2024-03-08T20:32:19.858 INFO:root:teuthology version: 0.0.1.dev262+ge691533 2024-03-08T20:32:19.861 DEBUG:teuthology.run:Teuthology command: teuthology --name rishabh-2024-03-08_19:54:47-fs-rishabh-2024mar8-testing-default-smithi --archive /home/teuthworker/archive/rishabh-2024-03-08_19:54:47-fs-rishabh-2024mar8-testing-default-smithi/7588131 --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/fuse objectstore-ec/bluestore-comp omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/random export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-workunit/direct_io}} --owner scheduled_rishabh@teuthology --verbose -- /home/teuthworker/archive/rishabh-2024-03-08_19:54:47-fs-rishabh-2024mar8-testing-default-smithi/7588131/orig.config.yaml 2024-03-08T20:32:19.916 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-08T20:32:20.004 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/rishabh-2024-03-08_19:54:47-fs-rishabh-2024mar8-testing-default-smithi/7588131 branch: rishabh-2024mar8 description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/fuse objectstore-ec/bluestore-comp omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/random export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-workunit/direct_io}} email: null first_in_suite: false job_id: '7588131' kernel: flavor: default kdb: true sha1: dd5c1cd171679b4e0ceeab3e8d62f25c2b4e6412 last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: rishabh-2024-03-08_19:54:47-fs-rishabh-2024mar8-testing-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: rishabh-2024mar8 ceph: cephfs: max_mds: 5 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_bal_replicate_threshold: 1 mds_export_ephemeral_random_max: 0.1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug cephsqlite: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - MON_DOWN sha1: 7d63482aa89628d733e298984a762389b625c761 subvols: create: 2 subvol_options: --size 25000000000 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - min: 3 name: mds_cache.dir_update - min: 3 name: mds_cache.dir_update_receipt - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount dry_run: true install: ceph: flavor: default sha1: 7d63482aa89628d733e298984a762389b625c761 kclient: client.0: mount_subvol_num: 1 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: rishabh-2024mar8 sha1: 7d63482aa89628d733e298984a762389b625c761 owner: scheduled_rishabh@teuthology priority: 70 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 1249 sha1: 7d63482aa89628d733e298984a762389b625c761 sleep_before_teardown: 0 subset: 481/512 suite: fs suite_branch: rishabh-2024mar8 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_7d63482aa89628d733e298984a762389b625c761/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 7d63482aa89628d733e298984a762389b625c761 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi107.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJHUqpW3DO5uh/UyCHrj4wHu9VgAgBFBM3KOq23ZnIIXBbGc51e1OwI4pL48TyShR7tAWT7OWIZaqaGRteH51no= smithi149.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGpna6/R7UJgNVAK3aRrFGF8fsUEJ0NGWiCWa9MYoBhKnZKZ1j7tDvHfMdVuXmpNKHo+IFqZuaP3Lrum5b/bDRA= smithi204.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGpHY52wqgFcelYlJ7D0yf12pxTEgCbwvmGmeK8S4HFDZ/4nEWVlPEtLpbNVhfgCTTiPaJMNAyvt+O/rU/tHA3Y= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - ceph-fuse: null - exec: mon.a: - ceph fs set cephfs balance_automate false - ceph fs subvolumegroup pin cephfs qa random 0.10 - check-counter: {} - exec: mon.a: - ceph mgr module enable snap_schedule - ceph config set mgr mgr/snap_schedule/allow_m_granularity true - ceph config set mgr mgr/snap_schedule/dump_on_update true - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - workunit: clients: all: - direct_io teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: e691533f9cbb33d85b2187bba20d7102f098636d timestamp: 2024-03-08_19:54:47 tube: smithi user: rishabh verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-03-08T20:32:20.004 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_7d63482aa89628d733e298984a762389b625c761/qa; will attempt to use it 2024-03-08T20:32:20.005 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_7d63482aa89628d733e298984a762389b625c761/qa/tasks 2024-03-08T20:32:20.005 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-03-08T20:32:20.007 INFO:teuthology.task.internal:Checking packages... 2024-03-08T20:32:20.030 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '7d63482aa89628d733e298984a762389b625c761' 2024-03-08T20:32:20.030 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-03-08T20:32:20.030 INFO:teuthology.packaging:ref: None 2024-03-08T20:32:20.030 INFO:teuthology.packaging:tag: None 2024-03-08T20:32:20.030 INFO:teuthology.packaging:branch: rishabh-2024mar8 2024-03-08T20:32:20.030 INFO:teuthology.packaging:sha1: 7d63482aa89628d733e298984a762389b625c761 2024-03-08T20:32:20.031 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=rishabh-2024mar8 2024-03-08T20:32:20.172 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-1965.g7d63482a 2024-03-08T20:32:20.173 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-03-08T20:32:20.180 INFO:teuthology.task.internal:no buildpackages task found 2024-03-08T20:32:20.180 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-03-08T20:32:20.225 INFO:teuthology.task.internal:Saving configuration 2024-03-08T20:32:20.245 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-03-08T20:32:20.265 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-08T20:32:20.291 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi107.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-03-08_19:54:47-fs-rishabh-2024mar8-testing-default-smithi/7588131', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-08 20:27:02.209988', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJHUqpW3DO5uh/UyCHrj4wHu9VgAgBFBM3KOq23ZnIIXBbGc51e1OwI4pL48TyShR7tAWT7OWIZaqaGRteH51no='} 2024-03-08T20:32:20.311 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi149.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-03-08_19:54:47-fs-rishabh-2024mar8-testing-default-smithi/7588131', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-08 20:27:02.208222', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGpna6/R7UJgNVAK3aRrFGF8fsUEJ0NGWiCWa9MYoBhKnZKZ1j7tDvHfMdVuXmpNKHo+IFqZuaP3Lrum5b/bDRA='} 2024-03-08T20:32:20.332 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi204.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-03-08_19:54:47-fs-rishabh-2024mar8-testing-default-smithi/7588131', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-08 20:27:02.206287', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGpHY52wqgFcelYlJ7D0yf12pxTEgCbwvmGmeK8S4HFDZ/4nEWVlPEtLpbNVhfgCTTiPaJMNAyvt+O/rU/tHA3Y='} 2024-03-08T20:32:20.333 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-03-08T20:32:20.339 INFO:teuthology.task.internal:roles: ubuntu@smithi107.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-03-08T20:32:20.339 INFO:teuthology.task.internal:roles: ubuntu@smithi149.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-03-08T20:32:20.339 INFO:teuthology.task.internal:roles: ubuntu@smithi204.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-03-08T20:32:20.339 INFO:teuthology.run_tasks:Running task console_log... 2024-03-08T20:32:20.461 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f35eeb280d0>, signals=[15]) 2024-03-08T20:32:20.461 INFO:teuthology.run_tasks:Running task internal.connect... 2024-03-08T20:32:20.467 INFO:teuthology.task.internal:Opening connections... 2024-03-08T20:32:20.467 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi107.front.sepia.ceph.com 2024-03-08T20:32:20.469 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi107.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-08T20:32:20.559 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi149.front.sepia.ceph.com 2024-03-08T20:32:20.560 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi149.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-08T20:32:20.650 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi204.front.sepia.ceph.com 2024-03-08T20:32:20.650 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi204.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-08T20:32:20.740 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-03-08T20:32:20.746 DEBUG:teuthology.orchestra.run.smithi107:> uname -m 2024-03-08T20:32:20.763 INFO:teuthology.orchestra.run.smithi107.stdout:x86_64 2024-03-08T20:32:20.764 DEBUG:teuthology.orchestra.run.smithi107:> cat /etc/os-release 2024-03-08T20:32:20.821 INFO:teuthology.orchestra.run.smithi107.stdout:NAME="CentOS Stream" 2024-03-08T20:32:20.821 INFO:teuthology.orchestra.run.smithi107.stdout:VERSION="9" 2024-03-08T20:32:20.821 INFO:teuthology.orchestra.run.smithi107.stdout:ID="centos" 2024-03-08T20:32:20.821 INFO:teuthology.orchestra.run.smithi107.stdout:ID_LIKE="rhel fedora" 2024-03-08T20:32:20.821 INFO:teuthology.orchestra.run.smithi107.stdout:VERSION_ID="9" 2024-03-08T20:32:20.821 INFO:teuthology.orchestra.run.smithi107.stdout:PLATFORM_ID="platform:el9" 2024-03-08T20:32:20.822 INFO:teuthology.orchestra.run.smithi107.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-08T20:32:20.822 INFO:teuthology.orchestra.run.smithi107.stdout:ANSI_COLOR="0;31" 2024-03-08T20:32:20.822 INFO:teuthology.orchestra.run.smithi107.stdout:LOGO="fedora-logo-icon" 2024-03-08T20:32:20.822 INFO:teuthology.orchestra.run.smithi107.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-08T20:32:20.822 INFO:teuthology.orchestra.run.smithi107.stdout:HOME_URL="https://centos.org/" 2024-03-08T20:32:20.822 INFO:teuthology.orchestra.run.smithi107.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-08T20:32:20.822 INFO:teuthology.orchestra.run.smithi107.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-08T20:32:20.822 INFO:teuthology.orchestra.run.smithi107.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-08T20:32:20.823 INFO:teuthology.lock.ops:Updating smithi107.front.sepia.ceph.com on lock server 2024-03-08T20:32:20.849 DEBUG:teuthology.orchestra.run.smithi149:> uname -m 2024-03-08T20:32:20.866 INFO:teuthology.orchestra.run.smithi149.stdout:x86_64 2024-03-08T20:32:20.867 DEBUG:teuthology.orchestra.run.smithi149:> cat /etc/os-release 2024-03-08T20:32:20.921 INFO:teuthology.orchestra.run.smithi149.stdout:NAME="CentOS Stream" 2024-03-08T20:32:20.921 INFO:teuthology.orchestra.run.smithi149.stdout:VERSION="9" 2024-03-08T20:32:20.921 INFO:teuthology.orchestra.run.smithi149.stdout:ID="centos" 2024-03-08T20:32:20.921 INFO:teuthology.orchestra.run.smithi149.stdout:ID_LIKE="rhel fedora" 2024-03-08T20:32:20.921 INFO:teuthology.orchestra.run.smithi149.stdout:VERSION_ID="9" 2024-03-08T20:32:20.921 INFO:teuthology.orchestra.run.smithi149.stdout:PLATFORM_ID="platform:el9" 2024-03-08T20:32:20.921 INFO:teuthology.orchestra.run.smithi149.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-08T20:32:20.921 INFO:teuthology.orchestra.run.smithi149.stdout:ANSI_COLOR="0;31" 2024-03-08T20:32:20.921 INFO:teuthology.orchestra.run.smithi149.stdout:LOGO="fedora-logo-icon" 2024-03-08T20:32:20.922 INFO:teuthology.orchestra.run.smithi149.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-08T20:32:20.922 INFO:teuthology.orchestra.run.smithi149.stdout:HOME_URL="https://centos.org/" 2024-03-08T20:32:20.922 INFO:teuthology.orchestra.run.smithi149.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-08T20:32:20.922 INFO:teuthology.orchestra.run.smithi149.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-08T20:32:20.922 INFO:teuthology.orchestra.run.smithi149.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-08T20:32:20.922 INFO:teuthology.lock.ops:Updating smithi149.front.sepia.ceph.com on lock server 2024-03-08T20:32:20.948 DEBUG:teuthology.orchestra.run.smithi204:> uname -m 2024-03-08T20:32:20.966 INFO:teuthology.orchestra.run.smithi204.stdout:x86_64 2024-03-08T20:32:20.967 DEBUG:teuthology.orchestra.run.smithi204:> cat /etc/os-release 2024-03-08T20:32:21.023 INFO:teuthology.orchestra.run.smithi204.stdout:NAME="CentOS Stream" 2024-03-08T20:32:21.023 INFO:teuthology.orchestra.run.smithi204.stdout:VERSION="9" 2024-03-08T20:32:21.024 INFO:teuthology.orchestra.run.smithi204.stdout:ID="centos" 2024-03-08T20:32:21.024 INFO:teuthology.orchestra.run.smithi204.stdout:ID_LIKE="rhel fedora" 2024-03-08T20:32:21.024 INFO:teuthology.orchestra.run.smithi204.stdout:VERSION_ID="9" 2024-03-08T20:32:21.024 INFO:teuthology.orchestra.run.smithi204.stdout:PLATFORM_ID="platform:el9" 2024-03-08T20:32:21.024 INFO:teuthology.orchestra.run.smithi204.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-08T20:32:21.024 INFO:teuthology.orchestra.run.smithi204.stdout:ANSI_COLOR="0;31" 2024-03-08T20:32:21.024 INFO:teuthology.orchestra.run.smithi204.stdout:LOGO="fedora-logo-icon" 2024-03-08T20:32:21.024 INFO:teuthology.orchestra.run.smithi204.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-08T20:32:21.024 INFO:teuthology.orchestra.run.smithi204.stdout:HOME_URL="https://centos.org/" 2024-03-08T20:32:21.024 INFO:teuthology.orchestra.run.smithi204.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-08T20:32:21.024 INFO:teuthology.orchestra.run.smithi204.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-08T20:32:21.024 INFO:teuthology.orchestra.run.smithi204.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-08T20:32:21.025 INFO:teuthology.lock.ops:Updating smithi204.front.sepia.ceph.com on lock server 2024-03-08T20:32:21.048 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-03-08T20:32:21.058 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-03-08T20:32:21.086 INFO:teuthology.task.internal:Checking for old test directory... 2024-03-08T20:32:21.087 DEBUG:teuthology.orchestra.run.smithi107:> test '!' -e /home/ubuntu/cephtest 2024-03-08T20:32:21.089 DEBUG:teuthology.orchestra.run.smithi149:> test '!' -e /home/ubuntu/cephtest 2024-03-08T20:32:21.094 DEBUG:teuthology.orchestra.run.smithi204:> test '!' -e /home/ubuntu/cephtest 2024-03-08T20:32:21.110 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-03-08T20:32:21.116 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-03-08T20:32:21.116 DEBUG:teuthology.orchestra.run.smithi107:> test -z $(ls -A /var/lib/ceph) 2024-03-08T20:32:21.147 DEBUG:teuthology.orchestra.run.smithi149:> test -z $(ls -A /var/lib/ceph) 2024-03-08T20:32:21.149 DEBUG:teuthology.orchestra.run.smithi204:> test -z $(ls -A /var/lib/ceph) 2024-03-08T20:32:21.258 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-03-08T20:32:21.330 INFO:teuthology.run_tasks:Running task kernel... 2024-03-08T20:32:21.345 INFO:teuthology.task.kernel:normalize config orig: {'flavor': 'default', 'kdb': True, 'sha1': 'dd5c1cd171679b4e0ceeab3e8d62f25c2b4e6412'} 2024-03-08T20:32:21.345 INFO:teuthology.task.kernel:config {'mon.a': {'flavor': 'default', 'kdb': True, 'sha1': 'dd5c1cd171679b4e0ceeab3e8d62f25c2b4e6412'}, 'mon.b': {'flavor': 'default', 'kdb': True, 'sha1': 'dd5c1cd171679b4e0ceeab3e8d62f25c2b4e6412'}, 'mon.c': {'flavor': 'default', 'kdb': True, 'sha1': 'dd5c1cd171679b4e0ceeab3e8d62f25c2b4e6412'}}, timeout 300 2024-03-08T20:32:21.346 DEBUG:teuthology.orchestra.run.smithi107:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-08T20:32:21.346 DEBUG:teuthology.orchestra.run.smithi149:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-08T20:32:21.347 DEBUG:teuthology.orchestra.run.smithi204:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-08T20:32:21.365 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-08T20:32:21.365 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-08T20:32:21.365 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-08T20:32:21.366 DEBUG:teuthology.task.kernel:sha1 for mon.c is dd5c1cd171679b4e0ceeab3e8d62f25c2b4e6412 2024-03-08T20:32:21.366 INFO:teuthology.task.kernel:Checking kernel version of mon.c, want "dd5c1cd171679b4e0ceeab3e8d62f25c2b4e6412"... 2024-03-08T20:32:21.367 DEBUG:teuthology.orchestra.run.smithi204:> uname -r 2024-03-08T20:32:21.367 DEBUG:teuthology.task.kernel:sha1 for mon.a is dd5c1cd171679b4e0ceeab3e8d62f25c2b4e6412 2024-03-08T20:32:21.367 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "dd5c1cd171679b4e0ceeab3e8d62f25c2b4e6412"... 2024-03-08T20:32:21.367 DEBUG:teuthology.orchestra.run.smithi107:> uname -r 2024-03-08T20:32:21.367 DEBUG:teuthology.task.kernel:sha1 for mon.b is dd5c1cd171679b4e0ceeab3e8d62f25c2b4e6412 2024-03-08T20:32:21.368 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "dd5c1cd171679b4e0ceeab3e8d62f25c2b4e6412"... 2024-03-08T20:32:21.368 DEBUG:teuthology.orchestra.run.smithi149:> uname -r 2024-03-08T20:32:21.422 INFO:teuthology.orchestra.run.smithi149.stdout:5.14.0-361.el9.x86_64 2024-03-08T20:32:21.423 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-361.el9.x86_64 vs dd5c1cd171679b4e0ceeab3e8d62f25c2b4e6412 2024-03-08T20:32:21.423 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-03-08T20:32:21.423 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=dd5c1cd171679b4e0ceeab3e8d62f25c2b4e6412 2024-03-08T20:32:21.428 INFO:teuthology.orchestra.run.smithi107.stdout:5.14.0-361.el9.x86_64 2024-03-08T20:32:21.428 INFO:teuthology.orchestra.run.smithi204.stdout:5.14.0-361.el9.x86_64 2024-03-08T20:32:21.428 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-361.el9.x86_64 vs dd5c1cd171679b4e0ceeab3e8d62f25c2b4e6412 2024-03-08T20:32:21.428 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-03-08T20:32:21.428 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=dd5c1cd171679b4e0ceeab3e8d62f25c2b4e6412 2024-03-08T20:32:21.429 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-361.el9.x86_64 vs dd5c1cd171679b4e0ceeab3e8d62f25c2b4e6412 2024-03-08T20:32:21.429 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-03-08T20:32:21.430 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=dd5c1cd171679b4e0ceeab3e8d62f25c2b4e6412 2024-03-08T20:32:21.654 DEBUG:teuthology.orchestra.run.smithi149:> sudo yum upgrade -y linux-firmware 2024-03-08T20:32:21.659 DEBUG:teuthology.orchestra.run.smithi204:> sudo yum upgrade -y linux-firmware 2024-03-08T20:32:21.660 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum upgrade -y linux-firmware 2024-03-08T20:32:23.746 INFO:teuthology.orchestra.run.smithi107.stdout:CentOS-9 - AppStream 16 MB/s | 6.7 MB 00:00 2024-03-08T20:32:23.848 INFO:teuthology.orchestra.run.smithi204.stdout:CentOS-9 - AppStream 13 MB/s | 6.7 MB 00:00 2024-03-08T20:32:23.891 INFO:teuthology.orchestra.run.smithi149.stdout:CentOS-9 - AppStream 14 MB/s | 6.7 MB 00:00 2024-03-08T20:32:25.363 INFO:teuthology.orchestra.run.smithi107.stdout:CentOS-9 - BaseOS 4.1 MB/s | 2.0 MB 00:00 2024-03-08T20:32:25.460 INFO:teuthology.orchestra.run.smithi204.stdout:CentOS-9 - BaseOS 3.9 MB/s | 2.0 MB 00:00 2024-03-08T20:32:25.483 INFO:teuthology.orchestra.run.smithi149.stdout:CentOS-9 - BaseOS 4.1 MB/s | 2.0 MB 00:00 2024-03-08T20:32:26.334 INFO:teuthology.orchestra.run.smithi204.stdout:CentOS Stream 9 - BaseOS 15 MB/s | 8.0 MB 00:00 2024-03-08T20:32:26.874 INFO:teuthology.orchestra.run.smithi149.stdout:CentOS Stream 9 - BaseOS 7.6 MB/s | 8.0 MB 00:01 2024-03-08T20:32:27.028 INFO:teuthology.orchestra.run.smithi107.stdout:CentOS Stream 9 - BaseOS 6.1 MB/s | 8.0 MB 00:01 2024-03-08T20:32:29.737 INFO:teuthology.orchestra.run.smithi107.stdout:CentOS Stream 9 - AppStream 12 MB/s | 19 MB 00:01 2024-03-08T20:32:31.418 INFO:teuthology.orchestra.run.smithi204.stdout:CentOS Stream 9 - AppStream 4.7 MB/s | 19 MB 00:04 2024-03-08T20:32:32.274 INFO:teuthology.orchestra.run.smithi149.stdout:CentOS Stream 9 - AppStream 4.4 MB/s | 19 MB 00:04 2024-03-08T20:32:33.746 INFO:teuthology.orchestra.run.smithi107.stdout:CentOS Stream 9 - CRB 11 MB/s | 6.1 MB 00:00 2024-03-08T20:32:35.213 INFO:teuthology.orchestra.run.smithi107.stdout:CentOS Stream 9 - Extras packages 44 kB/s | 15 kB 00:00 2024-03-08T20:32:35.444 INFO:teuthology.orchestra.run.smithi204.stdout:CentOS Stream 9 - CRB 11 MB/s | 6.1 MB 00:00 2024-03-08T20:32:36.223 INFO:teuthology.orchestra.run.smithi107.stdout:Extra Packages for Enterprise Linux 23 MB/s | 21 MB 00:00 2024-03-08T20:32:36.878 INFO:teuthology.orchestra.run.smithi204.stdout:CentOS Stream 9 - Extras packages 49 kB/s | 15 kB 00:00 2024-03-08T20:32:39.249 INFO:teuthology.orchestra.run.smithi149.stdout:CentOS Stream 9 - CRB 1.8 MB/s | 6.1 MB 00:03 2024-03-08T20:32:40.107 INFO:teuthology.orchestra.run.smithi204.stdout:Extra Packages for Enterprise Linux 6.6 MB/s | 21 MB 00:03 2024-03-08T20:32:40.742 INFO:teuthology.orchestra.run.smithi149.stdout:CentOS Stream 9 - Extras packages 42 kB/s | 15 kB 00:00 2024-03-08T20:32:41.347 INFO:teuthology.orchestra.run.smithi107.stdout:lab-extras 5.8 kB/s | 1.7 kB 00:00 2024-03-08T20:32:42.104 INFO:teuthology.orchestra.run.smithi149.stdout:Extra Packages for Enterprise Linux 17 MB/s | 21 MB 00:01 2024-03-08T20:32:43.618 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2024-03-08T20:32:43.630 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-03-08T20:32:43.630 INFO:teuthology.orchestra.run.smithi107.stdout: Package Arch Version Repository Size 2024-03-08T20:32:43.630 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-03-08T20:32:43.630 INFO:teuthology.orchestra.run.smithi107.stdout:Upgrading: 2024-03-08T20:32:43.630 INFO:teuthology.orchestra.run.smithi107.stdout: linux-firmware noarch 20240219-143.el9 CentOS-BaseOS 375 M 2024-03-08T20:32:43.630 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-08T20:32:43.630 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction Summary 2024-03-08T20:32:43.630 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-03-08T20:32:43.630 INFO:teuthology.orchestra.run.smithi107.stdout:Upgrade 1 Package 2024-03-08T20:32:43.630 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-08T20:32:43.630 INFO:teuthology.orchestra.run.smithi107.stdout:Total download size: 375 M 2024-03-08T20:32:43.630 INFO:teuthology.orchestra.run.smithi107.stdout:Downloading Packages: 2024-03-08T20:32:45.079 INFO:teuthology.orchestra.run.smithi204.stdout:lab-extras 12 kB/s | 1.7 kB 00:00 2024-03-08T20:32:47.026 INFO:teuthology.orchestra.run.smithi149.stdout:lab-extras 13 kB/s | 1.7 kB 00:00 2024-03-08T20:32:47.399 INFO:teuthology.orchestra.run.smithi204.stdout:Dependencies resolved. 2024-03-08T20:32:47.408 INFO:teuthology.orchestra.run.smithi204.stdout:================================================================================ 2024-03-08T20:32:47.408 INFO:teuthology.orchestra.run.smithi204.stdout: Package Arch Version Repository Size 2024-03-08T20:32:47.408 INFO:teuthology.orchestra.run.smithi204.stdout:================================================================================ 2024-03-08T20:32:47.408 INFO:teuthology.orchestra.run.smithi204.stdout:Upgrading: 2024-03-08T20:32:47.408 INFO:teuthology.orchestra.run.smithi204.stdout: linux-firmware noarch 20240219-143.el9 CentOS-BaseOS 375 M 2024-03-08T20:32:47.408 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-03-08T20:32:47.408 INFO:teuthology.orchestra.run.smithi204.stdout:Transaction Summary 2024-03-08T20:32:47.409 INFO:teuthology.orchestra.run.smithi204.stdout:================================================================================ 2024-03-08T20:32:47.409 INFO:teuthology.orchestra.run.smithi204.stdout:Upgrade 1 Package 2024-03-08T20:32:47.409 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-03-08T20:32:47.409 INFO:teuthology.orchestra.run.smithi204.stdout:Total download size: 375 M 2024-03-08T20:32:47.409 INFO:teuthology.orchestra.run.smithi204.stdout:Downloading Packages: 2024-03-08T20:32:49.270 INFO:teuthology.orchestra.run.smithi149.stdout:Dependencies resolved. 2024-03-08T20:32:49.286 INFO:teuthology.orchestra.run.smithi149.stdout:================================================================================ 2024-03-08T20:32:49.286 INFO:teuthology.orchestra.run.smithi149.stdout: Package Arch Version Repository Size 2024-03-08T20:32:49.286 INFO:teuthology.orchestra.run.smithi149.stdout:================================================================================ 2024-03-08T20:32:49.286 INFO:teuthology.orchestra.run.smithi149.stdout:Upgrading: 2024-03-08T20:32:49.287 INFO:teuthology.orchestra.run.smithi149.stdout: linux-firmware noarch 20240219-143.el9 CentOS-BaseOS 375 M 2024-03-08T20:32:49.287 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-03-08T20:32:49.287 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction Summary 2024-03-08T20:32:49.287 INFO:teuthology.orchestra.run.smithi149.stdout:================================================================================ 2024-03-08T20:32:49.287 INFO:teuthology.orchestra.run.smithi149.stdout:Upgrade 1 Package 2024-03-08T20:32:49.288 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-03-08T20:32:49.288 INFO:teuthology.orchestra.run.smithi149.stdout:Total download size: 375 M 2024-03-08T20:32:49.288 INFO:teuthology.orchestra.run.smithi149.stdout:Downloading Packages: 2024-03-08T20:32:58.338 INFO:teuthology.orchestra.run.smithi204.stdout:linux-firmware-20240219-143.el9.noarch.rpm 34 MB/s | 375 MB 00:10 2024-03-08T20:32:58.339 INFO:teuthology.orchestra.run.smithi204.stdout:-------------------------------------------------------------------------------- 2024-03-08T20:32:58.340 INFO:teuthology.orchestra.run.smithi204.stdout:Total 34 MB/s | 375 MB 00:10 2024-03-08T20:32:58.340 INFO:teuthology.orchestra.run.smithi204.stdout:Running transaction check 2024-03-08T20:32:58.430 INFO:teuthology.orchestra.run.smithi204.stdout:Transaction check succeeded. 2024-03-08T20:32:58.430 INFO:teuthology.orchestra.run.smithi204.stdout:Running transaction test 2024-03-08T20:32:59.288 INFO:teuthology.orchestra.run.smithi107.stdout:linux-firmware-20240219-143.el9.noarch.rpm 24 MB/s | 375 MB 00:15 2024-03-08T20:32:59.289 INFO:teuthology.orchestra.run.smithi107.stdout:-------------------------------------------------------------------------------- 2024-03-08T20:32:59.289 INFO:teuthology.orchestra.run.smithi107.stdout:Total 24 MB/s | 375 MB 00:15 2024-03-08T20:32:59.289 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction check 2024-03-08T20:32:59.388 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction check succeeded. 2024-03-08T20:32:59.388 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction test 2024-03-08T20:32:59.851 INFO:teuthology.orchestra.run.smithi149.stdout:linux-firmware-20240219-143.el9.noarch.rpm 35 MB/s | 375 MB 00:10 2024-03-08T20:32:59.852 INFO:teuthology.orchestra.run.smithi149.stdout:-------------------------------------------------------------------------------- 2024-03-08T20:32:59.853 INFO:teuthology.orchestra.run.smithi149.stdout:Total 35 MB/s | 375 MB 00:10 2024-03-08T20:32:59.853 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction check 2024-03-08T20:32:59.943 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction check succeeded. 2024-03-08T20:32:59.944 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction test 2024-03-08T20:33:00.864 INFO:teuthology.orchestra.run.smithi204.stdout:Transaction test succeeded. 2024-03-08T20:33:00.865 INFO:teuthology.orchestra.run.smithi204.stdout:Running transaction 2024-03-08T20:33:01.810 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction test succeeded. 2024-03-08T20:33:01.810 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction 2024-03-08T20:33:02.384 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction test succeeded. 2024-03-08T20:33:02.385 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction 2024-03-08T20:33:07.240 INFO:teuthology.orchestra.run.smithi204.stdout: Preparing : 1/1 2024-03-08T20:33:07.482 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : linux-firmware-20240219-143.el9.noarch 1/2 2024-03-08T20:33:07.565 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : linux-firmware-20230814-139.el9.noarch 2/2 2024-03-08T20:33:08.492 INFO:teuthology.orchestra.run.smithi107.stdout: Preparing : 1/1 2024-03-08T20:33:08.772 INFO:teuthology.orchestra.run.smithi107.stdout: Upgrading : linux-firmware-20240219-143.el9.noarch 1/2 2024-03-08T20:33:08.826 INFO:teuthology.orchestra.run.smithi149.stdout: Preparing : 1/1 2024-03-08T20:33:08.860 INFO:teuthology.orchestra.run.smithi107.stdout: Cleanup : linux-firmware-20230814-139.el9.noarch 2/2 2024-03-08T20:33:08.962 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: linux-firmware-20230814-139.el9.noarch 2/2 2024-03-08T20:33:08.963 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : linux-firmware-20240219-143.el9.noarch 1/2 2024-03-08T20:33:09.059 INFO:teuthology.orchestra.run.smithi149.stdout: Upgrading : linux-firmware-20240219-143.el9.noarch 1/2 2024-03-08T20:33:09.151 INFO:teuthology.orchestra.run.smithi149.stdout: Cleanup : linux-firmware-20230814-139.el9.noarch 2/2 2024-03-08T20:33:09.352 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : linux-firmware-20230814-139.el9.noarch 2/2 2024-03-08T20:33:09.352 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-03-08T20:33:09.353 INFO:teuthology.orchestra.run.smithi204.stdout:Upgraded: 2024-03-08T20:33:09.353 INFO:teuthology.orchestra.run.smithi204.stdout: linux-firmware-20240219-143.el9.noarch 2024-03-08T20:33:09.353 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-03-08T20:33:09.353 INFO:teuthology.orchestra.run.smithi204.stdout:Complete! 2024-03-08T20:33:09.624 INFO:teuthology.task.kernel:Downloading kernel dd5c1cd171679b4e0ceeab3e8d62f25c2b4e6412 on mon.c... 2024-03-08T20:33:09.625 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=dd5c1cd171679b4e0ceeab3e8d62f25c2b4e6412 2024-03-08T20:33:09.771 INFO:teuthology.task.kernel:fetching, builder baseurl is https://2.chacra.ceph.com/r/kernel/testing/dd5c1cd171679b4e0ceeab3e8d62f25c2b4e6412/centos/9/flavors/default/x86_64/ 2024-03-08T20:33:09.771 DEBUG:teuthology.orchestra.run.smithi204:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.8.0_rc7_gdd5c1cd17167-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://2.chacra.ceph.com/r/kernel/testing/dd5c1cd171679b4e0ceeab3e8d62f25c2b4e6412/centos/9/flavors/default/x86_64/ --input-file=- 2024-03-08T20:33:09.773 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi204.front.sepia.ceph.com to complete... 2024-03-08T20:33:10.016 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: linux-firmware-20230814-139.el9.noarch 2/2 2024-03-08T20:33:10.017 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : linux-firmware-20240219-143.el9.noarch 1/2 2024-03-08T20:33:10.389 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: linux-firmware-20230814-139.el9.noarch 2/2 2024-03-08T20:33:10.390 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : linux-firmware-20240219-143.el9.noarch 1/2 2024-03-08T20:33:10.436 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : linux-firmware-20230814-139.el9.noarch 2/2 2024-03-08T20:33:10.436 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-08T20:33:10.436 INFO:teuthology.orchestra.run.smithi107.stdout:Upgraded: 2024-03-08T20:33:10.436 INFO:teuthology.orchestra.run.smithi107.stdout: linux-firmware-20240219-143.el9.noarch 2024-03-08T20:33:10.436 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-08T20:33:10.437 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2024-03-08T20:33:10.766 INFO:teuthology.task.kernel:Downloading kernel dd5c1cd171679b4e0ceeab3e8d62f25c2b4e6412 on mon.a... 2024-03-08T20:33:10.766 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=dd5c1cd171679b4e0ceeab3e8d62f25c2b4e6412 2024-03-08T20:33:10.769 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : linux-firmware-20230814-139.el9.noarch 2/2 2024-03-08T20:33:10.769 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-03-08T20:33:10.770 INFO:teuthology.orchestra.run.smithi149.stdout:Upgraded: 2024-03-08T20:33:10.770 INFO:teuthology.orchestra.run.smithi149.stdout: linux-firmware-20240219-143.el9.noarch 2024-03-08T20:33:10.770 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-03-08T20:33:10.770 INFO:teuthology.orchestra.run.smithi149.stdout:Complete! 2024-03-08T20:33:10.998 INFO:teuthology.task.kernel:fetching, builder baseurl is https://2.chacra.ceph.com/r/kernel/testing/dd5c1cd171679b4e0ceeab3e8d62f25c2b4e6412/centos/9/flavors/default/x86_64/ 2024-03-08T20:33:10.998 DEBUG:teuthology.orchestra.run.smithi107:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.8.0_rc7_gdd5c1cd17167-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://2.chacra.ceph.com/r/kernel/testing/dd5c1cd171679b4e0ceeab3e8d62f25c2b4e6412/centos/9/flavors/default/x86_64/ --input-file=- 2024-03-08T20:33:11.000 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi107.front.sepia.ceph.com to complete... 2024-03-08T20:33:11.060 INFO:teuthology.task.kernel:Downloading kernel dd5c1cd171679b4e0ceeab3e8d62f25c2b4e6412 on mon.b... 2024-03-08T20:33:11.060 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=dd5c1cd171679b4e0ceeab3e8d62f25c2b4e6412 2024-03-08T20:33:11.293 INFO:teuthology.task.kernel:fetching, builder baseurl is https://2.chacra.ceph.com/r/kernel/testing/dd5c1cd171679b4e0ceeab3e8d62f25c2b4e6412/centos/9/flavors/default/x86_64/ 2024-03-08T20:33:11.293 DEBUG:teuthology.orchestra.run.smithi149:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.8.0_rc7_gdd5c1cd17167-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://2.chacra.ceph.com/r/kernel/testing/dd5c1cd171679b4e0ceeab3e8d62f25c2b4e6412/centos/9/flavors/default/x86_64/ --input-file=- 2024-03-08T20:33:11.296 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi149.front.sepia.ceph.com to complete... 2024-03-08T20:33:31.093 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08 20:33:31 URL:https://2.chacra.ceph.com/r/kernel/testing/dd5c1cd171679b4e0ceeab3e8d62f25c2b4e6412/centos/9/flavors/default/x86_64/kernel-6.8.0_rc7_gdd5c1cd17167-1.x86_64.rpm [336216379/336216379] -> "/tmp/kernel.x86_64.rpm" [1] 2024-03-08T20:33:31.093 INFO:teuthology.orchestra.run.smithi204.stderr:FINISHED --2024-03-08 20:33:31-- 2024-03-08T20:33:31.093 INFO:teuthology.orchestra.run.smithi204.stderr:Total wall clock time: 21s 2024-03-08T20:33:31.094 INFO:teuthology.orchestra.run.smithi204.stderr:Downloaded: 1 files, 321M in 21s (15.3 MB/s) 2024-03-08T20:33:31.096 INFO:teuthology.task.kernel:Installing kernel dd5c1cd171679b4e0ceeab3e8d62f25c2b4e6412 on mon.c... 2024-03-08T20:33:31.096 DEBUG:teuthology.orchestra.run.smithi204:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-03-08T20:33:33.028 INFO:teuthology.orchestra.run.smithi204.stdout:Verifying... ######################################## 2024-03-08T20:33:33.085 INFO:teuthology.orchestra.run.smithi204.stdout:Preparing... ######################################## 2024-03-08T20:33:33.164 INFO:teuthology.orchestra.run.smithi204.stdout:Updating / installing... 2024-03-08T20:33:39.812 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08 20:33:39 URL:https://2.chacra.ceph.com/r/kernel/testing/dd5c1cd171679b4e0ceeab3e8d62f25c2b4e6412/centos/9/flavors/default/x86_64/kernel-6.8.0_rc7_gdd5c1cd17167-1.x86_64.rpm [336216379/336216379] -> "/tmp/kernel.x86_64.rpm" [1] 2024-03-08T20:33:39.812 INFO:teuthology.orchestra.run.smithi107.stderr:FINISHED --2024-03-08 20:33:39-- 2024-03-08T20:33:39.812 INFO:teuthology.orchestra.run.smithi107.stderr:Total wall clock time: 29s 2024-03-08T20:33:39.812 INFO:teuthology.orchestra.run.smithi107.stderr:Downloaded: 1 files, 321M in 28s (11.3 MB/s) 2024-03-08T20:33:39.815 INFO:teuthology.task.kernel:Installing kernel dd5c1cd171679b4e0ceeab3e8d62f25c2b4e6412 on mon.a... 2024-03-08T20:33:39.815 DEBUG:teuthology.orchestra.run.smithi107:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-03-08T20:33:40.663 INFO:teuthology.orchestra.run.smithi204.stdout:kernel-6.8.0_rc7_gdd5c1cd17167-1 ######################################## 2024-03-08T20:33:41.743 INFO:teuthology.orchestra.run.smithi107.stdout:Verifying... ######################################## 2024-03-08T20:33:41.800 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing... ######################################## 2024-03-08T20:33:41.879 INFO:teuthology.orchestra.run.smithi107.stdout:Updating / installing... 2024-03-08T20:33:43.431 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08 20:33:43 URL:https://2.chacra.ceph.com/r/kernel/testing/dd5c1cd171679b4e0ceeab3e8d62f25c2b4e6412/centos/9/flavors/default/x86_64/kernel-6.8.0_rc7_gdd5c1cd17167-1.x86_64.rpm [336216379/336216379] -> "/tmp/kernel.x86_64.rpm" [1] 2024-03-08T20:33:43.431 INFO:teuthology.orchestra.run.smithi149.stderr:FINISHED --2024-03-08 20:33:43-- 2024-03-08T20:33:43.431 INFO:teuthology.orchestra.run.smithi149.stderr:Total wall clock time: 32s 2024-03-08T20:33:43.431 INFO:teuthology.orchestra.run.smithi149.stderr:Downloaded: 1 files, 321M in 32s (10.1 MB/s) 2024-03-08T20:33:43.434 INFO:teuthology.task.kernel:Installing kernel dd5c1cd171679b4e0ceeab3e8d62f25c2b4e6412 on mon.b... 2024-03-08T20:33:43.434 DEBUG:teuthology.orchestra.run.smithi149:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-03-08T20:33:45.366 INFO:teuthology.orchestra.run.smithi149.stdout:Verifying... ######################################## 2024-03-08T20:33:45.519 INFO:teuthology.orchestra.run.smithi149.stdout:Preparing... ######################################## 2024-03-08T20:33:45.598 INFO:teuthology.orchestra.run.smithi149.stdout:Updating / installing... 2024-03-08T20:33:49.435 INFO:teuthology.orchestra.run.smithi107.stdout:kernel-6.8.0_rc7_gdd5c1cd17167-1 ######################################## 2024-03-08T20:33:53.190 INFO:teuthology.orchestra.run.smithi149.stdout:kernel-6.8.0_rc7_gdd5c1cd17167-1 ######################################## 2024-03-08T20:33:56.837 INFO:teuthology.orchestra.run.smithi204.stderr:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-03-08T20:34:03.534 INFO:teuthology.orchestra.run.smithi107.stderr:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-03-08T20:34:06.288 INFO:teuthology.orchestra.run.smithi149.stderr:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-03-08T20:34:19.146 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi204.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-03-08T20:34:19.146 DEBUG:teuthology.orchestra.run.smithi204:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-03-08T20:34:19.218 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167 2024-03-08T20:34:19.219 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/System.map 2024-03-08T20:34:19.219 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/config 2024-03-08T20:34:19.219 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel 2024-03-08T20:34:19.219 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch 2024-03-08T20:34:19.219 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86 2024-03-08T20:34:19.219 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto 2024-03-08T20:34:19.219 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-03-08T20:34:19.219 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-03-08T20:34:19.219 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-03-08T20:34:19.219 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-03-08T20:34:19.219 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-03-08T20:34:19.220 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-03-08T20:34:19.220 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-03-08T20:34:19.220 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/crc32c-intel.ko 2024-03-08T20:34:19.220 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-03-08T20:34:19.220 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-03-08T20:34:19.220 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-03-08T20:34:19.220 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-03-08T20:34:19.220 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/serpent-avx2.ko 2024-03-08T20:34:19.220 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-03-08T20:34:19.220 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-03-08T20:34:19.220 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-03-08T20:34:19.220 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-03-08T20:34:19.220 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-03-08T20:34:19.221 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-03-08T20:34:19.221 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-03-08T20:34:19.221 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/kernel 2024-03-08T20:34:19.221 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/kernel/cpu 2024-03-08T20:34:19.221 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/kernel/cpu/mce 2024-03-08T20:34:19.221 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-03-08T20:34:19.221 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/kvm 2024-03-08T20:34:19.221 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/kvm/kvm-amd.ko 2024-03-08T20:34:19.221 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/kvm/kvm-intel.ko 2024-03-08T20:34:19.221 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/kvm/kvm.ko 2024-03-08T20:34:19.221 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto 2024-03-08T20:34:19.221 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/ansi_cprng.ko 2024-03-08T20:34:19.221 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/anubis.ko 2024-03-08T20:34:19.221 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/arc4.ko 2024-03-08T20:34:19.222 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/async_tx 2024-03-08T20:34:19.222 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/async_tx/async_memcpy.ko 2024-03-08T20:34:19.222 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/async_tx/async_pq.ko 2024-03-08T20:34:19.222 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/async_tx/async_raid6_recov.ko 2024-03-08T20:34:19.222 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/async_tx/async_tx.ko 2024-03-08T20:34:19.222 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/async_tx/async_xor.ko 2024-03-08T20:34:19.222 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/async_tx/raid6test.ko 2024-03-08T20:34:19.222 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/authenc.ko 2024-03-08T20:34:19.222 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/authencesn.ko 2024-03-08T20:34:19.222 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/blake2b_generic.ko 2024-03-08T20:34:19.222 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/blowfish_common.ko 2024-03-08T20:34:19.222 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/blowfish_generic.ko 2024-03-08T20:34:19.222 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/camellia_generic.ko 2024-03-08T20:34:19.223 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/cast5_generic.ko 2024-03-08T20:34:19.223 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/cast6_generic.ko 2024-03-08T20:34:19.223 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/cast_common.ko 2024-03-08T20:34:19.223 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/ccm.ko 2024-03-08T20:34:19.223 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/cmac.ko 2024-03-08T20:34:19.223 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/crc32_generic.ko 2024-03-08T20:34:19.223 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/crypto_engine.ko 2024-03-08T20:34:19.223 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/crypto_user.ko 2024-03-08T20:34:19.223 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/cts.ko 2024-03-08T20:34:19.223 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/des_generic.ko 2024-03-08T20:34:19.223 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/dh_generic.ko 2024-03-08T20:34:19.223 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/echainiv.ko 2024-03-08T20:34:19.223 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/essiv.ko 2024-03-08T20:34:19.224 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/fcrypt.ko 2024-03-08T20:34:19.224 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/khazad.ko 2024-03-08T20:34:19.224 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/lz4.ko 2024-03-08T20:34:19.224 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/lz4hc.ko 2024-03-08T20:34:19.224 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/md4.ko 2024-03-08T20:34:19.224 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/michael_mic.ko 2024-03-08T20:34:19.224 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/pcbc.ko 2024-03-08T20:34:19.224 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/pcrypt.ko 2024-03-08T20:34:19.224 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/rmd160.ko 2024-03-08T20:34:19.224 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/seed.ko 2024-03-08T20:34:19.224 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/serpent_generic.ko 2024-03-08T20:34:19.224 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/tcrypt.ko 2024-03-08T20:34:19.224 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/tea.ko 2024-03-08T20:34:19.224 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/twofish_common.ko 2024-03-08T20:34:19.225 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/twofish_generic.ko 2024-03-08T20:34:19.225 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/vmac.ko 2024-03-08T20:34:19.225 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/wp512.ko 2024-03-08T20:34:19.225 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/xcbc.ko 2024-03-08T20:34:19.225 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/xor.ko 2024-03-08T20:34:19.225 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/xxhash_generic.ko 2024-03-08T20:34:19.225 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers 2024-03-08T20:34:19.225 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/acpi 2024-03-08T20:34:19.225 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/acpi/acpi_ipmi.ko 2024-03-08T20:34:19.225 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/acpi/acpi_pad.ko 2024-03-08T20:34:19.225 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/acpi/custom_method.ko 2024-03-08T20:34:19.225 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/acpi/ec_sys.ko 2024-03-08T20:34:19.225 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/acpi/platform_profile.ko 2024-03-08T20:34:19.225 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/acpi/sbs.ko 2024-03-08T20:34:19.225 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/acpi/sbshc.ko 2024-03-08T20:34:19.226 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/acpi/video.ko 2024-03-08T20:34:19.226 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata 2024-03-08T20:34:19.226 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/acard-ahci.ko 2024-03-08T20:34:19.226 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/ahci_platform.ko 2024-03-08T20:34:19.226 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/ata_generic.ko 2024-03-08T20:34:19.226 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/libahci_platform.ko 2024-03-08T20:34:19.226 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_acpi.ko 2024-03-08T20:34:19.226 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_ali.ko 2024-03-08T20:34:19.226 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_amd.ko 2024-03-08T20:34:19.226 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_artop.ko 2024-03-08T20:34:19.226 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_atiixp.ko 2024-03-08T20:34:19.226 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_atp867x.ko 2024-03-08T20:34:19.226 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_cmd640.ko 2024-03-08T20:34:19.227 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_cmd64x.ko 2024-03-08T20:34:19.227 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_cypress.ko 2024-03-08T20:34:19.227 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_efar.ko 2024-03-08T20:34:19.227 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_hpt366.ko 2024-03-08T20:34:19.227 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_hpt37x.ko 2024-03-08T20:34:19.227 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_hpt3x2n.ko 2024-03-08T20:34:19.227 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_hpt3x3.ko 2024-03-08T20:34:19.227 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_it8213.ko 2024-03-08T20:34:19.227 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_it821x.ko 2024-03-08T20:34:19.227 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_jmicron.ko 2024-03-08T20:34:19.227 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_marvell.ko 2024-03-08T20:34:19.227 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_mpiix.ko 2024-03-08T20:34:19.227 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_netcell.ko 2024-03-08T20:34:19.227 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_ninja32.ko 2024-03-08T20:34:19.228 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_ns87410.ko 2024-03-08T20:34:19.228 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_ns87415.ko 2024-03-08T20:34:19.228 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_oldpiix.ko 2024-03-08T20:34:19.228 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_opti.ko 2024-03-08T20:34:19.228 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_optidma.ko 2024-03-08T20:34:19.228 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_pcmcia.ko 2024-03-08T20:34:19.228 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_pdc2027x.ko 2024-03-08T20:34:19.228 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-03-08T20:34:19.228 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_piccolo.ko 2024-03-08T20:34:19.228 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_rdc.ko 2024-03-08T20:34:19.228 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_sch.ko 2024-03-08T20:34:19.228 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_serverworks.ko 2024-03-08T20:34:19.228 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_sil680.ko 2024-03-08T20:34:19.229 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_sis.ko 2024-03-08T20:34:19.229 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_sl82c105.ko 2024-03-08T20:34:19.229 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_triflex.ko 2024-03-08T20:34:19.229 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_via.ko 2024-03-08T20:34:19.229 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pdc_adma.ko 2024-03-08T20:34:19.229 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/sata_inic162x.ko 2024-03-08T20:34:19.229 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/sata_mv.ko 2024-03-08T20:34:19.229 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/sata_nv.ko 2024-03-08T20:34:19.229 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/sata_promise.ko 2024-03-08T20:34:19.229 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/sata_qstor.ko 2024-03-08T20:34:19.229 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/sata_sil.ko 2024-03-08T20:34:19.229 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/sata_sil24.ko 2024-03-08T20:34:19.229 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/sata_sis.ko 2024-03-08T20:34:19.229 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/sata_svw.ko 2024-03-08T20:34:19.230 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/sata_sx4.ko 2024-03-08T20:34:19.230 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/sata_uli.ko 2024-03-08T20:34:19.230 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/sata_via.ko 2024-03-08T20:34:19.230 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/sata_vsc.ko 2024-03-08T20:34:19.230 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/auxdisplay 2024-03-08T20:34:19.230 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/auxdisplay/ks0108.ko 2024-03-08T20:34:19.230 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/base 2024-03-08T20:34:19.230 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/base/regmap 2024-03-08T20:34:19.230 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/base/regmap/regmap-i2c.ko 2024-03-08T20:34:19.230 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/base/regmap/regmap-mmio.ko 2024-03-08T20:34:19.230 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/bcma 2024-03-08T20:34:19.230 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/bcma/bcma.ko 2024-03-08T20:34:19.230 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block 2024-03-08T20:34:19.231 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/aoe 2024-03-08T20:34:19.231 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/aoe/aoe.ko 2024-03-08T20:34:19.231 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/brd.ko 2024-03-08T20:34:19.231 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/drbd 2024-03-08T20:34:19.231 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/drbd/drbd.ko 2024-03-08T20:34:19.231 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/floppy.ko 2024-03-08T20:34:19.231 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/loop.ko 2024-03-08T20:34:19.231 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/mtip32xx 2024-03-08T20:34:19.231 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-03-08T20:34:19.231 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/nbd.ko 2024-03-08T20:34:19.231 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/null_blk 2024-03-08T20:34:19.231 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/null_blk/null_blk.ko 2024-03-08T20:34:19.231 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/pktcdvd.ko 2024-03-08T20:34:19.231 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/rbd.ko 2024-03-08T20:34:19.232 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/virtio_blk.ko 2024-03-08T20:34:19.232 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/xen-blkback 2024-03-08T20:34:19.232 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-03-08T20:34:19.232 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/xen-blkfront.ko 2024-03-08T20:34:19.232 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/zram 2024-03-08T20:34:19.232 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/zram/zram.ko 2024-03-08T20:34:19.232 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char 2024-03-08T20:34:19.232 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/hangcheck-timer.ko 2024-03-08T20:34:19.232 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/hw_random 2024-03-08T20:34:19.232 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/hw_random/amd-rng.ko 2024-03-08T20:34:19.232 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/hw_random/intel-rng.ko 2024-03-08T20:34:19.232 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-03-08T20:34:19.232 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/hw_random/via-rng.ko 2024-03-08T20:34:19.233 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/hw_random/virtio-rng.ko 2024-03-08T20:34:19.233 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/ipmi 2024-03-08T20:34:19.233 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-03-08T20:34:19.233 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-03-08T20:34:19.233 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-03-08T20:34:19.233 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/ipmi/ipmi_si.ko 2024-03-08T20:34:19.233 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-03-08T20:34:19.233 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-03-08T20:34:19.233 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/lp.ko 2024-03-08T20:34:19.233 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/mwave 2024-03-08T20:34:19.233 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/mwave/mwave.ko 2024-03-08T20:34:19.233 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/ppdev.ko 2024-03-08T20:34:19.233 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/tlclk.ko 2024-03-08T20:34:19.233 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/tpm 2024-03-08T20:34:19.234 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/tpm/tpm.ko 2024-03-08T20:34:19.234 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/tpm/tpm_atmel.ko 2024-03-08T20:34:19.234 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/tpm/tpm_infineon.ko 2024-03-08T20:34:19.234 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/tpm/tpm_nsc.ko 2024-03-08T20:34:19.234 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/tpm/tpm_tis.ko 2024-03-08T20:34:19.234 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-03-08T20:34:19.234 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/uv_mmtimer.ko 2024-03-08T20:34:19.234 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/virtio_console.ko 2024-03-08T20:34:19.234 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/cpufreq 2024-03-08T20:34:19.234 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-03-08T20:34:19.234 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-03-08T20:34:19.234 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/cpufreq/p4-clockmod.ko 2024-03-08T20:34:19.234 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-03-08T20:34:19.235 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/cpufreq/powernow-k8.ko 2024-03-08T20:34:19.235 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/cpufreq/speedstep-lib.ko 2024-03-08T20:34:19.235 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/crypto 2024-03-08T20:34:19.235 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/crypto/ccp 2024-03-08T20:34:19.235 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-03-08T20:34:19.235 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/crypto/ccp/ccp.ko 2024-03-08T20:34:19.235 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/crypto/intel 2024-03-08T20:34:19.235 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/crypto/intel/qat 2024-03-08T20:34:19.235 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/crypto/intel/qat/qat_common 2024-03-08T20:34:19.235 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-03-08T20:34:19.235 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-03-08T20:34:19.235 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-03-08T20:34:19.235 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/crypto/padlock-aes.ko 2024-03-08T20:34:19.235 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/crypto/padlock-sha.ko 2024-03-08T20:34:19.236 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/crypto/virtio 2024-03-08T20:34:19.236 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-03-08T20:34:19.236 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/dca 2024-03-08T20:34:19.236 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/dca/dca.ko 2024-03-08T20:34:19.236 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/devfreq 2024-03-08T20:34:19.236 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-03-08T20:34:19.236 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/dma 2024-03-08T20:34:19.236 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/dma/dw 2024-03-08T20:34:19.236 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/dma/dw/dw_dmac.ko 2024-03-08T20:34:19.236 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/dma/ioat 2024-03-08T20:34:19.236 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/dma/ioat/ioatdma.ko 2024-03-08T20:34:19.236 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/edac 2024-03-08T20:34:19.236 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/edac/amd64_edac.ko 2024-03-08T20:34:19.237 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/edac/e752x_edac.ko 2024-03-08T20:34:19.237 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/edac/edac_mce_amd.ko 2024-03-08T20:34:19.237 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/edac/i3000_edac.ko 2024-03-08T20:34:19.237 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/edac/i3200_edac.ko 2024-03-08T20:34:19.237 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/edac/i5100_edac.ko 2024-03-08T20:34:19.237 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/edac/i5400_edac.ko 2024-03-08T20:34:19.237 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/edac/i7300_edac.ko 2024-03-08T20:34:19.237 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/edac/i7core_edac.ko 2024-03-08T20:34:19.237 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/edac/i82975x_edac.ko 2024-03-08T20:34:19.237 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/edac/ie31200_edac.ko 2024-03-08T20:34:19.237 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/edac/sb_edac.ko 2024-03-08T20:34:19.237 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/edac/x38_edac.ko 2024-03-08T20:34:19.237 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/firmware 2024-03-08T20:34:19.237 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/firmware/edd.ko 2024-03-08T20:34:19.237 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/firmware/iscsi_ibft.ko 2024-03-08T20:34:19.238 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/gpio 2024-03-08T20:34:19.238 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/gpio/gpio-ich.ko 2024-03-08T20:34:19.238 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/gpio/gpio-viperboard.ko 2024-03-08T20:34:19.238 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid 2024-03-08T20:34:19.238 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-appleir.ko 2024-03-08T20:34:19.238 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-aureal.ko 2024-03-08T20:34:19.238 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-axff.ko 2024-03-08T20:34:19.238 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-dr.ko 2024-03-08T20:34:19.238 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-elecom.ko 2024-03-08T20:34:19.238 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-elo.ko 2024-03-08T20:34:19.238 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-emsff.ko 2024-03-08T20:34:19.238 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-gaff.ko 2024-03-08T20:34:19.238 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-gt683r.ko 2024-03-08T20:34:19.239 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-gyration.ko 2024-03-08T20:34:19.239 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-holtek-kbd.ko 2024-03-08T20:34:19.239 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-holtek-mouse.ko 2024-03-08T20:34:19.239 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-holtekff.ko 2024-03-08T20:34:19.239 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-hyperv.ko 2024-03-08T20:34:19.239 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-icade.ko 2024-03-08T20:34:19.239 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-keytouch.ko 2024-03-08T20:34:19.239 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-kye.ko 2024-03-08T20:34:19.239 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-lcpower.ko 2024-03-08T20:34:19.239 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-led.ko 2024-03-08T20:34:19.239 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-lenovo.ko 2024-03-08T20:34:19.239 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-logitech-dj.ko 2024-03-08T20:34:19.239 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-03-08T20:34:19.240 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-multitouch.ko 2024-03-08T20:34:19.240 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-ortek.ko 2024-03-08T20:34:19.240 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-penmount.ko 2024-03-08T20:34:19.240 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-petalynx.ko 2024-03-08T20:34:19.240 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-picolcd.ko 2024-03-08T20:34:19.240 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-pl.ko 2024-03-08T20:34:19.240 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-plantronics.ko 2024-03-08T20:34:19.240 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-primax.ko 2024-03-08T20:34:19.240 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-rmi.ko 2024-03-08T20:34:19.240 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-roccat-arvo.ko 2024-03-08T20:34:19.240 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-roccat-common.ko 2024-03-08T20:34:19.240 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-roccat-isku.ko 2024-03-08T20:34:19.240 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-roccat-kone.ko 2024-03-08T20:34:19.240 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-03-08T20:34:19.241 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-roccat-konepure.ko 2024-03-08T20:34:19.241 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-03-08T20:34:19.241 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-roccat-lua.ko 2024-03-08T20:34:19.241 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-roccat-pyra.ko 2024-03-08T20:34:19.241 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-roccat-ryos.ko 2024-03-08T20:34:19.241 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-roccat-savu.ko 2024-03-08T20:34:19.241 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-roccat.ko 2024-03-08T20:34:19.241 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-saitek.ko 2024-03-08T20:34:19.241 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-samsung.ko 2024-03-08T20:34:19.241 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-sensor-hub.ko 2024-03-08T20:34:19.241 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-sjoy.ko 2024-03-08T20:34:19.241 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-sony.ko 2024-03-08T20:34:19.241 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-speedlink.ko 2024-03-08T20:34:19.241 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-steelseries.ko 2024-03-08T20:34:19.242 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-sunplus.ko 2024-03-08T20:34:19.242 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-thrustmaster.ko 2024-03-08T20:34:19.242 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-tivo.ko 2024-03-08T20:34:19.242 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-tmff.ko 2024-03-08T20:34:19.242 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-topseed.ko 2024-03-08T20:34:19.242 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-twinhan.ko 2024-03-08T20:34:19.242 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-uclogic.ko 2024-03-08T20:34:19.242 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-waltop.ko 2024-03-08T20:34:19.242 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-wiimote.ko 2024-03-08T20:34:19.242 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-xinmo.ko 2024-03-08T20:34:19.242 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-zpff.ko 2024-03-08T20:34:19.242 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-zydacron.ko 2024-03-08T20:34:19.242 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/uhid.ko 2024-03-08T20:34:19.242 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/wacom.ko 2024-03-08T20:34:19.243 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hv 2024-03-08T20:34:19.243 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hv/hv_balloon.ko 2024-03-08T20:34:19.243 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hv/hv_utils.ko 2024-03-08T20:34:19.243 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hv/hv_vmbus.ko 2024-03-08T20:34:19.243 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon 2024-03-08T20:34:19.243 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/abituguru.ko 2024-03-08T20:34:19.243 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/abituguru3.ko 2024-03-08T20:34:19.243 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/acpi_power_meter.ko 2024-03-08T20:34:19.243 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/ad7414.ko 2024-03-08T20:34:19.243 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/ad7418.ko 2024-03-08T20:34:19.243 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/adc128d818.ko 2024-03-08T20:34:19.243 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/adm1025.ko 2024-03-08T20:34:19.243 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/adm1026.ko 2024-03-08T20:34:19.244 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/adm1029.ko 2024-03-08T20:34:19.244 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/adm1031.ko 2024-03-08T20:34:19.244 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/adm9240.ko 2024-03-08T20:34:19.244 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/ads7828.ko 2024-03-08T20:34:19.244 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/adt7410.ko 2024-03-08T20:34:19.244 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/adt7411.ko 2024-03-08T20:34:19.244 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/adt7462.ko 2024-03-08T20:34:19.244 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/adt7470.ko 2024-03-08T20:34:19.244 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/adt7475.ko 2024-03-08T20:34:19.244 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/adt7x10.ko 2024-03-08T20:34:19.244 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/amc6821.ko 2024-03-08T20:34:19.244 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/applesmc.ko 2024-03-08T20:34:19.244 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/asb100.ko 2024-03-08T20:34:19.244 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/asc7621.ko 2024-03-08T20:34:19.244 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/asus_atk0110.ko 2024-03-08T20:34:19.245 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/atxp1.ko 2024-03-08T20:34:19.245 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/coretemp.ko 2024-03-08T20:34:19.245 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-03-08T20:34:19.245 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/dme1737.ko 2024-03-08T20:34:19.245 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/ds1621.ko 2024-03-08T20:34:19.245 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/ds620.ko 2024-03-08T20:34:19.245 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/emc1403.ko 2024-03-08T20:34:19.245 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/emc6w201.ko 2024-03-08T20:34:19.245 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/f71805f.ko 2024-03-08T20:34:19.245 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/f71882fg.ko 2024-03-08T20:34:19.245 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/f75375s.ko 2024-03-08T20:34:19.245 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/fam15h_power.ko 2024-03-08T20:34:19.246 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/fschmd.ko 2024-03-08T20:34:19.246 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/g760a.ko 2024-03-08T20:34:19.246 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/g762.ko 2024-03-08T20:34:19.246 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/gl518sm.ko 2024-03-08T20:34:19.246 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/gl520sm.ko 2024-03-08T20:34:19.246 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/hwmon-vid.ko 2024-03-08T20:34:19.246 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/i5500_temp.ko 2024-03-08T20:34:19.246 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/i5k_amb.ko 2024-03-08T20:34:19.246 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/ibmaem.ko 2024-03-08T20:34:19.246 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/ibmpex.ko 2024-03-08T20:34:19.246 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/ina209.ko 2024-03-08T20:34:19.246 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/ina2xx.ko 2024-03-08T20:34:19.246 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/it87.ko 2024-03-08T20:34:19.246 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/k10temp.ko 2024-03-08T20:34:19.247 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/k8temp.ko 2024-03-08T20:34:19.247 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/lineage-pem.ko 2024-03-08T20:34:19.247 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/lm63.ko 2024-03-08T20:34:19.247 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/lm73.ko 2024-03-08T20:34:19.247 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/lm75.ko 2024-03-08T20:34:19.247 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/lm77.ko 2024-03-08T20:34:19.247 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/lm78.ko 2024-03-08T20:34:19.247 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/lm80.ko 2024-03-08T20:34:19.247 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/lm83.ko 2024-03-08T20:34:19.247 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/lm85.ko 2024-03-08T20:34:19.247 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/lm87.ko 2024-03-08T20:34:19.247 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/lm90.ko 2024-03-08T20:34:19.247 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/lm92.ko 2024-03-08T20:34:19.247 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/lm93.ko 2024-03-08T20:34:19.248 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/lm95234.ko 2024-03-08T20:34:19.248 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/lm95241.ko 2024-03-08T20:34:19.248 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/lm95245.ko 2024-03-08T20:34:19.248 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/ltc2945.ko 2024-03-08T20:34:19.248 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/ltc4151.ko 2024-03-08T20:34:19.248 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/ltc4215.ko 2024-03-08T20:34:19.248 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/ltc4222.ko 2024-03-08T20:34:19.248 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/ltc4245.ko 2024-03-08T20:34:19.248 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/ltc4260.ko 2024-03-08T20:34:19.248 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/ltc4261.ko 2024-03-08T20:34:19.248 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/max16065.ko 2024-03-08T20:34:19.248 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/max1619.ko 2024-03-08T20:34:19.248 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/max1668.ko 2024-03-08T20:34:19.249 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/max197.ko 2024-03-08T20:34:19.249 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/max6639.ko 2024-03-08T20:34:19.249 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/max6650.ko 2024-03-08T20:34:19.249 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/max6697.ko 2024-03-08T20:34:19.249 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/mcp3021.ko 2024-03-08T20:34:19.249 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/nct6683.ko 2024-03-08T20:34:19.249 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/nct6775-core.ko 2024-03-08T20:34:19.249 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/nct6775.ko 2024-03-08T20:34:19.249 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/nct7802.ko 2024-03-08T20:34:19.249 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/pc87360.ko 2024-03-08T20:34:19.249 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/pc87427.ko 2024-03-08T20:34:19.249 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/pcf8591.ko 2024-03-08T20:34:19.249 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/pmbus 2024-03-08T20:34:19.249 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-03-08T20:34:19.250 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-03-08T20:34:19.250 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-03-08T20:34:19.250 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/pmbus/max16064.ko 2024-03-08T20:34:19.250 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/pmbus/max34440.ko 2024-03-08T20:34:19.250 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/pmbus/max8688.ko 2024-03-08T20:34:19.250 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-03-08T20:34:19.250 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-03-08T20:34:19.250 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-03-08T20:34:19.250 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-03-08T20:34:19.250 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-03-08T20:34:19.250 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-03-08T20:34:19.250 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/powr1220.ko 2024-03-08T20:34:19.250 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/sch5627.ko 2024-03-08T20:34:19.251 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/sch5636.ko 2024-03-08T20:34:19.251 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/sch56xx-common.ko 2024-03-08T20:34:19.251 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/sht15.ko 2024-03-08T20:34:19.251 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/sht21.ko 2024-03-08T20:34:19.251 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/shtc1.ko 2024-03-08T20:34:19.251 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/sis5595.ko 2024-03-08T20:34:19.251 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/smsc47b397.ko 2024-03-08T20:34:19.251 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/smsc47m1.ko 2024-03-08T20:34:19.251 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/smsc47m192.ko 2024-03-08T20:34:19.251 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/thmc50.ko 2024-03-08T20:34:19.251 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/tmp102.ko 2024-03-08T20:34:19.251 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/tmp103.ko 2024-03-08T20:34:19.251 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/tmp401.ko 2024-03-08T20:34:19.251 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/tmp421.ko 2024-03-08T20:34:19.252 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/via-cputemp.ko 2024-03-08T20:34:19.252 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/via686a.ko 2024-03-08T20:34:19.252 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/vt1211.ko 2024-03-08T20:34:19.252 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/vt8231.ko 2024-03-08T20:34:19.252 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/w83627ehf.ko 2024-03-08T20:34:19.252 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/w83627hf.ko 2024-03-08T20:34:19.252 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/w83781d.ko 2024-03-08T20:34:19.252 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/w83791d.ko 2024-03-08T20:34:19.252 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/w83792d.ko 2024-03-08T20:34:19.252 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/w83793.ko 2024-03-08T20:34:19.252 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/w83795.ko 2024-03-08T20:34:19.252 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/w83l785ts.ko 2024-03-08T20:34:19.252 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/w83l786ng.ko 2024-03-08T20:34:19.252 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c 2024-03-08T20:34:19.253 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/algos 2024-03-08T20:34:19.253 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-03-08T20:34:19.253 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-03-08T20:34:19.253 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses 2024-03-08T20:34:19.253 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-03-08T20:34:19.253 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-03-08T20:34:19.253 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-03-08T20:34:19.253 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-03-08T20:34:19.253 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-03-08T20:34:19.253 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-03-08T20:34:19.253 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-03-08T20:34:19.253 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-03-08T20:34:19.253 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-i801.ko 2024-03-08T20:34:19.254 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-isch.ko 2024-03-08T20:34:19.254 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-03-08T20:34:19.254 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-03-08T20:34:19.254 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-03-08T20:34:19.254 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-parport.ko 2024-03-08T20:34:19.254 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-03-08T20:34:19.254 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-03-08T20:34:19.254 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-03-08T20:34:19.254 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-03-08T20:34:19.254 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-03-08T20:34:19.254 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-03-08T20:34:19.254 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-via.ko 2024-03-08T20:34:19.254 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-03-08T20:34:19.254 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-03-08T20:34:19.255 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/i2c-dev.ko 2024-03-08T20:34:19.255 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/i2c-mux.ko 2024-03-08T20:34:19.255 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-03-08T20:34:19.255 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/i2c-smbus.ko 2024-03-08T20:34:19.255 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/i2c-stub.ko 2024-03-08T20:34:19.255 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband 2024-03-08T20:34:19.255 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/core 2024-03-08T20:34:19.255 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/core/ib_cm.ko 2024-03-08T20:34:19.255 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/core/ib_core.ko 2024-03-08T20:34:19.255 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/core/ib_umad.ko 2024-03-08T20:34:19.257 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-03-08T20:34:19.257 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/core/iw_cm.ko 2024-03-08T20:34:19.257 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/core/rdma_cm.ko 2024-03-08T20:34:19.257 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-03-08T20:34:19.257 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/hw 2024-03-08T20:34:19.257 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/hw/cxgb4 2024-03-08T20:34:19.257 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-03-08T20:34:19.257 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/hw/mlx4 2024-03-08T20:34:19.257 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-03-08T20:34:19.257 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/hw/mlx5 2024-03-08T20:34:19.258 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-03-08T20:34:19.258 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/hw/mthca 2024-03-08T20:34:19.258 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-03-08T20:34:19.258 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/ulp 2024-03-08T20:34:19.258 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/ulp/ipoib 2024-03-08T20:34:19.258 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-03-08T20:34:19.258 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/ulp/iser 2024-03-08T20:34:19.258 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-03-08T20:34:19.258 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/ulp/isert 2024-03-08T20:34:19.258 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-03-08T20:34:19.258 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/ulp/srp 2024-03-08T20:34:19.258 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-03-08T20:34:19.258 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/ulp/srpt 2024-03-08T20:34:19.259 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-03-08T20:34:19.259 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input 2024-03-08T20:34:19.259 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/gameport 2024-03-08T20:34:19.259 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-03-08T20:34:19.259 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/gameport/fm801-gp.ko 2024-03-08T20:34:19.259 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/gameport/gameport.ko 2024-03-08T20:34:19.259 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/joydev.ko 2024-03-08T20:34:19.259 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/keyboard 2024-03-08T20:34:19.259 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/keyboard/gpio_keys.ko 2024-03-08T20:34:19.259 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/mouse 2024-03-08T20:34:19.259 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/mouse/appletouch.ko 2024-03-08T20:34:19.259 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/mouse/bcm5974.ko 2024-03-08T20:34:19.259 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/mouse/cyapatp.ko 2024-03-08T20:34:19.259 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/mouse/elan_i2c.ko 2024-03-08T20:34:19.260 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/mouse/sermouse.ko 2024-03-08T20:34:19.260 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-03-08T20:34:19.260 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/mouse/synaptics_usb.ko 2024-03-08T20:34:19.260 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/mouse/vsxxxaa.ko 2024-03-08T20:34:19.260 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/rmi4 2024-03-08T20:34:19.260 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/rmi4/rmi_core.ko 2024-03-08T20:34:19.260 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/serio 2024-03-08T20:34:19.260 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/serio/altera_ps2.ko 2024-03-08T20:34:19.260 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/serio/arc_ps2.ko 2024-03-08T20:34:19.260 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-03-08T20:34:19.260 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/serio/serio_raw.ko 2024-03-08T20:34:19.260 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/sparse-keymap.ko 2024-03-08T20:34:19.260 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/leds 2024-03-08T20:34:19.260 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/leds/leds-blinkm.ko 2024-03-08T20:34:19.261 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/leds/leds-lm3530.ko 2024-03-08T20:34:19.261 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/leds/leds-lp3944.ko 2024-03-08T20:34:19.261 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/leds/leds-ss4200.ko 2024-03-08T20:34:19.261 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/leds/trigger 2024-03-08T20:34:19.261 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-03-08T20:34:19.261 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-03-08T20:34:19.261 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-03-08T20:34:19.261 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-03-08T20:34:19.261 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-03-08T20:34:19.261 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-03-08T20:34:19.261 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-03-08T20:34:19.261 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-03-08T20:34:19.261 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-03-08T20:34:19.261 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md 2024-03-08T20:34:19.262 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/bcache 2024-03-08T20:34:19.262 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/bcache/bcache.ko 2024-03-08T20:34:19.262 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/dm-bio-prison.ko 2024-03-08T20:34:19.262 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/dm-cache-smq.ko 2024-03-08T20:34:19.262 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/dm-cache.ko 2024-03-08T20:34:19.262 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/dm-crypt.ko 2024-03-08T20:34:19.262 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/dm-delay.ko 2024-03-08T20:34:19.262 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/dm-flakey.ko 2024-03-08T20:34:19.262 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/dm-log-userspace.ko 2024-03-08T20:34:19.262 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/dm-multipath.ko 2024-03-08T20:34:19.262 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/dm-queue-length.ko 2024-03-08T20:34:19.262 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/dm-raid.ko 2024-03-08T20:34:19.262 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/dm-round-robin.ko 2024-03-08T20:34:19.263 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/dm-service-time.ko 2024-03-08T20:34:19.263 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/dm-switch.ko 2024-03-08T20:34:19.263 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/dm-thin-pool.ko 2024-03-08T20:34:19.263 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/dm-verity.ko 2024-03-08T20:34:19.263 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/persistent-data 2024-03-08T20:34:19.263 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-03-08T20:34:19.263 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/raid0.ko 2024-03-08T20:34:19.263 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/raid1.ko 2024-03-08T20:34:19.263 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/raid10.ko 2024-03-08T20:34:19.263 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/raid456.ko 2024-03-08T20:34:19.263 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/message 2024-03-08T20:34:19.263 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/message/fusion 2024-03-08T20:34:19.263 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/message/fusion/mptbase.ko 2024-03-08T20:34:19.263 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/message/fusion/mptctl.ko 2024-03-08T20:34:19.264 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/message/fusion/mptfc.ko 2024-03-08T20:34:19.264 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/message/fusion/mptlan.ko 2024-03-08T20:34:19.264 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/message/fusion/mptsas.ko 2024-03-08T20:34:19.264 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/message/fusion/mptscsih.ko 2024-03-08T20:34:19.264 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/message/fusion/mptspi.ko 2024-03-08T20:34:19.264 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/mfd 2024-03-08T20:34:19.264 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/mfd/lpc_ich.ko 2024-03-08T20:34:19.264 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/mfd/lpc_sch.ko 2024-03-08T20:34:19.264 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/mfd/mfd-core.ko 2024-03-08T20:34:19.264 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/mfd/sm501.ko 2024-03-08T20:34:19.264 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/mfd/viperboard.ko 2024-03-08T20:34:19.264 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/mfd/vx855.ko 2024-03-08T20:34:19.264 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/mfd/wl1273-core.ko 2024-03-08T20:34:19.265 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc 2024-03-08T20:34:19.265 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/altera-stapl 2024-03-08T20:34:19.265 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-03-08T20:34:19.265 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/apds9802als.ko 2024-03-08T20:34:19.265 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/apds990x.ko 2024-03-08T20:34:19.265 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/bh1770glc.ko 2024-03-08T20:34:19.265 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/cb710 2024-03-08T20:34:19.265 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/cb710/cb710.ko 2024-03-08T20:34:19.265 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/eeprom 2024-03-08T20:34:19.265 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/eeprom/at24.ko 2024-03-08T20:34:19.265 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-03-08T20:34:19.265 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/eeprom/max6875.ko 2024-03-08T20:34:19.265 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/enclosure.ko 2024-03-08T20:34:19.265 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/hpilo.ko 2024-03-08T20:34:19.266 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/isl29003.ko 2024-03-08T20:34:19.266 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/isl29020.ko 2024-03-08T20:34:19.266 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/lis3lv02d 2024-03-08T20:34:19.266 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-03-08T20:34:19.266 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-03-08T20:34:19.266 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/mei 2024-03-08T20:34:19.266 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/mei/mei-me.ko 2024-03-08T20:34:19.266 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/mei/mei-txe.ko 2024-03-08T20:34:19.266 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/mei/mei.ko 2024-03-08T20:34:19.266 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/sgi-gru 2024-03-08T20:34:19.266 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/sgi-gru/gru.ko 2024-03-08T20:34:19.266 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/sgi-xp 2024-03-08T20:34:19.266 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/sgi-xp/xp.ko 2024-03-08T20:34:19.267 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/sgi-xp/xpc.ko 2024-03-08T20:34:19.267 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-03-08T20:34:19.267 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/tifm_7xx1.ko 2024-03-08T20:34:19.267 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/tifm_core.ko 2024-03-08T20:34:19.267 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/tsl2550.ko 2024-03-08T20:34:19.267 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/vmw_balloon.ko 2024-03-08T20:34:19.267 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/vmw_vmci 2024-03-08T20:34:19.267 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-03-08T20:34:19.267 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net 2024-03-08T20:34:19.267 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/bonding 2024-03-08T20:34:19.267 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/bonding/bonding.ko 2024-03-08T20:34:19.267 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/dsa 2024-03-08T20:34:19.267 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/dsa/b53 2024-03-08T20:34:19.267 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/dsa/b53/b53_common.ko 2024-03-08T20:34:19.268 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/dsa/bcm-sf2.ko 2024-03-08T20:34:19.268 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/dsa/mv88e6060.ko 2024-03-08T20:34:19.268 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/dsa/mv88e6xxx 2024-03-08T20:34:19.268 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-03-08T20:34:19.268 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/dummy.ko 2024-03-08T20:34:19.268 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/eql.ko 2024-03-08T20:34:19.268 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet 2024-03-08T20:34:19.268 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/3com 2024-03-08T20:34:19.268 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-03-08T20:34:19.268 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-03-08T20:34:19.268 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-03-08T20:34:19.268 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-03-08T20:34:19.268 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/8390 2024-03-08T20:34:19.268 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/8390/8390.ko 2024-03-08T20:34:19.269 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-03-08T20:34:19.269 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-03-08T20:34:19.269 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-03-08T20:34:19.269 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/adaptec 2024-03-08T20:34:19.269 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-03-08T20:34:19.269 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/agere 2024-03-08T20:34:19.269 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/agere/et131x.ko 2024-03-08T20:34:19.269 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/alteon 2024-03-08T20:34:19.269 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-03-08T20:34:19.269 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/altera 2024-03-08T20:34:19.269 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-03-08T20:34:19.269 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/amd 2024-03-08T20:34:19.269 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-03-08T20:34:19.269 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-03-08T20:34:19.270 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-03-08T20:34:19.270 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/atheros 2024-03-08T20:34:19.270 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/atheros/alx 2024-03-08T20:34:19.270 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-03-08T20:34:19.270 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/atheros/atl1c 2024-03-08T20:34:19.270 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-03-08T20:34:19.270 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/atheros/atl1e 2024-03-08T20:34:19.270 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-03-08T20:34:19.270 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/atheros/atlx 2024-03-08T20:34:19.270 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-03-08T20:34:19.270 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-03-08T20:34:19.270 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/broadcom 2024-03-08T20:34:19.270 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-03-08T20:34:19.271 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-03-08T20:34:19.271 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-03-08T20:34:19.271 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-03-08T20:34:19.271 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-03-08T20:34:19.271 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/broadcom/genet 2024-03-08T20:34:19.271 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-03-08T20:34:19.271 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-03-08T20:34:19.271 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/brocade 2024-03-08T20:34:19.271 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/brocade/bna 2024-03-08T20:34:19.271 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-03-08T20:34:19.271 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/chelsio 2024-03-08T20:34:19.271 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/chelsio/cxgb 2024-03-08T20:34:19.271 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-03-08T20:34:19.271 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-03-08T20:34:19.272 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-03-08T20:34:19.272 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-03-08T20:34:19.272 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-03-08T20:34:19.272 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-03-08T20:34:19.272 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-03-08T20:34:19.272 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-03-08T20:34:19.272 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-03-08T20:34:19.272 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/cisco 2024-03-08T20:34:19.272 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/cisco/enic 2024-03-08T20:34:19.272 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-03-08T20:34:19.272 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/dec 2024-03-08T20:34:19.272 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/dec/tulip 2024-03-08T20:34:19.272 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-03-08T20:34:19.272 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-03-08T20:34:19.273 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-03-08T20:34:19.273 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-03-08T20:34:19.273 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-03-08T20:34:19.273 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-03-08T20:34:19.273 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/dlink 2024-03-08T20:34:19.273 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-03-08T20:34:19.273 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-03-08T20:34:19.273 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/dnet.ko 2024-03-08T20:34:19.273 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/emulex 2024-03-08T20:34:19.273 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/emulex/benet 2024-03-08T20:34:19.273 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-03-08T20:34:19.273 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/ethoc.ko 2024-03-08T20:34:19.273 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/fealnx.ko 2024-03-08T20:34:19.273 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel 2024-03-08T20:34:19.274 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel/e100.ko 2024-03-08T20:34:19.274 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel/e1000 2024-03-08T20:34:19.274 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-03-08T20:34:19.274 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel/e1000e 2024-03-08T20:34:19.274 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-03-08T20:34:19.274 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel/fm10k 2024-03-08T20:34:19.274 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-03-08T20:34:19.274 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel/i40e 2024-03-08T20:34:19.274 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-03-08T20:34:19.274 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel/iavf 2024-03-08T20:34:19.274 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-03-08T20:34:19.274 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel/igb 2024-03-08T20:34:19.274 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-03-08T20:34:19.275 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel/igbvf 2024-03-08T20:34:19.275 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-03-08T20:34:19.275 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel/ixgbe 2024-03-08T20:34:19.275 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-03-08T20:34:19.275 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel/ixgbevf 2024-03-08T20:34:19.275 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-03-08T20:34:19.275 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/jme.ko 2024-03-08T20:34:19.275 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/marvell 2024-03-08T20:34:19.275 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-03-08T20:34:19.275 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/marvell/skge.ko 2024-03-08T20:34:19.275 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-03-08T20:34:19.275 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/mellanox 2024-03-08T20:34:19.275 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/mellanox/mlx4 2024-03-08T20:34:19.275 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-03-08T20:34:19.276 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-03-08T20:34:19.276 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/mellanox/mlx5 2024-03-08T20:34:19.276 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-03-08T20:34:19.276 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-03-08T20:34:19.276 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/micrel 2024-03-08T20:34:19.276 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-03-08T20:34:19.276 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/myricom 2024-03-08T20:34:19.276 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/myricom/myri10ge 2024-03-08T20:34:19.276 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-03-08T20:34:19.276 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/natsemi 2024-03-08T20:34:19.276 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-03-08T20:34:19.276 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-03-08T20:34:19.276 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/neterion 2024-03-08T20:34:19.276 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-03-08T20:34:19.276 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/nvidia 2024-03-08T20:34:19.277 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-03-08T20:34:19.277 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/packetengines 2024-03-08T20:34:19.277 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-03-08T20:34:19.277 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-03-08T20:34:19.277 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/qlogic 2024-03-08T20:34:19.277 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/qlogic/netxen 2024-03-08T20:34:19.277 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-03-08T20:34:19.277 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-03-08T20:34:19.277 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-03-08T20:34:19.277 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-03-08T20:34:19.277 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/rdc 2024-03-08T20:34:19.277 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-03-08T20:34:19.277 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/realtek 2024-03-08T20:34:19.278 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-03-08T20:34:19.278 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-03-08T20:34:19.278 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/realtek/atp.ko 2024-03-08T20:34:19.278 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-03-08T20:34:19.278 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/rocker 2024-03-08T20:34:19.278 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-03-08T20:34:19.278 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/sfc 2024-03-08T20:34:19.278 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-03-08T20:34:19.278 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/silan 2024-03-08T20:34:19.278 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-03-08T20:34:19.278 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/sis 2024-03-08T20:34:19.278 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/sis/sis190.ko 2024-03-08T20:34:19.278 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/sis/sis900.ko 2024-03-08T20:34:19.278 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/smsc 2024-03-08T20:34:19.279 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-03-08T20:34:19.279 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-03-08T20:34:19.279 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-03-08T20:34:19.279 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-03-08T20:34:19.279 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/stmicro 2024-03-08T20:34:19.279 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/stmicro/stmmac 2024-03-08T20:34:19.279 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-03-08T20:34:19.279 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-03-08T20:34:19.279 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/sun 2024-03-08T20:34:19.279 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/sun/cassini.ko 2024-03-08T20:34:19.279 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/sun/niu.ko 2024-03-08T20:34:19.279 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/sun/sungem.ko 2024-03-08T20:34:19.279 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-03-08T20:34:19.279 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/tehuti 2024-03-08T20:34:19.280 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-03-08T20:34:19.280 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/ti 2024-03-08T20:34:19.280 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/ti/tlan.ko 2024-03-08T20:34:19.280 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/via 2024-03-08T20:34:19.280 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-03-08T20:34:19.280 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-03-08T20:34:19.280 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/wiznet 2024-03-08T20:34:19.280 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-03-08T20:34:19.280 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-03-08T20:34:19.280 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/xircom 2024-03-08T20:34:19.280 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-03-08T20:34:19.280 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/geneve.ko 2024-03-08T20:34:19.280 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/hyperv 2024-03-08T20:34:19.280 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-03-08T20:34:19.281 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ieee802154 2024-03-08T20:34:19.281 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ieee802154/fakelb.ko 2024-03-08T20:34:19.281 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ifb.ko 2024-03-08T20:34:19.281 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ipvlan 2024-03-08T20:34:19.281 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ipvlan/ipvlan.ko 2024-03-08T20:34:19.281 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/macvlan.ko 2024-03-08T20:34:19.281 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/macvtap.ko 2024-03-08T20:34:19.281 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/mdio 2024-03-08T20:34:19.281 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/mdio.ko 2024-03-08T20:34:19.281 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-03-08T20:34:19.281 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-03-08T20:34:19.281 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/mdio/mdio-regmap.ko 2024-03-08T20:34:19.281 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/mii.ko 2024-03-08T20:34:19.281 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/net_failover.ko 2024-03-08T20:34:19.282 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/netconsole.ko 2024-03-08T20:34:19.282 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/nlmon.ko 2024-03-08T20:34:19.282 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/pcs 2024-03-08T20:34:19.282 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/pcs/pcs-lynx.ko 2024-03-08T20:34:19.282 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-03-08T20:34:19.282 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy 2024-03-08T20:34:19.282 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/amd.ko 2024-03-08T20:34:19.282 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-03-08T20:34:19.282 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-03-08T20:34:19.282 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/bcm7xxx.ko 2024-03-08T20:34:19.282 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/bcm87xx.ko 2024-03-08T20:34:19.282 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/broadcom.ko 2024-03-08T20:34:19.282 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/cicada.ko 2024-03-08T20:34:19.283 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/davicom.ko 2024-03-08T20:34:19.283 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/dp83640.ko 2024-03-08T20:34:19.283 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/et1011c.ko 2024-03-08T20:34:19.283 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/icplus.ko 2024-03-08T20:34:19.283 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/lxt.ko 2024-03-08T20:34:19.283 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/marvell.ko 2024-03-08T20:34:19.283 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/micrel.ko 2024-03-08T20:34:19.283 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/national.ko 2024-03-08T20:34:19.283 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/phylink.ko 2024-03-08T20:34:19.283 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/qsemi.ko 2024-03-08T20:34:19.283 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/realtek.ko 2024-03-08T20:34:19.283 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/smsc.ko 2024-03-08T20:34:19.283 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/ste10Xp.ko 2024-03-08T20:34:19.283 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/vitesse.ko 2024-03-08T20:34:19.284 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ppp 2024-03-08T20:34:19.284 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ppp/bsd_comp.ko 2024-03-08T20:34:19.284 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ppp/ppp_async.ko 2024-03-08T20:34:19.284 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ppp/ppp_deflate.ko 2024-03-08T20:34:19.284 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ppp/ppp_generic.ko 2024-03-08T20:34:19.284 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ppp/ppp_mppe.ko 2024-03-08T20:34:19.284 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ppp/ppp_synctty.ko 2024-03-08T20:34:19.284 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ppp/pppoe.ko 2024-03-08T20:34:19.284 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ppp/pppox.ko 2024-03-08T20:34:19.284 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ppp/pptp.ko 2024-03-08T20:34:19.284 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/slip 2024-03-08T20:34:19.284 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/slip/slhc.ko 2024-03-08T20:34:19.284 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/slip/slip.ko 2024-03-08T20:34:19.284 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/sungem_phy.ko 2024-03-08T20:34:19.284 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/tap.ko 2024-03-08T20:34:19.285 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/team 2024-03-08T20:34:19.285 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/team/team.ko 2024-03-08T20:34:19.285 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/team/team_mode_activebackup.ko 2024-03-08T20:34:19.285 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/team/team_mode_broadcast.ko 2024-03-08T20:34:19.285 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-03-08T20:34:19.285 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/team/team_mode_random.ko 2024-03-08T20:34:19.285 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-03-08T20:34:19.285 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/tun.ko 2024-03-08T20:34:19.285 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/veth.ko 2024-03-08T20:34:19.285 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/virtio_net.ko 2024-03-08T20:34:19.285 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/vmxnet3 2024-03-08T20:34:19.285 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-03-08T20:34:19.285 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/vxlan 2024-03-08T20:34:19.285 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/vxlan/vxlan.ko 2024-03-08T20:34:19.286 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/xen-netback 2024-03-08T20:34:19.286 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/xen-netback/xen-netback.ko 2024-03-08T20:34:19.286 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/xen-netfront.ko 2024-03-08T20:34:19.286 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ntb 2024-03-08T20:34:19.286 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ntb/ntb.ko 2024-03-08T20:34:19.286 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/nvme 2024-03-08T20:34:19.286 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/nvme/host 2024-03-08T20:34:19.286 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/nvme/host/nvme-core.ko 2024-03-08T20:34:19.286 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/nvme/host/nvme.ko 2024-03-08T20:34:19.286 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/parport 2024-03-08T20:34:19.286 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/parport/parport.ko 2024-03-08T20:34:19.286 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/parport/parport_cs.ko 2024-03-08T20:34:19.286 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/parport/parport_pc.ko 2024-03-08T20:34:19.287 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/parport/parport_serial.ko 2024-03-08T20:34:19.287 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/pci 2024-03-08T20:34:19.287 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/pci/hotplug 2024-03-08T20:34:19.287 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-03-08T20:34:19.287 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/pci/pcie 2024-03-08T20:34:19.287 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/pci/pcie/aer_inject.ko 2024-03-08T20:34:19.287 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/pci/xen-pcifront.ko 2024-03-08T20:34:19.287 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/pcmcia 2024-03-08T20:34:19.287 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/pcmcia/i82092.ko 2024-03-08T20:34:19.287 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/pcmcia/pd6729.ko 2024-03-08T20:34:19.287 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/pcmcia/yenta_socket.ko 2024-03-08T20:34:19.287 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/pinctrl 2024-03-08T20:34:19.287 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/pinctrl/intel 2024-03-08T20:34:19.288 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-03-08T20:34:19.288 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform 2024-03-08T20:34:19.288 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86 2024-03-08T20:34:19.288 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/acer-wmi.ko 2024-03-08T20:34:19.288 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/acerhdf.ko 2024-03-08T20:34:19.288 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/apple-gmux.ko 2024-03-08T20:34:19.288 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/asus-laptop.ko 2024-03-08T20:34:19.288 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-03-08T20:34:19.288 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/asus-wmi.ko 2024-03-08T20:34:19.288 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/classmate-laptop.ko 2024-03-08T20:34:19.288 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-03-08T20:34:19.288 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-03-08T20:34:19.288 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-03-08T20:34:19.288 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-03-08T20:34:19.288 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/hdaps.ko 2024-03-08T20:34:19.289 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/intel 2024-03-08T20:34:19.289 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-03-08T20:34:19.289 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/intel_ips.ko 2024-03-08T20:34:19.289 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/msi-wmi.ko 2024-03-08T20:34:19.289 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/mxm-wmi.ko 2024-03-08T20:34:19.289 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-03-08T20:34:19.289 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/samsung-laptop.ko 2024-03-08T20:34:19.289 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/samsung-q10.ko 2024-03-08T20:34:19.289 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/topstar-laptop.ko 2024-03-08T20:34:19.289 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-03-08T20:34:19.289 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/toshiba_haps.ko 2024-03-08T20:34:19.289 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/wmi-bmof.ko 2024-03-08T20:34:19.289 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/wmi.ko 2024-03-08T20:34:19.289 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/powercap 2024-03-08T20:34:19.290 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/powercap/intel_rapl_common.ko 2024-03-08T20:34:19.290 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/powercap/intel_rapl_msr.ko 2024-03-08T20:34:19.290 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/pps 2024-03-08T20:34:19.290 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/pps/clients 2024-03-08T20:34:19.290 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/pps/clients/pps-gpio.ko 2024-03-08T20:34:19.290 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/pps/clients/pps-ldisc.ko 2024-03-08T20:34:19.290 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/pps/clients/pps_parport.ko 2024-03-08T20:34:19.290 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/pps/pps_core.ko 2024-03-08T20:34:19.290 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ptp 2024-03-08T20:34:19.290 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ptp/ptp.ko 2024-03-08T20:34:19.290 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ptp/ptp_kvm.ko 2024-03-08T20:34:19.290 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc 2024-03-08T20:34:19.290 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-bq32k.ko 2024-03-08T20:34:19.290 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-ds1286.ko 2024-03-08T20:34:19.291 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-ds1307.ko 2024-03-08T20:34:19.291 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-ds1374.ko 2024-03-08T20:34:19.291 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-ds1511.ko 2024-03-08T20:34:19.291 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-ds1553.ko 2024-03-08T20:34:19.291 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-ds1672.ko 2024-03-08T20:34:19.291 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-ds1742.ko 2024-03-08T20:34:19.291 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-ds2404.ko 2024-03-08T20:34:19.291 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-ds3232.ko 2024-03-08T20:34:19.291 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-em3027.ko 2024-03-08T20:34:19.291 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-fm3130.ko 2024-03-08T20:34:19.291 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-isl12022.ko 2024-03-08T20:34:19.291 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-isl1208.ko 2024-03-08T20:34:19.291 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-m41t80.ko 2024-03-08T20:34:19.292 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-m48t35.ko 2024-03-08T20:34:19.292 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-m48t59.ko 2024-03-08T20:34:19.292 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-max6900.ko 2024-03-08T20:34:19.292 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-msm6242.ko 2024-03-08T20:34:19.292 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-pcf2127.ko 2024-03-08T20:34:19.292 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-pcf85063.ko 2024-03-08T20:34:19.292 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-pcf8523.ko 2024-03-08T20:34:19.292 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-pcf8563.ko 2024-03-08T20:34:19.292 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-pcf8583.ko 2024-03-08T20:34:19.292 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-rp5c01.ko 2024-03-08T20:34:19.292 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-rs5c372.ko 2024-03-08T20:34:19.292 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-03-08T20:34:19.292 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-rx8025.ko 2024-03-08T20:34:19.292 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-rx8581.ko 2024-03-08T20:34:19.293 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-03-08T20:34:19.293 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-x1205.ko 2024-03-08T20:34:19.293 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi 2024-03-08T20:34:19.293 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/3w-9xxx.ko 2024-03-08T20:34:19.293 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/3w-sas.ko 2024-03-08T20:34:19.293 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/3w-xxxx.ko 2024-03-08T20:34:19.293 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/BusLogic.ko 2024-03-08T20:34:19.293 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/a100u2w.ko 2024-03-08T20:34:19.293 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/aacraid 2024-03-08T20:34:19.293 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/aacraid/aacraid.ko 2024-03-08T20:34:19.293 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/advansys.ko 2024-03-08T20:34:19.293 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/aic7xxx 2024-03-08T20:34:19.293 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-03-08T20:34:19.293 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-03-08T20:34:19.293 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/am53c974.ko 2024-03-08T20:34:19.294 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/arcmsr 2024-03-08T20:34:19.294 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-03-08T20:34:19.294 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/atp870u.ko 2024-03-08T20:34:19.294 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/be2iscsi 2024-03-08T20:34:19.294 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-03-08T20:34:19.294 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/bfa 2024-03-08T20:34:19.294 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/bfa/bfa.ko 2024-03-08T20:34:19.294 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/bnx2fc 2024-03-08T20:34:19.294 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-03-08T20:34:19.294 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/bnx2i 2024-03-08T20:34:19.294 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-03-08T20:34:19.294 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/ch.ko 2024-03-08T20:34:19.294 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/csiostor 2024-03-08T20:34:19.294 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/csiostor/csiostor.ko 2024-03-08T20:34:19.295 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/cxgbi 2024-03-08T20:34:19.295 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/cxgbi/cxgb3i 2024-03-08T20:34:19.295 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-03-08T20:34:19.295 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/cxgbi/cxgb4i 2024-03-08T20:34:19.295 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-03-08T20:34:19.295 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-03-08T20:34:19.295 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/dc395x.ko 2024-03-08T20:34:19.295 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/device_handler 2024-03-08T20:34:19.295 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-03-08T20:34:19.295 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-03-08T20:34:19.295 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-03-08T20:34:19.295 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-03-08T20:34:19.295 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/dmx3191d.ko 2024-03-08T20:34:19.296 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/esas2r 2024-03-08T20:34:19.296 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/esas2r/esas2r.ko 2024-03-08T20:34:19.296 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/esp_scsi.ko 2024-03-08T20:34:19.296 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/fcoe 2024-03-08T20:34:19.296 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/fcoe/fcoe.ko 2024-03-08T20:34:19.296 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-03-08T20:34:19.296 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/fnic 2024-03-08T20:34:19.296 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/fnic/fnic.ko 2024-03-08T20:34:19.296 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/hpsa.ko 2024-03-08T20:34:19.296 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/hptiop.ko 2024-03-08T20:34:19.296 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/hv_storvsc.ko 2024-03-08T20:34:19.297 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/initio.ko 2024-03-08T20:34:19.297 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/ipr.ko 2024-03-08T20:34:19.297 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/ips.ko 2024-03-08T20:34:19.297 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/isci 2024-03-08T20:34:19.297 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/isci/isci.ko 2024-03-08T20:34:19.297 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-03-08T20:34:19.297 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/iscsi_tcp.ko 2024-03-08T20:34:19.297 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/libfc 2024-03-08T20:34:19.297 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/libfc/libfc.ko 2024-03-08T20:34:19.297 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/libiscsi.ko 2024-03-08T20:34:19.297 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/libiscsi_tcp.ko 2024-03-08T20:34:19.297 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/libsas 2024-03-08T20:34:19.298 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/libsas/libsas.ko 2024-03-08T20:34:19.298 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/lpfc 2024-03-08T20:34:19.298 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/lpfc/lpfc.ko 2024-03-08T20:34:19.298 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/megaraid 2024-03-08T20:34:19.298 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/megaraid.ko 2024-03-08T20:34:19.298 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-03-08T20:34:19.298 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-03-08T20:34:19.298 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-03-08T20:34:19.298 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/mpt3sas 2024-03-08T20:34:19.298 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-03-08T20:34:19.298 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/mvsas 2024-03-08T20:34:19.298 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/mvsas/mvsas.ko 2024-03-08T20:34:19.298 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/mvumi.ko 2024-03-08T20:34:19.299 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/pm8001 2024-03-08T20:34:19.299 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-03-08T20:34:19.299 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/pmcraid.ko 2024-03-08T20:34:19.299 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/qla1280.ko 2024-03-08T20:34:19.299 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/qla2xxx 2024-03-08T20:34:19.299 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-03-08T20:34:19.299 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-03-08T20:34:19.299 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/qla4xxx 2024-03-08T20:34:19.299 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-03-08T20:34:19.299 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/raid_class.ko 2024-03-08T20:34:19.299 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/scsi_debug.ko 2024-03-08T20:34:19.299 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/scsi_transport_fc.ko 2024-03-08T20:34:19.299 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-03-08T20:34:19.299 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/scsi_transport_sas.ko 2024-03-08T20:34:19.300 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/scsi_transport_spi.ko 2024-03-08T20:34:19.300 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/scsi_transport_srp.ko 2024-03-08T20:34:19.300 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/ses.ko 2024-03-08T20:34:19.300 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/st.ko 2024-03-08T20:34:19.300 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/stex.ko 2024-03-08T20:34:19.300 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/sym53c8xx_2 2024-03-08T20:34:19.300 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-03-08T20:34:19.300 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/virtio_scsi.ko 2024-03-08T20:34:19.300 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/vmw_pvscsi.ko 2024-03-08T20:34:19.300 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/wd719x.ko 2024-03-08T20:34:19.300 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ssb 2024-03-08T20:34:19.300 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ssb/ssb.ko 2024-03-08T20:34:19.300 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/target 2024-03-08T20:34:19.300 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/target/iscsi 2024-03-08T20:34:19.301 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-03-08T20:34:19.301 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/target/loopback 2024-03-08T20:34:19.301 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/target/loopback/tcm_loop.ko 2024-03-08T20:34:19.301 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/target/target_core_file.ko 2024-03-08T20:34:19.301 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/target/target_core_iblock.ko 2024-03-08T20:34:19.301 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/target/target_core_mod.ko 2024-03-08T20:34:19.301 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/target/target_core_pscsi.ko 2024-03-08T20:34:19.301 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/target/target_core_user.ko 2024-03-08T20:34:19.301 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/target/tcm_fc 2024-03-08T20:34:19.301 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-03-08T20:34:19.301 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal 2024-03-08T20:34:19.301 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel 2024-03-08T20:34:19.301 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel/int340x_thermal 2024-03-08T20:34:19.301 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-03-08T20:34:19.302 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-03-08T20:34:19.302 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-03-08T20:34:19.302 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-03-08T20:34:19.302 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-03-08T20:34:19.302 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-03-08T20:34:19.302 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-03-08T20:34:19.302 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-03-08T20:34:19.302 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-03-08T20:34:19.302 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-03-08T20:34:19.302 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-03-08T20:34:19.302 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-03-08T20:34:19.302 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-03-08T20:34:19.302 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-03-08T20:34:19.303 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-03-08T20:34:19.303 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-03-08T20:34:19.303 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-03-08T20:34:19.303 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-03-08T20:34:19.303 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/tty 2024-03-08T20:34:19.303 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/tty/ipwireless 2024-03-08T20:34:19.303 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-03-08T20:34:19.303 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/tty/n_gsm.ko 2024-03-08T20:34:19.303 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/tty/n_hdlc.ko 2024-03-08T20:34:19.303 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/tty/nozomi.ko 2024-03-08T20:34:19.303 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/tty/serial 2024-03-08T20:34:19.303 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/tty/serial/8250 2024-03-08T20:34:19.303 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-03-08T20:34:19.303 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/tty/serial/arc_uart.ko 2024-03-08T20:34:19.304 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/tty/serial/jsm 2024-03-08T20:34:19.304 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/tty/serial/jsm/jsm.ko 2024-03-08T20:34:19.304 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ufs 2024-03-08T20:34:19.304 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ufs/core 2024-03-08T20:34:19.304 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ufs/core/ufshcd-core.ko 2024-03-08T20:34:19.304 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ufs/host 2024-03-08T20:34:19.304 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-03-08T20:34:19.304 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/uio 2024-03-08T20:34:19.304 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/uio/uio.ko 2024-03-08T20:34:19.304 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/uio/uio_aec.ko 2024-03-08T20:34:19.304 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/uio/uio_cif.ko 2024-03-08T20:34:19.304 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/uio/uio_pci_generic.ko 2024-03-08T20:34:19.304 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/uio/uio_sercos3.ko 2024-03-08T20:34:19.304 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb 2024-03-08T20:34:19.305 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/class 2024-03-08T20:34:19.305 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/class/cdc-acm.ko 2024-03-08T20:34:19.305 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/class/cdc-wdm.ko 2024-03-08T20:34:19.305 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/class/usblp.ko 2024-03-08T20:34:19.305 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/class/usbtmc.ko 2024-03-08T20:34:19.305 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/host 2024-03-08T20:34:19.305 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/host/sl811-hcd.ko 2024-03-08T20:34:19.305 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/image 2024-03-08T20:34:19.305 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/image/mdc800.ko 2024-03-08T20:34:19.305 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/image/microtek.ko 2024-03-08T20:34:19.305 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/misc 2024-03-08T20:34:19.305 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/misc/adutux.ko 2024-03-08T20:34:19.305 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/misc/appledisplay.ko 2024-03-08T20:34:19.305 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/misc/emi26.ko 2024-03-08T20:34:19.306 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/misc/emi62.ko 2024-03-08T20:34:19.306 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/misc/ezusb.ko 2024-03-08T20:34:19.306 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/misc/idmouse.ko 2024-03-08T20:34:19.306 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/misc/iowarrior.ko 2024-03-08T20:34:19.306 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/misc/isight_firmware.ko 2024-03-08T20:34:19.306 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/misc/ldusb.ko 2024-03-08T20:34:19.306 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/misc/legousbtower.ko 2024-03-08T20:34:19.306 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/misc/sisusbvga 2024-03-08T20:34:19.306 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-03-08T20:34:19.306 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/misc/trancevibrator.ko 2024-03-08T20:34:19.306 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/misc/usb3503.ko 2024-03-08T20:34:19.306 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/misc/usblcd.ko 2024-03-08T20:34:19.306 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/misc/usbsevseg.ko 2024-03-08T20:34:19.307 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/misc/uss720.ko 2024-03-08T20:34:19.307 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/misc/yurex.ko 2024-03-08T20:34:19.307 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/phy 2024-03-08T20:34:19.307 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/phy/phy-generic.ko 2024-03-08T20:34:19.307 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial 2024-03-08T20:34:19.307 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/aircable.ko 2024-03-08T20:34:19.307 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/ark3116.ko 2024-03-08T20:34:19.307 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/belkin_sa.ko 2024-03-08T20:34:19.307 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/ch341.ko 2024-03-08T20:34:19.307 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/cp210x.ko 2024-03-08T20:34:19.307 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/cyberjack.ko 2024-03-08T20:34:19.307 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/cypress_m8.ko 2024-03-08T20:34:19.307 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/digi_acceleport.ko 2024-03-08T20:34:19.308 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/empeg.ko 2024-03-08T20:34:19.308 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/ftdi_sio.ko 2024-03-08T20:34:19.308 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/garmin_gps.ko 2024-03-08T20:34:19.308 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/io_edgeport.ko 2024-03-08T20:34:19.308 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/io_ti.ko 2024-03-08T20:34:19.308 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/ipaq.ko 2024-03-08T20:34:19.308 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/ipw.ko 2024-03-08T20:34:19.308 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/ir-usb.ko 2024-03-08T20:34:19.308 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-03-08T20:34:19.308 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/keyspan.ko 2024-03-08T20:34:19.308 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/keyspan_pda.ko 2024-03-08T20:34:19.308 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/kl5kusb105.ko 2024-03-08T20:34:19.308 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/kobil_sct.ko 2024-03-08T20:34:19.309 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/mct_u232.ko 2024-03-08T20:34:19.309 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/mos7720.ko 2024-03-08T20:34:19.309 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/mos7840.ko 2024-03-08T20:34:19.309 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/navman.ko 2024-03-08T20:34:19.309 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/omninet.ko 2024-03-08T20:34:19.309 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/opticon.ko 2024-03-08T20:34:19.309 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/option.ko 2024-03-08T20:34:19.309 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/oti6858.ko 2024-03-08T20:34:19.309 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/pl2303.ko 2024-03-08T20:34:19.309 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/qcaux.ko 2024-03-08T20:34:19.309 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/qcserial.ko 2024-03-08T20:34:19.309 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/quatech2.ko 2024-03-08T20:34:19.309 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/safe_serial.ko 2024-03-08T20:34:19.310 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/sierra.ko 2024-03-08T20:34:19.310 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/spcp8x5.ko 2024-03-08T20:34:19.310 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/ssu100.ko 2024-03-08T20:34:19.310 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/symbolserial.ko 2024-03-08T20:34:19.310 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-03-08T20:34:19.310 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-03-08T20:34:19.310 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/usb_debug.ko 2024-03-08T20:34:19.310 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/usb_wwan.ko 2024-03-08T20:34:19.310 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/visor.ko 2024-03-08T20:34:19.310 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/whiteheat.ko 2024-03-08T20:34:19.310 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/xsens_mt.ko 2024-03-08T20:34:19.310 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/storage 2024-03-08T20:34:19.310 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/storage/uas.ko 2024-03-08T20:34:19.311 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/storage/ums-alauda.ko 2024-03-08T20:34:19.311 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/storage/ums-cypress.ko 2024-03-08T20:34:19.311 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/storage/ums-datafab.ko 2024-03-08T20:34:19.311 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-03-08T20:34:19.311 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/storage/ums-freecom.ko 2024-03-08T20:34:19.311 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/storage/ums-isd200.ko 2024-03-08T20:34:19.311 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-03-08T20:34:19.311 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/storage/ums-karma.ko 2024-03-08T20:34:19.311 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/storage/ums-onetouch.ko 2024-03-08T20:34:19.311 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/storage/ums-realtek.ko 2024-03-08T20:34:19.311 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/storage/ums-sddr09.ko 2024-03-08T20:34:19.311 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/storage/ums-sddr55.ko 2024-03-08T20:34:19.311 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/storage/ums-usbat.ko 2024-03-08T20:34:19.311 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/storage/usb-storage.ko 2024-03-08T20:34:19.312 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/usbip 2024-03-08T20:34:19.312 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/usbip/usbip-core.ko 2024-03-08T20:34:19.312 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/usbip/usbip-host.ko 2024-03-08T20:34:19.312 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-03-08T20:34:19.312 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/vfio 2024-03-08T20:34:19.312 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/vfio/pci 2024-03-08T20:34:19.312 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-03-08T20:34:19.312 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/vfio/pci/vfio-pci.ko 2024-03-08T20:34:19.312 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/vfio/vfio.ko 2024-03-08T20:34:19.312 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-03-08T20:34:19.312 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/vhost 2024-03-08T20:34:19.312 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/vhost/vhost.ko 2024-03-08T20:34:19.312 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/vhost/vhost_iotlb.ko 2024-03-08T20:34:19.312 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/vhost/vhost_net.ko 2024-03-08T20:34:19.313 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/vhost/vhost_scsi.ko 2024-03-08T20:34:19.313 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/video 2024-03-08T20:34:19.313 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/video/backlight 2024-03-08T20:34:19.313 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/video/backlight/apple_bl.ko 2024-03-08T20:34:19.313 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/video/backlight/lcd.ko 2024-03-08T20:34:19.313 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/video/backlight/platform_lcd.ko 2024-03-08T20:34:19.313 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/virtio 2024-03-08T20:34:19.313 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/virtio/virtio.ko 2024-03-08T20:34:19.313 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/virtio/virtio_balloon.ko 2024-03-08T20:34:19.313 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/virtio/virtio_mmio.ko 2024-03-08T20:34:19.313 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/virtio/virtio_pci.ko 2024-03-08T20:34:19.313 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-03-08T20:34:19.313 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-03-08T20:34:19.313 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/virtio/virtio_ring.ko 2024-03-08T20:34:19.314 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/w1 2024-03-08T20:34:19.314 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/w1/masters 2024-03-08T20:34:19.314 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/w1/masters/ds2482.ko 2024-03-08T20:34:19.314 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/w1/masters/ds2490.ko 2024-03-08T20:34:19.314 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/w1/slaves 2024-03-08T20:34:19.314 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-03-08T20:34:19.314 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-03-08T20:34:19.314 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-03-08T20:34:19.314 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-03-08T20:34:19.314 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-03-08T20:34:19.314 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-03-08T20:34:19.314 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-03-08T20:34:19.314 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-03-08T20:34:19.315 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-03-08T20:34:19.315 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/w1/slaves/w1_smem.ko 2024-03-08T20:34:19.315 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/w1/slaves/w1_therm.ko 2024-03-08T20:34:19.315 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/w1/wire.ko 2024-03-08T20:34:19.315 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog 2024-03-08T20:34:19.315 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/alim1535_wdt.ko 2024-03-08T20:34:19.315 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/alim7101_wdt.ko 2024-03-08T20:34:19.315 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/f71808e_wdt.ko 2024-03-08T20:34:19.315 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/hpwdt.ko 2024-03-08T20:34:19.315 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/i6300esb.ko 2024-03-08T20:34:19.315 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-03-08T20:34:19.315 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/iTCO_wdt.ko 2024-03-08T20:34:19.315 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/ib700wdt.ko 2024-03-08T20:34:19.315 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/ibmasr.ko 2024-03-08T20:34:19.316 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-03-08T20:34:19.316 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/it8712f_wdt.ko 2024-03-08T20:34:19.316 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/it87_wdt.ko 2024-03-08T20:34:19.316 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/machzwd.ko 2024-03-08T20:34:19.316 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/nv_tco.ko 2024-03-08T20:34:19.316 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/pcwd_pci.ko 2024-03-08T20:34:19.316 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/pcwd_usb.ko 2024-03-08T20:34:19.316 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-03-08T20:34:19.316 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/sch311x_wdt.ko 2024-03-08T20:34:19.316 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/softdog.ko 2024-03-08T20:34:19.316 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/sp5100_tco.ko 2024-03-08T20:34:19.316 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/via_wdt.ko 2024-03-08T20:34:19.317 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-03-08T20:34:19.317 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/w83877f_wdt.ko 2024-03-08T20:34:19.317 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/w83977f_wdt.ko 2024-03-08T20:34:19.317 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/wdt_pci.ko 2024-03-08T20:34:19.317 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/xen_wdt.ko 2024-03-08T20:34:19.317 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/xen 2024-03-08T20:34:19.317 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/xen/xen-acpi-processor.ko 2024-03-08T20:34:19.317 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/xen/xen-evtchn.ko 2024-03-08T20:34:19.317 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/xen/xen-gntalloc.ko 2024-03-08T20:34:19.317 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/xen/xen-gntdev.ko 2024-03-08T20:34:19.317 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/xen/xen-pciback 2024-03-08T20:34:19.317 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-03-08T20:34:19.317 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/xen/xen-privcmd.ko 2024-03-08T20:34:19.317 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/xen/xenfs 2024-03-08T20:34:19.318 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/xen/xenfs/xenfs.ko 2024-03-08T20:34:19.318 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs 2024-03-08T20:34:19.318 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/9p 2024-03-08T20:34:19.318 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/9p/9p.ko 2024-03-08T20:34:19.318 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/affs 2024-03-08T20:34:19.318 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/affs/affs.ko 2024-03-08T20:34:19.318 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/befs 2024-03-08T20:34:19.318 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/befs/befs.ko 2024-03-08T20:34:19.318 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/binfmt_misc.ko 2024-03-08T20:34:19.318 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/btrfs 2024-03-08T20:34:19.318 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/btrfs/btrfs.ko 2024-03-08T20:34:19.318 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/cachefiles 2024-03-08T20:34:19.318 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/cachefiles/cachefiles.ko 2024-03-08T20:34:19.318 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/ceph 2024-03-08T20:34:19.319 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/ceph/ceph.ko 2024-03-08T20:34:19.319 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/coda 2024-03-08T20:34:19.319 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/coda/coda.ko 2024-03-08T20:34:19.319 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/cramfs 2024-03-08T20:34:19.319 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/cramfs/cramfs.ko 2024-03-08T20:34:19.319 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/dlm 2024-03-08T20:34:19.319 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/dlm/dlm.ko 2024-03-08T20:34:19.319 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/ecryptfs 2024-03-08T20:34:19.319 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/ecryptfs/ecryptfs.ko 2024-03-08T20:34:19.319 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/f2fs 2024-03-08T20:34:19.319 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/f2fs/f2fs.ko 2024-03-08T20:34:19.319 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/fat 2024-03-08T20:34:19.319 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/fat/fat.ko 2024-03-08T20:34:19.320 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/fat/msdos.ko 2024-03-08T20:34:19.320 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/fat/vfat.ko 2024-03-08T20:34:19.320 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/fuse 2024-03-08T20:34:19.320 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/fuse/cuse.ko 2024-03-08T20:34:19.320 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/fuse/fuse.ko 2024-03-08T20:34:19.320 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/gfs2 2024-03-08T20:34:19.320 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/gfs2/gfs2.ko 2024-03-08T20:34:19.320 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/hfs 2024-03-08T20:34:19.320 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/hfs/hfs.ko 2024-03-08T20:34:19.320 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/hfsplus 2024-03-08T20:34:19.320 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/hfsplus/hfsplus.ko 2024-03-08T20:34:19.320 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/isofs 2024-03-08T20:34:19.320 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/isofs/isofs.ko 2024-03-08T20:34:19.321 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/jfs 2024-03-08T20:34:19.321 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/jfs/jfs.ko 2024-03-08T20:34:19.321 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/lockd 2024-03-08T20:34:19.321 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/lockd/lockd.ko 2024-03-08T20:34:19.321 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/minix 2024-03-08T20:34:19.321 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/minix/minix.ko 2024-03-08T20:34:19.321 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/netfs 2024-03-08T20:34:19.321 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/netfs/netfs.ko 2024-03-08T20:34:19.321 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nfs 2024-03-08T20:34:19.321 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nfs/blocklayout 2024-03-08T20:34:19.321 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-03-08T20:34:19.321 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nfs/filelayout 2024-03-08T20:34:19.321 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-03-08T20:34:19.322 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nfs/flexfilelayout 2024-03-08T20:34:19.322 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-03-08T20:34:19.322 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nfs/nfs.ko 2024-03-08T20:34:19.322 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nfs/nfsv3.ko 2024-03-08T20:34:19.322 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nfs/nfsv4.ko 2024-03-08T20:34:19.322 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nfs_common 2024-03-08T20:34:19.322 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nfs_common/grace.ko 2024-03-08T20:34:19.322 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nfs_common/nfs_acl.ko 2024-03-08T20:34:19.322 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nfsd 2024-03-08T20:34:19.322 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nfsd/nfsd.ko 2024-03-08T20:34:19.322 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nilfs2 2024-03-08T20:34:19.322 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nilfs2/nilfs2.ko 2024-03-08T20:34:19.322 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls 2024-03-08T20:34:19.322 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/mac-celtic.ko 2024-03-08T20:34:19.323 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/mac-centeuro.ko 2024-03-08T20:34:19.323 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/mac-croatian.ko 2024-03-08T20:34:19.323 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/mac-cyrillic.ko 2024-03-08T20:34:19.323 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/mac-gaelic.ko 2024-03-08T20:34:19.323 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/mac-greek.ko 2024-03-08T20:34:19.323 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/mac-iceland.ko 2024-03-08T20:34:19.323 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/mac-inuit.ko 2024-03-08T20:34:19.323 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/mac-roman.ko 2024-03-08T20:34:19.323 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/mac-romanian.ko 2024-03-08T20:34:19.323 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/mac-turkish.ko 2024-03-08T20:34:19.323 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp1250.ko 2024-03-08T20:34:19.323 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp1251.ko 2024-03-08T20:34:19.323 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp1255.ko 2024-03-08T20:34:19.323 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp737.ko 2024-03-08T20:34:19.324 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp775.ko 2024-03-08T20:34:19.324 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp850.ko 2024-03-08T20:34:19.324 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp852.ko 2024-03-08T20:34:19.324 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp855.ko 2024-03-08T20:34:19.324 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp857.ko 2024-03-08T20:34:19.324 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp860.ko 2024-03-08T20:34:19.324 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp861.ko 2024-03-08T20:34:19.324 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp862.ko 2024-03-08T20:34:19.324 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp863.ko 2024-03-08T20:34:19.324 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp864.ko 2024-03-08T20:34:19.324 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp865.ko 2024-03-08T20:34:19.324 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp866.ko 2024-03-08T20:34:19.324 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp869.ko 2024-03-08T20:34:19.324 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp874.ko 2024-03-08T20:34:19.325 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp932.ko 2024-03-08T20:34:19.325 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp936.ko 2024-03-08T20:34:19.325 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp949.ko 2024-03-08T20:34:19.325 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp950.ko 2024-03-08T20:34:19.325 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_euc-jp.ko 2024-03-08T20:34:19.325 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_iso8859-1.ko 2024-03-08T20:34:19.325 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_iso8859-13.ko 2024-03-08T20:34:19.325 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_iso8859-14.ko 2024-03-08T20:34:19.325 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_iso8859-15.ko 2024-03-08T20:34:19.325 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_iso8859-2.ko 2024-03-08T20:34:19.325 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_iso8859-3.ko 2024-03-08T20:34:19.325 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_iso8859-4.ko 2024-03-08T20:34:19.325 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_iso8859-5.ko 2024-03-08T20:34:19.325 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_iso8859-6.ko 2024-03-08T20:34:19.326 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_iso8859-7.ko 2024-03-08T20:34:19.326 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_iso8859-9.ko 2024-03-08T20:34:19.326 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_koi8-r.ko 2024-03-08T20:34:19.326 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_koi8-ru.ko 2024-03-08T20:34:19.326 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_koi8-u.ko 2024-03-08T20:34:19.326 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_ucs2_utils.ko 2024-03-08T20:34:19.326 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_utf8.ko 2024-03-08T20:34:19.326 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/ocfs2 2024-03-08T20:34:19.326 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/ocfs2/cluster 2024-03-08T20:34:19.326 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-03-08T20:34:19.326 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/ocfs2/dlm 2024-03-08T20:34:19.326 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-03-08T20:34:19.327 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/ocfs2/dlmfs 2024-03-08T20:34:19.327 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-03-08T20:34:19.327 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/ocfs2/ocfs2.ko 2024-03-08T20:34:19.327 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-03-08T20:34:19.327 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-03-08T20:34:19.327 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-03-08T20:34:19.327 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/overlayfs 2024-03-08T20:34:19.327 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/overlayfs/overlay.ko 2024-03-08T20:34:19.327 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/pstore 2024-03-08T20:34:19.327 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/pstore/ramoops.ko 2024-03-08T20:34:19.327 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/reiserfs 2024-03-08T20:34:19.327 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/reiserfs/reiserfs.ko 2024-03-08T20:34:19.327 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/romfs 2024-03-08T20:34:19.327 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/romfs/romfs.ko 2024-03-08T20:34:19.328 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/smb 2024-03-08T20:34:19.328 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/smb/client 2024-03-08T20:34:19.328 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/smb/client/cifs.ko 2024-03-08T20:34:19.328 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/smb/common 2024-03-08T20:34:19.328 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/smb/common/cifs_arc4.ko 2024-03-08T20:34:19.328 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/smb/common/cifs_md4.ko 2024-03-08T20:34:19.328 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/squashfs 2024-03-08T20:34:19.328 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/squashfs/squashfs.ko 2024-03-08T20:34:19.328 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/sysv 2024-03-08T20:34:19.328 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/sysv/sysv.ko 2024-03-08T20:34:19.328 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/udf 2024-03-08T20:34:19.328 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/udf/udf.ko 2024-03-08T20:34:19.328 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/ufs 2024-03-08T20:34:19.328 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/ufs/ufs.ko 2024-03-08T20:34:19.328 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/xfs 2024-03-08T20:34:19.329 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/xfs/xfs.ko 2024-03-08T20:34:19.329 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/kernel 2024-03-08T20:34:19.329 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/kernel/rcu 2024-03-08T20:34:19.329 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/kernel/rcu/rcutorture.ko 2024-03-08T20:34:19.329 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/kernel/torture.ko 2024-03-08T20:34:19.329 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/kernel/trace 2024-03-08T20:34:19.329 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-03-08T20:34:19.329 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib 2024-03-08T20:34:19.329 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/asn1_encoder.ko 2024-03-08T20:34:19.329 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/crc-itu-t.ko 2024-03-08T20:34:19.329 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/crc8.ko 2024-03-08T20:34:19.329 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/crypto 2024-03-08T20:34:19.329 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/crypto/libarc4.ko 2024-03-08T20:34:19.330 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/crypto/libdes.ko 2024-03-08T20:34:19.330 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/libcrc32c.ko 2024-03-08T20:34:19.330 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/lru_cache.ko 2024-03-08T20:34:19.330 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/lz4 2024-03-08T20:34:19.330 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/lz4/lz4_compress.ko 2024-03-08T20:34:19.330 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/lz4/lz4hc_compress.ko 2024-03-08T20:34:19.330 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/math 2024-03-08T20:34:19.330 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/math/cordic.ko 2024-03-08T20:34:19.330 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/raid6 2024-03-08T20:34:19.330 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/raid6/raid6_pq.ko 2024-03-08T20:34:19.330 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/reed_solomon 2024-03-08T20:34:19.330 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/reed_solomon/reed_solomon.ko 2024-03-08T20:34:19.330 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/ts_bm.ko 2024-03-08T20:34:19.331 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/ts_fsm.ko 2024-03-08T20:34:19.331 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/ts_kmp.ko 2024-03-08T20:34:19.331 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/zstd 2024-03-08T20:34:19.331 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/zstd/zstd_compress.ko 2024-03-08T20:34:19.331 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/mm 2024-03-08T20:34:19.331 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/mm/hwpoison-inject.ko 2024-03-08T20:34:19.331 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net 2024-03-08T20:34:19.331 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/6lowpan 2024-03-08T20:34:19.331 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/6lowpan/6lowpan.ko 2024-03-08T20:34:19.331 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/6lowpan/nhc_dest.ko 2024-03-08T20:34:19.331 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/6lowpan/nhc_fragment.ko 2024-03-08T20:34:19.331 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/6lowpan/nhc_hop.ko 2024-03-08T20:34:19.331 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/6lowpan/nhc_ipv6.ko 2024-03-08T20:34:19.331 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/6lowpan/nhc_mobility.ko 2024-03-08T20:34:19.331 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/6lowpan/nhc_routing.ko 2024-03-08T20:34:19.332 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/6lowpan/nhc_udp.ko 2024-03-08T20:34:19.332 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/802 2024-03-08T20:34:19.332 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/802/garp.ko 2024-03-08T20:34:19.332 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/802/mrp.ko 2024-03-08T20:34:19.332 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/802/p8022.ko 2024-03-08T20:34:19.332 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/802/psnap.ko 2024-03-08T20:34:19.332 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/802/stp.ko 2024-03-08T20:34:19.332 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/8021q 2024-03-08T20:34:19.332 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/8021q/8021q.ko 2024-03-08T20:34:19.332 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/9p 2024-03-08T20:34:19.332 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/9p/9pnet.ko 2024-03-08T20:34:19.332 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/9p/9pnet_fd.ko 2024-03-08T20:34:19.332 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/9p/9pnet_rdma.ko 2024-03-08T20:34:19.332 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/9p/9pnet_virtio.ko 2024-03-08T20:34:19.333 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/batman-adv 2024-03-08T20:34:19.333 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/batman-adv/batman-adv.ko 2024-03-08T20:34:19.333 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge 2024-03-08T20:34:19.333 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/br_netfilter.ko 2024-03-08T20:34:19.333 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/bridge.ko 2024-03-08T20:34:19.333 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter 2024-03-08T20:34:19.333 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-03-08T20:34:19.333 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebt_among.ko 2024-03-08T20:34:19.333 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebt_arp.ko 2024-03-08T20:34:19.333 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-03-08T20:34:19.333 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-03-08T20:34:19.333 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebt_ip.ko 2024-03-08T20:34:19.333 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-03-08T20:34:19.333 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebt_limit.ko 2024-03-08T20:34:19.334 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebt_log.ko 2024-03-08T20:34:19.334 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebt_mark.ko 2024-03-08T20:34:19.334 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-03-08T20:34:19.334 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-03-08T20:34:19.334 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-03-08T20:34:19.334 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-03-08T20:34:19.334 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebt_snat.ko 2024-03-08T20:34:19.334 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebt_stp.ko 2024-03-08T20:34:19.334 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-03-08T20:34:19.334 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-03-08T20:34:19.334 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-03-08T20:34:19.334 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-03-08T20:34:19.334 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebtables.ko 2024-03-08T20:34:19.335 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-03-08T20:34:19.335 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-03-08T20:34:19.335 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-03-08T20:34:19.335 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ceph 2024-03-08T20:34:19.335 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ceph/libceph.ko 2024-03-08T20:34:19.335 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/core 2024-03-08T20:34:19.335 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/core/failover.ko 2024-03-08T20:34:19.335 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/core/pktgen.ko 2024-03-08T20:34:19.335 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/dccp 2024-03-08T20:34:19.335 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/dccp/dccp.ko 2024-03-08T20:34:19.335 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/dccp/dccp_diag.ko 2024-03-08T20:34:19.335 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/dccp/dccp_ipv4.ko 2024-03-08T20:34:19.335 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/dccp/dccp_ipv6.ko 2024-03-08T20:34:19.335 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/dns_resolver 2024-03-08T20:34:19.336 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/dns_resolver/dns_resolver.ko 2024-03-08T20:34:19.336 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/dsa 2024-03-08T20:34:19.336 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/dsa/dsa_core.ko 2024-03-08T20:34:19.336 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/dsa/tag_brcm.ko 2024-03-08T20:34:19.336 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/dsa/tag_dsa.ko 2024-03-08T20:34:19.336 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/dsa/tag_none.ko 2024-03-08T20:34:19.336 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/dsa/tag_trailer.ko 2024-03-08T20:34:19.336 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ieee802154 2024-03-08T20:34:19.336 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ieee802154/6lowpan 2024-03-08T20:34:19.336 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-03-08T20:34:19.336 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ieee802154/ieee802154.ko 2024-03-08T20:34:19.336 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ieee802154/ieee802154_socket.ko 2024-03-08T20:34:19.336 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4 2024-03-08T20:34:19.336 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/ah4.ko 2024-03-08T20:34:19.336 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/esp4.ko 2024-03-08T20:34:19.336 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/fou.ko 2024-03-08T20:34:19.337 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/gre.ko 2024-03-08T20:34:19.337 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/inet_diag.ko 2024-03-08T20:34:19.337 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/ip_gre.ko 2024-03-08T20:34:19.337 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/ip_tunnel.ko 2024-03-08T20:34:19.337 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/ip_vti.ko 2024-03-08T20:34:19.337 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/ipcomp.ko 2024-03-08T20:34:19.337 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/ipip.ko 2024-03-08T20:34:19.337 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter 2024-03-08T20:34:19.337 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/arp_tables.ko 2024-03-08T20:34:19.337 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-03-08T20:34:19.337 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-03-08T20:34:19.337 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-03-08T20:34:19.337 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-03-08T20:34:19.337 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-03-08T20:34:19.338 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-03-08T20:34:19.338 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-03-08T20:34:19.338 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-03-08T20:34:19.338 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-03-08T20:34:19.338 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/iptable_security.ko 2024-03-08T20:34:19.338 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-03-08T20:34:19.338 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-03-08T20:34:19.338 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-03-08T20:34:19.338 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-03-08T20:34:19.338 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-03-08T20:34:19.338 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-03-08T20:34:19.338 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-03-08T20:34:19.338 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-03-08T20:34:19.339 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-03-08T20:34:19.339 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-03-08T20:34:19.339 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/tcp_bic.ko 2024-03-08T20:34:19.339 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/tcp_dctcp.ko 2024-03-08T20:34:19.339 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/tcp_diag.ko 2024-03-08T20:34:19.339 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/tcp_highspeed.ko 2024-03-08T20:34:19.339 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/tcp_htcp.ko 2024-03-08T20:34:19.339 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/tcp_hybla.ko 2024-03-08T20:34:19.339 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/tcp_illinois.ko 2024-03-08T20:34:19.339 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/tcp_lp.ko 2024-03-08T20:34:19.339 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/tcp_scalable.ko 2024-03-08T20:34:19.339 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/tcp_vegas.ko 2024-03-08T20:34:19.339 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/tcp_veno.ko 2024-03-08T20:34:19.339 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/tcp_westwood.ko 2024-03-08T20:34:19.340 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/tcp_yeah.ko 2024-03-08T20:34:19.340 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/tunnel4.ko 2024-03-08T20:34:19.340 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/udp_diag.ko 2024-03-08T20:34:19.340 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/udp_tunnel.ko 2024-03-08T20:34:19.340 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/xfrm4_tunnel.ko 2024-03-08T20:34:19.340 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6 2024-03-08T20:34:19.340 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/ah6.ko 2024-03-08T20:34:19.340 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/esp6.ko 2024-03-08T20:34:19.340 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/fou6.ko 2024-03-08T20:34:19.340 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/ip6_tunnel.ko 2024-03-08T20:34:19.340 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-03-08T20:34:19.340 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/ip6_vti.ko 2024-03-08T20:34:19.340 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/ipcomp6.ko 2024-03-08T20:34:19.341 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter 2024-03-08T20:34:19.341 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-03-08T20:34:19.341 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-03-08T20:34:19.341 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-03-08T20:34:19.341 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-03-08T20:34:19.341 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-03-08T20:34:19.341 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-03-08T20:34:19.341 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-03-08T20:34:19.341 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-03-08T20:34:19.341 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-03-08T20:34:19.341 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-03-08T20:34:19.341 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-03-08T20:34:19.341 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-03-08T20:34:19.341 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-03-08T20:34:19.341 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-03-08T20:34:19.342 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-03-08T20:34:19.342 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-03-08T20:34:19.342 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-03-08T20:34:19.342 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-03-08T20:34:19.342 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-03-08T20:34:19.342 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-03-08T20:34:19.342 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-03-08T20:34:19.342 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-03-08T20:34:19.342 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-03-08T20:34:19.342 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-03-08T20:34:19.342 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/sit.ko 2024-03-08T20:34:19.342 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/tunnel6.ko 2024-03-08T20:34:19.342 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/xfrm6_tunnel.ko 2024-03-08T20:34:19.342 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/key 2024-03-08T20:34:19.343 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/key/af_key.ko 2024-03-08T20:34:19.343 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/l2tp 2024-03-08T20:34:19.343 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/l2tp/l2tp_core.ko 2024-03-08T20:34:19.343 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/l2tp/l2tp_debugfs.ko 2024-03-08T20:34:19.343 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/l2tp/l2tp_eth.ko 2024-03-08T20:34:19.343 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/l2tp/l2tp_ip.ko 2024-03-08T20:34:19.343 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/l2tp/l2tp_ip6.ko 2024-03-08T20:34:19.343 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/l2tp/l2tp_netlink.ko 2024-03-08T20:34:19.343 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/l2tp/l2tp_ppp.ko 2024-03-08T20:34:19.343 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/llc 2024-03-08T20:34:19.343 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/llc/llc.ko 2024-03-08T20:34:19.343 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/mac802154 2024-03-08T20:34:19.343 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/mac802154/mac802154.ko 2024-03-08T20:34:19.343 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/mpls 2024-03-08T20:34:19.344 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/mpls/mpls_gso.ko 2024-03-08T20:34:19.344 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter 2024-03-08T20:34:19.344 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipset 2024-03-08T20:34:19.344 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipset/ip_set.ko 2024-03-08T20:34:19.344 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-03-08T20:34:19.344 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-03-08T20:34:19.344 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-03-08T20:34:19.344 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-03-08T20:34:19.344 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-03-08T20:34:19.344 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-03-08T20:34:19.344 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-03-08T20:34:19.344 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-03-08T20:34:19.344 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-03-08T20:34:19.344 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-03-08T20:34:19.344 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-03-08T20:34:19.345 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-03-08T20:34:19.345 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-03-08T20:34:19.345 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-03-08T20:34:19.345 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-03-08T20:34:19.345 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipvs 2024-03-08T20:34:19.345 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipvs/ip_vs.ko 2024-03-08T20:34:19.345 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-03-08T20:34:19.345 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-03-08T20:34:19.345 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-03-08T20:34:19.345 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-03-08T20:34:19.345 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-03-08T20:34:19.345 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-03-08T20:34:19.345 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-03-08T20:34:19.345 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-03-08T20:34:19.345 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-03-08T20:34:19.346 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-03-08T20:34:19.346 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-03-08T20:34:19.346 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-03-08T20:34:19.346 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-03-08T20:34:19.346 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_conncount.ko 2024-03-08T20:34:19.346 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_conntrack.ko 2024-03-08T20:34:19.346 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-03-08T20:34:19.346 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-03-08T20:34:19.346 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-03-08T20:34:19.346 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_conntrack_h323.ko 2024-03-08T20:34:19.346 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_conntrack_irc.ko 2024-03-08T20:34:19.346 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-03-08T20:34:19.346 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-03-08T20:34:19.347 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-03-08T20:34:19.347 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_conntrack_sane.ko 2024-03-08T20:34:19.347 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_conntrack_sip.ko 2024-03-08T20:34:19.347 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-03-08T20:34:19.347 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-03-08T20:34:19.347 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_dup_netdev.ko 2024-03-08T20:34:19.347 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_flow_table.ko 2024-03-08T20:34:19.347 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_flow_table_inet.ko 2024-03-08T20:34:19.347 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_log_syslog.ko 2024-03-08T20:34:19.347 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_nat.ko 2024-03-08T20:34:19.347 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_nat_amanda.ko 2024-03-08T20:34:19.347 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_nat_ftp.ko 2024-03-08T20:34:19.347 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_nat_irc.ko 2024-03-08T20:34:19.348 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_nat_sip.ko 2024-03-08T20:34:19.348 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_nat_tftp.ko 2024-03-08T20:34:19.348 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_synproxy_core.ko 2024-03-08T20:34:19.348 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_tables.ko 2024-03-08T20:34:19.348 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nfnetlink.ko 2024-03-08T20:34:19.348 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nfnetlink_acct.ko 2024-03-08T20:34:19.348 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nfnetlink_log.ko 2024-03-08T20:34:19.348 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nfnetlink_osf.ko 2024-03-08T20:34:19.348 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nfnetlink_queue.ko 2024-03-08T20:34:19.348 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nft_chain_nat.ko 2024-03-08T20:34:19.348 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nft_compat.ko 2024-03-08T20:34:19.348 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nft_ct.ko 2024-03-08T20:34:19.348 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nft_dup_netdev.ko 2024-03-08T20:34:19.349 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nft_fib.ko 2024-03-08T20:34:19.349 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nft_fwd_netdev.ko 2024-03-08T20:34:19.349 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nft_hash.ko 2024-03-08T20:34:19.349 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nft_limit.ko 2024-03-08T20:34:19.349 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nft_log.ko 2024-03-08T20:34:19.349 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nft_masq.ko 2024-03-08T20:34:19.349 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nft_nat.ko 2024-03-08T20:34:19.349 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nft_numgen.ko 2024-03-08T20:34:19.349 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nft_queue.ko 2024-03-08T20:34:19.349 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nft_quota.ko 2024-03-08T20:34:19.349 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nft_redir.ko 2024-03-08T20:34:19.349 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nft_reject.ko 2024-03-08T20:34:19.349 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nft_reject_inet.ko 2024-03-08T20:34:19.350 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nft_socket.ko 2024-03-08T20:34:19.350 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nft_tunnel.ko 2024-03-08T20:34:19.350 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_AUDIT.ko 2024-03-08T20:34:19.350 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_CHECKSUM.ko 2024-03-08T20:34:19.350 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_CLASSIFY.ko 2024-03-08T20:34:19.350 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-03-08T20:34:19.350 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_CT.ko 2024-03-08T20:34:19.350 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_DSCP.ko 2024-03-08T20:34:19.350 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_HL.ko 2024-03-08T20:34:19.350 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_HMARK.ko 2024-03-08T20:34:19.350 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_IDLETIMER.ko 2024-03-08T20:34:19.350 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_LED.ko 2024-03-08T20:34:19.350 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_LOG.ko 2024-03-08T20:34:19.350 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_MASQUERADE.ko 2024-03-08T20:34:19.351 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_NETMAP.ko 2024-03-08T20:34:19.351 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_NFLOG.ko 2024-03-08T20:34:19.351 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_NFQUEUE.ko 2024-03-08T20:34:19.351 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_RATEEST.ko 2024-03-08T20:34:19.351 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_REDIRECT.ko 2024-03-08T20:34:19.351 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_SECMARK.ko 2024-03-08T20:34:19.351 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_TCPMSS.ko 2024-03-08T20:34:19.351 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-03-08T20:34:19.351 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_TEE.ko 2024-03-08T20:34:19.351 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_TPROXY.ko 2024-03-08T20:34:19.351 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_TRACE.ko 2024-03-08T20:34:19.351 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_addrtype.ko 2024-03-08T20:34:19.351 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_bpf.ko 2024-03-08T20:34:19.352 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_cgroup.ko 2024-03-08T20:34:19.352 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_cluster.ko 2024-03-08T20:34:19.352 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_comment.ko 2024-03-08T20:34:19.352 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_connbytes.ko 2024-03-08T20:34:19.352 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_connlabel.ko 2024-03-08T20:34:19.352 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_connlimit.ko 2024-03-08T20:34:19.352 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_connmark.ko 2024-03-08T20:34:19.352 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_conntrack.ko 2024-03-08T20:34:19.352 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_cpu.ko 2024-03-08T20:34:19.352 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_dccp.ko 2024-03-08T20:34:19.352 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_devgroup.ko 2024-03-08T20:34:19.352 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_dscp.ko 2024-03-08T20:34:19.352 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_ecn.ko 2024-03-08T20:34:19.352 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_esp.ko 2024-03-08T20:34:19.353 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_hashlimit.ko 2024-03-08T20:34:19.353 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_helper.ko 2024-03-08T20:34:19.353 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_hl.ko 2024-03-08T20:34:19.353 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_ipcomp.ko 2024-03-08T20:34:19.353 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_iprange.ko 2024-03-08T20:34:19.353 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_ipvs.ko 2024-03-08T20:34:19.353 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_l2tp.ko 2024-03-08T20:34:19.353 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_length.ko 2024-03-08T20:34:19.353 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_limit.ko 2024-03-08T20:34:19.353 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_mac.ko 2024-03-08T20:34:19.353 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_mark.ko 2024-03-08T20:34:19.353 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_multiport.ko 2024-03-08T20:34:19.353 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_nat.ko 2024-03-08T20:34:19.354 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_nfacct.ko 2024-03-08T20:34:19.354 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_osf.ko 2024-03-08T20:34:19.354 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_owner.ko 2024-03-08T20:34:19.354 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_physdev.ko 2024-03-08T20:34:19.354 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_pkttype.ko 2024-03-08T20:34:19.354 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_policy.ko 2024-03-08T20:34:19.354 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_quota.ko 2024-03-08T20:34:19.354 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_rateest.ko 2024-03-08T20:34:19.354 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_realm.ko 2024-03-08T20:34:19.354 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_recent.ko 2024-03-08T20:34:19.354 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_sctp.ko 2024-03-08T20:34:19.354 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_set.ko 2024-03-08T20:34:19.354 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_socket.ko 2024-03-08T20:34:19.355 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_state.ko 2024-03-08T20:34:19.355 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_statistic.ko 2024-03-08T20:34:19.355 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_string.ko 2024-03-08T20:34:19.355 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_tcpmss.ko 2024-03-08T20:34:19.355 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_time.ko 2024-03-08T20:34:19.355 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_u32.ko 2024-03-08T20:34:19.355 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netlink 2024-03-08T20:34:19.355 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netlink/netlink_diag.ko 2024-03-08T20:34:19.355 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/nsh 2024-03-08T20:34:19.355 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/nsh/nsh.ko 2024-03-08T20:34:19.355 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/openvswitch 2024-03-08T20:34:19.355 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/openvswitch/openvswitch.ko 2024-03-08T20:34:19.355 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/openvswitch/vport-geneve.ko 2024-03-08T20:34:19.356 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/openvswitch/vport-gre.ko 2024-03-08T20:34:19.356 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/openvswitch/vport-vxlan.ko 2024-03-08T20:34:19.356 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/packet 2024-03-08T20:34:19.356 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/packet/af_packet_diag.ko 2024-03-08T20:34:19.356 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/rds 2024-03-08T20:34:19.356 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/rds/rds.ko 2024-03-08T20:34:19.356 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/rds/rds_rdma.ko 2024-03-08T20:34:19.356 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/rds/rds_tcp.ko 2024-03-08T20:34:19.356 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched 2024-03-08T20:34:19.356 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/act_csum.ko 2024-03-08T20:34:19.356 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/act_gact.ko 2024-03-08T20:34:19.356 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/act_mirred.ko 2024-03-08T20:34:19.356 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/act_nat.ko 2024-03-08T20:34:19.356 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/act_pedit.ko 2024-03-08T20:34:19.357 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/act_police.ko 2024-03-08T20:34:19.357 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/act_simple.ko 2024-03-08T20:34:19.357 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/act_skbedit.ko 2024-03-08T20:34:19.357 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/act_vlan.ko 2024-03-08T20:34:19.357 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/cls_basic.ko 2024-03-08T20:34:19.357 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/cls_bpf.ko 2024-03-08T20:34:19.357 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/cls_flow.ko 2024-03-08T20:34:19.357 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/cls_fw.ko 2024-03-08T20:34:19.357 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/cls_route.ko 2024-03-08T20:34:19.357 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/cls_u32.ko 2024-03-08T20:34:19.357 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/em_cmp.ko 2024-03-08T20:34:19.357 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/em_ipset.ko 2024-03-08T20:34:19.357 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/em_meta.ko 2024-03-08T20:34:19.357 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/em_nbyte.ko 2024-03-08T20:34:19.358 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/em_text.ko 2024-03-08T20:34:19.358 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/em_u32.ko 2024-03-08T20:34:19.358 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_choke.ko 2024-03-08T20:34:19.358 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_codel.ko 2024-03-08T20:34:19.358 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_drr.ko 2024-03-08T20:34:19.358 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_fq.ko 2024-03-08T20:34:19.358 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_gred.ko 2024-03-08T20:34:19.358 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_hfsc.ko 2024-03-08T20:34:19.358 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_hhf.ko 2024-03-08T20:34:19.358 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_htb.ko 2024-03-08T20:34:19.358 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_ingress.ko 2024-03-08T20:34:19.358 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_mqprio.ko 2024-03-08T20:34:19.358 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_mqprio_lib.ko 2024-03-08T20:34:19.359 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_multiq.ko 2024-03-08T20:34:19.359 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_netem.ko 2024-03-08T20:34:19.359 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_pie.ko 2024-03-08T20:34:19.359 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_plug.ko 2024-03-08T20:34:19.359 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_prio.ko 2024-03-08T20:34:19.359 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_qfq.ko 2024-03-08T20:34:19.359 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_red.ko 2024-03-08T20:34:19.359 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_sfb.ko 2024-03-08T20:34:19.359 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_sfq.ko 2024-03-08T20:34:19.359 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_tbf.ko 2024-03-08T20:34:19.359 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_teql.ko 2024-03-08T20:34:19.359 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sctp 2024-03-08T20:34:19.359 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sctp/sctp.ko 2024-03-08T20:34:19.359 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sctp/sctp_diag.ko 2024-03-08T20:34:19.360 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sunrpc 2024-03-08T20:34:19.360 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sunrpc/auth_gss 2024-03-08T20:34:19.360 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-03-08T20:34:19.360 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-03-08T20:34:19.360 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sunrpc/sunrpc.ko 2024-03-08T20:34:19.360 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sunrpc/xprtrdma 2024-03-08T20:34:19.360 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-03-08T20:34:19.360 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/tipc 2024-03-08T20:34:19.360 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/tipc/diag.ko 2024-03-08T20:34:19.360 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/tipc/tipc.ko 2024-03-08T20:34:19.360 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/unix 2024-03-08T20:34:19.360 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/unix/unix_diag.ko 2024-03-08T20:34:19.360 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/vmw_vsock 2024-03-08T20:34:19.360 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-03-08T20:34:19.361 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-03-08T20:34:19.361 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/vmw_vsock/vsock.ko 2024-03-08T20:34:19.361 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/vmw_vsock/vsock_diag.ko 2024-03-08T20:34:19.361 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/vmw_vsock/vsock_loopback.ko 2024-03-08T20:34:19.361 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/xfrm 2024-03-08T20:34:19.361 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/xfrm/xfrm_ipcomp.ko 2024-03-08T20:34:19.361 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/security 2024-03-08T20:34:19.361 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/security/keys 2024-03-08T20:34:19.361 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/security/keys/encrypted-keys 2024-03-08T20:34:19.361 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-03-08T20:34:19.361 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/security/keys/trusted-keys 2024-03-08T20:34:19.361 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/security/keys/trusted-keys/trusted.ko 2024-03-08T20:34:19.361 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/virt 2024-03-08T20:34:19.362 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/virt/lib 2024-03-08T20:34:19.362 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/virt/lib/irqbypass.ko 2024-03-08T20:34:19.362 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/modules.alias 2024-03-08T20:34:19.362 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/modules.alias.bin 2024-03-08T20:34:19.362 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/modules.builtin 2024-03-08T20:34:19.362 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/modules.builtin.alias.bin 2024-03-08T20:34:19.362 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/modules.builtin.bin 2024-03-08T20:34:19.362 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/modules.builtin.modinfo 2024-03-08T20:34:19.362 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/modules.dep 2024-03-08T20:34:19.362 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/modules.dep.bin 2024-03-08T20:34:19.362 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/modules.devname 2024-03-08T20:34:19.362 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/modules.order 2024-03-08T20:34:19.362 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/modules.softdep 2024-03-08T20:34:19.362 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/modules.symbols 2024-03-08T20:34:19.362 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/modules.symbols.bin 2024-03-08T20:34:19.363 INFO:teuthology.orchestra.run.smithi204.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/vmlinuz 2024-03-08T20:34:19.367 DEBUG:teuthology.task.kernel:get_image_version: 6.8.0-rc7-gdd5c1cd17167 2024-03-08T20:34:19.368 DEBUG:teuthology.orchestra.run.smithi204:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-03-08T20:34:19.390 INFO:teuthology.orchestra.run.smithi204.stdout:postinstall scriptlet (using /bin/sh): 2024-03-08T20:34:19.390 INFO:teuthology.orchestra.run.smithi204.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-03-08T20:34:19.390 INFO:teuthology.orchestra.run.smithi204.stdout: /usr/bin/kernel-install add 6.8.0-rc7-gdd5c1cd17167 /lib/modules/6.8.0-rc7-gdd5c1cd17167/vmlinuz 2024-03-08T20:34:19.390 INFO:teuthology.orchestra.run.smithi204.stdout:fi 2024-03-08T20:34:19.390 INFO:teuthology.orchestra.run.smithi204.stdout:for file in vmlinuz System.map config; do 2024-03-08T20:34:19.391 INFO:teuthology.orchestra.run.smithi204.stdout: if ! cmp --silent "/lib/modules/6.8.0-rc7-gdd5c1cd17167/${file}" "/boot/${file}-6.8.0-rc7-gdd5c1cd17167"; then 2024-03-08T20:34:19.391 INFO:teuthology.orchestra.run.smithi204.stdout: cp "/lib/modules/6.8.0-rc7-gdd5c1cd17167/${file}" "/boot/${file}-6.8.0-rc7-gdd5c1cd17167" 2024-03-08T20:34:19.391 INFO:teuthology.orchestra.run.smithi204.stdout: fi 2024-03-08T20:34:19.391 INFO:teuthology.orchestra.run.smithi204.stdout:done 2024-03-08T20:34:19.391 INFO:teuthology.orchestra.run.smithi204.stdout:preuninstall scriptlet (using /bin/sh): 2024-03-08T20:34:19.391 INFO:teuthology.orchestra.run.smithi204.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2024-03-08T20:34:19.391 INFO:teuthology.orchestra.run.smithi204.stdout:new-kernel-pkg --remove 6.8.0-rc7-gdd5c1cd17167 --rminitrd --initrdfile=/boot/initramfs-6.8.0-rc7-gdd5c1cd17167.img 2024-03-08T20:34:19.391 INFO:teuthology.orchestra.run.smithi204.stdout:elif [ -x /usr/bin/kernel-install ]; then 2024-03-08T20:34:19.391 INFO:teuthology.orchestra.run.smithi204.stdout:kernel-install remove 6.8.0-rc7-gdd5c1cd17167 2024-03-08T20:34:19.391 INFO:teuthology.orchestra.run.smithi204.stdout:fi 2024-03-08T20:34:19.391 INFO:teuthology.orchestra.run.smithi204.stdout:postuninstall scriptlet (using /bin/sh): 2024-03-08T20:34:19.391 INFO:teuthology.orchestra.run.smithi204.stdout:if [ -x /sbin/update-bootloader ]; then 2024-03-08T20:34:19.391 INFO:teuthology.orchestra.run.smithi204.stdout:/sbin/update-bootloader --remove 6.8.0-rc7-gdd5c1cd17167 2024-03-08T20:34:19.391 INFO:teuthology.orchestra.run.smithi204.stdout:fi 2024-03-08T20:34:19.392 DEBUG:teuthology.orchestra.run.smithi204:> sudo rpm -qi grub2-tools 2024-03-08T20:34:19.467 INFO:teuthology.orchestra.run.smithi204.stdout:Name : grub2-tools 2024-03-08T20:34:19.467 INFO:teuthology.orchestra.run.smithi204.stdout:Epoch : 1 2024-03-08T20:34:19.467 INFO:teuthology.orchestra.run.smithi204.stdout:Version : 2.06 2024-03-08T20:34:19.467 INFO:teuthology.orchestra.run.smithi204.stdout:Release : 68.el9 2024-03-08T20:34:19.467 INFO:teuthology.orchestra.run.smithi204.stdout:Architecture: x86_64 2024-03-08T20:34:19.467 INFO:teuthology.orchestra.run.smithi204.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-03-08T20:34:19.468 INFO:teuthology.orchestra.run.smithi204.stdout:Group : Unspecified 2024-03-08T20:34:19.468 INFO:teuthology.orchestra.run.smithi204.stdout:Size : 8264254 2024-03-08T20:34:19.468 INFO:teuthology.orchestra.run.smithi204.stdout:License : GPLv3+ 2024-03-08T20:34:19.468 INFO:teuthology.orchestra.run.smithi204.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-03-08T20:34:19.468 INFO:teuthology.orchestra.run.smithi204.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-03-08T20:34:19.468 INFO:teuthology.orchestra.run.smithi204.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-03-08T20:34:19.468 INFO:teuthology.orchestra.run.smithi204.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-03-08T20:34:19.468 INFO:teuthology.orchestra.run.smithi204.stdout:Packager : builder@centos.org 2024-03-08T20:34:19.468 INFO:teuthology.orchestra.run.smithi204.stdout:Vendor : CentOS 2024-03-08T20:34:19.468 INFO:teuthology.orchestra.run.smithi204.stdout:URL : http://www.gnu.org/software/grub/ 2024-03-08T20:34:19.468 INFO:teuthology.orchestra.run.smithi204.stdout:Summary : Support tools for GRUB. 2024-03-08T20:34:19.468 INFO:teuthology.orchestra.run.smithi204.stdout:Description : 2024-03-08T20:34:19.468 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-03-08T20:34:19.468 INFO:teuthology.orchestra.run.smithi204.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-03-08T20:34:19.468 INFO:teuthology.orchestra.run.smithi204.stdout:customizable bootloader with modular architecture. It supports a rich 2024-03-08T20:34:19.469 INFO:teuthology.orchestra.run.smithi204.stdout:variety of kernel formats, file systems, computer architectures and 2024-03-08T20:34:19.469 INFO:teuthology.orchestra.run.smithi204.stdout:hardware devices. 2024-03-08T20:34:19.469 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-03-08T20:34:19.469 INFO:teuthology.orchestra.run.smithi204.stdout:This subpackage provides tools for support of all platforms. 2024-03-08T20:34:19.470 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-03-08T20:34:19.470 INFO:teuthology.task.kernel:Updating grub on smithi204 to boot 6.8.0-rc7-gdd5c1cd17167 2024-03-08T20:34:19.470 DEBUG:teuthology.orchestra.run.smithi204:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-03-08T20:34:20.061 INFO:teuthology.orchestra.run.smithi204.stderr:Generating grub configuration file ... 2024-03-08T20:34:21.380 INFO:teuthology.orchestra.run.smithi204.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-03-08T20:34:21.420 INFO:teuthology.orchestra.run.smithi204.stderr:done 2024-03-08T20:34:21.422 DEBUG:teuthology.orchestra.run.smithi204:> mktemp 2024-03-08T20:34:21.438 INFO:teuthology.orchestra.run.smithi204.stdout:/tmp/tmp.CAamKrXN2Z 2024-03-08T20:34:21.439 DEBUG:teuthology.orchestra.run.smithi204:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.CAamKrXN2Z 2024-03-08T20:34:21.506 DEBUG:teuthology.orchestra.run.smithi204:> sudo chmod 0666 /tmp/tmp.CAamKrXN2Z 2024-03-08T20:34:21.657 DEBUG:teuthology.orchestra.remote:smithi204:/tmp/tmp.CAamKrXN2Z is 6KB 2024-03-08T20:34:21.705 DEBUG:teuthology.orchestra.run.smithi204:> rm -fr /tmp/tmp.CAamKrXN2Z 2024-03-08T20:34:21.719 DEBUG:teuthology.orchestra.run.smithi204:> sudo /bin/ls /boot/loader/entries || true 2024-03-08T20:34:21.783 INFO:teuthology.orchestra.run.smithi204.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-03-08T20:34:21.783 INFO:teuthology.orchestra.run.smithi204.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-03-08T20:34:21.783 INFO:teuthology.orchestra.run.smithi204.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-03-08T20:34:21.784 INFO:teuthology.orchestra.run.smithi204.stdout:15949e3c88704300882482eddfd7baae-6.8.0-rc7-gdd5c1cd17167.conf 2024-03-08T20:34:21.785 DEBUG:teuthology.orchestra.run.smithi204:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-6.8.0-rc7-gdd5c1cd17167 2024-03-08T20:34:21.951 DEBUG:teuthology.orchestra.run.smithi204:> sudo shutdown -r now 2024-03-08T20:34:25.967 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi107.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-03-08T20:34:25.967 DEBUG:teuthology.orchestra.run.smithi107:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-03-08T20:34:26.043 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167 2024-03-08T20:34:26.043 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/System.map 2024-03-08T20:34:26.043 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/config 2024-03-08T20:34:26.043 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel 2024-03-08T20:34:26.043 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch 2024-03-08T20:34:26.044 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86 2024-03-08T20:34:26.044 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto 2024-03-08T20:34:26.044 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-03-08T20:34:26.044 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-03-08T20:34:26.044 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-03-08T20:34:26.044 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-03-08T20:34:26.044 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-03-08T20:34:26.044 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-03-08T20:34:26.044 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-03-08T20:34:26.044 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/crc32c-intel.ko 2024-03-08T20:34:26.044 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-03-08T20:34:26.044 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-03-08T20:34:26.044 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-03-08T20:34:26.044 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-03-08T20:34:26.045 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/serpent-avx2.ko 2024-03-08T20:34:26.045 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-03-08T20:34:26.045 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-03-08T20:34:26.045 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-03-08T20:34:26.045 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-03-08T20:34:26.045 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-03-08T20:34:26.045 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-03-08T20:34:26.045 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-03-08T20:34:26.045 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/kernel 2024-03-08T20:34:26.045 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/kernel/cpu 2024-03-08T20:34:26.045 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/kernel/cpu/mce 2024-03-08T20:34:26.045 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-03-08T20:34:26.045 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/kvm 2024-03-08T20:34:26.046 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/kvm/kvm-amd.ko 2024-03-08T20:34:26.046 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/kvm/kvm-intel.ko 2024-03-08T20:34:26.046 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/kvm/kvm.ko 2024-03-08T20:34:26.046 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto 2024-03-08T20:34:26.046 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/ansi_cprng.ko 2024-03-08T20:34:26.046 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/anubis.ko 2024-03-08T20:34:26.046 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/arc4.ko 2024-03-08T20:34:26.046 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/async_tx 2024-03-08T20:34:26.046 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/async_tx/async_memcpy.ko 2024-03-08T20:34:26.046 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/async_tx/async_pq.ko 2024-03-08T20:34:26.046 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/async_tx/async_raid6_recov.ko 2024-03-08T20:34:26.046 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/async_tx/async_tx.ko 2024-03-08T20:34:26.047 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/async_tx/async_xor.ko 2024-03-08T20:34:26.047 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/async_tx/raid6test.ko 2024-03-08T20:34:26.047 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/authenc.ko 2024-03-08T20:34:26.047 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/authencesn.ko 2024-03-08T20:34:26.047 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/blake2b_generic.ko 2024-03-08T20:34:26.047 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/blowfish_common.ko 2024-03-08T20:34:26.047 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/blowfish_generic.ko 2024-03-08T20:34:26.047 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/camellia_generic.ko 2024-03-08T20:34:26.047 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/cast5_generic.ko 2024-03-08T20:34:26.047 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/cast6_generic.ko 2024-03-08T20:34:26.047 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/cast_common.ko 2024-03-08T20:34:26.047 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/ccm.ko 2024-03-08T20:34:26.047 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/cmac.ko 2024-03-08T20:34:26.047 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/crc32_generic.ko 2024-03-08T20:34:26.048 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/crypto_engine.ko 2024-03-08T20:34:26.048 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/crypto_user.ko 2024-03-08T20:34:26.048 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/cts.ko 2024-03-08T20:34:26.048 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/des_generic.ko 2024-03-08T20:34:26.048 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/dh_generic.ko 2024-03-08T20:34:26.048 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/echainiv.ko 2024-03-08T20:34:26.048 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/essiv.ko 2024-03-08T20:34:26.048 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/fcrypt.ko 2024-03-08T20:34:26.048 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/khazad.ko 2024-03-08T20:34:26.048 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/lz4.ko 2024-03-08T20:34:26.048 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/lz4hc.ko 2024-03-08T20:34:26.048 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/md4.ko 2024-03-08T20:34:26.048 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/michael_mic.ko 2024-03-08T20:34:26.049 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/pcbc.ko 2024-03-08T20:34:26.049 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/pcrypt.ko 2024-03-08T20:34:26.049 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/rmd160.ko 2024-03-08T20:34:26.049 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/seed.ko 2024-03-08T20:34:26.049 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/serpent_generic.ko 2024-03-08T20:34:26.049 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/tcrypt.ko 2024-03-08T20:34:26.049 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/tea.ko 2024-03-08T20:34:26.049 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/twofish_common.ko 2024-03-08T20:34:26.049 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/twofish_generic.ko 2024-03-08T20:34:26.049 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/vmac.ko 2024-03-08T20:34:26.049 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/wp512.ko 2024-03-08T20:34:26.049 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/xcbc.ko 2024-03-08T20:34:26.049 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/xor.ko 2024-03-08T20:34:26.049 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/xxhash_generic.ko 2024-03-08T20:34:26.049 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers 2024-03-08T20:34:26.050 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/acpi 2024-03-08T20:34:26.050 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/acpi/acpi_ipmi.ko 2024-03-08T20:34:26.050 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/acpi/acpi_pad.ko 2024-03-08T20:34:26.050 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/acpi/custom_method.ko 2024-03-08T20:34:26.050 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/acpi/ec_sys.ko 2024-03-08T20:34:26.050 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/acpi/platform_profile.ko 2024-03-08T20:34:26.050 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/acpi/sbs.ko 2024-03-08T20:34:26.050 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/acpi/sbshc.ko 2024-03-08T20:34:26.050 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/acpi/video.ko 2024-03-08T20:34:26.050 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata 2024-03-08T20:34:26.050 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/acard-ahci.ko 2024-03-08T20:34:26.050 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/ahci_platform.ko 2024-03-08T20:34:26.050 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/ata_generic.ko 2024-03-08T20:34:26.051 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/libahci_platform.ko 2024-03-08T20:34:26.051 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_acpi.ko 2024-03-08T20:34:26.051 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_ali.ko 2024-03-08T20:34:26.051 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_amd.ko 2024-03-08T20:34:26.051 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_artop.ko 2024-03-08T20:34:26.051 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_atiixp.ko 2024-03-08T20:34:26.051 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_atp867x.ko 2024-03-08T20:34:26.051 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_cmd640.ko 2024-03-08T20:34:26.051 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_cmd64x.ko 2024-03-08T20:34:26.051 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_cypress.ko 2024-03-08T20:34:26.051 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_efar.ko 2024-03-08T20:34:26.051 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_hpt366.ko 2024-03-08T20:34:26.051 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_hpt37x.ko 2024-03-08T20:34:26.051 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_hpt3x2n.ko 2024-03-08T20:34:26.052 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_hpt3x3.ko 2024-03-08T20:34:26.052 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_it8213.ko 2024-03-08T20:34:26.052 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_it821x.ko 2024-03-08T20:34:26.052 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_jmicron.ko 2024-03-08T20:34:26.052 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_marvell.ko 2024-03-08T20:34:26.052 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_mpiix.ko 2024-03-08T20:34:26.052 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_netcell.ko 2024-03-08T20:34:26.052 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_ninja32.ko 2024-03-08T20:34:26.052 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_ns87410.ko 2024-03-08T20:34:26.052 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_ns87415.ko 2024-03-08T20:34:26.052 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_oldpiix.ko 2024-03-08T20:34:26.052 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_opti.ko 2024-03-08T20:34:26.052 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_optidma.ko 2024-03-08T20:34:26.052 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_pcmcia.ko 2024-03-08T20:34:26.053 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_pdc2027x.ko 2024-03-08T20:34:26.053 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-03-08T20:34:26.053 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_piccolo.ko 2024-03-08T20:34:26.053 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_rdc.ko 2024-03-08T20:34:26.053 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_sch.ko 2024-03-08T20:34:26.053 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_serverworks.ko 2024-03-08T20:34:26.053 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_sil680.ko 2024-03-08T20:34:26.053 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_sis.ko 2024-03-08T20:34:26.053 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_sl82c105.ko 2024-03-08T20:34:26.053 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_triflex.ko 2024-03-08T20:34:26.053 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_via.ko 2024-03-08T20:34:26.053 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pdc_adma.ko 2024-03-08T20:34:26.053 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/sata_inic162x.ko 2024-03-08T20:34:26.053 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/sata_mv.ko 2024-03-08T20:34:26.054 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/sata_nv.ko 2024-03-08T20:34:26.054 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/sata_promise.ko 2024-03-08T20:34:26.054 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/sata_qstor.ko 2024-03-08T20:34:26.054 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/sata_sil.ko 2024-03-08T20:34:26.054 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/sata_sil24.ko 2024-03-08T20:34:26.054 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/sata_sis.ko 2024-03-08T20:34:26.054 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/sata_svw.ko 2024-03-08T20:34:26.054 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/sata_sx4.ko 2024-03-08T20:34:26.054 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/sata_uli.ko 2024-03-08T20:34:26.054 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/sata_via.ko 2024-03-08T20:34:26.054 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/sata_vsc.ko 2024-03-08T20:34:26.054 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/auxdisplay 2024-03-08T20:34:26.054 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/auxdisplay/ks0108.ko 2024-03-08T20:34:26.055 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/base 2024-03-08T20:34:26.055 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/base/regmap 2024-03-08T20:34:26.055 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/base/regmap/regmap-i2c.ko 2024-03-08T20:34:26.055 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/base/regmap/regmap-mmio.ko 2024-03-08T20:34:26.055 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/bcma 2024-03-08T20:34:26.055 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/bcma/bcma.ko 2024-03-08T20:34:26.055 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block 2024-03-08T20:34:26.055 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/aoe 2024-03-08T20:34:26.055 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/aoe/aoe.ko 2024-03-08T20:34:26.055 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/brd.ko 2024-03-08T20:34:26.055 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/drbd 2024-03-08T20:34:26.055 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/drbd/drbd.ko 2024-03-08T20:34:26.055 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/floppy.ko 2024-03-08T20:34:26.055 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/loop.ko 2024-03-08T20:34:26.056 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/mtip32xx 2024-03-08T20:34:26.056 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-03-08T20:34:26.056 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/nbd.ko 2024-03-08T20:34:26.056 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/null_blk 2024-03-08T20:34:26.056 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/null_blk/null_blk.ko 2024-03-08T20:34:26.056 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/pktcdvd.ko 2024-03-08T20:34:26.056 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/rbd.ko 2024-03-08T20:34:26.056 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/virtio_blk.ko 2024-03-08T20:34:26.056 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/xen-blkback 2024-03-08T20:34:26.056 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-03-08T20:34:26.056 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/xen-blkfront.ko 2024-03-08T20:34:26.056 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/zram 2024-03-08T20:34:26.056 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/zram/zram.ko 2024-03-08T20:34:26.056 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char 2024-03-08T20:34:26.056 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/hangcheck-timer.ko 2024-03-08T20:34:26.057 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/hw_random 2024-03-08T20:34:26.057 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/hw_random/amd-rng.ko 2024-03-08T20:34:26.057 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/hw_random/intel-rng.ko 2024-03-08T20:34:26.057 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-03-08T20:34:26.057 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/hw_random/via-rng.ko 2024-03-08T20:34:26.057 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/hw_random/virtio-rng.ko 2024-03-08T20:34:26.057 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/ipmi 2024-03-08T20:34:26.057 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-03-08T20:34:26.057 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-03-08T20:34:26.057 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-03-08T20:34:26.057 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/ipmi/ipmi_si.ko 2024-03-08T20:34:26.057 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-03-08T20:34:26.057 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-03-08T20:34:26.058 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/lp.ko 2024-03-08T20:34:26.058 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/mwave 2024-03-08T20:34:26.058 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/mwave/mwave.ko 2024-03-08T20:34:26.058 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/ppdev.ko 2024-03-08T20:34:26.058 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/tlclk.ko 2024-03-08T20:34:26.058 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/tpm 2024-03-08T20:34:26.058 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/tpm/tpm.ko 2024-03-08T20:34:26.058 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/tpm/tpm_atmel.ko 2024-03-08T20:34:26.058 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/tpm/tpm_infineon.ko 2024-03-08T20:34:26.058 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/tpm/tpm_nsc.ko 2024-03-08T20:34:26.058 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/tpm/tpm_tis.ko 2024-03-08T20:34:26.058 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-03-08T20:34:26.058 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/uv_mmtimer.ko 2024-03-08T20:34:26.058 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/virtio_console.ko 2024-03-08T20:34:26.059 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/cpufreq 2024-03-08T20:34:26.059 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-03-08T20:34:26.059 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-03-08T20:34:26.059 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/cpufreq/p4-clockmod.ko 2024-03-08T20:34:26.059 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-03-08T20:34:26.059 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/cpufreq/powernow-k8.ko 2024-03-08T20:34:26.059 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/cpufreq/speedstep-lib.ko 2024-03-08T20:34:26.059 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/crypto 2024-03-08T20:34:26.059 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/crypto/ccp 2024-03-08T20:34:26.059 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-03-08T20:34:26.059 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/crypto/ccp/ccp.ko 2024-03-08T20:34:26.059 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/crypto/intel 2024-03-08T20:34:26.059 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/crypto/intel/qat 2024-03-08T20:34:26.059 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/crypto/intel/qat/qat_common 2024-03-08T20:34:26.060 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-03-08T20:34:26.060 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-03-08T20:34:26.060 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-03-08T20:34:26.060 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/crypto/padlock-aes.ko 2024-03-08T20:34:26.060 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/crypto/padlock-sha.ko 2024-03-08T20:34:26.060 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/crypto/virtio 2024-03-08T20:34:26.060 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-03-08T20:34:26.060 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/dca 2024-03-08T20:34:26.060 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/dca/dca.ko 2024-03-08T20:34:26.060 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/devfreq 2024-03-08T20:34:26.060 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-03-08T20:34:26.060 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/dma 2024-03-08T20:34:26.060 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/dma/dw 2024-03-08T20:34:26.060 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/dma/dw/dw_dmac.ko 2024-03-08T20:34:26.061 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/dma/ioat 2024-03-08T20:34:26.061 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/dma/ioat/ioatdma.ko 2024-03-08T20:34:26.061 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/edac 2024-03-08T20:34:26.061 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/edac/amd64_edac.ko 2024-03-08T20:34:26.061 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/edac/e752x_edac.ko 2024-03-08T20:34:26.061 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/edac/edac_mce_amd.ko 2024-03-08T20:34:26.061 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/edac/i3000_edac.ko 2024-03-08T20:34:26.061 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/edac/i3200_edac.ko 2024-03-08T20:34:26.061 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/edac/i5100_edac.ko 2024-03-08T20:34:26.061 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/edac/i5400_edac.ko 2024-03-08T20:34:26.061 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/edac/i7300_edac.ko 2024-03-08T20:34:26.061 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/edac/i7core_edac.ko 2024-03-08T20:34:26.061 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/edac/i82975x_edac.ko 2024-03-08T20:34:26.061 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/edac/ie31200_edac.ko 2024-03-08T20:34:26.061 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/edac/sb_edac.ko 2024-03-08T20:34:26.062 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/edac/x38_edac.ko 2024-03-08T20:34:26.062 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/firmware 2024-03-08T20:34:26.062 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/firmware/edd.ko 2024-03-08T20:34:26.062 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/firmware/iscsi_ibft.ko 2024-03-08T20:34:26.062 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/gpio 2024-03-08T20:34:26.062 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/gpio/gpio-ich.ko 2024-03-08T20:34:26.062 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/gpio/gpio-viperboard.ko 2024-03-08T20:34:26.062 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid 2024-03-08T20:34:26.062 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-appleir.ko 2024-03-08T20:34:26.062 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-aureal.ko 2024-03-08T20:34:26.062 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-axff.ko 2024-03-08T20:34:26.062 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-dr.ko 2024-03-08T20:34:26.062 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-elecom.ko 2024-03-08T20:34:26.063 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-elo.ko 2024-03-08T20:34:26.063 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-emsff.ko 2024-03-08T20:34:26.063 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-gaff.ko 2024-03-08T20:34:26.063 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-gt683r.ko 2024-03-08T20:34:26.063 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-gyration.ko 2024-03-08T20:34:26.063 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-holtek-kbd.ko 2024-03-08T20:34:26.063 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-holtek-mouse.ko 2024-03-08T20:34:26.063 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-holtekff.ko 2024-03-08T20:34:26.063 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-hyperv.ko 2024-03-08T20:34:26.063 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-icade.ko 2024-03-08T20:34:26.063 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-keytouch.ko 2024-03-08T20:34:26.063 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-kye.ko 2024-03-08T20:34:26.063 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-lcpower.ko 2024-03-08T20:34:26.063 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-led.ko 2024-03-08T20:34:26.063 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-lenovo.ko 2024-03-08T20:34:26.064 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-logitech-dj.ko 2024-03-08T20:34:26.064 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-03-08T20:34:26.064 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-multitouch.ko 2024-03-08T20:34:26.064 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-ortek.ko 2024-03-08T20:34:26.064 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-penmount.ko 2024-03-08T20:34:26.064 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-petalynx.ko 2024-03-08T20:34:26.064 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-picolcd.ko 2024-03-08T20:34:26.064 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-pl.ko 2024-03-08T20:34:26.064 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-plantronics.ko 2024-03-08T20:34:26.064 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-primax.ko 2024-03-08T20:34:26.064 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-rmi.ko 2024-03-08T20:34:26.064 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-roccat-arvo.ko 2024-03-08T20:34:26.064 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-roccat-common.ko 2024-03-08T20:34:26.064 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-roccat-isku.ko 2024-03-08T20:34:26.065 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-roccat-kone.ko 2024-03-08T20:34:26.065 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-03-08T20:34:26.065 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-roccat-konepure.ko 2024-03-08T20:34:26.065 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-03-08T20:34:26.065 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-roccat-lua.ko 2024-03-08T20:34:26.065 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-roccat-pyra.ko 2024-03-08T20:34:26.065 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-roccat-ryos.ko 2024-03-08T20:34:26.065 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-roccat-savu.ko 2024-03-08T20:34:26.065 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-roccat.ko 2024-03-08T20:34:26.065 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-saitek.ko 2024-03-08T20:34:26.065 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-samsung.ko 2024-03-08T20:34:26.065 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-sensor-hub.ko 2024-03-08T20:34:26.065 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-sjoy.ko 2024-03-08T20:34:26.065 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-sony.ko 2024-03-08T20:34:26.065 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-speedlink.ko 2024-03-08T20:34:26.066 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-steelseries.ko 2024-03-08T20:34:26.066 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-sunplus.ko 2024-03-08T20:34:26.066 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-thrustmaster.ko 2024-03-08T20:34:26.066 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-tivo.ko 2024-03-08T20:34:26.066 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-tmff.ko 2024-03-08T20:34:26.066 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-topseed.ko 2024-03-08T20:34:26.066 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-twinhan.ko 2024-03-08T20:34:26.066 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-uclogic.ko 2024-03-08T20:34:26.066 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-waltop.ko 2024-03-08T20:34:26.066 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-wiimote.ko 2024-03-08T20:34:26.066 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-xinmo.ko 2024-03-08T20:34:26.066 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-zpff.ko 2024-03-08T20:34:26.066 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-zydacron.ko 2024-03-08T20:34:26.066 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/uhid.ko 2024-03-08T20:34:26.067 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/wacom.ko 2024-03-08T20:34:26.067 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hv 2024-03-08T20:34:26.067 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hv/hv_balloon.ko 2024-03-08T20:34:26.067 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hv/hv_utils.ko 2024-03-08T20:34:26.067 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hv/hv_vmbus.ko 2024-03-08T20:34:26.067 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon 2024-03-08T20:34:26.067 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/abituguru.ko 2024-03-08T20:34:26.067 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/abituguru3.ko 2024-03-08T20:34:26.067 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/acpi_power_meter.ko 2024-03-08T20:34:26.067 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/ad7414.ko 2024-03-08T20:34:26.067 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/ad7418.ko 2024-03-08T20:34:26.067 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/adc128d818.ko 2024-03-08T20:34:26.067 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/adm1025.ko 2024-03-08T20:34:26.067 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/adm1026.ko 2024-03-08T20:34:26.067 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/adm1029.ko 2024-03-08T20:34:26.068 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/adm1031.ko 2024-03-08T20:34:26.068 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/adm9240.ko 2024-03-08T20:34:26.068 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/ads7828.ko 2024-03-08T20:34:26.068 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/adt7410.ko 2024-03-08T20:34:26.068 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/adt7411.ko 2024-03-08T20:34:26.068 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/adt7462.ko 2024-03-08T20:34:26.068 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/adt7470.ko 2024-03-08T20:34:26.068 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/adt7475.ko 2024-03-08T20:34:26.068 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/adt7x10.ko 2024-03-08T20:34:26.068 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/amc6821.ko 2024-03-08T20:34:26.068 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/applesmc.ko 2024-03-08T20:34:26.068 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/asb100.ko 2024-03-08T20:34:26.068 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/asc7621.ko 2024-03-08T20:34:26.068 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/asus_atk0110.ko 2024-03-08T20:34:26.069 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/atxp1.ko 2024-03-08T20:34:26.069 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/coretemp.ko 2024-03-08T20:34:26.069 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-03-08T20:34:26.069 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/dme1737.ko 2024-03-08T20:34:26.069 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/ds1621.ko 2024-03-08T20:34:26.069 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/ds620.ko 2024-03-08T20:34:26.069 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/emc1403.ko 2024-03-08T20:34:26.069 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/emc6w201.ko 2024-03-08T20:34:26.069 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/f71805f.ko 2024-03-08T20:34:26.069 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/f71882fg.ko 2024-03-08T20:34:26.069 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/f75375s.ko 2024-03-08T20:34:26.069 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/fam15h_power.ko 2024-03-08T20:34:26.069 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/fschmd.ko 2024-03-08T20:34:26.069 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/g760a.ko 2024-03-08T20:34:26.070 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/g762.ko 2024-03-08T20:34:26.070 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/gl518sm.ko 2024-03-08T20:34:26.070 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/gl520sm.ko 2024-03-08T20:34:26.070 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/hwmon-vid.ko 2024-03-08T20:34:26.070 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/i5500_temp.ko 2024-03-08T20:34:26.070 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/i5k_amb.ko 2024-03-08T20:34:26.070 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/ibmaem.ko 2024-03-08T20:34:26.070 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/ibmpex.ko 2024-03-08T20:34:26.070 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/ina209.ko 2024-03-08T20:34:26.070 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/ina2xx.ko 2024-03-08T20:34:26.070 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/it87.ko 2024-03-08T20:34:26.070 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/k10temp.ko 2024-03-08T20:34:26.070 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/k8temp.ko 2024-03-08T20:34:26.071 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/lineage-pem.ko 2024-03-08T20:34:26.071 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/lm63.ko 2024-03-08T20:34:26.071 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/lm73.ko 2024-03-08T20:34:26.071 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/lm75.ko 2024-03-08T20:34:26.071 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/lm77.ko 2024-03-08T20:34:26.071 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/lm78.ko 2024-03-08T20:34:26.071 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/lm80.ko 2024-03-08T20:34:26.071 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/lm83.ko 2024-03-08T20:34:26.071 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/lm85.ko 2024-03-08T20:34:26.071 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/lm87.ko 2024-03-08T20:34:26.071 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/lm90.ko 2024-03-08T20:34:26.071 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/lm92.ko 2024-03-08T20:34:26.071 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/lm93.ko 2024-03-08T20:34:26.071 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/lm95234.ko 2024-03-08T20:34:26.072 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/lm95241.ko 2024-03-08T20:34:26.072 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/lm95245.ko 2024-03-08T20:34:26.072 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/ltc2945.ko 2024-03-08T20:34:26.072 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/ltc4151.ko 2024-03-08T20:34:26.072 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/ltc4215.ko 2024-03-08T20:34:26.072 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/ltc4222.ko 2024-03-08T20:34:26.072 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/ltc4245.ko 2024-03-08T20:34:26.072 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/ltc4260.ko 2024-03-08T20:34:26.072 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/ltc4261.ko 2024-03-08T20:34:26.072 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/max16065.ko 2024-03-08T20:34:26.072 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/max1619.ko 2024-03-08T20:34:26.072 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/max1668.ko 2024-03-08T20:34:26.072 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/max197.ko 2024-03-08T20:34:26.072 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/max6639.ko 2024-03-08T20:34:26.073 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/max6650.ko 2024-03-08T20:34:26.073 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/max6697.ko 2024-03-08T20:34:26.073 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/mcp3021.ko 2024-03-08T20:34:26.073 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/nct6683.ko 2024-03-08T20:34:26.073 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/nct6775-core.ko 2024-03-08T20:34:26.073 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/nct6775.ko 2024-03-08T20:34:26.073 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/nct7802.ko 2024-03-08T20:34:26.073 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/pc87360.ko 2024-03-08T20:34:26.073 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/pc87427.ko 2024-03-08T20:34:26.073 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/pcf8591.ko 2024-03-08T20:34:26.073 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/pmbus 2024-03-08T20:34:26.073 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-03-08T20:34:26.073 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-03-08T20:34:26.074 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-03-08T20:34:26.074 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/pmbus/max16064.ko 2024-03-08T20:34:26.074 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/pmbus/max34440.ko 2024-03-08T20:34:26.074 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/pmbus/max8688.ko 2024-03-08T20:34:26.074 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-03-08T20:34:26.074 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-03-08T20:34:26.074 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-03-08T20:34:26.074 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-03-08T20:34:26.074 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-03-08T20:34:26.074 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-03-08T20:34:26.074 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/powr1220.ko 2024-03-08T20:34:26.074 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/sch5627.ko 2024-03-08T20:34:26.074 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/sch5636.ko 2024-03-08T20:34:26.075 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/sch56xx-common.ko 2024-03-08T20:34:26.075 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/sht15.ko 2024-03-08T20:34:26.075 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/sht21.ko 2024-03-08T20:34:26.075 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/shtc1.ko 2024-03-08T20:34:26.075 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/sis5595.ko 2024-03-08T20:34:26.075 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/smsc47b397.ko 2024-03-08T20:34:26.075 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/smsc47m1.ko 2024-03-08T20:34:26.075 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/smsc47m192.ko 2024-03-08T20:34:26.075 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/thmc50.ko 2024-03-08T20:34:26.075 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/tmp102.ko 2024-03-08T20:34:26.075 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/tmp103.ko 2024-03-08T20:34:26.075 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/tmp401.ko 2024-03-08T20:34:26.075 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/tmp421.ko 2024-03-08T20:34:26.075 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/via-cputemp.ko 2024-03-08T20:34:26.076 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/via686a.ko 2024-03-08T20:34:26.076 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/vt1211.ko 2024-03-08T20:34:26.076 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/vt8231.ko 2024-03-08T20:34:26.076 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/w83627ehf.ko 2024-03-08T20:34:26.076 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/w83627hf.ko 2024-03-08T20:34:26.076 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/w83781d.ko 2024-03-08T20:34:26.076 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/w83791d.ko 2024-03-08T20:34:26.076 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/w83792d.ko 2024-03-08T20:34:26.076 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/w83793.ko 2024-03-08T20:34:26.076 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/w83795.ko 2024-03-08T20:34:26.076 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/w83l785ts.ko 2024-03-08T20:34:26.076 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/w83l786ng.ko 2024-03-08T20:34:26.076 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c 2024-03-08T20:34:26.076 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/algos 2024-03-08T20:34:26.077 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-03-08T20:34:26.077 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-03-08T20:34:26.077 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses 2024-03-08T20:34:26.077 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-03-08T20:34:26.077 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-03-08T20:34:26.077 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-03-08T20:34:26.077 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-03-08T20:34:26.077 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-03-08T20:34:26.077 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-03-08T20:34:26.077 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-03-08T20:34:26.077 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-03-08T20:34:26.077 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-i801.ko 2024-03-08T20:34:26.077 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-isch.ko 2024-03-08T20:34:26.077 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-03-08T20:34:26.078 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-03-08T20:34:26.078 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-03-08T20:34:26.078 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-parport.ko 2024-03-08T20:34:26.078 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-03-08T20:34:26.078 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-03-08T20:34:26.078 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-03-08T20:34:26.078 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-03-08T20:34:26.078 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-03-08T20:34:26.078 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-03-08T20:34:26.078 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-via.ko 2024-03-08T20:34:26.078 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-03-08T20:34:26.078 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-03-08T20:34:26.078 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/i2c-dev.ko 2024-03-08T20:34:26.079 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/i2c-mux.ko 2024-03-08T20:34:26.079 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-03-08T20:34:26.079 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/i2c-smbus.ko 2024-03-08T20:34:26.079 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/i2c-stub.ko 2024-03-08T20:34:26.079 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband 2024-03-08T20:34:26.079 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/core 2024-03-08T20:34:26.079 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/core/ib_cm.ko 2024-03-08T20:34:26.079 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/core/ib_core.ko 2024-03-08T20:34:26.079 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/core/ib_umad.ko 2024-03-08T20:34:26.079 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-03-08T20:34:26.079 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/core/iw_cm.ko 2024-03-08T20:34:26.079 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/core/rdma_cm.ko 2024-03-08T20:34:26.079 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-03-08T20:34:26.079 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/hw 2024-03-08T20:34:26.079 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/hw/cxgb4 2024-03-08T20:34:26.080 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-03-08T20:34:26.080 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/hw/mlx4 2024-03-08T20:34:26.080 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-03-08T20:34:26.080 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/hw/mlx5 2024-03-08T20:34:26.080 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-03-08T20:34:26.080 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/hw/mthca 2024-03-08T20:34:26.080 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-03-08T20:34:26.080 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/ulp 2024-03-08T20:34:26.080 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/ulp/ipoib 2024-03-08T20:34:26.080 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-03-08T20:34:26.080 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/ulp/iser 2024-03-08T20:34:26.080 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-03-08T20:34:26.080 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/ulp/isert 2024-03-08T20:34:26.080 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-03-08T20:34:26.081 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/ulp/srp 2024-03-08T20:34:26.081 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-03-08T20:34:26.081 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/ulp/srpt 2024-03-08T20:34:26.081 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-03-08T20:34:26.081 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input 2024-03-08T20:34:26.081 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/gameport 2024-03-08T20:34:26.081 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-03-08T20:34:26.081 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/gameport/fm801-gp.ko 2024-03-08T20:34:26.081 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/gameport/gameport.ko 2024-03-08T20:34:26.081 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/joydev.ko 2024-03-08T20:34:26.081 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/keyboard 2024-03-08T20:34:26.081 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/keyboard/gpio_keys.ko 2024-03-08T20:34:26.081 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/mouse 2024-03-08T20:34:26.081 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/mouse/appletouch.ko 2024-03-08T20:34:26.082 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/mouse/bcm5974.ko 2024-03-08T20:34:26.082 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/mouse/cyapatp.ko 2024-03-08T20:34:26.082 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/mouse/elan_i2c.ko 2024-03-08T20:34:26.082 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/mouse/sermouse.ko 2024-03-08T20:34:26.082 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-03-08T20:34:26.082 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/mouse/synaptics_usb.ko 2024-03-08T20:34:26.082 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/mouse/vsxxxaa.ko 2024-03-08T20:34:26.082 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/rmi4 2024-03-08T20:34:26.082 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/rmi4/rmi_core.ko 2024-03-08T20:34:26.082 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/serio 2024-03-08T20:34:26.082 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/serio/altera_ps2.ko 2024-03-08T20:34:26.082 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/serio/arc_ps2.ko 2024-03-08T20:34:26.082 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-03-08T20:34:26.082 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/serio/serio_raw.ko 2024-03-08T20:34:26.082 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/sparse-keymap.ko 2024-03-08T20:34:26.083 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/leds 2024-03-08T20:34:26.083 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/leds/leds-blinkm.ko 2024-03-08T20:34:26.083 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/leds/leds-lm3530.ko 2024-03-08T20:34:26.083 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/leds/leds-lp3944.ko 2024-03-08T20:34:26.083 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/leds/leds-ss4200.ko 2024-03-08T20:34:26.083 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/leds/trigger 2024-03-08T20:34:26.083 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-03-08T20:34:26.083 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-03-08T20:34:26.083 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-03-08T20:34:26.083 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-03-08T20:34:26.083 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-03-08T20:34:26.083 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-03-08T20:34:26.083 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-03-08T20:34:26.083 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-03-08T20:34:26.084 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-03-08T20:34:26.084 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md 2024-03-08T20:34:26.084 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/bcache 2024-03-08T20:34:26.084 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/bcache/bcache.ko 2024-03-08T20:34:26.084 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/dm-bio-prison.ko 2024-03-08T20:34:26.084 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/dm-cache-smq.ko 2024-03-08T20:34:26.084 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/dm-cache.ko 2024-03-08T20:34:26.084 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/dm-crypt.ko 2024-03-08T20:34:26.084 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/dm-delay.ko 2024-03-08T20:34:26.084 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/dm-flakey.ko 2024-03-08T20:34:26.084 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/dm-log-userspace.ko 2024-03-08T20:34:26.084 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/dm-multipath.ko 2024-03-08T20:34:26.084 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/dm-queue-length.ko 2024-03-08T20:34:26.084 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/dm-raid.ko 2024-03-08T20:34:26.084 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/dm-round-robin.ko 2024-03-08T20:34:26.085 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/dm-service-time.ko 2024-03-08T20:34:26.085 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/dm-switch.ko 2024-03-08T20:34:26.085 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/dm-thin-pool.ko 2024-03-08T20:34:26.085 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/dm-verity.ko 2024-03-08T20:34:26.085 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/persistent-data 2024-03-08T20:34:26.085 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-03-08T20:34:26.085 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/raid0.ko 2024-03-08T20:34:26.085 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/raid1.ko 2024-03-08T20:34:26.085 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/raid10.ko 2024-03-08T20:34:26.085 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/raid456.ko 2024-03-08T20:34:26.085 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/message 2024-03-08T20:34:26.085 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/message/fusion 2024-03-08T20:34:26.085 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/message/fusion/mptbase.ko 2024-03-08T20:34:26.085 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/message/fusion/mptctl.ko 2024-03-08T20:34:26.086 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/message/fusion/mptfc.ko 2024-03-08T20:34:26.086 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/message/fusion/mptlan.ko 2024-03-08T20:34:26.086 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/message/fusion/mptsas.ko 2024-03-08T20:34:26.086 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/message/fusion/mptscsih.ko 2024-03-08T20:34:26.086 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/message/fusion/mptspi.ko 2024-03-08T20:34:26.086 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/mfd 2024-03-08T20:34:26.086 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/mfd/lpc_ich.ko 2024-03-08T20:34:26.086 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/mfd/lpc_sch.ko 2024-03-08T20:34:26.086 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/mfd/mfd-core.ko 2024-03-08T20:34:26.086 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/mfd/sm501.ko 2024-03-08T20:34:26.086 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/mfd/viperboard.ko 2024-03-08T20:34:26.086 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/mfd/vx855.ko 2024-03-08T20:34:26.086 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/mfd/wl1273-core.ko 2024-03-08T20:34:26.087 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc 2024-03-08T20:34:26.087 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/altera-stapl 2024-03-08T20:34:26.087 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-03-08T20:34:26.087 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/apds9802als.ko 2024-03-08T20:34:26.087 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/apds990x.ko 2024-03-08T20:34:26.087 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/bh1770glc.ko 2024-03-08T20:34:26.087 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/cb710 2024-03-08T20:34:26.087 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/cb710/cb710.ko 2024-03-08T20:34:26.087 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/eeprom 2024-03-08T20:34:26.087 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/eeprom/at24.ko 2024-03-08T20:34:26.087 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-03-08T20:34:26.087 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/eeprom/max6875.ko 2024-03-08T20:34:26.087 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/enclosure.ko 2024-03-08T20:34:26.087 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/hpilo.ko 2024-03-08T20:34:26.088 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/isl29003.ko 2024-03-08T20:34:26.088 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/isl29020.ko 2024-03-08T20:34:26.088 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/lis3lv02d 2024-03-08T20:34:26.088 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-03-08T20:34:26.088 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-03-08T20:34:26.088 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/mei 2024-03-08T20:34:26.088 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/mei/mei-me.ko 2024-03-08T20:34:26.088 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/mei/mei-txe.ko 2024-03-08T20:34:26.088 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/mei/mei.ko 2024-03-08T20:34:26.088 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/sgi-gru 2024-03-08T20:34:26.088 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/sgi-gru/gru.ko 2024-03-08T20:34:26.088 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/sgi-xp 2024-03-08T20:34:26.088 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/sgi-xp/xp.ko 2024-03-08T20:34:26.088 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/sgi-xp/xpc.ko 2024-03-08T20:34:26.089 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-03-08T20:34:26.089 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/tifm_7xx1.ko 2024-03-08T20:34:26.089 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/tifm_core.ko 2024-03-08T20:34:26.089 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/tsl2550.ko 2024-03-08T20:34:26.089 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/vmw_balloon.ko 2024-03-08T20:34:26.089 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/vmw_vmci 2024-03-08T20:34:26.089 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-03-08T20:34:26.089 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net 2024-03-08T20:34:26.089 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/bonding 2024-03-08T20:34:26.089 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/bonding/bonding.ko 2024-03-08T20:34:26.089 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/dsa 2024-03-08T20:34:26.089 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/dsa/b53 2024-03-08T20:34:26.089 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/dsa/b53/b53_common.ko 2024-03-08T20:34:26.089 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/dsa/bcm-sf2.ko 2024-03-08T20:34:26.090 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/dsa/mv88e6060.ko 2024-03-08T20:34:26.090 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/dsa/mv88e6xxx 2024-03-08T20:34:26.090 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-03-08T20:34:26.090 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/dummy.ko 2024-03-08T20:34:26.090 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/eql.ko 2024-03-08T20:34:26.090 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet 2024-03-08T20:34:26.090 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/3com 2024-03-08T20:34:26.090 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-03-08T20:34:26.090 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-03-08T20:34:26.090 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-03-08T20:34:26.090 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-03-08T20:34:26.090 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/8390 2024-03-08T20:34:26.090 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/8390/8390.ko 2024-03-08T20:34:26.091 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-03-08T20:34:26.091 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-03-08T20:34:26.091 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-03-08T20:34:26.091 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/adaptec 2024-03-08T20:34:26.091 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-03-08T20:34:26.091 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/agere 2024-03-08T20:34:26.091 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/agere/et131x.ko 2024-03-08T20:34:26.091 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/alteon 2024-03-08T20:34:26.091 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-03-08T20:34:26.091 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/altera 2024-03-08T20:34:26.091 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-03-08T20:34:26.091 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/amd 2024-03-08T20:34:26.091 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-03-08T20:34:26.091 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-03-08T20:34:26.092 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-03-08T20:34:26.092 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/atheros 2024-03-08T20:34:26.092 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/atheros/alx 2024-03-08T20:34:26.092 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-03-08T20:34:26.092 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/atheros/atl1c 2024-03-08T20:34:26.092 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-03-08T20:34:26.092 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/atheros/atl1e 2024-03-08T20:34:26.092 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-03-08T20:34:26.092 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/atheros/atlx 2024-03-08T20:34:26.092 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-03-08T20:34:26.092 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-03-08T20:34:26.092 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/broadcom 2024-03-08T20:34:26.092 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-03-08T20:34:26.092 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-03-08T20:34:26.093 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-03-08T20:34:26.093 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-03-08T20:34:26.093 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-03-08T20:34:26.093 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/broadcom/genet 2024-03-08T20:34:26.093 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-03-08T20:34:26.093 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-03-08T20:34:26.093 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/brocade 2024-03-08T20:34:26.093 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/brocade/bna 2024-03-08T20:34:26.093 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-03-08T20:34:26.093 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/chelsio 2024-03-08T20:34:26.093 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/chelsio/cxgb 2024-03-08T20:34:26.093 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-03-08T20:34:26.093 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-03-08T20:34:26.094 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-03-08T20:34:26.094 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-03-08T20:34:26.094 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-03-08T20:34:26.094 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-03-08T20:34:26.094 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-03-08T20:34:26.094 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-03-08T20:34:26.094 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-03-08T20:34:26.094 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/cisco 2024-03-08T20:34:26.094 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/cisco/enic 2024-03-08T20:34:26.094 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-03-08T20:34:26.094 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/dec 2024-03-08T20:34:26.094 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/dec/tulip 2024-03-08T20:34:26.094 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-03-08T20:34:26.094 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-03-08T20:34:26.095 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-03-08T20:34:26.095 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-03-08T20:34:26.095 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-03-08T20:34:26.095 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-03-08T20:34:26.095 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/dlink 2024-03-08T20:34:26.095 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-03-08T20:34:26.095 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-03-08T20:34:26.095 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/dnet.ko 2024-03-08T20:34:26.095 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/emulex 2024-03-08T20:34:26.095 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/emulex/benet 2024-03-08T20:34:26.095 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-03-08T20:34:26.095 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/ethoc.ko 2024-03-08T20:34:26.095 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/fealnx.ko 2024-03-08T20:34:26.095 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel 2024-03-08T20:34:26.095 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel/e100.ko 2024-03-08T20:34:26.096 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel/e1000 2024-03-08T20:34:26.096 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-03-08T20:34:26.096 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel/e1000e 2024-03-08T20:34:26.096 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-03-08T20:34:26.096 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel/fm10k 2024-03-08T20:34:26.096 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-03-08T20:34:26.096 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel/i40e 2024-03-08T20:34:26.096 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-03-08T20:34:26.096 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel/iavf 2024-03-08T20:34:26.096 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-03-08T20:34:26.096 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel/igb 2024-03-08T20:34:26.096 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-03-08T20:34:26.096 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel/igbvf 2024-03-08T20:34:26.096 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-03-08T20:34:26.097 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel/ixgbe 2024-03-08T20:34:26.097 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-03-08T20:34:26.097 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel/ixgbevf 2024-03-08T20:34:26.097 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-03-08T20:34:26.097 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/jme.ko 2024-03-08T20:34:26.097 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/marvell 2024-03-08T20:34:26.097 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-03-08T20:34:26.097 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/marvell/skge.ko 2024-03-08T20:34:26.097 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-03-08T20:34:26.097 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/mellanox 2024-03-08T20:34:26.097 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/mellanox/mlx4 2024-03-08T20:34:26.097 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-03-08T20:34:26.097 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-03-08T20:34:26.098 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/mellanox/mlx5 2024-03-08T20:34:26.098 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-03-08T20:34:26.098 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-03-08T20:34:26.098 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/micrel 2024-03-08T20:34:26.098 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-03-08T20:34:26.098 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/myricom 2024-03-08T20:34:26.098 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/myricom/myri10ge 2024-03-08T20:34:26.098 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-03-08T20:34:26.098 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/natsemi 2024-03-08T20:34:26.098 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-03-08T20:34:26.098 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-03-08T20:34:26.098 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/neterion 2024-03-08T20:34:26.098 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-03-08T20:34:26.098 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/nvidia 2024-03-08T20:34:26.098 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-03-08T20:34:26.099 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/packetengines 2024-03-08T20:34:26.099 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-03-08T20:34:26.099 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-03-08T20:34:26.099 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/qlogic 2024-03-08T20:34:26.099 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/qlogic/netxen 2024-03-08T20:34:26.099 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-03-08T20:34:26.099 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-03-08T20:34:26.099 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-03-08T20:34:26.099 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-03-08T20:34:26.099 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/rdc 2024-03-08T20:34:26.099 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-03-08T20:34:26.099 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/realtek 2024-03-08T20:34:26.099 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-03-08T20:34:26.099 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-03-08T20:34:26.099 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/realtek/atp.ko 2024-03-08T20:34:26.100 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-03-08T20:34:26.100 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/rocker 2024-03-08T20:34:26.100 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-03-08T20:34:26.100 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/sfc 2024-03-08T20:34:26.100 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-03-08T20:34:26.100 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/silan 2024-03-08T20:34:26.100 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-03-08T20:34:26.100 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/sis 2024-03-08T20:34:26.100 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/sis/sis190.ko 2024-03-08T20:34:26.100 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/sis/sis900.ko 2024-03-08T20:34:26.100 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/smsc 2024-03-08T20:34:26.100 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-03-08T20:34:26.100 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-03-08T20:34:26.100 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-03-08T20:34:26.101 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-03-08T20:34:26.101 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/stmicro 2024-03-08T20:34:26.101 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/stmicro/stmmac 2024-03-08T20:34:26.101 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-03-08T20:34:26.101 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-03-08T20:34:26.101 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/sun 2024-03-08T20:34:26.101 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/sun/cassini.ko 2024-03-08T20:34:26.101 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/sun/niu.ko 2024-03-08T20:34:26.101 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/sun/sungem.ko 2024-03-08T20:34:26.101 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-03-08T20:34:26.101 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/tehuti 2024-03-08T20:34:26.101 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-03-08T20:34:26.101 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/ti 2024-03-08T20:34:26.101 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/ti/tlan.ko 2024-03-08T20:34:26.102 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/via 2024-03-08T20:34:26.102 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-03-08T20:34:26.102 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-03-08T20:34:26.102 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/wiznet 2024-03-08T20:34:26.102 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-03-08T20:34:26.102 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-03-08T20:34:26.102 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/xircom 2024-03-08T20:34:26.102 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-03-08T20:34:26.102 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/geneve.ko 2024-03-08T20:34:26.102 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/hyperv 2024-03-08T20:34:26.102 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-03-08T20:34:26.102 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ieee802154 2024-03-08T20:34:26.102 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ieee802154/fakelb.ko 2024-03-08T20:34:26.102 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ifb.ko 2024-03-08T20:34:26.103 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ipvlan 2024-03-08T20:34:26.103 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ipvlan/ipvlan.ko 2024-03-08T20:34:26.103 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/macvlan.ko 2024-03-08T20:34:26.103 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/macvtap.ko 2024-03-08T20:34:26.103 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/mdio 2024-03-08T20:34:26.103 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/mdio.ko 2024-03-08T20:34:26.103 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-03-08T20:34:26.103 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-03-08T20:34:26.103 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/mdio/mdio-regmap.ko 2024-03-08T20:34:26.103 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/mii.ko 2024-03-08T20:34:26.103 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/net_failover.ko 2024-03-08T20:34:26.103 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/netconsole.ko 2024-03-08T20:34:26.103 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/nlmon.ko 2024-03-08T20:34:26.103 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/pcs 2024-03-08T20:34:26.103 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/pcs/pcs-lynx.ko 2024-03-08T20:34:26.104 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-03-08T20:34:26.104 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy 2024-03-08T20:34:26.104 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/amd.ko 2024-03-08T20:34:26.104 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-03-08T20:34:26.104 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-03-08T20:34:26.104 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/bcm7xxx.ko 2024-03-08T20:34:26.104 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/bcm87xx.ko 2024-03-08T20:34:26.104 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/broadcom.ko 2024-03-08T20:34:26.104 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/cicada.ko 2024-03-08T20:34:26.104 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/davicom.ko 2024-03-08T20:34:26.104 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/dp83640.ko 2024-03-08T20:34:26.104 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/et1011c.ko 2024-03-08T20:34:26.104 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/icplus.ko 2024-03-08T20:34:26.104 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/lxt.ko 2024-03-08T20:34:26.104 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/marvell.ko 2024-03-08T20:34:26.105 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/micrel.ko 2024-03-08T20:34:26.105 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/national.ko 2024-03-08T20:34:26.105 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/phylink.ko 2024-03-08T20:34:26.105 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/qsemi.ko 2024-03-08T20:34:26.105 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/realtek.ko 2024-03-08T20:34:26.105 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/smsc.ko 2024-03-08T20:34:26.105 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/ste10Xp.ko 2024-03-08T20:34:26.105 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/vitesse.ko 2024-03-08T20:34:26.105 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ppp 2024-03-08T20:34:26.105 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ppp/bsd_comp.ko 2024-03-08T20:34:26.105 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ppp/ppp_async.ko 2024-03-08T20:34:26.105 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ppp/ppp_deflate.ko 2024-03-08T20:34:26.105 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ppp/ppp_generic.ko 2024-03-08T20:34:26.105 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ppp/ppp_mppe.ko 2024-03-08T20:34:26.105 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ppp/ppp_synctty.ko 2024-03-08T20:34:26.106 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ppp/pppoe.ko 2024-03-08T20:34:26.106 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ppp/pppox.ko 2024-03-08T20:34:26.106 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ppp/pptp.ko 2024-03-08T20:34:26.106 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/slip 2024-03-08T20:34:26.106 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/slip/slhc.ko 2024-03-08T20:34:26.106 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/slip/slip.ko 2024-03-08T20:34:26.106 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/sungem_phy.ko 2024-03-08T20:34:26.106 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/tap.ko 2024-03-08T20:34:26.106 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/team 2024-03-08T20:34:26.106 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/team/team.ko 2024-03-08T20:34:26.106 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/team/team_mode_activebackup.ko 2024-03-08T20:34:26.106 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/team/team_mode_broadcast.ko 2024-03-08T20:34:26.106 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-03-08T20:34:26.106 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/team/team_mode_random.ko 2024-03-08T20:34:26.107 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-03-08T20:34:26.107 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/tun.ko 2024-03-08T20:34:26.107 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/veth.ko 2024-03-08T20:34:26.107 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/virtio_net.ko 2024-03-08T20:34:26.107 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/vmxnet3 2024-03-08T20:34:26.107 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-03-08T20:34:26.107 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/vxlan 2024-03-08T20:34:26.107 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/vxlan/vxlan.ko 2024-03-08T20:34:26.107 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/xen-netback 2024-03-08T20:34:26.107 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/xen-netback/xen-netback.ko 2024-03-08T20:34:26.107 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/xen-netfront.ko 2024-03-08T20:34:26.107 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ntb 2024-03-08T20:34:26.107 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ntb/ntb.ko 2024-03-08T20:34:26.107 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/nvme 2024-03-08T20:34:26.108 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/nvme/host 2024-03-08T20:34:26.108 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/nvme/host/nvme-core.ko 2024-03-08T20:34:26.108 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/nvme/host/nvme.ko 2024-03-08T20:34:26.108 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/parport 2024-03-08T20:34:26.108 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/parport/parport.ko 2024-03-08T20:34:26.108 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/parport/parport_cs.ko 2024-03-08T20:34:26.108 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/parport/parport_pc.ko 2024-03-08T20:34:26.108 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/parport/parport_serial.ko 2024-03-08T20:34:26.108 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/pci 2024-03-08T20:34:26.108 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/pci/hotplug 2024-03-08T20:34:26.108 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-03-08T20:34:26.108 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/pci/pcie 2024-03-08T20:34:26.108 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/pci/pcie/aer_inject.ko 2024-03-08T20:34:26.108 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/pci/xen-pcifront.ko 2024-03-08T20:34:26.109 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/pcmcia 2024-03-08T20:34:26.109 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/pcmcia/i82092.ko 2024-03-08T20:34:26.109 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/pcmcia/pd6729.ko 2024-03-08T20:34:26.109 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/pcmcia/yenta_socket.ko 2024-03-08T20:34:26.109 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/pinctrl 2024-03-08T20:34:26.109 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/pinctrl/intel 2024-03-08T20:34:26.109 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-03-08T20:34:26.109 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform 2024-03-08T20:34:26.109 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86 2024-03-08T20:34:26.109 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/acer-wmi.ko 2024-03-08T20:34:26.109 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/acerhdf.ko 2024-03-08T20:34:26.109 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/apple-gmux.ko 2024-03-08T20:34:26.109 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/asus-laptop.ko 2024-03-08T20:34:26.109 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-03-08T20:34:26.109 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/asus-wmi.ko 2024-03-08T20:34:26.110 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/classmate-laptop.ko 2024-03-08T20:34:26.110 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-03-08T20:34:26.110 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-03-08T20:34:26.110 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-03-08T20:34:26.110 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-03-08T20:34:26.110 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/hdaps.ko 2024-03-08T20:34:26.110 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/intel 2024-03-08T20:34:26.110 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-03-08T20:34:26.110 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/intel_ips.ko 2024-03-08T20:34:26.110 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/msi-wmi.ko 2024-03-08T20:34:26.110 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/mxm-wmi.ko 2024-03-08T20:34:26.110 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-03-08T20:34:26.110 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/samsung-laptop.ko 2024-03-08T20:34:26.111 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/samsung-q10.ko 2024-03-08T20:34:26.111 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/topstar-laptop.ko 2024-03-08T20:34:26.111 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-03-08T20:34:26.111 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/toshiba_haps.ko 2024-03-08T20:34:26.111 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/wmi-bmof.ko 2024-03-08T20:34:26.111 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/wmi.ko 2024-03-08T20:34:26.111 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/powercap 2024-03-08T20:34:26.111 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/powercap/intel_rapl_common.ko 2024-03-08T20:34:26.111 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/powercap/intel_rapl_msr.ko 2024-03-08T20:34:26.111 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/pps 2024-03-08T20:34:26.111 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/pps/clients 2024-03-08T20:34:26.111 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/pps/clients/pps-gpio.ko 2024-03-08T20:34:26.111 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/pps/clients/pps-ldisc.ko 2024-03-08T20:34:26.111 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/pps/clients/pps_parport.ko 2024-03-08T20:34:26.112 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/pps/pps_core.ko 2024-03-08T20:34:26.112 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ptp 2024-03-08T20:34:26.112 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ptp/ptp.ko 2024-03-08T20:34:26.112 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ptp/ptp_kvm.ko 2024-03-08T20:34:26.112 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc 2024-03-08T20:34:26.112 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-bq32k.ko 2024-03-08T20:34:26.112 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-ds1286.ko 2024-03-08T20:34:26.112 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-ds1307.ko 2024-03-08T20:34:26.112 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-ds1374.ko 2024-03-08T20:34:26.112 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-ds1511.ko 2024-03-08T20:34:26.112 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-ds1553.ko 2024-03-08T20:34:26.112 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-ds1672.ko 2024-03-08T20:34:26.112 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-ds1742.ko 2024-03-08T20:34:26.112 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-ds2404.ko 2024-03-08T20:34:26.113 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-ds3232.ko 2024-03-08T20:34:26.113 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-em3027.ko 2024-03-08T20:34:26.113 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-fm3130.ko 2024-03-08T20:34:26.113 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-isl12022.ko 2024-03-08T20:34:26.113 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-isl1208.ko 2024-03-08T20:34:26.113 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-m41t80.ko 2024-03-08T20:34:26.113 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-m48t35.ko 2024-03-08T20:34:26.113 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-m48t59.ko 2024-03-08T20:34:26.113 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-max6900.ko 2024-03-08T20:34:26.113 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-msm6242.ko 2024-03-08T20:34:26.113 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-pcf2127.ko 2024-03-08T20:34:26.113 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-pcf85063.ko 2024-03-08T20:34:26.113 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-pcf8523.ko 2024-03-08T20:34:26.113 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-pcf8563.ko 2024-03-08T20:34:26.113 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-pcf8583.ko 2024-03-08T20:34:26.114 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-rp5c01.ko 2024-03-08T20:34:26.114 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-rs5c372.ko 2024-03-08T20:34:26.114 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-03-08T20:34:26.114 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-rx8025.ko 2024-03-08T20:34:26.114 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-rx8581.ko 2024-03-08T20:34:26.114 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-03-08T20:34:26.114 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-x1205.ko 2024-03-08T20:34:26.114 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi 2024-03-08T20:34:26.114 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/3w-9xxx.ko 2024-03-08T20:34:26.114 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/3w-sas.ko 2024-03-08T20:34:26.114 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/3w-xxxx.ko 2024-03-08T20:34:26.114 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/BusLogic.ko 2024-03-08T20:34:26.114 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/a100u2w.ko 2024-03-08T20:34:26.114 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/aacraid 2024-03-08T20:34:26.114 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/aacraid/aacraid.ko 2024-03-08T20:34:26.115 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/advansys.ko 2024-03-08T20:34:26.115 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/aic7xxx 2024-03-08T20:34:26.115 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-03-08T20:34:26.115 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-03-08T20:34:26.115 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/am53c974.ko 2024-03-08T20:34:26.115 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/arcmsr 2024-03-08T20:34:26.115 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-03-08T20:34:26.115 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/atp870u.ko 2024-03-08T20:34:26.115 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/be2iscsi 2024-03-08T20:34:26.115 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-03-08T20:34:26.115 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/bfa 2024-03-08T20:34:26.115 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/bfa/bfa.ko 2024-03-08T20:34:26.115 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/bnx2fc 2024-03-08T20:34:26.115 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-03-08T20:34:26.116 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/bnx2i 2024-03-08T20:34:26.116 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-03-08T20:34:26.116 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/ch.ko 2024-03-08T20:34:26.116 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/csiostor 2024-03-08T20:34:26.116 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/csiostor/csiostor.ko 2024-03-08T20:34:26.116 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/cxgbi 2024-03-08T20:34:26.116 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/cxgbi/cxgb3i 2024-03-08T20:34:26.116 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-03-08T20:34:26.116 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/cxgbi/cxgb4i 2024-03-08T20:34:26.116 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-03-08T20:34:26.116 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-03-08T20:34:26.116 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/dc395x.ko 2024-03-08T20:34:26.116 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/device_handler 2024-03-08T20:34:26.116 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-03-08T20:34:26.116 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-03-08T20:34:26.116 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-03-08T20:34:26.117 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-03-08T20:34:26.117 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/dmx3191d.ko 2024-03-08T20:34:26.117 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/esas2r 2024-03-08T20:34:26.117 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/esas2r/esas2r.ko 2024-03-08T20:34:26.117 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/esp_scsi.ko 2024-03-08T20:34:26.117 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/fcoe 2024-03-08T20:34:26.117 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/fcoe/fcoe.ko 2024-03-08T20:34:26.117 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-03-08T20:34:26.117 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/fnic 2024-03-08T20:34:26.117 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/fnic/fnic.ko 2024-03-08T20:34:26.117 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/hpsa.ko 2024-03-08T20:34:26.117 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/hptiop.ko 2024-03-08T20:34:26.117 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/hv_storvsc.ko 2024-03-08T20:34:26.117 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/initio.ko 2024-03-08T20:34:26.117 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/ipr.ko 2024-03-08T20:34:26.118 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/ips.ko 2024-03-08T20:34:26.118 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/isci 2024-03-08T20:34:26.118 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/isci/isci.ko 2024-03-08T20:34:26.118 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-03-08T20:34:26.118 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/iscsi_tcp.ko 2024-03-08T20:34:26.118 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/libfc 2024-03-08T20:34:26.118 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/libfc/libfc.ko 2024-03-08T20:34:26.118 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/libiscsi.ko 2024-03-08T20:34:26.118 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/libiscsi_tcp.ko 2024-03-08T20:34:26.118 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/libsas 2024-03-08T20:34:26.118 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/libsas/libsas.ko 2024-03-08T20:34:26.118 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/lpfc 2024-03-08T20:34:26.118 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/lpfc/lpfc.ko 2024-03-08T20:34:26.119 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/megaraid 2024-03-08T20:34:26.119 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/megaraid.ko 2024-03-08T20:34:26.119 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-03-08T20:34:26.119 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-03-08T20:34:26.119 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-03-08T20:34:26.119 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/mpt3sas 2024-03-08T20:34:26.119 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-03-08T20:34:26.119 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/mvsas 2024-03-08T20:34:26.119 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/mvsas/mvsas.ko 2024-03-08T20:34:26.119 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/mvumi.ko 2024-03-08T20:34:26.119 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/pm8001 2024-03-08T20:34:26.119 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-03-08T20:34:26.119 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/pmcraid.ko 2024-03-08T20:34:26.119 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/qla1280.ko 2024-03-08T20:34:26.120 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/qla2xxx 2024-03-08T20:34:26.120 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-03-08T20:34:26.120 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-03-08T20:34:26.120 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/qla4xxx 2024-03-08T20:34:26.120 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-03-08T20:34:26.120 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/raid_class.ko 2024-03-08T20:34:26.120 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/scsi_debug.ko 2024-03-08T20:34:26.120 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/scsi_transport_fc.ko 2024-03-08T20:34:26.120 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-03-08T20:34:26.120 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/scsi_transport_sas.ko 2024-03-08T20:34:26.120 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/scsi_transport_spi.ko 2024-03-08T20:34:26.120 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/scsi_transport_srp.ko 2024-03-08T20:34:26.120 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/ses.ko 2024-03-08T20:34:26.120 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/st.ko 2024-03-08T20:34:26.121 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/stex.ko 2024-03-08T20:34:26.121 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/sym53c8xx_2 2024-03-08T20:34:26.121 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-03-08T20:34:26.121 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/virtio_scsi.ko 2024-03-08T20:34:26.121 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/vmw_pvscsi.ko 2024-03-08T20:34:26.121 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/wd719x.ko 2024-03-08T20:34:26.121 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ssb 2024-03-08T20:34:26.121 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ssb/ssb.ko 2024-03-08T20:34:26.121 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/target 2024-03-08T20:34:26.121 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/target/iscsi 2024-03-08T20:34:26.121 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-03-08T20:34:26.121 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/target/loopback 2024-03-08T20:34:26.121 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/target/loopback/tcm_loop.ko 2024-03-08T20:34:26.121 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/target/target_core_file.ko 2024-03-08T20:34:26.122 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/target/target_core_iblock.ko 2024-03-08T20:34:26.122 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/target/target_core_mod.ko 2024-03-08T20:34:26.122 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/target/target_core_pscsi.ko 2024-03-08T20:34:26.122 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/target/target_core_user.ko 2024-03-08T20:34:26.122 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/target/tcm_fc 2024-03-08T20:34:26.122 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-03-08T20:34:26.122 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal 2024-03-08T20:34:26.122 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel 2024-03-08T20:34:26.122 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel/int340x_thermal 2024-03-08T20:34:26.122 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-03-08T20:34:26.122 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-03-08T20:34:26.122 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-03-08T20:34:26.122 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-03-08T20:34:26.122 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-03-08T20:34:26.123 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-03-08T20:34:26.123 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-03-08T20:34:26.123 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-03-08T20:34:26.123 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-03-08T20:34:26.123 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-03-08T20:34:26.123 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-03-08T20:34:26.123 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-03-08T20:34:26.123 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-03-08T20:34:26.123 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-03-08T20:34:26.123 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-03-08T20:34:26.123 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-03-08T20:34:26.123 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-03-08T20:34:26.123 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-03-08T20:34:26.123 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/tty 2024-03-08T20:34:26.124 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/tty/ipwireless 2024-03-08T20:34:26.124 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-03-08T20:34:26.124 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/tty/n_gsm.ko 2024-03-08T20:34:26.124 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/tty/n_hdlc.ko 2024-03-08T20:34:26.124 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/tty/nozomi.ko 2024-03-08T20:34:26.124 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/tty/serial 2024-03-08T20:34:26.124 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/tty/serial/8250 2024-03-08T20:34:26.124 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-03-08T20:34:26.124 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/tty/serial/arc_uart.ko 2024-03-08T20:34:26.124 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/tty/serial/jsm 2024-03-08T20:34:26.124 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/tty/serial/jsm/jsm.ko 2024-03-08T20:34:26.124 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ufs 2024-03-08T20:34:26.124 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ufs/core 2024-03-08T20:34:26.124 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ufs/core/ufshcd-core.ko 2024-03-08T20:34:26.124 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ufs/host 2024-03-08T20:34:26.125 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-03-08T20:34:26.125 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/uio 2024-03-08T20:34:26.125 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/uio/uio.ko 2024-03-08T20:34:26.125 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/uio/uio_aec.ko 2024-03-08T20:34:26.125 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/uio/uio_cif.ko 2024-03-08T20:34:26.125 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/uio/uio_pci_generic.ko 2024-03-08T20:34:26.125 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/uio/uio_sercos3.ko 2024-03-08T20:34:26.125 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb 2024-03-08T20:34:26.125 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/class 2024-03-08T20:34:26.125 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/class/cdc-acm.ko 2024-03-08T20:34:26.125 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/class/cdc-wdm.ko 2024-03-08T20:34:26.125 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/class/usblp.ko 2024-03-08T20:34:26.125 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/class/usbtmc.ko 2024-03-08T20:34:26.125 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/host 2024-03-08T20:34:26.125 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/host/sl811-hcd.ko 2024-03-08T20:34:26.126 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/image 2024-03-08T20:34:26.126 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/image/mdc800.ko 2024-03-08T20:34:26.126 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/image/microtek.ko 2024-03-08T20:34:26.126 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/misc 2024-03-08T20:34:26.126 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/misc/adutux.ko 2024-03-08T20:34:26.126 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/misc/appledisplay.ko 2024-03-08T20:34:26.126 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/misc/emi26.ko 2024-03-08T20:34:26.126 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/misc/emi62.ko 2024-03-08T20:34:26.126 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/misc/ezusb.ko 2024-03-08T20:34:26.126 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/misc/idmouse.ko 2024-03-08T20:34:26.126 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/misc/iowarrior.ko 2024-03-08T20:34:26.126 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/misc/isight_firmware.ko 2024-03-08T20:34:26.126 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/misc/ldusb.ko 2024-03-08T20:34:26.126 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/misc/legousbtower.ko 2024-03-08T20:34:26.127 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/misc/sisusbvga 2024-03-08T20:34:26.127 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-03-08T20:34:26.127 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/misc/trancevibrator.ko 2024-03-08T20:34:26.127 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/misc/usb3503.ko 2024-03-08T20:34:26.127 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/misc/usblcd.ko 2024-03-08T20:34:26.127 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/misc/usbsevseg.ko 2024-03-08T20:34:26.127 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/misc/uss720.ko 2024-03-08T20:34:26.127 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/misc/yurex.ko 2024-03-08T20:34:26.127 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/phy 2024-03-08T20:34:26.127 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/phy/phy-generic.ko 2024-03-08T20:34:26.127 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial 2024-03-08T20:34:26.127 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/aircable.ko 2024-03-08T20:34:26.127 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/ark3116.ko 2024-03-08T20:34:26.127 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/belkin_sa.ko 2024-03-08T20:34:26.127 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/ch341.ko 2024-03-08T20:34:26.128 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/cp210x.ko 2024-03-08T20:34:26.128 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/cyberjack.ko 2024-03-08T20:34:26.128 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/cypress_m8.ko 2024-03-08T20:34:26.128 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/digi_acceleport.ko 2024-03-08T20:34:26.128 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/empeg.ko 2024-03-08T20:34:26.128 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/ftdi_sio.ko 2024-03-08T20:34:26.128 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/garmin_gps.ko 2024-03-08T20:34:26.128 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/io_edgeport.ko 2024-03-08T20:34:26.128 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/io_ti.ko 2024-03-08T20:34:26.128 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/ipaq.ko 2024-03-08T20:34:26.128 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/ipw.ko 2024-03-08T20:34:26.128 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/ir-usb.ko 2024-03-08T20:34:26.128 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-03-08T20:34:26.128 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/keyspan.ko 2024-03-08T20:34:26.129 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/keyspan_pda.ko 2024-03-08T20:34:26.129 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/kl5kusb105.ko 2024-03-08T20:34:26.129 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/kobil_sct.ko 2024-03-08T20:34:26.129 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/mct_u232.ko 2024-03-08T20:34:26.129 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/mos7720.ko 2024-03-08T20:34:26.129 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/mos7840.ko 2024-03-08T20:34:26.129 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/navman.ko 2024-03-08T20:34:26.129 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/omninet.ko 2024-03-08T20:34:26.129 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/opticon.ko 2024-03-08T20:34:26.129 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/option.ko 2024-03-08T20:34:26.129 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/oti6858.ko 2024-03-08T20:34:26.129 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/pl2303.ko 2024-03-08T20:34:26.129 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/qcaux.ko 2024-03-08T20:34:26.129 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/qcserial.ko 2024-03-08T20:34:26.129 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/quatech2.ko 2024-03-08T20:34:26.130 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/safe_serial.ko 2024-03-08T20:34:26.130 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/sierra.ko 2024-03-08T20:34:26.130 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/spcp8x5.ko 2024-03-08T20:34:26.130 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/ssu100.ko 2024-03-08T20:34:26.130 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/symbolserial.ko 2024-03-08T20:34:26.130 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-03-08T20:34:26.130 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-03-08T20:34:26.130 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/usb_debug.ko 2024-03-08T20:34:26.130 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/usb_wwan.ko 2024-03-08T20:34:26.130 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/visor.ko 2024-03-08T20:34:26.130 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/whiteheat.ko 2024-03-08T20:34:26.130 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/xsens_mt.ko 2024-03-08T20:34:26.130 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/storage 2024-03-08T20:34:26.130 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/storage/uas.ko 2024-03-08T20:34:26.130 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/storage/ums-alauda.ko 2024-03-08T20:34:26.131 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/storage/ums-cypress.ko 2024-03-08T20:34:26.131 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/storage/ums-datafab.ko 2024-03-08T20:34:26.131 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-03-08T20:34:26.131 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/storage/ums-freecom.ko 2024-03-08T20:34:26.131 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/storage/ums-isd200.ko 2024-03-08T20:34:26.131 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-03-08T20:34:26.131 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/storage/ums-karma.ko 2024-03-08T20:34:26.131 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/storage/ums-onetouch.ko 2024-03-08T20:34:26.131 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/storage/ums-realtek.ko 2024-03-08T20:34:26.131 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/storage/ums-sddr09.ko 2024-03-08T20:34:26.131 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/storage/ums-sddr55.ko 2024-03-08T20:34:26.131 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/storage/ums-usbat.ko 2024-03-08T20:34:26.131 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/storage/usb-storage.ko 2024-03-08T20:34:26.131 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/usbip 2024-03-08T20:34:26.132 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/usbip/usbip-core.ko 2024-03-08T20:34:26.132 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/usbip/usbip-host.ko 2024-03-08T20:34:26.132 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-03-08T20:34:26.132 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/vfio 2024-03-08T20:34:26.132 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/vfio/pci 2024-03-08T20:34:26.132 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-03-08T20:34:26.132 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/vfio/pci/vfio-pci.ko 2024-03-08T20:34:26.132 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/vfio/vfio.ko 2024-03-08T20:34:26.132 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-03-08T20:34:26.132 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/vhost 2024-03-08T20:34:26.132 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/vhost/vhost.ko 2024-03-08T20:34:26.132 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/vhost/vhost_iotlb.ko 2024-03-08T20:34:26.132 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/vhost/vhost_net.ko 2024-03-08T20:34:26.132 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/vhost/vhost_scsi.ko 2024-03-08T20:34:26.133 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/video 2024-03-08T20:34:26.133 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/video/backlight 2024-03-08T20:34:26.133 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/video/backlight/apple_bl.ko 2024-03-08T20:34:26.133 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/video/backlight/lcd.ko 2024-03-08T20:34:26.133 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/video/backlight/platform_lcd.ko 2024-03-08T20:34:26.133 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/virtio 2024-03-08T20:34:26.133 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/virtio/virtio.ko 2024-03-08T20:34:26.133 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/virtio/virtio_balloon.ko 2024-03-08T20:34:26.133 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/virtio/virtio_mmio.ko 2024-03-08T20:34:26.133 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/virtio/virtio_pci.ko 2024-03-08T20:34:26.133 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-03-08T20:34:26.133 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-03-08T20:34:26.133 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/virtio/virtio_ring.ko 2024-03-08T20:34:26.133 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/w1 2024-03-08T20:34:26.134 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/w1/masters 2024-03-08T20:34:26.134 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/w1/masters/ds2482.ko 2024-03-08T20:34:26.134 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/w1/masters/ds2490.ko 2024-03-08T20:34:26.134 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/w1/slaves 2024-03-08T20:34:26.134 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-03-08T20:34:26.134 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-03-08T20:34:26.134 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-03-08T20:34:26.134 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-03-08T20:34:26.134 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-03-08T20:34:26.134 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-03-08T20:34:26.134 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-03-08T20:34:26.134 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-03-08T20:34:26.134 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-03-08T20:34:26.134 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/w1/slaves/w1_smem.ko 2024-03-08T20:34:26.135 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/w1/slaves/w1_therm.ko 2024-03-08T20:34:26.135 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/w1/wire.ko 2024-03-08T20:34:26.135 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog 2024-03-08T20:34:26.135 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/alim1535_wdt.ko 2024-03-08T20:34:26.135 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/alim7101_wdt.ko 2024-03-08T20:34:26.135 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/f71808e_wdt.ko 2024-03-08T20:34:26.135 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/hpwdt.ko 2024-03-08T20:34:26.135 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/i6300esb.ko 2024-03-08T20:34:26.135 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-03-08T20:34:26.135 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/iTCO_wdt.ko 2024-03-08T20:34:26.135 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/ib700wdt.ko 2024-03-08T20:34:26.135 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/ibmasr.ko 2024-03-08T20:34:26.135 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-03-08T20:34:26.135 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/it8712f_wdt.ko 2024-03-08T20:34:26.135 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/it87_wdt.ko 2024-03-08T20:34:26.136 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/machzwd.ko 2024-03-08T20:34:26.136 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/nv_tco.ko 2024-03-08T20:34:26.136 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/pcwd_pci.ko 2024-03-08T20:34:26.136 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/pcwd_usb.ko 2024-03-08T20:34:26.136 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-03-08T20:34:26.136 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/sch311x_wdt.ko 2024-03-08T20:34:26.136 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/softdog.ko 2024-03-08T20:34:26.136 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/sp5100_tco.ko 2024-03-08T20:34:26.136 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/via_wdt.ko 2024-03-08T20:34:26.136 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-03-08T20:34:26.136 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/w83877f_wdt.ko 2024-03-08T20:34:26.136 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/w83977f_wdt.ko 2024-03-08T20:34:26.136 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/wdt_pci.ko 2024-03-08T20:34:26.136 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/xen_wdt.ko 2024-03-08T20:34:26.137 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/xen 2024-03-08T20:34:26.137 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/xen/xen-acpi-processor.ko 2024-03-08T20:34:26.137 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/xen/xen-evtchn.ko 2024-03-08T20:34:26.137 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/xen/xen-gntalloc.ko 2024-03-08T20:34:26.137 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/xen/xen-gntdev.ko 2024-03-08T20:34:26.137 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/xen/xen-pciback 2024-03-08T20:34:26.137 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-03-08T20:34:26.137 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/xen/xen-privcmd.ko 2024-03-08T20:34:26.137 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/xen/xenfs 2024-03-08T20:34:26.137 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/xen/xenfs/xenfs.ko 2024-03-08T20:34:26.137 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs 2024-03-08T20:34:26.137 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/9p 2024-03-08T20:34:26.137 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/9p/9p.ko 2024-03-08T20:34:26.137 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/affs 2024-03-08T20:34:26.137 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/affs/affs.ko 2024-03-08T20:34:26.138 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/befs 2024-03-08T20:34:26.138 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/befs/befs.ko 2024-03-08T20:34:26.138 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/binfmt_misc.ko 2024-03-08T20:34:26.138 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/btrfs 2024-03-08T20:34:26.138 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/btrfs/btrfs.ko 2024-03-08T20:34:26.138 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/cachefiles 2024-03-08T20:34:26.138 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/cachefiles/cachefiles.ko 2024-03-08T20:34:26.138 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/ceph 2024-03-08T20:34:26.138 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/ceph/ceph.ko 2024-03-08T20:34:26.138 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/coda 2024-03-08T20:34:26.138 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/coda/coda.ko 2024-03-08T20:34:26.138 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/cramfs 2024-03-08T20:34:26.138 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/cramfs/cramfs.ko 2024-03-08T20:34:26.139 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/dlm 2024-03-08T20:34:26.139 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/dlm/dlm.ko 2024-03-08T20:34:26.139 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/ecryptfs 2024-03-08T20:34:26.139 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/ecryptfs/ecryptfs.ko 2024-03-08T20:34:26.139 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/f2fs 2024-03-08T20:34:26.139 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/f2fs/f2fs.ko 2024-03-08T20:34:26.139 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/fat 2024-03-08T20:34:26.139 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/fat/fat.ko 2024-03-08T20:34:26.139 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/fat/msdos.ko 2024-03-08T20:34:26.139 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/fat/vfat.ko 2024-03-08T20:34:26.139 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/fuse 2024-03-08T20:34:26.139 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/fuse/cuse.ko 2024-03-08T20:34:26.139 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/fuse/fuse.ko 2024-03-08T20:34:26.139 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/gfs2 2024-03-08T20:34:26.140 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/gfs2/gfs2.ko 2024-03-08T20:34:26.140 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/hfs 2024-03-08T20:34:26.140 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/hfs/hfs.ko 2024-03-08T20:34:26.140 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/hfsplus 2024-03-08T20:34:26.140 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/hfsplus/hfsplus.ko 2024-03-08T20:34:26.140 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/isofs 2024-03-08T20:34:26.140 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/isofs/isofs.ko 2024-03-08T20:34:26.140 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/jfs 2024-03-08T20:34:26.140 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/jfs/jfs.ko 2024-03-08T20:34:26.140 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/lockd 2024-03-08T20:34:26.140 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/lockd/lockd.ko 2024-03-08T20:34:26.140 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/minix 2024-03-08T20:34:26.140 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/minix/minix.ko 2024-03-08T20:34:26.141 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/netfs 2024-03-08T20:34:26.141 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/netfs/netfs.ko 2024-03-08T20:34:26.141 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nfs 2024-03-08T20:34:26.141 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nfs/blocklayout 2024-03-08T20:34:26.141 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-03-08T20:34:26.141 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nfs/filelayout 2024-03-08T20:34:26.141 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-03-08T20:34:26.141 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nfs/flexfilelayout 2024-03-08T20:34:26.141 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-03-08T20:34:26.141 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nfs/nfs.ko 2024-03-08T20:34:26.141 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nfs/nfsv3.ko 2024-03-08T20:34:26.141 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nfs/nfsv4.ko 2024-03-08T20:34:26.141 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nfs_common 2024-03-08T20:34:26.141 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nfs_common/grace.ko 2024-03-08T20:34:26.142 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nfs_common/nfs_acl.ko 2024-03-08T20:34:26.142 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nfsd 2024-03-08T20:34:26.142 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nfsd/nfsd.ko 2024-03-08T20:34:26.142 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nilfs2 2024-03-08T20:34:26.142 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nilfs2/nilfs2.ko 2024-03-08T20:34:26.142 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls 2024-03-08T20:34:26.142 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/mac-celtic.ko 2024-03-08T20:34:26.142 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/mac-centeuro.ko 2024-03-08T20:34:26.142 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/mac-croatian.ko 2024-03-08T20:34:26.142 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/mac-cyrillic.ko 2024-03-08T20:34:26.142 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/mac-gaelic.ko 2024-03-08T20:34:26.142 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/mac-greek.ko 2024-03-08T20:34:26.142 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/mac-iceland.ko 2024-03-08T20:34:26.142 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/mac-inuit.ko 2024-03-08T20:34:26.143 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/mac-roman.ko 2024-03-08T20:34:26.143 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/mac-romanian.ko 2024-03-08T20:34:26.143 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/mac-turkish.ko 2024-03-08T20:34:26.143 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp1250.ko 2024-03-08T20:34:26.143 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp1251.ko 2024-03-08T20:34:26.143 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp1255.ko 2024-03-08T20:34:26.143 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp737.ko 2024-03-08T20:34:26.143 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp775.ko 2024-03-08T20:34:26.143 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp850.ko 2024-03-08T20:34:26.143 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp852.ko 2024-03-08T20:34:26.143 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp855.ko 2024-03-08T20:34:26.143 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp857.ko 2024-03-08T20:34:26.143 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp860.ko 2024-03-08T20:34:26.143 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp861.ko 2024-03-08T20:34:26.144 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp862.ko 2024-03-08T20:34:26.144 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp863.ko 2024-03-08T20:34:26.144 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp864.ko 2024-03-08T20:34:26.144 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp865.ko 2024-03-08T20:34:26.144 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp866.ko 2024-03-08T20:34:26.144 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp869.ko 2024-03-08T20:34:26.144 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp874.ko 2024-03-08T20:34:26.144 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp932.ko 2024-03-08T20:34:26.144 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp936.ko 2024-03-08T20:34:26.144 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp949.ko 2024-03-08T20:34:26.144 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp950.ko 2024-03-08T20:34:26.144 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_euc-jp.ko 2024-03-08T20:34:26.144 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_iso8859-1.ko 2024-03-08T20:34:26.144 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_iso8859-13.ko 2024-03-08T20:34:26.145 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_iso8859-14.ko 2024-03-08T20:34:26.145 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_iso8859-15.ko 2024-03-08T20:34:26.145 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_iso8859-2.ko 2024-03-08T20:34:26.145 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_iso8859-3.ko 2024-03-08T20:34:26.145 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_iso8859-4.ko 2024-03-08T20:34:26.145 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_iso8859-5.ko 2024-03-08T20:34:26.145 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_iso8859-6.ko 2024-03-08T20:34:26.145 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_iso8859-7.ko 2024-03-08T20:34:26.145 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_iso8859-9.ko 2024-03-08T20:34:26.145 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_koi8-r.ko 2024-03-08T20:34:26.145 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_koi8-ru.ko 2024-03-08T20:34:26.145 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_koi8-u.ko 2024-03-08T20:34:26.145 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_ucs2_utils.ko 2024-03-08T20:34:26.145 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_utf8.ko 2024-03-08T20:34:26.145 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/ocfs2 2024-03-08T20:34:26.146 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/ocfs2/cluster 2024-03-08T20:34:26.146 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-03-08T20:34:26.146 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/ocfs2/dlm 2024-03-08T20:34:26.146 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-03-08T20:34:26.146 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/ocfs2/dlmfs 2024-03-08T20:34:26.146 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-03-08T20:34:26.146 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/ocfs2/ocfs2.ko 2024-03-08T20:34:26.146 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-03-08T20:34:26.146 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-03-08T20:34:26.146 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-03-08T20:34:26.146 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/overlayfs 2024-03-08T20:34:26.146 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/overlayfs/overlay.ko 2024-03-08T20:34:26.146 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/pstore 2024-03-08T20:34:26.146 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/pstore/ramoops.ko 2024-03-08T20:34:26.147 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/reiserfs 2024-03-08T20:34:26.147 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/reiserfs/reiserfs.ko 2024-03-08T20:34:26.147 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/romfs 2024-03-08T20:34:26.147 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/romfs/romfs.ko 2024-03-08T20:34:26.147 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/smb 2024-03-08T20:34:26.147 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/smb/client 2024-03-08T20:34:26.147 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/smb/client/cifs.ko 2024-03-08T20:34:26.147 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/smb/common 2024-03-08T20:34:26.147 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/smb/common/cifs_arc4.ko 2024-03-08T20:34:26.147 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/smb/common/cifs_md4.ko 2024-03-08T20:34:26.147 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/squashfs 2024-03-08T20:34:26.147 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/squashfs/squashfs.ko 2024-03-08T20:34:26.147 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/sysv 2024-03-08T20:34:26.147 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/sysv/sysv.ko 2024-03-08T20:34:26.148 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/udf 2024-03-08T20:34:26.148 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/udf/udf.ko 2024-03-08T20:34:26.148 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/ufs 2024-03-08T20:34:26.148 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/ufs/ufs.ko 2024-03-08T20:34:26.148 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/xfs 2024-03-08T20:34:26.148 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/xfs/xfs.ko 2024-03-08T20:34:26.148 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/kernel 2024-03-08T20:34:26.148 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/kernel/rcu 2024-03-08T20:34:26.148 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/kernel/rcu/rcutorture.ko 2024-03-08T20:34:26.148 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/kernel/torture.ko 2024-03-08T20:34:26.148 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/kernel/trace 2024-03-08T20:34:26.148 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-03-08T20:34:26.148 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib 2024-03-08T20:34:26.148 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/asn1_encoder.ko 2024-03-08T20:34:26.148 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/crc-itu-t.ko 2024-03-08T20:34:26.149 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/crc8.ko 2024-03-08T20:34:26.149 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/crypto 2024-03-08T20:34:26.149 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/crypto/libarc4.ko 2024-03-08T20:34:26.149 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/crypto/libdes.ko 2024-03-08T20:34:26.149 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/libcrc32c.ko 2024-03-08T20:34:26.149 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/lru_cache.ko 2024-03-08T20:34:26.149 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/lz4 2024-03-08T20:34:26.149 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/lz4/lz4_compress.ko 2024-03-08T20:34:26.149 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/lz4/lz4hc_compress.ko 2024-03-08T20:34:26.149 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/math 2024-03-08T20:34:26.149 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/math/cordic.ko 2024-03-08T20:34:26.149 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/raid6 2024-03-08T20:34:26.149 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/raid6/raid6_pq.ko 2024-03-08T20:34:26.149 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/reed_solomon 2024-03-08T20:34:26.149 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/reed_solomon/reed_solomon.ko 2024-03-08T20:34:26.150 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/ts_bm.ko 2024-03-08T20:34:26.150 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/ts_fsm.ko 2024-03-08T20:34:26.150 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/ts_kmp.ko 2024-03-08T20:34:26.150 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/zstd 2024-03-08T20:34:26.150 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/zstd/zstd_compress.ko 2024-03-08T20:34:26.150 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/mm 2024-03-08T20:34:26.150 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/mm/hwpoison-inject.ko 2024-03-08T20:34:26.150 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net 2024-03-08T20:34:26.150 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/6lowpan 2024-03-08T20:34:26.150 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/6lowpan/6lowpan.ko 2024-03-08T20:34:26.150 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/6lowpan/nhc_dest.ko 2024-03-08T20:34:26.150 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/6lowpan/nhc_fragment.ko 2024-03-08T20:34:26.150 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/6lowpan/nhc_hop.ko 2024-03-08T20:34:26.150 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/6lowpan/nhc_ipv6.ko 2024-03-08T20:34:26.151 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/6lowpan/nhc_mobility.ko 2024-03-08T20:34:26.151 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/6lowpan/nhc_routing.ko 2024-03-08T20:34:26.151 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/6lowpan/nhc_udp.ko 2024-03-08T20:34:26.151 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/802 2024-03-08T20:34:26.151 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/802/garp.ko 2024-03-08T20:34:26.151 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/802/mrp.ko 2024-03-08T20:34:26.151 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/802/p8022.ko 2024-03-08T20:34:26.151 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/802/psnap.ko 2024-03-08T20:34:26.151 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/802/stp.ko 2024-03-08T20:34:26.151 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/8021q 2024-03-08T20:34:26.151 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/8021q/8021q.ko 2024-03-08T20:34:26.151 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/9p 2024-03-08T20:34:26.151 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/9p/9pnet.ko 2024-03-08T20:34:26.151 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/9p/9pnet_fd.ko 2024-03-08T20:34:26.151 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/9p/9pnet_rdma.ko 2024-03-08T20:34:26.152 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/9p/9pnet_virtio.ko 2024-03-08T20:34:26.152 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/batman-adv 2024-03-08T20:34:26.152 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/batman-adv/batman-adv.ko 2024-03-08T20:34:26.152 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge 2024-03-08T20:34:26.152 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/br_netfilter.ko 2024-03-08T20:34:26.152 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/bridge.ko 2024-03-08T20:34:26.152 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter 2024-03-08T20:34:26.152 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-03-08T20:34:26.152 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebt_among.ko 2024-03-08T20:34:26.152 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebt_arp.ko 2024-03-08T20:34:26.152 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-03-08T20:34:26.152 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-03-08T20:34:26.152 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebt_ip.ko 2024-03-08T20:34:26.152 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-03-08T20:34:26.153 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebt_limit.ko 2024-03-08T20:34:26.153 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebt_log.ko 2024-03-08T20:34:26.153 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebt_mark.ko 2024-03-08T20:34:26.153 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-03-08T20:34:26.153 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-03-08T20:34:26.153 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-03-08T20:34:26.153 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-03-08T20:34:26.153 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebt_snat.ko 2024-03-08T20:34:26.153 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebt_stp.ko 2024-03-08T20:34:26.153 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-03-08T20:34:26.153 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-03-08T20:34:26.153 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-03-08T20:34:26.153 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-03-08T20:34:26.153 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebtables.ko 2024-03-08T20:34:26.153 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-03-08T20:34:26.154 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-03-08T20:34:26.154 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-03-08T20:34:26.154 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ceph 2024-03-08T20:34:26.154 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ceph/libceph.ko 2024-03-08T20:34:26.154 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/core 2024-03-08T20:34:26.154 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/core/failover.ko 2024-03-08T20:34:26.154 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/core/pktgen.ko 2024-03-08T20:34:26.154 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/dccp 2024-03-08T20:34:26.154 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/dccp/dccp.ko 2024-03-08T20:34:26.154 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/dccp/dccp_diag.ko 2024-03-08T20:34:26.154 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/dccp/dccp_ipv4.ko 2024-03-08T20:34:26.154 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/dccp/dccp_ipv6.ko 2024-03-08T20:34:26.154 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/dns_resolver 2024-03-08T20:34:26.154 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/dns_resolver/dns_resolver.ko 2024-03-08T20:34:26.155 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/dsa 2024-03-08T20:34:26.155 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/dsa/dsa_core.ko 2024-03-08T20:34:26.155 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/dsa/tag_brcm.ko 2024-03-08T20:34:26.155 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/dsa/tag_dsa.ko 2024-03-08T20:34:26.155 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/dsa/tag_none.ko 2024-03-08T20:34:26.155 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/dsa/tag_trailer.ko 2024-03-08T20:34:26.155 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ieee802154 2024-03-08T20:34:26.155 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ieee802154/6lowpan 2024-03-08T20:34:26.155 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-03-08T20:34:26.155 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ieee802154/ieee802154.ko 2024-03-08T20:34:26.155 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ieee802154/ieee802154_socket.ko 2024-03-08T20:34:26.155 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4 2024-03-08T20:34:26.155 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/ah4.ko 2024-03-08T20:34:26.155 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/esp4.ko 2024-03-08T20:34:26.155 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/fou.ko 2024-03-08T20:34:26.156 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/gre.ko 2024-03-08T20:34:26.156 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/inet_diag.ko 2024-03-08T20:34:26.156 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/ip_gre.ko 2024-03-08T20:34:26.156 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/ip_tunnel.ko 2024-03-08T20:34:26.156 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/ip_vti.ko 2024-03-08T20:34:26.156 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/ipcomp.ko 2024-03-08T20:34:26.156 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/ipip.ko 2024-03-08T20:34:26.156 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter 2024-03-08T20:34:26.156 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/arp_tables.ko 2024-03-08T20:34:26.156 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-03-08T20:34:26.156 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-03-08T20:34:26.156 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-03-08T20:34:26.156 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-03-08T20:34:26.156 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-03-08T20:34:26.156 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-03-08T20:34:26.157 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-03-08T20:34:26.157 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-03-08T20:34:26.157 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-03-08T20:34:26.157 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/iptable_security.ko 2024-03-08T20:34:26.157 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-03-08T20:34:26.157 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-03-08T20:34:26.157 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-03-08T20:34:26.157 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-03-08T20:34:26.157 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-03-08T20:34:26.157 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-03-08T20:34:26.157 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-03-08T20:34:26.157 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-03-08T20:34:26.157 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-03-08T20:34:26.157 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-03-08T20:34:26.157 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/tcp_bic.ko 2024-03-08T20:34:26.158 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/tcp_dctcp.ko 2024-03-08T20:34:26.158 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/tcp_diag.ko 2024-03-08T20:34:26.158 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/tcp_highspeed.ko 2024-03-08T20:34:26.158 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/tcp_htcp.ko 2024-03-08T20:34:26.158 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/tcp_hybla.ko 2024-03-08T20:34:26.158 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/tcp_illinois.ko 2024-03-08T20:34:26.158 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/tcp_lp.ko 2024-03-08T20:34:26.158 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/tcp_scalable.ko 2024-03-08T20:34:26.158 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/tcp_vegas.ko 2024-03-08T20:34:26.158 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/tcp_veno.ko 2024-03-08T20:34:26.158 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/tcp_westwood.ko 2024-03-08T20:34:26.158 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/tcp_yeah.ko 2024-03-08T20:34:26.158 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/tunnel4.ko 2024-03-08T20:34:26.158 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/udp_diag.ko 2024-03-08T20:34:26.159 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/udp_tunnel.ko 2024-03-08T20:34:26.159 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/xfrm4_tunnel.ko 2024-03-08T20:34:26.159 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6 2024-03-08T20:34:26.159 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/ah6.ko 2024-03-08T20:34:26.159 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/esp6.ko 2024-03-08T20:34:26.159 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/fou6.ko 2024-03-08T20:34:26.159 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/ip6_tunnel.ko 2024-03-08T20:34:26.159 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-03-08T20:34:26.159 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/ip6_vti.ko 2024-03-08T20:34:26.159 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/ipcomp6.ko 2024-03-08T20:34:26.159 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter 2024-03-08T20:34:26.159 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-03-08T20:34:26.159 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-03-08T20:34:26.159 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-03-08T20:34:26.160 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-03-08T20:34:26.160 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-03-08T20:34:26.160 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-03-08T20:34:26.160 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-03-08T20:34:26.160 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-03-08T20:34:26.160 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-03-08T20:34:26.160 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-03-08T20:34:26.160 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-03-08T20:34:26.160 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-03-08T20:34:26.160 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-03-08T20:34:26.160 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-03-08T20:34:26.160 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-03-08T20:34:26.160 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-03-08T20:34:26.160 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-03-08T20:34:26.160 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-03-08T20:34:26.161 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-03-08T20:34:26.161 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-03-08T20:34:26.161 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-03-08T20:34:26.161 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-03-08T20:34:26.161 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-03-08T20:34:26.161 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-03-08T20:34:26.161 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/sit.ko 2024-03-08T20:34:26.161 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/tunnel6.ko 2024-03-08T20:34:26.161 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/xfrm6_tunnel.ko 2024-03-08T20:34:26.161 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/key 2024-03-08T20:34:26.161 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/key/af_key.ko 2024-03-08T20:34:26.161 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/l2tp 2024-03-08T20:34:26.161 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/l2tp/l2tp_core.ko 2024-03-08T20:34:26.161 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/l2tp/l2tp_debugfs.ko 2024-03-08T20:34:26.162 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/l2tp/l2tp_eth.ko 2024-03-08T20:34:26.162 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/l2tp/l2tp_ip.ko 2024-03-08T20:34:26.162 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/l2tp/l2tp_ip6.ko 2024-03-08T20:34:26.162 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/l2tp/l2tp_netlink.ko 2024-03-08T20:34:26.162 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/l2tp/l2tp_ppp.ko 2024-03-08T20:34:26.162 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/llc 2024-03-08T20:34:26.162 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/llc/llc.ko 2024-03-08T20:34:26.162 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/mac802154 2024-03-08T20:34:26.162 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/mac802154/mac802154.ko 2024-03-08T20:34:26.162 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/mpls 2024-03-08T20:34:26.162 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/mpls/mpls_gso.ko 2024-03-08T20:34:26.162 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter 2024-03-08T20:34:26.162 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipset 2024-03-08T20:34:26.163 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipset/ip_set.ko 2024-03-08T20:34:26.163 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-03-08T20:34:26.163 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-03-08T20:34:26.163 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-03-08T20:34:26.163 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-03-08T20:34:26.163 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-03-08T20:34:26.163 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-03-08T20:34:26.163 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-03-08T20:34:26.163 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-03-08T20:34:26.163 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-03-08T20:34:26.163 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-03-08T20:34:26.163 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-03-08T20:34:26.163 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-03-08T20:34:26.163 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-03-08T20:34:26.164 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-03-08T20:34:26.164 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-03-08T20:34:26.164 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipvs 2024-03-08T20:34:26.164 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipvs/ip_vs.ko 2024-03-08T20:34:26.164 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-03-08T20:34:26.164 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-03-08T20:34:26.164 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-03-08T20:34:26.164 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-03-08T20:34:26.164 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-03-08T20:34:26.164 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-03-08T20:34:26.164 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-03-08T20:34:26.164 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-03-08T20:34:26.164 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-03-08T20:34:26.164 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-03-08T20:34:26.164 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-03-08T20:34:26.165 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-03-08T20:34:26.165 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-03-08T20:34:26.165 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_conncount.ko 2024-03-08T20:34:26.165 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_conntrack.ko 2024-03-08T20:34:26.165 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-03-08T20:34:26.165 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-03-08T20:34:26.165 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-03-08T20:34:26.165 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_conntrack_h323.ko 2024-03-08T20:34:26.165 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_conntrack_irc.ko 2024-03-08T20:34:26.165 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-03-08T20:34:26.165 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-03-08T20:34:26.165 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-03-08T20:34:26.165 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_conntrack_sane.ko 2024-03-08T20:34:26.165 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_conntrack_sip.ko 2024-03-08T20:34:26.165 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-03-08T20:34:26.166 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-03-08T20:34:26.166 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_dup_netdev.ko 2024-03-08T20:34:26.166 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_flow_table.ko 2024-03-08T20:34:26.166 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_flow_table_inet.ko 2024-03-08T20:34:26.166 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_log_syslog.ko 2024-03-08T20:34:26.166 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_nat.ko 2024-03-08T20:34:26.166 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_nat_amanda.ko 2024-03-08T20:34:26.166 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_nat_ftp.ko 2024-03-08T20:34:26.166 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_nat_irc.ko 2024-03-08T20:34:26.166 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_nat_sip.ko 2024-03-08T20:34:26.166 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_nat_tftp.ko 2024-03-08T20:34:26.166 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_synproxy_core.ko 2024-03-08T20:34:26.166 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_tables.ko 2024-03-08T20:34:26.166 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nfnetlink.ko 2024-03-08T20:34:26.167 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nfnetlink_acct.ko 2024-03-08T20:34:26.167 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nfnetlink_log.ko 2024-03-08T20:34:26.167 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nfnetlink_osf.ko 2024-03-08T20:34:26.167 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nfnetlink_queue.ko 2024-03-08T20:34:26.167 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nft_chain_nat.ko 2024-03-08T20:34:26.167 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nft_compat.ko 2024-03-08T20:34:26.167 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nft_ct.ko 2024-03-08T20:34:26.167 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nft_dup_netdev.ko 2024-03-08T20:34:26.167 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nft_fib.ko 2024-03-08T20:34:26.167 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nft_fwd_netdev.ko 2024-03-08T20:34:26.167 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nft_hash.ko 2024-03-08T20:34:26.167 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nft_limit.ko 2024-03-08T20:34:26.167 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nft_log.ko 2024-03-08T20:34:26.167 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nft_masq.ko 2024-03-08T20:34:26.167 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nft_nat.ko 2024-03-08T20:34:26.168 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nft_numgen.ko 2024-03-08T20:34:26.168 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nft_queue.ko 2024-03-08T20:34:26.168 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nft_quota.ko 2024-03-08T20:34:26.168 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nft_redir.ko 2024-03-08T20:34:26.168 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nft_reject.ko 2024-03-08T20:34:26.168 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nft_reject_inet.ko 2024-03-08T20:34:26.168 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nft_socket.ko 2024-03-08T20:34:26.168 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nft_tunnel.ko 2024-03-08T20:34:26.168 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_AUDIT.ko 2024-03-08T20:34:26.168 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_CHECKSUM.ko 2024-03-08T20:34:26.168 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_CLASSIFY.ko 2024-03-08T20:34:26.168 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-03-08T20:34:26.168 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_CT.ko 2024-03-08T20:34:26.168 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_DSCP.ko 2024-03-08T20:34:26.169 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_HL.ko 2024-03-08T20:34:26.169 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_HMARK.ko 2024-03-08T20:34:26.169 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_IDLETIMER.ko 2024-03-08T20:34:26.169 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_LED.ko 2024-03-08T20:34:26.169 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_LOG.ko 2024-03-08T20:34:26.169 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_MASQUERADE.ko 2024-03-08T20:34:26.169 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_NETMAP.ko 2024-03-08T20:34:26.169 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_NFLOG.ko 2024-03-08T20:34:26.169 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_NFQUEUE.ko 2024-03-08T20:34:26.169 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_RATEEST.ko 2024-03-08T20:34:26.169 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_REDIRECT.ko 2024-03-08T20:34:26.169 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_SECMARK.ko 2024-03-08T20:34:26.169 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_TCPMSS.ko 2024-03-08T20:34:26.170 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-03-08T20:34:26.170 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_TEE.ko 2024-03-08T20:34:26.170 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_TPROXY.ko 2024-03-08T20:34:26.170 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_TRACE.ko 2024-03-08T20:34:26.170 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_addrtype.ko 2024-03-08T20:34:26.170 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_bpf.ko 2024-03-08T20:34:26.170 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_cgroup.ko 2024-03-08T20:34:26.170 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_cluster.ko 2024-03-08T20:34:26.170 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_comment.ko 2024-03-08T20:34:26.170 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_connbytes.ko 2024-03-08T20:34:26.170 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_connlabel.ko 2024-03-08T20:34:26.170 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_connlimit.ko 2024-03-08T20:34:26.170 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_connmark.ko 2024-03-08T20:34:26.170 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_conntrack.ko 2024-03-08T20:34:26.171 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_cpu.ko 2024-03-08T20:34:26.171 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_dccp.ko 2024-03-08T20:34:26.171 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_devgroup.ko 2024-03-08T20:34:26.171 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_dscp.ko 2024-03-08T20:34:26.171 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_ecn.ko 2024-03-08T20:34:26.171 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_esp.ko 2024-03-08T20:34:26.171 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_hashlimit.ko 2024-03-08T20:34:26.171 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_helper.ko 2024-03-08T20:34:26.171 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_hl.ko 2024-03-08T20:34:26.171 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_ipcomp.ko 2024-03-08T20:34:26.171 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_iprange.ko 2024-03-08T20:34:26.171 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_ipvs.ko 2024-03-08T20:34:26.171 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_l2tp.ko 2024-03-08T20:34:26.171 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_length.ko 2024-03-08T20:34:26.172 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_limit.ko 2024-03-08T20:34:26.172 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_mac.ko 2024-03-08T20:34:26.172 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_mark.ko 2024-03-08T20:34:26.172 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_multiport.ko 2024-03-08T20:34:26.172 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_nat.ko 2024-03-08T20:34:26.172 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_nfacct.ko 2024-03-08T20:34:26.172 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_osf.ko 2024-03-08T20:34:26.172 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_owner.ko 2024-03-08T20:34:26.172 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_physdev.ko 2024-03-08T20:34:26.172 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_pkttype.ko 2024-03-08T20:34:26.172 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_policy.ko 2024-03-08T20:34:26.172 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_quota.ko 2024-03-08T20:34:26.172 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_rateest.ko 2024-03-08T20:34:26.172 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_realm.ko 2024-03-08T20:34:26.172 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_recent.ko 2024-03-08T20:34:26.173 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_sctp.ko 2024-03-08T20:34:26.173 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_set.ko 2024-03-08T20:34:26.173 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_socket.ko 2024-03-08T20:34:26.173 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_state.ko 2024-03-08T20:34:26.173 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_statistic.ko 2024-03-08T20:34:26.173 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_string.ko 2024-03-08T20:34:26.173 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_tcpmss.ko 2024-03-08T20:34:26.173 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_time.ko 2024-03-08T20:34:26.173 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_u32.ko 2024-03-08T20:34:26.173 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netlink 2024-03-08T20:34:26.173 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netlink/netlink_diag.ko 2024-03-08T20:34:26.173 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/nsh 2024-03-08T20:34:26.173 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/nsh/nsh.ko 2024-03-08T20:34:26.173 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/openvswitch 2024-03-08T20:34:26.174 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/openvswitch/openvswitch.ko 2024-03-08T20:34:26.174 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/openvswitch/vport-geneve.ko 2024-03-08T20:34:26.174 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/openvswitch/vport-gre.ko 2024-03-08T20:34:26.174 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/openvswitch/vport-vxlan.ko 2024-03-08T20:34:26.174 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/packet 2024-03-08T20:34:26.174 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/packet/af_packet_diag.ko 2024-03-08T20:34:26.174 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/rds 2024-03-08T20:34:26.174 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/rds/rds.ko 2024-03-08T20:34:26.174 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/rds/rds_rdma.ko 2024-03-08T20:34:26.174 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/rds/rds_tcp.ko 2024-03-08T20:34:26.174 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched 2024-03-08T20:34:26.174 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/act_csum.ko 2024-03-08T20:34:26.174 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/act_gact.ko 2024-03-08T20:34:26.174 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/act_mirred.ko 2024-03-08T20:34:26.175 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/act_nat.ko 2024-03-08T20:34:26.175 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/act_pedit.ko 2024-03-08T20:34:26.175 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/act_police.ko 2024-03-08T20:34:26.175 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/act_simple.ko 2024-03-08T20:34:26.175 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/act_skbedit.ko 2024-03-08T20:34:26.175 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/act_vlan.ko 2024-03-08T20:34:26.175 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/cls_basic.ko 2024-03-08T20:34:26.175 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/cls_bpf.ko 2024-03-08T20:34:26.175 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/cls_flow.ko 2024-03-08T20:34:26.175 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/cls_fw.ko 2024-03-08T20:34:26.175 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/cls_route.ko 2024-03-08T20:34:26.175 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/cls_u32.ko 2024-03-08T20:34:26.175 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/em_cmp.ko 2024-03-08T20:34:26.175 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/em_ipset.ko 2024-03-08T20:34:26.176 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/em_meta.ko 2024-03-08T20:34:26.176 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/em_nbyte.ko 2024-03-08T20:34:26.176 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/em_text.ko 2024-03-08T20:34:26.176 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/em_u32.ko 2024-03-08T20:34:26.176 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_choke.ko 2024-03-08T20:34:26.176 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_codel.ko 2024-03-08T20:34:26.176 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_drr.ko 2024-03-08T20:34:26.176 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_fq.ko 2024-03-08T20:34:26.176 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_gred.ko 2024-03-08T20:34:26.176 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_hfsc.ko 2024-03-08T20:34:26.176 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_hhf.ko 2024-03-08T20:34:26.176 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_htb.ko 2024-03-08T20:34:26.176 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_ingress.ko 2024-03-08T20:34:26.176 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_mqprio.ko 2024-03-08T20:34:26.177 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_mqprio_lib.ko 2024-03-08T20:34:26.177 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_multiq.ko 2024-03-08T20:34:26.177 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_netem.ko 2024-03-08T20:34:26.177 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_pie.ko 2024-03-08T20:34:26.177 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_plug.ko 2024-03-08T20:34:26.177 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_prio.ko 2024-03-08T20:34:26.177 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_qfq.ko 2024-03-08T20:34:26.177 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_red.ko 2024-03-08T20:34:26.177 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_sfb.ko 2024-03-08T20:34:26.177 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_sfq.ko 2024-03-08T20:34:26.177 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_tbf.ko 2024-03-08T20:34:26.177 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_teql.ko 2024-03-08T20:34:26.177 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sctp 2024-03-08T20:34:26.177 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sctp/sctp.ko 2024-03-08T20:34:26.178 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sctp/sctp_diag.ko 2024-03-08T20:34:26.178 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sunrpc 2024-03-08T20:34:26.178 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sunrpc/auth_gss 2024-03-08T20:34:26.178 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-03-08T20:34:26.178 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-03-08T20:34:26.178 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sunrpc/sunrpc.ko 2024-03-08T20:34:26.178 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sunrpc/xprtrdma 2024-03-08T20:34:26.178 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-03-08T20:34:26.178 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/tipc 2024-03-08T20:34:26.178 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/tipc/diag.ko 2024-03-08T20:34:26.178 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/tipc/tipc.ko 2024-03-08T20:34:26.178 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/unix 2024-03-08T20:34:26.178 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/unix/unix_diag.ko 2024-03-08T20:34:26.178 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/vmw_vsock 2024-03-08T20:34:26.179 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-03-08T20:34:26.179 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-03-08T20:34:26.179 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/vmw_vsock/vsock.ko 2024-03-08T20:34:26.179 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/vmw_vsock/vsock_diag.ko 2024-03-08T20:34:26.179 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/vmw_vsock/vsock_loopback.ko 2024-03-08T20:34:26.179 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/xfrm 2024-03-08T20:34:26.179 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/xfrm/xfrm_ipcomp.ko 2024-03-08T20:34:26.179 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/security 2024-03-08T20:34:26.179 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/security/keys 2024-03-08T20:34:26.179 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/security/keys/encrypted-keys 2024-03-08T20:34:26.179 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-03-08T20:34:26.179 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/security/keys/trusted-keys 2024-03-08T20:34:26.179 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/security/keys/trusted-keys/trusted.ko 2024-03-08T20:34:26.179 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/virt 2024-03-08T20:34:26.180 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/virt/lib 2024-03-08T20:34:26.180 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/virt/lib/irqbypass.ko 2024-03-08T20:34:26.180 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/modules.alias 2024-03-08T20:34:26.180 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/modules.alias.bin 2024-03-08T20:34:26.180 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/modules.builtin 2024-03-08T20:34:26.180 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/modules.builtin.alias.bin 2024-03-08T20:34:26.180 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/modules.builtin.bin 2024-03-08T20:34:26.180 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/modules.builtin.modinfo 2024-03-08T20:34:26.180 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/modules.dep 2024-03-08T20:34:26.180 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/modules.dep.bin 2024-03-08T20:34:26.180 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/modules.devname 2024-03-08T20:34:26.180 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/modules.order 2024-03-08T20:34:26.180 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/modules.softdep 2024-03-08T20:34:26.180 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/modules.symbols 2024-03-08T20:34:26.181 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/modules.symbols.bin 2024-03-08T20:34:26.181 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/vmlinuz 2024-03-08T20:34:26.184 DEBUG:teuthology.task.kernel:get_image_version: 6.8.0-rc7-gdd5c1cd17167 2024-03-08T20:34:26.184 DEBUG:teuthology.orchestra.run.smithi107:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-03-08T20:34:26.206 INFO:teuthology.orchestra.run.smithi107.stdout:postinstall scriptlet (using /bin/sh): 2024-03-08T20:34:26.206 INFO:teuthology.orchestra.run.smithi107.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-03-08T20:34:26.206 INFO:teuthology.orchestra.run.smithi107.stdout: /usr/bin/kernel-install add 6.8.0-rc7-gdd5c1cd17167 /lib/modules/6.8.0-rc7-gdd5c1cd17167/vmlinuz 2024-03-08T20:34:26.206 INFO:teuthology.orchestra.run.smithi107.stdout:fi 2024-03-08T20:34:26.206 INFO:teuthology.orchestra.run.smithi107.stdout:for file in vmlinuz System.map config; do 2024-03-08T20:34:26.209 INFO:teuthology.orchestra.run.smithi107.stdout: if ! cmp --silent "/lib/modules/6.8.0-rc7-gdd5c1cd17167/${file}" "/boot/${file}-6.8.0-rc7-gdd5c1cd17167"; then 2024-03-08T20:34:26.209 INFO:teuthology.orchestra.run.smithi107.stdout: cp "/lib/modules/6.8.0-rc7-gdd5c1cd17167/${file}" "/boot/${file}-6.8.0-rc7-gdd5c1cd17167" 2024-03-08T20:34:26.209 INFO:teuthology.orchestra.run.smithi107.stdout: fi 2024-03-08T20:34:26.209 INFO:teuthology.orchestra.run.smithi107.stdout:done 2024-03-08T20:34:26.209 INFO:teuthology.orchestra.run.smithi107.stdout:preuninstall scriptlet (using /bin/sh): 2024-03-08T20:34:26.209 INFO:teuthology.orchestra.run.smithi107.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2024-03-08T20:34:26.209 INFO:teuthology.orchestra.run.smithi107.stdout:new-kernel-pkg --remove 6.8.0-rc7-gdd5c1cd17167 --rminitrd --initrdfile=/boot/initramfs-6.8.0-rc7-gdd5c1cd17167.img 2024-03-08T20:34:26.209 INFO:teuthology.orchestra.run.smithi107.stdout:elif [ -x /usr/bin/kernel-install ]; then 2024-03-08T20:34:26.209 INFO:teuthology.orchestra.run.smithi107.stdout:kernel-install remove 6.8.0-rc7-gdd5c1cd17167 2024-03-08T20:34:26.209 INFO:teuthology.orchestra.run.smithi107.stdout:fi 2024-03-08T20:34:26.210 INFO:teuthology.orchestra.run.smithi107.stdout:postuninstall scriptlet (using /bin/sh): 2024-03-08T20:34:26.210 INFO:teuthology.orchestra.run.smithi107.stdout:if [ -x /sbin/update-bootloader ]; then 2024-03-08T20:34:26.210 INFO:teuthology.orchestra.run.smithi107.stdout:/sbin/update-bootloader --remove 6.8.0-rc7-gdd5c1cd17167 2024-03-08T20:34:26.210 INFO:teuthology.orchestra.run.smithi107.stdout:fi 2024-03-08T20:34:26.211 DEBUG:teuthology.orchestra.run.smithi107:> sudo rpm -qi grub2-tools 2024-03-08T20:34:26.284 INFO:teuthology.orchestra.run.smithi107.stdout:Name : grub2-tools 2024-03-08T20:34:26.284 INFO:teuthology.orchestra.run.smithi107.stdout:Epoch : 1 2024-03-08T20:34:26.284 INFO:teuthology.orchestra.run.smithi107.stdout:Version : 2.06 2024-03-08T20:34:26.284 INFO:teuthology.orchestra.run.smithi107.stdout:Release : 68.el9 2024-03-08T20:34:26.284 INFO:teuthology.orchestra.run.smithi107.stdout:Architecture: x86_64 2024-03-08T20:34:26.285 INFO:teuthology.orchestra.run.smithi107.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-03-08T20:34:26.285 INFO:teuthology.orchestra.run.smithi107.stdout:Group : Unspecified 2024-03-08T20:34:26.285 INFO:teuthology.orchestra.run.smithi107.stdout:Size : 8264254 2024-03-08T20:34:26.285 INFO:teuthology.orchestra.run.smithi107.stdout:License : GPLv3+ 2024-03-08T20:34:26.285 INFO:teuthology.orchestra.run.smithi107.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-03-08T20:34:26.285 INFO:teuthology.orchestra.run.smithi107.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-03-08T20:34:26.285 INFO:teuthology.orchestra.run.smithi107.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-03-08T20:34:26.285 INFO:teuthology.orchestra.run.smithi107.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-03-08T20:34:26.285 INFO:teuthology.orchestra.run.smithi107.stdout:Packager : builder@centos.org 2024-03-08T20:34:26.285 INFO:teuthology.orchestra.run.smithi107.stdout:Vendor : CentOS 2024-03-08T20:34:26.285 INFO:teuthology.orchestra.run.smithi107.stdout:URL : http://www.gnu.org/software/grub/ 2024-03-08T20:34:26.285 INFO:teuthology.orchestra.run.smithi107.stdout:Summary : Support tools for GRUB. 2024-03-08T20:34:26.285 INFO:teuthology.orchestra.run.smithi107.stdout:Description : 2024-03-08T20:34:26.285 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-08T20:34:26.285 INFO:teuthology.orchestra.run.smithi107.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-03-08T20:34:26.285 INFO:teuthology.orchestra.run.smithi107.stdout:customizable bootloader with modular architecture. It supports a rich 2024-03-08T20:34:26.286 INFO:teuthology.orchestra.run.smithi107.stdout:variety of kernel formats, file systems, computer architectures and 2024-03-08T20:34:26.286 INFO:teuthology.orchestra.run.smithi107.stdout:hardware devices. 2024-03-08T20:34:26.286 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-08T20:34:26.286 INFO:teuthology.orchestra.run.smithi107.stdout:This subpackage provides tools for support of all platforms. 2024-03-08T20:34:26.287 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-03-08T20:34:26.287 INFO:teuthology.task.kernel:Updating grub on smithi107 to boot 6.8.0-rc7-gdd5c1cd17167 2024-03-08T20:34:26.287 DEBUG:teuthology.orchestra.run.smithi107:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-03-08T20:34:26.916 INFO:teuthology.orchestra.run.smithi107.stderr:Generating grub configuration file ... 2024-03-08T20:34:28.348 INFO:teuthology.orchestra.run.smithi107.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-03-08T20:34:28.379 INFO:teuthology.orchestra.run.smithi107.stderr:done 2024-03-08T20:34:28.381 DEBUG:teuthology.orchestra.run.smithi107:> mktemp 2024-03-08T20:34:28.397 INFO:teuthology.orchestra.run.smithi107.stdout:/tmp/tmp.66ILxWaY0d 2024-03-08T20:34:28.397 DEBUG:teuthology.orchestra.run.smithi107:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.66ILxWaY0d 2024-03-08T20:34:28.465 DEBUG:teuthology.orchestra.run.smithi107:> sudo chmod 0666 /tmp/tmp.66ILxWaY0d 2024-03-08T20:34:28.616 DEBUG:teuthology.orchestra.remote:smithi107:/tmp/tmp.66ILxWaY0d is 6KB 2024-03-08T20:34:28.644 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi149.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-03-08T20:34:28.644 DEBUG:teuthology.orchestra.run.smithi149:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-03-08T20:34:28.665 DEBUG:teuthology.orchestra.run.smithi107:> rm -fr /tmp/tmp.66ILxWaY0d 2024-03-08T20:34:28.679 DEBUG:teuthology.orchestra.run.smithi107:> sudo /bin/ls /boot/loader/entries || true 2024-03-08T20:34:28.718 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167 2024-03-08T20:34:28.718 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/System.map 2024-03-08T20:34:28.718 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/config 2024-03-08T20:34:28.718 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel 2024-03-08T20:34:28.718 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch 2024-03-08T20:34:28.718 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86 2024-03-08T20:34:28.718 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto 2024-03-08T20:34:28.718 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-03-08T20:34:28.718 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-03-08T20:34:28.719 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-03-08T20:34:28.719 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-03-08T20:34:28.719 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-03-08T20:34:28.719 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-03-08T20:34:28.719 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-03-08T20:34:28.719 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/crc32c-intel.ko 2024-03-08T20:34:28.719 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-03-08T20:34:28.719 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-03-08T20:34:28.719 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-03-08T20:34:28.719 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-03-08T20:34:28.719 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/serpent-avx2.ko 2024-03-08T20:34:28.719 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-03-08T20:34:28.719 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-03-08T20:34:28.719 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-03-08T20:34:28.720 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-03-08T20:34:28.720 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-03-08T20:34:28.720 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-03-08T20:34:28.720 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-03-08T20:34:28.720 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/kernel 2024-03-08T20:34:28.720 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/kernel/cpu 2024-03-08T20:34:28.720 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/kernel/cpu/mce 2024-03-08T20:34:28.720 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-03-08T20:34:28.720 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/kvm 2024-03-08T20:34:28.720 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/kvm/kvm-amd.ko 2024-03-08T20:34:28.720 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/kvm/kvm-intel.ko 2024-03-08T20:34:28.720 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/kvm/kvm.ko 2024-03-08T20:34:28.720 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto 2024-03-08T20:34:28.720 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/ansi_cprng.ko 2024-03-08T20:34:28.721 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/anubis.ko 2024-03-08T20:34:28.721 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/arc4.ko 2024-03-08T20:34:28.721 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/async_tx 2024-03-08T20:34:28.721 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/async_tx/async_memcpy.ko 2024-03-08T20:34:28.721 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/async_tx/async_pq.ko 2024-03-08T20:34:28.721 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/async_tx/async_raid6_recov.ko 2024-03-08T20:34:28.721 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/async_tx/async_tx.ko 2024-03-08T20:34:28.721 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/async_tx/async_xor.ko 2024-03-08T20:34:28.721 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/async_tx/raid6test.ko 2024-03-08T20:34:28.721 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/authenc.ko 2024-03-08T20:34:28.721 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/authencesn.ko 2024-03-08T20:34:28.721 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/blake2b_generic.ko 2024-03-08T20:34:28.721 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/blowfish_common.ko 2024-03-08T20:34:28.721 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/blowfish_generic.ko 2024-03-08T20:34:28.722 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/camellia_generic.ko 2024-03-08T20:34:28.722 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/cast5_generic.ko 2024-03-08T20:34:28.722 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/cast6_generic.ko 2024-03-08T20:34:28.722 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/cast_common.ko 2024-03-08T20:34:28.722 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/ccm.ko 2024-03-08T20:34:28.722 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/cmac.ko 2024-03-08T20:34:28.722 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/crc32_generic.ko 2024-03-08T20:34:28.722 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/crypto_engine.ko 2024-03-08T20:34:28.722 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/crypto_user.ko 2024-03-08T20:34:28.722 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/cts.ko 2024-03-08T20:34:28.722 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/des_generic.ko 2024-03-08T20:34:28.722 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/dh_generic.ko 2024-03-08T20:34:28.722 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/echainiv.ko 2024-03-08T20:34:28.722 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/essiv.ko 2024-03-08T20:34:28.723 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/fcrypt.ko 2024-03-08T20:34:28.723 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/khazad.ko 2024-03-08T20:34:28.723 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/lz4.ko 2024-03-08T20:34:28.723 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/lz4hc.ko 2024-03-08T20:34:28.723 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/md4.ko 2024-03-08T20:34:28.723 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/michael_mic.ko 2024-03-08T20:34:28.723 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/pcbc.ko 2024-03-08T20:34:28.723 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/pcrypt.ko 2024-03-08T20:34:28.723 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/rmd160.ko 2024-03-08T20:34:28.723 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/seed.ko 2024-03-08T20:34:28.723 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/serpent_generic.ko 2024-03-08T20:34:28.723 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/tcrypt.ko 2024-03-08T20:34:28.723 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/tea.ko 2024-03-08T20:34:28.723 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/twofish_common.ko 2024-03-08T20:34:28.724 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/twofish_generic.ko 2024-03-08T20:34:28.724 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/vmac.ko 2024-03-08T20:34:28.724 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/wp512.ko 2024-03-08T20:34:28.724 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/xcbc.ko 2024-03-08T20:34:28.724 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/xor.ko 2024-03-08T20:34:28.724 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/xxhash_generic.ko 2024-03-08T20:34:28.724 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers 2024-03-08T20:34:28.724 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/acpi 2024-03-08T20:34:28.724 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/acpi/acpi_ipmi.ko 2024-03-08T20:34:28.724 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/acpi/acpi_pad.ko 2024-03-08T20:34:28.724 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/acpi/custom_method.ko 2024-03-08T20:34:28.724 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/acpi/ec_sys.ko 2024-03-08T20:34:28.724 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/acpi/platform_profile.ko 2024-03-08T20:34:28.724 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/acpi/sbs.ko 2024-03-08T20:34:28.725 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/acpi/sbshc.ko 2024-03-08T20:34:28.725 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/acpi/video.ko 2024-03-08T20:34:28.725 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata 2024-03-08T20:34:28.725 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/acard-ahci.ko 2024-03-08T20:34:28.725 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/ahci_platform.ko 2024-03-08T20:34:28.725 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/ata_generic.ko 2024-03-08T20:34:28.725 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/libahci_platform.ko 2024-03-08T20:34:28.725 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_acpi.ko 2024-03-08T20:34:28.725 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_ali.ko 2024-03-08T20:34:28.725 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_amd.ko 2024-03-08T20:34:28.725 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_artop.ko 2024-03-08T20:34:28.725 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_atiixp.ko 2024-03-08T20:34:28.725 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_atp867x.ko 2024-03-08T20:34:28.725 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_cmd640.ko 2024-03-08T20:34:28.726 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_cmd64x.ko 2024-03-08T20:34:28.726 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_cypress.ko 2024-03-08T20:34:28.726 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_efar.ko 2024-03-08T20:34:28.726 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_hpt366.ko 2024-03-08T20:34:28.726 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_hpt37x.ko 2024-03-08T20:34:28.726 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_hpt3x2n.ko 2024-03-08T20:34:28.726 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_hpt3x3.ko 2024-03-08T20:34:28.726 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_it8213.ko 2024-03-08T20:34:28.726 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_it821x.ko 2024-03-08T20:34:28.726 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_jmicron.ko 2024-03-08T20:34:28.726 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_marvell.ko 2024-03-08T20:34:28.726 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_mpiix.ko 2024-03-08T20:34:28.726 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_netcell.ko 2024-03-08T20:34:28.727 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_ninja32.ko 2024-03-08T20:34:28.727 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_ns87410.ko 2024-03-08T20:34:28.727 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_ns87415.ko 2024-03-08T20:34:28.727 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_oldpiix.ko 2024-03-08T20:34:28.727 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_opti.ko 2024-03-08T20:34:28.727 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_optidma.ko 2024-03-08T20:34:28.727 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_pcmcia.ko 2024-03-08T20:34:28.727 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_pdc2027x.ko 2024-03-08T20:34:28.727 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-03-08T20:34:28.727 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_piccolo.ko 2024-03-08T20:34:28.727 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_rdc.ko 2024-03-08T20:34:28.727 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_sch.ko 2024-03-08T20:34:28.727 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_serverworks.ko 2024-03-08T20:34:28.727 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_sil680.ko 2024-03-08T20:34:28.728 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_sis.ko 2024-03-08T20:34:28.728 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_sl82c105.ko 2024-03-08T20:34:28.728 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_triflex.ko 2024-03-08T20:34:28.728 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_via.ko 2024-03-08T20:34:28.728 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pdc_adma.ko 2024-03-08T20:34:28.728 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/sata_inic162x.ko 2024-03-08T20:34:28.728 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/sata_mv.ko 2024-03-08T20:34:28.728 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/sata_nv.ko 2024-03-08T20:34:28.728 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/sata_promise.ko 2024-03-08T20:34:28.728 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/sata_qstor.ko 2024-03-08T20:34:28.728 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/sata_sil.ko 2024-03-08T20:34:28.728 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/sata_sil24.ko 2024-03-08T20:34:28.728 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/sata_sis.ko 2024-03-08T20:34:28.729 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/sata_svw.ko 2024-03-08T20:34:28.729 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/sata_sx4.ko 2024-03-08T20:34:28.729 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/sata_uli.ko 2024-03-08T20:34:28.729 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/sata_via.ko 2024-03-08T20:34:28.729 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/sata_vsc.ko 2024-03-08T20:34:28.729 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/auxdisplay 2024-03-08T20:34:28.729 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/auxdisplay/ks0108.ko 2024-03-08T20:34:28.729 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/base 2024-03-08T20:34:28.729 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/base/regmap 2024-03-08T20:34:28.729 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/base/regmap/regmap-i2c.ko 2024-03-08T20:34:28.729 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/base/regmap/regmap-mmio.ko 2024-03-08T20:34:28.730 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/bcma 2024-03-08T20:34:28.730 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/bcma/bcma.ko 2024-03-08T20:34:28.730 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block 2024-03-08T20:34:28.730 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/aoe 2024-03-08T20:34:28.730 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/aoe/aoe.ko 2024-03-08T20:34:28.730 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/brd.ko 2024-03-08T20:34:28.731 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/drbd 2024-03-08T20:34:28.731 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/drbd/drbd.ko 2024-03-08T20:34:28.731 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/floppy.ko 2024-03-08T20:34:28.731 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/loop.ko 2024-03-08T20:34:28.731 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/mtip32xx 2024-03-08T20:34:28.731 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-03-08T20:34:28.731 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/nbd.ko 2024-03-08T20:34:28.731 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/null_blk 2024-03-08T20:34:28.731 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/null_blk/null_blk.ko 2024-03-08T20:34:28.731 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/pktcdvd.ko 2024-03-08T20:34:28.731 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/rbd.ko 2024-03-08T20:34:28.731 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/virtio_blk.ko 2024-03-08T20:34:28.731 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/xen-blkback 2024-03-08T20:34:28.732 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-03-08T20:34:28.732 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/xen-blkfront.ko 2024-03-08T20:34:28.732 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/zram 2024-03-08T20:34:28.732 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/zram/zram.ko 2024-03-08T20:34:28.732 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char 2024-03-08T20:34:28.732 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/hangcheck-timer.ko 2024-03-08T20:34:28.732 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/hw_random 2024-03-08T20:34:28.732 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/hw_random/amd-rng.ko 2024-03-08T20:34:28.732 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/hw_random/intel-rng.ko 2024-03-08T20:34:28.732 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-03-08T20:34:28.732 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/hw_random/via-rng.ko 2024-03-08T20:34:28.732 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/hw_random/virtio-rng.ko 2024-03-08T20:34:28.732 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/ipmi 2024-03-08T20:34:28.732 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-03-08T20:34:28.733 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-03-08T20:34:28.733 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-03-08T20:34:28.733 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/ipmi/ipmi_si.ko 2024-03-08T20:34:28.733 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-03-08T20:34:28.733 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-03-08T20:34:28.733 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/lp.ko 2024-03-08T20:34:28.733 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/mwave 2024-03-08T20:34:28.733 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/mwave/mwave.ko 2024-03-08T20:34:28.733 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/ppdev.ko 2024-03-08T20:34:28.733 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/tlclk.ko 2024-03-08T20:34:28.733 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/tpm 2024-03-08T20:34:28.733 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/tpm/tpm.ko 2024-03-08T20:34:28.733 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/tpm/tpm_atmel.ko 2024-03-08T20:34:28.733 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/tpm/tpm_infineon.ko 2024-03-08T20:34:28.734 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/tpm/tpm_nsc.ko 2024-03-08T20:34:28.734 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/tpm/tpm_tis.ko 2024-03-08T20:34:28.734 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-03-08T20:34:28.734 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/uv_mmtimer.ko 2024-03-08T20:34:28.734 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/virtio_console.ko 2024-03-08T20:34:28.734 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/cpufreq 2024-03-08T20:34:28.734 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-03-08T20:34:28.734 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-03-08T20:34:28.734 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/cpufreq/p4-clockmod.ko 2024-03-08T20:34:28.734 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-03-08T20:34:28.734 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/cpufreq/powernow-k8.ko 2024-03-08T20:34:28.734 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/cpufreq/speedstep-lib.ko 2024-03-08T20:34:28.735 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/crypto 2024-03-08T20:34:28.735 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/crypto/ccp 2024-03-08T20:34:28.735 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-03-08T20:34:28.735 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/crypto/ccp/ccp.ko 2024-03-08T20:34:28.735 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/crypto/intel 2024-03-08T20:34:28.735 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/crypto/intel/qat 2024-03-08T20:34:28.735 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/crypto/intel/qat/qat_common 2024-03-08T20:34:28.735 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-03-08T20:34:28.735 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-03-08T20:34:28.735 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-03-08T20:34:28.735 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/crypto/padlock-aes.ko 2024-03-08T20:34:28.735 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/crypto/padlock-sha.ko 2024-03-08T20:34:28.735 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/crypto/virtio 2024-03-08T20:34:28.735 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-03-08T20:34:28.735 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/dca 2024-03-08T20:34:28.736 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/dca/dca.ko 2024-03-08T20:34:28.736 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/devfreq 2024-03-08T20:34:28.736 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-03-08T20:34:28.736 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/dma 2024-03-08T20:34:28.736 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/dma/dw 2024-03-08T20:34:28.736 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/dma/dw/dw_dmac.ko 2024-03-08T20:34:28.736 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/dma/ioat 2024-03-08T20:34:28.736 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/dma/ioat/ioatdma.ko 2024-03-08T20:34:28.736 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/edac 2024-03-08T20:34:28.736 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/edac/amd64_edac.ko 2024-03-08T20:34:28.736 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/edac/e752x_edac.ko 2024-03-08T20:34:28.736 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/edac/edac_mce_amd.ko 2024-03-08T20:34:28.736 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/edac/i3000_edac.ko 2024-03-08T20:34:28.736 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/edac/i3200_edac.ko 2024-03-08T20:34:28.737 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/edac/i5100_edac.ko 2024-03-08T20:34:28.737 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/edac/i5400_edac.ko 2024-03-08T20:34:28.737 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/edac/i7300_edac.ko 2024-03-08T20:34:28.737 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/edac/i7core_edac.ko 2024-03-08T20:34:28.737 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/edac/i82975x_edac.ko 2024-03-08T20:34:28.737 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/edac/ie31200_edac.ko 2024-03-08T20:34:28.737 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/edac/sb_edac.ko 2024-03-08T20:34:28.737 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/edac/x38_edac.ko 2024-03-08T20:34:28.737 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/firmware 2024-03-08T20:34:28.737 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/firmware/edd.ko 2024-03-08T20:34:28.737 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/firmware/iscsi_ibft.ko 2024-03-08T20:34:28.737 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/gpio 2024-03-08T20:34:28.738 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/gpio/gpio-ich.ko 2024-03-08T20:34:28.738 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/gpio/gpio-viperboard.ko 2024-03-08T20:34:28.738 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid 2024-03-08T20:34:28.738 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-appleir.ko 2024-03-08T20:34:28.738 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-aureal.ko 2024-03-08T20:34:28.738 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-axff.ko 2024-03-08T20:34:28.738 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-dr.ko 2024-03-08T20:34:28.738 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-elecom.ko 2024-03-08T20:34:28.738 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-elo.ko 2024-03-08T20:34:28.738 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-emsff.ko 2024-03-08T20:34:28.738 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-gaff.ko 2024-03-08T20:34:28.739 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-gt683r.ko 2024-03-08T20:34:28.739 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-gyration.ko 2024-03-08T20:34:28.739 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-holtek-kbd.ko 2024-03-08T20:34:28.739 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-holtek-mouse.ko 2024-03-08T20:34:28.739 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-holtekff.ko 2024-03-08T20:34:28.739 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-hyperv.ko 2024-03-08T20:34:28.739 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-icade.ko 2024-03-08T20:34:28.739 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-keytouch.ko 2024-03-08T20:34:28.739 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-kye.ko 2024-03-08T20:34:28.739 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-lcpower.ko 2024-03-08T20:34:28.739 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-led.ko 2024-03-08T20:34:28.739 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-lenovo.ko 2024-03-08T20:34:28.739 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-logitech-dj.ko 2024-03-08T20:34:28.739 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-03-08T20:34:28.739 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-multitouch.ko 2024-03-08T20:34:28.740 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-ortek.ko 2024-03-08T20:34:28.740 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-penmount.ko 2024-03-08T20:34:28.740 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-petalynx.ko 2024-03-08T20:34:28.740 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-picolcd.ko 2024-03-08T20:34:28.740 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-pl.ko 2024-03-08T20:34:28.740 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-plantronics.ko 2024-03-08T20:34:28.740 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-primax.ko 2024-03-08T20:34:28.740 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-rmi.ko 2024-03-08T20:34:28.740 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-roccat-arvo.ko 2024-03-08T20:34:28.740 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-roccat-common.ko 2024-03-08T20:34:28.740 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-roccat-isku.ko 2024-03-08T20:34:28.740 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-roccat-kone.ko 2024-03-08T20:34:28.740 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-03-08T20:34:28.740 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-roccat-konepure.ko 2024-03-08T20:34:28.741 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-03-08T20:34:28.741 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-roccat-lua.ko 2024-03-08T20:34:28.741 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-roccat-pyra.ko 2024-03-08T20:34:28.741 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-roccat-ryos.ko 2024-03-08T20:34:28.741 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-roccat-savu.ko 2024-03-08T20:34:28.741 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-roccat.ko 2024-03-08T20:34:28.741 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-saitek.ko 2024-03-08T20:34:28.741 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-samsung.ko 2024-03-08T20:34:28.741 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-sensor-hub.ko 2024-03-08T20:34:28.741 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-sjoy.ko 2024-03-08T20:34:28.741 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-sony.ko 2024-03-08T20:34:28.741 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-speedlink.ko 2024-03-08T20:34:28.741 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-steelseries.ko 2024-03-08T20:34:28.741 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-sunplus.ko 2024-03-08T20:34:28.742 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-thrustmaster.ko 2024-03-08T20:34:28.742 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-tivo.ko 2024-03-08T20:34:28.742 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-tmff.ko 2024-03-08T20:34:28.742 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-topseed.ko 2024-03-08T20:34:28.742 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-twinhan.ko 2024-03-08T20:34:28.742 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-uclogic.ko 2024-03-08T20:34:28.742 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-waltop.ko 2024-03-08T20:34:28.742 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-wiimote.ko 2024-03-08T20:34:28.742 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-xinmo.ko 2024-03-08T20:34:28.742 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-zpff.ko 2024-03-08T20:34:28.742 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-zydacron.ko 2024-03-08T20:34:28.742 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/uhid.ko 2024-03-08T20:34:28.742 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/wacom.ko 2024-03-08T20:34:28.743 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hv 2024-03-08T20:34:28.743 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hv/hv_balloon.ko 2024-03-08T20:34:28.743 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hv/hv_utils.ko 2024-03-08T20:34:28.743 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hv/hv_vmbus.ko 2024-03-08T20:34:28.743 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon 2024-03-08T20:34:28.743 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/abituguru.ko 2024-03-08T20:34:28.743 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/abituguru3.ko 2024-03-08T20:34:28.743 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/acpi_power_meter.ko 2024-03-08T20:34:28.743 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/ad7414.ko 2024-03-08T20:34:28.743 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/ad7418.ko 2024-03-08T20:34:28.743 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/adc128d818.ko 2024-03-08T20:34:28.743 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/adm1025.ko 2024-03-08T20:34:28.743 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/adm1026.ko 2024-03-08T20:34:28.744 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/adm1029.ko 2024-03-08T20:34:28.744 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/adm1031.ko 2024-03-08T20:34:28.744 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/adm9240.ko 2024-03-08T20:34:28.744 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/ads7828.ko 2024-03-08T20:34:28.744 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/adt7410.ko 2024-03-08T20:34:28.744 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/adt7411.ko 2024-03-08T20:34:28.744 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/adt7462.ko 2024-03-08T20:34:28.744 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/adt7470.ko 2024-03-08T20:34:28.744 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/adt7475.ko 2024-03-08T20:34:28.744 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/adt7x10.ko 2024-03-08T20:34:28.744 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/amc6821.ko 2024-03-08T20:34:28.744 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/applesmc.ko 2024-03-08T20:34:28.744 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/asb100.ko 2024-03-08T20:34:28.744 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/asc7621.ko 2024-03-08T20:34:28.745 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/asus_atk0110.ko 2024-03-08T20:34:28.745 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/atxp1.ko 2024-03-08T20:34:28.745 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/coretemp.ko 2024-03-08T20:34:28.745 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-03-08T20:34:28.745 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/dme1737.ko 2024-03-08T20:34:28.745 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/ds1621.ko 2024-03-08T20:34:28.745 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/ds620.ko 2024-03-08T20:34:28.745 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/emc1403.ko 2024-03-08T20:34:28.745 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/emc6w201.ko 2024-03-08T20:34:28.745 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/f71805f.ko 2024-03-08T20:34:28.745 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/f71882fg.ko 2024-03-08T20:34:28.745 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/f75375s.ko 2024-03-08T20:34:28.745 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/fam15h_power.ko 2024-03-08T20:34:28.745 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/fschmd.ko 2024-03-08T20:34:28.746 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/g760a.ko 2024-03-08T20:34:28.746 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/g762.ko 2024-03-08T20:34:28.746 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/gl518sm.ko 2024-03-08T20:34:28.746 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/gl520sm.ko 2024-03-08T20:34:28.746 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/hwmon-vid.ko 2024-03-08T20:34:28.746 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/i5500_temp.ko 2024-03-08T20:34:28.746 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/i5k_amb.ko 2024-03-08T20:34:28.746 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/ibmaem.ko 2024-03-08T20:34:28.746 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/ibmpex.ko 2024-03-08T20:34:28.746 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/ina209.ko 2024-03-08T20:34:28.746 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/ina2xx.ko 2024-03-08T20:34:28.746 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/it87.ko 2024-03-08T20:34:28.746 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/k10temp.ko 2024-03-08T20:34:28.747 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/k8temp.ko 2024-03-08T20:34:28.747 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/lineage-pem.ko 2024-03-08T20:34:28.747 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/lm63.ko 2024-03-08T20:34:28.747 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/lm73.ko 2024-03-08T20:34:28.747 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/lm75.ko 2024-03-08T20:34:28.747 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/lm77.ko 2024-03-08T20:34:28.747 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/lm78.ko 2024-03-08T20:34:28.747 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/lm80.ko 2024-03-08T20:34:28.747 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/lm83.ko 2024-03-08T20:34:28.747 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/lm85.ko 2024-03-08T20:34:28.747 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/lm87.ko 2024-03-08T20:34:28.747 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/lm90.ko 2024-03-08T20:34:28.747 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/lm92.ko 2024-03-08T20:34:28.747 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/lm93.ko 2024-03-08T20:34:28.748 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/lm95234.ko 2024-03-08T20:34:28.748 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/lm95241.ko 2024-03-08T20:34:28.748 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/lm95245.ko 2024-03-08T20:34:28.748 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/ltc2945.ko 2024-03-08T20:34:28.748 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/ltc4151.ko 2024-03-08T20:34:28.748 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/ltc4215.ko 2024-03-08T20:34:28.748 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/ltc4222.ko 2024-03-08T20:34:28.748 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/ltc4245.ko 2024-03-08T20:34:28.748 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/ltc4260.ko 2024-03-08T20:34:28.748 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/ltc4261.ko 2024-03-08T20:34:28.748 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/max16065.ko 2024-03-08T20:34:28.748 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/max1619.ko 2024-03-08T20:34:28.748 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/max1668.ko 2024-03-08T20:34:28.748 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/max197.ko 2024-03-08T20:34:28.749 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/max6639.ko 2024-03-08T20:34:28.749 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/max6650.ko 2024-03-08T20:34:28.749 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/max6697.ko 2024-03-08T20:34:28.749 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/mcp3021.ko 2024-03-08T20:34:28.749 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/nct6683.ko 2024-03-08T20:34:28.749 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/nct6775-core.ko 2024-03-08T20:34:28.749 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/nct6775.ko 2024-03-08T20:34:28.749 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/nct7802.ko 2024-03-08T20:34:28.749 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/pc87360.ko 2024-03-08T20:34:28.749 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/pc87427.ko 2024-03-08T20:34:28.749 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/pcf8591.ko 2024-03-08T20:34:28.749 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/pmbus 2024-03-08T20:34:28.749 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-03-08T20:34:28.749 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-03-08T20:34:28.750 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-03-08T20:34:28.750 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/pmbus/max16064.ko 2024-03-08T20:34:28.750 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/pmbus/max34440.ko 2024-03-08T20:34:28.750 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/pmbus/max8688.ko 2024-03-08T20:34:28.750 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-03-08T20:34:28.750 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-03-08T20:34:28.750 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-03-08T20:34:28.750 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-03-08T20:34:28.750 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-03-08T20:34:28.750 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-03-08T20:34:28.750 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/powr1220.ko 2024-03-08T20:34:28.750 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/sch5627.ko 2024-03-08T20:34:28.750 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/sch5636.ko 2024-03-08T20:34:28.750 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/sch56xx-common.ko 2024-03-08T20:34:28.751 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/sht15.ko 2024-03-08T20:34:28.751 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/sht21.ko 2024-03-08T20:34:28.751 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/shtc1.ko 2024-03-08T20:34:28.751 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/sis5595.ko 2024-03-08T20:34:28.751 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/smsc47b397.ko 2024-03-08T20:34:28.751 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/smsc47m1.ko 2024-03-08T20:34:28.751 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/smsc47m192.ko 2024-03-08T20:34:28.751 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/thmc50.ko 2024-03-08T20:34:28.751 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/tmp102.ko 2024-03-08T20:34:28.751 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/tmp103.ko 2024-03-08T20:34:28.751 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/tmp401.ko 2024-03-08T20:34:28.751 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/tmp421.ko 2024-03-08T20:34:28.751 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/via-cputemp.ko 2024-03-08T20:34:28.752 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/via686a.ko 2024-03-08T20:34:28.752 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/vt1211.ko 2024-03-08T20:34:28.752 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/vt8231.ko 2024-03-08T20:34:28.752 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/w83627ehf.ko 2024-03-08T20:34:28.752 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/w83627hf.ko 2024-03-08T20:34:28.752 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/w83781d.ko 2024-03-08T20:34:28.752 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/w83791d.ko 2024-03-08T20:34:28.752 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/w83792d.ko 2024-03-08T20:34:28.752 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/w83793.ko 2024-03-08T20:34:28.753 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/w83795.ko 2024-03-08T20:34:28.753 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/w83l785ts.ko 2024-03-08T20:34:28.753 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/w83l786ng.ko 2024-03-08T20:34:28.753 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c 2024-03-08T20:34:28.753 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/algos 2024-03-08T20:34:28.753 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-03-08T20:34:28.753 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-03-08T20:34:28.753 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses 2024-03-08T20:34:28.753 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-03-08T20:34:28.753 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-03-08T20:34:28.753 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-03-08T20:34:28.753 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-03-08T20:34:28.753 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-03-08T20:34:28.753 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-03-08T20:34:28.754 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-03-08T20:34:28.754 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-03-08T20:34:28.754 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-i801.ko 2024-03-08T20:34:28.754 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-isch.ko 2024-03-08T20:34:28.754 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-03-08T20:34:28.754 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-03-08T20:34:28.754 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-03-08T20:34:28.754 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-parport.ko 2024-03-08T20:34:28.754 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-03-08T20:34:28.754 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-03-08T20:34:28.754 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-03-08T20:34:28.754 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-03-08T20:34:28.754 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-03-08T20:34:28.754 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-03-08T20:34:28.755 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-via.ko 2024-03-08T20:34:28.755 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-03-08T20:34:28.755 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-03-08T20:34:28.755 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/i2c-dev.ko 2024-03-08T20:34:28.755 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/i2c-mux.ko 2024-03-08T20:34:28.755 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-03-08T20:34:28.755 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/i2c-smbus.ko 2024-03-08T20:34:28.755 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/i2c-stub.ko 2024-03-08T20:34:28.755 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband 2024-03-08T20:34:28.755 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/core 2024-03-08T20:34:28.755 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/core/ib_cm.ko 2024-03-08T20:34:28.755 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/core/ib_core.ko 2024-03-08T20:34:28.756 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/core/ib_umad.ko 2024-03-08T20:34:28.756 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-03-08T20:34:28.756 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/core/iw_cm.ko 2024-03-08T20:34:28.756 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/core/rdma_cm.ko 2024-03-08T20:34:28.756 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-03-08T20:34:28.756 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/hw 2024-03-08T20:34:28.756 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/hw/cxgb4 2024-03-08T20:34:28.756 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-03-08T20:34:28.756 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/hw/mlx4 2024-03-08T20:34:28.756 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-03-08T20:34:28.756 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/hw/mlx5 2024-03-08T20:34:28.756 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-03-08T20:34:28.756 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/hw/mthca 2024-03-08T20:34:28.757 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-03-08T20:34:28.757 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/ulp 2024-03-08T20:34:28.757 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/ulp/ipoib 2024-03-08T20:34:28.757 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-03-08T20:34:28.757 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/ulp/iser 2024-03-08T20:34:28.757 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-03-08T20:34:28.757 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/ulp/isert 2024-03-08T20:34:28.757 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-03-08T20:34:28.757 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/ulp/srp 2024-03-08T20:34:28.757 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-03-08T20:34:28.757 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/ulp/srpt 2024-03-08T20:34:28.757 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-03-08T20:34:28.757 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input 2024-03-08T20:34:28.757 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/gameport 2024-03-08T20:34:28.757 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-03-08T20:34:28.758 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/gameport/fm801-gp.ko 2024-03-08T20:34:28.758 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/gameport/gameport.ko 2024-03-08T20:34:28.758 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/joydev.ko 2024-03-08T20:34:28.758 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/keyboard 2024-03-08T20:34:28.758 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/keyboard/gpio_keys.ko 2024-03-08T20:34:28.758 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/mouse 2024-03-08T20:34:28.758 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/mouse/appletouch.ko 2024-03-08T20:34:28.758 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/mouse/bcm5974.ko 2024-03-08T20:34:28.758 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/mouse/cyapatp.ko 2024-03-08T20:34:28.758 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/mouse/elan_i2c.ko 2024-03-08T20:34:28.758 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/mouse/sermouse.ko 2024-03-08T20:34:28.758 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-03-08T20:34:28.758 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/mouse/synaptics_usb.ko 2024-03-08T20:34:28.758 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/mouse/vsxxxaa.ko 2024-03-08T20:34:28.759 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/rmi4 2024-03-08T20:34:28.759 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/rmi4/rmi_core.ko 2024-03-08T20:34:28.759 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/serio 2024-03-08T20:34:28.759 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/serio/altera_ps2.ko 2024-03-08T20:34:28.759 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/serio/arc_ps2.ko 2024-03-08T20:34:28.759 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-03-08T20:34:28.759 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/serio/serio_raw.ko 2024-03-08T20:34:28.759 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/sparse-keymap.ko 2024-03-08T20:34:28.759 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/leds 2024-03-08T20:34:28.759 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/leds/leds-blinkm.ko 2024-03-08T20:34:28.759 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/leds/leds-lm3530.ko 2024-03-08T20:34:28.759 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/leds/leds-lp3944.ko 2024-03-08T20:34:28.760 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/leds/leds-ss4200.ko 2024-03-08T20:34:28.760 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/leds/trigger 2024-03-08T20:34:28.760 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-03-08T20:34:28.760 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-03-08T20:34:28.760 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-03-08T20:34:28.760 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-03-08T20:34:28.760 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-03-08T20:34:28.760 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-03-08T20:34:28.760 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-03-08T20:34:28.760 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-03-08T20:34:28.760 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-03-08T20:34:28.760 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md 2024-03-08T20:34:28.760 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/bcache 2024-03-08T20:34:28.760 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/bcache/bcache.ko 2024-03-08T20:34:28.760 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/dm-bio-prison.ko 2024-03-08T20:34:28.761 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/dm-cache-smq.ko 2024-03-08T20:34:28.761 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/dm-cache.ko 2024-03-08T20:34:28.761 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/dm-crypt.ko 2024-03-08T20:34:28.761 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/dm-delay.ko 2024-03-08T20:34:28.761 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/dm-flakey.ko 2024-03-08T20:34:28.761 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/dm-log-userspace.ko 2024-03-08T20:34:28.761 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/dm-multipath.ko 2024-03-08T20:34:28.761 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/dm-queue-length.ko 2024-03-08T20:34:28.761 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/dm-raid.ko 2024-03-08T20:34:28.761 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/dm-round-robin.ko 2024-03-08T20:34:28.761 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/dm-service-time.ko 2024-03-08T20:34:28.761 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/dm-switch.ko 2024-03-08T20:34:28.761 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/dm-thin-pool.ko 2024-03-08T20:34:28.761 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/dm-verity.ko 2024-03-08T20:34:28.761 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/persistent-data 2024-03-08T20:34:28.762 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-03-08T20:34:28.762 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/raid0.ko 2024-03-08T20:34:28.762 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/raid1.ko 2024-03-08T20:34:28.762 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/raid10.ko 2024-03-08T20:34:28.762 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/raid456.ko 2024-03-08T20:34:28.762 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/message 2024-03-08T20:34:28.762 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/message/fusion 2024-03-08T20:34:28.762 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/message/fusion/mptbase.ko 2024-03-08T20:34:28.762 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/message/fusion/mptctl.ko 2024-03-08T20:34:28.762 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/message/fusion/mptfc.ko 2024-03-08T20:34:28.762 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/message/fusion/mptlan.ko 2024-03-08T20:34:28.762 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/message/fusion/mptsas.ko 2024-03-08T20:34:28.762 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/message/fusion/mptscsih.ko 2024-03-08T20:34:28.762 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/message/fusion/mptspi.ko 2024-03-08T20:34:28.763 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/mfd 2024-03-08T20:34:28.763 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/mfd/lpc_ich.ko 2024-03-08T20:34:28.763 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/mfd/lpc_sch.ko 2024-03-08T20:34:28.763 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/mfd/mfd-core.ko 2024-03-08T20:34:28.763 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/mfd/sm501.ko 2024-03-08T20:34:28.763 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/mfd/viperboard.ko 2024-03-08T20:34:28.763 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/mfd/vx855.ko 2024-03-08T20:34:28.763 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/mfd/wl1273-core.ko 2024-03-08T20:34:28.763 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc 2024-03-08T20:34:28.764 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/altera-stapl 2024-03-08T20:34:28.764 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-03-08T20:34:28.764 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/apds9802als.ko 2024-03-08T20:34:28.764 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/apds990x.ko 2024-03-08T20:34:28.764 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/bh1770glc.ko 2024-03-08T20:34:28.764 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/cb710 2024-03-08T20:34:28.764 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/cb710/cb710.ko 2024-03-08T20:34:28.764 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/eeprom 2024-03-08T20:34:28.764 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/eeprom/at24.ko 2024-03-08T20:34:28.764 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-03-08T20:34:28.764 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/eeprom/max6875.ko 2024-03-08T20:34:28.764 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/enclosure.ko 2024-03-08T20:34:28.764 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/hpilo.ko 2024-03-08T20:34:28.764 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/isl29003.ko 2024-03-08T20:34:28.764 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/isl29020.ko 2024-03-08T20:34:28.764 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/lis3lv02d 2024-03-08T20:34:28.765 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-03-08T20:34:28.765 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-03-08T20:34:28.765 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/mei 2024-03-08T20:34:28.765 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/mei/mei-me.ko 2024-03-08T20:34:28.765 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/mei/mei-txe.ko 2024-03-08T20:34:28.765 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/mei/mei.ko 2024-03-08T20:34:28.765 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/sgi-gru 2024-03-08T20:34:28.765 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/sgi-gru/gru.ko 2024-03-08T20:34:28.765 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/sgi-xp 2024-03-08T20:34:28.765 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/sgi-xp/xp.ko 2024-03-08T20:34:28.765 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/sgi-xp/xpc.ko 2024-03-08T20:34:28.765 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-03-08T20:34:28.765 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/tifm_7xx1.ko 2024-03-08T20:34:28.765 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/tifm_core.ko 2024-03-08T20:34:28.766 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/tsl2550.ko 2024-03-08T20:34:28.766 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/vmw_balloon.ko 2024-03-08T20:34:28.766 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/vmw_vmci 2024-03-08T20:34:28.766 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-03-08T20:34:28.766 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net 2024-03-08T20:34:28.766 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/bonding 2024-03-08T20:34:28.766 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/bonding/bonding.ko 2024-03-08T20:34:28.766 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/dsa 2024-03-08T20:34:28.766 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/dsa/b53 2024-03-08T20:34:28.766 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/dsa/b53/b53_common.ko 2024-03-08T20:34:28.766 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/dsa/bcm-sf2.ko 2024-03-08T20:34:28.766 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/dsa/mv88e6060.ko 2024-03-08T20:34:28.766 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/dsa/mv88e6xxx 2024-03-08T20:34:28.766 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-03-08T20:34:28.767 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/dummy.ko 2024-03-08T20:34:28.767 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/eql.ko 2024-03-08T20:34:28.767 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet 2024-03-08T20:34:28.767 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/3com 2024-03-08T20:34:28.767 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-03-08T20:34:28.767 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-03-08T20:34:28.767 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-03-08T20:34:28.767 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-03-08T20:34:28.767 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/8390 2024-03-08T20:34:28.767 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/8390/8390.ko 2024-03-08T20:34:28.767 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-03-08T20:34:28.767 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-03-08T20:34:28.767 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-03-08T20:34:28.767 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/adaptec 2024-03-08T20:34:28.767 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-03-08T20:34:28.768 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/agere 2024-03-08T20:34:28.768 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/agere/et131x.ko 2024-03-08T20:34:28.768 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/alteon 2024-03-08T20:34:28.768 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-03-08T20:34:28.768 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/altera 2024-03-08T20:34:28.768 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-03-08T20:34:28.768 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/amd 2024-03-08T20:34:28.768 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-03-08T20:34:28.768 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-03-08T20:34:28.768 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-03-08T20:34:28.768 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/atheros 2024-03-08T20:34:28.768 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/atheros/alx 2024-03-08T20:34:28.768 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-03-08T20:34:28.768 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/atheros/atl1c 2024-03-08T20:34:28.769 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-03-08T20:34:28.769 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/atheros/atl1e 2024-03-08T20:34:28.769 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-03-08T20:34:28.769 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/atheros/atlx 2024-03-08T20:34:28.769 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-03-08T20:34:28.769 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-03-08T20:34:28.769 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/broadcom 2024-03-08T20:34:28.769 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-03-08T20:34:28.769 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-03-08T20:34:28.769 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-03-08T20:34:28.769 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-03-08T20:34:28.769 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-03-08T20:34:28.769 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/broadcom/genet 2024-03-08T20:34:28.769 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-03-08T20:34:28.770 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-03-08T20:34:28.770 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/brocade 2024-03-08T20:34:28.770 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/brocade/bna 2024-03-08T20:34:28.770 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-03-08T20:34:28.770 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/chelsio 2024-03-08T20:34:28.770 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/chelsio/cxgb 2024-03-08T20:34:28.770 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-03-08T20:34:28.770 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-03-08T20:34:28.770 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-03-08T20:34:28.770 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-03-08T20:34:28.770 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-03-08T20:34:28.770 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-03-08T20:34:28.770 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-03-08T20:34:28.771 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-03-08T20:34:28.771 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-03-08T20:34:28.771 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/cisco 2024-03-08T20:34:28.771 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/cisco/enic 2024-03-08T20:34:28.771 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-03-08T20:34:28.771 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/dec 2024-03-08T20:34:28.771 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/dec/tulip 2024-03-08T20:34:28.771 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-03-08T20:34:28.771 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-03-08T20:34:28.771 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-03-08T20:34:28.771 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-03-08T20:34:28.771 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-03-08T20:34:28.771 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-03-08T20:34:28.771 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/dlink 2024-03-08T20:34:28.772 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-03-08T20:34:28.772 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-03-08T20:34:28.772 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/dnet.ko 2024-03-08T20:34:28.772 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/emulex 2024-03-08T20:34:28.772 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/emulex/benet 2024-03-08T20:34:28.772 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-03-08T20:34:28.772 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/ethoc.ko 2024-03-08T20:34:28.772 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/fealnx.ko 2024-03-08T20:34:28.772 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel 2024-03-08T20:34:28.772 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel/e100.ko 2024-03-08T20:34:28.772 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel/e1000 2024-03-08T20:34:28.772 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-03-08T20:34:28.772 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel/e1000e 2024-03-08T20:34:28.772 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-03-08T20:34:28.773 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel/fm10k 2024-03-08T20:34:28.773 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-03-08T20:34:28.773 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel/i40e 2024-03-08T20:34:28.773 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-03-08T20:34:28.773 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel/iavf 2024-03-08T20:34:28.773 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-03-08T20:34:28.773 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel/igb 2024-03-08T20:34:28.773 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-03-08T20:34:28.773 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel/igbvf 2024-03-08T20:34:28.773 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-03-08T20:34:28.774 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel/ixgbe 2024-03-08T20:34:28.774 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-03-08T20:34:28.774 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel/ixgbevf 2024-03-08T20:34:28.774 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-03-08T20:34:28.774 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/jme.ko 2024-03-08T20:34:28.774 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/marvell 2024-03-08T20:34:28.774 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-03-08T20:34:28.774 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/marvell/skge.ko 2024-03-08T20:34:28.774 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-03-08T20:34:28.774 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/mellanox 2024-03-08T20:34:28.774 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/mellanox/mlx4 2024-03-08T20:34:28.774 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-03-08T20:34:28.775 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-03-08T20:34:28.775 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/mellanox/mlx5 2024-03-08T20:34:28.775 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-03-08T20:34:28.775 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-03-08T20:34:28.775 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/micrel 2024-03-08T20:34:28.775 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-03-08T20:34:28.775 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/myricom 2024-03-08T20:34:28.775 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/myricom/myri10ge 2024-03-08T20:34:28.775 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-03-08T20:34:28.775 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/natsemi 2024-03-08T20:34:28.775 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-03-08T20:34:28.775 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-03-08T20:34:28.775 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/neterion 2024-03-08T20:34:28.775 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-03-08T20:34:28.776 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/nvidia 2024-03-08T20:34:28.776 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-03-08T20:34:28.776 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/packetengines 2024-03-08T20:34:28.776 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-03-08T20:34:28.776 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-03-08T20:34:28.776 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/qlogic 2024-03-08T20:34:28.776 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/qlogic/netxen 2024-03-08T20:34:28.776 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-03-08T20:34:28.776 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-03-08T20:34:28.776 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-03-08T20:34:28.776 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-03-08T20:34:28.776 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/rdc 2024-03-08T20:34:28.776 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-03-08T20:34:28.776 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/realtek 2024-03-08T20:34:28.776 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-03-08T20:34:28.777 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-03-08T20:34:28.777 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/realtek/atp.ko 2024-03-08T20:34:28.777 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-03-08T20:34:28.777 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/rocker 2024-03-08T20:34:28.777 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-03-08T20:34:28.777 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/sfc 2024-03-08T20:34:28.777 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-03-08T20:34:28.777 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/silan 2024-03-08T20:34:28.777 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-03-08T20:34:28.777 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/sis 2024-03-08T20:34:28.777 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/sis/sis190.ko 2024-03-08T20:34:28.777 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/sis/sis900.ko 2024-03-08T20:34:28.777 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/smsc 2024-03-08T20:34:28.777 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-03-08T20:34:28.778 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-03-08T20:34:28.778 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-03-08T20:34:28.778 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-03-08T20:34:28.778 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/stmicro 2024-03-08T20:34:28.778 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/stmicro/stmmac 2024-03-08T20:34:28.778 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-03-08T20:34:28.778 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-03-08T20:34:28.778 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/sun 2024-03-08T20:34:28.778 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/sun/cassini.ko 2024-03-08T20:34:28.778 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/sun/niu.ko 2024-03-08T20:34:28.778 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/sun/sungem.ko 2024-03-08T20:34:28.779 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-03-08T20:34:28.779 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/tehuti 2024-03-08T20:34:28.779 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-03-08T20:34:28.779 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/ti 2024-03-08T20:34:28.779 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/ti/tlan.ko 2024-03-08T20:34:28.779 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/via 2024-03-08T20:34:28.779 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-03-08T20:34:28.779 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-03-08T20:34:28.779 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/wiznet 2024-03-08T20:34:28.779 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-03-08T20:34:28.779 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-03-08T20:34:28.779 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/xircom 2024-03-08T20:34:28.779 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-03-08T20:34:28.779 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/geneve.ko 2024-03-08T20:34:28.779 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/hyperv 2024-03-08T20:34:28.780 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-03-08T20:34:28.780 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ieee802154 2024-03-08T20:34:28.780 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ieee802154/fakelb.ko 2024-03-08T20:34:28.780 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ifb.ko 2024-03-08T20:34:28.780 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ipvlan 2024-03-08T20:34:28.780 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ipvlan/ipvlan.ko 2024-03-08T20:34:28.780 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/macvlan.ko 2024-03-08T20:34:28.780 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/macvtap.ko 2024-03-08T20:34:28.780 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/mdio 2024-03-08T20:34:28.780 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/mdio.ko 2024-03-08T20:34:28.780 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-03-08T20:34:28.780 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-03-08T20:34:28.780 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/mdio/mdio-regmap.ko 2024-03-08T20:34:28.780 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/mii.ko 2024-03-08T20:34:28.781 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/net_failover.ko 2024-03-08T20:34:28.781 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/netconsole.ko 2024-03-08T20:34:28.781 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/nlmon.ko 2024-03-08T20:34:28.781 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/pcs 2024-03-08T20:34:28.781 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/pcs/pcs-lynx.ko 2024-03-08T20:34:28.781 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-03-08T20:34:28.781 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy 2024-03-08T20:34:28.781 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/amd.ko 2024-03-08T20:34:28.781 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-03-08T20:34:28.781 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-03-08T20:34:28.781 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/bcm7xxx.ko 2024-03-08T20:34:28.781 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/bcm87xx.ko 2024-03-08T20:34:28.781 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/broadcom.ko 2024-03-08T20:34:28.781 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/cicada.ko 2024-03-08T20:34:28.781 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/davicom.ko 2024-03-08T20:34:28.782 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/dp83640.ko 2024-03-08T20:34:28.782 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/et1011c.ko 2024-03-08T20:34:28.782 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/icplus.ko 2024-03-08T20:34:28.782 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/lxt.ko 2024-03-08T20:34:28.782 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/marvell.ko 2024-03-08T20:34:28.782 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/micrel.ko 2024-03-08T20:34:28.782 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/national.ko 2024-03-08T20:34:28.782 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/phylink.ko 2024-03-08T20:34:28.782 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/qsemi.ko 2024-03-08T20:34:28.782 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/realtek.ko 2024-03-08T20:34:28.782 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/smsc.ko 2024-03-08T20:34:28.782 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/ste10Xp.ko 2024-03-08T20:34:28.782 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/vitesse.ko 2024-03-08T20:34:28.783 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ppp 2024-03-08T20:34:28.783 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ppp/bsd_comp.ko 2024-03-08T20:34:28.783 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ppp/ppp_async.ko 2024-03-08T20:34:28.783 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ppp/ppp_deflate.ko 2024-03-08T20:34:28.783 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ppp/ppp_generic.ko 2024-03-08T20:34:28.783 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ppp/ppp_mppe.ko 2024-03-08T20:34:28.783 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ppp/ppp_synctty.ko 2024-03-08T20:34:28.783 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ppp/pppoe.ko 2024-03-08T20:34:28.783 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ppp/pppox.ko 2024-03-08T20:34:28.783 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ppp/pptp.ko 2024-03-08T20:34:28.783 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/slip 2024-03-08T20:34:28.783 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/slip/slhc.ko 2024-03-08T20:34:28.783 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/slip/slip.ko 2024-03-08T20:34:28.783 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/sungem_phy.ko 2024-03-08T20:34:28.784 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/tap.ko 2024-03-08T20:34:28.784 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/team 2024-03-08T20:34:28.784 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/team/team.ko 2024-03-08T20:34:28.784 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/team/team_mode_activebackup.ko 2024-03-08T20:34:28.784 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/team/team_mode_broadcast.ko 2024-03-08T20:34:28.784 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-03-08T20:34:28.784 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/team/team_mode_random.ko 2024-03-08T20:34:28.784 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-03-08T20:34:28.784 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/tun.ko 2024-03-08T20:34:28.785 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/veth.ko 2024-03-08T20:34:28.785 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/virtio_net.ko 2024-03-08T20:34:28.785 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/vmxnet3 2024-03-08T20:34:28.785 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-03-08T20:34:28.785 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/vxlan 2024-03-08T20:34:28.785 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/vxlan/vxlan.ko 2024-03-08T20:34:28.785 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/xen-netback 2024-03-08T20:34:28.785 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/xen-netback/xen-netback.ko 2024-03-08T20:34:28.785 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/xen-netfront.ko 2024-03-08T20:34:28.785 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ntb 2024-03-08T20:34:28.785 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ntb/ntb.ko 2024-03-08T20:34:28.785 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/nvme 2024-03-08T20:34:28.785 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/nvme/host 2024-03-08T20:34:28.785 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/nvme/host/nvme-core.ko 2024-03-08T20:34:28.786 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/nvme/host/nvme.ko 2024-03-08T20:34:28.786 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/parport 2024-03-08T20:34:28.786 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/parport/parport.ko 2024-03-08T20:34:28.786 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/parport/parport_cs.ko 2024-03-08T20:34:28.786 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/parport/parport_pc.ko 2024-03-08T20:34:28.786 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/parport/parport_serial.ko 2024-03-08T20:34:28.786 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/pci 2024-03-08T20:34:28.786 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/pci/hotplug 2024-03-08T20:34:28.786 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-03-08T20:34:28.786 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/pci/pcie 2024-03-08T20:34:28.786 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/pci/pcie/aer_inject.ko 2024-03-08T20:34:28.786 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/pci/xen-pcifront.ko 2024-03-08T20:34:28.786 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/pcmcia 2024-03-08T20:34:28.787 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/pcmcia/i82092.ko 2024-03-08T20:34:28.787 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/pcmcia/pd6729.ko 2024-03-08T20:34:28.787 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/pcmcia/yenta_socket.ko 2024-03-08T20:34:28.787 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/pinctrl 2024-03-08T20:34:28.787 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/pinctrl/intel 2024-03-08T20:34:28.787 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-03-08T20:34:28.787 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform 2024-03-08T20:34:28.787 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86 2024-03-08T20:34:28.787 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/acer-wmi.ko 2024-03-08T20:34:28.787 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/acerhdf.ko 2024-03-08T20:34:28.787 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/apple-gmux.ko 2024-03-08T20:34:28.787 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/asus-laptop.ko 2024-03-08T20:34:28.787 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-03-08T20:34:28.787 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/asus-wmi.ko 2024-03-08T20:34:28.787 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/classmate-laptop.ko 2024-03-08T20:34:28.788 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-03-08T20:34:28.788 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-03-08T20:34:28.788 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-03-08T20:34:28.788 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-03-08T20:34:28.788 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/hdaps.ko 2024-03-08T20:34:28.788 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/intel 2024-03-08T20:34:28.788 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-03-08T20:34:28.788 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/intel_ips.ko 2024-03-08T20:34:28.788 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/msi-wmi.ko 2024-03-08T20:34:28.788 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/mxm-wmi.ko 2024-03-08T20:34:28.788 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-03-08T20:34:28.789 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/samsung-laptop.ko 2024-03-08T20:34:28.789 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/samsung-q10.ko 2024-03-08T20:34:28.789 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/topstar-laptop.ko 2024-03-08T20:34:28.789 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-03-08T20:34:28.789 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/toshiba_haps.ko 2024-03-08T20:34:28.789 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/wmi-bmof.ko 2024-03-08T20:34:28.789 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/wmi.ko 2024-03-08T20:34:28.789 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/powercap 2024-03-08T20:34:28.789 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/powercap/intel_rapl_common.ko 2024-03-08T20:34:28.789 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/powercap/intel_rapl_msr.ko 2024-03-08T20:34:28.789 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/pps 2024-03-08T20:34:28.789 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/pps/clients 2024-03-08T20:34:28.789 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/pps/clients/pps-gpio.ko 2024-03-08T20:34:28.789 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/pps/clients/pps-ldisc.ko 2024-03-08T20:34:28.789 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/pps/clients/pps_parport.ko 2024-03-08T20:34:28.790 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/pps/pps_core.ko 2024-03-08T20:34:28.790 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ptp 2024-03-08T20:34:28.790 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ptp/ptp.ko 2024-03-08T20:34:28.790 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ptp/ptp_kvm.ko 2024-03-08T20:34:28.790 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc 2024-03-08T20:34:28.790 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-bq32k.ko 2024-03-08T20:34:28.790 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-ds1286.ko 2024-03-08T20:34:28.790 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-ds1307.ko 2024-03-08T20:34:28.790 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-ds1374.ko 2024-03-08T20:34:28.790 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-ds1511.ko 2024-03-08T20:34:28.790 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-ds1553.ko 2024-03-08T20:34:28.791 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-ds1672.ko 2024-03-08T20:34:28.791 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-ds1742.ko 2024-03-08T20:34:28.791 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-ds2404.ko 2024-03-08T20:34:28.791 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-ds3232.ko 2024-03-08T20:34:28.791 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-em3027.ko 2024-03-08T20:34:28.791 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-fm3130.ko 2024-03-08T20:34:28.791 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-isl12022.ko 2024-03-08T20:34:28.791 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-isl1208.ko 2024-03-08T20:34:28.791 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-m41t80.ko 2024-03-08T20:34:28.791 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-m48t35.ko 2024-03-08T20:34:28.791 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-m48t59.ko 2024-03-08T20:34:28.791 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-max6900.ko 2024-03-08T20:34:28.792 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-msm6242.ko 2024-03-08T20:34:28.792 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-pcf2127.ko 2024-03-08T20:34:28.792 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-pcf85063.ko 2024-03-08T20:34:28.792 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-pcf8523.ko 2024-03-08T20:34:28.792 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-pcf8563.ko 2024-03-08T20:34:28.792 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-pcf8583.ko 2024-03-08T20:34:28.792 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-rp5c01.ko 2024-03-08T20:34:28.792 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-rs5c372.ko 2024-03-08T20:34:28.792 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-03-08T20:34:28.792 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-rx8025.ko 2024-03-08T20:34:28.792 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-rx8581.ko 2024-03-08T20:34:28.792 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-03-08T20:34:28.792 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-x1205.ko 2024-03-08T20:34:28.792 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi 2024-03-08T20:34:28.792 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/3w-9xxx.ko 2024-03-08T20:34:28.793 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/3w-sas.ko 2024-03-08T20:34:28.793 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/3w-xxxx.ko 2024-03-08T20:34:28.793 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/BusLogic.ko 2024-03-08T20:34:28.793 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/a100u2w.ko 2024-03-08T20:34:28.793 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/aacraid 2024-03-08T20:34:28.793 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/aacraid/aacraid.ko 2024-03-08T20:34:28.793 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/advansys.ko 2024-03-08T20:34:28.793 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/aic7xxx 2024-03-08T20:34:28.793 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-03-08T20:34:28.793 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-03-08T20:34:28.793 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/am53c974.ko 2024-03-08T20:34:28.793 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/arcmsr 2024-03-08T20:34:28.793 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-03-08T20:34:28.793 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/atp870u.ko 2024-03-08T20:34:28.793 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/be2iscsi 2024-03-08T20:34:28.794 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-03-08T20:34:28.794 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/bfa 2024-03-08T20:34:28.794 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/bfa/bfa.ko 2024-03-08T20:34:28.794 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/bnx2fc 2024-03-08T20:34:28.794 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-03-08T20:34:28.794 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/bnx2i 2024-03-08T20:34:28.794 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-03-08T20:34:28.794 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/ch.ko 2024-03-08T20:34:28.794 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/csiostor 2024-03-08T20:34:28.794 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/csiostor/csiostor.ko 2024-03-08T20:34:28.794 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/cxgbi 2024-03-08T20:34:28.795 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/cxgbi/cxgb3i 2024-03-08T20:34:28.795 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-03-08T20:34:28.795 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/cxgbi/cxgb4i 2024-03-08T20:34:28.795 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-03-08T20:34:28.795 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-03-08T20:34:28.795 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/dc395x.ko 2024-03-08T20:34:28.795 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/device_handler 2024-03-08T20:34:28.795 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-03-08T20:34:28.795 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-03-08T20:34:28.795 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-03-08T20:34:28.795 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-03-08T20:34:28.796 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/dmx3191d.ko 2024-03-08T20:34:28.796 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/esas2r 2024-03-08T20:34:28.796 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/esas2r/esas2r.ko 2024-03-08T20:34:28.796 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/esp_scsi.ko 2024-03-08T20:34:28.796 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/fcoe 2024-03-08T20:34:28.796 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/fcoe/fcoe.ko 2024-03-08T20:34:28.796 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-03-08T20:34:28.796 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/fnic 2024-03-08T20:34:28.796 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/fnic/fnic.ko 2024-03-08T20:34:28.796 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/hpsa.ko 2024-03-08T20:34:28.796 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/hptiop.ko 2024-03-08T20:34:28.796 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/hv_storvsc.ko 2024-03-08T20:34:28.796 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/initio.ko 2024-03-08T20:34:28.796 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/ipr.ko 2024-03-08T20:34:28.796 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/ips.ko 2024-03-08T20:34:28.797 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/isci 2024-03-08T20:34:28.797 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/isci/isci.ko 2024-03-08T20:34:28.797 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-03-08T20:34:28.797 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/iscsi_tcp.ko 2024-03-08T20:34:28.797 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/libfc 2024-03-08T20:34:28.797 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/libfc/libfc.ko 2024-03-08T20:34:28.797 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/libiscsi.ko 2024-03-08T20:34:28.797 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/libiscsi_tcp.ko 2024-03-08T20:34:28.797 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/libsas 2024-03-08T20:34:28.797 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/libsas/libsas.ko 2024-03-08T20:34:28.797 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/lpfc 2024-03-08T20:34:28.797 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/lpfc/lpfc.ko 2024-03-08T20:34:28.797 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/megaraid 2024-03-08T20:34:28.797 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/megaraid.ko 2024-03-08T20:34:28.797 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-03-08T20:34:28.798 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-03-08T20:34:28.798 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-03-08T20:34:28.798 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/mpt3sas 2024-03-08T20:34:28.798 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-03-08T20:34:28.798 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/mvsas 2024-03-08T20:34:28.798 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/mvsas/mvsas.ko 2024-03-08T20:34:28.798 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/mvumi.ko 2024-03-08T20:34:28.798 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/pm8001 2024-03-08T20:34:28.798 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-03-08T20:34:28.798 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/pmcraid.ko 2024-03-08T20:34:28.798 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/qla1280.ko 2024-03-08T20:34:28.798 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/qla2xxx 2024-03-08T20:34:28.798 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-03-08T20:34:28.799 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-03-08T20:34:28.799 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/qla4xxx 2024-03-08T20:34:28.799 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-03-08T20:34:28.799 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/raid_class.ko 2024-03-08T20:34:28.799 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/scsi_debug.ko 2024-03-08T20:34:28.799 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/scsi_transport_fc.ko 2024-03-08T20:34:28.799 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-03-08T20:34:28.799 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/scsi_transport_sas.ko 2024-03-08T20:34:28.799 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/scsi_transport_spi.ko 2024-03-08T20:34:28.799 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/scsi_transport_srp.ko 2024-03-08T20:34:28.799 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/ses.ko 2024-03-08T20:34:28.800 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/st.ko 2024-03-08T20:34:28.800 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/stex.ko 2024-03-08T20:34:28.800 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/sym53c8xx_2 2024-03-08T20:34:28.800 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-03-08T20:34:28.800 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/virtio_scsi.ko 2024-03-08T20:34:28.800 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/vmw_pvscsi.ko 2024-03-08T20:34:28.800 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/wd719x.ko 2024-03-08T20:34:28.800 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ssb 2024-03-08T20:34:28.800 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ssb/ssb.ko 2024-03-08T20:34:28.800 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/target 2024-03-08T20:34:28.800 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/target/iscsi 2024-03-08T20:34:28.800 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-03-08T20:34:28.800 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/target/loopback 2024-03-08T20:34:28.800 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/target/loopback/tcm_loop.ko 2024-03-08T20:34:28.800 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/target/target_core_file.ko 2024-03-08T20:34:28.801 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/target/target_core_iblock.ko 2024-03-08T20:34:28.801 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/target/target_core_mod.ko 2024-03-08T20:34:28.801 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/target/target_core_pscsi.ko 2024-03-08T20:34:28.801 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/target/target_core_user.ko 2024-03-08T20:34:28.801 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/target/tcm_fc 2024-03-08T20:34:28.801 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-03-08T20:34:28.801 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal 2024-03-08T20:34:28.801 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel 2024-03-08T20:34:28.801 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel/int340x_thermal 2024-03-08T20:34:28.801 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-03-08T20:34:28.801 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-03-08T20:34:28.801 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-03-08T20:34:28.801 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-03-08T20:34:28.801 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-03-08T20:34:28.801 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-03-08T20:34:28.801 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-03-08T20:34:28.802 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-03-08T20:34:28.802 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-03-08T20:34:28.802 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-03-08T20:34:28.802 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-03-08T20:34:28.802 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-03-08T20:34:28.802 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-03-08T20:34:28.802 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-03-08T20:34:28.802 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-03-08T20:34:28.802 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-03-08T20:34:28.802 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-03-08T20:34:28.802 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-03-08T20:34:28.802 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/tty 2024-03-08T20:34:28.802 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/tty/ipwireless 2024-03-08T20:34:28.802 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-03-08T20:34:28.802 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/tty/n_gsm.ko 2024-03-08T20:34:28.803 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/tty/n_hdlc.ko 2024-03-08T20:34:28.803 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/tty/nozomi.ko 2024-03-08T20:34:28.803 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/tty/serial 2024-03-08T20:34:28.803 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/tty/serial/8250 2024-03-08T20:34:28.803 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-03-08T20:34:28.803 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/tty/serial/arc_uart.ko 2024-03-08T20:34:28.803 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/tty/serial/jsm 2024-03-08T20:34:28.803 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/tty/serial/jsm/jsm.ko 2024-03-08T20:34:28.803 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ufs 2024-03-08T20:34:28.803 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ufs/core 2024-03-08T20:34:28.803 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ufs/core/ufshcd-core.ko 2024-03-08T20:34:28.803 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ufs/host 2024-03-08T20:34:28.803 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-03-08T20:34:28.803 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/uio 2024-03-08T20:34:28.803 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/uio/uio.ko 2024-03-08T20:34:28.804 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/uio/uio_aec.ko 2024-03-08T20:34:28.804 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/uio/uio_cif.ko 2024-03-08T20:34:28.804 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/uio/uio_pci_generic.ko 2024-03-08T20:34:28.804 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/uio/uio_sercos3.ko 2024-03-08T20:34:28.804 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb 2024-03-08T20:34:28.804 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/class 2024-03-08T20:34:28.804 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/class/cdc-acm.ko 2024-03-08T20:34:28.804 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/class/cdc-wdm.ko 2024-03-08T20:34:28.804 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/class/usblp.ko 2024-03-08T20:34:28.804 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/class/usbtmc.ko 2024-03-08T20:34:28.804 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/host 2024-03-08T20:34:28.804 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/host/sl811-hcd.ko 2024-03-08T20:34:28.804 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/image 2024-03-08T20:34:28.804 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/image/mdc800.ko 2024-03-08T20:34:28.804 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/image/microtek.ko 2024-03-08T20:34:28.804 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/misc 2024-03-08T20:34:28.805 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/misc/adutux.ko 2024-03-08T20:34:28.805 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/misc/appledisplay.ko 2024-03-08T20:34:28.805 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/misc/emi26.ko 2024-03-08T20:34:28.805 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/misc/emi62.ko 2024-03-08T20:34:28.805 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/misc/ezusb.ko 2024-03-08T20:34:28.805 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/misc/idmouse.ko 2024-03-08T20:34:28.805 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/misc/iowarrior.ko 2024-03-08T20:34:28.805 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/misc/isight_firmware.ko 2024-03-08T20:34:28.806 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/misc/ldusb.ko 2024-03-08T20:34:28.806 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/misc/legousbtower.ko 2024-03-08T20:34:28.806 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/misc/sisusbvga 2024-03-08T20:34:28.806 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-03-08T20:34:28.806 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/misc/trancevibrator.ko 2024-03-08T20:34:28.806 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/misc/usb3503.ko 2024-03-08T20:34:28.806 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/misc/usblcd.ko 2024-03-08T20:34:28.806 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/misc/usbsevseg.ko 2024-03-08T20:34:28.806 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/misc/uss720.ko 2024-03-08T20:34:28.806 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/misc/yurex.ko 2024-03-08T20:34:28.806 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/phy 2024-03-08T20:34:28.806 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/phy/phy-generic.ko 2024-03-08T20:34:28.806 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial 2024-03-08T20:34:28.806 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/aircable.ko 2024-03-08T20:34:28.807 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/ark3116.ko 2024-03-08T20:34:28.807 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/belkin_sa.ko 2024-03-08T20:34:28.807 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/ch341.ko 2024-03-08T20:34:28.807 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/cp210x.ko 2024-03-08T20:34:28.807 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/cyberjack.ko 2024-03-08T20:34:28.807 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/cypress_m8.ko 2024-03-08T20:34:28.807 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/digi_acceleport.ko 2024-03-08T20:34:28.807 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/empeg.ko 2024-03-08T20:34:28.807 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/ftdi_sio.ko 2024-03-08T20:34:28.807 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/garmin_gps.ko 2024-03-08T20:34:28.807 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/io_edgeport.ko 2024-03-08T20:34:28.807 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/io_ti.ko 2024-03-08T20:34:28.807 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/ipaq.ko 2024-03-08T20:34:28.807 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/ipw.ko 2024-03-08T20:34:28.807 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/ir-usb.ko 2024-03-08T20:34:28.807 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-03-08T20:34:28.808 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/keyspan.ko 2024-03-08T20:34:28.808 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/keyspan_pda.ko 2024-03-08T20:34:28.808 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/kl5kusb105.ko 2024-03-08T20:34:28.808 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/kobil_sct.ko 2024-03-08T20:34:28.808 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/mct_u232.ko 2024-03-08T20:34:28.808 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/mos7720.ko 2024-03-08T20:34:28.808 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/mos7840.ko 2024-03-08T20:34:28.808 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/navman.ko 2024-03-08T20:34:28.808 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/omninet.ko 2024-03-08T20:34:28.808 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/opticon.ko 2024-03-08T20:34:28.808 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/option.ko 2024-03-08T20:34:28.808 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/oti6858.ko 2024-03-08T20:34:28.808 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/pl2303.ko 2024-03-08T20:34:28.808 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/qcaux.ko 2024-03-08T20:34:28.808 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/qcserial.ko 2024-03-08T20:34:28.809 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/quatech2.ko 2024-03-08T20:34:28.809 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/safe_serial.ko 2024-03-08T20:34:28.809 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/sierra.ko 2024-03-08T20:34:28.809 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/spcp8x5.ko 2024-03-08T20:34:28.809 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/ssu100.ko 2024-03-08T20:34:28.809 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/symbolserial.ko 2024-03-08T20:34:28.809 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-03-08T20:34:28.809 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-03-08T20:34:28.809 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/usb_debug.ko 2024-03-08T20:34:28.809 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/usb_wwan.ko 2024-03-08T20:34:28.809 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/visor.ko 2024-03-08T20:34:28.809 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/whiteheat.ko 2024-03-08T20:34:28.809 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/xsens_mt.ko 2024-03-08T20:34:28.810 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/storage 2024-03-08T20:34:28.810 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/storage/uas.ko 2024-03-08T20:34:28.810 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/storage/ums-alauda.ko 2024-03-08T20:34:28.810 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/storage/ums-cypress.ko 2024-03-08T20:34:28.810 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/storage/ums-datafab.ko 2024-03-08T20:34:28.810 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-03-08T20:34:28.810 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/storage/ums-freecom.ko 2024-03-08T20:34:28.810 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/storage/ums-isd200.ko 2024-03-08T20:34:28.810 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-03-08T20:34:28.810 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/storage/ums-karma.ko 2024-03-08T20:34:28.810 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/storage/ums-onetouch.ko 2024-03-08T20:34:28.810 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/storage/ums-realtek.ko 2024-03-08T20:34:28.810 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/storage/ums-sddr09.ko 2024-03-08T20:34:28.811 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/storage/ums-sddr55.ko 2024-03-08T20:34:28.811 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/storage/ums-usbat.ko 2024-03-08T20:34:28.811 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/storage/usb-storage.ko 2024-03-08T20:34:28.811 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/usbip 2024-03-08T20:34:28.811 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/usbip/usbip-core.ko 2024-03-08T20:34:28.811 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/usbip/usbip-host.ko 2024-03-08T20:34:28.811 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-03-08T20:34:28.811 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/vfio 2024-03-08T20:34:28.811 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/vfio/pci 2024-03-08T20:34:28.811 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-03-08T20:34:28.811 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/vfio/pci/vfio-pci.ko 2024-03-08T20:34:28.811 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/vfio/vfio.ko 2024-03-08T20:34:28.811 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-03-08T20:34:28.811 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/vhost 2024-03-08T20:34:28.811 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/vhost/vhost.ko 2024-03-08T20:34:28.811 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/vhost/vhost_iotlb.ko 2024-03-08T20:34:28.812 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/vhost/vhost_net.ko 2024-03-08T20:34:28.812 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/vhost/vhost_scsi.ko 2024-03-08T20:34:28.812 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/video 2024-03-08T20:34:28.812 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/video/backlight 2024-03-08T20:34:28.812 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/video/backlight/apple_bl.ko 2024-03-08T20:34:28.812 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/video/backlight/lcd.ko 2024-03-08T20:34:28.812 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/video/backlight/platform_lcd.ko 2024-03-08T20:34:28.812 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/virtio 2024-03-08T20:34:28.812 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/virtio/virtio.ko 2024-03-08T20:34:28.812 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/virtio/virtio_balloon.ko 2024-03-08T20:34:28.812 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/virtio/virtio_mmio.ko 2024-03-08T20:34:28.812 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/virtio/virtio_pci.ko 2024-03-08T20:34:28.812 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-03-08T20:34:28.812 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-03-08T20:34:28.812 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/virtio/virtio_ring.ko 2024-03-08T20:34:28.813 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/w1 2024-03-08T20:34:28.813 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/w1/masters 2024-03-08T20:34:28.813 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/w1/masters/ds2482.ko 2024-03-08T20:34:28.813 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/w1/masters/ds2490.ko 2024-03-08T20:34:28.813 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/w1/slaves 2024-03-08T20:34:28.813 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-03-08T20:34:28.813 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-03-08T20:34:28.813 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-03-08T20:34:28.813 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-03-08T20:34:28.813 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-03-08T20:34:28.813 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-03-08T20:34:28.813 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-03-08T20:34:28.813 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-03-08T20:34:28.813 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-03-08T20:34:28.813 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/w1/slaves/w1_smem.ko 2024-03-08T20:34:28.813 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/w1/slaves/w1_therm.ko 2024-03-08T20:34:28.814 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/w1/wire.ko 2024-03-08T20:34:28.814 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog 2024-03-08T20:34:28.814 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/alim1535_wdt.ko 2024-03-08T20:34:28.814 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/alim7101_wdt.ko 2024-03-08T20:34:28.814 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/f71808e_wdt.ko 2024-03-08T20:34:28.814 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/hpwdt.ko 2024-03-08T20:34:28.814 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/i6300esb.ko 2024-03-08T20:34:28.814 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-03-08T20:34:28.814 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/iTCO_wdt.ko 2024-03-08T20:34:28.814 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/ib700wdt.ko 2024-03-08T20:34:28.814 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/ibmasr.ko 2024-03-08T20:34:28.814 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-03-08T20:34:28.814 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/it8712f_wdt.ko 2024-03-08T20:34:28.814 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/it87_wdt.ko 2024-03-08T20:34:28.814 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/machzwd.ko 2024-03-08T20:34:28.815 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/nv_tco.ko 2024-03-08T20:34:28.815 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/pcwd_pci.ko 2024-03-08T20:34:28.815 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/pcwd_usb.ko 2024-03-08T20:34:28.815 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-03-08T20:34:28.815 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/sch311x_wdt.ko 2024-03-08T20:34:28.815 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/softdog.ko 2024-03-08T20:34:28.815 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/sp5100_tco.ko 2024-03-08T20:34:28.815 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/via_wdt.ko 2024-03-08T20:34:28.815 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-03-08T20:34:28.815 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/w83877f_wdt.ko 2024-03-08T20:34:28.815 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/w83977f_wdt.ko 2024-03-08T20:34:28.815 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/wdt_pci.ko 2024-03-08T20:34:28.815 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/xen_wdt.ko 2024-03-08T20:34:28.816 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/xen 2024-03-08T20:34:28.816 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/xen/xen-acpi-processor.ko 2024-03-08T20:34:28.816 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/xen/xen-evtchn.ko 2024-03-08T20:34:28.816 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/xen/xen-gntalloc.ko 2024-03-08T20:34:28.816 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/xen/xen-gntdev.ko 2024-03-08T20:34:28.816 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/xen/xen-pciback 2024-03-08T20:34:28.816 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-03-08T20:34:28.816 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/xen/xen-privcmd.ko 2024-03-08T20:34:28.816 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/xen/xenfs 2024-03-08T20:34:28.816 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/xen/xenfs/xenfs.ko 2024-03-08T20:34:28.816 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs 2024-03-08T20:34:28.816 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/9p 2024-03-08T20:34:28.817 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/9p/9p.ko 2024-03-08T20:34:28.817 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/affs 2024-03-08T20:34:28.817 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/affs/affs.ko 2024-03-08T20:34:28.817 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/befs 2024-03-08T20:34:28.817 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/befs/befs.ko 2024-03-08T20:34:28.817 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/binfmt_misc.ko 2024-03-08T20:34:28.817 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/btrfs 2024-03-08T20:34:28.817 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/btrfs/btrfs.ko 2024-03-08T20:34:28.817 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/cachefiles 2024-03-08T20:34:28.817 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/cachefiles/cachefiles.ko 2024-03-08T20:34:28.817 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/ceph 2024-03-08T20:34:28.817 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/ceph/ceph.ko 2024-03-08T20:34:28.817 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/coda 2024-03-08T20:34:28.817 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/coda/coda.ko 2024-03-08T20:34:28.817 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/cramfs 2024-03-08T20:34:28.817 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/cramfs/cramfs.ko 2024-03-08T20:34:28.818 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/dlm 2024-03-08T20:34:28.818 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/dlm/dlm.ko 2024-03-08T20:34:28.818 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/ecryptfs 2024-03-08T20:34:28.818 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/ecryptfs/ecryptfs.ko 2024-03-08T20:34:28.818 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/f2fs 2024-03-08T20:34:28.818 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/f2fs/f2fs.ko 2024-03-08T20:34:28.818 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/fat 2024-03-08T20:34:28.818 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/fat/fat.ko 2024-03-08T20:34:28.818 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/fat/msdos.ko 2024-03-08T20:34:28.818 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/fat/vfat.ko 2024-03-08T20:34:28.818 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/fuse 2024-03-08T20:34:28.818 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/fuse/cuse.ko 2024-03-08T20:34:28.818 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/fuse/fuse.ko 2024-03-08T20:34:28.818 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/gfs2 2024-03-08T20:34:28.819 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/gfs2/gfs2.ko 2024-03-08T20:34:28.819 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/hfs 2024-03-08T20:34:28.819 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/hfs/hfs.ko 2024-03-08T20:34:28.819 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/hfsplus 2024-03-08T20:34:28.819 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/hfsplus/hfsplus.ko 2024-03-08T20:34:28.819 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/isofs 2024-03-08T20:34:28.819 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/isofs/isofs.ko 2024-03-08T20:34:28.819 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/jfs 2024-03-08T20:34:28.819 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/jfs/jfs.ko 2024-03-08T20:34:28.819 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/lockd 2024-03-08T20:34:28.819 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/lockd/lockd.ko 2024-03-08T20:34:28.819 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/minix 2024-03-08T20:34:28.819 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/minix/minix.ko 2024-03-08T20:34:28.819 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/netfs 2024-03-08T20:34:28.820 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/netfs/netfs.ko 2024-03-08T20:34:28.820 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nfs 2024-03-08T20:34:28.820 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nfs/blocklayout 2024-03-08T20:34:28.820 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-03-08T20:34:28.820 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nfs/filelayout 2024-03-08T20:34:28.820 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-03-08T20:34:28.820 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nfs/flexfilelayout 2024-03-08T20:34:28.820 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-03-08T20:34:28.820 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nfs/nfs.ko 2024-03-08T20:34:28.820 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nfs/nfsv3.ko 2024-03-08T20:34:28.820 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nfs/nfsv4.ko 2024-03-08T20:34:28.820 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nfs_common 2024-03-08T20:34:28.820 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nfs_common/grace.ko 2024-03-08T20:34:28.820 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nfs_common/nfs_acl.ko 2024-03-08T20:34:28.820 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nfsd 2024-03-08T20:34:28.821 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nfsd/nfsd.ko 2024-03-08T20:34:28.821 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nilfs2 2024-03-08T20:34:28.821 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nilfs2/nilfs2.ko 2024-03-08T20:34:28.821 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls 2024-03-08T20:34:28.821 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/mac-celtic.ko 2024-03-08T20:34:28.821 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/mac-centeuro.ko 2024-03-08T20:34:28.821 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/mac-croatian.ko 2024-03-08T20:34:28.821 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/mac-cyrillic.ko 2024-03-08T20:34:28.821 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/mac-gaelic.ko 2024-03-08T20:34:28.821 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/mac-greek.ko 2024-03-08T20:34:28.821 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/mac-iceland.ko 2024-03-08T20:34:28.821 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/mac-inuit.ko 2024-03-08T20:34:28.822 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/mac-roman.ko 2024-03-08T20:34:28.822 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/mac-romanian.ko 2024-03-08T20:34:28.822 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/mac-turkish.ko 2024-03-08T20:34:28.822 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp1250.ko 2024-03-08T20:34:28.822 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp1251.ko 2024-03-08T20:34:28.822 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp1255.ko 2024-03-08T20:34:28.822 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp737.ko 2024-03-08T20:34:28.822 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp775.ko 2024-03-08T20:34:28.822 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp850.ko 2024-03-08T20:34:28.823 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp852.ko 2024-03-08T20:34:28.823 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp855.ko 2024-03-08T20:34:28.823 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp857.ko 2024-03-08T20:34:28.823 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp860.ko 2024-03-08T20:34:28.823 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp861.ko 2024-03-08T20:34:28.823 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp862.ko 2024-03-08T20:34:28.823 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp863.ko 2024-03-08T20:34:28.823 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp864.ko 2024-03-08T20:34:28.823 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp865.ko 2024-03-08T20:34:28.823 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp866.ko 2024-03-08T20:34:28.823 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp869.ko 2024-03-08T20:34:28.823 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp874.ko 2024-03-08T20:34:28.823 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp932.ko 2024-03-08T20:34:28.823 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp936.ko 2024-03-08T20:34:28.824 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp949.ko 2024-03-08T20:34:28.824 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp950.ko 2024-03-08T20:34:28.824 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_euc-jp.ko 2024-03-08T20:34:28.824 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_iso8859-1.ko 2024-03-08T20:34:28.824 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_iso8859-13.ko 2024-03-08T20:34:28.824 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_iso8859-14.ko 2024-03-08T20:34:28.824 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_iso8859-15.ko 2024-03-08T20:34:28.824 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_iso8859-2.ko 2024-03-08T20:34:28.824 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_iso8859-3.ko 2024-03-08T20:34:28.824 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_iso8859-4.ko 2024-03-08T20:34:28.824 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_iso8859-5.ko 2024-03-08T20:34:28.824 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_iso8859-6.ko 2024-03-08T20:34:28.824 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_iso8859-7.ko 2024-03-08T20:34:28.824 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_iso8859-9.ko 2024-03-08T20:34:28.824 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_koi8-r.ko 2024-03-08T20:34:28.825 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_koi8-ru.ko 2024-03-08T20:34:28.825 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_koi8-u.ko 2024-03-08T20:34:28.825 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_ucs2_utils.ko 2024-03-08T20:34:28.825 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_utf8.ko 2024-03-08T20:34:28.825 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/ocfs2 2024-03-08T20:34:28.825 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/ocfs2/cluster 2024-03-08T20:34:28.825 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-03-08T20:34:28.825 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/ocfs2/dlm 2024-03-08T20:34:28.825 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-03-08T20:34:28.825 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/ocfs2/dlmfs 2024-03-08T20:34:28.825 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-03-08T20:34:28.825 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/ocfs2/ocfs2.ko 2024-03-08T20:34:28.825 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-03-08T20:34:28.825 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-03-08T20:34:28.826 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-03-08T20:34:28.826 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/overlayfs 2024-03-08T20:34:28.826 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/overlayfs/overlay.ko 2024-03-08T20:34:28.826 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/pstore 2024-03-08T20:34:28.826 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/pstore/ramoops.ko 2024-03-08T20:34:28.826 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/reiserfs 2024-03-08T20:34:28.826 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/reiserfs/reiserfs.ko 2024-03-08T20:34:28.826 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/romfs 2024-03-08T20:34:28.826 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/romfs/romfs.ko 2024-03-08T20:34:28.826 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/smb 2024-03-08T20:34:28.826 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/smb/client 2024-03-08T20:34:28.826 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/smb/client/cifs.ko 2024-03-08T20:34:28.827 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/smb/common 2024-03-08T20:34:28.827 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/smb/common/cifs_arc4.ko 2024-03-08T20:34:28.827 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/smb/common/cifs_md4.ko 2024-03-08T20:34:28.827 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/squashfs 2024-03-08T20:34:28.827 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/squashfs/squashfs.ko 2024-03-08T20:34:28.827 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/sysv 2024-03-08T20:34:28.827 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/sysv/sysv.ko 2024-03-08T20:34:28.827 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/udf 2024-03-08T20:34:28.827 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/udf/udf.ko 2024-03-08T20:34:28.827 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/ufs 2024-03-08T20:34:28.827 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/ufs/ufs.ko 2024-03-08T20:34:28.827 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/xfs 2024-03-08T20:34:28.827 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/xfs/xfs.ko 2024-03-08T20:34:28.828 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/kernel 2024-03-08T20:34:28.828 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/kernel/rcu 2024-03-08T20:34:28.828 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/kernel/rcu/rcutorture.ko 2024-03-08T20:34:28.828 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/kernel/torture.ko 2024-03-08T20:34:28.828 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/kernel/trace 2024-03-08T20:34:28.828 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-03-08T20:34:28.828 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib 2024-03-08T20:34:28.828 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/asn1_encoder.ko 2024-03-08T20:34:28.828 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/crc-itu-t.ko 2024-03-08T20:34:28.828 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/crc8.ko 2024-03-08T20:34:28.828 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/crypto 2024-03-08T20:34:28.828 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/crypto/libarc4.ko 2024-03-08T20:34:28.828 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/crypto/libdes.ko 2024-03-08T20:34:28.828 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/libcrc32c.ko 2024-03-08T20:34:28.828 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/lru_cache.ko 2024-03-08T20:34:28.829 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/lz4 2024-03-08T20:34:28.829 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/lz4/lz4_compress.ko 2024-03-08T20:34:28.829 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/lz4/lz4hc_compress.ko 2024-03-08T20:34:28.829 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/math 2024-03-08T20:34:28.829 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/math/cordic.ko 2024-03-08T20:34:28.829 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/raid6 2024-03-08T20:34:28.829 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/raid6/raid6_pq.ko 2024-03-08T20:34:28.829 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/reed_solomon 2024-03-08T20:34:28.829 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/reed_solomon/reed_solomon.ko 2024-03-08T20:34:28.829 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/ts_bm.ko 2024-03-08T20:34:28.829 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/ts_fsm.ko 2024-03-08T20:34:28.829 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/ts_kmp.ko 2024-03-08T20:34:28.829 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/zstd 2024-03-08T20:34:28.829 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/zstd/zstd_compress.ko 2024-03-08T20:34:28.830 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/mm 2024-03-08T20:34:28.830 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/mm/hwpoison-inject.ko 2024-03-08T20:34:28.830 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net 2024-03-08T20:34:28.830 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/6lowpan 2024-03-08T20:34:28.830 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/6lowpan/6lowpan.ko 2024-03-08T20:34:28.830 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/6lowpan/nhc_dest.ko 2024-03-08T20:34:28.830 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/6lowpan/nhc_fragment.ko 2024-03-08T20:34:28.830 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/6lowpan/nhc_hop.ko 2024-03-08T20:34:28.830 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/6lowpan/nhc_ipv6.ko 2024-03-08T20:34:28.830 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/6lowpan/nhc_mobility.ko 2024-03-08T20:34:28.830 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/6lowpan/nhc_routing.ko 2024-03-08T20:34:28.830 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/6lowpan/nhc_udp.ko 2024-03-08T20:34:28.831 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/802 2024-03-08T20:34:28.831 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/802/garp.ko 2024-03-08T20:34:28.831 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/802/mrp.ko 2024-03-08T20:34:28.831 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/802/p8022.ko 2024-03-08T20:34:28.831 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/802/psnap.ko 2024-03-08T20:34:28.831 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/802/stp.ko 2024-03-08T20:34:28.831 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/8021q 2024-03-08T20:34:28.831 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/8021q/8021q.ko 2024-03-08T20:34:28.831 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/9p 2024-03-08T20:34:28.831 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/9p/9pnet.ko 2024-03-08T20:34:28.831 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/9p/9pnet_fd.ko 2024-03-08T20:34:28.831 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/9p/9pnet_rdma.ko 2024-03-08T20:34:28.831 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/9p/9pnet_virtio.ko 2024-03-08T20:34:28.831 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/batman-adv 2024-03-08T20:34:28.832 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/batman-adv/batman-adv.ko 2024-03-08T20:34:28.832 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge 2024-03-08T20:34:28.832 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/br_netfilter.ko 2024-03-08T20:34:28.832 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/bridge.ko 2024-03-08T20:34:28.832 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter 2024-03-08T20:34:28.832 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-03-08T20:34:28.832 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebt_among.ko 2024-03-08T20:34:28.832 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebt_arp.ko 2024-03-08T20:34:28.832 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-03-08T20:34:28.832 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-03-08T20:34:28.832 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebt_ip.ko 2024-03-08T20:34:28.832 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-03-08T20:34:28.832 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebt_limit.ko 2024-03-08T20:34:28.832 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebt_log.ko 2024-03-08T20:34:28.833 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebt_mark.ko 2024-03-08T20:34:28.833 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-03-08T20:34:28.833 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-03-08T20:34:28.833 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-03-08T20:34:28.833 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-03-08T20:34:28.833 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebt_snat.ko 2024-03-08T20:34:28.833 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebt_stp.ko 2024-03-08T20:34:28.833 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-03-08T20:34:28.833 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-03-08T20:34:28.833 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-03-08T20:34:28.833 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-03-08T20:34:28.833 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebtables.ko 2024-03-08T20:34:28.833 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-03-08T20:34:28.834 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-03-08T20:34:28.834 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-03-08T20:34:28.834 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ceph 2024-03-08T20:34:28.834 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ceph/libceph.ko 2024-03-08T20:34:28.834 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/core 2024-03-08T20:34:28.834 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/core/failover.ko 2024-03-08T20:34:28.834 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/core/pktgen.ko 2024-03-08T20:34:28.834 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/dccp 2024-03-08T20:34:28.834 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/dccp/dccp.ko 2024-03-08T20:34:28.834 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/dccp/dccp_diag.ko 2024-03-08T20:34:28.834 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/dccp/dccp_ipv4.ko 2024-03-08T20:34:28.835 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/dccp/dccp_ipv6.ko 2024-03-08T20:34:28.835 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/dns_resolver 2024-03-08T20:34:28.835 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/dns_resolver/dns_resolver.ko 2024-03-08T20:34:28.835 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/dsa 2024-03-08T20:34:28.835 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/dsa/dsa_core.ko 2024-03-08T20:34:28.835 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/dsa/tag_brcm.ko 2024-03-08T20:34:28.835 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/dsa/tag_dsa.ko 2024-03-08T20:34:28.835 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/dsa/tag_none.ko 2024-03-08T20:34:28.835 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/dsa/tag_trailer.ko 2024-03-08T20:34:28.835 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ieee802154 2024-03-08T20:34:28.835 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ieee802154/6lowpan 2024-03-08T20:34:28.835 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-03-08T20:34:28.835 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ieee802154/ieee802154.ko 2024-03-08T20:34:28.835 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ieee802154/ieee802154_socket.ko 2024-03-08T20:34:28.836 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4 2024-03-08T20:34:28.836 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/ah4.ko 2024-03-08T20:34:28.836 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/esp4.ko 2024-03-08T20:34:28.836 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/fou.ko 2024-03-08T20:34:28.836 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/gre.ko 2024-03-08T20:34:28.836 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/inet_diag.ko 2024-03-08T20:34:28.836 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/ip_gre.ko 2024-03-08T20:34:28.836 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/ip_tunnel.ko 2024-03-08T20:34:28.836 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/ip_vti.ko 2024-03-08T20:34:28.836 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/ipcomp.ko 2024-03-08T20:34:28.836 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/ipip.ko 2024-03-08T20:34:28.836 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter 2024-03-08T20:34:28.836 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/arp_tables.ko 2024-03-08T20:34:28.836 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-03-08T20:34:28.836 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-03-08T20:34:28.837 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-03-08T20:34:28.837 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-03-08T20:34:28.837 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-03-08T20:34:28.837 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-03-08T20:34:28.837 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-03-08T20:34:28.837 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-03-08T20:34:28.837 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-03-08T20:34:28.837 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/iptable_security.ko 2024-03-08T20:34:28.837 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-03-08T20:34:28.837 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-03-08T20:34:28.837 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-03-08T20:34:28.837 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-03-08T20:34:28.837 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-03-08T20:34:28.838 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-03-08T20:34:28.838 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-03-08T20:34:28.838 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-03-08T20:34:28.838 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-03-08T20:34:28.838 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-03-08T20:34:28.838 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/tcp_bic.ko 2024-03-08T20:34:28.838 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/tcp_dctcp.ko 2024-03-08T20:34:28.838 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/tcp_diag.ko 2024-03-08T20:34:28.838 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/tcp_highspeed.ko 2024-03-08T20:34:28.838 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/tcp_htcp.ko 2024-03-08T20:34:28.839 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/tcp_hybla.ko 2024-03-08T20:34:28.839 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/tcp_illinois.ko 2024-03-08T20:34:28.839 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/tcp_lp.ko 2024-03-08T20:34:28.839 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/tcp_scalable.ko 2024-03-08T20:34:28.839 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/tcp_vegas.ko 2024-03-08T20:34:28.839 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/tcp_veno.ko 2024-03-08T20:34:28.839 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/tcp_westwood.ko 2024-03-08T20:34:28.839 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/tcp_yeah.ko 2024-03-08T20:34:28.839 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/tunnel4.ko 2024-03-08T20:34:28.839 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/udp_diag.ko 2024-03-08T20:34:28.839 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/udp_tunnel.ko 2024-03-08T20:34:28.839 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/xfrm4_tunnel.ko 2024-03-08T20:34:28.839 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6 2024-03-08T20:34:28.840 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/ah6.ko 2024-03-08T20:34:28.840 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/esp6.ko 2024-03-08T20:34:28.840 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/fou6.ko 2024-03-08T20:34:28.840 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/ip6_tunnel.ko 2024-03-08T20:34:28.840 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-03-08T20:34:28.840 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/ip6_vti.ko 2024-03-08T20:34:28.840 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/ipcomp6.ko 2024-03-08T20:34:28.840 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter 2024-03-08T20:34:28.840 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-03-08T20:34:28.840 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-03-08T20:34:28.840 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-03-08T20:34:28.840 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-03-08T20:34:28.840 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-03-08T20:34:28.840 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-03-08T20:34:28.841 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-03-08T20:34:28.841 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-03-08T20:34:28.841 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-03-08T20:34:28.841 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-03-08T20:34:28.841 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-03-08T20:34:28.841 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-03-08T20:34:28.841 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-03-08T20:34:28.841 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-03-08T20:34:28.841 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-03-08T20:34:28.841 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-03-08T20:34:28.841 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-03-08T20:34:28.841 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-03-08T20:34:28.841 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-03-08T20:34:28.841 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-03-08T20:34:28.842 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-03-08T20:34:28.842 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-03-08T20:34:28.842 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-03-08T20:34:28.842 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-03-08T20:34:28.842 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/sit.ko 2024-03-08T20:34:28.842 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/tunnel6.ko 2024-03-08T20:34:28.842 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/xfrm6_tunnel.ko 2024-03-08T20:34:28.842 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/key 2024-03-08T20:34:28.842 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/key/af_key.ko 2024-03-08T20:34:28.842 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/l2tp 2024-03-08T20:34:28.842 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/l2tp/l2tp_core.ko 2024-03-08T20:34:28.842 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/l2tp/l2tp_debugfs.ko 2024-03-08T20:34:28.843 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/l2tp/l2tp_eth.ko 2024-03-08T20:34:28.843 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/l2tp/l2tp_ip.ko 2024-03-08T20:34:28.843 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/l2tp/l2tp_ip6.ko 2024-03-08T20:34:28.843 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/l2tp/l2tp_netlink.ko 2024-03-08T20:34:28.843 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/l2tp/l2tp_ppp.ko 2024-03-08T20:34:28.843 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/llc 2024-03-08T20:34:28.843 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/llc/llc.ko 2024-03-08T20:34:28.843 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/mac802154 2024-03-08T20:34:28.843 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/mac802154/mac802154.ko 2024-03-08T20:34:28.843 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/mpls 2024-03-08T20:34:28.843 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/mpls/mpls_gso.ko 2024-03-08T20:34:28.843 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter 2024-03-08T20:34:28.843 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipset 2024-03-08T20:34:28.843 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipset/ip_set.ko 2024-03-08T20:34:28.844 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-03-08T20:34:28.844 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-03-08T20:34:28.844 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-03-08T20:34:28.844 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-03-08T20:34:28.844 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-03-08T20:34:28.844 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-03-08T20:34:28.844 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-03-08T20:34:28.844 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-03-08T20:34:28.844 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-03-08T20:34:28.844 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-03-08T20:34:28.844 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-03-08T20:34:28.844 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-03-08T20:34:28.844 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-03-08T20:34:28.845 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-03-08T20:34:28.845 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-03-08T20:34:28.845 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipvs 2024-03-08T20:34:28.845 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipvs/ip_vs.ko 2024-03-08T20:34:28.845 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-03-08T20:34:28.845 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-03-08T20:34:28.845 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-03-08T20:34:28.845 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-03-08T20:34:28.845 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-03-08T20:34:28.845 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-03-08T20:34:28.845 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-03-08T20:34:28.845 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-03-08T20:34:28.845 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-03-08T20:34:28.846 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-03-08T20:34:28.846 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-03-08T20:34:28.846 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-03-08T20:34:28.846 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-03-08T20:34:28.846 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_conncount.ko 2024-03-08T20:34:28.846 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_conntrack.ko 2024-03-08T20:34:28.846 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-03-08T20:34:28.846 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-03-08T20:34:28.846 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-03-08T20:34:28.846 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_conntrack_h323.ko 2024-03-08T20:34:28.846 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_conntrack_irc.ko 2024-03-08T20:34:28.846 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-03-08T20:34:28.846 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-03-08T20:34:28.847 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-03-08T20:34:28.847 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_conntrack_sane.ko 2024-03-08T20:34:28.847 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_conntrack_sip.ko 2024-03-08T20:34:28.847 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-03-08T20:34:28.847 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-03-08T20:34:28.847 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_dup_netdev.ko 2024-03-08T20:34:28.847 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_flow_table.ko 2024-03-08T20:34:28.847 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_flow_table_inet.ko 2024-03-08T20:34:28.847 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_log_syslog.ko 2024-03-08T20:34:28.847 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_nat.ko 2024-03-08T20:34:28.847 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_nat_amanda.ko 2024-03-08T20:34:28.847 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_nat_ftp.ko 2024-03-08T20:34:28.847 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_nat_irc.ko 2024-03-08T20:34:28.847 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_nat_sip.ko 2024-03-08T20:34:28.848 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_nat_tftp.ko 2024-03-08T20:34:28.848 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_synproxy_core.ko 2024-03-08T20:34:28.848 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_tables.ko 2024-03-08T20:34:28.848 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nfnetlink.ko 2024-03-08T20:34:28.848 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nfnetlink_acct.ko 2024-03-08T20:34:28.848 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nfnetlink_log.ko 2024-03-08T20:34:28.848 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nfnetlink_osf.ko 2024-03-08T20:34:28.848 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nfnetlink_queue.ko 2024-03-08T20:34:28.848 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nft_chain_nat.ko 2024-03-08T20:34:28.848 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nft_compat.ko 2024-03-08T20:34:28.848 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nft_ct.ko 2024-03-08T20:34:28.848 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nft_dup_netdev.ko 2024-03-08T20:34:28.848 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nft_fib.ko 2024-03-08T20:34:28.848 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nft_fwd_netdev.ko 2024-03-08T20:34:28.848 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nft_hash.ko 2024-03-08T20:34:28.849 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nft_limit.ko 2024-03-08T20:34:28.849 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nft_log.ko 2024-03-08T20:34:28.849 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nft_masq.ko 2024-03-08T20:34:28.849 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nft_nat.ko 2024-03-08T20:34:28.849 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nft_numgen.ko 2024-03-08T20:34:28.849 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nft_queue.ko 2024-03-08T20:34:28.849 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nft_quota.ko 2024-03-08T20:34:28.849 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nft_redir.ko 2024-03-08T20:34:28.849 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nft_reject.ko 2024-03-08T20:34:28.849 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nft_reject_inet.ko 2024-03-08T20:34:28.849 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nft_socket.ko 2024-03-08T20:34:28.849 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nft_tunnel.ko 2024-03-08T20:34:28.849 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_AUDIT.ko 2024-03-08T20:34:28.849 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_CHECKSUM.ko 2024-03-08T20:34:28.849 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_CLASSIFY.ko 2024-03-08T20:34:28.850 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-03-08T20:34:28.850 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_CT.ko 2024-03-08T20:34:28.850 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_DSCP.ko 2024-03-08T20:34:28.850 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_HL.ko 2024-03-08T20:34:28.850 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_HMARK.ko 2024-03-08T20:34:28.850 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_IDLETIMER.ko 2024-03-08T20:34:28.850 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_LED.ko 2024-03-08T20:34:28.850 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_LOG.ko 2024-03-08T20:34:28.850 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_MASQUERADE.ko 2024-03-08T20:34:28.850 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_NETMAP.ko 2024-03-08T20:34:28.850 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_NFLOG.ko 2024-03-08T20:34:28.850 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_NFQUEUE.ko 2024-03-08T20:34:28.850 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_RATEEST.ko 2024-03-08T20:34:28.850 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_REDIRECT.ko 2024-03-08T20:34:28.851 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_SECMARK.ko 2024-03-08T20:34:28.851 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_TCPMSS.ko 2024-03-08T20:34:28.851 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-03-08T20:34:28.851 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_TEE.ko 2024-03-08T20:34:28.851 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_TPROXY.ko 2024-03-08T20:34:28.851 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_TRACE.ko 2024-03-08T20:34:28.851 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_addrtype.ko 2024-03-08T20:34:28.851 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_bpf.ko 2024-03-08T20:34:28.851 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_cgroup.ko 2024-03-08T20:34:28.851 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_cluster.ko 2024-03-08T20:34:28.851 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_comment.ko 2024-03-08T20:34:28.851 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_connbytes.ko 2024-03-08T20:34:28.851 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_connlabel.ko 2024-03-08T20:34:28.851 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_connlimit.ko 2024-03-08T20:34:28.852 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_connmark.ko 2024-03-08T20:34:28.852 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_conntrack.ko 2024-03-08T20:34:28.852 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_cpu.ko 2024-03-08T20:34:28.852 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_dccp.ko 2024-03-08T20:34:28.852 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_devgroup.ko 2024-03-08T20:34:28.852 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_dscp.ko 2024-03-08T20:34:28.852 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_ecn.ko 2024-03-08T20:34:28.852 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_esp.ko 2024-03-08T20:34:28.852 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_hashlimit.ko 2024-03-08T20:34:28.852 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_helper.ko 2024-03-08T20:34:28.852 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_hl.ko 2024-03-08T20:34:28.852 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_ipcomp.ko 2024-03-08T20:34:28.852 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_iprange.ko 2024-03-08T20:34:28.852 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_ipvs.ko 2024-03-08T20:34:28.853 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_l2tp.ko 2024-03-08T20:34:28.853 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_length.ko 2024-03-08T20:34:28.853 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_limit.ko 2024-03-08T20:34:28.853 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_mac.ko 2024-03-08T20:34:28.853 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_mark.ko 2024-03-08T20:34:28.853 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_multiport.ko 2024-03-08T20:34:28.853 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_nat.ko 2024-03-08T20:34:28.853 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_nfacct.ko 2024-03-08T20:34:28.853 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_osf.ko 2024-03-08T20:34:28.853 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_owner.ko 2024-03-08T20:34:28.853 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_physdev.ko 2024-03-08T20:34:28.853 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_pkttype.ko 2024-03-08T20:34:28.853 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_policy.ko 2024-03-08T20:34:28.853 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_quota.ko 2024-03-08T20:34:28.854 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_rateest.ko 2024-03-08T20:34:28.854 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_realm.ko 2024-03-08T20:34:28.854 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_recent.ko 2024-03-08T20:34:28.854 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_sctp.ko 2024-03-08T20:34:28.854 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_set.ko 2024-03-08T20:34:28.854 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_socket.ko 2024-03-08T20:34:28.854 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_state.ko 2024-03-08T20:34:28.854 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_statistic.ko 2024-03-08T20:34:28.854 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_string.ko 2024-03-08T20:34:28.854 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_tcpmss.ko 2024-03-08T20:34:28.854 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_time.ko 2024-03-08T20:34:28.854 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_u32.ko 2024-03-08T20:34:28.854 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netlink 2024-03-08T20:34:28.855 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netlink/netlink_diag.ko 2024-03-08T20:34:28.855 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/nsh 2024-03-08T20:34:28.855 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/nsh/nsh.ko 2024-03-08T20:34:28.855 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/openvswitch 2024-03-08T20:34:28.855 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/openvswitch/openvswitch.ko 2024-03-08T20:34:28.855 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/openvswitch/vport-geneve.ko 2024-03-08T20:34:28.855 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/openvswitch/vport-gre.ko 2024-03-08T20:34:28.855 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/openvswitch/vport-vxlan.ko 2024-03-08T20:34:28.855 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/packet 2024-03-08T20:34:28.855 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/packet/af_packet_diag.ko 2024-03-08T20:34:28.856 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/rds 2024-03-08T20:34:28.856 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/rds/rds.ko 2024-03-08T20:34:28.856 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/rds/rds_rdma.ko 2024-03-08T20:34:28.856 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/rds/rds_tcp.ko 2024-03-08T20:34:28.856 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched 2024-03-08T20:34:28.856 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/act_csum.ko 2024-03-08T20:34:28.856 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/act_gact.ko 2024-03-08T20:34:28.856 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/act_mirred.ko 2024-03-08T20:34:28.856 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/act_nat.ko 2024-03-08T20:34:28.856 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/act_pedit.ko 2024-03-08T20:34:28.856 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/act_police.ko 2024-03-08T20:34:28.856 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/act_simple.ko 2024-03-08T20:34:28.856 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/act_skbedit.ko 2024-03-08T20:34:28.856 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/act_vlan.ko 2024-03-08T20:34:28.857 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/cls_basic.ko 2024-03-08T20:34:28.857 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/cls_bpf.ko 2024-03-08T20:34:28.857 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/cls_flow.ko 2024-03-08T20:34:28.857 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/cls_fw.ko 2024-03-08T20:34:28.857 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/cls_route.ko 2024-03-08T20:34:28.857 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/cls_u32.ko 2024-03-08T20:34:28.857 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/em_cmp.ko 2024-03-08T20:34:28.857 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/em_ipset.ko 2024-03-08T20:34:28.857 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/em_meta.ko 2024-03-08T20:34:28.857 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/em_nbyte.ko 2024-03-08T20:34:28.857 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/em_text.ko 2024-03-08T20:34:28.857 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/em_u32.ko 2024-03-08T20:34:28.857 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_choke.ko 2024-03-08T20:34:28.858 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_codel.ko 2024-03-08T20:34:28.858 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_drr.ko 2024-03-08T20:34:28.858 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_fq.ko 2024-03-08T20:34:28.858 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_gred.ko 2024-03-08T20:34:28.858 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_hfsc.ko 2024-03-08T20:34:28.858 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_hhf.ko 2024-03-08T20:34:28.858 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_htb.ko 2024-03-08T20:34:28.858 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_ingress.ko 2024-03-08T20:34:28.858 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_mqprio.ko 2024-03-08T20:34:28.858 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_mqprio_lib.ko 2024-03-08T20:34:28.858 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_multiq.ko 2024-03-08T20:34:28.858 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_netem.ko 2024-03-08T20:34:28.859 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_pie.ko 2024-03-08T20:34:28.859 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_plug.ko 2024-03-08T20:34:28.859 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_prio.ko 2024-03-08T20:34:28.859 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_qfq.ko 2024-03-08T20:34:28.859 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_red.ko 2024-03-08T20:34:28.859 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_sfb.ko 2024-03-08T20:34:28.859 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_sfq.ko 2024-03-08T20:34:28.859 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_tbf.ko 2024-03-08T20:34:28.859 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_teql.ko 2024-03-08T20:34:28.859 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sctp 2024-03-08T20:34:28.859 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sctp/sctp.ko 2024-03-08T20:34:28.859 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sctp/sctp_diag.ko 2024-03-08T20:34:28.859 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sunrpc 2024-03-08T20:34:28.859 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sunrpc/auth_gss 2024-03-08T20:34:28.860 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-03-08T20:34:28.860 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-03-08T20:34:28.860 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sunrpc/sunrpc.ko 2024-03-08T20:34:28.860 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sunrpc/xprtrdma 2024-03-08T20:34:28.860 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-03-08T20:34:28.860 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/tipc 2024-03-08T20:34:28.860 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/tipc/diag.ko 2024-03-08T20:34:28.860 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/tipc/tipc.ko 2024-03-08T20:34:28.860 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/unix 2024-03-08T20:34:28.860 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/unix/unix_diag.ko 2024-03-08T20:34:28.860 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/vmw_vsock 2024-03-08T20:34:28.860 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-03-08T20:34:28.860 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-03-08T20:34:28.860 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/vmw_vsock/vsock.ko 2024-03-08T20:34:28.860 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/vmw_vsock/vsock_diag.ko 2024-03-08T20:34:28.861 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/vmw_vsock/vsock_loopback.ko 2024-03-08T20:34:28.861 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/xfrm 2024-03-08T20:34:28.861 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/xfrm/xfrm_ipcomp.ko 2024-03-08T20:34:28.861 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/security 2024-03-08T20:34:28.861 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/security/keys 2024-03-08T20:34:28.861 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/security/keys/encrypted-keys 2024-03-08T20:34:28.861 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-03-08T20:34:28.861 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/security/keys/trusted-keys 2024-03-08T20:34:28.861 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/security/keys/trusted-keys/trusted.ko 2024-03-08T20:34:28.861 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/virt 2024-03-08T20:34:28.861 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/virt/lib 2024-03-08T20:34:28.861 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/virt/lib/irqbypass.ko 2024-03-08T20:34:28.861 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/modules.alias 2024-03-08T20:34:28.861 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/modules.alias.bin 2024-03-08T20:34:28.862 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/modules.builtin 2024-03-08T20:34:28.862 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/modules.builtin.alias.bin 2024-03-08T20:34:28.862 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/modules.builtin.bin 2024-03-08T20:34:28.862 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/modules.builtin.modinfo 2024-03-08T20:34:28.862 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/modules.dep 2024-03-08T20:34:28.862 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/modules.dep.bin 2024-03-08T20:34:28.862 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/modules.devname 2024-03-08T20:34:28.862 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/modules.order 2024-03-08T20:34:28.862 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/modules.softdep 2024-03-08T20:34:28.862 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/modules.symbols 2024-03-08T20:34:28.862 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/modules.symbols.bin 2024-03-08T20:34:28.862 INFO:teuthology.orchestra.run.smithi149.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/vmlinuz 2024-03-08T20:34:28.867 DEBUG:teuthology.task.kernel:get_image_version: 6.8.0-rc7-gdd5c1cd17167 2024-03-08T20:34:28.867 DEBUG:teuthology.orchestra.run.smithi149:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-03-08T20:34:28.889 INFO:teuthology.orchestra.run.smithi149.stdout:postinstall scriptlet (using /bin/sh): 2024-03-08T20:34:28.889 INFO:teuthology.orchestra.run.smithi149.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-03-08T20:34:28.890 INFO:teuthology.orchestra.run.smithi149.stdout: /usr/bin/kernel-install add 6.8.0-rc7-gdd5c1cd17167 /lib/modules/6.8.0-rc7-gdd5c1cd17167/vmlinuz 2024-03-08T20:34:28.890 INFO:teuthology.orchestra.run.smithi149.stdout:fi 2024-03-08T20:34:28.890 INFO:teuthology.orchestra.run.smithi149.stdout:for file in vmlinuz System.map config; do 2024-03-08T20:34:28.890 INFO:teuthology.orchestra.run.smithi149.stdout: if ! cmp --silent "/lib/modules/6.8.0-rc7-gdd5c1cd17167/${file}" "/boot/${file}-6.8.0-rc7-gdd5c1cd17167"; then 2024-03-08T20:34:28.890 INFO:teuthology.orchestra.run.smithi149.stdout: cp "/lib/modules/6.8.0-rc7-gdd5c1cd17167/${file}" "/boot/${file}-6.8.0-rc7-gdd5c1cd17167" 2024-03-08T20:34:28.890 INFO:teuthology.orchestra.run.smithi149.stdout: fi 2024-03-08T20:34:28.890 INFO:teuthology.orchestra.run.smithi149.stdout:done 2024-03-08T20:34:28.890 INFO:teuthology.orchestra.run.smithi149.stdout:preuninstall scriptlet (using /bin/sh): 2024-03-08T20:34:28.890 INFO:teuthology.orchestra.run.smithi149.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2024-03-08T20:34:28.890 INFO:teuthology.orchestra.run.smithi149.stdout:new-kernel-pkg --remove 6.8.0-rc7-gdd5c1cd17167 --rminitrd --initrdfile=/boot/initramfs-6.8.0-rc7-gdd5c1cd17167.img 2024-03-08T20:34:28.890 INFO:teuthology.orchestra.run.smithi149.stdout:elif [ -x /usr/bin/kernel-install ]; then 2024-03-08T20:34:28.890 INFO:teuthology.orchestra.run.smithi149.stdout:kernel-install remove 6.8.0-rc7-gdd5c1cd17167 2024-03-08T20:34:28.891 INFO:teuthology.orchestra.run.smithi149.stdout:fi 2024-03-08T20:34:28.891 INFO:teuthology.orchestra.run.smithi149.stdout:postuninstall scriptlet (using /bin/sh): 2024-03-08T20:34:28.891 INFO:teuthology.orchestra.run.smithi149.stdout:if [ -x /sbin/update-bootloader ]; then 2024-03-08T20:34:28.891 INFO:teuthology.orchestra.run.smithi149.stdout:/sbin/update-bootloader --remove 6.8.0-rc7-gdd5c1cd17167 2024-03-08T20:34:28.891 INFO:teuthology.orchestra.run.smithi149.stdout:fi 2024-03-08T20:34:28.892 DEBUG:teuthology.orchestra.run.smithi149:> sudo rpm -qi grub2-tools 2024-03-08T20:34:28.898 INFO:teuthology.orchestra.run.smithi107.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-03-08T20:34:28.899 INFO:teuthology.orchestra.run.smithi107.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-03-08T20:34:28.899 INFO:teuthology.orchestra.run.smithi107.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-03-08T20:34:28.899 INFO:teuthology.orchestra.run.smithi107.stdout:15949e3c88704300882482eddfd7baae-6.8.0-rc7-gdd5c1cd17167.conf 2024-03-08T20:34:28.900 DEBUG:teuthology.orchestra.run.smithi107:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-6.8.0-rc7-gdd5c1cd17167 2024-03-08T20:34:28.966 INFO:teuthology.orchestra.run.smithi149.stdout:Name : grub2-tools 2024-03-08T20:34:28.966 INFO:teuthology.orchestra.run.smithi149.stdout:Epoch : 1 2024-03-08T20:34:28.966 INFO:teuthology.orchestra.run.smithi149.stdout:Version : 2.06 2024-03-08T20:34:28.966 INFO:teuthology.orchestra.run.smithi149.stdout:Release : 68.el9 2024-03-08T20:34:28.966 INFO:teuthology.orchestra.run.smithi149.stdout:Architecture: x86_64 2024-03-08T20:34:28.967 INFO:teuthology.orchestra.run.smithi149.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-03-08T20:34:28.967 INFO:teuthology.orchestra.run.smithi149.stdout:Group : Unspecified 2024-03-08T20:34:28.967 INFO:teuthology.orchestra.run.smithi149.stdout:Size : 8264254 2024-03-08T20:34:28.967 INFO:teuthology.orchestra.run.smithi149.stdout:License : GPLv3+ 2024-03-08T20:34:28.967 INFO:teuthology.orchestra.run.smithi149.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-03-08T20:34:28.967 INFO:teuthology.orchestra.run.smithi149.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-03-08T20:34:28.967 INFO:teuthology.orchestra.run.smithi149.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-03-08T20:34:28.967 INFO:teuthology.orchestra.run.smithi149.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-03-08T20:34:28.967 INFO:teuthology.orchestra.run.smithi149.stdout:Packager : builder@centos.org 2024-03-08T20:34:28.967 INFO:teuthology.orchestra.run.smithi149.stdout:Vendor : CentOS 2024-03-08T20:34:28.967 INFO:teuthology.orchestra.run.smithi149.stdout:URL : http://www.gnu.org/software/grub/ 2024-03-08T20:34:28.967 INFO:teuthology.orchestra.run.smithi149.stdout:Summary : Support tools for GRUB. 2024-03-08T20:34:28.967 INFO:teuthology.orchestra.run.smithi149.stdout:Description : 2024-03-08T20:34:28.967 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-03-08T20:34:28.968 INFO:teuthology.orchestra.run.smithi149.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-03-08T20:34:28.968 INFO:teuthology.orchestra.run.smithi149.stdout:customizable bootloader with modular architecture. It supports a rich 2024-03-08T20:34:28.968 INFO:teuthology.orchestra.run.smithi149.stdout:variety of kernel formats, file systems, computer architectures and 2024-03-08T20:34:28.968 INFO:teuthology.orchestra.run.smithi149.stdout:hardware devices. 2024-03-08T20:34:28.968 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-03-08T20:34:28.968 INFO:teuthology.orchestra.run.smithi149.stdout:This subpackage provides tools for support of all platforms. 2024-03-08T20:34:28.969 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-03-08T20:34:28.969 INFO:teuthology.task.kernel:Updating grub on smithi149 to boot 6.8.0-rc7-gdd5c1cd17167 2024-03-08T20:34:28.969 DEBUG:teuthology.orchestra.run.smithi149:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-03-08T20:34:29.063 DEBUG:teuthology.orchestra.run.smithi107:> sudo shutdown -r now 2024-03-08T20:34:29.530 INFO:teuthology.orchestra.run.smithi149.stderr:Generating grub configuration file ... 2024-03-08T20:34:30.910 INFO:teuthology.orchestra.run.smithi149.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-03-08T20:34:30.942 INFO:teuthology.orchestra.run.smithi149.stderr:done 2024-03-08T20:34:30.952 DEBUG:teuthology.orchestra.run.smithi149:> mktemp 2024-03-08T20:34:30.967 INFO:teuthology.orchestra.run.smithi149.stdout:/tmp/tmp.o1ztlVRKuY 2024-03-08T20:34:30.967 DEBUG:teuthology.orchestra.run.smithi149:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.o1ztlVRKuY 2024-03-08T20:34:31.048 DEBUG:teuthology.orchestra.run.smithi149:> sudo chmod 0666 /tmp/tmp.o1ztlVRKuY 2024-03-08T20:34:31.185 DEBUG:teuthology.orchestra.remote:smithi149:/tmp/tmp.o1ztlVRKuY is 6KB 2024-03-08T20:34:31.233 DEBUG:teuthology.orchestra.run.smithi149:> rm -fr /tmp/tmp.o1ztlVRKuY 2024-03-08T20:34:31.247 DEBUG:teuthology.orchestra.run.smithi149:> sudo /bin/ls /boot/loader/entries || true 2024-03-08T20:34:31.319 INFO:teuthology.orchestra.run.smithi149.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-03-08T20:34:31.319 INFO:teuthology.orchestra.run.smithi149.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-03-08T20:34:31.319 INFO:teuthology.orchestra.run.smithi149.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-03-08T20:34:31.319 INFO:teuthology.orchestra.run.smithi149.stdout:15949e3c88704300882482eddfd7baae-6.8.0-rc7-gdd5c1cd17167.conf 2024-03-08T20:34:31.319 DEBUG:teuthology.orchestra.run.smithi149:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-6.8.0-rc7-gdd5c1cd17167 2024-03-08T20:34:31.473 DEBUG:teuthology.orchestra.run.smithi149:> sudo shutdown -r now 2024-03-08T20:34:51.974 INFO:teuthology.task.kernel:Checking client mon.c for new kernel version... 2024-03-08T20:34:51.975 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi204.front.sepia.ceph.com' 2024-03-08T20:34:51.976 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi204.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-08T20:34:59.071 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-03-08T20:34:59.072 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi107.front.sepia.ceph.com' 2024-03-08T20:34:59.072 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi107.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-08T20:35:01.478 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-03-08T20:35:01.478 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi149.front.sepia.ceph.com' 2024-03-08T20:35:01.479 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi149.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-08T20:35:10.498 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.204 2024-03-08T20:35:17.406 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.107 2024-03-08T20:35:19.502 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi204.front.sepia.ceph.com' 2024-03-08T20:35:19.503 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi204.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-08T20:35:19.970 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.149 2024-03-08T20:35:22.558 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.204 2024-03-08T20:35:26.411 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi107.front.sepia.ceph.com' 2024-03-08T20:35:26.412 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi107.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-08T20:35:28.974 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi149.front.sepia.ceph.com' 2024-03-08T20:35:28.975 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi149.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-08T20:35:29.470 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.107 2024-03-08T20:35:32.030 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.149 2024-03-08T20:35:34.561 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi204.front.sepia.ceph.com' 2024-03-08T20:35:34.562 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi204.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-08T20:35:41.478 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi107.front.sepia.ceph.com' 2024-03-08T20:35:41.479 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi107.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-08T20:35:44.033 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi149.front.sepia.ceph.com' 2024-03-08T20:35:44.034 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi149.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-08T20:35:59.902 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.107 2024-03-08T20:36:14.914 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi107.front.sepia.ceph.com' 2024-03-08T20:36:14.915 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi107.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-08T20:36:17.278 DEBUG:teuthology.orchestra.run.smithi107:> true 2024-03-08T20:36:17.873 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi107.front.sepia.ceph.com' 2024-03-08T20:36:17.873 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "6.8.0-rc7-gdd5c1cd17167"... 2024-03-08T20:36:17.873 DEBUG:teuthology.orchestra.run.smithi107:> uname -r 2024-03-08T20:36:17.911 INFO:teuthology.orchestra.run.smithi107.stdout:6.8.0-rc7-gdd5c1cd17167 2024-03-08T20:36:17.911 DEBUG:teuthology.task.kernel:current kernel version is 6.8.0-rc7-gdd5c1cd17167 vs 6.8.0-rc7-gdd5c1cd17167 2024-03-08T20:36:17.911 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-03-08T20:36:17.911 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-03-08T20:36:18.912 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-03-08T20:36:18.912 DEBUG:teuthology.orchestra.run.smithi107:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-03-08T20:36:19.109 INFO:teuthology.orchestra.run.smithi107.stdout:ttyS1 2024-03-08T20:36:19.167 DEBUG:teuthology.parallel:result is None 2024-03-08T20:36:34.564 DEBUG:teuthology.orchestra.remote:timed out 2024-03-08T20:36:44.036 DEBUG:teuthology.orchestra.remote:timed out 2024-03-08T20:36:49.566 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi204.front.sepia.ceph.com' 2024-03-08T20:36:49.569 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi204.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-08T20:36:50.030 DEBUG:teuthology.orchestra.run.smithi204:> true 2024-03-08T20:36:50.676 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi204.front.sepia.ceph.com' 2024-03-08T20:36:50.677 INFO:teuthology.task.kernel:Checking kernel version of mon.c, want "6.8.0-rc7-gdd5c1cd17167"... 2024-03-08T20:36:50.677 DEBUG:teuthology.orchestra.run.smithi204:> uname -r 2024-03-08T20:36:50.727 INFO:teuthology.orchestra.run.smithi204.stdout:6.8.0-rc7-gdd5c1cd17167 2024-03-08T20:36:50.727 DEBUG:teuthology.task.kernel:current kernel version is 6.8.0-rc7-gdd5c1cd17167 vs 6.8.0-rc7-gdd5c1cd17167 2024-03-08T20:36:50.727 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-03-08T20:36:50.727 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-03-08T20:36:51.728 INFO:teuthology.task.kernel:Enabling kdb on mon.c... 2024-03-08T20:36:51.728 DEBUG:teuthology.orchestra.run.smithi204:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-03-08T20:36:51.863 INFO:teuthology.orchestra.run.smithi204.stdout:ttyS1 2024-03-08T20:36:51.924 DEBUG:teuthology.parallel:result is None 2024-03-08T20:36:59.036 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi149.front.sepia.ceph.com' 2024-03-08T20:36:59.037 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi149.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-08T20:36:59.517 DEBUG:teuthology.orchestra.run.smithi149:> true 2024-03-08T20:37:00.183 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi149.front.sepia.ceph.com' 2024-03-08T20:37:00.183 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "6.8.0-rc7-gdd5c1cd17167"... 2024-03-08T20:37:00.183 DEBUG:teuthology.orchestra.run.smithi149:> uname -r 2024-03-08T20:37:00.236 INFO:teuthology.orchestra.run.smithi149.stdout:6.8.0-rc7-gdd5c1cd17167 2024-03-08T20:37:00.236 DEBUG:teuthology.task.kernel:current kernel version is 6.8.0-rc7-gdd5c1cd17167 vs 6.8.0-rc7-gdd5c1cd17167 2024-03-08T20:37:00.236 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-03-08T20:37:00.236 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-03-08T20:37:01.237 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-03-08T20:37:01.238 DEBUG:teuthology.orchestra.run.smithi149:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-03-08T20:37:01.366 INFO:teuthology.orchestra.run.smithi149.stdout:ttyS1 2024-03-08T20:37:01.428 DEBUG:teuthology.parallel:result is None 2024-03-08T20:37:01.428 INFO:teuthology.run_tasks:Running task internal.base... 2024-03-08T20:37:01.437 INFO:teuthology.task.internal:Creating test directory... 2024-03-08T20:37:01.437 DEBUG:teuthology.orchestra.run.smithi107:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-08T20:37:01.446 DEBUG:teuthology.orchestra.run.smithi149:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-08T20:37:01.453 DEBUG:teuthology.orchestra.run.smithi204:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-08T20:37:01.507 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-03-08T20:37:01.514 INFO:teuthology.run_tasks:Running task internal.archive... 2024-03-08T20:37:01.539 INFO:teuthology.task.internal:Creating archive directory... 2024-03-08T20:37:01.539 DEBUG:teuthology.orchestra.run.smithi107:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-08T20:37:01.546 DEBUG:teuthology.orchestra.run.smithi149:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-08T20:37:01.552 DEBUG:teuthology.orchestra.run.smithi204:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-08T20:37:01.627 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-03-08T20:37:01.634 INFO:teuthology.task.internal:Enabling coredump saving... 2024-03-08T20:37:01.634 DEBUG:teuthology.orchestra.run.smithi107:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-08T20:37:01.640 DEBUG:teuthology.orchestra.run.smithi149:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-08T20:37:01.645 DEBUG:teuthology.orchestra.run.smithi204:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-08T20:37:01.737 INFO:teuthology.orchestra.run.smithi107.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-08T20:37:01.741 INFO:teuthology.orchestra.run.smithi149.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-08T20:37:01.750 INFO:teuthology.orchestra.run.smithi204.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-08T20:37:01.779 INFO:teuthology.orchestra.run.smithi107.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-08T20:37:01.783 INFO:teuthology.orchestra.run.smithi149.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-08T20:37:01.794 INFO:teuthology.orchestra.run.smithi204.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-08T20:37:01.799 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-03-08T20:37:01.810 INFO:teuthology.task.internal:Configuring sudo... 2024-03-08T20:37:01.811 DEBUG:teuthology.orchestra.run.smithi107:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-08T20:37:01.818 DEBUG:teuthology.orchestra.run.smithi149:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-08T20:37:01.825 DEBUG:teuthology.orchestra.run.smithi204:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-08T20:37:01.924 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-03-08T20:37:01.940 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-03-08T20:37:01.940 DEBUG:teuthology.orchestra.run.smithi107:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-08T20:37:01.948 DEBUG:teuthology.orchestra.run.smithi149:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-08T20:37:01.955 DEBUG:teuthology.orchestra.run.smithi204:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-08T20:37:02.009 DEBUG:teuthology.orchestra.run.smithi107:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-08T20:37:02.088 DEBUG:teuthology.orchestra.run.smithi107:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-08T20:37:02.205 DEBUG:teuthology.orchestra.run.smithi107:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-08T20:37:02.291 DEBUG:teuthology.orchestra.run.smithi107:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-08T20:37:02.383 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-03-08T20:37:02.384 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-08T20:37:02.477 DEBUG:teuthology.orchestra.run.smithi149:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-08T20:37:02.564 DEBUG:teuthology.orchestra.run.smithi149:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-08T20:37:02.702 DEBUG:teuthology.orchestra.run.smithi149:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-08T20:37:02.788 DEBUG:teuthology.orchestra.run.smithi149:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-08T20:37:02.879 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-03-08T20:37:02.880 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-08T20:37:02.972 DEBUG:teuthology.orchestra.run.smithi204:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-08T20:37:03.056 DEBUG:teuthology.orchestra.run.smithi204:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-08T20:37:03.183 DEBUG:teuthology.orchestra.run.smithi204:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-08T20:37:03.268 DEBUG:teuthology.orchestra.run.smithi204:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-08T20:37:03.365 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-03-08T20:37:03.365 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-08T20:37:03.459 DEBUG:teuthology.orchestra.run.smithi107:> sudo service rsyslog restart 2024-03-08T20:37:03.466 DEBUG:teuthology.orchestra.run.smithi149:> sudo service rsyslog restart 2024-03-08T20:37:03.473 DEBUG:teuthology.orchestra.run.smithi204:> sudo service rsyslog restart 2024-03-08T20:37:03.592 INFO:teuthology.orchestra.run.smithi149.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-08T20:37:03.594 INFO:teuthology.orchestra.run.smithi107.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-08T20:37:03.624 INFO:teuthology.orchestra.run.smithi204.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-08T20:37:04.109 INFO:teuthology.run_tasks:Running task internal.timer... 2024-03-08T20:37:04.116 INFO:teuthology.task.internal:Starting timer... 2024-03-08T20:37:04.116 INFO:teuthology.run_tasks:Running task pcp... 2024-03-08T20:37:04.191 INFO:teuthology.run_tasks:Running task selinux... 2024-03-08T20:37:04.220 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-03-08T20:37:04.221 DEBUG:teuthology.orchestra.run.smithi107:> sudo service auditd rotate 2024-03-08T20:37:04.337 INFO:teuthology.orchestra.run.smithi107.stdout:Rotating logs: 2024-03-08T20:37:04.342 DEBUG:teuthology.orchestra.run.smithi149:> sudo service auditd rotate 2024-03-08T20:37:04.479 INFO:teuthology.orchestra.run.smithi149.stdout:Rotating logs: 2024-03-08T20:37:04.484 DEBUG:teuthology.orchestra.run.smithi204:> sudo service auditd rotate 2024-03-08T20:37:04.617 INFO:teuthology.orchestra.run.smithi204.stdout:Rotating logs: 2024-03-08T20:37:04.621 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-03-08T20:37:04.622 DEBUG:teuthology.orchestra.run.smithi107:> /usr/sbin/getenforce 2024-03-08T20:37:04.687 INFO:teuthology.orchestra.run.smithi107.stdout:Permissive 2024-03-08T20:37:04.688 DEBUG:teuthology.orchestra.run.smithi149:> /usr/sbin/getenforce 2024-03-08T20:37:04.735 INFO:teuthology.orchestra.run.smithi149.stdout:Permissive 2024-03-08T20:37:04.735 DEBUG:teuthology.orchestra.run.smithi204:> /usr/sbin/getenforce 2024-03-08T20:37:04.787 INFO:teuthology.orchestra.run.smithi204.stdout:Permissive 2024-03-08T20:37:04.787 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi107.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi149.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi204.front.sepia.ceph.com': 'permissive'} 2024-03-08T20:37:04.787 DEBUG:teuthology.orchestra.run.smithi107:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-08T20:37:04.872 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-08T20:37:04.872 DEBUG:teuthology.orchestra.run.smithi149:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-08T20:37:04.954 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-08T20:37:04.954 DEBUG:teuthology.orchestra.run.smithi204:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-08T20:37:05.032 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-08T20:37:05.032 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-03-08T20:37:05.033 DEBUG:teuthology.orchestra.run.smithi107:> sudo /usr/sbin/setenforce permissive 2024-03-08T20:37:05.124 DEBUG:teuthology.orchestra.run.smithi149:> sudo /usr/sbin/setenforce permissive 2024-03-08T20:37:05.206 DEBUG:teuthology.orchestra.run.smithi204:> sudo /usr/sbin/setenforce permissive 2024-03-08T20:37:05.299 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-03-08T20:37:05.309 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-03-08T20:37:05.315 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-03-08T20:37:05.445 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-03-08T20:37:05.461 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-03-08T20:37:05.462 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi107.front.sepia.ceph.com,smithi149.front.sepia.ceph.com,smithi204.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-03-08T20:43:02.450 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi107.front.sepia.ceph.com'), Remote(name='ubuntu@smithi149.front.sepia.ceph.com'), Remote(name='ubuntu@smithi204.front.sepia.ceph.com')] 2024-03-08T20:43:02.451 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi107.front.sepia.ceph.com' 2024-03-08T20:43:02.452 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi107.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-08T20:43:02.601 DEBUG:teuthology.orchestra.run.smithi107:> true 2024-03-08T20:43:02.741 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi107.front.sepia.ceph.com' 2024-03-08T20:43:02.742 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi149.front.sepia.ceph.com' 2024-03-08T20:43:02.742 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi149.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-08T20:43:02.888 DEBUG:teuthology.orchestra.run.smithi149:> true 2024-03-08T20:43:03.032 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi149.front.sepia.ceph.com' 2024-03-08T20:43:03.032 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi204.front.sepia.ceph.com' 2024-03-08T20:43:03.032 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi204.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-08T20:43:03.175 DEBUG:teuthology.orchestra.run.smithi204:> true 2024-03-08T20:43:03.326 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi204.front.sepia.ceph.com' 2024-03-08T20:43:03.327 INFO:teuthology.run_tasks:Running task clock... 2024-03-08T20:43:03.337 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-03-08T20:43:03.338 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-08T20:43:03.338 DEBUG:teuthology.orchestra.run.smithi107:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-08T20:43:03.346 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-08T20:43:03.346 DEBUG:teuthology.orchestra.run.smithi149:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-08T20:43:03.353 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-08T20:43:03.353 DEBUG:teuthology.orchestra.run.smithi204:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-08T20:43:03.439 INFO:teuthology.orchestra.run.smithi107.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-08T20:43:03.452 INFO:teuthology.orchestra.run.smithi149.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-08T20:43:03.455 INFO:teuthology.orchestra.run.smithi204.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-08T20:43:03.487 INFO:teuthology.orchestra.run.smithi107.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-08T20:43:03.502 INFO:teuthology.orchestra.run.smithi149.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-08T20:43:03.505 INFO:teuthology.orchestra.run.smithi204.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-08T20:43:03.584 INFO:teuthology.orchestra.run.smithi107.stderr:sudo: ntpd: command not found 2024-03-08T20:43:03.592 INFO:teuthology.orchestra.run.smithi204.stderr:sudo: ntpd: command not found 2024-03-08T20:43:03.594 INFO:teuthology.orchestra.run.smithi149.stderr:sudo: ntpd: command not found 2024-03-08T20:43:03.623 INFO:teuthology.orchestra.run.smithi107.stdout:506 Cannot talk to daemon 2024-03-08T20:43:03.632 INFO:teuthology.orchestra.run.smithi204.stdout:506 Cannot talk to daemon 2024-03-08T20:43:03.633 INFO:teuthology.orchestra.run.smithi149.stdout:506 Cannot talk to daemon 2024-03-08T20:43:03.675 INFO:teuthology.orchestra.run.smithi107.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-08T20:43:03.683 INFO:teuthology.orchestra.run.smithi204.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-08T20:43:03.685 INFO:teuthology.orchestra.run.smithi149.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-08T20:43:03.725 INFO:teuthology.orchestra.run.smithi107.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-08T20:43:03.734 INFO:teuthology.orchestra.run.smithi204.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-08T20:43:03.738 INFO:teuthology.orchestra.run.smithi149.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-08T20:43:03.858 INFO:teuthology.orchestra.run.smithi107.stderr:bash: line 1: ntpq: command not found 2024-03-08T20:43:03.864 INFO:teuthology.orchestra.run.smithi204.stderr:bash: line 1: ntpq: command not found 2024-03-08T20:43:03.866 INFO:teuthology.orchestra.run.smithi107.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-08T20:43:03.866 INFO:teuthology.orchestra.run.smithi107.stdout:=============================================================================== 2024-03-08T20:43:03.866 INFO:teuthology.orchestra.run.smithi107.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-08T20:43:03.866 INFO:teuthology.orchestra.run.smithi107.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-08T20:43:03.866 INFO:teuthology.orchestra.run.smithi107.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-08T20:43:03.866 INFO:teuthology.orchestra.run.smithi107.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-08T20:43:03.872 INFO:teuthology.orchestra.run.smithi204.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-08T20:43:03.872 INFO:teuthology.orchestra.run.smithi204.stdout:=============================================================================== 2024-03-08T20:43:03.872 INFO:teuthology.orchestra.run.smithi204.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-08T20:43:03.872 INFO:teuthology.orchestra.run.smithi204.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-08T20:43:03.872 INFO:teuthology.orchestra.run.smithi204.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-08T20:43:03.872 INFO:teuthology.orchestra.run.smithi204.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-08T20:43:03.872 INFO:teuthology.orchestra.run.smithi149.stderr:bash: line 1: ntpq: command not found 2024-03-08T20:43:03.881 INFO:teuthology.orchestra.run.smithi149.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-08T20:43:03.881 INFO:teuthology.orchestra.run.smithi149.stdout:=============================================================================== 2024-03-08T20:43:03.882 INFO:teuthology.orchestra.run.smithi149.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-08T20:43:03.882 INFO:teuthology.orchestra.run.smithi149.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-08T20:43:03.882 INFO:teuthology.orchestra.run.smithi149.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-08T20:43:03.882 INFO:teuthology.orchestra.run.smithi149.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-08T20:43:03.883 INFO:teuthology.run_tasks:Running task install... 2024-03-08T20:43:03.892 DEBUG:teuthology.task.install:project ceph 2024-03-08T20:43:03.892 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '7d63482aa89628d733e298984a762389b625c761'}} 2024-03-08T20:43:03.892 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '7d63482aa89628d733e298984a762389b625c761'} 2024-03-08T20:43:03.892 INFO:teuthology.task.install:Using flavor: default 2024-03-08T20:43:03.898 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-03-08T20:43:03.898 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-03-08T20:43:03.898 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '7d63482aa89628d733e298984a762389b625c761', 'tag': None, 'wait_for_package': False} 2024-03-08T20:43:03.899 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=7d63482aa89628d733e298984a762389b625c761 2024-03-08T20:43:03.901 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '7d63482aa89628d733e298984a762389b625c761', 'tag': None, 'wait_for_package': False} 2024-03-08T20:43:03.901 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=7d63482aa89628d733e298984a762389b625c761 2024-03-08T20:43:03.902 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '7d63482aa89628d733e298984a762389b625c761', 'tag': None, 'wait_for_package': False} 2024-03-08T20:43:03.903 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=7d63482aa89628d733e298984a762389b625c761 2024-03-08T20:43:04.057 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/rishabh-2024mar8/7d63482aa89628d733e298984a762389b625c761/centos/9/flavors/default/ 2024-03-08T20:43:04.057 INFO:teuthology.task.install.rpm:Package version is 19.0.0-1965.g7d63482a 2024-03-08T20:43:04.070 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/rishabh-2024mar8/7d63482aa89628d733e298984a762389b625c761/centos/9/flavors/default/ 2024-03-08T20:43:04.070 INFO:teuthology.task.install.rpm:Package version is 19.0.0-1965.g7d63482a 2024-03-08T20:43:04.074 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/rishabh-2024mar8/7d63482aa89628d733e298984a762389b625c761/centos/9/flavors/default/ 2024-03-08T20:43:04.074 INFO:teuthology.task.install.rpm:Package version is 19.0.0-1965.g7d63482a 2024-03-08T20:43:04.183 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/rishabh-2024mar8/7d63482aa89628d733e298984a762389b625c761/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/rishabh-2024mar8/7d63482aa89628d733e298984a762389b625c761/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/rishabh-2024mar8/7d63482aa89628d733e298984a762389b625c761/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-08T20:43:04.183 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-03-08T20:43:04.183 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-08T20:43:04.200 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/rishabh-2024mar8/7d63482aa89628d733e298984a762389b625c761/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/rishabh-2024mar8/7d63482aa89628d733e298984a762389b625c761/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/rishabh-2024mar8/7d63482aa89628d733e298984a762389b625c761/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-08T20:43:04.200 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-03-08T20:43:04.200 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-08T20:43:04.286 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-03-08T20:43:04.286 DEBUG:teuthology.orchestra.run.smithi107:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/7d63482aa89628d733e298984a762389b625c761/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-08T20:43:04.300 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-03-08T20:43:04.300 DEBUG:teuthology.orchestra.run.smithi204:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/7d63482aa89628d733e298984a762389b625c761/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-08T20:43:04.323 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/rishabh-2024mar8/7d63482aa89628d733e298984a762389b625c761/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/rishabh-2024mar8/7d63482aa89628d733e298984a762389b625c761/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/rishabh-2024mar8/7d63482aa89628d733e298984a762389b625c761/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-08T20:43:04.323 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-03-08T20:43:04.324 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-08T20:43:04.421 DEBUG:teuthology.orchestra.run.smithi107:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-08T20:43:04.422 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-03-08T20:43:04.422 DEBUG:teuthology.orchestra.run.smithi149:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/7d63482aa89628d733e298984a762389b625c761/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-08T20:43:04.437 DEBUG:teuthology.orchestra.run.smithi204:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-08T20:43:04.547 DEBUG:teuthology.orchestra.run.smithi107:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-08T20:43:04.560 DEBUG:teuthology.orchestra.run.smithi204:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-08T20:43:04.562 DEBUG:teuthology.orchestra.run.smithi149:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-08T20:43:04.624 INFO:teuthology.orchestra.run.smithi107.stdout:check_obsoletes = 1 2024-03-08T20:43:04.628 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum clean all 2024-03-08T20:43:04.646 INFO:teuthology.orchestra.run.smithi204.stdout:check_obsoletes = 1 2024-03-08T20:43:04.651 DEBUG:teuthology.orchestra.run.smithi204:> sudo yum clean all 2024-03-08T20:43:04.694 DEBUG:teuthology.orchestra.run.smithi149:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-08T20:43:04.785 INFO:teuthology.orchestra.run.smithi149.stdout:check_obsoletes = 1 2024-03-08T20:43:04.789 DEBUG:teuthology.orchestra.run.smithi149:> sudo yum clean all 2024-03-08T20:43:04.985 INFO:teuthology.orchestra.run.smithi107.stdout:74 files removed 2024-03-08T20:43:05.005 INFO:teuthology.orchestra.run.smithi204.stdout:74 files removed 2024-03-08T20:43:05.023 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-03-08T20:43:05.044 DEBUG:teuthology.orchestra.run.smithi204:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-03-08T20:43:05.148 INFO:teuthology.orchestra.run.smithi149.stdout:74 files removed 2024-03-08T20:43:05.191 DEBUG:teuthology.orchestra.run.smithi149:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-03-08T20:43:05.917 INFO:teuthology.orchestra.run.smithi204.stdout:ceph packages for x86_64 227 kB/s | 82 kB 00:00 2024-03-08T20:43:06.343 INFO:teuthology.orchestra.run.smithi149.stdout:ceph packages for x86_64 129 kB/s | 82 kB 00:00 2024-03-08T20:43:06.386 INFO:teuthology.orchestra.run.smithi107.stdout:ceph packages for x86_64 95 kB/s | 82 kB 00:00 2024-03-08T20:43:06.743 INFO:teuthology.orchestra.run.smithi204.stdout:ceph noarch packages 14 kB/s | 10 kB 00:00 2024-03-08T20:43:07.006 INFO:teuthology.orchestra.run.smithi107.stdout:ceph noarch packages 19 kB/s | 10 kB 00:00 2024-03-08T20:43:07.018 INFO:teuthology.orchestra.run.smithi149.stdout:ceph noarch packages 17 kB/s | 10 kB 00:00 2024-03-08T20:43:07.201 INFO:teuthology.orchestra.run.smithi204.stdout:ceph source packages 5.0 kB/s | 1.9 kB 00:00 2024-03-08T20:43:07.669 INFO:teuthology.orchestra.run.smithi149.stdout:ceph source packages 3.3 kB/s | 1.9 kB 00:00 2024-03-08T20:43:07.679 INFO:teuthology.orchestra.run.smithi107.stdout:ceph source packages 3.2 kB/s | 1.9 kB 00:00 2024-03-08T20:43:08.422 INFO:teuthology.orchestra.run.smithi204.stdout:CentOS-9 - AppStream 5.8 MB/s | 6.7 MB 00:01 2024-03-08T20:43:08.774 INFO:teuthology.orchestra.run.smithi107.stdout:CentOS-9 - AppStream 6.5 MB/s | 6.7 MB 00:01 2024-03-08T20:43:08.788 INFO:teuthology.orchestra.run.smithi149.stdout:CentOS-9 - AppStream 6.4 MB/s | 6.7 MB 00:01 2024-03-08T20:43:10.730 INFO:teuthology.orchestra.run.smithi204.stdout:CentOS-9 - BaseOS 2.0 MB/s | 2.0 MB 00:00 2024-03-08T20:43:10.841 INFO:teuthology.orchestra.run.smithi149.stdout:CentOS-9 - BaseOS 2.6 MB/s | 2.0 MB 00:00 2024-03-08T20:43:10.891 INFO:teuthology.orchestra.run.smithi107.stdout:CentOS-9 - BaseOS 2.4 MB/s | 2.0 MB 00:00 2024-03-08T20:43:12.402 INFO:teuthology.orchestra.run.smithi149.stdout:CentOS Stream 9 - BaseOS 6.9 MB/s | 8.0 MB 00:01 2024-03-08T20:43:12.459 INFO:teuthology.orchestra.run.smithi204.stdout:CentOS Stream 9 - BaseOS 6.1 MB/s | 8.0 MB 00:01 2024-03-08T20:43:13.930 INFO:teuthology.orchestra.run.smithi107.stdout:CentOS Stream 9 - BaseOS 3.0 MB/s | 8.0 MB 00:02 2024-03-08T20:43:15.311 INFO:teuthology.orchestra.run.smithi149.stdout:CentOS Stream 9 - AppStream 11 MB/s | 19 MB 00:01 2024-03-08T20:43:15.527 INFO:teuthology.orchestra.run.smithi204.stdout:CentOS Stream 9 - AppStream 10 MB/s | 19 MB 00:01 2024-03-08T20:43:20.955 INFO:teuthology.orchestra.run.smithi149.stdout:CentOS Stream 9 - CRB 3.3 MB/s | 6.1 MB 00:01 2024-03-08T20:43:22.854 INFO:teuthology.orchestra.run.smithi149.stdout:CentOS Stream 9 - Extras packages 24 kB/s | 15 kB 00:00 2024-03-08T20:43:25.587 INFO:teuthology.orchestra.run.smithi149.stdout:Extra Packages for Enterprise Linux 8.1 MB/s | 21 MB 00:02 2024-03-08T20:43:31.440 INFO:teuthology.orchestra.run.smithi149.stdout:lab-extras 25 kB/s | 1.7 kB 00:00 2024-03-08T20:43:34.174 INFO:teuthology.orchestra.run.smithi149.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-08T20:43:34.176 INFO:teuthology.orchestra.run.smithi149.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-08T20:43:34.199 INFO:teuthology.orchestra.run.smithi149.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-03-08T20:43:34.201 INFO:teuthology.orchestra.run.smithi149.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-03-08T20:43:34.202 INFO:teuthology.orchestra.run.smithi149.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-03-08T20:43:34.204 INFO:teuthology.orchestra.run.smithi149.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-03-08T20:43:34.206 INFO:teuthology.orchestra.run.smithi149.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-03-08T20:43:34.210 INFO:teuthology.orchestra.run.smithi149.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-03-08T20:43:34.212 INFO:teuthology.orchestra.run.smithi149.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-03-08T20:43:34.300 INFO:teuthology.orchestra.run.smithi149.stdout:Dependencies resolved. 2024-03-08T20:43:34.332 INFO:teuthology.orchestra.run.smithi149.stdout:============================================================================================ 2024-03-08T20:43:34.332 INFO:teuthology.orchestra.run.smithi149.stdout: Package Arch Version Repository Size 2024-03-08T20:43:34.332 INFO:teuthology.orchestra.run.smithi149.stdout:============================================================================================ 2024-03-08T20:43:34.332 INFO:teuthology.orchestra.run.smithi149.stdout:Installing: 2024-03-08T20:43:34.332 INFO:teuthology.orchestra.run.smithi149.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-03-08T20:43:34.332 INFO:teuthology.orchestra.run.smithi149.stdout: ceph x86_64 2:19.0.0-1965.g7d63482a.el9 ceph 6.6 k 2024-03-08T20:43:34.333 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-base x86_64 2:19.0.0-1965.g7d63482a.el9 ceph 5.4 M 2024-03-08T20:43:34.333 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-fuse x86_64 2:19.0.0-1965.g7d63482a.el9 ceph 889 k 2024-03-08T20:43:34.333 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-1965.g7d63482a.el9 ceph 145 k 2024-03-08T20:43:34.333 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mgr x86_64 2:19.0.0-1965.g7d63482a.el9 ceph 1.5 M 2024-03-08T20:43:34.333 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mgr-cephadm noarch 2:19.0.0-1965.g7d63482a.el9 ceph-noarch 139 k 2024-03-08T20:43:34.333 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mgr-dashboard noarch 2:19.0.0-1965.g7d63482a.el9 ceph-noarch 1.8 M 2024-03-08T20:43:34.333 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-1965.g7d63482a.el9 ceph-noarch 7.4 M 2024-03-08T20:43:34.333 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mgr-rook noarch 2:19.0.0-1965.g7d63482a.el9 ceph-noarch 49 k 2024-03-08T20:43:34.333 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-radosgw x86_64 2:19.0.0-1965.g7d63482a.el9 ceph 9.8 M 2024-03-08T20:43:34.333 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-test x86_64 2:19.0.0-1965.g7d63482a.el9 ceph 47 M 2024-03-08T20:43:34.333 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-volume noarch 2:19.0.0-1965.g7d63482a.el9 ceph-noarch 264 k 2024-03-08T20:43:34.333 INFO:teuthology.orchestra.run.smithi149.stdout: cephadm noarch 2:19.0.0-1965.g7d63482a.el9 ceph-noarch 748 k 2024-03-08T20:43:34.333 INFO:teuthology.orchestra.run.smithi149.stdout: cephfs-mirror x86_64 2:19.0.0-1965.g7d63482a.el9 ceph 220 k 2024-03-08T20:43:34.334 INFO:teuthology.orchestra.run.smithi149.stdout: cephfs-shell x86_64 2:19.0.0-1965.g7d63482a.el9 ceph 27 k 2024-03-08T20:43:34.334 INFO:teuthology.orchestra.run.smithi149.stdout: cephfs-top noarch 2:19.0.0-1965.g7d63482a.el9 ceph-noarch 18 k 2024-03-08T20:43:34.334 INFO:teuthology.orchestra.run.smithi149.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-03-08T20:43:34.334 INFO:teuthology.orchestra.run.smithi149.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-03-08T20:43:34.334 INFO:teuthology.orchestra.run.smithi149.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-03-08T20:43:34.334 INFO:teuthology.orchestra.run.smithi149.stdout: libcephfs-devel x86_64 2:19.0.0-1965.g7d63482a.el9 ceph 31 k 2024-03-08T20:43:34.334 INFO:teuthology.orchestra.run.smithi149.stdout: libcephfs2 x86_64 2:19.0.0-1965.g7d63482a.el9 ceph 745 k 2024-03-08T20:43:34.334 INFO:teuthology.orchestra.run.smithi149.stdout: librados-devel x86_64 2:19.0.0-1965.g7d63482a.el9 ceph 127 k 2024-03-08T20:43:34.334 INFO:teuthology.orchestra.run.smithi149.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-03-08T20:43:34.334 INFO:teuthology.orchestra.run.smithi149.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-03-08T20:43:34.334 INFO:teuthology.orchestra.run.smithi149.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-03-08T20:43:34.334 INFO:teuthology.orchestra.run.smithi149.stdout: postgresql x86_64 13.11-1.el9 CentOS-AppStream 1.5 M 2024-03-08T20:43:34.335 INFO:teuthology.orchestra.run.smithi149.stdout: postgresql-contrib x86_64 13.11-1.el9 CentOS-AppStream 811 k 2024-03-08T20:43:34.335 INFO:teuthology.orchestra.run.smithi149.stdout: postgresql-server x86_64 13.11-1.el9 CentOS-AppStream 5.7 M 2024-03-08T20:43:34.335 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cephfs x86_64 2:19.0.0-1965.g7d63482a.el9 ceph 162 k 2024-03-08T20:43:34.335 INFO:teuthology.orchestra.run.smithi149.stdout: python3-rados x86_64 2:19.0.0-1965.g7d63482a.el9 ceph 320 k 2024-03-08T20:43:34.335 INFO:teuthology.orchestra.run.smithi149.stdout: python3-rbd x86_64 2:19.0.0-1965.g7d63482a.el9 ceph 298 k 2024-03-08T20:43:34.335 INFO:teuthology.orchestra.run.smithi149.stdout: python3-rgw x86_64 2:19.0.0-1965.g7d63482a.el9 ceph 99 k 2024-03-08T20:43:34.335 INFO:teuthology.orchestra.run.smithi149.stdout: rbd-fuse x86_64 2:19.0.0-1965.g7d63482a.el9 ceph 85 k 2024-03-08T20:43:34.335 INFO:teuthology.orchestra.run.smithi149.stdout: rbd-mirror x86_64 2:19.0.0-1965.g7d63482a.el9 ceph 3.1 M 2024-03-08T20:43:34.335 INFO:teuthology.orchestra.run.smithi149.stdout: rbd-nbd x86_64 2:19.0.0-1965.g7d63482a.el9 ceph 171 k 2024-03-08T20:43:34.335 INFO:teuthology.orchestra.run.smithi149.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-03-08T20:43:34.335 INFO:teuthology.orchestra.run.smithi149.stdout:Upgrading: 2024-03-08T20:43:34.335 INFO:teuthology.orchestra.run.smithi149.stdout: NetworkManager x86_64 1:1.46.0-1.el9 baseos 2.3 M 2024-03-08T20:43:34.335 INFO:teuthology.orchestra.run.smithi149.stdout: NetworkManager-libnm x86_64 1:1.46.0-1.el9 baseos 1.8 M 2024-03-08T20:43:34.336 INFO:teuthology.orchestra.run.smithi149.stdout: NetworkManager-team x86_64 1:1.46.0-1.el9 baseos 42 k 2024-03-08T20:43:34.336 INFO:teuthology.orchestra.run.smithi149.stdout: NetworkManager-tui x86_64 1:1.46.0-1.el9 baseos 248 k 2024-03-08T20:43:34.336 INFO:teuthology.orchestra.run.smithi149.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-03-08T20:43:34.336 INFO:teuthology.orchestra.run.smithi149.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-03-08T20:43:34.336 INFO:teuthology.orchestra.run.smithi149.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-03-08T20:43:34.336 INFO:teuthology.orchestra.run.smithi149.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-03-08T20:43:34.336 INFO:teuthology.orchestra.run.smithi149.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-03-08T20:43:34.336 INFO:teuthology.orchestra.run.smithi149.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-03-08T20:43:34.336 INFO:teuthology.orchestra.run.smithi149.stdout: libblkid x86_64 2.37.4-18.el9 baseos 108 k 2024-03-08T20:43:34.336 INFO:teuthology.orchestra.run.smithi149.stdout: libfdisk x86_64 2.37.4-18.el9 baseos 155 k 2024-03-08T20:43:34.336 INFO:teuthology.orchestra.run.smithi149.stdout: libmount x86_64 2.37.4-18.el9 baseos 136 k 2024-03-08T20:43:34.336 INFO:teuthology.orchestra.run.smithi149.stdout: librados2 x86_64 2:19.0.0-1965.g7d63482a.el9 ceph 3.4 M 2024-03-08T20:43:34.336 INFO:teuthology.orchestra.run.smithi149.stdout: librbd1 x86_64 2:19.0.0-1965.g7d63482a.el9 ceph 3.1 M 2024-03-08T20:43:34.337 INFO:teuthology.orchestra.run.smithi149.stdout: libsmartcols x86_64 2.37.4-18.el9 baseos 64 k 2024-03-08T20:43:34.337 INFO:teuthology.orchestra.run.smithi149.stdout: libuuid x86_64 2.37.4-18.el9 baseos 29 k 2024-03-08T20:43:34.337 INFO:teuthology.orchestra.run.smithi149.stdout: openssl x86_64 1:3.0.7-27.el9 baseos 1.2 M 2024-03-08T20:43:34.337 INFO:teuthology.orchestra.run.smithi149.stdout: openssl-libs x86_64 1:3.0.7-27.el9 baseos 2.2 M 2024-03-08T20:43:34.337 INFO:teuthology.orchestra.run.smithi149.stdout: util-linux x86_64 2.37.4-18.el9 baseos 2.3 M 2024-03-08T20:43:34.337 INFO:teuthology.orchestra.run.smithi149.stdout: util-linux-core x86_64 2.37.4-18.el9 baseos 465 k 2024-03-08T20:43:34.337 INFO:teuthology.orchestra.run.smithi149.stdout: util-linux-user x86_64 2.37.4-18.el9 baseos 31 k 2024-03-08T20:43:34.337 INFO:teuthology.orchestra.run.smithi149.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-03-08T20:43:34.337 INFO:teuthology.orchestra.run.smithi149.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-03-08T20:43:34.337 INFO:teuthology.orchestra.run.smithi149.stdout:Installing dependencies: 2024-03-08T20:43:34.337 INFO:teuthology.orchestra.run.smithi149.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-03-08T20:43:34.337 INFO:teuthology.orchestra.run.smithi149.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-03-08T20:43:34.337 INFO:teuthology.orchestra.run.smithi149.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-03-08T20:43:34.338 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-common x86_64 2:19.0.0-1965.g7d63482a.el9 ceph 20 M 2024-03-08T20:43:34.338 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-grafana-dashboards noarch 2:19.0.0-1965.g7d63482a.el9 ceph-noarch 34 k 2024-03-08T20:43:34.338 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mds x86_64 2:19.0.0-1965.g7d63482a.el9 ceph 2.1 M 2024-03-08T20:43:34.338 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mgr-modules-core noarch 2:19.0.0-1965.g7d63482a.el9 ceph-noarch 248 k 2024-03-08T20:43:34.338 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mon x86_64 2:19.0.0-1965.g7d63482a.el9 ceph 4.7 M 2024-03-08T20:43:34.338 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-osd x86_64 2:19.0.0-1965.g7d63482a.el9 ceph 17 M 2024-03-08T20:43:34.338 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-prometheus-alerts noarch 2:19.0.0-1965.g7d63482a.el9 ceph-noarch 16 k 2024-03-08T20:43:34.338 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-selinux x86_64 2:19.0.0-1965.g7d63482a.el9 ceph 25 k 2024-03-08T20:43:34.338 INFO:teuthology.orchestra.run.smithi149.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-03-08T20:43:34.338 INFO:teuthology.orchestra.run.smithi149.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-03-08T20:43:34.338 INFO:teuthology.orchestra.run.smithi149.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-03-08T20:43:34.338 INFO:teuthology.orchestra.run.smithi149.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-03-08T20:43:34.338 INFO:teuthology.orchestra.run.smithi149.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-03-08T20:43:34.339 INFO:teuthology.orchestra.run.smithi149.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-03-08T20:43:34.339 INFO:teuthology.orchestra.run.smithi149.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-03-08T20:43:34.339 INFO:teuthology.orchestra.run.smithi149.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-03-08T20:43:34.339 INFO:teuthology.orchestra.run.smithi149.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-03-08T20:43:34.339 INFO:teuthology.orchestra.run.smithi149.stdout: libcephsqlite x86_64 2:19.0.0-1965.g7d63482a.el9 ceph 163 k 2024-03-08T20:43:34.339 INFO:teuthology.orchestra.run.smithi149.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-03-08T20:43:34.339 INFO:teuthology.orchestra.run.smithi149.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-03-08T20:43:34.339 INFO:teuthology.orchestra.run.smithi149.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-03-08T20:43:34.339 INFO:teuthology.orchestra.run.smithi149.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-03-08T20:43:34.339 INFO:teuthology.orchestra.run.smithi149.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-03-08T20:43:34.339 INFO:teuthology.orchestra.run.smithi149.stdout: libradosstriper1 x86_64 2:19.0.0-1965.g7d63482a.el9 ceph 501 k 2024-03-08T20:43:34.339 INFO:teuthology.orchestra.run.smithi149.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-03-08T20:43:34.339 INFO:teuthology.orchestra.run.smithi149.stdout: librgw2 x86_64 2:19.0.0-1965.g7d63482a.el9 ceph 5.1 M 2024-03-08T20:43:34.340 INFO:teuthology.orchestra.run.smithi149.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-03-08T20:43:34.340 INFO:teuthology.orchestra.run.smithi149.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-03-08T20:43:34.340 INFO:teuthology.orchestra.run.smithi149.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-03-08T20:43:34.340 INFO:teuthology.orchestra.run.smithi149.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-03-08T20:43:34.340 INFO:teuthology.orchestra.run.smithi149.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-03-08T20:43:34.340 INFO:teuthology.orchestra.run.smithi149.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-03-08T20:43:34.340 INFO:teuthology.orchestra.run.smithi149.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-03-08T20:43:34.340 INFO:teuthology.orchestra.run.smithi149.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-03-08T20:43:34.340 INFO:teuthology.orchestra.run.smithi149.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-03-08T20:43:34.340 INFO:teuthology.orchestra.run.smithi149.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-03-08T20:43:34.340 INFO:teuthology.orchestra.run.smithi149.stdout: postgresql-private-libs x86_64 13.11-1.el9 CentOS-AppStream 134 k 2024-03-08T20:43:34.340 INFO:teuthology.orchestra.run.smithi149.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-03-08T20:43:34.340 INFO:teuthology.orchestra.run.smithi149.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-03-08T20:43:34.341 INFO:teuthology.orchestra.run.smithi149.stdout: python3-asyncssh noarch 2.13.2-4.el9 epel 548 k 2024-03-08T20:43:34.341 INFO:teuthology.orchestra.run.smithi149.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-03-08T20:43:34.341 INFO:teuthology.orchestra.run.smithi149.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-03-08T20:43:34.341 INFO:teuthology.orchestra.run.smithi149.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-03-08T20:43:34.341 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-03-08T20:43:34.341 INFO:teuthology.orchestra.run.smithi149.stdout: python3-ceph-argparse x86_64 2:19.0.0-1965.g7d63482a.el9 ceph 45 k 2024-03-08T20:43:34.341 INFO:teuthology.orchestra.run.smithi149.stdout: python3-ceph-common x86_64 2:19.0.0-1965.g7d63482a.el9 ceph 131 k 2024-03-08T20:43:34.341 INFO:teuthology.orchestra.run.smithi149.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-03-08T20:43:34.341 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-03-08T20:43:34.341 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-03-08T20:43:34.341 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-03-08T20:43:34.341 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-03-08T20:43:34.341 INFO:teuthology.orchestra.run.smithi149.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-03-08T20:43:34.342 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-03-08T20:43:34.342 INFO:teuthology.orchestra.run.smithi149.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-03-08T20:43:34.342 INFO:teuthology.orchestra.run.smithi149.stdout: python3-google-auth noarch 1:2.28.0-1.el9 epel 203 k 2024-03-08T20:43:34.342 INFO:teuthology.orchestra.run.smithi149.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-03-08T20:43:34.342 INFO:teuthology.orchestra.run.smithi149.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-03-08T20:43:34.342 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-03-08T20:43:34.342 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-03-08T20:43:34.342 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-03-08T20:43:34.342 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-03-08T20:43:34.342 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-03-08T20:43:34.342 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-03-08T20:43:34.342 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-03-08T20:43:34.343 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-03-08T20:43:34.343 INFO:teuthology.orchestra.run.smithi149.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-03-08T20:43:34.343 INFO:teuthology.orchestra.run.smithi149.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-03-08T20:43:34.343 INFO:teuthology.orchestra.run.smithi149.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-03-08T20:43:34.343 INFO:teuthology.orchestra.run.smithi149.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-03-08T20:43:34.343 INFO:teuthology.orchestra.run.smithi149.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-03-08T20:43:34.343 INFO:teuthology.orchestra.run.smithi149.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-03-08T20:43:34.343 INFO:teuthology.orchestra.run.smithi149.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-03-08T20:43:34.343 INFO:teuthology.orchestra.run.smithi149.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-03-08T20:43:34.343 INFO:teuthology.orchestra.run.smithi149.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-03-08T20:43:34.343 INFO:teuthology.orchestra.run.smithi149.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-03-08T20:43:34.343 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-03-08T20:43:34.343 INFO:teuthology.orchestra.run.smithi149.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-03-08T20:43:34.343 INFO:teuthology.orchestra.run.smithi149.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-03-08T20:43:34.343 INFO:teuthology.orchestra.run.smithi149.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-03-08T20:43:34.344 INFO:teuthology.orchestra.run.smithi149.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-03-08T20:43:34.344 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-03-08T20:43:34.344 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-03-08T20:43:34.344 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-03-08T20:43:34.344 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-03-08T20:43:34.344 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-03-08T20:43:34.344 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-03-08T20:43:34.344 INFO:teuthology.orchestra.run.smithi149.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-03-08T20:43:34.344 INFO:teuthology.orchestra.run.smithi149.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-03-08T20:43:34.344 INFO:teuthology.orchestra.run.smithi149.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-03-08T20:43:34.344 INFO:teuthology.orchestra.run.smithi149.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-03-08T20:43:34.344 INFO:teuthology.orchestra.run.smithi149.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-03-08T20:43:34.344 INFO:teuthology.orchestra.run.smithi149.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-03-08T20:43:34.344 INFO:teuthology.orchestra.run.smithi149.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-03-08T20:43:34.344 INFO:teuthology.orchestra.run.smithi149.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-03-08T20:43:34.345 INFO:teuthology.orchestra.run.smithi149.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-03-08T20:43:34.345 INFO:teuthology.orchestra.run.smithi149.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-03-08T20:43:34.345 INFO:teuthology.orchestra.run.smithi149.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-03-08T20:43:34.345 INFO:teuthology.orchestra.run.smithi149.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-03-08T20:43:34.345 INFO:teuthology.orchestra.run.smithi149.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-03-08T20:43:34.345 INFO:teuthology.orchestra.run.smithi149.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-03-08T20:43:34.345 INFO:teuthology.orchestra.run.smithi149.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-03-08T20:43:34.345 INFO:teuthology.orchestra.run.smithi149.stdout: qatlib x86_64 23.11.0-1.el9 appstream 216 k 2024-03-08T20:43:34.345 INFO:teuthology.orchestra.run.smithi149.stdout: qatzip-libs x86_64 1.1.2-2.el9 appstream 44 k 2024-03-08T20:43:34.345 INFO:teuthology.orchestra.run.smithi149.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-03-08T20:43:34.345 INFO:teuthology.orchestra.run.smithi149.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-03-08T20:43:34.345 INFO:teuthology.orchestra.run.smithi149.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-03-08T20:43:34.345 INFO:teuthology.orchestra.run.smithi149.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-03-08T20:43:34.345 INFO:teuthology.orchestra.run.smithi149.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-03-08T20:43:34.345 INFO:teuthology.orchestra.run.smithi149.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-03-08T20:43:34.346 INFO:teuthology.orchestra.run.smithi149.stdout:Installing weak dependencies: 2024-03-08T20:43:34.346 INFO:teuthology.orchestra.run.smithi149.stdout: qatlib-service x86_64 23.11.0-1.el9 appstream 35 k 2024-03-08T20:43:34.346 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-03-08T20:43:34.346 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction Summary 2024-03-08T20:43:34.346 INFO:teuthology.orchestra.run.smithi149.stdout:============================================================================================ 2024-03-08T20:43:34.346 INFO:teuthology.orchestra.run.smithi149.stdout:Install 150 Packages 2024-03-08T20:43:34.346 INFO:teuthology.orchestra.run.smithi149.stdout:Upgrade 24 Packages 2024-03-08T20:43:34.346 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-03-08T20:43:34.347 INFO:teuthology.orchestra.run.smithi149.stdout:Total download size: 223 M 2024-03-08T20:43:34.347 INFO:teuthology.orchestra.run.smithi149.stdout:Downloading Packages: 2024-03-08T20:43:34.826 INFO:teuthology.orchestra.run.smithi204.stdout:CentOS Stream 9 - CRB 407 kB/s | 6.1 MB 00:15 2024-03-08T20:43:35.541 INFO:teuthology.orchestra.run.smithi149.stdout:(1/174): ceph-19.0.0-1965.g7d63482a.el9.x86_64. 16 kB/s | 6.6 kB 00:00 2024-03-08T20:43:35.874 INFO:teuthology.orchestra.run.smithi149.stdout:(2/174): ceph-base-19.0.0-1965.g7d63482a.el9.x8 7.1 MB/s | 5.4 MB 00:00 2024-03-08T20:43:35.949 INFO:teuthology.orchestra.run.smithi149.stdout:(3/174): ceph-immutable-object-cache-19.0.0-196 1.9 MB/s | 145 kB 00:00 2024-03-08T20:43:36.208 INFO:teuthology.orchestra.run.smithi149.stdout:(4/174): ceph-mds-19.0.0-1965.g7d63482a.el9.x86 8.3 MB/s | 2.1 MB 00:00 2024-03-08T20:43:36.384 INFO:teuthology.orchestra.run.smithi149.stdout:(5/174): ceph-mgr-19.0.0-1965.g7d63482a.el9.x86 8.6 MB/s | 1.5 MB 00:00 2024-03-08T20:43:36.900 INFO:teuthology.orchestra.run.smithi149.stdout:(6/174): ceph-common-19.0.0-1965.g7d63482a.el9. 11 MB/s | 20 MB 00:01 2024-03-08T20:43:36.902 INFO:teuthology.orchestra.run.smithi204.stdout:CentOS Stream 9 - Extras packages 20 kB/s | 15 kB 00:00 2024-03-08T20:43:36.942 INFO:teuthology.orchestra.run.smithi149.stdout:(7/174): ceph-fuse-19.0.0-1965.g7d63482a.el9.x8 635 kB/s | 889 kB 00:01 2024-03-08T20:43:37.093 INFO:teuthology.orchestra.run.smithi149.stdout:(8/174): ceph-mon-19.0.0-1965.g7d63482a.el9.x86 6.6 MB/s | 4.7 MB 00:00 2024-03-08T20:43:37.143 INFO:teuthology.orchestra.run.smithi149.stdout:(9/174): ceph-selinux-19.0.0-1965.g7d63482a.el9 515 kB/s | 25 kB 00:00 2024-03-08T20:43:37.994 INFO:teuthology.orchestra.run.smithi149.stdout:(10/174): ceph-osd-19.0.0-1965.g7d63482a.el9.x8 15 MB/s | 17 MB 00:01 2024-03-08T20:43:38.085 INFO:teuthology.orchestra.run.smithi149.stdout:(11/174): cephfs-mirror-19.0.0-1965.g7d63482a.e 2.4 MB/s | 220 kB 00:00 2024-03-08T20:43:38.152 INFO:teuthology.orchestra.run.smithi149.stdout:(12/174): cephfs-shell-19.0.0-1965.g7d63482a.el 411 kB/s | 27 kB 00:00 2024-03-08T20:43:38.219 INFO:teuthology.orchestra.run.smithi149.stdout:(13/174): libcephfs-devel-19.0.0-1965.g7d63482a 475 kB/s | 31 kB 00:00 2024-03-08T20:43:38.329 INFO:teuthology.orchestra.run.smithi204.stdout:Extra Packages for Enterprise Linux 17 MB/s | 21 MB 00:01 2024-03-08T20:43:38.361 INFO:teuthology.orchestra.run.smithi149.stdout:(14/174): libcephfs2-19.0.0-1965.g7d63482a.el9. 5.2 MB/s | 745 kB 00:00 2024-03-08T20:43:38.436 INFO:teuthology.orchestra.run.smithi149.stdout:(15/174): libcephsqlite-19.0.0-1965.g7d63482a.e 2.2 MB/s | 163 kB 00:00 2024-03-08T20:43:38.519 INFO:teuthology.orchestra.run.smithi149.stdout:(16/174): librados-devel-19.0.0-1965.g7d63482a. 1.5 MB/s | 127 kB 00:00 2024-03-08T20:43:38.620 INFO:teuthology.orchestra.run.smithi149.stdout:(17/174): libradosstriper1-19.0.0-1965.g7d63482 4.9 MB/s | 501 kB 00:00 2024-03-08T20:43:39.070 INFO:teuthology.orchestra.run.smithi149.stdout:(18/174): librgw2-19.0.0-1965.g7d63482a.el9.x86 11 MB/s | 5.1 MB 00:00 2024-03-08T20:43:39.162 INFO:teuthology.orchestra.run.smithi149.stdout:(19/174): python3-ceph-argparse-19.0.0-1965.g7d 497 kB/s | 45 kB 00:00 2024-03-08T20:43:39.246 INFO:teuthology.orchestra.run.smithi149.stdout:(20/174): python3-ceph-common-19.0.0-1965.g7d63 1.6 MB/s | 131 kB 00:00 2024-03-08T20:43:39.321 INFO:teuthology.orchestra.run.smithi149.stdout:(21/174): python3-cephfs-19.0.0-1965.g7d63482a. 2.1 MB/s | 162 kB 00:00 2024-03-08T20:43:39.396 INFO:teuthology.orchestra.run.smithi149.stdout:(22/174): python3-rados-19.0.0-1965.g7d63482a.e 4.2 MB/s | 320 kB 00:00 2024-03-08T20:43:39.480 INFO:teuthology.orchestra.run.smithi149.stdout:(23/174): python3-rbd-19.0.0-1965.g7d63482a.el9 3.5 MB/s | 298 kB 00:00 2024-03-08T20:43:39.564 INFO:teuthology.orchestra.run.smithi149.stdout:(24/174): python3-rgw-19.0.0-1965.g7d63482a.el9 1.2 MB/s | 99 kB 00:00 2024-03-08T20:43:39.639 INFO:teuthology.orchestra.run.smithi149.stdout:(25/174): rbd-fuse-19.0.0-1965.g7d63482a.el9.x8 1.1 MB/s | 85 kB 00:00 2024-03-08T20:43:39.981 INFO:teuthology.orchestra.run.smithi149.stdout:(26/174): rbd-mirror-19.0.0-1965.g7d63482a.el9. 9.1 MB/s | 3.1 MB 00:00 2024-03-08T20:43:40.140 INFO:teuthology.orchestra.run.smithi149.stdout:(27/174): ceph-radosgw-19.0.0-1965.g7d63482a.el 3.1 MB/s | 9.8 MB 00:03 2024-03-08T20:43:40.173 INFO:teuthology.orchestra.run.smithi149.stdout:(28/174): rbd-nbd-19.0.0-1965.g7d63482a.el9.x86 894 kB/s | 171 kB 00:00 2024-03-08T20:43:40.232 INFO:teuthology.orchestra.run.smithi149.stdout:(29/174): ceph-grafana-dashboards-19.0.0-1965.g 374 kB/s | 34 kB 00:00 2024-03-08T20:43:40.257 INFO:teuthology.orchestra.run.smithi149.stdout:(30/174): ceph-mgr-cephadm-19.0.0-1965.g7d63482 1.6 MB/s | 139 kB 00:00 2024-03-08T20:43:40.949 INFO:teuthology.orchestra.run.smithi149.stdout:(31/174): ceph-test-19.0.0-1965.g7d63482a.el9.x 12 MB/s | 47 MB 00:03 2024-03-08T20:43:41.025 INFO:teuthology.orchestra.run.smithi149.stdout:(32/174): ceph-mgr-dashboard-19.0.0-1965.g7d634 2.3 MB/s | 1.8 MB 00:00 2024-03-08T20:43:41.058 INFO:teuthology.orchestra.run.smithi149.stdout:(33/174): ceph-mgr-modules-core-19.0.0-1965.g7d 2.3 MB/s | 248 kB 00:00 2024-03-08T20:43:41.108 INFO:teuthology.orchestra.run.smithi149.stdout:(34/174): ceph-prometheus-alerts-19.0.0-1965.g7 335 kB/s | 16 kB 00:00 2024-03-08T20:43:41.134 INFO:teuthology.orchestra.run.smithi149.stdout:(35/174): ceph-mgr-rook-19.0.0-1965.g7d63482a.e 458 kB/s | 49 kB 00:00 2024-03-08T20:43:41.167 INFO:teuthology.orchestra.run.smithi149.stdout:(36/174): ceph-volume-19.0.0-1965.g7d63482a.el9 4.5 MB/s | 264 kB 00:00 2024-03-08T20:43:41.218 INFO:teuthology.orchestra.run.smithi149.stdout:(37/174): cephfs-top-19.0.0-1965.g7d63482a.el9. 371 kB/s | 18 kB 00:00 2024-03-08T20:43:41.359 INFO:teuthology.orchestra.run.smithi149.stdout:(38/174): ceph-mgr-diskprediction-local-19.0.0- 6.7 MB/s | 7.4 MB 00:01 2024-03-08T20:43:41.468 INFO:teuthology.orchestra.run.smithi149.stdout:(39/174): cephadm-19.0.0-1965.g7d63482a.el9.noa 2.2 MB/s | 748 kB 00:00 2024-03-08T20:43:41.752 INFO:teuthology.orchestra.run.smithi149.stdout:(40/174): automake-1.16.2-8.el9.noarch.rpm 1.2 MB/s | 662 kB 00:00 2024-03-08T20:43:41.785 INFO:teuthology.orchestra.run.smithi149.stdout:(41/174): boost-program-options-1.75.0-8.el9.x8 409 kB/s | 106 kB 00:00 2024-03-08T20:43:41.844 INFO:teuthology.orchestra.run.smithi149.stdout:(42/174): flex-2.6.4-9.el9.x86_64.rpm 3.4 MB/s | 307 kB 00:00 2024-03-08T20:43:41.894 INFO:teuthology.orchestra.run.smithi149.stdout:(43/174): bison-3.7.4-5.el9.x86_64.rpm 1.9 MB/s | 920 kB 00:00 2024-03-08T20:43:41.919 INFO:teuthology.orchestra.run.smithi149.stdout:(44/174): flexiblas-3.0.4-8.el9.x86_64.rpm 241 kB/s | 32 kB 00:00 2024-03-08T20:43:42.228 INFO:teuthology.orchestra.run.smithi149.stdout:(45/174): flexiblas-netlib-3.0.4-8.el9.x86_64.r 7.8 MB/s | 3.0 MB 00:00 2024-03-08T20:43:42.253 INFO:teuthology.orchestra.run.smithi149.stdout:(46/174): libaio-devel-0.3.111-13.el9.x86_64.rp 31 kB/s | 10 kB 00:00 2024-03-08T20:43:42.278 INFO:teuthology.orchestra.run.smithi149.stdout:(47/174): flexiblas-openblas-openmp-3.0.4-8.el9 48 kB/s | 17 kB 00:00 2024-03-08T20:43:42.320 INFO:teuthology.orchestra.run.smithi149.stdout:(48/174): librabbitmq-0.11.0-7.el9.x86_64.rpm 674 kB/s | 44 kB 00:00 2024-03-08T20:43:42.345 INFO:teuthology.orchestra.run.smithi149.stdout:(49/174): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.7 MB/s | 159 kB 00:00 2024-03-08T20:43:42.454 INFO:teuthology.orchestra.run.smithi149.stdout:(50/174): librdkafka-1.6.1-102.el9.x86_64.rpm 3.7 MB/s | 661 kB 00:00 2024-03-08T20:43:42.487 INFO:teuthology.orchestra.run.smithi149.stdout:(51/174): libtool-2.4.6-45.el9.x86_64.rpm 3.4 MB/s | 578 kB 00:00 2024-03-08T20:43:42.512 INFO:teuthology.orchestra.run.smithi149.stdout:(52/174): libzstd-devel-1.5.1-2.el9.x86_64.rpm 279 kB/s | 46 kB 00:00 2024-03-08T20:43:42.571 INFO:teuthology.orchestra.run.smithi149.stdout:(53/174): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.8 MB/s | 282 kB 00:00 2024-03-08T20:43:42.605 INFO:teuthology.orchestra.run.smithi149.stdout:(54/174): lua-5.4.4-4.el9.x86_64.rpm 2.0 MB/s | 187 kB 00:00 2024-03-08T20:43:42.630 INFO:teuthology.orchestra.run.smithi149.stdout:(55/174): oniguruma-6.9.6-1.el9.5.x86_64.rpm 1.8 MB/s | 217 kB 00:00 2024-03-08T20:43:42.655 INFO:teuthology.orchestra.run.smithi149.stdout:(56/174): openblas-0.3.21-2.el9.x86_64.rpm 425 kB/s | 34 kB 00:00 2024-03-08T20:43:42.822 INFO:teuthology.orchestra.run.smithi149.stdout:(57/174): postgresql-13.11-1.el9.x86_64.rpm 8.0 MB/s | 1.5 MB 00:00 2024-03-08T20:43:42.923 INFO:teuthology.orchestra.run.smithi149.stdout:(58/174): postgresql-contrib-13.11-1.el9.x86_64 3.0 MB/s | 811 kB 00:00 2024-03-08T20:43:43.031 INFO:teuthology.orchestra.run.smithi149.stdout:(59/174): openblas-openmp-0.3.21-2.el9.x86_64.r 12 MB/s | 4.7 MB 00:00 2024-03-08T20:43:43.056 INFO:teuthology.orchestra.run.smithi149.stdout:(60/174): postgresql-private-libs-13.11-1.el9.x 605 kB/s | 134 kB 00:00 2024-03-08T20:43:43.198 INFO:teuthology.orchestra.run.smithi149.stdout:(61/174): postgresql-server-13.11-1.el9.x86_64. 21 MB/s | 5.7 MB 00:00 2024-03-08T20:43:43.232 INFO:teuthology.orchestra.run.smithi149.stdout:(62/174): python3-attrs-20.3.0-7.el9.noarch.rpm 474 kB/s | 78 kB 00:00 2024-03-08T20:43:43.282 INFO:teuthology.orchestra.run.smithi149.stdout:(63/174): protobuf-3.14.0-13.el9.x86_64.rpm 4.7 MB/s | 1.0 MB 00:00 2024-03-08T20:43:43.308 INFO:teuthology.orchestra.run.smithi149.stdout:(64/174): python3-jsonpatch-1.21-16.el9.noarch. 375 kB/s | 24 kB 00:00 2024-03-08T20:43:43.508 INFO:teuthology.orchestra.run.smithi149.stdout:(65/174): python3-babel-2.9.1-2.el9.noarch.rpm 22 MB/s | 5.8 MB 00:00 2024-03-08T20:43:43.534 INFO:teuthology.orchestra.run.smithi149.stdout:(66/174): python3-jsonpointer-2.0-4.el9.noarch. 72 kB/s | 18 kB 00:00 2024-03-08T20:43:43.558 INFO:teuthology.orchestra.run.smithi149.stdout:(67/174): python3-mako-1.1.4-6.el9.noarch.rpm 613 kB/s | 144 kB 00:00 2024-03-08T20:43:43.592 INFO:teuthology.orchestra.run.smithi149.stdout:(68/174): python3-markupsafe-1.1.1-12.el9.x86_6 553 kB/s | 32 kB 00:00 2024-03-08T20:43:43.725 INFO:teuthology.orchestra.run.smithi149.stdout:(69/174): python3-numpy-1.20.1-5.el9.x86_64.rpm 25 MB/s | 4.8 MB 00:00 2024-03-08T20:43:43.751 INFO:teuthology.orchestra.run.smithi149.stdout:(70/174): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.2 MB/s | 217 kB 00:00 2024-03-08T20:43:43.784 INFO:teuthology.orchestra.run.smithi149.stdout:(71/174): python3-oauthlib-3.1.1-5.el9.noarch.r 987 kB/s | 174 kB 00:00 2024-03-08T20:43:43.818 INFO:teuthology.orchestra.run.smithi149.stdout:(72/174): python3-packaging-20.9-5.el9.noarch.r 1.0 MB/s | 69 kB 00:00 2024-03-08T20:43:43.843 INFO:teuthology.orchestra.run.smithi149.stdout:(73/174): python3-prettytable-0.7.2-27.el9.noar 461 kB/s | 41 kB 00:00 2024-03-08T20:43:43.877 INFO:teuthology.orchestra.run.smithi149.stdout:(74/174): python3-protobuf-3.14.0-13.el9.noarch 2.6 MB/s | 238 kB 00:00 2024-03-08T20:43:43.910 INFO:teuthology.orchestra.run.smithi149.stdout:(75/174): python3-pyasn1-modules-0.4.8-6.el9.no 3.2 MB/s | 211 kB 00:00 2024-03-08T20:43:43.935 INFO:teuthology.orchestra.run.smithi149.stdout:(76/174): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.5 MB/s | 132 kB 00:00 2024-03-08T20:43:43.961 INFO:teuthology.orchestra.run.smithi149.stdout:(77/174): python3-pytz-2021.1-5.el9.noarch.rpm 582 kB/s | 47 kB 00:00 2024-03-08T20:43:43.978 INFO:teuthology.orchestra.run.smithi149.stdout:(78/174): python3-requests-oauthlib-1.3.0-12.el 661 kB/s | 43 kB 00:00 2024-03-08T20:43:44.003 INFO:teuthology.orchestra.run.smithi149.stdout:(79/174): python3-toml-0.10.2-6.el9.noarch.rpm 963 kB/s | 39 kB 00:00 2024-03-08T20:43:44.129 INFO:teuthology.orchestra.run.smithi149.stdout:(80/174): python3-wcwidth-0.2.5-8.el9.noarch.rp 274 kB/s | 41 kB 00:00 2024-03-08T20:43:44.129 INFO:teuthology.orchestra.run.smithi204.stdout:lab-extras 14 kB/s | 1.7 kB 00:00 2024-03-08T20:43:44.462 INFO:teuthology.orchestra.run.smithi149.stdout:(81/174): python3-scipy-1.6.2-8.el9.x86_64.rpm 31 MB/s | 16 MB 00:00 2024-03-08T20:43:44.495 INFO:teuthology.orchestra.run.smithi149.stdout:(82/174): socat-1.7.4.1-5.el9.x86_64.rpm 629 kB/s | 300 kB 00:00 2024-03-08T20:43:44.512 INFO:teuthology.orchestra.run.smithi149.stdout:(83/174): uuid-1.6.2-55.el9.x86_64.rpm 153 kB/s | 56 kB 00:00 2024-03-08T20:43:44.538 INFO:teuthology.orchestra.run.smithi149.stdout:(84/174): xmlstarlet-1.6.1-20.el9.x86_64.rpm 876 kB/s | 63 kB 00:00 2024-03-08T20:43:44.555 INFO:teuthology.orchestra.run.smithi149.stdout:(85/174): zlib-devel-1.2.11-40.el9.x86_64.rpm 1.1 MB/s | 44 kB 00:00 2024-03-08T20:43:44.580 INFO:teuthology.orchestra.run.smithi149.stdout:(86/174): fuse-2.9.9-15.el9.x86_64.rpm 1.2 MB/s | 79 kB 00:00 2024-03-08T20:43:44.614 INFO:teuthology.orchestra.run.smithi149.stdout:(87/174): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.2 MB/s | 241 kB 00:00 2024-03-08T20:43:44.672 INFO:teuthology.orchestra.run.smithi149.stdout:(88/174): python3-cryptography-36.0.1-4.el9.x86 9.9 MB/s | 1.1 MB 00:00 2024-03-08T20:43:44.697 INFO:teuthology.orchestra.run.smithi149.stdout:(89/174): python3-ply-3.11-14.el9.noarch.rpm 908 kB/s | 103 kB 00:00 2024-03-08T20:43:44.722 INFO:teuthology.orchestra.run.smithi149.stdout:(90/174): python3-pycparser-2.20-6.el9.noarch.r 1.2 MB/s | 124 kB 00:00 2024-03-08T20:43:44.756 INFO:teuthology.orchestra.run.smithi149.stdout:(91/174): python3-urllib3-1.26.5-5.el9.noarch.r 3.2 MB/s | 187 kB 00:00 2024-03-08T20:43:44.957 INFO:teuthology.orchestra.run.smithi149.stdout:(92/174): jq-1.6-16.el9.x86_64.rpm 730 kB/s | 187 kB 00:00 2024-03-08T20:43:45.015 INFO:teuthology.orchestra.run.smithi149.stdout:(93/174): python3-requests-2.25.1-8.el9.noarch. 2.1 MB/s | 125 kB 00:00 2024-03-08T20:43:45.049 INFO:teuthology.orchestra.run.smithi149.stdout:(94/174): libquadmath-11.4.1-3.el9.x86_64.rpm 669 kB/s | 195 kB 00:00 2024-03-08T20:43:45.074 INFO:teuthology.orchestra.run.smithi149.stdout:(95/174): elfutils-libelf-devel-0.190-2.el9.x86 371 kB/s | 21 kB 00:00 2024-03-08T20:43:45.116 INFO:teuthology.orchestra.run.smithi149.stdout:(96/174): libgfortran-11.4.1-3.el9.x86_64.rpm 2.0 MB/s | 803 kB 00:00 2024-03-08T20:43:45.141 INFO:teuthology.orchestra.run.smithi149.stdout:(97/174): libuuid-devel-2.37.4-18.el9.x86_64.rp 278 kB/s | 25 kB 00:00 2024-03-08T20:43:45.183 INFO:teuthology.orchestra.run.smithi149.stdout:(98/174): python3-devel-3.9.17-2.el9.x86_64.rpm 3.6 MB/s | 245 kB 00:00 2024-03-08T20:43:45.258 INFO:teuthology.orchestra.run.smithi149.stdout:(99/174): qatlib-23.11.0-1.el9.x86_64.rpm 2.9 MB/s | 216 kB 00:00 2024-03-08T20:43:45.284 INFO:teuthology.orchestra.run.smithi149.stdout:(100/174): python3-jinja2-2.11.3-5.el9.noarch.r 1.7 MB/s | 248 kB 00:00 2024-03-08T20:43:45.326 INFO:teuthology.orchestra.run.smithi149.stdout:(101/174): qatlib-service-23.11.0-1.el9.x86_64. 524 kB/s | 35 kB 00:00 2024-03-08T20:43:45.351 INFO:teuthology.orchestra.run.smithi149.stdout:(102/174): qatzip-libs-1.1.2-2.el9.x86_64.rpm 678 kB/s | 44 kB 00:00 2024-03-08T20:43:45.385 INFO:teuthology.orchestra.run.smithi149.stdout:(103/174): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 1.6 MB/s | 94 kB 00:00 2024-03-08T20:43:45.469 INFO:teuthology.orchestra.run.smithi149.stdout:(104/174): lua-devel-5.4.4-4.el9.x86_64.rpm 192 kB/s | 22 kB 00:00 2024-03-08T20:43:45.727 INFO:teuthology.orchestra.run.smithi149.stdout:(105/174): abseil-cpp-20211102.0-3.el9.x86_64.r 2.1 MB/s | 548 kB 00:00 2024-03-08T20:43:45.778 INFO:teuthology.orchestra.run.smithi149.stdout:(106/174): protobuf-compiler-3.14.0-13.el9.x86_ 2.2 MB/s | 863 kB 00:00 2024-03-08T20:43:45.811 INFO:teuthology.orchestra.run.smithi149.stdout:(107/174): gperftools-libs-2.9.1-2.el9.x86_64.r 3.6 MB/s | 309 kB 00:00 2024-03-08T20:43:45.903 INFO:teuthology.orchestra.run.smithi149.stdout:(108/174): grpc-data-1.46.7-10.el9.noarch.rpm 156 kB/s | 19 kB 00:00 2024-03-08T20:43:45.995 INFO:teuthology.orchestra.run.smithi149.stdout:(109/174): libarrow-doc-9.0.0-10.el9.noarch.rpm 285 kB/s | 26 kB 00:00 2024-03-08T20:43:46.095 INFO:teuthology.orchestra.run.smithi149.stdout:(110/174): libarrow-9.0.0-10.el9.x86_64.rpm 16 MB/s | 4.4 MB 00:00 2024-03-08T20:43:46.145 INFO:teuthology.orchestra.run.smithi149.stdout:(111/174): liboath-2.6.7-2.el9.x86_64.rpm 325 kB/s | 49 kB 00:00 2024-03-08T20:43:46.171 INFO:teuthology.orchestra.run.smithi149.stdout:(112/174): libunwind-1.6.2-1.el9.x86_64.rpm 906 kB/s | 67 kB 00:00 2024-03-08T20:43:46.254 INFO:teuthology.orchestra.run.smithi149.stdout:(113/174): parquet-libs-9.0.0-10.el9.x86_64.rpm 9.9 MB/s | 838 kB 00:00 2024-03-08T20:43:46.296 INFO:teuthology.orchestra.run.smithi149.stdout:(114/174): luarocks-3.9.2-1.el9.noarch.rpm 1.0 MB/s | 151 kB 00:00 2024-03-08T20:43:46.338 INFO:teuthology.orchestra.run.smithi149.stdout:(115/174): python3-asyncssh-2.13.2-4.el9.noarch 6.5 MB/s | 548 kB 00:00 2024-03-08T20:43:46.363 INFO:teuthology.orchestra.run.smithi149.stdout:(116/174): python3-bcrypt-3.2.2-1.el9.x86_64.rp 658 kB/s | 43 kB 00:00 2024-03-08T20:43:46.464 INFO:teuthology.orchestra.run.smithi149.stdout:(117/174): openssl-devel-3.0.7-27.el9.x86_64.rp 3.0 MB/s | 4.1 MB 00:01 2024-03-08T20:43:46.481 INFO:teuthology.orchestra.run.smithi149.stdout:(118/174): python3-certifi-2021.10.8-2.el9.noar 129 kB/s | 15 kB 00:00 2024-03-08T20:43:46.506 INFO:teuthology.orchestra.run.smithi149.stdout:(119/174): python3-cachetools-4.2.4-1.el9.noarc 193 kB/s | 32 kB 00:00 2024-03-08T20:43:46.548 INFO:teuthology.orchestra.run.smithi149.stdout:(120/174): python3-cheroot-8.6.0-4.el9.noarch.r 2.0 MB/s | 172 kB 00:00 2024-03-08T20:43:46.581 INFO:teuthology.orchestra.run.smithi149.stdout:(121/174): python3-cmd2-2.4.2-2.el9.noarch.rpm 4.0 MB/s | 303 kB 00:00 2024-03-08T20:43:46.607 INFO:teuthology.orchestra.run.smithi149.stdout:(122/174): python3-colorama-0.4.4-12.el9.noarch 610 kB/s | 35 kB 00:00 2024-03-08T20:43:46.657 INFO:teuthology.orchestra.run.smithi149.stdout:(123/174): python3-google-auth-2.28.0-1.el9.noa 2.6 MB/s | 203 kB 00:00 2024-03-08T20:43:46.724 INFO:teuthology.orchestra.run.smithi149.stdout:(124/174): python3-grpcio-tools-1.46.7-10.el9.x 2.1 MB/s | 144 kB 00:00 2024-03-08T20:43:46.783 INFO:teuthology.orchestra.run.smithi149.stdout:(125/174): python3-jaraco-8.2.1-3.el9.noarch.rp 186 kB/s | 11 kB 00:00 2024-03-08T20:43:46.816 INFO:teuthology.orchestra.run.smithi149.stdout:(126/174): python3-cherrypy-18.6.1-2.el9.noarch 1.0 MB/s | 358 kB 00:00 2024-03-08T20:43:46.842 INFO:teuthology.orchestra.run.smithi149.stdout:(127/174): python3-jaraco-classes-3.2.1-5.el9.n 306 kB/s | 18 kB 00:00 2024-03-08T20:43:46.871 INFO:teuthology.orchestra.run.smithi204.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-08T20:43:46.873 INFO:teuthology.orchestra.run.smithi204.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-08T20:43:46.883 INFO:teuthology.orchestra.run.smithi149.stdout:(128/174): python3-jaraco-collections-3.0.0-8.e 350 kB/s | 23 kB 00:00 2024-03-08T20:43:46.896 INFO:teuthology.orchestra.run.smithi204.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-03-08T20:43:46.897 INFO:teuthology.orchestra.run.smithi204.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-03-08T20:43:46.899 INFO:teuthology.orchestra.run.smithi204.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-03-08T20:43:46.900 INFO:teuthology.orchestra.run.smithi204.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-03-08T20:43:46.900 INFO:teuthology.orchestra.run.smithi149.stdout:(129/174): python3-jaraco-functools-3.5.0-2.el9 336 kB/s | 19 kB 00:00 2024-03-08T20:43:46.903 INFO:teuthology.orchestra.run.smithi204.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-03-08T20:43:46.906 INFO:teuthology.orchestra.run.smithi204.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-03-08T20:43:46.907 INFO:teuthology.orchestra.run.smithi204.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-03-08T20:43:46.984 INFO:teuthology.orchestra.run.smithi149.stdout:(130/174): python3-grpcio-1.46.7-10.el9.x86_64. 5.4 MB/s | 2.0 MB 00:00 2024-03-08T20:43:46.996 INFO:teuthology.orchestra.run.smithi204.stdout:Dependencies resolved. 2024-03-08T20:43:47.015 INFO:teuthology.orchestra.run.smithi204.stdout:============================================================================================ 2024-03-08T20:43:47.015 INFO:teuthology.orchestra.run.smithi204.stdout: Package Arch Version Repository Size 2024-03-08T20:43:47.015 INFO:teuthology.orchestra.run.smithi204.stdout:============================================================================================ 2024-03-08T20:43:47.015 INFO:teuthology.orchestra.run.smithi204.stdout:Installing: 2024-03-08T20:43:47.015 INFO:teuthology.orchestra.run.smithi204.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-03-08T20:43:47.015 INFO:teuthology.orchestra.run.smithi204.stdout: ceph x86_64 2:19.0.0-1965.g7d63482a.el9 ceph 6.6 k 2024-03-08T20:43:47.015 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-base x86_64 2:19.0.0-1965.g7d63482a.el9 ceph 5.4 M 2024-03-08T20:43:47.016 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-fuse x86_64 2:19.0.0-1965.g7d63482a.el9 ceph 889 k 2024-03-08T20:43:47.016 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-1965.g7d63482a.el9 ceph 145 k 2024-03-08T20:43:47.016 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr x86_64 2:19.0.0-1965.g7d63482a.el9 ceph 1.5 M 2024-03-08T20:43:47.016 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-cephadm noarch 2:19.0.0-1965.g7d63482a.el9 ceph-noarch 139 k 2024-03-08T20:43:47.016 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-dashboard noarch 2:19.0.0-1965.g7d63482a.el9 ceph-noarch 1.8 M 2024-03-08T20:43:47.016 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-1965.g7d63482a.el9 ceph-noarch 7.4 M 2024-03-08T20:43:47.016 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-rook noarch 2:19.0.0-1965.g7d63482a.el9 ceph-noarch 49 k 2024-03-08T20:43:47.016 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-radosgw x86_64 2:19.0.0-1965.g7d63482a.el9 ceph 9.8 M 2024-03-08T20:43:47.016 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-test x86_64 2:19.0.0-1965.g7d63482a.el9 ceph 47 M 2024-03-08T20:43:47.016 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-volume noarch 2:19.0.0-1965.g7d63482a.el9 ceph-noarch 264 k 2024-03-08T20:43:47.016 INFO:teuthology.orchestra.run.smithi204.stdout: cephadm noarch 2:19.0.0-1965.g7d63482a.el9 ceph-noarch 748 k 2024-03-08T20:43:47.016 INFO:teuthology.orchestra.run.smithi204.stdout: cephfs-mirror x86_64 2:19.0.0-1965.g7d63482a.el9 ceph 220 k 2024-03-08T20:43:47.016 INFO:teuthology.orchestra.run.smithi204.stdout: cephfs-shell x86_64 2:19.0.0-1965.g7d63482a.el9 ceph 27 k 2024-03-08T20:43:47.017 INFO:teuthology.orchestra.run.smithi204.stdout: cephfs-top noarch 2:19.0.0-1965.g7d63482a.el9 ceph-noarch 18 k 2024-03-08T20:43:47.017 INFO:teuthology.orchestra.run.smithi204.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-03-08T20:43:47.017 INFO:teuthology.orchestra.run.smithi204.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-03-08T20:43:47.017 INFO:teuthology.orchestra.run.smithi204.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-03-08T20:43:47.017 INFO:teuthology.orchestra.run.smithi204.stdout: libcephfs-devel x86_64 2:19.0.0-1965.g7d63482a.el9 ceph 31 k 2024-03-08T20:43:47.017 INFO:teuthology.orchestra.run.smithi204.stdout: libcephfs2 x86_64 2:19.0.0-1965.g7d63482a.el9 ceph 745 k 2024-03-08T20:43:47.017 INFO:teuthology.orchestra.run.smithi204.stdout: librados-devel x86_64 2:19.0.0-1965.g7d63482a.el9 ceph 127 k 2024-03-08T20:43:47.017 INFO:teuthology.orchestra.run.smithi204.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-03-08T20:43:47.017 INFO:teuthology.orchestra.run.smithi204.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-03-08T20:43:47.017 INFO:teuthology.orchestra.run.smithi204.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-03-08T20:43:47.017 INFO:teuthology.orchestra.run.smithi204.stdout: postgresql x86_64 13.11-1.el9 CentOS-AppStream 1.5 M 2024-03-08T20:43:47.017 INFO:teuthology.orchestra.run.smithi204.stdout: postgresql-contrib x86_64 13.11-1.el9 CentOS-AppStream 811 k 2024-03-08T20:43:47.018 INFO:teuthology.orchestra.run.smithi204.stdout: postgresql-server x86_64 13.11-1.el9 CentOS-AppStream 5.7 M 2024-03-08T20:43:47.018 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cephfs x86_64 2:19.0.0-1965.g7d63482a.el9 ceph 162 k 2024-03-08T20:43:47.018 INFO:teuthology.orchestra.run.smithi204.stdout: python3-rados x86_64 2:19.0.0-1965.g7d63482a.el9 ceph 320 k 2024-03-08T20:43:47.018 INFO:teuthology.orchestra.run.smithi204.stdout: python3-rbd x86_64 2:19.0.0-1965.g7d63482a.el9 ceph 298 k 2024-03-08T20:43:47.018 INFO:teuthology.orchestra.run.smithi204.stdout: python3-rgw x86_64 2:19.0.0-1965.g7d63482a.el9 ceph 99 k 2024-03-08T20:43:47.018 INFO:teuthology.orchestra.run.smithi204.stdout: rbd-fuse x86_64 2:19.0.0-1965.g7d63482a.el9 ceph 85 k 2024-03-08T20:43:47.018 INFO:teuthology.orchestra.run.smithi204.stdout: rbd-mirror x86_64 2:19.0.0-1965.g7d63482a.el9 ceph 3.1 M 2024-03-08T20:43:47.018 INFO:teuthology.orchestra.run.smithi204.stdout: rbd-nbd x86_64 2:19.0.0-1965.g7d63482a.el9 ceph 171 k 2024-03-08T20:43:47.018 INFO:teuthology.orchestra.run.smithi204.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-03-08T20:43:47.018 INFO:teuthology.orchestra.run.smithi204.stdout:Upgrading: 2024-03-08T20:43:47.018 INFO:teuthology.orchestra.run.smithi204.stdout: NetworkManager x86_64 1:1.46.0-1.el9 baseos 2.3 M 2024-03-08T20:43:47.018 INFO:teuthology.orchestra.run.smithi204.stdout: NetworkManager-libnm x86_64 1:1.46.0-1.el9 baseos 1.8 M 2024-03-08T20:43:47.018 INFO:teuthology.orchestra.run.smithi204.stdout: NetworkManager-team x86_64 1:1.46.0-1.el9 baseos 42 k 2024-03-08T20:43:47.019 INFO:teuthology.orchestra.run.smithi204.stdout: NetworkManager-tui x86_64 1:1.46.0-1.el9 baseos 248 k 2024-03-08T20:43:47.019 INFO:teuthology.orchestra.run.smithi204.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-03-08T20:43:47.019 INFO:teuthology.orchestra.run.smithi204.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-03-08T20:43:47.019 INFO:teuthology.orchestra.run.smithi204.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-03-08T20:43:47.019 INFO:teuthology.orchestra.run.smithi204.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-03-08T20:43:47.019 INFO:teuthology.orchestra.run.smithi204.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-03-08T20:43:47.019 INFO:teuthology.orchestra.run.smithi204.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-03-08T20:43:47.019 INFO:teuthology.orchestra.run.smithi204.stdout: libblkid x86_64 2.37.4-18.el9 baseos 108 k 2024-03-08T20:43:47.019 INFO:teuthology.orchestra.run.smithi204.stdout: libfdisk x86_64 2.37.4-18.el9 baseos 155 k 2024-03-08T20:43:47.019 INFO:teuthology.orchestra.run.smithi204.stdout: libmount x86_64 2.37.4-18.el9 baseos 136 k 2024-03-08T20:43:47.019 INFO:teuthology.orchestra.run.smithi204.stdout: librados2 x86_64 2:19.0.0-1965.g7d63482a.el9 ceph 3.4 M 2024-03-08T20:43:47.019 INFO:teuthology.orchestra.run.smithi204.stdout: librbd1 x86_64 2:19.0.0-1965.g7d63482a.el9 ceph 3.1 M 2024-03-08T20:43:47.020 INFO:teuthology.orchestra.run.smithi204.stdout: libsmartcols x86_64 2.37.4-18.el9 baseos 64 k 2024-03-08T20:43:47.020 INFO:teuthology.orchestra.run.smithi204.stdout: libuuid x86_64 2.37.4-18.el9 baseos 29 k 2024-03-08T20:43:47.020 INFO:teuthology.orchestra.run.smithi204.stdout: openssl x86_64 1:3.0.7-27.el9 baseos 1.2 M 2024-03-08T20:43:47.020 INFO:teuthology.orchestra.run.smithi204.stdout: openssl-libs x86_64 1:3.0.7-27.el9 baseos 2.2 M 2024-03-08T20:43:47.020 INFO:teuthology.orchestra.run.smithi204.stdout: util-linux x86_64 2.37.4-18.el9 baseos 2.3 M 2024-03-08T20:43:47.020 INFO:teuthology.orchestra.run.smithi204.stdout: util-linux-core x86_64 2.37.4-18.el9 baseos 465 k 2024-03-08T20:43:47.020 INFO:teuthology.orchestra.run.smithi204.stdout: util-linux-user x86_64 2.37.4-18.el9 baseos 31 k 2024-03-08T20:43:47.020 INFO:teuthology.orchestra.run.smithi204.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-03-08T20:43:47.020 INFO:teuthology.orchestra.run.smithi204.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-03-08T20:43:47.020 INFO:teuthology.orchestra.run.smithi204.stdout:Installing dependencies: 2024-03-08T20:43:47.020 INFO:teuthology.orchestra.run.smithi204.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-03-08T20:43:47.020 INFO:teuthology.orchestra.run.smithi204.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-03-08T20:43:47.020 INFO:teuthology.orchestra.run.smithi204.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-03-08T20:43:47.020 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-common x86_64 2:19.0.0-1965.g7d63482a.el9 ceph 20 M 2024-03-08T20:43:47.021 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-grafana-dashboards noarch 2:19.0.0-1965.g7d63482a.el9 ceph-noarch 34 k 2024-03-08T20:43:47.021 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mds x86_64 2:19.0.0-1965.g7d63482a.el9 ceph 2.1 M 2024-03-08T20:43:47.021 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-modules-core noarch 2:19.0.0-1965.g7d63482a.el9 ceph-noarch 248 k 2024-03-08T20:43:47.021 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mon x86_64 2:19.0.0-1965.g7d63482a.el9 ceph 4.7 M 2024-03-08T20:43:47.021 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-osd x86_64 2:19.0.0-1965.g7d63482a.el9 ceph 17 M 2024-03-08T20:43:47.021 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-prometheus-alerts noarch 2:19.0.0-1965.g7d63482a.el9 ceph-noarch 16 k 2024-03-08T20:43:47.021 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-selinux x86_64 2:19.0.0-1965.g7d63482a.el9 ceph 25 k 2024-03-08T20:43:47.021 INFO:teuthology.orchestra.run.smithi204.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-03-08T20:43:47.021 INFO:teuthology.orchestra.run.smithi204.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-03-08T20:43:47.021 INFO:teuthology.orchestra.run.smithi204.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-03-08T20:43:47.021 INFO:teuthology.orchestra.run.smithi204.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-03-08T20:43:47.021 INFO:teuthology.orchestra.run.smithi204.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-03-08T20:43:47.021 INFO:teuthology.orchestra.run.smithi204.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-03-08T20:43:47.022 INFO:teuthology.orchestra.run.smithi204.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-03-08T20:43:47.022 INFO:teuthology.orchestra.run.smithi204.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-03-08T20:43:47.022 INFO:teuthology.orchestra.run.smithi204.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-03-08T20:43:47.022 INFO:teuthology.orchestra.run.smithi204.stdout: libcephsqlite x86_64 2:19.0.0-1965.g7d63482a.el9 ceph 163 k 2024-03-08T20:43:47.022 INFO:teuthology.orchestra.run.smithi204.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-03-08T20:43:47.022 INFO:teuthology.orchestra.run.smithi204.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-03-08T20:43:47.022 INFO:teuthology.orchestra.run.smithi204.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-03-08T20:43:47.022 INFO:teuthology.orchestra.run.smithi204.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-03-08T20:43:47.022 INFO:teuthology.orchestra.run.smithi204.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-03-08T20:43:47.022 INFO:teuthology.orchestra.run.smithi204.stdout: libradosstriper1 x86_64 2:19.0.0-1965.g7d63482a.el9 ceph 501 k 2024-03-08T20:43:47.022 INFO:teuthology.orchestra.run.smithi204.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-03-08T20:43:47.022 INFO:teuthology.orchestra.run.smithi204.stdout: librgw2 x86_64 2:19.0.0-1965.g7d63482a.el9 ceph 5.1 M 2024-03-08T20:43:47.023 INFO:teuthology.orchestra.run.smithi204.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-03-08T20:43:47.023 INFO:teuthology.orchestra.run.smithi204.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-03-08T20:43:47.023 INFO:teuthology.orchestra.run.smithi204.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-03-08T20:43:47.023 INFO:teuthology.orchestra.run.smithi204.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-03-08T20:43:47.023 INFO:teuthology.orchestra.run.smithi204.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-03-08T20:43:47.023 INFO:teuthology.orchestra.run.smithi204.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-03-08T20:43:47.023 INFO:teuthology.orchestra.run.smithi204.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-03-08T20:43:47.023 INFO:teuthology.orchestra.run.smithi204.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-03-08T20:43:47.023 INFO:teuthology.orchestra.run.smithi204.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-03-08T20:43:47.023 INFO:teuthology.orchestra.run.smithi204.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-03-08T20:43:47.023 INFO:teuthology.orchestra.run.smithi204.stdout: postgresql-private-libs x86_64 13.11-1.el9 CentOS-AppStream 134 k 2024-03-08T20:43:47.023 INFO:teuthology.orchestra.run.smithi204.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-03-08T20:43:47.023 INFO:teuthology.orchestra.run.smithi204.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-03-08T20:43:47.023 INFO:teuthology.orchestra.run.smithi204.stdout: python3-asyncssh noarch 2.13.2-4.el9 epel 548 k 2024-03-08T20:43:47.024 INFO:teuthology.orchestra.run.smithi204.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-03-08T20:43:47.024 INFO:teuthology.orchestra.run.smithi204.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-03-08T20:43:47.024 INFO:teuthology.orchestra.run.smithi204.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-03-08T20:43:47.024 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-03-08T20:43:47.024 INFO:teuthology.orchestra.run.smithi204.stdout: python3-ceph-argparse x86_64 2:19.0.0-1965.g7d63482a.el9 ceph 45 k 2024-03-08T20:43:47.024 INFO:teuthology.orchestra.run.smithi204.stdout: python3-ceph-common x86_64 2:19.0.0-1965.g7d63482a.el9 ceph 131 k 2024-03-08T20:43:47.024 INFO:teuthology.orchestra.run.smithi204.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-03-08T20:43:47.024 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-03-08T20:43:47.024 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-03-08T20:43:47.024 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-03-08T20:43:47.024 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-03-08T20:43:47.024 INFO:teuthology.orchestra.run.smithi204.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-03-08T20:43:47.024 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-03-08T20:43:47.024 INFO:teuthology.orchestra.run.smithi204.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-03-08T20:43:47.025 INFO:teuthology.orchestra.run.smithi204.stdout: python3-google-auth noarch 1:2.28.0-1.el9 epel 203 k 2024-03-08T20:43:47.025 INFO:teuthology.orchestra.run.smithi204.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-03-08T20:43:47.025 INFO:teuthology.orchestra.run.smithi204.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-03-08T20:43:47.025 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-03-08T20:43:47.025 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-03-08T20:43:47.025 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-03-08T20:43:47.025 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-03-08T20:43:47.025 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-03-08T20:43:47.025 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-03-08T20:43:47.025 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-03-08T20:43:47.026 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-03-08T20:43:47.026 INFO:teuthology.orchestra.run.smithi204.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-03-08T20:43:47.026 INFO:teuthology.orchestra.run.smithi204.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-03-08T20:43:47.026 INFO:teuthology.orchestra.run.smithi204.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-03-08T20:43:47.026 INFO:teuthology.orchestra.run.smithi204.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-03-08T20:43:47.026 INFO:teuthology.orchestra.run.smithi204.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-03-08T20:43:47.026 INFO:teuthology.orchestra.run.smithi204.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-03-08T20:43:47.026 INFO:teuthology.orchestra.run.smithi204.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-03-08T20:43:47.026 INFO:teuthology.orchestra.run.smithi204.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-03-08T20:43:47.026 INFO:teuthology.orchestra.run.smithi204.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-03-08T20:43:47.026 INFO:teuthology.orchestra.run.smithi204.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-03-08T20:43:47.026 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-03-08T20:43:47.026 INFO:teuthology.orchestra.run.smithi204.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-03-08T20:43:47.027 INFO:teuthology.orchestra.run.smithi204.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-03-08T20:43:47.027 INFO:teuthology.orchestra.run.smithi204.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-03-08T20:43:47.027 INFO:teuthology.orchestra.run.smithi204.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-03-08T20:43:47.027 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-03-08T20:43:47.027 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-03-08T20:43:47.027 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-03-08T20:43:47.027 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-03-08T20:43:47.027 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-03-08T20:43:47.027 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-03-08T20:43:47.027 INFO:teuthology.orchestra.run.smithi204.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-03-08T20:43:47.027 INFO:teuthology.orchestra.run.smithi204.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-03-08T20:43:47.027 INFO:teuthology.orchestra.run.smithi204.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-03-08T20:43:47.027 INFO:teuthology.orchestra.run.smithi204.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-03-08T20:43:47.027 INFO:teuthology.orchestra.run.smithi204.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-03-08T20:43:47.028 INFO:teuthology.orchestra.run.smithi204.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-03-08T20:43:47.028 INFO:teuthology.orchestra.run.smithi204.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-03-08T20:43:47.028 INFO:teuthology.orchestra.run.smithi204.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-03-08T20:43:47.028 INFO:teuthology.orchestra.run.smithi204.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-03-08T20:43:47.028 INFO:teuthology.orchestra.run.smithi204.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-03-08T20:43:47.028 INFO:teuthology.orchestra.run.smithi204.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-03-08T20:43:47.028 INFO:teuthology.orchestra.run.smithi204.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-03-08T20:43:47.028 INFO:teuthology.orchestra.run.smithi204.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-03-08T20:43:47.028 INFO:teuthology.orchestra.run.smithi204.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-03-08T20:43:47.028 INFO:teuthology.orchestra.run.smithi204.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-03-08T20:43:47.028 INFO:teuthology.orchestra.run.smithi204.stdout: qatlib x86_64 23.11.0-1.el9 appstream 216 k 2024-03-08T20:43:47.028 INFO:teuthology.orchestra.run.smithi204.stdout: qatzip-libs x86_64 1.1.2-2.el9 appstream 44 k 2024-03-08T20:43:47.028 INFO:teuthology.orchestra.run.smithi204.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-03-08T20:43:47.029 INFO:teuthology.orchestra.run.smithi204.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-03-08T20:43:47.029 INFO:teuthology.orchestra.run.smithi204.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-03-08T20:43:47.029 INFO:teuthology.orchestra.run.smithi204.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-03-08T20:43:47.029 INFO:teuthology.orchestra.run.smithi204.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-03-08T20:43:47.029 INFO:teuthology.orchestra.run.smithi204.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-03-08T20:43:47.029 INFO:teuthology.orchestra.run.smithi204.stdout:Installing weak dependencies: 2024-03-08T20:43:47.029 INFO:teuthology.orchestra.run.smithi204.stdout: qatlib-service x86_64 23.11.0-1.el9 appstream 35 k 2024-03-08T20:43:47.029 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-03-08T20:43:47.029 INFO:teuthology.orchestra.run.smithi204.stdout:Transaction Summary 2024-03-08T20:43:47.029 INFO:teuthology.orchestra.run.smithi204.stdout:============================================================================================ 2024-03-08T20:43:47.030 INFO:teuthology.orchestra.run.smithi204.stdout:Install 150 Packages 2024-03-08T20:43:47.030 INFO:teuthology.orchestra.run.smithi204.stdout:Upgrade 24 Packages 2024-03-08T20:43:47.030 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-03-08T20:43:47.030 INFO:teuthology.orchestra.run.smithi204.stdout:Total download size: 223 M 2024-03-08T20:43:47.030 INFO:teuthology.orchestra.run.smithi204.stdout:Downloading Packages: 2024-03-08T20:43:47.031 INFO:teuthology.orchestra.run.smithi149.stdout:(131/174): python3-jaraco-text-3.2.0-6.el9.noar 134 kB/s | 20 kB 00:00 2024-03-08T20:43:47.060 INFO:teuthology.orchestra.run.smithi149.stdout:(132/174): python3-logutils-0.3.5-21.el9.noarch 623 kB/s | 46 kB 00:00 2024-03-08T20:43:47.110 INFO:teuthology.orchestra.run.smithi149.stdout:(133/174): python3-kubernetes-26.1.0-2.el9.noar 4.9 MB/s | 1.0 MB 00:00 2024-03-08T20:43:47.127 INFO:teuthology.orchestra.run.smithi149.stdout:(134/174): python3-more-itertools-8.12.0-2.el9. 838 kB/s | 79 kB 00:00 2024-03-08T20:43:47.152 INFO:teuthology.orchestra.run.smithi149.stdout:(135/174): python3-natsort-7.1.1-5.el9.noarch.r 630 kB/s | 58 kB 00:00 2024-03-08T20:43:47.186 INFO:teuthology.orchestra.run.smithi149.stdout:(136/174): python3-pecan-1.4.2-3.el9.noarch.rpm 3.6 MB/s | 272 kB 00:00 2024-03-08T20:43:47.202 INFO:teuthology.orchestra.run.smithi149.stdout:(137/174): python3-portend-3.1.0-2.el9.noarch.r 219 kB/s | 16 kB 00:00 2024-03-08T20:43:47.228 INFO:teuthology.orchestra.run.smithi149.stdout:(138/174): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.2 MB/s | 90 kB 00:00 2024-03-08T20:43:47.245 INFO:teuthology.orchestra.run.smithi149.stdout:(139/174): python3-pyperclip-1.8.2-2.el9.noarch 486 kB/s | 28 kB 00:00 2024-03-08T20:43:47.270 INFO:teuthology.orchestra.run.smithi149.stdout:(140/174): python3-repoze-lru-0.7-16.el9.noarch 462 kB/s | 31 kB 00:00 2024-03-08T20:43:47.303 INFO:teuthology.orchestra.run.smithi149.stdout:(141/174): python3-routes-2.5.1-5.el9.noarch.rp 2.5 MB/s | 188 kB 00:00 2024-03-08T20:43:47.320 INFO:teuthology.orchestra.run.smithi149.stdout:(142/174): python3-rsa-4.9-2.el9.noarch.rpm 790 kB/s | 59 kB 00:00 2024-03-08T20:43:47.337 INFO:teuthology.orchestra.run.smithi149.stdout:(143/174): python3-tempora-5.0.0-2.el9.noarch.r 538 kB/s | 36 kB 00:00 2024-03-08T20:43:47.371 INFO:teuthology.orchestra.run.smithi149.stdout:(144/174): python3-typing-extensions-4.4.0-2.el 766 kB/s | 51 kB 00:00 2024-03-08T20:43:47.405 INFO:teuthology.orchestra.run.smithi149.stdout:(145/174): python3-webob-1.8.7-6.el9.noarch.rpm 2.7 MB/s | 230 kB 00:00 2024-03-08T20:43:47.430 INFO:teuthology.orchestra.run.smithi149.stdout:(146/174): python3-websocket-client-1.2.3-2.el9 978 kB/s | 90 kB 00:00 2024-03-08T20:43:47.463 INFO:teuthology.orchestra.run.smithi149.stdout:(147/174): python3-werkzeug-2.0.3-3.el9.1.noarc 4.6 MB/s | 427 kB 00:00 2024-03-08T20:43:47.489 INFO:teuthology.orchestra.run.smithi149.stdout:(148/174): python3-zc-lockfile-2.0-10.el9.noarc 240 kB/s | 20 kB 00:00 2024-03-08T20:43:47.522 INFO:teuthology.orchestra.run.smithi149.stdout:(149/174): re2-20211101-3.el9.x86_64.rpm 2.0 MB/s | 192 kB 00:00 2024-03-08T20:43:47.614 INFO:teuthology.orchestra.run.smithi149.stdout:(150/174): thrift-0.15.0-2.el9.x86_64.rpm 11 MB/s | 1.6 MB 00:00 2024-03-08T20:43:47.690 INFO:teuthology.orchestra.run.smithi149.stdout:(151/174): xfsdump-3.1.12-4.el9.x86_64.rpm 4.2 MB/s | 321 kB 00:00 2024-03-08T20:43:47.857 INFO:teuthology.orchestra.run.smithi149.stdout:(152/174): librbd1-19.0.0-1965.g7d63482a.el9.x8 9.4 MB/s | 3.1 MB 00:00 2024-03-08T20:43:48.322 INFO:teuthology.orchestra.run.smithi204.stdout:(1/174): ceph-19.0.0-1965.g7d63482a.el9.x86_64. 16 kB/s | 6.6 kB 00:00 2024-03-08T20:43:48.348 INFO:teuthology.orchestra.run.smithi149.stdout:(153/174): NetworkManager-libnm-1.46.0-1.el9.x8 3.7 MB/s | 1.8 MB 00:00 2024-03-08T20:43:48.432 INFO:teuthology.orchestra.run.smithi149.stdout:(154/174): NetworkManager-team-1.46.0-1.el9.x86 512 kB/s | 42 kB 00:00 2024-03-08T20:43:48.583 INFO:teuthology.orchestra.run.smithi149.stdout:(155/174): NetworkManager-tui-1.46.0-1.el9.x86_ 1.6 MB/s | 248 kB 00:00 2024-03-08T20:43:48.589 INFO:teuthology.orchestra.run.smithi204.stdout:(2/174): ceph-base-19.0.0-1965.g7d63482a.el9.x8 7.9 MB/s | 5.4 MB 00:00 2024-03-08T20:43:48.639 INFO:teuthology.orchestra.run.smithi204.stdout:(3/174): ceph-immutable-object-cache-19.0.0-196 2.9 MB/s | 145 kB 00:00 2024-03-08T20:43:48.649 INFO:teuthology.orchestra.run.smithi149.stdout:(156/174): acl-2.3.1-4.el9.x86_64.rpm 1.1 MB/s | 71 kB 00:00 2024-03-08T20:43:48.798 INFO:teuthology.orchestra.run.smithi204.stdout:(4/174): ceph-mds-19.0.0-1965.g7d63482a.el9.x86 13 MB/s | 2.1 MB 00:00 2024-03-08T20:43:48.853 INFO:teuthology.orchestra.run.smithi149.stdout:(157/174): elfutils-debuginfod-client-0.190-2.e 180 kB/s | 36 kB 00:00 2024-03-08T20:43:48.915 INFO:teuthology.orchestra.run.smithi204.stdout:(5/174): ceph-mgr-19.0.0-1965.g7d63482a.el9.x86 13 MB/s | 1.5 MB 00:00 2024-03-08T20:43:49.034 INFO:teuthology.orchestra.run.smithi149.stdout:(158/174): elfutils-libelf-0.190-2.el9.x86_64.r 1.0 MB/s | 192 kB 00:00 2024-03-08T20:43:49.125 INFO:teuthology.orchestra.run.smithi149.stdout:(159/174): elfutils-libs-0.190-2.el9.x86_64.rpm 2.7 MB/s | 256 kB 00:00 2024-03-08T20:43:49.207 INFO:teuthology.orchestra.run.smithi204.stdout:(6/174): ceph-mon-19.0.0-1965.g7d63482a.el9.x86 16 MB/s | 4.7 MB 00:00 2024-03-08T20:43:49.209 INFO:teuthology.orchestra.run.smithi149.stdout:(160/174): libacl-2.3.1-4.el9.x86_64.rpm 278 kB/s | 23 kB 00:00 2024-03-08T20:43:49.293 INFO:teuthology.orchestra.run.smithi149.stdout:(161/174): librados2-19.0.0-1965.g7d63482a.el9. 1.9 MB/s | 3.4 MB 00:01 2024-03-08T20:43:49.321 INFO:teuthology.orchestra.run.smithi149.stdout:(162/174): libblkid-2.37.4-18.el9.x86_64.rpm 975 kB/s | 108 kB 00:00 2024-03-08T20:43:49.430 INFO:teuthology.orchestra.run.smithi149.stdout:(163/174): libmount-2.37.4-18.el9.x86_64.rpm 1.2 MB/s | 136 kB 00:00 2024-03-08T20:43:49.497 INFO:teuthology.orchestra.run.smithi149.stdout:(164/174): libsmartcols-2.37.4-18.el9.x86_64.rp 971 kB/s | 64 kB 00:00 2024-03-08T20:43:49.522 INFO:teuthology.orchestra.run.smithi149.stdout:(165/174): libfdisk-2.37.4-18.el9.x86_64.rpm 678 kB/s | 155 kB 00:00 2024-03-08T20:43:49.564 INFO:teuthology.orchestra.run.smithi149.stdout:(166/174): libuuid-2.37.4-18.el9.x86_64.rpm 431 kB/s | 29 kB 00:00 2024-03-08T20:43:49.848 INFO:teuthology.orchestra.run.smithi149.stdout:(167/174): openssl-3.0.7-27.el9.x86_64.rpm 3.7 MB/s | 1.2 MB 00:00 2024-03-08T20:43:49.990 INFO:teuthology.orchestra.run.smithi149.stdout:(168/174): NetworkManager-1.46.0-1.el9.x86_64.r 1.0 MB/s | 2.3 MB 00:02 2024-03-08T20:43:50.057 INFO:teuthology.orchestra.run.smithi204.stdout:(7/174): ceph-osd-19.0.0-1965.g7d63482a.el9.x86 20 MB/s | 17 MB 00:00 2024-03-08T20:43:50.107 INFO:teuthology.orchestra.run.smithi204.stdout:(8/174): ceph-fuse-19.0.0-1965.g7d63482a.el9.x8 498 kB/s | 889 kB 00:01 2024-03-08T20:43:50.249 INFO:teuthology.orchestra.run.smithi204.stdout:(9/174): ceph-selinux-19.0.0-1965.g7d63482a.el9 179 kB/s | 25 kB 00:00 2024-03-08T20:43:50.457 INFO:teuthology.orchestra.run.smithi149.stdout:(169/174): util-linux-2.37.4-18.el9.x86_64.rpm 3.8 MB/s | 2.3 MB 00:00 2024-03-08T20:43:50.524 INFO:teuthology.orchestra.run.smithi149.stdout:(170/174): util-linux-core-2.37.4-18.el9.x86_64 873 kB/s | 465 kB 00:00 2024-03-08T20:43:50.549 INFO:teuthology.orchestra.run.smithi149.stdout:(171/174): util-linux-user-2.37.4-18.el9.x86_64 342 kB/s | 31 kB 00:00 2024-03-08T20:43:50.649 INFO:teuthology.orchestra.run.smithi149.stdout:(172/174): openssl-libs-3.0.7-27.el9.x86_64.rpm 2.0 MB/s | 2.2 MB 00:01 2024-03-08T20:43:50.658 INFO:teuthology.orchestra.run.smithi204.stdout:(10/174): ceph-radosgw-19.0.0-1965.g7d63482a.el 16 MB/s | 9.8 MB 00:00 2024-03-08T20:43:50.675 INFO:teuthology.orchestra.run.smithi149.stdout:(173/174): libacl-devel-2.3.1-4.el9.x86_64.rpm 597 kB/s | 74 kB 00:00 2024-03-08T20:43:50.708 INFO:teuthology.orchestra.run.smithi204.stdout:(11/174): cephfs-mirror-19.0.0-1965.g7d63482a.e 4.4 MB/s | 220 kB 00:00 2024-03-08T20:43:50.750 INFO:teuthology.orchestra.run.smithi204.stdout:(12/174): cephfs-shell-19.0.0-1965.g7d63482a.el 661 kB/s | 27 kB 00:00 2024-03-08T20:43:50.792 INFO:teuthology.orchestra.run.smithi204.stdout:(13/174): libcephfs-devel-19.0.0-1965.g7d63482a 765 kB/s | 31 kB 00:00 2024-03-08T20:43:50.876 INFO:teuthology.orchestra.run.smithi204.stdout:(14/174): libcephfs2-19.0.0-1965.g7d63482a.el9. 8.8 MB/s | 745 kB 00:00 2024-03-08T20:43:50.934 INFO:teuthology.orchestra.run.smithi204.stdout:(15/174): libcephsqlite-19.0.0-1965.g7d63482a.e 2.8 MB/s | 163 kB 00:00 2024-03-08T20:43:50.985 INFO:teuthology.orchestra.run.smithi204.stdout:(16/174): librados-devel-19.0.0-1965.g7d63482a. 2.5 MB/s | 127 kB 00:00 2024-03-08T20:43:51.043 INFO:teuthology.orchestra.run.smithi204.stdout:(17/174): libradosstriper1-19.0.0-1965.g7d63482 8.5 MB/s | 501 kB 00:00 2024-03-08T20:43:51.369 INFO:teuthology.orchestra.run.smithi204.stdout:(18/174): librgw2-19.0.0-1965.g7d63482a.el9.x86 16 MB/s | 5.1 MB 00:00 2024-03-08T20:43:51.419 INFO:teuthology.orchestra.run.smithi204.stdout:(19/174): python3-ceph-argparse-19.0.0-1965.g7d 922 kB/s | 45 kB 00:00 2024-03-08T20:43:51.469 INFO:teuthology.orchestra.run.smithi204.stdout:(20/174): python3-ceph-common-19.0.0-1965.g7d63 2.6 MB/s | 131 kB 00:00 2024-03-08T20:43:51.520 INFO:teuthology.orchestra.run.smithi204.stdout:(21/174): python3-cephfs-19.0.0-1965.g7d63482a. 3.2 MB/s | 162 kB 00:00 2024-03-08T20:43:51.533 INFO:teuthology.orchestra.run.smithi149.stdout:(174/174): xfsprogs-6.3.0-1.el9.x86_64.rpm 1.1 MB/s | 1.1 MB 00:01 2024-03-08T20:43:51.542 INFO:teuthology.orchestra.run.smithi149.stdout:-------------------------------------------------------------------------------- 2024-03-08T20:43:51.542 INFO:teuthology.orchestra.run.smithi149.stdout:Total 13 MB/s | 223 MB 00:17 2024-03-08T20:43:51.578 INFO:teuthology.orchestra.run.smithi204.stdout:(22/174): python3-rados-19.0.0-1965.g7d63482a.e 5.5 MB/s | 320 kB 00:00 2024-03-08T20:43:51.662 INFO:teuthology.orchestra.run.smithi204.stdout:(23/174): python3-rbd-19.0.0-1965.g7d63482a.el9 3.5 MB/s | 298 kB 00:00 2024-03-08T20:43:51.704 INFO:teuthology.orchestra.run.smithi204.stdout:(24/174): python3-rgw-19.0.0-1965.g7d63482a.el9 2.4 MB/s | 99 kB 00:00 2024-03-08T20:43:51.754 INFO:teuthology.orchestra.run.smithi204.stdout:(25/174): rbd-fuse-19.0.0-1965.g7d63482a.el9.x8 1.7 MB/s | 85 kB 00:00 2024-03-08T20:43:51.954 INFO:teuthology.orchestra.run.smithi204.stdout:(26/174): rbd-mirror-19.0.0-1965.g7d63482a.el9. 16 MB/s | 3.1 MB 00:00 2024-03-08T20:43:51.997 INFO:teuthology.orchestra.run.smithi204.stdout:(27/174): rbd-nbd-19.0.0-1965.g7d63482a.el9.x86 4.1 MB/s | 171 kB 00:00 2024-03-08T20:43:52.047 INFO:teuthology.orchestra.run.smithi204.stdout:(28/174): ceph-grafana-dashboards-19.0.0-1965.g 690 kB/s | 34 kB 00:00 2024-03-08T20:43:52.097 INFO:teuthology.orchestra.run.smithi204.stdout:(29/174): ceph-mgr-cephadm-19.0.0-1965.g7d63482 2.8 MB/s | 139 kB 00:00 2024-03-08T20:43:52.230 INFO:teuthology.orchestra.run.smithi204.stdout:(30/174): ceph-mgr-dashboard-19.0.0-1965.g7d634 14 MB/s | 1.8 MB 00:00 2024-03-08T20:43:52.664 INFO:teuthology.orchestra.run.smithi204.stdout:(31/174): ceph-mgr-diskprediction-local-19.0.0- 17 MB/s | 7.4 MB 00:00 2024-03-08T20:43:52.748 INFO:teuthology.orchestra.run.smithi204.stdout:(32/174): ceph-mgr-modules-core-19.0.0-1965.g7d 2.9 MB/s | 248 kB 00:00 2024-03-08T20:43:52.798 INFO:teuthology.orchestra.run.smithi204.stdout:(33/174): ceph-mgr-rook-19.0.0-1965.g7d63482a.e 1.0 MB/s | 49 kB 00:00 2024-03-08T20:43:52.848 INFO:teuthology.orchestra.run.smithi204.stdout:(34/174): ceph-prometheus-alerts-19.0.0-1965.g7 333 kB/s | 16 kB 00:00 2024-03-08T20:43:52.907 INFO:teuthology.orchestra.run.smithi204.stdout:(35/174): ceph-volume-19.0.0-1965.g7d63482a.el9 4.5 MB/s | 264 kB 00:00 2024-03-08T20:43:52.982 INFO:teuthology.orchestra.run.smithi204.stdout:(36/174): cephadm-19.0.0-1965.g7d63482a.el9.noa 9.8 MB/s | 748 kB 00:00 2024-03-08T20:43:52.998 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction check 2024-03-08T20:43:53.032 INFO:teuthology.orchestra.run.smithi204.stdout:(37/174): cephfs-top-19.0.0-1965.g7d63482a.el9. 371 kB/s | 18 kB 00:00 2024-03-08T20:43:53.377 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction check succeeded. 2024-03-08T20:43:53.377 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction test 2024-03-08T20:43:53.541 INFO:teuthology.orchestra.run.smithi204.stdout:(38/174): automake-1.16.2-8.el9.noarch.rpm 1.3 MB/s | 662 kB 00:00 2024-03-08T20:43:53.691 INFO:teuthology.orchestra.run.smithi204.stdout:(39/174): bison-3.7.4-5.el9.x86_64.rpm 6.0 MB/s | 920 kB 00:00 2024-03-08T20:43:53.767 INFO:teuthology.orchestra.run.smithi204.stdout:(40/174): boost-program-options-1.75.0-8.el9.x8 1.4 MB/s | 106 kB 00:00 2024-03-08T20:43:53.859 INFO:teuthology.orchestra.run.smithi204.stdout:(41/174): flex-2.6.4-9.el9.x86_64.rpm 3.3 MB/s | 307 kB 00:00 2024-03-08T20:43:53.934 INFO:teuthology.orchestra.run.smithi204.stdout:(42/174): flexiblas-3.0.4-8.el9.x86_64.rpm 427 kB/s | 32 kB 00:00 2024-03-08T20:43:54.193 INFO:teuthology.orchestra.run.smithi204.stdout:(43/174): flexiblas-netlib-3.0.4-8.el9.x86_64.r 12 MB/s | 3.0 MB 00:00 2024-03-08T20:43:54.243 INFO:teuthology.orchestra.run.smithi204.stdout:(44/174): flexiblas-openblas-openmp-3.0.4-8.el9 349 kB/s | 17 kB 00:00 2024-03-08T20:43:54.302 INFO:teuthology.orchestra.run.smithi204.stdout:(45/174): libaio-devel-0.3.111-13.el9.x86_64.rp 174 kB/s | 10 kB 00:00 2024-03-08T20:43:54.373 INFO:teuthology.orchestra.run.smithi107.stdout:CentOS Stream 9 - AppStream 492 kB/s | 19 MB 00:39 2024-03-08T20:43:54.377 INFO:teuthology.orchestra.run.smithi204.stdout:(46/174): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.1 MB/s | 159 kB 00:00 2024-03-08T20:43:54.435 INFO:teuthology.orchestra.run.smithi204.stdout:(47/174): librabbitmq-0.11.0-7.el9.x86_64.rpm 766 kB/s | 44 kB 00:00 2024-03-08T20:43:54.511 INFO:teuthology.orchestra.run.smithi204.stdout:(48/174): librdkafka-1.6.1-102.el9.x86_64.rpm 8.7 MB/s | 661 kB 00:00 2024-03-08T20:43:54.578 INFO:teuthology.orchestra.run.smithi204.stdout:(49/174): libtool-2.4.6-45.el9.x86_64.rpm 8.6 MB/s | 578 kB 00:00 2024-03-08T20:43:54.628 INFO:teuthology.orchestra.run.smithi204.stdout:(50/174): libzstd-devel-1.5.1-2.el9.x86_64.rpm 933 kB/s | 46 kB 00:00 2024-03-08T20:43:54.687 INFO:teuthology.orchestra.run.smithi204.stdout:(51/174): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.8 MB/s | 282 kB 00:00 2024-03-08T20:43:54.737 INFO:teuthology.orchestra.run.smithi204.stdout:(52/174): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 187 kB 00:00 2024-03-08T20:43:54.804 INFO:teuthology.orchestra.run.smithi204.stdout:(53/174): oniguruma-6.9.6-1.el9.5.x86_64.rpm 3.2 MB/s | 217 kB 00:00 2024-03-08T20:43:54.854 INFO:teuthology.orchestra.run.smithi204.stdout:(54/174): openblas-0.3.21-2.el9.x86_64.rpm 696 kB/s | 34 kB 00:00 2024-03-08T20:43:55.063 INFO:teuthology.orchestra.run.smithi204.stdout:(55/174): openblas-openmp-0.3.21-2.el9.x86_64.r 23 MB/s | 4.7 MB 00:00 2024-03-08T20:43:55.163 INFO:teuthology.orchestra.run.smithi204.stdout:(56/174): postgresql-13.11-1.el9.x86_64.rpm 15 MB/s | 1.5 MB 00:00 2024-03-08T20:43:55.239 INFO:teuthology.orchestra.run.smithi204.stdout:(57/174): postgresql-contrib-13.11-1.el9.x86_64 11 MB/s | 811 kB 00:00 2024-03-08T20:43:55.497 INFO:teuthology.orchestra.run.smithi204.stdout:(58/174): ceph-common-19.0.0-1965.g7d63482a.el9 2.6 MB/s | 20 MB 00:07 2024-03-08T20:43:55.522 INFO:teuthology.orchestra.run.smithi204.stdout:(59/174): postgresql-private-libs-13.11-1.el9.x 473 kB/s | 134 kB 00:00 2024-03-08T20:43:55.772 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction test succeeded. 2024-03-08T20:43:55.772 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction 2024-03-08T20:43:55.773 INFO:teuthology.orchestra.run.smithi204.stdout:(60/174): postgresql-server-13.11-1.el9.x86_64. 21 MB/s | 5.7 MB 00:00 2024-03-08T20:43:55.815 INFO:teuthology.orchestra.run.smithi204.stdout:(61/174): protobuf-3.14.0-13.el9.x86_64.rpm 3.5 MB/s | 1.0 MB 00:00 2024-03-08T20:43:55.840 INFO:teuthology.orchestra.run.smithi204.stdout:(62/174): python3-attrs-20.3.0-7.el9.noarch.rpm 1.3 MB/s | 78 kB 00:00 2024-03-08T20:43:56.049 INFO:teuthology.orchestra.run.smithi204.stdout:(63/174): python3-babel-2.9.1-2.el9.noarch.rpm 25 MB/s | 5.8 MB 00:00 2024-03-08T20:43:56.074 INFO:teuthology.orchestra.run.smithi204.stdout:(64/174): python3-jsonpatch-1.21-16.el9.noarch. 106 kB/s | 24 kB 00:00 2024-03-08T20:43:56.132 INFO:teuthology.orchestra.run.smithi204.stdout:(65/174): python3-jsonpointer-2.0-4.el9.noarch. 316 kB/s | 18 kB 00:00 2024-03-08T20:43:56.157 INFO:teuthology.orchestra.run.smithi204.stdout:(66/174): python3-mako-1.1.4-6.el9.noarch.rpm 1.7 MB/s | 144 kB 00:00 2024-03-08T20:43:56.199 INFO:teuthology.orchestra.run.smithi204.stdout:(67/174): python3-markupsafe-1.1.1-12.el9.x86_6 481 kB/s | 32 kB 00:00 2024-03-08T20:43:56.425 INFO:teuthology.orchestra.run.smithi204.stdout:(68/174): python3-numpy-1.20.1-5.el9.x86_64.rpm 18 MB/s | 4.8 MB 00:00 2024-03-08T20:43:56.450 INFO:teuthology.orchestra.run.smithi204.stdout:(69/174): python3-numpy-f2py-1.20.1-5.el9.x86_6 883 kB/s | 217 kB 00:00 2024-03-08T20:43:56.509 INFO:teuthology.orchestra.run.smithi204.stdout:(70/174): python3-packaging-20.9-5.el9.noarch.r 1.2 MB/s | 69 kB 00:00 2024-03-08T20:43:56.534 INFO:teuthology.orchestra.run.smithi204.stdout:(71/174): python3-oauthlib-3.1.1-5.el9.noarch.r 2.1 MB/s | 174 kB 00:00 2024-03-08T20:43:56.576 INFO:teuthology.orchestra.run.smithi204.stdout:(72/174): python3-prettytable-0.7.2-27.el9.noar 632 kB/s | 41 kB 00:00 2024-03-08T20:43:56.610 INFO:teuthology.orchestra.run.smithi204.stdout:(73/174): python3-protobuf-3.14.0-13.el9.noarch 3.1 MB/s | 238 kB 00:00 2024-03-08T20:43:56.635 INFO:teuthology.orchestra.run.smithi204.stdout:(74/174): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.3 MB/s | 132 kB 00:00 2024-03-08T20:43:56.668 INFO:teuthology.orchestra.run.smithi204.stdout:(75/174): python3-pyasn1-modules-0.4.8-6.el9.no 3.7 MB/s | 211 kB 00:00 2024-03-08T20:43:56.685 INFO:teuthology.orchestra.run.smithi204.stdout:(76/174): python3-pytz-2021.1-5.el9.noarch.rpm 949 kB/s | 47 kB 00:00 2024-03-08T20:43:56.711 INFO:teuthology.orchestra.run.smithi204.stdout:(77/174): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 43 kB 00:00 2024-03-08T20:43:56.945 INFO:teuthology.orchestra.run.smithi204.stdout:(78/174): python3-toml-0.10.2-6.el9.noarch.rpm 169 kB/s | 39 kB 00:00 2024-03-08T20:43:57.228 INFO:teuthology.orchestra.run.smithi204.stdout:(79/174): python3-scipy-1.6.2-8.el9.x86_64.rpm 29 MB/s | 16 MB 00:00 2024-03-08T20:43:57.245 INFO:teuthology.orchestra.run.smithi204.stdout:(80/174): python3-wcwidth-0.2.5-8.el9.noarch.rp 143 kB/s | 41 kB 00:00 2024-03-08T20:43:57.312 INFO:teuthology.orchestra.run.smithi204.stdout:(81/174): socat-1.7.4.1-5.el9.x86_64.rpm 4.5 MB/s | 300 kB 00:00 2024-03-08T20:43:57.329 INFO:teuthology.orchestra.run.smithi204.stdout:(82/174): uuid-1.6.2-55.el9.x86_64.rpm 684 kB/s | 56 kB 00:00 2024-03-08T20:43:57.354 INFO:teuthology.orchestra.run.smithi204.stdout:(83/174): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.5 MB/s | 63 kB 00:00 2024-03-08T20:43:57.371 INFO:teuthology.orchestra.run.smithi204.stdout:(84/174): zlib-devel-1.2.11-40.el9.x86_64.rpm 1.0 MB/s | 44 kB 00:00 2024-03-08T20:43:57.405 INFO:teuthology.orchestra.run.smithi204.stdout:(85/174): fuse-2.9.9-15.el9.x86_64.rpm 1.6 MB/s | 79 kB 00:00 2024-03-08T20:43:57.439 INFO:teuthology.orchestra.run.smithi204.stdout:(86/174): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.5 MB/s | 241 kB 00:00 2024-03-08T20:43:57.505 INFO:teuthology.orchestra.run.smithi204.stdout:(87/174): python3-cryptography-36.0.1-4.el9.x86 12 MB/s | 1.1 MB 00:00 2024-03-08T20:43:57.530 INFO:teuthology.orchestra.run.smithi204.stdout:(88/174): python3-ply-3.11-14.el9.noarch.rpm 1.2 MB/s | 103 kB 00:00 2024-03-08T20:43:57.556 INFO:teuthology.orchestra.run.smithi204.stdout:(89/174): python3-pycparser-2.20-6.el9.noarch.r 2.5 MB/s | 124 kB 00:00 2024-03-08T20:43:57.589 INFO:teuthology.orchestra.run.smithi204.stdout:(90/174): python3-urllib3-1.26.5-5.el9.noarch.r 3.2 MB/s | 187 kB 00:00 2024-03-08T20:43:58.707 INFO:teuthology.orchestra.run.smithi204.stdout:(91/174): jq-1.6-16.el9.x86_64.rpm 163 kB/s | 187 kB 00:01 2024-03-08T20:43:59.466 INFO:teuthology.orchestra.run.smithi107.stdout:CentOS Stream 9 - CRB 4.7 MB/s | 6.1 MB 00:01 2024-03-08T20:43:59.665 INFO:teuthology.orchestra.run.smithi204.stdout:(92/174): libgfortran-11.4.1-3.el9.x86_64.rpm 387 kB/s | 803 kB 00:02 2024-03-08T20:43:59.873 INFO:teuthology.orchestra.run.smithi204.stdout:(93/174): libquadmath-11.4.1-3.el9.x86_64.rpm 168 kB/s | 195 kB 00:01 2024-03-08T20:44:00.099 INFO:teuthology.orchestra.run.smithi204.stdout:(94/174): elfutils-libelf-devel-0.190-2.el9.x86 96 kB/s | 21 kB 00:00 2024-03-08T20:44:00.282 INFO:teuthology.orchestra.run.smithi204.stdout:(95/174): libuuid-devel-2.37.4-18.el9.x86_64.rp 139 kB/s | 25 kB 00:00 2024-03-08T20:44:00.791 INFO:teuthology.orchestra.run.smithi204.stdout:(96/174): python3-requests-2.25.1-8.el9.noarch. 111 kB/s | 125 kB 00:01 2024-03-08T20:44:01.158 INFO:teuthology.orchestra.run.smithi107.stdout:CentOS Stream 9 - Extras packages 46 kB/s | 15 kB 00:00 2024-03-08T20:44:01.533 INFO:teuthology.orchestra.run.smithi204.stdout:(97/174): python3-devel-3.9.17-2.el9.x86_64.rpm 331 kB/s | 245 kB 00:00 2024-03-08T20:44:02.041 INFO:teuthology.orchestra.run.smithi204.stdout:(98/174): python3-jinja2-2.11.3-5.el9.noarch.rp 488 kB/s | 248 kB 00:00 2024-03-08T20:44:02.809 INFO:teuthology.orchestra.run.smithi204.stdout:(99/174): qatlib-23.11.0-1.el9.x86_64.rpm 283 kB/s | 216 kB 00:00 2024-03-08T20:44:02.951 INFO:teuthology.orchestra.run.smithi204.stdout:(100/174): qatlib-service-23.11.0-1.el9.x86_64. 247 kB/s | 35 kB 00:00 2024-03-08T20:44:03.104 INFO:teuthology.orchestra.run.smithi149.stdout: Preparing : 1/1 2024-03-08T20:44:03.234 INFO:teuthology.orchestra.run.smithi204.stdout:(101/174): qatzip-libs-1.1.2-2.el9.x86_64.rpm 157 kB/s | 44 kB 00:00 2024-03-08T20:44:03.241 INFO:teuthology.orchestra.run.smithi149.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/198 2024-03-08T20:44:03.386 INFO:teuthology.orchestra.run.smithi149.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/198 2024-03-08T20:44:03.526 INFO:teuthology.orchestra.run.smithi107.stdout:Extra Packages for Enterprise Linux 9.4 MB/s | 21 MB 00:02 2024-03-08T20:44:03.527 INFO:teuthology.orchestra.run.smithi149.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-08T20:44:03.570 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-08T20:44:03.657 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/198 2024-03-08T20:44:03.750 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/198 2024-03-08T20:44:03.800 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/198 2024-03-08T20:44:03.818 INFO:teuthology.orchestra.run.smithi204.stdout:(102/174): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 161 kB/s | 94 kB 00:00 2024-03-08T20:44:03.842 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/198 2024-03-08T20:44:03.938 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/198 2024-03-08T20:44:03.985 INFO:teuthology.orchestra.run.smithi204.stdout:(103/174): lua-devel-5.4.4-4.el9.x86_64.rpm 134 kB/s | 22 kB 00:00 2024-03-08T20:44:04.375 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/198 2024-03-08T20:44:04.407 INFO:teuthology.orchestra.run.smithi149.stdout: Upgrading : librados2-2:19.0.0-1965.g7d63482a.el9.x86_64 10/198 2024-03-08T20:44:04.556 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: librados2-2:19.0.0-1965.g7d63482a.el9.x86_64 10/198 2024-03-08T20:44:04.583 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libcephfs2-2:19.0.0-1965.g7d63482a.el9.x86_64 11/198 2024-03-08T20:44:04.702 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: libcephfs2-2:19.0.0-1965.g7d63482a.el9.x86_64 11/198 2024-03-08T20:44:04.749 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-rados-2:19.0.0-1965.g7d63482a.el9.x86_64 12/198 2024-03-08T20:44:04.814 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 13/198 2024-03-08T20:44:04.886 INFO:teuthology.orchestra.run.smithi204.stdout:(104/174): protobuf-compiler-3.14.0-13.el9.x86_ 960 kB/s | 863 kB 00:00 2024-03-08T20:44:05.098 INFO:teuthology.orchestra.run.smithi149.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/198 2024-03-08T20:44:05.155 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : postgresql-13.11-1.el9.x86_64 15/198 2024-03-08T20:44:05.181 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libcephsqlite-2:19.0.0-1965.g7d63482a.el9.x86_64 16/198 2024-03-08T20:44:05.329 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: libcephsqlite-2:19.0.0-1965.g7d63482a.el9.x86_64 16/198 2024-03-08T20:44:05.356 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libradosstriper1-2:19.0.0-1965.g7d63482a.el9.x86 17/198 2024-03-08T20:44:05.446 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: libradosstriper1-2:19.0.0-1965.g7d63482a.el9.x86 17/198 2024-03-08T20:44:05.492 INFO:teuthology.orchestra.run.smithi149.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/198 2024-03-08T20:44:05.517 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : qatlib-service-23.11.0-1.el9.x86_64 19/198 2024-03-08T20:44:05.581 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: qatlib-service-23.11.0-1.el9.x86_64 19/198 2024-03-08T20:44:05.627 INFO:teuthology.orchestra.run.smithi204.stdout:(105/174): abseil-cpp-20211102.0-3.el9.x86_64.r 740 kB/s | 548 kB 00:00 2024-03-08T20:44:05.752 INFO:teuthology.orchestra.run.smithi204.stdout:(106/174): gperftools-libs-2.9.1-2.el9.x86_64.r 2.4 MB/s | 309 kB 00:00 2024-03-08T20:44:05.844 INFO:teuthology.orchestra.run.smithi204.stdout:(107/174): grpc-data-1.46.7-10.el9.noarch.rpm 212 kB/s | 19 kB 00:00 2024-03-08T20:44:05.910 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: qatlib-23.11.0-1.el9.x86_64 20/198 2024-03-08T20:44:05.967 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : qatlib-23.11.0-1.el9.x86_64 20/198 2024-03-08T20:44:06.009 INFO:teuthology.orchestra.run.smithi149.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/198 2024-03-08T20:44:06.270 INFO:teuthology.orchestra.run.smithi204.stdout:(108/174): libarrow-9.0.0-10.el9.x86_64.rpm 10 MB/s | 4.4 MB 00:00 2024-03-08T20:44:06.319 INFO:teuthology.orchestra.run.smithi149.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/198 2024-03-08T20:44:06.370 INFO:teuthology.orchestra.run.smithi204.stdout:(109/174): libarrow-doc-9.0.0-10.el9.noarch.rpm 262 kB/s | 26 kB 00:00 2024-03-08T20:44:06.470 INFO:teuthology.orchestra.run.smithi204.stdout:(110/174): liboath-2.6.7-2.el9.x86_64.rpm 489 kB/s | 49 kB 00:00 2024-03-08T20:44:06.524 INFO:teuthology.orchestra.run.smithi149.stdout: Upgrading : NetworkManager-libnm-1:1.46.0-1.el9.x86_64 23/198 2024-03-08T20:44:06.579 INFO:teuthology.orchestra.run.smithi204.stdout:(111/174): libunwind-1.6.2-1.el9.x86_64.rpm 626 kB/s | 67 kB 00:00 2024-03-08T20:44:06.613 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: NetworkManager-libnm-1:1.46.0-1.el9.x86_64 23/198 2024-03-08T20:44:06.688 INFO:teuthology.orchestra.run.smithi204.stdout:(112/174): luarocks-3.9.2-1.el9.noarch.rpm 1.4 MB/s | 151 kB 00:00 2024-03-08T20:44:06.813 INFO:teuthology.orchestra.run.smithi204.stdout:(113/174): parquet-libs-9.0.0-10.el9.x86_64.rpm 6.6 MB/s | 838 kB 00:00 2024-03-08T20:44:06.858 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: NetworkManager-1:1.46.0-1.el9.x86_64 24/198 2024-03-08T20:44:06.910 INFO:teuthology.orchestra.run.smithi149.stdout: Upgrading : NetworkManager-1:1.46.0-1.el9.x86_64 24/198 2024-03-08T20:44:06.938 INFO:teuthology.orchestra.run.smithi204.stdout:(114/174): python3-asyncssh-2.13.2-4.el9.noarch 4.3 MB/s | 548 kB 00:00 2024-03-08T20:44:07.055 INFO:teuthology.orchestra.run.smithi204.stdout:(115/174): python3-bcrypt-3.2.2-1.el9.x86_64.rp 375 kB/s | 43 kB 00:00 2024-03-08T20:44:07.155 INFO:teuthology.orchestra.run.smithi204.stdout:(116/174): python3-cachetools-4.2.4-1.el9.noarc 325 kB/s | 32 kB 00:00 2024-03-08T20:44:07.256 INFO:teuthology.orchestra.run.smithi204.stdout:(117/174): python3-certifi-2021.10.8-2.el9.noar 151 kB/s | 15 kB 00:00 2024-03-08T20:44:07.269 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: NetworkManager-1:1.46.0-1.el9.x86_64 24/198 2024-03-08T20:44:07.356 INFO:teuthology.orchestra.run.smithi204.stdout:(118/174): python3-cheroot-8.6.0-4.el9.noarch.r 1.7 MB/s | 172 kB 00:00 2024-03-08T20:44:07.463 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/198 2024-03-08T20:44:07.465 INFO:teuthology.orchestra.run.smithi204.stdout:(119/174): python3-cherrypy-18.6.1-2.el9.noarch 3.3 MB/s | 358 kB 00:00 2024-03-08T20:44:07.573 INFO:teuthology.orchestra.run.smithi204.stdout:(120/174): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.8 MB/s | 303 kB 00:00 2024-03-08T20:44:07.596 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/198 2024-03-08T20:44:07.650 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/198 2024-03-08T20:44:07.665 INFO:teuthology.orchestra.run.smithi204.stdout:(121/174): python3-colorama-0.4.4-12.el9.noarch 388 kB/s | 35 kB 00:00 2024-03-08T20:44:07.714 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/198 2024-03-08T20:44:07.774 INFO:teuthology.orchestra.run.smithi204.stdout:(122/174): python3-google-auth-2.28.0-1.el9.noa 1.8 MB/s | 203 kB 00:00 2024-03-08T20:44:07.781 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/198 2024-03-08T20:44:07.905 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/198 2024-03-08T20:44:07.986 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 31/198 2024-03-08T20:44:07.994 INFO:teuthology.orchestra.run.smithi204.stdout:(123/174): python3-grpcio-1.46.7-10.el9.x86_64. 9.3 MB/s | 2.0 MB 00:00 2024-03-08T20:44:08.035 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 32/198 2024-03-08T20:44:08.108 INFO:teuthology.orchestra.run.smithi204.stdout:(124/174): python3-grpcio-tools-1.46.7-10.el9.x 1.3 MB/s | 144 kB 00:00 2024-03-08T20:44:08.136 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/198 2024-03-08T20:44:08.188 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/198 2024-03-08T20:44:08.200 INFO:teuthology.orchestra.run.smithi204.stdout:(125/174): python3-jaraco-8.2.1-3.el9.noarch.rp 118 kB/s | 11 kB 00:00 2024-03-08T20:44:08.252 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/198 2024-03-08T20:44:08.292 INFO:teuthology.orchestra.run.smithi204.stdout:(126/174): python3-jaraco-classes-3.2.1-5.el9.n 194 kB/s | 18 kB 00:00 2024-03-08T20:44:08.301 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/198 2024-03-08T20:44:08.336 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : lua-5.4.4-4.el9.x86_64 37/198 2024-03-08T20:44:08.392 INFO:teuthology.orchestra.run.smithi204.stdout:(127/174): python3-jaraco-collections-3.0.0-8.e 234 kB/s | 23 kB 00:00 2024-03-08T20:44:08.452 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 38/198 2024-03-08T20:44:08.489 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-ceph-common-2:19.0.0-1965.g7d63482a.el9. 39/198 2024-03-08T20:44:08.493 INFO:teuthology.orchestra.run.smithi204.stdout:(128/174): python3-jaraco-functools-3.5.0-2.el9 195 kB/s | 19 kB 00:00 2024-03-08T20:44:08.544 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-ceph-argparse-2:19.0.0-1965.g7d63482a.el 40/198 2024-03-08T20:44:08.585 INFO:teuthology.orchestra.run.smithi204.stdout:(129/174): python3-jaraco-text-3.2.0-6.el9.noar 217 kB/s | 20 kB 00:00 2024-03-08T20:44:08.604 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-cephfs-2:19.0.0-1965.g7d63482a.el9.x86_6 41/198 2024-03-08T20:44:08.710 INFO:teuthology.orchestra.run.smithi204.stdout:(130/174): python3-kubernetes-26.1.0-2.el9.noar 8.2 MB/s | 1.0 MB 00:00 2024-03-08T20:44:08.728 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/198 2024-03-08T20:44:08.785 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/198 2024-03-08T20:44:08.802 INFO:teuthology.orchestra.run.smithi204.stdout:(131/174): python3-logutils-0.3.5-21.el9.noarch 507 kB/s | 46 kB 00:00 2024-03-08T20:44:08.893 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/198 2024-03-08T20:44:08.894 INFO:teuthology.orchestra.run.smithi204.stdout:(132/174): python3-more-itertools-8.12.0-2.el9. 865 kB/s | 79 kB 00:00 2024-03-08T20:44:08.986 INFO:teuthology.orchestra.run.smithi204.stdout:(133/174): python3-natsort-7.1.1-5.el9.noarch.r 635 kB/s | 58 kB 00:00 2024-03-08T20:44:09.069 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/198 2024-03-08T20:44:09.094 INFO:teuthology.orchestra.run.smithi204.stdout:(134/174): python3-pecan-1.4.2-3.el9.noarch.rpm 2.5 MB/s | 272 kB 00:00 2024-03-08T20:44:09.135 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/198 2024-03-08T20:44:09.204 INFO:teuthology.orchestra.run.smithi204.stdout:(135/174): python3-portend-3.1.0-2.el9.noarch.r 152 kB/s | 16 kB 00:00 2024-03-08T20:44:09.275 INFO:teuthology.orchestra.run.smithi107.stdout:lab-extras 36 kB/s | 1.7 kB 00:00 2024-03-08T20:44:09.320 INFO:teuthology.orchestra.run.smithi204.stdout:(136/174): python3-pyOpenSSL-21.0.0-1.el9.noarc 775 kB/s | 90 kB 00:00 2024-03-08T20:44:09.421 INFO:teuthology.orchestra.run.smithi204.stdout:(137/174): python3-pyperclip-1.8.2-2.el9.noarch 284 kB/s | 28 kB 00:00 2024-03-08T20:44:09.521 INFO:teuthology.orchestra.run.smithi204.stdout:(138/174): python3-repoze-lru-0.7-16.el9.noarch 310 kB/s | 31 kB 00:00 2024-03-08T20:44:09.630 INFO:teuthology.orchestra.run.smithi204.stdout:(139/174): python3-routes-2.5.1-5.el9.noarch.rp 1.7 MB/s | 188 kB 00:00 2024-03-08T20:44:09.738 INFO:teuthology.orchestra.run.smithi204.stdout:(140/174): python3-rsa-4.9-2.el9.noarch.rpm 549 kB/s | 59 kB 00:00 2024-03-08T20:44:09.824 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/198 2024-03-08T20:44:09.855 INFO:teuthology.orchestra.run.smithi204.stdout:(141/174): python3-tempora-5.0.0-2.el9.noarch.r 308 kB/s | 36 kB 00:00 2024-03-08T20:44:09.925 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/198 2024-03-08T20:44:09.956 INFO:teuthology.orchestra.run.smithi204.stdout:(142/174): python3-typing-extensions-4.4.0-2.el 514 kB/s | 51 kB 00:00 2024-03-08T20:44:10.040 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/198 2024-03-08T20:44:10.073 INFO:teuthology.orchestra.run.smithi204.stdout:(143/174): python3-webob-1.8.7-6.el9.noarch.rpm 1.9 MB/s | 230 kB 00:00 2024-03-08T20:44:10.092 INFO:teuthology.orchestra.run.smithi149.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-08T20:44:10.116 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-08T20:44:10.165 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : qatzip-libs-1.1.2-2.el9.x86_64 51/198 2024-03-08T20:44:10.181 INFO:teuthology.orchestra.run.smithi204.stdout:(144/174): python3-websocket-client-1.2.3-2.el9 833 kB/s | 90 kB 00:00 2024-03-08T20:44:10.227 INFO:teuthology.orchestra.run.smithi149.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 52/198 2024-03-08T20:44:10.282 INFO:teuthology.orchestra.run.smithi204.stdout:(145/174): python3-werkzeug-2.0.3-3.el9.1.noarc 4.2 MB/s | 427 kB 00:00 2024-03-08T20:44:10.284 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : librados-devel-2:19.0.0-1965.g7d63482a.el9.x86_6 53/198 2024-03-08T20:44:10.325 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 54/198 2024-03-08T20:44:10.374 INFO:teuthology.orchestra.run.smithi204.stdout:(146/174): python3-zc-lockfile-2.0-10.el9.noarc 220 kB/s | 20 kB 00:00 2024-03-08T20:44:10.491 INFO:teuthology.orchestra.run.smithi204.stdout:(147/174): re2-20211101-3.el9.x86_64.rpm 1.7 MB/s | 192 kB 00:00 2024-03-08T20:44:10.649 INFO:teuthology.orchestra.run.smithi204.stdout:(148/174): thrift-0.15.0-2.el9.x86_64.rpm 10 MB/s | 1.6 MB 00:00 2024-03-08T20:44:10.709 INFO:teuthology.orchestra.run.smithi149.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 55/198 2024-03-08T20:44:10.909 INFO:teuthology.orchestra.run.smithi149.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 56/198 2024-03-08T20:44:10.997 INFO:teuthology.orchestra.run.smithi149.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 57/198 2024-03-08T20:44:11.050 INFO:teuthology.orchestra.run.smithi204.stdout:(149/174): librados2-19.0.0-1965.g7d63482a.el9. 8.5 MB/s | 3.4 MB 00:00 2024-03-08T20:44:11.065 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 58/198 2024-03-08T20:44:11.267 INFO:teuthology.orchestra.run.smithi204.stdout:(150/174): librbd1-19.0.0-1965.g7d63482a.el9.x8 15 MB/s | 3.1 MB 00:00 2024-03-08T20:44:11.271 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 59/198 2024-03-08T20:44:11.352 INFO:teuthology.orchestra.run.smithi149.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 60/198 2024-03-08T20:44:11.359 INFO:teuthology.orchestra.run.smithi204.stdout:(151/174): xfsdump-3.1.12-4.el9.x86_64.rpm 3.5 MB/s | 321 kB 00:00 2024-03-08T20:44:11.433 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 61/198 2024-03-08T20:44:11.532 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 62/198 2024-03-08T20:44:11.576 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 63/198 2024-03-08T20:44:11.616 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 64/198 2024-03-08T20:44:11.684 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 65/198 2024-03-08T20:44:11.724 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 66/198 2024-03-08T20:44:11.786 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 67/198 2024-03-08T20:44:11.844 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 68/198 2024-03-08T20:44:11.949 INFO:teuthology.orchestra.run.smithi107.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-08T20:44:11.951 INFO:teuthology.orchestra.run.smithi107.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-08T20:44:11.973 INFO:teuthology.orchestra.run.smithi107.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-03-08T20:44:11.975 INFO:teuthology.orchestra.run.smithi107.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-03-08T20:44:11.976 INFO:teuthology.orchestra.run.smithi107.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-03-08T20:44:11.978 INFO:teuthology.orchestra.run.smithi107.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-03-08T20:44:11.980 INFO:teuthology.orchestra.run.smithi107.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-03-08T20:44:11.984 INFO:teuthology.orchestra.run.smithi107.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-03-08T20:44:11.986 INFO:teuthology.orchestra.run.smithi107.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-03-08T20:44:12.034 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 69/198 2024-03-08T20:44:12.073 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2024-03-08T20:44:12.098 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 70/198 2024-03-08T20:44:12.107 INFO:teuthology.orchestra.run.smithi107.stdout:============================================================================================ 2024-03-08T20:44:12.108 INFO:teuthology.orchestra.run.smithi107.stdout: Package Arch Version Repository Size 2024-03-08T20:44:12.108 INFO:teuthology.orchestra.run.smithi107.stdout:============================================================================================ 2024-03-08T20:44:12.109 INFO:teuthology.orchestra.run.smithi107.stdout:Installing: 2024-03-08T20:44:12.109 INFO:teuthology.orchestra.run.smithi107.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-03-08T20:44:12.109 INFO:teuthology.orchestra.run.smithi107.stdout: ceph x86_64 2:19.0.0-1965.g7d63482a.el9 ceph 6.6 k 2024-03-08T20:44:12.109 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-base x86_64 2:19.0.0-1965.g7d63482a.el9 ceph 5.4 M 2024-03-08T20:44:12.109 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-fuse x86_64 2:19.0.0-1965.g7d63482a.el9 ceph 889 k 2024-03-08T20:44:12.109 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-1965.g7d63482a.el9 ceph 145 k 2024-03-08T20:44:12.109 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr x86_64 2:19.0.0-1965.g7d63482a.el9 ceph 1.5 M 2024-03-08T20:44:12.109 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-cephadm noarch 2:19.0.0-1965.g7d63482a.el9 ceph-noarch 139 k 2024-03-08T20:44:12.109 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-dashboard noarch 2:19.0.0-1965.g7d63482a.el9 ceph-noarch 1.8 M 2024-03-08T20:44:12.109 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-1965.g7d63482a.el9 ceph-noarch 7.4 M 2024-03-08T20:44:12.109 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-rook noarch 2:19.0.0-1965.g7d63482a.el9 ceph-noarch 49 k 2024-03-08T20:44:12.109 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-radosgw x86_64 2:19.0.0-1965.g7d63482a.el9 ceph 9.8 M 2024-03-08T20:44:12.109 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-test x86_64 2:19.0.0-1965.g7d63482a.el9 ceph 47 M 2024-03-08T20:44:12.110 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-volume noarch 2:19.0.0-1965.g7d63482a.el9 ceph-noarch 264 k 2024-03-08T20:44:12.110 INFO:teuthology.orchestra.run.smithi107.stdout: cephadm noarch 2:19.0.0-1965.g7d63482a.el9 ceph-noarch 748 k 2024-03-08T20:44:12.110 INFO:teuthology.orchestra.run.smithi107.stdout: cephfs-mirror x86_64 2:19.0.0-1965.g7d63482a.el9 ceph 220 k 2024-03-08T20:44:12.110 INFO:teuthology.orchestra.run.smithi107.stdout: cephfs-shell x86_64 2:19.0.0-1965.g7d63482a.el9 ceph 27 k 2024-03-08T20:44:12.110 INFO:teuthology.orchestra.run.smithi107.stdout: cephfs-top noarch 2:19.0.0-1965.g7d63482a.el9 ceph-noarch 18 k 2024-03-08T20:44:12.110 INFO:teuthology.orchestra.run.smithi107.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-03-08T20:44:12.110 INFO:teuthology.orchestra.run.smithi107.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-03-08T20:44:12.110 INFO:teuthology.orchestra.run.smithi107.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-03-08T20:44:12.110 INFO:teuthology.orchestra.run.smithi107.stdout: libcephfs-devel x86_64 2:19.0.0-1965.g7d63482a.el9 ceph 31 k 2024-03-08T20:44:12.110 INFO:teuthology.orchestra.run.smithi107.stdout: libcephfs2 x86_64 2:19.0.0-1965.g7d63482a.el9 ceph 745 k 2024-03-08T20:44:12.110 INFO:teuthology.orchestra.run.smithi107.stdout: librados-devel x86_64 2:19.0.0-1965.g7d63482a.el9 ceph 127 k 2024-03-08T20:44:12.110 INFO:teuthology.orchestra.run.smithi107.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-03-08T20:44:12.111 INFO:teuthology.orchestra.run.smithi107.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-03-08T20:44:12.111 INFO:teuthology.orchestra.run.smithi107.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-03-08T20:44:12.111 INFO:teuthology.orchestra.run.smithi107.stdout: postgresql x86_64 13.11-1.el9 CentOS-AppStream 1.5 M 2024-03-08T20:44:12.111 INFO:teuthology.orchestra.run.smithi107.stdout: postgresql-contrib x86_64 13.11-1.el9 CentOS-AppStream 811 k 2024-03-08T20:44:12.111 INFO:teuthology.orchestra.run.smithi107.stdout: postgresql-server x86_64 13.11-1.el9 CentOS-AppStream 5.7 M 2024-03-08T20:44:12.111 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cephfs x86_64 2:19.0.0-1965.g7d63482a.el9 ceph 162 k 2024-03-08T20:44:12.111 INFO:teuthology.orchestra.run.smithi107.stdout: python3-rados x86_64 2:19.0.0-1965.g7d63482a.el9 ceph 320 k 2024-03-08T20:44:12.111 INFO:teuthology.orchestra.run.smithi107.stdout: python3-rbd x86_64 2:19.0.0-1965.g7d63482a.el9 ceph 298 k 2024-03-08T20:44:12.111 INFO:teuthology.orchestra.run.smithi107.stdout: python3-rgw x86_64 2:19.0.0-1965.g7d63482a.el9 ceph 99 k 2024-03-08T20:44:12.111 INFO:teuthology.orchestra.run.smithi107.stdout: rbd-fuse x86_64 2:19.0.0-1965.g7d63482a.el9 ceph 85 k 2024-03-08T20:44:12.111 INFO:teuthology.orchestra.run.smithi107.stdout: rbd-mirror x86_64 2:19.0.0-1965.g7d63482a.el9 ceph 3.1 M 2024-03-08T20:44:12.111 INFO:teuthology.orchestra.run.smithi107.stdout: rbd-nbd x86_64 2:19.0.0-1965.g7d63482a.el9 ceph 171 k 2024-03-08T20:44:12.112 INFO:teuthology.orchestra.run.smithi107.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-03-08T20:44:12.112 INFO:teuthology.orchestra.run.smithi107.stdout:Upgrading: 2024-03-08T20:44:12.112 INFO:teuthology.orchestra.run.smithi107.stdout: NetworkManager x86_64 1:1.46.0-1.el9 baseos 2.3 M 2024-03-08T20:44:12.112 INFO:teuthology.orchestra.run.smithi107.stdout: NetworkManager-libnm x86_64 1:1.46.0-1.el9 baseos 1.8 M 2024-03-08T20:44:12.112 INFO:teuthology.orchestra.run.smithi107.stdout: NetworkManager-team x86_64 1:1.46.0-1.el9 baseos 42 k 2024-03-08T20:44:12.112 INFO:teuthology.orchestra.run.smithi107.stdout: NetworkManager-tui x86_64 1:1.46.0-1.el9 baseos 248 k 2024-03-08T20:44:12.112 INFO:teuthology.orchestra.run.smithi107.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-03-08T20:44:12.112 INFO:teuthology.orchestra.run.smithi107.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-03-08T20:44:12.112 INFO:teuthology.orchestra.run.smithi107.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-03-08T20:44:12.112 INFO:teuthology.orchestra.run.smithi107.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-03-08T20:44:12.112 INFO:teuthology.orchestra.run.smithi107.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-03-08T20:44:12.112 INFO:teuthology.orchestra.run.smithi107.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-03-08T20:44:12.112 INFO:teuthology.orchestra.run.smithi107.stdout: libblkid x86_64 2.37.4-18.el9 baseos 108 k 2024-03-08T20:44:12.112 INFO:teuthology.orchestra.run.smithi107.stdout: libfdisk x86_64 2.37.4-18.el9 baseos 155 k 2024-03-08T20:44:12.113 INFO:teuthology.orchestra.run.smithi107.stdout: libmount x86_64 2.37.4-18.el9 baseos 136 k 2024-03-08T20:44:12.113 INFO:teuthology.orchestra.run.smithi107.stdout: librados2 x86_64 2:19.0.0-1965.g7d63482a.el9 ceph 3.4 M 2024-03-08T20:44:12.113 INFO:teuthology.orchestra.run.smithi107.stdout: librbd1 x86_64 2:19.0.0-1965.g7d63482a.el9 ceph 3.1 M 2024-03-08T20:44:12.113 INFO:teuthology.orchestra.run.smithi107.stdout: libsmartcols x86_64 2.37.4-18.el9 baseos 64 k 2024-03-08T20:44:12.113 INFO:teuthology.orchestra.run.smithi107.stdout: libuuid x86_64 2.37.4-18.el9 baseos 29 k 2024-03-08T20:44:12.113 INFO:teuthology.orchestra.run.smithi107.stdout: openssl x86_64 1:3.0.7-27.el9 baseos 1.2 M 2024-03-08T20:44:12.113 INFO:teuthology.orchestra.run.smithi107.stdout: openssl-libs x86_64 1:3.0.7-27.el9 baseos 2.2 M 2024-03-08T20:44:12.113 INFO:teuthology.orchestra.run.smithi107.stdout: util-linux x86_64 2.37.4-18.el9 baseos 2.3 M 2024-03-08T20:44:12.113 INFO:teuthology.orchestra.run.smithi107.stdout: util-linux-core x86_64 2.37.4-18.el9 baseos 465 k 2024-03-08T20:44:12.113 INFO:teuthology.orchestra.run.smithi107.stdout: util-linux-user x86_64 2.37.4-18.el9 baseos 31 k 2024-03-08T20:44:12.113 INFO:teuthology.orchestra.run.smithi107.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-03-08T20:44:12.113 INFO:teuthology.orchestra.run.smithi107.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-03-08T20:44:12.113 INFO:teuthology.orchestra.run.smithi107.stdout:Installing dependencies: 2024-03-08T20:44:12.114 INFO:teuthology.orchestra.run.smithi107.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-03-08T20:44:12.114 INFO:teuthology.orchestra.run.smithi107.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-03-08T20:44:12.114 INFO:teuthology.orchestra.run.smithi107.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-03-08T20:44:12.114 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-common x86_64 2:19.0.0-1965.g7d63482a.el9 ceph 20 M 2024-03-08T20:44:12.114 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-grafana-dashboards noarch 2:19.0.0-1965.g7d63482a.el9 ceph-noarch 34 k 2024-03-08T20:44:12.114 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mds x86_64 2:19.0.0-1965.g7d63482a.el9 ceph 2.1 M 2024-03-08T20:44:12.114 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-modules-core noarch 2:19.0.0-1965.g7d63482a.el9 ceph-noarch 248 k 2024-03-08T20:44:12.114 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mon x86_64 2:19.0.0-1965.g7d63482a.el9 ceph 4.7 M 2024-03-08T20:44:12.114 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-osd x86_64 2:19.0.0-1965.g7d63482a.el9 ceph 17 M 2024-03-08T20:44:12.114 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-prometheus-alerts noarch 2:19.0.0-1965.g7d63482a.el9 ceph-noarch 16 k 2024-03-08T20:44:12.114 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-selinux x86_64 2:19.0.0-1965.g7d63482a.el9 ceph 25 k 2024-03-08T20:44:12.114 INFO:teuthology.orchestra.run.smithi107.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-03-08T20:44:12.115 INFO:teuthology.orchestra.run.smithi107.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-03-08T20:44:12.115 INFO:teuthology.orchestra.run.smithi107.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-03-08T20:44:12.115 INFO:teuthology.orchestra.run.smithi107.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-03-08T20:44:12.115 INFO:teuthology.orchestra.run.smithi107.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-03-08T20:44:12.115 INFO:teuthology.orchestra.run.smithi107.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-03-08T20:44:12.115 INFO:teuthology.orchestra.run.smithi107.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-03-08T20:44:12.115 INFO:teuthology.orchestra.run.smithi107.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-03-08T20:44:12.115 INFO:teuthology.orchestra.run.smithi107.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-03-08T20:44:12.115 INFO:teuthology.orchestra.run.smithi107.stdout: libcephsqlite x86_64 2:19.0.0-1965.g7d63482a.el9 ceph 163 k 2024-03-08T20:44:12.115 INFO:teuthology.orchestra.run.smithi107.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-03-08T20:44:12.115 INFO:teuthology.orchestra.run.smithi107.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-03-08T20:44:12.115 INFO:teuthology.orchestra.run.smithi107.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-03-08T20:44:12.115 INFO:teuthology.orchestra.run.smithi107.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-03-08T20:44:12.115 INFO:teuthology.orchestra.run.smithi107.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-03-08T20:44:12.116 INFO:teuthology.orchestra.run.smithi107.stdout: libradosstriper1 x86_64 2:19.0.0-1965.g7d63482a.el9 ceph 501 k 2024-03-08T20:44:12.116 INFO:teuthology.orchestra.run.smithi107.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-03-08T20:44:12.116 INFO:teuthology.orchestra.run.smithi107.stdout: librgw2 x86_64 2:19.0.0-1965.g7d63482a.el9 ceph 5.1 M 2024-03-08T20:44:12.116 INFO:teuthology.orchestra.run.smithi107.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-03-08T20:44:12.116 INFO:teuthology.orchestra.run.smithi107.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-03-08T20:44:12.116 INFO:teuthology.orchestra.run.smithi107.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-03-08T20:44:12.116 INFO:teuthology.orchestra.run.smithi107.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-03-08T20:44:12.116 INFO:teuthology.orchestra.run.smithi107.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-03-08T20:44:12.116 INFO:teuthology.orchestra.run.smithi107.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-03-08T20:44:12.116 INFO:teuthology.orchestra.run.smithi107.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-03-08T20:44:12.116 INFO:teuthology.orchestra.run.smithi107.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-03-08T20:44:12.116 INFO:teuthology.orchestra.run.smithi107.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-03-08T20:44:12.116 INFO:teuthology.orchestra.run.smithi107.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-03-08T20:44:12.116 INFO:teuthology.orchestra.run.smithi107.stdout: postgresql-private-libs x86_64 13.11-1.el9 CentOS-AppStream 134 k 2024-03-08T20:44:12.117 INFO:teuthology.orchestra.run.smithi107.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-03-08T20:44:12.117 INFO:teuthology.orchestra.run.smithi107.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-03-08T20:44:12.117 INFO:teuthology.orchestra.run.smithi107.stdout: python3-asyncssh noarch 2.13.2-4.el9 epel 548 k 2024-03-08T20:44:12.117 INFO:teuthology.orchestra.run.smithi107.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-03-08T20:44:12.117 INFO:teuthology.orchestra.run.smithi107.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-03-08T20:44:12.117 INFO:teuthology.orchestra.run.smithi107.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-03-08T20:44:12.117 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-03-08T20:44:12.117 INFO:teuthology.orchestra.run.smithi107.stdout: python3-ceph-argparse x86_64 2:19.0.0-1965.g7d63482a.el9 ceph 45 k 2024-03-08T20:44:12.117 INFO:teuthology.orchestra.run.smithi107.stdout: python3-ceph-common x86_64 2:19.0.0-1965.g7d63482a.el9 ceph 131 k 2024-03-08T20:44:12.117 INFO:teuthology.orchestra.run.smithi107.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-03-08T20:44:12.117 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-03-08T20:44:12.117 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-03-08T20:44:12.118 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-03-08T20:44:12.118 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-03-08T20:44:12.118 INFO:teuthology.orchestra.run.smithi107.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-03-08T20:44:12.118 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-03-08T20:44:12.118 INFO:teuthology.orchestra.run.smithi107.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-03-08T20:44:12.118 INFO:teuthology.orchestra.run.smithi107.stdout: python3-google-auth noarch 1:2.28.0-1.el9 epel 203 k 2024-03-08T20:44:12.118 INFO:teuthology.orchestra.run.smithi107.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-03-08T20:44:12.118 INFO:teuthology.orchestra.run.smithi107.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-03-08T20:44:12.118 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-03-08T20:44:12.119 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-03-08T20:44:12.119 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-03-08T20:44:12.119 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-03-08T20:44:12.119 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-03-08T20:44:12.119 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-03-08T20:44:12.119 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-03-08T20:44:12.119 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-03-08T20:44:12.119 INFO:teuthology.orchestra.run.smithi107.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-03-08T20:44:12.119 INFO:teuthology.orchestra.run.smithi107.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-03-08T20:44:12.119 INFO:teuthology.orchestra.run.smithi107.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-03-08T20:44:12.119 INFO:teuthology.orchestra.run.smithi107.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-03-08T20:44:12.119 INFO:teuthology.orchestra.run.smithi107.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-03-08T20:44:12.119 INFO:teuthology.orchestra.run.smithi107.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-03-08T20:44:12.119 INFO:teuthology.orchestra.run.smithi107.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-03-08T20:44:12.120 INFO:teuthology.orchestra.run.smithi107.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-03-08T20:44:12.120 INFO:teuthology.orchestra.run.smithi107.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-03-08T20:44:12.120 INFO:teuthology.orchestra.run.smithi107.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-03-08T20:44:12.120 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-03-08T20:44:12.120 INFO:teuthology.orchestra.run.smithi107.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-03-08T20:44:12.120 INFO:teuthology.orchestra.run.smithi107.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-03-08T20:44:12.120 INFO:teuthology.orchestra.run.smithi107.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-03-08T20:44:12.120 INFO:teuthology.orchestra.run.smithi107.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-03-08T20:44:12.120 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-03-08T20:44:12.120 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-03-08T20:44:12.120 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-03-08T20:44:12.120 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-03-08T20:44:12.120 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-03-08T20:44:12.121 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-03-08T20:44:12.121 INFO:teuthology.orchestra.run.smithi107.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-03-08T20:44:12.121 INFO:teuthology.orchestra.run.smithi107.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-03-08T20:44:12.121 INFO:teuthology.orchestra.run.smithi107.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-03-08T20:44:12.121 INFO:teuthology.orchestra.run.smithi107.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-03-08T20:44:12.121 INFO:teuthology.orchestra.run.smithi107.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-03-08T20:44:12.121 INFO:teuthology.orchestra.run.smithi107.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-03-08T20:44:12.121 INFO:teuthology.orchestra.run.smithi107.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-03-08T20:44:12.121 INFO:teuthology.orchestra.run.smithi107.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-03-08T20:44:12.121 INFO:teuthology.orchestra.run.smithi107.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-03-08T20:44:12.121 INFO:teuthology.orchestra.run.smithi107.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-03-08T20:44:12.121 INFO:teuthology.orchestra.run.smithi107.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-03-08T20:44:12.121 INFO:teuthology.orchestra.run.smithi107.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-03-08T20:44:12.121 INFO:teuthology.orchestra.run.smithi107.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-03-08T20:44:12.122 INFO:teuthology.orchestra.run.smithi107.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-03-08T20:44:12.122 INFO:teuthology.orchestra.run.smithi107.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-03-08T20:44:12.122 INFO:teuthology.orchestra.run.smithi107.stdout: qatlib x86_64 23.11.0-1.el9 appstream 216 k 2024-03-08T20:44:12.122 INFO:teuthology.orchestra.run.smithi107.stdout: qatzip-libs x86_64 1.1.2-2.el9 appstream 44 k 2024-03-08T20:44:12.122 INFO:teuthology.orchestra.run.smithi107.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-03-08T20:44:12.122 INFO:teuthology.orchestra.run.smithi107.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-03-08T20:44:12.122 INFO:teuthology.orchestra.run.smithi107.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-03-08T20:44:12.122 INFO:teuthology.orchestra.run.smithi107.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-03-08T20:44:12.122 INFO:teuthology.orchestra.run.smithi107.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-03-08T20:44:12.122 INFO:teuthology.orchestra.run.smithi107.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-03-08T20:44:12.122 INFO:teuthology.orchestra.run.smithi107.stdout:Installing weak dependencies: 2024-03-08T20:44:12.122 INFO:teuthology.orchestra.run.smithi107.stdout: qatlib-service x86_64 23.11.0-1.el9 appstream 35 k 2024-03-08T20:44:12.122 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-08T20:44:12.123 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction Summary 2024-03-08T20:44:12.123 INFO:teuthology.orchestra.run.smithi107.stdout:============================================================================================ 2024-03-08T20:44:12.123 INFO:teuthology.orchestra.run.smithi107.stdout:Install 150 Packages 2024-03-08T20:44:12.123 INFO:teuthology.orchestra.run.smithi107.stdout:Upgrade 24 Packages 2024-03-08T20:44:12.123 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-08T20:44:12.123 INFO:teuthology.orchestra.run.smithi107.stdout:Total download size: 223 M 2024-03-08T20:44:12.123 INFO:teuthology.orchestra.run.smithi107.stdout:Downloading Packages: 2024-03-08T20:44:12.136 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 71/198 2024-03-08T20:44:12.218 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 72/198 2024-03-08T20:44:12.362 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 73/198 2024-03-08T20:44:12.401 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-google-auth-1:2.28.0-1.el9.noarch 74/198 2024-03-08T20:44:12.447 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 75/198 2024-03-08T20:44:12.483 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 76/198 2024-03-08T20:44:12.516 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 77/198 2024-03-08T20:44:12.551 INFO:teuthology.orchestra.run.smithi204.stdout:(152/174): openssl-devel-3.0.7-27.el9.x86_64.rp 345 kB/s | 4.1 MB 00:12 2024-03-08T20:44:12.693 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 78/198 2024-03-08T20:44:12.875 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 79/198 2024-03-08T20:44:13.012 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 80/198 2024-03-08T20:44:13.068 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 81/198 2024-03-08T20:44:13.147 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-ply-3.11-14.el9.noarch 82/198 2024-03-08T20:44:13.223 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 83/198 2024-03-08T20:44:13.470 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 84/198 2024-03-08T20:44:13.532 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 85/198 2024-03-08T20:44:13.722 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 86/198 2024-03-08T20:44:13.770 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-asyncssh-2.13.2-4.el9.noarch 87/198 2024-03-08T20:44:13.810 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 88/198 2024-03-08T20:44:13.843 INFO:teuthology.orchestra.run.smithi107.stdout:(1/174): ceph-19.0.0-1965.g7d63482a.el9.x86_64. 17 kB/s | 6.6 kB 00:00 2024-03-08T20:44:13.848 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : fuse-2.9.9-15.el9.x86_64 89/198 2024-03-08T20:44:13.889 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 90/198 2024-03-08T20:44:13.934 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 91/198 2024-03-08T20:44:14.018 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : uuid-1.6.2-55.el9.x86_64 92/198 2024-03-08T20:44:14.071 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 93/198 2024-03-08T20:44:14.111 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 94/198 2024-03-08T20:44:14.235 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 95/198 2024-03-08T20:44:14.282 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 96/198 2024-03-08T20:44:14.321 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 97/198 2024-03-08T20:44:14.385 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 98/198 2024-03-08T20:44:14.432 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 99/198 2024-03-08T20:44:14.577 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 100/198 2024-03-08T20:44:14.692 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 101/198 2024-03-08T20:44:14.727 INFO:teuthology.orchestra.run.smithi107.stdout:(2/174): ceph-fuse-19.0.0-1965.g7d63482a.el9.x8 1.0 MB/s | 889 kB 00:00 2024-03-08T20:44:14.753 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 102/198 2024-03-08T20:44:14.788 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 103/198 2024-03-08T20:44:14.845 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 104/198 2024-03-08T20:44:14.918 INFO:teuthology.orchestra.run.smithi107.stdout:(3/174): ceph-immutable-object-cache-19.0.0-196 764 kB/s | 145 kB 00:00 2024-03-08T20:44:15.001 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-packaging-20.9-5.el9.noarch 105/198 2024-03-08T20:44:15.071 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 106/198 2024-03-08T20:44:15.794 INFO:teuthology.orchestra.run.smithi107.stdout:(4/174): ceph-mds-19.0.0-1965.g7d63482a.el9.x86 2.4 MB/s | 2.1 MB 00:00 2024-03-08T20:44:16.037 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 107/198 2024-03-08T20:44:16.095 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 108/198 2024-03-08T20:44:16.129 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 109/198 2024-03-08T20:44:16.186 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 110/198 2024-03-08T20:44:16.297 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 111/198 2024-03-08T20:44:16.318 INFO:teuthology.orchestra.run.smithi204.stdout:(153/174): NetworkManager-1.46.0-1.el9.x86_64.r 473 kB/s | 2.3 MB 00:04 2024-03-08T20:44:16.347 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 112/198 2024-03-08T20:44:16.352 INFO:teuthology.orchestra.run.smithi107.stdout:(5/174): ceph-mgr-19.0.0-1965.g7d63482a.el9.x86 2.7 MB/s | 1.5 MB 00:00 2024-03-08T20:44:16.560 INFO:teuthology.orchestra.run.smithi107.stdout:(6/174): ceph-base-19.0.0-1965.g7d63482a.el9.x8 1.7 MB/s | 5.4 MB 00:03 2024-03-08T20:44:16.581 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : openblas-0.3.21-2.el9.x86_64 113/198 2024-03-08T20:44:16.610 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 114/198 2024-03-08T20:44:16.708 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 115/198 2024-03-08T20:44:16.709 INFO:teuthology.orchestra.run.smithi204.stdout:(154/174): NetworkManager-libnm-1.46.0-1.el9.x8 448 kB/s | 1.8 MB 00:04 2024-03-08T20:44:17.226 INFO:teuthology.orchestra.run.smithi204.stdout:(155/174): NetworkManager-team-1.46.0-1.el9.x86 47 kB/s | 42 kB 00:00 2024-03-08T20:44:17.497 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 116/198 2024-03-08T20:44:17.536 INFO:teuthology.orchestra.run.smithi107.stdout:(7/174): ceph-common-19.0.0-1965.g7d63482a.el9. 4.9 MB/s | 20 MB 00:04 2024-03-08T20:44:17.654 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 117/198 2024-03-08T20:44:18.001 INFO:teuthology.orchestra.run.smithi204.stdout:(156/174): NetworkManager-tui-1.46.0-1.el9.x86_ 192 kB/s | 248 kB 00:01 2024-03-08T20:44:18.210 INFO:teuthology.orchestra.run.smithi204.stdout:(157/174): acl-2.3.1-4.el9.x86_64.rpm 73 kB/s | 71 kB 00:00 2024-03-08T20:44:18.336 INFO:teuthology.orchestra.run.smithi107.stdout:(8/174): ceph-mon-19.0.0-1965.g7d63482a.el9.x86 2.4 MB/s | 4.7 MB 00:01 2024-03-08T20:44:18.444 INFO:teuthology.orchestra.run.smithi107.stdout:(9/174): ceph-selinux-19.0.0-1965.g7d63482a.el9 234 kB/s | 25 kB 00:00 2024-03-08T20:44:18.902 INFO:teuthology.orchestra.run.smithi204.stdout:(158/174): elfutils-debuginfod-client-0.190-2.e 41 kB/s | 36 kB 00:00 2024-03-08T20:44:19.195 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 118/198 2024-03-08T20:44:19.298 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 119/198 2024-03-08T20:44:19.327 INFO:teuthology.orchestra.run.smithi204.stdout:(159/174): elfutils-libelf-0.190-2.el9.x86_64.r 172 kB/s | 192 kB 00:01 2024-03-08T20:44:19.367 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 120/198 2024-03-08T20:44:19.409 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : jq-1.6-16.el9.x86_64 121/198 2024-03-08T20:44:19.452 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 122/198 2024-03-08T20:44:19.831 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 123/198 2024-03-08T20:44:19.855 INFO:teuthology.orchestra.run.smithi149.stdout: Upgrading : librbd1-2:19.0.0-1965.g7d63482a.el9.x86_64 124/198 2024-03-08T20:44:19.942 INFO:teuthology.orchestra.run.smithi107.stdout:(10/174): ceph-radosgw-19.0.0-1965.g7d63482a.el 4.1 MB/s | 9.8 MB 00:02 2024-03-08T20:44:19.975 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: librbd1-2:19.0.0-1965.g7d63482a.el9.x86_64 124/198 2024-03-08T20:44:20.003 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-rbd-2:19.0.0-1965.g7d63482a.el9.x86_64 125/198 2024-03-08T20:44:20.036 INFO:teuthology.orchestra.run.smithi107.stdout:(11/174): cephfs-mirror-19.0.0-1965.g7d63482a.e 2.3 MB/s | 220 kB 00:00 2024-03-08T20:44:20.057 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 126/198 2024-03-08T20:44:20.094 INFO:teuthology.orchestra.run.smithi204.stdout:(160/174): elfutils-libs-0.190-2.el9.x86_64.rpm 215 kB/s | 256 kB 00:01 2024-03-08T20:44:20.111 INFO:teuthology.orchestra.run.smithi107.stdout:(12/174): cephfs-shell-19.0.0-1965.g7d63482a.el 363 kB/s | 27 kB 00:00 2024-03-08T20:44:20.119 INFO:teuthology.orchestra.run.smithi204.stdout:(161/174): libacl-2.3.1-4.el9.x86_64.rpm 29 kB/s | 23 kB 00:00 2024-03-08T20:44:20.187 INFO:teuthology.orchestra.run.smithi107.stdout:(13/174): libcephfs-devel-19.0.0-1965.g7d63482a 421 kB/s | 31 kB 00:00 2024-03-08T20:44:20.470 INFO:teuthology.orchestra.run.smithi107.stdout:(14/174): libcephfs2-19.0.0-1965.g7d63482a.el9. 2.6 MB/s | 745 kB 00:00 2024-03-08T20:44:20.545 INFO:teuthology.orchestra.run.smithi107.stdout:(15/174): libcephsqlite-19.0.0-1965.g7d63482a.e 2.2 MB/s | 163 kB 00:00 2024-03-08T20:44:20.560 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 127/198 2024-03-08T20:44:20.592 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : librgw2-2:19.0.0-1965.g7d63482a.el9.x86_64 128/198 2024-03-08T20:44:20.629 INFO:teuthology.orchestra.run.smithi107.stdout:(16/174): librados-devel-19.0.0-1965.g7d63482a. 1.5 MB/s | 127 kB 00:00 2024-03-08T20:44:20.683 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: librgw2-2:19.0.0-1965.g7d63482a.el9.x86_64 128/198 2024-03-08T20:44:20.711 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-rgw-2:19.0.0-1965.g7d63482a.el9.x86_64 129/198 2024-03-08T20:44:20.812 INFO:teuthology.orchestra.run.smithi107.stdout:(17/174): libradosstriper1-19.0.0-1965.g7d63482 2.7 MB/s | 501 kB 00:00 2024-03-08T20:44:21.120 INFO:teuthology.orchestra.run.smithi204.stdout:(162/174): libblkid-2.37.4-18.el9.x86_64.rpm 106 kB/s | 108 kB 00:01 2024-03-08T20:44:21.428 INFO:teuthology.orchestra.run.smithi204.stdout:(163/174): libfdisk-2.37.4-18.el9.x86_64.rpm 118 kB/s | 155 kB 00:01 2024-03-08T20:44:22.154 INFO:teuthology.orchestra.run.smithi107.stdout:(18/174): librgw2-19.0.0-1965.g7d63482a.el9.x86 3.8 MB/s | 5.1 MB 00:01 2024-03-08T20:44:22.238 INFO:teuthology.orchestra.run.smithi107.stdout:(19/174): python3-ceph-argparse-19.0.0-1965.g7d 548 kB/s | 45 kB 00:00 2024-03-08T20:44:22.330 INFO:teuthology.orchestra.run.smithi107.stdout:(20/174): python3-ceph-common-19.0.0-1965.g7d63 1.4 MB/s | 131 kB 00:00 2024-03-08T20:44:22.413 INFO:teuthology.orchestra.run.smithi107.stdout:(21/174): python3-cephfs-19.0.0-1965.g7d63482a. 1.9 MB/s | 162 kB 00:00 2024-03-08T20:44:22.564 INFO:teuthology.orchestra.run.smithi107.stdout:(22/174): python3-rados-19.0.0-1965.g7d63482a.e 2.1 MB/s | 320 kB 00:00 2024-03-08T20:44:22.645 INFO:teuthology.orchestra.run.smithi204.stdout:(164/174): ceph-test-19.0.0-1965.g7d63482a.el9. 1.5 MB/s | 47 MB 00:32 2024-03-08T20:44:22.670 INFO:teuthology.orchestra.run.smithi204.stdout:(165/174): libsmartcols-2.37.4-18.el9.x86_64.rp 51 kB/s | 64 kB 00:01 2024-03-08T20:44:22.695 INFO:teuthology.orchestra.run.smithi204.stdout:(166/174): libmount-2.37.4-18.el9.x86_64.rpm 86 kB/s | 136 kB 00:01 2024-03-08T20:44:22.714 INFO:teuthology.orchestra.run.smithi107.stdout:(23/174): python3-rbd-19.0.0-1965.g7d63482a.el9 2.0 MB/s | 298 kB 00:00 2024-03-08T20:44:22.797 INFO:teuthology.orchestra.run.smithi107.stdout:(24/174): python3-rgw-19.0.0-1965.g7d63482a.el9 1.2 MB/s | 99 kB 00:00 2024-03-08T20:44:22.881 INFO:teuthology.orchestra.run.smithi107.stdout:(25/174): rbd-fuse-19.0.0-1965.g7d63482a.el9.x8 1.0 MB/s | 85 kB 00:00 2024-03-08T20:44:23.232 INFO:teuthology.orchestra.run.smithi107.stdout:(26/174): ceph-osd-19.0.0-1965.g7d63482a.el9.x8 2.5 MB/s | 17 MB 00:06 2024-03-08T20:44:23.390 INFO:teuthology.orchestra.run.smithi107.stdout:(27/174): rbd-nbd-19.0.0-1965.g7d63482a.el9.x86 1.1 MB/s | 171 kB 00:00 2024-03-08T20:44:23.456 INFO:teuthology.orchestra.run.smithi204.stdout:(167/174): libuuid-2.37.4-18.el9.x86_64.rpm 35 kB/s | 29 kB 00:00 2024-03-08T20:44:23.474 INFO:teuthology.orchestra.run.smithi107.stdout:(28/174): ceph-grafana-dashboards-19.0.0-1965.g 411 kB/s | 34 kB 00:00 2024-03-08T20:44:23.492 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-common-2:19.0.0-1965.g7d63482a.el9.x86_64 130/198 2024-03-08T20:44:23.533 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-common-2:19.0.0-1965.g7d63482a.el9.x86_64 130/198 2024-03-08T20:44:23.574 INFO:teuthology.orchestra.run.smithi107.stdout:(29/174): ceph-mgr-cephadm-19.0.0-1965.g7d63482 1.4 MB/s | 139 kB 00:00 2024-03-08T20:44:23.824 INFO:teuthology.orchestra.run.smithi107.stdout:(30/174): rbd-mirror-19.0.0-1965.g7d63482a.el9. 3.3 MB/s | 3.1 MB 00:00 2024-03-08T20:44:24.155 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-common-2:19.0.0-1965.g7d63482a.el9.x86_64 130/198 2024-03-08T20:44:24.202 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-base-2:19.0.0-1965.g7d63482a.el9.x86_64 131/198 2024-03-08T20:44:24.291 INFO:teuthology.orchestra.run.smithi107.stdout:(31/174): ceph-mgr-dashboard-19.0.0-1965.g7d634 2.5 MB/s | 1.8 MB 00:00 2024-03-08T20:44:24.350 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-base-2:19.0.0-1965.g7d63482a.el9.x86_64 131/198 2024-03-08T20:44:24.350 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-03-08T20:44:24.350 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-03-08T20:44:24.350 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-03-08T20:44:24.395 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-selinux-2:19.0.0-1965.g7d63482a.el9.x86_64 132/198 2024-03-08T20:44:24.458 INFO:teuthology.orchestra.run.smithi107.stdout:(32/174): ceph-mgr-modules-core-19.0.0-1965.g7d 1.5 MB/s | 248 kB 00:00 2024-03-08T20:44:24.558 INFO:teuthology.orchestra.run.smithi107.stdout:(33/174): ceph-mgr-rook-19.0.0-1965.g7d63482a.e 497 kB/s | 49 kB 00:00 2024-03-08T20:44:24.658 INFO:teuthology.orchestra.run.smithi107.stdout:(34/174): ceph-prometheus-alerts-19.0.0-1965.g7 166 kB/s | 16 kB 00:00 2024-03-08T20:44:24.817 INFO:teuthology.orchestra.run.smithi107.stdout:(35/174): ceph-volume-19.0.0-1965.g7d63482a.el9 1.6 MB/s | 264 kB 00:00 2024-03-08T20:44:25.117 INFO:teuthology.orchestra.run.smithi107.stdout:(36/174): cephadm-19.0.0-1965.g7d63482a.el9.noa 2.4 MB/s | 748 kB 00:00 2024-03-08T20:44:25.209 INFO:teuthology.orchestra.run.smithi107.stdout:(37/174): cephfs-top-19.0.0-1965.g7d63482a.el9. 201 kB/s | 18 kB 00:00 2024-03-08T20:44:25.626 INFO:teuthology.orchestra.run.smithi107.stdout:(38/174): automake-1.16.2-8.el9.noarch.rpm 1.6 MB/s | 662 kB 00:00 2024-03-08T20:44:25.647 INFO:teuthology.orchestra.run.smithi204.stdout:(168/174): openssl-3.0.7-27.el9.x86_64.rpm 416 kB/s | 1.2 MB 00:02 2024-03-08T20:44:25.768 INFO:teuthology.orchestra.run.smithi107.stdout:(39/174): bison-3.7.4-5.el9.x86_64.rpm 6.4 MB/s | 920 kB 00:00 2024-03-08T20:44:25.910 INFO:teuthology.orchestra.run.smithi107.stdout:(40/174): ceph-mgr-diskprediction-local-19.0.0- 3.6 MB/s | 7.4 MB 00:02 2024-03-08T20:44:25.935 INFO:teuthology.orchestra.run.smithi107.stdout:(41/174): boost-program-options-1.75.0-8.el9.x8 636 kB/s | 106 kB 00:00 2024-03-08T20:44:25.985 INFO:teuthology.orchestra.run.smithi107.stdout:(42/174): flex-2.6.4-9.el9.x86_64.rpm 4.1 MB/s | 307 kB 00:00 2024-03-08T20:44:26.011 INFO:teuthology.orchestra.run.smithi107.stdout:(43/174): flexiblas-3.0.4-8.el9.x86_64.rpm 424 kB/s | 32 kB 00:00 2024-03-08T20:44:26.102 INFO:teuthology.orchestra.run.smithi107.stdout:(44/174): flexiblas-openblas-openmp-3.0.4-8.el9 188 kB/s | 17 kB 00:00 2024-03-08T20:44:26.261 INFO:teuthology.orchestra.run.smithi107.stdout:(45/174): flexiblas-netlib-3.0.4-8.el9.x86_64.r 11 MB/s | 3.0 MB 00:00 2024-03-08T20:44:26.278 INFO:teuthology.orchestra.run.smithi107.stdout:(46/174): libaio-devel-0.3.111-13.el9.x86_64.rp 60 kB/s | 10 kB 00:00 2024-03-08T20:44:26.353 INFO:teuthology.orchestra.run.smithi107.stdout:(47/174): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.1 MB/s | 159 kB 00:00 2024-03-08T20:44:26.378 INFO:teuthology.orchestra.run.smithi107.stdout:(48/174): librabbitmq-0.11.0-7.el9.x86_64.rpm 446 kB/s | 44 kB 00:00 2024-03-08T20:44:26.437 INFO:teuthology.orchestra.run.smithi107.stdout:(49/174): librdkafka-1.6.1-102.el9.x86_64.rpm 7.8 MB/s | 661 kB 00:00 2024-03-08T20:44:26.470 INFO:teuthology.orchestra.run.smithi107.stdout:(50/174): libtool-2.4.6-45.el9.x86_64.rpm 6.2 MB/s | 578 kB 00:00 2024-03-08T20:44:26.504 INFO:teuthology.orchestra.run.smithi107.stdout:(51/174): libzstd-devel-1.5.1-2.el9.x86_64.rpm 748 kB/s | 46 kB 00:00 2024-03-08T20:44:26.538 INFO:teuthology.orchestra.run.smithi107.stdout:(52/174): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.2 MB/s | 282 kB 00:00 2024-03-08T20:44:26.580 INFO:teuthology.orchestra.run.smithi107.stdout:(53/174): lua-5.4.4-4.el9.x86_64.rpm 2.4 MB/s | 187 kB 00:00 2024-03-08T20:44:26.613 INFO:teuthology.orchestra.run.smithi107.stdout:(54/174): oniguruma-6.9.6-1.el9.5.x86_64.rpm 2.8 MB/s | 217 kB 00:00 2024-03-08T20:44:26.647 INFO:teuthology.orchestra.run.smithi107.stdout:(55/174): openblas-0.3.21-2.el9.x86_64.rpm 530 kB/s | 34 kB 00:00 2024-03-08T20:44:27.006 INFO:teuthology.orchestra.run.smithi107.stdout:(56/174): openblas-openmp-0.3.21-2.el9.x86_64.r 12 MB/s | 4.7 MB 00:00 2024-03-08T20:44:27.064 INFO:teuthology.orchestra.run.smithi107.stdout:(57/174): postgresql-13.11-1.el9.x86_64.rpm 3.7 MB/s | 1.5 MB 00:00 2024-03-08T20:44:27.106 INFO:teuthology.orchestra.run.smithi107.stdout:(58/174): postgresql-contrib-13.11-1.el9.x86_64 8.2 MB/s | 811 kB 00:00 2024-03-08T20:44:27.131 INFO:teuthology.orchestra.run.smithi107.stdout:(59/174): postgresql-private-libs-13.11-1.el9.x 2.2 MB/s | 134 kB 00:00 2024-03-08T20:44:27.357 INFO:teuthology.orchestra.run.smithi107.stdout:(60/174): postgresql-server-13.11-1.el9.x86_64. 23 MB/s | 5.7 MB 00:00 2024-03-08T20:44:27.389 INFO:teuthology.orchestra.run.smithi204.stdout:(169/174): openssl-libs-3.0.7-27.el9.x86_64.rpm 470 kB/s | 2.2 MB 00:04 2024-03-08T20:44:27.398 INFO:teuthology.orchestra.run.smithi107.stdout:(61/174): protobuf-3.14.0-13.el9.x86_64.rpm 3.8 MB/s | 1.0 MB 00:00 2024-03-08T20:44:27.423 INFO:teuthology.orchestra.run.smithi204.stdout:(170/174): util-linux-core-2.37.4-18.el9.x86_64 262 kB/s | 465 kB 00:01 2024-03-08T20:44:27.449 INFO:teuthology.orchestra.run.smithi107.stdout:(62/174): python3-attrs-20.3.0-7.el9.noarch.rpm 1.5 MB/s | 78 kB 00:00 2024-03-08T20:44:27.632 INFO:teuthology.orchestra.run.smithi107.stdout:(63/174): python3-babel-2.9.1-2.el9.noarch.rpm 25 MB/s | 5.8 MB 00:00 2024-03-08T20:44:27.658 INFO:teuthology.orchestra.run.smithi107.stdout:(64/174): python3-jsonpatch-1.21-16.el9.noarch. 126 kB/s | 24 kB 00:00 2024-03-08T20:44:27.700 INFO:teuthology.orchestra.run.smithi107.stdout:(65/174): python3-jsonpointer-2.0-4.el9.noarch. 449 kB/s | 18 kB 00:00 2024-03-08T20:44:27.716 INFO:teuthology.orchestra.run.smithi107.stdout:(66/174): python3-mako-1.1.4-6.el9.noarch.rpm 2.5 MB/s | 144 kB 00:00 2024-03-08T20:44:27.750 INFO:teuthology.orchestra.run.smithi107.stdout:(67/174): python3-markupsafe-1.1.1-12.el9.x86_6 655 kB/s | 32 kB 00:00 2024-03-08T20:44:27.892 INFO:teuthology.orchestra.run.smithi107.stdout:(68/174): python3-numpy-1.20.1-5.el9.x86_64.rpm 27 MB/s | 4.8 MB 00:00 2024-03-08T20:44:27.925 INFO:teuthology.orchestra.run.smithi107.stdout:(69/174): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.3 MB/s | 217 kB 00:00 2024-03-08T20:44:27.951 INFO:teuthology.orchestra.run.smithi107.stdout:(70/174): python3-oauthlib-3.1.1-5.el9.noarch.r 3.1 MB/s | 174 kB 00:00 2024-03-08T20:44:27.976 INFO:teuthology.orchestra.run.smithi107.stdout:(71/174): python3-packaging-20.9-5.el9.noarch.r 1.4 MB/s | 69 kB 00:00 2024-03-08T20:44:28.001 INFO:teuthology.orchestra.run.smithi107.stdout:(72/174): python3-prettytable-0.7.2-27.el9.noar 841 kB/s | 41 kB 00:00 2024-03-08T20:44:28.035 INFO:teuthology.orchestra.run.smithi107.stdout:(73/174): python3-protobuf-3.14.0-13.el9.noarch 4.0 MB/s | 238 kB 00:00 2024-03-08T20:44:28.060 INFO:teuthology.orchestra.run.smithi107.stdout:(74/174): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.3 MB/s | 132 kB 00:00 2024-03-08T20:44:28.085 INFO:teuthology.orchestra.run.smithi107.stdout:(75/174): python3-pyasn1-modules-0.4.8-6.el9.no 4.3 MB/s | 211 kB 00:00 2024-03-08T20:44:28.111 INFO:teuthology.orchestra.run.smithi107.stdout:(76/174): python3-pytz-2021.1-5.el9.noarch.rpm 986 kB/s | 47 kB 00:00 2024-03-08T20:44:28.136 INFO:teuthology.orchestra.run.smithi107.stdout:(77/174): python3-requests-oauthlib-1.3.0-12.el 873 kB/s | 43 kB 00:00 2024-03-08T20:44:28.199 INFO:teuthology.orchestra.run.smithi204.stdout:(171/174): util-linux-user-2.37.4-18.el9.x86_64 38 kB/s | 31 kB 00:00 2024-03-08T20:44:28.291 INFO:teuthology.orchestra.run.smithi204.stdout:(172/174): util-linux-2.37.4-18.el9.x86_64.rpm 484 kB/s | 2.3 MB 00:04 2024-03-08T20:44:28.345 INFO:teuthology.orchestra.run.smithi107.stdout:(78/174): python3-toml-0.10.2-6.el9.noarch.rpm 186 kB/s | 39 kB 00:00 2024-03-08T20:44:28.349 INFO:teuthology.orchestra.run.smithi204.stdout:(173/174): libacl-devel-2.3.1-4.el9.x86_64.rpm 499 kB/s | 74 kB 00:00 2024-03-08T20:44:28.612 INFO:teuthology.orchestra.run.smithi107.stdout:(79/174): python3-scipy-1.6.2-8.el9.x86_64.rpm 31 MB/s | 16 MB 00:00 2024-03-08T20:44:28.637 INFO:teuthology.orchestra.run.smithi107.stdout:(80/174): python3-wcwidth-0.2.5-8.el9.noarch.rp 148 kB/s | 41 kB 00:00 2024-03-08T20:44:28.687 INFO:teuthology.orchestra.run.smithi107.stdout:(81/174): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 56 kB 00:00 2024-03-08T20:44:28.720 INFO:teuthology.orchestra.run.smithi107.stdout:(82/174): socat-1.7.4.1-5.el9.x86_64.rpm 3.6 MB/s | 300 kB 00:00 2024-03-08T20:44:28.746 INFO:teuthology.orchestra.run.smithi107.stdout:(83/174): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.1 MB/s | 63 kB 00:00 2024-03-08T20:44:28.771 INFO:teuthology.orchestra.run.smithi107.stdout:(84/174): zlib-devel-1.2.11-40.el9.x86_64.rpm 902 kB/s | 44 kB 00:00 2024-03-08T20:44:28.796 INFO:teuthology.orchestra.run.smithi107.stdout:(85/174): fuse-2.9.9-15.el9.x86_64.rpm 1.6 MB/s | 79 kB 00:00 2024-03-08T20:44:28.830 INFO:teuthology.orchestra.run.smithi107.stdout:(86/174): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.1 MB/s | 241 kB 00:00 2024-03-08T20:44:28.913 INFO:teuthology.orchestra.run.smithi107.stdout:(87/174): python3-cryptography-36.0.1-4.el9.x86 10 MB/s | 1.1 MB 00:00 2024-03-08T20:44:28.939 INFO:teuthology.orchestra.run.smithi107.stdout:(88/174): python3-ply-3.11-14.el9.noarch.rpm 1.0 MB/s | 103 kB 00:00 2024-03-08T20:44:28.964 INFO:teuthology.orchestra.run.smithi107.stdout:(89/174): python3-pycparser-2.20-6.el9.noarch.r 2.5 MB/s | 124 kB 00:00 2024-03-08T20:44:28.998 INFO:teuthology.orchestra.run.smithi107.stdout:(90/174): python3-urllib3-1.26.5-5.el9.noarch.r 3.2 MB/s | 187 kB 00:00 2024-03-08T20:44:29.182 INFO:teuthology.orchestra.run.smithi107.stdout:(91/174): jq-1.6-16.el9.x86_64.rpm 873 kB/s | 187 kB 00:00 2024-03-08T20:44:29.240 INFO:teuthology.orchestra.run.smithi107.stdout:(92/174): libquadmath-11.4.1-3.el9.x86_64.rpm 3.3 MB/s | 195 kB 00:00 2024-03-08T20:44:29.290 INFO:teuthology.orchestra.run.smithi107.stdout:(93/174): libgfortran-11.4.1-3.el9.x86_64.rpm 2.7 MB/s | 803 kB 00:00 2024-03-08T20:44:29.315 INFO:teuthology.orchestra.run.smithi107.stdout:(94/174): python3-requests-2.25.1-8.el9.noarch. 1.6 MB/s | 125 kB 00:00 2024-03-08T20:44:29.399 INFO:teuthology.orchestra.run.smithi107.stdout:(95/174): elfutils-libelf-devel-0.190-2.el9.x86 200 kB/s | 21 kB 00:00 2024-03-08T20:44:29.425 INFO:teuthology.orchestra.run.smithi107.stdout:(96/174): libuuid-devel-2.37.4-18.el9.x86_64.rp 234 kB/s | 25 kB 00:00 2024-03-08T20:44:29.599 INFO:teuthology.orchestra.run.smithi107.stdout:(97/174): python3-devel-3.9.17-2.el9.x86_64.rpm 1.4 MB/s | 245 kB 00:00 2024-03-08T20:44:29.691 INFO:teuthology.orchestra.run.smithi107.stdout:(98/174): python3-jinja2-2.11.3-5.el9.noarch.rp 2.7 MB/s | 248 kB 00:00 2024-03-08T20:44:29.792 INFO:teuthology.orchestra.run.smithi107.stdout:(99/174): qatlib-23.11.0-1.el9.x86_64.rpm 2.1 MB/s | 216 kB 00:00 2024-03-08T20:44:29.999 INFO:teuthology.orchestra.run.smithi107.stdout:(100/174): qatlib-service-23.11.0-1.el9.x86_64. 169 kB/s | 35 kB 00:00 2024-03-08T20:44:29.999 INFO:teuthology.orchestra.run.smithi204.stdout:(174/174): xfsprogs-6.3.0-1.el9.x86_64.rpm 427 kB/s | 1.1 MB 00:02 2024-03-08T20:44:30.012 INFO:teuthology.orchestra.run.smithi204.stdout:-------------------------------------------------------------------------------- 2024-03-08T20:44:30.012 INFO:teuthology.orchestra.run.smithi204.stdout:Total 5.2 MB/s | 223 MB 00:42 2024-03-08T20:44:30.059 INFO:teuthology.orchestra.run.smithi107.stdout:(101/174): qatzip-libs-1.1.2-2.el9.x86_64.rpm 750 kB/s | 44 kB 00:00 2024-03-08T20:44:30.119 INFO:teuthology.orchestra.run.smithi107.stdout:(102/174): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 1.6 MB/s | 94 kB 00:00 2024-03-08T20:44:30.193 INFO:teuthology.orchestra.run.smithi107.stdout:(103/174): lua-devel-5.4.4-4.el9.x86_64.rpm 300 kB/s | 22 kB 00:00 2024-03-08T20:44:30.410 INFO:teuthology.orchestra.run.smithi107.stdout:(104/174): protobuf-compiler-3.14.0-13.el9.x86_ 3.9 MB/s | 863 kB 00:00 2024-03-08T20:44:30.894 INFO:teuthology.orchestra.run.smithi107.stdout:(105/174): openssl-devel-3.0.7-27.el9.x86_64.rp 2.8 MB/s | 4.1 MB 00:01 2024-03-08T20:44:31.194 INFO:teuthology.orchestra.run.smithi107.stdout:(106/174): abseil-cpp-20211102.0-3.el9.x86_64.r 700 kB/s | 548 kB 00:00 2024-03-08T20:44:31.327 INFO:teuthology.orchestra.run.smithi107.stdout:(107/174): grpc-data-1.46.7-10.el9.noarch.rpm 147 kB/s | 19 kB 00:00 2024-03-08T20:44:31.472 INFO:teuthology.orchestra.run.smithi204.stdout:Running transaction check 2024-03-08T20:44:31.611 INFO:teuthology.orchestra.run.smithi107.stdout:(108/174): gperftools-libs-2.9.1-2.el9.x86_64.r 431 kB/s | 309 kB 00:00 2024-03-08T20:44:31.753 INFO:teuthology.orchestra.run.smithi107.stdout:(109/174): libarrow-doc-9.0.0-10.el9.noarch.rpm 184 kB/s | 26 kB 00:00 2024-03-08T20:44:31.853 INFO:teuthology.orchestra.run.smithi204.stdout:Transaction check succeeded. 2024-03-08T20:44:31.853 INFO:teuthology.orchestra.run.smithi204.stdout:Running transaction test 2024-03-08T20:44:31.878 INFO:teuthology.orchestra.run.smithi107.stdout:(110/174): libarrow-9.0.0-10.el9.x86_64.rpm 8.0 MB/s | 4.4 MB 00:00 2024-03-08T20:44:31.903 INFO:teuthology.orchestra.run.smithi107.stdout:(111/174): liboath-2.6.7-2.el9.x86_64.rpm 324 kB/s | 49 kB 00:00 2024-03-08T20:44:32.012 INFO:teuthology.orchestra.run.smithi107.stdout:(112/174): libunwind-1.6.2-1.el9.x86_64.rpm 508 kB/s | 67 kB 00:00 2024-03-08T20:44:32.095 INFO:teuthology.orchestra.run.smithi107.stdout:(113/174): luarocks-3.9.2-1.el9.noarch.rpm 790 kB/s | 151 kB 00:00 2024-03-08T20:44:32.304 INFO:teuthology.orchestra.run.smithi107.stdout:(114/174): parquet-libs-9.0.0-10.el9.x86_64.rpm 2.8 MB/s | 838 kB 00:00 2024-03-08T20:44:32.362 INFO:teuthology.orchestra.run.smithi107.stdout:(115/174): python3-asyncssh-2.13.2-4.el9.noarch 2.0 MB/s | 548 kB 00:00 2024-03-08T20:44:32.429 INFO:teuthology.orchestra.run.smithi107.stdout:(116/174): python3-bcrypt-3.2.2-1.el9.x86_64.rp 350 kB/s | 43 kB 00:00 2024-03-08T20:44:32.480 INFO:teuthology.orchestra.run.smithi107.stdout:(117/174): python3-cachetools-4.2.4-1.el9.noarc 276 kB/s | 32 kB 00:00 2024-03-08T20:44:32.555 INFO:teuthology.orchestra.run.smithi107.stdout:(118/174): python3-certifi-2021.10.8-2.el9.noar 120 kB/s | 15 kB 00:00 2024-03-08T20:44:32.614 INFO:teuthology.orchestra.run.smithi107.stdout:(119/174): python3-cheroot-8.6.0-4.el9.noarch.r 1.3 MB/s | 172 kB 00:00 2024-03-08T20:44:32.689 INFO:teuthology.orchestra.run.smithi107.stdout:(120/174): python3-cherrypy-18.6.1-2.el9.noarch 2.6 MB/s | 358 kB 00:00 2024-03-08T20:44:32.773 INFO:teuthology.orchestra.run.smithi107.stdout:(121/174): python3-cmd2-2.4.2-2.el9.noarch.rpm 1.9 MB/s | 303 kB 00:00 2024-03-08T20:44:32.806 INFO:teuthology.orchestra.run.smithi107.stdout:(122/174): python3-colorama-0.4.4-12.el9.noarch 303 kB/s | 35 kB 00:00 2024-03-08T20:44:32.915 INFO:teuthology.orchestra.run.smithi107.stdout:(123/174): python3-google-auth-2.28.0-1.el9.noa 1.4 MB/s | 203 kB 00:00 2024-03-08T20:44:33.015 INFO:teuthology.orchestra.run.smithi107.stdout:(124/174): python3-grpcio-1.46.7-10.el9.x86_64. 9.8 MB/s | 2.0 MB 00:00 2024-03-08T20:44:33.049 INFO:teuthology.orchestra.run.smithi107.stdout:(125/174): python3-grpcio-tools-1.46.7-10.el9.x 1.1 MB/s | 144 kB 00:00 2024-03-08T20:44:33.132 INFO:teuthology.orchestra.run.smithi107.stdout:(126/174): python3-jaraco-8.2.1-3.el9.noarch.rp 92 kB/s | 11 kB 00:00 2024-03-08T20:44:33.166 INFO:teuthology.orchestra.run.smithi107.stdout:(127/174): python3-jaraco-classes-3.2.1-5.el9.n 152 kB/s | 18 kB 00:00 2024-03-08T20:44:33.251 INFO:teuthology.orchestra.run.smithi107.stdout:(128/174): python3-jaraco-collections-3.0.0-8.e 199 kB/s | 23 kB 00:00 2024-03-08T20:44:33.283 INFO:teuthology.orchestra.run.smithi107.stdout:(129/174): python3-jaraco-functools-3.5.0-2.el9 166 kB/s | 19 kB 00:00 2024-03-08T20:44:33.383 INFO:teuthology.orchestra.run.smithi107.stdout:(130/174): python3-jaraco-text-3.2.0-6.el9.noar 148 kB/s | 20 kB 00:00 2024-03-08T20:44:33.509 INFO:teuthology.orchestra.run.smithi107.stdout:(131/174): python3-kubernetes-26.1.0-2.el9.noar 4.5 MB/s | 1.0 MB 00:00 2024-03-08T20:44:33.534 INFO:teuthology.orchestra.run.smithi107.stdout:(132/174): python3-logutils-0.3.5-21.el9.noarch 309 kB/s | 46 kB 00:00 2024-03-08T20:44:33.651 INFO:teuthology.orchestra.run.smithi107.stdout:(133/174): python3-more-itertools-8.12.0-2.el9. 560 kB/s | 79 kB 00:00 2024-03-08T20:44:33.676 INFO:teuthology.orchestra.run.smithi107.stdout:(134/174): python3-natsort-7.1.1-5.el9.noarch.r 409 kB/s | 58 kB 00:00 2024-03-08T20:44:33.801 INFO:teuthology.orchestra.run.smithi107.stdout:(135/174): python3-pecan-1.4.2-3.el9.noarch.rpm 1.8 MB/s | 272 kB 00:00 2024-03-08T20:44:33.827 INFO:teuthology.orchestra.run.smithi107.stdout:(136/174): python3-portend-3.1.0-2.el9.noarch.r 110 kB/s | 16 kB 00:00 2024-03-08T20:44:33.935 INFO:teuthology.orchestra.run.smithi107.stdout:(137/174): python3-pyOpenSSL-21.0.0-1.el9.noarc 677 kB/s | 90 kB 00:00 2024-03-08T20:44:33.961 INFO:teuthology.orchestra.run.smithi107.stdout:(138/174): python3-pyperclip-1.8.2-2.el9.noarch 212 kB/s | 28 kB 00:00 2024-03-08T20:44:34.077 INFO:teuthology.orchestra.run.smithi107.stdout:(139/174): python3-repoze-lru-0.7-16.el9.noarch 218 kB/s | 31 kB 00:00 2024-03-08T20:44:34.211 INFO:teuthology.orchestra.run.smithi107.stdout:(140/174): python3-rsa-4.9-2.el9.noarch.rpm 446 kB/s | 59 kB 00:00 2024-03-08T20:44:34.264 INFO:teuthology.orchestra.run.smithi204.stdout:Transaction test succeeded. 2024-03-08T20:44:34.264 INFO:teuthology.orchestra.run.smithi204.stdout:Running transaction 2024-03-08T20:44:34.345 INFO:teuthology.orchestra.run.smithi107.stdout:(141/174): python3-routes-2.5.1-5.el9.noarch.rp 492 kB/s | 188 kB 00:00 2024-03-08T20:44:34.361 INFO:teuthology.orchestra.run.smithi107.stdout:(142/174): python3-tempora-5.0.0-2.el9.noarch.r 239 kB/s | 36 kB 00:00 2024-03-08T20:44:34.495 INFO:teuthology.orchestra.run.smithi107.stdout:(143/174): python3-webob-1.8.7-6.el9.noarch.rpm 1.7 MB/s | 230 kB 00:00 2024-03-08T20:44:34.629 INFO:teuthology.orchestra.run.smithi107.stdout:(144/174): python3-websocket-client-1.2.3-2.el9 673 kB/s | 90 kB 00:00 2024-03-08T20:44:34.787 INFO:teuthology.orchestra.run.smithi107.stdout:(145/174): python3-werkzeug-2.0.3-3.el9.1.noarc 2.6 MB/s | 427 kB 00:00 2024-03-08T20:44:34.896 INFO:teuthology.orchestra.run.smithi107.stdout:(146/174): python3-zc-lockfile-2.0-10.el9.noarc 186 kB/s | 20 kB 00:00 2024-03-08T20:44:34.938 INFO:teuthology.orchestra.run.smithi107.stdout:(147/174): python3-typing-extensions-4.4.0-2.el 86 kB/s | 51 kB 00:00 2024-03-08T20:44:35.063 INFO:teuthology.orchestra.run.smithi107.stdout:(148/174): re2-20211101-3.el9.x86_64.rpm 1.1 MB/s | 192 kB 00:00 2024-03-08T20:44:36.397 INFO:teuthology.orchestra.run.smithi107.stdout:(149/174): librados2-19.0.0-1965.g7d63482a.el9. 2.5 MB/s | 3.4 MB 00:01 2024-03-08T20:44:37.237 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-selinux-2:19.0.0-1965.g7d63482a.el9.x86_64 132/198 2024-03-08T20:44:37.237 INFO:teuthology.orchestra.run.smithi149.stdout:skipping the directory /sys 2024-03-08T20:44:37.237 INFO:teuthology.orchestra.run.smithi149.stdout:skipping the directory /proc 2024-03-08T20:44:37.237 INFO:teuthology.orchestra.run.smithi149.stdout:skipping the directory /mnt 2024-03-08T20:44:37.238 INFO:teuthology.orchestra.run.smithi149.stdout:skipping the directory /var/tmp 2024-03-08T20:44:37.238 INFO:teuthology.orchestra.run.smithi149.stdout:skipping the directory /home 2024-03-08T20:44:37.238 INFO:teuthology.orchestra.run.smithi149.stdout:skipping the directory /root 2024-03-08T20:44:37.238 INFO:teuthology.orchestra.run.smithi149.stdout:skipping the directory /tmp 2024-03-08T20:44:37.238 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-03-08T20:44:37.459 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-volume-2:19.0.0-1965.g7d63482a.el9.noarch 133/198 2024-03-08T20:44:37.504 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-volume-2:19.0.0-1965.g7d63482a.el9.noarch 133/198 2024-03-08T20:44:37.505 INFO:teuthology.orchestra.run.smithi149.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-08T20:44:37.505 INFO:teuthology.orchestra.run.smithi149.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-03-08T20:44:37.505 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-03-08T20:44:37.697 INFO:teuthology.orchestra.run.smithi107.stdout:(150/174): ceph-test-19.0.0-1965.g7d63482a.el9. 2.5 MB/s | 47 MB 00:19 2024-03-08T20:44:37.798 INFO:teuthology.orchestra.run.smithi107.stdout:(151/174): librbd1-19.0.0-1965.g7d63482a.el9.x8 2.2 MB/s | 3.1 MB 00:01 2024-03-08T20:44:37.850 INFO:teuthology.orchestra.run.smithi107.stdout:(152/174): xfsdump-3.1.12-4.el9.x86_64.rpm 2.1 MB/s | 321 kB 00:00 2024-03-08T20:44:38.631 INFO:teuthology.orchestra.run.smithi107.stdout:(153/174): NetworkManager-1.46.0-1.el9.x86_64.r 2.8 MB/s | 2.3 MB 00:00 2024-03-08T20:44:38.698 INFO:teuthology.orchestra.run.smithi107.stdout:(154/174): NetworkManager-team-1.46.0-1.el9.x86 645 kB/s | 42 kB 00:00 2024-03-08T20:44:38.748 INFO:teuthology.orchestra.run.smithi107.stdout:(155/174): thrift-0.15.0-2.el9.x86_64.rpm 425 kB/s | 1.6 MB 00:03 2024-03-08T20:44:38.806 INFO:teuthology.orchestra.run.smithi107.stdout:(156/174): NetworkManager-libnm-1.46.0-1.el9.x8 1.9 MB/s | 1.8 MB 00:00 2024-03-08T20:44:38.840 INFO:teuthology.orchestra.run.smithi107.stdout:(157/174): NetworkManager-tui-1.46.0-1.el9.x86_ 1.7 MB/s | 248 kB 00:00 2024-03-08T20:44:38.874 INFO:teuthology.orchestra.run.smithi107.stdout:(158/174): elfutils-debuginfod-client-0.190-2.e 550 kB/s | 36 kB 00:00 2024-03-08T20:44:38.916 INFO:teuthology.orchestra.run.smithi107.stdout:(159/174): elfutils-libelf-0.190-2.el9.x86_64.r 2.5 MB/s | 192 kB 00:00 2024-03-08T20:44:38.966 INFO:teuthology.orchestra.run.smithi107.stdout:(160/174): acl-2.3.1-4.el9.x86_64.rpm 329 kB/s | 71 kB 00:00 2024-03-08T20:44:38.999 INFO:teuthology.orchestra.run.smithi107.stdout:(161/174): libacl-2.3.1-4.el9.x86_64.rpm 281 kB/s | 23 kB 00:00 2024-03-08T20:44:39.041 INFO:teuthology.orchestra.run.smithi107.stdout:(162/174): elfutils-libs-0.190-2.el9.x86_64.rpm 1.5 MB/s | 256 kB 00:00 2024-03-08T20:44:39.074 INFO:teuthology.orchestra.run.smithi107.stdout:(163/174): libfdisk-2.37.4-18.el9.x86_64.rpm 2.0 MB/s | 155 kB 00:00 2024-03-08T20:44:39.100 INFO:teuthology.orchestra.run.smithi107.stdout:(164/174): libblkid-2.37.4-18.el9.x86_64.rpm 816 kB/s | 108 kB 00:00 2024-03-08T20:44:39.150 INFO:teuthology.orchestra.run.smithi107.stdout:(165/174): libmount-2.37.4-18.el9.x86_64.rpm 1.2 MB/s | 136 kB 00:00 2024-03-08T20:44:39.176 INFO:teuthology.orchestra.run.smithi107.stdout:(166/174): libuuid-2.37.4-18.el9.x86_64.rpm 384 kB/s | 29 kB 00:00 2024-03-08T20:44:39.180 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-osd-2:19.0.0-1965.g7d63482a.el9.x86_64 134/198 2024-03-08T20:44:39.201 INFO:teuthology.orchestra.run.smithi107.stdout:(167/174): libsmartcols-2.37.4-18.el9.x86_64.rp 508 kB/s | 64 kB 00:00 2024-03-08T20:44:39.275 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-osd-2:19.0.0-1965.g7d63482a.el9.x86_64 134/198 2024-03-08T20:44:39.275 INFO:teuthology.orchestra.run.smithi149.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-08T20:44:39.275 INFO:teuthology.orchestra.run.smithi149.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-03-08T20:44:39.275 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-08T20:44:39.275 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-08T20:44:39.275 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-03-08T20:44:39.513 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-mds-2:19.0.0-1965.g7d63482a.el9.x86_64 135/198 2024-03-08T20:44:39.526 INFO:teuthology.orchestra.run.smithi107.stdout:(168/174): openssl-3.0.7-27.el9.x86_64.rpm 3.2 MB/s | 1.2 MB 00:00 2024-03-08T20:44:39.599 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-mds-2:19.0.0-1965.g7d63482a.el9.x86_64 135/198 2024-03-08T20:44:39.599 INFO:teuthology.orchestra.run.smithi149.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-08T20:44:39.599 INFO:teuthology.orchestra.run.smithi149.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-03-08T20:44:39.599 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-08T20:44:39.599 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-08T20:44:39.600 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-03-08T20:44:39.685 INFO:teuthology.orchestra.run.smithi107.stdout:(169/174): util-linux-2.37.4-18.el9.x86_64.rpm 4.7 MB/s | 2.3 MB 00:00 2024-03-08T20:44:39.727 INFO:teuthology.orchestra.run.smithi107.stdout:(170/174): util-linux-core-2.37.4-18.el9.x86_64 2.3 MB/s | 465 kB 00:00 2024-03-08T20:44:39.752 INFO:teuthology.orchestra.run.smithi107.stdout:(171/174): util-linux-user-2.37.4-18.el9.x86_64 471 kB/s | 31 kB 00:00 2024-03-08T20:44:39.994 INFO:teuthology.orchestra.run.smithi107.stdout:(172/174): xfsprogs-6.3.0-1.el9.x86_64.rpm 4.0 MB/s | 1.1 MB 00:00 2024-03-08T20:44:40.036 INFO:teuthology.orchestra.run.smithi107.stdout:(173/174): libacl-devel-2.3.1-4.el9.x86_64.rpm 263 kB/s | 74 kB 00:00 2024-03-08T20:44:40.080 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-mon-2:19.0.0-1965.g7d63482a.el9.x86_64 136/198 2024-03-08T20:44:40.155 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-mon-2:19.0.0-1965.g7d63482a.el9.x86_64 136/198 2024-03-08T20:44:40.155 INFO:teuthology.orchestra.run.smithi149.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-08T20:44:40.155 INFO:teuthology.orchestra.run.smithi149.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-03-08T20:44:40.155 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-08T20:44:40.155 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-08T20:44:40.155 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-03-08T20:44:40.283 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : automake-1.16.2-8.el9.noarch 137/198 2024-03-08T20:44:40.294 INFO:teuthology.orchestra.run.smithi107.stdout:(174/174): openssl-libs-3.0.7-27.el9.x86_64.rpm 1.9 MB/s | 2.2 MB 00:01 2024-03-08T20:44:40.303 INFO:teuthology.orchestra.run.smithi107.stdout:-------------------------------------------------------------------------------- 2024-03-08T20:44:40.303 INFO:teuthology.orchestra.run.smithi107.stdout:Total 7.9 MB/s | 223 MB 00:28 2024-03-08T20:44:40.832 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: cephadm-2:19.0.0-1965.g7d63482a.el9.noarch 138/198 2024-03-08T20:44:40.875 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : cephadm-2:19.0.0-1965.g7d63482a.el9.noarch 138/198 2024-03-08T20:44:40.955 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-1965.g7d63482a.e 139/198 2024-03-08T20:44:41.064 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-1965.g7d63482a. 140/198 2024-03-08T20:44:41.104 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-1965.g7d63482a.el9.noa 141/198 2024-03-08T20:44:41.417 INFO:teuthology.orchestra.run.smithi204.stdout: Preparing : 1/1 2024-03-08T20:44:41.422 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-1965.g7d63482a.el9.noa 141/198 2024-03-08T20:44:41.455 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-1965.g7d63482a.el9.n 142/198 2024-03-08T20:44:41.516 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/198 2024-03-08T20:44:41.555 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/198 2024-03-08T20:44:41.593 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-08T20:44:41.636 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-08T20:44:41.708 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction check 2024-03-08T20:44:41.755 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/198 2024-03-08T20:44:41.865 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/198 2024-03-08T20:44:41.919 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/198 2024-03-08T20:44:41.950 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/198 2024-03-08T20:44:42.063 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/198 2024-03-08T20:44:42.085 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction check succeeded. 2024-03-08T20:44:42.085 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction test 2024-03-08T20:44:42.494 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/198 2024-03-08T20:44:42.532 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : librados2-2:19.0.0-1965.g7d63482a.el9.x86_64 10/198 2024-03-08T20:44:42.577 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-1965.g7d63482a.el9.n 142/198 2024-03-08T20:44:42.613 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-1965.g7d6 143/198 2024-03-08T20:44:42.672 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: librados2-2:19.0.0-1965.g7d63482a.el9.x86_64 10/198 2024-03-08T20:44:42.699 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libcephfs2-2:19.0.0-1965.g7d63482a.el9.x86_64 11/198 2024-03-08T20:44:42.755 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-1965.g7d6 143/198 2024-03-08T20:44:42.843 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: libcephfs2-2:19.0.0-1965.g7d63482a.el9.x86_64 11/198 2024-03-08T20:44:42.879 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-rados-2:19.0.0-1965.g7d63482a.el9.x86_64 12/198 2024-03-08T20:44:42.925 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-1965.g7d63482a.el 144/198 2024-03-08T20:44:42.940 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 13/198 2024-03-08T20:44:42.957 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-mgr-2:19.0.0-1965.g7d63482a.el9.x86_64 145/198 2024-03-08T20:44:43.036 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-mgr-2:19.0.0-1965.g7d63482a.el9.x86_64 145/198 2024-03-08T20:44:43.036 INFO:teuthology.orchestra.run.smithi149.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-08T20:44:43.036 INFO:teuthology.orchestra.run.smithi149.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-03-08T20:44:43.036 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-08T20:44:43.036 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-08T20:44:43.036 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-03-08T20:44:43.091 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-mgr-rook-2:19.0.0-1965.g7d63482a.el9.noarch 146/198 2024-03-08T20:44:43.128 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-1965.g7d63482a.el9.noarch 146/198 2024-03-08T20:44:43.209 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-2:19.0.0-1965.g7d63482a.el9.x86_64 147/198 2024-03-08T20:44:43.234 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/198 2024-03-08T20:44:43.291 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : postgresql-13.11-1.el9.x86_64 15/198 2024-03-08T20:44:43.316 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libcephsqlite-2:19.0.0-1965.g7d63482a.el9.x86_64 16/198 2024-03-08T20:44:43.468 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: libcephsqlite-2:19.0.0-1965.g7d63482a.el9.x86_64 16/198 2024-03-08T20:44:43.503 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libradosstriper1-2:19.0.0-1965.g7d63482a.el9.x86 17/198 2024-03-08T20:44:43.593 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: libradosstriper1-2:19.0.0-1965.g7d63482a.el9.x86 17/198 2024-03-08T20:44:43.647 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/198 2024-03-08T20:44:43.680 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : qatlib-service-23.11.0-1.el9.x86_64 19/198 2024-03-08T20:44:43.750 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: qatlib-service-23.11.0-1.el9.x86_64 19/198 2024-03-08T20:44:44.073 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: qatlib-23.11.0-1.el9.x86_64 20/198 2024-03-08T20:44:44.113 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : qatlib-23.11.0-1.el9.x86_64 20/198 2024-03-08T20:44:44.150 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libtool-2.4.6-45.el9.x86_64 148/198 2024-03-08T20:44:44.163 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/198 2024-03-08T20:44:44.185 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-radosgw-2:19.0.0-1965.g7d63482a.el9.x86_64 149/198 2024-03-08T20:44:44.262 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-1965.g7d63482a.el9.x86_64 149/198 2024-03-08T20:44:44.262 INFO:teuthology.orchestra.run.smithi149.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-08T20:44:44.262 INFO:teuthology.orchestra.run.smithi149.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-03-08T20:44:44.262 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-08T20:44:44.262 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-08T20:44:44.262 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-03-08T20:44:44.311 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-1965.g7d634 150/198 2024-03-08T20:44:44.395 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-1965.g7d634 150/198 2024-03-08T20:44:44.395 INFO:teuthology.orchestra.run.smithi149.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-08T20:44:44.395 INFO:teuthology.orchestra.run.smithi149.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-03-08T20:44:44.395 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-03-08T20:44:44.481 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction test succeeded. 2024-03-08T20:44:44.481 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction 2024-03-08T20:44:44.501 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : cephfs-mirror-2:19.0.0-1965.g7d63482a.el9.x86_64 151/198 2024-03-08T20:44:44.533 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/198 2024-03-08T20:44:44.591 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-1965.g7d63482a.el9.x86_64 151/198 2024-03-08T20:44:44.591 INFO:teuthology.orchestra.run.smithi149.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-08T20:44:44.591 INFO:teuthology.orchestra.run.smithi149.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-03-08T20:44:44.591 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-08T20:44:44.591 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-08T20:44:44.592 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-03-08T20:44:44.687 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : NetworkManager-libnm-1:1.46.0-1.el9.x86_64 23/198 2024-03-08T20:44:44.768 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: NetworkManager-libnm-1:1.46.0-1.el9.x86_64 23/198 2024-03-08T20:44:44.962 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : rbd-mirror-2:19.0.0-1965.g7d63482a.el9.x86_64 152/198 2024-03-08T20:44:44.980 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: NetworkManager-1:1.46.0-1.el9.x86_64 24/198 2024-03-08T20:44:45.034 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: rbd-mirror-2:19.0.0-1965.g7d63482a.el9.x86_64 152/198 2024-03-08T20:44:45.034 INFO:teuthology.orchestra.run.smithi149.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-08T20:44:45.034 INFO:teuthology.orchestra.run.smithi149.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-03-08T20:44:45.034 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-08T20:44:45.034 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-08T20:44:45.034 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-03-08T20:44:45.057 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : NetworkManager-1:1.46.0-1.el9.x86_64 24/198 2024-03-08T20:44:45.386 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: NetworkManager-1:1.46.0-1.el9.x86_64 24/198 2024-03-08T20:44:45.564 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/198 2024-03-08T20:44:45.679 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/198 2024-03-08T20:44:45.721 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/198 2024-03-08T20:44:45.790 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/198 2024-03-08T20:44:45.856 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/198 2024-03-08T20:44:45.972 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/198 2024-03-08T20:44:46.046 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 31/198 2024-03-08T20:44:46.104 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 32/198 2024-03-08T20:44:46.198 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/198 2024-03-08T20:44:46.252 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/198 2024-03-08T20:44:46.315 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/198 2024-03-08T20:44:46.357 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/198 2024-03-08T20:44:46.388 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : lua-5.4.4-4.el9.x86_64 37/198 2024-03-08T20:44:46.500 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 38/198 2024-03-08T20:44:46.541 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-ceph-common-2:19.0.0-1965.g7d63482a.el9. 39/198 2024-03-08T20:44:46.586 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-ceph-argparse-2:19.0.0-1965.g7d63482a.el 40/198 2024-03-08T20:44:46.615 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-cephfs-2:19.0.0-1965.g7d63482a.el9.x86_6 41/198 2024-03-08T20:44:46.730 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/198 2024-03-08T20:44:46.792 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/198 2024-03-08T20:44:46.886 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/198 2024-03-08T20:44:47.064 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/198 2024-03-08T20:44:47.129 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/198 2024-03-08T20:44:47.778 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/198 2024-03-08T20:44:47.878 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/198 2024-03-08T20:44:47.977 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/198 2024-03-08T20:44:48.028 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-08T20:44:48.052 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-08T20:44:48.095 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : qatzip-libs-1.1.2-2.el9.x86_64 51/198 2024-03-08T20:44:48.163 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 52/198 2024-03-08T20:44:48.220 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : librados-devel-2:19.0.0-1965.g7d63482a.el9.x86_6 53/198 2024-03-08T20:44:48.269 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 54/198 2024-03-08T20:44:48.677 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 55/198 2024-03-08T20:44:48.920 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 56/198 2024-03-08T20:44:49.008 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 57/198 2024-03-08T20:44:49.071 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 58/198 2024-03-08T20:44:49.275 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 59/198 2024-03-08T20:44:49.380 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 60/198 2024-03-08T20:44:49.456 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 61/198 2024-03-08T20:44:49.552 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 62/198 2024-03-08T20:44:49.604 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 63/198 2024-03-08T20:44:49.646 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 64/198 2024-03-08T20:44:49.716 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 65/198 2024-03-08T20:44:49.762 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 66/198 2024-03-08T20:44:49.825 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 67/198 2024-03-08T20:44:49.835 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-test-2:19.0.0-1965.g7d63482a.el9.x86_64 153/198 2024-03-08T20:44:49.889 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 68/198 2024-03-08T20:44:49.905 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : rbd-fuse-2:19.0.0-1965.g7d63482a.el9.x86_64 154/198 2024-03-08T20:44:49.941 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : rbd-nbd-2:19.0.0-1965.g7d63482a.el9.x86_64 155/198 2024-03-08T20:44:49.979 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 156/198 2024-03-08T20:44:50.078 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 69/198 2024-03-08T20:44:50.133 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 70/198 2024-03-08T20:44:50.170 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 71/198 2024-03-08T20:44:50.219 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 72/198 2024-03-08T20:44:50.250 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : cephfs-shell-2:19.0.0-1965.g7d63482a.el9.x86_64 157/198 2024-03-08T20:44:50.348 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 73/198 2024-03-08T20:44:50.378 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 158/198 2024-03-08T20:44:50.406 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-google-auth-1:2.28.0-1.el9.noarch 74/198 2024-03-08T20:44:50.434 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-fuse-2:19.0.0-1965.g7d63482a.el9.x86_64 159/198 2024-03-08T20:44:50.458 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 75/198 2024-03-08T20:44:50.490 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 160/198 2024-03-08T20:44:50.494 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 76/198 2024-03-08T20:44:50.528 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 77/198 2024-03-08T20:44:50.560 INFO:teuthology.orchestra.run.smithi149.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/198 2024-03-08T20:44:50.612 INFO:teuthology.orchestra.run.smithi149.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 162/198 2024-03-08T20:44:50.648 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libcephfs-devel-2:19.0.0-1965.g7d63482a.el9.x86_ 163/198 2024-03-08T20:44:50.693 INFO:teuthology.orchestra.run.smithi149.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 164/198 2024-03-08T20:44:50.694 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 78/198 2024-03-08T20:44:50.745 INFO:teuthology.orchestra.run.smithi149.stdout: Upgrading : NetworkManager-team-1:1.46.0-1.el9.x86_64 165/198 2024-03-08T20:44:50.865 INFO:teuthology.orchestra.run.smithi149.stdout: Upgrading : NetworkManager-tui-1:1.46.0-1.el9.x86_64 166/198 2024-03-08T20:44:50.870 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 79/198 2024-03-08T20:44:50.972 INFO:teuthology.orchestra.run.smithi149.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 167/198 2024-03-08T20:44:51.001 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 80/198 2024-03-08T20:44:51.010 INFO:teuthology.orchestra.run.smithi149.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 168/198 2024-03-08T20:44:51.055 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 81/198 2024-03-08T20:44:51.117 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-ply-3.11-14.el9.noarch 82/198 2024-03-08T20:44:51.191 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 83/198 2024-03-08T20:44:51.444 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 84/198 2024-03-08T20:44:51.525 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 85/198 2024-03-08T20:44:51.694 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 86/198 2024-03-08T20:44:51.740 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-asyncssh-2.13.2-4.el9.noarch 87/198 2024-03-08T20:44:51.788 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 88/198 2024-03-08T20:44:51.788 INFO:teuthology.orchestra.run.smithi107.stdout: Preparing : 1/1 2024-03-08T20:44:51.826 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : fuse-2.9.9-15.el9.x86_64 89/198 2024-03-08T20:44:51.883 INFO:teuthology.orchestra.run.smithi107.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/198 2024-03-08T20:44:51.913 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-08T20:44:51.927 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 90/198 2024-03-08T20:44:51.951 INFO:teuthology.orchestra.run.smithi107.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/198 2024-03-08T20:44:51.957 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-08T20:44:51.995 INFO:teuthology.orchestra.run.smithi107.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-08T20:44:52.012 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-08T20:44:52.037 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-08T20:44:52.124 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/198 2024-03-08T20:44:52.243 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/198 2024-03-08T20:44:52.312 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/198 2024-03-08T20:44:52.327 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 91/198 2024-03-08T20:44:52.343 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/198 2024-03-08T20:44:52.398 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : uuid-1.6.2-55.el9.x86_64 92/198 2024-03-08T20:44:52.431 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/198 2024-03-08T20:44:52.463 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 93/198 2024-03-08T20:44:52.507 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 94/198 2024-03-08T20:44:52.648 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 95/198 2024-03-08T20:44:52.694 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 96/198 2024-03-08T20:44:52.736 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 97/198 2024-03-08T20:44:52.802 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 98/198 2024-03-08T20:44:52.839 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 99/198 2024-03-08T20:44:52.858 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/198 2024-03-08T20:44:52.908 INFO:teuthology.orchestra.run.smithi107.stdout: Upgrading : librados2-2:19.0.0-1965.g7d63482a.el9.x86_64 10/198 2024-03-08T20:44:52.981 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 100/198 2024-03-08T20:44:53.042 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: librados2-2:19.0.0-1965.g7d63482a.el9.x86_64 10/198 2024-03-08T20:44:53.075 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libcephfs2-2:19.0.0-1965.g7d63482a.el9.x86_64 11/198 2024-03-08T20:44:53.102 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 101/198 2024-03-08T20:44:53.172 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 102/198 2024-03-08T20:44:53.180 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: libcephfs2-2:19.0.0-1965.g7d63482a.el9.x86_64 11/198 2024-03-08T20:44:53.208 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 103/198 2024-03-08T20:44:53.245 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-rados-2:19.0.0-1965.g7d63482a.el9.x86_64 12/198 2024-03-08T20:44:53.266 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 104/198 2024-03-08T20:44:53.321 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 13/198 2024-03-08T20:44:53.347 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : cephfs-top-2:19.0.0-1965.g7d63482a.el9.noarch 170/198 2024-03-08T20:44:53.405 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-packaging-20.9-5.el9.noarch 105/198 2024-03-08T20:44:53.466 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 106/198 2024-03-08T20:44:53.466 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 171/198 2024-03-08T20:44:53.524 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 172/198 2024-03-08T20:44:53.609 INFO:teuthology.orchestra.run.smithi107.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/198 2024-03-08T20:44:53.660 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : flex-2.6.4-9.el9.x86_64 173/198 2024-03-08T20:44:53.663 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : postgresql-13.11-1.el9.x86_64 15/198 2024-03-08T20:44:53.688 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libcephsqlite-2:19.0.0-1965.g7d63482a.el9.x86_64 16/198 2024-03-08T20:44:53.726 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : bison-3.7.4-5.el9.x86_64 174/198 2024-03-08T20:44:53.759 INFO:teuthology.orchestra.run.smithi149.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 175/198 2024-03-08T20:44:53.793 INFO:teuthology.orchestra.run.smithi149.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 176/198 2024-03-08T20:44:53.837 INFO:teuthology.orchestra.run.smithi149.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 177/198 2024-03-08T20:44:53.837 INFO:teuthology.orchestra.run.smithi149.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-08T20:44:53.838 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: libcephsqlite-2:19.0.0-1965.g7d63482a.el9.x86_64 16/198 2024-03-08T20:44:53.873 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libradosstriper1-2:19.0.0-1965.g7d63482a.el9.x86 17/198 2024-03-08T20:44:53.962 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: libradosstriper1-2:19.0.0-1965.g7d63482a.el9.x86 17/198 2024-03-08T20:44:53.980 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-08T20:44:54.017 INFO:teuthology.orchestra.run.smithi107.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/198 2024-03-08T20:44:54.022 INFO:teuthology.orchestra.run.smithi149.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 179/198 2024-03-08T20:44:54.022 INFO:teuthology.orchestra.run.smithi149.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-08T20:44:54.049 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : qatlib-service-23.11.0-1.el9.x86_64 19/198 2024-03-08T20:44:54.111 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: qatlib-service-23.11.0-1.el9.x86_64 19/198 2024-03-08T20:44:54.152 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-08T20:44:54.202 INFO:teuthology.orchestra.run.smithi149.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 181/198 2024-03-08T20:44:54.258 INFO:teuthology.orchestra.run.smithi149.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 182/198 2024-03-08T20:44:54.292 INFO:teuthology.orchestra.run.smithi149.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 183/198 2024-03-08T20:44:54.324 INFO:teuthology.orchestra.run.smithi149.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 184/198 2024-03-08T20:44:54.400 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: qatlib-23.11.0-1.el9.x86_64 20/198 2024-03-08T20:44:54.409 INFO:teuthology.orchestra.run.smithi149.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 185/198 2024-03-08T20:44:54.432 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 107/198 2024-03-08T20:44:54.449 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : qatlib-23.11.0-1.el9.x86_64 20/198 2024-03-08T20:44:54.482 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 108/198 2024-03-08T20:44:54.488 INFO:teuthology.orchestra.run.smithi149.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 186/198 2024-03-08T20:44:54.509 INFO:teuthology.orchestra.run.smithi107.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/198 2024-03-08T20:44:54.515 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 109/198 2024-03-08T20:44:54.537 INFO:teuthology.orchestra.run.smithi149.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 187/198 2024-03-08T20:44:54.570 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 110/198 2024-03-08T20:44:54.571 INFO:teuthology.orchestra.run.smithi149.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 188/198 2024-03-08T20:44:54.605 INFO:teuthology.orchestra.run.smithi149.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 189/198 2024-03-08T20:44:54.638 INFO:teuthology.orchestra.run.smithi149.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 190/198 2024-03-08T20:44:54.671 INFO:teuthology.orchestra.run.smithi149.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 191/198 2024-03-08T20:44:54.682 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 111/198 2024-03-08T20:44:54.707 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-08T20:44:54.708 INFO:teuthology.orchestra.run.smithi149.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-08T20:44:54.716 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 112/198 2024-03-08T20:44:54.827 INFO:teuthology.orchestra.run.smithi107.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/198 2024-03-08T20:44:54.964 INFO:teuthology.orchestra.run.smithi107.stdout: Upgrading : NetworkManager-libnm-1:1.46.0-1.el9.x86_64 23/198 2024-03-08T20:44:54.971 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : openblas-0.3.21-2.el9.x86_64 113/198 2024-03-08T20:44:55.013 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 114/198 2024-03-08T20:44:55.047 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-08T20:44:55.047 INFO:teuthology.orchestra.run.smithi149.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-08T20:44:55.053 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: NetworkManager-libnm-1:1.46.0-1.el9.x86_64 23/198 2024-03-08T20:44:55.104 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 115/198 2024-03-08T20:44:55.194 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-08T20:44:55.224 INFO:teuthology.orchestra.run.smithi149.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 194/198 2024-03-08T20:44:55.244 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: NetworkManager-1:1.46.0-1.el9.x86_64 24/198 2024-03-08T20:44:55.266 INFO:teuthology.orchestra.run.smithi149.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 195/198 2024-03-08T20:44:55.301 INFO:teuthology.orchestra.run.smithi107.stdout: Upgrading : NetworkManager-1:1.46.0-1.el9.x86_64 24/198 2024-03-08T20:44:55.323 INFO:teuthology.orchestra.run.smithi149.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 196/198 2024-03-08T20:44:55.359 INFO:teuthology.orchestra.run.smithi149.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 197/198 2024-03-08T20:44:55.477 INFO:teuthology.orchestra.run.smithi149.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-08T20:44:55.609 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: NetworkManager-1:1.46.0-1.el9.x86_64 24/198 2024-03-08T20:44:55.736 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-08T20:44:55.736 INFO:teuthology.orchestra.run.smithi149.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-03-08T20:44:55.736 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-03-08T20:44:55.765 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/198 2024-03-08T20:44:55.889 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/198 2024-03-08T20:44:55.921 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 116/198 2024-03-08T20:44:55.923 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/198 2024-03-08T20:44:55.958 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/198 2024-03-08T20:44:56.023 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/198 2024-03-08T20:44:56.050 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 117/198 2024-03-08T20:44:56.135 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/198 2024-03-08T20:44:56.197 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 31/198 2024-03-08T20:44:56.240 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 32/198 2024-03-08T20:44:56.340 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/198 2024-03-08T20:44:56.386 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/198 2024-03-08T20:44:56.472 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/198 2024-03-08T20:44:56.525 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/198 2024-03-08T20:44:56.557 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : lua-5.4.4-4.el9.x86_64 37/198 2024-03-08T20:44:56.649 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 38/198 2024-03-08T20:44:56.684 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-ceph-common-2:19.0.0-1965.g7d63482a.el9. 39/198 2024-03-08T20:44:56.734 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-ceph-argparse-2:19.0.0-1965.g7d63482a.el 40/198 2024-03-08T20:44:56.766 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-cephfs-2:19.0.0-1965.g7d63482a.el9.x86_6 41/198 2024-03-08T20:44:56.877 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/198 2024-03-08T20:44:56.936 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/198 2024-03-08T20:44:57.061 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/198 2024-03-08T20:44:57.245 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/198 2024-03-08T20:44:57.314 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/198 2024-03-08T20:44:57.573 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 118/198 2024-03-08T20:44:57.668 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 119/198 2024-03-08T20:44:57.707 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 120/198 2024-03-08T20:44:57.737 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : jq-1.6-16.el9.x86_64 121/198 2024-03-08T20:44:57.778 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 122/198 2024-03-08T20:44:57.992 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/198 2024-03-08T20:44:58.093 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/198 2024-03-08T20:44:58.153 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 123/198 2024-03-08T20:44:58.184 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : librbd1-2:19.0.0-1965.g7d63482a.el9.x86_64 124/198 2024-03-08T20:44:58.218 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/198 2024-03-08T20:44:58.270 INFO:teuthology.orchestra.run.smithi107.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-08T20:44:58.295 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-08T20:44:58.304 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: librbd1-2:19.0.0-1965.g7d63482a.el9.x86_64 124/198 2024-03-08T20:44:58.332 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-rbd-2:19.0.0-1965.g7d63482a.el9.x86_64 125/198 2024-03-08T20:44:58.370 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : qatzip-libs-1.1.2-2.el9.x86_64 51/198 2024-03-08T20:44:58.387 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 126/198 2024-03-08T20:44:58.445 INFO:teuthology.orchestra.run.smithi107.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 52/198 2024-03-08T20:44:58.508 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : librados-devel-2:19.0.0-1965.g7d63482a.el9.x86_6 53/198 2024-03-08T20:44:58.553 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 54/198 2024-03-08T20:44:58.892 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 127/198 2024-03-08T20:44:58.929 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : librgw2-2:19.0.0-1965.g7d63482a.el9.x86_64 128/198 2024-03-08T20:44:58.960 INFO:teuthology.orchestra.run.smithi107.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 55/198 2024-03-08T20:44:59.003 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: librgw2-2:19.0.0-1965.g7d63482a.el9.x86_64 128/198 2024-03-08T20:44:59.031 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : python3-rgw-2:19.0.0-1965.g7d63482a.el9.x86_64 129/198 2024-03-08T20:44:59.167 INFO:teuthology.orchestra.run.smithi107.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 56/198 2024-03-08T20:44:59.257 INFO:teuthology.orchestra.run.smithi107.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 57/198 2024-03-08T20:44:59.330 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 58/198 2024-03-08T20:44:59.534 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 59/198 2024-03-08T20:44:59.595 INFO:teuthology.orchestra.run.smithi107.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 60/198 2024-03-08T20:44:59.693 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 61/198 2024-03-08T20:44:59.794 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 62/198 2024-03-08T20:44:59.837 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 63/198 2024-03-08T20:44:59.876 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 64/198 2024-03-08T20:44:59.945 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 65/198 2024-03-08T20:44:59.984 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 66/198 2024-03-08T20:45:00.046 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 67/198 2024-03-08T20:45:00.115 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 68/198 2024-03-08T20:45:00.306 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 69/198 2024-03-08T20:45:00.365 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 70/198 2024-03-08T20:45:00.403 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 71/198 2024-03-08T20:45:00.449 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-2:19.0.0-1965.g7d63482a.el9.x86_64 1/198 2024-03-08T20:45:00.449 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-base-2:19.0.0-1965.g7d63482a.el9.x86_64 2/198 2024-03-08T20:45:00.449 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-common-2:19.0.0-1965.g7d63482a.el9.x86_64 3/198 2024-03-08T20:45:00.449 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-fuse-2:19.0.0-1965.g7d63482a.el9.x86_64 4/198 2024-03-08T20:45:00.449 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-1965.g7d634 5/198 2024-03-08T20:45:00.449 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-mds-2:19.0.0-1965.g7d63482a.el9.x86_64 6/198 2024-03-08T20:45:00.449 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-mgr-2:19.0.0-1965.g7d63482a.el9.x86_64 7/198 2024-03-08T20:45:00.450 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-mon-2:19.0.0-1965.g7d63482a.el9.x86_64 8/198 2024-03-08T20:45:00.450 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-osd-2:19.0.0-1965.g7d63482a.el9.x86_64 9/198 2024-03-08T20:45:00.450 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-radosgw-2:19.0.0-1965.g7d63482a.el9.x86_64 10/198 2024-03-08T20:45:00.450 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-selinux-2:19.0.0-1965.g7d63482a.el9.x86_64 11/198 2024-03-08T20:45:00.450 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-test-2:19.0.0-1965.g7d63482a.el9.x86_64 12/198 2024-03-08T20:45:00.450 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : cephfs-mirror-2:19.0.0-1965.g7d63482a.el9.x86_64 13/198 2024-03-08T20:45:00.450 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : cephfs-shell-2:19.0.0-1965.g7d63482a.el9.x86_64 14/198 2024-03-08T20:45:00.450 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libcephfs-devel-2:19.0.0-1965.g7d63482a.el9.x86_ 15/198 2024-03-08T20:45:00.450 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libcephfs2-2:19.0.0-1965.g7d63482a.el9.x86_64 16/198 2024-03-08T20:45:00.450 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libcephsqlite-2:19.0.0-1965.g7d63482a.el9.x86_64 17/198 2024-03-08T20:45:00.450 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : librados-devel-2:19.0.0-1965.g7d63482a.el9.x86_6 18/198 2024-03-08T20:45:00.450 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libradosstriper1-2:19.0.0-1965.g7d63482a.el9.x86 19/198 2024-03-08T20:45:00.450 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : librgw2-2:19.0.0-1965.g7d63482a.el9.x86_64 20/198 2024-03-08T20:45:00.451 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-ceph-argparse-2:19.0.0-1965.g7d63482a.el 21/198 2024-03-08T20:45:00.451 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-ceph-common-2:19.0.0-1965.g7d63482a.el9. 22/198 2024-03-08T20:45:00.451 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-cephfs-2:19.0.0-1965.g7d63482a.el9.x86_6 23/198 2024-03-08T20:45:00.451 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-rados-2:19.0.0-1965.g7d63482a.el9.x86_64 24/198 2024-03-08T20:45:00.451 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-rbd-2:19.0.0-1965.g7d63482a.el9.x86_64 25/198 2024-03-08T20:45:00.451 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-rgw-2:19.0.0-1965.g7d63482a.el9.x86_64 26/198 2024-03-08T20:45:00.451 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : rbd-fuse-2:19.0.0-1965.g7d63482a.el9.x86_64 27/198 2024-03-08T20:45:00.451 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : rbd-mirror-2:19.0.0-1965.g7d63482a.el9.x86_64 28/198 2024-03-08T20:45:00.451 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : rbd-nbd-2:19.0.0-1965.g7d63482a.el9.x86_64 29/198 2024-03-08T20:45:00.452 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 72/198 2024-03-08T20:45:00.468 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-1965.g7d63482a. 30/198 2024-03-08T20:45:00.468 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-1965.g7d63482a.el9.noa 31/198 2024-03-08T20:45:00.468 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-1965.g7d63482a.el9.n 32/198 2024-03-08T20:45:00.468 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-1965.g7d6 33/198 2024-03-08T20:45:00.469 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-1965.g7d63482a.el 34/198 2024-03-08T20:45:00.469 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-mgr-rook-2:19.0.0-1965.g7d63482a.el9.noarch 35/198 2024-03-08T20:45:00.469 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-1965.g7d63482a.e 36/198 2024-03-08T20:45:00.469 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-volume-2:19.0.0-1965.g7d63482a.el9.noarch 37/198 2024-03-08T20:45:00.469 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : cephadm-2:19.0.0-1965.g7d63482a.el9.noarch 38/198 2024-03-08T20:45:00.469 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : cephfs-top-2:19.0.0-1965.g7d63482a.el9.noarch 39/198 2024-03-08T20:45:00.469 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/198 2024-03-08T20:45:00.469 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/198 2024-03-08T20:45:00.469 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/198 2024-03-08T20:45:00.469 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/198 2024-03-08T20:45:00.469 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/198 2024-03-08T20:45:00.469 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/198 2024-03-08T20:45:00.470 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/198 2024-03-08T20:45:00.470 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 47/198 2024-03-08T20:45:00.470 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 48/198 2024-03-08T20:45:00.470 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 49/198 2024-03-08T20:45:00.470 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 50/198 2024-03-08T20:45:00.470 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 51/198 2024-03-08T20:45:00.470 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 52/198 2024-03-08T20:45:00.470 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 53/198 2024-03-08T20:45:00.470 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : lua-5.4.4-4.el9.x86_64 54/198 2024-03-08T20:45:00.470 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 55/198 2024-03-08T20:45:00.470 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 56/198 2024-03-08T20:45:00.470 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 57/198 2024-03-08T20:45:00.470 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : postgresql-13.11-1.el9.x86_64 58/198 2024-03-08T20:45:00.471 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 59/198 2024-03-08T20:45:00.471 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 60/198 2024-03-08T20:45:00.471 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 61/198 2024-03-08T20:45:00.471 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 62/198 2024-03-08T20:45:00.471 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 63/198 2024-03-08T20:45:00.471 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 64/198 2024-03-08T20:45:00.471 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 65/198 2024-03-08T20:45:00.471 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 66/198 2024-03-08T20:45:00.471 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 67/198 2024-03-08T20:45:00.471 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 68/198 2024-03-08T20:45:00.471 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 69/198 2024-03-08T20:45:00.471 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 70/198 2024-03-08T20:45:00.471 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 71/198 2024-03-08T20:45:00.472 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 72/198 2024-03-08T20:45:00.472 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 73/198 2024-03-08T20:45:00.472 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 74/198 2024-03-08T20:45:00.472 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 75/198 2024-03-08T20:45:00.472 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 76/198 2024-03-08T20:45:00.472 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 77/198 2024-03-08T20:45:00.472 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 78/198 2024-03-08T20:45:00.472 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 79/198 2024-03-08T20:45:00.472 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 80/198 2024-03-08T20:45:00.472 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 81/198 2024-03-08T20:45:00.472 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 82/198 2024-03-08T20:45:00.472 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 83/198 2024-03-08T20:45:00.472 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 84/198 2024-03-08T20:45:00.473 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 85/198 2024-03-08T20:45:00.473 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 86/198 2024-03-08T20:45:00.473 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 87/198 2024-03-08T20:45:00.473 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 88/198 2024-03-08T20:45:00.473 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-ply-3.11-14.el9.noarch 89/198 2024-03-08T20:45:00.473 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 90/198 2024-03-08T20:45:00.473 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 91/198 2024-03-08T20:45:00.473 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : jq-1.6-16.el9.x86_64 92/198 2024-03-08T20:45:00.473 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 93/198 2024-03-08T20:45:00.473 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 94/198 2024-03-08T20:45:00.473 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 95/198 2024-03-08T20:45:00.473 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 96/198 2024-03-08T20:45:00.473 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 97/198 2024-03-08T20:45:00.474 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 98/198 2024-03-08T20:45:00.474 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 99/198 2024-03-08T20:45:00.474 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 100/198 2024-03-08T20:45:00.474 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : qatlib-23.11.0-1.el9.x86_64 101/198 2024-03-08T20:45:00.474 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : qatlib-service-23.11.0-1.el9.x86_64 102/198 2024-03-08T20:45:00.474 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : qatzip-libs-1.1.2-2.el9.x86_64 103/198 2024-03-08T20:45:00.474 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 104/198 2024-03-08T20:45:00.474 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/198 2024-03-08T20:45:00.474 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/198 2024-03-08T20:45:00.474 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/198 2024-03-08T20:45:00.474 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/198 2024-03-08T20:45:00.474 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/198 2024-03-08T20:45:00.474 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/198 2024-03-08T20:45:00.474 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/198 2024-03-08T20:45:00.475 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/198 2024-03-08T20:45:00.475 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/198 2024-03-08T20:45:00.475 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/198 2024-03-08T20:45:00.475 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-03-08T20:45:00.475 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-asyncssh-2.13.2-4.el9.noarch 116/198 2024-03-08T20:45:00.475 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/198 2024-03-08T20:45:00.475 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/198 2024-03-08T20:45:00.475 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/198 2024-03-08T20:45:00.475 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/198 2024-03-08T20:45:00.475 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/198 2024-03-08T20:45:00.475 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/198 2024-03-08T20:45:00.475 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/198 2024-03-08T20:45:00.475 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-google-auth-1:2.28.0-1.el9.noarch 124/198 2024-03-08T20:45:00.475 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/198 2024-03-08T20:45:00.476 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/198 2024-03-08T20:45:00.476 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/198 2024-03-08T20:45:00.476 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/198 2024-03-08T20:45:00.476 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/198 2024-03-08T20:45:00.476 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/198 2024-03-08T20:45:00.476 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/198 2024-03-08T20:45:00.476 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/198 2024-03-08T20:45:00.476 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/198 2024-03-08T20:45:00.476 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/198 2024-03-08T20:45:00.476 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/198 2024-03-08T20:45:00.476 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/198 2024-03-08T20:45:00.476 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/198 2024-03-08T20:45:00.476 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/198 2024-03-08T20:45:00.477 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/198 2024-03-08T20:45:00.477 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/198 2024-03-08T20:45:00.477 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/198 2024-03-08T20:45:00.477 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/198 2024-03-08T20:45:00.477 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/198 2024-03-08T20:45:00.477 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/198 2024-03-08T20:45:00.477 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/198 2024-03-08T20:45:00.477 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/198 2024-03-08T20:45:00.477 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/198 2024-03-08T20:45:00.477 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/198 2024-03-08T20:45:00.477 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/198 2024-03-08T20:45:00.477 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/198 2024-03-08T20:45:00.477 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : librados2-2:19.0.0-1965.g7d63482a.el9.x86_64 151/198 2024-03-08T20:45:00.477 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/198 2024-03-08T20:45:00.478 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : librbd1-2:19.0.0-1965.g7d63482a.el9.x86_64 153/198 2024-03-08T20:45:00.478 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/198 2024-03-08T20:45:00.478 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 155/198 2024-03-08T20:45:00.478 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 156/198 2024-03-08T20:45:00.478 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : NetworkManager-1:1.46.0-1.el9.x86_64 157/198 2024-03-08T20:45:00.478 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 158/198 2024-03-08T20:45:00.478 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : NetworkManager-libnm-1:1.46.0-1.el9.x86_64 159/198 2024-03-08T20:45:00.478 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 160/198 2024-03-08T20:45:00.478 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : NetworkManager-team-1:1.46.0-1.el9.x86_64 161/198 2024-03-08T20:45:00.478 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 162/198 2024-03-08T20:45:00.478 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : NetworkManager-tui-1:1.46.0-1.el9.x86_64 163/198 2024-03-08T20:45:00.478 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 164/198 2024-03-08T20:45:00.478 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : acl-2.3.1-4.el9.x86_64 165/198 2024-03-08T20:45:00.478 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : acl-2.3.1-3.el9.x86_64 166/198 2024-03-08T20:45:00.479 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 167/198 2024-03-08T20:45:00.479 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/198 2024-03-08T20:45:00.479 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 169/198 2024-03-08T20:45:00.479 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 170/198 2024-03-08T20:45:00.479 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 171/198 2024-03-08T20:45:00.479 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 172/198 2024-03-08T20:45:00.479 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 173/198 2024-03-08T20:45:00.479 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 174/198 2024-03-08T20:45:00.479 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 175/198 2024-03-08T20:45:00.479 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 176/198 2024-03-08T20:45:00.479 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 177/198 2024-03-08T20:45:00.479 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 178/198 2024-03-08T20:45:00.479 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 179/198 2024-03-08T20:45:00.479 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 180/198 2024-03-08T20:45:00.480 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 181/198 2024-03-08T20:45:00.480 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 182/198 2024-03-08T20:45:00.480 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 183/198 2024-03-08T20:45:00.480 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 184/198 2024-03-08T20:45:00.480 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 185/198 2024-03-08T20:45:00.480 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 186/198 2024-03-08T20:45:00.480 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 187/198 2024-03-08T20:45:00.480 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 188/198 2024-03-08T20:45:00.480 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 189/198 2024-03-08T20:45:00.480 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 190/198 2024-03-08T20:45:00.480 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 191/198 2024-03-08T20:45:00.480 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 192/198 2024-03-08T20:45:00.480 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 193/198 2024-03-08T20:45:00.480 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 194/198 2024-03-08T20:45:00.481 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 195/198 2024-03-08T20:45:00.481 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 196/198 2024-03-08T20:45:00.481 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 197/198 2024-03-08T20:45:00.579 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 73/198 2024-03-08T20:45:00.618 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-google-auth-1:2.28.0-1.el9.noarch 74/198 2024-03-08T20:45:00.663 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 75/198 2024-03-08T20:45:00.700 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 76/198 2024-03-08T20:45:00.742 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 77/198 2024-03-08T20:45:00.910 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 78/198 2024-03-08T20:45:01.082 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 79/198 2024-03-08T20:45:01.219 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 80/198 2024-03-08T20:45:01.271 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 81/198 2024-03-08T20:45:01.342 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-ply-3.11-14.el9.noarch 82/198 2024-03-08T20:45:01.416 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 83/198 2024-03-08T20:45:01.570 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-common-2:19.0.0-1965.g7d63482a.el9.x86_64 130/198 2024-03-08T20:45:01.602 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-common-2:19.0.0-1965.g7d63482a.el9.x86_64 130/198 2024-03-08T20:45:01.661 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 84/198 2024-03-08T20:45:01.726 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 85/198 2024-03-08T20:45:01.900 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 86/198 2024-03-08T20:45:01.949 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-asyncssh-2.13.2-4.el9.noarch 87/198 2024-03-08T20:45:01.988 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 88/198 2024-03-08T20:45:02.035 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : fuse-2.9.9-15.el9.x86_64 89/198 2024-03-08T20:45:02.083 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 90/198 2024-03-08T20:45:02.155 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 91/198 2024-03-08T20:45:02.227 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-common-2:19.0.0-1965.g7d63482a.el9.x86_64 130/198 2024-03-08T20:45:02.234 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : uuid-1.6.2-55.el9.x86_64 92/198 2024-03-08T20:45:02.281 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-base-2:19.0.0-1965.g7d63482a.el9.x86_64 131/198 2024-03-08T20:45:02.294 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 93/198 2024-03-08T20:45:02.340 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 94/198 2024-03-08T20:45:02.462 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-base-2:19.0.0-1965.g7d63482a.el9.x86_64 131/198 2024-03-08T20:45:02.462 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-03-08T20:45:02.462 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-03-08T20:45:02.462 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-03-08T20:45:02.468 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 95/198 2024-03-08T20:45:02.507 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-selinux-2:19.0.0-1965.g7d63482a.el9.x86_64 132/198 2024-03-08T20:45:02.510 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 96/198 2024-03-08T20:45:02.558 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 97/198 2024-03-08T20:45:02.612 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 98/198 2024-03-08T20:45:02.653 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 99/198 2024-03-08T20:45:02.793 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 100/198 2024-03-08T20:45:02.906 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 198/198 2024-03-08T20:45:02.907 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-03-08T20:45:02.907 INFO:teuthology.orchestra.run.smithi149.stdout:Upgraded: 2024-03-08T20:45:02.907 INFO:teuthology.orchestra.run.smithi149.stdout: NetworkManager-1:1.46.0-1.el9.x86_64 2024-03-08T20:45:02.907 INFO:teuthology.orchestra.run.smithi149.stdout: NetworkManager-libnm-1:1.46.0-1.el9.x86_64 2024-03-08T20:45:02.907 INFO:teuthology.orchestra.run.smithi149.stdout: NetworkManager-team-1:1.46.0-1.el9.x86_64 2024-03-08T20:45:02.907 INFO:teuthology.orchestra.run.smithi149.stdout: NetworkManager-tui-1:1.46.0-1.el9.x86_64 2024-03-08T20:45:02.907 INFO:teuthology.orchestra.run.smithi149.stdout: acl-2.3.1-4.el9.x86_64 2024-03-08T20:45:02.907 INFO:teuthology.orchestra.run.smithi149.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-03-08T20:45:02.907 INFO:teuthology.orchestra.run.smithi149.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-03-08T20:45:02.907 INFO:teuthology.orchestra.run.smithi149.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-03-08T20:45:02.907 INFO:teuthology.orchestra.run.smithi149.stdout: libacl-2.3.1-4.el9.x86_64 2024-03-08T20:45:02.907 INFO:teuthology.orchestra.run.smithi149.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-03-08T20:45:02.908 INFO:teuthology.orchestra.run.smithi149.stdout: libblkid-2.37.4-18.el9.x86_64 2024-03-08T20:45:02.908 INFO:teuthology.orchestra.run.smithi149.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-03-08T20:45:02.908 INFO:teuthology.orchestra.run.smithi149.stdout: libmount-2.37.4-18.el9.x86_64 2024-03-08T20:45:02.908 INFO:teuthology.orchestra.run.smithi149.stdout: librados2-2:19.0.0-1965.g7d63482a.el9.x86_64 2024-03-08T20:45:02.908 INFO:teuthology.orchestra.run.smithi149.stdout: librbd1-2:19.0.0-1965.g7d63482a.el9.x86_64 2024-03-08T20:45:02.908 INFO:teuthology.orchestra.run.smithi149.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-03-08T20:45:02.908 INFO:teuthology.orchestra.run.smithi149.stdout: libuuid-2.37.4-18.el9.x86_64 2024-03-08T20:45:02.908 INFO:teuthology.orchestra.run.smithi149.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-03-08T20:45:02.908 INFO:teuthology.orchestra.run.smithi149.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-03-08T20:45:02.908 INFO:teuthology.orchestra.run.smithi149.stdout: util-linux-2.37.4-18.el9.x86_64 2024-03-08T20:45:02.908 INFO:teuthology.orchestra.run.smithi149.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-03-08T20:45:02.908 INFO:teuthology.orchestra.run.smithi149.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-03-08T20:45:02.908 INFO:teuthology.orchestra.run.smithi149.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-03-08T20:45:02.909 INFO:teuthology.orchestra.run.smithi149.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-03-08T20:45:02.909 INFO:teuthology.orchestra.run.smithi149.stdout:Installed: 2024-03-08T20:45:02.909 INFO:teuthology.orchestra.run.smithi149.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-03-08T20:45:02.909 INFO:teuthology.orchestra.run.smithi149.stdout: automake-1.16.2-8.el9.noarch 2024-03-08T20:45:02.909 INFO:teuthology.orchestra.run.smithi149.stdout: bison-3.7.4-5.el9.x86_64 2024-03-08T20:45:02.909 INFO:teuthology.orchestra.run.smithi149.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-03-08T20:45:02.909 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-2:19.0.0-1965.g7d63482a.el9.x86_64 2024-03-08T20:45:02.909 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-base-2:19.0.0-1965.g7d63482a.el9.x86_64 2024-03-08T20:45:02.909 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-common-2:19.0.0-1965.g7d63482a.el9.x86_64 2024-03-08T20:45:02.909 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-fuse-2:19.0.0-1965.g7d63482a.el9.x86_64 2024-03-08T20:45:02.909 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-grafana-dashboards-2:19.0.0-1965.g7d63482a.el9.noarch 2024-03-08T20:45:02.909 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-immutable-object-cache-2:19.0.0-1965.g7d63482a.el9.x86_64 2024-03-08T20:45:02.909 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mds-2:19.0.0-1965.g7d63482a.el9.x86_64 2024-03-08T20:45:02.910 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mgr-2:19.0.0-1965.g7d63482a.el9.x86_64 2024-03-08T20:45:02.910 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mgr-cephadm-2:19.0.0-1965.g7d63482a.el9.noarch 2024-03-08T20:45:02.910 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mgr-dashboard-2:19.0.0-1965.g7d63482a.el9.noarch 2024-03-08T20:45:02.910 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mgr-diskprediction-local-2:19.0.0-1965.g7d63482a.el9.noarch 2024-03-08T20:45:02.910 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mgr-modules-core-2:19.0.0-1965.g7d63482a.el9.noarch 2024-03-08T20:45:02.910 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mgr-rook-2:19.0.0-1965.g7d63482a.el9.noarch 2024-03-08T20:45:02.910 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mon-2:19.0.0-1965.g7d63482a.el9.x86_64 2024-03-08T20:45:02.910 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-osd-2:19.0.0-1965.g7d63482a.el9.x86_64 2024-03-08T20:45:02.910 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-prometheus-alerts-2:19.0.0-1965.g7d63482a.el9.noarch 2024-03-08T20:45:02.910 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-radosgw-2:19.0.0-1965.g7d63482a.el9.x86_64 2024-03-08T20:45:02.910 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-selinux-2:19.0.0-1965.g7d63482a.el9.x86_64 2024-03-08T20:45:02.911 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-test-2:19.0.0-1965.g7d63482a.el9.x86_64 2024-03-08T20:45:02.911 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-volume-2:19.0.0-1965.g7d63482a.el9.noarch 2024-03-08T20:45:02.911 INFO:teuthology.orchestra.run.smithi149.stdout: cephadm-2:19.0.0-1965.g7d63482a.el9.noarch 2024-03-08T20:45:02.911 INFO:teuthology.orchestra.run.smithi149.stdout: cephfs-mirror-2:19.0.0-1965.g7d63482a.el9.x86_64 2024-03-08T20:45:02.911 INFO:teuthology.orchestra.run.smithi149.stdout: cephfs-shell-2:19.0.0-1965.g7d63482a.el9.x86_64 2024-03-08T20:45:02.911 INFO:teuthology.orchestra.run.smithi149.stdout: cephfs-top-2:19.0.0-1965.g7d63482a.el9.noarch 2024-03-08T20:45:02.911 INFO:teuthology.orchestra.run.smithi149.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-03-08T20:45:02.911 INFO:teuthology.orchestra.run.smithi149.stdout: flex-2.6.4-9.el9.x86_64 2024-03-08T20:45:02.911 INFO:teuthology.orchestra.run.smithi149.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-03-08T20:45:02.911 INFO:teuthology.orchestra.run.smithi149.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-03-08T20:45:02.911 INFO:teuthology.orchestra.run.smithi149.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-03-08T20:45:02.911 INFO:teuthology.orchestra.run.smithi149.stdout: fuse-2.9.9-15.el9.x86_64 2024-03-08T20:45:02.911 INFO:teuthology.orchestra.run.smithi149.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-03-08T20:45:02.911 INFO:teuthology.orchestra.run.smithi149.stdout: grpc-data-1.46.7-10.el9.noarch 2024-03-08T20:45:02.912 INFO:teuthology.orchestra.run.smithi149.stdout: jq-1.6-16.el9.x86_64 2024-03-08T20:45:02.912 INFO:teuthology.orchestra.run.smithi149.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-03-08T20:45:02.912 INFO:teuthology.orchestra.run.smithi149.stdout: libarrow-9.0.0-10.el9.x86_64 2024-03-08T20:45:02.912 INFO:teuthology.orchestra.run.smithi149.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-03-08T20:45:02.912 INFO:teuthology.orchestra.run.smithi149.stdout: libcephfs-devel-2:19.0.0-1965.g7d63482a.el9.x86_64 2024-03-08T20:45:02.912 INFO:teuthology.orchestra.run.smithi149.stdout: libcephfs2-2:19.0.0-1965.g7d63482a.el9.x86_64 2024-03-08T20:45:02.912 INFO:teuthology.orchestra.run.smithi149.stdout: libcephsqlite-2:19.0.0-1965.g7d63482a.el9.x86_64 2024-03-08T20:45:02.912 INFO:teuthology.orchestra.run.smithi149.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-03-08T20:45:02.912 INFO:teuthology.orchestra.run.smithi149.stdout: liboath-2.6.7-2.el9.x86_64 2024-03-08T20:45:02.912 INFO:teuthology.orchestra.run.smithi149.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-03-08T20:45:02.912 INFO:teuthology.orchestra.run.smithi149.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-03-08T20:45:02.912 INFO:teuthology.orchestra.run.smithi149.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-03-08T20:45:02.912 INFO:teuthology.orchestra.run.smithi149.stdout: librados-devel-2:19.0.0-1965.g7d63482a.el9.x86_64 2024-03-08T20:45:02.913 INFO:teuthology.orchestra.run.smithi149.stdout: libradosstriper1-2:19.0.0-1965.g7d63482a.el9.x86_64 2024-03-08T20:45:02.913 INFO:teuthology.orchestra.run.smithi149.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-03-08T20:45:02.913 INFO:teuthology.orchestra.run.smithi149.stdout: librgw2-2:19.0.0-1965.g7d63482a.el9.x86_64 2024-03-08T20:45:02.913 INFO:teuthology.orchestra.run.smithi149.stdout: libtool-2.4.6-45.el9.x86_64 2024-03-08T20:45:02.913 INFO:teuthology.orchestra.run.smithi149.stdout: libunwind-1.6.2-1.el9.x86_64 2024-03-08T20:45:02.913 INFO:teuthology.orchestra.run.smithi149.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-03-08T20:45:02.913 INFO:teuthology.orchestra.run.smithi149.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-03-08T20:45:02.913 INFO:teuthology.orchestra.run.smithi149.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-03-08T20:45:02.913 INFO:teuthology.orchestra.run.smithi149.stdout: lua-5.4.4-4.el9.x86_64 2024-03-08T20:45:02.913 INFO:teuthology.orchestra.run.smithi149.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-03-08T20:45:02.913 INFO:teuthology.orchestra.run.smithi149.stdout: luarocks-3.9.2-1.el9.noarch 2024-03-08T20:45:02.913 INFO:teuthology.orchestra.run.smithi149.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-03-08T20:45:02.913 INFO:teuthology.orchestra.run.smithi149.stdout: openblas-0.3.21-2.el9.x86_64 2024-03-08T20:45:02.913 INFO:teuthology.orchestra.run.smithi149.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-03-08T20:45:02.913 INFO:teuthology.orchestra.run.smithi149.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-03-08T20:45:02.914 INFO:teuthology.orchestra.run.smithi149.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-03-08T20:45:02.914 INFO:teuthology.orchestra.run.smithi149.stdout: postgresql-13.11-1.el9.x86_64 2024-03-08T20:45:02.914 INFO:teuthology.orchestra.run.smithi149.stdout: postgresql-contrib-13.11-1.el9.x86_64 2024-03-08T20:45:02.914 INFO:teuthology.orchestra.run.smithi149.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2024-03-08T20:45:02.914 INFO:teuthology.orchestra.run.smithi149.stdout: postgresql-server-13.11-1.el9.x86_64 2024-03-08T20:45:02.914 INFO:teuthology.orchestra.run.smithi149.stdout: protobuf-3.14.0-13.el9.x86_64 2024-03-08T20:45:02.914 INFO:teuthology.orchestra.run.smithi149.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-03-08T20:45:02.914 INFO:teuthology.orchestra.run.smithi149.stdout: python3-asyncssh-2.13.2-4.el9.noarch 2024-03-08T20:45:02.914 INFO:teuthology.orchestra.run.smithi149.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-03-08T20:45:02.914 INFO:teuthology.orchestra.run.smithi149.stdout: python3-babel-2.9.1-2.el9.noarch 2024-03-08T20:45:02.915 INFO:teuthology.orchestra.run.smithi149.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-03-08T20:45:02.915 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-03-08T20:45:02.915 INFO:teuthology.orchestra.run.smithi149.stdout: python3-ceph-argparse-2:19.0.0-1965.g7d63482a.el9.x86_64 2024-03-08T20:45:02.915 INFO:teuthology.orchestra.run.smithi149.stdout: python3-ceph-common-2:19.0.0-1965.g7d63482a.el9.x86_64 2024-03-08T20:45:02.915 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cephfs-2:19.0.0-1965.g7d63482a.el9.x86_64 2024-03-08T20:45:02.915 INFO:teuthology.orchestra.run.smithi149.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-03-08T20:45:02.915 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-03-08T20:45:02.915 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-03-08T20:45:02.915 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-03-08T20:45:02.915 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-03-08T20:45:02.915 INFO:teuthology.orchestra.run.smithi149.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-03-08T20:45:02.915 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-03-08T20:45:02.915 INFO:teuthology.orchestra.run.smithi149.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-03-08T20:45:02.915 INFO:teuthology.orchestra.run.smithi149.stdout: python3-google-auth-1:2.28.0-1.el9.noarch 2024-03-08T20:45:02.915 INFO:teuthology.orchestra.run.smithi149.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-03-08T20:45:02.916 INFO:teuthology.orchestra.run.smithi149.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-03-08T20:45:02.916 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-03-08T20:45:02.916 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-03-08T20:45:02.916 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-03-08T20:45:02.916 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-03-08T20:45:02.916 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-03-08T20:45:02.916 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-03-08T20:45:02.916 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-03-08T20:45:02.916 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-03-08T20:45:02.916 INFO:teuthology.orchestra.run.smithi149.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-03-08T20:45:02.916 INFO:teuthology.orchestra.run.smithi149.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-03-08T20:45:02.916 INFO:teuthology.orchestra.run.smithi149.stdout: python3-mako-1.1.4-6.el9.noarch 2024-03-08T20:45:02.916 INFO:teuthology.orchestra.run.smithi149.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-03-08T20:45:02.916 INFO:teuthology.orchestra.run.smithi149.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-03-08T20:45:02.916 INFO:teuthology.orchestra.run.smithi149.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-03-08T20:45:02.917 INFO:teuthology.orchestra.run.smithi149.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-03-08T20:45:02.917 INFO:teuthology.orchestra.run.smithi149.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-03-08T20:45:02.917 INFO:teuthology.orchestra.run.smithi149.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-03-08T20:45:02.917 INFO:teuthology.orchestra.run.smithi149.stdout: python3-packaging-20.9-5.el9.noarch 2024-03-08T20:45:02.917 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-03-08T20:45:02.917 INFO:teuthology.orchestra.run.smithi149.stdout: python3-ply-3.11-14.el9.noarch 2024-03-08T20:45:02.917 INFO:teuthology.orchestra.run.smithi149.stdout: python3-portend-3.1.0-2.el9.noarch 2024-03-08T20:45:02.917 INFO:teuthology.orchestra.run.smithi149.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-03-08T20:45:02.917 INFO:teuthology.orchestra.run.smithi149.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-03-08T20:45:02.917 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-03-08T20:45:02.917 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-03-08T20:45:02.917 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-03-08T20:45:02.917 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pycparser-2.20-6.el9.noarch 2024-03-08T20:45:02.918 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-03-08T20:45:02.918 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pytz-2021.1-5.el9.noarch 2024-03-08T20:45:02.918 INFO:teuthology.orchestra.run.smithi149.stdout: python3-rados-2:19.0.0-1965.g7d63482a.el9.x86_64 2024-03-08T20:45:02.918 INFO:teuthology.orchestra.run.smithi149.stdout: python3-rbd-2:19.0.0-1965.g7d63482a.el9.x86_64 2024-03-08T20:45:02.918 INFO:teuthology.orchestra.run.smithi149.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-03-08T20:45:02.918 INFO:teuthology.orchestra.run.smithi149.stdout: python3-requests-2.25.1-8.el9.noarch 2024-03-08T20:45:02.918 INFO:teuthology.orchestra.run.smithi149.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-03-08T20:45:02.918 INFO:teuthology.orchestra.run.smithi149.stdout: python3-rgw-2:19.0.0-1965.g7d63482a.el9.x86_64 2024-03-08T20:45:02.918 INFO:teuthology.orchestra.run.smithi149.stdout: python3-routes-2.5.1-5.el9.noarch 2024-03-08T20:45:02.918 INFO:teuthology.orchestra.run.smithi149.stdout: python3-rsa-4.9-2.el9.noarch 2024-03-08T20:45:02.918 INFO:teuthology.orchestra.run.smithi149.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-03-08T20:45:02.919 INFO:teuthology.orchestra.run.smithi149.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-03-08T20:45:02.919 INFO:teuthology.orchestra.run.smithi149.stdout: python3-toml-0.10.2-6.el9.noarch 2024-03-08T20:45:02.919 INFO:teuthology.orchestra.run.smithi149.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-03-08T20:45:02.919 INFO:teuthology.orchestra.run.smithi149.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-03-08T20:45:02.919 INFO:teuthology.orchestra.run.smithi149.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-03-08T20:45:02.919 INFO:teuthology.orchestra.run.smithi149.stdout: python3-webob-1.8.7-6.el9.noarch 2024-03-08T20:45:02.919 INFO:teuthology.orchestra.run.smithi149.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-03-08T20:45:02.919 INFO:teuthology.orchestra.run.smithi149.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-03-08T20:45:02.919 INFO:teuthology.orchestra.run.smithi149.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-03-08T20:45:02.919 INFO:teuthology.orchestra.run.smithi149.stdout: qatlib-23.11.0-1.el9.x86_64 2024-03-08T20:45:02.919 INFO:teuthology.orchestra.run.smithi149.stdout: qatlib-service-23.11.0-1.el9.x86_64 2024-03-08T20:45:02.919 INFO:teuthology.orchestra.run.smithi149.stdout: qatzip-libs-1.1.2-2.el9.x86_64 2024-03-08T20:45:02.919 INFO:teuthology.orchestra.run.smithi149.stdout: rbd-fuse-2:19.0.0-1965.g7d63482a.el9.x86_64 2024-03-08T20:45:02.919 INFO:teuthology.orchestra.run.smithi149.stdout: rbd-mirror-2:19.0.0-1965.g7d63482a.el9.x86_64 2024-03-08T20:45:02.920 INFO:teuthology.orchestra.run.smithi149.stdout: rbd-nbd-2:19.0.0-1965.g7d63482a.el9.x86_64 2024-03-08T20:45:02.920 INFO:teuthology.orchestra.run.smithi149.stdout: re2-1:20211101-3.el9.x86_64 2024-03-08T20:45:02.920 INFO:teuthology.orchestra.run.smithi149.stdout: socat-1.7.4.1-5.el9.x86_64 2024-03-08T20:45:02.920 INFO:teuthology.orchestra.run.smithi149.stdout: thrift-0.15.0-2.el9.x86_64 2024-03-08T20:45:02.920 INFO:teuthology.orchestra.run.smithi149.stdout: uuid-1.6.2-55.el9.x86_64 2024-03-08T20:45:02.920 INFO:teuthology.orchestra.run.smithi149.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-03-08T20:45:02.920 INFO:teuthology.orchestra.run.smithi149.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-03-08T20:45:02.920 INFO:teuthology.orchestra.run.smithi149.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-03-08T20:45:02.920 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-03-08T20:45:02.921 INFO:teuthology.orchestra.run.smithi149.stdout:Complete! 2024-03-08T20:45:02.925 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 101/198 2024-03-08T20:45:02.983 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 102/198 2024-03-08T20:45:03.020 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 103/198 2024-03-08T20:45:03.086 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 104/198 2024-03-08T20:45:03.234 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-packaging-20.9-5.el9.noarch 105/198 2024-03-08T20:45:03.301 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 106/198 2024-03-08T20:45:03.562 DEBUG:teuthology.parallel:result is None 2024-03-08T20:45:04.251 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 107/198 2024-03-08T20:45:04.310 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 108/198 2024-03-08T20:45:04.343 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 109/198 2024-03-08T20:45:04.411 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 110/198 2024-03-08T20:45:04.531 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 111/198 2024-03-08T20:45:04.568 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 112/198 2024-03-08T20:45:04.797 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : openblas-0.3.21-2.el9.x86_64 113/198 2024-03-08T20:45:04.833 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 114/198 2024-03-08T20:45:04.923 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 115/198 2024-03-08T20:45:05.709 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 116/198 2024-03-08T20:45:05.845 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 117/198 2024-03-08T20:45:07.355 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 118/198 2024-03-08T20:45:07.446 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 119/198 2024-03-08T20:45:07.485 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 120/198 2024-03-08T20:45:07.516 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : jq-1.6-16.el9.x86_64 121/198 2024-03-08T20:45:07.555 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 122/198 2024-03-08T20:45:07.940 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 123/198 2024-03-08T20:45:07.979 INFO:teuthology.orchestra.run.smithi107.stdout: Upgrading : librbd1-2:19.0.0-1965.g7d63482a.el9.x86_64 124/198 2024-03-08T20:45:08.099 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: librbd1-2:19.0.0-1965.g7d63482a.el9.x86_64 124/198 2024-03-08T20:45:08.135 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-rbd-2:19.0.0-1965.g7d63482a.el9.x86_64 125/198 2024-03-08T20:45:08.192 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 126/198 2024-03-08T20:45:08.706 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 127/198 2024-03-08T20:45:08.741 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : librgw2-2:19.0.0-1965.g7d63482a.el9.x86_64 128/198 2024-03-08T20:45:09.174 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: librgw2-2:19.0.0-1965.g7d63482a.el9.x86_64 128/198 2024-03-08T20:45:09.226 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-rgw-2:19.0.0-1965.g7d63482a.el9.x86_64 129/198 2024-03-08T20:45:11.801 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-common-2:19.0.0-1965.g7d63482a.el9.x86_64 130/198 2024-03-08T20:45:11.848 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-common-2:19.0.0-1965.g7d63482a.el9.x86_64 130/198 2024-03-08T20:45:12.468 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-common-2:19.0.0-1965.g7d63482a.el9.x86_64 130/198 2024-03-08T20:45:12.510 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-base-2:19.0.0-1965.g7d63482a.el9.x86_64 131/198 2024-03-08T20:45:12.658 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-base-2:19.0.0-1965.g7d63482a.el9.x86_64 131/198 2024-03-08T20:45:12.659 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-03-08T20:45:12.659 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-03-08T20:45:12.659 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-08T20:45:12.702 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-selinux-2:19.0.0-1965.g7d63482a.el9.x86_64 132/198 2024-03-08T20:45:15.307 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-selinux-2:19.0.0-1965.g7d63482a.el9.x86_64 132/198 2024-03-08T20:45:15.308 INFO:teuthology.orchestra.run.smithi204.stdout:skipping the directory /sys 2024-03-08T20:45:15.308 INFO:teuthology.orchestra.run.smithi204.stdout:skipping the directory /proc 2024-03-08T20:45:15.308 INFO:teuthology.orchestra.run.smithi204.stdout:skipping the directory /mnt 2024-03-08T20:45:15.308 INFO:teuthology.orchestra.run.smithi204.stdout:skipping the directory /var/tmp 2024-03-08T20:45:15.308 INFO:teuthology.orchestra.run.smithi204.stdout:skipping the directory /home 2024-03-08T20:45:15.308 INFO:teuthology.orchestra.run.smithi204.stdout:skipping the directory /root 2024-03-08T20:45:15.308 INFO:teuthology.orchestra.run.smithi204.stdout:skipping the directory /tmp 2024-03-08T20:45:15.308 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-03-08T20:45:15.531 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-volume-2:19.0.0-1965.g7d63482a.el9.noarch 133/198 2024-03-08T20:45:15.576 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-volume-2:19.0.0-1965.g7d63482a.el9.noarch 133/198 2024-03-08T20:45:15.576 INFO:teuthology.orchestra.run.smithi204.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-08T20:45:15.576 INFO:teuthology.orchestra.run.smithi204.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-03-08T20:45:15.576 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-03-08T20:45:17.175 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-osd-2:19.0.0-1965.g7d63482a.el9.x86_64 134/198 2024-03-08T20:45:17.260 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-osd-2:19.0.0-1965.g7d63482a.el9.x86_64 134/198 2024-03-08T20:45:17.261 INFO:teuthology.orchestra.run.smithi204.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-08T20:45:17.261 INFO:teuthology.orchestra.run.smithi204.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-03-08T20:45:17.261 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-08T20:45:17.261 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-08T20:45:17.261 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-03-08T20:45:17.484 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-mds-2:19.0.0-1965.g7d63482a.el9.x86_64 135/198 2024-03-08T20:45:17.552 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-mds-2:19.0.0-1965.g7d63482a.el9.x86_64 135/198 2024-03-08T20:45:17.552 INFO:teuthology.orchestra.run.smithi204.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-08T20:45:17.553 INFO:teuthology.orchestra.run.smithi204.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-03-08T20:45:17.553 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-08T20:45:17.553 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-08T20:45:17.553 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-03-08T20:45:18.035 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-mon-2:19.0.0-1965.g7d63482a.el9.x86_64 136/198 2024-03-08T20:45:18.104 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-mon-2:19.0.0-1965.g7d63482a.el9.x86_64 136/198 2024-03-08T20:45:18.104 INFO:teuthology.orchestra.run.smithi204.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-08T20:45:18.104 INFO:teuthology.orchestra.run.smithi204.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-03-08T20:45:18.104 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-08T20:45:18.104 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-08T20:45:18.104 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-03-08T20:45:18.229 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : automake-1.16.2-8.el9.noarch 137/198 2024-03-08T20:45:18.834 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: cephadm-2:19.0.0-1965.g7d63482a.el9.noarch 138/198 2024-03-08T20:45:18.872 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : cephadm-2:19.0.0-1965.g7d63482a.el9.noarch 138/198 2024-03-08T20:45:18.923 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-1965.g7d63482a.e 139/198 2024-03-08T20:45:19.023 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-1965.g7d63482a. 140/198 2024-03-08T20:45:19.078 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-1965.g7d63482a.el9.noa 141/198 2024-03-08T20:45:19.417 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-1965.g7d63482a.el9.noa 141/198 2024-03-08T20:45:19.451 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-1965.g7d63482a.el9.n 142/198 2024-03-08T20:45:20.575 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-1965.g7d63482a.el9.n 142/198 2024-03-08T20:45:20.601 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-1965.g7d6 143/198 2024-03-08T20:45:20.746 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-1965.g7d6 143/198 2024-03-08T20:45:20.929 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-1965.g7d63482a.el 144/198 2024-03-08T20:45:20.962 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-mgr-2:19.0.0-1965.g7d63482a.el9.x86_64 145/198 2024-03-08T20:45:21.034 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-mgr-2:19.0.0-1965.g7d63482a.el9.x86_64 145/198 2024-03-08T20:45:21.035 INFO:teuthology.orchestra.run.smithi204.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-08T20:45:21.035 INFO:teuthology.orchestra.run.smithi204.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-03-08T20:45:21.035 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-08T20:45:21.035 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-08T20:45:21.035 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-03-08T20:45:21.096 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-mgr-rook-2:19.0.0-1965.g7d63482a.el9.noarch 146/198 2024-03-08T20:45:21.133 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-1965.g7d63482a.el9.noarch 146/198 2024-03-08T20:45:21.223 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-2:19.0.0-1965.g7d63482a.el9.x86_64 147/198 2024-03-08T20:45:22.195 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libtool-2.4.6-45.el9.x86_64 148/198 2024-03-08T20:45:22.232 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-radosgw-2:19.0.0-1965.g7d63482a.el9.x86_64 149/198 2024-03-08T20:45:22.306 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-1965.g7d63482a.el9.x86_64 149/198 2024-03-08T20:45:22.307 INFO:teuthology.orchestra.run.smithi204.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-08T20:45:22.307 INFO:teuthology.orchestra.run.smithi204.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-03-08T20:45:22.307 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-08T20:45:22.307 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-08T20:45:22.307 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-03-08T20:45:22.366 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-1965.g7d634 150/198 2024-03-08T20:45:22.440 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-1965.g7d634 150/198 2024-03-08T20:45:22.441 INFO:teuthology.orchestra.run.smithi204.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-08T20:45:22.441 INFO:teuthology.orchestra.run.smithi204.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-03-08T20:45:22.441 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-03-08T20:45:22.517 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : cephfs-mirror-2:19.0.0-1965.g7d63482a.el9.x86_64 151/198 2024-03-08T20:45:22.591 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-1965.g7d63482a.el9.x86_64 151/198 2024-03-08T20:45:22.592 INFO:teuthology.orchestra.run.smithi204.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-08T20:45:22.592 INFO:teuthology.orchestra.run.smithi204.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-03-08T20:45:22.592 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-08T20:45:22.592 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-08T20:45:22.592 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-03-08T20:45:22.959 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : rbd-mirror-2:19.0.0-1965.g7d63482a.el9.x86_64 152/198 2024-03-08T20:45:23.047 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: rbd-mirror-2:19.0.0-1965.g7d63482a.el9.x86_64 152/198 2024-03-08T20:45:23.047 INFO:teuthology.orchestra.run.smithi204.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-08T20:45:23.048 INFO:teuthology.orchestra.run.smithi204.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-03-08T20:45:23.048 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-08T20:45:23.048 INFO:teuthology.orchestra.run.smithi204.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-08T20:45:23.048 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-03-08T20:45:25.545 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-selinux-2:19.0.0-1965.g7d63482a.el9.x86_64 132/198 2024-03-08T20:45:25.545 INFO:teuthology.orchestra.run.smithi107.stdout:skipping the directory /sys 2024-03-08T20:45:25.545 INFO:teuthology.orchestra.run.smithi107.stdout:skipping the directory /proc 2024-03-08T20:45:25.545 INFO:teuthology.orchestra.run.smithi107.stdout:skipping the directory /mnt 2024-03-08T20:45:25.545 INFO:teuthology.orchestra.run.smithi107.stdout:skipping the directory /var/tmp 2024-03-08T20:45:25.545 INFO:teuthology.orchestra.run.smithi107.stdout:skipping the directory /home 2024-03-08T20:45:25.546 INFO:teuthology.orchestra.run.smithi107.stdout:skipping the directory /root 2024-03-08T20:45:25.546 INFO:teuthology.orchestra.run.smithi107.stdout:skipping the directory /tmp 2024-03-08T20:45:25.546 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-08T20:45:25.792 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-volume-2:19.0.0-1965.g7d63482a.el9.noarch 133/198 2024-03-08T20:45:25.838 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-volume-2:19.0.0-1965.g7d63482a.el9.noarch 133/198 2024-03-08T20:45:25.838 INFO:teuthology.orchestra.run.smithi107.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-08T20:45:25.838 INFO:teuthology.orchestra.run.smithi107.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-03-08T20:45:25.838 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-08T20:45:27.463 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-osd-2:19.0.0-1965.g7d63482a.el9.x86_64 134/198 2024-03-08T20:45:27.540 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-osd-2:19.0.0-1965.g7d63482a.el9.x86_64 134/198 2024-03-08T20:45:27.541 INFO:teuthology.orchestra.run.smithi107.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-08T20:45:27.541 INFO:teuthology.orchestra.run.smithi107.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-03-08T20:45:27.541 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-08T20:45:27.541 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-08T20:45:27.541 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-08T20:45:27.771 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-mds-2:19.0.0-1965.g7d63482a.el9.x86_64 135/198 2024-03-08T20:45:27.849 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-mds-2:19.0.0-1965.g7d63482a.el9.x86_64 135/198 2024-03-08T20:45:27.849 INFO:teuthology.orchestra.run.smithi107.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-08T20:45:27.849 INFO:teuthology.orchestra.run.smithi107.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-03-08T20:45:27.849 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-08T20:45:27.849 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-08T20:45:27.849 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-08T20:45:28.151 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-test-2:19.0.0-1965.g7d63482a.el9.x86_64 153/198 2024-03-08T20:45:28.222 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : rbd-fuse-2:19.0.0-1965.g7d63482a.el9.x86_64 154/198 2024-03-08T20:45:28.259 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : rbd-nbd-2:19.0.0-1965.g7d63482a.el9.x86_64 155/198 2024-03-08T20:45:28.299 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 156/198 2024-03-08T20:45:28.330 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-mon-2:19.0.0-1965.g7d63482a.el9.x86_64 136/198 2024-03-08T20:45:28.397 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-mon-2:19.0.0-1965.g7d63482a.el9.x86_64 136/198 2024-03-08T20:45:28.397 INFO:teuthology.orchestra.run.smithi107.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-08T20:45:28.398 INFO:teuthology.orchestra.run.smithi107.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-03-08T20:45:28.398 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-08T20:45:28.398 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-08T20:45:28.398 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-08T20:45:28.507 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : automake-1.16.2-8.el9.noarch 137/198 2024-03-08T20:45:28.594 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : cephfs-shell-2:19.0.0-1965.g7d63482a.el9.x86_64 157/198 2024-03-08T20:45:28.725 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 158/198 2024-03-08T20:45:28.785 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : ceph-fuse-2:19.0.0-1965.g7d63482a.el9.x86_64 159/198 2024-03-08T20:45:28.847 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 160/198 2024-03-08T20:45:28.934 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/198 2024-03-08T20:45:28.988 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 162/198 2024-03-08T20:45:29.031 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libcephfs-devel-2:19.0.0-1965.g7d63482a.el9.x86_ 163/198 2024-03-08T20:45:29.075 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 164/198 2024-03-08T20:45:29.118 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: cephadm-2:19.0.0-1965.g7d63482a.el9.noarch 138/198 2024-03-08T20:45:29.127 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : NetworkManager-team-1:1.46.0-1.el9.x86_64 165/198 2024-03-08T20:45:29.158 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : cephadm-2:19.0.0-1965.g7d63482a.el9.noarch 138/198 2024-03-08T20:45:29.210 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-1965.g7d63482a.e 139/198 2024-03-08T20:45:29.245 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : NetworkManager-tui-1:1.46.0-1.el9.x86_64 166/198 2024-03-08T20:45:29.311 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-1965.g7d63482a. 140/198 2024-03-08T20:45:29.337 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-1965.g7d63482a.el9.noa 141/198 2024-03-08T20:45:29.359 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 167/198 2024-03-08T20:45:29.400 INFO:teuthology.orchestra.run.smithi204.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 168/198 2024-03-08T20:45:29.667 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-1965.g7d63482a.el9.noa 141/198 2024-03-08T20:45:29.705 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-1965.g7d63482a.el9.n 142/198 2024-03-08T20:45:30.240 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-08T20:45:30.281 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-08T20:45:30.337 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-08T20:45:30.820 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-1965.g7d63482a.el9.n 142/198 2024-03-08T20:45:30.847 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-1965.g7d6 143/198 2024-03-08T20:45:30.988 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-1965.g7d6 143/198 2024-03-08T20:45:31.161 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-1965.g7d63482a.el 144/198 2024-03-08T20:45:31.199 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-mgr-2:19.0.0-1965.g7d63482a.el9.x86_64 145/198 2024-03-08T20:45:31.290 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-mgr-2:19.0.0-1965.g7d63482a.el9.x86_64 145/198 2024-03-08T20:45:31.290 INFO:teuthology.orchestra.run.smithi107.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-08T20:45:31.290 INFO:teuthology.orchestra.run.smithi107.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-03-08T20:45:31.290 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-08T20:45:31.290 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-08T20:45:31.291 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-08T20:45:31.350 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-mgr-rook-2:19.0.0-1965.g7d63482a.el9.noarch 146/198 2024-03-08T20:45:31.389 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-1965.g7d63482a.el9.noarch 146/198 2024-03-08T20:45:31.518 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-2:19.0.0-1965.g7d63482a.el9.x86_64 147/198 2024-03-08T20:45:31.650 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : cephfs-top-2:19.0.0-1965.g7d63482a.el9.noarch 170/198 2024-03-08T20:45:31.745 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 171/198 2024-03-08T20:45:31.800 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 172/198 2024-03-08T20:45:31.917 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : flex-2.6.4-9.el9.x86_64 173/198 2024-03-08T20:45:31.956 INFO:teuthology.orchestra.run.smithi204.stdout: Installing : bison-3.7.4-5.el9.x86_64 174/198 2024-03-08T20:45:31.989 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 175/198 2024-03-08T20:45:32.023 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 176/198 2024-03-08T20:45:32.063 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 177/198 2024-03-08T20:45:32.063 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-08T20:45:32.229 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-08T20:45:32.270 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 179/198 2024-03-08T20:45:32.270 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-08T20:45:32.408 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-08T20:45:32.463 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 181/198 2024-03-08T20:45:32.465 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libtool-2.4.6-45.el9.x86_64 148/198 2024-03-08T20:45:32.502 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-radosgw-2:19.0.0-1965.g7d63482a.el9.x86_64 149/198 2024-03-08T20:45:32.506 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 182/198 2024-03-08T20:45:32.537 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 183/198 2024-03-08T20:45:32.563 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 184/198 2024-03-08T20:45:32.595 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-1965.g7d63482a.el9.x86_64 149/198 2024-03-08T20:45:32.595 INFO:teuthology.orchestra.run.smithi107.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-08T20:45:32.595 INFO:teuthology.orchestra.run.smithi107.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-03-08T20:45:32.595 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-08T20:45:32.595 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-08T20:45:32.595 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-08T20:45:32.642 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 185/198 2024-03-08T20:45:32.652 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-1965.g7d634 150/198 2024-03-08T20:45:32.725 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 186/198 2024-03-08T20:45:32.748 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-1965.g7d634 150/198 2024-03-08T20:45:32.748 INFO:teuthology.orchestra.run.smithi107.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-08T20:45:32.748 INFO:teuthology.orchestra.run.smithi107.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-03-08T20:45:32.748 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-08T20:45:32.769 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 187/198 2024-03-08T20:45:32.811 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 188/198 2024-03-08T20:45:32.820 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : cephfs-mirror-2:19.0.0-1965.g7d63482a.el9.x86_64 151/198 2024-03-08T20:45:32.861 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 189/198 2024-03-08T20:45:32.894 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 190/198 2024-03-08T20:45:32.923 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-1965.g7d63482a.el9.x86_64 151/198 2024-03-08T20:45:32.923 INFO:teuthology.orchestra.run.smithi107.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-08T20:45:32.923 INFO:teuthology.orchestra.run.smithi107.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-03-08T20:45:32.923 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-08T20:45:32.923 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-08T20:45:32.923 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-08T20:45:32.927 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 191/198 2024-03-08T20:45:32.964 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-08T20:45:32.965 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-08T20:45:33.294 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-08T20:45:33.294 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-08T20:45:33.304 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : rbd-mirror-2:19.0.0-1965.g7d63482a.el9.x86_64 152/198 2024-03-08T20:45:33.379 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: rbd-mirror-2:19.0.0-1965.g7d63482a.el9.x86_64 152/198 2024-03-08T20:45:33.379 INFO:teuthology.orchestra.run.smithi107.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-08T20:45:33.380 INFO:teuthology.orchestra.run.smithi107.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-03-08T20:45:33.380 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-08T20:45:33.380 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-08T20:45:33.380 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-08T20:45:33.417 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-08T20:45:33.447 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 194/198 2024-03-08T20:45:33.481 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 195/198 2024-03-08T20:45:33.537 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 196/198 2024-03-08T20:45:33.574 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 197/198 2024-03-08T20:45:33.692 INFO:teuthology.orchestra.run.smithi204.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-08T20:45:33.949 INFO:teuthology.orchestra.run.smithi204.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-08T20:45:33.949 INFO:teuthology.orchestra.run.smithi204.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-03-08T20:45:33.949 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-03-08T20:45:38.127 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-test-2:19.0.0-1965.g7d63482a.el9.x86_64 153/198 2024-03-08T20:45:38.180 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : rbd-fuse-2:19.0.0-1965.g7d63482a.el9.x86_64 154/198 2024-03-08T20:45:38.225 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : rbd-nbd-2:19.0.0-1965.g7d63482a.el9.x86_64 155/198 2024-03-08T20:45:38.263 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 156/198 2024-03-08T20:45:38.535 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : cephfs-shell-2:19.0.0-1965.g7d63482a.el9.x86_64 157/198 2024-03-08T20:45:38.670 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 158/198 2024-03-08T20:45:38.719 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-fuse-2:19.0.0-1965.g7d63482a.el9.x86_64 159/198 2024-03-08T20:45:38.770 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 160/198 2024-03-08T20:45:38.835 INFO:teuthology.orchestra.run.smithi107.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/198 2024-03-08T20:45:38.881 INFO:teuthology.orchestra.run.smithi107.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 162/198 2024-03-08T20:45:38.915 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libcephfs-devel-2:19.0.0-1965.g7d63482a.el9.x86_ 163/198 2024-03-08T20:45:38.961 INFO:teuthology.orchestra.run.smithi107.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 164/198 2024-03-08T20:45:39.015 INFO:teuthology.orchestra.run.smithi107.stdout: Upgrading : NetworkManager-team-1:1.46.0-1.el9.x86_64 165/198 2024-03-08T20:45:39.039 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-2:19.0.0-1965.g7d63482a.el9.x86_64 1/198 2024-03-08T20:45:39.040 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-base-2:19.0.0-1965.g7d63482a.el9.x86_64 2/198 2024-03-08T20:45:39.040 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-common-2:19.0.0-1965.g7d63482a.el9.x86_64 3/198 2024-03-08T20:45:39.040 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-fuse-2:19.0.0-1965.g7d63482a.el9.x86_64 4/198 2024-03-08T20:45:39.040 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-1965.g7d634 5/198 2024-03-08T20:45:39.040 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-mds-2:19.0.0-1965.g7d63482a.el9.x86_64 6/198 2024-03-08T20:45:39.040 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-mgr-2:19.0.0-1965.g7d63482a.el9.x86_64 7/198 2024-03-08T20:45:39.040 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-mon-2:19.0.0-1965.g7d63482a.el9.x86_64 8/198 2024-03-08T20:45:39.040 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-osd-2:19.0.0-1965.g7d63482a.el9.x86_64 9/198 2024-03-08T20:45:39.041 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-radosgw-2:19.0.0-1965.g7d63482a.el9.x86_64 10/198 2024-03-08T20:45:39.041 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-selinux-2:19.0.0-1965.g7d63482a.el9.x86_64 11/198 2024-03-08T20:45:39.041 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-test-2:19.0.0-1965.g7d63482a.el9.x86_64 12/198 2024-03-08T20:45:39.041 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : cephfs-mirror-2:19.0.0-1965.g7d63482a.el9.x86_64 13/198 2024-03-08T20:45:39.041 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : cephfs-shell-2:19.0.0-1965.g7d63482a.el9.x86_64 14/198 2024-03-08T20:45:39.041 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libcephfs-devel-2:19.0.0-1965.g7d63482a.el9.x86_ 15/198 2024-03-08T20:45:39.041 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libcephfs2-2:19.0.0-1965.g7d63482a.el9.x86_64 16/198 2024-03-08T20:45:39.041 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libcephsqlite-2:19.0.0-1965.g7d63482a.el9.x86_64 17/198 2024-03-08T20:45:39.041 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : librados-devel-2:19.0.0-1965.g7d63482a.el9.x86_6 18/198 2024-03-08T20:45:39.041 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libradosstriper1-2:19.0.0-1965.g7d63482a.el9.x86 19/198 2024-03-08T20:45:39.041 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : librgw2-2:19.0.0-1965.g7d63482a.el9.x86_64 20/198 2024-03-08T20:45:39.041 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-ceph-argparse-2:19.0.0-1965.g7d63482a.el 21/198 2024-03-08T20:45:39.041 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-ceph-common-2:19.0.0-1965.g7d63482a.el9. 22/198 2024-03-08T20:45:39.041 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-cephfs-2:19.0.0-1965.g7d63482a.el9.x86_6 23/198 2024-03-08T20:45:39.042 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-rados-2:19.0.0-1965.g7d63482a.el9.x86_64 24/198 2024-03-08T20:45:39.042 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-rbd-2:19.0.0-1965.g7d63482a.el9.x86_64 25/198 2024-03-08T20:45:39.042 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-rgw-2:19.0.0-1965.g7d63482a.el9.x86_64 26/198 2024-03-08T20:45:39.042 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : rbd-fuse-2:19.0.0-1965.g7d63482a.el9.x86_64 27/198 2024-03-08T20:45:39.042 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : rbd-mirror-2:19.0.0-1965.g7d63482a.el9.x86_64 28/198 2024-03-08T20:45:39.042 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : rbd-nbd-2:19.0.0-1965.g7d63482a.el9.x86_64 29/198 2024-03-08T20:45:39.042 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-1965.g7d63482a. 30/198 2024-03-08T20:45:39.042 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-1965.g7d63482a.el9.noa 31/198 2024-03-08T20:45:39.042 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-1965.g7d63482a.el9.n 32/198 2024-03-08T20:45:39.042 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-1965.g7d6 33/198 2024-03-08T20:45:39.042 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-1965.g7d63482a.el 34/198 2024-03-08T20:45:39.042 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-mgr-rook-2:19.0.0-1965.g7d63482a.el9.noarch 35/198 2024-03-08T20:45:39.043 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-1965.g7d63482a.e 36/198 2024-03-08T20:45:39.043 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : ceph-volume-2:19.0.0-1965.g7d63482a.el9.noarch 37/198 2024-03-08T20:45:39.043 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : cephadm-2:19.0.0-1965.g7d63482a.el9.noarch 38/198 2024-03-08T20:45:39.043 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : cephfs-top-2:19.0.0-1965.g7d63482a.el9.noarch 39/198 2024-03-08T20:45:39.043 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/198 2024-03-08T20:45:39.043 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/198 2024-03-08T20:45:39.043 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/198 2024-03-08T20:45:39.043 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/198 2024-03-08T20:45:39.043 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/198 2024-03-08T20:45:39.043 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/198 2024-03-08T20:45:39.043 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/198 2024-03-08T20:45:39.043 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 47/198 2024-03-08T20:45:39.043 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 48/198 2024-03-08T20:45:39.044 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 49/198 2024-03-08T20:45:39.044 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 50/198 2024-03-08T20:45:39.044 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 51/198 2024-03-08T20:45:39.044 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 52/198 2024-03-08T20:45:39.044 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 53/198 2024-03-08T20:45:39.044 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : lua-5.4.4-4.el9.x86_64 54/198 2024-03-08T20:45:39.044 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 55/198 2024-03-08T20:45:39.044 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 56/198 2024-03-08T20:45:39.044 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 57/198 2024-03-08T20:45:39.044 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : postgresql-13.11-1.el9.x86_64 58/198 2024-03-08T20:45:39.044 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 59/198 2024-03-08T20:45:39.044 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 60/198 2024-03-08T20:45:39.045 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 61/198 2024-03-08T20:45:39.045 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 62/198 2024-03-08T20:45:39.045 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 63/198 2024-03-08T20:45:39.045 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 64/198 2024-03-08T20:45:39.045 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 65/198 2024-03-08T20:45:39.045 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 66/198 2024-03-08T20:45:39.045 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 67/198 2024-03-08T20:45:39.045 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 68/198 2024-03-08T20:45:39.045 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 69/198 2024-03-08T20:45:39.045 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 70/198 2024-03-08T20:45:39.045 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 71/198 2024-03-08T20:45:39.045 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 72/198 2024-03-08T20:45:39.045 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 73/198 2024-03-08T20:45:39.046 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 74/198 2024-03-08T20:45:39.046 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 75/198 2024-03-08T20:45:39.046 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 76/198 2024-03-08T20:45:39.046 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 77/198 2024-03-08T20:45:39.046 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 78/198 2024-03-08T20:45:39.046 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 79/198 2024-03-08T20:45:39.049 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 80/198 2024-03-08T20:45:39.049 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 81/198 2024-03-08T20:45:39.049 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 82/198 2024-03-08T20:45:39.049 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 83/198 2024-03-08T20:45:39.049 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 84/198 2024-03-08T20:45:39.050 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 85/198 2024-03-08T20:45:39.050 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 86/198 2024-03-08T20:45:39.050 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 87/198 2024-03-08T20:45:39.050 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 88/198 2024-03-08T20:45:39.050 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-ply-3.11-14.el9.noarch 89/198 2024-03-08T20:45:39.050 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 90/198 2024-03-08T20:45:39.050 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 91/198 2024-03-08T20:45:39.050 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : jq-1.6-16.el9.x86_64 92/198 2024-03-08T20:45:39.050 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 93/198 2024-03-08T20:45:39.050 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 94/198 2024-03-08T20:45:39.050 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 95/198 2024-03-08T20:45:39.050 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 96/198 2024-03-08T20:45:39.051 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 97/198 2024-03-08T20:45:39.051 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 98/198 2024-03-08T20:45:39.051 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 99/198 2024-03-08T20:45:39.051 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 100/198 2024-03-08T20:45:39.051 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : qatlib-23.11.0-1.el9.x86_64 101/198 2024-03-08T20:45:39.051 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : qatlib-service-23.11.0-1.el9.x86_64 102/198 2024-03-08T20:45:39.051 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : qatzip-libs-1.1.2-2.el9.x86_64 103/198 2024-03-08T20:45:39.051 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 104/198 2024-03-08T20:45:39.051 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/198 2024-03-08T20:45:39.051 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/198 2024-03-08T20:45:39.051 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/198 2024-03-08T20:45:39.051 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/198 2024-03-08T20:45:39.051 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/198 2024-03-08T20:45:39.052 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/198 2024-03-08T20:45:39.052 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/198 2024-03-08T20:45:39.052 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/198 2024-03-08T20:45:39.052 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/198 2024-03-08T20:45:39.052 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/198 2024-03-08T20:45:39.052 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-03-08T20:45:39.052 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-asyncssh-2.13.2-4.el9.noarch 116/198 2024-03-08T20:45:39.052 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/198 2024-03-08T20:45:39.053 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/198 2024-03-08T20:45:39.053 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/198 2024-03-08T20:45:39.053 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/198 2024-03-08T20:45:39.053 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/198 2024-03-08T20:45:39.053 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/198 2024-03-08T20:45:39.053 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/198 2024-03-08T20:45:39.053 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-google-auth-1:2.28.0-1.el9.noarch 124/198 2024-03-08T20:45:39.053 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/198 2024-03-08T20:45:39.053 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/198 2024-03-08T20:45:39.053 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/198 2024-03-08T20:45:39.053 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/198 2024-03-08T20:45:39.053 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/198 2024-03-08T20:45:39.053 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/198 2024-03-08T20:45:39.054 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/198 2024-03-08T20:45:39.054 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/198 2024-03-08T20:45:39.054 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/198 2024-03-08T20:45:39.054 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/198 2024-03-08T20:45:39.054 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/198 2024-03-08T20:45:39.054 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/198 2024-03-08T20:45:39.054 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/198 2024-03-08T20:45:39.054 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/198 2024-03-08T20:45:39.054 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/198 2024-03-08T20:45:39.054 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/198 2024-03-08T20:45:39.054 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/198 2024-03-08T20:45:39.054 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/198 2024-03-08T20:45:39.055 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/198 2024-03-08T20:45:39.055 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/198 2024-03-08T20:45:39.055 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/198 2024-03-08T20:45:39.055 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/198 2024-03-08T20:45:39.055 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/198 2024-03-08T20:45:39.055 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/198 2024-03-08T20:45:39.055 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/198 2024-03-08T20:45:39.055 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/198 2024-03-08T20:45:39.055 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : librados2-2:19.0.0-1965.g7d63482a.el9.x86_64 151/198 2024-03-08T20:45:39.055 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/198 2024-03-08T20:45:39.055 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : librbd1-2:19.0.0-1965.g7d63482a.el9.x86_64 153/198 2024-03-08T20:45:39.055 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/198 2024-03-08T20:45:39.055 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 155/198 2024-03-08T20:45:39.056 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 156/198 2024-03-08T20:45:39.056 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : NetworkManager-1:1.46.0-1.el9.x86_64 157/198 2024-03-08T20:45:39.056 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 158/198 2024-03-08T20:45:39.056 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : NetworkManager-libnm-1:1.46.0-1.el9.x86_64 159/198 2024-03-08T20:45:39.056 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 160/198 2024-03-08T20:45:39.056 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : NetworkManager-team-1:1.46.0-1.el9.x86_64 161/198 2024-03-08T20:45:39.056 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 162/198 2024-03-08T20:45:39.056 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : NetworkManager-tui-1:1.46.0-1.el9.x86_64 163/198 2024-03-08T20:45:39.056 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 164/198 2024-03-08T20:45:39.056 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : acl-2.3.1-4.el9.x86_64 165/198 2024-03-08T20:45:39.056 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : acl-2.3.1-3.el9.x86_64 166/198 2024-03-08T20:45:39.056 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 167/198 2024-03-08T20:45:39.057 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/198 2024-03-08T20:45:39.057 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 169/198 2024-03-08T20:45:39.057 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 170/198 2024-03-08T20:45:39.057 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 171/198 2024-03-08T20:45:39.057 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 172/198 2024-03-08T20:45:39.057 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 173/198 2024-03-08T20:45:39.057 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 174/198 2024-03-08T20:45:39.057 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 175/198 2024-03-08T20:45:39.057 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 176/198 2024-03-08T20:45:39.057 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 177/198 2024-03-08T20:45:39.057 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 178/198 2024-03-08T20:45:39.057 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 179/198 2024-03-08T20:45:39.057 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 180/198 2024-03-08T20:45:39.058 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 181/198 2024-03-08T20:45:39.058 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 182/198 2024-03-08T20:45:39.058 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 183/198 2024-03-08T20:45:39.058 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 184/198 2024-03-08T20:45:39.058 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 185/198 2024-03-08T20:45:39.058 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 186/198 2024-03-08T20:45:39.058 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 187/198 2024-03-08T20:45:39.058 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 188/198 2024-03-08T20:45:39.058 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 189/198 2024-03-08T20:45:39.058 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 190/198 2024-03-08T20:45:39.059 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 191/198 2024-03-08T20:45:39.059 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 192/198 2024-03-08T20:45:39.059 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 193/198 2024-03-08T20:45:39.059 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 194/198 2024-03-08T20:45:39.059 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 195/198 2024-03-08T20:45:39.059 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 196/198 2024-03-08T20:45:39.059 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 197/198 2024-03-08T20:45:39.100 INFO:teuthology.orchestra.run.smithi107.stdout: Upgrading : NetworkManager-tui-1:1.46.0-1.el9.x86_64 166/198 2024-03-08T20:45:39.228 INFO:teuthology.orchestra.run.smithi107.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 167/198 2024-03-08T20:45:39.261 INFO:teuthology.orchestra.run.smithi107.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 168/198 2024-03-08T20:45:40.130 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-08T20:45:40.167 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-08T20:45:40.228 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-08T20:45:41.492 INFO:teuthology.orchestra.run.smithi204.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 198/198 2024-03-08T20:45:41.493 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-03-08T20:45:41.493 INFO:teuthology.orchestra.run.smithi204.stdout:Upgraded: 2024-03-08T20:45:41.493 INFO:teuthology.orchestra.run.smithi204.stdout: NetworkManager-1:1.46.0-1.el9.x86_64 2024-03-08T20:45:41.493 INFO:teuthology.orchestra.run.smithi204.stdout: NetworkManager-libnm-1:1.46.0-1.el9.x86_64 2024-03-08T20:45:41.493 INFO:teuthology.orchestra.run.smithi204.stdout: NetworkManager-team-1:1.46.0-1.el9.x86_64 2024-03-08T20:45:41.493 INFO:teuthology.orchestra.run.smithi204.stdout: NetworkManager-tui-1:1.46.0-1.el9.x86_64 2024-03-08T20:45:41.493 INFO:teuthology.orchestra.run.smithi204.stdout: acl-2.3.1-4.el9.x86_64 2024-03-08T20:45:41.493 INFO:teuthology.orchestra.run.smithi204.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-03-08T20:45:41.493 INFO:teuthology.orchestra.run.smithi204.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-03-08T20:45:41.493 INFO:teuthology.orchestra.run.smithi204.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-03-08T20:45:41.493 INFO:teuthology.orchestra.run.smithi204.stdout: libacl-2.3.1-4.el9.x86_64 2024-03-08T20:45:41.493 INFO:teuthology.orchestra.run.smithi204.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-03-08T20:45:41.494 INFO:teuthology.orchestra.run.smithi204.stdout: libblkid-2.37.4-18.el9.x86_64 2024-03-08T20:45:41.494 INFO:teuthology.orchestra.run.smithi204.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-03-08T20:45:41.494 INFO:teuthology.orchestra.run.smithi204.stdout: libmount-2.37.4-18.el9.x86_64 2024-03-08T20:45:41.494 INFO:teuthology.orchestra.run.smithi204.stdout: librados2-2:19.0.0-1965.g7d63482a.el9.x86_64 2024-03-08T20:45:41.494 INFO:teuthology.orchestra.run.smithi204.stdout: librbd1-2:19.0.0-1965.g7d63482a.el9.x86_64 2024-03-08T20:45:41.494 INFO:teuthology.orchestra.run.smithi204.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-03-08T20:45:41.494 INFO:teuthology.orchestra.run.smithi204.stdout: libuuid-2.37.4-18.el9.x86_64 2024-03-08T20:45:41.494 INFO:teuthology.orchestra.run.smithi204.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-03-08T20:45:41.494 INFO:teuthology.orchestra.run.smithi204.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-03-08T20:45:41.494 INFO:teuthology.orchestra.run.smithi204.stdout: util-linux-2.37.4-18.el9.x86_64 2024-03-08T20:45:41.494 INFO:teuthology.orchestra.run.smithi204.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-03-08T20:45:41.494 INFO:teuthology.orchestra.run.smithi204.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-03-08T20:45:41.494 INFO:teuthology.orchestra.run.smithi204.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-03-08T20:45:41.495 INFO:teuthology.orchestra.run.smithi204.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-03-08T20:45:41.495 INFO:teuthology.orchestra.run.smithi204.stdout:Installed: 2024-03-08T20:45:41.495 INFO:teuthology.orchestra.run.smithi204.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-03-08T20:45:41.495 INFO:teuthology.orchestra.run.smithi204.stdout: automake-1.16.2-8.el9.noarch 2024-03-08T20:45:41.495 INFO:teuthology.orchestra.run.smithi204.stdout: bison-3.7.4-5.el9.x86_64 2024-03-08T20:45:41.495 INFO:teuthology.orchestra.run.smithi204.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-03-08T20:45:41.495 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-2:19.0.0-1965.g7d63482a.el9.x86_64 2024-03-08T20:45:41.495 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-base-2:19.0.0-1965.g7d63482a.el9.x86_64 2024-03-08T20:45:41.495 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-common-2:19.0.0-1965.g7d63482a.el9.x86_64 2024-03-08T20:45:41.495 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-fuse-2:19.0.0-1965.g7d63482a.el9.x86_64 2024-03-08T20:45:41.495 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-grafana-dashboards-2:19.0.0-1965.g7d63482a.el9.noarch 2024-03-08T20:45:41.495 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-immutable-object-cache-2:19.0.0-1965.g7d63482a.el9.x86_64 2024-03-08T20:45:41.495 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mds-2:19.0.0-1965.g7d63482a.el9.x86_64 2024-03-08T20:45:41.496 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-2:19.0.0-1965.g7d63482a.el9.x86_64 2024-03-08T20:45:41.496 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-cephadm-2:19.0.0-1965.g7d63482a.el9.noarch 2024-03-08T20:45:41.496 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-dashboard-2:19.0.0-1965.g7d63482a.el9.noarch 2024-03-08T20:45:41.496 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-diskprediction-local-2:19.0.0-1965.g7d63482a.el9.noarch 2024-03-08T20:45:41.496 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-modules-core-2:19.0.0-1965.g7d63482a.el9.noarch 2024-03-08T20:45:41.496 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mgr-rook-2:19.0.0-1965.g7d63482a.el9.noarch 2024-03-08T20:45:41.496 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-mon-2:19.0.0-1965.g7d63482a.el9.x86_64 2024-03-08T20:45:41.496 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-osd-2:19.0.0-1965.g7d63482a.el9.x86_64 2024-03-08T20:45:41.496 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-prometheus-alerts-2:19.0.0-1965.g7d63482a.el9.noarch 2024-03-08T20:45:41.496 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-radosgw-2:19.0.0-1965.g7d63482a.el9.x86_64 2024-03-08T20:45:41.496 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-selinux-2:19.0.0-1965.g7d63482a.el9.x86_64 2024-03-08T20:45:41.496 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-test-2:19.0.0-1965.g7d63482a.el9.x86_64 2024-03-08T20:45:41.496 INFO:teuthology.orchestra.run.smithi204.stdout: ceph-volume-2:19.0.0-1965.g7d63482a.el9.noarch 2024-03-08T20:45:41.497 INFO:teuthology.orchestra.run.smithi204.stdout: cephadm-2:19.0.0-1965.g7d63482a.el9.noarch 2024-03-08T20:45:41.497 INFO:teuthology.orchestra.run.smithi204.stdout: cephfs-mirror-2:19.0.0-1965.g7d63482a.el9.x86_64 2024-03-08T20:45:41.497 INFO:teuthology.orchestra.run.smithi204.stdout: cephfs-shell-2:19.0.0-1965.g7d63482a.el9.x86_64 2024-03-08T20:45:41.497 INFO:teuthology.orchestra.run.smithi204.stdout: cephfs-top-2:19.0.0-1965.g7d63482a.el9.noarch 2024-03-08T20:45:41.497 INFO:teuthology.orchestra.run.smithi204.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-03-08T20:45:41.497 INFO:teuthology.orchestra.run.smithi204.stdout: flex-2.6.4-9.el9.x86_64 2024-03-08T20:45:41.497 INFO:teuthology.orchestra.run.smithi204.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-03-08T20:45:41.497 INFO:teuthology.orchestra.run.smithi204.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-03-08T20:45:41.497 INFO:teuthology.orchestra.run.smithi204.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-03-08T20:45:41.497 INFO:teuthology.orchestra.run.smithi204.stdout: fuse-2.9.9-15.el9.x86_64 2024-03-08T20:45:41.497 INFO:teuthology.orchestra.run.smithi204.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-03-08T20:45:41.497 INFO:teuthology.orchestra.run.smithi204.stdout: grpc-data-1.46.7-10.el9.noarch 2024-03-08T20:45:41.497 INFO:teuthology.orchestra.run.smithi204.stdout: jq-1.6-16.el9.x86_64 2024-03-08T20:45:41.497 INFO:teuthology.orchestra.run.smithi204.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-03-08T20:45:41.498 INFO:teuthology.orchestra.run.smithi204.stdout: libarrow-9.0.0-10.el9.x86_64 2024-03-08T20:45:41.498 INFO:teuthology.orchestra.run.smithi204.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-03-08T20:45:41.498 INFO:teuthology.orchestra.run.smithi204.stdout: libcephfs-devel-2:19.0.0-1965.g7d63482a.el9.x86_64 2024-03-08T20:45:41.498 INFO:teuthology.orchestra.run.smithi204.stdout: libcephfs2-2:19.0.0-1965.g7d63482a.el9.x86_64 2024-03-08T20:45:41.498 INFO:teuthology.orchestra.run.smithi204.stdout: libcephsqlite-2:19.0.0-1965.g7d63482a.el9.x86_64 2024-03-08T20:45:41.498 INFO:teuthology.orchestra.run.smithi204.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-03-08T20:45:41.498 INFO:teuthology.orchestra.run.smithi204.stdout: liboath-2.6.7-2.el9.x86_64 2024-03-08T20:45:41.498 INFO:teuthology.orchestra.run.smithi204.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-03-08T20:45:41.498 INFO:teuthology.orchestra.run.smithi204.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-03-08T20:45:41.498 INFO:teuthology.orchestra.run.smithi204.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-03-08T20:45:41.498 INFO:teuthology.orchestra.run.smithi204.stdout: librados-devel-2:19.0.0-1965.g7d63482a.el9.x86_64 2024-03-08T20:45:41.498 INFO:teuthology.orchestra.run.smithi204.stdout: libradosstriper1-2:19.0.0-1965.g7d63482a.el9.x86_64 2024-03-08T20:45:41.498 INFO:teuthology.orchestra.run.smithi204.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-03-08T20:45:41.499 INFO:teuthology.orchestra.run.smithi204.stdout: librgw2-2:19.0.0-1965.g7d63482a.el9.x86_64 2024-03-08T20:45:41.499 INFO:teuthology.orchestra.run.smithi204.stdout: libtool-2.4.6-45.el9.x86_64 2024-03-08T20:45:41.499 INFO:teuthology.orchestra.run.smithi204.stdout: libunwind-1.6.2-1.el9.x86_64 2024-03-08T20:45:41.499 INFO:teuthology.orchestra.run.smithi204.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-03-08T20:45:41.499 INFO:teuthology.orchestra.run.smithi204.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-03-08T20:45:41.499 INFO:teuthology.orchestra.run.smithi204.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-03-08T20:45:41.499 INFO:teuthology.orchestra.run.smithi204.stdout: lua-5.4.4-4.el9.x86_64 2024-03-08T20:45:41.499 INFO:teuthology.orchestra.run.smithi204.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-03-08T20:45:41.499 INFO:teuthology.orchestra.run.smithi204.stdout: luarocks-3.9.2-1.el9.noarch 2024-03-08T20:45:41.499 INFO:teuthology.orchestra.run.smithi204.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-03-08T20:45:41.499 INFO:teuthology.orchestra.run.smithi204.stdout: openblas-0.3.21-2.el9.x86_64 2024-03-08T20:45:41.499 INFO:teuthology.orchestra.run.smithi204.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-03-08T20:45:41.499 INFO:teuthology.orchestra.run.smithi204.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-03-08T20:45:41.500 INFO:teuthology.orchestra.run.smithi204.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-03-08T20:45:41.500 INFO:teuthology.orchestra.run.smithi204.stdout: postgresql-13.11-1.el9.x86_64 2024-03-08T20:45:41.500 INFO:teuthology.orchestra.run.smithi204.stdout: postgresql-contrib-13.11-1.el9.x86_64 2024-03-08T20:45:41.500 INFO:teuthology.orchestra.run.smithi204.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2024-03-08T20:45:41.500 INFO:teuthology.orchestra.run.smithi204.stdout: postgresql-server-13.11-1.el9.x86_64 2024-03-08T20:45:41.500 INFO:teuthology.orchestra.run.smithi204.stdout: protobuf-3.14.0-13.el9.x86_64 2024-03-08T20:45:41.500 INFO:teuthology.orchestra.run.smithi204.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-03-08T20:45:41.500 INFO:teuthology.orchestra.run.smithi204.stdout: python3-asyncssh-2.13.2-4.el9.noarch 2024-03-08T20:45:41.500 INFO:teuthology.orchestra.run.smithi204.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-03-08T20:45:41.500 INFO:teuthology.orchestra.run.smithi204.stdout: python3-babel-2.9.1-2.el9.noarch 2024-03-08T20:45:41.500 INFO:teuthology.orchestra.run.smithi204.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-03-08T20:45:41.500 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-03-08T20:45:41.500 INFO:teuthology.orchestra.run.smithi204.stdout: python3-ceph-argparse-2:19.0.0-1965.g7d63482a.el9.x86_64 2024-03-08T20:45:41.500 INFO:teuthology.orchestra.run.smithi204.stdout: python3-ceph-common-2:19.0.0-1965.g7d63482a.el9.x86_64 2024-03-08T20:45:41.501 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cephfs-2:19.0.0-1965.g7d63482a.el9.x86_64 2024-03-08T20:45:41.501 INFO:teuthology.orchestra.run.smithi204.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-03-08T20:45:41.501 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-03-08T20:45:41.501 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-03-08T20:45:41.501 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-03-08T20:45:41.501 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-03-08T20:45:41.501 INFO:teuthology.orchestra.run.smithi204.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-03-08T20:45:41.501 INFO:teuthology.orchestra.run.smithi204.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-03-08T20:45:41.501 INFO:teuthology.orchestra.run.smithi204.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-03-08T20:45:41.501 INFO:teuthology.orchestra.run.smithi204.stdout: python3-google-auth-1:2.28.0-1.el9.noarch 2024-03-08T20:45:41.501 INFO:teuthology.orchestra.run.smithi204.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-03-08T20:45:41.501 INFO:teuthology.orchestra.run.smithi204.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-03-08T20:45:41.501 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-03-08T20:45:41.501 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-03-08T20:45:41.502 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-03-08T20:45:41.502 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-03-08T20:45:41.502 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-03-08T20:45:41.502 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-03-08T20:45:41.502 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-03-08T20:45:41.502 INFO:teuthology.orchestra.run.smithi204.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-03-08T20:45:41.502 INFO:teuthology.orchestra.run.smithi204.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-03-08T20:45:41.502 INFO:teuthology.orchestra.run.smithi204.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-03-08T20:45:41.502 INFO:teuthology.orchestra.run.smithi204.stdout: python3-mako-1.1.4-6.el9.noarch 2024-03-08T20:45:41.502 INFO:teuthology.orchestra.run.smithi204.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-03-08T20:45:41.502 INFO:teuthology.orchestra.run.smithi204.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-03-08T20:45:41.502 INFO:teuthology.orchestra.run.smithi204.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-03-08T20:45:41.502 INFO:teuthology.orchestra.run.smithi204.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-03-08T20:45:41.503 INFO:teuthology.orchestra.run.smithi204.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-03-08T20:45:41.503 INFO:teuthology.orchestra.run.smithi204.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-03-08T20:45:41.503 INFO:teuthology.orchestra.run.smithi204.stdout: python3-packaging-20.9-5.el9.noarch 2024-03-08T20:45:41.503 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-03-08T20:45:41.503 INFO:teuthology.orchestra.run.smithi204.stdout: python3-ply-3.11-14.el9.noarch 2024-03-08T20:45:41.503 INFO:teuthology.orchestra.run.smithi204.stdout: python3-portend-3.1.0-2.el9.noarch 2024-03-08T20:45:41.503 INFO:teuthology.orchestra.run.smithi204.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-03-08T20:45:41.503 INFO:teuthology.orchestra.run.smithi204.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-03-08T20:45:41.503 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-03-08T20:45:41.503 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-03-08T20:45:41.503 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-03-08T20:45:41.503 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pycparser-2.20-6.el9.noarch 2024-03-08T20:45:41.503 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-03-08T20:45:41.503 INFO:teuthology.orchestra.run.smithi204.stdout: python3-pytz-2021.1-5.el9.noarch 2024-03-08T20:45:41.504 INFO:teuthology.orchestra.run.smithi204.stdout: python3-rados-2:19.0.0-1965.g7d63482a.el9.x86_64 2024-03-08T20:45:41.504 INFO:teuthology.orchestra.run.smithi204.stdout: python3-rbd-2:19.0.0-1965.g7d63482a.el9.x86_64 2024-03-08T20:45:41.504 INFO:teuthology.orchestra.run.smithi204.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-03-08T20:45:41.504 INFO:teuthology.orchestra.run.smithi204.stdout: python3-requests-2.25.1-8.el9.noarch 2024-03-08T20:45:41.504 INFO:teuthology.orchestra.run.smithi204.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-03-08T20:45:41.504 INFO:teuthology.orchestra.run.smithi204.stdout: python3-rgw-2:19.0.0-1965.g7d63482a.el9.x86_64 2024-03-08T20:45:41.504 INFO:teuthology.orchestra.run.smithi204.stdout: python3-routes-2.5.1-5.el9.noarch 2024-03-08T20:45:41.504 INFO:teuthology.orchestra.run.smithi204.stdout: python3-rsa-4.9-2.el9.noarch 2024-03-08T20:45:41.504 INFO:teuthology.orchestra.run.smithi204.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-03-08T20:45:41.504 INFO:teuthology.orchestra.run.smithi204.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-03-08T20:45:41.504 INFO:teuthology.orchestra.run.smithi204.stdout: python3-toml-0.10.2-6.el9.noarch 2024-03-08T20:45:41.504 INFO:teuthology.orchestra.run.smithi204.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-03-08T20:45:41.504 INFO:teuthology.orchestra.run.smithi204.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-03-08T20:45:41.505 INFO:teuthology.orchestra.run.smithi204.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-03-08T20:45:41.505 INFO:teuthology.orchestra.run.smithi204.stdout: python3-webob-1.8.7-6.el9.noarch 2024-03-08T20:45:41.505 INFO:teuthology.orchestra.run.smithi204.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-03-08T20:45:41.505 INFO:teuthology.orchestra.run.smithi204.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-03-08T20:45:41.505 INFO:teuthology.orchestra.run.smithi204.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-03-08T20:45:41.505 INFO:teuthology.orchestra.run.smithi204.stdout: qatlib-23.11.0-1.el9.x86_64 2024-03-08T20:45:41.505 INFO:teuthology.orchestra.run.smithi204.stdout: qatlib-service-23.11.0-1.el9.x86_64 2024-03-08T20:45:41.505 INFO:teuthology.orchestra.run.smithi204.stdout: qatzip-libs-1.1.2-2.el9.x86_64 2024-03-08T20:45:41.505 INFO:teuthology.orchestra.run.smithi204.stdout: rbd-fuse-2:19.0.0-1965.g7d63482a.el9.x86_64 2024-03-08T20:45:41.505 INFO:teuthology.orchestra.run.smithi204.stdout: rbd-mirror-2:19.0.0-1965.g7d63482a.el9.x86_64 2024-03-08T20:45:41.505 INFO:teuthology.orchestra.run.smithi204.stdout: rbd-nbd-2:19.0.0-1965.g7d63482a.el9.x86_64 2024-03-08T20:45:41.505 INFO:teuthology.orchestra.run.smithi204.stdout: re2-1:20211101-3.el9.x86_64 2024-03-08T20:45:41.505 INFO:teuthology.orchestra.run.smithi204.stdout: socat-1.7.4.1-5.el9.x86_64 2024-03-08T20:45:41.505 INFO:teuthology.orchestra.run.smithi204.stdout: thrift-0.15.0-2.el9.x86_64 2024-03-08T20:45:41.505 INFO:teuthology.orchestra.run.smithi204.stdout: uuid-1.6.2-55.el9.x86_64 2024-03-08T20:45:41.506 INFO:teuthology.orchestra.run.smithi204.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-03-08T20:45:41.506 INFO:teuthology.orchestra.run.smithi204.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-03-08T20:45:41.506 INFO:teuthology.orchestra.run.smithi204.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-03-08T20:45:41.506 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-03-08T20:45:41.506 INFO:teuthology.orchestra.run.smithi204.stdout:Complete! 2024-03-08T20:45:41.521 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : cephfs-top-2:19.0.0-1965.g7d63482a.el9.noarch 170/198 2024-03-08T20:45:41.615 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 171/198 2024-03-08T20:45:41.662 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 172/198 2024-03-08T20:45:41.794 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : flex-2.6.4-9.el9.x86_64 173/198 2024-03-08T20:45:41.835 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : bison-3.7.4-5.el9.x86_64 174/198 2024-03-08T20:45:41.868 INFO:teuthology.orchestra.run.smithi107.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 175/198 2024-03-08T20:45:41.902 INFO:teuthology.orchestra.run.smithi107.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 176/198 2024-03-08T20:45:41.944 INFO:teuthology.orchestra.run.smithi107.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 177/198 2024-03-08T20:45:41.945 INFO:teuthology.orchestra.run.smithi107.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-08T20:45:42.090 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-08T20:45:42.131 INFO:teuthology.orchestra.run.smithi107.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 179/198 2024-03-08T20:45:42.131 INFO:teuthology.orchestra.run.smithi107.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-08T20:45:42.153 DEBUG:teuthology.parallel:result is None 2024-03-08T20:45:42.253 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-08T20:45:42.336 INFO:teuthology.orchestra.run.smithi107.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 181/198 2024-03-08T20:45:42.384 INFO:teuthology.orchestra.run.smithi107.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 182/198 2024-03-08T20:45:42.418 INFO:teuthology.orchestra.run.smithi107.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 183/198 2024-03-08T20:45:42.451 INFO:teuthology.orchestra.run.smithi107.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 184/198 2024-03-08T20:45:42.524 INFO:teuthology.orchestra.run.smithi107.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 185/198 2024-03-08T20:45:42.600 INFO:teuthology.orchestra.run.smithi107.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 186/198 2024-03-08T20:45:42.655 INFO:teuthology.orchestra.run.smithi107.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 187/198 2024-03-08T20:45:42.689 INFO:teuthology.orchestra.run.smithi107.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 188/198 2024-03-08T20:45:42.723 INFO:teuthology.orchestra.run.smithi107.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 189/198 2024-03-08T20:45:42.756 INFO:teuthology.orchestra.run.smithi107.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 190/198 2024-03-08T20:45:42.790 INFO:teuthology.orchestra.run.smithi107.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 191/198 2024-03-08T20:45:42.826 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-08T20:45:42.826 INFO:teuthology.orchestra.run.smithi107.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-08T20:45:43.150 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-08T20:45:43.151 INFO:teuthology.orchestra.run.smithi107.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-08T20:45:43.289 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-08T20:45:43.320 INFO:teuthology.orchestra.run.smithi107.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 194/198 2024-03-08T20:45:43.353 INFO:teuthology.orchestra.run.smithi107.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 195/198 2024-03-08T20:45:43.411 INFO:teuthology.orchestra.run.smithi107.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 196/198 2024-03-08T20:45:43.446 INFO:teuthology.orchestra.run.smithi107.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 197/198 2024-03-08T20:45:43.573 INFO:teuthology.orchestra.run.smithi107.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-08T20:45:43.825 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-08T20:45:43.825 INFO:teuthology.orchestra.run.smithi107.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-03-08T20:45:43.826 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-08T20:45:48.834 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-2:19.0.0-1965.g7d63482a.el9.x86_64 1/198 2024-03-08T20:45:48.834 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-base-2:19.0.0-1965.g7d63482a.el9.x86_64 2/198 2024-03-08T20:45:48.834 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-common-2:19.0.0-1965.g7d63482a.el9.x86_64 3/198 2024-03-08T20:45:48.834 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-fuse-2:19.0.0-1965.g7d63482a.el9.x86_64 4/198 2024-03-08T20:45:48.834 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-1965.g7d634 5/198 2024-03-08T20:45:48.835 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-mds-2:19.0.0-1965.g7d63482a.el9.x86_64 6/198 2024-03-08T20:45:48.835 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-mgr-2:19.0.0-1965.g7d63482a.el9.x86_64 7/198 2024-03-08T20:45:48.835 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-mon-2:19.0.0-1965.g7d63482a.el9.x86_64 8/198 2024-03-08T20:45:48.835 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-osd-2:19.0.0-1965.g7d63482a.el9.x86_64 9/198 2024-03-08T20:45:48.835 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-radosgw-2:19.0.0-1965.g7d63482a.el9.x86_64 10/198 2024-03-08T20:45:48.835 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-selinux-2:19.0.0-1965.g7d63482a.el9.x86_64 11/198 2024-03-08T20:45:48.835 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-test-2:19.0.0-1965.g7d63482a.el9.x86_64 12/198 2024-03-08T20:45:48.835 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : cephfs-mirror-2:19.0.0-1965.g7d63482a.el9.x86_64 13/198 2024-03-08T20:45:48.835 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : cephfs-shell-2:19.0.0-1965.g7d63482a.el9.x86_64 14/198 2024-03-08T20:45:48.835 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libcephfs-devel-2:19.0.0-1965.g7d63482a.el9.x86_ 15/198 2024-03-08T20:45:48.835 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libcephfs2-2:19.0.0-1965.g7d63482a.el9.x86_64 16/198 2024-03-08T20:45:48.835 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libcephsqlite-2:19.0.0-1965.g7d63482a.el9.x86_64 17/198 2024-03-08T20:45:48.835 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : librados-devel-2:19.0.0-1965.g7d63482a.el9.x86_6 18/198 2024-03-08T20:45:48.836 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libradosstriper1-2:19.0.0-1965.g7d63482a.el9.x86 19/198 2024-03-08T20:45:48.836 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : librgw2-2:19.0.0-1965.g7d63482a.el9.x86_64 20/198 2024-03-08T20:45:48.836 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-ceph-argparse-2:19.0.0-1965.g7d63482a.el 21/198 2024-03-08T20:45:48.836 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-ceph-common-2:19.0.0-1965.g7d63482a.el9. 22/198 2024-03-08T20:45:48.836 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-cephfs-2:19.0.0-1965.g7d63482a.el9.x86_6 23/198 2024-03-08T20:45:48.836 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-rados-2:19.0.0-1965.g7d63482a.el9.x86_64 24/198 2024-03-08T20:45:48.836 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-rbd-2:19.0.0-1965.g7d63482a.el9.x86_64 25/198 2024-03-08T20:45:48.836 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-rgw-2:19.0.0-1965.g7d63482a.el9.x86_64 26/198 2024-03-08T20:45:48.836 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : rbd-fuse-2:19.0.0-1965.g7d63482a.el9.x86_64 27/198 2024-03-08T20:45:48.854 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : rbd-mirror-2:19.0.0-1965.g7d63482a.el9.x86_64 28/198 2024-03-08T20:45:48.855 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : rbd-nbd-2:19.0.0-1965.g7d63482a.el9.x86_64 29/198 2024-03-08T20:45:48.855 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-1965.g7d63482a. 30/198 2024-03-08T20:45:48.855 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-1965.g7d63482a.el9.noa 31/198 2024-03-08T20:45:48.855 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-1965.g7d63482a.el9.n 32/198 2024-03-08T20:45:48.855 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-1965.g7d6 33/198 2024-03-08T20:45:48.855 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-1965.g7d63482a.el 34/198 2024-03-08T20:45:48.856 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-mgr-rook-2:19.0.0-1965.g7d63482a.el9.noarch 35/198 2024-03-08T20:45:48.856 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-1965.g7d63482a.e 36/198 2024-03-08T20:45:48.856 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-volume-2:19.0.0-1965.g7d63482a.el9.noarch 37/198 2024-03-08T20:45:48.856 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : cephadm-2:19.0.0-1965.g7d63482a.el9.noarch 38/198 2024-03-08T20:45:48.856 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : cephfs-top-2:19.0.0-1965.g7d63482a.el9.noarch 39/198 2024-03-08T20:45:48.856 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/198 2024-03-08T20:45:48.856 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/198 2024-03-08T20:45:48.856 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/198 2024-03-08T20:45:48.857 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/198 2024-03-08T20:45:48.857 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/198 2024-03-08T20:45:48.857 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/198 2024-03-08T20:45:48.857 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/198 2024-03-08T20:45:48.857 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 47/198 2024-03-08T20:45:48.857 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 48/198 2024-03-08T20:45:48.857 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 49/198 2024-03-08T20:45:48.857 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 50/198 2024-03-08T20:45:48.857 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 51/198 2024-03-08T20:45:48.858 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 52/198 2024-03-08T20:45:48.858 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 53/198 2024-03-08T20:45:48.858 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : lua-5.4.4-4.el9.x86_64 54/198 2024-03-08T20:45:48.858 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 55/198 2024-03-08T20:45:48.858 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 56/198 2024-03-08T20:45:48.858 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 57/198 2024-03-08T20:45:48.858 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : postgresql-13.11-1.el9.x86_64 58/198 2024-03-08T20:45:48.858 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 59/198 2024-03-08T20:45:48.859 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 60/198 2024-03-08T20:45:48.859 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 61/198 2024-03-08T20:45:48.859 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 62/198 2024-03-08T20:45:48.859 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 63/198 2024-03-08T20:45:48.859 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 64/198 2024-03-08T20:45:48.859 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 65/198 2024-03-08T20:45:48.859 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 66/198 2024-03-08T20:45:48.859 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 67/198 2024-03-08T20:45:48.860 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 68/198 2024-03-08T20:45:48.860 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 69/198 2024-03-08T20:45:48.860 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 70/198 2024-03-08T20:45:48.860 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 71/198 2024-03-08T20:45:48.860 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 72/198 2024-03-08T20:45:48.860 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 73/198 2024-03-08T20:45:48.860 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 74/198 2024-03-08T20:45:48.860 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 75/198 2024-03-08T20:45:48.861 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 76/198 2024-03-08T20:45:48.861 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 77/198 2024-03-08T20:45:48.861 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 78/198 2024-03-08T20:45:48.861 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 79/198 2024-03-08T20:45:48.861 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 80/198 2024-03-08T20:45:48.861 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 81/198 2024-03-08T20:45:48.861 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 82/198 2024-03-08T20:45:48.861 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 83/198 2024-03-08T20:45:48.861 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 84/198 2024-03-08T20:45:48.861 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 85/198 2024-03-08T20:45:48.861 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 86/198 2024-03-08T20:45:48.861 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 87/198 2024-03-08T20:45:48.862 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 88/198 2024-03-08T20:45:48.862 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-ply-3.11-14.el9.noarch 89/198 2024-03-08T20:45:48.862 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 90/198 2024-03-08T20:45:48.862 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 91/198 2024-03-08T20:45:48.862 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : jq-1.6-16.el9.x86_64 92/198 2024-03-08T20:45:48.862 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 93/198 2024-03-08T20:45:48.862 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 94/198 2024-03-08T20:45:48.862 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 95/198 2024-03-08T20:45:48.862 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 96/198 2024-03-08T20:45:48.862 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 97/198 2024-03-08T20:45:48.862 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 98/198 2024-03-08T20:45:48.862 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 99/198 2024-03-08T20:45:48.862 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 100/198 2024-03-08T20:45:48.862 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : qatlib-23.11.0-1.el9.x86_64 101/198 2024-03-08T20:45:48.863 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : qatlib-service-23.11.0-1.el9.x86_64 102/198 2024-03-08T20:45:48.863 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : qatzip-libs-1.1.2-2.el9.x86_64 103/198 2024-03-08T20:45:48.863 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 104/198 2024-03-08T20:45:48.863 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/198 2024-03-08T20:45:48.863 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/198 2024-03-08T20:45:48.863 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/198 2024-03-08T20:45:48.863 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/198 2024-03-08T20:45:48.863 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/198 2024-03-08T20:45:48.863 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/198 2024-03-08T20:45:48.863 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/198 2024-03-08T20:45:48.863 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/198 2024-03-08T20:45:48.863 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/198 2024-03-08T20:45:48.863 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/198 2024-03-08T20:45:48.864 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-03-08T20:45:48.864 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-asyncssh-2.13.2-4.el9.noarch 116/198 2024-03-08T20:45:48.864 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/198 2024-03-08T20:45:48.864 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/198 2024-03-08T20:45:48.864 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/198 2024-03-08T20:45:48.864 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/198 2024-03-08T20:45:48.864 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/198 2024-03-08T20:45:48.864 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/198 2024-03-08T20:45:48.864 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/198 2024-03-08T20:45:48.864 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-google-auth-1:2.28.0-1.el9.noarch 124/198 2024-03-08T20:45:48.864 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/198 2024-03-08T20:45:48.864 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/198 2024-03-08T20:45:48.864 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/198 2024-03-08T20:45:48.864 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/198 2024-03-08T20:45:48.864 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/198 2024-03-08T20:45:48.865 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/198 2024-03-08T20:45:48.865 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/198 2024-03-08T20:45:48.865 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/198 2024-03-08T20:45:48.865 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/198 2024-03-08T20:45:48.865 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/198 2024-03-08T20:45:48.865 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/198 2024-03-08T20:45:48.865 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/198 2024-03-08T20:45:48.865 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/198 2024-03-08T20:45:48.865 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/198 2024-03-08T20:45:48.865 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/198 2024-03-08T20:45:48.865 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/198 2024-03-08T20:45:48.865 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/198 2024-03-08T20:45:48.865 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/198 2024-03-08T20:45:48.866 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/198 2024-03-08T20:45:48.866 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/198 2024-03-08T20:45:48.866 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/198 2024-03-08T20:45:48.866 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/198 2024-03-08T20:45:48.866 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/198 2024-03-08T20:45:48.866 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/198 2024-03-08T20:45:48.866 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/198 2024-03-08T20:45:48.866 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/198 2024-03-08T20:45:48.866 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : librados2-2:19.0.0-1965.g7d63482a.el9.x86_64 151/198 2024-03-08T20:45:48.866 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/198 2024-03-08T20:45:48.866 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : librbd1-2:19.0.0-1965.g7d63482a.el9.x86_64 153/198 2024-03-08T20:45:48.866 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/198 2024-03-08T20:45:48.866 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 155/198 2024-03-08T20:45:48.867 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 156/198 2024-03-08T20:45:48.867 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : NetworkManager-1:1.46.0-1.el9.x86_64 157/198 2024-03-08T20:45:48.867 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 158/198 2024-03-08T20:45:48.867 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : NetworkManager-libnm-1:1.46.0-1.el9.x86_64 159/198 2024-03-08T20:45:48.867 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 160/198 2024-03-08T20:45:48.867 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : NetworkManager-team-1:1.46.0-1.el9.x86_64 161/198 2024-03-08T20:45:48.867 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 162/198 2024-03-08T20:45:48.867 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : NetworkManager-tui-1:1.46.0-1.el9.x86_64 163/198 2024-03-08T20:45:48.867 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 164/198 2024-03-08T20:45:48.867 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : acl-2.3.1-4.el9.x86_64 165/198 2024-03-08T20:45:48.867 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : acl-2.3.1-3.el9.x86_64 166/198 2024-03-08T20:45:48.867 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 167/198 2024-03-08T20:45:48.867 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/198 2024-03-08T20:45:48.867 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 169/198 2024-03-08T20:45:48.868 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 170/198 2024-03-08T20:45:48.868 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 171/198 2024-03-08T20:45:48.868 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 172/198 2024-03-08T20:45:48.868 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 173/198 2024-03-08T20:45:48.868 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 174/198 2024-03-08T20:45:48.868 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 175/198 2024-03-08T20:45:48.868 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 176/198 2024-03-08T20:45:48.868 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 177/198 2024-03-08T20:45:48.868 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 178/198 2024-03-08T20:45:48.868 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 179/198 2024-03-08T20:45:48.868 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 180/198 2024-03-08T20:45:48.868 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 181/198 2024-03-08T20:45:48.868 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 182/198 2024-03-08T20:45:48.868 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 183/198 2024-03-08T20:45:48.869 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 184/198 2024-03-08T20:45:48.869 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 185/198 2024-03-08T20:45:48.869 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 186/198 2024-03-08T20:45:48.869 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 187/198 2024-03-08T20:45:48.869 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 188/198 2024-03-08T20:45:48.869 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 189/198 2024-03-08T20:45:48.869 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 190/198 2024-03-08T20:45:48.869 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 191/198 2024-03-08T20:45:48.869 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 192/198 2024-03-08T20:45:48.869 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 193/198 2024-03-08T20:45:48.869 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 194/198 2024-03-08T20:45:48.869 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 195/198 2024-03-08T20:45:48.870 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 196/198 2024-03-08T20:45:48.870 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 197/198 2024-03-08T20:45:51.291 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 198/198 2024-03-08T20:45:51.292 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-08T20:45:51.292 INFO:teuthology.orchestra.run.smithi107.stdout:Upgraded: 2024-03-08T20:45:51.292 INFO:teuthology.orchestra.run.smithi107.stdout: NetworkManager-1:1.46.0-1.el9.x86_64 2024-03-08T20:45:51.292 INFO:teuthology.orchestra.run.smithi107.stdout: NetworkManager-libnm-1:1.46.0-1.el9.x86_64 2024-03-08T20:45:51.292 INFO:teuthology.orchestra.run.smithi107.stdout: NetworkManager-team-1:1.46.0-1.el9.x86_64 2024-03-08T20:45:51.292 INFO:teuthology.orchestra.run.smithi107.stdout: NetworkManager-tui-1:1.46.0-1.el9.x86_64 2024-03-08T20:45:51.292 INFO:teuthology.orchestra.run.smithi107.stdout: acl-2.3.1-4.el9.x86_64 2024-03-08T20:45:51.292 INFO:teuthology.orchestra.run.smithi107.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-03-08T20:45:51.292 INFO:teuthology.orchestra.run.smithi107.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-03-08T20:45:51.292 INFO:teuthology.orchestra.run.smithi107.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-03-08T20:45:51.293 INFO:teuthology.orchestra.run.smithi107.stdout: libacl-2.3.1-4.el9.x86_64 2024-03-08T20:45:51.293 INFO:teuthology.orchestra.run.smithi107.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-03-08T20:45:51.293 INFO:teuthology.orchestra.run.smithi107.stdout: libblkid-2.37.4-18.el9.x86_64 2024-03-08T20:45:51.293 INFO:teuthology.orchestra.run.smithi107.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-03-08T20:45:51.293 INFO:teuthology.orchestra.run.smithi107.stdout: libmount-2.37.4-18.el9.x86_64 2024-03-08T20:45:51.293 INFO:teuthology.orchestra.run.smithi107.stdout: librados2-2:19.0.0-1965.g7d63482a.el9.x86_64 2024-03-08T20:45:51.293 INFO:teuthology.orchestra.run.smithi107.stdout: librbd1-2:19.0.0-1965.g7d63482a.el9.x86_64 2024-03-08T20:45:51.293 INFO:teuthology.orchestra.run.smithi107.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-03-08T20:45:51.293 INFO:teuthology.orchestra.run.smithi107.stdout: libuuid-2.37.4-18.el9.x86_64 2024-03-08T20:45:51.293 INFO:teuthology.orchestra.run.smithi107.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-03-08T20:45:51.293 INFO:teuthology.orchestra.run.smithi107.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-03-08T20:45:51.293 INFO:teuthology.orchestra.run.smithi107.stdout: util-linux-2.37.4-18.el9.x86_64 2024-03-08T20:45:51.293 INFO:teuthology.orchestra.run.smithi107.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-03-08T20:45:51.294 INFO:teuthology.orchestra.run.smithi107.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-03-08T20:45:51.294 INFO:teuthology.orchestra.run.smithi107.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-03-08T20:45:51.294 INFO:teuthology.orchestra.run.smithi107.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-03-08T20:45:51.294 INFO:teuthology.orchestra.run.smithi107.stdout:Installed: 2024-03-08T20:45:51.294 INFO:teuthology.orchestra.run.smithi107.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-03-08T20:45:51.294 INFO:teuthology.orchestra.run.smithi107.stdout: automake-1.16.2-8.el9.noarch 2024-03-08T20:45:51.294 INFO:teuthology.orchestra.run.smithi107.stdout: bison-3.7.4-5.el9.x86_64 2024-03-08T20:45:51.294 INFO:teuthology.orchestra.run.smithi107.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-03-08T20:45:51.294 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-2:19.0.0-1965.g7d63482a.el9.x86_64 2024-03-08T20:45:51.294 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-base-2:19.0.0-1965.g7d63482a.el9.x86_64 2024-03-08T20:45:51.294 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-common-2:19.0.0-1965.g7d63482a.el9.x86_64 2024-03-08T20:45:51.294 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-fuse-2:19.0.0-1965.g7d63482a.el9.x86_64 2024-03-08T20:45:51.295 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-grafana-dashboards-2:19.0.0-1965.g7d63482a.el9.noarch 2024-03-08T20:45:51.295 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-immutable-object-cache-2:19.0.0-1965.g7d63482a.el9.x86_64 2024-03-08T20:45:51.295 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mds-2:19.0.0-1965.g7d63482a.el9.x86_64 2024-03-08T20:45:51.295 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-2:19.0.0-1965.g7d63482a.el9.x86_64 2024-03-08T20:45:51.295 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-cephadm-2:19.0.0-1965.g7d63482a.el9.noarch 2024-03-08T20:45:51.295 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-dashboard-2:19.0.0-1965.g7d63482a.el9.noarch 2024-03-08T20:45:51.295 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-diskprediction-local-2:19.0.0-1965.g7d63482a.el9.noarch 2024-03-08T20:45:51.295 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-modules-core-2:19.0.0-1965.g7d63482a.el9.noarch 2024-03-08T20:45:51.295 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-rook-2:19.0.0-1965.g7d63482a.el9.noarch 2024-03-08T20:45:51.295 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mon-2:19.0.0-1965.g7d63482a.el9.x86_64 2024-03-08T20:45:51.295 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-osd-2:19.0.0-1965.g7d63482a.el9.x86_64 2024-03-08T20:45:51.295 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-prometheus-alerts-2:19.0.0-1965.g7d63482a.el9.noarch 2024-03-08T20:45:51.295 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-radosgw-2:19.0.0-1965.g7d63482a.el9.x86_64 2024-03-08T20:45:51.296 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-selinux-2:19.0.0-1965.g7d63482a.el9.x86_64 2024-03-08T20:45:51.296 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-test-2:19.0.0-1965.g7d63482a.el9.x86_64 2024-03-08T20:45:51.296 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-volume-2:19.0.0-1965.g7d63482a.el9.noarch 2024-03-08T20:45:51.296 INFO:teuthology.orchestra.run.smithi107.stdout: cephadm-2:19.0.0-1965.g7d63482a.el9.noarch 2024-03-08T20:45:51.296 INFO:teuthology.orchestra.run.smithi107.stdout: cephfs-mirror-2:19.0.0-1965.g7d63482a.el9.x86_64 2024-03-08T20:45:51.296 INFO:teuthology.orchestra.run.smithi107.stdout: cephfs-shell-2:19.0.0-1965.g7d63482a.el9.x86_64 2024-03-08T20:45:51.296 INFO:teuthology.orchestra.run.smithi107.stdout: cephfs-top-2:19.0.0-1965.g7d63482a.el9.noarch 2024-03-08T20:45:51.296 INFO:teuthology.orchestra.run.smithi107.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-03-08T20:45:51.296 INFO:teuthology.orchestra.run.smithi107.stdout: flex-2.6.4-9.el9.x86_64 2024-03-08T20:45:51.296 INFO:teuthology.orchestra.run.smithi107.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-03-08T20:45:51.296 INFO:teuthology.orchestra.run.smithi107.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-03-08T20:45:51.296 INFO:teuthology.orchestra.run.smithi107.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-03-08T20:45:51.296 INFO:teuthology.orchestra.run.smithi107.stdout: fuse-2.9.9-15.el9.x86_64 2024-03-08T20:45:51.296 INFO:teuthology.orchestra.run.smithi107.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-03-08T20:45:51.297 INFO:teuthology.orchestra.run.smithi107.stdout: grpc-data-1.46.7-10.el9.noarch 2024-03-08T20:45:51.297 INFO:teuthology.orchestra.run.smithi107.stdout: jq-1.6-16.el9.x86_64 2024-03-08T20:45:51.297 INFO:teuthology.orchestra.run.smithi107.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-03-08T20:45:51.297 INFO:teuthology.orchestra.run.smithi107.stdout: libarrow-9.0.0-10.el9.x86_64 2024-03-08T20:45:51.297 INFO:teuthology.orchestra.run.smithi107.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-03-08T20:45:51.297 INFO:teuthology.orchestra.run.smithi107.stdout: libcephfs-devel-2:19.0.0-1965.g7d63482a.el9.x86_64 2024-03-08T20:45:51.297 INFO:teuthology.orchestra.run.smithi107.stdout: libcephfs2-2:19.0.0-1965.g7d63482a.el9.x86_64 2024-03-08T20:45:51.297 INFO:teuthology.orchestra.run.smithi107.stdout: libcephsqlite-2:19.0.0-1965.g7d63482a.el9.x86_64 2024-03-08T20:45:51.297 INFO:teuthology.orchestra.run.smithi107.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-03-08T20:45:51.297 INFO:teuthology.orchestra.run.smithi107.stdout: liboath-2.6.7-2.el9.x86_64 2024-03-08T20:45:51.297 INFO:teuthology.orchestra.run.smithi107.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-03-08T20:45:51.297 INFO:teuthology.orchestra.run.smithi107.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-03-08T20:45:51.297 INFO:teuthology.orchestra.run.smithi107.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-03-08T20:45:51.298 INFO:teuthology.orchestra.run.smithi107.stdout: librados-devel-2:19.0.0-1965.g7d63482a.el9.x86_64 2024-03-08T20:45:51.298 INFO:teuthology.orchestra.run.smithi107.stdout: libradosstriper1-2:19.0.0-1965.g7d63482a.el9.x86_64 2024-03-08T20:45:51.298 INFO:teuthology.orchestra.run.smithi107.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-03-08T20:45:51.298 INFO:teuthology.orchestra.run.smithi107.stdout: librgw2-2:19.0.0-1965.g7d63482a.el9.x86_64 2024-03-08T20:45:51.298 INFO:teuthology.orchestra.run.smithi107.stdout: libtool-2.4.6-45.el9.x86_64 2024-03-08T20:45:51.298 INFO:teuthology.orchestra.run.smithi107.stdout: libunwind-1.6.2-1.el9.x86_64 2024-03-08T20:45:51.298 INFO:teuthology.orchestra.run.smithi107.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-03-08T20:45:51.298 INFO:teuthology.orchestra.run.smithi107.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-03-08T20:45:51.298 INFO:teuthology.orchestra.run.smithi107.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-03-08T20:45:51.298 INFO:teuthology.orchestra.run.smithi107.stdout: lua-5.4.4-4.el9.x86_64 2024-03-08T20:45:51.298 INFO:teuthology.orchestra.run.smithi107.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-03-08T20:45:51.298 INFO:teuthology.orchestra.run.smithi107.stdout: luarocks-3.9.2-1.el9.noarch 2024-03-08T20:45:51.299 INFO:teuthology.orchestra.run.smithi107.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-03-08T20:45:51.299 INFO:teuthology.orchestra.run.smithi107.stdout: openblas-0.3.21-2.el9.x86_64 2024-03-08T20:45:51.299 INFO:teuthology.orchestra.run.smithi107.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-03-08T20:45:51.299 INFO:teuthology.orchestra.run.smithi107.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-03-08T20:45:51.299 INFO:teuthology.orchestra.run.smithi107.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-03-08T20:45:51.299 INFO:teuthology.orchestra.run.smithi107.stdout: postgresql-13.11-1.el9.x86_64 2024-03-08T20:45:51.299 INFO:teuthology.orchestra.run.smithi107.stdout: postgresql-contrib-13.11-1.el9.x86_64 2024-03-08T20:45:51.299 INFO:teuthology.orchestra.run.smithi107.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2024-03-08T20:45:51.299 INFO:teuthology.orchestra.run.smithi107.stdout: postgresql-server-13.11-1.el9.x86_64 2024-03-08T20:45:51.299 INFO:teuthology.orchestra.run.smithi107.stdout: protobuf-3.14.0-13.el9.x86_64 2024-03-08T20:45:51.299 INFO:teuthology.orchestra.run.smithi107.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-03-08T20:45:51.299 INFO:teuthology.orchestra.run.smithi107.stdout: python3-asyncssh-2.13.2-4.el9.noarch 2024-03-08T20:45:51.299 INFO:teuthology.orchestra.run.smithi107.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-03-08T20:45:51.300 INFO:teuthology.orchestra.run.smithi107.stdout: python3-babel-2.9.1-2.el9.noarch 2024-03-08T20:45:51.300 INFO:teuthology.orchestra.run.smithi107.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-03-08T20:45:51.300 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-03-08T20:45:51.300 INFO:teuthology.orchestra.run.smithi107.stdout: python3-ceph-argparse-2:19.0.0-1965.g7d63482a.el9.x86_64 2024-03-08T20:45:51.300 INFO:teuthology.orchestra.run.smithi107.stdout: python3-ceph-common-2:19.0.0-1965.g7d63482a.el9.x86_64 2024-03-08T20:45:51.300 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cephfs-2:19.0.0-1965.g7d63482a.el9.x86_64 2024-03-08T20:45:51.300 INFO:teuthology.orchestra.run.smithi107.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-03-08T20:45:51.300 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-03-08T20:45:51.300 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-03-08T20:45:51.300 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-03-08T20:45:51.300 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-03-08T20:45:51.300 INFO:teuthology.orchestra.run.smithi107.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-03-08T20:45:51.301 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-03-08T20:45:51.301 INFO:teuthology.orchestra.run.smithi107.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-03-08T20:45:51.301 INFO:teuthology.orchestra.run.smithi107.stdout: python3-google-auth-1:2.28.0-1.el9.noarch 2024-03-08T20:45:51.301 INFO:teuthology.orchestra.run.smithi107.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-03-08T20:45:51.301 INFO:teuthology.orchestra.run.smithi107.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-03-08T20:45:51.301 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-03-08T20:45:51.301 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-03-08T20:45:51.301 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-03-08T20:45:51.301 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-03-08T20:45:51.301 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-03-08T20:45:51.301 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-03-08T20:45:51.302 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-03-08T20:45:51.302 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-03-08T20:45:51.302 INFO:teuthology.orchestra.run.smithi107.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-03-08T20:45:51.302 INFO:teuthology.orchestra.run.smithi107.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-03-08T20:45:51.302 INFO:teuthology.orchestra.run.smithi107.stdout: python3-mako-1.1.4-6.el9.noarch 2024-03-08T20:45:51.302 INFO:teuthology.orchestra.run.smithi107.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-03-08T20:45:51.302 INFO:teuthology.orchestra.run.smithi107.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-03-08T20:45:51.302 INFO:teuthology.orchestra.run.smithi107.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-03-08T20:45:51.302 INFO:teuthology.orchestra.run.smithi107.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-03-08T20:45:51.302 INFO:teuthology.orchestra.run.smithi107.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-03-08T20:45:51.302 INFO:teuthology.orchestra.run.smithi107.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-03-08T20:45:51.303 INFO:teuthology.orchestra.run.smithi107.stdout: python3-packaging-20.9-5.el9.noarch 2024-03-08T20:45:51.303 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-03-08T20:45:51.303 INFO:teuthology.orchestra.run.smithi107.stdout: python3-ply-3.11-14.el9.noarch 2024-03-08T20:45:51.303 INFO:teuthology.orchestra.run.smithi107.stdout: python3-portend-3.1.0-2.el9.noarch 2024-03-08T20:45:51.303 INFO:teuthology.orchestra.run.smithi107.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-03-08T20:45:51.303 INFO:teuthology.orchestra.run.smithi107.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-03-08T20:45:51.303 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-03-08T20:45:51.303 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-03-08T20:45:51.303 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-03-08T20:45:51.303 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pycparser-2.20-6.el9.noarch 2024-03-08T20:45:51.304 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-03-08T20:45:51.304 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pytz-2021.1-5.el9.noarch 2024-03-08T20:45:51.304 INFO:teuthology.orchestra.run.smithi107.stdout: python3-rados-2:19.0.0-1965.g7d63482a.el9.x86_64 2024-03-08T20:45:51.304 INFO:teuthology.orchestra.run.smithi107.stdout: python3-rbd-2:19.0.0-1965.g7d63482a.el9.x86_64 2024-03-08T20:45:51.304 INFO:teuthology.orchestra.run.smithi107.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-03-08T20:45:51.304 INFO:teuthology.orchestra.run.smithi107.stdout: python3-requests-2.25.1-8.el9.noarch 2024-03-08T20:45:51.304 INFO:teuthology.orchestra.run.smithi107.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-03-08T20:45:51.304 INFO:teuthology.orchestra.run.smithi107.stdout: python3-rgw-2:19.0.0-1965.g7d63482a.el9.x86_64 2024-03-08T20:45:51.304 INFO:teuthology.orchestra.run.smithi107.stdout: python3-routes-2.5.1-5.el9.noarch 2024-03-08T20:45:51.304 INFO:teuthology.orchestra.run.smithi107.stdout: python3-rsa-4.9-2.el9.noarch 2024-03-08T20:45:51.304 INFO:teuthology.orchestra.run.smithi107.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-03-08T20:45:51.304 INFO:teuthology.orchestra.run.smithi107.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-03-08T20:45:51.304 INFO:teuthology.orchestra.run.smithi107.stdout: python3-toml-0.10.2-6.el9.noarch 2024-03-08T20:45:51.304 INFO:teuthology.orchestra.run.smithi107.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-03-08T20:45:51.305 INFO:teuthology.orchestra.run.smithi107.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-03-08T20:45:51.305 INFO:teuthology.orchestra.run.smithi107.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-03-08T20:45:51.305 INFO:teuthology.orchestra.run.smithi107.stdout: python3-webob-1.8.7-6.el9.noarch 2024-03-08T20:45:51.305 INFO:teuthology.orchestra.run.smithi107.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-03-08T20:45:51.305 INFO:teuthology.orchestra.run.smithi107.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-03-08T20:45:51.305 INFO:teuthology.orchestra.run.smithi107.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-03-08T20:45:51.305 INFO:teuthology.orchestra.run.smithi107.stdout: qatlib-23.11.0-1.el9.x86_64 2024-03-08T20:45:51.305 INFO:teuthology.orchestra.run.smithi107.stdout: qatlib-service-23.11.0-1.el9.x86_64 2024-03-08T20:45:51.305 INFO:teuthology.orchestra.run.smithi107.stdout: qatzip-libs-1.1.2-2.el9.x86_64 2024-03-08T20:45:51.305 INFO:teuthology.orchestra.run.smithi107.stdout: rbd-fuse-2:19.0.0-1965.g7d63482a.el9.x86_64 2024-03-08T20:45:51.305 INFO:teuthology.orchestra.run.smithi107.stdout: rbd-mirror-2:19.0.0-1965.g7d63482a.el9.x86_64 2024-03-08T20:45:51.305 INFO:teuthology.orchestra.run.smithi107.stdout: rbd-nbd-2:19.0.0-1965.g7d63482a.el9.x86_64 2024-03-08T20:45:51.305 INFO:teuthology.orchestra.run.smithi107.stdout: re2-1:20211101-3.el9.x86_64 2024-03-08T20:45:51.306 INFO:teuthology.orchestra.run.smithi107.stdout: socat-1.7.4.1-5.el9.x86_64 2024-03-08T20:45:51.306 INFO:teuthology.orchestra.run.smithi107.stdout: thrift-0.15.0-2.el9.x86_64 2024-03-08T20:45:51.306 INFO:teuthology.orchestra.run.smithi107.stdout: uuid-1.6.2-55.el9.x86_64 2024-03-08T20:45:51.306 INFO:teuthology.orchestra.run.smithi107.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-03-08T20:45:51.306 INFO:teuthology.orchestra.run.smithi107.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-03-08T20:45:51.306 INFO:teuthology.orchestra.run.smithi107.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-03-08T20:45:51.306 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-08T20:45:51.307 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2024-03-08T20:45:51.986 DEBUG:teuthology.parallel:result is None 2024-03-08T20:45:51.987 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=7d63482aa89628d733e298984a762389b625c761 2024-03-08T20:45:52.335 DEBUG:teuthology.orchestra.run.smithi107:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-08T20:45:52.397 INFO:teuthology.orchestra.run.smithi107.stdout:19.0.0-1965.g7d63482a.el9 2024-03-08T20:45:52.397 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-1965.g7d63482a.el9 2024-03-08T20:45:52.397 INFO:teuthology.task.install:The correct ceph version 19.0.0-1965.g7d63482a is installed. 2024-03-08T20:45:52.399 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=7d63482aa89628d733e298984a762389b625c761 2024-03-08T20:45:52.549 DEBUG:teuthology.orchestra.run.smithi149:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-08T20:45:52.625 INFO:teuthology.orchestra.run.smithi149.stdout:19.0.0-1965.g7d63482a.el9 2024-03-08T20:45:52.625 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-1965.g7d63482a.el9 2024-03-08T20:45:52.625 INFO:teuthology.task.install:The correct ceph version 19.0.0-1965.g7d63482a is installed. 2024-03-08T20:45:52.627 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=7d63482aa89628d733e298984a762389b625c761 2024-03-08T20:45:52.998 DEBUG:teuthology.orchestra.run.smithi204:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-08T20:45:53.073 INFO:teuthology.orchestra.run.smithi204.stdout:19.0.0-1965.g7d63482a.el9 2024-03-08T20:45:53.073 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-1965.g7d63482a.el9 2024-03-08T20:45:53.073 INFO:teuthology.task.install:The correct ceph version 19.0.0-1965.g7d63482a is installed. 2024-03-08T20:45:53.075 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-03-08T20:45:53.075 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-03-08T20:45:53.076 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-08T20:45:53.158 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-03-08T20:45:53.158 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-08T20:45:53.257 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-03-08T20:45:53.257 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-08T20:45:53.359 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-03-08T20:45:53.359 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-03-08T20:45:53.359 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd of=/usr/bin/daemon-helper 2024-03-08T20:45:53.429 DEBUG:teuthology.orchestra.run.smithi107:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-08T20:45:53.541 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-03-08T20:45:53.541 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd of=/usr/bin/daemon-helper 2024-03-08T20:45:53.640 DEBUG:teuthology.orchestra.run.smithi149:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-08T20:45:53.779 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-03-08T20:45:53.780 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd of=/usr/bin/daemon-helper 2024-03-08T20:45:53.878 DEBUG:teuthology.orchestra.run.smithi204:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-08T20:45:54.016 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-03-08T20:45:54.016 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-03-08T20:45:54.016 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-08T20:45:54.090 DEBUG:teuthology.orchestra.run.smithi107:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-08T20:45:54.202 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-03-08T20:45:54.202 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-08T20:45:54.298 DEBUG:teuthology.orchestra.run.smithi149:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-08T20:45:54.437 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-03-08T20:45:54.437 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-08T20:45:54.527 DEBUG:teuthology.orchestra.run.smithi204:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-08T20:45:54.665 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-03-08T20:45:54.665 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-03-08T20:45:54.665 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd of=/usr/bin/stdin-killer 2024-03-08T20:45:54.737 DEBUG:teuthology.orchestra.run.smithi107:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-08T20:45:54.852 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-03-08T20:45:54.852 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd of=/usr/bin/stdin-killer 2024-03-08T20:45:54.951 DEBUG:teuthology.orchestra.run.smithi149:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-08T20:45:55.047 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-03-08T20:45:55.047 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd of=/usr/bin/stdin-killer 2024-03-08T20:45:55.145 DEBUG:teuthology.orchestra.run.smithi204:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-08T20:45:55.238 INFO:teuthology.run_tasks:Running task cephadm... 2024-03-08T20:45:55.336 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'max_mds': 5, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_bal_replicate_threshold': 1, 'mds_export_ephemeral_random_max': 0.1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug cephsqlite': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN'], 'sha1': '7d63482aa89628d733e298984a762389b625c761', 'subvols': {'create': 2, 'subvol_options': '--size 25000000000'}} 2024-03-08T20:45:55.336 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761 2024-03-08T20:45:55.337 INFO:tasks.cephadm:Cluster fsid is dc20f0f8-dd8c-11ee-95c6-87774f69a715 2024-03-08T20:45:55.337 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-03-08T20:45:55.337 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.107', 'mon.b': '172.21.15.149', 'mon.c': '172.21.15.204'} 2024-03-08T20:45:55.337 INFO:tasks.cephadm:First mon is mon.a on smithi107 2024-03-08T20:45:55.337 INFO:tasks.cephadm:First mgr is x 2024-03-08T20:45:55.337 INFO:tasks.cephadm:Normalizing hostnames... 2024-03-08T20:45:55.338 DEBUG:teuthology.orchestra.run.smithi107:> sudo hostname $(hostname -s) 2024-03-08T20:45:55.409 DEBUG:teuthology.orchestra.run.smithi149:> sudo hostname $(hostname -s) 2024-03-08T20:45:55.506 DEBUG:teuthology.orchestra.run.smithi204:> sudo hostname $(hostname -s) 2024-03-08T20:45:55.605 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-03-08T20:45:55.606 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=7d63482aa89628d733e298984a762389b625c761 2024-03-08T20:45:55.744 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '7d63482aa89628d733e298984a762389b625c761', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/77443/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-1965-g7d63482a', 'node_name': '172.21.2.1+braggi01', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-1965.g7d63482a'}, 'url': 'https://1.chacra.ceph.com/r/ceph/rishabh-2024mar8/7d63482aa89628d733e298984a762389b625c761/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-03-08 17:09:11.137784', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'rishabh-2024mar8', 'chacra_url': 'https://1.chacra.ceph.com/repos/ceph/rishabh-2024mar8/7d63482aa89628d733e298984a762389b625c761/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-03-08T20:45:55.882 INFO:tasks.util.chacra:got chacra host 1.chacra.ceph.com, ref rishabh-2024mar8, sha1 7d63482aa89628d733e298984a762389b625c761 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=7d63482aa89628d733e298984a762389b625c761 2024-03-08T20:45:55.883 INFO:tasks.cephadm:Discovered cachra url: https://1.chacra.ceph.com/binaries/ceph/rishabh-2024mar8/7d63482aa89628d733e298984a762389b625c761/centos/9/x86_64/flavors/default/cephadm 2024-03-08T20:45:55.884 DEBUG:teuthology.orchestra.run.smithi107:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/rishabh-2024mar8/7d63482aa89628d733e298984a762389b625c761/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-03-08T20:45:56.922 INFO:teuthology.orchestra.run.smithi107.stdout:-rw-r--r--. 1 ubuntu ubuntu 767402 Mar 8 20:45 /home/ubuntu/cephtest/cephadm 2024-03-08T20:45:56.922 DEBUG:teuthology.orchestra.run.smithi149:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/rishabh-2024mar8/7d63482aa89628d733e298984a762389b625c761/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-03-08T20:45:57.433 INFO:teuthology.orchestra.run.smithi149.stdout:-rw-r--r--. 1 ubuntu ubuntu 767402 Mar 8 20:45 /home/ubuntu/cephtest/cephadm 2024-03-08T20:45:57.433 DEBUG:teuthology.orchestra.run.smithi204:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/rishabh-2024mar8/7d63482aa89628d733e298984a762389b625c761/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-03-08T20:45:57.931 INFO:teuthology.orchestra.run.smithi204.stdout:-rw-r--r--. 1 ubuntu ubuntu 767402 Mar 8 20:45 /home/ubuntu/cephtest/cephadm 2024-03-08T20:45:57.932 DEBUG:teuthology.orchestra.run.smithi107:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-03-08T20:45:57.993 DEBUG:teuthology.orchestra.run.smithi149:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-03-08T20:45:58.054 DEBUG:teuthology.orchestra.run.smithi204:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-03-08T20:45:58.127 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761 on all hosts... 2024-03-08T20:45:58.128 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761 pull 2024-03-08T20:45:58.135 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761 pull 2024-03-08T20:45:58.142 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761 pull 2024-03-08T20:45:59.286 INFO:teuthology.orchestra.run.smithi149.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761... 2024-03-08T20:45:59.293 INFO:teuthology.orchestra.run.smithi204.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761... 2024-03-08T20:45:59.379 INFO:teuthology.orchestra.run.smithi107.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761... 2024-03-08T20:46:35.183 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-03-08T20:46:35.184 INFO:teuthology.orchestra.run.smithi149.stdout: "ceph_version": "ceph version 19.0.0-1965-g7d63482a (7d63482aa89628d733e298984a762389b625c761) squid (dev)", 2024-03-08T20:46:35.184 INFO:teuthology.orchestra.run.smithi149.stdout: "image_id": "a272146cb810b1fd4a53627f019472132e30966d5c25b01bfc44a001295f1249", 2024-03-08T20:46:35.184 INFO:teuthology.orchestra.run.smithi149.stdout: "repo_digests": [ 2024-03-08T20:46:35.184 INFO:teuthology.orchestra.run.smithi149.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd" 2024-03-08T20:46:35.184 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-03-08T20:46:35.184 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-03-08T20:46:42.838 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-03-08T20:46:42.839 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph_version": "ceph version 19.0.0-1965-g7d63482a (7d63482aa89628d733e298984a762389b625c761) squid (dev)", 2024-03-08T20:46:42.839 INFO:teuthology.orchestra.run.smithi107.stdout: "image_id": "a272146cb810b1fd4a53627f019472132e30966d5c25b01bfc44a001295f1249", 2024-03-08T20:46:42.839 INFO:teuthology.orchestra.run.smithi107.stdout: "repo_digests": [ 2024-03-08T20:46:42.839 INFO:teuthology.orchestra.run.smithi107.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd" 2024-03-08T20:46:42.839 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-03-08T20:46:42.839 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-03-08T20:46:51.588 INFO:teuthology.orchestra.run.smithi204.stdout:{ 2024-03-08T20:46:51.588 INFO:teuthology.orchestra.run.smithi204.stdout: "ceph_version": "ceph version 19.0.0-1965-g7d63482a (7d63482aa89628d733e298984a762389b625c761) squid (dev)", 2024-03-08T20:46:51.588 INFO:teuthology.orchestra.run.smithi204.stdout: "image_id": "a272146cb810b1fd4a53627f019472132e30966d5c25b01bfc44a001295f1249", 2024-03-08T20:46:51.588 INFO:teuthology.orchestra.run.smithi204.stdout: "repo_digests": [ 2024-03-08T20:46:51.588 INFO:teuthology.orchestra.run.smithi204.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd" 2024-03-08T20:46:51.588 INFO:teuthology.orchestra.run.smithi204.stdout: ] 2024-03-08T20:46:51.589 INFO:teuthology.orchestra.run.smithi204.stdout:} 2024-03-08T20:46:51.652 DEBUG:teuthology.orchestra.run.smithi107:> sudo mkdir -p /etc/ceph 2024-03-08T20:46:51.757 DEBUG:teuthology.orchestra.run.smithi149:> sudo mkdir -p /etc/ceph 2024-03-08T20:46:51.863 DEBUG:teuthology.orchestra.run.smithi204:> sudo mkdir -p /etc/ceph 2024-03-08T20:46:51.952 DEBUG:teuthology.orchestra.run.smithi107:> sudo chmod 777 /etc/ceph 2024-03-08T20:46:52.052 DEBUG:teuthology.orchestra.run.smithi149:> sudo chmod 777 /etc/ceph 2024-03-08T20:46:52.150 DEBUG:teuthology.orchestra.run.smithi204:> sudo chmod 777 /etc/ceph 2024-03-08T20:46:52.243 INFO:tasks.cephadm:Writing seed config... 2024-03-08T20:46:52.245 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-03-08T20:46:52.245 INFO:tasks.cephadm: override: [client] debug client = 20 2024-03-08T20:46:52.245 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-03-08T20:46:52.245 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-03-08T20:46:52.245 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-03-08T20:46:52.245 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-03-08T20:46:52.245 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-03-08T20:46:52.245 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-03-08T20:46:52.245 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-03-08T20:46:52.245 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-03-08T20:46:52.246 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-03-08T20:46:52.246 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-03-08T20:46:52.246 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-03-08T20:46:52.246 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-03-08T20:46:52.246 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-03-08T20:46:52.246 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-03-08T20:46:52.246 INFO:tasks.cephadm: override: [mds] mds_bal_replicate_threshold = 1 2024-03-08T20:46:52.246 INFO:tasks.cephadm: override: [mds] mds_export_ephemeral_random_max = 0.1 2024-03-08T20:46:52.246 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-03-08T20:46:52.246 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-03-08T20:46:52.246 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-03-08T20:46:52.246 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-03-08T20:46:52.246 INFO:tasks.cephadm: override: [mgr] debug cephsqlite = 20 2024-03-08T20:46:52.246 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-03-08T20:46:52.247 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-03-08T20:46:52.247 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-03-08T20:46:52.247 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-03-08T20:46:52.247 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-03-08T20:46:52.247 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-03-08T20:46:52.247 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-03-08T20:46:52.247 INFO:tasks.cephadm: override: [osd] bluestore compression mode = aggressive 2024-03-08T20:46:52.247 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-03-08T20:46:52.247 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-03-08T20:46:52.247 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-03-08T20:46:52.247 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-03-08T20:46:52.247 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-03-08T20:46:52.248 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-03-08T20:46:52.248 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-03-08T20:46:52.248 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-03-08T20:46:52.248 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-03-08T20:46:52.248 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-03-08T20:46:52.248 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-03-08T20:46:52.248 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-03-08T20:46:52.248 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-03-08T20:46:52.248 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10 2024-03-08T20:46:52.249 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-03-08T20:46:52.249 DEBUG:teuthology.orchestra.run.smithi107:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-03-08T20:46:52.305 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = dc20f0f8-dd8c-11ee-95c6-87774f69a715 [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore compression mode = aggressive bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug cephsqlite = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_bal_replicate_threshold = 1 mds_export_ephemeral_random_max = 0.1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-03-08T20:46:52.306 DEBUG:teuthology.orchestra.run.smithi107:mon.a> sudo journalctl -f -n 0 -u ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715@mon.a.service 2024-03-08T20:46:52.354 DEBUG:teuthology.orchestra.run.smithi107:mgr.x> sudo journalctl -f -n 0 -u ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715@mgr.x.service 2024-03-08T20:46:52.400 INFO:tasks.cephadm:Bootstrapping... 2024-03-08T20:46:52.401 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761 -v bootstrap --fsid dc20f0f8-dd8c-11ee-95c6-87774f69a715 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.107 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-03-08T20:46:52.674 INFO:teuthology.orchestra.run.smithi107.stdout:-------------------------------------------------------------------------------- 2024-03-08T20:46:52.674 INFO:teuthology.orchestra.run.smithi107.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761', '-v', 'bootstrap', '--fsid', 'dc20f0f8-dd8c-11ee-95c6-87774f69a715', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.107', '--skip-admin-label'] 2024-03-08T20:46:52.732 INFO:teuthology.orchestra.run.smithi107.stdout:/bin/podman: stdout 4.9.3-dev 2024-03-08T20:46:52.733 INFO:teuthology.orchestra.run.smithi107.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-03-08T20:46:52.733 INFO:teuthology.orchestra.run.smithi107.stdout:Verifying podman|docker is present... 2024-03-08T20:46:52.783 INFO:teuthology.orchestra.run.smithi107.stdout:/bin/podman: stdout 4.9.3-dev 2024-03-08T20:46:52.783 INFO:teuthology.orchestra.run.smithi107.stdout:Verifying lvm2 is present... 2024-03-08T20:46:52.784 INFO:teuthology.orchestra.run.smithi107.stdout:Verifying time synchronization is in place... 2024-03-08T20:46:52.799 INFO:teuthology.orchestra.run.smithi107.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-03-08T20:46:52.799 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-03-08T20:46:52.815 INFO:teuthology.orchestra.run.smithi107.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-03-08T20:46:52.815 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stdout inactive 2024-03-08T20:46:52.841 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stdout enabled 2024-03-08T20:46:52.862 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stdout active 2024-03-08T20:46:52.863 INFO:teuthology.orchestra.run.smithi107.stdout:Unit chronyd.service is enabled and running 2024-03-08T20:46:52.863 INFO:teuthology.orchestra.run.smithi107.stdout:Repeating the final host check... 2024-03-08T20:46:52.920 INFO:teuthology.orchestra.run.smithi107.stdout:/bin/podman: stdout 4.9.3-dev 2024-03-08T20:46:52.920 INFO:teuthology.orchestra.run.smithi107.stdout:podman (/bin/podman) version 4.9.3 is present 2024-03-08T20:46:52.920 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl is present 2024-03-08T20:46:52.920 INFO:teuthology.orchestra.run.smithi107.stdout:lvcreate is present 2024-03-08T20:46:52.936 INFO:teuthology.orchestra.run.smithi107.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-03-08T20:46:52.936 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-03-08T20:46:52.951 INFO:teuthology.orchestra.run.smithi107.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-03-08T20:46:52.952 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stdout inactive 2024-03-08T20:46:52.969 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stdout enabled 2024-03-08T20:46:52.984 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stdout active 2024-03-08T20:46:52.985 INFO:teuthology.orchestra.run.smithi107.stdout:Unit chronyd.service is enabled and running 2024-03-08T20:46:52.985 INFO:teuthology.orchestra.run.smithi107.stdout:Host looks OK 2024-03-08T20:46:52.985 INFO:teuthology.orchestra.run.smithi107.stdout:Cluster fsid: dc20f0f8-dd8c-11ee-95c6-87774f69a715 2024-03-08T20:46:52.986 INFO:teuthology.orchestra.run.smithi107.stdout:Acquiring lock 139910147178400 on /run/cephadm/dc20f0f8-dd8c-11ee-95c6-87774f69a715.lock 2024-03-08T20:46:52.986 INFO:teuthology.orchestra.run.smithi107.stdout:Lock 139910147178400 acquired on /run/cephadm/dc20f0f8-dd8c-11ee-95c6-87774f69a715.lock 2024-03-08T20:46:52.987 INFO:teuthology.orchestra.run.smithi107.stdout:Verifying IP 172.21.15.107 port 3300 ... 2024-03-08T20:46:52.988 INFO:teuthology.orchestra.run.smithi107.stdout:Verifying IP 172.21.15.107 port 6789 ... 2024-03-08T20:46:52.988 INFO:teuthology.orchestra.run.smithi107.stdout:Base mon IP(s) is [172.21.15.107:3300, 172.21.15.107:6789], mon addrv is [v2:172.21.15.107:3300,v1:172.21.15.107:6789] 2024-03-08T20:46:53.003 INFO:teuthology.orchestra.run.smithi107.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.107 metric 100 2024-03-08T20:46:53.003 INFO:teuthology.orchestra.run.smithi107.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.107 metric 100 2024-03-08T20:46:53.015 INFO:teuthology.orchestra.run.smithi107.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-03-08T20:46:53.015 INFO:teuthology.orchestra.run.smithi107.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 256 pref medium 2024-03-08T20:46:53.015 INFO:teuthology.orchestra.run.smithi107.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 1024 expires 1779sec hoplimit 64 pref medium 2024-03-08T20:46:53.026 INFO:teuthology.orchestra.run.smithi107.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-03-08T20:46:53.026 INFO:teuthology.orchestra.run.smithi107.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-03-08T20:46:53.026 INFO:teuthology.orchestra.run.smithi107.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-03-08T20:46:53.026 INFO:teuthology.orchestra.run.smithi107.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-03-08T20:46:53.026 INFO:teuthology.orchestra.run.smithi107.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:fe8f:47eb/64 scope link 2024-03-08T20:46:53.026 INFO:teuthology.orchestra.run.smithi107.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-03-08T20:46:53.028 INFO:teuthology.orchestra.run.smithi107.stdout:Mon IP `172.21.15.107` is in CIDR network `172.21.0.0/20` 2024-03-08T20:46:53.029 INFO:teuthology.orchestra.run.smithi107.stdout:Mon IP `172.21.15.107` is in CIDR network `172.21.0.0/20` 2024-03-08T20:46:53.029 INFO:teuthology.orchestra.run.smithi107.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-03-08T20:46:53.031 INFO:teuthology.orchestra.run.smithi107.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-03-08T20:46:53.033 INFO:teuthology.orchestra.run.smithi107.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761... 2024-03-08T20:46:53.690 INFO:teuthology.orchestra.run.smithi107.stdout:/bin/podman: stdout a272146cb810b1fd4a53627f019472132e30966d5c25b01bfc44a001295f1249 2024-03-08T20:46:53.690 INFO:teuthology.orchestra.run.smithi107.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761... 2024-03-08T20:46:53.690 INFO:teuthology.orchestra.run.smithi107.stdout:/bin/podman: stderr Getting image source signatures 2024-03-08T20:46:53.690 INFO:teuthology.orchestra.run.smithi107.stdout:/bin/podman: stderr Copying blob sha256:f58b4670ef4403d11d1c912aa26a4dbcf73447df5302e9dc66f6a7fc0d9d1e47 2024-03-08T20:46:53.690 INFO:teuthology.orchestra.run.smithi107.stdout:/bin/podman: stderr Copying blob sha256:7a785ebef89f89c1f7e1e5df571ab58da14c499794e7964e586765517f59740d 2024-03-08T20:46:53.690 INFO:teuthology.orchestra.run.smithi107.stdout:/bin/podman: stderr Copying config sha256:a272146cb810b1fd4a53627f019472132e30966d5c25b01bfc44a001295f1249 2024-03-08T20:46:53.690 INFO:teuthology.orchestra.run.smithi107.stdout:/bin/podman: stderr Writing manifest to image destination 2024-03-08T20:46:54.987 INFO:teuthology.orchestra.run.smithi107.stdout:ceph: stdout ceph version 19.0.0-1965-g7d63482a (7d63482aa89628d733e298984a762389b625c761) squid (dev) 2024-03-08T20:46:54.988 INFO:teuthology.orchestra.run.smithi107.stdout:Ceph version: ceph version 19.0.0-1965-g7d63482a (7d63482aa89628d733e298984a762389b625c761) squid (dev) 2024-03-08T20:46:54.988 INFO:teuthology.orchestra.run.smithi107.stdout:Extracting ceph user uid/gid from container image... 2024-03-08T20:46:56.123 INFO:teuthology.orchestra.run.smithi107.stdout:stat: stdout 167 167 2024-03-08T20:46:56.123 INFO:teuthology.orchestra.run.smithi107.stdout:Creating initial keys... 2024-03-08T20:46:57.330 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph-authtool: stdout AQBAeetlg0UJJRAAnJLbuc8VKxZBBiZNReUZVA== 2024-03-08T20:46:58.421 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph-authtool: stdout AQBBeetlosT0LhAAUZN9StZXxY2zQOpw80ILFA== 2024-03-08T20:46:59.600 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph-authtool: stdout AQBCeetloIO0NBAAGtxz5Oeo68Ci4AToc/3fOA== 2024-03-08T20:46:59.603 INFO:teuthology.orchestra.run.smithi107.stdout:Creating initial monmap... 2024-03-08T20:47:00.677 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-03-08T20:47:00.677 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-03-08T20:47:00.677 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to dc20f0f8-dd8c-11ee-95c6-87774f69a715 2024-03-08T20:47:00.677 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-03-08T20:47:00.677 INFO:teuthology.orchestra.run.smithi107.stdout:monmaptool for a [v2:172.21.15.107:3300,v1:172.21.15.107:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-03-08T20:47:00.677 INFO:teuthology.orchestra.run.smithi107.stdout:setting min_mon_release = quincy 2024-03-08T20:47:00.677 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/monmaptool: set fsid to dc20f0f8-dd8c-11ee-95c6-87774f69a715 2024-03-08T20:47:00.677 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-03-08T20:47:00.678 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-08T20:47:00.678 INFO:teuthology.orchestra.run.smithi107.stdout:Creating mon... 2024-03-08T20:47:01.888 INFO:teuthology.orchestra.run.smithi107.stdout:create mon.a on 2024-03-08T20:47:02.255 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-03-08T20:47:02.548 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-03-08T20:47:02.833 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715.target → /etc/systemd/system/ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715.target. 2024-03-08T20:47:02.833 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715.target → /etc/systemd/system/ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715.target. 2024-03-08T20:47:03.232 INFO:teuthology.orchestra.run.smithi107.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715@mon.a 2024-03-08T20:47:03.232 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stderr Failed to reset failed state of unit ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715@mon.a.service: Unit ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715@mon.a.service not loaded. 2024-03-08T20:47:03.502 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715.target.wants/ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715@mon.a.service → /etc/systemd/system/ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715@.service. 2024-03-08T20:47:04.111 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:04 smithi107 podman[30979]: 2024-03-08 20:47:04.105451728 +0000 UTC m=+0.356177726 container init 2641d23d6b2ca57a12100be8eec134a3c5478240818a1a02191ca6e97af658f4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mon-a, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=rishabh-2024mar8-7d63482, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-03-08T20:47:04.111 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:04 smithi107 podman[30979]: 2024-03-08 20:47:04.115233958 +0000 UTC m=+0.365959955 container start 2641d23d6b2ca57a12100be8eec134a3c5478240818a1a02191ca6e97af658f4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mon-a, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=rishabh-2024mar8-7d63482, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-08T20:47:04.210 INFO:teuthology.orchestra.run.smithi107.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-08T20:47:04.210 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stdout disabled 2024-03-08T20:47:04.225 INFO:teuthology.orchestra.run.smithi107.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-08T20:47:04.225 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stdout inactive 2024-03-08T20:47:04.225 INFO:teuthology.orchestra.run.smithi107.stdout:firewalld.service is not enabled 2024-03-08T20:47:04.225 INFO:teuthology.orchestra.run.smithi107.stdout:Not possible to enable service . firewalld.service is not available 2024-03-08T20:47:04.226 INFO:teuthology.orchestra.run.smithi107.stdout:Waiting for mon to start... 2024-03-08T20:47:04.226 INFO:teuthology.orchestra.run.smithi107.stdout:Waiting for mon... 2024-03-08T20:47:04.374 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:04 smithi107 bash[30979]: 2641d23d6b2ca57a12100be8eec134a3c5478240818a1a02191ca6e97af658f4 2024-03-08T20:47:04.374 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:04 smithi107 ceph-mon[30998]: mkfs dc20f0f8-dd8c-11ee-95c6-87774f69a715 2024-03-08T20:47:04.374 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:04 smithi107 systemd[1]: Started Ceph mon.a for dc20f0f8-dd8c-11ee-95c6-87774f69a715. 2024-03-08T20:47:04.374 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:04 smithi107 ceph-mon[30998]: mon.a is new leader, mons a in quorum (ranks 0) 2024-03-08T20:47:05.480 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:05 smithi107 ceph-mon[30998]: mon.a is new leader, mons a in quorum (ranks 0) 2024-03-08T20:47:05.480 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:05 smithi107 ceph-mon[30998]: monmap e1: 1 mons at {a=[v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-08T20:47:05.480 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:05 smithi107 ceph-mon[30998]: fsmap 2024-03-08T20:47:05.480 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:05 smithi107 ceph-mon[30998]: osdmap e1: 0 total, 0 up, 0 in 2024-03-08T20:47:05.480 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:05 smithi107 ceph-mon[30998]: mgrmap e1: no daemons active 2024-03-08T20:47:05.480 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:05 smithi107 ceph-mon[30998]: from='client.? 172.21.15.107:0/1879256915' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-03-08T20:47:07.845 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout cluster: 2024-03-08T20:47:07.845 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout id: dc20f0f8-dd8c-11ee-95c6-87774f69a715 2024-03-08T20:47:07.846 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-03-08T20:47:07.846 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout 2024-03-08T20:47:07.846 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout services: 2024-03-08T20:47:07.846 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.705826s) 2024-03-08T20:47:07.846 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-03-08T20:47:07.846 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-03-08T20:47:07.846 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout 2024-03-08T20:47:07.846 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout data: 2024-03-08T20:47:07.846 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-03-08T20:47:07.846 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-03-08T20:47:07.846 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-03-08T20:47:07.847 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout pgs: 2024-03-08T20:47:07.847 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout 2024-03-08T20:47:07.847 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:04.803+0000 7f1a6d600700 1 Processor -- start 2024-03-08T20:47:07.847 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:04.803+0000 7f1a6d600700 1 -- start start 2024-03-08T20:47:07.847 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:04.804+0000 7f1a6d600700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1a6810f230 0x7f1a6810f650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:47:07.847 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:04.804+0000 7f1a6d600700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a6810fc20 con 0x7f1a6810f230 2024-03-08T20:47:07.847 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:04.805+0000 7f1a67e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1a6810f230 0x7f1a6810f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:47:07.847 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:04.805+0000 7f1a67e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1a6810f230 0x7f1a6810f650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:56528/0 (socket says 172.21.15.107:56528) 2024-03-08T20:47:07.847 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:04.805+0000 7f1a67e00700 1 -- 172.21.15.107:0/2112036605 learned_addr learned my addr 172.21.15.107:0/2112036605 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T20:47:07.847 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:04.805+0000 7f1a67e00700 1 -- 172.21.15.107:0/2112036605 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a6810dcf0 con 0x7f1a6810f230 2024-03-08T20:47:07.848 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:04.805+0000 7f1a67e00700 1 --2- 172.21.15.107:0/2112036605 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1a6810f230 0x7f1a6810f650 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f1a580098a0 tx=0x7f1a58009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2324753bd0b5e42 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:47:07.848 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:04.806+0000 7f1a66a00700 1 -- 172.21.15.107:0/2112036605 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1a58004160 con 0x7f1a6810f230 2024-03-08T20:47:07.848 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:04.806+0000 7f1a66a00700 1 -- 172.21.15.107:0/2112036605 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a58004300 con 0x7f1a6810f230 2024-03-08T20:47:07.848 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:04.806+0000 7f1a66a00700 1 -- 172.21.15.107:0/2112036605 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1a580036a0 con 0x7f1a6810f230 2024-03-08T20:47:07.848 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:04.806+0000 7f1a6d600700 1 -- 172.21.15.107:0/2112036605 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1a6810f230 msgr2=0x7f1a6810f650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:47:07.848 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:04.806+0000 7f1a6d600700 1 --2- 172.21.15.107:0/2112036605 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1a6810f230 0x7f1a6810f650 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f1a580098a0 tx=0x7f1a58009bb0 comp rx=0 tx=0).stop 2024-03-08T20:47:07.848 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:04.807+0000 7f1a6d600700 1 -- 172.21.15.107:0/2112036605 shutdown_connections 2024-03-08T20:47:07.848 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:04.807+0000 7f1a6d600700 1 --2- 172.21.15.107:0/2112036605 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1a6810f230 0x7f1a6810f650 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:47:07.848 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:04.807+0000 7f1a6d600700 1 -- 172.21.15.107:0/2112036605 >> 172.21.15.107:0/2112036605 conn(0x7f1a6807cc30 msgr2=0x7f1a6807f050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:47:07.848 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:04.807+0000 7f1a6d600700 1 -- 172.21.15.107:0/2112036605 shutdown_connections 2024-03-08T20:47:07.848 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:04.807+0000 7f1a6d600700 1 -- 172.21.15.107:0/2112036605 wait complete. 2024-03-08T20:47:07.849 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:04.808+0000 7f1a6d600700 1 Processor -- start 2024-03-08T20:47:07.849 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:04.808+0000 7f1a6d600700 1 -- start start 2024-03-08T20:47:07.849 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:04.808+0000 7f1a6d600700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1a6810f230 0x7f1a681a1330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:47:07.849 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:04.808+0000 7f1a6d600700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a6810e1c0 con 0x7f1a6810f230 2024-03-08T20:47:07.849 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:04.809+0000 7f1a67e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1a6810f230 0x7f1a681a1330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:47:07.849 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:04.809+0000 7f1a67e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1a6810f230 0x7f1a681a1330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:56542/0 (socket says 172.21.15.107:56542) 2024-03-08T20:47:07.849 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:04.809+0000 7f1a67e00700 1 -- 172.21.15.107:0/1879256915 learned_addr learned my addr 172.21.15.107:0/1879256915 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T20:47:07.849 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:04.809+0000 7f1a67e00700 1 -- 172.21.15.107:0/1879256915 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a681a1870 con 0x7f1a6810f230 2024-03-08T20:47:07.849 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:04.810+0000 7f1a67e00700 1 --2- 172.21.15.107:0/1879256915 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1a6810f230 0x7f1a681a1330 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f1a58000c00 tx=0x7f1a58004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:47:07.850 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:04.810+0000 7f1a64c00700 1 -- 172.21.15.107:0/1879256915 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1a58003db0 con 0x7f1a6810f230 2024-03-08T20:47:07.850 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:04.810+0000 7f1a64c00700 1 -- 172.21.15.107:0/1879256915 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a58011630 con 0x7f1a6810f230 2024-03-08T20:47:07.850 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:04.810+0000 7f1a6d600700 1 -- 172.21.15.107:0/1879256915 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a681a1b00 con 0x7f1a6810f230 2024-03-08T20:47:07.850 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:04.810+0000 7f1a64c00700 1 -- 172.21.15.107:0/1879256915 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1a58011950 con 0x7f1a6810f230 2024-03-08T20:47:07.850 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:04.810+0000 7f1a6d600700 1 -- 172.21.15.107:0/1879256915 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a681a47f0 con 0x7f1a6810f230 2024-03-08T20:47:07.850 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:04.810+0000 7f1a64c00700 1 -- 172.21.15.107:0/1879256915 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f1a58011af0 con 0x7f1a6810f230 2024-03-08T20:47:07.850 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:04.810+0000 7f1a64c00700 1 -- 172.21.15.107:0/1879256915 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f1a5801c450 con 0x7f1a6810f230 2024-03-08T20:47:07.850 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:04.811+0000 7f1a6d600700 1 -- 172.21.15.107:0/1879256915 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a68063410 con 0x7f1a6810f230 2024-03-08T20:47:07.850 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:04.814+0000 7f1a64c00700 1 -- 172.21.15.107:0/1879256915 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73265 (secure 0 0 0) 0x7f1a58032020 con 0x7f1a6810f230 2024-03-08T20:47:07.850 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:04.893+0000 7f1a6d600700 1 -- 172.21.15.107:0/1879256915 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7f1a6802d830 con 0x7f1a6810f230 2024-03-08T20:47:07.850 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:04.894+0000 7f1a64c00700 1 -- 172.21.15.107:0/1879256915 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+317 (secure 0 0 0) 0x7f1a58004260 con 0x7f1a6810f230 2024-03-08T20:47:07.850 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:04.895+0000 7f1a6d600700 1 -- 172.21.15.107:0/1879256915 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1a6810f230 msgr2=0x7f1a681a1330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:47:07.850 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:04.895+0000 7f1a6d600700 1 --2- 172.21.15.107:0/1879256915 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1a6810f230 0x7f1a681a1330 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f1a58000c00 tx=0x7f1a58004750 comp rx=0 tx=0).stop 2024-03-08T20:47:07.851 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:04.895+0000 7f1a6d600700 1 -- 172.21.15.107:0/1879256915 shutdown_connections 2024-03-08T20:47:07.851 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:04.895+0000 7f1a6d600700 1 --2- 172.21.15.107:0/1879256915 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1a6810f230 0x7f1a681a1330 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:47:07.851 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:04.895+0000 7f1a6d600700 1 -- 172.21.15.107:0/1879256915 >> 172.21.15.107:0/1879256915 conn(0x7f1a6807cc30 msgr2=0x7f1a68197ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:47:07.851 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:04.895+0000 7f1a6d600700 1 -- 172.21.15.107:0/1879256915 shutdown_connections 2024-03-08T20:47:07.851 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:04.896+0000 7f1a6d600700 1 -- 172.21.15.107:0/1879256915 wait complete. 2024-03-08T20:47:07.851 INFO:teuthology.orchestra.run.smithi107.stdout:mon is available 2024-03-08T20:47:07.851 INFO:teuthology.orchestra.run.smithi107.stdout:Assimilating anything we can from ceph.conf... 2024-03-08T20:47:14.479 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:14 smithi107 ceph-mon[30998]: from='client.? 172.21.15.107:0/3843468555' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-03-08T20:47:14.480 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:14 smithi107 ceph-mon[30998]: from='client.? 172.21.15.107:0/3843468555' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-03-08T20:47:14.918 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout 2024-03-08T20:47:14.918 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout [global] 2024-03-08T20:47:14.918 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout fsid = dc20f0f8-dd8c-11ee-95c6-87774f69a715 2024-03-08T20:47:14.919 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.107:3300,v1:172.21.15.107:6789] 2024-03-08T20:47:14.919 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-03-08T20:47:14.919 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-03-08T20:47:14.919 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-03-08T20:47:14.919 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-03-08T20:47:14.919 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout 2024-03-08T20:47:14.919 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout [mgr] 2024-03-08T20:47:14.919 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-03-08T20:47:14.919 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout 2024-03-08T20:47:14.919 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout [osd] 2024-03-08T20:47:14.919 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-03-08T20:47:14.919 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-03-08T20:47:14.919 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-03-08T20:47:14.919 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-03-08T20:47:14.920 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-03-08T20:47:14.920 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:14.154+0000 7fbafde00700 1 Processor -- start 2024-03-08T20:47:14.920 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:14.154+0000 7fbafde00700 1 -- start start 2024-03-08T20:47:14.920 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:14.154+0000 7fbafde00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbaf8109960 0x7fbaf8109d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:47:14.920 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:14.154+0000 7fbafde00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbaf810a350 con 0x7fbaf8109960 2024-03-08T20:47:14.920 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:14.155+0000 7fbaf6a00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbaf8109960 0x7fbaf8109d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:47:14.920 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:14.155+0000 7fbaf6a00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbaf8109960 0x7fbaf8109d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:49930/0 (socket says 172.21.15.107:49930) 2024-03-08T20:47:14.920 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:14.155+0000 7fbaf6a00700 1 -- 172.21.15.107:0/3601068966 learned_addr learned my addr 172.21.15.107:0/3601068966 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T20:47:14.920 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:14.156+0000 7fbaf6a00700 1 -- 172.21.15.107:0/3601068966 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbaf810ab10 con 0x7fbaf8109960 2024-03-08T20:47:14.920 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:14.156+0000 7fbaf6a00700 1 --2- 172.21.15.107:0/3601068966 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbaf8109960 0x7fbaf8109d80 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fbae80098a0 tx=0x7fbae8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ba3721a80a4e5f0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:47:14.920 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:14.156+0000 7fbaf5600700 1 -- 172.21.15.107:0/3601068966 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbae8004160 con 0x7fbaf8109960 2024-03-08T20:47:14.920 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:14.156+0000 7fbaf5600700 1 -- 172.21.15.107:0/3601068966 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbae8004300 con 0x7fbaf8109960 2024-03-08T20:47:14.921 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:14.156+0000 7fbaf5600700 1 -- 172.21.15.107:0/3601068966 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbae80036a0 con 0x7fbaf8109960 2024-03-08T20:47:14.921 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:14.157+0000 7fbafde00700 1 -- 172.21.15.107:0/3601068966 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbaf8109960 msgr2=0x7fbaf8109d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:47:14.921 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:14.157+0000 7fbafde00700 1 --2- 172.21.15.107:0/3601068966 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbaf8109960 0x7fbaf8109d80 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fbae80098a0 tx=0x7fbae8009bb0 comp rx=0 tx=0).stop 2024-03-08T20:47:14.921 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:14.157+0000 7fbafde00700 1 -- 172.21.15.107:0/3601068966 shutdown_connections 2024-03-08T20:47:14.921 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:14.157+0000 7fbafde00700 1 --2- 172.21.15.107:0/3601068966 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbaf8109960 0x7fbaf8109d80 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:47:14.921 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:14.157+0000 7fbafde00700 1 -- 172.21.15.107:0/3601068966 >> 172.21.15.107:0/3601068966 conn(0x7fbaf807cdd0 msgr2=0x7fbaf81074f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:47:14.921 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:14.157+0000 7fbafde00700 1 -- 172.21.15.107:0/3601068966 shutdown_connections 2024-03-08T20:47:14.921 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:14.157+0000 7fbafde00700 1 -- 172.21.15.107:0/3601068966 wait complete. 2024-03-08T20:47:14.921 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:14.158+0000 7fbafde00700 1 Processor -- start 2024-03-08T20:47:14.921 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:14.159+0000 7fbafde00700 1 -- start start 2024-03-08T20:47:14.921 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:14.159+0000 7fbafde00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbaf8109960 0x7fbaf819d020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:47:14.921 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:14.159+0000 7fbafde00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbaf810b040 con 0x7fbaf8109960 2024-03-08T20:47:14.921 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:14.159+0000 7fbaf6a00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbaf8109960 0x7fbaf819d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:47:14.921 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:14.159+0000 7fbaf6a00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbaf8109960 0x7fbaf819d020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:49944/0 (socket says 172.21.15.107:49944) 2024-03-08T20:47:14.922 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:14.159+0000 7fbaf6a00700 1 -- 172.21.15.107:0/3843468555 learned_addr learned my addr 172.21.15.107:0/3843468555 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T20:47:14.922 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:14.159+0000 7fbaf6a00700 1 -- 172.21.15.107:0/3843468555 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbaf819d560 con 0x7fbaf8109960 2024-03-08T20:47:14.922 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:14.160+0000 7fbaf6a00700 1 --2- 172.21.15.107:0/3843468555 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbaf8109960 0x7fbaf819d020 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fbae8000c00 tx=0x7fbae8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:47:14.922 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:14.160+0000 7fbafca00700 1 -- 172.21.15.107:0/3843468555 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbae8003db0 con 0x7fbaf8109960 2024-03-08T20:47:14.922 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:14.160+0000 7fbafca00700 1 -- 172.21.15.107:0/3843468555 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbae8011630 con 0x7fbaf8109960 2024-03-08T20:47:14.922 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:14.160+0000 7fbafca00700 1 -- 172.21.15.107:0/3843468555 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbae8011950 con 0x7fbaf8109960 2024-03-08T20:47:14.922 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:14.160+0000 7fbafde00700 1 -- 172.21.15.107:0/3843468555 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbaf819d7f0 con 0x7fbaf8109960 2024-03-08T20:47:14.922 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:14.160+0000 7fbafde00700 1 -- 172.21.15.107:0/3843468555 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbaf81a04e0 con 0x7fbaf8109960 2024-03-08T20:47:14.922 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:14.160+0000 7fbafca00700 1 -- 172.21.15.107:0/3843468555 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fbae8011af0 con 0x7fbaf8109960 2024-03-08T20:47:14.922 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:14.161+0000 7fbafca00700 1 -- 172.21.15.107:0/3843468555 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fbae801c450 con 0x7fbaf8109960 2024-03-08T20:47:14.922 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:14.161+0000 7fbafde00700 1 -- 172.21.15.107:0/3843468555 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbaf8063410 con 0x7fbaf8109960 2024-03-08T20:47:14.922 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:14.164+0000 7fbafca00700 1 -- 172.21.15.107:0/3843468555 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73265 (secure 0 0 0) 0x7fbae8032020 con 0x7fbaf8109960 2024-03-08T20:47:14.922 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:14.240+0000 7fbafde00700 1 -- 172.21.15.107:0/3843468555 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7fbaf802d930 con 0x7fbaf8109960 2024-03-08T20:47:14.923 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:14.250+0000 7fbafca00700 1 -- 172.21.15.107:0/3843468555 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) v1 ==== 70+0+431 (secure 0 0 0) 0x7fbae8004260 con 0x7fbaf8109960 2024-03-08T20:47:14.923 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:14.252+0000 7fbafde00700 1 -- 172.21.15.107:0/3843468555 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbaf8109960 msgr2=0x7fbaf819d020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:47:14.923 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:14.252+0000 7fbafde00700 1 --2- 172.21.15.107:0/3843468555 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbaf8109960 0x7fbaf819d020 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fbae8000c00 tx=0x7fbae8004750 comp rx=0 tx=0).stop 2024-03-08T20:47:14.923 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:14.252+0000 7fbafca00700 1 -- 172.21.15.107:0/3843468555 <== mon.0 v2:172.21.15.107:3300/0 8 ==== config(26 keys) v1 ==== 1071+0+0 (secure 0 0 0) 0x7fbae801c650 con 0x7fbaf8109960 2024-03-08T20:47:14.923 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:14.252+0000 7fbafde00700 1 -- 172.21.15.107:0/3843468555 shutdown_connections 2024-03-08T20:47:14.923 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:14.252+0000 7fbafde00700 1 --2- 172.21.15.107:0/3843468555 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbaf8109960 0x7fbaf819d020 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:47:14.923 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:14.252+0000 7fbafde00700 1 -- 172.21.15.107:0/3843468555 >> 172.21.15.107:0/3843468555 conn(0x7fbaf807cdd0 msgr2=0x7fbaf8193cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:47:14.923 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:14.252+0000 7fbafde00700 1 -- 172.21.15.107:0/3843468555 shutdown_connections 2024-03-08T20:47:14.923 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:14.252+0000 7fbafde00700 1 -- 172.21.15.107:0/3843468555 wait complete. 2024-03-08T20:47:14.923 INFO:teuthology.orchestra.run.smithi107.stdout:Generating new minimal ceph.conf... 2024-03-08T20:47:15.968 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:15 smithi107 ceph-mon[30998]: from='client.? 172.21.15.107:0/2082360990' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:47:16.262 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:15.539+0000 7fab21200700 1 Processor -- start 2024-03-08T20:47:16.267 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:15.539+0000 7fab21200700 1 -- start start 2024-03-08T20:47:16.268 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:15.540+0000 7fab21200700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fab1c10ae40 0x7fab1c10b260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:47:16.268 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:15.540+0000 7fab21200700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab1c1093d0 con 0x7fab1c10ae40 2024-03-08T20:47:16.268 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:15.540+0000 7fab1b400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fab1c10ae40 0x7fab1c10b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:47:16.268 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:15.540+0000 7fab1b400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fab1c10ae40 0x7fab1c10b260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:49954/0 (socket says 172.21.15.107:49954) 2024-03-08T20:47:16.268 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:15.540+0000 7fab1b400700 1 -- 172.21.15.107:0/1766909948 learned_addr learned my addr 172.21.15.107:0/1766909948 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T20:47:16.268 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:15.541+0000 7fab1b400700 1 -- 172.21.15.107:0/1766909948 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab1c109550 con 0x7fab1c10ae40 2024-03-08T20:47:16.268 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:15.541+0000 7fab1b400700 1 --2- 172.21.15.107:0/1766909948 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fab1c10ae40 0x7fab1c10b260 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fab0c0098a0 tx=0x7fab0c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edfb28ca507d3048 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:47:16.268 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:15.542+0000 7fab1a000700 1 -- 172.21.15.107:0/1766909948 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fab0c004160 con 0x7fab1c10ae40 2024-03-08T20:47:16.268 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:15.542+0000 7fab1a000700 1 -- 172.21.15.107:0/1766909948 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1071+0+0 (secure 0 0 0) 0x7fab0c004300 con 0x7fab1c10ae40 2024-03-08T20:47:16.268 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:15.542+0000 7fab1a000700 1 -- 172.21.15.107:0/1766909948 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fab0c011a80 con 0x7fab1c10ae40 2024-03-08T20:47:16.268 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:15.542+0000 7fab21200700 1 -- 172.21.15.107:0/1766909948 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fab1c10ae40 msgr2=0x7fab1c10b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:47:16.268 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:15.542+0000 7fab21200700 1 --2- 172.21.15.107:0/1766909948 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fab1c10ae40 0x7fab1c10b260 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fab0c0098a0 tx=0x7fab0c009bb0 comp rx=0 tx=0).stop 2024-03-08T20:47:16.268 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:15.543+0000 7fab21200700 1 -- 172.21.15.107:0/1766909948 shutdown_connections 2024-03-08T20:47:16.269 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:15.543+0000 7fab21200700 1 --2- 172.21.15.107:0/1766909948 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fab1c10ae40 0x7fab1c10b260 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:47:16.269 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:15.543+0000 7fab21200700 1 -- 172.21.15.107:0/1766909948 >> 172.21.15.107:0/1766909948 conn(0x7fab1c07cc30 msgr2=0x7fab1c1074f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:47:16.269 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:15.543+0000 7fab21200700 1 -- 172.21.15.107:0/1766909948 shutdown_connections 2024-03-08T20:47:16.269 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:15.543+0000 7fab21200700 1 -- 172.21.15.107:0/1766909948 wait complete. 2024-03-08T20:47:16.269 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:15.544+0000 7fab21200700 1 Processor -- start 2024-03-08T20:47:16.269 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:15.544+0000 7fab21200700 1 -- start start 2024-03-08T20:47:16.269 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:15.544+0000 7fab21200700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fab1c10ae40 0x7fab1c19d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:47:16.269 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:15.544+0000 7fab21200700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab1c10dd10 con 0x7fab1c10ae40 2024-03-08T20:47:16.269 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:15.545+0000 7fab1b400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fab1c10ae40 0x7fab1c19d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:47:16.269 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:15.545+0000 7fab1b400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fab1c10ae40 0x7fab1c19d7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:49968/0 (socket says 172.21.15.107:49968) 2024-03-08T20:47:16.269 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:15.545+0000 7fab1b400700 1 -- 172.21.15.107:0/2082360990 learned_addr learned my addr 172.21.15.107:0/2082360990 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T20:47:16.269 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:15.545+0000 7fab1b400700 1 -- 172.21.15.107:0/2082360990 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab1c19dd10 con 0x7fab1c10ae40 2024-03-08T20:47:16.269 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:15.545+0000 7fab1b400700 1 --2- 172.21.15.107:0/2082360990 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fab1c10ae40 0x7fab1c19d7d0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fab0c000c00 tx=0x7fab0c011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:47:16.270 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:15.545+0000 7fab13e00700 1 -- 172.21.15.107:0/2082360990 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fab0c0039f0 con 0x7fab1c10ae40 2024-03-08T20:47:16.270 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:15.546+0000 7fab13e00700 1 -- 172.21.15.107:0/2082360990 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1071+0+0 (secure 0 0 0) 0x7fab0c003b90 con 0x7fab1c10ae40 2024-03-08T20:47:16.270 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:15.546+0000 7fab13e00700 1 -- 172.21.15.107:0/2082360990 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fab0c01a740 con 0x7fab1c10ae40 2024-03-08T20:47:16.270 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:15.546+0000 7fab21200700 1 -- 172.21.15.107:0/2082360990 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab1c19dfa0 con 0x7fab1c10ae40 2024-03-08T20:47:16.270 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:15.546+0000 7fab21200700 1 -- 172.21.15.107:0/2082360990 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab1c1a0c90 con 0x7fab1c10ae40 2024-03-08T20:47:16.270 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:15.546+0000 7fab13e00700 1 -- 172.21.15.107:0/2082360990 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fab0c01a9f0 con 0x7fab1c10ae40 2024-03-08T20:47:16.270 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:15.546+0000 7fab13e00700 1 -- 172.21.15.107:0/2082360990 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fab0c0193c0 con 0x7fab1c10ae40 2024-03-08T20:47:16.270 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:15.547+0000 7fab21200700 1 -- 172.21.15.107:0/2082360990 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab1c10de90 con 0x7fab1c10ae40 2024-03-08T20:47:16.270 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:15.550+0000 7fab13e00700 1 -- 172.21.15.107:0/2082360990 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73265 (secure 0 0 0) 0x7fab0c01ae20 con 0x7fab1c10ae40 2024-03-08T20:47:16.270 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:15.625+0000 7fab21200700 1 -- 172.21.15.107:0/2082360990 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7fab1c063410 con 0x7fab1c10ae40 2024-03-08T20:47:16.270 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:15.626+0000 7fab13e00700 1 -- 172.21.15.107:0/2082360990 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) v1 ==== 76+0+177 (secure 0 0 0) 0x7fab0c0248d0 con 0x7fab1c10ae40 2024-03-08T20:47:16.270 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:15.627+0000 7fab21200700 1 -- 172.21.15.107:0/2082360990 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fab1c10ae40 msgr2=0x7fab1c19d7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:47:16.271 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:15.627+0000 7fab21200700 1 --2- 172.21.15.107:0/2082360990 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fab1c10ae40 0x7fab1c19d7d0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fab0c000c00 tx=0x7fab0c011b00 comp rx=0 tx=0).stop 2024-03-08T20:47:16.271 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:15.627+0000 7fab21200700 1 -- 172.21.15.107:0/2082360990 shutdown_connections 2024-03-08T20:47:16.271 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:15.627+0000 7fab21200700 1 --2- 172.21.15.107:0/2082360990 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fab1c10ae40 0x7fab1c19d7d0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:47:16.271 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:15.627+0000 7fab21200700 1 -- 172.21.15.107:0/2082360990 >> 172.21.15.107:0/2082360990 conn(0x7fab1c07cc30 msgr2=0x7fab1c106d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:47:16.271 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:15.627+0000 7fab21200700 1 -- 172.21.15.107:0/2082360990 shutdown_connections 2024-03-08T20:47:16.271 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:15.628+0000 7fab21200700 1 -- 172.21.15.107:0/2082360990 wait complete. 2024-03-08T20:47:16.271 INFO:teuthology.orchestra.run.smithi107.stdout:Restarting the monitor... 2024-03-08T20:47:16.729 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:16 smithi107 systemd[1]: Stopping Ceph mon.a for dc20f0f8-dd8c-11ee-95c6-87774f69a715... 2024-03-08T20:47:17.157 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:16 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mon-a[30994]: 2024-03-08T20:47:16.739+0000 7fe359c00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-08T20:47:17.158 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:16 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mon-a[30994]: 2024-03-08T20:47:16.739+0000 7fe359c00700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-03-08T20:47:17.158 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:16 smithi107 podman[31255]: 2024-03-08 20:47:16.862953805 +0000 UTC m=+0.253994002 container died 2641d23d6b2ca57a12100be8eec134a3c5478240818a1a02191ca6e97af658f4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mon-a, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=rishabh-2024mar8-7d63482) 2024-03-08T20:47:17.479 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:17 smithi107 podman[31255]: 2024-03-08 20:47:17.273169484 +0000 UTC m=+0.664209705 container cleanup 2641d23d6b2ca57a12100be8eec134a3c5478240818a1a02191ca6e97af658f4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mon-a, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=rishabh-2024mar8-7d63482, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-03-08T20:47:17.479 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:17 smithi107 bash[31255]: ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mon-a 2024-03-08T20:47:17.979 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:17 smithi107 podman[31269]: 2024-03-08 20:47:17.54050149 +0000 UTC m=+0.657151985 container remove 2641d23d6b2ca57a12100be8eec134a3c5478240818a1a02191ca6e97af658f4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mon-a, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=rishabh-2024mar8-7d63482, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-08T20:47:18.394 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 systemd[1]: ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715@mon.a.service: Deactivated successfully. 2024-03-08T20:47:18.394 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 systemd[1]: Stopped Ceph mon.a for dc20f0f8-dd8c-11ee-95c6-87774f69a715. 2024-03-08T20:47:18.394 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 systemd[1]: ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715@mon.a.service: Consumed 1.852s CPU time. 2024-03-08T20:47:18.394 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 systemd[1]: Starting Ceph mon.a for dc20f0f8-dd8c-11ee-95c6-87774f69a715... 2024-03-08T20:47:18.671 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 podman[31363]: 2024-03-08 20:47:18.399532023 +0000 UTC m=+0.107129978 container create 80e0993b3b702919eb2a96b1a997815c7463c65afe695580e83d63562c53205f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mon-a, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=rishabh-2024mar8-7d63482, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-08T20:47:18.671 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 podman[31363]: 2024-03-08 20:47:18.323580507 +0000 UTC m=+0.031178441 image pull a272146cb810b1fd4a53627f019472132e30966d5c25b01bfc44a001295f1249 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761 2024-03-08T20:47:18.671 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 podman[31363]: 2024-03-08 20:47:18.619871013 +0000 UTC m=+0.327468991 container init 80e0993b3b702919eb2a96b1a997815c7463c65afe695580e83d63562c53205f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mon-a, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=rishabh-2024mar8-7d63482, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-08T20:47:18.671 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 podman[31363]: 2024-03-08 20:47:18.629675829 +0000 UTC m=+0.337273807 container start 80e0993b3b702919eb2a96b1a997815c7463c65afe695580e83d63562c53205f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mon-a, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, ceph=True, org.label-schema.vendor=CentOS, RELEASE=rishabh-2024mar8-7d63482, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-08T20:47:18.713 INFO:teuthology.orchestra.run.smithi107.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-03-08T20:47:18.980 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: set uid:gid to 167:167 (ceph:ceph) 2024-03-08T20:47:18.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: ceph version 19.0.0-1965-g7d63482a (7d63482aa89628d733e298984a762389b625c761) squid (dev), process ceph-mon, pid 2 2024-03-08T20:47:18.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: pidfile_write: ignore empty --pid-file 2024-03-08T20:47:18.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: load: jerasure load: lrc 2024-03-08T20:47:18.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: rocksdb: RocksDB version: 7.9.2 2024-03-08T20:47:18.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: rocksdb: Git sha 0 2024-03-08T20:47:18.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: rocksdb: Compile date 2024-03-08 15:06:51 2024-03-08T20:47:18.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: rocksdb: DB SUMMARY 2024-03-08T20:47:18.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: rocksdb: DB Session ID: WP486REQB4I8YR9W5IH5 2024-03-08T20:47:18.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: rocksdb: CURRENT file: CURRENT 2024-03-08T20:47:18.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: rocksdb: IDENTITY file: IDENTITY 2024-03-08T20:47:18.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-03-08T20:47:18.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-03-08T20:47:18.982 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 100354 ; 2024-03-08T20:47:18.982 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: rocksdb: Options.error_if_exists: 0 2024-03-08T20:47:18.982 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: rocksdb: Options.create_if_missing: 0 2024-03-08T20:47:18.982 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: rocksdb: Options.paranoid_checks: 1 2024-03-08T20:47:18.982 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: rocksdb: Options.flush_verify_memtable_count: 1 2024-03-08T20:47:18.982 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-03-08T20:47:18.982 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-03-08T20:47:18.982 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: rocksdb: Options.env: 0x556e88ceeb20 2024-03-08T20:47:18.982 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: rocksdb: Options.fs: PosixFileSystem 2024-03-08T20:47:18.982 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: rocksdb: Options.info_log: 0x556e89b6d920 2024-03-08T20:47:18.982 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: rocksdb: Options.max_file_opening_threads: 16 2024-03-08T20:47:18.982 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: rocksdb: Options.statistics: (nil) 2024-03-08T20:47:18.983 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: rocksdb: Options.use_fsync: 0 2024-03-08T20:47:18.983 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: rocksdb: Options.max_log_file_size: 0 2024-03-08T20:47:18.983 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-03-08T20:47:18.983 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: rocksdb: Options.log_file_time_to_roll: 0 2024-03-08T20:47:18.983 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: rocksdb: Options.keep_log_file_num: 1000 2024-03-08T20:47:18.983 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: rocksdb: Options.recycle_log_file_num: 0 2024-03-08T20:47:18.983 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: rocksdb: Options.allow_fallocate: 1 2024-03-08T20:47:18.983 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: rocksdb: Options.allow_mmap_reads: 0 2024-03-08T20:47:18.983 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: rocksdb: Options.allow_mmap_writes: 0 2024-03-08T20:47:18.983 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: rocksdb: Options.use_direct_reads: 0 2024-03-08T20:47:18.983 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-03-08T20:47:18.983 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: rocksdb: Options.create_missing_column_families: 0 2024-03-08T20:47:18.983 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: rocksdb: Options.db_log_dir: 2024-03-08T20:47:18.983 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: rocksdb: Options.wal_dir: 2024-03-08T20:47:18.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: rocksdb: Options.table_cache_numshardbits: 6 2024-03-08T20:47:18.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: rocksdb: Options.WAL_ttl_seconds: 0 2024-03-08T20:47:18.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: rocksdb: Options.WAL_size_limit_MB: 0 2024-03-08T20:47:18.985 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-03-08T20:47:18.985 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-03-08T20:47:18.985 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: rocksdb: Options.is_fd_close_on_exec: 1 2024-03-08T20:47:18.985 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: rocksdb: Options.advise_random_on_open: 1 2024-03-08T20:47:18.985 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: rocksdb: Options.db_write_buffer_size: 0 2024-03-08T20:47:18.985 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: rocksdb: Options.write_buffer_manager: 0x556e89b41400 2024-03-08T20:47:18.985 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-03-08T20:47:18.985 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-03-08T20:47:18.985 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: rocksdb: Options.use_adaptive_mutex: 0 2024-03-08T20:47:18.985 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: rocksdb: Options.rate_limiter: (nil) 2024-03-08T20:47:18.985 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-03-08T20:47:18.986 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: rocksdb: Options.wal_recovery_mode: 2 2024-03-08T20:47:18.986 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: rocksdb: Options.enable_thread_tracking: 0 2024-03-08T20:47:18.986 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: rocksdb: Options.enable_pipelined_write: 0 2024-03-08T20:47:18.986 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: rocksdb: Options.unordered_write: 0 2024-03-08T20:47:18.986 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-03-08T20:47:18.986 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-03-08T20:47:18.986 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-03-08T20:47:18.986 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-03-08T20:47:18.986 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: rocksdb: Options.row_cache: None 2024-03-08T20:47:18.986 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: rocksdb: Options.wal_filter: None 2024-03-08T20:47:18.986 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-03-08T20:47:18.987 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: rocksdb: Options.allow_ingest_behind: 0 2024-03-08T20:47:18.987 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: rocksdb: Options.two_write_queues: 0 2024-03-08T20:47:18.987 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: rocksdb: Options.manual_wal_flush: 0 2024-03-08T20:47:18.987 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: rocksdb: Options.wal_compression: 0 2024-03-08T20:47:18.987 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: rocksdb: Options.atomic_flush: 0 2024-03-08T20:47:18.987 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-03-08T20:47:18.987 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: rocksdb: Options.persist_stats_to_disk: 0 2024-03-08T20:47:18.987 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: rocksdb: Options.write_dbid_to_manifest: 0 2024-03-08T20:47:18.987 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: rocksdb: Options.log_readahead_size: 0 2024-03-08T20:47:18.988 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-03-08T20:47:18.988 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: rocksdb: Options.best_efforts_recovery: 0 2024-03-08T20:47:18.988 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-03-08T20:47:18.988 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-03-08T20:47:18.988 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: rocksdb: Options.allow_data_in_errors: 0 2024-03-08T20:47:18.988 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: rocksdb: Options.db_host_id: __hostname__ 2024-03-08T20:47:18.988 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: rocksdb: Options.enforce_single_del_contracts: true 2024-03-08T20:47:18.988 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: rocksdb: Options.max_background_jobs: 2 2024-03-08T20:47:18.988 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: rocksdb: Options.max_background_compactions: -1 2024-03-08T20:47:18.988 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: rocksdb: Options.max_subcompactions: 1 2024-03-08T20:47:18.989 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-03-08T20:47:18.989 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-03-08T20:47:18.989 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: rocksdb: Options.delayed_write_rate : 16777216 2024-03-08T20:47:18.989 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: rocksdb: Options.max_total_wal_size: 0 2024-03-08T20:47:18.989 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-03-08T20:47:18.989 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: rocksdb: Options.stats_dump_period_sec: 600 2024-03-08T20:47:18.990 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: rocksdb: Options.stats_persist_period_sec: 600 2024-03-08T20:47:18.990 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-03-08T20:47:18.990 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: rocksdb: Options.max_open_files: -1 2024-03-08T20:47:18.990 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: rocksdb: Options.bytes_per_sync: 0 2024-03-08T20:47:18.990 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: rocksdb: Options.wal_bytes_per_sync: 0 2024-03-08T20:47:18.990 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: rocksdb: Options.strict_bytes_per_sync: 0 2024-03-08T20:47:18.990 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: rocksdb: Options.compaction_readahead_size: 0 2024-03-08T20:47:18.990 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: rocksdb: Options.max_background_flushes: -1 2024-03-08T20:47:18.990 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: rocksdb: Compression algorithms supported: 2024-03-08T20:47:18.990 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-03-08T20:47:18.990 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: rocksdb: kZSTD supported: 0 2024-03-08T20:47:18.990 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: rocksdb: kXpressCompression supported: 0 2024-03-08T20:47:18.991 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: rocksdb: kLZ4HCCompression supported: 1 2024-03-08T20:47:18.991 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: rocksdb: kZlibCompression supported: 1 2024-03-08T20:47:18.991 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: rocksdb: kSnappyCompression supported: 1 2024-03-08T20:47:18.991 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: rocksdb: kLZ4Compression supported: 1 2024-03-08T20:47:18.991 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: rocksdb: kBZip2Compression supported: 0 2024-03-08T20:47:18.991 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: rocksdb: Fast CRC32 supported: Supported on x86 2024-03-08T20:47:18.991 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: rocksdb: DMutex implementation: pthread_mutex_t 2024-03-08T20:47:18.991 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-03-08T20:47:18.991 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-03-08T20:47:18.991 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-03-08T20:47:18.991 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: rocksdb: Options.merge_operator: 2024-03-08T20:47:18.991 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: rocksdb: Options.compaction_filter: None 2024-03-08T20:47:18.991 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: rocksdb: Options.compaction_filter_factory: None 2024-03-08T20:47:18.991 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: rocksdb: Options.sst_partitioner_factory: None 2024-03-08T20:47:18.991 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: rocksdb: Options.memtable_factory: SkipListFactory 2024-03-08T20:47:18.991 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: rocksdb: Options.table_factory: BlockBasedTable 2024-03-08T20:47:18.992 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x556e89b6da20) 2024-03-08T20:47:18.992 INFO:journalctl@ceph.mon.a.smithi107.stdout: cache_index_and_filter_blocks: 1 2024-03-08T20:47:18.992 INFO:journalctl@ceph.mon.a.smithi107.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-03-08T20:47:18.992 INFO:journalctl@ceph.mon.a.smithi107.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-03-08T20:47:18.992 INFO:journalctl@ceph.mon.a.smithi107.stdout: pin_top_level_index_and_filter: 1 2024-03-08T20:47:18.992 INFO:journalctl@ceph.mon.a.smithi107.stdout: index_type: 0 2024-03-08T20:47:18.992 INFO:journalctl@ceph.mon.a.smithi107.stdout: data_block_index_type: 0 2024-03-08T20:47:18.992 INFO:journalctl@ceph.mon.a.smithi107.stdout: index_shortening: 1 2024-03-08T20:47:18.992 INFO:journalctl@ceph.mon.a.smithi107.stdout: data_block_hash_table_util_ratio: 0.750000 2024-03-08T20:47:18.992 INFO:journalctl@ceph.mon.a.smithi107.stdout: checksum: 4 2024-03-08T20:47:18.992 INFO:journalctl@ceph.mon.a.smithi107.stdout: no_block_cache: 0 2024-03-08T20:47:18.992 INFO:journalctl@ceph.mon.a.smithi107.stdout: block_cache: 0x556e89b73090 2024-03-08T20:47:18.992 INFO:journalctl@ceph.mon.a.smithi107.stdout: block_cache_name: BinnedLRUCache 2024-03-08T20:47:18.992 INFO:journalctl@ceph.mon.a.smithi107.stdout: block_cache_options: 2024-03-08T20:47:18.993 INFO:journalctl@ceph.mon.a.smithi107.stdout: capacity : 536870912 2024-03-08T20:47:18.993 INFO:journalctl@ceph.mon.a.smithi107.stdout: num_shard_bits : 4 2024-03-08T20:47:18.993 INFO:journalctl@ceph.mon.a.smithi107.stdout: strict_capacity_limit : 0 2024-03-08T20:47:18.993 INFO:journalctl@ceph.mon.a.smithi107.stdout: high_pri_pool_ratio: 0.000 2024-03-08T20:47:18.993 INFO:journalctl@ceph.mon.a.smithi107.stdout: block_cache_compressed: (nil) 2024-03-08T20:47:18.993 INFO:journalctl@ceph.mon.a.smithi107.stdout: persistent_cache: (nil) 2024-03-08T20:47:18.993 INFO:journalctl@ceph.mon.a.smithi107.stdout: block_size: 4096 2024-03-08T20:47:18.993 INFO:journalctl@ceph.mon.a.smithi107.stdout: block_size_deviation: 10 2024-03-08T20:47:18.993 INFO:journalctl@ceph.mon.a.smithi107.stdout: block_restart_interval: 16 2024-03-08T20:47:18.993 INFO:journalctl@ceph.mon.a.smithi107.stdout: index_block_restart_interval: 1 2024-03-08T20:47:18.993 INFO:journalctl@ceph.mon.a.smithi107.stdout: metadata_block_size: 4096 2024-03-08T20:47:18.993 INFO:journalctl@ceph.mon.a.smithi107.stdout: partition_filters: 0 2024-03-08T20:47:18.993 INFO:journalctl@ceph.mon.a.smithi107.stdout: use_delta_encoding: 1 2024-03-08T20:47:18.993 INFO:journalctl@ceph.mon.a.smithi107.stdout: filter_policy: bloomfilter 2024-03-08T20:47:18.993 INFO:journalctl@ceph.mon.a.smithi107.stdout: whole_key_filtering: 1 2024-03-08T20:47:18.994 INFO:journalctl@ceph.mon.a.smithi107.stdout: verify_compression: 0 2024-03-08T20:47:18.994 INFO:journalctl@ceph.mon.a.smithi107.stdout: read_amp_bytes_per_bit: 0 2024-03-08T20:47:18.994 INFO:journalctl@ceph.mon.a.smithi107.stdout: format_version: 5 2024-03-08T20:47:18.994 INFO:journalctl@ceph.mon.a.smithi107.stdout: enable_index_compression: 1 2024-03-08T20:47:18.994 INFO:journalctl@ceph.mon.a.smithi107.stdout: block_align: 0 2024-03-08T20:47:18.994 INFO:journalctl@ceph.mon.a.smithi107.stdout: max_auto_readahead_size: 262144 2024-03-08T20:47:18.994 INFO:journalctl@ceph.mon.a.smithi107.stdout: prepopulate_block_cache: 0 2024-03-08T20:47:18.994 INFO:journalctl@ceph.mon.a.smithi107.stdout: initial_auto_readahead_size: 8192 2024-03-08T20:47:18.994 INFO:journalctl@ceph.mon.a.smithi107.stdout: num_file_reads_for_auto_readahead: 2 2024-03-08T20:47:18.994 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: rocksdb: Options.write_buffer_size: 33554432 2024-03-08T20:47:18.994 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: rocksdb: Options.max_write_buffer_number: 2 2024-03-08T20:47:18.994 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: rocksdb: Options.compression: NoCompression 2024-03-08T20:47:18.995 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: rocksdb: Options.bottommost_compression: Disabled 2024-03-08T20:47:18.995 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: rocksdb: Options.prefix_extractor: nullptr 2024-03-08T20:47:18.995 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-03-08T20:47:18.995 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: rocksdb: Options.num_levels: 7 2024-03-08T20:47:18.995 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-03-08T20:47:18.995 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-03-08T20:47:18.995 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-03-08T20:47:18.995 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-03-08T20:47:18.995 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-03-08T20:47:18.995 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-03-08T20:47:18.995 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-03-08T20:47:18.995 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-03-08T20:47:18.996 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-03-08T20:47:18.996 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-03-08T20:47:18.996 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-03-08T20:47:18.996 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-03-08T20:47:18.996 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: rocksdb: Options.compression_opts.window_bits: -14 2024-03-08T20:47:18.996 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: rocksdb: Options.compression_opts.level: 32767 2024-03-08T20:47:18.997 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: rocksdb: Options.compression_opts.strategy: 0 2024-03-08T20:47:18.997 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-03-08T20:47:18.997 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-03-08T20:47:18.997 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-03-08T20:47:18.997 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-03-08T20:47:18.997 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: rocksdb: Options.compression_opts.enabled: false 2024-03-08T20:47:18.997 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-03-08T20:47:18.997 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-03-08T20:47:18.997 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-03-08T20:47:18.997 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-03-08T20:47:18.997 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: rocksdb: Options.target_file_size_base: 67108864 2024-03-08T20:47:18.997 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: rocksdb: Options.target_file_size_multiplier: 1 2024-03-08T20:47:18.997 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-03-08T20:47:18.998 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-03-08T20:47:18.998 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-03-08T20:47:18.998 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-03-08T20:47:18.998 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-03-08T20:47:18.998 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-03-08T20:47:18.998 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-03-08T20:47:18.998 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-03-08T20:47:18.998 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-03-08T20:47:18.999 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-03-08T20:47:18.999 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-03-08T20:47:18.999 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-03-08T20:47:18.999 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-03-08T20:47:18.999 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: rocksdb: Options.arena_block_size: 1048576 2024-03-08T20:47:18.999 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-03-08T20:47:18.999 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-03-08T20:47:18.999 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: rocksdb: Options.disable_auto_compactions: 0 2024-03-08T20:47:18.999 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-03-08T20:47:18.999 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-03-08T20:47:18.999 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-03-08T20:47:18.999 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-03-08T20:47:18.999 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-03-08T20:47:18.999 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-03-08T20:47:18.999 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-03-08T20:47:19.000 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-03-08T20:47:19.000 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-03-08T20:47:19.000 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-03-08T20:47:19.000 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: rocksdb: Options.table_properties_collectors: 2024-03-08T20:47:19.000 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: rocksdb: Options.inplace_update_support: 0 2024-03-08T20:47:19.000 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: rocksdb: Options.inplace_update_num_locks: 10000 2024-03-08T20:47:19.000 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-03-08T20:47:19.000 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-03-08T20:47:19.000 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: rocksdb: Options.memtable_huge_page_size: 0 2024-03-08T20:47:19.000 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: rocksdb: Options.bloom_locality: 0 2024-03-08T20:47:19.000 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: rocksdb: Options.max_successive_merges: 0 2024-03-08T20:47:19.000 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: rocksdb: Options.optimize_filters_for_hits: 0 2024-03-08T20:47:19.000 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: rocksdb: Options.paranoid_file_checks: 0 2024-03-08T20:47:19.000 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: rocksdb: Options.force_consistency_checks: 1 2024-03-08T20:47:19.000 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: rocksdb: Options.report_bg_io_stats: 0 2024-03-08T20:47:19.000 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: rocksdb: Options.ttl: 2592000 2024-03-08T20:47:19.001 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: rocksdb: Options.periodic_compaction_seconds: 0 2024-03-08T20:47:19.001 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-03-08T20:47:19.001 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-03-08T20:47:19.001 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: rocksdb: Options.enable_blob_files: false 2024-03-08T20:47:19.001 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: rocksdb: Options.min_blob_size: 0 2024-03-08T20:47:19.001 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: rocksdb: Options.blob_file_size: 268435456 2024-03-08T20:47:19.001 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: rocksdb: Options.blob_compression_type: NoCompression 2024-03-08T20:47:19.001 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: rocksdb: Options.enable_blob_garbage_collection: false 2024-03-08T20:47:19.001 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-03-08T20:47:19.001 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-03-08T20:47:19.001 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-03-08T20:47:19.001 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: rocksdb: Options.blob_file_starting_level: 0 2024-03-08T20:47:19.001 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-03-08T20:47:19.001 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-03-08T20:47:19.001 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-03-08T20:47:19.002 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 79f9a859-e054-45de-87c3-f5c64a335f09 2024-03-08T20:47:19.002 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: rocksdb: EVENT_LOG_v1 {"time_micros": 1709930838686604, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-03-08T20:47:19.002 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-03-08T20:47:19.002 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: rocksdb: EVENT_LOG_v1 {"time_micros": 1709930838688455, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 95487, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 318, "table_properties": {"data_size": 93477, "index_size": 292, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 709, "raw_key_size": 13910, "raw_average_key_size": 50, "raw_value_size": 85915, "raw_average_value_size": 310, "num_data_blocks": 11, "num_entries": 277, "num_filter_entries": 277, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1709930838, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "79f9a859-e054-45de-87c3-f5c64a335f09", "db_session_id": "WP486REQB4I8YR9W5IH5", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-03-08T20:47:19.002 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: rocksdb: EVENT_LOG_v1 {"time_micros": 1709930838688617, "job": 1, "event": "recovery_finished"} 2024-03-08T20:47:19.002 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-03-08T20:47:19.002 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-03-08T20:47:19.002 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x556e89c66000 2024-03-08T20:47:19.002 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: rocksdb: DB pointer 0x556e89c50000 2024-03-08T20:47:19.002 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-03-08T20:47:19.002 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-03-08T20:47:19.003 INFO:journalctl@ceph.mon.a.smithi107.stdout: ** DB Stats ** 2024-03-08T20:47:19.003 INFO:journalctl@ceph.mon.a.smithi107.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-08T20:47:19.003 INFO:journalctl@ceph.mon.a.smithi107.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-03-08T20:47:19.003 INFO:journalctl@ceph.mon.a.smithi107.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-08T20:47:19.003 INFO:journalctl@ceph.mon.a.smithi107.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-08T20:47:19.003 INFO:journalctl@ceph.mon.a.smithi107.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-03-08T20:47:19.003 INFO:journalctl@ceph.mon.a.smithi107.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-08T20:47:19.003 INFO:journalctl@ceph.mon.a.smithi107.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-08T20:47:19.003 INFO:journalctl@ceph.mon.a.smithi107.stdout: 2024-03-08T20:47:19.003 INFO:journalctl@ceph.mon.a.smithi107.stdout: ** Compaction Stats [default] ** 2024-03-08T20:47:19.003 INFO:journalctl@ceph.mon.a.smithi107.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-08T20:47:19.003 INFO:journalctl@ceph.mon.a.smithi107.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-03-08T20:47:19.003 INFO:journalctl@ceph.mon.a.smithi107.stdout: L0 2/0 95.08 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 79.4 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-08T20:47:19.003 INFO:journalctl@ceph.mon.a.smithi107.stdout: Sum 2/0 95.08 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 79.4 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-08T20:47:19.004 INFO:journalctl@ceph.mon.a.smithi107.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 79.4 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-08T20:47:19.004 INFO:journalctl@ceph.mon.a.smithi107.stdout: 2024-03-08T20:47:19.004 INFO:journalctl@ceph.mon.a.smithi107.stdout: ** Compaction Stats [default] ** 2024-03-08T20:47:19.004 INFO:journalctl@ceph.mon.a.smithi107.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-08T20:47:19.004 INFO:journalctl@ceph.mon.a.smithi107.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-08T20:47:19.004 INFO:journalctl@ceph.mon.a.smithi107.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 79.4 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-08T20:47:19.004 INFO:journalctl@ceph.mon.a.smithi107.stdout: 2024-03-08T20:47:19.004 INFO:journalctl@ceph.mon.a.smithi107.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-03-08T20:47:19.004 INFO:journalctl@ceph.mon.a.smithi107.stdout: 2024-03-08T20:47:19.004 INFO:journalctl@ceph.mon.a.smithi107.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-08T20:47:19.004 INFO:journalctl@ceph.mon.a.smithi107.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-03-08T20:47:19.004 INFO:journalctl@ceph.mon.a.smithi107.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-03-08T20:47:19.005 INFO:journalctl@ceph.mon.a.smithi107.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-03-08T20:47:19.005 INFO:journalctl@ceph.mon.a.smithi107.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-03-08T20:47:19.005 INFO:journalctl@ceph.mon.a.smithi107.stdout: AddFile(Keys): cumulative 0, interval 0 2024-03-08T20:47:19.005 INFO:journalctl@ceph.mon.a.smithi107.stdout: Cumulative compaction: 0.00 GB write, 13.94 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-08T20:47:19.005 INFO:journalctl@ceph.mon.a.smithi107.stdout: Interval compaction: 0.00 GB write, 13.94 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-08T20:47:19.005 INFO:journalctl@ceph.mon.a.smithi107.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-03-08T20:47:19.005 INFO:journalctl@ceph.mon.a.smithi107.stdout: Block cache BinnedLRUCache@0x556e89b73090#2 capacity: 512.00 MB usage: 1.39 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.2e-05 secs_since: 0 2024-03-08T20:47:19.005 INFO:journalctl@ceph.mon.a.smithi107.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.89 KB,0.000169873%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-03-08T20:47:19.005 INFO:journalctl@ceph.mon.a.smithi107.stdout: 2024-03-08T20:47:19.005 INFO:journalctl@ceph.mon.a.smithi107.stdout: ** File Read Latency Histogram By Level [default] ** 2024-03-08T20:47:19.005 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: starting mon.a rank 0 at public addrs [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] at bind addrs [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid dc20f0f8-dd8c-11ee-95c6-87774f69a715 2024-03-08T20:47:19.005 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: mon.a@-1(???) e1 preinit fsid dc20f0f8-dd8c-11ee-95c6-87774f69a715 2024-03-08T20:47:19.005 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: mon.a@-1(???).mds e1 new map 2024-03-08T20:47:19.005 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: mon.a@-1(???).mds e1 print_map 2024-03-08T20:47:19.005 INFO:journalctl@ceph.mon.a.smithi107.stdout: e1 2024-03-08T20:47:19.005 INFO:journalctl@ceph.mon.a.smithi107.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-03-08T20:47:19.006 INFO:journalctl@ceph.mon.a.smithi107.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-03-08T20:47:19.006 INFO:journalctl@ceph.mon.a.smithi107.stdout: legacy client fscid: -1 2024-03-08T20:47:19.006 INFO:journalctl@ceph.mon.a.smithi107.stdout: 2024-03-08T20:47:19.006 INFO:journalctl@ceph.mon.a.smithi107.stdout: No filesystems configured 2024-03-08T20:47:19.006 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-03-08T20:47:19.006 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-03-08T20:47:19.006 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-03-08T20:47:19.006 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-03-08T20:47:19.006 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-03-08T20:47:19.006 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 bash[31363]: 80e0993b3b702919eb2a96b1a997815c7463c65afe695580e83d63562c53205f 2024-03-08T20:47:19.006 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 systemd[1]: Started Ceph mon.a for dc20f0f8-dd8c-11ee-95c6-87774f69a715. 2024-03-08T20:47:19.006 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: mon.a is new leader, mons a in quorum (ranks 0) 2024-03-08T20:47:19.006 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: monmap e1: 1 mons at {a=[v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-08T20:47:19.006 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: fsmap 2024-03-08T20:47:19.006 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: osdmap e1: 0 total, 0 up, 0 in 2024-03-08T20:47:19.007 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:18 smithi107 ceph-mon[31381]: mgrmap e1: no daemons active 2024-03-08T20:47:20.955 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:20.200+0000 7f2e7d200700 1 Processor -- start 2024-03-08T20:47:20.955 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:20.200+0000 7f2e7d200700 1 -- start start 2024-03-08T20:47:20.955 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:20.200+0000 7f2e7d200700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2e781099a0 0x7f2e78109dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:47:20.955 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:20.201+0000 7f2e7d200700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e7810a390 con 0x7f2e781099a0 2024-03-08T20:47:20.955 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:20.201+0000 7f2e75e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2e781099a0 0x7f2e78109dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:47:20.956 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:20.201+0000 7f2e75e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2e781099a0 0x7f2e78109dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:59436/0 (socket says 172.21.15.107:59436) 2024-03-08T20:47:20.956 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:20.201+0000 7f2e75e00700 1 -- 172.21.15.107:0/595477889 learned_addr learned my addr 172.21.15.107:0/595477889 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T20:47:20.956 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:20.202+0000 7f2e75e00700 1 -- 172.21.15.107:0/595477889 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e7810ab50 con 0x7f2e781099a0 2024-03-08T20:47:20.956 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:20.202+0000 7f2e75e00700 1 --2- 172.21.15.107:0/595477889 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2e781099a0 0x7f2e78109dc0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f2e680098a0 tx=0x7f2e68009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=897965955c153646 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:47:20.961 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:20.203+0000 7f2e74a00700 1 -- 172.21.15.107:0/595477889 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2e68004160 con 0x7f2e781099a0 2024-03-08T20:47:20.962 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:20.203+0000 7f2e74a00700 1 -- 172.21.15.107:0/595477889 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1071+0+0 (secure 0 0 0) 0x7f2e68004300 con 0x7f2e781099a0 2024-03-08T20:47:20.962 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:20.203+0000 7f2e74a00700 1 -- 172.21.15.107:0/595477889 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2e68011a80 con 0x7f2e781099a0 2024-03-08T20:47:20.962 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:20.203+0000 7f2e7d200700 1 -- 172.21.15.107:0/595477889 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2e781099a0 msgr2=0x7f2e78109dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:47:20.962 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:20.203+0000 7f2e7d200700 1 --2- 172.21.15.107:0/595477889 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2e781099a0 0x7f2e78109dc0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f2e680098a0 tx=0x7f2e68009bb0 comp rx=0 tx=0).stop 2024-03-08T20:47:20.962 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:20.203+0000 7f2e7d200700 1 -- 172.21.15.107:0/595477889 shutdown_connections 2024-03-08T20:47:20.962 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:20.203+0000 7f2e7d200700 1 --2- 172.21.15.107:0/595477889 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2e781099a0 0x7f2e78109dc0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:47:20.962 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:20.203+0000 7f2e7d200700 1 -- 172.21.15.107:0/595477889 >> 172.21.15.107:0/595477889 conn(0x7f2e7807cdf0 msgr2=0x7f2e781074f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:47:20.962 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:20.204+0000 7f2e7d200700 1 -- 172.21.15.107:0/595477889 shutdown_connections 2024-03-08T20:47:20.962 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:20.204+0000 7f2e7d200700 1 -- 172.21.15.107:0/595477889 wait complete. 2024-03-08T20:47:20.962 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:20.205+0000 7f2e7d200700 1 Processor -- start 2024-03-08T20:47:20.962 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:20.205+0000 7f2e7d200700 1 -- start start 2024-03-08T20:47:20.962 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:20.205+0000 7f2e7d200700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2e781099a0 0x7f2e7819d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:47:20.963 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:20.205+0000 7f2e7d200700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e7810df00 con 0x7f2e781099a0 2024-03-08T20:47:20.963 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:20.205+0000 7f2e75e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2e781099a0 0x7f2e7819d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:47:20.963 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:20.205+0000 7f2e75e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2e781099a0 0x7f2e7819d8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:59440/0 (socket says 172.21.15.107:59440) 2024-03-08T20:47:20.963 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:20.205+0000 7f2e75e00700 1 -- 172.21.15.107:0/4055793413 learned_addr learned my addr 172.21.15.107:0/4055793413 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T20:47:20.963 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:20.206+0000 7f2e75e00700 1 -- 172.21.15.107:0/4055793413 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e7819de20 con 0x7f2e781099a0 2024-03-08T20:47:20.963 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:20.206+0000 7f2e75e00700 1 --2- 172.21.15.107:0/4055793413 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2e781099a0 0x7f2e7819d8e0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f2e68000c00 tx=0x7f2e68004290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:47:20.963 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:20.206+0000 7f2e6ea00700 1 -- 172.21.15.107:0/4055793413 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2e680039c0 con 0x7f2e781099a0 2024-03-08T20:47:20.963 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:20.206+0000 7f2e6ea00700 1 -- 172.21.15.107:0/4055793413 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1071+0+0 (secure 0 0 0) 0x7f2e68003b60 con 0x7f2e781099a0 2024-03-08T20:47:20.963 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:20.206+0000 7f2e6ea00700 1 -- 172.21.15.107:0/4055793413 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2e6801a6f0 con 0x7f2e781099a0 2024-03-08T20:47:20.963 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:20.206+0000 7f2e7d200700 1 -- 172.21.15.107:0/4055793413 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e7819e0b0 con 0x7f2e781099a0 2024-03-08T20:47:20.963 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:20.207+0000 7f2e7d200700 1 -- 172.21.15.107:0/4055793413 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e781a0da0 con 0x7f2e781099a0 2024-03-08T20:47:20.963 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:20.207+0000 7f2e6ea00700 1 -- 172.21.15.107:0/4055793413 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f2e6802d3f0 con 0x7f2e781099a0 2024-03-08T20:47:20.964 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:20.207+0000 7f2e6ea00700 1 -- 172.21.15.107:0/4055793413 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f2e68019700 con 0x7f2e781099a0 2024-03-08T20:47:20.964 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:20.208+0000 7f2e7d200700 1 -- 172.21.15.107:0/4055793413 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e78063410 con 0x7f2e781099a0 2024-03-08T20:47:20.964 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:20.211+0000 7f2e6ea00700 1 -- 172.21.15.107:0/4055793413 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73265 (secure 0 0 0) 0x7f2e68024a40 con 0x7f2e781099a0 2024-03-08T20:47:20.964 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:20.286+0000 7f2e7d200700 1 -- 172.21.15.107:0/4055793413 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) v1 -- 0x7f2e781a0f70 con 0x7f2e781099a0 2024-03-08T20:47:20.964 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:20.291+0000 7f2e6ea00700 1 -- 172.21.15.107:0/4055793413 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) v1 ==== 124+0+0 (secure 0 0 0) 0x7f2e68032030 con 0x7f2e781099a0 2024-03-08T20:47:20.964 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:20.293+0000 7f2e7d200700 1 -- 172.21.15.107:0/4055793413 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2e781099a0 msgr2=0x7f2e7819d8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:47:20.964 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:20.293+0000 7f2e7d200700 1 --2- 172.21.15.107:0/4055793413 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2e781099a0 0x7f2e7819d8e0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f2e68000c00 tx=0x7f2e68004290 comp rx=0 tx=0).stop 2024-03-08T20:47:20.964 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:20.293+0000 7f2e7d200700 1 -- 172.21.15.107:0/4055793413 shutdown_connections 2024-03-08T20:47:20.964 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:20.293+0000 7f2e7d200700 1 --2- 172.21.15.107:0/4055793413 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2e781099a0 0x7f2e7819d8e0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:47:20.965 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:20.293+0000 7f2e7d200700 1 -- 172.21.15.107:0/4055793413 >> 172.21.15.107:0/4055793413 conn(0x7f2e7807cdf0 msgr2=0x7f2e78106c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:47:20.965 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:20.293+0000 7f2e7d200700 1 -- 172.21.15.107:0/4055793413 shutdown_connections 2024-03-08T20:47:20.965 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:20.294+0000 7f2e7d200700 1 -- 172.21.15.107:0/4055793413 wait complete. 2024-03-08T20:47:20.965 INFO:teuthology.orchestra.run.smithi107.stdout:Wrote config to /etc/ceph/ceph.conf 2024-03-08T20:47:21.002 INFO:teuthology.orchestra.run.smithi107.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-03-08T20:47:21.002 INFO:teuthology.orchestra.run.smithi107.stdout:Creating mgr... 2024-03-08T20:47:21.004 INFO:teuthology.orchestra.run.smithi107.stdout:Verifying port 0.0.0.0:9283 ... 2024-03-08T20:47:21.005 INFO:teuthology.orchestra.run.smithi107.stdout:Verifying port 0.0.0.0:8765 ... 2024-03-08T20:47:21.446 INFO:teuthology.orchestra.run.smithi107.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715@mgr.x 2024-03-08T20:47:21.446 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stderr Failed to reset failed state of unit ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715@mgr.x.service: Unit ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715@mgr.x.service not loaded. 2024-03-08T20:47:21.702 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715.target.wants/ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715@mgr.x.service → /etc/systemd/system/ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715@.service. 2024-03-08T20:47:21.729 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:21 smithi107 ceph-mon[31381]: from='client.? 172.21.15.107:0/4055793413' entity='client.admin' 2024-03-08T20:47:22.096 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 08 20:47:21 smithi107 systemd[1]: Starting Ceph mgr.x for dc20f0f8-dd8c-11ee-95c6-87774f69a715... 2024-03-08T20:47:22.370 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 08 20:47:22 smithi107 podman[31592]: 2024-03-08 20:47:22.006264866 +0000 UTC m=+0.035286792 image pull a272146cb810b1fd4a53627f019472132e30966d5c25b01bfc44a001295f1249 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761 2024-03-08T20:47:22.370 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 08 20:47:22 smithi107 podman[31592]: 2024-03-08 20:47:22.135485306 +0000 UTC m=+0.164507266 container create cec518020e7db1061653e72b3437af730f6aee622a32e17e9297c739964d334a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-x, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=rishabh-2024mar8-7d63482, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-03-08T20:47:22.370 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 08 20:47:22 smithi107 podman[31592]: 2024-03-08 20:47:22.374268283 +0000 UTC m=+0.403290245 container init cec518020e7db1061653e72b3437af730f6aee622a32e17e9297c739964d334a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-x, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=rishabh-2024mar8-7d63482, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-08T20:47:22.495 INFO:teuthology.orchestra.run.smithi107.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-08T20:47:22.495 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stdout disabled 2024-03-08T20:47:22.513 INFO:teuthology.orchestra.run.smithi107.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-08T20:47:22.514 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stdout inactive 2024-03-08T20:47:22.514 INFO:teuthology.orchestra.run.smithi107.stdout:firewalld.service is not enabled 2024-03-08T20:47:22.514 INFO:teuthology.orchestra.run.smithi107.stdout:Not possible to enable service . firewalld.service is not available 2024-03-08T20:47:22.536 INFO:teuthology.orchestra.run.smithi107.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-08T20:47:22.536 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stdout disabled 2024-03-08T20:47:22.555 INFO:teuthology.orchestra.run.smithi107.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-08T20:47:22.555 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stdout inactive 2024-03-08T20:47:22.556 INFO:teuthology.orchestra.run.smithi107.stdout:firewalld.service is not enabled 2024-03-08T20:47:22.556 INFO:teuthology.orchestra.run.smithi107.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-03-08T20:47:22.556 INFO:teuthology.orchestra.run.smithi107.stdout:Waiting for mgr to start... 2024-03-08T20:47:22.556 INFO:teuthology.orchestra.run.smithi107.stdout:Waiting for mgr... 2024-03-08T20:47:22.706 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 08 20:47:22 smithi107 podman[31592]: 2024-03-08 20:47:22.385981599 +0000 UTC m=+0.415003525 container start cec518020e7db1061653e72b3437af730f6aee622a32e17e9297c739964d334a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-x, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=rishabh-2024mar8-7d63482, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240307) 2024-03-08T20:47:22.706 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 08 20:47:22 smithi107 bash[31592]: cec518020e7db1061653e72b3437af730f6aee622a32e17e9297c739964d334a 2024-03-08T20:47:22.706 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 08 20:47:22 smithi107 systemd[1]: Started Ceph mgr.x for dc20f0f8-dd8c-11ee-95c6-87774f69a715. 2024-03-08T20:47:23.229 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 08 20:47:23 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-x[31606]: 2024-03-08T20:47:23.147+0000 7f466eaba200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-08T20:47:23.516 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:23 smithi107 ceph-mon[31381]: from='client.? 172.21.15.107:0/2719711665' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-08T20:47:23.516 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 08 20:47:23 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-x[31606]: 2024-03-08T20:47:23.290+0000 7f466eaba200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-08T20:47:23.947 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout 2024-03-08T20:47:23.947 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout { 2024-03-08T20:47:23.955 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "fsid": "dc20f0f8-dd8c-11ee-95c6-87774f69a715", 2024-03-08T20:47:23.955 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "health": { 2024-03-08T20:47:23.955 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-08T20:47:23.955 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-08T20:47:23.955 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-08T20:47:23.955 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-03-08T20:47:23.955 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-08T20:47:23.955 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-08T20:47:23.956 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout 0 2024-03-08T20:47:23.956 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout ], 2024-03-08T20:47:23.956 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-08T20:47:23.956 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "a" 2024-03-08T20:47:23.956 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout ], 2024-03-08T20:47:23.956 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "quorum_age": 4, 2024-03-08T20:47:23.956 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-08T20:47:23.956 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-08T20:47:23.956 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-03-08T20:47:23.957 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-08T20:47:23.957 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-03-08T20:47:23.957 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-08T20:47:23.957 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-08T20:47:23.957 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-08T20:47:23.957 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-08T20:47:23.957 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-08T20:47:23.957 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-08T20:47:23.957 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-08T20:47:23.957 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-08T20:47:23.957 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-03-08T20:47:23.957 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-08T20:47:23.957 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-08T20:47:23.957 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-08T20:47:23.958 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-08T20:47:23.958 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-08T20:47:23.958 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-08T20:47:23.958 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-08T20:47:23.958 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-08T20:47:23.958 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-08T20:47:23.958 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-03-08T20:47:23.958 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-08T20:47:23.958 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-08T20:47:23.958 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-08T20:47:23.958 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-08T20:47:23.958 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-03-08T20:47:23.958 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-08T20:47:23.958 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "available": false, 2024-03-08T20:47:23.959 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-08T20:47:23.959 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-08T20:47:23.959 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-08T20:47:23.959 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-08T20:47:23.959 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "restful" 2024-03-08T20:47:23.959 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout ], 2024-03-08T20:47:23.959 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-08T20:47:23.959 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-03-08T20:47:23.959 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-08T20:47:23.959 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-08T20:47:23.959 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "modified": "2024-03-08T20:47:04.190550+0000", 2024-03-08T20:47:23.959 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-08T20:47:23.959 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-03-08T20:47:23.959 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-08T20:47:23.959 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout } 2024-03-08T20:47:23.960 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:23.132+0000 7fcd84600700 1 Processor -- start 2024-03-08T20:47:23.960 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:23.133+0000 7fcd84600700 1 -- start start 2024-03-08T20:47:23.960 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:23.133+0000 7fcd84600700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcd7c108e70 0x7fcd7c109290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:47:23.960 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:23.133+0000 7fcd84600700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd7c107400 con 0x7fcd7c108e70 2024-03-08T20:47:23.960 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:23.133+0000 7fcd81c00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcd7c108e70 0x7fcd7c109290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:47:23.960 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:23.134+0000 7fcd81c00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcd7c108e70 0x7fcd7c109290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:59474/0 (socket says 172.21.15.107:59474) 2024-03-08T20:47:23.960 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:23.134+0000 7fcd81c00700 1 -- 172.21.15.107:0/3791874811 learned_addr learned my addr 172.21.15.107:0/3791874811 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T20:47:23.960 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:23.134+0000 7fcd81c00700 1 -- 172.21.15.107:0/3791874811 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd7c107580 con 0x7fcd7c108e70 2024-03-08T20:47:23.960 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:23.134+0000 7fcd81c00700 1 --2- 172.21.15.107:0/3791874811 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcd7c108e70 0x7fcd7c109290 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fcd700098a0 tx=0x7fcd70009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c73214f33535c40e server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:47:23.960 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:23.134+0000 7fcd7be00700 1 -- 172.21.15.107:0/3791874811 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcd70004160 con 0x7fcd7c108e70 2024-03-08T20:47:23.960 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:23.135+0000 7fcd7be00700 1 -- 172.21.15.107:0/3791874811 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1071+0+0 (secure 0 0 0) 0x7fcd70004300 con 0x7fcd7c108e70 2024-03-08T20:47:23.960 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:23.135+0000 7fcd7be00700 1 -- 172.21.15.107:0/3791874811 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcd70011a80 con 0x7fcd7c108e70 2024-03-08T20:47:23.964 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:23.135+0000 7fcd84600700 1 -- 172.21.15.107:0/3791874811 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcd7c108e70 msgr2=0x7fcd7c109290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:47:23.964 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:23.135+0000 7fcd84600700 1 --2- 172.21.15.107:0/3791874811 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcd7c108e70 0x7fcd7c109290 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fcd700098a0 tx=0x7fcd70009bb0 comp rx=0 tx=0).stop 2024-03-08T20:47:23.964 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:23.135+0000 7fcd84600700 1 -- 172.21.15.107:0/3791874811 shutdown_connections 2024-03-08T20:47:23.965 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:23.135+0000 7fcd84600700 1 --2- 172.21.15.107:0/3791874811 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcd7c108e70 0x7fcd7c109290 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:47:23.965 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:23.135+0000 7fcd84600700 1 -- 172.21.15.107:0/3791874811 >> 172.21.15.107:0/3791874811 conn(0x7fcd7c102da0 msgr2=0x7fcd7c1051c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:47:23.965 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:23.135+0000 7fcd84600700 1 -- 172.21.15.107:0/3791874811 shutdown_connections 2024-03-08T20:47:23.965 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:23.136+0000 7fcd84600700 1 -- 172.21.15.107:0/3791874811 wait complete. 2024-03-08T20:47:23.965 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:23.137+0000 7fcd84600700 1 Processor -- start 2024-03-08T20:47:23.965 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:23.137+0000 7fcd84600700 1 -- start start 2024-03-08T20:47:23.965 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:23.137+0000 7fcd84600700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcd7c108e70 0x7fcd7c07cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:47:23.965 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:23.137+0000 7fcd84600700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd7c10bd40 con 0x7fcd7c108e70 2024-03-08T20:47:23.965 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:23.137+0000 7fcd81c00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcd7c108e70 0x7fcd7c07cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:47:23.965 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:23.137+0000 7fcd81c00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcd7c108e70 0x7fcd7c07cb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:59486/0 (socket says 172.21.15.107:59486) 2024-03-08T20:47:23.965 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:23.137+0000 7fcd81c00700 1 -- 172.21.15.107:0/2719711665 learned_addr learned my addr 172.21.15.107:0/2719711665 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T20:47:23.965 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:23.138+0000 7fcd81c00700 1 -- 172.21.15.107:0/2719711665 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd7c07d0d0 con 0x7fcd7c108e70 2024-03-08T20:47:23.965 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:23.138+0000 7fcd81c00700 1 --2- 172.21.15.107:0/2719711665 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcd7c108e70 0x7fcd7c07cb90 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fcd70000c00 tx=0x7fcd70004290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:47:23.966 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:23.138+0000 7fcd7a000700 1 -- 172.21.15.107:0/2719711665 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcd700039c0 con 0x7fcd7c108e70 2024-03-08T20:47:23.966 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:23.138+0000 7fcd7a000700 1 -- 172.21.15.107:0/2719711665 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1071+0+0 (secure 0 0 0) 0x7fcd70003b60 con 0x7fcd7c108e70 2024-03-08T20:47:23.966 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:23.138+0000 7fcd7a000700 1 -- 172.21.15.107:0/2719711665 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcd7001a6f0 con 0x7fcd7c108e70 2024-03-08T20:47:23.966 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:23.138+0000 7fcd84600700 1 -- 172.21.15.107:0/2719711665 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd7c07b2c0 con 0x7fcd7c108e70 2024-03-08T20:47:23.966 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:23.138+0000 7fcd84600700 1 -- 172.21.15.107:0/2719711665 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd7c07b800 con 0x7fcd7c108e70 2024-03-08T20:47:23.966 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:23.139+0000 7fcd7a000700 1 -- 172.21.15.107:0/2719711665 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fcd7001a890 con 0x7fcd7c108e70 2024-03-08T20:47:23.966 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:23.139+0000 7fcd7a000700 1 -- 172.21.15.107:0/2719711665 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fcd7002dda0 con 0x7fcd7c108e70 2024-03-08T20:47:23.966 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:23.140+0000 7fcd84600700 1 -- 172.21.15.107:0/2719711665 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd7c10bec0 con 0x7fcd7c108e70 2024-03-08T20:47:23.966 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:23.144+0000 7fcd7a000700 1 -- 172.21.15.107:0/2719711665 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73265 (secure 0 0 0) 0x7fcd70004300 con 0x7fcd7c108e70 2024-03-08T20:47:23.966 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:23.235+0000 7fcd84600700 1 -- 172.21.15.107:0/2719711665 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fcd7c07bd00 con 0x7fcd7c108e70 2024-03-08T20:47:23.966 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:23.236+0000 7fcd7a000700 1 -- 172.21.15.107:0/2719711665 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1239 (secure 0 0 0) 0x7fcd70032020 con 0x7fcd7c108e70 2024-03-08T20:47:23.966 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:23.238+0000 7fcd84600700 1 -- 172.21.15.107:0/2719711665 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcd7c108e70 msgr2=0x7fcd7c07cb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:47:23.966 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:23.238+0000 7fcd84600700 1 --2- 172.21.15.107:0/2719711665 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcd7c108e70 0x7fcd7c07cb90 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fcd70000c00 tx=0x7fcd70004290 comp rx=0 tx=0).stop 2024-03-08T20:47:23.966 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:23.238+0000 7fcd84600700 1 -- 172.21.15.107:0/2719711665 shutdown_connections 2024-03-08T20:47:23.967 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:23.238+0000 7fcd84600700 1 --2- 172.21.15.107:0/2719711665 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcd7c108e70 0x7fcd7c07cb90 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:47:23.967 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:23.238+0000 7fcd84600700 1 -- 172.21.15.107:0/2719711665 >> 172.21.15.107:0/2719711665 conn(0x7fcd7c102da0 msgr2=0x7fcd7c1038f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:47:23.967 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:23.238+0000 7fcd84600700 1 -- 172.21.15.107:0/2719711665 shutdown_connections 2024-03-08T20:47:23.967 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:23.238+0000 7fcd84600700 1 -- 172.21.15.107:0/2719711665 wait complete. 2024-03-08T20:47:23.967 INFO:teuthology.orchestra.run.smithi107.stdout:mgr not available, waiting (1/15)... 2024-03-08T20:47:24.219 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 08 20:47:24 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-x[31606]: 2024-03-08T20:47:24.101+0000 7f466eaba200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-08T20:47:24.479 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 08 20:47:24 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-x[31606]: 2024-03-08T20:47:24.224+0000 7f466eaba200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-08T20:47:24.479 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 08 20:47:24 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-x[31606]: 2024-03-08T20:47:24.398+0000 7f466eaba200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-08T20:47:24.946 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 08 20:47:24 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-x[31606]: 2024-03-08T20:47:24.520+0000 7f466eaba200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-08T20:47:24.946 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 08 20:47:24 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-x[31606]: 2024-03-08T20:47:24.627+0000 7f466eaba200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-08T20:47:24.947 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 08 20:47:24 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-x[31606]: 2024-03-08T20:47:24.951+0000 7f466eaba200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-08T20:47:25.229 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 08 20:47:25 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-x[31606]: 2024-03-08T20:47:25.088+0000 7f466eaba200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-08T20:47:25.229 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 08 20:47:25 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-x[31606]: 2024-03-08T20:47:25.199+0000 7f466eaba200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-08T20:47:25.729 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 08 20:47:25 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-x[31606]: 2024-03-08T20:47:25.319+0000 7f466eaba200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-08T20:47:25.729 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 08 20:47:25 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-x[31606]: 2024-03-08T20:47:25.531+0000 7f466eaba200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-08T20:47:26.979 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:26 smithi107 ceph-mon[31381]: from='client.? 172.21.15.107:0/4020161618' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-08T20:47:26.979 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 08 20:47:26 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-x[31606]: 2024-03-08T20:47:26.918+0000 7f466eaba200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-08T20:47:27.475 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 08 20:47:27 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-x[31606]: 2024-03-08T20:47:27.099+0000 7f466eaba200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-08T20:47:27.487 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout 2024-03-08T20:47:27.487 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout { 2024-03-08T20:47:27.487 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "fsid": "dc20f0f8-dd8c-11ee-95c6-87774f69a715", 2024-03-08T20:47:27.487 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "health": { 2024-03-08T20:47:27.487 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-08T20:47:27.487 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-08T20:47:27.487 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-08T20:47:27.487 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-03-08T20:47:27.487 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-08T20:47:27.488 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-08T20:47:27.488 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout 0 2024-03-08T20:47:27.488 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout ], 2024-03-08T20:47:27.488 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-08T20:47:27.488 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "a" 2024-03-08T20:47:27.488 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout ], 2024-03-08T20:47:27.488 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "quorum_age": 7, 2024-03-08T20:47:27.488 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-08T20:47:27.488 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-08T20:47:27.488 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-03-08T20:47:27.488 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-08T20:47:27.488 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-03-08T20:47:27.488 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-08T20:47:27.488 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-08T20:47:27.489 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-08T20:47:27.489 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-08T20:47:27.497 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-08T20:47:27.497 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-08T20:47:27.497 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-08T20:47:27.497 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-08T20:47:27.498 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-03-08T20:47:27.498 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-08T20:47:27.498 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-08T20:47:27.498 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-08T20:47:27.498 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-08T20:47:27.498 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-08T20:47:27.498 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-08T20:47:27.498 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-08T20:47:27.498 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-08T20:47:27.498 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-08T20:47:27.498 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-03-08T20:47:27.498 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-08T20:47:27.498 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-08T20:47:27.499 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-08T20:47:27.499 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-08T20:47:27.499 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-03-08T20:47:27.499 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-08T20:47:27.499 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "available": false, 2024-03-08T20:47:27.499 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-08T20:47:27.499 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-08T20:47:27.499 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-08T20:47:27.499 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-08T20:47:27.499 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "restful" 2024-03-08T20:47:27.499 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout ], 2024-03-08T20:47:27.499 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-08T20:47:27.499 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-03-08T20:47:27.500 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-08T20:47:27.500 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-08T20:47:27.500 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "modified": "2024-03-08T20:47:04.190550+0000", 2024-03-08T20:47:27.500 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-08T20:47:27.500 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-03-08T20:47:27.500 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-08T20:47:27.500 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout } 2024-03-08T20:47:27.500 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:26.594+0000 7ff462600700 1 Processor -- start 2024-03-08T20:47:27.500 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:26.595+0000 7ff462600700 1 -- start start 2024-03-08T20:47:27.500 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:26.595+0000 7ff462600700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff45c07b340 0x7ff45c07b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:47:27.500 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:26.595+0000 7ff462600700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff45c07d020 con 0x7ff45c07b340 2024-03-08T20:47:27.500 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:26.595+0000 7ff45b400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff45c07b340 0x7ff45c07b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:47:27.501 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:26.596+0000 7ff45b400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff45c07b340 0x7ff45c07b760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:32842/0 (socket says 172.21.15.107:32842) 2024-03-08T20:47:27.501 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:26.596+0000 7ff45b400700 1 -- 172.21.15.107:0/3744687139 learned_addr learned my addr 172.21.15.107:0/3744687139 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T20:47:27.501 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:26.596+0000 7ff45b400700 1 -- 172.21.15.107:0/3744687139 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff45c07bca0 con 0x7ff45c07b340 2024-03-08T20:47:27.501 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:26.596+0000 7ff45b400700 1 --2- 172.21.15.107:0/3744687139 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff45c07b340 0x7ff45c07b760 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ff44c0098a0 tx=0x7ff44c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c641e1d4859e60c7 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:47:27.501 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:26.596+0000 7ff45a000700 1 -- 172.21.15.107:0/3744687139 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff44c004160 con 0x7ff45c07b340 2024-03-08T20:47:27.501 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:26.597+0000 7ff45a000700 1 -- 172.21.15.107:0/3744687139 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1071+0+0 (secure 0 0 0) 0x7ff44c004300 con 0x7ff45c07b340 2024-03-08T20:47:27.501 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:26.597+0000 7ff45a000700 1 -- 172.21.15.107:0/3744687139 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff44c011a80 con 0x7ff45c07b340 2024-03-08T20:47:27.501 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:26.597+0000 7ff462600700 1 -- 172.21.15.107:0/3744687139 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff45c07b340 msgr2=0x7ff45c07b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:47:27.501 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:26.597+0000 7ff462600700 1 --2- 172.21.15.107:0/3744687139 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff45c07b340 0x7ff45c07b760 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ff44c0098a0 tx=0x7ff44c009bb0 comp rx=0 tx=0).stop 2024-03-08T20:47:27.501 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:26.597+0000 7ff462600700 1 -- 172.21.15.107:0/3744687139 shutdown_connections 2024-03-08T20:47:27.501 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:26.597+0000 7ff462600700 1 --2- 172.21.15.107:0/3744687139 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff45c07b340 0x7ff45c07b760 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:47:27.501 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:26.597+0000 7ff462600700 1 -- 172.21.15.107:0/3744687139 >> 172.21.15.107:0/3744687139 conn(0x7ff45c102da0 msgr2=0x7ff45c1051c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:47:27.501 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:26.597+0000 7ff462600700 1 -- 172.21.15.107:0/3744687139 shutdown_connections 2024-03-08T20:47:27.502 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:26.597+0000 7ff462600700 1 -- 172.21.15.107:0/3744687139 wait complete. 2024-03-08T20:47:27.502 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:26.598+0000 7ff462600700 1 Processor -- start 2024-03-08T20:47:27.502 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:26.598+0000 7ff462600700 1 -- start start 2024-03-08T20:47:27.502 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:26.599+0000 7ff462600700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff45c07b340 0x7ff45c19da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:47:27.502 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:26.599+0000 7ff462600700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff45c10de90 con 0x7ff45c07b340 2024-03-08T20:47:27.502 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:26.599+0000 7ff45b400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff45c07b340 0x7ff45c19da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:47:27.502 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:26.599+0000 7ff45b400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff45c07b340 0x7ff45c19da40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:32846/0 (socket says 172.21.15.107:32846) 2024-03-08T20:47:27.502 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:26.599+0000 7ff45b400700 1 -- 172.21.15.107:0/4020161618 learned_addr learned my addr 172.21.15.107:0/4020161618 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T20:47:27.502 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:26.599+0000 7ff45b400700 1 -- 172.21.15.107:0/4020161618 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff45c19df80 con 0x7ff45c07b340 2024-03-08T20:47:27.502 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:26.600+0000 7ff45b400700 1 --2- 172.21.15.107:0/4020161618 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff45c07b340 0x7ff45c19da40 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ff44c011a50 tx=0x7ff44c004290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:47:27.502 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:26.600+0000 7ff453e00700 1 -- 172.21.15.107:0/4020161618 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff44c0039f0 con 0x7ff45c07b340 2024-03-08T20:47:27.502 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:26.600+0000 7ff453e00700 1 -- 172.21.15.107:0/4020161618 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1071+0+0 (secure 0 0 0) 0x7ff44c003b90 con 0x7ff45c07b340 2024-03-08T20:47:27.503 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:26.600+0000 7ff453e00700 1 -- 172.21.15.107:0/4020161618 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff44c01a600 con 0x7ff45c07b340 2024-03-08T20:47:27.503 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:26.600+0000 7ff462600700 1 -- 172.21.15.107:0/4020161618 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff45c19e210 con 0x7ff45c07b340 2024-03-08T20:47:27.503 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:26.600+0000 7ff462600700 1 -- 172.21.15.107:0/4020161618 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff45c1a0f00 con 0x7ff45c07b340 2024-03-08T20:47:27.503 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:26.600+0000 7ff453e00700 1 -- 172.21.15.107:0/4020161618 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7ff44c01ab30 con 0x7ff45c07b340 2024-03-08T20:47:27.503 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:26.600+0000 7ff453e00700 1 -- 172.21.15.107:0/4020161618 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7ff44c019700 con 0x7ff45c07b340 2024-03-08T20:47:27.503 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:26.601+0000 7ff462600700 1 -- 172.21.15.107:0/4020161618 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff45c063410 con 0x7ff45c07b340 2024-03-08T20:47:27.503 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:26.604+0000 7ff453e00700 1 -- 172.21.15.107:0/4020161618 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73265 (secure 0 0 0) 0x7ff44c024a40 con 0x7ff45c07b340 2024-03-08T20:47:27.503 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:26.692+0000 7ff462600700 1 -- 172.21.15.107:0/4020161618 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7ff45c1a12e0 con 0x7ff45c07b340 2024-03-08T20:47:27.503 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:26.694+0000 7ff453e00700 1 -- 172.21.15.107:0/4020161618 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1239 (secure 0 0 0) 0x7ff44c032030 con 0x7ff45c07b340 2024-03-08T20:47:27.503 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:26.695+0000 7ff462600700 1 -- 172.21.15.107:0/4020161618 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff45c07b340 msgr2=0x7ff45c19da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:47:27.503 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:26.695+0000 7ff462600700 1 --2- 172.21.15.107:0/4020161618 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff45c07b340 0x7ff45c19da40 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ff44c011a50 tx=0x7ff44c004290 comp rx=0 tx=0).stop 2024-03-08T20:47:27.504 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:26.696+0000 7ff462600700 1 -- 172.21.15.107:0/4020161618 shutdown_connections 2024-03-08T20:47:27.504 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:26.696+0000 7ff462600700 1 --2- 172.21.15.107:0/4020161618 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff45c07b340 0x7ff45c19da40 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:47:27.504 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:26.696+0000 7ff462600700 1 -- 172.21.15.107:0/4020161618 >> 172.21.15.107:0/4020161618 conn(0x7ff45c102da0 msgr2=0x7ff45c103910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:47:27.504 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:26.696+0000 7ff462600700 1 -- 172.21.15.107:0/4020161618 shutdown_connections 2024-03-08T20:47:27.504 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:26.696+0000 7ff462600700 1 -- 172.21.15.107:0/4020161618 wait complete. 2024-03-08T20:47:27.504 INFO:teuthology.orchestra.run.smithi107.stdout:mgr not available, waiting (2/15)... 2024-03-08T20:47:27.729 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 08 20:47:27 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-x[31606]: 2024-03-08T20:47:27.514+0000 7f466eaba200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-08T20:47:27.729 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 08 20:47:27 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-x[31606]: 2024-03-08T20:47:27.629+0000 7f466eaba200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-08T20:47:28.229 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 08 20:47:27 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-x[31606]: 2024-03-08T20:47:27.875+0000 7f466eaba200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-08T20:47:28.229 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 08 20:47:28 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-x[31606]: 2024-03-08T20:47:28.111+0000 7f466eaba200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-08T20:47:28.729 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 08 20:47:28 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-x[31606]: 2024-03-08T20:47:28.472+0000 7f466eaba200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-08T20:47:29.072 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 08 20:47:28 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-x[31606]: 2024-03-08T20:47:28.783+0000 7f466eaba200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-08T20:47:29.073 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 08 20:47:29 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-x[31606]: 2024-03-08T20:47:29.077+0000 7f466eaba200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-08T20:47:30.174 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 08 20:47:30 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-x[31606]: 2024-03-08T20:47:30.112+0000 7f466eaba200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-08T20:47:30.478 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:30 smithi107 ceph-mon[31381]: from='client.? 172.21.15.107:0/805431495' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-08T20:47:30.872 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout 2024-03-08T20:47:30.873 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout { 2024-03-08T20:47:30.873 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "fsid": "dc20f0f8-dd8c-11ee-95c6-87774f69a715", 2024-03-08T20:47:30.873 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "health": { 2024-03-08T20:47:30.873 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-08T20:47:30.873 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-08T20:47:30.873 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-08T20:47:30.873 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-03-08T20:47:30.873 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-08T20:47:30.873 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-08T20:47:30.873 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout 0 2024-03-08T20:47:30.873 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout ], 2024-03-08T20:47:30.873 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-08T20:47:30.873 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "a" 2024-03-08T20:47:30.873 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout ], 2024-03-08T20:47:30.874 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "quorum_age": 11, 2024-03-08T20:47:30.874 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-08T20:47:30.874 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-08T20:47:30.874 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-03-08T20:47:30.874 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-08T20:47:30.874 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-03-08T20:47:30.874 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-08T20:47:30.874 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-08T20:47:30.874 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-08T20:47:30.874 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-08T20:47:30.874 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-08T20:47:30.874 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-08T20:47:30.874 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-08T20:47:30.875 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-08T20:47:30.875 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-03-08T20:47:30.875 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-08T20:47:30.875 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-08T20:47:30.875 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-08T20:47:30.875 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-08T20:47:30.875 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-08T20:47:30.875 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-08T20:47:30.875 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-08T20:47:30.875 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-08T20:47:30.875 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-08T20:47:30.875 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-03-08T20:47:30.875 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-08T20:47:30.875 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-08T20:47:30.876 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-08T20:47:30.876 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-08T20:47:30.876 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-03-08T20:47:30.876 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-08T20:47:30.876 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "available": false, 2024-03-08T20:47:30.876 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-08T20:47:30.876 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-08T20:47:30.876 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-08T20:47:30.876 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-08T20:47:30.876 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "restful" 2024-03-08T20:47:30.876 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout ], 2024-03-08T20:47:30.876 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-08T20:47:30.876 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-03-08T20:47:30.876 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-08T20:47:30.877 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-08T20:47:30.877 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "modified": "2024-03-08T20:47:04.190550+0000", 2024-03-08T20:47:30.877 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-08T20:47:30.877 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-03-08T20:47:30.877 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-08T20:47:30.877 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout } 2024-03-08T20:47:30.877 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:30.075+0000 7f9cad000700 1 Processor -- start 2024-03-08T20:47:30.877 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:30.075+0000 7f9cad000700 1 -- start start 2024-03-08T20:47:30.877 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:30.075+0000 7f9cad000700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9ca807b320 0x7f9ca807b740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:47:30.877 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:30.075+0000 7f9cad000700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ca807d000 con 0x7f9ca807b320 2024-03-08T20:47:30.877 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:30.076+0000 7f9ca5e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9ca807b320 0x7f9ca807b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:47:30.877 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:30.076+0000 7f9ca5e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9ca807b320 0x7f9ca807b740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:32852/0 (socket says 172.21.15.107:32852) 2024-03-08T20:47:30.878 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:30.076+0000 7f9ca5e00700 1 -- 172.21.15.107:0/2057856851 learned_addr learned my addr 172.21.15.107:0/2057856851 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T20:47:30.878 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:30.076+0000 7f9ca5e00700 1 -- 172.21.15.107:0/2057856851 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ca807bc80 con 0x7f9ca807b320 2024-03-08T20:47:30.878 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:30.077+0000 7f9ca5e00700 1 --2- 172.21.15.107:0/2057856851 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9ca807b320 0x7f9ca807b740 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f9c980098a0 tx=0x7f9c98009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c84e7bc29007abad server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:47:30.878 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:30.077+0000 7f9ca4a00700 1 -- 172.21.15.107:0/2057856851 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9c98004160 con 0x7f9ca807b320 2024-03-08T20:47:30.878 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:30.077+0000 7f9ca4a00700 1 -- 172.21.15.107:0/2057856851 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1071+0+0 (secure 0 0 0) 0x7f9c98004300 con 0x7f9ca807b320 2024-03-08T20:47:30.878 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:30.077+0000 7f9ca4a00700 1 -- 172.21.15.107:0/2057856851 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9c98011a80 con 0x7f9ca807b320 2024-03-08T20:47:30.878 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:30.077+0000 7f9cad000700 1 -- 172.21.15.107:0/2057856851 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9ca807b320 msgr2=0x7f9ca807b740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:47:30.878 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:30.077+0000 7f9cad000700 1 --2- 172.21.15.107:0/2057856851 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9ca807b320 0x7f9ca807b740 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f9c980098a0 tx=0x7f9c98009bb0 comp rx=0 tx=0).stop 2024-03-08T20:47:30.878 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:30.078+0000 7f9cad000700 1 -- 172.21.15.107:0/2057856851 shutdown_connections 2024-03-08T20:47:30.878 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:30.078+0000 7f9cad000700 1 --2- 172.21.15.107:0/2057856851 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9ca807b320 0x7f9ca807b740 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:47:30.878 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:30.078+0000 7f9cad000700 1 -- 172.21.15.107:0/2057856851 >> 172.21.15.107:0/2057856851 conn(0x7f9ca8102da0 msgr2=0x7f9ca81051c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:47:30.878 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:30.078+0000 7f9cad000700 1 -- 172.21.15.107:0/2057856851 shutdown_connections 2024-03-08T20:47:30.878 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:30.078+0000 7f9cad000700 1 -- 172.21.15.107:0/2057856851 wait complete. 2024-03-08T20:47:30.879 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:30.079+0000 7f9cad000700 1 Processor -- start 2024-03-08T20:47:30.879 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:30.079+0000 7f9cad000700 1 -- start start 2024-03-08T20:47:30.879 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:30.079+0000 7f9cad000700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9ca807b320 0x7f9ca819da30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:47:30.879 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:30.079+0000 7f9cad000700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ca810de50 con 0x7f9ca807b320 2024-03-08T20:47:30.879 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:30.079+0000 7f9ca5e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9ca807b320 0x7f9ca819da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:47:30.879 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:30.079+0000 7f9ca5e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9ca807b320 0x7f9ca819da30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:32856/0 (socket says 172.21.15.107:32856) 2024-03-08T20:47:30.879 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:30.079+0000 7f9ca5e00700 1 -- 172.21.15.107:0/805431495 learned_addr learned my addr 172.21.15.107:0/805431495 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T20:47:30.879 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:30.080+0000 7f9ca5e00700 1 -- 172.21.15.107:0/805431495 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ca819df70 con 0x7f9ca807b320 2024-03-08T20:47:30.879 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:30.080+0000 7f9ca5e00700 1 --2- 172.21.15.107:0/805431495 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9ca807b320 0x7f9ca819da30 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f9c98011a50 tx=0x7f9c98004290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:47:30.879 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:30.080+0000 7f9c9ea00700 1 -- 172.21.15.107:0/805431495 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9c980039f0 con 0x7f9ca807b320 2024-03-08T20:47:30.879 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:30.080+0000 7f9c9ea00700 1 -- 172.21.15.107:0/805431495 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1071+0+0 (secure 0 0 0) 0x7f9c98003b90 con 0x7f9ca807b320 2024-03-08T20:47:30.879 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:30.080+0000 7f9c9ea00700 1 -- 172.21.15.107:0/805431495 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9c9801a600 con 0x7f9ca807b320 2024-03-08T20:47:30.879 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:30.080+0000 7f9cad000700 1 -- 172.21.15.107:0/805431495 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ca819e200 con 0x7f9ca807b320 2024-03-08T20:47:30.880 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:30.081+0000 7f9cad000700 1 -- 172.21.15.107:0/805431495 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ca81a0ef0 con 0x7f9ca807b320 2024-03-08T20:47:30.880 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:30.081+0000 7f9c9ea00700 1 -- 172.21.15.107:0/805431495 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f9c9801ac40 con 0x7f9ca807b320 2024-03-08T20:47:30.880 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:30.081+0000 7f9c9ea00700 1 -- 172.21.15.107:0/805431495 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f9c98019700 con 0x7f9ca807b320 2024-03-08T20:47:30.880 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:30.082+0000 7f9cad000700 1 -- 172.21.15.107:0/805431495 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ca8063410 con 0x7f9ca807b320 2024-03-08T20:47:30.880 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:30.084+0000 7f9c9ea00700 1 -- 172.21.15.107:0/805431495 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73265 (secure 0 0 0) 0x7f9c98024a40 con 0x7f9ca807b320 2024-03-08T20:47:30.880 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:30.162+0000 7f9cad000700 1 -- 172.21.15.107:0/805431495 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f9ca81a1260 con 0x7f9ca807b320 2024-03-08T20:47:30.880 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:30.163+0000 7f9c9ea00700 1 -- 172.21.15.107:0/805431495 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1240 (secure 0 0 0) 0x7f9c98024440 con 0x7f9ca807b320 2024-03-08T20:47:30.880 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:30.164+0000 7f9cad000700 1 -- 172.21.15.107:0/805431495 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9ca807b320 msgr2=0x7f9ca819da30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:47:30.880 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:30.164+0000 7f9cad000700 1 --2- 172.21.15.107:0/805431495 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9ca807b320 0x7f9ca819da30 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f9c98011a50 tx=0x7f9c98004290 comp rx=0 tx=0).stop 2024-03-08T20:47:30.880 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:30.164+0000 7f9cad000700 1 -- 172.21.15.107:0/805431495 shutdown_connections 2024-03-08T20:47:30.880 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:30.164+0000 7f9cad000700 1 --2- 172.21.15.107:0/805431495 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9ca807b320 0x7f9ca819da30 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:47:30.880 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:30.164+0000 7f9cad000700 1 -- 172.21.15.107:0/805431495 >> 172.21.15.107:0/805431495 conn(0x7f9ca8102da0 msgr2=0x7f9ca8103910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:47:30.880 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:30.164+0000 7f9cad000700 1 -- 172.21.15.107:0/805431495 shutdown_connections 2024-03-08T20:47:30.881 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:30.164+0000 7f9cad000700 1 -- 172.21.15.107:0/805431495 wait complete. 2024-03-08T20:47:30.881 INFO:teuthology.orchestra.run.smithi107.stdout:mgr not available, waiting (3/15)... 2024-03-08T20:47:31.229 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 08 20:47:30 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-x[31606]: 2024-03-08T20:47:30.827+0000 7f466eaba200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-08T20:47:31.229 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 08 20:47:30 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-x[31606]: 2024-03-08T20:47:30.959+0000 7f466eaba200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-08T20:47:31.729 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 08 20:47:31 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-x[31606]: 2024-03-08T20:47:31.234+0000 7f466eaba200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-08T20:47:31.729 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:31 smithi107 ceph-mon[31381]: Activating manager daemon x 2024-03-08T20:47:31.729 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:31 smithi107 ceph-mon[31381]: mgrmap e2: x(active, starting, since 0.0110016s) 2024-03-08T20:47:31.729 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:31 smithi107 ceph-mon[31381]: from='mgr.14100 172.21.15.107:0/3059430597' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-08T20:47:31.729 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:31 smithi107 ceph-mon[31381]: from='mgr.14100 172.21.15.107:0/3059430597' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-08T20:47:31.729 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:31 smithi107 ceph-mon[31381]: from='mgr.14100 172.21.15.107:0/3059430597' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-08T20:47:31.730 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:31 smithi107 ceph-mon[31381]: from='mgr.14100 172.21.15.107:0/3059430597' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-08T20:47:31.730 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:31 smithi107 ceph-mon[31381]: from='mgr.14100 172.21.15.107:0/3059430597' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-08T20:47:31.730 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:31 smithi107 ceph-mon[31381]: Manager daemon x is now available 2024-03-08T20:47:31.730 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:31 smithi107 ceph-mon[31381]: from='mgr.14100 172.21.15.107:0/3059430597' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-08T20:47:31.730 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:31 smithi107 ceph-mon[31381]: from='mgr.14100 172.21.15.107:0/3059430597' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-08T20:47:31.730 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:31 smithi107 ceph-mon[31381]: from='mgr.14100 172.21.15.107:0/3059430597' entity='mgr.x' 2024-03-08T20:47:31.730 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:31 smithi107 ceph-mon[31381]: from='mgr.14100 172.21.15.107:0/3059430597' entity='mgr.x' 2024-03-08T20:47:32.729 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:32 smithi107 ceph-mon[31381]: from='mgr.14100 172.21.15.107:0/3059430597' entity='mgr.x' 2024-03-08T20:47:32.729 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:32 smithi107 ceph-mon[31381]: mgrmap e3: x(active, since 1.02222s) 2024-03-08T20:47:34.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:34 smithi107 ceph-mon[31381]: mgrmap e4: x(active, since 2s) 2024-03-08T20:47:34.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:34 smithi107 ceph-mon[31381]: from='client.? 172.21.15.107:0/3727671691' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-08T20:47:34.448 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout 2024-03-08T20:47:34.448 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout { 2024-03-08T20:47:34.448 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "fsid": "dc20f0f8-dd8c-11ee-95c6-87774f69a715", 2024-03-08T20:47:34.459 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "health": { 2024-03-08T20:47:34.459 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-08T20:47:34.459 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-08T20:47:34.459 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-08T20:47:34.459 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-03-08T20:47:34.459 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-08T20:47:34.459 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-08T20:47:34.459 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout 0 2024-03-08T20:47:34.459 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout ], 2024-03-08T20:47:34.459 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-08T20:47:34.460 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "a" 2024-03-08T20:47:34.460 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout ], 2024-03-08T20:47:34.460 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "quorum_age": 15, 2024-03-08T20:47:34.460 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-08T20:47:34.460 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-08T20:47:34.460 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-03-08T20:47:34.460 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-08T20:47:34.460 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-03-08T20:47:34.460 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-08T20:47:34.460 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-08T20:47:34.460 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-08T20:47:34.460 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-08T20:47:34.460 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-08T20:47:34.460 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-08T20:47:34.461 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-08T20:47:34.461 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-08T20:47:34.461 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-03-08T20:47:34.461 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-08T20:47:34.461 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-08T20:47:34.461 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-08T20:47:34.461 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-08T20:47:34.461 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-08T20:47:34.461 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-08T20:47:34.461 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-08T20:47:34.461 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-08T20:47:34.461 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-08T20:47:34.461 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-03-08T20:47:34.462 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-08T20:47:34.462 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-08T20:47:34.462 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-08T20:47:34.462 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-08T20:47:34.462 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-03-08T20:47:34.462 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-08T20:47:34.462 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "available": true, 2024-03-08T20:47:34.462 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-08T20:47:34.462 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-08T20:47:34.462 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-08T20:47:34.462 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-08T20:47:34.462 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "restful" 2024-03-08T20:47:34.462 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout ], 2024-03-08T20:47:34.463 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-08T20:47:34.463 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-03-08T20:47:34.463 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-08T20:47:34.463 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-08T20:47:34.463 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "modified": "2024-03-08T20:47:04.190550+0000", 2024-03-08T20:47:34.463 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-08T20:47:34.463 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-03-08T20:47:34.463 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-08T20:47:34.463 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout } 2024-03-08T20:47:34.463 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:33.463+0000 7fde68000700 1 Processor -- start 2024-03-08T20:47:34.463 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:33.463+0000 7fde68000700 1 -- start start 2024-03-08T20:47:34.463 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:33.464+0000 7fde68000700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fde60105ff0 0x7fde60108420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:47:34.463 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:33.464+0000 7fde68000700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde6007ae70 con 0x7fde60105ff0 2024-03-08T20:47:34.463 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:33.464+0000 7fde65600700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fde60105ff0 0x7fde60108420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:47:34.463 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:33.464+0000 7fde65600700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fde60105ff0 0x7fde60108420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:32926/0 (socket says 172.21.15.107:32926) 2024-03-08T20:47:34.464 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:33.464+0000 7fde65600700 1 -- 172.21.15.107:0/3624682104 learned_addr learned my addr 172.21.15.107:0/3624682104 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T20:47:34.464 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:33.465+0000 7fde65600700 1 -- 172.21.15.107:0/3624682104 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde60108960 con 0x7fde60105ff0 2024-03-08T20:47:34.464 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:33.465+0000 7fde65600700 1 --2- 172.21.15.107:0/3624682104 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fde60105ff0 0x7fde60108420 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fde540098a0 tx=0x7fde54009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a0902c88309c097 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:47:34.464 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:33.465+0000 7fde5fe00700 1 -- 172.21.15.107:0/3624682104 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fde54004160 con 0x7fde60105ff0 2024-03-08T20:47:34.464 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:33.466+0000 7fde5fe00700 1 -- 172.21.15.107:0/3624682104 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1071+0+0 (secure 0 0 0) 0x7fde54004300 con 0x7fde60105ff0 2024-03-08T20:47:34.464 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:33.466+0000 7fde5fe00700 1 -- 172.21.15.107:0/3624682104 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fde54011a80 con 0x7fde60105ff0 2024-03-08T20:47:34.464 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:33.466+0000 7fde68000700 1 -- 172.21.15.107:0/3624682104 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fde60105ff0 msgr2=0x7fde60108420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:47:34.464 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:33.466+0000 7fde68000700 1 --2- 172.21.15.107:0/3624682104 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fde60105ff0 0x7fde60108420 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fde540098a0 tx=0x7fde54009bb0 comp rx=0 tx=0).stop 2024-03-08T20:47:34.464 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:33.466+0000 7fde68000700 1 -- 172.21.15.107:0/3624682104 shutdown_connections 2024-03-08T20:47:34.464 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:33.466+0000 7fde68000700 1 --2- 172.21.15.107:0/3624682104 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fde60105ff0 0x7fde60108420 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:47:34.464 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:33.466+0000 7fde68000700 1 -- 172.21.15.107:0/3624682104 >> 172.21.15.107:0/3624682104 conn(0x7fde60101d10 msgr2=0x7fde60104130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:47:34.464 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:33.467+0000 7fde68000700 1 -- 172.21.15.107:0/3624682104 shutdown_connections 2024-03-08T20:47:34.464 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:33.467+0000 7fde68000700 1 -- 172.21.15.107:0/3624682104 wait complete. 2024-03-08T20:47:34.465 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:33.468+0000 7fde68000700 1 Processor -- start 2024-03-08T20:47:34.465 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:33.468+0000 7fde68000700 1 -- start start 2024-03-08T20:47:34.465 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:33.468+0000 7fde68000700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fde60105ff0 0x7fde601a1c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:47:34.465 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:33.468+0000 7fde68000700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde60109990 con 0x7fde60105ff0 2024-03-08T20:47:34.465 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:33.468+0000 7fde65600700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fde60105ff0 0x7fde601a1c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:47:34.465 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:33.468+0000 7fde65600700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fde60105ff0 0x7fde601a1c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:32930/0 (socket says 172.21.15.107:32930) 2024-03-08T20:47:34.465 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:33.468+0000 7fde65600700 1 -- 172.21.15.107:0/3727671691 learned_addr learned my addr 172.21.15.107:0/3727671691 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T20:47:34.465 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:33.469+0000 7fde65600700 1 -- 172.21.15.107:0/3727671691 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde601a21c0 con 0x7fde60105ff0 2024-03-08T20:47:34.465 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:33.469+0000 7fde65600700 1 --2- 172.21.15.107:0/3727671691 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fde60105ff0 0x7fde601a1c80 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fde54006af0 tx=0x7fde54011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:47:34.465 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:33.469+0000 7fde5e000700 1 -- 172.21.15.107:0/3727671691 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fde540039e0 con 0x7fde60105ff0 2024-03-08T20:47:34.465 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:33.469+0000 7fde5e000700 1 -- 172.21.15.107:0/3727671691 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1071+0+0 (secure 0 0 0) 0x7fde54003b80 con 0x7fde60105ff0 2024-03-08T20:47:34.465 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:33.469+0000 7fde5e000700 1 -- 172.21.15.107:0/3727671691 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fde540196a0 con 0x7fde60105ff0 2024-03-08T20:47:34.465 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:33.469+0000 7fde68000700 1 -- 172.21.15.107:0/3727671691 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde601a2450 con 0x7fde60105ff0 2024-03-08T20:47:34.466 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:33.469+0000 7fde68000700 1 -- 172.21.15.107:0/3727671691 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde601a5140 con 0x7fde60105ff0 2024-03-08T20:47:34.466 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:33.470+0000 7fde68000700 1 -- 172.21.15.107:0/3727671691 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde60063410 con 0x7fde60105ff0 2024-03-08T20:47:34.466 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:33.470+0000 7fde5e000700 1 -- 172.21.15.107:0/3727671691 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 4) v1 ==== 46095+0+0 (secure 0 0 0) 0x7fde5401ab50 con 0x7fde60105ff0 2024-03-08T20:47:34.466 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:33.471+0000 7fde5e000700 1 --2- 172.21.15.107:0/3727671691 >> [v2:172.21.15.107:6800/3549936519,v1:172.21.15.107:6801/3549936519] conn(0x7fde48038ed0 0x7fde4803b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:47:34.466 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:33.471+0000 7fde5e000700 1 -- 172.21.15.107:0/3727671691 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fde5404d360 con 0x7fde60105ff0 2024-03-08T20:47:34.466 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:33.471+0000 7fde64c00700 1 --2- 172.21.15.107:0/3727671691 >> [v2:172.21.15.107:6800/3549936519,v1:172.21.15.107:6801/3549936519] conn(0x7fde48038ed0 0x7fde4803b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:47:34.466 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:33.472+0000 7fde64c00700 1 --2- 172.21.15.107:0/3727671691 >> [v2:172.21.15.107:6800/3549936519,v1:172.21.15.107:6801/3549936519] conn(0x7fde48038ed0 0x7fde4803b390 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fde4c006fd0 tx=0x7fde4c006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:47:34.466 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:33.478+0000 7fde5e000700 1 -- 172.21.15.107:0/3727671691 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7fde5404cd80 con 0x7fde60105ff0 2024-03-08T20:47:34.466 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:33.783+0000 7fde68000700 1 -- 172.21.15.107:0/3727671691 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fde6002daa0 con 0x7fde60105ff0 2024-03-08T20:47:34.466 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:33.784+0000 7fde5e000700 1 -- 172.21.15.107:0/3727671691 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1239 (secure 0 0 0) 0x7fde5401a430 con 0x7fde60105ff0 2024-03-08T20:47:34.466 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:33.787+0000 7fde68000700 1 -- 172.21.15.107:0/3727671691 >> [v2:172.21.15.107:6800/3549936519,v1:172.21.15.107:6801/3549936519] conn(0x7fde48038ed0 msgr2=0x7fde4803b390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:47:34.466 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:33.787+0000 7fde68000700 1 --2- 172.21.15.107:0/3727671691 >> [v2:172.21.15.107:6800/3549936519,v1:172.21.15.107:6801/3549936519] conn(0x7fde48038ed0 0x7fde4803b390 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fde4c006fd0 tx=0x7fde4c006e40 comp rx=0 tx=0).stop 2024-03-08T20:47:34.466 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:33.787+0000 7fde68000700 1 -- 172.21.15.107:0/3727671691 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fde60105ff0 msgr2=0x7fde601a1c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:47:34.467 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:33.787+0000 7fde68000700 1 --2- 172.21.15.107:0/3727671691 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fde60105ff0 0x7fde601a1c80 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fde54006af0 tx=0x7fde54011c40 comp rx=0 tx=0).stop 2024-03-08T20:47:34.467 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:33.787+0000 7fde68000700 1 -- 172.21.15.107:0/3727671691 shutdown_connections 2024-03-08T20:47:34.467 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:33.787+0000 7fde68000700 1 --2- 172.21.15.107:0/3727671691 >> [v2:172.21.15.107:6800/3549936519,v1:172.21.15.107:6801/3549936519] conn(0x7fde48038ed0 0x7fde4803b390 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:47:34.467 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:33.787+0000 7fde68000700 1 --2- 172.21.15.107:0/3727671691 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fde60105ff0 0x7fde601a1c80 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:47:34.467 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:33.787+0000 7fde68000700 1 -- 172.21.15.107:0/3727671691 >> 172.21.15.107:0/3727671691 conn(0x7fde60101d10 msgr2=0x7fde60104100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:47:34.467 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:33.787+0000 7fde68000700 1 -- 172.21.15.107:0/3727671691 shutdown_connections 2024-03-08T20:47:34.467 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:33.787+0000 7fde68000700 1 -- 172.21.15.107:0/3727671691 wait complete. 2024-03-08T20:47:34.467 INFO:teuthology.orchestra.run.smithi107.stdout:mgr is available 2024-03-08T20:47:35.621 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:35 smithi107 ceph-mon[31381]: from='client.? 172.21.15.107:0/549243204' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-03-08T20:47:35.969 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout 2024-03-08T20:47:35.969 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout [global] 2024-03-08T20:47:35.969 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout fsid = dc20f0f8-dd8c-11ee-95c6-87774f69a715 2024-03-08T20:47:35.969 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-03-08T20:47:35.969 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-03-08T20:47:35.970 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-03-08T20:47:35.970 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-03-08T20:47:35.970 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout 2024-03-08T20:47:35.970 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout [mgr] 2024-03-08T20:47:35.976 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-03-08T20:47:35.976 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout 2024-03-08T20:47:35.976 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout [osd] 2024-03-08T20:47:35.976 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-03-08T20:47:35.976 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-03-08T20:47:35.977 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-03-08T20:47:35.977 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-03-08T20:47:35.977 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-03-08T20:47:35.977 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:35.030+0000 7fa2f9a00700 1 Processor -- start 2024-03-08T20:47:35.977 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:35.030+0000 7fa2f9a00700 1 -- start start 2024-03-08T20:47:35.977 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:35.031+0000 7fa2f9a00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa2f410e0d0 0x7fa2f410e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:47:35.977 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:35.031+0000 7fa2f9a00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2f410eac0 con 0x7fa2f410e0d0 2024-03-08T20:47:35.977 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:35.031+0000 7fa2f2a00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa2f410e0d0 0x7fa2f410e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:47:35.977 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:35.031+0000 7fa2f2a00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa2f410e0d0 0x7fa2f410e4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:32936/0 (socket says 172.21.15.107:32936) 2024-03-08T20:47:35.977 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:35.031+0000 7fa2f2a00700 1 -- 172.21.15.107:0/2541458198 learned_addr learned my addr 172.21.15.107:0/2541458198 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T20:47:35.977 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:35.032+0000 7fa2f2a00700 1 -- 172.21.15.107:0/2541458198 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2f410ef10 con 0x7fa2f410e0d0 2024-03-08T20:47:35.977 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:35.032+0000 7fa2f2a00700 1 --2- 172.21.15.107:0/2541458198 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa2f410e0d0 0x7fa2f410e4f0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fa2e40098a0 tx=0x7fa2e4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75c10a5652190189 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:47:35.977 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:35.032+0000 7fa2f2000700 1 -- 172.21.15.107:0/2541458198 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa2e4004160 con 0x7fa2f410e0d0 2024-03-08T20:47:35.978 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:35.033+0000 7fa2f2000700 1 -- 172.21.15.107:0/2541458198 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1071+0+0 (secure 0 0 0) 0x7fa2e4004300 con 0x7fa2f410e0d0 2024-03-08T20:47:35.978 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:35.033+0000 7fa2f2000700 1 -- 172.21.15.107:0/2541458198 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa2e4011a80 con 0x7fa2f410e0d0 2024-03-08T20:47:35.978 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:35.033+0000 7fa2f9a00700 1 -- 172.21.15.107:0/2541458198 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa2f410e0d0 msgr2=0x7fa2f410e4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:47:35.978 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:35.033+0000 7fa2f9a00700 1 --2- 172.21.15.107:0/2541458198 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa2f410e0d0 0x7fa2f410e4f0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fa2e40098a0 tx=0x7fa2e4009bb0 comp rx=0 tx=0).stop 2024-03-08T20:47:35.978 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:35.033+0000 7fa2f9a00700 1 -- 172.21.15.107:0/2541458198 shutdown_connections 2024-03-08T20:47:35.978 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:35.033+0000 7fa2f9a00700 1 --2- 172.21.15.107:0/2541458198 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa2f410e0d0 0x7fa2f410e4f0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:47:35.978 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:35.033+0000 7fa2f9a00700 1 -- 172.21.15.107:0/2541458198 >> 172.21.15.107:0/2541458198 conn(0x7fa2f407cdd0 msgr2=0x7fa2f407f1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:47:35.978 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:35.033+0000 7fa2f9a00700 1 -- 172.21.15.107:0/2541458198 shutdown_connections 2024-03-08T20:47:35.978 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:35.034+0000 7fa2f9a00700 1 -- 172.21.15.107:0/2541458198 wait complete. 2024-03-08T20:47:35.978 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:35.035+0000 7fa2f9a00700 1 Processor -- start 2024-03-08T20:47:35.978 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:35.035+0000 7fa2f9a00700 1 -- start start 2024-03-08T20:47:35.978 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:35.035+0000 7fa2f9a00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa2f410e0d0 0x7fa2f41a1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:47:35.979 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:35.035+0000 7fa2f9a00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2f41122c0 con 0x7fa2f410e0d0 2024-03-08T20:47:35.979 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:35.035+0000 7fa2f2a00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa2f410e0d0 0x7fa2f41a1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:47:35.979 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:35.035+0000 7fa2f2a00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa2f410e0d0 0x7fa2f41a1cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:32938/0 (socket says 172.21.15.107:32938) 2024-03-08T20:47:35.979 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:35.035+0000 7fa2f2a00700 1 -- 172.21.15.107:0/549243204 learned_addr learned my addr 172.21.15.107:0/549243204 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T20:47:35.979 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:35.036+0000 7fa2f2a00700 1 -- 172.21.15.107:0/549243204 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2f41a2200 con 0x7fa2f410e0d0 2024-03-08T20:47:35.979 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:35.036+0000 7fa2f2a00700 1 --2- 172.21.15.107:0/549243204 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa2f410e0d0 0x7fa2f41a1cc0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fa2e4000c00 tx=0x7fa2e4011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:47:35.979 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:35.036+0000 7fa2eb400700 1 -- 172.21.15.107:0/549243204 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa2e40039f0 con 0x7fa2f410e0d0 2024-03-08T20:47:35.979 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:35.036+0000 7fa2eb400700 1 -- 172.21.15.107:0/549243204 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1071+0+0 (secure 0 0 0) 0x7fa2e4003b90 con 0x7fa2f410e0d0 2024-03-08T20:47:35.979 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:35.036+0000 7fa2eb400700 1 -- 172.21.15.107:0/549243204 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa2e401a740 con 0x7fa2f410e0d0 2024-03-08T20:47:35.979 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:35.036+0000 7fa2f9a00700 1 -- 172.21.15.107:0/549243204 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2f41a2490 con 0x7fa2f410e0d0 2024-03-08T20:47:35.979 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:35.036+0000 7fa2f9a00700 1 -- 172.21.15.107:0/549243204 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2f41a5180 con 0x7fa2f410e0d0 2024-03-08T20:47:35.979 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:35.037+0000 7fa2eb400700 1 -- 172.21.15.107:0/549243204 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 4) v1 ==== 46095+0+0 (secure 0 0 0) 0x7fa2e4024af0 con 0x7fa2f410e0d0 2024-03-08T20:47:35.979 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:35.038+0000 7fa2f9a00700 1 -- 172.21.15.107:0/549243204 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2f4063410 con 0x7fa2f410e0d0 2024-03-08T20:47:35.980 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:35.038+0000 7fa2eb400700 1 --2- 172.21.15.107:0/549243204 >> [v2:172.21.15.107:6800/3549936519,v1:172.21.15.107:6801/3549936519] conn(0x7fa2d8038f20 0x7fa2d803b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:47:35.980 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:35.038+0000 7fa2eb400700 1 -- 172.21.15.107:0/549243204 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fa2e404d140 con 0x7fa2f410e0d0 2024-03-08T20:47:35.980 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:35.038+0000 7fa2ebe00700 1 --2- 172.21.15.107:0/549243204 >> [v2:172.21.15.107:6800/3549936519,v1:172.21.15.107:6801/3549936519] conn(0x7fa2d8038f20 0x7fa2d803b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:47:35.980 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:35.039+0000 7fa2ebe00700 1 --2- 172.21.15.107:0/549243204 >> [v2:172.21.15.107:6800/3549936519,v1:172.21.15.107:6801/3549936519] conn(0x7fa2d8038f20 0x7fa2d803b3e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fa2dc006fd0 tx=0x7fa2dc006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:47:35.980 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:35.043+0000 7fa2eb400700 1 -- 172.21.15.107:0/549243204 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7fa2e404cb60 con 0x7fa2f410e0d0 2024-03-08T20:47:35.980 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:35.262+0000 7fa2f9a00700 1 -- 172.21.15.107:0/549243204 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7fa2f41a5350 con 0x7fa2f410e0d0 2024-03-08T20:47:35.980 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:35.264+0000 7fa2eb400700 1 -- 172.21.15.107:0/549243204 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) v1 ==== 70+0+373 (secure 0 0 0) 0x7fa2e402d3f0 con 0x7fa2f410e0d0 2024-03-08T20:47:35.980 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:35.267+0000 7fa2f9a00700 1 -- 172.21.15.107:0/549243204 >> [v2:172.21.15.107:6800/3549936519,v1:172.21.15.107:6801/3549936519] conn(0x7fa2d8038f20 msgr2=0x7fa2d803b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:47:35.980 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:35.267+0000 7fa2f9a00700 1 --2- 172.21.15.107:0/549243204 >> [v2:172.21.15.107:6800/3549936519,v1:172.21.15.107:6801/3549936519] conn(0x7fa2d8038f20 0x7fa2d803b3e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fa2dc006fd0 tx=0x7fa2dc006e40 comp rx=0 tx=0).stop 2024-03-08T20:47:35.980 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:35.267+0000 7fa2f9a00700 1 -- 172.21.15.107:0/549243204 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa2f410e0d0 msgr2=0x7fa2f41a1cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:47:35.981 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:35.267+0000 7fa2f9a00700 1 --2- 172.21.15.107:0/549243204 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa2f410e0d0 0x7fa2f41a1cc0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fa2e4000c00 tx=0x7fa2e4011b00 comp rx=0 tx=0).stop 2024-03-08T20:47:35.981 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:35.267+0000 7fa2f9a00700 1 -- 172.21.15.107:0/549243204 shutdown_connections 2024-03-08T20:47:35.981 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:35.267+0000 7fa2f9a00700 1 --2- 172.21.15.107:0/549243204 >> [v2:172.21.15.107:6800/3549936519,v1:172.21.15.107:6801/3549936519] conn(0x7fa2d8038f20 0x7fa2d803b3e0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:47:35.982 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:35.267+0000 7fa2f9a00700 1 --2- 172.21.15.107:0/549243204 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa2f410e0d0 0x7fa2f41a1cc0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:47:35.982 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:35.267+0000 7fa2f9a00700 1 -- 172.21.15.107:0/549243204 >> 172.21.15.107:0/549243204 conn(0x7fa2f407cdd0 msgr2=0x7fa2f407d930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:47:35.982 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:35.267+0000 7fa2f9a00700 1 -- 172.21.15.107:0/549243204 shutdown_connections 2024-03-08T20:47:35.982 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:35.268+0000 7fa2f9a00700 1 -- 172.21.15.107:0/549243204 wait complete. 2024-03-08T20:47:35.982 INFO:teuthology.orchestra.run.smithi107.stdout:Enabling cephadm module... 2024-03-08T20:47:37.228 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:36 smithi107 ceph-mon[31381]: from='client.? 172.21.15.107:0/1837993971' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-03-08T20:47:37.229 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 08 20:47:37 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-x[31606]: ignoring --setuser ceph since I am not root 2024-03-08T20:47:37.229 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 08 20:47:37 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-x[31606]: ignoring --setgroup ceph since I am not root 2024-03-08T20:47:37.595 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:36.565+0000 7f32e6800700 1 Processor -- start 2024-03-08T20:47:37.595 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:36.565+0000 7f32e6800700 1 -- start start 2024-03-08T20:47:37.596 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:36.565+0000 7f32e6800700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f32e010e0d0 0x7f32e010e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:47:37.596 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:36.565+0000 7f32e6800700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32e010eac0 con 0x7f32e010e0d0 2024-03-08T20:47:37.596 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:36.566+0000 7f32df400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f32e010e0d0 0x7f32e010e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:47:37.596 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:36.566+0000 7f32df400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f32e010e0d0 0x7f32e010e4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:39416/0 (socket says 172.21.15.107:39416) 2024-03-08T20:47:37.596 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:36.566+0000 7f32df400700 1 -- 172.21.15.107:0/3677586668 learned_addr learned my addr 172.21.15.107:0/3677586668 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T20:47:37.596 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:36.567+0000 7f32df400700 1 -- 172.21.15.107:0/3677586668 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32e010eef0 con 0x7f32e010e0d0 2024-03-08T20:47:37.596 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:36.567+0000 7f32df400700 1 --2- 172.21.15.107:0/3677586668 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f32e010e0d0 0x7f32e010e4f0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f32d00098a0 tx=0x7f32d0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41ebd936be54f778 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:47:37.596 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:36.567+0000 7f32de000700 1 -- 172.21.15.107:0/3677586668 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f32d0004160 con 0x7f32e010e0d0 2024-03-08T20:47:37.596 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:36.567+0000 7f32de000700 1 -- 172.21.15.107:0/3677586668 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1071+0+0 (secure 0 0 0) 0x7f32d0004300 con 0x7f32e010e0d0 2024-03-08T20:47:37.596 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:36.567+0000 7f32de000700 1 -- 172.21.15.107:0/3677586668 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f32d0011a80 con 0x7f32e010e0d0 2024-03-08T20:47:37.596 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:36.568+0000 7f32e6800700 1 -- 172.21.15.107:0/3677586668 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f32e010e0d0 msgr2=0x7f32e010e4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:47:37.596 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:36.568+0000 7f32e6800700 1 --2- 172.21.15.107:0/3677586668 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f32e010e0d0 0x7f32e010e4f0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f32d00098a0 tx=0x7f32d0009bb0 comp rx=0 tx=0).stop 2024-03-08T20:47:37.596 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:36.568+0000 7f32e6800700 1 -- 172.21.15.107:0/3677586668 shutdown_connections 2024-03-08T20:47:37.597 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:36.568+0000 7f32e6800700 1 --2- 172.21.15.107:0/3677586668 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f32e010e0d0 0x7f32e010e4f0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:47:37.597 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:36.568+0000 7f32e6800700 1 -- 172.21.15.107:0/3677586668 >> 172.21.15.107:0/3677586668 conn(0x7f32e007cdd0 msgr2=0x7f32e007f210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:47:37.597 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:36.568+0000 7f32e6800700 1 -- 172.21.15.107:0/3677586668 shutdown_connections 2024-03-08T20:47:37.597 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:36.568+0000 7f32e6800700 1 -- 172.21.15.107:0/3677586668 wait complete. 2024-03-08T20:47:37.597 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:36.569+0000 7f32e6800700 1 Processor -- start 2024-03-08T20:47:37.597 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:36.570+0000 7f32e6800700 1 -- start start 2024-03-08T20:47:37.597 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:36.570+0000 7f32e6800700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f32e010e0d0 0x7f32e01a1c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:47:37.597 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:36.570+0000 7f32e6800700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32e01122a0 con 0x7f32e010e0d0 2024-03-08T20:47:37.597 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:36.570+0000 7f32df400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f32e010e0d0 0x7f32e01a1c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:47:37.597 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:36.570+0000 7f32df400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f32e010e0d0 0x7f32e01a1c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:39428/0 (socket says 172.21.15.107:39428) 2024-03-08T20:47:37.597 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:36.570+0000 7f32df400700 1 -- 172.21.15.107:0/1837993971 learned_addr learned my addr 172.21.15.107:0/1837993971 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T20:47:37.597 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:36.571+0000 7f32df400700 1 -- 172.21.15.107:0/1837993971 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32e01a21b0 con 0x7f32e010e0d0 2024-03-08T20:47:37.597 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:36.571+0000 7f32df400700 1 --2- 172.21.15.107:0/1837993971 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f32e010e0d0 0x7f32e01a1c70 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f32d0000c00 tx=0x7f32d0011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:47:37.597 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:36.571+0000 7f32d7e00700 1 -- 172.21.15.107:0/1837993971 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f32d00039f0 con 0x7f32e010e0d0 2024-03-08T20:47:37.598 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:36.571+0000 7f32d7e00700 1 -- 172.21.15.107:0/1837993971 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1071+0+0 (secure 0 0 0) 0x7f32d0003b90 con 0x7f32e010e0d0 2024-03-08T20:47:37.598 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:36.571+0000 7f32e6800700 1 -- 172.21.15.107:0/1837993971 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32e01a2440 con 0x7f32e010e0d0 2024-03-08T20:47:37.598 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:36.571+0000 7f32d7e00700 1 -- 172.21.15.107:0/1837993971 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f32d001a740 con 0x7f32e010e0d0 2024-03-08T20:47:37.598 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:36.571+0000 7f32e6800700 1 -- 172.21.15.107:0/1837993971 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32e01a5130 con 0x7f32e010e0d0 2024-03-08T20:47:37.598 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:36.572+0000 7f32d7e00700 1 -- 172.21.15.107:0/1837993971 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 4) v1 ==== 46095+0+0 (secure 0 0 0) 0x7f32d0024440 con 0x7f32e010e0d0 2024-03-08T20:47:37.598 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:36.572+0000 7f32e6800700 1 -- 172.21.15.107:0/1837993971 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32e0063410 con 0x7f32e010e0d0 2024-03-08T20:47:37.598 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:36.573+0000 7f32d7e00700 1 --2- 172.21.15.107:0/1837993971 >> [v2:172.21.15.107:6800/3549936519,v1:172.21.15.107:6801/3549936519] conn(0x7f32c4038f20 0x7f32c403b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:47:37.598 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:36.573+0000 7f32d7e00700 1 -- 172.21.15.107:0/1837993971 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f32d004d0d0 con 0x7f32e010e0d0 2024-03-08T20:47:37.598 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:36.573+0000 7f32dea00700 1 --2- 172.21.15.107:0/1837993971 >> [v2:172.21.15.107:6800/3549936519,v1:172.21.15.107:6801/3549936519] conn(0x7f32c4038f20 0x7f32c403b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:47:37.598 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:36.574+0000 7f32dea00700 1 --2- 172.21.15.107:0/1837993971 >> [v2:172.21.15.107:6800/3549936519,v1:172.21.15.107:6801/3549936519] conn(0x7f32c4038f20 0x7f32c403b3e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f32cc006fd0 tx=0x7f32cc006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:47:37.598 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:36.578+0000 7f32d7e00700 1 -- 172.21.15.107:0/1837993971 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f32d0020240 con 0x7f32e010e0d0 2024-03-08T20:47:37.598 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:36.855+0000 7f32e6800700 1 -- 172.21.15.107:0/1837993971 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) v1 -- 0x7f32e01a56d0 con 0x7f32e010e0d0 2024-03-08T20:47:37.599 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:36.909+0000 7f32d7e00700 1 -- 172.21.15.107:0/1837993971 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) v1 ==== 86+0+0 (secure 0 0 0) 0x7f32d004caf0 con 0x7f32e010e0d0 2024-03-08T20:47:37.599 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:36.912+0000 7f32e6800700 1 -- 172.21.15.107:0/1837993971 >> [v2:172.21.15.107:6800/3549936519,v1:172.21.15.107:6801/3549936519] conn(0x7f32c4038f20 msgr2=0x7f32c403b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:47:37.599 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:36.912+0000 7f32e6800700 1 --2- 172.21.15.107:0/1837993971 >> [v2:172.21.15.107:6800/3549936519,v1:172.21.15.107:6801/3549936519] conn(0x7f32c4038f20 0x7f32c403b3e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f32cc006fd0 tx=0x7f32cc006e40 comp rx=0 tx=0).stop 2024-03-08T20:47:37.599 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:36.912+0000 7f32e6800700 1 -- 172.21.15.107:0/1837993971 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f32e010e0d0 msgr2=0x7f32e01a1c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:47:37.599 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:36.914+0000 7f32d7e00700 1 -- 172.21.15.107:0/1837993971 <== mon.0 v2:172.21.15.107:3300/0 8 ==== mgrmap(e 5) v1 ==== 46106+0+0 (secure 0 0 0) 0x7f32d002d3f0 con 0x7f32e010e0d0 2024-03-08T20:47:37.599 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:36.914+0000 7f32e6800700 1 --2- 172.21.15.107:0/1837993971 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f32e010e0d0 0x7f32e01a1c70 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f32d0000c00 tx=0x7f32d0011b00 comp rx=0 tx=0).stop 2024-03-08T20:47:37.599 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:36.914+0000 7f32e6800700 1 -- 172.21.15.107:0/1837993971 shutdown_connections 2024-03-08T20:47:37.599 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:36.914+0000 7f32e6800700 1 --2- 172.21.15.107:0/1837993971 >> [v2:172.21.15.107:6800/3549936519,v1:172.21.15.107:6801/3549936519] conn(0x7f32c4038f20 0x7f32c403b3e0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:47:37.599 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:36.914+0000 7f32e6800700 1 --2- 172.21.15.107:0/1837993971 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f32e010e0d0 0x7f32e01a1c70 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:47:37.599 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:36.914+0000 7f32e6800700 1 -- 172.21.15.107:0/1837993971 >> 172.21.15.107:0/1837993971 conn(0x7f32e007cdd0 msgr2=0x7f32e007d8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:47:37.599 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:36.914+0000 7f32e6800700 1 -- 172.21.15.107:0/1837993971 shutdown_connections 2024-03-08T20:47:37.599 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:36.914+0000 7f32e6800700 1 -- 172.21.15.107:0/1837993971 wait complete. 2024-03-08T20:47:37.890 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 08 20:47:37 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-x[31606]: 2024-03-08T20:47:37.683+0000 7ff4e6f9d200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-08T20:47:37.890 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 08 20:47:37 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-x[31606]: 2024-03-08T20:47:37.813+0000 7ff4e6f9d200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-08T20:47:38.181 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:37 smithi107 ceph-mon[31381]: from='client.? 172.21.15.107:0/1837993971' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-03-08T20:47:38.181 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:37 smithi107 ceph-mon[31381]: mgrmap e5: x(active, since 5s) 2024-03-08T20:47:38.744 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 08 20:47:38 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-x[31606]: 2024-03-08T20:47:38.627+0000 7ff4e6f9d200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-08T20:47:38.744 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 08 20:47:38 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-x[31606]: 2024-03-08T20:47:38.748+0000 7ff4e6f9d200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-08T20:47:39.047 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:38 smithi107 ceph-mon[31381]: from='client.? 172.21.15.107:0/1848655843' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-03-08T20:47:39.048 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 08 20:47:38 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-x[31606]: 2024-03-08T20:47:38.928+0000 7ff4e6f9d200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-08T20:47:39.048 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 08 20:47:39 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-x[31606]: 2024-03-08T20:47:39.052+0000 7ff4e6f9d200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-08T20:47:39.194 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout { 2024-03-08T20:47:39.194 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-03-08T20:47:39.194 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "available": true, 2024-03-08T20:47:39.195 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-03-08T20:47:39.195 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-03-08T20:47:39.195 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout } 2024-03-08T20:47:39.195 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:38.185+0000 7f92aaa00700 1 Processor -- start 2024-03-08T20:47:39.195 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:38.186+0000 7f92aaa00700 1 -- start start 2024-03-08T20:47:39.195 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:38.186+0000 7f92aaa00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f92a407ce70 0x7f92a407b320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:47:39.197 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:38.186+0000 7f92aaa00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92a407b8f0 con 0x7f92a407ce70 2024-03-08T20:47:39.197 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:38.187+0000 7f92a9600700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f92a407ce70 0x7f92a407b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:47:39.197 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:38.187+0000 7f92a9600700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f92a407ce70 0x7f92a407b320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:39456/0 (socket says 172.21.15.107:39456) 2024-03-08T20:47:39.197 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:38.187+0000 7f92a9600700 1 -- 172.21.15.107:0/3466712658 learned_addr learned my addr 172.21.15.107:0/3466712658 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T20:47:39.197 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:38.187+0000 7f92a9600700 1 -- 172.21.15.107:0/3466712658 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92a407ba70 con 0x7f92a407ce70 2024-03-08T20:47:39.197 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:38.187+0000 7f92a9600700 1 --2- 172.21.15.107:0/3466712658 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f92a407ce70 0x7f92a407b320 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f92980098a0 tx=0x7f9298009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=892695f863e7db8f server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:47:39.197 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:38.188+0000 7f92a3e00700 1 -- 172.21.15.107:0/3466712658 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9298004160 con 0x7f92a407ce70 2024-03-08T20:47:39.197 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:38.188+0000 7f92a3e00700 1 -- 172.21.15.107:0/3466712658 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1071+0+0 (secure 0 0 0) 0x7f9298004300 con 0x7f92a407ce70 2024-03-08T20:47:39.197 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:38.188+0000 7f92a3e00700 1 -- 172.21.15.107:0/3466712658 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9298011a80 con 0x7f92a407ce70 2024-03-08T20:47:39.199 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:38.189+0000 7f92aaa00700 1 -- 172.21.15.107:0/3466712658 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f92a407ce70 msgr2=0x7f92a407b320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:47:39.199 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:38.189+0000 7f92aaa00700 1 --2- 172.21.15.107:0/3466712658 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f92a407ce70 0x7f92a407b320 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f92980098a0 tx=0x7f9298009bb0 comp rx=0 tx=0).stop 2024-03-08T20:47:39.199 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:38.189+0000 7f92aaa00700 1 -- 172.21.15.107:0/3466712658 shutdown_connections 2024-03-08T20:47:39.199 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:38.189+0000 7f92aaa00700 1 --2- 172.21.15.107:0/3466712658 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f92a407ce70 0x7f92a407b320 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:47:39.199 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:38.189+0000 7f92aaa00700 1 -- 172.21.15.107:0/3466712658 >> 172.21.15.107:0/3466712658 conn(0x7f92a4102bb0 msgr2=0x7f92a4104fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:47:39.200 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:38.189+0000 7f92aaa00700 1 -- 172.21.15.107:0/3466712658 shutdown_connections 2024-03-08T20:47:39.200 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:38.189+0000 7f92aaa00700 1 -- 172.21.15.107:0/3466712658 wait complete. 2024-03-08T20:47:39.200 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:38.190+0000 7f92aaa00700 1 Processor -- start 2024-03-08T20:47:39.200 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:38.190+0000 7f92aaa00700 1 -- start start 2024-03-08T20:47:39.200 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:38.191+0000 7f92aaa00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f92a407ce70 0x7f92a419d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:47:39.200 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:38.191+0000 7f92aaa00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92a410dae0 con 0x7f92a407ce70 2024-03-08T20:47:39.200 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:38.191+0000 7f92a9600700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f92a407ce70 0x7f92a419d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:47:39.200 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:38.191+0000 7f92a9600700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f92a407ce70 0x7f92a419d660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:39468/0 (socket says 172.21.15.107:39468) 2024-03-08T20:47:39.202 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:38.191+0000 7f92a9600700 1 -- 172.21.15.107:0/1848655843 learned_addr learned my addr 172.21.15.107:0/1848655843 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T20:47:39.202 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:38.191+0000 7f92a9600700 1 -- 172.21.15.107:0/1848655843 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92a419dba0 con 0x7f92a407ce70 2024-03-08T20:47:39.202 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:38.192+0000 7f92a9600700 1 --2- 172.21.15.107:0/1848655843 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f92a407ce70 0x7f92a419d660 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f9298011a50 tx=0x7f9298011b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:47:39.202 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:38.192+0000 7f92a2000700 1 -- 172.21.15.107:0/1848655843 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9298003a80 con 0x7f92a407ce70 2024-03-08T20:47:39.202 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:38.192+0000 7f92a2000700 1 -- 172.21.15.107:0/1848655843 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1071+0+0 (secure 0 0 0) 0x7f9298003c20 con 0x7f92a407ce70 2024-03-08T20:47:39.202 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:38.192+0000 7f92a2000700 1 -- 172.21.15.107:0/1848655843 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f929801a650 con 0x7f92a407ce70 2024-03-08T20:47:39.202 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:38.192+0000 7f92aaa00700 1 -- 172.21.15.107:0/1848655843 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92a419de30 con 0x7f92a407ce70 2024-03-08T20:47:39.202 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:38.193+0000 7f92aaa00700 1 -- 172.21.15.107:0/1848655843 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92a41a0b20 con 0x7f92a407ce70 2024-03-08T20:47:39.203 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:38.194+0000 7f92a2000700 1 -- 172.21.15.107:0/1848655843 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 5) v1 ==== 46106+0+0 (secure 0 0 0) 0x7f9298024ad0 con 0x7f92a407ce70 2024-03-08T20:47:39.203 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:38.194+0000 7f92aaa00700 1 -- 172.21.15.107:0/1848655843 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92a4063410 con 0x7f92a407ce70 2024-03-08T20:47:39.203 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:38.194+0000 7f92a2000700 1 --2- 172.21.15.107:0/1848655843 >> [v2:172.21.15.107:6800/3549936519,v1:172.21.15.107:6801/3549936519] conn(0x7f928c038f70 0x7f928c03b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:47:39.203 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:38.194+0000 7f92a2000700 1 -- 172.21.15.107:0/1848655843 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f929804d280 con 0x7f92a407ce70 2024-03-08T20:47:39.203 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:38.194+0000 7f92a8c00700 1 -- 172.21.15.107:0/1848655843 >> [v2:172.21.15.107:6800/3549936519,v1:172.21.15.107:6801/3549936519] conn(0x7f928c038f70 msgr2=0x7f928c03b430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.107:6800/3549936519 2024-03-08T20:47:39.203 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:38.194+0000 7f92a8c00700 1 --2- 172.21.15.107:0/1848655843 >> [v2:172.21.15.107:6800/3549936519,v1:172.21.15.107:6801/3549936519] conn(0x7f928c038f70 0x7f928c03b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-08T20:47:39.203 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:38.200+0000 7f92a2000700 1 -- 172.21.15.107:0/1848655843 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f92980244f0 con 0x7f92a407ce70 2024-03-08T20:47:39.205 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:38.395+0000 7f92a8c00700 1 -- 172.21.15.107:0/1848655843 >> [v2:172.21.15.107:6800/3549936519,v1:172.21.15.107:6801/3549936519] conn(0x7f928c038f70 msgr2=0x7f928c03b430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.107:6800/3549936519 2024-03-08T20:47:39.205 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:38.395+0000 7f92a8c00700 1 --2- 172.21.15.107:0/1848655843 >> [v2:172.21.15.107:6800/3549936519,v1:172.21.15.107:6801/3549936519] conn(0x7f928c038f70 0x7f928c03b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-08T20:47:39.205 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:38.496+0000 7f92aaa00700 1 -- 172.21.15.107:0/1848655843 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f92a41a0cf0 con 0x7f92a407ce70 2024-03-08T20:47:39.205 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:38.497+0000 7f92a2000700 1 -- 172.21.15.107:0/1848655843 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) v1 ==== 56+0+88 (secure 0 0 0) 0x7f929802d3f0 con 0x7f92a407ce70 2024-03-08T20:47:39.205 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:38.500+0000 7f92aaa00700 1 -- 172.21.15.107:0/1848655843 >> [v2:172.21.15.107:6800/3549936519,v1:172.21.15.107:6801/3549936519] conn(0x7f928c038f70 msgr2=0x7f928c03b430 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-08T20:47:39.205 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:38.500+0000 7f92aaa00700 1 --2- 172.21.15.107:0/1848655843 >> [v2:172.21.15.107:6800/3549936519,v1:172.21.15.107:6801/3549936519] conn(0x7f928c038f70 0x7f928c03b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:47:39.205 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:38.500+0000 7f92aaa00700 1 -- 172.21.15.107:0/1848655843 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f92a407ce70 msgr2=0x7f92a419d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:47:39.205 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:38.500+0000 7f92aaa00700 1 --2- 172.21.15.107:0/1848655843 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f92a407ce70 0x7f92a419d660 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f9298011a50 tx=0x7f9298011b30 comp rx=0 tx=0).stop 2024-03-08T20:47:39.205 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:38.500+0000 7f92aaa00700 1 -- 172.21.15.107:0/1848655843 shutdown_connections 2024-03-08T20:47:39.205 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:38.500+0000 7f92aaa00700 1 --2- 172.21.15.107:0/1848655843 >> [v2:172.21.15.107:6800/3549936519,v1:172.21.15.107:6801/3549936519] conn(0x7f928c038f70 0x7f928c03b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:47:39.205 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:38.500+0000 7f92aaa00700 1 --2- 172.21.15.107:0/1848655843 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f92a407ce70 0x7f92a419d660 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:47:39.206 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:38.500+0000 7f92aaa00700 1 -- 172.21.15.107:0/1848655843 >> 172.21.15.107:0/1848655843 conn(0x7f92a4102bb0 msgr2=0x7f92a4103690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:47:39.206 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:38.501+0000 7f92aaa00700 1 -- 172.21.15.107:0/1848655843 shutdown_connections 2024-03-08T20:47:39.206 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:38.501+0000 7f92aaa00700 1 -- 172.21.15.107:0/1848655843 wait complete. 2024-03-08T20:47:39.206 INFO:teuthology.orchestra.run.smithi107.stdout:Waiting for the mgr to restart... 2024-03-08T20:47:39.206 INFO:teuthology.orchestra.run.smithi107.stdout:Waiting for mgr epoch 5... 2024-03-08T20:47:39.354 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 08 20:47:39 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-x[31606]: 2024-03-08T20:47:39.163+0000 7ff4e6f9d200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-08T20:47:39.633 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 08 20:47:39 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-x[31606]: 2024-03-08T20:47:39.511+0000 7ff4e6f9d200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-08T20:47:39.900 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 08 20:47:39 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-x[31606]: 2024-03-08T20:47:39.662+0000 7ff4e6f9d200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-08T20:47:39.900 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 08 20:47:39 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-x[31606]: 2024-03-08T20:47:39.779+0000 7ff4e6f9d200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-08T20:47:39.900 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 08 20:47:39 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-x[31606]: 2024-03-08T20:47:39.904+0000 7ff4e6f9d200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-08T20:47:40.228 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 08 20:47:40 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-x[31606]: 2024-03-08T20:47:40.121+0000 7ff4e6f9d200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-08T20:47:41.478 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 08 20:47:41 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-x[31606]: 2024-03-08T20:47:41.375+0000 7ff4e6f9d200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-08T20:47:41.951 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 08 20:47:41 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-x[31606]: 2024-03-08T20:47:41.546+0000 7ff4e6f9d200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-08T20:47:41.952 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 08 20:47:41 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-x[31606]: 2024-03-08T20:47:41.956+0000 7ff4e6f9d200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-08T20:47:42.228 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 08 20:47:42 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-x[31606]: 2024-03-08T20:47:42.074+0000 7ff4e6f9d200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-08T20:47:42.728 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 08 20:47:42 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-x[31606]: 2024-03-08T20:47:42.324+0000 7ff4e6f9d200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-08T20:47:42.729 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 08 20:47:42 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-x[31606]: 2024-03-08T20:47:42.570+0000 7ff4e6f9d200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-08T20:47:43.135 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 08 20:47:42 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-x[31606]: 2024-03-08T20:47:42.959+0000 7ff4e6f9d200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-08T20:47:43.478 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 08 20:47:43 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-x[31606]: 2024-03-08T20:47:43.285+0000 7ff4e6f9d200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-08T20:47:43.978 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 08 20:47:43 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-x[31606]: 2024-03-08T20:47:43.588+0000 7ff4e6f9d200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-08T20:47:44.978 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 08 20:47:44 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-x[31606]: 2024-03-08T20:47:44.606+0000 7ff4e6f9d200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-08T20:47:45.698 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 08 20:47:45 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-x[31606]: 2024-03-08T20:47:45.295+0000 7ff4e6f9d200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-08T20:47:45.698 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 08 20:47:45 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-x[31606]: 2024-03-08T20:47:45.424+0000 7ff4e6f9d200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-08T20:47:45.978 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 08 20:47:45 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-x[31606]: 2024-03-08T20:47:45.702+0000 7ff4e6f9d200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-08T20:47:45.979 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:45 smithi107 ceph-mon[31381]: Active manager daemon x restarted 2024-03-08T20:47:45.979 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:45 smithi107 ceph-mon[31381]: Activating manager daemon x 2024-03-08T20:47:45.979 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:45 smithi107 ceph-mon[31381]: osdmap e2: 0 total, 0 up, 0 in 2024-03-08T20:47:45.979 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:45 smithi107 ceph-mon[31381]: mgrmap e6: x(active, starting, since 0.00973792s) 2024-03-08T20:47:45.979 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:45 smithi107 ceph-mon[31381]: from='mgr.14120 172.21.15.107:0/283560839' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-08T20:47:45.979 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:45 smithi107 ceph-mon[31381]: from='mgr.14120 172.21.15.107:0/283560839' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-08T20:47:45.979 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:45 smithi107 ceph-mon[31381]: from='mgr.14120 172.21.15.107:0/283560839' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-08T20:47:45.979 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:45 smithi107 ceph-mon[31381]: from='mgr.14120 172.21.15.107:0/283560839' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-08T20:47:45.979 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:45 smithi107 ceph-mon[31381]: from='mgr.14120 172.21.15.107:0/283560839' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-08T20:47:45.979 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:45 smithi107 ceph-mon[31381]: Manager daemon x is now available 2024-03-08T20:47:45.979 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:45 smithi107 ceph-mon[31381]: from='mgr.14120 172.21.15.107:0/283560839' entity='mgr.x' 2024-03-08T20:47:46.978 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:46 smithi107 ceph-mon[31381]: Found migration_current of "None". Setting to last migration. 2024-03-08T20:47:46.979 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:46 smithi107 ceph-mon[31381]: from='mgr.14120 172.21.15.107:0/283560839' entity='mgr.x' 2024-03-08T20:47:46.979 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:46 smithi107 ceph-mon[31381]: from='mgr.14120 172.21.15.107:0/283560839' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T20:47:46.979 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:46 smithi107 ceph-mon[31381]: from='mgr.14120 172.21.15.107:0/283560839' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T20:47:46.979 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:46 smithi107 ceph-mon[31381]: from='mgr.14120 172.21.15.107:0/283560839' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T20:47:46.979 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:46 smithi107 ceph-mon[31381]: from='mgr.14120 172.21.15.107:0/283560839' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-08T20:47:46.979 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:46 smithi107 ceph-mon[31381]: from='mgr.14120 172.21.15.107:0/283560839' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-08T20:47:46.979 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:46 smithi107 ceph-mon[31381]: from='mgr.14120 172.21.15.107:0/283560839' entity='mgr.x' 2024-03-08T20:47:46.979 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:46 smithi107 ceph-mon[31381]: from='mgr.14120 172.21.15.107:0/283560839' entity='mgr.x' 2024-03-08T20:47:46.979 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:46 smithi107 ceph-mon[31381]: mgrmap e7: x(active, since 1.01348s) 2024-03-08T20:47:47.490 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout { 2024-03-08T20:47:47.491 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-03-08T20:47:47.491 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "initialized": true 2024-03-08T20:47:47.491 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout } 2024-03-08T20:47:47.491 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:39.790+0000 7fd88ec00700 1 Processor -- start 2024-03-08T20:47:47.494 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:39.790+0000 7fd88ec00700 1 -- start start 2024-03-08T20:47:47.494 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:39.791+0000 7fd88ec00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd88807b300 0x7fd88807b720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:47:47.494 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:39.791+0000 7fd88ec00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd88807cfe0 con 0x7fd88807b300 2024-03-08T20:47:47.494 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:39.791+0000 7fd887e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd88807b300 0x7fd88807b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:47:47.494 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:39.791+0000 7fd887e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd88807b300 0x7fd88807b720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:39478/0 (socket says 172.21.15.107:39478) 2024-03-08T20:47:47.494 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:39.791+0000 7fd887e00700 1 -- 172.21.15.107:0/600714586 learned_addr learned my addr 172.21.15.107:0/600714586 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T20:47:47.494 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:39.791+0000 7fd887e00700 1 -- 172.21.15.107:0/600714586 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd88807bc60 con 0x7fd88807b300 2024-03-08T20:47:47.494 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:39.792+0000 7fd887e00700 1 --2- 172.21.15.107:0/600714586 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd88807b300 0x7fd88807b720 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fd8780098a0 tx=0x7fd878009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adbd9a653ed30f99 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:47:47.494 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:39.792+0000 7fd887400700 1 -- 172.21.15.107:0/600714586 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd878004160 con 0x7fd88807b300 2024-03-08T20:47:47.494 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:39.792+0000 7fd887400700 1 -- 172.21.15.107:0/600714586 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1071+0+0 (secure 0 0 0) 0x7fd878004300 con 0x7fd88807b300 2024-03-08T20:47:47.494 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:39.792+0000 7fd887400700 1 -- 172.21.15.107:0/600714586 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd878011a80 con 0x7fd88807b300 2024-03-08T20:47:47.495 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:39.793+0000 7fd88ec00700 1 -- 172.21.15.107:0/600714586 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd88807b300 msgr2=0x7fd88807b720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:47:47.495 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:39.793+0000 7fd88ec00700 1 --2- 172.21.15.107:0/600714586 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd88807b300 0x7fd88807b720 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fd8780098a0 tx=0x7fd878009bb0 comp rx=0 tx=0).stop 2024-03-08T20:47:47.495 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:39.793+0000 7fd88ec00700 1 -- 172.21.15.107:0/600714586 shutdown_connections 2024-03-08T20:47:47.495 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:39.793+0000 7fd88ec00700 1 --2- 172.21.15.107:0/600714586 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd88807b300 0x7fd88807b720 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:47:47.495 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:39.793+0000 7fd88ec00700 1 -- 172.21.15.107:0/600714586 >> 172.21.15.107:0/600714586 conn(0x7fd888102da0 msgr2=0x7fd8881051c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:47:47.495 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:39.793+0000 7fd88ec00700 1 -- 172.21.15.107:0/600714586 shutdown_connections 2024-03-08T20:47:47.495 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:39.793+0000 7fd88ec00700 1 -- 172.21.15.107:0/600714586 wait complete. 2024-03-08T20:47:47.495 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:39.794+0000 7fd88ec00700 1 Processor -- start 2024-03-08T20:47:47.495 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:39.795+0000 7fd88ec00700 1 -- start start 2024-03-08T20:47:47.495 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:39.795+0000 7fd88ec00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd88807b300 0x7fd88819da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:47:47.495 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:39.795+0000 7fd88ec00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd88810de70 con 0x7fd88807b300 2024-03-08T20:47:47.495 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:39.795+0000 7fd887e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd88807b300 0x7fd88819da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:47:47.495 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:39.795+0000 7fd887e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd88807b300 0x7fd88819da20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:39482/0 (socket says 172.21.15.107:39482) 2024-03-08T20:47:47.501 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:39.795+0000 7fd887e00700 1 -- 172.21.15.107:0/1553299602 learned_addr learned my addr 172.21.15.107:0/1553299602 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T20:47:47.501 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:39.796+0000 7fd887e00700 1 -- 172.21.15.107:0/1553299602 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd88819df60 con 0x7fd88807b300 2024-03-08T20:47:47.501 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:39.796+0000 7fd887e00700 1 --2- 172.21.15.107:0/1553299602 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd88807b300 0x7fd88819da20 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fd878011a50 tx=0x7fd878004290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:47:47.501 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:39.796+0000 7fd885600700 1 -- 172.21.15.107:0/1553299602 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd8780039f0 con 0x7fd88807b300 2024-03-08T20:47:47.501 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:39.796+0000 7fd885600700 1 -- 172.21.15.107:0/1553299602 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1071+0+0 (secure 0 0 0) 0x7fd878003b90 con 0x7fd88807b300 2024-03-08T20:47:47.501 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:39.796+0000 7fd885600700 1 -- 172.21.15.107:0/1553299602 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd87801a600 con 0x7fd88807b300 2024-03-08T20:47:47.501 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:39.796+0000 7fd88ec00700 1 -- 172.21.15.107:0/1553299602 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd88819e1f0 con 0x7fd88807b300 2024-03-08T20:47:47.501 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:39.796+0000 7fd88ec00700 1 -- 172.21.15.107:0/1553299602 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8881a0ee0 con 0x7fd88807b300 2024-03-08T20:47:47.501 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:39.797+0000 7fd885600700 1 -- 172.21.15.107:0/1553299602 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 5) v1 ==== 46106+0+0 (secure 0 0 0) 0x7fd87801a8b0 con 0x7fd88807b300 2024-03-08T20:47:47.501 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:39.798+0000 7fd885600700 1 --2- 172.21.15.107:0/1553299602 >> [v2:172.21.15.107:6800/3549936519,v1:172.21.15.107:6801/3549936519] conn(0x7fd86c038f70 0x7fd86c03b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:47:47.501 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:39.798+0000 7fd885600700 1 -- 172.21.15.107:0/1553299602 --> [v2:172.21.15.107:6800/3549936519,v1:172.21.15.107:6801/3549936519] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd86c03bb40 con 0x7fd86c038f70 2024-03-08T20:47:47.502 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:39.798+0000 7fd885600700 1 -- 172.21.15.107:0/1553299602 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fd87804d280 con 0x7fd88807b300 2024-03-08T20:47:47.502 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:39.798+0000 7fd87f400700 1 -- 172.21.15.107:0/1553299602 >> [v2:172.21.15.107:6800/3549936519,v1:172.21.15.107:6801/3549936519] conn(0x7fd86c038f70 msgr2=0x7fd86c03b430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.107:6800/3549936519 2024-03-08T20:47:47.502 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:39.798+0000 7fd87f400700 1 --2- 172.21.15.107:0/1553299602 >> [v2:172.21.15.107:6800/3549936519,v1:172.21.15.107:6801/3549936519] conn(0x7fd86c038f70 0x7fd86c03b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-08T20:47:47.502 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:39.999+0000 7fd87f400700 1 -- 172.21.15.107:0/1553299602 >> [v2:172.21.15.107:6800/3549936519,v1:172.21.15.107:6801/3549936519] conn(0x7fd86c038f70 msgr2=0x7fd86c03b430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.107:6800/3549936519 2024-03-08T20:47:47.502 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:39.999+0000 7fd87f400700 1 --2- 172.21.15.107:0/1553299602 >> [v2:172.21.15.107:6800/3549936519,v1:172.21.15.107:6801/3549936519] conn(0x7fd86c038f70 0x7fd86c03b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-08T20:47:47.502 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:40.400+0000 7fd87f400700 1 -- 172.21.15.107:0/1553299602 >> [v2:172.21.15.107:6800/3549936519,v1:172.21.15.107:6801/3549936519] conn(0x7fd86c038f70 msgr2=0x7fd86c03b430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.107:6800/3549936519 2024-03-08T20:47:47.502 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:40.400+0000 7fd87f400700 1 --2- 172.21.15.107:0/1553299602 >> [v2:172.21.15.107:6800/3549936519,v1:172.21.15.107:6801/3549936519] conn(0x7fd86c038f70 0x7fd86c03b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-08T20:47:47.502 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:41.201+0000 7fd87f400700 1 -- 172.21.15.107:0/1553299602 >> [v2:172.21.15.107:6800/3549936519,v1:172.21.15.107:6801/3549936519] conn(0x7fd86c038f70 msgr2=0x7fd86c03b430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.107:6800/3549936519 2024-03-08T20:47:47.502 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:41.201+0000 7fd87f400700 1 --2- 172.21.15.107:0/1553299602 >> [v2:172.21.15.107:6800/3549936519,v1:172.21.15.107:6801/3549936519] conn(0x7fd86c038f70 0x7fd86c03b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-08T20:47:47.502 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:42.803+0000 7fd87f400700 1 -- 172.21.15.107:0/1553299602 >> [v2:172.21.15.107:6800/3549936519,v1:172.21.15.107:6801/3549936519] conn(0x7fd86c038f70 msgr2=0x7fd86c03b430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.107:6800/3549936519 2024-03-08T20:47:47.502 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:42.803+0000 7fd87f400700 1 --2- 172.21.15.107:0/1553299602 >> [v2:172.21.15.107:6800/3549936519,v1:172.21.15.107:6801/3549936519] conn(0x7fd86c038f70 0x7fd86c03b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-03-08T20:47:47.502 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:45.711+0000 7fd885600700 1 -- 172.21.15.107:0/1553299602 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mgrmap(e 6) v1 ==== 45873+0+0 (secure 0 0 0) 0x7fd87802d3f0 con 0x7fd88807b300 2024-03-08T20:47:47.502 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:45.711+0000 7fd885600700 1 -- 172.21.15.107:0/1553299602 >> [v2:172.21.15.107:6800/3549936519,v1:172.21.15.107:6801/3549936519] conn(0x7fd86c038f70 msgr2=0x7fd86c03b430 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-08T20:47:47.503 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:45.711+0000 7fd885600700 1 --2- 172.21.15.107:0/1553299602 >> [v2:172.21.15.107:6800/3549936519,v1:172.21.15.107:6801/3549936519] conn(0x7fd86c038f70 0x7fd86c03b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:47:47.503 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:46.715+0000 7fd885600700 1 -- 172.21.15.107:0/1553299602 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mgrmap(e 7) v1 ==== 46000+0+0 (secure 0 0 0) 0x7fd87804f0c0 con 0x7fd88807b300 2024-03-08T20:47:47.503 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:46.716+0000 7fd885600700 1 --2- 172.21.15.107:0/1553299602 >> [v2:172.21.15.107:6800/426372446,v1:172.21.15.107:6801/426372446] conn(0x7fd86c03c860 0x7fd86c03ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:47:47.503 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:46.716+0000 7fd885600700 1 -- 172.21.15.107:0/1553299602 --> [v2:172.21.15.107:6800/426372446,v1:172.21.15.107:6801/426372446] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd878022e80 con 0x7fd86c03c860 2024-03-08T20:47:47.503 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:46.718+0000 7fd87f400700 1 --2- 172.21.15.107:0/1553299602 >> [v2:172.21.15.107:6800/426372446,v1:172.21.15.107:6801/426372446] conn(0x7fd86c03c860 0x7fd86c03ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:47:47.503 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:46.721+0000 7fd87f400700 1 --2- 172.21.15.107:0/1553299602 >> [v2:172.21.15.107:6800/426372446,v1:172.21.15.107:6801/426372446] conn(0x7fd86c03c860 0x7fd86c03ec50 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fd870003a10 tx=0x7fd8700092b0 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:47:47.503 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:46.723+0000 7fd885600700 1 -- 172.21.15.107:0/1553299602 <== mgr.14120 v2:172.21.15.107:6800/426372446 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+7774 (secure 0 0 0) 0x7fd878022e80 con 0x7fd86c03c860 2024-03-08T20:47:47.503 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:46.730+0000 7fd88ec00700 1 -- 172.21.15.107:0/1553299602 --> [v2:172.21.15.107:6800/426372446,v1:172.21.15.107:6801/426372446] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7fd88819e6d0 con 0x7fd86c03c860 2024-03-08T20:47:47.503 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:46.730+0000 7fd885600700 1 -- 172.21.15.107:0/1553299602 <== mgr.14120 v2:172.21.15.107:6800/426372446 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7fd88819e6d0 con 0x7fd86c03c860 2024-03-08T20:47:47.503 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:46.731+0000 7fd88ec00700 1 -- 172.21.15.107:0/1553299602 >> [v2:172.21.15.107:6800/426372446,v1:172.21.15.107:6801/426372446] conn(0x7fd86c03c860 msgr2=0x7fd86c03ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:47:47.503 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:46.731+0000 7fd88ec00700 1 --2- 172.21.15.107:0/1553299602 >> [v2:172.21.15.107:6800/426372446,v1:172.21.15.107:6801/426372446] conn(0x7fd86c03c860 0x7fd86c03ec50 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fd870003a10 tx=0x7fd8700092b0 comp rx=0 tx=0).stop 2024-03-08T20:47:47.503 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:46.731+0000 7fd88ec00700 1 -- 172.21.15.107:0/1553299602 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd88807b300 msgr2=0x7fd88819da20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:47:47.504 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:46.731+0000 7fd88ec00700 1 --2- 172.21.15.107:0/1553299602 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd88807b300 0x7fd88819da20 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fd878011a50 tx=0x7fd878004290 comp rx=0 tx=0).stop 2024-03-08T20:47:47.504 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:46.731+0000 7fd88ec00700 1 -- 172.21.15.107:0/1553299602 shutdown_connections 2024-03-08T20:47:47.504 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:46.731+0000 7fd88ec00700 1 --2- 172.21.15.107:0/1553299602 >> [v2:172.21.15.107:6800/426372446,v1:172.21.15.107:6801/426372446] conn(0x7fd86c03c860 0x7fd86c03ec50 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:47:47.504 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:46.731+0000 7fd88ec00700 1 --2- 172.21.15.107:0/1553299602 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd88807b300 0x7fd88819da20 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:47:47.504 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:46.731+0000 7fd88ec00700 1 --2- 172.21.15.107:0/1553299602 >> [v2:172.21.15.107:6800/3549936519,v1:172.21.15.107:6801/3549936519] conn(0x7fd86c038f70 0x7fd86c03b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:47:47.504 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:46.731+0000 7fd88ec00700 1 -- 172.21.15.107:0/1553299602 >> 172.21.15.107:0/1553299602 conn(0x7fd888102da0 msgr2=0x7fd888103930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:47:47.504 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:46.731+0000 7fd88ec00700 1 -- 172.21.15.107:0/1553299602 shutdown_connections 2024-03-08T20:47:47.504 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:46.731+0000 7fd88ec00700 1 -- 172.21.15.107:0/1553299602 wait complete. 2024-03-08T20:47:47.504 INFO:teuthology.orchestra.run.smithi107.stdout:mgr epoch 5 is available 2024-03-08T20:47:47.504 INFO:teuthology.orchestra.run.smithi107.stdout:Setting orchestrator backend to cephadm... 2024-03-08T20:47:48.022 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:47 smithi107 ceph-mon[31381]: [08/Mar/2024:20:47:46] ENGINE Bus STARTING 2024-03-08T20:47:48.022 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:47 smithi107 ceph-mon[31381]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-03-08T20:47:48.022 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:47 smithi107 ceph-mon[31381]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-03-08T20:47:48.022 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:47 smithi107 ceph-mon[31381]: [08/Mar/2024:20:47:46] ENGINE Serving on https://172.21.15.107:7150 2024-03-08T20:47:48.022 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:47 smithi107 ceph-mon[31381]: [08/Mar/2024:20:47:46] ENGINE Serving on http://172.21.15.107:8765 2024-03-08T20:47:48.022 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:47 smithi107 ceph-mon[31381]: [08/Mar/2024:20:47:46] ENGINE Bus STARTED 2024-03-08T20:47:48.022 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:47 smithi107 ceph-mon[31381]: from='mgr.14120 172.21.15.107:0/283560839' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T20:47:48.978 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:48 smithi107 ceph-mon[31381]: mgrmap e8: x(active, since 2s) 2024-03-08T20:47:48.978 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:48 smithi107 ceph-mon[31381]: from='mgr.14120 172.21.15.107:0/283560839' entity='mgr.x' 2024-03-08T20:47:48.979 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:48 smithi107 ceph-mon[31381]: from='mgr.14120 172.21.15.107:0/283560839' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T20:47:49.150 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:48.108+0000 7fd583600700 1 Processor -- start 2024-03-08T20:47:49.151 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:48.108+0000 7fd583600700 1 -- start start 2024-03-08T20:47:49.152 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:48.109+0000 7fd583600700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd57c108e30 0x7fd57c109250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:47:49.152 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:48.109+0000 7fd583600700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd57c1073c0 con 0x7fd57c108e30 2024-03-08T20:47:49.152 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:48.109+0000 7fd580c00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd57c108e30 0x7fd57c109250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:47:49.152 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:48.109+0000 7fd580c00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd57c108e30 0x7fd57c109250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:39894/0 (socket says 172.21.15.107:39894) 2024-03-08T20:47:49.152 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:48.109+0000 7fd580c00700 1 -- 172.21.15.107:0/221181987 learned_addr learned my addr 172.21.15.107:0/221181987 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T20:47:49.152 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:48.110+0000 7fd580c00700 1 -- 172.21.15.107:0/221181987 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd57c107540 con 0x7fd57c108e30 2024-03-08T20:47:49.152 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:48.110+0000 7fd580c00700 1 --2- 172.21.15.107:0/221181987 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd57c108e30 0x7fd57c109250 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fd5640098a0 tx=0x7fd564009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec81358a7bc31e44 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:47:49.157 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:48.110+0000 7fd57be00700 1 -- 172.21.15.107:0/221181987 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd564004160 con 0x7fd57c108e30 2024-03-08T20:47:49.157 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:48.110+0000 7fd57be00700 1 -- 172.21.15.107:0/221181987 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1071+0+0 (secure 0 0 0) 0x7fd564004300 con 0x7fd57c108e30 2024-03-08T20:47:49.157 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:48.110+0000 7fd57be00700 1 -- 172.21.15.107:0/221181987 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd564011a80 con 0x7fd57c108e30 2024-03-08T20:47:49.157 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:48.111+0000 7fd583600700 1 -- 172.21.15.107:0/221181987 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd57c108e30 msgr2=0x7fd57c109250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:47:49.157 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:48.111+0000 7fd583600700 1 --2- 172.21.15.107:0/221181987 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd57c108e30 0x7fd57c109250 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fd5640098a0 tx=0x7fd564009bb0 comp rx=0 tx=0).stop 2024-03-08T20:47:49.157 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:48.111+0000 7fd583600700 1 -- 172.21.15.107:0/221181987 shutdown_connections 2024-03-08T20:47:49.157 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:48.111+0000 7fd583600700 1 --2- 172.21.15.107:0/221181987 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd57c108e30 0x7fd57c109250 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:47:49.157 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:48.111+0000 7fd583600700 1 -- 172.21.15.107:0/221181987 >> 172.21.15.107:0/221181987 conn(0x7fd57c102da0 msgr2=0x7fd57c1051c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:47:49.157 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:48.111+0000 7fd583600700 1 -- 172.21.15.107:0/221181987 shutdown_connections 2024-03-08T20:47:49.157 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:48.112+0000 7fd583600700 1 -- 172.21.15.107:0/221181987 wait complete. 2024-03-08T20:47:49.157 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:48.113+0000 7fd583600700 1 Processor -- start 2024-03-08T20:47:49.157 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:48.113+0000 7fd583600700 1 -- start start 2024-03-08T20:47:49.158 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:48.113+0000 7fd583600700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd57c108e30 0x7fd57c19b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:47:49.158 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:48.113+0000 7fd583600700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd57c10bd00 con 0x7fd57c108e30 2024-03-08T20:47:49.158 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:48.113+0000 7fd580c00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd57c108e30 0x7fd57c19b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:47:49.158 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:48.113+0000 7fd580c00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd57c108e30 0x7fd57c19b790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:39906/0 (socket says 172.21.15.107:39906) 2024-03-08T20:47:49.158 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:48.113+0000 7fd580c00700 1 -- 172.21.15.107:0/1707823593 learned_addr learned my addr 172.21.15.107:0/1707823593 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T20:47:49.158 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:48.114+0000 7fd580c00700 1 -- 172.21.15.107:0/1707823593 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd57c19bcd0 con 0x7fd57c108e30 2024-03-08T20:47:49.158 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:48.114+0000 7fd580c00700 1 --2- 172.21.15.107:0/1707823593 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd57c108e30 0x7fd57c19b790 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fd564006af0 tx=0x7fd564011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:47:49.158 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:48.114+0000 7fd57a000700 1 -- 172.21.15.107:0/1707823593 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd5640039e0 con 0x7fd57c108e30 2024-03-08T20:47:49.158 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:48.114+0000 7fd57a000700 1 -- 172.21.15.107:0/1707823593 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1071+0+0 (secure 0 0 0) 0x7fd564003b80 con 0x7fd57c108e30 2024-03-08T20:47:49.158 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:48.114+0000 7fd583600700 1 -- 172.21.15.107:0/1707823593 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd57c19bf60 con 0x7fd57c108e30 2024-03-08T20:47:49.158 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:48.114+0000 7fd57a000700 1 -- 172.21.15.107:0/1707823593 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd5640196a0 con 0x7fd57c108e30 2024-03-08T20:47:49.158 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:48.114+0000 7fd583600700 1 -- 172.21.15.107:0/1707823593 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd57c19ec50 con 0x7fd57c108e30 2024-03-08T20:47:49.158 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:48.116+0000 7fd57a000700 1 -- 172.21.15.107:0/1707823593 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 8) v1 ==== 46106+0+0 (secure 0 0 0) 0x7fd56401a430 con 0x7fd57c108e30 2024-03-08T20:47:49.159 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:48.116+0000 7fd572a00700 1 -- 172.21.15.107:0/1707823593 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd550005330 con 0x7fd57c108e30 2024-03-08T20:47:49.159 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:48.116+0000 7fd57a000700 1 --2- 172.21.15.107:0/1707823593 >> [v2:172.21.15.107:6800/426372446,v1:172.21.15.107:6801/426372446] conn(0x7fd55c038f20 0x7fd55c03b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:47:49.159 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:48.116+0000 7fd57a000700 1 -- 172.21.15.107:0/1707823593 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fd56404d390 con 0x7fd57c108e30 2024-03-08T20:47:49.159 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:48.116+0000 7fd573e00700 1 --2- 172.21.15.107:0/1707823593 >> [v2:172.21.15.107:6800/426372446,v1:172.21.15.107:6801/426372446] conn(0x7fd55c038f20 0x7fd55c03b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:47:49.159 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:48.117+0000 7fd573e00700 1 --2- 172.21.15.107:0/1707823593 >> [v2:172.21.15.107:6800/426372446,v1:172.21.15.107:6801/426372446] conn(0x7fd55c038f20 0x7fd55c03b3e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fd56c006fd0 tx=0x7fd56c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:47:49.159 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:48.122+0000 7fd57a000700 1 -- 172.21.15.107:0/1707823593 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7fd56401a720 con 0x7fd57c108e30 2024-03-08T20:47:49.159 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:48.358+0000 7fd572a00700 1 -- 172.21.15.107:0/1707823593 --> [v2:172.21.15.107:6800/426372446,v1:172.21.15.107:6801/426372446] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) v1 -- 0x7fd550000bc0 con 0x7fd55c038f20 2024-03-08T20:47:49.159 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:48.367+0000 7fd57a000700 1 -- 172.21.15.107:0/1707823593 <== mgr.14120 v2:172.21.15.107:6800/426372446 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fd550000bc0 con 0x7fd55c038f20 2024-03-08T20:47:49.159 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:48.370+0000 7fd572a00700 1 -- 172.21.15.107:0/1707823593 >> [v2:172.21.15.107:6800/426372446,v1:172.21.15.107:6801/426372446] conn(0x7fd55c038f20 msgr2=0x7fd55c03b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:47:49.159 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:48.370+0000 7fd572a00700 1 --2- 172.21.15.107:0/1707823593 >> [v2:172.21.15.107:6800/426372446,v1:172.21.15.107:6801/426372446] conn(0x7fd55c038f20 0x7fd55c03b3e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fd56c006fd0 tx=0x7fd56c006e40 comp rx=0 tx=0).stop 2024-03-08T20:47:49.159 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:48.370+0000 7fd572a00700 1 -- 172.21.15.107:0/1707823593 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd57c108e30 msgr2=0x7fd57c19b790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:47:49.159 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:48.370+0000 7fd572a00700 1 --2- 172.21.15.107:0/1707823593 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd57c108e30 0x7fd57c19b790 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fd564006af0 tx=0x7fd564011c40 comp rx=0 tx=0).stop 2024-03-08T20:47:49.159 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:48.370+0000 7fd572a00700 1 -- 172.21.15.107:0/1707823593 shutdown_connections 2024-03-08T20:47:49.160 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:48.370+0000 7fd572a00700 1 --2- 172.21.15.107:0/1707823593 >> [v2:172.21.15.107:6800/426372446,v1:172.21.15.107:6801/426372446] conn(0x7fd55c038f20 0x7fd55c03b3e0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:47:49.160 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:48.370+0000 7fd572a00700 1 --2- 172.21.15.107:0/1707823593 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd57c108e30 0x7fd57c19b790 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:47:49.160 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:48.370+0000 7fd572a00700 1 -- 172.21.15.107:0/1707823593 >> 172.21.15.107:0/1707823593 conn(0x7fd57c102da0 msgr2=0x7fd57c1038d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:47:49.160 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:48.370+0000 7fd572a00700 1 -- 172.21.15.107:0/1707823593 shutdown_connections 2024-03-08T20:47:49.160 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:48.370+0000 7fd572a00700 1 -- 172.21.15.107:0/1707823593 wait complete. 2024-03-08T20:47:49.973 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:49 smithi107 ceph-mon[31381]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-03-08T20:47:50.636 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout value unchanged 2024-03-08T20:47:50.636 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:49.719+0000 7fd145e00700 1 Processor -- start 2024-03-08T20:47:50.636 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:49.719+0000 7fd145e00700 1 -- start start 2024-03-08T20:47:50.636 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:49.720+0000 7fd145e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd140109960 0x7fd140109d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:47:50.636 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:49.720+0000 7fd145e00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd14010a350 con 0x7fd140109960 2024-03-08T20:47:50.636 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:49.720+0000 7fd13ea00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd140109960 0x7fd140109d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:47:50.636 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:49.720+0000 7fd13ea00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd140109960 0x7fd140109d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:39916/0 (socket says 172.21.15.107:39916) 2024-03-08T20:47:50.637 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:49.720+0000 7fd13ea00700 1 -- 172.21.15.107:0/1151074121 learned_addr learned my addr 172.21.15.107:0/1151074121 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T20:47:50.637 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:49.721+0000 7fd13ea00700 1 -- 172.21.15.107:0/1151074121 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd14010ab10 con 0x7fd140109960 2024-03-08T20:47:50.637 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:49.721+0000 7fd13ea00700 1 --2- 172.21.15.107:0/1151074121 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd140109960 0x7fd140109d80 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fd1300098a0 tx=0x7fd130009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ef4eb5307398b1c server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:47:50.637 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:49.721+0000 7fd13d600700 1 -- 172.21.15.107:0/1151074121 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd130004160 con 0x7fd140109960 2024-03-08T20:47:50.637 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:49.721+0000 7fd13d600700 1 -- 172.21.15.107:0/1151074121 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1071+0+0 (secure 0 0 0) 0x7fd130004300 con 0x7fd140109960 2024-03-08T20:47:50.637 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:49.721+0000 7fd13d600700 1 -- 172.21.15.107:0/1151074121 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd130011a80 con 0x7fd140109960 2024-03-08T20:47:50.637 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:49.722+0000 7fd145e00700 1 -- 172.21.15.107:0/1151074121 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd140109960 msgr2=0x7fd140109d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:47:50.637 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:49.722+0000 7fd145e00700 1 --2- 172.21.15.107:0/1151074121 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd140109960 0x7fd140109d80 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fd1300098a0 tx=0x7fd130009bb0 comp rx=0 tx=0).stop 2024-03-08T20:47:50.637 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:49.722+0000 7fd145e00700 1 -- 172.21.15.107:0/1151074121 shutdown_connections 2024-03-08T20:47:50.637 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:49.722+0000 7fd145e00700 1 --2- 172.21.15.107:0/1151074121 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd140109960 0x7fd140109d80 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:47:50.637 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:49.722+0000 7fd145e00700 1 -- 172.21.15.107:0/1151074121 >> 172.21.15.107:0/1151074121 conn(0x7fd14007cdd0 msgr2=0x7fd1401074f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:47:50.637 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:49.723+0000 7fd145e00700 1 -- 172.21.15.107:0/1151074121 shutdown_connections 2024-03-08T20:47:50.638 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:49.723+0000 7fd145e00700 1 -- 172.21.15.107:0/1151074121 wait complete. 2024-03-08T20:47:50.638 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:49.724+0000 7fd145e00700 1 Processor -- start 2024-03-08T20:47:50.638 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:49.724+0000 7fd145e00700 1 -- start start 2024-03-08T20:47:50.638 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:49.724+0000 7fd145e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd140109960 0x7fd14019d880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:47:50.638 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:49.724+0000 7fd145e00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd14010dec0 con 0x7fd140109960 2024-03-08T20:47:50.638 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:49.724+0000 7fd13ea00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd140109960 0x7fd14019d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:47:50.638 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:49.724+0000 7fd13ea00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd140109960 0x7fd14019d880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:39918/0 (socket says 172.21.15.107:39918) 2024-03-08T20:47:50.638 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:49.724+0000 7fd13ea00700 1 -- 172.21.15.107:0/1673603611 learned_addr learned my addr 172.21.15.107:0/1673603611 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T20:47:50.638 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:49.725+0000 7fd13ea00700 1 -- 172.21.15.107:0/1673603611 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd14019ddc0 con 0x7fd140109960 2024-03-08T20:47:50.638 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:49.725+0000 7fd13ea00700 1 --2- 172.21.15.107:0/1673603611 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd140109960 0x7fd14019d880 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fd130000c00 tx=0x7fd130011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:47:50.638 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:49.725+0000 7fd144a00700 1 -- 172.21.15.107:0/1673603611 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd1300039f0 con 0x7fd140109960 2024-03-08T20:47:50.639 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:49.725+0000 7fd144a00700 1 -- 172.21.15.107:0/1673603611 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1071+0+0 (secure 0 0 0) 0x7fd130003b90 con 0x7fd140109960 2024-03-08T20:47:50.639 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:49.725+0000 7fd145e00700 1 -- 172.21.15.107:0/1673603611 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd14019e050 con 0x7fd140109960 2024-03-08T20:47:50.639 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:49.725+0000 7fd144a00700 1 -- 172.21.15.107:0/1673603611 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd13001a740 con 0x7fd140109960 2024-03-08T20:47:50.639 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:49.725+0000 7fd145e00700 1 -- 172.21.15.107:0/1673603611 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1401a0d40 con 0x7fd140109960 2024-03-08T20:47:50.639 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:49.726+0000 7fd144a00700 1 -- 172.21.15.107:0/1673603611 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 8) v1 ==== 46106+0+0 (secure 0 0 0) 0x7fd13001a9f0 con 0x7fd140109960 2024-03-08T20:47:50.639 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:49.726+0000 7fd145e00700 1 -- 172.21.15.107:0/1673603611 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd140063410 con 0x7fd140109960 2024-03-08T20:47:50.639 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:49.727+0000 7fd144a00700 1 --2- 172.21.15.107:0/1673603611 >> [v2:172.21.15.107:6800/426372446,v1:172.21.15.107:6801/426372446] conn(0x7fd124038f70 0x7fd12403b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:47:50.639 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:49.727+0000 7fd144a00700 1 -- 172.21.15.107:0/1673603611 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fd13004d290 con 0x7fd140109960 2024-03-08T20:47:50.639 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:49.727+0000 7fd13e000700 1 --2- 172.21.15.107:0/1673603611 >> [v2:172.21.15.107:6800/426372446,v1:172.21.15.107:6801/426372446] conn(0x7fd124038f70 0x7fd12403b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:47:50.639 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:49.728+0000 7fd13e000700 1 --2- 172.21.15.107:0/1673603611 >> [v2:172.21.15.107:6800/426372446,v1:172.21.15.107:6801/426372446] conn(0x7fd124038f70 0x7fd12403b430 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fd12c006fd0 tx=0x7fd12c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:47:50.639 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:49.733+0000 7fd144a00700 1 -- 172.21.15.107:0/1673603611 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7fd13004cbe0 con 0x7fd140109960 2024-03-08T20:47:50.639 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:49.957+0000 7fd145e00700 1 -- 172.21.15.107:0/1673603611 --> [v2:172.21.15.107:6800/426372446,v1:172.21.15.107:6801/426372446] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) v1 -- 0x7fd140107370 con 0x7fd124038f70 2024-03-08T20:47:50.639 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:49.959+0000 7fd144a00700 1 -- 172.21.15.107:0/1673603611 <== mgr.14120 v2:172.21.15.107:6800/426372446 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+16 (secure 0 0 0) 0x7fd140107370 con 0x7fd124038f70 2024-03-08T20:47:50.640 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:49.961+0000 7fd145e00700 1 -- 172.21.15.107:0/1673603611 >> [v2:172.21.15.107:6800/426372446,v1:172.21.15.107:6801/426372446] conn(0x7fd124038f70 msgr2=0x7fd12403b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:47:50.640 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:49.961+0000 7fd145e00700 1 --2- 172.21.15.107:0/1673603611 >> [v2:172.21.15.107:6800/426372446,v1:172.21.15.107:6801/426372446] conn(0x7fd124038f70 0x7fd12403b430 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fd12c006fd0 tx=0x7fd12c006e40 comp rx=0 tx=0).stop 2024-03-08T20:47:50.640 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:49.961+0000 7fd145e00700 1 -- 172.21.15.107:0/1673603611 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd140109960 msgr2=0x7fd14019d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:47:50.640 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:49.961+0000 7fd145e00700 1 --2- 172.21.15.107:0/1673603611 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd140109960 0x7fd14019d880 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fd130000c00 tx=0x7fd130011b00 comp rx=0 tx=0).stop 2024-03-08T20:47:50.640 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:49.961+0000 7fd145e00700 1 -- 172.21.15.107:0/1673603611 shutdown_connections 2024-03-08T20:47:50.640 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:49.961+0000 7fd145e00700 1 --2- 172.21.15.107:0/1673603611 >> [v2:172.21.15.107:6800/426372446,v1:172.21.15.107:6801/426372446] conn(0x7fd124038f70 0x7fd12403b430 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:47:50.640 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:49.961+0000 7fd145e00700 1 --2- 172.21.15.107:0/1673603611 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd140109960 0x7fd14019d880 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:47:50.640 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:49.961+0000 7fd145e00700 1 -- 172.21.15.107:0/1673603611 >> 172.21.15.107:0/1673603611 conn(0x7fd14007cdd0 msgr2=0x7fd140106c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:47:50.640 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:49.962+0000 7fd145e00700 1 -- 172.21.15.107:0/1673603611 shutdown_connections 2024-03-08T20:47:50.640 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:49.962+0000 7fd145e00700 1 -- 172.21.15.107:0/1673603611 wait complete. 2024-03-08T20:47:50.640 INFO:teuthology.orchestra.run.smithi107.stdout:Generating ssh key... 2024-03-08T20:47:50.930 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:50 smithi107 ceph-mon[31381]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-03-08T20:47:52.225 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 08 20:47:51 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-x[31606]: Generating public/private rsa key pair. 2024-03-08T20:47:52.225 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 08 20:47:51 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-x[31606]: Your identification has been saved in /tmp/tmp1y6gii45/key. 2024-03-08T20:47:52.225 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 08 20:47:51 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-x[31606]: Your public key has been saved in /tmp/tmp1y6gii45/key.pub. 2024-03-08T20:47:52.225 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 08 20:47:51 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-x[31606]: The key fingerprint is: 2024-03-08T20:47:52.225 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 08 20:47:51 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-x[31606]: SHA256:Aoh7LIFDnWo8xJisDyzOAiObTdG2LZDskHF/+GkhWEs ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715 2024-03-08T20:47:52.225 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 08 20:47:51 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-x[31606]: The key's randomart image is: 2024-03-08T20:47:52.225 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 08 20:47:51 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-x[31606]: +---[RSA 3072]----+ 2024-03-08T20:47:52.226 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 08 20:47:51 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-x[31606]: |=*ooE | 2024-03-08T20:47:52.226 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 08 20:47:51 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-x[31606]: |B*=Ooo | 2024-03-08T20:47:52.226 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 08 20:47:51 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-x[31606]: |Xo++*oo | 2024-03-08T20:47:52.226 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 08 20:47:51 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-x[31606]: |B@o o=.o | 2024-03-08T20:47:52.226 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 08 20:47:51 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-x[31606]: |XO+ .= S | 2024-03-08T20:47:52.226 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 08 20:47:51 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-x[31606]: |+=o . . | 2024-03-08T20:47:52.226 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 08 20:47:51 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-x[31606]: |. | 2024-03-08T20:47:52.226 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 08 20:47:51 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-x[31606]: | | 2024-03-08T20:47:52.226 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 08 20:47:51 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-x[31606]: | | 2024-03-08T20:47:52.226 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 08 20:47:51 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-x[31606]: +----[SHA256]-----+ 2024-03-08T20:47:52.580 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:51.233+0000 7f3661600700 1 Processor -- start 2024-03-08T20:47:52.580 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:51.233+0000 7f3661600700 1 -- start start 2024-03-08T20:47:52.580 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:51.233+0000 7f3661600700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f365c107780 0x7f365c107ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:47:52.581 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:51.233+0000 7f3661600700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f365c108170 con 0x7f365c107780 2024-03-08T20:47:52.581 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:51.234+0000 7f365aa00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f365c107780 0x7f365c107ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:47:52.581 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:51.234+0000 7f365aa00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f365c107780 0x7f365c107ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:39920/0 (socket says 172.21.15.107:39920) 2024-03-08T20:47:52.581 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:51.234+0000 7f365aa00700 1 -- 172.21.15.107:0/1574813684 learned_addr learned my addr 172.21.15.107:0/1574813684 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T20:47:52.581 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:51.234+0000 7f365aa00700 1 -- 172.21.15.107:0/1574813684 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f365c108930 con 0x7f365c107780 2024-03-08T20:47:52.581 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:51.235+0000 7f365aa00700 1 --2- 172.21.15.107:0/1574813684 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f365c107780 0x7f365c107ba0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f364c0098a0 tx=0x7f364c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b32d122edb581391 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:47:52.581 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:51.235+0000 7f3659600700 1 -- 172.21.15.107:0/1574813684 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f364c004160 con 0x7f365c107780 2024-03-08T20:47:52.581 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:51.235+0000 7f3659600700 1 -- 172.21.15.107:0/1574813684 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1071+0+0 (secure 0 0 0) 0x7f364c004300 con 0x7f365c107780 2024-03-08T20:47:52.581 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:51.235+0000 7f3659600700 1 -- 172.21.15.107:0/1574813684 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f364c011a80 con 0x7f365c107780 2024-03-08T20:47:52.581 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:51.236+0000 7f3661600700 1 -- 172.21.15.107:0/1574813684 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f365c107780 msgr2=0x7f365c107ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:47:52.581 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:51.236+0000 7f3661600700 1 --2- 172.21.15.107:0/1574813684 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f365c107780 0x7f365c107ba0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f364c0098a0 tx=0x7f364c009bb0 comp rx=0 tx=0).stop 2024-03-08T20:47:52.581 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:51.236+0000 7f3661600700 1 -- 172.21.15.107:0/1574813684 shutdown_connections 2024-03-08T20:47:52.582 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:51.236+0000 7f3661600700 1 --2- 172.21.15.107:0/1574813684 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f365c107780 0x7f365c107ba0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:47:52.582 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:51.236+0000 7f3661600700 1 -- 172.21.15.107:0/1574813684 >> 172.21.15.107:0/1574813684 conn(0x7f365c102da0 msgr2=0x7f365c1051c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:47:52.582 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:51.236+0000 7f3661600700 1 -- 172.21.15.107:0/1574813684 shutdown_connections 2024-03-08T20:47:52.582 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:51.236+0000 7f3661600700 1 -- 172.21.15.107:0/1574813684 wait complete. 2024-03-08T20:47:52.582 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:51.237+0000 7f3661600700 1 Processor -- start 2024-03-08T20:47:52.582 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:51.237+0000 7f3661600700 1 -- start start 2024-03-08T20:47:52.582 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:51.238+0000 7f3661600700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f365c107780 0x7f365c07cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:47:52.582 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:51.238+0000 7f3661600700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f365c10bce0 con 0x7f365c107780 2024-03-08T20:47:52.582 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:51.238+0000 7f365aa00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f365c107780 0x7f365c07cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:47:52.582 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:51.238+0000 7f365aa00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f365c107780 0x7f365c07cb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:39934/0 (socket says 172.21.15.107:39934) 2024-03-08T20:47:52.582 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:51.238+0000 7f365aa00700 1 -- 172.21.15.107:0/1207605888 learned_addr learned my addr 172.21.15.107:0/1207605888 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T20:47:52.582 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:51.238+0000 7f365aa00700 1 -- 172.21.15.107:0/1207605888 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f365c07d0d0 con 0x7f365c107780 2024-03-08T20:47:52.582 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:51.238+0000 7f365aa00700 1 --2- 172.21.15.107:0/1207605888 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f365c107780 0x7f365c07cb90 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f364c000c00 tx=0x7f364c011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:47:52.583 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:51.239+0000 7f3653400700 1 -- 172.21.15.107:0/1207605888 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f364c0039f0 con 0x7f365c107780 2024-03-08T20:47:52.583 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:51.239+0000 7f3653400700 1 -- 172.21.15.107:0/1207605888 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1071+0+0 (secure 0 0 0) 0x7f364c003b90 con 0x7f365c107780 2024-03-08T20:47:52.583 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:51.239+0000 7f3661600700 1 -- 172.21.15.107:0/1207605888 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f365c07b2c0 con 0x7f365c107780 2024-03-08T20:47:52.583 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:51.239+0000 7f3653400700 1 -- 172.21.15.107:0/1207605888 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f364c01a740 con 0x7f365c107780 2024-03-08T20:47:52.583 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:51.239+0000 7f3661600700 1 -- 172.21.15.107:0/1207605888 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f365c07b800 con 0x7f365c107780 2024-03-08T20:47:52.583 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:51.240+0000 7f3653400700 1 -- 172.21.15.107:0/1207605888 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 8) v1 ==== 46106+0+0 (secure 0 0 0) 0x7f364c024af0 con 0x7f365c107780 2024-03-08T20:47:52.583 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:51.240+0000 7f3661600700 1 -- 172.21.15.107:0/1207605888 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f365c063410 con 0x7f365c107780 2024-03-08T20:47:52.583 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:51.240+0000 7f3653400700 1 --2- 172.21.15.107:0/1207605888 >> [v2:172.21.15.107:6800/426372446,v1:172.21.15.107:6801/426372446] conn(0x7f3640038f70 0x7f364003b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:47:52.583 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:51.240+0000 7f3653400700 1 -- 172.21.15.107:0/1207605888 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f364c04d1f0 con 0x7f365c107780 2024-03-08T20:47:52.583 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:51.241+0000 7f365a000700 1 --2- 172.21.15.107:0/1207605888 >> [v2:172.21.15.107:6800/426372446,v1:172.21.15.107:6801/426372446] conn(0x7f3640038f70 0x7f364003b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:47:52.583 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:51.242+0000 7f365a000700 1 --2- 172.21.15.107:0/1207605888 >> [v2:172.21.15.107:6800/426372446,v1:172.21.15.107:6801/426372446] conn(0x7f3640038f70 0x7f364003b430 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f3648006fd0 tx=0x7f3648006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:47:52.583 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:51.246+0000 7f3653400700 1 -- 172.21.15.107:0/1207605888 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f364c00bca0 con 0x7f365c107780 2024-03-08T20:47:52.583 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:51.465+0000 7f3661600700 1 -- 172.21.15.107:0/1207605888 --> [v2:172.21.15.107:6800/426372446,v1:172.21.15.107:6801/426372446] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f365c103fe0 con 0x7f3640038f70 2024-03-08T20:47:52.584 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:51.929+0000 7f3653400700 1 -- 172.21.15.107:0/1207605888 <== mgr.14120 v2:172.21.15.107:6800/426372446 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f365c103fe0 con 0x7f3640038f70 2024-03-08T20:47:52.584 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:51.932+0000 7f3661600700 1 -- 172.21.15.107:0/1207605888 >> [v2:172.21.15.107:6800/426372446,v1:172.21.15.107:6801/426372446] conn(0x7f3640038f70 msgr2=0x7f364003b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:47:52.584 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:51.932+0000 7f3661600700 1 --2- 172.21.15.107:0/1207605888 >> [v2:172.21.15.107:6800/426372446,v1:172.21.15.107:6801/426372446] conn(0x7f3640038f70 0x7f364003b430 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f3648006fd0 tx=0x7f3648006e40 comp rx=0 tx=0).stop 2024-03-08T20:47:52.584 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:51.932+0000 7f3661600700 1 -- 172.21.15.107:0/1207605888 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f365c107780 msgr2=0x7f365c07cb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:47:52.584 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:51.932+0000 7f3661600700 1 --2- 172.21.15.107:0/1207605888 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f365c107780 0x7f365c07cb90 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f364c000c00 tx=0x7f364c011b00 comp rx=0 tx=0).stop 2024-03-08T20:47:52.584 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:51.932+0000 7f3661600700 1 -- 172.21.15.107:0/1207605888 shutdown_connections 2024-03-08T20:47:52.584 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:51.932+0000 7f3661600700 1 --2- 172.21.15.107:0/1207605888 >> [v2:172.21.15.107:6800/426372446,v1:172.21.15.107:6801/426372446] conn(0x7f3640038f70 0x7f364003b430 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:47:52.584 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:51.932+0000 7f3661600700 1 --2- 172.21.15.107:0/1207605888 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f365c107780 0x7f365c07cb90 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:47:52.584 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:51.932+0000 7f3661600700 1 -- 172.21.15.107:0/1207605888 >> 172.21.15.107:0/1207605888 conn(0x7f365c102da0 msgr2=0x7f365c1038d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:47:52.584 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:51.933+0000 7f3661600700 1 -- 172.21.15.107:0/1207605888 shutdown_connections 2024-03-08T20:47:52.584 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:51.933+0000 7f3661600700 1 -- 172.21.15.107:0/1207605888 wait complete. 2024-03-08T20:47:53.130 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:52 smithi107 ceph-mon[31381]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-03-08T20:47:53.130 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:52 smithi107 ceph-mon[31381]: Generating ssh key... 2024-03-08T20:47:53.130 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:52 smithi107 ceph-mon[31381]: from='mgr.14120 172.21.15.107:0/283560839' entity='mgr.x' 2024-03-08T20:47:53.130 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:52 smithi107 ceph-mon[31381]: from='mgr.14120 172.21.15.107:0/283560839' entity='mgr.x' 2024-03-08T20:47:54.100 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:54 smithi107 ceph-mon[31381]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-03-08T20:47:54.116 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpSCnHfXT8N65+lAbF0tgR7lX8z7143TQ6tzGE7sZGK4PwBUbhtdrDAxlVOYdfjeZGKeP96nxBvR81AA5J+H2JHkdx/t4LTAZ7HOvSM22Khg1HZerSDFYbI1QxcO41edXWx2E4IyynZ6DxFCmHSptTTnIbNDFi5TcLWjh5nKBWqGsfjNXDZcxYrbdc9Iq3AtFj0zk9XLyzreBFSlq5ykKVuaE+XryB8qIkGYIW4jo5ApNP8VRi00cNBiPDzcQDbzyYmazSMDpEq0Eg1TZyGHI7y/iJxQ1IiP+s14mtDk5FpKgvbfKJvl1AQTYZEqEQKUieZ17g0Ykiijx8Z9P2DcKTwl1xKz7N2SAD0OmO6/f2TgtQQTTi3UZxyJc3om2DJSQfguoWau6itWFl4sWre3A6fWfXjaVLQGTIA1gbPTn/4QpvI39vFk+cLWEC0P7GlYiuxzYSpQPJbEs48ZvFzUvUugoMh4o8HoQAkZL5fqTOUxodpNaireU770p3YXGOyxk= ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715 2024-03-08T20:47:54.116 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:53.195+0000 7f8cd6200700 1 Processor -- start 2024-03-08T20:47:54.117 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:53.195+0000 7f8cd6200700 1 -- start start 2024-03-08T20:47:54.117 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:53.196+0000 7f8cd6200700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8cd0107780 0x7f8cd0107ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:47:54.117 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:53.196+0000 7f8cd6200700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cd0108170 con 0x7f8cd0107780 2024-03-08T20:47:54.117 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:53.196+0000 7f8ccf400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8cd0107780 0x7f8cd0107ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:47:54.117 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:53.196+0000 7f8ccf400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8cd0107780 0x7f8cd0107ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:39946/0 (socket says 172.21.15.107:39946) 2024-03-08T20:47:54.117 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:53.196+0000 7f8ccf400700 1 -- 172.21.15.107:0/4150660863 learned_addr learned my addr 172.21.15.107:0/4150660863 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T20:47:54.117 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:53.196+0000 7f8ccf400700 1 -- 172.21.15.107:0/4150660863 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cd0108930 con 0x7f8cd0107780 2024-03-08T20:47:54.117 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:53.197+0000 7f8ccf400700 1 --2- 172.21.15.107:0/4150660863 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8cd0107780 0x7f8cd0107ba0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f8cc00098a0 tx=0x7f8cc0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc147c8ad9aaf5e3 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:47:54.117 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:53.197+0000 7f8cce000700 1 -- 172.21.15.107:0/4150660863 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8cc0004160 con 0x7f8cd0107780 2024-03-08T20:47:54.118 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:53.197+0000 7f8cce000700 1 -- 172.21.15.107:0/4150660863 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1071+0+0 (secure 0 0 0) 0x7f8cc0004300 con 0x7f8cd0107780 2024-03-08T20:47:54.118 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:53.197+0000 7f8cce000700 1 -- 172.21.15.107:0/4150660863 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8cc0011a80 con 0x7f8cd0107780 2024-03-08T20:47:54.118 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:53.198+0000 7f8cd6200700 1 -- 172.21.15.107:0/4150660863 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8cd0107780 msgr2=0x7f8cd0107ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:47:54.118 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:53.198+0000 7f8cd6200700 1 --2- 172.21.15.107:0/4150660863 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8cd0107780 0x7f8cd0107ba0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f8cc00098a0 tx=0x7f8cc0009bb0 comp rx=0 tx=0).stop 2024-03-08T20:47:54.118 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:53.198+0000 7f8cd6200700 1 -- 172.21.15.107:0/4150660863 shutdown_connections 2024-03-08T20:47:54.118 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:53.198+0000 7f8cd6200700 1 --2- 172.21.15.107:0/4150660863 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8cd0107780 0x7f8cd0107ba0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:47:54.118 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:53.198+0000 7f8cd6200700 1 -- 172.21.15.107:0/4150660863 >> 172.21.15.107:0/4150660863 conn(0x7f8cd0102da0 msgr2=0x7f8cd01051c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:47:54.118 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:53.198+0000 7f8cd6200700 1 -- 172.21.15.107:0/4150660863 shutdown_connections 2024-03-08T20:47:54.119 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:53.198+0000 7f8cd6200700 1 -- 172.21.15.107:0/4150660863 wait complete. 2024-03-08T20:47:54.119 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:53.199+0000 7f8cd6200700 1 Processor -- start 2024-03-08T20:47:54.119 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:53.199+0000 7f8cd6200700 1 -- start start 2024-03-08T20:47:54.119 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:53.199+0000 7f8cd6200700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8cd0107780 0x7f8cd007cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:47:54.119 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:53.199+0000 7f8cd6200700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cd010bce0 con 0x7f8cd0107780 2024-03-08T20:47:54.119 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:53.200+0000 7f8ccf400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8cd0107780 0x7f8cd007cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:47:54.119 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:53.200+0000 7f8ccf400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8cd0107780 0x7f8cd007cb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:39956/0 (socket says 172.21.15.107:39956) 2024-03-08T20:47:54.120 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:53.200+0000 7f8ccf400700 1 -- 172.21.15.107:0/3574673561 learned_addr learned my addr 172.21.15.107:0/3574673561 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T20:47:54.120 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:53.200+0000 7f8ccf400700 1 -- 172.21.15.107:0/3574673561 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cd007d0d0 con 0x7f8cd0107780 2024-03-08T20:47:54.120 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:53.200+0000 7f8ccf400700 1 --2- 172.21.15.107:0/3574673561 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8cd0107780 0x7f8cd007cb90 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f8cc0000c00 tx=0x7f8cc0011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:47:54.120 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:53.201+0000 7f8cc7e00700 1 -- 172.21.15.107:0/3574673561 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8cc00039f0 con 0x7f8cd0107780 2024-03-08T20:47:54.120 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:53.201+0000 7f8cc7e00700 1 -- 172.21.15.107:0/3574673561 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1071+0+0 (secure 0 0 0) 0x7f8cc0003b90 con 0x7f8cd0107780 2024-03-08T20:47:54.120 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:53.201+0000 7f8cd6200700 1 -- 172.21.15.107:0/3574673561 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8cd007b2c0 con 0x7f8cd0107780 2024-03-08T20:47:54.120 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:53.201+0000 7f8cc7e00700 1 -- 172.21.15.107:0/3574673561 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8cc001a740 con 0x7f8cd0107780 2024-03-08T20:47:54.120 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:53.201+0000 7f8cd6200700 1 -- 172.21.15.107:0/3574673561 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8cd007b800 con 0x7f8cd0107780 2024-03-08T20:47:54.120 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:53.202+0000 7f8cc7e00700 1 -- 172.21.15.107:0/3574673561 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 8) v1 ==== 46106+0+0 (secure 0 0 0) 0x7f8cc0024af0 con 0x7f8cd0107780 2024-03-08T20:47:54.120 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:53.202+0000 7f8cd6200700 1 -- 172.21.15.107:0/3574673561 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8cd0063410 con 0x7f8cd0107780 2024-03-08T20:47:54.120 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:53.202+0000 7f8cc7e00700 1 --2- 172.21.15.107:0/3574673561 >> [v2:172.21.15.107:6800/426372446,v1:172.21.15.107:6801/426372446] conn(0x7f8cb4038f70 0x7f8cb403b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:47:54.121 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:53.202+0000 7f8cc7e00700 1 -- 172.21.15.107:0/3574673561 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f8cc004d230 con 0x7f8cd0107780 2024-03-08T20:47:54.121 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:53.203+0000 7f8ccea00700 1 --2- 172.21.15.107:0/3574673561 >> [v2:172.21.15.107:6800/426372446,v1:172.21.15.107:6801/426372446] conn(0x7f8cb4038f70 0x7f8cb403b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:47:54.121 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:53.203+0000 7f8ccea00700 1 --2- 172.21.15.107:0/3574673561 >> [v2:172.21.15.107:6800/426372446,v1:172.21.15.107:6801/426372446] conn(0x7f8cb4038f70 0x7f8cb403b430 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f8cbc006fd0 tx=0x7f8cbc006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:47:54.121 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:53.207+0000 7f8cc7e00700 1 -- 172.21.15.107:0/3574673561 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f8cc000e260 con 0x7f8cd0107780 2024-03-08T20:47:54.121 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:53.433+0000 7f8cd6200700 1 -- 172.21.15.107:0/3574673561 --> [v2:172.21.15.107:6800/426372446,v1:172.21.15.107:6801/426372446] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f8cd0103fe0 con 0x7f8cb4038f70 2024-03-08T20:47:54.121 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:53.435+0000 7f8cc7e00700 1 -- 172.21.15.107:0/3574673561 <== mgr.14120 v2:172.21.15.107:6800/426372446 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+595 (secure 0 0 0) 0x7f8cd0103fe0 con 0x7f8cb4038f70 2024-03-08T20:47:54.121 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:53.438+0000 7f8cd6200700 1 -- 172.21.15.107:0/3574673561 >> [v2:172.21.15.107:6800/426372446,v1:172.21.15.107:6801/426372446] conn(0x7f8cb4038f70 msgr2=0x7f8cb403b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:47:54.121 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:53.438+0000 7f8cd6200700 1 --2- 172.21.15.107:0/3574673561 >> [v2:172.21.15.107:6800/426372446,v1:172.21.15.107:6801/426372446] conn(0x7f8cb4038f70 0x7f8cb403b430 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f8cbc006fd0 tx=0x7f8cbc006e40 comp rx=0 tx=0).stop 2024-03-08T20:47:54.121 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:53.438+0000 7f8cd6200700 1 -- 172.21.15.107:0/3574673561 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8cd0107780 msgr2=0x7f8cd007cb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:47:54.121 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:53.438+0000 7f8cd6200700 1 --2- 172.21.15.107:0/3574673561 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8cd0107780 0x7f8cd007cb90 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f8cc0000c00 tx=0x7f8cc0011b00 comp rx=0 tx=0).stop 2024-03-08T20:47:54.122 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:53.438+0000 7f8cd6200700 1 -- 172.21.15.107:0/3574673561 shutdown_connections 2024-03-08T20:47:54.122 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:53.438+0000 7f8cd6200700 1 --2- 172.21.15.107:0/3574673561 >> [v2:172.21.15.107:6800/426372446,v1:172.21.15.107:6801/426372446] conn(0x7f8cb4038f70 0x7f8cb403b430 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:47:54.122 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:53.438+0000 7f8cd6200700 1 --2- 172.21.15.107:0/3574673561 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8cd0107780 0x7f8cd007cb90 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:47:54.122 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:53.438+0000 7f8cd6200700 1 -- 172.21.15.107:0/3574673561 >> 172.21.15.107:0/3574673561 conn(0x7f8cd0102da0 msgr2=0x7f8cd01038d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:47:54.122 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:53.439+0000 7f8cd6200700 1 -- 172.21.15.107:0/3574673561 shutdown_connections 2024-03-08T20:47:54.122 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:53.439+0000 7f8cd6200700 1 -- 172.21.15.107:0/3574673561 wait complete. 2024-03-08T20:47:54.122 INFO:teuthology.orchestra.run.smithi107.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-03-08T20:47:54.122 INFO:teuthology.orchestra.run.smithi107.stdout:Adding key to root@localhost authorized_keys... 2024-03-08T20:47:54.122 INFO:teuthology.orchestra.run.smithi107.stdout:Adding host smithi107... 2024-03-08T20:47:55.224 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:55 smithi107 ceph-mon[31381]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi107", "addr": "172.21.15.107", "target": ["mon-mgr", ""]}]: dispatch 2024-03-08T20:47:57.324 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:47:57 smithi107 ceph-mon[31381]: Deploying cephadm binary to smithi107 2024-03-08T20:48:01.871 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout Added host 'smithi107' with addr '172.21.15.107' 2024-03-08T20:48:01.871 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:54.703+0000 7fa025000700 1 Processor -- start 2024-03-08T20:48:01.871 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:54.704+0000 7fa025000700 1 -- start start 2024-03-08T20:48:01.871 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:54.704+0000 7fa025000700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa020107590 0x7fa0201079b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:48:01.871 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:54.704+0000 7fa025000700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa020107f80 con 0x7fa020107590 2024-03-08T20:48:01.871 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:54.704+0000 7fa01f400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa020107590 0x7fa0201079b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:48:01.872 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:54.705+0000 7fa01f400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa020107590 0x7fa0201079b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:39966/0 (socket says 172.21.15.107:39966) 2024-03-08T20:48:01.872 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:54.705+0000 7fa01f400700 1 -- 172.21.15.107:0/2779614801 learned_addr learned my addr 172.21.15.107:0/2779614801 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T20:48:01.872 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:54.705+0000 7fa01f400700 1 -- 172.21.15.107:0/2779614801 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa020108740 con 0x7fa020107590 2024-03-08T20:48:01.872 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:54.705+0000 7fa01f400700 1 --2- 172.21.15.107:0/2779614801 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa020107590 0x7fa0201079b0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fa0100098a0 tx=0x7fa010009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b373091bbd034c7 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:48:01.872 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:54.706+0000 7fa01e000700 1 -- 172.21.15.107:0/2779614801 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa010004160 con 0x7fa020107590 2024-03-08T20:48:01.872 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:54.706+0000 7fa01e000700 1 -- 172.21.15.107:0/2779614801 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1071+0+0 (secure 0 0 0) 0x7fa010004300 con 0x7fa020107590 2024-03-08T20:48:01.872 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:54.706+0000 7fa01e000700 1 -- 172.21.15.107:0/2779614801 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa010011a80 con 0x7fa020107590 2024-03-08T20:48:01.872 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:54.706+0000 7fa025000700 1 -- 172.21.15.107:0/2779614801 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa020107590 msgr2=0x7fa0201079b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:48:01.872 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:54.706+0000 7fa025000700 1 --2- 172.21.15.107:0/2779614801 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa020107590 0x7fa0201079b0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fa0100098a0 tx=0x7fa010009bb0 comp rx=0 tx=0).stop 2024-03-08T20:48:01.872 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:54.706+0000 7fa025000700 1 -- 172.21.15.107:0/2779614801 shutdown_connections 2024-03-08T20:48:01.872 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:54.706+0000 7fa025000700 1 --2- 172.21.15.107:0/2779614801 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa020107590 0x7fa0201079b0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:48:01.872 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:54.706+0000 7fa025000700 1 -- 172.21.15.107:0/2779614801 >> 172.21.15.107:0/2779614801 conn(0x7fa020102bb0 msgr2=0x7fa020104fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:48:01.872 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:54.707+0000 7fa025000700 1 -- 172.21.15.107:0/2779614801 shutdown_connections 2024-03-08T20:48:01.873 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:54.707+0000 7fa025000700 1 -- 172.21.15.107:0/2779614801 wait complete. 2024-03-08T20:48:01.873 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:54.708+0000 7fa025000700 1 Processor -- start 2024-03-08T20:48:01.873 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:54.708+0000 7fa025000700 1 -- start start 2024-03-08T20:48:01.873 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:54.708+0000 7fa025000700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa020107590 0x7fa02019b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:48:01.873 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:54.708+0000 7fa025000700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa02010baf0 con 0x7fa020107590 2024-03-08T20:48:01.873 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:54.708+0000 7fa01f400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa020107590 0x7fa02019b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:48:01.873 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:54.709+0000 7fa01f400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa020107590 0x7fa02019b490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:39976/0 (socket says 172.21.15.107:39976) 2024-03-08T20:48:01.873 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:54.709+0000 7fa01f400700 1 -- 172.21.15.107:0/1585501982 learned_addr learned my addr 172.21.15.107:0/1585501982 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T20:48:01.873 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:54.709+0000 7fa01f400700 1 -- 172.21.15.107:0/1585501982 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa02019b9d0 con 0x7fa020107590 2024-03-08T20:48:01.873 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:54.710+0000 7fa01f400700 1 --2- 172.21.15.107:0/1585501982 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa020107590 0x7fa02019b490 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fa010000c00 tx=0x7fa010011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:48:01.873 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:54.710+0000 7fa017e00700 1 -- 172.21.15.107:0/1585501982 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa0100039f0 con 0x7fa020107590 2024-03-08T20:48:01.873 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:54.710+0000 7fa025000700 1 -- 172.21.15.107:0/1585501982 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa02019bc60 con 0x7fa020107590 2024-03-08T20:48:01.873 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:54.710+0000 7fa017e00700 1 -- 172.21.15.107:0/1585501982 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1071+0+0 (secure 0 0 0) 0x7fa010003b90 con 0x7fa020107590 2024-03-08T20:48:01.873 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:54.710+0000 7fa017e00700 1 -- 172.21.15.107:0/1585501982 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa01001a740 con 0x7fa020107590 2024-03-08T20:48:01.874 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:54.710+0000 7fa025000700 1 -- 172.21.15.107:0/1585501982 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa02019e950 con 0x7fa020107590 2024-03-08T20:48:01.874 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:54.711+0000 7fa025000700 1 -- 172.21.15.107:0/1585501982 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa020063410 con 0x7fa020107590 2024-03-08T20:48:01.874 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:54.712+0000 7fa017e00700 1 -- 172.21.15.107:0/1585501982 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 8) v1 ==== 46106+0+0 (secure 0 0 0) 0x7fa010024af0 con 0x7fa020107590 2024-03-08T20:48:01.874 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:54.712+0000 7fa017e00700 1 --2- 172.21.15.107:0/1585501982 >> [v2:172.21.15.107:6800/426372446,v1:172.21.15.107:6801/426372446] conn(0x7fa004038fd0 0x7fa00403b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:48:01.874 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:54.712+0000 7fa017e00700 1 -- 172.21.15.107:0/1585501982 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fa01004d230 con 0x7fa020107590 2024-03-08T20:48:01.874 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:54.713+0000 7fa01ea00700 1 --2- 172.21.15.107:0/1585501982 >> [v2:172.21.15.107:6800/426372446,v1:172.21.15.107:6801/426372446] conn(0x7fa004038fd0 0x7fa00403b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:48:01.874 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:54.714+0000 7fa01ea00700 1 --2- 172.21.15.107:0/1585501982 >> [v2:172.21.15.107:6800/426372446,v1:172.21.15.107:6801/426372446] conn(0x7fa004038fd0 0x7fa00403b490 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fa00c006fd0 tx=0x7fa00c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:48:01.874 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:54.718+0000 7fa017e00700 1 -- 172.21.15.107:0/1585501982 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7fa01000e260 con 0x7fa020107590 2024-03-08T20:48:01.874 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:47:54.943+0000 7fa025000700 1 -- 172.21.15.107:0/1585501982 --> [v2:172.21.15.107:6800/426372446,v1:172.21.15.107:6801/426372446] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi107", "addr": "172.21.15.107", "target": ["mon-mgr", ""]}) v1 -- 0x7fa020103d90 con 0x7fa004038fd0 2024-03-08T20:48:01.874 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:01.174+0000 7fa017e00700 1 -- 172.21.15.107:0/1585501982 <== mgr.14120 v2:172.21.15.107:6800/426372446 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7fa020103d90 con 0x7fa004038fd0 2024-03-08T20:48:01.874 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:01.178+0000 7fa025000700 1 -- 172.21.15.107:0/1585501982 >> [v2:172.21.15.107:6800/426372446,v1:172.21.15.107:6801/426372446] conn(0x7fa004038fd0 msgr2=0x7fa00403b490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:48:01.874 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:01.178+0000 7fa025000700 1 --2- 172.21.15.107:0/1585501982 >> [v2:172.21.15.107:6800/426372446,v1:172.21.15.107:6801/426372446] conn(0x7fa004038fd0 0x7fa00403b490 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fa00c006fd0 tx=0x7fa00c006e40 comp rx=0 tx=0).stop 2024-03-08T20:48:01.874 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:01.178+0000 7fa025000700 1 -- 172.21.15.107:0/1585501982 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa020107590 msgr2=0x7fa02019b490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:48:01.874 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:01.178+0000 7fa025000700 1 --2- 172.21.15.107:0/1585501982 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa020107590 0x7fa02019b490 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fa010000c00 tx=0x7fa010011b00 comp rx=0 tx=0).stop 2024-03-08T20:48:01.875 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:01.178+0000 7fa025000700 1 -- 172.21.15.107:0/1585501982 shutdown_connections 2024-03-08T20:48:01.875 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:01.178+0000 7fa025000700 1 --2- 172.21.15.107:0/1585501982 >> [v2:172.21.15.107:6800/426372446,v1:172.21.15.107:6801/426372446] conn(0x7fa004038fd0 0x7fa00403b490 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:48:01.875 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:01.178+0000 7fa025000700 1 --2- 172.21.15.107:0/1585501982 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa020107590 0x7fa02019b490 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:48:01.875 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:01.178+0000 7fa025000700 1 -- 172.21.15.107:0/1585501982 >> 172.21.15.107:0/1585501982 conn(0x7fa020102bb0 msgr2=0x7fa020103680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:48:01.875 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:01.179+0000 7fa025000700 1 -- 172.21.15.107:0/1585501982 shutdown_connections 2024-03-08T20:48:01.875 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:01.179+0000 7fa025000700 1 -- 172.21.15.107:0/1585501982 wait complete. 2024-03-08T20:48:01.875 INFO:teuthology.orchestra.run.smithi107.stdout:Deploying unmanaged mon service... 2024-03-08T20:48:02.455 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:48:02 smithi107 ceph-mon[31381]: from='mgr.14120 172.21.15.107:0/283560839' entity='mgr.x' 2024-03-08T20:48:02.455 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:48:02 smithi107 ceph-mon[31381]: Added host smithi107 2024-03-08T20:48:02.455 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:48:02 smithi107 ceph-mon[31381]: from='mgr.14120 172.21.15.107:0/283560839' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T20:48:03.569 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-03-08T20:48:03.569 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:02.461+0000 7f67f6e00700 1 Processor -- start 2024-03-08T20:48:03.570 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:02.461+0000 7f67f6e00700 1 -- start start 2024-03-08T20:48:03.570 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:02.461+0000 7f67f6e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f67f01081e0 0x7f67f010a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:48:03.570 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:02.461+0000 7f67f6e00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67f007aed0 con 0x7f67f01081e0 2024-03-08T20:48:03.570 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:02.462+0000 7f67efe00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f67f01081e0 0x7f67f010a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:48:03.570 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:02.462+0000 7f67efe00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f67f01081e0 0x7f67f010a5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:33402/0 (socket says 172.21.15.107:33402) 2024-03-08T20:48:03.570 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:02.462+0000 7f67efe00700 1 -- 172.21.15.107:0/2863636681 learned_addr learned my addr 172.21.15.107:0/2863636681 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T20:48:03.570 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:02.463+0000 7f67efe00700 1 -- 172.21.15.107:0/2863636681 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67f010ab10 con 0x7f67f01081e0 2024-03-08T20:48:03.570 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:02.463+0000 7f67efe00700 1 --2- 172.21.15.107:0/2863636681 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f67f01081e0 0x7f67f010a5d0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f67e00098a0 tx=0x7f67e0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dafc3b6c5f5523fa server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:48:03.570 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:02.463+0000 7f67ef400700 1 -- 172.21.15.107:0/2863636681 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f67e0004160 con 0x7f67f01081e0 2024-03-08T20:48:03.570 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:02.463+0000 7f67ef400700 1 -- 172.21.15.107:0/2863636681 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1071+0+0 (secure 0 0 0) 0x7f67e0004300 con 0x7f67f01081e0 2024-03-08T20:48:03.570 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:02.463+0000 7f67ef400700 1 -- 172.21.15.107:0/2863636681 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f67e0011a80 con 0x7f67f01081e0 2024-03-08T20:48:03.570 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:02.464+0000 7f67f6e00700 1 -- 172.21.15.107:0/2863636681 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f67f01081e0 msgr2=0x7f67f010a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:48:03.570 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:02.464+0000 7f67f6e00700 1 --2- 172.21.15.107:0/2863636681 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f67f01081e0 0x7f67f010a5d0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f67e00098a0 tx=0x7f67e0009bb0 comp rx=0 tx=0).stop 2024-03-08T20:48:03.571 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:02.464+0000 7f67f6e00700 1 -- 172.21.15.107:0/2863636681 shutdown_connections 2024-03-08T20:48:03.571 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:02.464+0000 7f67f6e00700 1 --2- 172.21.15.107:0/2863636681 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f67f01081e0 0x7f67f010a5d0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:48:03.571 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:02.464+0000 7f67f6e00700 1 -- 172.21.15.107:0/2863636681 >> 172.21.15.107:0/2863636681 conn(0x7f67f0101c90 msgr2=0x7f67f01040b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:48:03.571 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:02.464+0000 7f67f6e00700 1 -- 172.21.15.107:0/2863636681 shutdown_connections 2024-03-08T20:48:03.571 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:02.464+0000 7f67f6e00700 1 -- 172.21.15.107:0/2863636681 wait complete. 2024-03-08T20:48:03.571 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:02.465+0000 7f67f6e00700 1 Processor -- start 2024-03-08T20:48:03.571 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:02.466+0000 7f67f6e00700 1 -- start start 2024-03-08T20:48:03.571 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:02.466+0000 7f67f6e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f67f01081e0 0x7f67f019d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:48:03.571 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:02.466+0000 7f67f6e00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67f010bae0 con 0x7f67f01081e0 2024-03-08T20:48:03.571 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:02.466+0000 7f67efe00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f67f01081e0 0x7f67f019d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:48:03.571 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:02.466+0000 7f67efe00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f67f01081e0 0x7f67f019d8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:33410/0 (socket says 172.21.15.107:33410) 2024-03-08T20:48:03.571 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:02.466+0000 7f67efe00700 1 -- 172.21.15.107:0/345424738 learned_addr learned my addr 172.21.15.107:0/345424738 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T20:48:03.571 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:02.467+0000 7f67efe00700 1 -- 172.21.15.107:0/345424738 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67f019ddf0 con 0x7f67f01081e0 2024-03-08T20:48:03.572 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:02.467+0000 7f67efe00700 1 --2- 172.21.15.107:0/345424738 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f67f01081e0 0x7f67f019d8b0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f67e0011a50 tx=0x7f67e0011b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:48:03.572 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:02.467+0000 7f67ed600700 1 -- 172.21.15.107:0/345424738 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f67e0003a80 con 0x7f67f01081e0 2024-03-08T20:48:03.572 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:02.467+0000 7f67ed600700 1 -- 172.21.15.107:0/345424738 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1071+0+0 (secure 0 0 0) 0x7f67e0003c20 con 0x7f67f01081e0 2024-03-08T20:48:03.574 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:02.467+0000 7f67ed600700 1 -- 172.21.15.107:0/345424738 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f67e001a650 con 0x7f67f01081e0 2024-03-08T20:48:03.574 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:02.467+0000 7f67f6e00700 1 -- 172.21.15.107:0/345424738 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67f019e080 con 0x7f67f01081e0 2024-03-08T20:48:03.574 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:02.467+0000 7f67f6e00700 1 -- 172.21.15.107:0/345424738 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67f01a0d70 con 0x7f67f01081e0 2024-03-08T20:48:03.574 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:02.468+0000 7f67ed600700 1 -- 172.21.15.107:0/345424738 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 8) v1 ==== 46106+0+0 (secure 0 0 0) 0x7f67e0024ad0 con 0x7f67f01081e0 2024-03-08T20:48:03.575 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:02.468+0000 7f67f6e00700 1 -- 172.21.15.107:0/345424738 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67f0063410 con 0x7f67f01081e0 2024-03-08T20:48:03.575 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:02.469+0000 7f67ed600700 1 --2- 172.21.15.107:0/345424738 >> [v2:172.21.15.107:6800/426372446,v1:172.21.15.107:6801/426372446] conn(0x7f67d4038f20 0x7f67d403b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:48:03.575 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:02.469+0000 7f67ed600700 1 -- 172.21.15.107:0/345424738 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f67e004d480 con 0x7f67f01081e0 2024-03-08T20:48:03.575 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:02.469+0000 7f67e7400700 1 --2- 172.21.15.107:0/345424738 >> [v2:172.21.15.107:6800/426372446,v1:172.21.15.107:6801/426372446] conn(0x7f67d4038f20 0x7f67d403b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:48:03.575 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:02.470+0000 7f67e7400700 1 --2- 172.21.15.107:0/345424738 >> [v2:172.21.15.107:6800/426372446,v1:172.21.15.107:6801/426372446] conn(0x7f67d4038f20 0x7f67d403b3e0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f67d8006fd0 tx=0x7f67d8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:48:03.575 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:02.474+0000 7f67ed600700 1 -- 172.21.15.107:0/345424738 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f67e00244f0 con 0x7f67f01081e0 2024-03-08T20:48:03.575 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:02.709+0000 7f67f6e00700 1 -- 172.21.15.107:0/345424738 --> [v2:172.21.15.107:6800/426372446,v1:172.21.15.107:6801/426372446] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f67f0102e70 con 0x7f67d4038f20 2024-03-08T20:48:03.575 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:02.721+0000 7f67ed600700 1 -- 172.21.15.107:0/345424738 <== mgr.14120 v2:172.21.15.107:6800/426372446 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f67f0102e70 con 0x7f67d4038f20 2024-03-08T20:48:03.575 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:02.724+0000 7f67f6e00700 1 -- 172.21.15.107:0/345424738 >> [v2:172.21.15.107:6800/426372446,v1:172.21.15.107:6801/426372446] conn(0x7f67d4038f20 msgr2=0x7f67d403b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:48:03.575 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:02.724+0000 7f67f6e00700 1 --2- 172.21.15.107:0/345424738 >> [v2:172.21.15.107:6800/426372446,v1:172.21.15.107:6801/426372446] conn(0x7f67d4038f20 0x7f67d403b3e0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f67d8006fd0 tx=0x7f67d8006e40 comp rx=0 tx=0).stop 2024-03-08T20:48:03.576 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:02.724+0000 7f67f6e00700 1 -- 172.21.15.107:0/345424738 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f67f01081e0 msgr2=0x7f67f019d8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:48:03.576 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:02.724+0000 7f67f6e00700 1 --2- 172.21.15.107:0/345424738 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f67f01081e0 0x7f67f019d8b0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f67e0011a50 tx=0x7f67e0011b30 comp rx=0 tx=0).stop 2024-03-08T20:48:03.576 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:02.724+0000 7f67f6e00700 1 -- 172.21.15.107:0/345424738 shutdown_connections 2024-03-08T20:48:03.576 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:02.724+0000 7f67f6e00700 1 --2- 172.21.15.107:0/345424738 >> [v2:172.21.15.107:6800/426372446,v1:172.21.15.107:6801/426372446] conn(0x7f67d4038f20 0x7f67d403b3e0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:48:03.576 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:02.724+0000 7f67f6e00700 1 --2- 172.21.15.107:0/345424738 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f67f01081e0 0x7f67f019d8b0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:48:03.576 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:02.724+0000 7f67f6e00700 1 -- 172.21.15.107:0/345424738 >> 172.21.15.107:0/345424738 conn(0x7f67f0101c90 msgr2=0x7f67f0102760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:48:03.576 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:02.724+0000 7f67f6e00700 1 -- 172.21.15.107:0/345424738 shutdown_connections 2024-03-08T20:48:03.576 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:02.724+0000 7f67f6e00700 1 -- 172.21.15.107:0/345424738 wait complete. 2024-03-08T20:48:03.576 INFO:teuthology.orchestra.run.smithi107.stdout:Deploying unmanaged mgr service... 2024-03-08T20:48:03.827 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:48:03 smithi107 ceph-mon[31381]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-08T20:48:03.827 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:48:03 smithi107 ceph-mon[31381]: Saving service mon spec with placement count:5 2024-03-08T20:48:03.827 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:48:03 smithi107 ceph-mon[31381]: from='mgr.14120 172.21.15.107:0/283560839' entity='mgr.x' 2024-03-08T20:48:06.091 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-03-08T20:48:06.091 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:05.169+0000 7fca60a00700 1 Processor -- start 2024-03-08T20:48:06.091 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:05.169+0000 7fca60a00700 1 -- start start 2024-03-08T20:48:06.091 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:05.170+0000 7fca60a00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fca5c1064a0 0x7fca5c1068c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:48:06.092 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:05.170+0000 7fca60a00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca5c106e90 con 0x7fca5c1064a0 2024-03-08T20:48:06.092 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:05.170+0000 7fca5b400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fca5c1064a0 0x7fca5c1068c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:48:06.092 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:05.170+0000 7fca5b400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fca5c1064a0 0x7fca5c1068c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:33422/0 (socket says 172.21.15.107:33422) 2024-03-08T20:48:06.092 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:05.170+0000 7fca5b400700 1 -- 172.21.15.107:0/2275101469 learned_addr learned my addr 172.21.15.107:0/2275101469 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T20:48:06.092 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:05.171+0000 7fca5b400700 1 -- 172.21.15.107:0/2275101469 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca5c1076a0 con 0x7fca5c1064a0 2024-03-08T20:48:06.092 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:05.171+0000 7fca5b400700 1 --2- 172.21.15.107:0/2275101469 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fca5c1064a0 0x7fca5c1068c0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fca4c0098a0 tx=0x7fca4c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebd02bdbb0a1c3f8 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:48:06.092 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:05.171+0000 7fca5a000700 1 -- 172.21.15.107:0/2275101469 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fca4c004160 con 0x7fca5c1064a0 2024-03-08T20:48:06.092 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:05.171+0000 7fca5a000700 1 -- 172.21.15.107:0/2275101469 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fca4c004300 con 0x7fca5c1064a0 2024-03-08T20:48:06.092 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:05.171+0000 7fca5a000700 1 -- 172.21.15.107:0/2275101469 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fca4c011a50 con 0x7fca5c1064a0 2024-03-08T20:48:06.092 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:05.172+0000 7fca60a00700 1 -- 172.21.15.107:0/2275101469 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fca5c1064a0 msgr2=0x7fca5c1068c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:48:06.092 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:05.172+0000 7fca60a00700 1 --2- 172.21.15.107:0/2275101469 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fca5c1064a0 0x7fca5c1068c0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fca4c0098a0 tx=0x7fca4c009bb0 comp rx=0 tx=0).stop 2024-03-08T20:48:06.092 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:05.172+0000 7fca60a00700 1 -- 172.21.15.107:0/2275101469 shutdown_connections 2024-03-08T20:48:06.092 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:05.172+0000 7fca60a00700 1 --2- 172.21.15.107:0/2275101469 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fca5c1064a0 0x7fca5c1068c0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:48:06.093 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:05.172+0000 7fca60a00700 1 -- 172.21.15.107:0/2275101469 >> 172.21.15.107:0/2275101469 conn(0x7fca5c101a60 msgr2=0x7fca5c103e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:48:06.093 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:05.173+0000 7fca60a00700 1 -- 172.21.15.107:0/2275101469 shutdown_connections 2024-03-08T20:48:06.093 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:05.173+0000 7fca60a00700 1 -- 172.21.15.107:0/2275101469 wait complete. 2024-03-08T20:48:06.093 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:05.174+0000 7fca60a00700 1 Processor -- start 2024-03-08T20:48:06.093 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:05.174+0000 7fca60a00700 1 -- start start 2024-03-08T20:48:06.093 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:05.174+0000 7fca60a00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fca5c1064a0 0x7fca5c19b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:48:06.093 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:05.174+0000 7fca60a00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca5c10aa50 con 0x7fca5c1064a0 2024-03-08T20:48:06.093 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:05.175+0000 7fca5b400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fca5c1064a0 0x7fca5c19b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:48:06.093 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:05.175+0000 7fca5b400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fca5c1064a0 0x7fca5c19b4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:33436/0 (socket says 172.21.15.107:33436) 2024-03-08T20:48:06.093 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:05.175+0000 7fca5b400700 1 -- 172.21.15.107:0/2097031678 learned_addr learned my addr 172.21.15.107:0/2097031678 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T20:48:06.093 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:05.175+0000 7fca5b400700 1 -- 172.21.15.107:0/2097031678 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca5c19b9e0 con 0x7fca5c1064a0 2024-03-08T20:48:06.093 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:05.175+0000 7fca5b400700 1 --2- 172.21.15.107:0/2097031678 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fca5c1064a0 0x7fca5c19b4a0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fca4c006ba0 tx=0x7fca4c011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:48:06.094 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:05.176+0000 7fca53e00700 1 -- 172.21.15.107:0/2097031678 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fca4c003af0 con 0x7fca5c1064a0 2024-03-08T20:48:06.094 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:05.176+0000 7fca53e00700 1 -- 172.21.15.107:0/2097031678 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fca4c01a8e0 con 0x7fca5c1064a0 2024-03-08T20:48:06.094 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:05.176+0000 7fca60a00700 1 -- 172.21.15.107:0/2097031678 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca5c19bc70 con 0x7fca5c1064a0 2024-03-08T20:48:06.094 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:05.176+0000 7fca53e00700 1 -- 172.21.15.107:0/2097031678 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fca4c024900 con 0x7fca5c1064a0 2024-03-08T20:48:06.094 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:05.176+0000 7fca60a00700 1 -- 172.21.15.107:0/2097031678 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca5c19e960 con 0x7fca5c1064a0 2024-03-08T20:48:06.094 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:05.177+0000 7fca53e00700 1 -- 172.21.15.107:0/2097031678 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 8) v1 ==== 46106+0+0 (secure 0 0 0) 0x7fca4c022020 con 0x7fca5c1064a0 2024-03-08T20:48:06.094 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:05.177+0000 7fca60a00700 1 -- 172.21.15.107:0/2097031678 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca5c063410 con 0x7fca5c1064a0 2024-03-08T20:48:06.094 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:05.178+0000 7fca53e00700 1 --2- 172.21.15.107:0/2097031678 >> [v2:172.21.15.107:6800/426372446,v1:172.21.15.107:6801/426372446] conn(0x7fca40038f20 0x7fca4003b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:48:06.094 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:05.178+0000 7fca53e00700 1 -- 172.21.15.107:0/2097031678 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fca4c04d220 con 0x7fca5c1064a0 2024-03-08T20:48:06.094 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:05.178+0000 7fca5aa00700 1 --2- 172.21.15.107:0/2097031678 >> [v2:172.21.15.107:6800/426372446,v1:172.21.15.107:6801/426372446] conn(0x7fca40038f20 0x7fca4003b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:48:06.094 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:05.179+0000 7fca5aa00700 1 --2- 172.21.15.107:0/2097031678 >> [v2:172.21.15.107:6800/426372446,v1:172.21.15.107:6801/426372446] conn(0x7fca40038f20 0x7fca4003b3e0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fca48006fd0 tx=0x7fca48006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:48:06.095 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:05.184+0000 7fca53e00700 1 -- 172.21.15.107:0/2097031678 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7fca4c02ab00 con 0x7fca5c1064a0 2024-03-08T20:48:06.095 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:05.423+0000 7fca60a00700 1 -- 172.21.15.107:0/2097031678 --> [v2:172.21.15.107:6800/426372446,v1:172.21.15.107:6801/426372446] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7fca5c02da80 con 0x7fca40038f20 2024-03-08T20:48:06.095 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:05.432+0000 7fca53e00700 1 -- 172.21.15.107:0/2097031678 <== mgr.14120 v2:172.21.15.107:6800/426372446 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7fca5c02da80 con 0x7fca40038f20 2024-03-08T20:48:06.095 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:05.435+0000 7fca60a00700 1 -- 172.21.15.107:0/2097031678 >> [v2:172.21.15.107:6800/426372446,v1:172.21.15.107:6801/426372446] conn(0x7fca40038f20 msgr2=0x7fca4003b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:48:06.095 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:05.435+0000 7fca60a00700 1 --2- 172.21.15.107:0/2097031678 >> [v2:172.21.15.107:6800/426372446,v1:172.21.15.107:6801/426372446] conn(0x7fca40038f20 0x7fca4003b3e0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fca48006fd0 tx=0x7fca48006e40 comp rx=0 tx=0).stop 2024-03-08T20:48:06.095 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:05.435+0000 7fca60a00700 1 -- 172.21.15.107:0/2097031678 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fca5c1064a0 msgr2=0x7fca5c19b4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:48:06.095 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:05.435+0000 7fca60a00700 1 --2- 172.21.15.107:0/2097031678 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fca5c1064a0 0x7fca5c19b4a0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fca4c006ba0 tx=0x7fca4c011ad0 comp rx=0 tx=0).stop 2024-03-08T20:48:06.095 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:05.435+0000 7fca60a00700 1 -- 172.21.15.107:0/2097031678 shutdown_connections 2024-03-08T20:48:06.095 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:05.435+0000 7fca60a00700 1 --2- 172.21.15.107:0/2097031678 >> [v2:172.21.15.107:6800/426372446,v1:172.21.15.107:6801/426372446] conn(0x7fca40038f20 0x7fca4003b3e0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:48:06.095 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:05.435+0000 7fca60a00700 1 --2- 172.21.15.107:0/2097031678 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fca5c1064a0 0x7fca5c19b4a0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:48:06.095 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:05.435+0000 7fca60a00700 1 -- 172.21.15.107:0/2097031678 >> 172.21.15.107:0/2097031678 conn(0x7fca5c101a60 msgr2=0x7fca5c102850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:48:06.095 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:05.435+0000 7fca60a00700 1 -- 172.21.15.107:0/2097031678 shutdown_connections 2024-03-08T20:48:06.095 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:05.435+0000 7fca60a00700 1 -- 172.21.15.107:0/2097031678 wait complete. 2024-03-08T20:48:06.357 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:48:06 smithi107 ceph-mon[31381]: from='mgr.14120 172.21.15.107:0/283560839' entity='mgr.x' 2024-03-08T20:48:06.357 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:48:06 smithi107 ceph-mon[31381]: from='mgr.14120 172.21.15.107:0/283560839' entity='mgr.x' 2024-03-08T20:48:06.357 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:48:06 smithi107 ceph-mon[31381]: from='mgr.14120 172.21.15.107:0/283560839' entity='mgr.x' 2024-03-08T20:48:07.163 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:48:07 smithi107 ceph-mon[31381]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-08T20:48:07.163 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:48:07 smithi107 ceph-mon[31381]: Saving service mgr spec with placement count:2 2024-03-08T20:48:07.163 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:48:07 smithi107 ceph-mon[31381]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T20:48:07.163 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:48:07 smithi107 ceph-mon[31381]: from='client.? 172.21.15.107:0/2698299597' entity='client.admin' 2024-03-08T20:48:08.210 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:48:08 smithi107 ceph-mon[31381]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T20:48:08.222 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:06.649+0000 7f28aec00700 1 Processor -- start 2024-03-08T20:48:08.222 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:06.649+0000 7f28aec00700 1 -- start start 2024-03-08T20:48:08.222 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:06.650+0000 7f28aec00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f28a807cf30 0x7f28a807b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:48:08.222 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:06.650+0000 7f28aec00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28a807b8b0 con 0x7f28a807cf30 2024-03-08T20:48:08.222 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:06.650+0000 7f28a7e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f28a807cf30 0x7f28a807b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:48:08.223 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:06.650+0000 7f28a7e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f28a807cf30 0x7f28a807b2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:59368/0 (socket says 172.21.15.107:59368) 2024-03-08T20:48:08.223 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:06.650+0000 7f28a7e00700 1 -- 172.21.15.107:0/3037510281 learned_addr learned my addr 172.21.15.107:0/3037510281 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T20:48:08.223 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:06.651+0000 7f28a7e00700 1 -- 172.21.15.107:0/3037510281 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28a807ba30 con 0x7f28a807cf30 2024-03-08T20:48:08.223 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:06.651+0000 7f28a7e00700 1 --2- 172.21.15.107:0/3037510281 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f28a807cf30 0x7f28a807b2e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f28980098a0 tx=0x7f2898009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1929d88bb0115e1 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:48:08.223 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:06.651+0000 7f28a6a00700 1 -- 172.21.15.107:0/3037510281 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2898004160 con 0x7f28a807cf30 2024-03-08T20:48:08.223 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:06.651+0000 7f28a6a00700 1 -- 172.21.15.107:0/3037510281 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f2898004300 con 0x7f28a807cf30 2024-03-08T20:48:08.223 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:06.651+0000 7f28a6a00700 1 -- 172.21.15.107:0/3037510281 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2898011a50 con 0x7f28a807cf30 2024-03-08T20:48:08.223 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:06.652+0000 7f28aec00700 1 -- 172.21.15.107:0/3037510281 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f28a807cf30 msgr2=0x7f28a807b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:48:08.223 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:06.652+0000 7f28aec00700 1 --2- 172.21.15.107:0/3037510281 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f28a807cf30 0x7f28a807b2e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f28980098a0 tx=0x7f2898009bb0 comp rx=0 tx=0).stop 2024-03-08T20:48:08.223 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:06.652+0000 7f28aec00700 1 -- 172.21.15.107:0/3037510281 shutdown_connections 2024-03-08T20:48:08.223 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:06.652+0000 7f28aec00700 1 --2- 172.21.15.107:0/3037510281 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f28a807cf30 0x7f28a807b2e0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:48:08.223 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:06.652+0000 7f28aec00700 1 -- 172.21.15.107:0/3037510281 >> 172.21.15.107:0/3037510281 conn(0x7f28a8102da0 msgr2=0x7f28a81051c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:48:08.224 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:06.653+0000 7f28aec00700 1 -- 172.21.15.107:0/3037510281 shutdown_connections 2024-03-08T20:48:08.224 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:06.653+0000 7f28aec00700 1 -- 172.21.15.107:0/3037510281 wait complete. 2024-03-08T20:48:08.224 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:06.654+0000 7f28aec00700 1 Processor -- start 2024-03-08T20:48:08.224 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:06.654+0000 7f28aec00700 1 -- start start 2024-03-08T20:48:08.224 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:06.655+0000 7f28aec00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f28a807cf30 0x7f28a819d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:48:08.224 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:06.655+0000 7f28aec00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28a810de70 con 0x7f28a807cf30 2024-03-08T20:48:08.224 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:06.655+0000 7f28a7e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f28a807cf30 0x7f28a819d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:48:08.224 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:06.655+0000 7f28a7e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f28a807cf30 0x7f28a819d960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:59372/0 (socket says 172.21.15.107:59372) 2024-03-08T20:48:08.224 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:06.655+0000 7f28a7e00700 1 -- 172.21.15.107:0/2698299597 learned_addr learned my addr 172.21.15.107:0/2698299597 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T20:48:08.224 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:06.655+0000 7f28a7e00700 1 -- 172.21.15.107:0/2698299597 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28a819dea0 con 0x7f28a807cf30 2024-03-08T20:48:08.224 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:06.656+0000 7f28a7e00700 1 --2- 172.21.15.107:0/2698299597 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f28a807cf30 0x7f28a819d960 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f2898011a00 tx=0x7f2898011ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:48:08.224 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:06.656+0000 7f28a4c00700 1 -- 172.21.15.107:0/2698299597 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2898003af0 con 0x7f28a807cf30 2024-03-08T20:48:08.224 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:06.656+0000 7f28a4c00700 1 -- 172.21.15.107:0/2698299597 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f289801a8e0 con 0x7f28a807cf30 2024-03-08T20:48:08.224 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:06.656+0000 7f28aec00700 1 -- 172.21.15.107:0/2698299597 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28a819e130 con 0x7f28a807cf30 2024-03-08T20:48:08.225 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:06.656+0000 7f28aec00700 1 -- 172.21.15.107:0/2698299597 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28a81a0e20 con 0x7f28a807cf30 2024-03-08T20:48:08.225 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:06.656+0000 7f28a4c00700 1 -- 172.21.15.107:0/2698299597 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2898024840 con 0x7f28a807cf30 2024-03-08T20:48:08.225 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:06.657+0000 7f28a4c00700 1 -- 172.21.15.107:0/2698299597 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 8) v1 ==== 46106+0+0 (secure 0 0 0) 0x7f2898022020 con 0x7f28a807cf30 2024-03-08T20:48:08.225 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:06.657+0000 7f289e000700 1 -- 172.21.15.107:0/2698299597 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2880005330 con 0x7f28a807cf30 2024-03-08T20:48:08.225 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:06.658+0000 7f28a4c00700 1 --2- 172.21.15.107:0/2698299597 >> [v2:172.21.15.107:6800/426372446,v1:172.21.15.107:6801/426372446] conn(0x7f288c038f70 0x7f288c03b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:48:08.225 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:06.658+0000 7f28a4c00700 1 -- 172.21.15.107:0/2698299597 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f289804e430 con 0x7f28a807cf30 2024-03-08T20:48:08.225 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:06.658+0000 7f28a7400700 1 --2- 172.21.15.107:0/2698299597 >> [v2:172.21.15.107:6800/426372446,v1:172.21.15.107:6801/426372446] conn(0x7f288c038f70 0x7f288c03b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:48:08.225 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:06.659+0000 7f28a7400700 1 --2- 172.21.15.107:0/2698299597 >> [v2:172.21.15.107:6800/426372446,v1:172.21.15.107:6801/426372446] conn(0x7f288c038f70 0x7f288c03b430 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f2894006fd0 tx=0x7f2894006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:48:08.225 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:06.664+0000 7f28a4c00700 1 -- 172.21.15.107:0/2698299597 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f289801adb0 con 0x7f28a807cf30 2024-03-08T20:48:08.225 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:06.894+0000 7f289e000700 1 -- 172.21.15.107:0/2698299597 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) v1 -- 0x7f2880005620 con 0x7f28a807cf30 2024-03-08T20:48:08.225 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:06.897+0000 7f28a4c00700 1 -- 172.21.15.107:0/2698299597 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) v1 ==== 142+0+0 (secure 0 0 0) 0x7f289804dd80 con 0x7f28a807cf30 2024-03-08T20:48:08.225 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:06.900+0000 7f289e000700 1 -- 172.21.15.107:0/2698299597 >> [v2:172.21.15.107:6800/426372446,v1:172.21.15.107:6801/426372446] conn(0x7f288c038f70 msgr2=0x7f288c03b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:48:08.225 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:06.900+0000 7f289e000700 1 --2- 172.21.15.107:0/2698299597 >> [v2:172.21.15.107:6800/426372446,v1:172.21.15.107:6801/426372446] conn(0x7f288c038f70 0x7f288c03b430 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f2894006fd0 tx=0x7f2894006e40 comp rx=0 tx=0).stop 2024-03-08T20:48:08.225 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:06.900+0000 7f289e000700 1 -- 172.21.15.107:0/2698299597 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f28a807cf30 msgr2=0x7f28a819d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:48:08.226 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:06.900+0000 7f289e000700 1 --2- 172.21.15.107:0/2698299597 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f28a807cf30 0x7f28a819d960 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f2898011a00 tx=0x7f2898011ae0 comp rx=0 tx=0).stop 2024-03-08T20:48:08.226 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:06.900+0000 7f289e000700 1 -- 172.21.15.107:0/2698299597 shutdown_connections 2024-03-08T20:48:08.226 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:06.900+0000 7f289e000700 1 --2- 172.21.15.107:0/2698299597 >> [v2:172.21.15.107:6800/426372446,v1:172.21.15.107:6801/426372446] conn(0x7f288c038f70 0x7f288c03b430 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:48:08.226 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:06.900+0000 7f289e000700 1 --2- 172.21.15.107:0/2698299597 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f28a807cf30 0x7f28a819d960 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:48:08.226 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:06.900+0000 7f289e000700 1 -- 172.21.15.107:0/2698299597 >> 172.21.15.107:0/2698299597 conn(0x7f28a8102da0 msgr2=0x7f28a8103860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:48:08.226 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:06.901+0000 7f289e000700 1 -- 172.21.15.107:0/2698299597 shutdown_connections 2024-03-08T20:48:08.226 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:06.901+0000 7f289e000700 1 -- 172.21.15.107:0/2698299597 wait complete. 2024-03-08T20:48:09.372 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:48:09 smithi107 ceph-mon[31381]: from='mgr.14120 172.21.15.107:0/283560839' entity='mgr.x' 2024-03-08T20:48:09.372 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:48:09 smithi107 ceph-mon[31381]: from='client.? 172.21.15.107:0/1312836612' entity='client.admin' 2024-03-08T20:48:09.729 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:08.822+0000 7f7a92600700 1 Processor -- start 2024-03-08T20:48:09.729 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:08.822+0000 7f7a92600700 1 -- start start 2024-03-08T20:48:09.729 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:08.822+0000 7f7a92600700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7a8c107740 0x7f7a8c107b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:48:09.729 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:08.822+0000 7f7a92600700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a8c108130 con 0x7f7a8c107740 2024-03-08T20:48:09.730 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:08.823+0000 7f7a8b400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7a8c107740 0x7f7a8c107b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:48:09.730 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:08.823+0000 7f7a8b400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7a8c107740 0x7f7a8c107b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:59382/0 (socket says 172.21.15.107:59382) 2024-03-08T20:48:09.730 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:08.823+0000 7f7a8b400700 1 -- 172.21.15.107:0/877013443 learned_addr learned my addr 172.21.15.107:0/877013443 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T20:48:09.730 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:08.823+0000 7f7a8b400700 1 -- 172.21.15.107:0/877013443 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a8c108940 con 0x7f7a8c107740 2024-03-08T20:48:09.730 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:08.824+0000 7f7a8b400700 1 --2- 172.21.15.107:0/877013443 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7a8c107740 0x7f7a8c107b60 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f7a7c0098a0 tx=0x7f7a7c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e49c10a5f1b8370 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:48:09.730 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:08.824+0000 7f7a8a000700 1 -- 172.21.15.107:0/877013443 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7a7c004160 con 0x7f7a8c107740 2024-03-08T20:48:09.730 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:08.824+0000 7f7a8a000700 1 -- 172.21.15.107:0/877013443 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f7a7c004300 con 0x7f7a8c107740 2024-03-08T20:48:09.730 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:08.824+0000 7f7a8a000700 1 -- 172.21.15.107:0/877013443 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7a7c011a50 con 0x7f7a8c107740 2024-03-08T20:48:09.730 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:08.825+0000 7f7a92600700 1 -- 172.21.15.107:0/877013443 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7a8c107740 msgr2=0x7f7a8c107b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:48:09.730 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:08.825+0000 7f7a92600700 1 --2- 172.21.15.107:0/877013443 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7a8c107740 0x7f7a8c107b60 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f7a7c0098a0 tx=0x7f7a7c009bb0 comp rx=0 tx=0).stop 2024-03-08T20:48:09.730 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:08.825+0000 7f7a92600700 1 -- 172.21.15.107:0/877013443 shutdown_connections 2024-03-08T20:48:09.730 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:08.825+0000 7f7a92600700 1 --2- 172.21.15.107:0/877013443 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7a8c107740 0x7f7a8c107b60 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:48:09.731 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:08.825+0000 7f7a92600700 1 -- 172.21.15.107:0/877013443 >> 172.21.15.107:0/877013443 conn(0x7f7a8c102d80 msgr2=0x7f7a8c1051a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:48:09.731 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:08.825+0000 7f7a92600700 1 -- 172.21.15.107:0/877013443 shutdown_connections 2024-03-08T20:48:09.731 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:08.825+0000 7f7a92600700 1 -- 172.21.15.107:0/877013443 wait complete. 2024-03-08T20:48:09.731 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:08.826+0000 7f7a92600700 1 Processor -- start 2024-03-08T20:48:09.731 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:08.826+0000 7f7a92600700 1 -- start start 2024-03-08T20:48:09.731 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:08.827+0000 7f7a92600700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7a8c107740 0x7f7a8c07cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:48:09.731 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:08.827+0000 7f7a92600700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a8c10bcf0 con 0x7f7a8c107740 2024-03-08T20:48:09.731 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:08.827+0000 7f7a8b400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7a8c107740 0x7f7a8c07cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:48:09.731 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:08.827+0000 7f7a8b400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7a8c107740 0x7f7a8c07cb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:59396/0 (socket says 172.21.15.107:59396) 2024-03-08T20:48:09.731 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:08.827+0000 7f7a8b400700 1 -- 172.21.15.107:0/1312836612 learned_addr learned my addr 172.21.15.107:0/1312836612 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T20:48:09.731 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:08.827+0000 7f7a8b400700 1 -- 172.21.15.107:0/1312836612 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a8c07d0d0 con 0x7f7a8c107740 2024-03-08T20:48:09.731 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:08.828+0000 7f7a8b400700 1 --2- 172.21.15.107:0/1312836612 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7a8c107740 0x7f7a8c07cb90 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f7a7c011a20 tx=0x7f7a7c011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:48:09.731 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:08.828+0000 7f7a83e00700 1 -- 172.21.15.107:0/1312836612 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7a7c003af0 con 0x7f7a8c107740 2024-03-08T20:48:09.731 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:08.828+0000 7f7a83e00700 1 -- 172.21.15.107:0/1312836612 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f7a7c01a8e0 con 0x7f7a8c107740 2024-03-08T20:48:09.732 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:08.828+0000 7f7a83e00700 1 -- 172.21.15.107:0/1312836612 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7a7c024860 con 0x7f7a8c107740 2024-03-08T20:48:09.732 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:08.828+0000 7f7a92600700 1 -- 172.21.15.107:0/1312836612 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a8c07b2c0 con 0x7f7a8c107740 2024-03-08T20:48:09.732 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:08.828+0000 7f7a92600700 1 -- 172.21.15.107:0/1312836612 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a8c07b800 con 0x7f7a8c107740 2024-03-08T20:48:09.732 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:08.830+0000 7f7a83e00700 1 -- 172.21.15.107:0/1312836612 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 8) v1 ==== 46106+0+0 (secure 0 0 0) 0x7f7a7c022020 con 0x7f7a8c107740 2024-03-08T20:48:09.732 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:08.830+0000 7f7a92600700 1 -- 172.21.15.107:0/1312836612 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a8c063410 con 0x7f7a8c107740 2024-03-08T20:48:09.732 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:08.830+0000 7f7a83e00700 1 --2- 172.21.15.107:0/1312836612 >> [v2:172.21.15.107:6800/426372446,v1:172.21.15.107:6801/426372446] conn(0x7f7a70038f70 0x7f7a7003b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:48:09.732 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:08.830+0000 7f7a83e00700 1 -- 172.21.15.107:0/1312836612 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f7a7c04d170 con 0x7f7a8c107740 2024-03-08T20:48:09.732 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:08.830+0000 7f7a8aa00700 1 --2- 172.21.15.107:0/1312836612 >> [v2:172.21.15.107:6800/426372446,v1:172.21.15.107:6801/426372446] conn(0x7f7a70038f70 0x7f7a7003b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:48:09.732 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:08.831+0000 7f7a8aa00700 1 --2- 172.21.15.107:0/1312836612 >> [v2:172.21.15.107:6800/426372446,v1:172.21.15.107:6801/426372446] conn(0x7f7a70038f70 0x7f7a7003b430 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f7a78006fd0 tx=0x7f7a78006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:48:09.732 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:08.836+0000 7f7a83e00700 1 -- 172.21.15.107:0/1312836612 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f7a7c02c3a0 con 0x7f7a8c107740 2024-03-08T20:48:09.732 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:09.066+0000 7f7a92600700 1 -- 172.21.15.107:0/1312836612 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) v1 -- 0x7f7a8c07ba80 con 0x7f7a8c107740 2024-03-08T20:48:09.732 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:09.069+0000 7f7a83e00700 1 -- 172.21.15.107:0/1312836612 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) v1 ==== 130+0+0 (secure 0 0 0) 0x7f7a7c016260 con 0x7f7a8c107740 2024-03-08T20:48:09.732 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:09.072+0000 7f7a92600700 1 -- 172.21.15.107:0/1312836612 >> [v2:172.21.15.107:6800/426372446,v1:172.21.15.107:6801/426372446] conn(0x7f7a70038f70 msgr2=0x7f7a7003b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:48:09.733 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:09.072+0000 7f7a92600700 1 --2- 172.21.15.107:0/1312836612 >> [v2:172.21.15.107:6800/426372446,v1:172.21.15.107:6801/426372446] conn(0x7f7a70038f70 0x7f7a7003b430 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f7a78006fd0 tx=0x7f7a78006e40 comp rx=0 tx=0).stop 2024-03-08T20:48:09.733 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:09.072+0000 7f7a92600700 1 -- 172.21.15.107:0/1312836612 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7a8c107740 msgr2=0x7f7a8c07cb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:48:09.733 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:09.072+0000 7f7a92600700 1 --2- 172.21.15.107:0/1312836612 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7a8c107740 0x7f7a8c07cb90 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f7a7c011a20 tx=0x7f7a7c011b00 comp rx=0 tx=0).stop 2024-03-08T20:48:09.733 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:09.072+0000 7f7a92600700 1 -- 172.21.15.107:0/1312836612 shutdown_connections 2024-03-08T20:48:09.733 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:09.072+0000 7f7a92600700 1 --2- 172.21.15.107:0/1312836612 >> [v2:172.21.15.107:6800/426372446,v1:172.21.15.107:6801/426372446] conn(0x7f7a70038f70 0x7f7a7003b430 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:48:09.733 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:09.072+0000 7f7a92600700 1 --2- 172.21.15.107:0/1312836612 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7a8c107740 0x7f7a8c07cb90 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:48:09.733 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:09.072+0000 7f7a92600700 1 -- 172.21.15.107:0/1312836612 >> 172.21.15.107:0/1312836612 conn(0x7f7a8c102d80 msgr2=0x7f7a8c103830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:48:09.733 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:09.072+0000 7f7a92600700 1 -- 172.21.15.107:0/1312836612 shutdown_connections 2024-03-08T20:48:09.733 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:09.072+0000 7f7a92600700 1 -- 172.21.15.107:0/1312836612 wait complete. 2024-03-08T20:48:09.733 INFO:teuthology.orchestra.run.smithi107.stdout:Enabling the dashboard module... 2024-03-08T20:48:10.298 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:48:10 smithi107 ceph-mon[31381]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T20:48:10.298 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:48:10 smithi107 ceph-mon[31381]: from='mgr.14120 172.21.15.107:0/283560839' entity='mgr.x' 2024-03-08T20:48:11.368 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:48:11 smithi107 ceph-mon[31381]: from='client.? 172.21.15.107:0/3987073128' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-03-08T20:48:11.707 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 08 20:48:11 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-x[31606]: ignoring --setuser ceph since I am not root 2024-03-08T20:48:11.707 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 08 20:48:11 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-x[31606]: ignoring --setgroup ceph since I am not root 2024-03-08T20:48:12.478 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:48:12 smithi107 ceph-mon[31381]: from='client.? 172.21.15.107:0/3987073128' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-03-08T20:48:12.478 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:48:12 smithi107 ceph-mon[31381]: mgrmap e9: x(active, since 25s) 2024-03-08T20:48:12.478 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 08 20:48:12 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-x[31606]: 2024-03-08T20:48:12.078+0000 7f8cbad18200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-08T20:48:12.478 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 08 20:48:12 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-x[31606]: 2024-03-08T20:48:12.197+0000 7f8cbad18200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-08T20:48:12.901 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:10.304+0000 7f7e8a800700 1 Processor -- start 2024-03-08T20:48:12.903 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:10.304+0000 7f7e8a800700 1 -- start start 2024-03-08T20:48:12.903 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:10.305+0000 7f7e8a800700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7e84108e30 0x7f7e84109250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:48:12.903 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:10.305+0000 7f7e8a800700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e841073c0 con 0x7f7e84108e30 2024-03-08T20:48:12.903 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:10.305+0000 7f7e83400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7e84108e30 0x7f7e84109250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:48:12.903 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:10.305+0000 7f7e83400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7e84108e30 0x7f7e84109250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:59402/0 (socket says 172.21.15.107:59402) 2024-03-08T20:48:12.904 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:10.306+0000 7f7e83400700 1 -- 172.21.15.107:0/2256678868 learned_addr learned my addr 172.21.15.107:0/2256678868 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T20:48:12.904 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:10.306+0000 7f7e83400700 1 -- 172.21.15.107:0/2256678868 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e84107540 con 0x7f7e84108e30 2024-03-08T20:48:12.904 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:10.306+0000 7f7e83400700 1 --2- 172.21.15.107:0/2256678868 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7e84108e30 0x7f7e84109250 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f7e740098a0 tx=0x7f7e74009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2416e71e26e3d0fd server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:48:12.904 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:10.307+0000 7f7e82000700 1 -- 172.21.15.107:0/2256678868 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7e74004160 con 0x7f7e84108e30 2024-03-08T20:48:12.904 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:10.307+0000 7f7e82000700 1 -- 172.21.15.107:0/2256678868 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f7e74004300 con 0x7f7e84108e30 2024-03-08T20:48:12.904 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:10.307+0000 7f7e82000700 1 -- 172.21.15.107:0/2256678868 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7e74011a50 con 0x7f7e84108e30 2024-03-08T20:48:12.904 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:10.307+0000 7f7e8a800700 1 -- 172.21.15.107:0/2256678868 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7e84108e30 msgr2=0x7f7e84109250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:48:12.904 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:10.307+0000 7f7e8a800700 1 --2- 172.21.15.107:0/2256678868 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7e84108e30 0x7f7e84109250 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f7e740098a0 tx=0x7f7e74009bb0 comp rx=0 tx=0).stop 2024-03-08T20:48:12.904 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:10.307+0000 7f7e8a800700 1 -- 172.21.15.107:0/2256678868 shutdown_connections 2024-03-08T20:48:12.904 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:10.307+0000 7f7e8a800700 1 --2- 172.21.15.107:0/2256678868 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7e84108e30 0x7f7e84109250 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:48:12.909 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:10.308+0000 7f7e8a800700 1 -- 172.21.15.107:0/2256678868 >> 172.21.15.107:0/2256678868 conn(0x7f7e84102da0 msgr2=0x7f7e841051c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:48:12.909 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:10.308+0000 7f7e8a800700 1 -- 172.21.15.107:0/2256678868 shutdown_connections 2024-03-08T20:48:12.909 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:10.308+0000 7f7e8a800700 1 -- 172.21.15.107:0/2256678868 wait complete. 2024-03-08T20:48:12.909 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:10.309+0000 7f7e8a800700 1 Processor -- start 2024-03-08T20:48:12.909 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:10.309+0000 7f7e8a800700 1 -- start start 2024-03-08T20:48:12.909 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:10.309+0000 7f7e8a800700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7e84108e30 0x7f7e8419d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:48:12.909 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:10.309+0000 7f7e8a800700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e8410bd00 con 0x7f7e84108e30 2024-03-08T20:48:12.909 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:10.310+0000 7f7e83400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7e84108e30 0x7f7e8419d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:48:12.909 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:10.310+0000 7f7e83400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7e84108e30 0x7f7e8419d940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:59414/0 (socket says 172.21.15.107:59414) 2024-03-08T20:48:12.909 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:10.310+0000 7f7e83400700 1 -- 172.21.15.107:0/3987073128 learned_addr learned my addr 172.21.15.107:0/3987073128 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T20:48:12.909 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:10.310+0000 7f7e83400700 1 -- 172.21.15.107:0/3987073128 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e8419de80 con 0x7f7e84108e30 2024-03-08T20:48:12.909 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:10.310+0000 7f7e83400700 1 --2- 172.21.15.107:0/3987073128 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7e84108e30 0x7f7e8419d940 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f7e74006b20 tx=0x7f7e74011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:48:12.910 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:10.311+0000 7f7e7be00700 1 -- 172.21.15.107:0/3987073128 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7e740039e0 con 0x7f7e84108e30 2024-03-08T20:48:12.910 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:10.311+0000 7f7e7be00700 1 -- 172.21.15.107:0/3987073128 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f7e7401a8e0 con 0x7f7e84108e30 2024-03-08T20:48:12.910 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:10.311+0000 7f7e8a800700 1 -- 172.21.15.107:0/3987073128 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e8419e110 con 0x7f7e84108e30 2024-03-08T20:48:12.910 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:10.311+0000 7f7e7be00700 1 -- 172.21.15.107:0/3987073128 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7e740247e0 con 0x7f7e84108e30 2024-03-08T20:48:12.910 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:10.311+0000 7f7e8a800700 1 -- 172.21.15.107:0/3987073128 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e841a0e00 con 0x7f7e84108e30 2024-03-08T20:48:12.910 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:10.312+0000 7f7e7be00700 1 -- 172.21.15.107:0/3987073128 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 8) v1 ==== 46106+0+0 (secure 0 0 0) 0x7f7e74022020 con 0x7f7e84108e30 2024-03-08T20:48:12.910 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:10.312+0000 7f7e8a800700 1 -- 172.21.15.107:0/3987073128 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e8410be80 con 0x7f7e84108e30 2024-03-08T20:48:12.910 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:10.313+0000 7f7e7be00700 1 --2- 172.21.15.107:0/3987073128 >> [v2:172.21.15.107:6800/426372446,v1:172.21.15.107:6801/426372446] conn(0x7f7e68038f20 0x7f7e6803b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:48:12.910 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:10.313+0000 7f7e7be00700 1 -- 172.21.15.107:0/3987073128 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f7e7404d240 con 0x7f7e84108e30 2024-03-08T20:48:12.910 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:10.313+0000 7f7e82a00700 1 --2- 172.21.15.107:0/3987073128 >> [v2:172.21.15.107:6800/426372446,v1:172.21.15.107:6801/426372446] conn(0x7f7e68038f20 0x7f7e6803b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:48:12.910 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:10.314+0000 7f7e82a00700 1 --2- 172.21.15.107:0/3987073128 >> [v2:172.21.15.107:6800/426372446,v1:172.21.15.107:6801/426372446] conn(0x7f7e68038f20 0x7f7e6803b3e0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f7e70006fd0 tx=0x7f7e70006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:48:12.910 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:10.318+0000 7f7e7be00700 1 -- 172.21.15.107:0/3987073128 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f7e7402ad60 con 0x7f7e84108e30 2024-03-08T20:48:12.911 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:10.606+0000 7f7e8a800700 1 -- 172.21.15.107:0/3987073128 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) v1 -- 0x7f7e841a1220 con 0x7f7e84108e30 2024-03-08T20:48:12.911 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:11.272+0000 7f7e7be00700 1 -- 172.21.15.107:0/3987073128 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) v1 ==== 88+0+0 (secure 0 0 0) 0x7f7e74021d60 con 0x7f7e84108e30 2024-03-08T20:48:12.911 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:11.276+0000 7f7e7be00700 1 -- 172.21.15.107:0/3987073128 <== mon.0 v2:172.21.15.107:3300/0 8 ==== mgrmap(e 9) v1 ==== 46119+0+0 (secure 0 0 0) 0x7f7e7402d3f0 con 0x7f7e84108e30 2024-03-08T20:48:12.911 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:11.278+0000 7f7e8a800700 1 -- 172.21.15.107:0/3987073128 >> [v2:172.21.15.107:6800/426372446,v1:172.21.15.107:6801/426372446] conn(0x7f7e68038f20 msgr2=0x7f7e6803b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:48:12.911 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:11.278+0000 7f7e8a800700 1 --2- 172.21.15.107:0/3987073128 >> [v2:172.21.15.107:6800/426372446,v1:172.21.15.107:6801/426372446] conn(0x7f7e68038f20 0x7f7e6803b3e0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f7e70006fd0 tx=0x7f7e70006e40 comp rx=0 tx=0).stop 2024-03-08T20:48:12.911 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:11.278+0000 7f7e8a800700 1 -- 172.21.15.107:0/3987073128 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7e84108e30 msgr2=0x7f7e8419d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:48:12.911 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:11.278+0000 7f7e8a800700 1 --2- 172.21.15.107:0/3987073128 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7e84108e30 0x7f7e8419d940 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f7e74006b20 tx=0x7f7e74011ad0 comp rx=0 tx=0).stop 2024-03-08T20:48:12.911 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:11.278+0000 7f7e8a800700 1 -- 172.21.15.107:0/3987073128 shutdown_connections 2024-03-08T20:48:12.911 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:11.278+0000 7f7e8a800700 1 --2- 172.21.15.107:0/3987073128 >> [v2:172.21.15.107:6800/426372446,v1:172.21.15.107:6801/426372446] conn(0x7f7e68038f20 0x7f7e6803b3e0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:48:12.911 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:11.278+0000 7f7e8a800700 1 --2- 172.21.15.107:0/3987073128 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7e84108e30 0x7f7e8419d940 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:48:12.911 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:11.278+0000 7f7e8a800700 1 -- 172.21.15.107:0/3987073128 >> 172.21.15.107:0/3987073128 conn(0x7f7e84102da0 msgr2=0x7f7e84103760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:48:12.911 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:11.278+0000 7f7e8a800700 1 -- 172.21.15.107:0/3987073128 shutdown_connections 2024-03-08T20:48:12.911 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:11.279+0000 7f7e8a800700 1 -- 172.21.15.107:0/3987073128 wait complete. 2024-03-08T20:48:13.227 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 08 20:48:12 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-x[31606]: 2024-03-08T20:48:12.996+0000 7f8cbad18200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-08T20:48:13.228 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 08 20:48:13 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-x[31606]: 2024-03-08T20:48:13.117+0000 7f8cbad18200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-08T20:48:13.527 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 08 20:48:13 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-x[31606]: 2024-03-08T20:48:13.302+0000 7f8cbad18200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-08T20:48:13.527 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 08 20:48:13 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-x[31606]: 2024-03-08T20:48:13.430+0000 7f8cbad18200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-08T20:48:13.825 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 08 20:48:13 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-x[31606]: 2024-03-08T20:48:13.550+0000 7f8cbad18200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-08T20:48:14.152 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:48:13 smithi107 ceph-mon[31381]: from='client.? 172.21.15.107:0/1544825749' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-03-08T20:48:14.152 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 08 20:48:13 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-x[31606]: 2024-03-08T20:48:13.895+0000 7f8cbad18200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-08T20:48:14.152 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 08 20:48:14 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-x[31606]: 2024-03-08T20:48:14.040+0000 7f8cbad18200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-08T20:48:14.152 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 08 20:48:14 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-x[31606]: 2024-03-08T20:48:14.158+0000 7f8cbad18200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-08T20:48:14.477 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 08 20:48:14 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-x[31606]: 2024-03-08T20:48:14.286+0000 7f8cbad18200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-08T20:48:14.512 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout { 2024-03-08T20:48:14.512 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-03-08T20:48:14.512 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "available": true, 2024-03-08T20:48:14.512 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-03-08T20:48:14.512 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-03-08T20:48:14.512 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout } 2024-03-08T20:48:14.512 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:13.533+0000 7f7380400700 1 Processor -- start 2024-03-08T20:48:14.512 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:13.533+0000 7f7380400700 1 -- start start 2024-03-08T20:48:14.512 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:13.534+0000 7f7380400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7378106050 0x7f7378106470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:48:14.512 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:13.534+0000 7f7380400700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f737807ae50 con 0x7f7378106050 2024-03-08T20:48:14.512 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:13.534+0000 7f737da00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7378106050 0x7f7378106470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:48:14.513 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:13.534+0000 7f737da00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7378106050 0x7f7378106470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:59434/0 (socket says 172.21.15.107:59434) 2024-03-08T20:48:14.513 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:13.534+0000 7f737da00700 1 -- 172.21.15.107:0/952212801 learned_addr learned my addr 172.21.15.107:0/952212801 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T20:48:14.513 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:13.535+0000 7f737da00700 1 -- 172.21.15.107:0/952212801 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73781069b0 con 0x7f7378106050 2024-03-08T20:48:14.513 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:13.535+0000 7f737da00700 1 --2- 172.21.15.107:0/952212801 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7378106050 0x7f7378106470 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f736c0098a0 tx=0x7f736c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=527c0a7851380b8a server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:48:14.513 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:13.536+0000 7f7377e00700 1 -- 172.21.15.107:0/952212801 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f736c004160 con 0x7f7378106050 2024-03-08T20:48:14.513 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:13.536+0000 7f7377e00700 1 -- 172.21.15.107:0/952212801 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f736c004300 con 0x7f7378106050 2024-03-08T20:48:14.513 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:13.536+0000 7f7377e00700 1 -- 172.21.15.107:0/952212801 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f736c011a50 con 0x7f7378106050 2024-03-08T20:48:14.513 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:13.536+0000 7f7380400700 1 -- 172.21.15.107:0/952212801 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7378106050 msgr2=0x7f7378106470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:48:14.513 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:13.536+0000 7f7380400700 1 --2- 172.21.15.107:0/952212801 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7378106050 0x7f7378106470 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f736c0098a0 tx=0x7f736c009bb0 comp rx=0 tx=0).stop 2024-03-08T20:48:14.513 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:13.537+0000 7f7380400700 1 -- 172.21.15.107:0/952212801 shutdown_connections 2024-03-08T20:48:14.513 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:13.537+0000 7f7380400700 1 --2- 172.21.15.107:0/952212801 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7378106050 0x7f7378106470 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:48:14.513 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:13.537+0000 7f7380400700 1 -- 172.21.15.107:0/952212801 >> 172.21.15.107:0/952212801 conn(0x7f7378101cd0 msgr2=0x7f73781040f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:48:14.514 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:13.537+0000 7f7380400700 1 -- 172.21.15.107:0/952212801 shutdown_connections 2024-03-08T20:48:14.514 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:13.537+0000 7f7380400700 1 -- 172.21.15.107:0/952212801 wait complete. 2024-03-08T20:48:14.514 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:13.538+0000 7f7380400700 1 Processor -- start 2024-03-08T20:48:14.514 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:13.538+0000 7f7380400700 1 -- start start 2024-03-08T20:48:14.514 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:13.538+0000 7f7380400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7378106050 0x7f737819d800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:48:14.514 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:13.538+0000 7f7380400700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7378107a20 con 0x7f7378106050 2024-03-08T20:48:14.514 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:13.539+0000 7f737da00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7378106050 0x7f737819d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:48:14.514 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:13.539+0000 7f737da00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7378106050 0x7f737819d800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:59438/0 (socket says 172.21.15.107:59438) 2024-03-08T20:48:14.514 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:13.539+0000 7f737da00700 1 -- 172.21.15.107:0/1544825749 learned_addr learned my addr 172.21.15.107:0/1544825749 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T20:48:14.515 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:13.539+0000 7f737da00700 1 -- 172.21.15.107:0/1544825749 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f737819dd40 con 0x7f7378106050 2024-03-08T20:48:14.515 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:13.539+0000 7f737da00700 1 --2- 172.21.15.107:0/1544825749 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7378106050 0x7f737819d800 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f736c006b20 tx=0x7f736c011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:48:14.515 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:13.539+0000 7f7376000700 1 -- 172.21.15.107:0/1544825749 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f736c003b70 con 0x7f7378106050 2024-03-08T20:48:14.515 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:13.540+0000 7f7376000700 1 -- 172.21.15.107:0/1544825749 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f736c01a8e0 con 0x7f7378106050 2024-03-08T20:48:14.515 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:13.540+0000 7f7380400700 1 -- 172.21.15.107:0/1544825749 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f737819dfd0 con 0x7f7378106050 2024-03-08T20:48:14.515 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:13.540+0000 7f7376000700 1 -- 172.21.15.107:0/1544825749 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f736c0249b0 con 0x7f7378106050 2024-03-08T20:48:14.515 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:13.540+0000 7f7380400700 1 -- 172.21.15.107:0/1544825749 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73781a0cc0 con 0x7f7378106050 2024-03-08T20:48:14.515 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:13.541+0000 7f7380400700 1 -- 172.21.15.107:0/1544825749 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7378063410 con 0x7f7378106050 2024-03-08T20:48:14.515 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:13.541+0000 7f7376000700 1 -- 172.21.15.107:0/1544825749 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 9) v1 ==== 46119+0+0 (secure 0 0 0) 0x7f736c022020 con 0x7f7378106050 2024-03-08T20:48:14.515 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:13.542+0000 7f7376000700 1 --2- 172.21.15.107:0/1544825749 >> [v2:172.21.15.107:6800/426372446,v1:172.21.15.107:6801/426372446] conn(0x7f7360038f70 0x7f736003b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:48:14.516 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:13.542+0000 7f7376000700 1 -- 172.21.15.107:0/1544825749 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f736c04d6d0 con 0x7f7378106050 2024-03-08T20:48:14.516 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:13.542+0000 7f737d000700 1 -- 172.21.15.107:0/1544825749 >> [v2:172.21.15.107:6800/426372446,v1:172.21.15.107:6801/426372446] conn(0x7f7360038f70 msgr2=0x7f736003b430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.107:6800/426372446 2024-03-08T20:48:14.516 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:13.542+0000 7f737d000700 1 --2- 172.21.15.107:0/1544825749 >> [v2:172.21.15.107:6800/426372446,v1:172.21.15.107:6801/426372446] conn(0x7f7360038f70 0x7f736003b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-08T20:48:14.516 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:13.551+0000 7f7376000700 1 -- 172.21.15.107:0/1544825749 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f736c02c3a0 con 0x7f7378106050 2024-03-08T20:48:14.516 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:13.743+0000 7f737d000700 1 -- 172.21.15.107:0/1544825749 >> [v2:172.21.15.107:6800/426372446,v1:172.21.15.107:6801/426372446] conn(0x7f7360038f70 msgr2=0x7f736003b430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.107:6800/426372446 2024-03-08T20:48:14.516 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:13.743+0000 7f737d000700 1 --2- 172.21.15.107:0/1544825749 >> [v2:172.21.15.107:6800/426372446,v1:172.21.15.107:6801/426372446] conn(0x7f7360038f70 0x7f736003b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-08T20:48:14.516 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:13.831+0000 7f7380400700 1 -- 172.21.15.107:0/1544825749 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f737802da10 con 0x7f7378106050 2024-03-08T20:48:14.516 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:13.832+0000 7f7376000700 1 -- 172.21.15.107:0/1544825749 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) v1 ==== 56+0+88 (secure 0 0 0) 0x7f736c00be50 con 0x7f7378106050 2024-03-08T20:48:14.516 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:13.836+0000 7f7380400700 1 -- 172.21.15.107:0/1544825749 >> [v2:172.21.15.107:6800/426372446,v1:172.21.15.107:6801/426372446] conn(0x7f7360038f70 msgr2=0x7f736003b430 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-08T20:48:14.516 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:13.836+0000 7f7380400700 1 --2- 172.21.15.107:0/1544825749 >> [v2:172.21.15.107:6800/426372446,v1:172.21.15.107:6801/426372446] conn(0x7f7360038f70 0x7f736003b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:48:14.517 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:13.836+0000 7f7380400700 1 -- 172.21.15.107:0/1544825749 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7378106050 msgr2=0x7f737819d800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:48:14.517 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:13.836+0000 7f7380400700 1 --2- 172.21.15.107:0/1544825749 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7378106050 0x7f737819d800 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f736c006b20 tx=0x7f736c011ad0 comp rx=0 tx=0).stop 2024-03-08T20:48:14.517 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:13.836+0000 7f7380400700 1 -- 172.21.15.107:0/1544825749 shutdown_connections 2024-03-08T20:48:14.517 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:13.836+0000 7f7380400700 1 --2- 172.21.15.107:0/1544825749 >> [v2:172.21.15.107:6800/426372446,v1:172.21.15.107:6801/426372446] conn(0x7f7360038f70 0x7f736003b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:48:14.517 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:13.836+0000 7f7380400700 1 --2- 172.21.15.107:0/1544825749 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7378106050 0x7f737819d800 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:48:14.517 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:13.836+0000 7f7380400700 1 -- 172.21.15.107:0/1544825749 >> 172.21.15.107:0/1544825749 conn(0x7f7378101cd0 msgr2=0x7f7378194470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:48:14.517 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:13.836+0000 7f7380400700 1 -- 172.21.15.107:0/1544825749 shutdown_connections 2024-03-08T20:48:14.517 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:13.836+0000 7f7380400700 1 -- 172.21.15.107:0/1544825749 wait complete. 2024-03-08T20:48:14.517 INFO:teuthology.orchestra.run.smithi107.stdout:Waiting for the mgr to restart... 2024-03-08T20:48:14.517 INFO:teuthology.orchestra.run.smithi107.stdout:Waiting for mgr epoch 9... 2024-03-08T20:48:14.782 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 08 20:48:14 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-x[31606]: 2024-03-08T20:48:14.509+0000 7f8cbad18200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-08T20:48:15.978 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 08 20:48:15 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-x[31606]: 2024-03-08T20:48:15.866+0000 7f8cbad18200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-08T20:48:16.438 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 08 20:48:16 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-x[31606]: 2024-03-08T20:48:16.034+0000 7f8cbad18200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-08T20:48:16.438 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 08 20:48:16 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-x[31606]: 2024-03-08T20:48:16.444+0000 7f8cbad18200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-08T20:48:16.728 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 08 20:48:16 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-x[31606]: 2024-03-08T20:48:16.562+0000 7f8cbad18200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-08T20:48:17.228 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 08 20:48:16 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-x[31606]: 2024-03-08T20:48:16.811+0000 7f8cbad18200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-08T20:48:17.228 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 08 20:48:17 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-x[31606]: 2024-03-08T20:48:17.052+0000 7f8cbad18200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-08T20:48:17.727 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 08 20:48:17 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-x[31606]: 2024-03-08T20:48:17.421+0000 7f8cbad18200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-08T20:48:18.030 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 08 20:48:17 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-x[31606]: 2024-03-08T20:48:17.737+0000 7f8cbad18200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-08T20:48:18.030 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 08 20:48:18 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-x[31606]: 2024-03-08T20:48:18.036+0000 7f8cbad18200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-08T20:48:19.477 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 08 20:48:19 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-x[31606]: 2024-03-08T20:48:19.049+0000 7f8cbad18200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-08T20:48:20.140 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 08 20:48:19 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-x[31606]: 2024-03-08T20:48:19.740+0000 7f8cbad18200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-08T20:48:20.140 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 08 20:48:19 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-x[31606]: 2024-03-08T20:48:19.868+0000 7f8cbad18200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-08T20:48:20.477 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 08 20:48:20 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-x[31606]: 2024-03-08T20:48:20.146+0000 7f8cbad18200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-08T20:48:20.477 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:48:20 smithi107 ceph-mon[31381]: Active manager daemon x restarted 2024-03-08T20:48:20.477 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:48:20 smithi107 ceph-mon[31381]: Activating manager daemon x 2024-03-08T20:48:20.477 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:48:20 smithi107 ceph-mon[31381]: osdmap e3: 0 total, 0 up, 0 in 2024-03-08T20:48:20.478 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:48:20 smithi107 ceph-mon[31381]: mgrmap e10: x(active, starting, since 0.00859289s) 2024-03-08T20:48:20.478 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:48:20 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-08T20:48:20.478 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:48:20 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-08T20:48:20.478 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:48:20 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-08T20:48:20.478 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:48:20 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-08T20:48:20.478 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:48:20 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-08T20:48:20.478 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:48:20 smithi107 ceph-mon[31381]: Manager daemon x is now available 2024-03-08T20:48:21.478 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:48:21 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T20:48:21.478 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:48:21 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T20:48:21.478 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:48:21 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-08T20:48:21.478 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:48:21 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-08T20:48:21.478 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:48:21 smithi107 ceph-mon[31381]: [08/Mar/2024:20:48:20] ENGINE Bus STARTING 2024-03-08T20:48:21.478 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:48:21 smithi107 ceph-mon[31381]: mgrmap e11: x(active, since 1.01002s) 2024-03-08T20:48:21.836 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout { 2024-03-08T20:48:21.837 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-03-08T20:48:21.837 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "initialized": true 2024-03-08T20:48:21.837 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout } 2024-03-08T20:48:21.837 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:15.112+0000 7f7613e00700 1 Processor -- start 2024-03-08T20:48:21.837 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:15.113+0000 7f7613e00700 1 -- start start 2024-03-08T20:48:21.837 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:15.113+0000 7f7613e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f760c106050 0x7f760c106470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:48:21.837 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:15.113+0000 7f7613e00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f760c07ae50 con 0x7f760c106050 2024-03-08T20:48:21.837 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:15.114+0000 7f7611400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f760c106050 0x7f760c106470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:48:21.837 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:15.114+0000 7f7611400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f760c106050 0x7f760c106470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:59442/0 (socket says 172.21.15.107:59442) 2024-03-08T20:48:21.837 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:15.114+0000 7f7611400700 1 -- 172.21.15.107:0/273659077 learned_addr learned my addr 172.21.15.107:0/273659077 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T20:48:21.837 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:15.114+0000 7f7611400700 1 -- 172.21.15.107:0/273659077 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f760c1069b0 con 0x7f760c106050 2024-03-08T20:48:21.837 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:15.114+0000 7f7611400700 1 --2- 172.21.15.107:0/273659077 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f760c106050 0x7f760c106470 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f76000098a0 tx=0x7f7600009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b83ed23967d6d4a server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:48:21.837 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:15.115+0000 7f760be00700 1 -- 172.21.15.107:0/273659077 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7600004160 con 0x7f760c106050 2024-03-08T20:48:21.838 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:15.115+0000 7f760be00700 1 -- 172.21.15.107:0/273659077 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f7600004300 con 0x7f760c106050 2024-03-08T20:48:21.838 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:15.115+0000 7f760be00700 1 -- 172.21.15.107:0/273659077 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7600011a50 con 0x7f760c106050 2024-03-08T20:48:21.838 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:15.115+0000 7f7613e00700 1 -- 172.21.15.107:0/273659077 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f760c106050 msgr2=0x7f760c106470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:48:21.838 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:15.115+0000 7f7613e00700 1 --2- 172.21.15.107:0/273659077 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f760c106050 0x7f760c106470 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f76000098a0 tx=0x7f7600009bb0 comp rx=0 tx=0).stop 2024-03-08T20:48:21.838 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:15.116+0000 7f7613e00700 1 -- 172.21.15.107:0/273659077 shutdown_connections 2024-03-08T20:48:21.838 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:15.116+0000 7f7613e00700 1 --2- 172.21.15.107:0/273659077 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f760c106050 0x7f760c106470 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:48:21.838 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:15.116+0000 7f7613e00700 1 -- 172.21.15.107:0/273659077 >> 172.21.15.107:0/273659077 conn(0x7f760c101cd0 msgr2=0x7f760c1040f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:48:21.838 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:15.116+0000 7f7613e00700 1 -- 172.21.15.107:0/273659077 shutdown_connections 2024-03-08T20:48:21.838 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:15.116+0000 7f7613e00700 1 -- 172.21.15.107:0/273659077 wait complete. 2024-03-08T20:48:21.838 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:15.117+0000 7f7613e00700 1 Processor -- start 2024-03-08T20:48:21.838 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:15.117+0000 7f7613e00700 1 -- start start 2024-03-08T20:48:21.838 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:15.117+0000 7f7613e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f760c106050 0x7f760c19d800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:48:21.839 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:15.117+0000 7f7613e00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f760c107a20 con 0x7f760c106050 2024-03-08T20:48:21.839 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:15.117+0000 7f7611400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f760c106050 0x7f760c19d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:48:21.839 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:15.117+0000 7f7611400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f760c106050 0x7f760c19d800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:59452/0 (socket says 172.21.15.107:59452) 2024-03-08T20:48:21.839 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:15.117+0000 7f7611400700 1 -- 172.21.15.107:0/3204318732 learned_addr learned my addr 172.21.15.107:0/3204318732 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T20:48:21.839 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:15.118+0000 7f7611400700 1 -- 172.21.15.107:0/3204318732 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f760c19dd40 con 0x7f760c106050 2024-03-08T20:48:21.839 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:15.118+0000 7f7611400700 1 --2- 172.21.15.107:0/3204318732 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f760c106050 0x7f760c19d800 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f7600006b20 tx=0x7f7600011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:48:21.839 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:15.118+0000 7f760a000700 1 -- 172.21.15.107:0/3204318732 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7600003b70 con 0x7f760c106050 2024-03-08T20:48:21.839 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:15.118+0000 7f760a000700 1 -- 172.21.15.107:0/3204318732 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f760001a8e0 con 0x7f760c106050 2024-03-08T20:48:21.839 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:15.118+0000 7f760a000700 1 -- 172.21.15.107:0/3204318732 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f76000249b0 con 0x7f760c106050 2024-03-08T20:48:21.839 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:15.118+0000 7f7613e00700 1 -- 172.21.15.107:0/3204318732 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f760c19dfd0 con 0x7f760c106050 2024-03-08T20:48:21.839 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:15.118+0000 7f7613e00700 1 -- 172.21.15.107:0/3204318732 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f760c1a0cc0 con 0x7f760c106050 2024-03-08T20:48:21.839 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:15.120+0000 7f760a000700 1 -- 172.21.15.107:0/3204318732 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 9) v1 ==== 46119+0+0 (secure 0 0 0) 0x7f7600022020 con 0x7f760c106050 2024-03-08T20:48:21.839 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:15.120+0000 7f760a000700 1 --2- 172.21.15.107:0/3204318732 >> [v2:172.21.15.107:6800/426372446,v1:172.21.15.107:6801/426372446] conn(0x7f75f4038f70 0x7f75f403b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:48:21.839 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:15.120+0000 7f760a000700 1 -- 172.21.15.107:0/3204318732 --> [v2:172.21.15.107:6800/426372446,v1:172.21.15.107:6801/426372446] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f75f403bb40 con 0x7f75f4038f70 2024-03-08T20:48:21.840 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:15.120+0000 7f760a000700 1 -- 172.21.15.107:0/3204318732 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f760004d680 con 0x7f760c106050 2024-03-08T20:48:21.840 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:15.121+0000 7f7610a00700 1 -- 172.21.15.107:0/3204318732 >> [v2:172.21.15.107:6800/426372446,v1:172.21.15.107:6801/426372446] conn(0x7f75f4038f70 msgr2=0x7f75f403b430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.107:6800/426372446 2024-03-08T20:48:21.840 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:15.121+0000 7f7610a00700 1 --2- 172.21.15.107:0/3204318732 >> [v2:172.21.15.107:6800/426372446,v1:172.21.15.107:6801/426372446] conn(0x7f75f4038f70 0x7f75f403b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-08T20:48:21.840 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:15.322+0000 7f7610a00700 1 -- 172.21.15.107:0/3204318732 >> [v2:172.21.15.107:6800/426372446,v1:172.21.15.107:6801/426372446] conn(0x7f75f4038f70 msgr2=0x7f75f403b430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.107:6800/426372446 2024-03-08T20:48:21.840 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:15.322+0000 7f7610a00700 1 --2- 172.21.15.107:0/3204318732 >> [v2:172.21.15.107:6800/426372446,v1:172.21.15.107:6801/426372446] conn(0x7f75f4038f70 0x7f75f403b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-08T20:48:21.840 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:15.723+0000 7f7610a00700 1 -- 172.21.15.107:0/3204318732 >> [v2:172.21.15.107:6800/426372446,v1:172.21.15.107:6801/426372446] conn(0x7f75f4038f70 msgr2=0x7f75f403b430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.107:6800/426372446 2024-03-08T20:48:21.840 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:15.723+0000 7f7610a00700 1 --2- 172.21.15.107:0/3204318732 >> [v2:172.21.15.107:6800/426372446,v1:172.21.15.107:6801/426372446] conn(0x7f75f4038f70 0x7f75f403b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-08T20:48:21.840 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:16.524+0000 7f7610a00700 1 -- 172.21.15.107:0/3204318732 >> [v2:172.21.15.107:6800/426372446,v1:172.21.15.107:6801/426372446] conn(0x7f75f4038f70 msgr2=0x7f75f403b430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.107:6800/426372446 2024-03-08T20:48:21.840 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:16.524+0000 7f7610a00700 1 --2- 172.21.15.107:0/3204318732 >> [v2:172.21.15.107:6800/426372446,v1:172.21.15.107:6801/426372446] conn(0x7f75f4038f70 0x7f75f403b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-08T20:48:21.840 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:18.126+0000 7f7610a00700 1 -- 172.21.15.107:0/3204318732 >> [v2:172.21.15.107:6800/426372446,v1:172.21.15.107:6801/426372446] conn(0x7f75f4038f70 msgr2=0x7f75f403b430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.107:6800/426372446 2024-03-08T20:48:21.840 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:18.126+0000 7f7610a00700 1 --2- 172.21.15.107:0/3204318732 >> [v2:172.21.15.107:6800/426372446,v1:172.21.15.107:6801/426372446] conn(0x7f75f4038f70 0x7f75f403b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-03-08T20:48:21.840 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:20.154+0000 7f760a000700 1 -- 172.21.15.107:0/3204318732 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mgrmap(e 10) v1 ==== 45886+0+0 (secure 0 0 0) 0x7f760000b3e0 con 0x7f760c106050 2024-03-08T20:48:21.840 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:20.155+0000 7f760a000700 1 -- 172.21.15.107:0/3204318732 >> [v2:172.21.15.107:6800/426372446,v1:172.21.15.107:6801/426372446] conn(0x7f75f4038f70 msgr2=0x7f75f403b430 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-08T20:48:21.841 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:20.155+0000 7f760a000700 1 --2- 172.21.15.107:0/3204318732 >> [v2:172.21.15.107:6800/426372446,v1:172.21.15.107:6801/426372446] conn(0x7f75f4038f70 0x7f75f403b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:48:21.841 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:21.156+0000 7f760a000700 1 -- 172.21.15.107:0/3204318732 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mgrmap(e 11) v1 ==== 46013+0+0 (secure 0 0 0) 0x7f7600018550 con 0x7f760c106050 2024-03-08T20:48:21.841 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:21.156+0000 7f760a000700 1 --2- 172.21.15.107:0/3204318732 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f75f403c9f0 0x7f75f403ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:48:21.841 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:21.156+0000 7f760a000700 1 -- 172.21.15.107:0/3204318732 --> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f75f403bb40 con 0x7f75f403c9f0 2024-03-08T20:48:21.841 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:21.157+0000 7f7610a00700 1 --2- 172.21.15.107:0/3204318732 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f75f403c9f0 0x7f75f403ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:48:21.841 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:21.158+0000 7f7610a00700 1 --2- 172.21.15.107:0/3204318732 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f75f403c9f0 0x7f75f403ede0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f75f8003a10 tx=0x7f75f80092b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:48:21.841 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:21.159+0000 7f760a000700 1 -- 172.21.15.107:0/3204318732 <== mgr.14152 v2:172.21.15.107:6800/3376636618 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+7774 (secure 0 0 0) 0x7f75f403bb40 con 0x7f75f403c9f0 2024-03-08T20:48:21.841 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:21.167+0000 7f7613e00700 1 -- 172.21.15.107:0/3204318732 --> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7f760c19e4b0 con 0x7f75f403c9f0 2024-03-08T20:48:21.841 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:21.168+0000 7f760a000700 1 -- 172.21.15.107:0/3204318732 <== mgr.14152 v2:172.21.15.107:6800/3376636618 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+52 (secure 0 0 0) 0x7f760c19e4b0 con 0x7f75f403c9f0 2024-03-08T20:48:21.842 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:21.168+0000 7f7613e00700 1 -- 172.21.15.107:0/3204318732 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f75f403c9f0 msgr2=0x7f75f403ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:48:21.842 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:21.168+0000 7f7613e00700 1 --2- 172.21.15.107:0/3204318732 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f75f403c9f0 0x7f75f403ede0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f75f8003a10 tx=0x7f75f80092b0 comp rx=0 tx=0).stop 2024-03-08T20:48:21.842 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:21.168+0000 7f7613e00700 1 -- 172.21.15.107:0/3204318732 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f760c106050 msgr2=0x7f760c19d800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:48:21.842 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:21.168+0000 7f7613e00700 1 --2- 172.21.15.107:0/3204318732 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f760c106050 0x7f760c19d800 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f7600006b20 tx=0x7f7600011ad0 comp rx=0 tx=0).stop 2024-03-08T20:48:21.842 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:21.168+0000 7f7613e00700 1 -- 172.21.15.107:0/3204318732 shutdown_connections 2024-03-08T20:48:21.842 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:21.168+0000 7f7613e00700 1 --2- 172.21.15.107:0/3204318732 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f75f403c9f0 0x7f75f403ede0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:48:21.842 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:21.168+0000 7f7613e00700 1 --2- 172.21.15.107:0/3204318732 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f760c106050 0x7f760c19d800 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:48:21.842 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:21.168+0000 7f7613e00700 1 --2- 172.21.15.107:0/3204318732 >> [v2:172.21.15.107:6800/426372446,v1:172.21.15.107:6801/426372446] conn(0x7f75f4038f70 0x7f75f403b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:48:21.842 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:21.168+0000 7f7613e00700 1 -- 172.21.15.107:0/3204318732 >> 172.21.15.107:0/3204318732 conn(0x7f760c101cd0 msgr2=0x7f760c194470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:48:21.842 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:21.168+0000 7f7613e00700 1 -- 172.21.15.107:0/3204318732 shutdown_connections 2024-03-08T20:48:21.842 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:21.168+0000 7f7613e00700 1 -- 172.21.15.107:0/3204318732 wait complete. 2024-03-08T20:48:21.842 INFO:teuthology.orchestra.run.smithi107.stdout:mgr epoch 9 is available 2024-03-08T20:48:21.843 INFO:teuthology.orchestra.run.smithi107.stdout:Generating a dashboard self-signed certificate... 2024-03-08T20:48:22.385 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:48:22 smithi107 ceph-mon[31381]: [08/Mar/2024:20:48:20] ENGINE Serving on http://172.21.15.107:8765 2024-03-08T20:48:22.385 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:48:22 smithi107 ceph-mon[31381]: [08/Mar/2024:20:48:20] ENGINE Serving on https://172.21.15.107:7150 2024-03-08T20:48:22.385 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:48:22 smithi107 ceph-mon[31381]: [08/Mar/2024:20:48:20] ENGINE Bus STARTED 2024-03-08T20:48:22.385 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:48:22 smithi107 ceph-mon[31381]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-03-08T20:48:22.385 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:48:22 smithi107 ceph-mon[31381]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-03-08T20:48:23.477 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:48:23 smithi107 ceph-mon[31381]: mgrmap e12: x(active, since 2s) 2024-03-08T20:48:23.477 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:48:23 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:48:23.477 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:48:23 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:48:23.676 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-03-08T20:48:23.676 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:22.547+0000 7f076d200700 1 Processor -- start 2024-03-08T20:48:23.676 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:22.547+0000 7f076d200700 1 -- start start 2024-03-08T20:48:23.677 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:22.548+0000 7f076d200700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f076810e0d0 0x7f076810e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:48:23.677 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:22.548+0000 7f076d200700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f076810eac0 con 0x7f076810e0d0 2024-03-08T20:48:23.677 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:22.548+0000 7f0765e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f076810e0d0 0x7f076810e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:48:23.677 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:22.548+0000 7f0765e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f076810e0d0 0x7f076810e4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:49506/0 (socket says 172.21.15.107:49506) 2024-03-08T20:48:23.677 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:22.548+0000 7f0765e00700 1 -- 172.21.15.107:0/685213997 learned_addr learned my addr 172.21.15.107:0/685213997 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T20:48:23.677 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:22.549+0000 7f0765e00700 1 -- 172.21.15.107:0/685213997 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f076810ef10 con 0x7f076810e0d0 2024-03-08T20:48:23.677 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:22.549+0000 7f0765e00700 1 --2- 172.21.15.107:0/685213997 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f076810e0d0 0x7f076810e4f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f07580098a0 tx=0x7f0758009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab21f589587912fa server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:48:23.677 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:22.549+0000 7f0765400700 1 -- 172.21.15.107:0/685213997 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0758004160 con 0x7f076810e0d0 2024-03-08T20:48:23.677 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:22.549+0000 7f0765400700 1 -- 172.21.15.107:0/685213997 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f0758004300 con 0x7f076810e0d0 2024-03-08T20:48:23.677 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:22.549+0000 7f0765400700 1 -- 172.21.15.107:0/685213997 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0758011a50 con 0x7f076810e0d0 2024-03-08T20:48:23.677 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:22.550+0000 7f076d200700 1 -- 172.21.15.107:0/685213997 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f076810e0d0 msgr2=0x7f076810e4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:48:23.677 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:22.550+0000 7f076d200700 1 --2- 172.21.15.107:0/685213997 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f076810e0d0 0x7f076810e4f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f07580098a0 tx=0x7f0758009bb0 comp rx=0 tx=0).stop 2024-03-08T20:48:23.678 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:22.550+0000 7f076d200700 1 -- 172.21.15.107:0/685213997 shutdown_connections 2024-03-08T20:48:23.678 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:22.550+0000 7f076d200700 1 --2- 172.21.15.107:0/685213997 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f076810e0d0 0x7f076810e4f0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:48:23.678 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:22.550+0000 7f076d200700 1 -- 172.21.15.107:0/685213997 >> 172.21.15.107:0/685213997 conn(0x7f076807cdd0 msgr2=0x7f076807f1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:48:23.678 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:22.550+0000 7f076d200700 1 -- 172.21.15.107:0/685213997 shutdown_connections 2024-03-08T20:48:23.678 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:22.550+0000 7f076d200700 1 -- 172.21.15.107:0/685213997 wait complete. 2024-03-08T20:48:23.678 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:22.551+0000 7f076d200700 1 Processor -- start 2024-03-08T20:48:23.678 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:22.551+0000 7f076d200700 1 -- start start 2024-03-08T20:48:23.678 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:22.552+0000 7f076d200700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f076810e0d0 0x7f07681a1c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:48:23.678 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:22.552+0000 7f076d200700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07681122c0 con 0x7f076810e0d0 2024-03-08T20:48:23.678 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:22.552+0000 7f0765e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f076810e0d0 0x7f07681a1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:48:23.678 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:22.552+0000 7f0765e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f076810e0d0 0x7f07681a1c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:49520/0 (socket says 172.21.15.107:49520) 2024-03-08T20:48:23.678 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:22.552+0000 7f0765e00700 1 -- 172.21.15.107:0/949401660 learned_addr learned my addr 172.21.15.107:0/949401660 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T20:48:23.679 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:22.552+0000 7f0765e00700 1 -- 172.21.15.107:0/949401660 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07681a2150 con 0x7f076810e0d0 2024-03-08T20:48:23.679 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:22.553+0000 7f0765e00700 1 --2- 172.21.15.107:0/949401660 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f076810e0d0 0x7f07681a1c10 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f0758006b20 tx=0x7f0758011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:48:23.679 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:22.553+0000 7f075ea00700 1 -- 172.21.15.107:0/949401660 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f07580039e0 con 0x7f076810e0d0 2024-03-08T20:48:23.679 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:22.553+0000 7f075ea00700 1 -- 172.21.15.107:0/949401660 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f075801a8e0 con 0x7f076810e0d0 2024-03-08T20:48:23.679 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:22.553+0000 7f076d200700 1 -- 172.21.15.107:0/949401660 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07681a23e0 con 0x7f076810e0d0 2024-03-08T20:48:23.679 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:22.553+0000 7f075ea00700 1 -- 172.21.15.107:0/949401660 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f07580247e0 con 0x7f076810e0d0 2024-03-08T20:48:23.679 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:22.553+0000 7f076d200700 1 -- 172.21.15.107:0/949401660 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07681a50d0 con 0x7f076810e0d0 2024-03-08T20:48:23.679 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:22.554+0000 7f075ea00700 1 -- 172.21.15.107:0/949401660 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 12) v1 ==== 46119+0+0 (secure 0 0 0) 0x7f0758022020 con 0x7f076810e0d0 2024-03-08T20:48:23.679 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:22.554+0000 7f076d200700 1 -- 172.21.15.107:0/949401660 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0768063410 con 0x7f076810e0d0 2024-03-08T20:48:23.679 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:22.555+0000 7f075ea00700 1 --2- 172.21.15.107:0/949401660 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f074c038fc0 0x7f074c03b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:48:23.679 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:22.555+0000 7f075ea00700 1 -- 172.21.15.107:0/949401660 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f075804d3e0 con 0x7f076810e0d0 2024-03-08T20:48:23.679 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:22.555+0000 7f075fe00700 1 --2- 172.21.15.107:0/949401660 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f074c038fc0 0x7f074c03b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:48:23.679 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:22.556+0000 7f075fe00700 1 --2- 172.21.15.107:0/949401660 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f074c038fc0 0x7f074c03b480 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f0750006fd0 tx=0x7f0750006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:48:23.680 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:22.560+0000 7f075ea00700 1 -- 172.21.15.107:0/949401660 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f075802ad60 con 0x7f076810e0d0 2024-03-08T20:48:23.680 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:22.793+0000 7f076d200700 1 -- 172.21.15.107:0/949401660 --> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) v1 -- 0x7f076807df70 con 0x7f074c038fc0 2024-03-08T20:48:23.680 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:22.932+0000 7f075ea00700 1 -- 172.21.15.107:0/949401660 <== mgr.14152 v2:172.21.15.107:6800/3376636618 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+32 (secure 0 0 0) 0x7f076807df70 con 0x7f074c038fc0 2024-03-08T20:48:23.680 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:22.935+0000 7f076d200700 1 -- 172.21.15.107:0/949401660 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f074c038fc0 msgr2=0x7f074c03b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:48:23.680 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:22.935+0000 7f076d200700 1 --2- 172.21.15.107:0/949401660 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f074c038fc0 0x7f074c03b480 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f0750006fd0 tx=0x7f0750006e40 comp rx=0 tx=0).stop 2024-03-08T20:48:23.680 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:22.935+0000 7f076d200700 1 -- 172.21.15.107:0/949401660 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f076810e0d0 msgr2=0x7f07681a1c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:48:23.680 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:22.935+0000 7f076d200700 1 --2- 172.21.15.107:0/949401660 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f076810e0d0 0x7f07681a1c10 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f0758006b20 tx=0x7f0758011ad0 comp rx=0 tx=0).stop 2024-03-08T20:48:23.680 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:22.936+0000 7f076d200700 1 -- 172.21.15.107:0/949401660 shutdown_connections 2024-03-08T20:48:23.680 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:22.936+0000 7f076d200700 1 --2- 172.21.15.107:0/949401660 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f074c038fc0 0x7f074c03b480 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:48:23.680 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:22.936+0000 7f076d200700 1 --2- 172.21.15.107:0/949401660 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f076810e0d0 0x7f07681a1c10 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:48:23.680 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:22.936+0000 7f076d200700 1 -- 172.21.15.107:0/949401660 >> 172.21.15.107:0/949401660 conn(0x7f076807cdd0 msgr2=0x7f076807d860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:48:23.680 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:22.936+0000 7f076d200700 1 -- 172.21.15.107:0/949401660 shutdown_connections 2024-03-08T20:48:23.680 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:22.936+0000 7f076d200700 1 -- 172.21.15.107:0/949401660 wait complete. 2024-03-08T20:48:23.680 INFO:teuthology.orchestra.run.smithi107.stdout:Creating initial admin user... 2024-03-08T20:48:24.510 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:48:24 smithi107 ceph-mon[31381]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-03-08T20:48:24.511 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:48:24 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:48:25.435 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$.qsQBvw4HIGUudet50SbPudFK602X2JINMA8OTzdNbF6iogKmaGXe", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1709930904, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-03-08T20:48:25.435 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:24.240+0000 7f20f3a00700 1 Processor -- start 2024-03-08T20:48:25.435 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:24.240+0000 7f20f3a00700 1 -- start start 2024-03-08T20:48:25.435 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:24.241+0000 7f20f3a00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f20ec107770 0x7f20ec107b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:48:25.435 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:24.241+0000 7f20f3a00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20ec108160 con 0x7f20ec107770 2024-03-08T20:48:25.435 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:24.241+0000 7f20f1000700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f20ec107770 0x7f20ec107b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:48:25.435 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:24.241+0000 7f20f1000700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f20ec107770 0x7f20ec107b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:49522/0 (socket says 172.21.15.107:49522) 2024-03-08T20:48:25.435 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:24.241+0000 7f20f1000700 1 -- 172.21.15.107:0/4184219284 learned_addr learned my addr 172.21.15.107:0/4184219284 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T20:48:25.435 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:24.242+0000 7f20f1000700 1 -- 172.21.15.107:0/4184219284 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20ec108920 con 0x7f20ec107770 2024-03-08T20:48:25.435 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:24.242+0000 7f20f1000700 1 --2- 172.21.15.107:0/4184219284 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f20ec107770 0x7f20ec107b90 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f20e00098a0 tx=0x7f20e0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d25cc2ab46717058 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:48:25.435 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:24.242+0000 7f20eb400700 1 -- 172.21.15.107:0/4184219284 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f20e0004160 con 0x7f20ec107770 2024-03-08T20:48:25.436 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:24.242+0000 7f20eb400700 1 -- 172.21.15.107:0/4184219284 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f20e0004300 con 0x7f20ec107770 2024-03-08T20:48:25.436 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:24.242+0000 7f20eb400700 1 -- 172.21.15.107:0/4184219284 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f20e0011a50 con 0x7f20ec107770 2024-03-08T20:48:25.436 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:24.243+0000 7f20f3a00700 1 -- 172.21.15.107:0/4184219284 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f20ec107770 msgr2=0x7f20ec107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:48:25.436 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:24.243+0000 7f20f3a00700 1 --2- 172.21.15.107:0/4184219284 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f20ec107770 0x7f20ec107b90 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f20e00098a0 tx=0x7f20e0009bb0 comp rx=0 tx=0).stop 2024-03-08T20:48:25.436 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:24.243+0000 7f20f3a00700 1 -- 172.21.15.107:0/4184219284 shutdown_connections 2024-03-08T20:48:25.436 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:24.243+0000 7f20f3a00700 1 --2- 172.21.15.107:0/4184219284 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f20ec107770 0x7f20ec107b90 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:48:25.436 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:24.243+0000 7f20f3a00700 1 -- 172.21.15.107:0/4184219284 >> 172.21.15.107:0/4184219284 conn(0x7f20ec102dd0 msgr2=0x7f20ec1051f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:48:25.436 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:24.244+0000 7f20f3a00700 1 -- 172.21.15.107:0/4184219284 shutdown_connections 2024-03-08T20:48:25.436 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:24.244+0000 7f20f3a00700 1 -- 172.21.15.107:0/4184219284 wait complete. 2024-03-08T20:48:25.436 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:24.245+0000 7f20f3a00700 1 Processor -- start 2024-03-08T20:48:25.436 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:24.245+0000 7f20f3a00700 1 -- start start 2024-03-08T20:48:25.436 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:24.245+0000 7f20f3a00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f20ec107770 0x7f20ec19b620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:48:25.436 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:24.245+0000 7f20f3a00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20ec10bcd0 con 0x7f20ec107770 2024-03-08T20:48:25.437 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:24.245+0000 7f20f1000700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f20ec107770 0x7f20ec19b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:48:25.437 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:24.245+0000 7f20f1000700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f20ec107770 0x7f20ec19b620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:49534/0 (socket says 172.21.15.107:49534) 2024-03-08T20:48:25.437 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:24.245+0000 7f20f1000700 1 -- 172.21.15.107:0/4053185135 learned_addr learned my addr 172.21.15.107:0/4053185135 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T20:48:25.437 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:24.246+0000 7f20f1000700 1 -- 172.21.15.107:0/4053185135 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20ec19bb60 con 0x7f20ec107770 2024-03-08T20:48:25.437 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:24.246+0000 7f20f1000700 1 --2- 172.21.15.107:0/4053185135 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f20ec107770 0x7f20ec19b620 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f20e0006b20 tx=0x7f20e0011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:48:25.437 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:24.246+0000 7f20e9600700 1 -- 172.21.15.107:0/4053185135 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f20e00039e0 con 0x7f20ec107770 2024-03-08T20:48:25.437 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:24.246+0000 7f20e9600700 1 -- 172.21.15.107:0/4053185135 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f20e001a8e0 con 0x7f20ec107770 2024-03-08T20:48:25.437 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:24.246+0000 7f20f3a00700 1 -- 172.21.15.107:0/4053185135 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20ec19bdf0 con 0x7f20ec107770 2024-03-08T20:48:25.437 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:24.246+0000 7f20e9600700 1 -- 172.21.15.107:0/4053185135 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f20e00247e0 con 0x7f20ec107770 2024-03-08T20:48:25.437 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:24.246+0000 7f20f3a00700 1 -- 172.21.15.107:0/4053185135 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20ec19eae0 con 0x7f20ec107770 2024-03-08T20:48:25.437 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:24.248+0000 7f20e9600700 1 -- 172.21.15.107:0/4053185135 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 12) v1 ==== 46119+0+0 (secure 0 0 0) 0x7f20e0022020 con 0x7f20ec107770 2024-03-08T20:48:25.437 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:24.248+0000 7f20f3a00700 1 -- 172.21.15.107:0/4053185135 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20ec063410 con 0x7f20ec107770 2024-03-08T20:48:25.437 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:24.248+0000 7f20e9600700 1 --2- 172.21.15.107:0/4053185135 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f20d4038f70 0x7f20d403b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:48:25.438 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:24.248+0000 7f20e9600700 1 -- 172.21.15.107:0/4053185135 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f20e004d400 con 0x7f20ec107770 2024-03-08T20:48:25.438 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:24.248+0000 7f20ebe00700 1 --2- 172.21.15.107:0/4053185135 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f20d4038f70 0x7f20d403b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:48:25.438 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:24.249+0000 7f20ebe00700 1 --2- 172.21.15.107:0/4053185135 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f20d4038f70 0x7f20d403b430 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f20d8006fd0 tx=0x7f20d8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:48:25.438 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:24.254+0000 7f20e9600700 1 -- 172.21.15.107:0/4053185135 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f20e002ad60 con 0x7f20ec107770 2024-03-08T20:48:25.438 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:24.517+0000 7f20f3a00700 1 -- 172.21.15.107:0/4053185135 --> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f20ec19c2d0 con 0x7f20d4038f70 2024-03-08T20:48:25.438 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:24.767+0000 7f20e9600700 1 -- 172.21.15.107:0/4053185135 <== mgr.14152 v2:172.21.15.107:6800/3376636618 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+252 (secure 0 0 0) 0x7f20ec19c2d0 con 0x7f20d4038f70 2024-03-08T20:48:25.438 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:24.770+0000 7f20f3a00700 1 -- 172.21.15.107:0/4053185135 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f20d4038f70 msgr2=0x7f20d403b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:48:25.438 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:24.770+0000 7f20f3a00700 1 --2- 172.21.15.107:0/4053185135 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f20d4038f70 0x7f20d403b430 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f20d8006fd0 tx=0x7f20d8006e40 comp rx=0 tx=0).stop 2024-03-08T20:48:25.438 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:24.770+0000 7f20f3a00700 1 -- 172.21.15.107:0/4053185135 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f20ec107770 msgr2=0x7f20ec19b620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:48:25.438 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:24.770+0000 7f20f3a00700 1 --2- 172.21.15.107:0/4053185135 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f20ec107770 0x7f20ec19b620 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f20e0006b20 tx=0x7f20e0011ad0 comp rx=0 tx=0).stop 2024-03-08T20:48:25.438 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:24.770+0000 7f20f3a00700 1 -- 172.21.15.107:0/4053185135 shutdown_connections 2024-03-08T20:48:25.438 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:24.770+0000 7f20f3a00700 1 --2- 172.21.15.107:0/4053185135 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f20d4038f70 0x7f20d403b430 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:48:25.438 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:24.770+0000 7f20f3a00700 1 --2- 172.21.15.107:0/4053185135 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f20ec107770 0x7f20ec19b620 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:48:25.439 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:24.770+0000 7f20f3a00700 1 -- 172.21.15.107:0/4053185135 >> 172.21.15.107:0/4053185135 conn(0x7f20ec102dd0 msgr2=0x7f20ec103860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:48:25.439 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:24.770+0000 7f20f3a00700 1 -- 172.21.15.107:0/4053185135 shutdown_connections 2024-03-08T20:48:25.439 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:24.770+0000 7f20f3a00700 1 -- 172.21.15.107:0/4053185135 wait complete. 2024-03-08T20:48:25.439 INFO:teuthology.orchestra.run.smithi107.stdout:Fetching dashboard port number... 2024-03-08T20:48:26.040 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:48:25 smithi107 ceph-mon[31381]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-08T20:48:26.040 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:48:25 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:48:26.040 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:48:25 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:48:26.844 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:48:26 smithi107 ceph-mon[31381]: from='client.? 172.21.15.107:0/263836076' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-03-08T20:48:27.782 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout 8443 2024-03-08T20:48:27.783 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:26.046+0000 7f6310200700 1 Processor -- start 2024-03-08T20:48:27.783 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:26.047+0000 7f6310200700 1 -- start start 2024-03-08T20:48:27.783 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:26.047+0000 7f6310200700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6308106670 0x7f6308106a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:48:27.783 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:26.047+0000 7f6310200700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6308107060 con 0x7f6308106670 2024-03-08T20:48:27.783 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:26.047+0000 7f630d800700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6308106670 0x7f6308106a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:48:27.783 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:26.047+0000 7f630d800700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6308106670 0x7f6308106a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:59962/0 (socket says 172.21.15.107:59962) 2024-03-08T20:48:27.783 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:26.047+0000 7f630d800700 1 -- 172.21.15.107:0/1940731381 learned_addr learned my addr 172.21.15.107:0/1940731381 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T20:48:27.783 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:26.048+0000 7f630d800700 1 -- 172.21.15.107:0/1940731381 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6308107870 con 0x7f6308106670 2024-03-08T20:48:27.783 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:26.049+0000 7f630d800700 1 --2- 172.21.15.107:0/1940731381 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6308106670 0x7f6308106a90 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f62fc00bb40 tx=0x7f62fc00be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e30cd4463258e944 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:48:27.783 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:26.049+0000 7f6307e00700 1 -- 172.21.15.107:0/1940731381 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f62fc00dbe0 con 0x7f6308106670 2024-03-08T20:48:27.784 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:26.049+0000 7f6307e00700 1 -- 172.21.15.107:0/1940731381 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f62fc0044e0 con 0x7f6308106670 2024-03-08T20:48:27.784 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:26.049+0000 7f6307e00700 1 -- 172.21.15.107:0/1940731381 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f62fc010470 con 0x7f6308106670 2024-03-08T20:48:27.784 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:26.050+0000 7f6310200700 1 -- 172.21.15.107:0/1940731381 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6308106670 msgr2=0x7f6308106a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:48:27.784 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:26.050+0000 7f6310200700 1 --2- 172.21.15.107:0/1940731381 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6308106670 0x7f6308106a90 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f62fc00bb40 tx=0x7f62fc00be50 comp rx=0 tx=0).stop 2024-03-08T20:48:27.784 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:26.050+0000 7f6310200700 1 -- 172.21.15.107:0/1940731381 shutdown_connections 2024-03-08T20:48:27.784 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:26.050+0000 7f6310200700 1 --2- 172.21.15.107:0/1940731381 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6308106670 0x7f6308106a90 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:48:27.784 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:26.050+0000 7f6310200700 1 -- 172.21.15.107:0/1940731381 >> 172.21.15.107:0/1940731381 conn(0x7f6308101c90 msgr2=0x7f63081040b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:48:27.784 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:26.050+0000 7f6310200700 1 -- 172.21.15.107:0/1940731381 shutdown_connections 2024-03-08T20:48:27.784 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:26.050+0000 7f6310200700 1 -- 172.21.15.107:0/1940731381 wait complete. 2024-03-08T20:48:27.784 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:26.051+0000 7f6310200700 1 Processor -- start 2024-03-08T20:48:27.784 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:26.052+0000 7f6310200700 1 -- start start 2024-03-08T20:48:27.784 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:26.052+0000 7f6310200700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6308106670 0x7f6308199450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:48:27.784 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:26.052+0000 7f6310200700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6308079ff0 con 0x7f6308106670 2024-03-08T20:48:27.785 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:26.052+0000 7f630d800700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6308106670 0x7f6308199450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:48:27.785 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:26.052+0000 7f630d800700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6308106670 0x7f6308199450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:59974/0 (socket says 172.21.15.107:59974) 2024-03-08T20:48:27.785 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:26.052+0000 7f630d800700 1 -- 172.21.15.107:0/263836076 learned_addr learned my addr 172.21.15.107:0/263836076 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T20:48:27.785 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:26.053+0000 7f630d800700 1 -- 172.21.15.107:0/263836076 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6308199990 con 0x7f6308106670 2024-03-08T20:48:27.785 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:26.053+0000 7f630d800700 1 --2- 172.21.15.107:0/263836076 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6308106670 0x7f6308199450 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f62fc003bf0 tx=0x7f62fc003fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:48:27.785 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:26.053+0000 7f6306000700 1 -- 172.21.15.107:0/263836076 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f62fc027070 con 0x7f6308106670 2024-03-08T20:48:27.785 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:26.053+0000 7f6306000700 1 -- 172.21.15.107:0/263836076 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f62fc010bd0 con 0x7f6308106670 2024-03-08T20:48:27.785 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:26.053+0000 7f6310200700 1 -- 172.21.15.107:0/263836076 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6308199c20 con 0x7f6308106670 2024-03-08T20:48:27.785 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:26.053+0000 7f6306000700 1 -- 172.21.15.107:0/263836076 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f62fc022b30 con 0x7f6308106670 2024-03-08T20:48:27.785 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:26.054+0000 7f6310200700 1 -- 172.21.15.107:0/263836076 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f630819a080 con 0x7f6308106670 2024-03-08T20:48:27.785 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:26.055+0000 7f6310200700 1 -- 172.21.15.107:0/263836076 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6308063410 con 0x7f6308106670 2024-03-08T20:48:27.785 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:26.055+0000 7f6306000700 1 -- 172.21.15.107:0/263836076 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 12) v1 ==== 46119+0+0 (secure 0 0 0) 0x7f62fc02a070 con 0x7f6308106670 2024-03-08T20:48:27.785 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:26.056+0000 7f6306000700 1 --2- 172.21.15.107:0/263836076 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f62f0038f70 0x7f62f003b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:48:27.785 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:26.056+0000 7f6306000700 1 -- 172.21.15.107:0/263836076 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f62fc01f070 con 0x7f6308106670 2024-03-08T20:48:27.786 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:26.056+0000 7f630ce00700 1 --2- 172.21.15.107:0/263836076 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f62f0038f70 0x7f62f003b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:48:27.786 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:26.057+0000 7f630ce00700 1 --2- 172.21.15.107:0/263836076 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f62f0038f70 0x7f62f003b430 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f62f4006fd0 tx=0x7f62f4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:48:27.786 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:26.061+0000 7f6306000700 1 -- 172.21.15.107:0/263836076 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f62fc015070 con 0x7f6308106670 2024-03-08T20:48:27.786 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:26.293+0000 7f6310200700 1 -- 172.21.15.107:0/263836076 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) v1 -- 0x7f63081929b0 con 0x7f6308106670 2024-03-08T20:48:27.786 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:26.294+0000 7f6306000700 1 -- 172.21.15.107:0/263836076 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) v1 ==== 112+0+5 (secure 0 0 0) 0x7f63081929b0 con 0x7f6308106670 2024-03-08T20:48:27.786 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:26.297+0000 7f6310200700 1 -- 172.21.15.107:0/263836076 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f62f0038f70 msgr2=0x7f62f003b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:48:27.786 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:26.297+0000 7f6310200700 1 --2- 172.21.15.107:0/263836076 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f62f0038f70 0x7f62f003b430 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f62f4006fd0 tx=0x7f62f4006e40 comp rx=0 tx=0).stop 2024-03-08T20:48:27.786 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:26.297+0000 7f6310200700 1 -- 172.21.15.107:0/263836076 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6308106670 msgr2=0x7f6308199450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:48:27.786 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:26.297+0000 7f6310200700 1 --2- 172.21.15.107:0/263836076 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6308106670 0x7f6308199450 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f62fc003bf0 tx=0x7f62fc003fb0 comp rx=0 tx=0).stop 2024-03-08T20:48:27.786 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:26.297+0000 7f6310200700 1 -- 172.21.15.107:0/263836076 shutdown_connections 2024-03-08T20:48:27.786 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:26.297+0000 7f6310200700 1 --2- 172.21.15.107:0/263836076 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f62f0038f70 0x7f62f003b430 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:48:27.786 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:26.297+0000 7f6310200700 1 --2- 172.21.15.107:0/263836076 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6308106670 0x7f6308199450 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:48:27.787 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:26.297+0000 7f6310200700 1 -- 172.21.15.107:0/263836076 >> 172.21.15.107:0/263836076 conn(0x7f6308101c90 msgr2=0x7f630818fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:48:27.787 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:26.297+0000 7f6310200700 1 -- 172.21.15.107:0/263836076 shutdown_connections 2024-03-08T20:48:27.787 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:26.297+0000 7f6310200700 1 -- 172.21.15.107:0/263836076 wait complete. 2024-03-08T20:48:27.813 INFO:teuthology.orchestra.run.smithi107.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-08T20:48:27.813 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stdout disabled 2024-03-08T20:48:27.838 INFO:teuthology.orchestra.run.smithi107.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-08T20:48:27.839 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stdout inactive 2024-03-08T20:48:27.839 INFO:teuthology.orchestra.run.smithi107.stdout:firewalld.service is not enabled 2024-03-08T20:48:27.839 INFO:teuthology.orchestra.run.smithi107.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-03-08T20:48:27.845 INFO:teuthology.orchestra.run.smithi107.stdout:Ceph Dashboard is now available at: 2024-03-08T20:48:27.846 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-08T20:48:27.846 INFO:teuthology.orchestra.run.smithi107.stdout: URL: https://smithi107.front.sepia.ceph.com:8443/ 2024-03-08T20:48:27.846 INFO:teuthology.orchestra.run.smithi107.stdout: User: admin 2024-03-08T20:48:27.846 INFO:teuthology.orchestra.run.smithi107.stdout: Password: 6rx93eiewx 2024-03-08T20:48:27.846 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-08T20:48:27.847 INFO:teuthology.orchestra.run.smithi107.stdout:Saving cluster configuration to /var/lib/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/config directory 2024-03-08T20:48:27.849 INFO:teuthology.orchestra.run.smithi107.stdout:Enabling autotune for osd_memory_target 2024-03-08T20:48:28.053 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:48:27 smithi107 ceph-mon[31381]: mgrmap e13: x(active, since 6s) 2024-03-08T20:48:29.344 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:28.517+0000 7f3a97a00700 1 Processor -- start 2024-03-08T20:48:29.344 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:28.518+0000 7f3a97a00700 1 -- start start 2024-03-08T20:48:29.344 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:28.518+0000 7f3a97a00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3a90109980 0x7f3a90109da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:48:29.345 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:28.518+0000 7f3a97a00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a9010a370 con 0x7f3a90109980 2024-03-08T20:48:29.345 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:28.519+0000 7f3a95000700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3a90109980 0x7f3a90109da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:48:29.345 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:28.519+0000 7f3a95000700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3a90109980 0x7f3a90109da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:59976/0 (socket says 172.21.15.107:59976) 2024-03-08T20:48:29.345 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:28.519+0000 7f3a95000700 1 -- 172.21.15.107:0/1793436410 learned_addr learned my addr 172.21.15.107:0/1793436410 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T20:48:29.345 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:28.519+0000 7f3a95000700 1 -- 172.21.15.107:0/1793436410 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a9010ab30 con 0x7f3a90109980 2024-03-08T20:48:29.345 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:28.519+0000 7f3a95000700 1 --2- 172.21.15.107:0/1793436410 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3a90109980 0x7f3a90109da0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f3a840098a0 tx=0x7f3a84009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f7939e921ad9bef server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:48:29.345 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:28.520+0000 7f3a8f400700 1 -- 172.21.15.107:0/1793436410 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3a84004160 con 0x7f3a90109980 2024-03-08T20:48:29.345 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:28.520+0000 7f3a8f400700 1 -- 172.21.15.107:0/1793436410 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f3a84004300 con 0x7f3a90109980 2024-03-08T20:48:29.350 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:28.520+0000 7f3a8f400700 1 -- 172.21.15.107:0/1793436410 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3a84011a50 con 0x7f3a90109980 2024-03-08T20:48:29.350 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:28.521+0000 7f3a97a00700 1 -- 172.21.15.107:0/1793436410 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3a90109980 msgr2=0x7f3a90109da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:48:29.350 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:28.521+0000 7f3a97a00700 1 --2- 172.21.15.107:0/1793436410 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3a90109980 0x7f3a90109da0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f3a840098a0 tx=0x7f3a84009bb0 comp rx=0 tx=0).stop 2024-03-08T20:48:29.350 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:28.521+0000 7f3a97a00700 1 -- 172.21.15.107:0/1793436410 shutdown_connections 2024-03-08T20:48:29.350 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:28.521+0000 7f3a97a00700 1 --2- 172.21.15.107:0/1793436410 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3a90109980 0x7f3a90109da0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:48:29.350 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:28.521+0000 7f3a97a00700 1 -- 172.21.15.107:0/1793436410 >> 172.21.15.107:0/1793436410 conn(0x7f3a9007cdd0 msgr2=0x7f3a901074f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:48:29.350 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:28.521+0000 7f3a97a00700 1 -- 172.21.15.107:0/1793436410 shutdown_connections 2024-03-08T20:48:29.350 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:28.521+0000 7f3a97a00700 1 -- 172.21.15.107:0/1793436410 wait complete. 2024-03-08T20:48:29.350 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:28.522+0000 7f3a97a00700 1 Processor -- start 2024-03-08T20:48:29.350 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:28.522+0000 7f3a97a00700 1 -- start start 2024-03-08T20:48:29.351 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:28.522+0000 7f3a97a00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3a90109980 0x7f3a9019d800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:48:29.351 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:28.522+0000 7f3a97a00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a9010dee0 con 0x7f3a90109980 2024-03-08T20:48:29.351 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:28.522+0000 7f3a95000700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3a90109980 0x7f3a9019d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:48:29.351 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:28.523+0000 7f3a95000700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3a90109980 0x7f3a9019d800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:59988/0 (socket says 172.21.15.107:59988) 2024-03-08T20:48:29.351 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:28.523+0000 7f3a95000700 1 -- 172.21.15.107:0/3942520564 learned_addr learned my addr 172.21.15.107:0/3942520564 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T20:48:29.351 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:28.523+0000 7f3a95000700 1 -- 172.21.15.107:0/3942520564 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a9019dd40 con 0x7f3a90109980 2024-03-08T20:48:29.351 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:28.523+0000 7f3a95000700 1 --2- 172.21.15.107:0/3942520564 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3a90109980 0x7f3a9019d800 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f3a84006b20 tx=0x7f3a84011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:48:29.351 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:28.523+0000 7f3a8d600700 1 -- 172.21.15.107:0/3942520564 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3a840039e0 con 0x7f3a90109980 2024-03-08T20:48:29.351 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:28.523+0000 7f3a8d600700 1 -- 172.21.15.107:0/3942520564 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f3a8401a8e0 con 0x7f3a90109980 2024-03-08T20:48:29.351 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:28.523+0000 7f3a97a00700 1 -- 172.21.15.107:0/3942520564 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a9019dfd0 con 0x7f3a90109980 2024-03-08T20:48:29.351 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:28.523+0000 7f3a8d600700 1 -- 172.21.15.107:0/3942520564 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3a840247e0 con 0x7f3a90109980 2024-03-08T20:48:29.351 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:28.524+0000 7f3a97a00700 1 -- 172.21.15.107:0/3942520564 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a901a0cc0 con 0x7f3a90109980 2024-03-08T20:48:29.352 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:28.525+0000 7f3a8d600700 1 -- 172.21.15.107:0/3942520564 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 13) v1 ==== 46163+0+0 (secure 0 0 0) 0x7f3a84022020 con 0x7f3a90109980 2024-03-08T20:48:29.352 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:28.525+0000 7f3a97a00700 1 -- 172.21.15.107:0/3942520564 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a90063410 con 0x7f3a90109980 2024-03-08T20:48:29.352 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:28.525+0000 7f3a8d600700 1 --2- 172.21.15.107:0/3942520564 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f3a78039060 0x7f3a7803b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:48:29.352 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:28.525+0000 7f3a8d600700 1 -- 172.21.15.107:0/3942520564 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f3a8404d400 con 0x7f3a90109980 2024-03-08T20:48:29.352 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:28.525+0000 7f3a8fe00700 1 --2- 172.21.15.107:0/3942520564 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f3a78039060 0x7f3a7803b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:48:29.352 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:28.526+0000 7f3a8fe00700 1 --2- 172.21.15.107:0/3942520564 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f3a78039060 0x7f3a7803b520 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f3a7c006fd0 tx=0x7f3a7c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:48:29.352 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:28.530+0000 7f3a8d600700 1 -- 172.21.15.107:0/3942520564 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f3a84017ca0 con 0x7f3a90109980 2024-03-08T20:48:29.352 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:28.752+0000 7f3a97a00700 1 -- 172.21.15.107:0/3942520564 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) v1 -- 0x7f3a901a0e90 con 0x7f3a90109980 2024-03-08T20:48:29.352 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:28.753+0000 7f3a8d600700 1 -- 172.21.15.107:0/3942520564 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) v1 ==== 127+0+0 (secure 0 0 0) 0x7f3a8402d3f0 con 0x7f3a90109980 2024-03-08T20:48:29.352 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:28.756+0000 7f3a97a00700 1 -- 172.21.15.107:0/3942520564 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f3a78039060 msgr2=0x7f3a7803b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:48:29.352 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:28.756+0000 7f3a97a00700 1 --2- 172.21.15.107:0/3942520564 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f3a78039060 0x7f3a7803b520 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f3a7c006fd0 tx=0x7f3a7c006e40 comp rx=0 tx=0).stop 2024-03-08T20:48:29.352 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:28.756+0000 7f3a97a00700 1 -- 172.21.15.107:0/3942520564 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3a90109980 msgr2=0x7f3a9019d800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:48:29.353 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:28.756+0000 7f3a97a00700 1 --2- 172.21.15.107:0/3942520564 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3a90109980 0x7f3a9019d800 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f3a84006b20 tx=0x7f3a84011ad0 comp rx=0 tx=0).stop 2024-03-08T20:48:29.353 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:28.756+0000 7f3a97a00700 1 -- 172.21.15.107:0/3942520564 shutdown_connections 2024-03-08T20:48:29.353 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:28.756+0000 7f3a97a00700 1 --2- 172.21.15.107:0/3942520564 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f3a78039060 0x7f3a7803b520 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:48:29.353 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:28.756+0000 7f3a97a00700 1 --2- 172.21.15.107:0/3942520564 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3a90109980 0x7f3a9019d800 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:48:29.353 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:28.756+0000 7f3a97a00700 1 -- 172.21.15.107:0/3942520564 >> 172.21.15.107:0/3942520564 conn(0x7f3a9007cdd0 msgr2=0x7f3a90106bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:48:29.353 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:28.757+0000 7f3a97a00700 1 -- 172.21.15.107:0/3942520564 shutdown_connections 2024-03-08T20:48:29.353 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:28.757+0000 7f3a97a00700 1 -- 172.21.15.107:0/3942520564 wait complete. 2024-03-08T20:48:31.050 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:30.030+0000 7eff4bc00700 1 Processor -- start 2024-03-08T20:48:31.050 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:30.031+0000 7eff4bc00700 1 -- start start 2024-03-08T20:48:31.050 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:30.031+0000 7eff4bc00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7eff4407b360 0x7eff4407b780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:48:31.050 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:30.031+0000 7eff4bc00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff4407d040 con 0x7eff4407b360 2024-03-08T20:48:31.050 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:30.031+0000 7eff49200700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7eff4407b360 0x7eff4407b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:48:31.050 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:30.031+0000 7eff49200700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7eff4407b360 0x7eff4407b780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:59990/0 (socket says 172.21.15.107:59990) 2024-03-08T20:48:31.050 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:30.031+0000 7eff49200700 1 -- 172.21.15.107:0/3711844057 learned_addr learned my addr 172.21.15.107:0/3711844057 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T20:48:31.050 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:30.032+0000 7eff49200700 1 -- 172.21.15.107:0/3711844057 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff4407bcc0 con 0x7eff4407b360 2024-03-08T20:48:31.050 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:30.032+0000 7eff49200700 1 --2- 172.21.15.107:0/3711844057 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7eff4407b360 0x7eff4407b780 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7eff34009e80 tx=0x7eff3400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24ff38914bb93629 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:48:31.050 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:30.032+0000 7eff43e00700 1 -- 172.21.15.107:0/3711844057 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7eff34017070 con 0x7eff4407b360 2024-03-08T20:48:31.051 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:30.032+0000 7eff43e00700 1 -- 172.21.15.107:0/3711844057 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7eff3400b8c0 con 0x7eff4407b360 2024-03-08T20:48:31.051 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:30.032+0000 7eff43e00700 1 -- 172.21.15.107:0/3711844057 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7eff34004490 con 0x7eff4407b360 2024-03-08T20:48:31.051 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:30.033+0000 7eff4bc00700 1 -- 172.21.15.107:0/3711844057 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7eff4407b360 msgr2=0x7eff4407b780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:48:31.051 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:30.033+0000 7eff4bc00700 1 --2- 172.21.15.107:0/3711844057 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7eff4407b360 0x7eff4407b780 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7eff34009e80 tx=0x7eff3400b330 comp rx=0 tx=0).stop 2024-03-08T20:48:31.051 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:30.033+0000 7eff4bc00700 1 -- 172.21.15.107:0/3711844057 shutdown_connections 2024-03-08T20:48:31.051 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:30.033+0000 7eff4bc00700 1 --2- 172.21.15.107:0/3711844057 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7eff4407b360 0x7eff4407b780 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:48:31.051 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:30.033+0000 7eff4bc00700 1 -- 172.21.15.107:0/3711844057 >> 172.21.15.107:0/3711844057 conn(0x7eff44102da0 msgr2=0x7eff441051c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:48:31.051 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:30.033+0000 7eff4bc00700 1 -- 172.21.15.107:0/3711844057 shutdown_connections 2024-03-08T20:48:31.051 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:30.033+0000 7eff4bc00700 1 -- 172.21.15.107:0/3711844057 wait complete. 2024-03-08T20:48:31.051 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:30.034+0000 7eff4bc00700 1 Processor -- start 2024-03-08T20:48:31.051 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:30.035+0000 7eff4bc00700 1 -- start start 2024-03-08T20:48:31.051 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:30.035+0000 7eff4bc00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7eff4407b360 0x7eff441a1d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:48:31.051 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:30.035+0000 7eff4bc00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff44109a10 con 0x7eff4407b360 2024-03-08T20:48:31.052 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:30.035+0000 7eff49200700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7eff4407b360 0x7eff441a1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:48:31.052 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:30.035+0000 7eff49200700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7eff4407b360 0x7eff441a1d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:60004/0 (socket says 172.21.15.107:60004) 2024-03-08T20:48:31.052 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:30.035+0000 7eff49200700 1 -- 172.21.15.107:0/3398095888 learned_addr learned my addr 172.21.15.107:0/3398095888 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T20:48:31.052 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:30.035+0000 7eff49200700 1 -- 172.21.15.107:0/3398095888 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff441a2290 con 0x7eff4407b360 2024-03-08T20:48:31.052 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:30.036+0000 7eff49200700 1 --2- 172.21.15.107:0/3398095888 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7eff4407b360 0x7eff441a1d50 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7eff34000f80 tx=0x7eff34003d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:48:31.052 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:30.036+0000 7eff42000700 1 -- 172.21.15.107:0/3398095888 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7eff34020020 con 0x7eff4407b360 2024-03-08T20:48:31.052 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:30.036+0000 7eff42000700 1 -- 172.21.15.107:0/3398095888 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7eff3400bba0 con 0x7eff4407b360 2024-03-08T20:48:31.052 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:30.036+0000 7eff4bc00700 1 -- 172.21.15.107:0/3398095888 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff441a2520 con 0x7eff4407b360 2024-03-08T20:48:31.052 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:30.036+0000 7eff42000700 1 -- 172.21.15.107:0/3398095888 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7eff34017040 con 0x7eff4407b360 2024-03-08T20:48:31.052 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:30.036+0000 7eff4bc00700 1 -- 172.21.15.107:0/3398095888 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff441a5210 con 0x7eff4407b360 2024-03-08T20:48:31.052 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:30.037+0000 7eff4bc00700 1 -- 172.21.15.107:0/3398095888 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff44063410 con 0x7eff4407b360 2024-03-08T20:48:31.052 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:30.037+0000 7eff42000700 1 -- 172.21.15.107:0/3398095888 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 13) v1 ==== 46163+0+0 (secure 0 0 0) 0x7eff34023020 con 0x7eff4407b360 2024-03-08T20:48:31.052 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:30.038+0000 7eff42000700 1 --2- 172.21.15.107:0/3398095888 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7eff28039070 0x7eff2803b530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:48:31.053 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:30.038+0000 7eff42000700 1 -- 172.21.15.107:0/3398095888 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7eff3400f050 con 0x7eff4407b360 2024-03-08T20:48:31.053 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:30.038+0000 7eff3be00700 1 --2- 172.21.15.107:0/3398095888 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7eff28039070 0x7eff2803b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:48:31.053 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:30.039+0000 7eff3be00700 1 --2- 172.21.15.107:0/3398095888 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7eff28039070 0x7eff2803b530 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7eff2c006fd0 tx=0x7eff2c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:48:31.053 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:30.043+0000 7eff42000700 1 -- 172.21.15.107:0/3398095888 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7eff3402a020 con 0x7eff4407b360 2024-03-08T20:48:31.053 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:30.377+0000 7eff4bc00700 1 -- 172.21.15.107:0/3398095888 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) v1 -- 0x7eff4402d8d0 con 0x7eff4407b360 2024-03-08T20:48:31.053 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:30.383+0000 7eff42000700 1 -- 172.21.15.107:0/3398095888 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) v1 ==== 153+0+0 (secure 0 0 0) 0x7eff3402c300 con 0x7eff4407b360 2024-03-08T20:48:31.053 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:30.385+0000 7eff4bc00700 1 -- 172.21.15.107:0/3398095888 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7eff28039070 msgr2=0x7eff2803b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:48:31.053 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:30.385+0000 7eff4bc00700 1 --2- 172.21.15.107:0/3398095888 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7eff28039070 0x7eff2803b530 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7eff2c006fd0 tx=0x7eff2c006e40 comp rx=0 tx=0).stop 2024-03-08T20:48:31.053 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:30.386+0000 7eff4bc00700 1 -- 172.21.15.107:0/3398095888 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7eff4407b360 msgr2=0x7eff441a1d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:48:31.053 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:30.386+0000 7eff4bc00700 1 --2- 172.21.15.107:0/3398095888 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7eff4407b360 0x7eff441a1d50 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7eff34000f80 tx=0x7eff34003d70 comp rx=0 tx=0).stop 2024-03-08T20:48:31.053 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:30.386+0000 7eff4bc00700 1 -- 172.21.15.107:0/3398095888 shutdown_connections 2024-03-08T20:48:31.053 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:30.386+0000 7eff4bc00700 1 --2- 172.21.15.107:0/3398095888 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7eff28039070 0x7eff2803b530 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:48:31.053 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:30.386+0000 7eff4bc00700 1 --2- 172.21.15.107:0/3398095888 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7eff4407b360 0x7eff441a1d50 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:48:31.054 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:30.386+0000 7eff4bc00700 1 -- 172.21.15.107:0/3398095888 >> 172.21.15.107:0/3398095888 conn(0x7eff44102da0 msgr2=0x7eff44198a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:48:31.054 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:30.386+0000 7eff4bc00700 1 -- 172.21.15.107:0/3398095888 shutdown_connections 2024-03-08T20:48:31.054 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-03-08T20:48:30.386+0000 7eff4bc00700 1 -- 172.21.15.107:0/3398095888 wait complete. 2024-03-08T20:48:31.054 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-03-08T20:48:31.054 INFO:teuthology.orchestra.run.smithi107.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-03-08T20:48:31.054 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-08T20:48:31.054 INFO:teuthology.orchestra.run.smithi107.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid dc20f0f8-dd8c-11ee-95c6-87774f69a715 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-03-08T20:48:31.054 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-08T20:48:31.054 INFO:teuthology.orchestra.run.smithi107.stdout:Or, if you are only running a single cluster on this host: 2024-03-08T20:48:31.054 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-08T20:48:31.054 INFO:teuthology.orchestra.run.smithi107.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-03-08T20:48:31.054 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-08T20:48:31.054 INFO:teuthology.orchestra.run.smithi107.stdout:Please consider enabling telemetry to help improve Ceph: 2024-03-08T20:48:31.054 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-08T20:48:31.055 INFO:teuthology.orchestra.run.smithi107.stdout: ceph telemetry on 2024-03-08T20:48:31.055 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-08T20:48:31.055 INFO:teuthology.orchestra.run.smithi107.stdout:For more information see: 2024-03-08T20:48:31.055 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-08T20:48:31.055 INFO:teuthology.orchestra.run.smithi107.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-03-08T20:48:31.055 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-08T20:48:31.055 INFO:teuthology.orchestra.run.smithi107.stdout:Bootstrap complete. 2024-03-08T20:48:31.171 INFO:tasks.cephadm:Fetching config... 2024-03-08T20:48:31.171 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-03-08T20:48:31.172 DEBUG:teuthology.orchestra.run.smithi107:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-03-08T20:48:31.226 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-03-08T20:48:31.226 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-03-08T20:48:31.226 DEBUG:teuthology.orchestra.run.smithi107:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-03-08T20:48:31.306 INFO:tasks.cephadm:Fetching mon keyring... 2024-03-08T20:48:31.306 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-03-08T20:48:31.306 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd if=/var/lib/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/mon.a/keyring of=/dev/stdout 2024-03-08T20:48:31.449 INFO:tasks.cephadm:Fetching pub ssh key... 2024-03-08T20:48:31.449 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-03-08T20:48:31.450 DEBUG:teuthology.orchestra.run.smithi107:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-03-08T20:48:31.548 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-03-08T20:48:31.548 DEBUG:teuthology.orchestra.run.smithi107:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpSCnHfXT8N65+lAbF0tgR7lX8z7143TQ6tzGE7sZGK4PwBUbhtdrDAxlVOYdfjeZGKeP96nxBvR81AA5J+H2JHkdx/t4LTAZ7HOvSM22Khg1HZerSDFYbI1QxcO41edXWx2E4IyynZ6DxFCmHSptTTnIbNDFi5TcLWjh5nKBWqGsfjNXDZcxYrbdc9Iq3AtFj0zk9XLyzreBFSlq5ykKVuaE+XryB8qIkGYIW4jo5ApNP8VRi00cNBiPDzcQDbzyYmazSMDpEq0Eg1TZyGHI7y/iJxQ1IiP+s14mtDk5FpKgvbfKJvl1AQTYZEqEQKUieZ17g0Ykiijx8Z9P2DcKTwl1xKz7N2SAD0OmO6/f2TgtQQTTi3UZxyJc3om2DJSQfguoWau6itWFl4sWre3A6fWfXjaVLQGTIA1gbPTn/4QpvI39vFk+cLWEC0P7GlYiuxzYSpQPJbEs48ZvFzUvUugoMh4o8HoQAkZL5fqTOUxodpNaireU770p3YXGOyxk= ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-03-08T20:48:31.614 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:48:31 smithi107 ceph-mon[31381]: from='client.? 172.21.15.107:0/3398095888' entity='client.admin' 2024-03-08T20:48:31.661 INFO:teuthology.orchestra.run.smithi107.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpSCnHfXT8N65+lAbF0tgR7lX8z7143TQ6tzGE7sZGK4PwBUbhtdrDAxlVOYdfjeZGKeP96nxBvR81AA5J+H2JHkdx/t4LTAZ7HOvSM22Khg1HZerSDFYbI1QxcO41edXWx2E4IyynZ6DxFCmHSptTTnIbNDFi5TcLWjh5nKBWqGsfjNXDZcxYrbdc9Iq3AtFj0zk9XLyzreBFSlq5ykKVuaE+XryB8qIkGYIW4jo5ApNP8VRi00cNBiPDzcQDbzyYmazSMDpEq0Eg1TZyGHI7y/iJxQ1IiP+s14mtDk5FpKgvbfKJvl1AQTYZEqEQKUieZ17g0Ykiijx8Z9P2DcKTwl1xKz7N2SAD0OmO6/f2TgtQQTTi3UZxyJc3om2DJSQfguoWau6itWFl4sWre3A6fWfXjaVLQGTIA1gbPTn/4QpvI39vFk+cLWEC0P7GlYiuxzYSpQPJbEs48ZvFzUvUugoMh4o8HoQAkZL5fqTOUxodpNaireU770p3YXGOyxk= ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715 2024-03-08T20:48:31.714 DEBUG:teuthology.orchestra.run.smithi149:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpSCnHfXT8N65+lAbF0tgR7lX8z7143TQ6tzGE7sZGK4PwBUbhtdrDAxlVOYdfjeZGKeP96nxBvR81AA5J+H2JHkdx/t4LTAZ7HOvSM22Khg1HZerSDFYbI1QxcO41edXWx2E4IyynZ6DxFCmHSptTTnIbNDFi5TcLWjh5nKBWqGsfjNXDZcxYrbdc9Iq3AtFj0zk9XLyzreBFSlq5ykKVuaE+XryB8qIkGYIW4jo5ApNP8VRi00cNBiPDzcQDbzyYmazSMDpEq0Eg1TZyGHI7y/iJxQ1IiP+s14mtDk5FpKgvbfKJvl1AQTYZEqEQKUieZ17g0Ykiijx8Z9P2DcKTwl1xKz7N2SAD0OmO6/f2TgtQQTTi3UZxyJc3om2DJSQfguoWau6itWFl4sWre3A6fWfXjaVLQGTIA1gbPTn/4QpvI39vFk+cLWEC0P7GlYiuxzYSpQPJbEs48ZvFzUvUugoMh4o8HoQAkZL5fqTOUxodpNaireU770p3YXGOyxk= ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-03-08T20:48:31.847 INFO:teuthology.orchestra.run.smithi149.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpSCnHfXT8N65+lAbF0tgR7lX8z7143TQ6tzGE7sZGK4PwBUbhtdrDAxlVOYdfjeZGKeP96nxBvR81AA5J+H2JHkdx/t4LTAZ7HOvSM22Khg1HZerSDFYbI1QxcO41edXWx2E4IyynZ6DxFCmHSptTTnIbNDFi5TcLWjh5nKBWqGsfjNXDZcxYrbdc9Iq3AtFj0zk9XLyzreBFSlq5ykKVuaE+XryB8qIkGYIW4jo5ApNP8VRi00cNBiPDzcQDbzyYmazSMDpEq0Eg1TZyGHI7y/iJxQ1IiP+s14mtDk5FpKgvbfKJvl1AQTYZEqEQKUieZ17g0Ykiijx8Z9P2DcKTwl1xKz7N2SAD0OmO6/f2TgtQQTTi3UZxyJc3om2DJSQfguoWau6itWFl4sWre3A6fWfXjaVLQGTIA1gbPTn/4QpvI39vFk+cLWEC0P7GlYiuxzYSpQPJbEs48ZvFzUvUugoMh4o8HoQAkZL5fqTOUxodpNaireU770p3YXGOyxk= ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715 2024-03-08T20:48:31.911 DEBUG:teuthology.orchestra.run.smithi204:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpSCnHfXT8N65+lAbF0tgR7lX8z7143TQ6tzGE7sZGK4PwBUbhtdrDAxlVOYdfjeZGKeP96nxBvR81AA5J+H2JHkdx/t4LTAZ7HOvSM22Khg1HZerSDFYbI1QxcO41edXWx2E4IyynZ6DxFCmHSptTTnIbNDFi5TcLWjh5nKBWqGsfjNXDZcxYrbdc9Iq3AtFj0zk9XLyzreBFSlq5ykKVuaE+XryB8qIkGYIW4jo5ApNP8VRi00cNBiPDzcQDbzyYmazSMDpEq0Eg1TZyGHI7y/iJxQ1IiP+s14mtDk5FpKgvbfKJvl1AQTYZEqEQKUieZ17g0Ykiijx8Z9P2DcKTwl1xKz7N2SAD0OmO6/f2TgtQQTTi3UZxyJc3om2DJSQfguoWau6itWFl4sWre3A6fWfXjaVLQGTIA1gbPTn/4QpvI39vFk+cLWEC0P7GlYiuxzYSpQPJbEs48ZvFzUvUugoMh4o8HoQAkZL5fqTOUxodpNaireU770p3YXGOyxk= ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-03-08T20:48:32.050 INFO:teuthology.orchestra.run.smithi204.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpSCnHfXT8N65+lAbF0tgR7lX8z7143TQ6tzGE7sZGK4PwBUbhtdrDAxlVOYdfjeZGKeP96nxBvR81AA5J+H2JHkdx/t4LTAZ7HOvSM22Khg1HZerSDFYbI1QxcO41edXWx2E4IyynZ6DxFCmHSptTTnIbNDFi5TcLWjh5nKBWqGsfjNXDZcxYrbdc9Iq3AtFj0zk9XLyzreBFSlq5ykKVuaE+XryB8qIkGYIW4jo5ApNP8VRi00cNBiPDzcQDbzyYmazSMDpEq0Eg1TZyGHI7y/iJxQ1IiP+s14mtDk5FpKgvbfKJvl1AQTYZEqEQKUieZ17g0Ykiijx8Z9P2DcKTwl1xKz7N2SAD0OmO6/f2TgtQQTTi3UZxyJc3om2DJSQfguoWau6itWFl4sWre3A6fWfXjaVLQGTIA1gbPTn/4QpvI39vFk+cLWEC0P7GlYiuxzYSpQPJbEs48ZvFzUvUugoMh4o8HoQAkZL5fqTOUxodpNaireU770p3YXGOyxk= ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715 2024-03-08T20:48:32.120 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dc20f0f8-dd8c-11ee-95c6-87774f69a715 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-03-08T20:48:32.532 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/mon.a/config 2024-03-08T20:48:35.261 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:48:35 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:48:35.261 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:48:35 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:48:35.261 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:48:35 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi107", "name": "osd_memory_target"}]: dispatch 2024-03-08T20:48:35.261 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:48:35 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:48:35.261 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:48:35 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T20:48:35.261 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:48:35 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:48:35.409 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:48:35.415+0000 7fdaa9c00700 1 -- 172.21.15.107:0/924629644 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdaa4075e10 msgr2=0x7fdaa4074270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:48:35.409 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:48:35.415+0000 7fdaa9c00700 1 --2- 172.21.15.107:0/924629644 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdaa4075e10 0x7fdaa4074270 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fda94009910 tx=0x7fda94009c20 comp rx=0 tx=0).stop 2024-03-08T20:48:35.409 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:48:35.415+0000 7fdaa9c00700 1 -- 172.21.15.107:0/924629644 shutdown_connections 2024-03-08T20:48:35.410 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:48:35.415+0000 7fdaa9c00700 1 --2- 172.21.15.107:0/924629644 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdaa4075e10 0x7fdaa4074270 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:48:35.410 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:48:35.415+0000 7fdaa9c00700 1 -- 172.21.15.107:0/924629644 >> 172.21.15.107:0/924629644 conn(0x7fdaa40fcda0 msgr2=0x7fdaa40ff1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:48:35.410 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:48:35.415+0000 7fdaa9c00700 1 -- 172.21.15.107:0/924629644 shutdown_connections 2024-03-08T20:48:35.410 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:48:35.415+0000 7fdaa9c00700 1 -- 172.21.15.107:0/924629644 wait complete. 2024-03-08T20:48:35.411 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:48:35.416+0000 7fdaa9c00700 1 Processor -- start 2024-03-08T20:48:35.411 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:48:35.416+0000 7fdaa9c00700 1 -- start start 2024-03-08T20:48:35.411 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:48:35.417+0000 7fdaa9c00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdaa4075e10 0x7fdaa419d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:48:35.411 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:48:35.417+0000 7fdaa9c00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdaa410dde0 con 0x7fdaa4075e10 2024-03-08T20:48:35.411 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:48:35.417+0000 7fdaa2a00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdaa4075e10 0x7fdaa419d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:48:35.412 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:48:35.417+0000 7fdaa2a00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdaa4075e10 0x7fdaa419d450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:60036/0 (socket says 172.21.15.107:60036) 2024-03-08T20:48:35.412 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:48:35.417+0000 7fdaa2a00700 1 -- 172.21.15.107:0/1046306349 learned_addr learned my addr 172.21.15.107:0/1046306349 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T20:48:35.412 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:48:35.417+0000 7fdaa2a00700 1 -- 172.21.15.107:0/1046306349 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdaa419d990 con 0x7fdaa4075e10 2024-03-08T20:48:35.412 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:48:35.418+0000 7fdaa2a00700 1 --2- 172.21.15.107:0/1046306349 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdaa4075e10 0x7fdaa419d450 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fda94005950 tx=0x7fda94004fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:48:35.412 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:48:35.418+0000 7fda9b400700 1 -- 172.21.15.107:0/1046306349 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fda94026070 con 0x7fdaa4075e10 2024-03-08T20:48:35.412 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:48:35.418+0000 7fda9b400700 1 -- 172.21.15.107:0/1046306349 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fda9400bc00 con 0x7fdaa4075e10 2024-03-08T20:48:35.412 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:48:35.418+0000 7fdaa9c00700 1 -- 172.21.15.107:0/1046306349 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdaa419dc20 con 0x7fdaa4075e10 2024-03-08T20:48:35.412 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:48:35.418+0000 7fda9b400700 1 -- 172.21.15.107:0/1046306349 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fda940039a0 con 0x7fdaa4075e10 2024-03-08T20:48:35.413 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:48:35.418+0000 7fdaa9c00700 1 -- 172.21.15.107:0/1046306349 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdaa41a0910 con 0x7fdaa4075e10 2024-03-08T20:48:35.413 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:48:35.419+0000 7fda9b400700 1 -- 172.21.15.107:0/1046306349 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 13) v1 ==== 46163+0+0 (secure 0 0 0) 0x7fda94029070 con 0x7fdaa4075e10 2024-03-08T20:48:35.414 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:48:35.419+0000 7fdaa9c00700 1 -- 172.21.15.107:0/1046306349 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdaa410e4c0 con 0x7fdaa4075e10 2024-03-08T20:48:35.414 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:48:35.420+0000 7fda9b400700 1 --2- 172.21.15.107:0/1046306349 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fda88039020 0x7fda8803b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:48:35.414 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:48:35.420+0000 7fda9b400700 1 -- 172.21.15.107:0/1046306349 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fda9401e070 con 0x7fdaa4075e10 2024-03-08T20:48:35.414 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:48:35.420+0000 7fdaa2000700 1 --2- 172.21.15.107:0/1046306349 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fda88039020 0x7fda8803b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:48:35.414 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:48:35.421+0000 7fdaa2000700 1 --2- 172.21.15.107:0/1046306349 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fda88039020 0x7fda8803b4e0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fda90006fd0 tx=0x7fda90006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:48:35.419 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:48:35.425+0000 7fda9b400700 1 -- 172.21.15.107:0/1046306349 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7fda94052020 con 0x7fdaa4075e10 2024-03-08T20:48:35.640 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:48:35.646+0000 7fdaa9c00700 1 -- 172.21.15.107:0/1046306349 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) v1 -- 0x7fdaa402d940 con 0x7fdaa4075e10 2024-03-08T20:48:35.644 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:48:35.650+0000 7fda9b400700 1 -- 172.21.15.107:0/1046306349 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) v1 ==== 125+0+0 (secure 0 0 0) 0x7fda94030020 con 0x7fdaa4075e10 2024-03-08T20:48:35.647 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:48:35.653+0000 7fdaa9c00700 1 -- 172.21.15.107:0/1046306349 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fda88039020 msgr2=0x7fda8803b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:48:35.647 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:48:35.653+0000 7fdaa9c00700 1 --2- 172.21.15.107:0/1046306349 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fda88039020 0x7fda8803b4e0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fda90006fd0 tx=0x7fda90006e40 comp rx=0 tx=0).stop 2024-03-08T20:48:35.647 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:48:35.653+0000 7fdaa9c00700 1 -- 172.21.15.107:0/1046306349 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdaa4075e10 msgr2=0x7fdaa419d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:48:35.647 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:48:35.653+0000 7fdaa9c00700 1 --2- 172.21.15.107:0/1046306349 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdaa4075e10 0x7fdaa419d450 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fda94005950 tx=0x7fda94004fa0 comp rx=0 tx=0).stop 2024-03-08T20:48:35.647 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:48:35.653+0000 7fdaa9c00700 1 -- 172.21.15.107:0/1046306349 shutdown_connections 2024-03-08T20:48:35.648 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:48:35.653+0000 7fdaa9c00700 1 --2- 172.21.15.107:0/1046306349 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fda88039020 0x7fda8803b4e0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:48:35.648 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:48:35.653+0000 7fdaa9c00700 1 --2- 172.21.15.107:0/1046306349 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdaa4075e10 0x7fdaa419d450 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:48:35.648 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:48:35.653+0000 7fdaa9c00700 1 -- 172.21.15.107:0/1046306349 >> 172.21.15.107:0/1046306349 conn(0x7fdaa40fcda0 msgr2=0x7fdaa40fd950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:48:35.648 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:48:35.654+0000 7fdaa9c00700 1 -- 172.21.15.107:0/1046306349 shutdown_connections 2024-03-08T20:48:35.648 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:48:35.654+0000 7fdaa9c00700 1 -- 172.21.15.107:0/1046306349 wait complete. 2024-03-08T20:48:36.374 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-03-08T20:48:36.374 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dc20f0f8-dd8c-11ee-95c6-87774f69a715 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-03-08T20:48:36.977 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:48:36 smithi107 ceph-mon[31381]: from='client.? 172.21.15.107:0/1046306349' entity='client.admin' 2024-03-08T20:48:36.977 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:48:36 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T20:48:36.978 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:48:36 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:48:37.160 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/mon.a/config 2024-03-08T20:48:39.161 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:48:39.167+0000 7f22f9200700 1 -- 172.21.15.107:0/369810153 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f22f4101820 msgr2=0x7f22f4101c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:48:39.162 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:48:39.167+0000 7f22f9200700 1 --2- 172.21.15.107:0/369810153 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f22f4101820 0x7f22f4101c40 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f22e4009910 tx=0x7f22e4009c20 comp rx=0 tx=0).stop 2024-03-08T20:48:39.162 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:48:39.167+0000 7f22f9200700 1 -- 172.21.15.107:0/369810153 shutdown_connections 2024-03-08T20:48:39.162 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:48:39.167+0000 7f22f9200700 1 --2- 172.21.15.107:0/369810153 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f22f4101820 0x7f22f4101c40 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:48:39.162 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:48:39.167+0000 7f22f9200700 1 -- 172.21.15.107:0/369810153 >> 172.21.15.107:0/369810153 conn(0x7f22f40fcdc0 msgr2=0x7f22f40ff200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:48:39.162 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:48:39.167+0000 7f22f9200700 1 -- 172.21.15.107:0/369810153 shutdown_connections 2024-03-08T20:48:39.162 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:48:39.167+0000 7f22f9200700 1 -- 172.21.15.107:0/369810153 wait complete. 2024-03-08T20:48:39.164 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:48:39.168+0000 7f22f9200700 1 Processor -- start 2024-03-08T20:48:39.164 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:48:39.168+0000 7f22f9200700 1 -- start start 2024-03-08T20:48:39.164 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:48:39.169+0000 7f22f9200700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f22f4101820 0x7f22f4199080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:48:39.164 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:48:39.169+0000 7f22f9200700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22f4107760 con 0x7f22f4101820 2024-03-08T20:48:39.164 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:48:39.169+0000 7f22f1e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f22f4101820 0x7f22f4199080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:48:39.164 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:48:39.169+0000 7f22f1e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f22f4101820 0x7f22f4199080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:51884/0 (socket says 172.21.15.107:51884) 2024-03-08T20:48:39.164 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:48:39.169+0000 7f22f1e00700 1 -- 172.21.15.107:0/2689736825 learned_addr learned my addr 172.21.15.107:0/2689736825 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T20:48:39.164 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:48:39.169+0000 7f22f1e00700 1 -- 172.21.15.107:0/2689736825 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22f41995c0 con 0x7f22f4101820 2024-03-08T20:48:39.164 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:48:39.170+0000 7f22f1e00700 1 --2- 172.21.15.107:0/2689736825 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f22f4101820 0x7f22f4199080 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f22e4005950 tx=0x7f22e4004fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:48:39.164 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:48:39.170+0000 7f22eaa00700 1 -- 172.21.15.107:0/2689736825 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f22e4026070 con 0x7f22f4101820 2024-03-08T20:48:39.165 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:48:39.170+0000 7f22eaa00700 1 -- 172.21.15.107:0/2689736825 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f22e400bc00 con 0x7f22f4101820 2024-03-08T20:48:39.165 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:48:39.170+0000 7f22eaa00700 1 -- 172.21.15.107:0/2689736825 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f22e40039a0 con 0x7f22f4101820 2024-03-08T20:48:39.165 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:48:39.170+0000 7f22f9200700 1 -- 172.21.15.107:0/2689736825 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22f4199850 con 0x7f22f4101820 2024-03-08T20:48:39.165 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:48:39.170+0000 7f22f9200700 1 -- 172.21.15.107:0/2689736825 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22f419c540 con 0x7f22f4101820 2024-03-08T20:48:39.166 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:48:39.171+0000 7f22f9200700 1 -- 172.21.15.107:0/2689736825 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22f4107ee0 con 0x7f22f4101820 2024-03-08T20:48:39.166 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:48:39.171+0000 7f22eaa00700 1 -- 172.21.15.107:0/2689736825 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 13) v1 ==== 46163+0+0 (secure 0 0 0) 0x7f22e4029070 con 0x7f22f4101820 2024-03-08T20:48:39.166 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:48:39.172+0000 7f22eaa00700 1 --2- 172.21.15.107:0/2689736825 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f22d8039020 0x7f22d803b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:48:39.166 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:48:39.172+0000 7f22eaa00700 1 -- 172.21.15.107:0/2689736825 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f22e401e070 con 0x7f22f4101820 2024-03-08T20:48:39.166 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:48:39.172+0000 7f22f1400700 1 --2- 172.21.15.107:0/2689736825 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f22d8039020 0x7f22d803b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:48:39.166 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:48:39.173+0000 7f22f1400700 1 --2- 172.21.15.107:0/2689736825 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f22d8039020 0x7f22d803b4e0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f22e0006fd0 tx=0x7f22e0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:48:39.171 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:48:39.177+0000 7f22eaa00700 1 -- 172.21.15.107:0/2689736825 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f22e4013070 con 0x7f22f4101820 2024-03-08T20:48:39.394 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:48:39.401+0000 7f22f9200700 1 -- 172.21.15.107:0/2689736825 --> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) v1 -- 0x7f22f402d940 con 0x7f22d8039020 2024-03-08T20:48:39.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:48:39.407+0000 7f22eaa00700 1 -- 172.21.15.107:0/2689736825 <== mgr.14152 v2:172.21.15.107:6800/3376636618 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f22f402d940 con 0x7f22d8039020 2024-03-08T20:48:39.403 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:48:39.410+0000 7f22f9200700 1 -- 172.21.15.107:0/2689736825 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f22d8039020 msgr2=0x7f22d803b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:48:39.403 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:48:39.410+0000 7f22f9200700 1 --2- 172.21.15.107:0/2689736825 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f22d8039020 0x7f22d803b4e0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f22e0006fd0 tx=0x7f22e0006e40 comp rx=0 tx=0).stop 2024-03-08T20:48:39.403 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:48:39.410+0000 7f22f9200700 1 -- 172.21.15.107:0/2689736825 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f22f4101820 msgr2=0x7f22f4199080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:48:39.403 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:48:39.410+0000 7f22f9200700 1 --2- 172.21.15.107:0/2689736825 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f22f4101820 0x7f22f4199080 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f22e4005950 tx=0x7f22e4004fa0 comp rx=0 tx=0).stop 2024-03-08T20:48:39.404 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:48:39.410+0000 7f22f9200700 1 -- 172.21.15.107:0/2689736825 shutdown_connections 2024-03-08T20:48:39.404 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:48:39.410+0000 7f22f9200700 1 --2- 172.21.15.107:0/2689736825 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f22d8039020 0x7f22d803b4e0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:48:39.404 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:48:39.410+0000 7f22f9200700 1 --2- 172.21.15.107:0/2689736825 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f22f4101820 0x7f22f4199080 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:48:39.404 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:48:39.410+0000 7f22f9200700 1 -- 172.21.15.107:0/2689736825 >> 172.21.15.107:0/2689736825 conn(0x7f22f40fcdc0 msgr2=0x7f22f40ff200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:48:39.405 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:48:39.410+0000 7f22f9200700 1 -- 172.21.15.107:0/2689736825 shutdown_connections 2024-03-08T20:48:39.405 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:48:39.410+0000 7f22f9200700 1 -- 172.21.15.107:0/2689736825 wait complete. 2024-03-08T20:48:40.220 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi149 2024-03-08T20:48:40.221 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-03-08T20:48:40.221 DEBUG:teuthology.orchestra.run.smithi149:> dd of=/etc/ceph/ceph.conf 2024-03-08T20:48:40.278 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-03-08T20:48:40.278 DEBUG:teuthology.orchestra.run.smithi149:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-08T20:48:40.372 INFO:tasks.cephadm:Adding host smithi149 to orchestrator... 2024-03-08T20:48:40.372 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dc20f0f8-dd8c-11ee-95c6-87774f69a715 -- ceph orch host add smithi149 2024-03-08T20:48:40.444 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:48:40 smithi107 ceph-mon[31381]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-03-08T20:48:40.444 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:48:40 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:48:40.444 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:48:40 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T20:48:40.444 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:48:40 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:48:40.445 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:48:40 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T20:48:40.786 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/mon.a/config 2024-03-08T20:48:41.641 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:48:41 smithi107 ceph-mon[31381]: Updating smithi107:/etc/ceph/ceph.conf 2024-03-08T20:48:41.641 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:48:41 smithi107 ceph-mon[31381]: Updating smithi107:/var/lib/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/config/ceph.conf 2024-03-08T20:48:41.642 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:48:41 smithi107 ceph-mon[31381]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T20:48:42.543 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:48:42 smithi107 ceph-mon[31381]: Updating smithi107:/etc/ceph/ceph.client.admin.keyring 2024-03-08T20:48:42.543 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:48:42 smithi107 ceph-mon[31381]: Updating smithi107:/var/lib/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/config/ceph.client.admin.keyring 2024-03-08T20:48:42.543 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:48:42 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:48:42.543 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:48:42 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:48:42.543 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:48:42 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:48:42.698 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:48:42.704+0000 7f914e400700 1 -- 172.21.15.107:0/226724514 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9148074210 msgr2=0x7f91480745f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:48:42.698 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:48:42.704+0000 7f914e400700 1 --2- 172.21.15.107:0/226724514 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9148074210 0x7f91480745f0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f9138009910 tx=0x7f9138009c20 comp rx=0 tx=0).stop 2024-03-08T20:48:42.699 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:48:42.704+0000 7f914e400700 1 -- 172.21.15.107:0/226724514 shutdown_connections 2024-03-08T20:48:42.699 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:48:42.704+0000 7f914e400700 1 --2- 172.21.15.107:0/226724514 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9148074210 0x7f91480745f0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:48:42.699 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:48:42.704+0000 7f914e400700 1 -- 172.21.15.107:0/226724514 >> 172.21.15.107:0/226724514 conn(0x7f91480fd2b0 msgr2=0x7f91480ff6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:48:42.699 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:48:42.705+0000 7f914e400700 1 -- 172.21.15.107:0/226724514 shutdown_connections 2024-03-08T20:48:42.699 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:48:42.705+0000 7f914e400700 1 -- 172.21.15.107:0/226724514 wait complete. 2024-03-08T20:48:42.700 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:48:42.706+0000 7f914e400700 1 Processor -- start 2024-03-08T20:48:42.700 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:48:42.706+0000 7f914e400700 1 -- start start 2024-03-08T20:48:42.700 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:48:42.706+0000 7f914e400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9148074210 0x7f9148199560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:48:42.700 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:48:42.706+0000 7f914e400700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f914810df00 con 0x7f9148074210 2024-03-08T20:48:42.700 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:48:42.706+0000 7f9147400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9148074210 0x7f9148199560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:48:42.700 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:48:42.706+0000 7f9147400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9148074210 0x7f9148199560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:51904/0 (socket says 172.21.15.107:51904) 2024-03-08T20:48:42.700 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:48:42.706+0000 7f9147400700 1 -- 172.21.15.107:0/1444087652 learned_addr learned my addr 172.21.15.107:0/1444087652 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T20:48:42.701 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:48:42.707+0000 7f9147400700 1 -- 172.21.15.107:0/1444087652 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9148199aa0 con 0x7f9148074210 2024-03-08T20:48:42.701 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:48:42.707+0000 7f9147400700 1 --2- 172.21.15.107:0/1444087652 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9148074210 0x7f9148199560 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f91380056d0 tx=0x7f91380057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:48:42.701 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:48:42.707+0000 7f913fe00700 1 -- 172.21.15.107:0/1444087652 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9138026070 con 0x7f9148074210 2024-03-08T20:48:42.702 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:48:42.707+0000 7f913fe00700 1 -- 172.21.15.107:0/1444087652 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f9138004370 con 0x7f9148074210 2024-03-08T20:48:42.702 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:48:42.707+0000 7f914e400700 1 -- 172.21.15.107:0/1444087652 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9148199d30 con 0x7f9148074210 2024-03-08T20:48:42.702 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:48:42.708+0000 7f913fe00700 1 -- 172.21.15.107:0/1444087652 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f91380039b0 con 0x7f9148074210 2024-03-08T20:48:42.702 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:48:42.708+0000 7f914e400700 1 -- 172.21.15.107:0/1444087652 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f914819a190 con 0x7f9148074210 2024-03-08T20:48:42.702 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:48:42.709+0000 7f914e400700 1 -- 172.21.15.107:0/1444087652 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9128005330 con 0x7f9148074210 2024-03-08T20:48:42.703 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:48:42.709+0000 7f913fe00700 1 -- 172.21.15.107:0/1444087652 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 13) v1 ==== 46163+0+0 (secure 0 0 0) 0x7f9138029070 con 0x7f9148074210 2024-03-08T20:48:42.703 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:48:42.710+0000 7f913fe00700 1 --2- 172.21.15.107:0/1444087652 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f912c038fc0 0x7f912c03b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:48:42.703 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:48:42.710+0000 7f913fe00700 1 -- 172.21.15.107:0/1444087652 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f913801e070 con 0x7f9148074210 2024-03-08T20:48:42.703 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:48:42.710+0000 7f9146a00700 1 --2- 172.21.15.107:0/1444087652 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f912c038fc0 0x7f912c03b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:48:42.704 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:48:42.710+0000 7f9146a00700 1 --2- 172.21.15.107:0/1444087652 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f912c038fc0 0x7f912c03b480 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f9134006fd0 tx=0x7f9134006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:48:42.712 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:48:42.718+0000 7f913fe00700 1 -- 172.21.15.107:0/1444087652 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f9138013070 con 0x7f9148074210 2024-03-08T20:48:42.939 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:48:42.945+0000 7f914e400700 1 -- 172.21.15.107:0/1444087652 --> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi149", "target": ["mon-mgr", ""]}) v1 -- 0x7f9128000bc0 con 0x7f912c038fc0 2024-03-08T20:48:43.727 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:48:43 smithi107 ceph-mon[31381]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T20:48:44.727 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:48:44 smithi107 ceph-mon[31381]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi149", "target": ["mon-mgr", ""]}]: dispatch 2024-03-08T20:48:45.727 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:48:45 smithi107 ceph-mon[31381]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T20:48:45.727 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:48:45 smithi107 ceph-mon[31381]: Deploying cephadm binary to smithi149 2024-03-08T20:48:46.727 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:48:46 smithi107 ceph-mon[31381]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T20:48:49.266 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:48:49.272+0000 7f913fe00700 1 -- 172.21.15.107:0/1444087652 <== mgr.14152 v2:172.21.15.107:6800/3376636618 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f9128000bc0 con 0x7f912c038fc0 2024-03-08T20:48:49.266 INFO:teuthology.orchestra.run.smithi107.stdout:Added host 'smithi149' with addr '172.21.15.149' 2024-03-08T20:48:49.269 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:48:49.275+0000 7f914e400700 1 -- 172.21.15.107:0/1444087652 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f912c038fc0 msgr2=0x7f912c03b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:48:49.270 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:48:49.275+0000 7f914e400700 1 --2- 172.21.15.107:0/1444087652 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f912c038fc0 0x7f912c03b480 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f9134006fd0 tx=0x7f9134006e40 comp rx=0 tx=0).stop 2024-03-08T20:48:49.270 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:48:49.275+0000 7f914e400700 1 -- 172.21.15.107:0/1444087652 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9148074210 msgr2=0x7f9148199560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:48:49.270 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:48:49.275+0000 7f914e400700 1 --2- 172.21.15.107:0/1444087652 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9148074210 0x7f9148199560 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f91380056d0 tx=0x7f91380057b0 comp rx=0 tx=0).stop 2024-03-08T20:48:49.270 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:48:49.275+0000 7f914e400700 1 -- 172.21.15.107:0/1444087652 shutdown_connections 2024-03-08T20:48:49.270 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:48:49.275+0000 7f914e400700 1 --2- 172.21.15.107:0/1444087652 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f912c038fc0 0x7f912c03b480 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:48:49.270 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:48:49.275+0000 7f914e400700 1 --2- 172.21.15.107:0/1444087652 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9148074210 0x7f9148199560 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:48:49.271 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:48:49.275+0000 7f914e400700 1 -- 172.21.15.107:0/1444087652 >> 172.21.15.107:0/1444087652 conn(0x7f91480fd2b0 msgr2=0x7f9148105a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:48:49.271 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:48:49.276+0000 7f914e400700 1 -- 172.21.15.107:0/1444087652 shutdown_connections 2024-03-08T20:48:49.271 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:48:49.276+0000 7f914e400700 1 -- 172.21.15.107:0/1444087652 wait complete. 2024-03-08T20:48:49.477 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:48:49 smithi107 ceph-mon[31381]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T20:48:50.080 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dc20f0f8-dd8c-11ee-95c6-87774f69a715 -- ceph orch host ls --format=json 2024-03-08T20:48:50.477 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:48:50 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:48:50.477 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:48:50 smithi107 ceph-mon[31381]: Added host smithi149 2024-03-08T20:48:50.477 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:48:50 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T20:48:50.477 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:48:50 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:48:50.523 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/mon.a/config 2024-03-08T20:48:51.686 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:48:51 smithi107 ceph-mon[31381]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T20:48:51.686 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:48:51 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:48:52.486 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:48:52.492+0000 7f2297e00700 1 -- 172.21.15.107:0/738403358 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2298107660 msgr2=0x7f2298107a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:48:52.486 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:48:52.492+0000 7f2297e00700 1 --2- 172.21.15.107:0/738403358 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2298107660 0x7f2298107a40 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f2288009910 tx=0x7f2288009c20 comp rx=0 tx=0).stop 2024-03-08T20:48:52.486 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:48:52.492+0000 7f2297e00700 1 -- 172.21.15.107:0/738403358 shutdown_connections 2024-03-08T20:48:52.486 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:48:52.492+0000 7f2297e00700 1 --2- 172.21.15.107:0/738403358 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2298107660 0x7f2298107a40 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:48:52.486 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:48:52.492+0000 7f2297e00700 1 -- 172.21.15.107:0/738403358 >> 172.21.15.107:0/738403358 conn(0x7f22980fd0e0 msgr2=0x7f22980ff500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:48:52.486 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:48:52.493+0000 7f2297e00700 1 -- 172.21.15.107:0/738403358 shutdown_connections 2024-03-08T20:48:52.486 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:48:52.493+0000 7f2297e00700 1 -- 172.21.15.107:0/738403358 wait complete. 2024-03-08T20:48:52.487 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:48:52.494+0000 7f2297e00700 1 Processor -- start 2024-03-08T20:48:52.487 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:48:52.494+0000 7f2297e00700 1 -- start start 2024-03-08T20:48:52.487 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:48:52.494+0000 7f2297e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2298107660 0x7f2298199230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:48:52.487 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:48:52.494+0000 7f2297e00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f229810bac0 con 0x7f2298107660 2024-03-08T20:48:52.488 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:48:52.494+0000 7f2296a00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2298107660 0x7f2298199230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:48:52.488 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:48:52.494+0000 7f2296a00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2298107660 0x7f2298199230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:33756/0 (socket says 172.21.15.107:33756) 2024-03-08T20:48:52.488 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:48:52.494+0000 7f2296a00700 1 -- 172.21.15.107:0/877494632 learned_addr learned my addr 172.21.15.107:0/877494632 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T20:48:52.488 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:48:52.495+0000 7f2296a00700 1 -- 172.21.15.107:0/877494632 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2298199770 con 0x7f2298107660 2024-03-08T20:48:52.488 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:48:52.495+0000 7f2296a00700 1 --2- 172.21.15.107:0/877494632 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2298107660 0x7f2298199230 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f22880092e0 tx=0x7f2288005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:48:52.489 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:48:52.495+0000 7f228f400700 1 -- 172.21.15.107:0/877494632 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2288026070 con 0x7f2298107660 2024-03-08T20:48:52.489 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:48:52.495+0000 7f228f400700 1 -- 172.21.15.107:0/877494632 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f228800b6f0 con 0x7f2298107660 2024-03-08T20:48:52.489 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:48:52.495+0000 7f228f400700 1 -- 172.21.15.107:0/877494632 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f22880037f0 con 0x7f2298107660 2024-03-08T20:48:52.489 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:48:52.495+0000 7f2297e00700 1 -- 172.21.15.107:0/877494632 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2298199a60 con 0x7f2298107660 2024-03-08T20:48:52.489 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:48:52.496+0000 7f2297e00700 1 -- 172.21.15.107:0/877494632 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f229819db00 con 0x7f2298107660 2024-03-08T20:48:52.489 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:48:52.497+0000 7f228f400700 1 -- 172.21.15.107:0/877494632 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 13) v1 ==== 46163+0+0 (secure 0 0 0) 0x7f2288029070 con 0x7f2298107660 2024-03-08T20:48:52.490 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:48:52.497+0000 7f2297e00700 1 -- 172.21.15.107:0/877494632 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2298102d10 con 0x7f2298107660 2024-03-08T20:48:52.490 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:48:52.497+0000 7f228f400700 1 --2- 172.21.15.107:0/877494632 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f227c039020 0x7f227c03b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:48:52.490 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:48:52.497+0000 7f228f400700 1 -- 172.21.15.107:0/877494632 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f228801e070 con 0x7f2298107660 2024-03-08T20:48:52.490 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:48:52.497+0000 7f228fe00700 1 --2- 172.21.15.107:0/877494632 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f227c039020 0x7f227c03b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:48:52.491 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:48:52.498+0000 7f228fe00700 1 --2- 172.21.15.107:0/877494632 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f227c039020 0x7f227c03b4e0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f2280006fd0 tx=0x7f2280006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:48:52.496 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:48:52.503+0000 7f228f400700 1 -- 172.21.15.107:0/877494632 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f2288013070 con 0x7f2298107660 2024-03-08T20:48:52.723 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:48:52.730+0000 7f2297e00700 1 -- 172.21.15.107:0/877494632 --> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f229819a590 con 0x7f227c039020 2024-03-08T20:48:52.726 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:48:52.732+0000 7f228f400700 1 -- 172.21.15.107:0/877494632 <== mgr.14152 v2:172.21.15.107:6800/3376636618 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+161 (secure 0 0 0) 0x7f229819a590 con 0x7f227c039020 2024-03-08T20:48:52.726 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-08T20:48:52.726 INFO:teuthology.orchestra.run.smithi107.stdout:[{"addr": "172.21.15.107", "hostname": "smithi107", "labels": [], "status": ""}, {"addr": "172.21.15.149", "hostname": "smithi149", "labels": [], "status": ""}] 2024-03-08T20:48:52.728 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:48:52.735+0000 7f2297e00700 1 -- 172.21.15.107:0/877494632 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f227c039020 msgr2=0x7f227c03b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:48:52.728 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:48:52.735+0000 7f2297e00700 1 --2- 172.21.15.107:0/877494632 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f227c039020 0x7f227c03b4e0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f2280006fd0 tx=0x7f2280006e40 comp rx=0 tx=0).stop 2024-03-08T20:48:52.729 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:48:52.735+0000 7f2297e00700 1 -- 172.21.15.107:0/877494632 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2298107660 msgr2=0x7f2298199230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:48:52.729 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:48:52.735+0000 7f2297e00700 1 --2- 172.21.15.107:0/877494632 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2298107660 0x7f2298199230 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f22880092e0 tx=0x7f2288005780 comp rx=0 tx=0).stop 2024-03-08T20:48:52.730 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:48:52.735+0000 7f2297e00700 1 -- 172.21.15.107:0/877494632 shutdown_connections 2024-03-08T20:48:52.730 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:48:52.735+0000 7f2297e00700 1 --2- 172.21.15.107:0/877494632 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f227c039020 0x7f227c03b4e0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:48:52.730 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:48:52.735+0000 7f2297e00700 1 --2- 172.21.15.107:0/877494632 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2298107660 0x7f2298199230 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:48:52.730 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:48:52.735+0000 7f2297e00700 1 -- 172.21.15.107:0/877494632 >> 172.21.15.107:0/877494632 conn(0x7f22980fd0e0 msgr2=0x7f2298074010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:48:52.730 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:48:52.736+0000 7f2297e00700 1 -- 172.21.15.107:0/877494632 shutdown_connections 2024-03-08T20:48:52.731 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:48:52.736+0000 7f2297e00700 1 -- 172.21.15.107:0/877494632 wait complete. 2024-03-08T20:48:53.354 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:48:53 smithi107 ceph-mon[31381]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T20:48:53.354 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:48:53 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:48:53.429 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi204 2024-03-08T20:48:53.429 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-03-08T20:48:53.429 DEBUG:teuthology.orchestra.run.smithi204:> dd of=/etc/ceph/ceph.conf 2024-03-08T20:48:53.489 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-03-08T20:48:53.489 DEBUG:teuthology.orchestra.run.smithi204:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-08T20:48:53.584 INFO:tasks.cephadm:Adding host smithi204 to orchestrator... 2024-03-08T20:48:53.584 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dc20f0f8-dd8c-11ee-95c6-87774f69a715 -- ceph orch host add smithi204 2024-03-08T20:48:54.028 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/mon.a/config 2024-03-08T20:48:54.449 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:48:54 smithi107 ceph-mon[31381]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-08T20:48:55.528 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:48:55 smithi107 ceph-mon[31381]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T20:48:55.948 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:48:55.955+0000 7f9d09000700 1 -- 172.21.15.107:0/3027666779 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9d041097f0 msgr2=0x7f9d04109bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:48:55.949 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:48:55.955+0000 7f9d09000700 1 --2- 172.21.15.107:0/3027666779 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9d041097f0 0x7f9d04109bd0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f9cf4009910 tx=0x7f9cf4009c20 comp rx=0 tx=0).stop 2024-03-08T20:48:55.949 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:48:55.955+0000 7f9d09000700 1 -- 172.21.15.107:0/3027666779 shutdown_connections 2024-03-08T20:48:55.949 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:48:55.955+0000 7f9d09000700 1 --2- 172.21.15.107:0/3027666779 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9d041097f0 0x7f9d04109bd0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:48:55.949 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:48:55.955+0000 7f9d09000700 1 -- 172.21.15.107:0/3027666779 >> 172.21.15.107:0/3027666779 conn(0x7f9d040ff320 msgr2=0x7f9d04101740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:48:55.949 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:48:55.955+0000 7f9d09000700 1 -- 172.21.15.107:0/3027666779 shutdown_connections 2024-03-08T20:48:55.949 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:48:55.955+0000 7f9d09000700 1 -- 172.21.15.107:0/3027666779 wait complete. 2024-03-08T20:48:55.950 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:48:55.956+0000 7f9d09000700 1 Processor -- start 2024-03-08T20:48:55.950 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:48:55.956+0000 7f9d09000700 1 -- start start 2024-03-08T20:48:55.950 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:48:55.957+0000 7f9d09000700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9d041097f0 0x7f9d04199260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:48:55.950 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:48:55.957+0000 7f9d09000700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d0410dce0 con 0x7f9d041097f0 2024-03-08T20:48:55.951 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:48:55.957+0000 7f9d03400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9d041097f0 0x7f9d04199260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:48:55.951 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:48:55.957+0000 7f9d03400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9d041097f0 0x7f9d04199260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:57346/0 (socket says 172.21.15.107:57346) 2024-03-08T20:48:55.951 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:48:55.957+0000 7f9d03400700 1 -- 172.21.15.107:0/3771531488 learned_addr learned my addr 172.21.15.107:0/3771531488 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T20:48:55.951 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:48:55.958+0000 7f9d03400700 1 -- 172.21.15.107:0/3771531488 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d041997a0 con 0x7f9d041097f0 2024-03-08T20:48:55.951 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:48:55.958+0000 7f9d03400700 1 --2- 172.21.15.107:0/3771531488 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9d041097f0 0x7f9d04199260 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f9cf4006010 tx=0x7f9cf4005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:48:55.951 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:48:55.958+0000 7f9cfbe00700 1 -- 172.21.15.107:0/3771531488 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9cf4026070 con 0x7f9d041097f0 2024-03-08T20:48:55.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:48:55.958+0000 7f9cfbe00700 1 -- 172.21.15.107:0/3771531488 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f9cf400bba0 con 0x7f9d041097f0 2024-03-08T20:48:55.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:48:55.958+0000 7f9cfbe00700 1 -- 172.21.15.107:0/3771531488 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9cf40038a0 con 0x7f9d041097f0 2024-03-08T20:48:55.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:48:55.958+0000 7f9d09000700 1 -- 172.21.15.107:0/3771531488 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d04199a90 con 0x7f9d041097f0 2024-03-08T20:48:55.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:48:55.958+0000 7f9d09000700 1 -- 172.21.15.107:0/3771531488 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d0419db30 con 0x7f9d041097f0 2024-03-08T20:48:55.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:48:55.959+0000 7f9d09000700 1 -- 172.21.15.107:0/3771531488 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d04104f30 con 0x7f9d041097f0 2024-03-08T20:48:55.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:48:55.960+0000 7f9cfbe00700 1 -- 172.21.15.107:0/3771531488 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 13) v1 ==== 46163+0+0 (secure 0 0 0) 0x7f9cf4029070 con 0x7f9d041097f0 2024-03-08T20:48:55.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:48:55.960+0000 7f9cfbe00700 1 --2- 172.21.15.107:0/3771531488 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f9ce8039020 0x7f9ce803b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:48:55.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:48:55.960+0000 7f9cfbe00700 1 -- 172.21.15.107:0/3771531488 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f9cf401e070 con 0x7f9d041097f0 2024-03-08T20:48:55.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:48:55.960+0000 7f9d02a00700 1 --2- 172.21.15.107:0/3771531488 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f9ce8039020 0x7f9ce803b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:48:55.954 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:48:55.961+0000 7f9d02a00700 1 --2- 172.21.15.107:0/3771531488 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f9ce8039020 0x7f9ce803b4e0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f9cf0006fd0 tx=0x7f9cf0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:48:55.959 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:48:55.966+0000 7f9cfbe00700 1 -- 172.21.15.107:0/3771531488 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f9cf4013070 con 0x7f9d041097f0 2024-03-08T20:48:56.183 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:48:56.190+0000 7f9d09000700 1 -- 172.21.15.107:0/3771531488 --> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi204", "target": ["mon-mgr", ""]}) v1 -- 0x7f9d0419a5c0 con 0x7f9ce8039020 2024-03-08T20:48:57.726 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:48:57 smithi107 ceph-mon[31381]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T20:48:57.727 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:48:57 smithi107 ceph-mon[31381]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi204", "target": ["mon-mgr", ""]}]: dispatch 2024-03-08T20:48:59.227 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:48:58 smithi107 ceph-mon[31381]: Deploying cephadm binary to smithi204 2024-03-08T20:48:59.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:48:58 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:48:59.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:48:58 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:48:59.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:48:58 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:48:59.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:48:58 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:48:59.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:48:58 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi149", "name": "osd_memory_target"}]: dispatch 2024-03-08T20:48:59.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:48:58 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:48:59.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:48:58 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T20:48:59.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:48:58 smithi107 ceph-mon[31381]: Updating smithi149:/etc/ceph/ceph.conf 2024-03-08T20:48:59.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:48:58 smithi107 ceph-mon[31381]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T20:49:00.226 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:48:59 smithi107 ceph-mon[31381]: Updating smithi149:/var/lib/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/config/ceph.conf 2024-03-08T20:49:01.226 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:49:00 smithi107 ceph-mon[31381]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T20:49:02.599 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:49:02.607+0000 7f9cfbe00700 1 -- 172.21.15.107:0/3771531488 <== mgr.14152 v2:172.21.15.107:6800/3376636618 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f9d0419a5c0 con 0x7f9ce8039020 2024-03-08T20:49:02.600 INFO:teuthology.orchestra.run.smithi107.stdout:Added host 'smithi204' with addr '172.21.15.204' 2024-03-08T20:49:02.603 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:49:02.611+0000 7f9d09000700 1 -- 172.21.15.107:0/3771531488 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f9ce8039020 msgr2=0x7f9ce803b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:49:02.603 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:49:02.611+0000 7f9d09000700 1 --2- 172.21.15.107:0/3771531488 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f9ce8039020 0x7f9ce803b4e0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f9cf0006fd0 tx=0x7f9cf0006e40 comp rx=0 tx=0).stop 2024-03-08T20:49:02.603 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:49:02.611+0000 7f9d09000700 1 -- 172.21.15.107:0/3771531488 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9d041097f0 msgr2=0x7f9d04199260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:49:02.603 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:49:02.611+0000 7f9d09000700 1 --2- 172.21.15.107:0/3771531488 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9d041097f0 0x7f9d04199260 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f9cf4006010 tx=0x7f9cf4005780 comp rx=0 tx=0).stop 2024-03-08T20:49:02.603 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:49:02.611+0000 7f9d09000700 1 -- 172.21.15.107:0/3771531488 shutdown_connections 2024-03-08T20:49:02.603 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:49:02.611+0000 7f9d09000700 1 --2- 172.21.15.107:0/3771531488 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f9ce8039020 0x7f9ce803b4e0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:49:02.604 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:49:02.611+0000 7f9d09000700 1 --2- 172.21.15.107:0/3771531488 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9d041097f0 0x7f9d04199260 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:49:02.604 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:49:02.611+0000 7f9d09000700 1 -- 172.21.15.107:0/3771531488 >> 172.21.15.107:0/3771531488 conn(0x7f9d040ff320 msgr2=0x7f9d041052f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:49:02.604 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:49:02.611+0000 7f9d09000700 1 -- 172.21.15.107:0/3771531488 shutdown_connections 2024-03-08T20:49:02.604 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:49:02.611+0000 7f9d09000700 1 -- 172.21.15.107:0/3771531488 wait complete. 2024-03-08T20:49:03.324 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:49:03 smithi107 ceph-mon[31381]: Updating smithi149:/etc/ceph/ceph.client.admin.keyring 2024-03-08T20:49:03.324 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:49:03 smithi107 ceph-mon[31381]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T20:49:03.324 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:49:03 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:49:03.382 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dc20f0f8-dd8c-11ee-95c6-87774f69a715 -- ceph orch host ls --format=json 2024-03-08T20:49:03.804 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/mon.a/config 2024-03-08T20:49:04.476 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:49:04 smithi107 ceph-mon[31381]: Updating smithi149:/var/lib/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/config/ceph.client.admin.keyring 2024-03-08T20:49:04.477 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:49:04 smithi107 ceph-mon[31381]: Added host smithi204 2024-03-08T20:49:04.477 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:49:04 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:49:04.477 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:49:04 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:49:04.477 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:49:04 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:49:04.477 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:49:04 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T20:49:05.675 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:49:05 smithi107 ceph-mon[31381]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T20:49:05.676 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:49:05 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:49:05.676 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:49:05 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:49:05.839 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:49:05.847+0000 7f29f1c00700 1 -- 172.21.15.107:0/3712920484 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f29ec1099a0 msgr2=0x7f29ec109d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:49:05.840 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:49:05.847+0000 7f29f1c00700 1 --2- 172.21.15.107:0/3712920484 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f29ec1099a0 0x7f29ec109d80 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f29dc009910 tx=0x7f29dc009c20 comp rx=0 tx=0).stop 2024-03-08T20:49:05.840 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:49:05.847+0000 7f29f1c00700 1 -- 172.21.15.107:0/3712920484 shutdown_connections 2024-03-08T20:49:05.840 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:49:05.847+0000 7f29f1c00700 1 --2- 172.21.15.107:0/3712920484 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f29ec1099a0 0x7f29ec109d80 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:49:05.840 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:49:05.847+0000 7f29f1c00700 1 -- 172.21.15.107:0/3712920484 >> 172.21.15.107:0/3712920484 conn(0x7f29ec0ff4d0 msgr2=0x7f29ec1018f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:49:05.840 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:49:05.847+0000 7f29f1c00700 1 -- 172.21.15.107:0/3712920484 shutdown_connections 2024-03-08T20:49:05.840 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:49:05.847+0000 7f29f1c00700 1 -- 172.21.15.107:0/3712920484 wait complete. 2024-03-08T20:49:05.841 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:49:05.848+0000 7f29f1c00700 1 Processor -- start 2024-03-08T20:49:05.841 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:49:05.848+0000 7f29f1c00700 1 -- start start 2024-03-08T20:49:05.841 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:49:05.849+0000 7f29f1c00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f29ec1099a0 0x7f29ec1993f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:49:05.841 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:49:05.849+0000 7f29f1c00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29ec10de90 con 0x7f29ec1099a0 2024-03-08T20:49:05.841 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:49:05.849+0000 7f29eaa00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f29ec1099a0 0x7f29ec1993f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:49:05.841 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:49:05.849+0000 7f29eaa00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f29ec1099a0 0x7f29ec1993f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:57366/0 (socket says 172.21.15.107:57366) 2024-03-08T20:49:05.842 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:49:05.849+0000 7f29eaa00700 1 -- 172.21.15.107:0/2712870742 learned_addr learned my addr 172.21.15.107:0/2712870742 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T20:49:05.842 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:49:05.850+0000 7f29eaa00700 1 -- 172.21.15.107:0/2712870742 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29ec199930 con 0x7f29ec1099a0 2024-03-08T20:49:05.842 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:49:05.850+0000 7f29eaa00700 1 --2- 172.21.15.107:0/2712870742 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f29ec1099a0 0x7f29ec1993f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f29dc006010 tx=0x7f29dc005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:49:05.842 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:49:05.850+0000 7f29e3400700 1 -- 172.21.15.107:0/2712870742 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f29dc026070 con 0x7f29ec1099a0 2024-03-08T20:49:05.843 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:49:05.850+0000 7f29e3400700 1 -- 172.21.15.107:0/2712870742 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f29dc00bba0 con 0x7f29ec1099a0 2024-03-08T20:49:05.843 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:49:05.850+0000 7f29e3400700 1 -- 172.21.15.107:0/2712870742 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f29dc0038a0 con 0x7f29ec1099a0 2024-03-08T20:49:05.843 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:49:05.850+0000 7f29f1c00700 1 -- 172.21.15.107:0/2712870742 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29ec199c20 con 0x7f29ec1099a0 2024-03-08T20:49:05.843 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:49:05.850+0000 7f29f1c00700 1 -- 172.21.15.107:0/2712870742 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29ec19dcc0 con 0x7f29ec1099a0 2024-03-08T20:49:05.843 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:49:05.851+0000 7f29e3400700 1 -- 172.21.15.107:0/2712870742 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 13) v1 ==== 46163+0+0 (secure 0 0 0) 0x7f29dc029070 con 0x7f29ec1099a0 2024-03-08T20:49:05.843 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:49:05.851+0000 7f29f1c00700 1 -- 172.21.15.107:0/2712870742 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29ec1050e0 con 0x7f29ec1099a0 2024-03-08T20:49:05.844 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:49:05.852+0000 7f29e3400700 1 --2- 172.21.15.107:0/2712870742 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f29d0039020 0x7f29d003b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:49:05.844 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:49:05.852+0000 7f29e3400700 1 -- 172.21.15.107:0/2712870742 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f29dc01e070 con 0x7f29ec1099a0 2024-03-08T20:49:05.844 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:49:05.852+0000 7f29ea000700 1 --2- 172.21.15.107:0/2712870742 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f29d0039020 0x7f29d003b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:49:05.844 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:49:05.853+0000 7f29ea000700 1 --2- 172.21.15.107:0/2712870742 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f29d0039020 0x7f29d003b4e0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f29d8006fd0 tx=0x7f29d8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:49:05.849 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:49:05.858+0000 7f29e3400700 1 -- 172.21.15.107:0/2712870742 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f29dc013070 con 0x7f29ec1099a0 2024-03-08T20:49:06.079 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:49:06.087+0000 7f29f1c00700 1 -- 172.21.15.107:0/2712870742 --> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f29ec19a750 con 0x7f29d0039020 2024-03-08T20:49:06.081 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:49:06.089+0000 7f29e3400700 1 -- 172.21.15.107:0/2712870742 <== mgr.14152 v2:172.21.15.107:6800/3376636618 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+241 (secure 0 0 0) 0x7f29ec19a750 con 0x7f29d0039020 2024-03-08T20:49:06.081 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-08T20:49:06.081 INFO:teuthology.orchestra.run.smithi107.stdout:[{"addr": "172.21.15.107", "hostname": "smithi107", "labels": [], "status": ""}, {"addr": "172.21.15.149", "hostname": "smithi149", "labels": [], "status": ""}, {"addr": "172.21.15.204", "hostname": "smithi204", "labels": [], "status": ""}] 2024-03-08T20:49:06.084 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:49:06.092+0000 7f29f1c00700 1 -- 172.21.15.107:0/2712870742 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f29d0039020 msgr2=0x7f29d003b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:49:06.085 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:49:06.092+0000 7f29f1c00700 1 --2- 172.21.15.107:0/2712870742 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f29d0039020 0x7f29d003b4e0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f29d8006fd0 tx=0x7f29d8006e40 comp rx=0 tx=0).stop 2024-03-08T20:49:06.085 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:49:06.092+0000 7f29f1c00700 1 -- 172.21.15.107:0/2712870742 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f29ec1099a0 msgr2=0x7f29ec1993f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:49:06.085 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:49:06.092+0000 7f29f1c00700 1 --2- 172.21.15.107:0/2712870742 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f29ec1099a0 0x7f29ec1993f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f29dc006010 tx=0x7f29dc005780 comp rx=0 tx=0).stop 2024-03-08T20:49:06.085 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:49:06.092+0000 7f29f1c00700 1 -- 172.21.15.107:0/2712870742 shutdown_connections 2024-03-08T20:49:06.085 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:49:06.092+0000 7f29f1c00700 1 --2- 172.21.15.107:0/2712870742 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f29d0039020 0x7f29d003b4e0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:49:06.085 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:49:06.092+0000 7f29f1c00700 1 --2- 172.21.15.107:0/2712870742 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f29ec1099a0 0x7f29ec1993f0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:49:06.085 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:49:06.092+0000 7f29f1c00700 1 -- 172.21.15.107:0/2712870742 >> 172.21.15.107:0/2712870742 conn(0x7f29ec0ff4d0 msgr2=0x7f29ec100260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:49:06.085 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:49:06.092+0000 7f29f1c00700 1 -- 172.21.15.107:0/2712870742 shutdown_connections 2024-03-08T20:49:06.085 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:49:06.092+0000 7f29f1c00700 1 -- 172.21.15.107:0/2712870742 wait complete. 2024-03-08T20:49:06.789 INFO:tasks.cephadm:Setting crush tunables to default 2024-03-08T20:49:06.789 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dc20f0f8-dd8c-11ee-95c6-87774f69a715 -- ceph osd crush tunables default 2024-03-08T20:49:07.216 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/mon.a/config 2024-03-08T20:49:07.640 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:49:07 smithi107 ceph-mon[31381]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-08T20:49:07.640 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:49:07 smithi107 ceph-mon[31381]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T20:49:07.640 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:49:07 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:49:09.189 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:49:09.196+0000 7f4b01e00700 1 -- 172.21.15.107:0/2998854114 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4afc102960 msgr2=0x7f4afc102d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:49:09.189 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:49:09.196+0000 7f4b01e00700 1 --2- 172.21.15.107:0/2998854114 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4afc102960 0x7f4afc102d40 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f4aec009910 tx=0x7f4aec009c20 comp rx=0 tx=0).stop 2024-03-08T20:49:09.189 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:49:09.196+0000 7f4b01e00700 1 -- 172.21.15.107:0/2998854114 shutdown_connections 2024-03-08T20:49:09.189 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:49:09.196+0000 7f4b01e00700 1 --2- 172.21.15.107:0/2998854114 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4afc102960 0x7f4afc102d40 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:49:09.189 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:49:09.196+0000 7f4b01e00700 1 -- 172.21.15.107:0/2998854114 >> 172.21.15.107:0/2998854114 conn(0x7f4afc0fc1d0 msgr2=0x7f4afc0fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:49:09.189 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:49:09.196+0000 7f4b01e00700 1 -- 172.21.15.107:0/2998854114 shutdown_connections 2024-03-08T20:49:09.190 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:49:09.196+0000 7f4b01e00700 1 -- 172.21.15.107:0/2998854114 wait complete. 2024-03-08T20:49:09.190 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:49:09.197+0000 7f4b01e00700 1 Processor -- start 2024-03-08T20:49:09.191 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:49:09.198+0000 7f4b01e00700 1 -- start start 2024-03-08T20:49:09.191 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:49:09.198+0000 7f4b01e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4afc102960 0x7f4afc199540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:49:09.191 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:49:09.198+0000 7f4b01e00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4afc10bd00 con 0x7f4afc102960 2024-03-08T20:49:09.191 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:49:09.198+0000 7f4afaa00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4afc102960 0x7f4afc199540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:49:09.191 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:49:09.198+0000 7f4afaa00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4afc102960 0x7f4afc199540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:43984/0 (socket says 172.21.15.107:43984) 2024-03-08T20:49:09.191 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:49:09.198+0000 7f4afaa00700 1 -- 172.21.15.107:0/1054927658 learned_addr learned my addr 172.21.15.107:0/1054927658 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T20:49:09.191 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:49:09.199+0000 7f4afaa00700 1 -- 172.21.15.107:0/1054927658 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4afc199a80 con 0x7f4afc102960 2024-03-08T20:49:09.191 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:49:09.199+0000 7f4afaa00700 1 --2- 172.21.15.107:0/1054927658 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4afc102960 0x7f4afc199540 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f4aec006010 tx=0x7f4aec005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:49:09.192 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:49:09.199+0000 7f4b00a00700 1 -- 172.21.15.107:0/1054927658 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4aec026070 con 0x7f4afc102960 2024-03-08T20:49:09.192 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:49:09.199+0000 7f4b00a00700 1 -- 172.21.15.107:0/1054927658 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f4aec00bc00 con 0x7f4afc102960 2024-03-08T20:49:09.192 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:49:09.199+0000 7f4b01e00700 1 -- 172.21.15.107:0/1054927658 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4afc199d10 con 0x7f4afc102960 2024-03-08T20:49:09.192 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:49:09.199+0000 7f4b00a00700 1 -- 172.21.15.107:0/1054927658 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4aec003910 con 0x7f4afc102960 2024-03-08T20:49:09.192 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:49:09.200+0000 7f4b01e00700 1 -- 172.21.15.107:0/1054927658 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4afc19a170 con 0x7f4afc102960 2024-03-08T20:49:09.193 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:49:09.201+0000 7f4b01e00700 1 -- 172.21.15.107:0/1054927658 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4afc06a350 con 0x7f4afc102960 2024-03-08T20:49:09.193 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:49:09.201+0000 7f4b00a00700 1 -- 172.21.15.107:0/1054927658 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 13) v1 ==== 46163+0+0 (secure 0 0 0) 0x7f4aec029070 con 0x7f4afc102960 2024-03-08T20:49:09.193 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:49:09.201+0000 7f4b00a00700 1 --2- 172.21.15.107:0/1054927658 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f4ae0039020 0x7f4ae003b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:49:09.193 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:49:09.201+0000 7f4b00a00700 1 -- 172.21.15.107:0/1054927658 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f4aec01e070 con 0x7f4afc102960 2024-03-08T20:49:09.194 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:49:09.202+0000 7f4afa000700 1 --2- 172.21.15.107:0/1054927658 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f4ae0039020 0x7f4ae003b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:49:09.194 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:49:09.202+0000 7f4afa000700 1 --2- 172.21.15.107:0/1054927658 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f4ae0039020 0x7f4ae003b4e0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f4ae8006fd0 tx=0x7f4ae8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:49:09.198 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:49:09.207+0000 7f4b00a00700 1 -- 172.21.15.107:0/1054927658 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f4aec013070 con 0x7f4afc102960 2024-03-08T20:49:09.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:49:09.434+0000 7f4b01e00700 1 -- 172.21.15.107:0/1054927658 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f4afc19a850 con 0x7f4afc102960 2024-03-08T20:49:09.726 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:49:09 smithi107 ceph-mon[31381]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T20:49:10.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:49:10.434+0000 7f4b00a00700 1 -- 172.21.15.107:0/1054927658 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) v1 ==== 124+0+0 (secure 0 0 0) 0x7f4aec00be60 con 0x7f4afc102960 2024-03-08T20:49:10.430 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:49:10.438+0000 7f4b01e00700 1 -- 172.21.15.107:0/1054927658 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f4ae0039020 msgr2=0x7f4ae003b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:49:10.430 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:49:10.438+0000 7f4b01e00700 1 --2- 172.21.15.107:0/1054927658 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f4ae0039020 0x7f4ae003b4e0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f4ae8006fd0 tx=0x7f4ae8006e40 comp rx=0 tx=0).stop 2024-03-08T20:49:10.430 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:49:10.438+0000 7f4b01e00700 1 -- 172.21.15.107:0/1054927658 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4afc102960 msgr2=0x7f4afc199540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:49:10.430 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:49:10.438+0000 7f4b01e00700 1 --2- 172.21.15.107:0/1054927658 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4afc102960 0x7f4afc199540 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f4aec006010 tx=0x7f4aec005780 comp rx=0 tx=0).stop 2024-03-08T20:49:10.430 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:49:10.438+0000 7f4b01e00700 1 -- 172.21.15.107:0/1054927658 shutdown_connections 2024-03-08T20:49:10.431 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:49:10.438+0000 7f4b01e00700 1 --2- 172.21.15.107:0/1054927658 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f4ae0039020 0x7f4ae003b4e0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:49:10.431 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:49:10.438+0000 7f4b01e00700 1 --2- 172.21.15.107:0/1054927658 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4afc102960 0x7f4afc199540 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:49:10.431 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:49:10.438+0000 7f4b01e00700 1 -- 172.21.15.107:0/1054927658 >> 172.21.15.107:0/1054927658 conn(0x7f4afc0fc1d0 msgr2=0x7f4afc107b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:49:10.431 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:49:10.438+0000 7f4b01e00700 1 -- 172.21.15.107:0/1054927658 shutdown_connections 2024-03-08T20:49:10.431 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:49:10.439+0000 7f4b01e00700 1 -- 172.21.15.107:0/1054927658 wait complete. 2024-03-08T20:49:10.431 INFO:teuthology.orchestra.run.smithi107.stderr:adjusted tunables profile to default 2024-03-08T20:49:10.726 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:49:10 smithi107 ceph-mon[31381]: from='client.? 172.21.15.107:0/1054927658' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-03-08T20:49:10.726 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:49:10 smithi107 ceph-mon[31381]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T20:49:11.214 INFO:tasks.cephadm:Adding mon.a on smithi107 2024-03-08T20:49:11.214 INFO:tasks.cephadm:Adding mon.b on smithi149 2024-03-08T20:49:11.214 INFO:tasks.cephadm:Adding mon.c on smithi204 2024-03-08T20:49:11.214 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dc20f0f8-dd8c-11ee-95c6-87774f69a715 -- ceph orch apply mon '3;smithi107:172.21.15.107=a;smithi149:172.21.15.149=b;smithi204:172.21.15.204=c' 2024-03-08T20:49:11.726 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:49:11 smithi107 ceph-mon[31381]: from='client.? 172.21.15.107:0/1054927658' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-03-08T20:49:11.726 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:49:11 smithi107 ceph-mon[31381]: osdmap e4: 0 total, 0 up, 0 in 2024-03-08T20:49:12.121 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-08T20:49:12.221 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-08T20:49:13.477 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:49:13 smithi107 ceph-mon[31381]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T20:49:13.477 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:49:13 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:49:13.477 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:49:13 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:49:13.477 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:49:13 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:49:13.477 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:49:13 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:49:13.477 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:49:13 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi204", "name": "osd_memory_target"}]: dispatch 2024-03-08T20:49:13.477 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:49:13 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:49:13.477 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:49:13 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T20:49:13.478 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:49:13 smithi107 ceph-mon[31381]: Updating smithi204:/etc/ceph/ceph.conf 2024-03-08T20:49:14.315 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:14.322+0000 7f0ebb000700 1 -- 172.21.15.204:0/509087682 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0eb4103a10 msgr2=0x7f0eb4103e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:49:14.315 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:14.322+0000 7f0ebb000700 1 --2- 172.21.15.204:0/509087682 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0eb4103a10 0x7f0eb4103e30 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f0ea4009910 tx=0x7f0ea4009c20 comp rx=0 tx=0).stop 2024-03-08T20:49:14.315 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:14.322+0000 7f0ebb000700 1 -- 172.21.15.204:0/509087682 shutdown_connections 2024-03-08T20:49:14.315 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:14.322+0000 7f0ebb000700 1 --2- 172.21.15.204:0/509087682 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0eb4103a10 0x7f0eb4103e30 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:49:14.315 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:14.322+0000 7f0ebb000700 1 -- 172.21.15.204:0/509087682 >> 172.21.15.204:0/509087682 conn(0x7f0eb40fefb0 msgr2=0x7f0eb41013f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:49:14.315 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:14.322+0000 7f0ebb000700 1 -- 172.21.15.204:0/509087682 shutdown_connections 2024-03-08T20:49:14.315 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:14.322+0000 7f0ebb000700 1 -- 172.21.15.204:0/509087682 wait complete. 2024-03-08T20:49:14.316 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:14.324+0000 7f0ebb000700 1 Processor -- start 2024-03-08T20:49:14.316 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:14.324+0000 7f0ebb000700 1 -- start start 2024-03-08T20:49:14.316 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:14.325+0000 7f0ebb000700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0eb4103a10 0x7f0eb4199040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:49:14.316 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:14.325+0000 7f0ebb000700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0eb4109950 con 0x7f0eb4103a10 2024-03-08T20:49:14.317 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:14.325+0000 7f0eb3e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0eb4103a10 0x7f0eb4199040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:49:14.317 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:14.325+0000 7f0eb3e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0eb4103a10 0x7f0eb4199040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.204:55572/0 (socket says 172.21.15.204:55572) 2024-03-08T20:49:14.317 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:14.325+0000 7f0eb3e00700 1 -- 172.21.15.204:0/1490128953 learned_addr learned my addr 172.21.15.204:0/1490128953 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-03-08T20:49:14.318 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:14.326+0000 7f0eb3e00700 1 -- 172.21.15.204:0/1490128953 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0eb4199580 con 0x7f0eb4103a10 2024-03-08T20:49:14.318 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:14.326+0000 7f0eb3e00700 1 --2- 172.21.15.204:0/1490128953 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0eb4103a10 0x7f0eb4199040 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f0ea4005950 tx=0x7f0ea4004fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:49:14.319 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:14.327+0000 7f0eb0c00700 1 -- 172.21.15.204:0/1490128953 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0ea4026070 con 0x7f0eb4103a10 2024-03-08T20:49:14.319 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:14.327+0000 7f0ebb000700 1 -- 172.21.15.204:0/1490128953 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0eb4199810 con 0x7f0eb4103a10 2024-03-08T20:49:14.319 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:14.327+0000 7f0eb0c00700 1 -- 172.21.15.204:0/1490128953 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f0ea400bc00 con 0x7f0eb4103a10 2024-03-08T20:49:14.319 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:14.327+0000 7f0eb0c00700 1 -- 172.21.15.204:0/1490128953 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0ea40039a0 con 0x7f0eb4103a10 2024-03-08T20:49:14.319 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:14.327+0000 7f0ebb000700 1 -- 172.21.15.204:0/1490128953 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0eb419c500 con 0x7f0eb4103a10 2024-03-08T20:49:14.320 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:14.328+0000 7f0ebb000700 1 -- 172.21.15.204:0/1490128953 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0eb410a0d0 con 0x7f0eb4103a10 2024-03-08T20:49:14.321 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:14.329+0000 7f0eb0c00700 1 -- 172.21.15.204:0/1490128953 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 13) v1 ==== 46163+0+0 (secure 0 0 0) 0x7f0ea4029020 con 0x7f0eb4103a10 2024-03-08T20:49:14.321 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:14.329+0000 7f0eb0c00700 1 --2- 172.21.15.204:0/1490128953 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f0e98038fc0 0x7f0e9803b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:49:14.321 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:14.329+0000 7f0eb0c00700 1 -- 172.21.15.204:0/1490128953 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f0ea401e070 con 0x7f0eb4103a10 2024-03-08T20:49:14.322 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:14.330+0000 7f0eb3400700 1 --2- 172.21.15.204:0/1490128953 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f0e98038fc0 0x7f0e9803b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:49:14.323 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:14.331+0000 7f0eb3400700 1 --2- 172.21.15.204:0/1490128953 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f0e98038fc0 0x7f0e9803b480 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f0ea0006fd0 tx=0x7f0ea0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:49:14.336 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:14.345+0000 7f0eb0c00700 1 -- 172.21.15.204:0/1490128953 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f0ea4013070 con 0x7f0eb4103a10 2024-03-08T20:49:14.476 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:49:14 smithi107 ceph-mon[31381]: Updating smithi204:/var/lib/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/config/ceph.conf 2024-03-08T20:49:14.572 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:14.580+0000 7f0ebb000700 1 -- 172.21.15.204:0/1490128953 --> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi107:172.21.15.107=a;smithi149:172.21.15.149=b;smithi204:172.21.15.204=c", "target": ["mon-mgr", ""]}) v1 -- 0x7f0eb402dac0 con 0x7f0e98038fc0 2024-03-08T20:49:14.590 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:14.598+0000 7f0eb0c00700 1 -- 172.21.15.204:0/1490128953 <== mgr.14152 v2:172.21.15.107:6800/3376636618 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f0eb402dac0 con 0x7f0e98038fc0 2024-03-08T20:49:14.590 INFO:teuthology.orchestra.run.smithi204.stdout:Scheduled mon update... 2024-03-08T20:49:14.593 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:14.600+0000 7f0ebb000700 1 -- 172.21.15.204:0/1490128953 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f0e98038fc0 msgr2=0x7f0e9803b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:49:14.593 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:14.601+0000 7f0ebb000700 1 --2- 172.21.15.204:0/1490128953 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f0e98038fc0 0x7f0e9803b480 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f0ea0006fd0 tx=0x7f0ea0006e40 comp rx=0 tx=0).stop 2024-03-08T20:49:14.593 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:14.601+0000 7f0ebb000700 1 -- 172.21.15.204:0/1490128953 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0eb4103a10 msgr2=0x7f0eb4199040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:49:14.593 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:14.601+0000 7f0ebb000700 1 --2- 172.21.15.204:0/1490128953 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0eb4103a10 0x7f0eb4199040 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f0ea4005950 tx=0x7f0ea4004fa0 comp rx=0 tx=0).stop 2024-03-08T20:49:14.593 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:14.601+0000 7f0ebb000700 1 -- 172.21.15.204:0/1490128953 shutdown_connections 2024-03-08T20:49:14.593 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:14.601+0000 7f0ebb000700 1 --2- 172.21.15.204:0/1490128953 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f0e98038fc0 0x7f0e9803b480 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:49:14.593 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:14.601+0000 7f0ebb000700 1 --2- 172.21.15.204:0/1490128953 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0eb4103a10 0x7f0eb4199040 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:49:14.594 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:14.601+0000 7f0ebb000700 1 -- 172.21.15.204:0/1490128953 >> 172.21.15.204:0/1490128953 conn(0x7f0eb40fefb0 msgr2=0x7f0eb40ffa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:49:14.594 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:14.601+0000 7f0ebb000700 1 -- 172.21.15.204:0/1490128953 shutdown_connections 2024-03-08T20:49:14.594 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:14.601+0000 7f0ebb000700 1 -- 172.21.15.204:0/1490128953 wait complete. 2024-03-08T20:49:15.332 DEBUG:teuthology.orchestra.run.smithi149:mon.b> sudo journalctl -f -n 0 -u ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715@mon.b.service 2024-03-08T20:49:15.340 DEBUG:teuthology.orchestra.run.smithi204:mon.c> sudo journalctl -f -n 0 -u ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715@mon.c.service 2024-03-08T20:49:15.347 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-03-08T20:49:15.347 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dc20f0f8-dd8c-11ee-95c6-87774f69a715 -- ceph mon dump -f json 2024-03-08T20:49:15.476 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:49:15 smithi107 ceph-mon[31381]: Updating smithi204:/etc/ceph/ceph.client.admin.keyring 2024-03-08T20:49:15.477 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:49:15 smithi107 ceph-mon[31381]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T20:49:15.477 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:49:15 smithi107 ceph-mon[31381]: Updating smithi204:/var/lib/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/config/ceph.client.admin.keyring 2024-03-08T20:49:15.477 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:49:15 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:49:15.477 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:49:15 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:49:15.477 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:49:15 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:49:15.477 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:49:15 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:49:15.477 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:49:15 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-08T20:49:15.477 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:49:15 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:49:15.786 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/config/ceph.conf 2024-03-08T20:49:16.476 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:49:16 smithi107 ceph-mon[31381]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi107:172.21.15.107=a;smithi149:172.21.15.149=b;smithi204:172.21.15.204=c", "target": ["mon-mgr", ""]}]: dispatch 2024-03-08T20:49:16.476 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:49:16 smithi107 ceph-mon[31381]: Saving service mon spec with placement smithi107:172.21.15.107=a;smithi149:172.21.15.149=b;smithi204:172.21.15.204=c;count:3 2024-03-08T20:49:16.476 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:49:16 smithi107 ceph-mon[31381]: Deploying daemon mon.c on smithi204 2024-03-08T20:49:17.476 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:49:17 smithi107 ceph-mon[31381]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T20:49:19.476 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:49:19 smithi107 ceph-mon[31381]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T20:49:19.599 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:19.606+0000 7f59d3e00700 1 -- 172.21.15.204:0/1988189677 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f59d410dcc0 msgr2=0x7f59d410e0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:49:19.599 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:19.606+0000 7f59d3e00700 1 --2- 172.21.15.204:0/1988189677 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f59d410dcc0 0x7f59d410e0a0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f59c4009910 tx=0x7f59c4009c20 comp rx=0 tx=0).stop 2024-03-08T20:49:19.599 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:19.606+0000 7f59d3e00700 1 -- 172.21.15.204:0/1988189677 shutdown_connections 2024-03-08T20:49:19.600 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:19.606+0000 7f59d3e00700 1 --2- 172.21.15.204:0/1988189677 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f59d410dcc0 0x7f59d410e0a0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:49:19.600 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:19.606+0000 7f59d3e00700 1 -- 172.21.15.204:0/1988189677 >> 172.21.15.204:0/1988189677 conn(0x7f59d40773b0 msgr2=0x7f59d40797d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:49:19.600 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:19.606+0000 7f59d3e00700 1 -- 172.21.15.204:0/1988189677 shutdown_connections 2024-03-08T20:49:19.600 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:19.606+0000 7f59d3e00700 1 -- 172.21.15.204:0/1988189677 wait complete. 2024-03-08T20:49:19.603 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:19.608+0000 7f59d3e00700 1 Processor -- start 2024-03-08T20:49:19.603 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:19.608+0000 7f59d3e00700 1 -- start start 2024-03-08T20:49:19.603 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:19.608+0000 7f59d3e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f59d410dcc0 0x7f59d419d580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:49:19.603 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:19.608+0000 7f59d3e00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59d41120c0 con 0x7f59d410dcc0 2024-03-08T20:49:19.603 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:19.609+0000 7f59d2a00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f59d410dcc0 0x7f59d419d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:49:19.604 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:19.609+0000 7f59d2a00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f59d410dcc0 0x7f59d419d580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.204:55604/0 (socket says 172.21.15.204:55604) 2024-03-08T20:49:19.604 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:19.609+0000 7f59d2a00700 1 -- 172.21.15.204:0/4167819322 learned_addr learned my addr 172.21.15.204:0/4167819322 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-03-08T20:49:19.604 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:19.610+0000 7f59d2a00700 1 -- 172.21.15.204:0/4167819322 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59d419dac0 con 0x7f59d410dcc0 2024-03-08T20:49:19.604 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:19.610+0000 7f59d2a00700 1 --2- 172.21.15.204:0/4167819322 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f59d410dcc0 0x7f59d419d580 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f59c40056d0 tx=0x7f59c40057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:49:19.604 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:19.610+0000 7f59d8a00700 1 -- 172.21.15.204:0/4167819322 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f59c4026070 con 0x7f59d410dcc0 2024-03-08T20:49:19.604 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:19.610+0000 7f59d3e00700 1 -- 172.21.15.204:0/4167819322 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59d419ddb0 con 0x7f59d410dcc0 2024-03-08T20:49:19.605 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:19.610+0000 7f59d8a00700 1 -- 172.21.15.204:0/4167819322 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f59c4004370 con 0x7f59d410dcc0 2024-03-08T20:49:19.605 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:19.610+0000 7f59d8a00700 1 -- 172.21.15.204:0/4167819322 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f59c40039b0 con 0x7f59d410dcc0 2024-03-08T20:49:19.605 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:19.610+0000 7f59d3e00700 1 -- 172.21.15.204:0/4167819322 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59d41a1e50 con 0x7f59d410dcc0 2024-03-08T20:49:19.606 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:19.612+0000 7f59d8a00700 1 -- 172.21.15.204:0/4167819322 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 13) v1 ==== 46163+0+0 (secure 0 0 0) 0x7f59c4029070 con 0x7f59d410dcc0 2024-03-08T20:49:19.606 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:19.612+0000 7f59d3e00700 1 -- 172.21.15.204:0/4167819322 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59d4109300 con 0x7f59d410dcc0 2024-03-08T20:49:19.606 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:19.612+0000 7f59d8a00700 1 --2- 172.21.15.204:0/4167819322 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f59b8038fc0 0x7f59b803b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:49:19.606 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:19.612+0000 7f59d8a00700 1 -- 172.21.15.204:0/4167819322 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f59c401e070 con 0x7f59d410dcc0 2024-03-08T20:49:19.606 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:19.613+0000 7f59d2000700 1 --2- 172.21.15.204:0/4167819322 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f59b8038fc0 0x7f59b803b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:49:19.606 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:19.614+0000 7f59d2000700 1 --2- 172.21.15.204:0/4167819322 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f59b8038fc0 0x7f59b803b480 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f59c0006fd0 tx=0x7f59c0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:49:19.620 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:19.628+0000 7f59d8a00700 1 -- 172.21.15.204:0/4167819322 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f59c4013070 con 0x7f59d410dcc0 2024-03-08T20:49:19.972 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:19.979+0000 7f59d3e00700 1 -- 172.21.15.204:0/4167819322 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f59d419e8e0 con 0x7f59d410dcc0 2024-03-08T20:49:19.973 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:19.981+0000 7f59d8a00700 1 -- 172.21.15.204:0/4167819322 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+749 (secure 0 0 0) 0x7f59c4030090 con 0x7f59d410dcc0 2024-03-08T20:49:19.973 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-03-08T20:49:19.973 INFO:teuthology.orchestra.run.smithi204.stdout:{"epoch":1,"fsid":"dc20f0f8-dd8c-11ee-95c6-87774f69a715","modified":"2024-03-08T20:47:00.037201Z","created":"2024-03-08T20:47:00.037201Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:3300","nonce":0},{"type":"v1","addr":"172.21.15.107:6789","nonce":0}]},"addr":"172.21.15.107:6789/0","public_addr":"172.21.15.107:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-08T20:49:19.976 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:19.983+0000 7f59d3e00700 1 -- 172.21.15.204:0/4167819322 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f59b8038fc0 msgr2=0x7f59b803b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:49:19.976 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:19.983+0000 7f59d3e00700 1 --2- 172.21.15.204:0/4167819322 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f59b8038fc0 0x7f59b803b480 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f59c0006fd0 tx=0x7f59c0006e40 comp rx=0 tx=0).stop 2024-03-08T20:49:19.976 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:19.983+0000 7f59d3e00700 1 -- 172.21.15.204:0/4167819322 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f59d410dcc0 msgr2=0x7f59d419d580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:49:19.976 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:19.983+0000 7f59d3e00700 1 --2- 172.21.15.204:0/4167819322 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f59d410dcc0 0x7f59d419d580 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f59c40056d0 tx=0x7f59c40057b0 comp rx=0 tx=0).stop 2024-03-08T20:49:19.976 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:19.984+0000 7f59d3e00700 1 -- 172.21.15.204:0/4167819322 shutdown_connections 2024-03-08T20:49:19.976 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:19.984+0000 7f59d3e00700 1 --2- 172.21.15.204:0/4167819322 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f59b8038fc0 0x7f59b803b480 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:49:19.976 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:19.984+0000 7f59d3e00700 1 --2- 172.21.15.204:0/4167819322 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f59d410dcc0 0x7f59d419d580 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:49:19.976 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:19.984+0000 7f59d3e00700 1 -- 172.21.15.204:0/4167819322 >> 172.21.15.204:0/4167819322 conn(0x7f59d40773b0 msgr2=0x7f59d4078cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:49:19.976 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:19.984+0000 7f59d3e00700 1 -- 172.21.15.204:0/4167819322 shutdown_connections 2024-03-08T20:49:19.976 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:19.984+0000 7f59d3e00700 1 -- 172.21.15.204:0/4167819322 wait complete. 2024-03-08T20:49:19.977 INFO:teuthology.orchestra.run.smithi204.stderr:dumped monmap epoch 1 2024-03-08T20:49:20.476 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:49:20 smithi107 ceph-mon[31381]: from='client.? 172.21.15.204:0/4167819322' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-08T20:49:21.476 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:49:21 smithi107 ceph-mon[31381]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T20:49:21.639 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 podman[34916]: 2024-03-08 20:49:21.395805567 +0000 UTC m=+0.130405228 container create c7d2a8d930206e45ef4add3e134a7be18d6cf2f728e047b8e4adb69bdd5f9f36 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mon-c, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=rishabh-2024mar8-7d63482, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-03-08T20:49:21.797 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-03-08T20:49:21.798 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dc20f0f8-dd8c-11ee-95c6-87774f69a715 -- ceph mon dump -f json 2024-03-08T20:49:22.137 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 podman[34916]: 2024-03-08 20:49:21.708102554 +0000 UTC m=+0.442702284 container init c7d2a8d930206e45ef4add3e134a7be18d6cf2f728e047b8e4adb69bdd5f9f36 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mon-c, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=rishabh-2024mar8-7d63482, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307) 2024-03-08T20:49:22.138 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 podman[34916]: 2024-03-08 20:49:21.718179791 +0000 UTC m=+0.452779457 container start c7d2a8d930206e45ef4add3e134a7be18d6cf2f728e047b8e4adb69bdd5f9f36 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mon-c, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=rishabh-2024mar8-7d63482, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-08T20:49:22.138 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: set uid:gid to 167:167 (ceph:ceph) 2024-03-08T20:49:22.138 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: ceph version 19.0.0-1965-g7d63482a (7d63482aa89628d733e298984a762389b625c761) squid (dev), process ceph-mon, pid 2 2024-03-08T20:49:22.138 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: pidfile_write: ignore empty --pid-file 2024-03-08T20:49:22.138 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: load: jerasure load: lrc 2024-03-08T20:49:22.138 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 bash[34916]: c7d2a8d930206e45ef4add3e134a7be18d6cf2f728e047b8e4adb69bdd5f9f36 2024-03-08T20:49:22.138 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: rocksdb: RocksDB version: 7.9.2 2024-03-08T20:49:22.138 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: rocksdb: Git sha 0 2024-03-08T20:49:22.138 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: rocksdb: Compile date 2024-03-08 15:06:51 2024-03-08T20:49:22.138 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: rocksdb: DB SUMMARY 2024-03-08T20:49:22.138 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: rocksdb: DB Session ID: 8SB1DBLN886US22PQMHV 2024-03-08T20:49:22.139 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: rocksdb: CURRENT file: CURRENT 2024-03-08T20:49:22.139 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: rocksdb: IDENTITY file: IDENTITY 2024-03-08T20:49:22.139 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-03-08T20:49:22.139 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-03-08T20:49:22.139 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2024-03-08T20:49:22.139 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: rocksdb: Options.error_if_exists: 0 2024-03-08T20:49:22.139 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: rocksdb: Options.create_if_missing: 0 2024-03-08T20:49:22.139 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: rocksdb: Options.paranoid_checks: 1 2024-03-08T20:49:22.139 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: rocksdb: Options.flush_verify_memtable_count: 1 2024-03-08T20:49:22.139 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-03-08T20:49:22.139 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-03-08T20:49:22.139 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: rocksdb: Options.env: 0x55bc3c201b20 2024-03-08T20:49:22.140 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: rocksdb: Options.fs: PosixFileSystem 2024-03-08T20:49:22.140 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: rocksdb: Options.info_log: 0x55bc3e79f920 2024-03-08T20:49:22.140 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: rocksdb: Options.max_file_opening_threads: 16 2024-03-08T20:49:22.140 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: rocksdb: Options.statistics: (nil) 2024-03-08T20:49:22.140 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: rocksdb: Options.use_fsync: 0 2024-03-08T20:49:22.140 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: rocksdb: Options.max_log_file_size: 0 2024-03-08T20:49:22.140 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-03-08T20:49:22.140 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: rocksdb: Options.log_file_time_to_roll: 0 2024-03-08T20:49:22.140 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: rocksdb: Options.keep_log_file_num: 1000 2024-03-08T20:49:22.140 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: rocksdb: Options.recycle_log_file_num: 0 2024-03-08T20:49:22.140 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: rocksdb: Options.allow_fallocate: 1 2024-03-08T20:49:22.140 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: rocksdb: Options.allow_mmap_reads: 0 2024-03-08T20:49:22.141 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: rocksdb: Options.allow_mmap_writes: 0 2024-03-08T20:49:22.141 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: rocksdb: Options.use_direct_reads: 0 2024-03-08T20:49:22.141 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-03-08T20:49:22.141 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: rocksdb: Options.create_missing_column_families: 0 2024-03-08T20:49:22.141 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: rocksdb: Options.db_log_dir: 2024-03-08T20:49:22.141 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: rocksdb: Options.wal_dir: 2024-03-08T20:49:22.141 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: rocksdb: Options.table_cache_numshardbits: 6 2024-03-08T20:49:22.141 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: rocksdb: Options.WAL_ttl_seconds: 0 2024-03-08T20:49:22.141 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: rocksdb: Options.WAL_size_limit_MB: 0 2024-03-08T20:49:22.141 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-03-08T20:49:22.141 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-03-08T20:49:22.141 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: rocksdb: Options.is_fd_close_on_exec: 1 2024-03-08T20:49:22.141 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: rocksdb: Options.advise_random_on_open: 1 2024-03-08T20:49:22.141 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: rocksdb: Options.db_write_buffer_size: 0 2024-03-08T20:49:22.142 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: rocksdb: Options.write_buffer_manager: 0x55bc3e773400 2024-03-08T20:49:22.142 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-03-08T20:49:22.142 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-03-08T20:49:22.142 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: rocksdb: Options.use_adaptive_mutex: 0 2024-03-08T20:49:22.142 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: rocksdb: Options.rate_limiter: (nil) 2024-03-08T20:49:22.142 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-03-08T20:49:22.142 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: rocksdb: Options.wal_recovery_mode: 2 2024-03-08T20:49:22.142 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: rocksdb: Options.enable_thread_tracking: 0 2024-03-08T20:49:22.142 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: rocksdb: Options.enable_pipelined_write: 0 2024-03-08T20:49:22.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: rocksdb: Options.unordered_write: 0 2024-03-08T20:49:22.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-03-08T20:49:22.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-03-08T20:49:22.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-03-08T20:49:22.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-03-08T20:49:22.143 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: rocksdb: Options.row_cache: None 2024-03-08T20:49:22.145 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: rocksdb: Options.wal_filter: None 2024-03-08T20:49:22.145 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-03-08T20:49:22.145 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: rocksdb: Options.allow_ingest_behind: 0 2024-03-08T20:49:22.145 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: rocksdb: Options.two_write_queues: 0 2024-03-08T20:49:22.145 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: rocksdb: Options.manual_wal_flush: 0 2024-03-08T20:49:22.145 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: rocksdb: Options.wal_compression: 0 2024-03-08T20:49:22.145 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: rocksdb: Options.atomic_flush: 0 2024-03-08T20:49:22.145 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-03-08T20:49:22.145 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: rocksdb: Options.persist_stats_to_disk: 0 2024-03-08T20:49:22.145 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: rocksdb: Options.write_dbid_to_manifest: 0 2024-03-08T20:49:22.146 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: rocksdb: Options.log_readahead_size: 0 2024-03-08T20:49:22.146 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-03-08T20:49:22.146 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: rocksdb: Options.best_efforts_recovery: 0 2024-03-08T20:49:22.146 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-03-08T20:49:22.146 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-03-08T20:49:22.146 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: rocksdb: Options.allow_data_in_errors: 0 2024-03-08T20:49:22.146 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: rocksdb: Options.db_host_id: __hostname__ 2024-03-08T20:49:22.146 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: rocksdb: Options.enforce_single_del_contracts: true 2024-03-08T20:49:22.146 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: rocksdb: Options.max_background_jobs: 2 2024-03-08T20:49:22.146 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: rocksdb: Options.max_background_compactions: -1 2024-03-08T20:49:22.146 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: rocksdb: Options.max_subcompactions: 1 2024-03-08T20:49:22.146 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-03-08T20:49:22.147 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-03-08T20:49:22.147 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: rocksdb: Options.delayed_write_rate : 16777216 2024-03-08T20:49:22.147 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: rocksdb: Options.max_total_wal_size: 0 2024-03-08T20:49:22.147 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-03-08T20:49:22.147 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: rocksdb: Options.stats_dump_period_sec: 600 2024-03-08T20:49:22.147 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: rocksdb: Options.stats_persist_period_sec: 600 2024-03-08T20:49:22.147 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-03-08T20:49:22.147 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: rocksdb: Options.max_open_files: -1 2024-03-08T20:49:22.147 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: rocksdb: Options.bytes_per_sync: 0 2024-03-08T20:49:22.147 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: rocksdb: Options.wal_bytes_per_sync: 0 2024-03-08T20:49:22.147 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: rocksdb: Options.strict_bytes_per_sync: 0 2024-03-08T20:49:22.147 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: rocksdb: Options.compaction_readahead_size: 0 2024-03-08T20:49:22.147 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: rocksdb: Options.max_background_flushes: -1 2024-03-08T20:49:22.147 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: rocksdb: Compression algorithms supported: 2024-03-08T20:49:22.148 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-03-08T20:49:22.148 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: rocksdb: kZSTD supported: 0 2024-03-08T20:49:22.148 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: rocksdb: kXpressCompression supported: 0 2024-03-08T20:49:22.148 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: rocksdb: kLZ4HCCompression supported: 1 2024-03-08T20:49:22.148 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: rocksdb: kZlibCompression supported: 1 2024-03-08T20:49:22.148 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: rocksdb: kSnappyCompression supported: 1 2024-03-08T20:49:22.148 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: rocksdb: kLZ4Compression supported: 1 2024-03-08T20:49:22.148 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: rocksdb: kBZip2Compression supported: 0 2024-03-08T20:49:22.148 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: rocksdb: Fast CRC32 supported: Supported on x86 2024-03-08T20:49:22.148 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: rocksdb: DMutex implementation: pthread_mutex_t 2024-03-08T20:49:22.148 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2024-03-08T20:49:22.148 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-03-08T20:49:22.148 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-03-08T20:49:22.149 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: rocksdb: Options.merge_operator: 2024-03-08T20:49:22.149 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: rocksdb: Options.compaction_filter: None 2024-03-08T20:49:22.149 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: rocksdb: Options.compaction_filter_factory: None 2024-03-08T20:49:22.149 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: rocksdb: Options.sst_partitioner_factory: None 2024-03-08T20:49:22.149 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: rocksdb: Options.memtable_factory: SkipListFactory 2024-03-08T20:49:22.149 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: rocksdb: Options.table_factory: BlockBasedTable 2024-03-08T20:49:22.149 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55bc3e79fa20) 2024-03-08T20:49:22.149 INFO:journalctl@ceph.mon.c.smithi204.stdout: cache_index_and_filter_blocks: 1 2024-03-08T20:49:22.149 INFO:journalctl@ceph.mon.c.smithi204.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-03-08T20:49:22.149 INFO:journalctl@ceph.mon.c.smithi204.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-03-08T20:49:22.149 INFO:journalctl@ceph.mon.c.smithi204.stdout: pin_top_level_index_and_filter: 1 2024-03-08T20:49:22.149 INFO:journalctl@ceph.mon.c.smithi204.stdout: index_type: 0 2024-03-08T20:49:22.149 INFO:journalctl@ceph.mon.c.smithi204.stdout: data_block_index_type: 0 2024-03-08T20:49:22.149 INFO:journalctl@ceph.mon.c.smithi204.stdout: index_shortening: 1 2024-03-08T20:49:22.150 INFO:journalctl@ceph.mon.c.smithi204.stdout: data_block_hash_table_util_ratio: 0.750000 2024-03-08T20:49:22.150 INFO:journalctl@ceph.mon.c.smithi204.stdout: checksum: 4 2024-03-08T20:49:22.150 INFO:journalctl@ceph.mon.c.smithi204.stdout: no_block_cache: 0 2024-03-08T20:49:22.150 INFO:journalctl@ceph.mon.c.smithi204.stdout: block_cache: 0x55bc3e7a5090 2024-03-08T20:49:22.150 INFO:journalctl@ceph.mon.c.smithi204.stdout: block_cache_name: BinnedLRUCache 2024-03-08T20:49:22.150 INFO:journalctl@ceph.mon.c.smithi204.stdout: block_cache_options: 2024-03-08T20:49:22.150 INFO:journalctl@ceph.mon.c.smithi204.stdout: capacity : 536870912 2024-03-08T20:49:22.150 INFO:journalctl@ceph.mon.c.smithi204.stdout: num_shard_bits : 4 2024-03-08T20:49:22.150 INFO:journalctl@ceph.mon.c.smithi204.stdout: strict_capacity_limit : 0 2024-03-08T20:49:22.150 INFO:journalctl@ceph.mon.c.smithi204.stdout: high_pri_pool_ratio: 0.000 2024-03-08T20:49:22.150 INFO:journalctl@ceph.mon.c.smithi204.stdout: block_cache_compressed: (nil) 2024-03-08T20:49:22.150 INFO:journalctl@ceph.mon.c.smithi204.stdout: persistent_cache: (nil) 2024-03-08T20:49:22.150 INFO:journalctl@ceph.mon.c.smithi204.stdout: block_size: 4096 2024-03-08T20:49:22.150 INFO:journalctl@ceph.mon.c.smithi204.stdout: block_size_deviation: 10 2024-03-08T20:49:22.151 INFO:journalctl@ceph.mon.c.smithi204.stdout: block_restart_interval: 16 2024-03-08T20:49:22.151 INFO:journalctl@ceph.mon.c.smithi204.stdout: index_block_restart_interval: 1 2024-03-08T20:49:22.151 INFO:journalctl@ceph.mon.c.smithi204.stdout: metadata_block_size: 4096 2024-03-08T20:49:22.151 INFO:journalctl@ceph.mon.c.smithi204.stdout: partition_filters: 0 2024-03-08T20:49:22.151 INFO:journalctl@ceph.mon.c.smithi204.stdout: use_delta_encoding: 1 2024-03-08T20:49:22.151 INFO:journalctl@ceph.mon.c.smithi204.stdout: filter_policy: bloomfilter 2024-03-08T20:49:22.151 INFO:journalctl@ceph.mon.c.smithi204.stdout: whole_key_filtering: 1 2024-03-08T20:49:22.151 INFO:journalctl@ceph.mon.c.smithi204.stdout: verify_compression: 0 2024-03-08T20:49:22.151 INFO:journalctl@ceph.mon.c.smithi204.stdout: read_amp_bytes_per_bit: 0 2024-03-08T20:49:22.151 INFO:journalctl@ceph.mon.c.smithi204.stdout: format_version: 5 2024-03-08T20:49:22.151 INFO:journalctl@ceph.mon.c.smithi204.stdout: enable_index_compression: 1 2024-03-08T20:49:22.151 INFO:journalctl@ceph.mon.c.smithi204.stdout: block_align: 0 2024-03-08T20:49:22.151 INFO:journalctl@ceph.mon.c.smithi204.stdout: max_auto_readahead_size: 262144 2024-03-08T20:49:22.151 INFO:journalctl@ceph.mon.c.smithi204.stdout: prepopulate_block_cache: 0 2024-03-08T20:49:22.152 INFO:journalctl@ceph.mon.c.smithi204.stdout: initial_auto_readahead_size: 8192 2024-03-08T20:49:22.152 INFO:journalctl@ceph.mon.c.smithi204.stdout: num_file_reads_for_auto_readahead: 2 2024-03-08T20:49:22.152 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: rocksdb: Options.write_buffer_size: 33554432 2024-03-08T20:49:22.152 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: rocksdb: Options.max_write_buffer_number: 2 2024-03-08T20:49:22.152 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: rocksdb: Options.compression: NoCompression 2024-03-08T20:49:22.152 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: rocksdb: Options.bottommost_compression: Disabled 2024-03-08T20:49:22.152 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: rocksdb: Options.prefix_extractor: nullptr 2024-03-08T20:49:22.152 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-03-08T20:49:22.152 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: rocksdb: Options.num_levels: 7 2024-03-08T20:49:22.152 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-03-08T20:49:22.152 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-03-08T20:49:22.152 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-03-08T20:49:22.152 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-03-08T20:49:22.152 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-03-08T20:49:22.153 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-03-08T20:49:22.153 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-03-08T20:49:22.153 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-03-08T20:49:22.153 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-03-08T20:49:22.153 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-03-08T20:49:22.153 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-03-08T20:49:22.153 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-03-08T20:49:22.153 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: rocksdb: Options.compression_opts.window_bits: -14 2024-03-08T20:49:22.153 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: rocksdb: Options.compression_opts.level: 32767 2024-03-08T20:49:22.153 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: rocksdb: Options.compression_opts.strategy: 0 2024-03-08T20:49:22.153 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-03-08T20:49:22.153 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-03-08T20:49:22.153 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-03-08T20:49:22.153 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-03-08T20:49:22.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: rocksdb: Options.compression_opts.enabled: false 2024-03-08T20:49:22.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-03-08T20:49:22.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-03-08T20:49:22.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-03-08T20:49:22.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-03-08T20:49:22.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: rocksdb: Options.target_file_size_base: 67108864 2024-03-08T20:49:22.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: rocksdb: Options.target_file_size_multiplier: 1 2024-03-08T20:49:22.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-03-08T20:49:22.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-03-08T20:49:22.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-03-08T20:49:22.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-03-08T20:49:22.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-03-08T20:49:22.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-03-08T20:49:22.154 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-03-08T20:49:22.155 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-03-08T20:49:22.155 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-03-08T20:49:22.155 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-03-08T20:49:22.155 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-03-08T20:49:22.155 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-03-08T20:49:22.155 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-03-08T20:49:22.155 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: rocksdb: Options.arena_block_size: 1048576 2024-03-08T20:49:22.155 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-03-08T20:49:22.155 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-03-08T20:49:22.155 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: rocksdb: Options.disable_auto_compactions: 0 2024-03-08T20:49:22.155 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-03-08T20:49:22.155 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-03-08T20:49:22.155 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-03-08T20:49:22.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-03-08T20:49:22.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-03-08T20:49:22.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-03-08T20:49:22.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-03-08T20:49:22.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-03-08T20:49:22.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-03-08T20:49:22.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-03-08T20:49:22.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: rocksdb: Options.table_properties_collectors: 2024-03-08T20:49:22.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: rocksdb: Options.inplace_update_support: 0 2024-03-08T20:49:22.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: rocksdb: Options.inplace_update_num_locks: 10000 2024-03-08T20:49:22.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-03-08T20:49:22.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-03-08T20:49:22.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: rocksdb: Options.memtable_huge_page_size: 0 2024-03-08T20:49:22.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: rocksdb: Options.bloom_locality: 0 2024-03-08T20:49:22.156 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: rocksdb: Options.max_successive_merges: 0 2024-03-08T20:49:22.157 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: rocksdb: Options.optimize_filters_for_hits: 0 2024-03-08T20:49:22.157 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: rocksdb: Options.paranoid_file_checks: 0 2024-03-08T20:49:22.157 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: rocksdb: Options.force_consistency_checks: 1 2024-03-08T20:49:22.157 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: rocksdb: Options.report_bg_io_stats: 0 2024-03-08T20:49:22.157 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: rocksdb: Options.ttl: 2592000 2024-03-08T20:49:22.157 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: rocksdb: Options.periodic_compaction_seconds: 0 2024-03-08T20:49:22.157 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-03-08T20:49:22.157 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-03-08T20:49:22.157 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: rocksdb: Options.enable_blob_files: false 2024-03-08T20:49:22.157 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: rocksdb: Options.min_blob_size: 0 2024-03-08T20:49:22.157 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: rocksdb: Options.blob_file_size: 268435456 2024-03-08T20:49:22.157 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: rocksdb: Options.blob_compression_type: NoCompression 2024-03-08T20:49:22.157 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: rocksdb: Options.enable_blob_garbage_collection: false 2024-03-08T20:49:22.157 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-03-08T20:49:22.158 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-03-08T20:49:22.158 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-03-08T20:49:22.158 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: rocksdb: Options.blob_file_starting_level: 0 2024-03-08T20:49:22.158 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-03-08T20:49:22.158 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-03-08T20:49:22.158 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-03-08T20:49:22.158 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 279f4cb0-450e-44df-acf5-d75a155dfe34 2024-03-08T20:49:22.158 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: rocksdb: EVENT_LOG_v1 {"time_micros": 1709930961779861, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-03-08T20:49:22.158 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-03-08T20:49:22.158 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 systemd[1]: Started Ceph mon.c for dc20f0f8-dd8c-11ee-95c6-87774f69a715. 2024-03-08T20:49:22.158 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: rocksdb: EVENT_LOG_v1 {"time_micros": 1709930961781126, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1622, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1709930961, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "279f4cb0-450e-44df-acf5-d75a155dfe34", "db_session_id": "8SB1DBLN886US22PQMHV", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-03-08T20:49:22.158 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: rocksdb: EVENT_LOG_v1 {"time_micros": 1709930961781475, "job": 1, "event": "recovery_finished"} 2024-03-08T20:49:22.158 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-03-08T20:49:22.159 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-03-08T20:49:22.159 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55bc3e898000 2024-03-08T20:49:22.159 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: rocksdb: DB pointer 0x55bc3e882000 2024-03-08T20:49:22.159 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-03-08T20:49:22.159 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-03-08T20:49:22.159 INFO:journalctl@ceph.mon.c.smithi204.stdout: ** DB Stats ** 2024-03-08T20:49:22.159 INFO:journalctl@ceph.mon.c.smithi204.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-08T20:49:22.159 INFO:journalctl@ceph.mon.c.smithi204.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-03-08T20:49:22.159 INFO:journalctl@ceph.mon.c.smithi204.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-08T20:49:22.159 INFO:journalctl@ceph.mon.c.smithi204.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-08T20:49:22.159 INFO:journalctl@ceph.mon.c.smithi204.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-03-08T20:49:22.159 INFO:journalctl@ceph.mon.c.smithi204.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-08T20:49:22.159 INFO:journalctl@ceph.mon.c.smithi204.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-08T20:49:22.159 INFO:journalctl@ceph.mon.c.smithi204.stdout: 2024-03-08T20:49:22.160 INFO:journalctl@ceph.mon.c.smithi204.stdout: ** Compaction Stats [default] ** 2024-03-08T20:49:22.160 INFO:journalctl@ceph.mon.c.smithi204.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-08T20:49:22.160 INFO:journalctl@ceph.mon.c.smithi204.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-03-08T20:49:22.160 INFO:journalctl@ceph.mon.c.smithi204.stdout: L0 1/0 1.58 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 1.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-08T20:49:22.160 INFO:journalctl@ceph.mon.c.smithi204.stdout: Sum 1/0 1.58 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 1.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-08T20:49:22.160 INFO:journalctl@ceph.mon.c.smithi204.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 1.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-08T20:49:22.160 INFO:journalctl@ceph.mon.c.smithi204.stdout: 2024-03-08T20:49:22.160 INFO:journalctl@ceph.mon.c.smithi204.stdout: ** Compaction Stats [default] ** 2024-03-08T20:49:22.160 INFO:journalctl@ceph.mon.c.smithi204.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-08T20:49:22.160 INFO:journalctl@ceph.mon.c.smithi204.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-08T20:49:22.160 INFO:journalctl@ceph.mon.c.smithi204.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-08T20:49:22.160 INFO:journalctl@ceph.mon.c.smithi204.stdout: 2024-03-08T20:49:22.160 INFO:journalctl@ceph.mon.c.smithi204.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-03-08T20:49:22.160 INFO:journalctl@ceph.mon.c.smithi204.stdout: 2024-03-08T20:49:22.160 INFO:journalctl@ceph.mon.c.smithi204.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-08T20:49:22.161 INFO:journalctl@ceph.mon.c.smithi204.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-03-08T20:49:22.161 INFO:journalctl@ceph.mon.c.smithi204.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-03-08T20:49:22.161 INFO:journalctl@ceph.mon.c.smithi204.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-03-08T20:49:22.161 INFO:journalctl@ceph.mon.c.smithi204.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-03-08T20:49:22.161 INFO:journalctl@ceph.mon.c.smithi204.stdout: AddFile(Keys): cumulative 0, interval 0 2024-03-08T20:49:22.161 INFO:journalctl@ceph.mon.c.smithi204.stdout: Cumulative compaction: 0.00 GB write, 0.18 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-08T20:49:22.161 INFO:journalctl@ceph.mon.c.smithi204.stdout: Interval compaction: 0.00 GB write, 0.18 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-08T20:49:22.161 INFO:journalctl@ceph.mon.c.smithi204.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-03-08T20:49:22.161 INFO:journalctl@ceph.mon.c.smithi204.stdout: Block cache BinnedLRUCache@0x55bc3e7a5090#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.3e-05 secs_since: 0 2024-03-08T20:49:22.161 INFO:journalctl@ceph.mon.c.smithi204.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-03-08T20:49:22.161 INFO:journalctl@ceph.mon.c.smithi204.stdout: 2024-03-08T20:49:22.161 INFO:journalctl@ceph.mon.c.smithi204.stdout: ** File Read Latency Histogram By Level [default] ** 2024-03-08T20:49:22.161 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-03-08T20:49:22.162 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: using public_addr v2:172.21.15.204:0/0 -> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-03-08T20:49:22.162 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: starting mon.c rank -1 at public addrs [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] at bind addrs [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid dc20f0f8-dd8c-11ee-95c6-87774f69a715 2024-03-08T20:49:22.162 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: mon.c@-1(???) e0 preinit fsid dc20f0f8-dd8c-11ee-95c6-87774f69a715 2024-03-08T20:49:22.162 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: mon.c@-1(synchronizing).mds e1 new map 2024-03-08T20:49:22.162 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: mon.c@-1(synchronizing).mds e1 print_map 2024-03-08T20:49:22.162 INFO:journalctl@ceph.mon.c.smithi204.stdout: e1 2024-03-08T20:49:22.162 INFO:journalctl@ceph.mon.c.smithi204.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-03-08T20:49:22.162 INFO:journalctl@ceph.mon.c.smithi204.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-03-08T20:49:22.162 INFO:journalctl@ceph.mon.c.smithi204.stdout: legacy client fscid: -1 2024-03-08T20:49:22.162 INFO:journalctl@ceph.mon.c.smithi204.stdout: 2024-03-08T20:49:22.163 INFO:journalctl@ceph.mon.c.smithi204.stdout: No filesystems configured 2024-03-08T20:49:22.163 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-03-08T20:49:22.163 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-03-08T20:49:22.163 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-03-08T20:49:22.163 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-03-08T20:49:22.163 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-03-08T20:49:22.163 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-03-08T20:49:22.163 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-03-08T20:49:22.163 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-03-08T20:49:22.163 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-03-08T20:49:22.163 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-03-08T20:49:22.163 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T20:49:22.163 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi149", "target": ["mon-mgr", ""]}]: dispatch 2024-03-08T20:49:22.163 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T20:49:22.164 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: Deploying cephadm binary to smithi149 2024-03-08T20:49:22.164 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T20:49:22.164 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T20:49:22.164 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:49:22.164 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: Added host smithi149 2024-03-08T20:49:22.164 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T20:49:22.164 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:49:22.164 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T20:49:22.164 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:49:22.164 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T20:49:22.164 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:49:22.164 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-08T20:49:22.164 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T20:49:22.165 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T20:49:22.165 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi204", "target": ["mon-mgr", ""]}]: dispatch 2024-03-08T20:49:22.165 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: Deploying cephadm binary to smithi204 2024-03-08T20:49:22.165 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:49:22.165 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:49:22.165 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:49:22.165 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:49:22.165 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi149", "name": "osd_memory_target"}]: dispatch 2024-03-08T20:49:22.165 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:49:22.165 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T20:49:22.165 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: Updating smithi149:/etc/ceph/ceph.conf 2024-03-08T20:49:22.165 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T20:49:22.165 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: Updating smithi149:/var/lib/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/config/ceph.conf 2024-03-08T20:49:22.165 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T20:49:22.165 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: Updating smithi149:/etc/ceph/ceph.client.admin.keyring 2024-03-08T20:49:22.166 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T20:49:22.166 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:49:22.166 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: Updating smithi149:/var/lib/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/config/ceph.client.admin.keyring 2024-03-08T20:49:22.166 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: Added host smithi204 2024-03-08T20:49:22.166 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:49:22.166 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:49:22.166 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:49:22.166 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T20:49:22.166 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T20:49:22.166 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:49:22.166 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:49:22.166 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-08T20:49:22.166 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T20:49:22.166 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:49:22.167 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T20:49:22.167 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: from='client.? 172.21.15.107:0/1054927658' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-03-08T20:49:22.167 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T20:49:22.167 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: from='client.? 172.21.15.107:0/1054927658' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-03-08T20:49:22.167 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: osdmap e4: 0 total, 0 up, 0 in 2024-03-08T20:49:22.167 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T20:49:22.167 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:49:22.167 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:49:22.167 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:49:22.167 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:49:22.167 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi204", "name": "osd_memory_target"}]: dispatch 2024-03-08T20:49:22.167 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:49:22.167 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T20:49:22.168 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: Updating smithi204:/etc/ceph/ceph.conf 2024-03-08T20:49:22.168 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: Updating smithi204:/var/lib/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/config/ceph.conf 2024-03-08T20:49:22.168 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: Updating smithi204:/etc/ceph/ceph.client.admin.keyring 2024-03-08T20:49:22.168 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T20:49:22.168 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: Updating smithi204:/var/lib/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/config/ceph.client.admin.keyring 2024-03-08T20:49:22.168 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:49:22.168 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:49:22.168 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:49:22.168 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:49:22.168 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-08T20:49:22.168 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:49:22.168 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi107:172.21.15.107=a;smithi149:172.21.15.149=b;smithi204:172.21.15.204=c", "target": ["mon-mgr", ""]}]: dispatch 2024-03-08T20:49:22.168 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: Saving service mon spec with placement smithi107:172.21.15.107=a;smithi149:172.21.15.149=b;smithi204:172.21.15.204=c;count:3 2024-03-08T20:49:22.168 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: Deploying daemon mon.c on smithi204 2024-03-08T20:49:22.168 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T20:49:22.169 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T20:49:22.169 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: from='client.? 172.21.15.204:0/4167819322' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-08T20:49:22.169 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T20:49:22.169 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:21 smithi204 ceph-mon[34935]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-03-08T20:49:22.197 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/mon.c/config 2024-03-08T20:49:27.026 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:27.033+0000 7ff491e00700 1 -- 172.21.15.204:0/1746889466 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff48c107f60 msgr2=0x7ff474005610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:49:27.026 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:27.033+0000 7ff491e00700 1 --2- 172.21.15.204:0/1746889466 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff48c107f60 0x7ff474005610 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7ff480005fa0 tx=0x7ff48000ffc0 comp rx=0 tx=0).stop 2024-03-08T20:49:27.026 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:27.034+0000 7ff491e00700 1 -- 172.21.15.204:0/1746889466 shutdown_connections 2024-03-08T20:49:27.026 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:27.034+0000 7ff491e00700 1 --2- 172.21.15.204:0/1746889466 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff48c107f60 0x7ff474005610 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:49:27.026 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:27.034+0000 7ff491e00700 1 -- 172.21.15.204:0/1746889466 >> 172.21.15.204:0/1746889466 conn(0x7ff48c0774b0 msgr2=0x7ff48c0798d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:49:27.027 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:27.034+0000 7ff491e00700 1 -- 172.21.15.204:0/1746889466 shutdown_connections 2024-03-08T20:49:27.027 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:27.034+0000 7ff491e00700 1 -- 172.21.15.204:0/1746889466 wait complete. 2024-03-08T20:49:27.029 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:27.037+0000 7ff491e00700 1 Processor -- start 2024-03-08T20:49:27.029 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:27.037+0000 7ff491e00700 1 -- start start 2024-03-08T20:49:27.030 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:27.038+0000 7ff491e00700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff48c107f60 0x7ff48c1a3440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:49:27.031 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:27.038+0000 7ff491e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff48c1a3980 0x7ff48c19d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:49:27.031 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:27.038+0000 7ff491e00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff48c10fc40 con 0x7ff48c1a3980 2024-03-08T20:49:27.031 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:27.038+0000 7ff491e00700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff48c10fac0 con 0x7ff48c107f60 2024-03-08T20:49:27.031 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:27.039+0000 7ff490a00700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff48c107f60 0x7ff48c1a3440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:49:27.031 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:27.039+0000 7ff490a00700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff48c107f60 0x7ff48c1a3440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:33692/0 (socket says 172.21.15.204:33692) 2024-03-08T20:49:27.031 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:27.039+0000 7ff490a00700 1 -- 172.21.15.204:0/4237351079 learned_addr learned my addr 172.21.15.204:0/4237351079 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-03-08T20:49:27.031 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:27.039+0000 7ff48be00700 1 --2- 172.21.15.204:0/4237351079 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff48c1a3980 0x7ff48c19d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:49:27.032 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:27.039+0000 7ff490a00700 1 -- 172.21.15.204:0/4237351079 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff48c107f60 msgr2=0x7ff48c1a3440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-03-08T20:49:27.032 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:27.039+0000 7ff490a00700 1 -- 172.21.15.204:0/4237351079 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff48c107f60 msgr2=0x7ff48c1a3440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-08T20:49:27.032 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:27.039+0000 7ff490a00700 1 --2- 172.21.15.204:0/4237351079 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff48c107f60 0x7ff48c1a3440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-08T20:49:27.032 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:27.039+0000 7ff490a00700 1 --2- 172.21.15.204:0/4237351079 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff48c107f60 0x7ff48c1a3440 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-08T20:49:27.032 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:27.040+0000 7ff48be00700 1 -- 172.21.15.204:0/4237351079 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff48c107f60 msgr2=0x7ff48c1a3440 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-08T20:49:27.032 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:27.040+0000 7ff48be00700 1 --2- 172.21.15.204:0/4237351079 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff48c107f60 0x7ff48c1a3440 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:49:27.032 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:27.040+0000 7ff48be00700 1 -- 172.21.15.204:0/4237351079 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff48c1a3fc0 con 0x7ff48c1a3980 2024-03-08T20:49:27.032 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:27.040+0000 7ff48be00700 1 --2- 172.21.15.204:0/4237351079 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff48c1a3980 0x7ff48c19d600 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7ff47800b4d0 tx=0x7ff47800b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:49:27.033 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:27.041+0000 7ff489600700 1 -- 172.21.15.204:0/4237351079 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7ff478011630 con 0x7ff48c1a3980 2024-03-08T20:49:27.033 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:27.041+0000 7ff489600700 1 -- 172.21.15.204:0/4237351079 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7ff478011c80 con 0x7ff48c1a3980 2024-03-08T20:49:27.033 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:27.041+0000 7ff489600700 1 -- 172.21.15.204:0/4237351079 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7ff478010cc0 con 0x7ff48c1a3980 2024-03-08T20:49:27.033 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:27.041+0000 7ff491e00700 1 -- 172.21.15.204:0/4237351079 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff48c19db40 con 0x7ff48c1a3980 2024-03-08T20:49:27.033 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:27.041+0000 7ff491e00700 1 -- 172.21.15.204:0/4237351079 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff48c19e0d0 con 0x7ff48c1a3980 2024-03-08T20:49:27.034 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:27.042+0000 7ff491e00700 1 -- 172.21.15.204:0/4237351079 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff48c19e440 con 0x7ff48c1a3980 2024-03-08T20:49:27.034 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:27.043+0000 7ff489600700 1 -- 172.21.15.204:0/4237351079 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 13) v1 ==== 46163+0+0 (secure 0 0 0) 0x7ff478018020 con 0x7ff48c1a3980 2024-03-08T20:49:27.035 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:27.043+0000 7ff489600700 1 --2- 172.21.15.204:0/4237351079 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7ff46c038dd0 0x7ff46c03b290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:49:27.035 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:27.043+0000 7ff489600700 1 -- 172.21.15.204:0/4237351079 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7ff47804b400 con 0x7ff48c1a3980 2024-03-08T20:49:27.035 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:27.044+0000 7ff490a00700 1 --2- 172.21.15.204:0/4237351079 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7ff46c038dd0 0x7ff46c03b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:49:27.036 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:27.045+0000 7ff490a00700 1 --2- 172.21.15.204:0/4237351079 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7ff46c038dd0 0x7ff46c03b290 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7ff48000fb20 tx=0x7ff48001d040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:49:27.049 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:27.058+0000 7ff489600700 1 -- 172.21.15.204:0/4237351079 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7ff47801f070 con 0x7ff48c1a3980 2024-03-08T20:49:27.227 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:49:26 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-08T20:49:27.227 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:49:26 smithi107 ceph-mon[31381]: mon.a calling monitor election 2024-03-08T20:49:27.227 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:49:26 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-08T20:49:27.227 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:49:26 smithi107 ceph-mon[31381]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T20:49:27.227 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:49:26 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-08T20:49:27.227 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:49:26 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-08T20:49:27.227 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:49:26 smithi107 ceph-mon[31381]: mon.c calling monitor election 2024-03-08T20:49:27.227 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:49:26 smithi107 ceph-mon[31381]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T20:49:27.227 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:49:26 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-08T20:49:27.228 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:49:26 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-08T20:49:27.228 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:49:26 smithi107 ceph-mon[31381]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T20:49:27.228 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:49:26 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-08T20:49:27.228 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:49:26 smithi107 ceph-mon[31381]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-08T20:49:27.228 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:49:26 smithi107 ceph-mon[31381]: monmap e2: 2 mons at {a=[v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0],c=[v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-08T20:49:27.228 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:49:26 smithi107 ceph-mon[31381]: fsmap 2024-03-08T20:49:27.228 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:49:26 smithi107 ceph-mon[31381]: osdmap e4: 0 total, 0 up, 0 in 2024-03-08T20:49:27.228 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:49:26 smithi107 ceph-mon[31381]: mgrmap e13: x(active, since 66s) 2024-03-08T20:49:27.228 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:49:26 smithi107 ceph-mon[31381]: overall HEALTH_OK 2024-03-08T20:49:27.228 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:49:26 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:49:27.229 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:49:26 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:49:27.372 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:27.380+0000 7ff491e00700 1 -- 172.21.15.204:0/4237351079 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7ff48c19eb30 con 0x7ff48c1a3980 2024-03-08T20:49:27.373 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:26 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-08T20:49:27.373 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:26 smithi204 ceph-mon[34935]: mon.a calling monitor election 2024-03-08T20:49:27.373 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:26 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-08T20:49:27.373 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:26 smithi204 ceph-mon[34935]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T20:49:27.374 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:26 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-08T20:49:27.374 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:26 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-08T20:49:27.374 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:26 smithi204 ceph-mon[34935]: mon.c calling monitor election 2024-03-08T20:49:27.374 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:26 smithi204 ceph-mon[34935]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T20:49:27.374 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:26 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-08T20:49:27.375 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:26 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-08T20:49:27.375 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:26 smithi204 ceph-mon[34935]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T20:49:27.375 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:26 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-08T20:49:27.375 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:26 smithi204 ceph-mon[34935]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-08T20:49:27.375 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:26 smithi204 ceph-mon[34935]: monmap e2: 2 mons at {a=[v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0],c=[v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-08T20:49:27.375 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:26 smithi204 ceph-mon[34935]: fsmap 2024-03-08T20:49:27.376 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:26 smithi204 ceph-mon[34935]: osdmap e4: 0 total, 0 up, 0 in 2024-03-08T20:49:27.376 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:26 smithi204 ceph-mon[34935]: mgrmap e13: x(active, since 66s) 2024-03-08T20:49:27.376 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:26 smithi204 ceph-mon[34935]: overall HEALTH_OK 2024-03-08T20:49:27.376 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:26 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:49:27.376 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:26 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:49:27.377 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:27.381+0000 7ff489600700 1 -- 172.21.15.204:0/4237351079 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) v1 ==== 95+0+1019 (secure 0 0 0) 0x7ff47800f540 con 0x7ff48c1a3980 2024-03-08T20:49:27.377 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-03-08T20:49:27.378 INFO:teuthology.orchestra.run.smithi204.stdout:{"epoch":2,"fsid":"dc20f0f8-dd8c-11ee-95c6-87774f69a715","modified":"2024-03-08T20:49:21.832570Z","created":"2024-03-08T20:47:00.037201Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:3300","nonce":0},{"type":"v1","addr":"172.21.15.107:6789","nonce":0}]},"addr":"172.21.15.107:6789/0","public_addr":"172.21.15.107:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:3300","nonce":0},{"type":"v1","addr":"172.21.15.204:6789","nonce":0}]},"addr":"172.21.15.204:6789/0","public_addr":"172.21.15.204:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-03-08T20:49:27.379 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:27.384+0000 7ff491e00700 1 -- 172.21.15.204:0/4237351079 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7ff46c038dd0 msgr2=0x7ff46c03b290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:49:27.379 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:27.384+0000 7ff491e00700 1 --2- 172.21.15.204:0/4237351079 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7ff46c038dd0 0x7ff46c03b290 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7ff48000fb20 tx=0x7ff48001d040 comp rx=0 tx=0).stop 2024-03-08T20:49:27.380 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:27.384+0000 7ff491e00700 1 -- 172.21.15.204:0/4237351079 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff48c1a3980 msgr2=0x7ff48c19d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:49:27.380 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:27.384+0000 7ff491e00700 1 --2- 172.21.15.204:0/4237351079 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff48c1a3980 0x7ff48c19d600 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7ff47800b4d0 tx=0x7ff47800b890 comp rx=0 tx=0).stop 2024-03-08T20:49:27.380 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:27.384+0000 7ff491e00700 1 -- 172.21.15.204:0/4237351079 shutdown_connections 2024-03-08T20:49:27.380 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:27.384+0000 7ff491e00700 1 --2- 172.21.15.204:0/4237351079 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff48c107f60 0x7ff48c1a3440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:49:27.380 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:27.384+0000 7ff491e00700 1 --2- 172.21.15.204:0/4237351079 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7ff46c038dd0 0x7ff46c03b290 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:49:27.380 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:27.384+0000 7ff491e00700 1 --2- 172.21.15.204:0/4237351079 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff48c1a3980 0x7ff48c19d600 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:49:27.381 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:27.384+0000 7ff491e00700 1 -- 172.21.15.204:0/4237351079 >> 172.21.15.204:0/4237351079 conn(0x7ff48c0774b0 msgr2=0x7ff48c078650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:49:27.381 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:27.384+0000 7ff491e00700 1 -- 172.21.15.204:0/4237351079 shutdown_connections 2024-03-08T20:49:27.381 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:27.386+0000 7ff491e00700 1 -- 172.21.15.204:0/4237351079 wait complete. 2024-03-08T20:49:27.381 INFO:teuthology.orchestra.run.smithi204.stderr:dumped monmap epoch 2 2024-03-08T20:49:28.044 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:27 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:49:28.044 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:27 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-08T20:49:28.044 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:27 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:49:28.044 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:27 smithi204 ceph-mon[34935]: Deploying daemon mon.b on smithi149 2024-03-08T20:49:28.044 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:27 smithi204 ceph-mon[34935]: from='client.? 172.21.15.204:0/4237351079' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-08T20:49:28.044 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:27 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-08T20:49:28.226 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:49:27 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:49:28.226 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:49:27 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-08T20:49:28.226 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:49:27 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:49:28.226 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:49:27 smithi107 ceph-mon[31381]: Deploying daemon mon.b on smithi149 2024-03-08T20:49:28.226 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:49:27 smithi107 ceph-mon[31381]: from='client.? 172.21.15.204:0/4237351079' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-08T20:49:28.226 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:49:27 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-08T20:49:29.104 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-03-08T20:49:29.104 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dc20f0f8-dd8c-11ee-95c6-87774f69a715 -- ceph mon dump -f json 2024-03-08T20:49:29.188 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:28 smithi204 ceph-mon[34935]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T20:49:29.226 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:49:28 smithi107 ceph-mon[31381]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T20:49:29.552 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/mon.c/config 2024-03-08T20:49:31.362 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:31.369+0000 7f79a0c00700 1 -- 172.21.15.204:0/1977739701 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f799c102b00 msgr2=0x7f799c102ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:49:31.362 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:31.369+0000 7f79a0c00700 1 --2- 172.21.15.204:0/1977739701 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f799c102b00 0x7f799c102ee0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f798c009910 tx=0x7f798c009c20 comp rx=0 tx=0).stop 2024-03-08T20:49:31.362 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:31.369+0000 7f79a0c00700 1 -- 172.21.15.204:0/1977739701 shutdown_connections 2024-03-08T20:49:31.362 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:31.369+0000 7f79a0c00700 1 --2- 172.21.15.204:0/1977739701 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f799c102b00 0x7f799c102ee0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:49:31.362 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:31.369+0000 7f79a0c00700 1 -- 172.21.15.204:0/1977739701 >> 172.21.15.204:0/1977739701 conn(0x7f799c0fc2f0 msgr2=0x7f799c0fe710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:49:31.363 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:31.369+0000 7f79a0c00700 1 -- 172.21.15.204:0/1977739701 shutdown_connections 2024-03-08T20:49:31.363 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:31.369+0000 7f79a0c00700 1 -- 172.21.15.204:0/1977739701 wait complete. 2024-03-08T20:49:31.363 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:31.370+0000 7f79a0c00700 1 Processor -- start 2024-03-08T20:49:31.363 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:31.370+0000 7f79a0c00700 1 -- start start 2024-03-08T20:49:31.364 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:31.371+0000 7f79a0c00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f799c102b00 0x7f799c197520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:49:31.364 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:31.371+0000 7f79a0c00700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f799c197a60 0x7f799c19bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:49:31.364 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:31.371+0000 7f79a0c00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f799c10be30 con 0x7f799c102b00 2024-03-08T20:49:31.364 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:31.371+0000 7f79a0c00700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f799c10bcb0 con 0x7f799c197a60 2024-03-08T20:49:31.365 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:31.371+0000 7f7999400700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f799c197a60 0x7f799c19bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:49:31.365 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:31.371+0000 7f7999400700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f799c197a60 0x7f799c19bdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:44788/0 (socket says 172.21.15.204:44788) 2024-03-08T20:49:31.365 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:31.371+0000 7f7999400700 1 -- 172.21.15.204:0/3772280724 learned_addr learned my addr 172.21.15.204:0/3772280724 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-03-08T20:49:31.365 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:31.371+0000 7f7999e00700 1 --2- 172.21.15.204:0/3772280724 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f799c102b00 0x7f799c197520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:49:31.365 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:31.372+0000 7f7999400700 1 -- 172.21.15.204:0/3772280724 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f799c197a60 msgr2=0x7f799c19bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-03-08T20:49:31.365 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:31.372+0000 7f7999400700 1 -- 172.21.15.204:0/3772280724 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f799c197a60 msgr2=0x7f799c19bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-08T20:49:31.366 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:31.372+0000 7f7999400700 1 --2- 172.21.15.204:0/3772280724 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f799c197a60 0x7f799c19bdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-08T20:49:31.366 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:31.372+0000 7f7999400700 1 --2- 172.21.15.204:0/3772280724 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f799c197a60 0x7f799c19bdf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-08T20:49:31.366 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:31.372+0000 7f7999e00700 1 -- 172.21.15.204:0/3772280724 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f799c197a60 msgr2=0x7f799c19bdf0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-08T20:49:31.366 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:31.372+0000 7f7999e00700 1 --2- 172.21.15.204:0/3772280724 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f799c197a60 0x7f799c19bdf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:49:31.366 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:31.372+0000 7f7999e00700 1 -- 172.21.15.204:0/3772280724 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f799c1980a0 con 0x7f799c102b00 2024-03-08T20:49:31.366 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:31.373+0000 7f7999e00700 1 --2- 172.21.15.204:0/3772280724 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f799c102b00 0x7f799c197520 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f798c014000 tx=0x7f798c005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:49:31.367 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:31.373+0000 7f7992a00700 1 -- 172.21.15.204:0/3772280724 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7f798c005dc0 con 0x7f799c102b00 2024-03-08T20:49:31.367 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:31.373+0000 7f79a0c00700 1 -- 172.21.15.204:0/3772280724 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f799c19c330 con 0x7f799c102b00 2024-03-08T20:49:31.367 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:31.373+0000 7f7992a00700 1 -- 172.21.15.204:0/3772280724 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f798c00be10 con 0x7f799c102b00 2024-03-08T20:49:31.367 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:31.373+0000 7f7992a00700 1 -- 172.21.15.204:0/3772280724 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7f798c022e30 con 0x7f799c102b00 2024-03-08T20:49:31.367 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:31.373+0000 7f79a0c00700 1 -- 172.21.15.204:0/3772280724 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f799c19c830 con 0x7f799c102b00 2024-03-08T20:49:31.367 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:31.374+0000 7f79a0c00700 1 -- 172.21.15.204:0/3772280724 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f799c06a840 con 0x7f799c102b00 2024-03-08T20:49:31.368 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:31.375+0000 7f7992a00700 1 -- 172.21.15.204:0/3772280724 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 13) v1 ==== 46163+0+0 (secure 0 0 0) 0x7f798c02a070 con 0x7f799c102b00 2024-03-08T20:49:31.368 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:31.375+0000 7f7992a00700 1 --2- 172.21.15.204:0/3772280724 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f7980038d60 0x7f798003b220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:49:31.368 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:31.375+0000 7f7992a00700 1 -- 172.21.15.204:0/3772280724 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f798c01e030 con 0x7f799c102b00 2024-03-08T20:49:31.368 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:31.376+0000 7f7999400700 1 --2- 172.21.15.204:0/3772280724 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f7980038d60 0x7f798003b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:49:31.369 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:31.377+0000 7f7999400700 1 --2- 172.21.15.204:0/3772280724 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f7980038d60 0x7f798003b220 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f7988009180 tx=0x7f7988008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:49:31.384 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:31.391+0000 7f7992a00700 1 -- 172.21.15.204:0/3772280724 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f798c013070 con 0x7f799c102b00 2024-03-08T20:49:31.476 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:49:31 smithi107 ceph-mon[31381]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T20:49:31.637 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:31 smithi204 ceph-mon[34935]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T20:49:31.689 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:31.696+0000 7f79a0c00700 1 -- 172.21.15.204:0/3772280724 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f799c19cae0 con 0x7f799c102b00 2024-03-08T20:49:31.689 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:31.697+0000 7f7992a00700 1 -- 172.21.15.204:0/3772280724 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) v1 ==== 95+0+1019 (secure 0 0 0) 0x7f798c022470 con 0x7f799c102b00 2024-03-08T20:49:31.690 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-03-08T20:49:31.690 INFO:teuthology.orchestra.run.smithi204.stdout:{"epoch":2,"fsid":"dc20f0f8-dd8c-11ee-95c6-87774f69a715","modified":"2024-03-08T20:49:21.832570Z","created":"2024-03-08T20:47:00.037201Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:3300","nonce":0},{"type":"v1","addr":"172.21.15.107:6789","nonce":0}]},"addr":"172.21.15.107:6789/0","public_addr":"172.21.15.107:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:3300","nonce":0},{"type":"v1","addr":"172.21.15.204:6789","nonce":0}]},"addr":"172.21.15.204:6789/0","public_addr":"172.21.15.204:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-03-08T20:49:31.693 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:31.700+0000 7f79a0c00700 1 -- 172.21.15.204:0/3772280724 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f7980038d60 msgr2=0x7f798003b220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:49:31.693 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:31.700+0000 7f79a0c00700 1 --2- 172.21.15.204:0/3772280724 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f7980038d60 0x7f798003b220 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f7988009180 tx=0x7f7988008040 comp rx=0 tx=0).stop 2024-03-08T20:49:31.693 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:31.700+0000 7f79a0c00700 1 -- 172.21.15.204:0/3772280724 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f799c102b00 msgr2=0x7f799c197520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:49:31.694 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:31.700+0000 7f79a0c00700 1 --2- 172.21.15.204:0/3772280724 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f799c102b00 0x7f799c197520 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f798c014000 tx=0x7f798c005030 comp rx=0 tx=0).stop 2024-03-08T20:49:31.694 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:31.700+0000 7f79a0c00700 1 -- 172.21.15.204:0/3772280724 shutdown_connections 2024-03-08T20:49:31.694 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:31.700+0000 7f79a0c00700 1 --2- 172.21.15.204:0/3772280724 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f7980038d60 0x7f798003b220 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:49:31.694 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:31.701+0000 7f79a0c00700 1 --2- 172.21.15.204:0/3772280724 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f799c102b00 0x7f799c197520 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:49:31.694 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:31.701+0000 7f79a0c00700 1 --2- 172.21.15.204:0/3772280724 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f799c197a60 0x7f799c19bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:49:31.694 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:31.701+0000 7f79a0c00700 1 -- 172.21.15.204:0/3772280724 >> 172.21.15.204:0/3772280724 conn(0x7f799c0fc2f0 msgr2=0x7f799c1058c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:49:31.694 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:31.701+0000 7f79a0c00700 1 -- 172.21.15.204:0/3772280724 shutdown_connections 2024-03-08T20:49:31.694 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:31.701+0000 7f79a0c00700 1 -- 172.21.15.204:0/3772280724 wait complete. 2024-03-08T20:49:31.695 INFO:teuthology.orchestra.run.smithi204.stderr:dumped monmap epoch 2 2024-03-08T20:49:32.273 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:32 smithi204 ceph-mon[34935]: from='client.? 172.21.15.204:0/3772280724' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-08T20:49:32.476 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:49:32 smithi107 ceph-mon[31381]: from='client.? 172.21.15.204:0/3772280724' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-08T20:49:32.523 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 podman[35023]: 2024-03-08 20:49:32.25991104 +0000 UTC m=+0.138631960 container create 8cecc7919a787e1ca250082d39d07931dc0f0e8e336b6b9c293973f59ad75088 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mon-b, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, RELEASE=rishabh-2024mar8-7d63482, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-08T20:49:32.524 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 podman[35023]: 2024-03-08 20:49:32.530494195 +0000 UTC m=+0.409215141 container init 8cecc7919a787e1ca250082d39d07931dc0f0e8e336b6b9c293973f59ad75088 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mon-b, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, RELEASE=rishabh-2024mar8-7d63482, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-08T20:49:32.775 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 podman[35023]: 2024-03-08 20:49:32.541001023 +0000 UTC m=+0.419721979 container start 8cecc7919a787e1ca250082d39d07931dc0f0e8e336b6b9c293973f59ad75088 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mon-b, org.label-schema.build-date=20240307, RELEASE=rishabh-2024mar8-7d63482, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-08T20:49:32.775 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: set uid:gid to 167:167 (ceph:ceph) 2024-03-08T20:49:32.775 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: ceph version 19.0.0-1965-g7d63482a (7d63482aa89628d733e298984a762389b625c761) squid (dev), process ceph-mon, pid 2 2024-03-08T20:49:32.775 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: pidfile_write: ignore empty --pid-file 2024-03-08T20:49:32.775 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: load: jerasure load: lrc 2024-03-08T20:49:32.775 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: rocksdb: RocksDB version: 7.9.2 2024-03-08T20:49:32.775 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: rocksdb: Git sha 0 2024-03-08T20:49:32.775 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: rocksdb: Compile date 2024-03-08 15:06:51 2024-03-08T20:49:32.776 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: rocksdb: DB SUMMARY 2024-03-08T20:49:32.776 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: rocksdb: DB Session ID: E9EJ9WZQMB2M9GT2A90H 2024-03-08T20:49:32.776 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: rocksdb: CURRENT file: CURRENT 2024-03-08T20:49:32.776 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: rocksdb: IDENTITY file: IDENTITY 2024-03-08T20:49:32.776 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-03-08T20:49:32.776 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2024-03-08T20:49:32.776 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 636 ; 2024-03-08T20:49:32.776 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: rocksdb: Options.error_if_exists: 0 2024-03-08T20:49:32.776 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: rocksdb: Options.create_if_missing: 0 2024-03-08T20:49:32.776 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: rocksdb: Options.paranoid_checks: 1 2024-03-08T20:49:32.776 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: rocksdb: Options.flush_verify_memtable_count: 1 2024-03-08T20:49:32.776 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-03-08T20:49:32.776 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-03-08T20:49:32.777 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: rocksdb: Options.env: 0x564915deeb20 2024-03-08T20:49:32.777 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: rocksdb: Options.fs: PosixFileSystem 2024-03-08T20:49:32.777 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: rocksdb: Options.info_log: 0x564916be3920 2024-03-08T20:49:32.777 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: rocksdb: Options.max_file_opening_threads: 16 2024-03-08T20:49:32.777 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: rocksdb: Options.statistics: (nil) 2024-03-08T20:49:32.777 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: rocksdb: Options.use_fsync: 0 2024-03-08T20:49:32.777 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: rocksdb: Options.max_log_file_size: 0 2024-03-08T20:49:32.777 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-03-08T20:49:32.777 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: rocksdb: Options.log_file_time_to_roll: 0 2024-03-08T20:49:32.777 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: rocksdb: Options.keep_log_file_num: 1000 2024-03-08T20:49:32.777 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: rocksdb: Options.recycle_log_file_num: 0 2024-03-08T20:49:32.777 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: rocksdb: Options.allow_fallocate: 1 2024-03-08T20:49:32.778 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: rocksdb: Options.allow_mmap_reads: 0 2024-03-08T20:49:32.778 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: rocksdb: Options.allow_mmap_writes: 0 2024-03-08T20:49:32.778 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: rocksdb: Options.use_direct_reads: 0 2024-03-08T20:49:32.779 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-03-08T20:49:32.779 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: rocksdb: Options.create_missing_column_families: 0 2024-03-08T20:49:32.779 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: rocksdb: Options.db_log_dir: 2024-03-08T20:49:32.779 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: rocksdb: Options.wal_dir: 2024-03-08T20:49:32.779 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: rocksdb: Options.table_cache_numshardbits: 6 2024-03-08T20:49:32.779 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: rocksdb: Options.WAL_ttl_seconds: 0 2024-03-08T20:49:32.779 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: rocksdb: Options.WAL_size_limit_MB: 0 2024-03-08T20:49:32.779 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-03-08T20:49:32.779 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-03-08T20:49:32.779 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: rocksdb: Options.is_fd_close_on_exec: 1 2024-03-08T20:49:32.779 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: rocksdb: Options.advise_random_on_open: 1 2024-03-08T20:49:32.779 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: rocksdb: Options.db_write_buffer_size: 0 2024-03-08T20:49:32.780 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: rocksdb: Options.write_buffer_manager: 0x564916bb7400 2024-03-08T20:49:32.780 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-03-08T20:49:32.780 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-03-08T20:49:32.780 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: rocksdb: Options.use_adaptive_mutex: 0 2024-03-08T20:49:32.780 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: rocksdb: Options.rate_limiter: (nil) 2024-03-08T20:49:32.780 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-03-08T20:49:32.780 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: rocksdb: Options.wal_recovery_mode: 2 2024-03-08T20:49:32.780 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: rocksdb: Options.enable_thread_tracking: 0 2024-03-08T20:49:32.780 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: rocksdb: Options.enable_pipelined_write: 0 2024-03-08T20:49:32.780 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: rocksdb: Options.unordered_write: 0 2024-03-08T20:49:32.780 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-03-08T20:49:32.780 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-03-08T20:49:32.780 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-03-08T20:49:32.781 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-03-08T20:49:32.781 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: rocksdb: Options.row_cache: None 2024-03-08T20:49:32.781 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: rocksdb: Options.wal_filter: None 2024-03-08T20:49:32.781 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-03-08T20:49:32.781 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: rocksdb: Options.allow_ingest_behind: 0 2024-03-08T20:49:32.781 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: rocksdb: Options.two_write_queues: 0 2024-03-08T20:49:32.781 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: rocksdb: Options.manual_wal_flush: 0 2024-03-08T20:49:32.781 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: rocksdb: Options.wal_compression: 0 2024-03-08T20:49:32.781 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: rocksdb: Options.atomic_flush: 0 2024-03-08T20:49:32.781 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-03-08T20:49:32.781 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: rocksdb: Options.persist_stats_to_disk: 0 2024-03-08T20:49:32.781 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: rocksdb: Options.write_dbid_to_manifest: 0 2024-03-08T20:49:32.781 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: rocksdb: Options.log_readahead_size: 0 2024-03-08T20:49:32.781 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-03-08T20:49:32.782 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: rocksdb: Options.best_efforts_recovery: 0 2024-03-08T20:49:32.782 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-03-08T20:49:32.782 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-03-08T20:49:32.782 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: rocksdb: Options.allow_data_in_errors: 0 2024-03-08T20:49:32.782 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: rocksdb: Options.db_host_id: __hostname__ 2024-03-08T20:49:32.782 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: rocksdb: Options.enforce_single_del_contracts: true 2024-03-08T20:49:32.782 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: rocksdb: Options.max_background_jobs: 2 2024-03-08T20:49:32.782 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: rocksdb: Options.max_background_compactions: -1 2024-03-08T20:49:32.782 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: rocksdb: Options.max_subcompactions: 1 2024-03-08T20:49:32.782 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-03-08T20:49:32.782 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-03-08T20:49:32.782 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: rocksdb: Options.delayed_write_rate : 16777216 2024-03-08T20:49:32.782 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: rocksdb: Options.max_total_wal_size: 0 2024-03-08T20:49:32.783 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-03-08T20:49:32.783 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: rocksdb: Options.stats_dump_period_sec: 600 2024-03-08T20:49:32.783 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: rocksdb: Options.stats_persist_period_sec: 600 2024-03-08T20:49:32.783 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-03-08T20:49:32.783 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: rocksdb: Options.max_open_files: -1 2024-03-08T20:49:32.783 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: rocksdb: Options.bytes_per_sync: 0 2024-03-08T20:49:32.783 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: rocksdb: Options.wal_bytes_per_sync: 0 2024-03-08T20:49:32.783 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: rocksdb: Options.strict_bytes_per_sync: 0 2024-03-08T20:49:32.783 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: rocksdb: Options.compaction_readahead_size: 0 2024-03-08T20:49:32.783 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: rocksdb: Options.max_background_flushes: -1 2024-03-08T20:49:32.784 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: rocksdb: Compression algorithms supported: 2024-03-08T20:49:32.784 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-03-08T20:49:32.784 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: rocksdb: kZSTD supported: 0 2024-03-08T20:49:32.784 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: rocksdb: kXpressCompression supported: 0 2024-03-08T20:49:32.784 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: rocksdb: kLZ4HCCompression supported: 1 2024-03-08T20:49:32.784 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: rocksdb: kZlibCompression supported: 1 2024-03-08T20:49:32.784 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: rocksdb: kSnappyCompression supported: 1 2024-03-08T20:49:32.784 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: rocksdb: kLZ4Compression supported: 1 2024-03-08T20:49:32.784 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: rocksdb: kBZip2Compression supported: 0 2024-03-08T20:49:32.784 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: rocksdb: Fast CRC32 supported: Supported on x86 2024-03-08T20:49:32.784 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: rocksdb: DMutex implementation: pthread_mutex_t 2024-03-08T20:49:32.784 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2024-03-08T20:49:32.784 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-03-08T20:49:32.785 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-03-08T20:49:32.785 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: rocksdb: Options.merge_operator: 2024-03-08T20:49:32.785 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: rocksdb: Options.compaction_filter: None 2024-03-08T20:49:32.785 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: rocksdb: Options.compaction_filter_factory: None 2024-03-08T20:49:32.785 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: rocksdb: Options.sst_partitioner_factory: None 2024-03-08T20:49:32.785 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: rocksdb: Options.memtable_factory: SkipListFactory 2024-03-08T20:49:32.785 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: rocksdb: Options.table_factory: BlockBasedTable 2024-03-08T20:49:32.785 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x564916be3a20) 2024-03-08T20:49:32.785 INFO:journalctl@ceph.mon.b.smithi149.stdout: cache_index_and_filter_blocks: 1 2024-03-08T20:49:32.785 INFO:journalctl@ceph.mon.b.smithi149.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-03-08T20:49:32.785 INFO:journalctl@ceph.mon.b.smithi149.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-03-08T20:49:32.785 INFO:journalctl@ceph.mon.b.smithi149.stdout: pin_top_level_index_and_filter: 1 2024-03-08T20:49:32.785 INFO:journalctl@ceph.mon.b.smithi149.stdout: index_type: 0 2024-03-08T20:49:32.786 INFO:journalctl@ceph.mon.b.smithi149.stdout: data_block_index_type: 0 2024-03-08T20:49:32.786 INFO:journalctl@ceph.mon.b.smithi149.stdout: index_shortening: 1 2024-03-08T20:49:32.786 INFO:journalctl@ceph.mon.b.smithi149.stdout: data_block_hash_table_util_ratio: 0.750000 2024-03-08T20:49:32.786 INFO:journalctl@ceph.mon.b.smithi149.stdout: checksum: 4 2024-03-08T20:49:32.786 INFO:journalctl@ceph.mon.b.smithi149.stdout: no_block_cache: 0 2024-03-08T20:49:32.786 INFO:journalctl@ceph.mon.b.smithi149.stdout: block_cache: 0x564916be9090 2024-03-08T20:49:32.786 INFO:journalctl@ceph.mon.b.smithi149.stdout: block_cache_name: BinnedLRUCache 2024-03-08T20:49:32.786 INFO:journalctl@ceph.mon.b.smithi149.stdout: block_cache_options: 2024-03-08T20:49:32.786 INFO:journalctl@ceph.mon.b.smithi149.stdout: capacity : 536870912 2024-03-08T20:49:32.786 INFO:journalctl@ceph.mon.b.smithi149.stdout: num_shard_bits : 4 2024-03-08T20:49:32.786 INFO:journalctl@ceph.mon.b.smithi149.stdout: strict_capacity_limit : 0 2024-03-08T20:49:32.786 INFO:journalctl@ceph.mon.b.smithi149.stdout: high_pri_pool_ratio: 0.000 2024-03-08T20:49:32.786 INFO:journalctl@ceph.mon.b.smithi149.stdout: block_cache_compressed: (nil) 2024-03-08T20:49:32.786 INFO:journalctl@ceph.mon.b.smithi149.stdout: persistent_cache: (nil) 2024-03-08T20:49:32.787 INFO:journalctl@ceph.mon.b.smithi149.stdout: block_size: 4096 2024-03-08T20:49:32.787 INFO:journalctl@ceph.mon.b.smithi149.stdout: block_size_deviation: 10 2024-03-08T20:49:32.787 INFO:journalctl@ceph.mon.b.smithi149.stdout: block_restart_interval: 16 2024-03-08T20:49:32.787 INFO:journalctl@ceph.mon.b.smithi149.stdout: index_block_restart_interval: 1 2024-03-08T20:49:32.787 INFO:journalctl@ceph.mon.b.smithi149.stdout: metadata_block_size: 4096 2024-03-08T20:49:32.787 INFO:journalctl@ceph.mon.b.smithi149.stdout: partition_filters: 0 2024-03-08T20:49:32.787 INFO:journalctl@ceph.mon.b.smithi149.stdout: use_delta_encoding: 1 2024-03-08T20:49:32.787 INFO:journalctl@ceph.mon.b.smithi149.stdout: filter_policy: bloomfilter 2024-03-08T20:49:32.787 INFO:journalctl@ceph.mon.b.smithi149.stdout: whole_key_filtering: 1 2024-03-08T20:49:32.787 INFO:journalctl@ceph.mon.b.smithi149.stdout: verify_compression: 0 2024-03-08T20:49:32.787 INFO:journalctl@ceph.mon.b.smithi149.stdout: read_amp_bytes_per_bit: 0 2024-03-08T20:49:32.787 INFO:journalctl@ceph.mon.b.smithi149.stdout: format_version: 5 2024-03-08T20:49:32.787 INFO:journalctl@ceph.mon.b.smithi149.stdout: enable_index_compression: 1 2024-03-08T20:49:32.787 INFO:journalctl@ceph.mon.b.smithi149.stdout: block_align: 0 2024-03-08T20:49:32.788 INFO:journalctl@ceph.mon.b.smithi149.stdout: max_auto_readahead_size: 262144 2024-03-08T20:49:32.788 INFO:journalctl@ceph.mon.b.smithi149.stdout: prepopulate_block_cache: 0 2024-03-08T20:49:32.788 INFO:journalctl@ceph.mon.b.smithi149.stdout: initial_auto_readahead_size: 8192 2024-03-08T20:49:32.788 INFO:journalctl@ceph.mon.b.smithi149.stdout: num_file_reads_for_auto_readahead: 2 2024-03-08T20:49:32.788 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: rocksdb: Options.write_buffer_size: 33554432 2024-03-08T20:49:32.788 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: rocksdb: Options.max_write_buffer_number: 2 2024-03-08T20:49:32.788 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: rocksdb: Options.compression: NoCompression 2024-03-08T20:49:32.788 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: rocksdb: Options.bottommost_compression: Disabled 2024-03-08T20:49:32.788 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: rocksdb: Options.prefix_extractor: nullptr 2024-03-08T20:49:32.788 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-03-08T20:49:32.788 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: rocksdb: Options.num_levels: 7 2024-03-08T20:49:32.788 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-03-08T20:49:32.790 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-03-08T20:49:32.790 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-03-08T20:49:32.790 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-03-08T20:49:32.790 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-03-08T20:49:32.790 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-03-08T20:49:32.790 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-03-08T20:49:32.790 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-03-08T20:49:32.790 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-03-08T20:49:32.790 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-03-08T20:49:32.791 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-03-08T20:49:32.791 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-03-08T20:49:32.791 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: rocksdb: Options.compression_opts.window_bits: -14 2024-03-08T20:49:32.791 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: rocksdb: Options.compression_opts.level: 32767 2024-03-08T20:49:32.791 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: rocksdb: Options.compression_opts.strategy: 0 2024-03-08T20:49:32.791 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-03-08T20:49:32.791 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-03-08T20:49:32.791 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-03-08T20:49:32.791 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-03-08T20:49:32.791 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: rocksdb: Options.compression_opts.enabled: false 2024-03-08T20:49:32.791 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-03-08T20:49:32.791 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-03-08T20:49:32.791 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-03-08T20:49:32.792 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-03-08T20:49:32.792 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: rocksdb: Options.target_file_size_base: 67108864 2024-03-08T20:49:32.792 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: rocksdb: Options.target_file_size_multiplier: 1 2024-03-08T20:49:32.792 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-03-08T20:49:32.792 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-03-08T20:49:32.792 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-03-08T20:49:32.792 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-03-08T20:49:32.792 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-03-08T20:49:32.792 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-03-08T20:49:32.792 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-03-08T20:49:32.792 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-03-08T20:49:32.792 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-03-08T20:49:32.792 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-03-08T20:49:32.793 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-03-08T20:49:32.793 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-03-08T20:49:32.793 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-03-08T20:49:32.793 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: rocksdb: Options.arena_block_size: 1048576 2024-03-08T20:49:32.793 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-03-08T20:49:32.793 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-03-08T20:49:32.793 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: rocksdb: Options.disable_auto_compactions: 0 2024-03-08T20:49:32.793 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-03-08T20:49:32.793 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-03-08T20:49:32.794 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-03-08T20:49:32.794 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-03-08T20:49:32.794 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-03-08T20:49:32.794 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-03-08T20:49:32.794 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-03-08T20:49:32.794 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-03-08T20:49:32.794 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-03-08T20:49:32.794 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-03-08T20:49:32.794 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: rocksdb: Options.table_properties_collectors: 2024-03-08T20:49:32.794 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: rocksdb: Options.inplace_update_support: 0 2024-03-08T20:49:32.794 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: rocksdb: Options.inplace_update_num_locks: 10000 2024-03-08T20:49:32.794 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-03-08T20:49:32.795 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-03-08T20:49:32.795 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: rocksdb: Options.memtable_huge_page_size: 0 2024-03-08T20:49:32.795 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: rocksdb: Options.bloom_locality: 0 2024-03-08T20:49:32.795 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: rocksdb: Options.max_successive_merges: 0 2024-03-08T20:49:32.795 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: rocksdb: Options.optimize_filters_for_hits: 0 2024-03-08T20:49:32.795 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: rocksdb: Options.paranoid_file_checks: 0 2024-03-08T20:49:32.795 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: rocksdb: Options.force_consistency_checks: 1 2024-03-08T20:49:32.795 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: rocksdb: Options.report_bg_io_stats: 0 2024-03-08T20:49:32.795 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: rocksdb: Options.ttl: 2592000 2024-03-08T20:49:32.795 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: rocksdb: Options.periodic_compaction_seconds: 0 2024-03-08T20:49:32.795 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-03-08T20:49:32.795 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-03-08T20:49:32.795 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: rocksdb: Options.enable_blob_files: false 2024-03-08T20:49:32.795 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: rocksdb: Options.min_blob_size: 0 2024-03-08T20:49:32.796 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: rocksdb: Options.blob_file_size: 268435456 2024-03-08T20:49:32.796 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: rocksdb: Options.blob_compression_type: NoCompression 2024-03-08T20:49:32.796 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: rocksdb: Options.enable_blob_garbage_collection: false 2024-03-08T20:49:32.796 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-03-08T20:49:32.796 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-03-08T20:49:32.796 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-03-08T20:49:32.796 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: rocksdb: Options.blob_file_starting_level: 0 2024-03-08T20:49:32.796 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-03-08T20:49:32.796 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-03-08T20:49:32.796 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-03-08T20:49:32.796 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 14e67aff-8475-4525-80e2-c4bd29961797 2024-03-08T20:49:32.796 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: rocksdb: EVENT_LOG_v1 {"time_micros": 1709930972599787, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-03-08T20:49:32.796 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-03-08T20:49:32.797 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: rocksdb: EVENT_LOG_v1 {"time_micros": 1709930972600700, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1747, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1709930972, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "14e67aff-8475-4525-80e2-c4bd29961797", "db_session_id": "E9EJ9WZQMB2M9GT2A90H", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-03-08T20:49:32.797 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: rocksdb: EVENT_LOG_v1 {"time_micros": 1709930972600969, "job": 1, "event": "recovery_finished"} 2024-03-08T20:49:32.797 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-03-08T20:49:32.797 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-03-08T20:49:32.797 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x564916cdc000 2024-03-08T20:49:32.797 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: rocksdb: DB pointer 0x564916cc6000 2024-03-08T20:49:32.797 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: mon.b does not exist in monmap, will attempt to join an existing cluster 2024-03-08T20:49:32.797 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-03-08T20:49:32.797 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-03-08T20:49:32.797 INFO:journalctl@ceph.mon.b.smithi149.stdout: ** DB Stats ** 2024-03-08T20:49:32.797 INFO:journalctl@ceph.mon.b.smithi149.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-08T20:49:32.797 INFO:journalctl@ceph.mon.b.smithi149.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-03-08T20:49:32.797 INFO:journalctl@ceph.mon.b.smithi149.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-08T20:49:32.798 INFO:journalctl@ceph.mon.b.smithi149.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-08T20:49:32.798 INFO:journalctl@ceph.mon.b.smithi149.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-03-08T20:49:32.798 INFO:journalctl@ceph.mon.b.smithi149.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-08T20:49:32.798 INFO:journalctl@ceph.mon.b.smithi149.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-08T20:49:32.798 INFO:journalctl@ceph.mon.b.smithi149.stdout: 2024-03-08T20:49:32.798 INFO:journalctl@ceph.mon.b.smithi149.stdout: ** Compaction Stats [default] ** 2024-03-08T20:49:32.798 INFO:journalctl@ceph.mon.b.smithi149.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-08T20:49:32.798 INFO:journalctl@ceph.mon.b.smithi149.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-03-08T20:49:32.798 INFO:journalctl@ceph.mon.b.smithi149.stdout: L0 1/0 1.71 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 1.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-08T20:49:32.798 INFO:journalctl@ceph.mon.b.smithi149.stdout: Sum 1/0 1.71 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 1.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-08T20:49:32.798 INFO:journalctl@ceph.mon.b.smithi149.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 1.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-08T20:49:32.798 INFO:journalctl@ceph.mon.b.smithi149.stdout: 2024-03-08T20:49:32.798 INFO:journalctl@ceph.mon.b.smithi149.stdout: ** Compaction Stats [default] ** 2024-03-08T20:49:32.799 INFO:journalctl@ceph.mon.b.smithi149.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-08T20:49:32.799 INFO:journalctl@ceph.mon.b.smithi149.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-08T20:49:32.799 INFO:journalctl@ceph.mon.b.smithi149.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-08T20:49:32.799 INFO:journalctl@ceph.mon.b.smithi149.stdout: 2024-03-08T20:49:32.799 INFO:journalctl@ceph.mon.b.smithi149.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-03-08T20:49:32.799 INFO:journalctl@ceph.mon.b.smithi149.stdout: 2024-03-08T20:49:32.799 INFO:journalctl@ceph.mon.b.smithi149.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-08T20:49:32.799 INFO:journalctl@ceph.mon.b.smithi149.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-03-08T20:49:32.799 INFO:journalctl@ceph.mon.b.smithi149.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-03-08T20:49:32.799 INFO:journalctl@ceph.mon.b.smithi149.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-03-08T20:49:32.799 INFO:journalctl@ceph.mon.b.smithi149.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-03-08T20:49:32.799 INFO:journalctl@ceph.mon.b.smithi149.stdout: AddFile(Keys): cumulative 0, interval 0 2024-03-08T20:49:32.799 INFO:journalctl@ceph.mon.b.smithi149.stdout: Cumulative compaction: 0.00 GB write, 0.30 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-08T20:49:32.800 INFO:journalctl@ceph.mon.b.smithi149.stdout: Interval compaction: 0.00 GB write, 0.30 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-08T20:49:32.800 INFO:journalctl@ceph.mon.b.smithi149.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-03-08T20:49:32.800 INFO:journalctl@ceph.mon.b.smithi149.stdout: Block cache BinnedLRUCache@0x564916be9090#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.1e-05 secs_since: 0 2024-03-08T20:49:32.800 INFO:journalctl@ceph.mon.b.smithi149.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-03-08T20:49:32.800 INFO:journalctl@ceph.mon.b.smithi149.stdout: 2024-03-08T20:49:32.800 INFO:journalctl@ceph.mon.b.smithi149.stdout: ** File Read Latency Histogram By Level [default] ** 2024-03-08T20:49:32.800 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: using public_addr v2:172.21.15.149:0/0 -> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] 2024-03-08T20:49:32.800 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: starting mon.b rank -1 at public addrs [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] at bind addrs [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid dc20f0f8-dd8c-11ee-95c6-87774f69a715 2024-03-08T20:49:32.800 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: mon.b@-1(???) e0 preinit fsid dc20f0f8-dd8c-11ee-95c6-87774f69a715 2024-03-08T20:49:32.800 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 bash[35023]: 8cecc7919a787e1ca250082d39d07931dc0f0e8e336b6b9c293973f59ad75088 2024-03-08T20:49:32.800 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 systemd[1]: Started Ceph mon.b for dc20f0f8-dd8c-11ee-95c6-87774f69a715. 2024-03-08T20:49:32.800 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: mon.b@-1(synchronizing).mds e1 new map 2024-03-08T20:49:32.800 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: mon.b@-1(synchronizing).mds e1 print_map 2024-03-08T20:49:32.801 INFO:journalctl@ceph.mon.b.smithi149.stdout: e1 2024-03-08T20:49:32.801 INFO:journalctl@ceph.mon.b.smithi149.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-03-08T20:49:32.801 INFO:journalctl@ceph.mon.b.smithi149.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-03-08T20:49:32.801 INFO:journalctl@ceph.mon.b.smithi149.stdout: legacy client fscid: -1 2024-03-08T20:49:32.801 INFO:journalctl@ceph.mon.b.smithi149.stdout: 2024-03-08T20:49:32.801 INFO:journalctl@ceph.mon.b.smithi149.stdout: No filesystems configured 2024-03-08T20:49:32.801 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-03-08T20:49:32.801 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-03-08T20:49:32.801 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-03-08T20:49:32.801 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-03-08T20:49:32.801 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-03-08T20:49:32.801 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-03-08T20:49:32.801 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-03-08T20:49:32.801 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-03-08T20:49:32.802 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-03-08T20:49:32.802 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-03-08T20:49:32.802 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T20:49:32.802 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi149", "target": ["mon-mgr", ""]}]: dispatch 2024-03-08T20:49:32.802 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T20:49:32.802 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: Deploying cephadm binary to smithi149 2024-03-08T20:49:32.802 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T20:49:32.802 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T20:49:32.802 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:49:32.802 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: Added host smithi149 2024-03-08T20:49:32.802 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T20:49:32.802 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:49:32.802 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T20:49:32.803 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:49:32.803 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T20:49:32.803 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:49:32.803 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-08T20:49:32.803 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T20:49:32.803 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T20:49:32.803 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi204", "target": ["mon-mgr", ""]}]: dispatch 2024-03-08T20:49:32.803 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: Deploying cephadm binary to smithi204 2024-03-08T20:49:32.803 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:49:32.803 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:49:32.803 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:49:32.803 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:49:32.803 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi149", "name": "osd_memory_target"}]: dispatch 2024-03-08T20:49:32.803 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:49:32.804 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T20:49:32.804 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: Updating smithi149:/etc/ceph/ceph.conf 2024-03-08T20:49:32.804 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T20:49:32.804 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: Updating smithi149:/var/lib/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/config/ceph.conf 2024-03-08T20:49:32.804 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T20:49:32.804 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: Updating smithi149:/etc/ceph/ceph.client.admin.keyring 2024-03-08T20:49:32.804 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T20:49:32.804 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:49:32.804 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: Updating smithi149:/var/lib/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/config/ceph.client.admin.keyring 2024-03-08T20:49:32.804 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: Added host smithi204 2024-03-08T20:49:32.804 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:49:32.804 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:49:32.804 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:49:32.804 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T20:49:32.805 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T20:49:32.805 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:49:32.805 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:49:32.805 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-08T20:49:32.805 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T20:49:32.805 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:49:32.805 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T20:49:32.805 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: from='client.? 172.21.15.107:0/1054927658' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-03-08T20:49:32.805 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T20:49:32.805 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: from='client.? 172.21.15.107:0/1054927658' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-03-08T20:49:32.805 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: osdmap e4: 0 total, 0 up, 0 in 2024-03-08T20:49:32.805 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T20:49:32.805 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:49:32.806 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:49:32.806 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:49:32.806 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:49:32.806 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi204", "name": "osd_memory_target"}]: dispatch 2024-03-08T20:49:32.806 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:49:32.806 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T20:49:32.806 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: Updating smithi204:/etc/ceph/ceph.conf 2024-03-08T20:49:32.806 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: Updating smithi204:/var/lib/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/config/ceph.conf 2024-03-08T20:49:32.806 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: Updating smithi204:/etc/ceph/ceph.client.admin.keyring 2024-03-08T20:49:32.806 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T20:49:32.806 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: Updating smithi204:/var/lib/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/config/ceph.client.admin.keyring 2024-03-08T20:49:32.806 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:49:32.806 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:49:32.806 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:49:32.807 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:49:32.807 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-08T20:49:32.807 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:49:32.807 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi107:172.21.15.107=a;smithi149:172.21.15.149=b;smithi204:172.21.15.204=c", "target": ["mon-mgr", ""]}]: dispatch 2024-03-08T20:49:32.807 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: Saving service mon spec with placement smithi107:172.21.15.107=a;smithi149:172.21.15.149=b;smithi204:172.21.15.204=c;count:3 2024-03-08T20:49:32.807 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: Deploying daemon mon.c on smithi204 2024-03-08T20:49:32.807 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T20:49:32.807 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T20:49:32.807 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: from='client.? 172.21.15.204:0/4167819322' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-08T20:49:32.807 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T20:49:32.807 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-08T20:49:32.807 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: mon.a calling monitor election 2024-03-08T20:49:32.807 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-08T20:49:32.808 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T20:49:32.808 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-08T20:49:32.808 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-08T20:49:32.808 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: mon.c calling monitor election 2024-03-08T20:49:32.808 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T20:49:32.808 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-08T20:49:32.808 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-08T20:49:32.808 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T20:49:32.808 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-08T20:49:32.808 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-08T20:49:32.808 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: monmap e2: 2 mons at {a=[v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0],c=[v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-08T20:49:32.808 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: fsmap 2024-03-08T20:49:32.808 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: osdmap e4: 0 total, 0 up, 0 in 2024-03-08T20:49:32.808 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: mgrmap e13: x(active, since 66s) 2024-03-08T20:49:32.809 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: overall HEALTH_OK 2024-03-08T20:49:32.809 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:49:32.809 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:49:32.809 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:49:32.809 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-08T20:49:32.809 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:49:32.809 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: Deploying daemon mon.b on smithi149 2024-03-08T20:49:32.809 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: from='client.? 172.21.15.204:0/4237351079' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-08T20:49:32.809 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-08T20:49:32.809 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T20:49:32.809 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T20:49:32.809 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: from='client.? 172.21.15.204:0/3772280724' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-08T20:49:32.809 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:32 smithi149 ceph-mon[35042]: mon.b@-1(synchronizing).paxosservice(auth 1..4) refresh upgraded, format 0 -> 3 2024-03-08T20:49:33.346 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-03-08T20:49:33.346 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dc20f0f8-dd8c-11ee-95c6-87774f69a715 -- ceph mon dump -f json 2024-03-08T20:49:33.784 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/mon.c/config 2024-03-08T20:49:37.976 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:49:37 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-08T20:49:37.977 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:49:37 smithi107 ceph-mon[31381]: mon.a calling monitor election 2024-03-08T20:49:37.977 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:49:37 smithi107 ceph-mon[31381]: mon.c calling monitor election 2024-03-08T20:49:37.977 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:49:37 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-08T20:49:37.977 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:49:37 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-08T20:49:37.977 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:49:37 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-08T20:49:37.977 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:49:37 smithi107 ceph-mon[31381]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T20:49:37.977 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:49:37 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-08T20:49:37.977 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:49:37 smithi107 ceph-mon[31381]: mon.b calling monitor election 2024-03-08T20:49:37.977 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:49:37 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-08T20:49:37.977 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:49:37 smithi107 ceph-mon[31381]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T20:49:37.977 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:49:37 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-08T20:49:37.977 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:49:37 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-08T20:49:37.978 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:49:37 smithi107 ceph-mon[31381]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-03-08T20:49:37.978 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:49:37 smithi107 ceph-mon[31381]: monmap e3: 3 mons at {a=[v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0],b=[v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0],c=[v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-08T20:49:37.978 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:49:37 smithi107 ceph-mon[31381]: fsmap 2024-03-08T20:49:37.978 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:49:37 smithi107 ceph-mon[31381]: osdmap e4: 0 total, 0 up, 0 in 2024-03-08T20:49:37.978 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:49:37 smithi107 ceph-mon[31381]: mgrmap e13: x(active, since 77s) 2024-03-08T20:49:37.978 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:49:37 smithi107 ceph-mon[31381]: overall HEALTH_OK 2024-03-08T20:49:37.978 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:49:37 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:49:37.978 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:49:37 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:49:38.024 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:37 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-08T20:49:38.025 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:37 smithi149 ceph-mon[35042]: mon.a calling monitor election 2024-03-08T20:49:38.025 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:37 smithi149 ceph-mon[35042]: mon.c calling monitor election 2024-03-08T20:49:38.025 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:37 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-08T20:49:38.025 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:37 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-08T20:49:38.025 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:37 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-08T20:49:38.025 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:37 smithi149 ceph-mon[35042]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T20:49:38.025 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:37 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-08T20:49:38.025 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:37 smithi149 ceph-mon[35042]: mon.b calling monitor election 2024-03-08T20:49:38.025 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:37 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-08T20:49:38.025 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:37 smithi149 ceph-mon[35042]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T20:49:38.025 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:37 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-08T20:49:38.025 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:37 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-08T20:49:38.026 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:37 smithi149 ceph-mon[35042]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-03-08T20:49:38.026 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:37 smithi149 ceph-mon[35042]: monmap e3: 3 mons at {a=[v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0],b=[v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0],c=[v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-08T20:49:38.026 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:37 smithi149 ceph-mon[35042]: fsmap 2024-03-08T20:49:38.026 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:37 smithi149 ceph-mon[35042]: osdmap e4: 0 total, 0 up, 0 in 2024-03-08T20:49:38.026 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:37 smithi149 ceph-mon[35042]: mgrmap e13: x(active, since 77s) 2024-03-08T20:49:38.026 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:37 smithi149 ceph-mon[35042]: overall HEALTH_OK 2024-03-08T20:49:38.026 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:37 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:49:38.026 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:37 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:49:38.137 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:37 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-08T20:49:38.137 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:37 smithi204 ceph-mon[34935]: mon.a calling monitor election 2024-03-08T20:49:38.137 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:37 smithi204 ceph-mon[34935]: mon.c calling monitor election 2024-03-08T20:49:38.137 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:37 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-08T20:49:38.137 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:37 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-08T20:49:38.137 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:37 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-08T20:49:38.137 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:37 smithi204 ceph-mon[34935]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T20:49:38.137 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:37 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-08T20:49:38.137 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:37 smithi204 ceph-mon[34935]: mon.b calling monitor election 2024-03-08T20:49:38.138 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:37 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-08T20:49:38.138 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:37 smithi204 ceph-mon[34935]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T20:49:38.138 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:37 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-08T20:49:38.138 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:37 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-08T20:49:38.138 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:37 smithi204 ceph-mon[34935]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-03-08T20:49:38.138 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:37 smithi204 ceph-mon[34935]: monmap e3: 3 mons at {a=[v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0],b=[v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0],c=[v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-08T20:49:38.138 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:37 smithi204 ceph-mon[34935]: fsmap 2024-03-08T20:49:38.138 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:37 smithi204 ceph-mon[34935]: osdmap e4: 0 total, 0 up, 0 in 2024-03-08T20:49:38.138 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:37 smithi204 ceph-mon[34935]: mgrmap e13: x(active, since 77s) 2024-03-08T20:49:38.138 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:37 smithi204 ceph-mon[34935]: overall HEALTH_OK 2024-03-08T20:49:38.138 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:37 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:49:38.138 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:37 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:49:38.834 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:38.841+0000 7f64aa600700 1 -- 172.21.15.204:0/145747970 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f64a4102910 msgr2=0x7f6488005610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:49:38.834 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:38.841+0000 7f64aa600700 1 --2- 172.21.15.204:0/145747970 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f64a4102910 0x7f6488005610 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f6494005fa0 tx=0x7f649400ffc0 comp rx=0 tx=0).stop 2024-03-08T20:49:38.834 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:38.841+0000 7f64aa600700 1 -- 172.21.15.204:0/145747970 shutdown_connections 2024-03-08T20:49:38.835 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:38.841+0000 7f64aa600700 1 --2- 172.21.15.204:0/145747970 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f64a4102910 0x7f6488005610 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:49:38.835 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:38.841+0000 7f64aa600700 1 -- 172.21.15.204:0/145747970 >> 172.21.15.204:0/145747970 conn(0x7f64a40fc120 msgr2=0x7f64a40fe540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:49:38.835 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:38.841+0000 7f64aa600700 1 -- 172.21.15.204:0/145747970 shutdown_connections 2024-03-08T20:49:38.835 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:38.841+0000 7f64aa600700 1 -- 172.21.15.204:0/145747970 wait complete. 2024-03-08T20:49:38.835 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:38.843+0000 7f64aa600700 1 Processor -- start 2024-03-08T20:49:38.836 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:38.843+0000 7f64aa600700 1 -- start start 2024-03-08T20:49:38.836 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:38.844+0000 7f64aa600700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f64a41996a0 0x7f64a4199a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:49:38.838 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:38.844+0000 7f64aa600700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f64a4199fc0 0x7f64a419df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:49:38.838 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:38.844+0000 7f64aa600700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f64a419e4b0 0x7f64a419e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:49:38.838 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:38.844+0000 7f64aa600700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64a41095e0 con 0x7f64a4199fc0 2024-03-08T20:49:38.838 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:38.844+0000 7f64aa600700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64a4109460 con 0x7f64a41996a0 2024-03-08T20:49:38.838 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:38.844+0000 7f64aa600700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64a4109760 con 0x7f64a419e4b0 2024-03-08T20:49:38.838 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:38.845+0000 7f649be00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f64a4199fc0 0x7f64a419df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:49:38.838 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:38.845+0000 7f64a9c00700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f64a419e4b0 0x7f64a419e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:49:38.838 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:38.845+0000 7f649be00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f64a4199fc0 0x7f64a419df70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.204:45728/0 (socket says 172.21.15.204:45728) 2024-03-08T20:49:38.839 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:38.845+0000 7f64a9c00700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f64a419e4b0 0x7f64a419e940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:44796/0 (socket says 172.21.15.204:44796) 2024-03-08T20:49:38.839 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:38.845+0000 7f649be00700 1 -- 172.21.15.204:0/1442025680 learned_addr learned my addr 172.21.15.204:0/1442025680 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-03-08T20:49:38.839 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:38.845+0000 7f64a9200700 1 --2- 172.21.15.204:0/1442025680 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f64a41996a0 0x7f64a4199a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:49:38.839 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:38.845+0000 7f64a9200700 1 -- 172.21.15.204:0/1442025680 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f64a41996a0 msgr2=0x7f64a4199a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-03-08T20:49:38.839 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:38.845+0000 7f64a9200700 1 -- 172.21.15.204:0/1442025680 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f64a41996a0 msgr2=0x7f64a4199a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-08T20:49:38.839 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:38.845+0000 7f64a9200700 1 --2- 172.21.15.204:0/1442025680 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f64a41996a0 0x7f64a4199a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-08T20:49:38.839 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:38.845+0000 7f64a9200700 1 --2- 172.21.15.204:0/1442025680 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f64a41996a0 0x7f64a4199a80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-08T20:49:38.839 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:38.845+0000 7f64a9c00700 1 -- 172.21.15.204:0/1442025680 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f64a41996a0 msgr2=0x7f64a4199a80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-08T20:49:38.839 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:38.845+0000 7f64a9c00700 1 --2- 172.21.15.204:0/1442025680 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f64a41996a0 0x7f64a4199a80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:49:38.839 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:38.845+0000 7f64a9c00700 1 -- 172.21.15.204:0/1442025680 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f64a4199fc0 msgr2=0x7f64a419df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:49:38.839 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:38.845+0000 7f64a9c00700 1 --2- 172.21.15.204:0/1442025680 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f64a4199fc0 0x7f64a419df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:49:38.839 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:38.846+0000 7f64a9c00700 1 -- 172.21.15.204:0/1442025680 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64a41a64f0 con 0x7f64a419e4b0 2024-03-08T20:49:38.840 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:38.846+0000 7f64a9c00700 1 --2- 172.21.15.204:0/1442025680 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f64a419e4b0 0x7f64a419e940 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f649c00bd50 tx=0x7f649c00bd80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:49:38.840 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:38.846+0000 7f64a2000700 1 -- 172.21.15.204:0/1442025680 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f649c019070 con 0x7f64a419e4b0 2024-03-08T20:49:38.840 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:38.846+0000 7f64a2000700 1 -- 172.21.15.204:0/1442025680 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f649c00cc10 con 0x7f64a419e4b0 2024-03-08T20:49:38.840 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:38.846+0000 7f64aa600700 1 -- 172.21.15.204:0/1442025680 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64a41a67e0 con 0x7f64a419e4b0 2024-03-08T20:49:38.840 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:38.846+0000 7f64a2000700 1 -- 172.21.15.204:0/1442025680 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f649c0125a0 con 0x7f64a419e4b0 2024-03-08T20:49:38.841 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:38.847+0000 7f64aa600700 1 -- 172.21.15.204:0/1442025680 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64a41a6d20 con 0x7f64a419e4b0 2024-03-08T20:49:38.841 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:38.848+0000 7f64aa600700 1 -- 172.21.15.204:0/1442025680 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64a4063410 con 0x7f64a419e4b0 2024-03-08T20:49:38.841 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:38.849+0000 7f64a2000700 1 -- 172.21.15.204:0/1442025680 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 13) v1 ==== 46163+0+0 (secure 0 0 0) 0x7f649c01c070 con 0x7f64a419e4b0 2024-03-08T20:49:38.841 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:38.849+0000 7f64a2000700 1 --2- 172.21.15.204:0/1442025680 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f6480039450 0x7f648003b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:49:38.841 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:38.849+0000 7f64a2000700 1 -- 172.21.15.204:0/1442025680 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f649c04b800 con 0x7f64a419e4b0 2024-03-08T20:49:38.841 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:38.850+0000 7f64a9200700 1 --2- 172.21.15.204:0/1442025680 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f6480039450 0x7f648003b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:49:38.842 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:38.851+0000 7f64a9200700 1 --2- 172.21.15.204:0/1442025680 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f6480039450 0x7f648003b910 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f64940137f0 tx=0x7f6494005ad0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:49:38.847 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:38.856+0000 7f64a2000700 1 -- 172.21.15.204:0/1442025680 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f649c007290 con 0x7f64a419e4b0 2024-03-08T20:49:39.023 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:38 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:49:39.023 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:38 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:49:39.023 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:38 smithi149 ceph-mon[35042]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T20:49:39.023 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:38 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T20:49:39.024 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:38 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-08T20:49:39.136 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:38 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:49:39.136 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:38 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:49:39.137 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:38 smithi204 ceph-mon[34935]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T20:49:39.137 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:38 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T20:49:39.137 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:38 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-08T20:49:39.218 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:39.227+0000 7f64aa600700 1 -- 172.21.15.204:0/1442025680 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f64a402d830 con 0x7f64a419e4b0 2024-03-08T20:49:39.219 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:39.227+0000 7f64a2000700 1 -- 172.21.15.204:0/1442025680 <== mon.1 v2:172.21.15.204:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) v1 ==== 95+0+1289 (secure 0 0 0) 0x7f649c01cc60 con 0x7f64a419e4b0 2024-03-08T20:49:39.219 INFO:teuthology.orchestra.run.smithi204.stdout: 2024-03-08T20:49:39.219 INFO:teuthology.orchestra.run.smithi204.stdout:{"epoch":3,"fsid":"dc20f0f8-dd8c-11ee-95c6-87774f69a715","modified":"2024-03-08T20:49:32.646152Z","created":"2024-03-08T20:47:00.037201Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:3300","nonce":0},{"type":"v1","addr":"172.21.15.107:6789","nonce":0}]},"addr":"172.21.15.107:6789/0","public_addr":"172.21.15.107:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:3300","nonce":0},{"type":"v1","addr":"172.21.15.204:6789","nonce":0}]},"addr":"172.21.15.204:6789/0","public_addr":"172.21.15.204:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:3300","nonce":0},{"type":"v1","addr":"172.21.15.149:6789","nonce":0}]},"addr":"172.21.15.149:6789/0","public_addr":"172.21.15.149:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1,2]} 2024-03-08T20:49:39.221 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:39.230+0000 7f64aa600700 1 -- 172.21.15.204:0/1442025680 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f6480039450 msgr2=0x7f648003b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:49:39.221 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:39.230+0000 7f64aa600700 1 --2- 172.21.15.204:0/1442025680 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f6480039450 0x7f648003b910 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f64940137f0 tx=0x7f6494005ad0 comp rx=0 tx=0).stop 2024-03-08T20:49:39.221 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:39.230+0000 7f64aa600700 1 -- 172.21.15.204:0/1442025680 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f64a419e4b0 msgr2=0x7f64a419e940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:49:39.222 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:39.230+0000 7f64aa600700 1 --2- 172.21.15.204:0/1442025680 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f64a419e4b0 0x7f64a419e940 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f649c00bd50 tx=0x7f649c00bd80 comp rx=0 tx=0).stop 2024-03-08T20:49:39.222 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:39.230+0000 7f64aa600700 1 -- 172.21.15.204:0/1442025680 shutdown_connections 2024-03-08T20:49:39.222 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:39.230+0000 7f64aa600700 1 --2- 172.21.15.204:0/1442025680 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f64a419e4b0 0x7f64a419e940 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:49:39.222 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:39.230+0000 7f64aa600700 1 --2- 172.21.15.204:0/1442025680 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f6480039450 0x7f648003b910 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:49:39.222 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:39.230+0000 7f64aa600700 1 --2- 172.21.15.204:0/1442025680 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f64a41996a0 0x7f64a4199a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:49:39.222 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:39.230+0000 7f64aa600700 1 --2- 172.21.15.204:0/1442025680 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f64a4199fc0 0x7f64a419df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:49:39.222 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:39.230+0000 7f64aa600700 1 -- 172.21.15.204:0/1442025680 >> 172.21.15.204:0/1442025680 conn(0x7f64a40fc120 msgr2=0x7f64a40fe3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:49:39.222 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:39.230+0000 7f64aa600700 1 -- 172.21.15.204:0/1442025680 shutdown_connections 2024-03-08T20:49:39.222 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:39.230+0000 7f64aa600700 1 -- 172.21.15.204:0/1442025680 wait complete. 2024-03-08T20:49:39.222 INFO:teuthology.orchestra.run.smithi204.stderr:dumped monmap epoch 3 2024-03-08T20:49:39.226 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:49:38 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:49:39.226 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:49:38 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:49:39.226 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:49:38 smithi107 ceph-mon[31381]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T20:49:39.226 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:49:38 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T20:49:39.226 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:49:38 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-08T20:49:40.024 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:39 smithi149 ceph-mon[35042]: from='client.? 172.21.15.204:0/1442025680' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-08T20:49:40.137 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:39 smithi204 ceph-mon[34935]: from='client.? 172.21.15.204:0/1442025680' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-08T20:49:40.225 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:49:39 smithi107 ceph-mon[31381]: from='client.? 172.21.15.204:0/1442025680' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-08T20:49:40.234 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-03-08T20:49:40.234 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dc20f0f8-dd8c-11ee-95c6-87774f69a715 -- ceph config generate-minimal-conf 2024-03-08T20:49:40.678 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/mon.a/config 2024-03-08T20:49:41.122 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:49:40 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:49:41.122 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:49:40 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:49:41.122 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:49:40 smithi107 ceph-mon[31381]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T20:49:41.123 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:49:40 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:49:41.123 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:49:40 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:49:41.123 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:49:40 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:49:41.123 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:49:40 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:49:41.123 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:49:40 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T20:49:41.273 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:40 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:49:41.274 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:40 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:49:41.274 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:40 smithi149 ceph-mon[35042]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T20:49:41.274 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:40 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:49:41.274 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:40 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:49:41.274 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:40 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:49:41.274 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:40 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:49:41.274 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:40 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T20:49:41.386 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:40 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:49:41.386 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:40 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:49:41.386 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:40 smithi204 ceph-mon[34935]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T20:49:41.387 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:40 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:49:41.387 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:40 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:49:41.387 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:40 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:49:41.387 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:40 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:49:41.387 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:40 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T20:49:42.060 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:49:41 smithi107 ceph-mon[31381]: Updating smithi107:/etc/ceph/ceph.conf 2024-03-08T20:49:42.060 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:49:41 smithi107 ceph-mon[31381]: Updating smithi149:/etc/ceph/ceph.conf 2024-03-08T20:49:42.060 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:49:41 smithi107 ceph-mon[31381]: Updating smithi204:/etc/ceph/ceph.conf 2024-03-08T20:49:42.060 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:49:41 smithi107 ceph-mon[31381]: Updating smithi107:/var/lib/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/config/ceph.conf 2024-03-08T20:49:42.060 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:49:41 smithi107 ceph-mon[31381]: Updating smithi149:/var/lib/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/config/ceph.conf 2024-03-08T20:49:42.060 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:49:41 smithi107 ceph-mon[31381]: Updating smithi204:/var/lib/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/config/ceph.conf 2024-03-08T20:49:42.060 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:49:41 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:49:42.060 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:49:41 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:49:42.273 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:41 smithi149 ceph-mon[35042]: Updating smithi107:/etc/ceph/ceph.conf 2024-03-08T20:49:42.274 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:41 smithi149 ceph-mon[35042]: Updating smithi149:/etc/ceph/ceph.conf 2024-03-08T20:49:42.274 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:41 smithi149 ceph-mon[35042]: Updating smithi204:/etc/ceph/ceph.conf 2024-03-08T20:49:42.274 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:41 smithi149 ceph-mon[35042]: Updating smithi107:/var/lib/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/config/ceph.conf 2024-03-08T20:49:42.274 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:41 smithi149 ceph-mon[35042]: Updating smithi149:/var/lib/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/config/ceph.conf 2024-03-08T20:49:42.274 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:41 smithi149 ceph-mon[35042]: Updating smithi204:/var/lib/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/config/ceph.conf 2024-03-08T20:49:42.274 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:41 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:49:42.274 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:41 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:49:42.387 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:41 smithi204 ceph-mon[34935]: Updating smithi107:/etc/ceph/ceph.conf 2024-03-08T20:49:42.387 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:41 smithi204 ceph-mon[34935]: Updating smithi149:/etc/ceph/ceph.conf 2024-03-08T20:49:42.387 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:41 smithi204 ceph-mon[34935]: Updating smithi204:/etc/ceph/ceph.conf 2024-03-08T20:49:42.387 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:41 smithi204 ceph-mon[34935]: Updating smithi107:/var/lib/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/config/ceph.conf 2024-03-08T20:49:42.387 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:41 smithi204 ceph-mon[34935]: Updating smithi149:/var/lib/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/config/ceph.conf 2024-03-08T20:49:42.387 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:41 smithi204 ceph-mon[34935]: Updating smithi204:/var/lib/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/config/ceph.conf 2024-03-08T20:49:42.387 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:41 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:49:42.387 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:41 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:49:42.785 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:49:42.792+0000 7ffb9a600700 1 -- 172.21.15.107:0/1422016578 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffb941077a0 msgr2=0x7ffb94107b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:49:42.785 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:49:42.792+0000 7ffb9a600700 1 --2- 172.21.15.107:0/1422016578 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffb941077a0 0x7ffb94107b80 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7ffb84009910 tx=0x7ffb84009c20 comp rx=0 tx=0).stop 2024-03-08T20:49:42.785 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:49:42.792+0000 7ffb9a600700 1 -- 172.21.15.107:0/1422016578 shutdown_connections 2024-03-08T20:49:42.785 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:49:42.792+0000 7ffb9a600700 1 --2- 172.21.15.107:0/1422016578 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffb941077a0 0x7ffb94107b80 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:49:42.785 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:49:42.792+0000 7ffb9a600700 1 -- 172.21.15.107:0/1422016578 >> 172.21.15.107:0/1422016578 conn(0x7ffb940fd2b0 msgr2=0x7ffb940ff6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:49:42.785 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:49:42.792+0000 7ffb9a600700 1 -- 172.21.15.107:0/1422016578 shutdown_connections 2024-03-08T20:49:42.785 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:49:42.792+0000 7ffb9a600700 1 -- 172.21.15.107:0/1422016578 wait complete. 2024-03-08T20:49:42.785 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:49:42.793+0000 7ffb9a600700 1 Processor -- start 2024-03-08T20:49:42.786 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:49:42.793+0000 7ffb9a600700 1 -- start start 2024-03-08T20:49:42.787 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:49:42.794+0000 7ffb9a600700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffb941077a0 0x7ffb94199670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:49:42.787 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:49:42.794+0000 7ffb9a600700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffb94199bb0 0x7ffb9419df40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:49:42.787 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:49:42.794+0000 7ffb9a600700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffb9419e480 0x7ffb9419e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:49:42.787 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:49:42.794+0000 7ffb9a600700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb9410bc90 con 0x7ffb94199bb0 2024-03-08T20:49:42.787 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:49:42.794+0000 7ffb9a600700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb9410bb10 con 0x7ffb9419e480 2024-03-08T20:49:42.787 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:49:42.794+0000 7ffb9a600700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb9410be10 con 0x7ffb941077a0 2024-03-08T20:49:42.787 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:49:42.794+0000 7ffb92a00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffb94199bb0 0x7ffb9419df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:49:42.787 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:49:42.794+0000 7ffb92a00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffb94199bb0 0x7ffb9419df40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46522/0 (socket says 172.21.15.107:46522) 2024-03-08T20:49:42.787 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:49:42.794+0000 7ffb92a00700 1 -- 172.21.15.107:0/589935363 learned_addr learned my addr 172.21.15.107:0/589935363 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T20:49:42.787 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:49:42.794+0000 7ffb93400700 1 --2- 172.21.15.107:0/589935363 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffb941077a0 0x7ffb94199670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:49:42.787 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:49:42.794+0000 7ffb92a00700 1 -- 172.21.15.107:0/589935363 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffb941077a0 msgr2=0x7ffb94199670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:49:42.787 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:49:42.794+0000 7ffb92a00700 1 --2- 172.21.15.107:0/589935363 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffb941077a0 0x7ffb94199670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:49:42.787 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:49:42.794+0000 7ffb92a00700 1 -- 172.21.15.107:0/589935363 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffb9419e480 msgr2=0x7ffb9419e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:49:42.788 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:49:42.794+0000 7ffb92a00700 1 --2- 172.21.15.107:0/589935363 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffb9419e480 0x7ffb9419e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:49:42.788 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:49:42.794+0000 7ffb92a00700 1 -- 172.21.15.107:0/589935363 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb941a61f0 con 0x7ffb94199bb0 2024-03-08T20:49:42.788 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:49:42.795+0000 7ffb92a00700 1 --2- 172.21.15.107:0/589935363 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffb94199bb0 0x7ffb9419df40 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7ffb8000b780 tx=0x7ffb8000bb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:49:42.788 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:49:42.795+0000 7ffb8be00700 1 -- 172.21.15.107:0/589935363 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb80004230 con 0x7ffb94199bb0 2024-03-08T20:49:42.789 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:49:42.795+0000 7ffb8be00700 1 -- 172.21.15.107:0/589935363 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7ffb800043d0 con 0x7ffb94199bb0 2024-03-08T20:49:42.789 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:49:42.795+0000 7ffb8be00700 1 -- 172.21.15.107:0/589935363 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb80011b80 con 0x7ffb94199bb0 2024-03-08T20:49:42.789 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:49:42.795+0000 7ffb9a600700 1 -- 172.21.15.107:0/589935363 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb941a6420 con 0x7ffb94199bb0 2024-03-08T20:49:42.789 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:49:42.795+0000 7ffb9a600700 1 -- 172.21.15.107:0/589935363 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb941a6910 con 0x7ffb94199bb0 2024-03-08T20:49:42.789 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:49:42.796+0000 7ffb9a600700 1 -- 172.21.15.107:0/589935363 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb94063410 con 0x7ffb94199bb0 2024-03-08T20:49:42.790 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:49:42.797+0000 7ffb8be00700 1 -- 172.21.15.107:0/589935363 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 13) v1 ==== 46163+0+0 (secure 0 0 0) 0x7ffb800036a0 con 0x7ffb94199bb0 2024-03-08T20:49:42.790 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:49:42.798+0000 7ffb8be00700 1 --2- 172.21.15.107:0/589935363 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7ffb78039450 0x7ffb7803b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:49:42.790 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:49:42.798+0000 7ffb8be00700 1 -- 172.21.15.107:0/589935363 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7ffb8004c760 con 0x7ffb94199bb0 2024-03-08T20:49:42.790 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:49:42.798+0000 7ffb93400700 1 --2- 172.21.15.107:0/589935363 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7ffb78039450 0x7ffb7803b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:49:42.790 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:49:42.799+0000 7ffb93400700 1 --2- 172.21.15.107:0/589935363 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7ffb78039450 0x7ffb7803b910 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7ffb84009750 tx=0x7ffb8401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:49:42.794 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:49:42.803+0000 7ffb8be00700 1 -- 172.21.15.107:0/589935363 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7ffb8001f020 con 0x7ffb94199bb0 2024-03-08T20:49:43.018 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:49:43.026+0000 7ffb9a600700 1 -- 172.21.15.107:0/589935363 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7ffb941a6c30 con 0x7ffb94199bb0 2024-03-08T20:49:43.018 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:49:43.026+0000 7ffb8be00700 1 -- 172.21.15.107:0/589935363 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) v1 ==== 76+0+277 (secure 0 0 0) 0x7ffb80010e60 con 0x7ffb94199bb0 2024-03-08T20:49:43.018 INFO:teuthology.orchestra.run.smithi107.stdout:# minimal ceph.conf for dc20f0f8-dd8c-11ee-95c6-87774f69a715 2024-03-08T20:49:43.019 INFO:teuthology.orchestra.run.smithi107.stdout:[global] 2024-03-08T20:49:43.019 INFO:teuthology.orchestra.run.smithi107.stdout: fsid = dc20f0f8-dd8c-11ee-95c6-87774f69a715 2024-03-08T20:49:43.019 INFO:teuthology.orchestra.run.smithi107.stdout: mon_host = [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] 2024-03-08T20:49:43.021 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:49:43.028+0000 7ffb9a600700 1 -- 172.21.15.107:0/589935363 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7ffb78039450 msgr2=0x7ffb7803b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:49:43.021 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:49:43.029+0000 7ffb9a600700 1 --2- 172.21.15.107:0/589935363 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7ffb78039450 0x7ffb7803b910 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7ffb84009750 tx=0x7ffb8401a040 comp rx=0 tx=0).stop 2024-03-08T20:49:43.021 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:49:43.029+0000 7ffb9a600700 1 -- 172.21.15.107:0/589935363 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffb94199bb0 msgr2=0x7ffb9419df40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:49:43.021 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:49:43.029+0000 7ffb9a600700 1 --2- 172.21.15.107:0/589935363 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffb94199bb0 0x7ffb9419df40 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7ffb8000b780 tx=0x7ffb8000bb40 comp rx=0 tx=0).stop 2024-03-08T20:49:43.021 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:49:43.029+0000 7ffb9a600700 1 -- 172.21.15.107:0/589935363 shutdown_connections 2024-03-08T20:49:43.021 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:49:43.029+0000 7ffb9a600700 1 --2- 172.21.15.107:0/589935363 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffb941077a0 0x7ffb94199670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:49:43.021 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:49:43.029+0000 7ffb9a600700 1 --2- 172.21.15.107:0/589935363 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7ffb78039450 0x7ffb7803b910 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:49:43.021 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:49:43.029+0000 7ffb9a600700 1 --2- 172.21.15.107:0/589935363 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffb9419e480 0x7ffb9419e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:49:43.021 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:49:43.029+0000 7ffb9a600700 1 --2- 172.21.15.107:0/589935363 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffb94199bb0 0x7ffb9419df40 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:49:43.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:49:43.029+0000 7ffb9a600700 1 -- 172.21.15.107:0/589935363 >> 172.21.15.107:0/589935363 conn(0x7ffb940fd2b0 msgr2=0x7ffb940fdca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:49:43.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:49:43.029+0000 7ffb9a600700 1 -- 172.21.15.107:0/589935363 shutdown_connections 2024-03-08T20:49:43.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:49:43.029+0000 7ffb9a600700 1 -- 172.21.15.107:0/589935363 wait complete. 2024-03-08T20:49:43.226 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:49:42 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:49:43.226 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:49:42 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:49:43.226 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:49:42 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:49:43.226 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:49:42 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:49:43.226 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:49:42 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:49:43.226 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:49:42 smithi107 ceph-mon[31381]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T20:49:43.226 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:49:42 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:49:43.227 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:49:42 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:49:43.227 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:49:42 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:49:43.227 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:49:42 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:49:43.227 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:49:42 smithi107 ceph-mon[31381]: Reconfiguring mon.a (unknown last config time)... 2024-03-08T20:49:43.227 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:49:42 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-08T20:49:43.227 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:49:42 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-08T20:49:43.227 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:49:42 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:49:43.227 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:49:42 smithi107 ceph-mon[31381]: Reconfiguring daemon mon.a on smithi107 2024-03-08T20:49:43.273 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:42 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:49:43.274 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:42 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:49:43.274 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:42 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:49:43.274 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:42 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:49:43.274 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:42 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:49:43.274 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:42 smithi149 ceph-mon[35042]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T20:49:43.274 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:42 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:49:43.274 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:42 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:49:43.274 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:42 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:49:43.274 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:42 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:49:43.274 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:42 smithi149 ceph-mon[35042]: Reconfiguring mon.a (unknown last config time)... 2024-03-08T20:49:43.274 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:42 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-08T20:49:43.274 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:42 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-08T20:49:43.275 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:42 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:49:43.275 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:42 smithi149 ceph-mon[35042]: Reconfiguring daemon mon.a on smithi107 2024-03-08T20:49:43.387 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:42 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:49:43.387 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:42 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:49:43.387 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:42 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:49:43.387 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:42 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:49:43.387 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:42 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:49:43.387 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:42 smithi204 ceph-mon[34935]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T20:49:43.387 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:42 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:49:43.387 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:42 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:49:43.387 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:42 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:49:43.388 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:42 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:49:43.388 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:42 smithi204 ceph-mon[34935]: Reconfiguring mon.a (unknown last config time)... 2024-03-08T20:49:43.388 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:42 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-08T20:49:43.388 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:42 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-08T20:49:43.388 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:42 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:49:43.388 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:42 smithi204 ceph-mon[34935]: Reconfiguring daemon mon.a on smithi107 2024-03-08T20:49:44.225 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:49:43 smithi107 ceph-mon[31381]: from='client.? 172.21.15.107:0/589935363' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:49:44.273 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:43 smithi149 ceph-mon[35042]: from='client.? 172.21.15.107:0/589935363' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:49:44.349 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-03-08T20:49:44.350 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-03-08T20:49:44.350 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd of=/etc/ceph/ceph.conf 2024-03-08T20:49:44.386 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:43 smithi204 ceph-mon[34935]: from='client.? 172.21.15.107:0/589935363' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:49:44.447 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-03-08T20:49:44.447 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-08T20:49:44.525 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-03-08T20:49:44.525 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd of=/etc/ceph/ceph.conf 2024-03-08T20:49:44.626 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-03-08T20:49:44.627 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-08T20:49:44.765 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-03-08T20:49:44.765 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd of=/etc/ceph/ceph.conf 2024-03-08T20:49:44.872 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-03-08T20:49:44.872 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-08T20:49:45.011 INFO:tasks.cephadm:Adding mgr.x on smithi107 2024-03-08T20:49:45.011 INFO:tasks.cephadm:Adding mgr.y on smithi149 2024-03-08T20:49:45.011 INFO:tasks.cephadm:Adding mgr.z on smithi204 2024-03-08T20:49:45.011 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dc20f0f8-dd8c-11ee-95c6-87774f69a715 -- ceph orch apply mgr '3;smithi107=x;smithi149=y;smithi204=z' 2024-03-08T20:49:45.130 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:44 smithi204 ceph-mon[34935]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T20:49:45.130 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:44 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:49:45.131 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:44 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:49:45.131 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:44 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-08T20:49:45.131 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:44 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-08T20:49:45.131 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:44 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:49:45.226 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:49:44 smithi107 ceph-mon[31381]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T20:49:45.226 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:49:44 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:49:45.226 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:49:44 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:49:45.227 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:49:44 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-08T20:49:45.227 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:49:44 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-08T20:49:45.227 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:49:44 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:49:45.273 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:44 smithi149 ceph-mon[35042]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T20:49:45.273 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:44 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:49:45.274 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:44 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:49:45.274 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:44 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-08T20:49:45.274 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:44 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-08T20:49:45.274 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:44 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:49:45.470 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/mon.c/config 2024-03-08T20:49:46.069 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:45 smithi149 ceph-mon[35042]: Reconfiguring mon.b (monmap changed)... 2024-03-08T20:49:46.069 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:45 smithi149 ceph-mon[35042]: Reconfiguring daemon mon.b on smithi149 2024-03-08T20:49:46.136 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:45 smithi204 ceph-mon[34935]: Reconfiguring mon.b (monmap changed)... 2024-03-08T20:49:46.136 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:45 smithi204 ceph-mon[34935]: Reconfiguring daemon mon.b on smithi149 2024-03-08T20:49:46.475 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:49:45 smithi107 ceph-mon[31381]: Reconfiguring mon.b (monmap changed)... 2024-03-08T20:49:46.476 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:49:45 smithi107 ceph-mon[31381]: Reconfiguring daemon mon.b on smithi149 2024-03-08T20:49:47.024 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:46 smithi149 ceph-mon[35042]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T20:49:47.024 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:47 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:49:47.024 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:47 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:49:47.024 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:47 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-08T20:49:47.024 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:47 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-08T20:49:47.024 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:47 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:49:47.290 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:46 smithi204 ceph-mon[34935]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T20:49:47.290 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:46 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:49:47.290 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:46 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:49:47.290 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:46 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-08T20:49:47.290 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:46 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-08T20:49:47.290 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:46 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:49:47.456 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:47.464+0000 7f9aad200700 1 -- 172.21.15.204:0/2375744556 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9aa8101910 msgr2=0x7f9aa8101d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:49:47.456 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:47.464+0000 7f9aad200700 1 --2- 172.21.15.204:0/2375744556 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9aa8101910 0x7f9aa8101d30 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f9a98009910 tx=0x7f9a98009c20 comp rx=0 tx=0).stop 2024-03-08T20:49:47.456 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:47.464+0000 7f9aad200700 1 -- 172.21.15.204:0/2375744556 shutdown_connections 2024-03-08T20:49:47.457 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:47.464+0000 7f9aad200700 1 --2- 172.21.15.204:0/2375744556 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9aa8101910 0x7f9aa8101d30 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:49:47.457 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:47.464+0000 7f9aad200700 1 -- 172.21.15.204:0/2375744556 >> 172.21.15.204:0/2375744556 conn(0x7f9aa80fceb0 msgr2=0x7f9aa80ff2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:49:47.457 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:47.464+0000 7f9aad200700 1 -- 172.21.15.204:0/2375744556 shutdown_connections 2024-03-08T20:49:47.457 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:47.465+0000 7f9aad200700 1 -- 172.21.15.204:0/2375744556 wait complete. 2024-03-08T20:49:47.458 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:47.466+0000 7f9aad200700 1 Processor -- start 2024-03-08T20:49:47.458 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:47.466+0000 7f9aad200700 1 -- start start 2024-03-08T20:49:47.458 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:47.466+0000 7f9aad200700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9aa8101910 0x7f9aa8199270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:49:47.459 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:47.466+0000 7f9aad200700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9aa81997b0 0x7f9aa819e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:49:47.459 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:47.466+0000 7f9aad200700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9aa8199c30 0x7f9aa81a0db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:49:47.459 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:47.466+0000 7f9aad200700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9aa8107850 con 0x7f9aa8101910 2024-03-08T20:49:47.459 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:47.466+0000 7f9aad200700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9aa81076d0 con 0x7f9aa81997b0 2024-03-08T20:49:47.459 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:47.466+0000 7f9aad200700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9aa81079d0 con 0x7f9aa8199c30 2024-03-08T20:49:47.459 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:47.467+0000 7f9aa6800700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9aa8199c30 0x7f9aa81a0db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:49:47.459 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:47.467+0000 7f9aa5e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9aa8101910 0x7f9aa8199270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:49:47.459 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:47.467+0000 7f9aa6800700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9aa8199c30 0x7f9aa81a0db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:33172/0 (socket says 172.21.15.204:33172) 2024-03-08T20:49:47.459 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:47.467+0000 7f9aa6800700 1 -- 172.21.15.204:0/238374231 learned_addr learned my addr 172.21.15.204:0/238374231 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-03-08T20:49:47.459 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:47.467+0000 7f9aa5400700 1 --2- 172.21.15.204:0/238374231 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9aa81997b0 0x7f9aa819e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:49:47.460 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:47.467+0000 7f9aa6800700 1 -- 172.21.15.204:0/238374231 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9aa81997b0 msgr2=0x7f9aa819e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:49:47.460 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:47.467+0000 7f9aa6800700 1 --2- 172.21.15.204:0/238374231 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9aa81997b0 0x7f9aa819e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:49:47.460 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:47.467+0000 7f9aa6800700 1 -- 172.21.15.204:0/238374231 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9aa8101910 msgr2=0x7f9aa8199270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:49:47.460 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:47.467+0000 7f9aa6800700 1 --2- 172.21.15.204:0/238374231 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9aa8101910 0x7f9aa8199270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:49:47.460 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:47.467+0000 7f9aa6800700 1 -- 172.21.15.204:0/238374231 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9aa81a1680 con 0x7f9aa8199c30 2024-03-08T20:49:47.460 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:47.467+0000 7f9aa5400700 1 --2- 172.21.15.204:0/238374231 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9aa81997b0 0x7f9aa819e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T20:49:47.461 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:47.467+0000 7f9aa6800700 1 --2- 172.21.15.204:0/238374231 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9aa8199c30 0x7f9aa81a0db0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f9aa000ba50 tx=0x7f9aa000bd60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:49:47.461 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:47.468+0000 7f9a9ea00700 1 -- 172.21.15.204:0/238374231 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9aa0019070 con 0x7f9aa8199c30 2024-03-08T20:49:47.461 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:47.468+0000 7f9aad200700 1 -- 172.21.15.204:0/238374231 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9aa81a1970 con 0x7f9aa8199c30 2024-03-08T20:49:47.461 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:47.468+0000 7f9a9ea00700 1 -- 172.21.15.204:0/238374231 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f9aa0014440 con 0x7f9aa8199c30 2024-03-08T20:49:47.461 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:47.468+0000 7f9a9ea00700 1 -- 172.21.15.204:0/238374231 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9aa000ce60 con 0x7f9aa8199c30 2024-03-08T20:49:47.462 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:47.468+0000 7f9aad200700 1 -- 172.21.15.204:0/238374231 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9aa81a1f30 con 0x7f9aa8199c30 2024-03-08T20:49:47.462 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:47.469+0000 7f9a9ea00700 1 -- 172.21.15.204:0/238374231 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 13) v1 ==== 46163+0+0 (secure 0 0 0) 0x7f9aa001c070 con 0x7f9aa8199c30 2024-03-08T20:49:47.462 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:47.469+0000 7f9aad200700 1 -- 172.21.15.204:0/238374231 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9aa8063410 con 0x7f9aa8199c30 2024-03-08T20:49:47.462 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:47.469+0000 7f9a9ea00700 1 --2- 172.21.15.204:0/238374231 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f9a8c039450 0x7f9a8c03b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:49:47.462 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:47.470+0000 7f9a9ea00700 1 -- 172.21.15.204:0/238374231 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f9aa004b670 con 0x7f9aa8199c30 2024-03-08T20:49:47.462 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:47.470+0000 7f9aa5e00700 1 --2- 172.21.15.204:0/238374231 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f9a8c039450 0x7f9a8c03b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:49:47.462 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:47.471+0000 7f9aa5e00700 1 --2- 172.21.15.204:0/238374231 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f9a8c039450 0x7f9a8c03b910 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f9a98005190 tx=0x7f9a9801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:49:47.467 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:47.476+0000 7f9a9ea00700 1 -- 172.21.15.204:0/238374231 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f9aa0012500 con 0x7f9aa8199c30 2024-03-08T20:49:47.475 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:49:46 smithi107 ceph-mon[31381]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T20:49:47.475 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:49:46 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:49:47.475 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:49:46 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:49:47.476 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:49:46 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-08T20:49:47.476 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:49:46 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-08T20:49:47.476 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:49:46 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:49:47.726 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:47.734+0000 7f9aad200700 1 -- 172.21.15.204:0/238374231 --> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi107=x;smithi149=y;smithi204=z", "target": ["mon-mgr", ""]}) v1 -- 0x7f9aa80622b0 con 0x7f9a8c039450 2024-03-08T20:49:47.740 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:47.749+0000 7f9a9ea00700 1 -- 172.21.15.204:0/238374231 <== mgr.14152 v2:172.21.15.107:6800/3376636618 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f9aa80622b0 con 0x7f9a8c039450 2024-03-08T20:49:47.740 INFO:teuthology.orchestra.run.smithi204.stdout:Scheduled mgr update... 2024-03-08T20:49:47.743 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:47.751+0000 7f9aad200700 1 -- 172.21.15.204:0/238374231 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f9a8c039450 msgr2=0x7f9a8c03b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:49:47.743 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:47.751+0000 7f9aad200700 1 --2- 172.21.15.204:0/238374231 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f9a8c039450 0x7f9a8c03b910 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f9a98005190 tx=0x7f9a9801a040 comp rx=0 tx=0).stop 2024-03-08T20:49:47.745 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:47.751+0000 7f9aad200700 1 -- 172.21.15.204:0/238374231 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9aa8199c30 msgr2=0x7f9aa81a0db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:49:47.745 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:47.751+0000 7f9aad200700 1 --2- 172.21.15.204:0/238374231 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9aa8199c30 0x7f9aa81a0db0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f9aa000ba50 tx=0x7f9aa000bd60 comp rx=0 tx=0).stop 2024-03-08T20:49:47.745 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:47.752+0000 7f9aad200700 1 -- 172.21.15.204:0/238374231 shutdown_connections 2024-03-08T20:49:47.745 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:47.752+0000 7f9aad200700 1 --2- 172.21.15.204:0/238374231 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9aa8199c30 0x7f9aa81a0db0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:49:47.745 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:47.752+0000 7f9aad200700 1 --2- 172.21.15.204:0/238374231 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f9a8c039450 0x7f9a8c03b910 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:49:47.745 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:47.752+0000 7f9aad200700 1 --2- 172.21.15.204:0/238374231 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9aa8101910 0x7f9aa8199270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:49:47.746 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:47.752+0000 7f9aad200700 1 --2- 172.21.15.204:0/238374231 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9aa81997b0 0x7f9aa819e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:49:47.746 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:47.752+0000 7f9aad200700 1 -- 172.21.15.204:0/238374231 >> 172.21.15.204:0/238374231 conn(0x7f9aa80fceb0 msgr2=0x7f9aa80fdb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:49:47.746 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:47.752+0000 7f9aad200700 1 -- 172.21.15.204:0/238374231 shutdown_connections 2024-03-08T20:49:47.746 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:49:47.752+0000 7f9aad200700 1 -- 172.21.15.204:0/238374231 wait complete. 2024-03-08T20:49:48.273 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:48 smithi149 ceph-mon[35042]: Reconfiguring mon.c (monmap changed)... 2024-03-08T20:49:48.273 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:48 smithi149 ceph-mon[35042]: Reconfiguring daemon mon.c on smithi204 2024-03-08T20:49:48.273 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:48 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:49:48.293 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:48 smithi204 ceph-mon[34935]: Reconfiguring mon.c (monmap changed)... 2024-03-08T20:49:48.293 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:48 smithi204 ceph-mon[34935]: Reconfiguring daemon mon.c on smithi204 2024-03-08T20:49:48.293 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:48 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:49:48.475 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:49:48 smithi107 ceph-mon[31381]: Reconfiguring mon.c (monmap changed)... 2024-03-08T20:49:48.475 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:49:48 smithi107 ceph-mon[31381]: Reconfiguring daemon mon.c on smithi204 2024-03-08T20:49:48.476 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:49:48 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:49:49.273 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:49 smithi149 ceph-mon[35042]: from='client.24106 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi107=x;smithi149=y;smithi204=z", "target": ["mon-mgr", ""]}]: dispatch 2024-03-08T20:49:49.274 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:49 smithi149 ceph-mon[35042]: Saving service mgr spec with placement smithi107=x;smithi149=y;smithi204=z;count:3 2024-03-08T20:49:49.274 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:49 smithi149 ceph-mon[35042]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T20:49:49.387 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:49 smithi204 ceph-mon[34935]: from='client.24106 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi107=x;smithi149=y;smithi204=z", "target": ["mon-mgr", ""]}]: dispatch 2024-03-08T20:49:49.387 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:49 smithi204 ceph-mon[34935]: Saving service mgr spec with placement smithi107=x;smithi149=y;smithi204=z;count:3 2024-03-08T20:49:49.387 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:49 smithi204 ceph-mon[34935]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T20:49:49.475 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:49:49 smithi107 ceph-mon[31381]: from='client.24106 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi107=x;smithi149=y;smithi204=z", "target": ["mon-mgr", ""]}]: dispatch 2024-03-08T20:49:49.476 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:49:49 smithi107 ceph-mon[31381]: Saving service mgr spec with placement smithi107=x;smithi149=y;smithi204=z;count:3 2024-03-08T20:49:49.476 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:49:49 smithi107 ceph-mon[31381]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T20:49:49.496 DEBUG:teuthology.orchestra.run.smithi149:mgr.y> sudo journalctl -f -n 0 -u ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715@mgr.y.service 2024-03-08T20:49:49.503 DEBUG:teuthology.orchestra.run.smithi204:mgr.z> sudo journalctl -f -n 0 -u ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715@mgr.z.service 2024-03-08T20:49:49.510 INFO:tasks.cephadm:Deploying OSDs... 2024-03-08T20:49:49.510 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-03-08T20:49:49.511 DEBUG:teuthology.orchestra.run.smithi107:> dd if=/scratch_devs of=/dev/stdout 2024-03-08T20:49:49.568 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-08T20:49:49.568 DEBUG:teuthology.orchestra.run.smithi107:> stat /dev/vg_nvme/lv_1 2024-03-08T20:49:49.621 INFO:teuthology.orchestra.run.smithi107.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-08T20:49:49.621 INFO:teuthology.orchestra.run.smithi107.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-08T20:49:49.621 INFO:teuthology.orchestra.run.smithi107.stdout:Device: 5h/5d Inode: 632 Links: 1 2024-03-08T20:49:49.621 INFO:teuthology.orchestra.run.smithi107.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-08T20:49:49.621 INFO:teuthology.orchestra.run.smithi107.stdout:Context: system_u:object_r:device_t:s0 2024-03-08T20:49:49.621 INFO:teuthology.orchestra.run.smithi107.stdout:Access: 2024-03-08 20:48:35.954990512 +0000 2024-03-08T20:49:49.621 INFO:teuthology.orchestra.run.smithi107.stdout:Modify: 2024-03-08 20:42:29.048523095 +0000 2024-03-08T20:49:49.621 INFO:teuthology.orchestra.run.smithi107.stdout:Change: 2024-03-08 20:42:29.048523095 +0000 2024-03-08T20:49:49.621 INFO:teuthology.orchestra.run.smithi107.stdout: Birth: 2024-03-08 20:42:29.048523095 +0000 2024-03-08T20:49:49.622 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-08T20:49:49.712 INFO:teuthology.orchestra.run.smithi107.stderr:1+0 records in 2024-03-08T20:49:49.712 INFO:teuthology.orchestra.run.smithi107.stderr:1+0 records out 2024-03-08T20:49:49.712 INFO:teuthology.orchestra.run.smithi107.stderr:512 bytes copied, 0.000366722 s, 1.4 MB/s 2024-03-08T20:49:49.716 DEBUG:teuthology.orchestra.run.smithi107:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-08T20:49:49.774 DEBUG:teuthology.orchestra.run.smithi107:> stat /dev/vg_nvme/lv_2 2024-03-08T20:49:49.823 INFO:teuthology.orchestra.run.smithi107.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-08T20:49:49.823 INFO:teuthology.orchestra.run.smithi107.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-08T20:49:49.823 INFO:teuthology.orchestra.run.smithi107.stdout:Device: 5h/5d Inode: 653 Links: 1 2024-03-08T20:49:49.823 INFO:teuthology.orchestra.run.smithi107.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-08T20:49:49.823 INFO:teuthology.orchestra.run.smithi107.stdout:Context: system_u:object_r:device_t:s0 2024-03-08T20:49:49.823 INFO:teuthology.orchestra.run.smithi107.stdout:Access: 2024-03-08 20:48:36.011989589 +0000 2024-03-08T20:49:49.823 INFO:teuthology.orchestra.run.smithi107.stdout:Modify: 2024-03-08 20:42:29.595514100 +0000 2024-03-08T20:49:49.823 INFO:teuthology.orchestra.run.smithi107.stdout:Change: 2024-03-08 20:42:29.595514100 +0000 2024-03-08T20:49:49.823 INFO:teuthology.orchestra.run.smithi107.stdout: Birth: 2024-03-08 20:42:29.595514100 +0000 2024-03-08T20:49:49.824 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-08T20:49:49.913 INFO:teuthology.orchestra.run.smithi107.stderr:1+0 records in 2024-03-08T20:49:49.913 INFO:teuthology.orchestra.run.smithi107.stderr:1+0 records out 2024-03-08T20:49:49.913 INFO:teuthology.orchestra.run.smithi107.stderr:512 bytes copied, 0.000354622 s, 1.4 MB/s 2024-03-08T20:49:49.917 DEBUG:teuthology.orchestra.run.smithi107:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-08T20:49:49.974 DEBUG:teuthology.orchestra.run.smithi107:> stat /dev/vg_nvme/lv_3 2024-03-08T20:49:50.020 INFO:teuthology.orchestra.run.smithi107.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-08T20:49:50.020 INFO:teuthology.orchestra.run.smithi107.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-08T20:49:50.020 INFO:teuthology.orchestra.run.smithi107.stdout:Device: 5h/5d Inode: 671 Links: 1 2024-03-08T20:49:50.020 INFO:teuthology.orchestra.run.smithi107.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-08T20:49:50.020 INFO:teuthology.orchestra.run.smithi107.stdout:Context: system_u:object_r:device_t:s0 2024-03-08T20:49:50.020 INFO:teuthology.orchestra.run.smithi107.stdout:Access: 2024-03-08 20:48:36.063988747 +0000 2024-03-08T20:49:50.021 INFO:teuthology.orchestra.run.smithi107.stdout:Modify: 2024-03-08 20:42:30.162504775 +0000 2024-03-08T20:49:50.021 INFO:teuthology.orchestra.run.smithi107.stdout:Change: 2024-03-08 20:42:30.162504775 +0000 2024-03-08T20:49:50.021 INFO:teuthology.orchestra.run.smithi107.stdout: Birth: 2024-03-08 20:42:30.162504775 +0000 2024-03-08T20:49:50.021 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-08T20:49:50.154 INFO:teuthology.orchestra.run.smithi107.stderr:1+0 records in 2024-03-08T20:49:50.155 INFO:teuthology.orchestra.run.smithi107.stderr:1+0 records out 2024-03-08T20:49:50.155 INFO:teuthology.orchestra.run.smithi107.stderr:512 bytes copied, 0.000331546 s, 1.5 MB/s 2024-03-08T20:49:50.159 DEBUG:teuthology.orchestra.run.smithi107:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-08T20:49:50.214 DEBUG:teuthology.orchestra.run.smithi107:> stat /dev/vg_nvme/lv_4 2024-03-08T20:49:50.261 INFO:teuthology.orchestra.run.smithi107.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-08T20:49:50.261 INFO:teuthology.orchestra.run.smithi107.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-08T20:49:50.261 INFO:teuthology.orchestra.run.smithi107.stdout:Device: 5h/5d Inode: 689 Links: 1 2024-03-08T20:49:50.261 INFO:teuthology.orchestra.run.smithi107.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-08T20:49:50.261 INFO:teuthology.orchestra.run.smithi107.stdout:Context: system_u:object_r:device_t:s0 2024-03-08T20:49:50.261 INFO:teuthology.orchestra.run.smithi107.stdout:Access: 2024-03-08 20:48:36.117987873 +0000 2024-03-08T20:49:50.261 INFO:teuthology.orchestra.run.smithi107.stdout:Modify: 2024-03-08 20:42:30.721495580 +0000 2024-03-08T20:49:50.261 INFO:teuthology.orchestra.run.smithi107.stdout:Change: 2024-03-08 20:42:30.721495580 +0000 2024-03-08T20:49:50.261 INFO:teuthology.orchestra.run.smithi107.stdout: Birth: 2024-03-08 20:42:30.721495580 +0000 2024-03-08T20:49:50.262 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-08T20:49:50.335 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:49:50 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:49:50.336 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:49:50 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:49:50.336 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:49:50 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T20:49:50.336 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:49:50 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:49:50.336 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:49:50 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T20:49:50.336 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:49:50 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:49:50.336 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:49:50 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-08T20:49:50.336 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:49:50 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-08T20:49:50.336 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:49:50 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-08T20:49:50.336 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:49:50 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:49:50.336 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:49:50 smithi107 ceph-mon[31381]: Deploying daemon mgr.z on smithi204 2024-03-08T20:49:50.337 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:50 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:49:50.337 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:50 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:49:50.337 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:50 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T20:49:50.337 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:50 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:49:50.337 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:50 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T20:49:50.337 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:50 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:49:50.337 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:50 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-08T20:49:50.337 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:50 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-08T20:49:50.337 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:50 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-08T20:49:50.337 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:50 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:49:50.337 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:50 smithi204 ceph-mon[34935]: Deploying daemon mgr.z on smithi204 2024-03-08T20:49:50.344 INFO:teuthology.orchestra.run.smithi107.stderr:1+0 records in 2024-03-08T20:49:50.344 INFO:teuthology.orchestra.run.smithi107.stderr:1+0 records out 2024-03-08T20:49:50.344 INFO:teuthology.orchestra.run.smithi107.stderr:512 bytes copied, 0.000332807 s, 1.5 MB/s 2024-03-08T20:49:50.347 DEBUG:teuthology.orchestra.run.smithi107:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-08T20:49:50.400 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-03-08T20:49:50.462 DEBUG:teuthology.orchestra.run.smithi149:> dd if=/scratch_devs of=/dev/stdout 2024-03-08T20:49:50.520 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-08T20:49:50.520 DEBUG:teuthology.orchestra.run.smithi149:> stat /dev/vg_nvme/lv_1 2024-03-08T20:49:50.525 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:50 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:49:50.525 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:50 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:49:50.525 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:50 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T20:49:50.526 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:50 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:49:50.526 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:50 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T20:49:50.526 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:50 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:49:50.526 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:50 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-08T20:49:50.526 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:50 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-08T20:49:50.526 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:50 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-08T20:49:50.526 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:50 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:49:50.526 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:50 smithi149 ceph-mon[35042]: Deploying daemon mgr.z on smithi204 2024-03-08T20:49:50.572 INFO:teuthology.orchestra.run.smithi149.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-08T20:49:50.572 INFO:teuthology.orchestra.run.smithi149.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-08T20:49:50.572 INFO:teuthology.orchestra.run.smithi149.stdout:Device: 5h/5d Inode: 632 Links: 1 2024-03-08T20:49:50.572 INFO:teuthology.orchestra.run.smithi149.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-08T20:49:50.572 INFO:teuthology.orchestra.run.smithi149.stdout:Context: system_u:object_r:device_t:s0 2024-03-08T20:49:50.572 INFO:teuthology.orchestra.run.smithi149.stdout:Access: 2024-03-08 20:49:31.751498130 +0000 2024-03-08T20:49:50.572 INFO:teuthology.orchestra.run.smithi149.stdout:Modify: 2024-03-08 20:42:30.380602181 +0000 2024-03-08T20:49:50.572 INFO:teuthology.orchestra.run.smithi149.stdout:Change: 2024-03-08 20:42:30.380602181 +0000 2024-03-08T20:49:50.572 INFO:teuthology.orchestra.run.smithi149.stdout: Birth: 2024-03-08 20:42:30.380602181 +0000 2024-03-08T20:49:50.573 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-08T20:49:50.660 INFO:teuthology.orchestra.run.smithi149.stderr:1+0 records in 2024-03-08T20:49:50.661 INFO:teuthology.orchestra.run.smithi149.stderr:1+0 records out 2024-03-08T20:49:50.661 INFO:teuthology.orchestra.run.smithi149.stderr:512 bytes copied, 0.000356476 s, 1.4 MB/s 2024-03-08T20:49:50.665 DEBUG:teuthology.orchestra.run.smithi149:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-08T20:49:50.722 DEBUG:teuthology.orchestra.run.smithi149:> stat /dev/vg_nvme/lv_2 2024-03-08T20:49:50.774 INFO:teuthology.orchestra.run.smithi149.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-08T20:49:50.774 INFO:teuthology.orchestra.run.smithi149.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-08T20:49:50.774 INFO:teuthology.orchestra.run.smithi149.stdout:Device: 5h/5d Inode: 651 Links: 1 2024-03-08T20:49:50.774 INFO:teuthology.orchestra.run.smithi149.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-08T20:49:50.774 INFO:teuthology.orchestra.run.smithi149.stdout:Context: system_u:object_r:device_t:s0 2024-03-08T20:49:50.774 INFO:teuthology.orchestra.run.smithi149.stdout:Access: 2024-03-08 20:49:31.752498110 +0000 2024-03-08T20:49:50.774 INFO:teuthology.orchestra.run.smithi149.stdout:Modify: 2024-03-08 20:42:30.967590503 +0000 2024-03-08T20:49:50.774 INFO:teuthology.orchestra.run.smithi149.stdout:Change: 2024-03-08 20:42:30.967590503 +0000 2024-03-08T20:49:50.775 INFO:teuthology.orchestra.run.smithi149.stdout: Birth: 2024-03-08 20:42:30.967590503 +0000 2024-03-08T20:49:50.775 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-08T20:49:50.863 INFO:teuthology.orchestra.run.smithi149.stderr:1+0 records in 2024-03-08T20:49:50.863 INFO:teuthology.orchestra.run.smithi149.stderr:1+0 records out 2024-03-08T20:49:50.864 INFO:teuthology.orchestra.run.smithi149.stderr:512 bytes copied, 0.000339353 s, 1.5 MB/s 2024-03-08T20:49:50.868 DEBUG:teuthology.orchestra.run.smithi149:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-08T20:49:50.922 DEBUG:teuthology.orchestra.run.smithi149:> stat /dev/vg_nvme/lv_3 2024-03-08T20:49:50.971 INFO:teuthology.orchestra.run.smithi149.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-08T20:49:50.971 INFO:teuthology.orchestra.run.smithi149.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-08T20:49:50.971 INFO:teuthology.orchestra.run.smithi149.stdout:Device: 5h/5d Inode: 669 Links: 1 2024-03-08T20:49:50.971 INFO:teuthology.orchestra.run.smithi149.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-08T20:49:50.971 INFO:teuthology.orchestra.run.smithi149.stdout:Context: system_u:object_r:device_t:s0 2024-03-08T20:49:50.971 INFO:teuthology.orchestra.run.smithi149.stdout:Access: 2024-03-08 20:49:31.752498110 +0000 2024-03-08T20:49:50.972 INFO:teuthology.orchestra.run.smithi149.stdout:Modify: 2024-03-08 20:42:31.532579263 +0000 2024-03-08T20:49:50.972 INFO:teuthology.orchestra.run.smithi149.stdout:Change: 2024-03-08 20:42:31.532579263 +0000 2024-03-08T20:49:50.972 INFO:teuthology.orchestra.run.smithi149.stdout: Birth: 2024-03-08 20:42:31.532579263 +0000 2024-03-08T20:49:50.972 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-08T20:49:51.052 INFO:teuthology.orchestra.run.smithi149.stderr:1+0 records in 2024-03-08T20:49:51.052 INFO:teuthology.orchestra.run.smithi149.stderr:1+0 records out 2024-03-08T20:49:51.053 INFO:teuthology.orchestra.run.smithi149.stderr:512 bytes copied, 0.000336163 s, 1.5 MB/s 2024-03-08T20:49:51.056 DEBUG:teuthology.orchestra.run.smithi149:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-08T20:49:51.110 DEBUG:teuthology.orchestra.run.smithi149:> stat /dev/vg_nvme/lv_4 2024-03-08T20:49:51.159 INFO:teuthology.orchestra.run.smithi149.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-08T20:49:51.160 INFO:teuthology.orchestra.run.smithi149.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-08T20:49:51.160 INFO:teuthology.orchestra.run.smithi149.stdout:Device: 5h/5d Inode: 691 Links: 1 2024-03-08T20:49:51.160 INFO:teuthology.orchestra.run.smithi149.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-08T20:49:51.160 INFO:teuthology.orchestra.run.smithi149.stdout:Context: system_u:object_r:device_t:s0 2024-03-08T20:49:51.160 INFO:teuthology.orchestra.run.smithi149.stdout:Access: 2024-03-08 20:49:31.753498090 +0000 2024-03-08T20:49:51.160 INFO:teuthology.orchestra.run.smithi149.stdout:Modify: 2024-03-08 20:42:32.108567804 +0000 2024-03-08T20:49:51.160 INFO:teuthology.orchestra.run.smithi149.stdout:Change: 2024-03-08 20:42:32.108567804 +0000 2024-03-08T20:49:51.160 INFO:teuthology.orchestra.run.smithi149.stdout: Birth: 2024-03-08 20:42:32.108567804 +0000 2024-03-08T20:49:51.160 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-08T20:49:51.240 INFO:teuthology.orchestra.run.smithi149.stderr:1+0 records in 2024-03-08T20:49:51.241 INFO:teuthology.orchestra.run.smithi149.stderr:1+0 records out 2024-03-08T20:49:51.241 INFO:teuthology.orchestra.run.smithi149.stderr:512 bytes copied, 0.000333006 s, 1.5 MB/s 2024-03-08T20:49:51.244 DEBUG:teuthology.orchestra.run.smithi149:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-08T20:49:51.295 DEBUG:teuthology.orchestra.run.smithi204:> set -ex 2024-03-08T20:49:51.295 DEBUG:teuthology.orchestra.run.smithi204:> dd if=/scratch_devs of=/dev/stdout 2024-03-08T20:49:51.347 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-08T20:49:51.347 DEBUG:teuthology.orchestra.run.smithi204:> stat /dev/vg_nvme/lv_1 2024-03-08T20:49:51.386 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:51 smithi204 ceph-mon[34935]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T20:49:51.388 INFO:teuthology.orchestra.run.smithi204.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-08T20:49:51.388 INFO:teuthology.orchestra.run.smithi204.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-08T20:49:51.389 INFO:teuthology.orchestra.run.smithi204.stdout:Device: 5h/5d Inode: 631 Links: 1 2024-03-08T20:49:51.389 INFO:teuthology.orchestra.run.smithi204.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-08T20:49:51.389 INFO:teuthology.orchestra.run.smithi204.stdout:Context: system_u:object_r:device_t:s0 2024-03-08T20:49:51.389 INFO:teuthology.orchestra.run.smithi204.stdout:Access: 2024-03-08 20:49:51.313703102 +0000 2024-03-08T20:49:51.389 INFO:teuthology.orchestra.run.smithi204.stdout:Modify: 2024-03-08 20:42:29.512624244 +0000 2024-03-08T20:49:51.389 INFO:teuthology.orchestra.run.smithi204.stdout:Change: 2024-03-08 20:42:29.512624244 +0000 2024-03-08T20:49:51.389 INFO:teuthology.orchestra.run.smithi204.stdout: Birth: 2024-03-08 20:42:29.512624244 +0000 2024-03-08T20:49:51.389 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-08T20:49:51.475 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:49:51 smithi107 ceph-mon[31381]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T20:49:51.523 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:51 smithi149 ceph-mon[35042]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T20:49:51.720 INFO:teuthology.orchestra.run.smithi204.stderr:1+0 records in 2024-03-08T20:49:51.720 INFO:teuthology.orchestra.run.smithi204.stderr:1+0 records out 2024-03-08T20:49:51.721 INFO:teuthology.orchestra.run.smithi204.stderr:512 bytes copied, 0.000309855 s, 1.7 MB/s 2024-03-08T20:49:51.724 DEBUG:teuthology.orchestra.run.smithi204:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-08T20:49:51.769 DEBUG:teuthology.orchestra.run.smithi204:> stat /dev/vg_nvme/lv_2 2024-03-08T20:49:51.849 INFO:teuthology.orchestra.run.smithi204.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-08T20:49:51.849 INFO:teuthology.orchestra.run.smithi204.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-08T20:49:51.849 INFO:teuthology.orchestra.run.smithi204.stdout:Device: 5h/5d Inode: 653 Links: 1 2024-03-08T20:49:51.849 INFO:teuthology.orchestra.run.smithi204.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-08T20:49:51.849 INFO:teuthology.orchestra.run.smithi204.stdout:Context: system_u:object_r:device_t:s0 2024-03-08T20:49:51.849 INFO:teuthology.orchestra.run.smithi204.stdout:Access: 2024-03-08 20:49:51.633698488 +0000 2024-03-08T20:49:51.849 INFO:teuthology.orchestra.run.smithi204.stdout:Modify: 2024-03-08 20:42:30.070615937 +0000 2024-03-08T20:49:51.850 INFO:teuthology.orchestra.run.smithi204.stdout:Change: 2024-03-08 20:42:30.070615937 +0000 2024-03-08T20:49:51.850 INFO:teuthology.orchestra.run.smithi204.stdout: Birth: 2024-03-08 20:42:30.070615937 +0000 2024-03-08T20:49:51.850 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-08T20:49:51.962 INFO:teuthology.orchestra.run.smithi204.stderr:1+0 records in 2024-03-08T20:49:51.962 INFO:teuthology.orchestra.run.smithi204.stderr:1+0 records out 2024-03-08T20:49:51.962 INFO:teuthology.orchestra.run.smithi204.stderr:512 bytes copied, 0.000339047 s, 1.5 MB/s 2024-03-08T20:49:51.965 DEBUG:teuthology.orchestra.run.smithi204:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-08T20:49:52.004 DEBUG:teuthology.orchestra.run.smithi204:> stat /dev/vg_nvme/lv_3 2024-03-08T20:49:52.091 INFO:teuthology.orchestra.run.smithi204.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-08T20:49:52.092 INFO:teuthology.orchestra.run.smithi204.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-08T20:49:52.092 INFO:teuthology.orchestra.run.smithi204.stdout:Device: 5h/5d Inode: 670 Links: 1 2024-03-08T20:49:52.092 INFO:teuthology.orchestra.run.smithi204.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-08T20:49:52.092 INFO:teuthology.orchestra.run.smithi204.stdout:Context: system_u:object_r:device_t:s0 2024-03-08T20:49:52.092 INFO:teuthology.orchestra.run.smithi204.stdout:Access: 2024-03-08 20:49:51.634698473 +0000 2024-03-08T20:49:52.092 INFO:teuthology.orchestra.run.smithi204.stdout:Modify: 2024-03-08 20:42:30.627607644 +0000 2024-03-08T20:49:52.092 INFO:teuthology.orchestra.run.smithi204.stdout:Change: 2024-03-08 20:42:30.627607644 +0000 2024-03-08T20:49:52.092 INFO:teuthology.orchestra.run.smithi204.stdout: Birth: 2024-03-08 20:42:30.627607644 +0000 2024-03-08T20:49:52.092 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-08T20:49:52.217 INFO:teuthology.orchestra.run.smithi204.stderr:1+0 records in 2024-03-08T20:49:52.217 INFO:teuthology.orchestra.run.smithi204.stderr:1+0 records out 2024-03-08T20:49:52.217 INFO:teuthology.orchestra.run.smithi204.stderr:512 bytes copied, 0.000318052 s, 1.6 MB/s 2024-03-08T20:49:52.220 DEBUG:teuthology.orchestra.run.smithi204:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-08T20:49:52.266 DEBUG:teuthology.orchestra.run.smithi204:> stat /dev/vg_nvme/lv_4 2024-03-08T20:49:52.358 INFO:teuthology.orchestra.run.smithi204.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-08T20:49:52.358 INFO:teuthology.orchestra.run.smithi204.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-08T20:49:52.359 INFO:teuthology.orchestra.run.smithi204.stdout:Device: 5h/5d Inode: 689 Links: 1 2024-03-08T20:49:52.359 INFO:teuthology.orchestra.run.smithi204.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-08T20:49:52.359 INFO:teuthology.orchestra.run.smithi204.stdout:Context: system_u:object_r:device_t:s0 2024-03-08T20:49:52.359 INFO:teuthology.orchestra.run.smithi204.stdout:Access: 2024-03-08 20:49:51.634698473 +0000 2024-03-08T20:49:52.359 INFO:teuthology.orchestra.run.smithi204.stdout:Modify: 2024-03-08 20:42:31.205599039 +0000 2024-03-08T20:49:52.359 INFO:teuthology.orchestra.run.smithi204.stdout:Change: 2024-03-08 20:42:31.205599039 +0000 2024-03-08T20:49:52.359 INFO:teuthology.orchestra.run.smithi204.stdout: Birth: 2024-03-08 20:42:31.205599039 +0000 2024-03-08T20:49:52.359 DEBUG:teuthology.orchestra.run.smithi204:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-08T20:49:52.367 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Mar 08 20:49:52 smithi204 podman[36801]: 2024-03-08 20:49:52.329466769 +0000 UTC m=+0.366160946 container start 6cff0bd156dc85b8535cf852375d41f3f744dbecbae8e925992a62e5e62ae8d9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-z, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=rishabh-2024mar8-7d63482, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-03-08T20:49:52.427 INFO:teuthology.orchestra.run.smithi204.stderr:1+0 records in 2024-03-08T20:49:52.427 INFO:teuthology.orchestra.run.smithi204.stderr:1+0 records out 2024-03-08T20:49:52.427 INFO:teuthology.orchestra.run.smithi204.stderr:512 bytes copied, 0.000283087 s, 1.8 MB/s 2024-03-08T20:49:52.431 DEBUG:teuthology.orchestra.run.smithi204:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-08T20:49:52.516 INFO:tasks.cephadm:Deploying osd.0 on smithi107 with /dev/vg_nvme/lv_4... 2024-03-08T20:49:52.516 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dc20f0f8-dd8c-11ee-95c6-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-03-08T20:49:52.636 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Mar 08 20:49:52 smithi204 ceph-mgr[36896]: -- 172.21.15.204:0/1708849694 <== mon.1 v2:172.21.15.204:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x557e4f34a5a0 con 0x557e4f31e800 2024-03-08T20:49:52.636 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Mar 08 20:49:52 smithi204 bash[36801]: 6cff0bd156dc85b8535cf852375d41f3f744dbecbae8e925992a62e5e62ae8d9 2024-03-08T20:49:52.636 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Mar 08 20:49:52 smithi204 systemd[1]: Started Ceph mgr.z for dc20f0f8-dd8c-11ee-95c6-87774f69a715. 2024-03-08T20:49:52.955 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/mon.a/config 2024-03-08T20:49:53.367 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:49:53 smithi107 ceph-mon[31381]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T20:49:53.368 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:49:53 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:49:53.368 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:49:53 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:49:53.368 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:49:53 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:49:53.368 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:49:53 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-08T20:49:53.368 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:49:53 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-08T20:49:53.368 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:49:53 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-08T20:49:53.368 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:49:53 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:49:53.386 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Mar 08 20:49:53 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-z[36870]: 2024-03-08T20:49:53.077+0000 7fef4c8a3200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-08T20:49:53.386 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Mar 08 20:49:53 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-z[36870]: 2024-03-08T20:49:53.193+0000 7fef4c8a3200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-08T20:49:53.386 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:53 smithi204 ceph-mon[34935]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T20:49:53.387 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:53 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:49:53.387 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:53 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:49:53.387 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:53 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:49:53.387 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:53 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-08T20:49:53.387 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:53 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-08T20:49:53.387 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:53 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-08T20:49:53.387 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:53 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:49:53.523 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:53 smithi149 ceph-mon[35042]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T20:49:53.523 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:53 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:49:53.523 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:53 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:49:53.523 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:53 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:49:53.523 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:53 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-08T20:49:53.523 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:53 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-08T20:49:53.523 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:53 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-08T20:49:53.524 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:53 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:49:54.235 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Mar 08 20:49:53 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-z[36870]: 2024-03-08T20:49:53.974+0000 7fef4c8a3200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-08T20:49:54.235 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Mar 08 20:49:54 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-z[36870]: 2024-03-08T20:49:54.091+0000 7fef4c8a3200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-08T20:49:54.402 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:49:54 smithi107 ceph-mon[31381]: Deploying daemon mgr.y on smithi149 2024-03-08T20:49:54.487 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Mar 08 20:49:54 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-z[36870]: 2024-03-08T20:49:54.267+0000 7fef4c8a3200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-08T20:49:54.487 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Mar 08 20:49:54 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-z[36870]: 2024-03-08T20:49:54.388+0000 7fef4c8a3200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-08T20:49:54.487 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:54 smithi204 ceph-mon[34935]: Deploying daemon mgr.y on smithi149 2024-03-08T20:49:54.523 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:54 smithi149 ceph-mon[35042]: Deploying daemon mgr.y on smithi149 2024-03-08T20:49:54.811 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Mar 08 20:49:54 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-z[36870]: 2024-03-08T20:49:54.495+0000 7fef4c8a3200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-08T20:49:55.062 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Mar 08 20:49:54 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-z[36870]: 2024-03-08T20:49:54.820+0000 7fef4c8a3200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-08T20:49:55.062 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Mar 08 20:49:54 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-z[36870]: 2024-03-08T20:49:54.959+0000 7fef4c8a3200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-08T20:49:55.386 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:55 smithi204 ceph-mon[34935]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T20:49:55.387 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Mar 08 20:49:55 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-z[36870]: 2024-03-08T20:49:55.070+0000 7fef4c8a3200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-08T20:49:55.387 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Mar 08 20:49:55 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-z[36870]: 2024-03-08T20:49:55.190+0000 7fef4c8a3200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-08T20:49:55.429 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:55 smithi149 ceph-mon[35042]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T20:49:55.534 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:49:55 smithi107 ceph-mon[31381]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T20:49:55.534 INFO:teuthology.orchestra.run.smithi107.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-03-08T20:49:55.534 INFO:teuthology.orchestra.run.smithi107.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-03-08T20:49:55.534 INFO:teuthology.orchestra.run.smithi107.stderr: stderr: 10+0 records in 2024-03-08T20:49:55.534 INFO:teuthology.orchestra.run.smithi107.stderr:10+0 records out 2024-03-08T20:49:55.534 INFO:teuthology.orchestra.run.smithi107.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0361879 s, 290 MB/s 2024-03-08T20:49:55.534 INFO:teuthology.orchestra.run.smithi107.stderr:--> Zapping successful for: 2024-03-08T20:49:55.765 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Mar 08 20:49:55 smithi149 systemd[1]: Starting Ceph mgr.y for dc20f0f8-dd8c-11ee-95c6-87774f69a715... 2024-03-08T20:49:55.886 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Mar 08 20:49:55 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-z[36870]: 2024-03-08T20:49:55.406+0000 7fef4c8a3200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-08T20:49:56.024 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Mar 08 20:49:55 smithi149 podman[36409]: 2024-03-08 20:49:55.678949521 +0000 UTC m=+0.031258401 image pull a272146cb810b1fd4a53627f019472132e30966d5c25b01bfc44a001295f1249 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd 2024-03-08T20:49:56.024 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Mar 08 20:49:55 smithi149 podman[36409]: 2024-03-08 20:49:55.810495501 +0000 UTC m=+0.162804418 container create 47118d6ce41b9f0ee58b1e2d5b80236fac84a34a388d680e12bad92fdb3c6fc6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-y, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=rishabh-2024mar8-7d63482, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=) 2024-03-08T20:49:56.356 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dc20f0f8-dd8c-11ee-95c6-87774f69a715 -- ceph orch daemon add osd smithi107:vg_nvme/lv_4 2024-03-08T20:49:56.523 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Mar 08 20:49:56 smithi149 podman[36409]: 2024-03-08 20:49:56.056024251 +0000 UTC m=+0.408333175 container init 47118d6ce41b9f0ee58b1e2d5b80236fac84a34a388d680e12bad92fdb3c6fc6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-y, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, RELEASE=rishabh-2024mar8-7d63482, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-08T20:49:56.523 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Mar 08 20:49:56 smithi149 podman[36409]: 2024-03-08 20:49:56.065678635 +0000 UTC m=+0.417987539 container start 47118d6ce41b9f0ee58b1e2d5b80236fac84a34a388d680e12bad92fdb3c6fc6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-y, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=rishabh-2024mar8-7d63482, CEPH_POINT_RELEASE=) 2024-03-08T20:49:56.523 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Mar 08 20:49:56 smithi149 ceph-mgr[36428]: -- 172.21.15.149:0/3170344089 <== mon.1 v2:172.21.15.204:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55fa03c885a0 con 0x55fa03c5c800 2024-03-08T20:49:56.523 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Mar 08 20:49:56 smithi149 bash[36409]: 47118d6ce41b9f0ee58b1e2d5b80236fac84a34a388d680e12bad92fdb3c6fc6 2024-03-08T20:49:56.523 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Mar 08 20:49:56 smithi149 systemd[1]: Started Ceph mgr.y for dc20f0f8-dd8c-11ee-95c6-87774f69a715. 2024-03-08T20:49:56.776 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/mon.a/config 2024-03-08T20:49:57.136 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Mar 08 20:49:56 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-z[36870]: 2024-03-08T20:49:56.699+0000 7fef4c8a3200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-08T20:49:57.136 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Mar 08 20:49:56 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-z[36870]: 2024-03-08T20:49:56.879+0000 7fef4c8a3200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-08T20:49:57.257 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Mar 08 20:49:56 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-y[36424]: 2024-03-08T20:49:56.820+0000 7f213fc40200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-08T20:49:57.257 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Mar 08 20:49:56 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-y[36424]: 2024-03-08T20:49:56.948+0000 7f213fc40200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-08T20:49:57.478 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:49:57 smithi107 ceph-mon[31381]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T20:49:57.478 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:49:57 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:49:57.478 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:49:57 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:49:57.478 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:49:57 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:49:57.479 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:49:57 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:49:57.479 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:49:57 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T20:49:57.523 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:57 smithi149 ceph-mon[35042]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T20:49:57.523 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:57 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:49:57.523 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:57 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:49:57.523 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:57 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:49:57.523 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:57 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:49:57.523 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:57 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T20:49:57.637 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Mar 08 20:49:57 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-z[36870]: 2024-03-08T20:49:57.309+0000 7fef4c8a3200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-08T20:49:57.637 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Mar 08 20:49:57 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-z[36870]: 2024-03-08T20:49:57.441+0000 7fef4c8a3200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-08T20:49:57.637 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:57 smithi204 ceph-mon[34935]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T20:49:57.637 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:57 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:49:57.637 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:57 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:49:57.637 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:57 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:49:57.638 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:57 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:49:57.638 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:57 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T20:49:57.963 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Mar 08 20:49:57 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-z[36870]: 2024-03-08T20:49:57.715+0000 7fef4c8a3200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-08T20:49:57.963 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Mar 08 20:49:57 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-z[36870]: 2024-03-08T20:49:57.971+0000 7fef4c8a3200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-08T20:49:58.089 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Mar 08 20:49:57 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-y[36424]: 2024-03-08T20:49:57.788+0000 7f213fc40200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-08T20:49:58.089 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Mar 08 20:49:57 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-y[36424]: 2024-03-08T20:49:57.913+0000 7f213fc40200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-08T20:49:58.089 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Mar 08 20:49:58 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-y[36424]: 2024-03-08T20:49:58.096+0000 7f213fc40200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-08T20:49:58.523 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Mar 08 20:49:58 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-y[36424]: 2024-03-08T20:49:58.222+0000 7f213fc40200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-08T20:49:58.523 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Mar 08 20:49:58 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-y[36424]: 2024-03-08T20:49:58.341+0000 7f213fc40200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-08T20:49:58.636 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Mar 08 20:49:58 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-z[36870]: 2024-03-08T20:49:58.354+0000 7fef4c8a3200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-08T20:49:58.752 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:49:58.759+0000 7f093ee00700 1 -- 172.21.15.107:0/1115076899 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0938101d50 msgr2=0x7f0938102170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:49:58.752 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:49:58.759+0000 7f093ee00700 1 --2- 172.21.15.107:0/1115076899 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0938101d50 0x7f0938102170 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f092c009990 tx=0x7f092c009ca0 comp rx=0 tx=0).stop 2024-03-08T20:49:58.752 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:49:58.759+0000 7f093ee00700 1 -- 172.21.15.107:0/1115076899 shutdown_connections 2024-03-08T20:49:58.752 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:49:58.759+0000 7f093ee00700 1 --2- 172.21.15.107:0/1115076899 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0938101d50 0x7f0938102170 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:49:58.752 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:49:58.759+0000 7f093ee00700 1 --2- 172.21.15.107:0/1115076899 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f09381038f0 0x7f093810a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:49:58.752 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:49:58.759+0000 7f093ee00700 1 --2- 172.21.15.107:0/1115076899 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0938102f50 0x7f09381033b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:49:58.753 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:49:58.759+0000 7f093ee00700 1 -- 172.21.15.107:0/1115076899 >> 172.21.15.107:0/1115076899 conn(0x7f09380fd2f0 msgr2=0x7f09380ff730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:49:58.753 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:49:58.759+0000 7f093ee00700 1 -- 172.21.15.107:0/1115076899 shutdown_connections 2024-03-08T20:49:58.753 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:49:58.759+0000 7f093ee00700 1 -- 172.21.15.107:0/1115076899 wait complete. 2024-03-08T20:49:58.755 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:49:58.760+0000 7f093ee00700 1 Processor -- start 2024-03-08T20:49:58.755 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:49:58.761+0000 7f093ee00700 1 -- start start 2024-03-08T20:49:58.755 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:49:58.761+0000 7f093ee00700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0938101d50 0x7f0938075b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:49:58.755 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:49:58.761+0000 7f093ee00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0938102f50 0x7f09380741d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:49:58.755 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:49:58.761+0000 7f093ee00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f09381038f0 0x7f0938074710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:49:58.756 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:49:58.761+0000 7f093ee00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f093810d890 con 0x7f0938102f50 2024-03-08T20:49:58.756 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:49:58.761+0000 7f093ee00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f093810d710 con 0x7f09381038f0 2024-03-08T20:49:58.756 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:49:58.761+0000 7f093ee00700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f093810da10 con 0x7f0938101d50 2024-03-08T20:49:58.756 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:49:58.761+0000 7f093d000700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0938102f50 0x7f09380741d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:49:58.756 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:49:58.761+0000 7f093d000700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0938102f50 0x7f09380741d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:43366/0 (socket says 172.21.15.107:43366) 2024-03-08T20:49:58.756 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:49:58.761+0000 7f093d000700 1 -- 172.21.15.107:0/2115443851 learned_addr learned my addr 172.21.15.107:0/2115443851 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T20:49:58.756 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:49:58.761+0000 7f093da00700 1 --2- 172.21.15.107:0/2115443851 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0938101d50 0x7f0938075b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:49:58.756 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:49:58.762+0000 7f093e400700 1 --2- 172.21.15.107:0/2115443851 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f09381038f0 0x7f0938074710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:49:58.756 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:49:58.762+0000 7f093d000700 1 -- 172.21.15.107:0/2115443851 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0938101d50 msgr2=0x7f0938075b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:49:58.756 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:49:58.762+0000 7f093d000700 1 --2- 172.21.15.107:0/2115443851 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0938101d50 0x7f0938075b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:49:58.756 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:49:58.762+0000 7f093d000700 1 -- 172.21.15.107:0/2115443851 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f09381038f0 msgr2=0x7f0938074710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:49:58.757 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:49:58.762+0000 7f093d000700 1 --2- 172.21.15.107:0/2115443851 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f09381038f0 0x7f0938074710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:49:58.757 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:49:58.762+0000 7f093d000700 1 -- 172.21.15.107:0/2115443851 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0938074c50 con 0x7f0938102f50 2024-03-08T20:49:58.757 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:49:58.762+0000 7f093e400700 1 --2- 172.21.15.107:0/2115443851 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f09381038f0 0x7f0938074710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T20:49:58.757 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:49:58.763+0000 7f093d000700 1 --2- 172.21.15.107:0/2115443851 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0938102f50 0x7f09380741d0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f092400b530 tx=0x7f092400b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:49:58.757 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:49:58.763+0000 7f0936000700 1 -- 172.21.15.107:0/2115443851 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0924015020 con 0x7f0938102f50 2024-03-08T20:49:58.757 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:49:58.763+0000 7f093ee00700 1 -- 172.21.15.107:0/2115443851 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0938074f40 con 0x7f0938102f50 2024-03-08T20:49:58.757 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:49:58.763+0000 7f0936000700 1 -- 172.21.15.107:0/2115443851 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f0924011ce0 con 0x7f0938102f50 2024-03-08T20:49:58.757 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:49:58.763+0000 7f093ee00700 1 -- 172.21.15.107:0/2115443851 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09381a6a10 con 0x7f0938102f50 2024-03-08T20:49:58.757 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:49:58.763+0000 7f0936000700 1 -- 172.21.15.107:0/2115443851 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0924010e20 con 0x7f0938102f50 2024-03-08T20:49:58.758 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:49:58.765+0000 7f0936000700 1 -- 172.21.15.107:0/2115443851 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 13) v1 ==== 46163+0+0 (secure 0 0 0) 0x7f0924018020 con 0x7f0938102f50 2024-03-08T20:49:58.758 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:49:58.765+0000 7f093ee00700 1 -- 172.21.15.107:0/2115443851 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0918005360 con 0x7f0938102f50 2024-03-08T20:49:58.759 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:49:58.765+0000 7f0936000700 1 --2- 172.21.15.107:0/2115443851 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f0920039200 0x7f092003b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:49:58.759 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:49:58.765+0000 7f0936000700 1 -- 172.21.15.107:0/2115443851 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f092404b880 con 0x7f0938102f50 2024-03-08T20:49:58.759 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:49:58.765+0000 7f093da00700 1 --2- 172.21.15.107:0/2115443851 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f0920039200 0x7f092003b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:49:58.759 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:49:58.766+0000 7f093da00700 1 --2- 172.21.15.107:0/2115443851 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f0920039200 0x7f092003b6c0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f092c006010 tx=0x7f092c0038e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:49:58.765 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:49:58.773+0000 7f0936000700 1 -- 172.21.15.107:0/2115443851 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f092401f020 con 0x7f0938102f50 2024-03-08T20:49:58.951 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Mar 08 20:49:58 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-y[36424]: 2024-03-08T20:49:58.689+0000 7f213fc40200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-08T20:49:58.951 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Mar 08 20:49:58 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-y[36424]: 2024-03-08T20:49:58.841+0000 7f213fc40200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-08T20:49:58.969 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Mar 08 20:49:58 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-z[36870]: 2024-03-08T20:49:58.674+0000 7fef4c8a3200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-08T20:49:58.996 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:49:59.004+0000 7f093ee00700 1 -- 172.21.15.107:0/2115443851 --> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f0918000bf0 con 0x7f0920039200 2024-03-08T20:49:59.273 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Mar 08 20:49:58 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-y[36424]: 2024-03-08T20:49:58.959+0000 7f213fc40200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-08T20:49:59.273 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Mar 08 20:49:59 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-y[36424]: 2024-03-08T20:49:59.098+0000 7f213fc40200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-08T20:49:59.327 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Mar 08 20:49:58 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-z[36870]: 2024-03-08T20:49:58.977+0000 7fef4c8a3200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-08T20:49:59.636 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:59 smithi204 ceph-mon[34935]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T20:49:59.637 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:59 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:49:59.637 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:59 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:49:59.637 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:59 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:49:59.637 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:59 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:49:59.637 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:59 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-08T20:49:59.637 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:59 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-08T20:49:59.637 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:49:59 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:49:59.725 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:49:59 smithi107 ceph-mon[31381]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T20:49:59.725 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:49:59 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:49:59.725 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:49:59 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:49:59.725 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:49:59 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:49:59.725 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:49:59 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:49:59.725 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:49:59 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-08T20:49:59.725 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:49:59 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-08T20:49:59.726 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:49:59 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:49:59.773 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Mar 08 20:49:59 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-y[36424]: 2024-03-08T20:49:59.329+0000 7f213fc40200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-08T20:49:59.773 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:59 smithi149 ceph-mon[35042]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T20:49:59.773 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:59 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:49:59.773 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:59 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:49:59.773 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:59 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:49:59.773 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:59 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:49:59.773 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:59 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-08T20:49:59.773 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:59 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-08T20:49:59.774 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:49:59 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:50:00.331 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Mar 08 20:50:00 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-z[36870]: 2024-03-08T20:49:59.999+0000 7fef4c8a3200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-08T20:50:00.475 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:50:00 smithi107 ceph-mon[31381]: from='client.14220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-08T20:50:00.476 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:50:00 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:50:00.476 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:50:00 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T20:50:00.476 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:50:00 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:50:00.476 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:50:00 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-08T20:50:00.476 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:50:00 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-08T20:50:00.476 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:50:00 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:50:00.476 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:50:00 smithi107 ceph-mon[31381]: overall HEALTH_OK 2024-03-08T20:50:00.476 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:50:00 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:50:00.636 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:50:00 smithi204 ceph-mon[34935]: from='client.14220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-08T20:50:00.636 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:50:00 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:50:00.636 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:50:00 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T20:50:00.636 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:50:00 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:50:00.637 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:50:00 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-08T20:50:00.637 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:50:00 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-08T20:50:00.637 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:50:00 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:50:00.637 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:50:00 smithi204 ceph-mon[34935]: overall HEALTH_OK 2024-03-08T20:50:00.637 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:50:00 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:50:00.642 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:50:00 smithi149 ceph-mon[35042]: from='client.14220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-08T20:50:00.642 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:50:00 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:50:00.642 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:50:00 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T20:50:00.642 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:50:00 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:50:00.642 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:50:00 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-08T20:50:00.642 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:50:00 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-08T20:50:00.642 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:50:00 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:50:00.643 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:50:00 smithi149 ceph-mon[35042]: overall HEALTH_OK 2024-03-08T20:50:00.643 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:50:00 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:50:01.023 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Mar 08 20:50:00 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-y[36424]: 2024-03-08T20:50:00.649+0000 7f213fc40200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-08T20:50:01.023 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Mar 08 20:50:00 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-y[36424]: 2024-03-08T20:50:00.817+0000 7f213fc40200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-08T20:50:01.096 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Mar 08 20:50:00 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-z[36870]: 2024-03-08T20:50:00.696+0000 7fef4c8a3200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-08T20:50:01.096 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Mar 08 20:50:00 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-z[36870]: 2024-03-08T20:50:00.823+0000 7fef4c8a3200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-08T20:50:01.386 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Mar 08 20:50:01 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-z[36870]: 2024-03-08T20:50:01.104+0000 7fef4c8a3200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-08T20:50:01.387 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:50:01 smithi204 ceph-mon[34935]: Reconfiguring mgr.x (unknown last config time)... 2024-03-08T20:50:01.387 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:50:01 smithi204 ceph-mon[34935]: Reconfiguring daemon mgr.x on smithi107 2024-03-08T20:50:01.387 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:50:01 smithi204 ceph-mon[34935]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T20:50:01.387 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:50:01 smithi204 ceph-mon[34935]: Standby manager daemon z started 2024-03-08T20:50:01.387 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:50:01 smithi204 ceph-mon[34935]: from='mgr.? 172.21.15.204:0/2328134280' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-03-08T20:50:01.387 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:50:01 smithi204 ceph-mon[34935]: from='mgr.? 172.21.15.204:0/2328134280' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-08T20:50:01.387 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:50:01 smithi204 ceph-mon[34935]: from='mgr.? 172.21.15.204:0/2328134280' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-03-08T20:50:01.387 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:50:01 smithi204 ceph-mon[34935]: from='mgr.? 172.21.15.204:0/2328134280' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-08T20:50:01.523 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Mar 08 20:50:01 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-y[36424]: 2024-03-08T20:50:01.237+0000 7f213fc40200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-08T20:50:01.523 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Mar 08 20:50:01 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-y[36424]: 2024-03-08T20:50:01.356+0000 7f213fc40200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-08T20:50:01.523 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:50:01 smithi149 ceph-mon[35042]: Reconfiguring mgr.x (unknown last config time)... 2024-03-08T20:50:01.523 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:50:01 smithi149 ceph-mon[35042]: Reconfiguring daemon mgr.x on smithi107 2024-03-08T20:50:01.523 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:50:01 smithi149 ceph-mon[35042]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T20:50:01.523 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:50:01 smithi149 ceph-mon[35042]: Standby manager daemon z started 2024-03-08T20:50:01.523 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:50:01 smithi149 ceph-mon[35042]: from='mgr.? 172.21.15.204:0/2328134280' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-03-08T20:50:01.524 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:50:01 smithi149 ceph-mon[35042]: from='mgr.? 172.21.15.204:0/2328134280' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-08T20:50:01.524 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:50:01 smithi149 ceph-mon[35042]: from='mgr.? 172.21.15.204:0/2328134280' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-03-08T20:50:01.524 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:50:01 smithi149 ceph-mon[35042]: from='mgr.? 172.21.15.204:0/2328134280' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-08T20:50:01.538 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:50:01.547+0000 7f0936000700 1 -- 172.21.15.107:0/2115443851 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mgrmap(e 14) v1 ==== 91699+0+0 (secure 0 0 0) 0x7f092401ed10 con 0x7f0938102f50 2024-03-08T20:50:01.539 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:50:01 smithi107 ceph-mon[31381]: Reconfiguring mgr.x (unknown last config time)... 2024-03-08T20:50:01.539 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:50:01 smithi107 ceph-mon[31381]: Reconfiguring daemon mgr.x on smithi107 2024-03-08T20:50:01.539 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:50:01 smithi107 ceph-mon[31381]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T20:50:01.540 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:50:01 smithi107 ceph-mon[31381]: Standby manager daemon z started 2024-03-08T20:50:01.540 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:50:01 smithi107 ceph-mon[31381]: from='mgr.? 172.21.15.204:0/2328134280' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-03-08T20:50:01.540 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:50:01 smithi107 ceph-mon[31381]: from='mgr.? 172.21.15.204:0/2328134280' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-08T20:50:01.540 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:50:01 smithi107 ceph-mon[31381]: from='mgr.? 172.21.15.204:0/2328134280' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-03-08T20:50:01.540 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:50:01 smithi107 ceph-mon[31381]: from='mgr.? 172.21.15.204:0/2328134280' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-08T20:50:02.023 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Mar 08 20:50:01 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-y[36424]: 2024-03-08T20:50:01.611+0000 7f213fc40200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-08T20:50:02.023 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Mar 08 20:50:01 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-y[36424]: 2024-03-08T20:50:01.853+0000 7f213fc40200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-08T20:50:02.523 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Mar 08 20:50:02 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-y[36424]: 2024-03-08T20:50:02.225+0000 7f213fc40200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-08T20:50:02.818 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:50:02 smithi107 ceph-mon[31381]: mgrmap e14: x(active, since 101s), standbys: z 2024-03-08T20:50:02.818 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:50:02 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-03-08T20:50:02.818 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:50:02 smithi107 ceph-mon[31381]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T20:50:02.818 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:50:02 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:50:02.818 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:50:02 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:50:02.818 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:50:02 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T20:50:02.818 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:50:02 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:50:02.818 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:50:02 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T20:50:02.818 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:50:02 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:50:02.850 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Mar 08 20:50:02 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-y[36424]: 2024-03-08T20:50:02.553+0000 7f213fc40200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-08T20:50:02.850 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:50:02 smithi149 ceph-mon[35042]: mgrmap e14: x(active, since 101s), standbys: z 2024-03-08T20:50:02.850 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:50:02 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-03-08T20:50:02.851 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:50:02 smithi149 ceph-mon[35042]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T20:50:02.851 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:50:02 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:50:02.851 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:50:02 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:50:02.851 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:50:02 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T20:50:02.851 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:50:02 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:50:02.851 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:50:02 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T20:50:02.851 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:50:02 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:50:02.886 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:50:02 smithi204 ceph-mon[34935]: mgrmap e14: x(active, since 101s), standbys: z 2024-03-08T20:50:02.886 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:50:02 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-03-08T20:50:02.886 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:50:02 smithi204 ceph-mon[34935]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T20:50:02.886 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:50:02 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:50:02.887 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:50:02 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:50:02.887 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:50:02 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T20:50:02.887 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:50:02 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:50:02.887 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:50:02 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T20:50:02.887 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:50:02 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:50:03.273 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Mar 08 20:50:02 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-y[36424]: 2024-03-08T20:50:02.857+0000 7f213fc40200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-08T20:50:04.163 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Mar 08 20:50:03 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-y[36424]: 2024-03-08T20:50:03.872+0000 7f213fc40200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-08T20:50:04.440 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:50:04 smithi204 ceph-mon[34935]: from='client.? 172.21.15.107:0/479991633' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8b1ddc64-4d20-4e85-ac66-f1c5ac599501"}]: dispatch 2024-03-08T20:50:04.440 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:50:04 smithi204 ceph-mon[34935]: from='client.? 172.21.15.107:0/479991633' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8b1ddc64-4d20-4e85-ac66-f1c5ac599501"}]': finished 2024-03-08T20:50:04.440 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:50:04 smithi204 ceph-mon[34935]: osdmap e5: 1 total, 0 up, 1 in 2024-03-08T20:50:04.440 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:50:04 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-08T20:50:04.475 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:50:04 smithi107 ceph-mon[31381]: from='client.? 172.21.15.107:0/479991633' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8b1ddc64-4d20-4e85-ac66-f1c5ac599501"}]: dispatch 2024-03-08T20:50:04.475 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:50:04 smithi107 ceph-mon[31381]: from='client.? 172.21.15.107:0/479991633' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8b1ddc64-4d20-4e85-ac66-f1c5ac599501"}]': finished 2024-03-08T20:50:04.475 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:50:04 smithi107 ceph-mon[31381]: osdmap e5: 1 total, 0 up, 1 in 2024-03-08T20:50:04.476 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:50:04 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-08T20:50:04.523 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:50:04 smithi149 ceph-mon[35042]: from='client.? 172.21.15.107:0/479991633' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8b1ddc64-4d20-4e85-ac66-f1c5ac599501"}]: dispatch 2024-03-08T20:50:04.523 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:50:04 smithi149 ceph-mon[35042]: from='client.? 172.21.15.107:0/479991633' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8b1ddc64-4d20-4e85-ac66-f1c5ac599501"}]': finished 2024-03-08T20:50:04.523 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:50:04 smithi149 ceph-mon[35042]: osdmap e5: 1 total, 0 up, 1 in 2024-03-08T20:50:04.523 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:50:04 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-08T20:50:04.972 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Mar 08 20:50:04 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-y[36424]: 2024-03-08T20:50:04.574+0000 7f213fc40200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-08T20:50:04.973 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Mar 08 20:50:04 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-y[36424]: 2024-03-08T20:50:04.701+0000 7f213fc40200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-08T20:50:05.182 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:50:05.191+0000 7f0936000700 1 -- 172.21.15.107:0/2115443851 <== mon.0 v2:172.21.15.107:3300/0 8 ==== mgrmap(e 15) v1 ==== 137235+0+0 (secure 0 0 0) 0x7f09240193b0 con 0x7f0938102f50 2024-03-08T20:50:05.273 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Mar 08 20:50:04 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-y[36424]: 2024-03-08T20:50:04.980+0000 7f213fc40200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-08T20:50:05.273 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:50:05 smithi149 ceph-mon[35042]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T20:50:05.273 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:50:05 smithi149 ceph-mon[35042]: Standby manager daemon y started 2024-03-08T20:50:05.274 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:50:05 smithi149 ceph-mon[35042]: from='mgr.? 172.21.15.149:0/3467881093' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-08T20:50:05.274 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:50:05 smithi149 ceph-mon[35042]: from='mgr.? 172.21.15.149:0/3467881093' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-08T20:50:05.274 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:50:05 smithi149 ceph-mon[35042]: from='client.? 172.21.15.107:0/2106910087' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-08T20:50:05.274 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:50:05 smithi149 ceph-mon[35042]: from='mgr.? 172.21.15.149:0/3467881093' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-08T20:50:05.274 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:50:05 smithi149 ceph-mon[35042]: from='mgr.? 172.21.15.149:0/3467881093' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-08T20:50:05.475 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:50:05 smithi107 ceph-mon[31381]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T20:50:05.475 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:50:05 smithi107 ceph-mon[31381]: Standby manager daemon y started 2024-03-08T20:50:05.476 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:50:05 smithi107 ceph-mon[31381]: from='mgr.? 172.21.15.149:0/3467881093' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-08T20:50:05.476 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:50:05 smithi107 ceph-mon[31381]: from='mgr.? 172.21.15.149:0/3467881093' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-08T20:50:05.476 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:50:05 smithi107 ceph-mon[31381]: from='client.? 172.21.15.107:0/2106910087' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-08T20:50:05.476 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:50:05 smithi107 ceph-mon[31381]: from='mgr.? 172.21.15.149:0/3467881093' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-08T20:50:05.476 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:50:05 smithi107 ceph-mon[31381]: from='mgr.? 172.21.15.149:0/3467881093' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-08T20:50:05.636 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:50:05 smithi204 ceph-mon[34935]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T20:50:05.636 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:50:05 smithi204 ceph-mon[34935]: Standby manager daemon y started 2024-03-08T20:50:05.636 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:50:05 smithi204 ceph-mon[34935]: from='mgr.? 172.21.15.149:0/3467881093' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-08T20:50:05.636 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:50:05 smithi204 ceph-mon[34935]: from='mgr.? 172.21.15.149:0/3467881093' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-08T20:50:05.637 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:50:05 smithi204 ceph-mon[34935]: from='client.? 172.21.15.107:0/2106910087' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-08T20:50:05.637 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:50:05 smithi204 ceph-mon[34935]: from='mgr.? 172.21.15.149:0/3467881093' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-08T20:50:05.637 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:50:05 smithi204 ceph-mon[34935]: from='mgr.? 172.21.15.149:0/3467881093' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-08T20:50:06.475 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:50:06 smithi107 ceph-mon[31381]: mgrmap e15: x(active, since 105s), standbys: z, y 2024-03-08T20:50:06.475 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:50:06 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-08T20:50:06.523 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:50:06 smithi149 ceph-mon[35042]: mgrmap e15: x(active, since 105s), standbys: z, y 2024-03-08T20:50:06.523 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:50:06 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-08T20:50:06.636 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:50:06 smithi204 ceph-mon[34935]: mgrmap e15: x(active, since 105s), standbys: z, y 2024-03-08T20:50:06.636 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:50:06 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-08T20:50:07.475 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:50:07 smithi107 ceph-mon[31381]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T20:50:07.523 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:50:07 smithi149 ceph-mon[35042]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T20:50:07.636 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:50:07 smithi204 ceph-mon[34935]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T20:50:09.475 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:50:09 smithi107 ceph-mon[31381]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T20:50:09.523 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:50:09 smithi149 ceph-mon[35042]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T20:50:09.636 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:50:09 smithi204 ceph-mon[34935]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T20:50:11.475 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:50:11 smithi107 ceph-mon[31381]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T20:50:11.523 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:50:11 smithi149 ceph-mon[35042]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T20:50:11.636 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:50:11 smithi204 ceph-mon[34935]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T20:50:13.475 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:50:13 smithi107 ceph-mon[31381]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T20:50:13.475 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:50:13 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-08T20:50:13.475 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:50:13 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:50:13.523 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:50:13 smithi149 ceph-mon[35042]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T20:50:13.523 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:50:13 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-08T20:50:13.523 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:50:13 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:50:13.636 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:50:13 smithi204 ceph-mon[34935]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T20:50:13.636 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:50:13 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-08T20:50:13.636 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:50:13 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:50:14.434 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:50:14 smithi107 ceph-mon[31381]: Deploying daemon osd.0 on smithi107 2024-03-08T20:50:14.522 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:50:14 smithi149 ceph-mon[35042]: Deploying daemon osd.0 on smithi107 2024-03-08T20:50:14.533 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:50:14 smithi204 ceph-mon[34935]: Deploying daemon osd.0 on smithi107 2024-03-08T20:50:15.476 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:50:15 smithi107 ceph-mon[31381]: pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T20:50:15.522 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:50:15 smithi149 ceph-mon[35042]: pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T20:50:15.635 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:50:15 smithi204 ceph-mon[34935]: pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T20:50:17.475 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:50:17 smithi107 ceph-mon[31381]: pgmap v53: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T20:50:17.522 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:50:17 smithi149 ceph-mon[35042]: pgmap v53: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T20:50:17.636 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:50:17 smithi204 ceph-mon[34935]: pgmap v53: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T20:50:19.522 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:50:19 smithi149 ceph-mon[35042]: pgmap v54: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T20:50:19.636 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:50:19 smithi204 ceph-mon[34935]: pgmap v54: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T20:50:19.726 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:50:19 smithi107 ceph-mon[31381]: pgmap v54: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T20:50:21.273 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:50:21 smithi149 ceph-mon[35042]: pgmap v55: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T20:50:21.273 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:50:21 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:50:21.273 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:50:21 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:50:21.273 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:50:21 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T20:50:21.636 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:50:21 smithi204 ceph-mon[34935]: pgmap v55: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T20:50:21.636 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:50:21 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:50:21.636 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:50:21 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:50:21.636 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:50:21 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T20:50:21.725 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:50:21 smithi107 ceph-mon[31381]: pgmap v55: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T20:50:21.725 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:50:21 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:50:21.725 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:50:21 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:50:21.725 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:50:21 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T20:50:23.515 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:50:23 smithi107 ceph-mon[31381]: pgmap v56: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T20:50:23.522 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:50:23 smithi149 ceph-mon[35042]: pgmap v56: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T20:50:23.635 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:50:23 smithi204 ceph-mon[34935]: pgmap v56: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T20:50:25.583 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:50:25 smithi107 ceph-mon[31381]: pgmap v57: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T20:50:25.583 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:50:25 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:50:25.583 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:50:25 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:50:25.583 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:50:25 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:50:25.583 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:50:25 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T20:50:25.583 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:50:25 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:50:25.583 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:50:25 smithi107 ceph-mon[31381]: from='osd.0 [v2:172.21.15.107:6802/2453946095,v1:172.21.15.107:6803/2453946095]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-08T20:50:25.636 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:50:25 smithi204 ceph-mon[34935]: pgmap v57: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T20:50:25.636 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:50:25 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:50:25.636 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:50:25 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:50:25.636 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:50:25 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:50:25.636 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:50:25 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T20:50:25.636 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:50:25 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:50:25.636 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:50:25 smithi204 ceph-mon[34935]: from='osd.0 [v2:172.21.15.107:6802/2453946095,v1:172.21.15.107:6803/2453946095]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-08T20:50:25.695 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:50:25.705+0000 7f0936000700 1 -- 172.21.15.107:0/2115443851 <== mgr.14152 v2:172.21.15.107:6800/3376636618 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f0918000bf0 con 0x7f0920039200 2024-03-08T20:50:25.696 INFO:teuthology.orchestra.run.smithi107.stdout:Created osd(s) 0 on host 'smithi107' 2024-03-08T20:50:25.699 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:50:25.708+0000 7f093ee00700 1 -- 172.21.15.107:0/2115443851 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f0920039200 msgr2=0x7f092003b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:50:25.699 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:50:25.708+0000 7f093ee00700 1 --2- 172.21.15.107:0/2115443851 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f0920039200 0x7f092003b6c0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f092c006010 tx=0x7f092c0038e0 comp rx=0 tx=0).stop 2024-03-08T20:50:25.699 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:50:25.709+0000 7f093ee00700 1 -- 172.21.15.107:0/2115443851 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0938102f50 msgr2=0x7f09380741d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:50:25.699 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:50:25.709+0000 7f093ee00700 1 --2- 172.21.15.107:0/2115443851 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0938102f50 0x7f09380741d0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f092400b530 tx=0x7f092400b840 comp rx=0 tx=0).stop 2024-03-08T20:50:25.700 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:50:25.709+0000 7f093ee00700 1 -- 172.21.15.107:0/2115443851 shutdown_connections 2024-03-08T20:50:25.700 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:50:25.709+0000 7f093ee00700 1 --2- 172.21.15.107:0/2115443851 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0938101d50 0x7f0938075b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:50:25.700 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:50:25.709+0000 7f093ee00700 1 --2- 172.21.15.107:0/2115443851 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f0920039200 0x7f092003b6c0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:50:25.700 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:50:25.709+0000 7f093ee00700 1 --2- 172.21.15.107:0/2115443851 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f09381038f0 0x7f0938074710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:50:25.700 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:50:25.709+0000 7f093ee00700 1 --2- 172.21.15.107:0/2115443851 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0938102f50 0x7f09380741d0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:50:25.700 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:50:25.709+0000 7f093ee00700 1 -- 172.21.15.107:0/2115443851 >> 172.21.15.107:0/2115443851 conn(0x7f09380fd2f0 msgr2=0x7f09381062d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:50:25.700 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:50:25.709+0000 7f093ee00700 1 -- 172.21.15.107:0/2115443851 shutdown_connections 2024-03-08T20:50:25.700 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:50:25.709+0000 7f093ee00700 1 -- 172.21.15.107:0/2115443851 wait complete. 2024-03-08T20:50:25.772 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:50:25 smithi149 ceph-mon[35042]: pgmap v57: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T20:50:25.772 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:50:25 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:50:25.772 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:50:25 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:50:25.773 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:50:25 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:50:25.773 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:50:25 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T20:50:25.773 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:50:25 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:50:25.773 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:50:25 smithi149 ceph-mon[35042]: from='osd.0 [v2:172.21.15.107:6802/2453946095,v1:172.21.15.107:6803/2453946095]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-08T20:50:26.430 DEBUG:teuthology.orchestra.run.smithi107:osd.0> sudo journalctl -f -n 0 -u ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715@osd.0.service 2024-03-08T20:50:26.435 INFO:tasks.cephadm:Deploying osd.1 on smithi149 with /dev/vg_nvme/lv_4... 2024-03-08T20:50:26.435 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dc20f0f8-dd8c-11ee-95c6-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-03-08T20:50:26.675 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:50:26 smithi107 ceph-mon[31381]: from='osd.0 [v2:172.21.15.107:6802/2453946095,v1:172.21.15.107:6803/2453946095]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-08T20:50:26.675 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:50:26 smithi107 ceph-mon[31381]: osdmap e6: 1 total, 0 up, 1 in 2024-03-08T20:50:26.675 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:50:26 smithi107 ceph-mon[31381]: from='osd.0 [v2:172.21.15.107:6802/2453946095,v1:172.21.15.107:6803/2453946095]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-03-08T20:50:26.675 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:50:26 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-08T20:50:26.675 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:50:26 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:50:26.675 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:50:26 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T20:50:26.675 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:50:26 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:50:26.774 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:50:26 smithi149 ceph-mon[35042]: from='osd.0 [v2:172.21.15.107:6802/2453946095,v1:172.21.15.107:6803/2453946095]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-08T20:50:26.774 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:50:26 smithi149 ceph-mon[35042]: osdmap e6: 1 total, 0 up, 1 in 2024-03-08T20:50:26.774 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:50:26 smithi149 ceph-mon[35042]: from='osd.0 [v2:172.21.15.107:6802/2453946095,v1:172.21.15.107:6803/2453946095]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-03-08T20:50:26.774 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:50:26 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-08T20:50:26.775 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:50:26 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:50:26.775 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:50:26 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T20:50:26.775 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:50:26 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:50:26.871 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/mon.b/config 2024-03-08T20:50:26.886 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:50:26 smithi204 ceph-mon[34935]: from='osd.0 [v2:172.21.15.107:6802/2453946095,v1:172.21.15.107:6803/2453946095]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-08T20:50:26.886 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:50:26 smithi204 ceph-mon[34935]: osdmap e6: 1 total, 0 up, 1 in 2024-03-08T20:50:26.886 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:50:26 smithi204 ceph-mon[34935]: from='osd.0 [v2:172.21.15.107:6802/2453946095,v1:172.21.15.107:6803/2453946095]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-03-08T20:50:26.886 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:50:26 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-08T20:50:26.886 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:50:26 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:50:26.886 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:50:26 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T20:50:26.886 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:50:26 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:50:27.226 INFO:journalctl@ceph.osd.0.smithi107.stdout:Mar 08 20:50:26 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-0[46080]: 2024-03-08T20:50:26.994+0000 7f34dde00700 -1 osd.0 0 waiting for initial osdmap 2024-03-08T20:50:27.226 INFO:journalctl@ceph.osd.0.smithi107.stdout:Mar 08 20:50:27 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-0[46080]: 2024-03-08T20:50:27.009+0000 7f34d3c00700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-08T20:50:27.712 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:50:27 smithi107 ceph-mon[31381]: pgmap v59: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T20:50:27.712 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:50:27 smithi107 ceph-mon[31381]: from='osd.0 [v2:172.21.15.107:6802/2453946095,v1:172.21.15.107:6803/2453946095]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]': finished 2024-03-08T20:50:27.713 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:50:27 smithi107 ceph-mon[31381]: osdmap e7: 1 total, 0 up, 1 in 2024-03-08T20:50:27.713 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:50:27 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-08T20:50:27.713 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:50:27 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-08T20:50:27.713 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:50:27 smithi107 ceph-mon[31381]: from='osd.0 [v2:172.21.15.107:6802/2453946095,v1:172.21.15.107:6803/2453946095]' entity='osd.0' 2024-03-08T20:50:27.748 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:50:27 smithi149 ceph-mon[35042]: pgmap v59: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T20:50:27.748 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:50:27 smithi149 ceph-mon[35042]: from='osd.0 [v2:172.21.15.107:6802/2453946095,v1:172.21.15.107:6803/2453946095]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]': finished 2024-03-08T20:50:27.748 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:50:27 smithi149 ceph-mon[35042]: osdmap e7: 1 total, 0 up, 1 in 2024-03-08T20:50:27.748 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:50:27 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-08T20:50:27.748 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:50:27 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-08T20:50:27.748 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:50:27 smithi149 ceph-mon[35042]: from='osd.0 [v2:172.21.15.107:6802/2453946095,v1:172.21.15.107:6803/2453946095]' entity='osd.0' 2024-03-08T20:50:27.886 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:50:27 smithi204 ceph-mon[34935]: pgmap v59: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T20:50:27.886 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:50:27 smithi204 ceph-mon[34935]: from='osd.0 [v2:172.21.15.107:6802/2453946095,v1:172.21.15.107:6803/2453946095]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]': finished 2024-03-08T20:50:27.886 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:50:27 smithi204 ceph-mon[34935]: osdmap e7: 1 total, 0 up, 1 in 2024-03-08T20:50:27.886 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:50:27 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-08T20:50:27.886 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:50:27 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-08T20:50:27.886 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:50:27 smithi204 ceph-mon[34935]: from='osd.0 [v2:172.21.15.107:6802/2453946095,v1:172.21.15.107:6803/2453946095]' entity='osd.0' 2024-03-08T20:50:28.672 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:50:28 smithi149 ceph-mon[35042]: purged_snaps scrub starts 2024-03-08T20:50:28.673 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:50:28 smithi149 ceph-mon[35042]: purged_snaps scrub ok 2024-03-08T20:50:28.673 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:50:28 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-08T20:50:28.673 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:50:28 smithi149 ceph-mon[35042]: osd.0 [v2:172.21.15.107:6802/2453946095,v1:172.21.15.107:6803/2453946095] boot 2024-03-08T20:50:28.673 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:50:28 smithi149 ceph-mon[35042]: osdmap e8: 1 total, 1 up, 1 in 2024-03-08T20:50:28.673 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:50:28 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-08T20:50:28.724 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:50:28 smithi107 ceph-mon[31381]: purged_snaps scrub starts 2024-03-08T20:50:28.725 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:50:28 smithi107 ceph-mon[31381]: purged_snaps scrub ok 2024-03-08T20:50:28.725 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:50:28 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-08T20:50:28.725 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:50:28 smithi107 ceph-mon[31381]: osd.0 [v2:172.21.15.107:6802/2453946095,v1:172.21.15.107:6803/2453946095] boot 2024-03-08T20:50:28.725 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:50:28 smithi107 ceph-mon[31381]: osdmap e8: 1 total, 1 up, 1 in 2024-03-08T20:50:28.725 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:50:28 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-08T20:50:28.886 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:50:28 smithi204 ceph-mon[34935]: purged_snaps scrub starts 2024-03-08T20:50:28.886 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:50:28 smithi204 ceph-mon[34935]: purged_snaps scrub ok 2024-03-08T20:50:28.886 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:50:28 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-08T20:50:28.886 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:50:28 smithi204 ceph-mon[34935]: osd.0 [v2:172.21.15.107:6802/2453946095,v1:172.21.15.107:6803/2453946095] boot 2024-03-08T20:50:28.886 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:50:28 smithi204 ceph-mon[34935]: osdmap e8: 1 total, 1 up, 1 in 2024-03-08T20:50:28.886 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:50:28 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-08T20:50:29.552 INFO:teuthology.orchestra.run.smithi149.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-03-08T20:50:29.552 INFO:teuthology.orchestra.run.smithi149.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-03-08T20:50:29.552 INFO:teuthology.orchestra.run.smithi149.stderr: stderr: 10+0 records in 2024-03-08T20:50:29.552 INFO:teuthology.orchestra.run.smithi149.stderr:10+0 records out 2024-03-08T20:50:29.552 INFO:teuthology.orchestra.run.smithi149.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0366026 s, 286 MB/s 2024-03-08T20:50:29.552 INFO:teuthology.orchestra.run.smithi149.stderr:--> Zapping successful for: 2024-03-08T20:50:29.725 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:50:29 smithi107 ceph-mon[31381]: pgmap v62: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T20:50:29.772 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:50:29 smithi149 ceph-mon[35042]: pgmap v62: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T20:50:29.885 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:50:29 smithi204 ceph-mon[34935]: pgmap v62: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-08T20:50:30.295 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dc20f0f8-dd8c-11ee-95c6-87774f69a715 -- ceph orch daemon add osd smithi149:vg_nvme/lv_4 2024-03-08T20:50:30.522 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:50:30 smithi149 ceph-mon[35042]: osdmap e9: 1 total, 1 up, 1 in 2024-03-08T20:50:30.718 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/mon.b/config 2024-03-08T20:50:30.724 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:50:30 smithi107 ceph-mon[31381]: osdmap e9: 1 total, 1 up, 1 in 2024-03-08T20:50:30.885 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:50:30 smithi204 ceph-mon[34935]: osdmap e9: 1 total, 1 up, 1 in 2024-03-08T20:50:31.725 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:50:31 smithi107 ceph-mon[31381]: pgmap v64: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-08T20:50:31.725 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:50:31 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:50:31.725 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:50:31 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:50:31.725 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:50:31 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-08T20:50:31.725 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:50:31 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:50:31.725 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:50:31 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:50:31.725 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:50:31 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T20:50:31.725 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:50:31 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:50:31.773 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:50:31 smithi149 ceph-mon[35042]: pgmap v64: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-08T20:50:31.773 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:50:31 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:50:31.773 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:50:31 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:50:31.773 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:50:31 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-08T20:50:31.773 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:50:31 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:50:31.773 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:50:31 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:50:31.773 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:50:31 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T20:50:31.773 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:50:31 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:50:31.886 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:50:31 smithi204 ceph-mon[34935]: pgmap v64: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-08T20:50:31.886 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:50:31 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:50:31.886 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:50:31 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:50:31.886 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:50:31 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-08T20:50:31.886 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:50:31 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:50:31.886 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:50:31 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:50:31.886 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:50:31 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T20:50:31.886 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:50:31 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:50:32.661 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:50:32.670+0000 7fbd13a00700 1 -- 172.21.15.149:0/2628734516 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbd0c109660 msgr2=0x7fbd0c109ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:50:32.661 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:50:32.670+0000 7fbd13a00700 1 --2- 172.21.15.149:0/2628734516 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbd0c109660 0x7fbd0c109ac0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fbcf80099c0 tx=0x7fbcf8009cd0 comp rx=0 tx=0).stop 2024-03-08T20:50:32.661 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:50:32.670+0000 7fbd13a00700 1 -- 172.21.15.149:0/2628734516 shutdown_connections 2024-03-08T20:50:32.662 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:50:32.670+0000 7fbd13a00700 1 --2- 172.21.15.149:0/2628734516 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbd0c108460 0x7fbd0c108880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:50:32.662 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:50:32.670+0000 7fbd13a00700 1 --2- 172.21.15.149:0/2628734516 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbd0c10a000 0x7fbd0c1108b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:50:32.662 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:50:32.670+0000 7fbd13a00700 1 --2- 172.21.15.149:0/2628734516 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbd0c109660 0x7fbd0c109ac0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:50:32.662 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:50:32.670+0000 7fbd13a00700 1 -- 172.21.15.149:0/2628734516 >> 172.21.15.149:0/2628734516 conn(0x7fbd0c077430 msgr2=0x7fbd0c079870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:50:32.662 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:50:32.670+0000 7fbd13a00700 1 -- 172.21.15.149:0/2628734516 shutdown_connections 2024-03-08T20:50:32.662 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:50:32.670+0000 7fbd13a00700 1 -- 172.21.15.149:0/2628734516 wait complete. 2024-03-08T20:50:32.662 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:50:32.671+0000 7fbd13a00700 1 Processor -- start 2024-03-08T20:50:32.662 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:50:32.671+0000 7fbd13a00700 1 -- start start 2024-03-08T20:50:32.664 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:50:32.672+0000 7fbd13a00700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbd0c108460 0x7fbd0c19e550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:50:32.664 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:50:32.672+0000 7fbd13a00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbd0c109660 0x7fbd0c19ea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:50:32.664 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:50:32.672+0000 7fbd13a00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbd0c10a000 0x7fbd0c1a5b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:50:32.664 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:50:32.672+0000 7fbd13a00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd0c113f00 con 0x7fbd0c10a000 2024-03-08T20:50:32.664 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:50:32.672+0000 7fbd13a00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd0c113d80 con 0x7fbd0c109660 2024-03-08T20:50:32.664 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:50:32.672+0000 7fbd13a00700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd0c114080 con 0x7fbd0c108460 2024-03-08T20:50:32.664 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:50:32.672+0000 7fbd0be00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbd0c109660 0x7fbd0c19ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:50:32.664 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:50:32.672+0000 7fbd0be00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbd0c109660 0x7fbd0c19ea90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.149:45470/0 (socket says 172.21.15.149:45470) 2024-03-08T20:50:32.664 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:50:32.672+0000 7fbd0be00700 1 -- 172.21.15.149:0/2896762635 learned_addr learned my addr 172.21.15.149:0/2896762635 (peer_addr_for_me v2:172.21.15.149:0/0) 2024-03-08T20:50:32.665 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:50:32.672+0000 7fbd11000700 1 --2- 172.21.15.149:0/2896762635 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbd0c108460 0x7fbd0c19e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:50:32.665 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:50:32.672+0000 7fbd0be00700 1 -- 172.21.15.149:0/2896762635 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbd0c108460 msgr2=0x7fbd0c19e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:50:32.665 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:50:32.672+0000 7fbd0be00700 1 --2- 172.21.15.149:0/2896762635 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbd0c108460 0x7fbd0c19e550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:50:32.665 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:50:32.672+0000 7fbd0be00700 1 -- 172.21.15.149:0/2896762635 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbd0c10a000 msgr2=0x7fbd0c1a5b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:50:32.665 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:50:32.672+0000 7fbd0be00700 1 --2- 172.21.15.149:0/2896762635 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbd0c10a000 0x7fbd0c1a5b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:50:32.665 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:50:32.673+0000 7fbd0be00700 1 -- 172.21.15.149:0/2896762635 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd0c1a61b0 con 0x7fbd0c109660 2024-03-08T20:50:32.665 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:50:32.673+0000 7fbd11000700 1 --2- 172.21.15.149:0/2896762635 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbd0c108460 0x7fbd0c19e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T20:50:32.665 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:50:32.673+0000 7fbd0be00700 1 --2- 172.21.15.149:0/2896762635 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbd0c109660 0x7fbd0c19ea90 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fbcf80036f0 tx=0x7fbcf800f740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:50:32.665 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:50:32.673+0000 7fbd09600700 1 -- 172.21.15.149:0/2896762635 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcf8009e60 con 0x7fbd0c109660 2024-03-08T20:50:32.665 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:50:32.673+0000 7fbd09600700 1 -- 172.21.15.149:0/2896762635 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fbcf8003d70 con 0x7fbd0c109660 2024-03-08T20:50:32.665 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:50:32.673+0000 7fbd09600700 1 -- 172.21.15.149:0/2896762635 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcf8005670 con 0x7fbd0c109660 2024-03-08T20:50:32.665 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:50:32.673+0000 7fbd13a00700 1 -- 172.21.15.149:0/2896762635 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd0c1a6440 con 0x7fbd0c109660 2024-03-08T20:50:32.665 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:50:32.673+0000 7fbd13a00700 1 -- 172.21.15.149:0/2896762635 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd0c1a6840 con 0x7fbd0c109660 2024-03-08T20:50:32.666 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:50:32.674+0000 7fbd13a00700 1 -- 172.21.15.149:0/2896762635 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd0c02d830 con 0x7fbd0c109660 2024-03-08T20:50:32.666 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:50:32.676+0000 7fbd09600700 1 -- 172.21.15.149:0/2896762635 <== mon.2 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 15) v1 ==== 137235+0+0 (secure 0 0 0) 0x7fbcf80038c0 con 0x7fbd0c109660 2024-03-08T20:50:32.669 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:50:32.679+0000 7fbd09600700 1 --2- 172.21.15.149:0/2896762635 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fbcf40a2880 0x7fbcf40a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:50:32.669 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:50:32.679+0000 7fbd09600700 1 -- 172.21.15.149:0/2896762635 <== mon.2 v2:172.21.15.149:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1767+0+0 (secure 0 0 0) 0x7fbcf80ce020 con 0x7fbd0c109660 2024-03-08T20:50:32.669 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:50:32.679+0000 7fbd11000700 1 --2- 172.21.15.149:0/2896762635 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fbcf40a2880 0x7fbcf40a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:50:32.670 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:50:32.680+0000 7fbd11000700 1 --2- 172.21.15.149:0/2896762635 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fbcf40a2880 0x7fbcf40a4d40 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fbd0c1094c0 tx=0x7fbd00009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:50:32.672 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:50:32.682+0000 7fbd09600700 1 -- 172.21.15.149:0/2896762635 <== mon.2 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7fbcf8013070 con 0x7fbd0c109660 2024-03-08T20:50:32.724 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:50:32 smithi107 ceph-mon[31381]: Detected new or changed devices on smithi107 2024-03-08T20:50:32.724 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:50:32 smithi107 ceph-mon[31381]: Adjusting osd_memory_target on smithi107 to 17090M 2024-03-08T20:50:32.772 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:50:32 smithi149 ceph-mon[35042]: Detected new or changed devices on smithi107 2024-03-08T20:50:32.772 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:50:32 smithi149 ceph-mon[35042]: Adjusting osd_memory_target on smithi107 to 17090M 2024-03-08T20:50:32.787 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:50:32 smithi204 ceph-mon[34935]: Detected new or changed devices on smithi107 2024-03-08T20:50:32.787 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:50:32 smithi204 ceph-mon[34935]: Adjusting osd_memory_target on smithi107 to 17090M 2024-03-08T20:50:32.902 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:50:32.911+0000 7fbd13a00700 1 -- 172.21.15.149:0/2896762635 --> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7fbd0c062220 con 0x7fbcf40a2880 2024-03-08T20:50:33.725 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:50:33 smithi107 ceph-mon[31381]: pgmap v65: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-08T20:50:33.725 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:50:33 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-08T20:50:33.725 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:50:33 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-08T20:50:33.725 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:50:33 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:50:33.772 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:50:33 smithi149 ceph-mon[35042]: pgmap v65: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-08T20:50:33.772 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:50:33 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-08T20:50:33.772 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:50:33 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-08T20:50:33.772 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:50:33 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:50:33.885 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:50:33 smithi204 ceph-mon[34935]: pgmap v65: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-08T20:50:33.886 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:50:33 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-08T20:50:33.886 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:50:33 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-08T20:50:33.886 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:50:33 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:50:34.724 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:50:34 smithi107 ceph-mon[31381]: from='client.34139 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-08T20:50:34.725 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:50:34 smithi107 ceph-mon[31381]: pgmap v66: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-08T20:50:34.772 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:50:34 smithi149 ceph-mon[35042]: from='client.34139 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-08T20:50:34.772 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:50:34 smithi149 ceph-mon[35042]: pgmap v66: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-08T20:50:34.885 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:50:34 smithi204 ceph-mon[34935]: from='client.34139 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-08T20:50:34.885 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:50:34 smithi204 ceph-mon[34935]: pgmap v66: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-08T20:50:37.522 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:50:37 smithi149 ceph-mon[35042]: pgmap v67: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-08T20:50:37.522 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:50:37 smithi149 ceph-mon[35042]: from='client.? 172.21.15.149:0/3430894417' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5a803c09-202a-4777-ba2a-1da8eaf26a54"}]: dispatch 2024-03-08T20:50:37.523 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:50:37 smithi149 ceph-mon[35042]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5a803c09-202a-4777-ba2a-1da8eaf26a54"}]: dispatch 2024-03-08T20:50:37.523 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:50:37 smithi149 ceph-mon[35042]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5a803c09-202a-4777-ba2a-1da8eaf26a54"}]': finished 2024-03-08T20:50:37.523 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:50:37 smithi149 ceph-mon[35042]: osdmap e10: 2 total, 1 up, 2 in 2024-03-08T20:50:37.523 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:50:37 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-08T20:50:37.635 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:50:37 smithi204 ceph-mon[34935]: pgmap v67: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-08T20:50:37.636 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:50:37 smithi204 ceph-mon[34935]: from='client.? 172.21.15.149:0/3430894417' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5a803c09-202a-4777-ba2a-1da8eaf26a54"}]: dispatch 2024-03-08T20:50:37.636 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:50:37 smithi204 ceph-mon[34935]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5a803c09-202a-4777-ba2a-1da8eaf26a54"}]: dispatch 2024-03-08T20:50:37.636 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:50:37 smithi204 ceph-mon[34935]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5a803c09-202a-4777-ba2a-1da8eaf26a54"}]': finished 2024-03-08T20:50:37.636 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:50:37 smithi204 ceph-mon[34935]: osdmap e10: 2 total, 1 up, 2 in 2024-03-08T20:50:37.636 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:50:37 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-08T20:50:37.724 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:50:37 smithi107 ceph-mon[31381]: pgmap v67: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-08T20:50:37.724 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:50:37 smithi107 ceph-mon[31381]: from='client.? 172.21.15.149:0/3430894417' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5a803c09-202a-4777-ba2a-1da8eaf26a54"}]: dispatch 2024-03-08T20:50:37.725 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:50:37 smithi107 ceph-mon[31381]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5a803c09-202a-4777-ba2a-1da8eaf26a54"}]: dispatch 2024-03-08T20:50:37.725 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:50:37 smithi107 ceph-mon[31381]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5a803c09-202a-4777-ba2a-1da8eaf26a54"}]': finished 2024-03-08T20:50:37.725 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:50:37 smithi107 ceph-mon[31381]: osdmap e10: 2 total, 1 up, 2 in 2024-03-08T20:50:37.725 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:50:37 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-08T20:50:38.522 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:50:38 smithi149 ceph-mon[35042]: from='client.? 172.21.15.149:0/1233931457' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-08T20:50:38.635 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:50:38 smithi204 ceph-mon[34935]: from='client.? 172.21.15.149:0/1233931457' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-08T20:50:38.724 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:50:38 smithi107 ceph-mon[31381]: from='client.? 172.21.15.149:0/1233931457' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-08T20:50:39.522 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:50:39 smithi149 ceph-mon[35042]: pgmap v69: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-08T20:50:39.635 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:50:39 smithi204 ceph-mon[34935]: pgmap v69: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-08T20:50:39.724 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:50:39 smithi107 ceph-mon[31381]: pgmap v69: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-08T20:50:41.523 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:50:41 smithi149 ceph-mon[35042]: pgmap v70: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-08T20:50:41.527 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:50:41 smithi107 ceph-mon[31381]: pgmap v70: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-08T20:50:41.635 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:50:41 smithi204 ceph-mon[34935]: pgmap v70: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-08T20:50:43.474 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:50:43 smithi107 ceph-mon[31381]: pgmap v71: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-08T20:50:43.522 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:50:43 smithi149 ceph-mon[35042]: pgmap v71: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-08T20:50:43.635 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:50:43 smithi204 ceph-mon[34935]: pgmap v71: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-08T20:50:45.522 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:50:45 smithi149 ceph-mon[35042]: pgmap v72: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-08T20:50:45.635 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:50:45 smithi204 ceph-mon[34935]: pgmap v72: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-08T20:50:45.724 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:50:45 smithi107 ceph-mon[31381]: pgmap v72: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-08T20:50:46.596 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:50:46 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-08T20:50:46.597 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:50:46 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:50:46.597 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:50:46 smithi149 ceph-mon[35042]: Deploying daemon osd.1 on smithi149 2024-03-08T20:50:46.635 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:50:46 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-08T20:50:46.635 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:50:46 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:50:46.635 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:50:46 smithi204 ceph-mon[34935]: Deploying daemon osd.1 on smithi149 2024-03-08T20:50:46.724 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:50:46 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-08T20:50:46.724 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:50:46 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:50:46.724 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:50:46 smithi107 ceph-mon[31381]: Deploying daemon osd.1 on smithi149 2024-03-08T20:50:47.420 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:50:47 smithi149 ceph-mon[35042]: pgmap v73: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-08T20:50:47.635 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:50:47 smithi204 ceph-mon[34935]: pgmap v73: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-08T20:50:47.724 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:50:47 smithi107 ceph-mon[31381]: pgmap v73: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-08T20:50:49.522 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:50:49 smithi149 ceph-mon[35042]: pgmap v74: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-08T20:50:49.635 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:50:49 smithi204 ceph-mon[34935]: pgmap v74: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-08T20:50:49.724 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:50:49 smithi107 ceph-mon[31381]: pgmap v74: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-08T20:50:51.635 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:50:51 smithi204 ceph-mon[34935]: pgmap v75: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-08T20:50:51.701 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:50:51 smithi149 ceph-mon[35042]: pgmap v75: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-08T20:50:51.724 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:50:51 smithi107 ceph-mon[31381]: pgmap v75: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-08T20:50:53.474 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:50:53 smithi107 ceph-mon[31381]: pgmap v76: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-08T20:50:53.474 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:50:53 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:50:53.474 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:50:53 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:50:53.474 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:50:53 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T20:50:53.569 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:50:53 smithi149 ceph-mon[35042]: pgmap v76: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-08T20:50:53.569 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:50:53 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:50:53.569 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:50:53 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:50:53.569 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:50:53 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T20:50:53.635 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:50:53 smithi204 ceph-mon[34935]: pgmap v76: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-08T20:50:53.635 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:50:53 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:50:53.635 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:50:53 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:50:53.635 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:50:53 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T20:50:55.512 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:50:55 smithi149 ceph-mon[35042]: pgmap v77: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-08T20:50:55.635 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:50:55 smithi204 ceph-mon[34935]: pgmap v77: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-08T20:50:55.724 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:50:55 smithi107 ceph-mon[31381]: pgmap v77: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-08T20:50:57.635 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:50:57 smithi204 ceph-mon[34935]: pgmap v78: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-08T20:50:57.636 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:50:57 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:50:57.636 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:50:57 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:50:57.636 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:50:57 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:50:57.636 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:50:57 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T20:50:57.636 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:50:57 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:50:57.636 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:50:57 smithi204 ceph-mon[34935]: from='osd.1 [v2:172.21.15.149:6800/3071316419,v1:172.21.15.149:6801/3071316419]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-08T20:50:57.637 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:50:57 smithi204 ceph-mon[34935]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-08T20:50:57.725 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:50:57 smithi107 ceph-mon[31381]: pgmap v78: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-08T20:50:57.725 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:50:57 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:50:57.725 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:50:57 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:50:57.725 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:50:57 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:50:57.725 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:50:57 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T20:50:57.726 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:50:57 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:50:57.726 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:50:57 smithi107 ceph-mon[31381]: from='osd.1 [v2:172.21.15.149:6800/3071316419,v1:172.21.15.149:6801/3071316419]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-08T20:50:57.726 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:50:57 smithi107 ceph-mon[31381]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-08T20:50:57.762 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:50:57 smithi149 ceph-mon[35042]: pgmap v78: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-08T20:50:57.762 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:50:57 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:50:57.762 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:50:57 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:50:57.762 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:50:57 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:50:57.762 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:50:57 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T20:50:57.763 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:50:57 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:50:57.763 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:50:57 smithi149 ceph-mon[35042]: from='osd.1 [v2:172.21.15.149:6800/3071316419,v1:172.21.15.149:6801/3071316419]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-08T20:50:57.763 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:50:57 smithi149 ceph-mon[35042]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-08T20:50:57.907 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:50:57.916+0000 7fbd09600700 1 -- 172.21.15.149:0/2896762635 <== mgr.14152 v2:172.21.15.107:6800/3376636618 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fbd0c062220 con 0x7fbcf40a2880 2024-03-08T20:50:57.908 INFO:teuthology.orchestra.run.smithi149.stdout:Created osd(s) 1 on host 'smithi149' 2024-03-08T20:50:57.910 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:50:57.920+0000 7fbd13a00700 1 -- 172.21.15.149:0/2896762635 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fbcf40a2880 msgr2=0x7fbcf40a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:50:57.911 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:50:57.920+0000 7fbd13a00700 1 --2- 172.21.15.149:0/2896762635 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fbcf40a2880 0x7fbcf40a4d40 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fbd0c1094c0 tx=0x7fbd00009260 comp rx=0 tx=0).stop 2024-03-08T20:50:57.911 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:50:57.920+0000 7fbd13a00700 1 -- 172.21.15.149:0/2896762635 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbd0c109660 msgr2=0x7fbd0c19ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:50:57.911 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:50:57.920+0000 7fbd13a00700 1 --2- 172.21.15.149:0/2896762635 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbd0c109660 0x7fbd0c19ea90 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fbcf80036f0 tx=0x7fbcf800f740 comp rx=0 tx=0).stop 2024-03-08T20:50:57.911 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:50:57.920+0000 7fbd13a00700 1 -- 172.21.15.149:0/2896762635 shutdown_connections 2024-03-08T20:50:57.911 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:50:57.920+0000 7fbd13a00700 1 --2- 172.21.15.149:0/2896762635 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbd0c108460 0x7fbd0c19e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:50:57.911 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:50:57.920+0000 7fbd13a00700 1 --2- 172.21.15.149:0/2896762635 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fbcf40a2880 0x7fbcf40a4d40 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:50:57.911 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:50:57.920+0000 7fbd13a00700 1 --2- 172.21.15.149:0/2896762635 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbd0c10a000 0x7fbd0c1a5b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:50:57.911 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:50:57.920+0000 7fbd13a00700 1 --2- 172.21.15.149:0/2896762635 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbd0c109660 0x7fbd0c19ea90 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:50:57.911 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:50:57.920+0000 7fbd13a00700 1 -- 172.21.15.149:0/2896762635 >> 172.21.15.149:0/2896762635 conn(0x7fbd0c077430 msgr2=0x7fbd0c10ca20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:50:57.912 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:50:57.920+0000 7fbd13a00700 1 -- 172.21.15.149:0/2896762635 shutdown_connections 2024-03-08T20:50:57.912 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:50:57.920+0000 7fbd13a00700 1 -- 172.21.15.149:0/2896762635 wait complete. 2024-03-08T20:50:58.599 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:50:58 smithi149 ceph-mon[35042]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-08T20:50:58.599 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:50:58 smithi149 ceph-mon[35042]: from='osd.1 [v2:172.21.15.149:6800/3071316419,v1:172.21.15.149:6801/3071316419]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-03-08T20:50:58.599 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:50:58 smithi149 ceph-mon[35042]: osdmap e11: 2 total, 1 up, 2 in 2024-03-08T20:50:58.599 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:50:58 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-08T20:50:58.599 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:50:58 smithi149 ceph-mon[35042]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-03-08T20:50:58.599 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:50:58 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:50:58.599 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:50:58 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:50:58.599 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:50:58 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T20:50:58.600 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:50:58 smithi149 ceph-mon[35042]: pgmap v80: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-08T20:50:58.644 DEBUG:teuthology.orchestra.run.smithi149:osd.1> sudo journalctl -f -n 0 -u ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715@osd.1.service 2024-03-08T20:50:58.648 INFO:tasks.cephadm:Deploying osd.2 on smithi204 with /dev/vg_nvme/lv_4... 2024-03-08T20:50:58.648 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dc20f0f8-dd8c-11ee-95c6-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-03-08T20:50:58.885 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:50:58 smithi204 ceph-mon[34935]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-08T20:50:58.886 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:50:58 smithi204 ceph-mon[34935]: from='osd.1 [v2:172.21.15.149:6800/3071316419,v1:172.21.15.149:6801/3071316419]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-03-08T20:50:58.886 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:50:58 smithi204 ceph-mon[34935]: osdmap e11: 2 total, 1 up, 2 in 2024-03-08T20:50:58.886 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:50:58 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-08T20:50:58.886 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:50:58 smithi204 ceph-mon[34935]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-03-08T20:50:58.886 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:50:58 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:50:58.886 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:50:58 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:50:58.886 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:50:58 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T20:50:58.886 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:50:58 smithi204 ceph-mon[34935]: pgmap v80: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-08T20:50:58.974 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:50:58 smithi107 ceph-mon[31381]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-08T20:50:58.974 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:50:58 smithi107 ceph-mon[31381]: from='osd.1 [v2:172.21.15.149:6800/3071316419,v1:172.21.15.149:6801/3071316419]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-03-08T20:50:58.975 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:50:58 smithi107 ceph-mon[31381]: osdmap e11: 2 total, 1 up, 2 in 2024-03-08T20:50:58.975 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:50:58 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-08T20:50:58.975 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:50:58 smithi107 ceph-mon[31381]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-03-08T20:50:58.975 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:50:58 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:50:58.975 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:50:58 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:50:58.975 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:50:58 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T20:50:58.975 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:50:58 smithi107 ceph-mon[31381]: pgmap v80: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-08T20:50:59.080 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/mon.c/config 2024-03-08T20:50:59.772 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:50:59 smithi149 ceph-mon[35042]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]': finished 2024-03-08T20:50:59.772 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:50:59 smithi149 ceph-mon[35042]: osdmap e12: 2 total, 1 up, 2 in 2024-03-08T20:50:59.772 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:50:59 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-08T20:50:59.772 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:50:59 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-08T20:50:59.885 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:50:59 smithi204 ceph-mon[34935]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]': finished 2024-03-08T20:50:59.885 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:50:59 smithi204 ceph-mon[34935]: osdmap e12: 2 total, 1 up, 2 in 2024-03-08T20:50:59.885 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:50:59 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-08T20:50:59.885 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:50:59 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-08T20:50:59.974 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:50:59 smithi107 ceph-mon[31381]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]': finished 2024-03-08T20:50:59.974 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:50:59 smithi107 ceph-mon[31381]: osdmap e12: 2 total, 1 up, 2 in 2024-03-08T20:50:59.975 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:50:59 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-08T20:50:59.975 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:50:59 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-08T20:51:00.772 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:51:00 smithi149 ceph-mon[35042]: purged_snaps scrub starts 2024-03-08T20:51:00.772 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:51:00 smithi149 ceph-mon[35042]: purged_snaps scrub ok 2024-03-08T20:51:00.773 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:51:00 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-08T20:51:00.773 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:51:00 smithi149 ceph-mon[35042]: osd.1 [v2:172.21.15.149:6800/3071316419,v1:172.21.15.149:6801/3071316419] boot 2024-03-08T20:51:00.773 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:51:00 smithi149 ceph-mon[35042]: osdmap e13: 2 total, 2 up, 2 in 2024-03-08T20:51:00.773 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:51:00 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-08T20:51:00.773 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:51:00 smithi149 ceph-mon[35042]: pgmap v83: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-08T20:51:00.789 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:51:00 smithi204 ceph-mon[34935]: purged_snaps scrub starts 2024-03-08T20:51:00.789 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:51:00 smithi204 ceph-mon[34935]: purged_snaps scrub ok 2024-03-08T20:51:00.790 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:51:00 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-08T20:51:00.790 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:51:00 smithi204 ceph-mon[34935]: osd.1 [v2:172.21.15.149:6800/3071316419,v1:172.21.15.149:6801/3071316419] boot 2024-03-08T20:51:00.790 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:51:00 smithi204 ceph-mon[34935]: osdmap e13: 2 total, 2 up, 2 in 2024-03-08T20:51:00.790 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:51:00 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-08T20:51:00.790 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:51:00 smithi204 ceph-mon[34935]: pgmap v83: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-08T20:51:00.974 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:51:00 smithi107 ceph-mon[31381]: purged_snaps scrub starts 2024-03-08T20:51:00.974 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:51:00 smithi107 ceph-mon[31381]: purged_snaps scrub ok 2024-03-08T20:51:00.975 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:51:00 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-08T20:51:00.975 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:51:00 smithi107 ceph-mon[31381]: osd.1 [v2:172.21.15.149:6800/3071316419,v1:172.21.15.149:6801/3071316419] boot 2024-03-08T20:51:00.975 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:51:00 smithi107 ceph-mon[31381]: osdmap e13: 2 total, 2 up, 2 in 2024-03-08T20:51:00.975 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:51:00 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-08T20:51:00.975 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:51:00 smithi107 ceph-mon[31381]: pgmap v83: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-08T20:51:01.763 INFO:teuthology.orchestra.run.smithi204.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-03-08T20:51:01.763 INFO:teuthology.orchestra.run.smithi204.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-03-08T20:51:01.763 INFO:teuthology.orchestra.run.smithi204.stderr: stderr: 10+0 records in 2024-03-08T20:51:01.763 INFO:teuthology.orchestra.run.smithi204.stderr:10+0 records out 2024-03-08T20:51:01.763 INFO:teuthology.orchestra.run.smithi204.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.034893 s, 301 MB/s 2024-03-08T20:51:01.763 INFO:teuthology.orchestra.run.smithi204.stderr:--> Zapping successful for: 2024-03-08T20:51:01.813 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:51:01 smithi204 ceph-mon[34935]: osdmap e14: 2 total, 2 up, 2 in 2024-03-08T20:51:01.974 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:51:01 smithi107 ceph-mon[31381]: osdmap e14: 2 total, 2 up, 2 in 2024-03-08T20:51:02.021 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:51:01 smithi149 ceph-mon[35042]: osdmap e14: 2 total, 2 up, 2 in 2024-03-08T20:51:02.498 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dc20f0f8-dd8c-11ee-95c6-87774f69a715 -- ceph orch daemon add osd smithi204:vg_nvme/lv_4 2024-03-08T20:51:02.885 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:51:02 smithi204 ceph-mon[34935]: pgmap v85: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-08T20:51:02.913 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/mon.c/config 2024-03-08T20:51:02.937 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:51:02 smithi149 ceph-mon[35042]: pgmap v85: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-08T20:51:02.974 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:51:02 smithi107 ceph-mon[31381]: pgmap v85: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-08T20:51:04.080 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:51:04 smithi204 ceph-mon[34935]: Detected new or changed devices on smithi149 2024-03-08T20:51:04.081 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:51:04 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:51:04.081 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:51:04 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:51:04.081 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:51:04 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-08T20:51:04.081 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:51:04 smithi204 ceph-mon[34935]: Adjusting osd_memory_target on smithi149 to 5818M 2024-03-08T20:51:04.081 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:51:04 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:51:04.081 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:51:04 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:51:04.081 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:51:04 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T20:51:04.081 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:51:04 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:51:04.272 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:51:04 smithi149 ceph-mon[35042]: Detected new or changed devices on smithi149 2024-03-08T20:51:04.272 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:51:04 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:51:04.272 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:51:04 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:51:04.272 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:51:04 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-08T20:51:04.272 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:51:04 smithi149 ceph-mon[35042]: Adjusting osd_memory_target on smithi149 to 5818M 2024-03-08T20:51:04.272 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:51:04 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:51:04.273 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:51:04 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:51:04.273 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:51:04 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T20:51:04.273 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:51:04 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:51:04.474 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:51:04 smithi107 ceph-mon[31381]: Detected new or changed devices on smithi149 2024-03-08T20:51:04.475 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:51:04 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:51:04.475 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:51:04 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:51:04.475 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:51:04 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-08T20:51:04.475 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:51:04 smithi107 ceph-mon[31381]: Adjusting osd_memory_target on smithi149 to 5818M 2024-03-08T20:51:04.475 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:51:04 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:51:04.475 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:51:04 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:51:04.475 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:51:04 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T20:51:04.475 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:51:04 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:51:04.854 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:51:04.863+0000 7faa1ac00700 1 -- 172.21.15.204:0/84065100 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faa14101e20 msgr2=0x7faa14102240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:51:04.854 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:51:04.863+0000 7faa1ac00700 1 --2- 172.21.15.204:0/84065100 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faa14101e20 0x7faa14102240 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7faa04009990 tx=0x7faa04009ca0 comp rx=0 tx=0).stop 2024-03-08T20:51:04.854 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:51:04.863+0000 7faa1ac00700 1 -- 172.21.15.204:0/84065100 shutdown_connections 2024-03-08T20:51:04.854 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:51:04.863+0000 7faa1ac00700 1 --2- 172.21.15.204:0/84065100 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faa14101e20 0x7faa14102240 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:51:04.854 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:51:04.863+0000 7faa1ac00700 1 --2- 172.21.15.204:0/84065100 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faa14103940 0x7faa1410a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:51:04.854 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:51:04.863+0000 7faa1ac00700 1 --2- 172.21.15.204:0/84065100 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faa14102f80 0x7faa14103400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:51:04.854 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:51:04.863+0000 7faa1ac00700 1 -- 172.21.15.204:0/84065100 >> 172.21.15.204:0/84065100 conn(0x7faa140fd380 msgr2=0x7faa140ff800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:51:04.854 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:51:04.864+0000 7faa1ac00700 1 -- 172.21.15.204:0/84065100 shutdown_connections 2024-03-08T20:51:04.854 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:51:04.864+0000 7faa1ac00700 1 -- 172.21.15.204:0/84065100 wait complete. 2024-03-08T20:51:04.856 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:51:04.865+0000 7faa1ac00700 1 Processor -- start 2024-03-08T20:51:04.856 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:51:04.865+0000 7faa1ac00700 1 -- start start 2024-03-08T20:51:04.856 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:51:04.865+0000 7faa1ac00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faa14101e20 0x7faa14197e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:51:04.856 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:51:04.865+0000 7faa1ac00700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faa14102f80 0x7faa141983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:51:04.857 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:51:04.866+0000 7faa1ac00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faa14103940 0x7faa1419f420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:51:04.857 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:51:04.866+0000 7faa1ac00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa1410d8b0 con 0x7faa14101e20 2024-03-08T20:51:04.857 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:51:04.866+0000 7faa1ac00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa1410d730 con 0x7faa14103940 2024-03-08T20:51:04.857 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:51:04.866+0000 7faa1ac00700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa1410da30 con 0x7faa14102f80 2024-03-08T20:51:04.857 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:51:04.866+0000 7faa13400700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faa14102f80 0x7faa141983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:51:04.857 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:51:04.866+0000 7faa13400700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faa14102f80 0x7faa141983a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:53940/0 (socket says 172.21.15.204:53940) 2024-03-08T20:51:04.857 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:51:04.866+0000 7faa13400700 1 -- 172.21.15.204:0/842636463 learned_addr learned my addr 172.21.15.204:0/842636463 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-03-08T20:51:04.857 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:51:04.866+0000 7faa13e00700 1 --2- 172.21.15.204:0/842636463 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faa14101e20 0x7faa14197e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:51:04.857 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:51:04.866+0000 7faa18c00700 1 --2- 172.21.15.204:0/842636463 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faa14103940 0x7faa1419f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:51:04.857 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:51:04.866+0000 7faa13400700 1 -- 172.21.15.204:0/842636463 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faa14103940 msgr2=0x7faa1419f420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:51:04.857 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:51:04.866+0000 7faa13400700 1 --2- 172.21.15.204:0/842636463 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faa14103940 0x7faa1419f420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:51:04.857 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:51:04.866+0000 7faa13400700 1 -- 172.21.15.204:0/842636463 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faa14101e20 msgr2=0x7faa14197e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:51:04.857 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:51:04.866+0000 7faa13400700 1 --2- 172.21.15.204:0/842636463 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faa14101e20 0x7faa14197e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:51:04.858 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:51:04.866+0000 7faa13400700 1 -- 172.21.15.204:0/842636463 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa1419fac0 con 0x7faa14102f80 2024-03-08T20:51:04.858 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:51:04.867+0000 7faa13400700 1 --2- 172.21.15.204:0/842636463 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faa14102f80 0x7faa141983a0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7faa0000c9e0 tx=0x7faa0000cda0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:51:04.858 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:51:04.867+0000 7faa10c00700 1 -- 172.21.15.204:0/842636463 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa00007ac0 con 0x7faa14102f80 2024-03-08T20:51:04.858 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:51:04.867+0000 7faa10c00700 1 -- 172.21.15.204:0/842636463 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7faa00007c60 con 0x7faa14102f80 2024-03-08T20:51:04.858 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:51:04.867+0000 7faa1ac00700 1 -- 172.21.15.204:0/842636463 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa1419fdb0 con 0x7faa14102f80 2024-03-08T20:51:04.858 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:51:04.867+0000 7faa10c00700 1 -- 172.21.15.204:0/842636463 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa00018660 con 0x7faa14102f80 2024-03-08T20:51:04.859 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:51:04.867+0000 7faa1ac00700 1 -- 172.21.15.204:0/842636463 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa141a0270 con 0x7faa14102f80 2024-03-08T20:51:04.859 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:51:04.868+0000 7faa1ac00700 1 -- 172.21.15.204:0/842636463 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa14068aa0 con 0x7faa14102f80 2024-03-08T20:51:04.861 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:51:04.872+0000 7faa10c00700 1 -- 172.21.15.204:0/842636463 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 15) v1 ==== 137235+0+0 (secure 0 0 0) 0x7faa0000f450 con 0x7faa14102f80 2024-03-08T20:51:04.863 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:51:04.873+0000 7faa10c00700 1 --2- 172.21.15.204:0/842636463 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fa9f80a2880 0x7fa9f80a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:51:04.863 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:51:04.873+0000 7faa10c00700 1 -- 172.21.15.204:0/842636463 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2309+0+0 (secure 0 0 0) 0x7faa000cd570 con 0x7faa14102f80 2024-03-08T20:51:04.863 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:51:04.874+0000 7faa13e00700 1 --2- 172.21.15.204:0/842636463 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fa9f80a2880 0x7fa9f80a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:51:04.864 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:51:04.875+0000 7faa13e00700 1 --2- 172.21.15.204:0/842636463 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fa9f80a2880 0x7fa9f80a4d40 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7faa04006010 tx=0x7faa04003870 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:51:04.866 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:51:04.877+0000 7faa10c00700 1 -- 172.21.15.204:0/842636463 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7faa00014030 con 0x7faa14102f80 2024-03-08T20:51:05.110 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:51:05.120+0000 7faa1ac00700 1 -- 172.21.15.204:0/842636463 --> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7faa14062220 con 0x7fa9f80a2880 2024-03-08T20:51:05.271 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:51:05 smithi149 ceph-mon[35042]: pgmap v86: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-08T20:51:05.385 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:51:05 smithi204 ceph-mon[34935]: pgmap v86: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-08T20:51:05.474 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:51:05 smithi107 ceph-mon[31381]: pgmap v86: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-08T20:51:06.272 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:51:06 smithi149 ceph-mon[35042]: from='client.24169 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-08T20:51:06.272 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:51:06 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-08T20:51:06.272 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:51:06 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-08T20:51:06.272 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:51:06 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:51:06.334 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:51:06 smithi204 ceph-mon[34935]: from='client.24169 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-08T20:51:06.334 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:51:06 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-08T20:51:06.334 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:51:06 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-08T20:51:06.334 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:51:06 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:51:06.474 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:51:06 smithi107 ceph-mon[31381]: from='client.24169 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-08T20:51:06.474 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:51:06 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-08T20:51:06.474 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:51:06 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-08T20:51:06.474 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:51:06 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:51:07.346 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:51:07 smithi204 ceph-mon[34935]: pgmap v87: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-08T20:51:07.474 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:51:07 smithi107 ceph-mon[31381]: pgmap v87: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-08T20:51:07.521 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:51:07 smithi149 ceph-mon[35042]: pgmap v87: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-08T20:51:09.522 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:51:09 smithi149 ceph-mon[35042]: pgmap v88: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-08T20:51:09.522 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:51:09 smithi149 ceph-mon[35042]: from='client.? 172.21.15.204:0/678689311' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "555c634f-347e-47dd-a943-e50ae2be073c"}]: dispatch 2024-03-08T20:51:09.522 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:51:09 smithi149 ceph-mon[35042]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "555c634f-347e-47dd-a943-e50ae2be073c"}]: dispatch 2024-03-08T20:51:09.522 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:51:09 smithi149 ceph-mon[35042]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "555c634f-347e-47dd-a943-e50ae2be073c"}]': finished 2024-03-08T20:51:09.522 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:51:09 smithi149 ceph-mon[35042]: osdmap e15: 3 total, 2 up, 3 in 2024-03-08T20:51:09.522 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:51:09 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-08T20:51:09.635 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:51:09 smithi204 ceph-mon[34935]: pgmap v88: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-08T20:51:09.635 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:51:09 smithi204 ceph-mon[34935]: from='client.? 172.21.15.204:0/678689311' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "555c634f-347e-47dd-a943-e50ae2be073c"}]: dispatch 2024-03-08T20:51:09.635 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:51:09 smithi204 ceph-mon[34935]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "555c634f-347e-47dd-a943-e50ae2be073c"}]: dispatch 2024-03-08T20:51:09.635 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:51:09 smithi204 ceph-mon[34935]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "555c634f-347e-47dd-a943-e50ae2be073c"}]': finished 2024-03-08T20:51:09.636 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:51:09 smithi204 ceph-mon[34935]: osdmap e15: 3 total, 2 up, 3 in 2024-03-08T20:51:09.636 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:51:09 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-08T20:51:09.724 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:51:09 smithi107 ceph-mon[31381]: pgmap v88: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-08T20:51:09.724 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:51:09 smithi107 ceph-mon[31381]: from='client.? 172.21.15.204:0/678689311' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "555c634f-347e-47dd-a943-e50ae2be073c"}]: dispatch 2024-03-08T20:51:09.724 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:51:09 smithi107 ceph-mon[31381]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "555c634f-347e-47dd-a943-e50ae2be073c"}]: dispatch 2024-03-08T20:51:09.724 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:51:09 smithi107 ceph-mon[31381]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "555c634f-347e-47dd-a943-e50ae2be073c"}]': finished 2024-03-08T20:51:09.724 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:51:09 smithi107 ceph-mon[31381]: osdmap e15: 3 total, 2 up, 3 in 2024-03-08T20:51:09.725 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:51:09 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-08T20:51:10.522 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:51:10 smithi149 ceph-mon[35042]: from='client.? 172.21.15.204:0/2473233699' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-08T20:51:10.635 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:51:10 smithi204 ceph-mon[34935]: from='client.? 172.21.15.204:0/2473233699' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-08T20:51:10.724 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:51:10 smithi107 ceph-mon[31381]: from='client.? 172.21.15.204:0/2473233699' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-08T20:51:11.521 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:51:11 smithi149 ceph-mon[35042]: pgmap v90: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-08T20:51:11.626 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:51:11 smithi107 ceph-mon[31381]: pgmap v90: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-08T20:51:11.634 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:51:11 smithi204 ceph-mon[34935]: pgmap v90: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-08T20:51:13.474 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:51:13 smithi107 ceph-mon[31381]: pgmap v91: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-08T20:51:13.521 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:51:13 smithi149 ceph-mon[35042]: pgmap v91: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-08T20:51:13.635 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:51:13 smithi204 ceph-mon[34935]: pgmap v91: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-08T20:51:15.521 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:51:15 smithi149 ceph-mon[35042]: pgmap v92: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-08T20:51:15.609 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:51:15 smithi204 ceph-mon[34935]: pgmap v92: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-08T20:51:15.724 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:51:15 smithi107 ceph-mon[31381]: pgmap v92: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-08T20:51:17.385 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:51:17 smithi204 ceph-mon[34935]: pgmap v93: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-08T20:51:17.724 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:51:17 smithi107 ceph-mon[31381]: pgmap v93: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-08T20:51:17.771 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:51:17 smithi149 ceph-mon[35042]: pgmap v93: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-08T20:51:18.572 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:51:18 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-08T20:51:18.573 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:51:18 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:51:18.723 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:51:18 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-08T20:51:18.724 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:51:18 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:51:18.771 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:51:18 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-08T20:51:18.771 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:51:18 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:51:19.480 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:51:19 smithi204 ceph-mon[34935]: Deploying daemon osd.2 on smithi204 2024-03-08T20:51:19.481 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:51:19 smithi204 ceph-mon[34935]: pgmap v94: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-08T20:51:19.724 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:51:19 smithi107 ceph-mon[31381]: Deploying daemon osd.2 on smithi204 2024-03-08T20:51:19.724 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:51:19 smithi107 ceph-mon[31381]: pgmap v94: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-08T20:51:19.771 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:51:19 smithi149 ceph-mon[35042]: Deploying daemon osd.2 on smithi204 2024-03-08T20:51:19.771 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:51:19 smithi149 ceph-mon[35042]: pgmap v94: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-08T20:51:21.521 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:51:21 smithi149 ceph-mon[35042]: pgmap v95: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-08T20:51:21.633 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:51:21 smithi204 ceph-mon[34935]: pgmap v95: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-08T20:51:21.723 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:51:21 smithi107 ceph-mon[31381]: pgmap v95: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-08T20:51:23.635 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:51:23 smithi204 ceph-mon[34935]: pgmap v96: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-08T20:51:23.724 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:51:23 smithi107 ceph-mon[31381]: pgmap v96: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-08T20:51:23.771 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:51:23 smithi149 ceph-mon[35042]: pgmap v96: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-08T20:51:25.384 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:51:25 smithi204 ceph-mon[34935]: pgmap v97: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-08T20:51:25.384 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:51:25 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:51:25.726 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:51:25 smithi107 ceph-mon[31381]: pgmap v97: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-08T20:51:25.726 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:51:25 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:51:25.771 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:51:25 smithi149 ceph-mon[35042]: pgmap v97: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-08T20:51:25.771 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:51:25 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:51:26.503 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:51:26 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:51:26.503 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:51:26 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T20:51:26.723 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:51:26 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:51:26.724 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:51:26 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T20:51:26.771 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:51:26 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:51:26.771 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:51:26 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T20:51:27.634 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:51:27 smithi204 ceph-mon[34935]: pgmap v98: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-08T20:51:27.723 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:51:27 smithi107 ceph-mon[31381]: pgmap v98: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-08T20:51:27.774 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:51:27 smithi149 ceph-mon[35042]: pgmap v98: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-08T20:51:29.548 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:51:29 smithi204 ceph-mon[34935]: pgmap v99: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-08T20:51:29.548 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:51:29 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:51:29.548 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:51:29 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:51:29.549 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:51:29 smithi204 ceph-mon[34935]: from='osd.2 [v2:172.21.15.204:6800/1087852630,v1:172.21.15.204:6801/1087852630]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-08T20:51:29.549 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:51:29 smithi204 ceph-mon[34935]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-08T20:51:29.671 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:51:29.681+0000 7faa10c00700 1 -- 172.21.15.204:0/842636463 <== mgr.14152 v2:172.21.15.107:6800/3376636618 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7faa14062220 con 0x7fa9f80a2880 2024-03-08T20:51:29.671 INFO:teuthology.orchestra.run.smithi204.stdout:Created osd(s) 2 on host 'smithi204' 2024-03-08T20:51:29.674 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:51:29.683+0000 7faa1ac00700 1 -- 172.21.15.204:0/842636463 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fa9f80a2880 msgr2=0x7fa9f80a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:51:29.674 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:51:29.683+0000 7faa1ac00700 1 --2- 172.21.15.204:0/842636463 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fa9f80a2880 0x7fa9f80a4d40 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7faa04006010 tx=0x7faa04003870 comp rx=0 tx=0).stop 2024-03-08T20:51:29.674 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:51:29.683+0000 7faa1ac00700 1 -- 172.21.15.204:0/842636463 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faa14102f80 msgr2=0x7faa141983a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:51:29.674 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:51:29.683+0000 7faa1ac00700 1 --2- 172.21.15.204:0/842636463 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faa14102f80 0x7faa141983a0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7faa0000c9e0 tx=0x7faa0000cda0 comp rx=0 tx=0).stop 2024-03-08T20:51:29.674 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:51:29.684+0000 7faa1ac00700 1 -- 172.21.15.204:0/842636463 shutdown_connections 2024-03-08T20:51:29.674 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:51:29.684+0000 7faa1ac00700 1 --2- 172.21.15.204:0/842636463 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fa9f80a2880 0x7fa9f80a4d40 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:51:29.674 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:51:29.684+0000 7faa1ac00700 1 --2- 172.21.15.204:0/842636463 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faa14103940 0x7faa1419f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:51:29.674 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:51:29.684+0000 7faa1ac00700 1 --2- 172.21.15.204:0/842636463 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faa14101e20 0x7faa14197e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:51:29.674 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:51:29.684+0000 7faa1ac00700 1 --2- 172.21.15.204:0/842636463 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faa14102f80 0x7faa141983a0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:51:29.674 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:51:29.684+0000 7faa1ac00700 1 -- 172.21.15.204:0/842636463 >> 172.21.15.204:0/842636463 conn(0x7faa140fd380 msgr2=0x7faa141062b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:51:29.675 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:51:29.684+0000 7faa1ac00700 1 -- 172.21.15.204:0/842636463 shutdown_connections 2024-03-08T20:51:29.675 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:51:29.684+0000 7faa1ac00700 1 -- 172.21.15.204:0/842636463 wait complete. 2024-03-08T20:51:29.723 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:51:29 smithi107 ceph-mon[31381]: pgmap v99: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-08T20:51:29.723 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:51:29 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:51:29.724 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:51:29 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:51:29.724 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:51:29 smithi107 ceph-mon[31381]: from='osd.2 [v2:172.21.15.204:6800/1087852630,v1:172.21.15.204:6801/1087852630]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-08T20:51:29.724 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:51:29 smithi107 ceph-mon[31381]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-08T20:51:29.771 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:51:29 smithi149 ceph-mon[35042]: pgmap v99: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-08T20:51:29.771 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:51:29 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:51:29.771 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:51:29 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:51:29.771 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:51:29 smithi149 ceph-mon[35042]: from='osd.2 [v2:172.21.15.204:6800/1087852630,v1:172.21.15.204:6801/1087852630]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-08T20:51:29.771 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:51:29 smithi149 ceph-mon[35042]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-08T20:51:30.572 DEBUG:teuthology.orchestra.run.smithi204:osd.2> sudo journalctl -f -n 0 -u ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715@osd.2.service 2024-03-08T20:51:30.578 INFO:tasks.cephadm:Deploying osd.3 on smithi107 with /dev/vg_nvme/lv_3... 2024-03-08T20:51:30.578 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dc20f0f8-dd8c-11ee-95c6-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-03-08T20:51:30.884 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:51:30 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:51:30.885 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:51:30 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:51:30.885 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:51:30 smithi204 ceph-mon[34935]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-08T20:51:30.885 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:51:30 smithi204 ceph-mon[34935]: from='osd.2 [v2:172.21.15.204:6800/1087852630,v1:172.21.15.204:6801/1087852630]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-03-08T20:51:30.885 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:51:30 smithi204 ceph-mon[34935]: osdmap e16: 3 total, 2 up, 3 in 2024-03-08T20:51:30.885 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:51:30 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-08T20:51:30.885 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:51:30 smithi204 ceph-mon[34935]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-03-08T20:51:30.885 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:51:30 smithi204 ceph-mon[34935]: pgmap v101: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-08T20:51:30.885 INFO:journalctl@ceph.osd.2.smithi204.stdout:Mar 08 20:51:30 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-2[40216]: 2024-03-08T20:51:30.702+0000 7f1992000700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-08T20:51:30.973 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:51:30 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:51:30.973 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:51:30 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:51:30.973 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:51:30 smithi107 ceph-mon[31381]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-08T20:51:30.974 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:51:30 smithi107 ceph-mon[31381]: from='osd.2 [v2:172.21.15.204:6800/1087852630,v1:172.21.15.204:6801/1087852630]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-03-08T20:51:30.974 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:51:30 smithi107 ceph-mon[31381]: osdmap e16: 3 total, 2 up, 3 in 2024-03-08T20:51:30.974 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:51:30 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-08T20:51:30.974 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:51:30 smithi107 ceph-mon[31381]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-03-08T20:51:30.974 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:51:30 smithi107 ceph-mon[31381]: pgmap v101: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-08T20:51:31.003 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/mon.a/config 2024-03-08T20:51:31.021 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:51:30 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:51:31.021 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:51:30 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:51:31.022 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:51:30 smithi149 ceph-mon[35042]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-08T20:51:31.022 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:51:30 smithi149 ceph-mon[35042]: from='osd.2 [v2:172.21.15.204:6800/1087852630,v1:172.21.15.204:6801/1087852630]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-03-08T20:51:31.022 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:51:30 smithi149 ceph-mon[35042]: osdmap e16: 3 total, 2 up, 3 in 2024-03-08T20:51:31.022 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:51:30 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-08T20:51:31.022 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:51:30 smithi149 ceph-mon[35042]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-03-08T20:51:31.022 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:51:30 smithi149 ceph-mon[35042]: pgmap v101: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-08T20:51:31.974 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:51:31 smithi107 ceph-mon[31381]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]': finished 2024-03-08T20:51:31.974 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:51:31 smithi107 ceph-mon[31381]: osdmap e17: 3 total, 2 up, 3 in 2024-03-08T20:51:31.974 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:51:31 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-08T20:51:31.974 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:51:31 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-08T20:51:32.021 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:51:31 smithi149 ceph-mon[35042]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]': finished 2024-03-08T20:51:32.021 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:51:31 smithi149 ceph-mon[35042]: osdmap e17: 3 total, 2 up, 3 in 2024-03-08T20:51:32.021 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:51:31 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-08T20:51:32.022 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:51:31 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-08T20:51:32.084 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:51:31 smithi204 ceph-mon[34935]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]': finished 2024-03-08T20:51:32.084 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:51:31 smithi204 ceph-mon[34935]: osdmap e17: 3 total, 2 up, 3 in 2024-03-08T20:51:32.084 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:51:31 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-08T20:51:32.084 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:51:31 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-08T20:51:32.974 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:51:32 smithi107 ceph-mon[31381]: purged_snaps scrub starts 2024-03-08T20:51:32.974 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:51:32 smithi107 ceph-mon[31381]: purged_snaps scrub ok 2024-03-08T20:51:32.974 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:51:32 smithi107 ceph-mon[31381]: osd.2 [v2:172.21.15.204:6800/1087852630,v1:172.21.15.204:6801/1087852630] boot 2024-03-08T20:51:32.974 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:51:32 smithi107 ceph-mon[31381]: osdmap e18: 3 total, 3 up, 3 in 2024-03-08T20:51:32.974 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:51:32 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-08T20:51:33.135 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:51:32 smithi204 ceph-mon[34935]: purged_snaps scrub starts 2024-03-08T20:51:33.135 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:51:32 smithi204 ceph-mon[34935]: purged_snaps scrub ok 2024-03-08T20:51:33.135 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:51:32 smithi204 ceph-mon[34935]: osd.2 [v2:172.21.15.204:6800/1087852630,v1:172.21.15.204:6801/1087852630] boot 2024-03-08T20:51:33.135 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:51:32 smithi204 ceph-mon[34935]: osdmap e18: 3 total, 3 up, 3 in 2024-03-08T20:51:33.135 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:51:32 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-08T20:51:33.271 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:51:32 smithi149 ceph-mon[35042]: purged_snaps scrub starts 2024-03-08T20:51:33.272 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:51:32 smithi149 ceph-mon[35042]: purged_snaps scrub ok 2024-03-08T20:51:33.272 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:51:32 smithi149 ceph-mon[35042]: osd.2 [v2:172.21.15.204:6800/1087852630,v1:172.21.15.204:6801/1087852630] boot 2024-03-08T20:51:33.272 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:51:32 smithi149 ceph-mon[35042]: osdmap e18: 3 total, 3 up, 3 in 2024-03-08T20:51:33.272 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:51:32 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-08T20:51:33.873 INFO:teuthology.orchestra.run.smithi107.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-03-08T20:51:33.873 INFO:teuthology.orchestra.run.smithi107.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-03-08T20:51:33.873 INFO:teuthology.orchestra.run.smithi107.stderr: stderr: 10+0 records in 2024-03-08T20:51:33.873 INFO:teuthology.orchestra.run.smithi107.stderr:10+0 records out 2024-03-08T20:51:33.873 INFO:teuthology.orchestra.run.smithi107.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0415975 s, 252 MB/s 2024-03-08T20:51:33.873 INFO:teuthology.orchestra.run.smithi107.stderr:--> Zapping successful for: 2024-03-08T20:51:34.134 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:51:33 smithi204 ceph-mon[34935]: pgmap v104: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-08T20:51:34.134 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:51:33 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-03-08T20:51:34.135 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:51:33 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-03-08T20:51:34.135 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:51:33 smithi204 ceph-mon[34935]: osdmap e19: 3 total, 3 up, 3 in 2024-03-08T20:51:34.135 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:51:33 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-03-08T20:51:34.223 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:51:33 smithi107 ceph-mon[31381]: pgmap v104: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-08T20:51:34.224 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:51:33 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-03-08T20:51:34.224 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:51:33 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-03-08T20:51:34.224 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:51:33 smithi107 ceph-mon[31381]: osdmap e19: 3 total, 3 up, 3 in 2024-03-08T20:51:34.224 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:51:33 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-03-08T20:51:34.271 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:51:33 smithi149 ceph-mon[35042]: pgmap v104: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-08T20:51:34.271 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:51:33 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-03-08T20:51:34.271 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:51:33 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-03-08T20:51:34.271 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:51:33 smithi149 ceph-mon[35042]: osdmap e19: 3 total, 3 up, 3 in 2024-03-08T20:51:34.271 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:51:33 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-03-08T20:51:34.710 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dc20f0f8-dd8c-11ee-95c6-87774f69a715 -- ceph orch daemon add osd smithi107:vg_nvme/lv_3 2024-03-08T20:51:34.852 INFO:journalctl@ceph.osd.0.smithi107.stdout:Mar 08 20:51:34 smithi107 sudo[50568]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-08T20:51:34.852 INFO:journalctl@ceph.osd.0.smithi107.stdout:Mar 08 20:51:34 smithi107 sudo[50568]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-08T20:51:34.852 INFO:journalctl@ceph.osd.0.smithi107.stdout:Mar 08 20:51:34 smithi107 sudo[50568]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-08T20:51:34.852 INFO:journalctl@ceph.osd.0.smithi107.stdout:Mar 08 20:51:34 smithi107 sudo[50568]: pam_unix(sudo:session): session closed for user root 2024-03-08T20:51:35.134 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:51:34 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-03-08T20:51:35.135 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:51:34 smithi204 ceph-mon[34935]: osdmap e20: 3 total, 3 up, 3 in 2024-03-08T20:51:35.135 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:51:34 smithi204 ceph-mon[34935]: pgmap v107: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-08T20:51:35.176 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/mon.a/config 2024-03-08T20:51:35.212 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:51:34 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-03-08T20:51:35.212 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:51:34 smithi107 ceph-mon[31381]: osdmap e20: 3 total, 3 up, 3 in 2024-03-08T20:51:35.212 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:51:34 smithi107 ceph-mon[31381]: pgmap v107: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-08T20:51:35.271 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:51:34 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-03-08T20:51:35.271 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:51:34 smithi149 ceph-mon[35042]: osdmap e20: 3 total, 3 up, 3 in 2024-03-08T20:51:35.271 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:51:34 smithi149 ceph-mon[35042]: pgmap v107: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-08T20:51:35.474 INFO:journalctl@ceph.osd.0.smithi107.stdout:Mar 08 20:51:35 smithi107 sudo[50635]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-08T20:51:35.474 INFO:journalctl@ceph.osd.0.smithi107.stdout:Mar 08 20:51:35 smithi107 sudo[50635]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-08T20:51:35.474 INFO:journalctl@ceph.osd.0.smithi107.stdout:Mar 08 20:51:35 smithi107 sudo[50635]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-08T20:51:35.474 INFO:journalctl@ceph.osd.0.smithi107.stdout:Mar 08 20:51:35 smithi107 sudo[50635]: pam_unix(sudo:session): session closed for user root 2024-03-08T20:51:36.135 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:51:35 smithi204 ceph-mon[34935]: mgrmap e16: x(active, since 3m), standbys: z, y 2024-03-08T20:51:36.135 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:51:35 smithi204 ceph-mon[34935]: osdmap e21: 3 total, 3 up, 3 in 2024-03-08T20:51:36.135 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:51:35 smithi204 ceph-mon[34935]: Detected new or changed devices on smithi204 2024-03-08T20:51:36.135 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:51:35 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:51:36.135 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:51:35 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:51:36.135 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:51:35 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-08T20:51:36.135 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:51:35 smithi204 ceph-mon[34935]: Adjusting osd_memory_target on smithi204 to 17091M 2024-03-08T20:51:36.135 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:51:35 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:51:36.136 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:51:35 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:51:36.136 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:51:35 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T20:51:36.136 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:51:35 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:51:36.136 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:51:35 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:51:36.136 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:51:35 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T20:51:36.136 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:51:35 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:51:36.136 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:51:35 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T20:51:36.136 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:51:35 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:51:36.224 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:51:35 smithi107 ceph-mon[31381]: mgrmap e16: x(active, since 3m), standbys: z, y 2024-03-08T20:51:36.224 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:51:35 smithi107 ceph-mon[31381]: osdmap e21: 3 total, 3 up, 3 in 2024-03-08T20:51:36.224 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:51:35 smithi107 ceph-mon[31381]: Detected new or changed devices on smithi204 2024-03-08T20:51:36.224 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:51:35 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:51:36.224 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:51:35 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:51:36.224 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:51:35 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-08T20:51:36.224 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:51:35 smithi107 ceph-mon[31381]: Adjusting osd_memory_target on smithi204 to 17091M 2024-03-08T20:51:36.224 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:51:35 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:51:36.224 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:51:35 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:51:36.224 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:51:35 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T20:51:36.225 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:51:35 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:51:36.225 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:51:35 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:51:36.225 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:51:35 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T20:51:36.225 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:51:35 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:51:36.225 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:51:35 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T20:51:36.225 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:51:35 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:51:36.272 INFO:journalctl@ceph.osd.1.smithi149.stdout:Mar 08 20:51:35 smithi149 sudo[43546]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-08T20:51:36.272 INFO:journalctl@ceph.osd.1.smithi149.stdout:Mar 08 20:51:35 smithi149 sudo[43546]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-08T20:51:36.272 INFO:journalctl@ceph.osd.1.smithi149.stdout:Mar 08 20:51:35 smithi149 sudo[43546]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-08T20:51:36.272 INFO:journalctl@ceph.osd.1.smithi149.stdout:Mar 08 20:51:35 smithi149 sudo[43546]: pam_unix(sudo:session): session closed for user root 2024-03-08T20:51:36.272 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:51:35 smithi149 ceph-mon[35042]: mgrmap e16: x(active, since 3m), standbys: z, y 2024-03-08T20:51:36.272 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:51:35 smithi149 ceph-mon[35042]: osdmap e21: 3 total, 3 up, 3 in 2024-03-08T20:51:36.272 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:51:35 smithi149 ceph-mon[35042]: Detected new or changed devices on smithi204 2024-03-08T20:51:36.272 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:51:35 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:51:36.272 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:51:35 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:51:36.272 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:51:35 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-08T20:51:36.272 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:51:35 smithi149 ceph-mon[35042]: Adjusting osd_memory_target on smithi204 to 17091M 2024-03-08T20:51:36.273 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:51:35 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:51:36.273 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:51:35 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:51:36.273 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:51:35 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T20:51:36.273 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:51:35 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:51:36.273 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:51:35 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:51:36.273 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:51:35 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T20:51:36.273 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:51:35 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:51:36.273 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:51:35 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T20:51:36.273 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:51:35 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:51:36.771 INFO:journalctl@ceph.osd.1.smithi149.stdout:Mar 08 20:51:36 smithi149 sudo[43580]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-08T20:51:36.771 INFO:journalctl@ceph.osd.1.smithi149.stdout:Mar 08 20:51:36 smithi149 sudo[43580]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-08T20:51:36.771 INFO:journalctl@ceph.osd.1.smithi149.stdout:Mar 08 20:51:36 smithi149 sudo[43580]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-08T20:51:36.771 INFO:journalctl@ceph.osd.1.smithi149.stdout:Mar 08 20:51:36 smithi149 sudo[43580]: pam_unix(sudo:session): session closed for user root 2024-03-08T20:51:37.125 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:51:36 smithi107 ceph-mon[31381]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-08T20:51:37.125 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:51:36 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:51:37.134 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:51:36 smithi204 ceph-mon[34935]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-08T20:51:37.134 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:51:36 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:51:37.134 INFO:journalctl@ceph.osd.2.smithi204.stdout:Mar 08 20:51:37 smithi204 sudo[44244]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-08T20:51:37.134 INFO:journalctl@ceph.osd.2.smithi204.stdout:Mar 08 20:51:37 smithi204 sudo[44244]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-08T20:51:37.135 INFO:journalctl@ceph.osd.2.smithi204.stdout:Mar 08 20:51:37 smithi204 sudo[44244]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-08T20:51:37.135 INFO:journalctl@ceph.osd.2.smithi204.stdout:Mar 08 20:51:37 smithi204 sudo[44244]: pam_unix(sudo:session): session closed for user root 2024-03-08T20:51:37.271 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:51:36 smithi149 ceph-mon[35042]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-08T20:51:37.271 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:51:36 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:51:37.599 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:51:37.610+0000 7f4fe8400700 1 -- 172.21.15.107:0/1792159472 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4fe0104060 msgr2=0x7f4fe0104480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:51:37.599 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:51:37.610+0000 7f4fe8400700 1 --2- 172.21.15.107:0/1792159472 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4fe0104060 0x7f4fe0104480 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f4fd400b0d0 tx=0x7f4fd400b3e0 comp rx=0 tx=0).stop 2024-03-08T20:51:37.599 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:51:37.610+0000 7f4fe8400700 1 -- 172.21.15.107:0/1792159472 shutdown_connections 2024-03-08T20:51:37.599 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:51:37.610+0000 7f4fe8400700 1 --2- 172.21.15.107:0/1792159472 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4fe0105c00 0x7f4fe010c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:51:37.599 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:51:37.610+0000 7f4fe8400700 1 --2- 172.21.15.107:0/1792159472 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4fe0104060 0x7f4fe0104480 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:51:37.599 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:51:37.610+0000 7f4fe8400700 1 --2- 172.21.15.107:0/1792159472 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4fe0105260 0x7f4fe01056c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:51:37.600 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:51:37.610+0000 7f4fe8400700 1 -- 172.21.15.107:0/1792159472 >> 172.21.15.107:0/1792159472 conn(0x7f4fe00ff600 msgr2=0x7f4fe0101a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:51:37.600 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:51:37.610+0000 7f4fe8400700 1 -- 172.21.15.107:0/1792159472 shutdown_connections 2024-03-08T20:51:37.600 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:51:37.610+0000 7f4fe8400700 1 -- 172.21.15.107:0/1792159472 wait complete. 2024-03-08T20:51:37.601 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:51:37.611+0000 7f4fe8400700 1 Processor -- start 2024-03-08T20:51:37.601 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:51:37.611+0000 7f4fe8400700 1 -- start start 2024-03-08T20:51:37.601 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:51:37.612+0000 7f4fe8400700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4fe0104060 0x7f4fe019a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:51:37.602 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:51:37.612+0000 7f4fe8400700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4fe0105260 0x7f4fe019a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:51:37.602 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:51:37.612+0000 7f4fe8400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4fe0105c00 0x7f4fe01a1760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:51:37.602 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:51:37.612+0000 7f4fe8400700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fe010fb20 con 0x7f4fe0105c00 2024-03-08T20:51:37.602 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:51:37.612+0000 7f4fe8400700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fe010f9a0 con 0x7f4fe0105260 2024-03-08T20:51:37.602 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:51:37.612+0000 7f4fe8400700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fe010fca0 con 0x7f4fe0104060 2024-03-08T20:51:37.602 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:51:37.612+0000 7f4fe6400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4fe0105c00 0x7f4fe01a1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:51:37.602 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:51:37.612+0000 7f4fe5a00700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4fe0104060 0x7f4fe019a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:51:37.602 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:51:37.612+0000 7f4fe6400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4fe0105c00 0x7f4fe01a1760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:45420/0 (socket says 172.21.15.107:45420) 2024-03-08T20:51:37.602 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:51:37.612+0000 7f4fe6400700 1 -- 172.21.15.107:0/2096380796 learned_addr learned my addr 172.21.15.107:0/2096380796 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T20:51:37.602 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:51:37.612+0000 7f4fe5000700 1 --2- 172.21.15.107:0/2096380796 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4fe0105260 0x7f4fe019a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:51:37.602 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:51:37.612+0000 7f4fe5a00700 1 -- 172.21.15.107:0/2096380796 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4fe0105260 msgr2=0x7f4fe019a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:51:37.602 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:51:37.612+0000 7f4fe5a00700 1 --2- 172.21.15.107:0/2096380796 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4fe0105260 0x7f4fe019a6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:51:37.602 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:51:37.612+0000 7f4fe5a00700 1 -- 172.21.15.107:0/2096380796 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4fe0105c00 msgr2=0x7f4fe01a1760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:51:37.602 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:51:37.612+0000 7f4fe5a00700 1 --2- 172.21.15.107:0/2096380796 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4fe0105c00 0x7f4fe01a1760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:51:37.603 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:51:37.612+0000 7f4fe5a00700 1 -- 172.21.15.107:0/2096380796 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fe01a1e00 con 0x7f4fe0104060 2024-03-08T20:51:37.603 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:51:37.613+0000 7f4fe5a00700 1 --2- 172.21.15.107:0/2096380796 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4fe0104060 0x7f4fe019a1a0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f4fd400b0d0 tx=0x7f4fd4004d00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:51:37.603 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:51:37.613+0000 7f4fde000700 1 -- 172.21.15.107:0/2096380796 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fd401f030 con 0x7f4fe0104060 2024-03-08T20:51:37.603 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:51:37.613+0000 7f4fe8400700 1 -- 172.21.15.107:0/2096380796 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fe01a2090 con 0x7f4fe0104060 2024-03-08T20:51:37.603 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:51:37.613+0000 7f4fe8400700 1 -- 172.21.15.107:0/2096380796 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fe01a2540 con 0x7f4fe0104060 2024-03-08T20:51:37.603 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:51:37.613+0000 7f4fde000700 1 -- 172.21.15.107:0/2096380796 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f4fd4005040 con 0x7f4fe0104060 2024-03-08T20:51:37.604 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:51:37.613+0000 7f4fde000700 1 -- 172.21.15.107:0/2096380796 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fd401b430 con 0x7f4fe0104060 2024-03-08T20:51:37.604 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:51:37.614+0000 7f4fe8400700 1 -- 172.21.15.107:0/2096380796 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fe0068aa0 con 0x7f4fe0104060 2024-03-08T20:51:37.606 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:51:37.617+0000 7f4fde000700 1 -- 172.21.15.107:0/2096380796 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 16) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f4fd4022030 con 0x7f4fe0104060 2024-03-08T20:51:37.607 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:51:37.618+0000 7f4fde000700 1 --2- 172.21.15.107:0/2096380796 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f4fc80a2910 0x7f4fc80a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:51:37.607 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:51:37.619+0000 7f4fde000700 1 -- 172.21.15.107:0/2096380796 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 3237+0+0 (secure 0 0 0) 0x7f4fd40cdfc0 con 0x7f4fe0104060 2024-03-08T20:51:37.607 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:51:37.619+0000 7f4fe5000700 1 --2- 172.21.15.107:0/2096380796 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f4fc80a2910 0x7f4fc80a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:51:37.608 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:51:37.619+0000 7f4fe5000700 1 --2- 172.21.15.107:0/2096380796 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f4fc80a2910 0x7f4fc80a4dd0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f4fcc005950 tx=0x7f4fcc0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:51:37.623 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:51:37.634+0000 7f4fde000700 1 -- 172.21.15.107:0/2096380796 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f4fd4029030 con 0x7f4fe0104060 2024-03-08T20:51:37.864 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:51:37.875+0000 7f4fe8400700 1 -- 172.21.15.107:0/2096380796 --> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f4fe0062220 con 0x7f4fc80a2910 2024-03-08T20:51:37.884 INFO:journalctl@ceph.osd.2.smithi204.stdout:Mar 08 20:51:37 smithi204 sudo[44247]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-08T20:51:37.884 INFO:journalctl@ceph.osd.2.smithi204.stdout:Mar 08 20:51:37 smithi204 sudo[44247]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-08T20:51:37.884 INFO:journalctl@ceph.osd.2.smithi204.stdout:Mar 08 20:51:37 smithi204 sudo[44247]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-08T20:51:37.885 INFO:journalctl@ceph.osd.2.smithi204.stdout:Mar 08 20:51:37 smithi204 sudo[44247]: pam_unix(sudo:session): session closed for user root 2024-03-08T20:51:38.473 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:51:38 smithi107 sudo[50779]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-08T20:51:38.473 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:51:38 smithi107 sudo[50779]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-08T20:51:38.473 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:51:38 smithi107 sudo[50779]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-08T20:51:38.474 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:51:38 smithi107 sudo[50779]: pam_unix(sudo:session): session closed for user root 2024-03-08T20:51:39.021 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:51:38 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:51:39.021 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:51:38 smithi149 ceph-mon[35042]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-08T20:51:39.134 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:51:38 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:51:39.134 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:51:38 smithi204 ceph-mon[34935]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-08T20:51:39.223 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:51:38 smithi107 sudo[50782]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-08T20:51:39.223 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:51:38 smithi107 sudo[50782]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-08T20:51:39.224 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:51:38 smithi107 sudo[50782]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-08T20:51:39.224 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:51:38 smithi107 sudo[50782]: pam_unix(sudo:session): session closed for user root 2024-03-08T20:51:39.224 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:51:38 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:51:39.224 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:51:38 smithi107 ceph-mon[31381]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-08T20:51:39.634 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:51:39 smithi204 sudo[44250]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-08T20:51:39.634 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:51:39 smithi204 sudo[44250]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-08T20:51:39.634 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:51:39 smithi204 sudo[44250]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-08T20:51:39.634 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:51:39 smithi204 sudo[44250]: pam_unix(sudo:session): session closed for user root 2024-03-08T20:51:39.971 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:51:39 smithi107 ceph-mon[31381]: from='client.24202 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-08T20:51:39.971 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:51:39 smithi107 ceph-mon[31381]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-08T20:51:39.971 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:51:39 smithi107 ceph-mon[31381]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-08T20:51:39.971 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:51:39 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-08T20:51:39.971 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:51:39 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-08T20:51:39.971 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:51:39 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-08T20:51:39.971 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:51:39 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-08T20:51:39.971 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:51:39 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-08T20:51:39.971 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:51:39 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-08T20:51:39.971 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:51:39 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-08T20:51:39.971 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:51:39 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-08T20:51:39.971 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:51:39 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:51:40.135 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:51:39 smithi204 sudo[44253]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-08T20:51:40.135 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:51:39 smithi204 sudo[44253]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-08T20:51:40.135 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:51:39 smithi204 sudo[44253]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-08T20:51:40.135 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:51:39 smithi204 sudo[44253]: pam_unix(sudo:session): session closed for user root 2024-03-08T20:51:40.135 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:51:39 smithi204 ceph-mon[34935]: from='client.24202 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-08T20:51:40.135 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:51:39 smithi204 ceph-mon[34935]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-08T20:51:40.135 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:51:39 smithi204 ceph-mon[34935]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-08T20:51:40.136 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:51:39 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-08T20:51:40.136 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:51:39 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-08T20:51:40.136 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:51:39 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-08T20:51:40.136 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:51:39 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-08T20:51:40.136 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:51:39 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-08T20:51:40.136 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:51:39 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-08T20:51:40.136 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:51:39 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-08T20:51:40.136 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:51:39 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-08T20:51:40.136 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:51:39 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:51:40.771 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:51:40 smithi149 sudo[43583]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-08T20:51:40.771 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:51:40 smithi149 sudo[43583]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-08T20:51:40.771 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:51:40 smithi149 sudo[43583]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-08T20:51:40.771 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:51:40 smithi149 sudo[43583]: pam_unix(sudo:session): session closed for user root 2024-03-08T20:51:41.174 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:51:40 smithi107 ceph-mon[31381]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-08T20:51:41.174 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:51:40 smithi107 ceph-mon[31381]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-08T20:51:41.174 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:51:40 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-08T20:51:41.174 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:51:40 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-08T20:51:41.174 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:51:40 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-08T20:51:41.174 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:51:40 smithi107 ceph-mon[31381]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-08T20:51:41.272 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:51:40 smithi149 sudo[43586]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-08T20:51:41.272 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:51:40 smithi149 sudo[43586]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-08T20:51:41.272 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:51:40 smithi149 sudo[43586]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-08T20:51:41.272 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:51:40 smithi149 sudo[43586]: pam_unix(sudo:session): session closed for user root 2024-03-08T20:51:41.272 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:51:40 smithi149 ceph-mon[35042]: from='client.24202 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-08T20:51:41.273 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:51:40 smithi149 ceph-mon[35042]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-08T20:51:41.273 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:51:40 smithi149 ceph-mon[35042]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-08T20:51:41.273 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:51:40 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-08T20:51:41.273 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:51:40 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-08T20:51:41.273 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:51:40 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-08T20:51:41.273 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:51:40 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-08T20:51:41.273 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:51:40 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-08T20:51:41.273 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:51:40 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-08T20:51:41.273 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:51:40 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-08T20:51:41.273 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:51:40 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-08T20:51:41.273 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:51:40 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:51:41.273 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:51:40 smithi149 ceph-mon[35042]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-08T20:51:41.273 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:51:40 smithi149 ceph-mon[35042]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-08T20:51:41.273 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:51:40 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-08T20:51:41.274 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:51:40 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-08T20:51:41.274 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:51:40 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-08T20:51:41.274 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:51:40 smithi149 ceph-mon[35042]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-08T20:51:41.384 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:51:40 smithi204 ceph-mon[34935]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-08T20:51:41.384 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:51:40 smithi204 ceph-mon[34935]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-08T20:51:41.384 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:51:40 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-08T20:51:41.385 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:51:40 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-08T20:51:41.385 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:51:40 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-08T20:51:41.385 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:51:40 smithi204 ceph-mon[34935]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-08T20:51:42.223 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:51:41 smithi107 ceph-mon[31381]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-08T20:51:42.224 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:51:41 smithi107 ceph-mon[31381]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-08T20:51:42.271 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:51:41 smithi149 ceph-mon[35042]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-08T20:51:42.271 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:51:41 smithi149 ceph-mon[35042]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-08T20:51:42.384 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:51:41 smithi204 ceph-mon[34935]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-08T20:51:42.384 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:51:41 smithi204 ceph-mon[34935]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-08T20:51:43.271 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:51:42 smithi149 ceph-mon[35042]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 3.3 KiB/s rd, 174 KiB/s wr, 10 op/s 2024-03-08T20:51:43.271 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:51:42 smithi149 ceph-mon[35042]: from='client.? 172.21.15.107:0/368956258' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9e433d97-e202-48e0-9ed1-f0988fa14307"}]: dispatch 2024-03-08T20:51:43.271 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:51:42 smithi149 ceph-mon[35042]: from='client.? 172.21.15.107:0/368956258' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9e433d97-e202-48e0-9ed1-f0988fa14307"}]': finished 2024-03-08T20:51:43.272 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:51:42 smithi149 ceph-mon[35042]: osdmap e22: 4 total, 3 up, 4 in 2024-03-08T20:51:43.272 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:51:42 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-08T20:51:43.384 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:51:42 smithi204 ceph-mon[34935]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 3.3 KiB/s rd, 174 KiB/s wr, 10 op/s 2024-03-08T20:51:43.384 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:51:42 smithi204 ceph-mon[34935]: from='client.? 172.21.15.107:0/368956258' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9e433d97-e202-48e0-9ed1-f0988fa14307"}]: dispatch 2024-03-08T20:51:43.385 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:51:42 smithi204 ceph-mon[34935]: from='client.? 172.21.15.107:0/368956258' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9e433d97-e202-48e0-9ed1-f0988fa14307"}]': finished 2024-03-08T20:51:43.385 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:51:42 smithi204 ceph-mon[34935]: osdmap e22: 4 total, 3 up, 4 in 2024-03-08T20:51:43.385 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:51:42 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-08T20:51:43.396 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:51:42 smithi107 ceph-mon[31381]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 3.3 KiB/s rd, 174 KiB/s wr, 10 op/s 2024-03-08T20:51:43.396 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:51:42 smithi107 ceph-mon[31381]: from='client.? 172.21.15.107:0/368956258' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9e433d97-e202-48e0-9ed1-f0988fa14307"}]: dispatch 2024-03-08T20:51:43.396 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:51:42 smithi107 ceph-mon[31381]: from='client.? 172.21.15.107:0/368956258' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9e433d97-e202-48e0-9ed1-f0988fa14307"}]': finished 2024-03-08T20:51:43.396 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:51:42 smithi107 ceph-mon[31381]: osdmap e22: 4 total, 3 up, 4 in 2024-03-08T20:51:43.396 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:51:42 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-08T20:51:44.271 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:51:43 smithi149 ceph-mon[35042]: from='client.? 172.21.15.107:0/4212551565' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-08T20:51:44.384 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:51:43 smithi204 ceph-mon[34935]: from='client.? 172.21.15.107:0/4212551565' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-08T20:51:44.473 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:51:43 smithi107 ceph-mon[31381]: from='client.? 172.21.15.107:0/4212551565' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-08T20:51:45.271 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:51:45 smithi149 ceph-mon[35042]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 2.5 KiB/s rd, 130 KiB/s wr, 8 op/s 2024-03-08T20:51:45.384 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:51:45 smithi204 ceph-mon[34935]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 2.5 KiB/s rd, 130 KiB/s wr, 8 op/s 2024-03-08T20:51:45.473 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:51:45 smithi107 ceph-mon[31381]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 2.5 KiB/s rd, 130 KiB/s wr, 8 op/s 2024-03-08T20:51:47.521 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:51:47 smithi149 ceph-mon[35042]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 3.0 KiB/s rd, 131 KiB/s wr, 9 op/s 2024-03-08T20:51:47.634 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:51:47 smithi204 ceph-mon[34935]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 3.0 KiB/s rd, 131 KiB/s wr, 9 op/s 2024-03-08T20:51:47.723 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:51:47 smithi107 ceph-mon[31381]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 3.0 KiB/s rd, 131 KiB/s wr, 9 op/s 2024-03-08T20:51:49.521 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:51:49 smithi149 ceph-mon[35042]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 3.0 KiB/s rd, 131 KiB/s wr, 9 op/s 2024-03-08T20:51:49.634 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:51:49 smithi204 ceph-mon[34935]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 3.0 KiB/s rd, 131 KiB/s wr, 9 op/s 2024-03-08T20:51:49.676 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:51:49 smithi107 ceph-mon[31381]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 3.0 KiB/s rd, 131 KiB/s wr, 9 op/s 2024-03-08T20:51:51.441 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:51:51 smithi107 ceph-mon[31381]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 3.0 KiB/s rd, 131 KiB/s wr, 9 op/s 2024-03-08T20:51:51.521 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:51:51 smithi149 ceph-mon[35042]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 3.0 KiB/s rd, 131 KiB/s wr, 9 op/s 2024-03-08T20:51:51.634 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:51:51 smithi204 ceph-mon[34935]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 3.0 KiB/s rd, 131 KiB/s wr, 9 op/s 2024-03-08T20:51:52.634 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:51:52 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-08T20:51:52.634 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:51:52 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:51:52.723 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:51:52 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-08T20:51:52.723 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:51:52 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:51:52.771 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:51:52 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-08T20:51:52.771 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:51:52 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:51:53.634 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:51:53 smithi204 ceph-mon[34935]: Deploying daemon osd.3 on smithi107 2024-03-08T20:51:53.634 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:51:53 smithi204 ceph-mon[34935]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 1023 B/s rd, 26 KiB/s wr, 2 op/s 2024-03-08T20:51:53.723 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:51:53 smithi107 ceph-mon[31381]: Deploying daemon osd.3 on smithi107 2024-03-08T20:51:53.723 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:51:53 smithi107 ceph-mon[31381]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 1023 B/s rd, 26 KiB/s wr, 2 op/s 2024-03-08T20:51:53.771 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:51:53 smithi149 ceph-mon[35042]: Deploying daemon osd.3 on smithi107 2024-03-08T20:51:53.771 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:51:53 smithi149 ceph-mon[35042]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 1023 B/s rd, 26 KiB/s wr, 2 op/s 2024-03-08T20:51:55.428 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:51:55 smithi107 ceph-mon[31381]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 899 B/s rd, 23 KiB/s wr, 2 op/s 2024-03-08T20:51:55.634 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:51:55 smithi204 ceph-mon[34935]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 899 B/s rd, 23 KiB/s wr, 2 op/s 2024-03-08T20:51:55.771 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:51:55 smithi149 ceph-mon[35042]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 899 B/s rd, 23 KiB/s wr, 2 op/s 2024-03-08T20:51:57.473 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:51:57 smithi107 ceph-mon[31381]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-03-08T20:51:57.634 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:51:57 smithi204 ceph-mon[34935]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-03-08T20:51:57.770 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:51:57 smithi149 ceph-mon[35042]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-03-08T20:51:59.473 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:51:59 smithi107 ceph-mon[31381]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-08T20:51:59.634 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:51:59 smithi204 ceph-mon[34935]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-08T20:51:59.770 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:51:59 smithi149 ceph-mon[35042]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-08T20:52:01.634 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:52:01 smithi204 ceph-mon[34935]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-08T20:52:01.634 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:52:01 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:52:01.634 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:52:01 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:52:01.634 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:52:01 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T20:52:01.645 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:52:01 smithi107 ceph-mon[31381]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-08T20:52:01.645 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:52:01 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:52:01.645 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:52:01 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:52:01.645 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:52:01 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T20:52:01.770 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:52:01 smithi149 ceph-mon[35042]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-08T20:52:01.771 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:52:01 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:52:01.771 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:52:01 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:52:01.771 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:52:01 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T20:52:03.473 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:52:03 smithi107 ceph-mon[31381]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-08T20:52:03.634 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:52:03 smithi204 ceph-mon[34935]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-08T20:52:03.770 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:52:03 smithi149 ceph-mon[35042]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-08T20:52:05.251 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:52:05 smithi107 ceph-mon[31381]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-08T20:52:05.384 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:52:05 smithi204 ceph-mon[34935]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-08T20:52:05.385 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:52:05 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:52:05.385 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:52:05 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:52:05.385 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:52:05 smithi204 ceph-mon[34935]: from='osd.3 [v2:172.21.15.107:6810/3123579789,v1:172.21.15.107:6811/3123579789]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-08T20:52:05.521 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:52:05 smithi149 ceph-mon[35042]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-08T20:52:05.521 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:52:05 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:52:05.521 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:52:05 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:52:05.521 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:52:05 smithi149 ceph-mon[35042]: from='osd.3 [v2:172.21.15.107:6810/3123579789,v1:172.21.15.107:6811/3123579789]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-08T20:52:05.569 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:52:05 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:52:05.569 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:52:05 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:52:05.569 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:52:05 smithi107 ceph-mon[31381]: from='osd.3 [v2:172.21.15.107:6810/3123579789,v1:172.21.15.107:6811/3123579789]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-08T20:52:05.569 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:52:05.578+0000 7f4fde000700 1 -- 172.21.15.107:0/2096380796 <== mgr.14152 v2:172.21.15.107:6800/3376636618 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f4fe0062220 con 0x7f4fc80a2910 2024-03-08T20:52:05.570 INFO:teuthology.orchestra.run.smithi107.stdout:Created osd(s) 3 on host 'smithi107' 2024-03-08T20:52:05.572 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:52:05.582+0000 7f4fe8400700 1 -- 172.21.15.107:0/2096380796 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f4fc80a2910 msgr2=0x7f4fc80a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:52:05.572 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:52:05.582+0000 7f4fe8400700 1 --2- 172.21.15.107:0/2096380796 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f4fc80a2910 0x7f4fc80a4dd0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f4fcc005950 tx=0x7f4fcc0058e0 comp rx=0 tx=0).stop 2024-03-08T20:52:05.572 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:52:05.582+0000 7f4fe8400700 1 -- 172.21.15.107:0/2096380796 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4fe0104060 msgr2=0x7f4fe019a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:52:05.572 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:52:05.582+0000 7f4fe8400700 1 --2- 172.21.15.107:0/2096380796 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4fe0104060 0x7f4fe019a1a0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f4fd400b0d0 tx=0x7f4fd4004d00 comp rx=0 tx=0).stop 2024-03-08T20:52:05.572 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:52:05.582+0000 7f4fe8400700 1 -- 172.21.15.107:0/2096380796 shutdown_connections 2024-03-08T20:52:05.572 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:52:05.582+0000 7f4fe8400700 1 --2- 172.21.15.107:0/2096380796 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4fe0104060 0x7f4fe019a1a0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:52:05.572 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:52:05.582+0000 7f4fe8400700 1 --2- 172.21.15.107:0/2096380796 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f4fc80a2910 0x7f4fc80a4dd0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:52:05.572 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:52:05.582+0000 7f4fe8400700 1 --2- 172.21.15.107:0/2096380796 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4fe0105c00 0x7f4fe01a1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:52:05.573 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:52:05.582+0000 7f4fe8400700 1 --2- 172.21.15.107:0/2096380796 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4fe0105260 0x7f4fe019a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:52:05.573 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:52:05.582+0000 7f4fe8400700 1 -- 172.21.15.107:0/2096380796 >> 172.21.15.107:0/2096380796 conn(0x7f4fe00ff600 msgr2=0x7f4fe01085d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:52:05.573 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:52:05.582+0000 7f4fe8400700 1 -- 172.21.15.107:0/2096380796 shutdown_connections 2024-03-08T20:52:05.573 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:52:05.582+0000 7f4fe8400700 1 -- 172.21.15.107:0/2096380796 wait complete. 2024-03-08T20:52:06.286 DEBUG:teuthology.orchestra.run.smithi107:osd.3> sudo journalctl -f -n 0 -u ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715@osd.3.service 2024-03-08T20:52:06.292 INFO:tasks.cephadm:Deploying osd.4 on smithi149 with /dev/vg_nvme/lv_3... 2024-03-08T20:52:06.292 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dc20f0f8-dd8c-11ee-95c6-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-03-08T20:52:06.473 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:52:06 smithi107 ceph-mon[31381]: from='osd.3 [v2:172.21.15.107:6810/3123579789,v1:172.21.15.107:6811/3123579789]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-08T20:52:06.473 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:52:06 smithi107 ceph-mon[31381]: osdmap e23: 4 total, 3 up, 4 in 2024-03-08T20:52:06.473 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:52:06 smithi107 ceph-mon[31381]: from='osd.3 [v2:172.21.15.107:6810/3123579789,v1:172.21.15.107:6811/3123579789]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-03-08T20:52:06.473 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:52:06 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-08T20:52:06.473 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:52:06 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:52:06.473 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:52:06 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T20:52:06.473 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:52:06 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:52:06.473 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:52:06 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:52:06.473 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:52:06 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:52:06.474 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:52:06 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T20:52:06.634 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:52:06 smithi204 ceph-mon[34935]: from='osd.3 [v2:172.21.15.107:6810/3123579789,v1:172.21.15.107:6811/3123579789]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-08T20:52:06.634 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:52:06 smithi204 ceph-mon[34935]: osdmap e23: 4 total, 3 up, 4 in 2024-03-08T20:52:06.634 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:52:06 smithi204 ceph-mon[34935]: from='osd.3 [v2:172.21.15.107:6810/3123579789,v1:172.21.15.107:6811/3123579789]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-03-08T20:52:06.635 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:52:06 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-08T20:52:06.635 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:52:06 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:52:06.635 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:52:06 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T20:52:06.635 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:52:06 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:52:06.635 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:52:06 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:52:06.635 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:52:06 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:52:06.635 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:52:06 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T20:52:06.727 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/mon.b/config 2024-03-08T20:52:06.770 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:52:06 smithi149 ceph-mon[35042]: from='osd.3 [v2:172.21.15.107:6810/3123579789,v1:172.21.15.107:6811/3123579789]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-08T20:52:06.770 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:52:06 smithi149 ceph-mon[35042]: osdmap e23: 4 total, 3 up, 4 in 2024-03-08T20:52:06.770 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:52:06 smithi149 ceph-mon[35042]: from='osd.3 [v2:172.21.15.107:6810/3123579789,v1:172.21.15.107:6811/3123579789]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-03-08T20:52:06.771 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:52:06 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-08T20:52:06.771 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:52:06 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:52:06.771 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:52:06 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T20:52:06.771 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:52:06 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:52:06.771 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:52:06 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:52:06.771 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:52:06 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:52:06.771 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:52:06 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T20:52:07.074 INFO:journalctl@ceph.osd.3.smithi107.stdout:Mar 08 20:52:06 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-3[53018]: 2024-03-08T20:52:06.901+0000 7fd3fb200700 -1 osd.3 0 waiting for initial osdmap 2024-03-08T20:52:07.074 INFO:journalctl@ceph.osd.3.smithi107.stdout:Mar 08 20:52:06 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-3[53018]: 2024-03-08T20:52:06.916+0000 7fd3f1000700 -1 osd.3 24 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-08T20:52:07.521 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:52:07 smithi149 ceph-mon[35042]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-08T20:52:07.521 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:52:07 smithi149 ceph-mon[35042]: from='osd.3 [v2:172.21.15.107:6810/3123579789,v1:172.21.15.107:6811/3123579789]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]': finished 2024-03-08T20:52:07.521 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:52:07 smithi149 ceph-mon[35042]: osdmap e24: 4 total, 3 up, 4 in 2024-03-08T20:52:07.521 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:52:07 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-08T20:52:07.521 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:52:07 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-08T20:52:07.521 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:52:07 smithi149 ceph-mon[35042]: from='osd.3 [v2:172.21.15.107:6810/3123579789,v1:172.21.15.107:6811/3123579789]' entity='osd.3' 2024-03-08T20:52:07.634 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:52:07 smithi204 ceph-mon[34935]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-08T20:52:07.634 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:52:07 smithi204 ceph-mon[34935]: from='osd.3 [v2:172.21.15.107:6810/3123579789,v1:172.21.15.107:6811/3123579789]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]': finished 2024-03-08T20:52:07.634 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:52:07 smithi204 ceph-mon[34935]: osdmap e24: 4 total, 3 up, 4 in 2024-03-08T20:52:07.634 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:52:07 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-08T20:52:07.634 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:52:07 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-08T20:52:07.634 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:52:07 smithi204 ceph-mon[34935]: from='osd.3 [v2:172.21.15.107:6810/3123579789,v1:172.21.15.107:6811/3123579789]' entity='osd.3' 2024-03-08T20:52:07.640 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:52:07 smithi107 ceph-mon[31381]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-08T20:52:07.640 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:52:07 smithi107 ceph-mon[31381]: from='osd.3 [v2:172.21.15.107:6810/3123579789,v1:172.21.15.107:6811/3123579789]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]': finished 2024-03-08T20:52:07.640 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:52:07 smithi107 ceph-mon[31381]: osdmap e24: 4 total, 3 up, 4 in 2024-03-08T20:52:07.640 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:52:07 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-08T20:52:07.640 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:52:07 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-08T20:52:07.640 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:52:07 smithi107 ceph-mon[31381]: from='osd.3 [v2:172.21.15.107:6810/3123579789,v1:172.21.15.107:6811/3123579789]' entity='osd.3' 2024-03-08T20:52:08.473 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:52:08 smithi107 ceph-mon[31381]: purged_snaps scrub starts 2024-03-08T20:52:08.473 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:52:08 smithi107 ceph-mon[31381]: purged_snaps scrub ok 2024-03-08T20:52:08.474 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:52:08 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-08T20:52:08.474 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:52:08 smithi107 ceph-mon[31381]: osd.3 [v2:172.21.15.107:6810/3123579789,v1:172.21.15.107:6811/3123579789] boot 2024-03-08T20:52:08.474 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:52:08 smithi107 ceph-mon[31381]: osdmap e25: 4 total, 4 up, 4 in 2024-03-08T20:52:08.474 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:52:08 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-08T20:52:08.607 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:52:08 smithi149 ceph-mon[35042]: purged_snaps scrub starts 2024-03-08T20:52:08.607 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:52:08 smithi149 ceph-mon[35042]: purged_snaps scrub ok 2024-03-08T20:52:08.607 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:52:08 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-08T20:52:08.608 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:52:08 smithi149 ceph-mon[35042]: osd.3 [v2:172.21.15.107:6810/3123579789,v1:172.21.15.107:6811/3123579789] boot 2024-03-08T20:52:08.608 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:52:08 smithi149 ceph-mon[35042]: osdmap e25: 4 total, 4 up, 4 in 2024-03-08T20:52:08.608 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:52:08 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-08T20:52:08.634 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:52:08 smithi204 ceph-mon[34935]: purged_snaps scrub starts 2024-03-08T20:52:08.634 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:52:08 smithi204 ceph-mon[34935]: purged_snaps scrub ok 2024-03-08T20:52:08.634 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:52:08 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-08T20:52:08.634 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:52:08 smithi204 ceph-mon[34935]: osd.3 [v2:172.21.15.107:6810/3123579789,v1:172.21.15.107:6811/3123579789] boot 2024-03-08T20:52:08.634 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:52:08 smithi204 ceph-mon[34935]: osdmap e25: 4 total, 4 up, 4 in 2024-03-08T20:52:08.634 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:52:08 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-08T20:52:09.571 INFO:teuthology.orchestra.run.smithi149.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-03-08T20:52:09.572 INFO:teuthology.orchestra.run.smithi149.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-03-08T20:52:09.572 INFO:teuthology.orchestra.run.smithi149.stderr: stderr: 10+0 records in 2024-03-08T20:52:09.572 INFO:teuthology.orchestra.run.smithi149.stderr:10+0 records out 2024-03-08T20:52:09.572 INFO:teuthology.orchestra.run.smithi149.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0351325 s, 298 MB/s 2024-03-08T20:52:09.572 INFO:teuthology.orchestra.run.smithi149.stderr:--> Zapping successful for: 2024-03-08T20:52:09.634 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:52:09 smithi204 ceph-mon[34935]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-08T20:52:09.634 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:52:09 smithi204 ceph-mon[34935]: osdmap e26: 4 total, 4 up, 4 in 2024-03-08T20:52:09.636 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:52:09 smithi149 ceph-mon[35042]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-08T20:52:09.636 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:52:09 smithi149 ceph-mon[35042]: osdmap e26: 4 total, 4 up, 4 in 2024-03-08T20:52:09.725 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:52:09 smithi107 ceph-mon[31381]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-08T20:52:09.725 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:52:09 smithi107 ceph-mon[31381]: osdmap e26: 4 total, 4 up, 4 in 2024-03-08T20:52:10.334 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dc20f0f8-dd8c-11ee-95c6-87774f69a715 -- ceph orch daemon add osd smithi149:vg_nvme/lv_3 2024-03-08T20:52:10.633 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:52:10 smithi204 ceph-mon[34935]: osdmap e27: 4 total, 4 up, 4 in 2024-03-08T20:52:10.723 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:52:10 smithi107 ceph-mon[31381]: osdmap e27: 4 total, 4 up, 4 in 2024-03-08T20:52:10.759 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/mon.b/config 2024-03-08T20:52:10.770 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:52:10 smithi149 ceph-mon[35042]: osdmap e27: 4 total, 4 up, 4 in 2024-03-08T20:52:11.473 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:52:11 smithi107 ceph-mon[31381]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-08T20:52:11.473 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:52:11 smithi107 ceph-mon[31381]: osdmap e28: 4 total, 4 up, 4 in 2024-03-08T20:52:11.473 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:52:11 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:52:11.473 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:52:11 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:52:11.473 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:52:11 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-08T20:52:11.474 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:52:11 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-08T20:52:11.474 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:52:11 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:52:11.474 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:52:11 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:52:11.474 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:52:11 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T20:52:11.474 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:52:11 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:52:11.520 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:52:11 smithi149 ceph-mon[35042]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-08T20:52:11.520 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:52:11 smithi149 ceph-mon[35042]: osdmap e28: 4 total, 4 up, 4 in 2024-03-08T20:52:11.520 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:52:11 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:52:11.520 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:52:11 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:52:11.521 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:52:11 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-08T20:52:11.521 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:52:11 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-08T20:52:11.521 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:52:11 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:52:11.521 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:52:11 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:52:11.521 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:52:11 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T20:52:11.521 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:52:11 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:52:11.633 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:52:11 smithi204 ceph-mon[34935]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-08T20:52:11.634 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:52:11 smithi204 ceph-mon[34935]: osdmap e28: 4 total, 4 up, 4 in 2024-03-08T20:52:11.634 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:52:11 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:52:11.634 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:52:11 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:52:11.634 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:52:11 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-08T20:52:11.634 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:52:11 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-08T20:52:11.634 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:52:11 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:52:11.634 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:52:11 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:52:11.634 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:52:11 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T20:52:11.634 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:52:11 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:52:12.643 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:52:12 smithi149 ceph-mon[35042]: Detected new or changed devices on smithi107 2024-03-08T20:52:12.643 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:52:12 smithi149 ceph-mon[35042]: Adjusting osd_memory_target on smithi107 to 8545M 2024-03-08T20:52:12.643 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:52:12 smithi149 ceph-mon[35042]: osdmap e29: 4 total, 4 up, 4 in 2024-03-08T20:52:12.723 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:52:12 smithi107 ceph-mon[31381]: Detected new or changed devices on smithi107 2024-03-08T20:52:12.723 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:52:12 smithi107 ceph-mon[31381]: Adjusting osd_memory_target on smithi107 to 8545M 2024-03-08T20:52:12.723 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:52:12 smithi107 ceph-mon[31381]: osdmap e29: 4 total, 4 up, 4 in 2024-03-08T20:52:12.884 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:52:12 smithi204 ceph-mon[34935]: Detected new or changed devices on smithi107 2024-03-08T20:52:12.884 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:52:12 smithi204 ceph-mon[34935]: Adjusting osd_memory_target on smithi107 to 8545M 2024-03-08T20:52:12.884 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:52:12 smithi204 ceph-mon[34935]: osdmap e29: 4 total, 4 up, 4 in 2024-03-08T20:52:12.925 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:52:12.936+0000 7f9329400700 1 -- 172.21.15.149:0/3711059940 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f932410a000 msgr2=0x7f93241108b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:52:12.925 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:52:12.936+0000 7f9329400700 1 --2- 172.21.15.149:0/3711059940 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f932410a000 0x7f93241108b0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f931c0099c0 tx=0x7f931c009cd0 comp rx=0 tx=0).stop 2024-03-08T20:52:12.925 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:52:12.936+0000 7f9329400700 1 -- 172.21.15.149:0/3711059940 shutdown_connections 2024-03-08T20:52:12.925 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:52:12.936+0000 7f9329400700 1 --2- 172.21.15.149:0/3711059940 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f932410a000 0x7f93241108b0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:52:12.925 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:52:12.936+0000 7f9329400700 1 --2- 172.21.15.149:0/3711059940 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9324108460 0x7f9324108880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:52:12.925 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:52:12.936+0000 7f9329400700 1 --2- 172.21.15.149:0/3711059940 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9324109660 0x7f9324109ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:52:12.925 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:52:12.936+0000 7f9329400700 1 -- 172.21.15.149:0/3711059940 >> 172.21.15.149:0/3711059940 conn(0x7f9324077430 msgr2=0x7f9324079870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:52:12.925 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:52:12.936+0000 7f9329400700 1 -- 172.21.15.149:0/3711059940 shutdown_connections 2024-03-08T20:52:12.925 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:52:12.936+0000 7f9329400700 1 -- 172.21.15.149:0/3711059940 wait complete. 2024-03-08T20:52:12.926 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:52:12.937+0000 7f9329400700 1 Processor -- start 2024-03-08T20:52:12.927 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:52:12.937+0000 7f9329400700 1 -- start start 2024-03-08T20:52:12.927 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:52:12.938+0000 7f9329400700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9324108460 0x7f932419e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:52:12.927 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:52:12.938+0000 7f9329400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9324109660 0x7f932419e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:52:12.927 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:52:12.938+0000 7f9329400700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f932410a000 0x7f93241a59d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:52:12.927 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:52:12.938+0000 7f9329400700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9324113ee0 con 0x7f9324109660 2024-03-08T20:52:12.927 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:52:12.938+0000 7f9329400700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9324113d60 con 0x7f932410a000 2024-03-08T20:52:12.927 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:52:12.938+0000 7f9329400700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9324114060 con 0x7f9324108460 2024-03-08T20:52:12.927 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:52:12.938+0000 7f9323200700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f932410a000 0x7f93241a59d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:52:12.927 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:52:12.938+0000 7f9323200700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f932410a000 0x7f93241a59d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.149:39368/0 (socket says 172.21.15.149:39368) 2024-03-08T20:52:12.927 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:52:12.938+0000 7f9323200700 1 -- 172.21.15.149:0/2390711744 learned_addr learned my addr 172.21.15.149:0/2390711744 (peer_addr_for_me v2:172.21.15.149:0/0) 2024-03-08T20:52:12.928 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:52:12.938+0000 7f9322800700 1 --2- 172.21.15.149:0/2390711744 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9324108460 0x7f932419e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:52:12.928 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:52:12.938+0000 7f9319e00700 1 --2- 172.21.15.149:0/2390711744 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9324109660 0x7f932419e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:52:12.928 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:52:12.939+0000 7f9323200700 1 -- 172.21.15.149:0/2390711744 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9324108460 msgr2=0x7f932419e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:52:12.928 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:52:12.939+0000 7f9323200700 1 --2- 172.21.15.149:0/2390711744 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9324108460 0x7f932419e410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:52:12.928 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:52:12.939+0000 7f9323200700 1 -- 172.21.15.149:0/2390711744 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9324109660 msgr2=0x7f932419e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:52:12.929 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:52:12.939+0000 7f9323200700 1 --2- 172.21.15.149:0/2390711744 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9324109660 0x7f932419e950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:52:12.929 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:52:12.939+0000 7f9323200700 1 -- 172.21.15.149:0/2390711744 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93241a6070 con 0x7f932410a000 2024-03-08T20:52:12.929 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:52:12.939+0000 7f9322800700 1 --2- 172.21.15.149:0/2390711744 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9324108460 0x7f932419e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T20:52:12.929 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:52:12.939+0000 7f9319e00700 1 --2- 172.21.15.149:0/2390711744 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9324109660 0x7f932419e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T20:52:12.929 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:52:12.939+0000 7f9323200700 1 --2- 172.21.15.149:0/2390711744 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f932410a000 0x7f93241a59d0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f931c005910 tx=0x7f931c009580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:52:12.929 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:52:12.939+0000 7f931be00700 1 -- 172.21.15.149:0/2390711744 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f931c027070 con 0x7f932410a000 2024-03-08T20:52:12.929 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:52:12.939+0000 7f9329400700 1 -- 172.21.15.149:0/2390711744 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93241a6300 con 0x7f932410a000 2024-03-08T20:52:12.929 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:52:12.939+0000 7f931be00700 1 -- 172.21.15.149:0/2390711744 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f931c004050 con 0x7f932410a000 2024-03-08T20:52:12.929 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:52:12.939+0000 7f931be00700 1 -- 172.21.15.149:0/2390711744 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f931c022e40 con 0x7f932410a000 2024-03-08T20:52:12.929 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:52:12.939+0000 7f9329400700 1 -- 172.21.15.149:0/2390711744 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93241a6700 con 0x7f932410a000 2024-03-08T20:52:12.930 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:52:12.941+0000 7f9329400700 1 -- 172.21.15.149:0/2390711744 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9324068aa0 con 0x7f932410a000 2024-03-08T20:52:12.931 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:52:12.943+0000 7f931be00700 1 -- 172.21.15.149:0/2390711744 <== mon.2 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 16) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f931c02a070 con 0x7f932410a000 2024-03-08T20:52:12.932 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:52:12.944+0000 7f931be00700 1 --2- 172.21.15.149:0/2390711744 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f93080a2890 0x7f93080a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:52:12.932 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:52:12.944+0000 7f931be00700 1 -- 172.21.15.149:0/2390711744 <== mon.2 v2:172.21.15.149:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 3669+0+0 (secure 0 0 0) 0x7f931c0ceb50 con 0x7f932410a000 2024-03-08T20:52:12.933 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:52:12.945+0000 7f9322800700 1 --2- 172.21.15.149:0/2390711744 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f93080a2890 0x7f93080a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:52:12.934 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:52:12.946+0000 7f9322800700 1 --2- 172.21.15.149:0/2390711744 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f93080a2890 0x7f93080a4d50 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f9314005950 tx=0x7f93140058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:52:12.935 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:52:12.947+0000 7f931be00700 1 -- 172.21.15.149:0/2390711744 <== mon.2 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f931c013070 con 0x7f932410a000 2024-03-08T20:52:13.171 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:52:13.182+0000 7f9329400700 1 -- 172.21.15.149:0/2390711744 --> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f9324062220 con 0x7f93080a2890 2024-03-08T20:52:13.520 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:52:13 smithi149 ceph-mon[35042]: pgmap v135: 1 pgs: 1 peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-08T20:52:13.521 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:52:13 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-08T20:52:13.521 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:52:13 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-08T20:52:13.521 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:52:13 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:52:13.723 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:52:13 smithi107 ceph-mon[31381]: pgmap v135: 1 pgs: 1 peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-08T20:52:13.723 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:52:13 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-08T20:52:13.723 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:52:13 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-08T20:52:13.723 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:52:13 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:52:13.884 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:52:13 smithi204 ceph-mon[34935]: pgmap v135: 1 pgs: 1 peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-08T20:52:13.884 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:52:13 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-08T20:52:13.884 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:52:13 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-08T20:52:13.884 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:52:13 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:52:14.723 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:52:14 smithi107 ceph-mon[31381]: from='client.34208 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-08T20:52:14.770 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:52:14 smithi149 ceph-mon[35042]: from='client.34208 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-08T20:52:14.884 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:52:14 smithi204 ceph-mon[34935]: from='client.34208 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-08T20:52:15.484 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:52:15 smithi149 ceph-mon[35042]: pgmap v136: 1 pgs: 1 peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 98 KiB/s, 0 objects/s recovering 2024-03-08T20:52:15.722 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:52:15 smithi107 ceph-mon[31381]: pgmap v136: 1 pgs: 1 peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 98 KiB/s, 0 objects/s recovering 2024-03-08T20:52:15.883 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:52:15 smithi204 ceph-mon[34935]: pgmap v136: 1 pgs: 1 peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 98 KiB/s, 0 objects/s recovering 2024-03-08T20:52:17.723 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:52:17 smithi107 ceph-mon[31381]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 84 KiB/s, 0 objects/s recovering 2024-03-08T20:52:17.723 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:52:17 smithi107 ceph-mon[31381]: from='client.? 172.21.15.149:0/216399567' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "27c53794-9d2b-487d-bfcf-ac4fd2ab061e"}]: dispatch 2024-03-08T20:52:17.723 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:52:17 smithi107 ceph-mon[31381]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "27c53794-9d2b-487d-bfcf-ac4fd2ab061e"}]: dispatch 2024-03-08T20:52:17.723 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:52:17 smithi107 ceph-mon[31381]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "27c53794-9d2b-487d-bfcf-ac4fd2ab061e"}]': finished 2024-03-08T20:52:17.723 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:52:17 smithi107 ceph-mon[31381]: osdmap e30: 5 total, 4 up, 5 in 2024-03-08T20:52:17.723 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:52:17 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-08T20:52:17.771 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:52:17 smithi149 ceph-mon[35042]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 84 KiB/s, 0 objects/s recovering 2024-03-08T20:52:17.771 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:52:17 smithi149 ceph-mon[35042]: from='client.? 172.21.15.149:0/216399567' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "27c53794-9d2b-487d-bfcf-ac4fd2ab061e"}]: dispatch 2024-03-08T20:52:17.771 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:52:17 smithi149 ceph-mon[35042]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "27c53794-9d2b-487d-bfcf-ac4fd2ab061e"}]: dispatch 2024-03-08T20:52:17.771 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:52:17 smithi149 ceph-mon[35042]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "27c53794-9d2b-487d-bfcf-ac4fd2ab061e"}]': finished 2024-03-08T20:52:17.772 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:52:17 smithi149 ceph-mon[35042]: osdmap e30: 5 total, 4 up, 5 in 2024-03-08T20:52:17.772 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:52:17 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-08T20:52:17.884 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:52:17 smithi204 ceph-mon[34935]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 84 KiB/s, 0 objects/s recovering 2024-03-08T20:52:17.884 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:52:17 smithi204 ceph-mon[34935]: from='client.? 172.21.15.149:0/216399567' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "27c53794-9d2b-487d-bfcf-ac4fd2ab061e"}]: dispatch 2024-03-08T20:52:17.884 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:52:17 smithi204 ceph-mon[34935]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "27c53794-9d2b-487d-bfcf-ac4fd2ab061e"}]: dispatch 2024-03-08T20:52:17.884 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:52:17 smithi204 ceph-mon[34935]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "27c53794-9d2b-487d-bfcf-ac4fd2ab061e"}]': finished 2024-03-08T20:52:17.884 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:52:17 smithi204 ceph-mon[34935]: osdmap e30: 5 total, 4 up, 5 in 2024-03-08T20:52:17.884 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:52:17 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-08T20:52:18.725 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:52:18 smithi107 ceph-mon[31381]: from='client.? 172.21.15.149:0/2167256965' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-08T20:52:18.770 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:52:18 smithi149 ceph-mon[35042]: from='client.? 172.21.15.149:0/2167256965' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-08T20:52:18.883 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:52:18 smithi204 ceph-mon[34935]: from='client.? 172.21.15.149:0/2167256965' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-08T20:52:19.722 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:52:19 smithi107 ceph-mon[31381]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-03-08T20:52:19.770 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:52:19 smithi149 ceph-mon[35042]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-03-08T20:52:19.883 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:52:19 smithi204 ceph-mon[34935]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-03-08T20:52:21.722 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:52:21 smithi107 ceph-mon[31381]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-08T20:52:21.770 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:52:21 smithi149 ceph-mon[35042]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-08T20:52:21.883 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:52:21 smithi204 ceph-mon[34935]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-08T20:52:23.723 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:52:23 smithi107 ceph-mon[31381]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-08T20:52:23.770 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:52:23 smithi149 ceph-mon[35042]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-08T20:52:23.883 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:52:23 smithi204 ceph-mon[34935]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-08T20:52:25.722 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:52:25 smithi107 ceph-mon[31381]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-08T20:52:25.770 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:52:25 smithi149 ceph-mon[35042]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-08T20:52:25.883 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:52:25 smithi204 ceph-mon[34935]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-08T20:52:26.722 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:52:26 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-08T20:52:26.723 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:52:26 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:52:26.770 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:52:26 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-08T20:52:26.770 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:52:26 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:52:26.883 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:52:26 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-08T20:52:26.883 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:52:26 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:52:27.464 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:52:27 smithi149 ceph-mon[35042]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-08T20:52:27.464 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:52:27 smithi149 ceph-mon[35042]: Deploying daemon osd.4 on smithi149 2024-03-08T20:52:27.722 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:52:27 smithi107 ceph-mon[31381]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-08T20:52:27.722 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:52:27 smithi107 ceph-mon[31381]: Deploying daemon osd.4 on smithi149 2024-03-08T20:52:27.884 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:52:27 smithi204 ceph-mon[34935]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-08T20:52:27.884 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:52:27 smithi204 ceph-mon[34935]: Deploying daemon osd.4 on smithi149 2024-03-08T20:52:28.520 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:52:28 smithi149 ceph-mon[35042]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-08T20:52:28.722 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:52:28 smithi107 ceph-mon[31381]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-08T20:52:28.883 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:52:28 smithi204 ceph-mon[34935]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-08T20:52:31.490 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:52:31 smithi149 ceph-mon[35042]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-08T20:52:31.633 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:52:31 smithi204 ceph-mon[34935]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-08T20:52:31.723 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:52:31 smithi107 ceph-mon[31381]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-08T20:52:33.520 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:52:33 smithi149 ceph-mon[35042]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-08T20:52:33.633 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:52:33 smithi204 ceph-mon[34935]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-08T20:52:33.722 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:52:33 smithi107 ceph-mon[31381]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-08T20:52:35.609 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:52:35 smithi149 ceph-mon[35042]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-08T20:52:35.609 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:52:35 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:52:35.609 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:52:35 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:52:35.609 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:52:35 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T20:52:35.633 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:52:35 smithi204 ceph-mon[34935]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-08T20:52:35.633 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:52:35 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:52:35.633 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:52:35 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:52:35.634 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:52:35 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T20:52:35.722 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:52:35 smithi107 ceph-mon[31381]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-08T20:52:35.723 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:52:35 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:52:35.723 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:52:35 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:52:35.723 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:52:35 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T20:52:37.722 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:52:37 smithi107 ceph-mon[31381]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-08T20:52:37.769 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:52:37 smithi149 ceph-mon[35042]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-08T20:52:37.883 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:52:37 smithi204 ceph-mon[34935]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-08T20:52:38.907 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:52:38 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:52:38.907 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:52:38 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:52:38.907 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:52:38 smithi149 ceph-mon[35042]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-08T20:52:38.907 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:52:38 smithi149 ceph-mon[35042]: from='osd.4 [v2:172.21.15.149:6808/726286918,v1:172.21.15.149:6809/726286918]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-08T20:52:38.907 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:52:38 smithi149 ceph-mon[35042]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-08T20:52:39.096 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:52:39.105+0000 7f931be00700 1 -- 172.21.15.149:0/2390711744 <== mgr.14152 v2:172.21.15.107:6800/3376636618 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f9324062220 con 0x7f93080a2890 2024-03-08T20:52:39.096 INFO:teuthology.orchestra.run.smithi149.stdout:Created osd(s) 4 on host 'smithi149' 2024-03-08T20:52:39.099 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:52:39.110+0000 7f9329400700 1 -- 172.21.15.149:0/2390711744 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f93080a2890 msgr2=0x7f93080a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:52:39.099 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:52:39.110+0000 7f9329400700 1 --2- 172.21.15.149:0/2390711744 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f93080a2890 0x7f93080a4d50 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f9314005950 tx=0x7f93140058e0 comp rx=0 tx=0).stop 2024-03-08T20:52:39.099 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:52:39.110+0000 7f9329400700 1 -- 172.21.15.149:0/2390711744 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f932410a000 msgr2=0x7f93241a59d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:52:39.100 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:52:39.110+0000 7f9329400700 1 --2- 172.21.15.149:0/2390711744 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f932410a000 0x7f93241a59d0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f931c005910 tx=0x7f931c009580 comp rx=0 tx=0).stop 2024-03-08T20:52:39.100 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:52:39.110+0000 7f9329400700 1 -- 172.21.15.149:0/2390711744 shutdown_connections 2024-03-08T20:52:39.100 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:52:39.110+0000 7f9329400700 1 --2- 172.21.15.149:0/2390711744 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9324108460 0x7f932419e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:52:39.100 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:52:39.110+0000 7f9329400700 1 --2- 172.21.15.149:0/2390711744 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f93080a2890 0x7f93080a4d50 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:52:39.100 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:52:39.110+0000 7f9329400700 1 --2- 172.21.15.149:0/2390711744 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f932410a000 0x7f93241a59d0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:52:39.100 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:52:39.110+0000 7f9329400700 1 --2- 172.21.15.149:0/2390711744 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9324109660 0x7f932419e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:52:39.100 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:52:39.110+0000 7f9329400700 1 -- 172.21.15.149:0/2390711744 >> 172.21.15.149:0/2390711744 conn(0x7f9324077430 msgr2=0x7f932410c900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:52:39.100 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:52:39.110+0000 7f9329400700 1 -- 172.21.15.149:0/2390711744 shutdown_connections 2024-03-08T20:52:39.100 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:52:39.111+0000 7f9329400700 1 -- 172.21.15.149:0/2390711744 wait complete. 2024-03-08T20:52:39.133 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:52:38 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:52:39.133 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:52:38 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:52:39.133 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:52:38 smithi204 ceph-mon[34935]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-08T20:52:39.133 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:52:38 smithi204 ceph-mon[34935]: from='osd.4 [v2:172.21.15.149:6808/726286918,v1:172.21.15.149:6809/726286918]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-08T20:52:39.133 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:52:38 smithi204 ceph-mon[34935]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-08T20:52:39.222 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:52:38 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:52:39.223 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:52:38 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:52:39.223 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:52:38 smithi107 ceph-mon[31381]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-08T20:52:39.223 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:52:38 smithi107 ceph-mon[31381]: from='osd.4 [v2:172.21.15.149:6808/726286918,v1:172.21.15.149:6809/726286918]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-08T20:52:39.223 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:52:38 smithi107 ceph-mon[31381]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-08T20:52:39.893 DEBUG:teuthology.orchestra.run.smithi149:osd.4> sudo journalctl -f -n 0 -u ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715@osd.4.service 2024-03-08T20:52:39.897 INFO:tasks.cephadm:Deploying osd.5 on smithi204 with /dev/vg_nvme/lv_3... 2024-03-08T20:52:39.897 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dc20f0f8-dd8c-11ee-95c6-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-03-08T20:52:40.071 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:52:39 smithi149 ceph-mon[35042]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-08T20:52:40.071 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:52:39 smithi149 ceph-mon[35042]: osdmap e31: 5 total, 4 up, 5 in 2024-03-08T20:52:40.071 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:52:39 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-08T20:52:40.072 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:52:39 smithi149 ceph-mon[35042]: from='osd.4 [v2:172.21.15.149:6808/726286918,v1:172.21.15.149:6809/726286918]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-03-08T20:52:40.072 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:52:39 smithi149 ceph-mon[35042]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-03-08T20:52:40.072 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:52:39 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:52:40.072 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:52:39 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:52:40.133 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:52:39 smithi204 ceph-mon[34935]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-08T20:52:40.133 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:52:39 smithi204 ceph-mon[34935]: osdmap e31: 5 total, 4 up, 5 in 2024-03-08T20:52:40.134 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:52:39 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-08T20:52:40.134 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:52:39 smithi204 ceph-mon[34935]: from='osd.4 [v2:172.21.15.149:6808/726286918,v1:172.21.15.149:6809/726286918]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-03-08T20:52:40.134 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:52:39 smithi204 ceph-mon[34935]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-03-08T20:52:40.134 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:52:39 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:52:40.134 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:52:39 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:52:40.222 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:52:39 smithi107 ceph-mon[31381]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-08T20:52:40.223 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:52:39 smithi107 ceph-mon[31381]: osdmap e31: 5 total, 4 up, 5 in 2024-03-08T20:52:40.223 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:52:39 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-08T20:52:40.223 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:52:39 smithi107 ceph-mon[31381]: from='osd.4 [v2:172.21.15.149:6808/726286918,v1:172.21.15.149:6809/726286918]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-03-08T20:52:40.223 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:52:39 smithi107 ceph-mon[31381]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-03-08T20:52:40.223 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:52:39 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:52:40.223 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:52:39 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:52:40.342 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/mon.c/config 2024-03-08T20:52:40.772 INFO:journalctl@ceph.osd.4.smithi149.stdout:Mar 08 20:52:40 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-4[46289]: 2024-03-08T20:52:40.425+0000 7f2988000700 -1 osd.4 0 waiting for initial osdmap 2024-03-08T20:52:40.772 INFO:journalctl@ceph.osd.4.smithi149.stdout:Mar 08 20:52:40 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-4[46289]: 2024-03-08T20:52:40.442+0000 7f297de00700 -1 osd.4 32 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-08T20:52:41.124 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:52:40 smithi107 ceph-mon[31381]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]': finished 2024-03-08T20:52:41.124 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:52:40 smithi107 ceph-mon[31381]: osdmap e32: 5 total, 4 up, 5 in 2024-03-08T20:52:41.124 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:52:40 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-08T20:52:41.124 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:52:40 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-08T20:52:41.124 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:52:40 smithi107 ceph-mon[31381]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-08T20:52:41.125 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:52:40 smithi107 ceph-mon[31381]: from='osd.4 ' entity='osd.4' 2024-03-08T20:52:41.133 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:52:40 smithi204 ceph-mon[34935]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]': finished 2024-03-08T20:52:41.133 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:52:40 smithi204 ceph-mon[34935]: osdmap e32: 5 total, 4 up, 5 in 2024-03-08T20:52:41.134 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:52:40 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-08T20:52:41.134 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:52:40 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-08T20:52:41.134 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:52:40 smithi204 ceph-mon[34935]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-08T20:52:41.134 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:52:40 smithi204 ceph-mon[34935]: from='osd.4 ' entity='osd.4' 2024-03-08T20:52:41.236 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:52:40 smithi149 ceph-mon[35042]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]': finished 2024-03-08T20:52:41.237 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:52:40 smithi149 ceph-mon[35042]: osdmap e32: 5 total, 4 up, 5 in 2024-03-08T20:52:41.237 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:52:40 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-08T20:52:41.237 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:52:40 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-08T20:52:41.237 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:52:40 smithi149 ceph-mon[35042]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-03-08T20:52:41.237 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:52:40 smithi149 ceph-mon[35042]: from='osd.4 ' entity='osd.4' 2024-03-08T20:52:42.061 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:52:41 smithi204 ceph-mon[34935]: purged_snaps scrub starts 2024-03-08T20:52:42.061 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:52:41 smithi204 ceph-mon[34935]: purged_snaps scrub ok 2024-03-08T20:52:42.062 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:52:41 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-08T20:52:42.062 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:52:41 smithi204 ceph-mon[34935]: osd.4 [v2:172.21.15.149:6808/726286918,v1:172.21.15.149:6809/726286918] boot 2024-03-08T20:52:42.062 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:52:41 smithi204 ceph-mon[34935]: osdmap e33: 5 total, 5 up, 5 in 2024-03-08T20:52:42.062 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:52:41 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-08T20:52:42.222 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:52:41 smithi107 ceph-mon[31381]: purged_snaps scrub starts 2024-03-08T20:52:42.223 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:52:41 smithi107 ceph-mon[31381]: purged_snaps scrub ok 2024-03-08T20:52:42.223 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:52:41 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-08T20:52:42.223 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:52:41 smithi107 ceph-mon[31381]: osd.4 [v2:172.21.15.149:6808/726286918,v1:172.21.15.149:6809/726286918] boot 2024-03-08T20:52:42.223 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:52:41 smithi107 ceph-mon[31381]: osdmap e33: 5 total, 5 up, 5 in 2024-03-08T20:52:42.223 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:52:41 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-08T20:52:42.270 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:52:41 smithi149 ceph-mon[35042]: purged_snaps scrub starts 2024-03-08T20:52:42.270 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:52:41 smithi149 ceph-mon[35042]: purged_snaps scrub ok 2024-03-08T20:52:42.271 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:52:41 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-08T20:52:42.271 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:52:41 smithi149 ceph-mon[35042]: osd.4 [v2:172.21.15.149:6808/726286918,v1:172.21.15.149:6809/726286918] boot 2024-03-08T20:52:42.271 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:52:41 smithi149 ceph-mon[35042]: osdmap e33: 5 total, 5 up, 5 in 2024-03-08T20:52:42.271 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:52:41 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-08T20:52:42.883 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:52:42 smithi204 ceph-mon[34935]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-08T20:52:43.176 INFO:teuthology.orchestra.run.smithi204.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-03-08T20:52:43.177 INFO:teuthology.orchestra.run.smithi204.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-03-08T20:52:43.177 INFO:teuthology.orchestra.run.smithi204.stderr: stderr: 10+0 records in 2024-03-08T20:52:43.177 INFO:teuthology.orchestra.run.smithi204.stderr:10+0 records out 2024-03-08T20:52:43.177 INFO:teuthology.orchestra.run.smithi204.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0404068 s, 260 MB/s 2024-03-08T20:52:43.177 INFO:teuthology.orchestra.run.smithi204.stderr:--> Zapping successful for: 2024-03-08T20:52:43.222 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:52:42 smithi107 ceph-mon[31381]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-08T20:52:43.270 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:52:42 smithi149 ceph-mon[35042]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-08T20:52:43.985 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dc20f0f8-dd8c-11ee-95c6-87774f69a715 -- ceph orch daemon add osd smithi204:vg_nvme/lv_3 2024-03-08T20:52:44.020 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:52:43 smithi149 ceph-mon[35042]: osdmap e34: 5 total, 5 up, 5 in 2024-03-08T20:52:44.071 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:52:43 smithi204 ceph-mon[34935]: osdmap e34: 5 total, 5 up, 5 in 2024-03-08T20:52:44.222 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:52:43 smithi107 ceph-mon[31381]: osdmap e34: 5 total, 5 up, 5 in 2024-03-08T20:52:44.419 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/mon.c/config 2024-03-08T20:52:45.134 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:52:44 smithi204 ceph-mon[34935]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-08T20:52:45.134 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:52:44 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:52:45.134 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:52:44 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:52:45.134 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:52:44 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-08T20:52:45.134 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:52:44 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-08T20:52:45.134 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:52:44 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:52:45.134 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:52:44 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:52:45.135 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:52:44 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T20:52:45.135 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:52:44 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:52:45.222 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:52:44 smithi107 ceph-mon[31381]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-08T20:52:45.223 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:52:44 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:52:45.223 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:52:44 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:52:45.223 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:52:44 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-08T20:52:45.223 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:52:44 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-08T20:52:45.223 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:52:44 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:52:45.223 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:52:44 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:52:45.223 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:52:44 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T20:52:45.223 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:52:44 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:52:45.270 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:52:44 smithi149 ceph-mon[35042]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-08T20:52:45.270 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:52:44 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:52:45.270 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:52:44 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:52:45.270 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:52:44 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-08T20:52:45.270 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:52:44 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-08T20:52:45.270 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:52:44 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:52:45.270 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:52:44 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:52:45.270 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:52:44 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T20:52:45.270 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:52:44 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:52:46.040 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:52:45 smithi204 ceph-mon[34935]: Detected new or changed devices on smithi149 2024-03-08T20:52:46.041 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:52:45 smithi204 ceph-mon[34935]: Adjusting osd_memory_target on smithi149 to 2909M 2024-03-08T20:52:46.041 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:52:45 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T20:52:46.041 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:52:45 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:52:46.041 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:52:45 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T20:52:46.041 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:52:45 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:52:46.222 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:52:45 smithi107 ceph-mon[31381]: Detected new or changed devices on smithi149 2024-03-08T20:52:46.222 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:52:45 smithi107 ceph-mon[31381]: Adjusting osd_memory_target on smithi149 to 2909M 2024-03-08T20:52:46.222 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:52:45 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T20:52:46.223 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:52:45 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:52:46.223 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:52:45 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T20:52:46.223 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:52:45 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:52:46.270 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:52:45 smithi149 ceph-mon[35042]: Detected new or changed devices on smithi149 2024-03-08T20:52:46.270 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:52:45 smithi149 ceph-mon[35042]: Adjusting osd_memory_target on smithi149 to 2909M 2024-03-08T20:52:46.270 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:52:45 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T20:52:46.270 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:52:45 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:52:46.270 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:52:45 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T20:52:46.270 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:52:45 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:52:46.617 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:52:46.628+0000 7f3dd6800700 1 -- 172.21.15.204:0/3762168862 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3dd0103930 msgr2=0x7f3dd010a250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:52:46.617 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:52:46.628+0000 7f3dd6800700 1 --2- 172.21.15.204:0/3762168862 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3dd0103930 0x7f3dd010a250 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f3dc800b0a0 tx=0x7f3dc800b3b0 comp rx=0 tx=0).stop 2024-03-08T20:52:46.617 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:52:46.629+0000 7f3dd6800700 1 -- 172.21.15.204:0/3762168862 shutdown_connections 2024-03-08T20:52:46.617 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:52:46.629+0000 7f3dd6800700 1 --2- 172.21.15.204:0/3762168862 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3dd0103930 0x7f3dd010a250 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:52:46.617 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:52:46.629+0000 7f3dd6800700 1 --2- 172.21.15.204:0/3762168862 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3dd0101e10 0x7f3dd0102230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:52:46.617 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:52:46.629+0000 7f3dd6800700 1 --2- 172.21.15.204:0/3762168862 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3dd0102f70 0x7f3dd01033f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:52:46.617 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:52:46.629+0000 7f3dd6800700 1 -- 172.21.15.204:0/3762168862 >> 172.21.15.204:0/3762168862 conn(0x7f3dd00fd3f0 msgr2=0x7f3dd00ff810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:52:46.618 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:52:46.629+0000 7f3dd6800700 1 -- 172.21.15.204:0/3762168862 shutdown_connections 2024-03-08T20:52:46.618 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:52:46.629+0000 7f3dd6800700 1 -- 172.21.15.204:0/3762168862 wait complete. 2024-03-08T20:52:46.619 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:52:46.630+0000 7f3dd6800700 1 Processor -- start 2024-03-08T20:52:46.619 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:52:46.630+0000 7f3dd6800700 1 -- start start 2024-03-08T20:52:46.619 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:52:46.630+0000 7f3dd6800700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3dd0101e10 0x7f3dd0199fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:52:46.620 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:52:46.630+0000 7f3dd6800700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3dd0102f70 0x7f3dd019a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:52:46.620 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:52:46.630+0000 7f3dd6800700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3dd0103930 0x7f3dd01a1560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:52:46.620 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:52:46.630+0000 7f3dd6800700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dd010d800 con 0x7f3dd0101e10 2024-03-08T20:52:46.620 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:52:46.630+0000 7f3dd6800700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dd010d680 con 0x7f3dd0102f70 2024-03-08T20:52:46.620 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:52:46.630+0000 7f3dd6800700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dd010d980 con 0x7f3dd0103930 2024-03-08T20:52:46.620 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:52:46.631+0000 7f3dcfe00700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3dd0103930 0x7f3dd01a1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:52:46.620 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:52:46.631+0000 7f3dcfe00700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3dd0103930 0x7f3dd01a1560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:35238/0 (socket says 172.21.15.204:35238) 2024-03-08T20:52:46.620 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:52:46.631+0000 7f3dcfe00700 1 -- 172.21.15.204:0/2364830932 learned_addr learned my addr 172.21.15.204:0/2364830932 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-03-08T20:52:46.620 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:52:46.631+0000 7f3dc7e00700 1 --2- 172.21.15.204:0/2364830932 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3dd0102f70 0x7f3dd019a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:52:46.620 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:52:46.631+0000 7f3dcf400700 1 --2- 172.21.15.204:0/2364830932 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3dd0101e10 0x7f3dd0199fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:52:46.620 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:52:46.631+0000 7f3dcfe00700 1 -- 172.21.15.204:0/2364830932 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3dd0102f70 msgr2=0x7f3dd019a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:52:46.620 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:52:46.631+0000 7f3dcfe00700 1 --2- 172.21.15.204:0/2364830932 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3dd0102f70 0x7f3dd019a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:52:46.620 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:52:46.631+0000 7f3dcfe00700 1 -- 172.21.15.204:0/2364830932 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3dd0101e10 msgr2=0x7f3dd0199fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:52:46.621 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:52:46.631+0000 7f3dcfe00700 1 --2- 172.21.15.204:0/2364830932 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3dd0101e10 0x7f3dd0199fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:52:46.621 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:52:46.631+0000 7f3dcfe00700 1 -- 172.21.15.204:0/2364830932 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dd01a1c00 con 0x7f3dd0103930 2024-03-08T20:52:46.621 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:52:46.631+0000 7f3dcfe00700 1 --2- 172.21.15.204:0/2364830932 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3dd0103930 0x7f3dd01a1560 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f3dc800ce80 tx=0x7f3dc800cf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:52:46.621 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:52:46.632+0000 7f3dccc00700 1 -- 172.21.15.204:0/2364830932 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dc8021070 con 0x7f3dd0103930 2024-03-08T20:52:46.621 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:52:46.632+0000 7f3dccc00700 1 -- 172.21.15.204:0/2364830932 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f3dc801d900 con 0x7f3dd0103930 2024-03-08T20:52:46.622 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:52:46.632+0000 7f3dd6800700 1 -- 172.21.15.204:0/2364830932 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3dd01a1e90 con 0x7f3dd0103930 2024-03-08T20:52:46.622 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:52:46.632+0000 7f3dccc00700 1 -- 172.21.15.204:0/2364830932 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dc8005370 con 0x7f3dd0103930 2024-03-08T20:52:46.622 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:52:46.632+0000 7f3dd6800700 1 -- 172.21.15.204:0/2364830932 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3dd01a2340 con 0x7f3dd0103930 2024-03-08T20:52:46.622 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:52:46.633+0000 7f3dd6800700 1 -- 172.21.15.204:0/2364830932 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3dd0068aa0 con 0x7f3dd0103930 2024-03-08T20:52:46.625 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:52:46.637+0000 7f3dccc00700 1 -- 172.21.15.204:0/2364830932 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 16) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f3dc8024070 con 0x7f3dd0103930 2024-03-08T20:52:46.626 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:52:46.638+0000 7f3dccc00700 1 --2- 172.21.15.204:0/2364830932 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f3db40a2890 0x7f3db40a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:52:46.626 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:52:46.638+0000 7f3dccc00700 1 -- 172.21.15.204:0/2364830932 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 4101+0+0 (secure 0 0 0) 0x7f3dc80cf550 con 0x7f3dd0103930 2024-03-08T20:52:46.626 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:52:46.639+0000 7f3dcf400700 1 --2- 172.21.15.204:0/2364830932 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f3db40a2890 0x7f3db40a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:52:46.627 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:52:46.640+0000 7f3dcf400700 1 --2- 172.21.15.204:0/2364830932 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f3db40a2890 0x7f3db40a4d50 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f3dc0005ea0 tx=0x7f3dc0005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:52:46.629 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:52:46.641+0000 7f3dccc00700 1 -- 172.21.15.204:0/2364830932 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f3dc802b020 con 0x7f3dd0103930 2024-03-08T20:52:46.865 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:52:46.877+0000 7f3dd6800700 1 -- 172.21.15.204:0/2364830932 --> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f3dd0075730 con 0x7f3db40a2890 2024-03-08T20:52:47.133 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:52:46 smithi204 ceph-mon[34935]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-08T20:52:47.222 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:52:46 smithi107 ceph-mon[31381]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-08T20:52:47.270 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:52:46 smithi149 ceph-mon[35042]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-08T20:52:48.133 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:52:47 smithi204 ceph-mon[34935]: from='client.24238 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-08T20:52:48.133 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:52:47 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-08T20:52:48.133 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:52:47 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-08T20:52:48.133 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:52:47 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:52:48.222 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:52:47 smithi107 ceph-mon[31381]: from='client.24238 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-08T20:52:48.222 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:52:47 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-08T20:52:48.222 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:52:47 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-08T20:52:48.223 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:52:47 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:52:48.270 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:52:47 smithi149 ceph-mon[35042]: from='client.24238 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-08T20:52:48.270 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:52:47 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-08T20:52:48.270 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:52:47 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-08T20:52:48.270 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:52:47 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:52:49.133 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:52:48 smithi204 ceph-mon[34935]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-08T20:52:49.223 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:52:48 smithi107 ceph-mon[31381]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-08T20:52:49.270 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:52:48 smithi149 ceph-mon[35042]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-08T20:52:51.520 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:52:51 smithi149 ceph-mon[35042]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-08T20:52:51.520 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:52:51 smithi149 ceph-mon[35042]: from='client.? 172.21.15.204:0/2344812524' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3a7778fb-46c5-40c0-842e-e988c45ea24d"}]: dispatch 2024-03-08T20:52:51.520 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:52:51 smithi149 ceph-mon[35042]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3a7778fb-46c5-40c0-842e-e988c45ea24d"}]: dispatch 2024-03-08T20:52:51.520 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:52:51 smithi149 ceph-mon[35042]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3a7778fb-46c5-40c0-842e-e988c45ea24d"}]': finished 2024-03-08T20:52:51.520 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:52:51 smithi149 ceph-mon[35042]: osdmap e35: 6 total, 5 up, 6 in 2024-03-08T20:52:51.520 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:52:51 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-08T20:52:51.634 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:52:51 smithi204 ceph-mon[34935]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-08T20:52:51.634 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:52:51 smithi204 ceph-mon[34935]: from='client.? 172.21.15.204:0/2344812524' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3a7778fb-46c5-40c0-842e-e988c45ea24d"}]: dispatch 2024-03-08T20:52:51.634 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:52:51 smithi204 ceph-mon[34935]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3a7778fb-46c5-40c0-842e-e988c45ea24d"}]: dispatch 2024-03-08T20:52:51.634 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:52:51 smithi204 ceph-mon[34935]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3a7778fb-46c5-40c0-842e-e988c45ea24d"}]': finished 2024-03-08T20:52:51.635 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:52:51 smithi204 ceph-mon[34935]: osdmap e35: 6 total, 5 up, 6 in 2024-03-08T20:52:51.635 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:52:51 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-08T20:52:51.722 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:52:51 smithi107 ceph-mon[31381]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-08T20:52:51.722 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:52:51 smithi107 ceph-mon[31381]: from='client.? 172.21.15.204:0/2344812524' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3a7778fb-46c5-40c0-842e-e988c45ea24d"}]: dispatch 2024-03-08T20:52:51.722 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:52:51 smithi107 ceph-mon[31381]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3a7778fb-46c5-40c0-842e-e988c45ea24d"}]: dispatch 2024-03-08T20:52:51.723 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:52:51 smithi107 ceph-mon[31381]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3a7778fb-46c5-40c0-842e-e988c45ea24d"}]': finished 2024-03-08T20:52:51.723 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:52:51 smithi107 ceph-mon[31381]: osdmap e35: 6 total, 5 up, 6 in 2024-03-08T20:52:51.723 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:52:51 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-08T20:52:52.633 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:52:52 smithi204 ceph-mon[34935]: from='client.? 172.21.15.204:0/3324027755' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-08T20:52:52.722 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:52:52 smithi107 ceph-mon[31381]: from='client.? 172.21.15.204:0/3324027755' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-08T20:52:52.770 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:52:52 smithi149 ceph-mon[35042]: from='client.? 172.21.15.204:0/3324027755' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-08T20:52:53.633 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:52:53 smithi204 ceph-mon[34935]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-08T20:52:53.722 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:52:53 smithi107 ceph-mon[31381]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-08T20:52:53.770 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:52:53 smithi149 ceph-mon[35042]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-08T20:52:55.633 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:52:55 smithi204 ceph-mon[34935]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-08T20:52:55.722 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:52:55 smithi107 ceph-mon[31381]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-08T20:52:55.770 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:52:55 smithi149 ceph-mon[35042]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-08T20:52:57.633 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:52:57 smithi204 ceph-mon[34935]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-08T20:52:57.722 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:52:57 smithi107 ceph-mon[31381]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-08T20:52:57.770 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:52:57 smithi149 ceph-mon[35042]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-08T20:52:59.634 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:52:59 smithi204 ceph-mon[34935]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-08T20:52:59.722 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:52:59 smithi107 ceph-mon[31381]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-08T20:52:59.769 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:52:59 smithi149 ceph-mon[35042]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-08T20:53:00.633 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:53:00 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-08T20:53:00.633 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:53:00 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:53:00.722 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:53:00 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-08T20:53:00.722 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:53:00 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:53:00.770 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:53:00 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-08T20:53:00.770 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:53:00 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:53:01.563 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:53:01 smithi204 ceph-mon[34935]: Deploying daemon osd.5 on smithi204 2024-03-08T20:53:01.563 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:53:01 smithi204 ceph-mon[34935]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-08T20:53:01.722 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:53:01 smithi107 ceph-mon[31381]: Deploying daemon osd.5 on smithi204 2024-03-08T20:53:01.722 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:53:01 smithi107 ceph-mon[31381]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-08T20:53:01.769 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:53:01 smithi149 ceph-mon[35042]: Deploying daemon osd.5 on smithi204 2024-03-08T20:53:01.770 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:53:01 smithi149 ceph-mon[35042]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-08T20:53:03.633 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:53:03 smithi204 ceph-mon[34935]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-08T20:53:03.722 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:53:03 smithi107 ceph-mon[31381]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-08T20:53:03.769 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:53:03 smithi149 ceph-mon[35042]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-08T20:53:05.342 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:53:05 smithi204 ceph-mon[34935]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-08T20:53:05.722 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:53:05 smithi107 ceph-mon[31381]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-08T20:53:05.769 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:53:05 smithi149 ceph-mon[35042]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-08T20:53:07.633 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:53:07 smithi204 ceph-mon[34935]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-08T20:53:07.722 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:53:07 smithi107 ceph-mon[31381]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-08T20:53:07.769 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:53:07 smithi149 ceph-mon[35042]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-08T20:53:08.633 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:53:08 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T20:53:08.633 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:53:08 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:53:08.633 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:53:08 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:53:08.722 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:53:08 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T20:53:08.722 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:53:08 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:53:08.722 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:53:08 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:53:08.770 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:53:08 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T20:53:08.770 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:53:08 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:53:08.770 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:53:08 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:53:09.630 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:53:09 smithi204 ceph-mon[34935]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-08T20:53:09.671 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:53:09 smithi149 ceph-mon[35042]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-08T20:53:09.722 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:53:09 smithi107 ceph-mon[31381]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-08T20:53:11.520 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:53:11 smithi149 ceph-mon[35042]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-08T20:53:11.633 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:53:11 smithi204 ceph-mon[34935]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-08T20:53:11.722 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:53:11 smithi107 ceph-mon[31381]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-08T20:53:12.633 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:53:12 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:53:12.633 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:53:12 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:53:12.633 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:53:12 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:53:12.633 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:53:12 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T20:53:12.633 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:53:12 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:53:12.633 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:53:12 smithi204 ceph-mon[34935]: from='osd.5 [v2:172.21.15.204:6808/3143887820,v1:172.21.15.204:6809/3143887820]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-08T20:53:12.634 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:53:12 smithi204 ceph-mon[34935]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-08T20:53:12.634 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:53:12 smithi204 ceph-mon[34935]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-08T20:53:12.872 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:53:12.884+0000 7f3dccc00700 1 -- 172.21.15.204:0/2364830932 <== mgr.14152 v2:172.21.15.107:6800/3376636618 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f3dd0075730 con 0x7f3db40a2890 2024-03-08T20:53:12.873 INFO:teuthology.orchestra.run.smithi204.stdout:Created osd(s) 5 on host 'smithi204' 2024-03-08T20:53:12.876 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:53:12.888+0000 7f3dd6800700 1 -- 172.21.15.204:0/2364830932 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f3db40a2890 msgr2=0x7f3db40a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:53:12.877 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:53:12.888+0000 7f3dd6800700 1 --2- 172.21.15.204:0/2364830932 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f3db40a2890 0x7f3db40a4d50 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f3dc0005ea0 tx=0x7f3dc0005e30 comp rx=0 tx=0).stop 2024-03-08T20:53:12.877 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:53:12.888+0000 7f3dd6800700 1 -- 172.21.15.204:0/2364830932 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3dd0103930 msgr2=0x7f3dd01a1560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:53:12.877 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:53:12.888+0000 7f3dd6800700 1 --2- 172.21.15.204:0/2364830932 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3dd0103930 0x7f3dd01a1560 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f3dc800ce80 tx=0x7f3dc800cf60 comp rx=0 tx=0).stop 2024-03-08T20:53:12.877 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:53:12.888+0000 7f3dd6800700 1 -- 172.21.15.204:0/2364830932 shutdown_connections 2024-03-08T20:53:12.877 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:53:12.888+0000 7f3dd6800700 1 --2- 172.21.15.204:0/2364830932 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3dd0103930 0x7f3dd01a1560 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:53:12.877 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:53:12.889+0000 7f3dd6800700 1 --2- 172.21.15.204:0/2364830932 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f3db40a2890 0x7f3db40a4d50 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:53:12.877 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:53:12.889+0000 7f3dd6800700 1 --2- 172.21.15.204:0/2364830932 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3dd0101e10 0x7f3dd0199fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:53:12.877 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:53:12.889+0000 7f3dd6800700 1 --2- 172.21.15.204:0/2364830932 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3dd0102f70 0x7f3dd019a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:53:12.877 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:53:12.889+0000 7f3dd6800700 1 -- 172.21.15.204:0/2364830932 >> 172.21.15.204:0/2364830932 conn(0x7f3dd00fd3f0 msgr2=0x7f3dd0074010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:53:12.877 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:53:12.889+0000 7f3dd6800700 1 -- 172.21.15.204:0/2364830932 shutdown_connections 2024-03-08T20:53:12.877 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:53:12.889+0000 7f3dd6800700 1 -- 172.21.15.204:0/2364830932 wait complete. 2024-03-08T20:53:12.972 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:53:12 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:53:12.972 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:53:12 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:53:12.972 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:53:12 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:53:12.972 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:53:12 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T20:53:12.972 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:53:12 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:53:12.973 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:53:12 smithi107 ceph-mon[31381]: from='osd.5 [v2:172.21.15.204:6808/3143887820,v1:172.21.15.204:6809/3143887820]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-08T20:53:12.973 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:53:12 smithi107 ceph-mon[31381]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-08T20:53:12.973 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:53:12 smithi107 ceph-mon[31381]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-08T20:53:13.020 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:53:12 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:53:13.020 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:53:12 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:53:13.020 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:53:12 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:53:13.020 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:53:12 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T20:53:13.020 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:53:12 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:53:13.020 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:53:12 smithi149 ceph-mon[35042]: from='osd.5 [v2:172.21.15.204:6808/3143887820,v1:172.21.15.204:6809/3143887820]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-08T20:53:13.020 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:53:12 smithi149 ceph-mon[35042]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-08T20:53:13.020 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:53:12 smithi149 ceph-mon[35042]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-08T20:53:13.713 DEBUG:teuthology.orchestra.run.smithi204:osd.5> sudo journalctl -f -n 0 -u ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715@osd.5.service 2024-03-08T20:53:13.717 INFO:tasks.cephadm:Deploying osd.6 on smithi107 with /dev/vg_nvme/lv_2... 2024-03-08T20:53:13.717 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dc20f0f8-dd8c-11ee-95c6-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-03-08T20:53:13.882 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:53:13 smithi204 ceph-mon[34935]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-08T20:53:13.883 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:53:13 smithi204 ceph-mon[34935]: osdmap e36: 6 total, 5 up, 6 in 2024-03-08T20:53:13.883 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:53:13 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-08T20:53:13.883 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:53:13 smithi204 ceph-mon[34935]: from='osd.5 [v2:172.21.15.204:6808/3143887820,v1:172.21.15.204:6809/3143887820]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-03-08T20:53:13.883 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:53:13 smithi204 ceph-mon[34935]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-03-08T20:53:13.883 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:53:13 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:53:13.883 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:53:13 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:53:13.883 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:53:13 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T20:53:13.972 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:53:13 smithi107 ceph-mon[31381]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-08T20:53:13.972 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:53:13 smithi107 ceph-mon[31381]: osdmap e36: 6 total, 5 up, 6 in 2024-03-08T20:53:13.972 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:53:13 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-08T20:53:13.972 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:53:13 smithi107 ceph-mon[31381]: from='osd.5 [v2:172.21.15.204:6808/3143887820,v1:172.21.15.204:6809/3143887820]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-03-08T20:53:13.972 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:53:13 smithi107 ceph-mon[31381]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-03-08T20:53:13.972 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:53:13 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:53:13.972 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:53:13 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:53:13.972 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:53:13 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T20:53:14.020 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:53:13 smithi149 ceph-mon[35042]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-08T20:53:14.020 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:53:13 smithi149 ceph-mon[35042]: osdmap e36: 6 total, 5 up, 6 in 2024-03-08T20:53:14.020 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:53:13 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-08T20:53:14.020 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:53:13 smithi149 ceph-mon[35042]: from='osd.5 [v2:172.21.15.204:6808/3143887820,v1:172.21.15.204:6809/3143887820]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-03-08T20:53:14.020 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:53:13 smithi149 ceph-mon[35042]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-03-08T20:53:14.020 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:53:13 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:53:14.020 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:53:13 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:53:14.020 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:53:13 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T20:53:14.165 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/mon.a/config 2024-03-08T20:53:14.329 INFO:journalctl@ceph.osd.5.smithi204.stdout:Mar 08 20:53:14 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-5[47140]: 2024-03-08T20:53:14.213+0000 7f0394600700 -1 osd.5 0 waiting for initial osdmap 2024-03-08T20:53:14.329 INFO:journalctl@ceph.osd.5.smithi204.stdout:Mar 08 20:53:14 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-5[47140]: 2024-03-08T20:53:14.234+0000 7f038a400700 -1 osd.5 38 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-08T20:53:14.633 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:53:14 smithi204 ceph-mon[34935]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]': finished 2024-03-08T20:53:14.633 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:53:14 smithi204 ceph-mon[34935]: osdmap e37: 6 total, 5 up, 6 in 2024-03-08T20:53:14.634 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:53:14 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-08T20:53:14.634 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:53:14 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-08T20:53:14.634 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:53:14 smithi204 ceph-mon[34935]: osdmap e38: 6 total, 5 up, 6 in 2024-03-08T20:53:14.634 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:53:14 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-08T20:53:14.634 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:53:14 smithi204 ceph-mon[34935]: from='osd.5 ' entity='osd.5' 2024-03-08T20:53:14.634 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:53:14 smithi204 ceph-mon[34935]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-08T20:53:14.972 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:53:14 smithi107 ceph-mon[31381]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]': finished 2024-03-08T20:53:14.972 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:53:14 smithi107 ceph-mon[31381]: osdmap e37: 6 total, 5 up, 6 in 2024-03-08T20:53:14.972 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:53:14 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-08T20:53:14.972 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:53:14 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-08T20:53:14.972 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:53:14 smithi107 ceph-mon[31381]: osdmap e38: 6 total, 5 up, 6 in 2024-03-08T20:53:14.972 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:53:14 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-08T20:53:14.972 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:53:14 smithi107 ceph-mon[31381]: from='osd.5 ' entity='osd.5' 2024-03-08T20:53:14.972 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:53:14 smithi107 ceph-mon[31381]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-08T20:53:15.020 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:53:14 smithi149 ceph-mon[35042]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]': finished 2024-03-08T20:53:15.020 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:53:14 smithi149 ceph-mon[35042]: osdmap e37: 6 total, 5 up, 6 in 2024-03-08T20:53:15.020 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:53:14 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-08T20:53:15.020 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:53:14 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-08T20:53:15.020 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:53:14 smithi149 ceph-mon[35042]: osdmap e38: 6 total, 5 up, 6 in 2024-03-08T20:53:15.020 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:53:14 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-08T20:53:15.020 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:53:14 smithi149 ceph-mon[35042]: from='osd.5 ' entity='osd.5' 2024-03-08T20:53:15.020 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:53:14 smithi149 ceph-mon[35042]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-03-08T20:53:15.659 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:53:15 smithi204 ceph-mon[34935]: purged_snaps scrub starts 2024-03-08T20:53:15.659 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:53:15 smithi204 ceph-mon[34935]: purged_snaps scrub ok 2024-03-08T20:53:15.659 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:53:15 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-08T20:53:15.659 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:53:15 smithi204 ceph-mon[34935]: osd.5 [v2:172.21.15.204:6808/3143887820,v1:172.21.15.204:6809/3143887820] boot 2024-03-08T20:53:15.659 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:53:15 smithi204 ceph-mon[34935]: osdmap e39: 6 total, 6 up, 6 in 2024-03-08T20:53:15.660 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:53:15 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-08T20:53:15.865 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:53:15 smithi107 ceph-mon[31381]: purged_snaps scrub starts 2024-03-08T20:53:15.866 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:53:15 smithi107 ceph-mon[31381]: purged_snaps scrub ok 2024-03-08T20:53:15.866 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:53:15 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-08T20:53:15.866 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:53:15 smithi107 ceph-mon[31381]: osd.5 [v2:172.21.15.204:6808/3143887820,v1:172.21.15.204:6809/3143887820] boot 2024-03-08T20:53:15.866 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:53:15 smithi107 ceph-mon[31381]: osdmap e39: 6 total, 6 up, 6 in 2024-03-08T20:53:15.866 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:53:15 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-08T20:53:16.020 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:53:15 smithi149 ceph-mon[35042]: purged_snaps scrub starts 2024-03-08T20:53:16.020 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:53:15 smithi149 ceph-mon[35042]: purged_snaps scrub ok 2024-03-08T20:53:16.020 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:53:15 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-08T20:53:16.020 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:53:15 smithi149 ceph-mon[35042]: osd.5 [v2:172.21.15.204:6808/3143887820,v1:172.21.15.204:6809/3143887820] boot 2024-03-08T20:53:16.020 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:53:15 smithi149 ceph-mon[35042]: osdmap e39: 6 total, 6 up, 6 in 2024-03-08T20:53:16.020 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:53:15 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-08T20:53:17.092 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:53:16 smithi107 ceph-mon[31381]: osdmap e40: 6 total, 6 up, 6 in 2024-03-08T20:53:17.093 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:53:16 smithi107 ceph-mon[31381]: pgmap v178: 1 pgs: 1 remapped+peering; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-03-08T20:53:17.093 INFO:teuthology.orchestra.run.smithi107.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-03-08T20:53:17.093 INFO:teuthology.orchestra.run.smithi107.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-03-08T20:53:17.093 INFO:teuthology.orchestra.run.smithi107.stderr: stderr: 10+0 records in 2024-03-08T20:53:17.093 INFO:teuthology.orchestra.run.smithi107.stderr:10+0 records out 2024-03-08T20:53:17.093 INFO:teuthology.orchestra.run.smithi107.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0332265 s, 316 MB/s 2024-03-08T20:53:17.093 INFO:teuthology.orchestra.run.smithi107.stderr:--> Zapping successful for: 2024-03-08T20:53:17.133 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:53:16 smithi204 ceph-mon[34935]: osdmap e40: 6 total, 6 up, 6 in 2024-03-08T20:53:17.133 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:53:16 smithi204 ceph-mon[34935]: pgmap v178: 1 pgs: 1 remapped+peering; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-03-08T20:53:17.269 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:53:16 smithi149 ceph-mon[35042]: osdmap e40: 6 total, 6 up, 6 in 2024-03-08T20:53:17.269 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:53:16 smithi149 ceph-mon[35042]: pgmap v178: 1 pgs: 1 remapped+peering; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-03-08T20:53:17.955 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dc20f0f8-dd8c-11ee-95c6-87774f69a715 -- ceph orch daemon add osd smithi107:vg_nvme/lv_2 2024-03-08T20:53:18.133 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:53:17 smithi204 ceph-mon[34935]: osdmap e41: 6 total, 6 up, 6 in 2024-03-08T20:53:18.221 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:53:17 smithi107 ceph-mon[31381]: osdmap e41: 6 total, 6 up, 6 in 2024-03-08T20:53:18.269 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:53:17 smithi149 ceph-mon[35042]: osdmap e41: 6 total, 6 up, 6 in 2024-03-08T20:53:18.366 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/mon.a/config 2024-03-08T20:53:19.133 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:53:18 smithi204 ceph-mon[34935]: osdmap e42: 6 total, 6 up, 6 in 2024-03-08T20:53:19.133 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:53:18 smithi204 ceph-mon[34935]: Detected new or changed devices on smithi204 2024-03-08T20:53:19.133 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:53:18 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:53:19.133 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:53:18 smithi204 ceph-mon[34935]: pgmap v181: 1 pgs: 1 remapped+peering; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-03-08T20:53:19.134 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:53:18 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:53:19.134 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:53:18 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-08T20:53:19.134 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:53:18 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-08T20:53:19.134 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:53:18 smithi204 ceph-mon[34935]: Adjusting osd_memory_target on smithi204 to 8545M 2024-03-08T20:53:19.134 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:53:18 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:53:19.134 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:53:18 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:53:19.134 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:53:18 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T20:53:19.134 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:53:18 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:53:19.222 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:53:18 smithi107 ceph-mon[31381]: osdmap e42: 6 total, 6 up, 6 in 2024-03-08T20:53:19.222 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:53:18 smithi107 ceph-mon[31381]: Detected new or changed devices on smithi204 2024-03-08T20:53:19.222 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:53:18 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:53:19.222 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:53:18 smithi107 ceph-mon[31381]: pgmap v181: 1 pgs: 1 remapped+peering; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-03-08T20:53:19.222 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:53:18 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:53:19.222 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:53:18 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-08T20:53:19.222 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:53:18 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-08T20:53:19.222 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:53:18 smithi107 ceph-mon[31381]: Adjusting osd_memory_target on smithi204 to 8545M 2024-03-08T20:53:19.223 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:53:18 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:53:19.223 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:53:18 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:53:19.223 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:53:18 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T20:53:19.223 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:53:18 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:53:19.270 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:53:18 smithi149 ceph-mon[35042]: osdmap e42: 6 total, 6 up, 6 in 2024-03-08T20:53:19.270 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:53:18 smithi149 ceph-mon[35042]: Detected new or changed devices on smithi204 2024-03-08T20:53:19.270 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:53:18 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:53:19.270 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:53:18 smithi149 ceph-mon[35042]: pgmap v181: 1 pgs: 1 remapped+peering; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-03-08T20:53:19.270 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:53:18 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:53:19.270 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:53:18 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-08T20:53:19.270 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:53:18 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-08T20:53:19.270 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:53:18 smithi149 ceph-mon[35042]: Adjusting osd_memory_target on smithi204 to 8545M 2024-03-08T20:53:19.270 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:53:18 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:53:19.270 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:53:18 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:53:19.270 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:53:18 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T20:53:19.270 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:53:18 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:53:19.972 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:53:19 smithi107 ceph-mon[31381]: osdmap e43: 6 total, 6 up, 6 in 2024-03-08T20:53:20.133 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:53:19 smithi204 ceph-mon[34935]: osdmap e43: 6 total, 6 up, 6 in 2024-03-08T20:53:20.269 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:53:19 smithi149 ceph-mon[35042]: osdmap e43: 6 total, 6 up, 6 in 2024-03-08T20:53:20.520 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:53:20.532+0000 7f5bb9000700 1 -- 172.21.15.107:0/738511867 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5bb410a000 msgr2=0x7f5bb41108b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:53:20.520 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:53:20.532+0000 7f5bb9000700 1 --2- 172.21.15.107:0/738511867 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5bb410a000 0x7f5bb41108b0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f5bac00b0d0 tx=0x7f5bac00b3e0 comp rx=0 tx=0).stop 2024-03-08T20:53:20.520 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:53:20.532+0000 7f5bb9000700 1 -- 172.21.15.107:0/738511867 shutdown_connections 2024-03-08T20:53:20.520 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:53:20.532+0000 7f5bb9000700 1 --2- 172.21.15.107:0/738511867 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5bb410a000 0x7f5bb41108b0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:53:20.520 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:53:20.532+0000 7f5bb9000700 1 --2- 172.21.15.107:0/738511867 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5bb4108460 0x7f5bb4108880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:53:20.520 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:53:20.532+0000 7f5bb9000700 1 --2- 172.21.15.107:0/738511867 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5bb4109660 0x7f5bb4109ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:53:20.520 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:53:20.533+0000 7f5bb9000700 1 -- 172.21.15.107:0/738511867 >> 172.21.15.107:0/738511867 conn(0x7f5bb4077430 msgr2=0x7f5bb4079870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:53:20.521 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:53:20.533+0000 7f5bb9000700 1 -- 172.21.15.107:0/738511867 shutdown_connections 2024-03-08T20:53:20.521 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:53:20.533+0000 7f5bb9000700 1 -- 172.21.15.107:0/738511867 wait complete. 2024-03-08T20:53:20.522 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:53:20.534+0000 7f5bb9000700 1 Processor -- start 2024-03-08T20:53:20.522 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:53:20.534+0000 7f5bb9000700 1 -- start start 2024-03-08T20:53:20.522 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:53:20.534+0000 7f5bb9000700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5bb4108460 0x7f5bb419e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:53:20.522 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:53:20.534+0000 7f5bb9000700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5bb4109660 0x7f5bb419e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:53:20.523 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:53:20.534+0000 7f5bb9000700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5bb410a000 0x7f5bb41a5a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:53:20.523 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:53:20.534+0000 7f5bb9000700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bb4113f30 con 0x7f5bb410a000 2024-03-08T20:53:20.523 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:53:20.534+0000 7f5bb9000700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bb4113db0 con 0x7f5bb4109660 2024-03-08T20:53:20.523 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:53:20.534+0000 7f5bb9000700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bb41140b0 con 0x7f5bb4108460 2024-03-08T20:53:20.523 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:53:20.534+0000 7f5bb2800700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5bb410a000 0x7f5bb41a5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:53:20.523 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:53:20.534+0000 7f5bb2800700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5bb410a000 0x7f5bb41a5a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48010/0 (socket says 172.21.15.107:48010) 2024-03-08T20:53:20.523 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:53:20.534+0000 7f5bb2800700 1 -- 172.21.15.107:0/1927864754 learned_addr learned my addr 172.21.15.107:0/1927864754 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T20:53:20.523 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:53:20.535+0000 7f5bb1e00700 1 --2- 172.21.15.107:0/1927864754 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5bb4108460 0x7f5bb419e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:53:20.523 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:53:20.535+0000 7f5bb1400700 1 --2- 172.21.15.107:0/1927864754 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5bb4109660 0x7f5bb419e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:53:20.523 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:53:20.535+0000 7f5bb2800700 1 -- 172.21.15.107:0/1927864754 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5bb4108460 msgr2=0x7f5bb419e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:53:20.523 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:53:20.535+0000 7f5bb2800700 1 --2- 172.21.15.107:0/1927864754 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5bb4108460 0x7f5bb419e4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:53:20.523 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:53:20.535+0000 7f5bb2800700 1 -- 172.21.15.107:0/1927864754 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5bb4109660 msgr2=0x7f5bb419e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:53:20.523 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:53:20.535+0000 7f5bb2800700 1 --2- 172.21.15.107:0/1927864754 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5bb4109660 0x7f5bb419e9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:53:20.523 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:53:20.535+0000 7f5bb2800700 1 -- 172.21.15.107:0/1927864754 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bb41a6110 con 0x7f5bb410a000 2024-03-08T20:53:20.524 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:53:20.535+0000 7f5bb2800700 1 --2- 172.21.15.107:0/1927864754 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5bb410a000 0x7f5bb41a5a70 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f5bac004cb0 tx=0x7f5bac004d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:53:20.524 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:53:20.536+0000 7f5baaa00700 1 -- 172.21.15.107:0/1927864754 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bac01f030 con 0x7f5bb410a000 2024-03-08T20:53:20.524 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:53:20.536+0000 7f5baaa00700 1 -- 172.21.15.107:0/1927864754 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f5bac007ac0 con 0x7f5bb410a000 2024-03-08T20:53:20.524 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:53:20.536+0000 7f5bb9000700 1 -- 172.21.15.107:0/1927864754 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5bb41a63a0 con 0x7f5bb410a000 2024-03-08T20:53:20.524 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:53:20.536+0000 7f5baaa00700 1 -- 172.21.15.107:0/1927864754 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bac01b690 con 0x7f5bb410a000 2024-03-08T20:53:20.525 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:53:20.536+0000 7f5bb9000700 1 -- 172.21.15.107:0/1927864754 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bb41a6850 con 0x7f5bb410a000 2024-03-08T20:53:20.525 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:53:20.537+0000 7f5bb9000700 1 -- 172.21.15.107:0/1927864754 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5bb4068aa0 con 0x7f5bb410a000 2024-03-08T20:53:20.526 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:53:20.539+0000 7f5baaa00700 1 -- 172.21.15.107:0/1927864754 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f5bac022030 con 0x7f5bb410a000 2024-03-08T20:53:20.527 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:53:20.540+0000 7f5baaa00700 1 --2- 172.21.15.107:0/1927864754 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f5b980a2910 0x7f5b980a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:53:20.527 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:53:20.540+0000 7f5baaa00700 1 -- 172.21.15.107:0/1927864754 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 4533+0+0 (secure 0 0 0) 0x7f5bac00f020 con 0x7f5bb410a000 2024-03-08T20:53:20.527 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:53:20.540+0000 7f5bb1e00700 1 --2- 172.21.15.107:0/1927864754 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f5b980a2910 0x7f5b980a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:53:20.528 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:53:20.541+0000 7f5bb1e00700 1 --2- 172.21.15.107:0/1927864754 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f5b980a2910 0x7f5b980a4dd0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f5ba4005950 tx=0x7f5ba40058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:53:20.533 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:53:20.547+0000 7f5baaa00700 1 -- 172.21.15.107:0/1927864754 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f5bac029020 con 0x7f5bb410a000 2024-03-08T20:53:20.767 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:53:20.780+0000 7f5bb9000700 1 -- 172.21.15.107:0/1927864754 --> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f5bb4062220 con 0x7f5b980a2910 2024-03-08T20:53:21.019 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:53:20 smithi149 ceph-mon[35042]: pgmap v183: 1 pgs: 1 remapped+peering; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-03-08T20:53:21.019 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:53:20 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-08T20:53:21.020 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:53:20 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-08T20:53:21.020 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:53:20 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:53:21.133 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:53:20 smithi204 ceph-mon[34935]: pgmap v183: 1 pgs: 1 remapped+peering; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-03-08T20:53:21.133 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:53:20 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-08T20:53:21.133 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:53:20 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-08T20:53:21.133 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:53:20 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:53:21.222 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:53:20 smithi107 ceph-mon[31381]: pgmap v183: 1 pgs: 1 remapped+peering; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-03-08T20:53:21.222 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:53:20 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-08T20:53:21.222 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:53:20 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-08T20:53:21.222 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:53:20 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:53:21.952 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:53:21 smithi107 ceph-mon[31381]: from='client.14352 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-08T20:53:22.133 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:53:21 smithi204 ceph-mon[34935]: from='client.14352 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-08T20:53:22.269 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:53:21 smithi149 ceph-mon[35042]: from='client.14352 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-08T20:53:23.132 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:53:22 smithi204 ceph-mon[34935]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-08T20:53:23.222 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:53:22 smithi107 ceph-mon[31381]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-08T20:53:23.269 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:53:22 smithi149 ceph-mon[35042]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-08T20:53:25.633 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:53:25 smithi204 ceph-mon[34935]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-03-08T20:53:25.633 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:53:25 smithi204 ceph-mon[34935]: from='client.? 172.21.15.107:0/23501352' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d2e05a56-be4f-4435-9b76-ff9aa01e4ac3"}]: dispatch 2024-03-08T20:53:25.633 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:53:25 smithi204 ceph-mon[34935]: from='client.? 172.21.15.107:0/23501352' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d2e05a56-be4f-4435-9b76-ff9aa01e4ac3"}]': finished 2024-03-08T20:53:25.633 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:53:25 smithi204 ceph-mon[34935]: osdmap e44: 7 total, 6 up, 7 in 2024-03-08T20:53:25.633 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:53:25 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-08T20:53:25.721 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:53:25 smithi107 ceph-mon[31381]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-03-08T20:53:25.722 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:53:25 smithi107 ceph-mon[31381]: from='client.? 172.21.15.107:0/23501352' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d2e05a56-be4f-4435-9b76-ff9aa01e4ac3"}]: dispatch 2024-03-08T20:53:25.722 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:53:25 smithi107 ceph-mon[31381]: from='client.? 172.21.15.107:0/23501352' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d2e05a56-be4f-4435-9b76-ff9aa01e4ac3"}]': finished 2024-03-08T20:53:25.722 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:53:25 smithi107 ceph-mon[31381]: osdmap e44: 7 total, 6 up, 7 in 2024-03-08T20:53:25.722 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:53:25 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-08T20:53:25.769 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:53:25 smithi149 ceph-mon[35042]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-03-08T20:53:25.770 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:53:25 smithi149 ceph-mon[35042]: from='client.? 172.21.15.107:0/23501352' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d2e05a56-be4f-4435-9b76-ff9aa01e4ac3"}]: dispatch 2024-03-08T20:53:25.770 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:53:25 smithi149 ceph-mon[35042]: from='client.? 172.21.15.107:0/23501352' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d2e05a56-be4f-4435-9b76-ff9aa01e4ac3"}]': finished 2024-03-08T20:53:25.770 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:53:25 smithi149 ceph-mon[35042]: osdmap e44: 7 total, 6 up, 7 in 2024-03-08T20:53:25.770 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:53:25 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-08T20:53:26.632 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:53:26 smithi204 ceph-mon[34935]: from='client.? 172.21.15.107:0/3509742386' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-08T20:53:26.721 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:53:26 smithi107 ceph-mon[31381]: from='client.? 172.21.15.107:0/3509742386' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-08T20:53:26.769 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:53:26 smithi149 ceph-mon[35042]: from='client.? 172.21.15.107:0/3509742386' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-08T20:53:27.472 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:53:27 smithi107 ceph-mon[31381]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-08T20:53:27.632 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:53:27 smithi204 ceph-mon[34935]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-08T20:53:27.769 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:53:27 smithi149 ceph-mon[35042]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-08T20:53:29.632 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:53:29 smithi204 ceph-mon[34935]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-03-08T20:53:29.721 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:53:29 smithi107 ceph-mon[31381]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-03-08T20:53:29.769 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:53:29 smithi149 ceph-mon[35042]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-03-08T20:53:31.471 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:53:31 smithi107 ceph-mon[31381]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-08T20:53:31.632 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:53:31 smithi204 ceph-mon[34935]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-08T20:53:31.769 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:53:31 smithi149 ceph-mon[35042]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-08T20:53:33.471 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:53:33 smithi107 ceph-mon[31381]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-03-08T20:53:33.632 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:53:33 smithi204 ceph-mon[34935]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-03-08T20:53:33.769 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:53:33 smithi149 ceph-mon[35042]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-03-08T20:53:34.632 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:53:34 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-03-08T20:53:34.633 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:53:34 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:53:34.722 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:53:34 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-03-08T20:53:34.722 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:53:34 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:53:34.769 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:53:34 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-03-08T20:53:34.769 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:53:34 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:53:35.383 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:53:35 smithi204 ceph-mon[34935]: Deploying daemon osd.6 on smithi107 2024-03-08T20:53:35.383 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:53:35 smithi204 ceph-mon[34935]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-03-08T20:53:35.432 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:53:35 smithi107 ceph-mon[31381]: Deploying daemon osd.6 on smithi107 2024-03-08T20:53:35.432 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:53:35 smithi107 ceph-mon[31381]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-03-08T20:53:35.769 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:53:35 smithi149 ceph-mon[35042]: Deploying daemon osd.6 on smithi107 2024-03-08T20:53:35.769 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:53:35 smithi149 ceph-mon[35042]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-03-08T20:53:37.472 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:53:37 smithi107 ceph-mon[31381]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-03-08T20:53:37.632 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:53:37 smithi204 ceph-mon[34935]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-03-08T20:53:37.769 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:53:37 smithi149 ceph-mon[35042]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-03-08T20:53:39.370 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:53:39 smithi107 ceph-mon[31381]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-03-08T20:53:39.632 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:53:39 smithi204 ceph-mon[34935]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-03-08T20:53:39.769 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:53:39 smithi149 ceph-mon[35042]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-03-08T20:53:41.632 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:53:41 smithi204 ceph-mon[34935]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-03-08T20:53:41.721 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:53:41 smithi107 ceph-mon[31381]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-03-08T20:53:41.769 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:53:41 smithi149 ceph-mon[35042]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-03-08T20:53:43.599 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:53:43 smithi107 ceph-mon[31381]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-03-08T20:53:43.599 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:53:43 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:53:43.599 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:53:43 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:53:43.599 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:53:43 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T20:53:43.632 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:53:43 smithi204 ceph-mon[34935]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-03-08T20:53:43.633 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:53:43 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:53:43.633 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:53:43 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:53:43.633 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:53:43 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T20:53:43.769 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:53:43 smithi149 ceph-mon[35042]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-03-08T20:53:43.769 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:53:43 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:53:43.770 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:53:43 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:53:43.770 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:53:43 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T20:53:45.634 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:53:45 smithi204 ceph-mon[34935]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-03-08T20:53:45.721 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:53:45 smithi107 ceph-mon[31381]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-03-08T20:53:45.769 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:53:45 smithi149 ceph-mon[35042]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-03-08T20:53:47.382 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:53:47 smithi107 ceph-mon[31381]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-03-08T20:53:47.382 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:53:47 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:53:47.382 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:53:47 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:53:47.382 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:53:47 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:53:47.383 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:53:47 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T20:53:47.383 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:53:47 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:53:47.383 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:53:47 smithi107 ceph-mon[31381]: from='osd.6 [v2:172.21.15.107:6818/2912452950,v1:172.21.15.107:6819/2912452950]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-03-08T20:53:47.514 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:53:47.525+0000 7f5baaa00700 1 -- 172.21.15.107:0/1927864754 <== mgr.14152 v2:172.21.15.107:6800/3376636618 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f5bb4062220 con 0x7f5b980a2910 2024-03-08T20:53:47.514 INFO:teuthology.orchestra.run.smithi107.stdout:Created osd(s) 6 on host 'smithi107' 2024-03-08T20:53:47.517 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:53:47.529+0000 7f5bb9000700 1 -- 172.21.15.107:0/1927864754 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f5b980a2910 msgr2=0x7f5b980a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:53:47.517 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:53:47.529+0000 7f5bb9000700 1 --2- 172.21.15.107:0/1927864754 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f5b980a2910 0x7f5b980a4dd0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f5ba4005950 tx=0x7f5ba40058e0 comp rx=0 tx=0).stop 2024-03-08T20:53:47.517 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:53:47.529+0000 7f5bb9000700 1 -- 172.21.15.107:0/1927864754 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5bb410a000 msgr2=0x7f5bb41a5a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:53:47.517 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:53:47.529+0000 7f5bb9000700 1 --2- 172.21.15.107:0/1927864754 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5bb410a000 0x7f5bb41a5a70 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f5bac004cb0 tx=0x7f5bac004d90 comp rx=0 tx=0).stop 2024-03-08T20:53:47.517 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:53:47.529+0000 7f5bb9000700 1 -- 172.21.15.107:0/1927864754 shutdown_connections 2024-03-08T20:53:47.518 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:53:47.529+0000 7f5bb9000700 1 --2- 172.21.15.107:0/1927864754 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5bb4108460 0x7f5bb419e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:53:47.518 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:53:47.529+0000 7f5bb9000700 1 --2- 172.21.15.107:0/1927864754 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f5b980a2910 0x7f5b980a4dd0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:53:47.518 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:53:47.529+0000 7f5bb9000700 1 --2- 172.21.15.107:0/1927864754 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5bb410a000 0x7f5bb41a5a70 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:53:47.518 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:53:47.529+0000 7f5bb9000700 1 --2- 172.21.15.107:0/1927864754 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5bb4109660 0x7f5bb419e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:53:47.518 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:53:47.529+0000 7f5bb9000700 1 -- 172.21.15.107:0/1927864754 >> 172.21.15.107:0/1927864754 conn(0x7f5bb4077430 msgr2=0x7f5bb410c890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:53:47.518 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:53:47.530+0000 7f5bb9000700 1 -- 172.21.15.107:0/1927864754 shutdown_connections 2024-03-08T20:53:47.518 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:53:47.530+0000 7f5bb9000700 1 -- 172.21.15.107:0/1927864754 wait complete. 2024-03-08T20:53:47.632 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:53:47 smithi204 ceph-mon[34935]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-03-08T20:53:47.633 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:53:47 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:53:47.633 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:53:47 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:53:47.633 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:53:47 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:53:47.633 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:53:47 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T20:53:47.633 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:53:47 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:53:47.633 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:53:47 smithi204 ceph-mon[34935]: from='osd.6 [v2:172.21.15.107:6818/2912452950,v1:172.21.15.107:6819/2912452950]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-03-08T20:53:47.769 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:53:47 smithi149 ceph-mon[35042]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-03-08T20:53:47.769 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:53:47 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:53:47.769 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:53:47 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:53:47.769 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:53:47 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:53:47.769 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:53:47 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T20:53:47.769 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:53:47 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:53:47.770 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:53:47 smithi149 ceph-mon[35042]: from='osd.6 [v2:172.21.15.107:6818/2912452950,v1:172.21.15.107:6819/2912452950]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-03-08T20:53:48.338 DEBUG:teuthology.orchestra.run.smithi107:osd.6> sudo journalctl -f -n 0 -u ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715@osd.6.service 2024-03-08T20:53:48.344 INFO:tasks.cephadm:Deploying osd.7 on smithi149 with /dev/vg_nvme/lv_2... 2024-03-08T20:53:48.344 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dc20f0f8-dd8c-11ee-95c6-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-03-08T20:53:48.626 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:53:48 smithi107 ceph-mon[31381]: from='osd.6 [v2:172.21.15.107:6818/2912452950,v1:172.21.15.107:6819/2912452950]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-03-08T20:53:48.626 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:53:48 smithi107 ceph-mon[31381]: osdmap e45: 7 total, 6 up, 7 in 2024-03-08T20:53:48.626 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:53:48 smithi107 ceph-mon[31381]: from='osd.6 [v2:172.21.15.107:6818/2912452950,v1:172.21.15.107:6819/2912452950]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-03-08T20:53:48.626 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:53:48 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-08T20:53:48.626 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:53:48 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:53:48.626 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:53:48 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:53:48.626 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:53:48 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T20:53:48.632 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:53:48 smithi204 ceph-mon[34935]: from='osd.6 [v2:172.21.15.107:6818/2912452950,v1:172.21.15.107:6819/2912452950]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-03-08T20:53:48.632 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:53:48 smithi204 ceph-mon[34935]: osdmap e45: 7 total, 6 up, 7 in 2024-03-08T20:53:48.632 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:53:48 smithi204 ceph-mon[34935]: from='osd.6 [v2:172.21.15.107:6818/2912452950,v1:172.21.15.107:6819/2912452950]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-03-08T20:53:48.632 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:53:48 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-08T20:53:48.633 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:53:48 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:53:48.633 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:53:48 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:53:48.633 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:53:48 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T20:53:48.769 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:53:48 smithi149 ceph-mon[35042]: from='osd.6 [v2:172.21.15.107:6818/2912452950,v1:172.21.15.107:6819/2912452950]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-03-08T20:53:48.769 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:53:48 smithi149 ceph-mon[35042]: osdmap e45: 7 total, 6 up, 7 in 2024-03-08T20:53:48.769 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:53:48 smithi149 ceph-mon[35042]: from='osd.6 [v2:172.21.15.107:6818/2912452950,v1:172.21.15.107:6819/2912452950]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-03-08T20:53:48.769 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:53:48 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-08T20:53:48.769 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:53:48 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:53:48.769 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:53:48 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:53:48.769 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:53:48 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T20:53:48.774 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/mon.b/config 2024-03-08T20:53:49.222 INFO:journalctl@ceph.osd.6.smithi107.stdout:Mar 08 20:53:48 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-6[59834]: 2024-03-08T20:53:48.933+0000 7f613aa00700 -1 osd.6 0 waiting for initial osdmap 2024-03-08T20:53:49.222 INFO:journalctl@ceph.osd.6.smithi107.stdout:Mar 08 20:53:48 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-6[59834]: 2024-03-08T20:53:48.959+0000 7f6130800700 -1 osd.6 46 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-08T20:53:49.519 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:53:49 smithi149 ceph-mon[35042]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-03-08T20:53:49.519 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:53:49 smithi149 ceph-mon[35042]: from='osd.6 [v2:172.21.15.107:6818/2912452950,v1:172.21.15.107:6819/2912452950]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]': finished 2024-03-08T20:53:49.519 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:53:49 smithi149 ceph-mon[35042]: osdmap e46: 7 total, 6 up, 7 in 2024-03-08T20:53:49.519 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:53:49 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-08T20:53:49.519 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:53:49 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-08T20:53:49.519 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:53:49 smithi149 ceph-mon[35042]: from='osd.6 [v2:172.21.15.107:6818/2912452950,v1:172.21.15.107:6819/2912452950]' entity='osd.6' 2024-03-08T20:53:49.632 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:53:49 smithi204 ceph-mon[34935]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-03-08T20:53:49.632 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:53:49 smithi204 ceph-mon[34935]: from='osd.6 [v2:172.21.15.107:6818/2912452950,v1:172.21.15.107:6819/2912452950]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]': finished 2024-03-08T20:53:49.632 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:53:49 smithi204 ceph-mon[34935]: osdmap e46: 7 total, 6 up, 7 in 2024-03-08T20:53:49.633 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:53:49 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-08T20:53:49.633 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:53:49 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-08T20:53:49.633 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:53:49 smithi204 ceph-mon[34935]: from='osd.6 [v2:172.21.15.107:6818/2912452950,v1:172.21.15.107:6819/2912452950]' entity='osd.6' 2024-03-08T20:53:49.722 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:53:49 smithi107 ceph-mon[31381]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-03-08T20:53:49.722 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:53:49 smithi107 ceph-mon[31381]: from='osd.6 [v2:172.21.15.107:6818/2912452950,v1:172.21.15.107:6819/2912452950]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]': finished 2024-03-08T20:53:49.722 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:53:49 smithi107 ceph-mon[31381]: osdmap e46: 7 total, 6 up, 7 in 2024-03-08T20:53:49.722 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:53:49 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-08T20:53:49.722 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:53:49 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-08T20:53:49.722 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:53:49 smithi107 ceph-mon[31381]: from='osd.6 [v2:172.21.15.107:6818/2912452950,v1:172.21.15.107:6819/2912452950]' entity='osd.6' 2024-03-08T20:53:50.401 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:53:50 smithi107 ceph-mon[31381]: purged_snaps scrub starts 2024-03-08T20:53:50.401 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:53:50 smithi107 ceph-mon[31381]: purged_snaps scrub ok 2024-03-08T20:53:50.401 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:53:50 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-08T20:53:50.402 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:53:50 smithi107 ceph-mon[31381]: osd.6 [v2:172.21.15.107:6818/2912452950,v1:172.21.15.107:6819/2912452950] boot 2024-03-08T20:53:50.402 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:53:50 smithi107 ceph-mon[31381]: osdmap e47: 7 total, 7 up, 7 in 2024-03-08T20:53:50.402 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:53:50 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-08T20:53:50.624 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:53:50 smithi149 ceph-mon[35042]: purged_snaps scrub starts 2024-03-08T20:53:50.624 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:53:50 smithi149 ceph-mon[35042]: purged_snaps scrub ok 2024-03-08T20:53:50.624 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:53:50 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-08T20:53:50.624 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:53:50 smithi149 ceph-mon[35042]: osd.6 [v2:172.21.15.107:6818/2912452950,v1:172.21.15.107:6819/2912452950] boot 2024-03-08T20:53:50.624 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:53:50 smithi149 ceph-mon[35042]: osdmap e47: 7 total, 7 up, 7 in 2024-03-08T20:53:50.624 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:53:50 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-08T20:53:50.632 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:53:50 smithi204 ceph-mon[34935]: purged_snaps scrub starts 2024-03-08T20:53:50.632 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:53:50 smithi204 ceph-mon[34935]: purged_snaps scrub ok 2024-03-08T20:53:50.632 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:53:50 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-08T20:53:50.632 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:53:50 smithi204 ceph-mon[34935]: osd.6 [v2:172.21.15.107:6818/2912452950,v1:172.21.15.107:6819/2912452950] boot 2024-03-08T20:53:50.632 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:53:50 smithi204 ceph-mon[34935]: osdmap e47: 7 total, 7 up, 7 in 2024-03-08T20:53:50.633 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:53:50 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-08T20:53:51.597 INFO:teuthology.orchestra.run.smithi149.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-03-08T20:53:51.598 INFO:teuthology.orchestra.run.smithi149.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-03-08T20:53:51.598 INFO:teuthology.orchestra.run.smithi149.stderr: stderr: 10+0 records in 2024-03-08T20:53:51.598 INFO:teuthology.orchestra.run.smithi149.stderr:10+0 records out 2024-03-08T20:53:51.598 INFO:teuthology.orchestra.run.smithi149.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0326772 s, 321 MB/s 2024-03-08T20:53:51.598 INFO:teuthology.orchestra.run.smithi149.stderr:--> Zapping successful for: 2024-03-08T20:53:51.655 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:53:51 smithi149 ceph-mon[35042]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-03-08T20:53:51.721 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:53:51 smithi107 ceph-mon[31381]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-03-08T20:53:51.883 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:53:51 smithi204 ceph-mon[34935]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-03-08T20:53:52.461 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dc20f0f8-dd8c-11ee-95c6-87774f69a715 -- ceph orch daemon add osd smithi149:vg_nvme/lv_2 2024-03-08T20:53:52.632 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:53:52 smithi107 ceph-mon[31381]: osdmap e48: 7 total, 7 up, 7 in 2024-03-08T20:53:52.769 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:53:52 smithi149 ceph-mon[35042]: osdmap e48: 7 total, 7 up, 7 in 2024-03-08T20:53:52.882 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:53:52 smithi204 ceph-mon[34935]: osdmap e48: 7 total, 7 up, 7 in 2024-03-08T20:53:52.897 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/mon.b/config 2024-03-08T20:53:53.722 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:53:53 smithi107 ceph-mon[31381]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-03-08T20:53:53.722 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:53:53 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:53:53.722 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:53:53 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:53:53.722 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:53:53 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-08T20:53:53.722 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:53:53 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-08T20:53:53.722 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:53:53 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-08T20:53:53.722 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:53:53 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:53:53.723 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:53:53 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:53:53.723 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:53:53 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T20:53:53.723 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:53:53 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:53:53.769 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:53:53 smithi149 ceph-mon[35042]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-03-08T20:53:53.770 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:53:53 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:53:53.770 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:53:53 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:53:53.770 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:53:53 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-08T20:53:53.770 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:53:53 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-08T20:53:53.770 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:53:53 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-08T20:53:53.770 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:53:53 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:53:53.770 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:53:53 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:53:53.770 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:53:53 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T20:53:53.770 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:53:53 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:53:53.883 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:53:53 smithi204 ceph-mon[34935]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-03-08T20:53:53.883 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:53:53 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:53:53.883 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:53:53 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:53:53.883 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:53:53 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-08T20:53:53.883 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:53:53 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-08T20:53:53.883 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:53:53 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-08T20:53:53.883 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:53:53 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:53:53.883 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:53:53 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:53:53.883 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:53:53 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T20:53:53.883 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:53:53 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:53:54.644 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:53:54 smithi149 ceph-mon[35042]: Detected new or changed devices on smithi107 2024-03-08T20:53:54.644 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:53:54 smithi149 ceph-mon[35042]: Adjusting osd_memory_target on smithi107 to 5696M 2024-03-08T20:53:54.721 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:53:54 smithi107 ceph-mon[31381]: Detected new or changed devices on smithi107 2024-03-08T20:53:54.721 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:53:54 smithi107 ceph-mon[31381]: Adjusting osd_memory_target on smithi107 to 5696M 2024-03-08T20:53:54.882 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:53:54 smithi204 ceph-mon[34935]: Detected new or changed devices on smithi107 2024-03-08T20:53:54.882 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:53:54 smithi204 ceph-mon[34935]: Adjusting osd_memory_target on smithi107 to 5696M 2024-03-08T20:53:55.137 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:53:55.150+0000 7facefa00700 1 -- 172.21.15.149:0/165227199 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7face8104060 msgr2=0x7face8104480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:53:55.138 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:53:55.150+0000 7facefa00700 1 --2- 172.21.15.149:0/165227199 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7face8104060 0x7face8104480 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7facdc0099c0 tx=0x7facdc009cd0 comp rx=0 tx=0).stop 2024-03-08T20:53:55.138 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:53:55.150+0000 7facefa00700 1 -- 172.21.15.149:0/165227199 shutdown_connections 2024-03-08T20:53:55.138 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:53:55.150+0000 7facefa00700 1 --2- 172.21.15.149:0/165227199 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7face8105c00 0x7face810c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:53:55.138 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:53:55.150+0000 7facefa00700 1 --2- 172.21.15.149:0/165227199 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7face8104060 0x7face8104480 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:53:55.138 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:53:55.150+0000 7facefa00700 1 --2- 172.21.15.149:0/165227199 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7face8105260 0x7face81056c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:53:55.138 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:53:55.150+0000 7facefa00700 1 -- 172.21.15.149:0/165227199 >> 172.21.15.149:0/165227199 conn(0x7face80ff600 msgr2=0x7face8101a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:53:55.138 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:53:55.150+0000 7facefa00700 1 -- 172.21.15.149:0/165227199 shutdown_connections 2024-03-08T20:53:55.138 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:53:55.150+0000 7facefa00700 1 -- 172.21.15.149:0/165227199 wait complete. 2024-03-08T20:53:55.139 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:53:55.152+0000 7facefa00700 1 Processor -- start 2024-03-08T20:53:55.140 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:53:55.152+0000 7facefa00700 1 -- start start 2024-03-08T20:53:55.140 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:53:55.152+0000 7facefa00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7face8104060 0x7face819a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:53:55.140 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:53:55.152+0000 7facefa00700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7face8105260 0x7face819a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:53:55.140 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:53:55.152+0000 7facefa00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7face8105c00 0x7face81a1780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:53:55.140 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:53:55.152+0000 7facefa00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7face810fb50 con 0x7face8104060 2024-03-08T20:53:55.140 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:53:55.152+0000 7facefa00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7face810f9d0 con 0x7face8105c00 2024-03-08T20:53:55.140 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:53:55.152+0000 7facefa00700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7face810fcd0 con 0x7face8105260 2024-03-08T20:53:55.140 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:53:55.152+0000 7faceda00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7face8105c00 0x7face81a1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:53:55.140 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:53:55.152+0000 7faceda00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7face8105c00 0x7face81a1780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.149:46418/0 (socket says 172.21.15.149:46418) 2024-03-08T20:53:55.140 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:53:55.152+0000 7faceda00700 1 -- 172.21.15.149:0/290618903 learned_addr learned my addr 172.21.15.149:0/290618903 (peer_addr_for_me v2:172.21.15.149:0/0) 2024-03-08T20:53:55.140 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:53:55.153+0000 7faced000700 1 --2- 172.21.15.149:0/290618903 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7face8104060 0x7face819a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:53:55.140 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:53:55.153+0000 7face7e00700 1 --2- 172.21.15.149:0/290618903 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7face8105260 0x7face819a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:53:55.141 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:53:55.153+0000 7faceda00700 1 -- 172.21.15.149:0/290618903 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7face8105260 msgr2=0x7face819a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:53:55.141 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:53:55.153+0000 7faceda00700 1 --2- 172.21.15.149:0/290618903 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7face8105260 0x7face819a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:53:55.141 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:53:55.153+0000 7faceda00700 1 -- 172.21.15.149:0/290618903 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7face8104060 msgr2=0x7face819a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:53:55.141 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:53:55.153+0000 7faceda00700 1 --2- 172.21.15.149:0/290618903 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7face8104060 0x7face819a1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:53:55.141 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:53:55.153+0000 7faceda00700 1 -- 172.21.15.149:0/290618903 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7face81a1e20 con 0x7face8105c00 2024-03-08T20:53:55.141 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:53:55.153+0000 7faceda00700 1 --2- 172.21.15.149:0/290618903 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7face8105c00 0x7face81a1780 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7face0007ee0 tx=0x7face000d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:53:55.142 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:53:55.153+0000 7face5600700 1 -- 172.21.15.149:0/290618903 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7face000dbd0 con 0x7face8105c00 2024-03-08T20:53:55.142 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:53:55.154+0000 7face5600700 1 -- 172.21.15.149:0/290618903 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7face000dd70 con 0x7face8105c00 2024-03-08T20:53:55.142 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:53:55.154+0000 7facefa00700 1 -- 172.21.15.149:0/290618903 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7face81a2110 con 0x7face8105c00 2024-03-08T20:53:55.142 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:53:55.154+0000 7face5600700 1 -- 172.21.15.149:0/290618903 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7face0005a00 con 0x7face8105c00 2024-03-08T20:53:55.142 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:53:55.154+0000 7facefa00700 1 -- 172.21.15.149:0/290618903 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7face81a2650 con 0x7face8105c00 2024-03-08T20:53:55.142 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:53:55.155+0000 7facefa00700 1 -- 172.21.15.149:0/290618903 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7face8068aa0 con 0x7face8105c00 2024-03-08T20:53:55.144 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:53:55.157+0000 7face5600700 1 -- 172.21.15.149:0/290618903 <== mon.2 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 16) v1 ==== 137291+0+0 (secure 0 0 0) 0x7face001a020 con 0x7face8105c00 2024-03-08T20:53:55.145 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:53:55.158+0000 7face5600700 1 --2- 172.21.15.149:0/290618903 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7facd00a2910 0x7facd00a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:53:55.145 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:53:55.159+0000 7face5600700 1 -- 172.21.15.149:0/290618903 <== mon.2 v2:172.21.15.149:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 4965+0+0 (secure 0 0 0) 0x7face00cdbb0 con 0x7face8105c00 2024-03-08T20:53:55.145 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:53:55.159+0000 7faced000700 1 --2- 172.21.15.149:0/290618903 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7facd00a2910 0x7facd00a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:53:55.146 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:53:55.160+0000 7faced000700 1 --2- 172.21.15.149:0/290618903 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7facd00a2910 0x7facd00a4dd0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7facdc000c00 tx=0x7facdc005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:53:55.148 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:53:55.162+0000 7face5600700 1 -- 172.21.15.149:0/290618903 <== mon.2 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7face0010070 con 0x7face8105c00 2024-03-08T20:53:55.381 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:53:55.394+0000 7facefa00700 1 -- 172.21.15.149:0/290618903 --> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7face80622e0 con 0x7facd00a2910 2024-03-08T20:53:55.651 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:53:55 smithi149 ceph-mon[35042]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-03-08T20:53:55.651 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:53:55 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-08T20:53:55.651 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:53:55 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-08T20:53:55.652 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:53:55 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:53:55.721 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:53:55 smithi107 ceph-mon[31381]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-03-08T20:53:55.721 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:53:55 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-08T20:53:55.721 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:53:55 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-08T20:53:55.722 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:53:55 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:53:55.882 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:53:55 smithi204 ceph-mon[34935]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-03-08T20:53:55.882 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:53:55 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-08T20:53:55.882 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:53:55 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-08T20:53:55.882 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:53:55 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:53:56.625 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:53:56 smithi149 ceph-mon[35042]: from='client.34265 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-08T20:53:56.721 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:53:56 smithi107 ceph-mon[31381]: from='client.34265 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-08T20:53:56.882 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:53:56 smithi204 ceph-mon[34935]: from='client.34265 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-08T20:53:57.721 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:53:57 smithi107 ceph-mon[31381]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-03-08T20:53:57.769 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:53:57 smithi149 ceph-mon[35042]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-03-08T20:53:57.882 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:53:57 smithi204 ceph-mon[34935]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-03-08T20:53:59.721 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:53:59 smithi107 ceph-mon[31381]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-03-08T20:53:59.721 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:53:59 smithi107 ceph-mon[31381]: from='client.? 172.21.15.149:0/2908858510' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a7dd26ee-c118-4a9a-9e19-142c2680902d"}]: dispatch 2024-03-08T20:53:59.721 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:53:59 smithi107 ceph-mon[31381]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a7dd26ee-c118-4a9a-9e19-142c2680902d"}]: dispatch 2024-03-08T20:53:59.722 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:53:59 smithi107 ceph-mon[31381]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a7dd26ee-c118-4a9a-9e19-142c2680902d"}]': finished 2024-03-08T20:53:59.722 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:53:59 smithi107 ceph-mon[31381]: osdmap e49: 8 total, 7 up, 8 in 2024-03-08T20:53:59.722 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:53:59 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-08T20:53:59.769 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:53:59 smithi149 ceph-mon[35042]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-03-08T20:53:59.769 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:53:59 smithi149 ceph-mon[35042]: from='client.? 172.21.15.149:0/2908858510' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a7dd26ee-c118-4a9a-9e19-142c2680902d"}]: dispatch 2024-03-08T20:53:59.769 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:53:59 smithi149 ceph-mon[35042]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a7dd26ee-c118-4a9a-9e19-142c2680902d"}]: dispatch 2024-03-08T20:53:59.770 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:53:59 smithi149 ceph-mon[35042]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a7dd26ee-c118-4a9a-9e19-142c2680902d"}]': finished 2024-03-08T20:53:59.770 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:53:59 smithi149 ceph-mon[35042]: osdmap e49: 8 total, 7 up, 8 in 2024-03-08T20:53:59.770 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:53:59 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-08T20:53:59.882 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:53:59 smithi204 ceph-mon[34935]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-03-08T20:53:59.882 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:53:59 smithi204 ceph-mon[34935]: from='client.? 172.21.15.149:0/2908858510' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a7dd26ee-c118-4a9a-9e19-142c2680902d"}]: dispatch 2024-03-08T20:53:59.882 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:53:59 smithi204 ceph-mon[34935]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a7dd26ee-c118-4a9a-9e19-142c2680902d"}]: dispatch 2024-03-08T20:53:59.883 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:53:59 smithi204 ceph-mon[34935]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a7dd26ee-c118-4a9a-9e19-142c2680902d"}]': finished 2024-03-08T20:53:59.883 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:53:59 smithi204 ceph-mon[34935]: osdmap e49: 8 total, 7 up, 8 in 2024-03-08T20:53:59.883 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:53:59 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-08T20:54:00.721 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:54:00 smithi107 ceph-mon[31381]: from='client.? 172.21.15.149:0/3886302618' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-08T20:54:00.769 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:54:00 smithi149 ceph-mon[35042]: from='client.? 172.21.15.149:0/3886302618' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-08T20:54:00.882 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:54:00 smithi204 ceph-mon[34935]: from='client.? 172.21.15.149:0/3886302618' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-08T20:54:01.721 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:54:01 smithi107 ceph-mon[31381]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-03-08T20:54:01.769 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:54:01 smithi149 ceph-mon[35042]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-03-08T20:54:01.882 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:54:01 smithi204 ceph-mon[34935]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-03-08T20:54:03.721 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:54:03 smithi107 ceph-mon[31381]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-03-08T20:54:03.768 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:54:03 smithi149 ceph-mon[35042]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-03-08T20:54:03.882 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:54:03 smithi204 ceph-mon[34935]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-03-08T20:54:05.721 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:54:05 smithi107 ceph-mon[31381]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-03-08T20:54:05.768 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:54:05 smithi149 ceph-mon[35042]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-03-08T20:54:05.882 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:54:05 smithi204 ceph-mon[34935]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-03-08T20:54:07.721 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:54:07 smithi107 ceph-mon[31381]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-03-08T20:54:07.769 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:54:07 smithi149 ceph-mon[35042]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-03-08T20:54:07.882 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:54:07 smithi204 ceph-mon[34935]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-03-08T20:54:08.768 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:54:08 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-03-08T20:54:08.768 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:54:08 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:54:08.882 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:54:08 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-03-08T20:54:08.882 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:54:08 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:54:08.971 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:54:08 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-03-08T20:54:08.971 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:54:08 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:54:09.518 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:54:09 smithi149 ceph-mon[35042]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-03-08T20:54:09.519 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:54:09 smithi149 ceph-mon[35042]: Deploying daemon osd.7 on smithi149 2024-03-08T20:54:09.882 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:54:09 smithi204 ceph-mon[34935]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-03-08T20:54:09.882 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:54:09 smithi204 ceph-mon[34935]: Deploying daemon osd.7 on smithi149 2024-03-08T20:54:09.971 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:54:09 smithi107 ceph-mon[31381]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-03-08T20:54:09.971 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:54:09 smithi107 ceph-mon[31381]: Deploying daemon osd.7 on smithi149 2024-03-08T20:54:10.769 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:54:10 smithi149 ceph-mon[35042]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-03-08T20:54:10.882 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:54:10 smithi204 ceph-mon[34935]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-03-08T20:54:10.971 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:54:10 smithi107 ceph-mon[31381]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-03-08T20:54:13.371 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:54:13 smithi149 ceph-mon[35042]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-03-08T20:54:13.632 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:54:13 smithi204 ceph-mon[34935]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-03-08T20:54:13.674 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:54:13 smithi107 ceph-mon[31381]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-03-08T20:54:15.518 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:54:15 smithi149 ceph-mon[35042]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-03-08T20:54:15.632 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:54:15 smithi204 ceph-mon[34935]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-03-08T20:54:15.720 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:54:15 smithi107 ceph-mon[31381]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-03-08T20:54:17.721 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:54:17 smithi107 ceph-mon[31381]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-03-08T20:54:17.721 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:54:17 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:54:17.721 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:54:17 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:54:17.721 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:54:17 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T20:54:17.724 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:54:17 smithi149 ceph-mon[35042]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-03-08T20:54:17.724 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:54:17 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:54:17.724 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:54:17 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:54:17.725 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:54:17 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T20:54:17.882 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:54:17 smithi204 ceph-mon[34935]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-03-08T20:54:17.882 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:54:17 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:54:17.882 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:54:17 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:54:17.882 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:54:17 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T20:54:19.574 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:54:19 smithi149 ceph-mon[35042]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-03-08T20:54:19.720 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:54:19 smithi107 ceph-mon[31381]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-03-08T20:54:19.881 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:54:19 smithi204 ceph-mon[34935]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-03-08T20:54:21.133 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:54:20 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:54:21.133 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:54:20 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:54:21.133 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:54:20 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:54:21.134 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:54:20 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T20:54:21.134 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:54:20 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:54:21.134 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:54:20 smithi204 ceph-mon[34935]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-03-08T20:54:21.134 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:54:20 smithi204 ceph-mon[34935]: from='osd.7 [v2:172.21.15.149:6816/3835863809,v1:172.21.15.149:6817/3835863809]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-08T20:54:21.134 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:54:20 smithi204 ceph-mon[34935]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-08T20:54:21.221 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:54:20 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:54:21.221 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:54:20 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:54:21.222 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:54:20 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:54:21.222 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:54:20 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T20:54:21.222 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:54:20 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:54:21.222 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:54:20 smithi107 ceph-mon[31381]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-03-08T20:54:21.222 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:54:20 smithi107 ceph-mon[31381]: from='osd.7 [v2:172.21.15.149:6816/3835863809,v1:172.21.15.149:6817/3835863809]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-08T20:54:21.223 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:54:20 smithi107 ceph-mon[31381]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-08T20:54:21.257 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:54:20 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:54:21.257 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:54:20 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:54:21.257 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:54:20 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:54:21.257 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:54:20 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T20:54:21.257 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:54:20 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:54:21.258 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:54:20 smithi149 ceph-mon[35042]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-03-08T20:54:21.258 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:54:20 smithi149 ceph-mon[35042]: from='osd.7 [v2:172.21.15.149:6816/3835863809,v1:172.21.15.149:6817/3835863809]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-08T20:54:21.258 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:54:20 smithi149 ceph-mon[35042]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-08T20:54:21.490 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:54:21.502+0000 7face5600700 1 -- 172.21.15.149:0/290618903 <== mgr.14152 v2:172.21.15.107:6800/3376636618 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7face80622e0 con 0x7facd00a2910 2024-03-08T20:54:21.490 INFO:teuthology.orchestra.run.smithi149.stdout:Created osd(s) 7 on host 'smithi149' 2024-03-08T20:54:21.494 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:54:21.506+0000 7facefa00700 1 -- 172.21.15.149:0/290618903 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7facd00a2910 msgr2=0x7facd00a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:54:21.494 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:54:21.506+0000 7facefa00700 1 --2- 172.21.15.149:0/290618903 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7facd00a2910 0x7facd00a4dd0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7facdc000c00 tx=0x7facdc005fb0 comp rx=0 tx=0).stop 2024-03-08T20:54:21.494 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:54:21.506+0000 7facefa00700 1 -- 172.21.15.149:0/290618903 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7face8105c00 msgr2=0x7face81a1780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:54:21.494 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:54:21.506+0000 7facefa00700 1 --2- 172.21.15.149:0/290618903 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7face8105c00 0x7face81a1780 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7face0007ee0 tx=0x7face000d3b0 comp rx=0 tx=0).stop 2024-03-08T20:54:21.494 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:54:21.506+0000 7facefa00700 1 -- 172.21.15.149:0/290618903 shutdown_connections 2024-03-08T20:54:21.494 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:54:21.506+0000 7facefa00700 1 --2- 172.21.15.149:0/290618903 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7facd00a2910 0x7facd00a4dd0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:54:21.494 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:54:21.506+0000 7facefa00700 1 --2- 172.21.15.149:0/290618903 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7face8105c00 0x7face81a1780 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:54:21.494 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:54:21.506+0000 7facefa00700 1 --2- 172.21.15.149:0/290618903 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7face8104060 0x7face819a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:54:21.494 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:54:21.506+0000 7facefa00700 1 --2- 172.21.15.149:0/290618903 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7face8105260 0x7face819a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:54:21.494 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:54:21.506+0000 7facefa00700 1 -- 172.21.15.149:0/290618903 >> 172.21.15.149:0/290618903 conn(0x7face80ff600 msgr2=0x7face8108580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:54:21.494 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:54:21.507+0000 7facefa00700 1 -- 172.21.15.149:0/290618903 shutdown_connections 2024-03-08T20:54:21.495 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:54:21.507+0000 7facefa00700 1 -- 172.21.15.149:0/290618903 wait complete. 2024-03-08T20:54:22.161 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:54:21 smithi149 ceph-mon[35042]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-03-08T20:54:22.161 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:54:21 smithi149 ceph-mon[35042]: osdmap e50: 8 total, 7 up, 8 in 2024-03-08T20:54:22.161 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:54:21 smithi149 ceph-mon[35042]: from='osd.7 [v2:172.21.15.149:6816/3835863809,v1:172.21.15.149:6817/3835863809]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-03-08T20:54:22.161 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:54:21 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-08T20:54:22.161 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:54:21 smithi149 ceph-mon[35042]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-03-08T20:54:22.162 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:54:21 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:54:22.162 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:54:21 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T20:54:22.162 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:54:21 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:54:22.214 DEBUG:teuthology.orchestra.run.smithi149:osd.7> sudo journalctl -f -n 0 -u ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715@osd.7.service 2024-03-08T20:54:22.219 INFO:tasks.cephadm:Deploying osd.8 on smithi204 with /dev/vg_nvme/lv_2... 2024-03-08T20:54:22.219 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dc20f0f8-dd8c-11ee-95c6-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-03-08T20:54:22.221 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:54:21 smithi107 ceph-mon[31381]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-03-08T20:54:22.221 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:54:21 smithi107 ceph-mon[31381]: osdmap e50: 8 total, 7 up, 8 in 2024-03-08T20:54:22.221 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:54:21 smithi107 ceph-mon[31381]: from='osd.7 [v2:172.21.15.149:6816/3835863809,v1:172.21.15.149:6817/3835863809]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-03-08T20:54:22.221 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:54:21 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-08T20:54:22.221 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:54:21 smithi107 ceph-mon[31381]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-03-08T20:54:22.221 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:54:21 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:54:22.221 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:54:21 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T20:54:22.221 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:54:21 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:54:22.306 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:54:21 smithi204 ceph-mon[34935]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-03-08T20:54:22.307 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:54:21 smithi204 ceph-mon[34935]: osdmap e50: 8 total, 7 up, 8 in 2024-03-08T20:54:22.307 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:54:21 smithi204 ceph-mon[34935]: from='osd.7 [v2:172.21.15.149:6816/3835863809,v1:172.21.15.149:6817/3835863809]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-03-08T20:54:22.307 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:54:21 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-08T20:54:22.307 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:54:21 smithi204 ceph-mon[34935]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-03-08T20:54:22.307 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:54:21 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:54:22.307 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:54:21 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T20:54:22.307 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:54:21 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:54:22.658 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/mon.c/config 2024-03-08T20:54:23.132 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:54:22 smithi204 ceph-mon[34935]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]': finished 2024-03-08T20:54:23.132 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:54:22 smithi204 ceph-mon[34935]: osdmap e51: 8 total, 7 up, 8 in 2024-03-08T20:54:23.132 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:54:22 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-08T20:54:23.132 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:54:22 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-08T20:54:23.132 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:54:22 smithi204 ceph-mon[34935]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-03-08T20:54:23.221 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:54:22 smithi107 ceph-mon[31381]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]': finished 2024-03-08T20:54:23.221 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:54:22 smithi107 ceph-mon[31381]: osdmap e51: 8 total, 7 up, 8 in 2024-03-08T20:54:23.221 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:54:22 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-08T20:54:23.221 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:54:22 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-08T20:54:23.221 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:54:22 smithi107 ceph-mon[31381]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-03-08T20:54:23.237 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:54:22 smithi149 ceph-mon[35042]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]': finished 2024-03-08T20:54:23.238 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:54:22 smithi149 ceph-mon[35042]: osdmap e51: 8 total, 7 up, 8 in 2024-03-08T20:54:23.238 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:54:22 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-08T20:54:23.238 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:54:22 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-08T20:54:23.238 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:54:22 smithi149 ceph-mon[35042]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-03-08T20:54:24.060 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:54:23 smithi149 ceph-mon[35042]: purged_snaps scrub starts 2024-03-08T20:54:24.061 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:54:23 smithi149 ceph-mon[35042]: purged_snaps scrub ok 2024-03-08T20:54:24.061 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:54:23 smithi149 ceph-mon[35042]: osd.7 [v2:172.21.15.149:6816/3835863809,v1:172.21.15.149:6817/3835863809] boot 2024-03-08T20:54:24.061 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:54:23 smithi149 ceph-mon[35042]: osdmap e52: 8 total, 8 up, 8 in 2024-03-08T20:54:24.061 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:54:23 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-08T20:54:24.061 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:54:23 smithi149 ceph-mon[35042]: osdmap e53: 8 total, 8 up, 8 in 2024-03-08T20:54:24.132 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:54:23 smithi204 ceph-mon[34935]: purged_snaps scrub starts 2024-03-08T20:54:24.132 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:54:23 smithi204 ceph-mon[34935]: purged_snaps scrub ok 2024-03-08T20:54:24.132 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:54:23 smithi204 ceph-mon[34935]: osd.7 [v2:172.21.15.149:6816/3835863809,v1:172.21.15.149:6817/3835863809] boot 2024-03-08T20:54:24.132 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:54:23 smithi204 ceph-mon[34935]: osdmap e52: 8 total, 8 up, 8 in 2024-03-08T20:54:24.132 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:54:23 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-08T20:54:24.132 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:54:23 smithi204 ceph-mon[34935]: osdmap e53: 8 total, 8 up, 8 in 2024-03-08T20:54:24.221 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:54:23 smithi107 ceph-mon[31381]: purged_snaps scrub starts 2024-03-08T20:54:24.221 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:54:23 smithi107 ceph-mon[31381]: purged_snaps scrub ok 2024-03-08T20:54:24.221 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:54:23 smithi107 ceph-mon[31381]: osd.7 [v2:172.21.15.149:6816/3835863809,v1:172.21.15.149:6817/3835863809] boot 2024-03-08T20:54:24.221 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:54:23 smithi107 ceph-mon[31381]: osdmap e52: 8 total, 8 up, 8 in 2024-03-08T20:54:24.221 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:54:23 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-08T20:54:24.221 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:54:23 smithi107 ceph-mon[31381]: osdmap e53: 8 total, 8 up, 8 in 2024-03-08T20:54:25.132 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:54:24 smithi204 ceph-mon[34935]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-03-08T20:54:25.132 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:54:24 smithi204 ceph-mon[34935]: osdmap e54: 8 total, 8 up, 8 in 2024-03-08T20:54:25.220 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:54:24 smithi107 ceph-mon[31381]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-03-08T20:54:25.221 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:54:24 smithi107 ceph-mon[31381]: osdmap e54: 8 total, 8 up, 8 in 2024-03-08T20:54:25.269 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:54:24 smithi149 ceph-mon[35042]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-03-08T20:54:25.269 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:54:24 smithi149 ceph-mon[35042]: osdmap e54: 8 total, 8 up, 8 in 2024-03-08T20:54:25.721 INFO:teuthology.orchestra.run.smithi204.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-03-08T20:54:25.721 INFO:teuthology.orchestra.run.smithi204.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-03-08T20:54:25.721 INFO:teuthology.orchestra.run.smithi204.stderr: stderr: 10+0 records in 2024-03-08T20:54:25.721 INFO:teuthology.orchestra.run.smithi204.stderr:10+0 records out 2024-03-08T20:54:25.721 INFO:teuthology.orchestra.run.smithi204.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0372065 s, 282 MB/s 2024-03-08T20:54:25.721 INFO:teuthology.orchestra.run.smithi204.stderr:--> Zapping successful for: 2024-03-08T20:54:26.559 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dc20f0f8-dd8c-11ee-95c6-87774f69a715 -- ceph orch daemon add osd smithi204:vg_nvme/lv_2 2024-03-08T20:54:26.990 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/mon.c/config 2024-03-08T20:54:27.018 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:54:26 smithi149 ceph-mon[35042]: osdmap e55: 8 total, 8 up, 8 in 2024-03-08T20:54:27.019 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:54:26 smithi149 ceph-mon[35042]: pgmap v228: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-03-08T20:54:27.019 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:54:26 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:54:27.019 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:54:26 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:54:27.019 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:54:26 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-08T20:54:27.019 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:54:26 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-08T20:54:27.019 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:54:26 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-08T20:54:27.020 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:54:26 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:54:27.020 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:54:26 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:54:27.020 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:54:26 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T20:54:27.020 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:54:26 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:54:27.221 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:54:26 smithi107 ceph-mon[31381]: osdmap e55: 8 total, 8 up, 8 in 2024-03-08T20:54:27.221 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:54:26 smithi107 ceph-mon[31381]: pgmap v228: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-03-08T20:54:27.221 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:54:26 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:54:27.222 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:54:26 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:54:27.222 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:54:26 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-08T20:54:27.222 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:54:26 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-08T20:54:27.222 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:54:26 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-08T20:54:27.222 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:54:26 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:54:27.222 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:54:26 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:54:27.222 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:54:26 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T20:54:27.222 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:54:26 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:54:27.313 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:54:26 smithi204 ceph-mon[34935]: osdmap e55: 8 total, 8 up, 8 in 2024-03-08T20:54:27.313 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:54:26 smithi204 ceph-mon[34935]: pgmap v228: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-03-08T20:54:27.313 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:54:26 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:54:27.313 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:54:26 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:54:27.313 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:54:26 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-08T20:54:27.313 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:54:26 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-08T20:54:27.313 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:54:26 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-08T20:54:27.313 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:54:26 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:54:27.313 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:54:26 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:54:27.314 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:54:26 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T20:54:27.314 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:54:26 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:54:28.220 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:54:27 smithi107 ceph-mon[31381]: Detected new or changed devices on smithi149 2024-03-08T20:54:28.221 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:54:27 smithi107 ceph-mon[31381]: Adjusting osd_memory_target on smithi149 to 1939M 2024-03-08T20:54:28.221 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:54:27 smithi107 ceph-mon[31381]: osdmap e56: 8 total, 8 up, 8 in 2024-03-08T20:54:28.268 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:54:27 smithi149 ceph-mon[35042]: Detected new or changed devices on smithi149 2024-03-08T20:54:28.268 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:54:27 smithi149 ceph-mon[35042]: Adjusting osd_memory_target on smithi149 to 1939M 2024-03-08T20:54:28.268 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:54:27 smithi149 ceph-mon[35042]: osdmap e56: 8 total, 8 up, 8 in 2024-03-08T20:54:28.381 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:54:27 smithi204 ceph-mon[34935]: Detected new or changed devices on smithi149 2024-03-08T20:54:28.382 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:54:27 smithi204 ceph-mon[34935]: Adjusting osd_memory_target on smithi149 to 1939M 2024-03-08T20:54:28.382 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:54:27 smithi204 ceph-mon[34935]: osdmap e56: 8 total, 8 up, 8 in 2024-03-08T20:54:29.030 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:54:28 smithi204 ceph-mon[34935]: pgmap v230: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-03-08T20:54:29.268 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:54:28 smithi149 ceph-mon[35042]: pgmap v230: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-03-08T20:54:29.345 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:54:29.358+0000 7fe24f000700 1 -- 172.21.15.204:0/2083586145 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe24810a000 msgr2=0x7fe2481108b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:54:29.345 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:54:29.358+0000 7fe24f000700 1 --2- 172.21.15.204:0/2083586145 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe24810a000 0x7fe2481108b0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fe24000b0a0 tx=0x7fe24000b3b0 comp rx=0 tx=0).stop 2024-03-08T20:54:29.345 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:54:29.358+0000 7fe24f000700 1 -- 172.21.15.204:0/2083586145 shutdown_connections 2024-03-08T20:54:29.345 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:54:29.359+0000 7fe24f000700 1 --2- 172.21.15.204:0/2083586145 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe24810a000 0x7fe2481108b0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:54:29.345 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:54:29.359+0000 7fe24f000700 1 --2- 172.21.15.204:0/2083586145 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe248108460 0x7fe248108880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:54:29.346 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:54:29.359+0000 7fe24f000700 1 --2- 172.21.15.204:0/2083586145 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe248109660 0x7fe248109ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:54:29.346 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:54:29.359+0000 7fe24f000700 1 -- 172.21.15.204:0/2083586145 >> 172.21.15.204:0/2083586145 conn(0x7fe248077430 msgr2=0x7fe248079870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:54:29.346 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:54:29.359+0000 7fe24f000700 1 -- 172.21.15.204:0/2083586145 shutdown_connections 2024-03-08T20:54:29.346 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:54:29.359+0000 7fe24f000700 1 -- 172.21.15.204:0/2083586145 wait complete. 2024-03-08T20:54:29.348 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:54:29.360+0000 7fe24f000700 1 Processor -- start 2024-03-08T20:54:29.348 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:54:29.360+0000 7fe24f000700 1 -- start start 2024-03-08T20:54:29.348 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:54:29.360+0000 7fe24f000700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe248108460 0x7fe24819e510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:54:29.348 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:54:29.360+0000 7fe24f000700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe248109660 0x7fe24819ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:54:29.348 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:54:29.360+0000 7fe24f000700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe24810a000 0x7fe2481a5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:54:29.348 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:54:29.360+0000 7fe24f000700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe248113f50 con 0x7fe24810a000 2024-03-08T20:54:29.348 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:54:29.360+0000 7fe24f000700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe248113dd0 con 0x7fe248109660 2024-03-08T20:54:29.348 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:54:29.360+0000 7fe24f000700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2481140d0 con 0x7fe248108460 2024-03-08T20:54:29.348 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:54:29.361+0000 7fe247e00700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe248108460 0x7fe24819e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:54:29.348 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:54:29.361+0000 7fe247e00700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe248108460 0x7fe24819e510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:47052/0 (socket says 172.21.15.204:47052) 2024-03-08T20:54:29.348 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:54:29.361+0000 7fe247e00700 1 -- 172.21.15.204:0/2366899952 learned_addr learned my addr 172.21.15.204:0/2366899952 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-03-08T20:54:29.349 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:54:29.361+0000 7fe24d000700 1 --2- 172.21.15.204:0/2366899952 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe24810a000 0x7fe2481a5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:54:29.349 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:54:29.361+0000 7fe247400700 1 --2- 172.21.15.204:0/2366899952 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe248109660 0x7fe24819ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:54:29.349 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:54:29.361+0000 7fe247e00700 1 -- 172.21.15.204:0/2366899952 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe248109660 msgr2=0x7fe24819ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:54:29.349 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:54:29.361+0000 7fe247e00700 1 --2- 172.21.15.204:0/2366899952 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe248109660 0x7fe24819ea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:54:29.349 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:54:29.361+0000 7fe247e00700 1 -- 172.21.15.204:0/2366899952 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe24810a000 msgr2=0x7fe2481a5ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:54:29.349 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:54:29.361+0000 7fe247e00700 1 --2- 172.21.15.204:0/2366899952 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe24810a000 0x7fe2481a5ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:54:29.349 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:54:29.361+0000 7fe247e00700 1 -- 172.21.15.204:0/2366899952 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2481a6170 con 0x7fe248108460 2024-03-08T20:54:29.349 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:54:29.362+0000 7fe247e00700 1 --2- 172.21.15.204:0/2366899952 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe248108460 0x7fe24819e510 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fe23800d9b0 tx=0x7fe23800dcc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:54:29.350 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:54:29.362+0000 7fe244c00700 1 -- 172.21.15.204:0/2366899952 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe238017070 con 0x7fe248108460 2024-03-08T20:54:29.350 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:54:29.362+0000 7fe244c00700 1 -- 172.21.15.204:0/2366899952 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fe238009de0 con 0x7fe248108460 2024-03-08T20:54:29.350 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:54:29.362+0000 7fe24f000700 1 -- 172.21.15.204:0/2366899952 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2481a6460 con 0x7fe248108460 2024-03-08T20:54:29.350 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:54:29.362+0000 7fe244c00700 1 -- 172.21.15.204:0/2366899952 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe23800f3c0 con 0x7fe248108460 2024-03-08T20:54:29.350 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:54:29.362+0000 7fe24f000700 1 -- 172.21.15.204:0/2366899952 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2481a69a0 con 0x7fe248108460 2024-03-08T20:54:29.350 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:54:29.363+0000 7fe24f000700 1 -- 172.21.15.204:0/2366899952 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe248068aa0 con 0x7fe248108460 2024-03-08T20:54:29.353 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:54:29.367+0000 7fe244c00700 1 -- 172.21.15.204:0/2366899952 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 16) v1 ==== 137291+0+0 (secure 0 0 0) 0x7fe238009930 con 0x7fe248108460 2024-03-08T20:54:29.355 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:54:29.368+0000 7fe244c00700 1 --2- 172.21.15.204:0/2366899952 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fe22c0a2910 0x7fe22c0a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:54:29.355 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:54:29.369+0000 7fe244c00700 1 -- 172.21.15.204:0/2366899952 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5397+0+0 (secure 0 0 0) 0x7fe2380cd990 con 0x7fe248108460 2024-03-08T20:54:29.355 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:54:29.369+0000 7fe247400700 1 --2- 172.21.15.204:0/2366899952 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fe22c0a2910 0x7fe22c0a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:54:29.356 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:54:29.370+0000 7fe247400700 1 --2- 172.21.15.204:0/2366899952 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fe22c0a2910 0x7fe22c0a4dd0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fe234005950 tx=0x7fe23400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:54:29.358 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:54:29.372+0000 7fe244c00700 1 -- 172.21.15.204:0/2366899952 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7fe23809b570 con 0x7fe248108460 2024-03-08T20:54:29.470 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:54:28 smithi107 ceph-mon[31381]: pgmap v230: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-03-08T20:54:29.588 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:54:29.602+0000 7fe24f000700 1 -- 172.21.15.204:0/2366899952 --> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7fe248062220 con 0x7fe22c0a2910 2024-03-08T20:54:30.268 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:54:29 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-08T20:54:30.268 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:54:29 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-08T20:54:30.268 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:54:29 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:54:30.381 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:54:29 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-08T20:54:30.381 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:54:29 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-08T20:54:30.381 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:54:29 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:54:30.470 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:54:29 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-08T20:54:30.471 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:54:29 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-08T20:54:30.471 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:54:29 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:54:31.268 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:54:31 smithi149 ceph-mon[35042]: from='client.24295 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-08T20:54:31.268 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:54:31 smithi149 ceph-mon[35042]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-03-08T20:54:31.381 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:54:31 smithi204 ceph-mon[34935]: from='client.24295 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-08T20:54:31.382 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:54:31 smithi204 ceph-mon[34935]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-03-08T20:54:31.470 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:54:31 smithi107 ceph-mon[31381]: from='client.24295 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-08T20:54:31.470 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:54:31 smithi107 ceph-mon[31381]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-03-08T20:54:33.518 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:54:33 smithi149 ceph-mon[35042]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-03-08T20:54:33.632 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:54:33 smithi204 ceph-mon[34935]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-03-08T20:54:33.720 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:54:33 smithi107 ceph-mon[31381]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-03-08T20:54:34.631 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:54:34 smithi204 ceph-mon[34935]: from='client.? 172.21.15.204:0/3425244568' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "baaaa8dd-0e1f-4bae-86bf-9e429d985f23"}]: dispatch 2024-03-08T20:54:34.631 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:54:34 smithi204 ceph-mon[34935]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "baaaa8dd-0e1f-4bae-86bf-9e429d985f23"}]: dispatch 2024-03-08T20:54:34.631 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:54:34 smithi204 ceph-mon[34935]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "baaaa8dd-0e1f-4bae-86bf-9e429d985f23"}]': finished 2024-03-08T20:54:34.632 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:54:34 smithi204 ceph-mon[34935]: osdmap e57: 9 total, 8 up, 9 in 2024-03-08T20:54:34.632 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:54:34 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-08T20:54:34.721 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:54:34 smithi107 ceph-mon[31381]: from='client.? 172.21.15.204:0/3425244568' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "baaaa8dd-0e1f-4bae-86bf-9e429d985f23"}]: dispatch 2024-03-08T20:54:34.721 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:54:34 smithi107 ceph-mon[31381]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "baaaa8dd-0e1f-4bae-86bf-9e429d985f23"}]: dispatch 2024-03-08T20:54:34.721 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:54:34 smithi107 ceph-mon[31381]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "baaaa8dd-0e1f-4bae-86bf-9e429d985f23"}]': finished 2024-03-08T20:54:34.721 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:54:34 smithi107 ceph-mon[31381]: osdmap e57: 9 total, 8 up, 9 in 2024-03-08T20:54:34.721 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:54:34 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-08T20:54:34.768 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:54:34 smithi149 ceph-mon[35042]: from='client.? 172.21.15.204:0/3425244568' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "baaaa8dd-0e1f-4bae-86bf-9e429d985f23"}]: dispatch 2024-03-08T20:54:34.768 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:54:34 smithi149 ceph-mon[35042]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "baaaa8dd-0e1f-4bae-86bf-9e429d985f23"}]: dispatch 2024-03-08T20:54:34.768 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:54:34 smithi149 ceph-mon[35042]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "baaaa8dd-0e1f-4bae-86bf-9e429d985f23"}]': finished 2024-03-08T20:54:34.768 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:54:34 smithi149 ceph-mon[35042]: osdmap e57: 9 total, 8 up, 9 in 2024-03-08T20:54:34.768 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:54:34 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-08T20:54:35.631 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:54:35 smithi204 ceph-mon[34935]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-03-08T20:54:35.631 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:54:35 smithi204 ceph-mon[34935]: from='client.? 172.21.15.204:0/2490885024' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-08T20:54:35.720 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:54:35 smithi107 ceph-mon[31381]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-03-08T20:54:35.720 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:54:35 smithi107 ceph-mon[31381]: from='client.? 172.21.15.204:0/2490885024' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-08T20:54:35.768 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:54:35 smithi149 ceph-mon[35042]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-03-08T20:54:35.768 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:54:35 smithi149 ceph-mon[35042]: from='client.? 172.21.15.204:0/2490885024' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-08T20:54:37.631 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:54:37 smithi204 ceph-mon[34935]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-03-08T20:54:37.720 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:54:37 smithi107 ceph-mon[31381]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-03-08T20:54:37.768 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:54:37 smithi149 ceph-mon[35042]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-03-08T20:54:39.570 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:54:39 smithi149 ceph-mon[35042]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-03-08T20:54:39.631 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:54:39 smithi204 ceph-mon[34935]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-03-08T20:54:39.720 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:54:39 smithi107 ceph-mon[31381]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-03-08T20:54:41.518 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:54:41 smithi149 ceph-mon[35042]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-03-08T20:54:41.631 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:54:41 smithi204 ceph-mon[34935]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-03-08T20:54:41.720 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:54:41 smithi107 ceph-mon[31381]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-03-08T20:54:43.631 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:54:43 smithi204 ceph-mon[34935]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-03-08T20:54:43.632 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:54:43 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-03-08T20:54:43.632 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:54:43 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:54:43.720 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:54:43 smithi107 ceph-mon[31381]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-03-08T20:54:43.720 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:54:43 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-03-08T20:54:43.721 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:54:43 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:54:43.768 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:54:43 smithi149 ceph-mon[35042]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-03-08T20:54:43.768 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:54:43 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-03-08T20:54:43.768 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:54:43 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:54:44.558 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:54:44 smithi204 ceph-mon[34935]: Deploying daemon osd.8 on smithi204 2024-03-08T20:54:44.720 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:54:44 smithi107 ceph-mon[31381]: Deploying daemon osd.8 on smithi204 2024-03-08T20:54:44.768 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:54:44 smithi149 ceph-mon[35042]: Deploying daemon osd.8 on smithi204 2024-03-08T20:54:45.631 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:54:45 smithi204 ceph-mon[34935]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-03-08T20:54:45.720 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:54:45 smithi107 ceph-mon[31381]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-03-08T20:54:45.768 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:54:45 smithi149 ceph-mon[35042]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-03-08T20:54:47.620 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:54:47 smithi204 ceph-mon[34935]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-03-08T20:54:47.720 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:54:47 smithi107 ceph-mon[31381]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-03-08T20:54:47.768 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:54:47 smithi149 ceph-mon[35042]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-03-08T20:54:49.633 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:54:49 smithi204 ceph-mon[34935]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-03-08T20:54:49.720 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:54:49 smithi107 ceph-mon[31381]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-03-08T20:54:49.768 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:54:49 smithi149 ceph-mon[35042]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-03-08T20:54:51.612 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:54:51 smithi204 ceph-mon[34935]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-03-08T20:54:51.720 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:54:51 smithi107 ceph-mon[31381]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-03-08T20:54:51.768 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:54:51 smithi149 ceph-mon[35042]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-03-08T20:54:53.381 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:54:53 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:54:53.381 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:54:53 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T20:54:53.381 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:54:53 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:54:53.381 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:54:53 smithi204 ceph-mon[34935]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-03-08T20:54:53.470 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:54:53 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:54:53.470 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:54:53 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T20:54:53.471 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:54:53 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:54:53.471 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:54:53 smithi107 ceph-mon[31381]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-03-08T20:54:53.518 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:54:53 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:54:53.518 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:54:53 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T20:54:53.518 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:54:53 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:54:53.518 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:54:53 smithi149 ceph-mon[35042]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-03-08T20:54:55.631 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:54:55 smithi204 ceph-mon[34935]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-03-08T20:54:55.720 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:54:55 smithi107 ceph-mon[31381]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-03-08T20:54:55.768 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:54:55 smithi149 ceph-mon[35042]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-03-08T20:54:57.061 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:54:57.075+0000 7fe244c00700 1 -- 172.21.15.204:0/2366899952 <== mgr.14152 v2:172.21.15.107:6800/3376636618 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fe248062220 con 0x7fe22c0a2910 2024-03-08T20:54:57.061 INFO:teuthology.orchestra.run.smithi204.stdout:Created osd(s) 8 on host 'smithi204' 2024-03-08T20:54:57.065 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:54:57.078+0000 7fe24f000700 1 -- 172.21.15.204:0/2366899952 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fe22c0a2910 msgr2=0x7fe22c0a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:54:57.065 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:54:57.078+0000 7fe24f000700 1 --2- 172.21.15.204:0/2366899952 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fe22c0a2910 0x7fe22c0a4dd0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fe234005950 tx=0x7fe23400b410 comp rx=0 tx=0).stop 2024-03-08T20:54:57.065 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:54:57.078+0000 7fe24f000700 1 -- 172.21.15.204:0/2366899952 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe248108460 msgr2=0x7fe24819e510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:54:57.065 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:54:57.078+0000 7fe24f000700 1 --2- 172.21.15.204:0/2366899952 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe248108460 0x7fe24819e510 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fe23800d9b0 tx=0x7fe23800dcc0 comp rx=0 tx=0).stop 2024-03-08T20:54:57.065 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:54:57.078+0000 7fe24f000700 1 -- 172.21.15.204:0/2366899952 shutdown_connections 2024-03-08T20:54:57.065 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:54:57.078+0000 7fe24f000700 1 --2- 172.21.15.204:0/2366899952 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe248108460 0x7fe24819e510 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:54:57.065 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:54:57.078+0000 7fe24f000700 1 --2- 172.21.15.204:0/2366899952 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fe22c0a2910 0x7fe22c0a4dd0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:54:57.065 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:54:57.078+0000 7fe24f000700 1 --2- 172.21.15.204:0/2366899952 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe24810a000 0x7fe2481a5ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:54:57.065 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:54:57.078+0000 7fe24f000700 1 --2- 172.21.15.204:0/2366899952 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe248109660 0x7fe24819ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:54:57.065 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:54:57.078+0000 7fe24f000700 1 -- 172.21.15.204:0/2366899952 >> 172.21.15.204:0/2366899952 conn(0x7fe248077430 msgr2=0x7fe24810c960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:54:57.065 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:54:57.078+0000 7fe24f000700 1 -- 172.21.15.204:0/2366899952 shutdown_connections 2024-03-08T20:54:57.065 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:54:57.079+0000 7fe24f000700 1 -- 172.21.15.204:0/2366899952 wait complete. 2024-03-08T20:54:57.131 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:54:56 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:54:57.131 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:54:56 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:54:57.131 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:54:56 smithi204 ceph-mon[34935]: from='osd.8 [v2:172.21.15.204:6816/1010601477,v1:172.21.15.204:6817/1010601477]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-08T20:54:57.131 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:54:56 smithi204 ceph-mon[34935]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-08T20:54:57.131 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:54:56 smithi204 ceph-mon[34935]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-03-08T20:54:57.220 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:54:56 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:54:57.220 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:54:56 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:54:57.221 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:54:56 smithi107 ceph-mon[31381]: from='osd.8 [v2:172.21.15.204:6816/1010601477,v1:172.21.15.204:6817/1010601477]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-08T20:54:57.221 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:54:56 smithi107 ceph-mon[31381]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-08T20:54:57.221 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:54:56 smithi107 ceph-mon[31381]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-03-08T20:54:57.268 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:54:56 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:54:57.268 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:54:56 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:54:57.268 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:54:56 smithi149 ceph-mon[35042]: from='osd.8 [v2:172.21.15.204:6816/1010601477,v1:172.21.15.204:6817/1010601477]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-08T20:54:57.268 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:54:56 smithi149 ceph-mon[35042]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-08T20:54:57.268 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:54:56 smithi149 ceph-mon[35042]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-03-08T20:54:57.903 DEBUG:teuthology.orchestra.run.smithi204:osd.8> sudo journalctl -f -n 0 -u ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715@osd.8.service 2024-03-08T20:54:57.909 INFO:tasks.cephadm:Deploying osd.9 on smithi107 with /dev/vg_nvme/lv_1... 2024-03-08T20:54:57.909 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dc20f0f8-dd8c-11ee-95c6-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-03-08T20:54:58.097 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:54:57 smithi204 ceph-mon[34935]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-03-08T20:54:58.098 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:54:57 smithi204 ceph-mon[34935]: from='osd.8 [v2:172.21.15.204:6816/1010601477,v1:172.21.15.204:6817/1010601477]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-03-08T20:54:58.098 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:54:57 smithi204 ceph-mon[34935]: osdmap e58: 9 total, 8 up, 9 in 2024-03-08T20:54:58.098 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:54:57 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-08T20:54:58.098 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:54:57 smithi204 ceph-mon[34935]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-03-08T20:54:58.098 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:54:57 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:54:58.098 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:54:57 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:54:58.220 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:54:57 smithi107 ceph-mon[31381]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-03-08T20:54:58.220 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:54:57 smithi107 ceph-mon[31381]: from='osd.8 [v2:172.21.15.204:6816/1010601477,v1:172.21.15.204:6817/1010601477]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-03-08T20:54:58.220 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:54:57 smithi107 ceph-mon[31381]: osdmap e58: 9 total, 8 up, 9 in 2024-03-08T20:54:58.220 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:54:57 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-08T20:54:58.221 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:54:57 smithi107 ceph-mon[31381]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-03-08T20:54:58.221 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:54:57 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:54:58.221 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:54:57 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:54:58.268 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:54:57 smithi149 ceph-mon[35042]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-03-08T20:54:58.268 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:54:57 smithi149 ceph-mon[35042]: from='osd.8 [v2:172.21.15.204:6816/1010601477,v1:172.21.15.204:6817/1010601477]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-03-08T20:54:58.268 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:54:57 smithi149 ceph-mon[35042]: osdmap e58: 9 total, 8 up, 9 in 2024-03-08T20:54:58.268 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:54:57 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-08T20:54:58.268 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:54:57 smithi149 ceph-mon[35042]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-03-08T20:54:58.269 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:54:57 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:54:58.269 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:54:57 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:54:58.358 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/mon.a/config 2024-03-08T20:54:59.221 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:54:58 smithi107 ceph-mon[31381]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]': finished 2024-03-08T20:54:59.221 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:54:58 smithi107 ceph-mon[31381]: osdmap e59: 9 total, 8 up, 9 in 2024-03-08T20:54:59.221 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:54:58 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-08T20:54:59.221 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:54:58 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-08T20:54:59.222 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:54:58 smithi107 ceph-mon[31381]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-03-08T20:54:59.222 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:54:58 smithi107 ceph-mon[31381]: osd.8 [v2:172.21.15.204:6816/1010601477,v1:172.21.15.204:6817/1010601477] boot 2024-03-08T20:54:59.222 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:54:58 smithi107 ceph-mon[31381]: osdmap e60: 9 total, 9 up, 9 in 2024-03-08T20:54:59.222 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:54:58 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-08T20:54:59.268 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:54:58 smithi149 ceph-mon[35042]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]': finished 2024-03-08T20:54:59.268 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:54:58 smithi149 ceph-mon[35042]: osdmap e59: 9 total, 8 up, 9 in 2024-03-08T20:54:59.268 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:54:58 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-08T20:54:59.269 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:54:58 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-08T20:54:59.269 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:54:58 smithi149 ceph-mon[35042]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-03-08T20:54:59.269 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:54:58 smithi149 ceph-mon[35042]: osd.8 [v2:172.21.15.204:6816/1010601477,v1:172.21.15.204:6817/1010601477] boot 2024-03-08T20:54:59.269 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:54:58 smithi149 ceph-mon[35042]: osdmap e60: 9 total, 9 up, 9 in 2024-03-08T20:54:59.269 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:54:58 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-08T20:54:59.309 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:54:58 smithi204 ceph-mon[34935]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]': finished 2024-03-08T20:54:59.309 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:54:58 smithi204 ceph-mon[34935]: osdmap e59: 9 total, 8 up, 9 in 2024-03-08T20:54:59.309 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:54:58 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-08T20:54:59.309 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:54:58 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-08T20:54:59.309 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:54:58 smithi204 ceph-mon[34935]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-03-08T20:54:59.309 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:54:58 smithi204 ceph-mon[34935]: osd.8 [v2:172.21.15.204:6816/1010601477,v1:172.21.15.204:6817/1010601477] boot 2024-03-08T20:54:59.309 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:54:58 smithi204 ceph-mon[34935]: osdmap e60: 9 total, 9 up, 9 in 2024-03-08T20:54:59.309 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:54:58 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-08T20:55:00.220 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:54:59 smithi107 ceph-mon[31381]: purged_snaps scrub starts 2024-03-08T20:55:00.220 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:54:59 smithi107 ceph-mon[31381]: purged_snaps scrub ok 2024-03-08T20:55:00.220 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:54:59 smithi107 ceph-mon[31381]: osdmap e61: 9 total, 9 up, 9 in 2024-03-08T20:55:00.268 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:54:59 smithi149 ceph-mon[35042]: purged_snaps scrub starts 2024-03-08T20:55:00.268 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:54:59 smithi149 ceph-mon[35042]: purged_snaps scrub ok 2024-03-08T20:55:00.268 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:54:59 smithi149 ceph-mon[35042]: osdmap e61: 9 total, 9 up, 9 in 2024-03-08T20:55:00.381 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:54:59 smithi204 ceph-mon[34935]: purged_snaps scrub starts 2024-03-08T20:55:00.381 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:54:59 smithi204 ceph-mon[34935]: purged_snaps scrub ok 2024-03-08T20:55:00.381 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:54:59 smithi204 ceph-mon[34935]: osdmap e61: 9 total, 9 up, 9 in 2024-03-08T20:55:01.199 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:55:00 smithi107 ceph-mon[31381]: pgmap v251: 1 pgs: 1 remapped+peering; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-03-08T20:55:01.199 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:55:00 smithi107 ceph-mon[31381]: osdmap e62: 9 total, 9 up, 9 in 2024-03-08T20:55:01.199 INFO:teuthology.orchestra.run.smithi107.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-03-08T20:55:01.199 INFO:teuthology.orchestra.run.smithi107.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-03-08T20:55:01.199 INFO:teuthology.orchestra.run.smithi107.stderr: stderr: 10+0 records in 2024-03-08T20:55:01.199 INFO:teuthology.orchestra.run.smithi107.stderr:10+0 records out 2024-03-08T20:55:01.200 INFO:teuthology.orchestra.run.smithi107.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0388688 s, 270 MB/s 2024-03-08T20:55:01.200 INFO:teuthology.orchestra.run.smithi107.stderr:--> Zapping successful for: 2024-03-08T20:55:01.268 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:55:00 smithi149 ceph-mon[35042]: pgmap v251: 1 pgs: 1 remapped+peering; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-03-08T20:55:01.268 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:55:00 smithi149 ceph-mon[35042]: osdmap e62: 9 total, 9 up, 9 in 2024-03-08T20:55:01.381 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:55:00 smithi204 ceph-mon[34935]: pgmap v251: 1 pgs: 1 remapped+peering; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-03-08T20:55:01.381 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:55:00 smithi204 ceph-mon[34935]: osdmap e62: 9 total, 9 up, 9 in 2024-03-08T20:55:02.167 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dc20f0f8-dd8c-11ee-95c6-87774f69a715 -- ceph orch daemon add osd smithi107:vg_nvme/lv_1 2024-03-08T20:55:02.591 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/mon.a/config 2024-03-08T20:55:03.057 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:55:02 smithi107 ceph-mon[31381]: osdmap e63: 9 total, 9 up, 9 in 2024-03-08T20:55:03.057 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:55:02 smithi107 ceph-mon[31381]: pgmap v254: 1 pgs: 1 remapped+peering; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-03-08T20:55:03.058 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:55:02 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:55:03.058 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:55:02 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:55:03.058 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:55:02 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-08T20:55:03.058 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:55:02 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-08T20:55:03.058 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:55:02 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-08T20:55:03.058 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:55:02 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:55:03.058 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:55:02 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:55:03.058 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:55:02 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T20:55:03.058 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:55:02 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:55:03.268 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:55:02 smithi149 ceph-mon[35042]: osdmap e63: 9 total, 9 up, 9 in 2024-03-08T20:55:03.268 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:55:02 smithi149 ceph-mon[35042]: pgmap v254: 1 pgs: 1 remapped+peering; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-03-08T20:55:03.268 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:55:02 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:55:03.268 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:55:02 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:55:03.269 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:55:02 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-08T20:55:03.269 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:55:02 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-08T20:55:03.269 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:55:02 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-08T20:55:03.269 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:55:02 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:55:03.269 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:55:02 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:55:03.269 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:55:02 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T20:55:03.269 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:55:02 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:55:03.381 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:55:02 smithi204 ceph-mon[34935]: osdmap e63: 9 total, 9 up, 9 in 2024-03-08T20:55:03.382 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:55:02 smithi204 ceph-mon[34935]: pgmap v254: 1 pgs: 1 remapped+peering; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-03-08T20:55:03.382 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:55:02 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:55:03.382 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:55:02 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:55:03.382 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:55:02 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-08T20:55:03.382 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:55:02 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-08T20:55:03.382 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:55:02 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-08T20:55:03.382 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:55:02 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:55:03.382 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:55:02 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:55:03.382 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:55:02 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T20:55:03.382 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:55:02 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:55:03.937 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:55:03 smithi107 ceph-mon[31381]: Detected new or changed devices on smithi204 2024-03-08T20:55:03.937 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:55:03 smithi107 ceph-mon[31381]: Adjusting osd_memory_target on smithi204 to 5697M 2024-03-08T20:55:03.937 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:55:03 smithi107 ceph-mon[31381]: osdmap e64: 9 total, 9 up, 9 in 2024-03-08T20:55:03.937 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:55:03 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T20:55:03.937 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:55:03 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:55:03.937 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:55:03 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T20:55:03.937 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:55:03 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:55:04.268 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:55:03 smithi149 ceph-mon[35042]: Detected new or changed devices on smithi204 2024-03-08T20:55:04.268 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:55:03 smithi149 ceph-mon[35042]: Adjusting osd_memory_target on smithi204 to 5697M 2024-03-08T20:55:04.268 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:55:03 smithi149 ceph-mon[35042]: osdmap e64: 9 total, 9 up, 9 in 2024-03-08T20:55:04.268 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:55:03 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T20:55:04.268 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:55:03 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:55:04.268 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:55:03 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T20:55:04.269 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:55:03 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:55:04.381 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:55:03 smithi204 ceph-mon[34935]: Detected new or changed devices on smithi204 2024-03-08T20:55:04.381 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:55:03 smithi204 ceph-mon[34935]: Adjusting osd_memory_target on smithi204 to 5697M 2024-03-08T20:55:04.381 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:55:03 smithi204 ceph-mon[34935]: osdmap e64: 9 total, 9 up, 9 in 2024-03-08T20:55:04.381 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:55:03 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T20:55:04.381 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:55:03 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:55:04.382 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:55:03 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T20:55:04.382 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:55:03 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:55:04.772 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:55:04.785+0000 7fcf86200700 1 -- 172.21.15.107:0/885883417 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcf80105c00 msgr2=0x7fcf8010c4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:55:04.773 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:55:04.785+0000 7fcf86200700 1 --2- 172.21.15.107:0/885883417 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcf80105c00 0x7fcf8010c4b0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fcf7800b0d0 tx=0x7fcf7800b3e0 comp rx=0 tx=0).stop 2024-03-08T20:55:04.773 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:55:04.786+0000 7fcf86200700 1 -- 172.21.15.107:0/885883417 shutdown_connections 2024-03-08T20:55:04.773 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:55:04.786+0000 7fcf86200700 1 --2- 172.21.15.107:0/885883417 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcf80104060 0x7fcf80104480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:55:04.773 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:55:04.786+0000 7fcf86200700 1 --2- 172.21.15.107:0/885883417 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcf80105c00 0x7fcf8010c4b0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:55:04.773 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:55:04.786+0000 7fcf86200700 1 --2- 172.21.15.107:0/885883417 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcf80105260 0x7fcf801056c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:55:04.773 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:55:04.786+0000 7fcf86200700 1 -- 172.21.15.107:0/885883417 >> 172.21.15.107:0/885883417 conn(0x7fcf800ff600 msgr2=0x7fcf80101a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:55:04.773 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:55:04.786+0000 7fcf86200700 1 -- 172.21.15.107:0/885883417 shutdown_connections 2024-03-08T20:55:04.773 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:55:04.786+0000 7fcf86200700 1 -- 172.21.15.107:0/885883417 wait complete. 2024-03-08T20:55:04.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:55:04.787+0000 7fcf86200700 1 Processor -- start 2024-03-08T20:55:04.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:55:04.787+0000 7fcf86200700 1 -- start start 2024-03-08T20:55:04.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:55:04.787+0000 7fcf86200700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcf80104060 0x7fcf8019a130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:55:04.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:55:04.787+0000 7fcf86200700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcf80105260 0x7fcf8019a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:55:04.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:55:04.787+0000 7fcf86200700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcf80105c00 0x7fcf801a16f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:55:04.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:55:04.787+0000 7fcf86200700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf8010fb40 con 0x7fcf80105260 2024-03-08T20:55:04.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:55:04.787+0000 7fcf86200700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf8010f9c0 con 0x7fcf80105c00 2024-03-08T20:55:04.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:55:04.787+0000 7fcf86200700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf8010fcc0 con 0x7fcf80104060 2024-03-08T20:55:04.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:55:04.788+0000 7fcf7ea00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcf80105260 0x7fcf8019a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:55:04.776 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:55:04.788+0000 7fcf7ea00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcf80105260 0x7fcf8019a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:35646/0 (socket says 172.21.15.107:35646) 2024-03-08T20:55:04.776 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:55:04.788+0000 7fcf7ea00700 1 -- 172.21.15.107:0/3323744814 learned_addr learned my addr 172.21.15.107:0/3323744814 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T20:55:04.776 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:55:04.788+0000 7fcf7f400700 1 --2- 172.21.15.107:0/3323744814 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcf80104060 0x7fcf8019a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:55:04.776 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:55:04.788+0000 7fcf7fe00700 1 --2- 172.21.15.107:0/3323744814 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcf80105c00 0x7fcf801a16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:55:04.776 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:55:04.788+0000 7fcf7ea00700 1 -- 172.21.15.107:0/3323744814 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcf80104060 msgr2=0x7fcf8019a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:55:04.776 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:55:04.788+0000 7fcf7ea00700 1 --2- 172.21.15.107:0/3323744814 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcf80104060 0x7fcf8019a130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:55:04.776 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:55:04.788+0000 7fcf7ea00700 1 -- 172.21.15.107:0/3323744814 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcf80105c00 msgr2=0x7fcf801a16f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:55:04.776 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:55:04.788+0000 7fcf7ea00700 1 --2- 172.21.15.107:0/3323744814 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcf80105c00 0x7fcf801a16f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:55:04.776 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:55:04.788+0000 7fcf7ea00700 1 -- 172.21.15.107:0/3323744814 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf801a1d90 con 0x7fcf80105260 2024-03-08T20:55:04.776 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:55:04.788+0000 7fcf7f400700 1 --2- 172.21.15.107:0/3323744814 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcf80104060 0x7fcf8019a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T20:55:04.776 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:55:04.789+0000 7fcf7ea00700 1 --2- 172.21.15.107:0/3323744814 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcf80105260 0x7fcf8019a670 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fcf6c00d980 tx=0x7fcf6c00dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:55:04.777 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:55:04.789+0000 7fcf77e00700 1 -- 172.21.15.107:0/3323744814 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf6c016070 con 0x7fcf80105260 2024-03-08T20:55:04.777 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:55:04.789+0000 7fcf77e00700 1 -- 172.21.15.107:0/3323744814 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fcf6c010460 con 0x7fcf80105260 2024-03-08T20:55:04.777 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:55:04.789+0000 7fcf86200700 1 -- 172.21.15.107:0/3323744814 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf801a2080 con 0x7fcf80105260 2024-03-08T20:55:04.777 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:55:04.789+0000 7fcf77e00700 1 -- 172.21.15.107:0/3323744814 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf6c00f620 con 0x7fcf80105260 2024-03-08T20:55:04.777 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:55:04.789+0000 7fcf86200700 1 -- 172.21.15.107:0/3323744814 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf801a2610 con 0x7fcf80105260 2024-03-08T20:55:04.777 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:55:04.790+0000 7fcf86200700 1 -- 172.21.15.107:0/3323744814 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf80068aa0 con 0x7fcf80105260 2024-03-08T20:55:04.779 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:55:04.792+0000 7fcf77e00700 1 -- 172.21.15.107:0/3323744814 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) v1 ==== 137291+0+0 (secure 0 0 0) 0x7fcf6c019070 con 0x7fcf80105260 2024-03-08T20:55:04.780 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:55:04.794+0000 7fcf77e00700 1 --2- 172.21.15.107:0/3323744814 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fcf640a2910 0x7fcf640a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:55:04.780 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:55:04.794+0000 7fcf77e00700 1 -- 172.21.15.107:0/3323744814 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5829+0+0 (secure 0 0 0) 0x7fcf6c0cdca0 con 0x7fcf80105260 2024-03-08T20:55:04.780 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:55:04.794+0000 7fcf7f400700 1 --2- 172.21.15.107:0/3323744814 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fcf640a2910 0x7fcf640a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:55:04.781 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:55:04.795+0000 7fcf7f400700 1 --2- 172.21.15.107:0/3323744814 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fcf640a2910 0x7fcf640a4dd0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fcf70005950 tx=0x7fcf700058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:55:04.783 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:55:04.796+0000 7fcf77e00700 1 -- 172.21.15.107:0/3323744814 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7fcf6c09b650 con 0x7fcf80105260 2024-03-08T20:55:05.021 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:55:05.034+0000 7fcf86200700 1 -- 172.21.15.107:0/3323744814 --> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7fcf80062220 con 0x7fcf640a2910 2024-03-08T20:55:05.220 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:55:04 smithi107 ceph-mon[31381]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 131 KiB/s, 0 objects/s recovering 2024-03-08T20:55:05.268 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:55:04 smithi149 ceph-mon[35042]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 131 KiB/s, 0 objects/s recovering 2024-03-08T20:55:05.381 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:55:04 smithi204 ceph-mon[34935]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 131 KiB/s, 0 objects/s recovering 2024-03-08T20:55:06.220 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:55:05 smithi107 ceph-mon[31381]: from='client.14415 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-08T20:55:06.220 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:55:05 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-08T20:55:06.220 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:55:05 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-08T20:55:06.221 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:55:05 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:55:06.268 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:55:05 smithi149 ceph-mon[35042]: from='client.14415 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-08T20:55:06.268 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:55:05 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-08T20:55:06.268 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:55:05 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-08T20:55:06.268 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:55:05 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:55:06.381 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:55:05 smithi204 ceph-mon[34935]: from='client.14415 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-08T20:55:06.381 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:55:05 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-08T20:55:06.381 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:55:05 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-08T20:55:06.381 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:55:05 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:55:07.170 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:55:06 smithi107 ceph-mon[31381]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-08T20:55:07.268 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:55:06 smithi149 ceph-mon[35042]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-08T20:55:07.381 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:55:06 smithi204 ceph-mon[34935]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-08T20:55:09.631 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:55:09 smithi204 ceph-mon[34935]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-03-08T20:55:09.631 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:55:09 smithi204 ceph-mon[34935]: from='client.? 172.21.15.107:0/4115939776' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8df80dfa-e7ff-410e-824c-b1dce111b02f"}]: dispatch 2024-03-08T20:55:09.631 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:55:09 smithi204 ceph-mon[34935]: from='client.? 172.21.15.107:0/4115939776' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8df80dfa-e7ff-410e-824c-b1dce111b02f"}]': finished 2024-03-08T20:55:09.632 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:55:09 smithi204 ceph-mon[34935]: osdmap e65: 10 total, 9 up, 10 in 2024-03-08T20:55:09.632 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:55:09 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-08T20:55:09.720 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:55:09 smithi107 ceph-mon[31381]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-03-08T20:55:09.720 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:55:09 smithi107 ceph-mon[31381]: from='client.? 172.21.15.107:0/4115939776' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8df80dfa-e7ff-410e-824c-b1dce111b02f"}]: dispatch 2024-03-08T20:55:09.720 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:55:09 smithi107 ceph-mon[31381]: from='client.? 172.21.15.107:0/4115939776' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8df80dfa-e7ff-410e-824c-b1dce111b02f"}]': finished 2024-03-08T20:55:09.720 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:55:09 smithi107 ceph-mon[31381]: osdmap e65: 10 total, 9 up, 10 in 2024-03-08T20:55:09.720 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:55:09 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-08T20:55:09.768 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:55:09 smithi149 ceph-mon[35042]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-03-08T20:55:09.768 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:55:09 smithi149 ceph-mon[35042]: from='client.? 172.21.15.107:0/4115939776' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8df80dfa-e7ff-410e-824c-b1dce111b02f"}]: dispatch 2024-03-08T20:55:09.768 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:55:09 smithi149 ceph-mon[35042]: from='client.? 172.21.15.107:0/4115939776' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8df80dfa-e7ff-410e-824c-b1dce111b02f"}]': finished 2024-03-08T20:55:09.768 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:55:09 smithi149 ceph-mon[35042]: osdmap e65: 10 total, 9 up, 10 in 2024-03-08T20:55:09.768 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:55:09 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-08T20:55:10.631 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:55:10 smithi204 ceph-mon[34935]: from='client.? 172.21.15.107:0/2422114558' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-08T20:55:10.720 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:55:10 smithi107 ceph-mon[31381]: from='client.? 172.21.15.107:0/2422114558' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-08T20:55:10.767 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:55:10 smithi149 ceph-mon[35042]: from='client.? 172.21.15.107:0/2422114558' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-08T20:55:11.631 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:55:11 smithi204 ceph-mon[34935]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-08T20:55:11.720 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:55:11 smithi107 ceph-mon[31381]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-08T20:55:11.768 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:55:11 smithi149 ceph-mon[35042]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-08T20:55:13.580 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:55:13 smithi107 ceph-mon[31381]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-03-08T20:55:13.631 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:55:13 smithi204 ceph-mon[34935]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-03-08T20:55:13.767 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:55:13 smithi149 ceph-mon[35042]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-03-08T20:55:15.373 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:55:15 smithi107 ceph-mon[31381]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-03-08T20:55:15.631 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:55:15 smithi204 ceph-mon[34935]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-03-08T20:55:15.767 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:55:15 smithi149 ceph-mon[35042]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-03-08T20:55:17.631 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:55:17 smithi204 ceph-mon[34935]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-03-08T20:55:17.721 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:55:17 smithi107 ceph-mon[31381]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-03-08T20:55:17.767 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:55:17 smithi149 ceph-mon[35042]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-03-08T20:55:18.469 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:55:18 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-03-08T20:55:18.470 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:55:18 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:55:18.631 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:55:18 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-03-08T20:55:18.631 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:55:18 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:55:18.768 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:55:18 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-03-08T20:55:18.768 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:55:18 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:55:19.572 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:55:19 smithi107 ceph-mon[31381]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-03-08T20:55:19.572 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:55:19 smithi107 ceph-mon[31381]: Deploying daemon osd.9 on smithi107 2024-03-08T20:55:19.631 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:55:19 smithi204 ceph-mon[34935]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-03-08T20:55:19.631 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:55:19 smithi204 ceph-mon[34935]: Deploying daemon osd.9 on smithi107 2024-03-08T20:55:19.767 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:55:19 smithi149 ceph-mon[35042]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-03-08T20:55:19.768 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:55:19 smithi149 ceph-mon[35042]: Deploying daemon osd.9 on smithi107 2024-03-08T20:55:21.630 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:55:21 smithi204 ceph-mon[34935]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-03-08T20:55:21.649 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:55:21 smithi107 ceph-mon[31381]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-03-08T20:55:21.767 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:55:21 smithi149 ceph-mon[35042]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-03-08T20:55:23.595 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:55:23 smithi107 ceph-mon[31381]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-03-08T20:55:23.630 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:55:23 smithi204 ceph-mon[34935]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-03-08T20:55:23.767 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:55:23 smithi149 ceph-mon[35042]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-03-08T20:55:25.470 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:55:25 smithi107 ceph-mon[31381]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-03-08T20:55:25.630 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:55:25 smithi204 ceph-mon[34935]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-03-08T20:55:25.767 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:55:25 smithi149 ceph-mon[35042]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-03-08T20:55:27.470 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:55:27 smithi107 ceph-mon[31381]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-03-08T20:55:27.470 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:55:27 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:55:27.470 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:55:27 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:55:27.470 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:55:27 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T20:55:27.631 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:55:27 smithi204 ceph-mon[34935]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-03-08T20:55:27.631 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:55:27 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:55:27.631 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:55:27 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:55:27.631 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:55:27 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T20:55:27.767 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:55:27 smithi149 ceph-mon[35042]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-03-08T20:55:27.768 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:55:27 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:55:27.768 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:55:27 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:55:27.768 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:55:27 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T20:55:29.630 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:55:29 smithi204 ceph-mon[34935]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-03-08T20:55:29.719 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:55:29 smithi107 ceph-mon[31381]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-03-08T20:55:29.767 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:55:29 smithi149 ceph-mon[35042]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-03-08T20:55:31.631 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:55:31 smithi204 ceph-mon[34935]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-03-08T20:55:31.631 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:55:31 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:55:31.631 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:55:31 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:55:31.631 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:55:31 smithi204 ceph-mon[34935]: from='osd.9 [v2:172.21.15.107:6826/3334759819,v1:172.21.15.107:6827/3334759819]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-03-08T20:55:31.719 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:55:31 smithi107 ceph-mon[31381]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-03-08T20:55:31.720 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:55:31 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:55:31.720 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:55:31 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:55:31.720 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:55:31 smithi107 ceph-mon[31381]: from='osd.9 [v2:172.21.15.107:6826/3334759819,v1:172.21.15.107:6827/3334759819]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-03-08T20:55:31.768 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:55:31 smithi149 ceph-mon[35042]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-03-08T20:55:31.768 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:55:31 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:55:31.768 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:55:31 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:55:31.768 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:55:31 smithi149 ceph-mon[35042]: from='osd.9 [v2:172.21.15.107:6826/3334759819,v1:172.21.15.107:6827/3334759819]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-03-08T20:55:32.064 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:55:32.078+0000 7fcf77e00700 1 -- 172.21.15.107:0/3323744814 <== mgr.14152 v2:172.21.15.107:6800/3376636618 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fcf80062220 con 0x7fcf640a2910 2024-03-08T20:55:32.064 INFO:teuthology.orchestra.run.smithi107.stdout:Created osd(s) 9 on host 'smithi107' 2024-03-08T20:55:32.066 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:55:32.081+0000 7fcf86200700 1 -- 172.21.15.107:0/3323744814 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fcf640a2910 msgr2=0x7fcf640a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:55:32.067 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:55:32.081+0000 7fcf86200700 1 --2- 172.21.15.107:0/3323744814 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fcf640a2910 0x7fcf640a4dd0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fcf70005950 tx=0x7fcf700058e0 comp rx=0 tx=0).stop 2024-03-08T20:55:32.067 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:55:32.081+0000 7fcf86200700 1 -- 172.21.15.107:0/3323744814 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcf80105260 msgr2=0x7fcf8019a670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:55:32.067 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:55:32.081+0000 7fcf86200700 1 --2- 172.21.15.107:0/3323744814 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcf80105260 0x7fcf8019a670 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fcf6c00d980 tx=0x7fcf6c00dd40 comp rx=0 tx=0).stop 2024-03-08T20:55:32.067 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:55:32.081+0000 7fcf86200700 1 -- 172.21.15.107:0/3323744814 shutdown_connections 2024-03-08T20:55:32.067 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:55:32.081+0000 7fcf86200700 1 --2- 172.21.15.107:0/3323744814 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcf80104060 0x7fcf8019a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:55:32.067 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:55:32.081+0000 7fcf86200700 1 --2- 172.21.15.107:0/3323744814 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fcf640a2910 0x7fcf640a4dd0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:55:32.067 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:55:32.081+0000 7fcf86200700 1 --2- 172.21.15.107:0/3323744814 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcf80105c00 0x7fcf801a16f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:55:32.067 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:55:32.081+0000 7fcf86200700 1 --2- 172.21.15.107:0/3323744814 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcf80105260 0x7fcf8019a670 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:55:32.067 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:55:32.081+0000 7fcf86200700 1 -- 172.21.15.107:0/3323744814 >> 172.21.15.107:0/3323744814 conn(0x7fcf800ff600 msgr2=0x7fcf80108560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:55:32.067 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:55:32.081+0000 7fcf86200700 1 -- 172.21.15.107:0/3323744814 shutdown_connections 2024-03-08T20:55:32.067 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:55:32.082+0000 7fcf86200700 1 -- 172.21.15.107:0/3323744814 wait complete. 2024-03-08T20:55:32.846 DEBUG:teuthology.orchestra.run.smithi107:osd.9> sudo journalctl -f -n 0 -u ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715@osd.9.service 2024-03-08T20:55:32.851 INFO:tasks.cephadm:Deploying osd.10 on smithi149 with /dev/vg_nvme/lv_1... 2024-03-08T20:55:32.851 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dc20f0f8-dd8c-11ee-95c6-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-03-08T20:55:33.131 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:55:32 smithi204 ceph-mon[34935]: from='osd.9 [v2:172.21.15.107:6826/3334759819,v1:172.21.15.107:6827/3334759819]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-03-08T20:55:33.131 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:55:32 smithi204 ceph-mon[34935]: osdmap e66: 10 total, 9 up, 10 in 2024-03-08T20:55:33.131 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:55:32 smithi204 ceph-mon[34935]: from='osd.9 [v2:172.21.15.107:6826/3334759819,v1:172.21.15.107:6827/3334759819]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-03-08T20:55:33.131 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:55:32 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-08T20:55:33.131 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:55:32 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:55:33.132 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:55:32 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T20:55:33.132 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:55:32 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:55:33.132 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:55:32 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:55:33.132 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:55:32 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:55:33.132 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:55:32 smithi204 ceph-mon[34935]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-03-08T20:55:33.132 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:55:32 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T20:55:33.220 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:55:32 smithi107 ceph-mon[31381]: from='osd.9 [v2:172.21.15.107:6826/3334759819,v1:172.21.15.107:6827/3334759819]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-03-08T20:55:33.220 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:55:32 smithi107 ceph-mon[31381]: osdmap e66: 10 total, 9 up, 10 in 2024-03-08T20:55:33.220 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:55:32 smithi107 ceph-mon[31381]: from='osd.9 [v2:172.21.15.107:6826/3334759819,v1:172.21.15.107:6827/3334759819]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-03-08T20:55:33.220 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:55:32 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-08T20:55:33.220 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:55:32 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:55:33.220 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:55:32 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T20:55:33.220 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:55:32 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:55:33.220 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:55:32 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:55:33.220 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:55:32 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:55:33.220 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:55:32 smithi107 ceph-mon[31381]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-03-08T20:55:33.220 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:55:32 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T20:55:33.267 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:55:32 smithi149 ceph-mon[35042]: from='osd.9 [v2:172.21.15.107:6826/3334759819,v1:172.21.15.107:6827/3334759819]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-03-08T20:55:33.267 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:55:32 smithi149 ceph-mon[35042]: osdmap e66: 10 total, 9 up, 10 in 2024-03-08T20:55:33.268 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:55:32 smithi149 ceph-mon[35042]: from='osd.9 [v2:172.21.15.107:6826/3334759819,v1:172.21.15.107:6827/3334759819]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-03-08T20:55:33.268 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:55:32 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-08T20:55:33.268 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:55:32 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:55:33.268 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:55:32 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T20:55:33.268 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:55:32 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:55:33.268 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:55:32 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:55:33.268 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:55:32 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:55:33.268 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:55:32 smithi149 ceph-mon[35042]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-03-08T20:55:33.268 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:55:32 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T20:55:33.293 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/mon.b/config 2024-03-08T20:55:33.693 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 20:55:33 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T20:55:33.395+0000 7f4539400700 -1 osd.9 0 waiting for initial osdmap 2024-03-08T20:55:33.693 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 20:55:33 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T20:55:33.431+0000 7f452e800700 -1 osd.9 67 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-08T20:55:33.970 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:55:33 smithi107 ceph-mon[31381]: from='osd.9 [v2:172.21.15.107:6826/3334759819,v1:172.21.15.107:6827/3334759819]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]': finished 2024-03-08T20:55:33.970 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:55:33 smithi107 ceph-mon[31381]: osdmap e67: 10 total, 9 up, 10 in 2024-03-08T20:55:33.970 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:55:33 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-08T20:55:33.970 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:55:33 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-08T20:55:33.971 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:55:33 smithi107 ceph-mon[31381]: from='osd.9 [v2:172.21.15.107:6826/3334759819,v1:172.21.15.107:6827/3334759819]' entity='osd.9' 2024-03-08T20:55:34.018 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:55:33 smithi149 ceph-mon[35042]: from='osd.9 [v2:172.21.15.107:6826/3334759819,v1:172.21.15.107:6827/3334759819]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]': finished 2024-03-08T20:55:34.018 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:55:33 smithi149 ceph-mon[35042]: osdmap e67: 10 total, 9 up, 10 in 2024-03-08T20:55:34.018 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:55:33 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-08T20:55:34.018 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:55:33 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-08T20:55:34.018 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:55:33 smithi149 ceph-mon[35042]: from='osd.9 [v2:172.21.15.107:6826/3334759819,v1:172.21.15.107:6827/3334759819]' entity='osd.9' 2024-03-08T20:55:34.131 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:55:33 smithi204 ceph-mon[34935]: from='osd.9 [v2:172.21.15.107:6826/3334759819,v1:172.21.15.107:6827/3334759819]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]': finished 2024-03-08T20:55:34.131 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:55:33 smithi204 ceph-mon[34935]: osdmap e67: 10 total, 9 up, 10 in 2024-03-08T20:55:34.131 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:55:33 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-08T20:55:34.131 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:55:33 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-08T20:55:34.131 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:55:33 smithi204 ceph-mon[34935]: from='osd.9 [v2:172.21.15.107:6826/3334759819,v1:172.21.15.107:6827/3334759819]' entity='osd.9' 2024-03-08T20:55:34.970 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:55:34 smithi107 ceph-mon[31381]: purged_snaps scrub starts 2024-03-08T20:55:34.970 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:55:34 smithi107 ceph-mon[31381]: purged_snaps scrub ok 2024-03-08T20:55:34.970 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:55:34 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-08T20:55:34.970 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:55:34 smithi107 ceph-mon[31381]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-03-08T20:55:34.970 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:55:34 smithi107 ceph-mon[31381]: osd.9 [v2:172.21.15.107:6826/3334759819,v1:172.21.15.107:6827/3334759819] boot 2024-03-08T20:55:34.970 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:55:34 smithi107 ceph-mon[31381]: osdmap e68: 10 total, 10 up, 10 in 2024-03-08T20:55:34.970 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:55:34 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-08T20:55:35.059 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:55:34 smithi149 ceph-mon[35042]: purged_snaps scrub starts 2024-03-08T20:55:35.059 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:55:34 smithi149 ceph-mon[35042]: purged_snaps scrub ok 2024-03-08T20:55:35.059 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:55:34 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-08T20:55:35.059 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:55:34 smithi149 ceph-mon[35042]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-03-08T20:55:35.059 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:55:34 smithi149 ceph-mon[35042]: osd.9 [v2:172.21.15.107:6826/3334759819,v1:172.21.15.107:6827/3334759819] boot 2024-03-08T20:55:35.059 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:55:34 smithi149 ceph-mon[35042]: osdmap e68: 10 total, 10 up, 10 in 2024-03-08T20:55:35.059 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:55:34 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-08T20:55:35.131 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:55:34 smithi204 ceph-mon[34935]: purged_snaps scrub starts 2024-03-08T20:55:35.131 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:55:34 smithi204 ceph-mon[34935]: purged_snaps scrub ok 2024-03-08T20:55:35.131 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:55:34 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-08T20:55:35.131 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:55:34 smithi204 ceph-mon[34935]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-03-08T20:55:35.131 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:55:34 smithi204 ceph-mon[34935]: osd.9 [v2:172.21.15.107:6826/3334759819,v1:172.21.15.107:6827/3334759819] boot 2024-03-08T20:55:35.131 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:55:34 smithi204 ceph-mon[34935]: osdmap e68: 10 total, 10 up, 10 in 2024-03-08T20:55:35.131 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:55:34 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-08T20:55:36.047 INFO:teuthology.orchestra.run.smithi149.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-03-08T20:55:36.048 INFO:teuthology.orchestra.run.smithi149.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-03-08T20:55:36.048 INFO:teuthology.orchestra.run.smithi149.stderr: stderr: 10+0 records in 2024-03-08T20:55:36.048 INFO:teuthology.orchestra.run.smithi149.stderr:10+0 records out 2024-03-08T20:55:36.048 INFO:teuthology.orchestra.run.smithi149.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.036338 s, 289 MB/s 2024-03-08T20:55:36.048 INFO:teuthology.orchestra.run.smithi149.stderr: stderr: 2024-03-08T20:55:36.048 INFO:teuthology.orchestra.run.smithi149.stderr:--> Zapping successful for: 2024-03-08T20:55:36.720 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:55:36 smithi107 ceph-mon[31381]: osdmap e69: 10 total, 10 up, 10 in 2024-03-08T20:55:36.767 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:55:36 smithi149 ceph-mon[35042]: osdmap e69: 10 total, 10 up, 10 in 2024-03-08T20:55:36.878 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dc20f0f8-dd8c-11ee-95c6-87774f69a715 -- ceph orch daemon add osd smithi149:vg_nvme/lv_1 2024-03-08T20:55:36.880 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:55:36 smithi204 ceph-mon[34935]: osdmap e69: 10 total, 10 up, 10 in 2024-03-08T20:55:37.324 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/mon.b/config 2024-03-08T20:55:37.469 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:55:37 smithi107 ceph-mon[31381]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-08T20:55:37.658 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:55:37 smithi149 ceph-mon[35042]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-08T20:55:37.880 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:55:37 smithi204 ceph-mon[34935]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-08T20:55:39.165 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:55:39 smithi149 ceph-mon[35042]: Detected new or changed devices on smithi107 2024-03-08T20:55:39.165 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:55:39 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:55:39.165 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:55:39 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:55:39.165 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:55:39 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-08T20:55:39.166 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:55:39 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-08T20:55:39.166 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:55:39 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-08T20:55:39.166 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:55:39 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-08T20:55:39.166 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:55:39 smithi149 ceph-mon[35042]: Adjusting osd_memory_target on smithi107 to 4272M 2024-03-08T20:55:39.166 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:55:39 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:55:39.166 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:55:39 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:55:39.166 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:55:39 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T20:55:39.166 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:55:39 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:55:39.166 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:55:39 smithi149 ceph-mon[35042]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-08T20:55:39.381 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:55:39 smithi204 ceph-mon[34935]: Detected new or changed devices on smithi107 2024-03-08T20:55:39.381 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:55:39 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:55:39.381 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:55:39 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:55:39.381 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:55:39 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-08T20:55:39.381 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:55:39 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-08T20:55:39.382 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:55:39 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-08T20:55:39.382 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:55:39 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-08T20:55:39.382 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:55:39 smithi204 ceph-mon[34935]: Adjusting osd_memory_target on smithi107 to 4272M 2024-03-08T20:55:39.382 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:55:39 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:55:39.382 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:55:39 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:55:39.382 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:55:39 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T20:55:39.382 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:55:39 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:55:39.382 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:55:39 smithi204 ceph-mon[34935]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-08T20:55:39.467 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:55:39.481+0000 7f9da8e00700 1 -- 172.21.15.149:0/3766355702 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9da4105bc0 msgr2=0x7f9da410c470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:55:39.467 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:55:39.481+0000 7f9da8e00700 1 --2- 172.21.15.149:0/3766355702 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9da4105bc0 0x7f9da410c470 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f9d9c00b0d0 tx=0x7f9d9c00b3e0 comp rx=0 tx=0).stop 2024-03-08T20:55:39.467 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:55:39.482+0000 7f9da8e00700 1 -- 172.21.15.149:0/3766355702 shutdown_connections 2024-03-08T20:55:39.467 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:55:39.482+0000 7f9da8e00700 1 --2- 172.21.15.149:0/3766355702 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9da4104020 0x7f9da4104440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:55:39.468 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:55:39.482+0000 7f9da8e00700 1 --2- 172.21.15.149:0/3766355702 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9da4105bc0 0x7f9da410c470 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:55:39.468 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:55:39.482+0000 7f9da8e00700 1 --2- 172.21.15.149:0/3766355702 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9da4105220 0x7f9da4105680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:55:39.468 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:55:39.482+0000 7f9da8e00700 1 -- 172.21.15.149:0/3766355702 >> 172.21.15.149:0/3766355702 conn(0x7f9da40ff5a0 msgr2=0x7f9da4101a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:55:39.468 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:55:39.482+0000 7f9da8e00700 1 -- 172.21.15.149:0/3766355702 shutdown_connections 2024-03-08T20:55:39.468 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:55:39.482+0000 7f9da8e00700 1 -- 172.21.15.149:0/3766355702 wait complete. 2024-03-08T20:55:39.469 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:55:39.483+0000 7f9da8e00700 1 Processor -- start 2024-03-08T20:55:39.470 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:55:39.483+0000 7f9da8e00700 1 -- start start 2024-03-08T20:55:39.470 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:55:39.483+0000 7f9da8e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9da4104020 0x7f9da419a150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:55:39.470 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:55:39.483+0000 7f9da8e00700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9da4105220 0x7f9da419a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:55:39.470 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:55:39.483+0000 7f9da8e00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9da4105bc0 0x7f9da41a1710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:55:39.470 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:55:39.483+0000 7f9da8e00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9da410fb10 con 0x7f9da4104020 2024-03-08T20:55:39.470 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:55:39.483+0000 7f9da8e00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9da410f990 con 0x7f9da4105bc0 2024-03-08T20:55:39.470 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:55:39.483+0000 7f9da8e00700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9da410fc90 con 0x7f9da4105220 2024-03-08T20:55:39.470 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:55:39.484+0000 7f9da2800700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9da4105bc0 0x7f9da41a1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:55:39.470 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:55:39.484+0000 7f9da2800700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9da4105bc0 0x7f9da41a1710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.149:48364/0 (socket says 172.21.15.149:48364) 2024-03-08T20:55:39.471 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:55:39.484+0000 7f9da2800700 1 -- 172.21.15.149:0/3529489643 learned_addr learned my addr 172.21.15.149:0/3529489643 (peer_addr_for_me v2:172.21.15.149:0/0) 2024-03-08T20:55:39.471 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:55:39.484+0000 7f9da1400700 1 --2- 172.21.15.149:0/3529489643 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9da4105220 0x7f9da419a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:55:39.471 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:55:39.484+0000 7f9da1e00700 1 --2- 172.21.15.149:0/3529489643 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9da4104020 0x7f9da419a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:55:39.471 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:55:39.484+0000 7f9da2800700 1 -- 172.21.15.149:0/3529489643 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9da4105220 msgr2=0x7f9da419a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:55:39.471 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:55:39.484+0000 7f9da2800700 1 --2- 172.21.15.149:0/3529489643 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9da4105220 0x7f9da419a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:55:39.471 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:55:39.484+0000 7f9da2800700 1 -- 172.21.15.149:0/3529489643 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9da4104020 msgr2=0x7f9da419a150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:55:39.471 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:55:39.484+0000 7f9da2800700 1 --2- 172.21.15.149:0/3529489643 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9da4104020 0x7f9da419a150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:55:39.471 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:55:39.484+0000 7f9da2800700 1 -- 172.21.15.149:0/3529489643 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9da41a1db0 con 0x7f9da4105bc0 2024-03-08T20:55:39.471 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:55:39.485+0000 7f9da2800700 1 --2- 172.21.15.149:0/3529489643 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9da4105bc0 0x7f9da41a1710 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f9d9c004c80 tx=0x7f9d9c004d60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:55:39.472 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:55:39 smithi107 ceph-mon[31381]: Detected new or changed devices on smithi107 2024-03-08T20:55:39.472 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:55:39 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:55:39.472 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:55:39 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:55:39.472 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:55:39 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-08T20:55:39.472 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:55:39 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-08T20:55:39.472 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:55:39 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-08T20:55:39.472 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:55:39 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-08T20:55:39.472 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:55:39 smithi107 ceph-mon[31381]: Adjusting osd_memory_target on smithi107 to 4272M 2024-03-08T20:55:39.472 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:55:39 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:55:39.473 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:55:39 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:55:39.473 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:55:39 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T20:55:39.473 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:55:39 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:55:39.473 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:55:39 smithi107 ceph-mon[31381]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-08T20:55:39.474 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:55:39.485+0000 7f9d9aa00700 1 -- 172.21.15.149:0/3529489643 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d9c01f030 con 0x7f9da4105bc0 2024-03-08T20:55:39.474 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:55:39.485+0000 7f9d9aa00700 1 -- 172.21.15.149:0/3529489643 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f9d9c007ac0 con 0x7f9da4105bc0 2024-03-08T20:55:39.474 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:55:39.485+0000 7f9da8e00700 1 -- 172.21.15.149:0/3529489643 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9da41a2040 con 0x7f9da4105bc0 2024-03-08T20:55:39.474 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:55:39.485+0000 7f9d9aa00700 1 -- 172.21.15.149:0/3529489643 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d9c01b690 con 0x7f9da4105bc0 2024-03-08T20:55:39.474 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:55:39.485+0000 7f9da8e00700 1 -- 172.21.15.149:0/3529489643 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9da41a2490 con 0x7f9da4105bc0 2024-03-08T20:55:39.474 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:55:39.486+0000 7f9da8e00700 1 -- 172.21.15.149:0/3529489643 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9da4068aa0 con 0x7f9da4105bc0 2024-03-08T20:55:39.474 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:55:39.488+0000 7f9d9aa00700 1 -- 172.21.15.149:0/3529489643 <== mon.2 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 16) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f9d9c022030 con 0x7f9da4105bc0 2024-03-08T20:55:39.475 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:55:39.490+0000 7f9d9aa00700 1 --2- 172.21.15.149:0/3529489643 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f9d880a2910 0x7f9d880a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:55:39.475 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:55:39.490+0000 7f9d9aa00700 1 -- 172.21.15.149:0/3529489643 <== mon.2 v2:172.21.15.149:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6261+0+0 (secure 0 0 0) 0x7f9d9c0cee20 con 0x7f9da4105bc0 2024-03-08T20:55:39.476 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:55:39.490+0000 7f9da1e00700 1 --2- 172.21.15.149:0/3529489643 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f9d880a2910 0x7f9d880a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:55:39.476 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:55:39.491+0000 7f9da1e00700 1 --2- 172.21.15.149:0/3529489643 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f9d880a2910 0x7f9d880a4dd0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f9d94005950 tx=0x7f9d940058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:55:39.478 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:55:39.493+0000 7f9d9aa00700 1 -- 172.21.15.149:0/3529489643 <== mon.2 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f9d9c09c6b0 con 0x7f9da4105bc0 2024-03-08T20:55:39.709 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:55:39.724+0000 7f9da8e00700 1 -- 172.21.15.149:0/3529489643 --> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f9da40622e0 con 0x7f9d880a2910 2024-03-08T20:55:40.380 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:55:40 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-08T20:55:40.381 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:55:40 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-08T20:55:40.381 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:55:40 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:55:40.469 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:55:40 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-08T20:55:40.470 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:55:40 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-08T20:55:40.470 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:55:40 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:55:40.517 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:55:40 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-08T20:55:40.517 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:55:40 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-08T20:55:40.517 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:55:40 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:55:41.267 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:55:41 smithi149 ceph-mon[35042]: from='client.34322 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-08T20:55:41.267 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:55:41 smithi149 ceph-mon[35042]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-08T20:55:41.380 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:55:41 smithi204 ceph-mon[34935]: from='client.34322 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-08T20:55:41.380 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:55:41 smithi204 ceph-mon[34935]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-08T20:55:41.469 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:55:41 smithi107 ceph-mon[31381]: from='client.34322 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi149:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-08T20:55:41.470 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:55:41 smithi107 ceph-mon[31381]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-08T20:55:43.630 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:55:43 smithi204 ceph-mon[34935]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-08T20:55:43.719 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:55:43 smithi107 ceph-mon[31381]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-08T20:55:43.767 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:55:43 smithi149 ceph-mon[35042]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-08T20:55:44.517 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:55:44 smithi149 ceph-mon[35042]: from='client.? 172.21.15.149:0/2474853321' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "338945b2-20c0-40b2-aaca-062cd4adee04"}]: dispatch 2024-03-08T20:55:44.517 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:55:44 smithi149 ceph-mon[35042]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "338945b2-20c0-40b2-aaca-062cd4adee04"}]: dispatch 2024-03-08T20:55:44.518 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:55:44 smithi149 ceph-mon[35042]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "338945b2-20c0-40b2-aaca-062cd4adee04"}]': finished 2024-03-08T20:55:44.518 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:55:44 smithi149 ceph-mon[35042]: osdmap e70: 11 total, 10 up, 11 in 2024-03-08T20:55:44.518 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:55:44 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-08T20:55:44.630 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:55:44 smithi204 ceph-mon[34935]: from='client.? 172.21.15.149:0/2474853321' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "338945b2-20c0-40b2-aaca-062cd4adee04"}]: dispatch 2024-03-08T20:55:44.630 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:55:44 smithi204 ceph-mon[34935]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "338945b2-20c0-40b2-aaca-062cd4adee04"}]: dispatch 2024-03-08T20:55:44.630 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:55:44 smithi204 ceph-mon[34935]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "338945b2-20c0-40b2-aaca-062cd4adee04"}]': finished 2024-03-08T20:55:44.631 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:55:44 smithi204 ceph-mon[34935]: osdmap e70: 11 total, 10 up, 11 in 2024-03-08T20:55:44.631 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:55:44 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-08T20:55:44.720 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:55:44 smithi107 ceph-mon[31381]: from='client.? 172.21.15.149:0/2474853321' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "338945b2-20c0-40b2-aaca-062cd4adee04"}]: dispatch 2024-03-08T20:55:44.720 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:55:44 smithi107 ceph-mon[31381]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "338945b2-20c0-40b2-aaca-062cd4adee04"}]: dispatch 2024-03-08T20:55:44.720 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:55:44 smithi107 ceph-mon[31381]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "338945b2-20c0-40b2-aaca-062cd4adee04"}]': finished 2024-03-08T20:55:44.720 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:55:44 smithi107 ceph-mon[31381]: osdmap e70: 11 total, 10 up, 11 in 2024-03-08T20:55:44.720 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:55:44 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-08T20:55:45.373 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:55:45 smithi107 ceph-mon[31381]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-08T20:55:45.373 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:55:45 smithi107 ceph-mon[31381]: from='client.? 172.21.15.149:0/2706810788' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-08T20:55:45.630 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:55:45 smithi204 ceph-mon[34935]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-08T20:55:45.630 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:55:45 smithi204 ceph-mon[34935]: from='client.? 172.21.15.149:0/2706810788' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-08T20:55:45.767 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:55:45 smithi149 ceph-mon[35042]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-08T20:55:45.767 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:55:45 smithi149 ceph-mon[35042]: from='client.? 172.21.15.149:0/2706810788' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-08T20:55:47.630 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:55:47 smithi204 ceph-mon[34935]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-08T20:55:47.719 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:55:47 smithi107 ceph-mon[31381]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-08T20:55:47.746 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:55:47 smithi149 ceph-mon[35042]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-08T20:55:49.630 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:55:49 smithi204 ceph-mon[34935]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-08T20:55:49.705 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:55:49 smithi149 ceph-mon[35042]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-08T20:55:49.719 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:55:49 smithi107 ceph-mon[31381]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-08T20:55:51.517 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:55:51 smithi149 ceph-mon[35042]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-08T20:55:51.630 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:55:51 smithi204 ceph-mon[34935]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-08T20:55:51.719 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:55:51 smithi107 ceph-mon[31381]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-08T20:55:53.630 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:55:53 smithi204 ceph-mon[34935]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-08T20:55:53.630 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:55:53 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-03-08T20:55:53.630 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:55:53 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:55:53.635 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:55:53 smithi149 ceph-mon[35042]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-08T20:55:53.636 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:55:53 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-03-08T20:55:53.636 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:55:53 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:55:53.719 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:55:53 smithi107 ceph-mon[31381]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-08T20:55:53.720 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:55:53 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-03-08T20:55:53.720 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:55:53 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:55:54.630 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:55:54 smithi204 ceph-mon[34935]: Deploying daemon osd.10 on smithi149 2024-03-08T20:55:54.657 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:55:54 smithi149 ceph-mon[35042]: Deploying daemon osd.10 on smithi149 2024-03-08T20:55:54.719 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:55:54 smithi107 ceph-mon[31381]: Deploying daemon osd.10 on smithi149 2024-03-08T20:55:55.517 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:55:55 smithi149 ceph-mon[35042]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-08T20:55:55.630 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:55:55 smithi204 ceph-mon[34935]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-08T20:55:55.719 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:55:55 smithi107 ceph-mon[31381]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-08T20:55:57.630 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:55:57 smithi204 ceph-mon[34935]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-08T20:55:57.644 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:55:57 smithi149 ceph-mon[35042]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-08T20:55:57.719 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:55:57 smithi107 ceph-mon[31381]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-08T20:55:59.517 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:55:59 smithi149 ceph-mon[35042]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-08T20:55:59.630 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:55:59 smithi204 ceph-mon[34935]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-08T20:55:59.719 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:55:59 smithi107 ceph-mon[31381]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-08T20:56:01.517 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:56:01 smithi149 ceph-mon[35042]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-08T20:56:01.517 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:56:01 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:56:01.517 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:56:01 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T20:56:01.517 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:56:01 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:56:01.630 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:56:01 smithi204 ceph-mon[34935]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-08T20:56:01.630 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:56:01 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:56:01.631 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:56:01 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T20:56:01.631 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:56:01 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:56:01.719 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:56:01 smithi107 ceph-mon[31381]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-08T20:56:01.720 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:56:01 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:56:01.720 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:56:01 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T20:56:01.720 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:56:01 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:56:03.719 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:56:03 smithi107 ceph-mon[31381]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-08T20:56:03.767 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:56:03 smithi149 ceph-mon[35042]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-08T20:56:03.880 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:56:03 smithi204 ceph-mon[34935]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-08T20:56:05.481 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:56:05 smithi149 ceph-mon[35042]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-08T20:56:05.481 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:56:05 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:56:05.481 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:56:05 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:56:05.481 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:56:05 smithi149 ceph-mon[35042]: from='osd.10 [v2:172.21.15.149:6824/3469284631,v1:172.21.15.149:6825/3469284631]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-08T20:56:05.482 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:56:05 smithi149 ceph-mon[35042]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-08T20:56:05.562 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:56:05.576+0000 7f9d9aa00700 1 -- 172.21.15.149:0/3529489643 <== mgr.14152 v2:172.21.15.107:6800/3376636618 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7f9da40622e0 con 0x7f9d880a2910 2024-03-08T20:56:05.562 INFO:teuthology.orchestra.run.smithi149.stdout:Created osd(s) 10 on host 'smithi149' 2024-03-08T20:56:05.565 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:56:05.579+0000 7f9da8e00700 1 -- 172.21.15.149:0/3529489643 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f9d880a2910 msgr2=0x7f9d880a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:56:05.565 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:56:05.579+0000 7f9da8e00700 1 --2- 172.21.15.149:0/3529489643 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f9d880a2910 0x7f9d880a4dd0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f9d94005950 tx=0x7f9d940058e0 comp rx=0 tx=0).stop 2024-03-08T20:56:05.565 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:56:05.579+0000 7f9da8e00700 1 -- 172.21.15.149:0/3529489643 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9da4105bc0 msgr2=0x7f9da41a1710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:56:05.565 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:56:05.579+0000 7f9da8e00700 1 --2- 172.21.15.149:0/3529489643 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9da4105bc0 0x7f9da41a1710 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f9d9c004c80 tx=0x7f9d9c004d60 comp rx=0 tx=0).stop 2024-03-08T20:56:05.566 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:56:05.579+0000 7f9da8e00700 1 -- 172.21.15.149:0/3529489643 shutdown_connections 2024-03-08T20:56:05.566 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:56:05.579+0000 7f9da8e00700 1 --2- 172.21.15.149:0/3529489643 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f9d880a2910 0x7f9d880a4dd0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:56:05.566 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:56:05.579+0000 7f9da8e00700 1 --2- 172.21.15.149:0/3529489643 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9da4105bc0 0x7f9da41a1710 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:56:05.566 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:56:05.579+0000 7f9da8e00700 1 --2- 172.21.15.149:0/3529489643 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9da4104020 0x7f9da419a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:56:05.566 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:56:05.579+0000 7f9da8e00700 1 --2- 172.21.15.149:0/3529489643 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9da4105220 0x7f9da419a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:56:05.566 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:56:05.579+0000 7f9da8e00700 1 -- 172.21.15.149:0/3529489643 >> 172.21.15.149:0/3529489643 conn(0x7f9da40ff5a0 msgr2=0x7f9da4108570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:56:05.566 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:56:05.579+0000 7f9da8e00700 1 -- 172.21.15.149:0/3529489643 shutdown_connections 2024-03-08T20:56:05.566 INFO:teuthology.orchestra.run.smithi149.stderr:2024-03-08T20:56:05.579+0000 7f9da8e00700 1 -- 172.21.15.149:0/3529489643 wait complete. 2024-03-08T20:56:05.719 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:56:05 smithi107 ceph-mon[31381]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-08T20:56:05.719 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:56:05 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:56:05.720 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:56:05 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:56:05.720 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:56:05 smithi107 ceph-mon[31381]: from='osd.10 [v2:172.21.15.149:6824/3469284631,v1:172.21.15.149:6825/3469284631]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-08T20:56:05.720 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:56:05 smithi107 ceph-mon[31381]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-08T20:56:05.880 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:56:05 smithi204 ceph-mon[34935]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-08T20:56:05.881 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:56:05 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:56:05.881 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:56:05 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:56:05.881 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:56:05 smithi204 ceph-mon[34935]: from='osd.10 [v2:172.21.15.149:6824/3469284631,v1:172.21.15.149:6825/3469284631]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-08T20:56:05.881 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:56:05 smithi204 ceph-mon[34935]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-08T20:56:06.323 DEBUG:teuthology.orchestra.run.smithi149:osd.10> sudo journalctl -f -n 0 -u ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715@osd.10.service 2024-03-08T20:56:06.328 INFO:tasks.cephadm:Deploying osd.11 on smithi204 with /dev/vg_nvme/lv_1... 2024-03-08T20:56:06.329 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dc20f0f8-dd8c-11ee-95c6-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-03-08T20:56:06.760 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/mon.c/config 2024-03-08T20:56:06.880 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:56:06 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:56:06.880 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:56:06 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:56:06.880 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:56:06 smithi204 ceph-mon[34935]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-03-08T20:56:06.881 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:56:06 smithi204 ceph-mon[34935]: from='osd.10 [v2:172.21.15.149:6824/3469284631,v1:172.21.15.149:6825/3469284631]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-03-08T20:56:06.881 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:56:06 smithi204 ceph-mon[34935]: osdmap e71: 11 total, 10 up, 11 in 2024-03-08T20:56:06.881 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:56:06 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-08T20:56:06.881 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:56:06 smithi204 ceph-mon[34935]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-03-08T20:56:06.881 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:56:06 smithi204 ceph-mon[34935]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-08T20:56:06.911 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:56:06 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:56:06.911 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:56:06 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:56:06.911 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:56:06 smithi149 ceph-mon[35042]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-03-08T20:56:06.911 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:56:06 smithi149 ceph-mon[35042]: from='osd.10 [v2:172.21.15.149:6824/3469284631,v1:172.21.15.149:6825/3469284631]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-03-08T20:56:06.911 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:56:06 smithi149 ceph-mon[35042]: osdmap e71: 11 total, 10 up, 11 in 2024-03-08T20:56:06.912 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:56:06 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-08T20:56:06.912 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:56:06 smithi149 ceph-mon[35042]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-03-08T20:56:06.912 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:56:06 smithi149 ceph-mon[35042]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-08T20:56:06.969 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:56:06 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:56:06.969 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:56:06 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:56:06.969 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:56:06 smithi107 ceph-mon[31381]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-03-08T20:56:06.970 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:56:06 smithi107 ceph-mon[31381]: from='osd.10 [v2:172.21.15.149:6824/3469284631,v1:172.21.15.149:6825/3469284631]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-03-08T20:56:06.970 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:56:06 smithi107 ceph-mon[31381]: osdmap e71: 11 total, 10 up, 11 in 2024-03-08T20:56:06.970 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:56:06 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-08T20:56:06.970 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:56:06 smithi107 ceph-mon[31381]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]: dispatch 2024-03-08T20:56:06.970 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:56:06 smithi107 ceph-mon[31381]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-03-08T20:56:07.478 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 20:56:07 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T20:56:07.232+0000 7f77a3200700 -1 osd.10 0 waiting for initial osdmap 2024-03-08T20:56:07.478 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 20:56:07 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T20:56:07.269+0000 7f7799000700 -1 osd.10 72 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-08T20:56:07.767 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:56:07 smithi149 ceph-mon[35042]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]': finished 2024-03-08T20:56:07.767 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:56:07 smithi149 ceph-mon[35042]: osdmap e72: 11 total, 10 up, 11 in 2024-03-08T20:56:07.767 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:56:07 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-08T20:56:07.767 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:56:07 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-08T20:56:07.767 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:56:07 smithi149 ceph-mon[35042]: from='osd.10 ' entity='osd.10' 2024-03-08T20:56:07.949 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:56:07 smithi204 ceph-mon[34935]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]': finished 2024-03-08T20:56:07.949 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:56:07 smithi204 ceph-mon[34935]: osdmap e72: 11 total, 10 up, 11 in 2024-03-08T20:56:07.949 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:56:07 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-08T20:56:07.949 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:56:07 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-08T20:56:07.949 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:56:07 smithi204 ceph-mon[34935]: from='osd.10 ' entity='osd.10' 2024-03-08T20:56:07.969 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:56:07 smithi107 ceph-mon[31381]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi149", "root=default"]}]': finished 2024-03-08T20:56:07.969 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:56:07 smithi107 ceph-mon[31381]: osdmap e72: 11 total, 10 up, 11 in 2024-03-08T20:56:07.969 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:56:07 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-08T20:56:07.969 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:56:07 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-08T20:56:07.969 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:56:07 smithi107 ceph-mon[31381]: from='osd.10 ' entity='osd.10' 2024-03-08T20:56:08.861 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:56:08 smithi204 ceph-mon[34935]: purged_snaps scrub starts 2024-03-08T20:56:08.861 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:56:08 smithi204 ceph-mon[34935]: purged_snaps scrub ok 2024-03-08T20:56:08.861 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:56:08 smithi204 ceph-mon[34935]: osd.10 [v2:172.21.15.149:6824/3469284631,v1:172.21.15.149:6825/3469284631] boot 2024-03-08T20:56:08.861 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:56:08 smithi204 ceph-mon[34935]: osdmap e73: 11 total, 11 up, 11 in 2024-03-08T20:56:08.862 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:56:08 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-08T20:56:08.862 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:56:08 smithi204 ceph-mon[34935]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 983 GiB / 983 GiB avail 2024-03-08T20:56:08.969 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:56:08 smithi107 ceph-mon[31381]: purged_snaps scrub starts 2024-03-08T20:56:08.969 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:56:08 smithi107 ceph-mon[31381]: purged_snaps scrub ok 2024-03-08T20:56:08.970 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:56:08 smithi107 ceph-mon[31381]: osd.10 [v2:172.21.15.149:6824/3469284631,v1:172.21.15.149:6825/3469284631] boot 2024-03-08T20:56:08.970 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:56:08 smithi107 ceph-mon[31381]: osdmap e73: 11 total, 11 up, 11 in 2024-03-08T20:56:08.970 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:56:08 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-08T20:56:08.970 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:56:08 smithi107 ceph-mon[31381]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 983 GiB / 983 GiB avail 2024-03-08T20:56:09.017 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:56:08 smithi149 ceph-mon[35042]: purged_snaps scrub starts 2024-03-08T20:56:09.017 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:56:08 smithi149 ceph-mon[35042]: purged_snaps scrub ok 2024-03-08T20:56:09.017 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:56:08 smithi149 ceph-mon[35042]: osd.10 [v2:172.21.15.149:6824/3469284631,v1:172.21.15.149:6825/3469284631] boot 2024-03-08T20:56:09.017 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:56:08 smithi149 ceph-mon[35042]: osdmap e73: 11 total, 11 up, 11 in 2024-03-08T20:56:09.018 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:56:08 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-08T20:56:09.018 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:56:08 smithi149 ceph-mon[35042]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 348 MiB used, 983 GiB / 983 GiB avail 2024-03-08T20:56:09.726 INFO:teuthology.orchestra.run.smithi204.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-03-08T20:56:09.727 INFO:teuthology.orchestra.run.smithi204.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-03-08T20:56:09.727 INFO:teuthology.orchestra.run.smithi204.stderr: stderr: 10+0 records in 2024-03-08T20:56:09.727 INFO:teuthology.orchestra.run.smithi204.stderr:10+0 records out 2024-03-08T20:56:09.727 INFO:teuthology.orchestra.run.smithi204.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0362402 s, 289 MB/s 2024-03-08T20:56:09.727 INFO:teuthology.orchestra.run.smithi204.stderr:--> Zapping successful for: 2024-03-08T20:56:09.969 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:56:09 smithi107 ceph-mon[31381]: osdmap e74: 11 total, 11 up, 11 in 2024-03-08T20:56:10.016 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:56:09 smithi149 ceph-mon[35042]: osdmap e74: 11 total, 11 up, 11 in 2024-03-08T20:56:10.130 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:56:09 smithi204 ceph-mon[34935]: osdmap e74: 11 total, 11 up, 11 in 2024-03-08T20:56:10.593 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dc20f0f8-dd8c-11ee-95c6-87774f69a715 -- ceph orch daemon add osd smithi204:vg_nvme/lv_1 2024-03-08T20:56:10.880 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:56:10 smithi204 ceph-mon[34935]: osdmap e75: 11 total, 11 up, 11 in 2024-03-08T20:56:10.880 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:56:10 smithi204 ceph-mon[34935]: pgmap v300: 1 pgs: 1 remapped+peering; 577 KiB data, 348 MiB used, 983 GiB / 983 GiB avail 2024-03-08T20:56:10.969 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:56:10 smithi107 ceph-mon[31381]: osdmap e75: 11 total, 11 up, 11 in 2024-03-08T20:56:10.969 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:56:10 smithi107 ceph-mon[31381]: pgmap v300: 1 pgs: 1 remapped+peering; 577 KiB data, 348 MiB used, 983 GiB / 983 GiB avail 2024-03-08T20:56:11.013 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/mon.c/config 2024-03-08T20:56:11.018 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:56:10 smithi149 ceph-mon[35042]: osdmap e75: 11 total, 11 up, 11 in 2024-03-08T20:56:11.018 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:56:10 smithi149 ceph-mon[35042]: pgmap v300: 1 pgs: 1 remapped+peering; 577 KiB data, 348 MiB used, 983 GiB / 983 GiB avail 2024-03-08T20:56:11.970 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:56:11 smithi107 ceph-mon[31381]: osdmap e76: 11 total, 11 up, 11 in 2024-03-08T20:56:11.970 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:56:11 smithi107 ceph-mon[31381]: Detected new or changed devices on smithi149 2024-03-08T20:56:11.970 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:56:11 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:56:11.970 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:56:11 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:56:11.970 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:56:11 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-08T20:56:11.970 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:56:11 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-08T20:56:11.970 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:56:11 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-08T20:56:11.970 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:56:11 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-08T20:56:11.971 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:56:11 smithi107 ceph-mon[31381]: Adjusting osd_memory_target on smithi149 to 1454M 2024-03-08T20:56:11.971 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:56:11 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:56:11.971 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:56:11 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:56:11.971 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:56:11 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T20:56:11.971 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:56:11 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:56:11.971 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:56:11 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T20:56:11.971 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:56:11 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:56:11.971 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:56:11 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T20:56:11.971 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:56:11 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:56:12.018 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:56:11 smithi149 ceph-mon[35042]: osdmap e76: 11 total, 11 up, 11 in 2024-03-08T20:56:12.018 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:56:11 smithi149 ceph-mon[35042]: Detected new or changed devices on smithi149 2024-03-08T20:56:12.018 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:56:11 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:56:12.018 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:56:11 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:56:12.019 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:56:11 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-08T20:56:12.019 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:56:11 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-08T20:56:12.019 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:56:11 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-08T20:56:12.019 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:56:11 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-08T20:56:12.019 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:56:11 smithi149 ceph-mon[35042]: Adjusting osd_memory_target on smithi149 to 1454M 2024-03-08T20:56:12.019 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:56:11 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:56:12.019 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:56:11 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:56:12.019 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:56:11 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T20:56:12.019 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:56:11 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:56:12.019 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:56:11 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T20:56:12.019 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:56:11 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:56:12.019 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:56:11 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T20:56:12.019 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:56:11 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:56:12.039 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:56:11 smithi204 ceph-mon[34935]: osdmap e76: 11 total, 11 up, 11 in 2024-03-08T20:56:12.039 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:56:11 smithi204 ceph-mon[34935]: Detected new or changed devices on smithi149 2024-03-08T20:56:12.039 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:56:11 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:56:12.039 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:56:11 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:56:12.039 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:56:11 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-08T20:56:12.039 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:56:11 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-08T20:56:12.040 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:56:11 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-08T20:56:12.040 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:56:11 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-08T20:56:12.040 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:56:11 smithi204 ceph-mon[34935]: Adjusting osd_memory_target on smithi149 to 1454M 2024-03-08T20:56:12.040 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:56:11 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:56:12.040 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:56:11 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:56:12.040 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:56:11 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T20:56:12.040 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:56:11 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:56:12.040 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:56:11 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T20:56:12.040 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:56:11 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:56:12.041 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:56:11 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T20:56:12.041 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:56:11 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:56:12.797 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:56:12 smithi204 ceph-mon[34935]: osdmap e77: 11 total, 11 up, 11 in 2024-03-08T20:56:12.797 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:56:12 smithi204 ceph-mon[34935]: pgmap v303: 1 pgs: 1 remapped+peering; 577 KiB data, 348 MiB used, 983 GiB / 983 GiB avail 2024-03-08T20:56:12.969 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:56:12 smithi107 ceph-mon[31381]: osdmap e77: 11 total, 11 up, 11 in 2024-03-08T20:56:12.969 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:56:12 smithi107 ceph-mon[31381]: pgmap v303: 1 pgs: 1 remapped+peering; 577 KiB data, 348 MiB used, 983 GiB / 983 GiB avail 2024-03-08T20:56:13.017 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:56:12 smithi149 ceph-mon[35042]: osdmap e77: 11 total, 11 up, 11 in 2024-03-08T20:56:13.017 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:56:12 smithi149 ceph-mon[35042]: pgmap v303: 1 pgs: 1 remapped+peering; 577 KiB data, 348 MiB used, 983 GiB / 983 GiB avail 2024-03-08T20:56:13.333 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:56:13.348+0000 7ff70fe00700 1 -- 172.21.15.204:0/2101408921 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff710109550 msgr2=0x7ff7101099b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:56:13.334 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:56:13.348+0000 7ff70fe00700 1 --2- 172.21.15.204:0/2101408921 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff710109550 0x7ff7101099b0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7ff6f8009990 tx=0x7ff6f8009ca0 comp rx=0 tx=0).stop 2024-03-08T20:56:13.334 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:56:13.348+0000 7ff70fe00700 1 -- 172.21.15.204:0/2101408921 shutdown_connections 2024-03-08T20:56:13.334 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:56:13.348+0000 7ff70fe00700 1 --2- 172.21.15.204:0/2101408921 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff710109ef0 0x7ff7101107a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:56:13.334 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:56:13.348+0000 7ff70fe00700 1 --2- 172.21.15.204:0/2101408921 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff710108350 0x7ff710108770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:56:13.334 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:56:13.348+0000 7ff70fe00700 1 --2- 172.21.15.204:0/2101408921 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff710109550 0x7ff7101099b0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:56:13.334 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:56:13.348+0000 7ff70fe00700 1 -- 172.21.15.204:0/2101408921 >> 172.21.15.204:0/2101408921 conn(0x7ff710077320 msgr2=0x7ff710079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:56:13.334 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:56:13.348+0000 7ff70fe00700 1 -- 172.21.15.204:0/2101408921 shutdown_connections 2024-03-08T20:56:13.334 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:56:13.348+0000 7ff70fe00700 1 -- 172.21.15.204:0/2101408921 wait complete. 2024-03-08T20:56:13.335 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:56:13.349+0000 7ff70fe00700 1 Processor -- start 2024-03-08T20:56:13.335 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:56:13.350+0000 7ff70fe00700 1 -- start start 2024-03-08T20:56:13.335 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:56:13.350+0000 7ff70fe00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff710108350 0x7ff71019e420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:56:13.336 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:56:13.350+0000 7ff70fe00700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff710109550 0x7ff71019e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:56:13.336 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:56:13.350+0000 7ff70fe00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff710109ef0 0x7ff7101a59e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:56:13.336 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:56:13.350+0000 7ff70fe00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff710113e40 con 0x7ff710109ef0 2024-03-08T20:56:13.336 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:56:13.350+0000 7ff70fe00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff710113cc0 con 0x7ff710108350 2024-03-08T20:56:13.336 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:56:13.350+0000 7ff70fe00700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff710113fc0 con 0x7ff710109550 2024-03-08T20:56:13.336 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:56:13.350+0000 7ff706000700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff710109550 0x7ff71019e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:56:13.336 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:56:13.350+0000 7ff706000700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff710109550 0x7ff71019e960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:53984/0 (socket says 172.21.15.204:53984) 2024-03-08T20:56:13.336 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:56:13.350+0000 7ff706000700 1 -- 172.21.15.204:0/2801417088 learned_addr learned my addr 172.21.15.204:0/2801417088 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-03-08T20:56:13.336 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:56:13.351+0000 7ff70ea00700 1 --2- 172.21.15.204:0/2801417088 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff710108350 0x7ff71019e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:56:13.336 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:56:13.351+0000 7ff70f400700 1 --2- 172.21.15.204:0/2801417088 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff710109ef0 0x7ff7101a59e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:56:13.337 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:56:13.351+0000 7ff706000700 1 -- 172.21.15.204:0/2801417088 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff710108350 msgr2=0x7ff71019e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:56:13.337 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:56:13.351+0000 7ff706000700 1 --2- 172.21.15.204:0/2801417088 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff710108350 0x7ff71019e420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:56:13.337 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:56:13.351+0000 7ff706000700 1 -- 172.21.15.204:0/2801417088 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff710109ef0 msgr2=0x7ff7101a59e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:56:13.337 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:56:13.351+0000 7ff706000700 1 --2- 172.21.15.204:0/2801417088 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff710109ef0 0x7ff7101a59e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:56:13.337 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:56:13.351+0000 7ff706000700 1 -- 172.21.15.204:0/2801417088 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7101a6080 con 0x7ff710109550 2024-03-08T20:56:13.337 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:56:13.351+0000 7ff706000700 1 --2- 172.21.15.204:0/2801417088 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff710109550 0x7ff71019e960 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7ff6f80057c0 tx=0x7ff6f80057f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:56:13.337 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:56:13.352+0000 7ff707e00700 1 -- 172.21.15.204:0/2801417088 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6f8027070 con 0x7ff710109550 2024-03-08T20:56:13.337 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:56:13.352+0000 7ff707e00700 1 -- 172.21.15.204:0/2801417088 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7ff6f8004080 con 0x7ff710109550 2024-03-08T20:56:13.338 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:56:13.352+0000 7ff707e00700 1 -- 172.21.15.204:0/2801417088 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6f8022de0 con 0x7ff710109550 2024-03-08T20:56:13.338 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:56:13.352+0000 7ff70fe00700 1 -- 172.21.15.204:0/2801417088 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7101a6310 con 0x7ff710109550 2024-03-08T20:56:13.338 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:56:13.352+0000 7ff70fe00700 1 -- 172.21.15.204:0/2801417088 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7101a6710 con 0x7ff710109550 2024-03-08T20:56:13.338 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:56:13.353+0000 7ff70fe00700 1 -- 172.21.15.204:0/2801417088 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff710068aa0 con 0x7ff710109550 2024-03-08T20:56:13.339 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:56:13.355+0000 7ff707e00700 1 -- 172.21.15.204:0/2801417088 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 16) v1 ==== 137291+0+0 (secure 0 0 0) 0x7ff6f802a070 con 0x7ff710109550 2024-03-08T20:56:13.342 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:56:13.357+0000 7ff707e00700 1 --2- 172.21.15.204:0/2801417088 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7ff6f40a2890 0x7ff6f40a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:56:13.342 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:56:13.357+0000 7ff707e00700 1 -- 172.21.15.204:0/2801417088 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6694+0+0 (secure 0 0 0) 0x7ff6f80cf360 con 0x7ff710109550 2024-03-08T20:56:13.342 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:56:13.358+0000 7ff70ea00700 1 --2- 172.21.15.204:0/2801417088 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7ff6f40a2890 0x7ff6f40a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:56:13.343 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:56:13.358+0000 7ff70ea00700 1 --2- 172.21.15.204:0/2801417088 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7ff6f40a2890 0x7ff6f40a4d50 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7ff7101093b0 tx=0x7ff70000a3b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:56:13.347 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:56:13.362+0000 7ff707e00700 1 -- 172.21.15.204:0/2801417088 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7ff6f805e260 con 0x7ff710109550 2024-03-08T20:56:13.579 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:56:13.594+0000 7ff70fe00700 1 -- 172.21.15.204:0/2801417088 --> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7ff710062220 con 0x7ff6f40a2890 2024-03-08T20:56:13.880 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:56:13 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-08T20:56:13.880 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:56:13 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-08T20:56:13.880 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:56:13 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:56:14.017 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:56:13 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-08T20:56:14.017 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:56:13 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-08T20:56:14.017 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:56:13 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:56:14.219 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:56:13 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-08T20:56:14.219 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:56:13 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-08T20:56:14.219 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:56:13 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:56:14.849 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:56:14 smithi204 ceph-mon[34935]: from='client.24349 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-08T20:56:14.850 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:56:14 smithi204 ceph-mon[34935]: pgmap v304: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 103 KiB/s, 0 objects/s recovering 2024-03-08T20:56:15.017 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:56:14 smithi149 ceph-mon[35042]: from='client.24349 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-08T20:56:15.017 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:56:14 smithi149 ceph-mon[35042]: pgmap v304: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 103 KiB/s, 0 objects/s recovering 2024-03-08T20:56:15.219 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:56:14 smithi107 ceph-mon[31381]: from='client.24349 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi204:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-08T20:56:15.219 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:56:14 smithi107 ceph-mon[31381]: pgmap v304: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 103 KiB/s, 0 objects/s recovering 2024-03-08T20:56:17.630 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:56:17 smithi204 ceph-mon[34935]: pgmap v305: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-03-08T20:56:17.719 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:56:17 smithi107 ceph-mon[31381]: pgmap v305: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-03-08T20:56:17.767 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:56:17 smithi149 ceph-mon[35042]: pgmap v305: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-03-08T20:56:18.630 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:56:18 smithi204 ceph-mon[34935]: from='client.? 172.21.15.204:0/2316015980' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "acccfe4f-d307-4a64-8f4d-fba27bfbe09d"}]: dispatch 2024-03-08T20:56:18.630 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:56:18 smithi204 ceph-mon[34935]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "acccfe4f-d307-4a64-8f4d-fba27bfbe09d"}]: dispatch 2024-03-08T20:56:18.630 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:56:18 smithi204 ceph-mon[34935]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "acccfe4f-d307-4a64-8f4d-fba27bfbe09d"}]': finished 2024-03-08T20:56:18.630 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:56:18 smithi204 ceph-mon[34935]: osdmap e78: 12 total, 11 up, 12 in 2024-03-08T20:56:18.631 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:56:18 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-08T20:56:18.719 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:56:18 smithi107 ceph-mon[31381]: from='client.? 172.21.15.204:0/2316015980' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "acccfe4f-d307-4a64-8f4d-fba27bfbe09d"}]: dispatch 2024-03-08T20:56:18.719 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:56:18 smithi107 ceph-mon[31381]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "acccfe4f-d307-4a64-8f4d-fba27bfbe09d"}]: dispatch 2024-03-08T20:56:18.720 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:56:18 smithi107 ceph-mon[31381]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "acccfe4f-d307-4a64-8f4d-fba27bfbe09d"}]': finished 2024-03-08T20:56:18.720 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:56:18 smithi107 ceph-mon[31381]: osdmap e78: 12 total, 11 up, 12 in 2024-03-08T20:56:18.720 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:56:18 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-08T20:56:18.767 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:56:18 smithi149 ceph-mon[35042]: from='client.? 172.21.15.204:0/2316015980' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "acccfe4f-d307-4a64-8f4d-fba27bfbe09d"}]: dispatch 2024-03-08T20:56:18.767 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:56:18 smithi149 ceph-mon[35042]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "acccfe4f-d307-4a64-8f4d-fba27bfbe09d"}]: dispatch 2024-03-08T20:56:18.767 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:56:18 smithi149 ceph-mon[35042]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "acccfe4f-d307-4a64-8f4d-fba27bfbe09d"}]': finished 2024-03-08T20:56:18.767 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:56:18 smithi149 ceph-mon[35042]: osdmap e78: 12 total, 11 up, 12 in 2024-03-08T20:56:18.768 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:56:18 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-08T20:56:19.630 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:56:19 smithi204 ceph-mon[34935]: pgmap v307: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-03-08T20:56:19.630 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:56:19 smithi204 ceph-mon[34935]: from='client.? 172.21.15.204:0/4018123698' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-08T20:56:19.719 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:56:19 smithi107 ceph-mon[31381]: pgmap v307: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-03-08T20:56:19.719 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:56:19 smithi107 ceph-mon[31381]: from='client.? 172.21.15.204:0/4018123698' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-08T20:56:19.767 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:56:19 smithi149 ceph-mon[35042]: pgmap v307: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-03-08T20:56:19.767 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:56:19 smithi149 ceph-mon[35042]: from='client.? 172.21.15.204:0/4018123698' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-08T20:56:21.631 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:56:21 smithi204 ceph-mon[34935]: pgmap v308: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-03-08T20:56:21.719 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:56:21 smithi107 ceph-mon[31381]: pgmap v308: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-03-08T20:56:21.767 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:56:21 smithi149 ceph-mon[35042]: pgmap v308: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-03-08T20:56:23.630 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:56:23 smithi204 ceph-mon[34935]: pgmap v309: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-08T20:56:23.719 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:56:23 smithi107 ceph-mon[31381]: pgmap v309: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-08T20:56:23.767 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:56:23 smithi149 ceph-mon[35042]: pgmap v309: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-08T20:56:25.513 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:56:25 smithi204 ceph-mon[34935]: pgmap v310: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-03-08T20:56:25.719 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:56:25 smithi107 ceph-mon[31381]: pgmap v310: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-03-08T20:56:25.767 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:56:25 smithi149 ceph-mon[35042]: pgmap v310: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-03-08T20:56:27.630 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:56:27 smithi204 ceph-mon[34935]: pgmap v311: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-03-08T20:56:27.630 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:56:27 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-03-08T20:56:27.630 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:56:27 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:56:27.719 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:56:27 smithi107 ceph-mon[31381]: pgmap v311: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-03-08T20:56:27.719 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:56:27 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-03-08T20:56:27.719 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:56:27 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:56:27.767 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:56:27 smithi149 ceph-mon[35042]: pgmap v311: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-03-08T20:56:27.767 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:56:27 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-03-08T20:56:27.767 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:56:27 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:56:28.630 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:56:28 smithi204 ceph-mon[34935]: Deploying daemon osd.11 on smithi204 2024-03-08T20:56:28.719 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:56:28 smithi107 ceph-mon[31381]: Deploying daemon osd.11 on smithi204 2024-03-08T20:56:28.767 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:56:28 smithi149 ceph-mon[35042]: Deploying daemon osd.11 on smithi204 2024-03-08T20:56:29.630 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:56:29 smithi204 ceph-mon[34935]: pgmap v312: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-03-08T20:56:29.719 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:56:29 smithi107 ceph-mon[31381]: pgmap v312: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-03-08T20:56:29.767 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:56:29 smithi149 ceph-mon[35042]: pgmap v312: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-03-08T20:56:31.612 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:56:31 smithi204 ceph-mon[34935]: pgmap v313: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-03-08T20:56:31.719 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:56:31 smithi107 ceph-mon[31381]: pgmap v313: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-03-08T20:56:31.766 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:56:31 smithi149 ceph-mon[35042]: pgmap v313: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-03-08T20:56:33.719 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:56:33 smithi107 ceph-mon[31381]: pgmap v314: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-03-08T20:56:33.766 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:56:33 smithi149 ceph-mon[35042]: pgmap v314: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-03-08T20:56:33.862 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:56:33 smithi204 ceph-mon[34935]: pgmap v314: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-03-08T20:56:35.719 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:56:35 smithi107 ceph-mon[31381]: pgmap v315: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-03-08T20:56:35.766 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:56:35 smithi149 ceph-mon[35042]: pgmap v315: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-03-08T20:56:35.880 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:56:35 smithi204 ceph-mon[34935]: pgmap v315: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-03-08T20:56:36.880 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:56:36 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:56:36.880 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:56:36 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T20:56:36.880 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:56:36 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:56:36.880 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:56:36 smithi204 ceph-mon[34935]: pgmap v316: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-03-08T20:56:36.969 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:56:36 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:56:36.970 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:56:36 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T20:56:36.970 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:56:36 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:56:36.970 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:56:36 smithi107 ceph-mon[31381]: pgmap v316: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-03-08T20:56:37.017 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:56:36 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:56:37.017 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:56:36 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T20:56:37.017 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:56:36 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:56:37.017 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:56:36 smithi149 ceph-mon[35042]: pgmap v316: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-03-08T20:56:39.629 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:56:39 smithi204 ceph-mon[34935]: pgmap v317: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-03-08T20:56:39.719 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:56:39 smithi107 ceph-mon[31381]: pgmap v317: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-03-08T20:56:39.719 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:56:39 smithi149 ceph-mon[35042]: pgmap v317: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-03-08T20:56:40.620 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:56:40 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:56:40.620 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:56:40 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:56:40.620 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:56:40 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:56:40.620 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:56:40 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T20:56:40.620 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:56:40 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:56:40.620 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:56:40 smithi204 ceph-mon[34935]: from='osd.11 [v2:172.21.15.204:6824/3097772473,v1:172.21.15.204:6825/3097772473]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-08T20:56:40.620 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:56:40 smithi204 ceph-mon[34935]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-08T20:56:40.719 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:56:40 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:56:40.719 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:56:40 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:56:40.719 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:56:40 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:56:40.719 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:56:40 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T20:56:40.719 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:56:40 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:56:40.719 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:56:40 smithi107 ceph-mon[31381]: from='osd.11 [v2:172.21.15.204:6824/3097772473,v1:172.21.15.204:6825/3097772473]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-08T20:56:40.719 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:56:40 smithi107 ceph-mon[31381]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-08T20:56:40.754 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:56:40.767+0000 7ff707e00700 1 -- 172.21.15.204:0/2801417088 <== mgr.14152 v2:172.21.15.107:6800/3376636618 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7ff710062220 con 0x7ff6f40a2890 2024-03-08T20:56:40.755 INFO:teuthology.orchestra.run.smithi204.stdout:Created osd(s) 11 on host 'smithi204' 2024-03-08T20:56:40.757 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:56:40.771+0000 7ff70fe00700 1 -- 172.21.15.204:0/2801417088 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7ff6f40a2890 msgr2=0x7ff6f40a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:56:40.758 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:56:40.771+0000 7ff70fe00700 1 --2- 172.21.15.204:0/2801417088 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7ff6f40a2890 0x7ff6f40a4d50 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7ff7101093b0 tx=0x7ff70000a3b0 comp rx=0 tx=0).stop 2024-03-08T20:56:40.758 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:56:40.771+0000 7ff70fe00700 1 -- 172.21.15.204:0/2801417088 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff710109550 msgr2=0x7ff71019e960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:56:40.758 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:56:40.771+0000 7ff70fe00700 1 --2- 172.21.15.204:0/2801417088 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff710109550 0x7ff71019e960 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7ff6f80057c0 tx=0x7ff6f80057f0 comp rx=0 tx=0).stop 2024-03-08T20:56:40.758 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:56:40.771+0000 7ff70fe00700 1 -- 172.21.15.204:0/2801417088 shutdown_connections 2024-03-08T20:56:40.758 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:56:40.771+0000 7ff70fe00700 1 --2- 172.21.15.204:0/2801417088 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7ff6f40a2890 0x7ff6f40a4d50 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:56:40.758 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:56:40.771+0000 7ff70fe00700 1 --2- 172.21.15.204:0/2801417088 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff710109ef0 0x7ff7101a59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:56:40.758 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:56:40.771+0000 7ff70fe00700 1 --2- 172.21.15.204:0/2801417088 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff710108350 0x7ff71019e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:56:40.759 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:56:40.771+0000 7ff70fe00700 1 --2- 172.21.15.204:0/2801417088 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff710109550 0x7ff71019e960 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:56:40.759 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:56:40.771+0000 7ff70fe00700 1 -- 172.21.15.204:0/2801417088 >> 172.21.15.204:0/2801417088 conn(0x7ff710077320 msgr2=0x7ff71010c7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:56:40.759 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:56:40.771+0000 7ff70fe00700 1 -- 172.21.15.204:0/2801417088 shutdown_connections 2024-03-08T20:56:40.759 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:56:40.771+0000 7ff70fe00700 1 -- 172.21.15.204:0/2801417088 wait complete. 2024-03-08T20:56:40.767 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:56:40 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:56:40.767 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:56:40 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:56:40.767 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:56:40 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:56:40.767 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:56:40 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T20:56:40.767 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:56:40 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:56:40.767 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:56:40 smithi149 ceph-mon[35042]: from='osd.11 [v2:172.21.15.204:6824/3097772473,v1:172.21.15.204:6825/3097772473]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-08T20:56:40.767 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:56:40 smithi149 ceph-mon[35042]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-08T20:56:41.553 DEBUG:teuthology.orchestra.run.smithi204:osd.11> sudo journalctl -f -n 0 -u ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715@osd.11.service 2024-03-08T20:56:41.558 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-03-08T20:56:41.558 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dc20f0f8-dd8c-11ee-95c6-87774f69a715 -- ceph osd stat -f json 2024-03-08T20:56:41.719 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:56:41 smithi107 ceph-mon[31381]: pgmap v318: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-03-08T20:56:41.719 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:56:41 smithi107 ceph-mon[31381]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-03-08T20:56:41.719 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:56:41 smithi107 ceph-mon[31381]: osdmap e79: 12 total, 11 up, 12 in 2024-03-08T20:56:41.719 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:56:41 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-08T20:56:41.719 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:56:41 smithi107 ceph-mon[31381]: from='osd.11 [v2:172.21.15.204:6824/3097772473,v1:172.21.15.204:6825/3097772473]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-03-08T20:56:41.719 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:56:41 smithi107 ceph-mon[31381]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-03-08T20:56:41.719 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:56:41 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:56:41.719 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:56:41 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T20:56:41.720 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:56:41 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:56:41.767 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:56:41 smithi149 ceph-mon[35042]: pgmap v318: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-03-08T20:56:41.767 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:56:41 smithi149 ceph-mon[35042]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-03-08T20:56:41.767 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:56:41 smithi149 ceph-mon[35042]: osdmap e79: 12 total, 11 up, 12 in 2024-03-08T20:56:41.767 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:56:41 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-08T20:56:41.767 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:56:41 smithi149 ceph-mon[35042]: from='osd.11 [v2:172.21.15.204:6824/3097772473,v1:172.21.15.204:6825/3097772473]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-03-08T20:56:41.767 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:56:41 smithi149 ceph-mon[35042]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-03-08T20:56:41.767 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:56:41 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:56:41.767 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:56:41 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T20:56:41.767 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:56:41 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:56:41.785 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:56:41 smithi204 ceph-mon[34935]: pgmap v318: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-03-08T20:56:41.785 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:56:41 smithi204 ceph-mon[34935]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-03-08T20:56:41.785 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:56:41 smithi204 ceph-mon[34935]: osdmap e79: 12 total, 11 up, 12 in 2024-03-08T20:56:41.785 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:56:41 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-08T20:56:41.785 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:56:41 smithi204 ceph-mon[34935]: from='osd.11 [v2:172.21.15.204:6824/3097772473,v1:172.21.15.204:6825/3097772473]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-03-08T20:56:41.785 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:56:41 smithi204 ceph-mon[34935]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]: dispatch 2024-03-08T20:56:41.785 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:56:41 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:56:41.785 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:56:41 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T20:56:41.785 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:56:41 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:56:42.003 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/mon.a/config 2024-03-08T20:56:42.381 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 20:56:42 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T20:56:42.129+0000 7fdccb400700 -1 osd.11 0 waiting for initial osdmap 2024-03-08T20:56:42.381 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 20:56:42 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T20:56:42.157+0000 7fdcc1200700 -1 osd.11 80 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-08T20:56:42.719 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:56:42 smithi107 ceph-mon[31381]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]': finished 2024-03-08T20:56:42.719 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:56:42 smithi107 ceph-mon[31381]: osdmap e80: 12 total, 11 up, 12 in 2024-03-08T20:56:42.719 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:56:42 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-08T20:56:42.719 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:56:42 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-08T20:56:42.719 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:56:42 smithi107 ceph-mon[31381]: from='osd.11 ' entity='osd.11' 2024-03-08T20:56:42.766 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:56:42 smithi149 ceph-mon[35042]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]': finished 2024-03-08T20:56:42.767 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:56:42 smithi149 ceph-mon[35042]: osdmap e80: 12 total, 11 up, 12 in 2024-03-08T20:56:42.767 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:56:42 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-08T20:56:42.767 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:56:42 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-08T20:56:42.767 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:56:42 smithi149 ceph-mon[35042]: from='osd.11 ' entity='osd.11' 2024-03-08T20:56:42.880 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:56:42 smithi204 ceph-mon[34935]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi204", "root=default"]}]': finished 2024-03-08T20:56:42.880 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:56:42 smithi204 ceph-mon[34935]: osdmap e80: 12 total, 11 up, 12 in 2024-03-08T20:56:42.880 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:56:42 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-08T20:56:42.880 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:56:42 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-08T20:56:42.880 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:56:42 smithi204 ceph-mon[34935]: from='osd.11 ' entity='osd.11' 2024-03-08T20:56:43.623 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:56:43 smithi204 ceph-mon[34935]: purged_snaps scrub starts 2024-03-08T20:56:43.623 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:56:43 smithi204 ceph-mon[34935]: purged_snaps scrub ok 2024-03-08T20:56:43.623 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:56:43 smithi204 ceph-mon[34935]: pgmap v321: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-03-08T20:56:43.623 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:56:43 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-08T20:56:43.623 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:56:43 smithi204 ceph-mon[34935]: osd.11 [v2:172.21.15.204:6824/3097772473,v1:172.21.15.204:6825/3097772473] boot 2024-03-08T20:56:43.623 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:56:43 smithi204 ceph-mon[34935]: osdmap e81: 12 total, 12 up, 12 in 2024-03-08T20:56:43.623 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:56:43 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-08T20:56:43.719 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:56:43 smithi107 ceph-mon[31381]: purged_snaps scrub starts 2024-03-08T20:56:43.719 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:56:43 smithi107 ceph-mon[31381]: purged_snaps scrub ok 2024-03-08T20:56:43.719 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:56:43 smithi107 ceph-mon[31381]: pgmap v321: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-03-08T20:56:43.719 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:56:43 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-08T20:56:43.719 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:56:43 smithi107 ceph-mon[31381]: osd.11 [v2:172.21.15.204:6824/3097772473,v1:172.21.15.204:6825/3097772473] boot 2024-03-08T20:56:43.719 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:56:43 smithi107 ceph-mon[31381]: osdmap e81: 12 total, 12 up, 12 in 2024-03-08T20:56:43.719 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:56:43 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-08T20:56:43.767 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:56:43 smithi149 ceph-mon[35042]: purged_snaps scrub starts 2024-03-08T20:56:43.767 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:56:43 smithi149 ceph-mon[35042]: purged_snaps scrub ok 2024-03-08T20:56:43.767 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:56:43 smithi149 ceph-mon[35042]: pgmap v321: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-03-08T20:56:43.767 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:56:43 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-08T20:56:43.768 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:56:43 smithi149 ceph-mon[35042]: osd.11 [v2:172.21.15.204:6824/3097772473,v1:172.21.15.204:6825/3097772473] boot 2024-03-08T20:56:43.768 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:56:43 smithi149 ceph-mon[35042]: osdmap e81: 12 total, 12 up, 12 in 2024-03-08T20:56:43.768 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:56:43 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-08T20:56:44.321 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:44.335+0000 7fe936c00700 1 -- 172.21.15.107:0/2487246734 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe930075e10 msgr2=0x7fe930074270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:56:44.321 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:44.335+0000 7fe936c00700 1 --2- 172.21.15.107:0/2487246734 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe930075e10 0x7fe930074270 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fe920009990 tx=0x7fe920009ca0 comp rx=0 tx=0).stop 2024-03-08T20:56:44.321 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:44.336+0000 7fe936c00700 1 -- 172.21.15.107:0/2487246734 shutdown_connections 2024-03-08T20:56:44.321 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:44.336+0000 7fe936c00700 1 --2- 172.21.15.107:0/2487246734 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe930109cf0 0x7fe93010c180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:56:44.321 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:44.336+0000 7fe936c00700 1 --2- 172.21.15.107:0/2487246734 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe930075e10 0x7fe930074270 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:56:44.321 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:44.336+0000 7fe936c00700 1 --2- 172.21.15.107:0/2487246734 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe930074840 0x7fe930074cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:56:44.321 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:44.336+0000 7fe936c00700 1 -- 172.21.15.107:0/2487246734 >> 172.21.15.107:0/2487246734 conn(0x7fe9300fd380 msgr2=0x7fe9300ff800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:56:44.321 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:44.336+0000 7fe936c00700 1 -- 172.21.15.107:0/2487246734 shutdown_connections 2024-03-08T20:56:44.321 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:44.336+0000 7fe936c00700 1 -- 172.21.15.107:0/2487246734 wait complete. 2024-03-08T20:56:44.322 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:44.337+0000 7fe936c00700 1 Processor -- start 2024-03-08T20:56:44.322 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:44.337+0000 7fe936c00700 1 -- start start 2024-03-08T20:56:44.322 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:44.338+0000 7fe936c00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe930074840 0x7fe93019a210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:56:44.322 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:44.338+0000 7fe936c00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe930075e10 0x7fe93019a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:56:44.322 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:44.338+0000 7fe936c00700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe930109cf0 0x7fe9301a17d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:56:44.322 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:44.338+0000 7fe936c00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe93010fb10 con 0x7fe930075e10 2024-03-08T20:56:44.322 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:44.338+0000 7fe936c00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe93010f990 con 0x7fe930074840 2024-03-08T20:56:44.323 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:44.338+0000 7fe936c00700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe93010fc90 con 0x7fe930109cf0 2024-03-08T20:56:44.324 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:44.338+0000 7fe92f400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe930075e10 0x7fe93019a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:56:44.324 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:44.338+0000 7fe92f400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe930075e10 0x7fe93019a750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46724/0 (socket says 172.21.15.107:46724) 2024-03-08T20:56:44.324 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:44.338+0000 7fe92f400700 1 -- 172.21.15.107:0/1097268722 learned_addr learned my addr 172.21.15.107:0/1097268722 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T20:56:44.324 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:44.338+0000 7fe92fe00700 1 --2- 172.21.15.107:0/1097268722 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe930074840 0x7fe93019a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:56:44.324 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:44.339+0000 7fe934c00700 1 --2- 172.21.15.107:0/1097268722 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe930109cf0 0x7fe9301a17d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:56:44.324 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:44.339+0000 7fe92f400700 1 -- 172.21.15.107:0/1097268722 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe930109cf0 msgr2=0x7fe9301a17d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:56:44.324 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:44.339+0000 7fe92f400700 1 --2- 172.21.15.107:0/1097268722 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe930109cf0 0x7fe9301a17d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:56:44.324 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:44.339+0000 7fe92f400700 1 -- 172.21.15.107:0/1097268722 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe930074840 msgr2=0x7fe93019a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:56:44.325 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:44.339+0000 7fe92f400700 1 --2- 172.21.15.107:0/1097268722 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe930074840 0x7fe93019a210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:56:44.325 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:44.339+0000 7fe92f400700 1 -- 172.21.15.107:0/1097268722 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9301a1e70 con 0x7fe930075e10 2024-03-08T20:56:44.325 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:44.339+0000 7fe92fe00700 1 --2- 172.21.15.107:0/1097268722 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe930074840 0x7fe93019a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T20:56:44.325 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:44.339+0000 7fe92f400700 1 --2- 172.21.15.107:0/1097268722 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe930075e10 0x7fe93019a750 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fe91c00c9a0 tx=0x7fe91c00cd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:56:44.325 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:44.339+0000 7fe92cc00700 1 -- 172.21.15.107:0/1097268722 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe91c004220 con 0x7fe930075e10 2024-03-08T20:56:44.325 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:44.340+0000 7fe936c00700 1 -- 172.21.15.107:0/1097268722 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9301a2160 con 0x7fe930075e10 2024-03-08T20:56:44.325 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:44.340+0000 7fe92cc00700 1 -- 172.21.15.107:0/1097268722 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fe91c007d10 con 0x7fe930075e10 2024-03-08T20:56:44.325 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:44.340+0000 7fe92cc00700 1 -- 172.21.15.107:0/1097268722 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe91c003e10 con 0x7fe930075e10 2024-03-08T20:56:44.325 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:44.340+0000 7fe936c00700 1 -- 172.21.15.107:0/1097268722 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9301a26f0 con 0x7fe930075e10 2024-03-08T20:56:44.325 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:44.341+0000 7fe936c00700 1 -- 172.21.15.107:0/1097268722 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe930068aa0 con 0x7fe930075e10 2024-03-08T20:56:44.327 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:44.343+0000 7fe92cc00700 1 -- 172.21.15.107:0/1097268722 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) v1 ==== 137291+0+0 (secure 0 0 0) 0x7fe91c007860 con 0x7fe930075e10 2024-03-08T20:56:44.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:44.344+0000 7fe92cc00700 1 --2- 172.21.15.107:0/1097268722 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fe9140a2910 0x7fe9140a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:56:44.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:44.344+0000 7fe92cc00700 1 -- 172.21.15.107:0/1097268722 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fe91c0ce570 con 0x7fe930075e10 2024-03-08T20:56:44.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:44.345+0000 7fe92fe00700 1 --2- 172.21.15.107:0/1097268722 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fe9140a2910 0x7fe9140a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:56:44.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:44.345+0000 7fe92fe00700 1 --2- 172.21.15.107:0/1097268722 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fe9140a2910 0x7fe9140a4dd0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fe920006010 tx=0x7fe920005e80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:56:44.331 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:44.347+0000 7fe92cc00700 1 -- 172.21.15.107:0/1097268722 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7fe91c014030 con 0x7fe930075e10 2024-03-08T20:56:44.562 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:44.577+0000 7fe936c00700 1 -- 172.21.15.107:0/1097268722 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7fe93002d860 con 0x7fe930075e10 2024-03-08T20:56:44.562 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:44.578+0000 7fe92cc00700 1 -- 172.21.15.107:0/1097268722 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v82) v1 ==== 74+0+133 (secure 0 0 0) 0x7fe91c09bb30 con 0x7fe930075e10 2024-03-08T20:56:44.562 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-08T20:56:44.565 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:44.580+0000 7fe936c00700 1 -- 172.21.15.107:0/1097268722 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fe9140a2910 msgr2=0x7fe9140a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:56:44.565 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:44.580+0000 7fe936c00700 1 --2- 172.21.15.107:0/1097268722 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fe9140a2910 0x7fe9140a4dd0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fe920006010 tx=0x7fe920005e80 comp rx=0 tx=0).stop 2024-03-08T20:56:44.565 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:44.580+0000 7fe936c00700 1 -- 172.21.15.107:0/1097268722 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe930075e10 msgr2=0x7fe93019a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:56:44.565 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:44.580+0000 7fe936c00700 1 --2- 172.21.15.107:0/1097268722 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe930075e10 0x7fe93019a750 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fe91c00c9a0 tx=0x7fe91c00cd60 comp rx=0 tx=0).stop 2024-03-08T20:56:44.565 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:44.580+0000 7fe936c00700 1 -- 172.21.15.107:0/1097268722 shutdown_connections 2024-03-08T20:56:44.565 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:44.580+0000 7fe936c00700 1 --2- 172.21.15.107:0/1097268722 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe930109cf0 0x7fe9301a17d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:56:44.565 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:44.580+0000 7fe936c00700 1 --2- 172.21.15.107:0/1097268722 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fe9140a2910 0x7fe9140a4dd0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:56:44.565 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:44.580+0000 7fe936c00700 1 --2- 172.21.15.107:0/1097268722 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe930074840 0x7fe93019a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:56:44.565 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:44.580+0000 7fe936c00700 1 --2- 172.21.15.107:0/1097268722 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe930075e10 0x7fe93019a750 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:56:44.565 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:44.580+0000 7fe936c00700 1 -- 172.21.15.107:0/1097268722 >> 172.21.15.107:0/1097268722 conn(0x7fe9300fd380 msgr2=0x7fe930105bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:56:44.565 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:44.581+0000 7fe936c00700 1 -- 172.21.15.107:0/1097268722 shutdown_connections 2024-03-08T20:56:44.565 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:44.581+0000 7fe936c00700 1 -- 172.21.15.107:0/1097268722 wait complete. 2024-03-08T20:56:45.434 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":82,"num_osds":12,"num_up_osds":12,"osd_up_since":1709931403,"num_in_osds":12,"osd_in_since":1709931377,"num_remapped_pgs":0} 2024-03-08T20:56:45.436 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761 shell --fsid dc20f0f8-dd8c-11ee-95c6-87774f69a715 -- ceph osd dump --format=json 2024-03-08T20:56:45.639 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:56:45 smithi107 ceph-mon[31381]: pgmap v323: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-08T20:56:45.640 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:56:45 smithi107 ceph-mon[31381]: osdmap e82: 12 total, 12 up, 12 in 2024-03-08T20:56:45.640 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:56:45 smithi107 ceph-mon[31381]: from='client.? 172.21.15.107:0/1097268722' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-08T20:56:45.767 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:56:45 smithi149 ceph-mon[35042]: pgmap v323: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-08T20:56:45.767 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:56:45 smithi149 ceph-mon[35042]: osdmap e82: 12 total, 12 up, 12 in 2024-03-08T20:56:45.767 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:56:45 smithi149 ceph-mon[35042]: from='client.? 172.21.15.107:0/1097268722' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-08T20:56:45.866 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/mon.a/config 2024-03-08T20:56:45.880 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:56:45 smithi204 ceph-mon[34935]: pgmap v323: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-08T20:56:45.880 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:56:45 smithi204 ceph-mon[34935]: osdmap e82: 12 total, 12 up, 12 in 2024-03-08T20:56:45.881 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:56:45 smithi204 ceph-mon[34935]: from='client.? 172.21.15.107:0/1097268722' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-08T20:56:47.615 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:56:47 smithi107 ceph-mon[31381]: pgmap v325: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-08T20:56:47.615 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:56:47 smithi107 ceph-mon[31381]: Detected new or changed devices on smithi204 2024-03-08T20:56:47.615 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:56:47 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:56:47.616 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:56:47 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:56:47.616 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:56:47 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-08T20:56:47.616 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:56:47 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-08T20:56:47.616 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:56:47 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-08T20:56:47.616 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:56:47 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-08T20:56:47.616 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:56:47 smithi107 ceph-mon[31381]: Adjusting osd_memory_target on smithi204 to 4272M 2024-03-08T20:56:47.616 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:56:47 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:56:47.616 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:56:47 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:56:47.616 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:56:47 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T20:56:47.616 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:56:47 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:56:47.767 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:56:47 smithi149 ceph-mon[35042]: pgmap v325: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-08T20:56:47.767 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:56:47 smithi149 ceph-mon[35042]: Detected new or changed devices on smithi204 2024-03-08T20:56:47.767 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:56:47 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:56:47.767 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:56:47 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:56:47.767 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:56:47 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-08T20:56:47.767 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:56:47 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-08T20:56:47.767 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:56:47 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-08T20:56:47.767 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:56:47 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-08T20:56:47.768 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:56:47 smithi149 ceph-mon[35042]: Adjusting osd_memory_target on smithi204 to 4272M 2024-03-08T20:56:47.768 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:56:47 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:56:47.768 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:56:47 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:56:47.768 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:56:47 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T20:56:47.768 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:56:47 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:56:47.880 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:56:47 smithi204 ceph-mon[34935]: pgmap v325: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-08T20:56:47.880 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:56:47 smithi204 ceph-mon[34935]: Detected new or changed devices on smithi204 2024-03-08T20:56:47.880 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:56:47 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:56:47.880 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:56:47 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:56:47.881 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:56:47 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-08T20:56:47.881 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:56:47 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-08T20:56:47.881 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:56:47 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-08T20:56:47.881 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:56:47 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-08T20:56:47.881 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:56:47 smithi204 ceph-mon[34935]: Adjusting osd_memory_target on smithi204 to 4272M 2024-03-08T20:56:47.881 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:56:47 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:56:47.881 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:56:47 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:56:47.881 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:56:47 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T20:56:47.881 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:56:47 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:56:48.141 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:48.156+0000 7f65a5400700 1 -- 172.21.15.107:0/1506530921 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f65a0109310 msgr2=0x7f65a0110ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:56:48.141 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:48.156+0000 7f65a5400700 1 --2- 172.21.15.107:0/1506530921 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f65a0109310 0x7f65a0110ee0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f659800b0d0 tx=0x7f659800b3e0 comp rx=0 tx=0).stop 2024-03-08T20:56:48.141 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:48.156+0000 7f65a5400700 1 -- 172.21.15.107:0/1506530921 shutdown_connections 2024-03-08T20:56:48.141 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:48.156+0000 7f65a5400700 1 --2- 172.21.15.107:0/1506530921 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f65a0104230 0x7f65a0104610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:56:48.141 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:48.156+0000 7f65a5400700 1 --2- 172.21.15.107:0/1506530921 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f65a0109310 0x7f65a0110ee0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:56:48.141 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:48.156+0000 7f65a5400700 1 --2- 172.21.15.107:0/1506530921 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f65a0104be0 0x7f65a0108c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:56:48.141 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:48.156+0000 7f65a5400700 1 -- 172.21.15.107:0/1506530921 >> 172.21.15.107:0/1506530921 conn(0x7f65a00ffac0 msgr2=0x7f65a0101ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:56:48.141 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:48.156+0000 7f65a5400700 1 -- 172.21.15.107:0/1506530921 shutdown_connections 2024-03-08T20:56:48.141 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:48.156+0000 7f65a5400700 1 -- 172.21.15.107:0/1506530921 wait complete. 2024-03-08T20:56:48.142 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:48.157+0000 7f65a5400700 1 Processor -- start 2024-03-08T20:56:48.143 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:48.157+0000 7f65a5400700 1 -- start start 2024-03-08T20:56:48.143 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:48.158+0000 7f65a5400700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f65a0104230 0x7f65a019a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:56:48.143 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:48.158+0000 7f65a5400700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f65a0104be0 0x7f65a019abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:56:48.143 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:48.158+0000 7f65a5400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f65a0109310 0x7f65a019ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:56:48.143 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:48.158+0000 7f65a5400700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65a0114200 con 0x7f65a0109310 2024-03-08T20:56:48.143 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:48.158+0000 7f65a5400700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65a0114080 con 0x7f65a0104230 2024-03-08T20:56:48.143 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:48.158+0000 7f65a5400700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65a0114380 con 0x7f65a0104be0 2024-03-08T20:56:48.143 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:48.158+0000 7f659f200700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f65a0109310 0x7f65a019ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:56:48.143 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:48.158+0000 7f659f200700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f65a0109310 0x7f65a019ef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:57370/0 (socket says 172.21.15.107:57370) 2024-03-08T20:56:48.143 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:48.158+0000 7f659f200700 1 -- 172.21.15.107:0/3012773386 learned_addr learned my addr 172.21.15.107:0/3012773386 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T20:56:48.143 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:48.159+0000 7f659e800700 1 --2- 172.21.15.107:0/3012773386 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f65a0104230 0x7f65a019a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:56:48.143 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:48.158+0000 7f659de00700 1 --2- 172.21.15.107:0/3012773386 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f65a0104be0 0x7f65a019abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:56:48.144 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:48.159+0000 7f659f200700 1 -- 172.21.15.107:0/3012773386 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f65a0104be0 msgr2=0x7f65a019abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:56:48.144 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:48.159+0000 7f659f200700 1 --2- 172.21.15.107:0/3012773386 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f65a0104be0 0x7f65a019abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:56:48.145 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:48.159+0000 7f659f200700 1 -- 172.21.15.107:0/3012773386 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f65a0104230 msgr2=0x7f65a019a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:56:48.145 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:48.159+0000 7f659f200700 1 --2- 172.21.15.107:0/3012773386 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f65a0104230 0x7f65a019a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:56:48.145 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:48.159+0000 7f659f200700 1 -- 172.21.15.107:0/3012773386 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65a019f650 con 0x7f65a0109310 2024-03-08T20:56:48.145 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:48.159+0000 7f659f200700 1 --2- 172.21.15.107:0/3012773386 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f65a0109310 0x7f65a019ef10 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f65980079a0 tx=0x7f6598007a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:56:48.145 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:48.159+0000 7f6597400700 1 -- 172.21.15.107:0/3012773386 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f659801f070 con 0x7f65a0109310 2024-03-08T20:56:48.145 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:48.159+0000 7f6597400700 1 -- 172.21.15.107:0/3012773386 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f6598003c00 con 0x7f65a0109310 2024-03-08T20:56:48.145 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:48.159+0000 7f65a5400700 1 -- 172.21.15.107:0/3012773386 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65a019f8e0 con 0x7f65a0109310 2024-03-08T20:56:48.145 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:48.159+0000 7f6597400700 1 -- 172.21.15.107:0/3012773386 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f659801b6e0 con 0x7f65a0109310 2024-03-08T20:56:48.145 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:48.160+0000 7f65a5400700 1 -- 172.21.15.107:0/3012773386 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65a01a71c0 con 0x7f65a0109310 2024-03-08T20:56:48.145 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:48.161+0000 7f65a5400700 1 -- 172.21.15.107:0/3012773386 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65a0068aa0 con 0x7f65a0109310 2024-03-08T20:56:48.146 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:48.163+0000 7f6597400700 1 -- 172.21.15.107:0/3012773386 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f6598022070 con 0x7f65a0109310 2024-03-08T20:56:48.148 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:48.164+0000 7f6597400700 1 --2- 172.21.15.107:0/3012773386 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f65840a2910 0x7f65840a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:56:48.148 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:48.164+0000 7f6597400700 1 -- 172.21.15.107:0/3012773386 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f659800f020 con 0x7f65a0109310 2024-03-08T20:56:48.148 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:48.164+0000 7f659e800700 1 --2- 172.21.15.107:0/3012773386 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f65840a2910 0x7f65840a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:56:48.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:48.165+0000 7f659e800700 1 --2- 172.21.15.107:0/3012773386 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f65840a2910 0x7f65840a4dd0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f6590005ea0 tx=0x7f6590005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:56:48.152 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:48.168+0000 7f6597400700 1 -- 172.21.15.107:0/3012773386 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f6598029020 con 0x7f65a0109310 2024-03-08T20:56:48.374 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:48.390+0000 7f65a5400700 1 -- 172.21.15.107:0/3012773386 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f65a002d830 con 0x7f65a0109310 2024-03-08T20:56:48.377 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:48.393+0000 7f6597400700 1 -- 172.21.15.107:0/3012773386 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) v1 ==== 74+0+18537 (secure 0 0 0) 0x7f659809ce50 con 0x7f65a0109310 2024-03-08T20:56:48.378 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-08T20:56:48.379 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":82,"fsid":"dc20f0f8-dd8c-11ee-95c6-87774f69a715","created":"2024-03-08T20:47:04.190034+0000","modified":"2024-03-08T20:56:44.482997+0000","last_up_change":"2024-03-08T20:56:43.135514+0000","last_in_change":"2024-03-08T20:56:17.577074+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-08T20:51:32.832786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8b1ddc64-4d20-4e85-ac66-f1c5ac599501","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":2453946095},{"type":"v1","addr":"172.21.15.107:6803","nonce":2453946095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":2453946095},{"type":"v1","addr":"172.21.15.107:6805","nonce":2453946095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":2453946095},{"type":"v1","addr":"172.21.15.107:6809","nonce":2453946095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":2453946095},{"type":"v1","addr":"172.21.15.107:6807","nonce":2453946095}]},"public_addr":"172.21.15.107:6803/2453946095","cluster_addr":"172.21.15.107:6805/2453946095","heartbeat_back_addr":"172.21.15.107:6809/2453946095","heartbeat_front_addr":"172.21.15.107:6807/2453946095","state":["exists","up"]},{"osd":1,"uuid":"5a803c09-202a-4777-ba2a-1da8eaf26a54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":3071316419},{"type":"v1","addr":"172.21.15.149:6801","nonce":3071316419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":3071316419},{"type":"v1","addr":"172.21.15.149:6803","nonce":3071316419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":3071316419},{"type":"v1","addr":"172.21.15.149:6807","nonce":3071316419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":3071316419},{"type":"v1","addr":"172.21.15.149:6805","nonce":3071316419}]},"public_addr":"172.21.15.149:6801/3071316419","cluster_addr":"172.21.15.149:6803/3071316419","heartbeat_back_addr":"172.21.15.149:6807/3071316419","heartbeat_front_addr":"172.21.15.149:6805/3071316419","state":["exists","up"]},{"osd":2,"uuid":"555c634f-347e-47dd-a943-e50ae2be073c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":1087852630},{"type":"v1","addr":"172.21.15.204:6801","nonce":1087852630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":1087852630},{"type":"v1","addr":"172.21.15.204:6803","nonce":1087852630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":1087852630},{"type":"v1","addr":"172.21.15.204:6807","nonce":1087852630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":1087852630},{"type":"v1","addr":"172.21.15.204:6805","nonce":1087852630}]},"public_addr":"172.21.15.204:6801/1087852630","cluster_addr":"172.21.15.204:6803/1087852630","heartbeat_back_addr":"172.21.15.204:6807/1087852630","heartbeat_front_addr":"172.21.15.204:6805/1087852630","state":["exists","up"]},{"osd":3,"uuid":"9e433d97-e202-48e0-9ed1-f0988fa14307","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":3123579789},{"type":"v1","addr":"172.21.15.107:6811","nonce":3123579789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":3123579789},{"type":"v1","addr":"172.21.15.107:6813","nonce":3123579789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":3123579789},{"type":"v1","addr":"172.21.15.107:6817","nonce":3123579789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":3123579789},{"type":"v1","addr":"172.21.15.107:6815","nonce":3123579789}]},"public_addr":"172.21.15.107:6811/3123579789","cluster_addr":"172.21.15.107:6813/3123579789","heartbeat_back_addr":"172.21.15.107:6817/3123579789","heartbeat_front_addr":"172.21.15.107:6815/3123579789","state":["exists","up"]},{"osd":4,"uuid":"27c53794-9d2b-487d-bfcf-ac4fd2ab061e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":726286918},{"type":"v1","addr":"172.21.15.149:6809","nonce":726286918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":726286918},{"type":"v1","addr":"172.21.15.149:6811","nonce":726286918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":726286918},{"type":"v1","addr":"172.21.15.149:6815","nonce":726286918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":726286918},{"type":"v1","addr":"172.21.15.149:6813","nonce":726286918}]},"public_addr":"172.21.15.149:6809/726286918","cluster_addr":"172.21.15.149:6811/726286918","heartbeat_back_addr":"172.21.15.149:6815/726286918","heartbeat_front_addr":"172.21.15.149:6813/726286918","state":["exists","up"]},{"osd":5,"uuid":"3a7778fb-46c5-40c0-842e-e988c45ea24d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":3143887820},{"type":"v1","addr":"172.21.15.204:6809","nonce":3143887820}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":3143887820},{"type":"v1","addr":"172.21.15.204:6811","nonce":3143887820}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":3143887820},{"type":"v1","addr":"172.21.15.204:6815","nonce":3143887820}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":3143887820},{"type":"v1","addr":"172.21.15.204:6813","nonce":3143887820}]},"public_addr":"172.21.15.204:6809/3143887820","cluster_addr":"172.21.15.204:6811/3143887820","heartbeat_back_addr":"172.21.15.204:6815/3143887820","heartbeat_front_addr":"172.21.15.204:6813/3143887820","state":["exists","up"]},{"osd":6,"uuid":"d2e05a56-be4f-4435-9b76-ff9aa01e4ac3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":2912452950},{"type":"v1","addr":"172.21.15.107:6819","nonce":2912452950}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":2912452950},{"type":"v1","addr":"172.21.15.107:6821","nonce":2912452950}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":2912452950},{"type":"v1","addr":"172.21.15.107:6825","nonce":2912452950}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":2912452950},{"type":"v1","addr":"172.21.15.107:6823","nonce":2912452950}]},"public_addr":"172.21.15.107:6819/2912452950","cluster_addr":"172.21.15.107:6821/2912452950","heartbeat_back_addr":"172.21.15.107:6825/2912452950","heartbeat_front_addr":"172.21.15.107:6823/2912452950","state":["exists","up"]},{"osd":7,"uuid":"a7dd26ee-c118-4a9a-9e19-142c2680902d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":3835863809},{"type":"v1","addr":"172.21.15.149:6817","nonce":3835863809}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":3835863809},{"type":"v1","addr":"172.21.15.149:6819","nonce":3835863809}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":3835863809},{"type":"v1","addr":"172.21.15.149:6823","nonce":3835863809}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":3835863809},{"type":"v1","addr":"172.21.15.149:6821","nonce":3835863809}]},"public_addr":"172.21.15.149:6817/3835863809","cluster_addr":"172.21.15.149:6819/3835863809","heartbeat_back_addr":"172.21.15.149:6823/3835863809","heartbeat_front_addr":"172.21.15.149:6821/3835863809","state":["exists","up"]},{"osd":8,"uuid":"baaaa8dd-0e1f-4bae-86bf-9e429d985f23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":1010601477},{"type":"v1","addr":"172.21.15.204:6817","nonce":1010601477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":1010601477},{"type":"v1","addr":"172.21.15.204:6819","nonce":1010601477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":1010601477},{"type":"v1","addr":"172.21.15.204:6823","nonce":1010601477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":1010601477},{"type":"v1","addr":"172.21.15.204:6821","nonce":1010601477}]},"public_addr":"172.21.15.204:6817/1010601477","cluster_addr":"172.21.15.204:6819/1010601477","heartbeat_back_addr":"172.21.15.204:6823/1010601477","heartbeat_front_addr":"172.21.15.204:6821/1010601477","state":["exists","up"]},{"osd":9,"uuid":"8df80dfa-e7ff-410e-824c-b1dce111b02f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":3334759819},{"type":"v1","addr":"172.21.15.107:6827","nonce":3334759819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3334759819},{"type":"v1","addr":"172.21.15.107:6829","nonce":3334759819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":3334759819},{"type":"v1","addr":"172.21.15.107:6833","nonce":3334759819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":3334759819},{"type":"v1","addr":"172.21.15.107:6831","nonce":3334759819}]},"public_addr":"172.21.15.107:6827/3334759819","cluster_addr":"172.21.15.107:6829/3334759819","heartbeat_back_addr":"172.21.15.107:6833/3334759819","heartbeat_front_addr":"172.21.15.107:6831/3334759819","state":["exists","up"]},{"osd":10,"uuid":"338945b2-20c0-40b2-aaca-062cd4adee04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":3469284631},{"type":"v1","addr":"172.21.15.149:6825","nonce":3469284631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":3469284631},{"type":"v1","addr":"172.21.15.149:6827","nonce":3469284631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":3469284631},{"type":"v1","addr":"172.21.15.149:6831","nonce":3469284631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":3469284631},{"type":"v1","addr":"172.21.15.149:6829","nonce":3469284631}]},"public_addr":"172.21.15.149:6825/3469284631","cluster_addr":"172.21.15.149:6827/3469284631","heartbeat_back_addr":"172.21.15.149:6831/3469284631","heartbeat_front_addr":"172.21.15.149:6829/3469284631","state":["exists","up"]},{"osd":11,"uuid":"acccfe4f-d307-4a64-8f4d-fba27bfbe09d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":3097772473},{"type":"v1","addr":"172.21.15.204:6825","nonce":3097772473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":3097772473},{"type":"v1","addr":"172.21.15.204:6827","nonce":3097772473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":3097772473},{"type":"v1","addr":"172.21.15.204:6831","nonce":3097772473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":3097772473},{"type":"v1","addr":"172.21.15.204:6829","nonce":3097772473}]},"public_addr":"172.21.15.204:6825/3097772473","cluster_addr":"172.21.15.204:6827/3097772473","heartbeat_back_addr":"172.21.15.204:6831/3097772473","heartbeat_front_addr":"172.21.15.204:6829/3097772473","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T20:50:25.862753+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T20:50:58.167663+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T20:51:30.312816+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T20:52:05.487917+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T20:52:39.348354+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T20:53:12.899002+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T20:53:47.630837+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T20:54:21.444552+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T20:54:57.101160+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T20:55:31.841786+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T20:56:05.827161+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T20:56:40.611097+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.107:0/1514666962":"2024-03-09T20:48:20.149406+0000","172.21.15.107:0/1886016707":"2024-03-09T20:48:20.149406+0000","172.21.15.107:6801/426372446":"2024-03-09T20:48:20.149406+0000","172.21.15.107:6800/426372446":"2024-03-09T20:48:20.149406+0000","172.21.15.107:0/2392547434":"2024-03-09T20:47:45.706790+0000","172.21.15.107:0/1206995985":"2024-03-09T20:47:45.706790+0000","172.21.15.107:6800/3549936519":"2024-03-09T20:47:45.706790+0000","172.21.15.107:6801/3549936519":"2024-03-09T20:47:45.706790+0000","172.21.15.107:0/2482452215":"2024-03-09T20:47:45.706790+0000","172.21.15.107:0/808124991":"2024-03-09T20:48:20.149406+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-08T20:56:48.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:48.396+0000 7f65a5400700 1 -- 172.21.15.107:0/3012773386 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f65840a2910 msgr2=0x7f65840a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:56:48.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:48.396+0000 7f65a5400700 1 --2- 172.21.15.107:0/3012773386 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f65840a2910 0x7f65840a4dd0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f6590005ea0 tx=0x7f6590005e30 comp rx=0 tx=0).stop 2024-03-08T20:56:48.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:48.396+0000 7f65a5400700 1 -- 172.21.15.107:0/3012773386 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f65a0109310 msgr2=0x7f65a019ef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:56:48.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:48.396+0000 7f65a5400700 1 --2- 172.21.15.107:0/3012773386 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f65a0109310 0x7f65a019ef10 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f65980079a0 tx=0x7f6598007a80 comp rx=0 tx=0).stop 2024-03-08T20:56:48.381 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:48.396+0000 7f65a5400700 1 -- 172.21.15.107:0/3012773386 shutdown_connections 2024-03-08T20:56:48.381 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:48.396+0000 7f65a5400700 1 --2- 172.21.15.107:0/3012773386 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f65840a2910 0x7f65840a4dd0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:56:48.381 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:48.396+0000 7f65a5400700 1 --2- 172.21.15.107:0/3012773386 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f65a0109310 0x7f65a019ef10 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:56:48.381 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:48.396+0000 7f65a5400700 1 --2- 172.21.15.107:0/3012773386 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f65a0104230 0x7f65a019a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:56:48.381 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:48.396+0000 7f65a5400700 1 --2- 172.21.15.107:0/3012773386 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f65a0104be0 0x7f65a019abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:56:48.381 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:48.396+0000 7f65a5400700 1 -- 172.21.15.107:0/3012773386 >> 172.21.15.107:0/3012773386 conn(0x7f65a00ffac0 msgr2=0x7f65a0100570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:56:48.381 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:48.396+0000 7f65a5400700 1 -- 172.21.15.107:0/3012773386 shutdown_connections 2024-03-08T20:56:48.381 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:48.396+0000 7f65a5400700 1 -- 172.21.15.107:0/3012773386 wait complete. 2024-03-08T20:56:48.719 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:56:48 smithi107 ceph-mon[31381]: from='client.? 172.21.15.107:0/3012773386' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-08T20:56:48.766 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:56:48 smithi149 ceph-mon[35042]: from='client.? 172.21.15.107:0/3012773386' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-08T20:56:48.880 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:56:48 smithi204 ceph-mon[34935]: from='client.? 172.21.15.107:0/3012773386' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-08T20:56:49.165 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-08T20:51:32.832786+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-08T20:56:49.165 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761 shell --fsid dc20f0f8-dd8c-11ee-95c6-87774f69a715 -- ceph osd pool get .mgr pg_num 2024-03-08T20:56:49.603 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/mon.a/config 2024-03-08T20:56:49.719 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:56:49 smithi107 ceph-mon[31381]: pgmap v326: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-08T20:56:49.766 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:56:49 smithi149 ceph-mon[35042]: pgmap v326: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-08T20:56:49.879 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:56:49 smithi204 ceph-mon[34935]: pgmap v326: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-08T20:56:51.698 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:56:51 smithi107 ceph-mon[31381]: pgmap v327: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-08T20:56:51.766 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:56:51 smithi149 ceph-mon[35042]: pgmap v327: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-08T20:56:51.868 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:51.883+0000 7f9189c00700 1 -- 172.21.15.107:0/2364460601 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9184106ff0 msgr2=0x7f918410ebc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:56:51.868 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:51.883+0000 7f9189c00700 1 --2- 172.21.15.107:0/2364460601 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9184106ff0 0x7f918410ebc0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f917c0099c0 tx=0x7f917c009cd0 comp rx=0 tx=0).stop 2024-03-08T20:56:51.868 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:51.883+0000 7f9189c00700 1 -- 172.21.15.107:0/2364460601 shutdown_connections 2024-03-08T20:56:51.868 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:51.883+0000 7f9189c00700 1 --2- 172.21.15.107:0/2364460601 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9184101f10 0x7f91841022f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:56:51.868 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:51.883+0000 7f9189c00700 1 --2- 172.21.15.107:0/2364460601 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9184106ff0 0x7f918410ebc0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:56:51.868 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:51.883+0000 7f9189c00700 1 --2- 172.21.15.107:0/2364460601 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f91841028c0 0x7f9184106910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:56:51.868 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:51.883+0000 7f9189c00700 1 -- 172.21.15.107:0/2364460601 >> 172.21.15.107:0/2364460601 conn(0x7f91840fd800 msgr2=0x7f91840ffc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:56:51.868 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:51.883+0000 7f9189c00700 1 -- 172.21.15.107:0/2364460601 shutdown_connections 2024-03-08T20:56:51.868 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:51.883+0000 7f9189c00700 1 -- 172.21.15.107:0/2364460601 wait complete. 2024-03-08T20:56:51.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:51.884+0000 7f9189c00700 1 Processor -- start 2024-03-08T20:56:51.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:51.884+0000 7f9189c00700 1 -- start start 2024-03-08T20:56:51.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:51.885+0000 7f9189c00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9184101f10 0x7f91840740e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:56:51.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:51.885+0000 7f9189c00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f91841028c0 0x7f9184074620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:56:51.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:51.885+0000 7f9189c00700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9184106ff0 0x7f9184074b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:56:51.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:51.885+0000 7f9189c00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9184111f00 con 0x7f91841028c0 2024-03-08T20:56:51.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:51.885+0000 7f9189c00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9184111d80 con 0x7f9184101f10 2024-03-08T20:56:51.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:51.885+0000 7f9189c00700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9184112080 con 0x7f9184106ff0 2024-03-08T20:56:51.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:51.885+0000 7f917b400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f91841028c0 0x7f9184074620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:56:51.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:51.885+0000 7f917b400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f91841028c0 0x7f9184074620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:57390/0 (socket says 172.21.15.107:57390) 2024-03-08T20:56:51.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:51.885+0000 7f917b400700 1 -- 172.21.15.107:0/1985151850 learned_addr learned my addr 172.21.15.107:0/1985151850 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T20:56:51.871 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:51.885+0000 7f9183e00700 1 --2- 172.21.15.107:0/1985151850 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9184101f10 0x7f91840740e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:56:51.871 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:51.885+0000 7f9189200700 1 --2- 172.21.15.107:0/1985151850 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9184106ff0 0x7f9184074b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:56:51.871 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:51.886+0000 7f917b400700 1 -- 172.21.15.107:0/1985151850 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9184106ff0 msgr2=0x7f9184074b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:56:51.871 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:51.886+0000 7f917b400700 1 --2- 172.21.15.107:0/1985151850 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9184106ff0 0x7f9184074b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:56:51.871 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:51.886+0000 7f917b400700 1 -- 172.21.15.107:0/1985151850 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9184101f10 msgr2=0x7f91840740e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:56:51.871 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:51.886+0000 7f917b400700 1 --2- 172.21.15.107:0/1985151850 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9184101f10 0x7f91840740e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:56:51.871 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:51.886+0000 7f917b400700 1 -- 172.21.15.107:0/1985151850 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91841a6920 con 0x7f91841028c0 2024-03-08T20:56:51.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:51.886+0000 7f917b400700 1 --2- 172.21.15.107:0/1985151850 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f91841028c0 0x7f9184074620 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f916c00b500 tx=0x7f916c00b8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:56:51.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:51.886+0000 7f9181600700 1 -- 172.21.15.107:0/1985151850 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f916c015020 con 0x7f91841028c0 2024-03-08T20:56:51.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:51.886+0000 7f9181600700 1 -- 172.21.15.107:0/1985151850 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f916c011ce0 con 0x7f91841028c0 2024-03-08T20:56:51.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:51.886+0000 7f9189c00700 1 -- 172.21.15.107:0/1985151850 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91841a6c10 con 0x7f91841028c0 2024-03-08T20:56:51.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:51.886+0000 7f9189c00700 1 -- 172.21.15.107:0/1985151850 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91841a71a0 con 0x7f91841028c0 2024-03-08T20:56:51.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:51.887+0000 7f9181600700 1 -- 172.21.15.107:0/1985151850 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f916c010d40 con 0x7f91841028c0 2024-03-08T20:56:51.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:51.888+0000 7f9189c00700 1 -- 172.21.15.107:0/1985151850 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9160005330 con 0x7f91841028c0 2024-03-08T20:56:51.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:51.890+0000 7f9181600700 1 -- 172.21.15.107:0/1985151850 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f916c018020 con 0x7f91841028c0 2024-03-08T20:56:51.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:51.892+0000 7f9181600700 1 --2- 172.21.15.107:0/1985151850 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f91680a2910 0x7f91680a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:56:51.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:51.892+0000 7f9181600700 1 -- 172.21.15.107:0/1985151850 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f916c0ce260 con 0x7f91841028c0 2024-03-08T20:56:51.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:51.892+0000 7f9183e00700 1 --2- 172.21.15.107:0/1985151850 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f91680a2910 0x7f91680a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:56:51.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:51.893+0000 7f9183e00700 1 --2- 172.21.15.107:0/1985151850 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f91680a2910 0x7f91680a4dd0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f91740060b0 tx=0x7f9174007b80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:56:51.879 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:51.895+0000 7f9181600700 1 -- 172.21.15.107:0/1985151850 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f916c01f020 con 0x7f91841028c0 2024-03-08T20:56:51.879 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:56:51 smithi204 ceph-mon[34935]: pgmap v327: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-08T20:56:52.103 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:52.118+0000 7f9189c00700 1 -- 172.21.15.107:0/1985151850 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f91600058f0 con 0x7f91841028c0 2024-03-08T20:56:52.103 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:52.119+0000 7f9181600700 1 -- 172.21.15.107:0/1985151850 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) v1 ==== 93+0+10 (secure 0 0 0) 0x7f916c09b700 con 0x7f91841028c0 2024-03-08T20:56:52.104 INFO:teuthology.orchestra.run.smithi107.stdout:pg_num: 1 2024-03-08T20:56:52.106 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:52.122+0000 7f9189c00700 1 -- 172.21.15.107:0/1985151850 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f91680a2910 msgr2=0x7f91680a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:56:52.106 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:52.122+0000 7f9189c00700 1 --2- 172.21.15.107:0/1985151850 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f91680a2910 0x7f91680a4dd0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f91740060b0 tx=0x7f9174007b80 comp rx=0 tx=0).stop 2024-03-08T20:56:52.106 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:52.122+0000 7f9189c00700 1 -- 172.21.15.107:0/1985151850 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f91841028c0 msgr2=0x7f9184074620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:56:52.106 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:52.122+0000 7f9189c00700 1 --2- 172.21.15.107:0/1985151850 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f91841028c0 0x7f9184074620 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f916c00b500 tx=0x7f916c00b8c0 comp rx=0 tx=0).stop 2024-03-08T20:56:52.107 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:52.122+0000 7f9189c00700 1 -- 172.21.15.107:0/1985151850 shutdown_connections 2024-03-08T20:56:52.107 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:52.122+0000 7f9189c00700 1 --2- 172.21.15.107:0/1985151850 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9184106ff0 0x7f9184074b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:56:52.107 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:52.122+0000 7f9189c00700 1 --2- 172.21.15.107:0/1985151850 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f91680a2910 0x7f91680a4dd0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:56:52.107 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:52.122+0000 7f9189c00700 1 --2- 172.21.15.107:0/1985151850 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9184101f10 0x7f91840740e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:56:52.107 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:52.122+0000 7f9189c00700 1 --2- 172.21.15.107:0/1985151850 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f91841028c0 0x7f9184074620 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:56:52.107 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:52.122+0000 7f9189c00700 1 -- 172.21.15.107:0/1985151850 >> 172.21.15.107:0/1985151850 conn(0x7f91840fd800 msgr2=0x7f91840ffb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:56:52.107 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:52.123+0000 7f9189c00700 1 -- 172.21.15.107:0/1985151850 shutdown_connections 2024-03-08T20:56:52.107 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:52.123+0000 7f9189c00700 1 -- 172.21.15.107:0/1985151850 wait complete. 2024-03-08T20:56:52.766 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:56:52 smithi149 ceph-mon[35042]: from='client.? 172.21.15.107:0/1985151850' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-08T20:56:52.879 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:56:52 smithi204 ceph-mon[34935]: from='client.? 172.21.15.107:0/1985151850' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-08T20:56:52.960 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:56:52 smithi107 ceph-mon[31381]: from='client.? 172.21.15.107:0/1985151850' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-08T20:56:53.009 INFO:tasks.cephadm:Adding mds.a on smithi107 2024-03-08T20:56:53.009 INFO:tasks.cephadm:Adding mds.d on smithi107 2024-03-08T20:56:53.009 INFO:tasks.cephadm:Adding mds.g on smithi107 2024-03-08T20:56:53.009 INFO:tasks.cephadm:Adding mds.j on smithi107 2024-03-08T20:56:53.009 INFO:tasks.cephadm:Adding mds.b on smithi149 2024-03-08T20:56:53.009 INFO:tasks.cephadm:Adding mds.e on smithi149 2024-03-08T20:56:53.009 INFO:tasks.cephadm:Adding mds.h on smithi149 2024-03-08T20:56:53.009 INFO:tasks.cephadm:Adding mds.k on smithi149 2024-03-08T20:56:53.010 INFO:tasks.cephadm:Adding mds.c on smithi204 2024-03-08T20:56:53.010 INFO:tasks.cephadm:Adding mds.f on smithi204 2024-03-08T20:56:53.010 INFO:tasks.cephadm:Adding mds.i on smithi204 2024-03-08T20:56:53.010 INFO:tasks.cephadm:Adding mds.l on smithi204 2024-03-08T20:56:53.010 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dc20f0f8-dd8c-11ee-95c6-87774f69a715 -- ceph orch apply mds all '12;smithi107=a;smithi107=d;smithi107=g;smithi107=j;smithi149=b;smithi149=e;smithi149=h;smithi149=k;smithi204=c;smithi204=f;smithi204=i;smithi204=l' 2024-03-08T20:56:53.455 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/mon.c/config 2024-03-08T20:56:53.766 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:56:53 smithi149 ceph-mon[35042]: pgmap v328: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-08T20:56:53.775 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:56:53 smithi204 ceph-mon[34935]: pgmap v328: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-08T20:56:53.968 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:56:53 smithi107 ceph-mon[31381]: pgmap v328: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-08T20:56:55.563 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:56:55 smithi204 ceph-mon[34935]: pgmap v329: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-08T20:56:55.745 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:56:55 smithi107 ceph-mon[31381]: pgmap v329: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-08T20:56:55.766 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:56:55 smithi149 ceph-mon[35042]: pgmap v329: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-08T20:56:55.841 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:56:55.856+0000 7fba3d800700 1 -- 172.21.15.204:0/2410101422 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fba38103940 msgr2=0x7fba3810a260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:56:55.841 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:56:55.856+0000 7fba3d800700 1 --2- 172.21.15.204:0/2410101422 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fba38103940 0x7fba3810a260 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fba3000b0a0 tx=0x7fba3000b3b0 comp rx=0 tx=0).stop 2024-03-08T20:56:55.841 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:56:55.856+0000 7fba3d800700 1 -- 172.21.15.204:0/2410101422 shutdown_connections 2024-03-08T20:56:55.841 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:56:55.856+0000 7fba3d800700 1 --2- 172.21.15.204:0/2410101422 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fba38103940 0x7fba3810a260 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:56:55.841 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:56:55.856+0000 7fba3d800700 1 --2- 172.21.15.204:0/2410101422 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fba38101e20 0x7fba38102240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:56:55.841 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:56:55.856+0000 7fba3d800700 1 --2- 172.21.15.204:0/2410101422 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fba38102f80 0x7fba38103400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:56:55.841 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:56:55.856+0000 7fba3d800700 1 -- 172.21.15.204:0/2410101422 >> 172.21.15.204:0/2410101422 conn(0x7fba380fd380 msgr2=0x7fba380ff800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:56:55.842 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:56:55.856+0000 7fba3d800700 1 -- 172.21.15.204:0/2410101422 shutdown_connections 2024-03-08T20:56:55.842 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:56:55.856+0000 7fba3d800700 1 -- 172.21.15.204:0/2410101422 wait complete. 2024-03-08T20:56:55.843 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:56:55.857+0000 7fba3d800700 1 Processor -- start 2024-03-08T20:56:55.843 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:56:55.858+0000 7fba3d800700 1 -- start start 2024-03-08T20:56:55.843 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:56:55.858+0000 7fba3d800700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fba38101e20 0x7fba38199ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:56:55.843 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:56:55.858+0000 7fba3d800700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fba38102f80 0x7fba3819a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:56:55.843 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:56:55.858+0000 7fba3d800700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fba38103940 0x7fba381a14b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:56:55.843 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:56:55.858+0000 7fba3d800700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba3810d7e0 con 0x7fba38103940 2024-03-08T20:56:55.843 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:56:55.858+0000 7fba3d800700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba3810d660 con 0x7fba38102f80 2024-03-08T20:56:55.843 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:56:55.858+0000 7fba3d800700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba3810d960 con 0x7fba38101e20 2024-03-08T20:56:55.844 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:56:55.858+0000 7fba36a00700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fba38101e20 0x7fba38199ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:56:55.844 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:56:55.858+0000 7fba36a00700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fba38101e20 0x7fba38199ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.204:40754/0 (socket says 172.21.15.204:40754) 2024-03-08T20:56:55.844 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:56:55.858+0000 7fba36a00700 1 -- 172.21.15.204:0/1010598198 learned_addr learned my addr 172.21.15.204:0/1010598198 (peer_addr_for_me v2:172.21.15.204:0/0) 2024-03-08T20:56:55.844 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:56:55.859+0000 7fba37400700 1 --2- 172.21.15.204:0/1010598198 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fba38103940 0x7fba381a14b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:56:55.844 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:56:55.859+0000 7fba2e000700 1 --2- 172.21.15.204:0/1010598198 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fba38102f80 0x7fba3819a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:56:55.844 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:56:55.859+0000 7fba36a00700 1 -- 172.21.15.204:0/1010598198 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fba38102f80 msgr2=0x7fba3819a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:56:55.844 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:56:55.859+0000 7fba36a00700 1 --2- 172.21.15.204:0/1010598198 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fba38102f80 0x7fba3819a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:56:55.844 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:56:55.859+0000 7fba36a00700 1 -- 172.21.15.204:0/1010598198 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fba38103940 msgr2=0x7fba381a14b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:56:55.844 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:56:55.859+0000 7fba36a00700 1 --2- 172.21.15.204:0/1010598198 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fba38103940 0x7fba381a14b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:56:55.844 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:56:55.859+0000 7fba36a00700 1 -- 172.21.15.204:0/1010598198 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba381a1b50 con 0x7fba38101e20 2024-03-08T20:56:55.844 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:56:55.859+0000 7fba2e000700 1 --2- 172.21.15.204:0/1010598198 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fba38102f80 0x7fba3819a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T20:56:55.844 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:56:55.859+0000 7fba37400700 1 --2- 172.21.15.204:0/1010598198 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fba38103940 0x7fba381a14b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T20:56:55.845 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:56:55.859+0000 7fba36a00700 1 --2- 172.21.15.204:0/1010598198 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fba38101e20 0x7fba38199ef0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fba2800c9e0 tx=0x7fba2800cda0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:56:55.845 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:56:55.859+0000 7fba2fe00700 1 -- 172.21.15.204:0/1010598198 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba28007ac0 con 0x7fba38101e20 2024-03-08T20:56:55.845 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:56:55.859+0000 7fba2fe00700 1 -- 172.21.15.204:0/1010598198 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fba28007c60 con 0x7fba38101e20 2024-03-08T20:56:55.845 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:56:55.859+0000 7fba3d800700 1 -- 172.21.15.204:0/1010598198 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba381a1e40 con 0x7fba38101e20 2024-03-08T20:56:55.845 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:56:55.859+0000 7fba2fe00700 1 -- 172.21.15.204:0/1010598198 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba28018660 con 0x7fba38101e20 2024-03-08T20:56:55.846 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:56:55.859+0000 7fba3d800700 1 -- 172.21.15.204:0/1010598198 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba381a22a0 con 0x7fba38101e20 2024-03-08T20:56:55.846 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:56:55.860+0000 7fba3d800700 1 -- 172.21.15.204:0/1010598198 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba38068ad0 con 0x7fba38101e20 2024-03-08T20:56:55.846 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:56:55.862+0000 7fba2fe00700 1 -- 172.21.15.204:0/1010598198 <== mon.1 v2:172.21.15.204:3300/0 4 ==== mgrmap(e 16) v1 ==== 137291+0+0 (secure 0 0 0) 0x7fba2800f450 con 0x7fba38101e20 2024-03-08T20:56:55.848 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:56:55.864+0000 7fba2fe00700 1 --2- 172.21.15.204:0/1010598198 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fba1c0a2910 0x7fba1c0a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:56:55.848 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:56:55.864+0000 7fba2fe00700 1 -- 172.21.15.204:0/1010598198 <== mon.1 v2:172.21.15.204:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fba280ce540 con 0x7fba38101e20 2024-03-08T20:56:55.848 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:56:55.864+0000 7fba2e000700 1 --2- 172.21.15.204:0/1010598198 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fba1c0a2910 0x7fba1c0a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:56:55.849 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:56:55.865+0000 7fba2e000700 1 --2- 172.21.15.204:0/1010598198 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fba1c0a2910 0x7fba1c0a4dd0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fba20005fd0 tx=0x7fba20005e20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:56:55.853 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:56:55.868+0000 7fba2fe00700 1 -- 172.21.15.204:0/1010598198 <== mon.1 v2:172.21.15.204:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7fba38068ad0 con 0x7fba38101e20 2024-03-08T20:56:56.087 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:56:56.102+0000 7fba3d800700 1 -- 172.21.15.204:0/1010598198 --> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi107=a;smithi107=d;smithi107=g;smithi107=j;smithi149=b;smithi149=e;smithi149=h;smithi149=k;smithi204=c;smithi204=f;smithi204=i;smithi204=l", "target": ["mon-mgr", ""]}) v1 -- 0x7fba38075830 con 0x7fba1c0a2910 2024-03-08T20:56:56.104 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:56:56.119+0000 7fba2fe00700 1 -- 172.21.15.204:0/1010598198 <== mgr.14152 v2:172.21.15.107:6800/3376636618 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+28 (secure 0 0 0) 0x7fba38075830 con 0x7fba1c0a2910 2024-03-08T20:56:56.104 INFO:teuthology.orchestra.run.smithi204.stdout:Scheduled mds.all update... 2024-03-08T20:56:56.107 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:56:56.122+0000 7fba3d800700 1 -- 172.21.15.204:0/1010598198 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fba1c0a2910 msgr2=0x7fba1c0a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:56:56.107 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:56:56.122+0000 7fba3d800700 1 --2- 172.21.15.204:0/1010598198 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fba1c0a2910 0x7fba1c0a4dd0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fba20005fd0 tx=0x7fba20005e20 comp rx=0 tx=0).stop 2024-03-08T20:56:56.107 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:56:56.122+0000 7fba3d800700 1 -- 172.21.15.204:0/1010598198 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fba38101e20 msgr2=0x7fba38199ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:56:56.107 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:56:56.122+0000 7fba3d800700 1 --2- 172.21.15.204:0/1010598198 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fba38101e20 0x7fba38199ef0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fba2800c9e0 tx=0x7fba2800cda0 comp rx=0 tx=0).stop 2024-03-08T20:56:56.107 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:56:56.122+0000 7fba3d800700 1 -- 172.21.15.204:0/1010598198 shutdown_connections 2024-03-08T20:56:56.107 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:56:56.122+0000 7fba3d800700 1 --2- 172.21.15.204:0/1010598198 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fba38101e20 0x7fba38199ef0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:56:56.107 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:56:56.122+0000 7fba3d800700 1 --2- 172.21.15.204:0/1010598198 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fba1c0a2910 0x7fba1c0a4dd0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:56:56.107 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:56:56.122+0000 7fba3d800700 1 --2- 172.21.15.204:0/1010598198 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fba38103940 0x7fba381a14b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:56:56.107 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:56:56.122+0000 7fba3d800700 1 --2- 172.21.15.204:0/1010598198 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fba38102f80 0x7fba3819a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:56:56.107 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:56:56.122+0000 7fba3d800700 1 -- 172.21.15.204:0/1010598198 >> 172.21.15.204:0/1010598198 conn(0x7fba380fd380 msgr2=0x7fba38074010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:56:56.107 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:56:56.122+0000 7fba3d800700 1 -- 172.21.15.204:0/1010598198 shutdown_connections 2024-03-08T20:56:56.107 INFO:teuthology.orchestra.run.smithi204.stderr:2024-03-08T20:56:56.122+0000 7fba3d800700 1 -- 172.21.15.204:0/1010598198 wait complete. 2024-03-08T20:56:56.911 DEBUG:teuthology.orchestra.run.smithi107:mds.a> sudo journalctl -f -n 0 -u ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715@mds.a.service 2024-03-08T20:56:56.919 DEBUG:teuthology.orchestra.run.smithi107:mds.d> sudo journalctl -f -n 0 -u ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715@mds.d.service 2024-03-08T20:56:56.965 DEBUG:teuthology.orchestra.run.smithi107:mds.g> sudo journalctl -f -n 0 -u ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715@mds.g.service 2024-03-08T20:56:57.014 DEBUG:teuthology.orchestra.run.smithi107:mds.j> sudo journalctl -f -n 0 -u ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715@mds.j.service 2024-03-08T20:56:57.061 DEBUG:teuthology.orchestra.run.smithi149:mds.b> sudo journalctl -f -n 0 -u ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715@mds.b.service 2024-03-08T20:56:57.069 DEBUG:teuthology.orchestra.run.smithi149:mds.e> sudo journalctl -f -n 0 -u ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715@mds.e.service 2024-03-08T20:56:57.116 DEBUG:teuthology.orchestra.run.smithi149:mds.h> sudo journalctl -f -n 0 -u ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715@mds.h.service 2024-03-08T20:56:57.130 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:56:57 smithi204 ceph-mon[34935]: from='client.24385 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi107=a;smithi107=d;smithi107=g;smithi107=j;smithi149=b;smithi149=e;smithi149=h;smithi149=k;smithi204=c;smithi204=f;smithi204=i;smithi204=l", "target": ["mon-mgr", ""]}]: dispatch 2024-03-08T20:56:57.130 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:56:57 smithi204 ceph-mon[34935]: Saving service mds.all spec with placement smithi107=a;smithi107=d;smithi107=g;smithi107=j;smithi149=b;smithi149=e;smithi149=h;smithi149=k;smithi204=c;smithi204=f;smithi204=i;smithi204=l;count:12 2024-03-08T20:56:57.130 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:56:57 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:56:57.130 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:56:57 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T20:56:57.130 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:56:57 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:56:57.130 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:56:57 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T20:56:57.130 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:56:57 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:56:57.130 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:56:57 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-08T20:56:57.130 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:56:57 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-08T20:56:57.131 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:56:57 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:56:57.131 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:56:57 smithi204 ceph-mon[34935]: Deploying daemon mds.l on smithi204 2024-03-08T20:56:57.131 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:56:57 smithi204 ceph-mon[34935]: pgmap v330: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-08T20:56:57.162 DEBUG:teuthology.orchestra.run.smithi149:mds.k> sudo journalctl -f -n 0 -u ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715@mds.k.service 2024-03-08T20:56:57.209 DEBUG:teuthology.orchestra.run.smithi204:mds.c> sudo journalctl -f -n 0 -u ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715@mds.c.service 2024-03-08T20:56:57.217 DEBUG:teuthology.orchestra.run.smithi204:mds.f> sudo journalctl -f -n 0 -u ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715@mds.f.service 2024-03-08T20:56:57.264 DEBUG:teuthology.orchestra.run.smithi204:mds.i> sudo journalctl -f -n 0 -u ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715@mds.i.service 2024-03-08T20:56:57.309 DEBUG:teuthology.orchestra.run.smithi204:mds.l> sudo journalctl -f -n 0 -u ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715@mds.l.service 2024-03-08T20:56:57.355 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-03-08T20:56:57.355 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-08T20:56:57.442 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:56:57 smithi107 ceph-mon[31381]: from='client.24385 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi107=a;smithi107=d;smithi107=g;smithi107=j;smithi149=b;smithi149=e;smithi149=h;smithi149=k;smithi204=c;smithi204=f;smithi204=i;smithi204=l", "target": ["mon-mgr", ""]}]: dispatch 2024-03-08T20:56:57.443 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:56:57 smithi107 ceph-mon[31381]: Saving service mds.all spec with placement smithi107=a;smithi107=d;smithi107=g;smithi107=j;smithi149=b;smithi149=e;smithi149=h;smithi149=k;smithi204=c;smithi204=f;smithi204=i;smithi204=l;count:12 2024-03-08T20:56:57.443 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:56:57 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:56:57.443 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:56:57 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T20:56:57.443 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:56:57 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:56:57.443 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:56:57 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T20:56:57.443 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:56:57 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:56:57.443 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:56:57 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-08T20:56:57.443 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:56:57 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-08T20:56:57.443 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:56:57 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:56:57.443 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:56:57 smithi107 ceph-mon[31381]: Deploying daemon mds.l on smithi204 2024-03-08T20:56:57.443 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:56:57 smithi107 ceph-mon[31381]: pgmap v330: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-08T20:56:57.517 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:56:57 smithi149 ceph-mon[35042]: from='client.24385 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi107=a;smithi107=d;smithi107=g;smithi107=j;smithi149=b;smithi149=e;smithi149=h;smithi149=k;smithi204=c;smithi204=f;smithi204=i;smithi204=l", "target": ["mon-mgr", ""]}]: dispatch 2024-03-08T20:56:57.517 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:56:57 smithi149 ceph-mon[35042]: Saving service mds.all spec with placement smithi107=a;smithi107=d;smithi107=g;smithi107=j;smithi149=b;smithi149=e;smithi149=h;smithi149=k;smithi204=c;smithi204=f;smithi204=i;smithi204=l;count:12 2024-03-08T20:56:57.517 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:56:57 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:56:57.517 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:56:57 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T20:56:57.517 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:56:57 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:56:57.517 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:56:57 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T20:56:57.517 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:56:57 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:56:57.517 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:56:57 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-08T20:56:57.518 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:56:57 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-08T20:56:57.518 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:56:57 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:56:57.518 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:56:57 smithi149 ceph-mon[35042]: Deploying daemon mds.l on smithi204 2024-03-08T20:56:57.518 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:56:57 smithi149 ceph-mon[35042]: pgmap v330: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-08T20:56:57.603 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:57.618+0000 7f614b000640 1 -- 172.21.15.107:0/2527152532 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f61441044f0 msgr2=0x7f61441068e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:56:57.603 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:57.618+0000 7f614b000640 1 --2- 172.21.15.107:0/2527152532 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f61441044f0 0x7f61441068e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f6130009900 tx=0x7f613002bc30 comp rx=0 tx=0).stop 2024-03-08T20:56:57.603 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:57.618+0000 7f614b000640 1 -- 172.21.15.107:0/2527152532 shutdown_connections 2024-03-08T20:56:57.603 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:57.618+0000 7f614b000640 1 --2- 172.21.15.107:0/2527152532 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6144106e20 0x7f6144109270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:56:57.603 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:57.618+0000 7f614b000640 1 --2- 172.21.15.107:0/2527152532 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f61441044f0 0x7f61441068e0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:56:57.603 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:57.618+0000 7f614b000640 1 --2- 172.21.15.107:0/2527152532 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6144101bc0 0x7f6144103fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:56:57.603 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:57.619+0000 7f614b000640 1 -- 172.21.15.107:0/2527152532 >> 172.21.15.107:0/2527152532 conn(0x7f61440fb5d0 msgr2=0x7f61440fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:56:57.604 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:57.619+0000 7f614b000640 1 -- 172.21.15.107:0/2527152532 shutdown_connections 2024-03-08T20:56:57.604 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:57.619+0000 7f614b000640 1 -- 172.21.15.107:0/2527152532 wait complete. 2024-03-08T20:56:57.606 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:57.620+0000 7f614b000640 1 Processor -- start 2024-03-08T20:56:57.606 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:57.620+0000 7f614b000640 1 -- start start 2024-03-08T20:56:57.606 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:57.620+0000 7f614b000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6144101bc0 0x7f61441995e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:56:57.606 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:57.620+0000 7f614b000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f61441044f0 0x7f6144199b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:56:57.606 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:57.620+0000 7f614b000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6144106e20 0x7f61441a0b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:56:57.606 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:57.620+0000 7f614b000640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f614410ccc0 con 0x7f6144106e20 2024-03-08T20:56:57.607 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:57.620+0000 7f614b000640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f614410cb40 con 0x7f61441044f0 2024-03-08T20:56:57.607 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:57.620+0000 7f614b000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f614410ce40 con 0x7f6144101bc0 2024-03-08T20:56:57.609 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:57.620+0000 7f614a600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6144106e20 0x7f61441a0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:56:57.609 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:57.621+0000 7f6149c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6144101bc0 0x7f61441995e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:56:57.609 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:57.621+0000 7f614a600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6144106e20 0x7f61441a0b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:56134/0 (socket says 172.21.15.107:56134) 2024-03-08T20:56:57.609 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:57.621+0000 7f614a600640 1 -- 172.21.15.107:0/3434641341 learned_addr learned my addr 172.21.15.107:0/3434641341 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T20:56:57.609 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:57.621+0000 7f6149200640 1 --2- 172.21.15.107:0/3434641341 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f61441044f0 0x7f6144199b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:56:57.609 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:57.621+0000 7f614a600640 1 -- 172.21.15.107:0/3434641341 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6144101bc0 msgr2=0x7f61441995e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:56:57.610 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:57.621+0000 7f614a600640 1 --2- 172.21.15.107:0/3434641341 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6144101bc0 0x7f61441995e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:56:57.610 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:57.621+0000 7f614a600640 1 -- 172.21.15.107:0/3434641341 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f61441044f0 msgr2=0x7f6144199b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:56:57.610 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:57.621+0000 7f614a600640 1 --2- 172.21.15.107:0/3434641341 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f61441044f0 0x7f6144199b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:56:57.610 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:57.621+0000 7f614a600640 1 -- 172.21.15.107:0/3434641341 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61441a1200 con 0x7f6144106e20 2024-03-08T20:56:57.610 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:57.621+0000 7f614a600640 1 --2- 172.21.15.107:0/3434641341 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6144106e20 0x7f61441a0b90 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f613c009fd0 tx=0x7f613c00b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:56:57.610 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:57.622+0000 7f6142000640 1 -- 172.21.15.107:0/3434641341 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f613c011830 con 0x7f6144106e20 2024-03-08T20:56:57.610 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:57.622+0000 7f614b000640 1 -- 172.21.15.107:0/3434641341 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61441a14f0 con 0x7f6144106e20 2024-03-08T20:56:57.610 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:57.622+0000 7f6142000640 1 -- 172.21.15.107:0/3434641341 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f613c016070 con 0x7f6144106e20 2024-03-08T20:56:57.610 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:57.622+0000 7f6142000640 1 -- 172.21.15.107:0/3434641341 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f613c010de0 con 0x7f6144106e20 2024-03-08T20:56:57.610 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:57.622+0000 7f614b000640 1 -- 172.21.15.107:0/3434641341 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61441a1a90 con 0x7f6144106e20 2024-03-08T20:56:57.611 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:57.623+0000 7f614b000640 1 -- 172.21.15.107:0/3434641341 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6144108910 con 0x7f6144106e20 2024-03-08T20:56:57.612 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:57.625+0000 7f6142000640 1 -- 172.21.15.107:0/3434641341 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f613c0119d0 con 0x7f6144106e20 2024-03-08T20:56:57.612 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:57.627+0000 7f6142000640 1 --2- 172.21.15.107:0/3434641341 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f612c0a2880 0x7f612c0a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:56:57.612 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:57.627+0000 7f6142000640 1 -- 172.21.15.107:0/3434641341 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f613c0cdc40 con 0x7f6144106e20 2024-03-08T20:56:57.612 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:57.627+0000 7f6149c00640 1 --2- 172.21.15.107:0/3434641341 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f612c0a2880 0x7f612c0a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:56:57.612 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:57.627+0000 7f6149c00640 1 --2- 172.21.15.107:0/3434641341 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f612c0a2880 0x7f612c0a4d40 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f6138005c40 tx=0x7f613800a590 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:56:57.616 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:57.633+0000 7f6142000640 1 -- 172.21.15.107:0/3434641341 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f613c09b0e0 con 0x7f6144106e20 2024-03-08T20:56:57.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:57.831+0000 7f614b000640 1 -- 172.21.15.107:0/3434641341 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6144061ef0 con 0x7f6144106e20 2024-03-08T20:56:57.818 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:57.833+0000 7f6142000640 1 -- 172.21.15.107:0/3434641341 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) v1 ==== 74+0+18537 (secure 0 0 0) 0x7f613c09a9e0 con 0x7f6144106e20 2024-03-08T20:56:57.818 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-08T20:56:57.818 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":82,"fsid":"dc20f0f8-dd8c-11ee-95c6-87774f69a715","created":"2024-03-08T20:47:04.190034+0000","modified":"2024-03-08T20:56:44.482997+0000","last_up_change":"2024-03-08T20:56:43.135514+0000","last_in_change":"2024-03-08T20:56:17.577074+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-08T20:51:32.832786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8b1ddc64-4d20-4e85-ac66-f1c5ac599501","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":2453946095},{"type":"v1","addr":"172.21.15.107:6803","nonce":2453946095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":2453946095},{"type":"v1","addr":"172.21.15.107:6805","nonce":2453946095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":2453946095},{"type":"v1","addr":"172.21.15.107:6809","nonce":2453946095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":2453946095},{"type":"v1","addr":"172.21.15.107:6807","nonce":2453946095}]},"public_addr":"172.21.15.107:6803/2453946095","cluster_addr":"172.21.15.107:6805/2453946095","heartbeat_back_addr":"172.21.15.107:6809/2453946095","heartbeat_front_addr":"172.21.15.107:6807/2453946095","state":["exists","up"]},{"osd":1,"uuid":"5a803c09-202a-4777-ba2a-1da8eaf26a54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":3071316419},{"type":"v1","addr":"172.21.15.149:6801","nonce":3071316419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":3071316419},{"type":"v1","addr":"172.21.15.149:6803","nonce":3071316419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":3071316419},{"type":"v1","addr":"172.21.15.149:6807","nonce":3071316419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":3071316419},{"type":"v1","addr":"172.21.15.149:6805","nonce":3071316419}]},"public_addr":"172.21.15.149:6801/3071316419","cluster_addr":"172.21.15.149:6803/3071316419","heartbeat_back_addr":"172.21.15.149:6807/3071316419","heartbeat_front_addr":"172.21.15.149:6805/3071316419","state":["exists","up"]},{"osd":2,"uuid":"555c634f-347e-47dd-a943-e50ae2be073c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":1087852630},{"type":"v1","addr":"172.21.15.204:6801","nonce":1087852630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":1087852630},{"type":"v1","addr":"172.21.15.204:6803","nonce":1087852630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":1087852630},{"type":"v1","addr":"172.21.15.204:6807","nonce":1087852630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":1087852630},{"type":"v1","addr":"172.21.15.204:6805","nonce":1087852630}]},"public_addr":"172.21.15.204:6801/1087852630","cluster_addr":"172.21.15.204:6803/1087852630","heartbeat_back_addr":"172.21.15.204:6807/1087852630","heartbeat_front_addr":"172.21.15.204:6805/1087852630","state":["exists","up"]},{"osd":3,"uuid":"9e433d97-e202-48e0-9ed1-f0988fa14307","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":3123579789},{"type":"v1","addr":"172.21.15.107:6811","nonce":3123579789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":3123579789},{"type":"v1","addr":"172.21.15.107:6813","nonce":3123579789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":3123579789},{"type":"v1","addr":"172.21.15.107:6817","nonce":3123579789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":3123579789},{"type":"v1","addr":"172.21.15.107:6815","nonce":3123579789}]},"public_addr":"172.21.15.107:6811/3123579789","cluster_addr":"172.21.15.107:6813/3123579789","heartbeat_back_addr":"172.21.15.107:6817/3123579789","heartbeat_front_addr":"172.21.15.107:6815/3123579789","state":["exists","up"]},{"osd":4,"uuid":"27c53794-9d2b-487d-bfcf-ac4fd2ab061e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":726286918},{"type":"v1","addr":"172.21.15.149:6809","nonce":726286918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":726286918},{"type":"v1","addr":"172.21.15.149:6811","nonce":726286918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":726286918},{"type":"v1","addr":"172.21.15.149:6815","nonce":726286918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":726286918},{"type":"v1","addr":"172.21.15.149:6813","nonce":726286918}]},"public_addr":"172.21.15.149:6809/726286918","cluster_addr":"172.21.15.149:6811/726286918","heartbeat_back_addr":"172.21.15.149:6815/726286918","heartbeat_front_addr":"172.21.15.149:6813/726286918","state":["exists","up"]},{"osd":5,"uuid":"3a7778fb-46c5-40c0-842e-e988c45ea24d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":3143887820},{"type":"v1","addr":"172.21.15.204:6809","nonce":3143887820}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":3143887820},{"type":"v1","addr":"172.21.15.204:6811","nonce":3143887820}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":3143887820},{"type":"v1","addr":"172.21.15.204:6815","nonce":3143887820}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":3143887820},{"type":"v1","addr":"172.21.15.204:6813","nonce":3143887820}]},"public_addr":"172.21.15.204:6809/3143887820","cluster_addr":"172.21.15.204:6811/3143887820","heartbeat_back_addr":"172.21.15.204:6815/3143887820","heartbeat_front_addr":"172.21.15.204:6813/3143887820","state":["exists","up"]},{"osd":6,"uuid":"d2e05a56-be4f-4435-9b76-ff9aa01e4ac3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":2912452950},{"type":"v1","addr":"172.21.15.107:6819","nonce":2912452950}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":2912452950},{"type":"v1","addr":"172.21.15.107:6821","nonce":2912452950}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":2912452950},{"type":"v1","addr":"172.21.15.107:6825","nonce":2912452950}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":2912452950},{"type":"v1","addr":"172.21.15.107:6823","nonce":2912452950}]},"public_addr":"172.21.15.107:6819/2912452950","cluster_addr":"172.21.15.107:6821/2912452950","heartbeat_back_addr":"172.21.15.107:6825/2912452950","heartbeat_front_addr":"172.21.15.107:6823/2912452950","state":["exists","up"]},{"osd":7,"uuid":"a7dd26ee-c118-4a9a-9e19-142c2680902d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":3835863809},{"type":"v1","addr":"172.21.15.149:6817","nonce":3835863809}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":3835863809},{"type":"v1","addr":"172.21.15.149:6819","nonce":3835863809}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":3835863809},{"type":"v1","addr":"172.21.15.149:6823","nonce":3835863809}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":3835863809},{"type":"v1","addr":"172.21.15.149:6821","nonce":3835863809}]},"public_addr":"172.21.15.149:6817/3835863809","cluster_addr":"172.21.15.149:6819/3835863809","heartbeat_back_addr":"172.21.15.149:6823/3835863809","heartbeat_front_addr":"172.21.15.149:6821/3835863809","state":["exists","up"]},{"osd":8,"uuid":"baaaa8dd-0e1f-4bae-86bf-9e429d985f23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":1010601477},{"type":"v1","addr":"172.21.15.204:6817","nonce":1010601477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":1010601477},{"type":"v1","addr":"172.21.15.204:6819","nonce":1010601477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":1010601477},{"type":"v1","addr":"172.21.15.204:6823","nonce":1010601477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":1010601477},{"type":"v1","addr":"172.21.15.204:6821","nonce":1010601477}]},"public_addr":"172.21.15.204:6817/1010601477","cluster_addr":"172.21.15.204:6819/1010601477","heartbeat_back_addr":"172.21.15.204:6823/1010601477","heartbeat_front_addr":"172.21.15.204:6821/1010601477","state":["exists","up"]},{"osd":9,"uuid":"8df80dfa-e7ff-410e-824c-b1dce111b02f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":3334759819},{"type":"v1","addr":"172.21.15.107:6827","nonce":3334759819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3334759819},{"type":"v1","addr":"172.21.15.107:6829","nonce":3334759819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":3334759819},{"type":"v1","addr":"172.21.15.107:6833","nonce":3334759819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":3334759819},{"type":"v1","addr":"172.21.15.107:6831","nonce":3334759819}]},"public_addr":"172.21.15.107:6827/3334759819","cluster_addr":"172.21.15.107:6829/3334759819","heartbeat_back_addr":"172.21.15.107:6833/3334759819","heartbeat_front_addr":"172.21.15.107:6831/3334759819","state":["exists","up"]},{"osd":10,"uuid":"338945b2-20c0-40b2-aaca-062cd4adee04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":3469284631},{"type":"v1","addr":"172.21.15.149:6825","nonce":3469284631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":3469284631},{"type":"v1","addr":"172.21.15.149:6827","nonce":3469284631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":3469284631},{"type":"v1","addr":"172.21.15.149:6831","nonce":3469284631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":3469284631},{"type":"v1","addr":"172.21.15.149:6829","nonce":3469284631}]},"public_addr":"172.21.15.149:6825/3469284631","cluster_addr":"172.21.15.149:6827/3469284631","heartbeat_back_addr":"172.21.15.149:6831/3469284631","heartbeat_front_addr":"172.21.15.149:6829/3469284631","state":["exists","up"]},{"osd":11,"uuid":"acccfe4f-d307-4a64-8f4d-fba27bfbe09d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":3097772473},{"type":"v1","addr":"172.21.15.204:6825","nonce":3097772473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":3097772473},{"type":"v1","addr":"172.21.15.204:6827","nonce":3097772473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":3097772473},{"type":"v1","addr":"172.21.15.204:6831","nonce":3097772473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":3097772473},{"type":"v1","addr":"172.21.15.204:6829","nonce":3097772473}]},"public_addr":"172.21.15.204:6825/3097772473","cluster_addr":"172.21.15.204:6827/3097772473","heartbeat_back_addr":"172.21.15.204:6831/3097772473","heartbeat_front_addr":"172.21.15.204:6829/3097772473","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T20:50:25.862753+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T20:50:58.167663+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T20:51:30.312816+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T20:52:05.487917+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T20:52:39.348354+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T20:53:12.899002+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T20:53:47.630837+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T20:54:21.444552+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T20:54:57.101160+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T20:55:31.841786+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T20:56:05.827161+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T20:56:40.611097+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.107:0/1514666962":"2024-03-09T20:48:20.149406+0000","172.21.15.107:0/1886016707":"2024-03-09T20:48:20.149406+0000","172.21.15.107:6801/426372446":"2024-03-09T20:48:20.149406+0000","172.21.15.107:6800/426372446":"2024-03-09T20:48:20.149406+0000","172.21.15.107:0/2392547434":"2024-03-09T20:47:45.706790+0000","172.21.15.107:0/1206995985":"2024-03-09T20:47:45.706790+0000","172.21.15.107:6800/3549936519":"2024-03-09T20:47:45.706790+0000","172.21.15.107:6801/3549936519":"2024-03-09T20:47:45.706790+0000","172.21.15.107:0/2482452215":"2024-03-09T20:47:45.706790+0000","172.21.15.107:0/808124991":"2024-03-09T20:48:20.149406+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-08T20:56:57.820 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:57.836+0000 7f614b000640 1 -- 172.21.15.107:0/3434641341 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f612c0a2880 msgr2=0x7f612c0a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:56:57.821 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:57.836+0000 7f614b000640 1 --2- 172.21.15.107:0/3434641341 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f612c0a2880 0x7f612c0a4d40 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f6138005c40 tx=0x7f613800a590 comp rx=0 tx=0).stop 2024-03-08T20:56:57.821 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:57.836+0000 7f614b000640 1 -- 172.21.15.107:0/3434641341 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6144106e20 msgr2=0x7f61441a0b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:56:57.821 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:57.836+0000 7f614b000640 1 --2- 172.21.15.107:0/3434641341 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6144106e20 0x7f61441a0b90 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f613c009fd0 tx=0x7f613c00b8e0 comp rx=0 tx=0).stop 2024-03-08T20:56:57.821 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:57.836+0000 7f614b000640 1 -- 172.21.15.107:0/3434641341 shutdown_connections 2024-03-08T20:56:57.821 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:57.836+0000 7f614b000640 1 --2- 172.21.15.107:0/3434641341 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f612c0a2880 0x7f612c0a4d40 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:56:57.821 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:57.836+0000 7f614b000640 1 --2- 172.21.15.107:0/3434641341 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6144106e20 0x7f61441a0b90 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:56:57.821 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:57.836+0000 7f614b000640 1 --2- 172.21.15.107:0/3434641341 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f61441044f0 0x7f6144199b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:56:57.821 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:57.836+0000 7f614b000640 1 --2- 172.21.15.107:0/3434641341 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6144101bc0 0x7f61441995e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:56:57.821 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:57.836+0000 7f614b000640 1 -- 172.21.15.107:0/3434641341 >> 172.21.15.107:0/3434641341 conn(0x7f61440fb5d0 msgr2=0x7f6144101270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:56:57.821 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:57.836+0000 7f614b000640 1 -- 172.21.15.107:0/3434641341 shutdown_connections 2024-03-08T20:56:57.821 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:57.836+0000 7f614b000640 1 -- 172.21.15.107:0/3434641341 wait complete. 2024-03-08T20:56:57.841 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-08T20:51:32.832786+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-08T20:56:57.842 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-08T20:56:58.061 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:58.077+0000 7fed62c00640 1 -- 172.21.15.107:0/415677574 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fed5c073d50 msgr2=0x7fed5c0741d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:56:58.061 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:58.077+0000 7fed62c00640 1 --2- 172.21.15.107:0/415677574 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fed5c073d50 0x7fed5c0741d0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fed44009930 tx=0x7fed4402bc30 comp rx=0 tx=0).stop 2024-03-08T20:56:58.061 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:58.077+0000 7fed62c00640 1 -- 172.21.15.107:0/415677574 shutdown_connections 2024-03-08T20:56:58.061 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:58.077+0000 7fed62c00640 1 --2- 172.21.15.107:0/415677574 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fed5c10d8d0 0x7fed5c10fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:56:58.061 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:58.077+0000 7fed62c00640 1 --2- 172.21.15.107:0/415677574 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fed5c073d50 0x7fed5c0741d0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:56:58.061 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:58.077+0000 7fed62c00640 1 --2- 172.21.15.107:0/415677574 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fed5c075330 0x7fed5c073780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:56:58.061 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:58.077+0000 7fed62c00640 1 -- 172.21.15.107:0/415677574 >> 172.21.15.107:0/415677574 conn(0x7fed5c0fc890 msgr2=0x7fed5c0fecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:56:58.062 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:58.077+0000 7fed62c00640 1 -- 172.21.15.107:0/415677574 shutdown_connections 2024-03-08T20:56:58.062 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:58.077+0000 7fed62c00640 1 -- 172.21.15.107:0/415677574 wait complete. 2024-03-08T20:56:58.064 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:58.078+0000 7fed62c00640 1 Processor -- start 2024-03-08T20:56:58.064 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:58.078+0000 7fed62c00640 1 -- start start 2024-03-08T20:56:58.064 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:58.078+0000 7fed62c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fed5c073d50 0x7fed5c19dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:56:58.064 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:58.078+0000 7fed62c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fed5c075330 0x7fed5c19e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:56:58.064 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:58.078+0000 7fed62c00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fed5c10d8d0 0x7fed5c1a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:56:58.064 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:58.078+0000 7fed62c00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed5c113470 con 0x7fed5c075330 2024-03-08T20:56:58.064 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:58.078+0000 7fed62c00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed5c1132f0 con 0x7fed5c10d8d0 2024-03-08T20:56:58.064 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:58.078+0000 7fed62c00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed5c1135f0 con 0x7fed5c073d50 2024-03-08T20:56:58.064 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:58.079+0000 7fed5b400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fed5c075330 0x7fed5c19e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:56:58.064 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:58.079+0000 7fed5b400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fed5c075330 0x7fed5c19e0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:56150/0 (socket says 172.21.15.107:56150) 2024-03-08T20:56:58.065 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:58.079+0000 7fed5b400640 1 -- 172.21.15.107:0/4122870591 learned_addr learned my addr 172.21.15.107:0/4122870591 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T20:56:58.065 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:58.079+0000 7fed5be00640 1 --2- 172.21.15.107:0/4122870591 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fed5c073d50 0x7fed5c19dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:56:58.065 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:58.079+0000 7fed60c00640 1 --2- 172.21.15.107:0/4122870591 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fed5c10d8d0 0x7fed5c1a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:56:58.065 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:58.079+0000 7fed5b400640 1 -- 172.21.15.107:0/4122870591 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fed5c073d50 msgr2=0x7fed5c19dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:56:58.065 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:58.079+0000 7fed5b400640 1 --2- 172.21.15.107:0/4122870591 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fed5c073d50 0x7fed5c19dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:56:58.065 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:58.079+0000 7fed5b400640 1 -- 172.21.15.107:0/4122870591 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fed5c10d8d0 msgr2=0x7fed5c1a5150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:56:58.065 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:58.079+0000 7fed5b400640 1 --2- 172.21.15.107:0/4122870591 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fed5c10d8d0 0x7fed5c1a5150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:56:58.065 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:58.079+0000 7fed5b400640 1 -- 172.21.15.107:0/4122870591 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed5c1a57c0 con 0x7fed5c075330 2024-03-08T20:56:58.065 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:58.079+0000 7fed5b400640 1 --2- 172.21.15.107:0/4122870591 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fed5c075330 0x7fed5c19e0e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fed44005ab0 tx=0x7fed440042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:56:58.065 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:58.080+0000 7fed58c00640 1 -- 172.21.15.107:0/4122870591 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed44043020 con 0x7fed5c075330 2024-03-08T20:56:58.065 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:58.080+0000 7fed58c00640 1 -- 172.21.15.107:0/4122870591 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fed440054c0 con 0x7fed5c075330 2024-03-08T20:56:58.065 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:58.080+0000 7fed58c00640 1 -- 172.21.15.107:0/4122870591 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed44003d10 con 0x7fed5c075330 2024-03-08T20:56:58.065 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:58.080+0000 7fed62c00640 1 -- 172.21.15.107:0/4122870591 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed5c1a5a50 con 0x7fed5c075330 2024-03-08T20:56:58.065 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:58.080+0000 7fed62c00640 1 -- 172.21.15.107:0/4122870591 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed5c1a5ff0 con 0x7fed5c075330 2024-03-08T20:56:58.066 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:58.081+0000 7fed62c00640 1 -- 172.21.15.107:0/4122870591 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed5c103aa0 con 0x7fed5c075330 2024-03-08T20:56:58.067 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:58.083+0000 7fed58c00640 1 -- 172.21.15.107:0/4122870591 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) v1 ==== 137291+0+0 (secure 0 0 0) 0x7fed44046020 con 0x7fed5c075330 2024-03-08T20:56:58.068 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:58.084+0000 7fed58c00640 1 --2- 172.21.15.107:0/4122870591 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fed400a2880 0x7fed400a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:56:58.068 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:58.084+0000 7fed58c00640 1 -- 172.21.15.107:0/4122870591 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fed440f0d70 con 0x7fed5c075330 2024-03-08T20:56:58.069 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:58.084+0000 7fed5be00640 1 --2- 172.21.15.107:0/4122870591 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fed400a2880 0x7fed400a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:56:58.069 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:58.085+0000 7fed5be00640 1 --2- 172.21.15.107:0/4122870591 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fed400a2880 0x7fed400a4d40 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fed54005fd0 tx=0x7fed54009870 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:56:58.070 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:58.086+0000 7fed58c00640 1 -- 172.21.15.107:0/4122870591 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7fed4404d020 con 0x7fed5c075330 2024-03-08T20:56:58.270 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:58.286+0000 7fed62c00640 1 -- 172.21.15.107:0/4122870591 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fed5c02d0c0 con 0x7fed5c075330 2024-03-08T20:56:58.270 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:58.287+0000 7fed58c00640 1 -- 172.21.15.107:0/4122870591 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) v1 ==== 93+0+10 (secure 0 0 0) 0x7fed440be210 con 0x7fed5c075330 2024-03-08T20:56:58.271 INFO:teuthology.orchestra.run.smithi107.stdout:pg_num: 1 2024-03-08T20:56:58.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:58.289+0000 7fed62c00640 1 -- 172.21.15.107:0/4122870591 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fed400a2880 msgr2=0x7fed400a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:56:58.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:58.289+0000 7fed62c00640 1 --2- 172.21.15.107:0/4122870591 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fed400a2880 0x7fed400a4d40 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fed54005fd0 tx=0x7fed54009870 comp rx=0 tx=0).stop 2024-03-08T20:56:58.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:58.289+0000 7fed62c00640 1 -- 172.21.15.107:0/4122870591 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fed5c075330 msgr2=0x7fed5c19e0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:56:58.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:58.289+0000 7fed62c00640 1 --2- 172.21.15.107:0/4122870591 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fed5c075330 0x7fed5c19e0e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fed44005ab0 tx=0x7fed440042f0 comp rx=0 tx=0).stop 2024-03-08T20:56:58.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:58.289+0000 7fed62c00640 1 -- 172.21.15.107:0/4122870591 shutdown_connections 2024-03-08T20:56:58.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:58.289+0000 7fed62c00640 1 --2- 172.21.15.107:0/4122870591 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fed400a2880 0x7fed400a4d40 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:56:58.274 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:58.289+0000 7fed62c00640 1 --2- 172.21.15.107:0/4122870591 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fed5c10d8d0 0x7fed5c1a5150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:56:58.274 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:58.289+0000 7fed62c00640 1 --2- 172.21.15.107:0/4122870591 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fed5c075330 0x7fed5c19e0e0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:56:58.274 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:58.289+0000 7fed62c00640 1 --2- 172.21.15.107:0/4122870591 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fed5c073d50 0x7fed5c19dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:56:58.274 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:58.289+0000 7fed62c00640 1 -- 172.21.15.107:0/4122870591 >> 172.21.15.107:0/4122870591 conn(0x7fed5c0fc890 msgr2=0x7fed5c100c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:56:58.274 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:58.289+0000 7fed62c00640 1 -- 172.21.15.107:0/4122870591 shutdown_connections 2024-03-08T20:56:58.274 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:58.289+0000 7fed62c00640 1 -- 172.21.15.107:0/4122870591 wait complete. 2024-03-08T20:56:58.294 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-08T20:56:58.378 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:56:58 smithi204 ceph-mon[34935]: from='client.? 172.21.15.107:0/3434641341' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-08T20:56:58.469 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:56:58 smithi107 ceph-mon[31381]: from='client.? 172.21.15.107:0/3434641341' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-08T20:56:58.504 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:58.520+0000 7f7872a00640 1 -- 172.21.15.107:0/3352484747 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f786c104720 msgr2=0x7f786c104ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:56:58.504 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:58.520+0000 7f7872a00640 1 --2- 172.21.15.107:0/3352484747 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f786c104720 0x7f786c104ba0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f7854009900 tx=0x7f785402bc40 comp rx=0 tx=0).stop 2024-03-08T20:56:58.505 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:58.520+0000 7f7872a00640 1 -- 172.21.15.107:0/3352484747 shutdown_connections 2024-03-08T20:56:58.505 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:58.520+0000 7f7872a00640 1 --2- 172.21.15.107:0/3352484747 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f786c1050e0 0x7f786c10b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:56:58.505 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:58.520+0000 7f7872a00640 1 --2- 172.21.15.107:0/3352484747 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f786c104720 0x7f786c104ba0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:56:58.505 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:58.520+0000 7f7872a00640 1 --2- 172.21.15.107:0/3352484747 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f786c103530 0x7f786c103950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:56:58.505 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:58.520+0000 7f7872a00640 1 -- 172.21.15.107:0/3352484747 >> 172.21.15.107:0/3352484747 conn(0x7f786c0feae0 msgr2=0x7f786c100f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:56:58.505 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:58.520+0000 7f7872a00640 1 -- 172.21.15.107:0/3352484747 shutdown_connections 2024-03-08T20:56:58.505 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:58.520+0000 7f7872a00640 1 -- 172.21.15.107:0/3352484747 wait complete. 2024-03-08T20:56:58.507 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:58.521+0000 7f7872a00640 1 Processor -- start 2024-03-08T20:56:58.507 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:58.521+0000 7f7872a00640 1 -- start start 2024-03-08T20:56:58.507 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:58.521+0000 7f7872a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f786c103530 0x7f786c199630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:56:58.507 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:58.521+0000 7f7872a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f786c104720 0x7f786c199b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:56:58.507 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:58.521+0000 7f7872a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f786c1050e0 0x7f786c1a0be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:56:58.507 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:58.521+0000 7f7872a00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f786c10f0c0 con 0x7f786c103530 2024-03-08T20:56:58.507 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:58.521+0000 7f7872a00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f786c10ef40 con 0x7f786c104720 2024-03-08T20:56:58.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:58.521+0000 7f7872a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f786c10f240 con 0x7f786c1050e0 2024-03-08T20:56:58.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:58.521+0000 7f786be00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f786c103530 0x7f786c199630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:56:58.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:58.521+0000 7f786be00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f786c103530 0x7f786c199630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:56170/0 (socket says 172.21.15.107:56170) 2024-03-08T20:56:58.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:58.521+0000 7f786be00640 1 -- 172.21.15.107:0/2360537466 learned_addr learned my addr 172.21.15.107:0/2360537466 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T20:56:58.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:58.522+0000 7f786be00640 1 -- 172.21.15.107:0/2360537466 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f786c1050e0 msgr2=0x7f786c1a0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:56:58.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:58.522+0000 7f786be00640 1 --2- 172.21.15.107:0/2360537466 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f786c1050e0 0x7f786c1a0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:56:58.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:58.522+0000 7f786be00640 1 -- 172.21.15.107:0/2360537466 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f786c104720 msgr2=0x7f786c199b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:56:58.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:58.522+0000 7f786be00640 1 --2- 172.21.15.107:0/2360537466 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f786c104720 0x7f786c199b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:56:58.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:58.522+0000 7f786be00640 1 -- 172.21.15.107:0/2360537466 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f786c1a1250 con 0x7f786c103530 2024-03-08T20:56:58.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:58.522+0000 7f786be00640 1 --2- 172.21.15.107:0/2360537466 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f786c103530 0x7f786c199630 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f785c00c970 tx=0x7f785c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:56:58.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:58.522+0000 7f7869600640 1 -- 172.21.15.107:0/2360537466 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f785c007c10 con 0x7f786c103530 2024-03-08T20:56:58.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:58.522+0000 7f7869600640 1 -- 172.21.15.107:0/2360537466 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f785c007db0 con 0x7f786c103530 2024-03-08T20:56:58.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:58.522+0000 7f7869600640 1 -- 172.21.15.107:0/2360537466 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f785c00f870 con 0x7f786c103530 2024-03-08T20:56:58.509 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:58.522+0000 7f7872a00640 1 -- 172.21.15.107:0/2360537466 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f786c1a1540 con 0x7f786c103530 2024-03-08T20:56:58.509 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:58.523+0000 7f7872a00640 1 -- 172.21.15.107:0/2360537466 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f786c1a1ae0 con 0x7f786c103530 2024-03-08T20:56:58.509 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:58.524+0000 7f7872a00640 1 -- 172.21.15.107:0/2360537466 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f786c10a7f0 con 0x7f786c103530 2024-03-08T20:56:58.509 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:58.526+0000 7f7869600640 1 -- 172.21.15.107:0/2360537466 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f785c00fa10 con 0x7f786c103530 2024-03-08T20:56:58.511 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:58.527+0000 7f7869600640 1 --2- 172.21.15.107:0/2360537466 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f78500a2880 0x7f78500a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:56:58.511 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:58.527+0000 7f7869600640 1 -- 172.21.15.107:0/2360537466 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f785c0ce9b0 con 0x7f786c103530 2024-03-08T20:56:58.511 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:58.527+0000 7f7863e00640 1 --2- 172.21.15.107:0/2360537466 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f78500a2880 0x7f78500a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:56:58.512 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:58.528+0000 7f7863e00640 1 --2- 172.21.15.107:0/2360537466 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f78500a2880 0x7f78500a4d40 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f7854005c20 tx=0x7f7854000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:56:58.515 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:58.531+0000 7f7869600640 1 -- 172.21.15.107:0/2360537466 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f785c09bf60 con 0x7f786c103530 2024-03-08T20:56:58.516 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:56:58 smithi149 ceph-mon[35042]: from='client.? 172.21.15.107:0/3434641341' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-08T20:56:58.750 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:58.765+0000 7f7872a00640 1 -- 172.21.15.107:0/2360537466 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f786c02d0b0 con 0x7f786c103530 2024-03-08T20:56:58.750 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:58.766+0000 7f7869600640 1 -- 172.21.15.107:0/2360537466 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) v1 ==== 93+0+535 (secure 0 0 0) 0x7f785c09b860 con 0x7f786c103530 2024-03-08T20:56:58.751 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-08T20:56:58.751 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-03-08T20:56:58.751 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 1 2024-03-08T20:56:58.754 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:58.770+0000 7f7872a00640 1 -- 172.21.15.107:0/2360537466 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f78500a2880 msgr2=0x7f78500a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:56:58.754 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:58.770+0000 7f7872a00640 1 --2- 172.21.15.107:0/2360537466 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f78500a2880 0x7f78500a4d40 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f7854005c20 tx=0x7f7854000bc0 comp rx=0 tx=0).stop 2024-03-08T20:56:58.754 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:58.770+0000 7f7872a00640 1 -- 172.21.15.107:0/2360537466 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f786c103530 msgr2=0x7f786c199630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:56:58.754 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:58.770+0000 7f7872a00640 1 --2- 172.21.15.107:0/2360537466 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f786c103530 0x7f786c199630 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f785c00c970 tx=0x7f785c00ce30 comp rx=0 tx=0).stop 2024-03-08T20:56:58.755 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:58.770+0000 7f7872a00640 1 -- 172.21.15.107:0/2360537466 shutdown_connections 2024-03-08T20:56:58.755 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:58.770+0000 7f7872a00640 1 --2- 172.21.15.107:0/2360537466 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f78500a2880 0x7f78500a4d40 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:56:58.755 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:58.770+0000 7f7872a00640 1 --2- 172.21.15.107:0/2360537466 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f786c1050e0 0x7f786c1a0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:56:58.755 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:58.770+0000 7f7872a00640 1 --2- 172.21.15.107:0/2360537466 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f786c104720 0x7f786c199b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:56:58.755 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:58.770+0000 7f7872a00640 1 --2- 172.21.15.107:0/2360537466 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f786c103530 0x7f786c199630 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:56:58.755 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:58.770+0000 7f7872a00640 1 -- 172.21.15.107:0/2360537466 >> 172.21.15.107:0/2360537466 conn(0x7f786c0feae0 msgr2=0x7f786c1079c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:56:58.755 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:58.770+0000 7f7872a00640 1 -- 172.21.15.107:0/2360537466 shutdown_connections 2024-03-08T20:56:58.755 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:56:58.770+0000 7f7872a00640 1 -- 172.21.15.107:0/2360537466 wait complete. 2024-03-08T20:56:59.254 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:56:59 smithi204 ceph-mon[34935]: from='client.? 172.21.15.107:0/4122870591' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-08T20:56:59.254 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:56:59 smithi204 ceph-mon[34935]: pgmap v331: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-08T20:56:59.254 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:56:59 smithi204 ceph-mon[34935]: from='client.? 172.21.15.107:0/2360537466' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-08T20:56:59.469 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:56:59 smithi107 ceph-mon[31381]: from='client.? 172.21.15.107:0/4122870591' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-08T20:56:59.469 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:56:59 smithi107 ceph-mon[31381]: pgmap v331: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-08T20:56:59.469 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:56:59 smithi107 ceph-mon[31381]: from='client.? 172.21.15.107:0/2360537466' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-08T20:56:59.516 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:56:59 smithi149 ceph-mon[35042]: from='client.? 172.21.15.107:0/4122870591' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-08T20:56:59.516 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:56:59 smithi149 ceph-mon[35042]: pgmap v331: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-08T20:56:59.516 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:56:59 smithi149 ceph-mon[35042]: from='client.? 172.21.15.107:0/2360537466' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-08T20:56:59.602 INFO:journalctl@ceph.mds.l.smithi204.stdout:Mar 08 20:56:59 smithi204 systemd[1]: Starting Ceph mds.l for dc20f0f8-dd8c-11ee-95c6-87774f69a715... 2024-03-08T20:56:59.880 INFO:journalctl@ceph.mds.l.smithi204.stdout:Mar 08 20:56:59 smithi204 podman[65367]: 2024-03-08 20:56:59.522165421 +0000 UTC m=+0.030838027 image pull a272146cb810b1fd4a53627f019472132e30966d5c25b01bfc44a001295f1249 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd 2024-03-08T20:56:59.880 INFO:journalctl@ceph.mds.l.smithi204.stdout:Mar 08 20:56:59 smithi204 podman[65367]: 2024-03-08 20:56:59.643708634 +0000 UTC m=+0.152381265 container create 859782e18848f12e8625c6bbbfc37b077526221cf043677aa87f0d19aaaf2a28 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-l, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, RELEASE=rishabh-2024mar8-7d63482, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-03-08T20:57:00.380 INFO:journalctl@ceph.mds.l.smithi204.stdout:Mar 08 20:56:59 smithi204 podman[65367]: 2024-03-08 20:56:59.942281169 +0000 UTC m=+0.450953810 container init 859782e18848f12e8625c6bbbfc37b077526221cf043677aa87f0d19aaaf2a28 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-l, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=rishabh-2024mar8-7d63482, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-03-08T20:57:00.380 INFO:journalctl@ceph.mds.l.smithi204.stdout:Mar 08 20:56:59 smithi204 podman[65367]: 2024-03-08 20:56:59.953523866 +0000 UTC m=+0.462196485 container start 859782e18848f12e8625c6bbbfc37b077526221cf043677aa87f0d19aaaf2a28 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-l, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=rishabh-2024mar8-7d63482, io.buildah.version=1.33.5, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307) 2024-03-08T20:57:00.380 INFO:journalctl@ceph.mds.l.smithi204.stdout:Mar 08 20:57:00 smithi204 ceph-mds[65386]: -- 172.21.15.204:0/537870838 <== mon.1 v2:172.21.15.204:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x564c251a7a40 con 0x564c2517ac00 2024-03-08T20:57:00.380 INFO:journalctl@ceph.mds.l.smithi204.stdout:Mar 08 20:57:00 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-l[65382]: starting mds.l at 2024-03-08T20:57:00.380 INFO:journalctl@ceph.mds.l.smithi204.stdout:Mar 08 20:57:00 smithi204 bash[65367]: 859782e18848f12e8625c6bbbfc37b077526221cf043677aa87f0d19aaaf2a28 2024-03-08T20:57:00.380 INFO:journalctl@ceph.mds.l.smithi204.stdout:Mar 08 20:57:00 smithi204 systemd[1]: Started Ceph mds.l for dc20f0f8-dd8c-11ee-95c6-87774f69a715. 2024-03-08T20:57:00.774 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-08T20:57:00.989 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:01.003+0000 7fb917400640 1 -- 172.21.15.107:0/3459833529 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb910104720 msgr2=0x7fb910104ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:57:00.989 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:01.003+0000 7fb917400640 1 --2- 172.21.15.107:0/3459833529 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb910104720 0x7fb910104ba0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fb8fc009900 tx=0x7fb8fc02bc30 comp rx=0 tx=0).stop 2024-03-08T20:57:00.989 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:01.004+0000 7fb917400640 1 -- 172.21.15.107:0/3459833529 shutdown_connections 2024-03-08T20:57:00.989 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:01.004+0000 7fb917400640 1 --2- 172.21.15.107:0/3459833529 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb9101050e0 0x7fb91010b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:00.989 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:01.004+0000 7fb917400640 1 --2- 172.21.15.107:0/3459833529 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb910104720 0x7fb910104ba0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:00.989 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:01.004+0000 7fb917400640 1 --2- 172.21.15.107:0/3459833529 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb910103530 0x7fb910103950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:00.989 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:01.004+0000 7fb917400640 1 -- 172.21.15.107:0/3459833529 >> 172.21.15.107:0/3459833529 conn(0x7fb9100feaa0 msgr2=0x7fb910100ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:57:00.989 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:01.004+0000 7fb917400640 1 -- 172.21.15.107:0/3459833529 shutdown_connections 2024-03-08T20:57:00.989 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:01.004+0000 7fb917400640 1 -- 172.21.15.107:0/3459833529 wait complete. 2024-03-08T20:57:00.992 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:01.005+0000 7fb917400640 1 Processor -- start 2024-03-08T20:57:00.992 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:01.005+0000 7fb917400640 1 -- start start 2024-03-08T20:57:00.992 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:01.005+0000 7fb917400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb910103530 0x7fb910199740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:57:00.992 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:01.005+0000 7fb917400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb910104720 0x7fb910199c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:57:00.992 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:01.005+0000 7fb917400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb9101050e0 0x7fb9101a0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:57:00.992 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:01.005+0000 7fb917400640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb91010f070 con 0x7fb910104720 2024-03-08T20:57:00.993 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:01.005+0000 7fb917400640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb91010eef0 con 0x7fb910103530 2024-03-08T20:57:00.993 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:01.005+0000 7fb917400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb91010f1f0 con 0x7fb9101050e0 2024-03-08T20:57:00.993 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:01.005+0000 7fb914a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb910103530 0x7fb910199740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:57:00.993 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:01.005+0000 7fb90fe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb910104720 0x7fb910199c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:57:00.993 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:01.005+0000 7fb914a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb910103530 0x7fb910199740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.107:60026/0 (socket says 172.21.15.107:60026) 2024-03-08T20:57:00.993 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:01.005+0000 7fb914a00640 1 -- 172.21.15.107:0/4005423400 learned_addr learned my addr 172.21.15.107:0/4005423400 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T20:57:00.993 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:01.006+0000 7fb915400640 1 --2- 172.21.15.107:0/4005423400 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb9101050e0 0x7fb9101a0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:57:00.993 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:01.006+0000 7fb914a00640 1 -- 172.21.15.107:0/4005423400 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb9101050e0 msgr2=0x7fb9101a0cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:57:00.993 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:01.006+0000 7fb914a00640 1 --2- 172.21.15.107:0/4005423400 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb9101050e0 0x7fb9101a0cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:00.993 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:01.006+0000 7fb914a00640 1 -- 172.21.15.107:0/4005423400 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb910104720 msgr2=0x7fb910199c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:57:00.993 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:01.006+0000 7fb914a00640 1 --2- 172.21.15.107:0/4005423400 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb910104720 0x7fb910199c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:00.993 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:01.006+0000 7fb914a00640 1 -- 172.21.15.107:0/4005423400 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9101a1360 con 0x7fb910103530 2024-03-08T20:57:00.993 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:01.006+0000 7fb915400640 1 --2- 172.21.15.107:0/4005423400 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb9101050e0 0x7fb9101a0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T20:57:00.993 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:01.006+0000 7fb914a00640 1 --2- 172.21.15.107:0/4005423400 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb910103530 0x7fb910199740 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fb90400d950 tx=0x7fb90400de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:57:00.994 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:01.006+0000 7fb90d600640 1 -- 172.21.15.107:0/4005423400 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb904017070 con 0x7fb910103530 2024-03-08T20:57:00.994 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:01.006+0000 7fb90d600640 1 -- 172.21.15.107:0/4005423400 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fb904009c30 con 0x7fb910103530 2024-03-08T20:57:00.994 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:01.006+0000 7fb917400640 1 -- 172.21.15.107:0/4005423400 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9101a1650 con 0x7fb910103530 2024-03-08T20:57:00.994 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:01.006+0000 7fb90d600640 1 -- 172.21.15.107:0/4005423400 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb90400f620 con 0x7fb910103530 2024-03-08T20:57:00.994 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:01.006+0000 7fb917400640 1 -- 172.21.15.107:0/4005423400 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9101a1b70 con 0x7fb910103530 2024-03-08T20:57:00.994 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:01.007+0000 7fb917400640 1 -- 172.21.15.107:0/4005423400 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb91010a920 con 0x7fb910103530 2024-03-08T20:57:00.995 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:01.010+0000 7fb90d600640 1 -- 172.21.15.107:0/4005423400 <== mon.2 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 16) v1 ==== 137291+0+0 (secure 0 0 0) 0x7fb90401a070 con 0x7fb910103530 2024-03-08T20:57:00.996 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:01.011+0000 7fb90d600640 1 --2- 172.21.15.107:0/4005423400 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fb8f80a2880 0x7fb8f80a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:57:00.997 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:01.012+0000 7fb90d600640 1 -- 172.21.15.107:0/4005423400 <== mon.2 v2:172.21.15.149:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fb9040ce0a0 con 0x7fb910103530 2024-03-08T20:57:00.997 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:01.012+0000 7fb90fe00640 1 --2- 172.21.15.107:0/4005423400 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fb8f80a2880 0x7fb8f80a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:57:00.997 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:01.013+0000 7fb90fe00640 1 --2- 172.21.15.107:0/4005423400 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fb8f80a2880 0x7fb8f80a4d40 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fb8fc009230 tx=0x7fb8fc005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:57:00.999 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:01.014+0000 7fb90d600640 1 -- 172.21.15.107:0/4005423400 <== mon.2 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7fb904009de0 con 0x7fb910103530 2024-03-08T20:57:01.219 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:01 smithi107 ceph-mon[31381]: mds.? [v2:172.21.15.204:6832/3404762782,v1:172.21.15.204:6833/3404762782] up:boot 2024-03-08T20:57:01.219 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:01 smithi107 ceph-mon[31381]: fsmap 1 up:standby 2024-03-08T20:57:01.219 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:01 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-08T20:57:01.219 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:01 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:57:01.219 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:01 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:57:01.219 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:01 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:57:01.219 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:01 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-08T20:57:01.219 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:01 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-08T20:57:01.219 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:01 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:57:01.220 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:01 smithi107 ceph-mon[31381]: Deploying daemon mds.k on smithi149 2024-03-08T20:57:01.220 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:01 smithi107 ceph-mon[31381]: pgmap v332: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-08T20:57:01.235 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:01.251+0000 7fb917400640 1 -- 172.21.15.107:0/4005423400 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb91002ce70 con 0x7fb910103530 2024-03-08T20:57:01.237 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:01.252+0000 7fb90d600640 1 -- 172.21.15.107:0/4005423400 <== mon.2 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1328 (secure 0 0 0) 0x7fb90409b5c0 con 0x7fb910103530 2024-03-08T20:57:01.237 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-08T20:57:01.237 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/3404762782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":3404762782},{"type":"v1","addr":"172.21.15.204:6833","nonce":3404762782}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2024-03-08T20:57:01.238 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 2 2024-03-08T20:57:01.240 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:01.255+0000 7fb917400640 1 -- 172.21.15.107:0/4005423400 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fb8f80a2880 msgr2=0x7fb8f80a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:57:01.240 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:01.255+0000 7fb917400640 1 --2- 172.21.15.107:0/4005423400 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fb8f80a2880 0x7fb8f80a4d40 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fb8fc009230 tx=0x7fb8fc005fd0 comp rx=0 tx=0).stop 2024-03-08T20:57:01.241 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:01.255+0000 7fb917400640 1 -- 172.21.15.107:0/4005423400 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb910103530 msgr2=0x7fb910199740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:57:01.241 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:01.255+0000 7fb917400640 1 --2- 172.21.15.107:0/4005423400 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb910103530 0x7fb910199740 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fb90400d950 tx=0x7fb90400de10 comp rx=0 tx=0).stop 2024-03-08T20:57:01.241 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:01.255+0000 7fb917400640 1 -- 172.21.15.107:0/4005423400 shutdown_connections 2024-03-08T20:57:01.241 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:01.255+0000 7fb917400640 1 --2- 172.21.15.107:0/4005423400 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fb8f80a2880 0x7fb8f80a4d40 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:01.241 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:01.255+0000 7fb917400640 1 --2- 172.21.15.107:0/4005423400 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb9101050e0 0x7fb9101a0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:01.241 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:01.255+0000 7fb917400640 1 --2- 172.21.15.107:0/4005423400 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb910104720 0x7fb910199c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:01.242 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:01.255+0000 7fb917400640 1 --2- 172.21.15.107:0/4005423400 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb910103530 0x7fb910199740 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:01.242 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:01.255+0000 7fb917400640 1 -- 172.21.15.107:0/4005423400 >> 172.21.15.107:0/4005423400 conn(0x7fb9100feaa0 msgr2=0x7fb910107af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:57:01.242 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:01.255+0000 7fb917400640 1 -- 172.21.15.107:0/4005423400 shutdown_connections 2024-03-08T20:57:01.242 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:01.256+0000 7fb917400640 1 -- 172.21.15.107:0/4005423400 wait complete. 2024-03-08T20:57:01.267 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:01 smithi149 ceph-mon[35042]: mds.? [v2:172.21.15.204:6832/3404762782,v1:172.21.15.204:6833/3404762782] up:boot 2024-03-08T20:57:01.267 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:01 smithi149 ceph-mon[35042]: fsmap 1 up:standby 2024-03-08T20:57:01.267 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:01 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-08T20:57:01.267 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:01 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:57:01.267 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:01 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:57:01.267 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:01 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:57:01.267 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:01 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-08T20:57:01.267 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:01 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-08T20:57:01.267 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:01 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:57:01.268 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:01 smithi149 ceph-mon[35042]: Deploying daemon mds.k on smithi149 2024-03-08T20:57:01.268 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:01 smithi149 ceph-mon[35042]: pgmap v332: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-08T20:57:01.380 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:01 smithi204 ceph-mon[34935]: mds.? [v2:172.21.15.204:6832/3404762782,v1:172.21.15.204:6833/3404762782] up:boot 2024-03-08T20:57:01.380 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:01 smithi204 ceph-mon[34935]: fsmap 1 up:standby 2024-03-08T20:57:01.380 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:01 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-08T20:57:01.380 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:01 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:57:01.380 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:01 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:57:01.380 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:01 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:57:01.380 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:01 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-08T20:57:01.380 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:01 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-08T20:57:01.381 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:01 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:57:01.381 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:01 smithi204 ceph-mon[34935]: Deploying daemon mds.k on smithi149 2024-03-08T20:57:01.381 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:01 smithi204 ceph-mon[34935]: pgmap v332: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-08T20:57:02.266 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:02 smithi149 ceph-mon[35042]: from='client.? 172.21.15.107:0/4005423400' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-08T20:57:02.379 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:02 smithi204 ceph-mon[34935]: from='client.? 172.21.15.107:0/4005423400' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-08T20:57:02.468 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:02 smithi107 ceph-mon[31381]: from='client.? 172.21.15.107:0/4005423400' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-08T20:57:03.229 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:03 smithi149 ceph-mon[35042]: pgmap v333: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-08T20:57:03.261 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-08T20:57:03.282 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:03 smithi204 ceph-mon[34935]: pgmap v333: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-08T20:57:03.349 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:03 smithi107 ceph-mon[31381]: pgmap v333: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-08T20:57:03.485 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:03.500+0000 7fe31d600640 1 -- 172.21.15.107:0/3576532355 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe318077890 msgr2=0x7fe318078130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:57:03.485 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:03.500+0000 7fe31d600640 1 --2- 172.21.15.107:0/3576532355 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe318077890 0x7fe318078130 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fe310009ca0 tx=0x7fe31002c0f0 comp rx=0 tx=0).stop 2024-03-08T20:57:03.485 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:03.500+0000 7fe31d600640 1 -- 172.21.15.107:0/3576532355 shutdown_connections 2024-03-08T20:57:03.485 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:03.500+0000 7fe31d600640 1 --2- 172.21.15.107:0/3576532355 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe318077890 0x7fe318078130 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:03.485 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:03.500+0000 7fe31d600640 1 --2- 172.21.15.107:0/3576532355 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe318076ef0 0x7fe318077350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:03.486 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:03.500+0000 7fe31d600640 1 --2- 172.21.15.107:0/3576532355 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe318075cb0 0x7fe3180760d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:03.486 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:03.500+0000 7fe31d600640 1 -- 172.21.15.107:0/3576532355 >> 172.21.15.107:0/3576532355 conn(0x7fe3180feaa0 msgr2=0x7fe318100ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:57:03.486 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:03.500+0000 7fe31d600640 1 -- 172.21.15.107:0/3576532355 shutdown_connections 2024-03-08T20:57:03.486 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:03.500+0000 7fe31d600640 1 -- 172.21.15.107:0/3576532355 wait complete. 2024-03-08T20:57:03.488 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:03.501+0000 7fe31d600640 1 Processor -- start 2024-03-08T20:57:03.488 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:03.501+0000 7fe31d600640 1 -- start start 2024-03-08T20:57:03.488 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:03.501+0000 7fe31d600640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe318075cb0 0x7fe31819db00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:57:03.488 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:03.501+0000 7fe31d600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe318076ef0 0x7fe31819e040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:57:03.488 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:03.501+0000 7fe31d600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe318077890 0x7fe3181a50b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:57:03.488 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:03.501+0000 7fe31d600640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe318113470 con 0x7fe318076ef0 2024-03-08T20:57:03.489 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:03.501+0000 7fe31d600640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3181132f0 con 0x7fe318075cb0 2024-03-08T20:57:03.489 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:03.501+0000 7fe31d600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3181135f0 con 0x7fe318077890 2024-03-08T20:57:03.489 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:03.501+0000 7fe316000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe318076ef0 0x7fe31819e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:57:03.489 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:03.502+0000 7fe316000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe318076ef0 0x7fe31819e040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:56206/0 (socket says 172.21.15.107:56206) 2024-03-08T20:57:03.489 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:03.502+0000 7fe316000640 1 -- 172.21.15.107:0/2785460512 learned_addr learned my addr 172.21.15.107:0/2785460512 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T20:57:03.489 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:03.502+0000 7fe316a00640 1 --2- 172.21.15.107:0/2785460512 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe318075cb0 0x7fe31819db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:57:03.489 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:03.502+0000 7fe317400640 1 --2- 172.21.15.107:0/2785460512 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe318077890 0x7fe3181a50b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:57:03.489 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:03.502+0000 7fe316000640 1 -- 172.21.15.107:0/2785460512 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe318077890 msgr2=0x7fe3181a50b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:57:03.489 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:03.502+0000 7fe316000640 1 --2- 172.21.15.107:0/2785460512 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe318077890 0x7fe3181a50b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:03.489 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:03.502+0000 7fe316000640 1 -- 172.21.15.107:0/2785460512 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe318075cb0 msgr2=0x7fe31819db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:57:03.489 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:03.502+0000 7fe316000640 1 --2- 172.21.15.107:0/2785460512 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe318075cb0 0x7fe31819db00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:03.489 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:03.502+0000 7fe316000640 1 -- 172.21.15.107:0/2785460512 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3181a5720 con 0x7fe318076ef0 2024-03-08T20:57:03.490 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:03.502+0000 7fe316000640 1 --2- 172.21.15.107:0/2785460512 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe318076ef0 0x7fe31819e040 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fe30400d770 tx=0x7fe30400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:57:03.490 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:03.502+0000 7fe30f400640 1 -- 172.21.15.107:0/2785460512 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe304017070 con 0x7fe318076ef0 2024-03-08T20:57:03.490 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:03.503+0000 7fe30f400640 1 -- 172.21.15.107:0/2785460512 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fe3040054d0 con 0x7fe318076ef0 2024-03-08T20:57:03.490 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:03.503+0000 7fe31d600640 1 -- 172.21.15.107:0/2785460512 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3181a5a10 con 0x7fe318076ef0 2024-03-08T20:57:03.490 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:03.503+0000 7fe30f400640 1 -- 172.21.15.107:0/2785460512 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe304004cf0 con 0x7fe318076ef0 2024-03-08T20:57:03.490 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:03.503+0000 7fe31d600640 1 -- 172.21.15.107:0/2785460512 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3181a5f10 con 0x7fe318076ef0 2024-03-08T20:57:03.490 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:03.504+0000 7fe31d600640 1 -- 172.21.15.107:0/2785460512 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe318110020 con 0x7fe318076ef0 2024-03-08T20:57:03.491 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:03.506+0000 7fe30f400640 1 -- 172.21.15.107:0/2785460512 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) v1 ==== 137291+0+0 (secure 0 0 0) 0x7fe304004e90 con 0x7fe318076ef0 2024-03-08T20:57:03.492 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:03.508+0000 7fe30f400640 1 --2- 172.21.15.107:0/2785460512 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fe2fc0a2800 0x7fe2fc0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:57:03.493 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:03.508+0000 7fe30f400640 1 -- 172.21.15.107:0/2785460512 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fe3040ce020 con 0x7fe318076ef0 2024-03-08T20:57:03.493 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:03.508+0000 7fe316a00640 1 --2- 172.21.15.107:0/2785460512 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fe2fc0a2800 0x7fe2fc0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:57:03.493 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:03.509+0000 7fe316a00640 1 --2- 172.21.15.107:0/2785460512 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fe2fc0a2800 0x7fe2fc0a4cc0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fe308009bd0 tx=0x7fe308005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:57:03.497 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:03.512+0000 7fe30f400640 1 -- 172.21.15.107:0/2785460512 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7fe30409b560 con 0x7fe318076ef0 2024-03-08T20:57:03.516 INFO:journalctl@ceph.mds.k.smithi149.stdout:Mar 08 20:57:03 smithi149 systemd[1]: Starting Ceph mds.k for dc20f0f8-dd8c-11ee-95c6-87774f69a715... 2024-03-08T20:57:03.737 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:03.751+0000 7fe31d600640 1 -- 172.21.15.107:0/2785460512 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe31802d0c0 con 0x7fe318076ef0 2024-03-08T20:57:03.737 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:03.752+0000 7fe30f400640 1 -- 172.21.15.107:0/2785460512 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1328 (secure 0 0 0) 0x7fe30409ae60 con 0x7fe318076ef0 2024-03-08T20:57:03.738 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-08T20:57:03.738 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/3404762782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":3404762782},{"type":"v1","addr":"172.21.15.204:6833","nonce":3404762782}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2024-03-08T20:57:03.738 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 2 2024-03-08T20:57:03.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:03.755+0000 7fe31d600640 1 -- 172.21.15.107:0/2785460512 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fe2fc0a2800 msgr2=0x7fe2fc0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:57:03.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:03.755+0000 7fe31d600640 1 --2- 172.21.15.107:0/2785460512 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fe2fc0a2800 0x7fe2fc0a4cc0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fe308009bd0 tx=0x7fe308005f90 comp rx=0 tx=0).stop 2024-03-08T20:57:03.741 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:03.755+0000 7fe31d600640 1 -- 172.21.15.107:0/2785460512 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe318076ef0 msgr2=0x7fe31819e040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:57:03.741 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:03.755+0000 7fe31d600640 1 --2- 172.21.15.107:0/2785460512 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe318076ef0 0x7fe31819e040 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fe30400d770 tx=0x7fe30400dc30 comp rx=0 tx=0).stop 2024-03-08T20:57:03.741 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:03.755+0000 7fe31d600640 1 -- 172.21.15.107:0/2785460512 shutdown_connections 2024-03-08T20:57:03.741 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:03.755+0000 7fe31d600640 1 --2- 172.21.15.107:0/2785460512 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fe2fc0a2800 0x7fe2fc0a4cc0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:03.741 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:03.755+0000 7fe31d600640 1 --2- 172.21.15.107:0/2785460512 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe318077890 0x7fe3181a50b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:03.741 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:03.755+0000 7fe31d600640 1 --2- 172.21.15.107:0/2785460512 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe318076ef0 0x7fe31819e040 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:03.741 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:03.755+0000 7fe31d600640 1 --2- 172.21.15.107:0/2785460512 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe318075cb0 0x7fe31819db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:03.742 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:03.755+0000 7fe31d600640 1 -- 172.21.15.107:0/2785460512 >> 172.21.15.107:0/2785460512 conn(0x7fe3180feaa0 msgr2=0x7fe318100200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:57:03.742 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:03.755+0000 7fe31d600640 1 -- 172.21.15.107:0/2785460512 shutdown_connections 2024-03-08T20:57:03.742 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:03.755+0000 7fe31d600640 1 -- 172.21.15.107:0/2785460512 wait complete. 2024-03-08T20:57:03.890 INFO:journalctl@ceph.mds.k.smithi149.stdout:Mar 08 20:57:03 smithi149 podman[64667]: 2024-03-08 20:57:03.533395252 +0000 UTC m=+0.042083251 image pull a272146cb810b1fd4a53627f019472132e30966d5c25b01bfc44a001295f1249 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd 2024-03-08T20:57:03.890 INFO:journalctl@ceph.mds.k.smithi149.stdout:Mar 08 20:57:03 smithi149 podman[64667]: 2024-03-08 20:57:03.6643623 +0000 UTC m=+0.173050300 container create 4bdbc491acceb2fa6e5863a64c8e0662249afcef287b2870c557152e597f3d65 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-k, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, RELEASE=rishabh-2024mar8-7d63482, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-03-08T20:57:04.266 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:04 smithi149 ceph-mon[35042]: from='client.? 172.21.15.107:0/2785460512' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-08T20:57:04.267 INFO:journalctl@ceph.mds.k.smithi149.stdout:Mar 08 20:57:03 smithi149 podman[64667]: 2024-03-08 20:57:03.919823478 +0000 UTC m=+0.428511481 container init 4bdbc491acceb2fa6e5863a64c8e0662249afcef287b2870c557152e597f3d65 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-k, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=rishabh-2024mar8-7d63482, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-03-08T20:57:04.267 INFO:journalctl@ceph.mds.k.smithi149.stdout:Mar 08 20:57:03 smithi149 podman[64667]: 2024-03-08 20:57:03.92992988 +0000 UTC m=+0.438617847 container start 4bdbc491acceb2fa6e5863a64c8e0662249afcef287b2870c557152e597f3d65 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-k, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=rishabh-2024mar8-7d63482, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-03-08T20:57:04.267 INFO:journalctl@ceph.mds.k.smithi149.stdout:Mar 08 20:57:03 smithi149 ceph-mds[64694]: -- 172.21.15.149:0/2505207063 <== mon.2 v2:172.21.15.149:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55644a9e3a40 con 0x55644a9b6800 2024-03-08T20:57:04.267 INFO:journalctl@ceph.mds.k.smithi149.stdout:Mar 08 20:57:03 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-k[64685]: starting mds.k at 2024-03-08T20:57:04.267 INFO:journalctl@ceph.mds.k.smithi149.stdout:Mar 08 20:57:04 smithi149 bash[64667]: 4bdbc491acceb2fa6e5863a64c8e0662249afcef287b2870c557152e597f3d65 2024-03-08T20:57:04.267 INFO:journalctl@ceph.mds.k.smithi149.stdout:Mar 08 20:57:04 smithi149 systemd[1]: Started Ceph mds.k for dc20f0f8-dd8c-11ee-95c6-87774f69a715. 2024-03-08T20:57:04.311 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:04 smithi204 ceph-mon[34935]: from='client.? 172.21.15.107:0/2785460512' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-08T20:57:04.468 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:04 smithi107 ceph-mon[31381]: from='client.? 172.21.15.107:0/2785460512' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-08T20:57:05.267 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:05 smithi149 ceph-mon[35042]: mds.? [v2:172.21.15.149:6832/1962161726,v1:172.21.15.149:6833/1962161726] up:boot 2024-03-08T20:57:05.267 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:05 smithi149 ceph-mon[35042]: fsmap 2 up:standby 2024-03-08T20:57:05.267 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:05 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-08T20:57:05.267 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:05 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:57:05.267 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:05 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:57:05.267 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:05 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:57:05.267 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:05 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-08T20:57:05.267 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:05 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-08T20:57:05.267 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:05 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:57:05.267 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:05 smithi149 ceph-mon[35042]: Deploying daemon mds.j on smithi107 2024-03-08T20:57:05.267 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:05 smithi149 ceph-mon[35042]: pgmap v334: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-08T20:57:05.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:05 smithi107 ceph-mon[31381]: mds.? [v2:172.21.15.149:6832/1962161726,v1:172.21.15.149:6833/1962161726] up:boot 2024-03-08T20:57:05.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:05 smithi107 ceph-mon[31381]: fsmap 2 up:standby 2024-03-08T20:57:05.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:05 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-08T20:57:05.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:05 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:57:05.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:05 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:57:05.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:05 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:57:05.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:05 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-08T20:57:05.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:05 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-08T20:57:05.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:05 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:57:05.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:05 smithi107 ceph-mon[31381]: Deploying daemon mds.j on smithi107 2024-03-08T20:57:05.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:05 smithi107 ceph-mon[31381]: pgmap v334: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-08T20:57:05.379 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:05 smithi204 ceph-mon[34935]: mds.? [v2:172.21.15.149:6832/1962161726,v1:172.21.15.149:6833/1962161726] up:boot 2024-03-08T20:57:05.379 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:05 smithi204 ceph-mon[34935]: fsmap 2 up:standby 2024-03-08T20:57:05.380 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:05 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-08T20:57:05.380 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:05 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:57:05.380 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:05 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:57:05.380 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:05 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:57:05.380 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:05 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-08T20:57:05.380 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:05 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-08T20:57:05.380 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:05 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:57:05.380 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:05 smithi204 ceph-mon[34935]: Deploying daemon mds.j on smithi107 2024-03-08T20:57:05.380 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:05 smithi204 ceph-mon[34935]: pgmap v334: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-08T20:57:05.759 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-08T20:57:05.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:05.995+0000 7fb754000640 1 -- 172.21.15.107:0/3811062664 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb74c101310 msgr2=0x7fb74c101730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:57:05.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:05.995+0000 7fb754000640 1 --2- 172.21.15.107:0/3811062664 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb74c101310 0x7fb74c101730 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fb744009ca0 tx=0x7fb74402c0f0 comp rx=0 tx=0).stop 2024-03-08T20:57:05.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:05.995+0000 7fb754000640 1 -- 172.21.15.107:0/3811062664 shutdown_connections 2024-03-08T20:57:05.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:05.995+0000 7fb754000640 1 --2- 172.21.15.107:0/3811062664 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb74c102ec0 0x7fb74c109750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:05.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:05.995+0000 7fb754000640 1 --2- 172.21.15.107:0/3811062664 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb74c102500 0x7fb74c102980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:05.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:05.995+0000 7fb754000640 1 --2- 172.21.15.107:0/3811062664 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb74c101310 0x7fb74c101730 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:05.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:05.995+0000 7fb754000640 1 -- 172.21.15.107:0/3811062664 >> 172.21.15.107:0/3811062664 conn(0x7fb74c0fc8a0 msgr2=0x7fb74c0fecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:57:05.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:05.995+0000 7fb754000640 1 -- 172.21.15.107:0/3811062664 shutdown_connections 2024-03-08T20:57:05.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:05.995+0000 7fb754000640 1 -- 172.21.15.107:0/3811062664 wait complete. 2024-03-08T20:57:05.983 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:05.996+0000 7fb754000640 1 Processor -- start 2024-03-08T20:57:05.983 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:05.996+0000 7fb754000640 1 -- start start 2024-03-08T20:57:05.983 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:05.996+0000 7fb754000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb74c101310 0x7fb74c199790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:57:05.984 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:05.996+0000 7fb754000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb74c102500 0x7fb74c199cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:57:05.984 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:05.996+0000 7fb754000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb74c102ec0 0x7fb74c1a0d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:57:05.984 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:05.996+0000 7fb754000640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb74c10ce50 con 0x7fb74c102ec0 2024-03-08T20:57:05.984 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:05.996+0000 7fb754000640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb74c10ccd0 con 0x7fb74c101310 2024-03-08T20:57:05.984 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:05.996+0000 7fb754000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb74c10cfd0 con 0x7fb74c102500 2024-03-08T20:57:05.984 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:05.997+0000 7fb752000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb74c102ec0 0x7fb74c1a0d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:57:05.984 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:05.997+0000 7fb752000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb74c102ec0 0x7fb74c1a0d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:41416/0 (socket says 172.21.15.107:41416) 2024-03-08T20:57:05.984 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:05.997+0000 7fb752000640 1 -- 172.21.15.107:0/1772797399 learned_addr learned my addr 172.21.15.107:0/1772797399 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T20:57:05.984 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:05.997+0000 7fb751600640 1 --2- 172.21.15.107:0/1772797399 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb74c101310 0x7fb74c199790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:57:05.984 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:05.997+0000 7fb750c00640 1 --2- 172.21.15.107:0/1772797399 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb74c102500 0x7fb74c199cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:57:05.984 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:05.997+0000 7fb752000640 1 -- 172.21.15.107:0/1772797399 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb74c102500 msgr2=0x7fb74c199cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:57:05.984 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:05.997+0000 7fb752000640 1 --2- 172.21.15.107:0/1772797399 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb74c102500 0x7fb74c199cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:05.984 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:05.997+0000 7fb752000640 1 -- 172.21.15.107:0/1772797399 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb74c101310 msgr2=0x7fb74c199790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:57:05.985 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:05.997+0000 7fb752000640 1 --2- 172.21.15.107:0/1772797399 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb74c101310 0x7fb74c199790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:05.985 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:05.997+0000 7fb752000640 1 -- 172.21.15.107:0/1772797399 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb74c1a13b0 con 0x7fb74c102ec0 2024-03-08T20:57:05.985 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:05.997+0000 7fb751600640 1 --2- 172.21.15.107:0/1772797399 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb74c101310 0x7fb74c199790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T20:57:05.985 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:05.997+0000 7fb752000640 1 --2- 172.21.15.107:0/1772797399 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb74c102ec0 0x7fb74c1a0d40 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fb73c00d750 tx=0x7fb73c00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:57:05.985 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:05.998+0000 7fb74a000640 1 -- 172.21.15.107:0/1772797399 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb73c017070 con 0x7fb74c102ec0 2024-03-08T20:57:05.985 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:05.998+0000 7fb754000640 1 -- 172.21.15.107:0/1772797399 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb74c1a16a0 con 0x7fb74c102ec0 2024-03-08T20:57:05.985 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:05.998+0000 7fb74a000640 1 -- 172.21.15.107:0/1772797399 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fb73c0054d0 con 0x7fb74c102ec0 2024-03-08T20:57:05.985 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:05.998+0000 7fb74a000640 1 -- 172.21.15.107:0/1772797399 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb73c004c60 con 0x7fb74c102ec0 2024-03-08T20:57:05.985 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:05.998+0000 7fb754000640 1 -- 172.21.15.107:0/1772797399 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb74c1a1ba0 con 0x7fb74c102ec0 2024-03-08T20:57:05.986 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:05.999+0000 7fb754000640 1 -- 172.21.15.107:0/1772797399 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb74c1a1e00 con 0x7fb74c102ec0 2024-03-08T20:57:05.986 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:06.001+0000 7fb74a000640 1 -- 172.21.15.107:0/1772797399 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) v1 ==== 137291+0+0 (secure 0 0 0) 0x7fb73c004e00 con 0x7fb74c102ec0 2024-03-08T20:57:05.987 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:06.002+0000 7fb74a000640 1 --2- 172.21.15.107:0/1772797399 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fb7300a2800 0x7fb7300a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:57:05.987 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:06.003+0000 7fb74a000640 1 -- 172.21.15.107:0/1772797399 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fb73c0ce0e0 con 0x7fb74c102ec0 2024-03-08T20:57:05.987 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:06.003+0000 7fb751600640 1 --2- 172.21.15.107:0/1772797399 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fb7300a2800 0x7fb7300a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:57:05.988 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:06.003+0000 7fb751600640 1 --2- 172.21.15.107:0/1772797399 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fb7300a2800 0x7fb7300a4cc0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fb74402c630 tx=0x7fb744032510 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:57:05.993 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:06.006+0000 7fb74a000640 1 -- 172.21.15.107:0/1772797399 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7fb73c09b580 con 0x7fb74c102ec0 2024-03-08T20:57:06.232 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:06.247+0000 7fb754000640 1 -- 172.21.15.107:0/1772797399 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb74c1a2190 con 0x7fb74c102ec0 2024-03-08T20:57:06.233 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:06.248+0000 7fb74a000640 1 -- 172.21.15.107:0/1772797399 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+2122 (secure 0 0 0) 0x7fb73c09ae80 con 0x7fb74c102ec0 2024-03-08T20:57:06.233 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-08T20:57:06.233 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/3404762782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":3404762782},{"type":"v1","addr":"172.21.15.204:6833","nonce":3404762782}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":34397,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/1962161726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":1962161726},{"type":"v1","addr":"172.21.15.149:6833","nonce":1962161726}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[]} 2024-03-08T20:57:06.233 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 3 2024-03-08T20:57:06.236 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:06.251+0000 7fb754000640 1 -- 172.21.15.107:0/1772797399 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fb7300a2800 msgr2=0x7fb7300a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:57:06.236 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:06.251+0000 7fb754000640 1 --2- 172.21.15.107:0/1772797399 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fb7300a2800 0x7fb7300a4cc0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fb74402c630 tx=0x7fb744032510 comp rx=0 tx=0).stop 2024-03-08T20:57:06.236 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:06.251+0000 7fb754000640 1 -- 172.21.15.107:0/1772797399 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb74c102ec0 msgr2=0x7fb74c1a0d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:57:06.237 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:06.251+0000 7fb754000640 1 --2- 172.21.15.107:0/1772797399 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb74c102ec0 0x7fb74c1a0d40 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fb73c00d750 tx=0x7fb73c00dc10 comp rx=0 tx=0).stop 2024-03-08T20:57:06.237 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:06.251+0000 7fb754000640 1 -- 172.21.15.107:0/1772797399 shutdown_connections 2024-03-08T20:57:06.237 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:06.251+0000 7fb754000640 1 --2- 172.21.15.107:0/1772797399 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fb7300a2800 0x7fb7300a4cc0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:06.237 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:06.251+0000 7fb754000640 1 --2- 172.21.15.107:0/1772797399 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb74c102ec0 0x7fb74c1a0d40 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:06.237 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:06.251+0000 7fb754000640 1 --2- 172.21.15.107:0/1772797399 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb74c102500 0x7fb74c199cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:06.237 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:06.251+0000 7fb754000640 1 --2- 172.21.15.107:0/1772797399 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb74c101310 0x7fb74c199790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:06.237 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:06.251+0000 7fb754000640 1 -- 172.21.15.107:0/1772797399 >> 172.21.15.107:0/1772797399 conn(0x7fb74c0fc8a0 msgr2=0x7fb74c073520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:57:06.237 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:06.251+0000 7fb754000640 1 -- 172.21.15.107:0/1772797399 shutdown_connections 2024-03-08T20:57:06.237 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:06.251+0000 7fb754000640 1 -- 172.21.15.107:0/1772797399 wait complete. 2024-03-08T20:57:06.613 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:06 smithi107 ceph-mon[31381]: from='client.? 172.21.15.107:0/1772797399' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-08T20:57:06.629 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:06 smithi204 ceph-mon[34935]: from='client.? 172.21.15.107:0/1772797399' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-08T20:57:06.766 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:06 smithi149 ceph-mon[35042]: from='client.? 172.21.15.107:0/1772797399' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-08T20:57:07.631 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:07 smithi204 ceph-mon[34935]: pgmap v335: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-08T20:57:07.653 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:07 smithi107 ceph-mon[31381]: pgmap v335: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-08T20:57:07.766 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:07 smithi149 ceph-mon[35042]: pgmap v335: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-08T20:57:07.955 INFO:journalctl@ceph.mds.j.smithi107.stdout:Mar 08 20:57:07 smithi107 podman[72483]: 2024-03-08 20:57:07.684463988 +0000 UTC m=+0.146139268 container create eee17e8f22b6e13bc0a41e9b669f9c0e67b19dd92698cffab1999ec6f44e8dce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-j, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=rishabh-2024mar8-7d63482) 2024-03-08T20:57:07.955 INFO:journalctl@ceph.mds.j.smithi107.stdout:Mar 08 20:57:07 smithi107 podman[72483]: 2024-03-08 20:57:07.96971695 +0000 UTC m=+0.431392236 container init eee17e8f22b6e13bc0a41e9b669f9c0e67b19dd92698cffab1999ec6f44e8dce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-j, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, ceph=True, RELEASE=rishabh-2024mar8-7d63482, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-03-08T20:57:08.219 INFO:journalctl@ceph.mds.j.smithi107.stdout:Mar 08 20:57:07 smithi107 podman[72483]: 2024-03-08 20:57:07.97974012 +0000 UTC m=+0.441415370 container start eee17e8f22b6e13bc0a41e9b669f9c0e67b19dd92698cffab1999ec6f44e8dce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-j, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, RELEASE=rishabh-2024mar8-7d63482, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-08T20:57:08.219 INFO:journalctl@ceph.mds.j.smithi107.stdout:Mar 08 20:57:08 smithi107 ceph-mds[72502]: -- 172.21.15.107:0/792070463 <== mon.0 v2:172.21.15.107:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55d8cb303a40 con 0x55d8cb2d7000 2024-03-08T20:57:08.219 INFO:journalctl@ceph.mds.j.smithi107.stdout:Mar 08 20:57:08 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-j[72498]: starting mds.j at 2024-03-08T20:57:08.219 INFO:journalctl@ceph.mds.j.smithi107.stdout:Mar 08 20:57:08 smithi107 bash[72483]: eee17e8f22b6e13bc0a41e9b669f9c0e67b19dd92698cffab1999ec6f44e8dce 2024-03-08T20:57:08.220 INFO:journalctl@ceph.mds.j.smithi107.stdout:Mar 08 20:57:08 smithi107 systemd[1]: Started Ceph mds.j for dc20f0f8-dd8c-11ee-95c6-87774f69a715. 2024-03-08T20:57:08.255 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-08T20:57:08.462 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:08.476+0000 7fd31f600640 1 -- 172.21.15.107:0/702077586 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd318104720 msgr2=0x7fd318104ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:57:08.462 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:08.476+0000 7fd31f600640 1 --2- 172.21.15.107:0/702077586 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd318104720 0x7fd318104ba0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fd304009900 tx=0x7fd30402bc30 comp rx=0 tx=0).stop 2024-03-08T20:57:08.462 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:08.477+0000 7fd31f600640 1 -- 172.21.15.107:0/702077586 shutdown_connections 2024-03-08T20:57:08.462 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:08.477+0000 7fd31f600640 1 --2- 172.21.15.107:0/702077586 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd3181050e0 0x7fd31810b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:08.462 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:08.477+0000 7fd31f600640 1 --2- 172.21.15.107:0/702077586 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd318104720 0x7fd318104ba0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:08.462 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:08.477+0000 7fd31f600640 1 --2- 172.21.15.107:0/702077586 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd318103530 0x7fd318103950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:08.462 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:08.477+0000 7fd31f600640 1 -- 172.21.15.107:0/702077586 >> 172.21.15.107:0/702077586 conn(0x7fd3180feae0 msgr2=0x7fd318100f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:57:08.462 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:08.477+0000 7fd31f600640 1 -- 172.21.15.107:0/702077586 shutdown_connections 2024-03-08T20:57:08.463 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:08.477+0000 7fd31f600640 1 -- 172.21.15.107:0/702077586 wait complete. 2024-03-08T20:57:08.464 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:08.478+0000 7fd31f600640 1 Processor -- start 2024-03-08T20:57:08.465 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:08.478+0000 7fd31f600640 1 -- start start 2024-03-08T20:57:08.465 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:08.478+0000 7fd31f600640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd318103530 0x7fd3180790a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:57:08.465 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:08.478+0000 7fd31f600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd318104720 0x7fd3180795e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:57:08.465 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:08.478+0000 7fd31f600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd3181050e0 0x7fd318075c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:57:08.465 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:08.478+0000 7fd31f600640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd31810f0c0 con 0x7fd318104720 2024-03-08T20:57:08.465 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:08.478+0000 7fd31f600640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd31810ef40 con 0x7fd318103530 2024-03-08T20:57:08.465 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:08.478+0000 7fd31f600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd31810f240 con 0x7fd3181050e0 2024-03-08T20:57:08.465 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:08.478+0000 7fd317e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd318104720 0x7fd3180795e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:57:08.465 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:08.478+0000 7fd317e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd318104720 0x7fd3180795e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:41462/0 (socket says 172.21.15.107:41462) 2024-03-08T20:57:08.465 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:08.478+0000 7fd317e00640 1 -- 172.21.15.107:0/1712434695 learned_addr learned my addr 172.21.15.107:0/1712434695 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T20:57:08.465 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:08.479+0000 7fd31d600640 1 --2- 172.21.15.107:0/1712434695 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd3181050e0 0x7fd318075c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:57:08.465 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:08.479+0000 7fd317e00640 1 -- 172.21.15.107:0/1712434695 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd3181050e0 msgr2=0x7fd318075c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:57:08.466 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:08.479+0000 7fd31cc00640 1 --2- 172.21.15.107:0/1712434695 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd318103530 0x7fd3180790a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:57:08.466 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:08.479+0000 7fd317e00640 1 --2- 172.21.15.107:0/1712434695 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd3181050e0 0x7fd318075c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:08.466 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:08.479+0000 7fd317e00640 1 -- 172.21.15.107:0/1712434695 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd318103530 msgr2=0x7fd3180790a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:57:08.466 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:08.479+0000 7fd317e00640 1 --2- 172.21.15.107:0/1712434695 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd318103530 0x7fd3180790a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:08.466 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:08.479+0000 7fd317e00640 1 -- 172.21.15.107:0/1712434695 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3180764d0 con 0x7fd318104720 2024-03-08T20:57:08.466 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:08.479+0000 7fd31d600640 1 --2- 172.21.15.107:0/1712434695 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd3181050e0 0x7fd318075c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T20:57:08.466 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:08.479+0000 7fd31cc00640 1 --2- 172.21.15.107:0/1712434695 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd318103530 0x7fd3180790a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T20:57:08.466 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:08.479+0000 7fd317e00640 1 --2- 172.21.15.107:0/1712434695 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd318104720 0x7fd3180795e0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fd304000c00 tx=0x7fd3040042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:57:08.466 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:08.479+0000 7fd315600640 1 -- 172.21.15.107:0/1712434695 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd304043020 con 0x7fd318104720 2024-03-08T20:57:08.466 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:08.479+0000 7fd315600640 1 -- 172.21.15.107:0/1712434695 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fd3040054c0 con 0x7fd318104720 2024-03-08T20:57:08.466 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:08.479+0000 7fd31f600640 1 -- 172.21.15.107:0/1712434695 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3181a5bc0 con 0x7fd318104720 2024-03-08T20:57:08.466 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:08.479+0000 7fd315600640 1 -- 172.21.15.107:0/1712434695 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd304003d80 con 0x7fd318104720 2024-03-08T20:57:08.466 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:08.480+0000 7fd31f600640 1 -- 172.21.15.107:0/1712434695 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3181a5fc0 con 0x7fd318104720 2024-03-08T20:57:08.467 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:08.481+0000 7fd31f600640 1 -- 172.21.15.107:0/1712434695 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd318068530 con 0x7fd318104720 2024-03-08T20:57:08.467 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:08.483+0000 7fd315600640 1 -- 172.21.15.107:0/1712434695 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) v1 ==== 137291+0+0 (secure 0 0 0) 0x7fd304046020 con 0x7fd318104720 2024-03-08T20:57:08.471 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:08.486+0000 7fd315600640 1 --2- 172.21.15.107:0/1712434695 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fd3000a2880 0x7fd3000a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:57:08.471 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:08.486+0000 7fd315600640 1 -- 172.21.15.107:0/1712434695 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fd3040f0a20 con 0x7fd318104720 2024-03-08T20:57:08.471 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:08.486+0000 7fd31cc00640 1 --2- 172.21.15.107:0/1712434695 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fd3000a2880 0x7fd3000a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:57:08.472 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:08.487+0000 7fd31cc00640 1 --2- 172.21.15.107:0/1712434695 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fd3000a2880 0x7fd3000a4d40 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fd318104580 tx=0x7fd30c005d30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:57:08.474 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:08.490+0000 7fd315600640 1 -- 172.21.15.107:0/1712434695 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7fd30404d020 con 0x7fd318104720 2024-03-08T20:57:08.714 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:08.729+0000 7fd31f600640 1 -- 172.21.15.107:0/1712434695 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd31802ce70 con 0x7fd318104720 2024-03-08T20:57:08.715 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:08.730+0000 7fd315600640 1 -- 172.21.15.107:0/1712434695 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) v1 ==== 93+0+2913 (secure 0 0 0) 0x7fd3040be080 con 0x7fd318104720 2024-03-08T20:57:08.715 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-08T20:57:08.715 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/133582743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":133582743},{"type":"v1","addr":"172.21.15.107:6835","nonce":133582743}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/3404762782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":3404762782},{"type":"v1","addr":"172.21.15.204:6833","nonce":3404762782}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":34397,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/1962161726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":1962161726},{"type":"v1","addr":"172.21.15.149:6833","nonce":1962161726}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[]} 2024-03-08T20:57:08.716 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 4 2024-03-08T20:57:08.718 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:08.733+0000 7fd31f600640 1 -- 172.21.15.107:0/1712434695 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fd3000a2880 msgr2=0x7fd3000a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:57:08.718 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:08.733+0000 7fd31f600640 1 --2- 172.21.15.107:0/1712434695 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fd3000a2880 0x7fd3000a4d40 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fd318104580 tx=0x7fd30c005d30 comp rx=0 tx=0).stop 2024-03-08T20:57:08.718 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:08.733+0000 7fd31f600640 1 -- 172.21.15.107:0/1712434695 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd318104720 msgr2=0x7fd3180795e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:57:08.718 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:08.733+0000 7fd31f600640 1 --2- 172.21.15.107:0/1712434695 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd318104720 0x7fd3180795e0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fd304000c00 tx=0x7fd3040042f0 comp rx=0 tx=0).stop 2024-03-08T20:57:08.718 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:08.733+0000 7fd31f600640 1 -- 172.21.15.107:0/1712434695 shutdown_connections 2024-03-08T20:57:08.718 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:08.733+0000 7fd31f600640 1 --2- 172.21.15.107:0/1712434695 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fd3000a2880 0x7fd3000a4d40 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:08.718 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:08.733+0000 7fd31f600640 1 --2- 172.21.15.107:0/1712434695 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd3181050e0 0x7fd318075c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:08.719 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:08.733+0000 7fd31f600640 1 --2- 172.21.15.107:0/1712434695 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd318104720 0x7fd3180795e0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:08.719 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:08.733+0000 7fd31f600640 1 --2- 172.21.15.107:0/1712434695 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd318103530 0x7fd3180790a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:08.719 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:08.733+0000 7fd31f600640 1 -- 172.21.15.107:0/1712434695 >> 172.21.15.107:0/1712434695 conn(0x7fd3180feae0 msgr2=0x7fd318107af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:57:08.719 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:08.733+0000 7fd31f600640 1 -- 172.21.15.107:0/1712434695 shutdown_connections 2024-03-08T20:57:08.719 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:08.733+0000 7fd31f600640 1 -- 172.21.15.107:0/1712434695 wait complete. 2024-03-08T20:57:09.469 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:09 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:57:09.470 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:09 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:57:09.470 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:09 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:57:09.470 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:09 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-08T20:57:09.470 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:09 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-08T20:57:09.470 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:09 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:57:09.470 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:09 smithi107 ceph-mon[31381]: Deploying daemon mds.d on smithi107 2024-03-08T20:57:09.470 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:09 smithi107 ceph-mon[31381]: pgmap v336: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-08T20:57:09.470 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:09 smithi107 ceph-mon[31381]: mds.? [v2:172.21.15.107:6834/133582743,v1:172.21.15.107:6835/133582743] up:boot 2024-03-08T20:57:09.470 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:09 smithi107 ceph-mon[31381]: fsmap 3 up:standby 2024-03-08T20:57:09.470 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:09 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-08T20:57:09.470 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:09 smithi107 ceph-mon[31381]: from='client.? 172.21.15.107:0/1712434695' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-08T20:57:09.516 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:09 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:57:09.517 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:09 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:57:09.517 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:09 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:57:09.517 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:09 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-08T20:57:09.517 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:09 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-08T20:57:09.517 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:09 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:57:09.517 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:09 smithi149 ceph-mon[35042]: Deploying daemon mds.d on smithi107 2024-03-08T20:57:09.517 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:09 smithi149 ceph-mon[35042]: pgmap v336: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-08T20:57:09.517 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:09 smithi149 ceph-mon[35042]: mds.? [v2:172.21.15.107:6834/133582743,v1:172.21.15.107:6835/133582743] up:boot 2024-03-08T20:57:09.517 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:09 smithi149 ceph-mon[35042]: fsmap 3 up:standby 2024-03-08T20:57:09.517 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:09 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-08T20:57:09.517 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:09 smithi149 ceph-mon[35042]: from='client.? 172.21.15.107:0/1712434695' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-08T20:57:09.630 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:09 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:57:09.630 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:09 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:57:09.630 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:09 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:57:09.630 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:09 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-08T20:57:09.630 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:09 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-08T20:57:09.630 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:09 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:57:09.631 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:09 smithi204 ceph-mon[34935]: Deploying daemon mds.d on smithi107 2024-03-08T20:57:09.631 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:09 smithi204 ceph-mon[34935]: pgmap v336: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-08T20:57:09.631 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:09 smithi204 ceph-mon[34935]: mds.? [v2:172.21.15.107:6834/133582743,v1:172.21.15.107:6835/133582743] up:boot 2024-03-08T20:57:09.631 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:09 smithi204 ceph-mon[34935]: fsmap 3 up:standby 2024-03-08T20:57:09.631 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:09 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-08T20:57:09.631 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:09 smithi204 ceph-mon[34935]: from='client.? 172.21.15.107:0/1712434695' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-08T20:57:10.737 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-08T20:57:11.153 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:11.167+0000 7f3f96a00640 1 -- 172.21.15.107:0/3517867592 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3f90075cb0 msgr2=0x7f3f900760d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:57:11.153 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:11.167+0000 7f3f96a00640 1 --2- 172.21.15.107:0/3517867592 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3f90075cb0 0x7f3f900760d0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f3f80009900 tx=0x7f3f8002bc30 comp rx=0 tx=0).stop 2024-03-08T20:57:11.153 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:11.167+0000 7f3f96a00640 1 -- 172.21.15.107:0/3517867592 shutdown_connections 2024-03-08T20:57:11.153 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:11.167+0000 7f3f96a00640 1 --2- 172.21.15.107:0/3517867592 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3f90077890 0x7f3f90078130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:11.153 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:11.167+0000 7f3f96a00640 1 --2- 172.21.15.107:0/3517867592 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3f90076ef0 0x7f3f90077350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:11.153 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:11.167+0000 7f3f96a00640 1 --2- 172.21.15.107:0/3517867592 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3f90075cb0 0x7f3f900760d0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:11.153 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:11.167+0000 7f3f96a00640 1 -- 172.21.15.107:0/3517867592 >> 172.21.15.107:0/3517867592 conn(0x7f3f900feae0 msgr2=0x7f3f90100f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:57:11.155 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:11.167+0000 7f3f96a00640 1 -- 172.21.15.107:0/3517867592 shutdown_connections 2024-03-08T20:57:11.155 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:11.168+0000 7f3f96a00640 1 -- 172.21.15.107:0/3517867592 wait complete. 2024-03-08T20:57:11.155 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:11.169+0000 7f3f96a00640 1 Processor -- start 2024-03-08T20:57:11.155 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:11.169+0000 7f3f96a00640 1 -- start start 2024-03-08T20:57:11.155 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:11.170+0000 7f3f96a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3f90075cb0 0x7f3f9006fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:57:11.155 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:11.170+0000 7f3f96a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3f90076ef0 0x7f3f90070210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:57:11.155 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:11.170+0000 7f3f96a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3f90077890 0x7f3f9019cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:57:11.155 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:11.170+0000 7f3f96a00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f90113510 con 0x7f3f90077890 2024-03-08T20:57:11.155 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:11.170+0000 7f3f96a00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f90113390 con 0x7f3f90076ef0 2024-03-08T20:57:11.155 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:11.170+0000 7f3f96a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f90113690 con 0x7f3f90075cb0 2024-03-08T20:57:11.156 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:11.170+0000 7f3f94a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3f90077890 0x7f3f9019cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:57:11.156 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:11.170+0000 7f3f94a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3f90077890 0x7f3f9019cc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:41480/0 (socket says 172.21.15.107:41480) 2024-03-08T20:57:11.156 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:11.170+0000 7f3f94a00640 1 -- 172.21.15.107:0/1581777351 learned_addr learned my addr 172.21.15.107:0/1581777351 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T20:57:11.156 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:11.170+0000 7f3f8f400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3f90076ef0 0x7f3f90070210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:57:11.156 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:11.171+0000 7f3f8fe00640 1 --2- 172.21.15.107:0/1581777351 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3f90075cb0 0x7f3f9006fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:57:11.156 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:11.171+0000 7f3f94a00640 1 -- 172.21.15.107:0/1581777351 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3f90075cb0 msgr2=0x7f3f9006fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:57:11.157 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:11.171+0000 7f3f94a00640 1 --2- 172.21.15.107:0/1581777351 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3f90075cb0 0x7f3f9006fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:11.157 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:11.171+0000 7f3f94a00640 1 -- 172.21.15.107:0/1581777351 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3f90076ef0 msgr2=0x7f3f90070210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:57:11.157 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:11.171+0000 7f3f94a00640 1 --2- 172.21.15.107:0/1581777351 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3f90076ef0 0x7f3f90070210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:11.157 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:11.171+0000 7f3f94a00640 1 -- 172.21.15.107:0/1581777351 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f9019d150 con 0x7f3f90077890 2024-03-08T20:57:11.157 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:11.172+0000 7f3f94a00640 1 --2- 172.21.15.107:0/1581777351 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3f90077890 0x7f3f9019cc10 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f3f8800b970 tx=0x7f3f8800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:57:11.158 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:11.173+0000 7f3f8cc00640 1 -- 172.21.15.107:0/1581777351 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f88019070 con 0x7f3f90077890 2024-03-08T20:57:11.158 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:11.173+0000 7f3f8cc00640 1 -- 172.21.15.107:0/1581777351 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f3f88014440 con 0x7f3f90077890 2024-03-08T20:57:11.158 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:11.173+0000 7f3f8cc00640 1 -- 172.21.15.107:0/1581777351 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f880075d0 con 0x7f3f90077890 2024-03-08T20:57:11.158 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:11.174+0000 7f3f96a00640 1 -- 172.21.15.107:0/1581777351 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f9019d440 con 0x7f3f90077890 2024-03-08T20:57:11.159 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:11.174+0000 7f3f96a00640 1 -- 172.21.15.107:0/1581777351 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f9019d9c0 con 0x7f3f90077890 2024-03-08T20:57:11.161 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:11.176+0000 7f3f96a00640 1 -- 172.21.15.107:0/1581777351 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f90068530 con 0x7f3f90077890 2024-03-08T20:57:11.164 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:11.180+0000 7f3f8cc00640 1 -- 172.21.15.107:0/1581777351 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f3f8801c070 con 0x7f3f90077890 2024-03-08T20:57:11.166 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:11.181+0000 7f3f8cc00640 1 --2- 172.21.15.107:0/1581777351 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f3f740a2950 0x7f3f740a4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:57:11.166 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:11.181+0000 7f3f8cc00640 1 -- 172.21.15.107:0/1581777351 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f3f880cdb20 con 0x7f3f90077890 2024-03-08T20:57:11.166 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:11.181+0000 7f3f8fe00640 1 --2- 172.21.15.107:0/1581777351 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f3f740a2950 0x7f3f740a4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:57:11.167 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:11.182+0000 7f3f8fe00640 1 --2- 172.21.15.107:0/1581777351 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f3f740a2950 0x7f3f740a4e10 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f3f80000c00 tx=0x7f3f80005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:57:11.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:11.187+0000 7f3f8cc00640 1 -- 172.21.15.107:0/1581777351 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f3f8809a970 con 0x7f3f90077890 2024-03-08T20:57:11.413 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:11.428+0000 7f3f96a00640 1 -- 172.21.15.107:0/1581777351 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3f9002d260 con 0x7f3f90077890 2024-03-08T20:57:11.413 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:11.429+0000 7f3f8cc00640 1 -- 172.21.15.107:0/1581777351 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) v1 ==== 93+0+2913 (secure 0 0 0) 0x7f3f8809ea10 con 0x7f3f90077890 2024-03-08T20:57:11.414 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-08T20:57:11.414 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/133582743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":133582743},{"type":"v1","addr":"172.21.15.107:6835","nonce":133582743}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/3404762782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":3404762782},{"type":"v1","addr":"172.21.15.204:6833","nonce":3404762782}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":34397,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/1962161726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":1962161726},{"type":"v1","addr":"172.21.15.149:6833","nonce":1962161726}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[]} 2024-03-08T20:57:11.414 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 4 2024-03-08T20:57:11.416 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:11.431+0000 7f3f96a00640 1 -- 172.21.15.107:0/1581777351 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f3f740a2950 msgr2=0x7f3f740a4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:57:11.416 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:11.431+0000 7f3f96a00640 1 --2- 172.21.15.107:0/1581777351 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f3f740a2950 0x7f3f740a4e10 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f3f80000c00 tx=0x7f3f80005fd0 comp rx=0 tx=0).stop 2024-03-08T20:57:11.416 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:11.431+0000 7f3f96a00640 1 -- 172.21.15.107:0/1581777351 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3f90077890 msgr2=0x7f3f9019cc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:57:11.416 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:11.431+0000 7f3f96a00640 1 --2- 172.21.15.107:0/1581777351 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3f90077890 0x7f3f9019cc10 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f3f8800b970 tx=0x7f3f8800be30 comp rx=0 tx=0).stop 2024-03-08T20:57:11.416 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:11.431+0000 7f3f96a00640 1 -- 172.21.15.107:0/1581777351 shutdown_connections 2024-03-08T20:57:11.417 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:11.431+0000 7f3f96a00640 1 --2- 172.21.15.107:0/1581777351 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f3f740a2950 0x7f3f740a4e10 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:11.417 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:11.431+0000 7f3f96a00640 1 --2- 172.21.15.107:0/1581777351 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3f90077890 0x7f3f9019cc10 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:11.417 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:11.431+0000 7f3f96a00640 1 --2- 172.21.15.107:0/1581777351 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3f90076ef0 0x7f3f90070210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:11.417 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:11.431+0000 7f3f96a00640 1 --2- 172.21.15.107:0/1581777351 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3f90075cb0 0x7f3f9006fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:11.417 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:11.431+0000 7f3f96a00640 1 -- 172.21.15.107:0/1581777351 >> 172.21.15.107:0/1581777351 conn(0x7f3f900feae0 msgr2=0x7f3f90100220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:57:11.417 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:11.431+0000 7f3f96a00640 1 -- 172.21.15.107:0/1581777351 shutdown_connections 2024-03-08T20:57:11.417 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:11.432+0000 7f3f96a00640 1 -- 172.21.15.107:0/1581777351 wait complete. 2024-03-08T20:57:11.629 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:11 smithi204 ceph-mon[34935]: pgmap v337: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-08T20:57:11.718 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:11 smithi107 ceph-mon[31381]: pgmap v337: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-08T20:57:11.766 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:11 smithi149 ceph-mon[35042]: pgmap v337: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-08T20:57:12.061 INFO:journalctl@ceph.mds.d.smithi107.stdout:Mar 08 20:57:11 smithi107 podman[72868]: 2024-03-08 20:57:11.772408768 +0000 UTC m=+0.156055944 container create d1bfb094648089fee45743483afde5b3705a8deeee706e70572d47baebad5962 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-d, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=rishabh-2024mar8-7d63482, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-03-08T20:57:12.061 INFO:journalctl@ceph.mds.d.smithi107.stdout:Mar 08 20:57:12 smithi107 podman[72868]: 2024-03-08 20:57:12.076295564 +0000 UTC m=+0.459942746 container init d1bfb094648089fee45743483afde5b3705a8deeee706e70572d47baebad5962 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-d, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=rishabh-2024mar8-7d63482, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-08T20:57:12.354 INFO:journalctl@ceph.mds.d.smithi107.stdout:Mar 08 20:57:12 smithi107 podman[72868]: 2024-03-08 20:57:12.08737608 +0000 UTC m=+0.471023233 container start d1bfb094648089fee45743483afde5b3705a8deeee706e70572d47baebad5962 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-d, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=rishabh-2024mar8-7d63482, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux ) 2024-03-08T20:57:12.355 INFO:journalctl@ceph.mds.d.smithi107.stdout:Mar 08 20:57:12 smithi107 ceph-mds[72887]: -- 172.21.15.107:0/2717601224 <== mon.0 v2:172.21.15.107:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55eb97d7ba40 con 0x55eb97d4ec00 2024-03-08T20:57:12.355 INFO:journalctl@ceph.mds.d.smithi107.stdout:Mar 08 20:57:12 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-d[72883]: starting mds.d at 2024-03-08T20:57:12.355 INFO:journalctl@ceph.mds.d.smithi107.stdout:Mar 08 20:57:12 smithi107 bash[72868]: d1bfb094648089fee45743483afde5b3705a8deeee706e70572d47baebad5962 2024-03-08T20:57:12.355 INFO:journalctl@ceph.mds.d.smithi107.stdout:Mar 08 20:57:12 smithi107 systemd[1]: Started Ceph mds.d for dc20f0f8-dd8c-11ee-95c6-87774f69a715. 2024-03-08T20:57:12.631 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:12 smithi204 ceph-mon[34935]: from='client.? 172.21.15.107:0/1581777351' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-08T20:57:12.631 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:12 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:57:12.631 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:12 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:57:12.631 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:12 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:57:12.631 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:12 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-08T20:57:12.631 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:12 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-08T20:57:12.631 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:12 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:57:12.719 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:12 smithi107 ceph-mon[31381]: from='client.? 172.21.15.107:0/1581777351' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-08T20:57:12.719 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:12 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:57:12.719 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:12 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:57:12.719 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:12 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:57:12.719 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:12 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-08T20:57:12.720 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:12 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-08T20:57:12.720 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:12 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:57:12.767 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:12 smithi149 ceph-mon[35042]: from='client.? 172.21.15.107:0/1581777351' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-08T20:57:12.767 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:12 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:57:12.767 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:12 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:57:12.767 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:12 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:57:12.767 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:12 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-08T20:57:12.767 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:12 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-08T20:57:12.767 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:12 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:57:13.436 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-08T20:57:13.620 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:13 smithi107 ceph-mon[31381]: pgmap v338: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-08T20:57:13.620 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:13 smithi107 ceph-mon[31381]: Deploying daemon mds.e on smithi149 2024-03-08T20:57:13.620 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:13 smithi107 ceph-mon[31381]: mds.? [v2:172.21.15.107:6836/3798982337,v1:172.21.15.107:6837/3798982337] up:boot 2024-03-08T20:57:13.621 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:13 smithi107 ceph-mon[31381]: fsmap 4 up:standby 2024-03-08T20:57:13.621 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:13 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-08T20:57:13.630 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:13 smithi204 ceph-mon[34935]: pgmap v338: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-08T20:57:13.630 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:13 smithi204 ceph-mon[34935]: Deploying daemon mds.e on smithi149 2024-03-08T20:57:13.630 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:13 smithi204 ceph-mon[34935]: mds.? [v2:172.21.15.107:6836/3798982337,v1:172.21.15.107:6837/3798982337] up:boot 2024-03-08T20:57:13.630 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:13 smithi204 ceph-mon[34935]: fsmap 4 up:standby 2024-03-08T20:57:13.630 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:13 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-08T20:57:13.658 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:13.673+0000 7f85abc00640 1 -- 172.21.15.107:0/3937363852 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f85a41094e0 msgr2=0x7f85a410fd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:57:13.659 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:13.673+0000 7f85abc00640 1 --2- 172.21.15.107:0/3937363852 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f85a41094e0 0x7f85a410fd70 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f859c009ca0 tx=0x7f859c02c100 comp rx=0 tx=0).stop 2024-03-08T20:57:13.659 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:13.674+0000 7f85abc00640 1 -- 172.21.15.107:0/3937363852 shutdown_connections 2024-03-08T20:57:13.659 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:13.674+0000 7f85abc00640 1 --2- 172.21.15.107:0/3937363852 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f85a41094e0 0x7f85a410fd70 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:13.659 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:13.674+0000 7f85abc00640 1 --2- 172.21.15.107:0/3937363852 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f85a4108b20 0x7f85a4108fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:13.659 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:13.674+0000 7f85abc00640 1 --2- 172.21.15.107:0/3937363852 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f85a4107930 0x7f85a4107d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:13.659 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:13.674+0000 7f85abc00640 1 -- 172.21.15.107:0/3937363852 >> 172.21.15.107:0/3937363852 conn(0x7f85a4076900 msgr2=0x7f85a4078d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:57:13.659 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:13.674+0000 7f85abc00640 1 -- 172.21.15.107:0/3937363852 shutdown_connections 2024-03-08T20:57:13.659 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:13.674+0000 7f85abc00640 1 -- 172.21.15.107:0/3937363852 wait complete. 2024-03-08T20:57:13.661 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:13.675+0000 7f85abc00640 1 Processor -- start 2024-03-08T20:57:13.661 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:13.675+0000 7f85abc00640 1 -- start start 2024-03-08T20:57:13.662 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:13.675+0000 7f85abc00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f85a4107930 0x7f85a419da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:57:13.662 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:13.675+0000 7f85abc00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f85a4108b20 0x7f85a419df60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:57:13.662 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:13.675+0000 7f85abc00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f85a41094e0 0x7f85a41a4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:57:13.662 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:13.675+0000 7f85abc00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85a4113440 con 0x7f85a41094e0 2024-03-08T20:57:13.662 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:13.675+0000 7f85abc00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85a41132c0 con 0x7f85a4107930 2024-03-08T20:57:13.662 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:13.675+0000 7f85abc00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85a41135c0 con 0x7f85a4108b20 2024-03-08T20:57:13.662 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:13.675+0000 7f85a9c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f85a41094e0 0x7f85a41a4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:57:13.662 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:13.675+0000 7f85a9c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f85a41094e0 0x7f85a41a4fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:41502/0 (socket says 172.21.15.107:41502) 2024-03-08T20:57:13.662 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:13.675+0000 7f85a9c00640 1 -- 172.21.15.107:0/1027198292 learned_addr learned my addr 172.21.15.107:0/1027198292 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T20:57:13.662 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:13.676+0000 7f85a3e00640 1 --2- 172.21.15.107:0/1027198292 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f85a4108b20 0x7f85a419df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:57:13.662 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:13.676+0000 7f85a9c00640 1 -- 172.21.15.107:0/1027198292 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f85a4108b20 msgr2=0x7f85a419df60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:57:13.662 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:13.676+0000 7f85a9c00640 1 --2- 172.21.15.107:0/1027198292 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f85a4108b20 0x7f85a419df60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:13.662 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:13.676+0000 7f85a9c00640 1 -- 172.21.15.107:0/1027198292 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f85a4107930 msgr2=0x7f85a419da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:57:13.663 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:13.676+0000 7f85a9200640 1 --2- 172.21.15.107:0/1027198292 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f85a4107930 0x7f85a419da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:57:13.663 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:13.676+0000 7f85a9c00640 1 --2- 172.21.15.107:0/1027198292 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f85a4107930 0x7f85a419da20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:13.663 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:13.676+0000 7f85a9c00640 1 -- 172.21.15.107:0/1027198292 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85a41a5640 con 0x7f85a41094e0 2024-03-08T20:57:13.663 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:13.676+0000 7f85a9c00640 1 --2- 172.21.15.107:0/1027198292 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f85a41094e0 0x7f85a41a4fd0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f859c02c710 tx=0x7f859c035dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:57:13.663 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:13.676+0000 7f85a1600640 1 -- 172.21.15.107:0/1027198292 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f859c039040 con 0x7f85a41094e0 2024-03-08T20:57:13.663 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:13.676+0000 7f85abc00640 1 -- 172.21.15.107:0/1027198292 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85a41a58d0 con 0x7f85a41094e0 2024-03-08T20:57:13.663 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:13.677+0000 7f85a1600640 1 -- 172.21.15.107:0/1027198292 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f859c044070 con 0x7f85a41094e0 2024-03-08T20:57:13.663 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:13.677+0000 7f85abc00640 1 -- 172.21.15.107:0/1027198292 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85a41a5dc0 con 0x7f85a41094e0 2024-03-08T20:57:13.663 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:13.677+0000 7f85a1600640 1 -- 172.21.15.107:0/1027198292 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f859c004690 con 0x7f85a41094e0 2024-03-08T20:57:13.663 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:13.678+0000 7f85abc00640 1 -- 172.21.15.107:0/1027198292 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85a410ec50 con 0x7f85a41094e0 2024-03-08T20:57:13.664 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:13 smithi149 ceph-mon[35042]: pgmap v338: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-08T20:57:13.664 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:13 smithi149 ceph-mon[35042]: Deploying daemon mds.e on smithi149 2024-03-08T20:57:13.664 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:13 smithi149 ceph-mon[35042]: mds.? [v2:172.21.15.107:6836/3798982337,v1:172.21.15.107:6837/3798982337] up:boot 2024-03-08T20:57:13.665 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:13 smithi149 ceph-mon[35042]: fsmap 4 up:standby 2024-03-08T20:57:13.665 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:13 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-08T20:57:13.665 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:13.680+0000 7f85a1600640 1 -- 172.21.15.107:0/1027198292 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f859c003970 con 0x7f85a41094e0 2024-03-08T20:57:13.666 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:13.681+0000 7f85a1600640 1 --2- 172.21.15.107:0/1027198292 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f858c0a2800 0x7f858c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:57:13.666 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:13.681+0000 7f85a1600640 1 -- 172.21.15.107:0/1027198292 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f859c0f0ff0 con 0x7f85a41094e0 2024-03-08T20:57:13.666 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:13.681+0000 7f85a9200640 1 --2- 172.21.15.107:0/1027198292 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f858c0a2800 0x7f858c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:57:13.667 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:13.682+0000 7f85a9200640 1 --2- 172.21.15.107:0/1027198292 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f858c0a2800 0x7f858c0a4cc0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f85a4108980 tx=0x7f8598005c60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:57:13.670 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:13.685+0000 7f85a1600640 1 -- 172.21.15.107:0/1027198292 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f859c0098b0 con 0x7f85a41094e0 2024-03-08T20:57:13.915 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:13.930+0000 7f85abc00640 1 -- 172.21.15.107:0/1027198292 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f85a402ce70 con 0x7f85a41094e0 2024-03-08T20:57:13.916 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:13.931+0000 7f85a1600640 1 -- 172.21.15.107:0/1027198292 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) v1 ==== 93+0+3707 (secure 0 0 0) 0x7f859c0be530 con 0x7f85a41094e0 2024-03-08T20:57:13.916 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-08T20:57:13.916 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/133582743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":133582743},{"type":"v1","addr":"172.21.15.107:6835","nonce":133582743}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/3798982337","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":3798982337},{"type":"v1","addr":"172.21.15.107:6837","nonce":3798982337}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/3404762782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":3404762782},{"type":"v1","addr":"172.21.15.204:6833","nonce":3404762782}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":34397,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/1962161726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":1962161726},{"type":"v1","addr":"172.21.15.149:6833","nonce":1962161726}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[]} 2024-03-08T20:57:13.917 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 5 2024-03-08T20:57:13.919 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:13.934+0000 7f85abc00640 1 -- 172.21.15.107:0/1027198292 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f858c0a2800 msgr2=0x7f858c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:57:13.919 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:13.934+0000 7f85abc00640 1 --2- 172.21.15.107:0/1027198292 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f858c0a2800 0x7f858c0a4cc0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f85a4108980 tx=0x7f8598005c60 comp rx=0 tx=0).stop 2024-03-08T20:57:13.919 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:13.934+0000 7f85abc00640 1 -- 172.21.15.107:0/1027198292 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f85a41094e0 msgr2=0x7f85a41a4fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:57:13.919 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:13.934+0000 7f85abc00640 1 --2- 172.21.15.107:0/1027198292 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f85a41094e0 0x7f85a41a4fd0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f859c02c710 tx=0x7f859c035dd0 comp rx=0 tx=0).stop 2024-03-08T20:57:13.920 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:13.934+0000 7f85abc00640 1 -- 172.21.15.107:0/1027198292 shutdown_connections 2024-03-08T20:57:13.920 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:13.934+0000 7f85abc00640 1 --2- 172.21.15.107:0/1027198292 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f858c0a2800 0x7f858c0a4cc0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:13.920 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:13.934+0000 7f85abc00640 1 --2- 172.21.15.107:0/1027198292 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f85a41094e0 0x7f85a41a4fd0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:13.920 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:13.934+0000 7f85abc00640 1 --2- 172.21.15.107:0/1027198292 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f85a4108b20 0x7f85a419df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:13.920 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:13.934+0000 7f85abc00640 1 --2- 172.21.15.107:0/1027198292 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f85a4107930 0x7f85a419da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:13.920 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:13.934+0000 7f85abc00640 1 -- 172.21.15.107:0/1027198292 >> 172.21.15.107:0/1027198292 conn(0x7f85a4076900 msgr2=0x7f85a410be20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:57:13.920 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:13.934+0000 7f85abc00640 1 -- 172.21.15.107:0/1027198292 shutdown_connections 2024-03-08T20:57:13.920 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:13.935+0000 7f85abc00640 1 -- 172.21.15.107:0/1027198292 wait complete. 2024-03-08T20:57:15.016 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:14 smithi149 ceph-mon[35042]: from='client.? 172.21.15.107:0/1027198292' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-08T20:57:15.016 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:14 smithi149 ceph-mon[35042]: pgmap v339: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-08T20:57:15.218 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:14 smithi107 ceph-mon[31381]: from='client.? 172.21.15.107:0/1027198292' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-08T20:57:15.218 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:14 smithi107 ceph-mon[31381]: pgmap v339: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-08T20:57:15.330 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:14 smithi204 ceph-mon[34935]: from='client.? 172.21.15.107:0/1027198292' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-08T20:57:15.330 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:14 smithi204 ceph-mon[34935]: pgmap v339: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-08T20:57:15.747 INFO:journalctl@ceph.mds.e.smithi149.stdout:Mar 08 20:57:15 smithi149 systemd[1]: Starting Ceph mds.e for dc20f0f8-dd8c-11ee-95c6-87774f69a715... 2024-03-08T20:57:15.941 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-08T20:57:16.017 INFO:journalctl@ceph.mds.e.smithi149.stdout:Mar 08 20:57:15 smithi149 podman[64986]: 2024-03-08 20:57:15.761365735 +0000 UTC m=+0.127918414 container create 112663c35378aa0d66806c78bb216536c2968fe367b8d25bec33bd87cc111981 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-e, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=rishabh-2024mar8-7d63482, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-08T20:57:16.017 INFO:journalctl@ceph.mds.e.smithi149.stdout:Mar 08 20:57:15 smithi149 podman[64986]: 2024-03-08 20:57:15.669853677 +0000 UTC m=+0.036406329 image pull a272146cb810b1fd4a53627f019472132e30966d5c25b01bfc44a001295f1249 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd 2024-03-08T20:57:16.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:16.189+0000 7f0664a00640 1 -- 172.21.15.107:0/2070991903 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0660102ec0 msgr2=0x7f0660109750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:57:16.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:16.189+0000 7f0664a00640 1 --2- 172.21.15.107:0/2070991903 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0660102ec0 0x7f0660109750 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f0658009ca0 tx=0x7f065802c0f0 comp rx=0 tx=0).stop 2024-03-08T20:57:16.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:16.190+0000 7f0664a00640 1 -- 172.21.15.107:0/2070991903 shutdown_connections 2024-03-08T20:57:16.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:16.190+0000 7f0664a00640 1 --2- 172.21.15.107:0/2070991903 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0660102ec0 0x7f0660109750 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:16.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:16.190+0000 7f0664a00640 1 --2- 172.21.15.107:0/2070991903 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0660102500 0x7f0660102980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:16.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:16.190+0000 7f0664a00640 1 --2- 172.21.15.107:0/2070991903 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0660101310 0x7f0660101730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:16.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:16.190+0000 7f0664a00640 1 -- 172.21.15.107:0/2070991903 >> 172.21.15.107:0/2070991903 conn(0x7f06600fc8a0 msgr2=0x7f06600fecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:57:16.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:16.190+0000 7f0664a00640 1 -- 172.21.15.107:0/2070991903 shutdown_connections 2024-03-08T20:57:16.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:16.190+0000 7f0664a00640 1 -- 172.21.15.107:0/2070991903 wait complete. 2024-03-08T20:57:16.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:16.191+0000 7f0664a00640 1 Processor -- start 2024-03-08T20:57:16.178 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:16.191+0000 7f0664a00640 1 -- start start 2024-03-08T20:57:16.178 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:16.191+0000 7f0664a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0660101310 0x7f06601974b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:57:16.178 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:16.191+0000 7f0664a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0660102500 0x7f06601979f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:57:16.178 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:16.191+0000 7f0664a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0660102ec0 0x7f066019ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:57:16.178 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:16.191+0000 7f0664a00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f066010ce50 con 0x7f0660102ec0 2024-03-08T20:57:16.178 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:16.191+0000 7f0664a00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f066010ccd0 con 0x7f0660101310 2024-03-08T20:57:16.178 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:16.191+0000 7f0664a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f066010cfd0 con 0x7f0660102500 2024-03-08T20:57:16.178 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:16.191+0000 7f065e800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0660102ec0 0x7f066019ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:57:16.178 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:16.191+0000 7f065e800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0660102ec0 0x7f066019ea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:40114/0 (socket says 172.21.15.107:40114) 2024-03-08T20:57:16.178 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:16.191+0000 7f065e800640 1 -- 172.21.15.107:0/3427358286 learned_addr learned my addr 172.21.15.107:0/3427358286 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T20:57:16.178 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:16.191+0000 7f065de00640 1 --2- 172.21.15.107:0/3427358286 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0660101310 0x7f06601974b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:57:16.179 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:16.192+0000 7f065d400640 1 --2- 172.21.15.107:0/3427358286 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0660102500 0x7f06601979f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:57:16.179 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:16.192+0000 7f065e800640 1 -- 172.21.15.107:0/3427358286 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0660102500 msgr2=0x7f06601979f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:57:16.179 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:16.192+0000 7f065e800640 1 --2- 172.21.15.107:0/3427358286 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0660102500 0x7f06601979f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:16.179 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:16.192+0000 7f065e800640 1 -- 172.21.15.107:0/3427358286 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0660101310 msgr2=0x7f06601974b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:57:16.179 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:16.192+0000 7f065e800640 1 --2- 172.21.15.107:0/3427358286 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0660101310 0x7f06601974b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:16.179 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:16.192+0000 7f065e800640 1 -- 172.21.15.107:0/3427358286 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f066019f0d0 con 0x7f0660102ec0 2024-03-08T20:57:16.179 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:16.192+0000 7f065e800640 1 --2- 172.21.15.107:0/3427358286 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0660102ec0 0x7f066019ea60 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f0658032380 tx=0x7f0658032820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:57:16.179 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:16.192+0000 7f0656a00640 1 -- 172.21.15.107:0/3427358286 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0658041020 con 0x7f0660102ec0 2024-03-08T20:57:16.179 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:16.192+0000 7f0656a00640 1 -- 172.21.15.107:0/3427358286 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f0658003e20 con 0x7f0660102ec0 2024-03-08T20:57:16.179 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:16.192+0000 7f0656a00640 1 -- 172.21.15.107:0/3427358286 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0658039040 con 0x7f0660102ec0 2024-03-08T20:57:16.179 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:16.192+0000 7f0664a00640 1 -- 172.21.15.107:0/3427358286 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f066019f360 con 0x7f0660102ec0 2024-03-08T20:57:16.179 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:16.192+0000 7f0664a00640 1 -- 172.21.15.107:0/3427358286 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f066019f850 con 0x7f0660102ec0 2024-03-08T20:57:16.180 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:16.194+0000 7f0664a00640 1 -- 172.21.15.107:0/3427358286 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06601086b0 con 0x7f0660102ec0 2024-03-08T20:57:16.180 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:16.195+0000 7f0656a00640 1 -- 172.21.15.107:0/3427358286 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f0658044020 con 0x7f0660102ec0 2024-03-08T20:57:16.182 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:16.197+0000 7f0656a00640 1 --2- 172.21.15.107:0/3427358286 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f06440a2800 0x7f06440a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:57:16.182 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:16.198+0000 7f0656a00640 1 -- 172.21.15.107:0/3427358286 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f06580f0970 con 0x7f0660102ec0 2024-03-08T20:57:16.182 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:16.198+0000 7f065de00640 1 --2- 172.21.15.107:0/3427358286 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f06440a2800 0x7f06440a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:57:16.183 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:16.198+0000 7f065de00640 1 --2- 172.21.15.107:0/3427358286 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f06440a2800 0x7f06440a4cc0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f0650005fd0 tx=0x7f065000b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:57:16.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:16.202+0000 7f0656a00640 1 -- 172.21.15.107:0/3427358286 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f065804b020 con 0x7f0660102ec0 2024-03-08T20:57:16.433 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:16.447+0000 7f0664a00640 1 -- 172.21.15.107:0/3427358286 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0660061ef0 con 0x7f0660102ec0 2024-03-08T20:57:16.433 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:16.449+0000 7f0656a00640 1 -- 172.21.15.107:0/3427358286 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4498 (secure 0 0 0) 0x7f06580bde10 con 0x7f0660102ec0 2024-03-08T20:57:16.434 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-08T20:57:16.434 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/133582743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":133582743},{"type":"v1","addr":"172.21.15.107:6835","nonce":133582743}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/3798982337","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":3798982337},{"type":"v1","addr":"172.21.15.107:6837","nonce":3798982337}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/3404762782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":3404762782},{"type":"v1","addr":"172.21.15.204:6833","nonce":3404762782}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":34397,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/1962161726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":1962161726},{"type":"v1","addr":"172.21.15.149:6833","nonce":1962161726}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":34415,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/442657066","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":442657066},{"type":"v1","addr":"172.21.15.149:6835","nonce":442657066}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[]} 2024-03-08T20:57:16.434 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 6 2024-03-08T20:57:16.439 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:16.451+0000 7f0664a00640 1 -- 172.21.15.107:0/3427358286 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f06440a2800 msgr2=0x7f06440a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:57:16.439 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:16.451+0000 7f0664a00640 1 --2- 172.21.15.107:0/3427358286 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f06440a2800 0x7f06440a4cc0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f0650005fd0 tx=0x7f065000b390 comp rx=0 tx=0).stop 2024-03-08T20:57:16.439 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:16.451+0000 7f0664a00640 1 -- 172.21.15.107:0/3427358286 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0660102ec0 msgr2=0x7f066019ea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:57:16.439 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:16.451+0000 7f0664a00640 1 --2- 172.21.15.107:0/3427358286 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0660102ec0 0x7f066019ea60 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f0658032380 tx=0x7f0658032820 comp rx=0 tx=0).stop 2024-03-08T20:57:16.439 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:16.451+0000 7f0664a00640 1 -- 172.21.15.107:0/3427358286 shutdown_connections 2024-03-08T20:57:16.440 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:16.451+0000 7f0664a00640 1 --2- 172.21.15.107:0/3427358286 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f06440a2800 0x7f06440a4cc0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:16.440 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:16.451+0000 7f0664a00640 1 --2- 172.21.15.107:0/3427358286 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0660102ec0 0x7f066019ea60 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:16.440 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:16.451+0000 7f0664a00640 1 --2- 172.21.15.107:0/3427358286 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0660102500 0x7f06601979f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:16.440 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:16.451+0000 7f0664a00640 1 --2- 172.21.15.107:0/3427358286 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0660101310 0x7f06601974b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:16.440 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:16.451+0000 7f0664a00640 1 -- 172.21.15.107:0/3427358286 >> 172.21.15.107:0/3427358286 conn(0x7f06600fc8a0 msgr2=0x7f0660105880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:57:16.440 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:16.452+0000 7f0664a00640 1 -- 172.21.15.107:0/3427358286 shutdown_connections 2024-03-08T20:57:16.440 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:16.452+0000 7f0664a00640 1 -- 172.21.15.107:0/3427358286 wait complete. 2024-03-08T20:57:16.517 INFO:journalctl@ceph.mds.e.smithi149.stdout:Mar 08 20:57:16 smithi149 podman[64986]: 2024-03-08 20:57:16.056814376 +0000 UTC m=+0.423367071 container init 112663c35378aa0d66806c78bb216536c2968fe367b8d25bec33bd87cc111981 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-e, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=rishabh-2024mar8-7d63482, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux ) 2024-03-08T20:57:16.517 INFO:journalctl@ceph.mds.e.smithi149.stdout:Mar 08 20:57:16 smithi149 podman[64986]: 2024-03-08 20:57:16.067079805 +0000 UTC m=+0.433632464 container start 112663c35378aa0d66806c78bb216536c2968fe367b8d25bec33bd87cc111981 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-e, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, RELEASE=rishabh-2024mar8-7d63482, org.label-schema.license=GPLv2) 2024-03-08T20:57:16.517 INFO:journalctl@ceph.mds.e.smithi149.stdout:Mar 08 20:57:16 smithi149 ceph-mds[65004]: -- 172.21.15.149:0/3066234007 <== mon.2 v2:172.21.15.149:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x560ed2a2ba40 con 0x560ed29ff000 2024-03-08T20:57:16.517 INFO:journalctl@ceph.mds.e.smithi149.stdout:Mar 08 20:57:16 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-e[65000]: starting mds.e at 2024-03-08T20:57:16.517 INFO:journalctl@ceph.mds.e.smithi149.stdout:Mar 08 20:57:16 smithi149 bash[64986]: 112663c35378aa0d66806c78bb216536c2968fe367b8d25bec33bd87cc111981 2024-03-08T20:57:16.517 INFO:journalctl@ceph.mds.e.smithi149.stdout:Mar 08 20:57:16 smithi149 systemd[1]: Started Ceph mds.e for dc20f0f8-dd8c-11ee-95c6-87774f69a715. 2024-03-08T20:57:17.469 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:17 smithi107 ceph-mon[31381]: mds.? [v2:172.21.15.149:6834/442657066,v1:172.21.15.149:6835/442657066] up:boot 2024-03-08T20:57:17.469 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:17 smithi107 ceph-mon[31381]: fsmap 5 up:standby 2024-03-08T20:57:17.469 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:17 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-08T20:57:17.469 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:17 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:57:17.469 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:17 smithi107 ceph-mon[31381]: pgmap v340: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-08T20:57:17.469 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:17 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:57:17.469 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:17 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:57:17.470 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:17 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-08T20:57:17.470 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:17 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-08T20:57:17.470 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:17 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:57:17.470 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:17 smithi107 ceph-mon[31381]: Deploying daemon mds.f on smithi204 2024-03-08T20:57:17.470 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:17 smithi107 ceph-mon[31381]: from='client.? 172.21.15.107:0/3427358286' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-08T20:57:17.504 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:17 smithi204 ceph-mon[34935]: mds.? [v2:172.21.15.149:6834/442657066,v1:172.21.15.149:6835/442657066] up:boot 2024-03-08T20:57:17.504 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:17 smithi204 ceph-mon[34935]: fsmap 5 up:standby 2024-03-08T20:57:17.504 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:17 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-08T20:57:17.504 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:17 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:57:17.504 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:17 smithi204 ceph-mon[34935]: pgmap v340: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-08T20:57:17.504 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:17 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:57:17.504 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:17 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:57:17.505 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:17 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-08T20:57:17.505 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:17 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-08T20:57:17.505 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:17 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:57:17.505 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:17 smithi204 ceph-mon[34935]: Deploying daemon mds.f on smithi204 2024-03-08T20:57:17.505 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:17 smithi204 ceph-mon[34935]: from='client.? 172.21.15.107:0/3427358286' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-08T20:57:17.516 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:17 smithi149 ceph-mon[35042]: mds.? [v2:172.21.15.149:6834/442657066,v1:172.21.15.149:6835/442657066] up:boot 2024-03-08T20:57:17.517 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:17 smithi149 ceph-mon[35042]: fsmap 5 up:standby 2024-03-08T20:57:17.517 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:17 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-08T20:57:17.517 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:17 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:57:17.517 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:17 smithi149 ceph-mon[35042]: pgmap v340: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-08T20:57:17.517 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:17 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:57:17.517 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:17 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:57:17.517 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:17 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-08T20:57:17.517 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:17 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-08T20:57:17.517 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:17 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:57:17.517 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:17 smithi149 ceph-mon[35042]: Deploying daemon mds.f on smithi204 2024-03-08T20:57:17.518 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:17 smithi149 ceph-mon[35042]: from='client.? 172.21.15.107:0/3427358286' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-08T20:57:18.460 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-08T20:57:18.679 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:18.694+0000 7f28b2e00640 1 -- 172.21.15.107:0/3486439476 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f28ac1050e0 msgr2=0x7f28ac10b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:57:18.679 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:18.694+0000 7f28b2e00640 1 --2- 172.21.15.107:0/3486439476 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f28ac1050e0 0x7f28ac10b970 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f28a4009ca0 tx=0x7f28a402c0f0 comp rx=0 tx=0).stop 2024-03-08T20:57:18.680 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:18.694+0000 7f28b2e00640 1 -- 172.21.15.107:0/3486439476 shutdown_connections 2024-03-08T20:57:18.680 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:18.694+0000 7f28b2e00640 1 --2- 172.21.15.107:0/3486439476 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f28ac1050e0 0x7f28ac10b970 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:18.680 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:18.694+0000 7f28b2e00640 1 --2- 172.21.15.107:0/3486439476 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f28ac104720 0x7f28ac104ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:18.680 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:18.694+0000 7f28b2e00640 1 --2- 172.21.15.107:0/3486439476 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f28ac103530 0x7f28ac103950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:18.680 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:18.694+0000 7f28b2e00640 1 -- 172.21.15.107:0/3486439476 >> 172.21.15.107:0/3486439476 conn(0x7f28ac0feae0 msgr2=0x7f28ac100f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:57:18.680 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:18.694+0000 7f28b2e00640 1 -- 172.21.15.107:0/3486439476 shutdown_connections 2024-03-08T20:57:18.680 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:18.694+0000 7f28b2e00640 1 -- 172.21.15.107:0/3486439476 wait complete. 2024-03-08T20:57:18.682 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:18.695+0000 7f28b2e00640 1 Processor -- start 2024-03-08T20:57:18.682 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:18.695+0000 7f28b2e00640 1 -- start start 2024-03-08T20:57:18.682 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:18.696+0000 7f28b2e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f28ac103530 0x7f28ac19db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:57:18.683 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:18.696+0000 7f28b2e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f28ac104720 0x7f28ac19e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:57:18.683 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:18.696+0000 7f28b2e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f28ac1050e0 0x7f28ac1a5130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:57:18.683 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:18.696+0000 7f28b2e00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28ac10f0d0 con 0x7f28ac104720 2024-03-08T20:57:18.683 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:18.696+0000 7f28b2e00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28ac10ef50 con 0x7f28ac103530 2024-03-08T20:57:18.683 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:18.696+0000 7f28b2e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28ac10f250 con 0x7f28ac1050e0 2024-03-08T20:57:18.683 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:18.696+0000 7f28ab400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f28ac104720 0x7f28ac19e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:57:18.683 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:18.696+0000 7f28ab400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f28ac104720 0x7f28ac19e0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:40138/0 (socket says 172.21.15.107:40138) 2024-03-08T20:57:18.683 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:18.696+0000 7f28ab400640 1 -- 172.21.15.107:0/2667997138 learned_addr learned my addr 172.21.15.107:0/2667997138 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T20:57:18.683 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:18.696+0000 7f28b0e00640 1 --2- 172.21.15.107:0/2667997138 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f28ac1050e0 0x7f28ac1a5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:57:18.683 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:18.696+0000 7f28abe00640 1 --2- 172.21.15.107:0/2667997138 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f28ac103530 0x7f28ac19db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:57:18.683 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:18.696+0000 7f28ab400640 1 -- 172.21.15.107:0/2667997138 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f28ac1050e0 msgr2=0x7f28ac1a5130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:57:18.683 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:18.696+0000 7f28ab400640 1 --2- 172.21.15.107:0/2667997138 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f28ac1050e0 0x7f28ac1a5130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:18.683 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:18.696+0000 7f28ab400640 1 -- 172.21.15.107:0/2667997138 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f28ac103530 msgr2=0x7f28ac19db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:57:18.684 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:18.696+0000 7f28ab400640 1 --2- 172.21.15.107:0/2667997138 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f28ac103530 0x7f28ac19db80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:18.684 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:18.696+0000 7f28ab400640 1 -- 172.21.15.107:0/2667997138 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28ac1a57a0 con 0x7f28ac104720 2024-03-08T20:57:18.684 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:18.696+0000 7f28b0e00640 1 --2- 172.21.15.107:0/2667997138 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f28ac1050e0 0x7f28ac1a5130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T20:57:18.684 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:18.696+0000 7f28abe00640 1 --2- 172.21.15.107:0/2667997138 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f28ac103530 0x7f28ac19db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T20:57:18.684 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:18.697+0000 7f28ab400640 1 --2- 172.21.15.107:0/2667997138 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f28ac104720 0x7f28ac19e0c0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f289800e9f0 tx=0x7f289800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:57:18.684 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:18.697+0000 7f28a8c00640 1 -- 172.21.15.107:0/2667997138 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f289800cd80 con 0x7f28ac104720 2024-03-08T20:57:18.684 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:18.697+0000 7f28a8c00640 1 -- 172.21.15.107:0/2667997138 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f28980108e0 con 0x7f28ac104720 2024-03-08T20:57:18.684 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:18.697+0000 7f28b2e00640 1 -- 172.21.15.107:0/2667997138 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28ac1a5a90 con 0x7f28ac104720 2024-03-08T20:57:18.684 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:18.697+0000 7f28a8c00640 1 -- 172.21.15.107:0/2667997138 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2898018920 con 0x7f28ac104720 2024-03-08T20:57:18.684 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:18.697+0000 7f28b2e00640 1 -- 172.21.15.107:0/2667997138 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28ac1a5fe0 con 0x7f28ac104720 2024-03-08T20:57:18.685 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:18.698+0000 7f28b2e00640 1 -- 172.21.15.107:0/2667997138 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28ac1a6270 con 0x7f28ac104720 2024-03-08T20:57:18.685 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:18.701+0000 7f28a8c00640 1 -- 172.21.15.107:0/2667997138 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f2898010a90 con 0x7f28ac104720 2024-03-08T20:57:18.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:18.702+0000 7f28a8c00640 1 --2- 172.21.15.107:0/2667997138 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f28900a2800 0x7f28900a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:57:18.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:18.702+0000 7f28a8c00640 1 -- 172.21.15.107:0/2667997138 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f28980ceab0 con 0x7f28ac104720 2024-03-08T20:57:18.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:18.702+0000 7f28abe00640 1 --2- 172.21.15.107:0/2667997138 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f28900a2800 0x7f28900a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:57:18.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:18.703+0000 7f28abe00640 1 --2- 172.21.15.107:0/2667997138 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f28900a2800 0x7f28900a4cc0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f289c006fd0 tx=0x7f289c008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:57:18.694 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:18.706+0000 7f28a8c00640 1 -- 172.21.15.107:0/2667997138 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f289809bf50 con 0x7f28ac104720 2024-03-08T20:57:18.928 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:18.943+0000 7f28b2e00640 1 -- 172.21.15.107:0/2667997138 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f28ac1a6600 con 0x7f28ac104720 2024-03-08T20:57:18.929 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:18.944+0000 7f28a8c00640 1 -- 172.21.15.107:0/2667997138 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4498 (secure 0 0 0) 0x7f289809b850 con 0x7f28ac104720 2024-03-08T20:57:18.929 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-08T20:57:18.929 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/133582743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":133582743},{"type":"v1","addr":"172.21.15.107:6835","nonce":133582743}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/3798982337","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":3798982337},{"type":"v1","addr":"172.21.15.107:6837","nonce":3798982337}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/3404762782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":3404762782},{"type":"v1","addr":"172.21.15.204:6833","nonce":3404762782}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":34397,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/1962161726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":1962161726},{"type":"v1","addr":"172.21.15.149:6833","nonce":1962161726}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":34415,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/442657066","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":442657066},{"type":"v1","addr":"172.21.15.149:6835","nonce":442657066}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[]} 2024-03-08T20:57:18.930 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 6 2024-03-08T20:57:18.932 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:18.947+0000 7f28b2e00640 1 -- 172.21.15.107:0/2667997138 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f28900a2800 msgr2=0x7f28900a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:57:18.932 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:18.947+0000 7f28b2e00640 1 --2- 172.21.15.107:0/2667997138 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f28900a2800 0x7f28900a4cc0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f289c006fd0 tx=0x7f289c008040 comp rx=0 tx=0).stop 2024-03-08T20:57:18.932 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:18.947+0000 7f28b2e00640 1 -- 172.21.15.107:0/2667997138 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f28ac104720 msgr2=0x7f28ac19e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:57:18.932 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:18.947+0000 7f28b2e00640 1 --2- 172.21.15.107:0/2667997138 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f28ac104720 0x7f28ac19e0c0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f289800e9f0 tx=0x7f289800eeb0 comp rx=0 tx=0).stop 2024-03-08T20:57:18.932 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:18.947+0000 7f28b2e00640 1 -- 172.21.15.107:0/2667997138 shutdown_connections 2024-03-08T20:57:18.932 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:18.947+0000 7f28b2e00640 1 --2- 172.21.15.107:0/2667997138 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f28900a2800 0x7f28900a4cc0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:18.932 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:18.947+0000 7f28b2e00640 1 --2- 172.21.15.107:0/2667997138 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f28ac1050e0 0x7f28ac1a5130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:18.932 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:18.947+0000 7f28b2e00640 1 --2- 172.21.15.107:0/2667997138 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f28ac104720 0x7f28ac19e0c0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:18.932 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:18.947+0000 7f28b2e00640 1 --2- 172.21.15.107:0/2667997138 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f28ac103530 0x7f28ac19db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:18.932 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:18.947+0000 7f28b2e00640 1 -- 172.21.15.107:0/2667997138 >> 172.21.15.107:0/2667997138 conn(0x7f28ac0feae0 msgr2=0x7f28ac077740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:57:18.932 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:18.947+0000 7f28b2e00640 1 -- 172.21.15.107:0/2667997138 shutdown_connections 2024-03-08T20:57:18.933 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:18.947+0000 7f28b2e00640 1 -- 172.21.15.107:0/2667997138 wait complete. 2024-03-08T20:57:19.630 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:19 smithi204 ceph-mon[34935]: pgmap v341: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-08T20:57:19.630 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:19 smithi204 ceph-mon[34935]: from='client.? 172.21.15.107:0/2667997138' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-08T20:57:19.670 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:19 smithi149 ceph-mon[35042]: pgmap v341: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-08T20:57:19.670 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:19 smithi149 ceph-mon[35042]: from='client.? 172.21.15.107:0/2667997138' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-08T20:57:19.718 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:19 smithi107 ceph-mon[31381]: pgmap v341: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-08T20:57:19.718 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:19 smithi107 ceph-mon[31381]: from='client.? 172.21.15.107:0/2667997138' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-08T20:57:19.946 INFO:journalctl@ceph.mds.f.smithi204.stdout:Mar 08 20:57:19 smithi204 podman[65682]: 2024-03-08 20:57:19.700222748 +0000 UTC m=+0.165829968 container create 05a8d06ffdecbaff4a9e91335b55fc86902670d41b43c5737cd37154a216508f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-f, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=rishabh-2024mar8-7d63482, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD) 2024-03-08T20:57:19.946 INFO:journalctl@ceph.mds.f.smithi204.stdout:Mar 08 20:57:19 smithi204 podman[65682]: 2024-03-08 20:57:19.962226724 +0000 UTC m=+0.427833908 container init 05a8d06ffdecbaff4a9e91335b55fc86902670d41b43c5737cd37154a216508f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-f, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, RELEASE=rishabh-2024mar8-7d63482, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-03-08T20:57:20.379 INFO:journalctl@ceph.mds.f.smithi204.stdout:Mar 08 20:57:19 smithi204 podman[65682]: 2024-03-08 20:57:19.970963983 +0000 UTC m=+0.436571193 container start 05a8d06ffdecbaff4a9e91335b55fc86902670d41b43c5737cd37154a216508f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-f, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=rishabh-2024mar8-7d63482, org.label-schema.license=GPLv2) 2024-03-08T20:57:20.379 INFO:journalctl@ceph.mds.f.smithi204.stdout:Mar 08 20:57:20 smithi204 ceph-mds[65709]: -- 172.21.15.204:0/610112166 <== mon.1 v2:172.21.15.204:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5600d7117a40 con 0x5600d70ea800 2024-03-08T20:57:20.379 INFO:journalctl@ceph.mds.f.smithi204.stdout:Mar 08 20:57:20 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-f[65700]: starting mds.f at 2024-03-08T20:57:20.380 INFO:journalctl@ceph.mds.f.smithi204.stdout:Mar 08 20:57:20 smithi204 bash[65682]: 05a8d06ffdecbaff4a9e91335b55fc86902670d41b43c5737cd37154a216508f 2024-03-08T20:57:20.380 INFO:journalctl@ceph.mds.f.smithi204.stdout:Mar 08 20:57:20 smithi204 systemd[1]: Started Ceph mds.f for dc20f0f8-dd8c-11ee-95c6-87774f69a715. 2024-03-08T20:57:20.952 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-08T20:57:21.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:21.186+0000 7fb9e6e00640 1 -- 172.21.15.107:0/2709778461 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb9e0107930 msgr2=0x7fb9e0107d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:57:21.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:21.186+0000 7fb9e6e00640 1 --2- 172.21.15.107:0/2709778461 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb9e0107930 0x7fb9e0107d50 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fb9d0009900 tx=0x7fb9d002bc30 comp rx=0 tx=0).stop 2024-03-08T20:57:21.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:21.187+0000 7fb9e6e00640 1 -- 172.21.15.107:0/2709778461 shutdown_connections 2024-03-08T20:57:21.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:21.187+0000 7fb9e6e00640 1 --2- 172.21.15.107:0/2709778461 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb9e01094e0 0x7fb9e010fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:21.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:21.187+0000 7fb9e6e00640 1 --2- 172.21.15.107:0/2709778461 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb9e0108b20 0x7fb9e0108fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:21.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:21.187+0000 7fb9e6e00640 1 --2- 172.21.15.107:0/2709778461 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb9e0107930 0x7fb9e0107d50 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:21.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:21.187+0000 7fb9e6e00640 1 -- 172.21.15.107:0/2709778461 >> 172.21.15.107:0/2709778461 conn(0x7fb9e0076900 msgr2=0x7fb9e0078d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:57:21.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:21.187+0000 7fb9e6e00640 1 -- 172.21.15.107:0/2709778461 shutdown_connections 2024-03-08T20:57:21.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:21.187+0000 7fb9e6e00640 1 -- 172.21.15.107:0/2709778461 wait complete. 2024-03-08T20:57:21.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:21.188+0000 7fb9e6e00640 1 Processor -- start 2024-03-08T20:57:21.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:21.188+0000 7fb9e6e00640 1 -- start start 2024-03-08T20:57:21.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:21.188+0000 7fb9e6e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb9e0107930 0x7fb9e019db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:57:21.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:21.188+0000 7fb9e6e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb9e0108b20 0x7fb9e019e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:57:21.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:21.188+0000 7fb9e6e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb9e01094e0 0x7fb9e01a5130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:57:21.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:21.188+0000 7fb9e6e00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9e0113480 con 0x7fb9e0107930 2024-03-08T20:57:21.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:21.188+0000 7fb9e6e00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9e0113300 con 0x7fb9e0108b20 2024-03-08T20:57:21.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:21.188+0000 7fb9e6e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9e0113600 con 0x7fb9e01094e0 2024-03-08T20:57:21.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:21.189+0000 7fb9dfe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb9e0107930 0x7fb9e019db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:57:21.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:21.189+0000 7fb9dfe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb9e0107930 0x7fb9e019db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:40152/0 (socket says 172.21.15.107:40152) 2024-03-08T20:57:21.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:21.189+0000 7fb9dfe00640 1 -- 172.21.15.107:0/3939022557 learned_addr learned my addr 172.21.15.107:0/3939022557 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T20:57:21.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:21.189+0000 7fb9df400640 1 --2- 172.21.15.107:0/3939022557 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb9e0108b20 0x7fb9e019e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:57:21.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:21.189+0000 7fb9e4e00640 1 --2- 172.21.15.107:0/3939022557 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb9e01094e0 0x7fb9e01a5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:57:21.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:21.189+0000 7fb9dfe00640 1 -- 172.21.15.107:0/3939022557 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb9e01094e0 msgr2=0x7fb9e01a5130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:57:21.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:21.189+0000 7fb9dfe00640 1 --2- 172.21.15.107:0/3939022557 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb9e01094e0 0x7fb9e01a5130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:21.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:21.189+0000 7fb9dfe00640 1 -- 172.21.15.107:0/3939022557 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb9e0108b20 msgr2=0x7fb9e019e0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:57:21.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:21.189+0000 7fb9dfe00640 1 --2- 172.21.15.107:0/3939022557 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb9e0108b20 0x7fb9e019e0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:21.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:21.189+0000 7fb9dfe00640 1 -- 172.21.15.107:0/3939022557 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9e01a57a0 con 0x7fb9e0107930 2024-03-08T20:57:21.176 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:21.190+0000 7fb9dfe00640 1 --2- 172.21.15.107:0/3939022557 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb9e0107930 0x7fb9e019db80 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fb9d0005ab0 tx=0x7fb9d00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:57:21.176 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:21.190+0000 7fb9dcc00640 1 -- 172.21.15.107:0/3939022557 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9d0043020 con 0x7fb9e0107930 2024-03-08T20:57:21.176 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:21.190+0000 7fb9dcc00640 1 -- 172.21.15.107:0/3939022557 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fb9d00054c0 con 0x7fb9e0107930 2024-03-08T20:57:21.176 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:21.190+0000 7fb9dcc00640 1 -- 172.21.15.107:0/3939022557 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9d0003d10 con 0x7fb9e0107930 2024-03-08T20:57:21.176 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:21.190+0000 7fb9e6e00640 1 -- 172.21.15.107:0/3939022557 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9e01a5a30 con 0x7fb9e0107930 2024-03-08T20:57:21.176 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:21.190+0000 7fb9e6e00640 1 -- 172.21.15.107:0/3939022557 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9e01a5f30 con 0x7fb9e0107930 2024-03-08T20:57:21.176 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:21.191+0000 7fb9e6e00640 1 -- 172.21.15.107:0/3939022557 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9e010eca0 con 0x7fb9e0107930 2024-03-08T20:57:21.178 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:21.193+0000 7fb9dcc00640 1 -- 172.21.15.107:0/3939022557 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) v1 ==== 137291+0+0 (secure 0 0 0) 0x7fb9d0046020 con 0x7fb9e0107930 2024-03-08T20:57:21.179 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:21.194+0000 7fb9dcc00640 1 --2- 172.21.15.107:0/3939022557 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fb9c40a2880 0x7fb9c40a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:57:21.179 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:21.195+0000 7fb9dcc00640 1 -- 172.21.15.107:0/3939022557 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fb9d00f0bf0 con 0x7fb9e0107930 2024-03-08T20:57:21.179 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:21.195+0000 7fb9df400640 1 --2- 172.21.15.107:0/3939022557 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fb9c40a2880 0x7fb9c40a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:57:21.180 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:21.196+0000 7fb9df400640 1 --2- 172.21.15.107:0/3939022557 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fb9c40a2880 0x7fb9c40a4d40 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fb9cc005fd0 tx=0x7fb9cc005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:57:21.184 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:21.200+0000 7fb9dcc00640 1 -- 172.21.15.107:0/3939022557 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7fb9d004d020 con 0x7fb9e0107930 2024-03-08T20:57:21.422 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:21.437+0000 7fb9e6e00640 1 -- 172.21.15.107:0/3939022557 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb9e002ce70 con 0x7fb9e0107930 2024-03-08T20:57:21.423 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:21.438+0000 7fb9dcc00640 1 -- 172.21.15.107:0/3939022557 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+5292 (secure 0 0 0) 0x7fb9d00be1b0 con 0x7fb9e0107930 2024-03-08T20:57:21.423 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 7 2024-03-08T20:57:21.424 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-08T20:57:21.424 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/133582743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":133582743},{"type":"v1","addr":"172.21.15.107:6835","nonce":133582743}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/3798982337","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":3798982337},{"type":"v1","addr":"172.21.15.107:6837","nonce":3798982337}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/3404762782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":3404762782},{"type":"v1","addr":"172.21.15.204:6833","nonce":3404762782}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24430,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/2072518651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":2072518651},{"type":"v1","addr":"172.21.15.204:6835","nonce":2072518651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":34397,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/1962161726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":1962161726},{"type":"v1","addr":"172.21.15.149:6833","nonce":1962161726}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":34415,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/442657066","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":442657066},{"type":"v1","addr":"172.21.15.149:6835","nonce":442657066}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[]} 2024-03-08T20:57:21.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:21.441+0000 7fb9e6e00640 1 -- 172.21.15.107:0/3939022557 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fb9c40a2880 msgr2=0x7fb9c40a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:57:21.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:21.441+0000 7fb9e6e00640 1 --2- 172.21.15.107:0/3939022557 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fb9c40a2880 0x7fb9c40a4d40 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fb9cc005fd0 tx=0x7fb9cc005850 comp rx=0 tx=0).stop 2024-03-08T20:57:21.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:21.441+0000 7fb9e6e00640 1 -- 172.21.15.107:0/3939022557 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb9e0107930 msgr2=0x7fb9e019db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:57:21.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:21.441+0000 7fb9e6e00640 1 --2- 172.21.15.107:0/3939022557 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb9e0107930 0x7fb9e019db80 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fb9d0005ab0 tx=0x7fb9d00042f0 comp rx=0 tx=0).stop 2024-03-08T20:57:21.427 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:21.441+0000 7fb9e6e00640 1 -- 172.21.15.107:0/3939022557 shutdown_connections 2024-03-08T20:57:21.427 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:21.441+0000 7fb9e6e00640 1 --2- 172.21.15.107:0/3939022557 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fb9c40a2880 0x7fb9c40a4d40 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:21.427 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:21.441+0000 7fb9e6e00640 1 --2- 172.21.15.107:0/3939022557 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb9e01094e0 0x7fb9e01a5130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:21.427 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:21.441+0000 7fb9e6e00640 1 --2- 172.21.15.107:0/3939022557 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb9e0108b20 0x7fb9e019e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:21.427 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:21.441+0000 7fb9e6e00640 1 --2- 172.21.15.107:0/3939022557 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb9e0107930 0x7fb9e019db80 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:21.427 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:21.441+0000 7fb9e6e00640 1 -- 172.21.15.107:0/3939022557 >> 172.21.15.107:0/3939022557 conn(0x7fb9e0076900 msgr2=0x7fb9e010be70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:57:21.427 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:21.441+0000 7fb9e6e00640 1 -- 172.21.15.107:0/3939022557 shutdown_connections 2024-03-08T20:57:21.427 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:21.441+0000 7fb9e6e00640 1 -- 172.21.15.107:0/3939022557 wait complete. 2024-03-08T20:57:21.441 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:21 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:57:21.442 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:21 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:57:21.442 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:21 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:57:21.442 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:21 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-08T20:57:21.442 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:21 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-08T20:57:21.442 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:21 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:57:21.442 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:21 smithi107 ceph-mon[31381]: Deploying daemon mds.c on smithi204 2024-03-08T20:57:21.442 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:21 smithi107 ceph-mon[31381]: pgmap v342: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-08T20:57:21.442 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:21 smithi107 ceph-mon[31381]: mds.? [v2:172.21.15.204:6834/2072518651,v1:172.21.15.204:6835/2072518651] up:boot 2024-03-08T20:57:21.442 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:21 smithi107 ceph-mon[31381]: fsmap 6 up:standby 2024-03-08T20:57:21.442 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:21 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-08T20:57:21.516 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:21 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:57:21.517 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:21 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:57:21.517 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:21 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:57:21.517 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:21 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-08T20:57:21.517 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:21 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-08T20:57:21.517 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:21 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:57:21.517 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:21 smithi149 ceph-mon[35042]: Deploying daemon mds.c on smithi204 2024-03-08T20:57:21.517 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:21 smithi149 ceph-mon[35042]: pgmap v342: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-08T20:57:21.517 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:21 smithi149 ceph-mon[35042]: mds.? [v2:172.21.15.204:6834/2072518651,v1:172.21.15.204:6835/2072518651] up:boot 2024-03-08T20:57:21.517 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:21 smithi149 ceph-mon[35042]: fsmap 6 up:standby 2024-03-08T20:57:21.517 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:21 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-08T20:57:21.518 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:21 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:57:21.518 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:21 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:57:21.518 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:21 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:57:21.518 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:21 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-08T20:57:21.518 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:21 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-08T20:57:21.518 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:21 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:57:21.518 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:21 smithi204 ceph-mon[34935]: Deploying daemon mds.c on smithi204 2024-03-08T20:57:21.518 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:21 smithi204 ceph-mon[34935]: pgmap v342: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-08T20:57:21.518 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:21 smithi204 ceph-mon[34935]: mds.? [v2:172.21.15.204:6834/2072518651,v1:172.21.15.204:6835/2072518651] up:boot 2024-03-08T20:57:21.518 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:21 smithi204 ceph-mon[34935]: fsmap 6 up:standby 2024-03-08T20:57:21.518 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:21 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-08T20:57:22.379 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:22 smithi204 ceph-mon[34935]: from='client.? 172.21.15.107:0/3939022557' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-08T20:57:22.516 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:22 smithi149 ceph-mon[35042]: from='client.? 172.21.15.107:0/3939022557' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-08T20:57:22.718 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:22 smithi107 ceph-mon[31381]: from='client.? 172.21.15.107:0/3939022557' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-08T20:57:23.446 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-08T20:57:23.516 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:23 smithi149 ceph-mon[35042]: pgmap v343: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-08T20:57:23.533 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:23 smithi107 ceph-mon[31381]: pgmap v343: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-08T20:57:23.629 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:23 smithi204 ceph-mon[34935]: pgmap v343: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-08T20:57:23.629 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 20:57:23 smithi204 systemd[1]: Starting Ceph mds.c for dc20f0f8-dd8c-11ee-95c6-87774f69a715... 2024-03-08T20:57:23.670 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:23.685+0000 7f2e7ee00640 1 -- 172.21.15.107:0/3947931286 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2e78075330 msgr2=0x7f2e78073780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:57:23.670 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:23.685+0000 7f2e7ee00640 1 --2- 172.21.15.107:0/3947931286 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2e78075330 0x7f2e78073780 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f2e68009900 tx=0x7f2e6802bc30 comp rx=0 tx=0).stop 2024-03-08T20:57:23.670 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:23.685+0000 7f2e7ee00640 1 -- 172.21.15.107:0/3947931286 shutdown_connections 2024-03-08T20:57:23.670 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:23.685+0000 7f2e7ee00640 1 --2- 172.21.15.107:0/3947931286 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2e781091d0 0x7f2e7810b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:23.670 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:23.685+0000 7f2e7ee00640 1 --2- 172.21.15.107:0/3947931286 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2e78073d50 0x7f2e780741d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:23.670 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:23.685+0000 7f2e7ee00640 1 --2- 172.21.15.107:0/3947931286 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2e78075330 0x7f2e78073780 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:23.671 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:23.685+0000 7f2e7ee00640 1 -- 172.21.15.107:0/3947931286 >> 172.21.15.107:0/3947931286 conn(0x7f2e780fc8a0 msgr2=0x7f2e780fecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:57:23.671 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:23.685+0000 7f2e7ee00640 1 -- 172.21.15.107:0/3947931286 shutdown_connections 2024-03-08T20:57:23.671 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:23.685+0000 7f2e7ee00640 1 -- 172.21.15.107:0/3947931286 wait complete. 2024-03-08T20:57:23.672 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:23.686+0000 7f2e7ee00640 1 Processor -- start 2024-03-08T20:57:23.672 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:23.686+0000 7f2e7ee00640 1 -- start start 2024-03-08T20:57:23.672 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:23.687+0000 7f2e7ee00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2e78073d50 0x7f2e781996c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:57:23.672 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:23.687+0000 7f2e7ee00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2e78075330 0x7f2e78199c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:57:23.672 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:23.687+0000 7f2e7ee00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2e781091d0 0x7f2e781a0c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:57:23.673 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:23.687+0000 7f2e7ee00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e7810f050 con 0x7f2e78073d50 2024-03-08T20:57:23.673 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:23.687+0000 7f2e7ee00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e7810eed0 con 0x7f2e781091d0 2024-03-08T20:57:23.673 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:23.687+0000 7f2e7ee00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e7810f1d0 con 0x7f2e78075330 2024-03-08T20:57:23.673 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:23.687+0000 7f2e77e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2e78073d50 0x7f2e781996c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:57:23.673 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:23.687+0000 7f2e77e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2e78073d50 0x7f2e781996c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:40170/0 (socket says 172.21.15.107:40170) 2024-03-08T20:57:23.673 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:23.687+0000 7f2e77e00640 1 -- 172.21.15.107:0/2876469708 learned_addr learned my addr 172.21.15.107:0/2876469708 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T20:57:23.673 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:23.687+0000 7f2e77400640 1 --2- 172.21.15.107:0/2876469708 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2e78075330 0x7f2e78199c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:57:23.673 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:23.687+0000 7f2e77e00640 1 -- 172.21.15.107:0/2876469708 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2e78075330 msgr2=0x7f2e78199c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:57:23.673 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:23.687+0000 7f2e77e00640 1 --2- 172.21.15.107:0/2876469708 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2e78075330 0x7f2e78199c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:23.673 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:23.687+0000 7f2e77e00640 1 -- 172.21.15.107:0/2876469708 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2e781091d0 msgr2=0x7f2e781a0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:57:23.673 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:23.687+0000 7f2e7ce00640 1 --2- 172.21.15.107:0/2876469708 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2e781091d0 0x7f2e781a0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:57:23.673 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:23.687+0000 7f2e77e00640 1 --2- 172.21.15.107:0/2876469708 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2e781091d0 0x7f2e781a0c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:23.673 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:23.687+0000 7f2e77e00640 1 -- 172.21.15.107:0/2876469708 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e781a12e0 con 0x7f2e78073d50 2024-03-08T20:57:23.673 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:23.688+0000 7f2e77e00640 1 --2- 172.21.15.107:0/2876469708 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2e78073d50 0x7f2e781996c0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f2e68000c00 tx=0x7f2e680042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:57:23.674 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:23.688+0000 7f2e74c00640 1 -- 172.21.15.107:0/2876469708 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e68043020 con 0x7f2e78073d50 2024-03-08T20:57:23.674 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:23.688+0000 7f2e74c00640 1 -- 172.21.15.107:0/2876469708 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f2e680054c0 con 0x7f2e78073d50 2024-03-08T20:57:23.675 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:23.688+0000 7f2e74c00640 1 -- 172.21.15.107:0/2876469708 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e68003d70 con 0x7f2e78073d50 2024-03-08T20:57:23.675 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:23.688+0000 7f2e7ee00640 1 -- 172.21.15.107:0/2876469708 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e781a1570 con 0x7f2e78073d50 2024-03-08T20:57:23.675 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:23.688+0000 7f2e7ee00640 1 -- 172.21.15.107:0/2876469708 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e781a1b10 con 0x7f2e78073d50 2024-03-08T20:57:23.675 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:23.689+0000 7f2e7ee00640 1 -- 172.21.15.107:0/2876469708 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e78107f00 con 0x7f2e78073d50 2024-03-08T20:57:23.675 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:23.691+0000 7f2e74c00640 1 -- 172.21.15.107:0/2876469708 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f2e68046020 con 0x7f2e78073d50 2024-03-08T20:57:23.677 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:23.692+0000 7f2e74c00640 1 --2- 172.21.15.107:0/2876469708 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f2e5c0a2880 0x7f2e5c0a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:57:23.677 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:23.693+0000 7f2e74c00640 1 -- 172.21.15.107:0/2876469708 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f2e680f0c10 con 0x7f2e78073d50 2024-03-08T20:57:23.677 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:23.693+0000 7f2e77400640 1 --2- 172.21.15.107:0/2876469708 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f2e5c0a2880 0x7f2e5c0a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:57:23.678 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:23.693+0000 7f2e77400640 1 --2- 172.21.15.107:0/2876469708 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f2e5c0a2880 0x7f2e5c0a4d40 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f2e64005e10 tx=0x7f2e64005da0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:57:23.681 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:23.696+0000 7f2e74c00640 1 -- 172.21.15.107:0/2876469708 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f2e6804d020 con 0x7f2e78073d50 2024-03-08T20:57:23.879 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 20:57:23 smithi204 podman[65993]: 2024-03-08 20:57:23.54955878 +0000 UTC m=+0.034248855 image pull a272146cb810b1fd4a53627f019472132e30966d5c25b01bfc44a001295f1249 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd 2024-03-08T20:57:23.879 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 20:57:23 smithi204 podman[65993]: 2024-03-08 20:57:23.673680049 +0000 UTC m=+0.158370162 container create b889757eaf94a48745a7e36278da2e4be29ea539739481d5951faa7465392322 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=rishabh-2024mar8-7d63482, maintainer=Guillaume Abrioux ) 2024-03-08T20:57:23.917 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:23.933+0000 7f2e7ee00640 1 -- 172.21.15.107:0/2876469708 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2e7802ce70 con 0x7f2e78073d50 2024-03-08T20:57:23.919 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:23.934+0000 7f2e74c00640 1 -- 172.21.15.107:0/2876469708 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+5292 (secure 0 0 0) 0x7f2e680be1d0 con 0x7f2e78073d50 2024-03-08T20:57:23.919 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-08T20:57:23.919 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/133582743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":133582743},{"type":"v1","addr":"172.21.15.107:6835","nonce":133582743}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/3798982337","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":3798982337},{"type":"v1","addr":"172.21.15.107:6837","nonce":3798982337}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/3404762782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":3404762782},{"type":"v1","addr":"172.21.15.204:6833","nonce":3404762782}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24430,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/2072518651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":2072518651},{"type":"v1","addr":"172.21.15.204:6835","nonce":2072518651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":34397,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/1962161726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":1962161726},{"type":"v1","addr":"172.21.15.149:6833","nonce":1962161726}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":34415,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/442657066","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":442657066},{"type":"v1","addr":"172.21.15.149:6835","nonce":442657066}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[]} 2024-03-08T20:57:23.919 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 7 2024-03-08T20:57:23.922 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:23.936+0000 7f2e7ee00640 1 -- 172.21.15.107:0/2876469708 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f2e5c0a2880 msgr2=0x7f2e5c0a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:57:23.922 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:23.936+0000 7f2e7ee00640 1 --2- 172.21.15.107:0/2876469708 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f2e5c0a2880 0x7f2e5c0a4d40 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f2e64005e10 tx=0x7f2e64005da0 comp rx=0 tx=0).stop 2024-03-08T20:57:23.922 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:23.936+0000 7f2e7ee00640 1 -- 172.21.15.107:0/2876469708 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2e78073d50 msgr2=0x7f2e781996c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:57:23.922 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:23.936+0000 7f2e7ee00640 1 --2- 172.21.15.107:0/2876469708 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2e78073d50 0x7f2e781996c0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f2e68000c00 tx=0x7f2e680042f0 comp rx=0 tx=0).stop 2024-03-08T20:57:23.922 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:23.937+0000 7f2e7ee00640 1 -- 172.21.15.107:0/2876469708 shutdown_connections 2024-03-08T20:57:23.922 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:23.937+0000 7f2e7ee00640 1 --2- 172.21.15.107:0/2876469708 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f2e5c0a2880 0x7f2e5c0a4d40 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:23.922 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:23.937+0000 7f2e7ee00640 1 --2- 172.21.15.107:0/2876469708 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2e781091d0 0x7f2e781a0c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:23.922 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:23.937+0000 7f2e7ee00640 1 --2- 172.21.15.107:0/2876469708 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2e78075330 0x7f2e78199c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:23.922 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:23.937+0000 7f2e7ee00640 1 --2- 172.21.15.107:0/2876469708 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2e78073d50 0x7f2e781996c0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:23.922 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:23.937+0000 7f2e7ee00640 1 -- 172.21.15.107:0/2876469708 >> 172.21.15.107:0/2876469708 conn(0x7f2e780fc8a0 msgr2=0x7f2e781050d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:57:23.922 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:23.937+0000 7f2e7ee00640 1 -- 172.21.15.107:0/2876469708 shutdown_connections 2024-03-08T20:57:23.922 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:23.937+0000 7f2e7ee00640 1 -- 172.21.15.107:0/2876469708 wait complete. 2024-03-08T20:57:24.379 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:24 smithi204 ceph-mon[34935]: from='client.? 172.21.15.107:0/2876469708' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-08T20:57:24.380 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 20:57:24 smithi204 podman[65993]: 2024-03-08 20:57:24.015384327 +0000 UTC m=+0.500074447 container init b889757eaf94a48745a7e36278da2e4be29ea539739481d5951faa7465392322 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, RELEASE=rishabh-2024mar8-7d63482, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-03-08T20:57:24.380 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 20:57:24 smithi204 podman[65993]: 2024-03-08 20:57:24.025201713 +0000 UTC m=+0.509891833 container start b889757eaf94a48745a7e36278da2e4be29ea539739481d5951faa7465392322 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=rishabh-2024mar8-7d63482, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-03-08T20:57:24.380 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 20:57:24 smithi204 ceph-mds[66012]: -- 172.21.15.204:0/1781913859 <== mon.1 v2:172.21.15.204:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5569260e9a40 con 0x5569260bc800 2024-03-08T20:57:24.380 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 20:57:24 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[66008]: starting mds.c at 2024-03-08T20:57:24.380 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 20:57:24 smithi204 bash[65993]: b889757eaf94a48745a7e36278da2e4be29ea539739481d5951faa7465392322 2024-03-08T20:57:24.380 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 20:57:24 smithi204 systemd[1]: Started Ceph mds.c for dc20f0f8-dd8c-11ee-95c6-87774f69a715. 2024-03-08T20:57:24.516 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:24 smithi149 ceph-mon[35042]: from='client.? 172.21.15.107:0/2876469708' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-08T20:57:24.718 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:24 smithi107 ceph-mon[31381]: from='client.? 172.21.15.107:0/2876469708' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-08T20:57:25.517 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:25 smithi149 ceph-mon[35042]: mds.? [v2:172.21.15.204:6836/42644905,v1:172.21.15.204:6837/42644905] up:boot 2024-03-08T20:57:25.517 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:25 smithi149 ceph-mon[35042]: fsmap 7 up:standby 2024-03-08T20:57:25.517 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:25 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-08T20:57:25.517 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:25 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:57:25.517 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:25 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:57:25.517 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:25 smithi149 ceph-mon[35042]: pgmap v344: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-08T20:57:25.517 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:25 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:57:25.517 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:25 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-08T20:57:25.517 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:25 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-08T20:57:25.518 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:25 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:57:25.518 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:25 smithi149 ceph-mon[35042]: Deploying daemon mds.h on smithi149 2024-03-08T20:57:25.617 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:25 smithi107 ceph-mon[31381]: mds.? [v2:172.21.15.204:6836/42644905,v1:172.21.15.204:6837/42644905] up:boot 2024-03-08T20:57:25.617 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:25 smithi107 ceph-mon[31381]: fsmap 7 up:standby 2024-03-08T20:57:25.618 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:25 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-08T20:57:25.618 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:25 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:57:25.618 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:25 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:57:25.618 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:25 smithi107 ceph-mon[31381]: pgmap v344: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-08T20:57:25.618 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:25 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:57:25.618 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:25 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-08T20:57:25.618 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:25 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-08T20:57:25.618 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:25 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:57:25.618 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:25 smithi107 ceph-mon[31381]: Deploying daemon mds.h on smithi149 2024-03-08T20:57:25.630 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:25 smithi204 ceph-mon[34935]: mds.? [v2:172.21.15.204:6836/42644905,v1:172.21.15.204:6837/42644905] up:boot 2024-03-08T20:57:25.630 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:25 smithi204 ceph-mon[34935]: fsmap 7 up:standby 2024-03-08T20:57:25.630 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:25 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-08T20:57:25.630 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:25 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:57:25.630 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:25 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:57:25.630 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:25 smithi204 ceph-mon[34935]: pgmap v344: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-08T20:57:25.630 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:25 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:57:25.630 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:25 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-08T20:57:25.630 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:25 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-08T20:57:25.630 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:25 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:57:25.631 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:25 smithi204 ceph-mon[34935]: Deploying daemon mds.h on smithi149 2024-03-08T20:57:25.940 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-08T20:57:26.159 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:26.175+0000 7f105ac00640 1 -- 172.21.15.107:0/1711963979 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1054108940 msgr2=0x7f1054108dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:57:26.159 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:26.175+0000 7f105ac00640 1 --2- 172.21.15.107:0/1711963979 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1054108940 0x7f1054108dc0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f1040009ca0 tx=0x7f104002c0f0 comp rx=0 tx=0).stop 2024-03-08T20:57:26.159 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:26.175+0000 7f105ac00640 1 -- 172.21.15.107:0/1711963979 shutdown_connections 2024-03-08T20:57:26.160 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:26.175+0000 7f105ac00640 1 --2- 172.21.15.107:0/1711963979 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1054109300 0x7f105410fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:26.160 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:26.175+0000 7f105ac00640 1 --2- 172.21.15.107:0/1711963979 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1054108940 0x7f1054108dc0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:26.160 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:26.175+0000 7f105ac00640 1 --2- 172.21.15.107:0/1711963979 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1054107750 0x7f1054107b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:26.160 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:26.175+0000 7f105ac00640 1 -- 172.21.15.107:0/1711963979 >> 172.21.15.107:0/1711963979 conn(0x7f1054076700 msgr2=0x7f1054078b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:57:26.160 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:26.175+0000 7f105ac00640 1 -- 172.21.15.107:0/1711963979 shutdown_connections 2024-03-08T20:57:26.160 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:26.175+0000 7f105ac00640 1 -- 172.21.15.107:0/1711963979 wait complete. 2024-03-08T20:57:26.162 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:26.176+0000 7f105ac00640 1 Processor -- start 2024-03-08T20:57:26.163 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:26.176+0000 7f105ac00640 1 -- start start 2024-03-08T20:57:26.163 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:26.177+0000 7f105ac00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1054107750 0x7f105419d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:57:26.163 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:26.177+0000 7f105ac00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1054108940 0x7f105419de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:57:26.163 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:26.177+0000 7f105ac00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1054109300 0x7f10541a4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:57:26.163 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:26.177+0000 7f105ac00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1054113260 con 0x7f1054109300 2024-03-08T20:57:26.163 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:26.177+0000 7f105ac00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10541130e0 con 0x7f1054108940 2024-03-08T20:57:26.163 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:26.177+0000 7f105ac00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10541133e0 con 0x7f1054107750 2024-03-08T20:57:26.163 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:26.177+0000 7f105a200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1054109300 0x7f10541a4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:57:26.163 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:26.177+0000 7f105a200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1054109300 0x7f10541a4ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:37508/0 (socket says 172.21.15.107:37508) 2024-03-08T20:57:26.163 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:26.177+0000 7f105a200640 1 -- 172.21.15.107:0/3590099025 learned_addr learned my addr 172.21.15.107:0/3590099025 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T20:57:26.163 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:26.177+0000 7f1058e00640 1 --2- 172.21.15.107:0/3590099025 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1054108940 0x7f105419de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:57:26.163 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:26.177+0000 7f1059800640 1 --2- 172.21.15.107:0/3590099025 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1054107750 0x7f105419d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:57:26.164 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:26.177+0000 7f105a200640 1 -- 172.21.15.107:0/3590099025 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1054107750 msgr2=0x7f105419d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:57:26.164 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:26.177+0000 7f105a200640 1 --2- 172.21.15.107:0/3590099025 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1054107750 0x7f105419d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:26.164 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:26.177+0000 7f105a200640 1 -- 172.21.15.107:0/3590099025 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1054108940 msgr2=0x7f105419de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:57:26.164 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:26.177+0000 7f105a200640 1 --2- 172.21.15.107:0/3590099025 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1054108940 0x7f105419de30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:26.164 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:26.177+0000 7f105a200640 1 -- 172.21.15.107:0/3590099025 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10541a5510 con 0x7f1054109300 2024-03-08T20:57:26.164 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:26.178+0000 7f105a200640 1 --2- 172.21.15.107:0/3590099025 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1054109300 0x7f10541a4ea0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f104c00c970 tx=0x7f104c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:57:26.164 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:26.178+0000 7f1052000640 1 -- 172.21.15.107:0/3590099025 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f104c007bb0 con 0x7f1054109300 2024-03-08T20:57:26.164 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:26.178+0000 7f1052000640 1 -- 172.21.15.107:0/3590099025 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f104c007d50 con 0x7f1054109300 2024-03-08T20:57:26.164 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:26.178+0000 7f105ac00640 1 -- 172.21.15.107:0/3590099025 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10541a5800 con 0x7f1054109300 2024-03-08T20:57:26.164 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:26.178+0000 7f1052000640 1 -- 172.21.15.107:0/3590099025 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f104c018700 con 0x7f1054109300 2024-03-08T20:57:26.164 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:26.178+0000 7f105ac00640 1 -- 172.21.15.107:0/3590099025 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10541a5cd0 con 0x7f1054109300 2024-03-08T20:57:26.165 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:26.179+0000 7f105ac00640 1 -- 172.21.15.107:0/3590099025 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f105410eb40 con 0x7f1054109300 2024-03-08T20:57:26.165 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:26.181+0000 7f1052000640 1 -- 172.21.15.107:0/3590099025 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f104c00fa20 con 0x7f1054109300 2024-03-08T20:57:26.167 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:26.183+0000 7f1052000640 1 --2- 172.21.15.107:0/3590099025 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f10380a2800 0x7f10380a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:57:26.167 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:26.184+0000 7f1052000640 1 -- 172.21.15.107:0/3590099025 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f104c0ce810 con 0x7f1054109300 2024-03-08T20:57:26.167 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:26.184+0000 7f1059800640 1 --2- 172.21.15.107:0/3590099025 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f10380a2800 0x7f10380a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:57:26.168 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:26.184+0000 7f1059800640 1 --2- 172.21.15.107:0/3590099025 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f10380a2800 0x7f10380a4cc0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f1044005fd0 tx=0x7f1044005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:57:26.171 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:26.188+0000 7f1052000640 1 -- 172.21.15.107:0/3590099025 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f104c014030 con 0x7f1054109300 2024-03-08T20:57:26.412 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:26.429+0000 7f105ac00640 1 -- 172.21.15.107:0/3590099025 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f105402ce70 con 0x7f1054109300 2024-03-08T20:57:26.414 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:26.430+0000 7f1052000640 1 -- 172.21.15.107:0/3590099025 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+6080 (secure 0 0 0) 0x7f104c09bcb0 con 0x7f1054109300 2024-03-08T20:57:26.414 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-08T20:57:26.414 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/133582743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":133582743},{"type":"v1","addr":"172.21.15.107:6835","nonce":133582743}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/3798982337","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":3798982337},{"type":"v1","addr":"172.21.15.107:6837","nonce":3798982337}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/3404762782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":3404762782},{"type":"v1","addr":"172.21.15.204:6833","nonce":3404762782}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24430,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/2072518651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":2072518651},{"type":"v1","addr":"172.21.15.204:6835","nonce":2072518651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24442,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6837/42644905","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":42644905},{"type":"v1","addr":"172.21.15.204:6837","nonce":42644905}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":34397,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/1962161726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":1962161726},{"type":"v1","addr":"172.21.15.149:6833","nonce":1962161726}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":34415,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/442657066","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":442657066},{"type":"v1","addr":"172.21.15.149:6835","nonce":442657066}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[]} 2024-03-08T20:57:26.415 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 8 2024-03-08T20:57:26.417 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:26.433+0000 7f105ac00640 1 -- 172.21.15.107:0/3590099025 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f10380a2800 msgr2=0x7f10380a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:57:26.417 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:26.433+0000 7f105ac00640 1 --2- 172.21.15.107:0/3590099025 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f10380a2800 0x7f10380a4cc0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f1044005fd0 tx=0x7f1044005850 comp rx=0 tx=0).stop 2024-03-08T20:57:26.417 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:26.433+0000 7f105ac00640 1 -- 172.21.15.107:0/3590099025 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1054109300 msgr2=0x7f10541a4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:57:26.417 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:26.433+0000 7f105ac00640 1 --2- 172.21.15.107:0/3590099025 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1054109300 0x7f10541a4ea0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f104c00c970 tx=0x7f104c00ce30 comp rx=0 tx=0).stop 2024-03-08T20:57:26.418 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:26.433+0000 7f105ac00640 1 -- 172.21.15.107:0/3590099025 shutdown_connections 2024-03-08T20:57:26.418 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:26.433+0000 7f105ac00640 1 --2- 172.21.15.107:0/3590099025 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f10380a2800 0x7f10380a4cc0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:26.418 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:26.433+0000 7f105ac00640 1 --2- 172.21.15.107:0/3590099025 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1054109300 0x7f10541a4ea0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:26.418 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:26.433+0000 7f105ac00640 1 --2- 172.21.15.107:0/3590099025 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1054108940 0x7f105419de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:26.418 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:26.433+0000 7f105ac00640 1 --2- 172.21.15.107:0/3590099025 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1054107750 0x7f105419d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:26.418 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:26.433+0000 7f105ac00640 1 -- 172.21.15.107:0/3590099025 >> 172.21.15.107:0/3590099025 conn(0x7f1054076700 msgr2=0x7f105410bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:57:26.418 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:26.433+0000 7f105ac00640 1 -- 172.21.15.107:0/3590099025 shutdown_connections 2024-03-08T20:57:26.418 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:26.434+0000 7f105ac00640 1 -- 172.21.15.107:0/3590099025 wait complete. 2024-03-08T20:57:27.582 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:27 smithi149 ceph-mon[35042]: pgmap v345: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-08T20:57:27.582 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:27 smithi149 ceph-mon[35042]: from='client.? 172.21.15.107:0/3590099025' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-08T20:57:27.629 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:27 smithi204 ceph-mon[34935]: pgmap v345: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-08T20:57:27.629 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:27 smithi204 ceph-mon[34935]: from='client.? 172.21.15.107:0/3590099025' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-08T20:57:27.718 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:27 smithi107 ceph-mon[31381]: pgmap v345: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-08T20:57:27.719 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:27 smithi107 ceph-mon[31381]: from='client.? 172.21.15.107:0/3590099025' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-08T20:57:27.928 INFO:journalctl@ceph.mds.h.smithi149.stdout:Mar 08 20:57:27 smithi149 systemd[1]: Starting Ceph mds.h for dc20f0f8-dd8c-11ee-95c6-87774f69a715... 2024-03-08T20:57:28.205 INFO:journalctl@ceph.mds.h.smithi149.stdout:Mar 08 20:57:27 smithi149 podman[65285]: 2024-03-08 20:57:27.847439948 +0000 UTC m=+0.033591832 image pull a272146cb810b1fd4a53627f019472132e30966d5c25b01bfc44a001295f1249 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd 2024-03-08T20:57:28.205 INFO:journalctl@ceph.mds.h.smithi149.stdout:Mar 08 20:57:27 smithi149 podman[65285]: 2024-03-08 20:57:27.949952377 +0000 UTC m=+0.136104300 container create 76101b0dc8103f89bc0f1d9b80bec5781a96e22b239a1a08d12a2c45d6c14cf0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-h, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=rishabh-2024mar8-7d63482, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-03-08T20:57:28.205 INFO:journalctl@ceph.mds.h.smithi149.stdout:Mar 08 20:57:28 smithi149 podman[65285]: 2024-03-08 20:57:28.220405608 +0000 UTC m=+0.406557534 container init 76101b0dc8103f89bc0f1d9b80bec5781a96e22b239a1a08d12a2c45d6c14cf0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-h, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=rishabh-2024mar8-7d63482, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-08T20:57:28.435 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-08T20:57:28.517 INFO:journalctl@ceph.mds.h.smithi149.stdout:Mar 08 20:57:28 smithi149 podman[65285]: 2024-03-08 20:57:28.231032257 +0000 UTC m=+0.417184233 container start 76101b0dc8103f89bc0f1d9b80bec5781a96e22b239a1a08d12a2c45d6c14cf0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-h, ceph=True, RELEASE=rishabh-2024mar8-7d63482, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-03-08T20:57:28.517 INFO:journalctl@ceph.mds.h.smithi149.stdout:Mar 08 20:57:28 smithi149 ceph-mds[65304]: -- 172.21.15.149:0/1391659688 <== mon.2 v2:172.21.15.149:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55aeb42a5a40 con 0x55aeb4278800 2024-03-08T20:57:28.517 INFO:journalctl@ceph.mds.h.smithi149.stdout:Mar 08 20:57:28 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-h[65300]: starting mds.h at 2024-03-08T20:57:28.517 INFO:journalctl@ceph.mds.h.smithi149.stdout:Mar 08 20:57:28 smithi149 bash[65285]: 76101b0dc8103f89bc0f1d9b80bec5781a96e22b239a1a08d12a2c45d6c14cf0 2024-03-08T20:57:28.517 INFO:journalctl@ceph.mds.h.smithi149.stdout:Mar 08 20:57:28 smithi149 systemd[1]: Started Ceph mds.h for dc20f0f8-dd8c-11ee-95c6-87774f69a715. 2024-03-08T20:57:28.653 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:28.669+0000 7f8d8d600640 1 -- 172.21.15.107:0/3340967123 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8d88075330 msgr2=0x7f8d88073780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:57:28.653 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:28.669+0000 7f8d8d600640 1 --2- 172.21.15.107:0/3340967123 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8d88075330 0x7f8d88073780 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f8d78009900 tx=0x7f8d7802bc30 comp rx=0 tx=0).stop 2024-03-08T20:57:28.653 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:28.669+0000 7f8d8d600640 1 -- 172.21.15.107:0/3340967123 shutdown_connections 2024-03-08T20:57:28.653 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:28.669+0000 7f8d8d600640 1 --2- 172.21.15.107:0/3340967123 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8d881091d0 0x7f8d8810b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:28.653 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:28.669+0000 7f8d8d600640 1 --2- 172.21.15.107:0/3340967123 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8d88073d50 0x7f8d880741d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:28.653 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:28.669+0000 7f8d8d600640 1 --2- 172.21.15.107:0/3340967123 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8d88075330 0x7f8d88073780 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:28.653 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:28.669+0000 7f8d8d600640 1 -- 172.21.15.107:0/3340967123 >> 172.21.15.107:0/3340967123 conn(0x7f8d880fc8a0 msgr2=0x7f8d880fecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:57:28.654 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:28.669+0000 7f8d8d600640 1 -- 172.21.15.107:0/3340967123 shutdown_connections 2024-03-08T20:57:28.654 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:28.669+0000 7f8d8d600640 1 -- 172.21.15.107:0/3340967123 wait complete. 2024-03-08T20:57:28.656 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:28.670+0000 7f8d8d600640 1 Processor -- start 2024-03-08T20:57:28.656 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:28.670+0000 7f8d8d600640 1 -- start start 2024-03-08T20:57:28.656 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:28.670+0000 7f8d8d600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8d88073d50 0x7f8d881997a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:57:28.656 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:28.670+0000 7f8d8d600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8d88075330 0x7f8d88199ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:57:28.656 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:28.670+0000 7f8d8d600640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8d881091d0 0x7f8d881a0d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:57:28.656 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:28.670+0000 7f8d8d600640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d8810f090 con 0x7f8d88075330 2024-03-08T20:57:28.656 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:28.670+0000 7f8d8d600640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d8810ef10 con 0x7f8d881091d0 2024-03-08T20:57:28.656 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:28.670+0000 7f8d8d600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d8810f210 con 0x7f8d88073d50 2024-03-08T20:57:28.657 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:28.671+0000 7f8d7e000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8d88075330 0x7f8d88199ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:57:28.657 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:28.671+0000 7f8d7e000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8d88075330 0x7f8d88199ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:37526/0 (socket says 172.21.15.107:37526) 2024-03-08T20:57:28.657 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:28.671+0000 7f8d7e000640 1 -- 172.21.15.107:0/1733644768 learned_addr learned my addr 172.21.15.107:0/1733644768 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T20:57:28.657 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:28.671+0000 7f8d86a00640 1 --2- 172.21.15.107:0/1733644768 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8d88073d50 0x7f8d881997a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:57:28.657 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:28.671+0000 7f8d87400640 1 --2- 172.21.15.107:0/1733644768 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8d881091d0 0x7f8d881a0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:57:28.657 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:28.671+0000 7f8d7e000640 1 -- 172.21.15.107:0/1733644768 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8d88073d50 msgr2=0x7f8d881997a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:57:28.657 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:28.671+0000 7f8d7e000640 1 --2- 172.21.15.107:0/1733644768 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8d88073d50 0x7f8d881997a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:28.657 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:28.671+0000 7f8d7e000640 1 -- 172.21.15.107:0/1733644768 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8d881091d0 msgr2=0x7f8d881a0d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:57:28.657 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:28.671+0000 7f8d7e000640 1 --2- 172.21.15.107:0/1733644768 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8d881091d0 0x7f8d881a0d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:28.657 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:28.671+0000 7f8d7e000640 1 -- 172.21.15.107:0/1733644768 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d881a13c0 con 0x7f8d88075330 2024-03-08T20:57:28.657 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:28.671+0000 7f8d87400640 1 --2- 172.21.15.107:0/1733644768 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8d881091d0 0x7f8d881a0d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T20:57:28.657 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:28.671+0000 7f8d7e000640 1 --2- 172.21.15.107:0/1733644768 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8d88075330 0x7f8d88199ce0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f8d7000d950 tx=0x7f8d7000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:57:28.657 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:28.672+0000 7f8d7fe00640 1 -- 172.21.15.107:0/1733644768 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d70017070 con 0x7f8d88075330 2024-03-08T20:57:28.657 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:28.672+0000 7f8d7fe00640 1 -- 172.21.15.107:0/1733644768 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f8d70009c30 con 0x7f8d88075330 2024-03-08T20:57:28.658 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:28.672+0000 7f8d8d600640 1 -- 172.21.15.107:0/1733644768 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d881a16b0 con 0x7f8d88075330 2024-03-08T20:57:28.658 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:28.672+0000 7f8d7fe00640 1 -- 172.21.15.107:0/1733644768 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d7000f620 con 0x7f8d88075330 2024-03-08T20:57:28.658 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:28.672+0000 7f8d8d600640 1 -- 172.21.15.107:0/1733644768 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d881a1c50 con 0x7f8d88075330 2024-03-08T20:57:28.658 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:28.673+0000 7f8d8d600640 1 -- 172.21.15.107:0/1733644768 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d88107f00 con 0x7f8d88075330 2024-03-08T20:57:28.658 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:28.675+0000 7f8d7fe00640 1 -- 172.21.15.107:0/1733644768 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f8d700109a0 con 0x7f8d88075330 2024-03-08T20:57:28.660 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:28.676+0000 7f8d7fe00640 1 --2- 172.21.15.107:0/1733644768 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f8d6c0a2880 0x7f8d6c0a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:57:28.660 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:28.676+0000 7f8d7fe00640 1 -- 172.21.15.107:0/1733644768 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f8d700ce230 con 0x7f8d88075330 2024-03-08T20:57:28.660 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:28.676+0000 7f8d86a00640 1 --2- 172.21.15.107:0/1733644768 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f8d6c0a2880 0x7f8d6c0a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:57:28.660 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:28.677+0000 7f8d86a00640 1 --2- 172.21.15.107:0/1733644768 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f8d6c0a2880 0x7f8d6c0a4d40 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f8d78000c00 tx=0x7f8d7802c7e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:57:28.664 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:28.680+0000 7f8d7fe00640 1 -- 172.21.15.107:0/1733644768 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f8d70009de0 con 0x7f8d88075330 2024-03-08T20:57:28.901 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:28.917+0000 7f8d8d600640 1 -- 172.21.15.107:0/1733644768 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8d88061ef0 con 0x7f8d88075330 2024-03-08T20:57:28.902 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:28.919+0000 7f8d7fe00640 1 -- 172.21.15.107:0/1733644768 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+6871 (secure 0 0 0) 0x7f8d7009b6d0 con 0x7f8d88075330 2024-03-08T20:57:28.903 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-08T20:57:28.903 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/133582743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":133582743},{"type":"v1","addr":"172.21.15.107:6835","nonce":133582743}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/3798982337","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":3798982337},{"type":"v1","addr":"172.21.15.107:6837","nonce":3798982337}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/3404762782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":3404762782},{"type":"v1","addr":"172.21.15.204:6833","nonce":3404762782}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24430,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/2072518651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":2072518651},{"type":"v1","addr":"172.21.15.204:6835","nonce":2072518651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24442,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6837/42644905","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":42644905},{"type":"v1","addr":"172.21.15.204:6837","nonce":42644905}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":34397,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/1962161726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":1962161726},{"type":"v1","addr":"172.21.15.149:6833","nonce":1962161726}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":34415,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/442657066","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":442657066},{"type":"v1","addr":"172.21.15.149:6835","nonce":442657066}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":34445,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6837/914546237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":914546237},{"type":"v1","addr":"172.21.15.149:6837","nonce":914546237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9}],"filesystems":[]} 2024-03-08T20:57:28.903 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 9 2024-03-08T20:57:28.905 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:28.921+0000 7f8d8d600640 1 -- 172.21.15.107:0/1733644768 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f8d6c0a2880 msgr2=0x7f8d6c0a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:57:28.905 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:28.921+0000 7f8d8d600640 1 --2- 172.21.15.107:0/1733644768 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f8d6c0a2880 0x7f8d6c0a4d40 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f8d78000c00 tx=0x7f8d7802c7e0 comp rx=0 tx=0).stop 2024-03-08T20:57:28.906 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:28.922+0000 7f8d8d600640 1 -- 172.21.15.107:0/1733644768 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8d88075330 msgr2=0x7f8d88199ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:57:28.906 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:28.922+0000 7f8d8d600640 1 --2- 172.21.15.107:0/1733644768 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8d88075330 0x7f8d88199ce0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f8d7000d950 tx=0x7f8d7000de10 comp rx=0 tx=0).stop 2024-03-08T20:57:28.906 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:28.922+0000 7f8d8d600640 1 -- 172.21.15.107:0/1733644768 shutdown_connections 2024-03-08T20:57:28.906 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:28.922+0000 7f8d8d600640 1 --2- 172.21.15.107:0/1733644768 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f8d6c0a2880 0x7f8d6c0a4d40 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:28.906 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:28.922+0000 7f8d8d600640 1 --2- 172.21.15.107:0/1733644768 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8d881091d0 0x7f8d881a0d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:28.906 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:28.922+0000 7f8d8d600640 1 --2- 172.21.15.107:0/1733644768 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8d88075330 0x7f8d88199ce0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:28.906 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:28.922+0000 7f8d8d600640 1 --2- 172.21.15.107:0/1733644768 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8d88073d50 0x7f8d881997a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:28.906 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:28.922+0000 7f8d8d600640 1 -- 172.21.15.107:0/1733644768 >> 172.21.15.107:0/1733644768 conn(0x7f8d880fc8a0 msgr2=0x7f8d881050d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:57:28.906 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:28.922+0000 7f8d8d600640 1 -- 172.21.15.107:0/1733644768 shutdown_connections 2024-03-08T20:57:28.906 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:28.922+0000 7f8d8d600640 1 -- 172.21.15.107:0/1733644768 wait complete. 2024-03-08T20:57:29.629 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:29 smithi204 ceph-mon[34935]: pgmap v346: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-08T20:57:29.630 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:29 smithi204 ceph-mon[34935]: mds.? [v2:172.21.15.149:6836/914546237,v1:172.21.15.149:6837/914546237] up:boot 2024-03-08T20:57:29.630 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:29 smithi204 ceph-mon[34935]: fsmap 8 up:standby 2024-03-08T20:57:29.630 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:29 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-08T20:57:29.630 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:29 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:57:29.630 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:29 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:57:29.630 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:29 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:57:29.630 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:29 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-08T20:57:29.630 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:29 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-08T20:57:29.630 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:29 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:57:29.630 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:29 smithi204 ceph-mon[34935]: Deploying daemon mds.i on smithi204 2024-03-08T20:57:29.630 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:29 smithi204 ceph-mon[34935]: from='client.? 172.21.15.107:0/1733644768' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-08T20:57:29.719 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:29 smithi107 ceph-mon[31381]: pgmap v346: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-08T20:57:29.719 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:29 smithi107 ceph-mon[31381]: mds.? [v2:172.21.15.149:6836/914546237,v1:172.21.15.149:6837/914546237] up:boot 2024-03-08T20:57:29.719 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:29 smithi107 ceph-mon[31381]: fsmap 8 up:standby 2024-03-08T20:57:29.719 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:29 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-08T20:57:29.719 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:29 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:57:29.719 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:29 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:57:29.720 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:29 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:57:29.720 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:29 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-08T20:57:29.720 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:29 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-08T20:57:29.720 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:29 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:57:29.720 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:29 smithi107 ceph-mon[31381]: Deploying daemon mds.i on smithi204 2024-03-08T20:57:29.720 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:29 smithi107 ceph-mon[31381]: from='client.? 172.21.15.107:0/1733644768' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-08T20:57:29.766 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:29 smithi149 ceph-mon[35042]: pgmap v346: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-08T20:57:29.767 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:29 smithi149 ceph-mon[35042]: mds.? [v2:172.21.15.149:6836/914546237,v1:172.21.15.149:6837/914546237] up:boot 2024-03-08T20:57:29.767 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:29 smithi149 ceph-mon[35042]: fsmap 8 up:standby 2024-03-08T20:57:29.767 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:29 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-08T20:57:29.767 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:29 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:57:29.767 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:29 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:57:29.767 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:29 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:57:29.767 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:29 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-08T20:57:29.767 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:29 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-08T20:57:29.767 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:29 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:57:29.767 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:29 smithi149 ceph-mon[35042]: Deploying daemon mds.i on smithi204 2024-03-08T20:57:29.767 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:29 smithi149 ceph-mon[35042]: from='client.? 172.21.15.107:0/1733644768' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-08T20:57:30.926 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-08T20:57:31.147 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:31.163+0000 7f6d1d000640 1 -- 172.21.15.107:0/1167276676 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6d18077890 msgr2=0x7f6d18078130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:57:31.147 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:31.163+0000 7f6d1d000640 1 --2- 172.21.15.107:0/1167276676 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6d18077890 0x7f6d18078130 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f6d10009ca0 tx=0x7f6d1002c100 comp rx=0 tx=0).stop 2024-03-08T20:57:31.147 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:31.163+0000 7f6d1d000640 1 -- 172.21.15.107:0/1167276676 shutdown_connections 2024-03-08T20:57:31.147 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:31.163+0000 7f6d1d000640 1 --2- 172.21.15.107:0/1167276676 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6d18077890 0x7f6d18078130 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:31.147 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:31.163+0000 7f6d1d000640 1 --2- 172.21.15.107:0/1167276676 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6d18076ef0 0x7f6d18077350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:31.147 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:31.163+0000 7f6d1d000640 1 --2- 172.21.15.107:0/1167276676 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6d18075cb0 0x7f6d180760d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:31.147 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:31.163+0000 7f6d1d000640 1 -- 172.21.15.107:0/1167276676 >> 172.21.15.107:0/1167276676 conn(0x7f6d180feaa0 msgr2=0x7f6d18100ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:57:31.147 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:31.163+0000 7f6d1d000640 1 -- 172.21.15.107:0/1167276676 shutdown_connections 2024-03-08T20:57:31.147 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:31.163+0000 7f6d1d000640 1 -- 172.21.15.107:0/1167276676 wait complete. 2024-03-08T20:57:31.150 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:31.164+0000 7f6d1d000640 1 Processor -- start 2024-03-08T20:57:31.150 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:31.164+0000 7f6d1d000640 1 -- start start 2024-03-08T20:57:31.150 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:31.164+0000 7f6d1d000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6d18075cb0 0x7f6d1819d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:57:31.150 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:31.164+0000 7f6d1d000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6d18076ef0 0x7f6d1819dea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:57:31.150 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:31.164+0000 7f6d1d000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6d18077890 0x7f6d181a4f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:57:31.150 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:31.164+0000 7f6d1d000640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d18113450 con 0x7f6d18076ef0 2024-03-08T20:57:31.150 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:31.164+0000 7f6d1d000640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d181132d0 con 0x7f6d18075cb0 2024-03-08T20:57:31.151 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:31.164+0000 7f6d1d000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d181135d0 con 0x7f6d18077890 2024-03-08T20:57:31.151 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:31.164+0000 7f6d0d400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6d18076ef0 0x7f6d1819dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:57:31.151 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:31.164+0000 7f6d0d400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6d18076ef0 0x7f6d1819dea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:37546/0 (socket says 172.21.15.107:37546) 2024-03-08T20:57:31.151 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:31.164+0000 7f6d0d400640 1 -- 172.21.15.107:0/653572795 learned_addr learned my addr 172.21.15.107:0/653572795 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T20:57:31.151 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:31.165+0000 7f6d15e00640 1 --2- 172.21.15.107:0/653572795 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6d18075cb0 0x7f6d1819d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:57:31.151 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:31.165+0000 7f6d0d400640 1 -- 172.21.15.107:0/653572795 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6d18077890 msgr2=0x7f6d181a4f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:57:31.151 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:31.165+0000 7f6d16800640 1 --2- 172.21.15.107:0/653572795 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6d18077890 0x7f6d181a4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:57:31.151 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:31.165+0000 7f6d0d400640 1 --2- 172.21.15.107:0/653572795 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6d18077890 0x7f6d181a4f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:31.152 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:31.165+0000 7f6d0d400640 1 -- 172.21.15.107:0/653572795 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6d18075cb0 msgr2=0x7f6d1819d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:57:31.152 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:31.165+0000 7f6d0d400640 1 --2- 172.21.15.107:0/653572795 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6d18075cb0 0x7f6d1819d960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:31.152 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:31.165+0000 7f6d0d400640 1 -- 172.21.15.107:0/653572795 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d181a5580 con 0x7f6d18076ef0 2024-03-08T20:57:31.152 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:31.165+0000 7f6d0d400640 1 --2- 172.21.15.107:0/653572795 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6d18076ef0 0x7f6d1819dea0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f6d0000c920 tx=0x7f6d0000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:57:31.152 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:31.165+0000 7f6d0f400640 1 -- 172.21.15.107:0/653572795 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d00007c70 con 0x7f6d18076ef0 2024-03-08T20:57:31.152 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:31.165+0000 7f6d0f400640 1 -- 172.21.15.107:0/653572795 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f6d00007e10 con 0x7f6d18076ef0 2024-03-08T20:57:31.152 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:31.165+0000 7f6d1d000640 1 -- 172.21.15.107:0/653572795 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d181a5870 con 0x7f6d18076ef0 2024-03-08T20:57:31.152 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:31.165+0000 7f6d0f400640 1 -- 172.21.15.107:0/653572795 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d00018820 con 0x7f6d18076ef0 2024-03-08T20:57:31.152 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:31.165+0000 7f6d1d000640 1 -- 172.21.15.107:0/653572795 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d181a5e10 con 0x7f6d18076ef0 2024-03-08T20:57:31.152 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:31.166+0000 7f6d1d000640 1 -- 172.21.15.107:0/653572795 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d1810fe80 con 0x7f6d18076ef0 2024-03-08T20:57:31.152 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:31.168+0000 7f6d0f400640 1 -- 172.21.15.107:0/653572795 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f6d000189c0 con 0x7f6d18076ef0 2024-03-08T20:57:31.153 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:31.170+0000 7f6d0f400640 1 --2- 172.21.15.107:0/653572795 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f6cf80a2900 0x7f6cf80a4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:57:31.153 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:31.170+0000 7f6d0f400640 1 -- 172.21.15.107:0/653572795 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f6d000ce7c0 con 0x7f6d18076ef0 2024-03-08T20:57:31.154 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:31.170+0000 7f6d15e00640 1 --2- 172.21.15.107:0/653572795 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f6cf80a2900 0x7f6cf80a4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:57:31.154 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:31.171+0000 7f6d15e00640 1 --2- 172.21.15.107:0/653572795 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f6cf80a2900 0x7f6cf80a4dc0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f6d0800ba40 tx=0x7f6d08005cc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:57:31.157 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:31.174+0000 7f6d0f400640 1 -- 172.21.15.107:0/653572795 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f6d00014030 con 0x7f6d18076ef0 2024-03-08T20:57:31.396 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:31.413+0000 7f6d1d000640 1 -- 172.21.15.107:0/653572795 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6d1802ce70 con 0x7f6d18076ef0 2024-03-08T20:57:31.397 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:31.414+0000 7f6d0f400640 1 -- 172.21.15.107:0/653572795 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+6871 (secure 0 0 0) 0x7f6d0000de20 con 0x7f6d18076ef0 2024-03-08T20:57:31.397 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-08T20:57:31.398 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/133582743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":133582743},{"type":"v1","addr":"172.21.15.107:6835","nonce":133582743}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/3798982337","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":3798982337},{"type":"v1","addr":"172.21.15.107:6837","nonce":3798982337}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/3404762782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":3404762782},{"type":"v1","addr":"172.21.15.204:6833","nonce":3404762782}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24430,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/2072518651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":2072518651},{"type":"v1","addr":"172.21.15.204:6835","nonce":2072518651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24442,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6837/42644905","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":42644905},{"type":"v1","addr":"172.21.15.204:6837","nonce":42644905}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":34397,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/1962161726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":1962161726},{"type":"v1","addr":"172.21.15.149:6833","nonce":1962161726}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":34415,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/442657066","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":442657066},{"type":"v1","addr":"172.21.15.149:6835","nonce":442657066}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":34445,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6837/914546237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":914546237},{"type":"v1","addr":"172.21.15.149:6837","nonce":914546237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9}],"filesystems":[]} 2024-03-08T20:57:31.398 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 9 2024-03-08T20:57:31.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:31.416+0000 7f6d1d000640 1 -- 172.21.15.107:0/653572795 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f6cf80a2900 msgr2=0x7f6cf80a4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:57:31.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:31.416+0000 7f6d1d000640 1 --2- 172.21.15.107:0/653572795 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f6cf80a2900 0x7f6cf80a4dc0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f6d0800ba40 tx=0x7f6d08005cc0 comp rx=0 tx=0).stop 2024-03-08T20:57:31.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:31.416+0000 7f6d1d000640 1 -- 172.21.15.107:0/653572795 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6d18076ef0 msgr2=0x7f6d1819dea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:57:31.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:31.416+0000 7f6d1d000640 1 --2- 172.21.15.107:0/653572795 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6d18076ef0 0x7f6d1819dea0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f6d0000c920 tx=0x7f6d0000cde0 comp rx=0 tx=0).stop 2024-03-08T20:57:31.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:31.416+0000 7f6d1d000640 1 -- 172.21.15.107:0/653572795 shutdown_connections 2024-03-08T20:57:31.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:31.416+0000 7f6d1d000640 1 --2- 172.21.15.107:0/653572795 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f6cf80a2900 0x7f6cf80a4dc0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:31.401 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:31.416+0000 7f6d1d000640 1 --2- 172.21.15.107:0/653572795 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6d18077890 0x7f6d181a4f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:31.401 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:31.416+0000 7f6d1d000640 1 --2- 172.21.15.107:0/653572795 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6d18076ef0 0x7f6d1819dea0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:31.401 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:31.416+0000 7f6d1d000640 1 --2- 172.21.15.107:0/653572795 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6d18075cb0 0x7f6d1819d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:31.401 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:31.416+0000 7f6d1d000640 1 -- 172.21.15.107:0/653572795 >> 172.21.15.107:0/653572795 conn(0x7f6d180feaa0 msgr2=0x7f6d181005a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:57:31.401 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:31.417+0000 7f6d1d000640 1 -- 172.21.15.107:0/653572795 shutdown_connections 2024-03-08T20:57:31.401 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:31.417+0000 7f6d1d000640 1 -- 172.21.15.107:0/653572795 wait complete. 2024-03-08T20:57:31.562 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:31 smithi204 ceph-mon[34935]: pgmap v347: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-08T20:57:31.718 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:31 smithi107 ceph-mon[31381]: pgmap v347: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-08T20:57:31.766 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:31 smithi149 ceph-mon[35042]: pgmap v347: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-08T20:57:31.880 INFO:journalctl@ceph.mds.i.smithi204.stdout:Mar 08 20:57:31 smithi204 systemd[1]: Starting Ceph mds.i for dc20f0f8-dd8c-11ee-95c6-87774f69a715... 2024-03-08T20:57:32.226 INFO:journalctl@ceph.mds.i.smithi204.stdout:Mar 08 20:57:31 smithi204 podman[66298]: 2024-03-08 20:57:31.843106727 +0000 UTC m=+0.035761109 image pull a272146cb810b1fd4a53627f019472132e30966d5c25b01bfc44a001295f1249 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd 2024-03-08T20:57:32.226 INFO:journalctl@ceph.mds.i.smithi204.stdout:Mar 08 20:57:31 smithi204 podman[66298]: 2024-03-08 20:57:31.945716171 +0000 UTC m=+0.138370558 container create 728070e41a85ef6dc95d06a9e86ddb324894b510b7348b37576273167f33b7cc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-i, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, RELEASE=rishabh-2024mar8-7d63482, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-08T20:57:32.226 INFO:journalctl@ceph.mds.i.smithi204.stdout:Mar 08 20:57:32 smithi204 podman[66298]: 2024-03-08 20:57:32.241621682 +0000 UTC m=+0.434276098 container init 728070e41a85ef6dc95d06a9e86ddb324894b510b7348b37576273167f33b7cc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-i, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, RELEASE=rishabh-2024mar8-7d63482, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-03-08T20:57:32.629 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:32 smithi204 ceph-mon[34935]: from='client.? 172.21.15.107:0/653572795' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-08T20:57:32.630 INFO:journalctl@ceph.mds.i.smithi204.stdout:Mar 08 20:57:32 smithi204 podman[66298]: 2024-03-08 20:57:32.251484219 +0000 UTC m=+0.444138608 container start 728070e41a85ef6dc95d06a9e86ddb324894b510b7348b37576273167f33b7cc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-i, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=rishabh-2024mar8-7d63482, io.buildah.version=1.33.5) 2024-03-08T20:57:32.630 INFO:journalctl@ceph.mds.i.smithi204.stdout:Mar 08 20:57:32 smithi204 ceph-mds[66316]: -- 172.21.15.204:0/3797468002 <== mon.1 v2:172.21.15.204:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55f933469a40 con 0x55f93343cc00 2024-03-08T20:57:32.630 INFO:journalctl@ceph.mds.i.smithi204.stdout:Mar 08 20:57:32 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-i[66312]: starting mds.i at 2024-03-08T20:57:32.630 INFO:journalctl@ceph.mds.i.smithi204.stdout:Mar 08 20:57:32 smithi204 bash[66298]: 728070e41a85ef6dc95d06a9e86ddb324894b510b7348b37576273167f33b7cc 2024-03-08T20:57:32.630 INFO:journalctl@ceph.mds.i.smithi204.stdout:Mar 08 20:57:32 smithi204 systemd[1]: Started Ceph mds.i for dc20f0f8-dd8c-11ee-95c6-87774f69a715. 2024-03-08T20:57:32.718 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:32 smithi107 ceph-mon[31381]: from='client.? 172.21.15.107:0/653572795' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-08T20:57:32.766 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:32 smithi149 ceph-mon[35042]: from='client.? 172.21.15.107:0/653572795' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-08T20:57:33.419 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-08T20:57:33.611 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:33.627+0000 7fb89bc00640 1 -- 172.21.15.107:0/3078431266 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb894103530 msgr2=0x7fb894103950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:57:33.611 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:33.627+0000 7fb89bc00640 1 --2- 172.21.15.107:0/3078431266 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb894103530 0x7fb894103950 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fb888009900 tx=0x7fb88802bc30 comp rx=0 tx=0).stop 2024-03-08T20:57:33.612 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:33.628+0000 7fb89bc00640 1 -- 172.21.15.107:0/3078431266 shutdown_connections 2024-03-08T20:57:33.612 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:33.628+0000 7fb89bc00640 1 --2- 172.21.15.107:0/3078431266 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb8941050e0 0x7fb89410b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:33.612 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:33.628+0000 7fb89bc00640 1 --2- 172.21.15.107:0/3078431266 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb894104720 0x7fb894104ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:33.612 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:33.628+0000 7fb89bc00640 1 --2- 172.21.15.107:0/3078431266 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb894103530 0x7fb894103950 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:33.612 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:33.628+0000 7fb89bc00640 1 -- 172.21.15.107:0/3078431266 >> 172.21.15.107:0/3078431266 conn(0x7fb8940feae0 msgr2=0x7fb894100f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:57:33.612 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:33.628+0000 7fb89bc00640 1 -- 172.21.15.107:0/3078431266 shutdown_connections 2024-03-08T20:57:33.612 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:33.628+0000 7fb89bc00640 1 -- 172.21.15.107:0/3078431266 wait complete. 2024-03-08T20:57:33.614 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:33.629+0000 7fb89bc00640 1 Processor -- start 2024-03-08T20:57:33.614 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:33.629+0000 7fb89bc00640 1 -- start start 2024-03-08T20:57:33.614 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:33.629+0000 7fb89bc00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb894103530 0x7fb8941996c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:57:33.614 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:33.629+0000 7fb89bc00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb894104720 0x7fb894199c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:57:33.614 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:33.629+0000 7fb89bc00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb8941050e0 0x7fb8941a0c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:57:33.614 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:33.629+0000 7fb89bc00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb89410f0c0 con 0x7fb894103530 2024-03-08T20:57:33.614 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:33.629+0000 7fb89bc00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb89410ef40 con 0x7fb894104720 2024-03-08T20:57:33.614 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:33.629+0000 7fb89bc00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb89410f240 con 0x7fb8941050e0 2024-03-08T20:57:33.614 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:33.630+0000 7fb899200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb894103530 0x7fb8941996c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:57:33.614 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:33.630+0000 7fb899200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb894103530 0x7fb8941996c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:37564/0 (socket says 172.21.15.107:37564) 2024-03-08T20:57:33.614 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:33.630+0000 7fb899200640 1 -- 172.21.15.107:0/565789331 learned_addr learned my addr 172.21.15.107:0/565789331 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T20:57:33.614 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:33.630+0000 7fb899200640 1 -- 172.21.15.107:0/565789331 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb8941050e0 msgr2=0x7fb8941a0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:57:33.614 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:33.630+0000 7fb899200640 1 --2- 172.21.15.107:0/565789331 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb8941050e0 0x7fb8941a0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:33.614 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:33.630+0000 7fb899200640 1 -- 172.21.15.107:0/565789331 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb894104720 msgr2=0x7fb894199c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:57:33.615 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:33.630+0000 7fb899200640 1 --2- 172.21.15.107:0/565789331 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb894104720 0x7fb894199c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:33.615 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:33.630+0000 7fb899200640 1 -- 172.21.15.107:0/565789331 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8941a12e0 con 0x7fb894103530 2024-03-08T20:57:33.615 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:33.630+0000 7fb899200640 1 --2- 172.21.15.107:0/565789331 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb894103530 0x7fb8941996c0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fb888005ab0 tx=0x7fb8880042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:57:33.615 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:33.630+0000 7fb891600640 1 -- 172.21.15.107:0/565789331 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb888043020 con 0x7fb894103530 2024-03-08T20:57:33.615 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:33.630+0000 7fb891600640 1 -- 172.21.15.107:0/565789331 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fb8880054c0 con 0x7fb894103530 2024-03-08T20:57:33.616 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:33.631+0000 7fb89bc00640 1 -- 172.21.15.107:0/565789331 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8941a1570 con 0x7fb894103530 2024-03-08T20:57:33.616 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:33.631+0000 7fb891600640 1 -- 172.21.15.107:0/565789331 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb888003d10 con 0x7fb894103530 2024-03-08T20:57:33.616 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:33.631+0000 7fb89bc00640 1 -- 172.21.15.107:0/565789331 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8941a1b10 con 0x7fb894103530 2024-03-08T20:57:33.616 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:33.632+0000 7fb89bc00640 1 -- 172.21.15.107:0/565789331 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb89410a7f0 con 0x7fb894103530 2024-03-08T20:57:33.617 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:33.634+0000 7fb891600640 1 -- 172.21.15.107:0/565789331 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) v1 ==== 137291+0+0 (secure 0 0 0) 0x7fb888046020 con 0x7fb894103530 2024-03-08T20:57:33.620 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:33.636+0000 7fb891600640 1 --2- 172.21.15.107:0/565789331 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fb87c0a2880 0x7fb87c0a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:57:33.620 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:33.636+0000 7fb891600640 1 -- 172.21.15.107:0/565789331 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fb8880f09f0 con 0x7fb894103530 2024-03-08T20:57:33.620 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:33.636+0000 7fb893e00640 1 --2- 172.21.15.107:0/565789331 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fb87c0a2880 0x7fb87c0a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:57:33.620 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:33.637+0000 7fb893e00640 1 --2- 172.21.15.107:0/565789331 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fb87c0a2880 0x7fb87c0a4d40 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fb89419abd0 tx=0x7fb880005c70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:57:33.623 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:33.639+0000 7fb891600640 1 -- 172.21.15.107:0/565789331 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7fb88804d020 con 0x7fb894103530 2024-03-08T20:57:33.700 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:33 smithi107 ceph-mon[31381]: pgmap v348: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-08T20:57:33.700 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:33 smithi107 ceph-mon[31381]: mds.? [v2:172.21.15.204:6838/1857316822,v1:172.21.15.204:6839/1857316822] up:boot 2024-03-08T20:57:33.700 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:33 smithi107 ceph-mon[31381]: fsmap 9 up:standby 2024-03-08T20:57:33.700 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:33 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-08T20:57:33.700 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:33 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:57:33.700 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:33 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:57:33.700 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:33 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:57:33.700 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:33 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-08T20:57:33.701 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:33 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-08T20:57:33.701 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:33 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:57:33.766 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:33 smithi149 ceph-mon[35042]: pgmap v348: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-08T20:57:33.766 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:33 smithi149 ceph-mon[35042]: mds.? [v2:172.21.15.204:6838/1857316822,v1:172.21.15.204:6839/1857316822] up:boot 2024-03-08T20:57:33.767 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:33 smithi149 ceph-mon[35042]: fsmap 9 up:standby 2024-03-08T20:57:33.767 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:33 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-08T20:57:33.767 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:33 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:57:33.767 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:33 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:57:33.767 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:33 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:57:33.767 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:33 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-08T20:57:33.767 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:33 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-08T20:57:33.767 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:33 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:57:33.869 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:33.885+0000 7fb89bc00640 1 -- 172.21.15.107:0/565789331 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb89402ce70 con 0x7fb894103530 2024-03-08T20:57:33.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:33.886+0000 7fb891600640 1 -- 172.21.15.107:0/565789331 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) v1 ==== 94+0+7667 (secure 0 0 0) 0x7fb89402ce70 con 0x7fb894103530 2024-03-08T20:57:33.870 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-08T20:57:33.871 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/133582743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":133582743},{"type":"v1","addr":"172.21.15.107:6835","nonce":133582743}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/3798982337","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":3798982337},{"type":"v1","addr":"172.21.15.107:6837","nonce":3798982337}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/3404762782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":3404762782},{"type":"v1","addr":"172.21.15.204:6833","nonce":3404762782}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24430,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/2072518651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":2072518651},{"type":"v1","addr":"172.21.15.204:6835","nonce":2072518651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24442,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6837/42644905","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":42644905},{"type":"v1","addr":"172.21.15.204:6837","nonce":42644905}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24457,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6839/1857316822","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":1857316822},{"type":"v1","addr":"172.21.15.204:6839","nonce":1857316822}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":34397,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/1962161726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":1962161726},{"type":"v1","addr":"172.21.15.149:6833","nonce":1962161726}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":34415,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/442657066","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":442657066},{"type":"v1","addr":"172.21.15.149:6835","nonce":442657066}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":34445,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6837/914546237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":914546237},{"type":"v1","addr":"172.21.15.149:6837","nonce":914546237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9}],"filesystems":[]} 2024-03-08T20:57:33.871 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 10 2024-03-08T20:57:33.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:33.890+0000 7fb89bc00640 1 -- 172.21.15.107:0/565789331 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fb87c0a2880 msgr2=0x7fb87c0a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:57:33.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:33.890+0000 7fb89bc00640 1 --2- 172.21.15.107:0/565789331 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fb87c0a2880 0x7fb87c0a4d40 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fb89419abd0 tx=0x7fb880005c70 comp rx=0 tx=0).stop 2024-03-08T20:57:33.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:33.890+0000 7fb89bc00640 1 -- 172.21.15.107:0/565789331 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb894103530 msgr2=0x7fb8941996c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:57:33.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:33.890+0000 7fb89bc00640 1 --2- 172.21.15.107:0/565789331 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb894103530 0x7fb8941996c0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fb888005ab0 tx=0x7fb8880042f0 comp rx=0 tx=0).stop 2024-03-08T20:57:33.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:33.890+0000 7fb89bc00640 1 -- 172.21.15.107:0/565789331 shutdown_connections 2024-03-08T20:57:33.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:33.890+0000 7fb89bc00640 1 --2- 172.21.15.107:0/565789331 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fb87c0a2880 0x7fb87c0a4d40 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:33.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:33.890+0000 7fb89bc00640 1 --2- 172.21.15.107:0/565789331 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb8941050e0 0x7fb8941a0c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:33.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:33.890+0000 7fb89bc00640 1 --2- 172.21.15.107:0/565789331 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb894104720 0x7fb894199c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:33.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:33.890+0000 7fb89bc00640 1 --2- 172.21.15.107:0/565789331 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb894103530 0x7fb8941996c0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:33.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:33.890+0000 7fb89bc00640 1 -- 172.21.15.107:0/565789331 >> 172.21.15.107:0/565789331 conn(0x7fb8940feae0 msgr2=0x7fb8941079c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:57:33.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:33.890+0000 7fb89bc00640 1 -- 172.21.15.107:0/565789331 shutdown_connections 2024-03-08T20:57:33.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:33.890+0000 7fb89bc00640 1 -- 172.21.15.107:0/565789331 wait complete. 2024-03-08T20:57:33.879 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:33 smithi204 ceph-mon[34935]: pgmap v348: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-08T20:57:33.879 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:33 smithi204 ceph-mon[34935]: mds.? [v2:172.21.15.204:6838/1857316822,v1:172.21.15.204:6839/1857316822] up:boot 2024-03-08T20:57:33.879 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:33 smithi204 ceph-mon[34935]: fsmap 9 up:standby 2024-03-08T20:57:33.880 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:33 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-08T20:57:33.880 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:33 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:57:33.880 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:33 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:57:33.880 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:33 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:57:33.880 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:33 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-08T20:57:33.880 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:33 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-08T20:57:33.880 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:33 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:57:34.718 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:34 smithi107 ceph-mon[31381]: Deploying daemon mds.g on smithi107 2024-03-08T20:57:34.718 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:34 smithi107 ceph-mon[31381]: from='client.? 172.21.15.107:0/565789331' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-08T20:57:34.766 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:34 smithi149 ceph-mon[35042]: Deploying daemon mds.g on smithi107 2024-03-08T20:57:34.766 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:34 smithi149 ceph-mon[35042]: from='client.? 172.21.15.107:0/565789331' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-08T20:57:34.880 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:34 smithi204 ceph-mon[34935]: Deploying daemon mds.g on smithi107 2024-03-08T20:57:34.880 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:34 smithi204 ceph-mon[34935]: from='client.? 172.21.15.107:0/565789331' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-08T20:57:35.469 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:35 smithi107 ceph-mon[31381]: pgmap v349: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-08T20:57:35.629 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:35 smithi204 ceph-mon[34935]: pgmap v349: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-08T20:57:35.766 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:35 smithi149 ceph-mon[35042]: pgmap v349: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-08T20:57:35.893 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-08T20:57:36.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:36.113+0000 7f3c71e00640 1 -- 172.21.15.107:0/1001783717 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3c6c0ffb80 msgr2=0x7f3c6c101fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:57:36.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:36.113+0000 7f3c71e00640 1 --2- 172.21.15.107:0/1001783717 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3c6c0ffb80 0x7f3c6c101fb0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f3c5c009900 tx=0x7f3c5c02bc30 comp rx=0 tx=0).stop 2024-03-08T20:57:36.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:36.113+0000 7f3c71e00640 1 -- 172.21.15.107:0/1001783717 shutdown_connections 2024-03-08T20:57:36.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:36.113+0000 7f3c71e00640 1 --2- 172.21.15.107:0/1001783717 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3c6c104ea0 0x7f3c6c105320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:36.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:36.113+0000 7f3c71e00640 1 --2- 172.21.15.107:0/1001783717 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3c6c1024f0 0x7f3c6c104960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:36.098 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:36.113+0000 7f3c71e00640 1 --2- 172.21.15.107:0/1001783717 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3c6c0ffb80 0x7f3c6c101fb0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:36.098 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:36.113+0000 7f3c71e00640 1 -- 172.21.15.107:0/1001783717 >> 172.21.15.107:0/1001783717 conn(0x7f3c6c0fb7a0 msgr2=0x7f3c6c0fdbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:57:36.098 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:36.113+0000 7f3c71e00640 1 -- 172.21.15.107:0/1001783717 shutdown_connections 2024-03-08T20:57:36.098 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:36.113+0000 7f3c71e00640 1 -- 172.21.15.107:0/1001783717 wait complete. 2024-03-08T20:57:36.100 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:36.114+0000 7f3c71e00640 1 Processor -- start 2024-03-08T20:57:36.100 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:36.114+0000 7f3c71e00640 1 -- start start 2024-03-08T20:57:36.100 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:36.115+0000 7f3c71e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3c6c0ffb80 0x7f3c6c19dc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:57:36.100 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:36.115+0000 7f3c71e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3c6c1024f0 0x7f3c6c19e150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:57:36.100 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:36.115+0000 7f3c71e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3c6c104ea0 0x7f3c6c1a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:57:36.100 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:36.115+0000 7f3c71e00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c6c113500 con 0x7f3c6c104ea0 2024-03-08T20:57:36.100 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:36.115+0000 7f3c71e00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c6c113380 con 0x7f3c6c1024f0 2024-03-08T20:57:36.100 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:36.115+0000 7f3c71e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c6c113680 con 0x7f3c6c0ffb80 2024-03-08T20:57:36.100 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:36.115+0000 7f3c6b400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3c6c104ea0 0x7f3c6c1a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:57:36.101 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:36.115+0000 7f3c6b400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3c6c104ea0 0x7f3c6c1a5170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:45496/0 (socket says 172.21.15.107:45496) 2024-03-08T20:57:36.101 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:36.115+0000 7f3c6b400640 1 -- 172.21.15.107:0/3567429221 learned_addr learned my addr 172.21.15.107:0/3567429221 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T20:57:36.101 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:36.115+0000 7f3c6aa00640 1 --2- 172.21.15.107:0/3567429221 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3c6c0ffb80 0x7f3c6c19dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:57:36.101 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:36.115+0000 7f3c63e00640 1 --2- 172.21.15.107:0/3567429221 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3c6c1024f0 0x7f3c6c19e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:57:36.101 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:36.115+0000 7f3c6b400640 1 -- 172.21.15.107:0/3567429221 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3c6c0ffb80 msgr2=0x7f3c6c19dc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:57:36.101 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:36.115+0000 7f3c6b400640 1 --2- 172.21.15.107:0/3567429221 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3c6c0ffb80 0x7f3c6c19dc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:36.101 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:36.115+0000 7f3c6b400640 1 -- 172.21.15.107:0/3567429221 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3c6c1024f0 msgr2=0x7f3c6c19e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:57:36.101 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:36.115+0000 7f3c6b400640 1 --2- 172.21.15.107:0/3567429221 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3c6c1024f0 0x7f3c6c19e150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:36.101 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:36.115+0000 7f3c6b400640 1 -- 172.21.15.107:0/3567429221 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c6c1a57e0 con 0x7f3c6c104ea0 2024-03-08T20:57:36.101 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:36.115+0000 7f3c63e00640 1 --2- 172.21.15.107:0/3567429221 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3c6c1024f0 0x7f3c6c19e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T20:57:36.101 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:36.116+0000 7f3c6b400640 1 --2- 172.21.15.107:0/3567429221 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3c6c104ea0 0x7f3c6c1a5170 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f3c6400eaa0 tx=0x7f3c6400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:57:36.101 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:36.116+0000 7f3c70a00640 1 -- 172.21.15.107:0/3567429221 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c6400cdb0 con 0x7f3c6c104ea0 2024-03-08T20:57:36.101 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:36.116+0000 7f3c71e00640 1 -- 172.21.15.107:0/3567429221 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c6c1a5ad0 con 0x7f3c6c104ea0 2024-03-08T20:57:36.101 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:36.116+0000 7f3c70a00640 1 -- 172.21.15.107:0/3567429221 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f3c64005020 con 0x7f3c6c104ea0 2024-03-08T20:57:36.102 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:36.116+0000 7f3c70a00640 1 -- 172.21.15.107:0/3567429221 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c640116d0 con 0x7f3c6c104ea0 2024-03-08T20:57:36.102 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:36.116+0000 7f3c71e00640 1 -- 172.21.15.107:0/3567429221 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c6c1a6020 con 0x7f3c6c104ea0 2024-03-08T20:57:36.102 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:36.117+0000 7f3c71e00640 1 -- 172.21.15.107:0/3567429221 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c6c0fd290 con 0x7f3c6c104ea0 2024-03-08T20:57:36.102 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:36.119+0000 7f3c70a00640 1 -- 172.21.15.107:0/3567429221 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f3c640051d0 con 0x7f3c6c104ea0 2024-03-08T20:57:36.104 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:36.120+0000 7f3c70a00640 1 --2- 172.21.15.107:0/3567429221 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f3c500a2800 0x7f3c500a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:57:36.104 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:36.120+0000 7f3c70a00640 1 -- 172.21.15.107:0/3567429221 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f3c640ce880 con 0x7f3c6c104ea0 2024-03-08T20:57:36.104 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:36.121+0000 7f3c6aa00640 1 --2- 172.21.15.107:0/3567429221 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f3c500a2800 0x7f3c500a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:57:36.104 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:36.121+0000 7f3c6aa00640 1 --2- 172.21.15.107:0/3567429221 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f3c500a2800 0x7f3c500a4cc0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f3c5c006010 tx=0x7f3c5c000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:57:36.108 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:36.125+0000 7f3c70a00640 1 -- 172.21.15.107:0/3567429221 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f3c64016030 con 0x7f3c6c104ea0 2024-03-08T20:57:36.359 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:36.375+0000 7f3c71e00640 1 -- 172.21.15.107:0/3567429221 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3c6c061f90 con 0x7f3c6c104ea0 2024-03-08T20:57:36.359 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:36.376+0000 7f3c70a00640 1 -- 172.21.15.107:0/3567429221 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) v1 ==== 94+0+7667 (secure 0 0 0) 0x7f3c6409bd20 con 0x7f3c6c104ea0 2024-03-08T20:57:36.360 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-08T20:57:36.360 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/133582743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":133582743},{"type":"v1","addr":"172.21.15.107:6835","nonce":133582743}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/3798982337","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":3798982337},{"type":"v1","addr":"172.21.15.107:6837","nonce":3798982337}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/3404762782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":3404762782},{"type":"v1","addr":"172.21.15.204:6833","nonce":3404762782}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24430,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/2072518651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":2072518651},{"type":"v1","addr":"172.21.15.204:6835","nonce":2072518651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24442,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6837/42644905","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":42644905},{"type":"v1","addr":"172.21.15.204:6837","nonce":42644905}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24457,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6839/1857316822","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":1857316822},{"type":"v1","addr":"172.21.15.204:6839","nonce":1857316822}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":34397,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/1962161726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":1962161726},{"type":"v1","addr":"172.21.15.149:6833","nonce":1962161726}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":34415,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/442657066","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":442657066},{"type":"v1","addr":"172.21.15.149:6835","nonce":442657066}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":34445,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6837/914546237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":914546237},{"type":"v1","addr":"172.21.15.149:6837","nonce":914546237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9}],"filesystems":[]} 2024-03-08T20:57:36.360 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 10 2024-03-08T20:57:36.362 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:36.378+0000 7f3c71e00640 1 -- 172.21.15.107:0/3567429221 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f3c500a2800 msgr2=0x7f3c500a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:57:36.362 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:36.378+0000 7f3c71e00640 1 --2- 172.21.15.107:0/3567429221 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f3c500a2800 0x7f3c500a4cc0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f3c5c006010 tx=0x7f3c5c000bc0 comp rx=0 tx=0).stop 2024-03-08T20:57:36.362 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:36.378+0000 7f3c71e00640 1 -- 172.21.15.107:0/3567429221 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3c6c104ea0 msgr2=0x7f3c6c1a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:57:36.362 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:36.378+0000 7f3c71e00640 1 --2- 172.21.15.107:0/3567429221 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3c6c104ea0 0x7f3c6c1a5170 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f3c6400eaa0 tx=0x7f3c6400ef60 comp rx=0 tx=0).stop 2024-03-08T20:57:36.363 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:36.378+0000 7f3c71e00640 1 -- 172.21.15.107:0/3567429221 shutdown_connections 2024-03-08T20:57:36.363 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:36.378+0000 7f3c71e00640 1 --2- 172.21.15.107:0/3567429221 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f3c500a2800 0x7f3c500a4cc0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:36.363 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:36.378+0000 7f3c71e00640 1 --2- 172.21.15.107:0/3567429221 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3c6c104ea0 0x7f3c6c1a5170 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:36.363 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:36.378+0000 7f3c71e00640 1 --2- 172.21.15.107:0/3567429221 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3c6c1024f0 0x7f3c6c19e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:36.363 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:36.378+0000 7f3c71e00640 1 --2- 172.21.15.107:0/3567429221 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3c6c0ffb80 0x7f3c6c19dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:36.363 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:36.378+0000 7f3c71e00640 1 -- 172.21.15.107:0/3567429221 >> 172.21.15.107:0/3567429221 conn(0x7f3c6c0fb7a0 msgr2=0x7f3c6c102eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:57:36.363 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:36.379+0000 7f3c71e00640 1 -- 172.21.15.107:0/3567429221 shutdown_connections 2024-03-08T20:57:36.363 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:36.379+0000 7f3c71e00640 1 -- 172.21.15.107:0/3567429221 wait complete. 2024-03-08T20:57:36.468 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:36 smithi107 ceph-mon[31381]: from='client.? 172.21.15.107:0/3567429221' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-08T20:57:36.766 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:36 smithi149 ceph-mon[35042]: from='client.? 172.21.15.107:0/3567429221' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-08T20:57:36.879 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:36 smithi204 ceph-mon[34935]: from='client.? 172.21.15.107:0/3567429221' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-08T20:57:37.699 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:37 smithi107 ceph-mon[31381]: pgmap v350: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-08T20:57:37.699 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:37 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:57:37.699 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:37 smithi107 ceph-mon[31381]: mds.? [v2:172.21.15.107:6838/3070059000,v1:172.21.15.107:6839/3070059000] up:boot 2024-03-08T20:57:37.699 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:37 smithi107 ceph-mon[31381]: fsmap 10 up:standby 2024-03-08T20:57:37.699 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:37 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-08T20:57:37.699 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:37 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:57:37.699 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:37 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:57:37.699 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:37 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-08T20:57:37.700 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:37 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-08T20:57:37.700 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:37 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:57:37.700 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:37 smithi107 ceph-mon[31381]: Deploying daemon mds.a on smithi107 2024-03-08T20:57:37.766 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:37 smithi149 ceph-mon[35042]: pgmap v350: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-08T20:57:37.767 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:37 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:57:37.767 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:37 smithi149 ceph-mon[35042]: mds.? [v2:172.21.15.107:6838/3070059000,v1:172.21.15.107:6839/3070059000] up:boot 2024-03-08T20:57:37.767 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:37 smithi149 ceph-mon[35042]: fsmap 10 up:standby 2024-03-08T20:57:37.767 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:37 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-08T20:57:37.767 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:37 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:57:37.767 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:37 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:57:37.767 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:37 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-08T20:57:37.767 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:37 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-08T20:57:37.767 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:37 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:57:37.767 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:37 smithi149 ceph-mon[35042]: Deploying daemon mds.a on smithi107 2024-03-08T20:57:37.880 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:37 smithi204 ceph-mon[34935]: pgmap v350: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-08T20:57:37.980 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:37 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:57:37.980 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:37 smithi204 ceph-mon[34935]: mds.? [v2:172.21.15.107:6838/3070059000,v1:172.21.15.107:6839/3070059000] up:boot 2024-03-08T20:57:37.980 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:37 smithi204 ceph-mon[34935]: fsmap 10 up:standby 2024-03-08T20:57:37.980 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:37 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-08T20:57:37.980 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:37 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:57:37.980 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:37 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:57:37.981 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:37 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-08T20:57:37.981 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:37 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-08T20:57:37.981 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:37 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:57:37.981 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:37 smithi204 ceph-mon[34935]: Deploying daemon mds.a on smithi107 2024-03-08T20:57:38.380 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-08T20:57:38.606 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:38.622+0000 7f378aa00640 1 -- 172.21.15.107:0/262313935 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f37841094e0 msgr2=0x7f378410fd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:57:38.607 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:38.622+0000 7f378aa00640 1 --2- 172.21.15.107:0/262313935 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f37841094e0 0x7f378410fd70 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f377c009ca0 tx=0x7f377c02c0f0 comp rx=0 tx=0).stop 2024-03-08T20:57:38.607 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:38.623+0000 7f378aa00640 1 -- 172.21.15.107:0/262313935 shutdown_connections 2024-03-08T20:57:38.607 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:38.623+0000 7f378aa00640 1 --2- 172.21.15.107:0/262313935 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f37841094e0 0x7f378410fd70 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:38.607 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:38.623+0000 7f378aa00640 1 --2- 172.21.15.107:0/262313935 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3784108b20 0x7f3784108fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:38.607 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:38.623+0000 7f378aa00640 1 --2- 172.21.15.107:0/262313935 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3784107930 0x7f3784107d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:38.607 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:38.623+0000 7f378aa00640 1 -- 172.21.15.107:0/262313935 >> 172.21.15.107:0/262313935 conn(0x7f3784076900 msgr2=0x7f3784078d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:57:38.608 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:38.623+0000 7f378aa00640 1 -- 172.21.15.107:0/262313935 shutdown_connections 2024-03-08T20:57:38.608 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:38.623+0000 7f378aa00640 1 -- 172.21.15.107:0/262313935 wait complete. 2024-03-08T20:57:38.611 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:38.624+0000 7f378aa00640 1 Processor -- start 2024-03-08T20:57:38.611 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:38.624+0000 7f378aa00640 1 -- start start 2024-03-08T20:57:38.612 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:38.624+0000 7f378aa00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3784107930 0x7f378419dae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:57:38.612 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:38.624+0000 7f378aa00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3784108b20 0x7f378419e020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:57:38.612 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:38.624+0000 7f378aa00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f37841094e0 0x7f37841a5090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:57:38.612 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:38.624+0000 7f378aa00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3784113460 con 0x7f3784108b20 2024-03-08T20:57:38.612 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:38.624+0000 7f378aa00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37841132e0 con 0x7f37841094e0 2024-03-08T20:57:38.612 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:38.624+0000 7f378aa00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37841135e0 con 0x7f3784107930 2024-03-08T20:57:38.612 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:38.625+0000 7f377b400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3784108b20 0x7f378419e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:57:38.612 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:38.625+0000 7f377b400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3784108b20 0x7f378419e020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:45530/0 (socket says 172.21.15.107:45530) 2024-03-08T20:57:38.612 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:38.625+0000 7f377b400640 1 -- 172.21.15.107:0/1800214228 learned_addr learned my addr 172.21.15.107:0/1800214228 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T20:57:38.612 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:38.625+0000 7f3788a00640 1 --2- 172.21.15.107:0/1800214228 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f37841094e0 0x7f37841a5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:57:38.612 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:38.625+0000 7f377b400640 1 -- 172.21.15.107:0/1800214228 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3784107930 msgr2=0x7f378419dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:57:38.612 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:38.625+0000 7f377b400640 1 --2- 172.21.15.107:0/1800214228 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3784107930 0x7f378419dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:38.613 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:38.625+0000 7f377b400640 1 -- 172.21.15.107:0/1800214228 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f37841094e0 msgr2=0x7f37841a5090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:57:38.613 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:38.625+0000 7f377b400640 1 --2- 172.21.15.107:0/1800214228 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f37841094e0 0x7f37841a5090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:38.613 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:38.625+0000 7f377b400640 1 -- 172.21.15.107:0/1800214228 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37841a5700 con 0x7f3784108b20 2024-03-08T20:57:38.613 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:38.625+0000 7f377b400640 1 --2- 172.21.15.107:0/1800214228 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3784108b20 0x7f378419e020 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f376c00c970 tx=0x7f376c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:57:38.613 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:38.625+0000 7f3781600640 1 -- 172.21.15.107:0/1800214228 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f376c007bb0 con 0x7f3784108b20 2024-03-08T20:57:38.613 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:38.626+0000 7f3781600640 1 -- 172.21.15.107:0/1800214228 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f376c007d50 con 0x7f3784108b20 2024-03-08T20:57:38.613 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:38.626+0000 7f378aa00640 1 -- 172.21.15.107:0/1800214228 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37841a59f0 con 0x7f3784108b20 2024-03-08T20:57:38.613 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:38.626+0000 7f3781600640 1 -- 172.21.15.107:0/1800214228 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f376c00f450 con 0x7f3784108b20 2024-03-08T20:57:38.613 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:38.626+0000 7f378aa00640 1 -- 172.21.15.107:0/1800214228 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37841a5f40 con 0x7f3784108b20 2024-03-08T20:57:38.613 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:38.627+0000 7f378aa00640 1 -- 172.21.15.107:0/1800214228 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f378410ec50 con 0x7f3784108b20 2024-03-08T20:57:38.615 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:38.629+0000 7f3781600640 1 -- 172.21.15.107:0/1800214228 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f376c004f20 con 0x7f3784108b20 2024-03-08T20:57:38.615 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:38.630+0000 7f3781600640 1 --2- 172.21.15.107:0/1800214228 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f37680a2880 0x7f37680a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:57:38.615 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:38.630+0000 7f3781600640 1 -- 172.21.15.107:0/1800214228 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f376c0ce3d0 con 0x7f3784108b20 2024-03-08T20:57:38.615 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:38.630+0000 7f3783e00640 1 --2- 172.21.15.107:0/1800214228 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f37680a2880 0x7f37680a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:57:38.615 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:38.631+0000 7f3783e00640 1 --2- 172.21.15.107:0/1800214228 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f37680a2880 0x7f37680a4d40 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f3784108980 tx=0x7f3774005c10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:57:38.618 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:38.634+0000 7f3781600640 1 -- 172.21.15.107:0/1800214228 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f376c014030 con 0x7f3784108b20 2024-03-08T20:57:38.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:38.889+0000 7f378aa00640 1 -- 172.21.15.107:0/1800214228 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f378402ce70 con 0x7f3784108b20 2024-03-08T20:57:38.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:38.891+0000 7f3781600640 1 -- 172.21.15.107:0/1800214228 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) v1 ==== 94+0+8462 (secure 0 0 0) 0x7f376c09b870 con 0x7f3784108b20 2024-03-08T20:57:38.876 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-08T20:57:38.876 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/133582743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":133582743},{"type":"v1","addr":"172.21.15.107:6835","nonce":133582743}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/3798982337","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":3798982337},{"type":"v1","addr":"172.21.15.107:6837","nonce":3798982337}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14625,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6839/3070059000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":3070059000},{"type":"v1","addr":"172.21.15.107:6839","nonce":3070059000}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/3404762782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":3404762782},{"type":"v1","addr":"172.21.15.204:6833","nonce":3404762782}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24430,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/2072518651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":2072518651},{"type":"v1","addr":"172.21.15.204:6835","nonce":2072518651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24442,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6837/42644905","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":42644905},{"type":"v1","addr":"172.21.15.204:6837","nonce":42644905}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24457,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6839/1857316822","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":1857316822},{"type":"v1","addr":"172.21.15.204:6839","nonce":1857316822}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":34397,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/1962161726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":1962161726},{"type":"v1","addr":"172.21.15.149:6833","nonce":1962161726}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":34415,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/442657066","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":442657066},{"type":"v1","addr":"172.21.15.149:6835","nonce":442657066}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":34445,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6837/914546237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":914546237},{"type":"v1","addr":"172.21.15.149:6837","nonce":914546237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9}],"filesystems":[]} 2024-03-08T20:57:38.876 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 11 2024-03-08T20:57:38.881 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:38.897+0000 7f378aa00640 1 -- 172.21.15.107:0/1800214228 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f37680a2880 msgr2=0x7f37680a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:57:38.881 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:38.897+0000 7f378aa00640 1 --2- 172.21.15.107:0/1800214228 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f37680a2880 0x7f37680a4d40 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f3784108980 tx=0x7f3774005c10 comp rx=0 tx=0).stop 2024-03-08T20:57:38.881 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:38.897+0000 7f378aa00640 1 -- 172.21.15.107:0/1800214228 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3784108b20 msgr2=0x7f378419e020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:57:38.881 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:38.897+0000 7f378aa00640 1 --2- 172.21.15.107:0/1800214228 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3784108b20 0x7f378419e020 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f376c00c970 tx=0x7f376c00ce30 comp rx=0 tx=0).stop 2024-03-08T20:57:38.881 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:38.897+0000 7f378aa00640 1 -- 172.21.15.107:0/1800214228 shutdown_connections 2024-03-08T20:57:38.881 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:38.897+0000 7f378aa00640 1 --2- 172.21.15.107:0/1800214228 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f37680a2880 0x7f37680a4d40 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:38.882 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:38.897+0000 7f378aa00640 1 --2- 172.21.15.107:0/1800214228 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f37841094e0 0x7f37841a5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:38.882 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:38.897+0000 7f378aa00640 1 --2- 172.21.15.107:0/1800214228 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3784108b20 0x7f378419e020 secure :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f376c00c970 tx=0x7f376c00ce30 comp rx=0 tx=0).stop 2024-03-08T20:57:38.882 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:38.897+0000 7f378aa00640 1 --2- 172.21.15.107:0/1800214228 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3784107930 0x7f378419dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:38.882 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:38.897+0000 7f378aa00640 1 -- 172.21.15.107:0/1800214228 >> 172.21.15.107:0/1800214228 conn(0x7f3784076900 msgr2=0x7f378410be20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:57:38.882 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:38.897+0000 7f378aa00640 1 -- 172.21.15.107:0/1800214228 shutdown_connections 2024-03-08T20:57:38.882 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:38.897+0000 7f378aa00640 1 -- 172.21.15.107:0/1800214228 wait complete. 2024-03-08T20:57:39.718 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:39 smithi107 ceph-mon[31381]: pgmap v351: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-08T20:57:39.718 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:39 smithi107 ceph-mon[31381]: from='client.? 172.21.15.107:0/1800214228' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-08T20:57:39.718 INFO:journalctl@ceph.mds.a.smithi107.stdout:Mar 08 20:57:39 smithi107 systemd[1]: Starting Ceph mds.a for dc20f0f8-dd8c-11ee-95c6-87774f69a715... 2024-03-08T20:57:39.766 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:39 smithi149 ceph-mon[35042]: pgmap v351: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-08T20:57:39.766 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:39 smithi149 ceph-mon[35042]: from='client.? 172.21.15.107:0/1800214228' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-08T20:57:39.879 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:39 smithi204 ceph-mon[34935]: pgmap v351: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-08T20:57:39.879 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:39 smithi204 ceph-mon[34935]: from='client.? 172.21.15.107:0/1800214228' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-08T20:57:40.143 INFO:journalctl@ceph.mds.a.smithi107.stdout:Mar 08 20:57:39 smithi107 podman[73942]: 2024-03-08 20:57:39.730549469 +0000 UTC m=+0.032375743 image pull a272146cb810b1fd4a53627f019472132e30966d5c25b01bfc44a001295f1249 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd 2024-03-08T20:57:40.143 INFO:journalctl@ceph.mds.a.smithi107.stdout:Mar 08 20:57:39 smithi107 podman[73942]: 2024-03-08 20:57:39.845179654 +0000 UTC m=+0.147005959 container create b3abd6c77684313e6b255934f6df11e4748740b54d37c859790bfb33748917de (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-a, maintainer=Guillaume Abrioux , RELEASE=rishabh-2024mar8-7d63482, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS) 2024-03-08T20:57:40.143 INFO:journalctl@ceph.mds.a.smithi107.stdout:Mar 08 20:57:40 smithi107 podman[73942]: 2024-03-08 20:57:40.159052415 +0000 UTC m=+0.460878764 container init b3abd6c77684313e6b255934f6df11e4748740b54d37c859790bfb33748917de (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-a, RELEASE=rishabh-2024mar8-7d63482, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, ceph=True) 2024-03-08T20:57:40.469 INFO:journalctl@ceph.mds.a.smithi107.stdout:Mar 08 20:57:40 smithi107 podman[73942]: 2024-03-08 20:57:40.170969821 +0000 UTC m=+0.472796098 container start b3abd6c77684313e6b255934f6df11e4748740b54d37c859790bfb33748917de (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-a, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=rishabh-2024mar8-7d63482, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-08T20:57:40.469 INFO:journalctl@ceph.mds.a.smithi107.stdout:Mar 08 20:57:40 smithi107 ceph-mds[73961]: -- 172.21.15.107:0/456802772 <== mon.0 v2:172.21.15.107:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x560f2ef95a40 con 0x560f2ef68800 2024-03-08T20:57:40.469 INFO:journalctl@ceph.mds.a.smithi107.stdout:Mar 08 20:57:40 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-a[73957]: starting mds.a at 2024-03-08T20:57:40.469 INFO:journalctl@ceph.mds.a.smithi107.stdout:Mar 08 20:57:40 smithi107 bash[73942]: b3abd6c77684313e6b255934f6df11e4748740b54d37c859790bfb33748917de 2024-03-08T20:57:40.469 INFO:journalctl@ceph.mds.a.smithi107.stdout:Mar 08 20:57:40 smithi107 systemd[1]: Started Ceph mds.a for dc20f0f8-dd8c-11ee-95c6-87774f69a715. 2024-03-08T20:57:40.902 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-08T20:57:41.127 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:41.144+0000 7f661da00640 1 -- 172.21.15.107:0/3969418901 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f66181050e0 msgr2=0x7f661810b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:57:41.128 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:41.144+0000 7f661da00640 1 --2- 172.21.15.107:0/3969418901 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f66181050e0 0x7f661810b970 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f6610009ca0 tx=0x7f661002c0f0 comp rx=0 tx=0).stop 2024-03-08T20:57:41.128 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:41.144+0000 7f661da00640 1 -- 172.21.15.107:0/3969418901 shutdown_connections 2024-03-08T20:57:41.128 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:41.144+0000 7f661da00640 1 --2- 172.21.15.107:0/3969418901 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f66181050e0 0x7f661810b970 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:41.128 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:41.144+0000 7f661da00640 1 --2- 172.21.15.107:0/3969418901 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6618104720 0x7f6618104ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:41.128 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:41.144+0000 7f661da00640 1 --2- 172.21.15.107:0/3969418901 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6618103530 0x7f6618103950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:41.128 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:41.144+0000 7f661da00640 1 -- 172.21.15.107:0/3969418901 >> 172.21.15.107:0/3969418901 conn(0x7f66180feae0 msgr2=0x7f6618100f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:57:41.128 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:41.144+0000 7f661da00640 1 -- 172.21.15.107:0/3969418901 shutdown_connections 2024-03-08T20:57:41.128 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:41.144+0000 7f661da00640 1 -- 172.21.15.107:0/3969418901 wait complete. 2024-03-08T20:57:41.130 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:41.145+0000 7f661da00640 1 Processor -- start 2024-03-08T20:57:41.130 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:41.145+0000 7f661da00640 1 -- start start 2024-03-08T20:57:41.130 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:41.145+0000 7f661da00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6618103530 0x7f66181996f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:57:41.131 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:41.145+0000 7f661da00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6618104720 0x7f6618199c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:57:41.131 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:41.145+0000 7f661da00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f66181050e0 0x7f66181a0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:57:41.131 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:41.145+0000 7f661da00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f661810f070 con 0x7f6618103530 2024-03-08T20:57:41.131 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:41.145+0000 7f661da00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f661810eef0 con 0x7f6618104720 2024-03-08T20:57:41.131 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:41.145+0000 7f661da00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f661810f1f0 con 0x7f66181050e0 2024-03-08T20:57:41.131 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:41.145+0000 7f6616a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6618103530 0x7f66181996f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:57:41.131 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:41.145+0000 7f6616a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6618103530 0x7f66181996f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:45574/0 (socket says 172.21.15.107:45574) 2024-03-08T20:57:41.131 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:41.145+0000 7f6616a00640 1 -- 172.21.15.107:0/55641892 learned_addr learned my addr 172.21.15.107:0/55641892 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T20:57:41.131 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:41.146+0000 7f660fe00640 1 --2- 172.21.15.107:0/55641892 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6618104720 0x7f6618199c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:57:41.131 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:41.146+0000 7f6617400640 1 --2- 172.21.15.107:0/55641892 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f66181050e0 0x7f66181a0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:57:41.131 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:41.146+0000 7f6616a00640 1 -- 172.21.15.107:0/55641892 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f66181050e0 msgr2=0x7f66181a0ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:57:41.131 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:41.146+0000 7f6616a00640 1 --2- 172.21.15.107:0/55641892 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f66181050e0 0x7f66181a0ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:41.132 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:41.146+0000 7f6616a00640 1 -- 172.21.15.107:0/55641892 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6618104720 msgr2=0x7f6618199c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:57:41.132 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:41.146+0000 7f6616a00640 1 --2- 172.21.15.107:0/55641892 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6618104720 0x7f6618199c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:41.132 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:41.146+0000 7f6616a00640 1 -- 172.21.15.107:0/55641892 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66181a1310 con 0x7f6618103530 2024-03-08T20:57:41.132 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:41.146+0000 7f6616a00640 1 --2- 172.21.15.107:0/55641892 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6618103530 0x7f66181996f0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f660800c970 tx=0x7f660800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:57:41.132 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:41.146+0000 7f660f400640 1 -- 172.21.15.107:0/55641892 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6608004220 con 0x7f6618103530 2024-03-08T20:57:41.132 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:41.146+0000 7f660f400640 1 -- 172.21.15.107:0/55641892 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f66080043c0 con 0x7f6618103530 2024-03-08T20:57:41.132 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:41.146+0000 7f660f400640 1 -- 172.21.15.107:0/55641892 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6608004a00 con 0x7f6618103530 2024-03-08T20:57:41.132 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:41.146+0000 7f661da00640 1 -- 172.21.15.107:0/55641892 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66181a1600 con 0x7f6618103530 2024-03-08T20:57:41.132 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:41.147+0000 7f661da00640 1 -- 172.21.15.107:0/55641892 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66181a1b00 con 0x7f6618103530 2024-03-08T20:57:41.133 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:41.148+0000 7f661da00640 1 -- 172.21.15.107:0/55641892 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f661810a8d0 con 0x7f6618103530 2024-03-08T20:57:41.133 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:41.149+0000 7f660f400640 1 -- 172.21.15.107:0/55641892 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f66080079b0 con 0x7f6618103530 2024-03-08T20:57:41.134 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:41.151+0000 7f660f400640 1 --2- 172.21.15.107:0/55641892 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f65fc0a2800 0x7f65fc0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:57:41.134 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:41.151+0000 7f660f400640 1 -- 172.21.15.107:0/55641892 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f66080ce3e0 con 0x7f6618103530 2024-03-08T20:57:41.134 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:41.151+0000 7f660fe00640 1 --2- 172.21.15.107:0/55641892 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f65fc0a2800 0x7f65fc0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:57:41.135 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:41.152+0000 7f660fe00640 1 --2- 172.21.15.107:0/55641892 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f65fc0a2800 0x7f65fc0a4cc0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f661819ac00 tx=0x7f660000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:57:41.138 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:41.155+0000 7f660f400640 1 -- 172.21.15.107:0/55641892 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f6608014030 con 0x7f6618103530 2024-03-08T20:57:41.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:41.397+0000 7f661da00640 1 -- 172.21.15.107:0/55641892 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f661802ce70 con 0x7f6618103530 2024-03-08T20:57:41.381 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:41.398+0000 7f660f400640 1 -- 172.21.15.107:0/55641892 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+9254 (secure 0 0 0) 0x7f660809b880 con 0x7f6618103530 2024-03-08T20:57:41.382 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-08T20:57:41.382 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/133582743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":133582743},{"type":"v1","addr":"172.21.15.107:6835","nonce":133582743}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/3798982337","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":3798982337},{"type":"v1","addr":"172.21.15.107:6837","nonce":3798982337}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14625,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6839/3070059000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":3070059000},{"type":"v1","addr":"172.21.15.107:6839","nonce":3070059000}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14637,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6841/747096194","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6840","nonce":747096194},{"type":"v1","addr":"172.21.15.107:6841","nonce":747096194}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/3404762782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":3404762782},{"type":"v1","addr":"172.21.15.204:6833","nonce":3404762782}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24430,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/2072518651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":2072518651},{"type":"v1","addr":"172.21.15.204:6835","nonce":2072518651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24442,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6837/42644905","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":42644905},{"type":"v1","addr":"172.21.15.204:6837","nonce":42644905}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24457,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6839/1857316822","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":1857316822},{"type":"v1","addr":"172.21.15.204:6839","nonce":1857316822}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":34397,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/1962161726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":1962161726},{"type":"v1","addr":"172.21.15.149:6833","nonce":1962161726}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":34415,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/442657066","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":442657066},{"type":"v1","addr":"172.21.15.149:6835","nonce":442657066}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":34445,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6837/914546237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":914546237},{"type":"v1","addr":"172.21.15.149:6837","nonce":914546237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9}],"filesystems":[]} 2024-03-08T20:57:41.382 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 12 2024-03-08T20:57:41.384 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:41.401+0000 7f661da00640 1 -- 172.21.15.107:0/55641892 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f65fc0a2800 msgr2=0x7f65fc0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:57:41.385 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:41.401+0000 7f661da00640 1 --2- 172.21.15.107:0/55641892 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f65fc0a2800 0x7f65fc0a4cc0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f661819ac00 tx=0x7f660000b410 comp rx=0 tx=0).stop 2024-03-08T20:57:41.385 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:41.401+0000 7f661da00640 1 -- 172.21.15.107:0/55641892 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6618103530 msgr2=0x7f66181996f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:57:41.385 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:41.401+0000 7f661da00640 1 --2- 172.21.15.107:0/55641892 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6618103530 0x7f66181996f0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f660800c970 tx=0x7f660800ce30 comp rx=0 tx=0).stop 2024-03-08T20:57:41.385 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:41.401+0000 7f661da00640 1 -- 172.21.15.107:0/55641892 shutdown_connections 2024-03-08T20:57:41.385 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:41.401+0000 7f661da00640 1 --2- 172.21.15.107:0/55641892 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f65fc0a2800 0x7f65fc0a4cc0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:41.385 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:41.401+0000 7f661da00640 1 --2- 172.21.15.107:0/55641892 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f66181050e0 0x7f66181a0ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:41.385 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:41.401+0000 7f661da00640 1 --2- 172.21.15.107:0/55641892 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6618104720 0x7f6618199c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:41.385 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:41.401+0000 7f661da00640 1 --2- 172.21.15.107:0/55641892 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6618103530 0x7f66181996f0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:41.385 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:41.401+0000 7f661da00640 1 -- 172.21.15.107:0/55641892 >> 172.21.15.107:0/55641892 conn(0x7f66180feae0 msgr2=0x7f6618107aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:57:41.385 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:41.401+0000 7f661da00640 1 -- 172.21.15.107:0/55641892 shutdown_connections 2024-03-08T20:57:41.385 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:41.401+0000 7f661da00640 1 -- 172.21.15.107:0/55641892 wait complete. 2024-03-08T20:57:41.629 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:41 smithi204 ceph-mon[34935]: pgmap v352: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-08T20:57:41.630 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:41 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:57:41.630 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:41 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:57:41.630 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:41 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:57:41.630 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:41 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-08T20:57:41.630 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:41 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-08T20:57:41.630 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:41 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:57:41.630 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:41 smithi204 ceph-mon[34935]: Deploying daemon mds.b on smithi149 2024-03-08T20:57:41.630 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:41 smithi204 ceph-mon[34935]: mds.? [v2:172.21.15.107:6840/747096194,v1:172.21.15.107:6841/747096194] up:boot 2024-03-08T20:57:41.630 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:41 smithi204 ceph-mon[34935]: fsmap 11 up:standby 2024-03-08T20:57:41.630 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:41 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-08T20:57:41.692 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:41 smithi149 ceph-mon[35042]: pgmap v352: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-08T20:57:41.692 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:41 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:57:41.692 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:41 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:57:41.692 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:41 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:57:41.692 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:41 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-08T20:57:41.692 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:41 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-08T20:57:41.692 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:41 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:57:41.692 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:41 smithi149 ceph-mon[35042]: Deploying daemon mds.b on smithi149 2024-03-08T20:57:41.692 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:41 smithi149 ceph-mon[35042]: mds.? [v2:172.21.15.107:6840/747096194,v1:172.21.15.107:6841/747096194] up:boot 2024-03-08T20:57:41.692 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:41 smithi149 ceph-mon[35042]: fsmap 11 up:standby 2024-03-08T20:57:41.692 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:41 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-08T20:57:41.718 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:41 smithi107 ceph-mon[31381]: pgmap v352: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-08T20:57:41.719 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:41 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:57:41.719 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:41 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:57:41.719 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:41 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:57:41.719 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:41 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-08T20:57:41.719 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:41 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-08T20:57:41.719 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:41 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:57:41.719 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:41 smithi107 ceph-mon[31381]: Deploying daemon mds.b on smithi149 2024-03-08T20:57:41.719 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:41 smithi107 ceph-mon[31381]: mds.? [v2:172.21.15.107:6840/747096194,v1:172.21.15.107:6841/747096194] up:boot 2024-03-08T20:57:41.719 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:41 smithi107 ceph-mon[31381]: fsmap 11 up:standby 2024-03-08T20:57:41.719 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:41 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-08T20:57:42.672 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:42 smithi149 ceph-mon[35042]: from='client.? 172.21.15.107:0/55641892' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-08T20:57:42.718 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:42 smithi107 ceph-mon[31381]: from='client.? 172.21.15.107:0/55641892' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-08T20:57:42.879 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:42 smithi204 ceph-mon[34935]: from='client.? 172.21.15.107:0/55641892' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-08T20:57:43.404 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-08T20:57:43.478 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:43 smithi149 ceph-mon[35042]: pgmap v353: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-08T20:57:43.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:43.641+0000 7fe90d200640 1 -- 172.21.15.107:0/1220837768 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe908103530 msgr2=0x7fe908103950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:57:43.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:43.641+0000 7fe90d200640 1 --2- 172.21.15.107:0/1220837768 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe908103530 0x7fe908103950 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fe8f8009900 tx=0x7fe8f802bc30 comp rx=0 tx=0).stop 2024-03-08T20:57:43.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:43.641+0000 7fe90d200640 1 -- 172.21.15.107:0/1220837768 shutdown_connections 2024-03-08T20:57:43.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:43.641+0000 7fe90d200640 1 --2- 172.21.15.107:0/1220837768 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe9081050e0 0x7fe90810b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:43.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:43.641+0000 7fe90d200640 1 --2- 172.21.15.107:0/1220837768 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe908104720 0x7fe908104ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:43.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:43.641+0000 7fe90d200640 1 --2- 172.21.15.107:0/1220837768 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe908103530 0x7fe908103950 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:43.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:43.641+0000 7fe90d200640 1 -- 172.21.15.107:0/1220837768 >> 172.21.15.107:0/1220837768 conn(0x7fe9080feae0 msgr2=0x7fe908100f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:57:43.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:43.641+0000 7fe90d200640 1 -- 172.21.15.107:0/1220837768 shutdown_connections 2024-03-08T20:57:43.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:43.641+0000 7fe90d200640 1 -- 172.21.15.107:0/1220837768 wait complete. 2024-03-08T20:57:43.627 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:43.642+0000 7fe90d200640 1 Processor -- start 2024-03-08T20:57:43.627 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:43.642+0000 7fe90d200640 1 -- start start 2024-03-08T20:57:43.628 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:43.642+0000 7fe90d200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe908103530 0x7fe908199780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:57:43.628 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:43.643+0000 7fe90d200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe908104720 0x7fe908199cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:57:43.628 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:43.643+0000 7fe90d200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe9081050e0 0x7fe9081a0d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:57:43.628 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:43.643+0000 7fe90d200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe90810f0c0 con 0x7fe908103530 2024-03-08T20:57:43.628 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:43.643+0000 7fe90d200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe90810ef40 con 0x7fe908104720 2024-03-08T20:57:43.628 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:43.643+0000 7fe90d200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe90810f240 con 0x7fe9081050e0 2024-03-08T20:57:43.628 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:43.643+0000 7fe905e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe908103530 0x7fe908199780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:57:43.628 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:43.643+0000 7fe905400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe908104720 0x7fe908199cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:57:43.628 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:43.643+0000 7fe905e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe908103530 0x7fe908199780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:45592/0 (socket says 172.21.15.107:45592) 2024-03-08T20:57:43.628 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:43.643+0000 7fe905e00640 1 -- 172.21.15.107:0/1510594529 learned_addr learned my addr 172.21.15.107:0/1510594529 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T20:57:43.628 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:43.643+0000 7fe906800640 1 --2- 172.21.15.107:0/1510594529 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe9081050e0 0x7fe9081a0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:57:43.628 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:43.643+0000 7fe905e00640 1 -- 172.21.15.107:0/1510594529 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe9081050e0 msgr2=0x7fe9081a0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:57:43.628 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:43.643+0000 7fe905e00640 1 --2- 172.21.15.107:0/1510594529 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe9081050e0 0x7fe9081a0d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:43.629 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:43.643+0000 7fe905e00640 1 -- 172.21.15.107:0/1510594529 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe908104720 msgr2=0x7fe908199cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:57:43.629 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:43.643+0000 7fe905e00640 1 --2- 172.21.15.107:0/1510594529 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe908104720 0x7fe908199cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:43.629 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:43.643+0000 7fe905e00640 1 -- 172.21.15.107:0/1510594529 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9081a13a0 con 0x7fe908103530 2024-03-08T20:57:43.629 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:43.643+0000 7fe906800640 1 --2- 172.21.15.107:0/1510594529 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe9081050e0 0x7fe9081a0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T20:57:43.629 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:43.643+0000 7fe905400640 1 --2- 172.21.15.107:0/1510594529 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe908104720 0x7fe908199cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T20:57:43.629 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:43.644+0000 7fe905e00640 1 --2- 172.21.15.107:0/1510594529 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe908103530 0x7fe908199780 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fe8f8000c00 tx=0x7fe8f80040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:57:43.629 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:43.644+0000 7fe8fea00640 1 -- 172.21.15.107:0/1510594529 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8f8043020 con 0x7fe908103530 2024-03-08T20:57:43.629 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:43.644+0000 7fe90d200640 1 -- 172.21.15.107:0/1510594529 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9081a1630 con 0x7fe908103530 2024-03-08T20:57:43.629 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:43.644+0000 7fe8fea00640 1 -- 172.21.15.107:0/1510594529 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fe8f80044c0 con 0x7fe908103530 2024-03-08T20:57:43.629 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:43.644+0000 7fe8fea00640 1 -- 172.21.15.107:0/1510594529 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8f8003c60 con 0x7fe908103530 2024-03-08T20:57:43.629 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:43.644+0000 7fe90d200640 1 -- 172.21.15.107:0/1510594529 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9081a1bd0 con 0x7fe908103530 2024-03-08T20:57:43.630 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:43.646+0000 7fe90d200640 1 -- 172.21.15.107:0/1510594529 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe90810a870 con 0x7fe908103530 2024-03-08T20:57:43.630 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:43.647+0000 7fe8fea00640 1 -- 172.21.15.107:0/1510594529 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) v1 ==== 137291+0+0 (secure 0 0 0) 0x7fe8f8046020 con 0x7fe908103530 2024-03-08T20:57:43.632 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:43.648+0000 7fe8fea00640 1 --2- 172.21.15.107:0/1510594529 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fe8ec0a2880 0x7fe8ec0a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:57:43.632 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:43.649+0000 7fe8fea00640 1 -- 172.21.15.107:0/1510594529 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fe8f80f0840 con 0x7fe908103530 2024-03-08T20:57:43.632 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:43.649+0000 7fe905400640 1 --2- 172.21.15.107:0/1510594529 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fe8ec0a2880 0x7fe8ec0a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:57:43.633 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:43.649+0000 7fe905400640 1 --2- 172.21.15.107:0/1510594529 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fe8ec0a2880 0x7fe8ec0a4d40 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fe90819ac90 tx=0x7fe8f000b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:57:43.640 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:43.657+0000 7fe8fea00640 1 -- 172.21.15.107:0/1510594529 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7fe8f804d020 con 0x7fe908103530 2024-03-08T20:57:43.647 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:43 smithi107 ceph-mon[31381]: pgmap v353: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-08T20:57:43.766 INFO:journalctl@ceph.mds.b.smithi149.stdout:Mar 08 20:57:43 smithi149 systemd[1]: Starting Ceph mds.b for dc20f0f8-dd8c-11ee-95c6-87774f69a715... 2024-03-08T20:57:43.879 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:43 smithi204 ceph-mon[34935]: pgmap v353: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-08T20:57:43.880 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:43.897+0000 7fe90d200640 1 -- 172.21.15.107:0/1510594529 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe90802ce70 con 0x7fe908103530 2024-03-08T20:57:43.882 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:43.898+0000 7fe8fea00640 1 -- 172.21.15.107:0/1510594529 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+9254 (secure 0 0 0) 0x7fe8f80bdce0 con 0x7fe908103530 2024-03-08T20:57:43.882 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-08T20:57:43.882 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/133582743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":133582743},{"type":"v1","addr":"172.21.15.107:6835","nonce":133582743}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/3798982337","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":3798982337},{"type":"v1","addr":"172.21.15.107:6837","nonce":3798982337}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14625,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6839/3070059000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":3070059000},{"type":"v1","addr":"172.21.15.107:6839","nonce":3070059000}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14637,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6841/747096194","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6840","nonce":747096194},{"type":"v1","addr":"172.21.15.107:6841","nonce":747096194}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/3404762782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":3404762782},{"type":"v1","addr":"172.21.15.204:6833","nonce":3404762782}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24430,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/2072518651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":2072518651},{"type":"v1","addr":"172.21.15.204:6835","nonce":2072518651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24442,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6837/42644905","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":42644905},{"type":"v1","addr":"172.21.15.204:6837","nonce":42644905}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24457,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6839/1857316822","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":1857316822},{"type":"v1","addr":"172.21.15.204:6839","nonce":1857316822}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":34397,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/1962161726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":1962161726},{"type":"v1","addr":"172.21.15.149:6833","nonce":1962161726}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":34415,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/442657066","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":442657066},{"type":"v1","addr":"172.21.15.149:6835","nonce":442657066}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":34445,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6837/914546237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":914546237},{"type":"v1","addr":"172.21.15.149:6837","nonce":914546237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9}],"filesystems":[]} 2024-03-08T20:57:43.883 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 12 2024-03-08T20:57:43.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:43.901+0000 7fe90d200640 1 -- 172.21.15.107:0/1510594529 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fe8ec0a2880 msgr2=0x7fe8ec0a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:57:43.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:43.901+0000 7fe90d200640 1 --2- 172.21.15.107:0/1510594529 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fe8ec0a2880 0x7fe8ec0a4d40 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fe90819ac90 tx=0x7fe8f000b390 comp rx=0 tx=0).stop 2024-03-08T20:57:43.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:43.901+0000 7fe90d200640 1 -- 172.21.15.107:0/1510594529 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe908103530 msgr2=0x7fe908199780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:57:43.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:43.901+0000 7fe90d200640 1 --2- 172.21.15.107:0/1510594529 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe908103530 0x7fe908199780 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fe8f8000c00 tx=0x7fe8f80040c0 comp rx=0 tx=0).stop 2024-03-08T20:57:43.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:43.901+0000 7fe90d200640 1 -- 172.21.15.107:0/1510594529 shutdown_connections 2024-03-08T20:57:43.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:43.901+0000 7fe90d200640 1 --2- 172.21.15.107:0/1510594529 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fe8ec0a2880 0x7fe8ec0a4d40 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:43.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:43.901+0000 7fe90d200640 1 --2- 172.21.15.107:0/1510594529 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe9081050e0 0x7fe9081a0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:43.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:43.901+0000 7fe90d200640 1 --2- 172.21.15.107:0/1510594529 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe908104720 0x7fe908199cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:43.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:43.901+0000 7fe90d200640 1 --2- 172.21.15.107:0/1510594529 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe908103530 0x7fe908199780 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:43.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:43.901+0000 7fe90d200640 1 -- 172.21.15.107:0/1510594529 >> 172.21.15.107:0/1510594529 conn(0x7fe9080feae0 msgr2=0x7fe908107a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:57:43.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:43.902+0000 7fe90d200640 1 -- 172.21.15.107:0/1510594529 shutdown_connections 2024-03-08T20:57:43.887 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:43.902+0000 7fe90d200640 1 -- 172.21.15.107:0/1510594529 wait complete. 2024-03-08T20:57:44.094 INFO:journalctl@ceph.mds.b.smithi149.stdout:Mar 08 20:57:43 smithi149 podman[65602]: 2024-03-08 20:57:43.740297325 +0000 UTC m=+0.033525789 image pull a272146cb810b1fd4a53627f019472132e30966d5c25b01bfc44a001295f1249 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd 2024-03-08T20:57:44.094 INFO:journalctl@ceph.mds.b.smithi149.stdout:Mar 08 20:57:43 smithi149 podman[65602]: 2024-03-08 20:57:43.855059794 +0000 UTC m=+0.148288295 container create e8a745fa0d1435df404db22d43b0e4216e2aadefca5629219b664e920814c4de (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=rishabh-2024mar8-7d63482, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240307, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=) 2024-03-08T20:57:44.094 INFO:journalctl@ceph.mds.b.smithi149.stdout:Mar 08 20:57:44 smithi149 podman[65602]: 2024-03-08 20:57:44.109168388 +0000 UTC m=+0.402396896 container init e8a745fa0d1435df404db22d43b0e4216e2aadefca5629219b664e920814c4de (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-b, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=rishabh-2024mar8-7d63482) 2024-03-08T20:57:44.402 INFO:journalctl@ceph.mds.b.smithi149.stdout:Mar 08 20:57:44 smithi149 podman[65602]: 2024-03-08 20:57:44.118610474 +0000 UTC m=+0.411838945 container start e8a745fa0d1435df404db22d43b0e4216e2aadefca5629219b664e920814c4de (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-b, org.label-schema.build-date=20240307, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=rishabh-2024mar8-7d63482, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, ceph=True) 2024-03-08T20:57:44.402 INFO:journalctl@ceph.mds.b.smithi149.stdout:Mar 08 20:57:44 smithi149 ceph-mds[65620]: -- 172.21.15.149:0/701399436 <== mon.2 v2:172.21.15.149:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x556d1945da40 con 0x556d19430800 2024-03-08T20:57:44.402 INFO:journalctl@ceph.mds.b.smithi149.stdout:Mar 08 20:57:44 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-b[65616]: starting mds.b at 2024-03-08T20:57:44.402 INFO:journalctl@ceph.mds.b.smithi149.stdout:Mar 08 20:57:44 smithi149 bash[65602]: e8a745fa0d1435df404db22d43b0e4216e2aadefca5629219b664e920814c4de 2024-03-08T20:57:44.402 INFO:journalctl@ceph.mds.b.smithi149.stdout:Mar 08 20:57:44 smithi149 systemd[1]: Started Ceph mds.b for dc20f0f8-dd8c-11ee-95c6-87774f69a715. 2024-03-08T20:57:44.718 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:44 smithi107 ceph-mon[31381]: from='client.? 172.21.15.107:0/1510594529' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-08T20:57:44.719 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:44 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:57:44.719 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:44 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:57:44.719 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:44 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:57:44.719 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:44 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:57:44.719 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:44 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:57:44.766 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:44 smithi149 ceph-mon[35042]: from='client.? 172.21.15.107:0/1510594529' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-08T20:57:44.766 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:44 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:57:44.766 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:44 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:57:44.766 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:44 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:57:44.766 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:44 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:57:44.766 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:44 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:57:44.879 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:44 smithi204 ceph-mon[34935]: from='client.? 172.21.15.107:0/1510594529' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-08T20:57:44.879 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:44 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:57:44.879 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:44 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:57:44.879 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:44 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:57:44.879 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:44 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:57:44.879 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:44 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:57:45.718 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:45 smithi107 ceph-mon[31381]: pgmap v354: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-08T20:57:45.718 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:45 smithi107 ceph-mon[31381]: mds.? [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] up:boot 2024-03-08T20:57:45.719 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:45 smithi107 ceph-mon[31381]: fsmap 12 up:standby 2024-03-08T20:57:45.719 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:45 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-08T20:57:45.719 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:45 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T20:57:45.719 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:45 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:57:45.766 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:45 smithi149 ceph-mon[35042]: pgmap v354: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-08T20:57:45.766 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:45 smithi149 ceph-mon[35042]: mds.? [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] up:boot 2024-03-08T20:57:45.766 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:45 smithi149 ceph-mon[35042]: fsmap 12 up:standby 2024-03-08T20:57:45.766 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:45 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-08T20:57:45.766 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:45 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T20:57:45.766 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:45 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:57:45.879 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:45 smithi204 ceph-mon[34935]: pgmap v354: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-08T20:57:45.879 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:45 smithi204 ceph-mon[34935]: mds.? [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] up:boot 2024-03-08T20:57:45.879 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:45 smithi204 ceph-mon[34935]: fsmap 12 up:standby 2024-03-08T20:57:45.879 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:45 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-08T20:57:45.879 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:45 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T20:57:45.879 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:45 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:57:45.905 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-08T20:57:46.114 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:46.130+0000 7f199ae00640 1 -- 172.21.15.107:0/321397855 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1994100cb0 msgr2=0x7f1994101130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:57:46.114 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:46.130+0000 7f199ae00640 1 --2- 172.21.15.107:0/321397855 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1994100cb0 0x7f1994101130 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f198c009900 tx=0x7f198c02bc30 comp rx=0 tx=0).stop 2024-03-08T20:57:46.114 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:46.130+0000 7f199ae00640 1 -- 172.21.15.107:0/321397855 shutdown_connections 2024-03-08T20:57:46.114 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:46.130+0000 7f199ae00640 1 --2- 172.21.15.107:0/321397855 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1994100cb0 0x7f1994101130 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:46.114 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:46.130+0000 7f199ae00640 1 --2- 172.21.15.107:0/321397855 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1994100310 0x7f1994100770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:46.114 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:46.130+0000 7f199ae00640 1 --2- 172.21.15.107:0/321397855 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f19940ff9b0 0x7f19940ffdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:46.114 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:46.130+0000 7f199ae00640 1 -- 172.21.15.107:0/321397855 >> 172.21.15.107:0/321397855 conn(0x7f19940fb5d0 msgr2=0x7f19940fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:57:46.114 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:46.130+0000 7f199ae00640 1 -- 172.21.15.107:0/321397855 shutdown_connections 2024-03-08T20:57:46.114 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:46.130+0000 7f199ae00640 1 -- 172.21.15.107:0/321397855 wait complete. 2024-03-08T20:57:46.117 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:46.131+0000 7f199ae00640 1 Processor -- start 2024-03-08T20:57:46.117 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:46.131+0000 7f199ae00640 1 -- start start 2024-03-08T20:57:46.117 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:46.131+0000 7f199ae00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f19940ff9b0 0x7f1994199510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:57:46.117 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:46.132+0000 7f199ae00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1994100310 0x7f1994199a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:57:46.117 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:46.132+0000 7f199ae00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1994100cb0 0x7f19941a0ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:57:46.117 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:46.132+0000 7f199ae00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f199410ee90 con 0x7f1994100cb0 2024-03-08T20:57:46.117 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:46.132+0000 7f199ae00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f199410ed10 con 0x7f19940ff9b0 2024-03-08T20:57:46.117 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:46.132+0000 7f199ae00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f199410f010 con 0x7f1994100310 2024-03-08T20:57:46.117 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:46.132+0000 7f199a400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1994100cb0 0x7f19941a0ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:57:46.117 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:46.132+0000 7f199a400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1994100cb0 0x7f19941a0ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:36842/0 (socket says 172.21.15.107:36842) 2024-03-08T20:57:46.117 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:46.132+0000 7f199a400640 1 -- 172.21.15.107:0/4382934 learned_addr learned my addr 172.21.15.107:0/4382934 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T20:57:46.117 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:46.132+0000 7f1999a00640 1 --2- 172.21.15.107:0/4382934 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f19940ff9b0 0x7f1994199510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:57:46.118 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:46.132+0000 7f1999000640 1 --2- 172.21.15.107:0/4382934 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1994100310 0x7f1994199a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:57:46.118 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:46.132+0000 7f199a400640 1 -- 172.21.15.107:0/4382934 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1994100310 msgr2=0x7f1994199a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:57:46.118 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:46.132+0000 7f199a400640 1 --2- 172.21.15.107:0/4382934 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1994100310 0x7f1994199a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:46.118 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:46.132+0000 7f199a400640 1 -- 172.21.15.107:0/4382934 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f19940ff9b0 msgr2=0x7f1994199510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:57:46.118 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:46.132+0000 7f199a400640 1 --2- 172.21.15.107:0/4382934 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f19940ff9b0 0x7f1994199510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:46.118 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:46.132+0000 7f199a400640 1 -- 172.21.15.107:0/4382934 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19941a1130 con 0x7f1994100cb0 2024-03-08T20:57:46.118 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:46.132+0000 7f1999000640 1 --2- 172.21.15.107:0/4382934 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1994100310 0x7f1994199a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T20:57:46.118 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:46.133+0000 7f199a400640 1 --2- 172.21.15.107:0/4382934 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1994100cb0 0x7f19941a0ac0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f198c009230 tx=0x7f198c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:57:46.118 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:46.133+0000 7f1992000640 1 -- 172.21.15.107:0/4382934 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f198c043020 con 0x7f1994100cb0 2024-03-08T20:57:46.118 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:46.133+0000 7f199ae00640 1 -- 172.21.15.107:0/4382934 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19941a13c0 con 0x7f1994100cb0 2024-03-08T20:57:46.118 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:46.133+0000 7f1992000640 1 -- 172.21.15.107:0/4382934 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f198c004aa0 con 0x7f1994100cb0 2024-03-08T20:57:46.118 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:46.133+0000 7f1992000640 1 -- 172.21.15.107:0/4382934 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f198c003c60 con 0x7f1994100cb0 2024-03-08T20:57:46.118 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:46.133+0000 7f199ae00640 1 -- 172.21.15.107:0/4382934 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19941a1960 con 0x7f1994100cb0 2024-03-08T20:57:46.119 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:46.134+0000 7f199ae00640 1 -- 172.21.15.107:0/4382934 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f199410a700 con 0x7f1994100cb0 2024-03-08T20:57:46.120 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:46.137+0000 7f1992000640 1 -- 172.21.15.107:0/4382934 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f198c046020 con 0x7f1994100cb0 2024-03-08T20:57:46.122 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:46.138+0000 7f1992000640 1 --2- 172.21.15.107:0/4382934 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f197c0a2800 0x7f197c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:57:46.122 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:46.138+0000 7f1992000640 1 -- 172.21.15.107:0/4382934 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f198c0f0a90 con 0x7f1994100cb0 2024-03-08T20:57:46.122 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:46.139+0000 7f1999a00640 1 --2- 172.21.15.107:0/4382934 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f197c0a2800 0x7f197c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:57:46.122 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:46.139+0000 7f1999a00640 1 --2- 172.21.15.107:0/4382934 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f197c0a2800 0x7f197c0a4cc0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f1988005fd0 tx=0x7f1988009520 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:57:46.128 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:46.144+0000 7f1992000640 1 -- 172.21.15.107:0/4382934 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f198c04d020 con 0x7f1994100cb0 2024-03-08T20:57:46.370 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:46.386+0000 7f199ae00640 1 -- 172.21.15.107:0/4382934 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1994061ef0 con 0x7f1994100cb0 2024-03-08T20:57:46.371 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:46.388+0000 7f1992000640 1 -- 172.21.15.107:0/4382934 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+10043 (secure 0 0 0) 0x7f198c0bdf70 con 0x7f1994100cb0 2024-03-08T20:57:46.371 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-08T20:57:46.371 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/133582743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":133582743},{"type":"v1","addr":"172.21.15.107:6835","nonce":133582743}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/3798982337","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":3798982337},{"type":"v1","addr":"172.21.15.107:6837","nonce":3798982337}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14625,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6839/3070059000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":3070059000},{"type":"v1","addr":"172.21.15.107:6839","nonce":3070059000}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14637,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6841/747096194","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6840","nonce":747096194},{"type":"v1","addr":"172.21.15.107:6841","nonce":747096194}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6833/3404762782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":3404762782},{"type":"v1","addr":"172.21.15.204:6833","nonce":3404762782}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24430,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6835/2072518651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":2072518651},{"type":"v1","addr":"172.21.15.204:6835","nonce":2072518651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24442,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6837/42644905","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":42644905},{"type":"v1","addr":"172.21.15.204:6837","nonce":42644905}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24457,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.204:6839/1857316822","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":1857316822},{"type":"v1","addr":"172.21.15.204:6839","nonce":1857316822}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":34397,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6833/1962161726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":1962161726},{"type":"v1","addr":"172.21.15.149:6833","nonce":1962161726}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":34415,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/442657066","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":442657066},{"type":"v1","addr":"172.21.15.149:6835","nonce":442657066}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":34445,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6837/914546237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":914546237},{"type":"v1","addr":"172.21.15.149:6837","nonce":914546237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":34475,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6839/55290596","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":55290596},{"type":"v1","addr":"172.21.15.149:6839","nonce":55290596}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13}],"filesystems":[]} 2024-03-08T20:57:46.372 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 13 2024-03-08T20:57:46.374 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:46.391+0000 7f199ae00640 1 -- 172.21.15.107:0/4382934 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f197c0a2800 msgr2=0x7f197c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:57:46.374 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:46.391+0000 7f199ae00640 1 --2- 172.21.15.107:0/4382934 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f197c0a2800 0x7f197c0a4cc0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f1988005fd0 tx=0x7f1988009520 comp rx=0 tx=0).stop 2024-03-08T20:57:46.374 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:46.391+0000 7f199ae00640 1 -- 172.21.15.107:0/4382934 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1994100cb0 msgr2=0x7f19941a0ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:57:46.375 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:46.391+0000 7f199ae00640 1 --2- 172.21.15.107:0/4382934 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1994100cb0 0x7f19941a0ac0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f198c009230 tx=0x7f198c0042f0 comp rx=0 tx=0).stop 2024-03-08T20:57:46.375 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:46.391+0000 7f199ae00640 1 -- 172.21.15.107:0/4382934 shutdown_connections 2024-03-08T20:57:46.375 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:46.391+0000 7f199ae00640 1 --2- 172.21.15.107:0/4382934 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f197c0a2800 0x7f197c0a4cc0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:46.375 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:46.391+0000 7f199ae00640 1 --2- 172.21.15.107:0/4382934 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1994100cb0 0x7f19941a0ac0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:46.375 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:46.391+0000 7f199ae00640 1 --2- 172.21.15.107:0/4382934 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1994100310 0x7f1994199a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:46.375 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:46.391+0000 7f199ae00640 1 --2- 172.21.15.107:0/4382934 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f19940ff9b0 0x7f1994199510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:46.375 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:46.391+0000 7f199ae00640 1 -- 172.21.15.107:0/4382934 >> 172.21.15.107:0/4382934 conn(0x7f19940fb5d0 msgr2=0x7f19940fd0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:57:46.375 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:46.391+0000 7f199ae00640 1 -- 172.21.15.107:0/4382934 shutdown_connections 2024-03-08T20:57:46.375 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:46.391+0000 7f199ae00640 1 -- 172.21.15.107:0/4382934 wait complete. 2024-03-08T20:57:46.394 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-08T20:57:46.599 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:46.615+0000 7f26d7c00640 1 -- 172.21.15.107:0/2751225735 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f26d0075cb0 msgr2=0x7f26d00760d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:57:46.599 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:46.615+0000 7f26d7c00640 1 --2- 172.21.15.107:0/2751225735 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f26d0075cb0 0x7f26d00760d0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f26c4009900 tx=0x7f26c402bc30 comp rx=0 tx=0).stop 2024-03-08T20:57:46.600 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:46.615+0000 7f26d7c00640 1 -- 172.21.15.107:0/2751225735 shutdown_connections 2024-03-08T20:57:46.600 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:46.616+0000 7f26d7c00640 1 --2- 172.21.15.107:0/2751225735 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f26d0077890 0x7f26d0078130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:46.600 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:46.616+0000 7f26d7c00640 1 --2- 172.21.15.107:0/2751225735 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f26d0076ef0 0x7f26d0077350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:46.600 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:46.616+0000 7f26d7c00640 1 --2- 172.21.15.107:0/2751225735 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f26d0075cb0 0x7f26d00760d0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:46.600 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:46.616+0000 7f26d7c00640 1 -- 172.21.15.107:0/2751225735 >> 172.21.15.107:0/2751225735 conn(0x7f26d00feaa0 msgr2=0x7f26d0100ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:57:46.600 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:46.616+0000 7f26d7c00640 1 -- 172.21.15.107:0/2751225735 shutdown_connections 2024-03-08T20:57:46.600 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:46.616+0000 7f26d7c00640 1 -- 172.21.15.107:0/2751225735 wait complete. 2024-03-08T20:57:46.602 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:46.617+0000 7f26d7c00640 1 Processor -- start 2024-03-08T20:57:46.602 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:46.617+0000 7f26d7c00640 1 -- start start 2024-03-08T20:57:46.602 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:46.617+0000 7f26d7c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f26d0075cb0 0x7f26d019dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:57:46.602 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:46.617+0000 7f26d7c00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f26d0076ef0 0x7f26d019e110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:57:46.602 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:46.617+0000 7f26d7c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f26d0077890 0x7f26d01a5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:57:46.602 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:46.617+0000 7f26d7c00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26d01134f0 con 0x7f26d0075cb0 2024-03-08T20:57:46.602 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:46.617+0000 7f26d7c00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26d0113370 con 0x7f26d0076ef0 2024-03-08T20:57:46.602 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:46.617+0000 7f26d7c00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26d0113670 con 0x7f26d0077890 2024-03-08T20:57:46.602 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:46.617+0000 7f26d5200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f26d0075cb0 0x7f26d019dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:57:46.602 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:46.617+0000 7f26d5200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f26d0075cb0 0x7f26d019dbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:36860/0 (socket says 172.21.15.107:36860) 2024-03-08T20:57:46.602 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:46.617+0000 7f26d5200640 1 -- 172.21.15.107:0/1319163914 learned_addr learned my addr 172.21.15.107:0/1319163914 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T20:57:46.602 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:46.618+0000 7f26cfe00640 1 --2- 172.21.15.107:0/1319163914 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f26d0076ef0 0x7f26d019e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:57:46.602 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:46.618+0000 7f26d5c00640 1 --2- 172.21.15.107:0/1319163914 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f26d0077890 0x7f26d01a5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:57:46.603 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:46.618+0000 7f26d5200640 1 -- 172.21.15.107:0/1319163914 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f26d0077890 msgr2=0x7f26d01a5180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:57:46.603 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:46.618+0000 7f26d5200640 1 --2- 172.21.15.107:0/1319163914 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f26d0077890 0x7f26d01a5180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:46.603 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:46.618+0000 7f26d5200640 1 -- 172.21.15.107:0/1319163914 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f26d0076ef0 msgr2=0x7f26d019e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:57:46.603 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:46.618+0000 7f26d5200640 1 --2- 172.21.15.107:0/1319163914 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f26d0076ef0 0x7f26d019e110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:46.603 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:46.618+0000 7f26d5200640 1 -- 172.21.15.107:0/1319163914 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26d01a57f0 con 0x7f26d0075cb0 2024-03-08T20:57:46.603 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:46.618+0000 7f26d5200640 1 --2- 172.21.15.107:0/1319163914 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f26d0075cb0 0x7f26d019dbd0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f26c4000c00 tx=0x7f26c40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:57:46.604 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:46.619+0000 7f26cd600640 1 -- 172.21.15.107:0/1319163914 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26c4043020 con 0x7f26d0075cb0 2024-03-08T20:57:46.604 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:46.619+0000 7f26cd600640 1 -- 172.21.15.107:0/1319163914 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f26c40054c0 con 0x7f26d0075cb0 2024-03-08T20:57:46.604 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:46.619+0000 7f26cd600640 1 -- 172.21.15.107:0/1319163914 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26c4003d00 con 0x7f26d0075cb0 2024-03-08T20:57:46.604 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:46.619+0000 7f26d7c00640 1 -- 172.21.15.107:0/1319163914 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26d01a5a80 con 0x7f26d0075cb0 2024-03-08T20:57:46.604 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:46.619+0000 7f26d7c00640 1 -- 172.21.15.107:0/1319163914 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26d01a5f80 con 0x7f26d0075cb0 2024-03-08T20:57:46.604 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:46.620+0000 7f26d7c00640 1 -- 172.21.15.107:0/1319163914 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26d01100e0 con 0x7f26d0075cb0 2024-03-08T20:57:46.607 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:46.623+0000 7f26cd600640 1 -- 172.21.15.107:0/1319163914 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f26c4046020 con 0x7f26d0075cb0 2024-03-08T20:57:46.608 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:46.625+0000 7f26cd600640 1 --2- 172.21.15.107:0/1319163914 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f26b80a2880 0x7f26b80a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:57:46.608 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:46.625+0000 7f26cd600640 1 -- 172.21.15.107:0/1319163914 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f26c40f0970 con 0x7f26d0075cb0 2024-03-08T20:57:46.608 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:46.625+0000 7f26cfe00640 1 --2- 172.21.15.107:0/1319163914 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f26b80a2880 0x7f26b80a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:57:46.609 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:46.626+0000 7f26cfe00640 1 --2- 172.21.15.107:0/1319163914 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f26b80a2880 0x7f26b80a4d40 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f26d019f0e0 tx=0x7f26bc009110 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:57:46.612 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:46.629+0000 7f26cd600640 1 -- 172.21.15.107:0/1319163914 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f26c404d020 con 0x7f26d0075cb0 2024-03-08T20:57:46.718 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:46 smithi107 ceph-mon[31381]: from='client.? 172.21.15.107:0/4382934' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-08T20:57:46.766 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:46 smithi149 ceph-mon[35042]: from='client.? 172.21.15.107:0/4382934' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-08T20:57:46.829 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:46.845+0000 7f26d7c00640 1 -- 172.21.15.107:0/1319163914 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f26d0061ef0 con 0x7f26d0075cb0 2024-03-08T20:57:46.830 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:46.846+0000 7f26cd600640 1 -- 172.21.15.107:0/1319163914 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) v1 ==== 74+0+18537 (secure 0 0 0) 0x7f26c4004b70 con 0x7f26d0075cb0 2024-03-08T20:57:46.830 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-08T20:57:46.831 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":82,"fsid":"dc20f0f8-dd8c-11ee-95c6-87774f69a715","created":"2024-03-08T20:47:04.190034+0000","modified":"2024-03-08T20:56:44.482997+0000","last_up_change":"2024-03-08T20:56:43.135514+0000","last_in_change":"2024-03-08T20:56:17.577074+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-08T20:51:32.832786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8b1ddc64-4d20-4e85-ac66-f1c5ac599501","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":2453946095},{"type":"v1","addr":"172.21.15.107:6803","nonce":2453946095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":2453946095},{"type":"v1","addr":"172.21.15.107:6805","nonce":2453946095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":2453946095},{"type":"v1","addr":"172.21.15.107:6809","nonce":2453946095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":2453946095},{"type":"v1","addr":"172.21.15.107:6807","nonce":2453946095}]},"public_addr":"172.21.15.107:6803/2453946095","cluster_addr":"172.21.15.107:6805/2453946095","heartbeat_back_addr":"172.21.15.107:6809/2453946095","heartbeat_front_addr":"172.21.15.107:6807/2453946095","state":["exists","up"]},{"osd":1,"uuid":"5a803c09-202a-4777-ba2a-1da8eaf26a54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":3071316419},{"type":"v1","addr":"172.21.15.149:6801","nonce":3071316419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":3071316419},{"type":"v1","addr":"172.21.15.149:6803","nonce":3071316419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":3071316419},{"type":"v1","addr":"172.21.15.149:6807","nonce":3071316419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":3071316419},{"type":"v1","addr":"172.21.15.149:6805","nonce":3071316419}]},"public_addr":"172.21.15.149:6801/3071316419","cluster_addr":"172.21.15.149:6803/3071316419","heartbeat_back_addr":"172.21.15.149:6807/3071316419","heartbeat_front_addr":"172.21.15.149:6805/3071316419","state":["exists","up"]},{"osd":2,"uuid":"555c634f-347e-47dd-a943-e50ae2be073c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":1087852630},{"type":"v1","addr":"172.21.15.204:6801","nonce":1087852630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":1087852630},{"type":"v1","addr":"172.21.15.204:6803","nonce":1087852630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":1087852630},{"type":"v1","addr":"172.21.15.204:6807","nonce":1087852630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":1087852630},{"type":"v1","addr":"172.21.15.204:6805","nonce":1087852630}]},"public_addr":"172.21.15.204:6801/1087852630","cluster_addr":"172.21.15.204:6803/1087852630","heartbeat_back_addr":"172.21.15.204:6807/1087852630","heartbeat_front_addr":"172.21.15.204:6805/1087852630","state":["exists","up"]},{"osd":3,"uuid":"9e433d97-e202-48e0-9ed1-f0988fa14307","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":3123579789},{"type":"v1","addr":"172.21.15.107:6811","nonce":3123579789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":3123579789},{"type":"v1","addr":"172.21.15.107:6813","nonce":3123579789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":3123579789},{"type":"v1","addr":"172.21.15.107:6817","nonce":3123579789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":3123579789},{"type":"v1","addr":"172.21.15.107:6815","nonce":3123579789}]},"public_addr":"172.21.15.107:6811/3123579789","cluster_addr":"172.21.15.107:6813/3123579789","heartbeat_back_addr":"172.21.15.107:6817/3123579789","heartbeat_front_addr":"172.21.15.107:6815/3123579789","state":["exists","up"]},{"osd":4,"uuid":"27c53794-9d2b-487d-bfcf-ac4fd2ab061e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":726286918},{"type":"v1","addr":"172.21.15.149:6809","nonce":726286918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":726286918},{"type":"v1","addr":"172.21.15.149:6811","nonce":726286918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":726286918},{"type":"v1","addr":"172.21.15.149:6815","nonce":726286918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":726286918},{"type":"v1","addr":"172.21.15.149:6813","nonce":726286918}]},"public_addr":"172.21.15.149:6809/726286918","cluster_addr":"172.21.15.149:6811/726286918","heartbeat_back_addr":"172.21.15.149:6815/726286918","heartbeat_front_addr":"172.21.15.149:6813/726286918","state":["exists","up"]},{"osd":5,"uuid":"3a7778fb-46c5-40c0-842e-e988c45ea24d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":3143887820},{"type":"v1","addr":"172.21.15.204:6809","nonce":3143887820}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":3143887820},{"type":"v1","addr":"172.21.15.204:6811","nonce":3143887820}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":3143887820},{"type":"v1","addr":"172.21.15.204:6815","nonce":3143887820}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":3143887820},{"type":"v1","addr":"172.21.15.204:6813","nonce":3143887820}]},"public_addr":"172.21.15.204:6809/3143887820","cluster_addr":"172.21.15.204:6811/3143887820","heartbeat_back_addr":"172.21.15.204:6815/3143887820","heartbeat_front_addr":"172.21.15.204:6813/3143887820","state":["exists","up"]},{"osd":6,"uuid":"d2e05a56-be4f-4435-9b76-ff9aa01e4ac3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":2912452950},{"type":"v1","addr":"172.21.15.107:6819","nonce":2912452950}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":2912452950},{"type":"v1","addr":"172.21.15.107:6821","nonce":2912452950}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":2912452950},{"type":"v1","addr":"172.21.15.107:6825","nonce":2912452950}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":2912452950},{"type":"v1","addr":"172.21.15.107:6823","nonce":2912452950}]},"public_addr":"172.21.15.107:6819/2912452950","cluster_addr":"172.21.15.107:6821/2912452950","heartbeat_back_addr":"172.21.15.107:6825/2912452950","heartbeat_front_addr":"172.21.15.107:6823/2912452950","state":["exists","up"]},{"osd":7,"uuid":"a7dd26ee-c118-4a9a-9e19-142c2680902d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":3835863809},{"type":"v1","addr":"172.21.15.149:6817","nonce":3835863809}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":3835863809},{"type":"v1","addr":"172.21.15.149:6819","nonce":3835863809}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":3835863809},{"type":"v1","addr":"172.21.15.149:6823","nonce":3835863809}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":3835863809},{"type":"v1","addr":"172.21.15.149:6821","nonce":3835863809}]},"public_addr":"172.21.15.149:6817/3835863809","cluster_addr":"172.21.15.149:6819/3835863809","heartbeat_back_addr":"172.21.15.149:6823/3835863809","heartbeat_front_addr":"172.21.15.149:6821/3835863809","state":["exists","up"]},{"osd":8,"uuid":"baaaa8dd-0e1f-4bae-86bf-9e429d985f23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":1010601477},{"type":"v1","addr":"172.21.15.204:6817","nonce":1010601477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":1010601477},{"type":"v1","addr":"172.21.15.204:6819","nonce":1010601477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":1010601477},{"type":"v1","addr":"172.21.15.204:6823","nonce":1010601477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":1010601477},{"type":"v1","addr":"172.21.15.204:6821","nonce":1010601477}]},"public_addr":"172.21.15.204:6817/1010601477","cluster_addr":"172.21.15.204:6819/1010601477","heartbeat_back_addr":"172.21.15.204:6823/1010601477","heartbeat_front_addr":"172.21.15.204:6821/1010601477","state":["exists","up"]},{"osd":9,"uuid":"8df80dfa-e7ff-410e-824c-b1dce111b02f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":3334759819},{"type":"v1","addr":"172.21.15.107:6827","nonce":3334759819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3334759819},{"type":"v1","addr":"172.21.15.107:6829","nonce":3334759819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":3334759819},{"type":"v1","addr":"172.21.15.107:6833","nonce":3334759819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":3334759819},{"type":"v1","addr":"172.21.15.107:6831","nonce":3334759819}]},"public_addr":"172.21.15.107:6827/3334759819","cluster_addr":"172.21.15.107:6829/3334759819","heartbeat_back_addr":"172.21.15.107:6833/3334759819","heartbeat_front_addr":"172.21.15.107:6831/3334759819","state":["exists","up"]},{"osd":10,"uuid":"338945b2-20c0-40b2-aaca-062cd4adee04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":3469284631},{"type":"v1","addr":"172.21.15.149:6825","nonce":3469284631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":3469284631},{"type":"v1","addr":"172.21.15.149:6827","nonce":3469284631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":3469284631},{"type":"v1","addr":"172.21.15.149:6831","nonce":3469284631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":3469284631},{"type":"v1","addr":"172.21.15.149:6829","nonce":3469284631}]},"public_addr":"172.21.15.149:6825/3469284631","cluster_addr":"172.21.15.149:6827/3469284631","heartbeat_back_addr":"172.21.15.149:6831/3469284631","heartbeat_front_addr":"172.21.15.149:6829/3469284631","state":["exists","up"]},{"osd":11,"uuid":"acccfe4f-d307-4a64-8f4d-fba27bfbe09d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":3097772473},{"type":"v1","addr":"172.21.15.204:6825","nonce":3097772473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":3097772473},{"type":"v1","addr":"172.21.15.204:6827","nonce":3097772473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":3097772473},{"type":"v1","addr":"172.21.15.204:6831","nonce":3097772473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":3097772473},{"type":"v1","addr":"172.21.15.204:6829","nonce":3097772473}]},"public_addr":"172.21.15.204:6825/3097772473","cluster_addr":"172.21.15.204:6827/3097772473","heartbeat_back_addr":"172.21.15.204:6831/3097772473","heartbeat_front_addr":"172.21.15.204:6829/3097772473","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T20:50:25.862753+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T20:50:58.167663+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T20:51:30.312816+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T20:52:05.487917+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T20:52:39.348354+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T20:53:12.899002+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T20:53:47.630837+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T20:54:21.444552+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T20:54:57.101160+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T20:55:31.841786+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T20:56:05.827161+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T20:56:40.611097+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.107:0/1514666962":"2024-03-09T20:48:20.149406+0000","172.21.15.107:0/1886016707":"2024-03-09T20:48:20.149406+0000","172.21.15.107:6801/426372446":"2024-03-09T20:48:20.149406+0000","172.21.15.107:6800/426372446":"2024-03-09T20:48:20.149406+0000","172.21.15.107:0/2392547434":"2024-03-09T20:47:45.706790+0000","172.21.15.107:0/1206995985":"2024-03-09T20:47:45.706790+0000","172.21.15.107:6800/3549936519":"2024-03-09T20:47:45.706790+0000","172.21.15.107:6801/3549936519":"2024-03-09T20:47:45.706790+0000","172.21.15.107:0/2482452215":"2024-03-09T20:47:45.706790+0000","172.21.15.107:0/808124991":"2024-03-09T20:48:20.149406+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-08T20:57:46.832 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:46.849+0000 7f26d7c00640 1 -- 172.21.15.107:0/1319163914 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f26b80a2880 msgr2=0x7f26b80a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:57:46.832 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:46.849+0000 7f26d7c00640 1 --2- 172.21.15.107:0/1319163914 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f26b80a2880 0x7f26b80a4d40 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f26d019f0e0 tx=0x7f26bc009110 comp rx=0 tx=0).stop 2024-03-08T20:57:46.833 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:46.849+0000 7f26d7c00640 1 -- 172.21.15.107:0/1319163914 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f26d0075cb0 msgr2=0x7f26d019dbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:57:46.833 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:46.849+0000 7f26d7c00640 1 --2- 172.21.15.107:0/1319163914 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f26d0075cb0 0x7f26d019dbd0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f26c4000c00 tx=0x7f26c40042f0 comp rx=0 tx=0).stop 2024-03-08T20:57:46.833 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:46.849+0000 7f26d7c00640 1 -- 172.21.15.107:0/1319163914 shutdown_connections 2024-03-08T20:57:46.833 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:46.849+0000 7f26d7c00640 1 --2- 172.21.15.107:0/1319163914 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f26b80a2880 0x7f26b80a4d40 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:46.833 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:46.849+0000 7f26d7c00640 1 --2- 172.21.15.107:0/1319163914 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f26d0077890 0x7f26d01a5180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:46.833 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:46.849+0000 7f26d7c00640 1 --2- 172.21.15.107:0/1319163914 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f26d0076ef0 0x7f26d019e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:46.833 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:46.849+0000 7f26d7c00640 1 --2- 172.21.15.107:0/1319163914 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f26d0075cb0 0x7f26d019dbd0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:46.833 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:46.849+0000 7f26d7c00640 1 -- 172.21.15.107:0/1319163914 >> 172.21.15.107:0/1319163914 conn(0x7f26d00feaa0 msgr2=0x7f26d0100240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:57:46.833 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:46.849+0000 7f26d7c00640 1 -- 172.21.15.107:0/1319163914 shutdown_connections 2024-03-08T20:57:46.833 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:46.849+0000 7f26d7c00640 1 -- 172.21.15.107:0/1319163914 wait complete. 2024-03-08T20:57:46.854 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-08T20:51:32.832786+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-08T20:57:46.854 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-08T20:57:46.879 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:46 smithi204 ceph-mon[34935]: from='client.? 172.21.15.107:0/4382934' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-08T20:57:47.064 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:47.080+0000 7f2809c00640 1 -- 172.21.15.107:0/2970925823 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2804073d50 msgr2=0x7f28040741d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:57:47.064 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:47.080+0000 7f2809c00640 1 --2- 172.21.15.107:0/2970925823 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2804073d50 0x7f28040741d0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f27ec009900 tx=0x7f27ec02bc30 comp rx=0 tx=0).stop 2024-03-08T20:57:47.064 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:47.080+0000 7f2809c00640 1 -- 172.21.15.107:0/2970925823 shutdown_connections 2024-03-08T20:57:47.065 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:47.080+0000 7f2809c00640 1 --2- 172.21.15.107:0/2970925823 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f28041091c0 0x7f280410b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:47.065 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:47.080+0000 7f2809c00640 1 --2- 172.21.15.107:0/2970925823 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2804073d50 0x7f28040741d0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:47.065 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:47.080+0000 7f2809c00640 1 --2- 172.21.15.107:0/2970925823 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2804075330 0x7f2804073780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:47.065 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:47.080+0000 7f2809c00640 1 -- 172.21.15.107:0/2970925823 >> 172.21.15.107:0/2970925823 conn(0x7f28040fc870 msgr2=0x7f28040fecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:57:47.065 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:47.081+0000 7f2809c00640 1 -- 172.21.15.107:0/2970925823 shutdown_connections 2024-03-08T20:57:47.065 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:47.081+0000 7f2809c00640 1 -- 172.21.15.107:0/2970925823 wait complete. 2024-03-08T20:57:47.067 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:47.081+0000 7f2809c00640 1 Processor -- start 2024-03-08T20:57:47.067 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:47.082+0000 7f2809c00640 1 -- start start 2024-03-08T20:57:47.067 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:47.082+0000 7f2809c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2804073d50 0x7f28041997a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:57:47.067 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:47.082+0000 7f2809c00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2804075330 0x7f2804199ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:57:47.067 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:47.082+0000 7f2809c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f28041091c0 0x7f28041a0d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:57:47.068 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:47.082+0000 7f2809c00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f280410f040 con 0x7f28041091c0 2024-03-08T20:57:47.068 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:47.082+0000 7f2809c00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f280410eec0 con 0x7f2804075330 2024-03-08T20:57:47.068 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:47.082+0000 7f2809c00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f280410f1c0 con 0x7f2804073d50 2024-03-08T20:57:47.068 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:47.082+0000 7f2803400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f28041091c0 0x7f28041a0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:57:47.068 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:47.082+0000 7f2803400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f28041091c0 0x7f28041a0d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:36872/0 (socket says 172.21.15.107:36872) 2024-03-08T20:57:47.068 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:47.082+0000 7f2803400640 1 -- 172.21.15.107:0/892806931 learned_addr learned my addr 172.21.15.107:0/892806931 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T20:57:47.068 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:47.082+0000 7f2802a00640 1 --2- 172.21.15.107:0/892806931 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2804073d50 0x7f28041997a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:57:47.068 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:47.082+0000 7f27fbe00640 1 --2- 172.21.15.107:0/892806931 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2804075330 0x7f2804199ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:57:47.068 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:47.083+0000 7f2803400640 1 -- 172.21.15.107:0/892806931 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2804073d50 msgr2=0x7f28041997a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:57:47.068 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:47.083+0000 7f2803400640 1 --2- 172.21.15.107:0/892806931 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2804073d50 0x7f28041997a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:47.068 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:47.083+0000 7f2803400640 1 -- 172.21.15.107:0/892806931 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2804075330 msgr2=0x7f2804199ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:57:47.068 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:47.083+0000 7f2803400640 1 --2- 172.21.15.107:0/892806931 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2804075330 0x7f2804199ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:47.068 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:47.083+0000 7f2803400640 1 -- 172.21.15.107:0/892806931 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28041a1370 con 0x7f28041091c0 2024-03-08T20:57:47.069 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:47.083+0000 7f27fbe00640 1 --2- 172.21.15.107:0/892806931 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2804075330 0x7f2804199ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T20:57:47.069 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:47.083+0000 7f2803400640 1 --2- 172.21.15.107:0/892806931 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f28041091c0 0x7f28041a0d00 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f27fc00eb10 tx=0x7f27fc00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:57:47.069 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:47.083+0000 7f27fb400640 1 -- 172.21.15.107:0/892806931 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27fc004b20 con 0x7f28041091c0 2024-03-08T20:57:47.069 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:47.083+0000 7f27fb400640 1 -- 172.21.15.107:0/892806931 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f27fc004cc0 con 0x7f28041091c0 2024-03-08T20:57:47.069 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:47.083+0000 7f2809c00640 1 -- 172.21.15.107:0/892806931 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28041a1660 con 0x7f28041091c0 2024-03-08T20:57:47.069 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:47.083+0000 7f27fb400640 1 -- 172.21.15.107:0/892806931 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27fc010690 con 0x7f28041091c0 2024-03-08T20:57:47.069 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:47.083+0000 7f2809c00640 1 -- 172.21.15.107:0/892806931 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28041a1b30 con 0x7f28041091c0 2024-03-08T20:57:47.070 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:47.085+0000 7f2809c00640 1 -- 172.21.15.107:0/892806931 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2804107ef0 con 0x7f28041091c0 2024-03-08T20:57:47.070 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:47.086+0000 7f27fb400640 1 -- 172.21.15.107:0/892806931 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f27fc005020 con 0x7f28041091c0 2024-03-08T20:57:47.071 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:47.087+0000 7f27fb400640 1 --2- 172.21.15.107:0/892806931 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f27e80a2800 0x7f27e80a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:57:47.071 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:47.088+0000 7f27fb400640 1 -- 172.21.15.107:0/892806931 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f27fc0cebf0 con 0x7f28041091c0 2024-03-08T20:57:47.071 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:47.088+0000 7f2802a00640 1 --2- 172.21.15.107:0/892806931 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f27e80a2800 0x7f27e80a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:57:47.071 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:47.088+0000 7f2802a00640 1 --2- 172.21.15.107:0/892806931 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f27e80a2800 0x7f27e80a4cc0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f27f40095f0 tx=0x7f27f4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:57:47.075 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:47.091+0000 7f27fb400640 1 -- 172.21.15.107:0/892806931 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f27fc09c090 con 0x7f28041091c0 2024-03-08T20:57:47.285 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:47.301+0000 7f2809c00640 1 -- 172.21.15.107:0/892806931 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2804061ef0 con 0x7f28041091c0 2024-03-08T20:57:47.285 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:47.302+0000 7f27fb400640 1 -- 172.21.15.107:0/892806931 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) v1 ==== 93+0+10 (secure 0 0 0) 0x7f27fc09b990 con 0x7f28041091c0 2024-03-08T20:57:47.286 INFO:teuthology.orchestra.run.smithi107.stdout:pg_num: 1 2024-03-08T20:57:47.288 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:47.305+0000 7f2809c00640 1 -- 172.21.15.107:0/892806931 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f27e80a2800 msgr2=0x7f27e80a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:57:47.288 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:47.305+0000 7f2809c00640 1 --2- 172.21.15.107:0/892806931 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f27e80a2800 0x7f27e80a4cc0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f27f40095f0 tx=0x7f27f4005f90 comp rx=0 tx=0).stop 2024-03-08T20:57:47.288 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:47.305+0000 7f2809c00640 1 -- 172.21.15.107:0/892806931 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f28041091c0 msgr2=0x7f28041a0d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:57:47.288 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:47.305+0000 7f2809c00640 1 --2- 172.21.15.107:0/892806931 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f28041091c0 0x7f28041a0d00 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f27fc00eb10 tx=0x7f27fc00c2d0 comp rx=0 tx=0).stop 2024-03-08T20:57:47.289 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:47.305+0000 7f2809c00640 1 -- 172.21.15.107:0/892806931 shutdown_connections 2024-03-08T20:57:47.289 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:47.305+0000 7f2809c00640 1 --2- 172.21.15.107:0/892806931 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f27e80a2800 0x7f27e80a4cc0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:47.289 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:47.305+0000 7f2809c00640 1 --2- 172.21.15.107:0/892806931 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f28041091c0 0x7f28041a0d00 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:47.289 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:47.305+0000 7f2809c00640 1 --2- 172.21.15.107:0/892806931 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2804075330 0x7f2804199ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:47.289 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:47.305+0000 7f2809c00640 1 --2- 172.21.15.107:0/892806931 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2804073d50 0x7f28041997a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:47.289 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:47.305+0000 7f2809c00640 1 -- 172.21.15.107:0/892806931 >> 172.21.15.107:0/892806931 conn(0x7f28040fc870 msgr2=0x7f28041050c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:57:47.289 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:47.305+0000 7f2809c00640 1 -- 172.21.15.107:0/892806931 shutdown_connections 2024-03-08T20:57:47.289 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:47.305+0000 7f2809c00640 1 -- 172.21.15.107:0/892806931 wait complete. 2024-03-08T20:57:47.306 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-03-08T20:57:47.504 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:47.520+0000 7f67d6200640 1 -- 172.21.15.107:0/976369285 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f67d0102d10 msgr2=0x7f67d01095a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:57:47.504 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:47.520+0000 7f67d6200640 1 --2- 172.21.15.107:0/976369285 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f67d0102d10 0x7f67d01095a0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f67c8009be0 tx=0x7f67c8034700 comp rx=0 tx=0).stop 2024-03-08T20:57:47.504 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:47.520+0000 7f67d6200640 1 -- 172.21.15.107:0/976369285 shutdown_connections 2024-03-08T20:57:47.504 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:47.520+0000 7f67d6200640 1 --2- 172.21.15.107:0/976369285 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f67d0102d10 0x7f67d01095a0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:47.504 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:47.520+0000 7f67d6200640 1 --2- 172.21.15.107:0/976369285 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f67d0102350 0x7f67d01027d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:47.505 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:47.520+0000 7f67d6200640 1 --2- 172.21.15.107:0/976369285 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f67d0101160 0x7f67d0101580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:47.505 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:47.520+0000 7f67d6200640 1 -- 172.21.15.107:0/976369285 >> 172.21.15.107:0/976369285 conn(0x7f67d00fc6d0 msgr2=0x7f67d00feb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:57:47.505 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:47.520+0000 7f67d6200640 1 -- 172.21.15.107:0/976369285 shutdown_connections 2024-03-08T20:57:47.505 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:47.520+0000 7f67d6200640 1 -- 172.21.15.107:0/976369285 wait complete. 2024-03-08T20:57:47.507 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:47.522+0000 7f67d6200640 1 Processor -- start 2024-03-08T20:57:47.507 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:47.522+0000 7f67d6200640 1 -- start start 2024-03-08T20:57:47.507 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:47.522+0000 7f67d6200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f67d0101160 0x7f67d0195100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:57:47.507 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:47.522+0000 7f67d6200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f67d0102350 0x7f67d0195640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:57:47.507 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:47.522+0000 7f67d6200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f67d0102d10 0x7f67d019c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:57:47.507 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:47.522+0000 7f67d6200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67d010aab0 con 0x7f67d0101160 2024-03-08T20:57:47.507 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:47.522+0000 7f67d6200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67d010a930 con 0x7f67d0102350 2024-03-08T20:57:47.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:47.522+0000 7f67d6200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67d010ac30 con 0x7f67d0102d10 2024-03-08T20:57:47.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:47.523+0000 7f67d4e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f67d0101160 0x7f67d0195100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:57:47.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:47.523+0000 7f67d4e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f67d0101160 0x7f67d0195100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:36878/0 (socket says 172.21.15.107:36878) 2024-03-08T20:57:47.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:47.523+0000 7f67cfe00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f67d0102350 0x7f67d0195640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:57:47.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:47.523+0000 7f67d4e00640 1 -- 172.21.15.107:0/1278364198 learned_addr learned my addr 172.21.15.107:0/1278364198 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T20:57:47.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:47.523+0000 7f67d5800640 1 --2- 172.21.15.107:0/1278364198 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f67d0102d10 0x7f67d019c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:57:47.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:47.523+0000 7f67d4e00640 1 -- 172.21.15.107:0/1278364198 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f67d0102d10 msgr2=0x7f67d019c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:57:47.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:47.523+0000 7f67d4e00640 1 --2- 172.21.15.107:0/1278364198 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f67d0102d10 0x7f67d019c6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:47.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:47.523+0000 7f67d4e00640 1 -- 172.21.15.107:0/1278364198 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f67d0102350 msgr2=0x7f67d0195640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:57:47.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:47.523+0000 7f67d4e00640 1 --2- 172.21.15.107:0/1278364198 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f67d0102350 0x7f67d0195640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:47.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:47.523+0000 7f67d4e00640 1 -- 172.21.15.107:0/1278364198 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67d019cd20 con 0x7f67d0101160 2024-03-08T20:57:47.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:47.523+0000 7f67cfe00640 1 --2- 172.21.15.107:0/1278364198 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f67d0102350 0x7f67d0195640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T20:57:47.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:47.523+0000 7f67d4e00640 1 --2- 172.21.15.107:0/1278364198 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f67d0101160 0x7f67d0195100 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f67c400e9a0 tx=0x7f67c400ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:57:47.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:47.524+0000 7f67cd600640 1 -- 172.21.15.107:0/1278364198 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67c400cd80 con 0x7f67d0101160 2024-03-08T20:57:47.509 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:47.524+0000 7f67cd600640 1 -- 172.21.15.107:0/1278364198 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f67c40108e0 con 0x7f67d0101160 2024-03-08T20:57:47.509 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:47.524+0000 7f67d6200640 1 -- 172.21.15.107:0/1278364198 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67d019cfb0 con 0x7f67d0101160 2024-03-08T20:57:47.509 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:47.524+0000 7f67cd600640 1 -- 172.21.15.107:0/1278364198 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67c4018950 con 0x7f67d0101160 2024-03-08T20:57:47.509 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:47.524+0000 7f67d6200640 1 -- 172.21.15.107:0/1278364198 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67d019d470 con 0x7f67d0101160 2024-03-08T20:57:47.509 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:47.525+0000 7f67d6200640 1 -- 172.21.15.107:0/1278364198 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67d01085a0 con 0x7f67d0101160 2024-03-08T20:57:47.511 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:47.528+0000 7f67cd600640 1 -- 172.21.15.107:0/1278364198 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f67c4010a90 con 0x7f67d0101160 2024-03-08T20:57:47.513 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:47.529+0000 7f67cd600640 1 --2- 172.21.15.107:0/1278364198 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f67b80a2800 0x7f67b80a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:57:47.513 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:47.529+0000 7f67cd600640 1 -- 172.21.15.107:0/1278364198 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f67c40ce9c0 con 0x7f67d0101160 2024-03-08T20:57:47.513 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:47.530+0000 7f67cfe00640 1 --2- 172.21.15.107:0/1278364198 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f67b80a2800 0x7f67b80a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:57:47.513 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:47.530+0000 7f67cfe00640 1 --2- 172.21.15.107:0/1278364198 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f67b80a2800 0x7f67b80a4cc0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f67bc00d5c0 tx=0x7f67bc00f040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:57:47.520 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:47.537+0000 7f67cd600640 1 -- 172.21.15.107:0/1278364198 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f67c4053260 con 0x7f67d0101160 2024-03-08T20:57:47.718 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:47 smithi107 ceph-mon[31381]: pgmap v355: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-08T20:57:47.718 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:47 smithi107 ceph-mon[31381]: from='client.? 172.21.15.107:0/1319163914' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-08T20:57:47.718 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:47 smithi107 ceph-mon[31381]: from='client.? 172.21.15.107:0/892806931' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-08T20:57:47.730 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:47.747+0000 7f67d6200640 1 -- 172.21.15.107:0/1278364198 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f67d0061f90 con 0x7f67d0101160 2024-03-08T20:57:47.731 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:47.748+0000 7f67cd600640 1 -- 172.21.15.107:0/1278364198 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v82) v1 ==== 84+0+65 (secure 0 0 0) 0x7f67c409be60 con 0x7f67d0101160 2024-03-08T20:57:47.731 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-08T20:57:47.731 INFO:teuthology.orchestra.run.smithi107.stdout:[ 2024-03-08T20:57:47.731 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-08T20:57:47.731 INFO:teuthology.orchestra.run.smithi107.stdout: "poolnum": 1, 2024-03-08T20:57:47.731 INFO:teuthology.orchestra.run.smithi107.stdout: "poolname": ".mgr" 2024-03-08T20:57:47.732 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T20:57:47.732 INFO:teuthology.orchestra.run.smithi107.stdout:] 2024-03-08T20:57:47.734 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:47.750+0000 7f67d6200640 1 -- 172.21.15.107:0/1278364198 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f67b80a2800 msgr2=0x7f67b80a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:57:47.734 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:47.750+0000 7f67d6200640 1 --2- 172.21.15.107:0/1278364198 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f67b80a2800 0x7f67b80a4cc0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f67bc00d5c0 tx=0x7f67bc00f040 comp rx=0 tx=0).stop 2024-03-08T20:57:47.734 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:47.750+0000 7f67d6200640 1 -- 172.21.15.107:0/1278364198 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f67d0101160 msgr2=0x7f67d0195100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:57:47.734 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:47.750+0000 7f67d6200640 1 --2- 172.21.15.107:0/1278364198 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f67d0101160 0x7f67d0195100 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f67c400e9a0 tx=0x7f67c400ee60 comp rx=0 tx=0).stop 2024-03-08T20:57:47.734 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:47.750+0000 7f67d6200640 1 -- 172.21.15.107:0/1278364198 shutdown_connections 2024-03-08T20:57:47.734 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:47.750+0000 7f67d6200640 1 --2- 172.21.15.107:0/1278364198 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f67b80a2800 0x7f67b80a4cc0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:47.734 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:47.750+0000 7f67d6200640 1 --2- 172.21.15.107:0/1278364198 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f67d0102d10 0x7f67d019c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:47.734 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:47.750+0000 7f67d6200640 1 --2- 172.21.15.107:0/1278364198 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f67d0102350 0x7f67d0195640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:47.734 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:47.750+0000 7f67d6200640 1 --2- 172.21.15.107:0/1278364198 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f67d0101160 0x7f67d0195100 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:47.734 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:47.750+0000 7f67d6200640 1 -- 172.21.15.107:0/1278364198 >> 172.21.15.107:0/1278364198 conn(0x7f67d00fc6d0 msgr2=0x7f67d0105770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:57:47.734 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:47.750+0000 7f67d6200640 1 -- 172.21.15.107:0/1278364198 shutdown_connections 2024-03-08T20:57:47.735 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:47.750+0000 7f67d6200640 1 -- 172.21.15.107:0/1278364198 wait complete. 2024-03-08T20:57:47.752 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-03-08T20:57:47.752 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-03-08T20:57:47.765 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:47 smithi149 ceph-mon[35042]: pgmap v355: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-08T20:57:47.765 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:47 smithi149 ceph-mon[35042]: from='client.? 172.21.15.107:0/1319163914' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-08T20:57:47.766 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:47 smithi149 ceph-mon[35042]: from='client.? 172.21.15.107:0/892806931' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-08T20:57:47.879 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:47 smithi204 ceph-mon[34935]: pgmap v355: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-08T20:57:47.879 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:47 smithi204 ceph-mon[34935]: from='client.? 172.21.15.107:0/1319163914' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-08T20:57:47.879 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:47 smithi204 ceph-mon[34935]: from='client.? 172.21.15.107:0/892806931' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-08T20:57:47.943 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:47.959+0000 7f5eeb400640 1 -- 172.21.15.107:0/4022186440 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5ee4075330 msgr2=0x7f5ee4073780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:57:47.943 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:47.959+0000 7f5eeb400640 1 --2- 172.21.15.107:0/4022186440 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5ee4075330 0x7f5ee4073780 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f5ed8009cd0 tx=0x7f5ed802c0f0 comp rx=0 tx=0).stop 2024-03-08T20:57:47.943 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:47.959+0000 7f5eeb400640 1 -- 172.21.15.107:0/4022186440 shutdown_connections 2024-03-08T20:57:47.943 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:47.959+0000 7f5eeb400640 1 --2- 172.21.15.107:0/4022186440 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5ee410d8d0 0x7f5ee410fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:47.943 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:47.959+0000 7f5eeb400640 1 --2- 172.21.15.107:0/4022186440 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5ee4073d50 0x7f5ee40741d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:47.943 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:47.959+0000 7f5eeb400640 1 --2- 172.21.15.107:0/4022186440 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5ee4075330 0x7f5ee4073780 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:47.944 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:47.959+0000 7f5eeb400640 1 -- 172.21.15.107:0/4022186440 >> 172.21.15.107:0/4022186440 conn(0x7f5ee40fc870 msgr2=0x7f5ee40fecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:57:47.944 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:47.959+0000 7f5eeb400640 1 -- 172.21.15.107:0/4022186440 shutdown_connections 2024-03-08T20:57:47.944 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:47.959+0000 7f5eeb400640 1 -- 172.21.15.107:0/4022186440 wait complete. 2024-03-08T20:57:47.946 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:47.960+0000 7f5eeb400640 1 Processor -- start 2024-03-08T20:57:47.946 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:47.960+0000 7f5eeb400640 1 -- start start 2024-03-08T20:57:47.946 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:47.961+0000 7f5eeb400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5ee4073d50 0x7f5ee419dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:57:47.946 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:47.961+0000 7f5eeb400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5ee4075330 0x7f5ee419e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:57:47.946 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:47.961+0000 7f5eeb400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5ee410d8d0 0x7f5ee41a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:57:47.946 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:47.961+0000 7f5eeb400640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ee4113420 con 0x7f5ee410d8d0 2024-03-08T20:57:47.946 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:47.961+0000 7f5eeb400640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ee41132a0 con 0x7f5ee4075330 2024-03-08T20:57:47.946 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:47.961+0000 7f5eeb400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ee41135a0 con 0x7f5ee4073d50 2024-03-08T20:57:47.947 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:47.961+0000 7f5ee9400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5ee410d8d0 0x7f5ee41a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:57:47.947 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:47.961+0000 7f5ee9400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5ee410d8d0 0x7f5ee41a5160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:36886/0 (socket says 172.21.15.107:36886) 2024-03-08T20:57:47.947 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:47.961+0000 7f5ee9400640 1 -- 172.21.15.107:0/4036949512 learned_addr learned my addr 172.21.15.107:0/4036949512 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T20:57:47.947 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:47.961+0000 7f5ee8a00640 1 --2- 172.21.15.107:0/4036949512 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5ee4073d50 0x7f5ee419dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:57:47.947 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:47.961+0000 7f5ee3e00640 1 --2- 172.21.15.107:0/4036949512 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5ee4075330 0x7f5ee419e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:57:47.947 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:47.961+0000 7f5ee9400640 1 -- 172.21.15.107:0/4036949512 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5ee4073d50 msgr2=0x7f5ee419dbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:57:47.947 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:47.961+0000 7f5ee9400640 1 --2- 172.21.15.107:0/4036949512 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5ee4073d50 0x7f5ee419dbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:47.947 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:47.961+0000 7f5ee9400640 1 -- 172.21.15.107:0/4036949512 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5ee4075330 msgr2=0x7f5ee419e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:57:47.947 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:47.961+0000 7f5ee9400640 1 --2- 172.21.15.107:0/4036949512 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5ee4075330 0x7f5ee419e0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:47.947 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:47.961+0000 7f5ee9400640 1 -- 172.21.15.107:0/4036949512 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ee41a57d0 con 0x7f5ee410d8d0 2024-03-08T20:57:47.947 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:47.962+0000 7f5ee9400640 1 --2- 172.21.15.107:0/4036949512 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5ee410d8d0 0x7f5ee41a5160 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f5edc00d750 tx=0x7f5edc00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:57:47.947 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:47.962+0000 7f5ee1600640 1 -- 172.21.15.107:0/4036949512 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5edc017070 con 0x7f5ee410d8d0 2024-03-08T20:57:47.947 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:47.962+0000 7f5ee1600640 1 -- 172.21.15.107:0/4036949512 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f5edc0054d0 con 0x7f5ee410d8d0 2024-03-08T20:57:47.947 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:47.962+0000 7f5eeb400640 1 -- 172.21.15.107:0/4036949512 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ee41a5ac0 con 0x7f5ee410d8d0 2024-03-08T20:57:47.948 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:47.962+0000 7f5ee1600640 1 -- 172.21.15.107:0/4036949512 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5edc004c60 con 0x7f5ee410d8d0 2024-03-08T20:57:47.948 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:47.962+0000 7f5eeb400640 1 -- 172.21.15.107:0/4036949512 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ee41a5fc0 con 0x7f5ee410d8d0 2024-03-08T20:57:47.948 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:47.963+0000 7f5eeb400640 1 -- 172.21.15.107:0/4036949512 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ee4103aa0 con 0x7f5ee410d8d0 2024-03-08T20:57:47.949 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:47.965+0000 7f5ee1600640 1 -- 172.21.15.107:0/4036949512 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f5edc004e00 con 0x7f5ee410d8d0 2024-03-08T20:57:47.950 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:47.967+0000 7f5ee1600640 1 --2- 172.21.15.107:0/4036949512 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f5ecc0a2800 0x7f5ecc0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:57:47.951 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:47.967+0000 7f5ee1600640 1 -- 172.21.15.107:0/4036949512 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f5edc0ce420 con 0x7f5ee410d8d0 2024-03-08T20:57:47.951 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:47.967+0000 7f5ee8a00640 1 --2- 172.21.15.107:0/4036949512 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f5ecc0a2800 0x7f5ecc0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:57:47.951 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:47.968+0000 7f5ee8a00640 1 --2- 172.21.15.107:0/4036949512 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f5ecc0a2800 0x7f5ecc0a4cc0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f5ed802c630 tx=0x7f5ed8032340 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:57:47.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:47.969+0000 7f5ee1600640 1 -- 172.21.15.107:0/4036949512 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f5edc09b8c0 con 0x7f5ee410d8d0 2024-03-08T20:57:48.151 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:48.167+0000 7f5eeb400640 1 -- 172.21.15.107:0/4036949512 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f5ee402ce70 con 0x7f5ee410d8d0 2024-03-08T20:57:48.734 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:48.750+0000 7f5ee1600640 1 -- 172.21.15.107:0/4036949512 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v83) v1 ==== 138+0+0 (secure 0 0 0) 0x7f5edc09b1c0 con 0x7f5ee410d8d0 2024-03-08T20:57:48.734 INFO:teuthology.orchestra.run.smithi107.stderr:pool 'cephfs_metadata' created 2024-03-08T20:57:48.737 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:48.753+0000 7f5eeb400640 1 -- 172.21.15.107:0/4036949512 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f5ecc0a2800 msgr2=0x7f5ecc0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:57:48.737 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:48.753+0000 7f5eeb400640 1 --2- 172.21.15.107:0/4036949512 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f5ecc0a2800 0x7f5ecc0a4cc0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f5ed802c630 tx=0x7f5ed8032340 comp rx=0 tx=0).stop 2024-03-08T20:57:48.737 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:48.753+0000 7f5eeb400640 1 -- 172.21.15.107:0/4036949512 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5ee410d8d0 msgr2=0x7f5ee41a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:57:48.737 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:48.753+0000 7f5eeb400640 1 --2- 172.21.15.107:0/4036949512 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5ee410d8d0 0x7f5ee41a5160 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f5edc00d750 tx=0x7f5edc00dc10 comp rx=0 tx=0).stop 2024-03-08T20:57:48.737 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:48.753+0000 7f5eeb400640 1 -- 172.21.15.107:0/4036949512 shutdown_connections 2024-03-08T20:57:48.737 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:48.753+0000 7f5eeb400640 1 --2- 172.21.15.107:0/4036949512 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f5ecc0a2800 0x7f5ecc0a4cc0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:48.737 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:48.753+0000 7f5eeb400640 1 --2- 172.21.15.107:0/4036949512 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5ee410d8d0 0x7f5ee41a5160 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:48.738 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:48.753+0000 7f5eeb400640 1 --2- 172.21.15.107:0/4036949512 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5ee4075330 0x7f5ee419e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:48.738 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:48.753+0000 7f5eeb400640 1 --2- 172.21.15.107:0/4036949512 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5ee4073d50 0x7f5ee419dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:48.738 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:48.753+0000 7f5eeb400640 1 -- 172.21.15.107:0/4036949512 >> 172.21.15.107:0/4036949512 conn(0x7f5ee40fc870 msgr2=0x7f5ee4100c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:57:48.738 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:48.754+0000 7f5eeb400640 1 -- 172.21.15.107:0/4036949512 shutdown_connections 2024-03-08T20:57:48.738 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:48.754+0000 7f5eeb400640 1 -- 172.21.15.107:0/4036949512 wait complete. 2024-03-08T20:57:48.761 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-03-08T20:57:48.956 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:48.973+0000 7f84fd600640 1 -- 172.21.15.107:0/4269494981 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f84f8104720 msgr2=0x7f84f8104ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:57:48.957 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:48.973+0000 7f84fd600640 1 --2- 172.21.15.107:0/4269494981 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f84f8104720 0x7f84f8104ba0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f84e0009900 tx=0x7f84e002bc30 comp rx=0 tx=0).stop 2024-03-08T20:57:48.957 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:48.973+0000 7f84fd600640 1 -- 172.21.15.107:0/4269494981 shutdown_connections 2024-03-08T20:57:48.957 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:48.973+0000 7f84fd600640 1 --2- 172.21.15.107:0/4269494981 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f84f81050e0 0x7f84f810b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:48.957 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:48.973+0000 7f84fd600640 1 --2- 172.21.15.107:0/4269494981 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f84f8104720 0x7f84f8104ba0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:48.957 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:48.973+0000 7f84fd600640 1 --2- 172.21.15.107:0/4269494981 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f84f8103530 0x7f84f8103950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:48.957 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:48.973+0000 7f84fd600640 1 -- 172.21.15.107:0/4269494981 >> 172.21.15.107:0/4269494981 conn(0x7f84f80feaa0 msgr2=0x7f84f8100ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:57:48.957 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:48.973+0000 7f84fd600640 1 -- 172.21.15.107:0/4269494981 shutdown_connections 2024-03-08T20:57:48.957 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:48.973+0000 7f84fd600640 1 -- 172.21.15.107:0/4269494981 wait complete. 2024-03-08T20:57:48.959 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:48.974+0000 7f84fd600640 1 Processor -- start 2024-03-08T20:57:48.960 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:48.974+0000 7f84fd600640 1 -- start start 2024-03-08T20:57:48.960 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:48.974+0000 7f84fd600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f84f8103530 0x7f84f81996f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:57:48.960 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:48.974+0000 7f84fd600640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f84f8104720 0x7f84f8199c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:57:48.960 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:48.974+0000 7f84fd600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f84f81050e0 0x7f84f81a0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:57:48.960 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:48.974+0000 7f84fd600640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84f810f070 con 0x7f84f81050e0 2024-03-08T20:57:48.960 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:48.974+0000 7f84fd600640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84f810eef0 con 0x7f84f8104720 2024-03-08T20:57:48.960 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:48.974+0000 7f84fd600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84f810f1f0 con 0x7f84f8103530 2024-03-08T20:57:48.960 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:48.975+0000 7f84f7400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f84f81050e0 0x7f84f81a0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:57:48.960 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:48.975+0000 7f84f7400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f84f81050e0 0x7f84f81a0ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:36892/0 (socket says 172.21.15.107:36892) 2024-03-08T20:57:48.960 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:48.975+0000 7f84f7400640 1 -- 172.21.15.107:0/1420170460 learned_addr learned my addr 172.21.15.107:0/1420170460 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T20:57:48.960 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:48.975+0000 7f84f6a00640 1 --2- 172.21.15.107:0/1420170460 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f84f8103530 0x7f84f81996f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:57:48.960 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:48.975+0000 7f84ee000640 1 --2- 172.21.15.107:0/1420170460 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f84f8104720 0x7f84f8199c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:57:48.961 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:48.975+0000 7f84f7400640 1 -- 172.21.15.107:0/1420170460 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f84f8103530 msgr2=0x7f84f81996f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:57:48.961 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:48.975+0000 7f84f7400640 1 --2- 172.21.15.107:0/1420170460 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f84f8103530 0x7f84f81996f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:48.961 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:48.975+0000 7f84f7400640 1 -- 172.21.15.107:0/1420170460 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f84f8104720 msgr2=0x7f84f8199c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:57:48.961 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:48.975+0000 7f84f7400640 1 --2- 172.21.15.107:0/1420170460 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f84f8104720 0x7f84f8199c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:48.961 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:48.975+0000 7f84f7400640 1 -- 172.21.15.107:0/1420170460 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84f81a1310 con 0x7f84f81050e0 2024-03-08T20:57:48.961 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:48.975+0000 7f84f7400640 1 --2- 172.21.15.107:0/1420170460 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f84f81050e0 0x7f84f81a0ca0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f84f000cad0 tx=0x7f84f000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:57:48.961 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:48.976+0000 7f84efe00640 1 -- 172.21.15.107:0/1420170460 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84f000ae20 con 0x7f84f81050e0 2024-03-08T20:57:48.961 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:48.976+0000 7f84efe00640 1 -- 172.21.15.107:0/1420170460 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f84f0005020 con 0x7f84f81050e0 2024-03-08T20:57:48.961 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:48.976+0000 7f84efe00640 1 -- 172.21.15.107:0/1420170460 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84f000f6b0 con 0x7f84f81050e0 2024-03-08T20:57:48.961 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:48.976+0000 7f84fd600640 1 -- 172.21.15.107:0/1420170460 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84f81a1600 con 0x7f84f81050e0 2024-03-08T20:57:48.961 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:48.976+0000 7f84fd600640 1 -- 172.21.15.107:0/1420170460 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84f81a1b00 con 0x7f84f81050e0 2024-03-08T20:57:48.962 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:48.977+0000 7f84fd600640 1 -- 172.21.15.107:0/1420170460 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84f810a920 con 0x7f84f81050e0 2024-03-08T20:57:48.962 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:48.979+0000 7f84efe00640 1 -- 172.21.15.107:0/1420170460 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f84f00051d0 con 0x7f84f81050e0 2024-03-08T20:57:48.964 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:48.980+0000 7f84efe00640 1 --2- 172.21.15.107:0/1420170460 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f84dc0a2800 0x7f84dc0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:57:48.964 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:48.981+0000 7f84efe00640 1 -- 172.21.15.107:0/1420170460 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7497+0+0 (secure 0 0 0) 0x7f84f00cec70 con 0x7f84f81050e0 2024-03-08T20:57:48.964 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:48.981+0000 7f84f6a00640 1 --2- 172.21.15.107:0/1420170460 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f84dc0a2800 0x7f84dc0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:57:48.965 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:48.982+0000 7f84f6a00640 1 --2- 172.21.15.107:0/1420170460 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f84dc0a2800 0x7f84dc0a4cc0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f84e80058c0 tx=0x7f84e8005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:57:48.967 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:48 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:57:48.968 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:48 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:57:48.968 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:48 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:57:48.968 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:48 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:57:48.968 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:48 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:57:48.968 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:48 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:57:48.968 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:48 smithi107 ceph-mon[31381]: from='client.? 172.21.15.107:0/1278364198' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-03-08T20:57:48.968 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:48.985+0000 7f84efe00640 1 -- 172.21.15.107:0/1420170460 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f84f0014030 con 0x7f84f81050e0 2024-03-08T20:57:49.016 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:48 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:57:49.016 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:48 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:57:49.016 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:48 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:57:49.016 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:48 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:57:49.016 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:48 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:57:49.016 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:48 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:57:49.016 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:48 smithi149 ceph-mon[35042]: from='client.? 172.21.15.107:0/1278364198' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-03-08T20:57:49.129 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:48 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:57:49.129 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:48 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:57:49.129 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:48 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:57:49.129 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:48 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:57:49.129 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:48 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:57:49.129 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:48 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:57:49.129 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:48 smithi204 ceph-mon[34935]: from='client.? 172.21.15.107:0/1278364198' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-03-08T20:57:49.168 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:49.185+0000 7f84fd600640 1 -- 172.21.15.107:0/1420170460 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f84f802ce70 con 0x7f84f81050e0 2024-03-08T20:57:49.736 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:49.753+0000 7f84efe00640 1 -- 172.21.15.107:0/1420170460 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v84) v1 ==== 170+0+0 (secure 0 0 0) 0x7f84f00cce20 con 0x7f84f81050e0 2024-03-08T20:57:49.736 INFO:teuthology.orchestra.run.smithi107.stderr:pool 'cephfs_data' created 2024-03-08T20:57:49.739 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:49.756+0000 7f84fd600640 1 -- 172.21.15.107:0/1420170460 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f84dc0a2800 msgr2=0x7f84dc0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:57:49.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:49.756+0000 7f84fd600640 1 --2- 172.21.15.107:0/1420170460 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f84dc0a2800 0x7f84dc0a4cc0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f84e80058c0 tx=0x7f84e8005850 comp rx=0 tx=0).stop 2024-03-08T20:57:49.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:49.756+0000 7f84fd600640 1 -- 172.21.15.107:0/1420170460 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f84f81050e0 msgr2=0x7f84f81a0ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:57:49.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:49.756+0000 7f84fd600640 1 --2- 172.21.15.107:0/1420170460 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f84f81050e0 0x7f84f81a0ca0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f84f000cad0 tx=0x7f84f000cf90 comp rx=0 tx=0).stop 2024-03-08T20:57:49.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:49.756+0000 7f84fd600640 1 -- 172.21.15.107:0/1420170460 shutdown_connections 2024-03-08T20:57:49.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:49.756+0000 7f84fd600640 1 --2- 172.21.15.107:0/1420170460 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f84dc0a2800 0x7f84dc0a4cc0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:49.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:49.756+0000 7f84fd600640 1 --2- 172.21.15.107:0/1420170460 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f84f81050e0 0x7f84f81a0ca0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:49.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:49.756+0000 7f84fd600640 1 --2- 172.21.15.107:0/1420170460 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f84f8104720 0x7f84f8199c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:49.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:49.756+0000 7f84fd600640 1 --2- 172.21.15.107:0/1420170460 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f84f8103530 0x7f84f81996f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:49.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:49.756+0000 7f84fd600640 1 -- 172.21.15.107:0/1420170460 >> 172.21.15.107:0/1420170460 conn(0x7f84f80feaa0 msgr2=0x7f84f8107af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:57:49.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:49.756+0000 7f84fd600640 1 -- 172.21.15.107:0/1420170460 shutdown_connections 2024-03-08T20:57:49.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:49.756+0000 7f84fd600640 1 -- 172.21.15.107:0/1420170460 wait complete. 2024-03-08T20:57:49.776 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-03-08T20:57:49.957 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:49.973+0000 7f7707a00640 1 -- 172.21.15.107:0/3499032917 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f77001070d0 msgr2=0x7f77001094c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:57:49.958 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:49.973+0000 7f7707a00640 1 --2- 172.21.15.107:0/3499032917 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f77001070d0 0x7f77001094c0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f76f8009ca0 tx=0x7f76f802c0f0 comp rx=0 tx=0).stop 2024-03-08T20:57:49.958 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:49.973+0000 7f7707a00640 1 -- 172.21.15.107:0/3499032917 shutdown_connections 2024-03-08T20:57:49.958 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:49.973+0000 7f7707a00640 1 --2- 172.21.15.107:0/3499032917 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f77001070d0 0x7f77001094c0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:49.958 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:49.973+0000 7f7707a00640 1 --2- 172.21.15.107:0/3499032917 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f77001047a0 0x7f7700106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:49.958 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:49.973+0000 7f7707a00640 1 --2- 172.21.15.107:0/3499032917 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7700101e70 0x7f7700104260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:49.958 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:49.973+0000 7f7707a00640 1 -- 172.21.15.107:0/3499032917 >> 172.21.15.107:0/3499032917 conn(0x7f77000fb7a0 msgr2=0x7f77000fdbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:57:49.958 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:49.973+0000 7f7707a00640 1 -- 172.21.15.107:0/3499032917 shutdown_connections 2024-03-08T20:57:49.958 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:49.973+0000 7f7707a00640 1 -- 172.21.15.107:0/3499032917 wait complete. 2024-03-08T20:57:49.960 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:49.974+0000 7f7707a00640 1 Processor -- start 2024-03-08T20:57:49.961 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:49.974+0000 7f7707a00640 1 -- start start 2024-03-08T20:57:49.961 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:49.974+0000 7f7707a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7700101e70 0x7f77001997a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:57:49.961 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:49.974+0000 7f7707a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f77001047a0 0x7f7700199ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:57:49.961 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:49.974+0000 7f7707a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f77001070d0 0x7f77001a0d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:57:49.961 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:49.974+0000 7f7707a00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f770010ce80 con 0x7f77001047a0 2024-03-08T20:57:49.961 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:49.974+0000 7f7707a00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f770010cd00 con 0x7f77001070d0 2024-03-08T20:57:49.961 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:49.974+0000 7f7707a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f770010d000 con 0x7f7700101e70 2024-03-08T20:57:49.961 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:49.975+0000 7f76ffe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f77001047a0 0x7f7700199ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:57:49.961 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:49.975+0000 7f7705000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7700101e70 0x7f77001997a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:57:49.961 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:49.975+0000 7f76ffe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f77001047a0 0x7f7700199ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:36920/0 (socket says 172.21.15.107:36920) 2024-03-08T20:57:49.961 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:49.975+0000 7f76ffe00640 1 -- 172.21.15.107:0/1285034179 learned_addr learned my addr 172.21.15.107:0/1285034179 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T20:57:49.961 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:49.975+0000 7f7705a00640 1 --2- 172.21.15.107:0/1285034179 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f77001070d0 0x7f77001a0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:57:49.962 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:49.975+0000 7f76ffe00640 1 -- 172.21.15.107:0/1285034179 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7700101e70 msgr2=0x7f77001997a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:57:49.962 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:49.975+0000 7f76ffe00640 1 --2- 172.21.15.107:0/1285034179 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7700101e70 0x7f77001997a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:49.962 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:49.975+0000 7f76ffe00640 1 -- 172.21.15.107:0/1285034179 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f77001070d0 msgr2=0x7f77001a0d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:57:49.962 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:49.975+0000 7f76ffe00640 1 --2- 172.21.15.107:0/1285034179 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f77001070d0 0x7f77001a0d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:49.962 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:49.975+0000 7f76ffe00640 1 -- 172.21.15.107:0/1285034179 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77001a13c0 con 0x7f77001047a0 2024-03-08T20:57:49.962 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:49.976+0000 7f76ffe00640 1 --2- 172.21.15.107:0/1285034179 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f77001047a0 0x7f7700199ce0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f76ec00e9c0 tx=0x7f76ec00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:57:49.962 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:49.976+0000 7f76fd600640 1 -- 172.21.15.107:0/1285034179 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76ec00cd80 con 0x7f77001047a0 2024-03-08T20:57:49.962 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:49.976+0000 7f76fd600640 1 -- 172.21.15.107:0/1285034179 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f76ec0054d0 con 0x7f77001047a0 2024-03-08T20:57:49.962 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:49.977+0000 7f76fd600640 1 -- 172.21.15.107:0/1285034179 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76ec010430 con 0x7f77001047a0 2024-03-08T20:57:49.962 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:49.977+0000 7f7707a00640 1 -- 172.21.15.107:0/1285034179 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77001a16b0 con 0x7f77001047a0 2024-03-08T20:57:49.962 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:49.977+0000 7f7707a00640 1 -- 172.21.15.107:0/1285034179 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77001a1c00 con 0x7f77001047a0 2024-03-08T20:57:49.963 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:49.978+0000 7f7707a00640 1 -- 172.21.15.107:0/1285034179 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7700108bc0 con 0x7f77001047a0 2024-03-08T20:57:49.965 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:49.982+0000 7f76fd600640 1 -- 172.21.15.107:0/1285034179 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f76ec004fb0 con 0x7f77001047a0 2024-03-08T20:57:49.967 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:49.984+0000 7f76fd600640 1 --2- 172.21.15.107:0/1285034179 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f76e80a2800 0x7f76e80a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:57:49.967 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:49.984+0000 7f76fd600640 1 -- 172.21.15.107:0/1285034179 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 7879+0+0 (secure 0 0 0) 0x7f76ec0cef70 con 0x7f77001047a0 2024-03-08T20:57:49.967 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:49.984+0000 7f7705000640 1 --2- 172.21.15.107:0/1285034179 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f76e80a2800 0x7f76e80a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:57:49.968 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:49 smithi107 ceph-mon[31381]: from='client.? 172.21.15.107:0/4036949512' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-03-08T20:57:49.968 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:49 smithi107 ceph-mon[31381]: pgmap v356: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-08T20:57:49.968 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:49 smithi107 ceph-mon[31381]: from='client.? 172.21.15.107:0/4036949512' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-03-08T20:57:49.968 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:49 smithi107 ceph-mon[31381]: osdmap e83: 12 total, 12 up, 12 in 2024-03-08T20:57:49.968 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:49 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:57:49.968 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:49 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T20:57:49.968 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:49 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:57:49.968 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:49 smithi107 ceph-mon[31381]: from='client.? 172.21.15.107:0/1420170460' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-03-08T20:57:49.968 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:49 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T20:57:49.968 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:49.985+0000 7f7705000640 1 --2- 172.21.15.107:0/1285034179 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f76e80a2800 0x7f76e80a4cc0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f76f4005fd0 tx=0x7f76f400b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:57:49.969 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:49.986+0000 7f76fd600640 1 -- 172.21.15.107:0/1285034179 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f76ec053260 con 0x7f77001047a0 2024-03-08T20:57:50.016 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:49 smithi149 ceph-mon[35042]: from='client.? 172.21.15.107:0/4036949512' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-03-08T20:57:50.016 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:49 smithi149 ceph-mon[35042]: pgmap v356: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-08T20:57:50.016 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:49 smithi149 ceph-mon[35042]: from='client.? 172.21.15.107:0/4036949512' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-03-08T20:57:50.016 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:49 smithi149 ceph-mon[35042]: osdmap e83: 12 total, 12 up, 12 in 2024-03-08T20:57:50.016 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:49 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:57:50.016 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:49 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T20:57:50.017 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:49 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:57:50.017 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:49 smithi149 ceph-mon[35042]: from='client.? 172.21.15.107:0/1420170460' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-03-08T20:57:50.017 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:49 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T20:57:50.129 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:49 smithi204 ceph-mon[34935]: from='client.? 172.21.15.107:0/4036949512' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-03-08T20:57:50.129 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:49 smithi204 ceph-mon[34935]: pgmap v356: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-08T20:57:50.130 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:49 smithi204 ceph-mon[34935]: from='client.? 172.21.15.107:0/4036949512' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-03-08T20:57:50.130 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:49 smithi204 ceph-mon[34935]: osdmap e83: 12 total, 12 up, 12 in 2024-03-08T20:57:50.130 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:49 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:57:50.130 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:49 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T20:57:50.130 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:49 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:57:50.130 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:49 smithi204 ceph-mon[34935]: from='client.? 172.21.15.107:0/1420170460' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-03-08T20:57:50.130 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:49 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T20:57:50.206 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:50.222+0000 7f7707a00640 1 -- 172.21.15.107:0/1285034179 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f770002ce70 con 0x7f77001047a0 2024-03-08T20:57:50.741 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:50.757+0000 7f76fd600640 1 -- 172.21.15.107:0/1285034179 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) v1 ==== 172+0+0 (secure 0 0 0) 0x7f76ec09c120 con 0x7f77001047a0 2024-03-08T20:57:50.741 INFO:teuthology.orchestra.run.smithi107.stderr:new fs with metadata pool 2 and data pool 3 2024-03-08T20:57:50.745 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:50.761+0000 7f7707a00640 1 -- 172.21.15.107:0/1285034179 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f76e80a2800 msgr2=0x7f76e80a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:57:50.745 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:50.761+0000 7f7707a00640 1 --2- 172.21.15.107:0/1285034179 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f76e80a2800 0x7f76e80a4cc0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f76f4005fd0 tx=0x7f76f400b390 comp rx=0 tx=0).stop 2024-03-08T20:57:50.745 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:50.761+0000 7f7707a00640 1 -- 172.21.15.107:0/1285034179 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f77001047a0 msgr2=0x7f7700199ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:57:50.745 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:50.761+0000 7f7707a00640 1 --2- 172.21.15.107:0/1285034179 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f77001047a0 0x7f7700199ce0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f76ec00e9c0 tx=0x7f76ec00ee80 comp rx=0 tx=0).stop 2024-03-08T20:57:50.745 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:50.762+0000 7f7707a00640 1 -- 172.21.15.107:0/1285034179 shutdown_connections 2024-03-08T20:57:50.745 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:50.762+0000 7f7707a00640 1 --2- 172.21.15.107:0/1285034179 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f76e80a2800 0x7f76e80a4cc0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:50.746 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:50.762+0000 7f7707a00640 1 --2- 172.21.15.107:0/1285034179 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f77001070d0 0x7f77001a0d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:50.746 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:50.762+0000 7f7707a00640 1 --2- 172.21.15.107:0/1285034179 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f77001047a0 0x7f7700199ce0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:50.746 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:50.762+0000 7f7707a00640 1 --2- 172.21.15.107:0/1285034179 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7700101e70 0x7f77001997a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:50.746 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:50.762+0000 7f7707a00640 1 -- 172.21.15.107:0/1285034179 >> 172.21.15.107:0/1285034179 conn(0x7f77000fb7a0 msgr2=0x7f7700101570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:57:50.746 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:50.762+0000 7f7707a00640 1 -- 172.21.15.107:0/1285034179 shutdown_connections 2024-03-08T20:57:50.746 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:50.763+0000 7f7707a00640 1 -- 172.21.15.107:0/1285034179 wait complete. 2024-03-08T20:57:50.780 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-08T20:57:50.956 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:50.972+0000 7f65d5800640 1 -- 172.21.15.107:0/636966356 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f65d0106ff0 msgr2=0x7f65d0109440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:57:50.956 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:50.972+0000 7f65d5800640 1 --2- 172.21.15.107:0/636966356 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f65d0106ff0 0x7f65d0109440 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f65c8009ca0 tx=0x7f65c802c100 comp rx=0 tx=0).stop 2024-03-08T20:57:50.956 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:50.972+0000 7f65d5800640 1 -- 172.21.15.107:0/636966356 shutdown_connections 2024-03-08T20:57:50.956 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:50.972+0000 7f65d5800640 1 --2- 172.21.15.107:0/636966356 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f65d0106ff0 0x7f65d0109440 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:50.957 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:50.972+0000 7f65d5800640 1 --2- 172.21.15.107:0/636966356 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f65d01046c0 0x7f65d0106ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:50.957 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:50.972+0000 7f65d5800640 1 --2- 172.21.15.107:0/636966356 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f65d0101d90 0x7f65d0104180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:50.957 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:50.972+0000 7f65d5800640 1 -- 172.21.15.107:0/636966356 >> 172.21.15.107:0/636966356 conn(0x7f65d00fb780 msgr2=0x7f65d00fdbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:57:50.957 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:50.972+0000 7f65d5800640 1 -- 172.21.15.107:0/636966356 shutdown_connections 2024-03-08T20:57:50.957 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:50.972+0000 7f65d5800640 1 -- 172.21.15.107:0/636966356 wait complete. 2024-03-08T20:57:50.959 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:50.973+0000 7f65d5800640 1 Processor -- start 2024-03-08T20:57:50.959 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:50.973+0000 7f65d5800640 1 -- start start 2024-03-08T20:57:50.959 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:50.974+0000 7f65d5800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f65d0101d90 0x7f65d0101700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:57:50.959 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:50.974+0000 7f65d5800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f65d01046c0 0x7f65d00ffd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:57:50.959 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:50.974+0000 7f65d5800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f65d0106ff0 0x7f65d0100280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:57:50.959 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:50.974+0000 7f65d5800640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65d010ce50 con 0x7f65d0101d90 2024-03-08T20:57:50.959 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:50.974+0000 7f65d5800640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65d010ccd0 con 0x7f65d0106ff0 2024-03-08T20:57:50.960 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:50.974+0000 7f65d5800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65d010cfd0 con 0x7f65d01046c0 2024-03-08T20:57:50.960 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:50.974+0000 7f65cea00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f65d0101d90 0x7f65d0101700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:57:50.960 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:50.974+0000 7f65cea00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f65d0101d90 0x7f65d0101700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:36944/0 (socket says 172.21.15.107:36944) 2024-03-08T20:57:50.960 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:50.974+0000 7f65cea00640 1 -- 172.21.15.107:0/1830361171 learned_addr learned my addr 172.21.15.107:0/1830361171 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T20:57:50.960 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:50.974+0000 7f65ce000640 1 --2- 172.21.15.107:0/1830361171 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f65d01046c0 0x7f65d00ffd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:57:50.960 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:50.974+0000 7f65cf400640 1 --2- 172.21.15.107:0/1830361171 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f65d0106ff0 0x7f65d0100280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:57:50.960 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:50.974+0000 7f65cea00640 1 -- 172.21.15.107:0/1830361171 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f65d01046c0 msgr2=0x7f65d00ffd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:57:50.960 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:50.974+0000 7f65cea00640 1 --2- 172.21.15.107:0/1830361171 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f65d01046c0 0x7f65d00ffd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:50.960 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:50.974+0000 7f65cea00640 1 -- 172.21.15.107:0/1830361171 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f65d0106ff0 msgr2=0x7f65d0100280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:57:50.960 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:50.974+0000 7f65cea00640 1 --2- 172.21.15.107:0/1830361171 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f65d0106ff0 0x7f65d0100280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:50.960 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:50.974+0000 7f65cea00640 1 -- 172.21.15.107:0/1830361171 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65d0100b10 con 0x7f65d0101d90 2024-03-08T20:57:50.960 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:50.975+0000 7f65cea00640 1 --2- 172.21.15.107:0/1830361171 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f65d0101d90 0x7f65d0101700 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f65c000c920 tx=0x7f65c000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:57:50.961 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:50.975+0000 7f65c7400640 1 -- 172.21.15.107:0/1830361171 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65c0007c70 con 0x7f65d0101d90 2024-03-08T20:57:50.961 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:50.975+0000 7f65c7400640 1 -- 172.21.15.107:0/1830361171 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f65c0007e10 con 0x7f65d0101d90 2024-03-08T20:57:50.961 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:50.975+0000 7f65c7400640 1 -- 172.21.15.107:0/1830361171 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65c0018820 con 0x7f65d0101d90 2024-03-08T20:57:50.961 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:50.975+0000 7f65d5800640 1 -- 172.21.15.107:0/1830361171 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65d01a1570 con 0x7f65d0101d90 2024-03-08T20:57:50.961 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:50.975+0000 7f65d5800640 1 -- 172.21.15.107:0/1830361171 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65d01a1a70 con 0x7f65d0101d90 2024-03-08T20:57:50.961 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:50.976+0000 7f65d5800640 1 -- 172.21.15.107:0/1830361171 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65d00fd270 con 0x7f65d0101d90 2024-03-08T20:57:50.963 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:50.979+0000 7f65c7400640 1 -- 172.21.15.107:0/1830361171 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f65c00189c0 con 0x7f65d0101d90 2024-03-08T20:57:50.964 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:50.981+0000 7f65c7400640 1 --2- 172.21.15.107:0/1830361171 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f65b40a2800 0x7f65b40a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:57:50.964 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:50.981+0000 7f65c7400640 1 -- 172.21.15.107:0/1830361171 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f65c00cee20 con 0x7f65d0101d90 2024-03-08T20:57:50.964 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:50.981+0000 7f65ce000640 1 --2- 172.21.15.107:0/1830361171 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f65b40a2800 0x7f65b40a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:57:50.965 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:50.982+0000 7f65ce000640 1 --2- 172.21.15.107:0/1830361171 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f65b40a2800 0x7f65b40a4cc0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f65bc005fd0 tx=0x7f65bc005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:57:50.966 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:50.983+0000 7f65c7400640 1 -- 172.21.15.107:0/1830361171 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f65c0014030 con 0x7f65d0101d90 2024-03-08T20:57:51.016 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:50 smithi149 ceph-mon[35042]: from='client.? 172.21.15.107:0/1420170460' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-03-08T20:57:51.016 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:50 smithi149 ceph-mon[35042]: osdmap e84: 12 total, 12 up, 12 in 2024-03-08T20:57:51.016 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:50 smithi149 ceph-mon[35042]: from='client.? 172.21.15.107:0/1285034179' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-03-08T20:57:51.016 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:50 smithi149 ceph-mon[35042]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-03-08T20:57:51.016 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:50 smithi149 ceph-mon[35042]: pgmap v359: 97 pgs: 11 creating+peering, 82 unknown, 4 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-08T20:57:51.073 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:50 smithi204 ceph-mon[34935]: from='client.? 172.21.15.107:0/1420170460' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-03-08T20:57:51.073 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:50 smithi204 ceph-mon[34935]: osdmap e84: 12 total, 12 up, 12 in 2024-03-08T20:57:51.073 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:50 smithi204 ceph-mon[34935]: from='client.? 172.21.15.107:0/1285034179' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-03-08T20:57:51.074 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:50 smithi204 ceph-mon[34935]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-03-08T20:57:51.074 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:50 smithi204 ceph-mon[34935]: pgmap v359: 97 pgs: 11 creating+peering, 82 unknown, 4 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-08T20:57:51.164 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:51.180+0000 7f65d5800640 1 -- 172.21.15.107:0/1830361171 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f65d002d100 con 0x7f65d0101d90 2024-03-08T20:57:51.166 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:51.182+0000 7f65c7400640 1 -- 172.21.15.107:0/1830361171 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+22313 (secure 0 0 0) 0x7f65c009c0a0 con 0x7f65d0101d90 2024-03-08T20:57:51.166 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-08T20:57:51.167 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":85,"fsid":"dc20f0f8-dd8c-11ee-95c6-87774f69a715","created":"2024-03-08T20:47:04.190034+0000","modified":"2024-03-08T20:57:50.223797+0000","last_up_change":"2024-03-08T20:56:43.135514+0000","last_in_change":"2024-03-08T20:56:17.577074+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-08T20:51:32.832786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-08T20:57:48.721227+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-08T20:57:49.707779+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8b1ddc64-4d20-4e85-ac66-f1c5ac599501","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":2453946095},{"type":"v1","addr":"172.21.15.107:6803","nonce":2453946095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":2453946095},{"type":"v1","addr":"172.21.15.107:6805","nonce":2453946095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":2453946095},{"type":"v1","addr":"172.21.15.107:6809","nonce":2453946095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":2453946095},{"type":"v1","addr":"172.21.15.107:6807","nonce":2453946095}]},"public_addr":"172.21.15.107:6803/2453946095","cluster_addr":"172.21.15.107:6805/2453946095","heartbeat_back_addr":"172.21.15.107:6809/2453946095","heartbeat_front_addr":"172.21.15.107:6807/2453946095","state":["exists","up"]},{"osd":1,"uuid":"5a803c09-202a-4777-ba2a-1da8eaf26a54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":3071316419},{"type":"v1","addr":"172.21.15.149:6801","nonce":3071316419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":3071316419},{"type":"v1","addr":"172.21.15.149:6803","nonce":3071316419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":3071316419},{"type":"v1","addr":"172.21.15.149:6807","nonce":3071316419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":3071316419},{"type":"v1","addr":"172.21.15.149:6805","nonce":3071316419}]},"public_addr":"172.21.15.149:6801/3071316419","cluster_addr":"172.21.15.149:6803/3071316419","heartbeat_back_addr":"172.21.15.149:6807/3071316419","heartbeat_front_addr":"172.21.15.149:6805/3071316419","state":["exists","up"]},{"osd":2,"uuid":"555c634f-347e-47dd-a943-e50ae2be073c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":1087852630},{"type":"v1","addr":"172.21.15.204:6801","nonce":1087852630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":1087852630},{"type":"v1","addr":"172.21.15.204:6803","nonce":1087852630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":1087852630},{"type":"v1","addr":"172.21.15.204:6807","nonce":1087852630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":1087852630},{"type":"v1","addr":"172.21.15.204:6805","nonce":1087852630}]},"public_addr":"172.21.15.204:6801/1087852630","cluster_addr":"172.21.15.204:6803/1087852630","heartbeat_back_addr":"172.21.15.204:6807/1087852630","heartbeat_front_addr":"172.21.15.204:6805/1087852630","state":["exists","up"]},{"osd":3,"uuid":"9e433d97-e202-48e0-9ed1-f0988fa14307","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":3123579789},{"type":"v1","addr":"172.21.15.107:6811","nonce":3123579789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":3123579789},{"type":"v1","addr":"172.21.15.107:6813","nonce":3123579789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":3123579789},{"type":"v1","addr":"172.21.15.107:6817","nonce":3123579789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":3123579789},{"type":"v1","addr":"172.21.15.107:6815","nonce":3123579789}]},"public_addr":"172.21.15.107:6811/3123579789","cluster_addr":"172.21.15.107:6813/3123579789","heartbeat_back_addr":"172.21.15.107:6817/3123579789","heartbeat_front_addr":"172.21.15.107:6815/3123579789","state":["exists","up"]},{"osd":4,"uuid":"27c53794-9d2b-487d-bfcf-ac4fd2ab061e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":726286918},{"type":"v1","addr":"172.21.15.149:6809","nonce":726286918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":726286918},{"type":"v1","addr":"172.21.15.149:6811","nonce":726286918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":726286918},{"type":"v1","addr":"172.21.15.149:6815","nonce":726286918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":726286918},{"type":"v1","addr":"172.21.15.149:6813","nonce":726286918}]},"public_addr":"172.21.15.149:6809/726286918","cluster_addr":"172.21.15.149:6811/726286918","heartbeat_back_addr":"172.21.15.149:6815/726286918","heartbeat_front_addr":"172.21.15.149:6813/726286918","state":["exists","up"]},{"osd":5,"uuid":"3a7778fb-46c5-40c0-842e-e988c45ea24d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":3143887820},{"type":"v1","addr":"172.21.15.204:6809","nonce":3143887820}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":3143887820},{"type":"v1","addr":"172.21.15.204:6811","nonce":3143887820}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":3143887820},{"type":"v1","addr":"172.21.15.204:6815","nonce":3143887820}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":3143887820},{"type":"v1","addr":"172.21.15.204:6813","nonce":3143887820}]},"public_addr":"172.21.15.204:6809/3143887820","cluster_addr":"172.21.15.204:6811/3143887820","heartbeat_back_addr":"172.21.15.204:6815/3143887820","heartbeat_front_addr":"172.21.15.204:6813/3143887820","state":["exists","up"]},{"osd":6,"uuid":"d2e05a56-be4f-4435-9b76-ff9aa01e4ac3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":2912452950},{"type":"v1","addr":"172.21.15.107:6819","nonce":2912452950}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":2912452950},{"type":"v1","addr":"172.21.15.107:6821","nonce":2912452950}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":2912452950},{"type":"v1","addr":"172.21.15.107:6825","nonce":2912452950}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":2912452950},{"type":"v1","addr":"172.21.15.107:6823","nonce":2912452950}]},"public_addr":"172.21.15.107:6819/2912452950","cluster_addr":"172.21.15.107:6821/2912452950","heartbeat_back_addr":"172.21.15.107:6825/2912452950","heartbeat_front_addr":"172.21.15.107:6823/2912452950","state":["exists","up"]},{"osd":7,"uuid":"a7dd26ee-c118-4a9a-9e19-142c2680902d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":3835863809},{"type":"v1","addr":"172.21.15.149:6817","nonce":3835863809}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":3835863809},{"type":"v1","addr":"172.21.15.149:6819","nonce":3835863809}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":3835863809},{"type":"v1","addr":"172.21.15.149:6823","nonce":3835863809}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":3835863809},{"type":"v1","addr":"172.21.15.149:6821","nonce":3835863809}]},"public_addr":"172.21.15.149:6817/3835863809","cluster_addr":"172.21.15.149:6819/3835863809","heartbeat_back_addr":"172.21.15.149:6823/3835863809","heartbeat_front_addr":"172.21.15.149:6821/3835863809","state":["exists","up"]},{"osd":8,"uuid":"baaaa8dd-0e1f-4bae-86bf-9e429d985f23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":1010601477},{"type":"v1","addr":"172.21.15.204:6817","nonce":1010601477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":1010601477},{"type":"v1","addr":"172.21.15.204:6819","nonce":1010601477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":1010601477},{"type":"v1","addr":"172.21.15.204:6823","nonce":1010601477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":1010601477},{"type":"v1","addr":"172.21.15.204:6821","nonce":1010601477}]},"public_addr":"172.21.15.204:6817/1010601477","cluster_addr":"172.21.15.204:6819/1010601477","heartbeat_back_addr":"172.21.15.204:6823/1010601477","heartbeat_front_addr":"172.21.15.204:6821/1010601477","state":["exists","up"]},{"osd":9,"uuid":"8df80dfa-e7ff-410e-824c-b1dce111b02f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":3334759819},{"type":"v1","addr":"172.21.15.107:6827","nonce":3334759819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3334759819},{"type":"v1","addr":"172.21.15.107:6829","nonce":3334759819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":3334759819},{"type":"v1","addr":"172.21.15.107:6833","nonce":3334759819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":3334759819},{"type":"v1","addr":"172.21.15.107:6831","nonce":3334759819}]},"public_addr":"172.21.15.107:6827/3334759819","cluster_addr":"172.21.15.107:6829/3334759819","heartbeat_back_addr":"172.21.15.107:6833/3334759819","heartbeat_front_addr":"172.21.15.107:6831/3334759819","state":["exists","up"]},{"osd":10,"uuid":"338945b2-20c0-40b2-aaca-062cd4adee04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":3469284631},{"type":"v1","addr":"172.21.15.149:6825","nonce":3469284631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":3469284631},{"type":"v1","addr":"172.21.15.149:6827","nonce":3469284631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":3469284631},{"type":"v1","addr":"172.21.15.149:6831","nonce":3469284631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":3469284631},{"type":"v1","addr":"172.21.15.149:6829","nonce":3469284631}]},"public_addr":"172.21.15.149:6825/3469284631","cluster_addr":"172.21.15.149:6827/3469284631","heartbeat_back_addr":"172.21.15.149:6831/3469284631","heartbeat_front_addr":"172.21.15.149:6829/3469284631","state":["exists","up"]},{"osd":11,"uuid":"acccfe4f-d307-4a64-8f4d-fba27bfbe09d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":3097772473},{"type":"v1","addr":"172.21.15.204:6825","nonce":3097772473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":3097772473},{"type":"v1","addr":"172.21.15.204:6827","nonce":3097772473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":3097772473},{"type":"v1","addr":"172.21.15.204:6831","nonce":3097772473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":3097772473},{"type":"v1","addr":"172.21.15.204:6829","nonce":3097772473}]},"public_addr":"172.21.15.204:6825/3097772473","cluster_addr":"172.21.15.204:6827/3097772473","heartbeat_back_addr":"172.21.15.204:6831/3097772473","heartbeat_front_addr":"172.21.15.204:6829/3097772473","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T20:50:25.862753+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T20:50:58.167663+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T20:51:30.312816+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T20:52:05.487917+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T20:52:39.348354+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T20:53:12.899002+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T20:53:47.630837+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T20:54:21.444552+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T20:54:57.101160+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T20:55:31.841786+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T20:56:05.827161+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T20:56:40.611097+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.107:0/1514666962":"2024-03-09T20:48:20.149406+0000","172.21.15.107:0/1886016707":"2024-03-09T20:48:20.149406+0000","172.21.15.107:6801/426372446":"2024-03-09T20:48:20.149406+0000","172.21.15.107:6800/426372446":"2024-03-09T20:48:20.149406+0000","172.21.15.107:0/2392547434":"2024-03-09T20:47:45.706790+0000","172.21.15.107:0/1206995985":"2024-03-09T20:47:45.706790+0000","172.21.15.107:6800/3549936519":"2024-03-09T20:47:45.706790+0000","172.21.15.107:6801/3549936519":"2024-03-09T20:47:45.706790+0000","172.21.15.107:0/2482452215":"2024-03-09T20:47:45.706790+0000","172.21.15.107:0/808124991":"2024-03-09T20:48:20.149406+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-08T20:57:51.169 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:51.185+0000 7f65d5800640 1 -- 172.21.15.107:0/1830361171 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f65b40a2800 msgr2=0x7f65b40a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:57:51.169 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:51.185+0000 7f65d5800640 1 --2- 172.21.15.107:0/1830361171 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f65b40a2800 0x7f65b40a4cc0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f65bc005fd0 tx=0x7f65bc005850 comp rx=0 tx=0).stop 2024-03-08T20:57:51.169 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:51.185+0000 7f65d5800640 1 -- 172.21.15.107:0/1830361171 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f65d0101d90 msgr2=0x7f65d0101700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:57:51.169 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:51.185+0000 7f65d5800640 1 --2- 172.21.15.107:0/1830361171 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f65d0101d90 0x7f65d0101700 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f65c000c920 tx=0x7f65c000cde0 comp rx=0 tx=0).stop 2024-03-08T20:57:51.169 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:51.186+0000 7f65d5800640 1 -- 172.21.15.107:0/1830361171 shutdown_connections 2024-03-08T20:57:51.169 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:51.186+0000 7f65d5800640 1 --2- 172.21.15.107:0/1830361171 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f65b40a2800 0x7f65b40a4cc0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:51.169 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:51.186+0000 7f65d5800640 1 --2- 172.21.15.107:0/1830361171 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f65d0106ff0 0x7f65d0100280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:51.170 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:51.186+0000 7f65d5800640 1 --2- 172.21.15.107:0/1830361171 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f65d01046c0 0x7f65d00ffd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:51.170 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:51.186+0000 7f65d5800640 1 --2- 172.21.15.107:0/1830361171 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f65d0101d90 0x7f65d0101700 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:51.170 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:51.186+0000 7f65d5800640 1 -- 172.21.15.107:0/1830361171 >> 172.21.15.107:0/1830361171 conn(0x7f65d00fb780 msgr2=0x7f65d01026d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:57:51.170 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:51.186+0000 7f65d5800640 1 -- 172.21.15.107:0/1830361171 shutdown_connections 2024-03-08T20:57:51.170 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:51.186+0000 7f65d5800640 1 -- 172.21.15.107:0/1830361171 wait complete. 2024-03-08T20:57:51.183 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:50 smithi107 ceph-mon[31381]: from='client.? 172.21.15.107:0/1420170460' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-03-08T20:57:51.184 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:50 smithi107 ceph-mon[31381]: osdmap e84: 12 total, 12 up, 12 in 2024-03-08T20:57:51.184 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:50 smithi107 ceph-mon[31381]: from='client.? 172.21.15.107:0/1285034179' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-03-08T20:57:51.184 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:50 smithi107 ceph-mon[31381]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-03-08T20:57:51.184 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:50 smithi107 ceph-mon[31381]: pgmap v359: 97 pgs: 11 creating+peering, 82 unknown, 4 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-08T20:57:51.187 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-08T20:57:51.389 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:51.405+0000 7f987da00640 1 -- 172.21.15.107:0/969247836 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9878104720 msgr2=0x7f9878104ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:57:51.389 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:51.405+0000 7f987da00640 1 --2- 172.21.15.107:0/969247836 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9878104720 0x7f9878104ba0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f9860009900 tx=0x7f986002bc30 comp rx=0 tx=0).stop 2024-03-08T20:57:51.389 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:51.405+0000 7f987da00640 1 -- 172.21.15.107:0/969247836 shutdown_connections 2024-03-08T20:57:51.389 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:51.405+0000 7f987da00640 1 --2- 172.21.15.107:0/969247836 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f98781050e0 0x7f987810b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:51.389 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:51.405+0000 7f987da00640 1 --2- 172.21.15.107:0/969247836 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9878104720 0x7f9878104ba0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:51.389 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:51.405+0000 7f987da00640 1 --2- 172.21.15.107:0/969247836 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9878103530 0x7f9878103950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:51.389 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:51.405+0000 7f987da00640 1 -- 172.21.15.107:0/969247836 >> 172.21.15.107:0/969247836 conn(0x7f98780feae0 msgr2=0x7f9878100f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:57:51.389 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:51.405+0000 7f987da00640 1 -- 172.21.15.107:0/969247836 shutdown_connections 2024-03-08T20:57:51.390 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:51.405+0000 7f987da00640 1 -- 172.21.15.107:0/969247836 wait complete. 2024-03-08T20:57:51.392 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:51.406+0000 7f987da00640 1 Processor -- start 2024-03-08T20:57:51.392 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:51.406+0000 7f987da00640 1 -- start start 2024-03-08T20:57:51.392 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:51.407+0000 7f987da00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9878103530 0x7f9878199750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:57:51.392 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:51.407+0000 7f987da00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9878104720 0x7f9878199c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:57:51.392 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:51.407+0000 7f987da00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f98781050e0 0x7f98781a0d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:57:51.392 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:51.407+0000 7f987da00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f987810f0c0 con 0x7f9878104720 2024-03-08T20:57:51.392 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:51.407+0000 7f987da00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f987810ef40 con 0x7f9878103530 2024-03-08T20:57:51.392 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:51.407+0000 7f987da00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f987810f240 con 0x7f98781050e0 2024-03-08T20:57:51.393 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:51.407+0000 7f986fe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9878104720 0x7f9878199c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:57:51.393 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:51.407+0000 7f986fe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9878104720 0x7f9878199c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:36964/0 (socket says 172.21.15.107:36964) 2024-03-08T20:57:51.393 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:51.407+0000 7f986fe00640 1 -- 172.21.15.107:0/1625301174 learned_addr learned my addr 172.21.15.107:0/1625301174 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T20:57:51.393 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:51.407+0000 7f9876a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9878103530 0x7f9878199750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:57:51.393 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:51.407+0000 7f9877400640 1 --2- 172.21.15.107:0/1625301174 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f98781050e0 0x7f98781a0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:57:51.393 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:51.407+0000 7f986fe00640 1 -- 172.21.15.107:0/1625301174 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f98781050e0 msgr2=0x7f98781a0d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:57:51.393 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:51.407+0000 7f986fe00640 1 --2- 172.21.15.107:0/1625301174 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f98781050e0 0x7f98781a0d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:51.393 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:51.407+0000 7f986fe00640 1 -- 172.21.15.107:0/1625301174 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9878103530 msgr2=0x7f9878199750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:57:51.393 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:51.407+0000 7f986fe00640 1 --2- 172.21.15.107:0/1625301174 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9878103530 0x7f9878199750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:51.393 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:51.407+0000 7f986fe00640 1 -- 172.21.15.107:0/1625301174 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98781a1370 con 0x7f9878104720 2024-03-08T20:57:51.393 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:51.407+0000 7f9877400640 1 --2- 172.21.15.107:0/1625301174 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f98781050e0 0x7f98781a0d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T20:57:51.393 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:51.408+0000 7f986fe00640 1 --2- 172.21.15.107:0/1625301174 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9878104720 0x7f9878199c90 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f9860000c00 tx=0x7f98600042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:57:51.394 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:51.408+0000 7f986f400640 1 -- 172.21.15.107:0/1625301174 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9860043020 con 0x7f9878104720 2024-03-08T20:57:51.394 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:51.408+0000 7f986f400640 1 -- 172.21.15.107:0/1625301174 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f98600054c0 con 0x7f9878104720 2024-03-08T20:57:51.394 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:51.408+0000 7f987da00640 1 -- 172.21.15.107:0/1625301174 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98781a1600 con 0x7f9878104720 2024-03-08T20:57:51.394 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:51.408+0000 7f986f400640 1 -- 172.21.15.107:0/1625301174 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9860003d70 con 0x7f9878104720 2024-03-08T20:57:51.394 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:51.408+0000 7f987da00640 1 -- 172.21.15.107:0/1625301174 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98781a1ba0 con 0x7f9878104720 2024-03-08T20:57:51.394 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:51.409+0000 7f987da00640 1 -- 172.21.15.107:0/1625301174 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f987810a870 con 0x7f9878104720 2024-03-08T20:57:51.394 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:51.411+0000 7f986f400640 1 -- 172.21.15.107:0/1625301174 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f9860046020 con 0x7f9878104720 2024-03-08T20:57:51.395 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:51.412+0000 7f986f400640 1 --2- 172.21.15.107:0/1625301174 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f985c0a2880 0x7f985c0a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:57:51.396 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:51.412+0000 7f986f400640 1 -- 172.21.15.107:0/1625301174 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f98600f0fe0 con 0x7f9878104720 2024-03-08T20:57:51.396 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:51.412+0000 7f9876a00640 1 --2- 172.21.15.107:0/1625301174 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f985c0a2880 0x7f985c0a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:57:51.396 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:51.413+0000 7f9876a00640 1 --2- 172.21.15.107:0/1625301174 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f985c0a2880 0x7f985c0a4d40 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f9878104580 tx=0x7f986800b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:57:51.399 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:51.416+0000 7f986f400640 1 -- 172.21.15.107:0/1625301174 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f986004d020 con 0x7f9878104720 2024-03-08T20:57:51.605 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:51.622+0000 7f987da00640 1 -- 172.21.15.107:0/1625301174 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9878061ef0 con 0x7f9878104720 2024-03-08T20:57:51.608 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:51.624+0000 7f986f400640 1 -- 172.21.15.107:0/1625301174 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+22313 (secure 0 0 0) 0x7f98600be250 con 0x7f9878104720 2024-03-08T20:57:51.608 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-08T20:57:51.608 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":85,"fsid":"dc20f0f8-dd8c-11ee-95c6-87774f69a715","created":"2024-03-08T20:47:04.190034+0000","modified":"2024-03-08T20:57:50.223797+0000","last_up_change":"2024-03-08T20:56:43.135514+0000","last_in_change":"2024-03-08T20:56:17.577074+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-08T20:51:32.832786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-08T20:57:48.721227+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-08T20:57:49.707779+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8b1ddc64-4d20-4e85-ac66-f1c5ac599501","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":2453946095},{"type":"v1","addr":"172.21.15.107:6803","nonce":2453946095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":2453946095},{"type":"v1","addr":"172.21.15.107:6805","nonce":2453946095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":2453946095},{"type":"v1","addr":"172.21.15.107:6809","nonce":2453946095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":2453946095},{"type":"v1","addr":"172.21.15.107:6807","nonce":2453946095}]},"public_addr":"172.21.15.107:6803/2453946095","cluster_addr":"172.21.15.107:6805/2453946095","heartbeat_back_addr":"172.21.15.107:6809/2453946095","heartbeat_front_addr":"172.21.15.107:6807/2453946095","state":["exists","up"]},{"osd":1,"uuid":"5a803c09-202a-4777-ba2a-1da8eaf26a54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":3071316419},{"type":"v1","addr":"172.21.15.149:6801","nonce":3071316419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":3071316419},{"type":"v1","addr":"172.21.15.149:6803","nonce":3071316419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":3071316419},{"type":"v1","addr":"172.21.15.149:6807","nonce":3071316419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":3071316419},{"type":"v1","addr":"172.21.15.149:6805","nonce":3071316419}]},"public_addr":"172.21.15.149:6801/3071316419","cluster_addr":"172.21.15.149:6803/3071316419","heartbeat_back_addr":"172.21.15.149:6807/3071316419","heartbeat_front_addr":"172.21.15.149:6805/3071316419","state":["exists","up"]},{"osd":2,"uuid":"555c634f-347e-47dd-a943-e50ae2be073c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":1087852630},{"type":"v1","addr":"172.21.15.204:6801","nonce":1087852630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":1087852630},{"type":"v1","addr":"172.21.15.204:6803","nonce":1087852630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":1087852630},{"type":"v1","addr":"172.21.15.204:6807","nonce":1087852630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":1087852630},{"type":"v1","addr":"172.21.15.204:6805","nonce":1087852630}]},"public_addr":"172.21.15.204:6801/1087852630","cluster_addr":"172.21.15.204:6803/1087852630","heartbeat_back_addr":"172.21.15.204:6807/1087852630","heartbeat_front_addr":"172.21.15.204:6805/1087852630","state":["exists","up"]},{"osd":3,"uuid":"9e433d97-e202-48e0-9ed1-f0988fa14307","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":3123579789},{"type":"v1","addr":"172.21.15.107:6811","nonce":3123579789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":3123579789},{"type":"v1","addr":"172.21.15.107:6813","nonce":3123579789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":3123579789},{"type":"v1","addr":"172.21.15.107:6817","nonce":3123579789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":3123579789},{"type":"v1","addr":"172.21.15.107:6815","nonce":3123579789}]},"public_addr":"172.21.15.107:6811/3123579789","cluster_addr":"172.21.15.107:6813/3123579789","heartbeat_back_addr":"172.21.15.107:6817/3123579789","heartbeat_front_addr":"172.21.15.107:6815/3123579789","state":["exists","up"]},{"osd":4,"uuid":"27c53794-9d2b-487d-bfcf-ac4fd2ab061e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":726286918},{"type":"v1","addr":"172.21.15.149:6809","nonce":726286918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":726286918},{"type":"v1","addr":"172.21.15.149:6811","nonce":726286918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":726286918},{"type":"v1","addr":"172.21.15.149:6815","nonce":726286918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":726286918},{"type":"v1","addr":"172.21.15.149:6813","nonce":726286918}]},"public_addr":"172.21.15.149:6809/726286918","cluster_addr":"172.21.15.149:6811/726286918","heartbeat_back_addr":"172.21.15.149:6815/726286918","heartbeat_front_addr":"172.21.15.149:6813/726286918","state":["exists","up"]},{"osd":5,"uuid":"3a7778fb-46c5-40c0-842e-e988c45ea24d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":3143887820},{"type":"v1","addr":"172.21.15.204:6809","nonce":3143887820}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":3143887820},{"type":"v1","addr":"172.21.15.204:6811","nonce":3143887820}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":3143887820},{"type":"v1","addr":"172.21.15.204:6815","nonce":3143887820}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":3143887820},{"type":"v1","addr":"172.21.15.204:6813","nonce":3143887820}]},"public_addr":"172.21.15.204:6809/3143887820","cluster_addr":"172.21.15.204:6811/3143887820","heartbeat_back_addr":"172.21.15.204:6815/3143887820","heartbeat_front_addr":"172.21.15.204:6813/3143887820","state":["exists","up"]},{"osd":6,"uuid":"d2e05a56-be4f-4435-9b76-ff9aa01e4ac3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":2912452950},{"type":"v1","addr":"172.21.15.107:6819","nonce":2912452950}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":2912452950},{"type":"v1","addr":"172.21.15.107:6821","nonce":2912452950}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":2912452950},{"type":"v1","addr":"172.21.15.107:6825","nonce":2912452950}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":2912452950},{"type":"v1","addr":"172.21.15.107:6823","nonce":2912452950}]},"public_addr":"172.21.15.107:6819/2912452950","cluster_addr":"172.21.15.107:6821/2912452950","heartbeat_back_addr":"172.21.15.107:6825/2912452950","heartbeat_front_addr":"172.21.15.107:6823/2912452950","state":["exists","up"]},{"osd":7,"uuid":"a7dd26ee-c118-4a9a-9e19-142c2680902d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":3835863809},{"type":"v1","addr":"172.21.15.149:6817","nonce":3835863809}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":3835863809},{"type":"v1","addr":"172.21.15.149:6819","nonce":3835863809}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":3835863809},{"type":"v1","addr":"172.21.15.149:6823","nonce":3835863809}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":3835863809},{"type":"v1","addr":"172.21.15.149:6821","nonce":3835863809}]},"public_addr":"172.21.15.149:6817/3835863809","cluster_addr":"172.21.15.149:6819/3835863809","heartbeat_back_addr":"172.21.15.149:6823/3835863809","heartbeat_front_addr":"172.21.15.149:6821/3835863809","state":["exists","up"]},{"osd":8,"uuid":"baaaa8dd-0e1f-4bae-86bf-9e429d985f23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":1010601477},{"type":"v1","addr":"172.21.15.204:6817","nonce":1010601477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":1010601477},{"type":"v1","addr":"172.21.15.204:6819","nonce":1010601477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":1010601477},{"type":"v1","addr":"172.21.15.204:6823","nonce":1010601477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":1010601477},{"type":"v1","addr":"172.21.15.204:6821","nonce":1010601477}]},"public_addr":"172.21.15.204:6817/1010601477","cluster_addr":"172.21.15.204:6819/1010601477","heartbeat_back_addr":"172.21.15.204:6823/1010601477","heartbeat_front_addr":"172.21.15.204:6821/1010601477","state":["exists","up"]},{"osd":9,"uuid":"8df80dfa-e7ff-410e-824c-b1dce111b02f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":3334759819},{"type":"v1","addr":"172.21.15.107:6827","nonce":3334759819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3334759819},{"type":"v1","addr":"172.21.15.107:6829","nonce":3334759819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":3334759819},{"type":"v1","addr":"172.21.15.107:6833","nonce":3334759819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":3334759819},{"type":"v1","addr":"172.21.15.107:6831","nonce":3334759819}]},"public_addr":"172.21.15.107:6827/3334759819","cluster_addr":"172.21.15.107:6829/3334759819","heartbeat_back_addr":"172.21.15.107:6833/3334759819","heartbeat_front_addr":"172.21.15.107:6831/3334759819","state":["exists","up"]},{"osd":10,"uuid":"338945b2-20c0-40b2-aaca-062cd4adee04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":3469284631},{"type":"v1","addr":"172.21.15.149:6825","nonce":3469284631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":3469284631},{"type":"v1","addr":"172.21.15.149:6827","nonce":3469284631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":3469284631},{"type":"v1","addr":"172.21.15.149:6831","nonce":3469284631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":3469284631},{"type":"v1","addr":"172.21.15.149:6829","nonce":3469284631}]},"public_addr":"172.21.15.149:6825/3469284631","cluster_addr":"172.21.15.149:6827/3469284631","heartbeat_back_addr":"172.21.15.149:6831/3469284631","heartbeat_front_addr":"172.21.15.149:6829/3469284631","state":["exists","up"]},{"osd":11,"uuid":"acccfe4f-d307-4a64-8f4d-fba27bfbe09d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":3097772473},{"type":"v1","addr":"172.21.15.204:6825","nonce":3097772473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":3097772473},{"type":"v1","addr":"172.21.15.204:6827","nonce":3097772473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":3097772473},{"type":"v1","addr":"172.21.15.204:6831","nonce":3097772473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":3097772473},{"type":"v1","addr":"172.21.15.204:6829","nonce":3097772473}]},"public_addr":"172.21.15.204:6825/3097772473","cluster_addr":"172.21.15.204:6827/3097772473","heartbeat_back_addr":"172.21.15.204:6831/3097772473","heartbeat_front_addr":"172.21.15.204:6829/3097772473","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T20:50:25.862753+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T20:50:58.167663+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T20:51:30.312816+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T20:52:05.487917+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T20:52:39.348354+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T20:53:12.899002+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T20:53:47.630837+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T20:54:21.444552+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T20:54:57.101160+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T20:55:31.841786+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T20:56:05.827161+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T20:56:40.611097+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.107:0/1514666962":"2024-03-09T20:48:20.149406+0000","172.21.15.107:0/1886016707":"2024-03-09T20:48:20.149406+0000","172.21.15.107:6801/426372446":"2024-03-09T20:48:20.149406+0000","172.21.15.107:6800/426372446":"2024-03-09T20:48:20.149406+0000","172.21.15.107:0/2392547434":"2024-03-09T20:47:45.706790+0000","172.21.15.107:0/1206995985":"2024-03-09T20:47:45.706790+0000","172.21.15.107:6800/3549936519":"2024-03-09T20:47:45.706790+0000","172.21.15.107:6801/3549936519":"2024-03-09T20:47:45.706790+0000","172.21.15.107:0/2482452215":"2024-03-09T20:47:45.706790+0000","172.21.15.107:0/808124991":"2024-03-09T20:48:20.149406+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-08T20:57:51.611 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:51.627+0000 7f987da00640 1 -- 172.21.15.107:0/1625301174 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f985c0a2880 msgr2=0x7f985c0a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:57:51.611 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:51.627+0000 7f987da00640 1 --2- 172.21.15.107:0/1625301174 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f985c0a2880 0x7f985c0a4d40 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f9878104580 tx=0x7f986800b390 comp rx=0 tx=0).stop 2024-03-08T20:57:51.611 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:51.628+0000 7f987da00640 1 -- 172.21.15.107:0/1625301174 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9878104720 msgr2=0x7f9878199c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:57:51.612 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:51.628+0000 7f987da00640 1 --2- 172.21.15.107:0/1625301174 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9878104720 0x7f9878199c90 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f9860000c00 tx=0x7f98600042f0 comp rx=0 tx=0).stop 2024-03-08T20:57:51.612 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:51.628+0000 7f987da00640 1 -- 172.21.15.107:0/1625301174 shutdown_connections 2024-03-08T20:57:51.612 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:51.628+0000 7f987da00640 1 --2- 172.21.15.107:0/1625301174 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f985c0a2880 0x7f985c0a4d40 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:51.612 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:51.628+0000 7f987da00640 1 --2- 172.21.15.107:0/1625301174 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f98781050e0 0x7f98781a0d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:51.612 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:51.628+0000 7f987da00640 1 --2- 172.21.15.107:0/1625301174 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9878104720 0x7f9878199c90 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:51.612 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:51.628+0000 7f987da00640 1 --2- 172.21.15.107:0/1625301174 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9878103530 0x7f9878199750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:51.612 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:51.628+0000 7f987da00640 1 -- 172.21.15.107:0/1625301174 >> 172.21.15.107:0/1625301174 conn(0x7f98780feae0 msgr2=0x7f9878107a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:57:51.612 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:51.628+0000 7f987da00640 1 -- 172.21.15.107:0/1625301174 shutdown_connections 2024-03-08T20:57:51.612 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:51.628+0000 7f987da00640 1 -- 172.21.15.107:0/1625301174 wait complete. 2024-03-08T20:57:51.634 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-03-08T20:57:51.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:51.839+0000 7fcd1d600640 1 -- 172.21.15.107:0/101256965 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcd181050e0 msgr2=0x7fcd1810b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:57:51.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:51.839+0000 7fcd1d600640 1 --2- 172.21.15.107:0/101256965 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcd181050e0 0x7fcd1810b970 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fcd10009ca0 tx=0x7fcd1002c100 comp rx=0 tx=0).stop 2024-03-08T20:57:51.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:51.839+0000 7fcd1d600640 1 -- 172.21.15.107:0/101256965 shutdown_connections 2024-03-08T20:57:51.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:51.839+0000 7fcd1d600640 1 --2- 172.21.15.107:0/101256965 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcd181050e0 0x7fcd1810b970 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:51.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:51.839+0000 7fcd1d600640 1 --2- 172.21.15.107:0/101256965 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcd18104720 0x7fcd18104ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:51.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:51.839+0000 7fcd1d600640 1 --2- 172.21.15.107:0/101256965 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcd18103530 0x7fcd18103950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:51.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:51.839+0000 7fcd1d600640 1 -- 172.21.15.107:0/101256965 >> 172.21.15.107:0/101256965 conn(0x7fcd180feaa0 msgr2=0x7fcd18100ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:57:51.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:51.839+0000 7fcd1d600640 1 -- 172.21.15.107:0/101256965 shutdown_connections 2024-03-08T20:57:51.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:51.839+0000 7fcd1d600640 1 -- 172.21.15.107:0/101256965 wait complete. 2024-03-08T20:57:51.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:51.840+0000 7fcd1d600640 1 Processor -- start 2024-03-08T20:57:51.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:51.840+0000 7fcd1d600640 1 -- start start 2024-03-08T20:57:51.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:51.840+0000 7fcd1d600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcd18103530 0x7fcd181996e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:57:51.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:51.840+0000 7fcd1d600640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcd18104720 0x7fcd18199c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:57:51.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:51.840+0000 7fcd1d600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcd181050e0 0x7fcd181a0c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:57:51.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:51.840+0000 7fcd1d600640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd1810f040 con 0x7fcd18103530 2024-03-08T20:57:51.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:51.840+0000 7fcd1d600640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd1810eec0 con 0x7fcd18104720 2024-03-08T20:57:51.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:51.840+0000 7fcd1d600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd1810f1c0 con 0x7fcd181050e0 2024-03-08T20:57:51.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:51.840+0000 7fcd16a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcd18103530 0x7fcd181996e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:57:51.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:51.841+0000 7fcd16a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcd18103530 0x7fcd181996e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:36978/0 (socket says 172.21.15.107:36978) 2024-03-08T20:57:51.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:51.841+0000 7fcd16a00640 1 -- 172.21.15.107:0/2425978498 learned_addr learned my addr 172.21.15.107:0/2425978498 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T20:57:51.827 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:51.841+0000 7fcd16000640 1 --2- 172.21.15.107:0/2425978498 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcd18104720 0x7fcd18199c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:57:51.827 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:51.841+0000 7fcd17400640 1 --2- 172.21.15.107:0/2425978498 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcd181050e0 0x7fcd181a0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:57:51.827 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:51.841+0000 7fcd16a00640 1 -- 172.21.15.107:0/2425978498 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcd181050e0 msgr2=0x7fcd181a0c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:57:51.827 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:51.841+0000 7fcd16a00640 1 --2- 172.21.15.107:0/2425978498 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcd181050e0 0x7fcd181a0c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:51.827 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:51.841+0000 7fcd16a00640 1 -- 172.21.15.107:0/2425978498 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcd18104720 msgr2=0x7fcd18199c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:57:51.827 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:51.841+0000 7fcd16a00640 1 --2- 172.21.15.107:0/2425978498 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcd18104720 0x7fcd18199c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:51.827 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:51.841+0000 7fcd16a00640 1 -- 172.21.15.107:0/2425978498 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd181a1300 con 0x7fcd18103530 2024-03-08T20:57:51.827 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:51.841+0000 7fcd16000640 1 --2- 172.21.15.107:0/2425978498 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcd18104720 0x7fcd18199c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T20:57:51.827 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:51.841+0000 7fcd16a00640 1 --2- 172.21.15.107:0/2425978498 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcd18103530 0x7fcd181996e0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fcd0800d950 tx=0x7fcd0800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:57:51.827 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:51.842+0000 7fcd0f400640 1 -- 172.21.15.107:0/2425978498 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd08017020 con 0x7fcd18103530 2024-03-08T20:57:51.827 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:51.842+0000 7fcd0f400640 1 -- 172.21.15.107:0/2425978498 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fcd08009c30 con 0x7fcd18103530 2024-03-08T20:57:51.827 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:51.842+0000 7fcd0f400640 1 -- 172.21.15.107:0/2425978498 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd0800f6b0 con 0x7fcd18103530 2024-03-08T20:57:51.827 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:51.842+0000 7fcd1d600640 1 -- 172.21.15.107:0/2425978498 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd181a15f0 con 0x7fcd18103530 2024-03-08T20:57:51.827 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:51.842+0000 7fcd1d600640 1 -- 172.21.15.107:0/2425978498 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd181a1af0 con 0x7fcd18103530 2024-03-08T20:57:51.828 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:51.843+0000 7fcd1d600640 1 -- 172.21.15.107:0/2425978498 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd1810a850 con 0x7fcd18103530 2024-03-08T20:57:51.829 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:51.846+0000 7fcd0f400640 1 -- 172.21.15.107:0/2425978498 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) v1 ==== 137291+0+0 (secure 0 0 0) 0x7fcd0801a020 con 0x7fcd18103530 2024-03-08T20:57:51.830 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:51.847+0000 7fcd0f400640 1 --2- 172.21.15.107:0/2425978498 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fccfc0a2800 0x7fccfc0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:57:51.831 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:51.848+0000 7fcd0f400640 1 -- 172.21.15.107:0/2425978498 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fcd080ce890 con 0x7fcd18103530 2024-03-08T20:57:51.831 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:51.848+0000 7fcd16000640 1 --2- 172.21.15.107:0/2425978498 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fccfc0a2800 0x7fccfc0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:57:51.831 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:51.848+0000 7fcd16000640 1 --2- 172.21.15.107:0/2425978498 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fccfc0a2800 0x7fccfc0a4cc0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fcd1819abf0 tx=0x7fcd04005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:57:51.837 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:51.854+0000 7fcd0f400640 1 -- 172.21.15.107:0/2425978498 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7fcd08021020 con 0x7fcd18103530 2024-03-08T20:57:51.968 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:51 smithi107 ceph-mon[31381]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-08T20:57:51.968 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:51 smithi107 ceph-mon[31381]: from='client.? 172.21.15.107:0/1285034179' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-03-08T20:57:51.968 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:51 smithi107 ceph-mon[31381]: osdmap e85: 12 total, 12 up, 12 in 2024-03-08T20:57:51.968 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:51 smithi107 ceph-mon[31381]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-03-08T20:57:51.968 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:51 smithi107 ceph-mon[31381]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-03-08T20:57:51.968 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:51 smithi107 ceph-mon[31381]: from='client.? 172.21.15.107:0/1830361171' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-08T20:57:51.968 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:51 smithi107 ceph-mon[31381]: from='client.? 172.21.15.107:0/1625301174' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-08T20:57:52.098 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:52.114+0000 7fcd1d600640 1 -- 172.21.15.107:0/2425978498 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fcd18061f20 con 0x7fcd18103530 2024-03-08T20:57:52.129 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:51 smithi204 ceph-mon[34935]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-08T20:57:52.129 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:51 smithi204 ceph-mon[34935]: from='client.? 172.21.15.107:0/1285034179' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-03-08T20:57:52.129 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:51 smithi204 ceph-mon[34935]: osdmap e85: 12 total, 12 up, 12 in 2024-03-08T20:57:52.129 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:51 smithi204 ceph-mon[34935]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-03-08T20:57:52.129 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:51 smithi204 ceph-mon[34935]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-03-08T20:57:52.129 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:51 smithi204 ceph-mon[34935]: from='client.? 172.21.15.107:0/1830361171' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-08T20:57:52.129 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:51 smithi204 ceph-mon[34935]: from='client.? 172.21.15.107:0/1625301174' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-08T20:57:52.134 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:51 smithi149 ceph-mon[35042]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-08T20:57:52.134 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:51 smithi149 ceph-mon[35042]: from='client.? 172.21.15.107:0/1285034179' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-03-08T20:57:52.134 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:51 smithi149 ceph-mon[35042]: osdmap e85: 12 total, 12 up, 12 in 2024-03-08T20:57:52.134 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:51 smithi149 ceph-mon[35042]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-03-08T20:57:52.135 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:51 smithi149 ceph-mon[35042]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-03-08T20:57:52.135 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:51 smithi149 ceph-mon[35042]: from='client.? 172.21.15.107:0/1830361171' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-08T20:57:52.135 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:51 smithi149 ceph-mon[35042]: from='client.? 172.21.15.107:0/1625301174' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-08T20:57:52.788 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:52.804+0000 7fcd0f400640 1 -- 172.21.15.107:0/2425978498 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) v1 ==== 118+0+0 (secure 0 0 0) 0x7fcd08009de0 con 0x7fcd18103530 2024-03-08T20:57:52.791 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:52.808+0000 7fcd1d600640 1 -- 172.21.15.107:0/2425978498 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fccfc0a2800 msgr2=0x7fccfc0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:57:52.791 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:52.808+0000 7fcd1d600640 1 --2- 172.21.15.107:0/2425978498 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fccfc0a2800 0x7fccfc0a4cc0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fcd1819abf0 tx=0x7fcd04005f90 comp rx=0 tx=0).stop 2024-03-08T20:57:52.792 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:52.808+0000 7fcd1d600640 1 -- 172.21.15.107:0/2425978498 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcd18103530 msgr2=0x7fcd181996e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:57:52.792 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:52.808+0000 7fcd1d600640 1 --2- 172.21.15.107:0/2425978498 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcd18103530 0x7fcd181996e0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fcd0800d950 tx=0x7fcd0800de10 comp rx=0 tx=0).stop 2024-03-08T20:57:52.792 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:52.808+0000 7fcd1d600640 1 -- 172.21.15.107:0/2425978498 shutdown_connections 2024-03-08T20:57:52.792 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:52.808+0000 7fcd1d600640 1 --2- 172.21.15.107:0/2425978498 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fccfc0a2800 0x7fccfc0a4cc0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:52.792 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:52.808+0000 7fcd1d600640 1 --2- 172.21.15.107:0/2425978498 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcd181050e0 0x7fcd181a0c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:52.792 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:52.808+0000 7fcd1d600640 1 --2- 172.21.15.107:0/2425978498 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcd18104720 0x7fcd18199c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:52.792 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:52.808+0000 7fcd1d600640 1 --2- 172.21.15.107:0/2425978498 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcd18103530 0x7fcd181996e0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:52.792 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:52.808+0000 7fcd1d600640 1 -- 172.21.15.107:0/2425978498 >> 172.21.15.107:0/2425978498 conn(0x7fcd180feaa0 msgr2=0x7fcd18107a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:57:52.792 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:52.808+0000 7fcd1d600640 1 -- 172.21.15.107:0/2425978498 shutdown_connections 2024-03-08T20:57:52.792 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:52.808+0000 7fcd1d600640 1 -- 172.21.15.107:0/2425978498 wait complete. 2024-03-08T20:57:52.816 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 5, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--size 25000000000'}} 2024-03-08T20:57:52.816 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 5 2024-03-08T20:57:53.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:53.049+0000 7f0e3be00640 1 -- 172.21.15.107:0/1413437921 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0e3c108a00 msgr2=0x7f0e3c108e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:57:53.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:53.049+0000 7f0e3be00640 1 --2- 172.21.15.107:0/1413437921 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0e3c108a00 0x7f0e3c108e80 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f0e24009900 tx=0x7f0e2402bc50 comp rx=0 tx=0).stop 2024-03-08T20:57:53.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:53.050+0000 7f0e3be00640 1 -- 172.21.15.107:0/1413437921 shutdown_connections 2024-03-08T20:57:53.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:53.050+0000 7f0e3be00640 1 --2- 172.21.15.107:0/1413437921 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0e3c1093c0 0x7f0e3c10fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:53.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:53.050+0000 7f0e3be00640 1 --2- 172.21.15.107:0/1413437921 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0e3c108a00 0x7f0e3c108e80 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:53.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:53.050+0000 7f0e3be00640 1 --2- 172.21.15.107:0/1413437921 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0e3c107810 0x7f0e3c107c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:53.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:53.050+0000 7f0e3be00640 1 -- 172.21.15.107:0/1413437921 >> 172.21.15.107:0/1413437921 conn(0x7f0e3c0767c0 msgr2=0x7f0e3c078be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:57:53.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:53.050+0000 7f0e3be00640 1 -- 172.21.15.107:0/1413437921 shutdown_connections 2024-03-08T20:57:53.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:53.050+0000 7f0e3be00640 1 -- 172.21.15.107:0/1413437921 wait complete. 2024-03-08T20:57:53.036 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:53.051+0000 7f0e3be00640 1 Processor -- start 2024-03-08T20:57:53.036 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:53.051+0000 7f0e3be00640 1 -- start start 2024-03-08T20:57:53.036 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:53.051+0000 7f0e3be00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0e3c107810 0x7f0e3c19da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:57:53.036 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:53.051+0000 7f0e3be00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0e3c108a00 0x7f0e3c19dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:57:53.036 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:53.051+0000 7f0e3be00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0e3c1093c0 0x7f0e3c1a5010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:57:53.036 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:53.051+0000 7f0e3be00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e3c113390 con 0x7f0e3c108a00 2024-03-08T20:57:53.036 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:53.051+0000 7f0e3be00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e3c113210 con 0x7f0e3c1093c0 2024-03-08T20:57:53.037 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:53.051+0000 7f0e3be00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e3c113510 con 0x7f0e3c107810 2024-03-08T20:57:53.037 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:53.051+0000 7f0e32000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0e3c108a00 0x7f0e3c19dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:57:53.037 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:53.051+0000 7f0e32000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0e3c108a00 0x7f0e3c19dfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:36984/0 (socket says 172.21.15.107:36984) 2024-03-08T20:57:53.037 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:53.051+0000 7f0e32000640 1 -- 172.21.15.107:0/785556163 learned_addr learned my addr 172.21.15.107:0/785556163 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T20:57:53.037 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:53.052+0000 7f0e3aa00640 1 --2- 172.21.15.107:0/785556163 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0e3c107810 0x7f0e3c19da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:57:53.037 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:53.052+0000 7f0e32000640 1 -- 172.21.15.107:0/785556163 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0e3c107810 msgr2=0x7f0e3c19da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:57:53.037 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:53.052+0000 7f0e32000640 1 --2- 172.21.15.107:0/785556163 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0e3c107810 0x7f0e3c19da60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:53.037 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:53.052+0000 7f0e3b400640 1 --2- 172.21.15.107:0/785556163 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0e3c1093c0 0x7f0e3c1a5010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:57:53.037 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:53.052+0000 7f0e32000640 1 -- 172.21.15.107:0/785556163 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0e3c1093c0 msgr2=0x7f0e3c1a5010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:57:53.037 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:53.052+0000 7f0e32000640 1 --2- 172.21.15.107:0/785556163 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0e3c1093c0 0x7f0e3c1a5010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:53.037 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:53.052+0000 7f0e32000640 1 -- 172.21.15.107:0/785556163 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e3c1a5680 con 0x7f0e3c108a00 2024-03-08T20:57:53.037 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:53.052+0000 7f0e32000640 1 --2- 172.21.15.107:0/785556163 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0e3c108a00 0x7f0e3c19dfa0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f0e24000c00 tx=0x7f0e240042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:57:53.038 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:53.052+0000 7f0e33e00640 1 -- 172.21.15.107:0/785556163 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e24043020 con 0x7f0e3c108a00 2024-03-08T20:57:53.038 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:53.052+0000 7f0e33e00640 1 -- 172.21.15.107:0/785556163 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f0e240054c0 con 0x7f0e3c108a00 2024-03-08T20:57:53.038 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:53.053+0000 7f0e3be00640 1 -- 172.21.15.107:0/785556163 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e3c1a5910 con 0x7f0e3c108a00 2024-03-08T20:57:53.038 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:53.053+0000 7f0e33e00640 1 -- 172.21.15.107:0/785556163 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e24003d80 con 0x7f0e3c108a00 2024-03-08T20:57:53.038 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:53.053+0000 7f0e3be00640 1 -- 172.21.15.107:0/785556163 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e3c1a5eb0 con 0x7f0e3c108a00 2024-03-08T20:57:53.038 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:53.054+0000 7f0e3be00640 1 -- 172.21.15.107:0/785556163 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e3c10ebb0 con 0x7f0e3c108a00 2024-03-08T20:57:53.039 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:53.056+0000 7f0e33e00640 1 -- 172.21.15.107:0/785556163 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f0e24046020 con 0x7f0e3c108a00 2024-03-08T20:57:53.041 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:53.058+0000 7f0e33e00640 1 --2- 172.21.15.107:0/785556163 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f0e200a2880 0x7f0e200a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:57:53.041 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:53.058+0000 7f0e33e00640 1 -- 172.21.15.107:0/785556163 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f0e240f12c0 con 0x7f0e3c108a00 2024-03-08T20:57:53.041 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:53.058+0000 7f0e3aa00640 1 --2- 172.21.15.107:0/785556163 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f0e200a2880 0x7f0e200a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:57:53.042 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:53.059+0000 7f0e3aa00640 1 --2- 172.21.15.107:0/785556163 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f0e200a2880 0x7f0e200a4d40 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f0e2c005db0 tx=0x7f0e2c005d40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:57:53.044 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:53.061+0000 7f0e33e00640 1 -- 172.21.15.107:0/785556163 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f0e2404d020 con 0x7f0e3c108a00 2024-03-08T20:57:53.130 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:52 smithi204 ceph-mon[34935]: mds.? [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] up:active 2024-03-08T20:57:53.130 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:52 smithi204 ceph-mon[34935]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-08T20:57:53.130 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:52 smithi204 ceph-mon[34935]: osdmap e86: 12 total, 12 up, 12 in 2024-03-08T20:57:53.130 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:52 smithi204 ceph-mon[34935]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-08T20:57:53.130 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:52 smithi204 ceph-mon[34935]: from='client.? 172.21.15.107:0/2425978498' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-03-08T20:57:53.130 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:52 smithi204 ceph-mon[34935]: pgmap v362: 97 pgs: 28 creating+peering, 51 unknown, 18 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-03-08T20:57:53.130 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:52 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:57:53.131 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:52 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:57:53.131 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:52 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:57:53.131 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:52 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:57:53.131 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:52 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:57:53.131 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:52 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:57:53.131 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:52 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:57:53.131 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:52 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T20:57:53.131 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:52 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:57:53.218 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:52 smithi107 ceph-mon[31381]: mds.? [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] up:active 2024-03-08T20:57:53.219 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:52 smithi107 ceph-mon[31381]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-08T20:57:53.219 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:52 smithi107 ceph-mon[31381]: osdmap e86: 12 total, 12 up, 12 in 2024-03-08T20:57:53.219 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:52 smithi107 ceph-mon[31381]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-08T20:57:53.219 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:52 smithi107 ceph-mon[31381]: from='client.? 172.21.15.107:0/2425978498' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-03-08T20:57:53.219 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:52 smithi107 ceph-mon[31381]: pgmap v362: 97 pgs: 28 creating+peering, 51 unknown, 18 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-03-08T20:57:53.219 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:52 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:57:53.219 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:52 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:57:53.219 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:52 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:57:53.219 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:52 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:57:53.219 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:52 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:57:53.219 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:52 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:57:53.220 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:52 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:57:53.220 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:52 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T20:57:53.220 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:52 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:57:53.266 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:52 smithi149 ceph-mon[35042]: mds.? [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] up:active 2024-03-08T20:57:53.266 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:52 smithi149 ceph-mon[35042]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-08T20:57:53.266 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:52 smithi149 ceph-mon[35042]: osdmap e86: 12 total, 12 up, 12 in 2024-03-08T20:57:53.266 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:52 smithi149 ceph-mon[35042]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-08T20:57:53.267 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:52 smithi149 ceph-mon[35042]: from='client.? 172.21.15.107:0/2425978498' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-03-08T20:57:53.267 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:52 smithi149 ceph-mon[35042]: pgmap v362: 97 pgs: 28 creating+peering, 51 unknown, 18 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-03-08T20:57:53.267 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:52 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:57:53.267 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:52 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:57:53.267 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:52 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:57:53.267 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:52 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:57:53.267 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:52 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:57:53.268 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:52 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:57:53.268 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:52 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:57:53.268 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:52 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T20:57:53.268 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:52 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:57:53.282 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:53.299+0000 7f0e3be00640 1 -- 172.21.15.107:0/785556163 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} v 0) v1 -- 0x7f0e3c02d000 con 0x7f0e3c108a00 2024-03-08T20:57:53.791 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:53.807+0000 7f0e33e00640 1 -- 172.21.15.107:0/785556163 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]=0 v18) v1 ==== 105+0+0 (secure 0 0 0) 0x7f0e240be530 con 0x7f0e3c108a00 2024-03-08T20:57:53.796 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:53.812+0000 7f0e3be00640 1 -- 172.21.15.107:0/785556163 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f0e200a2880 msgr2=0x7f0e200a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:57:53.796 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:53.812+0000 7f0e3be00640 1 --2- 172.21.15.107:0/785556163 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f0e200a2880 0x7f0e200a4d40 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f0e2c005db0 tx=0x7f0e2c005d40 comp rx=0 tx=0).stop 2024-03-08T20:57:53.796 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:53.812+0000 7f0e3be00640 1 -- 172.21.15.107:0/785556163 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0e3c108a00 msgr2=0x7f0e3c19dfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:57:53.796 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:53.812+0000 7f0e3be00640 1 --2- 172.21.15.107:0/785556163 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0e3c108a00 0x7f0e3c19dfa0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f0e24000c00 tx=0x7f0e240042f0 comp rx=0 tx=0).stop 2024-03-08T20:57:53.796 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:53.812+0000 7f0e3be00640 1 -- 172.21.15.107:0/785556163 shutdown_connections 2024-03-08T20:57:53.796 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:53.812+0000 7f0e3be00640 1 --2- 172.21.15.107:0/785556163 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f0e200a2880 0x7f0e200a4d40 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:53.797 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:53.812+0000 7f0e3be00640 1 --2- 172.21.15.107:0/785556163 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0e3c1093c0 0x7f0e3c1a5010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:53.797 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:53.812+0000 7f0e3be00640 1 --2- 172.21.15.107:0/785556163 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0e3c108a00 0x7f0e3c19dfa0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:53.797 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:53.812+0000 7f0e3be00640 1 --2- 172.21.15.107:0/785556163 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0e3c107810 0x7f0e3c19da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:53.797 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:53.812+0000 7f0e3be00640 1 -- 172.21.15.107:0/785556163 >> 172.21.15.107:0/785556163 conn(0x7f0e3c0767c0 msgr2=0x7f0e3c10bd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:57:53.797 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:53.813+0000 7f0e3be00640 1 -- 172.21.15.107:0/785556163 shutdown_connections 2024-03-08T20:57:53.797 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:53.813+0000 7f0e3be00640 1 -- 172.21.15.107:0/785556163 wait complete. 2024-03-08T20:57:53.823 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2024-03-08T20:57:53.968 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:53 smithi107 ceph-mon[31381]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-03-08T20:57:53.968 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:53 smithi107 ceph-mon[31381]: Cluster is now healthy 2024-03-08T20:57:53.969 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:53 smithi107 ceph-mon[31381]: from='client.? 172.21.15.107:0/2425978498' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-03-08T20:57:53.969 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:53 smithi107 ceph-mon[31381]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-08T20:57:53.969 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:53 smithi107 ceph-mon[31381]: from='client.? 172.21.15.107:0/785556163' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-03-08T20:57:54.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:54.045+0000 7f5df7e00640 1 -- 172.21.15.107:0/780935738 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5df8104fa0 msgr2=0x7f5df810b830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:57:54.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:54.045+0000 7f5df7e00640 1 --2- 172.21.15.107:0/780935738 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5df8104fa0 0x7f5df810b830 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f5df0009ca0 tx=0x7f5df002c0f0 comp rx=0 tx=0).stop 2024-03-08T20:57:54.030 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:54.045+0000 7f5df7e00640 1 -- 172.21.15.107:0/780935738 shutdown_connections 2024-03-08T20:57:54.030 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:54.045+0000 7f5df7e00640 1 --2- 172.21.15.107:0/780935738 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5df8104fa0 0x7f5df810b830 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:54.030 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:54.045+0000 7f5df7e00640 1 --2- 172.21.15.107:0/780935738 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5df81045e0 0x7f5df8104a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:54.030 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:54.046+0000 7f5df7e00640 1 --2- 172.21.15.107:0/780935738 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5df81033f0 0x7f5df8103810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:54.030 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:54.046+0000 7f5df7e00640 1 -- 172.21.15.107:0/780935738 >> 172.21.15.107:0/780935738 conn(0x7f5df80fe960 msgr2=0x7f5df8100da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:57:54.030 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:54.046+0000 7f5df7e00640 1 -- 172.21.15.107:0/780935738 shutdown_connections 2024-03-08T20:57:54.030 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:54.046+0000 7f5df7e00640 1 -- 172.21.15.107:0/780935738 wait complete. 2024-03-08T20:57:54.032 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:54.047+0000 7f5df7e00640 1 Processor -- start 2024-03-08T20:57:54.032 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:54.047+0000 7f5df7e00640 1 -- start start 2024-03-08T20:57:54.032 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:54.047+0000 7f5df7e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5df81033f0 0x7f5df81995c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:57:54.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:54.047+0000 7f5df7e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5df81045e0 0x7f5df8199b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:57:54.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:54.047+0000 7f5df7e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5df8104fa0 0x7f5df81a0b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:57:54.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:54.047+0000 7f5df7e00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5df810ef30 con 0x7f5df81045e0 2024-03-08T20:57:54.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:54.047+0000 7f5df7e00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5df810edb0 con 0x7f5df8104fa0 2024-03-08T20:57:54.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:54.047+0000 7f5df7e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5df810f0b0 con 0x7f5df81033f0 2024-03-08T20:57:54.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:54.047+0000 7f5df6000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5df81045e0 0x7f5df8199b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:57:54.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:54.048+0000 7f5df6000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5df81045e0 0x7f5df8199b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:37002/0 (socket says 172.21.15.107:37002) 2024-03-08T20:57:54.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:54.048+0000 7f5df6000640 1 -- 172.21.15.107:0/1269219154 learned_addr learned my addr 172.21.15.107:0/1269219154 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T20:57:54.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:54.048+0000 7f5df6a00640 1 --2- 172.21.15.107:0/1269219154 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5df81033f0 0x7f5df81995c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:57:54.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:54.048+0000 7f5df7400640 1 --2- 172.21.15.107:0/1269219154 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5df8104fa0 0x7f5df81a0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:57:54.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:54.048+0000 7f5df6000640 1 -- 172.21.15.107:0/1269219154 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5df81033f0 msgr2=0x7f5df81995c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:57:54.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:54.048+0000 7f5df6000640 1 --2- 172.21.15.107:0/1269219154 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5df81033f0 0x7f5df81995c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:54.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:54.048+0000 7f5df6000640 1 -- 172.21.15.107:0/1269219154 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5df8104fa0 msgr2=0x7f5df81a0b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:57:54.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:54.048+0000 7f5df6000640 1 --2- 172.21.15.107:0/1269219154 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5df8104fa0 0x7f5df81a0b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:54.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:54.048+0000 7f5df6000640 1 -- 172.21.15.107:0/1269219154 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5df81a11e0 con 0x7f5df81045e0 2024-03-08T20:57:54.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:54.048+0000 7f5df6000640 1 --2- 172.21.15.107:0/1269219154 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5df81045e0 0x7f5df8199b00 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f5de400d950 tx=0x7f5de400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:57:54.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:54.048+0000 7f5def400640 1 -- 172.21.15.107:0/1269219154 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5de4017070 con 0x7f5df81045e0 2024-03-08T20:57:54.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:54.049+0000 7f5df7e00640 1 -- 172.21.15.107:0/1269219154 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5df81a14d0 con 0x7f5df81045e0 2024-03-08T20:57:54.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:54.049+0000 7f5def400640 1 -- 172.21.15.107:0/1269219154 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f5de4009c30 con 0x7f5df81045e0 2024-03-08T20:57:54.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:54.049+0000 7f5def400640 1 -- 172.21.15.107:0/1269219154 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5de400f620 con 0x7f5df81045e0 2024-03-08T20:57:54.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:54.049+0000 7f5df7e00640 1 -- 172.21.15.107:0/1269219154 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5df81a1a70 con 0x7f5df81045e0 2024-03-08T20:57:54.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:54.050+0000 7f5df7e00640 1 -- 172.21.15.107:0/1269219154 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5df810a760 con 0x7f5df81045e0 2024-03-08T20:57:54.036 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:54.053+0000 7f5def400640 1 -- 172.21.15.107:0/1269219154 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f5de40109a0 con 0x7f5df81045e0 2024-03-08T20:57:54.037 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:54.054+0000 7f5def400640 1 --2- 172.21.15.107:0/1269219154 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f5ddc0a2800 0x7f5ddc0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:57:54.037 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:54.054+0000 7f5def400640 1 -- 172.21.15.107:0/1269219154 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f5de40ce610 con 0x7f5df81045e0 2024-03-08T20:57:54.038 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:54.054+0000 7f5df6a00640 1 --2- 172.21.15.107:0/1269219154 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f5ddc0a2800 0x7f5ddc0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:57:54.038 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:54.055+0000 7f5df6a00640 1 --2- 172.21.15.107:0/1269219154 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f5ddc0a2800 0x7f5ddc0a4cc0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f5de8005db0 tx=0x7f5de8005d40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:57:54.042 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:54.059+0000 7f5def400640 1 -- 172.21.15.107:0/1269219154 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f5de4009de0 con 0x7f5df81045e0 2024-03-08T20:57:54.130 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:53 smithi204 ceph-mon[34935]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-03-08T20:57:54.130 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:53 smithi204 ceph-mon[34935]: Cluster is now healthy 2024-03-08T20:57:54.130 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:53 smithi204 ceph-mon[34935]: from='client.? 172.21.15.107:0/2425978498' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-03-08T20:57:54.131 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:53 smithi204 ceph-mon[34935]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-08T20:57:54.131 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:53 smithi204 ceph-mon[34935]: from='client.? 172.21.15.107:0/785556163' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-03-08T20:57:54.266 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:53 smithi149 ceph-mon[35042]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-03-08T20:57:54.266 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:53 smithi149 ceph-mon[35042]: Cluster is now healthy 2024-03-08T20:57:54.266 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:53 smithi149 ceph-mon[35042]: from='client.? 172.21.15.107:0/2425978498' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-03-08T20:57:54.266 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:53 smithi149 ceph-mon[35042]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-08T20:57:54.266 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:53 smithi149 ceph-mon[35042]: from='client.? 172.21.15.107:0/785556163' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-03-08T20:57:54.280 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:54.297+0000 7f5df7e00640 1 -- 172.21.15.107:0/1269219154 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) v1 -- 0x7f5df802d0b0 con 0x7f5df81045e0 2024-03-08T20:57:54.766 INFO:journalctl@ceph.mds.b.smithi149.stdout:Mar 08 20:57:54 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-b[65616]: 2024-03-08T20:57:54.429+0000 7fb508800700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-08T20:57:54.802 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:54.818+0000 7f5def400640 1 -- 172.21.15.107:0/1269219154 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v20) v1 ==== 121+0+0 (secure 0 0 0) 0x7f5de409b800 con 0x7f5df81045e0 2024-03-08T20:57:54.806 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:54.822+0000 7f5df7e00640 1 -- 172.21.15.107:0/1269219154 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f5ddc0a2800 msgr2=0x7f5ddc0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:57:54.806 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:54.822+0000 7f5df7e00640 1 --2- 172.21.15.107:0/1269219154 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f5ddc0a2800 0x7f5ddc0a4cc0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f5de8005db0 tx=0x7f5de8005d40 comp rx=0 tx=0).stop 2024-03-08T20:57:54.806 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:54.822+0000 7f5df7e00640 1 -- 172.21.15.107:0/1269219154 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5df81045e0 msgr2=0x7f5df8199b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:57:54.806 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:54.822+0000 7f5df7e00640 1 --2- 172.21.15.107:0/1269219154 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5df81045e0 0x7f5df8199b00 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f5de400d950 tx=0x7f5de400de10 comp rx=0 tx=0).stop 2024-03-08T20:57:54.806 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:54.822+0000 7f5df7e00640 1 -- 172.21.15.107:0/1269219154 shutdown_connections 2024-03-08T20:57:54.806 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:54.822+0000 7f5df7e00640 1 --2- 172.21.15.107:0/1269219154 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f5ddc0a2800 0x7f5ddc0a4cc0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:54.806 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:54.822+0000 7f5df7e00640 1 --2- 172.21.15.107:0/1269219154 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5df8104fa0 0x7f5df81a0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:54.807 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:54.822+0000 7f5df7e00640 1 --2- 172.21.15.107:0/1269219154 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5df81045e0 0x7f5df8199b00 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:54.807 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:54.822+0000 7f5df7e00640 1 --2- 172.21.15.107:0/1269219154 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5df81033f0 0x7f5df81995c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:54.807 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:54.822+0000 7f5df7e00640 1 -- 172.21.15.107:0/1269219154 >> 172.21.15.107:0/1269219154 conn(0x7f5df80fe960 msgr2=0x7f5df8107930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:57:54.807 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:54.823+0000 7f5df7e00640 1 -- 172.21.15.107:0/1269219154 shutdown_connections 2024-03-08T20:57:54.807 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:54.823+0000 7f5df7e00640 1 -- 172.21.15.107:0/1269219154 wait complete. 2024-03-08T20:57:54.828 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2024-03-08T20:57:55.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:55.045+0000 7fe8d9c00640 1 -- 172.21.15.107:0/1327746868 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe8d4107910 msgr2=0x7fe8d4107d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:57:55.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:55.045+0000 7fe8d9c00640 1 --2- 172.21.15.107:0/1327746868 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe8d4107910 0x7fe8d4107d30 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fe8c4009900 tx=0x7fe8c402bc30 comp rx=0 tx=0).stop 2024-03-08T20:57:55.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:55.045+0000 7fe8d9c00640 1 -- 172.21.15.107:0/1327746868 shutdown_connections 2024-03-08T20:57:55.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:55.045+0000 7fe8d9c00640 1 --2- 172.21.15.107:0/1327746868 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe8d41094c0 0x7fe8d410fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:55.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:55.045+0000 7fe8d9c00640 1 --2- 172.21.15.107:0/1327746868 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe8d4108b00 0x7fe8d4108f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:55.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:55.045+0000 7fe8d9c00640 1 --2- 172.21.15.107:0/1327746868 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe8d4107910 0x7fe8d4107d30 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:55.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:55.045+0000 7fe8d9c00640 1 -- 172.21.15.107:0/1327746868 >> 172.21.15.107:0/1327746868 conn(0x7fe8d40768b0 msgr2=0x7fe8d4078cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:57:55.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:55.045+0000 7fe8d9c00640 1 -- 172.21.15.107:0/1327746868 shutdown_connections 2024-03-08T20:57:55.030 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:55.045+0000 7fe8d9c00640 1 -- 172.21.15.107:0/1327746868 wait complete. 2024-03-08T20:57:55.032 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:55.046+0000 7fe8d9c00640 1 Processor -- start 2024-03-08T20:57:55.032 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:55.046+0000 7fe8d9c00640 1 -- start start 2024-03-08T20:57:55.032 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:55.047+0000 7fe8d9c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe8d4107910 0x7fe8d419dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:57:55.032 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:55.047+0000 7fe8d9c00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe8d4108b00 0x7fe8d419e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:57:55.032 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:55.047+0000 7fe8d9c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe8d41094c0 0x7fe8d41a5190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:57:55.032 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:55.047+0000 7fe8d9c00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8d4113500 con 0x7fe8d41094c0 2024-03-08T20:57:55.032 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:55.047+0000 7fe8d9c00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8d4113380 con 0x7fe8d4108b00 2024-03-08T20:57:55.032 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:55.047+0000 7fe8d9c00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8d4113680 con 0x7fe8d4107910 2024-03-08T20:57:55.032 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:55.047+0000 7fe8d3400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe8d41094c0 0x7fe8d41a5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:57:55.032 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:55.047+0000 7fe8d3400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe8d41094c0 0x7fe8d41a5190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:37022/0 (socket says 172.21.15.107:37022) 2024-03-08T20:57:55.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:55.047+0000 7fe8d3400640 1 -- 172.21.15.107:0/2503358342 learned_addr learned my addr 172.21.15.107:0/2503358342 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T20:57:55.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:55.047+0000 7fe8cbe00640 1 --2- 172.21.15.107:0/2503358342 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe8d4108b00 0x7fe8d419e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:57:55.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:55.047+0000 7fe8d2a00640 1 --2- 172.21.15.107:0/2503358342 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe8d4107910 0x7fe8d419dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:57:55.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:55.047+0000 7fe8d3400640 1 -- 172.21.15.107:0/2503358342 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe8d4107910 msgr2=0x7fe8d419dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:57:55.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:55.047+0000 7fe8d3400640 1 --2- 172.21.15.107:0/2503358342 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe8d4107910 0x7fe8d419dc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:55.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:55.047+0000 7fe8d3400640 1 -- 172.21.15.107:0/2503358342 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe8d4108b00 msgr2=0x7fe8d419e160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:57:55.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:55.047+0000 7fe8d3400640 1 --2- 172.21.15.107:0/2503358342 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe8d4108b00 0x7fe8d419e160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:55.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:55.047+0000 7fe8d3400640 1 -- 172.21.15.107:0/2503358342 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8d41a5830 con 0x7fe8d41094c0 2024-03-08T20:57:55.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:55.047+0000 7fe8d2a00640 1 --2- 172.21.15.107:0/2503358342 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe8d4107910 0x7fe8d419dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T20:57:55.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:55.048+0000 7fe8d3400640 1 --2- 172.21.15.107:0/2503358342 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe8d41094c0 0x7fe8d41a5190 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fe8cc00b970 tx=0x7fe8cc00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:57:55.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:55.048+0000 7fe8cb400640 1 -- 172.21.15.107:0/2503358342 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8cc019070 con 0x7fe8d41094c0 2024-03-08T20:57:55.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:55.048+0000 7fe8d9c00640 1 -- 172.21.15.107:0/2503358342 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8d41a5b20 con 0x7fe8d41094c0 2024-03-08T20:57:55.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:55.048+0000 7fe8cb400640 1 -- 172.21.15.107:0/2503358342 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fe8cc014440 con 0x7fe8d41094c0 2024-03-08T20:57:55.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:55.048+0000 7fe8cb400640 1 -- 172.21.15.107:0/2503358342 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8cc0075d0 con 0x7fe8d41094c0 2024-03-08T20:57:55.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:55.048+0000 7fe8d9c00640 1 -- 172.21.15.107:0/2503358342 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8d41a6020 con 0x7fe8d41094c0 2024-03-08T20:57:55.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:55.049+0000 7fe8d9c00640 1 -- 172.21.15.107:0/2503358342 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8d410ed00 con 0x7fe8d41094c0 2024-03-08T20:57:55.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:55.051+0000 7fe8cb400640 1 -- 172.21.15.107:0/2503358342 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) v1 ==== 137291+0+0 (secure 0 0 0) 0x7fe8cc01c070 con 0x7fe8d41094c0 2024-03-08T20:57:55.036 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:55.052+0000 7fe8cb400640 1 --2- 172.21.15.107:0/2503358342 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fe8b40a2800 0x7fe8b40a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:57:55.036 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:55.052+0000 7fe8cb400640 1 -- 172.21.15.107:0/2503358342 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fe8cc0ce080 con 0x7fe8d41094c0 2024-03-08T20:57:55.036 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:55.053+0000 7fe8d2a00640 1 --2- 172.21.15.107:0/2503358342 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fe8b40a2800 0x7fe8b40a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:57:55.036 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:55.053+0000 7fe8d2a00640 1 --2- 172.21.15.107:0/2503358342 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fe8b40a2800 0x7fe8b40a4cc0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fe8c4036040 tx=0x7fe8c4000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:57:55.040 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:55.057+0000 7fe8cb400640 1 -- 172.21.15.107:0/2503358342 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7fe8cc09b1d0 con 0x7fe8d41094c0 2024-03-08T20:57:55.129 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:54 smithi204 ceph-mon[34935]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-03-08T20:57:55.129 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:54 smithi204 ceph-mon[34935]: from='client.? 172.21.15.107:0/785556163' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-03-08T20:57:55.129 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:54 smithi204 ceph-mon[34935]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-03-08T20:57:55.129 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:54 smithi204 ceph-mon[34935]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-08T20:57:55.129 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:54 smithi204 ceph-mon[34935]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-03-08T20:57:55.129 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:54 smithi204 ceph-mon[34935]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-03-08T20:57:55.130 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:54 smithi204 ceph-mon[34935]: from='client.? 172.21.15.107:0/1269219154' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-03-08T20:57:55.130 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:54 smithi204 ceph-mon[34935]: pgmap v363: 97 pgs: 19 creating+peering, 78 active+clean; 578 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s wr, 7 op/s 2024-03-08T20:57:55.218 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:54 smithi107 ceph-mon[31381]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-03-08T20:57:55.218 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:54 smithi107 ceph-mon[31381]: from='client.? 172.21.15.107:0/785556163' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-03-08T20:57:55.219 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:54 smithi107 ceph-mon[31381]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-03-08T20:57:55.219 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:54 smithi107 ceph-mon[31381]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-08T20:57:55.219 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:54 smithi107 ceph-mon[31381]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-03-08T20:57:55.219 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:54 smithi107 ceph-mon[31381]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-03-08T20:57:55.219 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:54 smithi107 ceph-mon[31381]: from='client.? 172.21.15.107:0/1269219154' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-03-08T20:57:55.219 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:54 smithi107 ceph-mon[31381]: pgmap v363: 97 pgs: 19 creating+peering, 78 active+clean; 578 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s wr, 7 op/s 2024-03-08T20:57:55.266 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:54 smithi149 ceph-mon[35042]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-03-08T20:57:55.266 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:54 smithi149 ceph-mon[35042]: from='client.? 172.21.15.107:0/785556163' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-03-08T20:57:55.266 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:54 smithi149 ceph-mon[35042]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-03-08T20:57:55.266 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:54 smithi149 ceph-mon[35042]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-03-08T20:57:55.267 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:54 smithi149 ceph-mon[35042]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-03-08T20:57:55.267 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:54 smithi149 ceph-mon[35042]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-03-08T20:57:55.267 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:54 smithi149 ceph-mon[35042]: from='client.? 172.21.15.107:0/1269219154' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-03-08T20:57:55.267 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:54 smithi149 ceph-mon[35042]: pgmap v363: 97 pgs: 19 creating+peering, 78 active+clean; 578 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s wr, 7 op/s 2024-03-08T20:57:55.280 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:55.296+0000 7fe8d9c00640 1 -- 172.21.15.107:0/2503358342 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) v1 -- 0x7fe8d4061f90 con 0x7fe8d41094c0 2024-03-08T20:57:55.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:55.832+0000 7fe8cb400640 1 -- 172.21.15.107:0/2503358342 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v22) v1 ==== 115+0+0 (secure 0 0 0) 0x7fe8cc09aad0 con 0x7fe8d41094c0 2024-03-08T20:57:55.819 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:55.836+0000 7fe8d9c00640 1 -- 172.21.15.107:0/2503358342 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fe8b40a2800 msgr2=0x7fe8b40a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:57:55.819 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:55.836+0000 7fe8d9c00640 1 --2- 172.21.15.107:0/2503358342 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fe8b40a2800 0x7fe8b40a4cc0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fe8c4036040 tx=0x7fe8c4000bc0 comp rx=0 tx=0).stop 2024-03-08T20:57:55.820 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:55.836+0000 7fe8d9c00640 1 -- 172.21.15.107:0/2503358342 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe8d41094c0 msgr2=0x7fe8d41a5190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:57:55.820 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:55.836+0000 7fe8d9c00640 1 --2- 172.21.15.107:0/2503358342 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe8d41094c0 0x7fe8d41a5190 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fe8cc00b970 tx=0x7fe8cc00be30 comp rx=0 tx=0).stop 2024-03-08T20:57:55.820 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:55.836+0000 7fe8d9c00640 1 -- 172.21.15.107:0/2503358342 shutdown_connections 2024-03-08T20:57:55.820 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:55.836+0000 7fe8d9c00640 1 --2- 172.21.15.107:0/2503358342 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fe8b40a2800 0x7fe8b40a4cc0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:55.820 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:55.836+0000 7fe8d9c00640 1 --2- 172.21.15.107:0/2503358342 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe8d41094c0 0x7fe8d41a5190 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:55.820 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:55.836+0000 7fe8d9c00640 1 --2- 172.21.15.107:0/2503358342 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe8d4108b00 0x7fe8d419e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:55.820 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:55.836+0000 7fe8d9c00640 1 --2- 172.21.15.107:0/2503358342 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe8d4107910 0x7fe8d419dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:55.820 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:55.836+0000 7fe8d9c00640 1 -- 172.21.15.107:0/2503358342 >> 172.21.15.107:0/2503358342 conn(0x7fe8d40768b0 msgr2=0x7fe8d410bed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:57:55.820 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:55.836+0000 7fe8d9c00640 1 -- 172.21.15.107:0/2503358342 shutdown_connections 2024-03-08T20:57:55.820 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:55.836+0000 7fe8d9c00640 1 -- 172.21.15.107:0/2503358342 wait complete. 2024-03-08T20:57:55.846 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--size 25000000000'} subvols for filesystem 'cephfs' 2024-03-08T20:57:55.846 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create cephfs qa 2024-03-08T20:57:56.042 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:56.058+0000 7f9a91000640 1 -- 172.21.15.107:0/3195095731 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9a8c1050c0 msgr2=0x7f9a8c10b950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:57:56.042 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:56.058+0000 7f9a91000640 1 --2- 172.21.15.107:0/3195095731 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9a8c1050c0 0x7f9a8c10b950 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f9a84009ca0 tx=0x7f9a8402c100 comp rx=0 tx=0).stop 2024-03-08T20:57:56.042 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:56.058+0000 7f9a91000640 1 -- 172.21.15.107:0/3195095731 shutdown_connections 2024-03-08T20:57:56.042 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:56.058+0000 7f9a91000640 1 --2- 172.21.15.107:0/3195095731 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9a8c1050c0 0x7f9a8c10b950 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:56.042 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:56.058+0000 7f9a91000640 1 --2- 172.21.15.107:0/3195095731 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9a8c104700 0x7f9a8c104b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:56.042 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:56.058+0000 7f9a91000640 1 --2- 172.21.15.107:0/3195095731 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9a8c103510 0x7f9a8c103930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:56.043 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:56.058+0000 7f9a91000640 1 -- 172.21.15.107:0/3195095731 >> 172.21.15.107:0/3195095731 conn(0x7f9a8c0feaa0 msgr2=0x7f9a8c100ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:57:56.043 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:56.058+0000 7f9a91000640 1 -- 172.21.15.107:0/3195095731 shutdown_connections 2024-03-08T20:57:56.043 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:56.058+0000 7f9a91000640 1 -- 172.21.15.107:0/3195095731 wait complete. 2024-03-08T20:57:56.045 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:56.059+0000 7f9a91000640 1 Processor -- start 2024-03-08T20:57:56.045 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:56.059+0000 7f9a91000640 1 -- start start 2024-03-08T20:57:56.045 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:56.059+0000 7f9a91000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9a8c103510 0x7f9a8c1996d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:57:56.045 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:56.059+0000 7f9a91000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9a8c104700 0x7f9a8c199c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:57:56.045 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:56.059+0000 7f9a91000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9a8c1050c0 0x7f9a8c1a0c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:57:56.045 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:56.059+0000 7f9a91000640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a8c10f050 con 0x7f9a8c103510 2024-03-08T20:57:56.045 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:56.059+0000 7f9a91000640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a8c10eed0 con 0x7f9a8c104700 2024-03-08T20:57:56.045 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:56.059+0000 7f9a91000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a8c10f1d0 con 0x7f9a8c1050c0 2024-03-08T20:57:56.045 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:56.060+0000 7f9a89e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9a8c103510 0x7f9a8c1996d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:57:56.046 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:56.060+0000 7f9a89e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9a8c103510 0x7f9a8c1996d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:57090/0 (socket says 172.21.15.107:57090) 2024-03-08T20:57:56.046 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:56.060+0000 7f9a89e00640 1 -- 172.21.15.107:0/379127635 learned_addr learned my addr 172.21.15.107:0/379127635 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T20:57:56.046 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:56.060+0000 7f9a83e00640 1 --2- 172.21.15.107:0/379127635 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9a8c104700 0x7f9a8c199c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:57:56.046 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:56.060+0000 7f9a8a800640 1 --2- 172.21.15.107:0/379127635 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9a8c1050c0 0x7f9a8c1a0c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:57:56.046 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:56.060+0000 7f9a89e00640 1 -- 172.21.15.107:0/379127635 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9a8c1050c0 msgr2=0x7f9a8c1a0c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:57:56.046 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:56.060+0000 7f9a89e00640 1 --2- 172.21.15.107:0/379127635 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9a8c1050c0 0x7f9a8c1a0c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:56.046 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:56.060+0000 7f9a89e00640 1 -- 172.21.15.107:0/379127635 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9a8c104700 msgr2=0x7f9a8c199c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:57:56.046 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:56.060+0000 7f9a89e00640 1 --2- 172.21.15.107:0/379127635 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9a8c104700 0x7f9a8c199c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:56.046 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:56.060+0000 7f9a89e00640 1 -- 172.21.15.107:0/379127635 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a8c1a12f0 con 0x7f9a8c103510 2024-03-08T20:57:56.046 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:56.060+0000 7f9a8a800640 1 --2- 172.21.15.107:0/379127635 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9a8c1050c0 0x7f9a8c1a0c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T20:57:56.046 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:56.061+0000 7f9a89e00640 1 --2- 172.21.15.107:0/379127635 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9a8c103510 0x7f9a8c1996d0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f9a7c00c970 tx=0x7f9a7c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:57:56.046 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:56.061+0000 7f9a82a00640 1 -- 172.21.15.107:0/379127635 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a7c004220 con 0x7f9a8c103510 2024-03-08T20:57:56.046 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:56.061+0000 7f9a82a00640 1 -- 172.21.15.107:0/379127635 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f9a7c0043c0 con 0x7f9a8c103510 2024-03-08T20:57:56.047 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:56.061+0000 7f9a91000640 1 -- 172.21.15.107:0/379127635 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a8c1a15e0 con 0x7f9a8c103510 2024-03-08T20:57:56.047 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:56.061+0000 7f9a82a00640 1 -- 172.21.15.107:0/379127635 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a7c00f450 con 0x7f9a8c103510 2024-03-08T20:57:56.047 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:56.061+0000 7f9a91000640 1 -- 172.21.15.107:0/379127635 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a8c1a1b30 con 0x7f9a8c103510 2024-03-08T20:57:56.047 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:56.062+0000 7f9a91000640 1 -- 172.21.15.107:0/379127635 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a8c10a830 con 0x7f9a8c103510 2024-03-08T20:57:56.047 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:56.064+0000 7f9a82a00640 1 -- 172.21.15.107:0/379127635 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f9a7c0079b0 con 0x7f9a8c103510 2024-03-08T20:57:56.049 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:56.066+0000 7f9a82a00640 1 --2- 172.21.15.107:0/379127635 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f9a6c0a2800 0x7f9a6c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:57:56.050 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:56.066+0000 7f9a82a00640 1 -- 172.21.15.107:0/379127635 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f9a7c0ce800 con 0x7f9a8c103510 2024-03-08T20:57:56.050 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:56.067+0000 7f9a83e00640 1 --2- 172.21.15.107:0/379127635 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f9a6c0a2800 0x7f9a6c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:57:56.050 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:56.067+0000 7f9a83e00640 1 --2- 172.21.15.107:0/379127635 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f9a6c0a2800 0x7f9a6c0a4cc0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f9a74005fd0 tx=0x7f9a74005e20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:57:56.054 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:56.071+0000 7f9a82a00640 1 -- 172.21.15.107:0/379127635 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f9a7c014030 con 0x7f9a8c103510 2024-03-08T20:57:56.129 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:55 smithi204 ceph-mon[34935]: from='client.? 172.21.15.107:0/1269219154' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-03-08T20:57:56.129 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:55 smithi204 ceph-mon[34935]: mds.? [v2:172.21.15.149:6836/914546237,v1:172.21.15.149:6837/914546237] up:active 2024-03-08T20:57:56.130 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:55 smithi204 ceph-mon[34935]: daemon mds.e assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-03-08T20:57:56.130 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:55 smithi204 ceph-mon[34935]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-03-08T20:57:56.130 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:55 smithi204 ceph-mon[34935]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:creating} 9 up:standby 2024-03-08T20:57:56.130 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:55 smithi204 ceph-mon[34935]: daemon mds.e is now active in filesystem cephfs as rank 2 2024-03-08T20:57:56.130 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:55 smithi204 ceph-mon[34935]: from='client.? 172.21.15.107:0/2503358342' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-03-08T20:57:56.218 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:55 smithi107 ceph-mon[31381]: from='client.? 172.21.15.107:0/1269219154' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-03-08T20:57:56.218 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:55 smithi107 ceph-mon[31381]: mds.? [v2:172.21.15.149:6836/914546237,v1:172.21.15.149:6837/914546237] up:active 2024-03-08T20:57:56.218 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:55 smithi107 ceph-mon[31381]: daemon mds.e assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-03-08T20:57:56.218 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:55 smithi107 ceph-mon[31381]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-03-08T20:57:56.219 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:55 smithi107 ceph-mon[31381]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:creating} 9 up:standby 2024-03-08T20:57:56.219 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:55 smithi107 ceph-mon[31381]: daemon mds.e is now active in filesystem cephfs as rank 2 2024-03-08T20:57:56.219 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:55 smithi107 ceph-mon[31381]: from='client.? 172.21.15.107:0/2503358342' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-03-08T20:57:56.266 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:55 smithi149 ceph-mon[35042]: from='client.? 172.21.15.107:0/1269219154' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-03-08T20:57:56.266 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:55 smithi149 ceph-mon[35042]: mds.? [v2:172.21.15.149:6836/914546237,v1:172.21.15.149:6837/914546237] up:active 2024-03-08T20:57:56.266 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:55 smithi149 ceph-mon[35042]: daemon mds.e assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-03-08T20:57:56.266 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:55 smithi149 ceph-mon[35042]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-03-08T20:57:56.266 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:55 smithi149 ceph-mon[35042]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:creating} 9 up:standby 2024-03-08T20:57:56.266 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:55 smithi149 ceph-mon[35042]: daemon mds.e is now active in filesystem cephfs as rank 2 2024-03-08T20:57:56.267 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:55 smithi149 ceph-mon[35042]: from='client.? 172.21.15.107:0/2503358342' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-03-08T20:57:56.282 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:56.298+0000 7f9a91000640 1 -- 172.21.15.107:0/379127635 --> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) v1 -- 0x7f9a8c061730 con 0x7f9a6c0a2800 2024-03-08T20:57:56.719 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 08 20:57:56 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-x[31606]: 2024-03-08T20:57:56.313+0000 7f8c10800700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-08T20:57:56.719 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 08 20:57:56 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-x[31606]: 2024-03-08T20:57:56.313+0000 7f8c10800700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-08T20:57:56.719 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 08 20:57:56 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-x[31606]: 2024-03-08T20:57:56.313+0000 7f8c10800700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-08T20:57:56.719 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 08 20:57:56 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-x[31606]: 2024-03-08T20:57:56.313+0000 7f8c10800700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-08T20:57:56.719 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 08 20:57:56 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-x[31606]: 2024-03-08T20:57:56.313+0000 7f8c10800700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-08T20:57:56.883 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:56.900+0000 7f9a82a00640 1 -- 172.21.15.107:0/379127635 <== mgr.14152 v2:172.21.15.107:6800/3376636618 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f9a8c061730 con 0x7f9a6c0a2800 2024-03-08T20:57:56.887 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:56.904+0000 7f9a91000640 1 -- 172.21.15.107:0/379127635 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f9a6c0a2800 msgr2=0x7f9a6c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:57:56.887 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:56.904+0000 7f9a91000640 1 --2- 172.21.15.107:0/379127635 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f9a6c0a2800 0x7f9a6c0a4cc0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f9a74005fd0 tx=0x7f9a74005e20 comp rx=0 tx=0).stop 2024-03-08T20:57:56.887 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:56.904+0000 7f9a91000640 1 -- 172.21.15.107:0/379127635 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9a8c103510 msgr2=0x7f9a8c1996d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:57:56.888 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:56.904+0000 7f9a91000640 1 --2- 172.21.15.107:0/379127635 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9a8c103510 0x7f9a8c1996d0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f9a7c00c970 tx=0x7f9a7c00ce30 comp rx=0 tx=0).stop 2024-03-08T20:57:56.888 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:56.904+0000 7f9a91000640 1 -- 172.21.15.107:0/379127635 shutdown_connections 2024-03-08T20:57:56.888 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:56.904+0000 7f9a91000640 1 --2- 172.21.15.107:0/379127635 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f9a6c0a2800 0x7f9a6c0a4cc0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:56.888 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:56.904+0000 7f9a91000640 1 --2- 172.21.15.107:0/379127635 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9a8c1050c0 0x7f9a8c1a0c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:56.888 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:56.904+0000 7f9a91000640 1 --2- 172.21.15.107:0/379127635 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9a8c104700 0x7f9a8c199c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:56.888 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:56.904+0000 7f9a91000640 1 --2- 172.21.15.107:0/379127635 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9a8c103510 0x7f9a8c1996d0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:56.888 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:56.904+0000 7f9a91000640 1 -- 172.21.15.107:0/379127635 >> 172.21.15.107:0/379127635 conn(0x7f9a8c0feaa0 msgr2=0x7f9a8c107a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:57:56.888 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:56.904+0000 7f9a91000640 1 -- 172.21.15.107:0/379127635 shutdown_connections 2024-03-08T20:57:56.888 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:56.904+0000 7f9a91000640 1 -- 172.21.15.107:0/379127635 wait complete. 2024-03-08T20:57:56.912 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_0 --group_name qa 2024-03-08T20:57:57.124 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:57.140+0000 7fbbab400640 1 -- 172.21.15.107:0/3276242496 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbba4103530 msgr2=0x7fbba4103950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:57:57.124 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:57.140+0000 7fbbab400640 1 --2- 172.21.15.107:0/3276242496 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbba4103530 0x7fbba4103950 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fbb98009900 tx=0x7fbb9802bc30 comp rx=0 tx=0).stop 2024-03-08T20:57:57.124 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:57.140+0000 7fbbab400640 1 -- 172.21.15.107:0/3276242496 shutdown_connections 2024-03-08T20:57:57.125 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:57.140+0000 7fbbab400640 1 --2- 172.21.15.107:0/3276242496 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbba41050e0 0x7fbba410b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:57.125 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:57.140+0000 7fbbab400640 1 --2- 172.21.15.107:0/3276242496 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbba4104720 0x7fbba4104ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:57.125 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:57.140+0000 7fbbab400640 1 --2- 172.21.15.107:0/3276242496 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbba4103530 0x7fbba4103950 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:57.125 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:57.141+0000 7fbbab400640 1 -- 172.21.15.107:0/3276242496 >> 172.21.15.107:0/3276242496 conn(0x7fbba40feaa0 msgr2=0x7fbba4100ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:57:57.125 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:57.141+0000 7fbbab400640 1 -- 172.21.15.107:0/3276242496 shutdown_connections 2024-03-08T20:57:57.125 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:57.141+0000 7fbbab400640 1 -- 172.21.15.107:0/3276242496 wait complete. 2024-03-08T20:57:57.127 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:57.141+0000 7fbbab400640 1 Processor -- start 2024-03-08T20:57:57.127 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:57.142+0000 7fbbab400640 1 -- start start 2024-03-08T20:57:57.127 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:57.142+0000 7fbbab400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbba4103530 0x7fbba4199970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:57:57.127 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:57.142+0000 7fbbab400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbba4104720 0x7fbba4199eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:57:57.127 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:57.142+0000 7fbbab400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbba41050e0 0x7fbba41a0ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:57:57.127 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:57.142+0000 7fbbab400640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbba410f180 con 0x7fbba4103530 2024-03-08T20:57:57.127 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:57.142+0000 7fbbab400640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbba410f000 con 0x7fbba4104720 2024-03-08T20:57:57.128 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:57.142+0000 7fbbab400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbba410f300 con 0x7fbba41050e0 2024-03-08T20:57:57.128 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:57.142+0000 7fbba8a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbba4103530 0x7fbba4199970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:57:57.128 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:57.142+0000 7fbba8a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbba4103530 0x7fbba4199970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:57134/0 (socket says 172.21.15.107:57134) 2024-03-08T20:57:57.128 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:57.142+0000 7fbba8a00640 1 -- 172.21.15.107:0/1773456973 learned_addr learned my addr 172.21.15.107:0/1773456973 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T20:57:57.128 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:57.143+0000 7fbba3e00640 1 --2- 172.21.15.107:0/1773456973 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbba4104720 0x7fbba4199eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:57:57.128 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:57.143+0000 7fbba9400640 1 --2- 172.21.15.107:0/1773456973 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbba41050e0 0x7fbba41a0ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:57:57.128 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:57.143+0000 7fbba8a00640 1 -- 172.21.15.107:0/1773456973 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbba41050e0 msgr2=0x7fbba41a0ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:57:57.128 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:57.143+0000 7fbba8a00640 1 --2- 172.21.15.107:0/1773456973 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbba41050e0 0x7fbba41a0ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:57.128 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:57.143+0000 7fbba8a00640 1 -- 172.21.15.107:0/1773456973 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbba4104720 msgr2=0x7fbba4199eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:57:57.128 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:57.143+0000 7fbba8a00640 1 --2- 172.21.15.107:0/1773456973 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbba4104720 0x7fbba4199eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:57.128 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:57.143+0000 7fbba8a00640 1 -- 172.21.15.107:0/1773456973 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbba41a1540 con 0x7fbba4103530 2024-03-08T20:57:57.128 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:57.143+0000 7fbba3e00640 1 --2- 172.21.15.107:0/1773456973 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbba4104720 0x7fbba4199eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T20:57:57.128 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:57.143+0000 7fbba9400640 1 --2- 172.21.15.107:0/1773456973 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbba41050e0 0x7fbba41a0ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T20:57:57.129 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:57.143+0000 7fbba8a00640 1 --2- 172.21.15.107:0/1773456973 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbba4103530 0x7fbba4199970 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fbb98005ab0 tx=0x7fbb980042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:57:57.129 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:57.143+0000 7fbba1600640 1 -- 172.21.15.107:0/1773456973 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb98043020 con 0x7fbba4103530 2024-03-08T20:57:57.129 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:57.143+0000 7fbba1600640 1 -- 172.21.15.107:0/1773456973 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fbb980054c0 con 0x7fbba4103530 2024-03-08T20:57:57.129 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:57.143+0000 7fbbab400640 1 -- 172.21.15.107:0/1773456973 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbba41a17d0 con 0x7fbba4103530 2024-03-08T20:57:57.129 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:57.143+0000 7fbba1600640 1 -- 172.21.15.107:0/1773456973 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb98003de0 con 0x7fbba4103530 2024-03-08T20:57:57.129 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:57.143+0000 7fbbab400640 1 -- 172.21.15.107:0/1773456973 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbba41a1d70 con 0x7fbba4103530 2024-03-08T20:57:57.129 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:56 smithi204 ceph-mon[34935]: from='client.? 172.21.15.107:0/2503358342' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-03-08T20:57:57.130 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:56 smithi204 ceph-mon[34935]: mds.? [v2:172.21.15.149:6834/442657066,v1:172.21.15.149:6835/442657066] up:active 2024-03-08T20:57:57.130 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:56 smithi204 ceph-mon[34935]: daemon mds.k assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-03-08T20:57:57.130 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:56 smithi204 ceph-mon[34935]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 9 up:standby 2024-03-08T20:57:57.130 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:56 smithi204 ceph-mon[34935]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=e=up:active,3=k=up:creating} 8 up:standby 2024-03-08T20:57:57.130 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:56 smithi204 ceph-mon[34935]: daemon mds.k is now active in filesystem cephfs as rank 3 2024-03-08T20:57:57.130 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:56 smithi204 ceph-mon[34935]: from='client.14733 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-08T20:57:57.130 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:56 smithi204 ceph-mon[34935]: pgmap v364: 97 pgs: 97 active+clean; 580 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s wr, 10 op/s 2024-03-08T20:57:57.130 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:57.145+0000 7fbbab400640 1 -- 172.21.15.107:0/1773456973 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbba410aa10 con 0x7fbba4103530 2024-03-08T20:57:57.130 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:57.146+0000 7fbba1600640 1 -- 172.21.15.107:0/1773456973 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) v1 ==== 137291+0+0 (secure 0 0 0) 0x7fbb98046020 con 0x7fbba4103530 2024-03-08T20:57:57.131 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:57.147+0000 7fbba1600640 1 --2- 172.21.15.107:0/1773456973 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fbb8c0a2800 0x7fbb8c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:57:57.131 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:57.148+0000 7fbba1600640 1 -- 172.21.15.107:0/1773456973 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fbb980f0e90 con 0x7fbba4103530 2024-03-08T20:57:57.131 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:57.148+0000 7fbba3e00640 1 --2- 172.21.15.107:0/1773456973 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fbb8c0a2800 0x7fbb8c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:57:57.132 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:57.149+0000 7fbba3e00640 1 --2- 172.21.15.107:0/1773456973 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fbb8c0a2800 0x7fbb8c0a4cc0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fbb90006fd0 tx=0x7fbb90008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:57:57.135 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:57.152+0000 7fbba1600640 1 -- 172.21.15.107:0/1773456973 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7fbb9804d020 con 0x7fbba4103530 2024-03-08T20:57:57.218 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:56 smithi107 ceph-mon[31381]: from='client.? 172.21.15.107:0/2503358342' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-03-08T20:57:57.218 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:56 smithi107 ceph-mon[31381]: mds.? [v2:172.21.15.149:6834/442657066,v1:172.21.15.149:6835/442657066] up:active 2024-03-08T20:57:57.218 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:56 smithi107 ceph-mon[31381]: daemon mds.k assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-03-08T20:57:57.219 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:56 smithi107 ceph-mon[31381]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 9 up:standby 2024-03-08T20:57:57.219 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:56 smithi107 ceph-mon[31381]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=e=up:active,3=k=up:creating} 8 up:standby 2024-03-08T20:57:57.219 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:56 smithi107 ceph-mon[31381]: daemon mds.k is now active in filesystem cephfs as rank 3 2024-03-08T20:57:57.219 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:56 smithi107 ceph-mon[31381]: from='client.14733 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-08T20:57:57.219 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:56 smithi107 ceph-mon[31381]: pgmap v364: 97 pgs: 97 active+clean; 580 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s wr, 10 op/s 2024-03-08T20:57:57.266 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:56 smithi149 ceph-mon[35042]: from='client.? 172.21.15.107:0/2503358342' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-03-08T20:57:57.266 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:56 smithi149 ceph-mon[35042]: mds.? [v2:172.21.15.149:6834/442657066,v1:172.21.15.149:6835/442657066] up:active 2024-03-08T20:57:57.266 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:56 smithi149 ceph-mon[35042]: daemon mds.k assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-03-08T20:57:57.266 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:56 smithi149 ceph-mon[35042]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 9 up:standby 2024-03-08T20:57:57.266 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:56 smithi149 ceph-mon[35042]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=e=up:active,3=k=up:creating} 8 up:standby 2024-03-08T20:57:57.266 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:56 smithi149 ceph-mon[35042]: daemon mds.k is now active in filesystem cephfs as rank 3 2024-03-08T20:57:57.267 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:56 smithi149 ceph-mon[35042]: from='client.14733 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-08T20:57:57.267 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:56 smithi149 ceph-mon[35042]: pgmap v364: 97 pgs: 97 active+clean; 580 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s wr, 10 op/s 2024-03-08T20:57:57.361 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:57.377+0000 7fbbab400640 1 -- 172.21.15.107:0/1773456973 --> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) v1 -- 0x7fbba40617c0 con 0x7fbb8c0a2800 2024-03-08T20:57:57.423 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:57.440+0000 7fbba1600640 1 -- 172.21.15.107:0/1773456973 <== mgr.14152 v2:172.21.15.107:6800/3376636618 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fbba40617c0 con 0x7fbb8c0a2800 2024-03-08T20:57:57.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:57.442+0000 7fbbab400640 1 -- 172.21.15.107:0/1773456973 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fbb8c0a2800 msgr2=0x7fbb8c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:57:57.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:57.442+0000 7fbbab400640 1 --2- 172.21.15.107:0/1773456973 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fbb8c0a2800 0x7fbb8c0a4cc0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fbb90006fd0 tx=0x7fbb90008040 comp rx=0 tx=0).stop 2024-03-08T20:57:57.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:57.442+0000 7fbbab400640 1 -- 172.21.15.107:0/1773456973 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbba4103530 msgr2=0x7fbba4199970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:57:57.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:57.442+0000 7fbbab400640 1 --2- 172.21.15.107:0/1773456973 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbba4103530 0x7fbba4199970 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fbb98005ab0 tx=0x7fbb980042f0 comp rx=0 tx=0).stop 2024-03-08T20:57:57.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:57.443+0000 7fbbab400640 1 -- 172.21.15.107:0/1773456973 shutdown_connections 2024-03-08T20:57:57.427 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:57.443+0000 7fbbab400640 1 --2- 172.21.15.107:0/1773456973 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fbb8c0a2800 0x7fbb8c0a4cc0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:57.427 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:57.443+0000 7fbbab400640 1 --2- 172.21.15.107:0/1773456973 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbba41050e0 0x7fbba41a0ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:57.427 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:57.443+0000 7fbbab400640 1 --2- 172.21.15.107:0/1773456973 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbba4104720 0x7fbba4199eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:57.427 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:57.443+0000 7fbbab400640 1 --2- 172.21.15.107:0/1773456973 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbba4103530 0x7fbba4199970 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:57.427 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:57.443+0000 7fbbab400640 1 -- 172.21.15.107:0/1773456973 >> 172.21.15.107:0/1773456973 conn(0x7fbba40feaa0 msgr2=0x7fbba4107be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:57:57.427 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:57.443+0000 7fbbab400640 1 -- 172.21.15.107:0/1773456973 shutdown_connections 2024-03-08T20:57:57.427 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:57.443+0000 7fbbab400640 1 -- 172.21.15.107:0/1773456973 wait complete. 2024-03-08T20:57:57.445 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-03-08T20:57:57.658 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:57.675+0000 7f07ad400640 1 -- 172.21.15.107:0/3498077770 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f07a81050e0 msgr2=0x7f07a810b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:57:57.659 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:57.675+0000 7f07ad400640 1 --2- 172.21.15.107:0/3498077770 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f07a81050e0 0x7f07a810b970 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f07a0009900 tx=0x7f07a002bc30 comp rx=0 tx=0).stop 2024-03-08T20:57:57.659 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:57.675+0000 7f07ad400640 1 -- 172.21.15.107:0/3498077770 shutdown_connections 2024-03-08T20:57:57.659 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:57.675+0000 7f07ad400640 1 --2- 172.21.15.107:0/3498077770 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f07a81050e0 0x7f07a810b970 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:57.659 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:57.675+0000 7f07ad400640 1 --2- 172.21.15.107:0/3498077770 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f07a8104720 0x7f07a8104ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:57.659 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:57.675+0000 7f07ad400640 1 --2- 172.21.15.107:0/3498077770 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f07a8103530 0x7f07a8103950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:57.659 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:57.675+0000 7f07ad400640 1 -- 172.21.15.107:0/3498077770 >> 172.21.15.107:0/3498077770 conn(0x7f07a80feaa0 msgr2=0x7f07a8100ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:57:57.659 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:57.675+0000 7f07ad400640 1 -- 172.21.15.107:0/3498077770 shutdown_connections 2024-03-08T20:57:57.659 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:57.675+0000 7f07ad400640 1 -- 172.21.15.107:0/3498077770 wait complete. 2024-03-08T20:57:57.662 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:57.676+0000 7f07ad400640 1 Processor -- start 2024-03-08T20:57:57.662 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:57.676+0000 7f07ad400640 1 -- start start 2024-03-08T20:57:57.662 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:57.676+0000 7f07ad400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f07a8103530 0x7f07a8199650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:57:57.662 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:57.676+0000 7f07ad400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f07a8104720 0x7f07a8199b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:57:57.662 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:57.676+0000 7f07ad400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f07a81050e0 0x7f07a81a0c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:57:57.662 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:57.676+0000 7f07ad400640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07a810f0c0 con 0x7f07a81050e0 2024-03-08T20:57:57.662 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:57.676+0000 7f07ad400640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07a810ef40 con 0x7f07a8103530 2024-03-08T20:57:57.662 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:57.676+0000 7f07ad400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07a810f240 con 0x7f07a8104720 2024-03-08T20:57:57.663 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:57.676+0000 7f07a7200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f07a81050e0 0x7f07a81a0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:57:57.663 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:57.676+0000 7f07a7200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f07a81050e0 0x7f07a81a0c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:57154/0 (socket says 172.21.15.107:57154) 2024-03-08T20:57:57.663 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:57.676+0000 7f07a7200640 1 -- 172.21.15.107:0/1901937846 learned_addr learned my addr 172.21.15.107:0/1901937846 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T20:57:57.663 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:57.677+0000 7f07a6800640 1 --2- 172.21.15.107:0/1901937846 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f07a8103530 0x7f07a8199650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:57:57.663 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:57.677+0000 7f079de00640 1 --2- 172.21.15.107:0/1901937846 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f07a8104720 0x7f07a8199b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:57:57.663 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:57.677+0000 7f07a7200640 1 -- 172.21.15.107:0/1901937846 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f07a8104720 msgr2=0x7f07a8199b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:57:57.663 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:57.677+0000 7f07a7200640 1 --2- 172.21.15.107:0/1901937846 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f07a8104720 0x7f07a8199b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:57.663 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:57.677+0000 7f07a7200640 1 -- 172.21.15.107:0/1901937846 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f07a8103530 msgr2=0x7f07a8199650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:57:57.663 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:57.677+0000 7f07a7200640 1 --2- 172.21.15.107:0/1901937846 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f07a8103530 0x7f07a8199650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:57.663 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:57.677+0000 7f07a7200640 1 -- 172.21.15.107:0/1901937846 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07a81a1270 con 0x7f07a81050e0 2024-03-08T20:57:57.663 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:57.677+0000 7f07a6800640 1 --2- 172.21.15.107:0/1901937846 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f07a8103530 0x7f07a8199650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T20:57:57.663 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:57.677+0000 7f07a7200640 1 --2- 172.21.15.107:0/1901937846 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f07a81050e0 0x7f07a81a0c00 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f07a002c170 tx=0x7f07a0004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:57:57.663 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:57.677+0000 7f079fe00640 1 -- 172.21.15.107:0/1901937846 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07a0043020 con 0x7f07a81050e0 2024-03-08T20:57:57.663 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:57.677+0000 7f079fe00640 1 -- 172.21.15.107:0/1901937846 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f07a00052b0 con 0x7f07a81050e0 2024-03-08T20:57:57.664 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:57.677+0000 7f079fe00640 1 -- 172.21.15.107:0/1901937846 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07a0003d00 con 0x7f07a81050e0 2024-03-08T20:57:57.664 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:57.677+0000 7f07ad400640 1 -- 172.21.15.107:0/1901937846 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07a81a1500 con 0x7f07a81050e0 2024-03-08T20:57:57.664 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:57.678+0000 7f07ad400640 1 -- 172.21.15.107:0/1901937846 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07a81a1aa0 con 0x7f07a81050e0 2024-03-08T20:57:57.664 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:57.679+0000 7f07ad400640 1 -- 172.21.15.107:0/1901937846 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07a810a780 con 0x7f07a81050e0 2024-03-08T20:57:57.665 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:57.680+0000 7f079fe00640 1 -- 172.21.15.107:0/1901937846 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f07a0046020 con 0x7f07a81050e0 2024-03-08T20:57:57.665 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:57.682+0000 7f079fe00640 1 --2- 172.21.15.107:0/1901937846 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f078c0a2880 0x7f078c0a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:57:57.665 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:57.682+0000 7f079fe00640 1 -- 172.21.15.107:0/1901937846 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f07a00f0fe0 con 0x7f07a81050e0 2024-03-08T20:57:57.666 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:57.682+0000 7f07a6800640 1 --2- 172.21.15.107:0/1901937846 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f078c0a2880 0x7f078c0a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:57:57.666 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:57.683+0000 7f07a6800640 1 --2- 172.21.15.107:0/1901937846 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f078c0a2880 0x7f078c0a4d40 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f07a8104580 tx=0x7f0798008370 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:57:57.669 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:57.686+0000 7f079fe00640 1 -- 172.21.15.107:0/1901937846 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f07a004d020 con 0x7f07a81050e0 2024-03-08T20:57:57.911 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:57.927+0000 7f07ad400640 1 -- 172.21.15.107:0/1901937846 --> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) v1 -- 0x7f07a80617c0 con 0x7f078c0a2880 2024-03-08T20:57:57.919 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:57.935+0000 7f079fe00640 1 -- 172.21.15.107:0/1901937846 <== mgr.14152 v2:172.21.15.107:6800/3376636618 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+54 (secure 0 0 0) 0x7f07a80617c0 con 0x7f078c0a2880 2024-03-08T20:57:57.919 INFO:teuthology.orchestra.run.smithi107.stdout:/volumes/qa/sv_0/2193ae44-16ef-4c95-bf3c-e3e15e292573 2024-03-08T20:57:57.921 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:57.938+0000 7f07ad400640 1 -- 172.21.15.107:0/1901937846 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f078c0a2880 msgr2=0x7f078c0a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:57:57.922 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:57.938+0000 7f07ad400640 1 --2- 172.21.15.107:0/1901937846 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f078c0a2880 0x7f078c0a4d40 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f07a8104580 tx=0x7f0798008370 comp rx=0 tx=0).stop 2024-03-08T20:57:57.922 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:57.938+0000 7f07ad400640 1 -- 172.21.15.107:0/1901937846 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f07a81050e0 msgr2=0x7f07a81a0c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:57:57.922 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:57.938+0000 7f07ad400640 1 --2- 172.21.15.107:0/1901937846 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f07a81050e0 0x7f07a81a0c00 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f07a002c170 tx=0x7f07a0004270 comp rx=0 tx=0).stop 2024-03-08T20:57:57.922 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:57.938+0000 7f07ad400640 1 -- 172.21.15.107:0/1901937846 shutdown_connections 2024-03-08T20:57:57.922 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:57.938+0000 7f07ad400640 1 --2- 172.21.15.107:0/1901937846 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f078c0a2880 0x7f078c0a4d40 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:57.922 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:57.938+0000 7f07ad400640 1 --2- 172.21.15.107:0/1901937846 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f07a81050e0 0x7f07a81a0c00 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:57.922 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:57.938+0000 7f07ad400640 1 --2- 172.21.15.107:0/1901937846 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f07a8104720 0x7f07a8199b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:57.922 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:57.938+0000 7f07ad400640 1 --2- 172.21.15.107:0/1901937846 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f07a8103530 0x7f07a8199650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:57.922 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:57.938+0000 7f07ad400640 1 -- 172.21.15.107:0/1901937846 >> 172.21.15.107:0/1901937846 conn(0x7f07a80feaa0 msgr2=0x7f07a8107950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:57:57.922 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:57.938+0000 7f07ad400640 1 -- 172.21.15.107:0/1901937846 shutdown_connections 2024-03-08T20:57:57.922 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:57.938+0000 7f07ad400640 1 -- 172.21.15.107:0/1901937846 wait complete. 2024-03-08T20:57:57.938 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_1 --group_name qa 2024-03-08T20:57:58.129 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:58.145+0000 7f7a21800640 1 -- 172.21.15.107:0/1773144691 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7a1c104700 msgr2=0x7f7a1c104b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:57:58.129 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:58.145+0000 7f7a21800640 1 --2- 172.21.15.107:0/1773144691 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7a1c104700 0x7f7a1c104b80 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f7a04009900 tx=0x7f7a0402bc30 comp rx=0 tx=0).stop 2024-03-08T20:57:58.129 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:58.145+0000 7f7a21800640 1 -- 172.21.15.107:0/1773144691 shutdown_connections 2024-03-08T20:57:58.129 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:58.145+0000 7f7a21800640 1 --2- 172.21.15.107:0/1773144691 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7a1c1050c0 0x7f7a1c10b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:58.129 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:58.145+0000 7f7a21800640 1 --2- 172.21.15.107:0/1773144691 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7a1c104700 0x7f7a1c104b80 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:58.129 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:58.145+0000 7f7a21800640 1 --2- 172.21.15.107:0/1773144691 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7a1c103510 0x7f7a1c103930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:58.130 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:58.145+0000 7f7a21800640 1 -- 172.21.15.107:0/1773144691 >> 172.21.15.107:0/1773144691 conn(0x7f7a1c0feaa0 msgr2=0x7f7a1c100ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:57:58.130 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:58.145+0000 7f7a21800640 1 -- 172.21.15.107:0/1773144691 shutdown_connections 2024-03-08T20:57:58.130 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:58.145+0000 7f7a21800640 1 -- 172.21.15.107:0/1773144691 wait complete. 2024-03-08T20:57:58.130 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:57 smithi204 ceph-mon[34935]: mds.? [v2:172.21.15.149:6832/1962161726,v1:172.21.15.149:6833/1962161726] up:active 2024-03-08T20:57:58.130 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:57 smithi204 ceph-mon[34935]: daemon mds.i assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-03-08T20:57:58.131 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:57 smithi204 ceph-mon[34935]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-03-08T20:57:58.131 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:57 smithi204 ceph-mon[34935]: Cluster is now healthy 2024-03-08T20:57:58.131 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:57 smithi204 ceph-mon[34935]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=e=up:active,3=k=up:active} 8 up:standby 2024-03-08T20:57:58.131 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:57 smithi204 ceph-mon[34935]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-03-08T20:57:58.131 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:57 smithi204 ceph-mon[34935]: daemon mds.i is now active in filesystem cephfs as rank 4 2024-03-08T20:57:58.131 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:57 smithi204 ceph-mon[34935]: from='client.14745 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-08T20:57:58.133 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:58.146+0000 7f7a21800640 1 Processor -- start 2024-03-08T20:57:58.133 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:58.146+0000 7f7a21800640 1 -- start start 2024-03-08T20:57:58.133 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:58.147+0000 7f7a21800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7a1c103510 0x7f7a1c199750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:57:58.133 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:58.147+0000 7f7a21800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7a1c104700 0x7f7a1c199c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:57:58.133 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:58.147+0000 7f7a21800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7a1c1050c0 0x7f7a1c1a0d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:57:58.133 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:58.147+0000 7f7a21800640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a1c10f0a0 con 0x7f7a1c1050c0 2024-03-08T20:57:58.133 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:58.147+0000 7f7a21800640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a1c10ef20 con 0x7f7a1c104700 2024-03-08T20:57:58.133 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:58.147+0000 7f7a21800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a1c10f220 con 0x7f7a1c103510 2024-03-08T20:57:58.133 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:58.147+0000 7f7a1b400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7a1c1050c0 0x7f7a1c1a0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:57:58.134 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:58.147+0000 7f7a1b400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7a1c1050c0 0x7f7a1c1a0d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:57180/0 (socket says 172.21.15.107:57180) 2024-03-08T20:57:58.134 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:58.147+0000 7f7a1b400640 1 -- 172.21.15.107:0/3690662059 learned_addr learned my addr 172.21.15.107:0/3690662059 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T20:57:58.134 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:58.147+0000 7f7a13e00640 1 --2- 172.21.15.107:0/3690662059 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7a1c104700 0x7f7a1c199c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:57:58.134 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:58.147+0000 7f7a1aa00640 1 --2- 172.21.15.107:0/3690662059 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7a1c103510 0x7f7a1c199750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:57:58.134 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:58.147+0000 7f7a1b400640 1 -- 172.21.15.107:0/3690662059 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7a1c103510 msgr2=0x7f7a1c199750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:57:58.134 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:58.147+0000 7f7a1b400640 1 --2- 172.21.15.107:0/3690662059 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7a1c103510 0x7f7a1c199750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:58.134 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:58.147+0000 7f7a1b400640 1 -- 172.21.15.107:0/3690662059 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7a1c104700 msgr2=0x7f7a1c199c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:57:58.134 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:58.147+0000 7f7a1b400640 1 --2- 172.21.15.107:0/3690662059 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7a1c104700 0x7f7a1c199c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:58.134 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:58.147+0000 7f7a1b400640 1 -- 172.21.15.107:0/3690662059 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a1c1a1370 con 0x7f7a1c1050c0 2024-03-08T20:57:58.134 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:58.148+0000 7f7a1aa00640 1 --2- 172.21.15.107:0/3690662059 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7a1c103510 0x7f7a1c199750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T20:57:58.134 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:58.148+0000 7f7a1b400640 1 --2- 172.21.15.107:0/3690662059 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7a1c1050c0 0x7f7a1c1a0d00 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f7a1400eaa0 tx=0x7f7a1400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:57:58.134 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:58.148+0000 7f7a13400640 1 -- 172.21.15.107:0/3690662059 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a1400cdb0 con 0x7f7a1c1050c0 2024-03-08T20:57:58.135 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:58.148+0000 7f7a13400640 1 -- 172.21.15.107:0/3690662059 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f7a14005020 con 0x7f7a1c1050c0 2024-03-08T20:57:58.135 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:58.148+0000 7f7a13400640 1 -- 172.21.15.107:0/3690662059 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a140116d0 con 0x7f7a1c1050c0 2024-03-08T20:57:58.135 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:58.148+0000 7f7a21800640 1 -- 172.21.15.107:0/3690662059 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a1c1a1660 con 0x7f7a1c1050c0 2024-03-08T20:57:58.135 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:58.148+0000 7f7a21800640 1 -- 172.21.15.107:0/3690662059 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a1c1a1b30 con 0x7f7a1c1050c0 2024-03-08T20:57:58.135 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:58.149+0000 7f7a21800640 1 -- 172.21.15.107:0/3690662059 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a1c10a850 con 0x7f7a1c1050c0 2024-03-08T20:57:58.135 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:58.151+0000 7f7a13400640 1 -- 172.21.15.107:0/3690662059 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f7a140051d0 con 0x7f7a1c1050c0 2024-03-08T20:57:58.136 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:58.153+0000 7f7a13400640 1 --2- 172.21.15.107:0/3690662059 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f7a000a2880 0x7f7a000a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:57:58.136 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:58.153+0000 7f7a13400640 1 -- 172.21.15.107:0/3690662059 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f7a140cebb0 con 0x7f7a1c1050c0 2024-03-08T20:57:58.136 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:58.153+0000 7f7a1aa00640 1 --2- 172.21.15.107:0/3690662059 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f7a000a2880 0x7f7a000a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:57:58.137 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:58.154+0000 7f7a1aa00640 1 --2- 172.21.15.107:0/3690662059 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f7a000a2880 0x7f7a000a4d40 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f7a1c104560 tx=0x7f7a0c005e70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:57:58.140 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:58.157+0000 7f7a13400640 1 -- 172.21.15.107:0/3690662059 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f7a14016030 con 0x7f7a1c1050c0 2024-03-08T20:57:58.218 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:57 smithi107 ceph-mon[31381]: mds.? [v2:172.21.15.149:6832/1962161726,v1:172.21.15.149:6833/1962161726] up:active 2024-03-08T20:57:58.219 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:57 smithi107 ceph-mon[31381]: daemon mds.i assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-03-08T20:57:58.219 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:57 smithi107 ceph-mon[31381]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-03-08T20:57:58.219 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:57 smithi107 ceph-mon[31381]: Cluster is now healthy 2024-03-08T20:57:58.219 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:57 smithi107 ceph-mon[31381]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=e=up:active,3=k=up:active} 8 up:standby 2024-03-08T20:57:58.219 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:57 smithi107 ceph-mon[31381]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-03-08T20:57:58.219 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:57 smithi107 ceph-mon[31381]: daemon mds.i is now active in filesystem cephfs as rank 4 2024-03-08T20:57:58.219 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:57 smithi107 ceph-mon[31381]: from='client.14745 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-08T20:57:58.266 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:57 smithi149 ceph-mon[35042]: mds.? [v2:172.21.15.149:6832/1962161726,v1:172.21.15.149:6833/1962161726] up:active 2024-03-08T20:57:58.266 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:57 smithi149 ceph-mon[35042]: daemon mds.i assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-03-08T20:57:58.266 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:57 smithi149 ceph-mon[35042]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-03-08T20:57:58.266 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:57 smithi149 ceph-mon[35042]: Cluster is now healthy 2024-03-08T20:57:58.266 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:57 smithi149 ceph-mon[35042]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=e=up:active,3=k=up:active} 8 up:standby 2024-03-08T20:57:58.267 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:57 smithi149 ceph-mon[35042]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-03-08T20:57:58.267 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:57 smithi149 ceph-mon[35042]: daemon mds.i is now active in filesystem cephfs as rank 4 2024-03-08T20:57:58.267 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:57 smithi149 ceph-mon[35042]: from='client.14745 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-08T20:57:58.368 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:58.385+0000 7f7a21800640 1 -- 172.21.15.107:0/3690662059 --> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) v1 -- 0x7f7a1c061730 con 0x7f7a000a2880 2024-03-08T20:57:58.431 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:58.448+0000 7f7a13400640 1 -- 172.21.15.107:0/3690662059 <== mgr.14152 v2:172.21.15.107:6800/3376636618 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f7a1c061730 con 0x7f7a000a2880 2024-03-08T20:57:58.435 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:58.451+0000 7f7a21800640 1 -- 172.21.15.107:0/3690662059 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f7a000a2880 msgr2=0x7f7a000a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:57:58.435 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:58.451+0000 7f7a21800640 1 --2- 172.21.15.107:0/3690662059 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f7a000a2880 0x7f7a000a4d40 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f7a1c104560 tx=0x7f7a0c005e70 comp rx=0 tx=0).stop 2024-03-08T20:57:58.435 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:58.451+0000 7f7a21800640 1 -- 172.21.15.107:0/3690662059 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7a1c1050c0 msgr2=0x7f7a1c1a0d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:57:58.435 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:58.451+0000 7f7a21800640 1 --2- 172.21.15.107:0/3690662059 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7a1c1050c0 0x7f7a1c1a0d00 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f7a1400eaa0 tx=0x7f7a1400ef60 comp rx=0 tx=0).stop 2024-03-08T20:57:58.435 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:58.451+0000 7f7a21800640 1 -- 172.21.15.107:0/3690662059 shutdown_connections 2024-03-08T20:57:58.435 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:58.451+0000 7f7a21800640 1 --2- 172.21.15.107:0/3690662059 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f7a000a2880 0x7f7a000a4d40 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:58.435 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:58.451+0000 7f7a21800640 1 --2- 172.21.15.107:0/3690662059 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7a1c1050c0 0x7f7a1c1a0d00 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:58.435 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:58.451+0000 7f7a21800640 1 --2- 172.21.15.107:0/3690662059 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7a1c104700 0x7f7a1c199c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:58.436 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:58.451+0000 7f7a21800640 1 --2- 172.21.15.107:0/3690662059 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7a1c103510 0x7f7a1c199750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:58.436 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:58.451+0000 7f7a21800640 1 -- 172.21.15.107:0/3690662059 >> 172.21.15.107:0/3690662059 conn(0x7f7a1c0feaa0 msgr2=0x7f7a1c107a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:57:58.436 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:58.452+0000 7f7a21800640 1 -- 172.21.15.107:0/3690662059 shutdown_connections 2024-03-08T20:57:58.436 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:58.452+0000 7f7a21800640 1 -- 172.21.15.107:0/3690662059 wait complete. 2024-03-08T20:57:58.459 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-03-08T20:57:58.679 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:58.696+0000 7f338de00640 1 -- 172.21.15.107:0/1200738942 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3388101300 msgr2=0x7f3388101720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:57:58.680 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:58.696+0000 7f338de00640 1 --2- 172.21.15.107:0/1200738942 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3388101300 0x7f3388101720 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f3378009ca0 tx=0x7f337802c100 comp rx=0 tx=0).stop 2024-03-08T20:57:58.680 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:58.696+0000 7f338de00640 1 -- 172.21.15.107:0/1200738942 shutdown_connections 2024-03-08T20:57:58.680 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:58.696+0000 7f338de00640 1 --2- 172.21.15.107:0/1200738942 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3388102eb0 0x7f3388109740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:58.680 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:58.696+0000 7f338de00640 1 --2- 172.21.15.107:0/1200738942 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f33881024f0 0x7f3388102970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:58.680 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:58.696+0000 7f338de00640 1 --2- 172.21.15.107:0/1200738942 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3388101300 0x7f3388101720 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:58.680 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:58.696+0000 7f338de00640 1 -- 172.21.15.107:0/1200738942 >> 172.21.15.107:0/1200738942 conn(0x7f33880fc890 msgr2=0x7f33880fecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:57:58.680 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:58.696+0000 7f338de00640 1 -- 172.21.15.107:0/1200738942 shutdown_connections 2024-03-08T20:57:58.680 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:58.696+0000 7f338de00640 1 -- 172.21.15.107:0/1200738942 wait complete. 2024-03-08T20:57:58.682 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:58.697+0000 7f338de00640 1 Processor -- start 2024-03-08T20:57:58.682 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:58.697+0000 7f338de00640 1 -- start start 2024-03-08T20:57:58.682 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:58.697+0000 7f338de00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3388101300 0x7f3388199760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:57:58.682 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:58.697+0000 7f338de00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f33881024f0 0x7f3388199ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:57:58.683 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:58.697+0000 7f338de00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3388102eb0 0x7f33881a0d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:57:58.683 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:58.697+0000 7f338de00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f338810ce40 con 0x7f33881024f0 2024-03-08T20:57:58.683 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:58.697+0000 7f338de00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f338810ccc0 con 0x7f3388101300 2024-03-08T20:57:58.683 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:58.697+0000 7f338de00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f338810cfc0 con 0x7f3388102eb0 2024-03-08T20:57:58.683 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:58.698+0000 7f337e000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f33881024f0 0x7f3388199ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:57:58.683 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:58.698+0000 7f337e000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f33881024f0 0x7f3388199ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:57192/0 (socket says 172.21.15.107:57192) 2024-03-08T20:57:58.683 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:58.698+0000 7f337e000640 1 -- 172.21.15.107:0/2832883391 learned_addr learned my addr 172.21.15.107:0/2832883391 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T20:57:58.683 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:58.698+0000 7f3386a00640 1 --2- 172.21.15.107:0/2832883391 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3388101300 0x7f3388199760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:57:58.683 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:58.698+0000 7f337e000640 1 -- 172.21.15.107:0/2832883391 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3388102eb0 msgr2=0x7f33881a0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:57:58.683 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:58.698+0000 7f337e000640 1 --2- 172.21.15.107:0/2832883391 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3388102eb0 0x7f33881a0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:58.683 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:58.698+0000 7f337e000640 1 -- 172.21.15.107:0/2832883391 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3388101300 msgr2=0x7f3388199760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:57:58.683 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:58.698+0000 7f337e000640 1 --2- 172.21.15.107:0/2832883391 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3388101300 0x7f3388199760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:58.683 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:58.698+0000 7f337e000640 1 -- 172.21.15.107:0/2832883391 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33881a1380 con 0x7f33881024f0 2024-03-08T20:57:58.684 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:58.698+0000 7f337e000640 1 --2- 172.21.15.107:0/2832883391 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f33881024f0 0x7f3388199ca0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f337000c920 tx=0x7f337000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:57:58.684 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:58.699+0000 7f338ca00640 1 -- 172.21.15.107:0/2832883391 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3370007c70 con 0x7f33881024f0 2024-03-08T20:57:58.684 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:58.699+0000 7f338ca00640 1 -- 172.21.15.107:0/2832883391 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f3370007e10 con 0x7f33881024f0 2024-03-08T20:57:58.684 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:58.699+0000 7f338de00640 1 -- 172.21.15.107:0/2832883391 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33881a1670 con 0x7f33881024f0 2024-03-08T20:57:58.684 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:58.699+0000 7f338ca00640 1 -- 172.21.15.107:0/2832883391 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3370018820 con 0x7f33881024f0 2024-03-08T20:57:58.684 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:58.699+0000 7f338de00640 1 -- 172.21.15.107:0/2832883391 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33881a1bc0 con 0x7f33881024f0 2024-03-08T20:57:58.684 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:58.700+0000 7f338de00640 1 -- 172.21.15.107:0/2832883391 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33881a1e80 con 0x7f33881024f0 2024-03-08T20:57:58.685 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:58.702+0000 7f338ca00640 1 -- 172.21.15.107:0/2832883391 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f33700189c0 con 0x7f33881024f0 2024-03-08T20:57:58.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:58.703+0000 7f338ca00640 1 --2- 172.21.15.107:0/2832883391 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f336c0a2800 0x7f336c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:57:58.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:58.703+0000 7f338ca00640 1 -- 172.21.15.107:0/2832883391 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f33700ceb10 con 0x7f33881024f0 2024-03-08T20:57:58.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:58.703+0000 7f3386a00640 1 --2- 172.21.15.107:0/2832883391 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f336c0a2800 0x7f336c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:57:58.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:58.704+0000 7f3386a00640 1 --2- 172.21.15.107:0/2832883391 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f336c0a2800 0x7f336c0a4cc0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f3378006010 tx=0x7f3378000f40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:57:58.690 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:58.707+0000 7f338ca00640 1 -- 172.21.15.107:0/2832883391 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f3370014030 con 0x7f33881024f0 2024-03-08T20:57:58.902 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:58.918+0000 7f338ca00640 1 -- 172.21.15.107:0/2832883391 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mgrmap(e 17) v1 ==== 137342+0+0 (secure 0 0 0) 0x7f3370097c70 con 0x7f33881024f0 2024-03-08T20:57:58.937 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:58.953+0000 7f338de00640 1 -- 172.21.15.107:0/2832883391 --> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f3388074c40 con 0x7f336c0a2800 2024-03-08T20:57:58.946 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:58.962+0000 7f338ca00640 1 -- 172.21.15.107:0/2832883391 <== mgr.14152 v2:172.21.15.107:6800/3376636618 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+54 (secure 0 0 0) 0x7f3388074c40 con 0x7f336c0a2800 2024-03-08T20:57:58.946 INFO:teuthology.orchestra.run.smithi107.stdout:/volumes/qa/sv_1/d20dc1fd-10e1-4a20-b637-4eb9c343cd28 2024-03-08T20:57:58.948 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:58.965+0000 7f338de00640 1 -- 172.21.15.107:0/2832883391 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f336c0a2800 msgr2=0x7f336c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:57:58.950 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:58.965+0000 7f338de00640 1 --2- 172.21.15.107:0/2832883391 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f336c0a2800 0x7f336c0a4cc0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f3378006010 tx=0x7f3378000f40 comp rx=0 tx=0).stop 2024-03-08T20:57:58.950 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:58.965+0000 7f338de00640 1 -- 172.21.15.107:0/2832883391 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f33881024f0 msgr2=0x7f3388199ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:57:58.950 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:58.965+0000 7f338de00640 1 --2- 172.21.15.107:0/2832883391 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f33881024f0 0x7f3388199ca0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f337000c920 tx=0x7f337000cde0 comp rx=0 tx=0).stop 2024-03-08T20:57:58.950 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:58.965+0000 7f338de00640 1 -- 172.21.15.107:0/2832883391 shutdown_connections 2024-03-08T20:57:58.950 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:58.965+0000 7f338de00640 1 --2- 172.21.15.107:0/2832883391 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f336c0a2800 0x7f336c0a4cc0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:58.950 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:58.965+0000 7f338de00640 1 --2- 172.21.15.107:0/2832883391 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3388102eb0 0x7f33881a0d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:58.950 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:58.965+0000 7f338de00640 1 --2- 172.21.15.107:0/2832883391 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f33881024f0 0x7f3388199ca0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:58.950 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:58.965+0000 7f338de00640 1 --2- 172.21.15.107:0/2832883391 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3388101300 0x7f3388199760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:58.950 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:58.965+0000 7f338de00640 1 -- 172.21.15.107:0/2832883391 >> 172.21.15.107:0/2832883391 conn(0x7f33880fc890 msgr2=0x7f3388073520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:57:58.951 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:58.965+0000 7f338de00640 1 -- 172.21.15.107:0/2832883391 shutdown_connections 2024-03-08T20:57:58.951 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:58.965+0000 7f338de00640 1 -- 172.21.15.107:0/2832883391 wait complete. 2024-03-08T20:57:58.969 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-08T20:57:59.129 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:58 smithi204 ceph-mon[34935]: mds.? [v2:172.21.15.204:6838/1857316822,v1:172.21.15.204:6839/1857316822] up:active 2024-03-08T20:57:59.129 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:58 smithi204 ceph-mon[34935]: fsmap cephfs:5 7 up:standby 5 up:active 2024-03-08T20:57:59.129 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:58 smithi204 ceph-mon[34935]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-08T20:57:59.130 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:58 smithi204 ceph-mon[34935]: from='client.14751 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-03-08T20:57:59.130 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:58 smithi204 ceph-mon[34935]: pgmap v365: 97 pgs: 97 active+clean; 601 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s wr, 16 op/s 2024-03-08T20:57:59.130 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:58 smithi204 ceph-mon[34935]: from='client.14757 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-08T20:57:59.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:59.191+0000 7f3934800640 1 -- 172.21.15.107:0/2523607074 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f392c1094e0 msgr2=0x7f392c10fd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:57:59.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:59.191+0000 7f3934800640 1 --2- 172.21.15.107:0/2523607074 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f392c1094e0 0x7f392c10fd70 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f3928009ca0 tx=0x7f392802c0f0 comp rx=0 tx=0).stop 2024-03-08T20:57:59.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:59.191+0000 7f3934800640 1 -- 172.21.15.107:0/2523607074 shutdown_connections 2024-03-08T20:57:59.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:59.191+0000 7f3934800640 1 --2- 172.21.15.107:0/2523607074 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f392c1094e0 0x7f392c10fd70 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:59.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:59.191+0000 7f3934800640 1 --2- 172.21.15.107:0/2523607074 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f392c108b20 0x7f392c108fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:59.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:59.191+0000 7f3934800640 1 --2- 172.21.15.107:0/2523607074 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f392c107930 0x7f392c107d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:59.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:59.191+0000 7f3934800640 1 -- 172.21.15.107:0/2523607074 >> 172.21.15.107:0/2523607074 conn(0x7f392c076900 msgr2=0x7f392c078d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:57:59.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:59.191+0000 7f3934800640 1 -- 172.21.15.107:0/2523607074 shutdown_connections 2024-03-08T20:57:59.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:59.191+0000 7f3934800640 1 -- 172.21.15.107:0/2523607074 wait complete. 2024-03-08T20:57:59.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:59.192+0000 7f3934800640 1 Processor -- start 2024-03-08T20:57:59.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:59.192+0000 7f3934800640 1 -- start start 2024-03-08T20:57:59.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:59.192+0000 7f3934800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f392c107930 0x7f392c19dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:57:59.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:59.192+0000 7f3934800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f392c108b20 0x7f392c19e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:57:59.178 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:59.192+0000 7f3934800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f392c1094e0 0x7f392c1a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:57:59.178 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:59.192+0000 7f3934800640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f392c113440 con 0x7f392c1094e0 2024-03-08T20:57:59.178 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:59.192+0000 7f3934800640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f392c1132c0 con 0x7f392c108b20 2024-03-08T20:57:59.178 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:59.192+0000 7f3934800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f392c1135c0 con 0x7f392c107930 2024-03-08T20:57:59.178 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:59.193+0000 7f3932800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f392c1094e0 0x7f392c1a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:57:59.178 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:59.193+0000 7f3932800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f392c1094e0 0x7f392c1a5070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:57212/0 (socket says 172.21.15.107:57212) 2024-03-08T20:57:59.178 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:59.193+0000 7f3932800640 1 -- 172.21.15.107:0/378324771 learned_addr learned my addr 172.21.15.107:0/378324771 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T20:57:59.178 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:59.193+0000 7f3932800640 1 -- 172.21.15.107:0/378324771 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f392c107930 msgr2=0x7f392c19dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:57:59.178 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:59.193+0000 7f3932800640 1 --2- 172.21.15.107:0/378324771 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f392c107930 0x7f392c19dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:59.178 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:59.193+0000 7f3931400640 1 --2- 172.21.15.107:0/378324771 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f392c108b20 0x7f392c19e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:57:59.178 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:59.193+0000 7f3932800640 1 -- 172.21.15.107:0/378324771 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f392c108b20 msgr2=0x7f392c19e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:57:59.178 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:59.193+0000 7f3932800640 1 --2- 172.21.15.107:0/378324771 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f392c108b20 0x7f392c19e000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:59.179 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:59.193+0000 7f3932800640 1 -- 172.21.15.107:0/378324771 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f392c1a56e0 con 0x7f392c1094e0 2024-03-08T20:57:59.179 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:59.193+0000 7f3932800640 1 --2- 172.21.15.107:0/378324771 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f392c1094e0 0x7f392c1a5070 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f392802c700 tx=0x7f39280328a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:57:59.179 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:59.194+0000 7f3926a00640 1 -- 172.21.15.107:0/378324771 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3928041020 con 0x7f392c1094e0 2024-03-08T20:57:59.179 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:59.194+0000 7f3926a00640 1 -- 172.21.15.107:0/378324771 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f3928003e20 con 0x7f392c1094e0 2024-03-08T20:57:59.179 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:59.194+0000 7f3934800640 1 -- 172.21.15.107:0/378324771 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f392c1a5970 con 0x7f392c1094e0 2024-03-08T20:57:59.179 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:59.194+0000 7f3926a00640 1 -- 172.21.15.107:0/378324771 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3928039040 con 0x7f392c1094e0 2024-03-08T20:57:59.179 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:59.194+0000 7f3934800640 1 -- 172.21.15.107:0/378324771 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f392c1a5eb0 con 0x7f392c1094e0 2024-03-08T20:57:59.179 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:59.195+0000 7f3934800640 1 -- 172.21.15.107:0/378324771 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f392c10ecd0 con 0x7f392c1094e0 2024-03-08T20:57:59.180 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:59.197+0000 7f3926a00640 1 -- 172.21.15.107:0/378324771 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) v1 ==== 137342+0+0 (secure 0 0 0) 0x7f3928044020 con 0x7f392c1094e0 2024-03-08T20:57:59.181 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:59.198+0000 7f3926a00640 1 --2- 172.21.15.107:0/378324771 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f39140a2910 0x7f39140a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:57:59.181 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:59.198+0000 7f3926a00640 1 -- 172.21.15.107:0/378324771 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f39280f0de0 con 0x7f392c1094e0 2024-03-08T20:57:59.181 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:59.198+0000 7f3931e00640 1 --2- 172.21.15.107:0/378324771 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f39140a2910 0x7f39140a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:57:59.182 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:59.199+0000 7f3931e00640 1 --2- 172.21.15.107:0/378324771 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f39140a2910 0x7f39140a4dd0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f3920005fd0 tx=0x7f392000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:57:59.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:59.203+0000 7f3926a00640 1 -- 172.21.15.107:0/378324771 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f392804b020 con 0x7f392c1094e0 2024-03-08T20:57:59.218 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:58 smithi107 ceph-mon[31381]: mds.? [v2:172.21.15.204:6838/1857316822,v1:172.21.15.204:6839/1857316822] up:active 2024-03-08T20:57:59.218 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:58 smithi107 ceph-mon[31381]: fsmap cephfs:5 7 up:standby 5 up:active 2024-03-08T20:57:59.218 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:58 smithi107 ceph-mon[31381]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-08T20:57:59.218 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:58 smithi107 ceph-mon[31381]: from='client.14751 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-03-08T20:57:59.218 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:58 smithi107 ceph-mon[31381]: pgmap v365: 97 pgs: 97 active+clean; 601 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s wr, 16 op/s 2024-03-08T20:57:59.218 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:58 smithi107 ceph-mon[31381]: from='client.14757 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-08T20:57:59.266 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:58 smithi149 ceph-mon[35042]: mds.? [v2:172.21.15.204:6838/1857316822,v1:172.21.15.204:6839/1857316822] up:active 2024-03-08T20:57:59.266 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:58 smithi149 ceph-mon[35042]: fsmap cephfs:5 7 up:standby 5 up:active 2024-03-08T20:57:59.266 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:58 smithi149 ceph-mon[35042]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-08T20:57:59.266 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:58 smithi149 ceph-mon[35042]: from='client.14751 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-03-08T20:57:59.266 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:58 smithi149 ceph-mon[35042]: pgmap v365: 97 pgs: 97 active+clean; 601 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s wr, 16 op/s 2024-03-08T20:57:59.266 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:58 smithi149 ceph-mon[35042]: from='client.14757 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-08T20:57:59.424 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:59.441+0000 7f3934800640 1 -- 172.21.15.107:0/378324771 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f392c02ce70 con 0x7f392c1094e0 2024-03-08T20:57:59.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:59.442+0000 7f3926a00640 1 -- 172.21.15.107:0/378324771 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+11393 (secure 0 0 0) 0x7f392803b2f0 con 0x7f392c1094e0 2024-03-08T20:57:59.426 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-08T20:57:59.426 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/133582743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":133582743},{"type":"v1","addr":"172.21.15.107:6835","nonce":133582743}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/3798982337","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":3798982337},{"type":"v1","addr":"172.21.15.107:6837","nonce":3798982337}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-08T20:57:50.223752+0000","modified":"2024-03-08T20:57:57.874727+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34475,"mds_1":34445,"mds_2":34415,"mds_3":34397,"mds_4":24457},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14625":{"gid":14625,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6839/3070059000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":3070059000},{"type":"v1","addr":"172.21.15.107:6839","nonce":3070059000}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14637":{"gid":14637,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6841/747096194","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6840","nonce":747096194},{"type":"v1","addr":"172.21.15.107:6841","nonce":747096194}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24400":{"gid":24400,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6833/3404762782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":3404762782},{"type":"v1","addr":"172.21.15.204:6833","nonce":3404762782}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24430":{"gid":24430,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/2072518651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":2072518651},{"type":"v1","addr":"172.21.15.204:6835","nonce":2072518651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24442":{"gid":24442,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6837/42644905","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":42644905},{"type":"v1","addr":"172.21.15.204:6837","nonce":42644905}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24457":{"gid":24457,"name":"i","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.204:6839/1857316822","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":1857316822},{"type":"v1","addr":"172.21.15.204:6839","nonce":1857316822}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34397":{"gid":34397,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":14,"addr":"172.21.15.149:6833/1962161726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":1962161726},{"type":"v1","addr":"172.21.15.149:6833","nonce":1962161726}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34415":{"gid":34415,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.149:6835/442657066","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":442657066},{"type":"v1","addr":"172.21.15.149:6835","nonce":442657066}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34445":{"gid":34445,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.149:6837/914546237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":914546237},{"type":"v1","addr":"172.21.15.149:6837","nonce":914546237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34475":{"gid":34475,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.149:6839/55290596","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":55290596},{"type":"v1","addr":"172.21.15.149:6839","nonce":55290596}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-08T20:57:59.427 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 27 2024-03-08T20:57:59.429 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:59.445+0000 7f3934800640 1 -- 172.21.15.107:0/378324771 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f39140a2910 msgr2=0x7f39140a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:57:59.429 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:59.445+0000 7f3934800640 1 --2- 172.21.15.107:0/378324771 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f39140a2910 0x7f39140a4dd0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f3920005fd0 tx=0x7f392000b410 comp rx=0 tx=0).stop 2024-03-08T20:57:59.429 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:59.445+0000 7f3934800640 1 -- 172.21.15.107:0/378324771 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f392c1094e0 msgr2=0x7f392c1a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:57:59.429 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:59.445+0000 7f3934800640 1 --2- 172.21.15.107:0/378324771 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f392c1094e0 0x7f392c1a5070 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f392802c700 tx=0x7f39280328a0 comp rx=0 tx=0).stop 2024-03-08T20:57:59.429 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:59.445+0000 7f3934800640 1 -- 172.21.15.107:0/378324771 shutdown_connections 2024-03-08T20:57:59.429 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:59.445+0000 7f3934800640 1 --2- 172.21.15.107:0/378324771 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f39140a2910 0x7f39140a4dd0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:59.429 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:59.445+0000 7f3934800640 1 --2- 172.21.15.107:0/378324771 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f392c1094e0 0x7f392c1a5070 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:59.429 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:59.445+0000 7f3934800640 1 --2- 172.21.15.107:0/378324771 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f392c108b20 0x7f392c19e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:59.429 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:59.445+0000 7f3934800640 1 --2- 172.21.15.107:0/378324771 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f392c107930 0x7f392c19dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:59.429 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:59.445+0000 7f3934800640 1 -- 172.21.15.107:0/378324771 >> 172.21.15.107:0/378324771 conn(0x7f392c076900 msgr2=0x7f392c10bea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:57:59.429 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:59.446+0000 7f3934800640 1 -- 172.21.15.107:0/378324771 shutdown_connections 2024-03-08T20:57:59.429 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:59.446+0000 7f3934800640 1 -- 172.21.15.107:0/378324771 wait complete. 2024-03-08T20:57:59.448 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-08T20:57:59.654 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:59.670+0000 7f7982c00640 1 -- 172.21.15.107:0/1403743886 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f797c104720 msgr2=0x7f797c104ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:57:59.654 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:59.670+0000 7f7982c00640 1 --2- 172.21.15.107:0/1403743886 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f797c104720 0x7f797c104ba0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f7964009900 tx=0x7f796402bc30 comp rx=0 tx=0).stop 2024-03-08T20:57:59.654 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:59.670+0000 7f7982c00640 1 -- 172.21.15.107:0/1403743886 shutdown_connections 2024-03-08T20:57:59.654 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:59.670+0000 7f7982c00640 1 --2- 172.21.15.107:0/1403743886 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f797c1050e0 0x7f797c10b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:59.654 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:59.670+0000 7f7982c00640 1 --2- 172.21.15.107:0/1403743886 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f797c104720 0x7f797c104ba0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:59.655 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:59.670+0000 7f7982c00640 1 --2- 172.21.15.107:0/1403743886 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f797c103530 0x7f797c103950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:59.655 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:59.670+0000 7f7982c00640 1 -- 172.21.15.107:0/1403743886 >> 172.21.15.107:0/1403743886 conn(0x7f797c0feae0 msgr2=0x7f797c100f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:57:59.655 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:59.670+0000 7f7982c00640 1 -- 172.21.15.107:0/1403743886 shutdown_connections 2024-03-08T20:57:59.655 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:59.670+0000 7f7982c00640 1 -- 172.21.15.107:0/1403743886 wait complete. 2024-03-08T20:57:59.657 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:59.671+0000 7f7982c00640 1 Processor -- start 2024-03-08T20:57:59.657 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:59.671+0000 7f7982c00640 1 -- start start 2024-03-08T20:57:59.657 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:59.672+0000 7f7982c00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f797c103530 0x7f797c199770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:57:59.658 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:59.672+0000 7f7982c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f797c104720 0x7f797c199cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:57:59.658 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:59.672+0000 7f7982c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f797c1050e0 0x7f797c1a0d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:57:59.658 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:59.672+0000 7f7982c00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f797c10f120 con 0x7f797c1050e0 2024-03-08T20:57:59.658 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:59.672+0000 7f7982c00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f797c10efa0 con 0x7f797c103530 2024-03-08T20:57:59.658 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:59.672+0000 7f7982c00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f797c10f2a0 con 0x7f797c104720 2024-03-08T20:57:59.658 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:59.672+0000 7f7980c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f797c1050e0 0x7f797c1a0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:57:59.658 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:59.672+0000 7f7980c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f797c1050e0 0x7f797c1a0d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:57238/0 (socket says 172.21.15.107:57238) 2024-03-08T20:57:59.658 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:59.672+0000 7f7980c00640 1 -- 172.21.15.107:0/1608738739 learned_addr learned my addr 172.21.15.107:0/1608738739 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T20:57:59.658 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:59.672+0000 7f7973400640 1 --2- 172.21.15.107:0/1608738739 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f797c104720 0x7f797c199cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:57:59.658 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:59.672+0000 7f797be00640 1 --2- 172.21.15.107:0/1608738739 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f797c103530 0x7f797c199770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:57:59.659 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:59.672+0000 7f7980c00640 1 -- 172.21.15.107:0/1608738739 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f797c104720 msgr2=0x7f797c199cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:57:59.659 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:59.672+0000 7f7980c00640 1 --2- 172.21.15.107:0/1608738739 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f797c104720 0x7f797c199cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:59.659 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:59.672+0000 7f7980c00640 1 -- 172.21.15.107:0/1608738739 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f797c103530 msgr2=0x7f797c199770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:57:59.659 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:59.672+0000 7f7980c00640 1 --2- 172.21.15.107:0/1608738739 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f797c103530 0x7f797c199770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:59.659 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:59.672+0000 7f7980c00640 1 -- 172.21.15.107:0/1608738739 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f797c1a1390 con 0x7f797c1050e0 2024-03-08T20:57:59.659 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:59.673+0000 7f7980c00640 1 --2- 172.21.15.107:0/1608738739 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f797c1050e0 0x7f797c1a0d20 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f797400cce0 tx=0x7f797400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:57:59.659 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:59.673+0000 7f7979600640 1 -- 172.21.15.107:0/1608738739 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f797400f450 con 0x7f797c1050e0 2024-03-08T20:57:59.659 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:59.673+0000 7f7979600640 1 -- 172.21.15.107:0/1608738739 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f797400faa0 con 0x7f797c1050e0 2024-03-08T20:57:59.659 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:59.673+0000 7f7982c00640 1 -- 172.21.15.107:0/1608738739 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f797c1a1680 con 0x7f797c1050e0 2024-03-08T20:57:59.659 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:59.673+0000 7f7979600640 1 -- 172.21.15.107:0/1608738739 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7974018b70 con 0x7f797c1050e0 2024-03-08T20:57:59.659 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:59.673+0000 7f7982c00640 1 -- 172.21.15.107:0/1608738739 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f797c1a1c00 con 0x7f797c1050e0 2024-03-08T20:57:59.660 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:59.674+0000 7f7982c00640 1 -- 172.21.15.107:0/1608738739 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f797c10a850 con 0x7f797c1050e0 2024-03-08T20:57:59.660 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:59.676+0000 7f7979600640 1 -- 172.21.15.107:0/1608738739 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) v1 ==== 137342+0+0 (secure 0 0 0) 0x7f7974018d10 con 0x7f797c1050e0 2024-03-08T20:57:59.661 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:59.677+0000 7f7979600640 1 --2- 172.21.15.107:0/1608738739 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f79600a2890 0x7f79600a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:57:59.661 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:59.678+0000 7f7979600640 1 -- 172.21.15.107:0/1608738739 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f79740cef60 con 0x7f797c1050e0 2024-03-08T20:57:59.661 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:59.678+0000 7f797be00640 1 --2- 172.21.15.107:0/1608738739 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f79600a2890 0x7f79600a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:57:59.662 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:59.678+0000 7f797be00640 1 --2- 172.21.15.107:0/1608738739 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f79600a2890 0x7f79600a4d50 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f797c104580 tx=0x7f796c008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:57:59.665 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:59.682+0000 7f7979600640 1 -- 172.21.15.107:0/1608738739 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f7974014030 con 0x7f797c1050e0 2024-03-08T20:57:59.766 INFO:journalctl@ceph.mds.b.smithi149.stdout:Mar 08 20:57:59 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-b[65616]: 2024-03-08T20:57:59.430+0000 7fb508800700 -1 mds.pinger is_rank_lagging: rank=3 was never sent ping request. 2024-03-08T20:57:59.766 INFO:journalctl@ceph.mds.b.smithi149.stdout:Mar 08 20:57:59 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-b[65616]: 2024-03-08T20:57:59.430+0000 7fb508800700 -1 mds.pinger is_rank_lagging: rank=4 was never sent ping request. 2024-03-08T20:57:59.864 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:59.880+0000 7f7982c00640 1 -- 172.21.15.107:0/1608738739 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f797c061ef0 con 0x7f797c1050e0 2024-03-08T20:57:59.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:59.883+0000 7f7979600640 1 -- 172.21.15.107:0/1608738739 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+22300 (secure 0 0 0) 0x7f797409c050 con 0x7f797c1050e0 2024-03-08T20:57:59.867 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-08T20:57:59.867 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":86,"fsid":"dc20f0f8-dd8c-11ee-95c6-87774f69a715","created":"2024-03-08T20:47:04.190034+0000","modified":"2024-03-08T20:57:51.779814+0000","last_up_change":"2024-03-08T20:56:43.135514+0000","last_in_change":"2024-03-08T20:56:17.577074+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-08T20:51:32.832786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-08T20:57:48.721227+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-08T20:57:49.707779+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8b1ddc64-4d20-4e85-ac66-f1c5ac599501","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":2453946095},{"type":"v1","addr":"172.21.15.107:6803","nonce":2453946095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":2453946095},{"type":"v1","addr":"172.21.15.107:6805","nonce":2453946095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":2453946095},{"type":"v1","addr":"172.21.15.107:6809","nonce":2453946095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":2453946095},{"type":"v1","addr":"172.21.15.107:6807","nonce":2453946095}]},"public_addr":"172.21.15.107:6803/2453946095","cluster_addr":"172.21.15.107:6805/2453946095","heartbeat_back_addr":"172.21.15.107:6809/2453946095","heartbeat_front_addr":"172.21.15.107:6807/2453946095","state":["exists","up"]},{"osd":1,"uuid":"5a803c09-202a-4777-ba2a-1da8eaf26a54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":3071316419},{"type":"v1","addr":"172.21.15.149:6801","nonce":3071316419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":3071316419},{"type":"v1","addr":"172.21.15.149:6803","nonce":3071316419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":3071316419},{"type":"v1","addr":"172.21.15.149:6807","nonce":3071316419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":3071316419},{"type":"v1","addr":"172.21.15.149:6805","nonce":3071316419}]},"public_addr":"172.21.15.149:6801/3071316419","cluster_addr":"172.21.15.149:6803/3071316419","heartbeat_back_addr":"172.21.15.149:6807/3071316419","heartbeat_front_addr":"172.21.15.149:6805/3071316419","state":["exists","up"]},{"osd":2,"uuid":"555c634f-347e-47dd-a943-e50ae2be073c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":1087852630},{"type":"v1","addr":"172.21.15.204:6801","nonce":1087852630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":1087852630},{"type":"v1","addr":"172.21.15.204:6803","nonce":1087852630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":1087852630},{"type":"v1","addr":"172.21.15.204:6807","nonce":1087852630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":1087852630},{"type":"v1","addr":"172.21.15.204:6805","nonce":1087852630}]},"public_addr":"172.21.15.204:6801/1087852630","cluster_addr":"172.21.15.204:6803/1087852630","heartbeat_back_addr":"172.21.15.204:6807/1087852630","heartbeat_front_addr":"172.21.15.204:6805/1087852630","state":["exists","up"]},{"osd":3,"uuid":"9e433d97-e202-48e0-9ed1-f0988fa14307","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":3123579789},{"type":"v1","addr":"172.21.15.107:6811","nonce":3123579789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":3123579789},{"type":"v1","addr":"172.21.15.107:6813","nonce":3123579789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":3123579789},{"type":"v1","addr":"172.21.15.107:6817","nonce":3123579789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":3123579789},{"type":"v1","addr":"172.21.15.107:6815","nonce":3123579789}]},"public_addr":"172.21.15.107:6811/3123579789","cluster_addr":"172.21.15.107:6813/3123579789","heartbeat_back_addr":"172.21.15.107:6817/3123579789","heartbeat_front_addr":"172.21.15.107:6815/3123579789","state":["exists","up"]},{"osd":4,"uuid":"27c53794-9d2b-487d-bfcf-ac4fd2ab061e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":726286918},{"type":"v1","addr":"172.21.15.149:6809","nonce":726286918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":726286918},{"type":"v1","addr":"172.21.15.149:6811","nonce":726286918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":726286918},{"type":"v1","addr":"172.21.15.149:6815","nonce":726286918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":726286918},{"type":"v1","addr":"172.21.15.149:6813","nonce":726286918}]},"public_addr":"172.21.15.149:6809/726286918","cluster_addr":"172.21.15.149:6811/726286918","heartbeat_back_addr":"172.21.15.149:6815/726286918","heartbeat_front_addr":"172.21.15.149:6813/726286918","state":["exists","up"]},{"osd":5,"uuid":"3a7778fb-46c5-40c0-842e-e988c45ea24d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":3143887820},{"type":"v1","addr":"172.21.15.204:6809","nonce":3143887820}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":3143887820},{"type":"v1","addr":"172.21.15.204:6811","nonce":3143887820}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":3143887820},{"type":"v1","addr":"172.21.15.204:6815","nonce":3143887820}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":3143887820},{"type":"v1","addr":"172.21.15.204:6813","nonce":3143887820}]},"public_addr":"172.21.15.204:6809/3143887820","cluster_addr":"172.21.15.204:6811/3143887820","heartbeat_back_addr":"172.21.15.204:6815/3143887820","heartbeat_front_addr":"172.21.15.204:6813/3143887820","state":["exists","up"]},{"osd":6,"uuid":"d2e05a56-be4f-4435-9b76-ff9aa01e4ac3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":2912452950},{"type":"v1","addr":"172.21.15.107:6819","nonce":2912452950}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":2912452950},{"type":"v1","addr":"172.21.15.107:6821","nonce":2912452950}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":2912452950},{"type":"v1","addr":"172.21.15.107:6825","nonce":2912452950}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":2912452950},{"type":"v1","addr":"172.21.15.107:6823","nonce":2912452950}]},"public_addr":"172.21.15.107:6819/2912452950","cluster_addr":"172.21.15.107:6821/2912452950","heartbeat_back_addr":"172.21.15.107:6825/2912452950","heartbeat_front_addr":"172.21.15.107:6823/2912452950","state":["exists","up"]},{"osd":7,"uuid":"a7dd26ee-c118-4a9a-9e19-142c2680902d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":3835863809},{"type":"v1","addr":"172.21.15.149:6817","nonce":3835863809}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":3835863809},{"type":"v1","addr":"172.21.15.149:6819","nonce":3835863809}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":3835863809},{"type":"v1","addr":"172.21.15.149:6823","nonce":3835863809}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":3835863809},{"type":"v1","addr":"172.21.15.149:6821","nonce":3835863809}]},"public_addr":"172.21.15.149:6817/3835863809","cluster_addr":"172.21.15.149:6819/3835863809","heartbeat_back_addr":"172.21.15.149:6823/3835863809","heartbeat_front_addr":"172.21.15.149:6821/3835863809","state":["exists","up"]},{"osd":8,"uuid":"baaaa8dd-0e1f-4bae-86bf-9e429d985f23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":1010601477},{"type":"v1","addr":"172.21.15.204:6817","nonce":1010601477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":1010601477},{"type":"v1","addr":"172.21.15.204:6819","nonce":1010601477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":1010601477},{"type":"v1","addr":"172.21.15.204:6823","nonce":1010601477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":1010601477},{"type":"v1","addr":"172.21.15.204:6821","nonce":1010601477}]},"public_addr":"172.21.15.204:6817/1010601477","cluster_addr":"172.21.15.204:6819/1010601477","heartbeat_back_addr":"172.21.15.204:6823/1010601477","heartbeat_front_addr":"172.21.15.204:6821/1010601477","state":["exists","up"]},{"osd":9,"uuid":"8df80dfa-e7ff-410e-824c-b1dce111b02f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":3334759819},{"type":"v1","addr":"172.21.15.107:6827","nonce":3334759819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3334759819},{"type":"v1","addr":"172.21.15.107:6829","nonce":3334759819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":3334759819},{"type":"v1","addr":"172.21.15.107:6833","nonce":3334759819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":3334759819},{"type":"v1","addr":"172.21.15.107:6831","nonce":3334759819}]},"public_addr":"172.21.15.107:6827/3334759819","cluster_addr":"172.21.15.107:6829/3334759819","heartbeat_back_addr":"172.21.15.107:6833/3334759819","heartbeat_front_addr":"172.21.15.107:6831/3334759819","state":["exists","up"]},{"osd":10,"uuid":"338945b2-20c0-40b2-aaca-062cd4adee04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":3469284631},{"type":"v1","addr":"172.21.15.149:6825","nonce":3469284631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":3469284631},{"type":"v1","addr":"172.21.15.149:6827","nonce":3469284631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":3469284631},{"type":"v1","addr":"172.21.15.149:6831","nonce":3469284631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":3469284631},{"type":"v1","addr":"172.21.15.149:6829","nonce":3469284631}]},"public_addr":"172.21.15.149:6825/3469284631","cluster_addr":"172.21.15.149:6827/3469284631","heartbeat_back_addr":"172.21.15.149:6831/3469284631","heartbeat_front_addr":"172.21.15.149:6829/3469284631","state":["exists","up"]},{"osd":11,"uuid":"acccfe4f-d307-4a64-8f4d-fba27bfbe09d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":3097772473},{"type":"v1","addr":"172.21.15.204:6825","nonce":3097772473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":3097772473},{"type":"v1","addr":"172.21.15.204:6827","nonce":3097772473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":3097772473},{"type":"v1","addr":"172.21.15.204:6831","nonce":3097772473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":3097772473},{"type":"v1","addr":"172.21.15.204:6829","nonce":3097772473}]},"public_addr":"172.21.15.204:6825/3097772473","cluster_addr":"172.21.15.204:6827/3097772473","heartbeat_back_addr":"172.21.15.204:6831/3097772473","heartbeat_front_addr":"172.21.15.204:6829/3097772473","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T20:50:25.862753+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T20:50:58.167663+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T20:51:30.312816+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T20:52:05.487917+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T20:52:39.348354+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T20:53:12.899002+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T20:53:47.630837+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T20:54:21.444552+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T20:54:57.101160+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T20:55:31.841786+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T20:56:05.827161+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T20:56:40.611097+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.107:0/1514666962":"2024-03-09T20:48:20.149406+0000","172.21.15.107:0/1886016707":"2024-03-09T20:48:20.149406+0000","172.21.15.107:6801/426372446":"2024-03-09T20:48:20.149406+0000","172.21.15.107:6800/426372446":"2024-03-09T20:48:20.149406+0000","172.21.15.107:0/2392547434":"2024-03-09T20:47:45.706790+0000","172.21.15.107:0/1206995985":"2024-03-09T20:47:45.706790+0000","172.21.15.107:6800/3549936519":"2024-03-09T20:47:45.706790+0000","172.21.15.107:6801/3549936519":"2024-03-09T20:47:45.706790+0000","172.21.15.107:0/2482452215":"2024-03-09T20:47:45.706790+0000","172.21.15.107:0/808124991":"2024-03-09T20:48:20.149406+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-08T20:57:59.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:59.886+0000 7f7982c00640 1 -- 172.21.15.107:0/1608738739 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f79600a2890 msgr2=0x7f79600a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:57:59.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:59.886+0000 7f7982c00640 1 --2- 172.21.15.107:0/1608738739 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f79600a2890 0x7f79600a4d50 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f797c104580 tx=0x7f796c008040 comp rx=0 tx=0).stop 2024-03-08T20:57:59.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:59.886+0000 7f7982c00640 1 -- 172.21.15.107:0/1608738739 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f797c1050e0 msgr2=0x7f797c1a0d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:57:59.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:59.886+0000 7f7982c00640 1 --2- 172.21.15.107:0/1608738739 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f797c1050e0 0x7f797c1a0d20 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f797400cce0 tx=0x7f797400a620 comp rx=0 tx=0).stop 2024-03-08T20:57:59.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:59.886+0000 7f7982c00640 1 -- 172.21.15.107:0/1608738739 shutdown_connections 2024-03-08T20:57:59.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:59.886+0000 7f7982c00640 1 --2- 172.21.15.107:0/1608738739 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f79600a2890 0x7f79600a4d50 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:59.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:59.886+0000 7f7982c00640 1 --2- 172.21.15.107:0/1608738739 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f797c1050e0 0x7f797c1a0d20 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:59.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:59.886+0000 7f7982c00640 1 --2- 172.21.15.107:0/1608738739 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f797c104720 0x7f797c199cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:59.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:59.886+0000 7f7982c00640 1 --2- 172.21.15.107:0/1608738739 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f797c103530 0x7f797c199770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:57:59.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:59.886+0000 7f7982c00640 1 -- 172.21.15.107:0/1608738739 >> 172.21.15.107:0/1608738739 conn(0x7f797c0feae0 msgr2=0x7f797c107a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:57:59.871 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:59.887+0000 7f7982c00640 1 -- 172.21.15.107:0/1608738739 shutdown_connections 2024-03-08T20:57:59.871 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:57:59.887+0000 7f7982c00640 1 -- 172.21.15.107:0/1608738739 wait complete. 2024-03-08T20:57:59.891 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-03-08T20:57:59.892 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-08T20:58:00.100 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:00.115+0000 7fe92e600640 1 -- 172.21.15.107:0/2961887071 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe928108b20 msgr2=0x7fe928108fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:58:00.100 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:00.115+0000 7fe92e600640 1 --2- 172.21.15.107:0/2961887071 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe928108b20 0x7fe928108fa0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fe910009900 tx=0x7fe91002bc30 comp rx=0 tx=0).stop 2024-03-08T20:58:00.101 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:00.116+0000 7fe92e600640 1 -- 172.21.15.107:0/2961887071 shutdown_connections 2024-03-08T20:58:00.101 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:00.116+0000 7fe92e600640 1 --2- 172.21.15.107:0/2961887071 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe9281094e0 0x7fe92810fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:58:00.101 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:00.116+0000 7fe92e600640 1 --2- 172.21.15.107:0/2961887071 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe928108b20 0x7fe928108fa0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:58:00.101 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:00.116+0000 7fe92e600640 1 --2- 172.21.15.107:0/2961887071 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe928107930 0x7fe928107d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:58:00.101 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:00.116+0000 7fe92e600640 1 -- 172.21.15.107:0/2961887071 >> 172.21.15.107:0/2961887071 conn(0x7fe928076900 msgr2=0x7fe928078d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:58:00.101 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:00.116+0000 7fe92e600640 1 -- 172.21.15.107:0/2961887071 shutdown_connections 2024-03-08T20:58:00.101 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:00.116+0000 7fe92e600640 1 -- 172.21.15.107:0/2961887071 wait complete. 2024-03-08T20:58:00.103 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:00.117+0000 7fe92e600640 1 Processor -- start 2024-03-08T20:58:00.103 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:00.117+0000 7fe92e600640 1 -- start start 2024-03-08T20:58:00.103 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:00.117+0000 7fe92e600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe928107930 0x7fe92819dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:58:00.103 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:00.117+0000 7fe92e600640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe928108b20 0x7fe92819e110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:58:00.103 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:00.117+0000 7fe92e600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe9281094e0 0x7fe9281a5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:58:00.103 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:00.117+0000 7fe92e600640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe928113530 con 0x7fe928107930 2024-03-08T20:58:00.103 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:00.117+0000 7fe92e600640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9281133b0 con 0x7fe928108b20 2024-03-08T20:58:00.103 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:00.117+0000 7fe92e600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9281136b0 con 0x7fe9281094e0 2024-03-08T20:58:00.103 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:00.117+0000 7fe927400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe928107930 0x7fe92819dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:58:00.104 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:00.117+0000 7fe927400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe928107930 0x7fe92819dbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:57258/0 (socket says 172.21.15.107:57258) 2024-03-08T20:58:00.104 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:00.117+0000 7fe927400640 1 -- 172.21.15.107:0/3314459932 learned_addr learned my addr 172.21.15.107:0/3314459932 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T20:58:00.104 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:00.118+0000 7fe927400640 1 -- 172.21.15.107:0/3314459932 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe9281094e0 msgr2=0x7fe9281a5180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:58:00.104 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:00.118+0000 7fe927400640 1 --2- 172.21.15.107:0/3314459932 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe9281094e0 0x7fe9281a5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:58:00.104 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:00.118+0000 7fe927400640 1 -- 172.21.15.107:0/3314459932 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe928108b20 msgr2=0x7fe92819e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:58:00.104 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:00.118+0000 7fe91fe00640 1 --2- 172.21.15.107:0/3314459932 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe928108b20 0x7fe92819e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:58:00.104 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:00.118+0000 7fe927400640 1 --2- 172.21.15.107:0/3314459932 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe928108b20 0x7fe92819e110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:58:00.104 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:00.118+0000 7fe927400640 1 -- 172.21.15.107:0/3314459932 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9281a57f0 con 0x7fe928107930 2024-03-08T20:58:00.104 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:00.118+0000 7fe927400640 1 --2- 172.21.15.107:0/3314459932 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe928107930 0x7fe92819dbd0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fe91800d950 tx=0x7fe91800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:58:00.104 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:00.118+0000 7fe924c00640 1 -- 172.21.15.107:0/3314459932 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe918017070 con 0x7fe928107930 2024-03-08T20:58:00.104 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:00.118+0000 7fe92e600640 1 -- 172.21.15.107:0/3314459932 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9281a5ae0 con 0x7fe928107930 2024-03-08T20:58:00.104 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:00.118+0000 7fe924c00640 1 -- 172.21.15.107:0/3314459932 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fe918009c30 con 0x7fe928107930 2024-03-08T20:58:00.104 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:00.118+0000 7fe924c00640 1 -- 172.21.15.107:0/3314459932 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe91800f620 con 0x7fe928107930 2024-03-08T20:58:00.104 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:00.119+0000 7fe92e600640 1 -- 172.21.15.107:0/3314459932 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9281a6060 con 0x7fe928107930 2024-03-08T20:58:00.105 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:00.120+0000 7fe92e600640 1 -- 172.21.15.107:0/3314459932 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe92810ecd0 con 0x7fe928107930 2024-03-08T20:58:00.106 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:00.122+0000 7fe924c00640 1 -- 172.21.15.107:0/3314459932 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) v1 ==== 137342+0+0 (secure 0 0 0) 0x7fe91801a070 con 0x7fe928107930 2024-03-08T20:58:00.107 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:00.123+0000 7fe924c00640 1 --2- 172.21.15.107:0/3314459932 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fe90c0a2890 0x7fe90c0a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:58:00.107 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:00.123+0000 7fe924c00640 1 -- 172.21.15.107:0/3314459932 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fe9180ce620 con 0x7fe928107930 2024-03-08T20:58:00.107 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:00.123+0000 7fe91fe00640 1 --2- 172.21.15.107:0/3314459932 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fe90c0a2890 0x7fe90c0a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:58:00.108 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:00.124+0000 7fe91fe00640 1 --2- 172.21.15.107:0/3314459932 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fe90c0a2890 0x7fe90c0a4d50 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fe9100096c0 tx=0x7fe910005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:58:00.111 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:00.127+0000 7fe924c00640 1 -- 172.21.15.107:0/3314459932 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7fe918009de0 con 0x7fe928107930 2024-03-08T20:58:00.218 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:59 smithi107 ceph-mon[31381]: mgrmap e17: x(active, since 9m), standbys: z, y 2024-03-08T20:58:00.218 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:59 smithi107 ceph-mon[31381]: from='client.14763 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-08T20:58:00.218 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:59 smithi107 ceph-mon[31381]: from='client.? 172.21.15.107:0/378324771' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-08T20:58:00.218 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:57:59 smithi107 ceph-mon[31381]: from='client.? 172.21.15.107:0/1608738739' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-08T20:58:00.266 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:59 smithi149 ceph-mon[35042]: mgrmap e17: x(active, since 9m), standbys: z, y 2024-03-08T20:58:00.266 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:59 smithi149 ceph-mon[35042]: from='client.14763 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-08T20:58:00.266 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:59 smithi149 ceph-mon[35042]: from='client.? 172.21.15.107:0/378324771' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-08T20:58:00.266 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:57:59 smithi149 ceph-mon[35042]: from='client.? 172.21.15.107:0/1608738739' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-08T20:58:00.307 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:00.323+0000 7fe92e600640 1 -- 172.21.15.107:0/3314459932 --> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe928061730 con 0x7fe90c0a2890 2024-03-08T20:58:00.333 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:00.349+0000 7fe924c00640 1 -- 172.21.15.107:0/3314459932 <== mgr.14152 v2:172.21.15.107:6800/3376636618 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+259033 (secure 0 0 0) 0x7fe928061730 con 0x7fe90c0a2890 2024-03-08T20:58:00.337 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-08T20:58:00.339 INFO:teuthology.orchestra.run.smithi107.stderr:dumped all 2024-03-08T20:58:00.341 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:00.355+0000 7fe92e600640 1 -- 172.21.15.107:0/3314459932 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fe90c0a2890 msgr2=0x7fe90c0a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:58:00.341 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:00.355+0000 7fe92e600640 1 --2- 172.21.15.107:0/3314459932 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fe90c0a2890 0x7fe90c0a4d50 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fe9100096c0 tx=0x7fe910005fd0 comp rx=0 tx=0).stop 2024-03-08T20:58:00.341 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:00.355+0000 7fe92e600640 1 -- 172.21.15.107:0/3314459932 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe928107930 msgr2=0x7fe92819dbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:58:00.341 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:00.355+0000 7fe92e600640 1 --2- 172.21.15.107:0/3314459932 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe928107930 0x7fe92819dbd0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fe91800d950 tx=0x7fe91800de10 comp rx=0 tx=0).stop 2024-03-08T20:58:00.341 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:00.355+0000 7fe92e600640 1 -- 172.21.15.107:0/3314459932 shutdown_connections 2024-03-08T20:58:00.341 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:00.355+0000 7fe92e600640 1 --2- 172.21.15.107:0/3314459932 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fe90c0a2890 0x7fe90c0a4d50 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:58:00.341 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:00.355+0000 7fe92e600640 1 --2- 172.21.15.107:0/3314459932 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe9281094e0 0x7fe9281a5180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:58:00.341 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:00.355+0000 7fe92e600640 1 --2- 172.21.15.107:0/3314459932 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe928108b20 0x7fe92819e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:58:00.342 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:00.355+0000 7fe92e600640 1 --2- 172.21.15.107:0/3314459932 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe928107930 0x7fe92819dbd0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:58:00.342 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:00.355+0000 7fe92e600640 1 -- 172.21.15.107:0/3314459932 >> 172.21.15.107:0/3314459932 conn(0x7fe928076900 msgr2=0x7fe92810bea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:58:00.342 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:00.355+0000 7fe92e600640 1 -- 172.21.15.107:0/3314459932 shutdown_connections 2024-03-08T20:58:00.342 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:00.355+0000 7fe92e600640 1 -- 172.21.15.107:0/3314459932 wait complete. 2024-03-08T20:58:00.359 INFO:teuthology.orchestra.run.smithi107.stdout:{"pg_ready":true,"pg_map":{"version":366,"stamp":"2024-03-08T20:58:00.314953+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":617191,"num_objects":92,"num_object_clones":0,"num_object_copies":276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":92,"num_whiteouts":0,"num_read":150,"num_read_kb":131,"num_write":385,"num_write_kb":3010,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":60,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":312,"ondisk_log_size":312,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":368828,"kb_used_data":11352,"kb_used_omap":18,"kb_used_meta":357357,"kb_avail":1124572996,"statfs":{"total":1151940427776,"available":1151562747904,"internally_reserved":0,"allocated":11624448,"data_stored":7127524,"data_compressed":107202,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19138,"internal_metadata":365933886},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1],"upper_bound":8},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":26823,"num_objects":90,"num_object_clones":0,"num_object_copies":270,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":90,"num_whiteouts":0,"num_read":14,"num_read_kb":12,"num_write":204,"num_write_kb":72,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":60,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.555043"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-08T20:57:51.805111+0000","last_change":"2024-03-08T20:57:50.800421+0000","last_active":"2024-03-08T20:57:51.805111+0000","last_peered":"2024-03-08T20:57:51.805111+0000","last_clean":"2024-03-08T20:57:51.805111+0000","last_became_active":"2024-03-08T20:57:50.799682+0000","last_became_peered":"2024-03-08T20:57:50.799682+0000","last_unstale":"2024-03-08T20:57:51.805111+0000","last_undegraded":"2024-03-08T20:57:51.805111+0000","last_fullsized":"2024-03-08T20:57:51.805111+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T07:25:11.394329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-08T20:57:52.073778+0000","last_change":"2024-03-08T20:57:50.800332+0000","last_active":"2024-03-08T20:57:52.073778+0000","last_peered":"2024-03-08T20:57:52.073778+0000","last_clean":"2024-03-08T20:57:52.073778+0000","last_became_active":"2024-03-08T20:57:50.799981+0000","last_became_peered":"2024-03-08T20:57:50.799981+0000","last_unstale":"2024-03-08T20:57:52.073778+0000","last_undegraded":"2024-03-08T20:57:52.073778+0000","last_fullsized":"2024-03-08T20:57:52.073778+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T06:08:19.618115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-08T20:57:52.073846+0000","last_change":"2024-03-08T20:57:50.800982+0000","last_active":"2024-03-08T20:57:52.073846+0000","last_peered":"2024-03-08T20:57:52.073846+0000","last_clean":"2024-03-08T20:57:52.073846+0000","last_became_active":"2024-03-08T20:57:50.800661+0000","last_became_peered":"2024-03-08T20:57:50.800661+0000","last_unstale":"2024-03-08T20:57:52.073846+0000","last_undegraded":"2024-03-08T20:57:52.073846+0000","last_fullsized":"2024-03-08T20:57:52.073846+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T22:33:08.869516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-08T20:57:52.075049+0000","last_change":"2024-03-08T20:57:50.792110+0000","last_active":"2024-03-08T20:57:52.075049+0000","last_peered":"2024-03-08T20:57:52.075049+0000","last_clean":"2024-03-08T20:57:52.075049+0000","last_became_active":"2024-03-08T20:57:50.790781+0000","last_became_peered":"2024-03-08T20:57:50.790781+0000","last_unstale":"2024-03-08T20:57:52.075049+0000","last_undegraded":"2024-03-08T20:57:52.075049+0000","last_fullsized":"2024-03-08T20:57:52.075049+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T05:28:50.939269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-08T20:57:52.073859+0000","last_change":"2024-03-08T20:57:50.804087+0000","last_active":"2024-03-08T20:57:52.073859+0000","last_peered":"2024-03-08T20:57:52.073859+0000","last_clean":"2024-03-08T20:57:52.073859+0000","last_became_active":"2024-03-08T20:57:50.803704+0000","last_became_peered":"2024-03-08T20:57:50.803704+0000","last_unstale":"2024-03-08T20:57:52.073859+0000","last_undegraded":"2024-03-08T20:57:52.073859+0000","last_fullsized":"2024-03-08T20:57:52.073859+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T21:23:16.770787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-08T20:57:52.075693+0000","last_change":"2024-03-08T20:57:50.789245+0000","last_active":"2024-03-08T20:57:52.075693+0000","last_peered":"2024-03-08T20:57:52.075693+0000","last_clean":"2024-03-08T20:57:52.075693+0000","last_became_active":"2024-03-08T20:57:50.788921+0000","last_became_peered":"2024-03-08T20:57:50.788921+0000","last_unstale":"2024-03-08T20:57:52.075693+0000","last_undegraded":"2024-03-08T20:57:52.075693+0000","last_fullsized":"2024-03-08T20:57:52.075693+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T08:57:30.368714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-08T20:57:52.073207+0000","last_change":"2024-03-08T20:57:50.800725+0000","last_active":"2024-03-08T20:57:52.073207+0000","last_peered":"2024-03-08T20:57:52.073207+0000","last_clean":"2024-03-08T20:57:52.073207+0000","last_became_active":"2024-03-08T20:57:50.800317+0000","last_became_peered":"2024-03-08T20:57:50.800317+0000","last_unstale":"2024-03-08T20:57:52.073207+0000","last_undegraded":"2024-03-08T20:57:52.073207+0000","last_fullsized":"2024-03-08T20:57:52.073207+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T05:44:01.402547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-08T20:57:51.791880+0000","last_change":"2024-03-08T20:57:50.801543+0000","last_active":"2024-03-08T20:57:51.791880+0000","last_peered":"2024-03-08T20:57:51.791880+0000","last_clean":"2024-03-08T20:57:51.791880+0000","last_became_active":"2024-03-08T20:57:50.801111+0000","last_became_peered":"2024-03-08T20:57:50.801111+0000","last_unstale":"2024-03-08T20:57:51.791880+0000","last_undegraded":"2024-03-08T20:57:51.791880+0000","last_fullsized":"2024-03-08T20:57:51.791880+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T06:32:55.234372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-08T20:57:51.795132+0000","last_change":"2024-03-08T20:57:50.798169+0000","last_active":"2024-03-08T20:57:51.795132+0000","last_peered":"2024-03-08T20:57:51.795132+0000","last_clean":"2024-03-08T20:57:51.795132+0000","last_became_active":"2024-03-08T20:57:50.797817+0000","last_became_peered":"2024-03-08T20:57:50.797817+0000","last_unstale":"2024-03-08T20:57:51.795132+0000","last_undegraded":"2024-03-08T20:57:51.795132+0000","last_fullsized":"2024-03-08T20:57:51.795132+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T04:01:46.179415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-08T20:57:52.075986+0000","last_change":"2024-03-08T20:57:50.790757+0000","last_active":"2024-03-08T20:57:52.075986+0000","last_peered":"2024-03-08T20:57:52.075986+0000","last_clean":"2024-03-08T20:57:52.075986+0000","last_became_active":"2024-03-08T20:57:50.790219+0000","last_became_peered":"2024-03-08T20:57:50.790219+0000","last_unstale":"2024-03-08T20:57:52.075986+0000","last_undegraded":"2024-03-08T20:57:52.075986+0000","last_fullsized":"2024-03-08T20:57:52.075986+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T07:55:54.945727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-08T20:57:52.074250+0000","last_change":"2024-03-08T20:57:50.809102+0000","last_active":"2024-03-08T20:57:52.074250+0000","last_peered":"2024-03-08T20:57:52.074250+0000","last_clean":"2024-03-08T20:57:52.074250+0000","last_became_active":"2024-03-08T20:57:50.808647+0000","last_became_peered":"2024-03-08T20:57:50.808647+0000","last_unstale":"2024-03-08T20:57:52.074250+0000","last_undegraded":"2024-03-08T20:57:52.074250+0000","last_fullsized":"2024-03-08T20:57:52.074250+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T00:29:26.317611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-08T20:57:52.074880+0000","last_change":"2024-03-08T20:57:50.791791+0000","last_active":"2024-03-08T20:57:52.074880+0000","last_peered":"2024-03-08T20:57:52.074880+0000","last_clean":"2024-03-08T20:57:52.074880+0000","last_became_active":"2024-03-08T20:57:50.790264+0000","last_became_peered":"2024-03-08T20:57:50.790264+0000","last_unstale":"2024-03-08T20:57:52.074880+0000","last_undegraded":"2024-03-08T20:57:52.074880+0000","last_fullsized":"2024-03-08T20:57:52.074880+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T08:54:30.221330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-08T20:57:52.073409+0000","last_change":"2024-03-08T20:57:50.793990+0000","last_active":"2024-03-08T20:57:52.073409+0000","last_peered":"2024-03-08T20:57:52.073409+0000","last_clean":"2024-03-08T20:57:52.073409+0000","last_became_active":"2024-03-08T20:57:50.792304+0000","last_became_peered":"2024-03-08T20:57:50.792304+0000","last_unstale":"2024-03-08T20:57:52.073409+0000","last_undegraded":"2024-03-08T20:57:52.073409+0000","last_fullsized":"2024-03-08T20:57:52.073409+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T00:21:57.491218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-08T20:57:52.073068+0000","last_change":"2024-03-08T20:57:50.787621+0000","last_active":"2024-03-08T20:57:52.073068+0000","last_peered":"2024-03-08T20:57:52.073068+0000","last_clean":"2024-03-08T20:57:52.073068+0000","last_became_active":"2024-03-08T20:57:50.786461+0000","last_became_peered":"2024-03-08T20:57:50.786461+0000","last_unstale":"2024-03-08T20:57:52.073068+0000","last_undegraded":"2024-03-08T20:57:52.073068+0000","last_fullsized":"2024-03-08T20:57:52.073068+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T08:35:32.513942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-08T20:57:52.073416+0000","last_change":"2024-03-08T20:57:50.789157+0000","last_active":"2024-03-08T20:57:52.073416+0000","last_peered":"2024-03-08T20:57:52.073416+0000","last_clean":"2024-03-08T20:57:52.073416+0000","last_became_active":"2024-03-08T20:57:50.788828+0000","last_became_peered":"2024-03-08T20:57:50.788828+0000","last_unstale":"2024-03-08T20:57:52.073416+0000","last_undegraded":"2024-03-08T20:57:52.073416+0000","last_fullsized":"2024-03-08T20:57:52.073416+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T05:43:53.398306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-08T20:57:51.793598+0000","last_change":"2024-03-08T20:57:50.796888+0000","last_active":"2024-03-08T20:57:51.793598+0000","last_peered":"2024-03-08T20:57:51.793598+0000","last_clean":"2024-03-08T20:57:51.793598+0000","last_became_active":"2024-03-08T20:57:50.795797+0000","last_became_peered":"2024-03-08T20:57:50.795797+0000","last_unstale":"2024-03-08T20:57:51.793598+0000","last_undegraded":"2024-03-08T20:57:51.793598+0000","last_fullsized":"2024-03-08T20:57:51.793598+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T00:43:53.838877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-08T20:57:51.805178+0000","last_change":"2024-03-08T20:57:50.801849+0000","last_active":"2024-03-08T20:57:51.805178+0000","last_peered":"2024-03-08T20:57:51.805178+0000","last_clean":"2024-03-08T20:57:51.805178+0000","last_became_active":"2024-03-08T20:57:50.801375+0000","last_became_peered":"2024-03-08T20:57:50.801375+0000","last_unstale":"2024-03-08T20:57:51.805178+0000","last_undegraded":"2024-03-08T20:57:51.805178+0000","last_fullsized":"2024-03-08T20:57:51.805178+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T03:16:44.652264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-08T20:57:52.074580+0000","last_change":"2024-03-08T20:57:50.790493+0000","last_active":"2024-03-08T20:57:52.074580+0000","last_peered":"2024-03-08T20:57:52.074580+0000","last_clean":"2024-03-08T20:57:52.074580+0000","last_became_active":"2024-03-08T20:57:50.789074+0000","last_became_peered":"2024-03-08T20:57:50.789074+0000","last_unstale":"2024-03-08T20:57:52.074580+0000","last_undegraded":"2024-03-08T20:57:52.074580+0000","last_fullsized":"2024-03-08T20:57:52.074580+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T05:52:45.700669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-08T20:57:52.073357+0000","last_change":"2024-03-08T20:57:50.794348+0000","last_active":"2024-03-08T20:57:52.073357+0000","last_peered":"2024-03-08T20:57:52.073357+0000","last_clean":"2024-03-08T20:57:52.073357+0000","last_became_active":"2024-03-08T20:57:50.793962+0000","last_became_peered":"2024-03-08T20:57:50.793962+0000","last_unstale":"2024-03-08T20:57:52.073357+0000","last_undegraded":"2024-03-08T20:57:52.073357+0000","last_fullsized":"2024-03-08T20:57:52.073357+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T01:38:05.146148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"86'13","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-08T20:57:57.897048+0000","last_change":"2024-03-08T20:57:49.792250+0000","last_active":"2024-03-08T20:57:57.897048+0000","last_peered":"2024-03-08T20:57:57.897048+0000","last_clean":"2024-03-08T20:57:57.897048+0000","last_became_active":"2024-03-08T20:57:49.791863+0000","last_became_peered":"2024-03-08T20:57:49.791863+0000","last_unstale":"2024-03-08T20:57:57.897048+0000","last_undegraded":"2024-03-08T20:57:57.897048+0000","last_fullsized":"2024-03-08T20:57:57.897048+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_clean_scrub_stamp":"2024-03-08T20:57:48.739829+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T05:37:40.942754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-08T20:57:51.793649+0000","last_change":"2024-03-08T20:57:50.802789+0000","last_active":"2024-03-08T20:57:51.793649+0000","last_peered":"2024-03-08T20:57:51.793649+0000","last_clean":"2024-03-08T20:57:51.793649+0000","last_became_active":"2024-03-08T20:57:50.802437+0000","last_became_peered":"2024-03-08T20:57:50.802437+0000","last_unstale":"2024-03-08T20:57:51.793649+0000","last_undegraded":"2024-03-08T20:57:51.793649+0000","last_fullsized":"2024-03-08T20:57:51.793649+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T23:47:39.882241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-08T20:57:53.381760+0000","last_change":"2024-03-08T20:57:49.785390+0000","last_active":"2024-03-08T20:57:53.381760+0000","last_peered":"2024-03-08T20:57:53.381760+0000","last_clean":"2024-03-08T20:57:53.381760+0000","last_became_active":"2024-03-08T20:57:49.784506+0000","last_became_peered":"2024-03-08T20:57:49.784506+0000","last_unstale":"2024-03-08T20:57:53.381760+0000","last_undegraded":"2024-03-08T20:57:53.381760+0000","last_fullsized":"2024-03-08T20:57:53.381760+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_clean_scrub_stamp":"2024-03-08T20:57:48.739829+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T03:21:54.331376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-08T20:57:52.074165+0000","last_change":"2024-03-08T20:57:50.796868+0000","last_active":"2024-03-08T20:57:52.074165+0000","last_peered":"2024-03-08T20:57:52.074165+0000","last_clean":"2024-03-08T20:57:52.074165+0000","last_became_active":"2024-03-08T20:57:50.796163+0000","last_became_peered":"2024-03-08T20:57:50.796163+0000","last_unstale":"2024-03-08T20:57:52.074165+0000","last_undegraded":"2024-03-08T20:57:52.074165+0000","last_fullsized":"2024-03-08T20:57:52.074165+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T21:26:46.335292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"86'12","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-08T20:57:57.882244+0000","last_change":"2024-03-08T20:57:49.794940+0000","last_active":"2024-03-08T20:57:57.882244+0000","last_peered":"2024-03-08T20:57:57.882244+0000","last_clean":"2024-03-08T20:57:57.882244+0000","last_became_active":"2024-03-08T20:57:49.794386+0000","last_became_peered":"2024-03-08T20:57:49.794386+0000","last_unstale":"2024-03-08T20:57:57.882244+0000","last_undegraded":"2024-03-08T20:57:57.882244+0000","last_fullsized":"2024-03-08T20:57:57.882244+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_clean_scrub_stamp":"2024-03-08T20:57:48.739829+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T04:21:09.953351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-08T20:57:51.794091+0000","last_change":"2024-03-08T20:57:50.794804+0000","last_active":"2024-03-08T20:57:51.794091+0000","last_peered":"2024-03-08T20:57:51.794091+0000","last_clean":"2024-03-08T20:57:51.794091+0000","last_became_active":"2024-03-08T20:57:50.794380+0000","last_became_peered":"2024-03-08T20:57:50.794380+0000","last_unstale":"2024-03-08T20:57:51.794091+0000","last_undegraded":"2024-03-08T20:57:51.794091+0000","last_fullsized":"2024-03-08T20:57:51.794091+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T05:32:21.015740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"86'5","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-08T20:57:56.893654+0000","last_change":"2024-03-08T20:57:49.790881+0000","last_active":"2024-03-08T20:57:56.893654+0000","last_peered":"2024-03-08T20:57:56.893654+0000","last_clean":"2024-03-08T20:57:56.893654+0000","last_became_active":"2024-03-08T20:57:49.790562+0000","last_became_peered":"2024-03-08T20:57:49.790562+0000","last_unstale":"2024-03-08T20:57:56.893654+0000","last_undegraded":"2024-03-08T20:57:56.893654+0000","last_fullsized":"2024-03-08T20:57:56.893654+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_clean_scrub_stamp":"2024-03-08T20:57:48.739829+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T23:57:40.542756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-08T20:57:51.794667+0000","last_change":"2024-03-08T20:57:50.799021+0000","last_active":"2024-03-08T20:57:51.794667+0000","last_peered":"2024-03-08T20:57:51.794667+0000","last_clean":"2024-03-08T20:57:51.794667+0000","last_became_active":"2024-03-08T20:57:50.795494+0000","last_became_peered":"2024-03-08T20:57:50.795494+0000","last_unstale":"2024-03-08T20:57:51.794667+0000","last_undegraded":"2024-03-08T20:57:51.794667+0000","last_fullsized":"2024-03-08T20:57:51.794667+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T21:26:40.936789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"86'5","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-08T20:57:53.846241+0000","last_change":"2024-03-08T20:57:49.791167+0000","last_active":"2024-03-08T20:57:53.846241+0000","last_peered":"2024-03-08T20:57:53.846241+0000","last_clean":"2024-03-08T20:57:53.846241+0000","last_became_active":"2024-03-08T20:57:49.789840+0000","last_became_peered":"2024-03-08T20:57:49.789840+0000","last_unstale":"2024-03-08T20:57:53.846241+0000","last_undegraded":"2024-03-08T20:57:53.846241+0000","last_fullsized":"2024-03-08T20:57:53.846241+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_clean_scrub_stamp":"2024-03-08T20:57:48.739829+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T23:14:51.103583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-08T20:57:52.073889+0000","last_change":"2024-03-08T20:57:50.812656+0000","last_active":"2024-03-08T20:57:52.073889+0000","last_peered":"2024-03-08T20:57:52.073889+0000","last_clean":"2024-03-08T20:57:52.073889+0000","last_became_active":"2024-03-08T20:57:50.812281+0000","last_became_peered":"2024-03-08T20:57:50.812281+0000","last_unstale":"2024-03-08T20:57:52.073889+0000","last_undegraded":"2024-03-08T20:57:52.073889+0000","last_fullsized":"2024-03-08T20:57:52.073889+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T05:40:20.985798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"86'7","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-08T20:57:55.862604+0000","last_change":"2024-03-08T20:57:49.792874+0000","last_active":"2024-03-08T20:57:55.862604+0000","last_peered":"2024-03-08T20:57:55.862604+0000","last_clean":"2024-03-08T20:57:55.862604+0000","last_became_active":"2024-03-08T20:57:49.792418+0000","last_became_peered":"2024-03-08T20:57:49.792418+0000","last_unstale":"2024-03-08T20:57:55.862604+0000","last_undegraded":"2024-03-08T20:57:55.862604+0000","last_fullsized":"2024-03-08T20:57:55.862604+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_clean_scrub_stamp":"2024-03-08T20:57:48.739829+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T02:23:12.888435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-08T20:57:51.794237+0000","last_change":"2024-03-08T20:57:50.805567+0000","last_active":"2024-03-08T20:57:51.794237+0000","last_peered":"2024-03-08T20:57:51.794237+0000","last_clean":"2024-03-08T20:57:51.794237+0000","last_became_active":"2024-03-08T20:57:50.805120+0000","last_became_peered":"2024-03-08T20:57:50.805120+0000","last_unstale":"2024-03-08T20:57:51.794237+0000","last_undegraded":"2024-03-08T20:57:51.794237+0000","last_fullsized":"2024-03-08T20:57:51.794237+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T06:18:14.297971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"86'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-08T20:57:54.859110+0000","last_change":"2024-03-08T20:57:49.771806+0000","last_active":"2024-03-08T20:57:54.859110+0000","last_peered":"2024-03-08T20:57:54.859110+0000","last_clean":"2024-03-08T20:57:54.859110+0000","last_became_active":"2024-03-08T20:57:49.771447+0000","last_became_peered":"2024-03-08T20:57:49.771447+0000","last_unstale":"2024-03-08T20:57:54.859110+0000","last_undegraded":"2024-03-08T20:57:54.859110+0000","last_fullsized":"2024-03-08T20:57:54.859110+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_clean_scrub_stamp":"2024-03-08T20:57:48.739829+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T03:16:37.913599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-08T20:57:51.791871+0000","last_change":"2024-03-08T20:57:50.792915+0000","last_active":"2024-03-08T20:57:51.791871+0000","last_peered":"2024-03-08T20:57:51.791871+0000","last_clean":"2024-03-08T20:57:51.791871+0000","last_became_active":"2024-03-08T20:57:50.792617+0000","last_became_peered":"2024-03-08T20:57:50.792617+0000","last_unstale":"2024-03-08T20:57:51.791871+0000","last_undegraded":"2024-03-08T20:57:51.791871+0000","last_fullsized":"2024-03-08T20:57:51.791871+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T01:59:30.427751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"86'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-08T20:57:51.793443+0000","last_change":"2024-03-08T20:57:49.789383+0000","last_active":"2024-03-08T20:57:51.793443+0000","last_peered":"2024-03-08T20:57:51.793443+0000","last_clean":"2024-03-08T20:57:51.793443+0000","last_became_active":"2024-03-08T20:57:49.788975+0000","last_became_peered":"2024-03-08T20:57:49.788975+0000","last_unstale":"2024-03-08T20:57:51.793443+0000","last_undegraded":"2024-03-08T20:57:51.793443+0000","last_fullsized":"2024-03-08T20:57:51.793443+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_clean_scrub_stamp":"2024-03-08T20:57:48.739829+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T07:42:21.366476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-08T20:57:52.073843+0000","last_change":"2024-03-08T20:57:50.785377+0000","last_active":"2024-03-08T20:57:52.073843+0000","last_peered":"2024-03-08T20:57:52.073843+0000","last_clean":"2024-03-08T20:57:52.073843+0000","last_became_active":"2024-03-08T20:57:50.785067+0000","last_became_peered":"2024-03-08T20:57:50.785067+0000","last_unstale":"2024-03-08T20:57:52.073843+0000","last_undegraded":"2024-03-08T20:57:52.073843+0000","last_fullsized":"2024-03-08T20:57:52.073843+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T02:49:10.388474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"86'4","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-08T20:57:53.842906+0000","last_change":"2024-03-08T20:57:49.784190+0000","last_active":"2024-03-08T20:57:53.842906+0000","last_peered":"2024-03-08T20:57:53.842906+0000","last_clean":"2024-03-08T20:57:53.842906+0000","last_became_active":"2024-03-08T20:57:49.783805+0000","last_became_peered":"2024-03-08T20:57:49.783805+0000","last_unstale":"2024-03-08T20:57:53.842906+0000","last_undegraded":"2024-03-08T20:57:53.842906+0000","last_fullsized":"2024-03-08T20:57:53.842906+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_clean_scrub_stamp":"2024-03-08T20:57:48.739829+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T03:16:12.983559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-08T20:57:51.794207+0000","last_change":"2024-03-08T20:57:50.795647+0000","last_active":"2024-03-08T20:57:51.794207+0000","last_peered":"2024-03-08T20:57:51.794207+0000","last_clean":"2024-03-08T20:57:51.794207+0000","last_became_active":"2024-03-08T20:57:50.793179+0000","last_became_peered":"2024-03-08T20:57:50.793179+0000","last_unstale":"2024-03-08T20:57:51.794207+0000","last_undegraded":"2024-03-08T20:57:51.794207+0000","last_fullsized":"2024-03-08T20:57:51.794207+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T05:56:14.051601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"86'4","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-08T20:57:56.894378+0000","last_change":"2024-03-08T20:57:49.795093+0000","last_active":"2024-03-08T20:57:56.894378+0000","last_peered":"2024-03-08T20:57:56.894378+0000","last_clean":"2024-03-08T20:57:56.894378+0000","last_became_active":"2024-03-08T20:57:49.794617+0000","last_became_peered":"2024-03-08T20:57:49.794617+0000","last_unstale":"2024-03-08T20:57:56.894378+0000","last_undegraded":"2024-03-08T20:57:56.894378+0000","last_fullsized":"2024-03-08T20:57:56.894378+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_clean_scrub_stamp":"2024-03-08T20:57:48.739829+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T05:56:38.669695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-08T20:57:52.073978+0000","last_change":"2024-03-08T20:57:50.806013+0000","last_active":"2024-03-08T20:57:52.073978+0000","last_peered":"2024-03-08T20:57:52.073978+0000","last_clean":"2024-03-08T20:57:52.073978+0000","last_became_active":"2024-03-08T20:57:50.805702+0000","last_became_peered":"2024-03-08T20:57:50.805702+0000","last_unstale":"2024-03-08T20:57:52.073978+0000","last_undegraded":"2024-03-08T20:57:52.073978+0000","last_fullsized":"2024-03-08T20:57:52.073978+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T03:27:37.118410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"86'4","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-08T20:57:51.794867+0000","last_change":"2024-03-08T20:57:49.776991+0000","last_active":"2024-03-08T20:57:51.794867+0000","last_peered":"2024-03-08T20:57:51.794867+0000","last_clean":"2024-03-08T20:57:51.794867+0000","last_became_active":"2024-03-08T20:57:49.776566+0000","last_became_peered":"2024-03-08T20:57:49.776566+0000","last_unstale":"2024-03-08T20:57:51.794867+0000","last_undegraded":"2024-03-08T20:57:51.794867+0000","last_fullsized":"2024-03-08T20:57:51.794867+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_clean_scrub_stamp":"2024-03-08T20:57:48.739829+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T21:02:37.726353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-08T20:57:51.794371+0000","last_change":"2024-03-08T20:57:50.793976+0000","last_active":"2024-03-08T20:57:51.794371+0000","last_peered":"2024-03-08T20:57:51.794371+0000","last_clean":"2024-03-08T20:57:51.794371+0000","last_became_active":"2024-03-08T20:57:50.792472+0000","last_became_peered":"2024-03-08T20:57:50.792472+0000","last_unstale":"2024-03-08T20:57:51.794371+0000","last_undegraded":"2024-03-08T20:57:51.794371+0000","last_fullsized":"2024-03-08T20:57:51.794371+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T06:01:03.360171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"86'17","reported_seq":32,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-08T20:57:57.417728+0000","last_change":"2024-03-08T20:57:49.791290+0000","last_active":"2024-03-08T20:57:57.417728+0000","last_peered":"2024-03-08T20:57:57.417728+0000","last_clean":"2024-03-08T20:57:57.417728+0000","last_became_active":"2024-03-08T20:57:49.790747+0000","last_became_peered":"2024-03-08T20:57:49.790747+0000","last_unstale":"2024-03-08T20:57:57.417728+0000","last_undegraded":"2024-03-08T20:57:57.417728+0000","last_fullsized":"2024-03-08T20:57:57.417728+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_clean_scrub_stamp":"2024-03-08T20:57:48.739829+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T02:49:12.984238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20895,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19,"num_write_kb":24,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-08T20:57:52.073954+0000","last_change":"2024-03-08T20:57:50.800031+0000","last_active":"2024-03-08T20:57:52.073954+0000","last_peered":"2024-03-08T20:57:52.073954+0000","last_clean":"2024-03-08T20:57:52.073954+0000","last_became_active":"2024-03-08T20:57:50.799663+0000","last_became_peered":"2024-03-08T20:57:50.799663+0000","last_unstale":"2024-03-08T20:57:52.073954+0000","last_undegraded":"2024-03-08T20:57:52.073954+0000","last_fullsized":"2024-03-08T20:57:52.073954+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T05:37:50.734330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"86'9","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-08T20:57:56.889292+0000","last_change":"2024-03-08T20:57:49.791884+0000","last_active":"2024-03-08T20:57:56.889292+0000","last_peered":"2024-03-08T20:57:56.889292+0000","last_clean":"2024-03-08T20:57:56.889292+0000","last_became_active":"2024-03-08T20:57:49.789314+0000","last_became_peered":"2024-03-08T20:57:49.789314+0000","last_unstale":"2024-03-08T20:57:56.889292+0000","last_undegraded":"2024-03-08T20:57:56.889292+0000","last_fullsized":"2024-03-08T20:57:56.889292+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_clean_scrub_stamp":"2024-03-08T20:57:48.739829+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T08:44:20.490703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-08T20:57:52.074277+0000","last_change":"2024-03-08T20:57:50.787609+0000","last_active":"2024-03-08T20:57:52.074277+0000","last_peered":"2024-03-08T20:57:52.074277+0000","last_clean":"2024-03-08T20:57:52.074277+0000","last_became_active":"2024-03-08T20:57:50.787171+0000","last_became_peered":"2024-03-08T20:57:50.787171+0000","last_unstale":"2024-03-08T20:57:52.074277+0000","last_undegraded":"2024-03-08T20:57:52.074277+0000","last_fullsized":"2024-03-08T20:57:52.074277+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T05:57:54.023942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"86'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-08T20:57:53.843588+0000","last_change":"2024-03-08T20:57:49.785642+0000","last_active":"2024-03-08T20:57:53.843588+0000","last_peered":"2024-03-08T20:57:53.843588+0000","last_clean":"2024-03-08T20:57:53.843588+0000","last_became_active":"2024-03-08T20:57:49.785180+0000","last_became_peered":"2024-03-08T20:57:49.785180+0000","last_unstale":"2024-03-08T20:57:53.843588+0000","last_undegraded":"2024-03-08T20:57:53.843588+0000","last_fullsized":"2024-03-08T20:57:53.843588+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_clean_scrub_stamp":"2024-03-08T20:57:48.739829+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T03:21:03.121253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-08T20:57:52.073757+0000","last_change":"2024-03-08T20:57:50.805179+0000","last_active":"2024-03-08T20:57:52.073757+0000","last_peered":"2024-03-08T20:57:52.073757+0000","last_clean":"2024-03-08T20:57:52.073757+0000","last_became_active":"2024-03-08T20:57:50.804801+0000","last_became_peered":"2024-03-08T20:57:50.804801+0000","last_unstale":"2024-03-08T20:57:52.073757+0000","last_undegraded":"2024-03-08T20:57:52.073757+0000","last_fullsized":"2024-03-08T20:57:52.073757+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T08:35:50.224508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"86'5","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-08T20:57:55.856902+0000","last_change":"2024-03-08T20:57:49.787198+0000","last_active":"2024-03-08T20:57:55.856902+0000","last_peered":"2024-03-08T20:57:55.856902+0000","last_clean":"2024-03-08T20:57:55.856902+0000","last_became_active":"2024-03-08T20:57:49.785966+0000","last_became_peered":"2024-03-08T20:57:49.785966+0000","last_unstale":"2024-03-08T20:57:55.856902+0000","last_undegraded":"2024-03-08T20:57:55.856902+0000","last_fullsized":"2024-03-08T20:57:55.856902+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_clean_scrub_stamp":"2024-03-08T20:57:48.739829+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T21:39:03.900857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-08T20:57:52.073164+0000","last_change":"2024-03-08T20:57:50.802887+0000","last_active":"2024-03-08T20:57:52.073164+0000","last_peered":"2024-03-08T20:57:52.073164+0000","last_clean":"2024-03-08T20:57:52.073164+0000","last_became_active":"2024-03-08T20:57:50.802530+0000","last_became_peered":"2024-03-08T20:57:50.802530+0000","last_unstale":"2024-03-08T20:57:52.073164+0000","last_undegraded":"2024-03-08T20:57:52.073164+0000","last_fullsized":"2024-03-08T20:57:52.073164+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T07:08:25.980812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"86'17","reported_seq":33,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-08T20:57:56.888941+0000","last_change":"2024-03-08T20:57:49.793528+0000","last_active":"2024-03-08T20:57:56.888941+0000","last_peered":"2024-03-08T20:57:56.888941+0000","last_clean":"2024-03-08T20:57:56.888941+0000","last_became_active":"2024-03-08T20:57:49.792509+0000","last_became_peered":"2024-03-08T20:57:49.792509+0000","last_unstale":"2024-03-08T20:57:56.888941+0000","last_undegraded":"2024-03-08T20:57:56.888941+0000","last_fullsized":"2024-03-08T20:57:56.888941+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_clean_scrub_stamp":"2024-03-08T20:57:48.739829+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T03:45:20.912280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-08T20:57:51.793333+0000","last_change":"2024-03-08T20:57:50.796843+0000","last_active":"2024-03-08T20:57:51.793333+0000","last_peered":"2024-03-08T20:57:51.793333+0000","last_clean":"2024-03-08T20:57:51.793333+0000","last_became_active":"2024-03-08T20:57:50.796232+0000","last_became_peered":"2024-03-08T20:57:50.796232+0000","last_unstale":"2024-03-08T20:57:51.793333+0000","last_undegraded":"2024-03-08T20:57:51.793333+0000","last_fullsized":"2024-03-08T20:57:51.793333+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T06:12:38.334736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"86'2","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-08T20:57:55.848274+0000","last_change":"2024-03-08T20:57:49.789376+0000","last_active":"2024-03-08T20:57:55.848274+0000","last_peered":"2024-03-08T20:57:55.848274+0000","last_clean":"2024-03-08T20:57:55.848274+0000","last_became_active":"2024-03-08T20:57:49.785791+0000","last_became_peered":"2024-03-08T20:57:49.785791+0000","last_unstale":"2024-03-08T20:57:55.848274+0000","last_undegraded":"2024-03-08T20:57:55.848274+0000","last_fullsized":"2024-03-08T20:57:55.848274+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_clean_scrub_stamp":"2024-03-08T20:57:48.739829+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T08:52:02.675985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-08T20:57:51.805054+0000","last_change":"2024-03-08T20:57:50.814625+0000","last_active":"2024-03-08T20:57:51.805054+0000","last_peered":"2024-03-08T20:57:51.805054+0000","last_clean":"2024-03-08T20:57:51.805054+0000","last_became_active":"2024-03-08T20:57:50.814220+0000","last_became_peered":"2024-03-08T20:57:50.814220+0000","last_unstale":"2024-03-08T20:57:51.805054+0000","last_undegraded":"2024-03-08T20:57:51.805054+0000","last_fullsized":"2024-03-08T20:57:51.805054+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T08:35:57.659511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"86'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-08T20:57:52.074373+0000","last_change":"2024-03-08T20:57:49.777922+0000","last_active":"2024-03-08T20:57:52.074373+0000","last_peered":"2024-03-08T20:57:52.074373+0000","last_clean":"2024-03-08T20:57:52.074373+0000","last_became_active":"2024-03-08T20:57:49.775083+0000","last_became_peered":"2024-03-08T20:57:49.775083+0000","last_unstale":"2024-03-08T20:57:52.074373+0000","last_undegraded":"2024-03-08T20:57:52.074373+0000","last_fullsized":"2024-03-08T20:57:52.074373+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_clean_scrub_stamp":"2024-03-08T20:57:48.739829+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T01:55:02.679999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"86'1","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-08T20:57:52.686908+0000","last_change":"2024-03-08T20:57:50.778774+0000","last_active":"2024-03-08T20:57:52.686908+0000","last_peered":"2024-03-08T20:57:52.686908+0000","last_clean":"2024-03-08T20:57:52.686908+0000","last_became_active":"2024-03-08T20:57:50.778194+0000","last_became_peered":"2024-03-08T20:57:50.778194+0000","last_unstale":"2024-03-08T20:57:52.686908+0000","last_undegraded":"2024-03-08T20:57:52.686908+0000","last_fullsized":"2024-03-08T20:57:52.686908+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T21:41:56.321304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"86'5","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-08T20:57:57.895945+0000","last_change":"2024-03-08T20:57:49.787818+0000","last_active":"2024-03-08T20:57:57.895945+0000","last_peered":"2024-03-08T20:57:57.895945+0000","last_clean":"2024-03-08T20:57:57.895945+0000","last_became_active":"2024-03-08T20:57:49.787401+0000","last_became_peered":"2024-03-08T20:57:49.787401+0000","last_unstale":"2024-03-08T20:57:57.895945+0000","last_undegraded":"2024-03-08T20:57:57.895945+0000","last_fullsized":"2024-03-08T20:57:57.895945+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_clean_scrub_stamp":"2024-03-08T20:57:48.739829+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T08:23:19.683962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-08T20:57:52.075297+0000","last_change":"2024-03-08T20:57:50.790922+0000","last_active":"2024-03-08T20:57:52.075297+0000","last_peered":"2024-03-08T20:57:52.075297+0000","last_clean":"2024-03-08T20:57:52.075297+0000","last_became_active":"2024-03-08T20:57:50.789993+0000","last_became_peered":"2024-03-08T20:57:50.789993+0000","last_unstale":"2024-03-08T20:57:52.075297+0000","last_undegraded":"2024-03-08T20:57:52.075297+0000","last_fullsized":"2024-03-08T20:57:52.075297+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T00:27:01.639941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-08T20:57:51.793549+0000","last_change":"2024-03-08T20:57:50.800505+0000","last_active":"2024-03-08T20:57:51.793549+0000","last_peered":"2024-03-08T20:57:51.793549+0000","last_clean":"2024-03-08T20:57:51.793549+0000","last_became_active":"2024-03-08T20:57:50.797459+0000","last_became_peered":"2024-03-08T20:57:50.797459+0000","last_unstale":"2024-03-08T20:57:51.793549+0000","last_undegraded":"2024-03-08T20:57:51.793549+0000","last_fullsized":"2024-03-08T20:57:51.793549+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T21:24:10.734888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"86'4","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-08T20:57:54.860484+0000","last_change":"2024-03-08T20:57:49.779056+0000","last_active":"2024-03-08T20:57:54.860484+0000","last_peered":"2024-03-08T20:57:54.860484+0000","last_clean":"2024-03-08T20:57:54.860484+0000","last_became_active":"2024-03-08T20:57:49.778714+0000","last_became_peered":"2024-03-08T20:57:49.778714+0000","last_unstale":"2024-03-08T20:57:54.860484+0000","last_undegraded":"2024-03-08T20:57:54.860484+0000","last_fullsized":"2024-03-08T20:57:54.860484+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_clean_scrub_stamp":"2024-03-08T20:57:48.739829+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T04:05:10.685148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-08T20:57:51.793317+0000","last_change":"2024-03-08T20:57:50.812509+0000","last_active":"2024-03-08T20:57:51.793317+0000","last_peered":"2024-03-08T20:57:51.793317+0000","last_clean":"2024-03-08T20:57:51.793317+0000","last_became_active":"2024-03-08T20:57:50.812146+0000","last_became_peered":"2024-03-08T20:57:50.812146+0000","last_unstale":"2024-03-08T20:57:51.793317+0000","last_undegraded":"2024-03-08T20:57:51.793317+0000","last_fullsized":"2024-03-08T20:57:51.793317+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T06:08:07.680098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'5","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-08T20:57:55.857423+0000","last_change":"2024-03-08T20:57:49.774994+0000","last_active":"2024-03-08T20:57:55.857423+0000","last_peered":"2024-03-08T20:57:55.857423+0000","last_clean":"2024-03-08T20:57:55.857423+0000","last_became_active":"2024-03-08T20:57:49.774501+0000","last_became_peered":"2024-03-08T20:57:49.774501+0000","last_unstale":"2024-03-08T20:57:55.857423+0000","last_undegraded":"2024-03-08T20:57:55.857423+0000","last_fullsized":"2024-03-08T20:57:55.857423+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_clean_scrub_stamp":"2024-03-08T20:57:48.739829+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T03:38:35.294082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":347,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-08T20:57:52.073201+0000","last_change":"2024-03-08T20:56:11.724397+0000","last_active":"2024-03-08T20:57:52.073201+0000","last_peered":"2024-03-08T20:57:52.073201+0000","last_clean":"2024-03-08T20:57:52.073201+0000","last_became_active":"2024-03-08T20:56:11.723212+0000","last_became_peered":"2024-03-08T20:56:11.723212+0000","last_unstale":"2024-03-08T20:57:52.073201+0000","last_undegraded":"2024-03-08T20:57:52.073201+0000","last_fullsized":"2024-03-08T20:57:52.073201+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:51:32.833935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:51:32.833935+0000","last_clean_scrub_stamp":"2024-03-08T20:51:32.833935+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T03:31:48.964514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-08T20:57:52.072747+0000","last_change":"2024-03-08T20:57:50.792894+0000","last_active":"2024-03-08T20:57:52.072747+0000","last_peered":"2024-03-08T20:57:52.072747+0000","last_clean":"2024-03-08T20:57:52.072747+0000","last_became_active":"2024-03-08T20:57:50.792471+0000","last_became_peered":"2024-03-08T20:57:50.792471+0000","last_unstale":"2024-03-08T20:57:52.072747+0000","last_undegraded":"2024-03-08T20:57:52.072747+0000","last_fullsized":"2024-03-08T20:57:52.072747+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T00:45:39.696601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'4","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-08T20:57:54.855670+0000","last_change":"2024-03-08T20:57:49.787880+0000","last_active":"2024-03-08T20:57:54.855670+0000","last_peered":"2024-03-08T20:57:54.855670+0000","last_clean":"2024-03-08T20:57:54.855670+0000","last_became_active":"2024-03-08T20:57:49.787450+0000","last_became_peered":"2024-03-08T20:57:49.787450+0000","last_unstale":"2024-03-08T20:57:54.855670+0000","last_undegraded":"2024-03-08T20:57:54.855670+0000","last_fullsized":"2024-03-08T20:57:54.855670+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_clean_scrub_stamp":"2024-03-08T20:57:48.739829+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T04:52:36.895259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"86'4","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-08T20:57:56.896666+0000","last_change":"2024-03-08T20:57:49.787473+0000","last_active":"2024-03-08T20:57:56.896666+0000","last_peered":"2024-03-08T20:57:56.896666+0000","last_clean":"2024-03-08T20:57:56.896666+0000","last_became_active":"2024-03-08T20:57:49.787145+0000","last_became_peered":"2024-03-08T20:57:49.787145+0000","last_unstale":"2024-03-08T20:57:56.896666+0000","last_undegraded":"2024-03-08T20:57:56.896666+0000","last_fullsized":"2024-03-08T20:57:56.896666+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_clean_scrub_stamp":"2024-03-08T20:57:48.739829+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T05:47:14.471850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-08T20:57:52.073056+0000","last_change":"2024-03-08T20:57:50.789546+0000","last_active":"2024-03-08T20:57:52.073056+0000","last_peered":"2024-03-08T20:57:52.073056+0000","last_clean":"2024-03-08T20:57:52.073056+0000","last_became_active":"2024-03-08T20:57:50.789195+0000","last_became_peered":"2024-03-08T20:57:50.789195+0000","last_unstale":"2024-03-08T20:57:52.073056+0000","last_undegraded":"2024-03-08T20:57:52.073056+0000","last_fullsized":"2024-03-08T20:57:52.073056+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T02:43:31.371303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"86'10","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-08T20:57:58.774929+0000","last_change":"2024-03-08T20:57:49.787907+0000","last_active":"2024-03-08T20:57:58.774929+0000","last_peered":"2024-03-08T20:57:58.774929+0000","last_clean":"2024-03-08T20:57:58.774929+0000","last_became_active":"2024-03-08T20:57:49.787601+0000","last_became_peered":"2024-03-08T20:57:49.787601+0000","last_unstale":"2024-03-08T20:57:58.774929+0000","last_undegraded":"2024-03-08T20:57:58.774929+0000","last_fullsized":"2024-03-08T20:57:58.774929+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_clean_scrub_stamp":"2024-03-08T20:57:48.739829+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T00:52:09.562020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-08T20:57:51.793583+0000","last_change":"2024-03-08T20:57:50.798773+0000","last_active":"2024-03-08T20:57:51.793583+0000","last_peered":"2024-03-08T20:57:51.793583+0000","last_clean":"2024-03-08T20:57:51.793583+0000","last_became_active":"2024-03-08T20:57:50.798396+0000","last_became_peered":"2024-03-08T20:57:50.798396+0000","last_unstale":"2024-03-08T20:57:51.793583+0000","last_undegraded":"2024-03-08T20:57:51.793583+0000","last_fullsized":"2024-03-08T20:57:51.793583+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T00:26:41.614622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"86'5","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-08T20:57:53.832726+0000","last_change":"2024-03-08T20:57:49.777474+0000","last_active":"2024-03-08T20:57:53.832726+0000","last_peered":"2024-03-08T20:57:53.832726+0000","last_clean":"2024-03-08T20:57:53.832726+0000","last_became_active":"2024-03-08T20:57:49.777084+0000","last_became_peered":"2024-03-08T20:57:49.777084+0000","last_unstale":"2024-03-08T20:57:53.832726+0000","last_undegraded":"2024-03-08T20:57:53.832726+0000","last_fullsized":"2024-03-08T20:57:53.832726+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_clean_scrub_stamp":"2024-03-08T20:57:48.739829+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T05:10:23.032788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-08T20:57:52.073940+0000","last_change":"2024-03-08T20:57:50.805061+0000","last_active":"2024-03-08T20:57:52.073940+0000","last_peered":"2024-03-08T20:57:52.073940+0000","last_clean":"2024-03-08T20:57:52.073940+0000","last_became_active":"2024-03-08T20:57:50.804709+0000","last_became_peered":"2024-03-08T20:57:50.804709+0000","last_unstale":"2024-03-08T20:57:52.073940+0000","last_undegraded":"2024-03-08T20:57:52.073940+0000","last_fullsized":"2024-03-08T20:57:52.073940+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T08:25:34.003449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-08T20:57:53.841513+0000","last_change":"2024-03-08T20:57:49.776677+0000","last_active":"2024-03-08T20:57:53.841513+0000","last_peered":"2024-03-08T20:57:53.841513+0000","last_clean":"2024-03-08T20:57:53.841513+0000","last_became_active":"2024-03-08T20:57:49.776296+0000","last_became_peered":"2024-03-08T20:57:49.776296+0000","last_unstale":"2024-03-08T20:57:53.841513+0000","last_undegraded":"2024-03-08T20:57:53.841513+0000","last_fullsized":"2024-03-08T20:57:53.841513+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_clean_scrub_stamp":"2024-03-08T20:57:48.739829+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T07:49:46.042037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-08T20:57:52.073666+0000","last_change":"2024-03-08T20:57:50.806141+0000","last_active":"2024-03-08T20:57:52.073666+0000","last_peered":"2024-03-08T20:57:52.073666+0000","last_clean":"2024-03-08T20:57:52.073666+0000","last_became_active":"2024-03-08T20:57:50.805026+0000","last_became_peered":"2024-03-08T20:57:50.805026+0000","last_unstale":"2024-03-08T20:57:52.073666+0000","last_undegraded":"2024-03-08T20:57:52.073666+0000","last_fullsized":"2024-03-08T20:57:52.073666+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T03:49:04.162888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"86'7","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-08T20:57:56.896343+0000","last_change":"2024-03-08T20:57:49.779675+0000","last_active":"2024-03-08T20:57:56.896343+0000","last_peered":"2024-03-08T20:57:56.896343+0000","last_clean":"2024-03-08T20:57:56.896343+0000","last_became_active":"2024-03-08T20:57:49.774270+0000","last_became_peered":"2024-03-08T20:57:49.774270+0000","last_unstale":"2024-03-08T20:57:56.896343+0000","last_undegraded":"2024-03-08T20:57:56.896343+0000","last_fullsized":"2024-03-08T20:57:56.896343+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_clean_scrub_stamp":"2024-03-08T20:57:48.739829+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T02:19:09.992649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-08T20:57:51.804720+0000","last_change":"2024-03-08T20:57:50.797461+0000","last_active":"2024-03-08T20:57:51.804720+0000","last_peered":"2024-03-08T20:57:51.804720+0000","last_clean":"2024-03-08T20:57:51.804720+0000","last_became_active":"2024-03-08T20:57:50.796928+0000","last_became_peered":"2024-03-08T20:57:50.796928+0000","last_unstale":"2024-03-08T20:57:51.804720+0000","last_undegraded":"2024-03-08T20:57:51.804720+0000","last_fullsized":"2024-03-08T20:57:51.804720+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T04:01:44.009461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"86'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-08T20:57:54.830216+0000","last_change":"2024-03-08T20:57:49.794659+0000","last_active":"2024-03-08T20:57:54.830216+0000","last_peered":"2024-03-08T20:57:54.830216+0000","last_clean":"2024-03-08T20:57:54.830216+0000","last_became_active":"2024-03-08T20:57:49.792426+0000","last_became_peered":"2024-03-08T20:57:49.792426+0000","last_unstale":"2024-03-08T20:57:54.830216+0000","last_undegraded":"2024-03-08T20:57:54.830216+0000","last_fullsized":"2024-03-08T20:57:54.830216+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_clean_scrub_stamp":"2024-03-08T20:57:48.739829+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T01:30:41.123131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-08T20:57:51.792840+0000","last_change":"2024-03-08T20:57:50.798691+0000","last_active":"2024-03-08T20:57:51.792840+0000","last_peered":"2024-03-08T20:57:51.792840+0000","last_clean":"2024-03-08T20:57:51.792840+0000","last_became_active":"2024-03-08T20:57:50.797876+0000","last_became_peered":"2024-03-08T20:57:50.797876+0000","last_unstale":"2024-03-08T20:57:51.792840+0000","last_undegraded":"2024-03-08T20:57:51.792840+0000","last_fullsized":"2024-03-08T20:57:51.792840+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T06:51:11.067375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-08T20:57:53.841990+0000","last_change":"2024-03-08T20:57:49.794648+0000","last_active":"2024-03-08T20:57:53.841990+0000","last_peered":"2024-03-08T20:57:53.841990+0000","last_clean":"2024-03-08T20:57:53.841990+0000","last_became_active":"2024-03-08T20:57:49.793316+0000","last_became_peered":"2024-03-08T20:57:49.793316+0000","last_unstale":"2024-03-08T20:57:53.841990+0000","last_undegraded":"2024-03-08T20:57:53.841990+0000","last_fullsized":"2024-03-08T20:57:53.841990+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_clean_scrub_stamp":"2024-03-08T20:57:48.739829+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T02:16:26.006212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-08T20:57:52.073761+0000","last_change":"2024-03-08T20:57:50.797850+0000","last_active":"2024-03-08T20:57:52.073761+0000","last_peered":"2024-03-08T20:57:52.073761+0000","last_clean":"2024-03-08T20:57:52.073761+0000","last_became_active":"2024-03-08T20:57:50.797530+0000","last_became_peered":"2024-03-08T20:57:50.797530+0000","last_unstale":"2024-03-08T20:57:52.073761+0000","last_undegraded":"2024-03-08T20:57:52.073761+0000","last_fullsized":"2024-03-08T20:57:52.073761+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T02:12:52.023644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"86'5","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-08T20:57:55.861183+0000","last_change":"2024-03-08T20:57:49.784671+0000","last_active":"2024-03-08T20:57:55.861183+0000","last_peered":"2024-03-08T20:57:55.861183+0000","last_clean":"2024-03-08T20:57:55.861183+0000","last_became_active":"2024-03-08T20:57:49.783870+0000","last_became_peered":"2024-03-08T20:57:49.783870+0000","last_unstale":"2024-03-08T20:57:55.861183+0000","last_undegraded":"2024-03-08T20:57:55.861183+0000","last_fullsized":"2024-03-08T20:57:55.861183+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_clean_scrub_stamp":"2024-03-08T20:57:48.739829+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T02:29:16.769221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-08T20:57:52.075006+0000","last_change":"2024-03-08T20:57:50.796522+0000","last_active":"2024-03-08T20:57:52.075006+0000","last_peered":"2024-03-08T20:57:52.075006+0000","last_clean":"2024-03-08T20:57:52.075006+0000","last_became_active":"2024-03-08T20:57:50.796061+0000","last_became_peered":"2024-03-08T20:57:50.796061+0000","last_unstale":"2024-03-08T20:57:52.075006+0000","last_undegraded":"2024-03-08T20:57:52.075006+0000","last_fullsized":"2024-03-08T20:57:52.075006+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T21:35:06.609042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-08T20:57:51.794556+0000","last_change":"2024-03-08T20:57:49.778240+0000","last_active":"2024-03-08T20:57:51.794556+0000","last_peered":"2024-03-08T20:57:51.794556+0000","last_clean":"2024-03-08T20:57:51.794556+0000","last_became_active":"2024-03-08T20:57:49.777798+0000","last_became_peered":"2024-03-08T20:57:49.777798+0000","last_unstale":"2024-03-08T20:57:51.794556+0000","last_undegraded":"2024-03-08T20:57:51.794556+0000","last_fullsized":"2024-03-08T20:57:51.794556+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_clean_scrub_stamp":"2024-03-08T20:57:48.739829+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T05:20:24.614659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"86'1","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-08T20:57:53.972947+0000","last_change":"2024-03-08T20:57:50.803435+0000","last_active":"2024-03-08T20:57:53.972947+0000","last_peered":"2024-03-08T20:57:53.972947+0000","last_clean":"2024-03-08T20:57:53.972947+0000","last_became_active":"2024-03-08T20:57:50.803074+0000","last_became_peered":"2024-03-08T20:57:50.803074+0000","last_unstale":"2024-03-08T20:57:53.972947+0000","last_undegraded":"2024-03-08T20:57:53.972947+0000","last_fullsized":"2024-03-08T20:57:53.972947+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T21:52:02.791159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-08T20:57:52.074495+0000","last_change":"2024-03-08T20:57:49.781604+0000","last_active":"2024-03-08T20:57:52.074495+0000","last_peered":"2024-03-08T20:57:52.074495+0000","last_clean":"2024-03-08T20:57:52.074495+0000","last_became_active":"2024-03-08T20:57:49.781274+0000","last_became_peered":"2024-03-08T20:57:49.781274+0000","last_unstale":"2024-03-08T20:57:52.074495+0000","last_undegraded":"2024-03-08T20:57:52.074495+0000","last_fullsized":"2024-03-08T20:57:52.074495+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_clean_scrub_stamp":"2024-03-08T20:57:48.739829+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T22:29:03.420779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-08T20:57:52.074143+0000","last_change":"2024-03-08T20:57:50.800863+0000","last_active":"2024-03-08T20:57:52.074143+0000","last_peered":"2024-03-08T20:57:52.074143+0000","last_clean":"2024-03-08T20:57:52.074143+0000","last_became_active":"2024-03-08T20:57:50.798916+0000","last_became_peered":"2024-03-08T20:57:50.798916+0000","last_unstale":"2024-03-08T20:57:52.074143+0000","last_undegraded":"2024-03-08T20:57:52.074143+0000","last_fullsized":"2024-03-08T20:57:52.074143+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T04:37:06.666245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-08T20:57:51.804884+0000","last_change":"2024-03-08T20:57:50.813502+0000","last_active":"2024-03-08T20:57:51.804884+0000","last_peered":"2024-03-08T20:57:51.804884+0000","last_clean":"2024-03-08T20:57:51.804884+0000","last_became_active":"2024-03-08T20:57:50.813150+0000","last_became_peered":"2024-03-08T20:57:50.813150+0000","last_unstale":"2024-03-08T20:57:51.804884+0000","last_undegraded":"2024-03-08T20:57:51.804884+0000","last_fullsized":"2024-03-08T20:57:51.804884+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T00:01:47.902909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-08T20:57:52.072901+0000","last_change":"2024-03-08T20:57:50.795055+0000","last_active":"2024-03-08T20:57:52.072901+0000","last_peered":"2024-03-08T20:57:52.072901+0000","last_clean":"2024-03-08T20:57:52.072901+0000","last_became_active":"2024-03-08T20:57:50.794559+0000","last_became_peered":"2024-03-08T20:57:50.794559+0000","last_unstale":"2024-03-08T20:57:52.072901+0000","last_undegraded":"2024-03-08T20:57:52.072901+0000","last_fullsized":"2024-03-08T20:57:52.072901+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T08:06:18.990542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-08T20:57:52.072961+0000","last_change":"2024-03-08T20:57:50.784871+0000","last_active":"2024-03-08T20:57:52.072961+0000","last_peered":"2024-03-08T20:57:52.072961+0000","last_clean":"2024-03-08T20:57:52.072961+0000","last_became_active":"2024-03-08T20:57:50.784553+0000","last_became_peered":"2024-03-08T20:57:50.784553+0000","last_unstale":"2024-03-08T20:57:52.072961+0000","last_undegraded":"2024-03-08T20:57:52.072961+0000","last_fullsized":"2024-03-08T20:57:52.072961+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T02:13:13.570078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-08T20:57:52.073692+0000","last_change":"2024-03-08T20:57:50.794051+0000","last_active":"2024-03-08T20:57:52.073692+0000","last_peered":"2024-03-08T20:57:52.073692+0000","last_clean":"2024-03-08T20:57:52.073692+0000","last_became_active":"2024-03-08T20:57:50.793547+0000","last_became_peered":"2024-03-08T20:57:50.793547+0000","last_unstale":"2024-03-08T20:57:52.073692+0000","last_undegraded":"2024-03-08T20:57:52.073692+0000","last_fullsized":"2024-03-08T20:57:52.073692+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T04:13:52.705247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-08T20:57:52.073281+0000","last_change":"2024-03-08T20:57:50.789449+0000","last_active":"2024-03-08T20:57:52.073281+0000","last_peered":"2024-03-08T20:57:52.073281+0000","last_clean":"2024-03-08T20:57:52.073281+0000","last_became_active":"2024-03-08T20:57:50.789106+0000","last_became_peered":"2024-03-08T20:57:50.789106+0000","last_unstale":"2024-03-08T20:57:52.073281+0000","last_undegraded":"2024-03-08T20:57:52.073281+0000","last_fullsized":"2024-03-08T20:57:52.073281+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T02:12:47.198726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-08T20:57:51.804816+0000","last_change":"2024-03-08T20:57:50.814360+0000","last_active":"2024-03-08T20:57:51.804816+0000","last_peered":"2024-03-08T20:57:51.804816+0000","last_clean":"2024-03-08T20:57:51.804816+0000","last_became_active":"2024-03-08T20:57:50.813966+0000","last_became_peered":"2024-03-08T20:57:50.813966+0000","last_unstale":"2024-03-08T20:57:51.804816+0000","last_undegraded":"2024-03-08T20:57:51.804816+0000","last_fullsized":"2024-03-08T20:57:51.804816+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T04:39:10.324938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-08T20:57:52.073448+0000","last_change":"2024-03-08T20:57:50.803658+0000","last_active":"2024-03-08T20:57:52.073448+0000","last_peered":"2024-03-08T20:57:52.073448+0000","last_clean":"2024-03-08T20:57:52.073448+0000","last_became_active":"2024-03-08T20:57:50.797593+0000","last_became_peered":"2024-03-08T20:57:50.797593+0000","last_unstale":"2024-03-08T20:57:52.073448+0000","last_undegraded":"2024-03-08T20:57:52.073448+0000","last_fullsized":"2024-03-08T20:57:52.073448+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T21:45:33.579087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-08T20:57:51.793416+0000","last_change":"2024-03-08T20:57:50.789913+0000","last_active":"2024-03-08T20:57:51.793416+0000","last_peered":"2024-03-08T20:57:51.793416+0000","last_clean":"2024-03-08T20:57:51.793416+0000","last_became_active":"2024-03-08T20:57:50.789492+0000","last_became_peered":"2024-03-08T20:57:50.789492+0000","last_unstale":"2024-03-08T20:57:51.793416+0000","last_undegraded":"2024-03-08T20:57:51.793416+0000","last_fullsized":"2024-03-08T20:57:51.793416+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T01:25:14.329005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-08T20:57:51.793346+0000","last_change":"2024-03-08T20:57:50.801743+0000","last_active":"2024-03-08T20:57:51.793346+0000","last_peered":"2024-03-08T20:57:51.793346+0000","last_clean":"2024-03-08T20:57:51.793346+0000","last_became_active":"2024-03-08T20:57:50.801261+0000","last_became_peered":"2024-03-08T20:57:50.801261+0000","last_unstale":"2024-03-08T20:57:51.793346+0000","last_undegraded":"2024-03-08T20:57:51.793346+0000","last_fullsized":"2024-03-08T20:57:51.793346+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T23:00:15.605070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-08T20:57:51.792139+0000","last_change":"2024-03-08T20:57:50.800759+0000","last_active":"2024-03-08T20:57:51.792139+0000","last_peered":"2024-03-08T20:57:51.792139+0000","last_clean":"2024-03-08T20:57:51.792139+0000","last_became_active":"2024-03-08T20:57:50.798328+0000","last_became_peered":"2024-03-08T20:57:50.798328+0000","last_unstale":"2024-03-08T20:57:51.792139+0000","last_undegraded":"2024-03-08T20:57:51.792139+0000","last_fullsized":"2024-03-08T20:57:51.792139+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T05:43:00.160071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-08T20:57:51.794830+0000","last_change":"2024-03-08T20:57:50.802262+0000","last_active":"2024-03-08T20:57:51.794830+0000","last_peered":"2024-03-08T20:57:51.794830+0000","last_clean":"2024-03-08T20:57:51.794830+0000","last_became_active":"2024-03-08T20:57:50.801786+0000","last_became_peered":"2024-03-08T20:57:50.801786+0000","last_unstale":"2024-03-08T20:57:51.794830+0000","last_undegraded":"2024-03-08T20:57:51.794830+0000","last_fullsized":"2024-03-08T20:57:51.794830+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T00:47:48.400329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-08T20:57:52.073201+0000","last_change":"2024-03-08T20:57:50.790819+0000","last_active":"2024-03-08T20:57:52.073201+0000","last_peered":"2024-03-08T20:57:52.073201+0000","last_clean":"2024-03-08T20:57:52.073201+0000","last_became_active":"2024-03-08T20:57:50.786840+0000","last_became_peered":"2024-03-08T20:57:50.786840+0000","last_unstale":"2024-03-08T20:57:52.073201+0000","last_undegraded":"2024-03-08T20:57:52.073201+0000","last_fullsized":"2024-03-08T20:57:52.073201+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T06:52:19.567245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-08T20:57:51.804858+0000","last_change":"2024-03-08T20:57:50.806693+0000","last_active":"2024-03-08T20:57:51.804858+0000","last_peered":"2024-03-08T20:57:51.804858+0000","last_clean":"2024-03-08T20:57:51.804858+0000","last_became_active":"2024-03-08T20:57:50.806299+0000","last_became_peered":"2024-03-08T20:57:50.806299+0000","last_unstale":"2024-03-08T20:57:51.804858+0000","last_undegraded":"2024-03-08T20:57:51.804858+0000","last_fullsized":"2024-03-08T20:57:51.804858+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T02:04:07.567265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":351,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":280946,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":26589,"num_objects":88,"num_object_clones":0,"num_object_copies":264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":88,"num_whiteouts":0,"num_read":14,"num_read_kb":12,"num_write":202,"num_write_kb":70,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":60,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":380928,"data_stored":58906,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":173,"ondisk_log_size":173,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892350993,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32092,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713060,"statfs":{"total":95995035648,"available":95962173440,"internally_reserved":0,"allocated":937984,"data_stored":444207,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612632,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32188,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712964,"statfs":{"total":95995035648,"available":95962075136,"internally_reserved":0,"allocated":1036288,"data_stored":1043331,"data_compressed":35734,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1599,"internal_metadata":31914433},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776159,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32116,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713036,"statfs":{"total":95995035648,"available":95962148864,"internally_reserved":0,"allocated":962560,"data_stored":444137,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037798,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":958464,"data_stored":444515,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299436,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1011712,"data_stored":1036642,"data_compressed":35734,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462963,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32080,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713072,"statfs":{"total":95995035648,"available":95962185728,"internally_reserved":0,"allocated":925696,"data_stored":443015,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574933,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":937984,"data_stored":444627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738461,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32120,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713032,"statfs":{"total":95995035648,"available":95962144768,"internally_reserved":0,"allocated":966656,"data_stored":445193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1],"upper_bound":8},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411408,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":983040,"data_stored":459755,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182472,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32112,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713040,"statfs":{"total":95995035648,"available":95962152960,"internally_reserved":0,"allocated":958464,"data_stored":443562,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920834,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32096,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713056,"statfs":{"total":95995035648,"available":95962169344,"internally_reserved":0,"allocated":942080,"data_stored":443681,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724603,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32156,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712996,"statfs":{"total":95995035648,"available":95962107904,"internally_reserved":0,"allocated":1003520,"data_stored":1034859,"data_compressed":35734,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34092,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34092,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34092,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35734,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35734,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35734,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35734,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35734,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":22867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":666,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":22631,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-08T20:58:00.364 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-08T20:58:00.379 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:59 smithi204 ceph-mon[34935]: mgrmap e17: x(active, since 9m), standbys: z, y 2024-03-08T20:58:00.379 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:59 smithi204 ceph-mon[34935]: from='client.14763 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-08T20:58:00.379 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:59 smithi204 ceph-mon[34935]: from='client.? 172.21.15.107:0/378324771' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-08T20:58:00.379 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:57:59 smithi204 ceph-mon[34935]: from='client.? 172.21.15.107:0/1608738739' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-08T20:58:00.613 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:00.629+0000 7fc632800640 1 -- 172.21.15.107:0/1628176311 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc62c104720 msgr2=0x7fc62c104ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:58:00.614 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:00.629+0000 7fc632800640 1 --2- 172.21.15.107:0/1628176311 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc62c104720 0x7fc62c104ba0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fc614009900 tx=0x7fc61402bc30 comp rx=0 tx=0).stop 2024-03-08T20:58:00.614 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:00.629+0000 7fc632800640 1 -- 172.21.15.107:0/1628176311 shutdown_connections 2024-03-08T20:58:00.614 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:00.629+0000 7fc632800640 1 --2- 172.21.15.107:0/1628176311 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc62c1050e0 0x7fc62c10b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:58:00.614 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:00.629+0000 7fc632800640 1 --2- 172.21.15.107:0/1628176311 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc62c104720 0x7fc62c104ba0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:58:00.614 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:00.629+0000 7fc632800640 1 --2- 172.21.15.107:0/1628176311 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc62c103530 0x7fc62c103950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:58:00.614 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:00.629+0000 7fc632800640 1 -- 172.21.15.107:0/1628176311 >> 172.21.15.107:0/1628176311 conn(0x7fc62c0feae0 msgr2=0x7fc62c100f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:58:00.614 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:00.629+0000 7fc632800640 1 -- 172.21.15.107:0/1628176311 shutdown_connections 2024-03-08T20:58:00.614 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:00.629+0000 7fc632800640 1 -- 172.21.15.107:0/1628176311 wait complete. 2024-03-08T20:58:00.616 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:00.630+0000 7fc632800640 1 Processor -- start 2024-03-08T20:58:00.616 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:00.630+0000 7fc632800640 1 -- start start 2024-03-08T20:58:00.616 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:00.630+0000 7fc632800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc62c103530 0x7fc62c199760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:58:00.617 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:00.630+0000 7fc632800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc62c104720 0x7fc62c199ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:58:00.617 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:00.630+0000 7fc632800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc62c1050e0 0x7fc62c1a0d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:58:00.617 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:00.630+0000 7fc632800640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc62c10f0c0 con 0x7fc62c104720 2024-03-08T20:58:00.617 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:00.630+0000 7fc632800640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc62c10ef40 con 0x7fc62c103530 2024-03-08T20:58:00.617 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:00.630+0000 7fc632800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc62c10f240 con 0x7fc62c1050e0 2024-03-08T20:58:00.617 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:00.631+0000 7fc622a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc62c104720 0x7fc62c199ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:58:00.617 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:00.631+0000 7fc622a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc62c104720 0x7fc62c199ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:57280/0 (socket says 172.21.15.107:57280) 2024-03-08T20:58:00.617 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:00.631+0000 7fc622a00640 1 -- 172.21.15.107:0/3189015615 learned_addr learned my addr 172.21.15.107:0/3189015615 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T20:58:00.617 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:00.631+0000 7fc62b400640 1 --2- 172.21.15.107:0/3189015615 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc62c103530 0x7fc62c199760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:58:00.617 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:00.631+0000 7fc622a00640 1 -- 172.21.15.107:0/3189015615 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc62c1050e0 msgr2=0x7fc62c1a0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:58:00.617 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:00.631+0000 7fc62be00640 1 --2- 172.21.15.107:0/3189015615 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc62c1050e0 0x7fc62c1a0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:58:00.617 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:00.631+0000 7fc622a00640 1 --2- 172.21.15.107:0/3189015615 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc62c1050e0 0x7fc62c1a0d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:58:00.617 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:00.631+0000 7fc622a00640 1 -- 172.21.15.107:0/3189015615 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc62c103530 msgr2=0x7fc62c199760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:58:00.618 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:00.631+0000 7fc622a00640 1 --2- 172.21.15.107:0/3189015615 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc62c103530 0x7fc62c199760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:58:00.618 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:00.631+0000 7fc622a00640 1 -- 172.21.15.107:0/3189015615 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc62c1a1380 con 0x7fc62c104720 2024-03-08T20:58:00.618 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:00.631+0000 7fc62b400640 1 --2- 172.21.15.107:0/3189015615 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc62c103530 0x7fc62c199760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T20:58:00.618 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:00.631+0000 7fc62be00640 1 --2- 172.21.15.107:0/3189015615 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc62c1050e0 0x7fc62c1a0d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T20:58:00.618 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:00.631+0000 7fc622a00640 1 --2- 172.21.15.107:0/3189015615 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc62c104720 0x7fc62c199ca0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fc614000c00 tx=0x7fc6140042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:58:00.618 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:00.632+0000 7fc628c00640 1 -- 172.21.15.107:0/3189015615 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc614043020 con 0x7fc62c104720 2024-03-08T20:58:00.618 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:00.632+0000 7fc628c00640 1 -- 172.21.15.107:0/3189015615 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fc6140054c0 con 0x7fc62c104720 2024-03-08T20:58:00.618 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:00.632+0000 7fc632800640 1 -- 172.21.15.107:0/3189015615 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc62c1a1610 con 0x7fc62c104720 2024-03-08T20:58:00.618 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:00.632+0000 7fc628c00640 1 -- 172.21.15.107:0/3189015615 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc614003d70 con 0x7fc62c104720 2024-03-08T20:58:00.618 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:00.632+0000 7fc632800640 1 -- 172.21.15.107:0/3189015615 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc62c1a1bb0 con 0x7fc62c104720 2024-03-08T20:58:00.619 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:00.633+0000 7fc632800640 1 -- 172.21.15.107:0/3189015615 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc62c10a8d0 con 0x7fc62c104720 2024-03-08T20:58:00.620 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:00.636+0000 7fc628c00640 1 -- 172.21.15.107:0/3189015615 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) v1 ==== 137342+0+0 (secure 0 0 0) 0x7fc614046020 con 0x7fc62c104720 2024-03-08T20:58:00.621 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:00.637+0000 7fc628c00640 1 --2- 172.21.15.107:0/3189015615 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fc6100a2910 0x7fc6100a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:58:00.622 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:00.638+0000 7fc628c00640 1 -- 172.21.15.107:0/3189015615 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fc6140f12e0 con 0x7fc62c104720 2024-03-08T20:58:00.622 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:00.638+0000 7fc62b400640 1 --2- 172.21.15.107:0/3189015615 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fc6100a2910 0x7fc6100a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:58:00.622 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:00.638+0000 7fc62b400640 1 --2- 172.21.15.107:0/3189015615 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fc6100a2910 0x7fc6100a4dd0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fc62c104580 tx=0x7fc61c005e70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:58:00.624 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:00.640+0000 7fc628c00640 1 -- 172.21.15.107:0/3189015615 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7fc61404d020 con 0x7fc62c104720 2024-03-08T20:58:00.819 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:00.835+0000 7fc632800640 1 -- 172.21.15.107:0/3189015615 --> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc62c061730 con 0x7fc6100a2910 2024-03-08T20:58:00.828 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:00.844+0000 7fc628c00640 1 -- 172.21.15.107:0/3189015615 <== mgr.14152 v2:172.21.15.107:6800/3376636618 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+259033 (secure 0 0 0) 0x7fc62c061730 con 0x7fc6100a2910 2024-03-08T20:58:00.831 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-08T20:58:00.833 INFO:teuthology.orchestra.run.smithi107.stderr:dumped all 2024-03-08T20:58:00.834 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:00.849+0000 7fc632800640 1 -- 172.21.15.107:0/3189015615 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fc6100a2910 msgr2=0x7fc6100a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:58:00.835 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:00.849+0000 7fc632800640 1 --2- 172.21.15.107:0/3189015615 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fc6100a2910 0x7fc6100a4dd0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fc62c104580 tx=0x7fc61c005e70 comp rx=0 tx=0).stop 2024-03-08T20:58:00.835 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:00.849+0000 7fc632800640 1 -- 172.21.15.107:0/3189015615 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc62c104720 msgr2=0x7fc62c199ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:58:00.835 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:00.849+0000 7fc632800640 1 --2- 172.21.15.107:0/3189015615 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc62c104720 0x7fc62c199ca0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fc614000c00 tx=0x7fc6140042f0 comp rx=0 tx=0).stop 2024-03-08T20:58:00.835 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:00.850+0000 7fc632800640 1 -- 172.21.15.107:0/3189015615 shutdown_connections 2024-03-08T20:58:00.835 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:00.850+0000 7fc632800640 1 --2- 172.21.15.107:0/3189015615 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fc6100a2910 0x7fc6100a4dd0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:58:00.835 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:00.850+0000 7fc632800640 1 --2- 172.21.15.107:0/3189015615 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc62c1050e0 0x7fc62c1a0d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:58:00.835 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:00.850+0000 7fc632800640 1 --2- 172.21.15.107:0/3189015615 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc62c104720 0x7fc62c199ca0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:58:00.835 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:00.850+0000 7fc632800640 1 --2- 172.21.15.107:0/3189015615 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc62c103530 0x7fc62c199760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:58:00.835 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:00.850+0000 7fc632800640 1 -- 172.21.15.107:0/3189015615 >> 172.21.15.107:0/3189015615 conn(0x7fc62c0feae0 msgr2=0x7fc62c107aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:58:00.835 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:00.850+0000 7fc632800640 1 -- 172.21.15.107:0/3189015615 shutdown_connections 2024-03-08T20:58:00.835 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:00.850+0000 7fc632800640 1 -- 172.21.15.107:0/3189015615 wait complete. 2024-03-08T20:58:00.852 INFO:teuthology.orchestra.run.smithi107.stdout:{"pg_ready":true,"pg_map":{"version":366,"stamp":"2024-03-08T20:58:00.314953+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":617191,"num_objects":92,"num_object_clones":0,"num_object_copies":276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":92,"num_whiteouts":0,"num_read":150,"num_read_kb":131,"num_write":385,"num_write_kb":3010,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":60,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":312,"ondisk_log_size":312,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":368828,"kb_used_data":11352,"kb_used_omap":18,"kb_used_meta":357357,"kb_avail":1124572996,"statfs":{"total":1151940427776,"available":1151562747904,"internally_reserved":0,"allocated":11624448,"data_stored":7127524,"data_compressed":107202,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19138,"internal_metadata":365933886},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1],"upper_bound":8},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":26823,"num_objects":90,"num_object_clones":0,"num_object_copies":270,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":90,"num_whiteouts":0,"num_read":14,"num_read_kb":12,"num_write":204,"num_write_kb":72,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":60,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.555043"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-08T20:57:51.805111+0000","last_change":"2024-03-08T20:57:50.800421+0000","last_active":"2024-03-08T20:57:51.805111+0000","last_peered":"2024-03-08T20:57:51.805111+0000","last_clean":"2024-03-08T20:57:51.805111+0000","last_became_active":"2024-03-08T20:57:50.799682+0000","last_became_peered":"2024-03-08T20:57:50.799682+0000","last_unstale":"2024-03-08T20:57:51.805111+0000","last_undegraded":"2024-03-08T20:57:51.805111+0000","last_fullsized":"2024-03-08T20:57:51.805111+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T07:25:11.394329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-08T20:57:52.073778+0000","last_change":"2024-03-08T20:57:50.800332+0000","last_active":"2024-03-08T20:57:52.073778+0000","last_peered":"2024-03-08T20:57:52.073778+0000","last_clean":"2024-03-08T20:57:52.073778+0000","last_became_active":"2024-03-08T20:57:50.799981+0000","last_became_peered":"2024-03-08T20:57:50.799981+0000","last_unstale":"2024-03-08T20:57:52.073778+0000","last_undegraded":"2024-03-08T20:57:52.073778+0000","last_fullsized":"2024-03-08T20:57:52.073778+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T06:08:19.618115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-08T20:57:52.073846+0000","last_change":"2024-03-08T20:57:50.800982+0000","last_active":"2024-03-08T20:57:52.073846+0000","last_peered":"2024-03-08T20:57:52.073846+0000","last_clean":"2024-03-08T20:57:52.073846+0000","last_became_active":"2024-03-08T20:57:50.800661+0000","last_became_peered":"2024-03-08T20:57:50.800661+0000","last_unstale":"2024-03-08T20:57:52.073846+0000","last_undegraded":"2024-03-08T20:57:52.073846+0000","last_fullsized":"2024-03-08T20:57:52.073846+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T22:33:08.869516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-08T20:57:52.075049+0000","last_change":"2024-03-08T20:57:50.792110+0000","last_active":"2024-03-08T20:57:52.075049+0000","last_peered":"2024-03-08T20:57:52.075049+0000","last_clean":"2024-03-08T20:57:52.075049+0000","last_became_active":"2024-03-08T20:57:50.790781+0000","last_became_peered":"2024-03-08T20:57:50.790781+0000","last_unstale":"2024-03-08T20:57:52.075049+0000","last_undegraded":"2024-03-08T20:57:52.075049+0000","last_fullsized":"2024-03-08T20:57:52.075049+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T05:28:50.939269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-08T20:57:52.073859+0000","last_change":"2024-03-08T20:57:50.804087+0000","last_active":"2024-03-08T20:57:52.073859+0000","last_peered":"2024-03-08T20:57:52.073859+0000","last_clean":"2024-03-08T20:57:52.073859+0000","last_became_active":"2024-03-08T20:57:50.803704+0000","last_became_peered":"2024-03-08T20:57:50.803704+0000","last_unstale":"2024-03-08T20:57:52.073859+0000","last_undegraded":"2024-03-08T20:57:52.073859+0000","last_fullsized":"2024-03-08T20:57:52.073859+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T21:23:16.770787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-08T20:57:52.075693+0000","last_change":"2024-03-08T20:57:50.789245+0000","last_active":"2024-03-08T20:57:52.075693+0000","last_peered":"2024-03-08T20:57:52.075693+0000","last_clean":"2024-03-08T20:57:52.075693+0000","last_became_active":"2024-03-08T20:57:50.788921+0000","last_became_peered":"2024-03-08T20:57:50.788921+0000","last_unstale":"2024-03-08T20:57:52.075693+0000","last_undegraded":"2024-03-08T20:57:52.075693+0000","last_fullsized":"2024-03-08T20:57:52.075693+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T08:57:30.368714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-08T20:57:52.073207+0000","last_change":"2024-03-08T20:57:50.800725+0000","last_active":"2024-03-08T20:57:52.073207+0000","last_peered":"2024-03-08T20:57:52.073207+0000","last_clean":"2024-03-08T20:57:52.073207+0000","last_became_active":"2024-03-08T20:57:50.800317+0000","last_became_peered":"2024-03-08T20:57:50.800317+0000","last_unstale":"2024-03-08T20:57:52.073207+0000","last_undegraded":"2024-03-08T20:57:52.073207+0000","last_fullsized":"2024-03-08T20:57:52.073207+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T05:44:01.402547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-08T20:57:51.791880+0000","last_change":"2024-03-08T20:57:50.801543+0000","last_active":"2024-03-08T20:57:51.791880+0000","last_peered":"2024-03-08T20:57:51.791880+0000","last_clean":"2024-03-08T20:57:51.791880+0000","last_became_active":"2024-03-08T20:57:50.801111+0000","last_became_peered":"2024-03-08T20:57:50.801111+0000","last_unstale":"2024-03-08T20:57:51.791880+0000","last_undegraded":"2024-03-08T20:57:51.791880+0000","last_fullsized":"2024-03-08T20:57:51.791880+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T06:32:55.234372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-08T20:57:51.795132+0000","last_change":"2024-03-08T20:57:50.798169+0000","last_active":"2024-03-08T20:57:51.795132+0000","last_peered":"2024-03-08T20:57:51.795132+0000","last_clean":"2024-03-08T20:57:51.795132+0000","last_became_active":"2024-03-08T20:57:50.797817+0000","last_became_peered":"2024-03-08T20:57:50.797817+0000","last_unstale":"2024-03-08T20:57:51.795132+0000","last_undegraded":"2024-03-08T20:57:51.795132+0000","last_fullsized":"2024-03-08T20:57:51.795132+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T04:01:46.179415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-08T20:57:52.075986+0000","last_change":"2024-03-08T20:57:50.790757+0000","last_active":"2024-03-08T20:57:52.075986+0000","last_peered":"2024-03-08T20:57:52.075986+0000","last_clean":"2024-03-08T20:57:52.075986+0000","last_became_active":"2024-03-08T20:57:50.790219+0000","last_became_peered":"2024-03-08T20:57:50.790219+0000","last_unstale":"2024-03-08T20:57:52.075986+0000","last_undegraded":"2024-03-08T20:57:52.075986+0000","last_fullsized":"2024-03-08T20:57:52.075986+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T07:55:54.945727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-08T20:57:52.074250+0000","last_change":"2024-03-08T20:57:50.809102+0000","last_active":"2024-03-08T20:57:52.074250+0000","last_peered":"2024-03-08T20:57:52.074250+0000","last_clean":"2024-03-08T20:57:52.074250+0000","last_became_active":"2024-03-08T20:57:50.808647+0000","last_became_peered":"2024-03-08T20:57:50.808647+0000","last_unstale":"2024-03-08T20:57:52.074250+0000","last_undegraded":"2024-03-08T20:57:52.074250+0000","last_fullsized":"2024-03-08T20:57:52.074250+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T00:29:26.317611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-08T20:57:52.074880+0000","last_change":"2024-03-08T20:57:50.791791+0000","last_active":"2024-03-08T20:57:52.074880+0000","last_peered":"2024-03-08T20:57:52.074880+0000","last_clean":"2024-03-08T20:57:52.074880+0000","last_became_active":"2024-03-08T20:57:50.790264+0000","last_became_peered":"2024-03-08T20:57:50.790264+0000","last_unstale":"2024-03-08T20:57:52.074880+0000","last_undegraded":"2024-03-08T20:57:52.074880+0000","last_fullsized":"2024-03-08T20:57:52.074880+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T08:54:30.221330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-08T20:57:52.073409+0000","last_change":"2024-03-08T20:57:50.793990+0000","last_active":"2024-03-08T20:57:52.073409+0000","last_peered":"2024-03-08T20:57:52.073409+0000","last_clean":"2024-03-08T20:57:52.073409+0000","last_became_active":"2024-03-08T20:57:50.792304+0000","last_became_peered":"2024-03-08T20:57:50.792304+0000","last_unstale":"2024-03-08T20:57:52.073409+0000","last_undegraded":"2024-03-08T20:57:52.073409+0000","last_fullsized":"2024-03-08T20:57:52.073409+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T00:21:57.491218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-08T20:57:52.073068+0000","last_change":"2024-03-08T20:57:50.787621+0000","last_active":"2024-03-08T20:57:52.073068+0000","last_peered":"2024-03-08T20:57:52.073068+0000","last_clean":"2024-03-08T20:57:52.073068+0000","last_became_active":"2024-03-08T20:57:50.786461+0000","last_became_peered":"2024-03-08T20:57:50.786461+0000","last_unstale":"2024-03-08T20:57:52.073068+0000","last_undegraded":"2024-03-08T20:57:52.073068+0000","last_fullsized":"2024-03-08T20:57:52.073068+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T08:35:32.513942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-08T20:57:52.073416+0000","last_change":"2024-03-08T20:57:50.789157+0000","last_active":"2024-03-08T20:57:52.073416+0000","last_peered":"2024-03-08T20:57:52.073416+0000","last_clean":"2024-03-08T20:57:52.073416+0000","last_became_active":"2024-03-08T20:57:50.788828+0000","last_became_peered":"2024-03-08T20:57:50.788828+0000","last_unstale":"2024-03-08T20:57:52.073416+0000","last_undegraded":"2024-03-08T20:57:52.073416+0000","last_fullsized":"2024-03-08T20:57:52.073416+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T05:43:53.398306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-08T20:57:51.793598+0000","last_change":"2024-03-08T20:57:50.796888+0000","last_active":"2024-03-08T20:57:51.793598+0000","last_peered":"2024-03-08T20:57:51.793598+0000","last_clean":"2024-03-08T20:57:51.793598+0000","last_became_active":"2024-03-08T20:57:50.795797+0000","last_became_peered":"2024-03-08T20:57:50.795797+0000","last_unstale":"2024-03-08T20:57:51.793598+0000","last_undegraded":"2024-03-08T20:57:51.793598+0000","last_fullsized":"2024-03-08T20:57:51.793598+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T00:43:53.838877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-08T20:57:51.805178+0000","last_change":"2024-03-08T20:57:50.801849+0000","last_active":"2024-03-08T20:57:51.805178+0000","last_peered":"2024-03-08T20:57:51.805178+0000","last_clean":"2024-03-08T20:57:51.805178+0000","last_became_active":"2024-03-08T20:57:50.801375+0000","last_became_peered":"2024-03-08T20:57:50.801375+0000","last_unstale":"2024-03-08T20:57:51.805178+0000","last_undegraded":"2024-03-08T20:57:51.805178+0000","last_fullsized":"2024-03-08T20:57:51.805178+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T03:16:44.652264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-08T20:57:52.074580+0000","last_change":"2024-03-08T20:57:50.790493+0000","last_active":"2024-03-08T20:57:52.074580+0000","last_peered":"2024-03-08T20:57:52.074580+0000","last_clean":"2024-03-08T20:57:52.074580+0000","last_became_active":"2024-03-08T20:57:50.789074+0000","last_became_peered":"2024-03-08T20:57:50.789074+0000","last_unstale":"2024-03-08T20:57:52.074580+0000","last_undegraded":"2024-03-08T20:57:52.074580+0000","last_fullsized":"2024-03-08T20:57:52.074580+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T05:52:45.700669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-08T20:57:52.073357+0000","last_change":"2024-03-08T20:57:50.794348+0000","last_active":"2024-03-08T20:57:52.073357+0000","last_peered":"2024-03-08T20:57:52.073357+0000","last_clean":"2024-03-08T20:57:52.073357+0000","last_became_active":"2024-03-08T20:57:50.793962+0000","last_became_peered":"2024-03-08T20:57:50.793962+0000","last_unstale":"2024-03-08T20:57:52.073357+0000","last_undegraded":"2024-03-08T20:57:52.073357+0000","last_fullsized":"2024-03-08T20:57:52.073357+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T01:38:05.146148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"86'13","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-08T20:57:57.897048+0000","last_change":"2024-03-08T20:57:49.792250+0000","last_active":"2024-03-08T20:57:57.897048+0000","last_peered":"2024-03-08T20:57:57.897048+0000","last_clean":"2024-03-08T20:57:57.897048+0000","last_became_active":"2024-03-08T20:57:49.791863+0000","last_became_peered":"2024-03-08T20:57:49.791863+0000","last_unstale":"2024-03-08T20:57:57.897048+0000","last_undegraded":"2024-03-08T20:57:57.897048+0000","last_fullsized":"2024-03-08T20:57:57.897048+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_clean_scrub_stamp":"2024-03-08T20:57:48.739829+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T05:37:40.942754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-08T20:57:51.793649+0000","last_change":"2024-03-08T20:57:50.802789+0000","last_active":"2024-03-08T20:57:51.793649+0000","last_peered":"2024-03-08T20:57:51.793649+0000","last_clean":"2024-03-08T20:57:51.793649+0000","last_became_active":"2024-03-08T20:57:50.802437+0000","last_became_peered":"2024-03-08T20:57:50.802437+0000","last_unstale":"2024-03-08T20:57:51.793649+0000","last_undegraded":"2024-03-08T20:57:51.793649+0000","last_fullsized":"2024-03-08T20:57:51.793649+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T23:47:39.882241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-08T20:57:53.381760+0000","last_change":"2024-03-08T20:57:49.785390+0000","last_active":"2024-03-08T20:57:53.381760+0000","last_peered":"2024-03-08T20:57:53.381760+0000","last_clean":"2024-03-08T20:57:53.381760+0000","last_became_active":"2024-03-08T20:57:49.784506+0000","last_became_peered":"2024-03-08T20:57:49.784506+0000","last_unstale":"2024-03-08T20:57:53.381760+0000","last_undegraded":"2024-03-08T20:57:53.381760+0000","last_fullsized":"2024-03-08T20:57:53.381760+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_clean_scrub_stamp":"2024-03-08T20:57:48.739829+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T03:21:54.331376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-08T20:57:52.074165+0000","last_change":"2024-03-08T20:57:50.796868+0000","last_active":"2024-03-08T20:57:52.074165+0000","last_peered":"2024-03-08T20:57:52.074165+0000","last_clean":"2024-03-08T20:57:52.074165+0000","last_became_active":"2024-03-08T20:57:50.796163+0000","last_became_peered":"2024-03-08T20:57:50.796163+0000","last_unstale":"2024-03-08T20:57:52.074165+0000","last_undegraded":"2024-03-08T20:57:52.074165+0000","last_fullsized":"2024-03-08T20:57:52.074165+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T21:26:46.335292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"86'12","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-08T20:57:57.882244+0000","last_change":"2024-03-08T20:57:49.794940+0000","last_active":"2024-03-08T20:57:57.882244+0000","last_peered":"2024-03-08T20:57:57.882244+0000","last_clean":"2024-03-08T20:57:57.882244+0000","last_became_active":"2024-03-08T20:57:49.794386+0000","last_became_peered":"2024-03-08T20:57:49.794386+0000","last_unstale":"2024-03-08T20:57:57.882244+0000","last_undegraded":"2024-03-08T20:57:57.882244+0000","last_fullsized":"2024-03-08T20:57:57.882244+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_clean_scrub_stamp":"2024-03-08T20:57:48.739829+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T04:21:09.953351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-08T20:57:51.794091+0000","last_change":"2024-03-08T20:57:50.794804+0000","last_active":"2024-03-08T20:57:51.794091+0000","last_peered":"2024-03-08T20:57:51.794091+0000","last_clean":"2024-03-08T20:57:51.794091+0000","last_became_active":"2024-03-08T20:57:50.794380+0000","last_became_peered":"2024-03-08T20:57:50.794380+0000","last_unstale":"2024-03-08T20:57:51.794091+0000","last_undegraded":"2024-03-08T20:57:51.794091+0000","last_fullsized":"2024-03-08T20:57:51.794091+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T05:32:21.015740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"86'5","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-08T20:57:56.893654+0000","last_change":"2024-03-08T20:57:49.790881+0000","last_active":"2024-03-08T20:57:56.893654+0000","last_peered":"2024-03-08T20:57:56.893654+0000","last_clean":"2024-03-08T20:57:56.893654+0000","last_became_active":"2024-03-08T20:57:49.790562+0000","last_became_peered":"2024-03-08T20:57:49.790562+0000","last_unstale":"2024-03-08T20:57:56.893654+0000","last_undegraded":"2024-03-08T20:57:56.893654+0000","last_fullsized":"2024-03-08T20:57:56.893654+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_clean_scrub_stamp":"2024-03-08T20:57:48.739829+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T23:57:40.542756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-08T20:57:51.794667+0000","last_change":"2024-03-08T20:57:50.799021+0000","last_active":"2024-03-08T20:57:51.794667+0000","last_peered":"2024-03-08T20:57:51.794667+0000","last_clean":"2024-03-08T20:57:51.794667+0000","last_became_active":"2024-03-08T20:57:50.795494+0000","last_became_peered":"2024-03-08T20:57:50.795494+0000","last_unstale":"2024-03-08T20:57:51.794667+0000","last_undegraded":"2024-03-08T20:57:51.794667+0000","last_fullsized":"2024-03-08T20:57:51.794667+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T21:26:40.936789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"86'5","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-08T20:57:53.846241+0000","last_change":"2024-03-08T20:57:49.791167+0000","last_active":"2024-03-08T20:57:53.846241+0000","last_peered":"2024-03-08T20:57:53.846241+0000","last_clean":"2024-03-08T20:57:53.846241+0000","last_became_active":"2024-03-08T20:57:49.789840+0000","last_became_peered":"2024-03-08T20:57:49.789840+0000","last_unstale":"2024-03-08T20:57:53.846241+0000","last_undegraded":"2024-03-08T20:57:53.846241+0000","last_fullsized":"2024-03-08T20:57:53.846241+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_clean_scrub_stamp":"2024-03-08T20:57:48.739829+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T23:14:51.103583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-08T20:57:52.073889+0000","last_change":"2024-03-08T20:57:50.812656+0000","last_active":"2024-03-08T20:57:52.073889+0000","last_peered":"2024-03-08T20:57:52.073889+0000","last_clean":"2024-03-08T20:57:52.073889+0000","last_became_active":"2024-03-08T20:57:50.812281+0000","last_became_peered":"2024-03-08T20:57:50.812281+0000","last_unstale":"2024-03-08T20:57:52.073889+0000","last_undegraded":"2024-03-08T20:57:52.073889+0000","last_fullsized":"2024-03-08T20:57:52.073889+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T05:40:20.985798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"86'7","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-08T20:57:55.862604+0000","last_change":"2024-03-08T20:57:49.792874+0000","last_active":"2024-03-08T20:57:55.862604+0000","last_peered":"2024-03-08T20:57:55.862604+0000","last_clean":"2024-03-08T20:57:55.862604+0000","last_became_active":"2024-03-08T20:57:49.792418+0000","last_became_peered":"2024-03-08T20:57:49.792418+0000","last_unstale":"2024-03-08T20:57:55.862604+0000","last_undegraded":"2024-03-08T20:57:55.862604+0000","last_fullsized":"2024-03-08T20:57:55.862604+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_clean_scrub_stamp":"2024-03-08T20:57:48.739829+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T02:23:12.888435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-08T20:57:51.794237+0000","last_change":"2024-03-08T20:57:50.805567+0000","last_active":"2024-03-08T20:57:51.794237+0000","last_peered":"2024-03-08T20:57:51.794237+0000","last_clean":"2024-03-08T20:57:51.794237+0000","last_became_active":"2024-03-08T20:57:50.805120+0000","last_became_peered":"2024-03-08T20:57:50.805120+0000","last_unstale":"2024-03-08T20:57:51.794237+0000","last_undegraded":"2024-03-08T20:57:51.794237+0000","last_fullsized":"2024-03-08T20:57:51.794237+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T06:18:14.297971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"86'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-08T20:57:54.859110+0000","last_change":"2024-03-08T20:57:49.771806+0000","last_active":"2024-03-08T20:57:54.859110+0000","last_peered":"2024-03-08T20:57:54.859110+0000","last_clean":"2024-03-08T20:57:54.859110+0000","last_became_active":"2024-03-08T20:57:49.771447+0000","last_became_peered":"2024-03-08T20:57:49.771447+0000","last_unstale":"2024-03-08T20:57:54.859110+0000","last_undegraded":"2024-03-08T20:57:54.859110+0000","last_fullsized":"2024-03-08T20:57:54.859110+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_clean_scrub_stamp":"2024-03-08T20:57:48.739829+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T03:16:37.913599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-08T20:57:51.791871+0000","last_change":"2024-03-08T20:57:50.792915+0000","last_active":"2024-03-08T20:57:51.791871+0000","last_peered":"2024-03-08T20:57:51.791871+0000","last_clean":"2024-03-08T20:57:51.791871+0000","last_became_active":"2024-03-08T20:57:50.792617+0000","last_became_peered":"2024-03-08T20:57:50.792617+0000","last_unstale":"2024-03-08T20:57:51.791871+0000","last_undegraded":"2024-03-08T20:57:51.791871+0000","last_fullsized":"2024-03-08T20:57:51.791871+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T01:59:30.427751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"86'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-08T20:57:51.793443+0000","last_change":"2024-03-08T20:57:49.789383+0000","last_active":"2024-03-08T20:57:51.793443+0000","last_peered":"2024-03-08T20:57:51.793443+0000","last_clean":"2024-03-08T20:57:51.793443+0000","last_became_active":"2024-03-08T20:57:49.788975+0000","last_became_peered":"2024-03-08T20:57:49.788975+0000","last_unstale":"2024-03-08T20:57:51.793443+0000","last_undegraded":"2024-03-08T20:57:51.793443+0000","last_fullsized":"2024-03-08T20:57:51.793443+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_clean_scrub_stamp":"2024-03-08T20:57:48.739829+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T07:42:21.366476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-08T20:57:52.073843+0000","last_change":"2024-03-08T20:57:50.785377+0000","last_active":"2024-03-08T20:57:52.073843+0000","last_peered":"2024-03-08T20:57:52.073843+0000","last_clean":"2024-03-08T20:57:52.073843+0000","last_became_active":"2024-03-08T20:57:50.785067+0000","last_became_peered":"2024-03-08T20:57:50.785067+0000","last_unstale":"2024-03-08T20:57:52.073843+0000","last_undegraded":"2024-03-08T20:57:52.073843+0000","last_fullsized":"2024-03-08T20:57:52.073843+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T02:49:10.388474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"86'4","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-08T20:57:53.842906+0000","last_change":"2024-03-08T20:57:49.784190+0000","last_active":"2024-03-08T20:57:53.842906+0000","last_peered":"2024-03-08T20:57:53.842906+0000","last_clean":"2024-03-08T20:57:53.842906+0000","last_became_active":"2024-03-08T20:57:49.783805+0000","last_became_peered":"2024-03-08T20:57:49.783805+0000","last_unstale":"2024-03-08T20:57:53.842906+0000","last_undegraded":"2024-03-08T20:57:53.842906+0000","last_fullsized":"2024-03-08T20:57:53.842906+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_clean_scrub_stamp":"2024-03-08T20:57:48.739829+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T03:16:12.983559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-08T20:57:51.794207+0000","last_change":"2024-03-08T20:57:50.795647+0000","last_active":"2024-03-08T20:57:51.794207+0000","last_peered":"2024-03-08T20:57:51.794207+0000","last_clean":"2024-03-08T20:57:51.794207+0000","last_became_active":"2024-03-08T20:57:50.793179+0000","last_became_peered":"2024-03-08T20:57:50.793179+0000","last_unstale":"2024-03-08T20:57:51.794207+0000","last_undegraded":"2024-03-08T20:57:51.794207+0000","last_fullsized":"2024-03-08T20:57:51.794207+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T05:56:14.051601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"86'4","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-08T20:57:56.894378+0000","last_change":"2024-03-08T20:57:49.795093+0000","last_active":"2024-03-08T20:57:56.894378+0000","last_peered":"2024-03-08T20:57:56.894378+0000","last_clean":"2024-03-08T20:57:56.894378+0000","last_became_active":"2024-03-08T20:57:49.794617+0000","last_became_peered":"2024-03-08T20:57:49.794617+0000","last_unstale":"2024-03-08T20:57:56.894378+0000","last_undegraded":"2024-03-08T20:57:56.894378+0000","last_fullsized":"2024-03-08T20:57:56.894378+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_clean_scrub_stamp":"2024-03-08T20:57:48.739829+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T05:56:38.669695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-08T20:57:52.073978+0000","last_change":"2024-03-08T20:57:50.806013+0000","last_active":"2024-03-08T20:57:52.073978+0000","last_peered":"2024-03-08T20:57:52.073978+0000","last_clean":"2024-03-08T20:57:52.073978+0000","last_became_active":"2024-03-08T20:57:50.805702+0000","last_became_peered":"2024-03-08T20:57:50.805702+0000","last_unstale":"2024-03-08T20:57:52.073978+0000","last_undegraded":"2024-03-08T20:57:52.073978+0000","last_fullsized":"2024-03-08T20:57:52.073978+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T03:27:37.118410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"86'4","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-08T20:57:51.794867+0000","last_change":"2024-03-08T20:57:49.776991+0000","last_active":"2024-03-08T20:57:51.794867+0000","last_peered":"2024-03-08T20:57:51.794867+0000","last_clean":"2024-03-08T20:57:51.794867+0000","last_became_active":"2024-03-08T20:57:49.776566+0000","last_became_peered":"2024-03-08T20:57:49.776566+0000","last_unstale":"2024-03-08T20:57:51.794867+0000","last_undegraded":"2024-03-08T20:57:51.794867+0000","last_fullsized":"2024-03-08T20:57:51.794867+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_clean_scrub_stamp":"2024-03-08T20:57:48.739829+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T21:02:37.726353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-08T20:57:51.794371+0000","last_change":"2024-03-08T20:57:50.793976+0000","last_active":"2024-03-08T20:57:51.794371+0000","last_peered":"2024-03-08T20:57:51.794371+0000","last_clean":"2024-03-08T20:57:51.794371+0000","last_became_active":"2024-03-08T20:57:50.792472+0000","last_became_peered":"2024-03-08T20:57:50.792472+0000","last_unstale":"2024-03-08T20:57:51.794371+0000","last_undegraded":"2024-03-08T20:57:51.794371+0000","last_fullsized":"2024-03-08T20:57:51.794371+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T06:01:03.360171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"86'17","reported_seq":32,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-08T20:57:57.417728+0000","last_change":"2024-03-08T20:57:49.791290+0000","last_active":"2024-03-08T20:57:57.417728+0000","last_peered":"2024-03-08T20:57:57.417728+0000","last_clean":"2024-03-08T20:57:57.417728+0000","last_became_active":"2024-03-08T20:57:49.790747+0000","last_became_peered":"2024-03-08T20:57:49.790747+0000","last_unstale":"2024-03-08T20:57:57.417728+0000","last_undegraded":"2024-03-08T20:57:57.417728+0000","last_fullsized":"2024-03-08T20:57:57.417728+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_clean_scrub_stamp":"2024-03-08T20:57:48.739829+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T02:49:12.984238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20895,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19,"num_write_kb":24,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-08T20:57:52.073954+0000","last_change":"2024-03-08T20:57:50.800031+0000","last_active":"2024-03-08T20:57:52.073954+0000","last_peered":"2024-03-08T20:57:52.073954+0000","last_clean":"2024-03-08T20:57:52.073954+0000","last_became_active":"2024-03-08T20:57:50.799663+0000","last_became_peered":"2024-03-08T20:57:50.799663+0000","last_unstale":"2024-03-08T20:57:52.073954+0000","last_undegraded":"2024-03-08T20:57:52.073954+0000","last_fullsized":"2024-03-08T20:57:52.073954+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T05:37:50.734330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"86'9","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-08T20:57:56.889292+0000","last_change":"2024-03-08T20:57:49.791884+0000","last_active":"2024-03-08T20:57:56.889292+0000","last_peered":"2024-03-08T20:57:56.889292+0000","last_clean":"2024-03-08T20:57:56.889292+0000","last_became_active":"2024-03-08T20:57:49.789314+0000","last_became_peered":"2024-03-08T20:57:49.789314+0000","last_unstale":"2024-03-08T20:57:56.889292+0000","last_undegraded":"2024-03-08T20:57:56.889292+0000","last_fullsized":"2024-03-08T20:57:56.889292+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_clean_scrub_stamp":"2024-03-08T20:57:48.739829+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T08:44:20.490703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-08T20:57:52.074277+0000","last_change":"2024-03-08T20:57:50.787609+0000","last_active":"2024-03-08T20:57:52.074277+0000","last_peered":"2024-03-08T20:57:52.074277+0000","last_clean":"2024-03-08T20:57:52.074277+0000","last_became_active":"2024-03-08T20:57:50.787171+0000","last_became_peered":"2024-03-08T20:57:50.787171+0000","last_unstale":"2024-03-08T20:57:52.074277+0000","last_undegraded":"2024-03-08T20:57:52.074277+0000","last_fullsized":"2024-03-08T20:57:52.074277+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T05:57:54.023942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"86'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-08T20:57:53.843588+0000","last_change":"2024-03-08T20:57:49.785642+0000","last_active":"2024-03-08T20:57:53.843588+0000","last_peered":"2024-03-08T20:57:53.843588+0000","last_clean":"2024-03-08T20:57:53.843588+0000","last_became_active":"2024-03-08T20:57:49.785180+0000","last_became_peered":"2024-03-08T20:57:49.785180+0000","last_unstale":"2024-03-08T20:57:53.843588+0000","last_undegraded":"2024-03-08T20:57:53.843588+0000","last_fullsized":"2024-03-08T20:57:53.843588+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_clean_scrub_stamp":"2024-03-08T20:57:48.739829+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T03:21:03.121253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-08T20:57:52.073757+0000","last_change":"2024-03-08T20:57:50.805179+0000","last_active":"2024-03-08T20:57:52.073757+0000","last_peered":"2024-03-08T20:57:52.073757+0000","last_clean":"2024-03-08T20:57:52.073757+0000","last_became_active":"2024-03-08T20:57:50.804801+0000","last_became_peered":"2024-03-08T20:57:50.804801+0000","last_unstale":"2024-03-08T20:57:52.073757+0000","last_undegraded":"2024-03-08T20:57:52.073757+0000","last_fullsized":"2024-03-08T20:57:52.073757+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T08:35:50.224508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"86'5","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-08T20:57:55.856902+0000","last_change":"2024-03-08T20:57:49.787198+0000","last_active":"2024-03-08T20:57:55.856902+0000","last_peered":"2024-03-08T20:57:55.856902+0000","last_clean":"2024-03-08T20:57:55.856902+0000","last_became_active":"2024-03-08T20:57:49.785966+0000","last_became_peered":"2024-03-08T20:57:49.785966+0000","last_unstale":"2024-03-08T20:57:55.856902+0000","last_undegraded":"2024-03-08T20:57:55.856902+0000","last_fullsized":"2024-03-08T20:57:55.856902+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_clean_scrub_stamp":"2024-03-08T20:57:48.739829+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T21:39:03.900857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-08T20:57:52.073164+0000","last_change":"2024-03-08T20:57:50.802887+0000","last_active":"2024-03-08T20:57:52.073164+0000","last_peered":"2024-03-08T20:57:52.073164+0000","last_clean":"2024-03-08T20:57:52.073164+0000","last_became_active":"2024-03-08T20:57:50.802530+0000","last_became_peered":"2024-03-08T20:57:50.802530+0000","last_unstale":"2024-03-08T20:57:52.073164+0000","last_undegraded":"2024-03-08T20:57:52.073164+0000","last_fullsized":"2024-03-08T20:57:52.073164+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T07:08:25.980812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"86'17","reported_seq":33,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-08T20:57:56.888941+0000","last_change":"2024-03-08T20:57:49.793528+0000","last_active":"2024-03-08T20:57:56.888941+0000","last_peered":"2024-03-08T20:57:56.888941+0000","last_clean":"2024-03-08T20:57:56.888941+0000","last_became_active":"2024-03-08T20:57:49.792509+0000","last_became_peered":"2024-03-08T20:57:49.792509+0000","last_unstale":"2024-03-08T20:57:56.888941+0000","last_undegraded":"2024-03-08T20:57:56.888941+0000","last_fullsized":"2024-03-08T20:57:56.888941+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_clean_scrub_stamp":"2024-03-08T20:57:48.739829+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T03:45:20.912280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-08T20:57:51.793333+0000","last_change":"2024-03-08T20:57:50.796843+0000","last_active":"2024-03-08T20:57:51.793333+0000","last_peered":"2024-03-08T20:57:51.793333+0000","last_clean":"2024-03-08T20:57:51.793333+0000","last_became_active":"2024-03-08T20:57:50.796232+0000","last_became_peered":"2024-03-08T20:57:50.796232+0000","last_unstale":"2024-03-08T20:57:51.793333+0000","last_undegraded":"2024-03-08T20:57:51.793333+0000","last_fullsized":"2024-03-08T20:57:51.793333+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T06:12:38.334736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"86'2","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-08T20:57:55.848274+0000","last_change":"2024-03-08T20:57:49.789376+0000","last_active":"2024-03-08T20:57:55.848274+0000","last_peered":"2024-03-08T20:57:55.848274+0000","last_clean":"2024-03-08T20:57:55.848274+0000","last_became_active":"2024-03-08T20:57:49.785791+0000","last_became_peered":"2024-03-08T20:57:49.785791+0000","last_unstale":"2024-03-08T20:57:55.848274+0000","last_undegraded":"2024-03-08T20:57:55.848274+0000","last_fullsized":"2024-03-08T20:57:55.848274+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_clean_scrub_stamp":"2024-03-08T20:57:48.739829+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T08:52:02.675985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-08T20:57:51.805054+0000","last_change":"2024-03-08T20:57:50.814625+0000","last_active":"2024-03-08T20:57:51.805054+0000","last_peered":"2024-03-08T20:57:51.805054+0000","last_clean":"2024-03-08T20:57:51.805054+0000","last_became_active":"2024-03-08T20:57:50.814220+0000","last_became_peered":"2024-03-08T20:57:50.814220+0000","last_unstale":"2024-03-08T20:57:51.805054+0000","last_undegraded":"2024-03-08T20:57:51.805054+0000","last_fullsized":"2024-03-08T20:57:51.805054+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T08:35:57.659511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"86'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-08T20:57:52.074373+0000","last_change":"2024-03-08T20:57:49.777922+0000","last_active":"2024-03-08T20:57:52.074373+0000","last_peered":"2024-03-08T20:57:52.074373+0000","last_clean":"2024-03-08T20:57:52.074373+0000","last_became_active":"2024-03-08T20:57:49.775083+0000","last_became_peered":"2024-03-08T20:57:49.775083+0000","last_unstale":"2024-03-08T20:57:52.074373+0000","last_undegraded":"2024-03-08T20:57:52.074373+0000","last_fullsized":"2024-03-08T20:57:52.074373+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_clean_scrub_stamp":"2024-03-08T20:57:48.739829+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T01:55:02.679999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"86'1","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-08T20:57:52.686908+0000","last_change":"2024-03-08T20:57:50.778774+0000","last_active":"2024-03-08T20:57:52.686908+0000","last_peered":"2024-03-08T20:57:52.686908+0000","last_clean":"2024-03-08T20:57:52.686908+0000","last_became_active":"2024-03-08T20:57:50.778194+0000","last_became_peered":"2024-03-08T20:57:50.778194+0000","last_unstale":"2024-03-08T20:57:52.686908+0000","last_undegraded":"2024-03-08T20:57:52.686908+0000","last_fullsized":"2024-03-08T20:57:52.686908+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T21:41:56.321304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"86'5","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-08T20:57:57.895945+0000","last_change":"2024-03-08T20:57:49.787818+0000","last_active":"2024-03-08T20:57:57.895945+0000","last_peered":"2024-03-08T20:57:57.895945+0000","last_clean":"2024-03-08T20:57:57.895945+0000","last_became_active":"2024-03-08T20:57:49.787401+0000","last_became_peered":"2024-03-08T20:57:49.787401+0000","last_unstale":"2024-03-08T20:57:57.895945+0000","last_undegraded":"2024-03-08T20:57:57.895945+0000","last_fullsized":"2024-03-08T20:57:57.895945+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_clean_scrub_stamp":"2024-03-08T20:57:48.739829+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T08:23:19.683962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-08T20:57:52.075297+0000","last_change":"2024-03-08T20:57:50.790922+0000","last_active":"2024-03-08T20:57:52.075297+0000","last_peered":"2024-03-08T20:57:52.075297+0000","last_clean":"2024-03-08T20:57:52.075297+0000","last_became_active":"2024-03-08T20:57:50.789993+0000","last_became_peered":"2024-03-08T20:57:50.789993+0000","last_unstale":"2024-03-08T20:57:52.075297+0000","last_undegraded":"2024-03-08T20:57:52.075297+0000","last_fullsized":"2024-03-08T20:57:52.075297+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T00:27:01.639941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-08T20:57:51.793549+0000","last_change":"2024-03-08T20:57:50.800505+0000","last_active":"2024-03-08T20:57:51.793549+0000","last_peered":"2024-03-08T20:57:51.793549+0000","last_clean":"2024-03-08T20:57:51.793549+0000","last_became_active":"2024-03-08T20:57:50.797459+0000","last_became_peered":"2024-03-08T20:57:50.797459+0000","last_unstale":"2024-03-08T20:57:51.793549+0000","last_undegraded":"2024-03-08T20:57:51.793549+0000","last_fullsized":"2024-03-08T20:57:51.793549+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T21:24:10.734888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"86'4","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-08T20:57:54.860484+0000","last_change":"2024-03-08T20:57:49.779056+0000","last_active":"2024-03-08T20:57:54.860484+0000","last_peered":"2024-03-08T20:57:54.860484+0000","last_clean":"2024-03-08T20:57:54.860484+0000","last_became_active":"2024-03-08T20:57:49.778714+0000","last_became_peered":"2024-03-08T20:57:49.778714+0000","last_unstale":"2024-03-08T20:57:54.860484+0000","last_undegraded":"2024-03-08T20:57:54.860484+0000","last_fullsized":"2024-03-08T20:57:54.860484+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_clean_scrub_stamp":"2024-03-08T20:57:48.739829+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T04:05:10.685148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-08T20:57:51.793317+0000","last_change":"2024-03-08T20:57:50.812509+0000","last_active":"2024-03-08T20:57:51.793317+0000","last_peered":"2024-03-08T20:57:51.793317+0000","last_clean":"2024-03-08T20:57:51.793317+0000","last_became_active":"2024-03-08T20:57:50.812146+0000","last_became_peered":"2024-03-08T20:57:50.812146+0000","last_unstale":"2024-03-08T20:57:51.793317+0000","last_undegraded":"2024-03-08T20:57:51.793317+0000","last_fullsized":"2024-03-08T20:57:51.793317+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T06:08:07.680098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'5","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-08T20:57:55.857423+0000","last_change":"2024-03-08T20:57:49.774994+0000","last_active":"2024-03-08T20:57:55.857423+0000","last_peered":"2024-03-08T20:57:55.857423+0000","last_clean":"2024-03-08T20:57:55.857423+0000","last_became_active":"2024-03-08T20:57:49.774501+0000","last_became_peered":"2024-03-08T20:57:49.774501+0000","last_unstale":"2024-03-08T20:57:55.857423+0000","last_undegraded":"2024-03-08T20:57:55.857423+0000","last_fullsized":"2024-03-08T20:57:55.857423+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_clean_scrub_stamp":"2024-03-08T20:57:48.739829+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T03:38:35.294082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":347,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-08T20:57:52.073201+0000","last_change":"2024-03-08T20:56:11.724397+0000","last_active":"2024-03-08T20:57:52.073201+0000","last_peered":"2024-03-08T20:57:52.073201+0000","last_clean":"2024-03-08T20:57:52.073201+0000","last_became_active":"2024-03-08T20:56:11.723212+0000","last_became_peered":"2024-03-08T20:56:11.723212+0000","last_unstale":"2024-03-08T20:57:52.073201+0000","last_undegraded":"2024-03-08T20:57:52.073201+0000","last_fullsized":"2024-03-08T20:57:52.073201+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:51:32.833935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:51:32.833935+0000","last_clean_scrub_stamp":"2024-03-08T20:51:32.833935+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T03:31:48.964514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-08T20:57:52.072747+0000","last_change":"2024-03-08T20:57:50.792894+0000","last_active":"2024-03-08T20:57:52.072747+0000","last_peered":"2024-03-08T20:57:52.072747+0000","last_clean":"2024-03-08T20:57:52.072747+0000","last_became_active":"2024-03-08T20:57:50.792471+0000","last_became_peered":"2024-03-08T20:57:50.792471+0000","last_unstale":"2024-03-08T20:57:52.072747+0000","last_undegraded":"2024-03-08T20:57:52.072747+0000","last_fullsized":"2024-03-08T20:57:52.072747+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T00:45:39.696601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'4","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-08T20:57:54.855670+0000","last_change":"2024-03-08T20:57:49.787880+0000","last_active":"2024-03-08T20:57:54.855670+0000","last_peered":"2024-03-08T20:57:54.855670+0000","last_clean":"2024-03-08T20:57:54.855670+0000","last_became_active":"2024-03-08T20:57:49.787450+0000","last_became_peered":"2024-03-08T20:57:49.787450+0000","last_unstale":"2024-03-08T20:57:54.855670+0000","last_undegraded":"2024-03-08T20:57:54.855670+0000","last_fullsized":"2024-03-08T20:57:54.855670+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_clean_scrub_stamp":"2024-03-08T20:57:48.739829+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T04:52:36.895259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"86'4","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-08T20:57:56.896666+0000","last_change":"2024-03-08T20:57:49.787473+0000","last_active":"2024-03-08T20:57:56.896666+0000","last_peered":"2024-03-08T20:57:56.896666+0000","last_clean":"2024-03-08T20:57:56.896666+0000","last_became_active":"2024-03-08T20:57:49.787145+0000","last_became_peered":"2024-03-08T20:57:49.787145+0000","last_unstale":"2024-03-08T20:57:56.896666+0000","last_undegraded":"2024-03-08T20:57:56.896666+0000","last_fullsized":"2024-03-08T20:57:56.896666+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_clean_scrub_stamp":"2024-03-08T20:57:48.739829+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T05:47:14.471850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-08T20:57:52.073056+0000","last_change":"2024-03-08T20:57:50.789546+0000","last_active":"2024-03-08T20:57:52.073056+0000","last_peered":"2024-03-08T20:57:52.073056+0000","last_clean":"2024-03-08T20:57:52.073056+0000","last_became_active":"2024-03-08T20:57:50.789195+0000","last_became_peered":"2024-03-08T20:57:50.789195+0000","last_unstale":"2024-03-08T20:57:52.073056+0000","last_undegraded":"2024-03-08T20:57:52.073056+0000","last_fullsized":"2024-03-08T20:57:52.073056+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T02:43:31.371303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"86'10","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-08T20:57:58.774929+0000","last_change":"2024-03-08T20:57:49.787907+0000","last_active":"2024-03-08T20:57:58.774929+0000","last_peered":"2024-03-08T20:57:58.774929+0000","last_clean":"2024-03-08T20:57:58.774929+0000","last_became_active":"2024-03-08T20:57:49.787601+0000","last_became_peered":"2024-03-08T20:57:49.787601+0000","last_unstale":"2024-03-08T20:57:58.774929+0000","last_undegraded":"2024-03-08T20:57:58.774929+0000","last_fullsized":"2024-03-08T20:57:58.774929+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_clean_scrub_stamp":"2024-03-08T20:57:48.739829+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T00:52:09.562020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-08T20:57:51.793583+0000","last_change":"2024-03-08T20:57:50.798773+0000","last_active":"2024-03-08T20:57:51.793583+0000","last_peered":"2024-03-08T20:57:51.793583+0000","last_clean":"2024-03-08T20:57:51.793583+0000","last_became_active":"2024-03-08T20:57:50.798396+0000","last_became_peered":"2024-03-08T20:57:50.798396+0000","last_unstale":"2024-03-08T20:57:51.793583+0000","last_undegraded":"2024-03-08T20:57:51.793583+0000","last_fullsized":"2024-03-08T20:57:51.793583+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T00:26:41.614622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"86'5","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-08T20:57:53.832726+0000","last_change":"2024-03-08T20:57:49.777474+0000","last_active":"2024-03-08T20:57:53.832726+0000","last_peered":"2024-03-08T20:57:53.832726+0000","last_clean":"2024-03-08T20:57:53.832726+0000","last_became_active":"2024-03-08T20:57:49.777084+0000","last_became_peered":"2024-03-08T20:57:49.777084+0000","last_unstale":"2024-03-08T20:57:53.832726+0000","last_undegraded":"2024-03-08T20:57:53.832726+0000","last_fullsized":"2024-03-08T20:57:53.832726+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_clean_scrub_stamp":"2024-03-08T20:57:48.739829+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T05:10:23.032788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-08T20:57:52.073940+0000","last_change":"2024-03-08T20:57:50.805061+0000","last_active":"2024-03-08T20:57:52.073940+0000","last_peered":"2024-03-08T20:57:52.073940+0000","last_clean":"2024-03-08T20:57:52.073940+0000","last_became_active":"2024-03-08T20:57:50.804709+0000","last_became_peered":"2024-03-08T20:57:50.804709+0000","last_unstale":"2024-03-08T20:57:52.073940+0000","last_undegraded":"2024-03-08T20:57:52.073940+0000","last_fullsized":"2024-03-08T20:57:52.073940+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T08:25:34.003449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-08T20:57:53.841513+0000","last_change":"2024-03-08T20:57:49.776677+0000","last_active":"2024-03-08T20:57:53.841513+0000","last_peered":"2024-03-08T20:57:53.841513+0000","last_clean":"2024-03-08T20:57:53.841513+0000","last_became_active":"2024-03-08T20:57:49.776296+0000","last_became_peered":"2024-03-08T20:57:49.776296+0000","last_unstale":"2024-03-08T20:57:53.841513+0000","last_undegraded":"2024-03-08T20:57:53.841513+0000","last_fullsized":"2024-03-08T20:57:53.841513+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_clean_scrub_stamp":"2024-03-08T20:57:48.739829+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T07:49:46.042037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-08T20:57:52.073666+0000","last_change":"2024-03-08T20:57:50.806141+0000","last_active":"2024-03-08T20:57:52.073666+0000","last_peered":"2024-03-08T20:57:52.073666+0000","last_clean":"2024-03-08T20:57:52.073666+0000","last_became_active":"2024-03-08T20:57:50.805026+0000","last_became_peered":"2024-03-08T20:57:50.805026+0000","last_unstale":"2024-03-08T20:57:52.073666+0000","last_undegraded":"2024-03-08T20:57:52.073666+0000","last_fullsized":"2024-03-08T20:57:52.073666+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T03:49:04.162888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"86'7","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-08T20:57:56.896343+0000","last_change":"2024-03-08T20:57:49.779675+0000","last_active":"2024-03-08T20:57:56.896343+0000","last_peered":"2024-03-08T20:57:56.896343+0000","last_clean":"2024-03-08T20:57:56.896343+0000","last_became_active":"2024-03-08T20:57:49.774270+0000","last_became_peered":"2024-03-08T20:57:49.774270+0000","last_unstale":"2024-03-08T20:57:56.896343+0000","last_undegraded":"2024-03-08T20:57:56.896343+0000","last_fullsized":"2024-03-08T20:57:56.896343+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_clean_scrub_stamp":"2024-03-08T20:57:48.739829+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T02:19:09.992649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-08T20:57:51.804720+0000","last_change":"2024-03-08T20:57:50.797461+0000","last_active":"2024-03-08T20:57:51.804720+0000","last_peered":"2024-03-08T20:57:51.804720+0000","last_clean":"2024-03-08T20:57:51.804720+0000","last_became_active":"2024-03-08T20:57:50.796928+0000","last_became_peered":"2024-03-08T20:57:50.796928+0000","last_unstale":"2024-03-08T20:57:51.804720+0000","last_undegraded":"2024-03-08T20:57:51.804720+0000","last_fullsized":"2024-03-08T20:57:51.804720+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T04:01:44.009461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"86'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-08T20:57:54.830216+0000","last_change":"2024-03-08T20:57:49.794659+0000","last_active":"2024-03-08T20:57:54.830216+0000","last_peered":"2024-03-08T20:57:54.830216+0000","last_clean":"2024-03-08T20:57:54.830216+0000","last_became_active":"2024-03-08T20:57:49.792426+0000","last_became_peered":"2024-03-08T20:57:49.792426+0000","last_unstale":"2024-03-08T20:57:54.830216+0000","last_undegraded":"2024-03-08T20:57:54.830216+0000","last_fullsized":"2024-03-08T20:57:54.830216+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_clean_scrub_stamp":"2024-03-08T20:57:48.739829+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T01:30:41.123131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-08T20:57:51.792840+0000","last_change":"2024-03-08T20:57:50.798691+0000","last_active":"2024-03-08T20:57:51.792840+0000","last_peered":"2024-03-08T20:57:51.792840+0000","last_clean":"2024-03-08T20:57:51.792840+0000","last_became_active":"2024-03-08T20:57:50.797876+0000","last_became_peered":"2024-03-08T20:57:50.797876+0000","last_unstale":"2024-03-08T20:57:51.792840+0000","last_undegraded":"2024-03-08T20:57:51.792840+0000","last_fullsized":"2024-03-08T20:57:51.792840+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T06:51:11.067375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-08T20:57:53.841990+0000","last_change":"2024-03-08T20:57:49.794648+0000","last_active":"2024-03-08T20:57:53.841990+0000","last_peered":"2024-03-08T20:57:53.841990+0000","last_clean":"2024-03-08T20:57:53.841990+0000","last_became_active":"2024-03-08T20:57:49.793316+0000","last_became_peered":"2024-03-08T20:57:49.793316+0000","last_unstale":"2024-03-08T20:57:53.841990+0000","last_undegraded":"2024-03-08T20:57:53.841990+0000","last_fullsized":"2024-03-08T20:57:53.841990+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_clean_scrub_stamp":"2024-03-08T20:57:48.739829+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T02:16:26.006212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-08T20:57:52.073761+0000","last_change":"2024-03-08T20:57:50.797850+0000","last_active":"2024-03-08T20:57:52.073761+0000","last_peered":"2024-03-08T20:57:52.073761+0000","last_clean":"2024-03-08T20:57:52.073761+0000","last_became_active":"2024-03-08T20:57:50.797530+0000","last_became_peered":"2024-03-08T20:57:50.797530+0000","last_unstale":"2024-03-08T20:57:52.073761+0000","last_undegraded":"2024-03-08T20:57:52.073761+0000","last_fullsized":"2024-03-08T20:57:52.073761+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T02:12:52.023644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"86'5","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-08T20:57:55.861183+0000","last_change":"2024-03-08T20:57:49.784671+0000","last_active":"2024-03-08T20:57:55.861183+0000","last_peered":"2024-03-08T20:57:55.861183+0000","last_clean":"2024-03-08T20:57:55.861183+0000","last_became_active":"2024-03-08T20:57:49.783870+0000","last_became_peered":"2024-03-08T20:57:49.783870+0000","last_unstale":"2024-03-08T20:57:55.861183+0000","last_undegraded":"2024-03-08T20:57:55.861183+0000","last_fullsized":"2024-03-08T20:57:55.861183+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_clean_scrub_stamp":"2024-03-08T20:57:48.739829+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T02:29:16.769221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-08T20:57:52.075006+0000","last_change":"2024-03-08T20:57:50.796522+0000","last_active":"2024-03-08T20:57:52.075006+0000","last_peered":"2024-03-08T20:57:52.075006+0000","last_clean":"2024-03-08T20:57:52.075006+0000","last_became_active":"2024-03-08T20:57:50.796061+0000","last_became_peered":"2024-03-08T20:57:50.796061+0000","last_unstale":"2024-03-08T20:57:52.075006+0000","last_undegraded":"2024-03-08T20:57:52.075006+0000","last_fullsized":"2024-03-08T20:57:52.075006+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T21:35:06.609042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-08T20:57:51.794556+0000","last_change":"2024-03-08T20:57:49.778240+0000","last_active":"2024-03-08T20:57:51.794556+0000","last_peered":"2024-03-08T20:57:51.794556+0000","last_clean":"2024-03-08T20:57:51.794556+0000","last_became_active":"2024-03-08T20:57:49.777798+0000","last_became_peered":"2024-03-08T20:57:49.777798+0000","last_unstale":"2024-03-08T20:57:51.794556+0000","last_undegraded":"2024-03-08T20:57:51.794556+0000","last_fullsized":"2024-03-08T20:57:51.794556+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_clean_scrub_stamp":"2024-03-08T20:57:48.739829+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T05:20:24.614659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"86'1","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-08T20:57:53.972947+0000","last_change":"2024-03-08T20:57:50.803435+0000","last_active":"2024-03-08T20:57:53.972947+0000","last_peered":"2024-03-08T20:57:53.972947+0000","last_clean":"2024-03-08T20:57:53.972947+0000","last_became_active":"2024-03-08T20:57:50.803074+0000","last_became_peered":"2024-03-08T20:57:50.803074+0000","last_unstale":"2024-03-08T20:57:53.972947+0000","last_undegraded":"2024-03-08T20:57:53.972947+0000","last_fullsized":"2024-03-08T20:57:53.972947+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T21:52:02.791159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-08T20:57:52.074495+0000","last_change":"2024-03-08T20:57:49.781604+0000","last_active":"2024-03-08T20:57:52.074495+0000","last_peered":"2024-03-08T20:57:52.074495+0000","last_clean":"2024-03-08T20:57:52.074495+0000","last_became_active":"2024-03-08T20:57:49.781274+0000","last_became_peered":"2024-03-08T20:57:49.781274+0000","last_unstale":"2024-03-08T20:57:52.074495+0000","last_undegraded":"2024-03-08T20:57:52.074495+0000","last_fullsized":"2024-03-08T20:57:52.074495+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_clean_scrub_stamp":"2024-03-08T20:57:48.739829+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T22:29:03.420779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-08T20:57:52.074143+0000","last_change":"2024-03-08T20:57:50.800863+0000","last_active":"2024-03-08T20:57:52.074143+0000","last_peered":"2024-03-08T20:57:52.074143+0000","last_clean":"2024-03-08T20:57:52.074143+0000","last_became_active":"2024-03-08T20:57:50.798916+0000","last_became_peered":"2024-03-08T20:57:50.798916+0000","last_unstale":"2024-03-08T20:57:52.074143+0000","last_undegraded":"2024-03-08T20:57:52.074143+0000","last_fullsized":"2024-03-08T20:57:52.074143+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T04:37:06.666245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-08T20:57:51.804884+0000","last_change":"2024-03-08T20:57:50.813502+0000","last_active":"2024-03-08T20:57:51.804884+0000","last_peered":"2024-03-08T20:57:51.804884+0000","last_clean":"2024-03-08T20:57:51.804884+0000","last_became_active":"2024-03-08T20:57:50.813150+0000","last_became_peered":"2024-03-08T20:57:50.813150+0000","last_unstale":"2024-03-08T20:57:51.804884+0000","last_undegraded":"2024-03-08T20:57:51.804884+0000","last_fullsized":"2024-03-08T20:57:51.804884+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T00:01:47.902909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-08T20:57:52.072901+0000","last_change":"2024-03-08T20:57:50.795055+0000","last_active":"2024-03-08T20:57:52.072901+0000","last_peered":"2024-03-08T20:57:52.072901+0000","last_clean":"2024-03-08T20:57:52.072901+0000","last_became_active":"2024-03-08T20:57:50.794559+0000","last_became_peered":"2024-03-08T20:57:50.794559+0000","last_unstale":"2024-03-08T20:57:52.072901+0000","last_undegraded":"2024-03-08T20:57:52.072901+0000","last_fullsized":"2024-03-08T20:57:52.072901+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T08:06:18.990542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-08T20:57:52.072961+0000","last_change":"2024-03-08T20:57:50.784871+0000","last_active":"2024-03-08T20:57:52.072961+0000","last_peered":"2024-03-08T20:57:52.072961+0000","last_clean":"2024-03-08T20:57:52.072961+0000","last_became_active":"2024-03-08T20:57:50.784553+0000","last_became_peered":"2024-03-08T20:57:50.784553+0000","last_unstale":"2024-03-08T20:57:52.072961+0000","last_undegraded":"2024-03-08T20:57:52.072961+0000","last_fullsized":"2024-03-08T20:57:52.072961+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T02:13:13.570078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-08T20:57:52.073692+0000","last_change":"2024-03-08T20:57:50.794051+0000","last_active":"2024-03-08T20:57:52.073692+0000","last_peered":"2024-03-08T20:57:52.073692+0000","last_clean":"2024-03-08T20:57:52.073692+0000","last_became_active":"2024-03-08T20:57:50.793547+0000","last_became_peered":"2024-03-08T20:57:50.793547+0000","last_unstale":"2024-03-08T20:57:52.073692+0000","last_undegraded":"2024-03-08T20:57:52.073692+0000","last_fullsized":"2024-03-08T20:57:52.073692+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T04:13:52.705247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-08T20:57:52.073281+0000","last_change":"2024-03-08T20:57:50.789449+0000","last_active":"2024-03-08T20:57:52.073281+0000","last_peered":"2024-03-08T20:57:52.073281+0000","last_clean":"2024-03-08T20:57:52.073281+0000","last_became_active":"2024-03-08T20:57:50.789106+0000","last_became_peered":"2024-03-08T20:57:50.789106+0000","last_unstale":"2024-03-08T20:57:52.073281+0000","last_undegraded":"2024-03-08T20:57:52.073281+0000","last_fullsized":"2024-03-08T20:57:52.073281+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T02:12:47.198726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-08T20:57:51.804816+0000","last_change":"2024-03-08T20:57:50.814360+0000","last_active":"2024-03-08T20:57:51.804816+0000","last_peered":"2024-03-08T20:57:51.804816+0000","last_clean":"2024-03-08T20:57:51.804816+0000","last_became_active":"2024-03-08T20:57:50.813966+0000","last_became_peered":"2024-03-08T20:57:50.813966+0000","last_unstale":"2024-03-08T20:57:51.804816+0000","last_undegraded":"2024-03-08T20:57:51.804816+0000","last_fullsized":"2024-03-08T20:57:51.804816+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T04:39:10.324938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-08T20:57:52.073448+0000","last_change":"2024-03-08T20:57:50.803658+0000","last_active":"2024-03-08T20:57:52.073448+0000","last_peered":"2024-03-08T20:57:52.073448+0000","last_clean":"2024-03-08T20:57:52.073448+0000","last_became_active":"2024-03-08T20:57:50.797593+0000","last_became_peered":"2024-03-08T20:57:50.797593+0000","last_unstale":"2024-03-08T20:57:52.073448+0000","last_undegraded":"2024-03-08T20:57:52.073448+0000","last_fullsized":"2024-03-08T20:57:52.073448+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T21:45:33.579087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-08T20:57:51.793416+0000","last_change":"2024-03-08T20:57:50.789913+0000","last_active":"2024-03-08T20:57:51.793416+0000","last_peered":"2024-03-08T20:57:51.793416+0000","last_clean":"2024-03-08T20:57:51.793416+0000","last_became_active":"2024-03-08T20:57:50.789492+0000","last_became_peered":"2024-03-08T20:57:50.789492+0000","last_unstale":"2024-03-08T20:57:51.793416+0000","last_undegraded":"2024-03-08T20:57:51.793416+0000","last_fullsized":"2024-03-08T20:57:51.793416+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T01:25:14.329005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-08T20:57:51.793346+0000","last_change":"2024-03-08T20:57:50.801743+0000","last_active":"2024-03-08T20:57:51.793346+0000","last_peered":"2024-03-08T20:57:51.793346+0000","last_clean":"2024-03-08T20:57:51.793346+0000","last_became_active":"2024-03-08T20:57:50.801261+0000","last_became_peered":"2024-03-08T20:57:50.801261+0000","last_unstale":"2024-03-08T20:57:51.793346+0000","last_undegraded":"2024-03-08T20:57:51.793346+0000","last_fullsized":"2024-03-08T20:57:51.793346+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T23:00:15.605070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-08T20:57:51.792139+0000","last_change":"2024-03-08T20:57:50.800759+0000","last_active":"2024-03-08T20:57:51.792139+0000","last_peered":"2024-03-08T20:57:51.792139+0000","last_clean":"2024-03-08T20:57:51.792139+0000","last_became_active":"2024-03-08T20:57:50.798328+0000","last_became_peered":"2024-03-08T20:57:50.798328+0000","last_unstale":"2024-03-08T20:57:51.792139+0000","last_undegraded":"2024-03-08T20:57:51.792139+0000","last_fullsized":"2024-03-08T20:57:51.792139+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T05:43:00.160071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-08T20:57:51.794830+0000","last_change":"2024-03-08T20:57:50.802262+0000","last_active":"2024-03-08T20:57:51.794830+0000","last_peered":"2024-03-08T20:57:51.794830+0000","last_clean":"2024-03-08T20:57:51.794830+0000","last_became_active":"2024-03-08T20:57:50.801786+0000","last_became_peered":"2024-03-08T20:57:50.801786+0000","last_unstale":"2024-03-08T20:57:51.794830+0000","last_undegraded":"2024-03-08T20:57:51.794830+0000","last_fullsized":"2024-03-08T20:57:51.794830+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T00:47:48.400329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-08T20:57:52.073201+0000","last_change":"2024-03-08T20:57:50.790819+0000","last_active":"2024-03-08T20:57:52.073201+0000","last_peered":"2024-03-08T20:57:52.073201+0000","last_clean":"2024-03-08T20:57:52.073201+0000","last_became_active":"2024-03-08T20:57:50.786840+0000","last_became_peered":"2024-03-08T20:57:50.786840+0000","last_unstale":"2024-03-08T20:57:52.073201+0000","last_undegraded":"2024-03-08T20:57:52.073201+0000","last_fullsized":"2024-03-08T20:57:52.073201+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T06:52:19.567245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-08T20:57:51.804858+0000","last_change":"2024-03-08T20:57:50.806693+0000","last_active":"2024-03-08T20:57:51.804858+0000","last_peered":"2024-03-08T20:57:51.804858+0000","last_clean":"2024-03-08T20:57:51.804858+0000","last_became_active":"2024-03-08T20:57:50.806299+0000","last_became_peered":"2024-03-08T20:57:50.806299+0000","last_unstale":"2024-03-08T20:57:51.804858+0000","last_undegraded":"2024-03-08T20:57:51.804858+0000","last_fullsized":"2024-03-08T20:57:51.804858+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T02:04:07.567265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":351,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":280946,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":26589,"num_objects":88,"num_object_clones":0,"num_object_copies":264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":88,"num_whiteouts":0,"num_read":14,"num_read_kb":12,"num_write":202,"num_write_kb":70,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":60,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":380928,"data_stored":58906,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":173,"ondisk_log_size":173,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892350993,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32092,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713060,"statfs":{"total":95995035648,"available":95962173440,"internally_reserved":0,"allocated":937984,"data_stored":444207,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612632,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32188,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712964,"statfs":{"total":95995035648,"available":95962075136,"internally_reserved":0,"allocated":1036288,"data_stored":1043331,"data_compressed":35734,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1599,"internal_metadata":31914433},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776159,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32116,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713036,"statfs":{"total":95995035648,"available":95962148864,"internally_reserved":0,"allocated":962560,"data_stored":444137,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037798,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":958464,"data_stored":444515,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299436,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1011712,"data_stored":1036642,"data_compressed":35734,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462963,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32080,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713072,"statfs":{"total":95995035648,"available":95962185728,"internally_reserved":0,"allocated":925696,"data_stored":443015,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574933,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":937984,"data_stored":444627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738461,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32120,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713032,"statfs":{"total":95995035648,"available":95962144768,"internally_reserved":0,"allocated":966656,"data_stored":445193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1],"upper_bound":8},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411408,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":983040,"data_stored":459755,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182472,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32112,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713040,"statfs":{"total":95995035648,"available":95962152960,"internally_reserved":0,"allocated":958464,"data_stored":443562,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920834,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32096,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713056,"statfs":{"total":95995035648,"available":95962169344,"internally_reserved":0,"allocated":942080,"data_stored":443681,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724603,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32156,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712996,"statfs":{"total":95995035648,"available":95962107904,"internally_reserved":0,"allocated":1003520,"data_stored":1034859,"data_compressed":35734,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34092,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34092,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34092,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35734,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35734,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35734,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35734,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35734,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":22867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":666,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":22631,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-08T20:58:00.858 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-03-08T20:58:00.859 INFO:tasks.cephadm:Setting up client nodes... 2024-03-08T20:58:00.859 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dc20f0f8-dd8c-11ee-95c6-87774f69a715 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-03-08T20:58:01.217 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:58:00 smithi107 ceph-mon[31381]: pgmap v366: 97 pgs: 97 active+clean; 603 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 7.5 KiB/s wr, 22 op/s 2024-03-08T20:58:01.218 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:58:00 smithi107 ceph-mon[31381]: from='client.14781 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-08T20:58:01.266 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:58:00 smithi149 ceph-mon[35042]: pgmap v366: 97 pgs: 97 active+clean; 603 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 7.5 KiB/s wr, 22 op/s 2024-03-08T20:58:01.266 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:58:00 smithi149 ceph-mon[35042]: from='client.14781 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-08T20:58:01.344 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/mon.a/config 2024-03-08T20:58:01.379 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:58:00 smithi204 ceph-mon[34935]: pgmap v366: 97 pgs: 97 active+clean; 603 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 7.5 KiB/s wr, 22 op/s 2024-03-08T20:58:01.379 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:58:00 smithi204 ceph-mon[34935]: from='client.14781 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-08T20:58:02.218 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:58:01 smithi107 ceph-mon[31381]: from='client.14787 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-08T20:58:02.265 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:58:01 smithi149 ceph-mon[35042]: from='client.14787 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-08T20:58:02.379 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:58:01 smithi204 ceph-mon[34935]: from='client.14787 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-08T20:58:03.139 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:58:02 smithi107 ceph-mon[31381]: pgmap v367: 97 pgs: 97 active+clean; 603 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 8.4 KiB/s wr, 24 op/s 2024-03-08T20:58:03.265 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:58:02 smithi149 ceph-mon[35042]: pgmap v367: 97 pgs: 97 active+clean; 603 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 8.4 KiB/s wr, 24 op/s 2024-03-08T20:58:03.281 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:58:02 smithi204 ceph-mon[34935]: pgmap v367: 97 pgs: 97 active+clean; 603 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 8.4 KiB/s wr, 24 op/s 2024-03-08T20:58:03.677 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:03.692+0000 7f8e74e00700 1 -- 172.21.15.107:0/2114876917 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8e70109480 msgr2=0x7f8e701098e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:58:03.677 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:03.692+0000 7f8e74e00700 1 --2- 172.21.15.107:0/2114876917 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8e70109480 0x7f8e701098e0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f8e5c0099c0 tx=0x7f8e5c009cd0 comp rx=0 tx=0).stop 2024-03-08T20:58:03.677 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:03.692+0000 7f8e74e00700 1 -- 172.21.15.107:0/2114876917 shutdown_connections 2024-03-08T20:58:03.677 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:03.692+0000 7f8e74e00700 1 --2- 172.21.15.107:0/2114876917 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8e70109e20 0x7f8e701106d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:58:03.677 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:03.692+0000 7f8e74e00700 1 --2- 172.21.15.107:0/2114876917 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8e70108280 0x7f8e701086a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:58:03.677 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:03.692+0000 7f8e74e00700 1 --2- 172.21.15.107:0/2114876917 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8e70109480 0x7f8e701098e0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:58:03.677 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:03.692+0000 7f8e74e00700 1 -- 172.21.15.107:0/2114876917 >> 172.21.15.107:0/2114876917 conn(0x7f8e70077250 msgr2=0x7f8e70079690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:58:03.678 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:03.692+0000 7f8e74e00700 1 -- 172.21.15.107:0/2114876917 shutdown_connections 2024-03-08T20:58:03.678 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:03.692+0000 7f8e74e00700 1 -- 172.21.15.107:0/2114876917 wait complete. 2024-03-08T20:58:03.679 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:03.694+0000 7f8e74e00700 1 Processor -- start 2024-03-08T20:58:03.679 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:03.694+0000 7f8e74e00700 1 -- start start 2024-03-08T20:58:03.679 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:03.694+0000 7f8e74e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8e70108280 0x7f8e7019e270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:58:03.679 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:03.694+0000 7f8e74e00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8e70109480 0x7f8e7019e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:58:03.679 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:03.694+0000 7f8e74e00700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8e70109e20 0x7f8e701a5830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:58:03.679 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:03.694+0000 7f8e74e00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e70113d50 con 0x7f8e70108280 2024-03-08T20:58:03.679 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:03.694+0000 7f8e74e00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e70113bd0 con 0x7f8e70109480 2024-03-08T20:58:03.680 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:03.694+0000 7f8e74e00700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e70113ed0 con 0x7f8e70109e20 2024-03-08T20:58:03.680 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:03.695+0000 7f8e6f400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8e70108280 0x7f8e7019e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:58:03.680 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:03.695+0000 7f8e6f400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8e70108280 0x7f8e7019e270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:57292/0 (socket says 172.21.15.107:57292) 2024-03-08T20:58:03.680 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:03.695+0000 7f8e6f400700 1 -- 172.21.15.107:0/1490673712 learned_addr learned my addr 172.21.15.107:0/1490673712 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T20:58:03.682 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:03.695+0000 7f8e6ea00700 1 --2- 172.21.15.107:0/1490673712 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8e70109480 0x7f8e7019e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:58:03.682 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:03.695+0000 7f8e6fe00700 1 --2- 172.21.15.107:0/1490673712 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8e70109e20 0x7f8e701a5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:58:03.682 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:03.695+0000 7f8e6f400700 1 -- 172.21.15.107:0/1490673712 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8e70109e20 msgr2=0x7f8e701a5830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:58:03.682 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:03.695+0000 7f8e6f400700 1 --2- 172.21.15.107:0/1490673712 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8e70109e20 0x7f8e701a5830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:58:03.682 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:03.695+0000 7f8e6f400700 1 -- 172.21.15.107:0/1490673712 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8e70109480 msgr2=0x7f8e7019e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:58:03.683 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:03.695+0000 7f8e6f400700 1 --2- 172.21.15.107:0/1490673712 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8e70109480 0x7f8e7019e7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:58:03.683 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:03.695+0000 7f8e6f400700 1 -- 172.21.15.107:0/1490673712 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e701a5ed0 con 0x7f8e70108280 2024-03-08T20:58:03.683 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:03.695+0000 7f8e6fe00700 1 --2- 172.21.15.107:0/1490673712 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8e70109e20 0x7f8e701a5830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T20:58:03.683 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:03.696+0000 7f8e6f400700 1 --2- 172.21.15.107:0/1490673712 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8e70108280 0x7f8e7019e270 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f8e6000d9b0 tx=0x7f8e6000dd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:58:03.683 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:03.696+0000 7f8e67e00700 1 -- 172.21.15.107:0/1490673712 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e60017030 con 0x7f8e70108280 2024-03-08T20:58:03.683 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:03.696+0000 7f8e67e00700 1 -- 172.21.15.107:0/1490673712 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f8e600044d0 con 0x7f8e70108280 2024-03-08T20:58:03.683 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:03.696+0000 7f8e74e00700 1 -- 172.21.15.107:0/1490673712 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e701a61c0 con 0x7f8e70108280 2024-03-08T20:58:03.683 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:03.696+0000 7f8e67e00700 1 -- 172.21.15.107:0/1490673712 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e600039e0 con 0x7f8e70108280 2024-03-08T20:58:03.683 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:03.696+0000 7f8e74e00700 1 -- 172.21.15.107:0/1490673712 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e701a6700 con 0x7f8e70108280 2024-03-08T20:58:03.683 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:03.697+0000 7f8e74e00700 1 -- 172.21.15.107:0/1490673712 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e70063410 con 0x7f8e70108280 2024-03-08T20:58:03.684 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:03.699+0000 7f8e67e00700 1 -- 172.21.15.107:0/1490673712 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) v1 ==== 137342+0+0 (secure 0 0 0) 0x7f8e6001a070 con 0x7f8e70108280 2024-03-08T20:58:03.685 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:03.701+0000 7f8e67e00700 1 --2- 172.21.15.107:0/1490673712 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f8e540a29d0 0x7f8e540a4e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:58:03.685 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:03.701+0000 7f8e67e00700 1 -- 172.21.15.107:0/1490673712 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f8e600ce8b0 con 0x7f8e70108280 2024-03-08T20:58:03.685 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:03.701+0000 7f8e6ea00700 1 --2- 172.21.15.107:0/1490673712 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f8e540a29d0 0x7f8e540a4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:58:03.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:03.702+0000 7f8e6ea00700 1 --2- 172.21.15.107:0/1490673712 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f8e540a29d0 0x7f8e540a4e90 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f8e5c0038f0 tx=0x7f8e5c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:58:03.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:03.703+0000 7f8e67e00700 1 -- 172.21.15.107:0/1490673712 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f8e6009b8a0 con 0x7f8e70108280 2024-03-08T20:58:03.998 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:04.013+0000 7f8e74e00700 1 -- 172.21.15.107:0/1490673712 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7f8e7002d970 con 0x7f8e70108280 2024-03-08T20:58:04.002 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:04.018+0000 7f8e67e00700 1 -- 172.21.15.107:0/1490673712 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v33) v1 ==== 170+0+59 (secure 0 0 0) 0x7f8e7002d970 con 0x7f8e70108280 2024-03-08T20:58:04.002 INFO:teuthology.orchestra.run.smithi107.stdout:[client.0] 2024-03-08T20:58:04.003 INFO:teuthology.orchestra.run.smithi107.stdout: key = AQDce+tlHNv1ABAAjJDkRkd+FqHEx2ZwKEOlag== 2024-03-08T20:58:04.006 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:04.021+0000 7f8e74e00700 1 -- 172.21.15.107:0/1490673712 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f8e540a29d0 msgr2=0x7f8e540a4e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:58:04.006 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:04.021+0000 7f8e74e00700 1 --2- 172.21.15.107:0/1490673712 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f8e540a29d0 0x7f8e540a4e90 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f8e5c0038f0 tx=0x7f8e5c005fb0 comp rx=0 tx=0).stop 2024-03-08T20:58:04.006 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:04.021+0000 7f8e74e00700 1 -- 172.21.15.107:0/1490673712 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8e70108280 msgr2=0x7f8e7019e270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:58:04.006 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:04.021+0000 7f8e74e00700 1 --2- 172.21.15.107:0/1490673712 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8e70108280 0x7f8e7019e270 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f8e6000d9b0 tx=0x7f8e6000dd70 comp rx=0 tx=0).stop 2024-03-08T20:58:04.006 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:04.021+0000 7f8e74e00700 1 -- 172.21.15.107:0/1490673712 shutdown_connections 2024-03-08T20:58:04.006 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:04.021+0000 7f8e74e00700 1 --2- 172.21.15.107:0/1490673712 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8e70109e20 0x7f8e701a5830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:58:04.006 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:04.021+0000 7f8e74e00700 1 --2- 172.21.15.107:0/1490673712 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f8e540a29d0 0x7f8e540a4e90 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:58:04.006 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:04.021+0000 7f8e74e00700 1 --2- 172.21.15.107:0/1490673712 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8e70108280 0x7f8e7019e270 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:58:04.006 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:04.021+0000 7f8e74e00700 1 --2- 172.21.15.107:0/1490673712 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8e70109480 0x7f8e7019e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:58:04.006 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:04.021+0000 7f8e74e00700 1 -- 172.21.15.107:0/1490673712 >> 172.21.15.107:0/1490673712 conn(0x7f8e70077250 msgr2=0x7f8e7010c6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:58:04.006 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:04.021+0000 7f8e74e00700 1 -- 172.21.15.107:0/1490673712 shutdown_connections 2024-03-08T20:58:04.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:04.021+0000 7f8e74e00700 1 -- 172.21.15.107:0/1490673712 wait complete. 2024-03-08T20:58:04.309 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:58:04 smithi204 ceph-mon[34935]: from='client.? 172.21.15.107:0/1490673712' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-08T20:58:04.309 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:58:04 smithi204 ceph-mon[34935]: from='client.? 172.21.15.107:0/1490673712' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-08T20:58:04.373 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:58:04 smithi107 ceph-mon[31381]: from='client.? 172.21.15.107:0/1490673712' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-08T20:58:04.373 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:58:04 smithi107 ceph-mon[31381]: from='client.? 172.21.15.107:0/1490673712' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-08T20:58:04.516 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:58:04 smithi149 ceph-mon[35042]: from='client.? 172.21.15.107:0/1490673712' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-08T20:58:04.516 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:58:04 smithi149 ceph-mon[35042]: from='client.? 172.21.15.107:0/1490673712' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-08T20:58:04.831 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-03-08T20:58:04.831 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-03-08T20:58:04.832 DEBUG:teuthology.orchestra.run.smithi107:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-03-08T20:58:04.978 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-03-08T20:58:04.978 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-03-08T20:58:04.978 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761 shell --fsid dc20f0f8-dd8c-11ee-95c6-87774f69a715 -- ceph mgr dump --format=json 2024-03-08T20:58:05.379 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:58:05 smithi204 ceph-mon[34935]: pgmap v368: 97 pgs: 97 active+clean; 625 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 9.2 KiB/s wr, 24 op/s 2024-03-08T20:58:05.468 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:58:05 smithi107 ceph-mon[31381]: pgmap v368: 97 pgs: 97 active+clean; 625 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 9.2 KiB/s wr, 24 op/s 2024-03-08T20:58:05.471 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/mon.a/config 2024-03-08T20:58:05.515 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:58:05 smithi149 ceph-mon[35042]: pgmap v368: 97 pgs: 97 active+clean; 625 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 9.2 KiB/s wr, 24 op/s 2024-03-08T20:58:07.468 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:58:07 smithi107 ceph-mon[31381]: pgmap v369: 97 pgs: 97 active+clean; 625 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 8.2 KiB/s wr, 23 op/s 2024-03-08T20:58:07.629 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:58:07 smithi204 ceph-mon[34935]: pgmap v369: 97 pgs: 97 active+clean; 625 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 8.2 KiB/s wr, 23 op/s 2024-03-08T20:58:07.723 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:07.739+0000 7fb32c600700 1 -- 172.21.15.107:0/1748060949 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb324069fd0 msgr2=0x7fb32406a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:58:07.724 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:07.739+0000 7fb32c600700 1 --2- 172.21.15.107:0/1748060949 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb324069fd0 0x7fb32406a430 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fb310009990 tx=0x7fb310009ca0 comp rx=0 tx=0).stop 2024-03-08T20:58:07.724 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:07.739+0000 7fb32c600700 1 -- 172.21.15.107:0/1748060949 shutdown_connections 2024-03-08T20:58:07.724 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:07.739+0000 7fb32c600700 1 --2- 172.21.15.107:0/1748060949 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb3241047e0 0x7fb324104c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:58:07.724 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:07.739+0000 7fb32c600700 1 --2- 172.21.15.107:0/1748060949 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb324106cd0 0x7fb324069a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:58:07.724 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:07.739+0000 7fb32c600700 1 --2- 172.21.15.107:0/1748060949 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb324069fd0 0x7fb32406a430 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:58:07.724 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:07.739+0000 7fb32c600700 1 -- 172.21.15.107:0/1748060949 >> 172.21.15.107:0/1748060949 conn(0x7fb324077c10 msgr2=0x7fb324078020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:58:07.724 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:07.739+0000 7fb32c600700 1 -- 172.21.15.107:0/1748060949 shutdown_connections 2024-03-08T20:58:07.724 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:07.739+0000 7fb32c600700 1 -- 172.21.15.107:0/1748060949 wait complete. 2024-03-08T20:58:07.727 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:07.740+0000 7fb32c600700 1 Processor -- start 2024-03-08T20:58:07.727 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:07.740+0000 7fb32c600700 1 -- start start 2024-03-08T20:58:07.727 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:07.740+0000 7fb32c600700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb324069fd0 0x7fb3241a4820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:58:07.727 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:07.740+0000 7fb32c600700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb3241047e0 0x7fb3241a4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:58:07.728 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:07.740+0000 7fb32c600700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb324106cd0 0x7fb32419e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:58:07.728 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:07.740+0000 7fb32c600700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb324118660 con 0x7fb3241047e0 2024-03-08T20:58:07.728 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:07.740+0000 7fb32c600700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3241184e0 con 0x7fb324069fd0 2024-03-08T20:58:07.728 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:07.740+0000 7fb32c600700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3241187e0 con 0x7fb324106cd0 2024-03-08T20:58:07.728 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:07.740+0000 7fb329200700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb3241047e0 0x7fb3241a4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:58:07.728 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:07.741+0000 7fb329200700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb3241047e0 0x7fb3241a4d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:59772/0 (socket says 172.21.15.107:59772) 2024-03-08T20:58:07.728 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:07.741+0000 7fb329200700 1 -- 172.21.15.107:0/15517806 learned_addr learned my addr 172.21.15.107:0/15517806 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T20:58:07.729 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:07.741+0000 7fb329200700 1 -- 172.21.15.107:0/15517806 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb324106cd0 msgr2=0x7fb32419e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:58:07.729 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:07.741+0000 7fb329200700 1 --2- 172.21.15.107:0/15517806 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb324106cd0 0x7fb32419e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:58:07.729 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:07.741+0000 7fb329200700 1 -- 172.21.15.107:0/15517806 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb324069fd0 msgr2=0x7fb3241a4820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:58:07.729 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:07.741+0000 7fb329c00700 1 --2- 172.21.15.107:0/15517806 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb324069fd0 0x7fb3241a4820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:58:07.729 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:07.741+0000 7fb329200700 1 --2- 172.21.15.107:0/15517806 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb324069fd0 0x7fb3241a4820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:58:07.729 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:07.741+0000 7fb329200700 1 -- 172.21.15.107:0/15517806 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb32419f140 con 0x7fb3241047e0 2024-03-08T20:58:07.729 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:07.741+0000 7fb329c00700 1 --2- 172.21.15.107:0/15517806 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb324069fd0 0x7fb3241a4820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T20:58:07.729 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:07.741+0000 7fb329200700 1 --2- 172.21.15.107:0/15517806 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb3241047e0 0x7fb3241a4d60 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fb310000c00 tx=0x7fb3100047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:58:07.729 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:07.742+0000 7fb322000700 1 -- 172.21.15.107:0/15517806 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb310027070 con 0x7fb3241047e0 2024-03-08T20:58:07.730 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:07.742+0000 7fb32c600700 1 -- 172.21.15.107:0/15517806 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb32419f3d0 con 0x7fb3241047e0 2024-03-08T20:58:07.730 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:07.742+0000 7fb322000700 1 -- 172.21.15.107:0/15517806 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fb3100040c0 con 0x7fb3241047e0 2024-03-08T20:58:07.730 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:07.742+0000 7fb322000700 1 -- 172.21.15.107:0/15517806 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb31000f460 con 0x7fb3241047e0 2024-03-08T20:58:07.730 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:07.742+0000 7fb32c600700 1 -- 172.21.15.107:0/15517806 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3241ab6f0 con 0x7fb3241047e0 2024-03-08T20:58:07.730 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:07.743+0000 7fb32c600700 1 -- 172.21.15.107:0/15517806 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb324068aa0 con 0x7fb3241047e0 2024-03-08T20:58:07.731 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:07.745+0000 7fb322000700 1 -- 172.21.15.107:0/15517806 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) v1 ==== 137342+0+0 (secure 0 0 0) 0x7fb31002a070 con 0x7fb3241047e0 2024-03-08T20:58:07.731 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:07.746+0000 7fb322000700 1 --2- 172.21.15.107:0/15517806 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fb30c0a2a20 0x7fb30c0a4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:58:07.731 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:07.746+0000 7fb322000700 1 -- 172.21.15.107:0/15517806 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fb3100d0330 con 0x7fb3241047e0 2024-03-08T20:58:07.731 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:07.746+0000 7fb329c00700 1 --2- 172.21.15.107:0/15517806 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fb30c0a2a20 0x7fb30c0a4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:58:07.731 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:07.747+0000 7fb329c00700 1 --2- 172.21.15.107:0/15517806 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fb30c0a2a20 0x7fb30c0a4ee0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fb32419fb20 tx=0x7fb318008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:58:07.733 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:07.748+0000 7fb322000700 1 -- 172.21.15.107:0/15517806 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7fb310031070 con 0x7fb3241047e0 2024-03-08T20:58:07.766 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:58:07 smithi149 ceph-mon[35042]: pgmap v369: 97 pgs: 97 active+clean; 625 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 8.2 KiB/s wr, 23 op/s 2024-03-08T20:58:08.016 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:08.031+0000 7fb32c600700 1 -- 172.21.15.107:0/15517806 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7fb3241ab8c0 con 0x7fb3241047e0 2024-03-08T20:58:08.024 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:08.040+0000 7fb322000700 1 -- 172.21.15.107:0/15517806 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) v1 ==== 74+0+262667 (secure 0 0 0) 0x7fb310013070 con 0x7fb3241047e0 2024-03-08T20:58:08.031 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-08T20:58:08.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:08.046+0000 7fb32c600700 1 -- 172.21.15.107:0/15517806 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fb30c0a2a20 msgr2=0x7fb30c0a4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:58:08.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:08.046+0000 7fb32c600700 1 --2- 172.21.15.107:0/15517806 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fb30c0a2a20 0x7fb30c0a4ee0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fb32419fb20 tx=0x7fb318008040 comp rx=0 tx=0).stop 2024-03-08T20:58:08.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:08.046+0000 7fb32c600700 1 -- 172.21.15.107:0/15517806 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb3241047e0 msgr2=0x7fb3241a4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:58:08.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:08.046+0000 7fb32c600700 1 --2- 172.21.15.107:0/15517806 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb3241047e0 0x7fb3241a4d60 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fb310000c00 tx=0x7fb3100047b0 comp rx=0 tx=0).stop 2024-03-08T20:58:08.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:08.047+0000 7fb32c600700 1 -- 172.21.15.107:0/15517806 shutdown_connections 2024-03-08T20:58:08.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:08.047+0000 7fb32c600700 1 --2- 172.21.15.107:0/15517806 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb324106cd0 0x7fb32419e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:58:08.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:08.047+0000 7fb32c600700 1 --2- 172.21.15.107:0/15517806 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fb30c0a2a20 0x7fb30c0a4ee0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:58:08.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:08.047+0000 7fb32c600700 1 --2- 172.21.15.107:0/15517806 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb324069fd0 0x7fb3241a4820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:58:08.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:08.047+0000 7fb32c600700 1 --2- 172.21.15.107:0/15517806 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb3241047e0 0x7fb3241a4d60 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:58:08.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:08.047+0000 7fb32c600700 1 -- 172.21.15.107:0/15517806 >> 172.21.15.107:0/15517806 conn(0x7fb324077c10 msgr2=0x7fb3241017b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:58:08.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:08.047+0000 7fb32c600700 1 -- 172.21.15.107:0/15517806 shutdown_connections 2024-03-08T20:58:08.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:08.047+0000 7fb32c600700 1 -- 172.21.15.107:0/15517806 wait complete. 2024-03-08T20:58:08.629 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:58:08 smithi204 ceph-mon[34935]: from='client.? 172.21.15.107:0/15517806' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-03-08T20:58:08.718 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:58:08 smithi107 ceph-mon[31381]: from='client.? 172.21.15.107:0/15517806' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-03-08T20:58:08.765 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:58:08 smithi149 ceph-mon[35042]: from='client.? 172.21.15.107:0/15517806' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-03-08T20:58:08.943 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":17,"flags":0,"active_gid":14152,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":3376636618},{"type":"v1","addr":"172.21.15.107:6801","nonce":3376636618}]},"active_addr":"172.21.15.107:6801/3376636618","active_change":"2024-03-08T20:48:20.149578+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24112,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":34109,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.107:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.107:0","nonce":552952579}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.107:0","nonce":458107598}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.107:0","nonce":3691962590}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.107:0","nonce":356063043}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.107:0","nonce":3589054898}]}]} 2024-03-08T20:58:08.948 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-03-08T20:58:08.948 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-03-08T20:58:08.949 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761 shell --fsid dc20f0f8-dd8c-11ee-95c6-87774f69a715 -- ceph osd dump --format=json 2024-03-08T20:58:09.409 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/mon.a/config 2024-03-08T20:58:09.629 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:58:09 smithi204 ceph-mon[34935]: pgmap v370: 97 pgs: 97 active+clean; 633 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.6 KiB/s wr, 23 op/s 2024-03-08T20:58:09.668 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:58:09 smithi149 ceph-mon[35042]: pgmap v370: 97 pgs: 97 active+clean; 633 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.6 KiB/s wr, 23 op/s 2024-03-08T20:58:09.718 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:58:09 smithi107 ceph-mon[31381]: pgmap v370: 97 pgs: 97 active+clean; 633 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.6 KiB/s wr, 23 op/s 2024-03-08T20:58:11.468 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:58:11 smithi107 ceph-mon[31381]: pgmap v371: 97 pgs: 97 active+clean; 633 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.6 KiB/s wr, 19 op/s 2024-03-08T20:58:11.727 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:11.742+0000 7f58cb200700 1 -- 172.21.15.107:0/1529321760 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f58c410e890 msgr2=0x7f58c4110c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:58:11.727 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:11.742+0000 7f58cb200700 1 --2- 172.21.15.107:0/1529321760 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f58c410e890 0x7f58c4110c80 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f58bc00b0a0 tx=0x7f58bc00b3b0 comp rx=0 tx=0).stop 2024-03-08T20:58:11.727 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:11.742+0000 7f58cb200700 1 -- 172.21.15.107:0/1529321760 shutdown_connections 2024-03-08T20:58:11.727 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:11.742+0000 7f58cb200700 1 --2- 172.21.15.107:0/1529321760 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f58c4069a90 0x7f58c4069e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:58:11.727 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:11.742+0000 7f58cb200700 1 --2- 172.21.15.107:0/1529321760 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f58c410e890 0x7f58c4110c80 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:58:11.727 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:11.742+0000 7f58cb200700 1 --2- 172.21.15.107:0/1529321760 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f58c406a3b0 0x7f58c410e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:58:11.727 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:11.742+0000 7f58cb200700 1 -- 172.21.15.107:0/1529321760 >> 172.21.15.107:0/1529321760 conn(0x7f58c4077c10 msgr2=0x7f58c4078020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:58:11.727 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:11.742+0000 7f58cb200700 1 -- 172.21.15.107:0/1529321760 shutdown_connections 2024-03-08T20:58:11.727 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:11.742+0000 7f58cb200700 1 -- 172.21.15.107:0/1529321760 wait complete. 2024-03-08T20:58:11.729 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:11.743+0000 7f58cb200700 1 Processor -- start 2024-03-08T20:58:11.729 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:11.743+0000 7f58cb200700 1 -- start start 2024-03-08T20:58:11.729 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:11.744+0000 7f58cb200700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f58c4069a90 0x7f58c419a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:58:11.730 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:11.744+0000 7f58cb200700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f58c406a3b0 0x7f58c419ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:58:11.730 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:11.744+0000 7f58cb200700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f58c410e890 0x7f58c419f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:58:11.730 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:11.744+0000 7f58cb200700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58c4114140 con 0x7f58c410e890 2024-03-08T20:58:11.730 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:11.744+0000 7f58cb200700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58c4113fc0 con 0x7f58c4069a90 2024-03-08T20:58:11.730 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:11.744+0000 7f58cb200700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58c41142c0 con 0x7f58c406a3b0 2024-03-08T20:58:11.730 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:11.744+0000 7f58c9200700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f58c410e890 0x7f58c419f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:58:11.730 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:11.744+0000 7f58c9200700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f58c410e890 0x7f58c419f010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:59800/0 (socket says 172.21.15.107:59800) 2024-03-08T20:58:11.730 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:11.744+0000 7f58c9200700 1 -- 172.21.15.107:0/2132264560 learned_addr learned my addr 172.21.15.107:0/2132264560 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T20:58:11.730 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:11.744+0000 7f58c3e00700 1 --2- 172.21.15.107:0/2132264560 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f58c4069a90 0x7f58c419a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:58:11.730 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:11.744+0000 7f58c3400700 1 --2- 172.21.15.107:0/2132264560 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f58c406a3b0 0x7f58c419ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:58:11.730 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:11.745+0000 7f58c9200700 1 -- 172.21.15.107:0/2132264560 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f58c406a3b0 msgr2=0x7f58c419ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:58:11.730 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:11.745+0000 7f58c9200700 1 --2- 172.21.15.107:0/2132264560 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f58c406a3b0 0x7f58c419ac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:58:11.731 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:11.745+0000 7f58c9200700 1 -- 172.21.15.107:0/2132264560 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f58c4069a90 msgr2=0x7f58c419a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:58:11.731 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:11.745+0000 7f58c9200700 1 --2- 172.21.15.107:0/2132264560 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f58c4069a90 0x7f58c419a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:58:11.731 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:11.745+0000 7f58c9200700 1 -- 172.21.15.107:0/2132264560 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58c419f750 con 0x7f58c410e890 2024-03-08T20:58:11.731 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:11.745+0000 7f58c9200700 1 --2- 172.21.15.107:0/2132264560 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f58c410e890 0x7f58c419f010 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f58bc0062a0 tx=0x7f58bc004ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:58:11.731 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:11.745+0000 7f58c0c00700 1 -- 172.21.15.107:0/2132264560 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58bc021070 con 0x7f58c410e890 2024-03-08T20:58:11.731 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:11.745+0000 7f58cb200700 1 -- 172.21.15.107:0/2132264560 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58c419f9e0 con 0x7f58c410e890 2024-03-08T20:58:11.731 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:11.745+0000 7f58c0c00700 1 -- 172.21.15.107:0/2132264560 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f58bc007e30 con 0x7f58c410e890 2024-03-08T20:58:11.731 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:11.745+0000 7f58c0c00700 1 -- 172.21.15.107:0/2132264560 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58bc0047a0 con 0x7f58c410e890 2024-03-08T20:58:11.731 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:11.745+0000 7f58cb200700 1 -- 172.21.15.107:0/2132264560 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58c41a72c0 con 0x7f58c410e890 2024-03-08T20:58:11.732 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:11.747+0000 7f58cb200700 1 -- 172.21.15.107:0/2132264560 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58c4068aa0 con 0x7f58c410e890 2024-03-08T20:58:11.732 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:11.748+0000 7f58c0c00700 1 -- 172.21.15.107:0/2132264560 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) v1 ==== 137342+0+0 (secure 0 0 0) 0x7f58bc007980 con 0x7f58c410e890 2024-03-08T20:58:11.734 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:11.750+0000 7f58c0c00700 1 --2- 172.21.15.107:0/2132264560 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f58a80a29a0 0x7f58a80a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:58:11.734 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:11.750+0000 7f58c0c00700 1 -- 172.21.15.107:0/2132264560 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f58bc00f020 con 0x7f58c410e890 2024-03-08T20:58:11.734 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:11.750+0000 7f58c3e00700 1 --2- 172.21.15.107:0/2132264560 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f58a80a29a0 0x7f58a80a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:58:11.735 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:11.751+0000 7f58c3e00700 1 --2- 172.21.15.107:0/2132264560 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f58a80a29a0 0x7f58a80a4e60 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f58b400ac50 tx=0x7f58b400a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:58:11.738 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:11.754+0000 7f58c0c00700 1 -- 172.21.15.107:0/2132264560 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f58bc02a020 con 0x7f58c410e890 2024-03-08T20:58:11.766 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:58:11 smithi149 ceph-mon[35042]: pgmap v371: 97 pgs: 97 active+clean; 633 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.6 KiB/s wr, 19 op/s 2024-03-08T20:58:11.879 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:58:11 smithi204 ceph-mon[34935]: pgmap v371: 97 pgs: 97 active+clean; 633 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.6 KiB/s wr, 19 op/s 2024-03-08T20:58:11.968 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:11.984+0000 7f58cb200700 1 -- 172.21.15.107:0/2132264560 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f58c402d830 con 0x7f58c410e890 2024-03-08T20:58:11.971 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:11.986+0000 7f58c0c00700 1 -- 172.21.15.107:0/2132264560 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+22300 (secure 0 0 0) 0x7f58bc09d0b0 con 0x7f58c410e890 2024-03-08T20:58:11.971 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-08T20:58:11.971 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":86,"fsid":"dc20f0f8-dd8c-11ee-95c6-87774f69a715","created":"2024-03-08T20:47:04.190034+0000","modified":"2024-03-08T20:57:51.779814+0000","last_up_change":"2024-03-08T20:56:43.135514+0000","last_in_change":"2024-03-08T20:56:17.577074+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-08T20:51:32.832786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-08T20:57:48.721227+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-08T20:57:49.707779+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8b1ddc64-4d20-4e85-ac66-f1c5ac599501","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":2453946095},{"type":"v1","addr":"172.21.15.107:6803","nonce":2453946095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":2453946095},{"type":"v1","addr":"172.21.15.107:6805","nonce":2453946095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":2453946095},{"type":"v1","addr":"172.21.15.107:6809","nonce":2453946095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":2453946095},{"type":"v1","addr":"172.21.15.107:6807","nonce":2453946095}]},"public_addr":"172.21.15.107:6803/2453946095","cluster_addr":"172.21.15.107:6805/2453946095","heartbeat_back_addr":"172.21.15.107:6809/2453946095","heartbeat_front_addr":"172.21.15.107:6807/2453946095","state":["exists","up"]},{"osd":1,"uuid":"5a803c09-202a-4777-ba2a-1da8eaf26a54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":3071316419},{"type":"v1","addr":"172.21.15.149:6801","nonce":3071316419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":3071316419},{"type":"v1","addr":"172.21.15.149:6803","nonce":3071316419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":3071316419},{"type":"v1","addr":"172.21.15.149:6807","nonce":3071316419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":3071316419},{"type":"v1","addr":"172.21.15.149:6805","nonce":3071316419}]},"public_addr":"172.21.15.149:6801/3071316419","cluster_addr":"172.21.15.149:6803/3071316419","heartbeat_back_addr":"172.21.15.149:6807/3071316419","heartbeat_front_addr":"172.21.15.149:6805/3071316419","state":["exists","up"]},{"osd":2,"uuid":"555c634f-347e-47dd-a943-e50ae2be073c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":1087852630},{"type":"v1","addr":"172.21.15.204:6801","nonce":1087852630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":1087852630},{"type":"v1","addr":"172.21.15.204:6803","nonce":1087852630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":1087852630},{"type":"v1","addr":"172.21.15.204:6807","nonce":1087852630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":1087852630},{"type":"v1","addr":"172.21.15.204:6805","nonce":1087852630}]},"public_addr":"172.21.15.204:6801/1087852630","cluster_addr":"172.21.15.204:6803/1087852630","heartbeat_back_addr":"172.21.15.204:6807/1087852630","heartbeat_front_addr":"172.21.15.204:6805/1087852630","state":["exists","up"]},{"osd":3,"uuid":"9e433d97-e202-48e0-9ed1-f0988fa14307","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":3123579789},{"type":"v1","addr":"172.21.15.107:6811","nonce":3123579789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":3123579789},{"type":"v1","addr":"172.21.15.107:6813","nonce":3123579789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":3123579789},{"type":"v1","addr":"172.21.15.107:6817","nonce":3123579789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":3123579789},{"type":"v1","addr":"172.21.15.107:6815","nonce":3123579789}]},"public_addr":"172.21.15.107:6811/3123579789","cluster_addr":"172.21.15.107:6813/3123579789","heartbeat_back_addr":"172.21.15.107:6817/3123579789","heartbeat_front_addr":"172.21.15.107:6815/3123579789","state":["exists","up"]},{"osd":4,"uuid":"27c53794-9d2b-487d-bfcf-ac4fd2ab061e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":726286918},{"type":"v1","addr":"172.21.15.149:6809","nonce":726286918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":726286918},{"type":"v1","addr":"172.21.15.149:6811","nonce":726286918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":726286918},{"type":"v1","addr":"172.21.15.149:6815","nonce":726286918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":726286918},{"type":"v1","addr":"172.21.15.149:6813","nonce":726286918}]},"public_addr":"172.21.15.149:6809/726286918","cluster_addr":"172.21.15.149:6811/726286918","heartbeat_back_addr":"172.21.15.149:6815/726286918","heartbeat_front_addr":"172.21.15.149:6813/726286918","state":["exists","up"]},{"osd":5,"uuid":"3a7778fb-46c5-40c0-842e-e988c45ea24d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":3143887820},{"type":"v1","addr":"172.21.15.204:6809","nonce":3143887820}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":3143887820},{"type":"v1","addr":"172.21.15.204:6811","nonce":3143887820}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":3143887820},{"type":"v1","addr":"172.21.15.204:6815","nonce":3143887820}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":3143887820},{"type":"v1","addr":"172.21.15.204:6813","nonce":3143887820}]},"public_addr":"172.21.15.204:6809/3143887820","cluster_addr":"172.21.15.204:6811/3143887820","heartbeat_back_addr":"172.21.15.204:6815/3143887820","heartbeat_front_addr":"172.21.15.204:6813/3143887820","state":["exists","up"]},{"osd":6,"uuid":"d2e05a56-be4f-4435-9b76-ff9aa01e4ac3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":2912452950},{"type":"v1","addr":"172.21.15.107:6819","nonce":2912452950}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":2912452950},{"type":"v1","addr":"172.21.15.107:6821","nonce":2912452950}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":2912452950},{"type":"v1","addr":"172.21.15.107:6825","nonce":2912452950}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":2912452950},{"type":"v1","addr":"172.21.15.107:6823","nonce":2912452950}]},"public_addr":"172.21.15.107:6819/2912452950","cluster_addr":"172.21.15.107:6821/2912452950","heartbeat_back_addr":"172.21.15.107:6825/2912452950","heartbeat_front_addr":"172.21.15.107:6823/2912452950","state":["exists","up"]},{"osd":7,"uuid":"a7dd26ee-c118-4a9a-9e19-142c2680902d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":3835863809},{"type":"v1","addr":"172.21.15.149:6817","nonce":3835863809}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":3835863809},{"type":"v1","addr":"172.21.15.149:6819","nonce":3835863809}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":3835863809},{"type":"v1","addr":"172.21.15.149:6823","nonce":3835863809}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":3835863809},{"type":"v1","addr":"172.21.15.149:6821","nonce":3835863809}]},"public_addr":"172.21.15.149:6817/3835863809","cluster_addr":"172.21.15.149:6819/3835863809","heartbeat_back_addr":"172.21.15.149:6823/3835863809","heartbeat_front_addr":"172.21.15.149:6821/3835863809","state":["exists","up"]},{"osd":8,"uuid":"baaaa8dd-0e1f-4bae-86bf-9e429d985f23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":1010601477},{"type":"v1","addr":"172.21.15.204:6817","nonce":1010601477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":1010601477},{"type":"v1","addr":"172.21.15.204:6819","nonce":1010601477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":1010601477},{"type":"v1","addr":"172.21.15.204:6823","nonce":1010601477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":1010601477},{"type":"v1","addr":"172.21.15.204:6821","nonce":1010601477}]},"public_addr":"172.21.15.204:6817/1010601477","cluster_addr":"172.21.15.204:6819/1010601477","heartbeat_back_addr":"172.21.15.204:6823/1010601477","heartbeat_front_addr":"172.21.15.204:6821/1010601477","state":["exists","up"]},{"osd":9,"uuid":"8df80dfa-e7ff-410e-824c-b1dce111b02f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":3334759819},{"type":"v1","addr":"172.21.15.107:6827","nonce":3334759819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3334759819},{"type":"v1","addr":"172.21.15.107:6829","nonce":3334759819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":3334759819},{"type":"v1","addr":"172.21.15.107:6833","nonce":3334759819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":3334759819},{"type":"v1","addr":"172.21.15.107:6831","nonce":3334759819}]},"public_addr":"172.21.15.107:6827/3334759819","cluster_addr":"172.21.15.107:6829/3334759819","heartbeat_back_addr":"172.21.15.107:6833/3334759819","heartbeat_front_addr":"172.21.15.107:6831/3334759819","state":["exists","up"]},{"osd":10,"uuid":"338945b2-20c0-40b2-aaca-062cd4adee04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":3469284631},{"type":"v1","addr":"172.21.15.149:6825","nonce":3469284631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":3469284631},{"type":"v1","addr":"172.21.15.149:6827","nonce":3469284631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":3469284631},{"type":"v1","addr":"172.21.15.149:6831","nonce":3469284631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":3469284631},{"type":"v1","addr":"172.21.15.149:6829","nonce":3469284631}]},"public_addr":"172.21.15.149:6825/3469284631","cluster_addr":"172.21.15.149:6827/3469284631","heartbeat_back_addr":"172.21.15.149:6831/3469284631","heartbeat_front_addr":"172.21.15.149:6829/3469284631","state":["exists","up"]},{"osd":11,"uuid":"acccfe4f-d307-4a64-8f4d-fba27bfbe09d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":3097772473},{"type":"v1","addr":"172.21.15.204:6825","nonce":3097772473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":3097772473},{"type":"v1","addr":"172.21.15.204:6827","nonce":3097772473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":3097772473},{"type":"v1","addr":"172.21.15.204:6831","nonce":3097772473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":3097772473},{"type":"v1","addr":"172.21.15.204:6829","nonce":3097772473}]},"public_addr":"172.21.15.204:6825/3097772473","cluster_addr":"172.21.15.204:6827/3097772473","heartbeat_back_addr":"172.21.15.204:6831/3097772473","heartbeat_front_addr":"172.21.15.204:6829/3097772473","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T20:50:25.862753+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T20:50:58.167663+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T20:51:30.312816+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T20:52:05.487917+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T20:52:39.348354+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T20:53:12.899002+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T20:53:47.630837+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T20:54:21.444552+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T20:54:57.101160+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T20:55:31.841786+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T20:56:05.827161+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T20:56:40.611097+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.107:0/1514666962":"2024-03-09T20:48:20.149406+0000","172.21.15.107:0/1886016707":"2024-03-09T20:48:20.149406+0000","172.21.15.107:6801/426372446":"2024-03-09T20:48:20.149406+0000","172.21.15.107:6800/426372446":"2024-03-09T20:48:20.149406+0000","172.21.15.107:0/2392547434":"2024-03-09T20:47:45.706790+0000","172.21.15.107:0/1206995985":"2024-03-09T20:47:45.706790+0000","172.21.15.107:6800/3549936519":"2024-03-09T20:47:45.706790+0000","172.21.15.107:6801/3549936519":"2024-03-09T20:47:45.706790+0000","172.21.15.107:0/2482452215":"2024-03-09T20:47:45.706790+0000","172.21.15.107:0/808124991":"2024-03-09T20:48:20.149406+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-08T20:58:11.974 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:11.990+0000 7f58cb200700 1 -- 172.21.15.107:0/2132264560 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f58a80a29a0 msgr2=0x7f58a80a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:58:11.974 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:11.990+0000 7f58cb200700 1 --2- 172.21.15.107:0/2132264560 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f58a80a29a0 0x7f58a80a4e60 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f58b400ac50 tx=0x7f58b400a380 comp rx=0 tx=0).stop 2024-03-08T20:58:11.975 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:11.990+0000 7f58cb200700 1 -- 172.21.15.107:0/2132264560 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f58c410e890 msgr2=0x7f58c419f010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:58:11.975 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:11.990+0000 7f58cb200700 1 --2- 172.21.15.107:0/2132264560 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f58c410e890 0x7f58c419f010 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f58bc0062a0 tx=0x7f58bc004ce0 comp rx=0 tx=0).stop 2024-03-08T20:58:11.975 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:11.990+0000 7f58cb200700 1 -- 172.21.15.107:0/2132264560 shutdown_connections 2024-03-08T20:58:11.975 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:11.990+0000 7f58cb200700 1 --2- 172.21.15.107:0/2132264560 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f58a80a29a0 0x7f58a80a4e60 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:58:11.975 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:11.990+0000 7f58cb200700 1 --2- 172.21.15.107:0/2132264560 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f58c410e890 0x7f58c419f010 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:58:11.975 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:11.990+0000 7f58cb200700 1 --2- 172.21.15.107:0/2132264560 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f58c4069a90 0x7f58c419a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:58:11.975 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:11.990+0000 7f58cb200700 1 --2- 172.21.15.107:0/2132264560 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f58c406a3b0 0x7f58c419ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:58:11.975 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:11.990+0000 7f58cb200700 1 -- 172.21.15.107:0/2132264560 >> 172.21.15.107:0/2132264560 conn(0x7f58c4077c10 msgr2=0x7f58c410fe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:58:11.975 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:11.990+0000 7f58cb200700 1 -- 172.21.15.107:0/2132264560 shutdown_connections 2024-03-08T20:58:11.975 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:11.990+0000 7f58cb200700 1 -- 172.21.15.107:0/2132264560 wait complete. 2024-03-08T20:58:12.516 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:58:12 smithi149 ceph-mon[35042]: from='client.? 172.21.15.107:0/2132264560' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-08T20:58:12.718 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:58:12 smithi107 ceph-mon[31381]: from='client.? 172.21.15.107:0/2132264560' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-08T20:58:12.872 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-03-08T20:58:12.873 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761 shell --fsid dc20f0f8-dd8c-11ee-95c6-87774f69a715 -- ceph osd dump --format=json 2024-03-08T20:58:12.879 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:58:12 smithi204 ceph-mon[34935]: from='client.? 172.21.15.107:0/2132264560' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-08T20:58:13.313 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/mon.a/config 2024-03-08T20:58:13.490 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:58:13 smithi107 ceph-mon[31381]: pgmap v372: 97 pgs: 97 active+clean; 633 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.2 KiB/s wr, 13 op/s 2024-03-08T20:58:13.765 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:58:13 smithi149 ceph-mon[35042]: pgmap v372: 97 pgs: 97 active+clean; 633 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.2 KiB/s wr, 13 op/s 2024-03-08T20:58:13.879 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:58:13 smithi204 ceph-mon[34935]: pgmap v372: 97 pgs: 97 active+clean; 633 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.2 KiB/s wr, 13 op/s 2024-03-08T20:58:15.630 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:58:15 smithi204 ceph-mon[34935]: pgmap v373: 97 pgs: 97 active+clean; 638 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 3.3 KiB/s wr, 12 op/s 2024-03-08T20:58:15.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:15.650+0000 7f3715600700 1 -- 172.21.15.107:0/3816480156 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3710104230 msgr2=0x7f3710104610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:58:15.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:15.650+0000 7f3715600700 1 --2- 172.21.15.107:0/3816480156 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3710104230 0x7f3710104610 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f37000099c0 tx=0x7f3700009cd0 comp rx=0 tx=0).stop 2024-03-08T20:58:15.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:15.650+0000 7f3715600700 1 -- 172.21.15.107:0/3816480156 shutdown_connections 2024-03-08T20:58:15.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:15.650+0000 7f3715600700 1 --2- 172.21.15.107:0/3816480156 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3710109310 0x7f3710110ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:58:15.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:15.650+0000 7f3715600700 1 --2- 172.21.15.107:0/3816480156 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3710104230 0x7f3710104610 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:58:15.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:15.650+0000 7f3715600700 1 --2- 172.21.15.107:0/3816480156 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3710104be0 0x7f3710108c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:58:15.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:15.650+0000 7f3715600700 1 -- 172.21.15.107:0/3816480156 >> 172.21.15.107:0/3816480156 conn(0x7f37100ffac0 msgr2=0x7f3710101ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:58:15.636 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:15.650+0000 7f3715600700 1 -- 172.21.15.107:0/3816480156 shutdown_connections 2024-03-08T20:58:15.636 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:15.650+0000 7f3715600700 1 -- 172.21.15.107:0/3816480156 wait complete. 2024-03-08T20:58:15.636 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:15.651+0000 7f3715600700 1 Processor -- start 2024-03-08T20:58:15.636 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:15.652+0000 7f3715600700 1 -- start start 2024-03-08T20:58:15.637 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:15.652+0000 7f3715600700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3710104230 0x7f371019a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:58:15.637 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:15.652+0000 7f3715600700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3710104be0 0x7f371019ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:58:15.637 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:15.652+0000 7f3715600700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3710109310 0x7f371019ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:58:15.637 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:15.652+0000 7f3715600700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37101141f0 con 0x7f3710104230 2024-03-08T20:58:15.637 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:15.652+0000 7f3715600700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3710114070 con 0x7f3710109310 2024-03-08T20:58:15.637 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:15.652+0000 7f3715600700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3710114370 con 0x7f3710104be0 2024-03-08T20:58:15.637 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:15.652+0000 7f370ea00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3710104230 0x7f371019a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:58:15.637 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:15.652+0000 7f370ea00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3710104230 0x7f371019a6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:59808/0 (socket says 172.21.15.107:59808) 2024-03-08T20:58:15.637 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:15.652+0000 7f370ea00700 1 -- 172.21.15.107:0/1052363655 learned_addr learned my addr 172.21.15.107:0/1052363655 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T20:58:15.638 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:15.653+0000 7f370e000700 1 --2- 172.21.15.107:0/1052363655 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3710104be0 0x7f371019ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:58:15.638 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:15.653+0000 7f370f400700 1 --2- 172.21.15.107:0/1052363655 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3710109310 0x7f371019ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:58:15.638 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:15.653+0000 7f370ea00700 1 -- 172.21.15.107:0/1052363655 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3710104be0 msgr2=0x7f371019ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:58:15.638 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:15.653+0000 7f370ea00700 1 --2- 172.21.15.107:0/1052363655 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3710104be0 0x7f371019ac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:58:15.639 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:15.653+0000 7f370ea00700 1 -- 172.21.15.107:0/1052363655 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3710109310 msgr2=0x7f371019ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:58:15.639 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:15.653+0000 7f370ea00700 1 --2- 172.21.15.107:0/1052363655 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3710109310 0x7f371019ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:58:15.639 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:15.653+0000 7f370ea00700 1 -- 172.21.15.107:0/1052363655 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f371019f6d0 con 0x7f3710104230 2024-03-08T20:58:15.639 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:15.653+0000 7f370f400700 1 --2- 172.21.15.107:0/1052363655 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3710109310 0x7f371019ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T20:58:15.639 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:15.653+0000 7f370ea00700 1 --2- 172.21.15.107:0/1052363655 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3710104230 0x7f371019a6c0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f3700005b40 tx=0x7f3700005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:58:15.639 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:15.653+0000 7f3707400700 1 -- 172.21.15.107:0/1052363655 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3700027070 con 0x7f3710104230 2024-03-08T20:58:15.639 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:15.654+0000 7f3707400700 1 -- 172.21.15.107:0/1052363655 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f3700003fd0 con 0x7f3710104230 2024-03-08T20:58:15.639 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:15.654+0000 7f3715600700 1 -- 172.21.15.107:0/1052363655 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f371019f960 con 0x7f3710104230 2024-03-08T20:58:15.639 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:15.654+0000 7f3707400700 1 -- 172.21.15.107:0/1052363655 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3700022c60 con 0x7f3710104230 2024-03-08T20:58:15.639 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:15.654+0000 7f3715600700 1 -- 172.21.15.107:0/1052363655 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37101a7240 con 0x7f3710104230 2024-03-08T20:58:15.639 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:15.655+0000 7f3715600700 1 -- 172.21.15.107:0/1052363655 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3710105710 con 0x7f3710104230 2024-03-08T20:58:15.641 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:15.657+0000 7f3707400700 1 -- 172.21.15.107:0/1052363655 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) v1 ==== 137342+0+0 (secure 0 0 0) 0x7f370002a070 con 0x7f3710104230 2024-03-08T20:58:15.642 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:15.658+0000 7f3707400700 1 --2- 172.21.15.107:0/1052363655 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f36f40a2920 0x7f36f40a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:58:15.642 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:15.658+0000 7f3707400700 1 -- 172.21.15.107:0/1052363655 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f3700022e00 con 0x7f3710104230 2024-03-08T20:58:15.642 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:15.658+0000 7f370e000700 1 --2- 172.21.15.107:0/1052363655 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f36f40a2920 0x7f36f40a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:58:15.643 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:15.659+0000 7f370e000700 1 --2- 172.21.15.107:0/1052363655 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f36f40a2920 0x7f36f40a4de0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f36fc005950 tx=0x7f36fc0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:58:15.647 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:15.663+0000 7f3707400700 1 -- 172.21.15.107:0/1052363655 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f370005e260 con 0x7f3710104230 2024-03-08T20:58:15.718 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:58:15 smithi107 ceph-mon[31381]: pgmap v373: 97 pgs: 97 active+clean; 638 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 3.3 KiB/s wr, 12 op/s 2024-03-08T20:58:15.765 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:58:15 smithi149 ceph-mon[35042]: pgmap v373: 97 pgs: 97 active+clean; 638 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 3.3 KiB/s wr, 12 op/s 2024-03-08T20:58:15.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:15.889+0000 7f3715600700 1 -- 172.21.15.107:0/1052363655 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f371002d830 con 0x7f3710104230 2024-03-08T20:58:15.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:15.893+0000 7f3707400700 1 -- 172.21.15.107:0/1052363655 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+22300 (secure 0 0 0) 0x7f3700013070 con 0x7f3710104230 2024-03-08T20:58:15.878 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-08T20:58:15.878 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":86,"fsid":"dc20f0f8-dd8c-11ee-95c6-87774f69a715","created":"2024-03-08T20:47:04.190034+0000","modified":"2024-03-08T20:57:51.779814+0000","last_up_change":"2024-03-08T20:56:43.135514+0000","last_in_change":"2024-03-08T20:56:17.577074+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-08T20:51:32.832786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-08T20:57:48.721227+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-08T20:57:49.707779+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8b1ddc64-4d20-4e85-ac66-f1c5ac599501","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":2453946095},{"type":"v1","addr":"172.21.15.107:6803","nonce":2453946095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":2453946095},{"type":"v1","addr":"172.21.15.107:6805","nonce":2453946095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":2453946095},{"type":"v1","addr":"172.21.15.107:6809","nonce":2453946095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":2453946095},{"type":"v1","addr":"172.21.15.107:6807","nonce":2453946095}]},"public_addr":"172.21.15.107:6803/2453946095","cluster_addr":"172.21.15.107:6805/2453946095","heartbeat_back_addr":"172.21.15.107:6809/2453946095","heartbeat_front_addr":"172.21.15.107:6807/2453946095","state":["exists","up"]},{"osd":1,"uuid":"5a803c09-202a-4777-ba2a-1da8eaf26a54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":3071316419},{"type":"v1","addr":"172.21.15.149:6801","nonce":3071316419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":3071316419},{"type":"v1","addr":"172.21.15.149:6803","nonce":3071316419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":3071316419},{"type":"v1","addr":"172.21.15.149:6807","nonce":3071316419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":3071316419},{"type":"v1","addr":"172.21.15.149:6805","nonce":3071316419}]},"public_addr":"172.21.15.149:6801/3071316419","cluster_addr":"172.21.15.149:6803/3071316419","heartbeat_back_addr":"172.21.15.149:6807/3071316419","heartbeat_front_addr":"172.21.15.149:6805/3071316419","state":["exists","up"]},{"osd":2,"uuid":"555c634f-347e-47dd-a943-e50ae2be073c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":1087852630},{"type":"v1","addr":"172.21.15.204:6801","nonce":1087852630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":1087852630},{"type":"v1","addr":"172.21.15.204:6803","nonce":1087852630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":1087852630},{"type":"v1","addr":"172.21.15.204:6807","nonce":1087852630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":1087852630},{"type":"v1","addr":"172.21.15.204:6805","nonce":1087852630}]},"public_addr":"172.21.15.204:6801/1087852630","cluster_addr":"172.21.15.204:6803/1087852630","heartbeat_back_addr":"172.21.15.204:6807/1087852630","heartbeat_front_addr":"172.21.15.204:6805/1087852630","state":["exists","up"]},{"osd":3,"uuid":"9e433d97-e202-48e0-9ed1-f0988fa14307","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":3123579789},{"type":"v1","addr":"172.21.15.107:6811","nonce":3123579789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":3123579789},{"type":"v1","addr":"172.21.15.107:6813","nonce":3123579789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":3123579789},{"type":"v1","addr":"172.21.15.107:6817","nonce":3123579789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":3123579789},{"type":"v1","addr":"172.21.15.107:6815","nonce":3123579789}]},"public_addr":"172.21.15.107:6811/3123579789","cluster_addr":"172.21.15.107:6813/3123579789","heartbeat_back_addr":"172.21.15.107:6817/3123579789","heartbeat_front_addr":"172.21.15.107:6815/3123579789","state":["exists","up"]},{"osd":4,"uuid":"27c53794-9d2b-487d-bfcf-ac4fd2ab061e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":726286918},{"type":"v1","addr":"172.21.15.149:6809","nonce":726286918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":726286918},{"type":"v1","addr":"172.21.15.149:6811","nonce":726286918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":726286918},{"type":"v1","addr":"172.21.15.149:6815","nonce":726286918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":726286918},{"type":"v1","addr":"172.21.15.149:6813","nonce":726286918}]},"public_addr":"172.21.15.149:6809/726286918","cluster_addr":"172.21.15.149:6811/726286918","heartbeat_back_addr":"172.21.15.149:6815/726286918","heartbeat_front_addr":"172.21.15.149:6813/726286918","state":["exists","up"]},{"osd":5,"uuid":"3a7778fb-46c5-40c0-842e-e988c45ea24d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":3143887820},{"type":"v1","addr":"172.21.15.204:6809","nonce":3143887820}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":3143887820},{"type":"v1","addr":"172.21.15.204:6811","nonce":3143887820}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":3143887820},{"type":"v1","addr":"172.21.15.204:6815","nonce":3143887820}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":3143887820},{"type":"v1","addr":"172.21.15.204:6813","nonce":3143887820}]},"public_addr":"172.21.15.204:6809/3143887820","cluster_addr":"172.21.15.204:6811/3143887820","heartbeat_back_addr":"172.21.15.204:6815/3143887820","heartbeat_front_addr":"172.21.15.204:6813/3143887820","state":["exists","up"]},{"osd":6,"uuid":"d2e05a56-be4f-4435-9b76-ff9aa01e4ac3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":2912452950},{"type":"v1","addr":"172.21.15.107:6819","nonce":2912452950}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":2912452950},{"type":"v1","addr":"172.21.15.107:6821","nonce":2912452950}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":2912452950},{"type":"v1","addr":"172.21.15.107:6825","nonce":2912452950}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":2912452950},{"type":"v1","addr":"172.21.15.107:6823","nonce":2912452950}]},"public_addr":"172.21.15.107:6819/2912452950","cluster_addr":"172.21.15.107:6821/2912452950","heartbeat_back_addr":"172.21.15.107:6825/2912452950","heartbeat_front_addr":"172.21.15.107:6823/2912452950","state":["exists","up"]},{"osd":7,"uuid":"a7dd26ee-c118-4a9a-9e19-142c2680902d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":3835863809},{"type":"v1","addr":"172.21.15.149:6817","nonce":3835863809}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":3835863809},{"type":"v1","addr":"172.21.15.149:6819","nonce":3835863809}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":3835863809},{"type":"v1","addr":"172.21.15.149:6823","nonce":3835863809}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":3835863809},{"type":"v1","addr":"172.21.15.149:6821","nonce":3835863809}]},"public_addr":"172.21.15.149:6817/3835863809","cluster_addr":"172.21.15.149:6819/3835863809","heartbeat_back_addr":"172.21.15.149:6823/3835863809","heartbeat_front_addr":"172.21.15.149:6821/3835863809","state":["exists","up"]},{"osd":8,"uuid":"baaaa8dd-0e1f-4bae-86bf-9e429d985f23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":1010601477},{"type":"v1","addr":"172.21.15.204:6817","nonce":1010601477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":1010601477},{"type":"v1","addr":"172.21.15.204:6819","nonce":1010601477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":1010601477},{"type":"v1","addr":"172.21.15.204:6823","nonce":1010601477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":1010601477},{"type":"v1","addr":"172.21.15.204:6821","nonce":1010601477}]},"public_addr":"172.21.15.204:6817/1010601477","cluster_addr":"172.21.15.204:6819/1010601477","heartbeat_back_addr":"172.21.15.204:6823/1010601477","heartbeat_front_addr":"172.21.15.204:6821/1010601477","state":["exists","up"]},{"osd":9,"uuid":"8df80dfa-e7ff-410e-824c-b1dce111b02f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":3334759819},{"type":"v1","addr":"172.21.15.107:6827","nonce":3334759819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3334759819},{"type":"v1","addr":"172.21.15.107:6829","nonce":3334759819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":3334759819},{"type":"v1","addr":"172.21.15.107:6833","nonce":3334759819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":3334759819},{"type":"v1","addr":"172.21.15.107:6831","nonce":3334759819}]},"public_addr":"172.21.15.107:6827/3334759819","cluster_addr":"172.21.15.107:6829/3334759819","heartbeat_back_addr":"172.21.15.107:6833/3334759819","heartbeat_front_addr":"172.21.15.107:6831/3334759819","state":["exists","up"]},{"osd":10,"uuid":"338945b2-20c0-40b2-aaca-062cd4adee04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":3469284631},{"type":"v1","addr":"172.21.15.149:6825","nonce":3469284631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":3469284631},{"type":"v1","addr":"172.21.15.149:6827","nonce":3469284631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":3469284631},{"type":"v1","addr":"172.21.15.149:6831","nonce":3469284631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":3469284631},{"type":"v1","addr":"172.21.15.149:6829","nonce":3469284631}]},"public_addr":"172.21.15.149:6825/3469284631","cluster_addr":"172.21.15.149:6827/3469284631","heartbeat_back_addr":"172.21.15.149:6831/3469284631","heartbeat_front_addr":"172.21.15.149:6829/3469284631","state":["exists","up"]},{"osd":11,"uuid":"acccfe4f-d307-4a64-8f4d-fba27bfbe09d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":3097772473},{"type":"v1","addr":"172.21.15.204:6825","nonce":3097772473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":3097772473},{"type":"v1","addr":"172.21.15.204:6827","nonce":3097772473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":3097772473},{"type":"v1","addr":"172.21.15.204:6831","nonce":3097772473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":3097772473},{"type":"v1","addr":"172.21.15.204:6829","nonce":3097772473}]},"public_addr":"172.21.15.204:6825/3097772473","cluster_addr":"172.21.15.204:6827/3097772473","heartbeat_back_addr":"172.21.15.204:6831/3097772473","heartbeat_front_addr":"172.21.15.204:6829/3097772473","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T20:50:25.862753+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T20:50:58.167663+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T20:51:30.312816+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T20:52:05.487917+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T20:52:39.348354+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T20:53:12.899002+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T20:53:47.630837+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T20:54:21.444552+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T20:54:57.101160+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T20:55:31.841786+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T20:56:05.827161+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T20:56:40.611097+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.107:0/1514666962":"2024-03-09T20:48:20.149406+0000","172.21.15.107:0/1886016707":"2024-03-09T20:48:20.149406+0000","172.21.15.107:6801/426372446":"2024-03-09T20:48:20.149406+0000","172.21.15.107:6800/426372446":"2024-03-09T20:48:20.149406+0000","172.21.15.107:0/2392547434":"2024-03-09T20:47:45.706790+0000","172.21.15.107:0/1206995985":"2024-03-09T20:47:45.706790+0000","172.21.15.107:6800/3549936519":"2024-03-09T20:47:45.706790+0000","172.21.15.107:6801/3549936519":"2024-03-09T20:47:45.706790+0000","172.21.15.107:0/2482452215":"2024-03-09T20:47:45.706790+0000","172.21.15.107:0/808124991":"2024-03-09T20:48:20.149406+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-08T20:58:15.880 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:15.896+0000 7f3715600700 1 -- 172.21.15.107:0/1052363655 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f36f40a2920 msgr2=0x7f36f40a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:58:15.880 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:15.896+0000 7f3715600700 1 --2- 172.21.15.107:0/1052363655 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f36f40a2920 0x7f36f40a4de0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f36fc005950 tx=0x7f36fc0058e0 comp rx=0 tx=0).stop 2024-03-08T20:58:15.880 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:15.896+0000 7f3715600700 1 -- 172.21.15.107:0/1052363655 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3710104230 msgr2=0x7f371019a6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:58:15.880 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:15.896+0000 7f3715600700 1 --2- 172.21.15.107:0/1052363655 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3710104230 0x7f371019a6c0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f3700005b40 tx=0x7f3700005e80 comp rx=0 tx=0).stop 2024-03-08T20:58:15.881 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:15.896+0000 7f3715600700 1 -- 172.21.15.107:0/1052363655 shutdown_connections 2024-03-08T20:58:15.881 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:15.896+0000 7f3715600700 1 --2- 172.21.15.107:0/1052363655 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f36f40a2920 0x7f36f40a4de0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:58:15.881 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:15.896+0000 7f3715600700 1 --2- 172.21.15.107:0/1052363655 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3710109310 0x7f371019ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:58:15.881 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:15.896+0000 7f3715600700 1 --2- 172.21.15.107:0/1052363655 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3710104230 0x7f371019a6c0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:58:15.881 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:15.896+0000 7f3715600700 1 --2- 172.21.15.107:0/1052363655 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3710104be0 0x7f371019ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:58:15.881 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:15.896+0000 7f3715600700 1 -- 172.21.15.107:0/1052363655 >> 172.21.15.107:0/1052363655 conn(0x7f37100ffac0 msgr2=0x7f3710101e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:58:15.881 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:15.896+0000 7f3715600700 1 -- 172.21.15.107:0/1052363655 shutdown_connections 2024-03-08T20:58:15.881 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:15.896+0000 7f3715600700 1 -- 172.21.15.107:0/1052363655 wait complete. 2024-03-08T20:58:16.698 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:58:16 smithi107 ceph-mon[31381]: from='client.? 172.21.15.107:0/1052363655' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-08T20:58:16.754 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761 shell --fsid dc20f0f8-dd8c-11ee-95c6-87774f69a715 -- ceph tell osd.0 flush_pg_stats 2024-03-08T20:58:16.754 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761 shell --fsid dc20f0f8-dd8c-11ee-95c6-87774f69a715 -- ceph tell osd.1 flush_pg_stats 2024-03-08T20:58:16.755 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761 shell --fsid dc20f0f8-dd8c-11ee-95c6-87774f69a715 -- ceph tell osd.2 flush_pg_stats 2024-03-08T20:58:16.755 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761 shell --fsid dc20f0f8-dd8c-11ee-95c6-87774f69a715 -- ceph tell osd.3 flush_pg_stats 2024-03-08T20:58:16.755 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761 shell --fsid dc20f0f8-dd8c-11ee-95c6-87774f69a715 -- ceph tell osd.4 flush_pg_stats 2024-03-08T20:58:16.756 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761 shell --fsid dc20f0f8-dd8c-11ee-95c6-87774f69a715 -- ceph tell osd.5 flush_pg_stats 2024-03-08T20:58:16.756 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761 shell --fsid dc20f0f8-dd8c-11ee-95c6-87774f69a715 -- ceph tell osd.6 flush_pg_stats 2024-03-08T20:58:16.756 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761 shell --fsid dc20f0f8-dd8c-11ee-95c6-87774f69a715 -- ceph tell osd.7 flush_pg_stats 2024-03-08T20:58:16.757 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761 shell --fsid dc20f0f8-dd8c-11ee-95c6-87774f69a715 -- ceph tell osd.8 flush_pg_stats 2024-03-08T20:58:16.757 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761 shell --fsid dc20f0f8-dd8c-11ee-95c6-87774f69a715 -- ceph tell osd.9 flush_pg_stats 2024-03-08T20:58:16.758 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761 shell --fsid dc20f0f8-dd8c-11ee-95c6-87774f69a715 -- ceph tell osd.10 flush_pg_stats 2024-03-08T20:58:16.758 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761 shell --fsid dc20f0f8-dd8c-11ee-95c6-87774f69a715 -- ceph tell osd.11 flush_pg_stats 2024-03-08T20:58:16.766 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:58:16 smithi149 ceph-mon[35042]: from='client.? 172.21.15.107:0/1052363655' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-08T20:58:16.879 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:58:16 smithi204 ceph-mon[34935]: from='client.? 172.21.15.107:0/1052363655' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-08T20:58:17.702 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/mon.a/config 2024-03-08T20:58:17.717 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:58:17 smithi107 ceph-mon[31381]: pgmap v374: 97 pgs: 97 active+clean; 638 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.4 KiB/s wr, 10 op/s 2024-03-08T20:58:17.765 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:58:17 smithi149 ceph-mon[35042]: pgmap v374: 97 pgs: 97 active+clean; 638 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.4 KiB/s wr, 10 op/s 2024-03-08T20:58:17.820 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/mon.a/config 2024-03-08T20:58:17.844 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/mon.a/config 2024-03-08T20:58:17.879 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:58:17 smithi204 ceph-mon[34935]: pgmap v374: 97 pgs: 97 active+clean; 638 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.4 KiB/s wr, 10 op/s 2024-03-08T20:58:17.945 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/mon.a/config 2024-03-08T20:58:19.718 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:58:19 smithi107 ceph-mon[31381]: pgmap v375: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-03-08T20:58:19.765 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:58:19 smithi149 ceph-mon[35042]: pgmap v375: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-03-08T20:58:19.841 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/mon.a/config 2024-03-08T20:58:19.844 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/mon.a/config 2024-03-08T20:58:19.846 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/mon.a/config 2024-03-08T20:58:19.847 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/mon.a/config 2024-03-08T20:58:19.849 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/mon.a/config 2024-03-08T20:58:19.854 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/mon.a/config 2024-03-08T20:58:19.858 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/mon.a/config 2024-03-08T20:58:19.863 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/mon.a/config 2024-03-08T20:58:19.878 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:58:19 smithi204 ceph-mon[34935]: pgmap v375: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-03-08T20:58:20.718 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:58:20 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]: dispatch 2024-03-08T20:58:20.719 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:58:20 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 4]}]: dispatch 2024-03-08T20:58:20.719 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:58:20 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]: dispatch 2024-03-08T20:58:20.719 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:58:20 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]: dispatch 2024-03-08T20:58:20.719 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:58:20 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 6]}]: dispatch 2024-03-08T20:58:20.719 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:58:20 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [5, 9]}]: dispatch 2024-03-08T20:58:20.719 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:58:20 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8]}]: dispatch 2024-03-08T20:58:20.719 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:58:20 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3b", "id": [3, 9]}]: dispatch 2024-03-08T20:58:20.719 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:58:20 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]: dispatch 2024-03-08T20:58:20.766 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:58:20 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]: dispatch 2024-03-08T20:58:20.766 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:58:20 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 4]}]: dispatch 2024-03-08T20:58:20.766 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:58:20 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]: dispatch 2024-03-08T20:58:20.766 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:58:20 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]: dispatch 2024-03-08T20:58:20.766 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:58:20 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 6]}]: dispatch 2024-03-08T20:58:20.766 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:58:20 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [5, 9]}]: dispatch 2024-03-08T20:58:20.766 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:58:20 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8]}]: dispatch 2024-03-08T20:58:20.766 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:58:20 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3b", "id": [3, 9]}]: dispatch 2024-03-08T20:58:20.766 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:58:20 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]: dispatch 2024-03-08T20:58:20.879 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:58:20 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]: dispatch 2024-03-08T20:58:20.879 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:58:20 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 4]}]: dispatch 2024-03-08T20:58:20.879 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:58:20 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]: dispatch 2024-03-08T20:58:20.879 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:58:20 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]: dispatch 2024-03-08T20:58:20.879 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:58:20 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 6]}]: dispatch 2024-03-08T20:58:20.879 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:58:20 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [5, 9]}]: dispatch 2024-03-08T20:58:20.880 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:58:20 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8]}]: dispatch 2024-03-08T20:58:20.880 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:58:20 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3b", "id": [3, 9]}]: dispatch 2024-03-08T20:58:20.880 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:58:20 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]: dispatch 2024-03-08T20:58:21.706 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:58:21 smithi149 ceph-mon[35042]: pgmap v376: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 852 B/s wr, 9 op/s 2024-03-08T20:58:21.706 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:58:21 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]': finished 2024-03-08T20:58:21.706 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:58:21 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 4]}]': finished 2024-03-08T20:58:21.707 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:58:21 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]': finished 2024-03-08T20:58:21.707 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:58:21 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]': finished 2024-03-08T20:58:21.707 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:58:21 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 6]}]': finished 2024-03-08T20:58:21.707 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:58:21 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [5, 9]}]': finished 2024-03-08T20:58:21.707 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:58:21 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8]}]': finished 2024-03-08T20:58:21.707 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:58:21 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3b", "id": [3, 9]}]': finished 2024-03-08T20:58:21.707 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:58:21 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]': finished 2024-03-08T20:58:21.707 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:58:21 smithi149 ceph-mon[35042]: osdmap e87: 12 total, 12 up, 12 in 2024-03-08T20:58:21.718 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:58:21 smithi107 ceph-mon[31381]: pgmap v376: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 852 B/s wr, 9 op/s 2024-03-08T20:58:21.718 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:58:21 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]': finished 2024-03-08T20:58:21.718 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:58:21 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 4]}]': finished 2024-03-08T20:58:21.718 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:58:21 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]': finished 2024-03-08T20:58:21.718 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:58:21 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]': finished 2024-03-08T20:58:21.718 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:58:21 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 6]}]': finished 2024-03-08T20:58:21.718 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:58:21 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [5, 9]}]': finished 2024-03-08T20:58:21.718 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:58:21 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8]}]': finished 2024-03-08T20:58:21.719 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:58:21 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3b", "id": [3, 9]}]': finished 2024-03-08T20:58:21.719 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:58:21 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]': finished 2024-03-08T20:58:21.719 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:58:21 smithi107 ceph-mon[31381]: osdmap e87: 12 total, 12 up, 12 in 2024-03-08T20:58:21.879 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:58:21 smithi204 ceph-mon[34935]: pgmap v376: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 852 B/s wr, 9 op/s 2024-03-08T20:58:21.879 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:58:21 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]': finished 2024-03-08T20:58:21.880 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:58:21 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 4]}]': finished 2024-03-08T20:58:21.880 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:58:21 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]': finished 2024-03-08T20:58:21.880 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:58:21 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]': finished 2024-03-08T20:58:21.880 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:58:21 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 6]}]': finished 2024-03-08T20:58:21.880 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:58:21 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [5, 9]}]': finished 2024-03-08T20:58:21.880 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:58:21 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8]}]': finished 2024-03-08T20:58:21.880 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:58:21 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3b", "id": [3, 9]}]': finished 2024-03-08T20:58:21.880 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:58:21 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]': finished 2024-03-08T20:58:21.880 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:58:21 smithi204 ceph-mon[34935]: osdmap e87: 12 total, 12 up, 12 in 2024-03-08T20:58:22.555 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:58:22 smithi107 ceph-mon[31381]: osdmap e88: 12 total, 12 up, 12 in 2024-03-08T20:58:22.765 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:58:22 smithi149 ceph-mon[35042]: osdmap e88: 12 total, 12 up, 12 in 2024-03-08T20:58:22.878 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:58:22 smithi204 ceph-mon[34935]: osdmap e88: 12 total, 12 up, 12 in 2024-03-08T20:58:23.669 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:58:23 smithi107 ceph-mon[31381]: pgmap v379: 97 pgs: 1 peering, 96 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 511 B/s wr, 6 op/s; 146 B/s, 0 objects/s recovering 2024-03-08T20:58:23.765 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:58:23 smithi149 ceph-mon[35042]: pgmap v379: 97 pgs: 1 peering, 96 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 511 B/s wr, 6 op/s; 146 B/s, 0 objects/s recovering 2024-03-08T20:58:23.879 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:58:23 smithi204 ceph-mon[34935]: pgmap v379: 97 pgs: 1 peering, 96 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 511 B/s wr, 6 op/s; 146 B/s, 0 objects/s recovering 2024-03-08T20:58:25.765 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:58:25 smithi149 ceph-mon[35042]: pgmap v380: 97 pgs: 1 peering, 96 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 511 B/s wr, 5 op/s; 7.3 KiB/s, 1 objects/s recovering 2024-03-08T20:58:25.862 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:58:25 smithi107 ceph-mon[31381]: pgmap v380: 97 pgs: 1 peering, 96 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 511 B/s wr, 5 op/s; 7.3 KiB/s, 1 objects/s recovering 2024-03-08T20:58:25.878 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:58:25 smithi204 ceph-mon[34935]: pgmap v380: 97 pgs: 1 peering, 96 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 511 B/s wr, 5 op/s; 7.3 KiB/s, 1 objects/s recovering 2024-03-08T20:58:27.765 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:58:27 smithi149 ceph-mon[35042]: pgmap v381: 97 pgs: 97 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 3 op/s; 7.3 KiB/s, 1 objects/s recovering 2024-03-08T20:58:27.789 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:58:27 smithi107 ceph-mon[31381]: pgmap v381: 97 pgs: 97 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 3 op/s; 7.3 KiB/s, 1 objects/s recovering 2024-03-08T20:58:27.879 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:58:27 smithi204 ceph-mon[34935]: pgmap v381: 97 pgs: 97 active+clean; 640 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 3 op/s; 7.3 KiB/s, 1 objects/s recovering 2024-03-08T20:58:29.766 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:58:29 smithi149 ceph-mon[35042]: pgmap v382: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 383 B/s wr, 6 op/s; 7.3 KiB/s, 1 objects/s recovering 2024-03-08T20:58:29.796 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:58:29 smithi107 ceph-mon[31381]: pgmap v382: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 383 B/s wr, 6 op/s; 7.3 KiB/s, 1 objects/s recovering 2024-03-08T20:58:29.879 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:58:29 smithi204 ceph-mon[34935]: pgmap v382: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 383 B/s wr, 6 op/s; 7.3 KiB/s, 1 objects/s recovering 2024-03-08T20:58:31.410 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:31.154+0000 7f47fa600700 1 -- 172.21.15.107:0/525210628 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f47f4104230 msgr2=0x7f47f4104610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:58:31.410 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:31.154+0000 7f47fa600700 1 --2- 172.21.15.107:0/525210628 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f47f4104230 0x7f47f4104610 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f47e40099c0 tx=0x7f47e4009cd0 comp rx=0 tx=0).stop 2024-03-08T20:58:31.410 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:31.154+0000 7f47fa600700 1 -- 172.21.15.107:0/525210628 shutdown_connections 2024-03-08T20:58:31.410 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:31.154+0000 7f47fa600700 1 --2- 172.21.15.107:0/525210628 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f47f4109310 0x7f47f4110ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:58:31.411 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:31.154+0000 7f47fa600700 1 --2- 172.21.15.107:0/525210628 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f47f4104230 0x7f47f4104610 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:58:31.411 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:31.154+0000 7f47fa600700 1 --2- 172.21.15.107:0/525210628 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f47f4104be0 0x7f47f4108c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:58:31.411 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:31.154+0000 7f47fa600700 1 -- 172.21.15.107:0/525210628 >> 172.21.15.107:0/525210628 conn(0x7f47f40ffac0 msgr2=0x7f47f4101ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:58:31.411 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:31.154+0000 7f47fa600700 1 -- 172.21.15.107:0/525210628 shutdown_connections 2024-03-08T20:58:31.411 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:31.154+0000 7f47fa600700 1 -- 172.21.15.107:0/525210628 wait complete. 2024-03-08T20:58:31.411 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:31.156+0000 7f47fa600700 1 Processor -- start 2024-03-08T20:58:31.411 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:31.156+0000 7f47fa600700 1 -- start start 2024-03-08T20:58:31.411 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:31.156+0000 7f47fa600700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f47f4104230 0x7f47f419a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:58:31.411 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:31.156+0000 7f47fa600700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f47f4104be0 0x7f47f419ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:58:31.411 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:31.156+0000 7f47fa600700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f47f4109310 0x7f47f419efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:58:31.411 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:31.156+0000 7f47fa600700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47f4114220 con 0x7f47f4109310 2024-03-08T20:58:31.411 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:31.156+0000 7f47fa600700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47f41140a0 con 0x7f47f4104230 2024-03-08T20:58:31.412 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:31.156+0000 7f47fa600700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47f41143a0 con 0x7f47f4104be0 2024-03-08T20:58:31.412 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:31.157+0000 7f47f3e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f47f4109310 0x7f47f419efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:58:31.412 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:31.157+0000 7f47f3e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f47f4109310 0x7f47f419efd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:54054/0 (socket says 172.21.15.107:54054) 2024-03-08T20:58:31.412 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:31.157+0000 7f47f3e00700 1 -- 172.21.15.107:0/3175134464 learned_addr learned my addr 172.21.15.107:0/3175134464 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T20:58:31.412 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:31.157+0000 7f47f3400700 1 --2- 172.21.15.107:0/3175134464 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f47f4104230 0x7f47f419a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:58:31.412 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:31.157+0000 7f47f2a00700 1 --2- 172.21.15.107:0/3175134464 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f47f4104be0 0x7f47f419ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:58:31.412 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:31.157+0000 7f47f3e00700 1 -- 172.21.15.107:0/3175134464 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f47f4104be0 msgr2=0x7f47f419ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:58:31.412 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:31.157+0000 7f47f3e00700 1 --2- 172.21.15.107:0/3175134464 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f47f4104be0 0x7f47f419ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:58:31.412 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:31.157+0000 7f47f3e00700 1 -- 172.21.15.107:0/3175134464 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f47f4104230 msgr2=0x7f47f419a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:58:31.412 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:31.157+0000 7f47f3e00700 1 --2- 172.21.15.107:0/3175134464 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f47f4104230 0x7f47f419a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:58:31.412 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:31.157+0000 7f47f3e00700 1 -- 172.21.15.107:0/3175134464 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47f419f710 con 0x7f47f4109310 2024-03-08T20:58:31.412 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:31.157+0000 7f47f3e00700 1 --2- 172.21.15.107:0/3175134464 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f47f4109310 0x7f47f419efd0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f47ec00ada0 tx=0x7f47ec00c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:58:31.412 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:31.158+0000 7f47ebe00700 1 -- 172.21.15.107:0/3175134464 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47ec017020 con 0x7f47f4109310 2024-03-08T20:58:31.413 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:31.158+0000 7f47ebe00700 1 -- 172.21.15.107:0/3175134464 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f47ec00ce30 con 0x7f47f4109310 2024-03-08T20:58:31.413 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:31.158+0000 7f47fa600700 1 -- 172.21.15.107:0/3175134464 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47f419fa00 con 0x7f47f4109310 2024-03-08T20:58:31.413 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:31.158+0000 7f47ebe00700 1 -- 172.21.15.107:0/3175134464 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47ec005460 con 0x7f47f4109310 2024-03-08T20:58:31.413 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:31.158+0000 7f47fa600700 1 -- 172.21.15.107:0/3175134464 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47f41a7280 con 0x7f47f4109310 2024-03-08T20:58:31.413 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:31.159+0000 7f47fa600700 1 -- 172.21.15.107:0/3175134464 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f47f4062530 con 0x7f47f4109310 2024-03-08T20:58:31.413 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:31.161+0000 7f47ebe00700 1 -- 172.21.15.107:0/3175134464 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) v1 ==== 137342+0+0 (secure 0 0 0) 0x7f47ec00c980 con 0x7f47f4109310 2024-03-08T20:58:31.413 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:31.162+0000 7f47ebe00700 1 --2- 172.21.15.107:0/3175134464 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f47d80a29a0 0x7f47d80a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:58:31.413 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:31.162+0000 7f47ebe00700 1 -- 172.21.15.107:0/3175134464 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f47ec0ced80 con 0x7f47f4109310 2024-03-08T20:58:31.413 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:31.162+0000 7f47f3400700 1 --2- 172.21.15.107:0/3175134464 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f47d80a29a0 0x7f47d80a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:58:31.413 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:31.163+0000 7f47ebe00700 1 --2- 172.21.15.107:0/3175134464 >> [v2:172.21.15.149:6800/3071316419,v1:172.21.15.149:6801/3071316419] conn(0x7f47d80ac560 0x7f47d80aea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:58:31.413 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:31.163+0000 7f47ebe00700 1 -- 172.21.15.107:0/3175134464 --> [v2:172.21.15.149:6800/3071316419,v1:172.21.15.149:6801/3071316419] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f47ec022e50 con 0x7f47d80ac560 2024-03-08T20:58:31.413 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:31.163+0000 7f47ebe00700 1 -- 172.21.15.107:0/3175134464 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) v2 ==== 24+0+0 (secure 0 0 0) 0x7f47ec0cf130 con 0x7f47f4109310 2024-03-08T20:58:31.413 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:31.163+0000 7f47f3400700 1 --2- 172.21.15.107:0/3175134464 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f47d80a29a0 0x7f47d80a4e60 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f47e4009fd0 tx=0x7f47e4005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:58:31.413 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:31.163+0000 7f47f2a00700 1 --2- 172.21.15.107:0/3175134464 >> [v2:172.21.15.149:6800/3071316419,v1:172.21.15.149:6801/3071316419] conn(0x7f47d80ac560 0x7f47d80aea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:58:31.414 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:31.164+0000 7f47f2a00700 1 --2- 172.21.15.107:0/3175134464 >> [v2:172.21.15.149:6800/3071316419,v1:172.21.15.149:6801/3071316419] conn(0x7f47d80ac560 0x7f47d80aea00 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:58:31.414 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:31.167+0000 7f47ebe00700 1 -- 172.21.15.107:0/3175134464 <== osd.1 v2:172.21.15.149:6800/3071316419 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f47ec022e50 con 0x7f47d80ac560 2024-03-08T20:58:31.414 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:31.185+0000 7f47fa600700 1 -- 172.21.15.107:0/3175134464 --> [v2:172.21.15.149:6800/3071316419,v1:172.21.15.149:6801/3071316419] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f47f4062960 con 0x7f47d80ac560 2024-03-08T20:58:31.414 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:31.186+0000 7f47ebe00700 1 -- 172.21.15.107:0/3175134464 <== osd.1 v2:172.21.15.149:6800/3071316419 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f47f4062960 con 0x7f47d80ac560 2024-03-08T20:58:31.414 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:31.186+0000 7f47fa600700 1 -- 172.21.15.107:0/3175134464 >> [v2:172.21.15.149:6800/3071316419,v1:172.21.15.149:6801/3071316419] conn(0x7f47d80ac560 msgr2=0x7f47d80aea00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:58:31.414 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:31.186+0000 7f47fa600700 1 --2- 172.21.15.107:0/3175134464 >> [v2:172.21.15.149:6800/3071316419,v1:172.21.15.149:6801/3071316419] conn(0x7f47d80ac560 0x7f47d80aea00 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:58:31.415 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:31.187+0000 7f47fa600700 1 -- 172.21.15.107:0/3175134464 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f47d80a29a0 msgr2=0x7f47d80a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:58:31.415 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:31.187+0000 7f47fa600700 1 --2- 172.21.15.107:0/3175134464 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f47d80a29a0 0x7f47d80a4e60 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f47e4009fd0 tx=0x7f47e4005fb0 comp rx=0 tx=0).stop 2024-03-08T20:58:31.415 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:31.187+0000 7f47fa600700 1 -- 172.21.15.107:0/3175134464 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f47f4109310 msgr2=0x7f47f419efd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:58:31.415 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:31.187+0000 7f47fa600700 1 --2- 172.21.15.107:0/3175134464 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f47f4109310 0x7f47f419efd0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f47ec00ada0 tx=0x7f47ec00c330 comp rx=0 tx=0).stop 2024-03-08T20:58:31.415 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:31.187+0000 7f47f3e00700 1 -- 172.21.15.107:0/3175134464 reap_dead start 2024-03-08T20:58:31.415 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:31.187+0000 7f47fa600700 1 -- 172.21.15.107:0/3175134464 shutdown_connections 2024-03-08T20:58:31.415 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:31.187+0000 7f47fa600700 1 -- 172.21.15.107:0/3175134464 >> 172.21.15.107:0/3175134464 conn(0x7f47f40ffac0 msgr2=0x7f47f4101dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:58:31.415 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:31.187+0000 7f47fa600700 1 -- 172.21.15.107:0/3175134464 shutdown_connections 2024-03-08T20:58:31.415 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:31.187+0000 7f47fa600700 1 -- 172.21.15.107:0/3175134464 wait complete. 2024-03-08T20:58:31.703 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:58:31 smithi107 ceph-mon[31381]: pgmap v383: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 311 B/s wr, 5 op/s; 5.9 KiB/s, 1 objects/s recovering 2024-03-08T20:58:31.765 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:58:31 smithi149 ceph-mon[35042]: pgmap v383: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 311 B/s wr, 5 op/s; 5.9 KiB/s, 1 objects/s recovering 2024-03-08T20:58:31.878 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:58:31 smithi204 ceph-mon[34935]: pgmap v383: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 311 B/s wr, 5 op/s; 5.9 KiB/s, 1 objects/s recovering 2024-03-08T20:58:33.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:58:33 smithi107 ceph-mon[31381]: pgmap v384: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 282 B/s wr, 6 op/s; 5.3 KiB/s, 1 objects/s recovering 2024-03-08T20:58:33.765 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:58:33 smithi149 ceph-mon[35042]: pgmap v384: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 282 B/s wr, 6 op/s; 5.3 KiB/s, 1 objects/s recovering 2024-03-08T20:58:33.878 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:58:33 smithi204 ceph-mon[34935]: pgmap v384: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 282 B/s wr, 6 op/s; 5.3 KiB/s, 1 objects/s recovering 2024-03-08T20:58:33.903 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:32.315+0000 7f2824a00700 1 -- 172.21.15.107:0/4128648813 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2820108db0 msgr2=0x7f28201152c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:58:33.903 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:32.315+0000 7f2824a00700 1 --2- 172.21.15.107:0/4128648813 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2820108db0 0x7f28201152c0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f281800b0a0 tx=0x7f281800b3b0 comp rx=0 tx=0).stop 2024-03-08T20:58:33.903 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:32.315+0000 7f2824a00700 1 -- 172.21.15.107:0/4128648813 shutdown_connections 2024-03-08T20:58:33.903 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:32.316+0000 7f2824a00700 1 --2- 172.21.15.107:0/4128648813 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2820108db0 0x7f28201152c0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:58:33.903 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:32.316+0000 7f2824a00700 1 --2- 172.21.15.107:0/4128648813 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f282010e480 0x7f282010e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:58:33.903 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:32.316+0000 7f2824a00700 1 --2- 172.21.15.107:0/4128648813 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f28201083f0 0x7f2820108870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:58:33.903 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:32.316+0000 7f2824a00700 1 -- 172.21.15.107:0/4128648813 >> 172.21.15.107:0/4128648813 conn(0x7f28200755b0 msgr2=0x7f28200779d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:58:33.903 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:32.316+0000 7f2824a00700 1 -- 172.21.15.107:0/4128648813 shutdown_connections 2024-03-08T20:58:33.903 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:32.316+0000 7f2824a00700 1 -- 172.21.15.107:0/4128648813 wait complete. 2024-03-08T20:58:33.904 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:32.317+0000 7f2824a00700 1 Processor -- start 2024-03-08T20:58:33.904 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:32.317+0000 7f2824a00700 1 -- start start 2024-03-08T20:58:33.904 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:32.317+0000 7f2824a00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f28201083f0 0x7f282019ead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:58:33.904 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:32.317+0000 7f2824a00700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2820108db0 0x7f282019f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:58:33.904 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:32.317+0000 7f2824a00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f282010e480 0x7f28201a33d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:58:33.904 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:32.317+0000 7f2824a00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28201185c0 con 0x7f282010e480 2024-03-08T20:58:33.904 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:32.317+0000 7f2824a00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2820118440 con 0x7f28201083f0 2024-03-08T20:58:33.904 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:32.317+0000 7f2824a00700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2820118740 con 0x7f2820108db0 2024-03-08T20:58:33.904 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:32.318+0000 7f281e800700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f282010e480 0x7f28201a33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:58:33.904 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:32.318+0000 7f281e800700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f282010e480 0x7f28201a33d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:54076/0 (socket says 172.21.15.107:54076) 2024-03-08T20:58:33.904 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:32.318+0000 7f281e800700 1 -- 172.21.15.107:0/3193998252 learned_addr learned my addr 172.21.15.107:0/3193998252 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T20:58:33.904 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:32.318+0000 7f281de00700 1 --2- 172.21.15.107:0/3193998252 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f28201083f0 0x7f282019ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:58:33.904 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:32.318+0000 7f281d400700 1 --2- 172.21.15.107:0/3193998252 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2820108db0 0x7f282019f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:58:33.905 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:32.318+0000 7f281e800700 1 -- 172.21.15.107:0/3193998252 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2820108db0 msgr2=0x7f282019f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:58:33.905 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:32.318+0000 7f281e800700 1 --2- 172.21.15.107:0/3193998252 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2820108db0 0x7f282019f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:58:33.905 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:32.318+0000 7f281e800700 1 -- 172.21.15.107:0/3193998252 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f28201083f0 msgr2=0x7f282019ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:58:33.905 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:32.318+0000 7f281e800700 1 --2- 172.21.15.107:0/3193998252 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f28201083f0 0x7f282019ead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:58:33.905 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:32.318+0000 7f281e800700 1 -- 172.21.15.107:0/3193998252 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28201a3ad0 con 0x7f282010e480 2024-03-08T20:58:33.905 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:32.319+0000 7f281e800700 1 --2- 172.21.15.107:0/3193998252 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f282010e480 0x7f28201a33d0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f281800b830 tx=0x7f2818004d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:58:33.905 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:32.319+0000 7f2816a00700 1 -- 172.21.15.107:0/3193998252 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2818021070 con 0x7f282010e480 2024-03-08T20:58:33.905 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:32.319+0000 7f2816a00700 1 -- 172.21.15.107:0/3193998252 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f2818003c00 con 0x7f282010e480 2024-03-08T20:58:33.905 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:32.319+0000 7f2824a00700 1 -- 172.21.15.107:0/3193998252 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28201a3d60 con 0x7f282010e480 2024-03-08T20:58:33.905 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:32.319+0000 7f2816a00700 1 -- 172.21.15.107:0/3193998252 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2818004790 con 0x7f282010e480 2024-03-08T20:58:33.905 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:32.319+0000 7f2824a00700 1 -- 172.21.15.107:0/3193998252 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28201ab680 con 0x7f282010e480 2024-03-08T20:58:33.905 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:32.320+0000 7f2824a00700 1 -- 172.21.15.107:0/3193998252 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f28200624b0 con 0x7f282010e480 2024-03-08T20:58:33.905 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:32.322+0000 7f2816a00700 1 -- 172.21.15.107:0/3193998252 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) v1 ==== 137342+0+0 (secure 0 0 0) 0x7f2818007a00 con 0x7f282010e480 2024-03-08T20:58:33.905 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:32.323+0000 7f2816a00700 1 --2- 172.21.15.107:0/3193998252 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f28040a2920 0x7f28040a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:58:33.906 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:32.324+0000 7f2816a00700 1 -- 172.21.15.107:0/3193998252 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f28180d0260 con 0x7f282010e480 2024-03-08T20:58:33.906 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:32.324+0000 7f281de00700 1 --2- 172.21.15.107:0/3193998252 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f28040a2920 0x7f28040a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:58:33.906 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:32.324+0000 7f2816a00700 1 --2- 172.21.15.107:0/3193998252 >> [v2:172.21.15.107:6818/2912452950,v1:172.21.15.107:6819/2912452950] conn(0x7f28040ac4e0 0x7f28040ae980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:58:33.906 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:32.324+0000 7f2816a00700 1 -- 172.21.15.107:0/3193998252 --> [v2:172.21.15.107:6818/2912452950,v1:172.21.15.107:6819/2912452950] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f28040af050 con 0x7f28040ac4e0 2024-03-08T20:58:33.906 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:32.324+0000 7f2816a00700 1 -- 172.21.15.107:0/3193998252 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) v2 ==== 24+0+0 (secure 0 0 0) 0x7f28180d0610 con 0x7f282010e480 2024-03-08T20:58:33.906 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:32.324+0000 7f281d400700 1 --2- 172.21.15.107:0/3193998252 >> [v2:172.21.15.107:6818/2912452950,v1:172.21.15.107:6819/2912452950] conn(0x7f28040ac4e0 0x7f28040ae980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:58:33.906 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:32.324+0000 7f281de00700 1 --2- 172.21.15.107:0/3193998252 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f28040a2920 0x7f28040a4de0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f2810005950 tx=0x7f28100058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:58:33.906 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:32.325+0000 7f281d400700 1 --2- 172.21.15.107:0/3193998252 >> [v2:172.21.15.107:6818/2912452950,v1:172.21.15.107:6819/2912452950] conn(0x7f28040ac4e0 0x7f28040ae980 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:58:33.906 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:32.326+0000 7f2816a00700 1 -- 172.21.15.107:0/3193998252 <== osd.6 v2:172.21.15.107:6818/2912452950 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f28040af050 con 0x7f28040ac4e0 2024-03-08T20:58:33.906 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:32.344+0000 7f2824a00700 1 -- 172.21.15.107:0/3193998252 --> [v2:172.21.15.107:6818/2912452950,v1:172.21.15.107:6819/2912452950] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f2820062930 con 0x7f28040ac4e0 2024-03-08T20:58:33.907 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:32.344+0000 7f2816a00700 1 -- 172.21.15.107:0/3193998252 <== osd.6 v2:172.21.15.107:6818/2912452950 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f2820062930 con 0x7f28040ac4e0 2024-03-08T20:58:33.907 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:32.345+0000 7f2824a00700 1 -- 172.21.15.107:0/3193998252 >> [v2:172.21.15.107:6818/2912452950,v1:172.21.15.107:6819/2912452950] conn(0x7f28040ac4e0 msgr2=0x7f28040ae980 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:58:33.907 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:32.345+0000 7f2824a00700 1 --2- 172.21.15.107:0/3193998252 >> [v2:172.21.15.107:6818/2912452950,v1:172.21.15.107:6819/2912452950] conn(0x7f28040ac4e0 0x7f28040ae980 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:58:33.907 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:32.345+0000 7f2824a00700 1 -- 172.21.15.107:0/3193998252 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f28040a2920 msgr2=0x7f28040a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:58:33.907 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:32.345+0000 7f2824a00700 1 --2- 172.21.15.107:0/3193998252 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f28040a2920 0x7f28040a4de0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f2810005950 tx=0x7f28100058e0 comp rx=0 tx=0).stop 2024-03-08T20:58:33.907 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:32.345+0000 7f2824a00700 1 -- 172.21.15.107:0/3193998252 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f282010e480 msgr2=0x7f28201a33d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:58:33.907 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:32.345+0000 7f2824a00700 1 --2- 172.21.15.107:0/3193998252 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f282010e480 0x7f28201a33d0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f281800b830 tx=0x7f2818004d90 comp rx=0 tx=0).stop 2024-03-08T20:58:33.907 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:32.345+0000 7f281e800700 1 -- 172.21.15.107:0/3193998252 reap_dead start 2024-03-08T20:58:33.907 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:32.345+0000 7f2824a00700 1 -- 172.21.15.107:0/3193998252 shutdown_connections 2024-03-08T20:58:33.908 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:32.345+0000 7f2824a00700 1 -- 172.21.15.107:0/3193998252 >> 172.21.15.107:0/3193998252 conn(0x7f28200755b0 msgr2=0x7f2820075ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:58:33.908 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:32.345+0000 7f2824a00700 1 -- 172.21.15.107:0/3193998252 shutdown_connections 2024-03-08T20:58:33.908 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:32.345+0000 7f2824a00700 1 -- 172.21.15.107:0/3193998252 wait complete. 2024-03-08T20:58:35.576 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:58:35 smithi107 ceph-mon[31381]: pgmap v385: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s; 4.8 KiB/s, 0 objects/s recovering 2024-03-08T20:58:35.664 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:35.680+0000 7fef5d600700 1 -- 172.21.15.107:0/1259842624 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fef58102090 msgr2=0x7fef58102470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:58:35.664 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:35.680+0000 7fef5d600700 1 --2- 172.21.15.107:0/1259842624 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fef58102090 0x7fef58102470 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fef480099c0 tx=0x7fef48009cd0 comp rx=0 tx=0).stop 2024-03-08T20:58:35.664 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:35.680+0000 7fef5d600700 1 -- 172.21.15.107:0/1259842624 shutdown_connections 2024-03-08T20:58:35.664 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:35.680+0000 7fef5d600700 1 --2- 172.21.15.107:0/1259842624 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fef58107160 0x7fef5810ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:58:35.664 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:35.680+0000 7fef5d600700 1 --2- 172.21.15.107:0/1259842624 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fef58102090 0x7fef58102470 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:58:35.664 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:35.680+0000 7fef5d600700 1 --2- 172.21.15.107:0/1259842624 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fef58102a40 0x7fef58106a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:58:35.664 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:35.680+0000 7fef5d600700 1 -- 172.21.15.107:0/1259842624 >> 172.21.15.107:0/1259842624 conn(0x7fef58078f40 msgr2=0x7fef58079350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:58:35.664 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:35.681+0000 7fef5d600700 1 -- 172.21.15.107:0/1259842624 shutdown_connections 2024-03-08T20:58:35.664 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:35.681+0000 7fef5d600700 1 -- 172.21.15.107:0/1259842624 wait complete. 2024-03-08T20:58:35.665 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:35.682+0000 7fef5d600700 1 Processor -- start 2024-03-08T20:58:35.666 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:35.682+0000 7fef5d600700 1 -- start start 2024-03-08T20:58:35.666 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:35.682+0000 7fef5d600700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fef58102090 0x7fef581a0460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:58:35.666 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:35.682+0000 7fef5d600700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fef58102a40 0x7fef581a09a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:58:35.666 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:35.682+0000 7fef5d600700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fef58107160 0x7fef5819a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:58:35.666 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:35.682+0000 7fef5d600700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef581120c0 con 0x7fef58102090 2024-03-08T20:58:35.666 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:35.682+0000 7fef5d600700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef58111f40 con 0x7fef58107160 2024-03-08T20:58:35.666 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:35.682+0000 7fef5d600700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef58112240 con 0x7fef58102a40 2024-03-08T20:58:35.666 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:35.682+0000 7fef56a00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fef58102090 0x7fef581a0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:58:35.666 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:35.683+0000 7fef56a00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fef58102090 0x7fef581a0460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:54126/0 (socket says 172.21.15.107:54126) 2024-03-08T20:58:35.666 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:35.683+0000 7fef56a00700 1 -- 172.21.15.107:0/1219068717 learned_addr learned my addr 172.21.15.107:0/1219068717 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T20:58:35.667 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:35.683+0000 7fef56a00700 1 -- 172.21.15.107:0/1219068717 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fef58102a40 msgr2=0x7fef581a09a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:58:35.667 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:35.683+0000 7fef56000700 1 --2- 172.21.15.107:0/1219068717 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fef58102a40 0x7fef581a09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:58:35.667 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:35.683+0000 7fef57400700 1 --2- 172.21.15.107:0/1219068717 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fef58107160 0x7fef5819a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:58:35.668 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:35.683+0000 7fef56a00700 1 --2- 172.21.15.107:0/1219068717 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fef58102a40 0x7fef581a09a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:58:35.668 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:35.683+0000 7fef56a00700 1 -- 172.21.15.107:0/1219068717 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fef58107160 msgr2=0x7fef5819a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:58:35.668 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:35.683+0000 7fef56a00700 1 --2- 172.21.15.107:0/1219068717 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fef58107160 0x7fef5819a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:58:35.668 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:35.683+0000 7fef56a00700 1 -- 172.21.15.107:0/1219068717 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef5819ad80 con 0x7fef58102090 2024-03-08T20:58:35.668 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:35.683+0000 7fef57400700 1 --2- 172.21.15.107:0/1219068717 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fef58107160 0x7fef5819a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T20:58:35.668 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:35.683+0000 7fef56a00700 1 --2- 172.21.15.107:0/1219068717 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fef58102090 0x7fef581a0460 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fef48009990 tx=0x7fef48003730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:58:35.668 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:35.684+0000 7fef4f400700 1 -- 172.21.15.107:0/1219068717 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef48026070 con 0x7fef58102090 2024-03-08T20:58:35.668 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:35.684+0000 7fef4f400700 1 -- 172.21.15.107:0/1219068717 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fef480040d0 con 0x7fef58102090 2024-03-08T20:58:35.668 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:35.684+0000 7fef5d600700 1 -- 172.21.15.107:0/1219068717 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef5819b010 con 0x7fef58102090 2024-03-08T20:58:35.668 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:35.684+0000 7fef4f400700 1 -- 172.21.15.107:0/1219068717 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef48005290 con 0x7fef58102090 2024-03-08T20:58:35.668 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:35.684+0000 7fef5d600700 1 -- 172.21.15.107:0/1219068717 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef581a7330 con 0x7fef58102090 2024-03-08T20:58:35.669 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:35.685+0000 7fef5d600700 1 -- 172.21.15.107:0/1219068717 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fef580752f0 con 0x7fef58102090 2024-03-08T20:58:35.670 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:35.687+0000 7fef4f400700 1 -- 172.21.15.107:0/1219068717 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) v1 ==== 137342+0+0 (secure 0 0 0) 0x7fef48029070 con 0x7fef58102090 2024-03-08T20:58:35.671 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:35.688+0000 7fef4f400700 1 --2- 172.21.15.107:0/1219068717 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fef3c0a2920 0x7fef3c0a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:58:35.671 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:35.688+0000 7fef56000700 1 --2- 172.21.15.107:0/1219068717 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fef3c0a2920 0x7fef3c0a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:58:35.671 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:35.688+0000 7fef4f400700 1 -- 172.21.15.107:0/1219068717 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fef480cff40 con 0x7fef58102090 2024-03-08T20:58:35.672 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:35.689+0000 7fef4f400700 1 --2- 172.21.15.107:0/1219068717 >> [v2:172.21.15.149:6816/3835863809,v1:172.21.15.149:6817/3835863809] conn(0x7fef3c0ac4e0 0x7fef3c0ae980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:58:35.672 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:35.689+0000 7fef4f400700 1 -- 172.21.15.107:0/1219068717 --> [v2:172.21.15.149:6816/3835863809,v1:172.21.15.149:6817/3835863809] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fef3c0af050 con 0x7fef3c0ac4e0 2024-03-08T20:58:35.672 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:35.689+0000 7fef4f400700 1 -- 172.21.15.107:0/1219068717 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) v2 ==== 24+0+0 (secure 0 0 0) 0x7fef480d02f0 con 0x7fef58102090 2024-03-08T20:58:35.672 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:35.689+0000 7fef56000700 1 --2- 172.21.15.107:0/1219068717 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fef3c0a2920 0x7fef3c0a4de0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fef44006fd0 tx=0x7fef44008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:58:35.672 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:35.689+0000 7fef57400700 1 --2- 172.21.15.107:0/1219068717 >> [v2:172.21.15.149:6816/3835863809,v1:172.21.15.149:6817/3835863809] conn(0x7fef3c0ac4e0 0x7fef3c0ae980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:58:35.673 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:35.690+0000 7fef57400700 1 --2- 172.21.15.107:0/1219068717 >> [v2:172.21.15.149:6816/3835863809,v1:172.21.15.149:6817/3835863809] conn(0x7fef3c0ac4e0 0x7fef3c0ae980 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:58:35.676 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:35.693+0000 7fef4f400700 1 -- 172.21.15.107:0/1219068717 <== osd.7 v2:172.21.15.149:6816/3835863809 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fef3c0af050 con 0x7fef3c0ac4e0 2024-03-08T20:58:35.693 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:35.711+0000 7fef5d600700 1 -- 172.21.15.107:0/1219068717 --> [v2:172.21.15.149:6816/3835863809,v1:172.21.15.149:6817/3835863809] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fef58103560 con 0x7fef3c0ac4e0 2024-03-08T20:58:35.694 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:35.712+0000 7fef4f400700 1 -- 172.21.15.107:0/1219068717 <== osd.7 v2:172.21.15.149:6816/3835863809 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fef58103560 con 0x7fef3c0ac4e0 2024-03-08T20:58:35.696 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:35.712+0000 7fef5d600700 1 -- 172.21.15.107:0/1219068717 >> [v2:172.21.15.149:6816/3835863809,v1:172.21.15.149:6817/3835863809] conn(0x7fef3c0ac4e0 msgr2=0x7fef3c0ae980 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:58:35.696 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:35.712+0000 7fef5d600700 1 --2- 172.21.15.107:0/1219068717 >> [v2:172.21.15.149:6816/3835863809,v1:172.21.15.149:6817/3835863809] conn(0x7fef3c0ac4e0 0x7fef3c0ae980 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:58:35.696 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:35.712+0000 7fef5d600700 1 -- 172.21.15.107:0/1219068717 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fef3c0a2920 msgr2=0x7fef3c0a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:58:35.696 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:35.712+0000 7fef5d600700 1 --2- 172.21.15.107:0/1219068717 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fef3c0a2920 0x7fef3c0a4de0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fef44006fd0 tx=0x7fef44008040 comp rx=0 tx=0).stop 2024-03-08T20:58:35.696 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:35.712+0000 7fef5d600700 1 -- 172.21.15.107:0/1219068717 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fef58102090 msgr2=0x7fef581a0460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:58:35.696 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:35.712+0000 7fef5d600700 1 --2- 172.21.15.107:0/1219068717 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fef58102090 0x7fef581a0460 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fef48009990 tx=0x7fef48003730 comp rx=0 tx=0).stop 2024-03-08T20:58:35.696 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:35.712+0000 7fef57400700 1 -- 172.21.15.107:0/1219068717 reap_dead start 2024-03-08T20:58:35.696 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:35.713+0000 7fef5d600700 1 -- 172.21.15.107:0/1219068717 shutdown_connections 2024-03-08T20:58:35.696 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:35.713+0000 7fef5d600700 1 -- 172.21.15.107:0/1219068717 >> 172.21.15.107:0/1219068717 conn(0x7fef58078f40 msgr2=0x7fef581001d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:58:35.696 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:35.713+0000 7fef5d600700 1 -- 172.21.15.107:0/1219068717 shutdown_connections 2024-03-08T20:58:35.696 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:35.713+0000 7fef5d600700 1 -- 172.21.15.107:0/1219068717 wait complete. 2024-03-08T20:58:35.765 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:58:35 smithi149 ceph-mon[35042]: pgmap v385: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s; 4.8 KiB/s, 0 objects/s recovering 2024-03-08T20:58:35.878 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:58:35 smithi204 ceph-mon[34935]: pgmap v385: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s; 4.8 KiB/s, 0 objects/s recovering 2024-03-08T20:58:36.349 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:35.585+0000 7f62f7c00700 1 -- 172.21.15.107:0/927129628 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f62f0074930 msgr2=0x7f62f010e2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:58:36.349 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:35.585+0000 7f62f7c00700 1 --2- 172.21.15.107:0/927129628 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f62f0074930 0x7f62f010e2a0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f62dc009990 tx=0x7f62dc009ca0 comp rx=0 tx=0).stop 2024-03-08T20:58:36.349 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:35.586+0000 7f62f7c00700 1 -- 172.21.15.107:0/927129628 shutdown_connections 2024-03-08T20:58:36.349 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:35.586+0000 7f62f7c00700 1 --2- 172.21.15.107:0/927129628 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f62f0074010 0x7f62f00743f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:58:36.349 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:35.586+0000 7f62f7c00700 1 --2- 172.21.15.107:0/927129628 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f62f010e7e0 0x7f62f0110bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:58:36.349 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:35.586+0000 7f62f7c00700 1 --2- 172.21.15.107:0/927129628 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f62f0074930 0x7f62f010e2a0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:58:36.349 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:35.586+0000 7f62f7c00700 1 -- 172.21.15.107:0/927129628 >> 172.21.15.107:0/927129628 conn(0x7f62f0078f40 msgr2=0x7f62f0079350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:58:36.349 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:35.586+0000 7f62f7c00700 1 -- 172.21.15.107:0/927129628 shutdown_connections 2024-03-08T20:58:36.350 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:35.586+0000 7f62f7c00700 1 -- 172.21.15.107:0/927129628 wait complete. 2024-03-08T20:58:36.350 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:35.587+0000 7f62f7c00700 1 Processor -- start 2024-03-08T20:58:36.350 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:35.587+0000 7f62f7c00700 1 -- start start 2024-03-08T20:58:36.350 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:35.587+0000 7f62f7c00700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f62f0074010 0x7f62f019a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:58:36.350 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:35.587+0000 7f62f7c00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f62f0074930 0x7f62f019ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:58:36.350 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:35.587+0000 7f62f7c00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f62f010e7e0 0x7f62f019efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:58:36.350 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:35.588+0000 7f62f7c00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62f01142a0 con 0x7f62f0074930 2024-03-08T20:58:36.350 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:35.588+0000 7f62f7c00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62f0114120 con 0x7f62f010e7e0 2024-03-08T20:58:36.350 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:35.588+0000 7f62f7c00700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62f0114420 con 0x7f62f0074010 2024-03-08T20:58:36.350 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:35.588+0000 7f62efe00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f62f0074930 0x7f62f019ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:58:36.350 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:35.588+0000 7f62efe00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f62f0074930 0x7f62f019ac70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:54108/0 (socket says 172.21.15.107:54108) 2024-03-08T20:58:36.350 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:35.588+0000 7f62efe00700 1 -- 172.21.15.107:0/2179020166 learned_addr learned my addr 172.21.15.107:0/2179020166 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T20:58:36.350 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:35.588+0000 7f62f5c00700 1 --2- 172.21.15.107:0/2179020166 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f62f010e7e0 0x7f62f019efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:58:36.351 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:35.588+0000 7f62f5200700 1 --2- 172.21.15.107:0/2179020166 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f62f0074010 0x7f62f019a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:58:36.351 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:35.588+0000 7f62efe00700 1 -- 172.21.15.107:0/2179020166 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f62f0074010 msgr2=0x7f62f019a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:58:36.351 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:35.588+0000 7f62efe00700 1 --2- 172.21.15.107:0/2179020166 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f62f0074010 0x7f62f019a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:58:36.351 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:35.588+0000 7f62efe00700 1 -- 172.21.15.107:0/2179020166 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f62f010e7e0 msgr2=0x7f62f019efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:58:36.351 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:35.588+0000 7f62efe00700 1 --2- 172.21.15.107:0/2179020166 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f62f010e7e0 0x7f62f019efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:58:36.351 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:35.588+0000 7f62efe00700 1 -- 172.21.15.107:0/2179020166 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62f019f6f0 con 0x7f62f0074930 2024-03-08T20:58:36.351 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:35.589+0000 7f62efe00700 1 --2- 172.21.15.107:0/2179020166 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f62f0074930 0x7f62f019ac70 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f62dc000c00 tx=0x7f62dc0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:58:36.351 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:35.589+0000 7f62ed600700 1 -- 172.21.15.107:0/2179020166 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62dc027070 con 0x7f62f0074930 2024-03-08T20:58:36.351 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:35.589+0000 7f62f7c00700 1 -- 172.21.15.107:0/2179020166 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62f019f980 con 0x7f62f0074930 2024-03-08T20:58:36.351 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:35.589+0000 7f62ed600700 1 -- 172.21.15.107:0/2179020166 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f62dc0040c0 con 0x7f62f0074930 2024-03-08T20:58:36.351 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:35.589+0000 7f62ed600700 1 -- 172.21.15.107:0/2179020166 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62dc00f460 con 0x7f62f0074930 2024-03-08T20:58:36.351 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:35.589+0000 7f62f7c00700 1 -- 172.21.15.107:0/2179020166 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62f01a7260 con 0x7f62f0074930 2024-03-08T20:58:36.351 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:35.590+0000 7f62f7c00700 1 -- 172.21.15.107:0/2179020166 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f62f00ff9d0 con 0x7f62f0074930 2024-03-08T20:58:36.352 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:35.593+0000 7f62ed600700 1 -- 172.21.15.107:0/2179020166 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) v1 ==== 137342+0+0 (secure 0 0 0) 0x7f62dc02a070 con 0x7f62f0074930 2024-03-08T20:58:36.352 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:35.594+0000 7f62ed600700 1 --2- 172.21.15.107:0/2179020166 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f62d80a29a0 0x7f62d80a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:58:36.352 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:35.595+0000 7f62ed600700 1 -- 172.21.15.107:0/2179020166 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f62dc0d0650 con 0x7f62f0074930 2024-03-08T20:58:36.352 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:35.595+0000 7f62f5200700 1 --2- 172.21.15.107:0/2179020166 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f62d80a29a0 0x7f62d80a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:58:36.352 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:35.595+0000 7f62ed600700 1 --2- 172.21.15.107:0/2179020166 >> [v2:172.21.15.204:6800/1087852630,v1:172.21.15.204:6801/1087852630] conn(0x7f62d80ac560 0x7f62d80aea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:58:36.352 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:35.595+0000 7f62ed600700 1 -- 172.21.15.107:0/2179020166 --> [v2:172.21.15.204:6800/1087852630,v1:172.21.15.204:6801/1087852630] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f62dc029e50 con 0x7f62d80ac560 2024-03-08T20:58:36.352 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:35.595+0000 7f62ed600700 1 -- 172.21.15.107:0/2179020166 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) v2 ==== 24+0+0 (secure 0 0 0) 0x7f62dc0d0a00 con 0x7f62f0074930 2024-03-08T20:58:36.352 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:35.596+0000 7f62f5c00700 1 --2- 172.21.15.107:0/2179020166 >> [v2:172.21.15.204:6800/1087852630,v1:172.21.15.204:6801/1087852630] conn(0x7f62d80ac560 0x7f62d80aea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:58:36.352 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:35.596+0000 7f62f5200700 1 --2- 172.21.15.107:0/2179020166 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f62d80a29a0 0x7f62d80a4e60 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f62e4005950 tx=0x7f62e4005eb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:58:36.352 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:35.596+0000 7f62f5c00700 1 --2- 172.21.15.107:0/2179020166 >> [v2:172.21.15.204:6800/1087852630,v1:172.21.15.204:6801/1087852630] conn(0x7f62d80ac560 0x7f62d80aea00 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:58:36.352 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:35.599+0000 7f62ed600700 1 -- 172.21.15.107:0/2179020166 <== osd.2 v2:172.21.15.204:6800/1087852630 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f62dc029e50 con 0x7f62d80ac560 2024-03-08T20:58:36.353 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:35.617+0000 7f62f7c00700 1 -- 172.21.15.107:0/2179020166 --> [v2:172.21.15.204:6800/1087852630,v1:172.21.15.204:6801/1087852630] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f62f0075740 con 0x7f62d80ac560 2024-03-08T20:58:36.353 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:35.618+0000 7f62ed600700 1 -- 172.21.15.107:0/2179020166 <== osd.2 v2:172.21.15.204:6800/1087852630 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f62f0075740 con 0x7f62d80ac560 2024-03-08T20:58:36.353 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:35.619+0000 7f62f7c00700 1 -- 172.21.15.107:0/2179020166 >> [v2:172.21.15.204:6800/1087852630,v1:172.21.15.204:6801/1087852630] conn(0x7f62d80ac560 msgr2=0x7f62d80aea00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:58:36.353 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:35.619+0000 7f62f7c00700 1 --2- 172.21.15.107:0/2179020166 >> [v2:172.21.15.204:6800/1087852630,v1:172.21.15.204:6801/1087852630] conn(0x7f62d80ac560 0x7f62d80aea00 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:58:36.353 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:35.619+0000 7f62f7c00700 1 -- 172.21.15.107:0/2179020166 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f62d80a29a0 msgr2=0x7f62d80a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:58:36.353 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:35.619+0000 7f62f7c00700 1 --2- 172.21.15.107:0/2179020166 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f62d80a29a0 0x7f62d80a4e60 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f62e4005950 tx=0x7f62e4005eb0 comp rx=0 tx=0).stop 2024-03-08T20:58:36.353 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:35.619+0000 7f62f7c00700 1 -- 172.21.15.107:0/2179020166 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f62f0074930 msgr2=0x7f62f019ac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:58:36.353 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:35.619+0000 7f62f7c00700 1 --2- 172.21.15.107:0/2179020166 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f62f0074930 0x7f62f019ac70 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f62dc000c00 tx=0x7f62dc0047b0 comp rx=0 tx=0).stop 2024-03-08T20:58:36.354 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:35.619+0000 7f62f5c00700 1 -- 172.21.15.107:0/2179020166 reap_dead start 2024-03-08T20:58:36.354 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:35.619+0000 7f62f7c00700 1 -- 172.21.15.107:0/2179020166 shutdown_connections 2024-03-08T20:58:36.354 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:35.619+0000 7f62f7c00700 1 -- 172.21.15.107:0/2179020166 >> 172.21.15.107:0/2179020166 conn(0x7f62f0078f40 msgr2=0x7f62f00feff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:58:36.354 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:35.619+0000 7f62f7c00700 1 -- 172.21.15.107:0/2179020166 shutdown_connections 2024-03-08T20:58:36.354 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:35.619+0000 7f62f7c00700 1 -- 172.21.15.107:0/2179020166 wait complete. 2024-03-08T20:58:36.955 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:34.198+0000 7fed04200700 1 -- 172.21.15.107:0/255868670 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fecfc102a40 msgr2=0x7fecfc106a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:58:36.955 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:34.198+0000 7fed04200700 1 --2- 172.21.15.107:0/255868670 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fecfc102a40 0x7fecfc106a80 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fece80099c0 tx=0x7fece8009cd0 comp rx=0 tx=0).stop 2024-03-08T20:58:36.955 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:34.198+0000 7fed04200700 1 -- 172.21.15.107:0/255868670 shutdown_connections 2024-03-08T20:58:36.955 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:34.198+0000 7fed04200700 1 --2- 172.21.15.107:0/255868670 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fecfc102090 0x7fecfc102470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:58:36.956 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:34.198+0000 7fed04200700 1 --2- 172.21.15.107:0/255868670 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fecfc107160 0x7fecfc10ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:58:36.956 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:34.198+0000 7fed04200700 1 --2- 172.21.15.107:0/255868670 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fecfc102a40 0x7fecfc106a80 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:58:36.956 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:34.198+0000 7fed04200700 1 -- 172.21.15.107:0/255868670 >> 172.21.15.107:0/255868670 conn(0x7fecfc078f40 msgr2=0x7fecfc079350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:58:36.956 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:34.198+0000 7fed04200700 1 -- 172.21.15.107:0/255868670 shutdown_connections 2024-03-08T20:58:36.956 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:34.198+0000 7fed04200700 1 -- 172.21.15.107:0/255868670 wait complete. 2024-03-08T20:58:36.956 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:34.200+0000 7fed04200700 1 Processor -- start 2024-03-08T20:58:36.956 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:34.200+0000 7fed04200700 1 -- start start 2024-03-08T20:58:36.956 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:34.200+0000 7fed04200700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fecfc102090 0x7fecfc19a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:58:36.957 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:34.200+0000 7fed04200700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fecfc102a40 0x7fecfc19ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:58:36.957 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:34.200+0000 7fed04200700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fecfc107160 0x7fecfc19f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:58:36.957 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:34.200+0000 7fed04200700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecfc111fd0 con 0x7fecfc102a40 2024-03-08T20:58:36.957 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:34.200+0000 7fed04200700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecfc111e50 con 0x7fecfc102090 2024-03-08T20:58:36.957 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:34.200+0000 7fed04200700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecfc112150 con 0x7fecfc107160 2024-03-08T20:58:36.957 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:34.200+0000 7fed00e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fecfc102a40 0x7fecfc19ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:58:36.957 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:34.200+0000 7fed00e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fecfc102a40 0x7fecfc19ac80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:54092/0 (socket says 172.21.15.107:54092) 2024-03-08T20:58:36.957 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:34.200+0000 7fed00e00700 1 -- 172.21.15.107:0/2221607780 learned_addr learned my addr 172.21.15.107:0/2221607780 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T20:58:36.957 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:34.201+0000 7fed00e00700 1 -- 172.21.15.107:0/2221607780 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fecfc107160 msgr2=0x7fecfc19f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:58:36.957 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:34.201+0000 7fed01800700 1 --2- 172.21.15.107:0/2221607780 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fecfc102090 0x7fecfc19a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:58:36.958 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:34.201+0000 7fed02200700 1 --2- 172.21.15.107:0/2221607780 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fecfc107160 0x7fecfc19f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:58:36.958 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:34.201+0000 7fed00e00700 1 --2- 172.21.15.107:0/2221607780 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fecfc107160 0x7fecfc19f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:58:36.958 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:34.201+0000 7fed00e00700 1 -- 172.21.15.107:0/2221607780 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fecfc102090 msgr2=0x7fecfc19a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:58:36.958 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:34.201+0000 7fed00e00700 1 --2- 172.21.15.107:0/2221607780 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fecfc102090 0x7fecfc19a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:58:36.958 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:34.201+0000 7fed00e00700 1 -- 172.21.15.107:0/2221607780 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecfc19f750 con 0x7fecfc102a40 2024-03-08T20:58:36.958 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:34.201+0000 7fed01800700 1 --2- 172.21.15.107:0/2221607780 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fecfc102090 0x7fecfc19a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T20:58:36.958 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:34.201+0000 7fed02200700 1 --2- 172.21.15.107:0/2221607780 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fecfc107160 0x7fecfc19f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T20:58:36.958 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:34.201+0000 7fed00e00700 1 --2- 172.21.15.107:0/2221607780 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fecfc102a40 0x7fecfc19ac80 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fece80059d0 tx=0x7fece8003680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:58:36.958 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:34.201+0000 7fecfa000700 1 -- 172.21.15.107:0/2221607780 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fece8027070 con 0x7fecfc102a40 2024-03-08T20:58:36.958 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:34.201+0000 7fed04200700 1 -- 172.21.15.107:0/2221607780 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fecfc19f9e0 con 0x7fecfc102a40 2024-03-08T20:58:36.959 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:34.201+0000 7fecfa000700 1 -- 172.21.15.107:0/2221607780 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fece8004050 con 0x7fecfc102a40 2024-03-08T20:58:36.959 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:34.201+0000 7fecfa000700 1 -- 172.21.15.107:0/2221607780 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fece8022d80 con 0x7fecfc102a40 2024-03-08T20:58:36.959 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:34.202+0000 7fed04200700 1 -- 172.21.15.107:0/2221607780 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fecfc1a72c0 con 0x7fecfc102a40 2024-03-08T20:58:36.959 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:34.203+0000 7fed04200700 1 -- 172.21.15.107:0/2221607780 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fecfc10e3d0 con 0x7fecfc102a40 2024-03-08T20:58:36.959 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:34.205+0000 7fecfa000700 1 -- 172.21.15.107:0/2221607780 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) v1 ==== 137342+0+0 (secure 0 0 0) 0x7fece802a070 con 0x7fecfc102a40 2024-03-08T20:58:36.959 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:34.207+0000 7fecfa000700 1 --2- 172.21.15.107:0/2221607780 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fece40a2920 0x7fece40a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:58:36.959 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:34.207+0000 7fecfa000700 1 -- 172.21.15.107:0/2221607780 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fece80d06d0 con 0x7fecfc102a40 2024-03-08T20:58:36.959 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:34.207+0000 7fed01800700 1 --2- 172.21.15.107:0/2221607780 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fece40a2920 0x7fece40a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:58:36.959 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:34.207+0000 7fecfa000700 1 --2- 172.21.15.107:0/2221607780 >> [v2:172.21.15.204:6816/1010601477,v1:172.21.15.204:6817/1010601477] conn(0x7fece40ac4e0 0x7fece40ae980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:58:36.959 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:34.207+0000 7fecfa000700 1 -- 172.21.15.107:0/2221607780 --> [v2:172.21.15.204:6816/1010601477,v1:172.21.15.204:6817/1010601477] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fece8023e80 con 0x7fece40ac4e0 2024-03-08T20:58:36.960 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:34.207+0000 7fecfa000700 1 -- 172.21.15.107:0/2221607780 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) v2 ==== 24+0+0 (secure 0 0 0) 0x7fece80d0a80 con 0x7fecfc102a40 2024-03-08T20:58:36.960 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:34.208+0000 7fed01800700 1 --2- 172.21.15.107:0/2221607780 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fece40a2920 0x7fece40a4de0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fecf0005fd0 tx=0x7fecf0005dc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:58:36.960 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:34.208+0000 7fed02200700 1 --2- 172.21.15.107:0/2221607780 >> [v2:172.21.15.204:6816/1010601477,v1:172.21.15.204:6817/1010601477] conn(0x7fece40ac4e0 0x7fece40ae980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:58:36.960 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:34.209+0000 7fed02200700 1 --2- 172.21.15.107:0/2221607780 >> [v2:172.21.15.204:6816/1010601477,v1:172.21.15.204:6817/1010601477] conn(0x7fece40ac4e0 0x7fece40ae980 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:58:36.961 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:34.211+0000 7fecfa000700 1 -- 172.21.15.107:0/2221607780 <== osd.8 v2:172.21.15.204:6816/1010601477 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fece8023e80 con 0x7fece40ac4e0 2024-03-08T20:58:36.961 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:34.230+0000 7fed04200700 1 -- 172.21.15.107:0/2221607780 --> [v2:172.21.15.204:6816/1010601477,v1:172.21.15.204:6817/1010601477] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fecfc103560 con 0x7fece40ac4e0 2024-03-08T20:58:36.961 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:34.231+0000 7fecfa000700 1 -- 172.21.15.107:0/2221607780 <== osd.8 v2:172.21.15.204:6816/1010601477 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fecfc103560 con 0x7fece40ac4e0 2024-03-08T20:58:36.961 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:34.231+0000 7fed04200700 1 -- 172.21.15.107:0/2221607780 >> [v2:172.21.15.204:6816/1010601477,v1:172.21.15.204:6817/1010601477] conn(0x7fece40ac4e0 msgr2=0x7fece40ae980 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:58:36.961 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:34.231+0000 7fed04200700 1 --2- 172.21.15.107:0/2221607780 >> [v2:172.21.15.204:6816/1010601477,v1:172.21.15.204:6817/1010601477] conn(0x7fece40ac4e0 0x7fece40ae980 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:58:36.961 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:34.231+0000 7fed04200700 1 -- 172.21.15.107:0/2221607780 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fece40a2920 msgr2=0x7fece40a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:58:36.961 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:34.231+0000 7fed04200700 1 --2- 172.21.15.107:0/2221607780 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fece40a2920 0x7fece40a4de0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fecf0005fd0 tx=0x7fecf0005dc0 comp rx=0 tx=0).stop 2024-03-08T20:58:36.961 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:34.231+0000 7fed04200700 1 -- 172.21.15.107:0/2221607780 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fecfc102a40 msgr2=0x7fecfc19ac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:58:36.961 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:34.231+0000 7fed04200700 1 --2- 172.21.15.107:0/2221607780 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fecfc102a40 0x7fecfc19ac80 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fece80059d0 tx=0x7fece8003680 comp rx=0 tx=0).stop 2024-03-08T20:58:36.962 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:34.231+0000 7fed02200700 1 -- 172.21.15.107:0/2221607780 reap_dead start 2024-03-08T20:58:36.962 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:34.231+0000 7fed04200700 1 -- 172.21.15.107:0/2221607780 shutdown_connections 2024-03-08T20:58:36.962 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:34.231+0000 7fed04200700 1 -- 172.21.15.107:0/2221607780 >> 172.21.15.107:0/2221607780 conn(0x7fecfc078f40 msgr2=0x7fecfc1001d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:58:36.962 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:34.231+0000 7fed04200700 1 -- 172.21.15.107:0/2221607780 shutdown_connections 2024-03-08T20:58:36.962 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:34.232+0000 7fed04200700 1 -- 172.21.15.107:0/2221607780 wait complete. 2024-03-08T20:58:37.718 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:58:37 smithi107 ceph-mon[31381]: pgmap v386: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 255 B/s wr, 9 op/s 2024-03-08T20:58:37.765 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:58:37 smithi149 ceph-mon[35042]: pgmap v386: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 255 B/s wr, 9 op/s 2024-03-08T20:58:37.878 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:58:37 smithi204 ceph-mon[34935]: pgmap v386: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 255 B/s wr, 9 op/s 2024-03-08T20:58:38.145 INFO:teuthology.orchestra.run.smithi107.stdout:257698037806 2024-03-08T20:58:38.146 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761 shell --fsid dc20f0f8-dd8c-11ee-95c6-87774f69a715 -- ceph osd last-stat-seq osd.8 2024-03-08T20:58:38.237 INFO:teuthology.orchestra.run.smithi107.stdout:55834574941 2024-03-08T20:58:38.237 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761 shell --fsid dc20f0f8-dd8c-11ee-95c6-87774f69a715 -- ceph osd last-stat-seq osd.1 2024-03-08T20:58:39.208 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:39.225+0000 7f3a55e00700 1 -- 172.21.15.107:0/2380345289 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3a50107e10 msgr2=0x7f3a501081f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:58:39.208 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:39.225+0000 7f3a55e00700 1 --2- 172.21.15.107:0/2380345289 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3a50107e10 0x7f3a501081f0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f3a400099c0 tx=0x7f3a40009cd0 comp rx=0 tx=0).stop 2024-03-08T20:58:39.209 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:39.225+0000 7f3a55e00700 1 -- 172.21.15.107:0/2380345289 shutdown_connections 2024-03-08T20:58:39.209 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:39.225+0000 7f3a55e00700 1 --2- 172.21.15.107:0/2380345289 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3a501027f0 0x7f3a5010ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:58:39.209 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:39.225+0000 7f3a55e00700 1 --2- 172.21.15.107:0/2380345289 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3a50107e10 0x7f3a501081f0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:58:39.209 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:39.225+0000 7f3a55e00700 1 --2- 172.21.15.107:0/2380345289 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3a50101e30 0x7f3a501022b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:58:39.209 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:39.225+0000 7f3a55e00700 1 -- 172.21.15.107:0/2380345289 >> 172.21.15.107:0/2380345289 conn(0x7f3a50078f10 msgr2=0x7f3a50079320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:58:39.209 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:39.225+0000 7f3a55e00700 1 -- 172.21.15.107:0/2380345289 shutdown_connections 2024-03-08T20:58:39.209 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:39.225+0000 7f3a55e00700 1 -- 172.21.15.107:0/2380345289 wait complete. 2024-03-08T20:58:39.211 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:39.226+0000 7f3a55e00700 1 Processor -- start 2024-03-08T20:58:39.211 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:39.226+0000 7f3a55e00700 1 -- start start 2024-03-08T20:58:39.211 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:39.227+0000 7f3a55e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3a50101e30 0x7f3a5019a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:58:39.211 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:39.227+0000 7f3a55e00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3a501027f0 0x7f3a5019ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:58:39.211 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:39.227+0000 7f3a55e00700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3a50107e10 0x7f3a5019efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:58:39.211 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:39.227+0000 7f3a55e00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a50111fc0 con 0x7f3a50101e30 2024-03-08T20:58:39.211 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:39.227+0000 7f3a55e00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a50111e40 con 0x7f3a501027f0 2024-03-08T20:58:39.211 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:39.227+0000 7f3a55e00700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a50112140 con 0x7f3a50107e10 2024-03-08T20:58:39.211 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:39.227+0000 7f3a4ea00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3a50101e30 0x7f3a5019a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:58:39.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:39.227+0000 7f3a4ea00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3a50101e30 0x7f3a5019a710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:60158/0 (socket says 172.21.15.107:60158) 2024-03-08T20:58:39.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:39.227+0000 7f3a4ea00700 1 -- 172.21.15.107:0/3117716712 learned_addr learned my addr 172.21.15.107:0/3117716712 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T20:58:39.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:39.227+0000 7f3a46000700 1 --2- 172.21.15.107:0/3117716712 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3a501027f0 0x7f3a5019ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:58:39.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:39.227+0000 7f3a4f400700 1 --2- 172.21.15.107:0/3117716712 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3a50107e10 0x7f3a5019efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:58:39.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:39.228+0000 7f3a4ea00700 1 -- 172.21.15.107:0/3117716712 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3a50107e10 msgr2=0x7f3a5019efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:58:39.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:39.228+0000 7f3a4ea00700 1 --2- 172.21.15.107:0/3117716712 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3a50107e10 0x7f3a5019efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:58:39.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:39.228+0000 7f3a4ea00700 1 -- 172.21.15.107:0/3117716712 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3a501027f0 msgr2=0x7f3a5019ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:58:39.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:39.228+0000 7f3a4ea00700 1 --2- 172.21.15.107:0/3117716712 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3a501027f0 0x7f3a5019ac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:58:39.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:39.228+0000 7f3a4ea00700 1 -- 172.21.15.107:0/3117716712 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a5019f720 con 0x7f3a50101e30 2024-03-08T20:58:39.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:39.228+0000 7f3a4ea00700 1 --2- 172.21.15.107:0/3117716712 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3a50101e30 0x7f3a5019a710 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f3a40005b40 tx=0x7f3a40005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:58:39.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:39.228+0000 7f3a54a00700 1 -- 172.21.15.107:0/3117716712 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a40027070 con 0x7f3a50101e30 2024-03-08T20:58:39.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:39.228+0000 7f3a54a00700 1 -- 172.21.15.107:0/3117716712 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f3a40003fd0 con 0x7f3a50101e30 2024-03-08T20:58:39.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:39.228+0000 7f3a55e00700 1 -- 172.21.15.107:0/3117716712 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a5019f9b0 con 0x7f3a50101e30 2024-03-08T20:58:39.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:39.228+0000 7f3a54a00700 1 -- 172.21.15.107:0/3117716712 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a40022c60 con 0x7f3a50101e30 2024-03-08T20:58:39.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:39.229+0000 7f3a55e00700 1 -- 172.21.15.107:0/3117716712 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a501a7290 con 0x7f3a50101e30 2024-03-08T20:58:39.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:39.229+0000 7f3a55e00700 1 -- 172.21.15.107:0/3117716712 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f3a50075d40 con 0x7f3a50101e30 2024-03-08T20:58:39.214 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:39.232+0000 7f3a54a00700 1 -- 172.21.15.107:0/3117716712 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) v1 ==== 137342+0+0 (secure 0 0 0) 0x7f3a4002a070 con 0x7f3a50101e30 2024-03-08T20:58:39.216 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:39.233+0000 7f3a54a00700 1 --2- 172.21.15.107:0/3117716712 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f3a340a29a0 0x7f3a340a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:58:39.216 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:39.233+0000 7f3a54a00700 1 -- 172.21.15.107:0/3117716712 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f3a40022e00 con 0x7f3a50101e30 2024-03-08T20:58:39.216 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:39.233+0000 7f3a46000700 1 --2- 172.21.15.107:0/3117716712 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f3a340a29a0 0x7f3a340a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:58:39.217 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:39.234+0000 7f3a54a00700 1 --2- 172.21.15.107:0/3117716712 >> [v2:172.21.15.149:6824/3469284631,v1:172.21.15.149:6825/3469284631] conn(0x7f3a340ac560 0x7f3a340aea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:58:39.217 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:39.234+0000 7f3a54a00700 1 -- 172.21.15.107:0/3117716712 --> [v2:172.21.15.149:6824/3469284631,v1:172.21.15.149:6825/3469284631] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3a40009030 con 0x7f3a340ac560 2024-03-08T20:58:39.217 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:39.234+0000 7f3a54a00700 1 -- 172.21.15.107:0/3117716712 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3a400d0920 con 0x7f3a50101e30 2024-03-08T20:58:39.217 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:39.234+0000 7f3a46000700 1 --2- 172.21.15.107:0/3117716712 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f3a340a29a0 0x7f3a340a4e60 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f3a38005950 tx=0x7f3a380058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:58:39.217 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:39.234+0000 7f3a4f400700 1 --2- 172.21.15.107:0/3117716712 >> [v2:172.21.15.149:6824/3469284631,v1:172.21.15.149:6825/3469284631] conn(0x7f3a340ac560 0x7f3a340aea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:58:39.218 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:39.235+0000 7f3a4f400700 1 --2- 172.21.15.107:0/3117716712 >> [v2:172.21.15.149:6824/3469284631,v1:172.21.15.149:6825/3469284631] conn(0x7f3a340ac560 0x7f3a340aea00 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:58:39.220 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:39.237+0000 7f3a54a00700 1 -- 172.21.15.107:0/3117716712 <== osd.10 v2:172.21.15.149:6824/3469284631 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f3a40009030 con 0x7f3a340ac560 2024-03-08T20:58:39.238 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:39.255+0000 7f3a55e00700 1 -- 172.21.15.107:0/3117716712 --> [v2:172.21.15.149:6824/3469284631,v1:172.21.15.149:6825/3469284631] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f3a501034c0 con 0x7f3a340ac560 2024-03-08T20:58:39.239 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:39.256+0000 7f3a54a00700 1 -- 172.21.15.107:0/3117716712 <== osd.10 v2:172.21.15.149:6824/3469284631 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f3a501034c0 con 0x7f3a340ac560 2024-03-08T20:58:39.240 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:39.257+0000 7f3a55e00700 1 -- 172.21.15.107:0/3117716712 >> [v2:172.21.15.149:6824/3469284631,v1:172.21.15.149:6825/3469284631] conn(0x7f3a340ac560 msgr2=0x7f3a340aea00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:58:39.240 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:39.257+0000 7f3a55e00700 1 --2- 172.21.15.107:0/3117716712 >> [v2:172.21.15.149:6824/3469284631,v1:172.21.15.149:6825/3469284631] conn(0x7f3a340ac560 0x7f3a340aea00 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:58:39.240 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:39.257+0000 7f3a55e00700 1 -- 172.21.15.107:0/3117716712 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f3a340a29a0 msgr2=0x7f3a340a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:58:39.240 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:39.257+0000 7f3a55e00700 1 --2- 172.21.15.107:0/3117716712 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f3a340a29a0 0x7f3a340a4e60 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f3a38005950 tx=0x7f3a380058e0 comp rx=0 tx=0).stop 2024-03-08T20:58:39.240 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:39.257+0000 7f3a55e00700 1 -- 172.21.15.107:0/3117716712 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3a50101e30 msgr2=0x7f3a5019a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:58:39.241 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:39.257+0000 7f3a55e00700 1 --2- 172.21.15.107:0/3117716712 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3a50101e30 0x7f3a5019a710 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f3a40005b40 tx=0x7f3a40005e80 comp rx=0 tx=0).stop 2024-03-08T20:58:39.241 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:39.257+0000 7f3a4f400700 1 -- 172.21.15.107:0/3117716712 reap_dead start 2024-03-08T20:58:39.241 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:39.257+0000 7f3a55e00700 1 -- 172.21.15.107:0/3117716712 shutdown_connections 2024-03-08T20:58:39.241 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:39.257+0000 7f3a55e00700 1 -- 172.21.15.107:0/3117716712 >> 172.21.15.107:0/3117716712 conn(0x7f3a50078f10 msgr2=0x7f3a50074310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:58:39.241 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:39.257+0000 7f3a55e00700 1 -- 172.21.15.107:0/3117716712 shutdown_connections 2024-03-08T20:58:39.241 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:39.257+0000 7f3a55e00700 1 -- 172.21.15.107:0/3117716712 wait complete. 2024-03-08T20:58:39.717 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:58:39 smithi107 ceph-mon[31381]: pgmap v387: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 255 B/s wr, 11 op/s 2024-03-08T20:58:39.765 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:58:39 smithi149 ceph-mon[35042]: pgmap v387: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 255 B/s wr, 11 op/s 2024-03-08T20:58:39.867 INFO:teuthology.orchestra.run.smithi107.stdout:201863462971 2024-03-08T20:58:39.867 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761 shell --fsid dc20f0f8-dd8c-11ee-95c6-87774f69a715 -- ceph osd last-stat-seq osd.6 2024-03-08T20:58:39.878 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:58:39 smithi204 ceph-mon[34935]: pgmap v387: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 255 B/s wr, 11 op/s 2024-03-08T20:58:40.019 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:39.451+0000 7f2320800700 1 -- 172.21.15.107:0/3660190226 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2318102090 msgr2=0x7f2318102470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:58:40.019 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:39.451+0000 7f2320800700 1 --2- 172.21.15.107:0/3660190226 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2318102090 0x7f2318102470 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f230c0099c0 tx=0x7f230c009cd0 comp rx=0 tx=0).stop 2024-03-08T20:58:40.019 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:39.451+0000 7f2320800700 1 -- 172.21.15.107:0/3660190226 shutdown_connections 2024-03-08T20:58:40.019 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:39.451+0000 7f2320800700 1 --2- 172.21.15.107:0/3660190226 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2318107160 0x7f231810ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:58:40.019 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:39.451+0000 7f2320800700 1 --2- 172.21.15.107:0/3660190226 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2318102090 0x7f2318102470 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:58:40.019 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:39.451+0000 7f2320800700 1 --2- 172.21.15.107:0/3660190226 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2318102a40 0x7f2318106a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:58:40.019 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:39.451+0000 7f2320800700 1 -- 172.21.15.107:0/3660190226 >> 172.21.15.107:0/3660190226 conn(0x7f2318078f40 msgr2=0x7f2318079350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:58:40.019 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:39.451+0000 7f2320800700 1 -- 172.21.15.107:0/3660190226 shutdown_connections 2024-03-08T20:58:40.020 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:39.451+0000 7f2320800700 1 -- 172.21.15.107:0/3660190226 wait complete. 2024-03-08T20:58:40.020 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:39.453+0000 7f2320800700 1 Processor -- start 2024-03-08T20:58:40.020 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:39.453+0000 7f2320800700 1 -- start start 2024-03-08T20:58:40.020 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:39.453+0000 7f2320800700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2318102090 0x7f2318198520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:58:40.020 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:39.453+0000 7f2320800700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2318102a40 0x7f2318198a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:58:40.020 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:39.453+0000 7f2320800700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2318107160 0x7f231819cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:58:40.020 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:39.453+0000 7f2320800700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2318112010 con 0x7f2318102a40 2024-03-08T20:58:40.020 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:39.453+0000 7f2320800700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2318111e90 con 0x7f2318102090 2024-03-08T20:58:40.020 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:39.453+0000 7f2320800700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2318112190 con 0x7f2318107160 2024-03-08T20:58:40.020 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:39.454+0000 7f231d400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2318102a40 0x7f2318198a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:58:40.020 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:39.454+0000 7f231d400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2318102a40 0x7f2318198a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:60170/0 (socket says 172.21.15.107:60170) 2024-03-08T20:58:40.020 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:39.454+0000 7f231d400700 1 -- 172.21.15.107:0/1503631415 learned_addr learned my addr 172.21.15.107:0/1503631415 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T20:58:40.020 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:39.454+0000 7f231de00700 1 --2- 172.21.15.107:0/1503631415 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2318102090 0x7f2318198520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:58:40.021 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:39.454+0000 7f231e800700 1 --2- 172.21.15.107:0/1503631415 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2318107160 0x7f231819cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:58:40.021 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:39.454+0000 7f231d400700 1 -- 172.21.15.107:0/1503631415 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2318107160 msgr2=0x7f231819cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:58:40.021 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:39.454+0000 7f231d400700 1 --2- 172.21.15.107:0/1503631415 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2318107160 0x7f231819cdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:58:40.021 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:39.454+0000 7f231d400700 1 -- 172.21.15.107:0/1503631415 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2318102090 msgr2=0x7f2318198520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:58:40.021 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:39.454+0000 7f231d400700 1 --2- 172.21.15.107:0/1503631415 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2318102090 0x7f2318198520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:58:40.021 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:39.454+0000 7f231d400700 1 -- 172.21.15.107:0/1503631415 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f231819d530 con 0x7f2318102a40 2024-03-08T20:58:40.021 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:39.454+0000 7f231de00700 1 --2- 172.21.15.107:0/1503631415 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2318102090 0x7f2318198520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T20:58:40.021 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:39.454+0000 7f231e800700 1 --2- 172.21.15.107:0/1503631415 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2318107160 0x7f231819cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T20:58:40.021 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:39.454+0000 7f231d400700 1 --2- 172.21.15.107:0/1503631415 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2318102a40 0x7f2318198a60 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f230800d950 tx=0x7f230800dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:58:40.021 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:39.455+0000 7f2312a00700 1 -- 172.21.15.107:0/1503631415 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2308017070 con 0x7f2318102a40 2024-03-08T20:58:40.021 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:39.455+0000 7f2312a00700 1 -- 172.21.15.107:0/1503631415 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f230800de30 con 0x7f2318102a40 2024-03-08T20:58:40.021 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:39.455+0000 7f2312a00700 1 -- 172.21.15.107:0/1503631415 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2308005440 con 0x7f2318102a40 2024-03-08T20:58:40.021 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:39.455+0000 7f2320800700 1 -- 172.21.15.107:0/1503631415 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f231819d820 con 0x7f2318102a40 2024-03-08T20:58:40.021 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:39.455+0000 7f2320800700 1 -- 172.21.15.107:0/1503631415 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23181a50a0 con 0x7f2318102a40 2024-03-08T20:58:40.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:39.456+0000 7f2320800700 1 -- 172.21.15.107:0/1503631415 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f23180624f0 con 0x7f2318102a40 2024-03-08T20:58:40.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:39.458+0000 7f2312a00700 1 -- 172.21.15.107:0/1503631415 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) v1 ==== 137342+0+0 (secure 0 0 0) 0x7f23080055e0 con 0x7f2318102a40 2024-03-08T20:58:40.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:39.459+0000 7f2312a00700 1 --2- 172.21.15.107:0/1503631415 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f23000a2920 0x7f23000a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:58:40.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:39.460+0000 7f2312a00700 1 -- 172.21.15.107:0/1503631415 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f23080ce900 con 0x7f2318102a40 2024-03-08T20:58:40.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:39.460+0000 7f231de00700 1 --2- 172.21.15.107:0/1503631415 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f23000a2920 0x7f23000a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:58:40.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:39.460+0000 7f2312a00700 1 --2- 172.21.15.107:0/1503631415 >> [v2:172.21.15.149:6808/726286918,v1:172.21.15.149:6809/726286918] conn(0x7f23000ac4e0 0x7f23000ae980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:58:40.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:39.460+0000 7f2312a00700 1 -- 172.21.15.107:0/1503631415 --> [v2:172.21.15.149:6808/726286918,v1:172.21.15.149:6809/726286918] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f23000af050 con 0x7f23000ac4e0 2024-03-08T20:58:40.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:39.460+0000 7f2312a00700 1 -- 172.21.15.107:0/1503631415 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) v2 ==== 24+0+0 (secure 0 0 0) 0x7f23080cecb0 con 0x7f2318102a40 2024-03-08T20:58:40.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:39.460+0000 7f231de00700 1 --2- 172.21.15.107:0/1503631415 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f23000a2920 0x7f23000a4de0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f230c00b580 tx=0x7f230c009f30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:58:40.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:39.461+0000 7f231e800700 1 --2- 172.21.15.107:0/1503631415 >> [v2:172.21.15.149:6808/726286918,v1:172.21.15.149:6809/726286918] conn(0x7f23000ac4e0 0x7f23000ae980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:58:40.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:39.461+0000 7f231e800700 1 --2- 172.21.15.107:0/1503631415 >> [v2:172.21.15.149:6808/726286918,v1:172.21.15.149:6809/726286918] conn(0x7f23000ac4e0 0x7f23000ae980 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:58:40.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:39.464+0000 7f2312a00700 1 -- 172.21.15.107:0/1503631415 <== osd.4 v2:172.21.15.149:6808/726286918 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f23000af050 con 0x7f23000ac4e0 2024-03-08T20:58:40.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:39.481+0000 7f2320800700 1 -- 172.21.15.107:0/1503631415 --> [v2:172.21.15.149:6808/726286918,v1:172.21.15.149:6809/726286918] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f2318062970 con 0x7f23000ac4e0 2024-03-08T20:58:40.023 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:39.482+0000 7f2312a00700 1 -- 172.21.15.107:0/1503631415 <== osd.4 v2:172.21.15.149:6808/726286918 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f2318062970 con 0x7f23000ac4e0 2024-03-08T20:58:40.023 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:39.482+0000 7f2320800700 1 -- 172.21.15.107:0/1503631415 >> [v2:172.21.15.149:6808/726286918,v1:172.21.15.149:6809/726286918] conn(0x7f23000ac4e0 msgr2=0x7f23000ae980 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:58:40.023 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:39.483+0000 7f2320800700 1 --2- 172.21.15.107:0/1503631415 >> [v2:172.21.15.149:6808/726286918,v1:172.21.15.149:6809/726286918] conn(0x7f23000ac4e0 0x7f23000ae980 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:58:40.023 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:39.483+0000 7f2320800700 1 -- 172.21.15.107:0/1503631415 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f23000a2920 msgr2=0x7f23000a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:58:40.023 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:39.483+0000 7f2320800700 1 --2- 172.21.15.107:0/1503631415 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f23000a2920 0x7f23000a4de0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f230c00b580 tx=0x7f230c009f30 comp rx=0 tx=0).stop 2024-03-08T20:58:40.024 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:39.483+0000 7f2320800700 1 -- 172.21.15.107:0/1503631415 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2318102a40 msgr2=0x7f2318198a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:58:40.024 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:39.483+0000 7f2320800700 1 --2- 172.21.15.107:0/1503631415 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2318102a40 0x7f2318198a60 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f230800d950 tx=0x7f230800dd10 comp rx=0 tx=0).stop 2024-03-08T20:58:40.024 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:39.483+0000 7f231e800700 1 -- 172.21.15.107:0/1503631415 reap_dead start 2024-03-08T20:58:40.024 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:39.483+0000 7f2320800700 1 -- 172.21.15.107:0/1503631415 shutdown_connections 2024-03-08T20:58:40.024 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:39.483+0000 7f2320800700 1 -- 172.21.15.107:0/1503631415 >> 172.21.15.107:0/1503631415 conn(0x7f2318078f40 msgr2=0x7f2318105e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:58:40.024 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:39.483+0000 7f2320800700 1 -- 172.21.15.107:0/1503631415 shutdown_connections 2024-03-08T20:58:40.024 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:39.483+0000 7f2320800700 1 -- 172.21.15.107:0/1503631415 wait complete. 2024-03-08T20:58:41.458 INFO:teuthology.orchestra.run.smithi107.stdout:77309411416 2024-03-08T20:58:41.458 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761 shell --fsid dc20f0f8-dd8c-11ee-95c6-87774f69a715 -- ceph osd last-stat-seq osd.2 2024-03-08T20:58:41.608 INFO:teuthology.orchestra.run.smithi107.stdout:223338299444 2024-03-08T20:58:41.608 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761 shell --fsid dc20f0f8-dd8c-11ee-95c6-87774f69a715 -- ceph osd last-stat-seq osd.7 2024-03-08T20:58:41.668 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:58:41 smithi107 ceph-mon[31381]: pgmap v388: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-03-08T20:58:41.796 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:41.127+0000 7fb17be00700 1 -- 172.21.15.107:0/290700249 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb17c108f20 msgr2=0x7fb17c10cf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:58:41.797 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:41.127+0000 7fb17be00700 1 --2- 172.21.15.107:0/290700249 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb17c108f20 0x7fb17c10cf20 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fb1680099c0 tx=0x7fb168009cd0 comp rx=0 tx=0).stop 2024-03-08T20:58:41.797 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:41.127+0000 7fb17be00700 1 -- 172.21.15.107:0/290700249 shutdown_connections 2024-03-08T20:58:41.797 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:41.127+0000 7fb17be00700 1 --2- 172.21.15.107:0/290700249 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb17c108570 0x7fb17c108950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:58:41.797 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:41.127+0000 7fb17be00700 1 --2- 172.21.15.107:0/290700249 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb17c10d600 0x7fb17c1151d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:58:41.797 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:41.127+0000 7fb17be00700 1 --2- 172.21.15.107:0/290700249 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb17c108f20 0x7fb17c10cf20 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:58:41.797 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:41.127+0000 7fb17be00700 1 -- 172.21.15.107:0/290700249 >> 172.21.15.107:0/290700249 conn(0x7fb17c077a80 msgr2=0x7fb17c079ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:58:41.797 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:41.127+0000 7fb17be00700 1 -- 172.21.15.107:0/290700249 shutdown_connections 2024-03-08T20:58:41.797 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:41.127+0000 7fb17be00700 1 -- 172.21.15.107:0/290700249 wait complete. 2024-03-08T20:58:41.797 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:41.128+0000 7fb17be00700 1 Processor -- start 2024-03-08T20:58:41.797 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:41.128+0000 7fb17be00700 1 -- start start 2024-03-08T20:58:41.797 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:41.129+0000 7fb17be00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb17c108570 0x7fb17c1a45a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:58:41.797 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:41.129+0000 7fb17be00700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb17c108f20 0x7fb17c1a4ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:58:41.797 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:41.129+0000 7fb17be00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb17c10d600 0x7fb17c19e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:58:41.798 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:41.129+0000 7fb17be00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb17c118470 con 0x7fb17c108570 2024-03-08T20:58:41.798 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:41.129+0000 7fb17be00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb17c1182f0 con 0x7fb17c10d600 2024-03-08T20:58:41.798 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:41.129+0000 7fb17be00700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb17c1185f0 con 0x7fb17c108f20 2024-03-08T20:58:41.798 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:41.129+0000 7fb17aa00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb17c108570 0x7fb17c1a45a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:58:41.798 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:41.129+0000 7fb17aa00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb17c108570 0x7fb17c1a45a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:60184/0 (socket says 172.21.15.107:60184) 2024-03-08T20:58:41.798 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:41.129+0000 7fb17aa00700 1 -- 172.21.15.107:0/3237059925 learned_addr learned my addr 172.21.15.107:0/3237059925 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T20:58:41.798 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:41.129+0000 7fb17a000700 1 --2- 172.21.15.107:0/3237059925 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb17c108f20 0x7fb17c1a4ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:58:41.798 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:41.130+0000 7fb17b400700 1 --2- 172.21.15.107:0/3237059925 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb17c10d600 0x7fb17c19e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:58:41.798 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:41.130+0000 7fb17aa00700 1 -- 172.21.15.107:0/3237059925 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb17c108f20 msgr2=0x7fb17c1a4ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:58:41.798 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:41.130+0000 7fb17aa00700 1 --2- 172.21.15.107:0/3237059925 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb17c108f20 0x7fb17c1a4ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:58:41.798 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:41.130+0000 7fb17aa00700 1 -- 172.21.15.107:0/3237059925 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb17c10d600 msgr2=0x7fb17c19e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:58:41.798 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:41.130+0000 7fb17aa00700 1 --2- 172.21.15.107:0/3237059925 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb17c10d600 0x7fb17c19e670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:58:41.799 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:41.130+0000 7fb17aa00700 1 -- 172.21.15.107:0/3237059925 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb17c19ef50 con 0x7fb17c108570 2024-03-08T20:58:41.799 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:41.130+0000 7fb17aa00700 1 --2- 172.21.15.107:0/3237059925 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb17c108570 0x7fb17c1a45a0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fb16c00c9b0 tx=0x7fb16c00cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:58:41.799 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:41.130+0000 7fb173400700 1 -- 172.21.15.107:0/3237059925 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb16c007b00 con 0x7fb17c108570 2024-03-08T20:58:41.799 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:41.130+0000 7fb17be00700 1 -- 172.21.15.107:0/3237059925 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb17c19f240 con 0x7fb17c108570 2024-03-08T20:58:41.799 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:41.130+0000 7fb173400700 1 -- 172.21.15.107:0/3237059925 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fb16c007ca0 con 0x7fb17c108570 2024-03-08T20:58:41.799 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:41.130+0000 7fb173400700 1 -- 172.21.15.107:0/3237059925 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb16c018730 con 0x7fb17c108570 2024-03-08T20:58:41.799 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:41.130+0000 7fb17be00700 1 -- 172.21.15.107:0/3237059925 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb17c1ab520 con 0x7fb17c108570 2024-03-08T20:58:41.799 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:41.131+0000 7fb17be00700 1 -- 172.21.15.107:0/3237059925 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb17c062570 con 0x7fb17c108570 2024-03-08T20:58:41.799 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:41.133+0000 7fb173400700 1 -- 172.21.15.107:0/3237059925 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) v1 ==== 137342+0+0 (secure 0 0 0) 0x7fb16c00f450 con 0x7fb17c108570 2024-03-08T20:58:41.799 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:41.135+0000 7fb173400700 1 --2- 172.21.15.107:0/3237059925 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fb1600a29a0 0x7fb1600a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:58:41.799 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:41.135+0000 7fb173400700 1 -- 172.21.15.107:0/3237059925 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fb16c0cef80 con 0x7fb17c108570 2024-03-08T20:58:41.799 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:41.135+0000 7fb17a000700 1 --2- 172.21.15.107:0/3237059925 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fb1600a29a0 0x7fb1600a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:58:41.799 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:41.135+0000 7fb173400700 1 --2- 172.21.15.107:0/3237059925 >> [v2:172.21.15.107:6826/3334759819,v1:172.21.15.107:6827/3334759819] conn(0x7fb1600ac560 0x7fb1600aea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:58:41.799 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:41.135+0000 7fb173400700 1 -- 172.21.15.107:0/3237059925 --> [v2:172.21.15.107:6826/3334759819,v1:172.21.15.107:6827/3334759819] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb1600af0d0 con 0x7fb1600ac560 2024-03-08T20:58:41.800 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:41.135+0000 7fb173400700 1 -- 172.21.15.107:0/3237059925 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb16c0cf460 con 0x7fb17c108570 2024-03-08T20:58:41.800 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:41.135+0000 7fb17b400700 1 --2- 172.21.15.107:0/3237059925 >> [v2:172.21.15.107:6826/3334759819,v1:172.21.15.107:6827/3334759819] conn(0x7fb1600ac560 0x7fb1600aea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:58:41.800 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:41.136+0000 7fb17a000700 1 --2- 172.21.15.107:0/3237059925 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fb1600a29a0 0x7fb1600a4e60 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fb168000c00 tx=0x7fb168005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:58:41.800 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:41.136+0000 7fb17b400700 1 --2- 172.21.15.107:0/3237059925 >> [v2:172.21.15.107:6826/3334759819,v1:172.21.15.107:6827/3334759819] conn(0x7fb1600ac560 0x7fb1600aea00 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:58:41.800 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:41.137+0000 7fb173400700 1 -- 172.21.15.107:0/3237059925 <== osd.9 v2:172.21.15.107:6826/3334759819 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fb1600af0d0 con 0x7fb1600ac560 2024-03-08T20:58:41.801 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:41.155+0000 7fb17be00700 1 -- 172.21.15.107:0/3237059925 --> [v2:172.21.15.107:6826/3334759819,v1:172.21.15.107:6827/3334759819] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fb17c0629f0 con 0x7fb1600ac560 2024-03-08T20:58:41.801 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:41.156+0000 7fb173400700 1 -- 172.21.15.107:0/3237059925 <== osd.9 v2:172.21.15.107:6826/3334759819 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fb17c0629f0 con 0x7fb1600ac560 2024-03-08T20:58:41.801 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:41.156+0000 7fb17be00700 1 -- 172.21.15.107:0/3237059925 >> [v2:172.21.15.107:6826/3334759819,v1:172.21.15.107:6827/3334759819] conn(0x7fb1600ac560 msgr2=0x7fb1600aea00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:58:41.801 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:41.156+0000 7fb17be00700 1 --2- 172.21.15.107:0/3237059925 >> [v2:172.21.15.107:6826/3334759819,v1:172.21.15.107:6827/3334759819] conn(0x7fb1600ac560 0x7fb1600aea00 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:58:41.801 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:41.157+0000 7fb17be00700 1 -- 172.21.15.107:0/3237059925 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fb1600a29a0 msgr2=0x7fb1600a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:58:41.801 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:41.157+0000 7fb17be00700 1 --2- 172.21.15.107:0/3237059925 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fb1600a29a0 0x7fb1600a4e60 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fb168000c00 tx=0x7fb168005fd0 comp rx=0 tx=0).stop 2024-03-08T20:58:41.801 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:41.157+0000 7fb17be00700 1 -- 172.21.15.107:0/3237059925 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb17c108570 msgr2=0x7fb17c1a45a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:58:41.801 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:41.157+0000 7fb17be00700 1 --2- 172.21.15.107:0/3237059925 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb17c108570 0x7fb17c1a45a0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fb16c00c9b0 tx=0x7fb16c00cd70 comp rx=0 tx=0).stop 2024-03-08T20:58:41.801 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:41.157+0000 7fb17b400700 1 -- 172.21.15.107:0/3237059925 reap_dead start 2024-03-08T20:58:41.801 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:41.157+0000 7fb17be00700 1 -- 172.21.15.107:0/3237059925 shutdown_connections 2024-03-08T20:58:41.801 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:41.157+0000 7fb17be00700 1 -- 172.21.15.107:0/3237059925 >> 172.21.15.107:0/3237059925 conn(0x7fb17c077a80 msgr2=0x7fb17c078600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:58:41.801 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:41.157+0000 7fb17be00700 1 -- 172.21.15.107:0/3237059925 shutdown_connections 2024-03-08T20:58:41.801 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:41.157+0000 7fb17be00700 1 -- 172.21.15.107:0/3237059925 wait complete. 2024-03-08T20:58:41.878 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:58:41 smithi204 ceph-mon[34935]: pgmap v388: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-03-08T20:58:42.015 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:58:41 smithi149 ceph-mon[35042]: pgmap v388: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-03-08T20:58:42.846 INFO:teuthology.orchestra.run.smithi107.stdout:292057776168 2024-03-08T20:58:42.847 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761 shell --fsid dc20f0f8-dd8c-11ee-95c6-87774f69a715 -- ceph osd last-stat-seq osd.9 2024-03-08T20:58:43.340 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:42.279+0000 7f04c8400700 1 -- 172.21.15.107:0/605621846 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f04c0107160 msgr2=0x7f04c010ed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:58:43.340 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:42.279+0000 7f04c8400700 1 --2- 172.21.15.107:0/605621846 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f04c0107160 0x7f04c010ed30 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f04b800b0d0 tx=0x7f04b800b3e0 comp rx=0 tx=0).stop 2024-03-08T20:58:43.340 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:42.280+0000 7f04c8400700 1 -- 172.21.15.107:0/605621846 shutdown_connections 2024-03-08T20:58:43.340 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:42.280+0000 7f04c8400700 1 --2- 172.21.15.107:0/605621846 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f04c0107160 0x7f04c010ed30 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:58:43.340 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:42.280+0000 7f04c8400700 1 --2- 172.21.15.107:0/605621846 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f04c0102090 0x7f04c0102470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:58:43.340 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:42.280+0000 7f04c8400700 1 --2- 172.21.15.107:0/605621846 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f04c0102a40 0x7f04c0106a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:58:43.340 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:42.280+0000 7f04c8400700 1 -- 172.21.15.107:0/605621846 >> 172.21.15.107:0/605621846 conn(0x7f04c0078f40 msgr2=0x7f04c0079350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:58:43.340 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:42.280+0000 7f04c8400700 1 -- 172.21.15.107:0/605621846 shutdown_connections 2024-03-08T20:58:43.340 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:42.280+0000 7f04c8400700 1 -- 172.21.15.107:0/605621846 wait complete. 2024-03-08T20:58:43.340 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:42.281+0000 7f04c8400700 1 Processor -- start 2024-03-08T20:58:43.340 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:42.281+0000 7f04c8400700 1 -- start start 2024-03-08T20:58:43.341 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:42.281+0000 7f04c8400700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f04c0102090 0x7f04c019a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:58:43.341 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:42.281+0000 7f04c8400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f04c0102a40 0x7f04c019abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:58:43.341 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:42.281+0000 7f04c8400700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f04c0107160 0x7f04c019ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:58:43.341 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:42.281+0000 7f04c8400700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04c0112030 con 0x7f04c0102a40 2024-03-08T20:58:43.341 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:42.281+0000 7f04c8400700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04c0111eb0 con 0x7f04c0107160 2024-03-08T20:58:43.341 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:42.281+0000 7f04c8400700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04c01121b0 con 0x7f04c0102090 2024-03-08T20:58:43.341 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:42.281+0000 7f04c5000700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f04c0102a40 0x7f04c019abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:58:43.341 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:42.281+0000 7f04c5000700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f04c0102a40 0x7f04c019abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:60212/0 (socket says 172.21.15.107:60212) 2024-03-08T20:58:43.341 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:42.281+0000 7f04c5000700 1 -- 172.21.15.107:0/2689318703 learned_addr learned my addr 172.21.15.107:0/2689318703 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T20:58:43.341 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:42.282+0000 7f04c6400700 1 --2- 172.21.15.107:0/2689318703 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f04c0107160 0x7f04c019ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:58:43.341 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:42.282+0000 7f04c5000700 1 -- 172.21.15.107:0/2689318703 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f04c0102090 msgr2=0x7f04c019a690 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-03-08T20:58:43.341 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:42.282+0000 7f04c5000700 1 --2- 172.21.15.107:0/2689318703 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f04c0102090 0x7f04c019a690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:58:43.342 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:42.282+0000 7f04c5000700 1 -- 172.21.15.107:0/2689318703 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f04c0107160 msgr2=0x7f04c019ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:58:43.342 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:42.282+0000 7f04c5000700 1 --2- 172.21.15.107:0/2689318703 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f04c0107160 0x7f04c019ef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:58:43.342 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:42.282+0000 7f04c5000700 1 -- 172.21.15.107:0/2689318703 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04c019f6a0 con 0x7f04c0102a40 2024-03-08T20:58:43.342 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:42.282+0000 7f04c5000700 1 --2- 172.21.15.107:0/2689318703 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f04c0102a40 0x7f04c019abd0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f04ac00c970 tx=0x7f04ac00cd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:58:43.342 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:42.282+0000 7f04be000700 1 -- 172.21.15.107:0/2689318703 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04ac007b00 con 0x7f04c0102a40 2024-03-08T20:58:43.342 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:42.282+0000 7f04c8400700 1 -- 172.21.15.107:0/2689318703 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04c019f990 con 0x7f04c0102a40 2024-03-08T20:58:43.342 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:42.282+0000 7f04be000700 1 -- 172.21.15.107:0/2689318703 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f04ac004ce0 con 0x7f04c0102a40 2024-03-08T20:58:43.342 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:42.282+0000 7f04be000700 1 -- 172.21.15.107:0/2689318703 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04ac0055f0 con 0x7f04c0102a40 2024-03-08T20:58:43.342 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:42.283+0000 7f04c8400700 1 -- 172.21.15.107:0/2689318703 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04c01a7210 con 0x7f04c0102a40 2024-03-08T20:58:43.342 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:42.283+0000 7f04c8400700 1 -- 172.21.15.107:0/2689318703 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f04c0100f40 con 0x7f04c0102a40 2024-03-08T20:58:43.342 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:42.285+0000 7f04be000700 1 -- 172.21.15.107:0/2689318703 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) v1 ==== 137342+0+0 (secure 0 0 0) 0x7f04ac007ca0 con 0x7f04c0102a40 2024-03-08T20:58:43.342 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:42.287+0000 7f04be000700 1 --2- 172.21.15.107:0/2689318703 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f04a80a2920 0x7f04a80a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:58:43.342 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:42.287+0000 7f04be000700 1 -- 172.21.15.107:0/2689318703 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f04ac0cedc0 con 0x7f04c0102a40 2024-03-08T20:58:43.343 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:42.287+0000 7f04c5a00700 1 --2- 172.21.15.107:0/2689318703 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f04a80a2920 0x7f04a80a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:58:43.343 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:42.287+0000 7f04be000700 1 --2- 172.21.15.107:0/2689318703 >> [v2:172.21.15.107:6802/2453946095,v1:172.21.15.107:6803/2453946095] conn(0x7f04a80ac4e0 0x7f04a80ae980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:58:43.343 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:42.287+0000 7f04be000700 1 -- 172.21.15.107:0/2689318703 --> [v2:172.21.15.107:6802/2453946095,v1:172.21.15.107:6803/2453946095] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f04ac012ec0 con 0x7f04a80ac4e0 2024-03-08T20:58:43.343 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:42.287+0000 7f04be000700 1 -- 172.21.15.107:0/2689318703 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) v2 ==== 24+0+0 (secure 0 0 0) 0x7f04ac0cf170 con 0x7f04c0102a40 2024-03-08T20:58:43.343 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:42.287+0000 7f04c6400700 1 --2- 172.21.15.107:0/2689318703 >> [v2:172.21.15.107:6802/2453946095,v1:172.21.15.107:6803/2453946095] conn(0x7f04a80ac4e0 0x7f04a80ae980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:58:43.343 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:42.288+0000 7f04c5a00700 1 --2- 172.21.15.107:0/2689318703 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f04a80a2920 0x7f04a80a4de0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f04b400a9b0 tx=0x7f04b4005c90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:58:43.343 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:42.288+0000 7f04c6400700 1 --2- 172.21.15.107:0/2689318703 >> [v2:172.21.15.107:6802/2453946095,v1:172.21.15.107:6803/2453946095] conn(0x7f04a80ac4e0 0x7f04a80ae980 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:58:43.343 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:42.289+0000 7f04be000700 1 -- 172.21.15.107:0/2689318703 <== osd.0 v2:172.21.15.107:6802/2453946095 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f04ac012ec0 con 0x7f04a80ac4e0 2024-03-08T20:58:43.343 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:42.311+0000 7f04c8400700 1 -- 172.21.15.107:0/2689318703 --> [v2:172.21.15.107:6802/2453946095,v1:172.21.15.107:6803/2453946095] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f04c0103560 con 0x7f04a80ac4e0 2024-03-08T20:58:43.343 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:42.311+0000 7f04be000700 1 -- 172.21.15.107:0/2689318703 <== osd.0 v2:172.21.15.107:6802/2453946095 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f04c0103560 con 0x7f04a80ac4e0 2024-03-08T20:58:43.344 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:42.312+0000 7f04c8400700 1 -- 172.21.15.107:0/2689318703 >> [v2:172.21.15.107:6802/2453946095,v1:172.21.15.107:6803/2453946095] conn(0x7f04a80ac4e0 msgr2=0x7f04a80ae980 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:58:43.344 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:42.312+0000 7f04c8400700 1 --2- 172.21.15.107:0/2689318703 >> [v2:172.21.15.107:6802/2453946095,v1:172.21.15.107:6803/2453946095] conn(0x7f04a80ac4e0 0x7f04a80ae980 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:58:43.344 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:42.312+0000 7f04c8400700 1 -- 172.21.15.107:0/2689318703 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f04a80a2920 msgr2=0x7f04a80a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:58:43.344 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:42.312+0000 7f04c8400700 1 --2- 172.21.15.107:0/2689318703 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f04a80a2920 0x7f04a80a4de0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f04b400a9b0 tx=0x7f04b4005c90 comp rx=0 tx=0).stop 2024-03-08T20:58:43.344 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:42.312+0000 7f04c8400700 1 -- 172.21.15.107:0/2689318703 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f04c0102a40 msgr2=0x7f04c019abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:58:43.344 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:42.312+0000 7f04c8400700 1 --2- 172.21.15.107:0/2689318703 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f04c0102a40 0x7f04c019abd0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f04ac00c970 tx=0x7f04ac00cd30 comp rx=0 tx=0).stop 2024-03-08T20:58:43.344 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:42.312+0000 7f04c6400700 1 -- 172.21.15.107:0/2689318703 reap_dead start 2024-03-08T20:58:43.344 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:42.312+0000 7f04c8400700 1 -- 172.21.15.107:0/2689318703 shutdown_connections 2024-03-08T20:58:43.345 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:42.312+0000 7f04c8400700 1 -- 172.21.15.107:0/2689318703 >> 172.21.15.107:0/2689318703 conn(0x7f04c0078f40 msgr2=0x7f04c010d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:58:43.345 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:42.312+0000 7f04c8400700 1 -- 172.21.15.107:0/2689318703 shutdown_connections 2024-03-08T20:58:43.345 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:42.312+0000 7f04c8400700 1 -- 172.21.15.107:0/2689318703 wait complete. 2024-03-08T20:58:43.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:58:43 smithi107 ceph-mon[31381]: pgmap v389: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 10 op/s 2024-03-08T20:58:43.878 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:58:43 smithi204 ceph-mon[34935]: pgmap v389: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 10 op/s 2024-03-08T20:58:44.015 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:58:43 smithi149 ceph-mon[35042]: pgmap v389: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 10 op/s 2024-03-08T20:58:44.390 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:44.406+0000 7f6fa5c00700 1 -- 172.21.15.107:0/3705851378 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6fa0100f70 msgr2=0x7f6fa0101350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:58:44.390 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:44.406+0000 7f6fa5c00700 1 --2- 172.21.15.107:0/3705851378 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6fa0100f70 0x7f6fa0101350 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f6f9000b0d0 tx=0x7f6f9000b3e0 comp rx=0 tx=0).stop 2024-03-08T20:58:44.391 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:44.406+0000 7f6fa5c00700 1 -- 172.21.15.107:0/3705851378 shutdown_connections 2024-03-08T20:58:44.391 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:44.406+0000 7f6fa5c00700 1 --2- 172.21.15.107:0/3705851378 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6fa0105fc0 0x7f6fa010db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:58:44.391 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:44.406+0000 7f6fa5c00700 1 --2- 172.21.15.107:0/3705851378 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6fa0100f70 0x7f6fa0101350 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:58:44.391 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:44.406+0000 7f6fa5c00700 1 --2- 172.21.15.107:0/3705851378 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6fa0101920 0x7f6fa01058e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:58:44.391 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:44.406+0000 7f6fa5c00700 1 -- 172.21.15.107:0/3705851378 >> 172.21.15.107:0/3705851378 conn(0x7f6fa0076d30 msgr2=0x7f6fa0077140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:58:44.391 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:44.406+0000 7f6fa5c00700 1 -- 172.21.15.107:0/3705851378 shutdown_connections 2024-03-08T20:58:44.391 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:44.406+0000 7f6fa5c00700 1 -- 172.21.15.107:0/3705851378 wait complete. 2024-03-08T20:58:44.391 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:44.407+0000 7f6fa5c00700 1 Processor -- start 2024-03-08T20:58:44.391 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:44.408+0000 7f6fa5c00700 1 -- start start 2024-03-08T20:58:44.393 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:44.408+0000 7f6fa5c00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6fa0100f70 0x7f6fa0102980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:58:44.393 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:44.408+0000 7f6fa5c00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6fa0101920 0x7f6fa0102ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:58:44.393 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:44.408+0000 7f6fa5c00700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6fa0105fc0 0x7f6fa0103400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:58:44.393 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:44.408+0000 7f6fa5c00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fa0110ec0 con 0x7f6fa0101920 2024-03-08T20:58:44.393 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:44.408+0000 7f6fa5c00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fa0110d40 con 0x7f6fa0100f70 2024-03-08T20:58:44.394 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:44.408+0000 7f6fa5c00700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fa0111040 con 0x7f6fa0105fc0 2024-03-08T20:58:44.394 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:44.408+0000 7f6f96000700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6fa0101920 0x7f6fa0102ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:58:44.394 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:44.408+0000 7f6f96000700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6fa0101920 0x7f6fa0102ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:60282/0 (socket says 172.21.15.107:60282) 2024-03-08T20:58:44.394 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:44.408+0000 7f6f96000700 1 -- 172.21.15.107:0/314645090 learned_addr learned my addr 172.21.15.107:0/314645090 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T20:58:44.394 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:44.409+0000 7f6f9ea00700 1 --2- 172.21.15.107:0/314645090 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6fa0100f70 0x7f6fa0102980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:58:44.394 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:44.409+0000 7f6f9f400700 1 --2- 172.21.15.107:0/314645090 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6fa0105fc0 0x7f6fa0103400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:58:44.394 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:44.409+0000 7f6f96000700 1 -- 172.21.15.107:0/314645090 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6fa0105fc0 msgr2=0x7f6fa0103400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:58:44.394 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:44.409+0000 7f6f96000700 1 --2- 172.21.15.107:0/314645090 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6fa0105fc0 0x7f6fa0103400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:58:44.394 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:44.409+0000 7f6f96000700 1 -- 172.21.15.107:0/314645090 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6fa0100f70 msgr2=0x7f6fa0102980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:58:44.394 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:44.409+0000 7f6f96000700 1 --2- 172.21.15.107:0/314645090 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6fa0100f70 0x7f6fa0102980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:58:44.394 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:44.409+0000 7f6f96000700 1 -- 172.21.15.107:0/314645090 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fa01a6a20 con 0x7f6fa0101920 2024-03-08T20:58:44.395 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:44.409+0000 7f6f96000700 1 --2- 172.21.15.107:0/314645090 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6fa0101920 0x7f6fa0102ec0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f6f8800c8f0 tx=0x7f6f8800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:58:44.395 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:44.409+0000 7f6f97e00700 1 -- 172.21.15.107:0/314645090 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f88007b00 con 0x7f6fa0101920 2024-03-08T20:58:44.395 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:44.409+0000 7f6f97e00700 1 -- 172.21.15.107:0/314645090 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f6f88007ca0 con 0x7f6fa0101920 2024-03-08T20:58:44.395 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:44.409+0000 7f6fa5c00700 1 -- 172.21.15.107:0/314645090 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fa01a6d10 con 0x7f6fa0101920 2024-03-08T20:58:44.395 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:44.409+0000 7f6f97e00700 1 -- 172.21.15.107:0/314645090 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f88018660 con 0x7f6fa0101920 2024-03-08T20:58:44.395 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:44.410+0000 7f6fa5c00700 1 -- 172.21.15.107:0/314645090 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fa01a71f0 con 0x7f6fa0101920 2024-03-08T20:58:44.395 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:44.411+0000 7f6fa5c00700 1 -- 172.21.15.107:0/314645090 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f6fa0062470 con 0x7f6fa0101920 2024-03-08T20:58:44.396 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:44.413+0000 7f6f97e00700 1 -- 172.21.15.107:0/314645090 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) v1 ==== 137342+0+0 (secure 0 0 0) 0x7f6f8800f450 con 0x7f6fa0101920 2024-03-08T20:58:44.397 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:44.414+0000 7f6f97e00700 1 --2- 172.21.15.107:0/314645090 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f6f840a2920 0x7f6f840a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:58:44.397 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:44.414+0000 7f6f97e00700 1 -- 172.21.15.107:0/314645090 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f6f880cec50 con 0x7f6fa0101920 2024-03-08T20:58:44.397 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:44.414+0000 7f6f9ea00700 1 --2- 172.21.15.107:0/314645090 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f6f840a2920 0x7f6f840a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:58:44.398 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:44.415+0000 7f6f97e00700 1 --2- 172.21.15.107:0/314645090 >> [v2:172.21.15.107:6810/3123579789,v1:172.21.15.107:6811/3123579789] conn(0x7f6f840ac4e0 0x7f6f840ae980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:58:44.398 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:44.415+0000 7f6f97e00700 1 -- 172.21.15.107:0/314645090 --> [v2:172.21.15.107:6810/3123579789,v1:172.21.15.107:6811/3123579789] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6f840af050 con 0x7f6f840ac4e0 2024-03-08T20:58:44.398 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:44.415+0000 7f6f97e00700 1 -- 172.21.15.107:0/314645090 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6f880d1270 con 0x7f6fa0101920 2024-03-08T20:58:44.398 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:44.415+0000 7f6f9f400700 1 --2- 172.21.15.107:0/314645090 >> [v2:172.21.15.107:6810/3123579789,v1:172.21.15.107:6811/3123579789] conn(0x7f6f840ac4e0 0x7f6f840ae980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:58:44.398 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:44.415+0000 7f6f9ea00700 1 --2- 172.21.15.107:0/314645090 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f6f840a2920 0x7f6f840a4de0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f6f90009fd0 tx=0x7f6f9000b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:58:44.399 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:44.415+0000 7f6f9f400700 1 --2- 172.21.15.107:0/314645090 >> [v2:172.21.15.107:6810/3123579789,v1:172.21.15.107:6811/3123579789] conn(0x7f6f840ac4e0 0x7f6f840ae980 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:58:44.399 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:44.416+0000 7f6f97e00700 1 -- 172.21.15.107:0/314645090 <== osd.3 v2:172.21.15.107:6810/3123579789 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f6f840af050 con 0x7f6f840ac4e0 2024-03-08T20:58:44.417 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:44.434+0000 7f6fa5c00700 1 -- 172.21.15.107:0/314645090 --> [v2:172.21.15.107:6810/3123579789,v1:172.21.15.107:6811/3123579789] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f6fa00628a0 con 0x7f6f840ac4e0 2024-03-08T20:58:44.418 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:44.435+0000 7f6f97e00700 1 -- 172.21.15.107:0/314645090 <== osd.3 v2:172.21.15.107:6810/3123579789 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f6fa00628a0 con 0x7f6f840ac4e0 2024-03-08T20:58:44.419 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:44.435+0000 7f6fa5c00700 1 -- 172.21.15.107:0/314645090 >> [v2:172.21.15.107:6810/3123579789,v1:172.21.15.107:6811/3123579789] conn(0x7f6f840ac4e0 msgr2=0x7f6f840ae980 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:58:44.419 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:44.435+0000 7f6fa5c00700 1 --2- 172.21.15.107:0/314645090 >> [v2:172.21.15.107:6810/3123579789,v1:172.21.15.107:6811/3123579789] conn(0x7f6f840ac4e0 0x7f6f840ae980 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:58:44.419 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:44.436+0000 7f6fa5c00700 1 -- 172.21.15.107:0/314645090 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f6f840a2920 msgr2=0x7f6f840a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:58:44.419 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:44.436+0000 7f6fa5c00700 1 --2- 172.21.15.107:0/314645090 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f6f840a2920 0x7f6f840a4de0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f6f90009fd0 tx=0x7f6f9000b040 comp rx=0 tx=0).stop 2024-03-08T20:58:44.419 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:44.436+0000 7f6fa5c00700 1 -- 172.21.15.107:0/314645090 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6fa0101920 msgr2=0x7f6fa0102ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:58:44.419 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:44.436+0000 7f6fa5c00700 1 --2- 172.21.15.107:0/314645090 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6fa0101920 0x7f6fa0102ec0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f6f8800c8f0 tx=0x7f6f8800ccb0 comp rx=0 tx=0).stop 2024-03-08T20:58:44.419 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:44.436+0000 7f6f9f400700 1 -- 172.21.15.107:0/314645090 reap_dead start 2024-03-08T20:58:44.419 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:44.436+0000 7f6fa5c00700 1 -- 172.21.15.107:0/314645090 shutdown_connections 2024-03-08T20:58:44.420 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:44.436+0000 7f6fa5c00700 1 -- 172.21.15.107:0/314645090 >> 172.21.15.107:0/314645090 conn(0x7f6fa0076d30 msgr2=0x7f6fa006a700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:58:44.420 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:44.436+0000 7f6fa5c00700 1 -- 172.21.15.107:0/314645090 shutdown_connections 2024-03-08T20:58:44.420 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:44.436+0000 7f6fa5c00700 1 -- 172.21.15.107:0/314645090 wait complete. 2024-03-08T20:58:44.431 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:43.988+0000 7f3d5dc00700 1 -- 172.21.15.107:0/2785139747 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3d58101440 msgr2=0x7f3d5810e100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:58:44.431 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:43.988+0000 7f3d5dc00700 1 --2- 172.21.15.107:0/2785139747 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3d58101440 0x7f3d5810e100 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f3d44009990 tx=0x7f3d44009ca0 comp rx=0 tx=0).stop 2024-03-08T20:58:44.431 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:43.988+0000 7f3d5dc00700 1 -- 172.21.15.107:0/2785139747 shutdown_connections 2024-03-08T20:58:44.431 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:43.988+0000 7f3d5dc00700 1 --2- 172.21.15.107:0/2785139747 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3d58100a90 0x7f3d58100e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:58:44.431 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:43.988+0000 7f3d5dc00700 1 --2- 172.21.15.107:0/2785139747 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3d5810e640 0x7f3d58110a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:58:44.431 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:43.988+0000 7f3d5dc00700 1 --2- 172.21.15.107:0/2785139747 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3d58101440 0x7f3d5810e100 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:58:44.431 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:43.988+0000 7f3d5dc00700 1 -- 172.21.15.107:0/2785139747 >> 172.21.15.107:0/2785139747 conn(0x7f3d580fc6b0 msgr2=0x7f3d580fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:58:44.431 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:43.988+0000 7f3d5dc00700 1 -- 172.21.15.107:0/2785139747 shutdown_connections 2024-03-08T20:58:44.431 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:43.988+0000 7f3d5dc00700 1 -- 172.21.15.107:0/2785139747 wait complete. 2024-03-08T20:58:44.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:43.989+0000 7f3d5dc00700 1 Processor -- start 2024-03-08T20:58:44.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:43.990+0000 7f3d5dc00700 1 -- start start 2024-03-08T20:58:44.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:43.990+0000 7f3d5dc00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3d58100a90 0x7f3d5819a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:58:44.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:43.990+0000 7f3d5dc00700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3d58101440 0x7f3d5819abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:58:44.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:43.990+0000 7f3d5dc00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3d5810e640 0x7f3d5819ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:58:44.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:43.990+0000 7f3d5dc00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d581141a0 con 0x7f3d5810e640 2024-03-08T20:58:44.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:43.990+0000 7f3d5dc00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d58114020 con 0x7f3d58100a90 2024-03-08T20:58:44.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:43.990+0000 7f3d5dc00700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d58114320 con 0x7f3d58101440 2024-03-08T20:58:44.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:43.990+0000 7f3d5d200700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3d5810e640 0x7f3d5819ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:58:44.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:43.990+0000 7f3d5d200700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3d5810e640 0x7f3d5819ef80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:60258/0 (socket says 172.21.15.107:60258) 2024-03-08T20:58:44.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:43.990+0000 7f3d5d200700 1 -- 172.21.15.107:0/1242254404 learned_addr learned my addr 172.21.15.107:0/1242254404 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T20:58:44.433 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:43.991+0000 7f3d57400700 1 --2- 172.21.15.107:0/1242254404 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3d58101440 0x7f3d5819abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:58:44.433 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:43.991+0000 7f3d57e00700 1 --2- 172.21.15.107:0/1242254404 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3d58100a90 0x7f3d5819a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:58:44.433 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:43.991+0000 7f3d5d200700 1 -- 172.21.15.107:0/1242254404 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3d58101440 msgr2=0x7f3d5819abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:58:44.433 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:43.991+0000 7f3d5d200700 1 --2- 172.21.15.107:0/1242254404 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3d58101440 0x7f3d5819abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:58:44.433 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:43.991+0000 7f3d5d200700 1 -- 172.21.15.107:0/1242254404 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3d58100a90 msgr2=0x7f3d5819a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:58:44.433 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:43.991+0000 7f3d5d200700 1 --2- 172.21.15.107:0/1242254404 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3d58100a90 0x7f3d5819a6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:58:44.433 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:43.991+0000 7f3d5d200700 1 -- 172.21.15.107:0/1242254404 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d5819f6c0 con 0x7f3d5810e640 2024-03-08T20:58:44.433 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:43.991+0000 7f3d57400700 1 --2- 172.21.15.107:0/1242254404 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3d58101440 0x7f3d5819abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T20:58:44.433 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:43.991+0000 7f3d5d200700 1 --2- 172.21.15.107:0/1242254404 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3d5810e640 0x7f3d5819ef80 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f3d5000b9e0 tx=0x7f3d5000bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:58:44.433 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:43.991+0000 7f3d54c00700 1 -- 172.21.15.107:0/1242254404 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d50019070 con 0x7f3d5810e640 2024-03-08T20:58:44.434 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:43.992+0000 7f3d5dc00700 1 -- 172.21.15.107:0/1242254404 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d5819f9b0 con 0x7f3d5810e640 2024-03-08T20:58:44.434 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:43.992+0000 7f3d54c00700 1 -- 172.21.15.107:0/1242254404 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f3d50014440 con 0x7f3d5810e640 2024-03-08T20:58:44.434 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:43.992+0000 7f3d54c00700 1 -- 172.21.15.107:0/1242254404 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d5000ce60 con 0x7f3d5810e640 2024-03-08T20:58:44.434 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:43.992+0000 7f3d5dc00700 1 -- 172.21.15.107:0/1242254404 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d581a7230 con 0x7f3d5810e640 2024-03-08T20:58:44.434 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:43.993+0000 7f3d5dc00700 1 -- 172.21.15.107:0/1242254404 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f3d58062570 con 0x7f3d5810e640 2024-03-08T20:58:44.434 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:43.995+0000 7f3d54c00700 1 -- 172.21.15.107:0/1242254404 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) v1 ==== 137342+0+0 (secure 0 0 0) 0x7f3d5001c070 con 0x7f3d5810e640 2024-03-08T20:58:44.434 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:43.997+0000 7f3d54c00700 1 --2- 172.21.15.107:0/1242254404 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f3d3c0a2920 0x7f3d3c0a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:58:44.434 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:43.997+0000 7f3d54c00700 1 -- 172.21.15.107:0/1242254404 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f3d500ce6d0 con 0x7f3d5810e640 2024-03-08T20:58:44.435 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:43.997+0000 7f3d57e00700 1 --2- 172.21.15.107:0/1242254404 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f3d3c0a2920 0x7f3d3c0a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:58:44.435 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:43.997+0000 7f3d54c00700 1 --2- 172.21.15.107:0/1242254404 >> [v2:172.21.15.204:6824/3097772473,v1:172.21.15.204:6825/3097772473] conn(0x7f3d3c0ac4e0 0x7f3d3c0ae980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:58:44.435 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:43.997+0000 7f3d54c00700 1 -- 172.21.15.107:0/1242254404 --> [v2:172.21.15.204:6824/3097772473,v1:172.21.15.204:6825/3097772473] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3d50017e70 con 0x7f3d3c0ac4e0 2024-03-08T20:58:44.435 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:43.997+0000 7f3d54c00700 1 -- 172.21.15.107:0/1242254404 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3d500cea80 con 0x7f3d5810e640 2024-03-08T20:58:44.435 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:43.998+0000 7f3d57e00700 1 --2- 172.21.15.107:0/1242254404 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f3d3c0a2920 0x7f3d3c0a4de0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f3d48005950 tx=0x7f3d48005eb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:58:44.435 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:43.998+0000 7f3d57400700 1 --2- 172.21.15.107:0/1242254404 >> [v2:172.21.15.204:6824/3097772473,v1:172.21.15.204:6825/3097772473] conn(0x7f3d3c0ac4e0 0x7f3d3c0ae980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:58:44.435 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:43.999+0000 7f3d57400700 1 --2- 172.21.15.107:0/1242254404 >> [v2:172.21.15.204:6824/3097772473,v1:172.21.15.204:6825/3097772473] conn(0x7f3d3c0ac4e0 0x7f3d3c0ae980 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:58:44.435 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:44.001+0000 7f3d54c00700 1 -- 172.21.15.107:0/1242254404 <== osd.11 v2:172.21.15.204:6824/3097772473 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f3d50017e70 con 0x7f3d3c0ac4e0 2024-03-08T20:58:44.435 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:44.026+0000 7f3d5dc00700 1 -- 172.21.15.107:0/1242254404 --> [v2:172.21.15.204:6824/3097772473,v1:172.21.15.204:6825/3097772473] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f3d580629f0 con 0x7f3d3c0ac4e0 2024-03-08T20:58:44.435 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:44.026+0000 7f3d54c00700 1 -- 172.21.15.107:0/1242254404 <== osd.11 v2:172.21.15.204:6824/3097772473 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f3d580629f0 con 0x7f3d3c0ac4e0 2024-03-08T20:58:44.435 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:44.027+0000 7f3d5dc00700 1 -- 172.21.15.107:0/1242254404 >> [v2:172.21.15.204:6824/3097772473,v1:172.21.15.204:6825/3097772473] conn(0x7f3d3c0ac4e0 msgr2=0x7f3d3c0ae980 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:58:44.435 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:44.027+0000 7f3d5dc00700 1 --2- 172.21.15.107:0/1242254404 >> [v2:172.21.15.204:6824/3097772473,v1:172.21.15.204:6825/3097772473] conn(0x7f3d3c0ac4e0 0x7f3d3c0ae980 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:58:44.436 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:44.027+0000 7f3d5dc00700 1 -- 172.21.15.107:0/1242254404 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f3d3c0a2920 msgr2=0x7f3d3c0a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:58:44.436 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:44.027+0000 7f3d5dc00700 1 --2- 172.21.15.107:0/1242254404 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f3d3c0a2920 0x7f3d3c0a4de0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f3d48005950 tx=0x7f3d48005eb0 comp rx=0 tx=0).stop 2024-03-08T20:58:44.436 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:44.027+0000 7f3d5dc00700 1 -- 172.21.15.107:0/1242254404 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3d5810e640 msgr2=0x7f3d5819ef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:58:44.436 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:44.027+0000 7f3d5dc00700 1 --2- 172.21.15.107:0/1242254404 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3d5810e640 0x7f3d5819ef80 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f3d5000b9e0 tx=0x7f3d5000bda0 comp rx=0 tx=0).stop 2024-03-08T20:58:44.436 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:44.027+0000 7f3d5d200700 1 -- 172.21.15.107:0/1242254404 reap_dead start 2024-03-08T20:58:44.436 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:44.027+0000 7f3d5dc00700 1 -- 172.21.15.107:0/1242254404 shutdown_connections 2024-03-08T20:58:44.436 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:44.027+0000 7f3d5dc00700 1 -- 172.21.15.107:0/1242254404 >> 172.21.15.107:0/1242254404 conn(0x7f3d580fc6b0 msgr2=0x7f3d580feaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:58:44.436 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:44.027+0000 7f3d5dc00700 1 -- 172.21.15.107:0/1242254404 shutdown_connections 2024-03-08T20:58:44.436 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:44.027+0000 7f3d5dc00700 1 -- 172.21.15.107:0/1242254404 wait complete. 2024-03-08T20:58:45.575 INFO:teuthology.orchestra.run.smithi107.stdout:107374182482 2024-03-08T20:58:45.576 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761 shell --fsid dc20f0f8-dd8c-11ee-95c6-87774f69a715 -- ceph osd last-stat-seq osd.3 2024-03-08T20:58:45.878 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:58:45 smithi204 ceph-mon[34935]: pgmap v390: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 10 op/s 2024-03-08T20:58:45.961 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:58:45 smithi107 ceph-mon[31381]: pgmap v390: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 10 op/s 2024-03-08T20:58:46.013 INFO:teuthology.orchestra.run.smithi107.stdout:313532612641 2024-03-08T20:58:46.014 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761 shell --fsid dc20f0f8-dd8c-11ee-95c6-87774f69a715 -- ceph osd last-stat-seq osd.10 2024-03-08T20:58:46.015 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:58:45 smithi149 ceph-mon[35042]: pgmap v390: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 10 op/s 2024-03-08T20:58:46.606 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:42.622+0000 7f775be00700 1 -- 172.21.15.107:0/2615418478 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f775c107000 msgr2=0x7f775c10ebd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:58:46.606 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:42.622+0000 7f775be00700 1 --2- 172.21.15.107:0/2615418478 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f775c107000 0x7f775c10ebd0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f77540099c0 tx=0x7f7754009cd0 comp rx=0 tx=0).stop 2024-03-08T20:58:46.606 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:42.622+0000 7f775be00700 1 -- 172.21.15.107:0/2615418478 shutdown_connections 2024-03-08T20:58:46.606 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:42.622+0000 7f775be00700 1 --2- 172.21.15.107:0/2615418478 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f775c107000 0x7f775c10ebd0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:58:46.606 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:42.622+0000 7f775be00700 1 --2- 172.21.15.107:0/2615418478 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f775c101f20 0x7f775c102300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:58:46.606 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:42.622+0000 7f775be00700 1 --2- 172.21.15.107:0/2615418478 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f775c1028d0 0x7f775c106920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:58:46.606 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:42.622+0000 7f775be00700 1 -- 172.21.15.107:0/2615418478 >> 172.21.15.107:0/2615418478 conn(0x7f775c0fd7b0 msgr2=0x7f775c0ffbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:58:46.606 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:42.622+0000 7f775be00700 1 -- 172.21.15.107:0/2615418478 shutdown_connections 2024-03-08T20:58:46.606 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:42.622+0000 7f775be00700 1 -- 172.21.15.107:0/2615418478 wait complete. 2024-03-08T20:58:46.606 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:42.623+0000 7f775be00700 1 Processor -- start 2024-03-08T20:58:46.607 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:42.623+0000 7f775be00700 1 -- start start 2024-03-08T20:58:46.607 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:42.624+0000 7f775be00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f775c101f20 0x7f775c0740e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:58:46.607 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:42.624+0000 7f775be00700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f775c1028d0 0x7f775c074620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:58:46.607 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:42.624+0000 7f775be00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f775c107000 0x7f775c074b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:58:46.607 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:42.624+0000 7f775be00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f775c111eb0 con 0x7f775c107000 2024-03-08T20:58:46.607 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:42.624+0000 7f775be00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f775c111d30 con 0x7f775c101f20 2024-03-08T20:58:46.607 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:42.624+0000 7f775be00700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f775c112030 con 0x7f775c1028d0 2024-03-08T20:58:46.607 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:42.624+0000 7f775b400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f775c107000 0x7f775c074b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:58:46.607 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:42.624+0000 7f775b400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f775c107000 0x7f775c074b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:60242/0 (socket says 172.21.15.107:60242) 2024-03-08T20:58:46.607 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:42.624+0000 7f775b400700 1 -- 172.21.15.107:0/2596934204 learned_addr learned my addr 172.21.15.107:0/2596934204 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T20:58:46.607 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:42.624+0000 7f7753e00700 1 --2- 172.21.15.107:0/2596934204 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f775c1028d0 0x7f775c074620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:58:46.607 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:42.624+0000 7f775aa00700 1 --2- 172.21.15.107:0/2596934204 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f775c101f20 0x7f775c0740e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:58:46.607 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:42.625+0000 7f775b400700 1 -- 172.21.15.107:0/2596934204 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f775c1028d0 msgr2=0x7f775c074620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:58:46.608 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:42.625+0000 7f775b400700 1 --2- 172.21.15.107:0/2596934204 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f775c1028d0 0x7f775c074620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:58:46.608 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:42.625+0000 7f775b400700 1 -- 172.21.15.107:0/2596934204 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f775c101f20 msgr2=0x7f775c0740e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:58:46.608 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:42.625+0000 7f775b400700 1 --2- 172.21.15.107:0/2596934204 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f775c101f20 0x7f775c0740e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:58:46.608 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:42.625+0000 7f775b400700 1 -- 172.21.15.107:0/2596934204 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f775c1a67d0 con 0x7f775c107000 2024-03-08T20:58:46.608 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:42.625+0000 7f775b400700 1 --2- 172.21.15.107:0/2596934204 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f775c107000 0x7f775c074b60 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f77540059d0 tx=0x7f7754009f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:58:46.608 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:42.625+0000 7f7760a00700 1 -- 172.21.15.107:0/2596934204 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7754027070 con 0x7f775c107000 2024-03-08T20:58:46.609 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:42.625+0000 7f7760a00700 1 -- 172.21.15.107:0/2596934204 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f7754004010 con 0x7f775c107000 2024-03-08T20:58:46.609 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:42.625+0000 7f7760a00700 1 -- 172.21.15.107:0/2596934204 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7754022ce0 con 0x7f775c107000 2024-03-08T20:58:46.609 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:42.625+0000 7f775be00700 1 -- 172.21.15.107:0/2596934204 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f775c1a6a60 con 0x7f775c107000 2024-03-08T20:58:46.609 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:42.625+0000 7f775be00700 1 -- 172.21.15.107:0/2596934204 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f775c1a6e60 con 0x7f775c107000 2024-03-08T20:58:46.609 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:42.626+0000 7f775be00700 1 -- 172.21.15.107:0/2596934204 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f775c0feb60 con 0x7f775c107000 2024-03-08T20:58:46.609 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:42.629+0000 7f7760a00700 1 -- 172.21.15.107:0/2596934204 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) v1 ==== 137342+0+0 (secure 0 0 0) 0x7f775402a070 con 0x7f775c107000 2024-03-08T20:58:46.609 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:42.630+0000 7f7760a00700 1 --2- 172.21.15.107:0/2596934204 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f77400a2920 0x7f77400a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:58:46.609 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:42.630+0000 7f7760a00700 1 -- 172.21.15.107:0/2596934204 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f77540d0340 con 0x7f775c107000 2024-03-08T20:58:46.610 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:42.630+0000 7f775aa00700 1 --2- 172.21.15.107:0/2596934204 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f77400a2920 0x7f77400a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:58:46.610 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:42.630+0000 7f7760a00700 1 --2- 172.21.15.107:0/2596934204 >> [v2:172.21.15.204:6808/3143887820,v1:172.21.15.204:6809/3143887820] conn(0x7f77400ac4e0 0x7f77400ae980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:58:46.610 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:42.630+0000 7f7760a00700 1 -- 172.21.15.107:0/2596934204 --> [v2:172.21.15.204:6808/3143887820,v1:172.21.15.204:6809/3143887820] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f77400af050 con 0x7f77400ac4e0 2024-03-08T20:58:46.610 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:42.630+0000 7f7760a00700 1 -- 172.21.15.107:0/2596934204 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) v2 ==== 24+0+0 (secure 0 0 0) 0x7f77540d06f0 con 0x7f775c107000 2024-03-08T20:58:46.610 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:42.631+0000 7f775aa00700 1 --2- 172.21.15.107:0/2596934204 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f77400a2920 0x7f77400a4de0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f774c00e270 tx=0x7f774c009040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:58:46.610 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:42.631+0000 7f7753e00700 1 --2- 172.21.15.107:0/2596934204 >> [v2:172.21.15.204:6808/3143887820,v1:172.21.15.204:6809/3143887820] conn(0x7f77400ac4e0 0x7f77400ae980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:58:46.610 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:42.632+0000 7f7753e00700 1 --2- 172.21.15.107:0/2596934204 >> [v2:172.21.15.204:6808/3143887820,v1:172.21.15.204:6809/3143887820] conn(0x7f77400ac4e0 0x7f77400ae980 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:58:46.610 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:42.634+0000 7f7760a00700 1 -- 172.21.15.107:0/2596934204 <== osd.5 v2:172.21.15.204:6808/3143887820 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f77400af050 con 0x7f77400ac4e0 2024-03-08T20:58:46.610 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:42.651+0000 7f775be00700 1 -- 172.21.15.107:0/2596934204 --> [v2:172.21.15.204:6808/3143887820,v1:172.21.15.204:6809/3143887820] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f775c103400 con 0x7f77400ac4e0 2024-03-08T20:58:46.610 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:42.652+0000 7f7760a00700 1 -- 172.21.15.107:0/2596934204 <== osd.5 v2:172.21.15.204:6808/3143887820 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f775c103400 con 0x7f77400ac4e0 2024-03-08T20:58:46.610 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:42.653+0000 7f775be00700 1 -- 172.21.15.107:0/2596934204 >> [v2:172.21.15.204:6808/3143887820,v1:172.21.15.204:6809/3143887820] conn(0x7f77400ac4e0 msgr2=0x7f77400ae980 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:58:46.610 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:42.653+0000 7f775be00700 1 --2- 172.21.15.107:0/2596934204 >> [v2:172.21.15.204:6808/3143887820,v1:172.21.15.204:6809/3143887820] conn(0x7f77400ac4e0 0x7f77400ae980 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:58:46.611 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:42.653+0000 7f775be00700 1 -- 172.21.15.107:0/2596934204 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f77400a2920 msgr2=0x7f77400a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:58:46.611 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:42.653+0000 7f775be00700 1 --2- 172.21.15.107:0/2596934204 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f77400a2920 0x7f77400a4de0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f774c00e270 tx=0x7f774c009040 comp rx=0 tx=0).stop 2024-03-08T20:58:46.611 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:42.653+0000 7f775be00700 1 -- 172.21.15.107:0/2596934204 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f775c107000 msgr2=0x7f775c074b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:58:46.611 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:42.653+0000 7f775be00700 1 --2- 172.21.15.107:0/2596934204 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f775c107000 0x7f775c074b60 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f77540059d0 tx=0x7f7754009f80 comp rx=0 tx=0).stop 2024-03-08T20:58:46.611 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:42.653+0000 7f775b400700 1 -- 172.21.15.107:0/2596934204 reap_dead start 2024-03-08T20:58:46.611 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:42.653+0000 7f775be00700 1 -- 172.21.15.107:0/2596934204 shutdown_connections 2024-03-08T20:58:46.611 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:42.653+0000 7f775be00700 1 -- 172.21.15.107:0/2596934204 >> 172.21.15.107:0/2596934204 conn(0x7f775c0fd7b0 msgr2=0x7f775c0fe1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:58:46.611 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:42.654+0000 7f775be00700 1 -- 172.21.15.107:0/2596934204 shutdown_connections 2024-03-08T20:58:46.611 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:58:42.654+0000 7f775be00700 1 -- 172.21.15.107:0/2596934204 wait complete. 2024-03-08T20:58:46.989 INFO:teuthology.orchestra.run.smithi107.stdout:141733920843 2024-03-08T20:58:46.989 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761 shell --fsid dc20f0f8-dd8c-11ee-95c6-87774f69a715 -- ceph osd last-stat-seq osd.4 2024-03-08T20:58:47.599 INFO:teuthology.orchestra.run.smithi107.stdout:167503724613 2024-03-08T20:58:47.599 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761 shell --fsid dc20f0f8-dd8c-11ee-95c6-87774f69a715 -- ceph osd last-stat-seq osd.5 2024-03-08T20:58:47.878 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:58:47 smithi204 ceph-mon[34935]: pgmap v391: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 9 op/s 2024-03-08T20:58:47.967 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:58:47 smithi107 ceph-mon[31381]: pgmap v391: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 9 op/s 2024-03-08T20:58:48.015 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:58:47 smithi149 ceph-mon[35042]: pgmap v391: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 9 op/s 2024-03-08T20:58:48.110 INFO:teuthology.orchestra.run.smithi107.stdout:347892351003 2024-03-08T20:58:48.111 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761 shell --fsid dc20f0f8-dd8c-11ee-95c6-87774f69a715 -- ceph osd last-stat-seq osd.11 2024-03-08T20:58:48.503 INFO:teuthology.orchestra.run.smithi107.stdout:34359738470 2024-03-08T20:58:48.504 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761 shell --fsid dc20f0f8-dd8c-11ee-95c6-87774f69a715 -- ceph osd last-stat-seq osd.0 2024-03-08T20:58:48.619 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/mon.a/config 2024-03-08T20:58:48.646 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/mon.a/config 2024-03-08T20:58:48.670 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/mon.a/config 2024-03-08T20:58:48.679 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/mon.a/config 2024-03-08T20:58:48.692 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/mon.a/config 2024-03-08T20:58:48.694 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/mon.a/config 2024-03-08T20:58:48.702 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/mon.a/config 2024-03-08T20:58:48.711 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/mon.a/config 2024-03-08T20:58:48.716 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/mon.a/config 2024-03-08T20:58:48.716 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/mon.a/config 2024-03-08T20:58:48.718 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:58:48 smithi107 ceph-mon[31381]: pgmap v392: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-03-08T20:58:48.878 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:58:48 smithi204 ceph-mon[34935]: pgmap v392: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-03-08T20:58:49.014 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/mon.a/config 2024-03-08T20:58:49.015 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:58:48 smithi149 ceph-mon[35042]: pgmap v392: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-03-08T20:58:51.495 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:58:51 smithi107 ceph-mon[31381]: pgmap v393: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-03-08T20:58:51.765 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:58:51 smithi149 ceph-mon[35042]: pgmap v393: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-03-08T20:58:51.878 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:58:51 smithi204 ceph-mon[34935]: pgmap v393: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-03-08T20:58:53.697 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:58:53 smithi107 ceph-mon[31381]: pgmap v394: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-08T20:58:53.698 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:58:53 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T20:58:53.765 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:58:53 smithi149 ceph-mon[35042]: pgmap v394: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-08T20:58:53.765 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:58:53 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T20:58:53.878 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:58:53 smithi204 ceph-mon[34935]: pgmap v394: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-03-08T20:58:53.878 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:58:53 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T20:58:55.015 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:58:54 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:58:55.015 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:58:54 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:58:55.015 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:58:54 smithi149 ceph-mon[35042]: pgmap v395: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-03-08T20:58:55.128 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:58:54 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:58:55.128 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:58:54 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:58:55.128 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:58:54 smithi204 ceph-mon[34935]: pgmap v395: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-03-08T20:58:55.217 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:58:54 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:58:55.218 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:58:54 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:58:55.218 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:58:54 smithi107 ceph-mon[31381]: pgmap v395: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-03-08T20:58:56.217 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:58:55 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:58:56.218 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:58:55 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:58:56.218 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:58:55 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:58:56.218 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:58:55 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:58:56.265 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:58:55 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:58:56.266 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:58:55 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:58:56.266 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:58:55 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:58:56.266 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:58:55 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:58:56.378 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:58:55 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:58:56.378 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:58:55 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:58:56.378 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:58:55 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:58:56.378 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:58:55 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:58:57.217 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:58:56 smithi107 ceph-mon[31381]: pgmap v396: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 9 op/s 2024-03-08T20:58:57.217 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:58:56 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:58:57.218 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:58:56 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:58:57.265 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:58:56 smithi149 ceph-mon[35042]: pgmap v396: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 9 op/s 2024-03-08T20:58:57.265 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:58:56 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:58:57.265 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:58:56 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:58:57.378 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:58:56 smithi204 ceph-mon[34935]: pgmap v396: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 9 op/s 2024-03-08T20:58:57.379 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:58:56 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:58:57.379 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:58:56 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:58:59.672 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:58:59 smithi107 ceph-mon[31381]: pgmap v397: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-03-08T20:58:59.765 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:58:59 smithi149 ceph-mon[35042]: pgmap v397: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-03-08T20:58:59.878 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:58:59 smithi204 ceph-mon[34935]: pgmap v397: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-03-08T20:59:01.718 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:59:01 smithi107 ceph-mon[31381]: pgmap v398: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-08T20:59:01.765 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:59:01 smithi149 ceph-mon[35042]: pgmap v398: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-08T20:59:01.878 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:59:01 smithi204 ceph-mon[34935]: pgmap v398: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-08T20:59:01.979 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:01.929+0000 7f6260600700 1 -- 172.21.15.107:0/464365002 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f625806a430 msgr2=0x7f625810ec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:59:01.979 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:01.929+0000 7f6260600700 1 --2- 172.21.15.107:0/464365002 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f625806a430 0x7f625810ec70 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f625000b0d0 tx=0x7f625000b3e0 comp rx=0 tx=0).stop 2024-03-08T20:59:01.979 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:01.929+0000 7f6260600700 1 -- 172.21.15.107:0/464365002 shutdown_connections 2024-03-08T20:59:01.979 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:01.929+0000 7f6260600700 1 --2- 172.21.15.107:0/464365002 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6258107e30 0x7f6258108210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:01.979 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:01.929+0000 7f6260600700 1 --2- 172.21.15.107:0/464365002 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f625806a430 0x7f625810ec70 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:01.979 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:01.929+0000 7f6260600700 1 --2- 172.21.15.107:0/464365002 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6258069a90 0x7f6258069ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:01.979 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:01.929+0000 7f6260600700 1 -- 172.21.15.107:0/464365002 >> 172.21.15.107:0/464365002 conn(0x7f6258077c10 msgr2=0x7f6258078020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:59:01.979 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:01.929+0000 7f6260600700 1 -- 172.21.15.107:0/464365002 shutdown_connections 2024-03-08T20:59:01.979 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:01.929+0000 7f6260600700 1 -- 172.21.15.107:0/464365002 wait complete. 2024-03-08T20:59:01.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:01.930+0000 7f6260600700 1 Processor -- start 2024-03-08T20:59:01.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:01.931+0000 7f6260600700 1 -- start start 2024-03-08T20:59:01.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:01.931+0000 7f6260600700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6258069a90 0x7f6258100db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:01.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:01.931+0000 7f6260600700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f625806a430 0x7f62581012f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:01.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:01.931+0000 7f6260600700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6258107e30 0x7f6258101830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:01.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:01.931+0000 7f6260600700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6258111fb0 con 0x7f6258069a90 2024-03-08T20:59:01.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:01.931+0000 7f6260600700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6258111e30 con 0x7f625806a430 2024-03-08T20:59:01.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:01.931+0000 7f6260600700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6258112130 con 0x7f6258107e30 2024-03-08T20:59:01.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:01.931+0000 7f625dc00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6258069a90 0x7f6258100db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:59:01.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:01.931+0000 7f625dc00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6258069a90 0x7f6258100db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:45888/0 (socket says 172.21.15.107:45888) 2024-03-08T20:59:01.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:01.931+0000 7f625dc00700 1 -- 172.21.15.107:0/2950283342 learned_addr learned my addr 172.21.15.107:0/2950283342 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T20:59:01.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:01.932+0000 7f625e600700 1 --2- 172.21.15.107:0/2950283342 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6258107e30 0x7f6258101830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:59:01.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:01.932+0000 7f625d200700 1 --2- 172.21.15.107:0/2950283342 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f625806a430 0x7f62581012f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:59:01.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:01.932+0000 7f625dc00700 1 -- 172.21.15.107:0/2950283342 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6258107e30 msgr2=0x7f6258101830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:59:01.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:01.932+0000 7f625dc00700 1 --2- 172.21.15.107:0/2950283342 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6258107e30 0x7f6258101830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:01.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:01.932+0000 7f625dc00700 1 -- 172.21.15.107:0/2950283342 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f625806a430 msgr2=0x7f62581012f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:59:01.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:01.932+0000 7f625dc00700 1 --2- 172.21.15.107:0/2950283342 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f625806a430 0x7f62581012f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:01.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:01.932+0000 7f625dc00700 1 -- 172.21.15.107:0/2950283342 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62581a69b0 con 0x7f6258069a90 2024-03-08T20:59:01.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:01.933+0000 7f625dc00700 1 --2- 172.21.15.107:0/2950283342 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6258069a90 0x7f6258100db0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f624c00d980 tx=0x7f624c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:59:01.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:01.933+0000 7f6256000700 1 -- 172.21.15.107:0/2950283342 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f624c017070 con 0x7f6258069a90 2024-03-08T20:59:01.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:01.933+0000 7f6256000700 1 -- 172.21.15.107:0/2950283342 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f624c005020 con 0x7f6258069a90 2024-03-08T20:59:01.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:01.933+0000 7f6260600700 1 -- 172.21.15.107:0/2950283342 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62581a6ca0 con 0x7f6258069a90 2024-03-08T20:59:01.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:01.933+0000 7f6256000700 1 -- 172.21.15.107:0/2950283342 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f624c00fa90 con 0x7f6258069a90 2024-03-08T20:59:01.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:01.933+0000 7f6260600700 1 -- 172.21.15.107:0/2950283342 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62581a7190 con 0x7f6258069a90 2024-03-08T20:59:01.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:01.934+0000 7f6260600700 1 -- 172.21.15.107:0/2950283342 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6258068aa0 con 0x7f6258069a90 2024-03-08T20:59:01.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:01.936+0000 7f6256000700 1 -- 172.21.15.107:0/2950283342 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) v1 ==== 137342+0+0 (secure 0 0 0) 0x7f624c0051d0 con 0x7f6258069a90 2024-03-08T20:59:01.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:01.937+0000 7f6256000700 1 --2- 172.21.15.107:0/2950283342 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f62400a2920 0x7f62400a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:01.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:01.937+0000 7f6256000700 1 -- 172.21.15.107:0/2950283342 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f624c0ce9e0 con 0x7f6258069a90 2024-03-08T20:59:01.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:01.938+0000 7f625d200700 1 --2- 172.21.15.107:0/2950283342 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f62400a2920 0x7f62400a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:59:01.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:01.938+0000 7f625d200700 1 --2- 172.21.15.107:0/2950283342 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f62400a2920 0x7f62400a4de0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f6258078280 tx=0x7f624400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:59:01.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:01.942+0000 7f6256000700 1 -- 172.21.15.107:0/2950283342 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f624c020020 con 0x7f6258069a90 2024-03-08T20:59:02.153 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:02.169+0000 7f6260600700 1 -- 172.21.15.107:0/2950283342 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f625802d9f0 con 0x7f6258069a90 2024-03-08T20:59:02.154 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:02.170+0000 7f6256000700 1 -- 172.21.15.107:0/2950283342 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f624c09b8e0 con 0x7f6258069a90 2024-03-08T20:59:02.154 INFO:teuthology.orchestra.run.smithi107.stdout:201863462976 2024-03-08T20:59:02.157 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:02.172+0000 7f6260600700 1 -- 172.21.15.107:0/2950283342 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f62400a2920 msgr2=0x7f62400a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:59:02.157 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:02.172+0000 7f6260600700 1 --2- 172.21.15.107:0/2950283342 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f62400a2920 0x7f62400a4de0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f6258078280 tx=0x7f624400b410 comp rx=0 tx=0).stop 2024-03-08T20:59:02.157 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:02.173+0000 7f6260600700 1 -- 172.21.15.107:0/2950283342 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6258069a90 msgr2=0x7f6258100db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:59:02.157 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:02.173+0000 7f6260600700 1 --2- 172.21.15.107:0/2950283342 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6258069a90 0x7f6258100db0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f624c00d980 tx=0x7f624c00dc90 comp rx=0 tx=0).stop 2024-03-08T20:59:02.157 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:02.173+0000 7f6260600700 1 -- 172.21.15.107:0/2950283342 shutdown_connections 2024-03-08T20:59:02.157 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:02.173+0000 7f6260600700 1 --2- 172.21.15.107:0/2950283342 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6258107e30 0x7f6258101830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:02.157 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:02.173+0000 7f6260600700 1 --2- 172.21.15.107:0/2950283342 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f62400a2920 0x7f62400a4de0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:02.157 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:02.173+0000 7f6260600700 1 --2- 172.21.15.107:0/2950283342 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6258069a90 0x7f6258100db0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:02.158 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:02.173+0000 7f6260600700 1 --2- 172.21.15.107:0/2950283342 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f625806a430 0x7f62581012f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:02.158 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:02.173+0000 7f6260600700 1 -- 172.21.15.107:0/2950283342 >> 172.21.15.107:0/2950283342 conn(0x7f6258077c10 msgr2=0x7f62580ff9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:59:02.158 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:02.173+0000 7f6260600700 1 -- 172.21.15.107:0/2950283342 shutdown_connections 2024-03-08T20:59:02.158 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:02.173+0000 7f6260600700 1 -- 172.21.15.107:0/2950283342 wait complete. 2024-03-08T20:59:02.295 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:02.311+0000 7fc35f800700 1 -- 172.21.15.107:0/731839304 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc358108db0 msgr2=0x7fc3581152c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:59:02.295 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:02.311+0000 7fc35f800700 1 --2- 172.21.15.107:0/731839304 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc358108db0 0x7fc3581152c0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fc35000b0a0 tx=0x7fc35000b3b0 comp rx=0 tx=0).stop 2024-03-08T20:59:02.295 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:02.311+0000 7fc35f800700 1 -- 172.21.15.107:0/731839304 shutdown_connections 2024-03-08T20:59:02.295 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:02.311+0000 7fc35f800700 1 --2- 172.21.15.107:0/731839304 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc358108db0 0x7fc3581152c0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:02.296 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:02.311+0000 7fc35f800700 1 --2- 172.21.15.107:0/731839304 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc35810e480 0x7fc35810e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:02.296 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:02.311+0000 7fc35f800700 1 --2- 172.21.15.107:0/731839304 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc3581083f0 0x7fc358108870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:02.296 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:02.311+0000 7fc35f800700 1 -- 172.21.15.107:0/731839304 >> 172.21.15.107:0/731839304 conn(0x7fc3580755b0 msgr2=0x7fc3580779d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:59:02.296 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:02.311+0000 7fc35f800700 1 -- 172.21.15.107:0/731839304 shutdown_connections 2024-03-08T20:59:02.296 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:02.311+0000 7fc35f800700 1 -- 172.21.15.107:0/731839304 wait complete. 2024-03-08T20:59:02.297 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:02.312+0000 7fc35f800700 1 Processor -- start 2024-03-08T20:59:02.297 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:02.312+0000 7fc35f800700 1 -- start start 2024-03-08T20:59:02.297 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:02.313+0000 7fc35f800700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc3581083f0 0x7fc3581a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:02.297 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:02.313+0000 7fc35f800700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc358108db0 0x7fc3581a4be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:02.297 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:02.313+0000 7fc35f800700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc35810e480 0x7fc35819e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:02.297 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:02.313+0000 7fc35f800700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc358118610 con 0x7fc3581083f0 2024-03-08T20:59:02.298 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:02.313+0000 7fc35f800700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc358118490 con 0x7fc358108db0 2024-03-08T20:59:02.298 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:02.313+0000 7fc35f800700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc358118790 con 0x7fc35810e480 2024-03-08T20:59:02.298 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:02.313+0000 7fc35ce00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc3581083f0 0x7fc3581a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:59:02.298 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:02.313+0000 7fc35ce00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc3581083f0 0x7fc3581a46a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:45902/0 (socket says 172.21.15.107:45902) 2024-03-08T20:59:02.298 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:02.313+0000 7fc35ce00700 1 -- 172.21.15.107:0/3254370519 learned_addr learned my addr 172.21.15.107:0/3254370519 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T20:59:02.298 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:02.313+0000 7fc35d800700 1 --2- 172.21.15.107:0/3254370519 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc35810e480 0x7fc35819e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:59:02.298 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:02.313+0000 7fc357e00700 1 --2- 172.21.15.107:0/3254370519 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc358108db0 0x7fc3581a4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:59:02.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:02.314+0000 7fc35ce00700 1 -- 172.21.15.107:0/3254370519 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc35810e480 msgr2=0x7fc35819e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:59:02.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:02.314+0000 7fc35ce00700 1 --2- 172.21.15.107:0/3254370519 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc35810e480 0x7fc35819e870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:02.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:02.314+0000 7fc35ce00700 1 -- 172.21.15.107:0/3254370519 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc358108db0 msgr2=0x7fc3581a4be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:59:02.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:02.314+0000 7fc35ce00700 1 --2- 172.21.15.107:0/3254370519 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc358108db0 0x7fc3581a4be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:02.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:02.314+0000 7fc35ce00700 1 -- 172.21.15.107:0/3254370519 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc35819f050 con 0x7fc3581083f0 2024-03-08T20:59:02.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:02.314+0000 7fc35d800700 1 --2- 172.21.15.107:0/3254370519 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc35810e480 0x7fc35819e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T20:59:02.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:02.314+0000 7fc357e00700 1 --2- 172.21.15.107:0/3254370519 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc358108db0 0x7fc3581a4be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T20:59:02.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:02.314+0000 7fc35ce00700 1 --2- 172.21.15.107:0/3254370519 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc3581083f0 0x7fc3581a46a0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fc34c00c9b0 tx=0x7fc34c00cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:59:02.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:02.314+0000 7fc355600700 1 -- 172.21.15.107:0/3254370519 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc34c007b00 con 0x7fc3581083f0 2024-03-08T20:59:02.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:02.315+0000 7fc355600700 1 -- 172.21.15.107:0/3254370519 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fc34c007ca0 con 0x7fc3581083f0 2024-03-08T20:59:02.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:02.315+0000 7fc35f800700 1 -- 172.21.15.107:0/3254370519 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc35819f340 con 0x7fc3581083f0 2024-03-08T20:59:02.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:02.315+0000 7fc355600700 1 -- 172.21.15.107:0/3254370519 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc34c018730 con 0x7fc3581083f0 2024-03-08T20:59:02.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:02.315+0000 7fc35f800700 1 -- 172.21.15.107:0/3254370519 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3581ab500 con 0x7fc3581083f0 2024-03-08T20:59:02.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:02.316+0000 7fc35f800700 1 -- 172.21.15.107:0/3254370519 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc358109b30 con 0x7fc3581083f0 2024-03-08T20:59:02.301 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:02.318+0000 7fc355600700 1 -- 172.21.15.107:0/3254370519 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) v1 ==== 137342+0+0 (secure 0 0 0) 0x7fc34c00f450 con 0x7fc3581083f0 2024-03-08T20:59:02.303 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:02.319+0000 7fc355600700 1 --2- 172.21.15.107:0/3254370519 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fc3400a29a0 0x7fc3400a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:02.303 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:02.319+0000 7fc355600700 1 -- 172.21.15.107:0/3254370519 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fc34c0ceba0 con 0x7fc3581083f0 2024-03-08T20:59:02.303 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:02.320+0000 7fc357e00700 1 --2- 172.21.15.107:0/3254370519 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fc3400a29a0 0x7fc3400a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:59:02.304 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:02.320+0000 7fc357e00700 1 --2- 172.21.15.107:0/3254370519 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fc3400a29a0 0x7fc3400a4e60 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fc35810d810 tx=0x7fc344005f30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:59:02.307 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:02.324+0000 7fc355600700 1 -- 172.21.15.107:0/3254370519 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7fc34c014030 con 0x7fc3581083f0 2024-03-08T20:59:02.548 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:02.564+0000 7fc35f800700 1 -- 172.21.15.107:0/3254370519 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fc35819fd50 con 0x7fc3581083f0 2024-03-08T20:59:02.548 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:02.564+0000 7fc355600700 1 -- 172.21.15.107:0/3254370519 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fc34c09baa0 con 0x7fc3581083f0 2024-03-08T20:59:02.548 INFO:teuthology.orchestra.run.smithi107.stdout:77309411421 2024-03-08T20:59:02.550 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:02.567+0000 7fc35f800700 1 -- 172.21.15.107:0/3254370519 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fc3400a29a0 msgr2=0x7fc3400a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:59:02.550 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:02.567+0000 7fc35f800700 1 --2- 172.21.15.107:0/3254370519 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fc3400a29a0 0x7fc3400a4e60 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fc35810d810 tx=0x7fc344005f30 comp rx=0 tx=0).stop 2024-03-08T20:59:02.550 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:02.567+0000 7fc35f800700 1 -- 172.21.15.107:0/3254370519 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc3581083f0 msgr2=0x7fc3581a46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:59:02.551 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:02.567+0000 7fc35f800700 1 --2- 172.21.15.107:0/3254370519 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc3581083f0 0x7fc3581a46a0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fc34c00c9b0 tx=0x7fc34c00cd70 comp rx=0 tx=0).stop 2024-03-08T20:59:02.551 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:02.567+0000 7fc35f800700 1 -- 172.21.15.107:0/3254370519 shutdown_connections 2024-03-08T20:59:02.551 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:02.567+0000 7fc35f800700 1 --2- 172.21.15.107:0/3254370519 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc35810e480 0x7fc35819e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:02.551 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:02.567+0000 7fc35f800700 1 --2- 172.21.15.107:0/3254370519 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fc3400a29a0 0x7fc3400a4e60 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:02.551 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:02.567+0000 7fc35f800700 1 --2- 172.21.15.107:0/3254370519 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc3581083f0 0x7fc3581a46a0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:02.551 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:02.567+0000 7fc35f800700 1 --2- 172.21.15.107:0/3254370519 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc358108db0 0x7fc3581a4be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:02.551 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:02.567+0000 7fc35f800700 1 -- 172.21.15.107:0/3254370519 >> 172.21.15.107:0/3254370519 conn(0x7fc3580755b0 msgr2=0x7fc358075ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:59:02.551 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:02.567+0000 7fc35f800700 1 -- 172.21.15.107:0/3254370519 shutdown_connections 2024-03-08T20:59:02.552 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:02.567+0000 7fc35f800700 1 -- 172.21.15.107:0/3254370519 wait complete. 2024-03-08T20:59:02.718 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:59:02 smithi107 ceph-mon[31381]: from='client.? 172.21.15.107:0/2950283342' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-03-08T20:59:02.765 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:59:02 smithi149 ceph-mon[35042]: from='client.? 172.21.15.107:0/2950283342' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-03-08T20:59:02.878 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:59:02 smithi204 ceph-mon[34935]: from='client.? 172.21.15.107:0/2950283342' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-03-08T20:59:03.031 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:03.046+0000 7f5f94800700 1 -- 172.21.15.107:0/4025259550 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5f8c104c40 msgr2=0x7f5f8c108c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:59:03.031 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:03.046+0000 7f5f94800700 1 --2- 172.21.15.107:0/4025259550 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5f8c104c40 0x7f5f8c108c90 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f5f780099c0 tx=0x7f5f78009cd0 comp rx=0 tx=0).stop 2024-03-08T20:59:03.031 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:03.046+0000 7f5f94800700 1 -- 172.21.15.107:0/4025259550 shutdown_connections 2024-03-08T20:59:03.031 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:03.046+0000 7f5f94800700 1 --2- 172.21.15.107:0/4025259550 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5f8c109370 0x7f5f8c110f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:03.031 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:03.046+0000 7f5f94800700 1 --2- 172.21.15.107:0/4025259550 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5f8c104290 0x7f5f8c104670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:03.031 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:03.047+0000 7f5f94800700 1 --2- 172.21.15.107:0/4025259550 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5f8c104c40 0x7f5f8c108c90 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:03.032 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:03.047+0000 7f5f94800700 1 -- 172.21.15.107:0/4025259550 >> 172.21.15.107:0/4025259550 conn(0x7f5f8c0ffb80 msgr2=0x7f5f8c101fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:59:03.032 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:03.047+0000 7f5f94800700 1 -- 172.21.15.107:0/4025259550 shutdown_connections 2024-03-08T20:59:03.032 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:03.047+0000 7f5f94800700 1 -- 172.21.15.107:0/4025259550 wait complete. 2024-03-08T20:59:03.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:03.048+0000 7f5f94800700 1 Processor -- start 2024-03-08T20:59:03.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:03.048+0000 7f5f94800700 1 -- start start 2024-03-08T20:59:03.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:03.048+0000 7f5f94800700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5f8c104290 0x7f5f8c1a0440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:03.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:03.048+0000 7f5f94800700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5f8c104c40 0x7f5f8c1a0980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:03.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:03.048+0000 7f5f94800700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5f8c109370 0x7f5f8c19a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:03.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:03.048+0000 7f5f94800700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f8c1142b0 con 0x7f5f8c104290 2024-03-08T20:59:03.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:03.048+0000 7f5f94800700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f8c114130 con 0x7f5f8c104c40 2024-03-08T20:59:03.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:03.048+0000 7f5f94800700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f8c114430 con 0x7f5f8c109370 2024-03-08T20:59:03.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:03.049+0000 7f5f91e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5f8c104290 0x7f5f8c1a0440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:59:03.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:03.049+0000 7f5f91e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5f8c104290 0x7f5f8c1a0440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:45910/0 (socket says 172.21.15.107:45910) 2024-03-08T20:59:03.035 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:03.049+0000 7f5f91e00700 1 -- 172.21.15.107:0/3570733864 learned_addr learned my addr 172.21.15.107:0/3570733864 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T20:59:03.035 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:03.049+0000 7f5f91400700 1 --2- 172.21.15.107:0/3570733864 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5f8c104c40 0x7f5f8c1a0980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:59:03.035 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:03.049+0000 7f5f92800700 1 --2- 172.21.15.107:0/3570733864 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5f8c109370 0x7f5f8c19a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:59:03.035 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:03.049+0000 7f5f91e00700 1 -- 172.21.15.107:0/3570733864 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5f8c109370 msgr2=0x7f5f8c19a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:59:03.035 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:03.049+0000 7f5f91e00700 1 --2- 172.21.15.107:0/3570733864 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5f8c109370 0x7f5f8c19a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:03.035 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:03.049+0000 7f5f91e00700 1 -- 172.21.15.107:0/3570733864 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5f8c104c40 msgr2=0x7f5f8c1a0980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:59:03.035 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:03.049+0000 7f5f91e00700 1 --2- 172.21.15.107:0/3570733864 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5f8c104c40 0x7f5f8c1a0980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:03.035 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:03.049+0000 7f5f91e00700 1 -- 172.21.15.107:0/3570733864 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f8c19adb0 con 0x7f5f8c104290 2024-03-08T20:59:03.035 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:03.049+0000 7f5f92800700 1 --2- 172.21.15.107:0/3570733864 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5f8c109370 0x7f5f8c19a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T20:59:03.035 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:03.049+0000 7f5f91e00700 1 --2- 172.21.15.107:0/3570733864 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5f8c104290 0x7f5f8c1a0440 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f5f8000c9b0 tx=0x7f5f8000cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:59:03.035 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:03.050+0000 7f5f86a00700 1 -- 172.21.15.107:0/3570733864 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f80007b00 con 0x7f5f8c104290 2024-03-08T20:59:03.035 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:03.050+0000 7f5f86a00700 1 -- 172.21.15.107:0/3570733864 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f5f80007ca0 con 0x7f5f8c104290 2024-03-08T20:59:03.036 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:03.050+0000 7f5f86a00700 1 -- 172.21.15.107:0/3570733864 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f80018730 con 0x7f5f8c104290 2024-03-08T20:59:03.036 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:03.050+0000 7f5f94800700 1 -- 172.21.15.107:0/3570733864 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f8c19b0a0 con 0x7f5f8c104290 2024-03-08T20:59:03.036 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:03.050+0000 7f5f94800700 1 -- 172.21.15.107:0/3570733864 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f8c1a7310 con 0x7f5f8c104290 2024-03-08T20:59:03.036 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:03.051+0000 7f5f94800700 1 -- 172.21.15.107:0/3570733864 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f8c068aa0 con 0x7f5f8c104290 2024-03-08T20:59:03.036 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:03.053+0000 7f5f86a00700 1 -- 172.21.15.107:0/3570733864 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) v1 ==== 137342+0+0 (secure 0 0 0) 0x7f5f8000f450 con 0x7f5f8c104290 2024-03-08T20:59:03.038 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:03.054+0000 7f5f86a00700 1 --2- 172.21.15.107:0/3570733864 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f5f740a29a0 0x7f5f740a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:03.038 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:03.054+0000 7f5f86a00700 1 -- 172.21.15.107:0/3570733864 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f5f800cec40 con 0x7f5f8c104290 2024-03-08T20:59:03.038 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:03.055+0000 7f5f91400700 1 --2- 172.21.15.107:0/3570733864 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f5f740a29a0 0x7f5f740a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:59:03.039 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:03.055+0000 7f5f91400700 1 --2- 172.21.15.107:0/3570733864 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f5f740a29a0 0x7f5f740a4e60 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f5f78000c00 tx=0x7f5f78005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:59:03.042 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:03.058+0000 7f5f86a00700 1 -- 172.21.15.107:0/3570733864 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f5f80014030 con 0x7f5f8c104290 2024-03-08T20:59:03.269 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:03.285+0000 7f5f94800700 1 -- 172.21.15.107:0/3570733864 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f5f8c19bb30 con 0x7f5f8c104290 2024-03-08T20:59:03.270 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:03.286+0000 7f5f86a00700 1 -- 172.21.15.107:0/3570733864 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f5f8009bb40 con 0x7f5f8c104290 2024-03-08T20:59:03.270 INFO:teuthology.orchestra.run.smithi107.stdout:141733920848 2024-03-08T20:59:03.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:03.289+0000 7f5f94800700 1 -- 172.21.15.107:0/3570733864 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f5f740a29a0 msgr2=0x7f5f740a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:59:03.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:03.289+0000 7f5f94800700 1 --2- 172.21.15.107:0/3570733864 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f5f740a29a0 0x7f5f740a4e60 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f5f78000c00 tx=0x7f5f78005fb0 comp rx=0 tx=0).stop 2024-03-08T20:59:03.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:03.289+0000 7f5f94800700 1 -- 172.21.15.107:0/3570733864 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5f8c104290 msgr2=0x7f5f8c1a0440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:59:03.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:03.289+0000 7f5f94800700 1 --2- 172.21.15.107:0/3570733864 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5f8c104290 0x7f5f8c1a0440 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f5f8000c9b0 tx=0x7f5f8000cd70 comp rx=0 tx=0).stop 2024-03-08T20:59:03.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:03.289+0000 7f5f94800700 1 -- 172.21.15.107:0/3570733864 shutdown_connections 2024-03-08T20:59:03.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:03.289+0000 7f5f94800700 1 --2- 172.21.15.107:0/3570733864 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5f8c109370 0x7f5f8c19a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:03.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:03.289+0000 7f5f94800700 1 --2- 172.21.15.107:0/3570733864 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f5f740a29a0 0x7f5f740a4e60 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:03.274 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:03.289+0000 7f5f94800700 1 --2- 172.21.15.107:0/3570733864 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5f8c104290 0x7f5f8c1a0440 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:03.274 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:03.289+0000 7f5f94800700 1 --2- 172.21.15.107:0/3570733864 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5f8c104c40 0x7f5f8c1a0980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:03.274 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:03.289+0000 7f5f94800700 1 -- 172.21.15.107:0/3570733864 >> 172.21.15.107:0/3570733864 conn(0x7f5f8c0ffb80 msgr2=0x7f5f8c101e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:59:03.274 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:03.289+0000 7f5f94800700 1 -- 172.21.15.107:0/3570733864 shutdown_connections 2024-03-08T20:59:03.274 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:03.289+0000 7f5f94800700 1 -- 172.21.15.107:0/3570733864 wait complete. 2024-03-08T20:59:03.717 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:59:03 smithi107 ceph-mon[31381]: pgmap v399: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-08T20:59:03.718 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:59:03 smithi107 ceph-mon[31381]: from='client.? 172.21.15.107:0/3254370519' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-03-08T20:59:03.718 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:59:03 smithi107 ceph-mon[31381]: from='client.? 172.21.15.107:0/3570733864' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-03-08T20:59:03.765 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:59:03 smithi149 ceph-mon[35042]: pgmap v399: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-08T20:59:03.765 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:59:03 smithi149 ceph-mon[35042]: from='client.? 172.21.15.107:0/3254370519' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-03-08T20:59:03.765 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:59:03 smithi149 ceph-mon[35042]: from='client.? 172.21.15.107:0/3570733864' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-03-08T20:59:03.780 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:59:03 smithi204 ceph-mon[34935]: pgmap v399: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-08T20:59:03.780 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:59:03 smithi204 ceph-mon[34935]: from='client.? 172.21.15.107:0/3254370519' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-03-08T20:59:03.780 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:59:03 smithi204 ceph-mon[34935]: from='client.? 172.21.15.107:0/3570733864' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-03-08T20:59:05.203 INFO:tasks.cephadm.ceph_manager.ceph:need seq 141733920843 got 141733920848 for osd.4 2024-03-08T20:59:05.203 DEBUG:teuthology.parallel:result is None 2024-03-08T20:59:05.707 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:59:05 smithi107 ceph-mon[31381]: pgmap v400: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-03-08T20:59:05.765 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:59:05 smithi149 ceph-mon[35042]: pgmap v400: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-03-08T20:59:05.878 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:59:05 smithi204 ceph-mon[34935]: pgmap v400: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-03-08T20:59:06.555 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:59:06 smithi107 ceph-mon[31381]: from='client.? 172.21.15.107:0/1049643103' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-03-08T20:59:06.741 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:06.756+0000 7fb78d200700 1 -- 172.21.15.107:0/339637104 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb788101680 msgr2=0x7fb78810db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:59:06.741 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:06.756+0000 7fb78d200700 1 --2- 172.21.15.107:0/339637104 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb788101680 0x7fb78810db60 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fb78000b0a0 tx=0x7fb78000b3b0 comp rx=0 tx=0).stop 2024-03-08T20:59:06.741 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:06.757+0000 7fb78d200700 1 -- 172.21.15.107:0/339637104 shutdown_connections 2024-03-08T20:59:06.741 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:06.757+0000 7fb78d200700 1 --2- 172.21.15.107:0/339637104 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb788101680 0x7fb78810db60 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:06.741 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:06.757+0000 7fb78d200700 1 --2- 172.21.15.107:0/339637104 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb788106d20 0x7fb788107100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:06.741 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:06.757+0000 7fb78d200700 1 --2- 172.21.15.107:0/339637104 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb788100ce0 0x7fb788101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:06.741 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:06.757+0000 7fb78d200700 1 -- 172.21.15.107:0/339637104 >> 172.21.15.107:0/339637104 conn(0x7fb788077c10 msgr2=0x7fb788078020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:59:06.741 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:06.757+0000 7fb78d200700 1 -- 172.21.15.107:0/339637104 shutdown_connections 2024-03-08T20:59:06.741 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:06.757+0000 7fb78d200700 1 -- 172.21.15.107:0/339637104 wait complete. 2024-03-08T20:59:06.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:06.758+0000 7fb78d200700 1 Processor -- start 2024-03-08T20:59:06.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:06.758+0000 7fb78d200700 1 -- start start 2024-03-08T20:59:06.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:06.758+0000 7fb78d200700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb788100ce0 0x7fb788102990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:06.744 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:06.758+0000 7fb78d200700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb788101680 0x7fb788102ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:06.744 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:06.758+0000 7fb78d200700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb788106d20 0x7fb788103410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:06.744 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:06.758+0000 7fb78d200700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb788110f20 con 0x7fb788106d20 2024-03-08T20:59:06.744 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:06.758+0000 7fb78d200700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb788110da0 con 0x7fb788101680 2024-03-08T20:59:06.744 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:06.758+0000 7fb78d200700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7881110a0 con 0x7fb788100ce0 2024-03-08T20:59:06.744 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:06.759+0000 7fb786800700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb788106d20 0x7fb788103410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:59:06.744 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:06.759+0000 7fb786800700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb788106d20 0x7fb788103410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:57168/0 (socket says 172.21.15.107:57168) 2024-03-08T20:59:06.744 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:06.759+0000 7fb786800700 1 -- 172.21.15.107:0/1329896680 learned_addr learned my addr 172.21.15.107:0/1329896680 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T20:59:06.744 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:06.759+0000 7fb785400700 1 --2- 172.21.15.107:0/1329896680 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb788101680 0x7fb788102ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:59:06.744 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:06.759+0000 7fb785e00700 1 --2- 172.21.15.107:0/1329896680 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb788100ce0 0x7fb788102990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:59:06.744 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:06.759+0000 7fb786800700 1 -- 172.21.15.107:0/1329896680 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb788100ce0 msgr2=0x7fb788102990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:59:06.745 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:06.759+0000 7fb786800700 1 --2- 172.21.15.107:0/1329896680 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb788100ce0 0x7fb788102990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:06.745 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:06.759+0000 7fb786800700 1 -- 172.21.15.107:0/1329896680 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb788101680 msgr2=0x7fb788102ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:59:06.745 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:06.759+0000 7fb786800700 1 --2- 172.21.15.107:0/1329896680 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb788101680 0x7fb788102ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:06.745 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:06.759+0000 7fb786800700 1 -- 172.21.15.107:0/1329896680 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7881a69f0 con 0x7fb788106d20 2024-03-08T20:59:06.745 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:06.759+0000 7fb786800700 1 --2- 172.21.15.107:0/1329896680 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb788106d20 0x7fb788103410 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fb78000b070 tx=0x7fb78000cf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:59:06.745 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:06.760+0000 7fb77ea00700 1 -- 172.21.15.107:0/1329896680 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb780021070 con 0x7fb788106d20 2024-03-08T20:59:06.745 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:06.760+0000 7fb77ea00700 1 -- 172.21.15.107:0/1329896680 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fb780007cf0 con 0x7fb788106d20 2024-03-08T20:59:06.745 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:06.760+0000 7fb78d200700 1 -- 172.21.15.107:0/1329896680 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7881a6c80 con 0x7fb788106d20 2024-03-08T20:59:06.745 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:06.760+0000 7fb77ea00700 1 -- 172.21.15.107:0/1329896680 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb780005440 con 0x7fb788106d20 2024-03-08T20:59:06.746 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:06.760+0000 7fb78d200700 1 -- 172.21.15.107:0/1329896680 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7881a7130 con 0x7fb788106d20 2024-03-08T20:59:06.746 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:06.762+0000 7fb78d200700 1 -- 172.21.15.107:0/1329896680 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb788068aa0 con 0x7fb788106d20 2024-03-08T20:59:06.747 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:06.763+0000 7fb77ea00700 1 -- 172.21.15.107:0/1329896680 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) v1 ==== 137342+0+0 (secure 0 0 0) 0x7fb780024070 con 0x7fb788106d20 2024-03-08T20:59:06.748 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:06.765+0000 7fb77ea00700 1 --2- 172.21.15.107:0/1329896680 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fb76c0a29a0 0x7fb76c0a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:06.749 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:06.765+0000 7fb77ea00700 1 -- 172.21.15.107:0/1329896680 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fb7800cfea0 con 0x7fb788106d20 2024-03-08T20:59:06.749 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:06.765+0000 7fb785e00700 1 --2- 172.21.15.107:0/1329896680 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fb76c0a29a0 0x7fb76c0a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:59:06.749 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:06.766+0000 7fb785e00700 1 --2- 172.21.15.107:0/1329896680 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fb76c0a29a0 0x7fb76c0a4e60 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fb778005fd0 tx=0x7fb77800a560 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:59:06.753 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:06.769+0000 7fb77ea00700 1 -- 172.21.15.107:0/1329896680 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7fb78009ce30 con 0x7fb788106d20 2024-03-08T20:59:06.765 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:59:06 smithi149 ceph-mon[35042]: from='client.? 172.21.15.107:0/1049643103' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-03-08T20:59:06.878 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:59:06 smithi204 ceph-mon[34935]: from='client.? 172.21.15.107:0/1049643103' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-03-08T20:59:06.970 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:06.986+0000 7fb78d200700 1 -- 172.21.15.107:0/1329896680 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7fb7881a7460 con 0x7fb788106d20 2024-03-08T20:59:06.970 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:06.987+0000 7fb77ea00700 1 -- 172.21.15.107:0/1329896680 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb78009c730 con 0x7fb788106d20 2024-03-08T20:59:06.971 INFO:teuthology.orchestra.run.smithi107.stdout:167503724618 2024-03-08T20:59:06.973 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:06.989+0000 7fb78d200700 1 -- 172.21.15.107:0/1329896680 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fb76c0a29a0 msgr2=0x7fb76c0a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:59:06.973 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:06.989+0000 7fb78d200700 1 --2- 172.21.15.107:0/1329896680 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fb76c0a29a0 0x7fb76c0a4e60 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fb778005fd0 tx=0x7fb77800a560 comp rx=0 tx=0).stop 2024-03-08T20:59:06.973 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:06.989+0000 7fb78d200700 1 -- 172.21.15.107:0/1329896680 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb788106d20 msgr2=0x7fb788103410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:59:06.974 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:06.989+0000 7fb78d200700 1 --2- 172.21.15.107:0/1329896680 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb788106d20 0x7fb788103410 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fb78000b070 tx=0x7fb78000cf00 comp rx=0 tx=0).stop 2024-03-08T20:59:06.974 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:06.989+0000 7fb78d200700 1 -- 172.21.15.107:0/1329896680 shutdown_connections 2024-03-08T20:59:06.974 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:06.989+0000 7fb78d200700 1 --2- 172.21.15.107:0/1329896680 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb788100ce0 0x7fb788102990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:06.974 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:06.989+0000 7fb78d200700 1 --2- 172.21.15.107:0/1329896680 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fb76c0a29a0 0x7fb76c0a4e60 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:06.974 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:06.989+0000 7fb78d200700 1 --2- 172.21.15.107:0/1329896680 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb788106d20 0x7fb788103410 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:06.974 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:06.989+0000 7fb78d200700 1 --2- 172.21.15.107:0/1329896680 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb788101680 0x7fb788102ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:06.974 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:06.989+0000 7fb78d200700 1 -- 172.21.15.107:0/1329896680 >> 172.21.15.107:0/1329896680 conn(0x7fb788077c10 msgr2=0x7fb788100010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:59:06.974 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:06.990+0000 7fb78d200700 1 -- 172.21.15.107:0/1329896680 shutdown_connections 2024-03-08T20:59:06.974 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:06.990+0000 7fb78d200700 1 -- 172.21.15.107:0/1329896680 wait complete. 2024-03-08T20:59:07.718 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:59:07 smithi107 ceph-mon[31381]: pgmap v401: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-08T20:59:07.718 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:59:07 smithi107 ceph-mon[31381]: from='client.? 172.21.15.107:0/2884605620' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-03-08T20:59:07.718 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:59:07 smithi107 ceph-mon[31381]: from='client.? 172.21.15.107:0/1329896680' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-03-08T20:59:07.765 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:59:07 smithi149 ceph-mon[35042]: pgmap v401: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-08T20:59:07.765 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:59:07 smithi149 ceph-mon[35042]: from='client.? 172.21.15.107:0/2884605620' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-03-08T20:59:07.765 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:59:07 smithi149 ceph-mon[35042]: from='client.? 172.21.15.107:0/1329896680' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-03-08T20:59:07.864 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:05.886+0000 7f5962800700 1 -- 172.21.15.107:0/72735024 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f595c06a320 msgr2=0x7f595c06a700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:59:07.864 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:05.886+0000 7f5962800700 1 --2- 172.21.15.107:0/72735024 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f595c06a320 0x7f595c06a700 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f594c009990 tx=0x7f594c009ca0 comp rx=0 tx=0).stop 2024-03-08T20:59:07.864 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:05.886+0000 7f5962800700 1 -- 172.21.15.107:0/72735024 shutdown_connections 2024-03-08T20:59:07.864 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:05.886+0000 7f5962800700 1 --2- 172.21.15.107:0/72735024 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f595c10e8c0 0x7f595c110cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:07.864 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:05.886+0000 7f5962800700 1 --2- 172.21.15.107:0/72735024 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f595c06a320 0x7f595c06a700 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:07.864 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:05.886+0000 7f5962800700 1 --2- 172.21.15.107:0/72735024 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f595c06ac40 0x7f595c10e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:07.864 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:05.886+0000 7f5962800700 1 -- 172.21.15.107:0/72735024 >> 172.21.15.107:0/72735024 conn(0x7f595c076d30 msgr2=0x7f595c077140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:59:07.865 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:05.886+0000 7f5962800700 1 -- 172.21.15.107:0/72735024 shutdown_connections 2024-03-08T20:59:07.865 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:05.886+0000 7f5962800700 1 -- 172.21.15.107:0/72735024 wait complete. 2024-03-08T20:59:07.865 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:05.887+0000 7f5962800700 1 Processor -- start 2024-03-08T20:59:07.865 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:05.887+0000 7f5962800700 1 -- start start 2024-03-08T20:59:07.865 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:05.887+0000 7f5962800700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f595c06a320 0x7f595c1a0560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:07.865 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:05.887+0000 7f5962800700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f595c06ac40 0x7f595c1a0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:07.865 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:05.887+0000 7f5962800700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f595c10e8c0 0x7f595c19a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:07.865 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:05.887+0000 7f5962800700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f595c114210 con 0x7f595c10e8c0 2024-03-08T20:59:07.865 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:05.887+0000 7f5962800700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f595c114090 con 0x7f595c06a320 2024-03-08T20:59:07.865 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:05.887+0000 7f5962800700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f595c114390 con 0x7f595c06ac40 2024-03-08T20:59:07.865 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:05.888+0000 7f595be00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f595c10e8c0 0x7f595c19a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:59:07.865 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:05.888+0000 7f595be00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f595c10e8c0 0x7f595c19a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:45926/0 (socket says 172.21.15.107:45926) 2024-03-08T20:59:07.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:05.888+0000 7f595be00700 1 -- 172.21.15.107:0/1049643103 learned_addr learned my addr 172.21.15.107:0/1049643103 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T20:59:07.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:05.888+0000 7f595b400700 1 --2- 172.21.15.107:0/1049643103 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f595c06a320 0x7f595c1a0560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:59:07.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:05.888+0000 7f595aa00700 1 --2- 172.21.15.107:0/1049643103 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f595c06ac40 0x7f595c1a0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:59:07.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:05.888+0000 7f595be00700 1 -- 172.21.15.107:0/1049643103 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f595c06ac40 msgr2=0x7f595c1a0aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:59:07.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:05.888+0000 7f595be00700 1 --2- 172.21.15.107:0/1049643103 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f595c06ac40 0x7f595c1a0aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:07.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:05.888+0000 7f595be00700 1 -- 172.21.15.107:0/1049643103 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f595c06a320 msgr2=0x7f595c1a0560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:59:07.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:05.888+0000 7f595be00700 1 --2- 172.21.15.107:0/1049643103 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f595c06a320 0x7f595c1a0560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:07.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:05.888+0000 7f595be00700 1 -- 172.21.15.107:0/1049643103 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f595c19af10 con 0x7f595c10e8c0 2024-03-08T20:59:07.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:05.888+0000 7f595be00700 1 --2- 172.21.15.107:0/1049643103 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f595c10e8c0 0x7f595c19a630 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f595400ada0 tx=0x7f595400c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:59:07.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:05.889+0000 7f5953e00700 1 -- 172.21.15.107:0/1049643103 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5954017020 con 0x7f595c10e8c0 2024-03-08T20:59:07.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:05.889+0000 7f5953e00700 1 -- 172.21.15.107:0/1049643103 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f595400ce30 con 0x7f595c10e8c0 2024-03-08T20:59:07.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:05.889+0000 7f5953e00700 1 -- 172.21.15.107:0/1049643103 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5954005460 con 0x7f595c10e8c0 2024-03-08T20:59:07.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:05.889+0000 7f5962800700 1 -- 172.21.15.107:0/1049643103 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f595c19b200 con 0x7f595c10e8c0 2024-03-08T20:59:07.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:05.889+0000 7f5962800700 1 -- 172.21.15.107:0/1049643103 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f595c1a74e0 con 0x7f595c10e8c0 2024-03-08T20:59:07.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:05.890+0000 7f5962800700 1 -- 172.21.15.107:0/1049643103 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f595c1012e0 con 0x7f595c10e8c0 2024-03-08T20:59:07.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:05.893+0000 7f5953e00700 1 -- 172.21.15.107:0/1049643103 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) v1 ==== 137342+0+0 (secure 0 0 0) 0x7f595400c980 con 0x7f595c10e8c0 2024-03-08T20:59:07.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:05.894+0000 7f5953e00700 1 --2- 172.21.15.107:0/1049643103 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f59400a2920 0x7f59400a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:07.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:05.894+0000 7f5953e00700 1 -- 172.21.15.107:0/1049643103 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f59540ce490 con 0x7f595c10e8c0 2024-03-08T20:59:07.867 INFO:teuthology.orchestra.run.smithi107.stdout:223338299450 2024-03-08T20:59:07.868 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:05.895+0000 7f595b400700 1 --2- 172.21.15.107:0/1049643103 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f59400a2920 0x7f59400a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:59:07.868 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:05.896+0000 7f595b400700 1 --2- 172.21.15.107:0/1049643103 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f59400a2920 0x7f59400a4de0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f594c00b5c0 tx=0x7f594c01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:59:07.868 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:05.899+0000 7f5953e00700 1 -- 172.21.15.107:0/1049643103 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f5954021020 con 0x7f595c10e8c0 2024-03-08T20:59:07.868 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:06.132+0000 7f5962800700 1 -- 172.21.15.107:0/1049643103 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f595c19b430 con 0x7f595c10e8c0 2024-03-08T20:59:07.869 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:06.132+0000 7f5953e00700 1 -- 172.21.15.107:0/1049643103 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f595409b460 con 0x7f595c10e8c0 2024-03-08T20:59:07.869 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:06.135+0000 7f5962800700 1 -- 172.21.15.107:0/1049643103 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f59400a2920 msgr2=0x7f59400a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:59:07.869 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:06.135+0000 7f5962800700 1 --2- 172.21.15.107:0/1049643103 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f59400a2920 0x7f59400a4de0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f594c00b5c0 tx=0x7f594c01a040 comp rx=0 tx=0).stop 2024-03-08T20:59:07.869 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:06.135+0000 7f5962800700 1 -- 172.21.15.107:0/1049643103 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f595c10e8c0 msgr2=0x7f595c19a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:59:07.869 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:06.135+0000 7f5962800700 1 --2- 172.21.15.107:0/1049643103 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f595c10e8c0 0x7f595c19a630 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f595400ada0 tx=0x7f595400c330 comp rx=0 tx=0).stop 2024-03-08T20:59:07.869 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:06.135+0000 7f5962800700 1 -- 172.21.15.107:0/1049643103 shutdown_connections 2024-03-08T20:59:07.869 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:06.135+0000 7f5962800700 1 --2- 172.21.15.107:0/1049643103 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f59400a2920 0x7f59400a4de0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:07.869 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:06.135+0000 7f5962800700 1 --2- 172.21.15.107:0/1049643103 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f595c10e8c0 0x7f595c19a630 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:07.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:06.135+0000 7f5962800700 1 --2- 172.21.15.107:0/1049643103 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f595c06a320 0x7f595c1a0560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:07.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:06.135+0000 7f5962800700 1 --2- 172.21.15.107:0/1049643103 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f595c06ac40 0x7f595c1a0aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:07.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:06.135+0000 7f5962800700 1 -- 172.21.15.107:0/1049643103 >> 172.21.15.107:0/1049643103 conn(0x7f595c076d30 msgr2=0x7f595c10ffa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:59:07.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:06.135+0000 7f5962800700 1 -- 172.21.15.107:0/1049643103 shutdown_connections 2024-03-08T20:59:07.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:06.135+0000 7f5962800700 1 -- 172.21.15.107:0/1049643103 wait complete. 2024-03-08T20:59:07.878 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:59:07 smithi204 ceph-mon[34935]: pgmap v401: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-08T20:59:07.878 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:59:07 smithi204 ceph-mon[34935]: from='client.? 172.21.15.107:0/2884605620' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-03-08T20:59:07.879 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:59:07 smithi204 ceph-mon[34935]: from='client.? 172.21.15.107:0/1329896680' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-03-08T20:59:08.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:06.314+0000 7f8d50400700 1 -- 172.21.15.107:0/4260212407 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8d48107160 msgr2=0x7f8d4810ed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:59:08.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:06.314+0000 7f8d50400700 1 --2- 172.21.15.107:0/4260212407 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8d48107160 0x7f8d4810ed30 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f8d4000b0d0 tx=0x7f8d4000b3e0 comp rx=0 tx=0).stop 2024-03-08T20:59:08.292 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:06.314+0000 7f8d50400700 1 -- 172.21.15.107:0/4260212407 shutdown_connections 2024-03-08T20:59:08.292 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:06.314+0000 7f8d50400700 1 --2- 172.21.15.107:0/4260212407 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8d48107160 0x7f8d4810ed30 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:08.292 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:06.314+0000 7f8d50400700 1 --2- 172.21.15.107:0/4260212407 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8d48102090 0x7f8d48102470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:08.292 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:06.314+0000 7f8d50400700 1 --2- 172.21.15.107:0/4260212407 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8d48102a40 0x7f8d48106a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:08.292 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:06.314+0000 7f8d50400700 1 -- 172.21.15.107:0/4260212407 >> 172.21.15.107:0/4260212407 conn(0x7f8d48078f40 msgr2=0x7f8d48079350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:59:08.292 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:06.314+0000 7f8d50400700 1 -- 172.21.15.107:0/4260212407 shutdown_connections 2024-03-08T20:59:08.292 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:06.314+0000 7f8d50400700 1 -- 172.21.15.107:0/4260212407 wait complete. 2024-03-08T20:59:08.292 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:06.315+0000 7f8d50400700 1 Processor -- start 2024-03-08T20:59:08.292 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:06.316+0000 7f8d50400700 1 -- start start 2024-03-08T20:59:08.292 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:06.316+0000 7f8d50400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8d48102090 0x7f8d4819a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:08.292 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:06.316+0000 7f8d50400700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8d48102a40 0x7f8d4819abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:08.292 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:06.316+0000 7f8d50400700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8d48107160 0x7f8d4819ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:08.292 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:06.316+0000 7f8d50400700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d48112070 con 0x7f8d48102090 2024-03-08T20:59:08.293 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:06.316+0000 7f8d50400700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d48111ef0 con 0x7f8d48102a40 2024-03-08T20:59:08.293 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:06.316+0000 7f8d50400700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d481121f0 con 0x7f8d48107160 2024-03-08T20:59:08.293 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:06.316+0000 7f8d4da00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8d48102090 0x7f8d4819a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:59:08.293 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:06.316+0000 7f8d4da00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8d48102090 0x7f8d4819a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:57144/0 (socket says 172.21.15.107:57144) 2024-03-08T20:59:08.293 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:06.316+0000 7f8d4da00700 1 -- 172.21.15.107:0/2884605620 learned_addr learned my addr 172.21.15.107:0/2884605620 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T20:59:08.293 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:06.317+0000 7f8d4da00700 1 -- 172.21.15.107:0/2884605620 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8d48107160 msgr2=0x7f8d4819ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:59:08.293 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:06.317+0000 7f8d4da00700 1 --2- 172.21.15.107:0/2884605620 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8d48107160 0x7f8d4819ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:08.293 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:06.317+0000 7f8d4da00700 1 -- 172.21.15.107:0/2884605620 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8d48102a40 msgr2=0x7f8d4819abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:59:08.293 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:06.317+0000 7f8d4da00700 1 --2- 172.21.15.107:0/2884605620 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8d48102a40 0x7f8d4819abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:08.293 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:06.317+0000 7f8d4da00700 1 -- 172.21.15.107:0/2884605620 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d4819f690 con 0x7f8d48102090 2024-03-08T20:59:08.293 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:06.317+0000 7f8d4da00700 1 --2- 172.21.15.107:0/2884605620 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8d48102090 0x7f8d4819a680 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f8d3c00c9b0 tx=0x7f8d3c00cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:59:08.293 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:06.317+0000 7f8d46000700 1 -- 172.21.15.107:0/2884605620 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d3c007b00 con 0x7f8d48102090 2024-03-08T20:59:08.293 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:06.317+0000 7f8d46000700 1 -- 172.21.15.107:0/2884605620 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f8d3c007ca0 con 0x7f8d48102090 2024-03-08T20:59:08.294 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:06.317+0000 7f8d50400700 1 -- 172.21.15.107:0/2884605620 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d4819f980 con 0x7f8d48102090 2024-03-08T20:59:08.294 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:06.317+0000 7f8d46000700 1 -- 172.21.15.107:0/2884605620 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d3c018730 con 0x7f8d48102090 2024-03-08T20:59:08.294 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:06.318+0000 7f8d50400700 1 -- 172.21.15.107:0/2884605620 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d481a7200 con 0x7f8d48102090 2024-03-08T20:59:08.294 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:06.319+0000 7f8d50400700 1 -- 172.21.15.107:0/2884605620 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d48068aa0 con 0x7f8d48102090 2024-03-08T20:59:08.294 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:06.321+0000 7f8d46000700 1 -- 172.21.15.107:0/2884605620 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) v1 ==== 137342+0+0 (secure 0 0 0) 0x7f8d3c00f450 con 0x7f8d48102090 2024-03-08T20:59:08.294 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:06.322+0000 7f8d46000700 1 --2- 172.21.15.107:0/2884605620 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f8d300a29a0 0x7f8d300a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:08.294 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:06.322+0000 7f8d46000700 1 -- 172.21.15.107:0/2884605620 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f8d3c0cef60 con 0x7f8d48102090 2024-03-08T20:59:08.294 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:06.322+0000 7f8d4d000700 1 --2- 172.21.15.107:0/2884605620 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f8d300a29a0 0x7f8d300a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:59:08.294 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:06.323+0000 7f8d4d000700 1 --2- 172.21.15.107:0/2884605620 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f8d300a29a0 0x7f8d300a4e60 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f8d481089d0 tx=0x7f8d34006d20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:59:08.294 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:06.325+0000 7f8d46000700 1 -- 172.21.15.107:0/2884605620 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f8d3c014030 con 0x7f8d48102090 2024-03-08T20:59:08.294 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:06.552+0000 7f8d50400700 1 -- 172.21.15.107:0/2884605620 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) v1 -- 0x7f8d4819b9e0 con 0x7f8d48102090 2024-03-08T20:59:08.294 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:06.552+0000 7f8d46000700 1 -- 172.21.15.107:0/2884605620 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f8d3c09bee0 con 0x7f8d48102090 2024-03-08T20:59:08.295 INFO:teuthology.orchestra.run.smithi107.stdout:257698037812 2024-03-08T20:59:08.295 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:06.555+0000 7f8d50400700 1 -- 172.21.15.107:0/2884605620 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f8d300a29a0 msgr2=0x7f8d300a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:59:08.295 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:06.555+0000 7f8d50400700 1 --2- 172.21.15.107:0/2884605620 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f8d300a29a0 0x7f8d300a4e60 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f8d481089d0 tx=0x7f8d34006d20 comp rx=0 tx=0).stop 2024-03-08T20:59:08.295 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:06.555+0000 7f8d50400700 1 -- 172.21.15.107:0/2884605620 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8d48102090 msgr2=0x7f8d4819a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:59:08.295 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:06.555+0000 7f8d50400700 1 --2- 172.21.15.107:0/2884605620 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8d48102090 0x7f8d4819a680 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f8d3c00c9b0 tx=0x7f8d3c00cd70 comp rx=0 tx=0).stop 2024-03-08T20:59:08.296 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:06.555+0000 7f8d50400700 1 -- 172.21.15.107:0/2884605620 shutdown_connections 2024-03-08T20:59:08.296 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:06.555+0000 7f8d50400700 1 --2- 172.21.15.107:0/2884605620 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8d48107160 0x7f8d4819ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:08.296 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:06.555+0000 7f8d50400700 1 --2- 172.21.15.107:0/2884605620 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f8d300a29a0 0x7f8d300a4e60 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:08.296 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:06.555+0000 7f8d50400700 1 --2- 172.21.15.107:0/2884605620 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8d48102090 0x7f8d4819a680 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:08.296 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:06.555+0000 7f8d50400700 1 --2- 172.21.15.107:0/2884605620 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8d48102a40 0x7f8d4819abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:08.296 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:06.555+0000 7f8d50400700 1 -- 172.21.15.107:0/2884605620 >> 172.21.15.107:0/2884605620 conn(0x7f8d48078f40 msgr2=0x7f8d4810d5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:59:08.296 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:06.555+0000 7f8d50400700 1 -- 172.21.15.107:0/2884605620 shutdown_connections 2024-03-08T20:59:08.296 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:06.555+0000 7f8d50400700 1 -- 172.21.15.107:0/2884605620 wait complete. 2024-03-08T20:59:08.788 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411416 got 77309411421 for osd.2 2024-03-08T20:59:08.788 DEBUG:teuthology.parallel:result is None 2024-03-08T20:59:09.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:59:09 smithi107 ceph-mon[31381]: pgmap v402: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-03-08T20:59:09.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:59:09 smithi107 ceph-mon[31381]: from='client.? 172.21.15.107:0/2625518369' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-03-08T20:59:09.612 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:08.351+0000 7fad85000700 1 -- 172.21.15.107:0/377731414 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fad80109210 msgr2=0x7fad80110de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:59:09.612 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:08.351+0000 7fad85000700 1 --2- 172.21.15.107:0/377731414 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fad80109210 0x7fad80110de0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fad7800b0d0 tx=0x7fad7800b3e0 comp rx=0 tx=0).stop 2024-03-08T20:59:09.612 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:08.351+0000 7fad85000700 1 -- 172.21.15.107:0/377731414 shutdown_connections 2024-03-08T20:59:09.612 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:08.351+0000 7fad85000700 1 --2- 172.21.15.107:0/377731414 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fad80104130 0x7fad80104510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:09.612 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:08.351+0000 7fad85000700 1 --2- 172.21.15.107:0/377731414 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fad80109210 0x7fad80110de0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:09.612 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:08.351+0000 7fad85000700 1 --2- 172.21.15.107:0/377731414 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fad80104ae0 0x7fad80108b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:09.612 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:08.351+0000 7fad85000700 1 -- 172.21.15.107:0/377731414 >> 172.21.15.107:0/377731414 conn(0x7fad800ffa00 msgr2=0x7fad80101e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:59:09.612 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:08.351+0000 7fad85000700 1 -- 172.21.15.107:0/377731414 shutdown_connections 2024-03-08T20:59:09.612 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:08.351+0000 7fad85000700 1 -- 172.21.15.107:0/377731414 wait complete. 2024-03-08T20:59:09.612 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:08.352+0000 7fad85000700 1 Processor -- start 2024-03-08T20:59:09.613 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:08.352+0000 7fad85000700 1 -- start start 2024-03-08T20:59:09.613 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:08.352+0000 7fad85000700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fad80104130 0x7fad8019a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:09.613 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:08.352+0000 7fad85000700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fad80104ae0 0x7fad8019aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:09.613 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:08.352+0000 7fad85000700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fad80109210 0x7fad8019ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:09.613 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:08.352+0000 7fad85000700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad801140d0 con 0x7fad80104ae0 2024-03-08T20:59:09.613 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:08.352+0000 7fad85000700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad80113f50 con 0x7fad80109210 2024-03-08T20:59:09.613 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:08.352+0000 7fad85000700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad80114250 con 0x7fad80104130 2024-03-08T20:59:09.613 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:08.353+0000 7fad7ea00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fad80104ae0 0x7fad8019aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:59:09.613 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:08.353+0000 7fad7ea00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fad80104ae0 0x7fad8019aa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:57192/0 (socket says 172.21.15.107:57192) 2024-03-08T20:59:09.613 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:08.353+0000 7fad7ea00700 1 -- 172.21.15.107:0/2625518369 learned_addr learned my addr 172.21.15.107:0/2625518369 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T20:59:09.613 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:08.353+0000 7fad7f400700 1 --2- 172.21.15.107:0/2625518369 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fad80104130 0x7fad8019a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:59:09.613 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:08.353+0000 7fad7fe00700 1 --2- 172.21.15.107:0/2625518369 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fad80109210 0x7fad8019ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:59:09.613 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:08.353+0000 7fad7ea00700 1 -- 172.21.15.107:0/2625518369 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fad80104130 msgr2=0x7fad8019a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:59:09.613 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:08.353+0000 7fad7ea00700 1 --2- 172.21.15.107:0/2625518369 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fad80104130 0x7fad8019a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:09.614 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:08.353+0000 7fad7ea00700 1 -- 172.21.15.107:0/2625518369 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fad80109210 msgr2=0x7fad8019ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:59:09.614 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:08.353+0000 7fad7ea00700 1 --2- 172.21.15.107:0/2625518369 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fad80109210 0x7fad8019ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:09.614 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:08.353+0000 7fad7ea00700 1 -- 172.21.15.107:0/2625518369 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad8019f4a0 con 0x7fad80104ae0 2024-03-08T20:59:09.614 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:08.353+0000 7fad7ea00700 1 --2- 172.21.15.107:0/2625518369 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fad80104ae0 0x7fad8019aa20 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fad6c00c9e0 tx=0x7fad6c00cda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:59:09.614 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:08.354+0000 7fad77e00700 1 -- 172.21.15.107:0/2625518369 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad6c007aa0 con 0x7fad80104ae0 2024-03-08T20:59:09.614 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:08.354+0000 7fad77e00700 1 -- 172.21.15.107:0/2625518369 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fad6c007c40 con 0x7fad80104ae0 2024-03-08T20:59:09.614 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:08.354+0000 7fad85000700 1 -- 172.21.15.107:0/2625518369 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad8019f790 con 0x7fad80104ae0 2024-03-08T20:59:09.614 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:08.354+0000 7fad77e00700 1 -- 172.21.15.107:0/2625518369 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad6c018660 con 0x7fad80104ae0 2024-03-08T20:59:09.614 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:08.354+0000 7fad85000700 1 -- 172.21.15.107:0/2625518369 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad801a7010 con 0x7fad80104ae0 2024-03-08T20:59:09.614 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:08.355+0000 7fad85000700 1 -- 172.21.15.107:0/2625518369 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad80068aa0 con 0x7fad80104ae0 2024-03-08T20:59:09.614 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:08.357+0000 7fad77e00700 1 -- 172.21.15.107:0/2625518369 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) v1 ==== 137342+0+0 (secure 0 0 0) 0x7fad6c00f450 con 0x7fad80104ae0 2024-03-08T20:59:09.614 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:08.358+0000 7fad77e00700 1 --2- 172.21.15.107:0/2625518369 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fad640a29a0 0x7fad640a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:09.614 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:08.358+0000 7fad77e00700 1 -- 172.21.15.107:0/2625518369 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fad6c0ceb00 con 0x7fad80104ae0 2024-03-08T20:59:09.614 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:08.358+0000 7fad7f400700 1 --2- 172.21.15.107:0/2625518369 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fad640a29a0 0x7fad640a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:59:09.615 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:08.359+0000 7fad7f400700 1 --2- 172.21.15.107:0/2625518369 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fad640a29a0 0x7fad640a4e60 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fad70005950 tx=0x7fad70005eb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:59:09.615 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:08.362+0000 7fad77e00700 1 -- 172.21.15.107:0/2625518369 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7fad6c014030 con 0x7fad80104ae0 2024-03-08T20:59:09.615 INFO:teuthology.orchestra.run.smithi107.stdout:107374182486 2024-03-08T20:59:09.615 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:08.598+0000 7fad85000700 1 -- 172.21.15.107:0/2625518369 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fad8019b7f0 con 0x7fad80104ae0 2024-03-08T20:59:09.616 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:08.599+0000 7fad77e00700 1 -- 172.21.15.107:0/2625518369 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fad6c09ba00 con 0x7fad80104ae0 2024-03-08T20:59:09.616 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:08.602+0000 7fad85000700 1 -- 172.21.15.107:0/2625518369 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fad640a29a0 msgr2=0x7fad640a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:59:09.616 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:08.602+0000 7fad85000700 1 --2- 172.21.15.107:0/2625518369 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fad640a29a0 0x7fad640a4e60 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fad70005950 tx=0x7fad70005eb0 comp rx=0 tx=0).stop 2024-03-08T20:59:09.616 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:08.602+0000 7fad85000700 1 -- 172.21.15.107:0/2625518369 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fad80104ae0 msgr2=0x7fad8019aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:59:09.616 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:08.602+0000 7fad85000700 1 --2- 172.21.15.107:0/2625518369 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fad80104ae0 0x7fad8019aa20 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fad6c00c9e0 tx=0x7fad6c00cda0 comp rx=0 tx=0).stop 2024-03-08T20:59:09.616 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:08.602+0000 7fad85000700 1 -- 172.21.15.107:0/2625518369 shutdown_connections 2024-03-08T20:59:09.616 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:08.602+0000 7fad85000700 1 --2- 172.21.15.107:0/2625518369 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fad80104130 0x7fad8019a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:09.616 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:08.602+0000 7fad85000700 1 --2- 172.21.15.107:0/2625518369 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fad640a29a0 0x7fad640a4e60 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:09.616 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:08.602+0000 7fad85000700 1 --2- 172.21.15.107:0/2625518369 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fad80109210 0x7fad8019ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:09.616 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:08.602+0000 7fad85000700 1 --2- 172.21.15.107:0/2625518369 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fad80104ae0 0x7fad8019aa20 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:09.616 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:08.602+0000 7fad85000700 1 -- 172.21.15.107:0/2625518369 >> 172.21.15.107:0/2625518369 conn(0x7fad800ffa00 msgr2=0x7fad801003f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:59:09.616 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:08.603+0000 7fad85000700 1 -- 172.21.15.107:0/2625518369 shutdown_connections 2024-03-08T20:59:09.616 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:08.603+0000 7fad85000700 1 -- 172.21.15.107:0/2625518369 wait complete. 2024-03-08T20:59:09.765 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:59:09 smithi149 ceph-mon[35042]: pgmap v402: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-03-08T20:59:09.765 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:59:09 smithi149 ceph-mon[35042]: from='client.? 172.21.15.107:0/2625518369' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-03-08T20:59:09.878 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:59:09 smithi204 ceph-mon[34935]: pgmap v402: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-03-08T20:59:09.878 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:59:09 smithi204 ceph-mon[34935]: from='client.? 172.21.15.107:0/2625518369' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-03-08T20:59:09.955 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:09.690+0000 7f14acc00700 1 -- 172.21.15.107:0/4023669947 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f14a8104230 msgr2=0x7f14a8104610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:59:09.955 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:09.690+0000 7f14acc00700 1 --2- 172.21.15.107:0/4023669947 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f14a8104230 0x7f14a8104610 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f14980099c0 tx=0x7f1498009cd0 comp rx=0 tx=0).stop 2024-03-08T20:59:09.955 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:09.690+0000 7f14acc00700 1 -- 172.21.15.107:0/4023669947 shutdown_connections 2024-03-08T20:59:09.955 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:09.690+0000 7f14acc00700 1 --2- 172.21.15.107:0/4023669947 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f14a8109310 0x7f14a8110ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:09.955 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:09.690+0000 7f14acc00700 1 --2- 172.21.15.107:0/4023669947 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f14a8104230 0x7f14a8104610 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:09.955 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:09.690+0000 7f14acc00700 1 --2- 172.21.15.107:0/4023669947 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f14a8104be0 0x7f14a8108c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:09.955 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:09.690+0000 7f14acc00700 1 -- 172.21.15.107:0/4023669947 >> 172.21.15.107:0/4023669947 conn(0x7f14a80ffac0 msgr2=0x7f14a8101ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:59:09.955 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:09.690+0000 7f14acc00700 1 -- 172.21.15.107:0/4023669947 shutdown_connections 2024-03-08T20:59:09.955 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:09.690+0000 7f14acc00700 1 -- 172.21.15.107:0/4023669947 wait complete. 2024-03-08T20:59:09.955 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:09.691+0000 7f14acc00700 1 Processor -- start 2024-03-08T20:59:09.955 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:09.691+0000 7f14acc00700 1 -- start start 2024-03-08T20:59:09.955 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:09.692+0000 7f14acc00700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f14a8104230 0x7f14a819a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:09.956 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:09.692+0000 7f14acc00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f14a8104be0 0x7f14a819abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:09.956 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:09.692+0000 7f14acc00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f14a8109310 0x7f14a819ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:09.956 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:09.692+0000 7f14acc00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14a81141c0 con 0x7f14a8109310 2024-03-08T20:59:09.956 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:09.692+0000 7f14acc00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14a8114040 con 0x7f14a8104be0 2024-03-08T20:59:09.956 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:09.692+0000 7f14acc00700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14a8114340 con 0x7f14a8104230 2024-03-08T20:59:09.956 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:09.692+0000 7f14a6800700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f14a8109310 0x7f14a819ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:59:09.956 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:09.692+0000 7f14a6800700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f14a8109310 0x7f14a819ef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:57212/0 (socket says 172.21.15.107:57212) 2024-03-08T20:59:09.956 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:09.692+0000 7f14a6800700 1 -- 172.21.15.107:0/902308239 learned_addr learned my addr 172.21.15.107:0/902308239 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T20:59:09.956 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:09.692+0000 7f14a5e00700 1 --2- 172.21.15.107:0/902308239 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f14a8104230 0x7f14a819a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:59:09.956 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:09.692+0000 7f14a5400700 1 --2- 172.21.15.107:0/902308239 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f14a8104be0 0x7f14a819abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:59:09.956 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:09.693+0000 7f14a6800700 1 -- 172.21.15.107:0/902308239 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f14a8104230 msgr2=0x7f14a819a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:59:09.956 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:09.693+0000 7f14a6800700 1 --2- 172.21.15.107:0/902308239 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f14a8104230 0x7f14a819a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:09.956 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:09.693+0000 7f14a6800700 1 -- 172.21.15.107:0/902308239 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f14a8104be0 msgr2=0x7f14a819abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:59:09.957 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:09.693+0000 7f14a6800700 1 --2- 172.21.15.107:0/902308239 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f14a8104be0 0x7f14a819abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:09.957 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:09.693+0000 7f14a6800700 1 -- 172.21.15.107:0/902308239 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14a819f6a0 con 0x7f14a8109310 2024-03-08T20:59:09.957 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:09.693+0000 7f14a6800700 1 --2- 172.21.15.107:0/902308239 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f14a8109310 0x7f14a819ef60 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f14a000ada0 tx=0x7f14a000c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:59:09.957 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:09.693+0000 7f149ea00700 1 -- 172.21.15.107:0/902308239 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14a0017020 con 0x7f14a8109310 2024-03-08T20:59:09.957 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:09.693+0000 7f14acc00700 1 -- 172.21.15.107:0/902308239 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14a819f990 con 0x7f14a8109310 2024-03-08T20:59:09.957 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:09.693+0000 7f149ea00700 1 -- 172.21.15.107:0/902308239 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f14a000ce30 con 0x7f14a8109310 2024-03-08T20:59:09.957 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:09.693+0000 7f149ea00700 1 -- 172.21.15.107:0/902308239 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14a0005460 con 0x7f14a8109310 2024-03-08T20:59:09.957 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:09.693+0000 7f14acc00700 1 -- 172.21.15.107:0/902308239 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14a81a7210 con 0x7f14a8109310 2024-03-08T20:59:09.957 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:09.694+0000 7f14acc00700 1 -- 172.21.15.107:0/902308239 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14a8068aa0 con 0x7f14a8109310 2024-03-08T20:59:09.957 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:09.696+0000 7f149ea00700 1 -- 172.21.15.107:0/902308239 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) v1 ==== 137342+0+0 (secure 0 0 0) 0x7f14a000c980 con 0x7f14a8109310 2024-03-08T20:59:09.957 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:09.698+0000 7f149ea00700 1 --2- 172.21.15.107:0/902308239 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f148c0a2920 0x7f148c0a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:09.957 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:09.699+0000 7f149ea00700 1 -- 172.21.15.107:0/902308239 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f14a00ce440 con 0x7f14a8109310 2024-03-08T20:59:09.957 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:09.699+0000 7f14a5e00700 1 --2- 172.21.15.107:0/902308239 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f148c0a2920 0x7f148c0a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:59:09.957 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:09.699+0000 7f14a5e00700 1 --2- 172.21.15.107:0/902308239 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f148c0a2920 0x7f148c0a4de0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f1498009fd0 tx=0x7f1498005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:59:09.958 INFO:teuthology.orchestra.run.smithi107.stdout:347892351007 2024-03-08T20:59:09.958 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:09.703+0000 7f149ea00700 1 -- 172.21.15.107:0/902308239 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f14a0021020 con 0x7f14a8109310 2024-03-08T20:59:09.958 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:09.937+0000 7f14acc00700 1 -- 172.21.15.107:0/902308239 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) v1 -- 0x7f14a802d830 con 0x7f14a8109310 2024-03-08T20:59:09.958 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:09.938+0000 7f149ea00700 1 -- 172.21.15.107:0/902308239 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7f14a009b3d0 con 0x7f14a8109310 2024-03-08T20:59:09.958 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:09.941+0000 7f14acc00700 1 -- 172.21.15.107:0/902308239 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f148c0a2920 msgr2=0x7f148c0a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:59:09.958 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:09.941+0000 7f14acc00700 1 --2- 172.21.15.107:0/902308239 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f148c0a2920 0x7f148c0a4de0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f1498009fd0 tx=0x7f1498005fd0 comp rx=0 tx=0).stop 2024-03-08T20:59:09.958 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:09.941+0000 7f14acc00700 1 -- 172.21.15.107:0/902308239 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f14a8109310 msgr2=0x7f14a819ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:59:09.959 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:09.941+0000 7f14acc00700 1 --2- 172.21.15.107:0/902308239 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f14a8109310 0x7f14a819ef60 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f14a000ada0 tx=0x7f14a000c330 comp rx=0 tx=0).stop 2024-03-08T20:59:09.959 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:09.941+0000 7f14acc00700 1 -- 172.21.15.107:0/902308239 shutdown_connections 2024-03-08T20:59:09.959 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:09.941+0000 7f14acc00700 1 --2- 172.21.15.107:0/902308239 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f14a8104230 0x7f14a819a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:09.959 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:09.941+0000 7f14acc00700 1 --2- 172.21.15.107:0/902308239 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f148c0a2920 0x7f148c0a4de0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:09.959 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:09.941+0000 7f14acc00700 1 --2- 172.21.15.107:0/902308239 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f14a8109310 0x7f14a819ef60 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:09.959 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:09.941+0000 7f14acc00700 1 --2- 172.21.15.107:0/902308239 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f14a8104be0 0x7f14a819abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:09.959 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:09.941+0000 7f14acc00700 1 -- 172.21.15.107:0/902308239 >> 172.21.15.107:0/902308239 conn(0x7f14a80ffac0 msgr2=0x7f14a81017f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:59:09.959 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:09.941+0000 7f14acc00700 1 -- 172.21.15.107:0/902308239 shutdown_connections 2024-03-08T20:59:09.959 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:09.941+0000 7f14acc00700 1 -- 172.21.15.107:0/902308239 wait complete. 2024-03-08T20:59:10.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:10.448+0000 7fb2f3200700 1 -- 172.21.15.107:0/4179950549 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb2f41028e0 msgr2=0x7fb2f4106930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:59:10.433 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:10.448+0000 7fb2f3200700 1 --2- 172.21.15.107:0/4179950549 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb2f41028e0 0x7fb2f4106930 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fb2e00099c0 tx=0x7fb2e0009cd0 comp rx=0 tx=0).stop 2024-03-08T20:59:10.433 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:10.448+0000 7fb2f3200700 1 -- 172.21.15.107:0/4179950549 shutdown_connections 2024-03-08T20:59:10.433 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:10.448+0000 7fb2f3200700 1 --2- 172.21.15.107:0/4179950549 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb2f4101f30 0x7fb2f4102310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:10.433 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:10.448+0000 7fb2f3200700 1 --2- 172.21.15.107:0/4179950549 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb2f4107010 0x7fb2f410ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:10.433 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:10.448+0000 7fb2f3200700 1 --2- 172.21.15.107:0/4179950549 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb2f41028e0 0x7fb2f4106930 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:10.433 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:10.448+0000 7fb2f3200700 1 -- 172.21.15.107:0/4179950549 >> 172.21.15.107:0/4179950549 conn(0x7fb2f40fd7e0 msgr2=0x7fb2f40ffc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:59:10.433 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:10.449+0000 7fb2f3200700 1 -- 172.21.15.107:0/4179950549 shutdown_connections 2024-03-08T20:59:10.433 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:10.449+0000 7fb2f3200700 1 -- 172.21.15.107:0/4179950549 wait complete. 2024-03-08T20:59:10.434 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:10.450+0000 7fb2f3200700 1 Processor -- start 2024-03-08T20:59:10.434 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:10.450+0000 7fb2f3200700 1 -- start start 2024-03-08T20:59:10.434 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:10.450+0000 7fb2f3200700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb2f4101f30 0x7fb2f4198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:10.434 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:10.450+0000 7fb2f3200700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb2f41028e0 0x7fb2f41988b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:10.434 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:10.450+0000 7fb2f3200700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb2f4107010 0x7fb2f419cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:10.435 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:10.450+0000 7fb2f3200700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2f4111e80 con 0x7fb2f41028e0 2024-03-08T20:59:10.435 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:10.450+0000 7fb2f3200700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2f4111d00 con 0x7fb2f4107010 2024-03-08T20:59:10.435 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:10.450+0000 7fb2f3200700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2f4112000 con 0x7fb2f4101f30 2024-03-08T20:59:10.435 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:10.451+0000 7fb2f1400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb2f41028e0 0x7fb2f41988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:59:10.435 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:10.451+0000 7fb2f1400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb2f41028e0 0x7fb2f41988b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:57220/0 (socket says 172.21.15.107:57220) 2024-03-08T20:59:10.435 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:10.451+0000 7fb2f1400700 1 -- 172.21.15.107:0/2297023795 learned_addr learned my addr 172.21.15.107:0/2297023795 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T20:59:10.436 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:10.451+0000 7fb2f2800700 1 --2- 172.21.15.107:0/2297023795 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb2f4107010 0x7fb2f419cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:59:10.436 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:10.451+0000 7fb2f1e00700 1 --2- 172.21.15.107:0/2297023795 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb2f4101f30 0x7fb2f4198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:59:10.436 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:10.451+0000 7fb2f1400700 1 -- 172.21.15.107:0/2297023795 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb2f4101f30 msgr2=0x7fb2f4198370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:59:10.436 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:10.451+0000 7fb2f1400700 1 --2- 172.21.15.107:0/2297023795 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb2f4101f30 0x7fb2f4198370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:10.436 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:10.451+0000 7fb2f1400700 1 -- 172.21.15.107:0/2297023795 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb2f4107010 msgr2=0x7fb2f419cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:59:10.436 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:10.451+0000 7fb2f1400700 1 --2- 172.21.15.107:0/2297023795 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb2f4107010 0x7fb2f419cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:10.436 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:10.451+0000 7fb2f1400700 1 -- 172.21.15.107:0/2297023795 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2f419d380 con 0x7fb2f41028e0 2024-03-08T20:59:10.436 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:10.451+0000 7fb2f2800700 1 --2- 172.21.15.107:0/2297023795 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb2f4107010 0x7fb2f419cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T20:59:10.436 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:10.452+0000 7fb2f1400700 1 --2- 172.21.15.107:0/2297023795 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb2f41028e0 0x7fb2f41988b0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fb2e00059d0 tx=0x7fb2e0005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:59:10.436 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:10.452+0000 7fb2eaa00700 1 -- 172.21.15.107:0/2297023795 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2e0027070 con 0x7fb2f41028e0 2024-03-08T20:59:10.437 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:10.452+0000 7fb2eaa00700 1 -- 172.21.15.107:0/2297023795 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fb2e0003fd0 con 0x7fb2f41028e0 2024-03-08T20:59:10.437 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:10.452+0000 7fb2f3200700 1 -- 172.21.15.107:0/2297023795 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2f419d610 con 0x7fb2f41028e0 2024-03-08T20:59:10.437 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:10.452+0000 7fb2eaa00700 1 -- 172.21.15.107:0/2297023795 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2e0022bd0 con 0x7fb2f41028e0 2024-03-08T20:59:10.437 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:10.452+0000 7fb2f3200700 1 -- 172.21.15.107:0/2297023795 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2f41a4ef0 con 0x7fb2f41028e0 2024-03-08T20:59:10.437 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:10.453+0000 7fb2f3200700 1 -- 172.21.15.107:0/2297023795 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2f4103410 con 0x7fb2f41028e0 2024-03-08T20:59:10.442 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:10.458+0000 7fb2eaa00700 1 -- 172.21.15.107:0/2297023795 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) v1 ==== 137342+0+0 (secure 0 0 0) 0x7fb2e002a070 con 0x7fb2f41028e0 2024-03-08T20:59:10.443 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:10.459+0000 7fb2eaa00700 1 --2- 172.21.15.107:0/2297023795 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fb2d80a2920 0x7fb2d80a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:10.443 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:10.460+0000 7fb2eaa00700 1 -- 172.21.15.107:0/2297023795 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fb2e00d0200 con 0x7fb2f41028e0 2024-03-08T20:59:10.443 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:10.460+0000 7fb2f1e00700 1 --2- 172.21.15.107:0/2297023795 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fb2d80a2920 0x7fb2d80a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:59:10.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:10.460+0000 7fb2f1e00700 1 --2- 172.21.15.107:0/2297023795 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fb2d80a2920 0x7fb2d80a4de0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fb2e4005fd0 tx=0x7fb2e400a560 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:59:10.445 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:10.461+0000 7fb2eaa00700 1 -- 172.21.15.107:0/2297023795 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7fb2e0031020 con 0x7fb2f41028e0 2024-03-08T20:59:10.521 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:59:10 smithi107 ceph-mon[31381]: from='client.? 172.21.15.107:0/902308239' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-03-08T20:59:10.671 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:10.687+0000 7fb2f3200700 1 -- 172.21.15.107:0/2297023795 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) v1 -- 0x7fb2f41996d0 con 0x7fb2f41028e0 2024-03-08T20:59:10.671 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:10.687+0000 7fb2eaa00700 1 -- 172.21.15.107:0/2297023795 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb2e0013070 con 0x7fb2f41028e0 2024-03-08T20:59:10.671 INFO:teuthology.orchestra.run.smithi107.stdout:292057776174 2024-03-08T20:59:10.674 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:10.690+0000 7fb2f3200700 1 -- 172.21.15.107:0/2297023795 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fb2d80a2920 msgr2=0x7fb2d80a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:59:10.674 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:10.690+0000 7fb2f3200700 1 --2- 172.21.15.107:0/2297023795 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fb2d80a2920 0x7fb2d80a4de0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fb2e4005fd0 tx=0x7fb2e400a560 comp rx=0 tx=0).stop 2024-03-08T20:59:10.674 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:10.690+0000 7fb2f3200700 1 -- 172.21.15.107:0/2297023795 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb2f41028e0 msgr2=0x7fb2f41988b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:59:10.675 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:10.690+0000 7fb2f3200700 1 --2- 172.21.15.107:0/2297023795 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb2f41028e0 0x7fb2f41988b0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fb2e00059d0 tx=0x7fb2e0005e80 comp rx=0 tx=0).stop 2024-03-08T20:59:10.675 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:10.690+0000 7fb2f3200700 1 -- 172.21.15.107:0/2297023795 shutdown_connections 2024-03-08T20:59:10.675 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:10.691+0000 7fb2f3200700 1 --2- 172.21.15.107:0/2297023795 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb2f4101f30 0x7fb2f4198370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:10.675 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:10.691+0000 7fb2f3200700 1 --2- 172.21.15.107:0/2297023795 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fb2d80a2920 0x7fb2d80a4de0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:10.675 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:10.691+0000 7fb2f3200700 1 --2- 172.21.15.107:0/2297023795 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb2f4107010 0x7fb2f419cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:10.675 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:10.691+0000 7fb2f3200700 1 --2- 172.21.15.107:0/2297023795 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb2f41028e0 0x7fb2f41988b0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:10.675 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:10.691+0000 7fb2f3200700 1 -- 172.21.15.107:0/2297023795 >> 172.21.15.107:0/2297023795 conn(0x7fb2f40fd7e0 msgr2=0x7fb2f40feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:59:10.675 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:10.691+0000 7fb2f3200700 1 -- 172.21.15.107:0/2297023795 shutdown_connections 2024-03-08T20:59:10.675 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:10.691+0000 7fb2f3200700 1 -- 172.21.15.107:0/2297023795 wait complete. 2024-03-08T20:59:10.751 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:10.696+0000 7f5aafa00700 1 -- 172.21.15.107:0/3862426470 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5aa8107e30 msgr2=0x7f5aa8108210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:59:10.751 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:10.696+0000 7f5aafa00700 1 --2- 172.21.15.107:0/3862426470 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5aa8107e30 0x7f5aa8108210 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f5a9c0099c0 tx=0x7f5a9c009cd0 comp rx=0 tx=0).stop 2024-03-08T20:59:10.751 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:10.696+0000 7f5aafa00700 1 -- 172.21.15.107:0/3862426470 shutdown_connections 2024-03-08T20:59:10.752 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:10.696+0000 7f5aafa00700 1 --2- 172.21.15.107:0/3862426470 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5aa806a430 0x7f5aa810ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:10.752 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:10.696+0000 7f5aafa00700 1 --2- 172.21.15.107:0/3862426470 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5aa8107e30 0x7f5aa8108210 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:10.752 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:10.696+0000 7f5aafa00700 1 --2- 172.21.15.107:0/3862426470 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5aa8069a90 0x7f5aa8069ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:10.752 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:10.696+0000 7f5aafa00700 1 -- 172.21.15.107:0/3862426470 >> 172.21.15.107:0/3862426470 conn(0x7f5aa8077c10 msgr2=0x7f5aa8078020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:59:10.752 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:10.696+0000 7f5aafa00700 1 -- 172.21.15.107:0/3862426470 shutdown_connections 2024-03-08T20:59:10.752 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:10.696+0000 7f5aafa00700 1 -- 172.21.15.107:0/3862426470 wait complete. 2024-03-08T20:59:10.752 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:10.697+0000 7f5aafa00700 1 Processor -- start 2024-03-08T20:59:10.752 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:10.697+0000 7f5aafa00700 1 -- start start 2024-03-08T20:59:10.752 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:10.698+0000 7f5aafa00700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5aa8069a90 0x7f5aa81a05b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:10.752 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:10.698+0000 7f5aafa00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5aa806a430 0x7f5aa81a0af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:10.752 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:10.698+0000 7f5aafa00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5aa8107e30 0x7f5aa819a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:10.752 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:10.698+0000 7f5aafa00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5aa8111ff0 con 0x7f5aa8107e30 2024-03-08T20:59:10.752 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:10.698+0000 7f5aafa00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5aa8111e70 con 0x7f5aa806a430 2024-03-08T20:59:10.752 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:10.698+0000 7f5aafa00700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5aa8112170 con 0x7f5aa8069a90 2024-03-08T20:59:10.753 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:10.698+0000 7f5aada00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5aa8107e30 0x7f5aa819a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:59:10.753 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:10.698+0000 7f5aada00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5aa8107e30 0x7f5aa819a6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:57234/0 (socket says 172.21.15.107:57234) 2024-03-08T20:59:10.753 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:10.698+0000 7f5aada00700 1 -- 172.21.15.107:0/15711633 learned_addr learned my addr 172.21.15.107:0/15711633 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T20:59:10.753 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:10.698+0000 7f5aad000700 1 --2- 172.21.15.107:0/15711633 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5aa8069a90 0x7f5aa81a05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:59:10.753 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:10.699+0000 7f5aada00700 1 -- 172.21.15.107:0/15711633 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5aa8069a90 msgr2=0x7f5aa81a05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:59:10.753 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:10.699+0000 7f5aada00700 1 --2- 172.21.15.107:0/15711633 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5aa8069a90 0x7f5aa81a05b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:10.753 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:10.699+0000 7f5aada00700 1 -- 172.21.15.107:0/15711633 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5aa806a430 msgr2=0x7f5aa81a0af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:59:10.753 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:10.699+0000 7f5aada00700 1 --2- 172.21.15.107:0/15711633 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5aa806a430 0x7f5aa81a0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:10.753 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:10.699+0000 7f5aada00700 1 -- 172.21.15.107:0/15711633 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5aa819afb0 con 0x7f5aa8107e30 2024-03-08T20:59:10.753 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:10.699+0000 7f5aada00700 1 --2- 172.21.15.107:0/15711633 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5aa8107e30 0x7f5aa819a6d0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f5aa000b970 tx=0x7f5aa000bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:59:10.753 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:10.699+0000 7f5aa5600700 1 -- 172.21.15.107:0/15711633 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5aa0019070 con 0x7f5aa8107e30 2024-03-08T20:59:10.753 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:10.699+0000 7f5aa5600700 1 -- 172.21.15.107:0/15711633 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f5aa0014440 con 0x7f5aa8107e30 2024-03-08T20:59:10.753 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:10.699+0000 7f5aa5600700 1 -- 172.21.15.107:0/15711633 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5aa000ce60 con 0x7f5aa8107e30 2024-03-08T20:59:10.753 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:10.699+0000 7f5aafa00700 1 -- 172.21.15.107:0/15711633 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5aa819b2a0 con 0x7f5aa8107e30 2024-03-08T20:59:10.754 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:10.699+0000 7f5aafa00700 1 -- 172.21.15.107:0/15711633 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5aa81a7530 con 0x7f5aa8107e30 2024-03-08T20:59:10.754 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:10.701+0000 7f5aafa00700 1 -- 172.21.15.107:0/15711633 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5aa81034e0 con 0x7f5aa8107e30 2024-03-08T20:59:10.754 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:10.703+0000 7f5aa5600700 1 -- 172.21.15.107:0/15711633 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) v1 ==== 137342+0+0 (secure 0 0 0) 0x7f5aa001c070 con 0x7f5aa8107e30 2024-03-08T20:59:10.754 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:10.705+0000 7f5aa5600700 1 --2- 172.21.15.107:0/15711633 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f5a900a29a0 0x7f5a900a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:10.754 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:10.705+0000 7f5aa5600700 1 -- 172.21.15.107:0/15711633 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f5aa00cdef0 con 0x7f5aa8107e30 2024-03-08T20:59:10.754 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:10.705+0000 7f5aad000700 1 --2- 172.21.15.107:0/15711633 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f5a900a29a0 0x7f5a900a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:59:10.755 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:10.706+0000 7f5aad000700 1 --2- 172.21.15.107:0/15711633 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f5a900a29a0 0x7f5a900a4e60 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f5a9c009fd0 tx=0x7f5a9c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:59:10.755 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:10.708+0000 7f5aa5600700 1 -- 172.21.15.107:0/15711633 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f5aa009af00 con 0x7f5aa8107e30 2024-03-08T20:59:10.765 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:59:10 smithi149 ceph-mon[35042]: from='client.? 172.21.15.107:0/902308239' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-03-08T20:59:10.878 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:59:10 smithi204 ceph-mon[34935]: from='client.? 172.21.15.107:0/902308239' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-03-08T20:59:10.939 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:10.955+0000 7f5aafa00700 1 -- 172.21.15.107:0/15711633 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f5aa819bbe0 con 0x7f5aa8107e30 2024-03-08T20:59:10.939 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:10.956+0000 7f5aa5600700 1 -- 172.21.15.107:0/15711633 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f5aa819bbe0 con 0x7f5aa8107e30 2024-03-08T20:59:10.939 INFO:teuthology.orchestra.run.smithi107.stdout:55834574948 2024-03-08T20:59:10.942 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:10.958+0000 7f5aafa00700 1 -- 172.21.15.107:0/15711633 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f5a900a29a0 msgr2=0x7f5a900a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:59:10.942 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:10.958+0000 7f5aafa00700 1 --2- 172.21.15.107:0/15711633 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f5a900a29a0 0x7f5a900a4e60 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f5a9c009fd0 tx=0x7f5a9c005fb0 comp rx=0 tx=0).stop 2024-03-08T20:59:10.942 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:10.958+0000 7f5aafa00700 1 -- 172.21.15.107:0/15711633 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5aa8107e30 msgr2=0x7f5aa819a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:59:10.943 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:10.958+0000 7f5aafa00700 1 --2- 172.21.15.107:0/15711633 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5aa8107e30 0x7f5aa819a6d0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f5aa000b970 tx=0x7f5aa000bd30 comp rx=0 tx=0).stop 2024-03-08T20:59:10.943 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:10.959+0000 7f5aafa00700 1 -- 172.21.15.107:0/15711633 shutdown_connections 2024-03-08T20:59:10.943 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:10.959+0000 7f5aafa00700 1 --2- 172.21.15.107:0/15711633 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5aa8069a90 0x7f5aa81a05b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:10.943 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:10.959+0000 7f5aafa00700 1 --2- 172.21.15.107:0/15711633 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f5a900a29a0 0x7f5a900a4e60 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:10.943 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:10.959+0000 7f5aafa00700 1 --2- 172.21.15.107:0/15711633 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5aa8107e30 0x7f5aa819a6d0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:10.943 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:10.959+0000 7f5aafa00700 1 --2- 172.21.15.107:0/15711633 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5aa806a430 0x7f5aa81a0af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:10.943 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:10.959+0000 7f5aafa00700 1 -- 172.21.15.107:0/15711633 >> 172.21.15.107:0/15711633 conn(0x7f5aa8077c10 msgr2=0x7f5aa80ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:59:10.943 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:10.959+0000 7f5aafa00700 1 -- 172.21.15.107:0/15711633 shutdown_connections 2024-03-08T20:59:10.943 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:10.959+0000 7f5aafa00700 1 -- 172.21.15.107:0/15711633 wait complete. 2024-03-08T20:59:11.372 INFO:tasks.cephadm.ceph_manager.ceph:need seq 201863462971 got 201863462976 for osd.6 2024-03-08T20:59:11.372 DEBUG:teuthology.parallel:result is None 2024-03-08T20:59:11.515 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:59:11 smithi149 ceph-mon[35042]: pgmap v403: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-08T20:59:11.515 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:59:11 smithi149 ceph-mon[35042]: from='client.? 172.21.15.107:0/2297023795' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-03-08T20:59:11.515 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:59:11 smithi149 ceph-mon[35042]: from='client.? 172.21.15.107:0/15711633' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-03-08T20:59:11.577 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:59:11 smithi107 ceph-mon[31381]: pgmap v403: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-08T20:59:11.577 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:59:11 smithi107 ceph-mon[31381]: from='client.? 172.21.15.107:0/2297023795' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-03-08T20:59:11.577 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:59:11 smithi107 ceph-mon[31381]: from='client.? 172.21.15.107:0/15711633' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-03-08T20:59:11.878 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:59:11 smithi204 ceph-mon[34935]: pgmap v403: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-08T20:59:11.878 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:59:11 smithi204 ceph-mon[34935]: from='client.? 172.21.15.107:0/2297023795' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-03-08T20:59:11.879 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:59:11 smithi204 ceph-mon[34935]: from='client.? 172.21.15.107:0/15711633' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-03-08T20:59:12.231 INFO:tasks.cephadm.ceph_manager.ceph:need seq 223338299444 got 223338299450 for osd.7 2024-03-08T20:59:12.231 DEBUG:teuthology.parallel:result is None 2024-03-08T20:59:12.569 INFO:tasks.cephadm.ceph_manager.ceph:need seq 347892351003 got 347892351007 for osd.11 2024-03-08T20:59:12.569 DEBUG:teuthology.parallel:result is None 2024-03-08T20:59:12.903 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:12.919+0000 7fd9c2800700 1 -- 172.21.15.107:0/1873320344 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd9bc06a3b0 msgr2=0x7fd9bc10e350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:59:12.903 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:12.919+0000 7fd9c2800700 1 --2- 172.21.15.107:0/1873320344 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd9bc06a3b0 0x7fd9bc10e350 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fd9a8009990 tx=0x7fd9a8009ca0 comp rx=0 tx=0).stop 2024-03-08T20:59:12.903 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:12.919+0000 7fd9c2800700 1 -- 172.21.15.107:0/1873320344 shutdown_connections 2024-03-08T20:59:12.903 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:12.919+0000 7fd9c2800700 1 --2- 172.21.15.107:0/1873320344 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd9bc069a90 0x7fd9bc069e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:12.903 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:12.919+0000 7fd9c2800700 1 --2- 172.21.15.107:0/1873320344 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd9bc10e890 0x7fd9bc110c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:12.903 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:12.919+0000 7fd9c2800700 1 --2- 172.21.15.107:0/1873320344 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd9bc06a3b0 0x7fd9bc10e350 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:12.904 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:12.919+0000 7fd9c2800700 1 -- 172.21.15.107:0/1873320344 >> 172.21.15.107:0/1873320344 conn(0x7fd9bc077c10 msgr2=0x7fd9bc078020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:59:12.904 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:12.919+0000 7fd9c2800700 1 -- 172.21.15.107:0/1873320344 shutdown_connections 2024-03-08T20:59:12.904 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:12.919+0000 7fd9c2800700 1 -- 172.21.15.107:0/1873320344 wait complete. 2024-03-08T20:59:12.906 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:12.920+0000 7fd9c2800700 1 Processor -- start 2024-03-08T20:59:12.906 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:12.920+0000 7fd9c2800700 1 -- start start 2024-03-08T20:59:12.906 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:12.921+0000 7fd9c2800700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd9bc069a90 0x7fd9bc19a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:12.906 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:12.921+0000 7fd9c2800700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd9bc06a3b0 0x7fd9bc19ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:12.906 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:12.921+0000 7fd9c2800700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd9bc10e890 0x7fd9bc19efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:12.906 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:12.921+0000 7fd9c2800700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9bc1141c0 con 0x7fd9bc069a90 2024-03-08T20:59:12.906 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:12.921+0000 7fd9c2800700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9bc114040 con 0x7fd9bc06a3b0 2024-03-08T20:59:12.907 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:12.921+0000 7fd9c2800700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9bc114340 con 0x7fd9bc10e890 2024-03-08T20:59:12.907 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:12.921+0000 7fd9bb400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd9bc069a90 0x7fd9bc19a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:59:12.907 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:12.921+0000 7fd9bb400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd9bc069a90 0x7fd9bc19a750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:57252/0 (socket says 172.21.15.107:57252) 2024-03-08T20:59:12.907 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:12.921+0000 7fd9bb400700 1 -- 172.21.15.107:0/1270064435 learned_addr learned my addr 172.21.15.107:0/1270064435 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T20:59:12.907 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:12.921+0000 7fd9baa00700 1 --2- 172.21.15.107:0/1270064435 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd9bc06a3b0 0x7fd9bc19ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:59:12.907 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:12.921+0000 7fd9bbe00700 1 --2- 172.21.15.107:0/1270064435 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd9bc10e890 0x7fd9bc19efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:59:12.907 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:12.921+0000 7fd9bb400700 1 -- 172.21.15.107:0/1270064435 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd9bc10e890 msgr2=0x7fd9bc19efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:59:12.908 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:12.921+0000 7fd9bb400700 1 --2- 172.21.15.107:0/1270064435 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd9bc10e890 0x7fd9bc19efd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:12.908 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:12.921+0000 7fd9bb400700 1 -- 172.21.15.107:0/1270064435 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd9bc06a3b0 msgr2=0x7fd9bc19ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:59:12.908 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:12.921+0000 7fd9bb400700 1 --2- 172.21.15.107:0/1270064435 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd9bc06a3b0 0x7fd9bc19ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:12.908 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:12.921+0000 7fd9bb400700 1 -- 172.21.15.107:0/1270064435 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9bc19f710 con 0x7fd9bc069a90 2024-03-08T20:59:12.908 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:12.922+0000 7fd9bb400700 1 --2- 172.21.15.107:0/1270064435 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd9bc069a90 0x7fd9bc19a750 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fd9ac00d980 tx=0x7fd9ac00dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:59:12.908 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:12.922+0000 7fd9b3e00700 1 -- 172.21.15.107:0/1270064435 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9ac017070 con 0x7fd9bc069a90 2024-03-08T20:59:12.908 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:12.922+0000 7fd9b3e00700 1 -- 172.21.15.107:0/1270064435 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fd9ac004830 con 0x7fd9bc069a90 2024-03-08T20:59:12.908 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:12.922+0000 7fd9c2800700 1 -- 172.21.15.107:0/1270064435 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9bc19fa00 con 0x7fd9bc069a90 2024-03-08T20:59:12.908 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:12.922+0000 7fd9b3e00700 1 -- 172.21.15.107:0/1270064435 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9ac005440 con 0x7fd9bc069a90 2024-03-08T20:59:12.908 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:12.922+0000 7fd9c2800700 1 -- 172.21.15.107:0/1270064435 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9bc1a7280 con 0x7fd9bc069a90 2024-03-08T20:59:12.909 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:12.924+0000 7fd9c2800700 1 -- 172.21.15.107:0/1270064435 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9bc068aa0 con 0x7fd9bc069a90 2024-03-08T20:59:12.909 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:12.925+0000 7fd9b3e00700 1 -- 172.21.15.107:0/1270064435 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) v1 ==== 137342+0+0 (secure 0 0 0) 0x7fd9ac0055e0 con 0x7fd9bc069a90 2024-03-08T20:59:12.910 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:12.927+0000 7fd9b3e00700 1 --2- 172.21.15.107:0/1270064435 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fd9a00a2920 0x7fd9a00a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:12.911 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:12.927+0000 7fd9b3e00700 1 -- 172.21.15.107:0/1270064435 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fd9ac0ceb70 con 0x7fd9bc069a90 2024-03-08T20:59:12.911 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:12.927+0000 7fd9baa00700 1 --2- 172.21.15.107:0/1270064435 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fd9a00a2920 0x7fd9a00a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:59:12.911 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:12.928+0000 7fd9baa00700 1 --2- 172.21.15.107:0/1270064435 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fd9a00a2920 0x7fd9a00a4de0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fd9a800b5c0 tx=0x7fd9a801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:59:12.914 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:12.931+0000 7fd9b3e00700 1 -- 172.21.15.107:0/1270064435 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7fd9ac020020 con 0x7fd9bc069a90 2024-03-08T20:59:13.146 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:13.160+0000 7fd9c2800700 1 -- 172.21.15.107:0/1270064435 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) v1 -- 0x7fd9bc02d830 con 0x7fd9bc069a90 2024-03-08T20:59:13.146 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:13.161+0000 7fd9b3e00700 1 -- 172.21.15.107:0/1270064435 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7fd9ac09ba70 con 0x7fd9bc069a90 2024-03-08T20:59:13.146 INFO:teuthology.orchestra.run.smithi107.stdout:313532612647 2024-03-08T20:59:13.148 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:13.164+0000 7fd9c2800700 1 -- 172.21.15.107:0/1270064435 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fd9a00a2920 msgr2=0x7fd9a00a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:59:13.148 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:13.164+0000 7fd9c2800700 1 --2- 172.21.15.107:0/1270064435 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fd9a00a2920 0x7fd9a00a4de0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fd9a800b5c0 tx=0x7fd9a801a040 comp rx=0 tx=0).stop 2024-03-08T20:59:13.148 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:13.164+0000 7fd9c2800700 1 -- 172.21.15.107:0/1270064435 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd9bc069a90 msgr2=0x7fd9bc19a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:59:13.148 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:13.164+0000 7fd9c2800700 1 --2- 172.21.15.107:0/1270064435 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd9bc069a90 0x7fd9bc19a750 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fd9ac00d980 tx=0x7fd9ac00dd40 comp rx=0 tx=0).stop 2024-03-08T20:59:13.148 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:13.164+0000 7fd9c2800700 1 -- 172.21.15.107:0/1270064435 shutdown_connections 2024-03-08T20:59:13.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:13.164+0000 7fd9c2800700 1 --2- 172.21.15.107:0/1270064435 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd9bc10e890 0x7fd9bc19efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:13.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:13.164+0000 7fd9c2800700 1 --2- 172.21.15.107:0/1270064435 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fd9a00a2920 0x7fd9a00a4de0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:13.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:13.164+0000 7fd9c2800700 1 --2- 172.21.15.107:0/1270064435 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd9bc069a90 0x7fd9bc19a750 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:13.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:13.164+0000 7fd9c2800700 1 --2- 172.21.15.107:0/1270064435 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd9bc06a3b0 0x7fd9bc19ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:13.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:13.164+0000 7fd9c2800700 1 -- 172.21.15.107:0/1270064435 >> 172.21.15.107:0/1270064435 conn(0x7fd9bc077c10 msgr2=0x7fd9bc10fe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:59:13.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:13.165+0000 7fd9c2800700 1 -- 172.21.15.107:0/1270064435 shutdown_connections 2024-03-08T20:59:13.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:13.165+0000 7fd9c2800700 1 -- 172.21.15.107:0/1270064435 wait complete. 2024-03-08T20:59:13.303 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574941 got 55834574948 for osd.1 2024-03-08T20:59:13.303 DEBUG:teuthology.parallel:result is None 2024-03-08T20:59:13.717 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:59:13 smithi107 ceph-mon[31381]: pgmap v404: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-08T20:59:13.717 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:59:13 smithi107 ceph-mon[31381]: from='client.? 172.21.15.107:0/1270064435' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-03-08T20:59:13.765 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:59:13 smithi149 ceph-mon[35042]: pgmap v404: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-08T20:59:13.765 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:59:13 smithi149 ceph-mon[35042]: from='client.? 172.21.15.107:0/1270064435' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-03-08T20:59:13.878 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:59:13 smithi204 ceph-mon[34935]: pgmap v404: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-08T20:59:13.878 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:59:13 smithi204 ceph-mon[34935]: from='client.? 172.21.15.107:0/1270064435' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-03-08T20:59:14.253 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182482 got 107374182486 for osd.3 2024-03-08T20:59:14.253 DEBUG:teuthology.parallel:result is None 2024-03-08T20:59:14.775 INFO:tasks.cephadm.ceph_manager.ceph:need seq 257698037806 got 257698037812 for osd.8 2024-03-08T20:59:14.775 DEBUG:teuthology.parallel:result is None 2024-03-08T20:59:15.431 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/mon.a/config 2024-03-08T20:59:15.460 INFO:tasks.cephadm.ceph_manager.ceph:need seq 167503724613 got 167503724618 for osd.5 2024-03-08T20:59:15.460 DEBUG:teuthology.parallel:result is None 2024-03-08T20:59:15.663 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:59:15 smithi107 ceph-mon[31381]: pgmap v405: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-03-08T20:59:15.765 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:59:15 smithi149 ceph-mon[35042]: pgmap v405: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-03-08T20:59:15.878 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:59:15 smithi204 ceph-mon[34935]: pgmap v405: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-03-08T20:59:15.974 INFO:tasks.cephadm.ceph_manager.ceph:need seq 292057776168 got 292057776174 for osd.9 2024-03-08T20:59:15.974 DEBUG:teuthology.parallel:result is None 2024-03-08T20:59:16.261 INFO:tasks.cephadm.ceph_manager.ceph:need seq 313532612641 got 313532612647 for osd.10 2024-03-08T20:59:16.261 DEBUG:teuthology.parallel:result is None 2024-03-08T20:59:17.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:59:17 smithi107 ceph-mon[31381]: pgmap v406: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-08T20:59:17.765 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:59:17 smithi149 ceph-mon[35042]: pgmap v406: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-08T20:59:17.878 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:59:17 smithi204 ceph-mon[34935]: pgmap v406: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-08T20:59:19.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:19.022+0000 7fe22e600700 1 -- 172.21.15.107:0/3460556064 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe228107100 msgr2=0x7fe22810ecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:59:19.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:19.022+0000 7fe22e600700 1 --2- 172.21.15.107:0/3460556064 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe228107100 0x7fe22810ecd0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fe22000b0d0 tx=0x7fe22000b3e0 comp rx=0 tx=0).stop 2024-03-08T20:59:19.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:19.022+0000 7fe22e600700 1 -- 172.21.15.107:0/3460556064 shutdown_connections 2024-03-08T20:59:19.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:19.022+0000 7fe22e600700 1 --2- 172.21.15.107:0/3460556064 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe228102020 0x7fe228102400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:19.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:19.022+0000 7fe22e600700 1 --2- 172.21.15.107:0/3460556064 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe228107100 0x7fe22810ecd0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:19.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:19.022+0000 7fe22e600700 1 --2- 172.21.15.107:0/3460556064 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe2281029d0 0x7fe228106a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:19.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:19.022+0000 7fe22e600700 1 -- 172.21.15.107:0/3460556064 >> 172.21.15.107:0/3460556064 conn(0x7fe2280fd890 msgr2=0x7fe2280ffcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:59:19.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:19.023+0000 7fe22e600700 1 -- 172.21.15.107:0/3460556064 shutdown_connections 2024-03-08T20:59:19.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:19.023+0000 7fe22e600700 1 -- 172.21.15.107:0/3460556064 wait complete. 2024-03-08T20:59:19.009 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:19.024+0000 7fe22e600700 1 Processor -- start 2024-03-08T20:59:19.009 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:19.024+0000 7fe22e600700 1 -- start start 2024-03-08T20:59:19.009 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:19.024+0000 7fe22e600700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe228102020 0x7fe2281984e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:19.009 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:19.024+0000 7fe22e600700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe2281029d0 0x7fe228198a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:19.009 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:19.024+0000 7fe22e600700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe228107100 0x7fe22819cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:19.009 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:19.024+0000 7fe22e600700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe228112010 con 0x7fe2281029d0 2024-03-08T20:59:19.009 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:19.024+0000 7fe22e600700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe228111e90 con 0x7fe228102020 2024-03-08T20:59:19.009 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:19.024+0000 7fe22e600700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe228112190 con 0x7fe228107100 2024-03-08T20:59:19.010 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:19.025+0000 7fe226a00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe2281029d0 0x7fe228198a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:59:19.010 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:19.025+0000 7fe226a00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe2281029d0 0x7fe228198a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:51588/0 (socket says 172.21.15.107:51588) 2024-03-08T20:59:19.010 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:19.025+0000 7fe226a00700 1 -- 172.21.15.107:0/1294791047 learned_addr learned my addr 172.21.15.107:0/1294791047 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T20:59:19.010 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:19.025+0000 7fe227400700 1 --2- 172.21.15.107:0/1294791047 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe228102020 0x7fe2281984e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:59:19.010 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:19.025+0000 7fe227e00700 1 --2- 172.21.15.107:0/1294791047 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe228107100 0x7fe22819cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:59:19.010 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:19.025+0000 7fe226a00700 1 -- 172.21.15.107:0/1294791047 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe228107100 msgr2=0x7fe22819cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:59:19.010 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:19.025+0000 7fe226a00700 1 --2- 172.21.15.107:0/1294791047 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe228107100 0x7fe22819cdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:19.010 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:19.025+0000 7fe226a00700 1 -- 172.21.15.107:0/1294791047 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe228102020 msgr2=0x7fe2281984e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:59:19.010 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:19.025+0000 7fe226a00700 1 --2- 172.21.15.107:0/1294791047 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe228102020 0x7fe2281984e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:19.010 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:19.025+0000 7fe226a00700 1 -- 172.21.15.107:0/1294791047 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe22819d4f0 con 0x7fe2281029d0 2024-03-08T20:59:19.010 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:19.025+0000 7fe227400700 1 --2- 172.21.15.107:0/1294791047 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe228102020 0x7fe2281984e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T20:59:19.010 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:19.026+0000 7fe226a00700 1 --2- 172.21.15.107:0/1294791047 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe2281029d0 0x7fe228198a20 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fe21400d980 tx=0x7fe21400dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:59:19.011 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:19.026+0000 7fe21fe00700 1 -- 172.21.15.107:0/1294791047 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe214017070 con 0x7fe2281029d0 2024-03-08T20:59:19.011 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:19.026+0000 7fe21fe00700 1 -- 172.21.15.107:0/1294791047 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fe214010460 con 0x7fe2281029d0 2024-03-08T20:59:19.011 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:19.026+0000 7fe22e600700 1 -- 172.21.15.107:0/1294791047 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe22819d7e0 con 0x7fe2281029d0 2024-03-08T20:59:19.011 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:19.026+0000 7fe21fe00700 1 -- 172.21.15.107:0/1294791047 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe21400de60 con 0x7fe2281029d0 2024-03-08T20:59:19.011 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:19.026+0000 7fe22e600700 1 -- 172.21.15.107:0/1294791047 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2281a5060 con 0x7fe2281029d0 2024-03-08T20:59:19.011 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:19.027+0000 7fe22e600700 1 -- 172.21.15.107:0/1294791047 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe228068aa0 con 0x7fe2281029d0 2024-03-08T20:59:19.012 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:19.029+0000 7fe21fe00700 1 -- 172.21.15.107:0/1294791047 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) v1 ==== 137342+0+0 (secure 0 0 0) 0x7fe214010610 con 0x7fe2281029d0 2024-03-08T20:59:19.014 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:19.031+0000 7fe21fe00700 1 --2- 172.21.15.107:0/1294791047 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fe20c0a2920 0x7fe20c0a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:19.015 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:19.031+0000 7fe21fe00700 1 -- 172.21.15.107:0/1294791047 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fe2140ce790 con 0x7fe2281029d0 2024-03-08T20:59:19.015 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:19.031+0000 7fe227400700 1 --2- 172.21.15.107:0/1294791047 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fe20c0a2920 0x7fe20c0a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:59:19.015 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:19.032+0000 7fe227400700 1 --2- 172.21.15.107:0/1294791047 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fe20c0a2920 0x7fe20c0a4de0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fe218005950 tx=0x7fe218005f30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:59:19.019 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:19.035+0000 7fe21fe00700 1 -- 172.21.15.107:0/1294791047 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7fe21409b690 con 0x7fe2281029d0 2024-03-08T20:59:19.260 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:19.276+0000 7fe22e600700 1 -- 172.21.15.107:0/1294791047 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fe22802d830 con 0x7fe2281029d0 2024-03-08T20:59:19.260 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:19.277+0000 7fe21fe00700 1 -- 172.21.15.107:0/1294791047 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fe21409af90 con 0x7fe2281029d0 2024-03-08T20:59:19.261 INFO:teuthology.orchestra.run.smithi107.stdout:34359738477 2024-03-08T20:59:19.264 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:19.280+0000 7fe22e600700 1 -- 172.21.15.107:0/1294791047 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fe20c0a2920 msgr2=0x7fe20c0a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:59:19.264 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:19.280+0000 7fe22e600700 1 --2- 172.21.15.107:0/1294791047 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fe20c0a2920 0x7fe20c0a4de0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fe218005950 tx=0x7fe218005f30 comp rx=0 tx=0).stop 2024-03-08T20:59:19.264 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:19.280+0000 7fe22e600700 1 -- 172.21.15.107:0/1294791047 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe2281029d0 msgr2=0x7fe228198a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:59:19.264 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:19.280+0000 7fe22e600700 1 --2- 172.21.15.107:0/1294791047 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe2281029d0 0x7fe228198a20 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fe21400d980 tx=0x7fe21400dd40 comp rx=0 tx=0).stop 2024-03-08T20:59:19.264 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:19.280+0000 7fe22e600700 1 -- 172.21.15.107:0/1294791047 shutdown_connections 2024-03-08T20:59:19.264 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:19.280+0000 7fe22e600700 1 --2- 172.21.15.107:0/1294791047 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe228107100 0x7fe22819cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:19.264 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:19.280+0000 7fe22e600700 1 --2- 172.21.15.107:0/1294791047 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fe20c0a2920 0x7fe20c0a4de0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:19.264 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:19.280+0000 7fe22e600700 1 --2- 172.21.15.107:0/1294791047 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe228102020 0x7fe2281984e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:19.264 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:19.280+0000 7fe22e600700 1 --2- 172.21.15.107:0/1294791047 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe2281029d0 0x7fe228198a20 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:19.264 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:19.280+0000 7fe22e600700 1 -- 172.21.15.107:0/1294791047 >> 172.21.15.107:0/1294791047 conn(0x7fe2280fd890 msgr2=0x7fe2280ffbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:59:19.265 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:19.280+0000 7fe22e600700 1 -- 172.21.15.107:0/1294791047 shutdown_connections 2024-03-08T20:59:19.265 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:19.280+0000 7fe22e600700 1 -- 172.21.15.107:0/1294791047 wait complete. 2024-03-08T20:59:19.690 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:59:19 smithi107 ceph-mon[31381]: pgmap v407: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-03-08T20:59:19.690 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:59:19 smithi107 ceph-mon[31381]: from='client.? 172.21.15.107:0/1294791047' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-03-08T20:59:19.765 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:59:19 smithi149 ceph-mon[35042]: pgmap v407: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-03-08T20:59:19.765 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:59:19 smithi149 ceph-mon[35042]: from='client.? 172.21.15.107:0/1294791047' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-03-08T20:59:19.878 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:59:19 smithi204 ceph-mon[34935]: pgmap v407: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-03-08T20:59:19.878 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:59:19 smithi204 ceph-mon[34935]: from='client.? 172.21.15.107:0/1294791047' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-03-08T20:59:20.116 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738470 got 34359738477 for osd.0 2024-03-08T20:59:20.116 DEBUG:teuthology.parallel:result is None 2024-03-08T20:59:20.116 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-03-08T20:59:20.116 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761 shell --fsid dc20f0f8-dd8c-11ee-95c6-87774f69a715 -- ceph pg dump --format=json 2024-03-08T20:59:20.603 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/mon.a/config 2024-03-08T20:59:21.628 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:59:21 smithi204 ceph-mon[34935]: pgmap v408: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-08T20:59:21.628 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:59:21 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:59:21.628 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:59:21 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:59:21.660 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:59:21 smithi107 ceph-mon[31381]: pgmap v408: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-08T20:59:21.660 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:59:21 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:59:21.660 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:59:21 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:59:21.765 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:59:21 smithi149 ceph-mon[35042]: pgmap v408: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-08T20:59:21.765 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:59:21 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:59:21.765 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:59:21 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:59:22.684 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:59:22 smithi107 ceph-mon[31381]: mgrmap e18: x(active, since 11m), standbys: z, y 2024-03-08T20:59:22.684 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:59:22 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:59:22.684 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:59:22 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T20:59:22.684 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:59:22 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:59:22.765 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:59:22 smithi149 ceph-mon[35042]: mgrmap e18: x(active, since 11m), standbys: z, y 2024-03-08T20:59:22.765 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:59:22 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:59:22.765 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:59:22 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T20:59:22.765 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:59:22 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:59:22.850 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:22.866+0000 7f60ade00700 1 -- 172.21.15.107:0/662786469 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f60a8108e10 msgr2=0x7f60a8115320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:59:22.850 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:22.866+0000 7f60ade00700 1 --2- 172.21.15.107:0/662786469 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f60a8108e10 0x7f60a8115320 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f60a000b0a0 tx=0x7f60a000b3b0 comp rx=0 tx=0).stop 2024-03-08T20:59:22.850 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:22.866+0000 7f60ade00700 1 -- 172.21.15.107:0/662786469 shutdown_connections 2024-03-08T20:59:22.850 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:22.866+0000 7f60ade00700 1 --2- 172.21.15.107:0/662786469 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f60a810e4e0 0x7f60a810e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:22.850 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:22.866+0000 7f60ade00700 1 --2- 172.21.15.107:0/662786469 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f60a8108e10 0x7f60a8115320 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:22.850 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:22.866+0000 7f60ade00700 1 --2- 172.21.15.107:0/662786469 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f60a8108450 0x7f60a81088d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:22.850 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:22.866+0000 7f60ade00700 1 -- 172.21.15.107:0/662786469 >> 172.21.15.107:0/662786469 conn(0x7f60a8076430 msgr2=0x7f60a8078850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:59:22.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:22.866+0000 7f60ade00700 1 -- 172.21.15.107:0/662786469 shutdown_connections 2024-03-08T20:59:22.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:22.866+0000 7f60ade00700 1 -- 172.21.15.107:0/662786469 wait complete. 2024-03-08T20:59:22.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:22.867+0000 7f60ade00700 1 Processor -- start 2024-03-08T20:59:22.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:22.867+0000 7f60ade00700 1 -- start start 2024-03-08T20:59:22.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:22.868+0000 7f60ade00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f60a8108450 0x7f60a81a46d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:22.853 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:22.868+0000 7f60ade00700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f60a8108e10 0x7f60a81a4c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:22.853 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:22.868+0000 7f60ade00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f60a810e4e0 0x7f60a819e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:22.853 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:22.868+0000 7f60ade00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60a8118610 con 0x7f60a8108450 2024-03-08T20:59:22.853 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:22.868+0000 7f60ade00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60a8118490 con 0x7f60a810e4e0 2024-03-08T20:59:22.853 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:22.868+0000 7f60ade00700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60a8118790 con 0x7f60a8108e10 2024-03-08T20:59:22.853 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:22.868+0000 7f60a6a00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f60a8108450 0x7f60a81a46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:59:22.853 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:22.868+0000 7f60a6a00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f60a8108450 0x7f60a81a46d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:51610/0 (socket says 172.21.15.107:51610) 2024-03-08T20:59:22.853 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:22.868+0000 7f60a6a00700 1 -- 172.21.15.107:0/3601817122 learned_addr learned my addr 172.21.15.107:0/3601817122 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T20:59:22.853 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:22.868+0000 7f60a6000700 1 --2- 172.21.15.107:0/3601817122 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f60a8108e10 0x7f60a81a4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:59:22.853 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:22.869+0000 7f60a7400700 1 --2- 172.21.15.107:0/3601817122 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f60a810e4e0 0x7f60a819e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:59:22.853 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:22.869+0000 7f60a6a00700 1 -- 172.21.15.107:0/3601817122 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f60a8108e10 msgr2=0x7f60a81a4c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:59:22.853 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:22.869+0000 7f60a6a00700 1 --2- 172.21.15.107:0/3601817122 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f60a8108e10 0x7f60a81a4c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:22.853 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:22.869+0000 7f60a6a00700 1 -- 172.21.15.107:0/3601817122 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f60a810e4e0 msgr2=0x7f60a819e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:59:22.853 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:22.869+0000 7f60a6a00700 1 --2- 172.21.15.107:0/3601817122 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f60a810e4e0 0x7f60a819e890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:22.854 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:22.869+0000 7f60a6a00700 1 -- 172.21.15.107:0/3601817122 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60a819eff0 con 0x7f60a8108450 2024-03-08T20:59:22.854 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:22.869+0000 7f60a6a00700 1 --2- 172.21.15.107:0/3601817122 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f60a8108450 0x7f60a81a46d0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f609800c970 tx=0x7f609800cd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:59:22.854 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:22.869+0000 7f60aca00700 1 -- 172.21.15.107:0/3601817122 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6098007aa0 con 0x7f60a8108450 2024-03-08T20:59:22.854 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:22.869+0000 7f60ade00700 1 -- 172.21.15.107:0/3601817122 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60a819f2e0 con 0x7f60a8108450 2024-03-08T20:59:22.854 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:22.870+0000 7f60ade00700 1 -- 172.21.15.107:0/3601817122 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60a81ab5a0 con 0x7f60a8108450 2024-03-08T20:59:22.855 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:22.870+0000 7f60aca00700 1 -- 172.21.15.107:0/3601817122 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f6098007c40 con 0x7f60a8108450 2024-03-08T20:59:22.855 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:22.870+0000 7f60aca00700 1 -- 172.21.15.107:0/3601817122 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f609800f6b0 con 0x7f60a8108450 2024-03-08T20:59:22.855 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:22.871+0000 7f60ade00700 1 -- 172.21.15.107:0/3601817122 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6088005330 con 0x7f60a8108450 2024-03-08T20:59:22.856 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:22.873+0000 7f60aca00700 1 -- 172.21.15.107:0/3601817122 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f609800f850 con 0x7f60a8108450 2024-03-08T20:59:22.857 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:22.874+0000 7f60aca00700 1 --2- 172.21.15.107:0/3601817122 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f608c0a2890 0x7f608c0a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:22.858 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:22.874+0000 7f60aca00700 1 -- 172.21.15.107:0/3601817122 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f60980cec30 con 0x7f60a8108450 2024-03-08T20:59:22.858 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:22.874+0000 7f60a6000700 1 --2- 172.21.15.107:0/3601817122 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f608c0a2890 0x7f608c0a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:59:22.858 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:22.875+0000 7f60a6000700 1 --2- 172.21.15.107:0/3601817122 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f608c0a2890 0x7f608c0a4d50 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f6094005ea0 tx=0x7f6094005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:59:22.862 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:22.879+0000 7f60aca00700 1 -- 172.21.15.107:0/3601817122 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f609809bd20 con 0x7f60a8108450 2024-03-08T20:59:22.878 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:59:22 smithi204 ceph-mon[34935]: mgrmap e18: x(active, since 11m), standbys: z, y 2024-03-08T20:59:22.878 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:59:22 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T20:59:22.878 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:59:22 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T20:59:22.878 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:59:22 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T20:59:23.083 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:23.099+0000 7f60ade00700 1 -- 172.21.15.107:0/3601817122 --> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f6088000bc0 con 0x7f608c0a2890 2024-03-08T20:59:23.093 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:23.109+0000 7f60aca00700 1 -- 172.21.15.107:0/3601817122 <== mgr.14152 v2:172.21.15.107:6800/3376636618 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+259180 (secure 0 0 0) 0x7f6088000bc0 con 0x7f608c0a2890 2024-03-08T20:59:23.094 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-08T20:59:23.099 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:23.115+0000 7f60ade00700 1 -- 172.21.15.107:0/3601817122 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f608c0a2890 msgr2=0x7f608c0a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:59:23.099 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:23.115+0000 7f60ade00700 1 --2- 172.21.15.107:0/3601817122 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f608c0a2890 0x7f608c0a4d50 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f6094005ea0 tx=0x7f6094005e30 comp rx=0 tx=0).stop 2024-03-08T20:59:23.099 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:23.115+0000 7f60ade00700 1 -- 172.21.15.107:0/3601817122 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f60a8108450 msgr2=0x7f60a81a46d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:59:23.099 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:23.115+0000 7f60ade00700 1 --2- 172.21.15.107:0/3601817122 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f60a8108450 0x7f60a81a46d0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f609800c970 tx=0x7f609800cd30 comp rx=0 tx=0).stop 2024-03-08T20:59:23.099 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:23.116+0000 7f60ade00700 1 -- 172.21.15.107:0/3601817122 shutdown_connections 2024-03-08T20:59:23.099 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:23.116+0000 7f60ade00700 1 --2- 172.21.15.107:0/3601817122 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f608c0a2890 0x7f608c0a4d50 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:23.099 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:23.116+0000 7f60ade00700 1 --2- 172.21.15.107:0/3601817122 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f60a810e4e0 0x7f60a819e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:23.099 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:23.116+0000 7f60ade00700 1 --2- 172.21.15.107:0/3601817122 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f60a8108450 0x7f60a81a46d0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:23.099 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:23.116+0000 7f60ade00700 1 --2- 172.21.15.107:0/3601817122 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f60a8108e10 0x7f60a81a4c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:23.100 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:23.116+0000 7f60ade00700 1 -- 172.21.15.107:0/3601817122 >> 172.21.15.107:0/3601817122 conn(0x7f60a8076430 msgr2=0x7f60a8076950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:59:23.100 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:23.116+0000 7f60ade00700 1 -- 172.21.15.107:0/3601817122 shutdown_connections 2024-03-08T20:59:23.100 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:23.116+0000 7f60ade00700 1 -- 172.21.15.107:0/3601817122 wait complete. 2024-03-08T20:59:23.101 INFO:teuthology.orchestra.run.smithi107.stderr:dumped all 2024-03-08T20:59:23.765 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:59:23 smithi149 ceph-mon[35042]: pgmap v409: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-08T20:59:23.878 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:59:23 smithi204 ceph-mon[34935]: pgmap v409: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-08T20:59:23.967 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:59:23 smithi107 ceph-mon[31381]: pgmap v409: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-08T20:59:24.037 INFO:teuthology.orchestra.run.smithi107.stdout:{"pg_ready":true,"pg_map":{"version":409,"stamp":"2024-03-08T20:59:22.348938+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":657047,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":950,"num_read_kb":603,"num_write":420,"num_write_kb":3073,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":1240299,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":497,"ondisk_log_size":497,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":369320,"kb_used_data":11804,"kb_used_omap":18,"kb_used_meta":357357,"kb_avail":1124572504,"statfs":{"total":1151940427776,"available":1151562244096,"internally_reserved":0,"allocated":12087296,"data_stored":7493934,"data_compressed":190168,"data_compressed_allocated":253952,"data_compressed_original":2039808,"omap_allocated":19138,"internal_metadata":365933886},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":120,"num_read_kb":55,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005242"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-08T20:58:21.461098+0000","last_change":"2024-03-08T20:57:50.800421+0000","last_active":"2024-03-08T20:58:21.461098+0000","last_peered":"2024-03-08T20:58:21.461098+0000","last_clean":"2024-03-08T20:58:21.461098+0000","last_became_active":"2024-03-08T20:57:50.799682+0000","last_became_peered":"2024-03-08T20:57:50.799682+0000","last_unstale":"2024-03-08T20:58:21.461098+0000","last_undegraded":"2024-03-08T20:58:21.461098+0000","last_fullsized":"2024-03-08T20:58:21.461098+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T07:25:11.394329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-08T20:58:21.466749+0000","last_change":"2024-03-08T20:57:50.800332+0000","last_active":"2024-03-08T20:58:21.466749+0000","last_peered":"2024-03-08T20:58:21.466749+0000","last_clean":"2024-03-08T20:58:21.466749+0000","last_became_active":"2024-03-08T20:57:50.799981+0000","last_became_peered":"2024-03-08T20:57:50.799981+0000","last_unstale":"2024-03-08T20:58:21.466749+0000","last_undegraded":"2024-03-08T20:58:21.466749+0000","last_fullsized":"2024-03-08T20:58:21.466749+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T06:08:19.618115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-08T20:58:21.466632+0000","last_change":"2024-03-08T20:57:50.800982+0000","last_active":"2024-03-08T20:58:21.466632+0000","last_peered":"2024-03-08T20:58:21.466632+0000","last_clean":"2024-03-08T20:58:21.466632+0000","last_became_active":"2024-03-08T20:57:50.800661+0000","last_became_peered":"2024-03-08T20:57:50.800661+0000","last_unstale":"2024-03-08T20:58:21.466632+0000","last_undegraded":"2024-03-08T20:58:21.466632+0000","last_fullsized":"2024-03-08T20:58:21.466632+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T22:33:08.869516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-08T20:58:21.483192+0000","last_change":"2024-03-08T20:58:21.481750+0000","last_active":"2024-03-08T20:58:21.483192+0000","last_peered":"2024-03-08T20:58:21.483192+0000","last_clean":"2024-03-08T20:58:21.483192+0000","last_became_active":"2024-03-08T20:58:21.480626+0000","last_became_peered":"2024-03-08T20:58:21.480626+0000","last_unstale":"2024-03-08T20:58:21.483192+0000","last_undegraded":"2024-03-08T20:58:21.483192+0000","last_fullsized":"2024-03-08T20:58:21.483192+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T03:51:34.944508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-08T20:58:21.691140+0000","last_change":"2024-03-08T20:58:21.691140+0000","last_active":"2024-03-08T20:58:21.691140+0000","last_peered":"2024-03-08T20:58:21.691140+0000","last_clean":"2024-03-08T20:58:21.691140+0000","last_became_active":"2024-03-08T20:58:21.689521+0000","last_became_peered":"2024-03-08T20:58:21.689521+0000","last_unstale":"2024-03-08T20:58:21.691140+0000","last_undegraded":"2024-03-08T20:58:21.691140+0000","last_fullsized":"2024-03-08T20:58:21.691140+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T23:15:52.861725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6],"acting":[8,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-08T20:58:21.470357+0000","last_change":"2024-03-08T20:57:50.789245+0000","last_active":"2024-03-08T20:58:21.470357+0000","last_peered":"2024-03-08T20:58:21.470357+0000","last_clean":"2024-03-08T20:58:21.470357+0000","last_became_active":"2024-03-08T20:57:50.788921+0000","last_became_peered":"2024-03-08T20:57:50.788921+0000","last_unstale":"2024-03-08T20:58:21.470357+0000","last_undegraded":"2024-03-08T20:58:21.470357+0000","last_fullsized":"2024-03-08T20:58:21.470357+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T08:57:30.368714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-08T20:58:21.460186+0000","last_change":"2024-03-08T20:57:50.800725+0000","last_active":"2024-03-08T20:58:21.460186+0000","last_peered":"2024-03-08T20:58:21.460186+0000","last_clean":"2024-03-08T20:58:21.460186+0000","last_became_active":"2024-03-08T20:57:50.800317+0000","last_became_peered":"2024-03-08T20:57:50.800317+0000","last_unstale":"2024-03-08T20:58:21.460186+0000","last_undegraded":"2024-03-08T20:58:21.460186+0000","last_fullsized":"2024-03-08T20:58:21.460186+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T05:44:01.402547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-08T20:58:21.685377+0000","last_change":"2024-03-08T20:57:50.801543+0000","last_active":"2024-03-08T20:58:21.685377+0000","last_peered":"2024-03-08T20:58:21.685377+0000","last_clean":"2024-03-08T20:58:21.685377+0000","last_became_active":"2024-03-08T20:57:50.801111+0000","last_became_peered":"2024-03-08T20:57:50.801111+0000","last_unstale":"2024-03-08T20:58:21.685377+0000","last_undegraded":"2024-03-08T20:58:21.685377+0000","last_fullsized":"2024-03-08T20:58:21.685377+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T06:32:55.234372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"88'11","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-08T20:59:20.324538+0000","last_change":"2024-03-08T20:57:50.798169+0000","last_active":"2024-03-08T20:59:20.324538+0000","last_peered":"2024-03-08T20:59:20.324538+0000","last_clean":"2024-03-08T20:59:20.324538+0000","last_became_active":"2024-03-08T20:57:50.797817+0000","last_became_peered":"2024-03-08T20:57:50.797817+0000","last_unstale":"2024-03-08T20:59:20.324538+0000","last_undegraded":"2024-03-08T20:59:20.324538+0000","last_fullsized":"2024-03-08T20:59:20.324538+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T04:01:46.179415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-08T20:58:21.472323+0000","last_change":"2024-03-08T20:57:50.790757+0000","last_active":"2024-03-08T20:58:21.472323+0000","last_peered":"2024-03-08T20:58:21.472323+0000","last_clean":"2024-03-08T20:58:21.472323+0000","last_became_active":"2024-03-08T20:57:50.790219+0000","last_became_peered":"2024-03-08T20:57:50.790219+0000","last_unstale":"2024-03-08T20:58:21.472323+0000","last_undegraded":"2024-03-08T20:58:21.472323+0000","last_fullsized":"2024-03-08T20:58:21.472323+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T07:55:54.945727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-08T20:58:21.461124+0000","last_change":"2024-03-08T20:57:50.809102+0000","last_active":"2024-03-08T20:58:21.461124+0000","last_peered":"2024-03-08T20:58:21.461124+0000","last_clean":"2024-03-08T20:58:21.461124+0000","last_became_active":"2024-03-08T20:57:50.808647+0000","last_became_peered":"2024-03-08T20:57:50.808647+0000","last_unstale":"2024-03-08T20:58:21.461124+0000","last_undegraded":"2024-03-08T20:58:21.461124+0000","last_fullsized":"2024-03-08T20:58:21.461124+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T00:29:26.317611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-08T20:59:20.322508+0000","last_change":"2024-03-08T20:57:50.791791+0000","last_active":"2024-03-08T20:59:20.322508+0000","last_peered":"2024-03-08T20:59:20.322508+0000","last_clean":"2024-03-08T20:59:20.322508+0000","last_became_active":"2024-03-08T20:57:50.790264+0000","last_became_peered":"2024-03-08T20:57:50.790264+0000","last_unstale":"2024-03-08T20:59:20.322508+0000","last_undegraded":"2024-03-08T20:59:20.322508+0000","last_fullsized":"2024-03-08T20:59:20.322508+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T08:54:30.221330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"88'10","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-08T20:59:20.335640+0000","last_change":"2024-03-08T20:57:50.793990+0000","last_active":"2024-03-08T20:59:20.335640+0000","last_peered":"2024-03-08T20:59:20.335640+0000","last_clean":"2024-03-08T20:59:20.335640+0000","last_became_active":"2024-03-08T20:57:50.792304+0000","last_became_peered":"2024-03-08T20:57:50.792304+0000","last_unstale":"2024-03-08T20:59:20.335640+0000","last_undegraded":"2024-03-08T20:59:20.335640+0000","last_fullsized":"2024-03-08T20:59:20.335640+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T00:21:57.491218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-08T20:58:21.465926+0000","last_change":"2024-03-08T20:57:50.787621+0000","last_active":"2024-03-08T20:58:21.465926+0000","last_peered":"2024-03-08T20:58:21.465926+0000","last_clean":"2024-03-08T20:58:21.465926+0000","last_became_active":"2024-03-08T20:57:50.786461+0000","last_became_peered":"2024-03-08T20:57:50.786461+0000","last_unstale":"2024-03-08T20:58:21.465926+0000","last_undegraded":"2024-03-08T20:58:21.465926+0000","last_fullsized":"2024-03-08T20:58:21.465926+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T08:35:32.513942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-08T20:58:21.465834+0000","last_change":"2024-03-08T20:57:50.789157+0000","last_active":"2024-03-08T20:58:21.465834+0000","last_peered":"2024-03-08T20:58:21.465834+0000","last_clean":"2024-03-08T20:58:21.465834+0000","last_became_active":"2024-03-08T20:57:50.788828+0000","last_became_peered":"2024-03-08T20:57:50.788828+0000","last_unstale":"2024-03-08T20:58:21.465834+0000","last_undegraded":"2024-03-08T20:58:21.465834+0000","last_fullsized":"2024-03-08T20:58:21.465834+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T05:43:53.398306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-08T20:59:20.337357+0000","last_change":"2024-03-08T20:57:50.796888+0000","last_active":"2024-03-08T20:59:20.337357+0000","last_peered":"2024-03-08T20:59:20.337357+0000","last_clean":"2024-03-08T20:59:20.337357+0000","last_became_active":"2024-03-08T20:57:50.795797+0000","last_became_peered":"2024-03-08T20:57:50.795797+0000","last_unstale":"2024-03-08T20:59:20.337357+0000","last_undegraded":"2024-03-08T20:59:20.337357+0000","last_fullsized":"2024-03-08T20:59:20.337357+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T00:43:53.838877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-08T20:58:21.460764+0000","last_change":"2024-03-08T20:57:50.801849+0000","last_active":"2024-03-08T20:58:21.460764+0000","last_peered":"2024-03-08T20:58:21.460764+0000","last_clean":"2024-03-08T20:58:21.460764+0000","last_became_active":"2024-03-08T20:57:50.801375+0000","last_became_peered":"2024-03-08T20:57:50.801375+0000","last_unstale":"2024-03-08T20:58:21.460764+0000","last_undegraded":"2024-03-08T20:58:21.460764+0000","last_fullsized":"2024-03-08T20:58:21.460764+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T03:16:44.652264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-08T20:58:21.458733+0000","last_change":"2024-03-08T20:57:50.790493+0000","last_active":"2024-03-08T20:58:21.458733+0000","last_peered":"2024-03-08T20:58:21.458733+0000","last_clean":"2024-03-08T20:58:21.458733+0000","last_became_active":"2024-03-08T20:57:50.789074+0000","last_became_peered":"2024-03-08T20:57:50.789074+0000","last_unstale":"2024-03-08T20:58:21.458733+0000","last_undegraded":"2024-03-08T20:58:21.458733+0000","last_fullsized":"2024-03-08T20:58:21.458733+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T05:52:45.700669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-08T20:58:21.469003+0000","last_change":"2024-03-08T20:57:50.794348+0000","last_active":"2024-03-08T20:58:21.469003+0000","last_peered":"2024-03-08T20:58:21.469003+0000","last_clean":"2024-03-08T20:58:21.469003+0000","last_became_active":"2024-03-08T20:57:50.793962+0000","last_became_peered":"2024-03-08T20:57:50.793962+0000","last_unstale":"2024-03-08T20:58:21.469003+0000","last_undegraded":"2024-03-08T20:58:21.469003+0000","last_fullsized":"2024-03-08T20:58:21.469003+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T01:38:05.146148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"86'13","reported_seq":115,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-08T20:59:18.109696+0000","last_change":"2024-03-08T20:57:49.792250+0000","last_active":"2024-03-08T20:59:18.109696+0000","last_peered":"2024-03-08T20:59:18.109696+0000","last_clean":"2024-03-08T20:59:18.109696+0000","last_became_active":"2024-03-08T20:57:49.791863+0000","last_became_peered":"2024-03-08T20:57:49.791863+0000","last_unstale":"2024-03-08T20:59:18.109696+0000","last_undegraded":"2024-03-08T20:59:18.109696+0000","last_fullsized":"2024-03-08T20:59:18.109696+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_clean_scrub_stamp":"2024-03-08T20:57:48.739829+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T05:37:40.942754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":83,"num_read_kb":83,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-08T20:58:21.686701+0000","last_change":"2024-03-08T20:57:50.802789+0000","last_active":"2024-03-08T20:58:21.686701+0000","last_peered":"2024-03-08T20:58:21.686701+0000","last_clean":"2024-03-08T20:58:21.686701+0000","last_became_active":"2024-03-08T20:57:50.802437+0000","last_became_peered":"2024-03-08T20:57:50.802437+0000","last_unstale":"2024-03-08T20:58:21.686701+0000","last_undegraded":"2024-03-08T20:58:21.686701+0000","last_fullsized":"2024-03-08T20:58:21.686701+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T23:47:39.882241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-08T20:58:21.687084+0000","last_change":"2024-03-08T20:57:49.785390+0000","last_active":"2024-03-08T20:58:21.687084+0000","last_peered":"2024-03-08T20:58:21.687084+0000","last_clean":"2024-03-08T20:58:21.687084+0000","last_became_active":"2024-03-08T20:57:49.784506+0000","last_became_peered":"2024-03-08T20:57:49.784506+0000","last_unstale":"2024-03-08T20:58:21.687084+0000","last_undegraded":"2024-03-08T20:58:21.687084+0000","last_fullsized":"2024-03-08T20:58:21.687084+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_clean_scrub_stamp":"2024-03-08T20:57:48.739829+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T03:21:54.331376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-08T20:58:21.467172+0000","last_change":"2024-03-08T20:57:50.796868+0000","last_active":"2024-03-08T20:58:21.467172+0000","last_peered":"2024-03-08T20:58:21.467172+0000","last_clean":"2024-03-08T20:58:21.467172+0000","last_became_active":"2024-03-08T20:57:50.796163+0000","last_became_peered":"2024-03-08T20:57:50.796163+0000","last_unstale":"2024-03-08T20:58:21.467172+0000","last_undegraded":"2024-03-08T20:58:21.467172+0000","last_fullsized":"2024-03-08T20:58:21.467172+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T21:26:46.335292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"86'12","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-08T20:58:21.474019+0000","last_change":"2024-03-08T20:57:49.794940+0000","last_active":"2024-03-08T20:58:21.474019+0000","last_peered":"2024-03-08T20:58:21.474019+0000","last_clean":"2024-03-08T20:58:21.474019+0000","last_became_active":"2024-03-08T20:57:49.794386+0000","last_became_peered":"2024-03-08T20:57:49.794386+0000","last_unstale":"2024-03-08T20:58:21.474019+0000","last_undegraded":"2024-03-08T20:58:21.474019+0000","last_fullsized":"2024-03-08T20:58:21.474019+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_clean_scrub_stamp":"2024-03-08T20:57:48.739829+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T04:21:09.953351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-08T20:59:20.326490+0000","last_change":"2024-03-08T20:57:50.794804+0000","last_active":"2024-03-08T20:59:20.326490+0000","last_peered":"2024-03-08T20:59:20.326490+0000","last_clean":"2024-03-08T20:59:20.326490+0000","last_became_active":"2024-03-08T20:57:50.794380+0000","last_became_peered":"2024-03-08T20:57:50.794380+0000","last_unstale":"2024-03-08T20:59:20.326490+0000","last_undegraded":"2024-03-08T20:59:20.326490+0000","last_fullsized":"2024-03-08T20:59:20.326490+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T05:32:21.015740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"86'5","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-08T20:58:21.476574+0000","last_change":"2024-03-08T20:57:49.790881+0000","last_active":"2024-03-08T20:58:21.476574+0000","last_peered":"2024-03-08T20:58:21.476574+0000","last_clean":"2024-03-08T20:58:21.476574+0000","last_became_active":"2024-03-08T20:57:49.790562+0000","last_became_peered":"2024-03-08T20:57:49.790562+0000","last_unstale":"2024-03-08T20:58:21.476574+0000","last_undegraded":"2024-03-08T20:58:21.476574+0000","last_fullsized":"2024-03-08T20:58:21.476574+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_clean_scrub_stamp":"2024-03-08T20:57:48.739829+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T23:57:40.542756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"88'15","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-08T20:59:20.329105+0000","last_change":"2024-03-08T20:57:50.799021+0000","last_active":"2024-03-08T20:59:20.329105+0000","last_peered":"2024-03-08T20:59:20.329105+0000","last_clean":"2024-03-08T20:59:20.329105+0000","last_became_active":"2024-03-08T20:57:50.795494+0000","last_became_peered":"2024-03-08T20:57:50.795494+0000","last_unstale":"2024-03-08T20:59:20.329105+0000","last_undegraded":"2024-03-08T20:59:20.329105+0000","last_fullsized":"2024-03-08T20:59:20.329105+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T21:26:40.936789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"86'5","reported_seq":109,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-08T20:59:19.123544+0000","last_change":"2024-03-08T20:57:49.791167+0000","last_active":"2024-03-08T20:59:19.123544+0000","last_peered":"2024-03-08T20:59:19.123544+0000","last_clean":"2024-03-08T20:59:19.123544+0000","last_became_active":"2024-03-08T20:57:49.789840+0000","last_became_peered":"2024-03-08T20:57:49.789840+0000","last_unstale":"2024-03-08T20:59:19.123544+0000","last_undegraded":"2024-03-08T20:59:19.123544+0000","last_fullsized":"2024-03-08T20:59:19.123544+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_clean_scrub_stamp":"2024-03-08T20:57:48.739829+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T23:14:51.103583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":85,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-08T20:58:21.472421+0000","last_change":"2024-03-08T20:57:50.812656+0000","last_active":"2024-03-08T20:58:21.472421+0000","last_peered":"2024-03-08T20:58:21.472421+0000","last_clean":"2024-03-08T20:58:21.472421+0000","last_became_active":"2024-03-08T20:57:50.812281+0000","last_became_peered":"2024-03-08T20:57:50.812281+0000","last_unstale":"2024-03-08T20:58:21.472421+0000","last_undegraded":"2024-03-08T20:58:21.472421+0000","last_fullsized":"2024-03-08T20:58:21.472421+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T05:40:20.985798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"86'7","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-08T20:58:21.470056+0000","last_change":"2024-03-08T20:57:49.792874+0000","last_active":"2024-03-08T20:58:21.470056+0000","last_peered":"2024-03-08T20:58:21.470056+0000","last_clean":"2024-03-08T20:58:21.470056+0000","last_became_active":"2024-03-08T20:57:49.792418+0000","last_became_peered":"2024-03-08T20:57:49.792418+0000","last_unstale":"2024-03-08T20:58:21.470056+0000","last_undegraded":"2024-03-08T20:58:21.470056+0000","last_fullsized":"2024-03-08T20:58:21.470056+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_clean_scrub_stamp":"2024-03-08T20:57:48.739829+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T02:23:12.888435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-08T20:59:20.323530+0000","last_change":"2024-03-08T20:57:50.805567+0000","last_active":"2024-03-08T20:59:20.323530+0000","last_peered":"2024-03-08T20:59:20.323530+0000","last_clean":"2024-03-08T20:59:20.323530+0000","last_became_active":"2024-03-08T20:57:50.805120+0000","last_became_peered":"2024-03-08T20:57:50.805120+0000","last_unstale":"2024-03-08T20:59:20.323530+0000","last_undegraded":"2024-03-08T20:59:20.323530+0000","last_fullsized":"2024-03-08T20:59:20.323530+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T06:18:14.297971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"86'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-08T20:58:21.686521+0000","last_change":"2024-03-08T20:57:49.771806+0000","last_active":"2024-03-08T20:58:21.686521+0000","last_peered":"2024-03-08T20:58:21.686521+0000","last_clean":"2024-03-08T20:58:21.686521+0000","last_became_active":"2024-03-08T20:57:49.771447+0000","last_became_peered":"2024-03-08T20:57:49.771447+0000","last_unstale":"2024-03-08T20:58:21.686521+0000","last_undegraded":"2024-03-08T20:58:21.686521+0000","last_fullsized":"2024-03-08T20:58:21.686521+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_clean_scrub_stamp":"2024-03-08T20:57:48.739829+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T03:16:37.913599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-08T20:58:21.685406+0000","last_change":"2024-03-08T20:57:50.792915+0000","last_active":"2024-03-08T20:58:21.685406+0000","last_peered":"2024-03-08T20:58:21.685406+0000","last_clean":"2024-03-08T20:58:21.685406+0000","last_became_active":"2024-03-08T20:57:50.792617+0000","last_became_peered":"2024-03-08T20:57:50.792617+0000","last_unstale":"2024-03-08T20:58:21.685406+0000","last_undegraded":"2024-03-08T20:58:21.685406+0000","last_fullsized":"2024-03-08T20:58:21.685406+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T01:59:30.427751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"86'3","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-08T20:58:21.469835+0000","last_change":"2024-03-08T20:57:49.789383+0000","last_active":"2024-03-08T20:58:21.469835+0000","last_peered":"2024-03-08T20:58:21.469835+0000","last_clean":"2024-03-08T20:58:21.469835+0000","last_became_active":"2024-03-08T20:57:49.788975+0000","last_became_peered":"2024-03-08T20:57:49.788975+0000","last_unstale":"2024-03-08T20:58:21.469835+0000","last_undegraded":"2024-03-08T20:58:21.469835+0000","last_fullsized":"2024-03-08T20:58:21.469835+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_clean_scrub_stamp":"2024-03-08T20:57:48.739829+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T07:42:21.366476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-08T20:58:21.457884+0000","last_change":"2024-03-08T20:57:50.785377+0000","last_active":"2024-03-08T20:58:21.457884+0000","last_peered":"2024-03-08T20:58:21.457884+0000","last_clean":"2024-03-08T20:58:21.457884+0000","last_became_active":"2024-03-08T20:57:50.785067+0000","last_became_peered":"2024-03-08T20:57:50.785067+0000","last_unstale":"2024-03-08T20:58:21.457884+0000","last_undegraded":"2024-03-08T20:58:21.457884+0000","last_fullsized":"2024-03-08T20:58:21.457884+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T02:49:10.388474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"86'4","reported_seq":183,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-08T20:59:17.116618+0000","last_change":"2024-03-08T20:57:49.784190+0000","last_active":"2024-03-08T20:59:17.116618+0000","last_peered":"2024-03-08T20:59:17.116618+0000","last_clean":"2024-03-08T20:59:17.116618+0000","last_became_active":"2024-03-08T20:57:49.783805+0000","last_became_peered":"2024-03-08T20:57:49.783805+0000","last_unstale":"2024-03-08T20:59:17.116618+0000","last_undegraded":"2024-03-08T20:59:17.116618+0000","last_fullsized":"2024-03-08T20:59:17.116618+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_clean_scrub_stamp":"2024-03-08T20:57:48.739829+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T03:16:12.983559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":160,"num_read_kb":160,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"88'10","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-08T20:59:20.326363+0000","last_change":"2024-03-08T20:57:50.795647+0000","last_active":"2024-03-08T20:59:20.326363+0000","last_peered":"2024-03-08T20:59:20.326363+0000","last_clean":"2024-03-08T20:59:20.326363+0000","last_became_active":"2024-03-08T20:57:50.793179+0000","last_became_peered":"2024-03-08T20:57:50.793179+0000","last_unstale":"2024-03-08T20:59:20.326363+0000","last_undegraded":"2024-03-08T20:59:20.326363+0000","last_fullsized":"2024-03-08T20:59:20.326363+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T05:56:14.051601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"86'4","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-08T20:58:21.460488+0000","last_change":"2024-03-08T20:57:49.795093+0000","last_active":"2024-03-08T20:58:21.460488+0000","last_peered":"2024-03-08T20:58:21.460488+0000","last_clean":"2024-03-08T20:58:21.460488+0000","last_became_active":"2024-03-08T20:57:49.794617+0000","last_became_peered":"2024-03-08T20:57:49.794617+0000","last_unstale":"2024-03-08T20:58:21.460488+0000","last_undegraded":"2024-03-08T20:58:21.460488+0000","last_fullsized":"2024-03-08T20:58:21.460488+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_clean_scrub_stamp":"2024-03-08T20:57:48.739829+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T05:56:38.669695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-08T20:58:21.480052+0000","last_change":"2024-03-08T20:58:21.480052+0000","last_active":"2024-03-08T20:58:21.480052+0000","last_peered":"2024-03-08T20:58:21.480052+0000","last_clean":"2024-03-08T20:58:21.480052+0000","last_became_active":"2024-03-08T20:58:21.478856+0000","last_became_peered":"2024-03-08T20:58:21.478856+0000","last_unstale":"2024-03-08T20:58:21.480052+0000","last_undegraded":"2024-03-08T20:58:21.480052+0000","last_fullsized":"2024-03-08T20:58:21.480052+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T02:57:07.905740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,5],"acting":[0,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"86'5","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-08T20:58:21.473105+0000","last_change":"2024-03-08T20:57:49.776991+0000","last_active":"2024-03-08T20:58:21.473105+0000","last_peered":"2024-03-08T20:58:21.473105+0000","last_clean":"2024-03-08T20:58:21.473105+0000","last_became_active":"2024-03-08T20:57:49.776566+0000","last_became_peered":"2024-03-08T20:57:49.776566+0000","last_unstale":"2024-03-08T20:58:21.473105+0000","last_undegraded":"2024-03-08T20:58:21.473105+0000","last_fullsized":"2024-03-08T20:58:21.473105+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_clean_scrub_stamp":"2024-03-08T20:57:48.739829+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T21:02:37.726353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"88'11","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-08T20:59:20.328988+0000","last_change":"2024-03-08T20:57:50.793976+0000","last_active":"2024-03-08T20:59:20.328988+0000","last_peered":"2024-03-08T20:59:20.328988+0000","last_clean":"2024-03-08T20:59:20.328988+0000","last_became_active":"2024-03-08T20:57:50.792472+0000","last_became_peered":"2024-03-08T20:57:50.792472+0000","last_unstale":"2024-03-08T20:59:20.328988+0000","last_undegraded":"2024-03-08T20:59:20.328988+0000","last_fullsized":"2024-03-08T20:59:20.328988+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T06:01:03.360171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"88'32","reported_seq":239,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-08T20:59:17.653612+0000","last_change":"2024-03-08T20:58:21.502531+0000","last_active":"2024-03-08T20:59:17.653612+0000","last_peered":"2024-03-08T20:59:17.653612+0000","last_clean":"2024-03-08T20:59:17.653612+0000","last_became_active":"2024-03-08T20:58:21.483763+0000","last_became_peered":"2024-03-08T20:58:21.483763+0000","last_unstale":"2024-03-08T20:59:17.653612+0000","last_undegraded":"2024-03-08T20:59:17.653612+0000","last_fullsized":"2024-03-08T20:59:17.653612+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_clean_scrub_stamp":"2024-03-08T20:57:48.739829+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T00:16:01.128478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":60023,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":155,"num_read_kb":134,"num_write":34,"num_write_kb":71,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":58371,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-08T20:58:21.473290+0000","last_change":"2024-03-08T20:57:50.800031+0000","last_active":"2024-03-08T20:58:21.473290+0000","last_peered":"2024-03-08T20:58:21.473290+0000","last_clean":"2024-03-08T20:58:21.473290+0000","last_became_active":"2024-03-08T20:57:50.799663+0000","last_became_peered":"2024-03-08T20:57:50.799663+0000","last_unstale":"2024-03-08T20:58:21.473290+0000","last_undegraded":"2024-03-08T20:58:21.473290+0000","last_fullsized":"2024-03-08T20:58:21.473290+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T05:37:50.734330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"86'9","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-08T20:58:21.467186+0000","last_change":"2024-03-08T20:57:49.791884+0000","last_active":"2024-03-08T20:58:21.467186+0000","last_peered":"2024-03-08T20:58:21.467186+0000","last_clean":"2024-03-08T20:58:21.467186+0000","last_became_active":"2024-03-08T20:57:49.789314+0000","last_became_peered":"2024-03-08T20:57:49.789314+0000","last_unstale":"2024-03-08T20:58:21.467186+0000","last_undegraded":"2024-03-08T20:58:21.467186+0000","last_fullsized":"2024-03-08T20:58:21.467186+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_clean_scrub_stamp":"2024-03-08T20:57:48.739829+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T08:44:20.490703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-08T20:59:20.335062+0000","last_change":"2024-03-08T20:57:50.787609+0000","last_active":"2024-03-08T20:59:20.335062+0000","last_peered":"2024-03-08T20:59:20.335062+0000","last_clean":"2024-03-08T20:59:20.335062+0000","last_became_active":"2024-03-08T20:57:50.787171+0000","last_became_peered":"2024-03-08T20:57:50.787171+0000","last_unstale":"2024-03-08T20:59:20.335062+0000","last_undegraded":"2024-03-08T20:59:20.335062+0000","last_fullsized":"2024-03-08T20:59:20.335062+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T05:57:54.023942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"86'5","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-08T20:58:21.470581+0000","last_change":"2024-03-08T20:57:49.785642+0000","last_active":"2024-03-08T20:58:21.470581+0000","last_peered":"2024-03-08T20:58:21.470581+0000","last_clean":"2024-03-08T20:58:21.470581+0000","last_became_active":"2024-03-08T20:57:49.785180+0000","last_became_peered":"2024-03-08T20:57:49.785180+0000","last_unstale":"2024-03-08T20:58:21.470581+0000","last_undegraded":"2024-03-08T20:58:21.470581+0000","last_fullsized":"2024-03-08T20:58:21.470581+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_clean_scrub_stamp":"2024-03-08T20:57:48.739829+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T03:21:03.121253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-08T20:58:21.461515+0000","last_change":"2024-03-08T20:57:50.805179+0000","last_active":"2024-03-08T20:58:21.461515+0000","last_peered":"2024-03-08T20:58:21.461515+0000","last_clean":"2024-03-08T20:58:21.461515+0000","last_became_active":"2024-03-08T20:57:50.804801+0000","last_became_peered":"2024-03-08T20:57:50.804801+0000","last_unstale":"2024-03-08T20:58:21.461515+0000","last_undegraded":"2024-03-08T20:58:21.461515+0000","last_fullsized":"2024-03-08T20:58:21.461515+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T08:35:50.224508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"86'5","reported_seq":110,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-08T20:59:20.125295+0000","last_change":"2024-03-08T20:57:49.787198+0000","last_active":"2024-03-08T20:59:20.125295+0000","last_peered":"2024-03-08T20:59:20.125295+0000","last_clean":"2024-03-08T20:59:20.125295+0000","last_became_active":"2024-03-08T20:57:49.785966+0000","last_became_peered":"2024-03-08T20:57:49.785966+0000","last_unstale":"2024-03-08T20:59:20.125295+0000","last_undegraded":"2024-03-08T20:59:20.125295+0000","last_fullsized":"2024-03-08T20:59:20.125295+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_clean_scrub_stamp":"2024-03-08T20:57:48.739829+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T21:39:03.900857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":87,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-08T20:58:21.460165+0000","last_change":"2024-03-08T20:57:50.802887+0000","last_active":"2024-03-08T20:58:21.460165+0000","last_peered":"2024-03-08T20:58:21.460165+0000","last_clean":"2024-03-08T20:58:21.460165+0000","last_became_active":"2024-03-08T20:57:50.802530+0000","last_became_peered":"2024-03-08T20:57:50.802530+0000","last_unstale":"2024-03-08T20:58:21.460165+0000","last_undegraded":"2024-03-08T20:58:21.460165+0000","last_fullsized":"2024-03-08T20:58:21.460165+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T07:08:25.980812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"86'17","reported_seq":37,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-08T20:58:21.470025+0000","last_change":"2024-03-08T20:57:49.793528+0000","last_active":"2024-03-08T20:58:21.470025+0000","last_peered":"2024-03-08T20:58:21.470025+0000","last_clean":"2024-03-08T20:58:21.470025+0000","last_became_active":"2024-03-08T20:57:49.792509+0000","last_became_peered":"2024-03-08T20:57:49.792509+0000","last_unstale":"2024-03-08T20:58:21.470025+0000","last_undegraded":"2024-03-08T20:58:21.470025+0000","last_fullsized":"2024-03-08T20:58:21.470025+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_clean_scrub_stamp":"2024-03-08T20:57:48.739829+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T03:45:20.912280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-08T20:58:21.485064+0000","last_change":"2024-03-08T20:58:21.484109+0000","last_active":"2024-03-08T20:58:21.485064+0000","last_peered":"2024-03-08T20:58:21.485064+0000","last_clean":"2024-03-08T20:58:21.485064+0000","last_became_active":"2024-03-08T20:58:21.483326+0000","last_became_peered":"2024-03-08T20:58:21.483326+0000","last_unstale":"2024-03-08T20:58:21.485064+0000","last_undegraded":"2024-03-08T20:58:21.485064+0000","last_fullsized":"2024-03-08T20:58:21.485064+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T00:16:20.462107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.f","version":"86'2","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-08T20:58:21.491048+0000","last_change":"2024-03-08T20:58:21.489895+0000","last_active":"2024-03-08T20:58:21.491048+0000","last_peered":"2024-03-08T20:58:21.491048+0000","last_clean":"2024-03-08T20:58:21.491048+0000","last_became_active":"2024-03-08T20:58:21.476612+0000","last_became_peered":"2024-03-08T20:58:21.476612+0000","last_unstale":"2024-03-08T20:58:21.491048+0000","last_undegraded":"2024-03-08T20:58:21.491048+0000","last_fullsized":"2024-03-08T20:58:21.491048+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_clean_scrub_stamp":"2024-03-08T20:57:48.739829+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T21:44:54.967671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-08T20:58:21.460580+0000","last_change":"2024-03-08T20:57:50.814625+0000","last_active":"2024-03-08T20:58:21.460580+0000","last_peered":"2024-03-08T20:58:21.460580+0000","last_clean":"2024-03-08T20:58:21.460580+0000","last_became_active":"2024-03-08T20:57:50.814220+0000","last_became_peered":"2024-03-08T20:57:50.814220+0000","last_unstale":"2024-03-08T20:58:21.460580+0000","last_undegraded":"2024-03-08T20:58:21.460580+0000","last_fullsized":"2024-03-08T20:58:21.460580+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T08:35:57.659511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"86'1","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-08T20:58:21.458294+0000","last_change":"2024-03-08T20:57:49.777922+0000","last_active":"2024-03-08T20:58:21.458294+0000","last_peered":"2024-03-08T20:58:21.458294+0000","last_clean":"2024-03-08T20:58:21.458294+0000","last_became_active":"2024-03-08T20:57:49.775083+0000","last_became_peered":"2024-03-08T20:57:49.775083+0000","last_unstale":"2024-03-08T20:58:21.458294+0000","last_undegraded":"2024-03-08T20:58:21.458294+0000","last_fullsized":"2024-03-08T20:58:21.458294+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_clean_scrub_stamp":"2024-03-08T20:57:48.739829+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T01:55:02.679999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"86'1","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-08T20:58:21.471362+0000","last_change":"2024-03-08T20:57:50.778774+0000","last_active":"2024-03-08T20:58:21.471362+0000","last_peered":"2024-03-08T20:58:21.471362+0000","last_clean":"2024-03-08T20:58:21.471362+0000","last_became_active":"2024-03-08T20:57:50.778194+0000","last_became_peered":"2024-03-08T20:57:50.778194+0000","last_unstale":"2024-03-08T20:58:21.471362+0000","last_undegraded":"2024-03-08T20:58:21.471362+0000","last_fullsized":"2024-03-08T20:58:21.471362+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T21:41:56.321304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"86'5","reported_seq":106,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-08T20:59:17.108440+0000","last_change":"2024-03-08T20:57:49.787818+0000","last_active":"2024-03-08T20:59:17.108440+0000","last_peered":"2024-03-08T20:59:17.108440+0000","last_clean":"2024-03-08T20:59:17.108440+0000","last_became_active":"2024-03-08T20:57:49.787401+0000","last_became_peered":"2024-03-08T20:57:49.787401+0000","last_unstale":"2024-03-08T20:59:17.108440+0000","last_undegraded":"2024-03-08T20:59:17.108440+0000","last_fullsized":"2024-03-08T20:59:17.108440+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_clean_scrub_stamp":"2024-03-08T20:57:48.739829+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T08:23:19.683962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":82,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-08T20:58:21.469537+0000","last_change":"2024-03-08T20:57:50.790922+0000","last_active":"2024-03-08T20:58:21.469537+0000","last_peered":"2024-03-08T20:58:21.469537+0000","last_clean":"2024-03-08T20:58:21.469537+0000","last_became_active":"2024-03-08T20:57:50.789993+0000","last_became_peered":"2024-03-08T20:57:50.789993+0000","last_unstale":"2024-03-08T20:58:21.469537+0000","last_undegraded":"2024-03-08T20:58:21.469537+0000","last_fullsized":"2024-03-08T20:58:21.469537+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T00:27:01.639941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"88'8","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-08T20:59:20.320761+0000","last_change":"2024-03-08T20:58:21.690462+0000","last_active":"2024-03-08T20:59:20.320761+0000","last_peered":"2024-03-08T20:59:20.320761+0000","last_clean":"2024-03-08T20:59:20.320761+0000","last_became_active":"2024-03-08T20:58:21.689310+0000","last_became_peered":"2024-03-08T20:58:21.689310+0000","last_unstale":"2024-03-08T20:59:20.320761+0000","last_undegraded":"2024-03-08T20:59:20.320761+0000","last_fullsized":"2024-03-08T20:59:20.320761+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T04:22:50.128243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"86'5","reported_seq":108,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-08T20:59:19.121582+0000","last_change":"2024-03-08T20:57:49.779056+0000","last_active":"2024-03-08T20:59:19.121582+0000","last_peered":"2024-03-08T20:59:19.121582+0000","last_clean":"2024-03-08T20:59:19.121582+0000","last_became_active":"2024-03-08T20:57:49.778714+0000","last_became_peered":"2024-03-08T20:57:49.778714+0000","last_unstale":"2024-03-08T20:59:19.121582+0000","last_undegraded":"2024-03-08T20:59:19.121582+0000","last_fullsized":"2024-03-08T20:59:19.121582+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_clean_scrub_stamp":"2024-03-08T20:57:48.739829+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T04:05:10.685148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":84,"num_read_kb":84,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-08T20:58:21.685972+0000","last_change":"2024-03-08T20:57:50.812509+0000","last_active":"2024-03-08T20:58:21.685972+0000","last_peered":"2024-03-08T20:58:21.685972+0000","last_clean":"2024-03-08T20:58:21.685972+0000","last_became_active":"2024-03-08T20:57:50.812146+0000","last_became_peered":"2024-03-08T20:57:50.812146+0000","last_unstale":"2024-03-08T20:58:21.685972+0000","last_undegraded":"2024-03-08T20:58:21.685972+0000","last_fullsized":"2024-03-08T20:58:21.685972+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T06:08:07.680098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'6","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-08T20:58:21.466310+0000","last_change":"2024-03-08T20:57:49.774994+0000","last_active":"2024-03-08T20:58:21.466310+0000","last_peered":"2024-03-08T20:58:21.466310+0000","last_clean":"2024-03-08T20:58:21.466310+0000","last_became_active":"2024-03-08T20:57:49.774501+0000","last_became_peered":"2024-03-08T20:57:49.774501+0000","last_unstale":"2024-03-08T20:58:21.466310+0000","last_undegraded":"2024-03-08T20:58:21.466310+0000","last_fullsized":"2024-03-08T20:58:21.466310+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_clean_scrub_stamp":"2024-03-08T20:57:48.739829+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T03:38:35.294082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":352,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-08T20:58:21.460695+0000","last_change":"2024-03-08T20:56:11.724397+0000","last_active":"2024-03-08T20:58:21.460695+0000","last_peered":"2024-03-08T20:58:21.460695+0000","last_clean":"2024-03-08T20:58:21.460695+0000","last_became_active":"2024-03-08T20:56:11.723212+0000","last_became_peered":"2024-03-08T20:56:11.723212+0000","last_unstale":"2024-03-08T20:58:21.460695+0000","last_undegraded":"2024-03-08T20:58:21.460695+0000","last_fullsized":"2024-03-08T20:58:21.460695+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:51:32.833935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:51:32.833935+0000","last_clean_scrub_stamp":"2024-03-08T20:51:32.833935+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T03:31:48.964514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"88'5","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-08T20:59:20.318879+0000","last_change":"2024-03-08T20:57:50.792894+0000","last_active":"2024-03-08T20:59:20.318879+0000","last_peered":"2024-03-08T20:59:20.318879+0000","last_clean":"2024-03-08T20:59:20.318879+0000","last_became_active":"2024-03-08T20:57:50.792471+0000","last_became_peered":"2024-03-08T20:57:50.792471+0000","last_unstale":"2024-03-08T20:59:20.318879+0000","last_undegraded":"2024-03-08T20:59:20.318879+0000","last_fullsized":"2024-03-08T20:59:20.318879+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T00:45:39.696601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'4","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-08T20:58:21.460906+0000","last_change":"2024-03-08T20:57:49.787880+0000","last_active":"2024-03-08T20:58:21.460906+0000","last_peered":"2024-03-08T20:58:21.460906+0000","last_clean":"2024-03-08T20:58:21.460906+0000","last_became_active":"2024-03-08T20:57:49.787450+0000","last_became_peered":"2024-03-08T20:57:49.787450+0000","last_unstale":"2024-03-08T20:58:21.460906+0000","last_undegraded":"2024-03-08T20:58:21.460906+0000","last_fullsized":"2024-03-08T20:58:21.460906+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_clean_scrub_stamp":"2024-03-08T20:57:48.739829+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T04:52:36.895259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"86'4","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-08T20:58:21.473560+0000","last_change":"2024-03-08T20:57:49.787473+0000","last_active":"2024-03-08T20:58:21.473560+0000","last_peered":"2024-03-08T20:58:21.473560+0000","last_clean":"2024-03-08T20:58:21.473560+0000","last_became_active":"2024-03-08T20:57:49.787145+0000","last_became_peered":"2024-03-08T20:57:49.787145+0000","last_unstale":"2024-03-08T20:58:21.473560+0000","last_undegraded":"2024-03-08T20:58:21.473560+0000","last_fullsized":"2024-03-08T20:58:21.473560+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_clean_scrub_stamp":"2024-03-08T20:57:48.739829+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T05:47:14.471850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-08T20:59:20.334977+0000","last_change":"2024-03-08T20:57:50.789546+0000","last_active":"2024-03-08T20:59:20.334977+0000","last_peered":"2024-03-08T20:59:20.334977+0000","last_clean":"2024-03-08T20:59:20.334977+0000","last_became_active":"2024-03-08T20:57:50.789195+0000","last_became_peered":"2024-03-08T20:57:50.789195+0000","last_unstale":"2024-03-08T20:59:20.334977+0000","last_undegraded":"2024-03-08T20:59:20.334977+0000","last_fullsized":"2024-03-08T20:59:20.334977+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T02:43:31.371303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"86'12","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-08T20:58:21.458864+0000","last_change":"2024-03-08T20:57:49.787907+0000","last_active":"2024-03-08T20:58:21.458864+0000","last_peered":"2024-03-08T20:58:21.458864+0000","last_clean":"2024-03-08T20:58:21.458864+0000","last_became_active":"2024-03-08T20:57:49.787601+0000","last_became_peered":"2024-03-08T20:57:49.787601+0000","last_unstale":"2024-03-08T20:58:21.458864+0000","last_undegraded":"2024-03-08T20:58:21.458864+0000","last_fullsized":"2024-03-08T20:58:21.458864+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_clean_scrub_stamp":"2024-03-08T20:57:48.739829+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T00:52:09.562020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-08T20:58:21.686073+0000","last_change":"2024-03-08T20:57:50.798773+0000","last_active":"2024-03-08T20:58:21.686073+0000","last_peered":"2024-03-08T20:58:21.686073+0000","last_clean":"2024-03-08T20:58:21.686073+0000","last_became_active":"2024-03-08T20:57:50.798396+0000","last_became_peered":"2024-03-08T20:57:50.798396+0000","last_unstale":"2024-03-08T20:58:21.686073+0000","last_undegraded":"2024-03-08T20:58:21.686073+0000","last_fullsized":"2024-03-08T20:58:21.686073+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T00:26:41.614622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"86'7","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-08T20:58:21.466937+0000","last_change":"2024-03-08T20:57:49.777474+0000","last_active":"2024-03-08T20:58:21.466937+0000","last_peered":"2024-03-08T20:58:21.466937+0000","last_clean":"2024-03-08T20:58:21.466937+0000","last_became_active":"2024-03-08T20:57:49.777084+0000","last_became_peered":"2024-03-08T20:57:49.777084+0000","last_unstale":"2024-03-08T20:58:21.466937+0000","last_undegraded":"2024-03-08T20:58:21.466937+0000","last_fullsized":"2024-03-08T20:58:21.466937+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_clean_scrub_stamp":"2024-03-08T20:57:48.739829+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T05:10:23.032788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-08T20:58:21.466907+0000","last_change":"2024-03-08T20:57:50.805061+0000","last_active":"2024-03-08T20:58:21.466907+0000","last_peered":"2024-03-08T20:58:21.466907+0000","last_clean":"2024-03-08T20:58:21.466907+0000","last_became_active":"2024-03-08T20:57:50.804709+0000","last_became_peered":"2024-03-08T20:57:50.804709+0000","last_unstale":"2024-03-08T20:58:21.466907+0000","last_undegraded":"2024-03-08T20:58:21.466907+0000","last_fullsized":"2024-03-08T20:58:21.466907+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T08:25:34.003449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-08T20:58:21.479347+0000","last_change":"2024-03-08T20:57:49.776677+0000","last_active":"2024-03-08T20:58:21.479347+0000","last_peered":"2024-03-08T20:58:21.479347+0000","last_clean":"2024-03-08T20:58:21.479347+0000","last_became_active":"2024-03-08T20:57:49.776296+0000","last_became_peered":"2024-03-08T20:57:49.776296+0000","last_unstale":"2024-03-08T20:58:21.479347+0000","last_undegraded":"2024-03-08T20:58:21.479347+0000","last_fullsized":"2024-03-08T20:58:21.479347+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_clean_scrub_stamp":"2024-03-08T20:57:48.739829+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T07:49:46.042037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-08T20:58:21.471498+0000","last_change":"2024-03-08T20:57:50.806141+0000","last_active":"2024-03-08T20:58:21.471498+0000","last_peered":"2024-03-08T20:58:21.471498+0000","last_clean":"2024-03-08T20:58:21.471498+0000","last_became_active":"2024-03-08T20:57:50.805026+0000","last_became_peered":"2024-03-08T20:57:50.805026+0000","last_unstale":"2024-03-08T20:58:21.471498+0000","last_undegraded":"2024-03-08T20:58:21.471498+0000","last_fullsized":"2024-03-08T20:58:21.471498+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T03:49:04.162888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"86'7","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-08T20:58:21.685648+0000","last_change":"2024-03-08T20:57:49.779675+0000","last_active":"2024-03-08T20:58:21.685648+0000","last_peered":"2024-03-08T20:58:21.685648+0000","last_clean":"2024-03-08T20:58:21.685648+0000","last_became_active":"2024-03-08T20:57:49.774270+0000","last_became_peered":"2024-03-08T20:57:49.774270+0000","last_unstale":"2024-03-08T20:58:21.685648+0000","last_undegraded":"2024-03-08T20:58:21.685648+0000","last_fullsized":"2024-03-08T20:58:21.685648+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_clean_scrub_stamp":"2024-03-08T20:57:48.739829+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T02:19:09.992649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-08T20:58:21.459535+0000","last_change":"2024-03-08T20:57:50.797461+0000","last_active":"2024-03-08T20:58:21.459535+0000","last_peered":"2024-03-08T20:58:21.459535+0000","last_clean":"2024-03-08T20:58:21.459535+0000","last_became_active":"2024-03-08T20:57:50.796928+0000","last_became_peered":"2024-03-08T20:57:50.796928+0000","last_unstale":"2024-03-08T20:58:21.459535+0000","last_undegraded":"2024-03-08T20:58:21.459535+0000","last_fullsized":"2024-03-08T20:58:21.459535+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T04:01:44.009461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"86'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-08T20:58:21.468247+0000","last_change":"2024-03-08T20:57:49.794659+0000","last_active":"2024-03-08T20:58:21.468247+0000","last_peered":"2024-03-08T20:58:21.468247+0000","last_clean":"2024-03-08T20:58:21.468247+0000","last_became_active":"2024-03-08T20:57:49.792426+0000","last_became_peered":"2024-03-08T20:57:49.792426+0000","last_unstale":"2024-03-08T20:58:21.468247+0000","last_undegraded":"2024-03-08T20:58:21.468247+0000","last_fullsized":"2024-03-08T20:58:21.468247+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_clean_scrub_stamp":"2024-03-08T20:57:48.739829+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T01:30:41.123131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-08T20:58:21.686046+0000","last_change":"2024-03-08T20:57:50.798691+0000","last_active":"2024-03-08T20:58:21.686046+0000","last_peered":"2024-03-08T20:58:21.686046+0000","last_clean":"2024-03-08T20:58:21.686046+0000","last_became_active":"2024-03-08T20:57:50.797876+0000","last_became_peered":"2024-03-08T20:57:50.797876+0000","last_unstale":"2024-03-08T20:58:21.686046+0000","last_undegraded":"2024-03-08T20:58:21.686046+0000","last_fullsized":"2024-03-08T20:58:21.686046+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T06:51:11.067375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-08T20:58:21.497089+0000","last_change":"2024-03-08T20:58:21.496411+0000","last_active":"2024-03-08T20:58:21.497089+0000","last_peered":"2024-03-08T20:58:21.497089+0000","last_clean":"2024-03-08T20:58:21.497089+0000","last_became_active":"2024-03-08T20:58:21.481560+0000","last_became_peered":"2024-03-08T20:58:21.481560+0000","last_unstale":"2024-03-08T20:58:21.497089+0000","last_undegraded":"2024-03-08T20:58:21.497089+0000","last_fullsized":"2024-03-08T20:58:21.497089+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_clean_scrub_stamp":"2024-03-08T20:57:48.739829+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T00:05:51.303844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-08T20:58:21.460905+0000","last_change":"2024-03-08T20:57:50.797850+0000","last_active":"2024-03-08T20:58:21.460905+0000","last_peered":"2024-03-08T20:58:21.460905+0000","last_clean":"2024-03-08T20:58:21.460905+0000","last_became_active":"2024-03-08T20:57:50.797530+0000","last_became_peered":"2024-03-08T20:57:50.797530+0000","last_unstale":"2024-03-08T20:58:21.460905+0000","last_undegraded":"2024-03-08T20:58:21.460905+0000","last_fullsized":"2024-03-08T20:58:21.460905+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T02:12:52.023644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"86'5","reported_seq":80,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-08T20:59:20.662650+0000","last_change":"2024-03-08T20:58:21.490543+0000","last_active":"2024-03-08T20:59:20.662650+0000","last_peered":"2024-03-08T20:59:20.662650+0000","last_clean":"2024-03-08T20:59:20.662650+0000","last_became_active":"2024-03-08T20:58:21.476411+0000","last_became_peered":"2024-03-08T20:58:21.476411+0000","last_unstale":"2024-03-08T20:59:20.662650+0000","last_undegraded":"2024-03-08T20:59:20.662650+0000","last_fullsized":"2024-03-08T20:59:20.662650+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_clean_scrub_stamp":"2024-03-08T20:57:48.739829+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T02:22:29.586738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":61,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":1170,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,0],"acting":[4,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-08T20:58:21.461413+0000","last_change":"2024-03-08T20:57:50.796522+0000","last_active":"2024-03-08T20:58:21.461413+0000","last_peered":"2024-03-08T20:58:21.461413+0000","last_clean":"2024-03-08T20:58:21.461413+0000","last_became_active":"2024-03-08T20:57:50.796061+0000","last_became_peered":"2024-03-08T20:57:50.796061+0000","last_unstale":"2024-03-08T20:58:21.461413+0000","last_undegraded":"2024-03-08T20:58:21.461413+0000","last_fullsized":"2024-03-08T20:58:21.461413+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T21:35:06.609042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"86'3","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-08T20:58:21.472371+0000","last_change":"2024-03-08T20:57:49.778240+0000","last_active":"2024-03-08T20:58:21.472371+0000","last_peered":"2024-03-08T20:58:21.472371+0000","last_clean":"2024-03-08T20:58:21.472371+0000","last_became_active":"2024-03-08T20:57:49.777798+0000","last_became_peered":"2024-03-08T20:57:49.777798+0000","last_unstale":"2024-03-08T20:58:21.472371+0000","last_undegraded":"2024-03-08T20:58:21.472371+0000","last_fullsized":"2024-03-08T20:58:21.472371+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_clean_scrub_stamp":"2024-03-08T20:57:48.739829+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T05:20:24.614659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"86'1","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-08T20:58:21.460341+0000","last_change":"2024-03-08T20:57:50.803435+0000","last_active":"2024-03-08T20:58:21.460341+0000","last_peered":"2024-03-08T20:58:21.460341+0000","last_clean":"2024-03-08T20:58:21.460341+0000","last_became_active":"2024-03-08T20:57:50.803074+0000","last_became_peered":"2024-03-08T20:57:50.803074+0000","last_unstale":"2024-03-08T20:58:21.460341+0000","last_undegraded":"2024-03-08T20:58:21.460341+0000","last_fullsized":"2024-03-08T20:58:21.460341+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T21:52:02.791159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"86'5","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-08T20:58:21.686832+0000","last_change":"2024-03-08T20:57:49.781604+0000","last_active":"2024-03-08T20:58:21.686832+0000","last_peered":"2024-03-08T20:58:21.686832+0000","last_clean":"2024-03-08T20:58:21.686832+0000","last_became_active":"2024-03-08T20:57:49.781274+0000","last_became_peered":"2024-03-08T20:57:49.781274+0000","last_unstale":"2024-03-08T20:58:21.686832+0000","last_undegraded":"2024-03-08T20:58:21.686832+0000","last_fullsized":"2024-03-08T20:58:21.686832+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_clean_scrub_stamp":"2024-03-08T20:57:48.739829+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T22:29:03.420779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-08T20:58:21.461482+0000","last_change":"2024-03-08T20:57:50.800863+0000","last_active":"2024-03-08T20:58:21.461482+0000","last_peered":"2024-03-08T20:58:21.461482+0000","last_clean":"2024-03-08T20:58:21.461482+0000","last_became_active":"2024-03-08T20:57:50.798916+0000","last_became_peered":"2024-03-08T20:57:50.798916+0000","last_unstale":"2024-03-08T20:58:21.461482+0000","last_undegraded":"2024-03-08T20:58:21.461482+0000","last_fullsized":"2024-03-08T20:58:21.461482+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T04:37:06.666245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-08T20:58:21.459703+0000","last_change":"2024-03-08T20:57:50.813502+0000","last_active":"2024-03-08T20:58:21.459703+0000","last_peered":"2024-03-08T20:58:21.459703+0000","last_clean":"2024-03-08T20:58:21.459703+0000","last_became_active":"2024-03-08T20:57:50.813150+0000","last_became_peered":"2024-03-08T20:57:50.813150+0000","last_unstale":"2024-03-08T20:58:21.459703+0000","last_undegraded":"2024-03-08T20:58:21.459703+0000","last_fullsized":"2024-03-08T20:58:21.459703+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T00:01:47.902909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-08T20:58:21.467378+0000","last_change":"2024-03-08T20:57:50.795055+0000","last_active":"2024-03-08T20:58:21.467378+0000","last_peered":"2024-03-08T20:58:21.467378+0000","last_clean":"2024-03-08T20:58:21.467378+0000","last_became_active":"2024-03-08T20:57:50.794559+0000","last_became_peered":"2024-03-08T20:57:50.794559+0000","last_unstale":"2024-03-08T20:58:21.467378+0000","last_undegraded":"2024-03-08T20:58:21.467378+0000","last_fullsized":"2024-03-08T20:58:21.467378+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T08:06:18.990542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-08T20:58:21.467903+0000","last_change":"2024-03-08T20:57:50.784871+0000","last_active":"2024-03-08T20:58:21.467903+0000","last_peered":"2024-03-08T20:58:21.467903+0000","last_clean":"2024-03-08T20:58:21.467903+0000","last_became_active":"2024-03-08T20:57:50.784553+0000","last_became_peered":"2024-03-08T20:57:50.784553+0000","last_unstale":"2024-03-08T20:58:21.467903+0000","last_undegraded":"2024-03-08T20:58:21.467903+0000","last_fullsized":"2024-03-08T20:58:21.467903+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T02:13:13.570078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-08T20:58:21.466294+0000","last_change":"2024-03-08T20:57:50.794051+0000","last_active":"2024-03-08T20:58:21.466294+0000","last_peered":"2024-03-08T20:58:21.466294+0000","last_clean":"2024-03-08T20:58:21.466294+0000","last_became_active":"2024-03-08T20:57:50.793547+0000","last_became_peered":"2024-03-08T20:57:50.793547+0000","last_unstale":"2024-03-08T20:58:21.466294+0000","last_undegraded":"2024-03-08T20:58:21.466294+0000","last_fullsized":"2024-03-08T20:58:21.466294+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T04:13:52.705247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"88'4","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-08T20:59:20.337479+0000","last_change":"2024-03-08T20:57:50.789449+0000","last_active":"2024-03-08T20:59:20.337479+0000","last_peered":"2024-03-08T20:59:20.337479+0000","last_clean":"2024-03-08T20:59:20.337479+0000","last_became_active":"2024-03-08T20:57:50.789106+0000","last_became_peered":"2024-03-08T20:57:50.789106+0000","last_unstale":"2024-03-08T20:59:20.337479+0000","last_undegraded":"2024-03-08T20:59:20.337479+0000","last_fullsized":"2024-03-08T20:59:20.337479+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T02:12:47.198726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-08T20:58:21.460418+0000","last_change":"2024-03-08T20:57:50.814360+0000","last_active":"2024-03-08T20:58:21.460418+0000","last_peered":"2024-03-08T20:58:21.460418+0000","last_clean":"2024-03-08T20:58:21.460418+0000","last_became_active":"2024-03-08T20:57:50.813966+0000","last_became_peered":"2024-03-08T20:57:50.813966+0000","last_unstale":"2024-03-08T20:58:21.460418+0000","last_undegraded":"2024-03-08T20:58:21.460418+0000","last_fullsized":"2024-03-08T20:58:21.460418+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T04:39:10.324938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-08T20:58:21.469596+0000","last_change":"2024-03-08T20:57:50.803658+0000","last_active":"2024-03-08T20:58:21.469596+0000","last_peered":"2024-03-08T20:58:21.469596+0000","last_clean":"2024-03-08T20:58:21.469596+0000","last_became_active":"2024-03-08T20:57:50.797593+0000","last_became_peered":"2024-03-08T20:57:50.797593+0000","last_unstale":"2024-03-08T20:58:21.469596+0000","last_undegraded":"2024-03-08T20:58:21.469596+0000","last_fullsized":"2024-03-08T20:58:21.469596+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T21:45:33.579087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-08T20:58:21.685927+0000","last_change":"2024-03-08T20:57:50.789913+0000","last_active":"2024-03-08T20:58:21.685927+0000","last_peered":"2024-03-08T20:58:21.685927+0000","last_clean":"2024-03-08T20:58:21.685927+0000","last_became_active":"2024-03-08T20:57:50.789492+0000","last_became_peered":"2024-03-08T20:57:50.789492+0000","last_unstale":"2024-03-08T20:58:21.685927+0000","last_undegraded":"2024-03-08T20:58:21.685927+0000","last_fullsized":"2024-03-08T20:58:21.685927+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T01:25:14.329005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"88'6","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-08T20:59:20.325216+0000","last_change":"2024-03-08T20:57:50.801743+0000","last_active":"2024-03-08T20:59:20.325216+0000","last_peered":"2024-03-08T20:59:20.325216+0000","last_clean":"2024-03-08T20:59:20.325216+0000","last_became_active":"2024-03-08T20:57:50.801261+0000","last_became_peered":"2024-03-08T20:57:50.801261+0000","last_unstale":"2024-03-08T20:59:20.325216+0000","last_undegraded":"2024-03-08T20:59:20.325216+0000","last_fullsized":"2024-03-08T20:59:20.325216+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T23:00:15.605070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-08T20:58:21.685451+0000","last_change":"2024-03-08T20:57:50.800759+0000","last_active":"2024-03-08T20:58:21.685451+0000","last_peered":"2024-03-08T20:58:21.685451+0000","last_clean":"2024-03-08T20:58:21.685451+0000","last_became_active":"2024-03-08T20:57:50.798328+0000","last_became_peered":"2024-03-08T20:57:50.798328+0000","last_unstale":"2024-03-08T20:58:21.685451+0000","last_undegraded":"2024-03-08T20:58:21.685451+0000","last_fullsized":"2024-03-08T20:58:21.685451+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T05:43:00.160071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"88'10","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-08T20:59:20.330953+0000","last_change":"2024-03-08T20:57:50.802262+0000","last_active":"2024-03-08T20:59:20.330953+0000","last_peered":"2024-03-08T20:59:20.330953+0000","last_clean":"2024-03-08T20:59:20.330953+0000","last_became_active":"2024-03-08T20:57:50.801786+0000","last_became_peered":"2024-03-08T20:57:50.801786+0000","last_unstale":"2024-03-08T20:59:20.330953+0000","last_undegraded":"2024-03-08T20:59:20.330953+0000","last_fullsized":"2024-03-08T20:59:20.330953+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T00:47:48.400329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"88'11","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-08T20:59:20.338908+0000","last_change":"2024-03-08T20:57:50.790819+0000","last_active":"2024-03-08T20:59:20.338908+0000","last_peered":"2024-03-08T20:59:20.338908+0000","last_clean":"2024-03-08T20:59:20.338908+0000","last_became_active":"2024-03-08T20:57:50.786840+0000","last_became_peered":"2024-03-08T20:57:50.786840+0000","last_unstale":"2024-03-08T20:59:20.338908+0000","last_undegraded":"2024-03-08T20:59:20.338908+0000","last_fullsized":"2024-03-08T20:59:20.338908+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T06:52:19.567245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-08T20:58:21.460163+0000","last_change":"2024-03-08T20:57:50.806693+0000","last_active":"2024-03-08T20:58:21.460163+0000","last_peered":"2024-03-08T20:58:21.460163+0000","last_clean":"2024-03-08T20:58:21.460163+0000","last_became_active":"2024-03-08T20:57:50.806299+0000","last_became_peered":"2024-03-08T20:57:50.806299+0000","last_unstale":"2024-03-08T20:58:21.460163+0000","last_undegraded":"2024-03-08T20:58:21.460163+0000","last_fullsized":"2024-03-08T20:58:21.460163+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T02:04:07.567265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":154,"ondisk_log_size":154,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":280946,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":66445,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":814,"num_read_kb":484,"num_write":237,"num_write_kb":133,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":59563,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":512000,"data_stored":199833,"data_compressed":13306,"data_compressed_allocated":20480,"data_compressed_original":73728,"omap_allocated":0,"internal_metadata":0},"log_size":206,"ondisk_log_size":206,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351010,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32128,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713024,"statfs":{"total":95995035648,"available":95962136576,"internally_reserved":0,"allocated":966656,"data_stored":461426,"data_compressed":5805,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612649,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32264,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712888,"statfs":{"total":95995035648,"available":95961997312,"internally_reserved":0,"allocated":1114112,"data_stored":1112456,"data_compressed":43393,"data_compressed_allocated":57344,"data_compressed_original":614400,"omap_allocated":1599,"internal_metadata":31914433},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776176,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32148,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713004,"statfs":{"total":95995035648,"available":95962116096,"internally_reserved":0,"allocated":987136,"data_stored":461334,"data_compressed":5805,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037815,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":978944,"data_stored":461690,"data_compressed":5805,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299453,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1036288,"data_stored":1053839,"data_compressed":41539,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462981,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32132,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713020,"statfs":{"total":95995035648,"available":95962132480,"internally_reserved":0,"allocated":970752,"data_stored":461051,"data_compressed":5805,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574951,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1028096,"data_stored":521987,"data_compressed":7659,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738478,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32148,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713004,"statfs":{"total":95995035648,"available":95962116096,"internally_reserved":0,"allocated":991232,"data_stored":462390,"data_compressed":5805,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411425,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":974848,"data_stored":461014,"data_compressed":5805,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182489,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32148,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713004,"statfs":{"total":95995035648,"available":95962116096,"internally_reserved":0,"allocated":987136,"data_stored":460876,"data_compressed":5805,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920852,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32156,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712996,"statfs":{"total":95995035648,"available":95962107904,"internally_reserved":0,"allocated":1003520,"data_stored":522354,"data_compressed":15403,"data_compressed_allocated":20480,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724621,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32200,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712952,"statfs":{"total":95995035648,"available":95962062848,"internally_reserved":0,"allocated":1048576,"data_stored":1053517,"data_compressed":41539,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34092,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34092,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34092,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35734,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35734,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35734,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35734,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35734,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":62024,"data_compressed":1854,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":802,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":62142,"data_compressed":9598,"data_compressed_allocated":12288,"data_compressed_original":57344,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":61759,"data_compressed":1854,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-08T20:59:24.042 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761 shell --fsid dc20f0f8-dd8c-11ee-95c6-87774f69a715 -- ceph pg dump --format=json 2024-03-08T20:59:24.525 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/mon.a/config 2024-03-08T20:59:24.765 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:59:24 smithi149 ceph-mon[35042]: from='client.14961 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-08T20:59:24.878 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:59:24 smithi204 ceph-mon[34935]: from='client.14961 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-08T20:59:24.898 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:59:24 smithi107 ceph-mon[31381]: from='client.14961 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-08T20:59:25.765 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:59:25 smithi149 ceph-mon[35042]: pgmap v410: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 255 B/s wr, 10 op/s 2024-03-08T20:59:25.878 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:59:25 smithi204 ceph-mon[34935]: pgmap v410: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 255 B/s wr, 10 op/s 2024-03-08T20:59:25.967 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:59:25 smithi107 ceph-mon[31381]: pgmap v410: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 255 B/s wr, 10 op/s 2024-03-08T20:59:26.869 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:26.869+0000 7fad92800700 1 -- 172.21.15.107:0/1953693944 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fad8c074010 msgr2=0x7fad8c0743f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:59:26.869 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:26.869+0000 7fad92800700 1 --2- 172.21.15.107:0/1953693944 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fad8c074010 0x7fad8c0743f0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fad7c009990 tx=0x7fad7c009ca0 comp rx=0 tx=0).stop 2024-03-08T20:59:26.869 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:26.869+0000 7fad92800700 1 -- 172.21.15.107:0/1953693944 shutdown_connections 2024-03-08T20:59:26.869 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:26.869+0000 7fad92800700 1 --2- 172.21.15.107:0/1953693944 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fad8c10e7e0 0x7fad8c110bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:26.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:26.869+0000 7fad92800700 1 --2- 172.21.15.107:0/1953693944 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fad8c074010 0x7fad8c0743f0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:26.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:26.869+0000 7fad92800700 1 --2- 172.21.15.107:0/1953693944 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fad8c074930 0x7fad8c10e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:26.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:26.869+0000 7fad92800700 1 -- 172.21.15.107:0/1953693944 >> 172.21.15.107:0/1953693944 conn(0x7fad8c078f10 msgr2=0x7fad8c079320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:59:26.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:26.869+0000 7fad92800700 1 -- 172.21.15.107:0/1953693944 shutdown_connections 2024-03-08T20:59:26.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:26.870+0000 7fad92800700 1 -- 172.21.15.107:0/1953693944 wait complete. 2024-03-08T20:59:26.871 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:26.871+0000 7fad92800700 1 Processor -- start 2024-03-08T20:59:26.871 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:26.871+0000 7fad92800700 1 -- start start 2024-03-08T20:59:26.871 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:26.871+0000 7fad92800700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fad8c074010 0x7fad8c19a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:26.871 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:26.871+0000 7fad92800700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fad8c074930 0x7fad8c19ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:26.871 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:26.871+0000 7fad92800700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fad8c10e7e0 0x7fad8c19efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:26.871 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:26.871+0000 7fad92800700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad8c1141b0 con 0x7fad8c074930 2024-03-08T20:59:26.871 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:26.871+0000 7fad92800700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad8c114030 con 0x7fad8c074010 2024-03-08T20:59:26.871 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:26.871+0000 7fad92800700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad8c114330 con 0x7fad8c10e7e0 2024-03-08T20:59:26.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:26.871+0000 7fad8aa00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fad8c074930 0x7fad8c19ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:59:26.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:26.871+0000 7fad8aa00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fad8c074930 0x7fad8c19ac20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:55226/0 (socket says 172.21.15.107:55226) 2024-03-08T20:59:26.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:26.871+0000 7fad8aa00700 1 -- 172.21.15.107:0/3254285048 learned_addr learned my addr 172.21.15.107:0/3254285048 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T20:59:26.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:26.872+0000 7fad8b400700 1 --2- 172.21.15.107:0/3254285048 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fad8c074010 0x7fad8c19a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:59:26.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:26.872+0000 7fad8aa00700 1 -- 172.21.15.107:0/3254285048 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fad8c10e7e0 msgr2=0x7fad8c19efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:59:26.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:26.872+0000 7fad8be00700 1 --2- 172.21.15.107:0/3254285048 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fad8c10e7e0 0x7fad8c19efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:59:26.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:26.872+0000 7fad8aa00700 1 --2- 172.21.15.107:0/3254285048 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fad8c10e7e0 0x7fad8c19efb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:26.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:26.872+0000 7fad8aa00700 1 -- 172.21.15.107:0/3254285048 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fad8c074010 msgr2=0x7fad8c19a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:59:26.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:26.872+0000 7fad8aa00700 1 --2- 172.21.15.107:0/3254285048 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fad8c074010 0x7fad8c19a6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:26.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:26.872+0000 7fad8aa00700 1 -- 172.21.15.107:0/3254285048 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad8c19f6f0 con 0x7fad8c074930 2024-03-08T20:59:26.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:26.872+0000 7fad8b400700 1 --2- 172.21.15.107:0/3254285048 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fad8c074010 0x7fad8c19a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T20:59:26.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:26.872+0000 7fad8aa00700 1 --2- 172.21.15.107:0/3254285048 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fad8c074930 0x7fad8c19ac20 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fad7800d950 tx=0x7fad7800dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:59:26.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:26.872+0000 7fad83e00700 1 -- 172.21.15.107:0/3254285048 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad78016070 con 0x7fad8c074930 2024-03-08T20:59:26.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:26.872+0000 7fad92800700 1 -- 172.21.15.107:0/3254285048 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad8c19f9e0 con 0x7fad8c074930 2024-03-08T20:59:26.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:26.872+0000 7fad83e00700 1 -- 172.21.15.107:0/3254285048 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fad78010460 con 0x7fad8c074930 2024-03-08T20:59:26.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:26.873+0000 7fad83e00700 1 -- 172.21.15.107:0/3254285048 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad78020060 con 0x7fad8c074930 2024-03-08T20:59:26.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:26.873+0000 7fad92800700 1 -- 172.21.15.107:0/3254285048 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad8c1a7260 con 0x7fad8c074930 2024-03-08T20:59:26.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:26.874+0000 7fad92800700 1 -- 172.21.15.107:0/3254285048 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad8c068aa0 con 0x7fad8c074930 2024-03-08T20:59:26.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:26.876+0000 7fad83e00700 1 -- 172.21.15.107:0/3254285048 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) v1 ==== 137291+0+0 (secure 0 0 0) 0x7fad78019070 con 0x7fad8c074930 2024-03-08T20:59:26.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:26.878+0000 7fad83e00700 1 --2- 172.21.15.107:0/3254285048 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fad700a2890 0x7fad700a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:26.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:26.879+0000 7fad83e00700 1 -- 172.21.15.107:0/3254285048 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fad780cea80 con 0x7fad8c074930 2024-03-08T20:59:26.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:26.879+0000 7fad8b400700 1 --2- 172.21.15.107:0/3254285048 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fad700a2890 0x7fad700a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:59:26.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:26.879+0000 7fad8b400700 1 --2- 172.21.15.107:0/3254285048 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fad700a2890 0x7fad700a4d50 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fad7c00b5c0 tx=0x7fad7c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:59:26.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:26.882+0000 7fad83e00700 1 -- 172.21.15.107:0/3254285048 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7fad7809bae0 con 0x7fad8c074930 2024-03-08T20:59:27.080 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:27.098+0000 7fad92800700 1 -- 172.21.15.107:0/3254285048 --> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fad8c062220 con 0x7fad700a2890 2024-03-08T20:59:27.094 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:27.111+0000 7fad83e00700 1 -- 172.21.15.107:0/3254285048 <== mgr.14152 v2:172.21.15.107:6800/3376636618 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+259305 (secure 0 0 0) 0x7fad8c062220 con 0x7fad700a2890 2024-03-08T20:59:27.095 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-08T20:59:27.102 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:27.118+0000 7fad92800700 1 -- 172.21.15.107:0/3254285048 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fad700a2890 msgr2=0x7fad700a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:59:27.102 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:27.118+0000 7fad92800700 1 --2- 172.21.15.107:0/3254285048 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fad700a2890 0x7fad700a4d50 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fad7c00b5c0 tx=0x7fad7c005fb0 comp rx=0 tx=0).stop 2024-03-08T20:59:27.103 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:27.118+0000 7fad92800700 1 -- 172.21.15.107:0/3254285048 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fad8c074930 msgr2=0x7fad8c19ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:59:27.103 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:27.118+0000 7fad92800700 1 --2- 172.21.15.107:0/3254285048 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fad8c074930 0x7fad8c19ac20 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fad7800d950 tx=0x7fad7800dd10 comp rx=0 tx=0).stop 2024-03-08T20:59:27.103 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:27.118+0000 7fad92800700 1 -- 172.21.15.107:0/3254285048 shutdown_connections 2024-03-08T20:59:27.103 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:27.118+0000 7fad92800700 1 --2- 172.21.15.107:0/3254285048 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fad8c10e7e0 0x7fad8c19efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:27.103 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:27.118+0000 7fad92800700 1 --2- 172.21.15.107:0/3254285048 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fad700a2890 0x7fad700a4d50 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:27.103 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:27.118+0000 7fad92800700 1 --2- 172.21.15.107:0/3254285048 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fad8c074010 0x7fad8c19a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:27.103 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:27.118+0000 7fad92800700 1 --2- 172.21.15.107:0/3254285048 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fad8c074930 0x7fad8c19ac20 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:27.103 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:27.118+0000 7fad92800700 1 -- 172.21.15.107:0/3254285048 >> 172.21.15.107:0/3254285048 conn(0x7fad8c078f10 msgr2=0x7fad8c0ff010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:59:27.103 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:27.119+0000 7fad92800700 1 -- 172.21.15.107:0/3254285048 shutdown_connections 2024-03-08T20:59:27.103 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:27.119+0000 7fad92800700 1 -- 172.21.15.107:0/3254285048 wait complete. 2024-03-08T20:59:27.104 INFO:teuthology.orchestra.run.smithi107.stderr:dumped all 2024-03-08T20:59:27.765 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:59:27 smithi149 ceph-mon[35042]: pgmap v411: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 255 B/s wr, 10 op/s 2024-03-08T20:59:27.859 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:59:27 smithi107 ceph-mon[31381]: pgmap v411: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 255 B/s wr, 10 op/s 2024-03-08T20:59:27.878 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:59:27 smithi204 ceph-mon[34935]: pgmap v411: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 255 B/s wr, 10 op/s 2024-03-08T20:59:27.931 INFO:teuthology.orchestra.run.smithi107.stdout:{"pg_ready":true,"pg_map":{"version":411,"stamp":"2024-03-08T20:59:26.350961+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":657296,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":991,"num_read_kb":629,"num_write":425,"num_write_kb":3076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":1240299,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":849,"ondisk_log_size":849,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":369276,"kb_used_data":11804,"kb_used_omap":18,"kb_used_meta":357357,"kb_avail":1124572548,"statfs":{"total":1151940427776,"available":1151562289152,"internally_reserved":0,"allocated":12087296,"data_stored":7494432,"data_compressed":190168,"data_compressed_allocated":253952,"data_compressed_original":2039808,"omap_allocated":19138,"internal_metadata":365933886},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":39,"apply_latency_ms":39,"commit_latency_ns":39000000,"apply_latency_ns":39000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":249,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":116,"num_read_kb":61,"num_write":5,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005470"},"pg_stats":[{"pgid":"3.3f","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-08T20:59:20.338374+0000","last_change":"2024-03-08T20:57:50.800421+0000","last_active":"2024-03-08T20:59:20.338374+0000","last_peered":"2024-03-08T20:59:20.338374+0000","last_clean":"2024-03-08T20:59:20.338374+0000","last_became_active":"2024-03-08T20:57:50.799682+0000","last_became_peered":"2024-03-08T20:57:50.799682+0000","last_unstale":"2024-03-08T20:59:20.338374+0000","last_undegraded":"2024-03-08T20:59:20.338374+0000","last_fullsized":"2024-03-08T20:59:20.338374+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T07:25:11.394329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"88'4","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-08T20:59:20.315777+0000","last_change":"2024-03-08T20:57:50.800332+0000","last_active":"2024-03-08T20:59:20.315777+0000","last_peered":"2024-03-08T20:59:20.315777+0000","last_clean":"2024-03-08T20:59:20.315777+0000","last_became_active":"2024-03-08T20:57:50.799981+0000","last_became_peered":"2024-03-08T20:57:50.799981+0000","last_unstale":"2024-03-08T20:59:20.315777+0000","last_undegraded":"2024-03-08T20:59:20.315777+0000","last_fullsized":"2024-03-08T20:59:20.315777+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T06:08:19.618115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-08T20:59:20.324775+0000","last_change":"2024-03-08T20:57:50.800982+0000","last_active":"2024-03-08T20:59:20.324775+0000","last_peered":"2024-03-08T20:59:20.324775+0000","last_clean":"2024-03-08T20:59:20.324775+0000","last_became_active":"2024-03-08T20:57:50.800661+0000","last_became_peered":"2024-03-08T20:57:50.800661+0000","last_unstale":"2024-03-08T20:59:20.324775+0000","last_undegraded":"2024-03-08T20:59:20.324775+0000","last_fullsized":"2024-03-08T20:59:20.324775+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T22:33:08.869516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"88'8","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-08T20:59:20.314115+0000","last_change":"2024-03-08T20:58:21.481750+0000","last_active":"2024-03-08T20:59:20.314115+0000","last_peered":"2024-03-08T20:59:20.314115+0000","last_clean":"2024-03-08T20:59:20.314115+0000","last_became_active":"2024-03-08T20:58:21.480626+0000","last_became_peered":"2024-03-08T20:58:21.480626+0000","last_unstale":"2024-03-08T20:59:20.314115+0000","last_undegraded":"2024-03-08T20:59:20.314115+0000","last_fullsized":"2024-03-08T20:59:20.314115+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T03:51:34.944508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"88'4","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-08T20:59:20.318816+0000","last_change":"2024-03-08T20:58:21.691140+0000","last_active":"2024-03-08T20:59:20.318816+0000","last_peered":"2024-03-08T20:59:20.318816+0000","last_clean":"2024-03-08T20:59:20.318816+0000","last_became_active":"2024-03-08T20:58:21.689521+0000","last_became_peered":"2024-03-08T20:58:21.689521+0000","last_unstale":"2024-03-08T20:59:20.318816+0000","last_undegraded":"2024-03-08T20:59:20.318816+0000","last_fullsized":"2024-03-08T20:59:20.318816+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T23:15:52.861725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6],"acting":[8,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"88'6","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-08T20:59:20.319910+0000","last_change":"2024-03-08T20:57:50.789245+0000","last_active":"2024-03-08T20:59:20.319910+0000","last_peered":"2024-03-08T20:59:20.319910+0000","last_clean":"2024-03-08T20:59:20.319910+0000","last_became_active":"2024-03-08T20:57:50.788921+0000","last_became_peered":"2024-03-08T20:57:50.788921+0000","last_unstale":"2024-03-08T20:59:20.319910+0000","last_undegraded":"2024-03-08T20:59:20.319910+0000","last_fullsized":"2024-03-08T20:59:20.319910+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T08:57:30.368714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"88'5","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-08T20:59:20.326074+0000","last_change":"2024-03-08T20:57:50.800725+0000","last_active":"2024-03-08T20:59:20.326074+0000","last_peered":"2024-03-08T20:59:20.326074+0000","last_clean":"2024-03-08T20:59:20.326074+0000","last_became_active":"2024-03-08T20:57:50.800317+0000","last_became_peered":"2024-03-08T20:57:50.800317+0000","last_unstale":"2024-03-08T20:59:20.326074+0000","last_undegraded":"2024-03-08T20:59:20.326074+0000","last_fullsized":"2024-03-08T20:59:20.326074+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T05:44:01.402547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-08T20:59:20.323789+0000","last_change":"2024-03-08T20:57:50.801543+0000","last_active":"2024-03-08T20:59:20.323789+0000","last_peered":"2024-03-08T20:59:20.323789+0000","last_clean":"2024-03-08T20:59:20.323789+0000","last_became_active":"2024-03-08T20:57:50.801111+0000","last_became_peered":"2024-03-08T20:57:50.801111+0000","last_unstale":"2024-03-08T20:59:20.323789+0000","last_undegraded":"2024-03-08T20:59:20.323789+0000","last_fullsized":"2024-03-08T20:59:20.323789+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T06:32:55.234372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"88'11","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-08T20:59:20.324538+0000","last_change":"2024-03-08T20:57:50.798169+0000","last_active":"2024-03-08T20:59:20.324538+0000","last_peered":"2024-03-08T20:59:20.324538+0000","last_clean":"2024-03-08T20:59:20.324538+0000","last_became_active":"2024-03-08T20:57:50.797817+0000","last_became_peered":"2024-03-08T20:57:50.797817+0000","last_unstale":"2024-03-08T20:59:20.324538+0000","last_undegraded":"2024-03-08T20:59:20.324538+0000","last_fullsized":"2024-03-08T20:59:20.324538+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T04:01:46.179415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"88'16","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-08T20:59:20.320182+0000","last_change":"2024-03-08T20:57:50.790757+0000","last_active":"2024-03-08T20:59:20.320182+0000","last_peered":"2024-03-08T20:59:20.320182+0000","last_clean":"2024-03-08T20:59:20.320182+0000","last_became_active":"2024-03-08T20:57:50.790219+0000","last_became_peered":"2024-03-08T20:57:50.790219+0000","last_unstale":"2024-03-08T20:59:20.320182+0000","last_undegraded":"2024-03-08T20:59:20.320182+0000","last_fullsized":"2024-03-08T20:59:20.320182+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T07:55:54.945727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"88'6","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-08T20:59:20.330255+0000","last_change":"2024-03-08T20:57:50.809102+0000","last_active":"2024-03-08T20:59:20.330255+0000","last_peered":"2024-03-08T20:59:20.330255+0000","last_clean":"2024-03-08T20:59:20.330255+0000","last_became_active":"2024-03-08T20:57:50.808647+0000","last_became_peered":"2024-03-08T20:57:50.808647+0000","last_unstale":"2024-03-08T20:59:20.330255+0000","last_undegraded":"2024-03-08T20:59:20.330255+0000","last_fullsized":"2024-03-08T20:59:20.330255+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T00:29:26.317611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-08T20:59:20.322508+0000","last_change":"2024-03-08T20:57:50.791791+0000","last_active":"2024-03-08T20:59:20.322508+0000","last_peered":"2024-03-08T20:59:20.322508+0000","last_clean":"2024-03-08T20:59:20.322508+0000","last_became_active":"2024-03-08T20:57:50.790264+0000","last_became_peered":"2024-03-08T20:57:50.790264+0000","last_unstale":"2024-03-08T20:59:20.322508+0000","last_undegraded":"2024-03-08T20:59:20.322508+0000","last_fullsized":"2024-03-08T20:59:20.322508+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T08:54:30.221330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"88'10","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-08T20:59:20.335640+0000","last_change":"2024-03-08T20:57:50.793990+0000","last_active":"2024-03-08T20:59:20.335640+0000","last_peered":"2024-03-08T20:59:20.335640+0000","last_clean":"2024-03-08T20:59:20.335640+0000","last_became_active":"2024-03-08T20:57:50.792304+0000","last_became_peered":"2024-03-08T20:57:50.792304+0000","last_unstale":"2024-03-08T20:59:20.335640+0000","last_undegraded":"2024-03-08T20:59:20.335640+0000","last_fullsized":"2024-03-08T20:59:20.335640+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T00:21:57.491218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-08T20:59:20.325375+0000","last_change":"2024-03-08T20:57:50.787621+0000","last_active":"2024-03-08T20:59:20.325375+0000","last_peered":"2024-03-08T20:59:20.325375+0000","last_clean":"2024-03-08T20:59:20.325375+0000","last_became_active":"2024-03-08T20:57:50.786461+0000","last_became_peered":"2024-03-08T20:57:50.786461+0000","last_unstale":"2024-03-08T20:59:20.325375+0000","last_undegraded":"2024-03-08T20:59:20.325375+0000","last_fullsized":"2024-03-08T20:59:20.325375+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T08:35:32.513942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-08T20:59:20.326602+0000","last_change":"2024-03-08T20:57:50.789157+0000","last_active":"2024-03-08T20:59:20.326602+0000","last_peered":"2024-03-08T20:59:20.326602+0000","last_clean":"2024-03-08T20:59:20.326602+0000","last_became_active":"2024-03-08T20:57:50.788828+0000","last_became_peered":"2024-03-08T20:57:50.788828+0000","last_unstale":"2024-03-08T20:59:20.326602+0000","last_undegraded":"2024-03-08T20:59:20.326602+0000","last_fullsized":"2024-03-08T20:59:20.326602+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T05:43:53.398306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-08T20:59:20.337357+0000","last_change":"2024-03-08T20:57:50.796888+0000","last_active":"2024-03-08T20:59:20.337357+0000","last_peered":"2024-03-08T20:59:20.337357+0000","last_clean":"2024-03-08T20:59:20.337357+0000","last_became_active":"2024-03-08T20:57:50.795797+0000","last_became_peered":"2024-03-08T20:57:50.795797+0000","last_unstale":"2024-03-08T20:59:20.337357+0000","last_undegraded":"2024-03-08T20:59:20.337357+0000","last_fullsized":"2024-03-08T20:59:20.337357+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T00:43:53.838877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-08T20:59:20.338380+0000","last_change":"2024-03-08T20:57:50.801849+0000","last_active":"2024-03-08T20:59:20.338380+0000","last_peered":"2024-03-08T20:59:20.338380+0000","last_clean":"2024-03-08T20:59:20.338380+0000","last_became_active":"2024-03-08T20:57:50.801375+0000","last_became_peered":"2024-03-08T20:57:50.801375+0000","last_unstale":"2024-03-08T20:59:20.338380+0000","last_undegraded":"2024-03-08T20:59:20.338380+0000","last_fullsized":"2024-03-08T20:59:20.338380+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T03:16:44.652264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"88'10","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-08T20:59:20.321667+0000","last_change":"2024-03-08T20:57:50.790493+0000","last_active":"2024-03-08T20:59:20.321667+0000","last_peered":"2024-03-08T20:59:20.321667+0000","last_clean":"2024-03-08T20:59:20.321667+0000","last_became_active":"2024-03-08T20:57:50.789074+0000","last_became_peered":"2024-03-08T20:57:50.789074+0000","last_unstale":"2024-03-08T20:59:20.321667+0000","last_undegraded":"2024-03-08T20:59:20.321667+0000","last_fullsized":"2024-03-08T20:59:20.321667+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T05:52:45.700669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-08T20:59:20.320867+0000","last_change":"2024-03-08T20:57:50.794348+0000","last_active":"2024-03-08T20:59:20.320867+0000","last_peered":"2024-03-08T20:59:20.320867+0000","last_clean":"2024-03-08T20:59:20.320867+0000","last_became_active":"2024-03-08T20:57:50.793962+0000","last_became_peered":"2024-03-08T20:57:50.793962+0000","last_unstale":"2024-03-08T20:59:20.320867+0000","last_undegraded":"2024-03-08T20:59:20.320867+0000","last_fullsized":"2024-03-08T20:59:20.320867+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T01:38:05.146148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"86'13","reported_seq":120,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-08T20:59:23.123121+0000","last_change":"2024-03-08T20:57:49.792250+0000","last_active":"2024-03-08T20:59:23.123121+0000","last_peered":"2024-03-08T20:59:23.123121+0000","last_clean":"2024-03-08T20:59:23.123121+0000","last_became_active":"2024-03-08T20:57:49.791863+0000","last_became_peered":"2024-03-08T20:57:49.791863+0000","last_unstale":"2024-03-08T20:59:23.123121+0000","last_undegraded":"2024-03-08T20:59:23.123121+0000","last_fullsized":"2024-03-08T20:59:23.123121+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_clean_scrub_stamp":"2024-03-08T20:57:48.739829+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T05:37:40.942754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":88,"num_read_kb":88,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"88'5","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-08T20:59:20.319546+0000","last_change":"2024-03-08T20:57:50.802789+0000","last_active":"2024-03-08T20:59:20.319546+0000","last_peered":"2024-03-08T20:59:20.319546+0000","last_clean":"2024-03-08T20:59:20.319546+0000","last_became_active":"2024-03-08T20:57:50.802437+0000","last_became_peered":"2024-03-08T20:57:50.802437+0000","last_unstale":"2024-03-08T20:59:20.319546+0000","last_undegraded":"2024-03-08T20:59:20.319546+0000","last_fullsized":"2024-03-08T20:59:20.319546+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T23:47:39.882241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-08T20:58:21.687084+0000","last_change":"2024-03-08T20:57:49.785390+0000","last_active":"2024-03-08T20:58:21.687084+0000","last_peered":"2024-03-08T20:58:21.687084+0000","last_clean":"2024-03-08T20:58:21.687084+0000","last_became_active":"2024-03-08T20:57:49.784506+0000","last_became_peered":"2024-03-08T20:57:49.784506+0000","last_unstale":"2024-03-08T20:58:21.687084+0000","last_undegraded":"2024-03-08T20:58:21.687084+0000","last_fullsized":"2024-03-08T20:58:21.687084+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_clean_scrub_stamp":"2024-03-08T20:57:48.739829+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T03:21:54.331376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"88'6","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-08T20:59:20.337598+0000","last_change":"2024-03-08T20:57:50.796868+0000","last_active":"2024-03-08T20:59:20.337598+0000","last_peered":"2024-03-08T20:59:20.337598+0000","last_clean":"2024-03-08T20:59:20.337598+0000","last_became_active":"2024-03-08T20:57:50.796163+0000","last_became_peered":"2024-03-08T20:57:50.796163+0000","last_unstale":"2024-03-08T20:59:20.337598+0000","last_undegraded":"2024-03-08T20:59:20.337598+0000","last_fullsized":"2024-03-08T20:59:20.337598+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T21:26:46.335292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"86'12","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-08T20:58:21.474019+0000","last_change":"2024-03-08T20:57:49.794940+0000","last_active":"2024-03-08T20:58:21.474019+0000","last_peered":"2024-03-08T20:58:21.474019+0000","last_clean":"2024-03-08T20:58:21.474019+0000","last_became_active":"2024-03-08T20:57:49.794386+0000","last_became_peered":"2024-03-08T20:57:49.794386+0000","last_unstale":"2024-03-08T20:58:21.474019+0000","last_undegraded":"2024-03-08T20:58:21.474019+0000","last_fullsized":"2024-03-08T20:58:21.474019+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_clean_scrub_stamp":"2024-03-08T20:57:48.739829+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T04:21:09.953351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-08T20:59:20.326490+0000","last_change":"2024-03-08T20:57:50.794804+0000","last_active":"2024-03-08T20:59:20.326490+0000","last_peered":"2024-03-08T20:59:20.326490+0000","last_clean":"2024-03-08T20:59:20.326490+0000","last_became_active":"2024-03-08T20:57:50.794380+0000","last_became_peered":"2024-03-08T20:57:50.794380+0000","last_unstale":"2024-03-08T20:59:20.326490+0000","last_undegraded":"2024-03-08T20:59:20.326490+0000","last_fullsized":"2024-03-08T20:59:20.326490+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T05:32:21.015740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"86'5","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-08T20:58:21.476574+0000","last_change":"2024-03-08T20:57:49.790881+0000","last_active":"2024-03-08T20:58:21.476574+0000","last_peered":"2024-03-08T20:58:21.476574+0000","last_clean":"2024-03-08T20:58:21.476574+0000","last_became_active":"2024-03-08T20:57:49.790562+0000","last_became_peered":"2024-03-08T20:57:49.790562+0000","last_unstale":"2024-03-08T20:58:21.476574+0000","last_undegraded":"2024-03-08T20:58:21.476574+0000","last_fullsized":"2024-03-08T20:58:21.476574+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_clean_scrub_stamp":"2024-03-08T20:57:48.739829+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T23:57:40.542756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"88'15","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-08T20:59:20.329105+0000","last_change":"2024-03-08T20:57:50.799021+0000","last_active":"2024-03-08T20:59:20.329105+0000","last_peered":"2024-03-08T20:59:20.329105+0000","last_clean":"2024-03-08T20:59:20.329105+0000","last_became_active":"2024-03-08T20:57:50.795494+0000","last_became_peered":"2024-03-08T20:57:50.795494+0000","last_unstale":"2024-03-08T20:59:20.329105+0000","last_undegraded":"2024-03-08T20:59:20.329105+0000","last_fullsized":"2024-03-08T20:59:20.329105+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T21:26:40.936789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"86'5","reported_seq":114,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-08T20:59:24.138004+0000","last_change":"2024-03-08T20:57:49.791167+0000","last_active":"2024-03-08T20:59:24.138004+0000","last_peered":"2024-03-08T20:59:24.138004+0000","last_clean":"2024-03-08T20:59:24.138004+0000","last_became_active":"2024-03-08T20:57:49.789840+0000","last_became_peered":"2024-03-08T20:57:49.789840+0000","last_unstale":"2024-03-08T20:59:24.138004+0000","last_undegraded":"2024-03-08T20:59:24.138004+0000","last_fullsized":"2024-03-08T20:59:24.138004+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_clean_scrub_stamp":"2024-03-08T20:57:48.739829+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T23:14:51.103583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":90,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-08T20:59:20.317727+0000","last_change":"2024-03-08T20:57:50.812656+0000","last_active":"2024-03-08T20:59:20.317727+0000","last_peered":"2024-03-08T20:59:20.317727+0000","last_clean":"2024-03-08T20:59:20.317727+0000","last_became_active":"2024-03-08T20:57:50.812281+0000","last_became_peered":"2024-03-08T20:57:50.812281+0000","last_unstale":"2024-03-08T20:59:20.317727+0000","last_undegraded":"2024-03-08T20:59:20.317727+0000","last_fullsized":"2024-03-08T20:59:20.317727+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T05:40:20.985798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"86'7","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-08T20:58:21.470056+0000","last_change":"2024-03-08T20:57:49.792874+0000","last_active":"2024-03-08T20:58:21.470056+0000","last_peered":"2024-03-08T20:58:21.470056+0000","last_clean":"2024-03-08T20:58:21.470056+0000","last_became_active":"2024-03-08T20:57:49.792418+0000","last_became_peered":"2024-03-08T20:57:49.792418+0000","last_unstale":"2024-03-08T20:58:21.470056+0000","last_undegraded":"2024-03-08T20:58:21.470056+0000","last_fullsized":"2024-03-08T20:58:21.470056+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_clean_scrub_stamp":"2024-03-08T20:57:48.739829+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T02:23:12.888435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-08T20:59:20.323530+0000","last_change":"2024-03-08T20:57:50.805567+0000","last_active":"2024-03-08T20:59:20.323530+0000","last_peered":"2024-03-08T20:59:20.323530+0000","last_clean":"2024-03-08T20:59:20.323530+0000","last_became_active":"2024-03-08T20:57:50.805120+0000","last_became_peered":"2024-03-08T20:57:50.805120+0000","last_unstale":"2024-03-08T20:59:20.323530+0000","last_undegraded":"2024-03-08T20:59:20.323530+0000","last_fullsized":"2024-03-08T20:59:20.323530+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T06:18:14.297971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"86'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-08T20:58:21.686521+0000","last_change":"2024-03-08T20:57:49.771806+0000","last_active":"2024-03-08T20:58:21.686521+0000","last_peered":"2024-03-08T20:58:21.686521+0000","last_clean":"2024-03-08T20:58:21.686521+0000","last_became_active":"2024-03-08T20:57:49.771447+0000","last_became_peered":"2024-03-08T20:57:49.771447+0000","last_unstale":"2024-03-08T20:58:21.686521+0000","last_undegraded":"2024-03-08T20:58:21.686521+0000","last_fullsized":"2024-03-08T20:58:21.686521+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_clean_scrub_stamp":"2024-03-08T20:57:48.739829+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T03:16:37.913599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-08T20:59:20.334823+0000","last_change":"2024-03-08T20:57:50.792915+0000","last_active":"2024-03-08T20:59:20.334823+0000","last_peered":"2024-03-08T20:59:20.334823+0000","last_clean":"2024-03-08T20:59:20.334823+0000","last_became_active":"2024-03-08T20:57:50.792617+0000","last_became_peered":"2024-03-08T20:57:50.792617+0000","last_unstale":"2024-03-08T20:59:20.334823+0000","last_undegraded":"2024-03-08T20:59:20.334823+0000","last_fullsized":"2024-03-08T20:59:20.334823+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T01:59:30.427751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"86'3","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-08T20:58:21.469835+0000","last_change":"2024-03-08T20:57:49.789383+0000","last_active":"2024-03-08T20:58:21.469835+0000","last_peered":"2024-03-08T20:58:21.469835+0000","last_clean":"2024-03-08T20:58:21.469835+0000","last_became_active":"2024-03-08T20:57:49.788975+0000","last_became_peered":"2024-03-08T20:57:49.788975+0000","last_unstale":"2024-03-08T20:58:21.469835+0000","last_undegraded":"2024-03-08T20:58:21.469835+0000","last_fullsized":"2024-03-08T20:58:21.469835+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_clean_scrub_stamp":"2024-03-08T20:57:48.739829+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T07:42:21.366476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"88'11","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-08T20:59:20.324785+0000","last_change":"2024-03-08T20:57:50.785377+0000","last_active":"2024-03-08T20:59:20.324785+0000","last_peered":"2024-03-08T20:59:20.324785+0000","last_clean":"2024-03-08T20:59:20.324785+0000","last_became_active":"2024-03-08T20:57:50.785067+0000","last_became_peered":"2024-03-08T20:57:50.785067+0000","last_unstale":"2024-03-08T20:59:20.324785+0000","last_undegraded":"2024-03-08T20:59:20.324785+0000","last_fullsized":"2024-03-08T20:59:20.324785+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T02:49:10.388474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"86'4","reported_seq":193,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-08T20:59:22.130977+0000","last_change":"2024-03-08T20:57:49.784190+0000","last_active":"2024-03-08T20:59:22.130977+0000","last_peered":"2024-03-08T20:59:22.130977+0000","last_clean":"2024-03-08T20:59:22.130977+0000","last_became_active":"2024-03-08T20:57:49.783805+0000","last_became_peered":"2024-03-08T20:57:49.783805+0000","last_unstale":"2024-03-08T20:59:22.130977+0000","last_undegraded":"2024-03-08T20:59:22.130977+0000","last_fullsized":"2024-03-08T20:59:22.130977+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_clean_scrub_stamp":"2024-03-08T20:57:48.739829+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T03:16:12.983559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":170,"num_read_kb":170,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"88'10","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-08T20:59:20.326363+0000","last_change":"2024-03-08T20:57:50.795647+0000","last_active":"2024-03-08T20:59:20.326363+0000","last_peered":"2024-03-08T20:59:20.326363+0000","last_clean":"2024-03-08T20:59:20.326363+0000","last_became_active":"2024-03-08T20:57:50.793179+0000","last_became_peered":"2024-03-08T20:57:50.793179+0000","last_unstale":"2024-03-08T20:59:20.326363+0000","last_undegraded":"2024-03-08T20:59:20.326363+0000","last_fullsized":"2024-03-08T20:59:20.326363+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T05:56:14.051601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"86'4","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-08T20:58:21.460488+0000","last_change":"2024-03-08T20:57:49.795093+0000","last_active":"2024-03-08T20:58:21.460488+0000","last_peered":"2024-03-08T20:58:21.460488+0000","last_clean":"2024-03-08T20:58:21.460488+0000","last_became_active":"2024-03-08T20:57:49.794617+0000","last_became_peered":"2024-03-08T20:57:49.794617+0000","last_unstale":"2024-03-08T20:58:21.460488+0000","last_undegraded":"2024-03-08T20:58:21.460488+0000","last_fullsized":"2024-03-08T20:58:21.460488+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_clean_scrub_stamp":"2024-03-08T20:57:48.739829+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T05:56:38.669695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"88'9","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-08T20:59:20.337273+0000","last_change":"2024-03-08T20:58:21.480052+0000","last_active":"2024-03-08T20:59:20.337273+0000","last_peered":"2024-03-08T20:59:20.337273+0000","last_clean":"2024-03-08T20:59:20.337273+0000","last_became_active":"2024-03-08T20:58:21.478856+0000","last_became_peered":"2024-03-08T20:58:21.478856+0000","last_unstale":"2024-03-08T20:59:20.337273+0000","last_undegraded":"2024-03-08T20:59:20.337273+0000","last_fullsized":"2024-03-08T20:59:20.337273+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T02:57:07.905740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,5],"acting":[0,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"86'5","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-08T20:58:21.473105+0000","last_change":"2024-03-08T20:57:49.776991+0000","last_active":"2024-03-08T20:58:21.473105+0000","last_peered":"2024-03-08T20:58:21.473105+0000","last_clean":"2024-03-08T20:58:21.473105+0000","last_became_active":"2024-03-08T20:57:49.776566+0000","last_became_peered":"2024-03-08T20:57:49.776566+0000","last_unstale":"2024-03-08T20:58:21.473105+0000","last_undegraded":"2024-03-08T20:58:21.473105+0000","last_fullsized":"2024-03-08T20:58:21.473105+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_clean_scrub_stamp":"2024-03-08T20:57:48.739829+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T21:02:37.726353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"88'11","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-08T20:59:20.328988+0000","last_change":"2024-03-08T20:57:50.793976+0000","last_active":"2024-03-08T20:59:20.328988+0000","last_peered":"2024-03-08T20:59:20.328988+0000","last_clean":"2024-03-08T20:59:20.328988+0000","last_became_active":"2024-03-08T20:57:50.792472+0000","last_became_peered":"2024-03-08T20:57:50.792472+0000","last_unstale":"2024-03-08T20:59:20.328988+0000","last_undegraded":"2024-03-08T20:59:20.328988+0000","last_fullsized":"2024-03-08T20:59:20.328988+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T06:01:03.360171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"88'35","reported_seq":253,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-08T20:59:22.669815+0000","last_change":"2024-03-08T20:58:21.502531+0000","last_active":"2024-03-08T20:59:22.669815+0000","last_peered":"2024-03-08T20:59:22.669815+0000","last_clean":"2024-03-08T20:59:22.669815+0000","last_became_active":"2024-03-08T20:58:21.483763+0000","last_became_peered":"2024-03-08T20:58:21.483763+0000","last_unstale":"2024-03-08T20:59:22.669815+0000","last_undegraded":"2024-03-08T20:59:22.669815+0000","last_fullsized":"2024-03-08T20:59:22.669815+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_clean_scrub_stamp":"2024-03-08T20:57:48.739829+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T00:16:01.128478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":60272,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":166,"num_read_kb":140,"num_write":37,"num_write_kb":74,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":58371,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"88'6","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-08T20:59:20.327280+0000","last_change":"2024-03-08T20:57:50.800031+0000","last_active":"2024-03-08T20:59:20.327280+0000","last_peered":"2024-03-08T20:59:20.327280+0000","last_clean":"2024-03-08T20:59:20.327280+0000","last_became_active":"2024-03-08T20:57:50.799663+0000","last_became_peered":"2024-03-08T20:57:50.799663+0000","last_unstale":"2024-03-08T20:59:20.327280+0000","last_undegraded":"2024-03-08T20:59:20.327280+0000","last_fullsized":"2024-03-08T20:59:20.327280+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T05:37:50.734330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"86'9","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-08T20:58:21.467186+0000","last_change":"2024-03-08T20:57:49.791884+0000","last_active":"2024-03-08T20:58:21.467186+0000","last_peered":"2024-03-08T20:58:21.467186+0000","last_clean":"2024-03-08T20:58:21.467186+0000","last_became_active":"2024-03-08T20:57:49.789314+0000","last_became_peered":"2024-03-08T20:57:49.789314+0000","last_unstale":"2024-03-08T20:58:21.467186+0000","last_undegraded":"2024-03-08T20:58:21.467186+0000","last_fullsized":"2024-03-08T20:58:21.467186+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_clean_scrub_stamp":"2024-03-08T20:57:48.739829+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T08:44:20.490703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-08T20:59:20.335062+0000","last_change":"2024-03-08T20:57:50.787609+0000","last_active":"2024-03-08T20:59:20.335062+0000","last_peered":"2024-03-08T20:59:20.335062+0000","last_clean":"2024-03-08T20:59:20.335062+0000","last_became_active":"2024-03-08T20:57:50.787171+0000","last_became_peered":"2024-03-08T20:57:50.787171+0000","last_unstale":"2024-03-08T20:59:20.335062+0000","last_undegraded":"2024-03-08T20:59:20.335062+0000","last_fullsized":"2024-03-08T20:59:20.335062+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T05:57:54.023942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"86'5","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-08T20:58:21.470581+0000","last_change":"2024-03-08T20:57:49.785642+0000","last_active":"2024-03-08T20:58:21.470581+0000","last_peered":"2024-03-08T20:58:21.470581+0000","last_clean":"2024-03-08T20:58:21.470581+0000","last_became_active":"2024-03-08T20:57:49.785180+0000","last_became_peered":"2024-03-08T20:57:49.785180+0000","last_unstale":"2024-03-08T20:58:21.470581+0000","last_undegraded":"2024-03-08T20:58:21.470581+0000","last_fullsized":"2024-03-08T20:58:21.470581+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_clean_scrub_stamp":"2024-03-08T20:57:48.739829+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T03:21:03.121253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"88'15","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-08T20:59:20.336545+0000","last_change":"2024-03-08T20:57:50.805179+0000","last_active":"2024-03-08T20:59:20.336545+0000","last_peered":"2024-03-08T20:59:20.336545+0000","last_clean":"2024-03-08T20:59:20.336545+0000","last_became_active":"2024-03-08T20:57:50.804801+0000","last_became_peered":"2024-03-08T20:57:50.804801+0000","last_unstale":"2024-03-08T20:59:20.336545+0000","last_undegraded":"2024-03-08T20:59:20.336545+0000","last_fullsized":"2024-03-08T20:59:20.336545+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T08:35:50.224508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"86'5","reported_seq":110,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-08T20:59:20.125295+0000","last_change":"2024-03-08T20:57:49.787198+0000","last_active":"2024-03-08T20:59:20.125295+0000","last_peered":"2024-03-08T20:59:20.125295+0000","last_clean":"2024-03-08T20:59:20.125295+0000","last_became_active":"2024-03-08T20:57:49.785966+0000","last_became_peered":"2024-03-08T20:57:49.785966+0000","last_unstale":"2024-03-08T20:59:20.125295+0000","last_undegraded":"2024-03-08T20:59:20.125295+0000","last_fullsized":"2024-03-08T20:59:20.125295+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_clean_scrub_stamp":"2024-03-08T20:57:48.739829+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T21:39:03.900857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":87,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"88'6","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-08T20:59:20.325652+0000","last_change":"2024-03-08T20:57:50.802887+0000","last_active":"2024-03-08T20:59:20.325652+0000","last_peered":"2024-03-08T20:59:20.325652+0000","last_clean":"2024-03-08T20:59:20.325652+0000","last_became_active":"2024-03-08T20:57:50.802530+0000","last_became_peered":"2024-03-08T20:57:50.802530+0000","last_unstale":"2024-03-08T20:59:20.325652+0000","last_undegraded":"2024-03-08T20:59:20.325652+0000","last_fullsized":"2024-03-08T20:59:20.325652+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T07:08:25.980812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"86'17","reported_seq":37,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-08T20:58:21.470025+0000","last_change":"2024-03-08T20:57:49.793528+0000","last_active":"2024-03-08T20:58:21.470025+0000","last_peered":"2024-03-08T20:58:21.470025+0000","last_clean":"2024-03-08T20:58:21.470025+0000","last_became_active":"2024-03-08T20:57:49.792509+0000","last_became_peered":"2024-03-08T20:57:49.792509+0000","last_unstale":"2024-03-08T20:58:21.470025+0000","last_undegraded":"2024-03-08T20:58:21.470025+0000","last_fullsized":"2024-03-08T20:58:21.470025+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_clean_scrub_stamp":"2024-03-08T20:57:48.739829+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T03:45:20.912280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"88'6","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-08T20:59:20.312145+0000","last_change":"2024-03-08T20:58:21.484109+0000","last_active":"2024-03-08T20:59:20.312145+0000","last_peered":"2024-03-08T20:59:20.312145+0000","last_clean":"2024-03-08T20:59:20.312145+0000","last_became_active":"2024-03-08T20:58:21.483326+0000","last_became_peered":"2024-03-08T20:58:21.483326+0000","last_unstale":"2024-03-08T20:59:20.312145+0000","last_undegraded":"2024-03-08T20:59:20.312145+0000","last_fullsized":"2024-03-08T20:59:20.312145+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T00:16:20.462107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.f","version":"86'2","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-08T20:58:21.491048+0000","last_change":"2024-03-08T20:58:21.489895+0000","last_active":"2024-03-08T20:58:21.491048+0000","last_peered":"2024-03-08T20:58:21.491048+0000","last_clean":"2024-03-08T20:58:21.491048+0000","last_became_active":"2024-03-08T20:58:21.476612+0000","last_became_peered":"2024-03-08T20:58:21.476612+0000","last_unstale":"2024-03-08T20:58:21.491048+0000","last_undegraded":"2024-03-08T20:58:21.491048+0000","last_fullsized":"2024-03-08T20:58:21.491048+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_clean_scrub_stamp":"2024-03-08T20:57:48.739829+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T21:44:54.967671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"88'5","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-08T20:59:20.318244+0000","last_change":"2024-03-08T20:57:50.814625+0000","last_active":"2024-03-08T20:59:20.318244+0000","last_peered":"2024-03-08T20:59:20.318244+0000","last_clean":"2024-03-08T20:59:20.318244+0000","last_became_active":"2024-03-08T20:57:50.814220+0000","last_became_peered":"2024-03-08T20:57:50.814220+0000","last_unstale":"2024-03-08T20:59:20.318244+0000","last_undegraded":"2024-03-08T20:59:20.318244+0000","last_fullsized":"2024-03-08T20:59:20.318244+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T08:35:57.659511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"86'1","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-08T20:58:21.458294+0000","last_change":"2024-03-08T20:57:49.777922+0000","last_active":"2024-03-08T20:58:21.458294+0000","last_peered":"2024-03-08T20:58:21.458294+0000","last_clean":"2024-03-08T20:58:21.458294+0000","last_became_active":"2024-03-08T20:57:49.775083+0000","last_became_peered":"2024-03-08T20:57:49.775083+0000","last_unstale":"2024-03-08T20:58:21.458294+0000","last_undegraded":"2024-03-08T20:58:21.458294+0000","last_fullsized":"2024-03-08T20:58:21.458294+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_clean_scrub_stamp":"2024-03-08T20:57:48.739829+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T01:55:02.679999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"88'5","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-08T20:59:20.322698+0000","last_change":"2024-03-08T20:57:50.778774+0000","last_active":"2024-03-08T20:59:20.322698+0000","last_peered":"2024-03-08T20:59:20.322698+0000","last_clean":"2024-03-08T20:59:20.322698+0000","last_became_active":"2024-03-08T20:57:50.778194+0000","last_became_peered":"2024-03-08T20:57:50.778194+0000","last_unstale":"2024-03-08T20:59:20.322698+0000","last_undegraded":"2024-03-08T20:59:20.322698+0000","last_fullsized":"2024-03-08T20:59:20.322698+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T21:41:56.321304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"86'5","reported_seq":111,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-08T20:59:22.121998+0000","last_change":"2024-03-08T20:57:49.787818+0000","last_active":"2024-03-08T20:59:22.121998+0000","last_peered":"2024-03-08T20:59:22.121998+0000","last_clean":"2024-03-08T20:59:22.121998+0000","last_became_active":"2024-03-08T20:57:49.787401+0000","last_became_peered":"2024-03-08T20:57:49.787401+0000","last_unstale":"2024-03-08T20:59:22.121998+0000","last_undegraded":"2024-03-08T20:59:22.121998+0000","last_fullsized":"2024-03-08T20:59:22.121998+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_clean_scrub_stamp":"2024-03-08T20:57:48.739829+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T08:23:19.683962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":87,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-08T20:59:20.325200+0000","last_change":"2024-03-08T20:57:50.790922+0000","last_active":"2024-03-08T20:59:20.325200+0000","last_peered":"2024-03-08T20:59:20.325200+0000","last_clean":"2024-03-08T20:59:20.325200+0000","last_became_active":"2024-03-08T20:57:50.789993+0000","last_became_peered":"2024-03-08T20:57:50.789993+0000","last_unstale":"2024-03-08T20:59:20.325200+0000","last_undegraded":"2024-03-08T20:59:20.325200+0000","last_fullsized":"2024-03-08T20:59:20.325200+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T00:27:01.639941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"88'8","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-08T20:59:20.320761+0000","last_change":"2024-03-08T20:58:21.690462+0000","last_active":"2024-03-08T20:59:20.320761+0000","last_peered":"2024-03-08T20:59:20.320761+0000","last_clean":"2024-03-08T20:59:20.320761+0000","last_became_active":"2024-03-08T20:58:21.689310+0000","last_became_peered":"2024-03-08T20:58:21.689310+0000","last_unstale":"2024-03-08T20:59:20.320761+0000","last_undegraded":"2024-03-08T20:59:20.320761+0000","last_fullsized":"2024-03-08T20:59:20.320761+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T04:22:50.128243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"86'5","reported_seq":113,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-08T20:59:24.135496+0000","last_change":"2024-03-08T20:57:49.779056+0000","last_active":"2024-03-08T20:59:24.135496+0000","last_peered":"2024-03-08T20:59:24.135496+0000","last_clean":"2024-03-08T20:59:24.135496+0000","last_became_active":"2024-03-08T20:57:49.778714+0000","last_became_peered":"2024-03-08T20:57:49.778714+0000","last_unstale":"2024-03-08T20:59:24.135496+0000","last_undegraded":"2024-03-08T20:59:24.135496+0000","last_fullsized":"2024-03-08T20:59:24.135496+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_clean_scrub_stamp":"2024-03-08T20:57:48.739829+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T04:05:10.685148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":89,"num_read_kb":89,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-08T20:59:20.331342+0000","last_change":"2024-03-08T20:57:50.812509+0000","last_active":"2024-03-08T20:59:20.331342+0000","last_peered":"2024-03-08T20:59:20.331342+0000","last_clean":"2024-03-08T20:59:20.331342+0000","last_became_active":"2024-03-08T20:57:50.812146+0000","last_became_peered":"2024-03-08T20:57:50.812146+0000","last_unstale":"2024-03-08T20:59:20.331342+0000","last_undegraded":"2024-03-08T20:59:20.331342+0000","last_fullsized":"2024-03-08T20:59:20.331342+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T06:08:07.680098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'6","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-08T20:58:21.466310+0000","last_change":"2024-03-08T20:57:49.774994+0000","last_active":"2024-03-08T20:58:21.466310+0000","last_peered":"2024-03-08T20:58:21.466310+0000","last_clean":"2024-03-08T20:58:21.466310+0000","last_became_active":"2024-03-08T20:57:49.774501+0000","last_became_peered":"2024-03-08T20:57:49.774501+0000","last_unstale":"2024-03-08T20:58:21.466310+0000","last_undegraded":"2024-03-08T20:58:21.466310+0000","last_fullsized":"2024-03-08T20:58:21.466310+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_clean_scrub_stamp":"2024-03-08T20:57:48.739829+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T03:38:35.294082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":352,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-08T20:58:21.460695+0000","last_change":"2024-03-08T20:56:11.724397+0000","last_active":"2024-03-08T20:58:21.460695+0000","last_peered":"2024-03-08T20:58:21.460695+0000","last_clean":"2024-03-08T20:58:21.460695+0000","last_became_active":"2024-03-08T20:56:11.723212+0000","last_became_peered":"2024-03-08T20:56:11.723212+0000","last_unstale":"2024-03-08T20:58:21.460695+0000","last_undegraded":"2024-03-08T20:58:21.460695+0000","last_fullsized":"2024-03-08T20:58:21.460695+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:51:32.833935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:51:32.833935+0000","last_clean_scrub_stamp":"2024-03-08T20:51:32.833935+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T03:31:48.964514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"88'5","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-08T20:59:20.318879+0000","last_change":"2024-03-08T20:57:50.792894+0000","last_active":"2024-03-08T20:59:20.318879+0000","last_peered":"2024-03-08T20:59:20.318879+0000","last_clean":"2024-03-08T20:59:20.318879+0000","last_became_active":"2024-03-08T20:57:50.792471+0000","last_became_peered":"2024-03-08T20:57:50.792471+0000","last_unstale":"2024-03-08T20:59:20.318879+0000","last_undegraded":"2024-03-08T20:59:20.318879+0000","last_fullsized":"2024-03-08T20:59:20.318879+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T00:45:39.696601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'4","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-08T20:58:21.460906+0000","last_change":"2024-03-08T20:57:49.787880+0000","last_active":"2024-03-08T20:58:21.460906+0000","last_peered":"2024-03-08T20:58:21.460906+0000","last_clean":"2024-03-08T20:58:21.460906+0000","last_became_active":"2024-03-08T20:57:49.787450+0000","last_became_peered":"2024-03-08T20:57:49.787450+0000","last_unstale":"2024-03-08T20:58:21.460906+0000","last_undegraded":"2024-03-08T20:58:21.460906+0000","last_fullsized":"2024-03-08T20:58:21.460906+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_clean_scrub_stamp":"2024-03-08T20:57:48.739829+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T04:52:36.895259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"86'4","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-08T20:58:21.473560+0000","last_change":"2024-03-08T20:57:49.787473+0000","last_active":"2024-03-08T20:58:21.473560+0000","last_peered":"2024-03-08T20:58:21.473560+0000","last_clean":"2024-03-08T20:58:21.473560+0000","last_became_active":"2024-03-08T20:57:49.787145+0000","last_became_peered":"2024-03-08T20:57:49.787145+0000","last_unstale":"2024-03-08T20:58:21.473560+0000","last_undegraded":"2024-03-08T20:58:21.473560+0000","last_fullsized":"2024-03-08T20:58:21.473560+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_clean_scrub_stamp":"2024-03-08T20:57:48.739829+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T05:47:14.471850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-08T20:59:20.334977+0000","last_change":"2024-03-08T20:57:50.789546+0000","last_active":"2024-03-08T20:59:20.334977+0000","last_peered":"2024-03-08T20:59:20.334977+0000","last_clean":"2024-03-08T20:59:20.334977+0000","last_became_active":"2024-03-08T20:57:50.789195+0000","last_became_peered":"2024-03-08T20:57:50.789195+0000","last_unstale":"2024-03-08T20:59:20.334977+0000","last_undegraded":"2024-03-08T20:59:20.334977+0000","last_fullsized":"2024-03-08T20:59:20.334977+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T02:43:31.371303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"88'13","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-08T20:58:21.459017+0000","last_change":"2024-03-08T20:57:49.787907+0000","last_active":"2024-03-08T20:58:21.459017+0000","last_peered":"2024-03-08T20:58:21.459017+0000","last_clean":"2024-03-08T20:58:21.459017+0000","last_became_active":"2024-03-08T20:57:49.787601+0000","last_became_peered":"2024-03-08T20:57:49.787601+0000","last_unstale":"2024-03-08T20:58:21.459017+0000","last_undegraded":"2024-03-08T20:58:21.459017+0000","last_fullsized":"2024-03-08T20:58:21.459017+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_clean_scrub_stamp":"2024-03-08T20:57:48.739829+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T00:52:09.562020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"88'5","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-08T20:59:20.328362+0000","last_change":"2024-03-08T20:57:50.798773+0000","last_active":"2024-03-08T20:59:20.328362+0000","last_peered":"2024-03-08T20:59:20.328362+0000","last_clean":"2024-03-08T20:59:20.328362+0000","last_became_active":"2024-03-08T20:57:50.798396+0000","last_became_peered":"2024-03-08T20:57:50.798396+0000","last_unstale":"2024-03-08T20:59:20.328362+0000","last_undegraded":"2024-03-08T20:59:20.328362+0000","last_fullsized":"2024-03-08T20:59:20.328362+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T00:26:41.614622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"86'7","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-08T20:58:21.466937+0000","last_change":"2024-03-08T20:57:49.777474+0000","last_active":"2024-03-08T20:58:21.466937+0000","last_peered":"2024-03-08T20:58:21.466937+0000","last_clean":"2024-03-08T20:58:21.466937+0000","last_became_active":"2024-03-08T20:57:49.777084+0000","last_became_peered":"2024-03-08T20:57:49.777084+0000","last_unstale":"2024-03-08T20:58:21.466937+0000","last_undegraded":"2024-03-08T20:58:21.466937+0000","last_fullsized":"2024-03-08T20:58:21.466937+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_clean_scrub_stamp":"2024-03-08T20:57:48.739829+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T05:10:23.032788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-08T20:59:20.334629+0000","last_change":"2024-03-08T20:57:50.805061+0000","last_active":"2024-03-08T20:59:20.334629+0000","last_peered":"2024-03-08T20:59:20.334629+0000","last_clean":"2024-03-08T20:59:20.334629+0000","last_became_active":"2024-03-08T20:57:50.804709+0000","last_became_peered":"2024-03-08T20:57:50.804709+0000","last_unstale":"2024-03-08T20:59:20.334629+0000","last_undegraded":"2024-03-08T20:59:20.334629+0000","last_fullsized":"2024-03-08T20:59:20.334629+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T08:25:34.003449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-08T20:58:21.479347+0000","last_change":"2024-03-08T20:57:49.776677+0000","last_active":"2024-03-08T20:58:21.479347+0000","last_peered":"2024-03-08T20:58:21.479347+0000","last_clean":"2024-03-08T20:58:21.479347+0000","last_became_active":"2024-03-08T20:57:49.776296+0000","last_became_peered":"2024-03-08T20:57:49.776296+0000","last_unstale":"2024-03-08T20:58:21.479347+0000","last_undegraded":"2024-03-08T20:58:21.479347+0000","last_fullsized":"2024-03-08T20:58:21.479347+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_clean_scrub_stamp":"2024-03-08T20:57:48.739829+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T07:49:46.042037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"88'2","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-08T20:59:20.274063+0000","last_change":"2024-03-08T20:57:50.806141+0000","last_active":"2024-03-08T20:59:20.274063+0000","last_peered":"2024-03-08T20:59:20.274063+0000","last_clean":"2024-03-08T20:59:20.274063+0000","last_became_active":"2024-03-08T20:57:50.805026+0000","last_became_peered":"2024-03-08T20:57:50.805026+0000","last_unstale":"2024-03-08T20:59:20.274063+0000","last_undegraded":"2024-03-08T20:59:20.274063+0000","last_fullsized":"2024-03-08T20:59:20.274063+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T03:49:04.162888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"86'7","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-08T20:58:21.685648+0000","last_change":"2024-03-08T20:57:49.779675+0000","last_active":"2024-03-08T20:58:21.685648+0000","last_peered":"2024-03-08T20:58:21.685648+0000","last_clean":"2024-03-08T20:58:21.685648+0000","last_became_active":"2024-03-08T20:57:49.774270+0000","last_became_peered":"2024-03-08T20:57:49.774270+0000","last_unstale":"2024-03-08T20:58:21.685648+0000","last_undegraded":"2024-03-08T20:58:21.685648+0000","last_fullsized":"2024-03-08T20:58:21.685648+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_clean_scrub_stamp":"2024-03-08T20:57:48.739829+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T02:19:09.992649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-08T20:59:20.336217+0000","last_change":"2024-03-08T20:57:50.797461+0000","last_active":"2024-03-08T20:59:20.336217+0000","last_peered":"2024-03-08T20:59:20.336217+0000","last_clean":"2024-03-08T20:59:20.336217+0000","last_became_active":"2024-03-08T20:57:50.796928+0000","last_became_peered":"2024-03-08T20:57:50.796928+0000","last_unstale":"2024-03-08T20:59:20.336217+0000","last_undegraded":"2024-03-08T20:59:20.336217+0000","last_fullsized":"2024-03-08T20:59:20.336217+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T04:01:44.009461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"86'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-08T20:58:21.468247+0000","last_change":"2024-03-08T20:57:49.794659+0000","last_active":"2024-03-08T20:58:21.468247+0000","last_peered":"2024-03-08T20:58:21.468247+0000","last_clean":"2024-03-08T20:58:21.468247+0000","last_became_active":"2024-03-08T20:57:49.792426+0000","last_became_peered":"2024-03-08T20:57:49.792426+0000","last_unstale":"2024-03-08T20:58:21.468247+0000","last_undegraded":"2024-03-08T20:58:21.468247+0000","last_fullsized":"2024-03-08T20:58:21.468247+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_clean_scrub_stamp":"2024-03-08T20:57:48.739829+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T01:30:41.123131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-08T20:59:20.338074+0000","last_change":"2024-03-08T20:57:50.798691+0000","last_active":"2024-03-08T20:59:20.338074+0000","last_peered":"2024-03-08T20:59:20.338074+0000","last_clean":"2024-03-08T20:59:20.338074+0000","last_became_active":"2024-03-08T20:57:50.797876+0000","last_became_peered":"2024-03-08T20:57:50.797876+0000","last_unstale":"2024-03-08T20:59:20.338074+0000","last_undegraded":"2024-03-08T20:59:20.338074+0000","last_fullsized":"2024-03-08T20:59:20.338074+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T06:51:11.067375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-08T20:58:21.497089+0000","last_change":"2024-03-08T20:58:21.496411+0000","last_active":"2024-03-08T20:58:21.497089+0000","last_peered":"2024-03-08T20:58:21.497089+0000","last_clean":"2024-03-08T20:58:21.497089+0000","last_became_active":"2024-03-08T20:58:21.481560+0000","last_became_peered":"2024-03-08T20:58:21.481560+0000","last_unstale":"2024-03-08T20:58:21.497089+0000","last_undegraded":"2024-03-08T20:58:21.497089+0000","last_fullsized":"2024-03-08T20:58:21.497089+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_clean_scrub_stamp":"2024-03-08T20:57:48.739829+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T00:05:51.303844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"88'5","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-08T20:59:20.336489+0000","last_change":"2024-03-08T20:57:50.797850+0000","last_active":"2024-03-08T20:59:20.336489+0000","last_peered":"2024-03-08T20:59:20.336489+0000","last_clean":"2024-03-08T20:59:20.336489+0000","last_became_active":"2024-03-08T20:57:50.797530+0000","last_became_peered":"2024-03-08T20:57:50.797530+0000","last_unstale":"2024-03-08T20:59:20.336489+0000","last_undegraded":"2024-03-08T20:59:20.336489+0000","last_fullsized":"2024-03-08T20:59:20.336489+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T02:12:52.023644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"86'5","reported_seq":80,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-08T20:59:20.662650+0000","last_change":"2024-03-08T20:58:21.490543+0000","last_active":"2024-03-08T20:59:20.662650+0000","last_peered":"2024-03-08T20:59:20.662650+0000","last_clean":"2024-03-08T20:59:20.662650+0000","last_became_active":"2024-03-08T20:58:21.476411+0000","last_became_peered":"2024-03-08T20:58:21.476411+0000","last_unstale":"2024-03-08T20:59:20.662650+0000","last_undegraded":"2024-03-08T20:59:20.662650+0000","last_fullsized":"2024-03-08T20:59:20.662650+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_clean_scrub_stamp":"2024-03-08T20:57:48.739829+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T02:22:29.586738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":61,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":1170,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,0],"acting":[4,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-08T20:59:20.330114+0000","last_change":"2024-03-08T20:57:50.796522+0000","last_active":"2024-03-08T20:59:20.330114+0000","last_peered":"2024-03-08T20:59:20.330114+0000","last_clean":"2024-03-08T20:59:20.330114+0000","last_became_active":"2024-03-08T20:57:50.796061+0000","last_became_peered":"2024-03-08T20:57:50.796061+0000","last_unstale":"2024-03-08T20:59:20.330114+0000","last_undegraded":"2024-03-08T20:59:20.330114+0000","last_fullsized":"2024-03-08T20:59:20.330114+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T21:35:06.609042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"86'3","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-08T20:58:21.472371+0000","last_change":"2024-03-08T20:57:49.778240+0000","last_active":"2024-03-08T20:58:21.472371+0000","last_peered":"2024-03-08T20:58:21.472371+0000","last_clean":"2024-03-08T20:58:21.472371+0000","last_became_active":"2024-03-08T20:57:49.777798+0000","last_became_peered":"2024-03-08T20:57:49.777798+0000","last_unstale":"2024-03-08T20:58:21.472371+0000","last_undegraded":"2024-03-08T20:58:21.472371+0000","last_fullsized":"2024-03-08T20:58:21.472371+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_clean_scrub_stamp":"2024-03-08T20:57:48.739829+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T05:20:24.614659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-08T20:59:20.325018+0000","last_change":"2024-03-08T20:57:50.803435+0000","last_active":"2024-03-08T20:59:20.325018+0000","last_peered":"2024-03-08T20:59:20.325018+0000","last_clean":"2024-03-08T20:59:20.325018+0000","last_became_active":"2024-03-08T20:57:50.803074+0000","last_became_peered":"2024-03-08T20:57:50.803074+0000","last_unstale":"2024-03-08T20:59:20.325018+0000","last_undegraded":"2024-03-08T20:59:20.325018+0000","last_fullsized":"2024-03-08T20:59:20.325018+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T21:52:02.791159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"86'5","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-08T20:58:21.686832+0000","last_change":"2024-03-08T20:57:49.781604+0000","last_active":"2024-03-08T20:58:21.686832+0000","last_peered":"2024-03-08T20:58:21.686832+0000","last_clean":"2024-03-08T20:58:21.686832+0000","last_became_active":"2024-03-08T20:57:49.781274+0000","last_became_peered":"2024-03-08T20:57:49.781274+0000","last_unstale":"2024-03-08T20:58:21.686832+0000","last_undegraded":"2024-03-08T20:58:21.686832+0000","last_fullsized":"2024-03-08T20:58:21.686832+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:48.739829+0000","last_clean_scrub_stamp":"2024-03-08T20:57:48.739829+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T22:29:03.420779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-08T20:59:20.322534+0000","last_change":"2024-03-08T20:57:50.800863+0000","last_active":"2024-03-08T20:59:20.322534+0000","last_peered":"2024-03-08T20:59:20.322534+0000","last_clean":"2024-03-08T20:59:20.322534+0000","last_became_active":"2024-03-08T20:57:50.798916+0000","last_became_peered":"2024-03-08T20:57:50.798916+0000","last_unstale":"2024-03-08T20:59:20.322534+0000","last_undegraded":"2024-03-08T20:59:20.322534+0000","last_fullsized":"2024-03-08T20:59:20.322534+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T04:37:06.666245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-08T20:59:20.321985+0000","last_change":"2024-03-08T20:57:50.813502+0000","last_active":"2024-03-08T20:59:20.321985+0000","last_peered":"2024-03-08T20:59:20.321985+0000","last_clean":"2024-03-08T20:59:20.321985+0000","last_became_active":"2024-03-08T20:57:50.813150+0000","last_became_peered":"2024-03-08T20:57:50.813150+0000","last_unstale":"2024-03-08T20:59:20.321985+0000","last_undegraded":"2024-03-08T20:59:20.321985+0000","last_fullsized":"2024-03-08T20:59:20.321985+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T00:01:47.902909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"88'12","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-08T20:59:20.319549+0000","last_change":"2024-03-08T20:57:50.795055+0000","last_active":"2024-03-08T20:59:20.319549+0000","last_peered":"2024-03-08T20:59:20.319549+0000","last_clean":"2024-03-08T20:59:20.319549+0000","last_became_active":"2024-03-08T20:57:50.794559+0000","last_became_peered":"2024-03-08T20:57:50.794559+0000","last_unstale":"2024-03-08T20:59:20.319549+0000","last_undegraded":"2024-03-08T20:59:20.319549+0000","last_fullsized":"2024-03-08T20:59:20.319549+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T08:06:18.990542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-08T20:59:20.333976+0000","last_change":"2024-03-08T20:57:50.784871+0000","last_active":"2024-03-08T20:59:20.333976+0000","last_peered":"2024-03-08T20:59:20.333976+0000","last_clean":"2024-03-08T20:59:20.333976+0000","last_became_active":"2024-03-08T20:57:50.784553+0000","last_became_peered":"2024-03-08T20:57:50.784553+0000","last_unstale":"2024-03-08T20:59:20.333976+0000","last_undegraded":"2024-03-08T20:59:20.333976+0000","last_fullsized":"2024-03-08T20:59:20.333976+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T02:13:13.570078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-08T20:59:20.335387+0000","last_change":"2024-03-08T20:57:50.794051+0000","last_active":"2024-03-08T20:59:20.335387+0000","last_peered":"2024-03-08T20:59:20.335387+0000","last_clean":"2024-03-08T20:59:20.335387+0000","last_became_active":"2024-03-08T20:57:50.793547+0000","last_became_peered":"2024-03-08T20:57:50.793547+0000","last_unstale":"2024-03-08T20:59:20.335387+0000","last_undegraded":"2024-03-08T20:59:20.335387+0000","last_fullsized":"2024-03-08T20:59:20.335387+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T04:13:52.705247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"88'4","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-08T20:59:20.337479+0000","last_change":"2024-03-08T20:57:50.789449+0000","last_active":"2024-03-08T20:59:20.337479+0000","last_peered":"2024-03-08T20:59:20.337479+0000","last_clean":"2024-03-08T20:59:20.337479+0000","last_became_active":"2024-03-08T20:57:50.789106+0000","last_became_peered":"2024-03-08T20:57:50.789106+0000","last_unstale":"2024-03-08T20:59:20.337479+0000","last_undegraded":"2024-03-08T20:59:20.337479+0000","last_fullsized":"2024-03-08T20:59:20.337479+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T02:12:47.198726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"88'6","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-08T20:59:20.318244+0000","last_change":"2024-03-08T20:57:50.814360+0000","last_active":"2024-03-08T20:59:20.318244+0000","last_peered":"2024-03-08T20:59:20.318244+0000","last_clean":"2024-03-08T20:59:20.318244+0000","last_became_active":"2024-03-08T20:57:50.813966+0000","last_became_peered":"2024-03-08T20:57:50.813966+0000","last_unstale":"2024-03-08T20:59:20.318244+0000","last_undegraded":"2024-03-08T20:59:20.318244+0000","last_fullsized":"2024-03-08T20:59:20.318244+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T04:39:10.324938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"88'15","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-08T20:59:20.321979+0000","last_change":"2024-03-08T20:57:50.803658+0000","last_active":"2024-03-08T20:59:20.321979+0000","last_peered":"2024-03-08T20:59:20.321979+0000","last_clean":"2024-03-08T20:59:20.321979+0000","last_became_active":"2024-03-08T20:57:50.797593+0000","last_became_peered":"2024-03-08T20:57:50.797593+0000","last_unstale":"2024-03-08T20:59:20.321979+0000","last_undegraded":"2024-03-08T20:59:20.321979+0000","last_fullsized":"2024-03-08T20:59:20.321979+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T21:45:33.579087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"88'2","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-08T20:59:20.302811+0000","last_change":"2024-03-08T20:57:50.789913+0000","last_active":"2024-03-08T20:59:20.302811+0000","last_peered":"2024-03-08T20:59:20.302811+0000","last_clean":"2024-03-08T20:59:20.302811+0000","last_became_active":"2024-03-08T20:57:50.789492+0000","last_became_peered":"2024-03-08T20:57:50.789492+0000","last_unstale":"2024-03-08T20:59:20.302811+0000","last_undegraded":"2024-03-08T20:59:20.302811+0000","last_fullsized":"2024-03-08T20:59:20.302811+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T01:25:14.329005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"88'6","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-08T20:59:20.325216+0000","last_change":"2024-03-08T20:57:50.801743+0000","last_active":"2024-03-08T20:59:20.325216+0000","last_peered":"2024-03-08T20:59:20.325216+0000","last_clean":"2024-03-08T20:59:20.325216+0000","last_became_active":"2024-03-08T20:57:50.801261+0000","last_became_peered":"2024-03-08T20:57:50.801261+0000","last_unstale":"2024-03-08T20:59:20.325216+0000","last_undegraded":"2024-03-08T20:59:20.325216+0000","last_fullsized":"2024-03-08T20:59:20.325216+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-09T23:00:15.605070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-08T20:59:20.316954+0000","last_change":"2024-03-08T20:57:50.800759+0000","last_active":"2024-03-08T20:59:20.316954+0000","last_peered":"2024-03-08T20:59:20.316954+0000","last_clean":"2024-03-08T20:59:20.316954+0000","last_became_active":"2024-03-08T20:57:50.798328+0000","last_became_peered":"2024-03-08T20:57:50.798328+0000","last_unstale":"2024-03-08T20:59:20.316954+0000","last_undegraded":"2024-03-08T20:59:20.316954+0000","last_fullsized":"2024-03-08T20:59:20.316954+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T05:43:00.160071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"88'10","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-08T20:59:20.330953+0000","last_change":"2024-03-08T20:57:50.802262+0000","last_active":"2024-03-08T20:59:20.330953+0000","last_peered":"2024-03-08T20:59:20.330953+0000","last_clean":"2024-03-08T20:59:20.330953+0000","last_became_active":"2024-03-08T20:57:50.801786+0000","last_became_peered":"2024-03-08T20:57:50.801786+0000","last_unstale":"2024-03-08T20:59:20.330953+0000","last_undegraded":"2024-03-08T20:59:20.330953+0000","last_fullsized":"2024-03-08T20:59:20.330953+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T00:47:48.400329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"88'11","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-08T20:59:20.338908+0000","last_change":"2024-03-08T20:57:50.790819+0000","last_active":"2024-03-08T20:59:20.338908+0000","last_peered":"2024-03-08T20:59:20.338908+0000","last_clean":"2024-03-08T20:59:20.338908+0000","last_became_active":"2024-03-08T20:57:50.786840+0000","last_became_peered":"2024-03-08T20:57:50.786840+0000","last_unstale":"2024-03-08T20:59:20.338908+0000","last_undegraded":"2024-03-08T20:59:20.338908+0000","last_fullsized":"2024-03-08T20:59:20.338908+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T06:52:19.567245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-08T20:59:20.310031+0000","last_change":"2024-03-08T20:57:50.806693+0000","last_active":"2024-03-08T20:59:20.310031+0000","last_peered":"2024-03-08T20:59:20.310031+0000","last_clean":"2024-03-08T20:59:20.310031+0000","last_became_active":"2024-03-08T20:57:50.806299+0000","last_became_peered":"2024-03-08T20:57:50.806299+0000","last_unstale":"2024-03-08T20:59:20.310031+0000","last_undegraded":"2024-03-08T20:59:20.310031+0000","last_fullsized":"2024-03-08T20:59:20.310031+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-08T20:57:49.750081+0000","last_clean_scrub_stamp":"2024-03-08T20:57:49.750081+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-10T02:04:07.567265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":280946,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":66694,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":855,"num_read_kb":510,"num_write":242,"num_write_kb":136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":59563,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":512000,"data_stored":200082,"data_compressed":13306,"data_compressed_allocated":20480,"data_compressed_original":73728,"omap_allocated":0,"internal_metadata":0},"log_size":210,"ondisk_log_size":210,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351011,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32120,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713032,"statfs":{"total":95995035648,"available":95962144768,"internally_reserved":0,"allocated":966656,"data_stored":461426,"data_compressed":5805,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612650,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32264,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712888,"statfs":{"total":95995035648,"available":95961997312,"internally_reserved":0,"allocated":1114112,"data_stored":1112705,"data_compressed":43393,"data_compressed_allocated":57344,"data_compressed_original":614400,"omap_allocated":1599,"internal_metadata":31914433},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776177,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32140,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713012,"statfs":{"total":95995035648,"available":95962124288,"internally_reserved":0,"allocated":987136,"data_stored":461334,"data_compressed":5805,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037816,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":978944,"data_stored":461690,"data_compressed":5805,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299454,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1036288,"data_stored":1053839,"data_compressed":41539,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462982,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32124,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713028,"statfs":{"total":95995035648,"available":95962140672,"internally_reserved":0,"allocated":970752,"data_stored":461051,"data_compressed":5805,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574952,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1028096,"data_stored":522236,"data_compressed":7659,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738479,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32144,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713008,"statfs":{"total":95995035648,"available":95962120192,"internally_reserved":0,"allocated":991232,"data_stored":462390,"data_compressed":5805,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411426,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":974848,"data_stored":461014,"data_compressed":5805,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182490,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32140,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713012,"statfs":{"total":95995035648,"available":95962124288,"internally_reserved":0,"allocated":987136,"data_stored":460876,"data_compressed":5805,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920852,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32156,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712996,"statfs":{"total":95995035648,"available":95962107904,"internally_reserved":0,"allocated":1003520,"data_stored":522354,"data_compressed":15403,"data_compressed_allocated":20480,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724622,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32200,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712952,"statfs":{"total":95995035648,"available":95962062848,"internally_reserved":0,"allocated":1048576,"data_stored":1053517,"data_compressed":41539,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34092,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34092,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34092,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35734,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35734,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35734,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35734,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35734,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":62024,"data_compressed":1854,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":802,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":62142,"data_compressed":9598,"data_compressed_allocated":12288,"data_compressed_original":57344,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":62008,"data_compressed":1854,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-08T20:59:27.937 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-03-08T20:59:27.937 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-03-08T20:59:27.938 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-03-08T20:59:27.938 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761 shell --fsid dc20f0f8-dd8c-11ee-95c6-87774f69a715 -- ceph health --format=json 2024-03-08T20:59:28.423 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/mon.a/config 2024-03-08T20:59:28.878 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:59:28 smithi204 ceph-mon[34935]: from='client.14967 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-08T20:59:28.880 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:59:28 smithi107 ceph-mon[31381]: from='client.14967 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-08T20:59:29.015 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:59:28 smithi149 ceph-mon[35042]: from='client.14967 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-08T20:59:29.717 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:59:29 smithi107 ceph-mon[31381]: pgmap v412: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 255 B/s wr, 11 op/s 2024-03-08T20:59:29.878 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:59:29 smithi204 ceph-mon[34935]: pgmap v412: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 255 B/s wr, 11 op/s 2024-03-08T20:59:30.015 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:59:29 smithi149 ceph-mon[35042]: pgmap v412: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 255 B/s wr, 11 op/s 2024-03-08T20:59:30.619 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:30.636+0000 7fd343000700 1 -- 172.21.15.107:0/740685750 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd33c104250 msgr2=0x7fd33c104630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:59:30.620 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:30.636+0000 7fd343000700 1 --2- 172.21.15.107:0/740685750 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd33c104250 0x7fd33c104630 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fd32c0099c0 tx=0x7fd32c009cd0 comp rx=0 tx=0).stop 2024-03-08T20:59:30.620 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:30.636+0000 7fd343000700 1 -- 172.21.15.107:0/740685750 shutdown_connections 2024-03-08T20:59:30.620 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:30.636+0000 7fd343000700 1 --2- 172.21.15.107:0/740685750 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd33c109330 0x7fd33c110f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:30.620 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:30.636+0000 7fd343000700 1 --2- 172.21.15.107:0/740685750 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd33c104250 0x7fd33c104630 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:30.620 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:30.636+0000 7fd343000700 1 --2- 172.21.15.107:0/740685750 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd33c104c00 0x7fd33c108c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:30.620 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:30.636+0000 7fd343000700 1 -- 172.21.15.107:0/740685750 >> 172.21.15.107:0/740685750 conn(0x7fd33c0ffac0 msgr2=0x7fd33c101ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:59:30.621 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:30.637+0000 7fd343000700 1 -- 172.21.15.107:0/740685750 shutdown_connections 2024-03-08T20:59:30.621 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:30.637+0000 7fd343000700 1 -- 172.21.15.107:0/740685750 wait complete. 2024-03-08T20:59:30.623 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:30.638+0000 7fd343000700 1 Processor -- start 2024-03-08T20:59:30.623 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:30.638+0000 7fd343000700 1 -- start start 2024-03-08T20:59:30.624 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:30.638+0000 7fd343000700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd33c104250 0x7fd33c19a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:30.624 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:30.638+0000 7fd343000700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd33c104c00 0x7fd33c19ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:30.624 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:30.638+0000 7fd343000700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd33c109330 0x7fd33c19ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:30.624 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:30.638+0000 7fd343000700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd33c1141f0 con 0x7fd33c109330 2024-03-08T20:59:30.624 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:30.638+0000 7fd343000700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd33c114070 con 0x7fd33c104250 2024-03-08T20:59:30.624 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:30.638+0000 7fd343000700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd33c114370 con 0x7fd33c104c00 2024-03-08T20:59:30.624 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:30.639+0000 7fd341000700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd33c109330 0x7fd33c19ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:59:30.624 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:30.639+0000 7fd341000700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd33c109330 0x7fd33c19ee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:55230/0 (socket says 172.21.15.107:55230) 2024-03-08T20:59:30.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:30.639+0000 7fd341000700 1 -- 172.21.15.107:0/3200584393 learned_addr learned my addr 172.21.15.107:0/3200584393 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T20:59:30.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:30.639+0000 7fd33be00700 1 --2- 172.21.15.107:0/3200584393 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd33c104250 0x7fd33c19a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:59:30.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:30.639+0000 7fd33b400700 1 --2- 172.21.15.107:0/3200584393 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd33c104c00 0x7fd33c19ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:59:30.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:30.639+0000 7fd341000700 1 -- 172.21.15.107:0/3200584393 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd33c104c00 msgr2=0x7fd33c19ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:59:30.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:30.639+0000 7fd341000700 1 --2- 172.21.15.107:0/3200584393 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd33c104c00 0x7fd33c19ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:30.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:30.639+0000 7fd341000700 1 -- 172.21.15.107:0/3200584393 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd33c104250 msgr2=0x7fd33c19a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:59:30.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:30.639+0000 7fd341000700 1 --2- 172.21.15.107:0/3200584393 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd33c104250 0x7fd33c19a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:30.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:30.639+0000 7fd341000700 1 -- 172.21.15.107:0/3200584393 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd33c19f5c0 con 0x7fd33c109330 2024-03-08T20:59:30.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:30.640+0000 7fd341000700 1 --2- 172.21.15.107:0/3200584393 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd33c109330 0x7fd33c19ee80 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fd334007ee0 tx=0x7fd33400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:59:30.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:30.640+0000 7fd338c00700 1 -- 172.21.15.107:0/3200584393 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd33400dbd0 con 0x7fd33c109330 2024-03-08T20:59:30.626 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:30.640+0000 7fd338c00700 1 -- 172.21.15.107:0/3200584393 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fd33400dd70 con 0x7fd33c109330 2024-03-08T20:59:30.626 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:30.640+0000 7fd338c00700 1 -- 172.21.15.107:0/3200584393 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd334005a20 con 0x7fd33c109330 2024-03-08T20:59:30.626 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:30.640+0000 7fd343000700 1 -- 172.21.15.107:0/3200584393 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd33c19f8b0 con 0x7fd33c109330 2024-03-08T20:59:30.626 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:30.640+0000 7fd343000700 1 -- 172.21.15.107:0/3200584393 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd33c1a7130 con 0x7fd33c109330 2024-03-08T20:59:30.626 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:30.641+0000 7fd343000700 1 -- 172.21.15.107:0/3200584393 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd33c068aa0 con 0x7fd33c109330 2024-03-08T20:59:30.626 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:30.643+0000 7fd338c00700 1 -- 172.21.15.107:0/3200584393 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) v1 ==== 137291+0+0 (secure 0 0 0) 0x7fd33401a020 con 0x7fd33c109330 2024-03-08T20:59:30.627 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:30.645+0000 7fd338c00700 1 --2- 172.21.15.107:0/3200584393 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fd3200a2910 0x7fd3200a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:30.627 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:30.645+0000 7fd338c00700 1 -- 172.21.15.107:0/3200584393 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fd3340ce170 con 0x7fd33c109330 2024-03-08T20:59:30.627 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:30.645+0000 7fd33be00700 1 --2- 172.21.15.107:0/3200584393 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fd3200a2910 0x7fd3200a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:59:30.628 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:30.645+0000 7fd33be00700 1 --2- 172.21.15.107:0/3200584393 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fd3200a2910 0x7fd3200a4dd0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fd32c009f50 tx=0x7fd32c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:59:30.633 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:30.650+0000 7fd338c00700 1 -- 172.21.15.107:0/3200584393 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7fd334010070 con 0x7fd33c109330 2024-03-08T20:59:30.899 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:30.916+0000 7fd343000700 1 -- 172.21.15.107:0/3200584393 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fd33c02d830 con 0x7fd33c109330 2024-03-08T20:59:30.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:30.917+0000 7fd338c00700 1 -- 172.21.15.107:0/3200584393 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7fd33409b1d0 con 0x7fd33c109330 2024-03-08T20:59:30.900 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-08T20:59:30.900 INFO:teuthology.orchestra.run.smithi107.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-03-08T20:59:30.903 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:30.920+0000 7fd343000700 1 -- 172.21.15.107:0/3200584393 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fd3200a2910 msgr2=0x7fd3200a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:59:30.903 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:30.920+0000 7fd343000700 1 --2- 172.21.15.107:0/3200584393 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fd3200a2910 0x7fd3200a4dd0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fd32c009f50 tx=0x7fd32c005fb0 comp rx=0 tx=0).stop 2024-03-08T20:59:30.903 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:30.920+0000 7fd343000700 1 -- 172.21.15.107:0/3200584393 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd33c109330 msgr2=0x7fd33c19ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:59:30.903 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:30.920+0000 7fd343000700 1 --2- 172.21.15.107:0/3200584393 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd33c109330 0x7fd33c19ee80 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fd334007ee0 tx=0x7fd33400d3b0 comp rx=0 tx=0).stop 2024-03-08T20:59:30.903 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:30.920+0000 7fd343000700 1 -- 172.21.15.107:0/3200584393 shutdown_connections 2024-03-08T20:59:30.903 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:30.920+0000 7fd343000700 1 --2- 172.21.15.107:0/3200584393 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fd3200a2910 0x7fd3200a4dd0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:30.903 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:30.920+0000 7fd343000700 1 --2- 172.21.15.107:0/3200584393 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd33c109330 0x7fd33c19ee80 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:30.903 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:30.920+0000 7fd343000700 1 --2- 172.21.15.107:0/3200584393 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd33c104250 0x7fd33c19a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:30.904 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:30.920+0000 7fd343000700 1 --2- 172.21.15.107:0/3200584393 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd33c104c00 0x7fd33c19ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:30.904 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:30.920+0000 7fd343000700 1 -- 172.21.15.107:0/3200584393 >> 172.21.15.107:0/3200584393 conn(0x7fd33c0ffac0 msgr2=0x7fd33c100620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:59:30.904 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:30.920+0000 7fd343000700 1 -- 172.21.15.107:0/3200584393 shutdown_connections 2024-03-08T20:59:30.904 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:30.920+0000 7fd343000700 1 -- 172.21.15.107:0/3200584393 wait complete. 2024-03-08T20:59:31.692 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:59:31 smithi107 ceph-mon[31381]: pgmap v413: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-03-08T20:59:31.692 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:59:31 smithi107 ceph-mon[31381]: from='client.? 172.21.15.107:0/3200584393' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-03-08T20:59:31.753 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-03-08T20:59:31.753 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-08T20:59:31.878 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:59:31 smithi204 ceph-mon[34935]: pgmap v413: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-03-08T20:59:31.878 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:59:31 smithi204 ceph-mon[34935]: from='client.? 172.21.15.107:0/3200584393' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-03-08T20:59:31.972 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:31.990+0000 7f7e51c00640 1 -- 172.21.15.107:0/4153784686 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7e4c1050e0 msgr2=0x7f7e4c10b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:59:31.973 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:31.990+0000 7f7e51c00640 1 --2- 172.21.15.107:0/4153784686 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7e4c1050e0 0x7f7e4c10b970 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f7e44009900 tx=0x7f7e4402bc30 comp rx=0 tx=0).stop 2024-03-08T20:59:31.973 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:31.990+0000 7f7e51c00640 1 -- 172.21.15.107:0/4153784686 shutdown_connections 2024-03-08T20:59:31.973 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:31.990+0000 7f7e51c00640 1 --2- 172.21.15.107:0/4153784686 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7e4c1050e0 0x7f7e4c10b970 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:31.973 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:31.990+0000 7f7e51c00640 1 --2- 172.21.15.107:0/4153784686 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7e4c104720 0x7f7e4c104ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:31.973 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:31.990+0000 7f7e51c00640 1 --2- 172.21.15.107:0/4153784686 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7e4c103530 0x7f7e4c103950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:31.973 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:31.990+0000 7f7e51c00640 1 -- 172.21.15.107:0/4153784686 >> 172.21.15.107:0/4153784686 conn(0x7f7e4c0feae0 msgr2=0x7f7e4c100f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:59:31.973 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:31.990+0000 7f7e51c00640 1 -- 172.21.15.107:0/4153784686 shutdown_connections 2024-03-08T20:59:31.973 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:31.990+0000 7f7e51c00640 1 -- 172.21.15.107:0/4153784686 wait complete. 2024-03-08T20:59:31.975 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:31.991+0000 7f7e51c00640 1 Processor -- start 2024-03-08T20:59:31.975 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:31.991+0000 7f7e51c00640 1 -- start start 2024-03-08T20:59:31.975 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:31.991+0000 7f7e51c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7e4c103530 0x7f7e4c199700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:31.975 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:31.991+0000 7f7e51c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7e4c104720 0x7f7e4c199c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:31.976 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:31.991+0000 7f7e51c00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7e4c1050e0 0x7f7e4c1a0cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:31.976 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:31.991+0000 7f7e51c00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e4c10f090 con 0x7f7e4c103530 2024-03-08T20:59:31.976 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:31.991+0000 7f7e51c00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e4c10ef10 con 0x7f7e4c1050e0 2024-03-08T20:59:31.976 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:31.991+0000 7f7e51c00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e4c10f210 con 0x7f7e4c104720 2024-03-08T20:59:31.976 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:31.991+0000 7f7e4aa00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7e4c103530 0x7f7e4c199700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:59:31.976 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:31.992+0000 7f7e4aa00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7e4c103530 0x7f7e4c199700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:55252/0 (socket says 172.21.15.107:55252) 2024-03-08T20:59:31.976 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:31.992+0000 7f7e4aa00640 1 -- 172.21.15.107:0/1989774735 learned_addr learned my addr 172.21.15.107:0/1989774735 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T20:59:31.976 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:31.992+0000 7f7e4a000640 1 --2- 172.21.15.107:0/1989774735 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7e4c104720 0x7f7e4c199c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:59:31.976 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:31.992+0000 7f7e4aa00640 1 -- 172.21.15.107:0/1989774735 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7e4c104720 msgr2=0x7f7e4c199c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:59:31.976 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:31.992+0000 7f7e4aa00640 1 --2- 172.21.15.107:0/1989774735 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7e4c104720 0x7f7e4c199c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:31.976 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:31.992+0000 7f7e4aa00640 1 -- 172.21.15.107:0/1989774735 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7e4c1050e0 msgr2=0x7f7e4c1a0cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:59:31.976 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:31.992+0000 7f7e4aa00640 1 --2- 172.21.15.107:0/1989774735 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7e4c1050e0 0x7f7e4c1a0cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:31.976 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:31.992+0000 7f7e4aa00640 1 -- 172.21.15.107:0/1989774735 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e4c1a1320 con 0x7f7e4c103530 2024-03-08T20:59:31.977 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:31.992+0000 7f7e4aa00640 1 --2- 172.21.15.107:0/1989774735 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7e4c103530 0x7f7e4c199700 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f7e3c00d770 tx=0x7f7e3c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:59:31.977 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:31.992+0000 7f7e43400640 1 -- 172.21.15.107:0/1989774735 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e3c017070 con 0x7f7e4c103530 2024-03-08T20:59:31.977 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:31.992+0000 7f7e43400640 1 -- 172.21.15.107:0/1989774735 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f7e3c009c30 con 0x7f7e4c103530 2024-03-08T20:59:31.977 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:31.992+0000 7f7e51c00640 1 -- 172.21.15.107:0/1989774735 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e4c1a1610 con 0x7f7e4c103530 2024-03-08T20:59:31.977 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:31.992+0000 7f7e43400640 1 -- 172.21.15.107:0/1989774735 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e3c003d00 con 0x7f7e4c103530 2024-03-08T20:59:31.977 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:31.993+0000 7f7e51c00640 1 -- 172.21.15.107:0/1989774735 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e4c1a1b60 con 0x7f7e4c103530 2024-03-08T20:59:31.977 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:31.994+0000 7f7e51c00640 1 -- 172.21.15.107:0/1989774735 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e4c10a850 con 0x7f7e4c103530 2024-03-08T20:59:31.978 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:31.995+0000 7f7e43400640 1 -- 172.21.15.107:0/1989774735 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f7e3c003ea0 con 0x7f7e4c103530 2024-03-08T20:59:31.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:31.998+0000 7f7e43400640 1 --2- 172.21.15.107:0/1989774735 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f7e2c0a2800 0x7f7e2c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:31.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:31.998+0000 7f7e43400640 1 -- 172.21.15.107:0/1989774735 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f7e3c0ce880 con 0x7f7e4c103530 2024-03-08T20:59:31.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:31.998+0000 7f7e4a000640 1 --2- 172.21.15.107:0/1989774735 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f7e2c0a2800 0x7f7e2c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:59:31.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:31.998+0000 7f7e4a000640 1 --2- 172.21.15.107:0/1989774735 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f7e2c0a2800 0x7f7e2c0a4cc0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f7e4c19ac10 tx=0x7f7e34008230 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:59:31.984 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:32.002+0000 7f7e43400640 1 -- 172.21.15.107:0/1989774735 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f7e3c09b8e0 con 0x7f7e4c103530 2024-03-08T20:59:32.015 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:59:31 smithi149 ceph-mon[35042]: pgmap v413: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-03-08T20:59:32.015 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:59:31 smithi149 ceph-mon[35042]: from='client.? 172.21.15.107:0/3200584393' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-03-08T20:59:32.183 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:32.201+0000 7f7e51c00640 1 -- 172.21.15.107:0/1989774735 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7e4c061f90 con 0x7f7e4c103530 2024-03-08T20:59:32.187 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:32.205+0000 7f7e43400640 1 -- 172.21.15.107:0/1989774735 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+22779 (secure 0 0 0) 0x7f7e3c009de0 con 0x7f7e4c103530 2024-03-08T20:59:32.188 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-08T20:59:32.188 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":88,"fsid":"dc20f0f8-dd8c-11ee-95c6-87774f69a715","created":"2024-03-08T20:47:04.190034+0000","modified":"2024-03-08T20:58:21.449936+0000","last_up_change":"2024-03-08T20:56:43.135514+0000","last_in_change":"2024-03-08T20:56:17.577074+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-08T20:51:32.832786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-08T20:57:48.721227+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-08T20:57:49.707779+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8b1ddc64-4d20-4e85-ac66-f1c5ac599501","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":2453946095},{"type":"v1","addr":"172.21.15.107:6803","nonce":2453946095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":2453946095},{"type":"v1","addr":"172.21.15.107:6805","nonce":2453946095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":2453946095},{"type":"v1","addr":"172.21.15.107:6809","nonce":2453946095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":2453946095},{"type":"v1","addr":"172.21.15.107:6807","nonce":2453946095}]},"public_addr":"172.21.15.107:6803/2453946095","cluster_addr":"172.21.15.107:6805/2453946095","heartbeat_back_addr":"172.21.15.107:6809/2453946095","heartbeat_front_addr":"172.21.15.107:6807/2453946095","state":["exists","up"]},{"osd":1,"uuid":"5a803c09-202a-4777-ba2a-1da8eaf26a54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":3071316419},{"type":"v1","addr":"172.21.15.149:6801","nonce":3071316419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":3071316419},{"type":"v1","addr":"172.21.15.149:6803","nonce":3071316419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":3071316419},{"type":"v1","addr":"172.21.15.149:6807","nonce":3071316419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":3071316419},{"type":"v1","addr":"172.21.15.149:6805","nonce":3071316419}]},"public_addr":"172.21.15.149:6801/3071316419","cluster_addr":"172.21.15.149:6803/3071316419","heartbeat_back_addr":"172.21.15.149:6807/3071316419","heartbeat_front_addr":"172.21.15.149:6805/3071316419","state":["exists","up"]},{"osd":2,"uuid":"555c634f-347e-47dd-a943-e50ae2be073c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":1087852630},{"type":"v1","addr":"172.21.15.204:6801","nonce":1087852630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":1087852630},{"type":"v1","addr":"172.21.15.204:6803","nonce":1087852630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":1087852630},{"type":"v1","addr":"172.21.15.204:6807","nonce":1087852630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":1087852630},{"type":"v1","addr":"172.21.15.204:6805","nonce":1087852630}]},"public_addr":"172.21.15.204:6801/1087852630","cluster_addr":"172.21.15.204:6803/1087852630","heartbeat_back_addr":"172.21.15.204:6807/1087852630","heartbeat_front_addr":"172.21.15.204:6805/1087852630","state":["exists","up"]},{"osd":3,"uuid":"9e433d97-e202-48e0-9ed1-f0988fa14307","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":3123579789},{"type":"v1","addr":"172.21.15.107:6811","nonce":3123579789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":3123579789},{"type":"v1","addr":"172.21.15.107:6813","nonce":3123579789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":3123579789},{"type":"v1","addr":"172.21.15.107:6817","nonce":3123579789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":3123579789},{"type":"v1","addr":"172.21.15.107:6815","nonce":3123579789}]},"public_addr":"172.21.15.107:6811/3123579789","cluster_addr":"172.21.15.107:6813/3123579789","heartbeat_back_addr":"172.21.15.107:6817/3123579789","heartbeat_front_addr":"172.21.15.107:6815/3123579789","state":["exists","up"]},{"osd":4,"uuid":"27c53794-9d2b-487d-bfcf-ac4fd2ab061e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":726286918},{"type":"v1","addr":"172.21.15.149:6809","nonce":726286918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":726286918},{"type":"v1","addr":"172.21.15.149:6811","nonce":726286918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":726286918},{"type":"v1","addr":"172.21.15.149:6815","nonce":726286918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":726286918},{"type":"v1","addr":"172.21.15.149:6813","nonce":726286918}]},"public_addr":"172.21.15.149:6809/726286918","cluster_addr":"172.21.15.149:6811/726286918","heartbeat_back_addr":"172.21.15.149:6815/726286918","heartbeat_front_addr":"172.21.15.149:6813/726286918","state":["exists","up"]},{"osd":5,"uuid":"3a7778fb-46c5-40c0-842e-e988c45ea24d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":3143887820},{"type":"v1","addr":"172.21.15.204:6809","nonce":3143887820}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":3143887820},{"type":"v1","addr":"172.21.15.204:6811","nonce":3143887820}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":3143887820},{"type":"v1","addr":"172.21.15.204:6815","nonce":3143887820}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":3143887820},{"type":"v1","addr":"172.21.15.204:6813","nonce":3143887820}]},"public_addr":"172.21.15.204:6809/3143887820","cluster_addr":"172.21.15.204:6811/3143887820","heartbeat_back_addr":"172.21.15.204:6815/3143887820","heartbeat_front_addr":"172.21.15.204:6813/3143887820","state":["exists","up"]},{"osd":6,"uuid":"d2e05a56-be4f-4435-9b76-ff9aa01e4ac3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":2912452950},{"type":"v1","addr":"172.21.15.107:6819","nonce":2912452950}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":2912452950},{"type":"v1","addr":"172.21.15.107:6821","nonce":2912452950}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":2912452950},{"type":"v1","addr":"172.21.15.107:6825","nonce":2912452950}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":2912452950},{"type":"v1","addr":"172.21.15.107:6823","nonce":2912452950}]},"public_addr":"172.21.15.107:6819/2912452950","cluster_addr":"172.21.15.107:6821/2912452950","heartbeat_back_addr":"172.21.15.107:6825/2912452950","heartbeat_front_addr":"172.21.15.107:6823/2912452950","state":["exists","up"]},{"osd":7,"uuid":"a7dd26ee-c118-4a9a-9e19-142c2680902d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":3835863809},{"type":"v1","addr":"172.21.15.149:6817","nonce":3835863809}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":3835863809},{"type":"v1","addr":"172.21.15.149:6819","nonce":3835863809}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":3835863809},{"type":"v1","addr":"172.21.15.149:6823","nonce":3835863809}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":3835863809},{"type":"v1","addr":"172.21.15.149:6821","nonce":3835863809}]},"public_addr":"172.21.15.149:6817/3835863809","cluster_addr":"172.21.15.149:6819/3835863809","heartbeat_back_addr":"172.21.15.149:6823/3835863809","heartbeat_front_addr":"172.21.15.149:6821/3835863809","state":["exists","up"]},{"osd":8,"uuid":"baaaa8dd-0e1f-4bae-86bf-9e429d985f23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":1010601477},{"type":"v1","addr":"172.21.15.204:6817","nonce":1010601477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":1010601477},{"type":"v1","addr":"172.21.15.204:6819","nonce":1010601477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":1010601477},{"type":"v1","addr":"172.21.15.204:6823","nonce":1010601477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":1010601477},{"type":"v1","addr":"172.21.15.204:6821","nonce":1010601477}]},"public_addr":"172.21.15.204:6817/1010601477","cluster_addr":"172.21.15.204:6819/1010601477","heartbeat_back_addr":"172.21.15.204:6823/1010601477","heartbeat_front_addr":"172.21.15.204:6821/1010601477","state":["exists","up"]},{"osd":9,"uuid":"8df80dfa-e7ff-410e-824c-b1dce111b02f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":3334759819},{"type":"v1","addr":"172.21.15.107:6827","nonce":3334759819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3334759819},{"type":"v1","addr":"172.21.15.107:6829","nonce":3334759819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":3334759819},{"type":"v1","addr":"172.21.15.107:6833","nonce":3334759819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":3334759819},{"type":"v1","addr":"172.21.15.107:6831","nonce":3334759819}]},"public_addr":"172.21.15.107:6827/3334759819","cluster_addr":"172.21.15.107:6829/3334759819","heartbeat_back_addr":"172.21.15.107:6833/3334759819","heartbeat_front_addr":"172.21.15.107:6831/3334759819","state":["exists","up"]},{"osd":10,"uuid":"338945b2-20c0-40b2-aaca-062cd4adee04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":3469284631},{"type":"v1","addr":"172.21.15.149:6825","nonce":3469284631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":3469284631},{"type":"v1","addr":"172.21.15.149:6827","nonce":3469284631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":3469284631},{"type":"v1","addr":"172.21.15.149:6831","nonce":3469284631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":3469284631},{"type":"v1","addr":"172.21.15.149:6829","nonce":3469284631}]},"public_addr":"172.21.15.149:6825/3469284631","cluster_addr":"172.21.15.149:6827/3469284631","heartbeat_back_addr":"172.21.15.149:6831/3469284631","heartbeat_front_addr":"172.21.15.149:6829/3469284631","state":["exists","up"]},{"osd":11,"uuid":"acccfe4f-d307-4a64-8f4d-fba27bfbe09d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":3097772473},{"type":"v1","addr":"172.21.15.204:6825","nonce":3097772473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":3097772473},{"type":"v1","addr":"172.21.15.204:6827","nonce":3097772473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":3097772473},{"type":"v1","addr":"172.21.15.204:6831","nonce":3097772473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":3097772473},{"type":"v1","addr":"172.21.15.204:6829","nonce":3097772473}]},"public_addr":"172.21.15.204:6825/3097772473","cluster_addr":"172.21.15.204:6827/3097772473","heartbeat_back_addr":"172.21.15.204:6831/3097772473","heartbeat_front_addr":"172.21.15.204:6829/3097772473","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T20:50:25.862753+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T20:50:58.167663+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T20:51:30.312816+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T20:52:05.487917+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T20:52:39.348354+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T20:53:12.899002+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T20:53:47.630837+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T20:54:21.444552+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T20:54:57.101160+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T20:55:31.841786+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T20:56:05.827161+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T20:56:40.611097+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.0","mappings":[{"from":4,"to":6}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.3b","mappings":[{"from":3,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.107:0/1514666962":"2024-03-09T20:48:20.149406+0000","172.21.15.107:0/1886016707":"2024-03-09T20:48:20.149406+0000","172.21.15.107:6801/426372446":"2024-03-09T20:48:20.149406+0000","172.21.15.107:6800/426372446":"2024-03-09T20:48:20.149406+0000","172.21.15.107:0/2392547434":"2024-03-09T20:47:45.706790+0000","172.21.15.107:0/1206995985":"2024-03-09T20:47:45.706790+0000","172.21.15.107:6800/3549936519":"2024-03-09T20:47:45.706790+0000","172.21.15.107:6801/3549936519":"2024-03-09T20:47:45.706790+0000","172.21.15.107:0/2482452215":"2024-03-09T20:47:45.706790+0000","172.21.15.107:0/808124991":"2024-03-09T20:48:20.149406+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-08T20:59:32.190 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:32.207+0000 7f7e51c00640 1 -- 172.21.15.107:0/1989774735 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f7e2c0a2800 msgr2=0x7f7e2c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:59:32.190 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:32.207+0000 7f7e51c00640 1 --2- 172.21.15.107:0/1989774735 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f7e2c0a2800 0x7f7e2c0a4cc0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f7e4c19ac10 tx=0x7f7e34008230 comp rx=0 tx=0).stop 2024-03-08T20:59:32.190 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:32.207+0000 7f7e51c00640 1 -- 172.21.15.107:0/1989774735 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7e4c103530 msgr2=0x7f7e4c199700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:59:32.190 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:32.207+0000 7f7e51c00640 1 --2- 172.21.15.107:0/1989774735 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7e4c103530 0x7f7e4c199700 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f7e3c00d770 tx=0x7f7e3c00dc30 comp rx=0 tx=0).stop 2024-03-08T20:59:32.190 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:32.207+0000 7f7e51c00640 1 -- 172.21.15.107:0/1989774735 shutdown_connections 2024-03-08T20:59:32.190 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:32.207+0000 7f7e51c00640 1 --2- 172.21.15.107:0/1989774735 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f7e2c0a2800 0x7f7e2c0a4cc0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:32.190 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:32.207+0000 7f7e51c00640 1 --2- 172.21.15.107:0/1989774735 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7e4c1050e0 0x7f7e4c1a0cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:32.191 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:32.207+0000 7f7e51c00640 1 --2- 172.21.15.107:0/1989774735 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7e4c104720 0x7f7e4c199c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:32.191 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:32.207+0000 7f7e51c00640 1 --2- 172.21.15.107:0/1989774735 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7e4c103530 0x7f7e4c199700 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:32.191 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:32.207+0000 7f7e51c00640 1 -- 172.21.15.107:0/1989774735 >> 172.21.15.107:0/1989774735 conn(0x7f7e4c0feae0 msgr2=0x7f7e4c107a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:59:32.191 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:32.208+0000 7f7e51c00640 1 -- 172.21.15.107:0/1989774735 shutdown_connections 2024-03-08T20:59:32.191 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:32.208+0000 7f7e51c00640 1 -- 172.21.15.107:0/1989774735 wait complete. 2024-03-08T20:59:32.211 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-08T20:51:32.832786+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-08T20:57:48.721227+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-08T20:57:49.707779+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-08T20:59:32.211 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-08T20:59:32.428 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:32.445+0000 7f230bc00640 1 -- 172.21.15.107:0/242331533 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f23041094c0 msgr2=0x7f230410fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:59:32.428 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:32.445+0000 7f230bc00640 1 --2- 172.21.15.107:0/242331533 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f23041094c0 0x7f230410fd50 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f22fc009ca0 tx=0x7f22fc02c0f0 comp rx=0 tx=0).stop 2024-03-08T20:59:32.428 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:32.445+0000 7f230bc00640 1 -- 172.21.15.107:0/242331533 shutdown_connections 2024-03-08T20:59:32.428 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:32.445+0000 7f230bc00640 1 --2- 172.21.15.107:0/242331533 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f23041094c0 0x7f230410fd50 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:32.428 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:32.445+0000 7f230bc00640 1 --2- 172.21.15.107:0/242331533 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2304108b00 0x7f2304108f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:32.429 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:32.445+0000 7f230bc00640 1 --2- 172.21.15.107:0/242331533 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2304107910 0x7f2304107d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:32.429 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:32.445+0000 7f230bc00640 1 -- 172.21.15.107:0/242331533 >> 172.21.15.107:0/242331533 conn(0x7f23040768b0 msgr2=0x7f2304078cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:59:32.429 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:32.445+0000 7f230bc00640 1 -- 172.21.15.107:0/242331533 shutdown_connections 2024-03-08T20:59:32.429 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:32.445+0000 7f230bc00640 1 -- 172.21.15.107:0/242331533 wait complete. 2024-03-08T20:59:32.431 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:32.446+0000 7f230bc00640 1 Processor -- start 2024-03-08T20:59:32.431 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:32.446+0000 7f230bc00640 1 -- start start 2024-03-08T20:59:32.431 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:32.447+0000 7f230bc00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2304107910 0x7f230419dae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:32.431 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:32.447+0000 7f230bc00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2304108b00 0x7f230419e020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:32.431 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:32.447+0000 7f230bc00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f23041094c0 0x7f23041a5090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:32.431 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:32.447+0000 7f230bc00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2304113440 con 0x7f2304107910 2024-03-08T20:59:32.431 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:32.447+0000 7f230bc00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23041132c0 con 0x7f2304108b00 2024-03-08T20:59:32.431 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:32.447+0000 7f230bc00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23041135c0 con 0x7f23041094c0 2024-03-08T20:59:32.431 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:32.447+0000 7f2309200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2304107910 0x7f230419dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:59:32.431 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:32.447+0000 7f2309200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2304107910 0x7f230419dae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:55260/0 (socket says 172.21.15.107:55260) 2024-03-08T20:59:32.431 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:32.447+0000 7f2309200640 1 -- 172.21.15.107:0/3348508303 learned_addr learned my addr 172.21.15.107:0/3348508303 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T20:59:32.431 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:32.447+0000 7f2309c00640 1 --2- 172.21.15.107:0/3348508303 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f23041094c0 0x7f23041a5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:59:32.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:32.447+0000 7f2309200640 1 -- 172.21.15.107:0/3348508303 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f23041094c0 msgr2=0x7f23041a5090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:59:32.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:32.447+0000 7f2303e00640 1 --2- 172.21.15.107:0/3348508303 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2304108b00 0x7f230419e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:59:32.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:32.447+0000 7f2309200640 1 --2- 172.21.15.107:0/3348508303 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f23041094c0 0x7f23041a5090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:32.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:32.447+0000 7f2309200640 1 -- 172.21.15.107:0/3348508303 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2304108b00 msgr2=0x7f230419e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:59:32.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:32.447+0000 7f2309200640 1 --2- 172.21.15.107:0/3348508303 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2304108b00 0x7f230419e020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:32.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:32.447+0000 7f2309200640 1 -- 172.21.15.107:0/3348508303 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23041a5700 con 0x7f2304107910 2024-03-08T20:59:32.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:32.448+0000 7f2309200640 1 --2- 172.21.15.107:0/3348508303 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2304107910 0x7f230419dae0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f22f800d950 tx=0x7f22f800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:59:32.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:32.448+0000 7f2301600640 1 -- 172.21.15.107:0/3348508303 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22f8017070 con 0x7f2304107910 2024-03-08T20:59:32.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:32.448+0000 7f2301600640 1 -- 172.21.15.107:0/3348508303 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f22f8009c30 con 0x7f2304107910 2024-03-08T20:59:32.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:32.448+0000 7f230bc00640 1 -- 172.21.15.107:0/3348508303 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23041a59f0 con 0x7f2304107910 2024-03-08T20:59:32.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:32.448+0000 7f2301600640 1 -- 172.21.15.107:0/3348508303 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22f800f620 con 0x7f2304107910 2024-03-08T20:59:32.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:32.448+0000 7f230bc00640 1 -- 172.21.15.107:0/3348508303 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23041a5f40 con 0x7f2304107910 2024-03-08T20:59:32.433 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:32.449+0000 7f230bc00640 1 -- 172.21.15.107:0/3348508303 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f230410ec30 con 0x7f2304107910 2024-03-08T20:59:32.433 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:32.451+0000 7f2301600640 1 -- 172.21.15.107:0/3348508303 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f22f801a070 con 0x7f2304107910 2024-03-08T20:59:32.435 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:32.453+0000 7f2301600640 1 --2- 172.21.15.107:0/3348508303 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f22ec0a2900 0x7f22ec0a4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:32.436 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:32.453+0000 7f2301600640 1 -- 172.21.15.107:0/3348508303 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f22f80ce590 con 0x7f2304107910 2024-03-08T20:59:32.436 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:32.453+0000 7f2303e00640 1 --2- 172.21.15.107:0/3348508303 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f22ec0a2900 0x7f22ec0a4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:59:32.436 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:32.454+0000 7f2303e00640 1 --2- 172.21.15.107:0/3348508303 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f22ec0a2900 0x7f22ec0a4dc0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f22f0005fd0 tx=0x7f22f0005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:59:32.437 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:32.455+0000 7f2301600640 1 -- 172.21.15.107:0/3348508303 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f22f8009de0 con 0x7f2304107910 2024-03-08T20:59:32.636 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:32.654+0000 7f230bc00640 1 -- 172.21.15.107:0/3348508303 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f230402d120 con 0x7f2304107910 2024-03-08T20:59:32.637 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:32.655+0000 7f2301600640 1 -- 172.21.15.107:0/3348508303 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) v1 ==== 93+0+10 (secure 0 0 0) 0x7f22f809b670 con 0x7f2304107910 2024-03-08T20:59:32.637 INFO:teuthology.orchestra.run.smithi107.stdout:pg_num: 1 2024-03-08T20:59:32.640 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:32.657+0000 7f230bc00640 1 -- 172.21.15.107:0/3348508303 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f22ec0a2900 msgr2=0x7f22ec0a4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:59:32.640 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:32.657+0000 7f230bc00640 1 --2- 172.21.15.107:0/3348508303 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f22ec0a2900 0x7f22ec0a4dc0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f22f0005fd0 tx=0x7f22f0005850 comp rx=0 tx=0).stop 2024-03-08T20:59:32.640 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:32.657+0000 7f230bc00640 1 -- 172.21.15.107:0/3348508303 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2304107910 msgr2=0x7f230419dae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:59:32.640 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:32.657+0000 7f230bc00640 1 --2- 172.21.15.107:0/3348508303 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2304107910 0x7f230419dae0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f22f800d950 tx=0x7f22f800de10 comp rx=0 tx=0).stop 2024-03-08T20:59:32.640 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:32.658+0000 7f230bc00640 1 -- 172.21.15.107:0/3348508303 shutdown_connections 2024-03-08T20:59:32.641 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:32.658+0000 7f230bc00640 1 --2- 172.21.15.107:0/3348508303 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f22ec0a2900 0x7f22ec0a4dc0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:32.641 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:32.658+0000 7f230bc00640 1 --2- 172.21.15.107:0/3348508303 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f23041094c0 0x7f23041a5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:32.641 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:32.658+0000 7f230bc00640 1 --2- 172.21.15.107:0/3348508303 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2304108b00 0x7f230419e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:32.641 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:32.658+0000 7f230bc00640 1 --2- 172.21.15.107:0/3348508303 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2304107910 0x7f230419dae0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:32.641 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:32.658+0000 7f230bc00640 1 -- 172.21.15.107:0/3348508303 >> 172.21.15.107:0/3348508303 conn(0x7f23040768b0 msgr2=0x7f230410be00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:59:32.641 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:32.658+0000 7f230bc00640 1 -- 172.21.15.107:0/3348508303 shutdown_connections 2024-03-08T20:59:32.641 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:32.658+0000 7f230bc00640 1 -- 172.21.15.107:0/3348508303 wait complete. 2024-03-08T20:59:32.660 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-08T20:59:32.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:32.889+0000 7f8036c00640 1 -- 172.21.15.107:0/3633821012 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8030075330 msgr2=0x7f8030073780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:59:32.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:32.889+0000 7f8036c00640 1 --2- 172.21.15.107:0/3633821012 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8030075330 0x7f8030073780 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f8020009cd0 tx=0x7f802002c0f0 comp rx=0 tx=0).stop 2024-03-08T20:59:32.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:32.889+0000 7f8036c00640 1 -- 172.21.15.107:0/3633821012 shutdown_connections 2024-03-08T20:59:32.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:32.889+0000 7f8036c00640 1 --2- 172.21.15.107:0/3633821012 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f803010d8d0 0x7f803010fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:32.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:32.889+0000 7f8036c00640 1 --2- 172.21.15.107:0/3633821012 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8030073d50 0x7f80300741d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:32.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:32.889+0000 7f8036c00640 1 --2- 172.21.15.107:0/3633821012 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8030075330 0x7f8030073780 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:32.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:32.889+0000 7f8036c00640 1 -- 172.21.15.107:0/3633821012 >> 172.21.15.107:0/3633821012 conn(0x7f80300fc890 msgr2=0x7f80300fecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:59:32.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:32.889+0000 7f8036c00640 1 -- 172.21.15.107:0/3633821012 shutdown_connections 2024-03-08T20:59:32.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:32.890+0000 7f8036c00640 1 -- 172.21.15.107:0/3633821012 wait complete. 2024-03-08T20:59:32.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:32.890+0000 7f8036c00640 1 Processor -- start 2024-03-08T20:59:32.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:32.891+0000 7f8036c00640 1 -- start start 2024-03-08T20:59:32.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:32.891+0000 7f8036c00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8030073d50 0x7f803019db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:32.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:32.891+0000 7f8036c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8030075330 0x7f803019e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:32.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:32.891+0000 7f8036c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f803010d8d0 0x7f80301a5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:32.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:32.891+0000 7f8036c00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8030113420 con 0x7f8030075330 2024-03-08T20:59:32.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:32.891+0000 7f8036c00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80301132a0 con 0x7f8030073d50 2024-03-08T20:59:32.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:32.891+0000 7f8036c00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80301135a0 con 0x7f803010d8d0 2024-03-08T20:59:32.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:32.891+0000 7f802f400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8030075330 0x7f803019e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:59:32.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:32.891+0000 7f802f400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8030075330 0x7f803019e090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:55286/0 (socket says 172.21.15.107:55286) 2024-03-08T20:59:32.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:32.891+0000 7f802f400640 1 -- 172.21.15.107:0/2656999649 learned_addr learned my addr 172.21.15.107:0/2656999649 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T20:59:32.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:32.891+0000 7f8034c00640 1 --2- 172.21.15.107:0/2656999649 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f803010d8d0 0x7f80301a5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:59:32.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:32.891+0000 7f802fe00640 1 --2- 172.21.15.107:0/2656999649 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8030073d50 0x7f803019db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:59:32.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:32.892+0000 7f802f400640 1 -- 172.21.15.107:0/2656999649 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f803010d8d0 msgr2=0x7f80301a5100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:59:32.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:32.892+0000 7f802f400640 1 --2- 172.21.15.107:0/2656999649 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f803010d8d0 0x7f80301a5100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:32.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:32.892+0000 7f802f400640 1 -- 172.21.15.107:0/2656999649 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8030073d50 msgr2=0x7f803019db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:59:32.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:32.892+0000 7f802f400640 1 --2- 172.21.15.107:0/2656999649 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8030073d50 0x7f803019db50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:32.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:32.892+0000 7f802f400640 1 -- 172.21.15.107:0/2656999649 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80301a5770 con 0x7f8030075330 2024-03-08T20:59:32.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:32.892+0000 7f802f400640 1 --2- 172.21.15.107:0/2656999649 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8030075330 0x7f803019e090 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f801c00d770 tx=0x7f801c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:59:32.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:32.892+0000 7f802cc00640 1 -- 172.21.15.107:0/2656999649 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f801c017070 con 0x7f8030075330 2024-03-08T20:59:32.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:32.892+0000 7f802cc00640 1 -- 172.21.15.107:0/2656999649 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f801c0054d0 con 0x7f8030075330 2024-03-08T20:59:32.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:32.892+0000 7f8036c00640 1 -- 172.21.15.107:0/2656999649 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80301a5a60 con 0x7f8030075330 2024-03-08T20:59:32.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:32.892+0000 7f802cc00640 1 -- 172.21.15.107:0/2656999649 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f801c004cf0 con 0x7f8030075330 2024-03-08T20:59:32.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:32.892+0000 7f8036c00640 1 -- 172.21.15.107:0/2656999649 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80301a5f60 con 0x7f8030075330 2024-03-08T20:59:32.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:32.893+0000 7f8036c00640 1 -- 172.21.15.107:0/2656999649 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8030103aa0 con 0x7f8030075330 2024-03-08T20:59:32.878 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:59:32 smithi204 ceph-mon[34935]: from='client.? 172.21.15.107:0/1989774735' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-08T20:59:32.878 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:32.896+0000 7f802cc00640 1 -- 172.21.15.107:0/2656999649 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f801c004e90 con 0x7f8030075330 2024-03-08T20:59:32.880 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:32.898+0000 7f802cc00640 1 --2- 172.21.15.107:0/2656999649 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f80140a2800 0x7f80140a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:32.881 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:32.898+0000 7f802cc00640 1 -- 172.21.15.107:0/2656999649 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f801c0ce5b0 con 0x7f8030075330 2024-03-08T20:59:32.881 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:32.898+0000 7f802fe00640 1 --2- 172.21.15.107:0/2656999649 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f80140a2800 0x7f80140a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:59:32.881 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:32.899+0000 7f802fe00640 1 --2- 172.21.15.107:0/2656999649 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f80140a2800 0x7f80140a4cc0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f802002c630 tx=0x7f8020000f40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:59:32.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:32.903+0000 7f802cc00640 1 -- 172.21.15.107:0/2656999649 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f801c09b610 con 0x7f8030075330 2024-03-08T20:59:32.967 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:59:32 smithi107 ceph-mon[31381]: from='client.? 172.21.15.107:0/1989774735' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-08T20:59:33.015 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:59:32 smithi149 ceph-mon[35042]: from='client.? 172.21.15.107:0/1989774735' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-08T20:59:33.084 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:33.101+0000 7f8036c00640 1 -- 172.21.15.107:0/2656999649 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f8030061ef0 con 0x7f8030075330 2024-03-08T20:59:33.084 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:33.102+0000 7f802cc00640 1 -- 172.21.15.107:0/2656999649 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) v1 ==== 104+0+11 (secure 0 0 0) 0x7f801c09af10 con 0x7f8030075330 2024-03-08T20:59:33.085 INFO:teuthology.orchestra.run.smithi107.stdout:pg_num: 32 2024-03-08T20:59:33.087 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:33.105+0000 7f8036c00640 1 -- 172.21.15.107:0/2656999649 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f80140a2800 msgr2=0x7f80140a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:59:33.087 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:33.105+0000 7f8036c00640 1 --2- 172.21.15.107:0/2656999649 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f80140a2800 0x7f80140a4cc0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f802002c630 tx=0x7f8020000f40 comp rx=0 tx=0).stop 2024-03-08T20:59:33.088 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:33.105+0000 7f8036c00640 1 -- 172.21.15.107:0/2656999649 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8030075330 msgr2=0x7f803019e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:59:33.088 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:33.105+0000 7f8036c00640 1 --2- 172.21.15.107:0/2656999649 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8030075330 0x7f803019e090 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f801c00d770 tx=0x7f801c00dc30 comp rx=0 tx=0).stop 2024-03-08T20:59:33.088 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:33.105+0000 7f8036c00640 1 -- 172.21.15.107:0/2656999649 shutdown_connections 2024-03-08T20:59:33.088 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:33.105+0000 7f8036c00640 1 --2- 172.21.15.107:0/2656999649 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f80140a2800 0x7f80140a4cc0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:33.088 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:33.105+0000 7f8036c00640 1 --2- 172.21.15.107:0/2656999649 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f803010d8d0 0x7f80301a5100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:33.088 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:33.105+0000 7f8036c00640 1 --2- 172.21.15.107:0/2656999649 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8030075330 0x7f803019e090 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:33.088 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:33.105+0000 7f8036c00640 1 --2- 172.21.15.107:0/2656999649 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8030073d50 0x7f803019db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:33.088 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:33.105+0000 7f8036c00640 1 -- 172.21.15.107:0/2656999649 >> 172.21.15.107:0/2656999649 conn(0x7f80300fc890 msgr2=0x7f8030100c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:59:33.088 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:33.105+0000 7f8036c00640 1 -- 172.21.15.107:0/2656999649 shutdown_connections 2024-03-08T20:59:33.088 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:33.105+0000 7f8036c00640 1 -- 172.21.15.107:0/2656999649 wait complete. 2024-03-08T20:59:33.107 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-08T20:59:33.324 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:33.341+0000 7f08f7e00640 1 -- 172.21.15.107:0/2158770482 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f08f8106eb0 msgr2=0x7f08f8109300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:59:33.324 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:33.341+0000 7f08f7e00640 1 --2- 172.21.15.107:0/2158770482 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f08f8106eb0 0x7f08f8109300 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f08f0009ca0 tx=0x7f08f002c0f0 comp rx=0 tx=0).stop 2024-03-08T20:59:33.324 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:33.341+0000 7f08f7e00640 1 -- 172.21.15.107:0/2158770482 shutdown_connections 2024-03-08T20:59:33.325 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:33.341+0000 7f08f7e00640 1 --2- 172.21.15.107:0/2158770482 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f08f8106eb0 0x7f08f8109300 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:33.325 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:33.341+0000 7f08f7e00640 1 --2- 172.21.15.107:0/2158770482 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f08f8104580 0x7f08f8106970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:33.325 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:33.341+0000 7f08f7e00640 1 --2- 172.21.15.107:0/2158770482 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f08f8101c50 0x7f08f8104040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:33.325 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:33.341+0000 7f08f7e00640 1 -- 172.21.15.107:0/2158770482 >> 172.21.15.107:0/2158770482 conn(0x7f08f80fb660 msgr2=0x7f08f80fda80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:59:33.325 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:33.341+0000 7f08f7e00640 1 -- 172.21.15.107:0/2158770482 shutdown_connections 2024-03-08T20:59:33.325 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:33.341+0000 7f08f7e00640 1 -- 172.21.15.107:0/2158770482 wait complete. 2024-03-08T20:59:33.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:33.342+0000 7f08f7e00640 1 Processor -- start 2024-03-08T20:59:33.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:33.342+0000 7f08f7e00640 1 -- start start 2024-03-08T20:59:33.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:33.343+0000 7f08f7e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f08f8101c50 0x7f08f81015c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:33.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:33.343+0000 7f08f7e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f08f8104580 0x7f08f80ffc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:33.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:33.343+0000 7f08f7e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f08f8106eb0 0x7f08f8100140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:33.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:33.343+0000 7f08f7e00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08f810cd10 con 0x7f08f8106eb0 2024-03-08T20:59:33.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:33.343+0000 7f08f7e00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08f810cb90 con 0x7f08f8101c50 2024-03-08T20:59:33.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:33.343+0000 7f08f7e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08f810ce90 con 0x7f08f8104580 2024-03-08T20:59:33.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:33.343+0000 7f08f7400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f08f8106eb0 0x7f08f8100140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:59:33.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:33.343+0000 7f08f7400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f08f8106eb0 0x7f08f8100140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:55302/0 (socket says 172.21.15.107:55302) 2024-03-08T20:59:33.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:33.343+0000 7f08f7400640 1 -- 172.21.15.107:0/2234851784 learned_addr learned my addr 172.21.15.107:0/2234851784 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T20:59:33.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:33.343+0000 7f08f6a00640 1 --2- 172.21.15.107:0/2234851784 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f08f8101c50 0x7f08f81015c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:59:33.331 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:33.343+0000 7f08f6000640 1 --2- 172.21.15.107:0/2234851784 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f08f8104580 0x7f08f80ffc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:59:33.331 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:33.343+0000 7f08f7400640 1 -- 172.21.15.107:0/2234851784 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f08f8104580 msgr2=0x7f08f80ffc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:59:33.331 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:33.343+0000 7f08f7400640 1 --2- 172.21.15.107:0/2234851784 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f08f8104580 0x7f08f80ffc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:33.331 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:33.343+0000 7f08f7400640 1 -- 172.21.15.107:0/2234851784 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f08f8101c50 msgr2=0x7f08f81015c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:59:33.331 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:33.343+0000 7f08f7400640 1 --2- 172.21.15.107:0/2234851784 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f08f8101c50 0x7f08f81015c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:33.331 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:33.343+0000 7f08f7400640 1 -- 172.21.15.107:0/2234851784 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08f81009d0 con 0x7f08f8106eb0 2024-03-08T20:59:33.331 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:33.344+0000 7f08f7400640 1 --2- 172.21.15.107:0/2234851784 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f08f8106eb0 0x7f08f8100140 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f08f0032380 tx=0x7f08f0032820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:59:33.331 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:33.344+0000 7f08ef400640 1 -- 172.21.15.107:0/2234851784 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08f0041020 con 0x7f08f8106eb0 2024-03-08T20:59:33.331 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:33.344+0000 7f08ef400640 1 -- 172.21.15.107:0/2234851784 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f08f0003e20 con 0x7f08f8106eb0 2024-03-08T20:59:33.332 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:33.344+0000 7f08f7e00640 1 -- 172.21.15.107:0/2234851784 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08f81a13d0 con 0x7f08f8106eb0 2024-03-08T20:59:33.332 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:33.344+0000 7f08ef400640 1 -- 172.21.15.107:0/2234851784 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08f0039040 con 0x7f08f8106eb0 2024-03-08T20:59:33.332 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:33.344+0000 7f08f7e00640 1 -- 172.21.15.107:0/2234851784 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08f81a1920 con 0x7f08f8106eb0 2024-03-08T20:59:33.332 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:33.345+0000 7f08f7e00640 1 -- 172.21.15.107:0/2234851784 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08f80fd150 con 0x7f08f8106eb0 2024-03-08T20:59:33.333 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:33.348+0000 7f08ef400640 1 -- 172.21.15.107:0/2234851784 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f08f0044020 con 0x7f08f8106eb0 2024-03-08T20:59:33.333 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:33.349+0000 7f08ef400640 1 --2- 172.21.15.107:0/2234851784 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f08dc0a2800 0x7f08dc0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:33.333 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:33.349+0000 7f08ef400640 1 -- 172.21.15.107:0/2234851784 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f08f00f0dc0 con 0x7f08f8106eb0 2024-03-08T20:59:33.334 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:33.349+0000 7f08f6a00640 1 --2- 172.21.15.107:0/2234851784 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f08dc0a2800 0x7f08dc0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:59:33.334 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:33.350+0000 7f08f6a00640 1 --2- 172.21.15.107:0/2234851784 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f08dc0a2800 0x7f08dc0a4cc0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f08e8005fd0 tx=0x7f08e8005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:59:33.336 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:33.353+0000 7f08ef400640 1 -- 172.21.15.107:0/2234851784 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f08f004b020 con 0x7f08f8106eb0 2024-03-08T20:59:33.534 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:33.552+0000 7f08f7e00640 1 -- 172.21.15.107:0/2234851784 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f08f8061ef0 con 0x7f08f8106eb0 2024-03-08T20:59:33.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:33.553+0000 7f08ef400640 1 -- 172.21.15.107:0/2234851784 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) v1 ==== 100+0+11 (secure 0 0 0) 0x7f08f00bde60 con 0x7f08f8106eb0 2024-03-08T20:59:33.536 INFO:teuthology.orchestra.run.smithi107.stdout:pg_num: 64 2024-03-08T20:59:33.538 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:33.556+0000 7f08f7e00640 1 -- 172.21.15.107:0/2234851784 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f08dc0a2800 msgr2=0x7f08dc0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:59:33.539 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:33.556+0000 7f08f7e00640 1 --2- 172.21.15.107:0/2234851784 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f08dc0a2800 0x7f08dc0a4cc0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f08e8005fd0 tx=0x7f08e8005850 comp rx=0 tx=0).stop 2024-03-08T20:59:33.539 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:33.556+0000 7f08f7e00640 1 -- 172.21.15.107:0/2234851784 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f08f8106eb0 msgr2=0x7f08f8100140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:59:33.539 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:33.556+0000 7f08f7e00640 1 --2- 172.21.15.107:0/2234851784 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f08f8106eb0 0x7f08f8100140 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f08f0032380 tx=0x7f08f0032820 comp rx=0 tx=0).stop 2024-03-08T20:59:33.539 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:33.556+0000 7f08f7e00640 1 -- 172.21.15.107:0/2234851784 shutdown_connections 2024-03-08T20:59:33.539 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:33.556+0000 7f08f7e00640 1 --2- 172.21.15.107:0/2234851784 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f08dc0a2800 0x7f08dc0a4cc0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:33.539 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:33.556+0000 7f08f7e00640 1 --2- 172.21.15.107:0/2234851784 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f08f8106eb0 0x7f08f8100140 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:33.539 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:33.556+0000 7f08f7e00640 1 --2- 172.21.15.107:0/2234851784 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f08f8104580 0x7f08f80ffc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:33.539 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:33.556+0000 7f08f7e00640 1 --2- 172.21.15.107:0/2234851784 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f08f8101c50 0x7f08f81015c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:33.539 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:33.556+0000 7f08f7e00640 1 -- 172.21.15.107:0/2234851784 >> 172.21.15.107:0/2234851784 conn(0x7f08f80fb660 msgr2=0x7f08f81025b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:59:33.539 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:33.556+0000 7f08f7e00640 1 -- 172.21.15.107:0/2234851784 shutdown_connections 2024-03-08T20:59:33.539 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:33.556+0000 7f08f7e00640 1 -- 172.21.15.107:0/2234851784 wait complete. 2024-03-08T20:59:33.558 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-08T20:59:33.772 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:33.789+0000 7f54fd200640 1 -- 172.21.15.107:0/1991219548 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f54f8101c80 msgr2=0x7f54f8104070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:59:33.772 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:33.789+0000 7f54fd200640 1 --2- 172.21.15.107:0/1991219548 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f54f8101c80 0x7f54f8104070 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f54e8009900 tx=0x7f54e802bc30 comp rx=0 tx=0).stop 2024-03-08T20:59:33.772 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:33.789+0000 7f54fd200640 1 -- 172.21.15.107:0/1991219548 shutdown_connections 2024-03-08T20:59:33.772 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:33.789+0000 7f54fd200640 1 --2- 172.21.15.107:0/1991219548 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f54f8106ee0 0x7f54f81092d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:33.772 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:33.789+0000 7f54fd200640 1 --2- 172.21.15.107:0/1991219548 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f54f81045b0 0x7f54f81069a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:33.773 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:33.789+0000 7f54fd200640 1 --2- 172.21.15.107:0/1991219548 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f54f8101c80 0x7f54f8104070 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:33.773 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:33.789+0000 7f54fd200640 1 -- 172.21.15.107:0/1991219548 >> 172.21.15.107:0/1991219548 conn(0x7f54f80fb590 msgr2=0x7f54f80fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:59:33.773 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:33.789+0000 7f54fd200640 1 -- 172.21.15.107:0/1991219548 shutdown_connections 2024-03-08T20:59:33.773 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:33.789+0000 7f54fd200640 1 -- 172.21.15.107:0/1991219548 wait complete. 2024-03-08T20:59:33.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:33.790+0000 7f54fd200640 1 Processor -- start 2024-03-08T20:59:33.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:33.791+0000 7f54fd200640 1 -- start start 2024-03-08T20:59:33.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:33.791+0000 7f54fd200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f54f8101c80 0x7f54f81995b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:33.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:33.791+0000 7f54fd200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f54f81045b0 0x7f54f8199af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:33.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:33.791+0000 7f54fd200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f54f8106ee0 0x7f54f81a0b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:33.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:33.791+0000 7f54fd200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54f810cc90 con 0x7f54f81045b0 2024-03-08T20:59:33.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:33.791+0000 7f54fd200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54f810cb10 con 0x7f54f8106ee0 2024-03-08T20:59:33.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:33.791+0000 7f54fd200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54f810ce10 con 0x7f54f8101c80 2024-03-08T20:59:33.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:33.791+0000 7f54f6a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f54f81045b0 0x7f54f8199af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:59:33.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:33.791+0000 7f54f6a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f54f81045b0 0x7f54f8199af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:55324/0 (socket says 172.21.15.107:55324) 2024-03-08T20:59:33.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:33.791+0000 7f54f6a00640 1 -- 172.21.15.107:0/1736516860 learned_addr learned my addr 172.21.15.107:0/1736516860 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T20:59:33.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:33.792+0000 7f54f7e00640 1 --2- 172.21.15.107:0/1736516860 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f54f8106ee0 0x7f54f81a0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:59:33.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:33.792+0000 7f54f7400640 1 --2- 172.21.15.107:0/1736516860 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f54f8101c80 0x7f54f81995b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:59:33.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:33.792+0000 7f54f6a00640 1 -- 172.21.15.107:0/1736516860 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f54f8101c80 msgr2=0x7f54f81995b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:59:33.776 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:33.792+0000 7f54f6a00640 1 --2- 172.21.15.107:0/1736516860 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f54f8101c80 0x7f54f81995b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:33.776 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:33.792+0000 7f54f6a00640 1 -- 172.21.15.107:0/1736516860 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f54f8106ee0 msgr2=0x7f54f81a0b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:59:33.776 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:33.792+0000 7f54f6a00640 1 --2- 172.21.15.107:0/1736516860 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f54f8106ee0 0x7f54f81a0b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:33.776 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:33.792+0000 7f54f6a00640 1 -- 172.21.15.107:0/1736516860 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54f81a11d0 con 0x7f54f81045b0 2024-03-08T20:59:33.776 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:33.792+0000 7f54f6a00640 1 --2- 172.21.15.107:0/1736516860 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f54f81045b0 0x7f54f8199af0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f54e400d770 tx=0x7f54e400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:59:33.776 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:33.792+0000 7f54efe00640 1 -- 172.21.15.107:0/1736516860 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54e4017070 con 0x7f54f81045b0 2024-03-08T20:59:33.776 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:33.792+0000 7f54efe00640 1 -- 172.21.15.107:0/1736516860 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f54e4009c30 con 0x7f54f81045b0 2024-03-08T20:59:33.776 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:33.792+0000 7f54fd200640 1 -- 172.21.15.107:0/1736516860 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54f81a14c0 con 0x7f54f81045b0 2024-03-08T20:59:33.777 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:33.792+0000 7f54efe00640 1 -- 172.21.15.107:0/1736516860 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54e400f620 con 0x7f54f81045b0 2024-03-08T20:59:33.777 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:33.793+0000 7f54fd200640 1 -- 172.21.15.107:0/1736516860 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54f81a1a10 con 0x7f54f81045b0 2024-03-08T20:59:33.777 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:33.794+0000 7f54fd200640 1 -- 172.21.15.107:0/1736516860 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54f81089d0 con 0x7f54f81045b0 2024-03-08T20:59:33.778 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:33.796+0000 7f54efe00640 1 -- 172.21.15.107:0/1736516860 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f54e401a070 con 0x7f54f81045b0 2024-03-08T20:59:33.780 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:33.798+0000 7f54efe00640 1 --2- 172.21.15.107:0/1736516860 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f54dc0a2800 0x7f54dc0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:33.780 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:33.798+0000 7f54efe00640 1 -- 172.21.15.107:0/1736516860 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f54e40ce610 con 0x7f54f81045b0 2024-03-08T20:59:33.780 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:33.798+0000 7f54f7400640 1 --2- 172.21.15.107:0/1736516860 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f54dc0a2800 0x7f54dc0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:59:33.781 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:33.799+0000 7f54f7400640 1 --2- 172.21.15.107:0/1736516860 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f54dc0a2800 0x7f54dc0a4cc0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f54e8000c00 tx=0x7f54e8005be0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:59:33.784 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:33.802+0000 7f54efe00640 1 -- 172.21.15.107:0/1736516860 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f54e4009de0 con 0x7f54f81045b0 2024-03-08T20:59:33.878 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:59:33 smithi204 ceph-mon[34935]: pgmap v414: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 10 op/s 2024-03-08T20:59:33.878 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:59:33 smithi204 ceph-mon[34935]: from='client.? 172.21.15.107:0/3348508303' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-08T20:59:33.878 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:59:33 smithi204 ceph-mon[34935]: from='client.? 172.21.15.107:0/2656999649' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-08T20:59:33.878 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:59:33 smithi204 ceph-mon[34935]: from='client.? 172.21.15.107:0/2234851784' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-08T20:59:33.967 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:59:33 smithi107 ceph-mon[31381]: pgmap v414: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 10 op/s 2024-03-08T20:59:33.967 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:59:33 smithi107 ceph-mon[31381]: from='client.? 172.21.15.107:0/3348508303' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-08T20:59:33.967 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:59:33 smithi107 ceph-mon[31381]: from='client.? 172.21.15.107:0/2656999649' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-08T20:59:33.967 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:59:33 smithi107 ceph-mon[31381]: from='client.? 172.21.15.107:0/2234851784' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-08T20:59:34.015 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:59:33 smithi149 ceph-mon[35042]: pgmap v414: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 10 op/s 2024-03-08T20:59:34.015 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:59:33 smithi149 ceph-mon[35042]: from='client.? 172.21.15.107:0/3348508303' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-08T20:59:34.015 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:59:33 smithi149 ceph-mon[35042]: from='client.? 172.21.15.107:0/2656999649' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-08T20:59:34.015 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:59:33 smithi149 ceph-mon[35042]: from='client.? 172.21.15.107:0/2234851784' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-08T20:59:34.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:34.040+0000 7f54fd200640 1 -- 172.21.15.107:0/1736516860 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f54f8061ef0 con 0x7f54f81045b0 2024-03-08T20:59:34.024 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:34.042+0000 7f54efe00640 1 -- 172.21.15.107:0/1736516860 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+11393 (secure 0 0 0) 0x7f54e409b6b0 con 0x7f54f81045b0 2024-03-08T20:59:34.024 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-08T20:59:34.025 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/133582743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":133582743},{"type":"v1","addr":"172.21.15.107:6835","nonce":133582743}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/3798982337","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":3798982337},{"type":"v1","addr":"172.21.15.107:6837","nonce":3798982337}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-08T20:57:50.223752+0000","modified":"2024-03-08T20:57:57.874727+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34475,"mds_1":34445,"mds_2":34415,"mds_3":34397,"mds_4":24457},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14625":{"gid":14625,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6839/3070059000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":3070059000},{"type":"v1","addr":"172.21.15.107:6839","nonce":3070059000}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14637":{"gid":14637,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6841/747096194","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6840","nonce":747096194},{"type":"v1","addr":"172.21.15.107:6841","nonce":747096194}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24400":{"gid":24400,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6833/3404762782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":3404762782},{"type":"v1","addr":"172.21.15.204:6833","nonce":3404762782}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24430":{"gid":24430,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/2072518651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":2072518651},{"type":"v1","addr":"172.21.15.204:6835","nonce":2072518651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24442":{"gid":24442,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6837/42644905","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":42644905},{"type":"v1","addr":"172.21.15.204:6837","nonce":42644905}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24457":{"gid":24457,"name":"i","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.204:6839/1857316822","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":1857316822},{"type":"v1","addr":"172.21.15.204:6839","nonce":1857316822}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34397":{"gid":34397,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":14,"addr":"172.21.15.149:6833/1962161726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":1962161726},{"type":"v1","addr":"172.21.15.149:6833","nonce":1962161726}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34415":{"gid":34415,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.149:6835/442657066","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":442657066},{"type":"v1","addr":"172.21.15.149:6835","nonce":442657066}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34445":{"gid":34445,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.149:6837/914546237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":914546237},{"type":"v1","addr":"172.21.15.149:6837","nonce":914546237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34475":{"gid":34475,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.149:6839/55290596","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":55290596},{"type":"v1","addr":"172.21.15.149:6839","nonce":55290596}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-08T20:59:34.025 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 27 2024-03-08T20:59:34.027 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:34.044+0000 7f54fd200640 1 -- 172.21.15.107:0/1736516860 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f54dc0a2800 msgr2=0x7f54dc0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:59:34.027 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:34.044+0000 7f54fd200640 1 --2- 172.21.15.107:0/1736516860 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f54dc0a2800 0x7f54dc0a4cc0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f54e8000c00 tx=0x7f54e8005be0 comp rx=0 tx=0).stop 2024-03-08T20:59:34.027 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:34.044+0000 7f54fd200640 1 -- 172.21.15.107:0/1736516860 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f54f81045b0 msgr2=0x7f54f8199af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:59:34.027 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:34.044+0000 7f54fd200640 1 --2- 172.21.15.107:0/1736516860 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f54f81045b0 0x7f54f8199af0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f54e400d770 tx=0x7f54e400dc30 comp rx=0 tx=0).stop 2024-03-08T20:59:34.027 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:34.044+0000 7f54fd200640 1 -- 172.21.15.107:0/1736516860 shutdown_connections 2024-03-08T20:59:34.027 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:34.044+0000 7f54fd200640 1 --2- 172.21.15.107:0/1736516860 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f54dc0a2800 0x7f54dc0a4cc0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:34.027 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:34.044+0000 7f54fd200640 1 --2- 172.21.15.107:0/1736516860 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f54f8106ee0 0x7f54f81a0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:34.027 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:34.044+0000 7f54fd200640 1 --2- 172.21.15.107:0/1736516860 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f54f81045b0 0x7f54f8199af0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:34.028 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:34.044+0000 7f54fd200640 1 --2- 172.21.15.107:0/1736516860 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f54f8101c80 0x7f54f81995b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:34.028 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:34.044+0000 7f54fd200640 1 -- 172.21.15.107:0/1736516860 >> 172.21.15.107:0/1736516860 conn(0x7f54f80fb590 msgr2=0x7f54f8101360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:59:34.028 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:34.045+0000 7f54fd200640 1 -- 172.21.15.107:0/1736516860 shutdown_connections 2024-03-08T20:59:34.028 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:34.045+0000 7f54fd200640 1 -- 172.21.15.107:0/1736516860 wait complete. 2024-03-08T20:59:34.046 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-08T20:59:34.257 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:34.274+0000 7efc61c00640 1 -- 172.21.15.107:0/1772423998 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efc5c104720 msgr2=0x7efc5c104ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:59:34.257 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:34.274+0000 7efc61c00640 1 --2- 172.21.15.107:0/1772423998 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efc5c104720 0x7efc5c104ba0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7efc44009900 tx=0x7efc4402bc30 comp rx=0 tx=0).stop 2024-03-08T20:59:34.257 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:34.274+0000 7efc61c00640 1 -- 172.21.15.107:0/1772423998 shutdown_connections 2024-03-08T20:59:34.257 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:34.274+0000 7efc61c00640 1 --2- 172.21.15.107:0/1772423998 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efc5c1050e0 0x7efc5c10b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:34.257 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:34.274+0000 7efc61c00640 1 --2- 172.21.15.107:0/1772423998 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efc5c104720 0x7efc5c104ba0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:34.257 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:34.274+0000 7efc61c00640 1 --2- 172.21.15.107:0/1772423998 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efc5c103530 0x7efc5c103950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:34.258 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:34.274+0000 7efc61c00640 1 -- 172.21.15.107:0/1772423998 >> 172.21.15.107:0/1772423998 conn(0x7efc5c0feae0 msgr2=0x7efc5c100f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:59:34.258 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:34.275+0000 7efc61c00640 1 -- 172.21.15.107:0/1772423998 shutdown_connections 2024-03-08T20:59:34.258 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:34.275+0000 7efc61c00640 1 -- 172.21.15.107:0/1772423998 wait complete. 2024-03-08T20:59:34.260 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:34.275+0000 7efc61c00640 1 Processor -- start 2024-03-08T20:59:34.260 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:34.276+0000 7efc61c00640 1 -- start start 2024-03-08T20:59:34.260 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:34.276+0000 7efc61c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efc5c103530 0x7efc5c199820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:34.260 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:34.276+0000 7efc61c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efc5c104720 0x7efc5c199d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:34.260 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:34.276+0000 7efc61c00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efc5c1050e0 0x7efc5c1a0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:34.260 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:34.276+0000 7efc61c00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc5c10f120 con 0x7efc5c103530 2024-03-08T20:59:34.261 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:34.276+0000 7efc61c00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc5c10efa0 con 0x7efc5c1050e0 2024-03-08T20:59:34.261 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:34.276+0000 7efc61c00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc5c10f2a0 con 0x7efc5c104720 2024-03-08T20:59:34.261 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:34.276+0000 7efc5aa00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efc5c103530 0x7efc5c199820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:59:34.261 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:34.276+0000 7efc5aa00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efc5c103530 0x7efc5c199820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:55352/0 (socket says 172.21.15.107:55352) 2024-03-08T20:59:34.261 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:34.276+0000 7efc5aa00640 1 -- 172.21.15.107:0/1591046412 learned_addr learned my addr 172.21.15.107:0/1591046412 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T20:59:34.261 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:34.276+0000 7efc5a000640 1 --2- 172.21.15.107:0/1591046412 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efc5c104720 0x7efc5c199d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:59:34.261 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:34.276+0000 7efc5b400640 1 --2- 172.21.15.107:0/1591046412 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efc5c1050e0 0x7efc5c1a0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:59:34.261 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:34.277+0000 7efc5aa00640 1 -- 172.21.15.107:0/1591046412 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efc5c104720 msgr2=0x7efc5c199d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:59:34.261 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:34.277+0000 7efc5aa00640 1 --2- 172.21.15.107:0/1591046412 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efc5c104720 0x7efc5c199d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:34.261 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:34.277+0000 7efc5aa00640 1 -- 172.21.15.107:0/1591046412 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efc5c1050e0 msgr2=0x7efc5c1a0dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:59:34.261 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:34.277+0000 7efc5aa00640 1 --2- 172.21.15.107:0/1591046412 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efc5c1050e0 0x7efc5c1a0dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:34.261 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:34.277+0000 7efc5aa00640 1 -- 172.21.15.107:0/1591046412 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc5c1a1440 con 0x7efc5c103530 2024-03-08T20:59:34.261 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:34.277+0000 7efc5b400640 1 --2- 172.21.15.107:0/1591046412 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efc5c1050e0 0x7efc5c1a0dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T20:59:34.262 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:34.277+0000 7efc5aa00640 1 --2- 172.21.15.107:0/1591046412 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efc5c103530 0x7efc5c199820 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7efc4c00b450 tx=0x7efc4c00b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:59:34.262 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:34.277+0000 7efc53400640 1 -- 172.21.15.107:0/1591046412 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc4c015020 con 0x7efc5c103530 2024-03-08T20:59:34.262 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:34.277+0000 7efc53400640 1 -- 172.21.15.107:0/1591046412 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7efc4c011d40 con 0x7efc5c103530 2024-03-08T20:59:34.262 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:34.277+0000 7efc53400640 1 -- 172.21.15.107:0/1591046412 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc4c00f340 con 0x7efc5c103530 2024-03-08T20:59:34.262 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:34.277+0000 7efc61c00640 1 -- 172.21.15.107:0/1591046412 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc5c1a1730 con 0x7efc5c103530 2024-03-08T20:59:34.262 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:34.277+0000 7efc61c00640 1 -- 172.21.15.107:0/1591046412 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc5c1a1cb0 con 0x7efc5c103530 2024-03-08T20:59:34.262 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:34.279+0000 7efc61c00640 1 -- 172.21.15.107:0/1591046412 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc5c10a8d0 con 0x7efc5c103530 2024-03-08T20:59:34.263 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:34.280+0000 7efc53400640 1 -- 172.21.15.107:0/1591046412 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) v1 ==== 137291+0+0 (secure 0 0 0) 0x7efc4c018020 con 0x7efc5c103530 2024-03-08T20:59:34.265 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:34.283+0000 7efc53400640 1 --2- 172.21.15.107:0/1591046412 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7efc400a2800 0x7efc400a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:34.265 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:34.283+0000 7efc5a000640 1 --2- 172.21.15.107:0/1591046412 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7efc400a2800 0x7efc400a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:59:34.265 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:34.283+0000 7efc53400640 1 -- 172.21.15.107:0/1591046412 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7efc4c0ce5b0 con 0x7efc5c103530 2024-03-08T20:59:34.265 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:34.283+0000 7efc5a000640 1 --2- 172.21.15.107:0/1591046412 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7efc400a2800 0x7efc400a4cc0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7efc5c19ad30 tx=0x7efc44000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:59:34.270 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:34.288+0000 7efc53400640 1 -- 172.21.15.107:0/1591046412 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7efc4c01f020 con 0x7efc5c103530 2024-03-08T20:59:34.467 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:34.485+0000 7efc61c00640 1 -- 172.21.15.107:0/1591046412 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efc5c061f20 con 0x7efc5c103530 2024-03-08T20:59:34.471 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:34.489+0000 7efc53400640 1 -- 172.21.15.107:0/1591046412 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+22779 (secure 0 0 0) 0x7efc4c00d040 con 0x7efc5c103530 2024-03-08T20:59:34.472 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-08T20:59:34.472 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":88,"fsid":"dc20f0f8-dd8c-11ee-95c6-87774f69a715","created":"2024-03-08T20:47:04.190034+0000","modified":"2024-03-08T20:58:21.449936+0000","last_up_change":"2024-03-08T20:56:43.135514+0000","last_in_change":"2024-03-08T20:56:17.577074+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-08T20:51:32.832786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-08T20:57:48.721227+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-08T20:57:49.707779+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8b1ddc64-4d20-4e85-ac66-f1c5ac599501","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":2453946095},{"type":"v1","addr":"172.21.15.107:6803","nonce":2453946095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":2453946095},{"type":"v1","addr":"172.21.15.107:6805","nonce":2453946095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":2453946095},{"type":"v1","addr":"172.21.15.107:6809","nonce":2453946095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":2453946095},{"type":"v1","addr":"172.21.15.107:6807","nonce":2453946095}]},"public_addr":"172.21.15.107:6803/2453946095","cluster_addr":"172.21.15.107:6805/2453946095","heartbeat_back_addr":"172.21.15.107:6809/2453946095","heartbeat_front_addr":"172.21.15.107:6807/2453946095","state":["exists","up"]},{"osd":1,"uuid":"5a803c09-202a-4777-ba2a-1da8eaf26a54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":3071316419},{"type":"v1","addr":"172.21.15.149:6801","nonce":3071316419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":3071316419},{"type":"v1","addr":"172.21.15.149:6803","nonce":3071316419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":3071316419},{"type":"v1","addr":"172.21.15.149:6807","nonce":3071316419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":3071316419},{"type":"v1","addr":"172.21.15.149:6805","nonce":3071316419}]},"public_addr":"172.21.15.149:6801/3071316419","cluster_addr":"172.21.15.149:6803/3071316419","heartbeat_back_addr":"172.21.15.149:6807/3071316419","heartbeat_front_addr":"172.21.15.149:6805/3071316419","state":["exists","up"]},{"osd":2,"uuid":"555c634f-347e-47dd-a943-e50ae2be073c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":1087852630},{"type":"v1","addr":"172.21.15.204:6801","nonce":1087852630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":1087852630},{"type":"v1","addr":"172.21.15.204:6803","nonce":1087852630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":1087852630},{"type":"v1","addr":"172.21.15.204:6807","nonce":1087852630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":1087852630},{"type":"v1","addr":"172.21.15.204:6805","nonce":1087852630}]},"public_addr":"172.21.15.204:6801/1087852630","cluster_addr":"172.21.15.204:6803/1087852630","heartbeat_back_addr":"172.21.15.204:6807/1087852630","heartbeat_front_addr":"172.21.15.204:6805/1087852630","state":["exists","up"]},{"osd":3,"uuid":"9e433d97-e202-48e0-9ed1-f0988fa14307","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":3123579789},{"type":"v1","addr":"172.21.15.107:6811","nonce":3123579789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":3123579789},{"type":"v1","addr":"172.21.15.107:6813","nonce":3123579789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":3123579789},{"type":"v1","addr":"172.21.15.107:6817","nonce":3123579789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":3123579789},{"type":"v1","addr":"172.21.15.107:6815","nonce":3123579789}]},"public_addr":"172.21.15.107:6811/3123579789","cluster_addr":"172.21.15.107:6813/3123579789","heartbeat_back_addr":"172.21.15.107:6817/3123579789","heartbeat_front_addr":"172.21.15.107:6815/3123579789","state":["exists","up"]},{"osd":4,"uuid":"27c53794-9d2b-487d-bfcf-ac4fd2ab061e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":726286918},{"type":"v1","addr":"172.21.15.149:6809","nonce":726286918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":726286918},{"type":"v1","addr":"172.21.15.149:6811","nonce":726286918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":726286918},{"type":"v1","addr":"172.21.15.149:6815","nonce":726286918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":726286918},{"type":"v1","addr":"172.21.15.149:6813","nonce":726286918}]},"public_addr":"172.21.15.149:6809/726286918","cluster_addr":"172.21.15.149:6811/726286918","heartbeat_back_addr":"172.21.15.149:6815/726286918","heartbeat_front_addr":"172.21.15.149:6813/726286918","state":["exists","up"]},{"osd":5,"uuid":"3a7778fb-46c5-40c0-842e-e988c45ea24d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":3143887820},{"type":"v1","addr":"172.21.15.204:6809","nonce":3143887820}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":3143887820},{"type":"v1","addr":"172.21.15.204:6811","nonce":3143887820}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":3143887820},{"type":"v1","addr":"172.21.15.204:6815","nonce":3143887820}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":3143887820},{"type":"v1","addr":"172.21.15.204:6813","nonce":3143887820}]},"public_addr":"172.21.15.204:6809/3143887820","cluster_addr":"172.21.15.204:6811/3143887820","heartbeat_back_addr":"172.21.15.204:6815/3143887820","heartbeat_front_addr":"172.21.15.204:6813/3143887820","state":["exists","up"]},{"osd":6,"uuid":"d2e05a56-be4f-4435-9b76-ff9aa01e4ac3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":2912452950},{"type":"v1","addr":"172.21.15.107:6819","nonce":2912452950}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":2912452950},{"type":"v1","addr":"172.21.15.107:6821","nonce":2912452950}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":2912452950},{"type":"v1","addr":"172.21.15.107:6825","nonce":2912452950}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":2912452950},{"type":"v1","addr":"172.21.15.107:6823","nonce":2912452950}]},"public_addr":"172.21.15.107:6819/2912452950","cluster_addr":"172.21.15.107:6821/2912452950","heartbeat_back_addr":"172.21.15.107:6825/2912452950","heartbeat_front_addr":"172.21.15.107:6823/2912452950","state":["exists","up"]},{"osd":7,"uuid":"a7dd26ee-c118-4a9a-9e19-142c2680902d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":3835863809},{"type":"v1","addr":"172.21.15.149:6817","nonce":3835863809}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":3835863809},{"type":"v1","addr":"172.21.15.149:6819","nonce":3835863809}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":3835863809},{"type":"v1","addr":"172.21.15.149:6823","nonce":3835863809}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":3835863809},{"type":"v1","addr":"172.21.15.149:6821","nonce":3835863809}]},"public_addr":"172.21.15.149:6817/3835863809","cluster_addr":"172.21.15.149:6819/3835863809","heartbeat_back_addr":"172.21.15.149:6823/3835863809","heartbeat_front_addr":"172.21.15.149:6821/3835863809","state":["exists","up"]},{"osd":8,"uuid":"baaaa8dd-0e1f-4bae-86bf-9e429d985f23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":1010601477},{"type":"v1","addr":"172.21.15.204:6817","nonce":1010601477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":1010601477},{"type":"v1","addr":"172.21.15.204:6819","nonce":1010601477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":1010601477},{"type":"v1","addr":"172.21.15.204:6823","nonce":1010601477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":1010601477},{"type":"v1","addr":"172.21.15.204:6821","nonce":1010601477}]},"public_addr":"172.21.15.204:6817/1010601477","cluster_addr":"172.21.15.204:6819/1010601477","heartbeat_back_addr":"172.21.15.204:6823/1010601477","heartbeat_front_addr":"172.21.15.204:6821/1010601477","state":["exists","up"]},{"osd":9,"uuid":"8df80dfa-e7ff-410e-824c-b1dce111b02f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":3334759819},{"type":"v1","addr":"172.21.15.107:6827","nonce":3334759819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3334759819},{"type":"v1","addr":"172.21.15.107:6829","nonce":3334759819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":3334759819},{"type":"v1","addr":"172.21.15.107:6833","nonce":3334759819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":3334759819},{"type":"v1","addr":"172.21.15.107:6831","nonce":3334759819}]},"public_addr":"172.21.15.107:6827/3334759819","cluster_addr":"172.21.15.107:6829/3334759819","heartbeat_back_addr":"172.21.15.107:6833/3334759819","heartbeat_front_addr":"172.21.15.107:6831/3334759819","state":["exists","up"]},{"osd":10,"uuid":"338945b2-20c0-40b2-aaca-062cd4adee04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":3469284631},{"type":"v1","addr":"172.21.15.149:6825","nonce":3469284631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":3469284631},{"type":"v1","addr":"172.21.15.149:6827","nonce":3469284631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":3469284631},{"type":"v1","addr":"172.21.15.149:6831","nonce":3469284631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":3469284631},{"type":"v1","addr":"172.21.15.149:6829","nonce":3469284631}]},"public_addr":"172.21.15.149:6825/3469284631","cluster_addr":"172.21.15.149:6827/3469284631","heartbeat_back_addr":"172.21.15.149:6831/3469284631","heartbeat_front_addr":"172.21.15.149:6829/3469284631","state":["exists","up"]},{"osd":11,"uuid":"acccfe4f-d307-4a64-8f4d-fba27bfbe09d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":3097772473},{"type":"v1","addr":"172.21.15.204:6825","nonce":3097772473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":3097772473},{"type":"v1","addr":"172.21.15.204:6827","nonce":3097772473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":3097772473},{"type":"v1","addr":"172.21.15.204:6831","nonce":3097772473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":3097772473},{"type":"v1","addr":"172.21.15.204:6829","nonce":3097772473}]},"public_addr":"172.21.15.204:6825/3097772473","cluster_addr":"172.21.15.204:6827/3097772473","heartbeat_back_addr":"172.21.15.204:6831/3097772473","heartbeat_front_addr":"172.21.15.204:6829/3097772473","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T20:50:25.862753+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T20:50:58.167663+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T20:51:30.312816+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T20:52:05.487917+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T20:52:39.348354+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T20:53:12.899002+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T20:53:47.630837+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T20:54:21.444552+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T20:54:57.101160+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T20:55:31.841786+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T20:56:05.827161+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T20:56:40.611097+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.0","mappings":[{"from":4,"to":6}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.3b","mappings":[{"from":3,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.107:0/1514666962":"2024-03-09T20:48:20.149406+0000","172.21.15.107:0/1886016707":"2024-03-09T20:48:20.149406+0000","172.21.15.107:6801/426372446":"2024-03-09T20:48:20.149406+0000","172.21.15.107:6800/426372446":"2024-03-09T20:48:20.149406+0000","172.21.15.107:0/2392547434":"2024-03-09T20:47:45.706790+0000","172.21.15.107:0/1206995985":"2024-03-09T20:47:45.706790+0000","172.21.15.107:6800/3549936519":"2024-03-09T20:47:45.706790+0000","172.21.15.107:6801/3549936519":"2024-03-09T20:47:45.706790+0000","172.21.15.107:0/2482452215":"2024-03-09T20:47:45.706790+0000","172.21.15.107:0/808124991":"2024-03-09T20:48:20.149406+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-08T20:59:34.474 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:34.491+0000 7efc61c00640 1 -- 172.21.15.107:0/1591046412 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7efc400a2800 msgr2=0x7efc400a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:59:34.474 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:34.491+0000 7efc61c00640 1 --2- 172.21.15.107:0/1591046412 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7efc400a2800 0x7efc400a4cc0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7efc5c19ad30 tx=0x7efc44000bc0 comp rx=0 tx=0).stop 2024-03-08T20:59:34.474 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:34.491+0000 7efc61c00640 1 -- 172.21.15.107:0/1591046412 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efc5c103530 msgr2=0x7efc5c199820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:59:34.474 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:34.491+0000 7efc61c00640 1 --2- 172.21.15.107:0/1591046412 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efc5c103530 0x7efc5c199820 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7efc4c00b450 tx=0x7efc4c00b910 comp rx=0 tx=0).stop 2024-03-08T20:59:34.474 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:34.491+0000 7efc61c00640 1 -- 172.21.15.107:0/1591046412 shutdown_connections 2024-03-08T20:59:34.474 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:34.491+0000 7efc61c00640 1 --2- 172.21.15.107:0/1591046412 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7efc400a2800 0x7efc400a4cc0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:34.474 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:34.491+0000 7efc61c00640 1 --2- 172.21.15.107:0/1591046412 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efc5c1050e0 0x7efc5c1a0dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:34.474 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:34.491+0000 7efc61c00640 1 --2- 172.21.15.107:0/1591046412 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efc5c104720 0x7efc5c199d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:34.474 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:34.491+0000 7efc61c00640 1 --2- 172.21.15.107:0/1591046412 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efc5c103530 0x7efc5c199820 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:34.474 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:34.491+0000 7efc61c00640 1 -- 172.21.15.107:0/1591046412 >> 172.21.15.107:0/1591046412 conn(0x7efc5c0feae0 msgr2=0x7efc5c107aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:59:34.475 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:34.492+0000 7efc61c00640 1 -- 172.21.15.107:0/1591046412 shutdown_connections 2024-03-08T20:59:34.475 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:34.492+0000 7efc61c00640 1 -- 172.21.15.107:0/1591046412 wait complete. 2024-03-08T20:59:34.493 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-08T20:51:32.832786+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-08T20:57:48.721227+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-08T20:57:49.707779+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-08T20:59:34.494 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-08T20:59:34.709 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:34.726+0000 7f56b8c00640 1 -- 172.21.15.107:0/1970639115 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f56b410cf10 msgr2=0x7f56b410f300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:59:34.709 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:34.726+0000 7f56b8c00640 1 --2- 172.21.15.107:0/1970639115 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f56b410cf10 0x7f56b410f300 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f569c009900 tx=0x7f569c02bc30 comp rx=0 tx=0).stop 2024-03-08T20:59:34.709 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:34.726+0000 7f56b8c00640 1 -- 172.21.15.107:0/1970639115 shutdown_connections 2024-03-08T20:59:34.709 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:34.726+0000 7f56b8c00640 1 --2- 172.21.15.107:0/1970639115 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f56b410f840 0x7f56b4111c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:34.709 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:34.726+0000 7f56b8c00640 1 --2- 172.21.15.107:0/1970639115 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f56b410cf10 0x7f56b410f300 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:34.709 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:34.726+0000 7f56b8c00640 1 --2- 172.21.15.107:0/1970639115 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f56b410a5e0 0x7f56b410c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:34.709 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:34.726+0000 7f56b8c00640 1 -- 172.21.15.107:0/1970639115 >> 172.21.15.107:0/1970639115 conn(0x7f56b406f180 msgr2=0x7f56b40715e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:59:34.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:34.726+0000 7f56b8c00640 1 -- 172.21.15.107:0/1970639115 shutdown_connections 2024-03-08T20:59:34.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:34.726+0000 7f56b8c00640 1 -- 172.21.15.107:0/1970639115 wait complete. 2024-03-08T20:59:34.712 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:34.727+0000 7f56b8c00640 1 Processor -- start 2024-03-08T20:59:34.712 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:34.727+0000 7f56b8c00640 1 -- start start 2024-03-08T20:59:34.712 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:34.728+0000 7f56b8c00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f56b410a5e0 0x7f56b419fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:34.712 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:34.728+0000 7f56b8c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f56b410cf10 0x7f56b41a0280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:34.712 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:34.728+0000 7f56b8c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f56b410f840 0x7f56b41a72f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:34.712 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:34.728+0000 7f56b8c00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56b4115600 con 0x7f56b410f840 2024-03-08T20:59:34.712 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:34.728+0000 7f56b8c00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56b4115480 con 0x7f56b410a5e0 2024-03-08T20:59:34.713 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:34.728+0000 7f56b8c00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56b4115780 con 0x7f56b410cf10 2024-03-08T20:59:34.713 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:34.728+0000 7f56b2800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f56b410f840 0x7f56b41a72f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:59:34.713 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:34.728+0000 7f56b2800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f56b410f840 0x7f56b41a72f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:55366/0 (socket says 172.21.15.107:55366) 2024-03-08T20:59:34.713 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:34.728+0000 7f56b2800640 1 -- 172.21.15.107:0/1806539752 learned_addr learned my addr 172.21.15.107:0/1806539752 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T20:59:34.713 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:34.728+0000 7f56b1e00640 1 --2- 172.21.15.107:0/1806539752 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f56b410a5e0 0x7f56b419fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:59:34.713 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:34.728+0000 7f56b1400640 1 --2- 172.21.15.107:0/1806539752 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f56b410cf10 0x7f56b41a0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:59:34.713 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:34.728+0000 7f56b2800640 1 -- 172.21.15.107:0/1806539752 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f56b410cf10 msgr2=0x7f56b41a0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:59:34.713 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:34.728+0000 7f56b2800640 1 --2- 172.21.15.107:0/1806539752 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f56b410cf10 0x7f56b41a0280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:34.713 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:34.728+0000 7f56b2800640 1 -- 172.21.15.107:0/1806539752 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f56b410a5e0 msgr2=0x7f56b419fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:59:34.713 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:34.728+0000 7f56b2800640 1 --2- 172.21.15.107:0/1806539752 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f56b410a5e0 0x7f56b419fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:34.713 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:34.728+0000 7f56b2800640 1 -- 172.21.15.107:0/1806539752 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56b41a7960 con 0x7f56b410f840 2024-03-08T20:59:34.713 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:34.729+0000 7f56b2800640 1 --2- 172.21.15.107:0/1806539752 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f56b410f840 0x7f56b41a72f0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f56ac00cad0 tx=0x7f56ac00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:59:34.714 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:34.729+0000 7f56aaa00640 1 -- 172.21.15.107:0/1806539752 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56ac00ae20 con 0x7f56b410f840 2024-03-08T20:59:34.714 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:34.729+0000 7f56aaa00640 1 -- 172.21.15.107:0/1806539752 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f56ac005020 con 0x7f56b410f840 2024-03-08T20:59:34.714 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:34.729+0000 7f56b8c00640 1 -- 172.21.15.107:0/1806539752 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56b41a7c50 con 0x7f56b410f840 2024-03-08T20:59:34.714 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:34.729+0000 7f56aaa00640 1 -- 172.21.15.107:0/1806539752 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56ac00f6b0 con 0x7f56b410f840 2024-03-08T20:59:34.714 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:34.729+0000 7f56b8c00640 1 -- 172.21.15.107:0/1806539752 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56b41a8120 con 0x7f56b410f840 2024-03-08T20:59:34.714 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:34.730+0000 7f56b8c00640 1 -- 172.21.15.107:0/1806539752 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56b4070c70 con 0x7f56b410f840 2024-03-08T20:59:34.715 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:34.732+0000 7f56aaa00640 1 -- 172.21.15.107:0/1806539752 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f56ac0051d0 con 0x7f56b410f840 2024-03-08T20:59:34.716 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:34.734+0000 7f56aaa00640 1 --2- 172.21.15.107:0/1806539752 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f56980a2800 0x7f56980a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:34.717 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:34.734+0000 7f56aaa00640 1 -- 172.21.15.107:0/1806539752 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f56ac0cebc0 con 0x7f56b410f840 2024-03-08T20:59:34.717 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:34.734+0000 7f56b1e00640 1 --2- 172.21.15.107:0/1806539752 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f56980a2800 0x7f56980a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:59:34.717 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:34.735+0000 7f56b1e00640 1 --2- 172.21.15.107:0/1806539752 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f56980a2800 0x7f56980a4cc0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f56a4005e10 tx=0x7f56a4005da0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:59:34.721 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:34.739+0000 7f56aaa00640 1 -- 172.21.15.107:0/1806539752 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f56ac014030 con 0x7f56b410f840 2024-03-08T20:59:34.878 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:59:34 smithi204 ceph-mon[34935]: from='client.? 172.21.15.107:0/1736516860' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-08T20:59:34.878 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:59:34 smithi204 ceph-mon[34935]: pgmap v415: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 255 B/s wr, 10 op/s 2024-03-08T20:59:34.878 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:59:34 smithi204 ceph-mon[34935]: from='client.? 172.21.15.107:0/1591046412' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-08T20:59:34.925 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:34.943+0000 7f56b8c00640 1 -- 172.21.15.107:0/1806539752 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f56b4061f90 con 0x7f56b410f840 2024-03-08T20:59:34.926 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:34.944+0000 7f56aaa00640 1 -- 172.21.15.107:0/1806539752 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) v1 ==== 93+0+10 (secure 0 0 0) 0x7f56ac09bc20 con 0x7f56b410f840 2024-03-08T20:59:34.926 INFO:teuthology.orchestra.run.smithi107.stdout:pg_num: 1 2024-03-08T20:59:34.929 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:34.946+0000 7f56b8c00640 1 -- 172.21.15.107:0/1806539752 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f56980a2800 msgr2=0x7f56980a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:59:34.929 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:34.946+0000 7f56b8c00640 1 --2- 172.21.15.107:0/1806539752 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f56980a2800 0x7f56980a4cc0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f56a4005e10 tx=0x7f56a4005da0 comp rx=0 tx=0).stop 2024-03-08T20:59:34.929 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:34.946+0000 7f56b8c00640 1 -- 172.21.15.107:0/1806539752 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f56b410f840 msgr2=0x7f56b41a72f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:59:34.930 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:34.946+0000 7f56b8c00640 1 --2- 172.21.15.107:0/1806539752 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f56b410f840 0x7f56b41a72f0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f56ac00cad0 tx=0x7f56ac00cf90 comp rx=0 tx=0).stop 2024-03-08T20:59:34.930 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:34.947+0000 7f56b8c00640 1 -- 172.21.15.107:0/1806539752 shutdown_connections 2024-03-08T20:59:34.930 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:34.947+0000 7f56b8c00640 1 --2- 172.21.15.107:0/1806539752 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f56980a2800 0x7f56980a4cc0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:34.930 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:34.947+0000 7f56b8c00640 1 --2- 172.21.15.107:0/1806539752 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f56b410f840 0x7f56b41a72f0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:34.930 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:34.947+0000 7f56b8c00640 1 --2- 172.21.15.107:0/1806539752 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f56b410cf10 0x7f56b41a0280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:34.930 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:34.947+0000 7f56b8c00640 1 --2- 172.21.15.107:0/1806539752 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f56b410a5e0 0x7f56b419fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:34.930 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:34.947+0000 7f56b8c00640 1 -- 172.21.15.107:0/1806539752 >> 172.21.15.107:0/1806539752 conn(0x7f56b406f180 msgr2=0x7f56b410b000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:59:34.930 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:34.947+0000 7f56b8c00640 1 -- 172.21.15.107:0/1806539752 shutdown_connections 2024-03-08T20:59:34.930 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:34.947+0000 7f56b8c00640 1 -- 172.21.15.107:0/1806539752 wait complete. 2024-03-08T20:59:34.945 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:59:34 smithi107 ceph-mon[31381]: from='client.? 172.21.15.107:0/1736516860' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-08T20:59:34.945 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:59:34 smithi107 ceph-mon[31381]: pgmap v415: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 255 B/s wr, 10 op/s 2024-03-08T20:59:34.945 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:59:34 smithi107 ceph-mon[31381]: from='client.? 172.21.15.107:0/1591046412' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-08T20:59:34.948 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-08T20:59:35.015 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:59:34 smithi149 ceph-mon[35042]: from='client.? 172.21.15.107:0/1736516860' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-08T20:59:35.015 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:59:34 smithi149 ceph-mon[35042]: pgmap v415: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 255 B/s wr, 10 op/s 2024-03-08T20:59:35.015 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:59:34 smithi149 ceph-mon[35042]: from='client.? 172.21.15.107:0/1591046412' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-08T20:59:35.154 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:35.171+0000 7fa376200640 1 -- 172.21.15.107:0/3914317900 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa3701093c0 msgr2=0x7fa37010fc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:59:35.154 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:35.171+0000 7fa376200640 1 --2- 172.21.15.107:0/3914317900 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa3701093c0 0x7fa37010fc50 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fa368009ca0 tx=0x7fa36802c100 comp rx=0 tx=0).stop 2024-03-08T20:59:35.155 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:35.171+0000 7fa376200640 1 -- 172.21.15.107:0/3914317900 shutdown_connections 2024-03-08T20:59:35.155 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:35.171+0000 7fa376200640 1 --2- 172.21.15.107:0/3914317900 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa3701093c0 0x7fa37010fc50 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:35.155 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:35.171+0000 7fa376200640 1 --2- 172.21.15.107:0/3914317900 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa370108a00 0x7fa370108e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:35.155 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:35.171+0000 7fa376200640 1 --2- 172.21.15.107:0/3914317900 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa370107810 0x7fa370107c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:35.155 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:35.171+0000 7fa376200640 1 -- 172.21.15.107:0/3914317900 >> 172.21.15.107:0/3914317900 conn(0x7fa3700767c0 msgr2=0x7fa370078be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:59:35.155 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:35.171+0000 7fa376200640 1 -- 172.21.15.107:0/3914317900 shutdown_connections 2024-03-08T20:59:35.155 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:35.171+0000 7fa376200640 1 -- 172.21.15.107:0/3914317900 wait complete. 2024-03-08T20:59:35.157 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:35.172+0000 7fa376200640 1 Processor -- start 2024-03-08T20:59:35.157 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:35.172+0000 7fa376200640 1 -- start start 2024-03-08T20:59:35.157 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:35.173+0000 7fa376200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa370107810 0x7fa37019da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:35.157 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:35.173+0000 7fa376200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa370108a00 0x7fa37019df60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:35.158 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:35.173+0000 7fa376200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa3701093c0 0x7fa3701a4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:35.158 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:35.173+0000 7fa376200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3701133b0 con 0x7fa370108a00 2024-03-08T20:59:35.158 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:35.173+0000 7fa376200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa370113230 con 0x7fa3701093c0 2024-03-08T20:59:35.158 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:35.173+0000 7fa376200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa370113530 con 0x7fa370107810 2024-03-08T20:59:35.158 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:35.173+0000 7fa36fe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa370108a00 0x7fa37019df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:59:35.158 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:35.173+0000 7fa36fe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa370108a00 0x7fa37019df60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:55390/0 (socket says 172.21.15.107:55390) 2024-03-08T20:59:35.158 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:35.173+0000 7fa36fe00640 1 -- 172.21.15.107:0/1437108986 learned_addr learned my addr 172.21.15.107:0/1437108986 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T20:59:35.158 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:35.173+0000 7fa374e00640 1 --2- 172.21.15.107:0/1437108986 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa370107810 0x7fa37019da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:59:35.158 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:35.173+0000 7fa375800640 1 --2- 172.21.15.107:0/1437108986 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa3701093c0 0x7fa3701a4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:59:35.158 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:35.173+0000 7fa36fe00640 1 -- 172.21.15.107:0/1437108986 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa370107810 msgr2=0x7fa37019da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:59:35.159 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:35.173+0000 7fa36fe00640 1 --2- 172.21.15.107:0/1437108986 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa370107810 0x7fa37019da20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:35.159 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:35.173+0000 7fa36fe00640 1 -- 172.21.15.107:0/1437108986 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa3701093c0 msgr2=0x7fa3701a4fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:59:35.159 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:35.173+0000 7fa36fe00640 1 --2- 172.21.15.107:0/1437108986 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa3701093c0 0x7fa3701a4fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:35.159 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:35.173+0000 7fa36fe00640 1 -- 172.21.15.107:0/1437108986 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3701a5640 con 0x7fa370108a00 2024-03-08T20:59:35.159 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:35.174+0000 7fa374e00640 1 --2- 172.21.15.107:0/1437108986 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa370107810 0x7fa37019da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T20:59:35.159 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:35.174+0000 7fa36fe00640 1 --2- 172.21.15.107:0/1437108986 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa370108a00 0x7fa37019df60 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fa35c00c890 tx=0x7fa35c00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:59:35.159 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:35.174+0000 7fa36d600640 1 -- 172.21.15.107:0/1437108986 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa35c004220 con 0x7fa370108a00 2024-03-08T20:59:35.159 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:35.174+0000 7fa36d600640 1 -- 172.21.15.107:0/1437108986 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa35c0043c0 con 0x7fa370108a00 2024-03-08T20:59:35.159 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:35.174+0000 7fa36d600640 1 -- 172.21.15.107:0/1437108986 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa35c004a00 con 0x7fa370108a00 2024-03-08T20:59:35.159 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:35.174+0000 7fa376200640 1 -- 172.21.15.107:0/1437108986 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3701a5930 con 0x7fa370108a00 2024-03-08T20:59:35.159 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:35.174+0000 7fa376200640 1 -- 172.21.15.107:0/1437108986 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3701a5eb0 con 0x7fa370108a00 2024-03-08T20:59:35.160 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:35.175+0000 7fa376200640 1 -- 172.21.15.107:0/1437108986 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa37010ebb0 con 0x7fa370108a00 2024-03-08T20:59:35.160 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:35.178+0000 7fa36d600640 1 -- 172.21.15.107:0/1437108986 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) v1 ==== 137291+0+0 (secure 0 0 0) 0x7fa35c004ba0 con 0x7fa370108a00 2024-03-08T20:59:35.163 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:35.180+0000 7fa36d600640 1 --2- 172.21.15.107:0/1437108986 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fa3580a2800 0x7fa3580a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:35.163 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:35.180+0000 7fa36d600640 1 -- 172.21.15.107:0/1437108986 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fa35c0ce820 con 0x7fa370108a00 2024-03-08T20:59:35.163 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:35.180+0000 7fa374e00640 1 --2- 172.21.15.107:0/1437108986 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fa3580a2800 0x7fa3580a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:59:35.163 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:35.181+0000 7fa374e00640 1 --2- 172.21.15.107:0/1437108986 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fa3580a2800 0x7fa3580a4cc0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fa364006fd0 tx=0x7fa364008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:59:35.167 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:35.184+0000 7fa36d600640 1 -- 172.21.15.107:0/1437108986 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7fa35c014030 con 0x7fa370108a00 2024-03-08T20:59:35.364 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:35.382+0000 7fa376200640 1 -- 172.21.15.107:0/1437108986 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa370061ef0 con 0x7fa370108a00 2024-03-08T20:59:35.365 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:35.383+0000 7fa36d600640 1 -- 172.21.15.107:0/1437108986 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa35c09b880 con 0x7fa370108a00 2024-03-08T20:59:35.365 INFO:teuthology.orchestra.run.smithi107.stdout:pg_num: 32 2024-03-08T20:59:35.368 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:35.386+0000 7fa376200640 1 -- 172.21.15.107:0/1437108986 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fa3580a2800 msgr2=0x7fa3580a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:59:35.368 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:35.386+0000 7fa376200640 1 --2- 172.21.15.107:0/1437108986 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fa3580a2800 0x7fa3580a4cc0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fa364006fd0 tx=0x7fa364008040 comp rx=0 tx=0).stop 2024-03-08T20:59:35.368 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:35.386+0000 7fa376200640 1 -- 172.21.15.107:0/1437108986 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa370108a00 msgr2=0x7fa37019df60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:59:35.368 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:35.386+0000 7fa376200640 1 --2- 172.21.15.107:0/1437108986 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa370108a00 0x7fa37019df60 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fa35c00c890 tx=0x7fa35c00cd50 comp rx=0 tx=0).stop 2024-03-08T20:59:35.369 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:35.386+0000 7fa376200640 1 -- 172.21.15.107:0/1437108986 shutdown_connections 2024-03-08T20:59:35.369 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:35.386+0000 7fa376200640 1 --2- 172.21.15.107:0/1437108986 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fa3580a2800 0x7fa3580a4cc0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:35.369 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:35.386+0000 7fa376200640 1 --2- 172.21.15.107:0/1437108986 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa3701093c0 0x7fa3701a4fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:35.369 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:35.386+0000 7fa376200640 1 --2- 172.21.15.107:0/1437108986 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa370108a00 0x7fa37019df60 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:35.369 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:35.386+0000 7fa376200640 1 --2- 172.21.15.107:0/1437108986 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa370107810 0x7fa37019da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:35.369 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:35.386+0000 7fa376200640 1 -- 172.21.15.107:0/1437108986 >> 172.21.15.107:0/1437108986 conn(0x7fa3700767c0 msgr2=0x7fa37010bd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:59:35.369 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:35.386+0000 7fa376200640 1 -- 172.21.15.107:0/1437108986 shutdown_connections 2024-03-08T20:59:35.369 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:35.386+0000 7fa376200640 1 -- 172.21.15.107:0/1437108986 wait complete. 2024-03-08T20:59:35.387 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-08T20:59:35.607 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:35.624+0000 7fa33d400640 1 -- 172.21.15.107:0/3029826515 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa338104720 msgr2=0x7fa338104ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:59:35.607 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:35.624+0000 7fa33d400640 1 --2- 172.21.15.107:0/3029826515 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa338104720 0x7fa338104ba0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fa320009900 tx=0x7fa32002bc30 comp rx=0 tx=0).stop 2024-03-08T20:59:35.607 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:35.625+0000 7fa33d400640 1 -- 172.21.15.107:0/3029826515 shutdown_connections 2024-03-08T20:59:35.608 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:35.625+0000 7fa33d400640 1 --2- 172.21.15.107:0/3029826515 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa3381050e0 0x7fa33810b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:35.608 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:35.625+0000 7fa33d400640 1 --2- 172.21.15.107:0/3029826515 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa338104720 0x7fa338104ba0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:35.608 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:35.625+0000 7fa33d400640 1 --2- 172.21.15.107:0/3029826515 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa338103530 0x7fa338103950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:35.608 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:35.625+0000 7fa33d400640 1 -- 172.21.15.107:0/3029826515 >> 172.21.15.107:0/3029826515 conn(0x7fa3380feaa0 msgr2=0x7fa338100ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:59:35.608 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:35.625+0000 7fa33d400640 1 -- 172.21.15.107:0/3029826515 shutdown_connections 2024-03-08T20:59:35.608 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:35.625+0000 7fa33d400640 1 -- 172.21.15.107:0/3029826515 wait complete. 2024-03-08T20:59:35.611 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:35.626+0000 7fa33d400640 1 Processor -- start 2024-03-08T20:59:35.611 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:35.626+0000 7fa33d400640 1 -- start start 2024-03-08T20:59:35.611 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:35.626+0000 7fa33d400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa338103530 0x7fa338199720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:35.611 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:35.626+0000 7fa33d400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa338104720 0x7fa338199c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:35.611 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:35.626+0000 7fa33d400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa3381050e0 0x7fa3381a0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:35.611 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:35.626+0000 7fa33d400640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa33810f100 con 0x7fa3381050e0 2024-03-08T20:59:35.611 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:35.626+0000 7fa33d400640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa33810ef80 con 0x7fa338103530 2024-03-08T20:59:35.611 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:35.626+0000 7fa33d400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa33810f280 con 0x7fa338104720 2024-03-08T20:59:35.611 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:35.626+0000 7fa337200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa3381050e0 0x7fa3381a0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:59:35.611 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:35.626+0000 7fa337200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa3381050e0 0x7fa3381a0cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:55408/0 (socket says 172.21.15.107:55408) 2024-03-08T20:59:35.611 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:35.626+0000 7fa337200640 1 -- 172.21.15.107:0/4282867972 learned_addr learned my addr 172.21.15.107:0/4282867972 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T20:59:35.611 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:35.627+0000 7fa32de00640 1 --2- 172.21.15.107:0/4282867972 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa338104720 0x7fa338199c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:59:35.612 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:35.627+0000 7fa336800640 1 --2- 172.21.15.107:0/4282867972 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa338103530 0x7fa338199720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:59:35.612 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:35.627+0000 7fa337200640 1 -- 172.21.15.107:0/4282867972 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa338104720 msgr2=0x7fa338199c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:59:35.612 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:35.627+0000 7fa337200640 1 --2- 172.21.15.107:0/4282867972 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa338104720 0x7fa338199c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:35.612 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:35.627+0000 7fa337200640 1 -- 172.21.15.107:0/4282867972 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa338103530 msgr2=0x7fa338199720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:59:35.612 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:35.627+0000 7fa337200640 1 --2- 172.21.15.107:0/4282867972 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa338103530 0x7fa338199720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:35.612 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:35.627+0000 7fa337200640 1 -- 172.21.15.107:0/4282867972 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3381a1340 con 0x7fa3381050e0 2024-03-08T20:59:35.612 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:35.627+0000 7fa337200640 1 --2- 172.21.15.107:0/4282867972 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa3381050e0 0x7fa3381a0cd0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fa33000b970 tx=0x7fa33000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:59:35.613 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:35.627+0000 7fa32fe00640 1 -- 172.21.15.107:0/4282867972 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa330019070 con 0x7fa3381050e0 2024-03-08T20:59:35.613 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:35.627+0000 7fa32fe00640 1 -- 172.21.15.107:0/4282867972 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa330014440 con 0x7fa3381050e0 2024-03-08T20:59:35.613 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:35.627+0000 7fa33d400640 1 -- 172.21.15.107:0/4282867972 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3381a1630 con 0x7fa3381050e0 2024-03-08T20:59:35.613 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:35.627+0000 7fa32fe00640 1 -- 172.21.15.107:0/4282867972 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3300075d0 con 0x7fa3381050e0 2024-03-08T20:59:35.613 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:35.628+0000 7fa33d400640 1 -- 172.21.15.107:0/4282867972 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3381a1c00 con 0x7fa3381050e0 2024-03-08T20:59:35.613 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:35.629+0000 7fa33d400640 1 -- 172.21.15.107:0/4282867972 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa33810a850 con 0x7fa3381050e0 2024-03-08T20:59:35.613 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:35.631+0000 7fa32fe00640 1 -- 172.21.15.107:0/4282867972 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) v1 ==== 137291+0+0 (secure 0 0 0) 0x7fa33001c070 con 0x7fa3381050e0 2024-03-08T20:59:35.614 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:35.632+0000 7fa32fe00640 1 --2- 172.21.15.107:0/4282867972 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fa31c0a2800 0x7fa31c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:35.614 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:35.632+0000 7fa32fe00640 1 -- 172.21.15.107:0/4282867972 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fa3300cdfd0 con 0x7fa3381050e0 2024-03-08T20:59:35.614 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:35.632+0000 7fa336800640 1 --2- 172.21.15.107:0/4282867972 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fa31c0a2800 0x7fa31c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:59:35.615 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:35.633+0000 7fa336800640 1 --2- 172.21.15.107:0/4282867972 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fa31c0a2800 0x7fa31c0a4cc0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fa3280058c0 tx=0x7fa328005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:59:35.618 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:35.636+0000 7fa32fe00640 1 -- 172.21.15.107:0/4282867972 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7fa33009b110 con 0x7fa3381050e0 2024-03-08T20:59:35.717 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:59:35 smithi107 ceph-mon[31381]: from='client.? 172.21.15.107:0/1806539752' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-08T20:59:35.717 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:59:35 smithi107 ceph-mon[31381]: from='client.? 172.21.15.107:0/1437108986' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-08T20:59:35.814 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:35.832+0000 7fa33d400640 1 -- 172.21.15.107:0/4282867972 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa338061ef0 con 0x7fa3381050e0 2024-03-08T20:59:35.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:35.833+0000 7fa32fe00640 1 -- 172.21.15.107:0/4282867972 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) v1 ==== 100+0+11 (secure 0 0 0) 0x7fa33009aa10 con 0x7fa3381050e0 2024-03-08T20:59:35.815 INFO:teuthology.orchestra.run.smithi107.stdout:pg_num: 64 2024-03-08T20:59:35.818 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:35.836+0000 7fa33d400640 1 -- 172.21.15.107:0/4282867972 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fa31c0a2800 msgr2=0x7fa31c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:59:35.818 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:35.836+0000 7fa33d400640 1 --2- 172.21.15.107:0/4282867972 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fa31c0a2800 0x7fa31c0a4cc0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fa3280058c0 tx=0x7fa328005850 comp rx=0 tx=0).stop 2024-03-08T20:59:35.818 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:35.836+0000 7fa33d400640 1 -- 172.21.15.107:0/4282867972 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa3381050e0 msgr2=0x7fa3381a0cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:59:35.819 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:35.836+0000 7fa33d400640 1 --2- 172.21.15.107:0/4282867972 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa3381050e0 0x7fa3381a0cd0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fa33000b970 tx=0x7fa33000be30 comp rx=0 tx=0).stop 2024-03-08T20:59:35.819 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:35.836+0000 7fa33d400640 1 -- 172.21.15.107:0/4282867972 shutdown_connections 2024-03-08T20:59:35.819 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:35.836+0000 7fa33d400640 1 --2- 172.21.15.107:0/4282867972 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fa31c0a2800 0x7fa31c0a4cc0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:35.819 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:35.836+0000 7fa33d400640 1 --2- 172.21.15.107:0/4282867972 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa3381050e0 0x7fa3381a0cd0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:35.819 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:35.836+0000 7fa33d400640 1 --2- 172.21.15.107:0/4282867972 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa338104720 0x7fa338199c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:35.819 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:35.836+0000 7fa33d400640 1 --2- 172.21.15.107:0/4282867972 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa338103530 0x7fa338199720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:35.819 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:35.836+0000 7fa33d400640 1 -- 172.21.15.107:0/4282867972 >> 172.21.15.107:0/4282867972 conn(0x7fa3380feaa0 msgr2=0x7fa338107a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:59:35.819 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:35.836+0000 7fa33d400640 1 -- 172.21.15.107:0/4282867972 shutdown_connections 2024-03-08T20:59:35.819 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:35.836+0000 7fa33d400640 1 -- 172.21.15.107:0/4282867972 wait complete. 2024-03-08T20:59:35.837 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-08T20:59:35.878 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:59:35 smithi204 ceph-mon[34935]: from='client.? 172.21.15.107:0/1806539752' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-08T20:59:35.878 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:59:35 smithi204 ceph-mon[34935]: from='client.? 172.21.15.107:0/1437108986' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-08T20:59:36.015 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:59:35 smithi149 ceph-mon[35042]: from='client.? 172.21.15.107:0/1806539752' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-08T20:59:36.015 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:59:35 smithi149 ceph-mon[35042]: from='client.? 172.21.15.107:0/1437108986' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-08T20:59:36.049 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:36.067+0000 7f6926200640 1 -- 172.21.15.107:0/2413433759 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6920104720 msgr2=0x7f6920104ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:59:36.050 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:36.067+0000 7f6926200640 1 --2- 172.21.15.107:0/2413433759 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6920104720 0x7f6920104ba0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f690c009900 tx=0x7f690c02bc30 comp rx=0 tx=0).stop 2024-03-08T20:59:36.050 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:36.067+0000 7f6926200640 1 -- 172.21.15.107:0/2413433759 shutdown_connections 2024-03-08T20:59:36.050 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:36.067+0000 7f6926200640 1 --2- 172.21.15.107:0/2413433759 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f69201050e0 0x7f692010b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:36.050 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:36.067+0000 7f6926200640 1 --2- 172.21.15.107:0/2413433759 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6920104720 0x7f6920104ba0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:36.050 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:36.067+0000 7f6926200640 1 --2- 172.21.15.107:0/2413433759 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6920103530 0x7f6920103950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:36.050 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:36.067+0000 7f6926200640 1 -- 172.21.15.107:0/2413433759 >> 172.21.15.107:0/2413433759 conn(0x7f69200feae0 msgr2=0x7f6920100f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:59:36.050 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:36.067+0000 7f6926200640 1 -- 172.21.15.107:0/2413433759 shutdown_connections 2024-03-08T20:59:36.050 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:36.067+0000 7f6926200640 1 -- 172.21.15.107:0/2413433759 wait complete. 2024-03-08T20:59:36.052 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:36.068+0000 7f6926200640 1 Processor -- start 2024-03-08T20:59:36.052 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:36.068+0000 7f6926200640 1 -- start start 2024-03-08T20:59:36.053 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:36.068+0000 7f6926200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6920103530 0x7f6920199770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:36.053 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:36.068+0000 7f6926200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6920104720 0x7f6920199cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:36.053 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:36.068+0000 7f6926200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f69201050e0 0x7f69201a0d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:36.053 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:36.068+0000 7f6926200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f692010f0c0 con 0x7f69201050e0 2024-03-08T20:59:36.053 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:36.068+0000 7f6926200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f692010ef40 con 0x7f6920103530 2024-03-08T20:59:36.053 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:36.068+0000 7f6926200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f692010f240 con 0x7f6920104720 2024-03-08T20:59:36.053 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:36.069+0000 7f691fe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f69201050e0 0x7f69201a0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:59:36.053 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:36.069+0000 7f691fe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f69201050e0 0x7f69201a0d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:56852/0 (socket says 172.21.15.107:56852) 2024-03-08T20:59:36.053 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:36.069+0000 7f691fe00640 1 -- 172.21.15.107:0/1570852810 learned_addr learned my addr 172.21.15.107:0/1570852810 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T20:59:36.053 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:36.069+0000 7f691f400640 1 --2- 172.21.15.107:0/1570852810 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6920103530 0x7f6920199770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:59:36.053 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:36.069+0000 7f691ea00640 1 --2- 172.21.15.107:0/1570852810 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6920104720 0x7f6920199cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:59:36.053 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:36.069+0000 7f691fe00640 1 -- 172.21.15.107:0/1570852810 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6920104720 msgr2=0x7f6920199cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:59:36.053 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:36.069+0000 7f691fe00640 1 --2- 172.21.15.107:0/1570852810 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6920104720 0x7f6920199cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:36.054 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:36.069+0000 7f691fe00640 1 -- 172.21.15.107:0/1570852810 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6920103530 msgr2=0x7f6920199770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:59:36.054 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:36.069+0000 7f691fe00640 1 --2- 172.21.15.107:0/1570852810 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6920103530 0x7f6920199770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:36.054 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:36.069+0000 7f691fe00640 1 -- 172.21.15.107:0/1570852810 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69201a1390 con 0x7f69201050e0 2024-03-08T20:59:36.054 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:36.069+0000 7f691f400640 1 --2- 172.21.15.107:0/1570852810 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6920103530 0x7f6920199770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T20:59:36.054 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:36.069+0000 7f691fe00640 1 --2- 172.21.15.107:0/1570852810 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f69201050e0 0x7f69201a0d20 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f691800eaa0 tx=0x7f691800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:59:36.054 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:36.069+0000 7f6917e00640 1 -- 172.21.15.107:0/1570852810 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f691800cdb0 con 0x7f69201050e0 2024-03-08T20:59:36.054 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:36.070+0000 7f6926200640 1 -- 172.21.15.107:0/1570852810 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69201a1680 con 0x7f69201050e0 2024-03-08T20:59:36.054 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:36.070+0000 7f6917e00640 1 -- 172.21.15.107:0/1570852810 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f6918005020 con 0x7f69201050e0 2024-03-08T20:59:36.054 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:36.070+0000 7f6917e00640 1 -- 172.21.15.107:0/1570852810 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69180116d0 con 0x7f69201050e0 2024-03-08T20:59:36.054 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:36.070+0000 7f6926200640 1 -- 172.21.15.107:0/1570852810 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69201a1bd0 con 0x7f69201050e0 2024-03-08T20:59:36.055 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:36.071+0000 7f6926200640 1 -- 172.21.15.107:0/1570852810 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f692010a870 con 0x7f69201050e0 2024-03-08T20:59:36.055 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:36.073+0000 7f6917e00640 1 -- 172.21.15.107:0/1570852810 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f69180051d0 con 0x7f69201050e0 2024-03-08T20:59:36.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:36.075+0000 7f6917e00640 1 --2- 172.21.15.107:0/1570852810 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f69040a2880 0x7f69040a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:36.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:36.075+0000 7f6917e00640 1 -- 172.21.15.107:0/1570852810 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f69180cecd0 con 0x7f69201050e0 2024-03-08T20:59:36.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:36.075+0000 7f691f400640 1 --2- 172.21.15.107:0/1570852810 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f69040a2880 0x7f69040a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:59:36.058 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:36.076+0000 7f691f400640 1 --2- 172.21.15.107:0/1570852810 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f69040a2880 0x7f69040a4d40 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f6920104580 tx=0x7f6910005e70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:59:36.061 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:36.079+0000 7f6917e00640 1 -- 172.21.15.107:0/1570852810 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f6918016030 con 0x7f69201050e0 2024-03-08T20:59:36.303 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:36.320+0000 7f6926200640 1 -- 172.21.15.107:0/1570852810 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6920061ef0 con 0x7f69201050e0 2024-03-08T20:59:36.304 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:36.322+0000 7f6917e00640 1 -- 172.21.15.107:0/1570852810 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+11393 (secure 0 0 0) 0x7f691809bd30 con 0x7f69201050e0 2024-03-08T20:59:36.304 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-08T20:59:36.305 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/133582743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":133582743},{"type":"v1","addr":"172.21.15.107:6835","nonce":133582743}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/3798982337","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":3798982337},{"type":"v1","addr":"172.21.15.107:6837","nonce":3798982337}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-08T20:57:50.223752+0000","modified":"2024-03-08T20:57:57.874727+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34475,"mds_1":34445,"mds_2":34415,"mds_3":34397,"mds_4":24457},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14625":{"gid":14625,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6839/3070059000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":3070059000},{"type":"v1","addr":"172.21.15.107:6839","nonce":3070059000}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14637":{"gid":14637,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6841/747096194","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6840","nonce":747096194},{"type":"v1","addr":"172.21.15.107:6841","nonce":747096194}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24400":{"gid":24400,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6833/3404762782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":3404762782},{"type":"v1","addr":"172.21.15.204:6833","nonce":3404762782}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24430":{"gid":24430,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/2072518651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":2072518651},{"type":"v1","addr":"172.21.15.204:6835","nonce":2072518651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24442":{"gid":24442,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6837/42644905","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":42644905},{"type":"v1","addr":"172.21.15.204:6837","nonce":42644905}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24457":{"gid":24457,"name":"i","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.204:6839/1857316822","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":1857316822},{"type":"v1","addr":"172.21.15.204:6839","nonce":1857316822}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34397":{"gid":34397,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":14,"addr":"172.21.15.149:6833/1962161726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":1962161726},{"type":"v1","addr":"172.21.15.149:6833","nonce":1962161726}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34415":{"gid":34415,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.149:6835/442657066","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":442657066},{"type":"v1","addr":"172.21.15.149:6835","nonce":442657066}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34445":{"gid":34445,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.149:6837/914546237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":914546237},{"type":"v1","addr":"172.21.15.149:6837","nonce":914546237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34475":{"gid":34475,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.149:6839/55290596","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":55290596},{"type":"v1","addr":"172.21.15.149:6839","nonce":55290596}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-08T20:59:36.305 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 27 2024-03-08T20:59:36.308 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:36.325+0000 7f6926200640 1 -- 172.21.15.107:0/1570852810 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f69040a2880 msgr2=0x7f69040a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:59:36.308 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:36.325+0000 7f6926200640 1 --2- 172.21.15.107:0/1570852810 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f69040a2880 0x7f69040a4d40 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f6920104580 tx=0x7f6910005e70 comp rx=0 tx=0).stop 2024-03-08T20:59:36.308 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:36.325+0000 7f6926200640 1 -- 172.21.15.107:0/1570852810 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f69201050e0 msgr2=0x7f69201a0d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:59:36.308 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:36.325+0000 7f6926200640 1 --2- 172.21.15.107:0/1570852810 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f69201050e0 0x7f69201a0d20 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f691800eaa0 tx=0x7f691800ef60 comp rx=0 tx=0).stop 2024-03-08T20:59:36.308 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:36.325+0000 7f6926200640 1 -- 172.21.15.107:0/1570852810 shutdown_connections 2024-03-08T20:59:36.308 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:36.325+0000 7f6926200640 1 --2- 172.21.15.107:0/1570852810 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f69040a2880 0x7f69040a4d40 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:36.308 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:36.325+0000 7f6926200640 1 --2- 172.21.15.107:0/1570852810 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f69201050e0 0x7f69201a0d20 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:36.308 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:36.325+0000 7f6926200640 1 --2- 172.21.15.107:0/1570852810 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6920104720 0x7f6920199cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:36.308 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:36.325+0000 7f6926200640 1 --2- 172.21.15.107:0/1570852810 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6920103530 0x7f6920199770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:36.308 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:36.325+0000 7f6926200640 1 -- 172.21.15.107:0/1570852810 >> 172.21.15.107:0/1570852810 conn(0x7f69200feae0 msgr2=0x7f6920107a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:59:36.308 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:36.325+0000 7f6926200640 1 -- 172.21.15.107:0/1570852810 shutdown_connections 2024-03-08T20:59:36.309 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:36.325+0000 7f6926200640 1 -- 172.21.15.107:0/1570852810 wait complete. 2024-03-08T20:59:36.327 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-08T20:59:36.534 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:36.551+0000 7f1ca4200640 1 -- 172.21.15.107:0/552243649 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1c9c103530 msgr2=0x7f1c9c103950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:59:36.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:36.551+0000 7f1ca4200640 1 --2- 172.21.15.107:0/552243649 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1c9c103530 0x7f1c9c103950 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f1c90009900 tx=0x7f1c9002bc30 comp rx=0 tx=0).stop 2024-03-08T20:59:36.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:36.552+0000 7f1ca4200640 1 -- 172.21.15.107:0/552243649 shutdown_connections 2024-03-08T20:59:36.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:36.552+0000 7f1ca4200640 1 --2- 172.21.15.107:0/552243649 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1c9c1050e0 0x7f1c9c10b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:36.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:36.552+0000 7f1ca4200640 1 --2- 172.21.15.107:0/552243649 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1c9c104720 0x7f1c9c104ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:36.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:36.552+0000 7f1ca4200640 1 --2- 172.21.15.107:0/552243649 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1c9c103530 0x7f1c9c103950 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:36.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:36.552+0000 7f1ca4200640 1 -- 172.21.15.107:0/552243649 >> 172.21.15.107:0/552243649 conn(0x7f1c9c0feaa0 msgr2=0x7f1c9c100ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:59:36.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:36.552+0000 7f1ca4200640 1 -- 172.21.15.107:0/552243649 shutdown_connections 2024-03-08T20:59:36.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:36.552+0000 7f1ca4200640 1 -- 172.21.15.107:0/552243649 wait complete. 2024-03-08T20:59:36.538 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:36.553+0000 7f1ca4200640 1 Processor -- start 2024-03-08T20:59:36.538 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:36.553+0000 7f1ca4200640 1 -- start start 2024-03-08T20:59:36.538 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:36.553+0000 7f1ca4200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1c9c103530 0x7f1c9c1997a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:36.538 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:36.553+0000 7f1ca4200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1c9c104720 0x7f1c9c199ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:36.538 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:36.553+0000 7f1ca4200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1c9c1050e0 0x7f1c9c1a0d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:36.538 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:36.553+0000 7f1ca4200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c9c10f0c0 con 0x7f1c9c1050e0 2024-03-08T20:59:36.538 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:36.553+0000 7f1ca4200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c9c10ef40 con 0x7f1c9c104720 2024-03-08T20:59:36.539 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:36.553+0000 7f1ca4200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c9c10f240 con 0x7f1c9c103530 2024-03-08T20:59:36.539 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:36.553+0000 7f1ca2200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1c9c1050e0 0x7f1c9c1a0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:59:36.539 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:36.553+0000 7f1ca2200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1c9c1050e0 0x7f1c9c1a0d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:56882/0 (socket says 172.21.15.107:56882) 2024-03-08T20:59:36.539 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:36.553+0000 7f1ca2200640 1 -- 172.21.15.107:0/1764362259 learned_addr learned my addr 172.21.15.107:0/1764362259 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T20:59:36.539 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:36.554+0000 7f1ca1800640 1 --2- 172.21.15.107:0/1764362259 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1c9c103530 0x7f1c9c1997a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:59:36.539 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:36.554+0000 7f1ca0e00640 1 --2- 172.21.15.107:0/1764362259 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1c9c104720 0x7f1c9c199ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:59:36.539 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:36.554+0000 7f1ca2200640 1 -- 172.21.15.107:0/1764362259 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1c9c103530 msgr2=0x7f1c9c1997a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:59:36.539 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:36.554+0000 7f1ca2200640 1 --2- 172.21.15.107:0/1764362259 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1c9c103530 0x7f1c9c1997a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:36.539 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:36.554+0000 7f1ca2200640 1 -- 172.21.15.107:0/1764362259 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1c9c104720 msgr2=0x7f1c9c199ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:59:36.539 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:36.554+0000 7f1ca2200640 1 --2- 172.21.15.107:0/1764362259 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1c9c104720 0x7f1c9c199ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:36.539 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:36.554+0000 7f1ca2200640 1 -- 172.21.15.107:0/1764362259 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c9c1a13c0 con 0x7f1c9c1050e0 2024-03-08T20:59:36.539 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:36.554+0000 7f1ca0e00640 1 --2- 172.21.15.107:0/1764362259 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1c9c104720 0x7f1c9c199ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T20:59:36.539 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:36.554+0000 7f1ca2200640 1 --2- 172.21.15.107:0/1764362259 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1c9c1050e0 0x7f1c9c1a0d50 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f1c9400eaa0 tx=0x7f1c9400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:59:36.540 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:36.554+0000 7f1c9a000640 1 -- 172.21.15.107:0/1764362259 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c9400cdb0 con 0x7f1c9c1050e0 2024-03-08T20:59:36.540 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:36.554+0000 7f1c9a000640 1 -- 172.21.15.107:0/1764362259 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f1c94005020 con 0x7f1c9c1050e0 2024-03-08T20:59:36.540 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:36.554+0000 7f1ca4200640 1 -- 172.21.15.107:0/1764362259 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c9c1a16b0 con 0x7f1c9c1050e0 2024-03-08T20:59:36.540 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:36.554+0000 7f1c9a000640 1 -- 172.21.15.107:0/1764362259 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c940116d0 con 0x7f1c9c1050e0 2024-03-08T20:59:36.540 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:36.555+0000 7f1ca4200640 1 -- 172.21.15.107:0/1764362259 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c9c1a1c00 con 0x7f1c9c1050e0 2024-03-08T20:59:36.540 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:36.556+0000 7f1ca4200640 1 -- 172.21.15.107:0/1764362259 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c9c10a8c0 con 0x7f1c9c1050e0 2024-03-08T20:59:36.540 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:36.558+0000 7f1c9a000640 1 -- 172.21.15.107:0/1764362259 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f1c940051d0 con 0x7f1c9c1050e0 2024-03-08T20:59:36.541 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:36.559+0000 7f1c9a000640 1 --2- 172.21.15.107:0/1764362259 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f1c840a2800 0x7f1c840a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:36.541 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:36.559+0000 7f1c9a000640 1 -- 172.21.15.107:0/1764362259 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f1c940ced00 con 0x7f1c9c1050e0 2024-03-08T20:59:36.541 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:36.559+0000 7f1ca1800640 1 --2- 172.21.15.107:0/1764362259 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f1c840a2800 0x7f1c840a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:59:36.542 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:36.560+0000 7f1ca1800640 1 --2- 172.21.15.107:0/1764362259 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f1c840a2800 0x7f1c840a4cc0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f1c9002c240 tx=0x7f1c90005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:59:36.545 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:36.563+0000 7f1c9a000640 1 -- 172.21.15.107:0/1764362259 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f1c94016030 con 0x7f1c9c1050e0 2024-03-08T20:59:36.717 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:59:36 smithi107 ceph-mon[31381]: from='client.? 172.21.15.107:0/4282867972' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-08T20:59:36.717 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:59:36 smithi107 ceph-mon[31381]: from='client.? 172.21.15.107:0/1570852810' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-08T20:59:36.717 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:59:36 smithi107 ceph-mon[31381]: pgmap v416: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 0 B/s wr, 9 op/s 2024-03-08T20:59:36.749 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:36.767+0000 7f1ca4200640 1 -- 172.21.15.107:0/1764362259 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1c9c061ef0 con 0x7f1c9c1050e0 2024-03-08T20:59:36.751 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:36.769+0000 7f1c9a000640 1 -- 172.21.15.107:0/1764362259 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+22779 (secure 0 0 0) 0x7f1c9409bd60 con 0x7f1c9c1050e0 2024-03-08T20:59:36.752 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-08T20:59:36.752 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":88,"fsid":"dc20f0f8-dd8c-11ee-95c6-87774f69a715","created":"2024-03-08T20:47:04.190034+0000","modified":"2024-03-08T20:58:21.449936+0000","last_up_change":"2024-03-08T20:56:43.135514+0000","last_in_change":"2024-03-08T20:56:17.577074+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-08T20:51:32.832786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-08T20:57:48.721227+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-08T20:57:49.707779+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8b1ddc64-4d20-4e85-ac66-f1c5ac599501","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":2453946095},{"type":"v1","addr":"172.21.15.107:6803","nonce":2453946095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":2453946095},{"type":"v1","addr":"172.21.15.107:6805","nonce":2453946095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":2453946095},{"type":"v1","addr":"172.21.15.107:6809","nonce":2453946095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":2453946095},{"type":"v1","addr":"172.21.15.107:6807","nonce":2453946095}]},"public_addr":"172.21.15.107:6803/2453946095","cluster_addr":"172.21.15.107:6805/2453946095","heartbeat_back_addr":"172.21.15.107:6809/2453946095","heartbeat_front_addr":"172.21.15.107:6807/2453946095","state":["exists","up"]},{"osd":1,"uuid":"5a803c09-202a-4777-ba2a-1da8eaf26a54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":3071316419},{"type":"v1","addr":"172.21.15.149:6801","nonce":3071316419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":3071316419},{"type":"v1","addr":"172.21.15.149:6803","nonce":3071316419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":3071316419},{"type":"v1","addr":"172.21.15.149:6807","nonce":3071316419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":3071316419},{"type":"v1","addr":"172.21.15.149:6805","nonce":3071316419}]},"public_addr":"172.21.15.149:6801/3071316419","cluster_addr":"172.21.15.149:6803/3071316419","heartbeat_back_addr":"172.21.15.149:6807/3071316419","heartbeat_front_addr":"172.21.15.149:6805/3071316419","state":["exists","up"]},{"osd":2,"uuid":"555c634f-347e-47dd-a943-e50ae2be073c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":1087852630},{"type":"v1","addr":"172.21.15.204:6801","nonce":1087852630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":1087852630},{"type":"v1","addr":"172.21.15.204:6803","nonce":1087852630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":1087852630},{"type":"v1","addr":"172.21.15.204:6807","nonce":1087852630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":1087852630},{"type":"v1","addr":"172.21.15.204:6805","nonce":1087852630}]},"public_addr":"172.21.15.204:6801/1087852630","cluster_addr":"172.21.15.204:6803/1087852630","heartbeat_back_addr":"172.21.15.204:6807/1087852630","heartbeat_front_addr":"172.21.15.204:6805/1087852630","state":["exists","up"]},{"osd":3,"uuid":"9e433d97-e202-48e0-9ed1-f0988fa14307","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":3123579789},{"type":"v1","addr":"172.21.15.107:6811","nonce":3123579789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":3123579789},{"type":"v1","addr":"172.21.15.107:6813","nonce":3123579789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":3123579789},{"type":"v1","addr":"172.21.15.107:6817","nonce":3123579789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":3123579789},{"type":"v1","addr":"172.21.15.107:6815","nonce":3123579789}]},"public_addr":"172.21.15.107:6811/3123579789","cluster_addr":"172.21.15.107:6813/3123579789","heartbeat_back_addr":"172.21.15.107:6817/3123579789","heartbeat_front_addr":"172.21.15.107:6815/3123579789","state":["exists","up"]},{"osd":4,"uuid":"27c53794-9d2b-487d-bfcf-ac4fd2ab061e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":726286918},{"type":"v1","addr":"172.21.15.149:6809","nonce":726286918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":726286918},{"type":"v1","addr":"172.21.15.149:6811","nonce":726286918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":726286918},{"type":"v1","addr":"172.21.15.149:6815","nonce":726286918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":726286918},{"type":"v1","addr":"172.21.15.149:6813","nonce":726286918}]},"public_addr":"172.21.15.149:6809/726286918","cluster_addr":"172.21.15.149:6811/726286918","heartbeat_back_addr":"172.21.15.149:6815/726286918","heartbeat_front_addr":"172.21.15.149:6813/726286918","state":["exists","up"]},{"osd":5,"uuid":"3a7778fb-46c5-40c0-842e-e988c45ea24d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":3143887820},{"type":"v1","addr":"172.21.15.204:6809","nonce":3143887820}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":3143887820},{"type":"v1","addr":"172.21.15.204:6811","nonce":3143887820}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":3143887820},{"type":"v1","addr":"172.21.15.204:6815","nonce":3143887820}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":3143887820},{"type":"v1","addr":"172.21.15.204:6813","nonce":3143887820}]},"public_addr":"172.21.15.204:6809/3143887820","cluster_addr":"172.21.15.204:6811/3143887820","heartbeat_back_addr":"172.21.15.204:6815/3143887820","heartbeat_front_addr":"172.21.15.204:6813/3143887820","state":["exists","up"]},{"osd":6,"uuid":"d2e05a56-be4f-4435-9b76-ff9aa01e4ac3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":2912452950},{"type":"v1","addr":"172.21.15.107:6819","nonce":2912452950}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":2912452950},{"type":"v1","addr":"172.21.15.107:6821","nonce":2912452950}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":2912452950},{"type":"v1","addr":"172.21.15.107:6825","nonce":2912452950}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":2912452950},{"type":"v1","addr":"172.21.15.107:6823","nonce":2912452950}]},"public_addr":"172.21.15.107:6819/2912452950","cluster_addr":"172.21.15.107:6821/2912452950","heartbeat_back_addr":"172.21.15.107:6825/2912452950","heartbeat_front_addr":"172.21.15.107:6823/2912452950","state":["exists","up"]},{"osd":7,"uuid":"a7dd26ee-c118-4a9a-9e19-142c2680902d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":3835863809},{"type":"v1","addr":"172.21.15.149:6817","nonce":3835863809}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":3835863809},{"type":"v1","addr":"172.21.15.149:6819","nonce":3835863809}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":3835863809},{"type":"v1","addr":"172.21.15.149:6823","nonce":3835863809}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":3835863809},{"type":"v1","addr":"172.21.15.149:6821","nonce":3835863809}]},"public_addr":"172.21.15.149:6817/3835863809","cluster_addr":"172.21.15.149:6819/3835863809","heartbeat_back_addr":"172.21.15.149:6823/3835863809","heartbeat_front_addr":"172.21.15.149:6821/3835863809","state":["exists","up"]},{"osd":8,"uuid":"baaaa8dd-0e1f-4bae-86bf-9e429d985f23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":1010601477},{"type":"v1","addr":"172.21.15.204:6817","nonce":1010601477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":1010601477},{"type":"v1","addr":"172.21.15.204:6819","nonce":1010601477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":1010601477},{"type":"v1","addr":"172.21.15.204:6823","nonce":1010601477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":1010601477},{"type":"v1","addr":"172.21.15.204:6821","nonce":1010601477}]},"public_addr":"172.21.15.204:6817/1010601477","cluster_addr":"172.21.15.204:6819/1010601477","heartbeat_back_addr":"172.21.15.204:6823/1010601477","heartbeat_front_addr":"172.21.15.204:6821/1010601477","state":["exists","up"]},{"osd":9,"uuid":"8df80dfa-e7ff-410e-824c-b1dce111b02f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":3334759819},{"type":"v1","addr":"172.21.15.107:6827","nonce":3334759819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3334759819},{"type":"v1","addr":"172.21.15.107:6829","nonce":3334759819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":3334759819},{"type":"v1","addr":"172.21.15.107:6833","nonce":3334759819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":3334759819},{"type":"v1","addr":"172.21.15.107:6831","nonce":3334759819}]},"public_addr":"172.21.15.107:6827/3334759819","cluster_addr":"172.21.15.107:6829/3334759819","heartbeat_back_addr":"172.21.15.107:6833/3334759819","heartbeat_front_addr":"172.21.15.107:6831/3334759819","state":["exists","up"]},{"osd":10,"uuid":"338945b2-20c0-40b2-aaca-062cd4adee04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":3469284631},{"type":"v1","addr":"172.21.15.149:6825","nonce":3469284631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":3469284631},{"type":"v1","addr":"172.21.15.149:6827","nonce":3469284631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":3469284631},{"type":"v1","addr":"172.21.15.149:6831","nonce":3469284631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":3469284631},{"type":"v1","addr":"172.21.15.149:6829","nonce":3469284631}]},"public_addr":"172.21.15.149:6825/3469284631","cluster_addr":"172.21.15.149:6827/3469284631","heartbeat_back_addr":"172.21.15.149:6831/3469284631","heartbeat_front_addr":"172.21.15.149:6829/3469284631","state":["exists","up"]},{"osd":11,"uuid":"acccfe4f-d307-4a64-8f4d-fba27bfbe09d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":3097772473},{"type":"v1","addr":"172.21.15.204:6825","nonce":3097772473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":3097772473},{"type":"v1","addr":"172.21.15.204:6827","nonce":3097772473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":3097772473},{"type":"v1","addr":"172.21.15.204:6831","nonce":3097772473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":3097772473},{"type":"v1","addr":"172.21.15.204:6829","nonce":3097772473}]},"public_addr":"172.21.15.204:6825/3097772473","cluster_addr":"172.21.15.204:6827/3097772473","heartbeat_back_addr":"172.21.15.204:6831/3097772473","heartbeat_front_addr":"172.21.15.204:6829/3097772473","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T20:50:25.862753+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T20:50:58.167663+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T20:51:30.312816+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T20:52:05.487917+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T20:52:39.348354+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T20:53:12.899002+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T20:53:47.630837+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T20:54:21.444552+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T20:54:57.101160+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T20:55:31.841786+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T20:56:05.827161+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T20:56:40.611097+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.0","mappings":[{"from":4,"to":6}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.3b","mappings":[{"from":3,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.107:0/1514666962":"2024-03-09T20:48:20.149406+0000","172.21.15.107:0/1886016707":"2024-03-09T20:48:20.149406+0000","172.21.15.107:6801/426372446":"2024-03-09T20:48:20.149406+0000","172.21.15.107:6800/426372446":"2024-03-09T20:48:20.149406+0000","172.21.15.107:0/2392547434":"2024-03-09T20:47:45.706790+0000","172.21.15.107:0/1206995985":"2024-03-09T20:47:45.706790+0000","172.21.15.107:6800/3549936519":"2024-03-09T20:47:45.706790+0000","172.21.15.107:6801/3549936519":"2024-03-09T20:47:45.706790+0000","172.21.15.107:0/2482452215":"2024-03-09T20:47:45.706790+0000","172.21.15.107:0/808124991":"2024-03-09T20:48:20.149406+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-08T20:59:36.754 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:36.771+0000 7f1ca4200640 1 -- 172.21.15.107:0/1764362259 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f1c840a2800 msgr2=0x7f1c840a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:59:36.754 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:36.771+0000 7f1ca4200640 1 --2- 172.21.15.107:0/1764362259 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f1c840a2800 0x7f1c840a4cc0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f1c9002c240 tx=0x7f1c90005fd0 comp rx=0 tx=0).stop 2024-03-08T20:59:36.754 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:36.771+0000 7f1ca4200640 1 -- 172.21.15.107:0/1764362259 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1c9c1050e0 msgr2=0x7f1c9c1a0d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:59:36.754 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:36.771+0000 7f1ca4200640 1 --2- 172.21.15.107:0/1764362259 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1c9c1050e0 0x7f1c9c1a0d50 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f1c9400eaa0 tx=0x7f1c9400ef60 comp rx=0 tx=0).stop 2024-03-08T20:59:36.755 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:36.772+0000 7f1ca4200640 1 -- 172.21.15.107:0/1764362259 shutdown_connections 2024-03-08T20:59:36.755 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:36.772+0000 7f1ca4200640 1 --2- 172.21.15.107:0/1764362259 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f1c840a2800 0x7f1c840a4cc0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:36.755 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:36.772+0000 7f1ca4200640 1 --2- 172.21.15.107:0/1764362259 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1c9c1050e0 0x7f1c9c1a0d50 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:36.755 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:36.772+0000 7f1ca4200640 1 --2- 172.21.15.107:0/1764362259 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1c9c104720 0x7f1c9c199ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:36.755 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:36.772+0000 7f1ca4200640 1 --2- 172.21.15.107:0/1764362259 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1c9c103530 0x7f1c9c1997a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:36.755 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:36.772+0000 7f1ca4200640 1 -- 172.21.15.107:0/1764362259 >> 172.21.15.107:0/1764362259 conn(0x7f1c9c0feaa0 msgr2=0x7f1c9c107a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:59:36.755 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:36.772+0000 7f1ca4200640 1 -- 172.21.15.107:0/1764362259 shutdown_connections 2024-03-08T20:59:36.755 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:36.772+0000 7f1ca4200640 1 -- 172.21.15.107:0/1764362259 wait complete. 2024-03-08T20:59:36.774 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-08T20:59:36.878 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:59:36 smithi204 ceph-mon[34935]: from='client.? 172.21.15.107:0/4282867972' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-08T20:59:36.878 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:59:36 smithi204 ceph-mon[34935]: from='client.? 172.21.15.107:0/1570852810' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-08T20:59:36.878 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:59:36 smithi204 ceph-mon[34935]: pgmap v416: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 0 B/s wr, 9 op/s 2024-03-08T20:59:36.985 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.002+0000 7f8dbf600640 1 -- 172.21.15.107:0/747964111 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8db8073d50 msgr2=0x7f8db80741d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:59:36.985 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.002+0000 7f8dbf600640 1 --2- 172.21.15.107:0/747964111 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8db8073d50 0x7f8db80741d0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f8da0009900 tx=0x7f8da002bc30 comp rx=0 tx=0).stop 2024-03-08T20:59:36.985 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.002+0000 7f8dbf600640 1 -- 172.21.15.107:0/747964111 shutdown_connections 2024-03-08T20:59:36.985 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.002+0000 7f8dbf600640 1 --2- 172.21.15.107:0/747964111 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8db81091d0 0x7f8db810b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:36.985 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.002+0000 7f8dbf600640 1 --2- 172.21.15.107:0/747964111 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8db8073d50 0x7f8db80741d0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:36.985 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.002+0000 7f8dbf600640 1 --2- 172.21.15.107:0/747964111 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8db8075330 0x7f8db8073780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:36.985 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.002+0000 7f8dbf600640 1 -- 172.21.15.107:0/747964111 >> 172.21.15.107:0/747964111 conn(0x7f8db80fc8a0 msgr2=0x7f8db80fecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:59:36.985 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.002+0000 7f8dbf600640 1 -- 172.21.15.107:0/747964111 shutdown_connections 2024-03-08T20:59:36.985 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.002+0000 7f8dbf600640 1 -- 172.21.15.107:0/747964111 wait complete. 2024-03-08T20:59:36.987 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.003+0000 7f8dbf600640 1 Processor -- start 2024-03-08T20:59:36.988 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.003+0000 7f8dbf600640 1 -- start start 2024-03-08T20:59:36.988 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.003+0000 7f8dbf600640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8db8073d50 0x7f8db81996e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:36.988 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.003+0000 7f8dbf600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8db8075330 0x7f8db8199c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:36.988 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.003+0000 7f8dbf600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8db81091d0 0x7f8db81a0c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:36.988 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.003+0000 7f8dbf600640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8db810f090 con 0x7f8db8075330 2024-03-08T20:59:36.988 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.003+0000 7f8dbf600640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8db810ef10 con 0x7f8db8073d50 2024-03-08T20:59:36.988 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.003+0000 7f8dbf600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8db810f210 con 0x7f8db81091d0 2024-03-08T20:59:36.988 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.004+0000 7f8db7e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8db8075330 0x7f8db8199c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:59:36.988 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.004+0000 7f8db7e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8db8075330 0x7f8db8199c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:56912/0 (socket says 172.21.15.107:56912) 2024-03-08T20:59:36.988 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.004+0000 7f8db7e00640 1 -- 172.21.15.107:0/1379066268 learned_addr learned my addr 172.21.15.107:0/1379066268 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T20:59:36.988 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.004+0000 7f8db7e00640 1 -- 172.21.15.107:0/1379066268 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8db81091d0 msgr2=0x7f8db81a0c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:59:36.989 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.004+0000 7f8db7e00640 1 --2- 172.21.15.107:0/1379066268 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8db81091d0 0x7f8db81a0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:36.989 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.004+0000 7f8db7e00640 1 -- 172.21.15.107:0/1379066268 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8db8073d50 msgr2=0x7f8db81996e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:59:36.989 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.004+0000 7f8dbcc00640 1 --2- 172.21.15.107:0/1379066268 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8db8073d50 0x7f8db81996e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:59:36.989 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.004+0000 7f8db7e00640 1 --2- 172.21.15.107:0/1379066268 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8db8073d50 0x7f8db81996e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:36.989 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.004+0000 7f8db7e00640 1 -- 172.21.15.107:0/1379066268 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8db81a1300 con 0x7f8db8075330 2024-03-08T20:59:36.989 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.004+0000 7f8db7e00640 1 --2- 172.21.15.107:0/1379066268 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8db8075330 0x7f8db8199c20 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f8da0000c00 tx=0x7f8da00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:59:36.989 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.005+0000 7f8db5600640 1 -- 172.21.15.107:0/1379066268 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8da0043020 con 0x7f8db8075330 2024-03-08T20:59:36.989 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.005+0000 7f8dbf600640 1 -- 172.21.15.107:0/1379066268 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8db81a1590 con 0x7f8db8075330 2024-03-08T20:59:36.989 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.005+0000 7f8db5600640 1 -- 172.21.15.107:0/1379066268 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f8da00054c0 con 0x7f8db8075330 2024-03-08T20:59:36.989 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.005+0000 7f8db5600640 1 -- 172.21.15.107:0/1379066268 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8da0003d80 con 0x7f8db8075330 2024-03-08T20:59:36.989 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.005+0000 7f8dbf600640 1 -- 172.21.15.107:0/1379066268 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8db81a1b30 con 0x7f8db8075330 2024-03-08T20:59:36.990 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.006+0000 7f8dbf600640 1 -- 172.21.15.107:0/1379066268 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8db8107f00 con 0x7f8db8075330 2024-03-08T20:59:36.990 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.008+0000 7f8db5600640 1 -- 172.21.15.107:0/1379066268 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f8da0046020 con 0x7f8db8075330 2024-03-08T20:59:36.991 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.009+0000 7f8db5600640 1 --2- 172.21.15.107:0/1379066268 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f8d9c0a2880 0x7f8d9c0a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:36.992 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.009+0000 7f8db5600640 1 -- 172.21.15.107:0/1379066268 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f8da00f0f00 con 0x7f8db8075330 2024-03-08T20:59:36.992 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.009+0000 7f8dbcc00640 1 --2- 172.21.15.107:0/1379066268 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f8d9c0a2880 0x7f8d9c0a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:59:36.992 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.010+0000 7f8dbcc00640 1 --2- 172.21.15.107:0/1379066268 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f8d9c0a2880 0x7f8d9c0a4d40 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f8db0008d20 tx=0x7f8db00082b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:59:36.995 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.013+0000 7f8db5600640 1 -- 172.21.15.107:0/1379066268 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f8da004d020 con 0x7f8db8075330 2024-03-08T20:59:37.015 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:59:36 smithi149 ceph-mon[35042]: from='client.? 172.21.15.107:0/4282867972' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-08T20:59:37.015 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:59:36 smithi149 ceph-mon[35042]: from='client.? 172.21.15.107:0/1570852810' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-08T20:59:37.016 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:59:36 smithi149 ceph-mon[35042]: pgmap v416: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 0 B/s wr, 9 op/s 2024-03-08T20:59:37.230 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.248+0000 7f8dbf600640 1 -- 172.21.15.107:0/1379066268 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8db8061f90 con 0x7f8db8075330 2024-03-08T20:59:37.231 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.249+0000 7f8db5600640 1 -- 172.21.15.107:0/1379066268 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+11393 (secure 0 0 0) 0x7f8da00be080 con 0x7f8db8075330 2024-03-08T20:59:37.232 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-08T20:59:37.232 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/133582743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":133582743},{"type":"v1","addr":"172.21.15.107:6835","nonce":133582743}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/3798982337","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":3798982337},{"type":"v1","addr":"172.21.15.107:6837","nonce":3798982337}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-08T20:57:50.223752+0000","modified":"2024-03-08T20:57:57.874727+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34475,"mds_1":34445,"mds_2":34415,"mds_3":34397,"mds_4":24457},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14625":{"gid":14625,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6839/3070059000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":3070059000},{"type":"v1","addr":"172.21.15.107:6839","nonce":3070059000}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14637":{"gid":14637,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6841/747096194","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6840","nonce":747096194},{"type":"v1","addr":"172.21.15.107:6841","nonce":747096194}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24400":{"gid":24400,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6833/3404762782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":3404762782},{"type":"v1","addr":"172.21.15.204:6833","nonce":3404762782}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24430":{"gid":24430,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/2072518651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":2072518651},{"type":"v1","addr":"172.21.15.204:6835","nonce":2072518651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24442":{"gid":24442,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6837/42644905","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":42644905},{"type":"v1","addr":"172.21.15.204:6837","nonce":42644905}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24457":{"gid":24457,"name":"i","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.204:6839/1857316822","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":1857316822},{"type":"v1","addr":"172.21.15.204:6839","nonce":1857316822}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34397":{"gid":34397,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":14,"addr":"172.21.15.149:6833/1962161726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":1962161726},{"type":"v1","addr":"172.21.15.149:6833","nonce":1962161726}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34415":{"gid":34415,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.149:6835/442657066","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":442657066},{"type":"v1","addr":"172.21.15.149:6835","nonce":442657066}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34445":{"gid":34445,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.149:6837/914546237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":914546237},{"type":"v1","addr":"172.21.15.149:6837","nonce":914546237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34475":{"gid":34475,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.149:6839/55290596","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":55290596},{"type":"v1","addr":"172.21.15.149:6839","nonce":55290596}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-08T20:59:37.232 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 27 2024-03-08T20:59:37.235 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.251+0000 7f8dbf600640 1 -- 172.21.15.107:0/1379066268 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f8d9c0a2880 msgr2=0x7f8d9c0a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:59:37.235 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.251+0000 7f8dbf600640 1 --2- 172.21.15.107:0/1379066268 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f8d9c0a2880 0x7f8d9c0a4d40 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f8db0008d20 tx=0x7f8db00082b0 comp rx=0 tx=0).stop 2024-03-08T20:59:37.235 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.252+0000 7f8dbf600640 1 -- 172.21.15.107:0/1379066268 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8db8075330 msgr2=0x7f8db8199c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:59:37.235 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.252+0000 7f8dbf600640 1 --2- 172.21.15.107:0/1379066268 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8db8075330 0x7f8db8199c20 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f8da0000c00 tx=0x7f8da00042f0 comp rx=0 tx=0).stop 2024-03-08T20:59:37.235 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.252+0000 7f8dbf600640 1 -- 172.21.15.107:0/1379066268 shutdown_connections 2024-03-08T20:59:37.235 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.252+0000 7f8dbf600640 1 --2- 172.21.15.107:0/1379066268 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f8d9c0a2880 0x7f8d9c0a4d40 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:37.235 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.252+0000 7f8dbf600640 1 --2- 172.21.15.107:0/1379066268 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8db81091d0 0x7f8db81a0c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:37.235 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.252+0000 7f8dbf600640 1 --2- 172.21.15.107:0/1379066268 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8db8075330 0x7f8db8199c20 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:37.235 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.252+0000 7f8dbf600640 1 --2- 172.21.15.107:0/1379066268 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8db8073d50 0x7f8db81996e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:37.235 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.252+0000 7f8dbf600640 1 -- 172.21.15.107:0/1379066268 >> 172.21.15.107:0/1379066268 conn(0x7f8db80fc8a0 msgr2=0x7f8db81050d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:59:37.236 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.252+0000 7f8dbf600640 1 -- 172.21.15.107:0/1379066268 shutdown_connections 2024-03-08T20:59:37.236 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.252+0000 7f8dbf600640 1 -- 172.21.15.107:0/1379066268 wait complete. 2024-03-08T20:59:37.254 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 27, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-03-08T20:57:50.223752+0000', 'modified': '2024-03-08T20:57:57.874727+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 34475, 'mds_1': 34445, 'mds_2': 34415, 'mds_3': 34397, 'mds_4': 24457}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14625': {'gid': 14625, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.107:6839/3070059000', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6838', 'nonce': 3070059000}, {'type': 'v1', 'addr': '172.21.15.107:6839', 'nonce': 3070059000}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_14637': {'gid': 14637, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.107:6841/747096194', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6840', 'nonce': 747096194}, {'type': 'v1', 'addr': '172.21.15.107:6841', 'nonce': 747096194}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24400': {'gid': 24400, 'name': 'l', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.204:6833/3404762782', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.204:6832', 'nonce': 3404762782}, {'type': 'v1', 'addr': '172.21.15.204:6833', 'nonce': 3404762782}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24430': {'gid': 24430, 'name': 'f', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.204:6835/2072518651', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.204:6834', 'nonce': 2072518651}, {'type': 'v1', 'addr': '172.21.15.204:6835', 'nonce': 2072518651}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24442': {'gid': 24442, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.204:6837/42644905', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.204:6836', 'nonce': 42644905}, {'type': 'v1', 'addr': '172.21.15.204:6837', 'nonce': 42644905}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24457': {'gid': 24457, 'name': 'i', 'rank': 4, 'incarnation': 25, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.204:6839/1857316822', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.204:6838', 'nonce': 1857316822}, {'type': 'v1', 'addr': '172.21.15.204:6839', 'nonce': 1857316822}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_34397': {'gid': 34397, 'name': 'k', 'rank': 3, 'incarnation': 23, 'state': 'up:active', 'state_seq': 14, 'addr': '172.21.15.149:6833/1962161726', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.149:6832', 'nonce': 1962161726}, {'type': 'v1', 'addr': '172.21.15.149:6833', 'nonce': 1962161726}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_34415': {'gid': 34415, 'name': 'e', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.149:6835/442657066', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.149:6834', 'nonce': 442657066}, {'type': 'v1', 'addr': '172.21.15.149:6835', 'nonce': 442657066}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_34445': {'gid': 34445, 'name': 'h', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.149:6837/914546237', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.149:6836', 'nonce': 914546237}, {'type': 'v1', 'addr': '172.21.15.149:6837', 'nonce': 914546237}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_34475': {'gid': 34475, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.149:6839/55290596', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.149:6838', 'nonce': 55290596}, {'type': 'v1', 'addr': '172.21.15.149:6839', 'nonce': 55290596}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-03-08T20:59:37.254 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-03-08T20:59:37.254 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-03-08T20:59:37.464 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.480+0000 7f662ae00640 1 -- 172.21.15.107:0/4011286372 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f66240ffa40 msgr2=0x7f6624101e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:59:37.464 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.480+0000 7f662ae00640 1 --2- 172.21.15.107:0/4011286372 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f66240ffa40 0x7f6624101e70 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f6618009900 tx=0x7f661802bc30 comp rx=0 tx=0).stop 2024-03-08T20:59:37.464 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.480+0000 7f662ae00640 1 -- 172.21.15.107:0/4011286372 shutdown_connections 2024-03-08T20:59:37.464 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.480+0000 7f662ae00640 1 --2- 172.21.15.107:0/4011286372 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6624104d60 0x7f66241051e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:37.464 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.480+0000 7f662ae00640 1 --2- 172.21.15.107:0/4011286372 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f66241023b0 0x7f6624104820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:37.464 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.480+0000 7f662ae00640 1 --2- 172.21.15.107:0/4011286372 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f66240ffa40 0x7f6624101e70 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:37.465 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.480+0000 7f662ae00640 1 -- 172.21.15.107:0/4011286372 >> 172.21.15.107:0/4011286372 conn(0x7f66240fb660 msgr2=0x7f66240fda80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:59:37.465 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.480+0000 7f662ae00640 1 -- 172.21.15.107:0/4011286372 shutdown_connections 2024-03-08T20:59:37.465 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.480+0000 7f662ae00640 1 -- 172.21.15.107:0/4011286372 wait complete. 2024-03-08T20:59:37.465 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.481+0000 7f662ae00640 1 Processor -- start 2024-03-08T20:59:37.465 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.481+0000 7f662ae00640 1 -- start start 2024-03-08T20:59:37.468 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.482+0000 7f662ae00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f66240ffa40 0x7f662419d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:37.468 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.482+0000 7f662ae00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f66241023b0 0x7f662419df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:37.468 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.482+0000 7f662ae00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6624104d60 0x7f66241a4f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:37.468 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.482+0000 7f662ae00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66241133a0 con 0x7f6624104d60 2024-03-08T20:59:37.468 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.482+0000 7f662ae00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6624113220 con 0x7f66241023b0 2024-03-08T20:59:37.468 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.482+0000 7f662ae00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6624113520 con 0x7f66240ffa40 2024-03-08T20:59:37.469 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.482+0000 7f662a400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6624104d60 0x7f66241a4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:59:37.469 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.482+0000 7f662a400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6624104d60 0x7f66241a4f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:56926/0 (socket says 172.21.15.107:56926) 2024-03-08T20:59:37.469 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.482+0000 7f662a400640 1 -- 172.21.15.107:0/424184027 learned_addr learned my addr 172.21.15.107:0/424184027 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T20:59:37.469 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.482+0000 7f6629000640 1 --2- 172.21.15.107:0/424184027 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f66241023b0 0x7f662419df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:59:37.469 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.482+0000 7f6629a00640 1 --2- 172.21.15.107:0/424184027 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f66240ffa40 0x7f662419d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:59:37.469 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.483+0000 7f662a400640 1 -- 172.21.15.107:0/424184027 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f66240ffa40 msgr2=0x7f662419d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:59:37.469 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.483+0000 7f662a400640 1 --2- 172.21.15.107:0/424184027 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f66240ffa40 0x7f662419d9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:37.469 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.483+0000 7f662a400640 1 -- 172.21.15.107:0/424184027 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f66241023b0 msgr2=0x7f662419df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:59:37.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.483+0000 7f662a400640 1 --2- 172.21.15.107:0/424184027 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f66241023b0 0x7f662419df20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:37.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.483+0000 7f662a400640 1 -- 172.21.15.107:0/424184027 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66241a5600 con 0x7f6624104d60 2024-03-08T20:59:37.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.483+0000 7f6629000640 1 --2- 172.21.15.107:0/424184027 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f66241023b0 0x7f662419df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T20:59:37.471 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.483+0000 7f662a400640 1 --2- 172.21.15.107:0/424184027 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6624104d60 0x7f66241a4f90 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f661c00b970 tx=0x7f661c00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:59:37.471 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.483+0000 7f6622000640 1 -- 172.21.15.107:0/424184027 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f661c019070 con 0x7f6624104d60 2024-03-08T20:59:37.471 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.483+0000 7f6622000640 1 -- 172.21.15.107:0/424184027 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f661c014440 con 0x7f6624104d60 2024-03-08T20:59:37.471 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.483+0000 7f662ae00640 1 -- 172.21.15.107:0/424184027 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66241a58f0 con 0x7f6624104d60 2024-03-08T20:59:37.471 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.483+0000 7f6622000640 1 -- 172.21.15.107:0/424184027 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f661c0075d0 con 0x7f6624104d60 2024-03-08T20:59:37.471 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.483+0000 7f662ae00640 1 -- 172.21.15.107:0/424184027 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66241a5ec0 con 0x7f6624104d60 2024-03-08T20:59:37.480 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.486+0000 7f6622000640 1 -- 172.21.15.107:0/424184027 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f661c01c070 con 0x7f6624104d60 2024-03-08T20:59:37.480 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.487+0000 7f662f00b740 1 Processor -- start 2024-03-08T20:59:37.480 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.487+0000 7f662f00b740 1 -- start start 2024-03-08T20:59:37.480 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.487+0000 7f662f00b740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563c9f915760 0x563c9f915b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:37.480 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.487+0000 7f662f00b740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563c9f916150 0x563c9f91ac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:37.481 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.487+0000 7f662f00b740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563c9f91b2c0 0x563c9f91d750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:37.481 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.487+0000 7f662f00b740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c9f87a6f0 con 0x563c9f91b2c0 2024-03-08T20:59:37.481 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.487+0000 7f662f00b740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c9f8239e0 con 0x563c9f916150 2024-03-08T20:59:37.481 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.487+0000 7f662f00b740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c9f876790 con 0x563c9f915760 2024-03-08T20:59:37.481 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.488+0000 7f6629000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563c9f91b2c0 0x563c9f91d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:59:37.481 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.488+0000 7f6629000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563c9f91b2c0 0x563c9f91d750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:56928/0 (socket says 172.21.15.107:56928) 2024-03-08T20:59:37.482 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.488+0000 7f6629000640 1 -- 172.21.15.107:0/2144782686 learned_addr learned my addr 172.21.15.107:0/2144782686 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T20:59:37.482 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.488+0000 7f6622000640 1 --2- 172.21.15.107:0/424184027 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f660c0a2800 0x7f660c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:37.482 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.488+0000 7f6629000640 1 --2- 172.21.15.107:0/2144782686 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563c9f915760 0x563c9f915b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:59:37.482 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.488+0000 7f6629a00640 1 --2- 172.21.15.107:0/2144782686 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563c9f916150 0x563c9f91ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:59:37.482 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.488+0000 7f6622000640 1 -- 172.21.15.107:0/424184027 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f661c0ce050 con 0x7f6624104d60 2024-03-08T20:59:37.482 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.488+0000 7f662a400640 1 --2- 172.21.15.107:0/424184027 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f660c0a2800 0x7f660c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:59:37.482 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.488+0000 7f6629000640 1 -- 172.21.15.107:0/2144782686 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563c9f915760 msgr2=0x563c9f915b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T20:59:37.482 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.488+0000 7f6629000640 1 --2- 172.21.15.107:0/2144782686 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563c9f915760 0x563c9f915b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:37.483 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.488+0000 7f6629000640 1 -- 172.21.15.107:0/2144782686 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563c9f916150 msgr2=0x563c9f91ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T20:59:37.483 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.488+0000 7f6629000640 1 --2- 172.21.15.107:0/2144782686 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563c9f916150 0x563c9f91ac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:37.483 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.488+0000 7f6629000640 1 -- 172.21.15.107:0/2144782686 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563c9f817720 con 0x563c9f91b2c0 2024-03-08T20:59:37.483 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.488+0000 7f6629000640 1 --2- 172.21.15.107:0/2144782686 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563c9f91b2c0 0x563c9f91d750 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f661000bfd0 tx=0x7f661000b500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dce02be50f4693e5 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:59:37.483 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.489+0000 7f662a400640 1 --2- 172.21.15.107:0/424184027 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f660c0a2800 0x7f660c0a4cc0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f661c0a2c60 tx=0x7f661c004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:59:37.483 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.489+0000 7f6616a00640 1 -- 172.21.15.107:0/2144782686 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6610004260 con 0x563c9f91b2c0 2024-03-08T20:59:37.483 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.489+0000 7f6616a00640 1 -- 172.21.15.107:0/2144782686 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f6610004400 con 0x563c9f91b2c0 2024-03-08T20:59:37.484 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.489+0000 7f6616a00640 1 -- 172.21.15.107:0/2144782686 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6610004df0 con 0x563c9f91b2c0 2024-03-08T20:59:37.484 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.489+0000 7f662f00b740 1 -- 172.21.15.107:0/2144782686 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563c9f91b2c0 msgr2=0x563c9f91d750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:59:37.484 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.489+0000 7f662f00b740 1 --2- 172.21.15.107:0/2144782686 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563c9f91b2c0 0x563c9f91d750 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f661000bfd0 tx=0x7f661000b500 comp rx=0 tx=0).stop 2024-03-08T20:59:37.484 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.490+0000 7f662f00b740 1 -- 172.21.15.107:0/2144782686 shutdown_connections 2024-03-08T20:59:37.484 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.490+0000 7f662f00b740 1 --2- 172.21.15.107:0/2144782686 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563c9f91b2c0 0x563c9f91d750 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:37.484 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.490+0000 7f662f00b740 1 --2- 172.21.15.107:0/2144782686 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563c9f916150 0x563c9f91ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:37.484 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.490+0000 7f662f00b740 1 --2- 172.21.15.107:0/2144782686 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563c9f915760 0x563c9f915b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:37.485 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.490+0000 7f662f00b740 1 -- 172.21.15.107:0/2144782686 >> 172.21.15.107:0/2144782686 conn(0x563c9f910cd0 msgr2=0x563c9f9110d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:59:37.485 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.490+0000 7f662f00b740 1 -- 172.21.15.107:0/2144782686 shutdown_connections 2024-03-08T20:59:37.485 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.490+0000 7f662f00b740 1 -- 172.21.15.107:0/2144782686 wait complete. 2024-03-08T20:59:37.485 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.490+0000 7f662f00b740 1 -- start start 2024-03-08T20:59:37.485 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.490+0000 7f662f00b740 1 Processor -- start 2024-03-08T20:59:37.485 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.491+0000 7f662f00b740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563c9f915760 0x563c9f91f970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:37.485 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.491+0000 7f662f00b740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563c9f916150 0x563c9f91feb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:37.486 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.491+0000 7f662f00b740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563c9f91b2c0 0x563c9f926ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:37.486 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.491+0000 7f662f00b740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c9f91e270 con 0x563c9f916150 2024-03-08T20:59:37.486 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.491+0000 7f662f00b740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c9f836e20 con 0x563c9f915760 2024-03-08T20:59:37.486 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.491+0000 7f662f00b740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c9f91e3f0 con 0x563c9f91b2c0 2024-03-08T20:59:37.486 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.491+0000 7f662f00b740 10 client.0 _ll_register_callbacks cb 0x563c9f8718d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T20:59:37.486 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.491+0000 7f6629a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563c9f916150 0x563c9f91feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:59:37.487 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.491+0000 7f6629a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563c9f916150 0x563c9f91feb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:56936/0 (socket says 172.21.15.107:56936) 2024-03-08T20:59:37.487 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.491+0000 7f6629a00640 1 -- 172.21.15.107:0/390198300 learned_addr learned my addr 172.21.15.107:0/390198300 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T20:59:37.487 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.491+0000 7f6629000640 1 --2- 172.21.15.107:0/390198300 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563c9f915760 0x563c9f91f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:59:37.487 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.491+0000 7f6629000640 1 --2- 172.21.15.107:0/390198300 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563c9f91b2c0 0x563c9f926ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:59:37.487 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.491+0000 7f6629a00640 1 -- 172.21.15.107:0/390198300 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563c9f91b2c0 msgr2=0x563c9f926ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T20:59:37.487 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.491+0000 7f6629a00640 1 --2- 172.21.15.107:0/390198300 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563c9f91b2c0 0x563c9f926ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:37.487 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.491+0000 7f6629a00640 1 -- 172.21.15.107:0/390198300 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563c9f915760 msgr2=0x563c9f91f970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T20:59:37.487 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.491+0000 7f6629a00640 1 --2- 172.21.15.107:0/390198300 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563c9f915760 0x563c9f91f970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:37.488 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.491+0000 7f6629a00640 1 -- 172.21.15.107:0/390198300 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563c9f58f2d0 con 0x563c9f916150 2024-03-08T20:59:37.488 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.491+0000 7f6629000640 1 --2- 172.21.15.107:0/390198300 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563c9f915760 0x563c9f91f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T20:59:37.488 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.492+0000 7f6629a00640 1 --2- 172.21.15.107:0/390198300 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563c9f916150 0x563c9f91feb0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f6618009900 tx=0x7f6618004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1686365fb09f3082 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:59:37.488 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.492+0000 7f660b400640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-03-08T20:59:37.488 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.492+0000 7f660b400640 1 -- 172.21.15.107:0/390198300 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6618005230 con 0x563c9f916150 2024-03-08T20:59:37.488 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.492+0000 7f660b400640 1 -- 172.21.15.107:0/390198300 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f66180053d0 con 0x563c9f916150 2024-03-08T20:59:37.488 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.492+0000 7f662f00b740 1 -- 172.21.15.107:0/390198300 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6610004400 con 0x563c9f916150 2024-03-08T20:59:37.489 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.492+0000 7f660b400640 1 -- 172.21.15.107:0/390198300 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f66180055d0 con 0x563c9f916150 2024-03-08T20:59:37.489 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.492+0000 7f662f00b740 10 client.15057 fetch_fsmap learned FSMap version 27 2024-03-08T20:59:37.489 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.492+0000 7f662f00b740 1 -- 172.21.15.107:0/390198300 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x563c9f6596b0 con 0x563c9f916150 2024-03-08T20:59:37.489 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.492+0000 7f660b400640 1 -- 172.21.15.107:0/390198300 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f6610004400 con 0x563c9f916150 2024-03-08T20:59:37.489 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.493+0000 7f660b400640 1 -- 172.21.15.107:0/390198300 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f6618003990 con 0x563c9f916150 2024-03-08T20:59:37.489 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.493+0000 7f662f00b740 10 client.15057 fetch_fsmap finished waiting for FSMap version 27 2024-03-08T20:59:37.489 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.493+0000 7f662f00b740 10 client.15057 resolve_mds: resolved name 'i' to daemon mds.i 2024-03-08T20:59:37.489 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.493+0000 7f662f00b740 20 client.15057 populate_metadata read hostname 'smithi107' 2024-03-08T20:59:37.490 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.493+0000 7f662f00b740 1 --2- 172.21.15.107:0/390198300 >> [v2:172.21.15.204:6838/1857316822,v1:172.21.15.204:6839/1857316822] conn(0x563c9f92c550 0x563c9f92b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:37.490 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.493+0000 7f662f00b740 4 client.15057 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-08T20:59:37.490 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.493+0000 7f662f00b740 1 -- 172.21.15.107:0/390198300 --> [v2:172.21.15.204:6838/1857316822,v1:172.21.15.204:6839/1857316822] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563c9f887c30 con 0x563c9f92c550 2024-03-08T20:59:37.490 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.493+0000 7f6629000640 1 --2- 172.21.15.107:0/390198300 >> [v2:172.21.15.204:6838/1857316822,v1:172.21.15.204:6839/1857316822] conn(0x563c9f92c550 0x563c9f92b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:59:37.490 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.494+0000 7f660b400640 10 client.15057 ms_handle_connect on v2:172.21.15.204:6838/1857316822 2024-03-08T20:59:37.490 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.494+0000 7f6629000640 1 --2- 172.21.15.107:0/390198300 >> [v2:172.21.15.204:6838/1857316822,v1:172.21.15.204:6839/1857316822] conn(0x563c9f92c550 0x563c9f92b5c0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=50ca5061619e7a27 server_cookie=251f7440539503e0 in_seq=0 out_seq=0 2024-03-08T20:59:37.490 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.496+0000 7f660b400640 1 -- 172.21.15.107:0/390198300 <== mds.4 v2:172.21.15.204:6838/1857316822 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x563c9f887c30 con 0x563c9f92c550 2024-03-08T20:59:37.490 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.496+0000 7f660b400640 10 client.15057 handle_command_reply: tid=0 2024-03-08T20:59:37.494 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.496+0000 7f662f00b740 1 client.15057 shutdown 2024-03-08T20:59:37.494 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.496+0000 7f662f00b740 1 -- 172.21.15.107:0/390198300 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563c9f916150 msgr2=0x563c9f91feb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:59:37.494 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.496+0000 7f662f00b740 1 --2- 172.21.15.107:0/390198300 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563c9f916150 0x563c9f91feb0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f6618009900 tx=0x7f6618004270 comp rx=0 tx=0).stop 2024-03-08T20:59:37.494 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.496+0000 7f662f00b740 1 -- 172.21.15.107:0/390198300 shutdown_connections 2024-03-08T20:59:37.494 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.496+0000 7f662f00b740 1 --2- 172.21.15.107:0/390198300 >> [v2:172.21.15.204:6838/1857316822,v1:172.21.15.204:6839/1857316822] conn(0x563c9f92c550 0x563c9f92b5c0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:37.494 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.496+0000 7f662f00b740 1 --2- 172.21.15.107:0/390198300 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563c9f91b2c0 0x563c9f926ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:37.494 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.496+0000 7f662f00b740 1 --2- 172.21.15.107:0/390198300 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563c9f916150 0x563c9f91feb0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:37.495 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.496+0000 7f662f00b740 1 --2- 172.21.15.107:0/390198300 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563c9f915760 0x563c9f91f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:37.495 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.496+0000 7f660b400640 0 client.15057 ms_handle_reset on v2:172.21.15.204:6838/1857316822 2024-03-08T20:59:37.495 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.496+0000 7f662f00b740 1 -- 172.21.15.107:0/390198300 >> 172.21.15.107:0/390198300 conn(0x563c9f910cd0 msgr2=0x563c9f90fee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:59:37.495 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.497+0000 7f662f00b740 1 -- 172.21.15.107:0/390198300 shutdown_connections 2024-03-08T20:59:37.495 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.497+0000 7f662f00b740 1 -- 172.21.15.107:0/390198300 wait complete. 2024-03-08T20:59:37.495 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.497+0000 7f662f00b740 20 client.15057 trim_cache size 0 max 16384 2024-03-08T20:59:37.502 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.513+0000 7f662f00b740 1 Processor -- start 2024-03-08T20:59:37.503 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.514+0000 7f662f00b740 1 -- start start 2024-03-08T20:59:37.503 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.514+0000 7f662f00b740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563c9f915760 0x563c9f92b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:37.503 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.514+0000 7f662f00b740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563c9f916150 0x563c9f926eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:37.503 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.514+0000 7f662f00b740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563c9f91b2c0 0x563c9f90ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:37.503 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.514+0000 7f662f00b740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c9f92bd70 con 0x563c9f915760 2024-03-08T20:59:37.503 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.514+0000 7f662f00b740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c9f92c3d0 con 0x563c9f916150 2024-03-08T20:59:37.503 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.514+0000 7f662f00b740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c9f92ca30 con 0x563c9f91b2c0 2024-03-08T20:59:37.503 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.514+0000 7f6629000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563c9f915760 0x563c9f92b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:59:37.503 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.514+0000 7f6629000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563c9f915760 0x563c9f92b5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:56950/0 (socket says 172.21.15.107:56950) 2024-03-08T20:59:37.504 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.514+0000 7f6629000640 1 -- 172.21.15.107:0/1416190122 learned_addr learned my addr 172.21.15.107:0/1416190122 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T20:59:37.504 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.514+0000 7f6629a00640 1 --2- 172.21.15.107:0/1416190122 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563c9f916150 0x563c9f926eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:59:37.504 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.514+0000 7f6629000640 1 -- 172.21.15.107:0/1416190122 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563c9f91b2c0 msgr2=0x563c9f90ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T20:59:37.504 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.514+0000 7f6629000640 1 --2- 172.21.15.107:0/1416190122 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563c9f91b2c0 0x563c9f90ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:37.504 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.514+0000 7f6629000640 1 -- 172.21.15.107:0/1416190122 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563c9f916150 msgr2=0x563c9f926eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T20:59:37.504 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.514+0000 7f6629000640 1 --2- 172.21.15.107:0/1416190122 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563c9f916150 0x563c9f926eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:37.504 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.514+0000 7f6629000640 1 -- 172.21.15.107:0/1416190122 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563c9f92bb00 con 0x563c9f915760 2024-03-08T20:59:37.505 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.515+0000 7f6629000640 1 --2- 172.21.15.107:0/1416190122 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563c9f915760 0x563c9f92b5c0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f661001a040 tx=0x7f661000b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc891ddea9ee8e0b server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:59:37.505 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.515+0000 7f6616000640 1 -- 172.21.15.107:0/1416190122 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f661002b5f0 con 0x563c9f915760 2024-03-08T20:59:37.505 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.515+0000 7f6616000640 1 -- 172.21.15.107:0/1416190122 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f661002b790 con 0x563c9f915760 2024-03-08T20:59:37.505 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.515+0000 7f6616000640 1 -- 172.21.15.107:0/1416190122 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6610010750 con 0x563c9f915760 2024-03-08T20:59:37.505 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.516+0000 7f662f00b740 1 -- 172.21.15.107:0/1416190122 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563c9f915760 msgr2=0x563c9f92b5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:59:37.505 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.516+0000 7f662f00b740 1 --2- 172.21.15.107:0/1416190122 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563c9f915760 0x563c9f92b5c0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f661001a040 tx=0x7f661000b890 comp rx=0 tx=0).stop 2024-03-08T20:59:37.505 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.516+0000 7f662f00b740 1 -- 172.21.15.107:0/1416190122 shutdown_connections 2024-03-08T20:59:37.505 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.516+0000 7f662f00b740 1 --2- 172.21.15.107:0/1416190122 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563c9f91b2c0 0x563c9f90ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:37.505 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.516+0000 7f662f00b740 1 --2- 172.21.15.107:0/1416190122 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563c9f916150 0x563c9f926eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:37.505 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.516+0000 7f662f00b740 1 --2- 172.21.15.107:0/1416190122 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563c9f915760 0x563c9f92b5c0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:37.506 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.516+0000 7f662f00b740 1 -- 172.21.15.107:0/1416190122 >> 172.21.15.107:0/1416190122 conn(0x563c9f910cd0 msgr2=0x563c9f9110b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:59:37.506 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.516+0000 7f662f00b740 1 -- 172.21.15.107:0/1416190122 shutdown_connections 2024-03-08T20:59:37.506 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.516+0000 7f662f00b740 1 -- 172.21.15.107:0/1416190122 wait complete. 2024-03-08T20:59:37.506 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.516+0000 7f662f00b740 1 -- start start 2024-03-08T20:59:37.506 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.516+0000 7f662f00b740 1 Processor -- start 2024-03-08T20:59:37.506 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.517+0000 7f662f00b740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563c9f915760 0x563c9f923ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:37.506 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.517+0000 7f662f00b740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563c9f916150 0x563c9f9241e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:37.506 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.517+0000 7f662f00b740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563c9f91b2c0 0x563c9f982150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:37.506 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.517+0000 7f662f00b740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c9f92d6f0 con 0x563c9f91b2c0 2024-03-08T20:59:37.507 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.517+0000 7f662f00b740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c9f92d090 con 0x563c9f916150 2024-03-08T20:59:37.507 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.517+0000 7f662f00b740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c9f92dd50 con 0x563c9f915760 2024-03-08T20:59:37.507 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.517+0000 7f662f00b740 10 client.0 _ll_register_callbacks cb 0x7f661000b6f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T20:59:37.507 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.517+0000 7f6629000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563c9f91b2c0 0x563c9f982150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:59:37.507 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.517+0000 7f6629000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563c9f91b2c0 0x563c9f982150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:56952/0 (socket says 172.21.15.107:56952) 2024-03-08T20:59:37.507 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.517+0000 7f6629000640 1 -- 172.21.15.107:0/4009738931 learned_addr learned my addr 172.21.15.107:0/4009738931 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T20:59:37.507 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.517+0000 7f6629000640 1 --2- 172.21.15.107:0/4009738931 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563c9f915760 0x563c9f923ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:59:37.507 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.517+0000 7f6629a00640 1 --2- 172.21.15.107:0/4009738931 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563c9f916150 0x563c9f9241e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:59:37.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.517+0000 7f6629000640 1 -- 172.21.15.107:0/4009738931 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563c9f915760 msgr2=0x563c9f923ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T20:59:37.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.517+0000 7f6629000640 1 --2- 172.21.15.107:0/4009738931 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563c9f915760 0x563c9f923ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:37.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.517+0000 7f6629000640 1 -- 172.21.15.107:0/4009738931 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563c9f916150 msgr2=0x563c9f9241e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T20:59:37.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.517+0000 7f6629000640 1 --2- 172.21.15.107:0/4009738931 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563c9f916150 0x563c9f9241e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:37.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.517+0000 7f6629000640 1 -- 172.21.15.107:0/4009738931 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563c9f982690 con 0x563c9f91b2c0 2024-03-08T20:59:37.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.517+0000 7f6629a00640 1 --2- 172.21.15.107:0/4009738931 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563c9f916150 0x563c9f9241e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T20:59:37.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.518+0000 7f6629000640 1 --2- 172.21.15.107:0/4009738931 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563c9f91b2c0 0x563c9f982150 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f661000bfa0 tx=0x7f661000b4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff2cbe87a8728332 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:59:37.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.518+0000 7f660b400640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-03-08T20:59:37.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.518+0000 7f660b400640 1 -- 172.21.15.107:0/4009738931 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f661003f060 con 0x563c9f91b2c0 2024-03-08T20:59:37.509 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.518+0000 7f660b400640 1 -- 172.21.15.107:0/4009738931 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f66100106f0 con 0x563c9f91b2c0 2024-03-08T20:59:37.509 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.518+0000 7f662f00b740 1 -- 172.21.15.107:0/4009738931 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f661002b790 con 0x563c9f91b2c0 2024-03-08T20:59:37.509 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.518+0000 7f660b400640 1 -- 172.21.15.107:0/4009738931 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6610010970 con 0x563c9f91b2c0 2024-03-08T20:59:37.509 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.518+0000 7f662f00b740 10 client.15063 fetch_fsmap learned FSMap version 27 2024-03-08T20:59:37.509 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.518+0000 7f662f00b740 1 -- 172.21.15.107:0/4009738931 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x563c9f983850 con 0x563c9f91b2c0 2024-03-08T20:59:37.509 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.518+0000 7f660b400640 1 -- 172.21.15.107:0/4009738931 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f661002b790 con 0x563c9f91b2c0 2024-03-08T20:59:37.509 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.519+0000 7f660b400640 1 -- 172.21.15.107:0/4009738931 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f6610036070 con 0x563c9f91b2c0 2024-03-08T20:59:37.509 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.519+0000 7f662f00b740 10 client.15063 fetch_fsmap finished waiting for FSMap version 27 2024-03-08T20:59:37.509 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.519+0000 7f662f00b740 10 client.15063 resolve_mds: resolved name 'i' to daemon mds.i 2024-03-08T20:59:37.510 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.519+0000 7f662f00b740 20 client.15063 populate_metadata read hostname 'smithi107' 2024-03-08T20:59:37.510 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.519+0000 7f662f00b740 1 --2- 172.21.15.107:0/4009738931 >> [v2:172.21.15.204:6838/1857316822,v1:172.21.15.204:6839/1857316822] conn(0x563c9f987850 0x563c9f983a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:37.510 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.519+0000 7f662f00b740 4 client.15063 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-08T20:59:37.510 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.519+0000 7f662f00b740 1 -- 172.21.15.107:0/4009738931 --> [v2:172.21.15.204:6838/1857316822,v1:172.21.15.204:6839/1857316822] -- command(tid 0: {"prefix": "status"}) v1 -- 0x563c9f881f80 con 0x563c9f987850 2024-03-08T20:59:37.510 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.519+0000 7f6629a00640 1 --2- 172.21.15.107:0/4009738931 >> [v2:172.21.15.204:6838/1857316822,v1:172.21.15.204:6839/1857316822] conn(0x563c9f987850 0x563c9f983a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:59:37.510 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.520+0000 7f6629a00640 1 --2- 172.21.15.107:0/4009738931 >> [v2:172.21.15.204:6838/1857316822,v1:172.21.15.204:6839/1857316822] conn(0x563c9f987850 0x563c9f983a20 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=a44d68dc1b0b9951 server_cookie=7049135fda0b752c in_seq=0 out_seq=0 2024-03-08T20:59:37.510 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.520+0000 7f660b400640 10 client.15063 ms_handle_connect on v2:172.21.15.204:6838/1857316822 2024-03-08T20:59:37.512 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.521+0000 7f660b400640 1 -- 172.21.15.107:0/4009738931 <== mds.4 v2:172.21.15.204:6838/1857316822 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x563c9f881f80 con 0x563c9f987850 2024-03-08T20:59:37.512 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.521+0000 7f660b400640 10 client.15063 handle_command_reply: tid=0 2024-03-08T20:59:37.512 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.521+0000 7f662f00b740 1 client.15063 shutdown 2024-03-08T20:59:37.512 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.521+0000 7f662f00b740 1 -- 172.21.15.107:0/4009738931 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563c9f91b2c0 msgr2=0x563c9f982150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:59:37.512 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.521+0000 7f662f00b740 1 --2- 172.21.15.107:0/4009738931 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563c9f91b2c0 0x563c9f982150 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f661000bfa0 tx=0x7f661000b4d0 comp rx=0 tx=0).stop 2024-03-08T20:59:37.512 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.521+0000 7f662f00b740 1 -- 172.21.15.107:0/4009738931 shutdown_connections 2024-03-08T20:59:37.512 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.521+0000 7f662f00b740 1 --2- 172.21.15.107:0/4009738931 >> [v2:172.21.15.204:6838/1857316822,v1:172.21.15.204:6839/1857316822] conn(0x563c9f987850 0x563c9f983a20 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:37.512 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.521+0000 7f662f00b740 1 --2- 172.21.15.107:0/4009738931 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563c9f91b2c0 0x563c9f982150 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:37.513 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.521+0000 7f662f00b740 1 --2- 172.21.15.107:0/4009738931 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563c9f916150 0x563c9f9241e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:37.513 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.521+0000 7f662f00b740 1 --2- 172.21.15.107:0/4009738931 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563c9f915760 0x563c9f923ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:37.513 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.521+0000 7f660b400640 0 client.15063 ms_handle_reset on v2:172.21.15.204:6838/1857316822 2024-03-08T20:59:37.513 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.521+0000 7f662f00b740 1 -- 172.21.15.107:0/4009738931 >> 172.21.15.107:0/4009738931 conn(0x563c9f910cd0 msgr2=0x563c9f90ba10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:59:37.513 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.521+0000 7f662f00b740 1 -- 172.21.15.107:0/4009738931 shutdown_connections 2024-03-08T20:59:37.513 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.521+0000 7f662f00b740 1 -- 172.21.15.107:0/4009738931 wait complete. 2024-03-08T20:59:37.513 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.521+0000 7f662f00b740 20 client.15063 trim_cache size 0 max 16384 2024-03-08T20:59:37.514 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-03-08T20:59:37.514 INFO:teuthology.orchestra.run.smithi107.stdout: "cluster_fsid": "dc20f0f8-dd8c-11ee-95c6-87774f69a715", 2024-03-08T20:59:37.514 INFO:teuthology.orchestra.run.smithi107.stdout: "whoami": 4, 2024-03-08T20:59:37.514 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 24457, 2024-03-08T20:59:37.514 INFO:teuthology.orchestra.run.smithi107.stdout: "want_state": "up:active", 2024-03-08T20:59:37.514 INFO:teuthology.orchestra.run.smithi107.stdout: "state": "up:active", 2024-03-08T20:59:37.514 INFO:teuthology.orchestra.run.smithi107.stdout: "fs_name": "cephfs", 2024-03-08T20:59:37.515 INFO:teuthology.orchestra.run.smithi107.stdout: "rank_uptime": 100.65229014800001, 2024-03-08T20:59:37.515 INFO:teuthology.orchestra.run.smithi107.stdout: "mdsmap_epoch": 27, 2024-03-08T20:59:37.515 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_epoch": 88, 2024-03-08T20:59:37.515 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_epoch_barrier": 86, 2024-03-08T20:59:37.515 INFO:teuthology.orchestra.run.smithi107.stdout: "uptime": 125.216842447 2024-03-08T20:59:37.515 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-03-08T20:59:37.516 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.522+0000 7f662ae00640 1 -- 172.21.15.107:0/424184027 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f660c0a2800 msgr2=0x7f660c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:59:37.516 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.522+0000 7f662ae00640 1 --2- 172.21.15.107:0/424184027 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f660c0a2800 0x7f660c0a4cc0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f661c0a2c60 tx=0x7f661c004a40 comp rx=0 tx=0).stop 2024-03-08T20:59:37.517 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.522+0000 7f662ae00640 1 -- 172.21.15.107:0/424184027 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6624104d60 msgr2=0x7f66241a4f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:59:37.517 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.522+0000 7f662ae00640 1 --2- 172.21.15.107:0/424184027 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6624104d60 0x7f66241a4f90 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f661c00b970 tx=0x7f661c00be30 comp rx=0 tx=0).stop 2024-03-08T20:59:37.517 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.522+0000 7f662ae00640 1 -- 172.21.15.107:0/424184027 shutdown_connections 2024-03-08T20:59:37.517 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.522+0000 7f662ae00640 1 --2- 172.21.15.107:0/424184027 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f660c0a2800 0x7f660c0a4cc0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:37.517 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.522+0000 7f662ae00640 1 --2- 172.21.15.107:0/424184027 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6624104d60 0x7f66241a4f90 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:37.517 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.522+0000 7f662ae00640 1 --2- 172.21.15.107:0/424184027 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f66241023b0 0x7f662419df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:37.517 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.522+0000 7f662ae00640 1 --2- 172.21.15.107:0/424184027 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f66240ffa40 0x7f662419d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:37.517 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.522+0000 7f662ae00640 1 -- 172.21.15.107:0/424184027 >> 172.21.15.107:0/424184027 conn(0x7f66240fb660 msgr2=0x7f6624102d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:59:37.517 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.522+0000 7f662ae00640 1 -- 172.21.15.107:0/424184027 shutdown_connections 2024-03-08T20:59:37.518 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.522+0000 7f662ae00640 1 -- 172.21.15.107:0/424184027 wait complete. 2024-03-08T20:59:37.521 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.k status 2024-03-08T20:59:37.730 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.746+0000 7fdc6ae00640 1 -- 172.21.15.107:0/3955317433 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdc64102d80 msgr2=0x7fdc64109610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:59:37.730 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.746+0000 7fdc6ae00640 1 --2- 172.21.15.107:0/3955317433 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdc64102d80 0x7fdc64109610 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fdc5c009ca0 tx=0x7fdc5c02c0f0 comp rx=0 tx=0).stop 2024-03-08T20:59:37.730 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.746+0000 7fdc6ae00640 1 -- 172.21.15.107:0/3955317433 shutdown_connections 2024-03-08T20:59:37.730 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.746+0000 7fdc6ae00640 1 --2- 172.21.15.107:0/3955317433 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdc64102d80 0x7fdc64109610 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:37.730 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.746+0000 7fdc6ae00640 1 --2- 172.21.15.107:0/3955317433 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdc641023c0 0x7fdc64102840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:37.730 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.746+0000 7fdc6ae00640 1 --2- 172.21.15.107:0/3955317433 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdc641011d0 0x7fdc641015f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:37.730 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.746+0000 7fdc6ae00640 1 -- 172.21.15.107:0/3955317433 >> 172.21.15.107:0/3955317433 conn(0x7fdc640fc760 msgr2=0x7fdc640feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:59:37.730 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.747+0000 7fdc6ae00640 1 -- 172.21.15.107:0/3955317433 shutdown_connections 2024-03-08T20:59:37.731 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.747+0000 7fdc6ae00640 1 -- 172.21.15.107:0/3955317433 wait complete. 2024-03-08T20:59:37.731 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.747+0000 7fdc6ae00640 1 Processor -- start 2024-03-08T20:59:37.731 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.748+0000 7fdc6ae00640 1 -- start start 2024-03-08T20:59:37.733 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.748+0000 7fdc6ae00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdc641011d0 0x7fdc64199570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:37.733 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.748+0000 7fdc6ae00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdc641023c0 0x7fdc64199ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:37.733 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.748+0000 7fdc6ae00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdc64102d80 0x7fdc641a0b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:37.733 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.748+0000 7fdc6ae00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc6410cd10 con 0x7fdc641023c0 2024-03-08T20:59:37.733 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.748+0000 7fdc6ae00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc6410cb90 con 0x7fdc641011d0 2024-03-08T20:59:37.733 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.748+0000 7fdc6ae00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc6410ce90 con 0x7fdc64102d80 2024-03-08T20:59:37.733 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.748+0000 7fdc69000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdc641023c0 0x7fdc64199ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:59:37.733 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.748+0000 7fdc69000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdc641023c0 0x7fdc64199ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:56980/0 (socket says 172.21.15.107:56980) 2024-03-08T20:59:37.734 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.748+0000 7fdc69000640 1 -- 172.21.15.107:0/549565978 learned_addr learned my addr 172.21.15.107:0/549565978 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T20:59:37.734 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.749+0000 7fdc69a00640 1 --2- 172.21.15.107:0/549565978 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdc641011d0 0x7fdc64199570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:59:37.734 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.749+0000 7fdc69000640 1 -- 172.21.15.107:0/549565978 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdc64102d80 msgr2=0x7fdc641a0b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:59:37.734 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.749+0000 7fdc6a400640 1 --2- 172.21.15.107:0/549565978 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdc64102d80 0x7fdc641a0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:59:37.734 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.749+0000 7fdc69000640 1 --2- 172.21.15.107:0/549565978 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdc64102d80 0x7fdc641a0b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:37.734 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.749+0000 7fdc69000640 1 -- 172.21.15.107:0/549565978 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdc641011d0 msgr2=0x7fdc64199570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:59:37.734 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.749+0000 7fdc69000640 1 --2- 172.21.15.107:0/549565978 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdc641011d0 0x7fdc64199570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:37.734 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.749+0000 7fdc69000640 1 -- 172.21.15.107:0/549565978 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc641a1190 con 0x7fdc641023c0 2024-03-08T20:59:37.734 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.749+0000 7fdc69a00640 1 --2- 172.21.15.107:0/549565978 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdc641011d0 0x7fdc64199570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T20:59:37.734 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.749+0000 7fdc6a400640 1 --2- 172.21.15.107:0/549565978 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdc64102d80 0x7fdc641a0b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T20:59:37.734 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.749+0000 7fdc69000640 1 --2- 172.21.15.107:0/549565978 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdc641023c0 0x7fdc64199ab0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fdc5000d950 tx=0x7fdc5000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:59:37.735 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.749+0000 7fdc62000640 1 -- 172.21.15.107:0/549565978 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc50017070 con 0x7fdc641023c0 2024-03-08T20:59:37.735 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.749+0000 7fdc62000640 1 -- 172.21.15.107:0/549565978 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fdc50009c30 con 0x7fdc641023c0 2024-03-08T20:59:37.735 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.749+0000 7fdc62000640 1 -- 172.21.15.107:0/549565978 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc5000f620 con 0x7fdc641023c0 2024-03-08T20:59:37.735 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.749+0000 7fdc6ae00640 1 -- 172.21.15.107:0/549565978 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc641a1480 con 0x7fdc641023c0 2024-03-08T20:59:37.735 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.749+0000 7fdc6ae00640 1 -- 172.21.15.107:0/549565978 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc641a1a20 con 0x7fdc641023c0 2024-03-08T20:59:37.737 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.753+0000 7fdc6f03a740 1 Processor -- start 2024-03-08T20:59:37.737 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.753+0000 7fdc6f03a740 1 -- start start 2024-03-08T20:59:37.737 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.753+0000 7fdc62000640 1 -- 172.21.15.107:0/549565978 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) v1 ==== 137291+0+0 (secure 0 0 0) 0x7fdc500109a0 con 0x7fdc641023c0 2024-03-08T20:59:37.737 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.753+0000 7fdc6f03a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ca9ce63760 0x55ca9ce63b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:37.738 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.753+0000 7fdc6f03a740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ca9ce64150 0x55ca9ce68c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:37.738 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.753+0000 7fdc6f03a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ca9ce692c0 0x55ca9ce6b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:37.738 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.753+0000 7fdc6f03a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca9cdc86f0 con 0x55ca9ce692c0 2024-03-08T20:59:37.738 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.753+0000 7fdc6f03a740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca9cd719e0 con 0x55ca9ce64150 2024-03-08T20:59:37.738 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.753+0000 7fdc6f03a740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca9cdc4790 con 0x55ca9ce63760 2024-03-08T20:59:37.738 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.754+0000 7fdc69000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ca9ce692c0 0x55ca9ce6b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:59:37.738 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.754+0000 7fdc69000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ca9ce692c0 0x55ca9ce6b750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:56992/0 (socket says 172.21.15.107:56992) 2024-03-08T20:59:37.738 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.754+0000 7fdc69000640 1 -- 172.21.15.107:0/1678271237 learned_addr learned my addr 172.21.15.107:0/1678271237 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T20:59:37.738 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.754+0000 7fdc69a00640 1 --2- 172.21.15.107:0/1678271237 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ca9ce64150 0x55ca9ce68c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:59:37.738 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.754+0000 7fdc6a400640 1 --2- 172.21.15.107:0/1678271237 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ca9ce63760 0x55ca9ce63b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:59:37.738 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.754+0000 7fdc69000640 1 -- 172.21.15.107:0/1678271237 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ca9ce63760 msgr2=0x55ca9ce63b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T20:59:37.739 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.754+0000 7fdc69000640 1 --2- 172.21.15.107:0/1678271237 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ca9ce63760 0x55ca9ce63b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:37.739 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.754+0000 7fdc69000640 1 -- 172.21.15.107:0/1678271237 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ca9ce64150 msgr2=0x55ca9ce68c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T20:59:37.739 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.754+0000 7fdc69000640 1 --2- 172.21.15.107:0/1678271237 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ca9ce64150 0x55ca9ce68c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:37.739 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.754+0000 7fdc69000640 1 -- 172.21.15.107:0/1678271237 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ca9cd65720 con 0x55ca9ce692c0 2024-03-08T20:59:37.739 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.754+0000 7fdc69a00640 1 --2- 172.21.15.107:0/1678271237 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ca9ce64150 0x55ca9ce68c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T20:59:37.739 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.755+0000 7fdc69000640 1 --2- 172.21.15.107:0/1678271237 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ca9ce692c0 0x55ca9ce6b750 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fdc500d4d00 tx=0x7fdc500d6750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ae02acb5ab3d4ba server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:59:37.739 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.755+0000 7fdc62000640 1 --2- 172.21.15.107:0/549565978 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fdc4c0a2880 0x7fdc4c0a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:37.739 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.755+0000 7fdc62000640 1 -- 172.21.15.107:0/549565978 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fdc500ce6e0 con 0x7fdc641023c0 2024-03-08T20:59:37.739 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.755+0000 7fdc56a00640 1 -- 172.21.15.107:0/1678271237 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc50097320 con 0x55ca9ce692c0 2024-03-08T20:59:37.739 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.755+0000 7fdc6a400640 1 --2- 172.21.15.107:0/549565978 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fdc4c0a2880 0x7fdc4c0a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:59:37.739 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.755+0000 7fdc56a00640 1 -- 172.21.15.107:0/1678271237 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fdc500db070 con 0x55ca9ce692c0 2024-03-08T20:59:37.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.755+0000 7fdc56a00640 1 -- 172.21.15.107:0/1678271237 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc5009a9e0 con 0x55ca9ce692c0 2024-03-08T20:59:37.744 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.756+0000 7fdc6f03a740 1 -- 172.21.15.107:0/1678271237 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ca9ce692c0 msgr2=0x55ca9ce6b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:59:37.744 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.756+0000 7fdc6a400640 1 --2- 172.21.15.107:0/549565978 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fdc4c0a2880 0x7fdc4c0a4d40 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fdc5c032420 tx=0x7fdc5c005f80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:59:37.744 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.756+0000 7fdc6f03a740 1 --2- 172.21.15.107:0/1678271237 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ca9ce692c0 0x55ca9ce6b750 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fdc500d4d00 tx=0x7fdc500d6750 comp rx=0 tx=0).stop 2024-03-08T20:59:37.744 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.756+0000 7fdc6f03a740 1 -- 172.21.15.107:0/1678271237 shutdown_connections 2024-03-08T20:59:37.744 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.756+0000 7fdc6f03a740 1 --2- 172.21.15.107:0/1678271237 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ca9ce692c0 0x55ca9ce6b750 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:37.744 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.756+0000 7fdc6f03a740 1 --2- 172.21.15.107:0/1678271237 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ca9ce64150 0x55ca9ce68c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:37.744 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.756+0000 7fdc6f03a740 1 --2- 172.21.15.107:0/1678271237 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ca9ce63760 0x55ca9ce63b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:37.744 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.756+0000 7fdc6f03a740 1 -- 172.21.15.107:0/1678271237 >> 172.21.15.107:0/1678271237 conn(0x55ca9ce5ecd0 msgr2=0x55ca9ce5f0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:59:37.744 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.756+0000 7fdc6f03a740 1 -- 172.21.15.107:0/1678271237 shutdown_connections 2024-03-08T20:59:37.745 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.756+0000 7fdc6f03a740 1 -- 172.21.15.107:0/1678271237 wait complete. 2024-03-08T20:59:37.745 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.756+0000 7fdc6f03a740 1 -- start start 2024-03-08T20:59:37.745 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.757+0000 7fdc6f03a740 1 Processor -- start 2024-03-08T20:59:37.745 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.757+0000 7fdc6f03a740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ca9ce63760 0x55ca9ce6d980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:37.745 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.757+0000 7fdc6f03a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ca9ce64150 0x55ca9ce6dec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:37.745 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.757+0000 7fdc6f03a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ca9ce692c0 0x55ca9ce74f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:37.745 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.757+0000 7fdc6f03a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca9ce6c270 con 0x55ca9ce692c0 2024-03-08T20:59:37.745 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.757+0000 7fdc6f03a740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca9cd84e20 con 0x55ca9ce63760 2024-03-08T20:59:37.745 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.757+0000 7fdc6f03a740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca9ce6c3f0 con 0x55ca9ce64150 2024-03-08T20:59:37.745 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.757+0000 7fdc6f03a740 10 client.0 _ll_register_callbacks cb 0x55ca9cdbf8d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T20:59:37.745 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.758+0000 7fdc6a400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ca9ce692c0 0x55ca9ce74f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:59:37.745 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.758+0000 7fdc69a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ca9ce63760 0x55ca9ce6d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:59:37.745 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.758+0000 7fdc69000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ca9ce64150 0x55ca9ce6dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:59:37.745 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.758+0000 7fdc6a400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ca9ce692c0 0x55ca9ce74f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:57006/0 (socket says 172.21.15.107:57006) 2024-03-08T20:59:37.746 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.758+0000 7fdc6a400640 1 -- 172.21.15.107:0/2545986746 learned_addr learned my addr 172.21.15.107:0/2545986746 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T20:59:37.746 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.758+0000 7fdc6a400640 1 -- 172.21.15.107:0/2545986746 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ca9ce64150 msgr2=0x55ca9ce6dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T20:59:37.746 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.758+0000 7fdc6a400640 1 --2- 172.21.15.107:0/2545986746 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ca9ce64150 0x55ca9ce6dec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:37.746 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.758+0000 7fdc6a400640 1 -- 172.21.15.107:0/2545986746 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ca9ce63760 msgr2=0x55ca9ce6d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T20:59:37.746 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.758+0000 7fdc6a400640 1 --2- 172.21.15.107:0/2545986746 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ca9ce63760 0x55ca9ce6d980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:37.746 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.758+0000 7fdc6a400640 1 -- 172.21.15.107:0/2545986746 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ca9cadd2d0 con 0x55ca9ce692c0 2024-03-08T20:59:37.746 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.758+0000 7fdc6a400640 1 --2- 172.21.15.107:0/2545986746 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ca9ce692c0 0x55ca9ce74f00 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fdc5c036000 tx=0x7fdc5c02c0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e1b4f9c8da36f74 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:59:37.746 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.759+0000 7fdc4b400640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-03-08T20:59:37.746 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.759+0000 7fdc4b400640 1 -- 172.21.15.107:0/2545986746 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc5c04f070 con 0x55ca9ce692c0 2024-03-08T20:59:37.746 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.759+0000 7fdc4b400640 1 -- 172.21.15.107:0/2545986746 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fdc5c0041b0 con 0x55ca9ce692c0 2024-03-08T20:59:37.746 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.759+0000 7fdc6f03a740 1 -- 172.21.15.107:0/2545986746 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdc500db070 con 0x55ca9ce692c0 2024-03-08T20:59:37.746 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.759+0000 7fdc4b400640 1 -- 172.21.15.107:0/2545986746 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdc5c05b050 con 0x55ca9ce692c0 2024-03-08T20:59:37.747 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.759+0000 7fdc6f03a740 10 client.15075 fetch_fsmap learned FSMap version 27 2024-03-08T20:59:37.747 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.759+0000 7fdc6f03a740 1 -- 172.21.15.107:0/2545986746 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55ca9cba76b0 con 0x55ca9ce692c0 2024-03-08T20:59:37.747 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.759+0000 7fdc4b400640 1 -- 172.21.15.107:0/2545986746 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fdc500db070 con 0x55ca9ce692c0 2024-03-08T20:59:37.747 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.759+0000 7fdc4b400640 1 -- 172.21.15.107:0/2545986746 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fdc5c04a070 con 0x55ca9ce692c0 2024-03-08T20:59:37.747 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.760+0000 7fdc6f03a740 10 client.15075 fetch_fsmap finished waiting for FSMap version 27 2024-03-08T20:59:37.747 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.760+0000 7fdc6f03a740 10 client.15075 resolve_mds: resolved name 'k' to daemon mds.k 2024-03-08T20:59:37.747 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.760+0000 7fdc6f03a740 20 client.15075 populate_metadata read hostname 'smithi107' 2024-03-08T20:59:37.747 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.760+0000 7fdc6f03a740 1 --2- 172.21.15.107:0/2545986746 >> [v2:172.21.15.149:6832/1962161726,v1:172.21.15.149:6833/1962161726] conn(0x55ca9ce7a550 0x55ca9ce795d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:37.747 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.760+0000 7fdc6f03a740 4 client.15075 mds_command: new command op to 34397 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-08T20:59:37.747 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.760+0000 7fdc6f03a740 1 -- 172.21.15.107:0/2545986746 --> [v2:172.21.15.149:6832/1962161726,v1:172.21.15.149:6833/1962161726] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ca9cdd5c30 con 0x55ca9ce7a550 2024-03-08T20:59:37.747 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.760+0000 7fdc69a00640 1 --2- 172.21.15.107:0/2545986746 >> [v2:172.21.15.149:6832/1962161726,v1:172.21.15.149:6833/1962161726] conn(0x55ca9ce7a550 0x55ca9ce795d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:59:37.747 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.761+0000 7fdc69a00640 1 --2- 172.21.15.107:0/2545986746 >> [v2:172.21.15.149:6832/1962161726,v1:172.21.15.149:6833/1962161726] conn(0x55ca9ce7a550 0x55ca9ce795d0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=17880b82eb37b6a8 server_cookie=1773fbeda171b4dc in_seq=0 out_seq=0 2024-03-08T20:59:37.747 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.761+0000 7fdc4b400640 10 client.15075 ms_handle_connect on v2:172.21.15.149:6832/1962161726 2024-03-08T20:59:37.749 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.763+0000 7fdc4b400640 1 -- 172.21.15.107:0/2545986746 <== mds.3 v2:172.21.15.149:6832/1962161726 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ca9cdd5c30 con 0x55ca9ce7a550 2024-03-08T20:59:37.749 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.763+0000 7fdc4b400640 10 client.15075 handle_command_reply: tid=0 2024-03-08T20:59:37.749 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.764+0000 7fdc6f03a740 1 client.15075 shutdown 2024-03-08T20:59:37.749 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.764+0000 7fdc6f03a740 1 -- 172.21.15.107:0/2545986746 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ca9ce692c0 msgr2=0x55ca9ce74f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:59:37.749 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.764+0000 7fdc6f03a740 1 --2- 172.21.15.107:0/2545986746 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ca9ce692c0 0x55ca9ce74f00 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fdc5c036000 tx=0x7fdc5c02c0f0 comp rx=0 tx=0).stop 2024-03-08T20:59:37.749 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.764+0000 7fdc6f03a740 1 -- 172.21.15.107:0/2545986746 shutdown_connections 2024-03-08T20:59:37.749 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.764+0000 7fdc6f03a740 1 --2- 172.21.15.107:0/2545986746 >> [v2:172.21.15.149:6832/1962161726,v1:172.21.15.149:6833/1962161726] conn(0x55ca9ce7a550 0x55ca9ce795d0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:37.749 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.764+0000 7fdc6f03a740 1 --2- 172.21.15.107:0/2545986746 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ca9ce692c0 0x55ca9ce74f00 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:37.749 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.764+0000 7fdc6f03a740 1 --2- 172.21.15.107:0/2545986746 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ca9ce64150 0x55ca9ce6dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:37.749 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.764+0000 7fdc6f03a740 1 --2- 172.21.15.107:0/2545986746 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ca9ce63760 0x55ca9ce6d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:37.750 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.764+0000 7fdc4b400640 0 client.15075 ms_handle_reset on v2:172.21.15.149:6832/1962161726 2024-03-08T20:59:37.750 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.764+0000 7fdc6f03a740 1 -- 172.21.15.107:0/2545986746 >> 172.21.15.107:0/2545986746 conn(0x55ca9ce5ecd0 msgr2=0x55ca9ce5dee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:59:37.750 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.764+0000 7fdc6f03a740 1 -- 172.21.15.107:0/2545986746 shutdown_connections 2024-03-08T20:59:37.750 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.764+0000 7fdc6f03a740 1 -- 172.21.15.107:0/2545986746 wait complete. 2024-03-08T20:59:37.750 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.764+0000 7fdc6f03a740 20 client.15075 trim_cache size 0 max 16384 2024-03-08T20:59:37.765 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.781+0000 7fdc6f03a740 1 Processor -- start 2024-03-08T20:59:37.765 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.781+0000 7fdc6f03a740 1 -- start start 2024-03-08T20:59:37.765 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.781+0000 7fdc6f03a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ca9ce63760 0x55ca9ce59a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:37.765 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.781+0000 7fdc6f03a740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ca9ce64150 0x55ca9ce74e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:37.765 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.781+0000 7fdc6f03a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ca9ce692c0 0x55ca9ce5f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:37.766 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.781+0000 7fdc6f03a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca9ce79d70 con 0x55ca9ce63760 2024-03-08T20:59:37.766 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.781+0000 7fdc6f03a740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca9ce7a3d0 con 0x55ca9ce64150 2024-03-08T20:59:37.766 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.781+0000 7fdc6f03a740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca9ce7aa30 con 0x55ca9ce692c0 2024-03-08T20:59:37.766 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.782+0000 7fdc69a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ca9ce63760 0x55ca9ce59a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:59:37.766 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.782+0000 7fdc69a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ca9ce63760 0x55ca9ce59a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:57018/0 (socket says 172.21.15.107:57018) 2024-03-08T20:59:37.766 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.782+0000 7fdc69a00640 1 -- 172.21.15.107:0/3062159359 learned_addr learned my addr 172.21.15.107:0/3062159359 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T20:59:37.766 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.782+0000 7fdc69000640 1 --2- 172.21.15.107:0/3062159359 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ca9ce64150 0x55ca9ce74e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:59:37.766 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.782+0000 7fdc6a400640 1 --2- 172.21.15.107:0/3062159359 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ca9ce692c0 0x55ca9ce5f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:59:37.766 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.782+0000 7fdc69a00640 1 -- 172.21.15.107:0/3062159359 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ca9ce692c0 msgr2=0x55ca9ce5f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T20:59:37.766 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.782+0000 7fdc69a00640 1 --2- 172.21.15.107:0/3062159359 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ca9ce692c0 0x55ca9ce5f0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:37.766 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.782+0000 7fdc69a00640 1 -- 172.21.15.107:0/3062159359 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ca9ce64150 msgr2=0x55ca9ce74e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T20:59:37.766 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.782+0000 7fdc69a00640 1 --2- 172.21.15.107:0/3062159359 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ca9ce64150 0x55ca9ce74e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:37.767 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.782+0000 7fdc69a00640 1 -- 172.21.15.107:0/3062159359 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ca9ce59f50 con 0x55ca9ce63760 2024-03-08T20:59:37.767 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.782+0000 7fdc69000640 1 --2- 172.21.15.107:0/3062159359 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ca9ce64150 0x55ca9ce74e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T20:59:37.767 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.782+0000 7fdc69a00640 1 --2- 172.21.15.107:0/3062159359 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ca9ce63760 0x55ca9ce59a10 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x55ca9ce64bc0 tx=0x7fdc580094e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a852fed4a06720c6 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:59:37.767 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.783+0000 7fdc56000640 1 -- 172.21.15.107:0/3062159359 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc580142d0 con 0x55ca9ce63760 2024-03-08T20:59:37.767 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.783+0000 7fdc56000640 1 -- 172.21.15.107:0/3062159359 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fdc5801e030 con 0x55ca9ce63760 2024-03-08T20:59:37.767 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.783+0000 7fdc56000640 1 -- 172.21.15.107:0/3062159359 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc580126b0 con 0x55ca9ce63760 2024-03-08T20:59:37.771 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.783+0000 7fdc6f03a740 1 -- 172.21.15.107:0/3062159359 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ca9ce63760 msgr2=0x55ca9ce59a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:59:37.771 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.783+0000 7fdc6f03a740 1 --2- 172.21.15.107:0/3062159359 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ca9ce63760 0x55ca9ce59a10 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x55ca9ce64bc0 tx=0x7fdc580094e0 comp rx=0 tx=0).stop 2024-03-08T20:59:37.771 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.783+0000 7fdc6f03a740 1 -- 172.21.15.107:0/3062159359 shutdown_connections 2024-03-08T20:59:37.772 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.783+0000 7fdc6f03a740 1 --2- 172.21.15.107:0/3062159359 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ca9ce692c0 0x55ca9ce5f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:37.772 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.783+0000 7fdc6f03a740 1 --2- 172.21.15.107:0/3062159359 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ca9ce64150 0x55ca9ce74e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:37.772 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.783+0000 7fdc6f03a740 1 --2- 172.21.15.107:0/3062159359 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ca9ce63760 0x55ca9ce59a10 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:37.772 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.783+0000 7fdc6f03a740 1 -- 172.21.15.107:0/3062159359 >> 172.21.15.107:0/3062159359 conn(0x55ca9ce5ecd0 msgr2=0x55ca9ce689b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:59:37.772 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.783+0000 7fdc6f03a740 1 -- 172.21.15.107:0/3062159359 shutdown_connections 2024-03-08T20:59:37.772 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.784+0000 7fdc6f03a740 1 -- 172.21.15.107:0/3062159359 wait complete. 2024-03-08T20:59:37.772 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.784+0000 7fdc6f03a740 1 -- start start 2024-03-08T20:59:37.772 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.784+0000 7fdc6f03a740 1 Processor -- start 2024-03-08T20:59:37.772 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.784+0000 7fdc6f03a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ca9ce63760 0x55ca9ce71eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:37.772 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.784+0000 7fdc6f03a740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ca9ce64150 0x55ca9ce723f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:37.772 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.784+0000 7fdc6f03a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ca9ce692c0 0x55ca9ced0150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:37.772 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.784+0000 7fdc6f03a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca9ce7b6f0 con 0x55ca9ce63760 2024-03-08T20:59:37.772 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.784+0000 7fdc6f03a740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca9ce7b090 con 0x55ca9ce64150 2024-03-08T20:59:37.772 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.784+0000 7fdc6f03a740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca9ce7bd50 con 0x55ca9ce692c0 2024-03-08T20:59:37.773 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.785+0000 7fdc6f03a740 10 client.0 _ll_register_callbacks cb 0x7fdc500d44e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T20:59:37.773 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.785+0000 7fdc69a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ca9ce63760 0x55ca9ce71eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:59:37.773 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.785+0000 7fdc69a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ca9ce63760 0x55ca9ce71eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:57024/0 (socket says 172.21.15.107:57024) 2024-03-08T20:59:37.773 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.785+0000 7fdc69a00640 1 -- 172.21.15.107:0/1724247820 learned_addr learned my addr 172.21.15.107:0/1724247820 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T20:59:37.773 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.785+0000 7fdc69000640 1 --2- 172.21.15.107:0/1724247820 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ca9ce64150 0x55ca9ce723f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:59:37.773 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.785+0000 7fdc69a00640 1 -- 172.21.15.107:0/1724247820 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ca9ce692c0 msgr2=0x55ca9ced0150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T20:59:37.773 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.785+0000 7fdc69a00640 1 --2- 172.21.15.107:0/1724247820 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ca9ce692c0 0x55ca9ced0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:37.773 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.785+0000 7fdc69a00640 1 -- 172.21.15.107:0/1724247820 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ca9ce64150 msgr2=0x55ca9ce723f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T20:59:37.773 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.785+0000 7fdc69a00640 1 --2- 172.21.15.107:0/1724247820 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ca9ce64150 0x55ca9ce723f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:37.773 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.785+0000 7fdc69a00640 1 -- 172.21.15.107:0/1724247820 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ca9ced0690 con 0x55ca9ce63760 2024-03-08T20:59:37.773 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.785+0000 7fdc69a00640 1 --2- 172.21.15.107:0/1724247820 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ca9ce63760 0x55ca9ce71eb0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fdc58005820 tx=0x7fdc580140d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9c9a17461eb1b88 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:59:37.773 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.785+0000 7fdc4b400640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-03-08T20:59:37.773 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.786+0000 7fdc4b400640 1 -- 172.21.15.107:0/1724247820 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc58030060 con 0x55ca9ce63760 2024-03-08T20:59:37.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.786+0000 7fdc4b400640 1 -- 172.21.15.107:0/1724247820 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fdc5801a4b0 con 0x55ca9ce63760 2024-03-08T20:59:37.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.786+0000 7fdc6f03a740 1 -- 172.21.15.107:0/1724247820 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdc5801e030 con 0x55ca9ce63760 2024-03-08T20:59:37.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.786+0000 7fdc4b400640 1 -- 172.21.15.107:0/1724247820 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdc580142c0 con 0x55ca9ce63760 2024-03-08T20:59:37.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.786+0000 7fdc6f03a740 10 client.15081 fetch_fsmap learned FSMap version 27 2024-03-08T20:59:37.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.786+0000 7fdc6f03a740 1 -- 172.21.15.107:0/1724247820 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55ca9ced1880 con 0x55ca9ce63760 2024-03-08T20:59:37.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.786+0000 7fdc4b400640 1 -- 172.21.15.107:0/1724247820 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fdc5801e030 con 0x55ca9ce63760 2024-03-08T20:59:37.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.786+0000 7fdc4b400640 1 -- 172.21.15.107:0/1724247820 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fdc58035070 con 0x55ca9ce63760 2024-03-08T20:59:37.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.786+0000 7fdc6f03a740 10 client.15081 fetch_fsmap finished waiting for FSMap version 27 2024-03-08T20:59:37.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.786+0000 7fdc6f03a740 10 client.15081 resolve_mds: resolved name 'k' to daemon mds.k 2024-03-08T20:59:37.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.786+0000 7fdc6f03a740 20 client.15081 populate_metadata read hostname 'smithi107' 2024-03-08T20:59:37.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.786+0000 7fdc6f03a740 1 --2- 172.21.15.107:0/1724247820 >> [v2:172.21.15.149:6832/1962161726,v1:172.21.15.149:6833/1962161726] conn(0x55ca9ced5c10 0x55ca9ced1b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:37.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.786+0000 7fdc6f03a740 4 client.15081 mds_command: new command op to 34397 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-08T20:59:37.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.786+0000 7fdc6f03a740 1 -- 172.21.15.107:0/1724247820 --> [v2:172.21.15.149:6832/1962161726,v1:172.21.15.149:6833/1962161726] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55ca9cdcff80 con 0x55ca9ced5c10 2024-03-08T20:59:37.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.787+0000 7fdc69000640 1 --2- 172.21.15.107:0/1724247820 >> [v2:172.21.15.149:6832/1962161726,v1:172.21.15.149:6833/1962161726] conn(0x55ca9ced5c10 0x55ca9ced1b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:59:37.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.788+0000 7fdc69000640 1 --2- 172.21.15.107:0/1724247820 >> [v2:172.21.15.149:6832/1962161726,v1:172.21.15.149:6833/1962161726] conn(0x55ca9ced5c10 0x55ca9ced1b60 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=f193c904f4997461 server_cookie=776f192ae0000a2f in_seq=0 out_seq=0 2024-03-08T20:59:37.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.788+0000 7fdc4b400640 10 client.15081 ms_handle_connect on v2:172.21.15.149:6832/1962161726 2024-03-08T20:59:37.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.788+0000 7fdc4b400640 1 -- 172.21.15.107:0/1724247820 <== mds.3 v2:172.21.15.149:6832/1962161726 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x55ca9cdcff80 con 0x55ca9ced5c10 2024-03-08T20:59:37.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.788+0000 7fdc4b400640 10 client.15081 handle_command_reply: tid=0 2024-03-08T20:59:37.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.788+0000 7fdc6f03a740 1 client.15081 shutdown 2024-03-08T20:59:37.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.789+0000 7fdc6f03a740 1 -- 172.21.15.107:0/1724247820 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ca9ce63760 msgr2=0x55ca9ce71eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:59:37.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.789+0000 7fdc6f03a740 1 --2- 172.21.15.107:0/1724247820 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ca9ce63760 0x55ca9ce71eb0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fdc58005820 tx=0x7fdc580140d0 comp rx=0 tx=0).stop 2024-03-08T20:59:37.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.789+0000 7fdc6f03a740 1 -- 172.21.15.107:0/1724247820 shutdown_connections 2024-03-08T20:59:37.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.789+0000 7fdc6f03a740 1 --2- 172.21.15.107:0/1724247820 >> [v2:172.21.15.149:6832/1962161726,v1:172.21.15.149:6833/1962161726] conn(0x55ca9ced5c10 0x55ca9ced1b60 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:37.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.789+0000 7fdc6f03a740 1 --2- 172.21.15.107:0/1724247820 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ca9ce692c0 0x55ca9ced0150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:37.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.789+0000 7fdc6f03a740 1 --2- 172.21.15.107:0/1724247820 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ca9ce64150 0x55ca9ce723f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:37.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.789+0000 7fdc6f03a740 1 --2- 172.21.15.107:0/1724247820 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ca9ce63760 0x55ca9ce71eb0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:37.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.789+0000 7fdc4b400640 0 client.15081 ms_handle_reset on v2:172.21.15.149:6832/1962161726 2024-03-08T20:59:37.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.789+0000 7fdc6f03a740 1 -- 172.21.15.107:0/1724247820 >> 172.21.15.107:0/1724247820 conn(0x55ca9ce5ecd0 msgr2=0x55ca9ce829f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:59:37.776 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.789+0000 7fdc6f03a740 1 -- 172.21.15.107:0/1724247820 shutdown_connections 2024-03-08T20:59:37.776 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.789+0000 7fdc6f03a740 1 -- 172.21.15.107:0/1724247820 wait complete. 2024-03-08T20:59:37.776 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.789+0000 7fdc6f03a740 20 client.15081 trim_cache size 0 max 16384 2024-03-08T20:59:37.776 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-03-08T20:59:37.776 INFO:teuthology.orchestra.run.smithi107.stdout: "cluster_fsid": "dc20f0f8-dd8c-11ee-95c6-87774f69a715", 2024-03-08T20:59:37.776 INFO:teuthology.orchestra.run.smithi107.stdout: "whoami": 3, 2024-03-08T20:59:37.776 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 34397, 2024-03-08T20:59:37.776 INFO:teuthology.orchestra.run.smithi107.stdout: "want_state": "up:active", 2024-03-08T20:59:37.776 INFO:teuthology.orchestra.run.smithi107.stdout: "state": "up:active", 2024-03-08T20:59:37.777 INFO:teuthology.orchestra.run.smithi107.stdout: "fs_name": "cephfs", 2024-03-08T20:59:37.777 INFO:teuthology.orchestra.run.smithi107.stdout: "rank_uptime": 101.944199164, 2024-03-08T20:59:37.777 INFO:teuthology.orchestra.run.smithi107.stdout: "mdsmap_epoch": 27, 2024-03-08T20:59:37.777 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_epoch": 88, 2024-03-08T20:59:37.777 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_epoch_barrier": 86, 2024-03-08T20:59:37.777 INFO:teuthology.orchestra.run.smithi107.stdout: "uptime": 153.81068683300001 2024-03-08T20:59:37.777 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-03-08T20:59:37.778 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.789+0000 7fdc6ae00640 1 -- 172.21.15.107:0/549565978 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fdc4c0a2880 msgr2=0x7fdc4c0a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:59:37.778 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.789+0000 7fdc6ae00640 1 --2- 172.21.15.107:0/549565978 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fdc4c0a2880 0x7fdc4c0a4d40 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fdc5c032420 tx=0x7fdc5c005f80 comp rx=0 tx=0).stop 2024-03-08T20:59:37.778 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.789+0000 7fdc6ae00640 1 -- 172.21.15.107:0/549565978 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdc641023c0 msgr2=0x7fdc64199ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:59:37.778 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.789+0000 7fdc6ae00640 1 --2- 172.21.15.107:0/549565978 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdc641023c0 0x7fdc64199ab0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fdc5000d950 tx=0x7fdc5000de10 comp rx=0 tx=0).stop 2024-03-08T20:59:37.778 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.789+0000 7fdc6ae00640 1 -- 172.21.15.107:0/549565978 shutdown_connections 2024-03-08T20:59:37.778 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.790+0000 7fdc6ae00640 1 --2- 172.21.15.107:0/549565978 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fdc4c0a2880 0x7fdc4c0a4d40 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:37.778 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.790+0000 7fdc6ae00640 1 --2- 172.21.15.107:0/549565978 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdc64102d80 0x7fdc641a0b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:37.778 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.790+0000 7fdc6ae00640 1 --2- 172.21.15.107:0/549565978 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdc641023c0 0x7fdc64199ab0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:37.778 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.790+0000 7fdc6ae00640 1 --2- 172.21.15.107:0/549565978 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdc641011d0 0x7fdc64199570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:37.778 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.790+0000 7fdc6ae00640 1 -- 172.21.15.107:0/549565978 >> 172.21.15.107:0/549565978 conn(0x7fdc640fc760 msgr2=0x7fdc64073520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:59:37.778 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.790+0000 7fdc6ae00640 1 -- 172.21.15.107:0/549565978 shutdown_connections 2024-03-08T20:59:37.779 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:37.790+0000 7fdc6ae00640 1 -- 172.21.15.107:0/549565978 wait complete. 2024-03-08T20:59:37.787 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:59:37 smithi107 ceph-mon[31381]: from='client.? 172.21.15.107:0/1764362259' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-08T20:59:37.787 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:59:37 smithi107 ceph-mon[31381]: from='client.? 172.21.15.107:0/1379066268' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-08T20:59:37.789 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.e status 2024-03-08T20:59:37.878 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:59:37 smithi204 ceph-mon[34935]: from='client.? 172.21.15.107:0/1764362259' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-08T20:59:37.878 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:59:37 smithi204 ceph-mon[34935]: from='client.? 172.21.15.107:0/1379066268' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-08T20:59:37.993 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.010+0000 7f3b27800640 1 -- 172.21.15.107:0/1363932587 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3b20108b00 msgr2=0x7f3b20108f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:59:37.993 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.010+0000 7f3b27800640 1 --2- 172.21.15.107:0/1363932587 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3b20108b00 0x7f3b20108f80 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f3b0c009900 tx=0x7f3b0c02bc30 comp rx=0 tx=0).stop 2024-03-08T20:59:37.993 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.010+0000 7f3b27800640 1 -- 172.21.15.107:0/1363932587 shutdown_connections 2024-03-08T20:59:37.993 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.010+0000 7f3b27800640 1 --2- 172.21.15.107:0/1363932587 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3b201094c0 0x7f3b2010fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:37.993 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.010+0000 7f3b27800640 1 --2- 172.21.15.107:0/1363932587 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3b20108b00 0x7f3b20108f80 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:37.993 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.010+0000 7f3b27800640 1 --2- 172.21.15.107:0/1363932587 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3b20107910 0x7f3b20107d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:37.993 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.010+0000 7f3b27800640 1 -- 172.21.15.107:0/1363932587 >> 172.21.15.107:0/1363932587 conn(0x7f3b200768b0 msgr2=0x7f3b20078cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:59:37.994 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.010+0000 7f3b27800640 1 -- 172.21.15.107:0/1363932587 shutdown_connections 2024-03-08T20:59:37.994 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.010+0000 7f3b27800640 1 -- 172.21.15.107:0/1363932587 wait complete. 2024-03-08T20:59:37.994 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.011+0000 7f3b27800640 1 Processor -- start 2024-03-08T20:59:37.996 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.011+0000 7f3b27800640 1 -- start start 2024-03-08T20:59:37.996 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.011+0000 7f3b27800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3b20107910 0x7f3b2019db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:37.996 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.011+0000 7f3b27800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3b20108b00 0x7f3b2019e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:37.996 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.011+0000 7f3b27800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3b201094c0 0x7f3b201a5110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:37.997 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.011+0000 7f3b27800640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b20113490 con 0x7f3b20107910 2024-03-08T20:59:37.997 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.011+0000 7f3b27800640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b20113310 con 0x7f3b201094c0 2024-03-08T20:59:37.997 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.011+0000 7f3b27800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b20113610 con 0x7f3b20108b00 2024-03-08T20:59:37.997 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.012+0000 7f3b24e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3b20107910 0x7f3b2019db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:59:37.997 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.012+0000 7f3b24e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3b20107910 0x7f3b2019db60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:57044/0 (socket says 172.21.15.107:57044) 2024-03-08T20:59:37.997 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.012+0000 7f3b24e00640 1 -- 172.21.15.107:0/2005066469 learned_addr learned my addr 172.21.15.107:0/2005066469 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T20:59:37.997 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.012+0000 7f3b25800640 1 --2- 172.21.15.107:0/2005066469 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3b201094c0 0x7f3b201a5110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:59:37.997 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.012+0000 7f3b1fe00640 1 --2- 172.21.15.107:0/2005066469 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3b20108b00 0x7f3b2019e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:59:37.997 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.012+0000 7f3b24e00640 1 -- 172.21.15.107:0/2005066469 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3b20108b00 msgr2=0x7f3b2019e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:59:37.998 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.012+0000 7f3b24e00640 1 --2- 172.21.15.107:0/2005066469 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3b20108b00 0x7f3b2019e0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:37.998 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.012+0000 7f3b24e00640 1 -- 172.21.15.107:0/2005066469 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3b201094c0 msgr2=0x7f3b201a5110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:59:37.998 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.012+0000 7f3b24e00640 1 --2- 172.21.15.107:0/2005066469 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3b201094c0 0x7f3b201a5110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:37.998 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.012+0000 7f3b24e00640 1 -- 172.21.15.107:0/2005066469 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b201a5780 con 0x7f3b20107910 2024-03-08T20:59:37.998 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.012+0000 7f3b1fe00640 1 --2- 172.21.15.107:0/2005066469 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3b20108b00 0x7f3b2019e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T20:59:37.998 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.012+0000 7f3b25800640 1 --2- 172.21.15.107:0/2005066469 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3b201094c0 0x7f3b201a5110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T20:59:37.998 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.012+0000 7f3b24e00640 1 --2- 172.21.15.107:0/2005066469 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3b20107910 0x7f3b2019db60 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f3b1400d950 tx=0x7f3b1400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:59:37.998 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.013+0000 7f3b1d600640 1 -- 172.21.15.107:0/2005066469 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b14017070 con 0x7f3b20107910 2024-03-08T20:59:37.998 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.013+0000 7f3b1d600640 1 -- 172.21.15.107:0/2005066469 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f3b14009c30 con 0x7f3b20107910 2024-03-08T20:59:37.998 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.013+0000 7f3b1d600640 1 -- 172.21.15.107:0/2005066469 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b1400f620 con 0x7f3b20107910 2024-03-08T20:59:37.998 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.013+0000 7f3b27800640 1 -- 172.21.15.107:0/2005066469 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b201a5a70 con 0x7f3b20107910 2024-03-08T20:59:37.999 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.013+0000 7f3b27800640 1 -- 172.21.15.107:0/2005066469 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b201a5f70 con 0x7f3b20107910 2024-03-08T20:59:38.005 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.016+0000 7f3b2a60d740 1 Processor -- start 2024-03-08T20:59:38.005 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.016+0000 7f3b1d600640 1 -- 172.21.15.107:0/2005066469 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f3b1401a070 con 0x7f3b20107910 2024-03-08T20:59:38.005 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.016+0000 7f3b2a60d740 1 -- start start 2024-03-08T20:59:38.006 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.016+0000 7f3b2a60d740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559211079590 0x5592110799b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:38.006 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.016+0000 7f3b2a60d740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559211079f80 0x55921107ea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:38.006 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.016+0000 7f3b2a60d740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55921107f0f0 0x559211081580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:38.006 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.016+0000 7f3b2a60d740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x559210fde6f0 con 0x559211079f80 2024-03-08T20:59:38.006 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.016+0000 7f3b2a60d740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x559210f879e0 con 0x55921107f0f0 2024-03-08T20:59:38.006 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.016+0000 7f3b2a60d740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x559210fda790 con 0x559211079590 2024-03-08T20:59:38.006 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.017+0000 7f3b24e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559211079f80 0x55921107ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:59:38.006 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.017+0000 7f3b24e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559211079f80 0x55921107ea80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:57050/0 (socket says 172.21.15.107:57050) 2024-03-08T20:59:38.006 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.017+0000 7f3b24e00640 1 -- 172.21.15.107:0/2780490482 learned_addr learned my addr 172.21.15.107:0/2780490482 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T20:59:38.006 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.017+0000 7f3b25800640 1 --2- 172.21.15.107:0/2780490482 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559211079590 0x5592110799b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:59:38.006 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.017+0000 7f3b1fe00640 1 --2- 172.21.15.107:0/2780490482 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55921107f0f0 0x559211081580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:59:38.006 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.017+0000 7f3b24e00640 1 -- 172.21.15.107:0/2780490482 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559211079590 msgr2=0x5592110799b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T20:59:38.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.017+0000 7f3b24e00640 1 --2- 172.21.15.107:0/2780490482 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559211079590 0x5592110799b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:38.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.017+0000 7f3b24e00640 1 -- 172.21.15.107:0/2780490482 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55921107f0f0 msgr2=0x559211081580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T20:59:38.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.017+0000 7f3b24e00640 1 --2- 172.21.15.107:0/2780490482 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55921107f0f0 0x559211081580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:38.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.017+0000 7f3b24e00640 1 -- 172.21.15.107:0/2780490482 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559210f7b720 con 0x559211079f80 2024-03-08T20:59:38.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.017+0000 7f3b25800640 1 --2- 172.21.15.107:0/2780490482 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559211079590 0x5592110799b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T20:59:38.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.017+0000 7f3b1fe00640 1 --2- 172.21.15.107:0/2780490482 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55921107f0f0 0x559211081580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T20:59:38.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.017+0000 7f3b1d600640 1 --2- 172.21.15.107:0/2005066469 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f3b080a2800 0x7f3b080a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:38.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.017+0000 7f3b24e00640 1 --2- 172.21.15.107:0/2780490482 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559211079f80 0x55921107ea80 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f3b140d4760 tx=0x7f3b140d4c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d318f6624b4f08ff server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:59:38.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.018+0000 7f3b1d600640 1 -- 172.21.15.107:0/2005066469 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f3b140ce540 con 0x7f3b20107910 2024-03-08T20:59:38.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.018+0000 7f3b25800640 1 --2- 172.21.15.107:0/2005066469 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f3b080a2800 0x7f3b080a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:59:38.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.018+0000 7f3b12000640 1 -- 172.21.15.107:0/2780490482 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b14097180 con 0x559211079f80 2024-03-08T20:59:38.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.018+0000 7f3b12000640 1 -- 172.21.15.107:0/2780490482 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f3b1409b5e0 con 0x559211079f80 2024-03-08T20:59:38.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.018+0000 7f3b12000640 1 -- 172.21.15.107:0/2780490482 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b1409a840 con 0x559211079f80 2024-03-08T20:59:38.008 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.019+0000 7f3b2a60d740 1 -- 172.21.15.107:0/2780490482 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559211079f80 msgr2=0x55921107ea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:59:38.008 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.019+0000 7f3b25800640 1 --2- 172.21.15.107:0/2005066469 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f3b080a2800 0x7f3b080a4cc0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x55921107a7e0 tx=0x7f3b180094f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:59:38.008 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.019+0000 7f3b2a60d740 1 --2- 172.21.15.107:0/2780490482 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559211079f80 0x55921107ea80 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f3b140d4760 tx=0x7f3b140d4c20 comp rx=0 tx=0).stop 2024-03-08T20:59:38.008 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.019+0000 7f3b2a60d740 1 -- 172.21.15.107:0/2780490482 shutdown_connections 2024-03-08T20:59:38.008 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.019+0000 7f3b2a60d740 1 --2- 172.21.15.107:0/2780490482 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55921107f0f0 0x559211081580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:38.008 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.019+0000 7f3b2a60d740 1 --2- 172.21.15.107:0/2780490482 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559211079f80 0x55921107ea80 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:38.008 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.019+0000 7f3b2a60d740 1 --2- 172.21.15.107:0/2780490482 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559211079590 0x5592110799b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:38.008 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.019+0000 7f3b2a60d740 1 -- 172.21.15.107:0/2780490482 >> 172.21.15.107:0/2780490482 conn(0x559211074b00 msgr2=0x559211074f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:59:38.008 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.019+0000 7f3b2a60d740 1 -- 172.21.15.107:0/2780490482 shutdown_connections 2024-03-08T20:59:38.008 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.019+0000 7f3b2a60d740 1 -- 172.21.15.107:0/2780490482 wait complete. 2024-03-08T20:59:38.008 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.019+0000 7f3b2a60d740 1 -- start start 2024-03-08T20:59:38.008 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.019+0000 7f3b2a60d740 1 Processor -- start 2024-03-08T20:59:38.008 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.020+0000 7f3b2a60d740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559211079590 0x5592110836c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:38.009 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.020+0000 7f3b2a60d740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559211079f80 0x559211083c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:38.009 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.020+0000 7f3b2a60d740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55921107f0f0 0x55921108ac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:38.009 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.020+0000 7f3b2a60d740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592110820a0 con 0x55921107f0f0 2024-03-08T20:59:38.009 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.020+0000 7f3b2a60d740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x559210f9ae20 con 0x559211079f80 2024-03-08T20:59:38.009 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.020+0000 7f3b2a60d740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x559211082220 con 0x559211079590 2024-03-08T20:59:38.009 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.020+0000 7f3b2a60d740 10 client.0 _ll_register_callbacks cb 0x559210fd58d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T20:59:38.009 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.020+0000 7f3b25800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55921107f0f0 0x55921108ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:59:38.009 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.020+0000 7f3b25800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55921107f0f0 0x55921108ac40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:57066/0 (socket says 172.21.15.107:57066) 2024-03-08T20:59:38.009 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.020+0000 7f3b25800640 1 -- 172.21.15.107:0/2625504890 learned_addr learned my addr 172.21.15.107:0/2625504890 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T20:59:38.009 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.020+0000 7f3b1fe00640 1 --2- 172.21.15.107:0/2625504890 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559211079f80 0x559211083c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:59:38.009 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.020+0000 7f3b24e00640 1 --2- 172.21.15.107:0/2625504890 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559211079590 0x5592110836c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:59:38.009 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.021+0000 7f3b25800640 1 -- 172.21.15.107:0/2625504890 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559211079590 msgr2=0x5592110836c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T20:59:38.009 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.021+0000 7f3b25800640 1 --2- 172.21.15.107:0/2625504890 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559211079590 0x5592110836c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:38.010 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.021+0000 7f3b25800640 1 -- 172.21.15.107:0/2625504890 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559211079f80 msgr2=0x559211083c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T20:59:38.010 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.021+0000 7f3b25800640 1 --2- 172.21.15.107:0/2625504890 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559211079f80 0x559211083c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:38.010 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.021+0000 7f3b25800640 1 -- 172.21.15.107:0/2625504890 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559210cf32d0 con 0x55921107f0f0 2024-03-08T20:59:38.010 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.021+0000 7f3b25800640 1 --2- 172.21.15.107:0/2625504890 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55921107f0f0 0x55921108ac40 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f3b18011c20 tx=0x7f3b18004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8c774f47a83b6e7 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:59:38.010 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.021+0000 7f3b06a00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-03-08T20:59:38.010 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.021+0000 7f3b06a00640 1 -- 172.21.15.107:0/2625504890 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b18011e50 con 0x55921107f0f0 2024-03-08T20:59:38.010 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.021+0000 7f3b06a00640 1 -- 172.21.15.107:0/2625504890 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f3b18004b20 con 0x55921107f0f0 2024-03-08T20:59:38.010 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.021+0000 7f3b2a60d740 1 -- 172.21.15.107:0/2625504890 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3b1409b5e0 con 0x55921107f0f0 2024-03-08T20:59:38.010 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.022+0000 7f3b06a00640 1 -- 172.21.15.107:0/2625504890 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3b18004d20 con 0x55921107f0f0 2024-03-08T20:59:38.010 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.022+0000 7f3b2a60d740 10 client.15093 fetch_fsmap learned FSMap version 27 2024-03-08T20:59:38.010 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.022+0000 7f3b2a60d740 1 -- 172.21.15.107:0/2625504890 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x559210dbd6b0 con 0x55921107f0f0 2024-03-08T20:59:38.010 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.022+0000 7f3b06a00640 1 -- 172.21.15.107:0/2625504890 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f3b1409b5e0 con 0x55921107f0f0 2024-03-08T20:59:38.010 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.022+0000 7f3b06a00640 1 -- 172.21.15.107:0/2625504890 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f3b18026570 con 0x55921107f0f0 2024-03-08T20:59:38.010 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.022+0000 7f3b2a60d740 10 client.15093 fetch_fsmap finished waiting for FSMap version 27 2024-03-08T20:59:38.011 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.022+0000 7f3b2a60d740 10 client.15093 resolve_mds: resolved name 'e' to daemon mds.e 2024-03-08T20:59:38.011 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.022+0000 7f3b2a60d740 20 client.15093 populate_metadata read hostname 'smithi107' 2024-03-08T20:59:38.011 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.022+0000 7f3b2a60d740 1 --2- 172.21.15.107:0/2625504890 >> [v2:172.21.15.149:6834/442657066,v1:172.21.15.149:6835/442657066] conn(0x5592110902d0 0x55921108f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:38.011 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.022+0000 7f3b2a60d740 4 client.15093 mds_command: new command op to 34415 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-08T20:59:38.011 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.022+0000 7f3b2a60d740 1 -- 172.21.15.107:0/2625504890 --> [v2:172.21.15.149:6834/442657066,v1:172.21.15.149:6835/442657066] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559210febc30 con 0x5592110902d0 2024-03-08T20:59:38.011 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.023+0000 7f3b24e00640 1 --2- 172.21.15.107:0/2625504890 >> [v2:172.21.15.149:6834/442657066,v1:172.21.15.149:6835/442657066] conn(0x5592110902d0 0x55921108f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:59:38.012 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.024+0000 7f3b24e00640 1 --2- 172.21.15.107:0/2625504890 >> [v2:172.21.15.149:6834/442657066,v1:172.21.15.149:6835/442657066] conn(0x5592110902d0 0x55921108f2e0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c9348b438c772ae3 server_cookie=250e674f950834ca in_seq=0 out_seq=0 2024-03-08T20:59:38.012 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.024+0000 7f3b06a00640 10 client.15093 ms_handle_connect on v2:172.21.15.149:6834/442657066 2024-03-08T20:59:38.012 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.025+0000 7f3b06a00640 1 -- 172.21.15.107:0/2625504890 <== mds.2 v2:172.21.15.149:6834/442657066 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x559210febc30 con 0x5592110902d0 2024-03-08T20:59:38.012 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.025+0000 7f3b06a00640 10 client.15093 handle_command_reply: tid=0 2024-03-08T20:59:38.012 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.026+0000 7f3b2a60d740 1 client.15093 shutdown 2024-03-08T20:59:38.012 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.026+0000 7f3b2a60d740 1 -- 172.21.15.107:0/2625504890 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55921107f0f0 msgr2=0x55921108ac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:59:38.013 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.026+0000 7f3b2a60d740 1 --2- 172.21.15.107:0/2625504890 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55921107f0f0 0x55921108ac40 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f3b18011c20 tx=0x7f3b18004a40 comp rx=0 tx=0).stop 2024-03-08T20:59:38.013 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.026+0000 7f3b2a60d740 1 -- 172.21.15.107:0/2625504890 shutdown_connections 2024-03-08T20:59:38.013 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.026+0000 7f3b2a60d740 1 --2- 172.21.15.107:0/2625504890 >> [v2:172.21.15.149:6834/442657066,v1:172.21.15.149:6835/442657066] conn(0x5592110902d0 0x55921108f2e0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:38.013 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.026+0000 7f3b2a60d740 1 --2- 172.21.15.107:0/2625504890 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55921107f0f0 0x55921108ac40 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:38.013 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.026+0000 7f3b2a60d740 1 --2- 172.21.15.107:0/2625504890 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559211079f80 0x559211083c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:38.013 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.026+0000 7f3b2a60d740 1 --2- 172.21.15.107:0/2625504890 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559211079590 0x5592110836c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:38.013 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.026+0000 7f3b06a00640 0 client.15093 ms_handle_reset on v2:172.21.15.149:6834/442657066 2024-03-08T20:59:38.013 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.026+0000 7f3b2a60d740 1 -- 172.21.15.107:0/2625504890 >> 172.21.15.107:0/2625504890 conn(0x559211074b00 msgr2=0x559211073c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:59:38.013 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.026+0000 7f3b2a60d740 1 -- 172.21.15.107:0/2625504890 shutdown_connections 2024-03-08T20:59:38.013 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.026+0000 7f3b2a60d740 1 -- 172.21.15.107:0/2625504890 wait complete. 2024-03-08T20:59:38.013 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.026+0000 7f3b2a60d740 20 client.15093 trim_cache size 0 max 16384 2024-03-08T20:59:38.014 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:59:37 smithi149 ceph-mon[35042]: from='client.? 172.21.15.107:0/1764362259' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-08T20:59:38.015 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:59:37 smithi149 ceph-mon[35042]: from='client.? 172.21.15.107:0/1379066268' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-08T20:59:38.026 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.043+0000 7f3b2a60d740 1 Processor -- start 2024-03-08T20:59:38.026 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.043+0000 7f3b2a60d740 1 -- start start 2024-03-08T20:59:38.026 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.044+0000 7f3b2a60d740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559211079590 0x55921108f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:38.026 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.044+0000 7f3b2a60d740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559211079f80 0x55921108ab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:38.026 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.044+0000 7f3b2a60d740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55921107f0f0 0x5592110836c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:38.026 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.044+0000 7f3b2a60d740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55921108faf0 con 0x559211079590 2024-03-08T20:59:38.026 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.044+0000 7f3b2a60d740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x559211090150 con 0x559211079f80 2024-03-08T20:59:38.026 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.044+0000 7f3b2a60d740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592110907b0 con 0x55921107f0f0 2024-03-08T20:59:38.031 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.044+0000 7f3b24e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559211079590 0x55921108f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:59:38.031 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.044+0000 7f3b24e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559211079590 0x55921108f2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:57078/0 (socket says 172.21.15.107:57078) 2024-03-08T20:59:38.031 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.044+0000 7f3b24e00640 1 -- 172.21.15.107:0/141589351 learned_addr learned my addr 172.21.15.107:0/141589351 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T20:59:38.031 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.044+0000 7f3b1fe00640 1 --2- 172.21.15.107:0/141589351 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559211079f80 0x55921108ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:59:38.031 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.044+0000 7f3b24e00640 1 -- 172.21.15.107:0/141589351 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55921107f0f0 msgr2=0x5592110836c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T20:59:38.031 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.044+0000 7f3b24e00640 1 --2- 172.21.15.107:0/141589351 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55921107f0f0 0x5592110836c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:38.031 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.044+0000 7f3b24e00640 1 -- 172.21.15.107:0/141589351 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559211079f80 msgr2=0x55921108ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T20:59:38.032 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.044+0000 7f3b24e00640 1 --2- 172.21.15.107:0/141589351 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559211079f80 0x55921108ab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:38.032 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.044+0000 7f3b24e00640 1 -- 172.21.15.107:0/141589351 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55921107a810 con 0x559211079590 2024-03-08T20:59:38.032 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.045+0000 7f3b24e00640 1 --2- 172.21.15.107:0/141589351 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559211079590 0x55921108f2e0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f3b1409abe0 tx=0x7f3b14005f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d69d278074b94b1 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:59:38.032 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.045+0000 7f3b11600640 1 -- 172.21.15.107:0/141589351 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b14021040 con 0x559211079590 2024-03-08T20:59:38.032 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.045+0000 7f3b11600640 1 -- 172.21.15.107:0/141589351 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f3b140d1d90 con 0x559211079590 2024-03-08T20:59:38.032 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.045+0000 7f3b11600640 1 -- 172.21.15.107:0/141589351 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b1409aa20 con 0x559211079590 2024-03-08T20:59:38.032 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.045+0000 7f3b2a60d740 1 -- 172.21.15.107:0/141589351 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559211079590 msgr2=0x55921108f2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:59:38.032 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.045+0000 7f3b2a60d740 1 --2- 172.21.15.107:0/141589351 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559211079590 0x55921108f2e0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f3b1409abe0 tx=0x7f3b14005f30 comp rx=0 tx=0).stop 2024-03-08T20:59:38.032 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.045+0000 7f3b2a60d740 1 -- 172.21.15.107:0/141589351 shutdown_connections 2024-03-08T20:59:38.032 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.045+0000 7f3b2a60d740 1 --2- 172.21.15.107:0/141589351 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55921107f0f0 0x5592110836c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:38.032 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.045+0000 7f3b2a60d740 1 --2- 172.21.15.107:0/141589351 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559211079f80 0x55921108ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:38.032 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.045+0000 7f3b2a60d740 1 --2- 172.21.15.107:0/141589351 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559211079590 0x55921108f2e0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:38.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.045+0000 7f3b2a60d740 1 -- 172.21.15.107:0/141589351 >> 172.21.15.107:0/141589351 conn(0x559211074b00 msgr2=0x55921106f8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:59:38.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.045+0000 7f3b2a60d740 1 -- 172.21.15.107:0/141589351 shutdown_connections 2024-03-08T20:59:38.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.046+0000 7f3b2a60d740 1 -- 172.21.15.107:0/141589351 wait complete. 2024-03-08T20:59:38.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.046+0000 7f3b2a60d740 1 -- start start 2024-03-08T20:59:38.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.046+0000 7f3b2a60d740 1 Processor -- start 2024-03-08T20:59:38.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.046+0000 7f3b2a60d740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559211079590 0x559211086f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:38.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.046+0000 7f3b2a60d740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559211079f80 0x559211087470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:38.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.046+0000 7f3b2a60d740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55921107f0f0 0x55921108a430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:38.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.046+0000 7f3b2a60d740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x559211091470 con 0x55921107f0f0 2024-03-08T20:59:38.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.046+0000 7f3b2a60d740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x559211090e10 con 0x559211079f80 2024-03-08T20:59:38.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.047+0000 7f3b2a60d740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x559211091ad0 con 0x559211079590 2024-03-08T20:59:38.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.047+0000 7f3b2a60d740 10 client.0 _ll_register_callbacks cb 0x7f3b140d4060 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T20:59:38.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.047+0000 7f3b25800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55921107f0f0 0x55921108a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:59:38.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.047+0000 7f3b25800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55921107f0f0 0x55921108a430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:57084/0 (socket says 172.21.15.107:57084) 2024-03-08T20:59:38.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.047+0000 7f3b25800640 1 -- 172.21.15.107:0/41292986 learned_addr learned my addr 172.21.15.107:0/41292986 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T20:59:38.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.047+0000 7f3b1fe00640 1 --2- 172.21.15.107:0/41292986 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559211079f80 0x559211087470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:59:38.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.047+0000 7f3b24e00640 1 --2- 172.21.15.107:0/41292986 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559211079590 0x559211086f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:59:38.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.047+0000 7f3b25800640 1 -- 172.21.15.107:0/41292986 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559211079590 msgr2=0x559211086f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T20:59:38.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.047+0000 7f3b25800640 1 --2- 172.21.15.107:0/41292986 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559211079590 0x559211086f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:38.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.047+0000 7f3b25800640 1 -- 172.21.15.107:0/41292986 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559211079f80 msgr2=0x559211087470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T20:59:38.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.047+0000 7f3b25800640 1 --2- 172.21.15.107:0/41292986 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559211079f80 0x559211087470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:38.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.047+0000 7f3b25800640 1 -- 172.21.15.107:0/41292986 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55921108aa20 con 0x55921107f0f0 2024-03-08T20:59:38.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.047+0000 7f3b1fe00640 1 --2- 172.21.15.107:0/41292986 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559211079f80 0x559211087470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T20:59:38.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.047+0000 7f3b24e00640 1 --2- 172.21.15.107:0/41292986 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559211079590 0x559211086f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T20:59:38.035 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.047+0000 7f3b25800640 1 --2- 172.21.15.107:0/41292986 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55921107f0f0 0x55921108a430 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f3b18010f70 tx=0x7f3b1800ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9b785e23892b8ba server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:59:38.035 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.047+0000 7f3b06a00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-03-08T20:59:38.035 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.048+0000 7f3b06a00640 1 -- 172.21.15.107:0/41292986 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b18011e60 con 0x55921107f0f0 2024-03-08T20:59:38.035 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.048+0000 7f3b06a00640 1 -- 172.21.15.107:0/41292986 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f3b18026600 con 0x55921107f0f0 2024-03-08T20:59:38.035 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.048+0000 7f3b2a60d740 1 -- 172.21.15.107:0/41292986 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3b140d1d90 con 0x55921107f0f0 2024-03-08T20:59:38.035 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.048+0000 7f3b06a00640 1 -- 172.21.15.107:0/41292986 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3b18031460 con 0x55921107f0f0 2024-03-08T20:59:38.035 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.048+0000 7f3b2a60d740 10 client.15099 fetch_fsmap learned FSMap version 27 2024-03-08T20:59:38.035 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.048+0000 7f3b2a60d740 1 -- 172.21.15.107:0/41292986 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5592110e6a40 con 0x55921107f0f0 2024-03-08T20:59:38.035 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.048+0000 7f3b06a00640 1 -- 172.21.15.107:0/41292986 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f3b140d1d90 con 0x55921107f0f0 2024-03-08T20:59:38.035 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.048+0000 7f3b06a00640 1 -- 172.21.15.107:0/41292986 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f3b18038020 con 0x55921107f0f0 2024-03-08T20:59:38.035 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.048+0000 7f3b2a60d740 10 client.15099 fetch_fsmap finished waiting for FSMap version 27 2024-03-08T20:59:38.035 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.048+0000 7f3b2a60d740 10 client.15099 resolve_mds: resolved name 'e' to daemon mds.e 2024-03-08T20:59:38.035 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.048+0000 7f3b2a60d740 20 client.15099 populate_metadata read hostname 'smithi107' 2024-03-08T20:59:38.035 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.048+0000 7f3b2a60d740 1 --2- 172.21.15.107:0/41292986 >> [v2:172.21.15.149:6834/442657066,v1:172.21.15.149:6835/442657066] conn(0x5592110eb120 0x5592110e6cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:38.036 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.049+0000 7f3b2a60d740 4 client.15099 mds_command: new command op to 34415 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-08T20:59:38.036 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.049+0000 7f3b2a60d740 1 -- 172.21.15.107:0/41292986 --> [v2:172.21.15.149:6834/442657066,v1:172.21.15.149:6835/442657066] -- command(tid 0: {"prefix": "status"}) v1 -- 0x559210fe5f80 con 0x5592110eb120 2024-03-08T20:59:38.037 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.049+0000 7f3b24e00640 1 --2- 172.21.15.107:0/41292986 >> [v2:172.21.15.149:6834/442657066,v1:172.21.15.149:6835/442657066] conn(0x5592110eb120 0x5592110e6cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:59:38.037 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.050+0000 7f3b24e00640 1 --2- 172.21.15.107:0/41292986 >> [v2:172.21.15.149:6834/442657066,v1:172.21.15.149:6835/442657066] conn(0x5592110eb120 0x5592110e6cd0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=fca61432045b837 server_cookie=7d2b2357418fc66e in_seq=0 out_seq=0 2024-03-08T20:59:38.037 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.050+0000 7f3b06a00640 10 client.15099 ms_handle_connect on v2:172.21.15.149:6834/442657066 2024-03-08T20:59:38.037 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.051+0000 7f3b06a00640 1 -- 172.21.15.107:0/41292986 <== mds.2 v2:172.21.15.149:6834/442657066 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x559210fe5f80 con 0x5592110eb120 2024-03-08T20:59:38.037 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.051+0000 7f3b06a00640 10 client.15099 handle_command_reply: tid=0 2024-03-08T20:59:38.037 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.051+0000 7f3b2a60d740 1 client.15099 shutdown 2024-03-08T20:59:38.037 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.051+0000 7f3b2a60d740 1 -- 172.21.15.107:0/41292986 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55921107f0f0 msgr2=0x55921108a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:59:38.037 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.051+0000 7f3b2a60d740 1 --2- 172.21.15.107:0/41292986 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55921107f0f0 0x55921108a430 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f3b18010f70 tx=0x7f3b1800ed60 comp rx=0 tx=0).stop 2024-03-08T20:59:38.038 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.051+0000 7f3b2a60d740 1 -- 172.21.15.107:0/41292986 shutdown_connections 2024-03-08T20:59:38.038 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.051+0000 7f3b2a60d740 1 --2- 172.21.15.107:0/41292986 >> [v2:172.21.15.149:6834/442657066,v1:172.21.15.149:6835/442657066] conn(0x5592110eb120 0x5592110e6cd0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:38.038 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.051+0000 7f3b2a60d740 1 --2- 172.21.15.107:0/41292986 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55921107f0f0 0x55921108a430 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:38.038 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.051+0000 7f3b2a60d740 1 --2- 172.21.15.107:0/41292986 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559211079f80 0x559211087470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:38.038 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.051+0000 7f3b06a00640 0 client.15099 ms_handle_reset on v2:172.21.15.149:6834/442657066 2024-03-08T20:59:38.038 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.051+0000 7f3b2a60d740 1 --2- 172.21.15.107:0/41292986 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559211079590 0x559211086f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:38.038 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.051+0000 7f3b2a60d740 1 -- 172.21.15.107:0/41292986 >> 172.21.15.107:0/41292986 conn(0x559211074b00 msgr2=0x559211098770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:59:38.038 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.051+0000 7f3b2a60d740 1 -- 172.21.15.107:0/41292986 shutdown_connections 2024-03-08T20:59:38.038 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.051+0000 7f3b2a60d740 1 -- 172.21.15.107:0/41292986 wait complete. 2024-03-08T20:59:38.038 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.051+0000 7f3b2a60d740 20 client.15099 trim_cache size 0 max 16384 2024-03-08T20:59:38.038 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-03-08T20:59:38.039 INFO:teuthology.orchestra.run.smithi107.stdout: "cluster_fsid": "dc20f0f8-dd8c-11ee-95c6-87774f69a715", 2024-03-08T20:59:38.039 INFO:teuthology.orchestra.run.smithi107.stdout: "whoami": 2, 2024-03-08T20:59:38.039 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 34415, 2024-03-08T20:59:38.039 INFO:teuthology.orchestra.run.smithi107.stdout: "want_state": "up:active", 2024-03-08T20:59:38.039 INFO:teuthology.orchestra.run.smithi107.stdout: "state": "up:active", 2024-03-08T20:59:38.039 INFO:teuthology.orchestra.run.smithi107.stdout: "fs_name": "cephfs", 2024-03-08T20:59:38.039 INFO:teuthology.orchestra.run.smithi107.stdout: "rank_uptime": 103.218288595, 2024-03-08T20:59:38.039 INFO:teuthology.orchestra.run.smithi107.stdout: "mdsmap_epoch": 27, 2024-03-08T20:59:38.039 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_epoch": 88, 2024-03-08T20:59:38.039 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_epoch_barrier": 86, 2024-03-08T20:59:38.039 INFO:teuthology.orchestra.run.smithi107.stdout: "uptime": 141.93189754299999 2024-03-08T20:59:38.039 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-03-08T20:59:38.040 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.052+0000 7f3b27800640 1 -- 172.21.15.107:0/2005066469 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f3b080a2800 msgr2=0x7f3b080a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:59:38.040 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.052+0000 7f3b27800640 1 --2- 172.21.15.107:0/2005066469 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f3b080a2800 0x7f3b080a4cc0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x55921107a7e0 tx=0x7f3b180094f0 comp rx=0 tx=0).stop 2024-03-08T20:59:38.040 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.052+0000 7f3b27800640 1 -- 172.21.15.107:0/2005066469 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3b20107910 msgr2=0x7f3b2019db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:59:38.040 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.052+0000 7f3b27800640 1 --2- 172.21.15.107:0/2005066469 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3b20107910 0x7f3b2019db60 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f3b1400d950 tx=0x7f3b1400de10 comp rx=0 tx=0).stop 2024-03-08T20:59:38.040 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.052+0000 7f3b27800640 1 -- 172.21.15.107:0/2005066469 shutdown_connections 2024-03-08T20:59:38.040 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.052+0000 7f3b27800640 1 --2- 172.21.15.107:0/2005066469 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f3b080a2800 0x7f3b080a4cc0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:38.040 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.052+0000 7f3b27800640 1 --2- 172.21.15.107:0/2005066469 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3b201094c0 0x7f3b201a5110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:38.040 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.052+0000 7f3b27800640 1 --2- 172.21.15.107:0/2005066469 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3b20108b00 0x7f3b2019e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:38.041 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.052+0000 7f3b27800640 1 --2- 172.21.15.107:0/2005066469 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3b20107910 0x7f3b2019db60 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:38.041 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.052+0000 7f3b27800640 1 -- 172.21.15.107:0/2005066469 >> 172.21.15.107:0/2005066469 conn(0x7f3b200768b0 msgr2=0x7f3b2010bed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:59:38.041 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.052+0000 7f3b27800640 1 -- 172.21.15.107:0/2005066469 shutdown_connections 2024-03-08T20:59:38.041 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.052+0000 7f3b27800640 1 -- 172.21.15.107:0/2005066469 wait complete. 2024-03-08T20:59:38.051 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2024-03-08T20:59:38.255 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.272+0000 7ff85c200640 1 -- 172.21.15.107:0/2129582490 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff854104720 msgr2=0x7ff854104ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:59:38.256 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.272+0000 7ff85c200640 1 --2- 172.21.15.107:0/2129582490 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff854104720 0x7ff854104ba0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7ff840009900 tx=0x7ff84002bc40 comp rx=0 tx=0).stop 2024-03-08T20:59:38.256 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.273+0000 7ff85c200640 1 -- 172.21.15.107:0/2129582490 shutdown_connections 2024-03-08T20:59:38.256 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.273+0000 7ff85c200640 1 --2- 172.21.15.107:0/2129582490 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff8541050e0 0x7ff85410b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:38.256 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.273+0000 7ff85c200640 1 --2- 172.21.15.107:0/2129582490 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff854104720 0x7ff854104ba0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:38.256 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.273+0000 7ff85c200640 1 --2- 172.21.15.107:0/2129582490 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff854103530 0x7ff854103950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:38.256 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.273+0000 7ff85c200640 1 -- 172.21.15.107:0/2129582490 >> 172.21.15.107:0/2129582490 conn(0x7ff8540feaa0 msgr2=0x7ff854100ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:59:38.256 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.273+0000 7ff85c200640 1 -- 172.21.15.107:0/2129582490 shutdown_connections 2024-03-08T20:59:38.256 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.273+0000 7ff85c200640 1 -- 172.21.15.107:0/2129582490 wait complete. 2024-03-08T20:59:38.259 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.274+0000 7ff85c200640 1 Processor -- start 2024-03-08T20:59:38.259 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.274+0000 7ff85c200640 1 -- start start 2024-03-08T20:59:38.259 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.274+0000 7ff85c200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff854103530 0x7ff8541997a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:38.259 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.274+0000 7ff85c200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff854104720 0x7ff854199ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:38.259 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.274+0000 7ff85c200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff8541050e0 0x7ff8541a0d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:38.259 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.274+0000 7ff85c200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff85410f100 con 0x7ff854104720 2024-03-08T20:59:38.260 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.274+0000 7ff85c200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff85410ef80 con 0x7ff854103530 2024-03-08T20:59:38.260 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.274+0000 7ff85c200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff85410f280 con 0x7ff8541050e0 2024-03-08T20:59:38.260 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.274+0000 7ff858e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff854104720 0x7ff854199ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:59:38.260 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.274+0000 7ff858e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff854104720 0x7ff854199ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:57108/0 (socket says 172.21.15.107:57108) 2024-03-08T20:59:38.260 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.274+0000 7ff858e00640 1 -- 172.21.15.107:0/4237187124 learned_addr learned my addr 172.21.15.107:0/4237187124 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T20:59:38.260 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.275+0000 7ff859800640 1 --2- 172.21.15.107:0/4237187124 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff854103530 0x7ff8541997a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:59:38.261 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.275+0000 7ff85a200640 1 --2- 172.21.15.107:0/4237187124 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff8541050e0 0x7ff8541a0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:59:38.261 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.275+0000 7ff858e00640 1 -- 172.21.15.107:0/4237187124 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff8541050e0 msgr2=0x7ff8541a0d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:59:38.261 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.275+0000 7ff858e00640 1 --2- 172.21.15.107:0/4237187124 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff8541050e0 0x7ff8541a0d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:38.261 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.275+0000 7ff858e00640 1 -- 172.21.15.107:0/4237187124 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff854103530 msgr2=0x7ff8541997a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:59:38.261 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.275+0000 7ff858e00640 1 --2- 172.21.15.107:0/4237187124 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff854103530 0x7ff8541997a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:38.261 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.275+0000 7ff858e00640 1 -- 172.21.15.107:0/4237187124 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8541a13c0 con 0x7ff854104720 2024-03-08T20:59:38.261 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.275+0000 7ff858e00640 1 --2- 172.21.15.107:0/4237187124 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff854104720 0x7ff854199ce0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7ff840000c00 tx=0x7ff8400042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:59:38.262 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.275+0000 7ff852000640 1 -- 172.21.15.107:0/4237187124 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff840043020 con 0x7ff854104720 2024-03-08T20:59:38.262 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.275+0000 7ff852000640 1 -- 172.21.15.107:0/4237187124 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7ff8400054c0 con 0x7ff854104720 2024-03-08T20:59:38.262 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.275+0000 7ff852000640 1 -- 172.21.15.107:0/4237187124 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff840003d80 con 0x7ff854104720 2024-03-08T20:59:38.262 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.275+0000 7ff85c200640 1 -- 172.21.15.107:0/4237187124 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8541a1650 con 0x7ff854104720 2024-03-08T20:59:38.262 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.276+0000 7ff85c200640 1 -- 172.21.15.107:0/4237187124 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8541a1c70 con 0x7ff854104720 2024-03-08T20:59:38.270 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.278+0000 7ff852000640 1 -- 172.21.15.107:0/4237187124 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) v1 ==== 137291+0+0 (secure 0 0 0) 0x7ff840046020 con 0x7ff854104720 2024-03-08T20:59:38.271 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.278+0000 7ff85eb22740 1 Processor -- start 2024-03-08T20:59:38.271 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.279+0000 7ff85eb22740 1 -- start start 2024-03-08T20:59:38.271 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.279+0000 7ff85eb22740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556cd620b590 0x556cd620b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:38.271 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.279+0000 7ff85eb22740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556cd620bf80 0x556cd6210a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:38.271 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.279+0000 7ff85eb22740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556cd62110f0 0x556cd6213580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:38.271 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.279+0000 7ff85eb22740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x556cd61706f0 con 0x556cd62110f0 2024-03-08T20:59:38.271 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.279+0000 7ff85eb22740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x556cd61199e0 con 0x556cd620b590 2024-03-08T20:59:38.271 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.279+0000 7ff85eb22740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x556cd616c790 con 0x556cd620bf80 2024-03-08T20:59:38.271 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.279+0000 7ff858e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556cd62110f0 0x556cd6213580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:59:38.272 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.279+0000 7ff858e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556cd62110f0 0x556cd6213580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:57124/0 (socket says 172.21.15.107:57124) 2024-03-08T20:59:38.272 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.279+0000 7ff858e00640 1 -- 172.21.15.107:0/2421065167 learned_addr learned my addr 172.21.15.107:0/2421065167 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T20:59:38.272 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.280+0000 7ff85a200640 1 --2- 172.21.15.107:0/2421065167 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556cd620b590 0x556cd620b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:59:38.272 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.280+0000 7ff859800640 1 --2- 172.21.15.107:0/2421065167 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556cd620bf80 0x556cd6210a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:59:38.272 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.280+0000 7ff858e00640 1 -- 172.21.15.107:0/2421065167 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556cd620bf80 msgr2=0x556cd6210a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T20:59:38.272 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.280+0000 7ff858e00640 1 --2- 172.21.15.107:0/2421065167 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556cd620bf80 0x556cd6210a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:38.272 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.280+0000 7ff858e00640 1 -- 172.21.15.107:0/2421065167 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556cd620b590 msgr2=0x556cd620b9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T20:59:38.272 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.280+0000 7ff858e00640 1 --2- 172.21.15.107:0/2421065167 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556cd620b590 0x556cd620b9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:38.272 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.280+0000 7ff858e00640 1 -- 172.21.15.107:0/2421065167 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556cd610d720 con 0x556cd62110f0 2024-03-08T20:59:38.272 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.280+0000 7ff852000640 1 --2- 172.21.15.107:0/4237187124 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7ff83c0a2880 0x7ff83c0a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:38.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.280+0000 7ff858e00640 1 --2- 172.21.15.107:0/2421065167 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556cd62110f0 0x556cd6213580 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7ff8400f4e10 tx=0x7ff8400f4f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e57fa26a2704ee2f server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:59:38.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.280+0000 7ff852000640 1 -- 172.21.15.107:0/4237187124 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7ff8400f0f00 con 0x7ff854104720 2024-03-08T20:59:38.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.280+0000 7ff85a200640 1 --2- 172.21.15.107:0/4237187124 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7ff83c0a2880 0x7ff83c0a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:59:38.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.280+0000 7ff846a00640 1 -- 172.21.15.107:0/2421065167 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8400b9c20 con 0x556cd62110f0 2024-03-08T20:59:38.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.280+0000 7ff846a00640 1 -- 172.21.15.107:0/2421065167 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7ff8400fe070 con 0x556cd62110f0 2024-03-08T20:59:38.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.280+0000 7ff846a00640 1 -- 172.21.15.107:0/2421065167 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8400bd2e0 con 0x556cd62110f0 2024-03-08T20:59:38.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.281+0000 7ff85eb22740 1 -- 172.21.15.107:0/2421065167 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556cd62110f0 msgr2=0x556cd6213580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:59:38.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.281+0000 7ff85eb22740 1 --2- 172.21.15.107:0/2421065167 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556cd62110f0 0x556cd6213580 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7ff8400f4e10 tx=0x7ff8400f4f00 comp rx=0 tx=0).stop 2024-03-08T20:59:38.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.281+0000 7ff85a200640 1 --2- 172.21.15.107:0/4237187124 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7ff83c0a2880 0x7ff83c0a4d40 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x556cd620c7e0 tx=0x7ff84c008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:59:38.274 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.281+0000 7ff85eb22740 1 -- 172.21.15.107:0/2421065167 shutdown_connections 2024-03-08T20:59:38.274 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.281+0000 7ff85eb22740 1 --2- 172.21.15.107:0/2421065167 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556cd62110f0 0x556cd6213580 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:38.274 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.281+0000 7ff85eb22740 1 --2- 172.21.15.107:0/2421065167 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556cd620bf80 0x556cd6210a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:38.274 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.281+0000 7ff85eb22740 1 --2- 172.21.15.107:0/2421065167 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556cd620b590 0x556cd620b9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:38.274 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.281+0000 7ff85eb22740 1 -- 172.21.15.107:0/2421065167 >> 172.21.15.107:0/2421065167 conn(0x556cd6206b00 msgr2=0x556cd6206f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:59:38.274 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.281+0000 7ff85eb22740 1 -- 172.21.15.107:0/2421065167 shutdown_connections 2024-03-08T20:59:38.274 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.281+0000 7ff85eb22740 1 -- 172.21.15.107:0/2421065167 wait complete. 2024-03-08T20:59:38.274 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.282+0000 7ff85eb22740 1 -- start start 2024-03-08T20:59:38.274 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.282+0000 7ff85eb22740 1 Processor -- start 2024-03-08T20:59:38.275 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.282+0000 7ff85eb22740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556cd620b590 0x556cd62154a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:38.275 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.282+0000 7ff85eb22740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556cd620bf80 0x556cd62159e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:38.275 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.282+0000 7ff85eb22740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556cd62110f0 0x556cd621ca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:38.275 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.282+0000 7ff85eb22740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x556cd6213e60 con 0x556cd620bf80 2024-03-08T20:59:38.275 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.282+0000 7ff85eb22740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x556cd612ce20 con 0x556cd62110f0 2024-03-08T20:59:38.275 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.282+0000 7ff85eb22740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x556cd6213fe0 con 0x556cd620b590 2024-03-08T20:59:38.275 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.282+0000 7ff85eb22740 10 client.0 _ll_register_callbacks cb 0x556cd61678d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T20:59:38.275 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.283+0000 7ff858e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556cd620bf80 0x556cd62159e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:59:38.275 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.283+0000 7ff858e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556cd620bf80 0x556cd62159e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:57132/0 (socket says 172.21.15.107:57132) 2024-03-08T20:59:38.275 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.283+0000 7ff858e00640 1 -- 172.21.15.107:0/594787849 learned_addr learned my addr 172.21.15.107:0/594787849 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T20:59:38.276 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.283+0000 7ff85a200640 1 --2- 172.21.15.107:0/594787849 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556cd62110f0 0x556cd621ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:59:38.276 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.283+0000 7ff858e00640 1 -- 172.21.15.107:0/594787849 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556cd620b590 msgr2=0x556cd62154a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T20:59:38.276 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.283+0000 7ff858e00640 1 --2- 172.21.15.107:0/594787849 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556cd620b590 0x556cd62154a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:38.276 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.283+0000 7ff858e00640 1 -- 172.21.15.107:0/594787849 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556cd62110f0 msgr2=0x556cd621ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T20:59:38.276 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.283+0000 7ff858e00640 1 --2- 172.21.15.107:0/594787849 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556cd62110f0 0x556cd621ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:38.276 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.283+0000 7ff858e00640 1 -- 172.21.15.107:0/594787849 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556cd5e852d0 con 0x556cd620bf80 2024-03-08T20:59:38.276 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.283+0000 7ff85a200640 1 --2- 172.21.15.107:0/594787849 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556cd62110f0 0x556cd621ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T20:59:38.276 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.283+0000 7ff858e00640 1 --2- 172.21.15.107:0/594787849 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556cd620bf80 0x556cd62159e0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7ff84002c220 tx=0x7ff840003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe2ff87c060859d server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:59:38.276 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.283+0000 7ff83b400640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-03-08T20:59:38.277 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.284+0000 7ff83b400640 1 -- 172.21.15.107:0/594787849 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8400bd4c0 con 0x556cd620bf80 2024-03-08T20:59:38.277 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.284+0000 7ff83b400640 1 -- 172.21.15.107:0/594787849 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7ff8400bbc80 con 0x556cd620bf80 2024-03-08T20:59:38.277 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.284+0000 7ff85eb22740 1 -- 172.21.15.107:0/594787849 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff8400fe070 con 0x556cd620bf80 2024-03-08T20:59:38.277 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.284+0000 7ff83b400640 1 -- 172.21.15.107:0/594787849 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff8400bd750 con 0x556cd620bf80 2024-03-08T20:59:38.277 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.284+0000 7ff85eb22740 10 client.15111 fetch_fsmap learned FSMap version 27 2024-03-08T20:59:38.277 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.284+0000 7ff85eb22740 1 -- 172.21.15.107:0/594787849 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x556cd5f4f6b0 con 0x556cd620bf80 2024-03-08T20:59:38.277 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.284+0000 7ff83b400640 1 -- 172.21.15.107:0/594787849 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7ff8400fe070 con 0x556cd620bf80 2024-03-08T20:59:38.277 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.284+0000 7ff83b400640 1 -- 172.21.15.107:0/594787849 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7ff8400f2030 con 0x556cd620bf80 2024-03-08T20:59:38.277 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.284+0000 7ff85eb22740 10 client.15111 fetch_fsmap finished waiting for FSMap version 27 2024-03-08T20:59:38.277 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.284+0000 7ff85eb22740 10 client.15111 resolve_mds: resolved name 'h' to daemon mds.h 2024-03-08T20:59:38.277 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.284+0000 7ff85eb22740 20 client.15111 populate_metadata read hostname 'smithi107' 2024-03-08T20:59:38.278 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.284+0000 7ff85eb22740 1 --2- 172.21.15.107:0/594787849 >> [v2:172.21.15.149:6836/914546237,v1:172.21.15.149:6837/914546237] conn(0x556cd6221d00 0x556cd6220f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:38.278 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.284+0000 7ff85eb22740 4 client.15111 mds_command: new command op to 34445 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-08T20:59:38.278 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.284+0000 7ff85eb22740 1 -- 172.21.15.107:0/594787849 --> [v2:172.21.15.149:6836/914546237,v1:172.21.15.149:6837/914546237] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556cd617dc30 con 0x556cd6221d00 2024-03-08T20:59:38.278 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.285+0000 7ff859800640 1 --2- 172.21.15.107:0/594787849 >> [v2:172.21.15.149:6836/914546237,v1:172.21.15.149:6837/914546237] conn(0x556cd6221d00 0x556cd6220f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:59:38.278 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.286+0000 7ff859800640 1 --2- 172.21.15.107:0/594787849 >> [v2:172.21.15.149:6836/914546237,v1:172.21.15.149:6837/914546237] conn(0x556cd6221d00 0x556cd6220f10 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=de8d8041bd8c4dda server_cookie=57f67afc0e2fe1ab in_seq=0 out_seq=0 2024-03-08T20:59:38.278 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.286+0000 7ff83b400640 10 client.15111 ms_handle_connect on v2:172.21.15.149:6836/914546237 2024-03-08T20:59:38.278 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.288+0000 7ff83b400640 1 -- 172.21.15.107:0/594787849 <== mds.1 v2:172.21.15.149:6836/914546237 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x556cd617dc30 con 0x556cd6221d00 2024-03-08T20:59:38.278 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.288+0000 7ff83b400640 10 client.15111 handle_command_reply: tid=0 2024-03-08T20:59:38.278 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.288+0000 7ff85eb22740 1 client.15111 shutdown 2024-03-08T20:59:38.278 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.288+0000 7ff85eb22740 1 -- 172.21.15.107:0/594787849 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556cd620bf80 msgr2=0x556cd62159e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:59:38.279 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.288+0000 7ff85eb22740 1 --2- 172.21.15.107:0/594787849 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556cd620bf80 0x556cd62159e0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7ff84002c220 tx=0x7ff840003f90 comp rx=0 tx=0).stop 2024-03-08T20:59:38.279 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.288+0000 7ff85eb22740 1 -- 172.21.15.107:0/594787849 shutdown_connections 2024-03-08T20:59:38.279 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.288+0000 7ff85eb22740 1 --2- 172.21.15.107:0/594787849 >> [v2:172.21.15.149:6836/914546237,v1:172.21.15.149:6837/914546237] conn(0x556cd6221d00 0x556cd6220f10 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:38.279 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.288+0000 7ff85eb22740 1 --2- 172.21.15.107:0/594787849 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556cd62110f0 0x556cd621ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:38.279 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.288+0000 7ff85eb22740 1 --2- 172.21.15.107:0/594787849 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556cd620bf80 0x556cd62159e0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:38.279 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.288+0000 7ff85eb22740 1 --2- 172.21.15.107:0/594787849 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556cd620b590 0x556cd62154a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:38.279 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.288+0000 7ff83b400640 0 client.15111 ms_handle_reset on v2:172.21.15.149:6836/914546237 2024-03-08T20:59:38.279 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.288+0000 7ff85eb22740 1 -- 172.21.15.107:0/594787849 >> 172.21.15.107:0/594787849 conn(0x556cd6206b00 msgr2=0x556cd6205c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:59:38.279 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.288+0000 7ff85eb22740 1 -- 172.21.15.107:0/594787849 shutdown_connections 2024-03-08T20:59:38.279 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.288+0000 7ff85eb22740 1 -- 172.21.15.107:0/594787849 wait complete. 2024-03-08T20:59:38.279 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.288+0000 7ff85eb22740 20 client.15111 trim_cache size 0 max 16384 2024-03-08T20:59:38.289 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.305+0000 7ff85eb22740 1 Processor -- start 2024-03-08T20:59:38.289 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.305+0000 7ff85eb22740 1 -- start start 2024-03-08T20:59:38.289 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.305+0000 7ff85eb22740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556cd620b590 0x556cd6205190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:38.290 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.306+0000 7ff85eb22740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556cd620bf80 0x556cd621c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:38.290 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.306+0000 7ff85eb22740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556cd62110f0 0x556cd62056d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:38.290 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.306+0000 7ff85eb22740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x556cd6221520 con 0x556cd620bf80 2024-03-08T20:59:38.290 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.306+0000 7ff85eb22740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x556cd6221b80 con 0x556cd620b590 2024-03-08T20:59:38.290 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.306+0000 7ff85eb22740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x556cd62221e0 con 0x556cd62110f0 2024-03-08T20:59:38.290 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.306+0000 7ff858e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556cd620bf80 0x556cd621c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:59:38.290 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.306+0000 7ff858e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556cd620bf80 0x556cd621c9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:57134/0 (socket says 172.21.15.107:57134) 2024-03-08T20:59:38.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.306+0000 7ff858e00640 1 -- 172.21.15.107:0/3180182495 learned_addr learned my addr 172.21.15.107:0/3180182495 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T20:59:38.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.306+0000 7ff859800640 1 --2- 172.21.15.107:0/3180182495 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556cd620b590 0x556cd6205190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:59:38.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.306+0000 7ff85a200640 1 --2- 172.21.15.107:0/3180182495 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556cd62110f0 0x556cd62056d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:59:38.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.306+0000 7ff858e00640 1 -- 172.21.15.107:0/3180182495 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556cd62110f0 msgr2=0x556cd62056d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T20:59:38.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.306+0000 7ff858e00640 1 --2- 172.21.15.107:0/3180182495 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556cd62110f0 0x556cd62056d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:38.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.306+0000 7ff858e00640 1 -- 172.21.15.107:0/3180182495 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556cd620b590 msgr2=0x556cd6205190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T20:59:38.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.306+0000 7ff858e00640 1 --2- 172.21.15.107:0/3180182495 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556cd620b590 0x556cd6205190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:38.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.306+0000 7ff858e00640 1 -- 172.21.15.107:0/3180182495 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556cd6221280 con 0x556cd620bf80 2024-03-08T20:59:38.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.307+0000 7ff858e00640 1 --2- 172.21.15.107:0/3180182495 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556cd620bf80 0x556cd621c9e0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7ff8400b9de0 tx=0x7ff8400045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d79bf3eaa0e8eeef server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:59:38.292 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.307+0000 7ff846000640 1 -- 172.21.15.107:0/3180182495 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8400bd4c0 con 0x556cd620bf80 2024-03-08T20:59:38.292 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.307+0000 7ff846000640 1 -- 172.21.15.107:0/3180182495 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7ff840004c50 con 0x556cd620bf80 2024-03-08T20:59:38.292 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.307+0000 7ff846000640 1 -- 172.21.15.107:0/3180182495 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8400bd2e0 con 0x556cd620bf80 2024-03-08T20:59:38.298 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.308+0000 7ff85eb22740 1 -- 172.21.15.107:0/3180182495 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556cd620bf80 msgr2=0x556cd621c9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:59:38.298 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.308+0000 7ff85eb22740 1 --2- 172.21.15.107:0/3180182495 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556cd620bf80 0x556cd621c9e0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7ff8400b9de0 tx=0x7ff8400045e0 comp rx=0 tx=0).stop 2024-03-08T20:59:38.298 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.308+0000 7ff85eb22740 1 -- 172.21.15.107:0/3180182495 shutdown_connections 2024-03-08T20:59:38.298 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.308+0000 7ff85eb22740 1 --2- 172.21.15.107:0/3180182495 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556cd62110f0 0x556cd62056d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:38.298 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.308+0000 7ff85eb22740 1 --2- 172.21.15.107:0/3180182495 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556cd620bf80 0x556cd621c9e0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:38.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.308+0000 7ff85eb22740 1 --2- 172.21.15.107:0/3180182495 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556cd620b590 0x556cd6205190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:38.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.308+0000 7ff85eb22740 1 -- 172.21.15.107:0/3180182495 >> 172.21.15.107:0/3180182495 conn(0x556cd6206b00 msgr2=0x556cd6206ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:59:38.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.308+0000 7ff85eb22740 1 -- 172.21.15.107:0/3180182495 shutdown_connections 2024-03-08T20:59:38.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.308+0000 7ff85eb22740 1 -- 172.21.15.107:0/3180182495 wait complete. 2024-03-08T20:59:38.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.308+0000 7ff85eb22740 1 -- start start 2024-03-08T20:59:38.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.308+0000 7ff85eb22740 1 Processor -- start 2024-03-08T20:59:38.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.309+0000 7ff85eb22740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556cd620b590 0x556cd621ca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:38.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.309+0000 7ff85eb22740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556cd620bf80 0x556cd62195d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:38.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.309+0000 7ff85eb22740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556cd62110f0 0x556cd6219c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:38.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.309+0000 7ff85eb22740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x556cd6222ea0 con 0x556cd620bf80 2024-03-08T20:59:38.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.309+0000 7ff85eb22740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x556cd6222840 con 0x556cd62110f0 2024-03-08T20:59:38.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.309+0000 7ff85eb22740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x556cd6223500 con 0x556cd620b590 2024-03-08T20:59:38.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.309+0000 7ff85eb22740 10 client.0 _ll_register_callbacks cb 0x7ff8400f4810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T20:59:38.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.309+0000 7ff858e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556cd620bf80 0x556cd62195d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:59:38.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.309+0000 7ff858e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556cd620bf80 0x556cd62195d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:57150/0 (socket says 172.21.15.107:57150) 2024-03-08T20:59:38.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.309+0000 7ff858e00640 1 -- 172.21.15.107:0/1374367266 learned_addr learned my addr 172.21.15.107:0/1374367266 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T20:59:38.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.309+0000 7ff859800640 1 --2- 172.21.15.107:0/1374367266 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556cd620b590 0x556cd621ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:59:38.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.309+0000 7ff85a200640 1 --2- 172.21.15.107:0/1374367266 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556cd62110f0 0x556cd6219c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:59:38.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.309+0000 7ff858e00640 1 -- 172.21.15.107:0/1374367266 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556cd620b590 msgr2=0x556cd621ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T20:59:38.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.309+0000 7ff858e00640 1 --2- 172.21.15.107:0/1374367266 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556cd620b590 0x556cd621ca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:38.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.309+0000 7ff858e00640 1 -- 172.21.15.107:0/1374367266 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556cd62110f0 msgr2=0x556cd6219c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T20:59:38.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.309+0000 7ff858e00640 1 --2- 172.21.15.107:0/1374367266 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556cd62110f0 0x556cd6219c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:38.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.309+0000 7ff858e00640 1 -- 172.21.15.107:0/1374367266 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556cd62779b0 con 0x556cd620bf80 2024-03-08T20:59:38.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.309+0000 7ff85a200640 1 --2- 172.21.15.107:0/1374367266 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556cd62110f0 0x556cd6219c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T20:59:38.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.310+0000 7ff858e00640 1 --2- 172.21.15.107:0/1374367266 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556cd620bf80 0x556cd62195d0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7ff8400bd680 tx=0x7ff8400c1ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64eb88db331f5011 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:59:38.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.310+0000 7ff83b400640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-03-08T20:59:38.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.310+0000 7ff83b400640 1 -- 172.21.15.107:0/1374367266 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8400b9c20 con 0x556cd620bf80 2024-03-08T20:59:38.301 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.310+0000 7ff83b400640 1 -- 172.21.15.107:0/1374367266 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7ff8400c5e90 con 0x556cd620bf80 2024-03-08T20:59:38.301 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.310+0000 7ff85eb22740 1 -- 172.21.15.107:0/1374367266 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff840004c50 con 0x556cd620bf80 2024-03-08T20:59:38.301 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.310+0000 7ff83b400640 1 -- 172.21.15.107:0/1374367266 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff8400bd2e0 con 0x556cd620bf80 2024-03-08T20:59:38.301 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.310+0000 7ff85eb22740 10 client.15117 fetch_fsmap learned FSMap version 27 2024-03-08T20:59:38.301 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.310+0000 7ff85eb22740 1 -- 172.21.15.107:0/1374367266 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x556cd62779b0 con 0x556cd620bf80 2024-03-08T20:59:38.301 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.310+0000 7ff83b400640 1 -- 172.21.15.107:0/1374367266 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7ff840004c50 con 0x556cd620bf80 2024-03-08T20:59:38.301 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.311+0000 7ff83b400640 1 -- 172.21.15.107:0/1374367266 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7ff84009c360 con 0x556cd620bf80 2024-03-08T20:59:38.301 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.311+0000 7ff85eb22740 10 client.15117 fetch_fsmap finished waiting for FSMap version 27 2024-03-08T20:59:38.301 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.311+0000 7ff85eb22740 10 client.15117 resolve_mds: resolved name 'h' to daemon mds.h 2024-03-08T20:59:38.301 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.311+0000 7ff85eb22740 20 client.15117 populate_metadata read hostname 'smithi107' 2024-03-08T20:59:38.301 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.311+0000 7ff85eb22740 1 --2- 172.21.15.107:0/1374367266 >> [v2:172.21.15.149:6836/914546237,v1:172.21.15.149:6837/914546237] conn(0x556cd627cd20 0x556cd6278cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:38.301 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.311+0000 7ff85eb22740 4 client.15117 mds_command: new command op to 34445 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-08T20:59:38.301 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.311+0000 7ff85eb22740 1 -- 172.21.15.107:0/1374367266 --> [v2:172.21.15.149:6836/914546237,v1:172.21.15.149:6837/914546237] -- command(tid 0: {"prefix": "status"}) v1 -- 0x556cd6177f80 con 0x556cd627cd20 2024-03-08T20:59:38.302 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.311+0000 7ff859800640 1 --2- 172.21.15.107:0/1374367266 >> [v2:172.21.15.149:6836/914546237,v1:172.21.15.149:6837/914546237] conn(0x556cd627cd20 0x556cd6278cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:59:38.302 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.312+0000 7ff859800640 1 --2- 172.21.15.107:0/1374367266 >> [v2:172.21.15.149:6836/914546237,v1:172.21.15.149:6837/914546237] conn(0x556cd627cd20 0x556cd6278cb0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=142d8c8e2417a165 server_cookie=9899feb469dc65c0 in_seq=0 out_seq=0 2024-03-08T20:59:38.302 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.312+0000 7ff83b400640 10 client.15117 ms_handle_connect on v2:172.21.15.149:6836/914546237 2024-03-08T20:59:38.302 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.313+0000 7ff83b400640 1 -- 172.21.15.107:0/1374367266 <== mds.1 v2:172.21.15.149:6836/914546237 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+321 (crc 0 0 0) 0x556cd6177f80 con 0x556cd627cd20 2024-03-08T20:59:38.302 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.313+0000 7ff83b400640 10 client.15117 handle_command_reply: tid=0 2024-03-08T20:59:38.302 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.313+0000 7ff85eb22740 1 client.15117 shutdown 2024-03-08T20:59:38.302 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.313+0000 7ff85eb22740 1 -- 172.21.15.107:0/1374367266 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556cd620bf80 msgr2=0x556cd62195d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:59:38.302 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.313+0000 7ff85eb22740 1 --2- 172.21.15.107:0/1374367266 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556cd620bf80 0x556cd62195d0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7ff8400bd680 tx=0x7ff8400c1ad0 comp rx=0 tx=0).stop 2024-03-08T20:59:38.302 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.313+0000 7ff85eb22740 1 -- 172.21.15.107:0/1374367266 shutdown_connections 2024-03-08T20:59:38.302 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.313+0000 7ff85eb22740 1 --2- 172.21.15.107:0/1374367266 >> [v2:172.21.15.149:6836/914546237,v1:172.21.15.149:6837/914546237] conn(0x556cd627cd20 0x556cd6278cb0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:38.302 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.313+0000 7ff85eb22740 1 --2- 172.21.15.107:0/1374367266 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556cd62110f0 0x556cd6219c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:38.302 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.313+0000 7ff85eb22740 1 --2- 172.21.15.107:0/1374367266 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556cd620bf80 0x556cd62195d0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:38.302 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.313+0000 7ff85eb22740 1 --2- 172.21.15.107:0/1374367266 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556cd620b590 0x556cd621ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:38.303 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.313+0000 7ff85eb22740 1 -- 172.21.15.107:0/1374367266 >> 172.21.15.107:0/1374367266 conn(0x556cd6206b00 msgr2=0x556cd6209d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:59:38.303 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.313+0000 7ff83b400640 0 client.15117 ms_handle_reset on v2:172.21.15.149:6836/914546237 2024-03-08T20:59:38.303 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.313+0000 7ff85eb22740 1 -- 172.21.15.107:0/1374367266 shutdown_connections 2024-03-08T20:59:38.303 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.313+0000 7ff85eb22740 1 -- 172.21.15.107:0/1374367266 wait complete. 2024-03-08T20:59:38.303 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.314+0000 7ff85eb22740 20 client.15117 trim_cache size 0 max 16384 2024-03-08T20:59:38.303 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-03-08T20:59:38.303 INFO:teuthology.orchestra.run.smithi107.stdout: "cluster_fsid": "dc20f0f8-dd8c-11ee-95c6-87774f69a715", 2024-03-08T20:59:38.303 INFO:teuthology.orchestra.run.smithi107.stdout: "whoami": 1, 2024-03-08T20:59:38.303 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 34445, 2024-03-08T20:59:38.303 INFO:teuthology.orchestra.run.smithi107.stdout: "want_state": "up:active", 2024-03-08T20:59:38.304 INFO:teuthology.orchestra.run.smithi107.stdout: "state": "up:active", 2024-03-08T20:59:38.304 INFO:teuthology.orchestra.run.smithi107.stdout: "fs_name": "cephfs", 2024-03-08T20:59:38.304 INFO:teuthology.orchestra.run.smithi107.stdout: "rank_uptime": 104.49492913, 2024-03-08T20:59:38.304 INFO:teuthology.orchestra.run.smithi107.stdout: "mdsmap_epoch": 27, 2024-03-08T20:59:38.304 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_epoch": 88, 2024-03-08T20:59:38.304 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_epoch_barrier": 86, 2024-03-08T20:59:38.304 INFO:teuthology.orchestra.run.smithi107.stdout: "uptime": 130.028290271 2024-03-08T20:59:38.304 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-03-08T20:59:38.305 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.314+0000 7ff85c200640 1 -- 172.21.15.107:0/4237187124 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7ff83c0a2880 msgr2=0x7ff83c0a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:59:38.305 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.314+0000 7ff85c200640 1 --2- 172.21.15.107:0/4237187124 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7ff83c0a2880 0x7ff83c0a4d40 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x556cd620c7e0 tx=0x7ff84c008040 comp rx=0 tx=0).stop 2024-03-08T20:59:38.305 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.314+0000 7ff85c200640 1 -- 172.21.15.107:0/4237187124 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff854104720 msgr2=0x7ff854199ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:59:38.305 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.314+0000 7ff85c200640 1 --2- 172.21.15.107:0/4237187124 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff854104720 0x7ff854199ce0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7ff840000c00 tx=0x7ff8400042f0 comp rx=0 tx=0).stop 2024-03-08T20:59:38.305 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.314+0000 7ff85c200640 1 -- 172.21.15.107:0/4237187124 shutdown_connections 2024-03-08T20:59:38.305 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.314+0000 7ff85c200640 1 --2- 172.21.15.107:0/4237187124 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7ff83c0a2880 0x7ff83c0a4d40 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:38.305 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.314+0000 7ff85c200640 1 --2- 172.21.15.107:0/4237187124 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff8541050e0 0x7ff8541a0d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:38.305 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.314+0000 7ff85c200640 1 --2- 172.21.15.107:0/4237187124 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff854104720 0x7ff854199ce0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:38.305 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.314+0000 7ff85c200640 1 --2- 172.21.15.107:0/4237187124 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff854103530 0x7ff8541997a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:38.305 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.314+0000 7ff85c200640 1 -- 172.21.15.107:0/4237187124 >> 172.21.15.107:0/4237187124 conn(0x7ff8540feaa0 msgr2=0x7ff854107b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:59:38.305 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.314+0000 7ff85c200640 1 -- 172.21.15.107:0/4237187124 shutdown_connections 2024-03-08T20:59:38.306 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.314+0000 7ff85c200640 1 -- 172.21.15.107:0/4237187124 wait complete. 2024-03-08T20:59:38.314 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-03-08T20:59:38.509 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.525+0000 7f2517000640 1 -- 172.21.15.107:0/3068913348 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2510107910 msgr2=0x7f2510107d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:59:38.509 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.525+0000 7f2517000640 1 --2- 172.21.15.107:0/3068913348 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2510107910 0x7f2510107d30 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f2500009900 tx=0x7f250002bc30 comp rx=0 tx=0).stop 2024-03-08T20:59:38.509 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.526+0000 7f2517000640 1 -- 172.21.15.107:0/3068913348 shutdown_connections 2024-03-08T20:59:38.509 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.526+0000 7f2517000640 1 --2- 172.21.15.107:0/3068913348 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f25101094c0 0x7f251010fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:38.510 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.526+0000 7f2517000640 1 --2- 172.21.15.107:0/3068913348 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2510108b00 0x7f2510108f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:38.510 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.526+0000 7f2517000640 1 --2- 172.21.15.107:0/3068913348 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2510107910 0x7f2510107d30 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:38.510 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.526+0000 7f2517000640 1 -- 172.21.15.107:0/3068913348 >> 172.21.15.107:0/3068913348 conn(0x7f25100768b0 msgr2=0x7f2510078cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:59:38.510 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.526+0000 7f2517000640 1 -- 172.21.15.107:0/3068913348 shutdown_connections 2024-03-08T20:59:38.510 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.526+0000 7f2517000640 1 -- 172.21.15.107:0/3068913348 wait complete. 2024-03-08T20:59:38.513 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.527+0000 7f2517000640 1 Processor -- start 2024-03-08T20:59:38.513 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.527+0000 7f2517000640 1 -- start start 2024-03-08T20:59:38.513 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.527+0000 7f2517000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2510107910 0x7f251019db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:38.513 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.527+0000 7f2517000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2510108b00 0x7f251019e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:38.513 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.527+0000 7f2517000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f25101094c0 0x7f25101a5120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:38.514 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.528+0000 7f2517000640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2510113460 con 0x7f2510107910 2024-03-08T20:59:38.514 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.528+0000 7f2517000640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25101132e0 con 0x7f2510108b00 2024-03-08T20:59:38.514 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.528+0000 7f2517000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25101135e0 con 0x7f25101094c0 2024-03-08T20:59:38.514 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.528+0000 7f250fe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2510107910 0x7f251019db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:59:38.514 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.528+0000 7f250fe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2510107910 0x7f251019db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:57174/0 (socket says 172.21.15.107:57174) 2024-03-08T20:59:38.514 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.528+0000 7f250fe00640 1 -- 172.21.15.107:0/1177089292 learned_addr learned my addr 172.21.15.107:0/1177089292 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T20:59:38.514 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.528+0000 7f2507400640 1 --2- 172.21.15.107:0/1177089292 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2510108b00 0x7f251019e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:59:38.514 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.528+0000 7f250fe00640 1 -- 172.21.15.107:0/1177089292 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f25101094c0 msgr2=0x7f25101a5120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:59:38.514 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.528+0000 7f250fe00640 1 --2- 172.21.15.107:0/1177089292 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f25101094c0 0x7f25101a5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:38.514 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.528+0000 7f250fe00640 1 -- 172.21.15.107:0/1177089292 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2510108b00 msgr2=0x7f251019e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:59:38.514 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.528+0000 7f250fe00640 1 --2- 172.21.15.107:0/1177089292 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2510108b00 0x7f251019e0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:38.514 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.528+0000 7f250fe00640 1 -- 172.21.15.107:0/1177089292 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25101a5790 con 0x7f2510107910 2024-03-08T20:59:38.515 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.528+0000 7f2507400640 1 --2- 172.21.15.107:0/1177089292 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2510108b00 0x7f251019e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T20:59:38.515 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.528+0000 7f250fe00640 1 --2- 172.21.15.107:0/1177089292 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2510107910 0x7f251019db70 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f2500005ab0 tx=0x7f25000042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:59:38.515 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.529+0000 7f250d600640 1 -- 172.21.15.107:0/1177089292 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2500043020 con 0x7f2510107910 2024-03-08T20:59:38.515 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.529+0000 7f250d600640 1 -- 172.21.15.107:0/1177089292 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f25000054c0 con 0x7f2510107910 2024-03-08T20:59:38.515 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.529+0000 7f2517000640 1 -- 172.21.15.107:0/1177089292 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25101a5a20 con 0x7f2510107910 2024-03-08T20:59:38.515 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.529+0000 7f250d600640 1 -- 172.21.15.107:0/1177089292 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2500003d10 con 0x7f2510107910 2024-03-08T20:59:38.515 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.529+0000 7f2517000640 1 -- 172.21.15.107:0/1177089292 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25101a5f70 con 0x7f2510107910 2024-03-08T20:59:38.518 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.532+0000 7f250d600640 1 -- 172.21.15.107:0/1177089292 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f2500046020 con 0x7f2510107910 2024-03-08T20:59:38.518 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.532+0000 7f2519c64740 1 Processor -- start 2024-03-08T20:59:38.518 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.532+0000 7f2519c64740 1 -- start start 2024-03-08T20:59:38.518 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.533+0000 7f2519c64740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557bf1608590 0x557bf16089b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:38.518 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.533+0000 7f2519c64740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557bf1608f80 0x557bf160da80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:38.519 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.533+0000 7f2519c64740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557bf160e0f0 0x557bf1610580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:38.519 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.533+0000 7f2519c64740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x557bf156d6f0 con 0x557bf160e0f0 2024-03-08T20:59:38.519 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.533+0000 7f2519c64740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x557bf15169e0 con 0x557bf1608f80 2024-03-08T20:59:38.519 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.533+0000 7f2519c64740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x557bf1569790 con 0x557bf1608590 2024-03-08T20:59:38.519 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.533+0000 7f2507400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557bf160e0f0 0x557bf1610580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:59:38.520 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.533+0000 7f2507400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557bf160e0f0 0x557bf1610580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:57186/0 (socket says 172.21.15.107:57186) 2024-03-08T20:59:38.520 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.533+0000 7f2507400640 1 -- 172.21.15.107:0/3942014051 learned_addr learned my addr 172.21.15.107:0/3942014051 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T20:59:38.520 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.533+0000 7f2515000640 1 --2- 172.21.15.107:0/3942014051 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557bf1608590 0x557bf16089b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:59:38.520 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.533+0000 7f250fe00640 1 --2- 172.21.15.107:0/3942014051 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557bf1608f80 0x557bf160da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:59:38.520 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.534+0000 7f2507400640 1 -- 172.21.15.107:0/3942014051 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557bf1608590 msgr2=0x557bf16089b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T20:59:38.520 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.534+0000 7f2507400640 1 --2- 172.21.15.107:0/3942014051 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557bf1608590 0x557bf16089b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:38.520 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.534+0000 7f2507400640 1 -- 172.21.15.107:0/3942014051 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557bf1608f80 msgr2=0x557bf160da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T20:59:38.520 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.534+0000 7f2507400640 1 --2- 172.21.15.107:0/3942014051 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557bf1608f80 0x557bf160da80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:38.520 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.534+0000 7f2507400640 1 -- 172.21.15.107:0/3942014051 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557bf150a720 con 0x557bf160e0f0 2024-03-08T20:59:38.520 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.534+0000 7f2507400640 1 --2- 172.21.15.107:0/3942014051 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557bf160e0f0 0x557bf1610580 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f24f8009ca0 tx=0x7f24f800e6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6889a94ca9a088f server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:59:38.521 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.534+0000 7f250d600640 1 --2- 172.21.15.107:0/1177089292 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f24f00a2800 0x7f24f00a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:38.521 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.534+0000 7f2505600640 1 -- 172.21.15.107:0/3942014051 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24f8016070 con 0x557bf160e0f0 2024-03-08T20:59:38.521 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.534+0000 7f2505600640 1 -- 172.21.15.107:0/3942014051 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f24f80054d0 con 0x557bf160e0f0 2024-03-08T20:59:38.521 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.534+0000 7f2505600640 1 -- 172.21.15.107:0/3942014051 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24f8011470 con 0x557bf160e0f0 2024-03-08T20:59:38.521 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.534+0000 7f250d600640 1 -- 172.21.15.107:0/1177089292 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f25000f0f40 con 0x7f2510107910 2024-03-08T20:59:38.521 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.534+0000 7f2515000640 1 --2- 172.21.15.107:0/1177089292 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f24f00a2800 0x7f24f00a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:59:38.521 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.535+0000 7f2519c64740 1 -- 172.21.15.107:0/3942014051 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557bf160e0f0 msgr2=0x557bf1610580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:59:38.521 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.535+0000 7f2515000640 1 --2- 172.21.15.107:0/1177089292 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f24f00a2800 0x7f24f00a4cc0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f2508005c40 tx=0x7f2508009480 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:59:38.521 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.535+0000 7f2519c64740 1 --2- 172.21.15.107:0/3942014051 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557bf160e0f0 0x557bf1610580 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f24f8009ca0 tx=0x7f24f800e6c0 comp rx=0 tx=0).stop 2024-03-08T20:59:38.521 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.535+0000 7f2519c64740 1 -- 172.21.15.107:0/3942014051 shutdown_connections 2024-03-08T20:59:38.522 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.535+0000 7f2519c64740 1 --2- 172.21.15.107:0/3942014051 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557bf160e0f0 0x557bf1610580 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:38.522 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.535+0000 7f2519c64740 1 --2- 172.21.15.107:0/3942014051 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557bf1608f80 0x557bf160da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:38.522 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.535+0000 7f2519c64740 1 --2- 172.21.15.107:0/3942014051 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557bf1608590 0x557bf16089b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:38.522 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.535+0000 7f2519c64740 1 -- 172.21.15.107:0/3942014051 >> 172.21.15.107:0/3942014051 conn(0x557bf1603b00 msgr2=0x557bf1603f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:59:38.522 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.535+0000 7f2519c64740 1 -- 172.21.15.107:0/3942014051 shutdown_connections 2024-03-08T20:59:38.522 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.535+0000 7f2519c64740 1 -- 172.21.15.107:0/3942014051 wait complete. 2024-03-08T20:59:38.527 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.536+0000 7f2519c64740 1 -- start start 2024-03-08T20:59:38.527 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.536+0000 7f2519c64740 1 Processor -- start 2024-03-08T20:59:38.527 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.536+0000 7f2519c64740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557bf1608590 0x557bf1612770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:38.527 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.537+0000 7f2519c64740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557bf1608f80 0x557bf1612cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:38.527 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.537+0000 7f2519c64740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557bf160e0f0 0x557bf1619cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:38.527 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.537+0000 7f2519c64740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x557bf16110a0 con 0x557bf1608590 2024-03-08T20:59:38.527 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.537+0000 7f2519c64740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x557bf1529e20 con 0x557bf160e0f0 2024-03-08T20:59:38.527 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.537+0000 7f2519c64740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x557bf1611220 con 0x557bf1608f80 2024-03-08T20:59:38.527 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.537+0000 7f2519c64740 10 client.0 _ll_register_callbacks cb 0x557bf15648d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T20:59:38.527 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.537+0000 7f250fe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557bf1608590 0x557bf1612770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:59:38.528 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.537+0000 7f250fe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557bf1608590 0x557bf1612770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:57194/0 (socket says 172.21.15.107:57194) 2024-03-08T20:59:38.528 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.537+0000 7f250fe00640 1 -- 172.21.15.107:0/77179013 learned_addr learned my addr 172.21.15.107:0/77179013 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T20:59:38.528 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.537+0000 7f2507400640 1 --2- 172.21.15.107:0/77179013 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557bf1608f80 0x557bf1612cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:59:38.528 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.537+0000 7f250fe00640 1 -- 172.21.15.107:0/77179013 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557bf1608f80 msgr2=0x557bf1612cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T20:59:38.528 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.537+0000 7f250fe00640 1 --2- 172.21.15.107:0/77179013 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557bf1608f80 0x557bf1612cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:38.528 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.537+0000 7f250fe00640 1 -- 172.21.15.107:0/77179013 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557bf160e0f0 msgr2=0x557bf1619cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T20:59:38.528 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.537+0000 7f250fe00640 1 --2- 172.21.15.107:0/77179013 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557bf160e0f0 0x557bf1619cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:38.528 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.537+0000 7f250fe00640 1 -- 172.21.15.107:0/77179013 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557bf12822d0 con 0x557bf1608590 2024-03-08T20:59:38.528 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.538+0000 7f250fe00640 1 --2- 172.21.15.107:0/77179013 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557bf1608590 0x557bf1612770 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f25000eea40 tx=0x7f2500003eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69b82536ca4759f0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:59:38.528 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.538+0000 7f24fe000640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-03-08T20:59:38.528 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.538+0000 7f24fe000640 1 -- 172.21.15.107:0/77179013 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25000d6060 con 0x557bf1608590 2024-03-08T20:59:38.528 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.538+0000 7f24fe000640 1 -- 172.21.15.107:0/77179013 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f25000035f0 con 0x557bf1608590 2024-03-08T20:59:38.529 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.538+0000 7f2519c64740 1 -- 172.21.15.107:0/77179013 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f24f80054d0 con 0x557bf1608590 2024-03-08T20:59:38.529 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.538+0000 7f24fe000640 1 -- 172.21.15.107:0/77179013 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f25000bd540 con 0x557bf1608590 2024-03-08T20:59:38.529 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.538+0000 7f2519c64740 10 client.15129 fetch_fsmap learned FSMap version 27 2024-03-08T20:59:38.529 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.538+0000 7f2519c64740 1 -- 172.21.15.107:0/77179013 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x557bf134c6b0 con 0x557bf1608590 2024-03-08T20:59:38.529 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.538+0000 7f24fe000640 1 -- 172.21.15.107:0/77179013 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f24f80054d0 con 0x557bf1608590 2024-03-08T20:59:38.529 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.538+0000 7f24fe000640 1 -- 172.21.15.107:0/77179013 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f25000c5d10 con 0x557bf1608590 2024-03-08T20:59:38.529 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.539+0000 7f2519c64740 10 client.15129 fetch_fsmap finished waiting for FSMap version 27 2024-03-08T20:59:38.529 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.539+0000 7f2519c64740 10 client.15129 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-08T20:59:38.529 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.539+0000 7f2519c64740 20 client.15129 populate_metadata read hostname 'smithi107' 2024-03-08T20:59:38.529 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.539+0000 7f2519c64740 1 --2- 172.21.15.107:0/77179013 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x557bf161f350 0x557bf161e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:38.529 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.539+0000 7f2519c64740 4 client.15129 mds_command: new command op to 34475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-08T20:59:38.530 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.539+0000 7f2519c64740 1 -- 172.21.15.107:0/77179013 --> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557bf157ac30 con 0x557bf161f350 2024-03-08T20:59:38.530 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.539+0000 7f2507400640 1 --2- 172.21.15.107:0/77179013 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x557bf161f350 0x557bf161e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:59:38.530 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.540+0000 7f2507400640 1 --2- 172.21.15.107:0/77179013 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x557bf161f350 0x557bf161e1e0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b68057566dd74b3 server_cookie=76a21a9e3e7c8f40 in_seq=0 out_seq=0 2024-03-08T20:59:38.530 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.540+0000 7f24fe000640 10 client.15129 ms_handle_connect on v2:172.21.15.149:6838/55290596 2024-03-08T20:59:38.530 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.542+0000 7f24fe000640 1 -- 172.21.15.107:0/77179013 <== mds.0 v2:172.21.15.149:6838/55290596 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x557bf157ac30 con 0x557bf161f350 2024-03-08T20:59:38.530 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.542+0000 7f24fe000640 10 client.15129 handle_command_reply: tid=0 2024-03-08T20:59:38.530 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.542+0000 7f2519c64740 1 client.15129 shutdown 2024-03-08T20:59:38.530 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.543+0000 7f2519c64740 1 -- 172.21.15.107:0/77179013 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557bf1608590 msgr2=0x557bf1612770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:59:38.530 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.543+0000 7f2519c64740 1 --2- 172.21.15.107:0/77179013 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557bf1608590 0x557bf1612770 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f25000eea40 tx=0x7f2500003eb0 comp rx=0 tx=0).stop 2024-03-08T20:59:38.531 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.543+0000 7f2519c64740 1 -- 172.21.15.107:0/77179013 shutdown_connections 2024-03-08T20:59:38.531 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.543+0000 7f2519c64740 1 --2- 172.21.15.107:0/77179013 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x557bf161f350 0x557bf161e1e0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:38.531 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.543+0000 7f2519c64740 1 --2- 172.21.15.107:0/77179013 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557bf160e0f0 0x557bf1619cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:38.531 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.543+0000 7f2519c64740 1 --2- 172.21.15.107:0/77179013 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557bf1608f80 0x557bf1612cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:38.531 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.543+0000 7f2519c64740 1 --2- 172.21.15.107:0/77179013 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557bf1608590 0x557bf1612770 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:38.531 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.543+0000 7f24fe000640 0 client.15129 ms_handle_reset on v2:172.21.15.149:6838/55290596 2024-03-08T20:59:38.531 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.543+0000 7f2519c64740 1 -- 172.21.15.107:0/77179013 >> 172.21.15.107:0/77179013 conn(0x557bf1603b00 msgr2=0x557bf1602c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:59:38.531 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.543+0000 7f2519c64740 1 -- 172.21.15.107:0/77179013 shutdown_connections 2024-03-08T20:59:38.531 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.543+0000 7f2519c64740 1 -- 172.21.15.107:0/77179013 wait complete. 2024-03-08T20:59:38.531 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.543+0000 7f2519c64740 20 client.15129 trim_cache size 0 max 16384 2024-03-08T20:59:38.549 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.560+0000 7f2519c64740 1 Processor -- start 2024-03-08T20:59:38.549 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.560+0000 7f2519c64740 1 -- start start 2024-03-08T20:59:38.549 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.560+0000 7f2519c64740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557bf1608590 0x557bf1612380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:38.549 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.560+0000 7f2519c64740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557bf1608f80 0x557bf1619cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:38.549 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.560+0000 7f2519c64740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557bf160e0f0 0x557bf16128c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:38.550 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.560+0000 7f2519c64740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x557bf161eb70 con 0x557bf1608f80 2024-03-08T20:59:38.551 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.560+0000 7f2519c64740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x557bf161f1d0 con 0x557bf160e0f0 2024-03-08T20:59:38.551 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.560+0000 7f2519c64740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x557bf161f830 con 0x557bf1608590 2024-03-08T20:59:38.551 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.561+0000 7f2507400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557bf1608f80 0x557bf1619cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:59:38.551 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.561+0000 7f2507400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557bf1608f80 0x557bf1619cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:57208/0 (socket says 172.21.15.107:57208) 2024-03-08T20:59:38.551 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.561+0000 7f2507400640 1 -- 172.21.15.107:0/1016509280 learned_addr learned my addr 172.21.15.107:0/1016509280 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T20:59:38.551 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.561+0000 7f2507400640 1 -- 172.21.15.107:0/1016509280 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557bf1608590 msgr2=0x557bf1612380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T20:59:38.551 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.561+0000 7f2507400640 1 --2- 172.21.15.107:0/1016509280 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557bf1608590 0x557bf1612380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:38.551 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.561+0000 7f2507400640 1 -- 172.21.15.107:0/1016509280 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557bf160e0f0 msgr2=0x557bf16128c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T20:59:38.551 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.561+0000 7f2507400640 1 --2- 172.21.15.107:0/1016509280 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557bf160e0f0 0x557bf16128c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:38.551 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.561+0000 7f2507400640 1 -- 172.21.15.107:0/1016509280 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557bf1609360 con 0x557bf1608f80 2024-03-08T20:59:38.551 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.561+0000 7f2507400640 1 --2- 172.21.15.107:0/1016509280 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557bf1608f80 0x557bf1619cb0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f24f8005940 tx=0x7f24f8005890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5052fe5fb08653 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:59:38.551 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.561+0000 7f2504c00640 1 -- 172.21.15.107:0/1016509280 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24f8016040 con 0x557bf1608f80 2024-03-08T20:59:38.552 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.562+0000 7f2504c00640 1 -- 172.21.15.107:0/1016509280 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f24f8026450 con 0x557bf1608f80 2024-03-08T20:59:38.552 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.562+0000 7f2504c00640 1 -- 172.21.15.107:0/1016509280 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24f8005260 con 0x557bf1608f80 2024-03-08T20:59:38.552 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.562+0000 7f2519c64740 1 -- 172.21.15.107:0/1016509280 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557bf1608f80 msgr2=0x557bf1619cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:59:38.552 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.562+0000 7f2519c64740 1 --2- 172.21.15.107:0/1016509280 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557bf1608f80 0x557bf1619cb0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f24f8005940 tx=0x7f24f8005890 comp rx=0 tx=0).stop 2024-03-08T20:59:38.552 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.562+0000 7f2519c64740 1 -- 172.21.15.107:0/1016509280 shutdown_connections 2024-03-08T20:59:38.552 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.562+0000 7f2519c64740 1 --2- 172.21.15.107:0/1016509280 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557bf160e0f0 0x557bf16128c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:38.552 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.562+0000 7f2519c64740 1 --2- 172.21.15.107:0/1016509280 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557bf1608f80 0x557bf1619cb0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:38.552 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.562+0000 7f2519c64740 1 --2- 172.21.15.107:0/1016509280 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557bf1608590 0x557bf1612380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:38.552 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.562+0000 7f2519c64740 1 -- 172.21.15.107:0/1016509280 >> 172.21.15.107:0/1016509280 conn(0x557bf1603b00 msgr2=0x557bf16277f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:59:38.552 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.562+0000 7f2519c64740 1 -- 172.21.15.107:0/1016509280 shutdown_connections 2024-03-08T20:59:38.552 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.562+0000 7f2519c64740 1 -- 172.21.15.107:0/1016509280 wait complete. 2024-03-08T20:59:38.552 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.563+0000 7f2519c64740 1 -- start start 2024-03-08T20:59:38.553 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.563+0000 7f2519c64740 1 Processor -- start 2024-03-08T20:59:38.553 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.563+0000 7f2519c64740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557bf1608590 0x557bf1616d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:38.553 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.563+0000 7f2519c64740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557bf1608f80 0x557bf16172d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:38.553 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.563+0000 7f2519c64740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557bf160e0f0 0x557bf1618270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:38.553 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.563+0000 7f2519c64740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x557bf16204f0 con 0x557bf1608590 2024-03-08T20:59:38.553 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.563+0000 7f2519c64740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x557bf161fe90 con 0x557bf1608f80 2024-03-08T20:59:38.553 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.563+0000 7f2519c64740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x557bf1620b50 con 0x557bf160e0f0 2024-03-08T20:59:38.553 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.563+0000 7f2519c64740 10 client.0 _ll_register_callbacks cb 0x7f24f8009460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T20:59:38.553 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.564+0000 7f250fe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557bf1608590 0x557bf1616d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:59:38.553 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.564+0000 7f250fe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557bf1608590 0x557bf1616d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:57216/0 (socket says 172.21.15.107:57216) 2024-03-08T20:59:38.553 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.564+0000 7f250fe00640 1 -- 172.21.15.107:0/3114209356 learned_addr learned my addr 172.21.15.107:0/3114209356 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T20:59:38.554 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.564+0000 7f250fe00640 1 -- 172.21.15.107:0/3114209356 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557bf160e0f0 msgr2=0x557bf1618270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T20:59:38.554 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.564+0000 7f250fe00640 1 --2- 172.21.15.107:0/3114209356 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557bf160e0f0 0x557bf1618270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:38.554 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.564+0000 7f2507400640 1 --2- 172.21.15.107:0/3114209356 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557bf1608f80 0x557bf16172d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:59:38.554 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.564+0000 7f250fe00640 1 -- 172.21.15.107:0/3114209356 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557bf1608f80 msgr2=0x557bf16172d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T20:59:38.554 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.564+0000 7f250fe00640 1 --2- 172.21.15.107:0/3114209356 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557bf1608f80 0x557bf16172d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:38.554 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.564+0000 7f250fe00640 1 -- 172.21.15.107:0/3114209356 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557bf1618860 con 0x557bf1608590 2024-03-08T20:59:38.554 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.564+0000 7f250fe00640 1 --2- 172.21.15.107:0/3114209356 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557bf1608590 0x557bf1616d90 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f25000bd990 tx=0x7f2500009560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7c5d43d9ace4018 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:59:38.554 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.564+0000 7f24fe000640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-03-08T20:59:38.555 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.564+0000 7f24fe000640 1 -- 172.21.15.107:0/3114209356 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25000044e0 con 0x557bf1608590 2024-03-08T20:59:38.555 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.564+0000 7f24fe000640 1 -- 172.21.15.107:0/3114209356 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f2500005210 con 0x557bf1608590 2024-03-08T20:59:38.555 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.564+0000 7f2519c64740 1 -- 172.21.15.107:0/3114209356 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f24f8026450 con 0x557bf1608590 2024-03-08T20:59:38.555 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.565+0000 7f24fe000640 1 -- 172.21.15.107:0/3114209356 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f25000bd360 con 0x557bf1608590 2024-03-08T20:59:38.555 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.565+0000 7f2519c64740 10 client.15135 fetch_fsmap learned FSMap version 27 2024-03-08T20:59:38.555 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.565+0000 7f2519c64740 1 -- 172.21.15.107:0/3114209356 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x557bf1619bd0 con 0x557bf1608590 2024-03-08T20:59:38.555 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.565+0000 7f24fe000640 1 -- 172.21.15.107:0/3114209356 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f24f8026450 con 0x557bf1608590 2024-03-08T20:59:38.555 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.565+0000 7f24fe000640 1 -- 172.21.15.107:0/3114209356 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f25000bce50 con 0x557bf1608590 2024-03-08T20:59:38.555 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.565+0000 7f2519c64740 10 client.15135 fetch_fsmap finished waiting for FSMap version 27 2024-03-08T20:59:38.555 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.565+0000 7f2519c64740 10 client.15135 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-08T20:59:38.555 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.565+0000 7f2519c64740 20 client.15135 populate_metadata read hostname 'smithi107' 2024-03-08T20:59:38.556 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.565+0000 7f2519c64740 1 --2- 172.21.15.107:0/3114209356 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x557bf167ae60 0x557bf1676f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:38.556 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.565+0000 7f2519c64740 4 client.15135 mds_command: new command op to 34475 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-08T20:59:38.556 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.565+0000 7f2519c64740 1 -- 172.21.15.107:0/3114209356 --> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] -- command(tid 0: {"prefix": "status"}) v1 -- 0x557bf1574f80 con 0x557bf167ae60 2024-03-08T20:59:38.556 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.566+0000 7f2507400640 1 --2- 172.21.15.107:0/3114209356 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x557bf167ae60 0x557bf1676f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:59:38.556 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.566+0000 7f2507400640 1 --2- 172.21.15.107:0/3114209356 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x557bf167ae60 0x557bf1676f60 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cbaf287d825d4e32 server_cookie=3c078e65b80e1c5c in_seq=0 out_seq=0 2024-03-08T20:59:38.556 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.566+0000 7f24fe000640 10 client.15135 ms_handle_connect on v2:172.21.15.149:6838/55290596 2024-03-08T20:59:38.557 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.567+0000 7f24fe000640 1 -- 172.21.15.107:0/3114209356 <== mds.0 v2:172.21.15.149:6838/55290596 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+322 (crc 0 0 0) 0x557bf1574f80 con 0x557bf167ae60 2024-03-08T20:59:38.557 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.567+0000 7f24fe000640 10 client.15135 handle_command_reply: tid=0 2024-03-08T20:59:38.557 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.567+0000 7f2519c64740 1 client.15135 shutdown 2024-03-08T20:59:38.557 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.567+0000 7f2519c64740 1 -- 172.21.15.107:0/3114209356 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557bf1608590 msgr2=0x557bf1616d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:59:38.558 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.567+0000 7f2519c64740 1 --2- 172.21.15.107:0/3114209356 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557bf1608590 0x557bf1616d90 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f25000bd990 tx=0x7f2500009560 comp rx=0 tx=0).stop 2024-03-08T20:59:38.558 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.567+0000 7f2519c64740 1 -- 172.21.15.107:0/3114209356 shutdown_connections 2024-03-08T20:59:38.558 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.567+0000 7f2519c64740 1 --2- 172.21.15.107:0/3114209356 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557bf160e0f0 0x557bf1618270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:38.558 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.567+0000 7f2519c64740 1 --2- 172.21.15.107:0/3114209356 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x557bf167ae60 0x557bf1676f60 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:38.558 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.567+0000 7f2519c64740 1 --2- 172.21.15.107:0/3114209356 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557bf1608f80 0x557bf16172d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:38.558 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.567+0000 7f2519c64740 1 --2- 172.21.15.107:0/3114209356 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557bf1608590 0x557bf1616d90 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:38.558 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.567+0000 7f24fe000640 0 client.15135 ms_handle_reset on v2:172.21.15.149:6838/55290596 2024-03-08T20:59:38.558 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.567+0000 7f2519c64740 1 -- 172.21.15.107:0/3114209356 >> 172.21.15.107:0/3114209356 conn(0x557bf1603b00 msgr2=0x557bf1612380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:59:38.558 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.568+0000 7f2519c64740 1 -- 172.21.15.107:0/3114209356 shutdown_connections 2024-03-08T20:59:38.558 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.568+0000 7f2519c64740 1 -- 172.21.15.107:0/3114209356 wait complete. 2024-03-08T20:59:38.558 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.568+0000 7f2519c64740 20 client.15135 trim_cache size 0 max 16384 2024-03-08T20:59:38.559 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-03-08T20:59:38.559 INFO:teuthology.orchestra.run.smithi107.stdout: "cluster_fsid": "dc20f0f8-dd8c-11ee-95c6-87774f69a715", 2024-03-08T20:59:38.559 INFO:teuthology.orchestra.run.smithi107.stdout: "whoami": 0, 2024-03-08T20:59:38.559 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 34475, 2024-03-08T20:59:38.559 INFO:teuthology.orchestra.run.smithi107.stdout: "want_state": "up:active", 2024-03-08T20:59:38.559 INFO:teuthology.orchestra.run.smithi107.stdout: "state": "up:active", 2024-03-08T20:59:38.559 INFO:teuthology.orchestra.run.smithi107.stdout: "fs_name": "cephfs", 2024-03-08T20:59:38.559 INFO:teuthology.orchestra.run.smithi107.stdout: "rank_uptime": 107.805748249, 2024-03-08T20:59:38.559 INFO:teuthology.orchestra.run.smithi107.stdout: "mdsmap_epoch": 27, 2024-03-08T20:59:38.559 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_epoch": 88, 2024-03-08T20:59:38.559 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_epoch_barrier": 86, 2024-03-08T20:59:38.559 INFO:teuthology.orchestra.run.smithi107.stdout: "uptime": 114.397180253 2024-03-08T20:59:38.560 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-03-08T20:59:38.560 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.568+0000 7f2517000640 1 -- 172.21.15.107:0/1177089292 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f24f00a2800 msgr2=0x7f24f00a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:59:38.560 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.568+0000 7f2517000640 1 --2- 172.21.15.107:0/1177089292 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f24f00a2800 0x7f24f00a4cc0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f2508005c40 tx=0x7f2508009480 comp rx=0 tx=0).stop 2024-03-08T20:59:38.561 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.568+0000 7f2517000640 1 -- 172.21.15.107:0/1177089292 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2510107910 msgr2=0x7f251019db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:59:38.561 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.568+0000 7f2517000640 1 --2- 172.21.15.107:0/1177089292 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2510107910 0x7f251019db70 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f2500005ab0 tx=0x7f25000042f0 comp rx=0 tx=0).stop 2024-03-08T20:59:38.561 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.568+0000 7f2517000640 1 -- 172.21.15.107:0/1177089292 shutdown_connections 2024-03-08T20:59:38.561 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.568+0000 7f2517000640 1 --2- 172.21.15.107:0/1177089292 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f24f00a2800 0x7f24f00a4cc0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:38.561 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.568+0000 7f2517000640 1 --2- 172.21.15.107:0/1177089292 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f25101094c0 0x7f25101a5120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:38.561 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.568+0000 7f2517000640 1 --2- 172.21.15.107:0/1177089292 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2510108b00 0x7f251019e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:38.561 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.568+0000 7f2517000640 1 --2- 172.21.15.107:0/1177089292 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2510107910 0x7f251019db70 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:38.561 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.568+0000 7f2517000640 1 -- 172.21.15.107:0/1177089292 >> 172.21.15.107:0/1177089292 conn(0x7f25100768b0 msgr2=0x7f251010bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:59:38.561 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.569+0000 7f2517000640 1 -- 172.21.15.107:0/1177089292 shutdown_connections 2024-03-08T20:59:38.561 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:38.569+0000 7f2517000640 1 -- 172.21.15.107:0/1177089292 wait complete. 2024-03-08T20:59:38.568 INFO:tasks.cephadm:Setup complete, yielding 2024-03-08T20:59:38.568 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-08T20:59:38.581 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi107.front.sepia.ceph.com 2024-03-08T20:59:38.581 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dc20f0f8-dd8c-11ee-95c6-87774f69a715 -- bash -c 'ceph orch status' 2024-03-08T20:59:38.878 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:59:38 smithi204 ceph-mon[34935]: pgmap v417: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s 2024-03-08T20:59:38.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:59:38 smithi107 ceph-mon[31381]: pgmap v417: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s 2024-03-08T20:59:39.015 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:59:38 smithi149 ceph-mon[35042]: pgmap v417: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s 2024-03-08T20:59:39.015 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/mon.a/config 2024-03-08T20:59:41.411 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:41.427+0000 7ff999000700 1 -- 172.21.15.107:0/2438880051 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff994107b60 msgr2=0x7ff994109f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:59:41.411 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:41.427+0000 7ff999000700 1 --2- 172.21.15.107:0/2438880051 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff994107b60 0x7ff994109f70 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7ff98c00b0a0 tx=0x7ff98c00b3b0 comp rx=0 tx=0).stop 2024-03-08T20:59:41.411 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:41.427+0000 7ff999000700 1 -- 172.21.15.107:0/2438880051 shutdown_connections 2024-03-08T20:59:41.411 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:41.427+0000 7ff999000700 1 --2- 172.21.15.107:0/2438880051 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff994107b60 0x7ff994109f70 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:41.411 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:41.427+0000 7ff999000700 1 --2- 172.21.15.107:0/2438880051 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff994102900 0x7ff994104cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:41.411 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:41.427+0000 7ff999000700 1 --2- 172.21.15.107:0/2438880051 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff994105230 0x7ff994107620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:41.411 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:41.427+0000 7ff999000700 1 -- 172.21.15.107:0/2438880051 >> 172.21.15.107:0/2438880051 conn(0x7ff9940fc2b0 msgr2=0x7ff9940fe6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:59:41.411 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:41.428+0000 7ff999000700 1 -- 172.21.15.107:0/2438880051 shutdown_connections 2024-03-08T20:59:41.412 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:41.428+0000 7ff999000700 1 -- 172.21.15.107:0/2438880051 wait complete. 2024-03-08T20:59:41.412 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:41.429+0000 7ff999000700 1 Processor -- start 2024-03-08T20:59:41.412 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:41.430+0000 7ff999000700 1 -- start start 2024-03-08T20:59:41.414 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:41.430+0000 7ff999000700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff994102900 0x7ff994197fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:41.414 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:41.430+0000 7ff999000700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff994105230 0x7ff9941984e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:41.414 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:41.430+0000 7ff999000700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff994107b60 0x7ff99419f560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:41.414 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:41.430+0000 7ff999000700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff99410d8f0 con 0x7ff994102900 2024-03-08T20:59:41.414 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:41.430+0000 7ff999000700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff99410d770 con 0x7ff994107b60 2024-03-08T20:59:41.414 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:41.430+0000 7ff999000700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff99410da70 con 0x7ff994105230 2024-03-08T20:59:41.414 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:41.431+0000 7ff991e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff994102900 0x7ff994197fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:59:41.415 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:41.431+0000 7ff991e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff994102900 0x7ff994197fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:57226/0 (socket says 172.21.15.107:57226) 2024-03-08T20:59:41.415 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:41.431+0000 7ff991e00700 1 -- 172.21.15.107:0/900301440 learned_addr learned my addr 172.21.15.107:0/900301440 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T20:59:41.415 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:41.431+0000 7ff992800700 1 --2- 172.21.15.107:0/900301440 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff994107b60 0x7ff99419f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:59:41.415 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:41.431+0000 7ff991400700 1 --2- 172.21.15.107:0/900301440 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff994105230 0x7ff9941984e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:59:41.415 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:41.431+0000 7ff991e00700 1 -- 172.21.15.107:0/900301440 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff994105230 msgr2=0x7ff9941984e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:59:41.415 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:41.431+0000 7ff991e00700 1 --2- 172.21.15.107:0/900301440 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff994105230 0x7ff9941984e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:41.415 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:41.431+0000 7ff991e00700 1 -- 172.21.15.107:0/900301440 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff994107b60 msgr2=0x7ff99419f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:59:41.415 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:41.431+0000 7ff991e00700 1 --2- 172.21.15.107:0/900301440 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff994107b60 0x7ff99419f560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:41.415 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:41.431+0000 7ff991e00700 1 -- 172.21.15.107:0/900301440 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff99419fc00 con 0x7ff994102900 2024-03-08T20:59:41.415 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:41.431+0000 7ff991400700 1 --2- 172.21.15.107:0/900301440 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff994105230 0x7ff9941984e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T20:59:41.415 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:41.432+0000 7ff991e00700 1 --2- 172.21.15.107:0/900301440 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff994102900 0x7ff994197fa0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7ff98400d980 tx=0x7ff98400dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:59:41.415 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:41.432+0000 7ff98aa00700 1 -- 172.21.15.107:0/900301440 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff984016070 con 0x7ff994102900 2024-03-08T20:59:41.416 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:41.432+0000 7ff98aa00700 1 -- 172.21.15.107:0/900301440 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7ff984010460 con 0x7ff994102900 2024-03-08T20:59:41.416 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:41.432+0000 7ff999000700 1 -- 172.21.15.107:0/900301440 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff99419fef0 con 0x7ff994102900 2024-03-08T20:59:41.416 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:41.432+0000 7ff98aa00700 1 -- 172.21.15.107:0/900301440 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff98400f620 con 0x7ff994102900 2024-03-08T20:59:41.416 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:41.432+0000 7ff999000700 1 -- 172.21.15.107:0/900301440 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9941a0350 con 0x7ff994102900 2024-03-08T20:59:41.416 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:41.433+0000 7ff999000700 1 -- 172.21.15.107:0/900301440 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff994068aa0 con 0x7ff994102900 2024-03-08T20:59:41.418 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:41.436+0000 7ff98aa00700 1 -- 172.21.15.107:0/900301440 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) v1 ==== 137291+0+0 (secure 0 0 0) 0x7ff984019070 con 0x7ff994102900 2024-03-08T20:59:41.420 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:41.438+0000 7ff98aa00700 1 --2- 172.21.15.107:0/900301440 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7ff9780a2910 0x7ff9780a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:41.420 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:41.438+0000 7ff98aa00700 1 -- 172.21.15.107:0/900301440 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7ff9840ce600 con 0x7ff994102900 2024-03-08T20:59:41.420 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:41.438+0000 7ff991400700 1 --2- 172.21.15.107:0/900301440 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7ff9780a2910 0x7ff9780a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:59:41.421 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:41.439+0000 7ff991400700 1 --2- 172.21.15.107:0/900301440 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7ff9780a2910 0x7ff9780a4dd0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7ff980005fd0 tx=0x7ff980005ee0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:59:41.422 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:41.440+0000 7ff98aa00700 1 -- 172.21.15.107:0/900301440 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7ff98409b660 con 0x7ff994102900 2024-03-08T20:59:41.665 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:41.683+0000 7ff999000700 1 -- 172.21.15.107:0/900301440 --> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff994062220 con 0x7ff9780a2910 2024-03-08T20:59:41.666 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:59:41 smithi107 ceph-mon[31381]: pgmap v418: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 8 op/s 2024-03-08T20:59:41.668 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:41.685+0000 7ff98aa00700 1 -- 172.21.15.107:0/900301440 <== mgr.14152 v2:172.21.15.107:6800/3376636618 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7ff994062220 con 0x7ff9780a2910 2024-03-08T20:59:41.668 INFO:teuthology.orchestra.run.smithi107.stdout:Backend: cephadm 2024-03-08T20:59:41.668 INFO:teuthology.orchestra.run.smithi107.stdout:Available: Yes 2024-03-08T20:59:41.668 INFO:teuthology.orchestra.run.smithi107.stdout:Paused: No 2024-03-08T20:59:41.670 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:41.688+0000 7ff999000700 1 -- 172.21.15.107:0/900301440 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7ff9780a2910 msgr2=0x7ff9780a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:59:41.670 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:41.688+0000 7ff999000700 1 --2- 172.21.15.107:0/900301440 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7ff9780a2910 0x7ff9780a4dd0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7ff980005fd0 tx=0x7ff980005ee0 comp rx=0 tx=0).stop 2024-03-08T20:59:41.670 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:41.688+0000 7ff999000700 1 -- 172.21.15.107:0/900301440 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff994102900 msgr2=0x7ff994197fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:59:41.671 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:41.688+0000 7ff999000700 1 --2- 172.21.15.107:0/900301440 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff994102900 0x7ff994197fa0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7ff98400d980 tx=0x7ff98400dd40 comp rx=0 tx=0).stop 2024-03-08T20:59:41.671 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:41.689+0000 7ff999000700 1 -- 172.21.15.107:0/900301440 shutdown_connections 2024-03-08T20:59:41.671 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:41.689+0000 7ff999000700 1 --2- 172.21.15.107:0/900301440 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7ff9780a2910 0x7ff9780a4dd0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:41.671 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:41.689+0000 7ff999000700 1 --2- 172.21.15.107:0/900301440 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff994107b60 0x7ff99419f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:41.671 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:41.689+0000 7ff999000700 1 --2- 172.21.15.107:0/900301440 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff994102900 0x7ff994197fa0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:41.671 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:41.689+0000 7ff999000700 1 --2- 172.21.15.107:0/900301440 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff994105230 0x7ff9941984e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:41.671 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:41.689+0000 7ff999000700 1 -- 172.21.15.107:0/900301440 >> 172.21.15.107:0/900301440 conn(0x7ff9940fc2b0 msgr2=0x7ff994103240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:59:41.672 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:41.689+0000 7ff999000700 1 -- 172.21.15.107:0/900301440 shutdown_connections 2024-03-08T20:59:41.672 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:41.689+0000 7ff999000700 1 -- 172.21.15.107:0/900301440 wait complete. 2024-03-08T20:59:41.718 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:59:41 smithi149 ceph-mon[35042]: pgmap v418: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 8 op/s 2024-03-08T20:59:41.878 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:59:41 smithi204 ceph-mon[34935]: pgmap v418: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 8 op/s 2024-03-08T20:59:42.439 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dc20f0f8-dd8c-11ee-95c6-87774f69a715 -- bash -c 'ceph orch ps' 2024-03-08T20:59:42.874 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/mon.a/config 2024-03-08T20:59:43.717 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:59:43 smithi107 ceph-mon[31381]: from='client.15141 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-08T20:59:43.717 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:59:43 smithi107 ceph-mon[31381]: pgmap v419: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 9 op/s 2024-03-08T20:59:43.765 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:59:43 smithi149 ceph-mon[35042]: from='client.15141 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-08T20:59:43.765 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:59:43 smithi149 ceph-mon[35042]: pgmap v419: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 9 op/s 2024-03-08T20:59:43.878 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:59:43 smithi204 ceph-mon[34935]: from='client.15141 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-08T20:59:43.878 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:59:43 smithi204 ceph-mon[34935]: pgmap v419: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 9 op/s 2024-03-08T20:59:45.122 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:45.139+0000 7f84a4c00700 1 -- 172.21.15.107:0/3031943913 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f84a01027c0 msgr2=0x7f84a0104bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:59:45.122 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:45.139+0000 7f84a4c00700 1 --2- 172.21.15.107:0/3031943913 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f84a01027c0 0x7f84a0104bb0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f8490009990 tx=0x7f8490009ca0 comp rx=0 tx=0).stop 2024-03-08T20:59:45.122 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:45.139+0000 7f84a4c00700 1 -- 172.21.15.107:0/3031943913 shutdown_connections 2024-03-08T20:59:45.123 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:45.139+0000 7f84a4c00700 1 --2- 172.21.15.107:0/3031943913 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f84a0107a20 0x7f84a0109e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:45.123 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:45.139+0000 7f84a4c00700 1 --2- 172.21.15.107:0/3031943913 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f84a01027c0 0x7f84a0104bb0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:45.123 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:45.139+0000 7f84a4c00700 1 --2- 172.21.15.107:0/3031943913 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f84a01050f0 0x7f84a01074e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:45.123 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:45.139+0000 7f84a4c00700 1 -- 172.21.15.107:0/3031943913 >> 172.21.15.107:0/3031943913 conn(0x7f84a00fc1b0 msgr2=0x7f84a00fe610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:59:45.123 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:45.139+0000 7f84a4c00700 1 -- 172.21.15.107:0/3031943913 shutdown_connections 2024-03-08T20:59:45.123 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:45.140+0000 7f84a4c00700 1 -- 172.21.15.107:0/3031943913 wait complete. 2024-03-08T20:59:45.124 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:45.141+0000 7f84a4c00700 1 Processor -- start 2024-03-08T20:59:45.124 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:45.141+0000 7f84a4c00700 1 -- start start 2024-03-08T20:59:45.125 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:45.141+0000 7f84a4c00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f84a01027c0 0x7f84a019a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:45.125 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:45.141+0000 7f84a4c00700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f84a01050f0 0x7f84a019a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:45.125 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:45.141+0000 7f84a4c00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f84a0107a20 0x7f84a01a1650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:45.125 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:45.141+0000 7f84a4c00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84a010d830 con 0x7f84a01027c0 2024-03-08T20:59:45.125 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:45.141+0000 7f84a4c00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84a010d6b0 con 0x7f84a0107a20 2024-03-08T20:59:45.125 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:45.141+0000 7f84a4c00700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84a010d9b0 con 0x7f84a01050f0 2024-03-08T20:59:45.125 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:45.141+0000 7f849f400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f84a01027c0 0x7f84a019a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:59:45.125 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:45.141+0000 7f849f400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f84a01027c0 0x7f84a019a090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:57242/0 (socket says 172.21.15.107:57242) 2024-03-08T20:59:45.125 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:45.142+0000 7f849f400700 1 -- 172.21.15.107:0/2560597074 learned_addr learned my addr 172.21.15.107:0/2560597074 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T20:59:45.125 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:45.142+0000 7f8497e00700 1 --2- 172.21.15.107:0/2560597074 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f84a01050f0 0x7f84a019a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:59:45.125 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:45.142+0000 7f849fe00700 1 --2- 172.21.15.107:0/2560597074 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f84a0107a20 0x7f84a01a1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:59:45.125 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:45.142+0000 7f849f400700 1 -- 172.21.15.107:0/2560597074 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f84a01050f0 msgr2=0x7f84a019a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:59:45.125 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:45.142+0000 7f849f400700 1 --2- 172.21.15.107:0/2560597074 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f84a01050f0 0x7f84a019a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:45.125 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:45.142+0000 7f849f400700 1 -- 172.21.15.107:0/2560597074 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f84a0107a20 msgr2=0x7f84a01a1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:59:45.125 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:45.142+0000 7f849f400700 1 --2- 172.21.15.107:0/2560597074 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f84a0107a20 0x7f84a01a1650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:45.126 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:45.142+0000 7f849f400700 1 -- 172.21.15.107:0/2560597074 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84a01a1cf0 con 0x7f84a01027c0 2024-03-08T20:59:45.126 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:45.142+0000 7f849f400700 1 --2- 172.21.15.107:0/2560597074 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f84a01027c0 0x7f84a019a090 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f8490006010 tx=0x7f8490003c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:59:45.126 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:45.143+0000 7f849cc00700 1 -- 172.21.15.107:0/2560597074 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8490027070 con 0x7f84a01027c0 2024-03-08T20:59:45.127 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:45.143+0000 7f849cc00700 1 -- 172.21.15.107:0/2560597074 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f849000bd90 con 0x7f84a01027c0 2024-03-08T20:59:45.127 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:45.143+0000 7f849cc00700 1 -- 172.21.15.107:0/2560597074 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f849000f460 con 0x7f84a01027c0 2024-03-08T20:59:45.127 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:45.143+0000 7f84a4c00700 1 -- 172.21.15.107:0/2560597074 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84a01a1f80 con 0x7f84a01027c0 2024-03-08T20:59:45.127 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:45.143+0000 7f84a4c00700 1 -- 172.21.15.107:0/2560597074 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84a01a2320 con 0x7f84a01027c0 2024-03-08T20:59:45.127 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:45.144+0000 7f84a4c00700 1 -- 172.21.15.107:0/2560597074 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84a0068ae0 con 0x7f84a01027c0 2024-03-08T20:59:45.128 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:45.146+0000 7f849cc00700 1 -- 172.21.15.107:0/2560597074 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f849002a070 con 0x7f84a01027c0 2024-03-08T20:59:45.130 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:45.147+0000 7f849cc00700 1 --2- 172.21.15.107:0/2560597074 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f84840a2910 0x7f84840a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:45.130 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:45.147+0000 7f849cc00700 1 -- 172.21.15.107:0/2560597074 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f84900cf9b0 con 0x7f84a01027c0 2024-03-08T20:59:45.130 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:45.147+0000 7f8497e00700 1 --2- 172.21.15.107:0/2560597074 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f84840a2910 0x7f84840a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:59:45.131 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:45.148+0000 7f8497e00700 1 --2- 172.21.15.107:0/2560597074 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f84840a2910 0x7f84840a4dd0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f8488005950 tx=0x7f84880058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:59:45.134 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:45.152+0000 7f849cc00700 1 -- 172.21.15.107:0/2560597074 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f8490013070 con 0x7f84a01027c0 2024-03-08T20:59:45.364 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:45.382+0000 7f84a4c00700 1 -- 172.21.15.107:0/2560597074 --> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f84a0062220 con 0x7f84840a2910 2024-03-08T20:59:45.405 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:45.423+0000 7f849cc00700 1 -- 172.21.15.107:0/2560597074 <== mgr.14152 v2:172.21.15.107:6800/3376636618 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+4123 (secure 0 0 0) 0x7f84a0062220 con 0x7f84840a2910 2024-03-08T20:59:45.407 INFO:teuthology.orchestra.run.smithi107.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-08T20:59:45.407 INFO:teuthology.orchestra.run.smithi107.stdout:mds.a smithi107 running (2m) 25s ago 2m 15.7M - 19.0.0-1965-g7d63482a a272146cb810 b3abd6c77684 2024-03-08T20:59:45.407 INFO:teuthology.orchestra.run.smithi107.stdout:mds.b smithi149 running (2m) 48s ago 2m 24.4M - 19.0.0-1965-g7d63482a a272146cb810 e8a745fa0d14 2024-03-08T20:59:45.407 INFO:teuthology.orchestra.run.smithi107.stdout:mds.c smithi204 running (2m) 49s ago 2m 16.3M - 19.0.0-1965-g7d63482a a272146cb810 b889757eaf94 2024-03-08T20:59:45.407 INFO:teuthology.orchestra.run.smithi107.stdout:mds.d smithi107 running (2m) 25s ago 2m 12.4M - 19.0.0-1965-g7d63482a a272146cb810 d1bfb0946480 2024-03-08T20:59:45.407 INFO:teuthology.orchestra.run.smithi107.stdout:mds.e smithi149 running (2m) 48s ago 2m 16.2M - 19.0.0-1965-g7d63482a a272146cb810 112663c35378 2024-03-08T20:59:45.407 INFO:teuthology.orchestra.run.smithi107.stdout:mds.f smithi204 running (2m) 49s ago 2m 15.4M - 19.0.0-1965-g7d63482a a272146cb810 05a8d06ffdec 2024-03-08T20:59:45.407 INFO:teuthology.orchestra.run.smithi107.stdout:mds.g smithi107 running (2m) 25s ago 2m 15.8M - 19.0.0-1965-g7d63482a a272146cb810 35370fcd36cd 2024-03-08T20:59:45.407 INFO:teuthology.orchestra.run.smithi107.stdout:mds.h smithi149 running (2m) 48s ago 2m 16.2M - 19.0.0-1965-g7d63482a a272146cb810 76101b0dc810 2024-03-08T20:59:45.407 INFO:teuthology.orchestra.run.smithi107.stdout:mds.i smithi204 running (2m) 49s ago 2m 15.9M - 19.0.0-1965-g7d63482a a272146cb810 728070e41a85 2024-03-08T20:59:45.407 INFO:teuthology.orchestra.run.smithi107.stdout:mds.j smithi107 running (2m) 25s ago 2m 12.4M - 19.0.0-1965-g7d63482a a272146cb810 eee17e8f22b6 2024-03-08T20:59:45.408 INFO:teuthology.orchestra.run.smithi107.stdout:mds.k smithi149 running (2m) 48s ago 2m 16.0M - 19.0.0-1965-g7d63482a a272146cb810 4bdbc491acce 2024-03-08T20:59:45.408 INFO:teuthology.orchestra.run.smithi107.stdout:mds.l smithi204 running (2m) 49s ago 2m 15.5M - 19.0.0-1965-g7d63482a a272146cb810 859782e18848 2024-03-08T20:59:45.408 INFO:teuthology.orchestra.run.smithi107.stdout:mgr.x smithi107 *:9283,8765 running (12m) 25s ago 12m 513M - 19.0.0-1965-g7d63482a a272146cb810 cec518020e7d 2024-03-08T20:59:45.408 INFO:teuthology.orchestra.run.smithi107.stdout:mgr.y smithi149 *:8443,8765 running (9m) 48s ago 9m 451M - 19.0.0-1965-g7d63482a a272146cb810 47118d6ce41b 2024-03-08T20:59:45.408 INFO:teuthology.orchestra.run.smithi107.stdout:mgr.z smithi204 *:8443,8765 running (9m) 49s ago 9m 451M - 19.0.0-1965-g7d63482a a272146cb810 6cff0bd156dc 2024-03-08T20:59:45.408 INFO:teuthology.orchestra.run.smithi107.stdout:mon.a smithi107 running (12m) 25s ago 12m 49.6M 2048M 19.0.0-1965-g7d63482a a272146cb810 80e0993b3b70 2024-03-08T20:59:45.408 INFO:teuthology.orchestra.run.smithi107.stdout:mon.b smithi149 running (10m) 48s ago 10m 39.4M 2048M 19.0.0-1965-g7d63482a a272146cb810 8cecc7919a78 2024-03-08T20:59:45.408 INFO:teuthology.orchestra.run.smithi107.stdout:mon.c smithi204 running (10m) 49s ago 10m 39.4M 2048M 19.0.0-1965-g7d63482a a272146cb810 c7d2a8d93020 2024-03-08T20:59:45.408 INFO:teuthology.orchestra.run.smithi107.stdout:osd.0 smithi107 running (9m) 25s ago 9m 53.0M 4272M 19.0.0-1965-g7d63482a a272146cb810 6269d4bb525d 2024-03-08T20:59:45.408 INFO:teuthology.orchestra.run.smithi107.stdout:osd.1 smithi149 running (8m) 48s ago 8m 45.6M 1454M 19.0.0-1965-g7d63482a a272146cb810 e9c05da3e3f3 2024-03-08T20:59:45.408 INFO:teuthology.orchestra.run.smithi107.stdout:osd.2 smithi204 running (8m) 49s ago 8m 46.3M 4272M 19.0.0-1965-g7d63482a a272146cb810 ff79fc65d95c 2024-03-08T20:59:45.408 INFO:teuthology.orchestra.run.smithi107.stdout:osd.3 smithi107 running (7m) 25s ago 7m 53.8M 4272M 19.0.0-1965-g7d63482a a272146cb810 bb1b3a2897bc 2024-03-08T20:59:45.408 INFO:teuthology.orchestra.run.smithi107.stdout:osd.4 smithi149 running (7m) 48s ago 7m 53.3M 1454M 19.0.0-1965-g7d63482a a272146cb810 ce49b985891b 2024-03-08T20:59:45.408 INFO:teuthology.orchestra.run.smithi107.stdout:osd.5 smithi204 running (6m) 49s ago 6m 53.6M 4272M 19.0.0-1965-g7d63482a a272146cb810 1a43b081163e 2024-03-08T20:59:45.409 INFO:teuthology.orchestra.run.smithi107.stdout:osd.6 smithi107 running (6m) 25s ago 6m 50.4M 4272M 19.0.0-1965-g7d63482a a272146cb810 b9c3ca32bbc4 2024-03-08T20:59:45.409 INFO:teuthology.orchestra.run.smithi107.stdout:osd.7 smithi149 running (5m) 48s ago 5m 42.5M 1454M 19.0.0-1965-g7d63482a a272146cb810 9d448c10c2d5 2024-03-08T20:59:45.409 INFO:teuthology.orchestra.run.smithi107.stdout:osd.8 smithi204 running (4m) 49s ago 4m 42.0M 4272M 19.0.0-1965-g7d63482a a272146cb810 2cf02f530e2f 2024-03-08T20:59:45.409 INFO:teuthology.orchestra.run.smithi107.stdout:osd.9 smithi107 running (4m) 25s ago 4m 52.6M 4272M 19.0.0-1965-g7d63482a a272146cb810 78eeaf6f4892 2024-03-08T20:59:45.409 INFO:teuthology.orchestra.run.smithi107.stdout:osd.10 smithi149 running (3m) 48s ago 3m 54.2M 1454M 19.0.0-1965-g7d63482a a272146cb810 d91396563d89 2024-03-08T20:59:45.409 INFO:teuthology.orchestra.run.smithi107.stdout:osd.11 smithi204 running (3m) 49s ago 3m 51.0M 4272M 19.0.0-1965-g7d63482a a272146cb810 3d7919046eb5 2024-03-08T20:59:45.409 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:45.426+0000 7f84a4c00700 1 -- 172.21.15.107:0/2560597074 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f84840a2910 msgr2=0x7f84840a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:59:45.410 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:45.426+0000 7f84a4c00700 1 --2- 172.21.15.107:0/2560597074 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f84840a2910 0x7f84840a4dd0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f8488005950 tx=0x7f84880058e0 comp rx=0 tx=0).stop 2024-03-08T20:59:45.410 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:45.426+0000 7f84a4c00700 1 -- 172.21.15.107:0/2560597074 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f84a01027c0 msgr2=0x7f84a019a090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:59:45.410 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:45.426+0000 7f84a4c00700 1 --2- 172.21.15.107:0/2560597074 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f84a01027c0 0x7f84a019a090 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f8490006010 tx=0x7f8490003c30 comp rx=0 tx=0).stop 2024-03-08T20:59:45.410 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:45.427+0000 7f84a4c00700 1 -- 172.21.15.107:0/2560597074 shutdown_connections 2024-03-08T20:59:45.410 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:45.427+0000 7f84a4c00700 1 --2- 172.21.15.107:0/2560597074 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f84840a2910 0x7f84840a4dd0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:45.410 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:45.427+0000 7f84a4c00700 1 --2- 172.21.15.107:0/2560597074 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f84a0107a20 0x7f84a01a1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:45.410 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:45.427+0000 7f84a4c00700 1 --2- 172.21.15.107:0/2560597074 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f84a01027c0 0x7f84a019a090 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:45.410 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:45.427+0000 7f84a4c00700 1 --2- 172.21.15.107:0/2560597074 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f84a01050f0 0x7f84a019a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:45.410 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:45.427+0000 7f84a4c00700 1 -- 172.21.15.107:0/2560597074 >> 172.21.15.107:0/2560597074 conn(0x7f84a00fc1b0 msgr2=0x7f84a0101f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:59:45.410 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:45.427+0000 7f84a4c00700 1 -- 172.21.15.107:0/2560597074 shutdown_connections 2024-03-08T20:59:45.410 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:45.427+0000 7f84a4c00700 1 -- 172.21.15.107:0/2560597074 wait complete. 2024-03-08T20:59:45.717 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:59:45 smithi107 ceph-mon[31381]: pgmap v420: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s 2024-03-08T20:59:45.764 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:59:45 smithi149 ceph-mon[35042]: pgmap v420: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s 2024-03-08T20:59:45.878 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:59:45 smithi204 ceph-mon[34935]: pgmap v420: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s 2024-03-08T20:59:46.203 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dc20f0f8-dd8c-11ee-95c6-87774f69a715 -- bash -c 'ceph orch ls' 2024-03-08T20:59:46.665 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/mon.a/config 2024-03-08T20:59:46.717 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:59:46 smithi107 ceph-mon[31381]: from='client.15147 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-08T20:59:46.764 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:59:46 smithi149 ceph-mon[35042]: from='client.15147 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-08T20:59:46.878 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:59:46 smithi204 ceph-mon[34935]: from='client.15147 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-08T20:59:47.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:59:47 smithi107 ceph-mon[31381]: pgmap v421: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 9 op/s 2024-03-08T20:59:47.764 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:59:47 smithi149 ceph-mon[35042]: pgmap v421: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 9 op/s 2024-03-08T20:59:47.878 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:59:47 smithi204 ceph-mon[34935]: pgmap v421: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 9 op/s 2024-03-08T20:59:48.950 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:48.967+0000 7fa95e600700 1 -- 172.21.15.107:0/547694745 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa958075e10 msgr2=0x7fa958074270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:59:48.950 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:48.967+0000 7fa95e600700 1 --2- 172.21.15.107:0/547694745 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa958075e10 0x7fa958074270 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fa948009990 tx=0x7fa948009ca0 comp rx=0 tx=0).stop 2024-03-08T20:59:48.950 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:48.967+0000 7fa95e600700 1 -- 172.21.15.107:0/547694745 shutdown_connections 2024-03-08T20:59:48.951 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:48.967+0000 7fa95e600700 1 --2- 172.21.15.107:0/547694745 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa95810e3e0 0x7fa9581107d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:48.951 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:48.967+0000 7fa95e600700 1 --2- 172.21.15.107:0/547694745 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa958075e10 0x7fa958074270 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:48.951 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:48.967+0000 7fa95e600700 1 --2- 172.21.15.107:0/547694745 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa958074840 0x7fa958074cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:48.951 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:48.967+0000 7fa95e600700 1 -- 172.21.15.107:0/547694745 >> 172.21.15.107:0/547694745 conn(0x7fa9580fd3c0 msgr2=0x7fa9580ff7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:59:48.951 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:48.967+0000 7fa95e600700 1 -- 172.21.15.107:0/547694745 shutdown_connections 2024-03-08T20:59:48.951 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:48.967+0000 7fa95e600700 1 -- 172.21.15.107:0/547694745 wait complete. 2024-03-08T20:59:48.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:48.968+0000 7fa95e600700 1 Processor -- start 2024-03-08T20:59:48.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:48.969+0000 7fa95e600700 1 -- start start 2024-03-08T20:59:48.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:48.969+0000 7fa95e600700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa958074840 0x7fa95819e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:48.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:48.969+0000 7fa95e600700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa958075e10 0x7fa95819eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:48.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:48.969+0000 7fa95e600700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa95810e3e0 0x7fa9581a5bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:48.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:48.969+0000 7fa95e600700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa958113ed0 con 0x7fa958075e10 2024-03-08T20:59:48.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:48.969+0000 7fa95e600700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa958113d50 con 0x7fa958074840 2024-03-08T20:59:48.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:48.969+0000 7fa95e600700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa958114050 con 0x7fa95810e3e0 2024-03-08T20:59:48.954 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:48.970+0000 7fa956a00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa958075e10 0x7fa95819eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:59:48.954 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:48.970+0000 7fa956a00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa958075e10 0x7fa95819eb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:49668/0 (socket says 172.21.15.107:49668) 2024-03-08T20:59:48.954 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:48.970+0000 7fa956a00700 1 -- 172.21.15.107:0/3100767417 learned_addr learned my addr 172.21.15.107:0/3100767417 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T20:59:48.954 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:48.970+0000 7fa957400700 1 --2- 172.21.15.107:0/3100767417 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa958074840 0x7fa95819e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:59:48.954 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:48.970+0000 7fa957e00700 1 --2- 172.21.15.107:0/3100767417 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa95810e3e0 0x7fa9581a5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:59:48.954 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:48.970+0000 7fa956a00700 1 -- 172.21.15.107:0/3100767417 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa95810e3e0 msgr2=0x7fa9581a5bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:59:48.954 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:48.970+0000 7fa956a00700 1 --2- 172.21.15.107:0/3100767417 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa95810e3e0 0x7fa9581a5bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:48.954 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:48.970+0000 7fa956a00700 1 -- 172.21.15.107:0/3100767417 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa958074840 msgr2=0x7fa95819e600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:59:48.954 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:48.970+0000 7fa956a00700 1 --2- 172.21.15.107:0/3100767417 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa958074840 0x7fa95819e600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:48.954 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:48.970+0000 7fa956a00700 1 -- 172.21.15.107:0/3100767417 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9581a6260 con 0x7fa958075e10 2024-03-08T20:59:48.955 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:48.971+0000 7fa956a00700 1 --2- 172.21.15.107:0/3100767417 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa958075e10 0x7fa95819eb40 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fa94400d9b0 tx=0x7fa94400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:59:48.955 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:48.971+0000 7fa94fe00700 1 -- 172.21.15.107:0/3100767417 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa944017030 con 0x7fa958075e10 2024-03-08T20:59:48.955 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:48.971+0000 7fa94fe00700 1 -- 172.21.15.107:0/3100767417 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa9440044d0 con 0x7fa958075e10 2024-03-08T20:59:48.956 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:48.971+0000 7fa94fe00700 1 -- 172.21.15.107:0/3100767417 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9440039e0 con 0x7fa958075e10 2024-03-08T20:59:48.956 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:48.971+0000 7fa95e600700 1 -- 172.21.15.107:0/3100767417 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9581a6550 con 0x7fa958075e10 2024-03-08T20:59:48.956 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:48.971+0000 7fa95e600700 1 -- 172.21.15.107:0/3100767417 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9581a6a40 con 0x7fa958075e10 2024-03-08T20:59:48.956 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:48.972+0000 7fa95e600700 1 -- 172.21.15.107:0/3100767417 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa958068aa0 con 0x7fa958075e10 2024-03-08T20:59:48.956 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:48.974+0000 7fa94fe00700 1 -- 172.21.15.107:0/3100767417 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) v1 ==== 137291+0+0 (secure 0 0 0) 0x7fa94401a070 con 0x7fa958075e10 2024-03-08T20:59:48.959 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:48.977+0000 7fa94fe00700 1 --2- 172.21.15.107:0/3100767417 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fa93c0a2910 0x7fa93c0a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:48.959 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:48.977+0000 7fa94fe00700 1 -- 172.21.15.107:0/3100767417 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fa9440ce350 con 0x7fa958075e10 2024-03-08T20:59:48.959 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:48.977+0000 7fa957400700 1 --2- 172.21.15.107:0/3100767417 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fa93c0a2910 0x7fa93c0a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:59:48.960 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:48.977+0000 7fa957400700 1 --2- 172.21.15.107:0/3100767417 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fa93c0a2910 0x7fa93c0a4dd0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fa948000c00 tx=0x7fa948005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:59:48.964 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:48.982+0000 7fa94fe00700 1 -- 172.21.15.107:0/3100767417 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7fa94409b3b0 con 0x7fa958075e10 2024-03-08T20:59:49.194 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:49.212+0000 7fa95e600700 1 -- 172.21.15.107:0/3100767417 --> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fa958062220 con 0x7fa93c0a2910 2024-03-08T20:59:49.211 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:49.228+0000 7fa94fe00700 1 -- 172.21.15.107:0/3100767417 <== mgr.14152 v2:172.21.15.107:6800/3376636618 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+980 (secure 0 0 0) 0x7fa958062220 con 0x7fa93c0a2910 2024-03-08T20:59:49.211 INFO:teuthology.orchestra.run.smithi107.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-03-08T20:59:49.211 INFO:teuthology.orchestra.run.smithi107.stdout:mds.all 12/12 53s ago 2m smithi107=a;smithi107=d;smithi107=g;smithi107=j;smithi149=b;smithi149=e;smithi149=h;smithi149=k;smithi204=c;smithi204=f;smithi204=i;smithi204=l;count:12 2024-03-08T20:59:49.211 INFO:teuthology.orchestra.run.smithi107.stdout:mgr 3/3 53s ago 10m smithi107=x;smithi149=y;smithi204=z;count:3 2024-03-08T20:59:49.211 INFO:teuthology.orchestra.run.smithi107.stdout:mon 3/3 53s ago 10m smithi107:172.21.15.107=a;smithi149:172.21.15.149=b;smithi204:172.21.15.204=c;count:3 2024-03-08T20:59:49.211 INFO:teuthology.orchestra.run.smithi107.stdout:osd 12 53s ago - 2024-03-08T20:59:49.214 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:49.231+0000 7fa95e600700 1 -- 172.21.15.107:0/3100767417 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fa93c0a2910 msgr2=0x7fa93c0a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:59:49.214 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:49.231+0000 7fa95e600700 1 --2- 172.21.15.107:0/3100767417 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fa93c0a2910 0x7fa93c0a4dd0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fa948000c00 tx=0x7fa948005fb0 comp rx=0 tx=0).stop 2024-03-08T20:59:49.214 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:49.231+0000 7fa95e600700 1 -- 172.21.15.107:0/3100767417 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa958075e10 msgr2=0x7fa95819eb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:59:49.214 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:49.231+0000 7fa95e600700 1 --2- 172.21.15.107:0/3100767417 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa958075e10 0x7fa95819eb40 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fa94400d9b0 tx=0x7fa94400dcc0 comp rx=0 tx=0).stop 2024-03-08T20:59:49.214 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:49.231+0000 7fa95e600700 1 -- 172.21.15.107:0/3100767417 shutdown_connections 2024-03-08T20:59:49.214 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:49.231+0000 7fa95e600700 1 --2- 172.21.15.107:0/3100767417 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa95810e3e0 0x7fa9581a5bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:49.214 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:49.231+0000 7fa95e600700 1 --2- 172.21.15.107:0/3100767417 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fa93c0a2910 0x7fa93c0a4dd0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:49.214 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:49.231+0000 7fa95e600700 1 --2- 172.21.15.107:0/3100767417 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa958074840 0x7fa95819e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:49.214 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:49.231+0000 7fa95e600700 1 --2- 172.21.15.107:0/3100767417 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa958075e10 0x7fa95819eb40 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:49.214 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:49.231+0000 7fa95e600700 1 -- 172.21.15.107:0/3100767417 >> 172.21.15.107:0/3100767417 conn(0x7fa9580fd3c0 msgr2=0x7fa958101780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:59:49.215 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:49.232+0000 7fa95e600700 1 -- 172.21.15.107:0/3100767417 shutdown_connections 2024-03-08T20:59:49.215 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:49.232+0000 7fa95e600700 1 -- 172.21.15.107:0/3100767417 wait complete. 2024-03-08T20:59:49.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:59:49 smithi107 ceph-mon[31381]: pgmap v422: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s 2024-03-08T20:59:49.764 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:59:49 smithi149 ceph-mon[35042]: pgmap v422: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s 2024-03-08T20:59:49.878 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:59:49 smithi204 ceph-mon[34935]: pgmap v422: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s 2024-03-08T20:59:49.993 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dc20f0f8-dd8c-11ee-95c6-87774f69a715 -- bash -c 'ceph orch host ls' 2024-03-08T20:59:50.450 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/mon.a/config 2024-03-08T20:59:50.717 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:59:50 smithi107 ceph-mon[31381]: from='client.15153 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-08T20:59:50.765 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:59:50 smithi149 ceph-mon[35042]: from='client.15153 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-08T20:59:50.878 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:59:50 smithi204 ceph-mon[34935]: from='client.15153 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-08T20:59:51.717 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:59:51 smithi107 ceph-mon[31381]: pgmap v423: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-08T20:59:51.764 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:59:51 smithi149 ceph-mon[35042]: pgmap v423: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-08T20:59:51.878 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:59:51 smithi204 ceph-mon[34935]: pgmap v423: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-08T20:59:52.698 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:52.715+0000 7f5865000700 1 -- 172.21.15.107:0/1683443008 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5860104060 msgr2=0x7f5860104480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:59:52.698 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:52.715+0000 7f5865000700 1 --2- 172.21.15.107:0/1683443008 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5860104060 0x7f5860104480 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f58500099c0 tx=0x7f5850009cd0 comp rx=0 tx=0).stop 2024-03-08T20:59:52.699 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:52.715+0000 7f5865000700 1 -- 172.21.15.107:0/1683443008 shutdown_connections 2024-03-08T20:59:52.699 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:52.715+0000 7f5865000700 1 --2- 172.21.15.107:0/1683443008 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5860105c00 0x7f586010c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:52.699 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:52.715+0000 7f5865000700 1 --2- 172.21.15.107:0/1683443008 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5860104060 0x7f5860104480 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:52.699 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:52.715+0000 7f5865000700 1 --2- 172.21.15.107:0/1683443008 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5860105260 0x7f58601056c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:52.699 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:52.715+0000 7f5865000700 1 -- 172.21.15.107:0/1683443008 >> 172.21.15.107:0/1683443008 conn(0x7f58600ff600 msgr2=0x7f5860101a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:59:52.699 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:52.715+0000 7f5865000700 1 -- 172.21.15.107:0/1683443008 shutdown_connections 2024-03-08T20:59:52.699 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:52.715+0000 7f5865000700 1 -- 172.21.15.107:0/1683443008 wait complete. 2024-03-08T20:59:52.701 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:52.717+0000 7f5865000700 1 Processor -- start 2024-03-08T20:59:52.702 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:52.717+0000 7f5865000700 1 -- start start 2024-03-08T20:59:52.702 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:52.717+0000 7f5865000700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5860104060 0x7f586019a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:52.702 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:52.717+0000 7f5865000700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5860105260 0x7f586019a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:52.702 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:52.717+0000 7f5865000700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5860105c00 0x7f58601a1760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:52.702 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:52.717+0000 7f5865000700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f586010faf0 con 0x7f5860105c00 2024-03-08T20:59:52.702 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:52.717+0000 7f5865000700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f586010f970 con 0x7f5860105260 2024-03-08T20:59:52.702 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:52.717+0000 7f5865000700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f586010fc70 con 0x7f5860104060 2024-03-08T20:59:52.702 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:52.718+0000 7f585e800700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5860105c00 0x7f58601a1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:59:52.702 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:52.718+0000 7f585e800700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5860105c00 0x7f58601a1760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:49682/0 (socket says 172.21.15.107:49682) 2024-03-08T20:59:52.702 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:52.718+0000 7f585e800700 1 -- 172.21.15.107:0/1982522480 learned_addr learned my addr 172.21.15.107:0/1982522480 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T20:59:52.702 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:52.718+0000 7f585d400700 1 --2- 172.21.15.107:0/1982522480 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5860105260 0x7f586019a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:59:52.703 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:52.718+0000 7f585de00700 1 --2- 172.21.15.107:0/1982522480 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5860104060 0x7f586019a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:59:52.703 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:52.718+0000 7f585e800700 1 -- 172.21.15.107:0/1982522480 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5860104060 msgr2=0x7f586019a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:59:52.703 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:52.718+0000 7f585e800700 1 --2- 172.21.15.107:0/1982522480 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5860104060 0x7f586019a1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:52.703 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:52.718+0000 7f585e800700 1 -- 172.21.15.107:0/1982522480 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5860105260 msgr2=0x7f586019a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:59:52.703 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:52.718+0000 7f585e800700 1 --2- 172.21.15.107:0/1982522480 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5860105260 0x7f586019a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:52.703 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:52.718+0000 7f585e800700 1 -- 172.21.15.107:0/1982522480 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58601a1e00 con 0x7f5860105c00 2024-03-08T20:59:52.703 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:52.718+0000 7f585de00700 1 --2- 172.21.15.107:0/1982522480 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5860104060 0x7f586019a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T20:59:52.703 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:52.719+0000 7f585e800700 1 --2- 172.21.15.107:0/1982522480 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5860105c00 0x7f58601a1760 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f5858007f10 tx=0x7f585800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:59:52.703 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:52.719+0000 7f5856a00700 1 -- 172.21.15.107:0/1982522480 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5858004bb0 con 0x7f5860105c00 2024-03-08T20:59:52.703 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:52.719+0000 7f5856a00700 1 -- 172.21.15.107:0/1982522480 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f5858004d50 con 0x7f5860105c00 2024-03-08T20:59:52.703 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:52.719+0000 7f5865000700 1 -- 172.21.15.107:0/1982522480 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58601a20f0 con 0x7f5860105c00 2024-03-08T20:59:52.703 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:52.719+0000 7f5856a00700 1 -- 172.21.15.107:0/1982522480 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5858005990 con 0x7f5860105c00 2024-03-08T20:59:52.703 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:52.719+0000 7f5865000700 1 -- 172.21.15.107:0/1982522480 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58601a25e0 con 0x7f5860105c00 2024-03-08T20:59:52.704 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:52.720+0000 7f5865000700 1 -- 172.21.15.107:0/1982522480 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5860068aa0 con 0x7f5860105c00 2024-03-08T20:59:52.704 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:52.722+0000 7f5856a00700 1 -- 172.21.15.107:0/1982522480 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f585801a020 con 0x7f5860105c00 2024-03-08T20:59:52.705 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:52.723+0000 7f5856a00700 1 --2- 172.21.15.107:0/1982522480 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f58440a2890 0x7f58440a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:52.706 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:52.724+0000 7f5856a00700 1 -- 172.21.15.107:0/1982522480 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f58580ce2c0 con 0x7f5860105c00 2024-03-08T20:59:52.706 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:52.724+0000 7f585de00700 1 --2- 172.21.15.107:0/1982522480 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f58440a2890 0x7f58440a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:59:52.706 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:52.724+0000 7f585de00700 1 --2- 172.21.15.107:0/1982522480 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f58440a2890 0x7f58440a4d50 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f5850000c00 tx=0x7f5850005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:59:52.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:52.728+0000 7f5856a00700 1 -- 172.21.15.107:0/1982522480 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f5858010070 con 0x7f5860105c00 2024-03-08T20:59:52.940 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:52.958+0000 7f5865000700 1 -- 172.21.15.107:0/1982522480 --> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f5860062220 con 0x7f58440a2890 2024-03-08T20:59:52.943 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:52.961+0000 7f5856a00700 1 -- 172.21.15.107:0/1982522480 <== mgr.14152 v2:172.21.15.107:6800/3376636618 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+191 (secure 0 0 0) 0x7f5860062220 con 0x7f58440a2890 2024-03-08T20:59:52.943 INFO:teuthology.orchestra.run.smithi107.stdout:HOST ADDR LABELS STATUS 2024-03-08T20:59:52.944 INFO:teuthology.orchestra.run.smithi107.stdout:smithi107 172.21.15.107 2024-03-08T20:59:52.944 INFO:teuthology.orchestra.run.smithi107.stdout:smithi149 172.21.15.149 2024-03-08T20:59:52.944 INFO:teuthology.orchestra.run.smithi107.stdout:smithi204 172.21.15.204 2024-03-08T20:59:52.944 INFO:teuthology.orchestra.run.smithi107.stdout:3 hosts in cluster 2024-03-08T20:59:52.946 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:52.964+0000 7f5865000700 1 -- 172.21.15.107:0/1982522480 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f58440a2890 msgr2=0x7f58440a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:59:52.946 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:52.964+0000 7f5865000700 1 --2- 172.21.15.107:0/1982522480 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f58440a2890 0x7f58440a4d50 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f5850000c00 tx=0x7f5850005fd0 comp rx=0 tx=0).stop 2024-03-08T20:59:52.946 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:52.964+0000 7f5865000700 1 -- 172.21.15.107:0/1982522480 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5860105c00 msgr2=0x7f58601a1760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:59:52.946 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:52.964+0000 7f5865000700 1 --2- 172.21.15.107:0/1982522480 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5860105c00 0x7f58601a1760 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f5858007f10 tx=0x7f585800d3b0 comp rx=0 tx=0).stop 2024-03-08T20:59:52.946 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:52.964+0000 7f5865000700 1 -- 172.21.15.107:0/1982522480 shutdown_connections 2024-03-08T20:59:52.947 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:52.964+0000 7f5865000700 1 --2- 172.21.15.107:0/1982522480 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5860104060 0x7f586019a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:52.947 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:52.964+0000 7f5865000700 1 --2- 172.21.15.107:0/1982522480 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f58440a2890 0x7f58440a4d50 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:52.947 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:52.964+0000 7f5865000700 1 --2- 172.21.15.107:0/1982522480 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5860105c00 0x7f58601a1760 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:52.947 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:52.964+0000 7f5865000700 1 --2- 172.21.15.107:0/1982522480 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5860105260 0x7f586019a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:52.947 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:52.964+0000 7f5865000700 1 -- 172.21.15.107:0/1982522480 >> 172.21.15.107:0/1982522480 conn(0x7f58600ff600 msgr2=0x7f58601085b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:59:52.947 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:52.964+0000 7f5865000700 1 -- 172.21.15.107:0/1982522480 shutdown_connections 2024-03-08T20:59:52.947 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:52.964+0000 7f5865000700 1 -- 172.21.15.107:0/1982522480 wait complete. 2024-03-08T20:59:53.717 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:59:53 smithi107 ceph-mon[31381]: pgmap v424: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-08T20:59:53.764 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:59:53 smithi149 ceph-mon[35042]: pgmap v424: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-08T20:59:53.797 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dc20f0f8-dd8c-11ee-95c6-87774f69a715 -- bash -c 'ceph orch device ls' 2024-03-08T20:59:53.878 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:59:53 smithi204 ceph-mon[34935]: pgmap v424: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-08T20:59:54.259 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/mon.a/config 2024-03-08T20:59:54.717 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:59:54 smithi107 ceph-mon[31381]: from='client.15159 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-08T20:59:54.764 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:59:54 smithi149 ceph-mon[35042]: from='client.15159 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-08T20:59:54.878 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:59:54 smithi204 ceph-mon[34935]: from='client.15159 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-08T20:59:55.632 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:59:55 smithi107 ceph-mon[31381]: pgmap v425: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-03-08T20:59:55.764 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:59:55 smithi149 ceph-mon[35042]: pgmap v425: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-03-08T20:59:55.878 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:59:55 smithi204 ceph-mon[34935]: pgmap v425: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-03-08T20:59:56.474 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:56.491+0000 7f23cc200700 1 -- 172.21.15.107:0/3366850346 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f23c4102f70 msgr2=0x7f23c41033f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:59:56.474 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:56.491+0000 7f23cc200700 1 --2- 172.21.15.107:0/3366850346 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f23c4102f70 0x7f23c41033f0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f23b0009990 tx=0x7f23b0009ca0 comp rx=0 tx=0).stop 2024-03-08T20:59:56.474 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:56.491+0000 7f23cc200700 1 -- 172.21.15.107:0/3366850346 shutdown_connections 2024-03-08T20:59:56.474 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:56.491+0000 7f23cc200700 1 --2- 172.21.15.107:0/3366850346 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f23c4103930 0x7f23c410a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:56.474 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:56.491+0000 7f23cc200700 1 --2- 172.21.15.107:0/3366850346 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f23c4101e10 0x7f23c4102230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:56.475 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:56.491+0000 7f23cc200700 1 --2- 172.21.15.107:0/3366850346 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f23c4102f70 0x7f23c41033f0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:56.475 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:56.491+0000 7f23cc200700 1 -- 172.21.15.107:0/3366850346 >> 172.21.15.107:0/3366850346 conn(0x7f23c40fd3f0 msgr2=0x7f23c40ff810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:59:56.475 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:56.492+0000 7f23cc200700 1 -- 172.21.15.107:0/3366850346 shutdown_connections 2024-03-08T20:59:56.475 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:56.492+0000 7f23cc200700 1 -- 172.21.15.107:0/3366850346 wait complete. 2024-03-08T20:59:56.476 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:56.493+0000 7f23cc200700 1 Processor -- start 2024-03-08T20:59:56.476 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:56.493+0000 7f23cc200700 1 -- start start 2024-03-08T20:59:56.476 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:56.493+0000 7f23cc200700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f23c4101e10 0x7f23c4197ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:56.477 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:56.493+0000 7f23cc200700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f23c4102f70 0x7f23c4198530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:56.477 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:56.493+0000 7f23cc200700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f23c4103930 0x7f23c419f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:56.477 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:56.493+0000 7f23cc200700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23c410d980 con 0x7f23c4102f70 2024-03-08T20:59:56.477 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:56.493+0000 7f23cc200700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23c410d800 con 0x7f23c4103930 2024-03-08T20:59:56.477 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:56.493+0000 7f23cc200700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23c410db00 con 0x7f23c4101e10 2024-03-08T20:59:56.477 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:56.493+0000 7f23c8e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f23c4102f70 0x7f23c4198530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:59:56.477 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:56.494+0000 7f23c8e00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f23c4102f70 0x7f23c4198530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:35938/0 (socket says 172.21.15.107:35938) 2024-03-08T20:59:56.477 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:56.494+0000 7f23c8e00700 1 -- 172.21.15.107:0/868711331 learned_addr learned my addr 172.21.15.107:0/868711331 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T20:59:56.477 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:56.494+0000 7f23c9800700 1 --2- 172.21.15.107:0/868711331 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f23c4101e10 0x7f23c4197ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:59:56.477 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:56.494+0000 7f23ca200700 1 --2- 172.21.15.107:0/868711331 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f23c4103930 0x7f23c419f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:59:56.477 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:56.494+0000 7f23c8e00700 1 -- 172.21.15.107:0/868711331 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f23c4101e10 msgr2=0x7f23c4197ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:59:56.477 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:56.494+0000 7f23c8e00700 1 --2- 172.21.15.107:0/868711331 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f23c4101e10 0x7f23c4197ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:56.477 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:56.494+0000 7f23c8e00700 1 -- 172.21.15.107:0/868711331 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f23c4103930 msgr2=0x7f23c419f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:59:56.478 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:56.494+0000 7f23c8e00700 1 --2- 172.21.15.107:0/868711331 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f23c4103930 0x7f23c419f5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:56.478 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:56.494+0000 7f23c8e00700 1 -- 172.21.15.107:0/868711331 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23c419fc50 con 0x7f23c4102f70 2024-03-08T20:59:56.478 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:56.494+0000 7f23c8e00700 1 --2- 172.21.15.107:0/868711331 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f23c4102f70 0x7f23c4198530 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f23b00057c0 tx=0x7f23b0005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:59:56.478 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:56.495+0000 7f23c2000700 1 -- 172.21.15.107:0/868711331 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23b0027070 con 0x7f23c4102f70 2024-03-08T20:59:56.479 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:56.495+0000 7f23c2000700 1 -- 172.21.15.107:0/868711331 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f23b0004080 con 0x7f23c4102f70 2024-03-08T20:59:56.479 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:56.495+0000 7f23cc200700 1 -- 172.21.15.107:0/868711331 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23c419fee0 con 0x7f23c4102f70 2024-03-08T20:59:56.479 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:56.495+0000 7f23c2000700 1 -- 172.21.15.107:0/868711331 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23b0022dd0 con 0x7f23c4102f70 2024-03-08T20:59:56.479 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:56.495+0000 7f23cc200700 1 -- 172.21.15.107:0/868711331 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23c41a0350 con 0x7f23c4102f70 2024-03-08T20:59:56.479 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:56.496+0000 7f23cc200700 1 -- 172.21.15.107:0/868711331 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23c4068aa0 con 0x7f23c4102f70 2024-03-08T20:59:56.480 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:56.498+0000 7f23c2000700 1 -- 172.21.15.107:0/868711331 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f23b002a070 con 0x7f23c4102f70 2024-03-08T20:59:56.482 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:56.500+0000 7f23c2000700 1 --2- 172.21.15.107:0/868711331 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f23ac0a2890 0x7f23ac0a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T20:59:56.482 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:56.500+0000 7f23c2000700 1 -- 172.21.15.107:0/868711331 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f23b00cf9e0 con 0x7f23c4102f70 2024-03-08T20:59:56.482 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:56.500+0000 7f23c9800700 1 --2- 172.21.15.107:0/868711331 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f23ac0a2890 0x7f23ac0a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T20:59:56.483 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:56.501+0000 7f23c9800700 1 --2- 172.21.15.107:0/868711331 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f23ac0a2890 0x7f23ac0a4d50 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f23b8009a10 tx=0x7f23b8009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T20:59:56.486 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:56.504+0000 7f23c2000700 1 -- 172.21.15.107:0/868711331 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f23b005e260 con 0x7f23c4102f70 2024-03-08T20:59:56.718 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:56.736+0000 7f23cc200700 1 -- 172.21.15.107:0/868711331 --> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f23c4062220 con 0x7f23ac0a2890 2024-03-08T20:59:56.726 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:56.740+0000 7f23c2000700 1 -- 172.21.15.107:0/868711331 <== mgr.14152 v2:172.21.15.107:6800/3376636618 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+532 (secure 0 0 0) 0x7f23c4062220 con 0x7f23ac0a2890 2024-03-08T20:59:56.726 INFO:teuthology.orchestra.run.smithi107.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-03-08T20:59:56.726 INFO:teuthology.orchestra.run.smithi107.stdout:smithi107 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400NK400BGN 372G No 4m ago Has a FileSystem, LVM detected 2024-03-08T20:59:56.726 INFO:teuthology.orchestra.run.smithi107.stdout:smithi149 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT623300CK400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-03-08T20:59:56.726 INFO:teuthology.orchestra.run.smithi107.stdout:smithi204 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT623300GD400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-03-08T20:59:56.727 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:56.743+0000 7f23cc200700 1 -- 172.21.15.107:0/868711331 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f23ac0a2890 msgr2=0x7f23ac0a4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:59:56.727 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:56.743+0000 7f23cc200700 1 --2- 172.21.15.107:0/868711331 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f23ac0a2890 0x7f23ac0a4d50 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f23b8009a10 tx=0x7f23b8009190 comp rx=0 tx=0).stop 2024-03-08T20:59:56.727 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:56.743+0000 7f23cc200700 1 -- 172.21.15.107:0/868711331 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f23c4102f70 msgr2=0x7f23c4198530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T20:59:56.727 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:56.743+0000 7f23cc200700 1 --2- 172.21.15.107:0/868711331 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f23c4102f70 0x7f23c4198530 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f23b00057c0 tx=0x7f23b0005ef0 comp rx=0 tx=0).stop 2024-03-08T20:59:56.727 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:56.744+0000 7f23cc200700 1 -- 172.21.15.107:0/868711331 shutdown_connections 2024-03-08T20:59:56.727 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:56.744+0000 7f23cc200700 1 --2- 172.21.15.107:0/868711331 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f23c4101e10 0x7f23c4197ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:56.727 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:56.744+0000 7f23cc200700 1 --2- 172.21.15.107:0/868711331 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f23ac0a2890 0x7f23ac0a4d50 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:56.727 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:56.744+0000 7f23cc200700 1 --2- 172.21.15.107:0/868711331 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f23c4103930 0x7f23c419f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:56.727 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:56.744+0000 7f23cc200700 1 --2- 172.21.15.107:0/868711331 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f23c4102f70 0x7f23c4198530 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T20:59:56.728 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:56.744+0000 7f23cc200700 1 -- 172.21.15.107:0/868711331 >> 172.21.15.107:0/868711331 conn(0x7f23c40fd3f0 msgr2=0x7f23c4106380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T20:59:56.728 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:56.744+0000 7f23cc200700 1 -- 172.21.15.107:0/868711331 shutdown_connections 2024-03-08T20:59:56.728 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T20:59:56.744+0000 7f23cc200700 1 -- 172.21.15.107:0/868711331 wait complete. 2024-03-08T20:59:57.575 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-08T20:59:57.591 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi107.front.sepia.ceph.com 2024-03-08T20:59:57.592 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dc20f0f8-dd8c-11ee-95c6-87774f69a715 -- bash -c 'ceph fs dump' 2024-03-08T20:59:57.717 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:59:57 smithi107 ceph-mon[31381]: pgmap v426: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-08T20:59:57.765 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:59:57 smithi149 ceph-mon[35042]: pgmap v426: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-08T20:59:57.878 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:59:57 smithi204 ceph-mon[34935]: pgmap v426: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-08T20:59:58.043 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/mon.a/config 2024-03-08T20:59:58.764 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:59:58 smithi149 ceph-mon[35042]: from='client.15165 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-08T20:59:58.880 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:59:58 smithi204 ceph-mon[34935]: from='client.15165 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-08T20:59:58.967 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:59:58 smithi107 ceph-mon[31381]: from='client.15165 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-08T20:59:59.764 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 20:59:59 smithi149 ceph-mon[35042]: pgmap v427: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-03-08T20:59:59.853 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 20:59:59 smithi107 ceph-mon[31381]: pgmap v427: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-03-08T20:59:59.878 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 20:59:59 smithi204 ceph-mon[34935]: pgmap v427: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-03-08T21:00:00.347 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:00.362+0000 7f715e200700 1 -- 172.21.15.107:0/1607105151 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7158105260 msgr2=0x7f71581056c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:00:00.347 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:00.362+0000 7f715e200700 1 --2- 172.21.15.107:0/1607105151 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7158105260 0x7f71581056c0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f71400099c0 tx=0x7f7140009cd0 comp rx=0 tx=0).stop 2024-03-08T21:00:00.347 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:00.362+0000 7f715e200700 1 -- 172.21.15.107:0/1607105151 shutdown_connections 2024-03-08T21:00:00.347 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:00.362+0000 7f715e200700 1 --2- 172.21.15.107:0/1607105151 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7158105c00 0x7f715810c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:00.348 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:00.362+0000 7f715e200700 1 --2- 172.21.15.107:0/1607105151 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7158104060 0x7f7158104480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:00.348 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:00.362+0000 7f715e200700 1 --2- 172.21.15.107:0/1607105151 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7158105260 0x7f71581056c0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:00.348 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:00.362+0000 7f715e200700 1 -- 172.21.15.107:0/1607105151 >> 172.21.15.107:0/1607105151 conn(0x7f71580ff600 msgr2=0x7f7158101a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:00:00.348 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:00.363+0000 7f715e200700 1 -- 172.21.15.107:0/1607105151 shutdown_connections 2024-03-08T21:00:00.348 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:00.363+0000 7f715e200700 1 -- 172.21.15.107:0/1607105151 wait complete. 2024-03-08T21:00:00.348 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:00.364+0000 7f715e200700 1 Processor -- start 2024-03-08T21:00:00.348 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:00.364+0000 7f715e200700 1 -- start start 2024-03-08T21:00:00.348 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:00.364+0000 7f715e200700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7158104060 0x7f715819a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:00:00.349 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:00.364+0000 7f715e200700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7158105260 0x7f715819a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:00:00.349 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:00.364+0000 7f715e200700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7158105c00 0x7f71581a1770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:00:00.349 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:00.364+0000 7f715e200700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f715810fba0 con 0x7f7158104060 2024-03-08T21:00:00.349 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:00.364+0000 7f715e200700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f715810fa20 con 0x7f7158105260 2024-03-08T21:00:00.349 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:00.364+0000 7f715e200700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f715810fd20 con 0x7f7158105c00 2024-03-08T21:00:00.349 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:00.364+0000 7f7157400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7158104060 0x7f715819a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:00:00.349 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:00.365+0000 7f7157400700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7158104060 0x7f715819a1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:35960/0 (socket says 172.21.15.107:35960) 2024-03-08T21:00:00.349 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:00.365+0000 7f7157400700 1 -- 172.21.15.107:0/2298657363 learned_addr learned my addr 172.21.15.107:0/2298657363 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:00:00.349 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:00.365+0000 7f7157e00700 1 --2- 172.21.15.107:0/2298657363 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7158105c00 0x7f71581a1770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:00:00.349 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:00.365+0000 7f714ea00700 1 --2- 172.21.15.107:0/2298657363 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7158105260 0x7f715819a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:00:00.349 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:00.365+0000 7f7157400700 1 -- 172.21.15.107:0/2298657363 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7158105c00 msgr2=0x7f71581a1770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:00:00.350 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:00.365+0000 7f7157400700 1 --2- 172.21.15.107:0/2298657363 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7158105c00 0x7f71581a1770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:00.351 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:00.365+0000 7f7157400700 1 -- 172.21.15.107:0/2298657363 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7158105260 msgr2=0x7f715819a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:00:00.351 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:00.365+0000 7f7157400700 1 --2- 172.21.15.107:0/2298657363 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7158105260 0x7f715819a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:00.351 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:00.365+0000 7f7157400700 1 -- 172.21.15.107:0/2298657363 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71581a1e10 con 0x7f7158104060 2024-03-08T21:00:00.351 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:00.365+0000 7f7157400700 1 --2- 172.21.15.107:0/2298657363 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7158104060 0x7f715819a1b0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f714800c9a0 tx=0x7f714800cd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:00:00.351 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:00.366+0000 7f7154c00700 1 -- 172.21.15.107:0/2298657363 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7148004220 con 0x7f7158104060 2024-03-08T21:00:00.351 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:00.366+0000 7f7154c00700 1 -- 172.21.15.107:0/2298657363 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f7148007d10 con 0x7f7158104060 2024-03-08T21:00:00.351 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:00.366+0000 7f7154c00700 1 -- 172.21.15.107:0/2298657363 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7148003e10 con 0x7f7158104060 2024-03-08T21:00:00.351 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:00.366+0000 7f715e200700 1 -- 172.21.15.107:0/2298657363 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71581a2100 con 0x7f7158104060 2024-03-08T21:00:00.351 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:00.366+0000 7f715e200700 1 -- 172.21.15.107:0/2298657363 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71581a2670 con 0x7f7158104060 2024-03-08T21:00:00.351 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:00.367+0000 7f715e200700 1 -- 172.21.15.107:0/2298657363 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7158068aa0 con 0x7f7158104060 2024-03-08T21:00:00.352 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:00.369+0000 7f7154c00700 1 -- 172.21.15.107:0/2298657363 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f7148007860 con 0x7f7158104060 2024-03-08T21:00:00.352 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:00.370+0000 7f7154c00700 1 --2- 172.21.15.107:0/2298657363 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f713c0a2910 0x7f713c0a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:00:00.353 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:00.371+0000 7f7154c00700 1 -- 172.21.15.107:0/2298657363 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f71480ce980 con 0x7f7158104060 2024-03-08T21:00:00.353 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:00.371+0000 7f714ea00700 1 --2- 172.21.15.107:0/2298657363 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f713c0a2910 0x7f713c0a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:00:00.353 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:00.371+0000 7f714ea00700 1 --2- 172.21.15.107:0/2298657363 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f713c0a2910 0x7f713c0a4dd0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f7140009fd0 tx=0x7f7140005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:00:00.357 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:00.375+0000 7f7154c00700 1 -- 172.21.15.107:0/2298657363 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f7148014030 con 0x7f7158104060 2024-03-08T21:00:00.629 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:00.646+0000 7f715e200700 1 -- 172.21.15.107:0/2298657363 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f71580629f0 con 0x7f7158104060 2024-03-08T21:00:00.630 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:00.647+0000 7f7154c00700 1 -- 172.21.15.107:0/2298657363 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 27 v27) v1 ==== 76+0+2971 (secure 0 0 0) 0x7f714809b9e0 con 0x7f7158104060 2024-03-08T21:00:00.631 INFO:teuthology.orchestra.run.smithi107.stdout:e27 2024-03-08T21:00:00.631 INFO:teuthology.orchestra.run.smithi107.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-08T21:00:00.631 INFO:teuthology.orchestra.run.smithi107.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-03-08T21:00:00.631 INFO:teuthology.orchestra.run.smithi107.stdout:legacy client fscid: 1 2024-03-08T21:00:00.631 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-08T21:00:00.631 INFO:teuthology.orchestra.run.smithi107.stdout:Filesystem 'cephfs' (1) 2024-03-08T21:00:00.632 INFO:teuthology.orchestra.run.smithi107.stdout:fs_name cephfs 2024-03-08T21:00:00.632 INFO:teuthology.orchestra.run.smithi107.stdout:epoch 27 2024-03-08T21:00:00.632 INFO:teuthology.orchestra.run.smithi107.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-03-08T21:00:00.632 INFO:teuthology.orchestra.run.smithi107.stdout:created 2024-03-08T20:57:50.223752+0000 2024-03-08T21:00:00.632 INFO:teuthology.orchestra.run.smithi107.stdout:modified 2024-03-08T20:57:57.874727+0000 2024-03-08T21:00:00.632 INFO:teuthology.orchestra.run.smithi107.stdout:tableserver 0 2024-03-08T21:00:00.632 INFO:teuthology.orchestra.run.smithi107.stdout:root 0 2024-03-08T21:00:00.632 INFO:teuthology.orchestra.run.smithi107.stdout:session_timeout 300 2024-03-08T21:00:00.632 INFO:teuthology.orchestra.run.smithi107.stdout:session_autoclose 300 2024-03-08T21:00:00.632 INFO:teuthology.orchestra.run.smithi107.stdout:max_file_size 1099511627776 2024-03-08T21:00:00.633 INFO:teuthology.orchestra.run.smithi107.stdout:max_xattr_size 65536 2024-03-08T21:00:00.633 INFO:teuthology.orchestra.run.smithi107.stdout:required_client_features {} 2024-03-08T21:00:00.633 INFO:teuthology.orchestra.run.smithi107.stdout:last_failure 0 2024-03-08T21:00:00.633 INFO:teuthology.orchestra.run.smithi107.stdout:last_failure_osd_epoch 0 2024-03-08T21:00:00.633 INFO:teuthology.orchestra.run.smithi107.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-03-08T21:00:00.633 INFO:teuthology.orchestra.run.smithi107.stdout:max_mds 5 2024-03-08T21:00:00.633 INFO:teuthology.orchestra.run.smithi107.stdout:in 0,1,2,3,4 2024-03-08T21:00:00.633 INFO:teuthology.orchestra.run.smithi107.stdout:up {0=34475,1=34445,2=34415,3=34397,4=24457} 2024-03-08T21:00:00.633 INFO:teuthology.orchestra.run.smithi107.stdout:failed 2024-03-08T21:00:00.633 INFO:teuthology.orchestra.run.smithi107.stdout:damaged 2024-03-08T21:00:00.633 INFO:teuthology.orchestra.run.smithi107.stdout:stopped 2024-03-08T21:00:00.633 INFO:teuthology.orchestra.run.smithi107.stdout:data_pools [3] 2024-03-08T21:00:00.633 INFO:teuthology.orchestra.run.smithi107.stdout:metadata_pool 2 2024-03-08T21:00:00.634 INFO:teuthology.orchestra.run.smithi107.stdout:inline_data disabled 2024-03-08T21:00:00.634 INFO:teuthology.orchestra.run.smithi107.stdout:balancer 2024-03-08T21:00:00.634 INFO:teuthology.orchestra.run.smithi107.stdout:bal_rank_mask -1 2024-03-08T21:00:00.634 INFO:teuthology.orchestra.run.smithi107.stdout:standby_count_wanted 0 2024-03-08T21:00:00.634 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.b{0:34475} state up:active seq 3 addr [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] compat {c=[1],r=[1],i=[fff]}] 2024-03-08T21:00:00.634 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.c{0:24442} state up:standby-replay seq 1 addr [v2:172.21.15.204:6836/42644905,v1:172.21.15.204:6837/42644905] compat {c=[1],r=[1],i=[fff]}] 2024-03-08T21:00:00.634 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.h{1:34445} state up:active seq 8 addr [v2:172.21.15.149:6836/914546237,v1:172.21.15.149:6837/914546237] compat {c=[1],r=[1],i=[fff]}] 2024-03-08T21:00:00.634 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.f{1:24430} state up:standby-replay seq 1 addr [v2:172.21.15.204:6834/2072518651,v1:172.21.15.204:6835/2072518651] compat {c=[1],r=[1],i=[fff]}] 2024-03-08T21:00:00.634 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.e{2:34415} state up:active seq 11 addr [v2:172.21.15.149:6834/442657066,v1:172.21.15.149:6835/442657066] compat {c=[1],r=[1],i=[fff]}] 2024-03-08T21:00:00.634 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.l{2:24400} state up:standby-replay seq 1 addr [v2:172.21.15.204:6832/3404762782,v1:172.21.15.204:6833/3404762782] compat {c=[1],r=[1],i=[fff]}] 2024-03-08T21:00:00.634 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.k{3:34397} state up:active seq 14 addr [v2:172.21.15.149:6832/1962161726,v1:172.21.15.149:6833/1962161726] compat {c=[1],r=[1],i=[fff]}] 2024-03-08T21:00:00.634 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.a{3:14637} state up:standby-replay seq 1 addr [v2:172.21.15.107:6840/747096194,v1:172.21.15.107:6841/747096194] compat {c=[1],r=[1],i=[fff]}] 2024-03-08T21:00:00.634 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.i{4:24457} state up:active seq 8 addr [v2:172.21.15.204:6838/1857316822,v1:172.21.15.204:6839/1857316822] compat {c=[1],r=[1],i=[fff]}] 2024-03-08T21:00:00.634 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.g{4:14625} state up:standby-replay seq 1 addr [v2:172.21.15.107:6838/3070059000,v1:172.21.15.107:6839/3070059000] compat {c=[1],r=[1],i=[fff]}] 2024-03-08T21:00:00.635 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-08T21:00:00.635 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-08T21:00:00.635 INFO:teuthology.orchestra.run.smithi107.stdout:Standby daemons: 2024-03-08T21:00:00.635 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-08T21:00:00.635 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.j{-1:14538} state up:standby seq 1 addr [v2:172.21.15.107:6834/133582743,v1:172.21.15.107:6835/133582743] compat {c=[1],r=[1],i=[fff]}] 2024-03-08T21:00:00.635 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.d{-1:14553} state up:standby seq 1 addr [v2:172.21.15.107:6836/3798982337,v1:172.21.15.107:6837/3798982337] compat {c=[1],r=[1],i=[fff]}] 2024-03-08T21:00:00.636 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:00.650+0000 7f715e200700 1 -- 172.21.15.107:0/2298657363 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f713c0a2910 msgr2=0x7f713c0a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:00:00.636 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:00.650+0000 7f715e200700 1 --2- 172.21.15.107:0/2298657363 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f713c0a2910 0x7f713c0a4dd0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f7140009fd0 tx=0x7f7140005fb0 comp rx=0 tx=0).stop 2024-03-08T21:00:00.636 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:00.650+0000 7f715e200700 1 -- 172.21.15.107:0/2298657363 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7158104060 msgr2=0x7f715819a1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:00:00.636 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:00.650+0000 7f715e200700 1 --2- 172.21.15.107:0/2298657363 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7158104060 0x7f715819a1b0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f714800c9a0 tx=0x7f714800cd60 comp rx=0 tx=0).stop 2024-03-08T21:00:00.636 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:00.651+0000 7f715e200700 1 -- 172.21.15.107:0/2298657363 shutdown_connections 2024-03-08T21:00:00.636 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:00.651+0000 7f715e200700 1 --2- 172.21.15.107:0/2298657363 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7158105c00 0x7f71581a1770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:00.636 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:00.651+0000 7f715e200700 1 --2- 172.21.15.107:0/2298657363 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f713c0a2910 0x7f713c0a4dd0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:00.636 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:00.651+0000 7f715e200700 1 --2- 172.21.15.107:0/2298657363 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7158104060 0x7f715819a1b0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:00.636 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:00.651+0000 7f715e200700 1 --2- 172.21.15.107:0/2298657363 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7158105260 0x7f715819a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:00.636 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:00.651+0000 7f715e200700 1 -- 172.21.15.107:0/2298657363 >> 172.21.15.107:0/2298657363 conn(0x7f71580ff600 msgr2=0x7f7158108560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:00:00.636 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:00.651+0000 7f715e200700 1 -- 172.21.15.107:0/2298657363 shutdown_connections 2024-03-08T21:00:00.636 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:00.651+0000 7f715e200700 1 -- 172.21.15.107:0/2298657363 wait complete. 2024-03-08T21:00:00.637 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 27 2024-03-08T21:00:00.765 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:00:00 smithi149 ceph-mon[35042]: overall HEALTH_OK 2024-03-08T21:00:00.878 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:00:00 smithi204 ceph-mon[34935]: overall HEALTH_OK 2024-03-08T21:00:00.967 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:00:00 smithi107 ceph-mon[31381]: overall HEALTH_OK 2024-03-08T21:00:01.623 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid dc20f0f8-dd8c-11ee-95c6-87774f69a715 -- bash -c 'ceph osd dump' 2024-03-08T21:00:01.764 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:00:01 smithi149 ceph-mon[35042]: pgmap v428: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-08T21:00:01.765 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:00:01 smithi149 ceph-mon[35042]: from='client.? 172.21.15.107:0/2298657363' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-08T21:00:01.878 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:00:01 smithi204 ceph-mon[34935]: pgmap v428: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-08T21:00:01.878 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:00:01 smithi204 ceph-mon[34935]: from='client.? 172.21.15.107:0/2298657363' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-08T21:00:01.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:00:01 smithi107 ceph-mon[31381]: pgmap v428: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-08T21:00:01.967 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:00:01 smithi107 ceph-mon[31381]: from='client.? 172.21.15.107:0/2298657363' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-08T21:00:02.097 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/mon.a/config 2024-03-08T21:00:03.764 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:00:03 smithi149 ceph-mon[35042]: pgmap v429: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-08T21:00:03.879 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:00:03 smithi204 ceph-mon[34935]: pgmap v429: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-08T21:00:03.913 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:00:03 smithi107 ceph-mon[31381]: pgmap v429: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-08T21:00:04.443 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:04.459+0000 7f6efde00700 1 -- 172.21.15.107:0/4154395538 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6ef8103970 msgr2=0x7f6ef810a290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:00:04.443 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:04.459+0000 7f6efde00700 1 --2- 172.21.15.107:0/4154395538 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6ef8103970 0x7f6ef810a290 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f6ef000b0a0 tx=0x7f6ef000b3b0 comp rx=0 tx=0).stop 2024-03-08T21:00:04.443 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:04.459+0000 7f6efde00700 1 -- 172.21.15.107:0/4154395538 shutdown_connections 2024-03-08T21:00:04.443 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:04.459+0000 7f6efde00700 1 --2- 172.21.15.107:0/4154395538 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6ef8101e50 0x7f6ef8102270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:04.443 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:04.459+0000 7f6efde00700 1 --2- 172.21.15.107:0/4154395538 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6ef8103970 0x7f6ef810a290 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:04.443 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:04.459+0000 7f6efde00700 1 --2- 172.21.15.107:0/4154395538 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6ef8102fb0 0x7f6ef8103430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:04.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:04.459+0000 7f6efde00700 1 -- 172.21.15.107:0/4154395538 >> 172.21.15.107:0/4154395538 conn(0x7f6ef80fd430 msgr2=0x7f6ef80ff850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:00:04.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:04.460+0000 7f6efde00700 1 -- 172.21.15.107:0/4154395538 shutdown_connections 2024-03-08T21:00:04.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:04.460+0000 7f6efde00700 1 -- 172.21.15.107:0/4154395538 wait complete. 2024-03-08T21:00:04.445 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:04.461+0000 7f6efde00700 1 Processor -- start 2024-03-08T21:00:04.445 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:04.461+0000 7f6efde00700 1 -- start start 2024-03-08T21:00:04.445 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:04.461+0000 7f6efde00700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6ef8101e50 0x7f6ef8075b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:00:04.445 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:04.461+0000 7f6efde00700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6ef8102fb0 0x7f6ef80741d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:00:04.445 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:04.461+0000 7f6efde00700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6ef8103970 0x7f6ef8074710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:00:04.445 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:04.461+0000 7f6efde00700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ef810d920 con 0x7f6ef8102fb0 2024-03-08T21:00:04.446 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:04.461+0000 7f6efde00700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ef810d7a0 con 0x7f6ef8103970 2024-03-08T21:00:04.446 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:04.461+0000 7f6efde00700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ef810daa0 con 0x7f6ef8101e50 2024-03-08T21:00:04.446 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:04.462+0000 7f6ef6000700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6ef8102fb0 0x7f6ef80741d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:00:04.446 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:04.462+0000 7f6ef6000700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6ef8102fb0 0x7f6ef80741d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:35986/0 (socket says 172.21.15.107:35986) 2024-03-08T21:00:04.446 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:04.462+0000 7f6ef6000700 1 -- 172.21.15.107:0/3660256740 learned_addr learned my addr 172.21.15.107:0/3660256740 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:00:04.446 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:04.462+0000 7f6ef6a00700 1 --2- 172.21.15.107:0/3660256740 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6ef8101e50 0x7f6ef8075b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:00:04.446 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:04.462+0000 7f6ef7400700 1 --2- 172.21.15.107:0/3660256740 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6ef8103970 0x7f6ef8074710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:00:04.446 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:04.462+0000 7f6ef6000700 1 -- 172.21.15.107:0/3660256740 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6ef8101e50 msgr2=0x7f6ef8075b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:00:04.446 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:04.462+0000 7f6ef6000700 1 --2- 172.21.15.107:0/3660256740 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6ef8101e50 0x7f6ef8075b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:04.446 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:04.462+0000 7f6ef6000700 1 -- 172.21.15.107:0/3660256740 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6ef8103970 msgr2=0x7f6ef8074710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:00:04.446 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:04.462+0000 7f6ef6000700 1 --2- 172.21.15.107:0/3660256740 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6ef8103970 0x7f6ef8074710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:04.446 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:04.462+0000 7f6ef6000700 1 -- 172.21.15.107:0/3660256740 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ef8074fc0 con 0x7f6ef8102fb0 2024-03-08T21:00:04.446 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:04.462+0000 7f6ef6000700 1 --2- 172.21.15.107:0/3660256740 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6ef8102fb0 0x7f6ef80741d0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f6ee400d950 tx=0x7f6ee400dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:00:04.447 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:04.463+0000 7f6efca00700 1 -- 172.21.15.107:0/3660256740 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ee4016070 con 0x7f6ef8102fb0 2024-03-08T21:00:04.447 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:04.463+0000 7f6efca00700 1 -- 172.21.15.107:0/3660256740 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f6ee4010460 con 0x7f6ef8102fb0 2024-03-08T21:00:04.447 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:04.463+0000 7f6efde00700 1 -- 172.21.15.107:0/3660256740 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ef81a1fe0 con 0x7f6ef8102fb0 2024-03-08T21:00:04.447 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:04.463+0000 7f6efca00700 1 -- 172.21.15.107:0/3660256740 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ee400f620 con 0x7f6ef8102fb0 2024-03-08T21:00:04.447 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:04.463+0000 7f6efde00700 1 -- 172.21.15.107:0/3660256740 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ef81a2490 con 0x7f6ef8102fb0 2024-03-08T21:00:04.448 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:04.464+0000 7f6efde00700 1 -- 172.21.15.107:0/3660256740 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ef8068aa0 con 0x7f6ef8102fb0 2024-03-08T21:00:04.449 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:04.466+0000 7f6efca00700 1 -- 172.21.15.107:0/3660256740 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f6ee4019070 con 0x7f6ef8102fb0 2024-03-08T21:00:04.450 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:04.467+0000 7f6efca00700 1 --2- 172.21.15.107:0/3660256740 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f6edc0a2910 0x7f6edc0a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:00:04.450 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:04.467+0000 7f6efca00700 1 -- 172.21.15.107:0/3660256740 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f6ee40ce680 con 0x7f6ef8102fb0 2024-03-08T21:00:04.450 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:04.467+0000 7f6ef6a00700 1 --2- 172.21.15.107:0/3660256740 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f6edc0a2910 0x7f6edc0a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:00:04.451 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:04.468+0000 7f6ef6a00700 1 --2- 172.21.15.107:0/3660256740 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f6edc0a2910 0x7f6edc0a4dd0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f6ee8009bf0 tx=0x7f6ee8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:00:04.455 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:04.472+0000 7f6efca00700 1 -- 172.21.15.107:0/3660256740 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f6ee409b6e0 con 0x7f6ef8102fb0 2024-03-08T21:00:04.681 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:04.697+0000 7f6efde00700 1 -- 172.21.15.107:0/3660256740 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd dump"} v 0) v1 -- 0x7f6ef8062a20 con 0x7f6ef8102fb0 2024-03-08T21:00:04.683 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:04.700+0000 7f6efca00700 1 -- 172.21.15.107:0/3660256740 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v88) v1 ==== 56+0+5341 (secure 0 0 0) 0x7f6ee409afe0 con 0x7f6ef8102fb0 2024-03-08T21:00:04.684 INFO:teuthology.orchestra.run.smithi107.stdout:epoch 88 2024-03-08T21:00:04.684 INFO:teuthology.orchestra.run.smithi107.stdout:fsid dc20f0f8-dd8c-11ee-95c6-87774f69a715 2024-03-08T21:00:04.685 INFO:teuthology.orchestra.run.smithi107.stdout:created 2024-03-08T20:47:04.190034+0000 2024-03-08T21:00:04.685 INFO:teuthology.orchestra.run.smithi107.stdout:modified 2024-03-08T20:58:21.449936+0000 2024-03-08T21:00:04.685 INFO:teuthology.orchestra.run.smithi107.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-03-08T21:00:04.685 INFO:teuthology.orchestra.run.smithi107.stdout:crush_version 26 2024-03-08T21:00:04.685 INFO:teuthology.orchestra.run.smithi107.stdout:full_ratio 0.95 2024-03-08T21:00:04.685 INFO:teuthology.orchestra.run.smithi107.stdout:backfillfull_ratio 0.9 2024-03-08T21:00:04.685 INFO:teuthology.orchestra.run.smithi107.stdout:nearfull_ratio 0.85 2024-03-08T21:00:04.685 INFO:teuthology.orchestra.run.smithi107.stdout:require_min_compat_client luminous 2024-03-08T21:00:04.685 INFO:teuthology.orchestra.run.smithi107.stdout:min_compat_client luminous 2024-03-08T21:00:04.685 INFO:teuthology.orchestra.run.smithi107.stdout:require_osd_release squid 2024-03-08T21:00:04.685 INFO:teuthology.orchestra.run.smithi107.stdout:stretch_mode_enabled false 2024-03-08T21:00:04.685 INFO:teuthology.orchestra.run.smithi107.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-03-08T21:00:04.685 INFO:teuthology.orchestra.run.smithi107.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 85 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-03-08T21:00:04.686 INFO:teuthology.orchestra.run.smithi107.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 86 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.50 2024-03-08T21:00:04.686 INFO:teuthology.orchestra.run.smithi107.stdout:max_osd 12 2024-03-08T21:00:04.686 INFO:teuthology.orchestra.run.smithi107.stdout:osd.0 up in weight 1 up_from 8 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.107:6802/2453946095,v1:172.21.15.107:6803/2453946095] [v2:172.21.15.107:6804/2453946095,v1:172.21.15.107:6805/2453946095] exists,up 8b1ddc64-4d20-4e85-ac66-f1c5ac599501 2024-03-08T21:00:04.686 INFO:teuthology.orchestra.run.smithi107.stdout:osd.1 up in weight 1 up_from 13 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.149:6800/3071316419,v1:172.21.15.149:6801/3071316419] [v2:172.21.15.149:6802/3071316419,v1:172.21.15.149:6803/3071316419] exists,up 5a803c09-202a-4777-ba2a-1da8eaf26a54 2024-03-08T21:00:04.686 INFO:teuthology.orchestra.run.smithi107.stdout:osd.2 up in weight 1 up_from 18 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.204:6800/1087852630,v1:172.21.15.204:6801/1087852630] [v2:172.21.15.204:6802/1087852630,v1:172.21.15.204:6803/1087852630] exists,up 555c634f-347e-47dd-a943-e50ae2be073c 2024-03-08T21:00:04.686 INFO:teuthology.orchestra.run.smithi107.stdout:osd.3 up in weight 1 up_from 25 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.107:6810/3123579789,v1:172.21.15.107:6811/3123579789] [v2:172.21.15.107:6812/3123579789,v1:172.21.15.107:6813/3123579789] exists,up 9e433d97-e202-48e0-9ed1-f0988fa14307 2024-03-08T21:00:04.686 INFO:teuthology.orchestra.run.smithi107.stdout:osd.4 up in weight 1 up_from 33 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.149:6808/726286918,v1:172.21.15.149:6809/726286918] [v2:172.21.15.149:6810/726286918,v1:172.21.15.149:6811/726286918] exists,up 27c53794-9d2b-487d-bfcf-ac4fd2ab061e 2024-03-08T21:00:04.686 INFO:teuthology.orchestra.run.smithi107.stdout:osd.5 up in weight 1 up_from 39 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.204:6808/3143887820,v1:172.21.15.204:6809/3143887820] [v2:172.21.15.204:6810/3143887820,v1:172.21.15.204:6811/3143887820] exists,up 3a7778fb-46c5-40c0-842e-e988c45ea24d 2024-03-08T21:00:04.686 INFO:teuthology.orchestra.run.smithi107.stdout:osd.6 up in weight 1 up_from 47 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.107:6818/2912452950,v1:172.21.15.107:6819/2912452950] [v2:172.21.15.107:6820/2912452950,v1:172.21.15.107:6821/2912452950] exists,up d2e05a56-be4f-4435-9b76-ff9aa01e4ac3 2024-03-08T21:00:04.686 INFO:teuthology.orchestra.run.smithi107.stdout:osd.7 up in weight 1 up_from 52 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.149:6816/3835863809,v1:172.21.15.149:6817/3835863809] [v2:172.21.15.149:6818/3835863809,v1:172.21.15.149:6819/3835863809] exists,up a7dd26ee-c118-4a9a-9e19-142c2680902d 2024-03-08T21:00:04.687 INFO:teuthology.orchestra.run.smithi107.stdout:osd.8 up in weight 1 up_from 60 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.204:6816/1010601477,v1:172.21.15.204:6817/1010601477] [v2:172.21.15.204:6818/1010601477,v1:172.21.15.204:6819/1010601477] exists,up baaaa8dd-0e1f-4bae-86bf-9e429d985f23 2024-03-08T21:00:04.687 INFO:teuthology.orchestra.run.smithi107.stdout:osd.9 up in weight 1 up_from 68 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.107:6826/3334759819,v1:172.21.15.107:6827/3334759819] [v2:172.21.15.107:6828/3334759819,v1:172.21.15.107:6829/3334759819] exists,up 8df80dfa-e7ff-410e-824c-b1dce111b02f 2024-03-08T21:00:04.687 INFO:teuthology.orchestra.run.smithi107.stdout:osd.10 up in weight 1 up_from 73 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.149:6824/3469284631,v1:172.21.15.149:6825/3469284631] [v2:172.21.15.149:6826/3469284631,v1:172.21.15.149:6827/3469284631] exists,up 338945b2-20c0-40b2-aaca-062cd4adee04 2024-03-08T21:00:04.687 INFO:teuthology.orchestra.run.smithi107.stdout:osd.11 up in weight 1 up_from 81 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.204:6824/3097772473,v1:172.21.15.204:6825/3097772473] [v2:172.21.15.204:6826/3097772473,v1:172.21.15.204:6827/3097772473] exists,up acccfe4f-d307-4a64-8f4d-fba27bfbe09d 2024-03-08T21:00:04.687 INFO:teuthology.orchestra.run.smithi107.stdout:pg_upmap_items 2.b [8,11] 2024-03-08T21:00:04.687 INFO:teuthology.orchestra.run.smithi107.stdout:pg_upmap_items 2.c [2,4] 2024-03-08T21:00:04.687 INFO:teuthology.orchestra.run.smithi107.stdout:pg_upmap_items 2.f [8,11] 2024-03-08T21:00:04.687 INFO:teuthology.orchestra.run.smithi107.stdout:pg_upmap_items 2.14 [2,4] 2024-03-08T21:00:04.687 INFO:teuthology.orchestra.run.smithi107.stdout:pg_upmap_items 3.0 [4,6] 2024-03-08T21:00:04.687 INFO:teuthology.orchestra.run.smithi107.stdout:pg_upmap_items 3.11 [5,9] 2024-03-08T21:00:04.687 INFO:teuthology.orchestra.run.smithi107.stdout:pg_upmap_items 3.17 [4,8] 2024-03-08T21:00:04.687 INFO:teuthology.orchestra.run.smithi107.stdout:pg_upmap_items 3.3b [3,9] 2024-03-08T21:00:04.687 INFO:teuthology.orchestra.run.smithi107.stdout:pg_upmap_items 3.3c [4,9] 2024-03-08T21:00:04.687 INFO:teuthology.orchestra.run.smithi107.stdout:blocklist 172.21.15.107:0/1514666962 expires 2024-03-09T20:48:20.149406+0000 2024-03-08T21:00:04.688 INFO:teuthology.orchestra.run.smithi107.stdout:blocklist 172.21.15.107:0/1886016707 expires 2024-03-09T20:48:20.149406+0000 2024-03-08T21:00:04.688 INFO:teuthology.orchestra.run.smithi107.stdout:blocklist 172.21.15.107:6801/426372446 expires 2024-03-09T20:48:20.149406+0000 2024-03-08T21:00:04.688 INFO:teuthology.orchestra.run.smithi107.stdout:blocklist 172.21.15.107:6800/426372446 expires 2024-03-09T20:48:20.149406+0000 2024-03-08T21:00:04.688 INFO:teuthology.orchestra.run.smithi107.stdout:blocklist 172.21.15.107:0/2392547434 expires 2024-03-09T20:47:45.706790+0000 2024-03-08T21:00:04.688 INFO:teuthology.orchestra.run.smithi107.stdout:blocklist 172.21.15.107:0/1206995985 expires 2024-03-09T20:47:45.706790+0000 2024-03-08T21:00:04.688 INFO:teuthology.orchestra.run.smithi107.stdout:blocklist 172.21.15.107:6800/3549936519 expires 2024-03-09T20:47:45.706790+0000 2024-03-08T21:00:04.688 INFO:teuthology.orchestra.run.smithi107.stdout:blocklist 172.21.15.107:6801/3549936519 expires 2024-03-09T20:47:45.706790+0000 2024-03-08T21:00:04.688 INFO:teuthology.orchestra.run.smithi107.stdout:blocklist 172.21.15.107:0/2482452215 expires 2024-03-09T20:47:45.706790+0000 2024-03-08T21:00:04.688 INFO:teuthology.orchestra.run.smithi107.stdout:blocklist 172.21.15.107:0/808124991 expires 2024-03-09T20:48:20.149406+0000 2024-03-08T21:00:04.689 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:04.704+0000 7f6efde00700 1 -- 172.21.15.107:0/3660256740 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f6edc0a2910 msgr2=0x7f6edc0a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:00:04.689 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:04.704+0000 7f6efde00700 1 --2- 172.21.15.107:0/3660256740 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f6edc0a2910 0x7f6edc0a4dd0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f6ee8009bf0 tx=0x7f6ee8009260 comp rx=0 tx=0).stop 2024-03-08T21:00:04.689 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:04.704+0000 7f6efde00700 1 -- 172.21.15.107:0/3660256740 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6ef8102fb0 msgr2=0x7f6ef80741d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:00:04.690 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:04.704+0000 7f6efde00700 1 --2- 172.21.15.107:0/3660256740 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6ef8102fb0 0x7f6ef80741d0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f6ee400d950 tx=0x7f6ee400dd10 comp rx=0 tx=0).stop 2024-03-08T21:00:04.690 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:04.705+0000 7f6efde00700 1 -- 172.21.15.107:0/3660256740 shutdown_connections 2024-03-08T21:00:04.690 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:04.705+0000 7f6efde00700 1 --2- 172.21.15.107:0/3660256740 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6ef8101e50 0x7f6ef8075b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:04.690 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:04.705+0000 7f6efde00700 1 --2- 172.21.15.107:0/3660256740 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f6edc0a2910 0x7f6edc0a4dd0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:04.690 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:04.705+0000 7f6efde00700 1 --2- 172.21.15.107:0/3660256740 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6ef8103970 0x7f6ef8074710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:04.690 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:04.705+0000 7f6efde00700 1 --2- 172.21.15.107:0/3660256740 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6ef8102fb0 0x7f6ef80741d0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:04.690 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:04.705+0000 7f6efde00700 1 -- 172.21.15.107:0/3660256740 >> 172.21.15.107:0/3660256740 conn(0x7f6ef80fd430 msgr2=0x7f6ef81062e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:00:04.690 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:04.705+0000 7f6efde00700 1 -- 172.21.15.107:0/3660256740 shutdown_connections 2024-03-08T21:00:04.690 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:04.705+0000 7f6efde00700 1 -- 172.21.15.107:0/3660256740 wait complete. 2024-03-08T21:00:05.495 INFO:teuthology.run_tasks:Running task fs.ready... 2024-03-08T21:00:05.507 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-08T21:00:05.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:00:05 smithi107 ceph-mon[31381]: pgmap v430: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-03-08T21:00:05.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:00:05 smithi107 ceph-mon[31381]: from='client.? 172.21.15.107:0/3660256740' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-03-08T21:00:05.728 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:05.745+0000 7fe34be00640 1 -- 172.21.15.107:0/1871758919 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe344101310 msgr2=0x7fe344101730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:00:05.728 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:05.745+0000 7fe34be00640 1 --2- 172.21.15.107:0/1871758919 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe344101310 0x7fe344101730 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fe338009900 tx=0x7fe33802bc30 comp rx=0 tx=0).stop 2024-03-08T21:00:05.729 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:05.745+0000 7fe34be00640 1 -- 172.21.15.107:0/1871758919 shutdown_connections 2024-03-08T21:00:05.729 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:05.745+0000 7fe34be00640 1 --2- 172.21.15.107:0/1871758919 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe344102ec0 0x7fe344109750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:05.729 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:05.745+0000 7fe34be00640 1 --2- 172.21.15.107:0/1871758919 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe344102500 0x7fe344102980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:05.729 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:05.745+0000 7fe34be00640 1 --2- 172.21.15.107:0/1871758919 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe344101310 0x7fe344101730 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:05.729 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:05.745+0000 7fe34be00640 1 -- 172.21.15.107:0/1871758919 >> 172.21.15.107:0/1871758919 conn(0x7fe3440fc8a0 msgr2=0x7fe3440fecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:00:05.729 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:05.745+0000 7fe34be00640 1 -- 172.21.15.107:0/1871758919 shutdown_connections 2024-03-08T21:00:05.729 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:05.745+0000 7fe34be00640 1 -- 172.21.15.107:0/1871758919 wait complete. 2024-03-08T21:00:05.731 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:05.746+0000 7fe34be00640 1 Processor -- start 2024-03-08T21:00:05.731 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:05.746+0000 7fe34be00640 1 -- start start 2024-03-08T21:00:05.731 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:05.746+0000 7fe34be00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe344101310 0x7fe344197590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:00:05.732 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:05.746+0000 7fe34be00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe344102500 0x7fe344197ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:00:05.732 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:05.746+0000 7fe34be00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe344102ec0 0x7fe34419eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:00:05.732 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:05.746+0000 7fe34be00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe34410ce20 con 0x7fe344101310 2024-03-08T21:00:05.732 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:05.746+0000 7fe34be00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe34410cca0 con 0x7fe344102500 2024-03-08T21:00:05.732 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:05.746+0000 7fe34be00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe34410cfa0 con 0x7fe344102ec0 2024-03-08T21:00:05.732 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:05.747+0000 7fe349400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe344101310 0x7fe344197590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:00:05.732 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:05.747+0000 7fe349400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe344101310 0x7fe344197590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:36000/0 (socket says 172.21.15.107:36000) 2024-03-08T21:00:05.732 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:05.747+0000 7fe349400640 1 -- 172.21.15.107:0/1564103115 learned_addr learned my addr 172.21.15.107:0/1564103115 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:00:05.732 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:05.747+0000 7fe348a00640 1 --2- 172.21.15.107:0/1564103115 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe344102500 0x7fe344197ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:00:05.732 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:05.747+0000 7fe349400640 1 -- 172.21.15.107:0/1564103115 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe344102ec0 msgr2=0x7fe34419eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:00:05.732 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:05.747+0000 7fe349400640 1 --2- 172.21.15.107:0/1564103115 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe344102ec0 0x7fe34419eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:05.732 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:05.747+0000 7fe349400640 1 -- 172.21.15.107:0/1564103115 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe344102500 msgr2=0x7fe344197ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:00:05.732 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:05.747+0000 7fe349400640 1 --2- 172.21.15.107:0/1564103115 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe344102500 0x7fe344197ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:05.732 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:05.747+0000 7fe349400640 1 -- 172.21.15.107:0/1564103115 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe34419f1b0 con 0x7fe344101310 2024-03-08T21:00:05.733 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:05.747+0000 7fe348a00640 1 --2- 172.21.15.107:0/1564103115 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe344102500 0x7fe344197ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T21:00:05.733 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:05.747+0000 7fe349400640 1 --2- 172.21.15.107:0/1564103115 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe344101310 0x7fe344197590 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fe338036040 tx=0x7fe3380042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:00:05.733 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:05.748+0000 7fe342000640 1 -- 172.21.15.107:0/1564103115 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe338043020 con 0x7fe344101310 2024-03-08T21:00:05.733 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:05.748+0000 7fe342000640 1 -- 172.21.15.107:0/1564103115 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fe338004aa0 con 0x7fe344101310 2024-03-08T21:00:05.733 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:05.748+0000 7fe342000640 1 -- 172.21.15.107:0/1564103115 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe338003de0 con 0x7fe344101310 2024-03-08T21:00:05.733 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:05.748+0000 7fe34be00640 1 -- 172.21.15.107:0/1564103115 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe34419f440 con 0x7fe344101310 2024-03-08T21:00:05.733 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:05.748+0000 7fe34be00640 1 -- 172.21.15.107:0/1564103115 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe34419f990 con 0x7fe344101310 2024-03-08T21:00:05.733 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:05.749+0000 7fe34be00640 1 -- 172.21.15.107:0/1564103115 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3441086a0 con 0x7fe344101310 2024-03-08T21:00:05.734 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:05.751+0000 7fe342000640 1 -- 172.21.15.107:0/1564103115 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) v1 ==== 137291+0+0 (secure 0 0 0) 0x7fe338046020 con 0x7fe344101310 2024-03-08T21:00:05.735 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:05.752+0000 7fe342000640 1 --2- 172.21.15.107:0/1564103115 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fe32c0a2800 0x7fe32c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:00:05.735 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:05.752+0000 7fe342000640 1 -- 172.21.15.107:0/1564103115 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fe3380f0f20 con 0x7fe344101310 2024-03-08T21:00:05.735 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:05.753+0000 7fe348a00640 1 --2- 172.21.15.107:0/1564103115 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fe32c0a2800 0x7fe32c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:00:05.736 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:05.753+0000 7fe348a00640 1 --2- 172.21.15.107:0/1564103115 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fe32c0a2800 0x7fe32c0a4cc0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fe344198aa0 tx=0x7fe334005d30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:00:05.739 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:05.757+0000 7fe342000640 1 -- 172.21.15.107:0/1564103115 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7fe33804d020 con 0x7fe344101310 2024-03-08T21:00:05.764 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:00:05 smithi149 ceph-mon[35042]: pgmap v430: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-03-08T21:00:05.764 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:00:05 smithi149 ceph-mon[35042]: from='client.? 172.21.15.107:0/3660256740' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-03-08T21:00:05.878 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:00:05 smithi204 ceph-mon[34935]: pgmap v430: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-03-08T21:00:05.878 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:00:05 smithi204 ceph-mon[34935]: from='client.? 172.21.15.107:0/3660256740' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-03-08T21:00:05.943 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:05.960+0000 7fe34be00640 1 -- 172.21.15.107:0/1564103115 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe344061f90 con 0x7fe344101310 2024-03-08T21:00:05.948 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:05.965+0000 7fe342000640 1 -- 172.21.15.107:0/1564103115 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+22779 (secure 0 0 0) 0x7fe3380bdfc0 con 0x7fe344101310 2024-03-08T21:00:05.948 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-08T21:00:05.948 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":88,"fsid":"dc20f0f8-dd8c-11ee-95c6-87774f69a715","created":"2024-03-08T20:47:04.190034+0000","modified":"2024-03-08T20:58:21.449936+0000","last_up_change":"2024-03-08T20:56:43.135514+0000","last_in_change":"2024-03-08T20:56:17.577074+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-08T20:51:32.832786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-08T20:57:48.721227+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-08T20:57:49.707779+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8b1ddc64-4d20-4e85-ac66-f1c5ac599501","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":2453946095},{"type":"v1","addr":"172.21.15.107:6803","nonce":2453946095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":2453946095},{"type":"v1","addr":"172.21.15.107:6805","nonce":2453946095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":2453946095},{"type":"v1","addr":"172.21.15.107:6809","nonce":2453946095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":2453946095},{"type":"v1","addr":"172.21.15.107:6807","nonce":2453946095}]},"public_addr":"172.21.15.107:6803/2453946095","cluster_addr":"172.21.15.107:6805/2453946095","heartbeat_back_addr":"172.21.15.107:6809/2453946095","heartbeat_front_addr":"172.21.15.107:6807/2453946095","state":["exists","up"]},{"osd":1,"uuid":"5a803c09-202a-4777-ba2a-1da8eaf26a54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":3071316419},{"type":"v1","addr":"172.21.15.149:6801","nonce":3071316419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":3071316419},{"type":"v1","addr":"172.21.15.149:6803","nonce":3071316419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":3071316419},{"type":"v1","addr":"172.21.15.149:6807","nonce":3071316419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":3071316419},{"type":"v1","addr":"172.21.15.149:6805","nonce":3071316419}]},"public_addr":"172.21.15.149:6801/3071316419","cluster_addr":"172.21.15.149:6803/3071316419","heartbeat_back_addr":"172.21.15.149:6807/3071316419","heartbeat_front_addr":"172.21.15.149:6805/3071316419","state":["exists","up"]},{"osd":2,"uuid":"555c634f-347e-47dd-a943-e50ae2be073c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":1087852630},{"type":"v1","addr":"172.21.15.204:6801","nonce":1087852630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":1087852630},{"type":"v1","addr":"172.21.15.204:6803","nonce":1087852630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":1087852630},{"type":"v1","addr":"172.21.15.204:6807","nonce":1087852630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":1087852630},{"type":"v1","addr":"172.21.15.204:6805","nonce":1087852630}]},"public_addr":"172.21.15.204:6801/1087852630","cluster_addr":"172.21.15.204:6803/1087852630","heartbeat_back_addr":"172.21.15.204:6807/1087852630","heartbeat_front_addr":"172.21.15.204:6805/1087852630","state":["exists","up"]},{"osd":3,"uuid":"9e433d97-e202-48e0-9ed1-f0988fa14307","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":3123579789},{"type":"v1","addr":"172.21.15.107:6811","nonce":3123579789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":3123579789},{"type":"v1","addr":"172.21.15.107:6813","nonce":3123579789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":3123579789},{"type":"v1","addr":"172.21.15.107:6817","nonce":3123579789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":3123579789},{"type":"v1","addr":"172.21.15.107:6815","nonce":3123579789}]},"public_addr":"172.21.15.107:6811/3123579789","cluster_addr":"172.21.15.107:6813/3123579789","heartbeat_back_addr":"172.21.15.107:6817/3123579789","heartbeat_front_addr":"172.21.15.107:6815/3123579789","state":["exists","up"]},{"osd":4,"uuid":"27c53794-9d2b-487d-bfcf-ac4fd2ab061e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":726286918},{"type":"v1","addr":"172.21.15.149:6809","nonce":726286918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":726286918},{"type":"v1","addr":"172.21.15.149:6811","nonce":726286918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":726286918},{"type":"v1","addr":"172.21.15.149:6815","nonce":726286918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":726286918},{"type":"v1","addr":"172.21.15.149:6813","nonce":726286918}]},"public_addr":"172.21.15.149:6809/726286918","cluster_addr":"172.21.15.149:6811/726286918","heartbeat_back_addr":"172.21.15.149:6815/726286918","heartbeat_front_addr":"172.21.15.149:6813/726286918","state":["exists","up"]},{"osd":5,"uuid":"3a7778fb-46c5-40c0-842e-e988c45ea24d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":3143887820},{"type":"v1","addr":"172.21.15.204:6809","nonce":3143887820}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":3143887820},{"type":"v1","addr":"172.21.15.204:6811","nonce":3143887820}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":3143887820},{"type":"v1","addr":"172.21.15.204:6815","nonce":3143887820}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":3143887820},{"type":"v1","addr":"172.21.15.204:6813","nonce":3143887820}]},"public_addr":"172.21.15.204:6809/3143887820","cluster_addr":"172.21.15.204:6811/3143887820","heartbeat_back_addr":"172.21.15.204:6815/3143887820","heartbeat_front_addr":"172.21.15.204:6813/3143887820","state":["exists","up"]},{"osd":6,"uuid":"d2e05a56-be4f-4435-9b76-ff9aa01e4ac3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":2912452950},{"type":"v1","addr":"172.21.15.107:6819","nonce":2912452950}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":2912452950},{"type":"v1","addr":"172.21.15.107:6821","nonce":2912452950}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":2912452950},{"type":"v1","addr":"172.21.15.107:6825","nonce":2912452950}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":2912452950},{"type":"v1","addr":"172.21.15.107:6823","nonce":2912452950}]},"public_addr":"172.21.15.107:6819/2912452950","cluster_addr":"172.21.15.107:6821/2912452950","heartbeat_back_addr":"172.21.15.107:6825/2912452950","heartbeat_front_addr":"172.21.15.107:6823/2912452950","state":["exists","up"]},{"osd":7,"uuid":"a7dd26ee-c118-4a9a-9e19-142c2680902d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":3835863809},{"type":"v1","addr":"172.21.15.149:6817","nonce":3835863809}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":3835863809},{"type":"v1","addr":"172.21.15.149:6819","nonce":3835863809}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":3835863809},{"type":"v1","addr":"172.21.15.149:6823","nonce":3835863809}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":3835863809},{"type":"v1","addr":"172.21.15.149:6821","nonce":3835863809}]},"public_addr":"172.21.15.149:6817/3835863809","cluster_addr":"172.21.15.149:6819/3835863809","heartbeat_back_addr":"172.21.15.149:6823/3835863809","heartbeat_front_addr":"172.21.15.149:6821/3835863809","state":["exists","up"]},{"osd":8,"uuid":"baaaa8dd-0e1f-4bae-86bf-9e429d985f23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":1010601477},{"type":"v1","addr":"172.21.15.204:6817","nonce":1010601477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":1010601477},{"type":"v1","addr":"172.21.15.204:6819","nonce":1010601477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":1010601477},{"type":"v1","addr":"172.21.15.204:6823","nonce":1010601477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":1010601477},{"type":"v1","addr":"172.21.15.204:6821","nonce":1010601477}]},"public_addr":"172.21.15.204:6817/1010601477","cluster_addr":"172.21.15.204:6819/1010601477","heartbeat_back_addr":"172.21.15.204:6823/1010601477","heartbeat_front_addr":"172.21.15.204:6821/1010601477","state":["exists","up"]},{"osd":9,"uuid":"8df80dfa-e7ff-410e-824c-b1dce111b02f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":3334759819},{"type":"v1","addr":"172.21.15.107:6827","nonce":3334759819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3334759819},{"type":"v1","addr":"172.21.15.107:6829","nonce":3334759819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":3334759819},{"type":"v1","addr":"172.21.15.107:6833","nonce":3334759819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":3334759819},{"type":"v1","addr":"172.21.15.107:6831","nonce":3334759819}]},"public_addr":"172.21.15.107:6827/3334759819","cluster_addr":"172.21.15.107:6829/3334759819","heartbeat_back_addr":"172.21.15.107:6833/3334759819","heartbeat_front_addr":"172.21.15.107:6831/3334759819","state":["exists","up"]},{"osd":10,"uuid":"338945b2-20c0-40b2-aaca-062cd4adee04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":3469284631},{"type":"v1","addr":"172.21.15.149:6825","nonce":3469284631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":3469284631},{"type":"v1","addr":"172.21.15.149:6827","nonce":3469284631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":3469284631},{"type":"v1","addr":"172.21.15.149:6831","nonce":3469284631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":3469284631},{"type":"v1","addr":"172.21.15.149:6829","nonce":3469284631}]},"public_addr":"172.21.15.149:6825/3469284631","cluster_addr":"172.21.15.149:6827/3469284631","heartbeat_back_addr":"172.21.15.149:6831/3469284631","heartbeat_front_addr":"172.21.15.149:6829/3469284631","state":["exists","up"]},{"osd":11,"uuid":"acccfe4f-d307-4a64-8f4d-fba27bfbe09d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":3097772473},{"type":"v1","addr":"172.21.15.204:6825","nonce":3097772473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":3097772473},{"type":"v1","addr":"172.21.15.204:6827","nonce":3097772473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":3097772473},{"type":"v1","addr":"172.21.15.204:6831","nonce":3097772473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":3097772473},{"type":"v1","addr":"172.21.15.204:6829","nonce":3097772473}]},"public_addr":"172.21.15.204:6825/3097772473","cluster_addr":"172.21.15.204:6827/3097772473","heartbeat_back_addr":"172.21.15.204:6831/3097772473","heartbeat_front_addr":"172.21.15.204:6829/3097772473","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T20:50:25.862753+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T20:50:58.167663+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T20:51:30.312816+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T20:52:05.487917+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T20:52:39.348354+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T20:53:12.899002+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T20:53:47.630837+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T20:54:21.444552+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T20:54:57.101160+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T20:55:31.841786+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T20:56:05.827161+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T20:56:40.611097+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.0","mappings":[{"from":4,"to":6}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.3b","mappings":[{"from":3,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.107:0/1514666962":"2024-03-09T20:48:20.149406+0000","172.21.15.107:0/1886016707":"2024-03-09T20:48:20.149406+0000","172.21.15.107:6801/426372446":"2024-03-09T20:48:20.149406+0000","172.21.15.107:6800/426372446":"2024-03-09T20:48:20.149406+0000","172.21.15.107:0/2392547434":"2024-03-09T20:47:45.706790+0000","172.21.15.107:0/1206995985":"2024-03-09T20:47:45.706790+0000","172.21.15.107:6800/3549936519":"2024-03-09T20:47:45.706790+0000","172.21.15.107:6801/3549936519":"2024-03-09T20:47:45.706790+0000","172.21.15.107:0/2482452215":"2024-03-09T20:47:45.706790+0000","172.21.15.107:0/808124991":"2024-03-09T20:48:20.149406+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-08T21:00:05.951 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:05.968+0000 7fe34be00640 1 -- 172.21.15.107:0/1564103115 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fe32c0a2800 msgr2=0x7fe32c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:00:05.951 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:05.968+0000 7fe34be00640 1 --2- 172.21.15.107:0/1564103115 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fe32c0a2800 0x7fe32c0a4cc0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fe344198aa0 tx=0x7fe334005d30 comp rx=0 tx=0).stop 2024-03-08T21:00:05.951 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:05.968+0000 7fe34be00640 1 -- 172.21.15.107:0/1564103115 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe344101310 msgr2=0x7fe344197590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:00:05.951 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:05.968+0000 7fe34be00640 1 --2- 172.21.15.107:0/1564103115 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe344101310 0x7fe344197590 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fe338036040 tx=0x7fe3380042f0 comp rx=0 tx=0).stop 2024-03-08T21:00:05.951 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:05.968+0000 7fe34be00640 1 -- 172.21.15.107:0/1564103115 shutdown_connections 2024-03-08T21:00:05.951 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:05.968+0000 7fe34be00640 1 --2- 172.21.15.107:0/1564103115 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fe32c0a2800 0x7fe32c0a4cc0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:05.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:05.968+0000 7fe34be00640 1 --2- 172.21.15.107:0/1564103115 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe344102ec0 0x7fe34419eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:05.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:05.968+0000 7fe34be00640 1 --2- 172.21.15.107:0/1564103115 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe344102500 0x7fe344197ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:05.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:05.968+0000 7fe34be00640 1 --2- 172.21.15.107:0/1564103115 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe344101310 0x7fe344197590 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:05.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:05.968+0000 7fe34be00640 1 -- 172.21.15.107:0/1564103115 >> 172.21.15.107:0/1564103115 conn(0x7fe3440fc8a0 msgr2=0x7fe344105870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:00:05.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:05.968+0000 7fe34be00640 1 -- 172.21.15.107:0/1564103115 shutdown_connections 2024-03-08T21:00:05.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:05.968+0000 7fe34be00640 1 -- 172.21.15.107:0/1564103115 wait complete. 2024-03-08T21:00:05.971 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-08T20:51:32.832786+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-08T20:57:48.721227+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-08T20:57:49.707779+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-08T21:00:05.972 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-08T21:00:06.192 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:06.208+0000 7f9eebc00640 1 -- 172.21.15.107:0/2659880034 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9ee4103530 msgr2=0x7f9ee4103950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:00:06.192 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:06.208+0000 7f9eebc00640 1 --2- 172.21.15.107:0/2659880034 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9ee4103530 0x7f9ee4103950 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f9ed4009900 tx=0x7f9ed402bc30 comp rx=0 tx=0).stop 2024-03-08T21:00:06.192 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:06.208+0000 7f9eebc00640 1 -- 172.21.15.107:0/2659880034 shutdown_connections 2024-03-08T21:00:06.192 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:06.208+0000 7f9eebc00640 1 --2- 172.21.15.107:0/2659880034 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9ee41050e0 0x7f9ee410b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:06.192 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:06.208+0000 7f9eebc00640 1 --2- 172.21.15.107:0/2659880034 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9ee4104720 0x7f9ee4104ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:06.192 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:06.208+0000 7f9eebc00640 1 --2- 172.21.15.107:0/2659880034 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9ee4103530 0x7f9ee4103950 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:06.192 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:06.208+0000 7f9eebc00640 1 -- 172.21.15.107:0/2659880034 >> 172.21.15.107:0/2659880034 conn(0x7f9ee40feaa0 msgr2=0x7f9ee4100ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:00:06.192 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:06.209+0000 7f9eebc00640 1 -- 172.21.15.107:0/2659880034 shutdown_connections 2024-03-08T21:00:06.192 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:06.209+0000 7f9eebc00640 1 -- 172.21.15.107:0/2659880034 wait complete. 2024-03-08T21:00:06.195 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:06.209+0000 7f9eebc00640 1 Processor -- start 2024-03-08T21:00:06.195 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:06.210+0000 7f9eebc00640 1 -- start start 2024-03-08T21:00:06.195 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:06.210+0000 7f9eebc00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9ee4103530 0x7f9ee4199780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:00:06.195 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:06.210+0000 7f9eebc00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9ee4104720 0x7f9ee4199cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:00:06.195 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:06.210+0000 7f9eebc00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9ee41050e0 0x7f9ee41a0d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:00:06.195 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:06.210+0000 7f9eebc00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ee410f070 con 0x7f9ee4103530 2024-03-08T21:00:06.195 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:06.210+0000 7f9eebc00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ee410eef0 con 0x7f9ee4104720 2024-03-08T21:00:06.195 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:06.210+0000 7f9eebc00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ee410f1f0 con 0x7f9ee41050e0 2024-03-08T21:00:06.195 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:06.210+0000 7f9ee9200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9ee4103530 0x7f9ee4199780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:00:06.195 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:06.210+0000 7f9ee9200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9ee4103530 0x7f9ee4199780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:56476/0 (socket says 172.21.15.107:56476) 2024-03-08T21:00:06.196 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:06.210+0000 7f9ee9200640 1 -- 172.21.15.107:0/1225049613 learned_addr learned my addr 172.21.15.107:0/1225049613 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:00:06.196 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:06.210+0000 7f9ee9c00640 1 --2- 172.21.15.107:0/1225049613 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9ee41050e0 0x7f9ee41a0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:00:06.196 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:06.210+0000 7f9edbe00640 1 --2- 172.21.15.107:0/1225049613 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9ee4104720 0x7f9ee4199cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:00:06.196 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:06.211+0000 7f9ee9200640 1 -- 172.21.15.107:0/1225049613 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9ee41050e0 msgr2=0x7f9ee41a0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:00:06.196 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:06.211+0000 7f9ee9200640 1 --2- 172.21.15.107:0/1225049613 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9ee41050e0 0x7f9ee41a0d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:06.196 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:06.211+0000 7f9ee9200640 1 -- 172.21.15.107:0/1225049613 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9ee4104720 msgr2=0x7f9ee4199cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:00:06.196 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:06.211+0000 7f9ee9200640 1 --2- 172.21.15.107:0/1225049613 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9ee4104720 0x7f9ee4199cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:06.196 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:06.211+0000 7f9ee9200640 1 -- 172.21.15.107:0/1225049613 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ee41a13a0 con 0x7f9ee4103530 2024-03-08T21:00:06.196 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:06.211+0000 7f9ee9c00640 1 --2- 172.21.15.107:0/1225049613 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9ee41050e0 0x7f9ee41a0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T21:00:06.196 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:06.211+0000 7f9edbe00640 1 --2- 172.21.15.107:0/1225049613 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9ee4104720 0x7f9ee4199cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T21:00:06.196 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:06.211+0000 7f9ee9200640 1 --2- 172.21.15.107:0/1225049613 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9ee4103530 0x7f9ee4199780 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f9ed4000c00 tx=0x7f9ed4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:00:06.196 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:06.211+0000 7f9ee2000640 1 -- 172.21.15.107:0/1225049613 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ed4043020 con 0x7f9ee4103530 2024-03-08T21:00:06.196 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:06.211+0000 7f9ee2000640 1 -- 172.21.15.107:0/1225049613 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f9ed4004aa0 con 0x7f9ee4103530 2024-03-08T21:00:06.196 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:06.211+0000 7f9eebc00640 1 -- 172.21.15.107:0/1225049613 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ee41a1630 con 0x7f9ee4103530 2024-03-08T21:00:06.197 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:06.211+0000 7f9ee2000640 1 -- 172.21.15.107:0/1225049613 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ed4003d00 con 0x7f9ee4103530 2024-03-08T21:00:06.197 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:06.211+0000 7f9eebc00640 1 -- 172.21.15.107:0/1225049613 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ee41a1bd0 con 0x7f9ee4103530 2024-03-08T21:00:06.197 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:06.213+0000 7f9eebc00640 1 -- 172.21.15.107:0/1225049613 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ee410a8c0 con 0x7f9ee4103530 2024-03-08T21:00:06.197 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:06.214+0000 7f9ee2000640 1 -- 172.21.15.107:0/1225049613 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f9ed4046020 con 0x7f9ee4103530 2024-03-08T21:00:06.199 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:06.216+0000 7f9ee2000640 1 --2- 172.21.15.107:0/1225049613 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f9ec80a2880 0x7f9ec80a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:00:06.199 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:06.216+0000 7f9ee2000640 1 -- 172.21.15.107:0/1225049613 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f9ed40f1060 con 0x7f9ee4103530 2024-03-08T21:00:06.199 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:06.216+0000 7f9edbe00640 1 --2- 172.21.15.107:0/1225049613 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f9ec80a2880 0x7f9ec80a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:00:06.199 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:06.217+0000 7f9edbe00640 1 --2- 172.21.15.107:0/1225049613 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f9ec80a2880 0x7f9ec80a4d40 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f9ee419ac90 tx=0x7f9ed0005e50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:00:06.203 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:06.220+0000 7f9ee2000640 1 -- 172.21.15.107:0/1225049613 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f9ed404d020 con 0x7f9ee4103530 2024-03-08T21:00:06.401 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:06.418+0000 7f9eebc00640 1 -- 172.21.15.107:0/1225049613 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9ee4061ef0 con 0x7f9ee4103530 2024-03-08T21:00:06.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:06.419+0000 7f9ee2000640 1 -- 172.21.15.107:0/1225049613 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9ed40be100 con 0x7f9ee4103530 2024-03-08T21:00:06.402 INFO:teuthology.orchestra.run.smithi107.stdout:pg_num: 1 2024-03-08T21:00:06.405 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:06.421+0000 7f9eebc00640 1 -- 172.21.15.107:0/1225049613 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f9ec80a2880 msgr2=0x7f9ec80a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:00:06.405 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:06.421+0000 7f9eebc00640 1 --2- 172.21.15.107:0/1225049613 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f9ec80a2880 0x7f9ec80a4d40 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f9ee419ac90 tx=0x7f9ed0005e50 comp rx=0 tx=0).stop 2024-03-08T21:00:06.405 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:06.421+0000 7f9eebc00640 1 -- 172.21.15.107:0/1225049613 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9ee4103530 msgr2=0x7f9ee4199780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:00:06.405 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:06.421+0000 7f9eebc00640 1 --2- 172.21.15.107:0/1225049613 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9ee4103530 0x7f9ee4199780 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f9ed4000c00 tx=0x7f9ed4004270 comp rx=0 tx=0).stop 2024-03-08T21:00:06.405 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:06.422+0000 7f9eebc00640 1 -- 172.21.15.107:0/1225049613 shutdown_connections 2024-03-08T21:00:06.405 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:06.422+0000 7f9eebc00640 1 --2- 172.21.15.107:0/1225049613 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f9ec80a2880 0x7f9ec80a4d40 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:06.405 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:06.422+0000 7f9eebc00640 1 --2- 172.21.15.107:0/1225049613 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9ee41050e0 0x7f9ee41a0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:06.405 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:06.422+0000 7f9eebc00640 1 --2- 172.21.15.107:0/1225049613 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9ee4104720 0x7f9ee4199cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:06.406 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:06.422+0000 7f9eebc00640 1 --2- 172.21.15.107:0/1225049613 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9ee4103530 0x7f9ee4199780 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:06.406 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:06.422+0000 7f9eebc00640 1 -- 172.21.15.107:0/1225049613 >> 172.21.15.107:0/1225049613 conn(0x7f9ee40feaa0 msgr2=0x7f9ee4107a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:00:06.406 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:06.422+0000 7f9eebc00640 1 -- 172.21.15.107:0/1225049613 shutdown_connections 2024-03-08T21:00:06.406 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:06.422+0000 7f9eebc00640 1 -- 172.21.15.107:0/1225049613 wait complete. 2024-03-08T21:00:06.424 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-08T21:00:06.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:06.651+0000 7f8434e00640 1 -- 172.21.15.107:0/3974700281 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8430107910 msgr2=0x7f8430107d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:00:06.636 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:06.651+0000 7f8434e00640 1 --2- 172.21.15.107:0/3974700281 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8430107910 0x7f8430107d30 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f8420009900 tx=0x7f842002bc30 comp rx=0 tx=0).stop 2024-03-08T21:00:06.636 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:06.651+0000 7f8434e00640 1 -- 172.21.15.107:0/3974700281 shutdown_connections 2024-03-08T21:00:06.636 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:06.651+0000 7f8434e00640 1 --2- 172.21.15.107:0/3974700281 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f84301094c0 0x7f843010fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:06.636 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:06.651+0000 7f8434e00640 1 --2- 172.21.15.107:0/3974700281 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8430108b00 0x7f8430108f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:06.636 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:06.651+0000 7f8434e00640 1 --2- 172.21.15.107:0/3974700281 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8430107910 0x7f8430107d30 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:06.636 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:06.651+0000 7f8434e00640 1 -- 172.21.15.107:0/3974700281 >> 172.21.15.107:0/3974700281 conn(0x7f84300768b0 msgr2=0x7f8430078cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:00:06.636 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:06.651+0000 7f8434e00640 1 -- 172.21.15.107:0/3974700281 shutdown_connections 2024-03-08T21:00:06.636 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:06.651+0000 7f8434e00640 1 -- 172.21.15.107:0/3974700281 wait complete. 2024-03-08T21:00:06.637 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:06.652+0000 7f8434e00640 1 Processor -- start 2024-03-08T21:00:06.637 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:06.652+0000 7f8434e00640 1 -- start start 2024-03-08T21:00:06.639 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:06.653+0000 7f8434e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8430107910 0x7f843019db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:00:06.639 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:06.653+0000 7f8434e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8430108b00 0x7f843019e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:00:06.639 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:06.653+0000 7f8434e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f84301094c0 0x7f84301a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:00:06.639 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:06.653+0000 7f8434e00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8430113490 con 0x7f8430108b00 2024-03-08T21:00:06.639 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:06.653+0000 7f8434e00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8430113310 con 0x7f84301094c0 2024-03-08T21:00:06.639 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:06.653+0000 7f8434e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8430113610 con 0x7f8430107910 2024-03-08T21:00:06.639 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:06.653+0000 7f8427e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8430108b00 0x7f843019e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:00:06.639 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:06.653+0000 7f8427e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8430108b00 0x7f843019e0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:56502/0 (socket says 172.21.15.107:56502) 2024-03-08T21:00:06.639 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:06.653+0000 7f8427e00640 1 -- 172.21.15.107:0/305019050 learned_addr learned my addr 172.21.15.107:0/305019050 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:00:06.639 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:06.653+0000 7f842de00640 1 --2- 172.21.15.107:0/305019050 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8430107910 0x7f843019db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:00:06.639 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:06.654+0000 7f842e800640 1 --2- 172.21.15.107:0/305019050 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f84301094c0 0x7f84301a5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:00:06.640 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:06.654+0000 7f8427e00640 1 -- 172.21.15.107:0/305019050 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8430107910 msgr2=0x7f843019db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:00:06.640 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:06.654+0000 7f8427e00640 1 --2- 172.21.15.107:0/305019050 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8430107910 0x7f843019db90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:06.640 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:06.654+0000 7f8427e00640 1 -- 172.21.15.107:0/305019050 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f84301094c0 msgr2=0x7f84301a5140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:00:06.640 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:06.654+0000 7f8427e00640 1 --2- 172.21.15.107:0/305019050 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f84301094c0 0x7f84301a5140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:06.640 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:06.654+0000 7f8427e00640 1 -- 172.21.15.107:0/305019050 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84301a57b0 con 0x7f8430108b00 2024-03-08T21:00:06.640 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:06.654+0000 7f842de00640 1 --2- 172.21.15.107:0/305019050 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8430107910 0x7f843019db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T21:00:06.640 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:06.654+0000 7f8427e00640 1 --2- 172.21.15.107:0/305019050 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8430108b00 0x7f843019e0d0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f841800c890 tx=0x7f841800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:00:06.640 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:06.654+0000 7f8426a00640 1 -- 172.21.15.107:0/305019050 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8418004220 con 0x7f8430108b00 2024-03-08T21:00:06.640 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:06.654+0000 7f8426a00640 1 -- 172.21.15.107:0/305019050 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f84180043c0 con 0x7f8430108b00 2024-03-08T21:00:06.640 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:06.654+0000 7f8434e00640 1 -- 172.21.15.107:0/305019050 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84301a5aa0 con 0x7f8430108b00 2024-03-08T21:00:06.640 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:06.654+0000 7f8426a00640 1 -- 172.21.15.107:0/305019050 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8418004a00 con 0x7f8430108b00 2024-03-08T21:00:06.640 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:06.655+0000 7f8434e00640 1 -- 172.21.15.107:0/305019050 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84301a5ff0 con 0x7f8430108b00 2024-03-08T21:00:06.640 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:06.656+0000 7f8434e00640 1 -- 172.21.15.107:0/305019050 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f843010ed00 con 0x7f8430108b00 2024-03-08T21:00:06.641 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:06.658+0000 7f8426a00640 1 -- 172.21.15.107:0/305019050 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f8418004ba0 con 0x7f8430108b00 2024-03-08T21:00:06.642 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:06.659+0000 7f8426a00640 1 --2- 172.21.15.107:0/305019050 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f84140a2800 0x7f84140a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:00:06.643 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:06.660+0000 7f8426a00640 1 -- 172.21.15.107:0/305019050 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f84180ce820 con 0x7f8430108b00 2024-03-08T21:00:06.643 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:06.660+0000 7f842de00640 1 --2- 172.21.15.107:0/305019050 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f84140a2800 0x7f84140a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:00:06.643 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:06.660+0000 7f842de00640 1 --2- 172.21.15.107:0/305019050 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f84140a2800 0x7f84140a4cc0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f842002c240 tx=0x7f8420005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:00:06.646 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:06.663+0000 7f8426a00640 1 -- 172.21.15.107:0/305019050 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f8418014030 con 0x7f8430108b00 2024-03-08T21:00:06.717 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:00:06 smithi107 ceph-mon[31381]: from='client.? 172.21.15.107:0/1564103115' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-08T21:00:06.717 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:00:06 smithi107 ceph-mon[31381]: from='client.? 172.21.15.107:0/1225049613' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-08T21:00:06.764 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:00:06 smithi149 ceph-mon[35042]: from='client.? 172.21.15.107:0/1564103115' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-08T21:00:06.765 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:00:06 smithi149 ceph-mon[35042]: from='client.? 172.21.15.107:0/1225049613' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-08T21:00:06.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:06.868+0000 7f8434e00640 1 -- 172.21.15.107:0/305019050 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f8430061ef0 con 0x7f8430108b00 2024-03-08T21:00:06.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:06.869+0000 7f8426a00640 1 -- 172.21.15.107:0/305019050 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) v1 ==== 104+0+11 (secure 0 0 0) 0x7f841809b880 con 0x7f8430108b00 2024-03-08T21:00:06.852 INFO:teuthology.orchestra.run.smithi107.stdout:pg_num: 32 2024-03-08T21:00:06.855 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:06.871+0000 7f8434e00640 1 -- 172.21.15.107:0/305019050 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f84140a2800 msgr2=0x7f84140a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:00:06.855 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:06.872+0000 7f8434e00640 1 --2- 172.21.15.107:0/305019050 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f84140a2800 0x7f84140a4cc0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f842002c240 tx=0x7f8420005fd0 comp rx=0 tx=0).stop 2024-03-08T21:00:06.855 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:06.872+0000 7f8434e00640 1 -- 172.21.15.107:0/305019050 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8430108b00 msgr2=0x7f843019e0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:00:06.855 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:06.872+0000 7f8434e00640 1 --2- 172.21.15.107:0/305019050 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8430108b00 0x7f843019e0d0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f841800c890 tx=0x7f841800cd50 comp rx=0 tx=0).stop 2024-03-08T21:00:06.855 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:06.872+0000 7f8434e00640 1 -- 172.21.15.107:0/305019050 shutdown_connections 2024-03-08T21:00:06.855 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:06.872+0000 7f8434e00640 1 --2- 172.21.15.107:0/305019050 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f84140a2800 0x7f84140a4cc0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:06.855 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:06.872+0000 7f8434e00640 1 --2- 172.21.15.107:0/305019050 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f84301094c0 0x7f84301a5140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:06.855 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:06.872+0000 7f8434e00640 1 --2- 172.21.15.107:0/305019050 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8430108b00 0x7f843019e0d0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:06.856 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:06.872+0000 7f8434e00640 1 --2- 172.21.15.107:0/305019050 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8430107910 0x7f843019db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:06.856 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:06.872+0000 7f8434e00640 1 -- 172.21.15.107:0/305019050 >> 172.21.15.107:0/305019050 conn(0x7f84300768b0 msgr2=0x7f843010bed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:00:06.856 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:06.872+0000 7f8434e00640 1 -- 172.21.15.107:0/305019050 shutdown_connections 2024-03-08T21:00:06.856 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:06.872+0000 7f8434e00640 1 -- 172.21.15.107:0/305019050 wait complete. 2024-03-08T21:00:06.875 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-08T21:00:06.878 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:00:06 smithi204 ceph-mon[34935]: from='client.? 172.21.15.107:0/1564103115' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-08T21:00:06.878 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:00:06 smithi204 ceph-mon[34935]: from='client.? 172.21.15.107:0/1225049613' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-08T21:00:07.089 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:07.106+0000 7faa1c000640 1 -- 172.21.15.107:0/3588568357 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faa14108b00 msgr2=0x7faa14108f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:00:07.089 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:07.106+0000 7faa1c000640 1 --2- 172.21.15.107:0/3588568357 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faa14108b00 0x7faa14108f80 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7faa04009900 tx=0x7faa0402bc30 comp rx=0 tx=0).stop 2024-03-08T21:00:07.090 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:07.106+0000 7faa1c000640 1 -- 172.21.15.107:0/3588568357 shutdown_connections 2024-03-08T21:00:07.090 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:07.106+0000 7faa1c000640 1 --2- 172.21.15.107:0/3588568357 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faa141094c0 0x7faa1410fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:07.090 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:07.106+0000 7faa1c000640 1 --2- 172.21.15.107:0/3588568357 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faa14108b00 0x7faa14108f80 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:07.090 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:07.106+0000 7faa1c000640 1 --2- 172.21.15.107:0/3588568357 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faa14107910 0x7faa14107d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:07.090 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:07.106+0000 7faa1c000640 1 -- 172.21.15.107:0/3588568357 >> 172.21.15.107:0/3588568357 conn(0x7faa140768b0 msgr2=0x7faa14078cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:00:07.090 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:07.106+0000 7faa1c000640 1 -- 172.21.15.107:0/3588568357 shutdown_connections 2024-03-08T21:00:07.090 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:07.106+0000 7faa1c000640 1 -- 172.21.15.107:0/3588568357 wait complete. 2024-03-08T21:00:07.092 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:07.107+0000 7faa1c000640 1 Processor -- start 2024-03-08T21:00:07.092 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:07.107+0000 7faa1c000640 1 -- start start 2024-03-08T21:00:07.093 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:07.107+0000 7faa1c000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faa14107910 0x7faa1419daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:00:07.093 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:07.107+0000 7faa1c000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faa14108b00 0x7faa1419e030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:00:07.093 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:07.107+0000 7faa1c000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faa141094c0 0x7faa141a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:00:07.093 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:07.107+0000 7faa1c000640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa14113420 con 0x7faa141094c0 2024-03-08T21:00:07.093 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:07.107+0000 7faa1c000640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa141132a0 con 0x7faa14108b00 2024-03-08T21:00:07.093 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:07.107+0000 7faa1c000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa141135a0 con 0x7faa14107910 2024-03-08T21:00:07.093 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:07.108+0000 7faa1a000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faa141094c0 0x7faa141a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:00:07.093 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:07.108+0000 7faa1a000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faa141094c0 0x7faa141a50a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:56518/0 (socket says 172.21.15.107:56518) 2024-03-08T21:00:07.093 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:07.108+0000 7faa1a000640 1 -- 172.21.15.107:0/2787922285 learned_addr learned my addr 172.21.15.107:0/2787922285 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:00:07.093 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:07.108+0000 7faa18c00640 1 --2- 172.21.15.107:0/2787922285 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faa14108b00 0x7faa1419e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:00:07.093 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:07.108+0000 7faa19600640 1 --2- 172.21.15.107:0/2787922285 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faa14107910 0x7faa1419daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:00:07.093 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:07.108+0000 7faa1a000640 1 -- 172.21.15.107:0/2787922285 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faa14107910 msgr2=0x7faa1419daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:00:07.093 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:07.108+0000 7faa1a000640 1 --2- 172.21.15.107:0/2787922285 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faa14107910 0x7faa1419daf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:07.094 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:07.108+0000 7faa1a000640 1 -- 172.21.15.107:0/2787922285 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faa14108b00 msgr2=0x7faa1419e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:00:07.094 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:07.108+0000 7faa1a000640 1 --2- 172.21.15.107:0/2787922285 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faa14108b00 0x7faa1419e030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:07.094 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:07.108+0000 7faa1a000640 1 -- 172.21.15.107:0/2787922285 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa141a5710 con 0x7faa141094c0 2024-03-08T21:00:07.094 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:07.108+0000 7faa18c00640 1 --2- 172.21.15.107:0/2787922285 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faa14108b00 0x7faa1419e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T21:00:07.094 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:07.108+0000 7faa19600640 1 --2- 172.21.15.107:0/2787922285 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faa14107910 0x7faa1419daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T21:00:07.094 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:07.109+0000 7faa1a000640 1 --2- 172.21.15.107:0/2787922285 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faa141094c0 0x7faa141a50a0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7faa0c00cad0 tx=0x7faa0c00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:00:07.094 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:07.109+0000 7faa12000640 1 -- 172.21.15.107:0/2787922285 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa0c00ae20 con 0x7faa141094c0 2024-03-08T21:00:07.094 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:07.109+0000 7faa12000640 1 -- 172.21.15.107:0/2787922285 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7faa0c005020 con 0x7faa141094c0 2024-03-08T21:00:07.094 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:07.109+0000 7faa1c000640 1 -- 172.21.15.107:0/2787922285 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa141a5a00 con 0x7faa141094c0 2024-03-08T21:00:07.094 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:07.109+0000 7faa12000640 1 -- 172.21.15.107:0/2787922285 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa0c00f6b0 con 0x7faa141094c0 2024-03-08T21:00:07.094 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:07.109+0000 7faa1c000640 1 -- 172.21.15.107:0/2787922285 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa141a5f50 con 0x7faa141094c0 2024-03-08T21:00:07.095 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:07.110+0000 7faa1c000640 1 -- 172.21.15.107:0/2787922285 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa1410ec80 con 0x7faa141094c0 2024-03-08T21:00:07.095 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:07.112+0000 7faa12000640 1 -- 172.21.15.107:0/2787922285 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) v1 ==== 137291+0+0 (secure 0 0 0) 0x7faa0c0051d0 con 0x7faa141094c0 2024-03-08T21:00:07.096 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:07.114+0000 7faa12000640 1 --2- 172.21.15.107:0/2787922285 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fa9fc0a2880 0x7fa9fc0a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:00:07.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:07.114+0000 7faa12000640 1 -- 172.21.15.107:0/2787922285 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7faa0c0ceb50 con 0x7faa141094c0 2024-03-08T21:00:07.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:07.114+0000 7faa19600640 1 --2- 172.21.15.107:0/2787922285 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fa9fc0a2880 0x7fa9fc0a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:00:07.098 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:07.115+0000 7faa19600640 1 --2- 172.21.15.107:0/2787922285 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fa9fc0a2880 0x7fa9fc0a4d40 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7faa14108960 tx=0x7faa08005db0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:00:07.101 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:07.118+0000 7faa12000640 1 -- 172.21.15.107:0/2787922285 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7faa0c014030 con 0x7faa141094c0 2024-03-08T21:00:07.298 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:07.315+0000 7faa1c000640 1 -- 172.21.15.107:0/2787922285 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7faa14061ef0 con 0x7faa141094c0 2024-03-08T21:00:07.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:07.316+0000 7faa12000640 1 -- 172.21.15.107:0/2787922285 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) v1 ==== 100+0+11 (secure 0 0 0) 0x7faa0c09bbb0 con 0x7faa141094c0 2024-03-08T21:00:07.299 INFO:teuthology.orchestra.run.smithi107.stdout:pg_num: 64 2024-03-08T21:00:07.302 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:07.319+0000 7faa1c000640 1 -- 172.21.15.107:0/2787922285 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fa9fc0a2880 msgr2=0x7fa9fc0a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:00:07.302 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:07.319+0000 7faa1c000640 1 --2- 172.21.15.107:0/2787922285 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fa9fc0a2880 0x7fa9fc0a4d40 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7faa14108960 tx=0x7faa08005db0 comp rx=0 tx=0).stop 2024-03-08T21:00:07.302 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:07.319+0000 7faa1c000640 1 -- 172.21.15.107:0/2787922285 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faa141094c0 msgr2=0x7faa141a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:00:07.302 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:07.319+0000 7faa1c000640 1 --2- 172.21.15.107:0/2787922285 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faa141094c0 0x7faa141a50a0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7faa0c00cad0 tx=0x7faa0c00cf90 comp rx=0 tx=0).stop 2024-03-08T21:00:07.302 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:07.319+0000 7faa1c000640 1 -- 172.21.15.107:0/2787922285 shutdown_connections 2024-03-08T21:00:07.303 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:07.319+0000 7faa1c000640 1 --2- 172.21.15.107:0/2787922285 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fa9fc0a2880 0x7fa9fc0a4d40 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:07.303 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:07.319+0000 7faa1c000640 1 --2- 172.21.15.107:0/2787922285 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faa141094c0 0x7faa141a50a0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:07.303 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:07.319+0000 7faa1c000640 1 --2- 172.21.15.107:0/2787922285 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faa14108b00 0x7faa1419e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:07.303 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:07.319+0000 7faa1c000640 1 --2- 172.21.15.107:0/2787922285 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faa14107910 0x7faa1419daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:07.303 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:07.319+0000 7faa1c000640 1 -- 172.21.15.107:0/2787922285 >> 172.21.15.107:0/2787922285 conn(0x7faa140768b0 msgr2=0x7faa1410be50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:00:07.303 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:07.319+0000 7faa1c000640 1 -- 172.21.15.107:0/2787922285 shutdown_connections 2024-03-08T21:00:07.303 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:07.319+0000 7faa1c000640 1 -- 172.21.15.107:0/2787922285 wait complete. 2024-03-08T21:00:07.323 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-08T21:00:07.529 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:07.545+0000 7f2a0be00640 1 -- 172.21.15.107:0/366051462 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2a0c073d50 msgr2=0x7f2a0c0741d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:00:07.529 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:07.545+0000 7f2a0be00640 1 --2- 172.21.15.107:0/366051462 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2a0c073d50 0x7f2a0c0741d0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f29f4009900 tx=0x7f29f402bc30 comp rx=0 tx=0).stop 2024-03-08T21:00:07.529 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:07.545+0000 7f2a0be00640 1 -- 172.21.15.107:0/366051462 shutdown_connections 2024-03-08T21:00:07.529 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:07.545+0000 7f2a0be00640 1 --2- 172.21.15.107:0/366051462 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2a0c109010 0x7f2a0c10b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:07.529 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:07.545+0000 7f2a0be00640 1 --2- 172.21.15.107:0/366051462 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2a0c073d50 0x7f2a0c0741d0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:07.529 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:07.545+0000 7f2a0be00640 1 --2- 172.21.15.107:0/366051462 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2a0c075330 0x7f2a0c073780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:07.529 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:07.545+0000 7f2a0be00640 1 -- 172.21.15.107:0/366051462 >> 172.21.15.107:0/366051462 conn(0x7f2a0c0fc6c0 msgr2=0x7f2a0c0feb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:00:07.530 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:07.546+0000 7f2a0be00640 1 -- 172.21.15.107:0/366051462 shutdown_connections 2024-03-08T21:00:07.530 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:07.546+0000 7f2a0be00640 1 -- 172.21.15.107:0/366051462 wait complete. 2024-03-08T21:00:07.532 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:07.547+0000 7f2a0be00640 1 Processor -- start 2024-03-08T21:00:07.532 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:07.547+0000 7f2a0be00640 1 -- start start 2024-03-08T21:00:07.532 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:07.547+0000 7f2a0be00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2a0c073d50 0x7f2a0c199690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:00:07.532 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:07.547+0000 7f2a0be00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2a0c075330 0x7f2a0c199bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:00:07.532 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:07.547+0000 7f2a0be00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2a0c109010 0x7f2a0c1a0c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:00:07.532 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:07.547+0000 7f2a0be00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a0c10ef90 con 0x7f2a0c109010 2024-03-08T21:00:07.532 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:07.547+0000 7f2a0be00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a0c10ee10 con 0x7f2a0c073d50 2024-03-08T21:00:07.532 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:07.547+0000 7f2a0be00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a0c10f110 con 0x7f2a0c075330 2024-03-08T21:00:07.532 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:07.547+0000 7f2a0b400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2a0c109010 0x7f2a0c1a0c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:00:07.532 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:07.547+0000 7f2a0b400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2a0c109010 0x7f2a0c1a0c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:56530/0 (socket says 172.21.15.107:56530) 2024-03-08T21:00:07.532 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:07.547+0000 7f2a0b400640 1 -- 172.21.15.107:0/228591807 learned_addr learned my addr 172.21.15.107:0/228591807 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:00:07.532 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:07.548+0000 7f2a0a000640 1 --2- 172.21.15.107:0/228591807 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2a0c075330 0x7f2a0c199bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:00:07.533 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:07.548+0000 7f2a0aa00640 1 --2- 172.21.15.107:0/228591807 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2a0c073d50 0x7f2a0c199690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:00:07.533 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:07.548+0000 7f2a0b400640 1 -- 172.21.15.107:0/228591807 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2a0c075330 msgr2=0x7f2a0c199bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:00:07.533 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:07.548+0000 7f2a0b400640 1 --2- 172.21.15.107:0/228591807 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2a0c075330 0x7f2a0c199bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:07.533 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:07.548+0000 7f2a0b400640 1 -- 172.21.15.107:0/228591807 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2a0c073d50 msgr2=0x7f2a0c199690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:00:07.533 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:07.548+0000 7f2a0b400640 1 --2- 172.21.15.107:0/228591807 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2a0c073d50 0x7f2a0c199690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:07.533 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:07.548+0000 7f2a0b400640 1 -- 172.21.15.107:0/228591807 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a0c1a12b0 con 0x7f2a0c109010 2024-03-08T21:00:07.533 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:07.548+0000 7f2a0b400640 1 --2- 172.21.15.107:0/228591807 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2a0c109010 0x7f2a0c1a0c40 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f2a0400b970 tx=0x7f2a0400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:00:07.533 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:07.548+0000 7f2a03400640 1 -- 172.21.15.107:0/228591807 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a04019070 con 0x7f2a0c109010 2024-03-08T21:00:07.533 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:07.548+0000 7f2a03400640 1 -- 172.21.15.107:0/228591807 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f2a04014440 con 0x7f2a0c109010 2024-03-08T21:00:07.533 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:07.548+0000 7f2a03400640 1 -- 172.21.15.107:0/228591807 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a040075d0 con 0x7f2a0c109010 2024-03-08T21:00:07.533 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:07.548+0000 7f2a0be00640 1 -- 172.21.15.107:0/228591807 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a0c1a15a0 con 0x7f2a0c109010 2024-03-08T21:00:07.533 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:07.549+0000 7f2a0be00640 1 -- 172.21.15.107:0/228591807 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a0c1a1b70 con 0x7f2a0c109010 2024-03-08T21:00:07.534 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:07.550+0000 7f2a0be00640 1 -- 172.21.15.107:0/228591807 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a0c107d40 con 0x7f2a0c109010 2024-03-08T21:00:07.534 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:07.551+0000 7f2a03400640 1 -- 172.21.15.107:0/228591807 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f2a0401c070 con 0x7f2a0c109010 2024-03-08T21:00:07.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:07.554+0000 7f2a03400640 1 --2- 172.21.15.107:0/228591807 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f29f00a2880 0x7f29f00a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:00:07.537 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:07.554+0000 7f2a03400640 1 -- 172.21.15.107:0/228591807 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f2a040ce060 con 0x7f2a0c109010 2024-03-08T21:00:07.537 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:07.554+0000 7f2a0aa00640 1 --2- 172.21.15.107:0/228591807 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f29f00a2880 0x7f29f00a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:00:07.537 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:07.554+0000 7f2a0aa00640 1 --2- 172.21.15.107:0/228591807 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f29f00a2880 0x7f29f00a4d40 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f29fc009890 tx=0x7f29fc008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:00:07.541 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:07.558+0000 7f2a03400640 1 -- 172.21.15.107:0/228591807 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f2a0409b0c0 con 0x7f2a0c109010 2024-03-08T21:00:07.717 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:00:07 smithi107 ceph-mon[31381]: pgmap v431: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-08T21:00:07.717 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:00:07 smithi107 ceph-mon[31381]: from='client.? 172.21.15.107:0/305019050' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-08T21:00:07.717 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:00:07 smithi107 ceph-mon[31381]: from='client.? 172.21.15.107:0/2787922285' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-08T21:00:07.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:07.792+0000 7f2a0be00640 1 -- 172.21.15.107:0/228591807 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2a0c02ce70 con 0x7f2a0c109010 2024-03-08T21:00:07.776 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:07.793+0000 7f2a03400640 1 -- 172.21.15.107:0/228591807 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+11393 (secure 0 0 0) 0x7f2a0409a9c0 con 0x7f2a0c109010 2024-03-08T21:00:07.777 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-08T21:00:07.777 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/133582743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":133582743},{"type":"v1","addr":"172.21.15.107:6835","nonce":133582743}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/3798982337","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":3798982337},{"type":"v1","addr":"172.21.15.107:6837","nonce":3798982337}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-08T20:57:50.223752+0000","modified":"2024-03-08T20:57:57.874727+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34475,"mds_1":34445,"mds_2":34415,"mds_3":34397,"mds_4":24457},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14625":{"gid":14625,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6839/3070059000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":3070059000},{"type":"v1","addr":"172.21.15.107:6839","nonce":3070059000}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14637":{"gid":14637,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6841/747096194","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6840","nonce":747096194},{"type":"v1","addr":"172.21.15.107:6841","nonce":747096194}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24400":{"gid":24400,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6833/3404762782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":3404762782},{"type":"v1","addr":"172.21.15.204:6833","nonce":3404762782}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24430":{"gid":24430,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/2072518651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":2072518651},{"type":"v1","addr":"172.21.15.204:6835","nonce":2072518651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24442":{"gid":24442,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6837/42644905","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":42644905},{"type":"v1","addr":"172.21.15.204:6837","nonce":42644905}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24457":{"gid":24457,"name":"i","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.204:6839/1857316822","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":1857316822},{"type":"v1","addr":"172.21.15.204:6839","nonce":1857316822}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34397":{"gid":34397,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":14,"addr":"172.21.15.149:6833/1962161726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":1962161726},{"type":"v1","addr":"172.21.15.149:6833","nonce":1962161726}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34415":{"gid":34415,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.149:6835/442657066","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":442657066},{"type":"v1","addr":"172.21.15.149:6835","nonce":442657066}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34445":{"gid":34445,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.149:6837/914546237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":914546237},{"type":"v1","addr":"172.21.15.149:6837","nonce":914546237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34475":{"gid":34475,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.149:6839/55290596","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":55290596},{"type":"v1","addr":"172.21.15.149:6839","nonce":55290596}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-08T21:00:07.777 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 27 2024-03-08T21:00:07.779 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:07.796+0000 7f2a0be00640 1 -- 172.21.15.107:0/228591807 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f29f00a2880 msgr2=0x7f29f00a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:00:07.779 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:07.796+0000 7f2a0be00640 1 --2- 172.21.15.107:0/228591807 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f29f00a2880 0x7f29f00a4d40 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f29fc009890 tx=0x7f29fc008040 comp rx=0 tx=0).stop 2024-03-08T21:00:07.779 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:07.796+0000 7f2a0be00640 1 -- 172.21.15.107:0/228591807 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2a0c109010 msgr2=0x7f2a0c1a0c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:00:07.779 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:07.796+0000 7f2a0be00640 1 --2- 172.21.15.107:0/228591807 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2a0c109010 0x7f2a0c1a0c40 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f2a0400b970 tx=0x7f2a0400be30 comp rx=0 tx=0).stop 2024-03-08T21:00:07.779 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:07.796+0000 7f2a0be00640 1 -- 172.21.15.107:0/228591807 shutdown_connections 2024-03-08T21:00:07.779 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:07.796+0000 7f2a0be00640 1 --2- 172.21.15.107:0/228591807 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f29f00a2880 0x7f29f00a4d40 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:07.780 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:07.796+0000 7f2a0be00640 1 --2- 172.21.15.107:0/228591807 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2a0c109010 0x7f2a0c1a0c40 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:07.780 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:07.796+0000 7f2a0be00640 1 --2- 172.21.15.107:0/228591807 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2a0c075330 0x7f2a0c199bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:07.780 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:07.796+0000 7f2a0be00640 1 --2- 172.21.15.107:0/228591807 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2a0c073d50 0x7f2a0c199690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:07.780 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:07.796+0000 7f2a0be00640 1 -- 172.21.15.107:0/228591807 >> 172.21.15.107:0/228591807 conn(0x7f2a0c0fc6c0 msgr2=0x7f2a0c104f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:00:07.780 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:07.796+0000 7f2a0be00640 1 -- 172.21.15.107:0/228591807 shutdown_connections 2024-03-08T21:00:07.780 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:07.796+0000 7f2a0be00640 1 -- 172.21.15.107:0/228591807 wait complete. 2024-03-08T21:00:07.798 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-08T21:00:07.878 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:00:07 smithi204 ceph-mon[34935]: pgmap v431: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-08T21:00:07.878 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:00:07 smithi204 ceph-mon[34935]: from='client.? 172.21.15.107:0/305019050' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-08T21:00:07.878 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:00:07 smithi204 ceph-mon[34935]: from='client.? 172.21.15.107:0/2787922285' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-08T21:00:08.006 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:08.022+0000 7fbafea00640 1 -- 172.21.15.107:0/87756515 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbaf8104720 msgr2=0x7fbaf8104ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:00:08.006 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:08.022+0000 7fbafea00640 1 --2- 172.21.15.107:0/87756515 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbaf8104720 0x7fbaf8104ba0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fbae0009900 tx=0x7fbae002bc50 comp rx=0 tx=0).stop 2024-03-08T21:00:08.006 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:08.023+0000 7fbafea00640 1 -- 172.21.15.107:0/87756515 shutdown_connections 2024-03-08T21:00:08.006 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:08.023+0000 7fbafea00640 1 --2- 172.21.15.107:0/87756515 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbaf81050e0 0x7fbaf810b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:08.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:08.023+0000 7fbafea00640 1 --2- 172.21.15.107:0/87756515 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbaf8104720 0x7fbaf8104ba0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:08.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:08.023+0000 7fbafea00640 1 --2- 172.21.15.107:0/87756515 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbaf8103530 0x7fbaf8103950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:08.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:08.023+0000 7fbafea00640 1 -- 172.21.15.107:0/87756515 >> 172.21.15.107:0/87756515 conn(0x7fbaf80feae0 msgr2=0x7fbaf8100f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:00:08.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:08.023+0000 7fbafea00640 1 -- 172.21.15.107:0/87756515 shutdown_connections 2024-03-08T21:00:08.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:08.023+0000 7fbafea00640 1 -- 172.21.15.107:0/87756515 wait complete. 2024-03-08T21:00:08.010 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:08.024+0000 7fbafea00640 1 Processor -- start 2024-03-08T21:00:08.010 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:08.024+0000 7fbafea00640 1 -- start start 2024-03-08T21:00:08.010 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:08.024+0000 7fbafea00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbaf8103530 0x7fbaf81998d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:00:08.010 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:08.024+0000 7fbafea00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbaf8104720 0x7fbaf8199e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:00:08.010 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:08.024+0000 7fbafea00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbaf81050e0 0x7fbaf81a0e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:00:08.010 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:08.024+0000 7fbafea00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbaf810f1a0 con 0x7fbaf8103530 2024-03-08T21:00:08.010 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:08.024+0000 7fbafea00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbaf810f020 con 0x7fbaf8104720 2024-03-08T21:00:08.010 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:08.024+0000 7fbafea00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbaf810f320 con 0x7fbaf81050e0 2024-03-08T21:00:08.010 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:08.024+0000 7fbaf7e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbaf8103530 0x7fbaf81998d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:00:08.010 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:08.024+0000 7fbaf7e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbaf8103530 0x7fbaf81998d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:56552/0 (socket says 172.21.15.107:56552) 2024-03-08T21:00:08.011 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:08.024+0000 7fbaf7e00640 1 -- 172.21.15.107:0/3282863309 learned_addr learned my addr 172.21.15.107:0/3282863309 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:00:08.011 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:08.025+0000 7fbaef400640 1 --2- 172.21.15.107:0/3282863309 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbaf8104720 0x7fbaf8199e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:00:08.011 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:08.025+0000 7fbaf7e00640 1 -- 172.21.15.107:0/3282863309 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbaf81050e0 msgr2=0x7fbaf81a0e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:00:08.011 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:08.025+0000 7fbafca00640 1 --2- 172.21.15.107:0/3282863309 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbaf81050e0 0x7fbaf81a0e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:00:08.011 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:08.025+0000 7fbaf7e00640 1 --2- 172.21.15.107:0/3282863309 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbaf81050e0 0x7fbaf81a0e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:08.011 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:08.025+0000 7fbaf7e00640 1 -- 172.21.15.107:0/3282863309 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbaf8104720 msgr2=0x7fbaf8199e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:00:08.011 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:08.025+0000 7fbaf7e00640 1 --2- 172.21.15.107:0/3282863309 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbaf8104720 0x7fbaf8199e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:08.011 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:08.025+0000 7fbaf7e00640 1 -- 172.21.15.107:0/3282863309 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbaf81a14f0 con 0x7fbaf8103530 2024-03-08T21:00:08.011 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:08.025+0000 7fbaef400640 1 --2- 172.21.15.107:0/3282863309 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbaf8104720 0x7fbaf8199e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T21:00:08.011 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:08.025+0000 7fbaf7e00640 1 --2- 172.21.15.107:0/3282863309 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbaf8103530 0x7fbaf81998d0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fbae800b9c0 tx=0x7fbae800be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:00:08.011 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:08.025+0000 7fbaf5600640 1 -- 172.21.15.107:0/3282863309 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbae8019070 con 0x7fbaf8103530 2024-03-08T21:00:08.011 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:08.025+0000 7fbaf5600640 1 -- 172.21.15.107:0/3282863309 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fbae800cba0 con 0x7fbaf8103530 2024-03-08T21:00:08.011 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:08.025+0000 7fbafea00640 1 -- 172.21.15.107:0/3282863309 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbaf81a17e0 con 0x7fbaf8103530 2024-03-08T21:00:08.011 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:08.026+0000 7fbaf5600640 1 -- 172.21.15.107:0/3282863309 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbae80125a0 con 0x7fbaf8103530 2024-03-08T21:00:08.012 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:08.026+0000 7fbafea00640 1 -- 172.21.15.107:0/3282863309 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbaf81a1e00 con 0x7fbaf8103530 2024-03-08T21:00:08.012 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:08.027+0000 7fbafea00640 1 -- 172.21.15.107:0/3282863309 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbaf810a970 con 0x7fbaf8103530 2024-03-08T21:00:08.012 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:08.029+0000 7fbaf5600640 1 -- 172.21.15.107:0/3282863309 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) v1 ==== 137291+0+0 (secure 0 0 0) 0x7fbae801c070 con 0x7fbaf8103530 2024-03-08T21:00:08.013 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:08.030+0000 7fbaf5600640 1 --2- 172.21.15.107:0/3282863309 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fbadc0a2880 0x7fbadc0a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:00:08.013 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:08.030+0000 7fbaf5600640 1 -- 172.21.15.107:0/3282863309 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fbae80ce5f0 con 0x7fbaf8103530 2024-03-08T21:00:08.013 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:08.030+0000 7fbaef400640 1 --2- 172.21.15.107:0/3282863309 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fbadc0a2880 0x7fbadc0a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:00:08.014 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:08.031+0000 7fbaef400640 1 --2- 172.21.15.107:0/3282863309 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fbadc0a2880 0x7fbadc0a4d40 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fbae0036040 tx=0x7fbae0005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:00:08.014 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:00:07 smithi149 ceph-mon[35042]: pgmap v431: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-08T21:00:08.014 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:00:07 smithi149 ceph-mon[35042]: from='client.? 172.21.15.107:0/305019050' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-08T21:00:08.014 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:00:07 smithi149 ceph-mon[35042]: from='client.? 172.21.15.107:0/2787922285' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-08T21:00:08.016 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:08.033+0000 7fbaf5600640 1 -- 172.21.15.107:0/3282863309 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7fbae809b690 con 0x7fbaf8103530 2024-03-08T21:00:08.214 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:08.231+0000 7fbafea00640 1 -- 172.21.15.107:0/3282863309 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbaf802d0e0 con 0x7fbaf8103530 2024-03-08T21:00:08.219 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:08.236+0000 7fbaf5600640 1 -- 172.21.15.107:0/3282863309 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+22779 (secure 0 0 0) 0x7fbae809af90 con 0x7fbaf8103530 2024-03-08T21:00:08.220 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-08T21:00:08.220 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":88,"fsid":"dc20f0f8-dd8c-11ee-95c6-87774f69a715","created":"2024-03-08T20:47:04.190034+0000","modified":"2024-03-08T20:58:21.449936+0000","last_up_change":"2024-03-08T20:56:43.135514+0000","last_in_change":"2024-03-08T20:56:17.577074+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-08T20:51:32.832786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-08T20:57:48.721227+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-08T20:57:49.707779+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8b1ddc64-4d20-4e85-ac66-f1c5ac599501","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":2453946095},{"type":"v1","addr":"172.21.15.107:6803","nonce":2453946095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":2453946095},{"type":"v1","addr":"172.21.15.107:6805","nonce":2453946095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":2453946095},{"type":"v1","addr":"172.21.15.107:6809","nonce":2453946095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":2453946095},{"type":"v1","addr":"172.21.15.107:6807","nonce":2453946095}]},"public_addr":"172.21.15.107:6803/2453946095","cluster_addr":"172.21.15.107:6805/2453946095","heartbeat_back_addr":"172.21.15.107:6809/2453946095","heartbeat_front_addr":"172.21.15.107:6807/2453946095","state":["exists","up"]},{"osd":1,"uuid":"5a803c09-202a-4777-ba2a-1da8eaf26a54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":3071316419},{"type":"v1","addr":"172.21.15.149:6801","nonce":3071316419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":3071316419},{"type":"v1","addr":"172.21.15.149:6803","nonce":3071316419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":3071316419},{"type":"v1","addr":"172.21.15.149:6807","nonce":3071316419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":3071316419},{"type":"v1","addr":"172.21.15.149:6805","nonce":3071316419}]},"public_addr":"172.21.15.149:6801/3071316419","cluster_addr":"172.21.15.149:6803/3071316419","heartbeat_back_addr":"172.21.15.149:6807/3071316419","heartbeat_front_addr":"172.21.15.149:6805/3071316419","state":["exists","up"]},{"osd":2,"uuid":"555c634f-347e-47dd-a943-e50ae2be073c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":1087852630},{"type":"v1","addr":"172.21.15.204:6801","nonce":1087852630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":1087852630},{"type":"v1","addr":"172.21.15.204:6803","nonce":1087852630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":1087852630},{"type":"v1","addr":"172.21.15.204:6807","nonce":1087852630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":1087852630},{"type":"v1","addr":"172.21.15.204:6805","nonce":1087852630}]},"public_addr":"172.21.15.204:6801/1087852630","cluster_addr":"172.21.15.204:6803/1087852630","heartbeat_back_addr":"172.21.15.204:6807/1087852630","heartbeat_front_addr":"172.21.15.204:6805/1087852630","state":["exists","up"]},{"osd":3,"uuid":"9e433d97-e202-48e0-9ed1-f0988fa14307","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":3123579789},{"type":"v1","addr":"172.21.15.107:6811","nonce":3123579789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":3123579789},{"type":"v1","addr":"172.21.15.107:6813","nonce":3123579789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":3123579789},{"type":"v1","addr":"172.21.15.107:6817","nonce":3123579789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":3123579789},{"type":"v1","addr":"172.21.15.107:6815","nonce":3123579789}]},"public_addr":"172.21.15.107:6811/3123579789","cluster_addr":"172.21.15.107:6813/3123579789","heartbeat_back_addr":"172.21.15.107:6817/3123579789","heartbeat_front_addr":"172.21.15.107:6815/3123579789","state":["exists","up"]},{"osd":4,"uuid":"27c53794-9d2b-487d-bfcf-ac4fd2ab061e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":726286918},{"type":"v1","addr":"172.21.15.149:6809","nonce":726286918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":726286918},{"type":"v1","addr":"172.21.15.149:6811","nonce":726286918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":726286918},{"type":"v1","addr":"172.21.15.149:6815","nonce":726286918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":726286918},{"type":"v1","addr":"172.21.15.149:6813","nonce":726286918}]},"public_addr":"172.21.15.149:6809/726286918","cluster_addr":"172.21.15.149:6811/726286918","heartbeat_back_addr":"172.21.15.149:6815/726286918","heartbeat_front_addr":"172.21.15.149:6813/726286918","state":["exists","up"]},{"osd":5,"uuid":"3a7778fb-46c5-40c0-842e-e988c45ea24d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":3143887820},{"type":"v1","addr":"172.21.15.204:6809","nonce":3143887820}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":3143887820},{"type":"v1","addr":"172.21.15.204:6811","nonce":3143887820}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":3143887820},{"type":"v1","addr":"172.21.15.204:6815","nonce":3143887820}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":3143887820},{"type":"v1","addr":"172.21.15.204:6813","nonce":3143887820}]},"public_addr":"172.21.15.204:6809/3143887820","cluster_addr":"172.21.15.204:6811/3143887820","heartbeat_back_addr":"172.21.15.204:6815/3143887820","heartbeat_front_addr":"172.21.15.204:6813/3143887820","state":["exists","up"]},{"osd":6,"uuid":"d2e05a56-be4f-4435-9b76-ff9aa01e4ac3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":2912452950},{"type":"v1","addr":"172.21.15.107:6819","nonce":2912452950}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":2912452950},{"type":"v1","addr":"172.21.15.107:6821","nonce":2912452950}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":2912452950},{"type":"v1","addr":"172.21.15.107:6825","nonce":2912452950}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":2912452950},{"type":"v1","addr":"172.21.15.107:6823","nonce":2912452950}]},"public_addr":"172.21.15.107:6819/2912452950","cluster_addr":"172.21.15.107:6821/2912452950","heartbeat_back_addr":"172.21.15.107:6825/2912452950","heartbeat_front_addr":"172.21.15.107:6823/2912452950","state":["exists","up"]},{"osd":7,"uuid":"a7dd26ee-c118-4a9a-9e19-142c2680902d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":3835863809},{"type":"v1","addr":"172.21.15.149:6817","nonce":3835863809}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":3835863809},{"type":"v1","addr":"172.21.15.149:6819","nonce":3835863809}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":3835863809},{"type":"v1","addr":"172.21.15.149:6823","nonce":3835863809}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":3835863809},{"type":"v1","addr":"172.21.15.149:6821","nonce":3835863809}]},"public_addr":"172.21.15.149:6817/3835863809","cluster_addr":"172.21.15.149:6819/3835863809","heartbeat_back_addr":"172.21.15.149:6823/3835863809","heartbeat_front_addr":"172.21.15.149:6821/3835863809","state":["exists","up"]},{"osd":8,"uuid":"baaaa8dd-0e1f-4bae-86bf-9e429d985f23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":1010601477},{"type":"v1","addr":"172.21.15.204:6817","nonce":1010601477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":1010601477},{"type":"v1","addr":"172.21.15.204:6819","nonce":1010601477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":1010601477},{"type":"v1","addr":"172.21.15.204:6823","nonce":1010601477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":1010601477},{"type":"v1","addr":"172.21.15.204:6821","nonce":1010601477}]},"public_addr":"172.21.15.204:6817/1010601477","cluster_addr":"172.21.15.204:6819/1010601477","heartbeat_back_addr":"172.21.15.204:6823/1010601477","heartbeat_front_addr":"172.21.15.204:6821/1010601477","state":["exists","up"]},{"osd":9,"uuid":"8df80dfa-e7ff-410e-824c-b1dce111b02f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":3334759819},{"type":"v1","addr":"172.21.15.107:6827","nonce":3334759819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3334759819},{"type":"v1","addr":"172.21.15.107:6829","nonce":3334759819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":3334759819},{"type":"v1","addr":"172.21.15.107:6833","nonce":3334759819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":3334759819},{"type":"v1","addr":"172.21.15.107:6831","nonce":3334759819}]},"public_addr":"172.21.15.107:6827/3334759819","cluster_addr":"172.21.15.107:6829/3334759819","heartbeat_back_addr":"172.21.15.107:6833/3334759819","heartbeat_front_addr":"172.21.15.107:6831/3334759819","state":["exists","up"]},{"osd":10,"uuid":"338945b2-20c0-40b2-aaca-062cd4adee04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":3469284631},{"type":"v1","addr":"172.21.15.149:6825","nonce":3469284631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":3469284631},{"type":"v1","addr":"172.21.15.149:6827","nonce":3469284631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":3469284631},{"type":"v1","addr":"172.21.15.149:6831","nonce":3469284631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":3469284631},{"type":"v1","addr":"172.21.15.149:6829","nonce":3469284631}]},"public_addr":"172.21.15.149:6825/3469284631","cluster_addr":"172.21.15.149:6827/3469284631","heartbeat_back_addr":"172.21.15.149:6831/3469284631","heartbeat_front_addr":"172.21.15.149:6829/3469284631","state":["exists","up"]},{"osd":11,"uuid":"acccfe4f-d307-4a64-8f4d-fba27bfbe09d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":3097772473},{"type":"v1","addr":"172.21.15.204:6825","nonce":3097772473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":3097772473},{"type":"v1","addr":"172.21.15.204:6827","nonce":3097772473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":3097772473},{"type":"v1","addr":"172.21.15.204:6831","nonce":3097772473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":3097772473},{"type":"v1","addr":"172.21.15.204:6829","nonce":3097772473}]},"public_addr":"172.21.15.204:6825/3097772473","cluster_addr":"172.21.15.204:6827/3097772473","heartbeat_back_addr":"172.21.15.204:6831/3097772473","heartbeat_front_addr":"172.21.15.204:6829/3097772473","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T20:50:25.862753+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T20:50:58.167663+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T20:51:30.312816+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T20:52:05.487917+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T20:52:39.348354+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T20:53:12.899002+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T20:53:47.630837+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T20:54:21.444552+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T20:54:57.101160+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T20:55:31.841786+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T20:56:05.827161+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T20:56:40.611097+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.0","mappings":[{"from":4,"to":6}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.3b","mappings":[{"from":3,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.107:0/1514666962":"2024-03-09T20:48:20.149406+0000","172.21.15.107:0/1886016707":"2024-03-09T20:48:20.149406+0000","172.21.15.107:6801/426372446":"2024-03-09T20:48:20.149406+0000","172.21.15.107:6800/426372446":"2024-03-09T20:48:20.149406+0000","172.21.15.107:0/2392547434":"2024-03-09T20:47:45.706790+0000","172.21.15.107:0/1206995985":"2024-03-09T20:47:45.706790+0000","172.21.15.107:6800/3549936519":"2024-03-09T20:47:45.706790+0000","172.21.15.107:6801/3549936519":"2024-03-09T20:47:45.706790+0000","172.21.15.107:0/2482452215":"2024-03-09T20:47:45.706790+0000","172.21.15.107:0/808124991":"2024-03-09T20:48:20.149406+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-08T21:00:08.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:08.239+0000 7fbafea00640 1 -- 172.21.15.107:0/3282863309 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fbadc0a2880 msgr2=0x7fbadc0a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:00:08.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:08.239+0000 7fbafea00640 1 --2- 172.21.15.107:0/3282863309 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fbadc0a2880 0x7fbadc0a4d40 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fbae0036040 tx=0x7fbae0005fd0 comp rx=0 tx=0).stop 2024-03-08T21:00:08.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:08.239+0000 7fbafea00640 1 -- 172.21.15.107:0/3282863309 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbaf8103530 msgr2=0x7fbaf81998d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:00:08.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:08.239+0000 7fbafea00640 1 --2- 172.21.15.107:0/3282863309 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbaf8103530 0x7fbaf81998d0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fbae800b9c0 tx=0x7fbae800be80 comp rx=0 tx=0).stop 2024-03-08T21:00:08.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:08.239+0000 7fbafea00640 1 -- 172.21.15.107:0/3282863309 shutdown_connections 2024-03-08T21:00:08.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:08.239+0000 7fbafea00640 1 --2- 172.21.15.107:0/3282863309 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fbadc0a2880 0x7fbadc0a4d40 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:08.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:08.239+0000 7fbafea00640 1 --2- 172.21.15.107:0/3282863309 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbaf81050e0 0x7fbaf81a0e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:08.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:08.239+0000 7fbafea00640 1 --2- 172.21.15.107:0/3282863309 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbaf8104720 0x7fbaf8199e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:08.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:08.239+0000 7fbafea00640 1 --2- 172.21.15.107:0/3282863309 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbaf8103530 0x7fbaf81998d0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:08.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:08.239+0000 7fbafea00640 1 -- 172.21.15.107:0/3282863309 >> 172.21.15.107:0/3282863309 conn(0x7fbaf80feae0 msgr2=0x7fbaf8107b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:00:08.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:08.239+0000 7fbafea00640 1 -- 172.21.15.107:0/3282863309 shutdown_connections 2024-03-08T21:00:08.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:08.239+0000 7fbafea00640 1 -- 172.21.15.107:0/3282863309 wait complete. 2024-03-08T21:00:08.241 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-08T20:51:32.832786+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-08T20:57:48.721227+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-08T20:57:49.707779+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-08T21:00:08.241 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-08T21:00:08.436 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:08.453+0000 7fb802200640 1 -- 172.21.15.107:0/2391092695 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb7fc104700 msgr2=0x7fb7fc104b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:00:08.437 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:08.453+0000 7fb802200640 1 --2- 172.21.15.107:0/2391092695 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb7fc104700 0x7fb7fc104b80 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fb7e4009900 tx=0x7fb7e402bc30 comp rx=0 tx=0).stop 2024-03-08T21:00:08.437 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:08.453+0000 7fb802200640 1 -- 172.21.15.107:0/2391092695 shutdown_connections 2024-03-08T21:00:08.437 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:08.453+0000 7fb802200640 1 --2- 172.21.15.107:0/2391092695 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb7fc1050c0 0x7fb7fc10b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:08.437 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:08.453+0000 7fb802200640 1 --2- 172.21.15.107:0/2391092695 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb7fc104700 0x7fb7fc104b80 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:08.437 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:08.453+0000 7fb802200640 1 --2- 172.21.15.107:0/2391092695 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb7fc103510 0x7fb7fc103930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:08.437 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:08.453+0000 7fb802200640 1 -- 172.21.15.107:0/2391092695 >> 172.21.15.107:0/2391092695 conn(0x7fb7fc0feaa0 msgr2=0x7fb7fc100ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:00:08.437 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:08.453+0000 7fb802200640 1 -- 172.21.15.107:0/2391092695 shutdown_connections 2024-03-08T21:00:08.437 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:08.453+0000 7fb802200640 1 -- 172.21.15.107:0/2391092695 wait complete. 2024-03-08T21:00:08.439 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:08.454+0000 7fb802200640 1 Processor -- start 2024-03-08T21:00:08.439 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:08.454+0000 7fb802200640 1 -- start start 2024-03-08T21:00:08.439 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:08.454+0000 7fb802200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb7fc103510 0x7fb7fc1996b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:00:08.440 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:08.454+0000 7fb802200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb7fc104700 0x7fb7fc199bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:00:08.440 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:08.454+0000 7fb802200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb7fc1050c0 0x7fb7fc1a0c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:00:08.440 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:08.454+0000 7fb802200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7fc10f050 con 0x7fb7fc1050c0 2024-03-08T21:00:08.440 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:08.454+0000 7fb802200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7fc10eed0 con 0x7fb7fc103510 2024-03-08T21:00:08.440 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:08.454+0000 7fb802200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7fc10f1d0 con 0x7fb7fc104700 2024-03-08T21:00:08.440 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:08.455+0000 7fb7fbe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb7fc1050c0 0x7fb7fc1a0c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:00:08.440 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:08.455+0000 7fb7fbe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb7fc1050c0 0x7fb7fc1a0c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:56576/0 (socket says 172.21.15.107:56576) 2024-03-08T21:00:08.440 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:08.455+0000 7fb7fbe00640 1 -- 172.21.15.107:0/3545772356 learned_addr learned my addr 172.21.15.107:0/3545772356 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:00:08.440 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:08.455+0000 7fb7fbe00640 1 -- 172.21.15.107:0/3545772356 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb7fc104700 msgr2=0x7fb7fc199bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:00:08.440 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:08.455+0000 7fb7fb400640 1 --2- 172.21.15.107:0/3545772356 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb7fc103510 0x7fb7fc1996b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:00:08.440 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:08.455+0000 7fb7f3e00640 1 --2- 172.21.15.107:0/3545772356 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb7fc104700 0x7fb7fc199bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:00:08.440 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:08.455+0000 7fb7fbe00640 1 --2- 172.21.15.107:0/3545772356 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb7fc104700 0x7fb7fc199bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:08.440 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:08.455+0000 7fb7fbe00640 1 -- 172.21.15.107:0/3545772356 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb7fc103510 msgr2=0x7fb7fc1996b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:00:08.441 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:08.455+0000 7fb7fbe00640 1 --2- 172.21.15.107:0/3545772356 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb7fc103510 0x7fb7fc1996b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:08.441 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:08.455+0000 7fb7fbe00640 1 -- 172.21.15.107:0/3545772356 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7fc1a12d0 con 0x7fb7fc1050c0 2024-03-08T21:00:08.441 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:08.455+0000 7fb7fbe00640 1 --2- 172.21.15.107:0/3545772356 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb7fc1050c0 0x7fb7fc1a0c60 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fb7f400cce0 tx=0x7fb7f400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:00:08.441 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:08.455+0000 7fb7f8c00640 1 -- 172.21.15.107:0/3545772356 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7f400f450 con 0x7fb7fc1050c0 2024-03-08T21:00:08.441 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:08.456+0000 7fb7f8c00640 1 -- 172.21.15.107:0/3545772356 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fb7f400faa0 con 0x7fb7fc1050c0 2024-03-08T21:00:08.441 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:08.456+0000 7fb802200640 1 -- 172.21.15.107:0/3545772356 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7fc1a15c0 con 0x7fb7fc1050c0 2024-03-08T21:00:08.442 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:08.456+0000 7fb7f8c00640 1 -- 172.21.15.107:0/3545772356 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7f4018b70 con 0x7fb7fc1050c0 2024-03-08T21:00:08.442 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:08.456+0000 7fb802200640 1 -- 172.21.15.107:0/3545772356 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7fc1a1ac0 con 0x7fb7fc1050c0 2024-03-08T21:00:08.442 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:08.457+0000 7fb802200640 1 -- 172.21.15.107:0/3545772356 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7fc10a830 con 0x7fb7fc1050c0 2024-03-08T21:00:08.442 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:08.459+0000 7fb7f8c00640 1 -- 172.21.15.107:0/3545772356 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) v1 ==== 137291+0+0 (secure 0 0 0) 0x7fb7f4018d10 con 0x7fb7fc1050c0 2024-03-08T21:00:08.443 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:08.460+0000 7fb7f8c00640 1 --2- 172.21.15.107:0/3545772356 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fb7e00a2800 0x7fb7e00a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:00:08.443 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:08.460+0000 7fb7f8c00640 1 -- 172.21.15.107:0/3545772356 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fb7f40cef70 con 0x7fb7fc1050c0 2024-03-08T21:00:08.443 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:08.460+0000 7fb7fb400640 1 --2- 172.21.15.107:0/3545772356 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fb7e00a2800 0x7fb7e00a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:00:08.443 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:08.461+0000 7fb7fb400640 1 --2- 172.21.15.107:0/3545772356 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fb7e00a2800 0x7fb7e00a4cc0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fb7ec005fd0 tx=0x7fb7ec005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:00:08.447 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:08.464+0000 7fb7f8c00640 1 -- 172.21.15.107:0/3545772356 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7fb7f4014030 con 0x7fb7fc1050c0 2024-03-08T21:00:08.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:08.652+0000 7fb802200640 1 -- 172.21.15.107:0/3545772356 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb7fc061ef0 con 0x7fb7fc1050c0 2024-03-08T21:00:08.636 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:08.653+0000 7fb7f8c00640 1 -- 172.21.15.107:0/3545772356 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb7f409bfd0 con 0x7fb7fc1050c0 2024-03-08T21:00:08.636 INFO:teuthology.orchestra.run.smithi107.stdout:pg_num: 1 2024-03-08T21:00:08.639 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:08.655+0000 7fb802200640 1 -- 172.21.15.107:0/3545772356 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fb7e00a2800 msgr2=0x7fb7e00a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:00:08.639 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:08.655+0000 7fb802200640 1 --2- 172.21.15.107:0/3545772356 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fb7e00a2800 0x7fb7e00a4cc0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fb7ec005fd0 tx=0x7fb7ec005850 comp rx=0 tx=0).stop 2024-03-08T21:00:08.639 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:08.655+0000 7fb802200640 1 -- 172.21.15.107:0/3545772356 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb7fc1050c0 msgr2=0x7fb7fc1a0c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:00:08.639 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:08.655+0000 7fb802200640 1 --2- 172.21.15.107:0/3545772356 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb7fc1050c0 0x7fb7fc1a0c60 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fb7f400cce0 tx=0x7fb7f400a620 comp rx=0 tx=0).stop 2024-03-08T21:00:08.639 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:08.656+0000 7fb802200640 1 -- 172.21.15.107:0/3545772356 shutdown_connections 2024-03-08T21:00:08.639 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:08.656+0000 7fb802200640 1 --2- 172.21.15.107:0/3545772356 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fb7e00a2800 0x7fb7e00a4cc0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:08.639 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:08.656+0000 7fb802200640 1 --2- 172.21.15.107:0/3545772356 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb7fc1050c0 0x7fb7fc1a0c60 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:08.639 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:08.656+0000 7fb802200640 1 --2- 172.21.15.107:0/3545772356 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb7fc104700 0x7fb7fc199bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:08.640 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:08.656+0000 7fb802200640 1 --2- 172.21.15.107:0/3545772356 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb7fc103510 0x7fb7fc1996b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:08.640 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:08.656+0000 7fb802200640 1 -- 172.21.15.107:0/3545772356 >> 172.21.15.107:0/3545772356 conn(0x7fb7fc0feaa0 msgr2=0x7fb7fc107a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:00:08.640 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:08.656+0000 7fb802200640 1 -- 172.21.15.107:0/3545772356 shutdown_connections 2024-03-08T21:00:08.640 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:08.656+0000 7fb802200640 1 -- 172.21.15.107:0/3545772356 wait complete. 2024-03-08T21:00:08.657 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-08T21:00:08.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:08.889+0000 7fe344600640 1 -- 172.21.15.107:0/2804042693 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe33c103510 msgr2=0x7fe33c103930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:00:08.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:08.889+0000 7fe344600640 1 --2- 172.21.15.107:0/2804042693 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe33c103510 0x7fe33c103930 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fe330009900 tx=0x7fe33002bc30 comp rx=0 tx=0).stop 2024-03-08T21:00:08.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:08.889+0000 7fe344600640 1 -- 172.21.15.107:0/2804042693 shutdown_connections 2024-03-08T21:00:08.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:08.889+0000 7fe344600640 1 --2- 172.21.15.107:0/2804042693 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe33c1050c0 0x7fe33c10b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:08.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:08.889+0000 7fe344600640 1 --2- 172.21.15.107:0/2804042693 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe33c104700 0x7fe33c104b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:08.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:08.889+0000 7fe344600640 1 --2- 172.21.15.107:0/2804042693 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe33c103510 0x7fe33c103930 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:08.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:08.889+0000 7fe344600640 1 -- 172.21.15.107:0/2804042693 >> 172.21.15.107:0/2804042693 conn(0x7fe33c0feaa0 msgr2=0x7fe33c100ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:00:08.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:08.889+0000 7fe344600640 1 -- 172.21.15.107:0/2804042693 shutdown_connections 2024-03-08T21:00:08.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:08.889+0000 7fe344600640 1 -- 172.21.15.107:0/2804042693 wait complete. 2024-03-08T21:00:08.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:08.890+0000 7fe344600640 1 Processor -- start 2024-03-08T21:00:08.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:08.890+0000 7fe344600640 1 -- start start 2024-03-08T21:00:08.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:08.890+0000 7fe344600640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe33c103510 0x7fe33c1996a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:00:08.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:08.890+0000 7fe344600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe33c104700 0x7fe33c199be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:00:08.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:08.890+0000 7fe344600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe33c1050c0 0x7fe33c1a0c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:00:08.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:08.890+0000 7fe344600640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe33c10f0a0 con 0x7fe33c1050c0 2024-03-08T21:00:08.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:08.890+0000 7fe344600640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe33c10ef20 con 0x7fe33c103510 2024-03-08T21:00:08.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:08.890+0000 7fe344600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe33c10f220 con 0x7fe33c104700 2024-03-08T21:00:08.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:08.891+0000 7fe342600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe33c1050c0 0x7fe33c1a0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:00:08.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:08.891+0000 7fe342600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe33c1050c0 0x7fe33c1a0c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:56588/0 (socket says 172.21.15.107:56588) 2024-03-08T21:00:08.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:08.891+0000 7fe342600640 1 -- 172.21.15.107:0/2488533707 learned_addr learned my addr 172.21.15.107:0/2488533707 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:00:08.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:08.891+0000 7fe341c00640 1 --2- 172.21.15.107:0/2488533707 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe33c103510 0x7fe33c1996a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:00:08.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:08.891+0000 7fe341200640 1 --2- 172.21.15.107:0/2488533707 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe33c104700 0x7fe33c199be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:00:08.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:08.891+0000 7fe342600640 1 -- 172.21.15.107:0/2488533707 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe33c104700 msgr2=0x7fe33c199be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:00:08.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:08.891+0000 7fe342600640 1 --2- 172.21.15.107:0/2488533707 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe33c104700 0x7fe33c199be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:08.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:08.891+0000 7fe342600640 1 -- 172.21.15.107:0/2488533707 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe33c103510 msgr2=0x7fe33c1996a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:00:08.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:08.891+0000 7fe342600640 1 --2- 172.21.15.107:0/2488533707 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe33c103510 0x7fe33c1996a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:08.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:08.891+0000 7fe342600640 1 -- 172.21.15.107:0/2488533707 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe33c1a12c0 con 0x7fe33c1050c0 2024-03-08T21:00:08.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:08.891+0000 7fe341c00640 1 --2- 172.21.15.107:0/2488533707 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe33c103510 0x7fe33c1996a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T21:00:08.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:08.891+0000 7fe342600640 1 --2- 172.21.15.107:0/2488533707 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe33c1050c0 0x7fe33c1a0c50 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fe33400eaa0 tx=0x7fe33400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:00:08.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:08.892+0000 7fe33a000640 1 -- 172.21.15.107:0/2488533707 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe33400cdb0 con 0x7fe33c1050c0 2024-03-08T21:00:08.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:08.892+0000 7fe344600640 1 -- 172.21.15.107:0/2488533707 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe33c1a15b0 con 0x7fe33c1050c0 2024-03-08T21:00:08.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:08.892+0000 7fe33a000640 1 -- 172.21.15.107:0/2488533707 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fe334005020 con 0x7fe33c1050c0 2024-03-08T21:00:08.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:08.892+0000 7fe33a000640 1 -- 172.21.15.107:0/2488533707 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3340116d0 con 0x7fe33c1050c0 2024-03-08T21:00:08.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:08.892+0000 7fe344600640 1 -- 172.21.15.107:0/2488533707 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe33c1a1ab0 con 0x7fe33c1050c0 2024-03-08T21:00:08.878 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:00:08 smithi204 ceph-mon[34935]: from='client.? 172.21.15.107:0/228591807' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-08T21:00:08.878 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:00:08 smithi204 ceph-mon[34935]: from='client.? 172.21.15.107:0/3282863309' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-08T21:00:08.878 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:08.893+0000 7fe344600640 1 -- 172.21.15.107:0/2488533707 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe33c10a7d0 con 0x7fe33c1050c0 2024-03-08T21:00:08.879 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:08.895+0000 7fe33a000640 1 -- 172.21.15.107:0/2488533707 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) v1 ==== 137291+0+0 (secure 0 0 0) 0x7fe3340051d0 con 0x7fe33c1050c0 2024-03-08T21:00:08.880 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:08.897+0000 7fe33a000640 1 --2- 172.21.15.107:0/2488533707 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fe3240a2800 0x7fe3240a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:00:08.880 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:08.897+0000 7fe33a000640 1 -- 172.21.15.107:0/2488533707 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fe3340ced60 con 0x7fe33c1050c0 2024-03-08T21:00:08.880 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:08.897+0000 7fe341c00640 1 --2- 172.21.15.107:0/2488533707 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fe3240a2800 0x7fe3240a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:00:08.881 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:08.898+0000 7fe341c00640 1 --2- 172.21.15.107:0/2488533707 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fe3240a2800 0x7fe3240a4cc0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fe330036040 tx=0x7fe330005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:00:08.884 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:08.901+0000 7fe33a000640 1 -- 172.21.15.107:0/2488533707 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7fe334016030 con 0x7fe33c1050c0 2024-03-08T21:00:08.967 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:00:08 smithi107 ceph-mon[31381]: from='client.? 172.21.15.107:0/228591807' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-08T21:00:08.967 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:00:08 smithi107 ceph-mon[31381]: from='client.? 172.21.15.107:0/3282863309' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-08T21:00:09.014 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:00:08 smithi149 ceph-mon[35042]: from='client.? 172.21.15.107:0/228591807' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-08T21:00:09.015 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:00:08 smithi149 ceph-mon[35042]: from='client.? 172.21.15.107:0/3282863309' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-08T21:00:09.081 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:09.098+0000 7fe344600640 1 -- 172.21.15.107:0/2488533707 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe33c061ef0 con 0x7fe33c1050c0 2024-03-08T21:00:09.081 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:09.098+0000 7fe33a000640 1 -- 172.21.15.107:0/2488533707 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe33409bdc0 con 0x7fe33c1050c0 2024-03-08T21:00:09.082 INFO:teuthology.orchestra.run.smithi107.stdout:pg_num: 32 2024-03-08T21:00:09.084 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:09.101+0000 7fe344600640 1 -- 172.21.15.107:0/2488533707 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fe3240a2800 msgr2=0x7fe3240a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:00:09.084 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:09.101+0000 7fe344600640 1 --2- 172.21.15.107:0/2488533707 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fe3240a2800 0x7fe3240a4cc0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fe330036040 tx=0x7fe330005fd0 comp rx=0 tx=0).stop 2024-03-08T21:00:09.084 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:09.101+0000 7fe344600640 1 -- 172.21.15.107:0/2488533707 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe33c1050c0 msgr2=0x7fe33c1a0c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:00:09.084 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:09.101+0000 7fe344600640 1 --2- 172.21.15.107:0/2488533707 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe33c1050c0 0x7fe33c1a0c50 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fe33400eaa0 tx=0x7fe33400ef60 comp rx=0 tx=0).stop 2024-03-08T21:00:09.084 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:09.101+0000 7fe344600640 1 -- 172.21.15.107:0/2488533707 shutdown_connections 2024-03-08T21:00:09.084 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:09.101+0000 7fe344600640 1 --2- 172.21.15.107:0/2488533707 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fe3240a2800 0x7fe3240a4cc0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:09.084 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:09.101+0000 7fe344600640 1 --2- 172.21.15.107:0/2488533707 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe33c1050c0 0x7fe33c1a0c50 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:09.085 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:09.101+0000 7fe344600640 1 --2- 172.21.15.107:0/2488533707 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe33c104700 0x7fe33c199be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:09.085 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:09.101+0000 7fe344600640 1 --2- 172.21.15.107:0/2488533707 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe33c103510 0x7fe33c1996a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:09.085 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:09.101+0000 7fe344600640 1 -- 172.21.15.107:0/2488533707 >> 172.21.15.107:0/2488533707 conn(0x7fe33c0feaa0 msgr2=0x7fe33c1079a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:00:09.085 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:09.101+0000 7fe344600640 1 -- 172.21.15.107:0/2488533707 shutdown_connections 2024-03-08T21:00:09.085 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:09.102+0000 7fe344600640 1 -- 172.21.15.107:0/2488533707 wait complete. 2024-03-08T21:00:09.101 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-08T21:00:09.308 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:09.324+0000 7f338f200640 1 -- 172.21.15.107:0/4056047915 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3390077890 msgr2=0x7f3390078130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:00:09.308 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:09.324+0000 7f338f200640 1 --2- 172.21.15.107:0/4056047915 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3390077890 0x7f3390078130 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f3388009ca0 tx=0x7f338802c100 comp rx=0 tx=0).stop 2024-03-08T21:00:09.308 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:09.324+0000 7f338f200640 1 -- 172.21.15.107:0/4056047915 shutdown_connections 2024-03-08T21:00:09.308 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:09.324+0000 7f338f200640 1 --2- 172.21.15.107:0/4056047915 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3390077890 0x7f3390078130 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:09.308 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:09.324+0000 7f338f200640 1 --2- 172.21.15.107:0/4056047915 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3390076ef0 0x7f3390077350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:09.308 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:09.324+0000 7f338f200640 1 --2- 172.21.15.107:0/4056047915 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3390075cb0 0x7f33900760d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:09.308 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:09.324+0000 7f338f200640 1 -- 172.21.15.107:0/4056047915 >> 172.21.15.107:0/4056047915 conn(0x7f33900fe960 msgr2=0x7f3390100da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:00:09.308 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:09.324+0000 7f338f200640 1 -- 172.21.15.107:0/4056047915 shutdown_connections 2024-03-08T21:00:09.308 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:09.324+0000 7f338f200640 1 -- 172.21.15.107:0/4056047915 wait complete. 2024-03-08T21:00:09.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:09.325+0000 7f338f200640 1 Processor -- start 2024-03-08T21:00:09.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:09.326+0000 7f338f200640 1 -- start start 2024-03-08T21:00:09.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:09.326+0000 7f338f200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3390075cb0 0x7f339019d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:00:09.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:09.326+0000 7f338f200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3390076ef0 0x7f339019de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:00:09.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:09.326+0000 7f338f200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3390077890 0x7f33901a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:00:09.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:09.326+0000 7f338f200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3390113330 con 0x7f3390077890 2024-03-08T21:00:09.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:09.326+0000 7f338f200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33901131b0 con 0x7f3390075cb0 2024-03-08T21:00:09.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:09.326+0000 7f338f200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33901134b0 con 0x7f3390076ef0 2024-03-08T21:00:09.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:09.326+0000 7f338e800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3390077890 0x7f33901a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:00:09.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:09.326+0000 7f338e800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3390077890 0x7f33901a4ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:56596/0 (socket says 172.21.15.107:56596) 2024-03-08T21:00:09.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:09.326+0000 7f338e800640 1 -- 172.21.15.107:0/682873847 learned_addr learned my addr 172.21.15.107:0/682873847 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:00:09.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:09.326+0000 7f338d400640 1 --2- 172.21.15.107:0/682873847 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3390076ef0 0x7f339019de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:00:09.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:09.326+0000 7f338de00640 1 --2- 172.21.15.107:0/682873847 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3390075cb0 0x7f339019d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:00:09.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:09.326+0000 7f338e800640 1 -- 172.21.15.107:0/682873847 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3390076ef0 msgr2=0x7f339019de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:00:09.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:09.326+0000 7f338e800640 1 --2- 172.21.15.107:0/682873847 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3390076ef0 0x7f339019de60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:09.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:09.326+0000 7f338e800640 1 -- 172.21.15.107:0/682873847 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3390075cb0 msgr2=0x7f339019d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:00:09.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:09.326+0000 7f338e800640 1 --2- 172.21.15.107:0/682873847 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3390075cb0 0x7f339019d920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:09.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:09.326+0000 7f338e800640 1 -- 172.21.15.107:0/682873847 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33901a5540 con 0x7f3390077890 2024-03-08T21:00:09.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:09.327+0000 7f338de00640 1 --2- 172.21.15.107:0/682873847 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3390075cb0 0x7f339019d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T21:00:09.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:09.327+0000 7f338e800640 1 --2- 172.21.15.107:0/682873847 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3390077890 0x7f33901a4ed0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f338802c710 tx=0x7f3388035dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:00:09.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:09.327+0000 7f3386a00640 1 -- 172.21.15.107:0/682873847 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3388039040 con 0x7f3390077890 2024-03-08T21:00:09.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:09.327+0000 7f3386a00640 1 -- 172.21.15.107:0/682873847 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f3388044070 con 0x7f3390077890 2024-03-08T21:00:09.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:09.327+0000 7f338f200640 1 -- 172.21.15.107:0/682873847 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33901a57d0 con 0x7f3390077890 2024-03-08T21:00:09.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:09.327+0000 7f3386a00640 1 -- 172.21.15.107:0/682873847 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33880046f0 con 0x7f3390077890 2024-03-08T21:00:09.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:09.327+0000 7f338f200640 1 -- 172.21.15.107:0/682873847 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33901a5d10 con 0x7f3390077890 2024-03-08T21:00:09.313 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:09.328+0000 7f338f200640 1 -- 172.21.15.107:0/682873847 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f339010fea0 con 0x7f3390077890 2024-03-08T21:00:09.313 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:09.330+0000 7f3386a00640 1 -- 172.21.15.107:0/682873847 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f3388003970 con 0x7f3390077890 2024-03-08T21:00:09.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:09.332+0000 7f3386a00640 1 --2- 172.21.15.107:0/682873847 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f33740a2880 0x7f33740a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:00:09.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:09.332+0000 7f3386a00640 1 -- 172.21.15.107:0/682873847 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f33880f1420 con 0x7f3390077890 2024-03-08T21:00:09.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:09.332+0000 7f338de00640 1 --2- 172.21.15.107:0/682873847 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f33740a2880 0x7f33740a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:00:09.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:09.333+0000 7f338de00640 1 --2- 172.21.15.107:0/682873847 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f33740a2880 0x7f33740a4d40 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f33800058c0 tx=0x7f3380005d30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:00:09.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:09.336+0000 7f3386a00640 1 -- 172.21.15.107:0/682873847 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f33880098b0 con 0x7f3390077890 2024-03-08T21:00:09.505 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:09.521+0000 7f338f200640 1 -- 172.21.15.107:0/682873847 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f3390061ef0 con 0x7f3390077890 2024-03-08T21:00:09.505 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:09.522+0000 7f3386a00640 1 -- 172.21.15.107:0/682873847 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) v1 ==== 100+0+11 (secure 0 0 0) 0x7f33880be480 con 0x7f3390077890 2024-03-08T21:00:09.505 INFO:teuthology.orchestra.run.smithi107.stdout:pg_num: 64 2024-03-08T21:00:09.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:09.524+0000 7f338f200640 1 -- 172.21.15.107:0/682873847 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f33740a2880 msgr2=0x7f33740a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:00:09.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:09.524+0000 7f338f200640 1 --2- 172.21.15.107:0/682873847 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f33740a2880 0x7f33740a4d40 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f33800058c0 tx=0x7f3380005d30 comp rx=0 tx=0).stop 2024-03-08T21:00:09.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:09.524+0000 7f338f200640 1 -- 172.21.15.107:0/682873847 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3390077890 msgr2=0x7f33901a4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:00:09.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:09.524+0000 7f338f200640 1 --2- 172.21.15.107:0/682873847 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3390077890 0x7f33901a4ed0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f338802c710 tx=0x7f3388035dd0 comp rx=0 tx=0).stop 2024-03-08T21:00:09.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:09.524+0000 7f338f200640 1 -- 172.21.15.107:0/682873847 shutdown_connections 2024-03-08T21:00:09.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:09.524+0000 7f338f200640 1 --2- 172.21.15.107:0/682873847 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f33740a2880 0x7f33740a4d40 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:09.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:09.524+0000 7f338f200640 1 --2- 172.21.15.107:0/682873847 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3390077890 0x7f33901a4ed0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:09.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:09.525+0000 7f338f200640 1 --2- 172.21.15.107:0/682873847 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3390076ef0 0x7f339019de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:09.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:09.525+0000 7f338f200640 1 --2- 172.21.15.107:0/682873847 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3390075cb0 0x7f339019d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:09.509 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:09.525+0000 7f338f200640 1 -- 172.21.15.107:0/682873847 >> 172.21.15.107:0/682873847 conn(0x7f33900fe960 msgr2=0x7f3390100030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:00:09.509 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:09.525+0000 7f338f200640 1 -- 172.21.15.107:0/682873847 shutdown_connections 2024-03-08T21:00:09.509 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:09.525+0000 7f338f200640 1 -- 172.21.15.107:0/682873847 wait complete. 2024-03-08T21:00:09.527 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-08T21:00:09.736 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:09.753+0000 7f9ea0c00640 1 -- 172.21.15.107:0/3938098638 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9e9c108940 msgr2=0x7f9e9c108dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:00:09.736 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:09.753+0000 7f9ea0c00640 1 --2- 172.21.15.107:0/3938098638 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9e9c108940 0x7f9e9c108dc0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f9e84009900 tx=0x7f9e8402bc40 comp rx=0 tx=0).stop 2024-03-08T21:00:09.736 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:09.753+0000 7f9ea0c00640 1 -- 172.21.15.107:0/3938098638 shutdown_connections 2024-03-08T21:00:09.736 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:09.753+0000 7f9ea0c00640 1 --2- 172.21.15.107:0/3938098638 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9e9c109300 0x7f9e9c10fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:09.737 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:09.753+0000 7f9ea0c00640 1 --2- 172.21.15.107:0/3938098638 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9e9c108940 0x7f9e9c108dc0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:09.737 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:09.753+0000 7f9ea0c00640 1 --2- 172.21.15.107:0/3938098638 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9e9c107750 0x7f9e9c107b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:09.737 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:09.753+0000 7f9ea0c00640 1 -- 172.21.15.107:0/3938098638 >> 172.21.15.107:0/3938098638 conn(0x7f9e9c076700 msgr2=0x7f9e9c078b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:00:09.737 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:09.753+0000 7f9ea0c00640 1 -- 172.21.15.107:0/3938098638 shutdown_connections 2024-03-08T21:00:09.737 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:09.753+0000 7f9ea0c00640 1 -- 172.21.15.107:0/3938098638 wait complete. 2024-03-08T21:00:09.739 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:09.754+0000 7f9ea0c00640 1 Processor -- start 2024-03-08T21:00:09.739 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:09.754+0000 7f9ea0c00640 1 -- start start 2024-03-08T21:00:09.739 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:09.754+0000 7f9ea0c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9e9c107750 0x7f9e9c19d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:00:09.739 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:09.754+0000 7f9ea0c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9e9c108940 0x7f9e9c19def0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:00:09.739 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:09.754+0000 7f9ea0c00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9e9c109300 0x7f9e9c1a4f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:00:09.739 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:09.754+0000 7f9ea0c00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e9c1132e0 con 0x7f9e9c107750 2024-03-08T21:00:09.739 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:09.754+0000 7f9ea0c00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e9c113160 con 0x7f9e9c109300 2024-03-08T21:00:09.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:09.754+0000 7f9ea0c00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e9c113460 con 0x7f9e9c108940 2024-03-08T21:00:09.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:09.754+0000 7f9e9b400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9e9c107750 0x7f9e9c19d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:00:09.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:09.754+0000 7f9e9b400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9e9c107750 0x7f9e9c19d9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:56600/0 (socket says 172.21.15.107:56600) 2024-03-08T21:00:09.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:09.754+0000 7f9e9b400640 1 -- 172.21.15.107:0/1495107928 learned_addr learned my addr 172.21.15.107:0/1495107928 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:00:09.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:09.755+0000 7f9e9b400640 1 -- 172.21.15.107:0/1495107928 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9e9c108940 msgr2=0x7f9e9c19def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:00:09.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:09.755+0000 7f9e93e00640 1 --2- 172.21.15.107:0/1495107928 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9e9c108940 0x7f9e9c19def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:00:09.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:09.755+0000 7f9e9b400640 1 --2- 172.21.15.107:0/1495107928 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9e9c108940 0x7f9e9c19def0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:09.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:09.755+0000 7f9e9be00640 1 --2- 172.21.15.107:0/1495107928 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9e9c109300 0x7f9e9c1a4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:00:09.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:09.755+0000 7f9e9b400640 1 -- 172.21.15.107:0/1495107928 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9e9c109300 msgr2=0x7f9e9c1a4f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:00:09.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:09.755+0000 7f9e9b400640 1 --2- 172.21.15.107:0/1495107928 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9e9c109300 0x7f9e9c1a4f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:09.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:09.755+0000 7f9e9b400640 1 -- 172.21.15.107:0/1495107928 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e9c1a55d0 con 0x7f9e9c107750 2024-03-08T21:00:09.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:09.755+0000 7f9e9b400640 1 --2- 172.21.15.107:0/1495107928 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9e9c107750 0x7f9e9c19d9b0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f9e8c00c970 tx=0x7f9e8c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:00:09.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:09.755+0000 7f9e98c00640 1 -- 172.21.15.107:0/1495107928 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e8c004220 con 0x7f9e9c107750 2024-03-08T21:00:09.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:09.755+0000 7f9e98c00640 1 -- 172.21.15.107:0/1495107928 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f9e8c0043c0 con 0x7f9e9c107750 2024-03-08T21:00:09.741 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:09.755+0000 7f9ea0c00640 1 -- 172.21.15.107:0/1495107928 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e9c1a58c0 con 0x7f9e9c107750 2024-03-08T21:00:09.741 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:09.755+0000 7f9e98c00640 1 -- 172.21.15.107:0/1495107928 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e8c00f450 con 0x7f9e9c107750 2024-03-08T21:00:09.741 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:09.756+0000 7f9ea0c00640 1 -- 172.21.15.107:0/1495107928 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e9c1a5ee0 con 0x7f9e9c107750 2024-03-08T21:00:09.741 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:09.757+0000 7f9ea0c00640 1 -- 172.21.15.107:0/1495107928 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e9c10eb40 con 0x7f9e9c107750 2024-03-08T21:00:09.741 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:09.759+0000 7f9e98c00640 1 -- 172.21.15.107:0/1495107928 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f9e8c0079b0 con 0x7f9e9c107750 2024-03-08T21:00:09.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:09.760+0000 7f9e98c00640 1 --2- 172.21.15.107:0/1495107928 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f9e800a2880 0x7f9e800a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:00:09.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:09.760+0000 7f9e98c00640 1 -- 172.21.15.107:0/1495107928 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f9e8c0ce870 con 0x7f9e9c107750 2024-03-08T21:00:09.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:09.760+0000 7f9e93e00640 1 --2- 172.21.15.107:0/1495107928 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f9e800a2880 0x7f9e800a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:00:09.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:09.761+0000 7f9e93e00640 1 --2- 172.21.15.107:0/1495107928 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f9e800a2880 0x7f9e800a4d40 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f9e84036040 tx=0x7f9e84000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:00:09.747 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:09.764+0000 7f9e98c00640 1 -- 172.21.15.107:0/1495107928 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f9e8c014030 con 0x7f9e9c107750 2024-03-08T21:00:09.878 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:00:09 smithi204 ceph-mon[34935]: pgmap v432: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-03-08T21:00:09.878 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:00:09 smithi204 ceph-mon[34935]: from='client.? 172.21.15.107:0/3545772356' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-08T21:00:09.878 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:00:09 smithi204 ceph-mon[34935]: from='client.? 172.21.15.107:0/2488533707' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-08T21:00:09.878 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:00:09 smithi204 ceph-mon[34935]: from='client.? 172.21.15.107:0/682873847' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-08T21:00:09.967 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:00:09 smithi107 ceph-mon[31381]: pgmap v432: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-03-08T21:00:09.967 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:00:09 smithi107 ceph-mon[31381]: from='client.? 172.21.15.107:0/3545772356' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-08T21:00:09.967 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:00:09 smithi107 ceph-mon[31381]: from='client.? 172.21.15.107:0/2488533707' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-08T21:00:09.967 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:00:09 smithi107 ceph-mon[31381]: from='client.? 172.21.15.107:0/682873847' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-08T21:00:09.990 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.007+0000 7f9ea0c00640 1 -- 172.21.15.107:0/1495107928 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9e9c02ce70 con 0x7f9e9c107750 2024-03-08T21:00:09.991 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.008+0000 7f9e98c00640 1 -- 172.21.15.107:0/1495107928 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+11393 (secure 0 0 0) 0x7f9e8c09b8d0 con 0x7f9e9c107750 2024-03-08T21:00:09.992 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-08T21:00:09.992 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14538,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/133582743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":133582743},{"type":"v1","addr":"172.21.15.107:6835","nonce":133582743}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/3798982337","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":3798982337},{"type":"v1","addr":"172.21.15.107:6837","nonce":3798982337}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-08T20:57:50.223752+0000","modified":"2024-03-08T20:57:57.874727+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34475,"mds_1":34445,"mds_2":34415,"mds_3":34397,"mds_4":24457},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14625":{"gid":14625,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6839/3070059000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":3070059000},{"type":"v1","addr":"172.21.15.107:6839","nonce":3070059000}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14637":{"gid":14637,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.107:6841/747096194","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6840","nonce":747096194},{"type":"v1","addr":"172.21.15.107:6841","nonce":747096194}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24400":{"gid":24400,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6833/3404762782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6832","nonce":3404762782},{"type":"v1","addr":"172.21.15.204:6833","nonce":3404762782}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24430":{"gid":24430,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6835/2072518651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6834","nonce":2072518651},{"type":"v1","addr":"172.21.15.204:6835","nonce":2072518651}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24442":{"gid":24442,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.204:6837/42644905","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6836","nonce":42644905},{"type":"v1","addr":"172.21.15.204:6837","nonce":42644905}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24457":{"gid":24457,"name":"i","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.204:6839/1857316822","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6838","nonce":1857316822},{"type":"v1","addr":"172.21.15.204:6839","nonce":1857316822}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34397":{"gid":34397,"name":"k","rank":3,"incarnation":23,"state":"up:active","state_seq":14,"addr":"172.21.15.149:6833/1962161726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":1962161726},{"type":"v1","addr":"172.21.15.149:6833","nonce":1962161726}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34415":{"gid":34415,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.149:6835/442657066","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":442657066},{"type":"v1","addr":"172.21.15.149:6835","nonce":442657066}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34445":{"gid":34445,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.149:6837/914546237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":914546237},{"type":"v1","addr":"172.21.15.149:6837","nonce":914546237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34475":{"gid":34475,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.149:6839/55290596","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6838","nonce":55290596},{"type":"v1","addr":"172.21.15.149:6839","nonce":55290596}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-08T21:00:09.992 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 27 2024-03-08T21:00:09.995 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.012+0000 7f9ea0c00640 1 -- 172.21.15.107:0/1495107928 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f9e800a2880 msgr2=0x7f9e800a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:00:09.995 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.012+0000 7f9ea0c00640 1 --2- 172.21.15.107:0/1495107928 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f9e800a2880 0x7f9e800a4d40 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f9e84036040 tx=0x7f9e84000bc0 comp rx=0 tx=0).stop 2024-03-08T21:00:09.995 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.012+0000 7f9ea0c00640 1 -- 172.21.15.107:0/1495107928 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9e9c107750 msgr2=0x7f9e9c19d9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:00:09.995 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.012+0000 7f9ea0c00640 1 --2- 172.21.15.107:0/1495107928 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9e9c107750 0x7f9e9c19d9b0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f9e8c00c970 tx=0x7f9e8c00ce30 comp rx=0 tx=0).stop 2024-03-08T21:00:09.995 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.012+0000 7f9ea0c00640 1 -- 172.21.15.107:0/1495107928 shutdown_connections 2024-03-08T21:00:09.996 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.012+0000 7f9ea0c00640 1 --2- 172.21.15.107:0/1495107928 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f9e800a2880 0x7f9e800a4d40 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:09.996 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.012+0000 7f9ea0c00640 1 --2- 172.21.15.107:0/1495107928 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9e9c109300 0x7f9e9c1a4f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:09.996 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.012+0000 7f9ea0c00640 1 --2- 172.21.15.107:0/1495107928 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9e9c108940 0x7f9e9c19def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:09.996 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.012+0000 7f9ea0c00640 1 --2- 172.21.15.107:0/1495107928 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9e9c107750 0x7f9e9c19d9b0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:09.996 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.012+0000 7f9ea0c00640 1 -- 172.21.15.107:0/1495107928 >> 172.21.15.107:0/1495107928 conn(0x7f9e9c076700 msgr2=0x7f9e9c10bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:00:09.996 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.012+0000 7f9ea0c00640 1 -- 172.21.15.107:0/1495107928 shutdown_connections 2024-03-08T21:00:09.996 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.012+0000 7f9ea0c00640 1 -- 172.21.15.107:0/1495107928 wait complete. 2024-03-08T21:00:10.014 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:00:09 smithi149 ceph-mon[35042]: pgmap v432: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-03-08T21:00:10.014 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:00:09 smithi149 ceph-mon[35042]: from='client.? 172.21.15.107:0/3545772356' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-08T21:00:10.014 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:00:09 smithi149 ceph-mon[35042]: from='client.? 172.21.15.107:0/2488533707' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-08T21:00:10.015 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:00:09 smithi149 ceph-mon[35042]: from='client.? 172.21.15.107:0/682873847' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-08T21:00:10.016 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-08T21:00:10.226 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.242+0000 7fdb51200640 1 -- 172.21.15.107:0/1644678267 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdb4c103530 msgr2=0x7fdb4c103950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:00:10.226 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.242+0000 7fdb51200640 1 --2- 172.21.15.107:0/1644678267 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdb4c103530 0x7fdb4c103950 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fdb3c009900 tx=0x7fdb3c02bc30 comp rx=0 tx=0).stop 2024-03-08T21:00:10.226 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.242+0000 7fdb51200640 1 -- 172.21.15.107:0/1644678267 shutdown_connections 2024-03-08T21:00:10.226 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.242+0000 7fdb51200640 1 --2- 172.21.15.107:0/1644678267 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdb4c1050e0 0x7fdb4c10b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:10.226 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.242+0000 7fdb51200640 1 --2- 172.21.15.107:0/1644678267 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdb4c104720 0x7fdb4c104ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:10.226 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.242+0000 7fdb51200640 1 --2- 172.21.15.107:0/1644678267 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdb4c103530 0x7fdb4c103950 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:10.226 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.242+0000 7fdb51200640 1 -- 172.21.15.107:0/1644678267 >> 172.21.15.107:0/1644678267 conn(0x7fdb4c0feae0 msgr2=0x7fdb4c100f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:00:10.226 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.242+0000 7fdb51200640 1 -- 172.21.15.107:0/1644678267 shutdown_connections 2024-03-08T21:00:10.226 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.242+0000 7fdb51200640 1 -- 172.21.15.107:0/1644678267 wait complete. 2024-03-08T21:00:10.228 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.243+0000 7fdb51200640 1 Processor -- start 2024-03-08T21:00:10.228 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.243+0000 7fdb51200640 1 -- start start 2024-03-08T21:00:10.229 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.244+0000 7fdb51200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdb4c103530 0x7fdb4c199730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:00:10.229 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.244+0000 7fdb51200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdb4c104720 0x7fdb4c199c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:00:10.229 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.244+0000 7fdb51200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdb4c1050e0 0x7fdb4c1a0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:00:10.229 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.244+0000 7fdb51200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb4c10f070 con 0x7fdb4c104720 2024-03-08T21:00:10.229 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.244+0000 7fdb51200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb4c10eef0 con 0x7fdb4c103530 2024-03-08T21:00:10.229 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.244+0000 7fdb51200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb4c10f1f0 con 0x7fdb4c1050e0 2024-03-08T21:00:10.229 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.244+0000 7fdb41400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdb4c104720 0x7fdb4c199c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:00:10.229 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.244+0000 7fdb41400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdb4c104720 0x7fdb4c199c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:56614/0 (socket says 172.21.15.107:56614) 2024-03-08T21:00:10.229 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.244+0000 7fdb41400640 1 -- 172.21.15.107:0/714023804 learned_addr learned my addr 172.21.15.107:0/714023804 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:00:10.229 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.244+0000 7fdb49e00640 1 --2- 172.21.15.107:0/714023804 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdb4c103530 0x7fdb4c199730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:00:10.229 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.244+0000 7fdb4a800640 1 --2- 172.21.15.107:0/714023804 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdb4c1050e0 0x7fdb4c1a0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:00:10.229 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.244+0000 7fdb41400640 1 -- 172.21.15.107:0/714023804 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdb4c1050e0 msgr2=0x7fdb4c1a0ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:00:10.229 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.244+0000 7fdb41400640 1 --2- 172.21.15.107:0/714023804 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdb4c1050e0 0x7fdb4c1a0ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:10.230 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.244+0000 7fdb41400640 1 -- 172.21.15.107:0/714023804 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdb4c103530 msgr2=0x7fdb4c199730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:00:10.230 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.244+0000 7fdb41400640 1 --2- 172.21.15.107:0/714023804 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdb4c103530 0x7fdb4c199730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:10.230 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.244+0000 7fdb41400640 1 -- 172.21.15.107:0/714023804 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb4c1a1350 con 0x7fdb4c104720 2024-03-08T21:00:10.230 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.245+0000 7fdb41400640 1 --2- 172.21.15.107:0/714023804 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdb4c104720 0x7fdb4c199c70 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fdb3400d770 tx=0x7fdb3400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:00:10.230 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.245+0000 7fdb43400640 1 -- 172.21.15.107:0/714023804 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb34017070 con 0x7fdb4c104720 2024-03-08T21:00:10.230 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.245+0000 7fdb51200640 1 -- 172.21.15.107:0/714023804 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb4c1a1640 con 0x7fdb4c104720 2024-03-08T21:00:10.230 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.245+0000 7fdb43400640 1 -- 172.21.15.107:0/714023804 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fdb340054d0 con 0x7fdb4c104720 2024-03-08T21:00:10.230 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.245+0000 7fdb51200640 1 -- 172.21.15.107:0/714023804 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb4c1a1b40 con 0x7fdb4c104720 2024-03-08T21:00:10.230 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.245+0000 7fdb43400640 1 -- 172.21.15.107:0/714023804 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb34020060 con 0x7fdb4c104720 2024-03-08T21:00:10.231 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.246+0000 7fdb51200640 1 -- 172.21.15.107:0/714023804 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb4c10a8d0 con 0x7fdb4c104720 2024-03-08T21:00:10.231 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.248+0000 7fdb43400640 1 -- 172.21.15.107:0/714023804 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) v1 ==== 137291+0+0 (secure 0 0 0) 0x7fdb34004c90 con 0x7fdb4c104720 2024-03-08T21:00:10.232 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.249+0000 7fdb43400640 1 --2- 172.21.15.107:0/714023804 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fdb300a2800 0x7fdb300a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:00:10.233 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.250+0000 7fdb43400640 1 -- 172.21.15.107:0/714023804 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fdb340ce670 con 0x7fdb4c104720 2024-03-08T21:00:10.233 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.250+0000 7fdb49e00640 1 --2- 172.21.15.107:0/714023804 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fdb300a2800 0x7fdb300a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:00:10.233 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.251+0000 7fdb49e00640 1 --2- 172.21.15.107:0/714023804 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fdb300a2800 0x7fdb300a4cc0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fdb3c006010 tx=0x7fdb3c005be0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:00:10.237 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.254+0000 7fdb43400640 1 -- 172.21.15.107:0/714023804 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7fdb3409b6d0 con 0x7fdb4c104720 2024-03-08T21:00:10.439 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.456+0000 7fdb51200640 1 -- 172.21.15.107:0/714023804 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdb4c061ef0 con 0x7fdb4c104720 2024-03-08T21:00:10.441 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.458+0000 7fdb43400640 1 -- 172.21.15.107:0/714023804 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+22779 (secure 0 0 0) 0x7fdb3409afd0 con 0x7fdb4c104720 2024-03-08T21:00:10.442 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-08T21:00:10.442 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":88,"fsid":"dc20f0f8-dd8c-11ee-95c6-87774f69a715","created":"2024-03-08T20:47:04.190034+0000","modified":"2024-03-08T20:58:21.449936+0000","last_up_change":"2024-03-08T20:56:43.135514+0000","last_in_change":"2024-03-08T20:56:17.577074+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-08T20:51:32.832786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-08T20:57:48.721227+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-08T20:57:49.707779+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8b1ddc64-4d20-4e85-ac66-f1c5ac599501","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":2453946095},{"type":"v1","addr":"172.21.15.107:6803","nonce":2453946095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":2453946095},{"type":"v1","addr":"172.21.15.107:6805","nonce":2453946095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":2453946095},{"type":"v1","addr":"172.21.15.107:6809","nonce":2453946095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":2453946095},{"type":"v1","addr":"172.21.15.107:6807","nonce":2453946095}]},"public_addr":"172.21.15.107:6803/2453946095","cluster_addr":"172.21.15.107:6805/2453946095","heartbeat_back_addr":"172.21.15.107:6809/2453946095","heartbeat_front_addr":"172.21.15.107:6807/2453946095","state":["exists","up"]},{"osd":1,"uuid":"5a803c09-202a-4777-ba2a-1da8eaf26a54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":3071316419},{"type":"v1","addr":"172.21.15.149:6801","nonce":3071316419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":3071316419},{"type":"v1","addr":"172.21.15.149:6803","nonce":3071316419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":3071316419},{"type":"v1","addr":"172.21.15.149:6807","nonce":3071316419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":3071316419},{"type":"v1","addr":"172.21.15.149:6805","nonce":3071316419}]},"public_addr":"172.21.15.149:6801/3071316419","cluster_addr":"172.21.15.149:6803/3071316419","heartbeat_back_addr":"172.21.15.149:6807/3071316419","heartbeat_front_addr":"172.21.15.149:6805/3071316419","state":["exists","up"]},{"osd":2,"uuid":"555c634f-347e-47dd-a943-e50ae2be073c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":1087852630},{"type":"v1","addr":"172.21.15.204:6801","nonce":1087852630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":1087852630},{"type":"v1","addr":"172.21.15.204:6803","nonce":1087852630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":1087852630},{"type":"v1","addr":"172.21.15.204:6807","nonce":1087852630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":1087852630},{"type":"v1","addr":"172.21.15.204:6805","nonce":1087852630}]},"public_addr":"172.21.15.204:6801/1087852630","cluster_addr":"172.21.15.204:6803/1087852630","heartbeat_back_addr":"172.21.15.204:6807/1087852630","heartbeat_front_addr":"172.21.15.204:6805/1087852630","state":["exists","up"]},{"osd":3,"uuid":"9e433d97-e202-48e0-9ed1-f0988fa14307","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":3123579789},{"type":"v1","addr":"172.21.15.107:6811","nonce":3123579789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":3123579789},{"type":"v1","addr":"172.21.15.107:6813","nonce":3123579789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":3123579789},{"type":"v1","addr":"172.21.15.107:6817","nonce":3123579789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":3123579789},{"type":"v1","addr":"172.21.15.107:6815","nonce":3123579789}]},"public_addr":"172.21.15.107:6811/3123579789","cluster_addr":"172.21.15.107:6813/3123579789","heartbeat_back_addr":"172.21.15.107:6817/3123579789","heartbeat_front_addr":"172.21.15.107:6815/3123579789","state":["exists","up"]},{"osd":4,"uuid":"27c53794-9d2b-487d-bfcf-ac4fd2ab061e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":726286918},{"type":"v1","addr":"172.21.15.149:6809","nonce":726286918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":726286918},{"type":"v1","addr":"172.21.15.149:6811","nonce":726286918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":726286918},{"type":"v1","addr":"172.21.15.149:6815","nonce":726286918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":726286918},{"type":"v1","addr":"172.21.15.149:6813","nonce":726286918}]},"public_addr":"172.21.15.149:6809/726286918","cluster_addr":"172.21.15.149:6811/726286918","heartbeat_back_addr":"172.21.15.149:6815/726286918","heartbeat_front_addr":"172.21.15.149:6813/726286918","state":["exists","up"]},{"osd":5,"uuid":"3a7778fb-46c5-40c0-842e-e988c45ea24d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":3143887820},{"type":"v1","addr":"172.21.15.204:6809","nonce":3143887820}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":3143887820},{"type":"v1","addr":"172.21.15.204:6811","nonce":3143887820}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":3143887820},{"type":"v1","addr":"172.21.15.204:6815","nonce":3143887820}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":3143887820},{"type":"v1","addr":"172.21.15.204:6813","nonce":3143887820}]},"public_addr":"172.21.15.204:6809/3143887820","cluster_addr":"172.21.15.204:6811/3143887820","heartbeat_back_addr":"172.21.15.204:6815/3143887820","heartbeat_front_addr":"172.21.15.204:6813/3143887820","state":["exists","up"]},{"osd":6,"uuid":"d2e05a56-be4f-4435-9b76-ff9aa01e4ac3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":2912452950},{"type":"v1","addr":"172.21.15.107:6819","nonce":2912452950}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":2912452950},{"type":"v1","addr":"172.21.15.107:6821","nonce":2912452950}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":2912452950},{"type":"v1","addr":"172.21.15.107:6825","nonce":2912452950}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":2912452950},{"type":"v1","addr":"172.21.15.107:6823","nonce":2912452950}]},"public_addr":"172.21.15.107:6819/2912452950","cluster_addr":"172.21.15.107:6821/2912452950","heartbeat_back_addr":"172.21.15.107:6825/2912452950","heartbeat_front_addr":"172.21.15.107:6823/2912452950","state":["exists","up"]},{"osd":7,"uuid":"a7dd26ee-c118-4a9a-9e19-142c2680902d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":3835863809},{"type":"v1","addr":"172.21.15.149:6817","nonce":3835863809}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":3835863809},{"type":"v1","addr":"172.21.15.149:6819","nonce":3835863809}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":3835863809},{"type":"v1","addr":"172.21.15.149:6823","nonce":3835863809}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":3835863809},{"type":"v1","addr":"172.21.15.149:6821","nonce":3835863809}]},"public_addr":"172.21.15.149:6817/3835863809","cluster_addr":"172.21.15.149:6819/3835863809","heartbeat_back_addr":"172.21.15.149:6823/3835863809","heartbeat_front_addr":"172.21.15.149:6821/3835863809","state":["exists","up"]},{"osd":8,"uuid":"baaaa8dd-0e1f-4bae-86bf-9e429d985f23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":1010601477},{"type":"v1","addr":"172.21.15.204:6817","nonce":1010601477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":1010601477},{"type":"v1","addr":"172.21.15.204:6819","nonce":1010601477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":1010601477},{"type":"v1","addr":"172.21.15.204:6823","nonce":1010601477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":1010601477},{"type":"v1","addr":"172.21.15.204:6821","nonce":1010601477}]},"public_addr":"172.21.15.204:6817/1010601477","cluster_addr":"172.21.15.204:6819/1010601477","heartbeat_back_addr":"172.21.15.204:6823/1010601477","heartbeat_front_addr":"172.21.15.204:6821/1010601477","state":["exists","up"]},{"osd":9,"uuid":"8df80dfa-e7ff-410e-824c-b1dce111b02f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":3334759819},{"type":"v1","addr":"172.21.15.107:6827","nonce":3334759819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3334759819},{"type":"v1","addr":"172.21.15.107:6829","nonce":3334759819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":3334759819},{"type":"v1","addr":"172.21.15.107:6833","nonce":3334759819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":3334759819},{"type":"v1","addr":"172.21.15.107:6831","nonce":3334759819}]},"public_addr":"172.21.15.107:6827/3334759819","cluster_addr":"172.21.15.107:6829/3334759819","heartbeat_back_addr":"172.21.15.107:6833/3334759819","heartbeat_front_addr":"172.21.15.107:6831/3334759819","state":["exists","up"]},{"osd":10,"uuid":"338945b2-20c0-40b2-aaca-062cd4adee04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":3469284631},{"type":"v1","addr":"172.21.15.149:6825","nonce":3469284631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":3469284631},{"type":"v1","addr":"172.21.15.149:6827","nonce":3469284631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":3469284631},{"type":"v1","addr":"172.21.15.149:6831","nonce":3469284631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":3469284631},{"type":"v1","addr":"172.21.15.149:6829","nonce":3469284631}]},"public_addr":"172.21.15.149:6825/3469284631","cluster_addr":"172.21.15.149:6827/3469284631","heartbeat_back_addr":"172.21.15.149:6831/3469284631","heartbeat_front_addr":"172.21.15.149:6829/3469284631","state":["exists","up"]},{"osd":11,"uuid":"acccfe4f-d307-4a64-8f4d-fba27bfbe09d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":3097772473},{"type":"v1","addr":"172.21.15.204:6825","nonce":3097772473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":3097772473},{"type":"v1","addr":"172.21.15.204:6827","nonce":3097772473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":3097772473},{"type":"v1","addr":"172.21.15.204:6831","nonce":3097772473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":3097772473},{"type":"v1","addr":"172.21.15.204:6829","nonce":3097772473}]},"public_addr":"172.21.15.204:6825/3097772473","cluster_addr":"172.21.15.204:6827/3097772473","heartbeat_back_addr":"172.21.15.204:6831/3097772473","heartbeat_front_addr":"172.21.15.204:6829/3097772473","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T20:50:25.862753+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T20:50:58.167663+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T20:51:30.312816+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T20:52:05.487917+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T20:52:39.348354+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T20:53:12.899002+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T20:53:47.630837+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T20:54:21.444552+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T20:54:57.101160+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T20:55:31.841786+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T20:56:05.827161+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T20:56:40.611097+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.0","mappings":[{"from":4,"to":6}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.3b","mappings":[{"from":3,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.107:0/1514666962":"2024-03-09T20:48:20.149406+0000","172.21.15.107:0/1886016707":"2024-03-09T20:48:20.149406+0000","172.21.15.107:6801/426372446":"2024-03-09T20:48:20.149406+0000","172.21.15.107:6800/426372446":"2024-03-09T20:48:20.149406+0000","172.21.15.107:0/2392547434":"2024-03-09T20:47:45.706790+0000","172.21.15.107:0/1206995985":"2024-03-09T20:47:45.706790+0000","172.21.15.107:6800/3549936519":"2024-03-09T20:47:45.706790+0000","172.21.15.107:6801/3549936519":"2024-03-09T20:47:45.706790+0000","172.21.15.107:0/2482452215":"2024-03-09T20:47:45.706790+0000","172.21.15.107:0/808124991":"2024-03-09T20:48:20.149406+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-08T21:00:10.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.461+0000 7fdb51200640 1 -- 172.21.15.107:0/714023804 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fdb300a2800 msgr2=0x7fdb300a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:00:10.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.461+0000 7fdb51200640 1 --2- 172.21.15.107:0/714023804 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fdb300a2800 0x7fdb300a4cc0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fdb3c006010 tx=0x7fdb3c005be0 comp rx=0 tx=0).stop 2024-03-08T21:00:10.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.461+0000 7fdb51200640 1 -- 172.21.15.107:0/714023804 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdb4c104720 msgr2=0x7fdb4c199c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:00:10.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.461+0000 7fdb51200640 1 --2- 172.21.15.107:0/714023804 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdb4c104720 0x7fdb4c199c70 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fdb3400d770 tx=0x7fdb3400dc30 comp rx=0 tx=0).stop 2024-03-08T21:00:10.445 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.461+0000 7fdb51200640 1 -- 172.21.15.107:0/714023804 shutdown_connections 2024-03-08T21:00:10.445 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.461+0000 7fdb51200640 1 --2- 172.21.15.107:0/714023804 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fdb300a2800 0x7fdb300a4cc0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:10.445 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.461+0000 7fdb51200640 1 --2- 172.21.15.107:0/714023804 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdb4c1050e0 0x7fdb4c1a0ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:10.445 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.461+0000 7fdb51200640 1 --2- 172.21.15.107:0/714023804 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdb4c104720 0x7fdb4c199c70 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:10.445 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.461+0000 7fdb51200640 1 --2- 172.21.15.107:0/714023804 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdb4c103530 0x7fdb4c199730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:10.445 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.461+0000 7fdb51200640 1 -- 172.21.15.107:0/714023804 >> 172.21.15.107:0/714023804 conn(0x7fdb4c0feae0 msgr2=0x7fdb4c107aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:00:10.445 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.461+0000 7fdb51200640 1 -- 172.21.15.107:0/714023804 shutdown_connections 2024-03-08T21:00:10.445 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.462+0000 7fdb51200640 1 -- 172.21.15.107:0/714023804 wait complete. 2024-03-08T21:00:10.464 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 27, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-03-08T20:57:50.223752+0000', 'modified': '2024-03-08T20:57:57.874727+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 34475, 'mds_1': 34445, 'mds_2': 34415, 'mds_3': 34397, 'mds_4': 24457}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14625': {'gid': 14625, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.107:6839/3070059000', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6838', 'nonce': 3070059000}, {'type': 'v1', 'addr': '172.21.15.107:6839', 'nonce': 3070059000}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_14637': {'gid': 14637, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.107:6841/747096194', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6840', 'nonce': 747096194}, {'type': 'v1', 'addr': '172.21.15.107:6841', 'nonce': 747096194}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24400': {'gid': 24400, 'name': 'l', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.204:6833/3404762782', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.204:6832', 'nonce': 3404762782}, {'type': 'v1', 'addr': '172.21.15.204:6833', 'nonce': 3404762782}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24430': {'gid': 24430, 'name': 'f', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.204:6835/2072518651', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.204:6834', 'nonce': 2072518651}, {'type': 'v1', 'addr': '172.21.15.204:6835', 'nonce': 2072518651}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24442': {'gid': 24442, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.204:6837/42644905', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.204:6836', 'nonce': 42644905}, {'type': 'v1', 'addr': '172.21.15.204:6837', 'nonce': 42644905}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24457': {'gid': 24457, 'name': 'i', 'rank': 4, 'incarnation': 25, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.204:6839/1857316822', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.204:6838', 'nonce': 1857316822}, {'type': 'v1', 'addr': '172.21.15.204:6839', 'nonce': 1857316822}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_34397': {'gid': 34397, 'name': 'k', 'rank': 3, 'incarnation': 23, 'state': 'up:active', 'state_seq': 14, 'addr': '172.21.15.149:6833/1962161726', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.149:6832', 'nonce': 1962161726}, {'type': 'v1', 'addr': '172.21.15.149:6833', 'nonce': 1962161726}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_34415': {'gid': 34415, 'name': 'e', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.149:6835/442657066', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.149:6834', 'nonce': 442657066}, {'type': 'v1', 'addr': '172.21.15.149:6835', 'nonce': 442657066}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_34445': {'gid': 34445, 'name': 'h', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.149:6837/914546237', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.149:6836', 'nonce': 914546237}, {'type': 'v1', 'addr': '172.21.15.149:6837', 'nonce': 914546237}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_34475': {'gid': 34475, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.149:6839/55290596', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.149:6838', 'nonce': 55290596}, {'type': 'v1', 'addr': '172.21.15.149:6839', 'nonce': 55290596}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-03-08T21:00:10.464 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-03-08T21:00:10.464 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-03-08T21:00:10.679 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.696+0000 7f8550400640 1 -- 172.21.15.107:0/4193535875 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8548104720 msgr2=0x7f8548104ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:00:10.679 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.696+0000 7f8550400640 1 --2- 172.21.15.107:0/4193535875 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8548104720 0x7f8548104ba0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f8534009900 tx=0x7f853402bc30 comp rx=0 tx=0).stop 2024-03-08T21:00:10.679 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.696+0000 7f8550400640 1 -- 172.21.15.107:0/4193535875 shutdown_connections 2024-03-08T21:00:10.679 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.696+0000 7f8550400640 1 --2- 172.21.15.107:0/4193535875 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f85481050e0 0x7f854810b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:10.680 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.696+0000 7f8550400640 1 --2- 172.21.15.107:0/4193535875 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8548104720 0x7f8548104ba0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:10.680 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.696+0000 7f8550400640 1 --2- 172.21.15.107:0/4193535875 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8548103530 0x7f8548103950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:10.680 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.696+0000 7f8550400640 1 -- 172.21.15.107:0/4193535875 >> 172.21.15.107:0/4193535875 conn(0x7f85480feaa0 msgr2=0x7f8548100ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:00:10.680 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.696+0000 7f8550400640 1 -- 172.21.15.107:0/4193535875 shutdown_connections 2024-03-08T21:00:10.680 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.696+0000 7f8550400640 1 -- 172.21.15.107:0/4193535875 wait complete. 2024-03-08T21:00:10.682 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.697+0000 7f8550400640 1 Processor -- start 2024-03-08T21:00:10.682 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.697+0000 7f8550400640 1 -- start start 2024-03-08T21:00:10.682 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.697+0000 7f8550400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8548103530 0x7f8548199730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:00:10.682 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.697+0000 7f8550400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8548104720 0x7f8548199c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:00:10.682 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.697+0000 7f8550400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f85481050e0 0x7f85481a0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:00:10.682 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.697+0000 7f8550400640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f854810f0c0 con 0x7f8548103530 2024-03-08T21:00:10.682 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.697+0000 7f8550400640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f854810ef40 con 0x7f85481050e0 2024-03-08T21:00:10.682 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.697+0000 7f8550400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f854810f240 con 0x7f8548104720 2024-03-08T21:00:10.682 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.697+0000 7f854da00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8548103530 0x7f8548199730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:00:10.682 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.698+0000 7f854da00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8548103530 0x7f8548199730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:56638/0 (socket says 172.21.15.107:56638) 2024-03-08T21:00:10.682 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.698+0000 7f854da00640 1 -- 172.21.15.107:0/3552253735 learned_addr learned my addr 172.21.15.107:0/3552253735 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:00:10.682 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.698+0000 7f854da00640 1 -- 172.21.15.107:0/3552253735 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8548104720 msgr2=0x7f8548199c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:00:10.683 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.698+0000 7f854da00640 1 --2- 172.21.15.107:0/3552253735 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8548104720 0x7f8548199c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:10.683 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.698+0000 7f854da00640 1 -- 172.21.15.107:0/3552253735 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f85481050e0 msgr2=0x7f85481a0ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:00:10.683 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.698+0000 7f854da00640 1 --2- 172.21.15.107:0/3552253735 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f85481050e0 0x7f85481a0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:10.683 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.698+0000 7f854da00640 1 -- 172.21.15.107:0/3552253735 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85481a1350 con 0x7f8548103530 2024-03-08T21:00:10.683 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.698+0000 7f854da00640 1 --2- 172.21.15.107:0/3552253735 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8548103530 0x7f8548199730 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f853800d770 tx=0x7f853800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:00:10.683 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.698+0000 7f8546000640 1 -- 172.21.15.107:0/3552253735 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8538017070 con 0x7f8548103530 2024-03-08T21:00:10.683 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.698+0000 7f8546000640 1 -- 172.21.15.107:0/3552253735 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f8538009c30 con 0x7f8548103530 2024-03-08T21:00:10.683 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.698+0000 7f8550400640 1 -- 172.21.15.107:0/3552253735 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85481a1640 con 0x7f8548103530 2024-03-08T21:00:10.683 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.698+0000 7f8546000640 1 -- 172.21.15.107:0/3552253735 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8538003d00 con 0x7f8548103530 2024-03-08T21:00:10.683 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.699+0000 7f8550400640 1 -- 172.21.15.107:0/3552253735 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85481a1b90 con 0x7f8548103530 2024-03-08T21:00:10.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.702+0000 7f8553236740 1 Processor -- start 2024-03-08T21:00:10.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.702+0000 7f8546000640 1 -- 172.21.15.107:0/3552253735 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f8538003ea0 con 0x7f8548103530 2024-03-08T21:00:10.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.702+0000 7f8553236740 1 -- start start 2024-03-08T21:00:10.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.702+0000 7f8553236740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563a62dac590 0x563a62dac9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:00:10.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.702+0000 7f8553236740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563a62dacf80 0x563a62db1a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:00:10.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.702+0000 7f8553236740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563a62db20f0 0x563a62db4580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:00:10.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.702+0000 7f8553236740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a62d116f0 con 0x563a62db20f0 2024-03-08T21:00:10.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.702+0000 7f8553236740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a62cba9e0 con 0x563a62dac590 2024-03-08T21:00:10.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.702+0000 7f8553236740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a62d0d790 con 0x563a62dacf80 2024-03-08T21:00:10.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.703+0000 7f854d000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563a62db20f0 0x563a62db4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:00:10.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.703+0000 7f854d000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563a62db20f0 0x563a62db4580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:56642/0 (socket says 172.21.15.107:56642) 2024-03-08T21:00:10.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.703+0000 7f854d000640 1 -- 172.21.15.107:0/773671464 learned_addr learned my addr 172.21.15.107:0/773671464 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:00:10.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.703+0000 7f854e400640 1 --2- 172.21.15.107:0/773671464 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563a62dac590 0x563a62dac9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:00:10.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.703+0000 7f854da00640 1 --2- 172.21.15.107:0/773671464 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563a62dacf80 0x563a62db1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:00:10.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.703+0000 7f854d000640 1 -- 172.21.15.107:0/773671464 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563a62dacf80 msgr2=0x563a62db1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:00:10.688 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.703+0000 7f854d000640 1 --2- 172.21.15.107:0/773671464 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563a62dacf80 0x563a62db1a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:10.688 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.703+0000 7f854d000640 1 -- 172.21.15.107:0/773671464 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563a62dac590 msgr2=0x563a62dac9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:00:10.688 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.703+0000 7f854d000640 1 --2- 172.21.15.107:0/773671464 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563a62dac590 0x563a62dac9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:10.688 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.703+0000 7f854d000640 1 -- 172.21.15.107:0/773671464 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563a62cae720 con 0x563a62db20f0 2024-03-08T21:00:10.688 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.703+0000 7f854e400640 1 --2- 172.21.15.107:0/773671464 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563a62dac590 0x563a62dac9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T21:00:10.688 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.704+0000 7f854d000640 1 --2- 172.21.15.107:0/773671464 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563a62db20f0 0x563a62db4580 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f8534036040 tx=0x7f8534009610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb83790a1af47c22 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:00:10.692 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.704+0000 7f853ea00640 1 -- 172.21.15.107:0/773671464 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8534042020 con 0x563a62db20f0 2024-03-08T21:00:10.692 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.704+0000 7f853ea00640 1 -- 172.21.15.107:0/773671464 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f853402cae0 con 0x563a62db20f0 2024-03-08T21:00:10.692 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.704+0000 7f853ea00640 1 -- 172.21.15.107:0/773671464 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8534003dc0 con 0x563a62db20f0 2024-03-08T21:00:10.692 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.704+0000 7f8546000640 1 --2- 172.21.15.107:0/3552253735 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f852c0a2800 0x7f852c0a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:00:10.692 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.705+0000 7f8546000640 1 -- 172.21.15.107:0/3552253735 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f85380ce860 con 0x7f8548103530 2024-03-08T21:00:10.692 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.705+0000 7f854e400640 1 --2- 172.21.15.107:0/3552253735 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f852c0a2800 0x7f852c0a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:00:10.692 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.705+0000 7f8553236740 1 -- 172.21.15.107:0/773671464 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563a62db20f0 msgr2=0x563a62db4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:00:10.693 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.705+0000 7f8553236740 1 --2- 172.21.15.107:0/773671464 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563a62db20f0 0x563a62db4580 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f8534036040 tx=0x7f8534009610 comp rx=0 tx=0).stop 2024-03-08T21:00:10.693 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.705+0000 7f8553236740 1 -- 172.21.15.107:0/773671464 shutdown_connections 2024-03-08T21:00:10.693 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.705+0000 7f8553236740 1 --2- 172.21.15.107:0/773671464 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563a62db20f0 0x563a62db4580 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:10.693 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.705+0000 7f8553236740 1 --2- 172.21.15.107:0/773671464 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563a62dacf80 0x563a62db1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:10.693 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.705+0000 7f8553236740 1 --2- 172.21.15.107:0/773671464 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563a62dac590 0x563a62dac9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:10.693 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.705+0000 7f8553236740 1 -- 172.21.15.107:0/773671464 >> 172.21.15.107:0/773671464 conn(0x563a62da7b00 msgr2=0x563a62da7f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:00:10.693 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.705+0000 7f8553236740 1 -- 172.21.15.107:0/773671464 shutdown_connections 2024-03-08T21:00:10.693 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.705+0000 7f8553236740 1 -- 172.21.15.107:0/773671464 wait complete. 2024-03-08T21:00:10.693 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.705+0000 7f854e400640 1 --2- 172.21.15.107:0/3552253735 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f852c0a2800 0x7f852c0a4cc0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f8540009270 tx=0x7f8540009540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:00:10.693 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.706+0000 7f8553236740 1 -- start start 2024-03-08T21:00:10.693 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.706+0000 7f8553236740 1 Processor -- start 2024-03-08T21:00:10.693 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.706+0000 7f8553236740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563a62dac590 0x563a62db6710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:00:10.693 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.706+0000 7f8553236740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563a62dacf80 0x563a62db6c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:00:10.694 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.706+0000 7f8553236740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563a62db20f0 0x563a62dbdc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:00:10.694 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.706+0000 7f8553236740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a62db5040 con 0x563a62dacf80 2024-03-08T21:00:10.694 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.706+0000 7f8553236740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a62ccde20 con 0x563a62dac590 2024-03-08T21:00:10.694 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.706+0000 7f8553236740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a62db51c0 con 0x563a62db20f0 2024-03-08T21:00:10.694 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.706+0000 7f8553236740 10 client.0 _ll_register_callbacks cb 0x563a62d088d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T21:00:10.694 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.706+0000 7f854d000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563a62dacf80 0x563a62db6c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:00:10.694 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.707+0000 7f854d000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563a62dacf80 0x563a62db6c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:56658/0 (socket says 172.21.15.107:56658) 2024-03-08T21:00:10.694 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.707+0000 7f854d000640 1 -- 172.21.15.107:0/2208418830 learned_addr learned my addr 172.21.15.107:0/2208418830 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:00:10.694 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.707+0000 7f854d000640 1 -- 172.21.15.107:0/2208418830 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563a62db20f0 msgr2=0x563a62dbdc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:00:10.694 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.707+0000 7f854d000640 1 --2- 172.21.15.107:0/2208418830 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563a62db20f0 0x563a62dbdc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:10.694 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.707+0000 7f854da00640 1 --2- 172.21.15.107:0/2208418830 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563a62dac590 0x563a62db6710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:00:10.694 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.707+0000 7f854d000640 1 -- 172.21.15.107:0/2208418830 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563a62dac590 msgr2=0x563a62db6710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:00:10.694 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.707+0000 7f854d000640 1 --2- 172.21.15.107:0/2208418830 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563a62dac590 0x563a62db6710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:10.694 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.707+0000 7f854d000640 1 -- 172.21.15.107:0/2208418830 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563a62a262d0 con 0x563a62dacf80 2024-03-08T21:00:10.695 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.707+0000 7f854d000640 1 --2- 172.21.15.107:0/2208418830 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563a62dacf80 0x563a62db6c50 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f853402c6a0 tx=0x7f8534003df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dab702305f08e45b server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:00:10.695 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.707+0000 7f8533400640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-03-08T21:00:10.695 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.707+0000 7f8533400640 1 -- 172.21.15.107:0/2208418830 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8534042030 con 0x563a62dacf80 2024-03-08T21:00:10.695 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.707+0000 7f8533400640 1 -- 172.21.15.107:0/2208418830 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f8534004450 con 0x563a62dacf80 2024-03-08T21:00:10.695 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.708+0000 7f8553236740 1 -- 172.21.15.107:0/2208418830 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f853402cae0 con 0x563a62dacf80 2024-03-08T21:00:10.695 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.708+0000 7f8533400640 1 -- 172.21.15.107:0/2208418830 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8534034440 con 0x563a62dacf80 2024-03-08T21:00:10.695 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.708+0000 7f8553236740 10 client.15255 fetch_fsmap learned FSMap version 27 2024-03-08T21:00:10.695 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.708+0000 7f8553236740 1 -- 172.21.15.107:0/2208418830 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x563a62af06b0 con 0x563a62dacf80 2024-03-08T21:00:10.695 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.708+0000 7f8533400640 1 -- 172.21.15.107:0/2208418830 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f853402cae0 con 0x563a62dacf80 2024-03-08T21:00:10.695 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.708+0000 7f8533400640 1 -- 172.21.15.107:0/2208418830 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f85340474b0 con 0x563a62dacf80 2024-03-08T21:00:10.695 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.708+0000 7f8553236740 10 client.15255 fetch_fsmap finished waiting for FSMap version 27 2024-03-08T21:00:10.696 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.708+0000 7f8553236740 10 client.15255 resolve_mds: resolved name 'i' to daemon mds.i 2024-03-08T21:00:10.696 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.708+0000 7f8553236740 20 client.15255 populate_metadata read hostname 'smithi107' 2024-03-08T21:00:10.696 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.708+0000 7f8553236740 1 --2- 172.21.15.107:0/2208418830 >> [v2:172.21.15.204:6838/1857316822,v1:172.21.15.204:6839/1857316822] conn(0x563a62dc2180 0x563a62dc72b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:00:10.696 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.708+0000 7f8553236740 4 client.15255 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-08T21:00:10.696 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.708+0000 7f8553236740 1 -- 172.21.15.107:0/2208418830 --> [v2:172.21.15.204:6838/1857316822,v1:172.21.15.204:6839/1857316822] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563a62d1ec30 con 0x563a62dc2180 2024-03-08T21:00:10.696 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.709+0000 7f854da00640 1 --2- 172.21.15.107:0/2208418830 >> [v2:172.21.15.204:6838/1857316822,v1:172.21.15.204:6839/1857316822] conn(0x563a62dc2180 0x563a62dc72b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:00:10.697 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.710+0000 7f854da00640 1 --2- 172.21.15.107:0/2208418830 >> [v2:172.21.15.204:6838/1857316822,v1:172.21.15.204:6839/1857316822] conn(0x563a62dc2180 0x563a62dc72b0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=8eb2d39ec1fda53b server_cookie=e7390b1c07304968 in_seq=0 out_seq=0 2024-03-08T21:00:10.697 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.710+0000 7f8533400640 10 client.15255 ms_handle_connect on v2:172.21.15.204:6838/1857316822 2024-03-08T21:00:10.697 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.712+0000 7f8533400640 1 -- 172.21.15.107:0/2208418830 <== mds.4 v2:172.21.15.204:6838/1857316822 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x563a62d1ec30 con 0x563a62dc2180 2024-03-08T21:00:10.697 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.712+0000 7f8533400640 10 client.15255 handle_command_reply: tid=0 2024-03-08T21:00:10.697 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.712+0000 7f8553236740 1 client.15255 shutdown 2024-03-08T21:00:10.697 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.712+0000 7f8553236740 1 -- 172.21.15.107:0/2208418830 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563a62dacf80 msgr2=0x563a62db6c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:00:10.698 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.712+0000 7f8553236740 1 --2- 172.21.15.107:0/2208418830 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563a62dacf80 0x563a62db6c50 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f853402c6a0 tx=0x7f8534003df0 comp rx=0 tx=0).stop 2024-03-08T21:00:10.698 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.712+0000 7f8553236740 1 -- 172.21.15.107:0/2208418830 shutdown_connections 2024-03-08T21:00:10.698 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.712+0000 7f8553236740 1 --2- 172.21.15.107:0/2208418830 >> [v2:172.21.15.204:6838/1857316822,v1:172.21.15.204:6839/1857316822] conn(0x563a62dc2180 0x563a62dc72b0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:10.698 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.712+0000 7f8553236740 1 --2- 172.21.15.107:0/2208418830 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563a62db20f0 0x563a62dbdc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:10.698 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.712+0000 7f8553236740 1 --2- 172.21.15.107:0/2208418830 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563a62dacf80 0x563a62db6c50 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:10.698 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.712+0000 7f8553236740 1 --2- 172.21.15.107:0/2208418830 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563a62dac590 0x563a62db6710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:10.698 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.712+0000 7f8533400640 0 client.15255 ms_handle_reset on v2:172.21.15.204:6838/1857316822 2024-03-08T21:00:10.698 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.712+0000 7f8553236740 1 -- 172.21.15.107:0/2208418830 >> 172.21.15.107:0/2208418830 conn(0x563a62da7b00 msgr2=0x563a62da6c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:00:10.698 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.712+0000 7f8553236740 1 -- 172.21.15.107:0/2208418830 shutdown_connections 2024-03-08T21:00:10.698 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.712+0000 7f8553236740 1 -- 172.21.15.107:0/2208418830 wait complete. 2024-03-08T21:00:10.698 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.712+0000 7f8553236740 20 client.15255 trim_cache size 0 max 16384 2024-03-08T21:00:10.713 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.729+0000 7f8553236740 1 Processor -- start 2024-03-08T21:00:10.713 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.729+0000 7f8553236740 1 -- start start 2024-03-08T21:00:10.713 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.730+0000 7f8553236740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563a62dac590 0x563a62db6bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:00:10.714 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.730+0000 7f8553236740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563a62dacf80 0x563a62dbdc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:00:10.714 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.730+0000 7f8553236740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563a62db20f0 0x563a62da63b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:00:10.714 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.730+0000 7f8553236740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a62dc2650 con 0x563a62dacf80 2024-03-08T21:00:10.714 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.730+0000 7f8553236740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a62dc2cb0 con 0x563a62dac590 2024-03-08T21:00:10.714 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.730+0000 7f8553236740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a62dc3310 con 0x563a62db20f0 2024-03-08T21:00:10.714 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.730+0000 7f854d000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563a62dacf80 0x563a62dbdc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:00:10.714 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.730+0000 7f854d000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563a62dacf80 0x563a62dbdc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:56668/0 (socket says 172.21.15.107:56668) 2024-03-08T21:00:10.714 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.730+0000 7f854d000640 1 -- 172.21.15.107:0/531878814 learned_addr learned my addr 172.21.15.107:0/531878814 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:00:10.714 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.730+0000 7f854d000640 1 -- 172.21.15.107:0/531878814 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563a62db20f0 msgr2=0x563a62da63b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:00:10.714 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.730+0000 7f854d000640 1 --2- 172.21.15.107:0/531878814 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563a62db20f0 0x563a62da63b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:10.714 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.730+0000 7f854d000640 1 -- 172.21.15.107:0/531878814 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563a62dac590 msgr2=0x563a62db6bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:00:10.714 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.730+0000 7f854d000640 1 --2- 172.21.15.107:0/531878814 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563a62dac590 0x563a62db6bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:10.714 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.730+0000 7f854d000640 1 -- 172.21.15.107:0/531878814 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563a62dad810 con 0x563a62dacf80 2024-03-08T21:00:10.715 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.731+0000 7f854d000640 1 --2- 172.21.15.107:0/531878814 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563a62dacf80 0x563a62dbdc50 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f8534003b10 tx=0x7f853402cd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7fde4899b59362a server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:00:10.716 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.731+0000 7f853e000640 1 -- 172.21.15.107:0/531878814 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8534042070 con 0x563a62dacf80 2024-03-08T21:00:10.717 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.731+0000 7f853e000640 1 -- 172.21.15.107:0/531878814 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f853404b070 con 0x563a62dacf80 2024-03-08T21:00:10.717 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.731+0000 7f853e000640 1 -- 172.21.15.107:0/531878814 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8534047420 con 0x563a62dacf80 2024-03-08T21:00:10.717 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.731+0000 7f8553236740 1 -- 172.21.15.107:0/531878814 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563a62dacf80 msgr2=0x563a62dbdc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:00:10.717 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.731+0000 7f8553236740 1 --2- 172.21.15.107:0/531878814 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563a62dacf80 0x563a62dbdc50 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f8534003b10 tx=0x7f853402cd00 comp rx=0 tx=0).stop 2024-03-08T21:00:10.717 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.731+0000 7f8553236740 1 -- 172.21.15.107:0/531878814 shutdown_connections 2024-03-08T21:00:10.717 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.731+0000 7f8553236740 1 --2- 172.21.15.107:0/531878814 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563a62db20f0 0x563a62da63b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:10.717 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.731+0000 7f8553236740 1 --2- 172.21.15.107:0/531878814 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563a62dacf80 0x563a62dbdc50 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:10.717 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.731+0000 7f8553236740 1 --2- 172.21.15.107:0/531878814 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563a62dac590 0x563a62db6bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:10.717 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.731+0000 7f8553236740 1 -- 172.21.15.107:0/531878814 >> 172.21.15.107:0/531878814 conn(0x563a62da7b00 msgr2=0x563a62da7ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:00:10.717 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.732+0000 7f8553236740 1 -- 172.21.15.107:0/531878814 shutdown_connections 2024-03-08T21:00:10.717 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.732+0000 7f8553236740 1 -- 172.21.15.107:0/531878814 wait complete. 2024-03-08T21:00:10.717 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.732+0000 7f8553236740 1 -- start start 2024-03-08T21:00:10.718 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.732+0000 7f8553236740 1 Processor -- start 2024-03-08T21:00:10.718 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.732+0000 7f8553236740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563a62dac590 0x563a62dba970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:00:10.718 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.733+0000 7f8553236740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563a62dacf80 0x563a62dbaeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:00:10.718 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.733+0000 7f8553236740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563a62db20f0 0x563a62e21780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:00:10.718 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.733+0000 7f8553236740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a62dc3fd0 con 0x563a62dacf80 2024-03-08T21:00:10.718 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.733+0000 7f8553236740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a62dc3970 con 0x563a62db20f0 2024-03-08T21:00:10.718 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.733+0000 7f8553236740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a62dc4630 con 0x563a62dac590 2024-03-08T21:00:10.718 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.733+0000 7f8553236740 10 client.0 _ll_register_callbacks cb 0x7f8534009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T21:00:10.718 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.733+0000 7f854d000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563a62dacf80 0x563a62dbaeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:00:10.718 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.733+0000 7f854d000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563a62dacf80 0x563a62dbaeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:56672/0 (socket says 172.21.15.107:56672) 2024-03-08T21:00:10.718 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.733+0000 7f854d000640 1 -- 172.21.15.107:0/616547533 learned_addr learned my addr 172.21.15.107:0/616547533 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:00:10.718 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.733+0000 7f854e400640 1 --2- 172.21.15.107:0/616547533 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563a62db20f0 0x563a62e21780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:00:10.719 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:00:10 smithi107 ceph-mon[31381]: from='client.? 172.21.15.107:0/1495107928' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-08T21:00:10.719 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:00:10 smithi107 ceph-mon[31381]: from='client.? 172.21.15.107:0/714023804' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-08T21:00:10.722 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.733+0000 7f854d000640 1 -- 172.21.15.107:0/616547533 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563a62dac590 msgr2=0x563a62dba970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:00:10.722 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.733+0000 7f854d000640 1 --2- 172.21.15.107:0/616547533 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563a62dac590 0x563a62dba970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:10.722 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.733+0000 7f854d000640 1 -- 172.21.15.107:0/616547533 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563a62db20f0 msgr2=0x563a62e21780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:00:10.722 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.733+0000 7f854d000640 1 --2- 172.21.15.107:0/616547533 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563a62db20f0 0x563a62e21780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:10.722 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.733+0000 7f854d000640 1 -- 172.21.15.107:0/616547533 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563a62dbb520 con 0x563a62dacf80 2024-03-08T21:00:10.722 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.734+0000 7f854d000640 1 --2- 172.21.15.107:0/616547533 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563a62dacf80 0x563a62dbaeb0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f853402ce10 tx=0x7f8534003900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b36d0e07abb6a9e7 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:00:10.722 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.734+0000 7f8533400640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-03-08T21:00:10.722 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.734+0000 7f8533400640 1 -- 172.21.15.107:0/616547533 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f853405d020 con 0x563a62dacf80 2024-03-08T21:00:10.722 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.734+0000 7f8533400640 1 -- 172.21.15.107:0/616547533 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f8534047bf0 con 0x563a62dacf80 2024-03-08T21:00:10.722 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.734+0000 7f8553236740 1 -- 172.21.15.107:0/616547533 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f853404b070 con 0x563a62dacf80 2024-03-08T21:00:10.722 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.734+0000 7f8533400640 1 -- 172.21.15.107:0/616547533 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8534042040 con 0x563a62dacf80 2024-03-08T21:00:10.723 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.734+0000 7f8553236740 10 client.15261 fetch_fsmap learned FSMap version 27 2024-03-08T21:00:10.723 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.734+0000 7f8553236740 1 -- 172.21.15.107:0/616547533 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x563a62e22dc0 con 0x563a62dacf80 2024-03-08T21:00:10.723 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.734+0000 7f8533400640 1 -- 172.21.15.107:0/616547533 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f853404b070 con 0x563a62dacf80 2024-03-08T21:00:10.723 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.735+0000 7f8533400640 1 -- 172.21.15.107:0/616547533 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f8534064430 con 0x563a62dacf80 2024-03-08T21:00:10.723 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.735+0000 7f8553236740 10 client.15261 fetch_fsmap finished waiting for FSMap version 27 2024-03-08T21:00:10.723 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.735+0000 7f8553236740 10 client.15261 resolve_mds: resolved name 'i' to daemon mds.i 2024-03-08T21:00:10.723 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.735+0000 7f8553236740 20 client.15261 populate_metadata read hostname 'smithi107' 2024-03-08T21:00:10.723 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.735+0000 7f8553236740 1 --2- 172.21.15.107:0/616547533 >> [v2:172.21.15.204:6838/1857316822,v1:172.21.15.204:6839/1857316822] conn(0x563a62e27090 0x563a62e22ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:00:10.723 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.735+0000 7f8553236740 4 client.15261 mds_command: new command op to 24457 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-08T21:00:10.723 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.735+0000 7f8553236740 1 -- 172.21.15.107:0/616547533 --> [v2:172.21.15.204:6838/1857316822,v1:172.21.15.204:6839/1857316822] -- command(tid 0: {"prefix": "status"}) v1 -- 0x563a62d18f80 con 0x563a62e27090 2024-03-08T21:00:10.723 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.735+0000 7f854da00640 1 --2- 172.21.15.107:0/616547533 >> [v2:172.21.15.204:6838/1857316822,v1:172.21.15.204:6839/1857316822] conn(0x563a62e27090 0x563a62e22ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:00:10.723 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.736+0000 7f8533400640 10 client.15261 ms_handle_connect on v2:172.21.15.204:6838/1857316822 2024-03-08T21:00:10.723 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.736+0000 7f854da00640 1 --2- 172.21.15.107:0/616547533 >> [v2:172.21.15.204:6838/1857316822,v1:172.21.15.204:6839/1857316822] conn(0x563a62e27090 0x563a62e22ff0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=a220b3b7196d4b64 server_cookie=867e05865d43803b in_seq=0 out_seq=0 2024-03-08T21:00:10.723 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.737+0000 7f8533400640 1 -- 172.21.15.107:0/616547533 <== mds.4 v2:172.21.15.204:6838/1857316822 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x563a62d18f80 con 0x563a62e27090 2024-03-08T21:00:10.724 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.737+0000 7f8533400640 10 client.15261 handle_command_reply: tid=0 2024-03-08T21:00:10.724 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.737+0000 7f8553236740 1 client.15261 shutdown 2024-03-08T21:00:10.724 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.737+0000 7f8553236740 1 -- 172.21.15.107:0/616547533 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563a62dacf80 msgr2=0x563a62dbaeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:00:10.724 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.737+0000 7f8553236740 1 --2- 172.21.15.107:0/616547533 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563a62dacf80 0x563a62dbaeb0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f853402ce10 tx=0x7f8534003900 comp rx=0 tx=0).stop 2024-03-08T21:00:10.724 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.737+0000 7f8553236740 1 -- 172.21.15.107:0/616547533 shutdown_connections 2024-03-08T21:00:10.724 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.737+0000 7f8553236740 1 --2- 172.21.15.107:0/616547533 >> [v2:172.21.15.204:6838/1857316822,v1:172.21.15.204:6839/1857316822] conn(0x563a62e27090 0x563a62e22ff0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:10.724 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.737+0000 7f8553236740 1 --2- 172.21.15.107:0/616547533 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563a62db20f0 0x563a62e21780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:10.724 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.737+0000 7f8553236740 1 --2- 172.21.15.107:0/616547533 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563a62dacf80 0x563a62dbaeb0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:10.724 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.737+0000 7f8553236740 1 --2- 172.21.15.107:0/616547533 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563a62dac590 0x563a62dba970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:10.724 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.737+0000 7f8533400640 0 client.15261 ms_handle_reset on v2:172.21.15.204:6838/1857316822 2024-03-08T21:00:10.724 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.737+0000 7f8553236740 1 -- 172.21.15.107:0/616547533 >> 172.21.15.107:0/616547533 conn(0x563a62da7b00 msgr2=0x563a62da63b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:00:10.724 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.737+0000 7f8553236740 1 -- 172.21.15.107:0/616547533 shutdown_connections 2024-03-08T21:00:10.724 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.737+0000 7f8553236740 1 -- 172.21.15.107:0/616547533 wait complete. 2024-03-08T21:00:10.724 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.737+0000 7f8553236740 20 client.15261 trim_cache size 0 max 16384 2024-03-08T21:00:10.725 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-03-08T21:00:10.725 INFO:teuthology.orchestra.run.smithi107.stdout: "cluster_fsid": "dc20f0f8-dd8c-11ee-95c6-87774f69a715", 2024-03-08T21:00:10.725 INFO:teuthology.orchestra.run.smithi107.stdout: "whoami": 4, 2024-03-08T21:00:10.725 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 24457, 2024-03-08T21:00:10.725 INFO:teuthology.orchestra.run.smithi107.stdout: "want_state": "up:active", 2024-03-08T21:00:10.725 INFO:teuthology.orchestra.run.smithi107.stdout: "state": "up:active", 2024-03-08T21:00:10.725 INFO:teuthology.orchestra.run.smithi107.stdout: "fs_name": "cephfs", 2024-03-08T21:00:10.725 INFO:teuthology.orchestra.run.smithi107.stdout: "rank_uptime": 133.86945147099999, 2024-03-08T21:00:10.725 INFO:teuthology.orchestra.run.smithi107.stdout: "mdsmap_epoch": 27, 2024-03-08T21:00:10.725 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_epoch": 88, 2024-03-08T21:00:10.725 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_epoch_barrier": 86, 2024-03-08T21:00:10.726 INFO:teuthology.orchestra.run.smithi107.stdout: "uptime": 158.43401424999999 2024-03-08T21:00:10.726 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-03-08T21:00:10.726 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.738+0000 7f8550400640 1 -- 172.21.15.107:0/3552253735 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f852c0a2800 msgr2=0x7f852c0a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:00:10.726 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.738+0000 7f8550400640 1 --2- 172.21.15.107:0/3552253735 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f852c0a2800 0x7f852c0a4cc0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f8540009270 tx=0x7f8540009540 comp rx=0 tx=0).stop 2024-03-08T21:00:10.727 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.738+0000 7f8550400640 1 -- 172.21.15.107:0/3552253735 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8548103530 msgr2=0x7f8548199730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:00:10.727 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.738+0000 7f8550400640 1 --2- 172.21.15.107:0/3552253735 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8548103530 0x7f8548199730 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f853800d770 tx=0x7f853800dc30 comp rx=0 tx=0).stop 2024-03-08T21:00:10.727 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.738+0000 7f8550400640 1 -- 172.21.15.107:0/3552253735 shutdown_connections 2024-03-08T21:00:10.727 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.738+0000 7f8550400640 1 --2- 172.21.15.107:0/3552253735 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f852c0a2800 0x7f852c0a4cc0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:10.727 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.738+0000 7f8550400640 1 --2- 172.21.15.107:0/3552253735 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f85481050e0 0x7f85481a0ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:10.727 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.738+0000 7f8550400640 1 --2- 172.21.15.107:0/3552253735 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8548104720 0x7f8548199c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:10.727 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.738+0000 7f8550400640 1 --2- 172.21.15.107:0/3552253735 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8548103530 0x7f8548199730 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:10.727 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.738+0000 7f8550400640 1 -- 172.21.15.107:0/3552253735 >> 172.21.15.107:0/3552253735 conn(0x7f85480feaa0 msgr2=0x7f8548107ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:00:10.727 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.738+0000 7f8550400640 1 -- 172.21.15.107:0/3552253735 shutdown_connections 2024-03-08T21:00:10.727 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.738+0000 7f8550400640 1 -- 172.21.15.107:0/3552253735 wait complete. 2024-03-08T21:00:10.738 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.k status 2024-03-08T21:00:10.878 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:00:10 smithi204 ceph-mon[34935]: from='client.? 172.21.15.107:0/1495107928' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-08T21:00:10.878 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:00:10 smithi204 ceph-mon[34935]: from='client.? 172.21.15.107:0/714023804' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-08T21:00:10.934 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.951+0000 7f8a07c00640 1 -- 172.21.15.107:0/1312525477 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8a00076ef0 msgr2=0x7f8a00077350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:00:10.934 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.951+0000 7f8a07c00640 1 --2- 172.21.15.107:0/1312525477 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8a00076ef0 0x7f8a00077350 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f89e8009ca0 tx=0x7f89e802c0f0 comp rx=0 tx=0).stop 2024-03-08T21:00:10.934 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.951+0000 7f8a07c00640 1 -- 172.21.15.107:0/1312525477 shutdown_connections 2024-03-08T21:00:10.935 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.951+0000 7f8a07c00640 1 --2- 172.21.15.107:0/1312525477 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a00077890 0x7f8a00078130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:10.935 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.951+0000 7f8a07c00640 1 --2- 172.21.15.107:0/1312525477 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8a00076ef0 0x7f8a00077350 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:10.935 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.951+0000 7f8a07c00640 1 --2- 172.21.15.107:0/1312525477 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8a00075cb0 0x7f8a000760d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:10.935 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.951+0000 7f8a07c00640 1 -- 172.21.15.107:0/1312525477 >> 172.21.15.107:0/1312525477 conn(0x7f8a000feaa0 msgr2=0x7f8a00100ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:00:10.935 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.951+0000 7f8a07c00640 1 -- 172.21.15.107:0/1312525477 shutdown_connections 2024-03-08T21:00:10.935 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.951+0000 7f8a07c00640 1 -- 172.21.15.107:0/1312525477 wait complete. 2024-03-08T21:00:10.937 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.952+0000 7f8a07c00640 1 Processor -- start 2024-03-08T21:00:10.938 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.952+0000 7f8a07c00640 1 -- start start 2024-03-08T21:00:10.938 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.952+0000 7f8a07c00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8a00075cb0 0x7f8a0019dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:00:10.938 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.952+0000 7f8a07c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a00076ef0 0x7f8a0019e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:00:10.938 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.952+0000 7f8a07c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8a00077890 0x7f8a001a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:00:10.938 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.952+0000 7f8a07c00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a001134b0 con 0x7f8a00077890 2024-03-08T21:00:10.938 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.952+0000 7f8a07c00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a00113330 con 0x7f8a00075cb0 2024-03-08T21:00:10.938 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.952+0000 7f8a07c00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a00113630 con 0x7f8a00076ef0 2024-03-08T21:00:10.938 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.953+0000 7f8a05c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8a00077890 0x7f8a001a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:00:10.938 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.953+0000 7f8a05c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8a00077890 0x7f8a001a5170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:56692/0 (socket says 172.21.15.107:56692) 2024-03-08T21:00:10.938 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.953+0000 7f8a05c00640 1 -- 172.21.15.107:0/3443908453 learned_addr learned my addr 172.21.15.107:0/3443908453 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:00:10.938 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.953+0000 7f8a05200640 1 --2- 172.21.15.107:0/3443908453 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8a00075cb0 0x7f8a0019dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:00:10.939 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.953+0000 7f8a05c00640 1 -- 172.21.15.107:0/3443908453 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a00076ef0 msgr2=0x7f8a0019e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:00:10.939 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.953+0000 7f8a05c00640 1 --2- 172.21.15.107:0/3443908453 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a00076ef0 0x7f8a0019e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:10.939 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.953+0000 7f8a05c00640 1 -- 172.21.15.107:0/3443908453 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8a00075cb0 msgr2=0x7f8a0019dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:00:10.939 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.953+0000 7f8a05c00640 1 --2- 172.21.15.107:0/3443908453 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8a00075cb0 0x7f8a0019dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:10.939 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.953+0000 7f8a05c00640 1 -- 172.21.15.107:0/3443908453 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a001a57e0 con 0x7f8a00077890 2024-03-08T21:00:10.939 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.953+0000 7f8a05200640 1 --2- 172.21.15.107:0/3443908453 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8a00075cb0 0x7f8a0019dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T21:00:10.939 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.953+0000 7f8a05c00640 1 --2- 172.21.15.107:0/3443908453 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8a00077890 0x7f8a001a5170 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f89f8009fd0 tx=0x7f89f800b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:00:10.939 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.954+0000 7f89fd600640 1 -- 172.21.15.107:0/3443908453 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89f8011830 con 0x7f8a00077890 2024-03-08T21:00:10.939 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.954+0000 7f89fd600640 1 -- 172.21.15.107:0/3443908453 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f89f8016070 con 0x7f8a00077890 2024-03-08T21:00:10.939 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.954+0000 7f8a07c00640 1 -- 172.21.15.107:0/3443908453 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a001a5ad0 con 0x7f8a00077890 2024-03-08T21:00:10.939 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.954+0000 7f89fd600640 1 -- 172.21.15.107:0/3443908453 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89f8010de0 con 0x7f8a00077890 2024-03-08T21:00:10.939 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.954+0000 7f8a07c00640 1 -- 172.21.15.107:0/3443908453 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a001a60a0 con 0x7f8a00077890 2024-03-08T21:00:10.941 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.957+0000 7f89fd600640 1 -- 172.21.15.107:0/3443908453 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f89f80119d0 con 0x7f8a00077890 2024-03-08T21:00:10.942 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.957+0000 7f8a0a844740 1 Processor -- start 2024-03-08T21:00:10.942 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.957+0000 7f8a0a844740 1 -- start start 2024-03-08T21:00:10.942 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.957+0000 7f8a0a844740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55559d9bf590 0x55559d9bf9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:00:10.942 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.957+0000 7f8a0a844740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55559d9bff80 0x55559d9c4a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:00:10.942 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.957+0000 7f8a0a844740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55559d9c50f0 0x55559d9c7580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:00:10.942 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.957+0000 7f8a0a844740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55559d9246f0 con 0x55559d9bf590 2024-03-08T21:00:10.942 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.957+0000 7f8a0a844740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55559d8cd9e0 con 0x55559d9c50f0 2024-03-08T21:00:10.942 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.957+0000 7f8a0a844740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55559d920790 con 0x55559d9bff80 2024-03-08T21:00:10.942 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.958+0000 7f89ffe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55559d9bf590 0x55559d9bf9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:00:10.942 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.958+0000 7f89ffe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55559d9bf590 0x55559d9bf9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:56704/0 (socket says 172.21.15.107:56704) 2024-03-08T21:00:10.942 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.958+0000 7f89ffe00640 1 -- 172.21.15.107:0/2292484881 learned_addr learned my addr 172.21.15.107:0/2292484881 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:00:10.942 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.958+0000 7f8a05200640 1 --2- 172.21.15.107:0/2292484881 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55559d9bff80 0x55559d9c4a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:00:10.943 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.958+0000 7f89ffe00640 1 -- 172.21.15.107:0/2292484881 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55559d9bff80 msgr2=0x55559d9c4a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:00:10.943 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.958+0000 7f89ffe00640 1 --2- 172.21.15.107:0/2292484881 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55559d9bff80 0x55559d9c4a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:10.943 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.958+0000 7f89ffe00640 1 -- 172.21.15.107:0/2292484881 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55559d9c50f0 msgr2=0x55559d9c7580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:00:10.943 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.958+0000 7f89ffe00640 1 --2- 172.21.15.107:0/2292484881 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55559d9c50f0 0x55559d9c7580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:10.943 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.958+0000 7f89ffe00640 1 -- 172.21.15.107:0/2292484881 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55559d8c1720 con 0x55559d9bf590 2024-03-08T21:00:10.943 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.958+0000 7f89fd600640 1 --2- 172.21.15.107:0/3443908453 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f89e00a2880 0x7f89e00a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:00:10.943 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.958+0000 7f89ffe00640 1 --2- 172.21.15.107:0/2292484881 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55559d9bf590 0x55559d9bf9b0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f89e8005e30 tx=0x7f89e8009800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f5798b7e278aefd server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:00:10.943 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.958+0000 7f89fd600640 1 -- 172.21.15.107:0/3443908453 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f89f80ce1e0 con 0x7f8a00077890 2024-03-08T21:00:10.943 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.959+0000 7f8a05200640 1 --2- 172.21.15.107:0/3443908453 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f89e00a2880 0x7f89e00a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:00:10.943 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.959+0000 7f89f6000640 1 -- 172.21.15.107:0/2292484881 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89e8044070 con 0x55559d9bf590 2024-03-08T21:00:10.943 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.959+0000 7f89f6000640 1 -- 172.21.15.107:0/2292484881 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f89e8032bb0 con 0x55559d9bf590 2024-03-08T21:00:10.943 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.959+0000 7f89f6000640 1 -- 172.21.15.107:0/2292484881 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89e8039030 con 0x55559d9bf590 2024-03-08T21:00:10.947 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.959+0000 7f8a0a844740 1 -- 172.21.15.107:0/2292484881 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55559d9bf590 msgr2=0x55559d9bf9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:00:10.947 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.959+0000 7f8a05200640 1 --2- 172.21.15.107:0/3443908453 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f89e00a2880 0x7f89e00a4d40 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x55559d9c0380 tx=0x7f89f0005e90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:00:10.948 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.960+0000 7f8a0a844740 1 --2- 172.21.15.107:0/2292484881 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55559d9bf590 0x55559d9bf9b0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f89e8005e30 tx=0x7f89e8009800 comp rx=0 tx=0).stop 2024-03-08T21:00:10.948 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.960+0000 7f8a0a844740 1 -- 172.21.15.107:0/2292484881 shutdown_connections 2024-03-08T21:00:10.948 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.960+0000 7f8a0a844740 1 --2- 172.21.15.107:0/2292484881 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55559d9c50f0 0x55559d9c7580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:10.948 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.960+0000 7f8a0a844740 1 --2- 172.21.15.107:0/2292484881 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55559d9bff80 0x55559d9c4a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:10.948 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.960+0000 7f8a0a844740 1 --2- 172.21.15.107:0/2292484881 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55559d9bf590 0x55559d9bf9b0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:10.948 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.960+0000 7f8a0a844740 1 -- 172.21.15.107:0/2292484881 >> 172.21.15.107:0/2292484881 conn(0x55559d9bab00 msgr2=0x55559d9baf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:00:10.948 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.960+0000 7f8a0a844740 1 -- 172.21.15.107:0/2292484881 shutdown_connections 2024-03-08T21:00:10.948 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.960+0000 7f8a0a844740 1 -- 172.21.15.107:0/2292484881 wait complete. 2024-03-08T21:00:10.948 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.960+0000 7f8a0a844740 1 -- start start 2024-03-08T21:00:10.948 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.960+0000 7f8a0a844740 1 Processor -- start 2024-03-08T21:00:10.948 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.961+0000 7f8a0a844740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55559d9bf590 0x55559d9c9780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:00:10.948 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.961+0000 7f8a0a844740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55559d9bff80 0x55559d9c9cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:00:10.948 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.961+0000 7f8a0a844740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55559d9c50f0 0x55559d9d0d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:00:10.949 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.961+0000 7f8a0a844740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55559d9c8040 con 0x55559d9bf590 2024-03-08T21:00:10.949 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.961+0000 7f8a0a844740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55559d8e0e20 con 0x55559d9bff80 2024-03-08T21:00:10.949 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.961+0000 7f8a0a844740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55559d9c81c0 con 0x55559d9c50f0 2024-03-08T21:00:10.949 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.961+0000 7f8a0a844740 10 client.0 _ll_register_callbacks cb 0x55559d91b8d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T21:00:10.949 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.961+0000 7f8a05200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55559d9bf590 0x55559d9c9780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:00:10.949 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.961+0000 7f8a05200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55559d9bf590 0x55559d9c9780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:56716/0 (socket says 172.21.15.107:56716) 2024-03-08T21:00:10.949 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.961+0000 7f8a05200640 1 -- 172.21.15.107:0/1768643624 learned_addr learned my addr 172.21.15.107:0/1768643624 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:00:10.949 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.961+0000 7f89ffe00640 1 --2- 172.21.15.107:0/1768643624 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55559d9bff80 0x55559d9c9cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:00:10.949 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.961+0000 7f8a05200640 1 -- 172.21.15.107:0/1768643624 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55559d9c50f0 msgr2=0x55559d9d0d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:00:10.949 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.961+0000 7f8a05200640 1 --2- 172.21.15.107:0/1768643624 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55559d9c50f0 0x55559d9d0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:10.949 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.961+0000 7f8a05200640 1 -- 172.21.15.107:0/1768643624 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55559d9bff80 msgr2=0x55559d9c9cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:00:10.949 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.961+0000 7f8a05200640 1 --2- 172.21.15.107:0/1768643624 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55559d9bff80 0x55559d9c9cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:10.949 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.961+0000 7f8a05200640 1 -- 172.21.15.107:0/1768643624 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55559d6392d0 con 0x55559d9bf590 2024-03-08T21:00:10.949 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.962+0000 7f8a05200640 1 --2- 172.21.15.107:0/1768643624 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55559d9bf590 0x55559d9c9780 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f89f0009fd0 tx=0x7f89f0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73ffac925d389914 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:00:10.950 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.962+0000 7f89eea00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-03-08T21:00:10.950 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.962+0000 7f89eea00640 1 -- 172.21.15.107:0/1768643624 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89f0022070 con 0x55559d9bf590 2024-03-08T21:00:10.950 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.962+0000 7f89eea00640 1 -- 172.21.15.107:0/1768643624 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f89f001e860 con 0x55559d9bf590 2024-03-08T21:00:10.950 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.962+0000 7f8a0a844740 1 -- 172.21.15.107:0/1768643624 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f89e8032bb0 con 0x55559d9bf590 2024-03-08T21:00:10.950 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.962+0000 7f89eea00640 1 -- 172.21.15.107:0/1768643624 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f89f002e050 con 0x55559d9bf590 2024-03-08T21:00:10.950 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.962+0000 7f8a0a844740 10 client.15273 fetch_fsmap learned FSMap version 27 2024-03-08T21:00:10.950 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.962+0000 7f8a0a844740 1 -- 172.21.15.107:0/1768643624 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55559d7036b0 con 0x55559d9bf590 2024-03-08T21:00:10.950 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.963+0000 7f89eea00640 1 -- 172.21.15.107:0/1768643624 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f89e8032bb0 con 0x55559d9bf590 2024-03-08T21:00:10.950 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.963+0000 7f89eea00640 1 -- 172.21.15.107:0/1768643624 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f89f0010ae0 con 0x55559d9bf590 2024-03-08T21:00:10.950 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.963+0000 7f8a0a844740 10 client.15273 fetch_fsmap finished waiting for FSMap version 27 2024-03-08T21:00:10.950 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.963+0000 7f8a0a844740 10 client.15273 resolve_mds: resolved name 'k' to daemon mds.k 2024-03-08T21:00:10.950 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.963+0000 7f8a0a844740 20 client.15273 populate_metadata read hostname 'smithi107' 2024-03-08T21:00:10.950 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.963+0000 7f8a0a844740 1 --2- 172.21.15.107:0/1768643624 >> [v2:172.21.15.149:6832/1962161726,v1:172.21.15.149:6833/1962161726] conn(0x55559d9d6460 0x55559d9d53e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:00:10.951 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.963+0000 7f8a0a844740 4 client.15273 mds_command: new command op to 34397 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-08T21:00:10.951 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.963+0000 7f8a0a844740 1 -- 172.21.15.107:0/1768643624 --> [v2:172.21.15.149:6832/1962161726,v1:172.21.15.149:6833/1962161726] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55559d931c30 con 0x55559d9d6460 2024-03-08T21:00:10.951 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.964+0000 7f89ffe00640 1 --2- 172.21.15.107:0/1768643624 >> [v2:172.21.15.149:6832/1962161726,v1:172.21.15.149:6833/1962161726] conn(0x55559d9d6460 0x55559d9d53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:00:10.951 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.964+0000 7f89ffe00640 1 --2- 172.21.15.107:0/1768643624 >> [v2:172.21.15.149:6832/1962161726,v1:172.21.15.149:6833/1962161726] conn(0x55559d9d6460 0x55559d9d53e0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=5f6e967b2cfa1ab0 server_cookie=4bdc1e8f91459864 in_seq=0 out_seq=0 2024-03-08T21:00:10.951 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.964+0000 7f89eea00640 10 client.15273 ms_handle_connect on v2:172.21.15.149:6832/1962161726 2024-03-08T21:00:10.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.966+0000 7f89eea00640 1 -- 172.21.15.107:0/1768643624 <== mds.3 v2:172.21.15.149:6832/1962161726 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55559d931c30 con 0x55559d9d6460 2024-03-08T21:00:10.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.966+0000 7f89eea00640 10 client.15273 handle_command_reply: tid=0 2024-03-08T21:00:10.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.966+0000 7f8a0a844740 1 client.15273 shutdown 2024-03-08T21:00:10.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.967+0000 7f8a0a844740 1 -- 172.21.15.107:0/1768643624 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55559d9bf590 msgr2=0x55559d9c9780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:00:10.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.967+0000 7f8a0a844740 1 --2- 172.21.15.107:0/1768643624 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55559d9bf590 0x55559d9c9780 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f89f0009fd0 tx=0x7f89f0004730 comp rx=0 tx=0).stop 2024-03-08T21:00:10.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.967+0000 7f8a0a844740 1 -- 172.21.15.107:0/1768643624 shutdown_connections 2024-03-08T21:00:10.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.967+0000 7f8a0a844740 1 --2- 172.21.15.107:0/1768643624 >> [v2:172.21.15.149:6832/1962161726,v1:172.21.15.149:6833/1962161726] conn(0x55559d9d6460 0x55559d9d53e0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:10.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.967+0000 7f8a0a844740 1 --2- 172.21.15.107:0/1768643624 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55559d9c50f0 0x55559d9d0d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:10.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.967+0000 7f8a0a844740 1 --2- 172.21.15.107:0/1768643624 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55559d9bff80 0x55559d9c9cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:10.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.967+0000 7f89eea00640 0 client.15273 ms_handle_reset on v2:172.21.15.149:6832/1962161726 2024-03-08T21:00:10.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.967+0000 7f8a0a844740 1 --2- 172.21.15.107:0/1768643624 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55559d9bf590 0x55559d9c9780 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:10.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.967+0000 7f8a0a844740 1 -- 172.21.15.107:0/1768643624 >> 172.21.15.107:0/1768643624 conn(0x55559d9bab00 msgr2=0x55559d9b9c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:00:10.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.967+0000 7f8a0a844740 1 -- 172.21.15.107:0/1768643624 shutdown_connections 2024-03-08T21:00:10.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.967+0000 7f8a0a844740 1 -- 172.21.15.107:0/1768643624 wait complete. 2024-03-08T21:00:10.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.967+0000 7f8a0a844740 20 client.15273 trim_cache size 0 max 16384 2024-03-08T21:00:10.968 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.984+0000 7f8a0a844740 1 Processor -- start 2024-03-08T21:00:10.969 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.984+0000 7f8a0a844740 1 -- start start 2024-03-08T21:00:10.969 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.984+0000 7f8a0a844740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55559d9bf590 0x55559d9baee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:00:10.969 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.984+0000 7f8a0a844740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55559d9bff80 0x55559d9d0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:00:10.969 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.984+0000 7f8a0a844740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55559d9c50f0 0x55559d9c9390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:00:10.969 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.984+0000 7f8a0a844740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55559d9d5c80 con 0x55559d9c50f0 2024-03-08T21:00:10.969 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.984+0000 7f8a0a844740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55559d9d62e0 con 0x55559d9bff80 2024-03-08T21:00:10.969 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.984+0000 7f8a0a844740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55559d9d6940 con 0x55559d9bf590 2024-03-08T21:00:10.969 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.985+0000 7f8a05c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55559d9c50f0 0x55559d9c9390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:00:10.969 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.985+0000 7f8a05c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55559d9c50f0 0x55559d9c9390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:56730/0 (socket says 172.21.15.107:56730) 2024-03-08T21:00:10.969 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.985+0000 7f8a05c00640 1 -- 172.21.15.107:0/1727989864 learned_addr learned my addr 172.21.15.107:0/1727989864 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:00:10.970 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.985+0000 7f89ffe00640 1 --2- 172.21.15.107:0/1727989864 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55559d9bff80 0x55559d9d0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:00:10.970 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.985+0000 7f8a05c00640 1 -- 172.21.15.107:0/1727989864 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55559d9bf590 msgr2=0x55559d9baee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:00:10.970 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.985+0000 7f8a05200640 1 --2- 172.21.15.107:0/1727989864 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55559d9bf590 0x55559d9baee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:00:10.970 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.985+0000 7f8a05c00640 1 --2- 172.21.15.107:0/1727989864 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55559d9bf590 0x55559d9baee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:10.970 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.985+0000 7f8a05c00640 1 -- 172.21.15.107:0/1727989864 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55559d9bff80 msgr2=0x55559d9d0cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:00:10.970 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.985+0000 7f8a05c00640 1 --2- 172.21.15.107:0/1727989864 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55559d9bff80 0x55559d9d0cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:10.970 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.985+0000 7f8a05c00640 1 -- 172.21.15.107:0/1727989864 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55559d9d77b0 con 0x55559d9c50f0 2024-03-08T21:00:10.970 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.985+0000 7f89ffe00640 1 --2- 172.21.15.107:0/1727989864 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55559d9bff80 0x55559d9d0cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T21:00:10.970 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.985+0000 7f8a05c00640 1 --2- 172.21.15.107:0/1727989864 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55559d9c50f0 0x55559d9c9390 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f89f8096fa0 tx=0x7f89f809a740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=386c115f9c257a1 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:00:10.970 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.985+0000 7f89f5600640 1 -- 172.21.15.107:0/1727989864 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89f809ab40 con 0x55559d9c50f0 2024-03-08T21:00:10.970 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.986+0000 7f89f5600640 1 -- 172.21.15.107:0/1727989864 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f89f809ec90 con 0x55559d9c50f0 2024-03-08T21:00:10.971 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.986+0000 7f89f5600640 1 -- 172.21.15.107:0/1727989864 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89f80971b0 con 0x55559d9c50f0 2024-03-08T21:00:10.975 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.986+0000 7f8a0a844740 1 -- 172.21.15.107:0/1727989864 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55559d9c50f0 msgr2=0x55559d9c9390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:00:10.975 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.986+0000 7f8a0a844740 1 --2- 172.21.15.107:0/1727989864 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55559d9c50f0 0x55559d9c9390 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f89f8096fa0 tx=0x7f89f809a740 comp rx=0 tx=0).stop 2024-03-08T21:00:10.975 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.986+0000 7f8a0a844740 1 -- 172.21.15.107:0/1727989864 shutdown_connections 2024-03-08T21:00:10.975 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.986+0000 7f8a0a844740 1 --2- 172.21.15.107:0/1727989864 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55559d9c50f0 0x55559d9c9390 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:10.975 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.986+0000 7f8a0a844740 1 --2- 172.21.15.107:0/1727989864 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55559d9bff80 0x55559d9d0cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:10.975 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.986+0000 7f8a0a844740 1 --2- 172.21.15.107:0/1727989864 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55559d9bf590 0x55559d9baee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:10.976 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.986+0000 7f8a0a844740 1 -- 172.21.15.107:0/1727989864 >> 172.21.15.107:0/1727989864 conn(0x55559d9bab00 msgr2=0x55559d9c47b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:00:10.976 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.986+0000 7f8a0a844740 1 -- 172.21.15.107:0/1727989864 shutdown_connections 2024-03-08T21:00:10.976 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.986+0000 7f8a0a844740 1 -- 172.21.15.107:0/1727989864 wait complete. 2024-03-08T21:00:10.976 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.987+0000 7f8a0a844740 1 -- start start 2024-03-08T21:00:10.976 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.987+0000 7f8a0a844740 1 Processor -- start 2024-03-08T21:00:10.976 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.987+0000 7f8a0a844740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55559d9bf590 0x55559d9ce090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:00:10.976 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.987+0000 7f8a0a844740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55559d9bff80 0x55559d9ce5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:00:10.976 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.987+0000 7f8a0a844740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55559d9c50f0 0x55559d9cf540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:00:10.976 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.987+0000 7f8a0a844740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55559d9d7600 con 0x55559d9bf590 2024-03-08T21:00:10.976 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.987+0000 7f8a0a844740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55559d9d6fa0 con 0x55559d9c50f0 2024-03-08T21:00:10.976 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.987+0000 7f8a0a844740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55559d9d7c60 con 0x55559d9bff80 2024-03-08T21:00:10.976 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.987+0000 7f8a0a844740 10 client.0 _ll_register_callbacks cb 0x7f89e8037070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T21:00:10.976 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.987+0000 7f8a05200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55559d9bf590 0x55559d9ce090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:00:10.977 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.988+0000 7f8a05200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55559d9bf590 0x55559d9ce090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:56740/0 (socket says 172.21.15.107:56740) 2024-03-08T21:00:10.977 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.988+0000 7f8a05200640 1 -- 172.21.15.107:0/2703833089 learned_addr learned my addr 172.21.15.107:0/2703833089 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:00:10.977 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.988+0000 7f8a05200640 1 -- 172.21.15.107:0/2703833089 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55559d9bff80 msgr2=0x55559d9ce5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:00:10.977 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.988+0000 7f8a05200640 1 --2- 172.21.15.107:0/2703833089 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55559d9bff80 0x55559d9ce5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:10.977 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.988+0000 7f8a05200640 1 -- 172.21.15.107:0/2703833089 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55559d9c50f0 msgr2=0x55559d9cf540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:00:10.977 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.988+0000 7f8a05200640 1 --2- 172.21.15.107:0/2703833089 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55559d9c50f0 0x55559d9cf540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:10.977 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.988+0000 7f8a05200640 1 -- 172.21.15.107:0/2703833089 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55559d9cfb30 con 0x55559d9bf590 2024-03-08T21:00:10.977 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.988+0000 7f8a05200640 1 --2- 172.21.15.107:0/2703833089 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55559d9bf590 0x55559d9ce090 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f89f0000c00 tx=0x7f89f0026680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=999da69013d28da server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:00:10.977 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.988+0000 7f89eea00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-03-08T21:00:10.977 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.988+0000 7f89eea00640 1 -- 172.21.15.107:0/2703833089 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89f0022070 con 0x55559d9bf590 2024-03-08T21:00:10.977 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.988+0000 7f89eea00640 1 -- 172.21.15.107:0/2703833089 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f89f0026c90 con 0x55559d9bf590 2024-03-08T21:00:10.977 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.988+0000 7f8a0a844740 1 -- 172.21.15.107:0/2703833089 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f89f809ec90 con 0x55559d9bf590 2024-03-08T21:00:10.977 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.989+0000 7f89eea00640 1 -- 172.21.15.107:0/2703833089 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f89f0004020 con 0x55559d9bf590 2024-03-08T21:00:10.977 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.989+0000 7f8a0a844740 10 client.15279 fetch_fsmap learned FSMap version 27 2024-03-08T21:00:10.978 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.989+0000 7f8a0a844740 1 -- 172.21.15.107:0/2703833089 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55559d9d0e10 con 0x55559d9bf590 2024-03-08T21:00:10.978 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.989+0000 7f89eea00640 1 -- 172.21.15.107:0/2703833089 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f89f809ec90 con 0x55559d9bf590 2024-03-08T21:00:10.978 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.989+0000 7f89eea00640 1 -- 172.21.15.107:0/2703833089 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f89f00047a0 con 0x55559d9bf590 2024-03-08T21:00:10.978 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.989+0000 7f8a0a844740 10 client.15279 fetch_fsmap finished waiting for FSMap version 27 2024-03-08T21:00:10.978 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.989+0000 7f8a0a844740 10 client.15279 resolve_mds: resolved name 'k' to daemon mds.k 2024-03-08T21:00:10.978 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.989+0000 7f8a0a844740 20 client.15279 populate_metadata read hostname 'smithi107' 2024-03-08T21:00:10.978 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.989+0000 7f8a0a844740 1 --2- 172.21.15.107:0/2703833089 >> [v2:172.21.15.149:6832/1962161726,v1:172.21.15.149:6833/1962161726] conn(0x55559da32460 0x55559da2e070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:00:10.978 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.989+0000 7f8a0a844740 4 client.15279 mds_command: new command op to 34397 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-08T21:00:10.978 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.989+0000 7f8a0a844740 1 -- 172.21.15.107:0/2703833089 --> [v2:172.21.15.149:6832/1962161726,v1:172.21.15.149:6833/1962161726] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55559d92bf80 con 0x55559da32460 2024-03-08T21:00:10.978 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.990+0000 7f89ffe00640 1 --2- 172.21.15.107:0/2703833089 >> [v2:172.21.15.149:6832/1962161726,v1:172.21.15.149:6833/1962161726] conn(0x55559da32460 0x55559da2e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:00:10.978 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.991+0000 7f89ffe00640 1 --2- 172.21.15.107:0/2703833089 >> [v2:172.21.15.149:6832/1962161726,v1:172.21.15.149:6833/1962161726] conn(0x55559da32460 0x55559da2e070 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=5dac4436bab54395 server_cookie=e0179c6d5be2b681 in_seq=0 out_seq=0 2024-03-08T21:00:10.978 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.991+0000 7f89eea00640 10 client.15279 ms_handle_connect on v2:172.21.15.149:6832/1962161726 2024-03-08T21:00:10.978 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.991+0000 7f89eea00640 1 -- 172.21.15.107:0/2703833089 <== mds.3 v2:172.21.15.149:6832/1962161726 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+322 (crc 0 0 0) 0x55559d92bf80 con 0x55559da32460 2024-03-08T21:00:10.979 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.991+0000 7f89eea00640 10 client.15279 handle_command_reply: tid=0 2024-03-08T21:00:10.979 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.991+0000 7f8a0a844740 1 client.15279 shutdown 2024-03-08T21:00:10.979 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.992+0000 7f8a0a844740 1 -- 172.21.15.107:0/2703833089 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55559d9bf590 msgr2=0x55559d9ce090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:00:10.979 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.992+0000 7f8a0a844740 1 --2- 172.21.15.107:0/2703833089 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55559d9bf590 0x55559d9ce090 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f89f0000c00 tx=0x7f89f0026680 comp rx=0 tx=0).stop 2024-03-08T21:00:10.979 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.992+0000 7f8a0a844740 1 -- 172.21.15.107:0/2703833089 shutdown_connections 2024-03-08T21:00:10.979 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.992+0000 7f8a0a844740 1 --2- 172.21.15.107:0/2703833089 >> [v2:172.21.15.149:6832/1962161726,v1:172.21.15.149:6833/1962161726] conn(0x55559da32460 0x55559da2e070 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:10.979 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.992+0000 7f8a0a844740 1 --2- 172.21.15.107:0/2703833089 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55559d9c50f0 0x55559d9cf540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:10.979 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.992+0000 7f8a0a844740 1 --2- 172.21.15.107:0/2703833089 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55559d9bff80 0x55559d9ce5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:10.979 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.992+0000 7f8a0a844740 1 --2- 172.21.15.107:0/2703833089 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55559d9bf590 0x55559d9ce090 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:10.979 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.992+0000 7f89eea00640 0 client.15279 ms_handle_reset on v2:172.21.15.149:6832/1962161726 2024-03-08T21:00:10.979 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.992+0000 7f8a0a844740 1 -- 172.21.15.107:0/2703833089 >> 172.21.15.107:0/2703833089 conn(0x55559d9bab00 msgr2=0x55559d9c4370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:00:10.979 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.992+0000 7f8a0a844740 1 -- 172.21.15.107:0/2703833089 shutdown_connections 2024-03-08T21:00:10.979 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.992+0000 7f8a0a844740 1 -- 172.21.15.107:0/2703833089 wait complete. 2024-03-08T21:00:10.979 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.992+0000 7f8a0a844740 20 client.15279 trim_cache size 0 max 16384 2024-03-08T21:00:10.980 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-03-08T21:00:10.980 INFO:teuthology.orchestra.run.smithi107.stdout: "cluster_fsid": "dc20f0f8-dd8c-11ee-95c6-87774f69a715", 2024-03-08T21:00:10.980 INFO:teuthology.orchestra.run.smithi107.stdout: "whoami": 3, 2024-03-08T21:00:10.980 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 34397, 2024-03-08T21:00:10.980 INFO:teuthology.orchestra.run.smithi107.stdout: "want_state": "up:active", 2024-03-08T21:00:10.980 INFO:teuthology.orchestra.run.smithi107.stdout: "state": "up:active", 2024-03-08T21:00:10.980 INFO:teuthology.orchestra.run.smithi107.stdout: "fs_name": "cephfs", 2024-03-08T21:00:10.980 INFO:teuthology.orchestra.run.smithi107.stdout: "rank_uptime": 135.148153745, 2024-03-08T21:00:10.980 INFO:teuthology.orchestra.run.smithi107.stdout: "mdsmap_epoch": 27, 2024-03-08T21:00:10.980 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_epoch": 88, 2024-03-08T21:00:10.981 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_epoch_barrier": 86, 2024-03-08T21:00:10.981 INFO:teuthology.orchestra.run.smithi107.stdout: "uptime": 187.014640595 2024-03-08T21:00:10.981 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-03-08T21:00:10.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.992+0000 7f8a07c00640 1 -- 172.21.15.107:0/3443908453 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f89e00a2880 msgr2=0x7f89e00a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:00:10.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.992+0000 7f8a07c00640 1 --2- 172.21.15.107:0/3443908453 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f89e00a2880 0x7f89e00a4d40 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x55559d9c0380 tx=0x7f89f0005e90 comp rx=0 tx=0).stop 2024-03-08T21:00:10.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.992+0000 7f8a07c00640 1 -- 172.21.15.107:0/3443908453 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8a00077890 msgr2=0x7f8a001a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:00:10.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.992+0000 7f8a07c00640 1 --2- 172.21.15.107:0/3443908453 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8a00077890 0x7f8a001a5170 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f89f8009fd0 tx=0x7f89f800b8e0 comp rx=0 tx=0).stop 2024-03-08T21:00:10.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.993+0000 7f8a07c00640 1 -- 172.21.15.107:0/3443908453 shutdown_connections 2024-03-08T21:00:10.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.993+0000 7f8a07c00640 1 --2- 172.21.15.107:0/3443908453 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f89e00a2880 0x7f89e00a4d40 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:10.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.993+0000 7f8a07c00640 1 --2- 172.21.15.107:0/3443908453 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8a00077890 0x7f8a001a5170 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:10.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.993+0000 7f8a07c00640 1 --2- 172.21.15.107:0/3443908453 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8a00076ef0 0x7f8a0019e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:10.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.993+0000 7f8a07c00640 1 --2- 172.21.15.107:0/3443908453 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8a00075cb0 0x7f8a0019dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:10.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.993+0000 7f8a07c00640 1 -- 172.21.15.107:0/3443908453 >> 172.21.15.107:0/3443908453 conn(0x7f8a000feaa0 msgr2=0x7f8a00101e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:00:10.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.993+0000 7f8a07c00640 1 -- 172.21.15.107:0/3443908453 shutdown_connections 2024-03-08T21:00:10.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:10.993+0000 7f8a07c00640 1 -- 172.21.15.107:0/3443908453 wait complete. 2024-03-08T21:00:10.993 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.e status 2024-03-08T21:00:11.014 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:00:10 smithi149 ceph-mon[35042]: from='client.? 172.21.15.107:0/1495107928' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-08T21:00:11.014 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:00:10 smithi149 ceph-mon[35042]: from='client.? 172.21.15.107:0/714023804' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-08T21:00:11.198 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.215+0000 7f2f56a00640 1 -- 172.21.15.107:0/231526244 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2f50103510 msgr2=0x7f2f50103930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:00:11.198 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.215+0000 7f2f56a00640 1 --2- 172.21.15.107:0/231526244 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2f50103510 0x7f2f50103930 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f2f40009900 tx=0x7f2f4002bc30 comp rx=0 tx=0).stop 2024-03-08T21:00:11.198 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.215+0000 7f2f56a00640 1 -- 172.21.15.107:0/231526244 shutdown_connections 2024-03-08T21:00:11.198 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.215+0000 7f2f56a00640 1 --2- 172.21.15.107:0/231526244 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2f501050c0 0x7f2f5010b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:11.198 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.215+0000 7f2f56a00640 1 --2- 172.21.15.107:0/231526244 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2f50104700 0x7f2f50104b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:11.198 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.215+0000 7f2f56a00640 1 --2- 172.21.15.107:0/231526244 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2f50103510 0x7f2f50103930 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:11.199 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.215+0000 7f2f56a00640 1 -- 172.21.15.107:0/231526244 >> 172.21.15.107:0/231526244 conn(0x7f2f500feaa0 msgr2=0x7f2f50100ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:00:11.199 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.215+0000 7f2f56a00640 1 -- 172.21.15.107:0/231526244 shutdown_connections 2024-03-08T21:00:11.199 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.215+0000 7f2f56a00640 1 -- 172.21.15.107:0/231526244 wait complete. 2024-03-08T21:00:11.201 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.216+0000 7f2f56a00640 1 Processor -- start 2024-03-08T21:00:11.201 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.216+0000 7f2f56a00640 1 -- start start 2024-03-08T21:00:11.201 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.216+0000 7f2f56a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2f50103510 0x7f2f50199730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:00:11.201 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.216+0000 7f2f56a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2f50104700 0x7f2f50199c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:00:11.201 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.216+0000 7f2f56a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2f501050c0 0x7f2f501a0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:00:11.201 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.216+0000 7f2f56a00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f5010f070 con 0x7f2f50104700 2024-03-08T21:00:11.201 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.216+0000 7f2f56a00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f5010eef0 con 0x7f2f50103510 2024-03-08T21:00:11.201 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.216+0000 7f2f56a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f5010f1f0 con 0x7f2f501050c0 2024-03-08T21:00:11.201 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.216+0000 7f2f47400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2f50104700 0x7f2f50199c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:00:11.201 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.216+0000 7f2f47400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2f50104700 0x7f2f50199c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:56744/0 (socket says 172.21.15.107:56744) 2024-03-08T21:00:11.201 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.216+0000 7f2f47400640 1 -- 172.21.15.107:0/3089414133 learned_addr learned my addr 172.21.15.107:0/3089414133 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:00:11.201 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.217+0000 7f2f47400640 1 -- 172.21.15.107:0/3089414133 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2f501050c0 msgr2=0x7f2f501a0ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:00:11.201 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.217+0000 7f2f47400640 1 --2- 172.21.15.107:0/3089414133 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2f501050c0 0x7f2f501a0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:11.202 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.217+0000 7f2f47400640 1 -- 172.21.15.107:0/3089414133 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2f50103510 msgr2=0x7f2f50199730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:00:11.202 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.217+0000 7f2f47400640 1 --2- 172.21.15.107:0/3089414133 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2f50103510 0x7f2f50199730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:11.202 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.217+0000 7f2f47400640 1 -- 172.21.15.107:0/3089414133 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f501a1350 con 0x7f2f50104700 2024-03-08T21:00:11.202 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.217+0000 7f2f47400640 1 --2- 172.21.15.107:0/3089414133 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2f50104700 0x7f2f50199c70 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f2f3800d950 tx=0x7f2f3800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:00:11.202 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.217+0000 7f2f4d600640 1 -- 172.21.15.107:0/3089414133 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f38017070 con 0x7f2f50104700 2024-03-08T21:00:11.202 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.217+0000 7f2f4d600640 1 -- 172.21.15.107:0/3089414133 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f2f38009c30 con 0x7f2f50104700 2024-03-08T21:00:11.202 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.217+0000 7f2f4d600640 1 -- 172.21.15.107:0/3089414133 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f3800f620 con 0x7f2f50104700 2024-03-08T21:00:11.202 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.217+0000 7f2f56a00640 1 -- 172.21.15.107:0/3089414133 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f501a1640 con 0x7f2f50104700 2024-03-08T21:00:11.202 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.218+0000 7f2f56a00640 1 -- 172.21.15.107:0/3089414133 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f501a1b90 con 0x7f2f50104700 2024-03-08T21:00:11.204 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.220+0000 7f2f595eb740 1 Processor -- start 2024-03-08T21:00:11.204 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.221+0000 7f2f595eb740 1 -- start start 2024-03-08T21:00:11.204 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.221+0000 7f2f4d600640 1 -- 172.21.15.107:0/3089414133 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f2f380109a0 con 0x7f2f50104700 2024-03-08T21:00:11.204 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.221+0000 7f2f595eb740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562698783590 0x5626987839b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:00:11.204 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.221+0000 7f2f595eb740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562698783f80 0x562698788a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:00:11.204 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.221+0000 7f2f595eb740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5626987890f0 0x56269878b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:00:11.204 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.221+0000 7f2f595eb740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626986e86f0 con 0x5626987890f0 2024-03-08T21:00:11.204 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.221+0000 7f2f595eb740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626986919e0 con 0x562698783f80 2024-03-08T21:00:11.204 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.221+0000 7f2f595eb740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626986e4790 con 0x562698783590 2024-03-08T21:00:11.206 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.221+0000 7f2f47400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5626987890f0 0x56269878b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:00:11.206 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.221+0000 7f2f47400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5626987890f0 0x56269878b580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:56748/0 (socket says 172.21.15.107:56748) 2024-03-08T21:00:11.206 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.221+0000 7f2f47400640 1 -- 172.21.15.107:0/815008117 learned_addr learned my addr 172.21.15.107:0/815008117 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:00:11.206 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.221+0000 7f2f54a00640 1 --2- 172.21.15.107:0/815008117 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562698783590 0x5626987839b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:00:11.206 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.222+0000 7f2f4fe00640 1 --2- 172.21.15.107:0/815008117 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562698783f80 0x562698788a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:00:11.206 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.222+0000 7f2f47400640 1 -- 172.21.15.107:0/815008117 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562698783590 msgr2=0x5626987839b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:00:11.206 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.222+0000 7f2f47400640 1 --2- 172.21.15.107:0/815008117 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562698783590 0x5626987839b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:11.206 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.222+0000 7f2f47400640 1 -- 172.21.15.107:0/815008117 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562698783f80 msgr2=0x562698788a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:00:11.206 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.222+0000 7f2f47400640 1 --2- 172.21.15.107:0/815008117 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562698783f80 0x562698788a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:11.206 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.222+0000 7f2f47400640 1 -- 172.21.15.107:0/815008117 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562698685720 con 0x5626987890f0 2024-03-08T21:00:11.206 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.222+0000 7f2f47400640 1 --2- 172.21.15.107:0/815008117 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5626987890f0 0x56269878b580 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f2f380d4d00 tx=0x7f2f380d6750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af4339900fe61497 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:00:11.207 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.222+0000 7f2f45200640 1 -- 172.21.15.107:0/815008117 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f380ced90 con 0x5626987890f0 2024-03-08T21:00:11.207 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.222+0000 7f2f4d600640 1 --2- 172.21.15.107:0/3089414133 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f2f300a2800 0x7f2f300a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:00:11.207 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.222+0000 7f2f45200640 1 -- 172.21.15.107:0/815008117 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f2f380db070 con 0x5626987890f0 2024-03-08T21:00:11.207 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.222+0000 7f2f45200640 1 -- 172.21.15.107:0/815008117 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f38099e00 con 0x5626987890f0 2024-03-08T21:00:11.207 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.223+0000 7f2f54a00640 1 --2- 172.21.15.107:0/3089414133 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f2f300a2800 0x7f2f300a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:00:11.207 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.223+0000 7f2f4d600640 1 -- 172.21.15.107:0/3089414133 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f2f380ce7a0 con 0x7f2f50104700 2024-03-08T21:00:11.210 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.223+0000 7f2f595eb740 1 -- 172.21.15.107:0/815008117 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5626987890f0 msgr2=0x56269878b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:00:11.210 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.223+0000 7f2f595eb740 1 --2- 172.21.15.107:0/815008117 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5626987890f0 0x56269878b580 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f2f380d4d00 tx=0x7f2f380d6750 comp rx=0 tx=0).stop 2024-03-08T21:00:11.210 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.223+0000 7f2f54a00640 1 --2- 172.21.15.107:0/3089414133 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f2f300a2800 0x7f2f300a4cc0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x5626987847e0 tx=0x7f2f480073a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:00:11.210 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.223+0000 7f2f595eb740 1 -- 172.21.15.107:0/815008117 shutdown_connections 2024-03-08T21:00:11.211 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.223+0000 7f2f595eb740 1 --2- 172.21.15.107:0/815008117 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5626987890f0 0x56269878b580 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:11.211 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.223+0000 7f2f595eb740 1 --2- 172.21.15.107:0/815008117 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562698783f80 0x562698788a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:11.211 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.223+0000 7f2f595eb740 1 --2- 172.21.15.107:0/815008117 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562698783590 0x5626987839b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:11.211 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.223+0000 7f2f595eb740 1 -- 172.21.15.107:0/815008117 >> 172.21.15.107:0/815008117 conn(0x56269877eb00 msgr2=0x56269877ef00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:00:11.211 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.224+0000 7f2f595eb740 1 -- 172.21.15.107:0/815008117 shutdown_connections 2024-03-08T21:00:11.211 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.224+0000 7f2f595eb740 1 -- 172.21.15.107:0/815008117 wait complete. 2024-03-08T21:00:11.211 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.224+0000 7f2f595eb740 1 -- start start 2024-03-08T21:00:11.211 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.224+0000 7f2f595eb740 1 Processor -- start 2024-03-08T21:00:11.211 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.224+0000 7f2f595eb740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562698783590 0x56269878d800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:00:11.211 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.225+0000 7f2f595eb740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562698783f80 0x56269878dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:00:11.211 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.225+0000 7f2f595eb740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5626987890f0 0x562698794d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:00:11.211 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.225+0000 7f2f595eb740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x56269878c0a0 con 0x562698783f80 2024-03-08T21:00:11.211 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.225+0000 7f2f595eb740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626986a4e20 con 0x5626987890f0 2024-03-08T21:00:11.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.225+0000 7f2f595eb740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56269878c220 con 0x562698783590 2024-03-08T21:00:11.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.225+0000 7f2f595eb740 10 client.0 _ll_register_callbacks cb 0x5626986df8d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T21:00:11.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.225+0000 7f2f47400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562698783f80 0x56269878dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:00:11.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.225+0000 7f2f47400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562698783f80 0x56269878dd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:56750/0 (socket says 172.21.15.107:56750) 2024-03-08T21:00:11.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.225+0000 7f2f47400640 1 -- 172.21.15.107:0/2703419554 learned_addr learned my addr 172.21.15.107:0/2703419554 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:00:11.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.225+0000 7f2f4fe00640 1 --2- 172.21.15.107:0/2703419554 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562698783590 0x56269878d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:00:11.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.225+0000 7f2f54a00640 1 --2- 172.21.15.107:0/2703419554 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5626987890f0 0x562698794d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:00:11.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.225+0000 7f2f47400640 1 -- 172.21.15.107:0/2703419554 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562698783590 msgr2=0x56269878d800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:00:11.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.225+0000 7f2f47400640 1 --2- 172.21.15.107:0/2703419554 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562698783590 0x56269878d800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:11.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.225+0000 7f2f47400640 1 -- 172.21.15.107:0/2703419554 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5626987890f0 msgr2=0x562698794d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:00:11.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.225+0000 7f2f47400640 1 --2- 172.21.15.107:0/2703419554 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5626987890f0 0x562698794d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:11.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.225+0000 7f2f47400640 1 -- 172.21.15.107:0/2703419554 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5626983fd2d0 con 0x562698783f80 2024-03-08T21:00:11.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.225+0000 7f2f47400640 1 --2- 172.21.15.107:0/2703419554 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562698783f80 0x56269878dd40 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f2f38000c00 tx=0x7f2f38099400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b651aeea80273d7 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:00:11.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.225+0000 7f2f3d600640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-03-08T21:00:11.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.226+0000 7f2f3d600640 1 -- 172.21.15.107:0/2703419554 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f38098da0 con 0x562698783f80 2024-03-08T21:00:11.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.226+0000 7f2f3d600640 1 -- 172.21.15.107:0/2703419554 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f2f380d47b0 con 0x562698783f80 2024-03-08T21:00:11.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.226+0000 7f2f595eb740 1 -- 172.21.15.107:0/2703419554 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2f380db070 con 0x562698783f80 2024-03-08T21:00:11.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.226+0000 7f2f3d600640 1 -- 172.21.15.107:0/2703419554 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2f38098fe0 con 0x562698783f80 2024-03-08T21:00:11.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.226+0000 7f2f595eb740 10 client.15291 fetch_fsmap learned FSMap version 27 2024-03-08T21:00:11.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.226+0000 7f2f595eb740 1 -- 172.21.15.107:0/2703419554 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5626984c76b0 con 0x562698783f80 2024-03-08T21:00:11.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.226+0000 7f2f3d600640 1 -- 172.21.15.107:0/2703419554 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f2f380db070 con 0x562698783f80 2024-03-08T21:00:11.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.226+0000 7f2f3d600640 1 -- 172.21.15.107:0/2703419554 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f2f38097770 con 0x562698783f80 2024-03-08T21:00:11.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.226+0000 7f2f595eb740 10 client.15291 fetch_fsmap finished waiting for FSMap version 27 2024-03-08T21:00:11.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.226+0000 7f2f595eb740 10 client.15291 resolve_mds: resolved name 'e' to daemon mds.e 2024-03-08T21:00:11.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.226+0000 7f2f595eb740 20 client.15291 populate_metadata read hostname 'smithi107' 2024-03-08T21:00:11.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.226+0000 7f2f595eb740 1 --2- 172.21.15.107:0/2703419554 >> [v2:172.21.15.149:6834/442657066,v1:172.21.15.149:6835/442657066] conn(0x562698799d60 0x56269879e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:00:11.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.226+0000 7f2f595eb740 4 client.15291 mds_command: new command op to 34415 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-08T21:00:11.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.226+0000 7f2f595eb740 1 -- 172.21.15.107:0/2703419554 --> [v2:172.21.15.149:6834/442657066,v1:172.21.15.149:6835/442657066] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5626986f5c30 con 0x562698799d60 2024-03-08T21:00:11.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.227+0000 7f2f4fe00640 1 --2- 172.21.15.107:0/2703419554 >> [v2:172.21.15.149:6834/442657066,v1:172.21.15.149:6835/442657066] conn(0x562698799d60 0x56269879e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:00:11.215 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.228+0000 7f2f3d600640 10 client.15291 ms_handle_connect on v2:172.21.15.149:6834/442657066 2024-03-08T21:00:11.215 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.228+0000 7f2f4fe00640 1 --2- 172.21.15.107:0/2703419554 >> [v2:172.21.15.149:6834/442657066,v1:172.21.15.149:6835/442657066] conn(0x562698799d60 0x56269879e1e0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9a5b642fe2b1efc0 server_cookie=793b667d709f9d60 in_seq=0 out_seq=0 2024-03-08T21:00:11.215 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.230+0000 7f2f3d600640 1 -- 172.21.15.107:0/2703419554 <== mds.2 v2:172.21.15.149:6834/442657066 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5626986f5c30 con 0x562698799d60 2024-03-08T21:00:11.215 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.230+0000 7f2f3d600640 10 client.15291 handle_command_reply: tid=0 2024-03-08T21:00:11.215 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.231+0000 7f2f595eb740 1 client.15291 shutdown 2024-03-08T21:00:11.215 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.231+0000 7f2f595eb740 1 -- 172.21.15.107:0/2703419554 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562698783f80 msgr2=0x56269878dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:00:11.215 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.231+0000 7f2f595eb740 1 --2- 172.21.15.107:0/2703419554 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562698783f80 0x56269878dd40 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f2f38000c00 tx=0x7f2f38099400 comp rx=0 tx=0).stop 2024-03-08T21:00:11.215 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.231+0000 7f2f595eb740 1 -- 172.21.15.107:0/2703419554 shutdown_connections 2024-03-08T21:00:11.215 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.231+0000 7f2f595eb740 1 --2- 172.21.15.107:0/2703419554 >> [v2:172.21.15.149:6834/442657066,v1:172.21.15.149:6835/442657066] conn(0x562698799d60 0x56269879e1e0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:11.215 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.231+0000 7f2f595eb740 1 --2- 172.21.15.107:0/2703419554 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5626987890f0 0x562698794d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:11.215 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.231+0000 7f2f3d600640 0 client.15291 ms_handle_reset on v2:172.21.15.149:6834/442657066 2024-03-08T21:00:11.215 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.231+0000 7f2f595eb740 1 --2- 172.21.15.107:0/2703419554 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562698783f80 0x56269878dd40 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:11.216 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.231+0000 7f2f595eb740 1 --2- 172.21.15.107:0/2703419554 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562698783590 0x56269878d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:11.216 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.231+0000 7f2f595eb740 1 -- 172.21.15.107:0/2703419554 >> 172.21.15.107:0/2703419554 conn(0x56269877eb00 msgr2=0x56269877dc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:00:11.216 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.231+0000 7f2f595eb740 1 -- 172.21.15.107:0/2703419554 shutdown_connections 2024-03-08T21:00:11.216 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.231+0000 7f2f595eb740 1 -- 172.21.15.107:0/2703419554 wait complete. 2024-03-08T21:00:11.216 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.231+0000 7f2f595eb740 20 client.15291 trim_cache size 0 max 16384 2024-03-08T21:00:11.234 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.249+0000 7f2f595eb740 1 Processor -- start 2024-03-08T21:00:11.234 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.250+0000 7f2f595eb740 1 -- start start 2024-03-08T21:00:11.234 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.250+0000 7f2f595eb740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562698783590 0x56269878bf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:00:11.234 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.250+0000 7f2f595eb740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562698783f80 0x562698794d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:00:11.234 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.250+0000 7f2f595eb740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5626987890f0 0x56269877eee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:00:11.235 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.250+0000 7f2f595eb740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x562698799580 con 0x5626987890f0 2024-03-08T21:00:11.235 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.250+0000 7f2f595eb740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x562698799be0 con 0x562698783590 2024-03-08T21:00:11.235 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.250+0000 7f2f595eb740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56269879a240 con 0x562698783f80 2024-03-08T21:00:11.235 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.250+0000 7f2f54a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5626987890f0 0x56269877eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:00:11.235 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.250+0000 7f2f54a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5626987890f0 0x56269877eee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:56756/0 (socket says 172.21.15.107:56756) 2024-03-08T21:00:11.235 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.250+0000 7f2f54a00640 1 -- 172.21.15.107:0/2327186056 learned_addr learned my addr 172.21.15.107:0/2327186056 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:00:11.235 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.250+0000 7f2f4fe00640 1 --2- 172.21.15.107:0/2327186056 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562698783590 0x56269878bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:00:11.235 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.251+0000 7f2f47400640 1 --2- 172.21.15.107:0/2327186056 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562698783f80 0x562698794d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:00:11.235 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.251+0000 7f2f54a00640 1 -- 172.21.15.107:0/2327186056 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562698783f80 msgr2=0x562698794d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:00:11.235 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.251+0000 7f2f54a00640 1 --2- 172.21.15.107:0/2327186056 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562698783f80 0x562698794d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:11.235 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.251+0000 7f2f54a00640 1 -- 172.21.15.107:0/2327186056 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562698783590 msgr2=0x56269878bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:00:11.235 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.251+0000 7f2f54a00640 1 --2- 172.21.15.107:0/2327186056 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562698783590 0x56269878bf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:11.235 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.251+0000 7f2f54a00640 1 -- 172.21.15.107:0/2327186056 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56269878e6a0 con 0x5626987890f0 2024-03-08T21:00:11.235 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.251+0000 7f2f4fe00640 1 --2- 172.21.15.107:0/2327186056 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562698783590 0x56269878bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T21:00:11.236 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.251+0000 7f2f54a00640 1 --2- 172.21.15.107:0/2327186056 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5626987890f0 0x56269877eee0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f2f480049b0 tx=0x7f2f48011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=911088a564b60d7 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:00:11.239 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.251+0000 7f2f3fe00640 1 -- 172.21.15.107:0/2327186056 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f4801d830 con 0x5626987890f0 2024-03-08T21:00:11.239 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.252+0000 7f2f3fe00640 1 -- 172.21.15.107:0/2327186056 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f2f48010e40 con 0x5626987890f0 2024-03-08T21:00:11.239 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.252+0000 7f2f3fe00640 1 -- 172.21.15.107:0/2327186056 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f4801cc80 con 0x5626987890f0 2024-03-08T21:00:11.239 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.252+0000 7f2f595eb740 1 -- 172.21.15.107:0/2327186056 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5626987890f0 msgr2=0x56269877eee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:00:11.239 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.252+0000 7f2f595eb740 1 --2- 172.21.15.107:0/2327186056 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5626987890f0 0x56269877eee0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f2f480049b0 tx=0x7f2f48011000 comp rx=0 tx=0).stop 2024-03-08T21:00:11.240 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.252+0000 7f2f595eb740 1 -- 172.21.15.107:0/2327186056 shutdown_connections 2024-03-08T21:00:11.240 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.252+0000 7f2f595eb740 1 --2- 172.21.15.107:0/2327186056 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5626987890f0 0x56269877eee0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:11.240 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.252+0000 7f2f595eb740 1 --2- 172.21.15.107:0/2327186056 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562698783f80 0x562698794d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:11.240 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.252+0000 7f2f595eb740 1 --2- 172.21.15.107:0/2327186056 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562698783590 0x56269878bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:11.240 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.252+0000 7f2f595eb740 1 -- 172.21.15.107:0/2327186056 >> 172.21.15.107:0/2327186056 conn(0x56269877eb00 msgr2=0x56269877dc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:00:11.240 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.252+0000 7f2f595eb740 1 -- 172.21.15.107:0/2327186056 shutdown_connections 2024-03-08T21:00:11.240 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.253+0000 7f2f595eb740 1 -- 172.21.15.107:0/2327186056 wait complete. 2024-03-08T21:00:11.240 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.253+0000 7f2f595eb740 1 -- start start 2024-03-08T21:00:11.240 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.253+0000 7f2f595eb740 1 Processor -- start 2024-03-08T21:00:11.240 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.253+0000 7f2f595eb740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562698783590 0x562698791a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:00:11.240 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.253+0000 7f2f595eb740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562698783f80 0x562698791f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:00:11.240 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.253+0000 7f2f595eb740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5626987890f0 0x5626987f86b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:00:11.240 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.253+0000 7f2f595eb740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x56269879af00 con 0x562698783f80 2024-03-08T21:00:11.241 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.253+0000 7f2f595eb740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x56269879a8a0 con 0x562698783590 2024-03-08T21:00:11.241 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.253+0000 7f2f595eb740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56269879b560 con 0x5626987890f0 2024-03-08T21:00:11.241 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.254+0000 7f2f595eb740 10 client.0 _ll_register_callbacks cb 0x7f2f380d44e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T21:00:11.241 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.254+0000 7f2f47400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562698783f80 0x562698791f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:00:11.241 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.254+0000 7f2f47400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562698783f80 0x562698791f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:56770/0 (socket says 172.21.15.107:56770) 2024-03-08T21:00:11.241 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.254+0000 7f2f47400640 1 -- 172.21.15.107:0/1442184221 learned_addr learned my addr 172.21.15.107:0/1442184221 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:00:11.241 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.254+0000 7f2f47400640 1 -- 172.21.15.107:0/1442184221 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5626987890f0 msgr2=0x5626987f86b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:00:11.241 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.254+0000 7f2f47400640 1 --2- 172.21.15.107:0/1442184221 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5626987890f0 0x5626987f86b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:11.241 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.254+0000 7f2f47400640 1 -- 172.21.15.107:0/1442184221 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562698783590 msgr2=0x562698791a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:00:11.241 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.254+0000 7f2f47400640 1 --2- 172.21.15.107:0/1442184221 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562698783590 0x562698791a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:11.241 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.254+0000 7f2f47400640 1 -- 172.21.15.107:0/1442184221 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562698794ff0 con 0x562698783f80 2024-03-08T21:00:11.241 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.254+0000 7f2f47400640 1 --2- 172.21.15.107:0/1442184221 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562698783f80 0x562698791f70 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f2f38099880 tx=0x7f2f380979a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1989938fc847b379 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:00:11.241 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.254+0000 7f2f3d600640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-03-08T21:00:11.241 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.254+0000 7f2f3d600640 1 -- 172.21.15.107:0/1442184221 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f38020070 con 0x562698783f80 2024-03-08T21:00:11.241 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.255+0000 7f2f3d600640 1 -- 172.21.15.107:0/1442184221 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f2f3809a550 con 0x562698783f80 2024-03-08T21:00:11.242 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.255+0000 7f2f595eb740 1 -- 172.21.15.107:0/1442184221 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5626986eff80 con 0x562698783f80 2024-03-08T21:00:11.242 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.255+0000 7f2f3d600640 1 -- 172.21.15.107:0/1442184221 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2f38099240 con 0x562698783f80 2024-03-08T21:00:11.242 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.255+0000 7f2f595eb740 10 client.15297 fetch_fsmap learned FSMap version 27 2024-03-08T21:00:11.242 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.255+0000 7f2f595eb740 1 -- 172.21.15.107:0/1442184221 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5626987f9c30 con 0x562698783f80 2024-03-08T21:00:11.242 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.255+0000 7f2f3d600640 1 -- 172.21.15.107:0/1442184221 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x5626986eff80 con 0x562698783f80 2024-03-08T21:00:11.242 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.255+0000 7f2f3d600640 1 -- 172.21.15.107:0/1442184221 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f2f38096e50 con 0x562698783f80 2024-03-08T21:00:11.242 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.255+0000 7f2f595eb740 10 client.15297 fetch_fsmap finished waiting for FSMap version 27 2024-03-08T21:00:11.242 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.255+0000 7f2f595eb740 10 client.15297 resolve_mds: resolved name 'e' to daemon mds.e 2024-03-08T21:00:11.242 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.255+0000 7f2f595eb740 20 client.15297 populate_metadata read hostname 'smithi107' 2024-03-08T21:00:11.242 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.255+0000 7f2f595eb740 1 --2- 172.21.15.107:0/1442184221 >> [v2:172.21.15.149:6834/442657066,v1:172.21.15.149:6835/442657066] conn(0x5626987fdfd0 0x5626987f9c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:00:11.242 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.255+0000 7f2f595eb740 4 client.15297 mds_command: new command op to 34415 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-08T21:00:11.242 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.255+0000 7f2f595eb740 1 -- 172.21.15.107:0/1442184221 --> [v2:172.21.15.149:6834/442657066,v1:172.21.15.149:6835/442657066] -- command(tid 0: {"prefix": "status"}) v1 -- 0x56269866e860 con 0x5626987fdfd0 2024-03-08T21:00:11.242 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.256+0000 7f2f4fe00640 1 --2- 172.21.15.107:0/1442184221 >> [v2:172.21.15.149:6834/442657066,v1:172.21.15.149:6835/442657066] conn(0x5626987fdfd0 0x5626987f9c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:00:11.244 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.257+0000 7f2f4fe00640 1 --2- 172.21.15.107:0/1442184221 >> [v2:172.21.15.149:6834/442657066,v1:172.21.15.149:6835/442657066] conn(0x5626987fdfd0 0x5626987f9c30 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=643d13e5bad7dfe7 server_cookie=4d5266000404d1e9 in_seq=0 out_seq=0 2024-03-08T21:00:11.244 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.257+0000 7f2f3d600640 10 client.15297 ms_handle_connect on v2:172.21.15.149:6834/442657066 2024-03-08T21:00:11.244 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.257+0000 7f2f3d600640 1 -- 172.21.15.107:0/1442184221 <== mds.2 v2:172.21.15.149:6834/442657066 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x56269866e860 con 0x5626987fdfd0 2024-03-08T21:00:11.244 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.257+0000 7f2f3d600640 10 client.15297 handle_command_reply: tid=0 2024-03-08T21:00:11.244 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.257+0000 7f2f595eb740 1 client.15297 shutdown 2024-03-08T21:00:11.244 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.257+0000 7f2f595eb740 1 -- 172.21.15.107:0/1442184221 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562698783f80 msgr2=0x562698791f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:00:11.244 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.257+0000 7f2f595eb740 1 --2- 172.21.15.107:0/1442184221 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562698783f80 0x562698791f70 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f2f38099880 tx=0x7f2f380979a0 comp rx=0 tx=0).stop 2024-03-08T21:00:11.244 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.258+0000 7f2f595eb740 1 -- 172.21.15.107:0/1442184221 shutdown_connections 2024-03-08T21:00:11.244 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.258+0000 7f2f595eb740 1 --2- 172.21.15.107:0/1442184221 >> [v2:172.21.15.149:6834/442657066,v1:172.21.15.149:6835/442657066] conn(0x5626987fdfd0 0x5626987f9c30 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:11.244 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.258+0000 7f2f595eb740 1 --2- 172.21.15.107:0/1442184221 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5626987890f0 0x5626987f86b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:11.244 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.258+0000 7f2f595eb740 1 --2- 172.21.15.107:0/1442184221 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562698783f80 0x562698791f70 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:11.244 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.258+0000 7f2f595eb740 1 --2- 172.21.15.107:0/1442184221 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562698783590 0x562698791a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:11.245 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.258+0000 7f2f3d600640 0 client.15297 ms_handle_reset on v2:172.21.15.149:6834/442657066 2024-03-08T21:00:11.245 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.258+0000 7f2f595eb740 1 -- 172.21.15.107:0/1442184221 >> 172.21.15.107:0/1442184221 conn(0x56269877eb00 msgr2=0x56269877d3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:00:11.245 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.258+0000 7f2f595eb740 1 -- 172.21.15.107:0/1442184221 shutdown_connections 2024-03-08T21:00:11.245 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.258+0000 7f2f595eb740 1 -- 172.21.15.107:0/1442184221 wait complete. 2024-03-08T21:00:11.245 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.258+0000 7f2f595eb740 20 client.15297 trim_cache size 0 max 16384 2024-03-08T21:00:11.245 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-03-08T21:00:11.245 INFO:teuthology.orchestra.run.smithi107.stdout: "cluster_fsid": "dc20f0f8-dd8c-11ee-95c6-87774f69a715", 2024-03-08T21:00:11.245 INFO:teuthology.orchestra.run.smithi107.stdout: "whoami": 2, 2024-03-08T21:00:11.245 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 34415, 2024-03-08T21:00:11.245 INFO:teuthology.orchestra.run.smithi107.stdout: "want_state": "up:active", 2024-03-08T21:00:11.246 INFO:teuthology.orchestra.run.smithi107.stdout: "state": "up:active", 2024-03-08T21:00:11.246 INFO:teuthology.orchestra.run.smithi107.stdout: "fs_name": "cephfs", 2024-03-08T21:00:11.246 INFO:teuthology.orchestra.run.smithi107.stdout: "rank_uptime": 136.42590008400001, 2024-03-08T21:00:11.246 INFO:teuthology.orchestra.run.smithi107.stdout: "mdsmap_epoch": 27, 2024-03-08T21:00:11.246 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_epoch": 88, 2024-03-08T21:00:11.246 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_epoch_barrier": 86, 2024-03-08T21:00:11.246 INFO:teuthology.orchestra.run.smithi107.stdout: "uptime": 175.139509269 2024-03-08T21:00:11.246 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-03-08T21:00:11.247 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.258+0000 7f2f56a00640 1 -- 172.21.15.107:0/3089414133 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f2f300a2800 msgr2=0x7f2f300a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:00:11.247 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.258+0000 7f2f56a00640 1 --2- 172.21.15.107:0/3089414133 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f2f300a2800 0x7f2f300a4cc0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x5626987847e0 tx=0x7f2f480073a0 comp rx=0 tx=0).stop 2024-03-08T21:00:11.247 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.258+0000 7f2f56a00640 1 -- 172.21.15.107:0/3089414133 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2f50104700 msgr2=0x7f2f50199c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:00:11.247 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.258+0000 7f2f56a00640 1 --2- 172.21.15.107:0/3089414133 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2f50104700 0x7f2f50199c70 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f2f3800d950 tx=0x7f2f3800de10 comp rx=0 tx=0).stop 2024-03-08T21:00:11.247 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.259+0000 7f2f56a00640 1 -- 172.21.15.107:0/3089414133 shutdown_connections 2024-03-08T21:00:11.247 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.259+0000 7f2f56a00640 1 --2- 172.21.15.107:0/3089414133 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f2f300a2800 0x7f2f300a4cc0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:11.247 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.259+0000 7f2f56a00640 1 --2- 172.21.15.107:0/3089414133 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2f501050c0 0x7f2f501a0ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:11.247 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.259+0000 7f2f56a00640 1 --2- 172.21.15.107:0/3089414133 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2f50104700 0x7f2f50199c70 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:11.247 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.259+0000 7f2f56a00640 1 --2- 172.21.15.107:0/3089414133 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2f50103510 0x7f2f50199730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:11.247 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.259+0000 7f2f56a00640 1 -- 172.21.15.107:0/3089414133 >> 172.21.15.107:0/3089414133 conn(0x7f2f500feaa0 msgr2=0x7f2f50107ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:00:11.248 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.259+0000 7f2f56a00640 1 -- 172.21.15.107:0/3089414133 shutdown_connections 2024-03-08T21:00:11.248 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.259+0000 7f2f56a00640 1 -- 172.21.15.107:0/3089414133 wait complete. 2024-03-08T21:00:11.261 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2024-03-08T21:00:11.472 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.489+0000 7fbdf8c00640 1 -- 172.21.15.107:0/2802569574 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbdf4108b00 msgr2=0x7fbdf4108f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:00:11.472 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.489+0000 7fbdf8c00640 1 --2- 172.21.15.107:0/2802569574 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbdf4108b00 0x7fbdf4108f80 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fbddc009900 tx=0x7fbddc02bc30 comp rx=0 tx=0).stop 2024-03-08T21:00:11.472 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.489+0000 7fbdf8c00640 1 -- 172.21.15.107:0/2802569574 shutdown_connections 2024-03-08T21:00:11.473 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.489+0000 7fbdf8c00640 1 --2- 172.21.15.107:0/2802569574 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbdf41094c0 0x7fbdf410fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:11.473 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.489+0000 7fbdf8c00640 1 --2- 172.21.15.107:0/2802569574 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbdf4108b00 0x7fbdf4108f80 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:11.473 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.489+0000 7fbdf8c00640 1 --2- 172.21.15.107:0/2802569574 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbdf4107910 0x7fbdf4107d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:11.473 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.489+0000 7fbdf8c00640 1 -- 172.21.15.107:0/2802569574 >> 172.21.15.107:0/2802569574 conn(0x7fbdf40768b0 msgr2=0x7fbdf4078cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:00:11.473 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.489+0000 7fbdf8c00640 1 -- 172.21.15.107:0/2802569574 shutdown_connections 2024-03-08T21:00:11.473 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.489+0000 7fbdf8c00640 1 -- 172.21.15.107:0/2802569574 wait complete. 2024-03-08T21:00:11.475 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.490+0000 7fbdf8c00640 1 Processor -- start 2024-03-08T21:00:11.475 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.490+0000 7fbdf8c00640 1 -- start start 2024-03-08T21:00:11.475 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.490+0000 7fbdf8c00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbdf4107910 0x7fbdf419db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:00:11.475 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.490+0000 7fbdf8c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbdf4108b00 0x7fbdf419e070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:00:11.475 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.490+0000 7fbdf8c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbdf41094c0 0x7fbdf41a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:00:11.475 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.490+0000 7fbdf8c00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdf4113490 con 0x7fbdf4108b00 2024-03-08T21:00:11.475 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.490+0000 7fbdf8c00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdf4113310 con 0x7fbdf4107910 2024-03-08T21:00:11.476 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.490+0000 7fbdf8c00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdf4113610 con 0x7fbdf41094c0 2024-03-08T21:00:11.476 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.490+0000 7fbde9400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbdf4108b00 0x7fbdf419e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:00:11.476 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.491+0000 7fbde9400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbdf4108b00 0x7fbdf419e070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:56788/0 (socket says 172.21.15.107:56788) 2024-03-08T21:00:11.476 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.491+0000 7fbde9400640 1 -- 172.21.15.107:0/4109952411 learned_addr learned my addr 172.21.15.107:0/4109952411 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:00:11.476 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.491+0000 7fbdf2800640 1 --2- 172.21.15.107:0/4109952411 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbdf41094c0 0x7fbdf41a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:00:11.476 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.491+0000 7fbde9400640 1 -- 172.21.15.107:0/4109952411 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbdf41094c0 msgr2=0x7fbdf41a50e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:00:11.476 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.491+0000 7fbdf1e00640 1 --2- 172.21.15.107:0/4109952411 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbdf4107910 0x7fbdf419db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:00:11.476 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.491+0000 7fbde9400640 1 --2- 172.21.15.107:0/4109952411 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbdf41094c0 0x7fbdf41a50e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:11.476 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.491+0000 7fbde9400640 1 -- 172.21.15.107:0/4109952411 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbdf4107910 msgr2=0x7fbdf419db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:00:11.476 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.491+0000 7fbde9400640 1 --2- 172.21.15.107:0/4109952411 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbdf4107910 0x7fbdf419db30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:11.476 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.491+0000 7fbde9400640 1 -- 172.21.15.107:0/4109952411 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdf41a5750 con 0x7fbdf4108b00 2024-03-08T21:00:11.477 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.491+0000 7fbde9400640 1 --2- 172.21.15.107:0/4109952411 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbdf4108b00 0x7fbdf419e070 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fbddc000c00 tx=0x7fbddc0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:00:11.477 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.491+0000 7fbdeb400640 1 -- 172.21.15.107:0/4109952411 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbddc043020 con 0x7fbdf4108b00 2024-03-08T21:00:11.477 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.492+0000 7fbdf8c00640 1 -- 172.21.15.107:0/4109952411 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbdf41a59e0 con 0x7fbdf4108b00 2024-03-08T21:00:11.477 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.492+0000 7fbdeb400640 1 -- 172.21.15.107:0/4109952411 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fbddc0054c0 con 0x7fbdf4108b00 2024-03-08T21:00:11.477 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.492+0000 7fbdeb400640 1 -- 172.21.15.107:0/4109952411 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbddc003d80 con 0x7fbdf4108b00 2024-03-08T21:00:11.477 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.492+0000 7fbdf8c00640 1 -- 172.21.15.107:0/4109952411 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbdf41a5f80 con 0x7fbdf4108b00 2024-03-08T21:00:11.479 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.495+0000 7fbdeb400640 1 -- 172.21.15.107:0/4109952411 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) v1 ==== 137291+0+0 (secure 0 0 0) 0x7fbddc046020 con 0x7fbdf4108b00 2024-03-08T21:00:11.479 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.495+0000 7fbdfb817740 1 Processor -- start 2024-03-08T21:00:11.479 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.495+0000 7fbdfb817740 1 -- start start 2024-03-08T21:00:11.479 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.495+0000 7fbdfb817740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564d3399b590 0x564d3399b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:00:11.479 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.495+0000 7fbdfb817740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564d3399bf80 0x564d339a0a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:00:11.479 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.495+0000 7fbdfb817740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564d339a10f0 0x564d339a3580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:00:11.479 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.495+0000 7fbdfb817740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d339006f0 con 0x564d3399b590 2024-03-08T21:00:11.480 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.495+0000 7fbdfb817740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d338a99e0 con 0x564d339a10f0 2024-03-08T21:00:11.480 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.495+0000 7fbdfb817740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d338fc790 con 0x564d3399bf80 2024-03-08T21:00:11.480 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.496+0000 7fbdf2800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564d3399b590 0x564d3399b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:00:11.480 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.496+0000 7fbdf2800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564d3399b590 0x564d3399b9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:56804/0 (socket says 172.21.15.107:56804) 2024-03-08T21:00:11.480 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.496+0000 7fbdf2800640 1 -- 172.21.15.107:0/3725433679 learned_addr learned my addr 172.21.15.107:0/3725433679 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:00:11.480 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.496+0000 7fbdf1e00640 1 --2- 172.21.15.107:0/3725433679 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564d3399bf80 0x564d339a0a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:00:11.480 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.496+0000 7fbde9400640 1 --2- 172.21.15.107:0/3725433679 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564d339a10f0 0x564d339a3580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:00:11.480 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.496+0000 7fbdf2800640 1 -- 172.21.15.107:0/3725433679 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564d3399bf80 msgr2=0x564d339a0a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:00:11.480 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.496+0000 7fbdf2800640 1 --2- 172.21.15.107:0/3725433679 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564d3399bf80 0x564d339a0a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:11.480 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.496+0000 7fbdf2800640 1 -- 172.21.15.107:0/3725433679 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564d339a10f0 msgr2=0x564d339a3580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:00:11.480 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.496+0000 7fbdf2800640 1 --2- 172.21.15.107:0/3725433679 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564d339a10f0 0x564d339a3580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:11.480 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.496+0000 7fbdf2800640 1 -- 172.21.15.107:0/3725433679 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564d3389d720 con 0x564d3399b590 2024-03-08T21:00:11.480 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.496+0000 7fbde9400640 1 --2- 172.21.15.107:0/3725433679 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564d339a10f0 0x564d339a3580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T21:00:11.485 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.497+0000 7fbdf2800640 1 --2- 172.21.15.107:0/3725433679 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564d3399b590 0x564d3399b9b0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fbdec008da0 tx=0x7fbdec00d710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a981bc20e2c7ff9a server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:00:11.485 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.497+0000 7fbde3000640 1 -- 172.21.15.107:0/3725433679 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdec00f830 con 0x564d3399b590 2024-03-08T21:00:11.485 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.497+0000 7fbde3000640 1 -- 172.21.15.107:0/3725433679 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fbdec00de20 con 0x564d3399b590 2024-03-08T21:00:11.485 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.497+0000 7fbde3000640 1 -- 172.21.15.107:0/3725433679 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdec018df0 con 0x564d3399b590 2024-03-08T21:00:11.485 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.497+0000 7fbdeb400640 1 --2- 172.21.15.107:0/4109952411 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fbdd40a2800 0x7fbdd40a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:00:11.485 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.497+0000 7fbdeb400640 1 -- 172.21.15.107:0/4109952411 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fbddc0f1060 con 0x7fbdf4108b00 2024-03-08T21:00:11.486 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.497+0000 7fbdf1e00640 1 --2- 172.21.15.107:0/4109952411 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fbdd40a2800 0x7fbdd40a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:00:11.486 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.498+0000 7fbdfb817740 1 -- 172.21.15.107:0/3725433679 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564d3399b590 msgr2=0x564d3399b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:00:11.486 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.498+0000 7fbdfb817740 1 --2- 172.21.15.107:0/3725433679 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564d3399b590 0x564d3399b9b0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fbdec008da0 tx=0x7fbdec00d710 comp rx=0 tx=0).stop 2024-03-08T21:00:11.486 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.498+0000 7fbdfb817740 1 -- 172.21.15.107:0/3725433679 shutdown_connections 2024-03-08T21:00:11.486 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.498+0000 7fbdfb817740 1 --2- 172.21.15.107:0/3725433679 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564d339a10f0 0x564d339a3580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:11.486 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.498+0000 7fbdfb817740 1 --2- 172.21.15.107:0/3725433679 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564d3399bf80 0x564d339a0a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:11.486 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.498+0000 7fbdfb817740 1 --2- 172.21.15.107:0/3725433679 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564d3399b590 0x564d3399b9b0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:11.486 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.498+0000 7fbdfb817740 1 -- 172.21.15.107:0/3725433679 >> 172.21.15.107:0/3725433679 conn(0x564d33996b00 msgr2=0x564d33996f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:00:11.486 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.498+0000 7fbdf1e00640 1 --2- 172.21.15.107:0/4109952411 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fbdd40a2800 0x7fbdd40a4cc0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x564d3399c380 tx=0x7fbde400b590 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:00:11.486 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.498+0000 7fbdfb817740 1 -- 172.21.15.107:0/3725433679 shutdown_connections 2024-03-08T21:00:11.486 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.498+0000 7fbdfb817740 1 -- 172.21.15.107:0/3725433679 wait complete. 2024-03-08T21:00:11.486 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.498+0000 7fbdfb817740 1 -- start start 2024-03-08T21:00:11.487 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.499+0000 7fbdfb817740 1 Processor -- start 2024-03-08T21:00:11.487 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.499+0000 7fbdfb817740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564d3399b590 0x564d339a5830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:00:11.487 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.499+0000 7fbdfb817740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564d3399bf80 0x564d339a5d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:00:11.487 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.499+0000 7fbdfb817740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564d339a10f0 0x564d339acd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:00:11.487 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.499+0000 7fbdfb817740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d339a40a0 con 0x564d3399bf80 2024-03-08T21:00:11.487 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.499+0000 7fbdfb817740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d338bce20 con 0x564d3399b590 2024-03-08T21:00:11.487 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.499+0000 7fbdfb817740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d339a4220 con 0x564d339a10f0 2024-03-08T21:00:11.487 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.499+0000 7fbdfb817740 10 client.0 _ll_register_callbacks cb 0x564d338f78d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T21:00:11.487 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.499+0000 7fbde9400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564d3399bf80 0x564d339a5d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:00:11.487 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.499+0000 7fbde9400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564d3399bf80 0x564d339a5d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:56810/0 (socket says 172.21.15.107:56810) 2024-03-08T21:00:11.487 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.499+0000 7fbde9400640 1 -- 172.21.15.107:0/3776454107 learned_addr learned my addr 172.21.15.107:0/3776454107 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:00:11.487 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.500+0000 7fbdf1e00640 1 --2- 172.21.15.107:0/3776454107 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564d3399b590 0x564d339a5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:00:11.487 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.500+0000 7fbde9400640 1 -- 172.21.15.107:0/3776454107 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564d339a10f0 msgr2=0x564d339acd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:00:11.487 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.500+0000 7fbde9400640 1 --2- 172.21.15.107:0/3776454107 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564d339a10f0 0x564d339acd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:11.488 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.500+0000 7fbde9400640 1 -- 172.21.15.107:0/3776454107 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564d3399b590 msgr2=0x564d339a5830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:00:11.488 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.500+0000 7fbde9400640 1 --2- 172.21.15.107:0/3776454107 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564d3399b590 0x564d339a5830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:11.488 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.500+0000 7fbde9400640 1 -- 172.21.15.107:0/3776454107 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564d336152d0 con 0x564d3399bf80 2024-03-08T21:00:11.488 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.500+0000 7fbde9400640 1 --2- 172.21.15.107:0/3776454107 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564d3399bf80 0x564d339a5d70 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fbddc0b9f80 tx=0x7fbddc004c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef31d193a2a8536 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:00:11.488 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.500+0000 7fbddb400640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-03-08T21:00:11.488 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.500+0000 7fbddb400640 1 -- 172.21.15.107:0/3776454107 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbddc0d5020 con 0x564d3399bf80 2024-03-08T21:00:11.488 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.500+0000 7fbddb400640 1 -- 172.21.15.107:0/3776454107 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fbddc0f30b0 con 0x564d3399bf80 2024-03-08T21:00:11.488 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.500+0000 7fbdfb817740 1 -- 172.21.15.107:0/3776454107 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbdec00de20 con 0x564d3399bf80 2024-03-08T21:00:11.488 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.500+0000 7fbddb400640 1 -- 172.21.15.107:0/3776454107 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbddc0bdb20 con 0x564d3399bf80 2024-03-08T21:00:11.488 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.500+0000 7fbdfb817740 10 client.15309 fetch_fsmap learned FSMap version 27 2024-03-08T21:00:11.488 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.500+0000 7fbdfb817740 1 -- 172.21.15.107:0/3776454107 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x564d336df6b0 con 0x564d3399bf80 2024-03-08T21:00:11.488 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.501+0000 7fbddb400640 1 -- 172.21.15.107:0/3776454107 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fbdec00de20 con 0x564d3399bf80 2024-03-08T21:00:11.488 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.501+0000 7fbddb400640 1 -- 172.21.15.107:0/3776454107 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fbddc0c5e30 con 0x564d3399bf80 2024-03-08T21:00:11.489 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.501+0000 7fbdfb817740 10 client.15309 fetch_fsmap finished waiting for FSMap version 27 2024-03-08T21:00:11.489 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.501+0000 7fbdfb817740 10 client.15309 resolve_mds: resolved name 'h' to daemon mds.h 2024-03-08T21:00:11.489 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.501+0000 7fbdfb817740 20 client.15309 populate_metadata read hostname 'smithi107' 2024-03-08T21:00:11.489 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.501+0000 7fbdfb817740 1 --2- 172.21.15.107:0/3776454107 >> [v2:172.21.15.149:6836/914546237,v1:172.21.15.149:6837/914546237] conn(0x564d339b2320 0x564d339b11a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:00:11.489 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.501+0000 7fbdfb817740 4 client.15309 mds_command: new command op to 34445 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-08T21:00:11.489 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.501+0000 7fbdfb817740 1 -- 172.21.15.107:0/3776454107 --> [v2:172.21.15.149:6836/914546237,v1:172.21.15.149:6837/914546237] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564d3390dc30 con 0x564d339b2320 2024-03-08T21:00:11.489 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.502+0000 7fbdf1e00640 1 --2- 172.21.15.107:0/3776454107 >> [v2:172.21.15.149:6836/914546237,v1:172.21.15.149:6837/914546237] conn(0x564d339b2320 0x564d339b11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:00:11.490 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.502+0000 7fbdf1e00640 1 --2- 172.21.15.107:0/3776454107 >> [v2:172.21.15.149:6836/914546237,v1:172.21.15.149:6837/914546237] conn(0x564d339b2320 0x564d339b11a0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a5b1936843c58b36 server_cookie=5a83033bb48806c9 in_seq=0 out_seq=0 2024-03-08T21:00:11.490 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.502+0000 7fbddb400640 10 client.15309 ms_handle_connect on v2:172.21.15.149:6836/914546237 2024-03-08T21:00:11.491 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.504+0000 7fbddb400640 1 -- 172.21.15.107:0/3776454107 <== mds.1 v2:172.21.15.149:6836/914546237 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x564d3390dc30 con 0x564d339b2320 2024-03-08T21:00:11.491 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.504+0000 7fbddb400640 10 client.15309 handle_command_reply: tid=0 2024-03-08T21:00:11.491 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.504+0000 7fbdfb817740 1 client.15309 shutdown 2024-03-08T21:00:11.491 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.504+0000 7fbdfb817740 1 -- 172.21.15.107:0/3776454107 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564d3399bf80 msgr2=0x564d339a5d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:00:11.491 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.504+0000 7fbdfb817740 1 --2- 172.21.15.107:0/3776454107 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564d3399bf80 0x564d339a5d70 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fbddc0b9f80 tx=0x7fbddc004c50 comp rx=0 tx=0).stop 2024-03-08T21:00:11.491 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.504+0000 7fbdfb817740 1 -- 172.21.15.107:0/3776454107 shutdown_connections 2024-03-08T21:00:11.491 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.504+0000 7fbdfb817740 1 --2- 172.21.15.107:0/3776454107 >> [v2:172.21.15.149:6836/914546237,v1:172.21.15.149:6837/914546237] conn(0x564d339b2320 0x564d339b11a0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:11.491 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.504+0000 7fbdfb817740 1 --2- 172.21.15.107:0/3776454107 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564d339a10f0 0x564d339acd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:11.491 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.504+0000 7fbdfb817740 1 --2- 172.21.15.107:0/3776454107 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564d3399bf80 0x564d339a5d70 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:11.491 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.504+0000 7fbdfb817740 1 --2- 172.21.15.107:0/3776454107 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564d3399b590 0x564d339a5830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:11.491 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.504+0000 7fbddb400640 0 client.15309 ms_handle_reset on v2:172.21.15.149:6836/914546237 2024-03-08T21:00:11.491 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.504+0000 7fbdfb817740 1 -- 172.21.15.107:0/3776454107 >> 172.21.15.107:0/3776454107 conn(0x564d33996b00 msgr2=0x564d33995c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:00:11.491 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.505+0000 7fbdfb817740 1 -- 172.21.15.107:0/3776454107 shutdown_connections 2024-03-08T21:00:11.491 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.505+0000 7fbdfb817740 1 -- 172.21.15.107:0/3776454107 wait complete. 2024-03-08T21:00:11.491 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.505+0000 7fbdfb817740 20 client.15309 trim_cache size 0 max 16384 2024-03-08T21:00:11.505 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.522+0000 7fbdfb817740 1 Processor -- start 2024-03-08T21:00:11.505 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.522+0000 7fbdfb817740 1 -- start start 2024-03-08T21:00:11.505 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.522+0000 7fbdfb817740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564d3399b590 0x564d339a07d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:00:11.505 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.522+0000 7fbdfb817740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564d3399bf80 0x564d339acd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:00:11.505 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.522+0000 7fbdfb817740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564d339a10f0 0x564d339ba7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:00:11.505 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.522+0000 7fbdfb817740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d339b1b40 con 0x564d339a10f0 2024-03-08T21:00:11.505 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.522+0000 7fbdfb817740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d339b21a0 con 0x564d3399bf80 2024-03-08T21:00:11.506 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.522+0000 7fbdfb817740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d339b2800 con 0x564d3399b590 2024-03-08T21:00:11.506 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.522+0000 7fbdf2800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564d339a10f0 0x564d339ba7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:00:11.506 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.522+0000 7fbdf2800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564d339a10f0 0x564d339ba7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:56820/0 (socket says 172.21.15.107:56820) 2024-03-08T21:00:11.506 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.522+0000 7fbdf2800640 1 -- 172.21.15.107:0/16483062 learned_addr learned my addr 172.21.15.107:0/16483062 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:00:11.507 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.522+0000 7fbdf2800640 1 -- 172.21.15.107:0/16483062 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564d3399b590 msgr2=0x564d339a07d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:00:11.507 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.522+0000 7fbdf1e00640 1 --2- 172.21.15.107:0/16483062 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564d3399b590 0x564d339a07d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:00:11.507 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.523+0000 7fbdf2800640 1 --2- 172.21.15.107:0/16483062 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564d3399b590 0x564d339a07d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:11.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.523+0000 7fbde9400640 1 --2- 172.21.15.107:0/16483062 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564d3399bf80 0x564d339acd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:00:11.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.523+0000 7fbdf2800640 1 -- 172.21.15.107:0/16483062 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564d3399bf80 msgr2=0x564d339acd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:00:11.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.523+0000 7fbdf2800640 1 --2- 172.21.15.107:0/16483062 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564d3399bf80 0x564d339acd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:11.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.523+0000 7fbdf2800640 1 -- 172.21.15.107:0/16483062 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564d339a0450 con 0x564d339a10f0 2024-03-08T21:00:11.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.523+0000 7fbdf2800640 1 --2- 172.21.15.107:0/16483062 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564d339a10f0 0x564d339ba7c0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fbdec0084f0 tx=0x7fbdec00dfb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18df3de96d39d10 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:00:11.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.523+0000 7fbde2600640 1 -- 172.21.15.107:0/16483062 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdec00f630 con 0x564d339a10f0 2024-03-08T21:00:11.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.523+0000 7fbde2600640 1 -- 172.21.15.107:0/16483062 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fbdec027070 con 0x564d339a10f0 2024-03-08T21:00:11.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.523+0000 7fbde2600640 1 -- 172.21.15.107:0/16483062 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdec018c90 con 0x564d339a10f0 2024-03-08T21:00:11.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.524+0000 7fbdfb817740 1 -- 172.21.15.107:0/16483062 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564d339a10f0 msgr2=0x564d339ba7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:00:11.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.524+0000 7fbdfb817740 1 --2- 172.21.15.107:0/16483062 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564d339a10f0 0x564d339ba7c0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fbdec0084f0 tx=0x7fbdec00dfb0 comp rx=0 tx=0).stop 2024-03-08T21:00:11.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.524+0000 7fbdfb817740 1 -- 172.21.15.107:0/16483062 shutdown_connections 2024-03-08T21:00:11.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.524+0000 7fbdfb817740 1 --2- 172.21.15.107:0/16483062 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564d339a10f0 0x564d339ba7c0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:11.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.524+0000 7fbdfb817740 1 --2- 172.21.15.107:0/16483062 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564d3399bf80 0x564d339acd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:11.509 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.524+0000 7fbdfb817740 1 --2- 172.21.15.107:0/16483062 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564d3399b590 0x564d339a07d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:11.509 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.524+0000 7fbdfb817740 1 -- 172.21.15.107:0/16483062 >> 172.21.15.107:0/16483062 conn(0x564d33996b00 msgr2=0x564d33996ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:00:11.509 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.524+0000 7fbdfb817740 1 -- 172.21.15.107:0/16483062 shutdown_connections 2024-03-08T21:00:11.509 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.524+0000 7fbdfb817740 1 -- 172.21.15.107:0/16483062 wait complete. 2024-03-08T21:00:11.509 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.524+0000 7fbdfb817740 1 -- start start 2024-03-08T21:00:11.509 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.525+0000 7fbdfb817740 1 Processor -- start 2024-03-08T21:00:11.512 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.525+0000 7fbdfb817740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564d3399b590 0x564d339a99b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:00:11.512 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.525+0000 7fbdfb817740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564d3399bf80 0x564d339a9ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:00:11.512 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.525+0000 7fbdfb817740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564d339a10f0 0x564d33a07f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:00:11.512 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.525+0000 7fbdfb817740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d339b34c0 con 0x564d3399bf80 2024-03-08T21:00:11.512 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.525+0000 7fbdfb817740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d339b2e60 con 0x564d3399b590 2024-03-08T21:00:11.512 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.525+0000 7fbdfb817740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d339b3b20 con 0x564d339a10f0 2024-03-08T21:00:11.512 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.525+0000 7fbdfb817740 10 client.0 _ll_register_callbacks cb 0x7fbdec008530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T21:00:11.513 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.525+0000 7fbde9400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564d3399bf80 0x564d339a9ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:00:11.513 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.525+0000 7fbde9400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564d3399bf80 0x564d339a9ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:56836/0 (socket says 172.21.15.107:56836) 2024-03-08T21:00:11.513 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.525+0000 7fbde9400640 1 -- 172.21.15.107:0/1754961191 learned_addr learned my addr 172.21.15.107:0/1754961191 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:00:11.513 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.525+0000 7fbde9400640 1 -- 172.21.15.107:0/1754961191 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564d339a10f0 msgr2=0x564d33a07f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:00:11.513 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.525+0000 7fbde9400640 1 --2- 172.21.15.107:0/1754961191 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564d339a10f0 0x564d33a07f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:11.513 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.525+0000 7fbde9400640 1 -- 172.21.15.107:0/1754961191 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564d3399b590 msgr2=0x564d339a99b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:00:11.513 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.525+0000 7fbde9400640 1 --2- 172.21.15.107:0/1754961191 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564d3399b590 0x564d339a99b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:11.513 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.525+0000 7fbde9400640 1 -- 172.21.15.107:0/1754961191 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564d339acfc0 con 0x564d3399bf80 2024-03-08T21:00:11.513 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.526+0000 7fbde9400640 1 --2- 172.21.15.107:0/1754961191 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564d3399bf80 0x564d339a9ef0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fbddc0bdab0 tx=0x7fbddc0bd240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30956e3b029ac330 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:00:11.513 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.526+0000 7fbddb400640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-03-08T21:00:11.513 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.526+0000 7fbddb400640 1 -- 172.21.15.107:0/1754961191 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbddc0d5070 con 0x564d3399bf80 2024-03-08T21:00:11.513 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.526+0000 7fbddb400640 1 -- 172.21.15.107:0/1754961191 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fbddc0d6be0 con 0x564d3399bf80 2024-03-08T21:00:11.514 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.526+0000 7fbdfb817740 1 -- 172.21.15.107:0/1754961191 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbdec027070 con 0x564d3399bf80 2024-03-08T21:00:11.514 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.526+0000 7fbddb400640 1 -- 172.21.15.107:0/1754961191 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbddc0c6040 con 0x564d3399bf80 2024-03-08T21:00:11.514 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.526+0000 7fbdfb817740 10 client.15315 fetch_fsmap learned FSMap version 27 2024-03-08T21:00:11.514 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.526+0000 7fbdfb817740 1 -- 172.21.15.107:0/1754961191 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x564d33a09510 con 0x564d3399bf80 2024-03-08T21:00:11.514 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.526+0000 7fbddb400640 1 -- 172.21.15.107:0/1754961191 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fbdec027070 con 0x564d3399bf80 2024-03-08T21:00:11.514 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.526+0000 7fbddb400640 1 -- 172.21.15.107:0/1754961191 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fbddc0b9870 con 0x564d3399bf80 2024-03-08T21:00:11.514 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.527+0000 7fbdfb817740 10 client.15315 fetch_fsmap finished waiting for FSMap version 27 2024-03-08T21:00:11.514 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.527+0000 7fbdfb817740 10 client.15315 resolve_mds: resolved name 'h' to daemon mds.h 2024-03-08T21:00:11.514 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.527+0000 7fbdfb817740 20 client.15315 populate_metadata read hostname 'smithi107' 2024-03-08T21:00:11.514 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.527+0000 7fbdfb817740 1 --2- 172.21.15.107:0/1754961191 >> [v2:172.21.15.149:6836/914546237,v1:172.21.15.149:6837/914546237] conn(0x564d33a0dac0 0x564d33a09800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:00:11.514 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.527+0000 7fbdfb817740 4 client.15315 mds_command: new command op to 34445 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-08T21:00:11.514 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.527+0000 7fbdfb817740 1 -- 172.21.15.107:0/1754961191 --> [v2:172.21.15.149:6836/914546237,v1:172.21.15.149:6837/914546237] -- command(tid 0: {"prefix": "status"}) v1 -- 0x564d33907f80 con 0x564d33a0dac0 2024-03-08T21:00:11.514 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.527+0000 7fbdf1e00640 1 --2- 172.21.15.107:0/1754961191 >> [v2:172.21.15.149:6836/914546237,v1:172.21.15.149:6837/914546237] conn(0x564d33a0dac0 0x564d33a09800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:00:11.515 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.528+0000 7fbdf1e00640 1 --2- 172.21.15.107:0/1754961191 >> [v2:172.21.15.149:6836/914546237,v1:172.21.15.149:6837/914546237] conn(0x564d33a0dac0 0x564d33a09800 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=21f978bcfae643c8 server_cookie=14f18b84a83778ef in_seq=0 out_seq=0 2024-03-08T21:00:11.515 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.528+0000 7fbddb400640 10 client.15315 ms_handle_connect on v2:172.21.15.149:6836/914546237 2024-03-08T21:00:11.515 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.529+0000 7fbddb400640 1 -- 172.21.15.107:0/1754961191 <== mds.1 v2:172.21.15.149:6836/914546237 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x564d33907f80 con 0x564d33a0dac0 2024-03-08T21:00:11.515 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.529+0000 7fbddb400640 10 client.15315 handle_command_reply: tid=0 2024-03-08T21:00:11.515 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.529+0000 7fbdfb817740 1 client.15315 shutdown 2024-03-08T21:00:11.515 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.529+0000 7fbdfb817740 1 -- 172.21.15.107:0/1754961191 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564d3399bf80 msgr2=0x564d339a9ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:00:11.515 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.529+0000 7fbdfb817740 1 --2- 172.21.15.107:0/1754961191 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564d3399bf80 0x564d339a9ef0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fbddc0bdab0 tx=0x7fbddc0bd240 comp rx=0 tx=0).stop 2024-03-08T21:00:11.515 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.529+0000 7fbdfb817740 1 -- 172.21.15.107:0/1754961191 shutdown_connections 2024-03-08T21:00:11.515 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.529+0000 7fbdfb817740 1 --2- 172.21.15.107:0/1754961191 >> [v2:172.21.15.149:6836/914546237,v1:172.21.15.149:6837/914546237] conn(0x564d33a0dac0 0x564d33a09800 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:11.515 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.529+0000 7fbdfb817740 1 --2- 172.21.15.107:0/1754961191 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564d339a10f0 0x564d33a07f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:11.515 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.529+0000 7fbdfb817740 1 --2- 172.21.15.107:0/1754961191 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564d3399bf80 0x564d339a9ef0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:11.515 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.529+0000 7fbdfb817740 1 --2- 172.21.15.107:0/1754961191 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564d3399b590 0x564d339a99b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:11.515 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.529+0000 7fbddb400640 0 client.15315 ms_handle_reset on v2:172.21.15.149:6836/914546237 2024-03-08T21:00:11.515 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.529+0000 7fbdfb817740 1 -- 172.21.15.107:0/1754961191 >> 172.21.15.107:0/1754961191 conn(0x564d33996b00 msgr2=0x564d33996ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:00:11.515 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.529+0000 7fbdfb817740 1 -- 172.21.15.107:0/1754961191 shutdown_connections 2024-03-08T21:00:11.516 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.529+0000 7fbdfb817740 1 -- 172.21.15.107:0/1754961191 wait complete. 2024-03-08T21:00:11.516 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.529+0000 7fbdfb817740 20 client.15315 trim_cache size 0 max 16384 2024-03-08T21:00:11.516 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-03-08T21:00:11.516 INFO:teuthology.orchestra.run.smithi107.stdout: "cluster_fsid": "dc20f0f8-dd8c-11ee-95c6-87774f69a715", 2024-03-08T21:00:11.516 INFO:teuthology.orchestra.run.smithi107.stdout: "whoami": 1, 2024-03-08T21:00:11.516 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 34445, 2024-03-08T21:00:11.516 INFO:teuthology.orchestra.run.smithi107.stdout: "want_state": "up:active", 2024-03-08T21:00:11.516 INFO:teuthology.orchestra.run.smithi107.stdout: "state": "up:active", 2024-03-08T21:00:11.516 INFO:teuthology.orchestra.run.smithi107.stdout: "fs_name": "cephfs", 2024-03-08T21:00:11.517 INFO:teuthology.orchestra.run.smithi107.stdout: "rank_uptime": 137.71176576799999, 2024-03-08T21:00:11.517 INFO:teuthology.orchestra.run.smithi107.stdout: "mdsmap_epoch": 27, 2024-03-08T21:00:11.517 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_epoch": 88, 2024-03-08T21:00:11.517 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_epoch_barrier": 86, 2024-03-08T21:00:11.517 INFO:teuthology.orchestra.run.smithi107.stdout: "uptime": 163.245127782 2024-03-08T21:00:11.517 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-03-08T21:00:11.518 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.530+0000 7fbdf8c00640 1 -- 172.21.15.107:0/4109952411 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fbdd40a2800 msgr2=0x7fbdd40a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:00:11.518 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.530+0000 7fbdf8c00640 1 --2- 172.21.15.107:0/4109952411 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fbdd40a2800 0x7fbdd40a4cc0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x564d3399c380 tx=0x7fbde400b590 comp rx=0 tx=0).stop 2024-03-08T21:00:11.518 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.530+0000 7fbdf8c00640 1 -- 172.21.15.107:0/4109952411 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbdf4108b00 msgr2=0x7fbdf419e070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:00:11.518 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.530+0000 7fbdf8c00640 1 --2- 172.21.15.107:0/4109952411 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbdf4108b00 0x7fbdf419e070 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fbddc000c00 tx=0x7fbddc0042f0 comp rx=0 tx=0).stop 2024-03-08T21:00:11.518 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.530+0000 7fbdf8c00640 1 -- 172.21.15.107:0/4109952411 shutdown_connections 2024-03-08T21:00:11.518 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.530+0000 7fbdf8c00640 1 --2- 172.21.15.107:0/4109952411 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fbdd40a2800 0x7fbdd40a4cc0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:11.518 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.530+0000 7fbdf8c00640 1 --2- 172.21.15.107:0/4109952411 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbdf41094c0 0x7fbdf41a50e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:11.518 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.530+0000 7fbdf8c00640 1 --2- 172.21.15.107:0/4109952411 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbdf4108b00 0x7fbdf419e070 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:11.518 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.530+0000 7fbdf8c00640 1 --2- 172.21.15.107:0/4109952411 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbdf4107910 0x7fbdf419db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:11.518 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.530+0000 7fbdf8c00640 1 -- 172.21.15.107:0/4109952411 >> 172.21.15.107:0/4109952411 conn(0x7fbdf40768b0 msgr2=0x7fbdf410be00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:00:11.518 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.530+0000 7fbdf8c00640 1 -- 172.21.15.107:0/4109952411 shutdown_connections 2024-03-08T21:00:11.519 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.530+0000 7fbdf8c00640 1 -- 172.21.15.107:0/4109952411 wait complete. 2024-03-08T21:00:11.530 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-03-08T21:00:11.735 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.751+0000 7fea57200640 1 -- 172.21.15.107:0/660235925 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fea581093c0 msgr2=0x7fea5810fc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:00:11.735 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.751+0000 7fea57200640 1 --2- 172.21.15.107:0/660235925 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fea581093c0 0x7fea5810fc50 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fea50009900 tx=0x7fea5002bc30 comp rx=0 tx=0).stop 2024-03-08T21:00:11.735 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.751+0000 7fea57200640 1 -- 172.21.15.107:0/660235925 shutdown_connections 2024-03-08T21:00:11.735 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.751+0000 7fea57200640 1 --2- 172.21.15.107:0/660235925 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fea581093c0 0x7fea5810fc50 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:11.735 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.751+0000 7fea57200640 1 --2- 172.21.15.107:0/660235925 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fea58108a00 0x7fea58108e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:11.735 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.751+0000 7fea57200640 1 --2- 172.21.15.107:0/660235925 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fea58107810 0x7fea58107c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:11.735 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.751+0000 7fea57200640 1 -- 172.21.15.107:0/660235925 >> 172.21.15.107:0/660235925 conn(0x7fea580767c0 msgr2=0x7fea58078be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:00:11.735 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.751+0000 7fea57200640 1 -- 172.21.15.107:0/660235925 shutdown_connections 2024-03-08T21:00:11.735 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.751+0000 7fea57200640 1 -- 172.21.15.107:0/660235925 wait complete. 2024-03-08T21:00:11.738 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.752+0000 7fea57200640 1 Processor -- start 2024-03-08T21:00:11.738 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.752+0000 7fea57200640 1 -- start start 2024-03-08T21:00:11.738 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.753+0000 7fea57200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fea58107810 0x7fea5819d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:00:11.738 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.753+0000 7fea57200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fea58108a00 0x7fea5819ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:00:11.738 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.753+0000 7fea57200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fea581093c0 0x7fea581a4f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:00:11.738 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.753+0000 7fea57200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea58113370 con 0x7fea58107810 2024-03-08T21:00:11.738 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.753+0000 7fea57200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea581131f0 con 0x7fea581093c0 2024-03-08T21:00:11.738 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.753+0000 7fea57200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea581134f0 con 0x7fea58108a00 2024-03-08T21:00:11.738 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.753+0000 7fea55e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fea58107810 0x7fea5819d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:00:11.738 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.753+0000 7fea55e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fea58107810 0x7fea5819d990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:56864/0 (socket says 172.21.15.107:56864) 2024-03-08T21:00:11.739 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.753+0000 7fea55e00640 1 -- 172.21.15.107:0/549511609 learned_addr learned my addr 172.21.15.107:0/549511609 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:00:11.739 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.753+0000 7fea55e00640 1 -- 172.21.15.107:0/549511609 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fea58108a00 msgr2=0x7fea5819ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:00:11.739 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.753+0000 7fea4d400640 1 --2- 172.21.15.107:0/549511609 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fea58108a00 0x7fea5819ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:00:11.739 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.753+0000 7fea55e00640 1 --2- 172.21.15.107:0/549511609 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fea58108a00 0x7fea5819ded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:11.739 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.753+0000 7fea55e00640 1 -- 172.21.15.107:0/549511609 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fea581093c0 msgr2=0x7fea581a4f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:00:11.739 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.753+0000 7fea55e00640 1 --2- 172.21.15.107:0/549511609 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fea581093c0 0x7fea581a4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:11.739 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.753+0000 7fea55e00640 1 -- 172.21.15.107:0/549511609 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea581a55b0 con 0x7fea58107810 2024-03-08T21:00:11.739 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.754+0000 7fea55e00640 1 --2- 172.21.15.107:0/549511609 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fea58107810 0x7fea5819d990 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fea4800ccb0 tx=0x7fea4800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:00:11.739 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.754+0000 7fea4f400640 1 -- 172.21.15.107:0/549511609 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea48003b70 con 0x7fea58107810 2024-03-08T21:00:11.739 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.754+0000 7fea4f400640 1 -- 172.21.15.107:0/549511609 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fea4800ae50 con 0x7fea58107810 2024-03-08T21:00:11.739 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.754+0000 7fea4f400640 1 -- 172.21.15.107:0/549511609 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea4800f6f0 con 0x7fea58107810 2024-03-08T21:00:11.739 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.754+0000 7fea57200640 1 -- 172.21.15.107:0/549511609 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea581a58a0 con 0x7fea58107810 2024-03-08T21:00:11.739 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.754+0000 7fea57200640 1 -- 172.21.15.107:0/549511609 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea581a5da0 con 0x7fea58107810 2024-03-08T21:00:11.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.757+0000 7fea4f400640 1 -- 172.21.15.107:0/549511609 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) v1 ==== 137291+0+0 (secure 0 0 0) 0x7fea48004000 con 0x7fea58107810 2024-03-08T21:00:11.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.757+0000 7fea5ffbe740 1 Processor -- start 2024-03-08T21:00:11.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.757+0000 7fea5ffbe740 1 -- start start 2024-03-08T21:00:11.742 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.758+0000 7fea5ffbe740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55701c682760 0x55701c682b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:00:11.742 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.758+0000 7fea5ffbe740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55701c683150 0x55701c687c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:00:11.742 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.758+0000 7fea5ffbe740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55701c6882c0 0x55701c68a750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:00:11.742 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.758+0000 7fea5ffbe740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55701c5e76f0 con 0x55701c683150 2024-03-08T21:00:11.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.758+0000 7fea5ffbe740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55701c5909e0 con 0x55701c682760 2024-03-08T21:00:11.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.758+0000 7fea5ffbe740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55701c5e3790 con 0x55701c6882c0 2024-03-08T21:00:11.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.758+0000 7fea55e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55701c683150 0x55701c687c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:00:11.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.758+0000 7fea55e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55701c683150 0x55701c687c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:56866/0 (socket says 172.21.15.107:56866) 2024-03-08T21:00:11.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.758+0000 7fea55e00640 1 -- 172.21.15.107:0/1212842480 learned_addr learned my addr 172.21.15.107:0/1212842480 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:00:11.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.758+0000 7fea56800640 1 --2- 172.21.15.107:0/1212842480 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55701c682760 0x55701c682b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:00:11.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.758+0000 7fea4d400640 1 --2- 172.21.15.107:0/1212842480 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55701c6882c0 0x55701c68a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:00:11.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.758+0000 7fea55e00640 1 -- 172.21.15.107:0/1212842480 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55701c6882c0 msgr2=0x55701c68a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:00:11.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.758+0000 7fea55e00640 1 --2- 172.21.15.107:0/1212842480 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55701c6882c0 0x55701c68a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:11.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.758+0000 7fea55e00640 1 -- 172.21.15.107:0/1212842480 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55701c682760 msgr2=0x55701c682b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:00:11.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.758+0000 7fea55e00640 1 --2- 172.21.15.107:0/1212842480 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55701c682760 0x55701c682b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:11.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.758+0000 7fea55e00640 1 -- 172.21.15.107:0/1212842480 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55701c584720 con 0x55701c683150 2024-03-08T21:00:11.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.759+0000 7fea4f400640 1 --2- 172.21.15.107:0/549511609 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fea3c0a2880 0x7fea3c0a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:00:11.744 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.759+0000 7fea55e00640 1 --2- 172.21.15.107:0/1212842480 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55701c683150 0x55701c687c50 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fea480d2ed0 tx=0x7fea480d7340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6396ea6c0396b59e server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:00:11.744 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.759+0000 7fea56800640 1 --2- 172.21.15.107:0/549511609 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fea3c0a2880 0x7fea3c0a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:00:11.744 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.759+0000 7fea4f400640 1 -- 172.21.15.107:0/549511609 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fea480cecc0 con 0x7fea58107810 2024-03-08T21:00:11.744 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.759+0000 7fea47400640 1 -- 172.21.15.107:0/1212842480 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea480978c0 con 0x55701c683150 2024-03-08T21:00:11.744 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.759+0000 7fea47400640 1 -- 172.21.15.107:0/1212842480 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fea480525e0 con 0x55701c683150 2024-03-08T21:00:11.744 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.759+0000 7fea47400640 1 -- 172.21.15.107:0/1212842480 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea480ab520 con 0x55701c683150 2024-03-08T21:00:11.749 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.760+0000 7fea5ffbe740 1 -- 172.21.15.107:0/1212842480 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55701c683150 msgr2=0x55701c687c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:00:11.749 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.760+0000 7fea56800640 1 --2- 172.21.15.107:0/549511609 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fea3c0a2880 0x7fea3c0a4d40 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x55701c6839b0 tx=0x7fea5002c740 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:00:11.749 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.760+0000 7fea5ffbe740 1 --2- 172.21.15.107:0/1212842480 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55701c683150 0x55701c687c50 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fea480d2ed0 tx=0x7fea480d7340 comp rx=0 tx=0).stop 2024-03-08T21:00:11.749 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.760+0000 7fea5ffbe740 1 -- 172.21.15.107:0/1212842480 shutdown_connections 2024-03-08T21:00:11.749 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.760+0000 7fea5ffbe740 1 --2- 172.21.15.107:0/1212842480 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55701c6882c0 0x55701c68a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:11.749 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.760+0000 7fea5ffbe740 1 --2- 172.21.15.107:0/1212842480 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55701c683150 0x55701c687c50 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:11.749 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.760+0000 7fea5ffbe740 1 --2- 172.21.15.107:0/1212842480 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55701c682760 0x55701c682b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:11.749 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.760+0000 7fea5ffbe740 1 -- 172.21.15.107:0/1212842480 >> 172.21.15.107:0/1212842480 conn(0x55701c67dcd0 msgr2=0x55701c67e0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:00:11.749 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.760+0000 7fea5ffbe740 1 -- 172.21.15.107:0/1212842480 shutdown_connections 2024-03-08T21:00:11.750 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.760+0000 7fea5ffbe740 1 -- 172.21.15.107:0/1212842480 wait complete. 2024-03-08T21:00:11.750 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.761+0000 7fea5ffbe740 1 -- start start 2024-03-08T21:00:11.750 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.761+0000 7fea5ffbe740 1 Processor -- start 2024-03-08T21:00:11.750 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.761+0000 7fea5ffbe740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55701c682760 0x55701c68c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:00:11.750 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.761+0000 7fea5ffbe740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55701c683150 0x55701c68cf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:00:11.750 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.761+0000 7fea5ffbe740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55701c6882c0 0x55701c693f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:00:11.750 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.761+0000 7fea5ffbe740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55701c68b270 con 0x55701c683150 2024-03-08T21:00:11.750 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.761+0000 7fea5ffbe740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55701c5a3e20 con 0x55701c6882c0 2024-03-08T21:00:11.750 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.761+0000 7fea5ffbe740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55701c68b3f0 con 0x55701c682760 2024-03-08T21:00:11.750 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.761+0000 7fea5ffbe740 10 client.0 _ll_register_callbacks cb 0x55701c5de8d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T21:00:11.750 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.761+0000 7fea4d400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55701c683150 0x55701c68cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:00:11.750 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.762+0000 7fea4d400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55701c683150 0x55701c68cf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:56872/0 (socket says 172.21.15.107:56872) 2024-03-08T21:00:11.751 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.762+0000 7fea4d400640 1 -- 172.21.15.107:0/3049834006 learned_addr learned my addr 172.21.15.107:0/3049834006 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:00:11.751 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.762+0000 7fea55e00640 1 --2- 172.21.15.107:0/3049834006 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55701c682760 0x55701c68c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:00:11.751 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.762+0000 7fea4d400640 1 -- 172.21.15.107:0/3049834006 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55701c682760 msgr2=0x55701c68c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:00:11.751 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.762+0000 7fea4d400640 1 --2- 172.21.15.107:0/3049834006 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55701c682760 0x55701c68c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:11.751 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.762+0000 7fea4d400640 1 -- 172.21.15.107:0/3049834006 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55701c6882c0 msgr2=0x55701c693f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:00:11.751 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.762+0000 7fea56800640 1 --2- 172.21.15.107:0/3049834006 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55701c6882c0 0x55701c693f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:00:11.751 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.762+0000 7fea4d400640 1 --2- 172.21.15.107:0/3049834006 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55701c6882c0 0x55701c693f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:11.751 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.762+0000 7fea4d400640 1 -- 172.21.15.107:0/3049834006 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55701c2fc2d0 con 0x55701c683150 2024-03-08T21:00:11.751 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.762+0000 7fea4d400640 1 --2- 172.21.15.107:0/3049834006 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55701c683150 0x55701c68cf10 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fea4000f6a0 tx=0x7fea4000fb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcb02f14b30829cf server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:00:11.752 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.762+0000 7fea3be00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-03-08T21:00:11.752 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.762+0000 7fea3be00640 1 -- 172.21.15.107:0/3049834006 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea40016070 con 0x55701c683150 2024-03-08T21:00:11.752 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.762+0000 7fea3be00640 1 -- 172.21.15.107:0/3049834006 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fea4000baa0 con 0x55701c683150 2024-03-08T21:00:11.752 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.763+0000 7fea5ffbe740 1 -- 172.21.15.107:0/3049834006 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fea480525e0 con 0x55701c683150 2024-03-08T21:00:11.752 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.763+0000 7fea3be00640 1 -- 172.21.15.107:0/3049834006 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fea4000bca0 con 0x55701c683150 2024-03-08T21:00:11.752 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.763+0000 7fea5ffbe740 10 client.15327 fetch_fsmap learned FSMap version 27 2024-03-08T21:00:11.752 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.763+0000 7fea5ffbe740 1 -- 172.21.15.107:0/3049834006 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55701c3c66b0 con 0x55701c683150 2024-03-08T21:00:11.752 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.763+0000 7fea3be00640 1 -- 172.21.15.107:0/3049834006 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fea480525e0 con 0x55701c683150 2024-03-08T21:00:11.752 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.763+0000 7fea3be00640 1 -- 172.21.15.107:0/3049834006 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fea40004c20 con 0x55701c683150 2024-03-08T21:00:11.752 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.763+0000 7fea5ffbe740 10 client.15327 fetch_fsmap finished waiting for FSMap version 27 2024-03-08T21:00:11.752 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.763+0000 7fea5ffbe740 10 client.15327 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-08T21:00:11.753 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.763+0000 7fea5ffbe740 20 client.15327 populate_metadata read hostname 'smithi107' 2024-03-08T21:00:11.753 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.763+0000 7fea5ffbe740 1 --2- 172.21.15.107:0/3049834006 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55701c6995e0 0x55701c698630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:00:11.753 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.763+0000 7fea5ffbe740 4 client.15327 mds_command: new command op to 34475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-08T21:00:11.753 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.763+0000 7fea5ffbe740 1 -- 172.21.15.107:0/3049834006 --> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55701c5f4c30 con 0x55701c6995e0 2024-03-08T21:00:11.753 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.764+0000 7fea55e00640 1 --2- 172.21.15.107:0/3049834006 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55701c6995e0 0x55701c698630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:00:11.753 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.765+0000 7fea55e00640 1 --2- 172.21.15.107:0/3049834006 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55701c6995e0 0x55701c698630 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a0ac8db7be688f8b server_cookie=605445e678229189 in_seq=0 out_seq=0 2024-03-08T21:00:11.753 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.765+0000 7fea3be00640 10 client.15327 ms_handle_connect on v2:172.21.15.149:6838/55290596 2024-03-08T21:00:11.754 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.767+0000 7fea3be00640 1 -- 172.21.15.107:0/3049834006 <== mds.0 v2:172.21.15.149:6838/55290596 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55701c5f4c30 con 0x55701c6995e0 2024-03-08T21:00:11.755 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.767+0000 7fea3be00640 10 client.15327 handle_command_reply: tid=0 2024-03-08T21:00:11.755 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.767+0000 7fea5ffbe740 1 client.15327 shutdown 2024-03-08T21:00:11.755 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.767+0000 7fea5ffbe740 1 -- 172.21.15.107:0/3049834006 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55701c683150 msgr2=0x55701c68cf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:00:11.755 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.767+0000 7fea5ffbe740 1 --2- 172.21.15.107:0/3049834006 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55701c683150 0x55701c68cf10 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fea4000f6a0 tx=0x7fea4000fb60 comp rx=0 tx=0).stop 2024-03-08T21:00:11.755 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.767+0000 7fea5ffbe740 1 -- 172.21.15.107:0/3049834006 shutdown_connections 2024-03-08T21:00:11.755 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.767+0000 7fea5ffbe740 1 --2- 172.21.15.107:0/3049834006 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55701c6995e0 0x55701c698630 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:11.755 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.767+0000 7fea5ffbe740 1 --2- 172.21.15.107:0/3049834006 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55701c6882c0 0x55701c693f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:11.755 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.767+0000 7fea5ffbe740 1 --2- 172.21.15.107:0/3049834006 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55701c683150 0x55701c68cf10 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:11.755 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.767+0000 7fea5ffbe740 1 --2- 172.21.15.107:0/3049834006 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55701c682760 0x55701c68c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:11.756 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.767+0000 7fea3be00640 0 client.15327 ms_handle_reset on v2:172.21.15.149:6838/55290596 2024-03-08T21:00:11.756 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.767+0000 7fea5ffbe740 1 -- 172.21.15.107:0/3049834006 >> 172.21.15.107:0/3049834006 conn(0x55701c67dcd0 msgr2=0x55701c67cee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:00:11.756 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.767+0000 7fea5ffbe740 1 -- 172.21.15.107:0/3049834006 shutdown_connections 2024-03-08T21:00:11.756 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.767+0000 7fea5ffbe740 1 -- 172.21.15.107:0/3049834006 wait complete. 2024-03-08T21:00:11.756 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.767+0000 7fea5ffbe740 20 client.15327 trim_cache size 0 max 16384 2024-03-08T21:00:11.768 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.784+0000 7fea5ffbe740 1 Processor -- start 2024-03-08T21:00:11.769 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.784+0000 7fea5ffbe740 1 -- start start 2024-03-08T21:00:11.769 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.785+0000 7fea5ffbe740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55701c682760 0x55701c67e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:00:11.769 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.785+0000 7fea5ffbe740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55701c683150 0x55701c693f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:00:11.769 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.785+0000 7fea5ffbe740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55701c6882c0 0x55701c68bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:00:11.769 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.785+0000 7fea5ffbe740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55701c698e00 con 0x55701c683150 2024-03-08T21:00:11.769 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.785+0000 7fea5ffbe740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55701c699460 con 0x55701c682760 2024-03-08T21:00:11.769 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.785+0000 7fea5ffbe740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55701c699ac0 con 0x55701c6882c0 2024-03-08T21:00:11.769 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.785+0000 7fea4d400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55701c683150 0x55701c693f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:00:11.769 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.785+0000 7fea4d400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55701c683150 0x55701c693f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:56874/0 (socket says 172.21.15.107:56874) 2024-03-08T21:00:11.769 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.785+0000 7fea4d400640 1 -- 172.21.15.107:0/930591330 learned_addr learned my addr 172.21.15.107:0/930591330 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:00:11.769 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.785+0000 7fea4d400640 1 -- 172.21.15.107:0/930591330 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55701c6882c0 msgr2=0x55701c68bcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:00:11.769 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.785+0000 7fea4d400640 1 --2- 172.21.15.107:0/930591330 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55701c6882c0 0x55701c68bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:11.770 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.785+0000 7fea4d400640 1 -- 172.21.15.107:0/930591330 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55701c682760 msgr2=0x55701c67e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:00:11.770 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.785+0000 7fea55e00640 1 --2- 172.21.15.107:0/930591330 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55701c682760 0x55701c67e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:00:11.770 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.785+0000 7fea4d400640 1 --2- 172.21.15.107:0/930591330 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55701c682760 0x55701c67e0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:11.770 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.785+0000 7fea4d400640 1 -- 172.21.15.107:0/930591330 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55701c6839e0 con 0x55701c683150 2024-03-08T21:00:11.770 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.786+0000 7fea4d400640 1 --2- 172.21.15.107:0/930591330 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55701c683150 0x55701c693f10 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fea4000e010 tx=0x7fea4000de80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71a6ba97abaf331d server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:00:11.770 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.786+0000 7fea46a00640 1 -- 172.21.15.107:0/930591330 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea40016070 con 0x55701c683150 2024-03-08T21:00:11.770 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.786+0000 7fea46a00640 1 -- 172.21.15.107:0/930591330 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fea400109b0 con 0x55701c683150 2024-03-08T21:00:11.770 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.786+0000 7fea46a00640 1 -- 172.21.15.107:0/930591330 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea400055b0 con 0x55701c683150 2024-03-08T21:00:11.773 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.786+0000 7fea5ffbe740 1 -- 172.21.15.107:0/930591330 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55701c683150 msgr2=0x55701c693f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:00:11.773 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.786+0000 7fea5ffbe740 1 --2- 172.21.15.107:0/930591330 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55701c683150 0x55701c693f10 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fea4000e010 tx=0x7fea4000de80 comp rx=0 tx=0).stop 2024-03-08T21:00:11.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.786+0000 7fea5ffbe740 1 -- 172.21.15.107:0/930591330 shutdown_connections 2024-03-08T21:00:11.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.786+0000 7fea5ffbe740 1 --2- 172.21.15.107:0/930591330 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55701c6882c0 0x55701c68bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:11.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.786+0000 7fea5ffbe740 1 --2- 172.21.15.107:0/930591330 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55701c683150 0x55701c693f10 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:11.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.786+0000 7fea5ffbe740 1 --2- 172.21.15.107:0/930591330 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55701c682760 0x55701c67e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:11.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.787+0000 7fea5ffbe740 1 -- 172.21.15.107:0/930591330 >> 172.21.15.107:0/930591330 conn(0x55701c67dcd0 msgr2=0x55701c687980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:00:11.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.787+0000 7fea5ffbe740 1 -- 172.21.15.107:0/930591330 shutdown_connections 2024-03-08T21:00:11.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.787+0000 7fea5ffbe740 1 -- 172.21.15.107:0/930591330 wait complete. 2024-03-08T21:00:11.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.787+0000 7fea5ffbe740 1 -- start start 2024-03-08T21:00:11.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.787+0000 7fea5ffbe740 1 Processor -- start 2024-03-08T21:00:11.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.787+0000 7fea5ffbe740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55701c682760 0x55701c6911b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:00:11.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.787+0000 7fea5ffbe740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55701c683150 0x55701c6916f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:00:11.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.787+0000 7fea5ffbe740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55701c6882c0 0x55701c6926b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:00:11.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.787+0000 7fea5ffbe740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55701c69a780 con 0x55701c683150 2024-03-08T21:00:11.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.787+0000 7fea5ffbe740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55701c69a120 con 0x55701c6882c0 2024-03-08T21:00:11.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.787+0000 7fea5ffbe740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55701c69ade0 con 0x55701c682760 2024-03-08T21:00:11.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.788+0000 7fea5ffbe740 10 client.0 _ll_register_callbacks cb 0x7fea480d26b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T21:00:11.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.788+0000 7fea4d400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55701c683150 0x55701c6916f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:00:11.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.788+0000 7fea4d400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55701c683150 0x55701c6916f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:56876/0 (socket says 172.21.15.107:56876) 2024-03-08T21:00:11.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.788+0000 7fea4d400640 1 -- 172.21.15.107:0/4254055959 learned_addr learned my addr 172.21.15.107:0/4254055959 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:00:11.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.788+0000 7fea4d400640 1 -- 172.21.15.107:0/4254055959 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55701c682760 msgr2=0x55701c6911b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:00:11.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.788+0000 7fea4d400640 1 --2- 172.21.15.107:0/4254055959 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55701c682760 0x55701c6911b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:11.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.788+0000 7fea4d400640 1 -- 172.21.15.107:0/4254055959 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55701c6882c0 msgr2=0x55701c6926b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:00:11.776 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.788+0000 7fea56800640 1 --2- 172.21.15.107:0/4254055959 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55701c6882c0 0x55701c6926b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:00:11.776 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.788+0000 7fea4d400640 1 --2- 172.21.15.107:0/4254055959 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55701c6882c0 0x55701c6926b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:11.776 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.788+0000 7fea4d400640 1 -- 172.21.15.107:0/4254055959 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55701c692ca0 con 0x55701c683150 2024-03-08T21:00:11.776 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.788+0000 7fea56800640 1 --2- 172.21.15.107:0/4254055959 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55701c6882c0 0x55701c6926b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T21:00:11.776 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.788+0000 7fea4d400640 1 --2- 172.21.15.107:0/4254055959 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55701c683150 0x55701c6916f0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fea40001eb0 tx=0x7fea40010660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15a77b3376754f40 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:00:11.776 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.788+0000 7fea3be00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-03-08T21:00:11.776 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.789+0000 7fea3be00640 1 -- 172.21.15.107:0/4254055959 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea40016040 con 0x55701c683150 2024-03-08T21:00:11.776 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.789+0000 7fea3be00640 1 -- 172.21.15.107:0/4254055959 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fea40010d70 con 0x55701c683150 2024-03-08T21:00:11.776 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.789+0000 7fea5ffbe740 1 -- 172.21.15.107:0/4254055959 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fea400109b0 con 0x55701c683150 2024-03-08T21:00:11.776 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.789+0000 7fea3be00640 1 -- 172.21.15.107:0/4254055959 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fea4000b6f0 con 0x55701c683150 2024-03-08T21:00:11.776 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.789+0000 7fea5ffbe740 10 client.15333 fetch_fsmap learned FSMap version 27 2024-03-08T21:00:11.776 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.789+0000 7fea5ffbe740 1 -- 172.21.15.107:0/4254055959 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55701c693fa0 con 0x55701c683150 2024-03-08T21:00:11.777 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.789+0000 7fea3be00640 1 -- 172.21.15.107:0/4254055959 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fea400109b0 con 0x55701c683150 2024-03-08T21:00:11.777 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.789+0000 7fea3be00640 1 -- 172.21.15.107:0/4254055959 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fea40004a00 con 0x55701c683150 2024-03-08T21:00:11.777 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.789+0000 7fea5ffbe740 10 client.15333 fetch_fsmap finished waiting for FSMap version 27 2024-03-08T21:00:11.777 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.790+0000 7fea5ffbe740 10 client.15333 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-08T21:00:11.777 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.790+0000 7fea5ffbe740 20 client.15333 populate_metadata read hostname 'smithi107' 2024-03-08T21:00:11.777 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.790+0000 7fea5ffbe740 1 --2- 172.21.15.107:0/4254055959 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55701c6f50b0 0x55701c6f11f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:00:11.777 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.790+0000 7fea5ffbe740 4 client.15333 mds_command: new command op to 34475 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-08T21:00:11.777 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.790+0000 7fea5ffbe740 1 -- 172.21.15.107:0/4254055959 --> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55701c5eef80 con 0x55701c6f50b0 2024-03-08T21:00:11.777 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.790+0000 7fea55e00640 1 --2- 172.21.15.107:0/4254055959 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55701c6f50b0 0x55701c6f11f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:00:11.779 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.791+0000 7fea3be00640 10 client.15333 ms_handle_connect on v2:172.21.15.149:6838/55290596 2024-03-08T21:00:11.779 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.791+0000 7fea55e00640 1 --2- 172.21.15.107:0/4254055959 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55701c6f50b0 0x55701c6f11f0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bbee97e26ab39cef server_cookie=386467122dc4fffe in_seq=0 out_seq=0 2024-03-08T21:00:11.779 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.792+0000 7fea3be00640 1 -- 172.21.15.107:0/4254055959 <== mds.0 v2:172.21.15.149:6838/55290596 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x55701c5eef80 con 0x55701c6f50b0 2024-03-08T21:00:11.779 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.792+0000 7fea3be00640 10 client.15333 handle_command_reply: tid=0 2024-03-08T21:00:11.779 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.792+0000 7fea5ffbe740 1 client.15333 shutdown 2024-03-08T21:00:11.779 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.792+0000 7fea5ffbe740 1 -- 172.21.15.107:0/4254055959 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55701c683150 msgr2=0x55701c6916f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:00:11.779 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.792+0000 7fea5ffbe740 1 --2- 172.21.15.107:0/4254055959 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55701c683150 0x55701c6916f0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fea40001eb0 tx=0x7fea40010660 comp rx=0 tx=0).stop 2024-03-08T21:00:11.779 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.792+0000 7fea5ffbe740 1 -- 172.21.15.107:0/4254055959 shutdown_connections 2024-03-08T21:00:11.780 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.792+0000 7fea5ffbe740 1 --2- 172.21.15.107:0/4254055959 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55701c6f50b0 0x55701c6f11f0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:11.780 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.792+0000 7fea5ffbe740 1 --2- 172.21.15.107:0/4254055959 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55701c6882c0 0x55701c6926b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:11.780 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.792+0000 7fea5ffbe740 1 --2- 172.21.15.107:0/4254055959 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55701c683150 0x55701c6916f0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:11.780 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.792+0000 7fea5ffbe740 1 --2- 172.21.15.107:0/4254055959 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55701c682760 0x55701c6911b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:11.780 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.792+0000 7fea3be00640 0 client.15333 ms_handle_reset on v2:172.21.15.149:6838/55290596 2024-03-08T21:00:11.780 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.792+0000 7fea5ffbe740 1 -- 172.21.15.107:0/4254055959 >> 172.21.15.107:0/4254055959 conn(0x55701c67dcd0 msgr2=0x55701c68c9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:00:11.780 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.792+0000 7fea5ffbe740 1 -- 172.21.15.107:0/4254055959 shutdown_connections 2024-03-08T21:00:11.780 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.792+0000 7fea5ffbe740 1 -- 172.21.15.107:0/4254055959 wait complete. 2024-03-08T21:00:11.780 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.792+0000 7fea5ffbe740 20 client.15333 trim_cache size 0 max 16384 2024-03-08T21:00:11.780 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-03-08T21:00:11.781 INFO:teuthology.orchestra.run.smithi107.stdout: "cluster_fsid": "dc20f0f8-dd8c-11ee-95c6-87774f69a715", 2024-03-08T21:00:11.781 INFO:teuthology.orchestra.run.smithi107.stdout: "whoami": 0, 2024-03-08T21:00:11.781 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 34475, 2024-03-08T21:00:11.781 INFO:teuthology.orchestra.run.smithi107.stdout: "want_state": "up:active", 2024-03-08T21:00:11.781 INFO:teuthology.orchestra.run.smithi107.stdout: "state": "up:active", 2024-03-08T21:00:11.781 INFO:teuthology.orchestra.run.smithi107.stdout: "fs_name": "cephfs", 2024-03-08T21:00:11.781 INFO:teuthology.orchestra.run.smithi107.stdout: "rank_uptime": 141.03123791600001, 2024-03-08T21:00:11.781 INFO:teuthology.orchestra.run.smithi107.stdout: "mdsmap_epoch": 27, 2024-03-08T21:00:11.781 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_epoch": 88, 2024-03-08T21:00:11.781 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_epoch_barrier": 86, 2024-03-08T21:00:11.782 INFO:teuthology.orchestra.run.smithi107.stdout: "uptime": 147.622673924 2024-03-08T21:00:11.782 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-03-08T21:00:11.783 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.793+0000 7fea57200640 1 -- 172.21.15.107:0/549511609 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fea3c0a2880 msgr2=0x7fea3c0a4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:00:11.783 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.793+0000 7fea57200640 1 --2- 172.21.15.107:0/549511609 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fea3c0a2880 0x7fea3c0a4d40 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x55701c6839b0 tx=0x7fea5002c740 comp rx=0 tx=0).stop 2024-03-08T21:00:11.783 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.793+0000 7fea57200640 1 -- 172.21.15.107:0/549511609 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fea58107810 msgr2=0x7fea5819d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:00:11.783 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.793+0000 7fea57200640 1 --2- 172.21.15.107:0/549511609 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fea58107810 0x7fea5819d990 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fea4800ccb0 tx=0x7fea4800a620 comp rx=0 tx=0).stop 2024-03-08T21:00:11.783 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.793+0000 7fea57200640 1 -- 172.21.15.107:0/549511609 shutdown_connections 2024-03-08T21:00:11.783 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.793+0000 7fea57200640 1 --2- 172.21.15.107:0/549511609 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fea3c0a2880 0x7fea3c0a4d40 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:11.783 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.793+0000 7fea57200640 1 --2- 172.21.15.107:0/549511609 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fea581093c0 0x7fea581a4f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:11.783 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.793+0000 7fea57200640 1 --2- 172.21.15.107:0/549511609 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fea58108a00 0x7fea5819ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:11.783 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.793+0000 7fea57200640 1 --2- 172.21.15.107:0/549511609 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fea58107810 0x7fea5819d990 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:11.783 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.793+0000 7fea57200640 1 -- 172.21.15.107:0/549511609 >> 172.21.15.107:0/549511609 conn(0x7fea580767c0 msgr2=0x7fea5810bca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:00:11.783 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.793+0000 7fea57200640 1 -- 172.21.15.107:0/549511609 shutdown_connections 2024-03-08T21:00:11.783 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:11.793+0000 7fea57200640 1 -- 172.21.15.107:0/549511609 wait complete. 2024-03-08T21:00:11.790 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:00:11 smithi107 ceph-mon[31381]: pgmap v433: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-08T21:00:11.792 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-03-08T21:00:11.817 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-03-08T21:00:11.817 INFO:tasks.ceph_fuse:config is {'client.0': {}} 2024-03-08T21:00:11.817 INFO:tasks.ceph_fuse:client.0 config is {'mount_subvol_num': 0} 2024-03-08T21:00:11.818 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-08T21:00:11.818 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-08T21:00:11.818 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T21:00:11.818 DEBUG:teuthology.orchestra.run.smithi107:> ip netns list 2024-03-08T21:00:11.870 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T21:00:11.871 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip link delete ceph-brx 2024-03-08T21:00:11.877 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:00:11 smithi204 ceph-mon[34935]: pgmap v433: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-08T21:00:11.917 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:00:11 smithi149 ceph-mon[35042]: pgmap v433: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-08T21:00:11.968 INFO:teuthology.orchestra.run.smithi107.stderr:Cannot find device "ceph-brx" 2024-03-08T21:00:11.972 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-08T21:00:11.972 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-03-08T21:00:11.973 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761 shell --fsid dc20f0f8-dd8c-11ee-95c6-87774f69a715 -- ceph fs ls 2024-03-08T21:00:12.427 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/mon.a/config 2024-03-08T21:00:13.878 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:00:13 smithi204 ceph-mon[34935]: pgmap v434: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-08T21:00:13.967 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:00:13 smithi107 ceph-mon[31381]: pgmap v434: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-08T21:00:14.014 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:00:13 smithi149 ceph-mon[35042]: pgmap v434: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-03-08T21:00:14.853 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:14.869+0000 7fa9e8800700 1 -- 172.21.15.107:0/1765352223 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa9e01018f0 msgr2=0x7fa9e0101d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:00:14.853 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:14.869+0000 7fa9e8800700 1 --2- 172.21.15.107:0/1765352223 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa9e01018f0 0x7fa9e0101d70 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fa9d0009990 tx=0x7fa9d0009ca0 comp rx=0 tx=0).stop 2024-03-08T21:00:14.853 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:00:14 smithi107 ceph-mon[31381]: pgmap v435: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-03-08T21:00:14.854 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:14.869+0000 7fa9e8800700 1 -- 172.21.15.107:0/1765352223 shutdown_connections 2024-03-08T21:00:14.854 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:14.869+0000 7fa9e8800700 1 --2- 172.21.15.107:0/1765352223 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa9e0105d70 0x7fa9e0106150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:14.854 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:14.869+0000 7fa9e8800700 1 --2- 172.21.15.107:0/1765352223 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa9e01022b0 0x7fa9e010dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:14.854 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:14.869+0000 7fa9e8800700 1 --2- 172.21.15.107:0/1765352223 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa9e01018f0 0x7fa9e0101d70 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:14.855 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:14.869+0000 7fa9e8800700 1 -- 172.21.15.107:0/1765352223 >> 172.21.15.107:0/1765352223 conn(0x7fa9e0076030 msgr2=0x7fa9e0076440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:00:14.855 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:14.869+0000 7fa9e8800700 1 -- 172.21.15.107:0/1765352223 shutdown_connections 2024-03-08T21:00:14.855 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:14.869+0000 7fa9e8800700 1 -- 172.21.15.107:0/1765352223 wait complete. 2024-03-08T21:00:14.855 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:14.871+0000 7fa9e8800700 1 Processor -- start 2024-03-08T21:00:14.855 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:14.871+0000 7fa9e8800700 1 -- start start 2024-03-08T21:00:14.855 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:14.871+0000 7fa9e8800700 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa9e01018f0 0x7fa9e0198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:00:14.855 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:14.871+0000 7fa9e8800700 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa9e01022b0 0x7fa9e01989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:00:14.855 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:14.871+0000 7fa9e8800700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa9e0105d70 0x7fa9e019cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:00:14.855 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:14.871+0000 7fa9e8800700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9e0110f20 con 0x7fa9e0105d70 2024-03-08T21:00:14.855 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:14.871+0000 7fa9e8800700 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9e0110da0 con 0x7fa9e01018f0 2024-03-08T21:00:14.855 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:14.871+0000 7fa9e8800700 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9e01110a0 con 0x7fa9e01022b0 2024-03-08T21:00:14.857 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:14.872+0000 7fa9e6800700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa9e0105d70 0x7fa9e019cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:00:14.857 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:14.872+0000 7fa9e6800700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa9e0105d70 0x7fa9e019cd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:56896/0 (socket says 172.21.15.107:56896) 2024-03-08T21:00:14.857 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:14.872+0000 7fa9e6800700 1 -- 172.21.15.107:0/912173656 learned_addr learned my addr 172.21.15.107:0/912173656 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:00:14.857 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:14.872+0000 7fa9e5e00700 1 --2- 172.21.15.107:0/912173656 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa9e01018f0 0x7fa9e0198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:00:14.857 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:14.872+0000 7fa9e5400700 1 --2- 172.21.15.107:0/912173656 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa9e01022b0 0x7fa9e01989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:00:14.857 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:14.872+0000 7fa9e6800700 1 -- 172.21.15.107:0/912173656 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa9e01022b0 msgr2=0x7fa9e01989a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:00:14.857 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:14.872+0000 7fa9e6800700 1 --2- 172.21.15.107:0/912173656 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa9e01022b0 0x7fa9e01989a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:14.857 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:14.872+0000 7fa9e6800700 1 -- 172.21.15.107:0/912173656 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa9e01018f0 msgr2=0x7fa9e0198460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:00:14.857 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:14.872+0000 7fa9e6800700 1 --2- 172.21.15.107:0/912173656 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa9e01018f0 0x7fa9e0198460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:14.857 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:14.872+0000 7fa9e6800700 1 -- 172.21.15.107:0/912173656 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9e019d470 con 0x7fa9e0105d70 2024-03-08T21:00:14.857 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:14.873+0000 7fa9e6800700 1 --2- 172.21.15.107:0/912173656 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa9e0105d70 0x7fa9e019cd30 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fa9dc007d60 tx=0x7fa9dc00a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:00:14.857 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:14.873+0000 7fa9daa00700 1 -- 172.21.15.107:0/912173656 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9dc00ac90 con 0x7fa9e0105d70 2024-03-08T21:00:14.857 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:14.873+0000 7fa9daa00700 1 -- 172.21.15.107:0/912173656 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa9dc00ae30 con 0x7fa9e0105d70 2024-03-08T21:00:14.857 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:14.873+0000 7fa9e8800700 1 -- 172.21.15.107:0/912173656 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9e019d760 con 0x7fa9e0105d70 2024-03-08T21:00:14.858 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:14.873+0000 7fa9e8800700 1 -- 172.21.15.107:0/912173656 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9e01029a0 con 0x7fa9e0105d70 2024-03-08T21:00:14.858 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:14.873+0000 7fa9daa00700 1 -- 172.21.15.107:0/912173656 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9dc0048d0 con 0x7fa9e0105d70 2024-03-08T21:00:14.858 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:14.874+0000 7fa9e8800700 1 -- 172.21.15.107:0/912173656 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9c4005330 con 0x7fa9e0105d70 2024-03-08T21:00:14.859 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:14.876+0000 7fa9daa00700 1 -- 172.21.15.107:0/912173656 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) v1 ==== 137291+0+0 (secure 0 0 0) 0x7fa9dc003a00 con 0x7fa9e0105d70 2024-03-08T21:00:14.861 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:14.878+0000 7fa9daa00700 1 --2- 172.21.15.107:0/912173656 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fa9c80a2910 0x7fa9c80a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:00:14.861 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:14.878+0000 7fa9daa00700 1 -- 172.21.15.107:0/912173656 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fa9dc0ceb00 con 0x7fa9e0105d70 2024-03-08T21:00:14.861 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:14.879+0000 7fa9e5e00700 1 --2- 172.21.15.107:0/912173656 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fa9c80a2910 0x7fa9c80a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:00:14.862 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:14.879+0000 7fa9e5e00700 1 --2- 172.21.15.107:0/912173656 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fa9c80a2910 0x7fa9c80a4dd0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fa9d4005950 tx=0x7fa9d400a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:00:14.865 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:14.883+0000 7fa9daa00700 1 -- 172.21.15.107:0/912173656 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7fa9dc09b510 con 0x7fa9e0105d70 2024-03-08T21:00:14.877 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:00:14 smithi204 ceph-mon[34935]: pgmap v435: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-03-08T21:00:15.014 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:00:14 smithi149 ceph-mon[35042]: pgmap v435: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-03-08T21:00:15.141 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:15.158+0000 7fa9e8800700 1 -- 172.21.15.107:0/912173656 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fa9c4005e30 con 0x7fa9e0105d70 2024-03-08T21:00:15.142 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:15.159+0000 7fa9daa00700 1 -- 172.21.15.107:0/912173656 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v27) v1 ==== 53+0+73 (secure 0 0 0) 0x7fa9dc09f5b0 con 0x7fa9e0105d70 2024-03-08T21:00:15.142 INFO:teuthology.orchestra.run.smithi107.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-08T21:00:15.145 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:15.162+0000 7fa9e8800700 1 -- 172.21.15.107:0/912173656 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fa9c80a2910 msgr2=0x7fa9c80a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:00:15.145 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:15.162+0000 7fa9e8800700 1 --2- 172.21.15.107:0/912173656 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fa9c80a2910 0x7fa9c80a4dd0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fa9d4005950 tx=0x7fa9d400a300 comp rx=0 tx=0).stop 2024-03-08T21:00:15.145 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:15.162+0000 7fa9e8800700 1 -- 172.21.15.107:0/912173656 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa9e0105d70 msgr2=0x7fa9e019cd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:00:15.145 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:15.162+0000 7fa9e8800700 1 --2- 172.21.15.107:0/912173656 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa9e0105d70 0x7fa9e019cd30 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fa9dc007d60 tx=0x7fa9dc00a340 comp rx=0 tx=0).stop 2024-03-08T21:00:15.146 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:15.162+0000 7fa9e8800700 1 -- 172.21.15.107:0/912173656 shutdown_connections 2024-03-08T21:00:15.146 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:15.162+0000 7fa9e8800700 1 --2- 172.21.15.107:0/912173656 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fa9c80a2910 0x7fa9c80a4dd0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:15.146 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:15.162+0000 7fa9e8800700 1 --2- 172.21.15.107:0/912173656 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa9e0105d70 0x7fa9e019cd30 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:15.146 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:15.162+0000 7fa9e8800700 1 --2- 172.21.15.107:0/912173656 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa9e01018f0 0x7fa9e0198460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:15.146 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:15.162+0000 7fa9e8800700 1 --2- 172.21.15.107:0/912173656 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa9e01022b0 0x7fa9e01989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:15.146 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:15.162+0000 7fa9e8800700 1 -- 172.21.15.107:0/912173656 >> 172.21.15.107:0/912173656 conn(0x7fa9e0076030 msgr2=0x7fa9e01008a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:00:15.146 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:15.162+0000 7fa9e8800700 1 -- 172.21.15.107:0/912173656 shutdown_connections 2024-03-08T21:00:15.146 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:15.162+0000 7fa9e8800700 1 -- 172.21.15.107:0/912173656 wait complete. 2024-03-08T21:00:15.829 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:00:15 smithi107 ceph-mon[31381]: from='client.? 172.21.15.107:0/912173656' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-08T21:00:15.878 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:00:15 smithi204 ceph-mon[34935]: from='client.? 172.21.15.107:0/912173656' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-08T21:00:16.014 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:00:15 smithi149 ceph-mon[35042]: from='client.? 172.21.15.107:0/912173656' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-08T21:00:16.121 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-08T21:00:16.121 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-08T21:00:16.122 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi107.front.sepia.ceph.com 2024-03-08T21:00:16.122 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-03-08T21:00:16.122 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-08T21:00:16.122 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-08T21:00:16.122 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-08T21:00:16.122 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-08T21:00:16.122 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-08T21:00:16.122 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T21:00:16.122 DEBUG:teuthology.orchestra.run.smithi107:> ip addr 2024-03-08T21:00:16.178 INFO:teuthology.orchestra.run.smithi107.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-08T21:00:16.178 INFO:teuthology.orchestra.run.smithi107.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-08T21:00:16.178 INFO:teuthology.orchestra.run.smithi107.stdout: inet 127.0.0.1/8 scope host lo 2024-03-08T21:00:16.178 INFO:teuthology.orchestra.run.smithi107.stdout: valid_lft forever preferred_lft forever 2024-03-08T21:00:16.178 INFO:teuthology.orchestra.run.smithi107.stdout: inet6 ::1/128 scope host 2024-03-08T21:00:16.178 INFO:teuthology.orchestra.run.smithi107.stdout: valid_lft forever preferred_lft forever 2024-03-08T21:00:16.179 INFO:teuthology.orchestra.run.smithi107.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-08T21:00:16.179 INFO:teuthology.orchestra.run.smithi107.stdout: link/ether 0c:c4:7a:d8:a2:0a brd ff:ff:ff:ff:ff:ff 2024-03-08T21:00:16.179 INFO:teuthology.orchestra.run.smithi107.stdout: altname enp7s0f0 2024-03-08T21:00:16.179 INFO:teuthology.orchestra.run.smithi107.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-08T21:00:16.179 INFO:teuthology.orchestra.run.smithi107.stdout: link/ether 0c:c4:7a:d8:a2:0b brd ff:ff:ff:ff:ff:ff 2024-03-08T21:00:16.179 INFO:teuthology.orchestra.run.smithi107.stdout: altname enp7s0f1 2024-03-08T21:00:16.179 INFO:teuthology.orchestra.run.smithi107.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-08T21:00:16.179 INFO:teuthology.orchestra.run.smithi107.stdout: link/ether 0c:c4:7a:8f:47:ea brd ff:ff:ff:ff:ff:ff 2024-03-08T21:00:16.179 INFO:teuthology.orchestra.run.smithi107.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-08T21:00:16.179 INFO:teuthology.orchestra.run.smithi107.stdout: link/ether 0c:c4:7a:8f:47:eb brd ff:ff:ff:ff:ff:ff 2024-03-08T21:00:16.179 INFO:teuthology.orchestra.run.smithi107.stdout: inet 172.21.15.107/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-03-08T21:00:16.179 INFO:teuthology.orchestra.run.smithi107.stdout: valid_lft 41761sec preferred_lft 41761sec 2024-03-08T21:00:16.180 INFO:teuthology.orchestra.run.smithi107.stdout: inet6 fe80::ec4:7aff:fe8f:47eb/64 scope link 2024-03-08T21:00:16.180 INFO:teuthology.orchestra.run.smithi107.stdout: valid_lft forever preferred_lft forever 2024-03-08T21:00:16.180 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-03-08T21:00:16.180 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T21:00:16.180 DEBUG:teuthology.orchestra.run.smithi107:> (cd / && exec bash -c ' 2024-03-08T21:00:16.180 DEBUG:teuthology.orchestra.run.smithi107:> set -e 2024-03-08T21:00:16.181 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip link add name ceph-brx type bridge 2024-03-08T21:00:16.181 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip addr flush dev ceph-brx 2024-03-08T21:00:16.181 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip link set ceph-brx up 2024-03-08T21:00:16.181 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-03-08T21:00:16.181 DEBUG:teuthology.orchestra.run.smithi107:> ') 2024-03-08T21:00:16.616 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T21:00:16.617 DEBUG:teuthology.orchestra.run.smithi107:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-08T21:00:16.689 INFO:teuthology.orchestra.run.smithi107.stdout:1 2024-03-08T21:00:16.692 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T21:00:16.693 DEBUG:teuthology.orchestra.run.smithi107:> route 2024-03-08T21:00:16.752 INFO:teuthology.orchestra.run.smithi107.stdout:Kernel IP routing table 2024-03-08T21:00:16.752 INFO:teuthology.orchestra.run.smithi107.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-08T21:00:16.752 INFO:teuthology.orchestra.run.smithi107.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-08T21:00:16.752 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-08T21:00:16.752 INFO:teuthology.orchestra.run.smithi107.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-08T21:00:16.753 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T21:00:16.753 DEBUG:teuthology.orchestra.run.smithi107:> (cd / && exec bash -c ' 2024-03-08T21:00:16.753 DEBUG:teuthology.orchestra.run.smithi107:> set -e 2024-03-08T21:00:16.753 DEBUG:teuthology.orchestra.run.smithi107:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-08T21:00:16.753 DEBUG:teuthology.orchestra.run.smithi107:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-08T21:00:16.753 DEBUG:teuthology.orchestra.run.smithi107:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-08T21:00:16.753 DEBUG:teuthology.orchestra.run.smithi107:> ') 2024-03-08T21:00:16.838 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:00:16 smithi107 ceph-mon[31381]: pgmap v436: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-08T21:00:16.877 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:00:16 smithi204 ceph-mon[34935]: pgmap v436: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-08T21:00:17.014 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:00:16 smithi149 ceph-mon[35042]: pgmap v436: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-03-08T21:00:17.420 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T21:00:17.421 DEBUG:teuthology.orchestra.run.smithi107:> ip netns list 2024-03-08T21:00:17.474 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T21:00:17.474 DEBUG:teuthology.orchestra.run.smithi107:> ip netns list-id 2024-03-08T21:00:17.524 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T21:00:17.525 DEBUG:teuthology.orchestra.run.smithi107:> (cd / && exec bash -c ' 2024-03-08T21:00:17.525 DEBUG:teuthology.orchestra.run.smithi107:> set -e 2024-03-08T21:00:17.525 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-08T21:00:17.525 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-03-08T21:00:17.525 DEBUG:teuthology.orchestra.run.smithi107:> ') 2024-03-08T21:00:17.662 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-03-08T21:00:17.662 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T21:00:17.662 DEBUG:teuthology.orchestra.run.smithi107:> (cd / && exec bash -c ' 2024-03-08T21:00:17.662 DEBUG:teuthology.orchestra.run.smithi107:> set -e 2024-03-08T21:00:17.662 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-03-08T21:00:17.662 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-03-08T21:00:17.662 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-03-08T21:00:17.663 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-03-08T21:00:17.663 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-03-08T21:00:17.663 DEBUG:teuthology.orchestra.run.smithi107:> ') 2024-03-08T21:00:17.930 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T21:00:17.931 DEBUG:teuthology.orchestra.run.smithi107:> (cd / && exec bash -c ' 2024-03-08T21:00:17.931 DEBUG:teuthology.orchestra.run.smithi107:> set -e 2024-03-08T21:00:17.931 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip link set brx.0 up 2024-03-08T21:00:17.931 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip link set dev brx.0 master ceph-brx 2024-03-08T21:00:17.931 DEBUG:teuthology.orchestra.run.smithi107:> ') 2024-03-08T21:00:18.092 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {'mount_subvol_num': 0} 2024-03-08T21:00:18.092 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-08T21:00:18.092 DEBUG:teuthology.orchestra.run.smithi107:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-03-08T21:00:18.135 INFO:teuthology.orchestra.run.smithi107.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-03-08T21:00:18.135 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-08T21:00:18.136 DEBUG:teuthology.orchestra.run.smithi107:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-03-08T21:00:18.226 DEBUG:teuthology.orchestra.run.smithi107:> sudo modprobe fuse 2024-03-08T21:00:18.314 DEBUG:teuthology.orchestra.run.smithi107:> cat /proc/self/mounts | awk '{print $2}' 2024-03-08T21:00:18.366 INFO:teuthology.orchestra.run.smithi107.stdout:/proc 2024-03-08T21:00:18.366 INFO:teuthology.orchestra.run.smithi107.stdout:/sys 2024-03-08T21:00:18.366 INFO:teuthology.orchestra.run.smithi107.stdout:/dev 2024-03-08T21:00:18.366 INFO:teuthology.orchestra.run.smithi107.stdout:/sys/kernel/security 2024-03-08T21:00:18.366 INFO:teuthology.orchestra.run.smithi107.stdout:/dev/shm 2024-03-08T21:00:18.366 INFO:teuthology.orchestra.run.smithi107.stdout:/dev/pts 2024-03-08T21:00:18.366 INFO:teuthology.orchestra.run.smithi107.stdout:/run 2024-03-08T21:00:18.366 INFO:teuthology.orchestra.run.smithi107.stdout:/sys/fs/cgroup 2024-03-08T21:00:18.366 INFO:teuthology.orchestra.run.smithi107.stdout:/sys/fs/pstore 2024-03-08T21:00:18.366 INFO:teuthology.orchestra.run.smithi107.stdout:/sys/fs/bpf 2024-03-08T21:00:18.367 INFO:teuthology.orchestra.run.smithi107.stdout:/ 2024-03-08T21:00:18.367 INFO:teuthology.orchestra.run.smithi107.stdout:/sys/fs/selinux 2024-03-08T21:00:18.367 INFO:teuthology.orchestra.run.smithi107.stdout:/proc/sys/fs/binfmt_misc 2024-03-08T21:00:18.367 INFO:teuthology.orchestra.run.smithi107.stdout:/dev/hugepages 2024-03-08T21:00:18.367 INFO:teuthology.orchestra.run.smithi107.stdout:/dev/mqueue 2024-03-08T21:00:18.367 INFO:teuthology.orchestra.run.smithi107.stdout:/sys/kernel/debug 2024-03-08T21:00:18.367 INFO:teuthology.orchestra.run.smithi107.stdout:/sys/kernel/tracing 2024-03-08T21:00:18.367 INFO:teuthology.orchestra.run.smithi107.stdout:/sys/fs/fuse/connections 2024-03-08T21:00:18.367 INFO:teuthology.orchestra.run.smithi107.stdout:/sys/kernel/config 2024-03-08T21:00:18.367 INFO:teuthology.orchestra.run.smithi107.stdout:/run/credentials/systemd-sysctl.service 2024-03-08T21:00:18.367 INFO:teuthology.orchestra.run.smithi107.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-03-08T21:00:18.367 INFO:teuthology.orchestra.run.smithi107.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-03-08T21:00:18.367 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/nfs/rpc_pipefs 2024-03-08T21:00:18.367 INFO:teuthology.orchestra.run.smithi107.stdout:/run/user/1000 2024-03-08T21:00:18.368 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/ceph 2024-03-08T21:00:18.368 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/containers/storage/overlay 2024-03-08T21:00:18.368 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/containers/storage/overlay/cc3f77fde58910700b166306bb59cae6394c9bcf80aa5054f52703bfcae6a1ef/merged 2024-03-08T21:00:18.368 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/containers/storage/overlay/89c0b3435dc98f1f357f507b635c0183531bd24ebf90e4bf035d506a4cdeeb27/merged 2024-03-08T21:00:18.368 INFO:teuthology.orchestra.run.smithi107.stdout:/run/user/0 2024-03-08T21:00:18.368 INFO:teuthology.orchestra.run.smithi107.stdout:/proc/sys/fs/binfmt_misc 2024-03-08T21:00:18.369 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/containers/storage/overlay/c4337a01827a8ae8ffaee98b425f951075d62a4a5735092c949bf8bbb7a48a81/merged 2024-03-08T21:00:18.369 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/containers/storage/overlay/973dc733abe047551bc6eacba7b63800d2ae4fa7a76c183b367be885d76f288c/merged 2024-03-08T21:00:18.369 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/containers/storage/overlay/4d03b3617accb1e88f42500de8b626beb3e0b00c26079b71937e21a88afa210c/merged 2024-03-08T21:00:18.369 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/containers/storage/overlay/eabce703cb492d668d9c25b3774087577d00fbf2c33ccea289f9f388b1fbbd38/merged 2024-03-08T21:00:18.369 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/containers/storage/overlay/36ebf2109c63bbafb6ca84e3c08ed7ea2d7c6a2f917099cfb6a2724287e057cb/merged 2024-03-08T21:00:18.369 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/containers/storage/overlay/cebccdc40745a1064427b844c8ecf2e7913226cba924cebcb71947053e36d96d/merged 2024-03-08T21:00:18.369 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/containers/storage/overlay/b7f034dc4ce8e0359f3e655a5d7f00f00dd603f0d0083c9828ac7baaa6200e8c/merged 2024-03-08T21:00:18.369 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/containers/storage/overlay/14dced56db323f542497d619fab83980a86d50566c274347d1bbf5acf1341a4a/merged 2024-03-08T21:00:18.369 INFO:teuthology.orchestra.run.smithi107.stdout:/run/netns 2024-03-08T21:00:18.369 INFO:teuthology.orchestra.run.smithi107.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-08T21:00:18.369 INFO:teuthology.orchestra.run.smithi107.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-08T21:00:18.370 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T21:00:18.371 DEBUG:teuthology.orchestra.run.smithi107:> ls /sys/fs/fuse/connections 2024-03-08T21:00:18.424 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-03-08T21:00:18.424 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0 --client_mountpoint=/volumes/qa/sv_0/2193ae44-16ef-4c95-bf3c-e3e15e292573 --client_fs=cephfs) 2024-03-08T21:00:18.430 DEBUG:teuthology.orchestra.run.smithi107:> sudo modprobe fuse 2024-03-08T21:00:18.505 DEBUG:teuthology.orchestra.run.smithi107:> cat /proc/self/mounts | awk '{print $2}' 2024-03-08T21:00:18.548 INFO:teuthology.orchestra.run.smithi107.stdout:/proc 2024-03-08T21:00:18.548 INFO:teuthology.orchestra.run.smithi107.stdout:/sys 2024-03-08T21:00:18.548 INFO:teuthology.orchestra.run.smithi107.stdout:/dev 2024-03-08T21:00:18.548 INFO:teuthology.orchestra.run.smithi107.stdout:/sys/kernel/security 2024-03-08T21:00:18.548 INFO:teuthology.orchestra.run.smithi107.stdout:/dev/shm 2024-03-08T21:00:18.548 INFO:teuthology.orchestra.run.smithi107.stdout:/dev/pts 2024-03-08T21:00:18.549 INFO:teuthology.orchestra.run.smithi107.stdout:/run 2024-03-08T21:00:18.549 INFO:teuthology.orchestra.run.smithi107.stdout:/sys/fs/cgroup 2024-03-08T21:00:18.549 INFO:teuthology.orchestra.run.smithi107.stdout:/sys/fs/pstore 2024-03-08T21:00:18.549 INFO:teuthology.orchestra.run.smithi107.stdout:/sys/fs/bpf 2024-03-08T21:00:18.549 INFO:teuthology.orchestra.run.smithi107.stdout:/ 2024-03-08T21:00:18.549 INFO:teuthology.orchestra.run.smithi107.stdout:/sys/fs/selinux 2024-03-08T21:00:18.549 INFO:teuthology.orchestra.run.smithi107.stdout:/proc/sys/fs/binfmt_misc 2024-03-08T21:00:18.549 INFO:teuthology.orchestra.run.smithi107.stdout:/dev/hugepages 2024-03-08T21:00:18.549 INFO:teuthology.orchestra.run.smithi107.stdout:/dev/mqueue 2024-03-08T21:00:18.549 INFO:teuthology.orchestra.run.smithi107.stdout:/sys/kernel/debug 2024-03-08T21:00:18.549 INFO:teuthology.orchestra.run.smithi107.stdout:/sys/kernel/tracing 2024-03-08T21:00:18.549 INFO:teuthology.orchestra.run.smithi107.stdout:/sys/fs/fuse/connections 2024-03-08T21:00:18.549 INFO:teuthology.orchestra.run.smithi107.stdout:/sys/kernel/config 2024-03-08T21:00:18.549 INFO:teuthology.orchestra.run.smithi107.stdout:/run/credentials/systemd-sysctl.service 2024-03-08T21:00:18.550 INFO:teuthology.orchestra.run.smithi107.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-03-08T21:00:18.550 INFO:teuthology.orchestra.run.smithi107.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-03-08T21:00:18.550 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/nfs/rpc_pipefs 2024-03-08T21:00:18.550 INFO:teuthology.orchestra.run.smithi107.stdout:/run/user/1000 2024-03-08T21:00:18.550 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/ceph 2024-03-08T21:00:18.550 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/containers/storage/overlay 2024-03-08T21:00:18.550 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/containers/storage/overlay/cc3f77fde58910700b166306bb59cae6394c9bcf80aa5054f52703bfcae6a1ef/merged 2024-03-08T21:00:18.550 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/containers/storage/overlay/89c0b3435dc98f1f357f507b635c0183531bd24ebf90e4bf035d506a4cdeeb27/merged 2024-03-08T21:00:18.550 INFO:teuthology.orchestra.run.smithi107.stdout:/run/user/0 2024-03-08T21:00:18.550 INFO:teuthology.orchestra.run.smithi107.stdout:/proc/sys/fs/binfmt_misc 2024-03-08T21:00:18.550 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/containers/storage/overlay/c4337a01827a8ae8ffaee98b425f951075d62a4a5735092c949bf8bbb7a48a81/merged 2024-03-08T21:00:18.550 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/containers/storage/overlay/973dc733abe047551bc6eacba7b63800d2ae4fa7a76c183b367be885d76f288c/merged 2024-03-08T21:00:18.550 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/containers/storage/overlay/4d03b3617accb1e88f42500de8b626beb3e0b00c26079b71937e21a88afa210c/merged 2024-03-08T21:00:18.550 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/containers/storage/overlay/eabce703cb492d668d9c25b3774087577d00fbf2c33ccea289f9f388b1fbbd38/merged 2024-03-08T21:00:18.551 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/containers/storage/overlay/36ebf2109c63bbafb6ca84e3c08ed7ea2d7c6a2f917099cfb6a2724287e057cb/merged 2024-03-08T21:00:18.551 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/containers/storage/overlay/cebccdc40745a1064427b844c8ecf2e7913226cba924cebcb71947053e36d96d/merged 2024-03-08T21:00:18.551 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/containers/storage/overlay/b7f034dc4ce8e0359f3e655a5d7f00f00dd603f0d0083c9828ac7baaa6200e8c/merged 2024-03-08T21:00:18.551 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/containers/storage/overlay/14dced56db323f542497d619fab83980a86d50566c274347d1bbf5acf1341a4a/merged 2024-03-08T21:00:18.551 INFO:teuthology.orchestra.run.smithi107.stdout:/run/netns 2024-03-08T21:00:18.551 INFO:teuthology.orchestra.run.smithi107.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-08T21:00:18.551 INFO:teuthology.orchestra.run.smithi107.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-08T21:00:18.552 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T21:00:18.552 DEBUG:teuthology.orchestra.run.smithi107:> ls /sys/fs/fuse/connections 2024-03-08T21:00:18.642 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi107.stderr:2024-03-08T21:00:18.659+0000 7f3b83b3d480 -1 init, newargv = 0x55ff1652c9a0 newargc=15 2024-03-08T21:00:18.642 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi107.stderr:ceph-fuse[84486]: starting ceph client 2024-03-08T21:00:18.704 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi107.stderr:ceph-fuse[84486]: starting fuse 2024-03-08T21:00:19.717 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:00:19 smithi107 ceph-mon[31381]: pgmap v437: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-03-08T21:00:19.739 DEBUG:teuthology.orchestra.run.smithi107:> sudo modprobe fuse 2024-03-08T21:00:19.764 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:00:19 smithi149 ceph-mon[35042]: pgmap v437: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-03-08T21:00:19.842 DEBUG:teuthology.orchestra.run.smithi107:> cat /proc/self/mounts | awk '{print $2}' 2024-03-08T21:00:19.878 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:00:19 smithi204 ceph-mon[34935]: pgmap v437: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-03-08T21:00:19.897 INFO:teuthology.orchestra.run.smithi107.stdout:/proc 2024-03-08T21:00:19.898 INFO:teuthology.orchestra.run.smithi107.stdout:/sys 2024-03-08T21:00:19.898 INFO:teuthology.orchestra.run.smithi107.stdout:/dev 2024-03-08T21:00:19.898 INFO:teuthology.orchestra.run.smithi107.stdout:/sys/kernel/security 2024-03-08T21:00:19.898 INFO:teuthology.orchestra.run.smithi107.stdout:/dev/shm 2024-03-08T21:00:19.898 INFO:teuthology.orchestra.run.smithi107.stdout:/dev/pts 2024-03-08T21:00:19.898 INFO:teuthology.orchestra.run.smithi107.stdout:/run 2024-03-08T21:00:19.898 INFO:teuthology.orchestra.run.smithi107.stdout:/sys/fs/cgroup 2024-03-08T21:00:19.898 INFO:teuthology.orchestra.run.smithi107.stdout:/sys/fs/pstore 2024-03-08T21:00:19.898 INFO:teuthology.orchestra.run.smithi107.stdout:/sys/fs/bpf 2024-03-08T21:00:19.898 INFO:teuthology.orchestra.run.smithi107.stdout:/ 2024-03-08T21:00:19.898 INFO:teuthology.orchestra.run.smithi107.stdout:/sys/fs/selinux 2024-03-08T21:00:19.899 INFO:teuthology.orchestra.run.smithi107.stdout:/proc/sys/fs/binfmt_misc 2024-03-08T21:00:19.899 INFO:teuthology.orchestra.run.smithi107.stdout:/dev/hugepages 2024-03-08T21:00:19.899 INFO:teuthology.orchestra.run.smithi107.stdout:/dev/mqueue 2024-03-08T21:00:19.899 INFO:teuthology.orchestra.run.smithi107.stdout:/sys/kernel/debug 2024-03-08T21:00:19.899 INFO:teuthology.orchestra.run.smithi107.stdout:/sys/kernel/tracing 2024-03-08T21:00:19.899 INFO:teuthology.orchestra.run.smithi107.stdout:/sys/fs/fuse/connections 2024-03-08T21:00:19.899 INFO:teuthology.orchestra.run.smithi107.stdout:/sys/kernel/config 2024-03-08T21:00:19.899 INFO:teuthology.orchestra.run.smithi107.stdout:/run/credentials/systemd-sysctl.service 2024-03-08T21:00:19.899 INFO:teuthology.orchestra.run.smithi107.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-03-08T21:00:19.899 INFO:teuthology.orchestra.run.smithi107.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-03-08T21:00:19.900 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/nfs/rpc_pipefs 2024-03-08T21:00:19.900 INFO:teuthology.orchestra.run.smithi107.stdout:/run/user/1000 2024-03-08T21:00:19.900 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/ceph 2024-03-08T21:00:19.900 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/containers/storage/overlay 2024-03-08T21:00:19.900 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/containers/storage/overlay/cc3f77fde58910700b166306bb59cae6394c9bcf80aa5054f52703bfcae6a1ef/merged 2024-03-08T21:00:19.900 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/containers/storage/overlay/89c0b3435dc98f1f357f507b635c0183531bd24ebf90e4bf035d506a4cdeeb27/merged 2024-03-08T21:00:19.900 INFO:teuthology.orchestra.run.smithi107.stdout:/run/user/0 2024-03-08T21:00:19.900 INFO:teuthology.orchestra.run.smithi107.stdout:/proc/sys/fs/binfmt_misc 2024-03-08T21:00:19.900 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/containers/storage/overlay/c4337a01827a8ae8ffaee98b425f951075d62a4a5735092c949bf8bbb7a48a81/merged 2024-03-08T21:00:19.900 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/containers/storage/overlay/973dc733abe047551bc6eacba7b63800d2ae4fa7a76c183b367be885d76f288c/merged 2024-03-08T21:00:19.901 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/containers/storage/overlay/4d03b3617accb1e88f42500de8b626beb3e0b00c26079b71937e21a88afa210c/merged 2024-03-08T21:00:19.901 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/containers/storage/overlay/eabce703cb492d668d9c25b3774087577d00fbf2c33ccea289f9f388b1fbbd38/merged 2024-03-08T21:00:19.901 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/containers/storage/overlay/36ebf2109c63bbafb6ca84e3c08ed7ea2d7c6a2f917099cfb6a2724287e057cb/merged 2024-03-08T21:00:19.901 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/containers/storage/overlay/cebccdc40745a1064427b844c8ecf2e7913226cba924cebcb71947053e36d96d/merged 2024-03-08T21:00:19.901 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/containers/storage/overlay/b7f034dc4ce8e0359f3e655a5d7f00f00dd603f0d0083c9828ac7baaa6200e8c/merged 2024-03-08T21:00:19.901 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/containers/storage/overlay/14dced56db323f542497d619fab83980a86d50566c274347d1bbf5acf1341a4a/merged 2024-03-08T21:00:19.901 INFO:teuthology.orchestra.run.smithi107.stdout:/run/netns 2024-03-08T21:00:19.901 INFO:teuthology.orchestra.run.smithi107.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-08T21:00:19.901 INFO:teuthology.orchestra.run.smithi107.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-08T21:00:19.901 INFO:teuthology.orchestra.run.smithi107.stdout:/home/ubuntu/cephtest/mnt.0 2024-03-08T21:00:19.902 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T21:00:19.902 DEBUG:teuthology.orchestra.run.smithi107:> ls /sys/fs/fuse/connections 2024-03-08T21:00:19.952 INFO:teuthology.orchestra.run.smithi107.stdout:118 2024-03-08T21:00:19.952 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [118] 2024-03-08T21:00:19.952 DEBUG:teuthology.orchestra.run.smithi107:> sudo stdin-killer -- python3 -c ' 2024-03-08T21:00:19.952 DEBUG:teuthology.orchestra.run.smithi107:> import glob 2024-03-08T21:00:19.952 DEBUG:teuthology.orchestra.run.smithi107:> import re 2024-03-08T21:00:19.953 DEBUG:teuthology.orchestra.run.smithi107:> import os 2024-03-08T21:00:19.953 DEBUG:teuthology.orchestra.run.smithi107:> import subprocess 2024-03-08T21:00:19.953 DEBUG:teuthology.orchestra.run.smithi107:> 2024-03-08T21:00:19.953 DEBUG:teuthology.orchestra.run.smithi107:> def _find_admin_socket(client_name): 2024-03-08T21:00:19.953 DEBUG:teuthology.orchestra.run.smithi107:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-03-08T21:00:19.953 DEBUG:teuthology.orchestra.run.smithi107:> files = glob.glob(asok_path) 2024-03-08T21:00:19.953 DEBUG:teuthology.orchestra.run.smithi107:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-03-08T21:00:19.953 DEBUG:teuthology.orchestra.run.smithi107:> 2024-03-08T21:00:19.953 DEBUG:teuthology.orchestra.run.smithi107:> # Given a non-glob path, it better be there 2024-03-08T21:00:19.953 DEBUG:teuthology.orchestra.run.smithi107:> if "*" not in asok_path: 2024-03-08T21:00:19.953 DEBUG:teuthology.orchestra.run.smithi107:> assert(len(files) == 1) 2024-03-08T21:00:19.953 DEBUG:teuthology.orchestra.run.smithi107:> return files[0] 2024-03-08T21:00:19.953 DEBUG:teuthology.orchestra.run.smithi107:> 2024-03-08T21:00:19.953 DEBUG:teuthology.orchestra.run.smithi107:> for f in files: 2024-03-08T21:00:19.954 DEBUG:teuthology.orchestra.run.smithi107:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-03-08T21:00:19.954 DEBUG:teuthology.orchestra.run.smithi107:> if os.path.exists("/proc/{0}".format(pid)): 2024-03-08T21:00:19.954 DEBUG:teuthology.orchestra.run.smithi107:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-03-08T21:00:19.954 DEBUG:teuthology.orchestra.run.smithi107:> contents = proc_f.read() 2024-03-08T21:00:19.954 DEBUG:teuthology.orchestra.run.smithi107:> if mountpoint in contents: 2024-03-08T21:00:19.954 DEBUG:teuthology.orchestra.run.smithi107:> return f 2024-03-08T21:00:19.954 DEBUG:teuthology.orchestra.run.smithi107:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-03-08T21:00:19.954 DEBUG:teuthology.orchestra.run.smithi107:> 2024-03-08T21:00:19.954 DEBUG:teuthology.orchestra.run.smithi107:> print(_find_admin_socket("client.0")) 2024-03-08T21:00:19.954 DEBUG:teuthology.orchestra.run.smithi107:> ' 2024-03-08T21:00:20.129 INFO:teuthology.orchestra.run.smithi107.stdout:/var/run/ceph/ceph-client.0.84486.asok 2024-03-08T21:00:20.134 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-08T21:00:20.147 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.84486.asok 2024-03-08T21:00:20.147 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T21:00:20.147 DEBUG:teuthology.orchestra.run.smithi107:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.84486.asok status 2024-03-08T21:00:20.393 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-03-08T21:00:20.394 INFO:teuthology.orchestra.run.smithi107.stdout: "metadata": { 2024-03-08T21:00:20.394 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph_sha1": "7d63482aa89628d733e298984a762389b625c761", 2024-03-08T21:00:20.394 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph_version": "ceph version 19.0.0-1965-g7d63482a (7d63482aa89628d733e298984a762389b625c761) squid (dev)", 2024-03-08T21:00:20.394 INFO:teuthology.orchestra.run.smithi107.stdout: "entity_id": "0", 2024-03-08T21:00:20.394 INFO:teuthology.orchestra.run.smithi107.stdout: "hostname": "smithi107", 2024-03-08T21:00:20.394 INFO:teuthology.orchestra.run.smithi107.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-03-08T21:00:20.394 INFO:teuthology.orchestra.run.smithi107.stdout: "pid": "84486", 2024-03-08T21:00:20.394 INFO:teuthology.orchestra.run.smithi107.stdout: "root": "/volumes/qa/sv_0/2193ae44-16ef-4c95-bf3c-e3e15e292573" 2024-03-08T21:00:20.394 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:00:20.394 INFO:teuthology.orchestra.run.smithi107.stdout: "dentry_count": 0, 2024-03-08T21:00:20.394 INFO:teuthology.orchestra.run.smithi107.stdout: "dentry_pinned_count": 0, 2024-03-08T21:00:20.394 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 15345, 2024-03-08T21:00:20.395 INFO:teuthology.orchestra.run.smithi107.stdout: "inst": { 2024-03-08T21:00:20.395 INFO:teuthology.orchestra.run.smithi107.stdout: "name": { 2024-03-08T21:00:20.395 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "client", 2024-03-08T21:00:20.395 INFO:teuthology.orchestra.run.smithi107.stdout: "num": 15345 2024-03-08T21:00:20.395 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:00:20.395 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": { 2024-03-08T21:00:20.395 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-03-08T21:00:20.395 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "192.168.0.1:0", 2024-03-08T21:00:20.395 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3517930426 2024-03-08T21:00:20.395 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:00:20.395 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:00:20.395 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": { 2024-03-08T21:00:20.395 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-03-08T21:00:20.395 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "192.168.0.1:0", 2024-03-08T21:00:20.396 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3517930426 2024-03-08T21:00:20.396 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:00:20.396 INFO:teuthology.orchestra.run.smithi107.stdout: "inst_str": "client.15345 192.168.0.1:0/3517930426", 2024-03-08T21:00:20.396 INFO:teuthology.orchestra.run.smithi107.stdout: "addr_str": "192.168.0.1:0/3517930426", 2024-03-08T21:00:20.396 INFO:teuthology.orchestra.run.smithi107.stdout: "inode_count": 5, 2024-03-08T21:00:20.396 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_epoch": 27, 2024-03-08T21:00:20.396 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_epoch": 88, 2024-03-08T21:00:20.396 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_epoch_barrier": 0, 2024-03-08T21:00:20.396 INFO:teuthology.orchestra.run.smithi107.stdout: "blocklisted": false, 2024-03-08T21:00:20.396 INFO:teuthology.orchestra.run.smithi107.stdout: "fs_name": "cephfs" 2024-03-08T21:00:20.396 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-03-08T21:00:20.409 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T21:00:20.409 DEBUG:teuthology.orchestra.run.smithi107:> stat --file-system '--printf=%T 2024-03-08T21:00:20.409 DEBUG:teuthology.orchestra.run.smithi107:> ' -- /home/ubuntu/cephtest/mnt.0 2024-03-08T21:00:20.463 INFO:teuthology.orchestra.run.smithi107.stdout:fuseblk 2024-03-08T21:00:20.463 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-03-08T21:00:20.463 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T21:00:20.463 DEBUG:teuthology.orchestra.run.smithi107:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-03-08T21:00:20.551 INFO:teuthology.run_tasks:Running task exec... 2024-03-08T21:00:20.563 INFO:teuthology.task.exec:Executing custom commands... 2024-03-08T21:00:20.563 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi107.front.sepia.ceph.com 2024-03-08T21:00:20.564 DEBUG:teuthology.orchestra.run.smithi107:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate false' 2024-03-08T21:00:20.772 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:20.786+0000 7f7023000640 1 -- 172.21.15.107:0/3621555743 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f701c103550 msgr2=0x7f701c103970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:00:20.773 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:20.786+0000 7f7023000640 1 --2- 172.21.15.107:0/3621555743 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f701c103550 0x7f701c103970 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f700c009900 tx=0x7f700c02bc30 comp rx=0 tx=0).stop 2024-03-08T21:00:20.773 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:20.786+0000 7f7023000640 1 -- 172.21.15.107:0/3621555743 shutdown_connections 2024-03-08T21:00:20.773 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:20.786+0000 7f7023000640 1 --2- 172.21.15.107:0/3621555743 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f701c105100 0x7f701c10b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:20.773 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:20.786+0000 7f7023000640 1 --2- 172.21.15.107:0/3621555743 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f701c104740 0x7f701c104bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:20.773 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:20.786+0000 7f7023000640 1 --2- 172.21.15.107:0/3621555743 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f701c103550 0x7f701c103970 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:20.773 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:20.786+0000 7f7023000640 1 -- 172.21.15.107:0/3621555743 >> 172.21.15.107:0/3621555743 conn(0x7f701c0feae0 msgr2=0x7f701c100f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:00:20.773 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:20.786+0000 7f7023000640 1 -- 172.21.15.107:0/3621555743 shutdown_connections 2024-03-08T21:00:20.773 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:20.786+0000 7f7023000640 1 -- 172.21.15.107:0/3621555743 wait complete. 2024-03-08T21:00:20.773 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:20.787+0000 7f7023000640 1 Processor -- start 2024-03-08T21:00:20.773 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:20.787+0000 7f7023000640 1 -- start start 2024-03-08T21:00:20.773 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:20.787+0000 7f7023000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f701c103550 0x7f701c1996a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:00:20.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:20.787+0000 7f7023000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f701c104740 0x7f701c199be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:00:20.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:20.788+0000 7f7023000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f701c105100 0x7f701c1a0c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:00:20.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:20.788+0000 7f7023000640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f701c10f120 con 0x7f701c104740 2024-03-08T21:00:20.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:20.788+0000 7f7023000640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f701c10efa0 con 0x7f701c103550 2024-03-08T21:00:20.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:20.788+0000 7f7023000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f701c10f2a0 con 0x7f701c105100 2024-03-08T21:00:20.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:20.788+0000 7f7013400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f701c104740 0x7f701c199be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:00:20.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:20.788+0000 7f7013400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f701c104740 0x7f701c199be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:34024/0 (socket says 172.21.15.107:34024) 2024-03-08T21:00:20.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:20.788+0000 7f7013400640 1 -- 172.21.15.107:0/3704687195 learned_addr learned my addr 172.21.15.107:0/3704687195 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:00:20.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:20.788+0000 7f701be00640 1 --2- 172.21.15.107:0/3704687195 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f701c103550 0x7f701c1996a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:00:20.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:20.788+0000 7f7021000640 1 --2- 172.21.15.107:0/3704687195 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f701c105100 0x7f701c1a0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:00:20.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:20.788+0000 7f7013400640 1 -- 172.21.15.107:0/3704687195 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f701c105100 msgr2=0x7f701c1a0c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:00:20.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:20.788+0000 7f7013400640 1 --2- 172.21.15.107:0/3704687195 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f701c105100 0x7f701c1a0c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:20.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:20.788+0000 7f7013400640 1 -- 172.21.15.107:0/3704687195 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f701c103550 msgr2=0x7f701c1996a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:00:20.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:20.788+0000 7f7013400640 1 --2- 172.21.15.107:0/3704687195 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f701c103550 0x7f701c1996a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:20.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:20.788+0000 7f7013400640 1 -- 172.21.15.107:0/3704687195 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f701c1a12c0 con 0x7f701c104740 2024-03-08T21:00:20.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:20.788+0000 7f7021000640 1 --2- 172.21.15.107:0/3704687195 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f701c105100 0x7f701c1a0c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T21:00:20.776 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:20.789+0000 7f7013400640 1 --2- 172.21.15.107:0/3704687195 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f701c104740 0x7f701c199be0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f700400c920 tx=0x7f700400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:00:20.776 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:20.789+0000 7f7019600640 1 -- 172.21.15.107:0/3704687195 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7004007c70 con 0x7f701c104740 2024-03-08T21:00:20.776 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:20.789+0000 7f7019600640 1 -- 172.21.15.107:0/3704687195 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f7004007e10 con 0x7f701c104740 2024-03-08T21:00:20.776 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:20.789+0000 7f7019600640 1 -- 172.21.15.107:0/3704687195 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7004018820 con 0x7f701c104740 2024-03-08T21:00:20.776 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:20.789+0000 7f7023000640 1 -- 172.21.15.107:0/3704687195 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f701c1a15b0 con 0x7f701c104740 2024-03-08T21:00:20.776 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:20.789+0000 7f7023000640 1 -- 172.21.15.107:0/3704687195 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f701c1a1b30 con 0x7f701c104740 2024-03-08T21:00:20.778 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:20.790+0000 7f7023000640 1 -- 172.21.15.107:0/3704687195 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f701c10a810 con 0x7f701c104740 2024-03-08T21:00:20.778 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:20.792+0000 7f7019600640 1 -- 172.21.15.107:0/3704687195 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f70040189c0 con 0x7f701c104740 2024-03-08T21:00:20.778 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:20.793+0000 7f7019600640 1 --2- 172.21.15.107:0/3704687195 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f70000a2920 0x7f70000a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:00:20.778 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:20.793+0000 7f7019600640 1 -- 172.21.15.107:0/3704687195 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f70040cec20 con 0x7f701c104740 2024-03-08T21:00:20.778 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:20.794+0000 7f701be00640 1 --2- 172.21.15.107:0/3704687195 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f70000a2920 0x7f70000a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:00:20.778 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:20.794+0000 7f701be00640 1 --2- 172.21.15.107:0/3704687195 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f70000a2920 0x7f70000a4de0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f700c02c240 tx=0x7f700c000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:00:20.780 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:20.798+0000 7f7019600640 1 -- 172.21.15.107:0/3704687195 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f7004014030 con 0x7f701c104740 2024-03-08T21:00:21.020 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:21.037+0000 7f7023000640 1 -- 172.21.15.107:0/3704687195 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"} v 0) v1 -- 0x7f701c02ce70 con 0x7f701c104740 2024-03-08T21:00:21.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:21.443+0000 7f7019600640 1 -- 172.21.15.107:0/3704687195 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]=0 v28) v1 ==== 118+0+0 (secure 0 0 0) 0x7f700409bc80 con 0x7f701c104740 2024-03-08T21:00:21.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:21.448+0000 7f7023000640 1 -- 172.21.15.107:0/3704687195 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f70000a2920 msgr2=0x7f70000a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:00:21.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:21.448+0000 7f7023000640 1 --2- 172.21.15.107:0/3704687195 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f70000a2920 0x7f70000a4de0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f700c02c240 tx=0x7f700c000bc0 comp rx=0 tx=0).stop 2024-03-08T21:00:21.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:21.448+0000 7f7023000640 1 -- 172.21.15.107:0/3704687195 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f701c104740 msgr2=0x7f701c199be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:00:21.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:21.448+0000 7f7023000640 1 --2- 172.21.15.107:0/3704687195 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f701c104740 0x7f701c199be0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f700400c920 tx=0x7f700400cde0 comp rx=0 tx=0).stop 2024-03-08T21:00:21.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:21.448+0000 7f7023000640 1 -- 172.21.15.107:0/3704687195 shutdown_connections 2024-03-08T21:00:21.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:21.448+0000 7f7023000640 1 --2- 172.21.15.107:0/3704687195 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f70000a2920 0x7f70000a4de0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:21.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:21.449+0000 7f7023000640 1 --2- 172.21.15.107:0/3704687195 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f701c105100 0x7f701c1a0c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:21.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:21.449+0000 7f7023000640 1 --2- 172.21.15.107:0/3704687195 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f701c104740 0x7f701c199be0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:21.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:21.449+0000 7f7023000640 1 --2- 172.21.15.107:0/3704687195 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f701c103550 0x7f701c1996a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:21.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:21.449+0000 7f7023000640 1 -- 172.21.15.107:0/3704687195 >> 172.21.15.107:0/3704687195 conn(0x7f701c0feae0 msgr2=0x7f701c1079e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:00:21.433 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:21.449+0000 7f7023000640 1 -- 172.21.15.107:0/3704687195 shutdown_connections 2024-03-08T21:00:21.433 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:21.449+0000 7f7023000640 1 -- 172.21.15.107:0/3704687195 wait complete. 2024-03-08T21:00:21.451 DEBUG:teuthology.orchestra.run.smithi107:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs subvolumegroup pin cephfs qa random 0.10' 2024-03-08T21:00:21.659 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:21.672+0000 7f670e400640 1 -- 172.21.15.107:0/1131857687 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6708101440 msgr2=0x7f67081018c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:00:21.659 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:21.672+0000 7f670e400640 1 --2- 172.21.15.107:0/1131857687 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6708101440 0x7f67081018c0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f66f4009900 tx=0x7f66f402bc30 comp rx=0 tx=0).stop 2024-03-08T21:00:21.659 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:21.673+0000 7f670e400640 1 -- 172.21.15.107:0/1131857687 shutdown_connections 2024-03-08T21:00:21.659 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:21.673+0000 7f670e400640 1 --2- 172.21.15.107:0/1131857687 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6708101e00 0x7f6708108690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:21.659 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:21.673+0000 7f670e400640 1 --2- 172.21.15.107:0/1131857687 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6708101440 0x7f67081018c0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:21.659 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:21.673+0000 7f670e400640 1 --2- 172.21.15.107:0/1131857687 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6708100250 0x7f6708100670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:21.659 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:21.673+0000 7f670e400640 1 -- 172.21.15.107:0/1131857687 >> 172.21.15.107:0/1131857687 conn(0x7f67080fb7c0 msgr2=0x7f67080fdc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:00:21.659 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:21.673+0000 7f670e400640 1 -- 172.21.15.107:0/1131857687 shutdown_connections 2024-03-08T21:00:21.660 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:21.673+0000 7f670e400640 1 -- 172.21.15.107:0/1131857687 wait complete. 2024-03-08T21:00:21.660 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:21.674+0000 7f670e400640 1 Processor -- start 2024-03-08T21:00:21.660 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:21.674+0000 7f670e400640 1 -- start start 2024-03-08T21:00:21.660 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:21.674+0000 7f670e400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6708100250 0x7f67081041e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:00:21.660 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:21.674+0000 7f670e400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6708101440 0x7f6708102820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:00:21.660 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:21.674+0000 7f670e400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6708101e00 0x7f6708102d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:00:21.660 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:21.674+0000 7f670e400640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f670810bde0 con 0x7f6708101e00 2024-03-08T21:00:21.661 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:21.674+0000 7f670e400640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f670810bc60 con 0x7f6708100250 2024-03-08T21:00:21.661 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:21.674+0000 7f670e400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f670810bf60 con 0x7f6708101440 2024-03-08T21:00:21.661 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:21.674+0000 7f6707e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6708101e00 0x7f6708102d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:00:21.661 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:21.674+0000 7f6707e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6708101e00 0x7f6708102d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:34036/0 (socket says 172.21.15.107:34036) 2024-03-08T21:00:21.661 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:21.674+0000 7f6707e00640 1 -- 172.21.15.107:0/1998191107 learned_addr learned my addr 172.21.15.107:0/1998191107 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:00:21.661 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:21.675+0000 7f6706a00640 1 --2- 172.21.15.107:0/1998191107 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6708101440 0x7f6708102820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:00:21.661 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:21.675+0000 7f6707400640 1 --2- 172.21.15.107:0/1998191107 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6708100250 0x7f67081041e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:00:21.661 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:21.675+0000 7f6707e00640 1 -- 172.21.15.107:0/1998191107 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6708101440 msgr2=0x7f6708102820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:00:21.662 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:21.675+0000 7f6707e00640 1 --2- 172.21.15.107:0/1998191107 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6708101440 0x7f6708102820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:21.662 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:21.675+0000 7f6707e00640 1 -- 172.21.15.107:0/1998191107 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6708100250 msgr2=0x7f67081041e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:00:21.662 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:21.675+0000 7f6707e00640 1 --2- 172.21.15.107:0/1998191107 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6708100250 0x7f67081041e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:21.662 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:21.675+0000 7f6707e00640 1 -- 172.21.15.107:0/1998191107 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67081035f0 con 0x7f6708101e00 2024-03-08T21:00:21.662 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:21.675+0000 7f6706a00640 1 --2- 172.21.15.107:0/1998191107 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6708101440 0x7f6708102820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T21:00:21.662 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:21.675+0000 7f6707400640 1 --2- 172.21.15.107:0/1998191107 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6708100250 0x7f67081041e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T21:00:21.662 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:21.675+0000 7f6707e00640 1 --2- 172.21.15.107:0/1998191107 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6708101e00 0x7f6708102d60 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f670000cad0 tx=0x7f670000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:00:21.663 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:21.675+0000 7f66ffe00640 1 -- 172.21.15.107:0/1998191107 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f670000ae20 con 0x7f6708101e00 2024-03-08T21:00:21.663 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:21.676+0000 7f66ffe00640 1 -- 172.21.15.107:0/1998191107 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f6700005020 con 0x7f6708101e00 2024-03-08T21:00:21.663 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:21.676+0000 7f670e400640 1 -- 172.21.15.107:0/1998191107 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67081a1610 con 0x7f6708101e00 2024-03-08T21:00:21.663 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:21.676+0000 7f66ffe00640 1 -- 172.21.15.107:0/1998191107 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f670000f6b0 con 0x7f6708101e00 2024-03-08T21:00:21.663 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:21.676+0000 7f670e400640 1 -- 172.21.15.107:0/1998191107 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67081a1bb0 con 0x7f6708101e00 2024-03-08T21:00:21.664 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:21.677+0000 7f670e400640 1 -- 172.21.15.107:0/1998191107 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67081a1ed0 con 0x7f6708101e00 2024-03-08T21:00:21.665 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:21.679+0000 7f66ffe00640 1 -- 172.21.15.107:0/1998191107 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) v1 ==== 137291+0+0 (secure 0 0 0) 0x7f67000051d0 con 0x7f6708101e00 2024-03-08T21:00:21.665 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:21.680+0000 7f66ffe00640 1 --2- 172.21.15.107:0/1998191107 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f66ec0a2920 0x7f66ec0a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:00:21.665 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:21.680+0000 7f66ffe00640 1 -- 172.21.15.107:0/1998191107 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f67000ceb90 con 0x7f6708101e00 2024-03-08T21:00:21.665 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:21.680+0000 7f6707400640 1 --2- 172.21.15.107:0/1998191107 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f66ec0a2920 0x7f66ec0a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:00:21.665 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:21.681+0000 7f6707400640 1 --2- 172.21.15.107:0/1998191107 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f66ec0a2920 0x7f66ec0a4de0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f67081012a0 tx=0x7f66f8005d30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:00:21.667 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:21.684+0000 7f66ffe00640 1 -- 172.21.15.107:0/1998191107 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f6700014030 con 0x7f6708101e00 2024-03-08T21:00:21.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:00:21 smithi107 ceph-mon[31381]: pgmap v438: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 255 B/s wr, 9 op/s 2024-03-08T21:00:21.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:00:21 smithi107 ceph-mon[31381]: from='client.? 172.21.15.107:0/3704687195' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-03-08T21:00:21.764 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:00:21 smithi149 ceph-mon[35042]: pgmap v438: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 255 B/s wr, 9 op/s 2024-03-08T21:00:21.765 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:00:21 smithi149 ceph-mon[35042]: from='client.? 172.21.15.107:0/3704687195' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-03-08T21:00:21.878 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:00:21 smithi204 ceph-mon[34935]: pgmap v438: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 255 B/s wr, 9 op/s 2024-03-08T21:00:21.878 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:00:21 smithi204 ceph-mon[34935]: from='client.? 172.21.15.107:0/3704687195' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-03-08T21:00:21.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:21.917+0000 7f670e400640 1 -- 172.21.15.107:0/1998191107 --> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}) v1 -- 0x7f67080fd9f0 con 0x7f66ec0a2920 2024-03-08T21:00:21.929 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:21.946+0000 7f66ffe00640 1 -- 172.21.15.107:0/1998191107 <== mgr.14152 v2:172.21.15.107:6800/3376636618 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3 (secure 0 0 0) 0x7f67080fd9f0 con 0x7f66ec0a2920 2024-03-08T21:00:21.930 INFO:teuthology.orchestra.run.smithi107.stdout:{} 2024-03-08T21:00:21.931 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:21.949+0000 7f670e400640 1 -- 172.21.15.107:0/1998191107 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f66ec0a2920 msgr2=0x7f66ec0a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:00:21.932 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:21.949+0000 7f670e400640 1 --2- 172.21.15.107:0/1998191107 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f66ec0a2920 0x7f66ec0a4de0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f67081012a0 tx=0x7f66f8005d30 comp rx=0 tx=0).stop 2024-03-08T21:00:21.932 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:21.949+0000 7f670e400640 1 -- 172.21.15.107:0/1998191107 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6708101e00 msgr2=0x7f6708102d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:00:21.932 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:21.949+0000 7f670e400640 1 --2- 172.21.15.107:0/1998191107 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6708101e00 0x7f6708102d60 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f670000cad0 tx=0x7f670000cf90 comp rx=0 tx=0).stop 2024-03-08T21:00:21.932 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:21.949+0000 7f670e400640 1 -- 172.21.15.107:0/1998191107 shutdown_connections 2024-03-08T21:00:21.932 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:21.949+0000 7f670e400640 1 --2- 172.21.15.107:0/1998191107 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f66ec0a2920 0x7f66ec0a4de0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:21.933 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:21.949+0000 7f670e400640 1 --2- 172.21.15.107:0/1998191107 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6708101e00 0x7f6708102d60 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:21.933 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:21.949+0000 7f670e400640 1 --2- 172.21.15.107:0/1998191107 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6708101440 0x7f6708102820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:21.933 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:21.949+0000 7f670e400640 1 --2- 172.21.15.107:0/1998191107 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6708100250 0x7f67081041e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:21.933 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:21.949+0000 7f670e400640 1 -- 172.21.15.107:0/1998191107 >> 172.21.15.107:0/1998191107 conn(0x7f67080fb7c0 msgr2=0x7f67080fd240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:00:21.933 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:21.949+0000 7f670e400640 1 -- 172.21.15.107:0/1998191107 shutdown_connections 2024-03-08T21:00:21.933 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:21.949+0000 7f670e400640 1 -- 172.21.15.107:0/1998191107 wait complete. 2024-03-08T21:00:21.949 INFO:teuthology.run_tasks:Running task check-counter... 2024-03-08T21:00:21.963 INFO:teuthology.run_tasks:Running task exec... 2024-03-08T21:00:21.989 INFO:teuthology.task.exec:Executing custom commands... 2024-03-08T21:00:21.989 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi107.front.sepia.ceph.com 2024-03-08T21:00:21.989 DEBUG:teuthology.orchestra.run.smithi107:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph mgr module enable snap_schedule' 2024-03-08T21:00:22.182 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:22.199+0000 7fc222400640 1 -- 172.21.15.107:0/2472011885 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc21c107010 msgr2=0x7fc21c109460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:00:22.182 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:22.199+0000 7fc222400640 1 --2- 172.21.15.107:0/2472011885 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc21c107010 0x7fc21c109460 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fc214009ca0 tx=0x7fc21402c0f0 comp rx=0 tx=0).stop 2024-03-08T21:00:22.183 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:22.200+0000 7fc222400640 1 -- 172.21.15.107:0/2472011885 shutdown_connections 2024-03-08T21:00:22.183 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:22.200+0000 7fc222400640 1 --2- 172.21.15.107:0/2472011885 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc21c107010 0x7fc21c109460 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:22.183 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:22.200+0000 7fc222400640 1 --2- 172.21.15.107:0/2472011885 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc21c1046e0 0x7fc21c106ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:22.183 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:22.200+0000 7fc222400640 1 --2- 172.21.15.107:0/2472011885 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc21c101db0 0x7fc21c1041a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:22.183 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:22.200+0000 7fc222400640 1 -- 172.21.15.107:0/2472011885 >> 172.21.15.107:0/2472011885 conn(0x7fc21c0fb7a0 msgr2=0x7fc21c0fdbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:00:22.184 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:22.200+0000 7fc222400640 1 -- 172.21.15.107:0/2472011885 shutdown_connections 2024-03-08T21:00:22.184 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:22.200+0000 7fc222400640 1 -- 172.21.15.107:0/2472011885 wait complete. 2024-03-08T21:00:22.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:22.201+0000 7fc222400640 1 Processor -- start 2024-03-08T21:00:22.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:22.201+0000 7fc222400640 1 -- start start 2024-03-08T21:00:22.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:22.201+0000 7fc222400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc21c101db0 0x7fc21c101660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:00:22.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:22.201+0000 7fc222400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc21c1046e0 0x7fc21c0ffd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:00:22.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:22.201+0000 7fc222400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc21c107010 0x7fc21c1002a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:00:22.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:22.201+0000 7fc222400640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc21c10ce70 con 0x7fc21c1046e0 2024-03-08T21:00:22.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:22.201+0000 7fc222400640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc21c10ccf0 con 0x7fc21c101db0 2024-03-08T21:00:22.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:22.201+0000 7fc222400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc21c10cff0 con 0x7fc21c107010 2024-03-08T21:00:22.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:22.202+0000 7fc21aa00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc21c1046e0 0x7fc21c0ffd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:00:22.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:22.202+0000 7fc21aa00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc21c1046e0 0x7fc21c0ffd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:34068/0 (socket says 172.21.15.107:34068) 2024-03-08T21:00:22.187 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:22.202+0000 7fc21aa00640 1 -- 172.21.15.107:0/1782390817 learned_addr learned my addr 172.21.15.107:0/1782390817 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:00:22.187 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:22.202+0000 7fc21b400640 1 --2- 172.21.15.107:0/1782390817 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc21c101db0 0x7fc21c101660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:00:22.187 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:22.202+0000 7fc21be00640 1 --2- 172.21.15.107:0/1782390817 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc21c107010 0x7fc21c1002a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:00:22.187 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:22.202+0000 7fc21aa00640 1 -- 172.21.15.107:0/1782390817 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc21c107010 msgr2=0x7fc21c1002a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:00:22.187 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:22.202+0000 7fc21aa00640 1 --2- 172.21.15.107:0/1782390817 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc21c107010 0x7fc21c1002a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:22.187 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:22.202+0000 7fc21aa00640 1 -- 172.21.15.107:0/1782390817 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc21c101db0 msgr2=0x7fc21c101660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:00:22.187 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:22.202+0000 7fc21aa00640 1 --2- 172.21.15.107:0/1782390817 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc21c101db0 0x7fc21c101660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:22.187 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:22.202+0000 7fc21aa00640 1 -- 172.21.15.107:0/1782390817 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc21c100a70 con 0x7fc21c1046e0 2024-03-08T21:00:22.187 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:22.202+0000 7fc21be00640 1 --2- 172.21.15.107:0/1782390817 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc21c107010 0x7fc21c1002a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T21:00:22.187 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:22.202+0000 7fc21aa00640 1 --2- 172.21.15.107:0/1782390817 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc21c1046e0 0x7fc21c0ffd60 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fc20800e9c0 tx=0x7fc20800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:00:22.188 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:22.203+0000 7fc213e00640 1 -- 172.21.15.107:0/1782390817 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc20800cd80 con 0x7fc21c1046e0 2024-03-08T21:00:22.188 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:22.203+0000 7fc213e00640 1 -- 172.21.15.107:0/1782390817 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fc2080054d0 con 0x7fc21c1046e0 2024-03-08T21:00:22.188 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:22.203+0000 7fc213e00640 1 -- 172.21.15.107:0/1782390817 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc208010430 con 0x7fc21c1046e0 2024-03-08T21:00:22.188 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:22.203+0000 7fc222400640 1 -- 172.21.15.107:0/1782390817 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc21c1a1550 con 0x7fc21c1046e0 2024-03-08T21:00:22.188 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:22.203+0000 7fc222400640 1 -- 172.21.15.107:0/1782390817 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc21c1a1af0 con 0x7fc21c1046e0 2024-03-08T21:00:22.188 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:22.205+0000 7fc222400640 1 -- 172.21.15.107:0/1782390817 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc21c0fd290 con 0x7fc21c1046e0 2024-03-08T21:00:22.190 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:22.207+0000 7fc213e00640 1 -- 172.21.15.107:0/1782390817 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) v1 ==== 137291+0+0 (secure 0 0 0) 0x7fc208004fd0 con 0x7fc21c1046e0 2024-03-08T21:00:22.192 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:22.209+0000 7fc213e00640 1 --2- 172.21.15.107:0/1782390817 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fc2000a2920 0x7fc2000a4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:00:22.192 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:22.209+0000 7fc213e00640 1 -- 172.21.15.107:0/1782390817 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fc2080ceec0 con 0x7fc21c1046e0 2024-03-08T21:00:22.192 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:22.209+0000 7fc21b400640 1 --2- 172.21.15.107:0/1782390817 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fc2000a2920 0x7fc2000a4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:00:22.193 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:22.210+0000 7fc21b400640 1 --2- 172.21.15.107:0/1782390817 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fc2000a2920 0x7fc2000a4de0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fc20c009bd0 tx=0x7fc20c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:00:22.197 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:22.214+0000 7fc213e00640 1 -- 172.21.15.107:0/1782390817 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7fc20809bf20 con 0x7fc21c1046e0 2024-03-08T21:00:22.216 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 08 21:00:21 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-x[31606]: 2024-03-08T21:00:21.924+0000 7f8c10800700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-08T21:00:22.216 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 08 21:00:21 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-x[31606]: 2024-03-08T21:00:21.924+0000 7f8c10800700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-08T21:00:22.217 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 08 21:00:21 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-x[31606]: 2024-03-08T21:00:21.924+0000 7f8c10800700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-08T21:00:22.217 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 08 21:00:21 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-x[31606]: 2024-03-08T21:00:21.924+0000 7f8c10800700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-08T21:00:22.217 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 08 21:00:21 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-x[31606]: 2024-03-08T21:00:21.924+0000 7f8c10800700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-08T21:00:22.456 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:22.473+0000 7fc222400640 1 -- 172.21.15.107:0/1782390817 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7fc21c061ef0 con 0x7fc21c1046e0 2024-03-08T21:00:22.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:00:22 smithi107 ceph-mon[31381]: from='client.? 172.21.15.107:0/3704687195' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-03-08T21:00:22.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:00:22 smithi107 ceph-mon[31381]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-08T21:00:22.717 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:00:22 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T21:00:22.764 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:00:22 smithi149 ceph-mon[35042]: from='client.? 172.21.15.107:0/3704687195' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-03-08T21:00:22.764 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:00:22 smithi149 ceph-mon[35042]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-08T21:00:22.764 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:00:22 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T21:00:22.877 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:00:22 smithi204 ceph-mon[34935]: from='client.? 172.21.15.107:0/3704687195' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-03-08T21:00:22.877 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:00:22 smithi204 ceph-mon[34935]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-08T21:00:22.877 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:00:22 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T21:00:23.457 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:23.473+0000 7fc213e00640 1 -- 172.21.15.107:0/1782390817 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v19) v1 ==== 92+0+0 (secure 0 0 0) 0x7fc20809b820 con 0x7fc21c1046e0 2024-03-08T21:00:23.460 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:23.477+0000 7fc222400640 1 -- 172.21.15.107:0/1782390817 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fc2000a2920 msgr2=0x7fc2000a4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:00:23.460 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:23.477+0000 7fc222400640 1 --2- 172.21.15.107:0/1782390817 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fc2000a2920 0x7fc2000a4de0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fc20c009bd0 tx=0x7fc20c005f90 comp rx=0 tx=0).stop 2024-03-08T21:00:23.460 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:23.477+0000 7fc222400640 1 -- 172.21.15.107:0/1782390817 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc21c1046e0 msgr2=0x7fc21c0ffd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:00:23.460 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:23.477+0000 7fc222400640 1 --2- 172.21.15.107:0/1782390817 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc21c1046e0 0x7fc21c0ffd60 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fc20800e9c0 tx=0x7fc20800ee80 comp rx=0 tx=0).stop 2024-03-08T21:00:23.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:23.477+0000 7fc222400640 1 -- 172.21.15.107:0/1782390817 shutdown_connections 2024-03-08T21:00:23.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:23.477+0000 7fc222400640 1 --2- 172.21.15.107:0/1782390817 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fc2000a2920 0x7fc2000a4de0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:23.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:23.477+0000 7fc222400640 1 --2- 172.21.15.107:0/1782390817 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc21c107010 0x7fc21c1002a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:23.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:23.477+0000 7fc222400640 1 --2- 172.21.15.107:0/1782390817 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc21c1046e0 0x7fc21c0ffd60 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:23.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:23.477+0000 7fc222400640 1 --2- 172.21.15.107:0/1782390817 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc21c101db0 0x7fc21c101660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:23.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:23.477+0000 7fc222400640 1 -- 172.21.15.107:0/1782390817 >> 172.21.15.107:0/1782390817 conn(0x7fc21c0fb7a0 msgr2=0x7fc21c102750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:00:23.462 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:23.478+0000 7fc222400640 1 -- 172.21.15.107:0/1782390817 shutdown_connections 2024-03-08T21:00:23.462 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:23.478+0000 7fc222400640 1 -- 172.21.15.107:0/1782390817 wait complete. 2024-03-08T21:00:23.487 DEBUG:teuthology.orchestra.run.smithi107:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/allow_m_granularity true' 2024-03-08T21:00:23.666 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:23.683+0000 7f94ab800640 1 -- 172.21.15.107:0/423536195 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f94a41070b0 msgr2=0x7f94a41094a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:00:23.666 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:23.683+0000 7f94ab800640 1 --2- 172.21.15.107:0/423536195 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f94a41070b0 0x7f94a41094a0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f949c009ca0 tx=0x7f949c02c0f0 comp rx=0 tx=0).stop 2024-03-08T21:00:23.666 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:23.683+0000 7f94ab800640 1 -- 172.21.15.107:0/423536195 shutdown_connections 2024-03-08T21:00:23.667 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:23.683+0000 7f94ab800640 1 --2- 172.21.15.107:0/423536195 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f94a41070b0 0x7f94a41094a0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:23.667 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:23.683+0000 7f94ab800640 1 --2- 172.21.15.107:0/423536195 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f94a4104780 0x7f94a4106b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:23.667 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:23.683+0000 7f94ab800640 1 --2- 172.21.15.107:0/423536195 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f94a4101e50 0x7f94a4104240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:23.667 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:23.683+0000 7f94ab800640 1 -- 172.21.15.107:0/423536195 >> 172.21.15.107:0/423536195 conn(0x7f94a40fb780 msgr2=0x7f94a40fdbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:00:23.667 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:23.683+0000 7f94ab800640 1 -- 172.21.15.107:0/423536195 shutdown_connections 2024-03-08T21:00:23.667 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:23.683+0000 7f94ab800640 1 -- 172.21.15.107:0/423536195 wait complete. 2024-03-08T21:00:23.670 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:23.684+0000 7f94ab800640 1 Processor -- start 2024-03-08T21:00:23.670 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:23.684+0000 7f94ab800640 1 -- start start 2024-03-08T21:00:23.670 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:23.684+0000 7f94ab800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f94a4101e50 0x7f94a4197460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:00:23.670 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:23.684+0000 7f94ab800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f94a4104780 0x7f94a41979a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:00:23.670 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:23.684+0000 7f94ab800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f94a41070b0 0x7f94a419ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:00:23.670 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:23.684+0000 7f94ab800640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94a410ce60 con 0x7f94a4104780 2024-03-08T21:00:23.670 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:23.684+0000 7f94ab800640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94a410cce0 con 0x7f94a41070b0 2024-03-08T21:00:23.670 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:23.684+0000 7f94ab800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94a410cfe0 con 0x7f94a4101e50 2024-03-08T21:00:23.671 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:23.685+0000 7f94a3e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f94a4104780 0x7f94a41979a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:00:23.671 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:23.685+0000 7f94a3e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f94a4104780 0x7f94a41979a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:34102/0 (socket says 172.21.15.107:34102) 2024-03-08T21:00:23.671 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:23.685+0000 7f94a3e00640 1 -- 172.21.15.107:0/3902874278 learned_addr learned my addr 172.21.15.107:0/3902874278 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:00:23.671 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:23.685+0000 7f94a9800640 1 --2- 172.21.15.107:0/3902874278 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f94a41070b0 0x7f94a419ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:00:23.671 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:23.685+0000 7f94a8e00640 1 --2- 172.21.15.107:0/3902874278 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f94a4101e50 0x7f94a4197460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:00:23.671 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:23.685+0000 7f94a3e00640 1 -- 172.21.15.107:0/3902874278 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f94a4101e50 msgr2=0x7f94a4197460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:00:23.671 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:23.685+0000 7f94a3e00640 1 --2- 172.21.15.107:0/3902874278 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f94a4101e50 0x7f94a4197460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:23.671 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:23.685+0000 7f94a3e00640 1 -- 172.21.15.107:0/3902874278 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f94a41070b0 msgr2=0x7f94a419ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:00:23.671 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:23.685+0000 7f94a3e00640 1 --2- 172.21.15.107:0/3902874278 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f94a41070b0 0x7f94a419ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:23.671 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:23.685+0000 7f94a3e00640 1 -- 172.21.15.107:0/3902874278 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94a419f080 con 0x7f94a4104780 2024-03-08T21:00:23.671 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:23.685+0000 7f94a9800640 1 --2- 172.21.15.107:0/3902874278 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f94a41070b0 0x7f94a419ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T21:00:23.671 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:23.685+0000 7f94a3e00640 1 --2- 172.21.15.107:0/3902874278 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f94a4104780 0x7f94a41979a0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f949000e9c0 tx=0x7f949000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:00:23.672 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:23.686+0000 7f94a1600640 1 -- 172.21.15.107:0/3902874278 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f949000cd80 con 0x7f94a4104780 2024-03-08T21:00:23.672 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:23.686+0000 7f94a1600640 1 -- 172.21.15.107:0/3902874278 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f94900054d0 con 0x7f94a4104780 2024-03-08T21:00:23.672 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:23.686+0000 7f94ab800640 1 -- 172.21.15.107:0/3902874278 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94a419f370 con 0x7f94a4104780 2024-03-08T21:00:23.672 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:23.686+0000 7f94a1600640 1 -- 172.21.15.107:0/3902874278 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9490010430 con 0x7f94a4104780 2024-03-08T21:00:23.672 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:23.686+0000 7f94ab800640 1 -- 172.21.15.107:0/3902874278 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94a419f8c0 con 0x7f94a4104780 2024-03-08T21:00:23.673 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:23.687+0000 7f94ab800640 1 -- 172.21.15.107:0/3902874278 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94a40fd270 con 0x7f94a4104780 2024-03-08T21:00:23.673 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:23.689+0000 7f94a1600640 1 -- 172.21.15.107:0/3902874278 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) v1 ==== 137359+0+0 (secure 0 0 0) 0x7f9490004fd0 con 0x7f94a4104780 2024-03-08T21:00:23.673 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:23.690+0000 7f94a1600640 1 --2- 172.21.15.107:0/3902874278 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f948c0a2990 0x7f948c0a4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:00:23.674 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:23.690+0000 7f94a8e00640 1 -- 172.21.15.107:0/3902874278 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f948c0a2990 msgr2=0x7f948c0a4e50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.107:6800/3376636618 2024-03-08T21:00:23.674 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:23.690+0000 7f94a1600640 1 -- 172.21.15.107:0/3902874278 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f94900cf060 con 0x7f94a4104780 2024-03-08T21:00:23.674 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:23.690+0000 7f94a8e00640 1 --2- 172.21.15.107:0/3902874278 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f948c0a2990 0x7f948c0a4e50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-08T21:00:23.682 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:23.699+0000 7f94a1600640 1 -- 172.21.15.107:0/3902874278 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f949009bf60 con 0x7f94a4104780 2024-03-08T21:00:23.717 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:00:23 smithi107 ceph-mon[31381]: from='client.15357 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-03-08T21:00:23.717 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:00:23 smithi107 ceph-mon[31381]: pgmap v439: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 597 B/s wr, 10 op/s 2024-03-08T21:00:23.717 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:00:23 smithi107 ceph-mon[31381]: from='client.? 172.21.15.107:0/1782390817' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-03-08T21:00:23.717 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:00:23 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T21:00:23.717 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:00:23 smithi107 ceph-mon[31381]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T21:00:23.717 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 08 21:00:23 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-x[31606]: ignoring --setuser ceph since I am not root 2024-03-08T21:00:23.717 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 08 21:00:23 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-x[31606]: ignoring --setgroup ceph since I am not root 2024-03-08T21:00:23.765 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:00:23 smithi149 ceph-mon[35042]: from='client.15357 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-03-08T21:00:23.765 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:00:23 smithi149 ceph-mon[35042]: pgmap v439: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 597 B/s wr, 10 op/s 2024-03-08T21:00:23.765 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:00:23 smithi149 ceph-mon[35042]: from='client.? 172.21.15.107:0/1782390817' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-03-08T21:00:23.765 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:00:23 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T21:00:23.765 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:00:23 smithi149 ceph-mon[35042]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T21:00:23.765 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Mar 08 21:00:23 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-y[36424]: ignoring --setuser ceph since I am not root 2024-03-08T21:00:23.765 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Mar 08 21:00:23 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-y[36424]: ignoring --setgroup ceph since I am not root 2024-03-08T21:00:23.766 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Mar 08 21:00:23 smithi149 ceph-mgr[36428]: -- 172.21.15.149:0/1044825058 <== mon.2 v2:172.21.15.149:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5575368c65a0 con 0x55753689a800 2024-03-08T21:00:23.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:23.891+0000 7f94a8e00640 1 -- 172.21.15.107:0/3902874278 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f948c0a2990 msgr2=0x7f948c0a4e50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.107:6800/3376636618 2024-03-08T21:00:23.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:23.891+0000 7f94a8e00640 1 --2- 172.21.15.107:0/3902874278 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f948c0a2990 0x7f948c0a4e50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-08T21:00:23.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:23.892+0000 7f94ab800640 1 -- 172.21.15.107:0/3902874278 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7f94a402ce70 con 0x7f94a4104780 2024-03-08T21:00:23.877 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:00:23 smithi204 ceph-mon[34935]: from='client.15357 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-03-08T21:00:23.877 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:00:23 smithi204 ceph-mon[34935]: pgmap v439: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 597 B/s wr, 10 op/s 2024-03-08T21:00:23.878 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:00:23 smithi204 ceph-mon[34935]: from='client.? 172.21.15.107:0/1782390817' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-03-08T21:00:23.878 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:00:23 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T21:00:23.878 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:00:23 smithi204 ceph-mon[34935]: from='mgr.14152 172.21.15.107:0/4222642378' entity='mgr.x' 2024-03-08T21:00:23.878 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Mar 08 21:00:23 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-z[36870]: ignoring --setuser ceph since I am not root 2024-03-08T21:00:23.878 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Mar 08 21:00:23 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-z[36870]: ignoring --setgroup ceph since I am not root 2024-03-08T21:00:23.878 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Mar 08 21:00:23 smithi204 ceph-mgr[36896]: -- 172.21.15.204:0/3689400728 <== mon.1 v2:172.21.15.204:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55ff2c5025a0 con 0x55ff2c4d6800 2024-03-08T21:00:23.880 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:23.897+0000 7f94a1600640 1 -- 172.21.15.107:0/3902874278 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v31)=0 v31) v1 ==== 138+0+0 (secure 0 0 0) 0x7f949009b860 con 0x7f94a4104780 2024-03-08T21:00:23.883 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:23.900+0000 7f94ab800640 1 -- 172.21.15.107:0/3902874278 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f948c0a2990 msgr2=0x7f948c0a4e50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-08T21:00:23.883 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:23.900+0000 7f94ab800640 1 --2- 172.21.15.107:0/3902874278 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f948c0a2990 0x7f948c0a4e50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:23.883 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:23.900+0000 7f94ab800640 1 -- 172.21.15.107:0/3902874278 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f94a4104780 msgr2=0x7f94a41979a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:00:23.883 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:23.900+0000 7f94ab800640 1 --2- 172.21.15.107:0/3902874278 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f94a4104780 0x7f94a41979a0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f949000e9c0 tx=0x7f949000ee80 comp rx=0 tx=0).stop 2024-03-08T21:00:23.884 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:23.900+0000 7f94ab800640 1 -- 172.21.15.107:0/3902874278 shutdown_connections 2024-03-08T21:00:23.884 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:23.900+0000 7f94ab800640 1 --2- 172.21.15.107:0/3902874278 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7f948c0a2990 0x7f948c0a4e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:23.884 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:23.900+0000 7f94ab800640 1 --2- 172.21.15.107:0/3902874278 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f94a41070b0 0x7f94a419ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:23.884 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:23.900+0000 7f94ab800640 1 --2- 172.21.15.107:0/3902874278 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f94a4104780 0x7f94a41979a0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:23.884 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:23.900+0000 7f94ab800640 1 --2- 172.21.15.107:0/3902874278 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f94a4101e50 0x7f94a4197460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:23.884 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:23.900+0000 7f94ab800640 1 -- 172.21.15.107:0/3902874278 >> 172.21.15.107:0/3902874278 conn(0x7f94a40fb780 msgr2=0x7f94a4102790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:00:23.884 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:23.900+0000 7f94ab800640 1 -- 172.21.15.107:0/3902874278 shutdown_connections 2024-03-08T21:00:23.884 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:23.900+0000 7f94ab800640 1 -- 172.21.15.107:0/3902874278 wait complete. 2024-03-08T21:00:23.902 DEBUG:teuthology.orchestra.run.smithi107:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/dump_on_update true' 2024-03-08T21:00:24.077 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:24.094+0000 7fee71c00640 1 -- 172.21.15.107:0/2470133504 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fee6c105100 msgr2=0x7fee6c10b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:00:24.078 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:24.094+0000 7fee71c00640 1 --2- 172.21.15.107:0/2470133504 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fee6c105100 0x7fee6c10b990 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fee64009ca0 tx=0x7fee6402c0f0 comp rx=0 tx=0).stop 2024-03-08T21:00:24.078 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:24.094+0000 7fee71c00640 1 -- 172.21.15.107:0/2470133504 shutdown_connections 2024-03-08T21:00:24.078 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:24.094+0000 7fee71c00640 1 --2- 172.21.15.107:0/2470133504 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fee6c105100 0x7fee6c10b990 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:24.078 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:24.094+0000 7fee71c00640 1 --2- 172.21.15.107:0/2470133504 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fee6c104740 0x7fee6c104bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:24.078 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:24.094+0000 7fee71c00640 1 --2- 172.21.15.107:0/2470133504 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fee6c103550 0x7fee6c103970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:24.078 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:24.094+0000 7fee71c00640 1 -- 172.21.15.107:0/2470133504 >> 172.21.15.107:0/2470133504 conn(0x7fee6c0feae0 msgr2=0x7fee6c100f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:00:24.078 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:24.094+0000 7fee71c00640 1 -- 172.21.15.107:0/2470133504 shutdown_connections 2024-03-08T21:00:24.078 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:24.094+0000 7fee71c00640 1 -- 172.21.15.107:0/2470133504 wait complete. 2024-03-08T21:00:24.080 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:24.095+0000 7fee71c00640 1 Processor -- start 2024-03-08T21:00:24.080 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:24.095+0000 7fee71c00640 1 -- start start 2024-03-08T21:00:24.081 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:24.096+0000 7fee71c00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fee6c103550 0x7fee6c199680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:00:24.081 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:24.096+0000 7fee71c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fee6c104740 0x7fee6c199bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:00:24.081 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:24.096+0000 7fee71c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fee6c105100 0x7fee6c1a0c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:00:24.081 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:24.096+0000 7fee71c00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee6c10f090 con 0x7fee6c105100 2024-03-08T21:00:24.081 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:24.096+0000 7fee71c00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee6c10ef10 con 0x7fee6c103550 2024-03-08T21:00:24.081 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:24.096+0000 7fee71c00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee6c10f210 con 0x7fee6c104740 2024-03-08T21:00:24.081 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:24.096+0000 7fee6b400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fee6c105100 0x7fee6c1a0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:00:24.081 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:24.096+0000 7fee6b400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fee6c105100 0x7fee6c1a0c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:34116/0 (socket says 172.21.15.107:34116) 2024-03-08T21:00:24.081 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:24.096+0000 7fee6b400640 1 -- 172.21.15.107:0/3957829241 learned_addr learned my addr 172.21.15.107:0/3957829241 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:00:24.081 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:24.096+0000 7fee6a000640 1 --2- 172.21.15.107:0/3957829241 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fee6c104740 0x7fee6c199bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:00:24.081 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:24.096+0000 7fee6aa00640 1 --2- 172.21.15.107:0/3957829241 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fee6c103550 0x7fee6c199680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:00:24.081 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:24.096+0000 7fee6b400640 1 -- 172.21.15.107:0/3957829241 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fee6c104740 msgr2=0x7fee6c199bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:00:24.082 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:24.096+0000 7fee6b400640 1 --2- 172.21.15.107:0/3957829241 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fee6c104740 0x7fee6c199bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:24.082 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:24.096+0000 7fee6b400640 1 -- 172.21.15.107:0/3957829241 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fee6c103550 msgr2=0x7fee6c199680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:00:24.082 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:24.096+0000 7fee6b400640 1 --2- 172.21.15.107:0/3957829241 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fee6c103550 0x7fee6c199680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:24.082 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:24.096+0000 7fee6b400640 1 -- 172.21.15.107:0/3957829241 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee6c1a12a0 con 0x7fee6c105100 2024-03-08T21:00:24.082 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:24.096+0000 7fee6aa00640 1 --2- 172.21.15.107:0/3957829241 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fee6c103550 0x7fee6c199680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T21:00:24.082 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:24.097+0000 7fee6b400640 1 --2- 172.21.15.107:0/3957829241 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fee6c105100 0x7fee6c1a0c30 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fee64032380 tx=0x7fee64032820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:00:24.082 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:24.097+0000 7fee63400640 1 -- 172.21.15.107:0/3957829241 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee64041020 con 0x7fee6c105100 2024-03-08T21:00:24.082 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:24.097+0000 7fee71c00640 1 -- 172.21.15.107:0/3957829241 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee6c1a1530 con 0x7fee6c105100 2024-03-08T21:00:24.082 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:24.097+0000 7fee63400640 1 -- 172.21.15.107:0/3957829241 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fee64003e20 con 0x7fee6c105100 2024-03-08T21:00:24.082 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:24.097+0000 7fee63400640 1 -- 172.21.15.107:0/3957829241 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee64039040 con 0x7fee6c105100 2024-03-08T21:00:24.082 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:24.097+0000 7fee71c00640 1 -- 172.21.15.107:0/3957829241 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee6c1a1a20 con 0x7fee6c105100 2024-03-08T21:00:24.083 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:24.098+0000 7fee71c00640 1 -- 172.21.15.107:0/3957829241 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee6c10a810 con 0x7fee6c105100 2024-03-08T21:00:24.083 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:24.100+0000 7fee63400640 1 -- 172.21.15.107:0/3957829241 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) v1 ==== 137359+0+0 (secure 0 0 0) 0x7fee64044020 con 0x7fee6c105100 2024-03-08T21:00:24.084 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:24.101+0000 7fee63400640 1 --2- 172.21.15.107:0/3957829241 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fee500a2990 0x7fee500a4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:00:24.084 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:24.102+0000 7fee6aa00640 1 -- 172.21.15.107:0/3957829241 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fee500a2990 msgr2=0x7fee500a4e50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.107:6800/3376636618 2024-03-08T21:00:24.084 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:24.102+0000 7fee63400640 1 -- 172.21.15.107:0/3957829241 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fee640f0db0 con 0x7fee6c105100 2024-03-08T21:00:24.085 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:24.102+0000 7fee6aa00640 1 --2- 172.21.15.107:0/3957829241 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fee500a2990 0x7fee500a4e50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-08T21:00:24.088 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:24.106+0000 7fee63400640 1 -- 172.21.15.107:0/3957829241 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7fee640f5070 con 0x7fee6c105100 2024-03-08T21:00:24.286 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:24.303+0000 7fee6aa00640 1 -- 172.21.15.107:0/3957829241 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fee500a2990 msgr2=0x7fee500a4e50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.107:6800/3376636618 2024-03-08T21:00:24.337 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:24.303+0000 7fee6aa00640 1 --2- 172.21.15.107:0/3957829241 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fee500a2990 0x7fee500a4e50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-08T21:00:24.339 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:24.309+0000 7fee71c00640 1 -- 172.21.15.107:0/3957829241 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7fee6c061f60 con 0x7fee6c105100 2024-03-08T21:00:24.339 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:24.314+0000 7fee63400640 1 -- 172.21.15.107:0/3957829241 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v32)=0 v32) v1 ==== 133+0+0 (secure 0 0 0) 0x7fee6404b020 con 0x7fee6c105100 2024-03-08T21:00:24.340 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:24.316+0000 7fee71c00640 1 -- 172.21.15.107:0/3957829241 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fee500a2990 msgr2=0x7fee500a4e50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-08T21:00:24.340 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:24.316+0000 7fee71c00640 1 --2- 172.21.15.107:0/3957829241 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fee500a2990 0x7fee500a4e50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:24.340 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:24.316+0000 7fee71c00640 1 -- 172.21.15.107:0/3957829241 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fee6c105100 msgr2=0x7fee6c1a0c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:00:24.340 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:24.316+0000 7fee71c00640 1 --2- 172.21.15.107:0/3957829241 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fee6c105100 0x7fee6c1a0c30 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fee64032380 tx=0x7fee64032820 comp rx=0 tx=0).stop 2024-03-08T21:00:24.340 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:24.316+0000 7fee71c00640 1 -- 172.21.15.107:0/3957829241 shutdown_connections 2024-03-08T21:00:24.340 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:24.316+0000 7fee71c00640 1 --2- 172.21.15.107:0/3957829241 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7fee500a2990 0x7fee500a4e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:24.340 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:24.316+0000 7fee71c00640 1 --2- 172.21.15.107:0/3957829241 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fee6c105100 0x7fee6c1a0c30 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:24.340 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:24.316+0000 7fee71c00640 1 --2- 172.21.15.107:0/3957829241 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fee6c104740 0x7fee6c199bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:24.340 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:24.316+0000 7fee71c00640 1 --2- 172.21.15.107:0/3957829241 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fee6c103550 0x7fee6c199680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:24.340 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:24.316+0000 7fee71c00640 1 -- 172.21.15.107:0/3957829241 >> 172.21.15.107:0/3957829241 conn(0x7fee6c0feae0 msgr2=0x7fee6c1079e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:00:24.340 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:24.317+0000 7fee71c00640 1 -- 172.21.15.107:0/3957829241 shutdown_connections 2024-03-08T21:00:24.340 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:24.317+0000 7fee71c00640 1 -- 172.21.15.107:0/3957829241 wait complete. 2024-03-08T21:00:24.341 DEBUG:teuthology.orchestra.run.smithi107:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m' 2024-03-08T21:00:24.532 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:24.549+0000 7efde7600640 1 -- 172.21.15.107:0/2473540667 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efde0102520 msgr2=0x7efde01029a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:00:24.532 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:24.549+0000 7efde7600640 1 --2- 172.21.15.107:0/2473540667 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efde0102520 0x7efde01029a0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7efdcc009900 tx=0x7efdcc02bc30 comp rx=0 tx=0).stop 2024-03-08T21:00:24.533 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:24.549+0000 7efde7600640 1 -- 172.21.15.107:0/2473540667 shutdown_connections 2024-03-08T21:00:24.533 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:24.549+0000 7efde7600640 1 --2- 172.21.15.107:0/2473540667 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efde0102ee0 0x7efde0109770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:24.533 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:24.549+0000 7efde7600640 1 --2- 172.21.15.107:0/2473540667 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efde0102520 0x7efde01029a0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:24.533 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:24.549+0000 7efde7600640 1 --2- 172.21.15.107:0/2473540667 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efde0101330 0x7efde0101750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:24.533 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:24.549+0000 7efde7600640 1 -- 172.21.15.107:0/2473540667 >> 172.21.15.107:0/2473540667 conn(0x7efde00fc880 msgr2=0x7efde00fece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:00:24.533 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:24.549+0000 7efde7600640 1 -- 172.21.15.107:0/2473540667 shutdown_connections 2024-03-08T21:00:24.533 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:24.549+0000 7efde7600640 1 -- 172.21.15.107:0/2473540667 wait complete. 2024-03-08T21:00:24.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:24.550+0000 7efde7600640 1 Processor -- start 2024-03-08T21:00:24.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:24.550+0000 7efde7600640 1 -- start start 2024-03-08T21:00:24.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:24.550+0000 7efde7600640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efde0101330 0x7efde01974e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:00:24.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:24.550+0000 7efde7600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efde0102520 0x7efde0197a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:00:24.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:24.550+0000 7efde7600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efde0102ee0 0x7efde019ea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:00:24.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:24.550+0000 7efde7600640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efde010cec0 con 0x7efde0102ee0 2024-03-08T21:00:24.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:24.550+0000 7efde7600640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efde010cd40 con 0x7efde0101330 2024-03-08T21:00:24.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:24.550+0000 7efde7600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efde010d040 con 0x7efde0102520 2024-03-08T21:00:24.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:24.551+0000 7efde5600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efde0102ee0 0x7efde019ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:00:24.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:24.551+0000 7efde5600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efde0102ee0 0x7efde019ea90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:34128/0 (socket says 172.21.15.107:34128) 2024-03-08T21:00:24.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:24.551+0000 7efde5600640 1 -- 172.21.15.107:0/776992101 learned_addr learned my addr 172.21.15.107:0/776992101 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:00:24.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:24.551+0000 7efde4c00640 1 --2- 172.21.15.107:0/776992101 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efde0101330 0x7efde01974e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:00:24.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:24.551+0000 7efddfe00640 1 --2- 172.21.15.107:0/776992101 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efde0102520 0x7efde0197a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:00:24.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:24.551+0000 7efde5600640 1 -- 172.21.15.107:0/776992101 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efde0102520 msgr2=0x7efde0197a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:00:24.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:24.551+0000 7efde5600640 1 --2- 172.21.15.107:0/776992101 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efde0102520 0x7efde0197a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:24.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:24.551+0000 7efde5600640 1 -- 172.21.15.107:0/776992101 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efde0101330 msgr2=0x7efde01974e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:00:24.537 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:24.551+0000 7efde5600640 1 --2- 172.21.15.107:0/776992101 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efde0101330 0x7efde01974e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:24.537 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:24.551+0000 7efde5600640 1 -- 172.21.15.107:0/776992101 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efde019f100 con 0x7efde0102ee0 2024-03-08T21:00:24.537 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:24.551+0000 7efddfe00640 1 --2- 172.21.15.107:0/776992101 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efde0102520 0x7efde0197a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T21:00:24.537 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:24.552+0000 7efde5600640 1 --2- 172.21.15.107:0/776992101 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efde0102ee0 0x7efde019ea90 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7efdd800eaa0 tx=0x7efdd800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:00:24.537 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:24.552+0000 7efddd600640 1 -- 172.21.15.107:0/776992101 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdd800cdb0 con 0x7efde0102ee0 2024-03-08T21:00:24.537 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:24.552+0000 7efddd600640 1 -- 172.21.15.107:0/776992101 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7efdd8005020 con 0x7efde0102ee0 2024-03-08T21:00:24.537 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:24.552+0000 7efde7600640 1 -- 172.21.15.107:0/776992101 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efde019f3f0 con 0x7efde0102ee0 2024-03-08T21:00:24.537 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:24.552+0000 7efddd600640 1 -- 172.21.15.107:0/776992101 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdd80116d0 con 0x7efde0102ee0 2024-03-08T21:00:24.537 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:24.552+0000 7efde7600640 1 -- 172.21.15.107:0/776992101 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efde019f960 con 0x7efde0102ee0 2024-03-08T21:00:24.537 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:24.554+0000 7efdd2a00640 1 -- 172.21.15.107:0/776992101 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efde0061ef0 con 0x7efde0102ee0 2024-03-08T21:00:24.540 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:24.557+0000 7efddd600640 1 -- 172.21.15.107:0/776992101 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) v1 ==== 137359+0+0 (secure 0 0 0) 0x7efdd8004a40 con 0x7efde0102ee0 2024-03-08T21:00:24.541 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:24.558+0000 7efddd600640 1 --2- 172.21.15.107:0/776992101 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7efdc80a2a10 0x7efdc80a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:00:24.541 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:24.558+0000 7efde4c00640 1 -- 172.21.15.107:0/776992101 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7efdc80a2a10 msgr2=0x7efdc80a4ed0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.107:6800/3376636618 2024-03-08T21:00:24.541 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:24.558+0000 7efddd600640 1 -- 172.21.15.107:0/776992101 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7efdd80ce9a0 con 0x7efde0102ee0 2024-03-08T21:00:24.541 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:24.558+0000 7efde4c00640 1 --2- 172.21.15.107:0/776992101 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7efdc80a2a10 0x7efdc80a4ed0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-08T21:00:24.546 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:24.563+0000 7efddd600640 1 -- 172.21.15.107:0/776992101 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7efdd8016030 con 0x7efde0102ee0 2024-03-08T21:00:24.628 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:00:24 smithi204 ceph-mon[34935]: from='client.? 172.21.15.107:0/1782390817' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-03-08T21:00:24.628 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:00:24 smithi204 ceph-mon[34935]: mgrmap e19: x(active, since 12m), standbys: z, y 2024-03-08T21:00:24.628 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:00:24 smithi204 ceph-mon[34935]: from='client.? 172.21.15.107:0/3902874278' entity='client.admin' 2024-03-08T21:00:24.628 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:00:24 smithi204 ceph-mon[34935]: from='client.? 172.21.15.107:0/3957829241' entity='client.admin' 2024-03-08T21:00:24.628 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Mar 08 21:00:24 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-z[36870]: 2024-03-08T21:00:24.379+0000 7fdcbb60b200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-08T21:00:24.628 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Mar 08 21:00:24 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-z[36870]: 2024-03-08T21:00:24.505+0000 7fdcbb60b200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-08T21:00:24.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:00:24 smithi107 ceph-mon[31381]: from='client.? 172.21.15.107:0/1782390817' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-03-08T21:00:24.717 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:00:24 smithi107 ceph-mon[31381]: mgrmap e19: x(active, since 12m), standbys: z, y 2024-03-08T21:00:24.717 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:00:24 smithi107 ceph-mon[31381]: from='client.? 172.21.15.107:0/3902874278' entity='client.admin' 2024-03-08T21:00:24.717 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:00:24 smithi107 ceph-mon[31381]: from='client.? 172.21.15.107:0/3957829241' entity='client.admin' 2024-03-08T21:00:24.717 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 08 21:00:24 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-x[31606]: 2024-03-08T21:00:24.375+0000 7f65d41ae200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-08T21:00:24.717 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 08 21:00:24 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-x[31606]: 2024-03-08T21:00:24.520+0000 7f65d41ae200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-08T21:00:24.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:24.760+0000 7efde4c00640 1 -- 172.21.15.107:0/776992101 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7efdc80a2a10 msgr2=0x7efdc80a4ed0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.107:6800/3376636618 2024-03-08T21:00:24.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:24.760+0000 7efde4c00640 1 --2- 172.21.15.107:0/776992101 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7efdc80a2a10 0x7efdc80a4ed0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-08T21:00:24.764 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:00:24 smithi149 ceph-mon[35042]: from='client.? 172.21.15.107:0/1782390817' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-03-08T21:00:24.764 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:00:24 smithi149 ceph-mon[35042]: mgrmap e19: x(active, since 12m), standbys: z, y 2024-03-08T21:00:24.764 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:00:24 smithi149 ceph-mon[35042]: from='client.? 172.21.15.107:0/3902874278' entity='client.admin' 2024-03-08T21:00:24.764 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:00:24 smithi149 ceph-mon[35042]: from='client.? 172.21.15.107:0/3957829241' entity='client.admin' 2024-03-08T21:00:24.765 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Mar 08 21:00:24 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-y[36424]: 2024-03-08T21:00:24.376+0000 7f30cebf2200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-08T21:00:24.765 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Mar 08 21:00:24 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-y[36424]: 2024-03-08T21:00:24.531+0000 7f30cebf2200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-08T21:00:24.773 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:24.790+0000 7efdd2a00640 1 -- 172.21.15.107:0/776992101 --> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7efde00622b0 con 0x7efdc80a2a10 2024-03-08T21:00:25.144 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:25.161+0000 7efde4c00640 1 -- 172.21.15.107:0/776992101 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7efdc80a2a10 msgr2=0x7efdc80a4ed0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.107:6800/3376636618 2024-03-08T21:00:25.144 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:25.161+0000 7efde4c00640 1 --2- 172.21.15.107:0/776992101 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7efdc80a2a10 0x7efdc80a4ed0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-08T21:00:25.618 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 08 21:00:25 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-x[31606]: 2024-03-08T21:00:25.336+0000 7f65d41ae200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-08T21:00:25.618 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 08 21:00:25 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-x[31606]: 2024-03-08T21:00:25.455+0000 7f65d41ae200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-08T21:00:25.618 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 08 21:00:25 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-x[31606]: 2024-03-08T21:00:25.634+0000 7f65d41ae200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-08T21:00:25.627 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Mar 08 21:00:25 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-z[36870]: 2024-03-08T21:00:25.355+0000 7fdcbb60b200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-08T21:00:25.627 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Mar 08 21:00:25 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-z[36870]: 2024-03-08T21:00:25.491+0000 7fdcbb60b200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-08T21:00:25.673 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Mar 08 21:00:25 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-y[36424]: 2024-03-08T21:00:25.391+0000 7f30cebf2200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-08T21:00:25.674 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Mar 08 21:00:25 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-y[36424]: 2024-03-08T21:00:25.510+0000 7f30cebf2200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-08T21:00:25.674 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Mar 08 21:00:25 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-y[36424]: 2024-03-08T21:00:25.690+0000 7f30cebf2200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-08T21:00:25.904 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 08 21:00:25 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-x[31606]: 2024-03-08T21:00:25.757+0000 7f65d41ae200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-08T21:00:25.904 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 08 21:00:25 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-x[31606]: 2024-03-08T21:00:25.867+0000 7f65d41ae200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-08T21:00:25.945 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:25.963+0000 7efde4c00640 1 -- 172.21.15.107:0/776992101 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7efdc80a2a10 msgr2=0x7efdc80a4ed0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.107:6800/3376636618 2024-03-08T21:00:25.945 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:25.963+0000 7efde4c00640 1 --2- 172.21.15.107:0/776992101 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7efdc80a2a10 0x7efdc80a4ed0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-08T21:00:26.014 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Mar 08 21:00:25 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-y[36424]: 2024-03-08T21:00:25.813+0000 7f30cebf2200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-08T21:00:26.014 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Mar 08 21:00:25 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-y[36424]: 2024-03-08T21:00:25.926+0000 7f30cebf2200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-08T21:00:26.128 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Mar 08 21:00:25 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-z[36870]: 2024-03-08T21:00:25.687+0000 7fdcbb60b200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-08T21:00:26.128 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Mar 08 21:00:25 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-z[36870]: 2024-03-08T21:00:25.813+0000 7fdcbb60b200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-08T21:00:26.128 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Mar 08 21:00:25 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-z[36870]: 2024-03-08T21:00:25.926+0000 7fdcbb60b200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-08T21:00:26.441 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 08 21:00:26 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-x[31606]: 2024-03-08T21:00:26.204+0000 7f65d41ae200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-08T21:00:26.442 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 08 21:00:26 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-x[31606]: 2024-03-08T21:00:26.346+0000 7f65d41ae200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-08T21:00:26.442 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 08 21:00:26 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-x[31606]: 2024-03-08T21:00:26.457+0000 7f65d41ae200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-08T21:00:26.495 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Mar 08 21:00:26 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-z[36870]: 2024-03-08T21:00:26.258+0000 7fdcbb60b200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-08T21:00:26.495 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Mar 08 21:00:26 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-z[36870]: 2024-03-08T21:00:26.398+0000 7fdcbb60b200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-08T21:00:26.502 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Mar 08 21:00:26 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-y[36424]: 2024-03-08T21:00:26.261+0000 7f30cebf2200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-08T21:00:26.502 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Mar 08 21:00:26 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-y[36424]: 2024-03-08T21:00:26.404+0000 7f30cebf2200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-08T21:00:26.717 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 08 21:00:26 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-x[31606]: 2024-03-08T21:00:26.580+0000 7f65d41ae200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-08T21:00:26.764 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Mar 08 21:00:26 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-y[36424]: 2024-03-08T21:00:26.519+0000 7f30cebf2200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-08T21:00:26.765 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Mar 08 21:00:26 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-y[36424]: 2024-03-08T21:00:26.644+0000 7f30cebf2200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-08T21:00:26.838 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Mar 08 21:00:26 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-z[36870]: 2024-03-08T21:00:26.511+0000 7fdcbb60b200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-08T21:00:26.838 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Mar 08 21:00:26 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-z[36870]: 2024-03-08T21:00:26.634+0000 7fdcbb60b200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-08T21:00:27.127 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Mar 08 21:00:26 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-z[36870]: 2024-03-08T21:00:26.855+0000 7fdcbb60b200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-08T21:00:27.216 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 08 21:00:26 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-x[31606]: 2024-03-08T21:00:26.801+0000 7f65d41ae200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-08T21:00:27.264 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Mar 08 21:00:26 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-y[36424]: 2024-03-08T21:00:26.866+0000 7f30cebf2200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-08T21:00:27.547 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:27.565+0000 7efde4c00640 1 -- 172.21.15.107:0/776992101 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7efdc80a2a10 msgr2=0x7efdc80a4ed0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.107:6800/3376636618 2024-03-08T21:00:27.547 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:27.565+0000 7efde4c00640 1 --2- 172.21.15.107:0/776992101 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7efdc80a2a10 0x7efdc80a4ed0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-03-08T21:00:28.377 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Mar 08 21:00:28 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-z[36870]: 2024-03-08T21:00:28.131+0000 7fdcbb60b200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-08T21:00:28.377 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Mar 08 21:00:28 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-z[36870]: 2024-03-08T21:00:28.301+0000 7fdcbb60b200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-08T21:00:28.467 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 08 21:00:28 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-x[31606]: 2024-03-08T21:00:28.068+0000 7f65d41ae200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-08T21:00:28.467 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 08 21:00:28 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-x[31606]: 2024-03-08T21:00:28.239+0000 7f65d41ae200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-08T21:00:28.514 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Mar 08 21:00:28 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-y[36424]: 2024-03-08T21:00:28.143+0000 7f30cebf2200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-08T21:00:28.514 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Mar 08 21:00:28 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-y[36424]: 2024-03-08T21:00:28.315+0000 7f30cebf2200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-08T21:00:28.967 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 08 21:00:28 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-x[31606]: 2024-03-08T21:00:28.654+0000 7f65d41ae200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-08T21:00:28.967 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 08 21:00:28 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-x[31606]: 2024-03-08T21:00:28.773+0000 7f65d41ae200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-08T21:00:29.014 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Mar 08 21:00:28 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-y[36424]: 2024-03-08T21:00:28.738+0000 7f30cebf2200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-08T21:00:29.015 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Mar 08 21:00:28 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-y[36424]: 2024-03-08T21:00:28.856+0000 7f30cebf2200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-08T21:00:29.079 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Mar 08 21:00:28 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-z[36870]: 2024-03-08T21:00:28.718+0000 7fdcbb60b200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-08T21:00:29.080 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Mar 08 21:00:28 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-z[36870]: 2024-03-08T21:00:28.837+0000 7fdcbb60b200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-08T21:00:29.080 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Mar 08 21:00:29 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-z[36870]: 2024-03-08T21:00:29.093+0000 7fdcbb60b200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-08T21:00:29.377 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Mar 08 21:00:29 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-z[36870]: 2024-03-08T21:00:29.337+0000 7fdcbb60b200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-08T21:00:29.467 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 08 21:00:29 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-x[31606]: 2024-03-08T21:00:29.029+0000 7f65d41ae200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-08T21:00:29.467 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 08 21:00:29 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-x[31606]: 2024-03-08T21:00:29.272+0000 7f65d41ae200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-08T21:00:29.514 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Mar 08 21:00:29 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-y[36424]: 2024-03-08T21:00:29.113+0000 7f30cebf2200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-08T21:00:29.515 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Mar 08 21:00:29 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-y[36424]: 2024-03-08T21:00:29.357+0000 7f30cebf2200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-08T21:00:29.955 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 08 21:00:29 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-x[31606]: 2024-03-08T21:00:29.648+0000 7f65d41ae200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-08T21:00:30.014 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Mar 08 21:00:29 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-y[36424]: 2024-03-08T21:00:29.732+0000 7f30cebf2200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-08T21:00:30.019 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Mar 08 21:00:29 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-z[36870]: 2024-03-08T21:00:29.712+0000 7fdcbb60b200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-08T21:00:30.216 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 08 21:00:29 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-x[31606]: 2024-03-08T21:00:29.973+0000 7f65d41ae200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-08T21:00:30.325 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Mar 08 21:00:30 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-z[36870]: 2024-03-08T21:00:30.036+0000 7fdcbb60b200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-08T21:00:30.352 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Mar 08 21:00:30 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-y[36424]: 2024-03-08T21:00:30.063+0000 7f30cebf2200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-08T21:00:30.628 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Mar 08 21:00:30 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-z[36870]: 2024-03-08T21:00:30.342+0000 7fdcbb60b200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-08T21:00:30.716 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 08 21:00:30 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-x[31606]: 2024-03-08T21:00:30.278+0000 7f65d41ae200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-08T21:00:30.751 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:30.769+0000 7efde4c00640 1 -- 172.21.15.107:0/776992101 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7efdc80a2a10 msgr2=0x7efdc80a4ed0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.107:6800/3376636618 2024-03-08T21:00:30.751 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:30.769+0000 7efde4c00640 1 --2- 172.21.15.107:0/776992101 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7efdc80a2a10 0x7efdc80a4ed0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 6.400000 2024-03-08T21:00:30.764 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Mar 08 21:00:30 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-y[36424]: 2024-03-08T21:00:30.369+0000 7f30cebf2200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-08T21:00:31.627 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Mar 08 21:00:31 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-z[36870]: 2024-03-08T21:00:31.373+0000 7fdcbb60b200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-08T21:00:31.717 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 08 21:00:31 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-x[31606]: 2024-03-08T21:00:31.310+0000 7f65d41ae200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-08T21:00:31.764 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Mar 08 21:00:31 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-y[36424]: 2024-03-08T21:00:31.401+0000 7f30cebf2200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-08T21:00:32.377 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Mar 08 21:00:32 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-z[36870]: 2024-03-08T21:00:32.075+0000 7fdcbb60b200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-08T21:00:32.377 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Mar 08 21:00:32 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-z[36870]: 2024-03-08T21:00:32.205+0000 7fdcbb60b200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-08T21:00:32.410 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 08 21:00:32 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-x[31606]: 2024-03-08T21:00:32.011+0000 7f65d41ae200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-08T21:00:32.411 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 08 21:00:32 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-x[31606]: 2024-03-08T21:00:32.142+0000 7f65d41ae200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-08T21:00:32.437 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:32.455+0000 7efddd600640 1 -- 172.21.15.107:0/776992101 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mgrmap(e 20) v1 ==== 136975+0+0 (secure 0 0 0) 0x7efdd8097850 con 0x7efde0102ee0 2024-03-08T21:00:32.438 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:32.457+0000 7efddd600640 1 -- 172.21.15.107:0/776992101 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7efdc80a2a10 msgr2=0x7efdc80a4ed0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-08T21:00:32.439 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:32.457+0000 7efddd600640 1 --2- 172.21.15.107:0/776992101 >> [v2:172.21.15.107:6800/3376636618,v1:172.21.15.107:6801/3376636618] conn(0x7efdc80a2a10 0x7efdc80a4ed0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:32.471 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Mar 08 21:00:32 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-y[36424]: 2024-03-08T21:00:32.113+0000 7f30cebf2200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-08T21:00:32.472 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Mar 08 21:00:32 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-y[36424]: 2024-03-08T21:00:32.243+0000 7f30cebf2200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-08T21:00:32.716 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 08 21:00:32 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-x[31606]: 2024-03-08T21:00:32.427+0000 7f65d41ae200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-08T21:00:32.717 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:00:32 smithi107 ceph-mon[31381]: Active manager daemon x restarted 2024-03-08T21:00:32.717 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:00:32 smithi107 ceph-mon[31381]: Activating manager daemon x 2024-03-08T21:00:32.717 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:00:32 smithi107 ceph-mon[31381]: osdmap e89: 12 total, 12 up, 12 in 2024-03-08T21:00:32.717 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:00:32 smithi107 ceph-mon[31381]: mgrmap e20: x(active, starting, since 0.0285492s), standbys: z, y 2024-03-08T21:00:32.717 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:00:32 smithi107 ceph-mon[31381]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-08T21:00:32.717 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:00:32 smithi107 ceph-mon[31381]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-08T21:00:32.718 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:00:32 smithi107 ceph-mon[31381]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-08T21:00:32.718 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:00:32 smithi107 ceph-mon[31381]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-08T21:00:32.718 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:00:32 smithi107 ceph-mon[31381]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-08T21:00:32.718 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:00:32 smithi107 ceph-mon[31381]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-08T21:00:32.718 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:00:32 smithi107 ceph-mon[31381]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-08T21:00:32.718 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:00:32 smithi107 ceph-mon[31381]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-08T21:00:32.718 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:00:32 smithi107 ceph-mon[31381]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-08T21:00:32.718 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:00:32 smithi107 ceph-mon[31381]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-08T21:00:32.718 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:00:32 smithi107 ceph-mon[31381]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-08T21:00:32.718 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:00:32 smithi107 ceph-mon[31381]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-08T21:00:32.718 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:00:32 smithi107 ceph-mon[31381]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-08T21:00:32.718 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:00:32 smithi107 ceph-mon[31381]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-08T21:00:32.718 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:00:32 smithi107 ceph-mon[31381]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-08T21:00:32.719 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:00:32 smithi107 ceph-mon[31381]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-08T21:00:32.719 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:00:32 smithi107 ceph-mon[31381]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-03-08T21:00:32.719 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:00:32 smithi107 ceph-mon[31381]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-08T21:00:32.719 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:00:32 smithi107 ceph-mon[31381]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-08T21:00:32.719 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:00:32 smithi107 ceph-mon[31381]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-08T21:00:32.719 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:00:32 smithi107 ceph-mon[31381]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-08T21:00:32.719 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:00:32 smithi107 ceph-mon[31381]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-08T21:00:32.719 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:00:32 smithi107 ceph-mon[31381]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-08T21:00:32.719 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:00:32 smithi107 ceph-mon[31381]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-08T21:00:32.719 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:00:32 smithi107 ceph-mon[31381]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-08T21:00:32.719 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:00:32 smithi107 ceph-mon[31381]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-08T21:00:32.719 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:00:32 smithi107 ceph-mon[31381]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-08T21:00:32.719 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:00:32 smithi107 ceph-mon[31381]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-08T21:00:32.720 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:00:32 smithi107 ceph-mon[31381]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-08T21:00:32.720 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:00:32 smithi107 ceph-mon[31381]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-08T21:00:32.720 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:00:32 smithi107 ceph-mon[31381]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-08T21:00:32.720 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:00:32 smithi107 ceph-mon[31381]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-08T21:00:32.720 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:00:32 smithi107 ceph-mon[31381]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-08T21:00:32.764 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Mar 08 21:00:32 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-y[36424]: 2024-03-08T21:00:32.533+0000 7f30cebf2200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-08T21:00:32.765 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:00:32 smithi149 ceph-mon[35042]: Active manager daemon x restarted 2024-03-08T21:00:32.765 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:00:32 smithi149 ceph-mon[35042]: Activating manager daemon x 2024-03-08T21:00:32.765 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:00:32 smithi149 ceph-mon[35042]: osdmap e89: 12 total, 12 up, 12 in 2024-03-08T21:00:32.765 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:00:32 smithi149 ceph-mon[35042]: mgrmap e20: x(active, starting, since 0.0285492s), standbys: z, y 2024-03-08T21:00:32.765 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:00:32 smithi149 ceph-mon[35042]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-08T21:00:32.766 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:00:32 smithi149 ceph-mon[35042]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-08T21:00:32.766 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:00:32 smithi149 ceph-mon[35042]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-08T21:00:32.766 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:00:32 smithi149 ceph-mon[35042]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-08T21:00:32.766 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:00:32 smithi149 ceph-mon[35042]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-08T21:00:32.766 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:00:32 smithi149 ceph-mon[35042]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-08T21:00:32.766 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:00:32 smithi149 ceph-mon[35042]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-08T21:00:32.766 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:00:32 smithi149 ceph-mon[35042]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-08T21:00:32.766 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:00:32 smithi149 ceph-mon[35042]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-08T21:00:32.766 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:00:32 smithi149 ceph-mon[35042]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-08T21:00:32.766 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:00:32 smithi149 ceph-mon[35042]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-08T21:00:32.766 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:00:32 smithi149 ceph-mon[35042]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-08T21:00:32.766 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:00:32 smithi149 ceph-mon[35042]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-08T21:00:32.767 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:00:32 smithi149 ceph-mon[35042]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-08T21:00:32.767 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:00:32 smithi149 ceph-mon[35042]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-08T21:00:32.767 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:00:32 smithi149 ceph-mon[35042]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-08T21:00:32.767 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:00:32 smithi149 ceph-mon[35042]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-03-08T21:00:32.767 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:00:32 smithi149 ceph-mon[35042]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-08T21:00:32.767 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:00:32 smithi149 ceph-mon[35042]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-08T21:00:32.767 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:00:32 smithi149 ceph-mon[35042]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-08T21:00:32.767 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:00:32 smithi149 ceph-mon[35042]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-08T21:00:32.767 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:00:32 smithi149 ceph-mon[35042]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-08T21:00:32.767 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:00:32 smithi149 ceph-mon[35042]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-08T21:00:32.767 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:00:32 smithi149 ceph-mon[35042]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-08T21:00:32.767 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:00:32 smithi149 ceph-mon[35042]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-08T21:00:32.767 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:00:32 smithi149 ceph-mon[35042]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-08T21:00:32.767 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:00:32 smithi149 ceph-mon[35042]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-08T21:00:32.768 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:00:32 smithi149 ceph-mon[35042]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-08T21:00:32.768 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:00:32 smithi149 ceph-mon[35042]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-08T21:00:32.768 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:00:32 smithi149 ceph-mon[35042]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-08T21:00:32.768 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:00:32 smithi149 ceph-mon[35042]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-08T21:00:32.768 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:00:32 smithi149 ceph-mon[35042]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-08T21:00:32.768 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:00:32 smithi149 ceph-mon[35042]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-08T21:00:32.785 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:00:32 smithi204 ceph-mon[34935]: Active manager daemon x restarted 2024-03-08T21:00:32.786 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:00:32 smithi204 ceph-mon[34935]: Activating manager daemon x 2024-03-08T21:00:32.786 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:00:32 smithi204 ceph-mon[34935]: osdmap e89: 12 total, 12 up, 12 in 2024-03-08T21:00:32.786 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:00:32 smithi204 ceph-mon[34935]: mgrmap e20: x(active, starting, since 0.0285492s), standbys: z, y 2024-03-08T21:00:32.787 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:00:32 smithi204 ceph-mon[34935]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-08T21:00:32.788 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:00:32 smithi204 ceph-mon[34935]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-08T21:00:32.788 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:00:32 smithi204 ceph-mon[34935]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-08T21:00:32.788 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:00:32 smithi204 ceph-mon[34935]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-08T21:00:32.788 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:00:32 smithi204 ceph-mon[34935]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-08T21:00:32.788 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:00:32 smithi204 ceph-mon[34935]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-08T21:00:32.788 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:00:32 smithi204 ceph-mon[34935]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-08T21:00:32.789 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:00:32 smithi204 ceph-mon[34935]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-08T21:00:32.789 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:00:32 smithi204 ceph-mon[34935]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-08T21:00:32.789 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:00:32 smithi204 ceph-mon[34935]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-08T21:00:32.789 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:00:32 smithi204 ceph-mon[34935]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-08T21:00:32.789 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:00:32 smithi204 ceph-mon[34935]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-08T21:00:32.789 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:00:32 smithi204 ceph-mon[34935]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-08T21:00:32.789 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:00:32 smithi204 ceph-mon[34935]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-08T21:00:32.790 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:00:32 smithi204 ceph-mon[34935]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-08T21:00:32.790 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:00:32 smithi204 ceph-mon[34935]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-08T21:00:32.790 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:00:32 smithi204 ceph-mon[34935]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-03-08T21:00:32.790 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:00:32 smithi204 ceph-mon[34935]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-08T21:00:32.791 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:00:32 smithi204 ceph-mon[34935]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-08T21:00:32.792 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:00:32 smithi204 ceph-mon[34935]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-08T21:00:32.792 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:00:32 smithi204 ceph-mon[34935]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-08T21:00:32.793 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:00:32 smithi204 ceph-mon[34935]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-08T21:00:32.793 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:00:32 smithi204 ceph-mon[34935]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-08T21:00:32.793 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Mar 08 21:00:32 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-z[36870]: 2024-03-08T21:00:32.500+0000 7fdcbb60b200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-08T21:00:32.794 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:00:32 smithi204 ceph-mon[34935]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-08T21:00:32.794 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:00:32 smithi204 ceph-mon[34935]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-08T21:00:32.794 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:00:32 smithi204 ceph-mon[34935]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-08T21:00:32.794 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:00:32 smithi204 ceph-mon[34935]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-08T21:00:32.794 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:00:32 smithi204 ceph-mon[34935]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-08T21:00:32.794 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:00:32 smithi204 ceph-mon[34935]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-08T21:00:32.794 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:00:32 smithi204 ceph-mon[34935]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-08T21:00:32.794 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:00:32 smithi204 ceph-mon[34935]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-08T21:00:32.794 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:00:32 smithi204 ceph-mon[34935]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-08T21:00:32.794 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:00:32 smithi204 ceph-mon[34935]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-08T21:00:33.222 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 08 21:00:32 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-x[31606]: 2024-03-08T21:00:32.985+0000 7f652a600700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-08T21:00:33.222 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 08 21:00:32 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-x[31606]: 2024-03-08T21:00:32.985+0000 7f652a600700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-08T21:00:33.223 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 08 21:00:32 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-x[31606]: 2024-03-08T21:00:32.985+0000 7f652a600700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-08T21:00:33.223 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 08 21:00:32 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-x[31606]: 2024-03-08T21:00:32.985+0000 7f652a600700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-08T21:00:33.223 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 08 21:00:32 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-x[31606]: 2024-03-08T21:00:32.985+0000 7f652a600700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-08T21:00:33.223 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 08 21:00:32 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-x[31606]: 2024-03-08T21:00:32.985+0000 7f6526a00700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-08T21:00:33.223 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 08 21:00:32 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-x[31606]: 2024-03-08T21:00:32.985+0000 7f6526a00700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-08T21:00:33.223 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 08 21:00:32 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-x[31606]: 2024-03-08T21:00:32.985+0000 7f6526a00700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-08T21:00:33.224 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 08 21:00:32 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-x[31606]: 2024-03-08T21:00:32.985+0000 7f6526a00700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-08T21:00:33.224 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 08 21:00:32 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-x[31606]: 2024-03-08T21:00:32.985+0000 7f6526a00700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-08T21:00:33.502 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:33.519+0000 7efddd600640 1 -- 172.21.15.107:0/776992101 <== mon.0 v2:172.21.15.107:3300/0 8 ==== mgrmap(e 21) v1 ==== 137102+0+0 (secure 0 0 0) 0x7efdd8022350 con 0x7efde0102ee0 2024-03-08T21:00:33.504 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:33.520+0000 7efddd600640 1 --2- 172.21.15.107:0/776992101 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7efdc8051790 0x7efdc8051b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:00:33.504 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:33.520+0000 7efddd600640 1 -- 172.21.15.107:0/776992101 --> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7efdc803cae0 con 0x7efdc8051790 2024-03-08T21:00:33.512 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:33.530+0000 7efde4c00640 1 --2- 172.21.15.107:0/776992101 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7efdc8051790 0x7efdc8051b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:00:33.520 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:33.538+0000 7efde4c00640 1 --2- 172.21.15.107:0/776992101 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7efdc8051790 0x7efdc8051b70 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7efde0102380 tx=0x7efdd4005f90 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:00:33.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:33.643+0000 7efddd600640 1 -- 172.21.15.107:0/776992101 <== mgr.15375 v2:172.21.15.107:6800/2792338347 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7efdc803cae0 con 0x7efdc8051790 2024-03-08T21:00:33.626 INFO:teuthology.orchestra.run.smithi107.stdout:Schedule set for path / 2024-03-08T21:00:33.628 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:33.646+0000 7efdd2a00640 1 -- 172.21.15.107:0/776992101 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7efdc8051790 msgr2=0x7efdc8051b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:00:33.628 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:33.646+0000 7efdd2a00640 1 --2- 172.21.15.107:0/776992101 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7efdc8051790 0x7efdc8051b70 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7efde0102380 tx=0x7efdd4005f90 comp rx=0 tx=0).stop 2024-03-08T21:00:33.628 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:33.646+0000 7efdd2a00640 1 -- 172.21.15.107:0/776992101 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efde0102ee0 msgr2=0x7efde019ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:00:33.628 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:33.646+0000 7efdd2a00640 1 --2- 172.21.15.107:0/776992101 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efde0102ee0 0x7efde019ea90 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7efdd800eaa0 tx=0x7efdd800ef60 comp rx=0 tx=0).stop 2024-03-08T21:00:33.629 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:33.646+0000 7efde5600640 1 -- 172.21.15.107:0/776992101 reap_dead start 2024-03-08T21:00:33.629 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:33.646+0000 7efdd2a00640 1 -- 172.21.15.107:0/776992101 shutdown_connections 2024-03-08T21:00:33.629 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:33.646+0000 7efdd2a00640 1 -- 172.21.15.107:0/776992101 >> 172.21.15.107:0/776992101 conn(0x7efde00fc880 msgr2=0x7efde0105820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:00:33.629 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:33.646+0000 7efdd2a00640 1 -- 172.21.15.107:0/776992101 shutdown_connections 2024-03-08T21:00:33.629 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:33.646+0000 7efdd2a00640 1 -- 172.21.15.107:0/776992101 wait complete. 2024-03-08T21:00:33.645 DEBUG:teuthology.orchestra.run.smithi107:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h' 2024-03-08T21:00:33.802 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:00:33 smithi204 ceph-mon[34935]: Manager daemon x is now available 2024-03-08T21:00:33.802 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:00:33 smithi204 ceph-mon[34935]: Standby manager daemon z restarted 2024-03-08T21:00:33.802 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:00:33 smithi204 ceph-mon[34935]: Standby manager daemon z started 2024-03-08T21:00:33.802 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:00:33 smithi204 ceph-mon[34935]: from='mgr.? 172.21.15.204:0/3693169975' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-03-08T21:00:33.802 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:00:33 smithi204 ceph-mon[34935]: from='mgr.? 172.21.15.204:0/3693169975' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-08T21:00:33.802 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:00:33 smithi204 ceph-mon[34935]: from='mgr.? 172.21.15.204:0/3693169975' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-03-08T21:00:33.803 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:00:33 smithi204 ceph-mon[34935]: from='mgr.? 172.21.15.204:0/3693169975' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-08T21:00:33.803 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:00:33 smithi204 ceph-mon[34935]: Standby manager daemon y restarted 2024-03-08T21:00:33.803 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:00:33 smithi204 ceph-mon[34935]: Standby manager daemon y started 2024-03-08T21:00:33.803 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:00:33 smithi204 ceph-mon[34935]: from='mgr.? 172.21.15.149:0/2166939354' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-08T21:00:33.803 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:00:33 smithi204 ceph-mon[34935]: from='mgr.? 172.21.15.149:0/2166939354' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-08T21:00:33.803 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:00:33 smithi204 ceph-mon[34935]: from='mgr.? 172.21.15.149:0/2166939354' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-08T21:00:33.803 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:00:33 smithi204 ceph-mon[34935]: from='mgr.? 172.21.15.149:0/2166939354' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-08T21:00:33.803 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:00:33 smithi204 ceph-mon[34935]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T21:00:33.803 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:00:33 smithi204 ceph-mon[34935]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T21:00:33.803 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:00:33 smithi204 ceph-mon[34935]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-08T21:00:33.803 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:00:33 smithi204 ceph-mon[34935]: from='mgr.15375 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-08T21:00:33.803 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:00:33 smithi204 ceph-mon[34935]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-08T21:00:33.803 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:00:33 smithi204 ceph-mon[34935]: from='mgr.15375 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-08T21:00:33.820 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:33.838+0000 7f5744c00640 1 -- 172.21.15.107:0/4188010361 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5740103550 msgr2=0x7f5740103970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:00:33.820 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:33.838+0000 7f5744c00640 1 --2- 172.21.15.107:0/4188010361 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5740103550 0x7f5740103970 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f5730009900 tx=0x7f573002bc40 comp rx=0 tx=0).stop 2024-03-08T21:00:33.820 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:33.838+0000 7f5744c00640 1 -- 172.21.15.107:0/4188010361 shutdown_connections 2024-03-08T21:00:33.820 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:33.838+0000 7f5744c00640 1 --2- 172.21.15.107:0/4188010361 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5740105100 0x7f574010b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:33.820 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:33.838+0000 7f5744c00640 1 --2- 172.21.15.107:0/4188010361 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5740104740 0x7f5740104bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:33.820 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:33.838+0000 7f5744c00640 1 --2- 172.21.15.107:0/4188010361 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5740103550 0x7f5740103970 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:33.821 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:33.838+0000 7f5744c00640 1 -- 172.21.15.107:0/4188010361 >> 172.21.15.107:0/4188010361 conn(0x7f57400feae0 msgr2=0x7f5740100f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:00:33.821 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:33.838+0000 7f5744c00640 1 -- 172.21.15.107:0/4188010361 shutdown_connections 2024-03-08T21:00:33.821 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:33.838+0000 7f5744c00640 1 -- 172.21.15.107:0/4188010361 wait complete. 2024-03-08T21:00:33.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:33.839+0000 7f5744c00640 1 Processor -- start 2024-03-08T21:00:33.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:33.839+0000 7f5744c00640 1 -- start start 2024-03-08T21:00:33.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:33.840+0000 7f5744c00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5740103550 0x7f57401996f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:00:33.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:33.840+0000 7f5744c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5740104740 0x7f5740199c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:00:33.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:33.840+0000 7f5744c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5740105100 0x7f57401a0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:00:33.824 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:33.840+0000 7f5744c00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f574010f090 con 0x7f5740104740 2024-03-08T21:00:33.824 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:33.840+0000 7f5744c00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f574010ef10 con 0x7f5740103550 2024-03-08T21:00:33.824 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:33.840+0000 7f5744c00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f574010f210 con 0x7f5740105100 2024-03-08T21:00:33.824 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:33.840+0000 7f573d400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5740104740 0x7f5740199c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:00:33.824 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:33.840+0000 7f573d400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5740104740 0x7f5740199c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:58236/0 (socket says 172.21.15.107:58236) 2024-03-08T21:00:33.824 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:33.840+0000 7f573d400640 1 -- 172.21.15.107:0/4107548792 learned_addr learned my addr 172.21.15.107:0/4107548792 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:00:33.824 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:33.840+0000 7f573de00640 1 --2- 172.21.15.107:0/4107548792 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5740103550 0x7f57401996f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:00:33.824 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:33.840+0000 7f573e800640 1 --2- 172.21.15.107:0/4107548792 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5740105100 0x7f57401a0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:00:33.824 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:33.840+0000 7f573d400640 1 -- 172.21.15.107:0/4107548792 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5740105100 msgr2=0x7f57401a0ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:00:33.824 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:33.840+0000 7f573d400640 1 --2- 172.21.15.107:0/4107548792 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5740105100 0x7f57401a0ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:33.824 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:33.840+0000 7f573d400640 1 -- 172.21.15.107:0/4107548792 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5740103550 msgr2=0x7f57401996f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:00:33.824 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:33.840+0000 7f573d400640 1 --2- 172.21.15.107:0/4107548792 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5740103550 0x7f57401996f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:33.824 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:33.840+0000 7f573d400640 1 -- 172.21.15.107:0/4107548792 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57401a1310 con 0x7f5740104740 2024-03-08T21:00:33.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:33.841+0000 7f573d400640 1 --2- 172.21.15.107:0/4107548792 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5740104740 0x7f5740199c30 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f572c00d950 tx=0x7f572c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:00:33.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:33.841+0000 7f5736a00640 1 -- 172.21.15.107:0/4107548792 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f572c017070 con 0x7f5740104740 2024-03-08T21:00:33.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:33.841+0000 7f5736a00640 1 -- 172.21.15.107:0/4107548792 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f572c009c30 con 0x7f5740104740 2024-03-08T21:00:33.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:33.841+0000 7f5736a00640 1 -- 172.21.15.107:0/4107548792 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f572c00f620 con 0x7f5740104740 2024-03-08T21:00:33.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:33.841+0000 7f5744c00640 1 -- 172.21.15.107:0/4107548792 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57401a1600 con 0x7f5740104740 2024-03-08T21:00:33.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:33.841+0000 7f5744c00640 1 -- 172.21.15.107:0/4107548792 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57401a1b50 con 0x7f5740104740 2024-03-08T21:00:33.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:33.842+0000 7f5744c00640 1 -- 172.21.15.107:0/4107548792 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f574010a890 con 0x7f5740104740 2024-03-08T21:00:33.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:33.845+0000 7f5736a00640 1 -- 172.21.15.107:0/4107548792 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 21) v1 ==== 137102+0+0 (secure 0 0 0) 0x7f572c0109a0 con 0x7f5740104740 2024-03-08T21:00:33.828 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:33.846+0000 7f5736a00640 1 --2- 172.21.15.107:0/4107548792 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f57240a26b0 0x7f57240a4b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:00:33.828 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:33.846+0000 7f5736a00640 1 -- 172.21.15.107:0/4107548792 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f572c0ce370 con 0x7f5740104740 2024-03-08T21:00:33.828 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:33.846+0000 7f573de00640 1 --2- 172.21.15.107:0/4107548792 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f57240a26b0 0x7f57240a4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:00:33.829 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:33.847+0000 7f573de00640 1 --2- 172.21.15.107:0/4107548792 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f57240a26b0 0x7f57240a4b70 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f5730005ab0 tx=0x7f5730005fd0 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:00:33.837 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:33.855+0000 7f5736a00640 1 -- 172.21.15.107:0/4107548792 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f572c009de0 con 0x7f5740104740 2024-03-08T21:00:33.967 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:00:33 smithi107 ceph-mon[31381]: Manager daemon x is now available 2024-03-08T21:00:33.968 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:00:33 smithi107 ceph-mon[31381]: Standby manager daemon z restarted 2024-03-08T21:00:33.968 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:00:33 smithi107 ceph-mon[31381]: Standby manager daemon z started 2024-03-08T21:00:33.968 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:00:33 smithi107 ceph-mon[31381]: from='mgr.? 172.21.15.204:0/3693169975' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-03-08T21:00:33.968 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:00:33 smithi107 ceph-mon[31381]: from='mgr.? 172.21.15.204:0/3693169975' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-08T21:00:33.968 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:00:33 smithi107 ceph-mon[31381]: from='mgr.? 172.21.15.204:0/3693169975' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-03-08T21:00:33.968 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:00:33 smithi107 ceph-mon[31381]: from='mgr.? 172.21.15.204:0/3693169975' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-08T21:00:33.968 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:00:33 smithi107 ceph-mon[31381]: Standby manager daemon y restarted 2024-03-08T21:00:33.968 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:00:33 smithi107 ceph-mon[31381]: Standby manager daemon y started 2024-03-08T21:00:33.968 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:00:33 smithi107 ceph-mon[31381]: from='mgr.? 172.21.15.149:0/2166939354' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-08T21:00:33.968 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:00:33 smithi107 ceph-mon[31381]: from='mgr.? 172.21.15.149:0/2166939354' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-08T21:00:33.969 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:00:33 smithi107 ceph-mon[31381]: from='mgr.? 172.21.15.149:0/2166939354' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-08T21:00:33.969 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:00:33 smithi107 ceph-mon[31381]: from='mgr.? 172.21.15.149:0/2166939354' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-08T21:00:33.969 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:00:33 smithi107 ceph-mon[31381]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T21:00:33.969 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:00:33 smithi107 ceph-mon[31381]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T21:00:33.969 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:00:33 smithi107 ceph-mon[31381]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-08T21:00:33.969 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:00:33 smithi107 ceph-mon[31381]: from='mgr.15375 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-08T21:00:33.969 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:00:33 smithi107 ceph-mon[31381]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-08T21:00:33.969 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:00:33 smithi107 ceph-mon[31381]: from='mgr.15375 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-08T21:00:34.015 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:00:33 smithi149 ceph-mon[35042]: Manager daemon x is now available 2024-03-08T21:00:34.015 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:00:33 smithi149 ceph-mon[35042]: Standby manager daemon z restarted 2024-03-08T21:00:34.015 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:00:33 smithi149 ceph-mon[35042]: Standby manager daemon z started 2024-03-08T21:00:34.015 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:00:33 smithi149 ceph-mon[35042]: from='mgr.? 172.21.15.204:0/3693169975' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-03-08T21:00:34.016 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:00:33 smithi149 ceph-mon[35042]: from='mgr.? 172.21.15.204:0/3693169975' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-08T21:00:34.016 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:00:33 smithi149 ceph-mon[35042]: from='mgr.? 172.21.15.204:0/3693169975' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-03-08T21:00:34.016 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:00:33 smithi149 ceph-mon[35042]: from='mgr.? 172.21.15.204:0/3693169975' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-08T21:00:34.016 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:00:33 smithi149 ceph-mon[35042]: Standby manager daemon y restarted 2024-03-08T21:00:34.016 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:00:33 smithi149 ceph-mon[35042]: Standby manager daemon y started 2024-03-08T21:00:34.016 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:00:33 smithi149 ceph-mon[35042]: from='mgr.? 172.21.15.149:0/2166939354' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-08T21:00:34.016 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:00:33 smithi149 ceph-mon[35042]: from='mgr.? 172.21.15.149:0/2166939354' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-08T21:00:34.016 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:00:33 smithi149 ceph-mon[35042]: from='mgr.? 172.21.15.149:0/2166939354' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-08T21:00:34.016 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:00:33 smithi149 ceph-mon[35042]: from='mgr.? 172.21.15.149:0/2166939354' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-08T21:00:34.017 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:00:33 smithi149 ceph-mon[35042]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T21:00:34.017 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:00:33 smithi149 ceph-mon[35042]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T21:00:34.017 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:00:33 smithi149 ceph-mon[35042]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-08T21:00:34.017 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:00:33 smithi149 ceph-mon[35042]: from='mgr.15375 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-08T21:00:34.017 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:00:33 smithi149 ceph-mon[35042]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-08T21:00:34.017 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:00:33 smithi149 ceph-mon[35042]: from='mgr.15375 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-08T21:00:34.062 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:34.079+0000 7f5744c00640 1 -- 172.21.15.107:0/4107548792 --> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}) v1 -- 0x7f5740061890 con 0x7f57240a26b0 2024-03-08T21:00:34.188 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:34.206+0000 7f5736a00640 1 -- 172.21.15.107:0/4107548792 <== mgr.15375 v2:172.21.15.107:6800/2792338347 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+26 (secure 0 0 0) 0x7f5740061890 con 0x7f57240a26b0 2024-03-08T21:00:34.189 INFO:teuthology.orchestra.run.smithi107.stdout:Retention added to path / 2024-03-08T21:00:34.192 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:34.210+0000 7f5744c00640 1 -- 172.21.15.107:0/4107548792 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f57240a26b0 msgr2=0x7f57240a4b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:00:34.192 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:34.210+0000 7f5744c00640 1 --2- 172.21.15.107:0/4107548792 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f57240a26b0 0x7f57240a4b70 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f5730005ab0 tx=0x7f5730005fd0 comp rx=0 tx=0).stop 2024-03-08T21:00:34.192 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:34.210+0000 7f5744c00640 1 -- 172.21.15.107:0/4107548792 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5740104740 msgr2=0x7f5740199c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:00:34.192 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:34.210+0000 7f5744c00640 1 --2- 172.21.15.107:0/4107548792 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5740104740 0x7f5740199c30 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f572c00d950 tx=0x7f572c00de10 comp rx=0 tx=0).stop 2024-03-08T21:00:34.193 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:34.210+0000 7f5744c00640 1 -- 172.21.15.107:0/4107548792 shutdown_connections 2024-03-08T21:00:34.193 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:34.210+0000 7f5744c00640 1 --2- 172.21.15.107:0/4107548792 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f57240a26b0 0x7f57240a4b70 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:34.193 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:34.210+0000 7f5744c00640 1 --2- 172.21.15.107:0/4107548792 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5740105100 0x7f57401a0ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:34.194 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:34.210+0000 7f5744c00640 1 --2- 172.21.15.107:0/4107548792 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5740104740 0x7f5740199c30 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:34.194 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:34.210+0000 7f5744c00640 1 --2- 172.21.15.107:0/4107548792 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5740103550 0x7f57401996f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:34.194 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:34.210+0000 7f5744c00640 1 -- 172.21.15.107:0/4107548792 >> 172.21.15.107:0/4107548792 conn(0x7f57400feae0 msgr2=0x7f5740107a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:00:34.194 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:34.210+0000 7f5744c00640 1 -- 172.21.15.107:0/4107548792 shutdown_connections 2024-03-08T21:00:34.194 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:34.210+0000 7f5744c00640 1 -- 172.21.15.107:0/4107548792 wait complete. 2024-03-08T21:00:34.210 DEBUG:teuthology.orchestra.run.smithi107:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-03-08T21:00:34.392 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:34.410+0000 7f093e200640 1 -- 172.21.15.107:0/1916641283 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0938102ed0 msgr2=0x7f0938109760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:00:34.393 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:34.410+0000 7f093e200640 1 --2- 172.21.15.107:0/1916641283 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0938102ed0 0x7f0938109760 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f0930009ca0 tx=0x7f093002c0f0 comp rx=0 tx=0).stop 2024-03-08T21:00:34.393 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:34.410+0000 7f093e200640 1 -- 172.21.15.107:0/1916641283 shutdown_connections 2024-03-08T21:00:34.393 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:34.410+0000 7f093e200640 1 --2- 172.21.15.107:0/1916641283 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0938102ed0 0x7f0938109760 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:34.393 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:34.410+0000 7f093e200640 1 --2- 172.21.15.107:0/1916641283 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0938102510 0x7f0938102990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:34.393 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:34.410+0000 7f093e200640 1 --2- 172.21.15.107:0/1916641283 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0938101320 0x7f0938101740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:34.393 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:34.410+0000 7f093e200640 1 -- 172.21.15.107:0/1916641283 >> 172.21.15.107:0/1916641283 conn(0x7f09380fc8b0 msgr2=0x7f09380fecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:00:34.393 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:34.410+0000 7f093e200640 1 -- 172.21.15.107:0/1916641283 shutdown_connections 2024-03-08T21:00:34.393 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:34.411+0000 7f093e200640 1 -- 172.21.15.107:0/1916641283 wait complete. 2024-03-08T21:00:34.395 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:34.411+0000 7f093e200640 1 Processor -- start 2024-03-08T21:00:34.395 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:34.411+0000 7f093e200640 1 -- start start 2024-03-08T21:00:34.395 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:34.412+0000 7f093e200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0938101320 0x7f0938199610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:00:34.395 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:34.412+0000 7f093e200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0938102510 0x7f0938199b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:00:34.396 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:34.412+0000 7f093e200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0938102ed0 0x7f09381a0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:00:34.396 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:34.412+0000 7f093e200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f093810ce60 con 0x7f0938102ed0 2024-03-08T21:00:34.396 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:34.412+0000 7f093e200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f093810cce0 con 0x7f0938102510 2024-03-08T21:00:34.396 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:34.412+0000 7f093e200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f093810cfe0 con 0x7f0938101320 2024-03-08T21:00:34.396 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:34.412+0000 7f0937e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0938102ed0 0x7f09381a0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:00:34.396 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:34.412+0000 7f0937e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0938102ed0 0x7f09381a0bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:58258/0 (socket says 172.21.15.107:58258) 2024-03-08T21:00:34.396 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:34.412+0000 7f0937e00640 1 -- 172.21.15.107:0/3425327371 learned_addr learned my addr 172.21.15.107:0/3425327371 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:00:34.396 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:34.412+0000 7f092ea00640 1 --2- 172.21.15.107:0/3425327371 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0938102510 0x7f0938199b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:00:34.396 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:34.412+0000 7f0937400640 1 --2- 172.21.15.107:0/3425327371 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0938101320 0x7f0938199610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:00:34.396 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:34.412+0000 7f0937e00640 1 -- 172.21.15.107:0/3425327371 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0938101320 msgr2=0x7f0938199610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:00:34.396 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:34.412+0000 7f0937e00640 1 --2- 172.21.15.107:0/3425327371 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0938101320 0x7f0938199610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:34.396 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:34.412+0000 7f0937e00640 1 -- 172.21.15.107:0/3425327371 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0938102510 msgr2=0x7f0938199b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:00:34.396 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:34.412+0000 7f0937e00640 1 --2- 172.21.15.107:0/3425327371 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0938102510 0x7f0938199b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:34.396 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:34.412+0000 7f0937e00640 1 -- 172.21.15.107:0/3425327371 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09381a1230 con 0x7f0938102ed0 2024-03-08T21:00:34.397 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:34.413+0000 7f0937e00640 1 --2- 172.21.15.107:0/3425327371 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0938102ed0 0x7f09381a0bc0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f093002c700 tx=0x7f09300328a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:00:34.397 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:34.413+0000 7f0934c00640 1 -- 172.21.15.107:0/3425327371 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0930041020 con 0x7f0938102ed0 2024-03-08T21:00:34.397 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:34.413+0000 7f0934c00640 1 -- 172.21.15.107:0/3425327371 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f0930003e20 con 0x7f0938102ed0 2024-03-08T21:00:34.397 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:34.413+0000 7f0934c00640 1 -- 172.21.15.107:0/3425327371 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0930039040 con 0x7f0938102ed0 2024-03-08T21:00:34.397 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:34.413+0000 7f093e200640 1 -- 172.21.15.107:0/3425327371 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09381a14c0 con 0x7f0938102ed0 2024-03-08T21:00:34.397 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:34.413+0000 7f093e200640 1 -- 172.21.15.107:0/3425327371 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09381a1960 con 0x7f0938102ed0 2024-03-08T21:00:34.397 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:34.414+0000 7f093e200640 1 -- 172.21.15.107:0/3425327371 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09381a1bc0 con 0x7f0938102ed0 2024-03-08T21:00:34.398 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:34.416+0000 7f0934c00640 1 -- 172.21.15.107:0/3425327371 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 21) v1 ==== 137102+0+0 (secure 0 0 0) 0x7f0930044020 con 0x7f0938102ed0 2024-03-08T21:00:34.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:34.417+0000 7f0934c00640 1 --2- 172.21.15.107:0/3425327371 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f09180a2730 0x7f09180a4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:00:34.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:34.417+0000 7f0934c00640 1 -- 172.21.15.107:0/3425327371 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f09300f08d0 con 0x7f0938102ed0 2024-03-08T21:00:34.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:34.418+0000 7f0937400640 1 --2- 172.21.15.107:0/3425327371 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f09180a2730 0x7f09180a4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:00:34.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:34.418+0000 7f0937400640 1 --2- 172.21.15.107:0/3425327371 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f09180a2730 0x7f09180a4bf0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f0938102370 tx=0x7f0928005f90 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:00:34.405 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:34.422+0000 7f0934c00640 1 -- 172.21.15.107:0/3425327371 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f093004b020 con 0x7f0938102ed0 2024-03-08T21:00:34.535 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:00:34 smithi204 ceph-mon[34935]: mgrmap e21: x(active, since 1.09548s), standbys: y, z 2024-03-08T21:00:34.535 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:00:34 smithi204 ceph-mon[34935]: from='client.15405 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-03-08T21:00:34.535 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:00:34 smithi204 ceph-mon[34935]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-08T21:00:34.554 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:34.571+0000 7f0934c00640 1 -- 172.21.15.107:0/3425327371 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mgrmap(e 22) v1 ==== 137411+0+0 (secure 0 0 0) 0x7f09300b9a60 con 0x7f0938102ed0 2024-03-08T21:00:34.641 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:34.659+0000 7f093e200640 1 -- 172.21.15.107:0/3425327371 --> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) v1 -- 0x7f0938074c40 con 0x7f09180a2730 2024-03-08T21:00:34.659 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:34.677+0000 7f0934c00640 1 -- 172.21.15.107:0/3425327371 <== mgr.15375 v2:172.21.15.107:6800/2792338347 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+298 (secure 0 0 0) 0x7f0938074c40 con 0x7f09180a2730 2024-03-08T21:00:34.660 INFO:teuthology.orchestra.run.smithi107.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-03-08T00:00:00", "created": "2024-03-08T21:00:33", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2024-03-08T21:00:34.663 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:34.679+0000 7f093e200640 1 -- 172.21.15.107:0/3425327371 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f09180a2730 msgr2=0x7f09180a4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:00:34.663 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:34.679+0000 7f093e200640 1 --2- 172.21.15.107:0/3425327371 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f09180a2730 0x7f09180a4bf0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f0938102370 tx=0x7f0928005f90 comp rx=0 tx=0).stop 2024-03-08T21:00:34.663 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:34.679+0000 7f093e200640 1 -- 172.21.15.107:0/3425327371 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0938102ed0 msgr2=0x7f09381a0bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:00:34.663 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:34.679+0000 7f093e200640 1 --2- 172.21.15.107:0/3425327371 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0938102ed0 0x7f09381a0bc0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f093002c700 tx=0x7f09300328a0 comp rx=0 tx=0).stop 2024-03-08T21:00:34.663 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:34.679+0000 7f093e200640 1 -- 172.21.15.107:0/3425327371 shutdown_connections 2024-03-08T21:00:34.663 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:34.679+0000 7f093e200640 1 --2- 172.21.15.107:0/3425327371 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f09180a2730 0x7f09180a4bf0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:34.664 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:34.679+0000 7f093e200640 1 --2- 172.21.15.107:0/3425327371 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0938102ed0 0x7f09381a0bc0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:34.664 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:34.679+0000 7f093e200640 1 --2- 172.21.15.107:0/3425327371 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0938102510 0x7f0938199b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:34.664 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:34.679+0000 7f093e200640 1 --2- 172.21.15.107:0/3425327371 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0938101320 0x7f0938199610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:34.664 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:34.679+0000 7f093e200640 1 -- 172.21.15.107:0/3425327371 >> 172.21.15.107:0/3425327371 conn(0x7f09380fc8b0 msgr2=0x7f0938073520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:00:34.664 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:34.680+0000 7f093e200640 1 -- 172.21.15.107:0/3425327371 shutdown_connections 2024-03-08T21:00:34.664 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:34.680+0000 7f093e200640 1 -- 172.21.15.107:0/3425327371 wait complete. 2024-03-08T21:00:34.678 DEBUG:teuthology.orchestra.run.smithi107:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-03-08T21:00:34.864 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:34.882+0000 7f2b39600640 1 -- 172.21.15.107:0/34436432 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2b341046e0 msgr2=0x7f2b34106ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:00:34.865 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:34.882+0000 7f2b39600640 1 --2- 172.21.15.107:0/34436432 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2b341046e0 0x7f2b34106ad0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f2b1c009900 tx=0x7f2b1c02bc30 comp rx=0 tx=0).stop 2024-03-08T21:00:34.865 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:34.882+0000 7f2b39600640 1 -- 172.21.15.107:0/34436432 shutdown_connections 2024-03-08T21:00:34.865 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:34.882+0000 7f2b39600640 1 --2- 172.21.15.107:0/34436432 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2b34107010 0x7f2b34109460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:34.865 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:34.882+0000 7f2b39600640 1 --2- 172.21.15.107:0/34436432 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2b341046e0 0x7f2b34106ad0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:34.865 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:34.882+0000 7f2b39600640 1 --2- 172.21.15.107:0/34436432 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2b34101db0 0x7f2b341041a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:34.865 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:34.882+0000 7f2b39600640 1 -- 172.21.15.107:0/34436432 >> 172.21.15.107:0/34436432 conn(0x7f2b340fb7a0 msgr2=0x7f2b340fdbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:00:34.865 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:34.882+0000 7f2b39600640 1 -- 172.21.15.107:0/34436432 shutdown_connections 2024-03-08T21:00:34.865 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:34.882+0000 7f2b39600640 1 -- 172.21.15.107:0/34436432 wait complete. 2024-03-08T21:00:34.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:34.883+0000 7f2b39600640 1 Processor -- start 2024-03-08T21:00:34.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:34.883+0000 7f2b39600640 1 -- start start 2024-03-08T21:00:34.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:34.883+0000 7f2b39600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2b34101db0 0x7f2b34197520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:00:34.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:34.883+0000 7f2b39600640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2b341046e0 0x7f2b34197a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:00:34.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:34.883+0000 7f2b39600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2b34107010 0x7f2b3419ead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:00:34.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:34.883+0000 7f2b39600640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b3410ce70 con 0x7f2b34101db0 2024-03-08T21:00:34.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:34.883+0000 7f2b39600640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b3410ccf0 con 0x7f2b341046e0 2024-03-08T21:00:34.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:34.883+0000 7f2b39600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b3410cff0 con 0x7f2b34107010 2024-03-08T21:00:34.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:34.884+0000 7f2b32a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2b34101db0 0x7f2b34197520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:00:34.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:34.884+0000 7f2b32a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2b34101db0 0x7f2b34197520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:58290/0 (socket says 172.21.15.107:58290) 2024-03-08T21:00:34.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:34.884+0000 7f2b32a00640 1 -- 172.21.15.107:0/1590327896 learned_addr learned my addr 172.21.15.107:0/1590327896 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:00:34.868 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:34.884+0000 7f2b2be00640 1 --2- 172.21.15.107:0/1590327896 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2b341046e0 0x7f2b34197a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:00:34.868 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:34.884+0000 7f2b33400640 1 --2- 172.21.15.107:0/1590327896 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2b34107010 0x7f2b3419ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:00:34.869 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:34.884+0000 7f2b2be00640 1 -- 172.21.15.107:0/1590327896 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2b34107010 msgr2=0x7f2b3419ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:00:34.869 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:34.884+0000 7f2b2be00640 1 --2- 172.21.15.107:0/1590327896 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2b34107010 0x7f2b3419ead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:34.869 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:34.884+0000 7f2b2be00640 1 -- 172.21.15.107:0/1590327896 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2b34101db0 msgr2=0x7f2b34197520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:00:34.869 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:34.884+0000 7f2b2be00640 1 --2- 172.21.15.107:0/1590327896 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2b34101db0 0x7f2b34197520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:34.869 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:34.884+0000 7f2b2be00640 1 -- 172.21.15.107:0/1590327896 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b3419f140 con 0x7f2b341046e0 2024-03-08T21:00:34.869 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:34.884+0000 7f2b32a00640 1 --2- 172.21.15.107:0/1590327896 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2b34101db0 0x7f2b34197520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T21:00:34.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:34.884+0000 7f2b2be00640 1 --2- 172.21.15.107:0/1590327896 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2b341046e0 0x7f2b34197a60 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f2b1c036040 tx=0x7f2b1c005dc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:00:34.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:34.885+0000 7f2b2b400640 1 -- 172.21.15.107:0/1590327896 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b1c043020 con 0x7f2b341046e0 2024-03-08T21:00:34.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:34.885+0000 7f2b2b400640 1 -- 172.21.15.107:0/1590327896 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f2b1c004350 con 0x7f2b341046e0 2024-03-08T21:00:34.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:34.885+0000 7f2b39600640 1 -- 172.21.15.107:0/1590327896 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b3419f3d0 con 0x7f2b341046e0 2024-03-08T21:00:34.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:34.885+0000 7f2b2b400640 1 -- 172.21.15.107:0/1590327896 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b1c003ab0 con 0x7f2b341046e0 2024-03-08T21:00:34.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:34.885+0000 7f2b39600640 1 -- 172.21.15.107:0/1590327896 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b3419f940 con 0x7f2b341046e0 2024-03-08T21:00:34.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:34.886+0000 7f2b39600640 1 -- 172.21.15.107:0/1590327896 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b340fd290 con 0x7f2b341046e0 2024-03-08T21:00:34.871 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:34.888+0000 7f2b2b400640 1 -- 172.21.15.107:0/1590327896 <== mon.2 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 22) v1 ==== 137411+0+0 (secure 0 0 0) 0x7f2b1c046020 con 0x7f2b341046e0 2024-03-08T21:00:34.871 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:34.889+0000 7f2b2b400640 1 --2- 172.21.15.107:0/1590327896 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f2b180a2a90 0x7f2b180a4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:00:34.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:34.889+0000 7f2b2b400640 1 -- 172.21.15.107:0/1590327896 <== mon.2 v2:172.21.15.149:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f2b1c0f13c0 con 0x7f2b341046e0 2024-03-08T21:00:34.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:34.889+0000 7f2b32a00640 1 --2- 172.21.15.107:0/1590327896 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f2b180a2a90 0x7f2b180a4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:00:34.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:34.890+0000 7f2b32a00640 1 --2- 172.21.15.107:0/1590327896 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f2b180a2a90 0x7f2b180a4f50 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f2b24011550 tx=0x7f2b24005f90 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:00:34.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:34.893+0000 7f2b2b400640 1 -- 172.21.15.107:0/1590327896 <== mon.2 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f2b1c04d020 con 0x7f2b341046e0 2024-03-08T21:00:34.967 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:00:34 smithi107 ceph-mon[31381]: mgrmap e21: x(active, since 1.09548s), standbys: y, z 2024-03-08T21:00:34.967 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:00:34 smithi107 ceph-mon[31381]: from='client.15405 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-03-08T21:00:34.967 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:00:34 smithi107 ceph-mon[31381]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-08T21:00:35.016 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:00:34 smithi149 ceph-mon[35042]: mgrmap e21: x(active, since 1.09548s), standbys: y, z 2024-03-08T21:00:35.017 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:00:34 smithi149 ceph-mon[35042]: from='client.15405 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-03-08T21:00:35.017 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:00:34 smithi149 ceph-mon[35042]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-08T21:00:35.106 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:35.124+0000 7f2b39600640 1 -- 172.21.15.107:0/1590327896 --> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f2b34061730 con 0x7f2b180a2a90 2024-03-08T21:00:35.115 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:35.133+0000 7f2b2b400640 1 -- 172.21.15.107:0/1590327896 <== mgr.15375 v2:172.21.15.107:6800/2792338347 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+10 (secure 0 0 0) 0x7f2b34061730 con 0x7f2b180a2a90 2024-03-08T21:00:35.116 INFO:teuthology.orchestra.run.smithi107.stdout:/ 1m 6m3h 2024-03-08T21:00:35.118 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:35.136+0000 7f2b39600640 1 -- 172.21.15.107:0/1590327896 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f2b180a2a90 msgr2=0x7f2b180a4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:00:35.119 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:35.136+0000 7f2b39600640 1 --2- 172.21.15.107:0/1590327896 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f2b180a2a90 0x7f2b180a4f50 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f2b24011550 tx=0x7f2b24005f90 comp rx=0 tx=0).stop 2024-03-08T21:00:35.119 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:35.136+0000 7f2b39600640 1 -- 172.21.15.107:0/1590327896 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2b341046e0 msgr2=0x7f2b34197a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:00:35.119 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:35.136+0000 7f2b39600640 1 --2- 172.21.15.107:0/1590327896 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2b341046e0 0x7f2b34197a60 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f2b1c036040 tx=0x7f2b1c005dc0 comp rx=0 tx=0).stop 2024-03-08T21:00:35.119 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:35.136+0000 7f2b39600640 1 -- 172.21.15.107:0/1590327896 shutdown_connections 2024-03-08T21:00:35.119 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:35.136+0000 7f2b39600640 1 --2- 172.21.15.107:0/1590327896 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f2b180a2a90 0x7f2b180a4f50 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:35.119 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:35.136+0000 7f2b39600640 1 --2- 172.21.15.107:0/1590327896 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2b34107010 0x7f2b3419ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:35.119 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:35.136+0000 7f2b39600640 1 --2- 172.21.15.107:0/1590327896 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2b341046e0 0x7f2b34197a60 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:35.119 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:35.136+0000 7f2b39600640 1 --2- 172.21.15.107:0/1590327896 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2b34101db0 0x7f2b34197520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:35.119 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:35.136+0000 7f2b39600640 1 -- 172.21.15.107:0/1590327896 >> 172.21.15.107:0/1590327896 conn(0x7f2b340fb7a0 msgr2=0x7f2b341028f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:00:35.120 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:35.136+0000 7f2b39600640 1 -- 172.21.15.107:0/1590327896 shutdown_connections 2024-03-08T21:00:35.120 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:00:35.136+0000 7f2b39600640 1 -- 172.21.15.107:0/1590327896 wait complete. 2024-03-08T21:00:35.135 DEBUG:teuthology.orchestra.run.smithi107:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2024-03-08T21:00:35.209 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2024-03-08T21:00:35.394 INFO:teuthology.run_tasks:Running task background_exec... 2024-03-08T21:00:35.479 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi107.front.sepia.ceph.com 2024-03-08T21:00:35.479 DEBUG:teuthology.orchestra.run.smithi107:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2024-03-08T21:00:35.489 INFO:teuthology.run_tasks:Running task workunit... 2024-03-08T21:00:35.570 INFO:tasks.workunit:Pulling workunits from ref 7d63482aa89628d733e298984a762389b625c761 2024-03-08T21:00:35.571 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-03-08T21:00:35.571 INFO:tasks.workunit:timeout=3h 2024-03-08T21:00:35.571 INFO:tasks.workunit:cleanup=True 2024-03-08T21:00:35.571 DEBUG:teuthology.orchestra.run.smithi107:> stat -- /home/ubuntu/cephtest/mnt.0 2024-03-08T21:00:35.611 INFO:teuthology.orchestra.run.smithi107.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-03-08T21:00:35.611 INFO:teuthology.orchestra.run.smithi107.stdout: Size: 0 Blocks: 1 IO Block: 4194304 directory 2024-03-08T21:00:35.611 INFO:teuthology.orchestra.run.smithi107.stdout:Device: 76h/118d Inode: 1 Links: 2 2024-03-08T21:00:35.611 INFO:teuthology.orchestra.run.smithi107.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-08T21:00:35.611 INFO:teuthology.orchestra.run.smithi107.stdout:Context: system_u:object_r:fusefs_t:s0 2024-03-08T21:00:35.611 INFO:teuthology.orchestra.run.smithi107.stdout:Access: 2024-03-08 20:57:57.387801505 +0000 2024-03-08T21:00:35.612 INFO:teuthology.orchestra.run.smithi107.stdout:Modify: 2024-03-08 20:57:57.387801505 +0000 2024-03-08T21:00:35.612 INFO:teuthology.orchestra.run.smithi107.stdout:Change: 2024-03-08 21:00:20.563047771 +0000 2024-03-08T21:00:35.612 INFO:teuthology.orchestra.run.smithi107.stdout: Birth: - 2024-03-08T21:00:35.612 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-03-08T21:00:35.612 DEBUG:teuthology.orchestra.run.smithi107:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-03-08T21:00:35.749 DEBUG:teuthology.orchestra.run.smithi107:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 7d63482aa89628d733e298984a762389b625c761 2024-03-08T21:00:35.801 INFO:tasks.workunit.client.0.smithi107.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-03-08T21:00:35.877 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:00:35 smithi204 ceph-mon[34935]: [08/Mar/2024:21:00:33] ENGINE Bus STARTING 2024-03-08T21:00:35.878 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:00:35 smithi204 ceph-mon[34935]: [08/Mar/2024:21:00:33] ENGINE Serving on http://172.21.15.107:8765 2024-03-08T21:00:35.878 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:00:35 smithi204 ceph-mon[34935]: [08/Mar/2024:21:00:33] ENGINE Serving on https://172.21.15.107:7150 2024-03-08T21:00:35.878 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:00:35 smithi204 ceph-mon[34935]: [08/Mar/2024:21:00:33] ENGINE Bus STARTED 2024-03-08T21:00:35.878 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:00:35 smithi204 ceph-mon[34935]: from='client.15447 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-03-08T21:00:35.878 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:00:35 smithi204 ceph-mon[34935]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-08T21:00:35.878 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:00:35 smithi204 ceph-mon[34935]: mgrmap e22: x(active, since 2s), standbys: y, z 2024-03-08T21:00:35.878 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:00:35 smithi204 ceph-mon[34935]: from='mgr.15375 ' entity='mgr.x' 2024-03-08T21:00:35.878 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:00:35 smithi204 ceph-mon[34935]: from='mgr.15375 ' entity='mgr.x' 2024-03-08T21:00:35.967 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:00:35 smithi107 ceph-mon[31381]: [08/Mar/2024:21:00:33] ENGINE Bus STARTING 2024-03-08T21:00:35.967 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:00:35 smithi107 ceph-mon[31381]: [08/Mar/2024:21:00:33] ENGINE Serving on http://172.21.15.107:8765 2024-03-08T21:00:35.967 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:00:35 smithi107 ceph-mon[31381]: [08/Mar/2024:21:00:33] ENGINE Serving on https://172.21.15.107:7150 2024-03-08T21:00:35.967 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:00:35 smithi107 ceph-mon[31381]: [08/Mar/2024:21:00:33] ENGINE Bus STARTED 2024-03-08T21:00:35.967 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:00:35 smithi107 ceph-mon[31381]: from='client.15447 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-03-08T21:00:35.967 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:00:35 smithi107 ceph-mon[31381]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-08T21:00:35.967 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:00:35 smithi107 ceph-mon[31381]: mgrmap e22: x(active, since 2s), standbys: y, z 2024-03-08T21:00:35.967 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:00:35 smithi107 ceph-mon[31381]: from='mgr.15375 ' entity='mgr.x' 2024-03-08T21:00:35.967 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:00:35 smithi107 ceph-mon[31381]: from='mgr.15375 ' entity='mgr.x' 2024-03-08T21:00:36.014 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:00:35 smithi149 ceph-mon[35042]: [08/Mar/2024:21:00:33] ENGINE Bus STARTING 2024-03-08T21:00:36.015 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:00:35 smithi149 ceph-mon[35042]: [08/Mar/2024:21:00:33] ENGINE Serving on http://172.21.15.107:8765 2024-03-08T21:00:36.015 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:00:35 smithi149 ceph-mon[35042]: [08/Mar/2024:21:00:33] ENGINE Serving on https://172.21.15.107:7150 2024-03-08T21:00:36.015 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:00:35 smithi149 ceph-mon[35042]: [08/Mar/2024:21:00:33] ENGINE Bus STARTED 2024-03-08T21:00:36.015 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:00:35 smithi149 ceph-mon[35042]: from='client.15447 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-03-08T21:00:36.015 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:00:35 smithi149 ceph-mon[35042]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-08T21:00:36.015 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:00:35 smithi149 ceph-mon[35042]: mgrmap e22: x(active, since 2s), standbys: y, z 2024-03-08T21:00:36.015 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:00:35 smithi149 ceph-mon[35042]: from='mgr.15375 ' entity='mgr.x' 2024-03-08T21:00:36.015 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:00:35 smithi149 ceph-mon[35042]: from='mgr.15375 ' entity='mgr.x' 2024-03-08T21:00:36.877 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:00:36 smithi204 ceph-mon[34935]: from='client.15453 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-03-08T21:00:36.877 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:00:36 smithi204 ceph-mon[34935]: from='client.34889 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-08T21:00:36.877 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:00:36 smithi204 ceph-mon[34935]: from='mgr.15375 ' entity='mgr.x' 2024-03-08T21:00:36.877 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:00:36 smithi204 ceph-mon[34935]: from='mgr.15375 ' entity='mgr.x' 2024-03-08T21:00:36.878 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:00:36 smithi204 ceph-mon[34935]: from='mgr.15375 ' entity='mgr.x' 2024-03-08T21:00:36.878 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:00:36 smithi204 ceph-mon[34935]: from='mgr.15375 ' entity='mgr.x' 2024-03-08T21:00:36.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:00:36 smithi107 ceph-mon[31381]: from='client.15453 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-03-08T21:00:36.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:00:36 smithi107 ceph-mon[31381]: from='client.34889 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-08T21:00:36.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:00:36 smithi107 ceph-mon[31381]: from='mgr.15375 ' entity='mgr.x' 2024-03-08T21:00:36.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:00:36 smithi107 ceph-mon[31381]: from='mgr.15375 ' entity='mgr.x' 2024-03-08T21:00:36.967 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:00:36 smithi107 ceph-mon[31381]: from='mgr.15375 ' entity='mgr.x' 2024-03-08T21:00:36.967 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:00:36 smithi107 ceph-mon[31381]: from='mgr.15375 ' entity='mgr.x' 2024-03-08T21:00:37.014 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:00:36 smithi149 ceph-mon[35042]: from='client.15453 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-03-08T21:00:37.014 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:00:36 smithi149 ceph-mon[35042]: from='client.34889 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-08T21:00:37.014 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:00:36 smithi149 ceph-mon[35042]: from='mgr.15375 ' entity='mgr.x' 2024-03-08T21:00:37.014 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:00:36 smithi149 ceph-mon[35042]: from='mgr.15375 ' entity='mgr.x' 2024-03-08T21:00:37.014 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:00:36 smithi149 ceph-mon[35042]: from='mgr.15375 ' entity='mgr.x' 2024-03-08T21:00:37.014 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:00:36 smithi149 ceph-mon[35042]: from='mgr.15375 ' entity='mgr.x' 2024-03-08T21:00:37.877 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:00:37 smithi204 ceph-mon[34935]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-08T21:00:37.877 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:00:37 smithi204 ceph-mon[34935]: mgrmap e23: x(active, since 4s), standbys: y, z 2024-03-08T21:00:37.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:00:37 smithi107 ceph-mon[31381]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-08T21:00:37.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:00:37 smithi107 ceph-mon[31381]: mgrmap e23: x(active, since 4s), standbys: y, z 2024-03-08T21:00:38.014 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:00:37 smithi149 ceph-mon[35042]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-08T21:00:38.014 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:00:37 smithi149 ceph-mon[35042]: mgrmap e23: x(active, since 4s), standbys: y, z 2024-03-08T21:00:38.877 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:00:38 smithi204 ceph-mon[34935]: from='mgr.15375 ' entity='mgr.x' 2024-03-08T21:00:38.878 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:00:38 smithi204 ceph-mon[34935]: from='mgr.15375 ' entity='mgr.x' 2024-03-08T21:00:38.878 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:00:38 smithi204 ceph-mon[34935]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-08T21:00:38.878 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:00:38 smithi204 ceph-mon[34935]: from='mgr.15375 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-08T21:00:38.878 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:00:38 smithi204 ceph-mon[34935]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-08T21:00:38.878 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:00:38 smithi204 ceph-mon[34935]: from='mgr.15375 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-08T21:00:38.878 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:00:38 smithi204 ceph-mon[34935]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-08T21:00:38.878 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:00:38 smithi204 ceph-mon[34935]: from='mgr.15375 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-08T21:00:38.878 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:00:38 smithi204 ceph-mon[34935]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-08T21:00:38.878 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:00:38 smithi204 ceph-mon[34935]: from='mgr.15375 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-08T21:00:38.878 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:00:38 smithi204 ceph-mon[34935]: Adjusting osd_memory_target on smithi107 to 176.7M 2024-03-08T21:00:38.878 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:00:38 smithi204 ceph-mon[34935]: Unable to set osd_memory_target on smithi107 to 185328640: error parsing value: Value '185328640' is below minimum 939524096 2024-03-08T21:00:38.879 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:00:38 smithi204 ceph-mon[34935]: from='mgr.15375 ' entity='mgr.x' 2024-03-08T21:00:38.879 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:00:38 smithi204 ceph-mon[34935]: from='mgr.15375 ' entity='mgr.x' 2024-03-08T21:00:38.879 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:00:38 smithi204 ceph-mon[34935]: from='mgr.15375 ' entity='mgr.x' 2024-03-08T21:00:38.879 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:00:38 smithi204 ceph-mon[34935]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-08T21:00:38.879 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:00:38 smithi204 ceph-mon[34935]: from='mgr.15375 ' entity='mgr.x' 2024-03-08T21:00:38.879 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:00:38 smithi204 ceph-mon[34935]: from='mgr.15375 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-08T21:00:38.879 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:00:38 smithi204 ceph-mon[34935]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi149", "name": "osd_memory_target"}]: dispatch 2024-03-08T21:00:38.879 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:00:38 smithi204 ceph-mon[34935]: from='mgr.15375 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi149", "name": "osd_memory_target"}]: dispatch 2024-03-08T21:00:38.879 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:00:38 smithi204 ceph-mon[34935]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-08T21:00:38.879 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:00:38 smithi204 ceph-mon[34935]: from='mgr.15375 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-08T21:00:38.879 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:00:38 smithi204 ceph-mon[34935]: from='mgr.15375 ' entity='mgr.x' cmd='[{"prefix": "config rm", "who": "osd/host:smithi149", "name": "osd_memory_target"}]': finished 2024-03-08T21:00:38.879 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:00:38 smithi204 ceph-mon[34935]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-08T21:00:38.879 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:00:38 smithi204 ceph-mon[34935]: from='mgr.15375 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-08T21:00:38.879 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:00:38 smithi204 ceph-mon[34935]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-08T21:00:38.880 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:00:38 smithi204 ceph-mon[34935]: from='mgr.15375 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-08T21:00:38.880 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:00:38 smithi204 ceph-mon[34935]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T21:00:38.880 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:00:38 smithi204 ceph-mon[34935]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T21:00:38.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:00:38 smithi107 ceph-mon[31381]: from='mgr.15375 ' entity='mgr.x' 2024-03-08T21:00:38.967 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:00:38 smithi107 ceph-mon[31381]: from='mgr.15375 ' entity='mgr.x' 2024-03-08T21:00:38.967 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:00:38 smithi107 ceph-mon[31381]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-08T21:00:38.967 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:00:38 smithi107 ceph-mon[31381]: from='mgr.15375 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-08T21:00:38.967 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:00:38 smithi107 ceph-mon[31381]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-08T21:00:38.967 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:00:38 smithi107 ceph-mon[31381]: from='mgr.15375 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-08T21:00:38.967 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:00:38 smithi107 ceph-mon[31381]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-08T21:00:38.967 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:00:38 smithi107 ceph-mon[31381]: from='mgr.15375 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-08T21:00:38.967 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:00:38 smithi107 ceph-mon[31381]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-08T21:00:38.967 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:00:38 smithi107 ceph-mon[31381]: from='mgr.15375 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-08T21:00:38.967 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:00:38 smithi107 ceph-mon[31381]: Adjusting osd_memory_target on smithi107 to 176.7M 2024-03-08T21:00:38.967 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:00:38 smithi107 ceph-mon[31381]: Unable to set osd_memory_target on smithi107 to 185328640: error parsing value: Value '185328640' is below minimum 939524096 2024-03-08T21:00:38.967 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:00:38 smithi107 ceph-mon[31381]: from='mgr.15375 ' entity='mgr.x' 2024-03-08T21:00:38.967 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:00:38 smithi107 ceph-mon[31381]: from='mgr.15375 ' entity='mgr.x' 2024-03-08T21:00:38.967 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:00:38 smithi107 ceph-mon[31381]: from='mgr.15375 ' entity='mgr.x' 2024-03-08T21:00:38.968 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:00:38 smithi107 ceph-mon[31381]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-08T21:00:38.968 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:00:38 smithi107 ceph-mon[31381]: from='mgr.15375 ' entity='mgr.x' 2024-03-08T21:00:38.968 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:00:38 smithi107 ceph-mon[31381]: from='mgr.15375 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-08T21:00:38.968 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:00:38 smithi107 ceph-mon[31381]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi149", "name": "osd_memory_target"}]: dispatch 2024-03-08T21:00:38.968 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:00:38 smithi107 ceph-mon[31381]: from='mgr.15375 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi149", "name": "osd_memory_target"}]: dispatch 2024-03-08T21:00:38.968 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:00:38 smithi107 ceph-mon[31381]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-08T21:00:38.968 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:00:38 smithi107 ceph-mon[31381]: from='mgr.15375 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-08T21:00:38.968 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:00:38 smithi107 ceph-mon[31381]: from='mgr.15375 ' entity='mgr.x' cmd='[{"prefix": "config rm", "who": "osd/host:smithi149", "name": "osd_memory_target"}]': finished 2024-03-08T21:00:38.968 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:00:38 smithi107 ceph-mon[31381]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-08T21:00:38.968 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:00:38 smithi107 ceph-mon[31381]: from='mgr.15375 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-08T21:00:38.968 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:00:38 smithi107 ceph-mon[31381]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-08T21:00:38.968 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:00:38 smithi107 ceph-mon[31381]: from='mgr.15375 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-08T21:00:38.968 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:00:38 smithi107 ceph-mon[31381]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T21:00:38.968 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:00:38 smithi107 ceph-mon[31381]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T21:00:39.014 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:00:38 smithi149 ceph-mon[35042]: from='mgr.15375 ' entity='mgr.x' 2024-03-08T21:00:39.014 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:00:38 smithi149 ceph-mon[35042]: from='mgr.15375 ' entity='mgr.x' 2024-03-08T21:00:39.015 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:00:38 smithi149 ceph-mon[35042]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-08T21:00:39.015 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:00:38 smithi149 ceph-mon[35042]: from='mgr.15375 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-08T21:00:39.015 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:00:38 smithi149 ceph-mon[35042]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-08T21:00:39.015 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:00:38 smithi149 ceph-mon[35042]: from='mgr.15375 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-08T21:00:39.015 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:00:38 smithi149 ceph-mon[35042]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-08T21:00:39.015 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:00:38 smithi149 ceph-mon[35042]: from='mgr.15375 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-08T21:00:39.015 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:00:38 smithi149 ceph-mon[35042]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-08T21:00:39.015 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:00:38 smithi149 ceph-mon[35042]: from='mgr.15375 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-08T21:00:39.015 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:00:38 smithi149 ceph-mon[35042]: Adjusting osd_memory_target on smithi107 to 176.7M 2024-03-08T21:00:39.015 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:00:38 smithi149 ceph-mon[35042]: Unable to set osd_memory_target on smithi107 to 185328640: error parsing value: Value '185328640' is below minimum 939524096 2024-03-08T21:00:39.015 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:00:38 smithi149 ceph-mon[35042]: from='mgr.15375 ' entity='mgr.x' 2024-03-08T21:00:39.015 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:00:38 smithi149 ceph-mon[35042]: from='mgr.15375 ' entity='mgr.x' 2024-03-08T21:00:39.015 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:00:38 smithi149 ceph-mon[35042]: from='mgr.15375 ' entity='mgr.x' 2024-03-08T21:00:39.015 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:00:38 smithi149 ceph-mon[35042]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-08T21:00:39.016 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:00:38 smithi149 ceph-mon[35042]: from='mgr.15375 ' entity='mgr.x' 2024-03-08T21:00:39.016 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:00:38 smithi149 ceph-mon[35042]: from='mgr.15375 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-08T21:00:39.016 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:00:38 smithi149 ceph-mon[35042]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi149", "name": "osd_memory_target"}]: dispatch 2024-03-08T21:00:39.016 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:00:38 smithi149 ceph-mon[35042]: from='mgr.15375 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi149", "name": "osd_memory_target"}]: dispatch 2024-03-08T21:00:39.016 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:00:38 smithi149 ceph-mon[35042]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-08T21:00:39.016 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:00:38 smithi149 ceph-mon[35042]: from='mgr.15375 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-08T21:00:39.016 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:00:38 smithi149 ceph-mon[35042]: from='mgr.15375 ' entity='mgr.x' cmd='[{"prefix": "config rm", "who": "osd/host:smithi149", "name": "osd_memory_target"}]': finished 2024-03-08T21:00:39.016 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:00:38 smithi149 ceph-mon[35042]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-08T21:00:39.016 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:00:38 smithi149 ceph-mon[35042]: from='mgr.15375 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-08T21:00:39.016 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:00:38 smithi149 ceph-mon[35042]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-08T21:00:39.016 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:00:38 smithi149 ceph-mon[35042]: from='mgr.15375 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-08T21:00:39.016 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:00:38 smithi149 ceph-mon[35042]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T21:00:39.016 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:00:38 smithi149 ceph-mon[35042]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T21:00:39.877 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:00:39 smithi204 ceph-mon[34935]: Adjusting osd_memory_target on smithi204 to 176.7M 2024-03-08T21:00:39.877 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:00:39 smithi204 ceph-mon[34935]: Unable to set osd_memory_target on smithi204 to 185360896: error parsing value: Value '185360896' is below minimum 939524096 2024-03-08T21:00:39.877 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:00:39 smithi204 ceph-mon[34935]: Updating smithi107:/etc/ceph/ceph.conf 2024-03-08T21:00:39.877 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:00:39 smithi204 ceph-mon[34935]: Updating smithi149:/etc/ceph/ceph.conf 2024-03-08T21:00:39.877 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:00:39 smithi204 ceph-mon[34935]: Updating smithi204:/etc/ceph/ceph.conf 2024-03-08T21:00:39.878 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:00:39 smithi204 ceph-mon[34935]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-08T21:00:39.878 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:00:39 smithi204 ceph-mon[34935]: Updating smithi107:/var/lib/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/config/ceph.conf 2024-03-08T21:00:39.878 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:00:39 smithi204 ceph-mon[34935]: Updating smithi149:/var/lib/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/config/ceph.conf 2024-03-08T21:00:39.878 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:00:39 smithi204 ceph-mon[34935]: Updating smithi204:/var/lib/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/config/ceph.conf 2024-03-08T21:00:39.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:00:39 smithi107 ceph-mon[31381]: Adjusting osd_memory_target on smithi204 to 176.7M 2024-03-08T21:00:39.967 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:00:39 smithi107 ceph-mon[31381]: Unable to set osd_memory_target on smithi204 to 185360896: error parsing value: Value '185360896' is below minimum 939524096 2024-03-08T21:00:39.967 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:00:39 smithi107 ceph-mon[31381]: Updating smithi107:/etc/ceph/ceph.conf 2024-03-08T21:00:39.967 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:00:39 smithi107 ceph-mon[31381]: Updating smithi149:/etc/ceph/ceph.conf 2024-03-08T21:00:39.967 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:00:39 smithi107 ceph-mon[31381]: Updating smithi204:/etc/ceph/ceph.conf 2024-03-08T21:00:39.967 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:00:39 smithi107 ceph-mon[31381]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-08T21:00:39.967 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:00:39 smithi107 ceph-mon[31381]: Updating smithi107:/var/lib/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/config/ceph.conf 2024-03-08T21:00:39.967 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:00:39 smithi107 ceph-mon[31381]: Updating smithi149:/var/lib/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/config/ceph.conf 2024-03-08T21:00:39.967 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:00:39 smithi107 ceph-mon[31381]: Updating smithi204:/var/lib/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/config/ceph.conf 2024-03-08T21:00:40.014 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:00:39 smithi149 ceph-mon[35042]: Adjusting osd_memory_target on smithi204 to 176.7M 2024-03-08T21:00:40.014 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:00:39 smithi149 ceph-mon[35042]: Unable to set osd_memory_target on smithi204 to 185360896: error parsing value: Value '185360896' is below minimum 939524096 2024-03-08T21:00:40.014 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:00:39 smithi149 ceph-mon[35042]: Updating smithi107:/etc/ceph/ceph.conf 2024-03-08T21:00:40.014 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:00:39 smithi149 ceph-mon[35042]: Updating smithi149:/etc/ceph/ceph.conf 2024-03-08T21:00:40.014 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:00:39 smithi149 ceph-mon[35042]: Updating smithi204:/etc/ceph/ceph.conf 2024-03-08T21:00:40.014 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:00:39 smithi149 ceph-mon[35042]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-08T21:00:40.014 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:00:39 smithi149 ceph-mon[35042]: Updating smithi107:/var/lib/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/config/ceph.conf 2024-03-08T21:00:40.015 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:00:39 smithi149 ceph-mon[35042]: Updating smithi149:/var/lib/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/config/ceph.conf 2024-03-08T21:00:40.015 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:00:39 smithi149 ceph-mon[35042]: Updating smithi204:/var/lib/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/config/ceph.conf 2024-03-08T21:00:40.877 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:00:40 smithi204 ceph-mon[34935]: Updating smithi107:/etc/ceph/ceph.client.admin.keyring 2024-03-08T21:00:40.878 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:00:40 smithi204 ceph-mon[34935]: Updating smithi149:/etc/ceph/ceph.client.admin.keyring 2024-03-08T21:00:40.878 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:00:40 smithi204 ceph-mon[34935]: Updating smithi204:/etc/ceph/ceph.client.admin.keyring 2024-03-08T21:00:40.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:00:40 smithi107 ceph-mon[31381]: Updating smithi107:/etc/ceph/ceph.client.admin.keyring 2024-03-08T21:00:40.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:00:40 smithi107 ceph-mon[31381]: Updating smithi149:/etc/ceph/ceph.client.admin.keyring 2024-03-08T21:00:40.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:00:40 smithi107 ceph-mon[31381]: Updating smithi204:/etc/ceph/ceph.client.admin.keyring 2024-03-08T21:00:41.014 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:00:40 smithi149 ceph-mon[35042]: Updating smithi107:/etc/ceph/ceph.client.admin.keyring 2024-03-08T21:00:41.014 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:00:40 smithi149 ceph-mon[35042]: Updating smithi149:/etc/ceph/ceph.client.admin.keyring 2024-03-08T21:00:41.014 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:00:40 smithi149 ceph-mon[35042]: Updating smithi204:/etc/ceph/ceph.client.admin.keyring 2024-03-08T21:00:41.878 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:00:41 smithi204 ceph-mon[34935]: Updating smithi107:/var/lib/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/config/ceph.client.admin.keyring 2024-03-08T21:00:41.878 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:00:41 smithi204 ceph-mon[34935]: Updating smithi149:/var/lib/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/config/ceph.client.admin.keyring 2024-03-08T21:00:41.878 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:00:41 smithi204 ceph-mon[34935]: Updating smithi204:/var/lib/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/config/ceph.client.admin.keyring 2024-03-08T21:00:41.879 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:00:41 smithi204 ceph-mon[34935]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 133 KiB/s wr, 80 op/s 2024-03-08T21:00:41.879 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:00:41 smithi204 ceph-mon[34935]: from='mgr.15375 ' entity='mgr.x' 2024-03-08T21:00:41.879 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:00:41 smithi204 ceph-mon[34935]: from='mgr.15375 ' entity='mgr.x' 2024-03-08T21:00:41.879 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:00:41 smithi204 ceph-mon[34935]: from='mgr.15375 ' entity='mgr.x' 2024-03-08T21:00:41.879 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:00:41 smithi204 ceph-mon[34935]: from='mgr.15375 ' entity='mgr.x' 2024-03-08T21:00:41.879 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:00:41 smithi204 ceph-mon[34935]: from='mgr.15375 ' entity='mgr.x' 2024-03-08T21:00:41.879 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:00:41 smithi204 ceph-mon[34935]: from='mgr.15375 ' entity='mgr.x' 2024-03-08T21:00:41.879 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:00:41 smithi204 ceph-mon[34935]: from='mgr.15375 ' entity='mgr.x' 2024-03-08T21:00:41.880 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:00:41 smithi204 ceph-mon[34935]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T21:00:41.880 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:00:41 smithi204 ceph-mon[34935]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T21:00:41.880 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:00:41 smithi204 ceph-mon[34935]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T21:00:41.880 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:00:41 smithi204 ceph-mon[34935]: from='mgr.15375 ' entity='mgr.x' 2024-03-08T21:00:41.917 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:00:41 smithi149 ceph-mon[35042]: Updating smithi107:/var/lib/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/config/ceph.client.admin.keyring 2024-03-08T21:00:41.917 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:00:41 smithi149 ceph-mon[35042]: Updating smithi149:/var/lib/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/config/ceph.client.admin.keyring 2024-03-08T21:00:41.917 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:00:41 smithi149 ceph-mon[35042]: Updating smithi204:/var/lib/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/config/ceph.client.admin.keyring 2024-03-08T21:00:41.917 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:00:41 smithi149 ceph-mon[35042]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 133 KiB/s wr, 80 op/s 2024-03-08T21:00:41.918 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:00:41 smithi149 ceph-mon[35042]: from='mgr.15375 ' entity='mgr.x' 2024-03-08T21:00:41.918 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:00:41 smithi149 ceph-mon[35042]: from='mgr.15375 ' entity='mgr.x' 2024-03-08T21:00:41.918 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:00:41 smithi149 ceph-mon[35042]: from='mgr.15375 ' entity='mgr.x' 2024-03-08T21:00:41.918 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:00:41 smithi149 ceph-mon[35042]: from='mgr.15375 ' entity='mgr.x' 2024-03-08T21:00:41.918 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:00:41 smithi149 ceph-mon[35042]: from='mgr.15375 ' entity='mgr.x' 2024-03-08T21:00:41.918 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:00:41 smithi149 ceph-mon[35042]: from='mgr.15375 ' entity='mgr.x' 2024-03-08T21:00:41.918 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:00:41 smithi149 ceph-mon[35042]: from='mgr.15375 ' entity='mgr.x' 2024-03-08T21:00:41.918 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:00:41 smithi149 ceph-mon[35042]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T21:00:41.918 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:00:41 smithi149 ceph-mon[35042]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T21:00:41.918 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:00:41 smithi149 ceph-mon[35042]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T21:00:41.918 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:00:41 smithi149 ceph-mon[35042]: from='mgr.15375 ' entity='mgr.x' 2024-03-08T21:00:41.967 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:00:41 smithi107 ceph-mon[31381]: Updating smithi107:/var/lib/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/config/ceph.client.admin.keyring 2024-03-08T21:00:41.967 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:00:41 smithi107 ceph-mon[31381]: Updating smithi149:/var/lib/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/config/ceph.client.admin.keyring 2024-03-08T21:00:41.967 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:00:41 smithi107 ceph-mon[31381]: Updating smithi204:/var/lib/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/config/ceph.client.admin.keyring 2024-03-08T21:00:41.967 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:00:41 smithi107 ceph-mon[31381]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 133 KiB/s wr, 80 op/s 2024-03-08T21:00:41.967 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:00:41 smithi107 ceph-mon[31381]: from='mgr.15375 ' entity='mgr.x' 2024-03-08T21:00:41.967 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:00:41 smithi107 ceph-mon[31381]: from='mgr.15375 ' entity='mgr.x' 2024-03-08T21:00:41.967 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:00:41 smithi107 ceph-mon[31381]: from='mgr.15375 ' entity='mgr.x' 2024-03-08T21:00:41.968 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:00:41 smithi107 ceph-mon[31381]: from='mgr.15375 ' entity='mgr.x' 2024-03-08T21:00:41.968 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:00:41 smithi107 ceph-mon[31381]: from='mgr.15375 ' entity='mgr.x' 2024-03-08T21:00:41.968 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:00:41 smithi107 ceph-mon[31381]: from='mgr.15375 ' entity='mgr.x' 2024-03-08T21:00:41.968 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:00:41 smithi107 ceph-mon[31381]: from='mgr.15375 ' entity='mgr.x' 2024-03-08T21:00:41.968 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:00:41 smithi107 ceph-mon[31381]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T21:00:41.968 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:00:41 smithi107 ceph-mon[31381]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T21:00:41.968 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:00:41 smithi107 ceph-mon[31381]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T21:00:41.968 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:00:41 smithi107 ceph-mon[31381]: from='mgr.15375 ' entity='mgr.x' 2024-03-08T21:00:43.877 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:00:43 smithi204 ceph-mon[34935]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 103 KiB/s wr, 62 op/s 2024-03-08T21:00:43.967 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:00:43 smithi107 ceph-mon[31381]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 103 KiB/s wr, 62 op/s 2024-03-08T21:00:44.014 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:00:43 smithi149 ceph-mon[35042]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 103 KiB/s wr, 62 op/s 2024-03-08T21:00:45.877 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:00:45 smithi204 ceph-mon[34935]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 85 KiB/s wr, 55 op/s 2024-03-08T21:00:45.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:00:45 smithi107 ceph-mon[31381]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 85 KiB/s wr, 55 op/s 2024-03-08T21:00:46.014 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:00:45 smithi149 ceph-mon[35042]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 85 KiB/s wr, 55 op/s 2024-03-08T21:00:47.877 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:00:47 smithi204 ceph-mon[34935]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 77 KiB/s wr, 50 op/s 2024-03-08T21:00:47.967 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:00:47 smithi107 ceph-mon[31381]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 77 KiB/s wr, 50 op/s 2024-03-08T21:00:48.014 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:00:47 smithi149 ceph-mon[35042]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 77 KiB/s wr, 50 op/s 2024-03-08T21:00:48.740 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.758+0000 7fd0aee00640 1 -- 172.21.15.107:0/2966162655 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd0a8075820 msgr2=0x7fd0a8075c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:00:48.740 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.758+0000 7fd0aee00640 1 --2- 172.21.15.107:0/2966162655 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd0a8075820 0x7fd0a8075c00 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fd098009900 tx=0x7fd09802bc40 comp rx=0 tx=0).stop 2024-03-08T21:00:48.740 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.758+0000 7fd0aee00640 1 -- 172.21.15.107:0/2966162655 shutdown_connections 2024-03-08T21:00:48.740 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.758+0000 7fd0aee00640 1 --2- 172.21.15.107:0/2966162655 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd0a8112150 0x7fd0a8114540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:48.741 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.758+0000 7fd0aee00640 1 --2- 172.21.15.107:0/2966162655 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd0a8076140 0x7fd0a8111c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:48.741 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.758+0000 7fd0aee00640 1 --2- 172.21.15.107:0/2966162655 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd0a8075820 0x7fd0a8075c00 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:48.741 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.758+0000 7fd0aee00640 1 -- 172.21.15.107:0/2966162655 >> 172.21.15.107:0/2966162655 conn(0x7fd0a80fefe0 msgr2=0x7fd0a8101400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:00:48.741 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.758+0000 7fd0aee00640 1 -- 172.21.15.107:0/2966162655 shutdown_connections 2024-03-08T21:00:48.741 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.758+0000 7fd0aee00640 1 -- 172.21.15.107:0/2966162655 wait complete. 2024-03-08T21:00:48.743 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.759+0000 7fd0aee00640 1 Processor -- start 2024-03-08T21:00:48.743 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.759+0000 7fd0aee00640 1 -- start start 2024-03-08T21:00:48.743 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.759+0000 7fd0aee00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd0a8075820 0x7fd0a819e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:00:48.744 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.759+0000 7fd0aee00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd0a8076140 0x7fd0a819e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:00:48.744 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.759+0000 7fd0aee00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd0a8112150 0x7fd0a81a2960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:00:48.744 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.759+0000 7fd0aee00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0a8117b60 con 0x7fd0a8112150 2024-03-08T21:00:48.744 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.759+0000 7fd0aee00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0a81179e0 con 0x7fd0a8076140 2024-03-08T21:00:48.744 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.759+0000 7fd0aee00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0a8117ce0 con 0x7fd0a8075820 2024-03-08T21:00:48.744 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.760+0000 7fd0ace00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd0a8112150 0x7fd0a81a2960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:00:48.744 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.760+0000 7fd0ace00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd0a8112150 0x7fd0a81a2960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:38460/0 (socket says 172.21.15.107:38460) 2024-03-08T21:00:48.744 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.760+0000 7fd0ace00640 1 -- 172.21.15.107:0/2748930162 learned_addr learned my addr 172.21.15.107:0/2748930162 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:00:48.744 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.760+0000 7fd09f400640 1 --2- 172.21.15.107:0/2748930162 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd0a8076140 0x7fd0a819e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:00:48.744 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.760+0000 7fd0a7e00640 1 --2- 172.21.15.107:0/2748930162 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd0a8075820 0x7fd0a819e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:00:48.744 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.760+0000 7fd0ace00640 1 -- 172.21.15.107:0/2748930162 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd0a8075820 msgr2=0x7fd0a819e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:00:48.744 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.760+0000 7fd0ace00640 1 --2- 172.21.15.107:0/2748930162 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd0a8075820 0x7fd0a819e0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:48.744 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.760+0000 7fd0ace00640 1 -- 172.21.15.107:0/2748930162 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd0a8076140 msgr2=0x7fd0a819e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:00:48.745 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.760+0000 7fd0ace00640 1 --2- 172.21.15.107:0/2748930162 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd0a8076140 0x7fd0a819e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:48.745 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.760+0000 7fd0ace00640 1 -- 172.21.15.107:0/2748930162 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0a81a3090 con 0x7fd0a8112150 2024-03-08T21:00:48.745 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.760+0000 7fd0a7e00640 1 --2- 172.21.15.107:0/2748930162 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd0a8075820 0x7fd0a819e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T21:00:48.745 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.761+0000 7fd0ace00640 1 --2- 172.21.15.107:0/2748930162 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd0a8112150 0x7fd0a81a2960 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fd0a000e9d0 tx=0x7fd0a000ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:00:48.745 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.761+0000 7fd0a5600640 1 -- 172.21.15.107:0/2748930162 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0a000cd80 con 0x7fd0a8112150 2024-03-08T21:00:48.745 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.761+0000 7fd0aee00640 1 -- 172.21.15.107:0/2748930162 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0a81a3380 con 0x7fd0a8112150 2024-03-08T21:00:48.745 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.761+0000 7fd0a5600640 1 -- 172.21.15.107:0/2748930162 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fd0a0005020 con 0x7fd0a8112150 2024-03-08T21:00:48.745 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.761+0000 7fd0a5600640 1 -- 172.21.15.107:0/2748930162 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0a00116d0 con 0x7fd0a8112150 2024-03-08T21:00:48.745 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.761+0000 7fd0aee00640 1 -- 172.21.15.107:0/2748930162 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0a81aac10 con 0x7fd0a8112150 2024-03-08T21:00:48.747 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.765+0000 7fd0b1b89740 1 Processor -- start 2024-03-08T21:00:48.747 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.765+0000 7fd0a5600640 1 -- 172.21.15.107:0/2748930162 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 23) v1 ==== 137467+0+0 (secure 0 0 0) 0x7fd0a00051d0 con 0x7fd0a8112150 2024-03-08T21:00:48.749 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.766+0000 7fd0b1b89740 1 -- start start 2024-03-08T21:00:48.749 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.766+0000 7fd0b1b89740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563a6098ec80 0x563a6098f060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:00:48.749 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.766+0000 7fd0b1b89740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563a6098f5a0 0x563a60997e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:00:48.749 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.766+0000 7fd0b1b89740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563a60998370 0x563a6099a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:00:48.749 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.766+0000 7fd0b1b89740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a608f23d0 con 0x563a6098f5a0 2024-03-08T21:00:48.749 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.766+0000 7fd0b1b89740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a6089b6b0 con 0x563a60998370 2024-03-08T21:00:48.749 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.766+0000 7fd0b1b89740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a608ee470 con 0x563a6098ec80 2024-03-08T21:00:48.750 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.766+0000 7fd0a7e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563a6098f5a0 0x563a60997e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:00:48.750 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.766+0000 7fd0a7e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563a6098f5a0 0x563a60997e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:38472/0 (socket says 172.21.15.107:38472) 2024-03-08T21:00:48.750 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.766+0000 7fd0a7e00640 1 -- 172.21.15.107:0/2466058279 learned_addr learned my addr 172.21.15.107:0/2466058279 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:00:48.750 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.767+0000 7fd09f400640 1 --2- 172.21.15.107:0/2466058279 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563a6098ec80 0x563a6098f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:00:48.750 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.767+0000 7fd0a7e00640 1 -- 172.21.15.107:0/2466058279 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563a6098ec80 msgr2=0x563a6098f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:00:48.750 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.767+0000 7fd0a7e00640 1 --2- 172.21.15.107:0/2466058279 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563a6098ec80 0x563a6098f060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:48.750 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.767+0000 7fd09f400640 1 --2- 172.21.15.107:0/2466058279 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563a60998370 0x563a6099a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:00:48.750 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.767+0000 7fd0a7e00640 1 -- 172.21.15.107:0/2466058279 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563a60998370 msgr2=0x563a6099a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:00:48.750 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.767+0000 7fd0a7e00640 1 --2- 172.21.15.107:0/2466058279 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563a60998370 0x563a6099a760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:48.750 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.767+0000 7fd0a7e00640 1 -- 172.21.15.107:0/2466058279 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563a608c1b50 con 0x563a6098f5a0 2024-03-08T21:00:48.750 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.767+0000 7fd0a5600640 1 --2- 172.21.15.107:0/2748930162 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7fd08c0a2ab0 0x7fd08c0a4f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:00:48.750 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.767+0000 7fd0a7e00640 1 --2- 172.21.15.107:0/2466058279 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563a6098f5a0 0x563a60997e30 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fd098000c00 tx=0x7fd098009680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a592be0573aa751e server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:00:48.750 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.767+0000 7fd0a5600640 1 -- 172.21.15.107:0/2748930162 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fd0a00cf2c0 con 0x7fd0a8112150 2024-03-08T21:00:48.751 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.767+0000 7fd09f400640 1 --2- 172.21.15.107:0/2748930162 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7fd08c0a2ab0 0x7fd08c0a4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:00:48.751 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.767+0000 7fd09d600640 1 -- 172.21.15.107:0/2466058279 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd09802c890 con 0x563a6098f5a0 2024-03-08T21:00:48.751 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.767+0000 7fd09d600640 1 -- 172.21.15.107:0/2466058279 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fd09802ca30 con 0x563a6098f5a0 2024-03-08T21:00:48.751 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.767+0000 7fd09d600640 1 -- 172.21.15.107:0/2466058279 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd098004920 con 0x563a6098f5a0 2024-03-08T21:00:48.754 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.768+0000 7fd0b1b89740 1 -- 172.21.15.107:0/2466058279 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563a6098f5a0 msgr2=0x563a60997e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:00:48.754 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.768+0000 7fd09f400640 1 --2- 172.21.15.107:0/2748930162 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7fd08c0a2ab0 0x7fd08c0a4f70 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x563a6098a9c0 tx=0x7fd0900046c0 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:00:48.754 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.768+0000 7fd0b1b89740 1 --2- 172.21.15.107:0/2466058279 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563a6098f5a0 0x563a60997e30 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fd098000c00 tx=0x7fd098009680 comp rx=0 tx=0).stop 2024-03-08T21:00:48.754 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.768+0000 7fd0b1b89740 1 -- 172.21.15.107:0/2466058279 shutdown_connections 2024-03-08T21:00:48.754 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.768+0000 7fd0b1b89740 1 --2- 172.21.15.107:0/2466058279 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563a60998370 0x563a6099a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:48.755 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.768+0000 7fd0b1b89740 1 --2- 172.21.15.107:0/2466058279 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563a6098f5a0 0x563a60997e30 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:48.755 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.768+0000 7fd0b1b89740 1 --2- 172.21.15.107:0/2466058279 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563a6098ec80 0x563a6098f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:48.755 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.768+0000 7fd0b1b89740 1 -- 172.21.15.107:0/2466058279 >> 172.21.15.107:0/2466058279 conn(0x563a609895a0 msgr2=0x563a60989980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:00:48.755 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.768+0000 7fd0b1b89740 1 -- 172.21.15.107:0/2466058279 shutdown_connections 2024-03-08T21:00:48.755 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.768+0000 7fd0b1b89740 1 -- 172.21.15.107:0/2466058279 wait complete. 2024-03-08T21:00:48.755 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.769+0000 7fd0b1b89740 1 -- start start 2024-03-08T21:00:48.755 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.769+0000 7fd0b1b89740 1 Processor -- start 2024-03-08T21:00:48.755 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.769+0000 7fd0b1b89740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563a6098ec80 0x563a60997ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:00:48.755 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.770+0000 7fd0b1b89740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563a6098f5a0 0x563a60995070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:00:48.755 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.770+0000 7fd0b1b89740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563a60998370 0x563a609955d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:00:48.755 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.770+0000 7fd0b1b89740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a6099af40 con 0x563a60998370 2024-03-08T21:00:48.755 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.770+0000 7fd0b1b89740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a608ae9f0 con 0x563a6098ec80 2024-03-08T21:00:48.756 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.770+0000 7fd0b1b89740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a6099b0c0 con 0x563a6098f5a0 2024-03-08T21:00:48.756 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.770+0000 7fd0b1b89740 10 client.0 _ll_register_callbacks cb 0x563a60606a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T21:00:48.756 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.770+0000 7fd0ace00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563a60998370 0x563a609955d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:00:48.756 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.770+0000 7fd0ace00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563a60998370 0x563a609955d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:38478/0 (socket says 172.21.15.107:38478) 2024-03-08T21:00:48.756 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.770+0000 7fd0ace00640 1 -- 172.21.15.107:0/1979748447 learned_addr learned my addr 172.21.15.107:0/1979748447 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:00:48.756 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.770+0000 7fd0a7e00640 1 --2- 172.21.15.107:0/1979748447 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563a6098ec80 0x563a60997ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:00:48.756 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.770+0000 7fd09f400640 1 --2- 172.21.15.107:0/1979748447 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563a6098f5a0 0x563a60995070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:00:48.756 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.770+0000 7fd0ace00640 1 -- 172.21.15.107:0/1979748447 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563a6098f5a0 msgr2=0x563a60995070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:00:48.756 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.770+0000 7fd0ace00640 1 --2- 172.21.15.107:0/1979748447 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563a6098f5a0 0x563a60995070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:48.756 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.770+0000 7fd0ace00640 1 -- 172.21.15.107:0/1979748447 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563a6098ec80 msgr2=0x563a60997ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:00:48.756 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.770+0000 7fd0ace00640 1 --2- 172.21.15.107:0/1979748447 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563a6098ec80 0x563a60997ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:48.756 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.770+0000 7fd0ace00640 1 -- 172.21.15.107:0/1979748447 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563a60607210 con 0x563a60998370 2024-03-08T21:00:48.756 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.771+0000 7fd0ace00640 1 --2- 172.21.15.107:0/1979748447 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563a60998370 0x563a609955d0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fd0a009b6d0 tx=0x7fd0a009b190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a131c9db12c309b server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:00:48.757 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.771+0000 7fd096000640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-03-08T21:00:48.757 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.771+0000 7fd096000640 1 -- 172.21.15.107:0/1979748447 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0a001f040 con 0x563a60998370 2024-03-08T21:00:48.757 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.771+0000 7fd096000640 1 -- 172.21.15.107:0/1979748447 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fd0a00ccb90 con 0x563a60998370 2024-03-08T21:00:48.757 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.771+0000 7fd0b1b89740 1 -- 172.21.15.107:0/1979748447 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd09802ca30 con 0x563a60998370 2024-03-08T21:00:48.757 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.771+0000 7fd096000640 1 -- 172.21.15.107:0/1979748447 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd0a00ccdb0 con 0x563a60998370 2024-03-08T21:00:48.757 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.771+0000 7fd0b1b89740 10 client.15477 fetch_fsmap learned FSMap version 28 2024-03-08T21:00:48.757 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.771+0000 7fd0b1b89740 1 -- 172.21.15.107:0/1979748447 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x563a606d3620 con 0x563a60998370 2024-03-08T21:00:48.757 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.771+0000 7fd096000640 1 -- 172.21.15.107:0/1979748447 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fd09802ca30 con 0x563a60998370 2024-03-08T21:00:48.757 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.772+0000 7fd096000640 1 -- 172.21.15.107:0/1979748447 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 28) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fd0a000f040 con 0x563a60998370 2024-03-08T21:00:48.757 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.772+0000 7fd0b1b89740 10 client.15477 fetch_fsmap finished waiting for FSMap version 28 2024-03-08T21:00:48.757 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.772+0000 7fd0b1b89740 10 client.15477 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-08T21:00:48.757 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.772+0000 7fd0b1b89740 20 client.15477 populate_metadata read hostname 'smithi107' 2024-03-08T21:00:48.757 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.772+0000 7fd0b1b89740 1 --2- 172.21.15.107:0/1979748447 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x563a609a81a0 0x563a609aa5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:00:48.758 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.772+0000 7fd0b1b89740 4 client.15477 mds_command: new command op to 34475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-08T21:00:48.758 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.772+0000 7fd0b1b89740 1 -- 172.21.15.107:0/1979748447 --> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563a608ff910 con 0x563a609a81a0 2024-03-08T21:00:48.758 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.772+0000 7fd0a7e00640 1 --2- 172.21.15.107:0/1979748447 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x563a609a81a0 0x563a609aa5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:00:48.759 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.773+0000 7fd0a7e00640 1 --2- 172.21.15.107:0/1979748447 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x563a609a81a0 0x563a609aa5b0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d6f79df059692603 server_cookie=96eb89062739699 in_seq=0 out_seq=0 2024-03-08T21:00:48.759 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.773+0000 7fd096000640 10 client.15477 ms_handle_connect on v2:172.21.15.149:6838/55290596 2024-03-08T21:00:48.759 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.775+0000 7fd096000640 1 -- 172.21.15.107:0/1979748447 <== mds.0 v2:172.21.15.149:6838/55290596 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x563a608ff910 con 0x563a609a81a0 2024-03-08T21:00:48.759 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.775+0000 7fd096000640 10 client.15477 handle_command_reply: tid=0 2024-03-08T21:00:48.759 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.775+0000 7fd0b1b89740 1 client.15477 shutdown 2024-03-08T21:00:48.759 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.775+0000 7fd0b1b89740 1 -- 172.21.15.107:0/1979748447 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563a60998370 msgr2=0x563a609955d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:00:48.759 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.775+0000 7fd0b1b89740 1 --2- 172.21.15.107:0/1979748447 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563a60998370 0x563a609955d0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fd0a009b6d0 tx=0x7fd0a009b190 comp rx=0 tx=0).stop 2024-03-08T21:00:48.759 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.775+0000 7fd0b1b89740 1 -- 172.21.15.107:0/1979748447 shutdown_connections 2024-03-08T21:00:48.760 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.775+0000 7fd0b1b89740 1 --2- 172.21.15.107:0/1979748447 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563a60998370 0x563a609955d0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:48.760 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.775+0000 7fd0b1b89740 1 --2- 172.21.15.107:0/1979748447 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563a6098f5a0 0x563a60995070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:48.760 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.775+0000 7fd0b1b89740 1 --2- 172.21.15.107:0/1979748447 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x563a609a81a0 0x563a609aa5b0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:48.760 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.775+0000 7fd0b1b89740 1 --2- 172.21.15.107:0/1979748447 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563a6098ec80 0x563a60997ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:48.760 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.775+0000 7fd096000640 0 client.15477 ms_handle_reset on v2:172.21.15.149:6838/55290596 2024-03-08T21:00:48.760 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.775+0000 7fd0b1b89740 1 -- 172.21.15.107:0/1979748447 >> 172.21.15.107:0/1979748447 conn(0x563a609895a0 msgr2=0x563a6098b2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:00:48.760 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.775+0000 7fd0b1b89740 1 -- 172.21.15.107:0/1979748447 shutdown_connections 2024-03-08T21:00:48.760 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.775+0000 7fd0b1b89740 1 -- 172.21.15.107:0/1979748447 wait complete. 2024-03-08T21:00:48.760 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.776+0000 7fd0b1b89740 20 client.15477 trim_cache size 0 max 16384 2024-03-08T21:00:48.773 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.790+0000 7fd0b1b89740 1 Processor -- start 2024-03-08T21:00:48.773 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.790+0000 7fd0b1b89740 1 -- start start 2024-03-08T21:00:48.773 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.790+0000 7fd0b1b89740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563a6098ec80 0x563a609977a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:00:48.773 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.790+0000 7fd0b1b89740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563a6098f5a0 0x563a60997ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:00:48.773 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.790+0000 7fd0b1b89740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563a60998370 0x563a609922d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:00:48.773 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.790+0000 7fd0b1b89740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0a81b3250 con 0x563a60998370 2024-03-08T21:00:48.773 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.790+0000 7fd0b1b89740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0a81b4bd0 con 0x563a6098f5a0 2024-03-08T21:00:48.773 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.790+0000 7fd0b1b89740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0a81b38b0 con 0x563a6098ec80 2024-03-08T21:00:48.773 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.791+0000 7fd0ace00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563a60998370 0x563a609922d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:00:48.773 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.791+0000 7fd0ace00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563a60998370 0x563a609922d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:38492/0 (socket says 172.21.15.107:38492) 2024-03-08T21:00:48.773 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.791+0000 7fd0ace00640 1 -- 172.21.15.107:0/1919282799 learned_addr learned my addr 172.21.15.107:0/1919282799 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:00:48.774 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.791+0000 7fd0a7e00640 1 --2- 172.21.15.107:0/1919282799 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563a6098ec80 0x563a609977a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:00:48.774 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.791+0000 7fd09f400640 1 --2- 172.21.15.107:0/1919282799 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563a6098f5a0 0x563a60997ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:00:48.774 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.791+0000 7fd0ace00640 1 -- 172.21.15.107:0/1919282799 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563a6098ec80 msgr2=0x563a609977a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:00:48.774 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.791+0000 7fd0ace00640 1 --2- 172.21.15.107:0/1919282799 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563a6098ec80 0x563a609977a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:48.774 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.791+0000 7fd0ace00640 1 -- 172.21.15.107:0/1919282799 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563a6098f5a0 msgr2=0x563a60997ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:00:48.774 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.791+0000 7fd0ace00640 1 --2- 172.21.15.107:0/1919282799 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563a6098f5a0 0x563a60997ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:48.774 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.791+0000 7fd0ace00640 1 -- 172.21.15.107:0/1919282799 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563a60984e60 con 0x563a60998370 2024-03-08T21:00:48.778 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.792+0000 7fd0ace00640 1 --2- 172.21.15.107:0/1919282799 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563a60998370 0x563a609922d0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fd0a00a2270 tx=0x7fd0a0009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cceee4ef7c381e6e server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:00:48.778 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.792+0000 7fd09cc00640 1 -- 172.21.15.107:0/1919282799 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0a001f030 con 0x563a60998370 2024-03-08T21:00:48.778 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.792+0000 7fd09cc00640 1 -- 172.21.15.107:0/1919282799 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fd0a00a3ee0 con 0x563a60998370 2024-03-08T21:00:48.778 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.792+0000 7fd09cc00640 1 -- 172.21.15.107:0/1919282799 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0a00a2070 con 0x563a60998370 2024-03-08T21:00:48.778 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.792+0000 7fd0b1b89740 1 -- 172.21.15.107:0/1919282799 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563a60998370 msgr2=0x563a609922d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:00:48.778 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.792+0000 7fd0b1b89740 1 --2- 172.21.15.107:0/1919282799 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563a60998370 0x563a609922d0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fd0a00a2270 tx=0x7fd0a0009220 comp rx=0 tx=0).stop 2024-03-08T21:00:48.778 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.792+0000 7fd0b1b89740 1 -- 172.21.15.107:0/1919282799 shutdown_connections 2024-03-08T21:00:48.778 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.792+0000 7fd0b1b89740 1 --2- 172.21.15.107:0/1919282799 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563a60998370 0x563a609922d0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:48.779 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.792+0000 7fd0b1b89740 1 --2- 172.21.15.107:0/1919282799 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563a6098f5a0 0x563a60997ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:48.779 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.792+0000 7fd0b1b89740 1 --2- 172.21.15.107:0/1919282799 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563a6098ec80 0x563a609977a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:48.779 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.792+0000 7fd0b1b89740 1 -- 172.21.15.107:0/1919282799 >> 172.21.15.107:0/1919282799 conn(0x563a609895a0 msgr2=0x563a60995030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:00:48.779 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.793+0000 7fd0b1b89740 1 -- 172.21.15.107:0/1919282799 shutdown_connections 2024-03-08T21:00:48.779 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.793+0000 7fd0b1b89740 1 -- 172.21.15.107:0/1919282799 wait complete. 2024-03-08T21:00:48.779 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.793+0000 7fd0b1b89740 1 -- start start 2024-03-08T21:00:48.779 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.793+0000 7fd0b1b89740 1 Processor -- start 2024-03-08T21:00:48.779 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.794+0000 7fd0b1b89740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563a6098ec80 0x563a6099df60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:00:48.779 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.794+0000 7fd0b1b89740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563a6098f5a0 0x563a6099e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:00:48.779 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.794+0000 7fd0b1b89740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563a60998370 0x563a6099ea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:00:48.779 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.794+0000 7fd0b1b89740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0a81b4570 con 0x563a60998370 2024-03-08T21:00:48.779 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.794+0000 7fd0b1b89740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0a81b3f10 con 0x563a6098f5a0 2024-03-08T21:00:48.779 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.794+0000 7fd0b1b89740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0a81b14f0 con 0x563a6098ec80 2024-03-08T21:00:48.780 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.794+0000 7fd0b1b89740 10 client.0 _ll_register_callbacks cb 0x7fd098009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T21:00:48.780 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.794+0000 7fd0ace00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563a60998370 0x563a6099ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:00:48.780 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.794+0000 7fd0ace00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563a60998370 0x563a6099ea00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:38498/0 (socket says 172.21.15.107:38498) 2024-03-08T21:00:48.780 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.794+0000 7fd0ace00640 1 -- 172.21.15.107:0/3076362542 learned_addr learned my addr 172.21.15.107:0/3076362542 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:00:48.780 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.794+0000 7fd09f400640 1 --2- 172.21.15.107:0/3076362542 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563a6098f5a0 0x563a6099e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:00:48.780 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.794+0000 7fd0a7e00640 1 --2- 172.21.15.107:0/3076362542 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563a6098ec80 0x563a6099df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:00:48.780 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.794+0000 7fd0ace00640 1 -- 172.21.15.107:0/3076362542 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563a6098ec80 msgr2=0x563a6099df60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:00:48.780 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.794+0000 7fd0ace00640 1 --2- 172.21.15.107:0/3076362542 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563a6098ec80 0x563a6099df60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:48.780 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.794+0000 7fd0ace00640 1 -- 172.21.15.107:0/3076362542 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563a6098f5a0 msgr2=0x563a6099e4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:00:48.780 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.794+0000 7fd0ace00640 1 --2- 172.21.15.107:0/3076362542 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563a6098f5a0 0x563a6099e4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:48.780 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.794+0000 7fd0ace00640 1 -- 172.21.15.107:0/3076362542 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563a609fe9b0 con 0x563a60998370 2024-03-08T21:00:48.780 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.795+0000 7fd0ace00640 1 --2- 172.21.15.107:0/3076362542 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563a60998370 0x563a6099ea00 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fd0a0097ba0 tx=0x7fd0a009f950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12354f930b218dd2 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:00:48.780 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.795+0000 7fd096000640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-03-08T21:00:48.780 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.795+0000 7fd096000640 1 -- 172.21.15.107:0/3076362542 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0a00b8070 con 0x563a60998370 2024-03-08T21:00:48.781 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.795+0000 7fd096000640 1 -- 172.21.15.107:0/3076362542 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fd0a009ab90 con 0x563a60998370 2024-03-08T21:00:48.781 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.795+0000 7fd0b1b89740 1 -- 172.21.15.107:0/3076362542 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd0a00a3ee0 con 0x563a60998370 2024-03-08T21:00:48.781 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.795+0000 7fd096000640 1 -- 172.21.15.107:0/3076362542 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd0a001f070 con 0x563a60998370 2024-03-08T21:00:48.781 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.795+0000 7fd0b1b89740 10 client.15483 fetch_fsmap learned FSMap version 28 2024-03-08T21:00:48.781 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.795+0000 7fd0b1b89740 1 -- 172.21.15.107:0/3076362542 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x563a609ffcc0 con 0x563a60998370 2024-03-08T21:00:48.781 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.795+0000 7fd096000640 1 -- 172.21.15.107:0/3076362542 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fd0a00a3ee0 con 0x563a60998370 2024-03-08T21:00:48.781 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.796+0000 7fd096000640 1 -- 172.21.15.107:0/3076362542 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 28) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fd0a00a07b0 con 0x563a60998370 2024-03-08T21:00:48.781 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.796+0000 7fd0b1b89740 10 client.15483 fetch_fsmap finished waiting for FSMap version 28 2024-03-08T21:00:48.781 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.796+0000 7fd0b1b89740 10 client.15483 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-08T21:00:48.781 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.796+0000 7fd0b1b89740 20 client.15483 populate_metadata read hostname 'smithi107' 2024-03-08T21:00:48.781 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.796+0000 7fd0b1b89740 1 --2- 172.21.15.107:0/3076362542 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x563a60a000d0 0x563a60a024f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:00:48.781 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.796+0000 7fd0b1b89740 4 client.15483 mds_command: new command op to 34475 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-08T21:00:48.781 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.796+0000 7fd0b1b89740 1 -- 172.21.15.107:0/3076362542 --> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x563a608f9c60 con 0x563a60a000d0 2024-03-08T21:00:48.781 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.796+0000 7fd0a7e00640 1 --2- 172.21.15.107:0/3076362542 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x563a60a000d0 0x563a60a024f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:00:48.782 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.797+0000 7fd0a7e00640 1 --2- 172.21.15.107:0/3076362542 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x563a60a000d0 0x563a60a024f0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=502d400e54dd4888 server_cookie=79a3725accc56452 in_seq=0 out_seq=0 2024-03-08T21:00:48.782 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.797+0000 7fd096000640 10 client.15483 ms_handle_connect on v2:172.21.15.149:6838/55290596 2024-03-08T21:00:48.811 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.828+0000 7fd096000640 1 -- 172.21.15.107:0/3076362542 <== mds.0 v2:172.21.15.149:6838/55290596 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x563a608f9c60 con 0x563a60a000d0 2024-03-08T21:00:48.811 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.828+0000 7fd096000640 10 client.15483 handle_command_reply: tid=0 2024-03-08T21:00:48.811 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.828+0000 7fd0b1b89740 1 client.15483 shutdown 2024-03-08T21:00:48.811 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.828+0000 7fd0b1b89740 1 -- 172.21.15.107:0/3076362542 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563a60998370 msgr2=0x563a6099ea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:00:48.811 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.828+0000 7fd0b1b89740 1 --2- 172.21.15.107:0/3076362542 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563a60998370 0x563a6099ea00 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fd0a0097ba0 tx=0x7fd0a009f950 comp rx=0 tx=0).stop 2024-03-08T21:00:48.811 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.828+0000 7fd0b1b89740 1 -- 172.21.15.107:0/3076362542 shutdown_connections 2024-03-08T21:00:48.811 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.828+0000 7fd0b1b89740 1 --2- 172.21.15.107:0/3076362542 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563a60998370 0x563a6099ea00 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:48.812 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.828+0000 7fd0b1b89740 1 --2- 172.21.15.107:0/3076362542 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x563a60a000d0 0x563a60a024f0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:48.812 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.829+0000 7fd0b1b89740 1 --2- 172.21.15.107:0/3076362542 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563a6098f5a0 0x563a6099e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:48.812 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.829+0000 7fd096000640 0 client.15483 ms_handle_reset on v2:172.21.15.149:6838/55290596 2024-03-08T21:00:48.812 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.829+0000 7fd0b1b89740 1 --2- 172.21.15.107:0/3076362542 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563a6098ec80 0x563a6099df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:48.812 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.829+0000 7fd0b1b89740 1 -- 172.21.15.107:0/3076362542 >> 172.21.15.107:0/3076362542 conn(0x563a609895a0 msgr2=0x563a6098a9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:00:48.812 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.829+0000 7fd0b1b89740 1 -- 172.21.15.107:0/3076362542 shutdown_connections 2024-03-08T21:00:48.812 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.829+0000 7fd0b1b89740 1 -- 172.21.15.107:0/3076362542 wait complete. 2024-03-08T21:00:48.812 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.829+0000 7fd0b1b89740 20 client.15483 trim_cache size 0 max 16384 2024-03-08T21:00:48.813 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:{ 2024-03-08T21:00:48.813 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "message": "", 2024-03-08T21:00:48.813 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "return_code": 0 2024-03-08T21:00:48.813 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:} 2024-03-08T21:00:48.814 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.830+0000 7fd0aee00640 1 -- 172.21.15.107:0/2748930162 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7fd08c0a2ab0 msgr2=0x7fd08c0a4f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:00:48.814 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.830+0000 7fd0aee00640 1 --2- 172.21.15.107:0/2748930162 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7fd08c0a2ab0 0x7fd08c0a4f70 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x563a6098a9c0 tx=0x7fd0900046c0 comp rx=0 tx=0).stop 2024-03-08T21:00:48.814 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.830+0000 7fd0aee00640 1 -- 172.21.15.107:0/2748930162 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd0a8112150 msgr2=0x7fd0a81a2960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:00:48.814 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.830+0000 7fd0aee00640 1 --2- 172.21.15.107:0/2748930162 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd0a8112150 0x7fd0a81a2960 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fd0a000e9d0 tx=0x7fd0a000ee90 comp rx=0 tx=0).stop 2024-03-08T21:00:48.814 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.830+0000 7fd0aee00640 1 -- 172.21.15.107:0/2748930162 shutdown_connections 2024-03-08T21:00:48.814 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.830+0000 7fd0aee00640 1 --2- 172.21.15.107:0/2748930162 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7fd08c0a2ab0 0x7fd08c0a4f70 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:48.814 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.830+0000 7fd0aee00640 1 --2- 172.21.15.107:0/2748930162 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd0a8112150 0x7fd0a81a2960 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:48.814 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.830+0000 7fd0aee00640 1 --2- 172.21.15.107:0/2748930162 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd0a8076140 0x7fd0a819e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:48.814 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.830+0000 7fd0aee00640 1 --2- 172.21.15.107:0/2748930162 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd0a8075820 0x7fd0a819e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:00:48.815 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.830+0000 7fd0aee00640 1 -- 172.21.15.107:0/2748930162 >> 172.21.15.107:0/2748930162 conn(0x7fd0a80fefe0 msgr2=0x7fd0a81013d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:00:48.815 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.830+0000 7fd0aee00640 1 -- 172.21.15.107:0/2748930162 shutdown_connections 2024-03-08T21:00:48.815 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:00:48.831+0000 7fd0aee00640 1 -- 172.21.15.107:0/2748930162 wait complete. 2024-03-08T21:00:49.877 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:00:49 smithi204 ceph-mon[34935]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 77 KiB/s wr, 50 op/s 2024-03-08T21:00:49.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:00:49 smithi107 ceph-mon[31381]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 77 KiB/s wr, 50 op/s 2024-03-08T21:00:50.014 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:00:49 smithi149 ceph-mon[35042]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 77 KiB/s wr, 50 op/s 2024-03-08T21:00:50.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:00:50 smithi107 ceph-mon[31381]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 77 KiB/s wr, 55 op/s 2024-03-08T21:00:51.015 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:00:50 smithi149 ceph-mon[35042]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 77 KiB/s wr, 55 op/s 2024-03-08T21:00:51.127 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:00:50 smithi204 ceph-mon[34935]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 77 KiB/s wr, 55 op/s 2024-03-08T21:00:53.877 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:00:53 smithi204 ceph-mon[34935]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 597 B/s wr, 8 op/s 2024-03-08T21:00:53.950 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:00:53 smithi149 ceph-mon[35042]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 597 B/s wr, 8 op/s 2024-03-08T21:00:53.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:00:53 smithi107 ceph-mon[31381]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 597 B/s wr, 8 op/s 2024-03-08T21:00:55.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:00:55 smithi107 ceph-mon[31381]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 2.1 KiB/s wr, 17 op/s 2024-03-08T21:00:55.877 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:00:55 smithi204 ceph-mon[34935]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 2.1 KiB/s wr, 17 op/s 2024-03-08T21:00:56.014 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:00:55 smithi149 ceph-mon[35042]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 2.1 KiB/s wr, 17 op/s 2024-03-08T21:00:57.877 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:00:57 smithi204 ceph-mon[34935]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.0 KiB/s wr, 12 op/s 2024-03-08T21:00:57.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:00:57 smithi107 ceph-mon[31381]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.0 KiB/s wr, 12 op/s 2024-03-08T21:00:58.014 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:00:57 smithi149 ceph-mon[35042]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.0 KiB/s wr, 12 op/s 2024-03-08T21:00:59.878 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:00:59 smithi204 ceph-mon[34935]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.0 KiB/s wr, 12 op/s 2024-03-08T21:00:59.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:00:59 smithi107 ceph-mon[31381]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.0 KiB/s wr, 12 op/s 2024-03-08T21:01:00.022 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:00:59 smithi149 ceph-mon[35042]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.0 KiB/s wr, 12 op/s 2024-03-08T21:01:00.467 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 08 21:01:00 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-x[31606]: 2024-03-08T21:01:00.232+0000 7f64ffe00700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-08T21:01:00.467 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 08 21:01:00 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-x[31606]: 2024-03-08T21:01:00.232+0000 7f64ffe00700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-08T21:01:00.467 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 08 21:01:00 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-x[31606]: 2024-03-08T21:01:00.232+0000 7f64ffe00700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-08T21:01:00.467 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 08 21:01:00 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-x[31606]: 2024-03-08T21:01:00.232+0000 7f64ffe00700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-08T21:01:00.467 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 08 21:01:00 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-x[31606]: 2024-03-08T21:01:00.232+0000 7f64ffe00700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-08T21:01:01.835 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:01:01 smithi149 ceph-mon[35042]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 2.0 KiB/s wr, 16 op/s 2024-03-08T21:01:01.835 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:01:01 smithi149 ceph-mon[35042]: mgrmap e24: x(active, since 28s), standbys: y, z 2024-03-08T21:01:01.838 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:01:01 smithi204 ceph-mon[34935]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 2.0 KiB/s wr, 16 op/s 2024-03-08T21:01:01.838 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:01:01 smithi204 ceph-mon[34935]: mgrmap e24: x(active, since 28s), standbys: y, z 2024-03-08T21:01:01.961 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:01.974+0000 7f598bc00640 1 -- 172.21.15.107:0/1398900724 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5984109590 msgr2=0x7f5984109970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:01:01.961 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:01.974+0000 7f598bc00640 1 --2- 172.21.15.107:0/1398900724 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5984109590 0x7f5984109970 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f5978009900 tx=0x7f597802bc40 comp rx=0 tx=0).stop 2024-03-08T21:01:01.961 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:01.974+0000 7f598bc00640 1 -- 172.21.15.107:0/1398900724 shutdown_connections 2024-03-08T21:01:01.961 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:01.974+0000 7f598bc00640 1 --2- 172.21.15.107:0/1398900724 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5984103e90 0x7f5984110350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:01:01.961 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:01.974+0000 7f598bc00640 1 --2- 172.21.15.107:0/1398900724 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f59841034d0 0x7f5984103950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:01:01.961 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:01.974+0000 7f598bc00640 1 --2- 172.21.15.107:0/1398900724 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5984109590 0x7f5984109970 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:01:01.961 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:01.974+0000 7f598bc00640 1 -- 172.21.15.107:0/1398900724 >> 172.21.15.107:0/1398900724 conn(0x7f59840fefe0 msgr2=0x7f5984101400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:01:01.962 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:01.975+0000 7f598bc00640 1 -- 172.21.15.107:0/1398900724 shutdown_connections 2024-03-08T21:01:01.962 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:01.975+0000 7f598bc00640 1 -- 172.21.15.107:0/1398900724 wait complete. 2024-03-08T21:01:01.962 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:01.976+0000 7f598bc00640 1 Processor -- start 2024-03-08T21:01:01.962 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:01.976+0000 7f598bc00640 1 -- start start 2024-03-08T21:01:01.962 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:01.976+0000 7f598bc00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f59841034d0 0x7f598419f910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:01:01.962 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:01.976+0000 7f598bc00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5984103e90 0x7f598419fe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:01:01.962 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:01.976+0000 7f598bc00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5984109590 0x7f5984199b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:01:01.962 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:01.976+0000 7f598bc00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59841137d0 con 0x7f5984103e90 2024-03-08T21:01:01.962 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:01.976+0000 7f598bc00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5984113650 con 0x7f5984109590 2024-03-08T21:01:01.962 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:01.976+0000 7f598bc00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5984113950 con 0x7f59841034d0 2024-03-08T21:01:01.962 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:01.976+0000 7f5983e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5984103e90 0x7f598419fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:01:01.962 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:01.976+0000 7f5983e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5984103e90 0x7f598419fe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:42052/0 (socket says 172.21.15.107:42052) 2024-03-08T21:01:01.963 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:01.976+0000 7f5983e00640 1 -- 172.21.15.107:0/1023141434 learned_addr learned my addr 172.21.15.107:0/1023141434 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:01:01.963 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:01.977+0000 7f5989200640 1 --2- 172.21.15.107:0/1023141434 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f59841034d0 0x7f598419f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:01:01.963 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:01.977+0000 7f5989c00640 1 --2- 172.21.15.107:0/1023141434 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5984109590 0x7f5984199b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:01:01.963 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:01.977+0000 7f5983e00640 1 -- 172.21.15.107:0/1023141434 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f59841034d0 msgr2=0x7f598419f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:01:01.963 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:01.977+0000 7f5983e00640 1 --2- 172.21.15.107:0/1023141434 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f59841034d0 0x7f598419f910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:01:01.963 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:01.977+0000 7f5983e00640 1 -- 172.21.15.107:0/1023141434 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5984109590 msgr2=0x7f5984199b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:01:01.963 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:01.977+0000 7f5983e00640 1 --2- 172.21.15.107:0/1023141434 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5984109590 0x7f5984199b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:01:01.963 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:01.977+0000 7f5983e00640 1 -- 172.21.15.107:0/1023141434 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f598419a290 con 0x7f5984103e90 2024-03-08T21:01:01.963 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:01.977+0000 7f5983e00640 1 --2- 172.21.15.107:0/1023141434 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5984103e90 0x7f598419fe50 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f597000c890 tx=0x7f597000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:01:01.963 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:01.977+0000 7f5981600640 1 -- 172.21.15.107:0/1023141434 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5970004220 con 0x7f5984103e90 2024-03-08T21:01:01.963 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:01.978+0000 7f5981600640 1 -- 172.21.15.107:0/1023141434 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f59700043c0 con 0x7f5984103e90 2024-03-08T21:01:01.963 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:01.978+0000 7f598bc00640 1 -- 172.21.15.107:0/1023141434 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f598419a580 con 0x7f5984103e90 2024-03-08T21:01:01.963 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:01.978+0000 7f5981600640 1 -- 172.21.15.107:0/1023141434 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5970004a00 con 0x7f5984103e90 2024-03-08T21:01:01.963 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:01.978+0000 7f598bc00640 1 -- 172.21.15.107:0/1023141434 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59841a67d0 con 0x7f5984103e90 2024-03-08T21:01:01.967 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:01.981+0000 7f598e9ad740 1 Processor -- start 2024-03-08T21:01:01.967 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:01.981+0000 7f598e9ad740 1 -- start start 2024-03-08T21:01:01.967 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:01.981+0000 7f598e9ad740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5568bd476c80 0x5568bd477060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:01:01.968 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:01.981+0000 7f598e9ad740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5568bd4775a0 0x5568bd47fe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:01:01.968 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:01.981+0000 7f5981600640 1 -- 172.21.15.107:0/1023141434 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 24) v1 ==== 137524+0+0 (secure 0 0 0) 0x7f5970004ba0 con 0x7f5984103e90 2024-03-08T21:01:01.968 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:01.981+0000 7f598e9ad740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5568bd480370 0x5568bd482760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:01:01.968 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:01.981+0000 7f598e9ad740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568bd3da3d0 con 0x5568bd476c80 2024-03-08T21:01:01.968 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:01.981+0000 7f598e9ad740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568bd3836b0 con 0x5568bd4775a0 2024-03-08T21:01:01.968 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:01.981+0000 7f598e9ad740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568bd3d6470 con 0x5568bd480370 2024-03-08T21:01:01.968 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:01.982+0000 7f5989c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5568bd476c80 0x5568bd477060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:01:01.968 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:01.982+0000 7f5989c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5568bd476c80 0x5568bd477060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:42054/0 (socket says 172.21.15.107:42054) 2024-03-08T21:01:01.968 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:01.982+0000 7f5989c00640 1 -- 172.21.15.107:0/257653794 learned_addr learned my addr 172.21.15.107:0/257653794 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:01:01.968 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:01.982+0000 7f5989200640 1 --2- 172.21.15.107:0/257653794 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5568bd4775a0 0x5568bd47fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:01:01.968 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:01.982+0000 7f5989c00640 1 -- 172.21.15.107:0/257653794 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5568bd480370 msgr2=0x5568bd482760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:01:01.968 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:01.982+0000 7f5989c00640 1 --2- 172.21.15.107:0/257653794 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5568bd480370 0x5568bd482760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:01:01.968 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:01.982+0000 7f5989c00640 1 -- 172.21.15.107:0/257653794 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5568bd4775a0 msgr2=0x5568bd47fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:01:01.969 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:01.982+0000 7f5989c00640 1 --2- 172.21.15.107:0/257653794 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5568bd4775a0 0x5568bd47fe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:01:01.969 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:01.982+0000 7f5989c00640 1 -- 172.21.15.107:0/257653794 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5568bd3a9b50 con 0x5568bd476c80 2024-03-08T21:01:01.969 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:01.982+0000 7f5989200640 1 --2- 172.21.15.107:0/257653794 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5568bd4775a0 0x5568bd47fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T21:01:01.969 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:01.982+0000 7f5989c00640 1 --2- 172.21.15.107:0/257653794 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5568bd476c80 0x5568bd477060 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f597c009f20 tx=0x7f597c00c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4723eabbf1b3a11 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:01:01.969 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:01.983+0000 7f5976000640 1 -- 172.21.15.107:0/257653794 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f597c01a070 con 0x5568bd476c80 2024-03-08T21:01:01.969 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:01.983+0000 7f5976000640 1 -- 172.21.15.107:0/257653794 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f597c0158c0 con 0x5568bd476c80 2024-03-08T21:01:01.969 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:01.983+0000 7f5976000640 1 -- 172.21.15.107:0/257653794 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f597c00a8a0 con 0x5568bd476c80 2024-03-08T21:01:01.969 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:01.983+0000 7f5981600640 1 --2- 172.21.15.107:0/1023141434 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f596c0a2b30 0x7f596c0a4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:01:01.969 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:01.983+0000 7f5981600640 1 -- 172.21.15.107:0/1023141434 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f59700cef20 con 0x7f5984103e90 2024-03-08T21:01:01.969 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:01.983+0000 7f5989200640 1 --2- 172.21.15.107:0/1023141434 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f596c0a2b30 0x7f596c0a4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:01:01.969 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:01.984+0000 7f598e9ad740 1 -- 172.21.15.107:0/257653794 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5568bd476c80 msgr2=0x5568bd477060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:01:01.969 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:01.984+0000 7f598e9ad740 1 --2- 172.21.15.107:0/257653794 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5568bd476c80 0x5568bd477060 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f597c009f20 tx=0x7f597c00c8b0 comp rx=0 tx=0).stop 2024-03-08T21:01:01.969 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:01.984+0000 7f598e9ad740 1 -- 172.21.15.107:0/257653794 shutdown_connections 2024-03-08T21:01:01.969 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:01.984+0000 7f598e9ad740 1 --2- 172.21.15.107:0/257653794 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5568bd480370 0x5568bd482760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:01:01.970 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:01.984+0000 7f598e9ad740 1 --2- 172.21.15.107:0/257653794 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5568bd4775a0 0x5568bd47fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:01:01.970 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:01.984+0000 7f598e9ad740 1 --2- 172.21.15.107:0/257653794 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5568bd476c80 0x5568bd477060 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:01:01.970 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:01.984+0000 7f598e9ad740 1 -- 172.21.15.107:0/257653794 >> 172.21.15.107:0/257653794 conn(0x5568bd4715a0 msgr2=0x5568bd471980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:01:01.970 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:01:01 smithi107 ceph-mon[31381]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 2.0 KiB/s wr, 16 op/s 2024-03-08T21:01:01.970 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:01:01 smithi107 ceph-mon[31381]: mgrmap e24: x(active, since 28s), standbys: y, z 2024-03-08T21:01:01.974 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:01.984+0000 7f598e9ad740 1 -- 172.21.15.107:0/257653794 shutdown_connections 2024-03-08T21:01:01.974 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:01.984+0000 7f5989200640 1 --2- 172.21.15.107:0/1023141434 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f596c0a2b30 0x7f596c0a4ff0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f5984108700 tx=0x7f5978005a20 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:01:01.974 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:01.984+0000 7f598e9ad740 1 -- 172.21.15.107:0/257653794 wait complete. 2024-03-08T21:01:01.974 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:01.985+0000 7f598e9ad740 1 -- start start 2024-03-08T21:01:01.974 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:01.985+0000 7f598e9ad740 1 Processor -- start 2024-03-08T21:01:01.974 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:01.985+0000 7f598e9ad740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5568bd476c80 0x5568bd47fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:01:01.974 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:01.985+0000 7f598e9ad740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5568bd4775a0 0x5568bd47d130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:01:01.975 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:01.985+0000 7f598e9ad740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5568bd480370 0x5568bd47d870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:01:01.975 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:01.985+0000 7f598e9ad740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568bd482fa0 con 0x5568bd4775a0 2024-03-08T21:01:01.975 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:01.985+0000 7f598e9ad740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568bd3969f0 con 0x5568bd476c80 2024-03-08T21:01:01.975 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:01.985+0000 7f598e9ad740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568bd483120 con 0x5568bd480370 2024-03-08T21:01:01.975 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:01.985+0000 7f598e9ad740 10 client.0 _ll_register_callbacks cb 0x5568bd0eea00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T21:01:01.975 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:01.986+0000 7f5983e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5568bd4775a0 0x5568bd47d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:01:01.975 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:01.986+0000 7f5983e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5568bd4775a0 0x5568bd47d130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:42070/0 (socket says 172.21.15.107:42070) 2024-03-08T21:01:01.975 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:01.986+0000 7f5983e00640 1 -- 172.21.15.107:0/2670087245 learned_addr learned my addr 172.21.15.107:0/2670087245 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:01:01.975 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:01.986+0000 7f5989200640 1 --2- 172.21.15.107:0/2670087245 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5568bd476c80 0x5568bd47fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:01:01.975 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:01.986+0000 7f5983e00640 1 -- 172.21.15.107:0/2670087245 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5568bd480370 msgr2=0x5568bd47d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:01:01.975 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:01.986+0000 7f5983e00640 1 --2- 172.21.15.107:0/2670087245 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5568bd480370 0x5568bd47d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:01:01.975 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:01.986+0000 7f5983e00640 1 -- 172.21.15.107:0/2670087245 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5568bd476c80 msgr2=0x5568bd47fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:01:01.975 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:01.986+0000 7f5983e00640 1 --2- 172.21.15.107:0/2670087245 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5568bd476c80 0x5568bd47fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:01:01.975 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:01.986+0000 7f5983e00640 1 -- 172.21.15.107:0/2670087245 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5568bd0ef210 con 0x5568bd4775a0 2024-03-08T21:01:01.976 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:01.986+0000 7f5989200640 1 --2- 172.21.15.107:0/2670087245 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5568bd476c80 0x5568bd47fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T21:01:01.976 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:01.986+0000 7f5983e00640 1 --2- 172.21.15.107:0/2670087245 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5568bd4775a0 0x5568bd47d130 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f5970097720 tx=0x7f597009ac20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=878d042857e60225 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:01:01.976 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:01.986+0000 7f596aa00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-03-08T21:01:01.976 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:01.987+0000 7f596aa00640 1 -- 172.21.15.107:0/2670087245 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f597001d040 con 0x5568bd4775a0 2024-03-08T21:01:01.976 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:01.987+0000 7f596aa00640 1 -- 172.21.15.107:0/2670087245 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f597009f120 con 0x5568bd4775a0 2024-03-08T21:01:01.976 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:01.987+0000 7f598e9ad740 1 -- 172.21.15.107:0/2670087245 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f597c0158c0 con 0x5568bd4775a0 2024-03-08T21:01:01.976 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:01.987+0000 7f596aa00640 1 -- 172.21.15.107:0/2670087245 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f597009f410 con 0x5568bd4775a0 2024-03-08T21:01:01.976 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:01.987+0000 7f598e9ad740 10 client.15501 fetch_fsmap learned FSMap version 28 2024-03-08T21:01:01.976 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:01.987+0000 7f598e9ad740 1 -- 172.21.15.107:0/2670087245 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5568bd1bb620 con 0x5568bd4775a0 2024-03-08T21:01:01.976 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:01.987+0000 7f596aa00640 1 -- 172.21.15.107:0/2670087245 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f597c0158c0 con 0x5568bd4775a0 2024-03-08T21:01:01.976 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:01.987+0000 7f596aa00640 1 -- 172.21.15.107:0/2670087245 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 28) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f597009a830 con 0x5568bd4775a0 2024-03-08T21:01:01.976 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:01.987+0000 7f598e9ad740 10 client.15501 fetch_fsmap finished waiting for FSMap version 28 2024-03-08T21:01:01.976 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:01.987+0000 7f598e9ad740 10 client.15501 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-08T21:01:01.976 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:01.987+0000 7f598e9ad740 20 client.15501 populate_metadata read hostname 'smithi107' 2024-03-08T21:01:01.976 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:01.987+0000 7f598e9ad740 1 --2- 172.21.15.107:0/2670087245 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x5568bd490190 0x5568bd492580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:01:01.977 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:01.987+0000 7f598e9ad740 4 client.15501 mds_command: new command op to 34475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-08T21:01:01.977 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:01.987+0000 7f598e9ad740 1 -- 172.21.15.107:0/2670087245 --> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5568bd3e7910 con 0x5568bd490190 2024-03-08T21:01:01.977 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:01.988+0000 7f5989200640 1 --2- 172.21.15.107:0/2670087245 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x5568bd490190 0x5568bd492580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:01:01.977 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:01.989+0000 7f5989200640 1 --2- 172.21.15.107:0/2670087245 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x5568bd490190 0x5568bd492580 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=103956ba87f4d3b4 server_cookie=de876c6e9fa599d6 in_seq=0 out_seq=0 2024-03-08T21:01:01.977 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:01.989+0000 7f596aa00640 10 client.15501 ms_handle_connect on v2:172.21.15.149:6838/55290596 2024-03-08T21:01:01.977 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:01.991+0000 7f596aa00640 1 -- 172.21.15.107:0/2670087245 <== mds.0 v2:172.21.15.149:6838/55290596 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5568bd3e7910 con 0x5568bd490190 2024-03-08T21:01:01.977 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:01.991+0000 7f596aa00640 10 client.15501 handle_command_reply: tid=0 2024-03-08T21:01:01.977 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:01.991+0000 7f598e9ad740 1 client.15501 shutdown 2024-03-08T21:01:01.977 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:01.991+0000 7f598e9ad740 1 -- 172.21.15.107:0/2670087245 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5568bd4775a0 msgr2=0x5568bd47d130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:01:01.977 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:01.991+0000 7f598e9ad740 1 --2- 172.21.15.107:0/2670087245 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5568bd4775a0 0x5568bd47d130 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f5970097720 tx=0x7f597009ac20 comp rx=0 tx=0).stop 2024-03-08T21:01:01.977 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:01.991+0000 7f598e9ad740 1 -- 172.21.15.107:0/2670087245 shutdown_connections 2024-03-08T21:01:01.977 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:01.991+0000 7f598e9ad740 1 --2- 172.21.15.107:0/2670087245 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5568bd480370 0x5568bd47d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:01:01.977 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:01.991+0000 7f598e9ad740 1 --2- 172.21.15.107:0/2670087245 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5568bd4775a0 0x5568bd47d130 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:01:01.977 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:01.991+0000 7f598e9ad740 1 --2- 172.21.15.107:0/2670087245 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x5568bd490190 0x5568bd492580 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:01:01.978 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:01.991+0000 7f598e9ad740 1 --2- 172.21.15.107:0/2670087245 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5568bd476c80 0x5568bd47fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:01:01.978 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:01.991+0000 7f598e9ad740 1 -- 172.21.15.107:0/2670087245 >> 172.21.15.107:0/2670087245 conn(0x5568bd4715a0 msgr2=0x5568bd4732f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:01:01.978 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:01.991+0000 7f596aa00640 0 client.15501 ms_handle_reset on v2:172.21.15.149:6838/55290596 2024-03-08T21:01:01.978 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:01.991+0000 7f598e9ad740 1 -- 172.21.15.107:0/2670087245 shutdown_connections 2024-03-08T21:01:01.978 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:01.991+0000 7f598e9ad740 1 -- 172.21.15.107:0/2670087245 wait complete. 2024-03-08T21:01:01.978 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:01.991+0000 7f598e9ad740 20 client.15501 trim_cache size 0 max 16384 2024-03-08T21:01:01.989 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:02.005+0000 7f598e9ad740 1 Processor -- start 2024-03-08T21:01:01.989 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:02.006+0000 7f598e9ad740 1 -- start start 2024-03-08T21:01:01.989 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:02.006+0000 7f598e9ad740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5568bd476c80 0x5568bd47f860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:01:01.989 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:02.006+0000 7f598e9ad740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5568bd4775a0 0x5568bd47fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:01:01.989 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:02.006+0000 7f598e9ad740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5568bd480370 0x5568bd47a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:01:01.989 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:02.006+0000 7f598e9ad740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59841aee50 con 0x5568bd4775a0 2024-03-08T21:01:01.989 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:02.006+0000 7f598e9ad740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59841b07d0 con 0x5568bd480370 2024-03-08T21:01:01.989 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:02.006+0000 7f598e9ad740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59841af4b0 con 0x5568bd476c80 2024-03-08T21:01:01.990 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:02.006+0000 7f5983e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5568bd4775a0 0x5568bd47fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:01:01.990 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:02.006+0000 7f5983e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5568bd4775a0 0x5568bd47fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:42076/0 (socket says 172.21.15.107:42076) 2024-03-08T21:01:01.990 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:02.006+0000 7f5983e00640 1 -- 172.21.15.107:0/3678626161 learned_addr learned my addr 172.21.15.107:0/3678626161 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:01:01.990 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:02.006+0000 7f5983e00640 1 -- 172.21.15.107:0/3678626161 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5568bd476c80 msgr2=0x5568bd47f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:01:01.990 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:02.006+0000 7f5989200640 1 --2- 172.21.15.107:0/3678626161 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5568bd476c80 0x5568bd47f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:01:01.990 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:02.007+0000 7f5989c00640 1 --2- 172.21.15.107:0/3678626161 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5568bd480370 0x5568bd47a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:01:01.990 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:02.007+0000 7f5983e00640 1 --2- 172.21.15.107:0/3678626161 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5568bd476c80 0x5568bd47f860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:01:01.990 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:02.007+0000 7f5983e00640 1 -- 172.21.15.107:0/3678626161 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5568bd480370 msgr2=0x5568bd47a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:01:01.990 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:02.007+0000 7f5983e00640 1 --2- 172.21.15.107:0/3678626161 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5568bd480370 0x5568bd47a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:01:01.990 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:02.007+0000 7f5983e00640 1 -- 172.21.15.107:0/3678626161 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5568bd4928b0 con 0x5568bd4775a0 2024-03-08T21:01:01.990 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:02.007+0000 7f5989c00640 1 --2- 172.21.15.107:0/3678626161 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5568bd480370 0x5568bd47a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T21:01:01.991 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:02.007+0000 7f5983e00640 1 --2- 172.21.15.107:0/3678626161 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5568bd4775a0 0x5568bd47fda0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f5970097c90 tx=0x7f597009ad50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=346b68309d5da896 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:01:01.991 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:02.007+0000 7f5975600640 1 -- 172.21.15.107:0/3678626161 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f597001d070 con 0x5568bd4775a0 2024-03-08T21:01:01.991 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:02.007+0000 7f5975600640 1 -- 172.21.15.107:0/3678626161 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f59700a3a60 con 0x5568bd4775a0 2024-03-08T21:01:01.991 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:02.007+0000 7f5975600640 1 -- 172.21.15.107:0/3678626161 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f597009a950 con 0x5568bd4775a0 2024-03-08T21:01:01.994 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:02.008+0000 7f598e9ad740 1 -- 172.21.15.107:0/3678626161 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5568bd4775a0 msgr2=0x5568bd47fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:01:01.994 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:02.008+0000 7f598e9ad740 1 --2- 172.21.15.107:0/3678626161 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5568bd4775a0 0x5568bd47fda0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f5970097c90 tx=0x7f597009ad50 comp rx=0 tx=0).stop 2024-03-08T21:01:01.994 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:02.008+0000 7f598e9ad740 1 -- 172.21.15.107:0/3678626161 shutdown_connections 2024-03-08T21:01:01.994 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:02.008+0000 7f598e9ad740 1 --2- 172.21.15.107:0/3678626161 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5568bd480370 0x5568bd47a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:01:01.995 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:02.008+0000 7f598e9ad740 1 --2- 172.21.15.107:0/3678626161 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5568bd4775a0 0x5568bd47fda0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:01:01.995 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:02.008+0000 7f598e9ad740 1 --2- 172.21.15.107:0/3678626161 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5568bd476c80 0x5568bd47f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:01:01.995 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:02.008+0000 7f598e9ad740 1 -- 172.21.15.107:0/3678626161 >> 172.21.15.107:0/3678626161 conn(0x5568bd4715a0 msgr2=0x5568bd471980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:01:01.995 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:02.008+0000 7f598e9ad740 1 -- 172.21.15.107:0/3678626161 shutdown_connections 2024-03-08T21:01:01.995 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:02.008+0000 7f598e9ad740 1 -- 172.21.15.107:0/3678626161 wait complete. 2024-03-08T21:01:01.995 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:02.008+0000 7f598e9ad740 1 -- start start 2024-03-08T21:01:01.995 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:02.008+0000 7f598e9ad740 1 Processor -- start 2024-03-08T21:01:01.995 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:02.009+0000 7f598e9ad740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5568bd476c80 0x5568bd4863c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:01:01.995 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:02.009+0000 7f598e9ad740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5568bd4775a0 0x5568bd486900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:01:01.995 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:02.009+0000 7f598e9ad740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5568bd480370 0x5568bd4e6670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:01:01.995 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:02.009+0000 7f598e9ad740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59841b0170 con 0x5568bd476c80 2024-03-08T21:01:01.996 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:02.009+0000 7f598e9ad740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59841afb10 con 0x5568bd480370 2024-03-08T21:01:01.996 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:02.009+0000 7f598e9ad740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59841ad0f0 con 0x5568bd4775a0 2024-03-08T21:01:01.996 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:02.009+0000 7f598e9ad740 10 client.0 _ll_register_callbacks cb 0x7f597c009710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T21:01:01.996 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:02.009+0000 7f5989200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5568bd476c80 0x5568bd4863c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:01:01.996 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:02.009+0000 7f5989200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5568bd476c80 0x5568bd4863c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:42092/0 (socket says 172.21.15.107:42092) 2024-03-08T21:01:01.996 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:02.009+0000 7f5989200640 1 -- 172.21.15.107:0/656753405 learned_addr learned my addr 172.21.15.107:0/656753405 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:01:01.996 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:02.009+0000 7f5989200640 1 -- 172.21.15.107:0/656753405 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5568bd4775a0 msgr2=0x5568bd486900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:01:01.996 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:02.009+0000 7f5989200640 1 --2- 172.21.15.107:0/656753405 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5568bd4775a0 0x5568bd486900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:01:01.996 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:02.009+0000 7f5989200640 1 -- 172.21.15.107:0/656753405 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5568bd480370 msgr2=0x5568bd4e6670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:01:01.996 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:02.009+0000 7f5989200640 1 --2- 172.21.15.107:0/656753405 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5568bd480370 0x5568bd4e6670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:01:01.996 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:02.009+0000 7f5989200640 1 -- 172.21.15.107:0/656753405 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5568bd4e6bb0 con 0x5568bd476c80 2024-03-08T21:01:01.996 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:02.010+0000 7f5989200640 1 --2- 172.21.15.107:0/656753405 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5568bd476c80 0x5568bd4863c0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f5978000c00 tx=0x7f597802bc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f3b2ea6affc077f server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:01:01.996 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:02.010+0000 7f596aa00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-03-08T21:01:01.997 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:02.010+0000 7f596aa00640 1 -- 172.21.15.107:0/656753405 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5978005610 con 0x5568bd476c80 2024-03-08T21:01:01.997 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:02.010+0000 7f596aa00640 1 -- 172.21.15.107:0/656753405 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f59780046c0 con 0x5568bd476c80 2024-03-08T21:01:01.997 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:02.010+0000 7f598e9ad740 1 -- 172.21.15.107:0/656753405 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f59700a3a60 con 0x5568bd476c80 2024-03-08T21:01:01.997 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:02.010+0000 7f596aa00640 1 -- 172.21.15.107:0/656753405 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f597805e050 con 0x5568bd476c80 2024-03-08T21:01:01.997 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:02.010+0000 7f598e9ad740 10 client.15507 fetch_fsmap learned FSMap version 28 2024-03-08T21:01:01.997 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:02.010+0000 7f598e9ad740 1 -- 172.21.15.107:0/656753405 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5568bd4e7df0 con 0x5568bd476c80 2024-03-08T21:01:01.997 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:02.010+0000 7f596aa00640 1 -- 172.21.15.107:0/656753405 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f59700a3a60 con 0x5568bd476c80 2024-03-08T21:01:01.997 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:02.011+0000 7f596aa00640 1 -- 172.21.15.107:0/656753405 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 28) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f597804a040 con 0x5568bd476c80 2024-03-08T21:01:01.997 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:02.011+0000 7f598e9ad740 10 client.15507 fetch_fsmap finished waiting for FSMap version 28 2024-03-08T21:01:01.997 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:02.011+0000 7f598e9ad740 10 client.15507 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-08T21:01:01.997 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:02.011+0000 7f598e9ad740 20 client.15507 populate_metadata read hostname 'smithi107' 2024-03-08T21:01:01.997 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:02.011+0000 7f598e9ad740 1 --2- 172.21.15.107:0/656753405 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x5568bd4e84d0 0x5568bd4ea970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:01:01.997 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:02.011+0000 7f598e9ad740 4 client.15507 mds_command: new command op to 34475 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-08T21:01:01.997 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:02.011+0000 7f598e9ad740 1 -- 172.21.15.107:0/656753405 --> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5568bd3e1c60 con 0x5568bd4e84d0 2024-03-08T21:01:01.998 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:02.011+0000 7f5983e00640 1 --2- 172.21.15.107:0/656753405 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x5568bd4e84d0 0x5568bd4ea970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:01:01.998 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:02.012+0000 7f5983e00640 1 --2- 172.21.15.107:0/656753405 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x5568bd4e84d0 0x5568bd4ea970 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cc0ada6c88ec0aa6 server_cookie=ccb9c1c23bdbaa90 in_seq=0 out_seq=0 2024-03-08T21:01:01.998 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:02.012+0000 7f596aa00640 10 client.15507 ms_handle_connect on v2:172.21.15.149:6838/55290596 2024-03-08T21:01:03.877 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:01:03 smithi204 ceph-mon[34935]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.5 KiB/s wr, 12 op/s 2024-03-08T21:01:03.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:01:03 smithi107 ceph-mon[31381]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.5 KiB/s wr, 12 op/s 2024-03-08T21:01:04.014 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:01:03 smithi149 ceph-mon[35042]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.5 KiB/s wr, 12 op/s 2024-03-08T21:01:04.171 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:04.189+0000 7f596aa00640 1 -- 172.21.15.107:0/656753405 <== mds.0 v2:172.21.15.149:6838/55290596 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5568bd3e1c60 con 0x5568bd4e84d0 2024-03-08T21:01:04.171 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:04.189+0000 7f596aa00640 10 client.15507 handle_command_reply: tid=0 2024-03-08T21:01:04.171 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:04.189+0000 7f598e9ad740 1 client.15507 shutdown 2024-03-08T21:01:04.172 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:04.189+0000 7f598e9ad740 1 -- 172.21.15.107:0/656753405 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5568bd476c80 msgr2=0x5568bd4863c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:01:04.172 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:04.189+0000 7f598e9ad740 1 --2- 172.21.15.107:0/656753405 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5568bd476c80 0x5568bd4863c0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f5978000c00 tx=0x7f597802bc40 comp rx=0 tx=0).stop 2024-03-08T21:01:04.172 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:04.189+0000 7f598e9ad740 1 -- 172.21.15.107:0/656753405 shutdown_connections 2024-03-08T21:01:04.172 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:04.189+0000 7f598e9ad740 1 --2- 172.21.15.107:0/656753405 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x5568bd4e84d0 0x5568bd4ea970 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:01:04.172 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:04.189+0000 7f598e9ad740 1 --2- 172.21.15.107:0/656753405 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5568bd480370 0x5568bd4e6670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:01:04.172 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:04.189+0000 7f598e9ad740 1 --2- 172.21.15.107:0/656753405 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5568bd4775a0 0x5568bd486900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:01:04.172 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:04.189+0000 7f598e9ad740 1 --2- 172.21.15.107:0/656753405 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5568bd476c80 0x5568bd4863c0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:01:04.172 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:04.189+0000 7f596aa00640 0 client.15507 ms_handle_reset on v2:172.21.15.149:6838/55290596 2024-03-08T21:01:04.173 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:04.189+0000 7f598e9ad740 1 -- 172.21.15.107:0/656753405 >> 172.21.15.107:0/656753405 conn(0x5568bd4715a0 msgr2=0x5568bd472920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:01:04.173 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:04.189+0000 7f598e9ad740 1 -- 172.21.15.107:0/656753405 shutdown_connections 2024-03-08T21:01:04.173 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:04.189+0000 7f598e9ad740 1 -- 172.21.15.107:0/656753405 wait complete. 2024-03-08T21:01:04.173 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:04.190+0000 7f598e9ad740 20 client.15507 trim_cache size 0 max 16384 2024-03-08T21:01:04.173 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:{ 2024-03-08T21:01:04.173 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "message": "", 2024-03-08T21:01:04.174 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "return_code": 0 2024-03-08T21:01:04.174 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:} 2024-03-08T21:01:04.175 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:04.191+0000 7f598bc00640 1 -- 172.21.15.107:0/1023141434 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f596c0a2b30 msgr2=0x7f596c0a4ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:01:04.175 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:04.191+0000 7f598bc00640 1 --2- 172.21.15.107:0/1023141434 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f596c0a2b30 0x7f596c0a4ff0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f5984108700 tx=0x7f5978005a20 comp rx=0 tx=0).stop 2024-03-08T21:01:04.175 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:04.191+0000 7f598bc00640 1 -- 172.21.15.107:0/1023141434 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5984103e90 msgr2=0x7f598419fe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:01:04.175 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:04.191+0000 7f598bc00640 1 --2- 172.21.15.107:0/1023141434 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5984103e90 0x7f598419fe50 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f597000c890 tx=0x7f597000cd50 comp rx=0 tx=0).stop 2024-03-08T21:01:04.175 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:04.191+0000 7f598bc00640 1 -- 172.21.15.107:0/1023141434 shutdown_connections 2024-03-08T21:01:04.175 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:04.191+0000 7f598bc00640 1 --2- 172.21.15.107:0/1023141434 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f596c0a2b30 0x7f596c0a4ff0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:01:04.175 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:04.192+0000 7f598bc00640 1 --2- 172.21.15.107:0/1023141434 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5984109590 0x7f5984199b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:01:04.175 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:04.192+0000 7f598bc00640 1 --2- 172.21.15.107:0/1023141434 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5984103e90 0x7f598419fe50 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:01:04.175 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:04.192+0000 7f598bc00640 1 --2- 172.21.15.107:0/1023141434 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f59841034d0 0x7f598419f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:01:04.175 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:04.192+0000 7f598bc00640 1 -- 172.21.15.107:0/1023141434 >> 172.21.15.107:0/1023141434 conn(0x7f59840fefe0 msgr2=0x7f5984100770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:01:04.175 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:04.192+0000 7f598bc00640 1 -- 172.21.15.107:0/1023141434 shutdown_connections 2024-03-08T21:01:04.175 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:04.192+0000 7f598bc00640 1 -- 172.21.15.107:0/1023141434 wait complete. 2024-03-08T21:01:05.255 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:01:04 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[66008]: *** Caught signal (Segmentation fault) ** 2024-03-08T21:01:05.255 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:01:04 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[66008]: in thread 7fc5c5e00700 thread_name:md_log_replay 2024-03-08T21:01:05.255 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:01:04 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[66008]: ceph version 19.0.0-1965-g7d63482a (7d63482aa89628d733e298984a762389b625c761) squid (dev) 2024-03-08T21:01:05.255 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:01:04 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[66008]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fc5da40fd20] 2024-03-08T21:01:05.255 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:01:04 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[66008]: 2: /usr/bin/ceph-mds(+0x503ee2) [0x55692431eee2] 2024-03-08T21:01:05.255 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:01:04 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[66008]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x556924322773] 2024-03-08T21:01:05.255 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:01:04 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[66008]: 4: (MDLog::_replay_thread()+0x617) [0x55692437e587] 2024-03-08T21:01:05.256 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:01:04 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[66008]: 5: (MDLog::ReplayThread::entry()+0x11) [0x556924387091] 2024-03-08T21:01:05.256 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:01:04 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[66008]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7fc5da4051ca] 2024-03-08T21:01:05.256 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:01:04 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[66008]: 7: clone() 2024-03-08T21:01:05.256 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:01:04 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[66008]: 2024-03-08T21:01:04.971+0000 7fc5c5e00700 -1 *** Caught signal (Segmentation fault) ** 2024-03-08T21:01:05.256 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:01:04 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[66008]: in thread 7fc5c5e00700 thread_name:md_log_replay 2024-03-08T21:01:05.256 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:01:04 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[66008]: 2024-03-08T21:01:05.256 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:01:04 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[66008]: ceph version 19.0.0-1965-g7d63482a (7d63482aa89628d733e298984a762389b625c761) squid (dev) 2024-03-08T21:01:05.256 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:01:04 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[66008]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fc5da40fd20] 2024-03-08T21:01:05.256 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:01:04 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[66008]: 2: /usr/bin/ceph-mds(+0x503ee2) [0x55692431eee2] 2024-03-08T21:01:05.256 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:01:04 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[66008]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x556924322773] 2024-03-08T21:01:05.256 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:01:04 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[66008]: 4: (MDLog::_replay_thread()+0x617) [0x55692437e587] 2024-03-08T21:01:05.256 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:01:04 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[66008]: 5: (MDLog::ReplayThread::entry()+0x11) [0x556924387091] 2024-03-08T21:01:05.256 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:01:04 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[66008]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7fc5da4051ca] 2024-03-08T21:01:05.256 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:01:04 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[66008]: 7: clone() 2024-03-08T21:01:05.256 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:01:04 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[66008]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-08T21:01:05.257 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:01:04 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[66008]: 2024-03-08T21:01:05.257 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:01:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[66008]: 0> 2024-03-08T21:01:04.971+0000 7fc5c5e00700 -1 *** Caught signal (Segmentation fault) ** 2024-03-08T21:01:05.257 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:01:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[66008]: in thread 7fc5c5e00700 thread_name:md_log_replay 2024-03-08T21:01:05.257 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:01:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[66008]: 2024-03-08T21:01:05.257 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:01:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[66008]: ceph version 19.0.0-1965-g7d63482a (7d63482aa89628d733e298984a762389b625c761) squid (dev) 2024-03-08T21:01:05.257 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:01:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[66008]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fc5da40fd20] 2024-03-08T21:01:05.257 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:01:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[66008]: 2: /usr/bin/ceph-mds(+0x503ee2) [0x55692431eee2] 2024-03-08T21:01:05.257 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:01:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[66008]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x556924322773] 2024-03-08T21:01:05.257 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:01:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[66008]: 4: (MDLog::_replay_thread()+0x617) [0x55692437e587] 2024-03-08T21:01:05.258 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:01:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[66008]: 5: (MDLog::ReplayThread::entry()+0x11) [0x556924387091] 2024-03-08T21:01:05.258 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:01:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[66008]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7fc5da4051ca] 2024-03-08T21:01:05.258 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:01:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[66008]: 7: clone() 2024-03-08T21:01:05.258 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:01:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[66008]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-08T21:01:05.258 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:01:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[66008]: 2024-03-08T21:01:05.258 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:01:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[66008]: 0> 2024-03-08T21:01:04.971+0000 7fc5c5e00700 -1 *** Caught signal (Segmentation fault) ** 2024-03-08T21:01:05.258 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:01:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[66008]: in thread 7fc5c5e00700 thread_name:md_log_replay 2024-03-08T21:01:05.258 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:01:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[66008]: 2024-03-08T21:01:05.258 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:01:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[66008]: ceph version 19.0.0-1965-g7d63482a (7d63482aa89628d733e298984a762389b625c761) squid (dev) 2024-03-08T21:01:05.258 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:01:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[66008]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fc5da40fd20] 2024-03-08T21:01:05.258 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:01:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[66008]: 2: /usr/bin/ceph-mds(+0x503ee2) [0x55692431eee2] 2024-03-08T21:01:05.258 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:01:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[66008]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x556924322773] 2024-03-08T21:01:05.258 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:01:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[66008]: 4: (MDLog::_replay_thread()+0x617) [0x55692437e587] 2024-03-08T21:01:05.258 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:01:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[66008]: 5: (MDLog::ReplayThread::entry()+0x11) [0x556924387091] 2024-03-08T21:01:05.259 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:01:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[66008]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7fc5da4051ca] 2024-03-08T21:01:05.259 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:01:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[66008]: 7: clone() 2024-03-08T21:01:05.259 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:01:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[66008]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-08T21:01:05.259 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:01:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[66008]: 2024-03-08T21:01:05.259 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:01:05 smithi204 podman[69729]: 2024-03-08 21:01:05.080721773 +0000 UTC m=+0.038387410 container died b889757eaf94a48745a7e36278da2e4be29ea539739481d5951faa7465392322 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=rishabh-2024mar8-7d63482, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-03-08T21:01:05.877 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:01:05 smithi204 ceph-mon[34935]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 24 KiB/s wr, 32 op/s 2024-03-08T21:01:05.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:01:05 smithi107 ceph-mon[31381]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 24 KiB/s wr, 32 op/s 2024-03-08T21:01:06.014 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:01:05 smithi149 ceph-mon[35042]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 24 KiB/s wr, 32 op/s 2024-03-08T21:01:06.354 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:01:05 smithi204 podman[69729]: 2024-03-08 21:01:05.919296865 +0000 UTC m=+0.876962523 container remove b889757eaf94a48745a7e36278da2e4be29ea539739481d5951faa7465392322 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=rishabh-2024mar8-7d63482, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-08T21:01:06.354 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:01:05 smithi204 systemd[1]: ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715@mds.c.service: Main process exited, code=exited, status=139/n/a 2024-03-08T21:01:06.627 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:01:06 smithi204 systemd[1]: ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715@mds.c.service: Failed with result 'exit-code'. 2024-03-08T21:01:06.627 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:01:06 smithi204 systemd[1]: ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715@mds.c.service: Consumed 2.120s CPU time. 2024-03-08T21:01:07.877 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:01:07 smithi204 ceph-mon[34935]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-03-08T21:01:07.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:01:07 smithi107 ceph-mon[31381]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-03-08T21:01:08.014 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:01:07 smithi149 ceph-mon[35042]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-03-08T21:01:09.877 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:01:09 smithi204 ceph-mon[34935]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-03-08T21:01:09.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:01:09 smithi107 ceph-mon[31381]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-03-08T21:01:10.014 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:01:09 smithi149 ceph-mon[35042]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-03-08T21:01:11.877 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:01:11 smithi204 ceph-mon[34935]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 28 op/s 2024-03-08T21:01:11.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:01:11 smithi107 ceph-mon[31381]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 28 op/s 2024-03-08T21:01:12.014 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:01:11 smithi149 ceph-mon[35042]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 28 op/s 2024-03-08T21:01:13.764 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:01:13 smithi149 ceph-mon[35042]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-03-08T21:01:13.877 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:01:13 smithi204 ceph-mon[34935]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-03-08T21:01:13.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:01:13 smithi107 ceph-mon[31381]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-03-08T21:01:15.877 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:01:15 smithi204 ceph-mon[34935]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 KiB/s wr, 27 op/s 2024-03-08T21:01:15.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:01:15 smithi107 ceph-mon[31381]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 KiB/s wr, 27 op/s 2024-03-08T21:01:16.014 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:01:15 smithi149 ceph-mon[35042]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 KiB/s wr, 27 op/s 2024-03-08T21:01:16.776 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:01:16 smithi204 systemd[1]: ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715@mds.c.service: Scheduled restart job, restart counter is at 1. 2024-03-08T21:01:16.776 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:01:16 smithi204 systemd[1]: Stopped Ceph mds.c for dc20f0f8-dd8c-11ee-95c6-87774f69a715. 2024-03-08T21:01:16.776 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:01:16 smithi204 systemd[1]: ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715@mds.c.service: Consumed 2.120s CPU time. 2024-03-08T21:01:16.776 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:01:16 smithi204 systemd[1]: Starting Ceph mds.c for dc20f0f8-dd8c-11ee-95c6-87774f69a715... 2024-03-08T21:01:17.116 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:01:16 smithi204 podman[69831]: 2024-03-08 21:01:16.701126281 +0000 UTC m=+0.036028007 image pull a272146cb810b1fd4a53627f019472132e30966d5c25b01bfc44a001295f1249 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd 2024-03-08T21:01:17.116 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:01:16 smithi204 podman[69831]: 2024-03-08 21:01:16.841199684 +0000 UTC m=+0.176101442 container create 1633bce08122b382f994e613bbd2c64c80c2d7ffbef441b57ad2f7ac547c941c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=rishabh-2024mar8-7d63482, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-03-08T21:01:17.324 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.341+0000 7fb9abe00640 1 -- 172.21.15.107:0/4049798619 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb9ac106410 msgr2=0x7fb9ac10dfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:01:17.324 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.341+0000 7fb9abe00640 1 --2- 172.21.15.107:0/4049798619 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb9ac106410 0x7fb9ac10dfe0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fb9a4009ca0 tx=0x7fb9a402c0f0 comp rx=0 tx=0).stop 2024-03-08T21:01:17.324 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.341+0000 7fb9abe00640 1 -- 172.21.15.107:0/4049798619 shutdown_connections 2024-03-08T21:01:17.324 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.341+0000 7fb9abe00640 1 --2- 172.21.15.107:0/4049798619 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb9ac106410 0x7fb9ac10dfe0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:01:17.324 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.341+0000 7fb9abe00640 1 --2- 172.21.15.107:0/4049798619 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb9ac101cf0 0x7fb9ac105d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:01:17.325 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.341+0000 7fb9abe00640 1 --2- 172.21.15.107:0/4049798619 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb9ac101340 0x7fb9ac101720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:01:17.325 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.341+0000 7fb9abe00640 1 -- 172.21.15.107:0/4049798619 >> 172.21.15.107:0/4049798619 conn(0x7fb9ac0fcc40 msgr2=0x7fb9ac0ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:01:17.325 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.341+0000 7fb9abe00640 1 -- 172.21.15.107:0/4049798619 shutdown_connections 2024-03-08T21:01:17.325 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.341+0000 7fb9abe00640 1 -- 172.21.15.107:0/4049798619 wait complete. 2024-03-08T21:01:17.327 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.342+0000 7fb9abe00640 1 Processor -- start 2024-03-08T21:01:17.327 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.342+0000 7fb9abe00640 1 -- start start 2024-03-08T21:01:17.327 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.343+0000 7fb9abe00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb9ac101340 0x7fb9ac19d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:01:17.327 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.343+0000 7fb9abe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb9ac106410 0x7fb9ac19db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:01:17.327 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.343+0000 7fb9abe00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb9ac1976e0 0x7fb9ac197b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:01:17.328 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.343+0000 7fb9abe00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9ac111380 con 0x7fb9ac106410 2024-03-08T21:01:17.328 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.343+0000 7fb9abe00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9ac111200 con 0x7fb9ac101340 2024-03-08T21:01:17.328 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.343+0000 7fb9abe00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9ac111500 con 0x7fb9ac1976e0 2024-03-08T21:01:17.328 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.343+0000 7fb9aa000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb9ac106410 0x7fb9ac19db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:01:17.328 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.343+0000 7fb9aa000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb9ac106410 0x7fb9ac19db60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:39448/0 (socket says 172.21.15.107:39448) 2024-03-08T21:01:17.328 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.343+0000 7fb9aa000640 1 -- 172.21.15.107:0/892881 learned_addr learned my addr 172.21.15.107:0/892881 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:01:17.328 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.343+0000 7fb9aaa00640 1 --2- 172.21.15.107:0/892881 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb9ac101340 0x7fb9ac19d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:01:17.328 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.343+0000 7fb9ab400640 1 --2- 172.21.15.107:0/892881 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb9ac1976e0 0x7fb9ac197b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:01:17.328 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.344+0000 7fb9aa000640 1 -- 172.21.15.107:0/892881 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb9ac1976e0 msgr2=0x7fb9ac197b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:01:17.328 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.344+0000 7fb9aa000640 1 --2- 172.21.15.107:0/892881 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb9ac1976e0 0x7fb9ac197b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:01:17.328 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.344+0000 7fb9aa000640 1 -- 172.21.15.107:0/892881 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb9ac101340 msgr2=0x7fb9ac19d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:01:17.328 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.344+0000 7fb9aa000640 1 --2- 172.21.15.107:0/892881 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb9ac101340 0x7fb9ac19d620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:01:17.328 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.344+0000 7fb9aa000640 1 -- 172.21.15.107:0/892881 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9ac1982e0 con 0x7fb9ac106410 2024-03-08T21:01:17.328 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.344+0000 7fb9aa000640 1 --2- 172.21.15.107:0/892881 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb9ac106410 0x7fb9ac19db60 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fb99800d740 tx=0x7fb99800dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:01:17.329 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.344+0000 7fb9a3400640 1 -- 172.21.15.107:0/892881 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb998017070 con 0x7fb9ac106410 2024-03-08T21:01:17.329 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.344+0000 7fb9abe00640 1 -- 172.21.15.107:0/892881 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9ac1a43c0 con 0x7fb9ac106410 2024-03-08T21:01:17.329 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.344+0000 7fb9a3400640 1 -- 172.21.15.107:0/892881 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fb99800de50 con 0x7fb9ac106410 2024-03-08T21:01:17.329 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.345+0000 7fb9abe00640 1 -- 172.21.15.107:0/892881 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9ac1a47c0 con 0x7fb9ac106410 2024-03-08T21:01:17.329 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.345+0000 7fb9a3400640 1 -- 172.21.15.107:0/892881 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb998020060 con 0x7fb9ac106410 2024-03-08T21:01:17.330 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.348+0000 7fb9b4413740 1 Processor -- start 2024-03-08T21:01:17.330 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.348+0000 7fb9b4413740 1 -- start start 2024-03-08T21:01:17.332 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.348+0000 7fb9b4413740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558354726290 0x558354726670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:01:17.332 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.348+0000 7fb9b4413740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558354726bb0 0x55835472f440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:01:17.332 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.348+0000 7fb9b4413740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55835472f980 0x558354731d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:01:17.332 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.348+0000 7fb9b4413740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583546853d0 con 0x558354726290 2024-03-08T21:01:17.332 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.348+0000 7fb9b4413740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55835462e6b0 con 0x55835472f980 2024-03-08T21:01:17.332 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.348+0000 7fb9b4413740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x558354681470 con 0x558354726bb0 2024-03-08T21:01:17.332 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.349+0000 7fb9a3400640 1 -- 172.21.15.107:0/892881 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 24) v1 ==== 137524+0+0 (secure 0 0 0) 0x7fb998004d00 con 0x7fb9ac106410 2024-03-08T21:01:17.332 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.349+0000 7fb9ab400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558354726290 0x558354726670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:01:17.332 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.349+0000 7fb9ab400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558354726290 0x558354726670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:39452/0 (socket says 172.21.15.107:39452) 2024-03-08T21:01:17.333 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.349+0000 7fb9ab400640 1 -- 172.21.15.107:0/2184071984 learned_addr learned my addr 172.21.15.107:0/2184071984 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:01:17.333 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.349+0000 7fb9aaa00640 1 --2- 172.21.15.107:0/2184071984 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558354726bb0 0x55835472f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:01:17.333 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.349+0000 7fb9ab400640 1 -- 172.21.15.107:0/2184071984 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558354726bb0 msgr2=0x55835472f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:01:17.333 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.349+0000 7fb9ab400640 1 --2- 172.21.15.107:0/2184071984 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558354726bb0 0x55835472f440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:01:17.333 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.349+0000 7fb9ab400640 1 -- 172.21.15.107:0/2184071984 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55835472f980 msgr2=0x558354731d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:01:17.333 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.349+0000 7fb9ab400640 1 --2- 172.21.15.107:0/2184071984 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55835472f980 0x558354731d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:01:17.333 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.349+0000 7fb9ab400640 1 -- 172.21.15.107:0/2184071984 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558354654b50 con 0x558354726290 2024-03-08T21:01:17.333 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.349+0000 7fb9aaa00640 1 --2- 172.21.15.107:0/2184071984 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558354726bb0 0x55835472f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T21:01:17.333 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.350+0000 7fb9ab400640 1 --2- 172.21.15.107:0/2184071984 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558354726290 0x558354726670 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fb9a402c700 tx=0x7fb9a4032340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc1bb179cbd6b0e server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:01:17.336 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.350+0000 7fb997400640 1 -- 172.21.15.107:0/2184071984 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9a4043020 con 0x558354726290 2024-03-08T21:01:17.336 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.350+0000 7fb997400640 1 -- 172.21.15.107:0/2184071984 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fb9a4032bb0 con 0x558354726290 2024-03-08T21:01:17.336 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.350+0000 7fb997400640 1 -- 172.21.15.107:0/2184071984 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9a4039040 con 0x558354726290 2024-03-08T21:01:17.336 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.350+0000 7fb9a3400640 1 --2- 172.21.15.107:0/892881 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7fb9900a2a80 0x7fb9900a4f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:01:17.336 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.351+0000 7fb9a3400640 1 -- 172.21.15.107:0/892881 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fb9980ceb40 con 0x7fb9ac106410 2024-03-08T21:01:17.336 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.351+0000 7fb9aaa00640 1 --2- 172.21.15.107:0/892881 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7fb9900a2a80 0x7fb9900a4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:01:17.337 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.351+0000 7fb9b4413740 1 -- 172.21.15.107:0/2184071984 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558354726290 msgr2=0x558354726670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:01:17.337 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.351+0000 7fb9b4413740 1 --2- 172.21.15.107:0/2184071984 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558354726290 0x558354726670 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fb9a402c700 tx=0x7fb9a4032340 comp rx=0 tx=0).stop 2024-03-08T21:01:17.337 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.351+0000 7fb9b4413740 1 -- 172.21.15.107:0/2184071984 shutdown_connections 2024-03-08T21:01:17.337 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.351+0000 7fb9b4413740 1 --2- 172.21.15.107:0/2184071984 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55835472f980 0x558354731d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:01:17.337 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.351+0000 7fb9b4413740 1 --2- 172.21.15.107:0/2184071984 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558354726bb0 0x55835472f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:01:17.337 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.351+0000 7fb9b4413740 1 --2- 172.21.15.107:0/2184071984 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558354726290 0x558354726670 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:01:17.337 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.351+0000 7fb9b4413740 1 -- 172.21.15.107:0/2184071984 >> 172.21.15.107:0/2184071984 conn(0x558354720bb0 msgr2=0x558354720f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:01:17.337 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.351+0000 7fb9b4413740 1 -- 172.21.15.107:0/2184071984 shutdown_connections 2024-03-08T21:01:17.337 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.351+0000 7fb9b4413740 1 -- 172.21.15.107:0/2184071984 wait complete. 2024-03-08T21:01:17.337 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.352+0000 7fb9aaa00640 1 --2- 172.21.15.107:0/892881 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7fb9900a2a80 0x7fb9900a4f40 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x558354725d40 tx=0x7fb99c005f50 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:01:17.337 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.352+0000 7fb9b4413740 1 -- start start 2024-03-08T21:01:17.337 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.352+0000 7fb9b4413740 1 Processor -- start 2024-03-08T21:01:17.337 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.352+0000 7fb9b4413740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558354726290 0x55835472f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:01:17.338 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.352+0000 7fb9b4413740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558354726bb0 0x55835472c780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:01:17.338 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.353+0000 7fb9b4413740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55835472f980 0x55835472cec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:01:17.338 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.353+0000 7fb9b4413740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x558354732550 con 0x55835472f980 2024-03-08T21:01:17.338 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.353+0000 7fb9b4413740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583546419f0 con 0x558354726bb0 2024-03-08T21:01:17.338 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.353+0000 7fb9b4413740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583547326d0 con 0x558354726290 2024-03-08T21:01:17.338 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.353+0000 7fb9b4413740 10 client.0 _ll_register_callbacks cb 0x558354399a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T21:01:17.338 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.353+0000 7fb9ab400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55835472f980 0x55835472cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:01:17.338 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.353+0000 7fb9ab400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55835472f980 0x55835472cec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:39468/0 (socket says 172.21.15.107:39468) 2024-03-08T21:01:17.338 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.353+0000 7fb9ab400640 1 -- 172.21.15.107:0/3508630778 learned_addr learned my addr 172.21.15.107:0/3508630778 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:01:17.338 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.353+0000 7fb9aa000640 1 --2- 172.21.15.107:0/3508630778 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558354726bb0 0x55835472c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:01:17.338 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.353+0000 7fb9aaa00640 1 --2- 172.21.15.107:0/3508630778 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558354726290 0x55835472f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:01:17.338 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.353+0000 7fb9ab400640 1 -- 172.21.15.107:0/3508630778 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558354726290 msgr2=0x55835472f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:01:17.338 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.353+0000 7fb9ab400640 1 --2- 172.21.15.107:0/3508630778 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558354726290 0x55835472f3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:01:17.338 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.353+0000 7fb9ab400640 1 -- 172.21.15.107:0/3508630778 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558354726bb0 msgr2=0x55835472c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:01:17.339 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.353+0000 7fb9ab400640 1 --2- 172.21.15.107:0/3508630778 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558354726bb0 0x55835472c780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:01:17.339 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.353+0000 7fb9ab400640 1 -- 172.21.15.107:0/3508630778 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55835439a210 con 0x55835472f980 2024-03-08T21:01:17.339 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.354+0000 7fb9ab400640 1 --2- 172.21.15.107:0/3508630778 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55835472f980 0x55835472cec0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fb9a402c630 tx=0x7fb9a402c0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b9a412e89f252f7 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:01:17.339 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.354+0000 7fb98be00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-03-08T21:01:17.339 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.354+0000 7fb98be00640 1 -- 172.21.15.107:0/3508630778 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9a404c070 con 0x55835472f980 2024-03-08T21:01:17.339 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.354+0000 7fb9b4413740 1 -- 172.21.15.107:0/3508630778 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb9a4032bb0 con 0x55835472f980 2024-03-08T21:01:17.339 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.354+0000 7fb98be00640 1 -- 172.21.15.107:0/3508630778 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fb9a403f470 con 0x55835472f980 2024-03-08T21:01:17.341 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.354+0000 7fb98be00640 1 -- 172.21.15.107:0/3508630778 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb9a4058050 con 0x55835472f980 2024-03-08T21:01:17.341 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.354+0000 7fb9b4413740 10 client.15525 fetch_fsmap learned FSMap version 28 2024-03-08T21:01:17.341 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.354+0000 7fb9b4413740 1 -- 172.21.15.107:0/3508630778 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x558354466620 con 0x55835472f980 2024-03-08T21:01:17.342 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.355+0000 7fb98be00640 1 -- 172.21.15.107:0/3508630778 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fb9a4032bb0 con 0x55835472f980 2024-03-08T21:01:17.342 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.355+0000 7fb98be00640 1 -- 172.21.15.107:0/3508630778 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 28) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fb9a40040d0 con 0x55835472f980 2024-03-08T21:01:17.342 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.355+0000 7fb9b4413740 10 client.15525 fetch_fsmap finished waiting for FSMap version 28 2024-03-08T21:01:17.342 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.355+0000 7fb9b4413740 10 client.15525 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-08T21:01:17.342 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.355+0000 7fb9b4413740 20 client.15525 populate_metadata read hostname 'smithi107' 2024-03-08T21:01:17.342 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.355+0000 7fb9b4413740 1 --2- 172.21.15.107:0/3508630778 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55835473f690 0x558354741a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:01:17.342 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.355+0000 7fb9b4413740 4 client.15525 mds_command: new command op to 34475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-08T21:01:17.342 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.355+0000 7fb9b4413740 1 -- 172.21.15.107:0/3508630778 --> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558354692910 con 0x55835473f690 2024-03-08T21:01:17.342 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.356+0000 7fb9aaa00640 1 --2- 172.21.15.107:0/3508630778 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55835473f690 0x558354741a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:01:17.343 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.356+0000 7fb98be00640 10 client.15525 ms_handle_connect on v2:172.21.15.149:6838/55290596 2024-03-08T21:01:17.343 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.356+0000 7fb9aaa00640 1 --2- 172.21.15.107:0/3508630778 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55835473f690 0x558354741a80 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=90688c49be04896a server_cookie=57990d0a23eb84fd in_seq=0 out_seq=0 2024-03-08T21:01:17.343 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.358+0000 7fb98be00640 1 -- 172.21.15.107:0/3508630778 <== mds.0 v2:172.21.15.149:6838/55290596 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x558354692910 con 0x55835473f690 2024-03-08T21:01:17.343 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.358+0000 7fb98be00640 10 client.15525 handle_command_reply: tid=0 2024-03-08T21:01:17.343 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.358+0000 7fb9b4413740 1 client.15525 shutdown 2024-03-08T21:01:17.343 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.358+0000 7fb9b4413740 1 -- 172.21.15.107:0/3508630778 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55835472f980 msgr2=0x55835472cec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:01:17.343 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.358+0000 7fb9b4413740 1 --2- 172.21.15.107:0/3508630778 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55835472f980 0x55835472cec0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fb9a402c630 tx=0x7fb9a402c0f0 comp rx=0 tx=0).stop 2024-03-08T21:01:17.343 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.358+0000 7fb9b4413740 1 -- 172.21.15.107:0/3508630778 shutdown_connections 2024-03-08T21:01:17.343 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.358+0000 7fb9b4413740 1 --2- 172.21.15.107:0/3508630778 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55835472f980 0x55835472cec0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:01:17.343 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.358+0000 7fb9b4413740 1 --2- 172.21.15.107:0/3508630778 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55835473f690 0x558354741a80 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:01:17.343 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.358+0000 7fb9b4413740 1 --2- 172.21.15.107:0/3508630778 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558354726bb0 0x55835472c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:01:17.343 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.358+0000 7fb9b4413740 1 --2- 172.21.15.107:0/3508630778 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558354726290 0x55835472f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:01:17.343 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.359+0000 7fb9b4413740 1 -- 172.21.15.107:0/3508630778 >> 172.21.15.107:0/3508630778 conn(0x558354720bb0 msgr2=0x558354722900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:01:17.343 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.359+0000 7fb98be00640 0 client.15525 ms_handle_reset on v2:172.21.15.149:6838/55290596 2024-03-08T21:01:17.343 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.359+0000 7fb9b4413740 1 -- 172.21.15.107:0/3508630778 shutdown_connections 2024-03-08T21:01:17.344 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.359+0000 7fb9b4413740 1 -- 172.21.15.107:0/3508630778 wait complete. 2024-03-08T21:01:17.344 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.359+0000 7fb9b4413740 20 client.15525 trim_cache size 0 max 16384 2024-03-08T21:01:17.356 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.372+0000 7fb9b4413740 1 Processor -- start 2024-03-08T21:01:17.357 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.373+0000 7fb9b4413740 1 -- start start 2024-03-08T21:01:17.357 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.373+0000 7fb9b4413740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558354726290 0x55835472b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:01:17.357 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.373+0000 7fb9b4413740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558354726bb0 0x55835472eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:01:17.357 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.373+0000 7fb9b4413740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55835472f980 0x55835472f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:01:17.357 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.373+0000 7fb9b4413740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9ac1aa820 con 0x55835472f980 2024-03-08T21:01:17.357 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.373+0000 7fb9b4413740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9ac1ac1a0 con 0x558354726bb0 2024-03-08T21:01:17.357 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.373+0000 7fb9b4413740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9ac1aae80 con 0x558354726290 2024-03-08T21:01:17.357 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.373+0000 7fb9ab400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55835472f980 0x55835472f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:01:17.357 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.373+0000 7fb9ab400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55835472f980 0x55835472f3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:39476/0 (socket says 172.21.15.107:39476) 2024-03-08T21:01:17.357 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.373+0000 7fb9ab400640 1 -- 172.21.15.107:0/2544473597 learned_addr learned my addr 172.21.15.107:0/2544473597 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:01:17.357 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.373+0000 7fb9aa000640 1 --2- 172.21.15.107:0/2544473597 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558354726bb0 0x55835472eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:01:17.357 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.373+0000 7fb9aaa00640 1 --2- 172.21.15.107:0/2544473597 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558354726290 0x55835472b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:01:17.357 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.374+0000 7fb9ab400640 1 -- 172.21.15.107:0/2544473597 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558354726290 msgr2=0x55835472b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:01:17.358 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.374+0000 7fb9ab400640 1 --2- 172.21.15.107:0/2544473597 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558354726290 0x55835472b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:01:17.358 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.374+0000 7fb9ab400640 1 -- 172.21.15.107:0/2544473597 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558354726bb0 msgr2=0x55835472eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:01:17.358 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.374+0000 7fb9ab400640 1 --2- 172.21.15.107:0/2544473597 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558354726bb0 0x55835472eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:01:17.358 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.374+0000 7fb9ab400640 1 -- 172.21.15.107:0/2544473597 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558354746900 con 0x55835472f980 2024-03-08T21:01:17.358 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.374+0000 7fb9aaa00640 1 --2- 172.21.15.107:0/2544473597 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558354726290 0x55835472b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T21:01:17.358 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.374+0000 7fb9ab400640 1 --2- 172.21.15.107:0/2544473597 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55835472f980 0x55835472f3f0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fb9a4056040 tx=0x7fb9a4004810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c619a0f860cacb60 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:01:17.358 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.374+0000 7fb996a00640 1 -- 172.21.15.107:0/2544473597 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9a404c040 con 0x55835472f980 2024-03-08T21:01:17.358 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.374+0000 7fb996a00640 1 -- 172.21.15.107:0/2544473597 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fb9a40041f0 con 0x55835472f980 2024-03-08T21:01:17.358 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.374+0000 7fb996a00640 1 -- 172.21.15.107:0/2544473597 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9a4043030 con 0x55835472f980 2024-03-08T21:01:17.361 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.375+0000 7fb9b4413740 1 -- 172.21.15.107:0/2544473597 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55835472f980 msgr2=0x55835472f3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:01:17.361 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.375+0000 7fb9b4413740 1 --2- 172.21.15.107:0/2544473597 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55835472f980 0x55835472f3f0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fb9a4056040 tx=0x7fb9a4004810 comp rx=0 tx=0).stop 2024-03-08T21:01:17.361 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.375+0000 7fb9b4413740 1 -- 172.21.15.107:0/2544473597 shutdown_connections 2024-03-08T21:01:17.361 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.375+0000 7fb9b4413740 1 --2- 172.21.15.107:0/2544473597 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55835472f980 0x55835472f3f0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:01:17.361 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.375+0000 7fb9b4413740 1 --2- 172.21.15.107:0/2544473597 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558354726bb0 0x55835472eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:01:17.361 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.375+0000 7fb9b4413740 1 --2- 172.21.15.107:0/2544473597 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x558354726290 0x55835472b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:01:17.361 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.375+0000 7fb9b4413740 1 -- 172.21.15.107:0/2544473597 >> 172.21.15.107:0/2544473597 conn(0x558354720bb0 msgr2=0x55835472af20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:01:17.362 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.375+0000 7fb9b4413740 1 -- 172.21.15.107:0/2544473597 shutdown_connections 2024-03-08T21:01:17.362 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.375+0000 7fb9b4413740 1 -- 172.21.15.107:0/2544473597 wait complete. 2024-03-08T21:01:17.362 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.375+0000 7fb9b4413740 1 -- start start 2024-03-08T21:01:17.362 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.376+0000 7fb9b4413740 1 Processor -- start 2024-03-08T21:01:17.362 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.376+0000 7fb9b4413740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558354726290 0x558354736890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:01:17.362 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.376+0000 7fb9b4413740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558354726bb0 0x558354736dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:01:17.362 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.376+0000 7fb9b4413740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55835472f980 0x558354734e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:01:17.362 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.376+0000 7fb9b4413740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9ac1abb40 con 0x558354726bb0 2024-03-08T21:01:17.362 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.376+0000 7fb9b4413740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9ac1ab4e0 con 0x558354726290 2024-03-08T21:01:17.362 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.376+0000 7fb9b4413740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9ac1a8ae0 con 0x55835472f980 2024-03-08T21:01:17.362 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.376+0000 7fb9b4413740 10 client.0 _ll_register_callbacks cb 0x7fb9a4037070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T21:01:17.362 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.376+0000 7fb9aa000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558354726bb0 0x558354736dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:01:17.362 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.376+0000 7fb9aa000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558354726bb0 0x558354736dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:39490/0 (socket says 172.21.15.107:39490) 2024-03-08T21:01:17.363 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.376+0000 7fb9aa000640 1 -- 172.21.15.107:0/1597427827 learned_addr learned my addr 172.21.15.107:0/1597427827 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:01:17.363 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.377+0000 7fb9aaa00640 1 --2- 172.21.15.107:0/1597427827 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558354726290 0x558354736890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:01:17.363 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.377+0000 7fb9aa000640 1 -- 172.21.15.107:0/1597427827 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55835472f980 msgr2=0x558354734e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:01:17.363 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.377+0000 7fb9aa000640 1 --2- 172.21.15.107:0/1597427827 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55835472f980 0x558354734e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:01:17.363 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.377+0000 7fb9aa000640 1 -- 172.21.15.107:0/1597427827 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558354726290 msgr2=0x558354736890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:01:17.363 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.377+0000 7fb9aa000640 1 --2- 172.21.15.107:0/1597427827 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558354726290 0x558354736890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:01:17.363 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.377+0000 7fb9aa000640 1 -- 172.21.15.107:0/1597427827 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558354735630 con 0x558354726bb0 2024-03-08T21:01:17.363 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.377+0000 7fb9aa000640 1 --2- 172.21.15.107:0/1597427827 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558354726bb0 0x558354736dd0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fb99809aed0 tx=0x7fb998009e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99c3d8612a58d9f4 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:01:17.363 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.377+0000 7fb98be00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-03-08T21:01:17.363 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.377+0000 7fb98be00640 1 -- 172.21.15.107:0/1597427827 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9980cc5e0 con 0x558354726bb0 2024-03-08T21:01:17.363 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.377+0000 7fb98be00640 1 -- 172.21.15.107:0/1597427827 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fb99809a390 con 0x558354726bb0 2024-03-08T21:01:17.363 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.377+0000 7fb9b4413740 1 -- 172.21.15.107:0/1597427827 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb9a40041f0 con 0x558354726bb0 2024-03-08T21:01:17.363 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.377+0000 7fb98be00640 1 -- 172.21.15.107:0/1597427827 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb99809f090 con 0x558354726bb0 2024-03-08T21:01:17.363 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.378+0000 7fb9b4413740 10 client.15531 fetch_fsmap learned FSMap version 28 2024-03-08T21:01:17.364 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.378+0000 7fb9b4413740 1 -- 172.21.15.107:0/1597427827 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5583547963c0 con 0x558354726bb0 2024-03-08T21:01:17.364 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.378+0000 7fb98be00640 1 -- 172.21.15.107:0/1597427827 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fb9a40041f0 con 0x558354726bb0 2024-03-08T21:01:17.364 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.378+0000 7fb98be00640 1 -- 172.21.15.107:0/1597427827 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 28) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fb998095ab0 con 0x558354726bb0 2024-03-08T21:01:17.364 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.378+0000 7fb9b4413740 10 client.15531 fetch_fsmap finished waiting for FSMap version 28 2024-03-08T21:01:17.364 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.378+0000 7fb9b4413740 10 client.15531 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-08T21:01:17.364 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.378+0000 7fb9b4413740 20 client.15531 populate_metadata read hostname 'smithi107' 2024-03-08T21:01:17.364 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.378+0000 7fb9b4413740 1 --2- 172.21.15.107:0/1597427827 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x558354796b40 0x558354798f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:01:17.364 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.378+0000 7fb9b4413740 4 client.15531 mds_command: new command op to 34475 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-08T21:01:17.364 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.378+0000 7fb9b4413740 1 -- 172.21.15.107:0/1597427827 --> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55835468cc60 con 0x558354796b40 2024-03-08T21:01:17.364 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.379+0000 7fb9aaa00640 1 --2- 172.21.15.107:0/1597427827 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x558354796b40 0x558354798f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:01:17.365 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.379+0000 7fb9aaa00640 1 --2- 172.21.15.107:0/1597427827 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x558354796b40 0x558354798f60 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7f0b4af600a157ab server_cookie=9ba0727816f72b83 in_seq=0 out_seq=0 2024-03-08T21:01:17.365 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.379+0000 7fb98be00640 10 client.15531 ms_handle_connect on v2:172.21.15.149:6838/55290596 2024-03-08T21:01:17.373 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.390+0000 7fb98be00640 1 -- 172.21.15.107:0/1597427827 <== mds.0 v2:172.21.15.149:6838/55290596 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55835468cc60 con 0x558354796b40 2024-03-08T21:01:17.373 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.390+0000 7fb98be00640 10 client.15531 handle_command_reply: tid=0 2024-03-08T21:01:17.373 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.390+0000 7fb9b4413740 1 client.15531 shutdown 2024-03-08T21:01:17.373 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.390+0000 7fb9b4413740 1 -- 172.21.15.107:0/1597427827 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558354726bb0 msgr2=0x558354736dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:01:17.373 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.390+0000 7fb9b4413740 1 --2- 172.21.15.107:0/1597427827 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558354726bb0 0x558354736dd0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fb99809aed0 tx=0x7fb998009e00 comp rx=0 tx=0).stop 2024-03-08T21:01:17.373 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.390+0000 7fb9b4413740 1 -- 172.21.15.107:0/1597427827 shutdown_connections 2024-03-08T21:01:17.373 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.390+0000 7fb9b4413740 1 --2- 172.21.15.107:0/1597427827 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55835472f980 0x558354734e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:01:17.373 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.390+0000 7fb9b4413740 1 --2- 172.21.15.107:0/1597427827 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558354726bb0 0x558354736dd0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:01:17.373 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.390+0000 7fb9b4413740 1 --2- 172.21.15.107:0/1597427827 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x558354796b40 0x558354798f60 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:01:17.373 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.390+0000 7fb9b4413740 1 --2- 172.21.15.107:0/1597427827 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558354726290 0x558354736890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:01:17.374 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.390+0000 7fb9b4413740 1 -- 172.21.15.107:0/1597427827 >> 172.21.15.107:0/1597427827 conn(0x558354720bb0 msgr2=0x55835472b7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:01:17.374 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.390+0000 7fb98be00640 0 client.15531 ms_handle_reset on v2:172.21.15.149:6838/55290596 2024-03-08T21:01:17.374 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.390+0000 7fb9b4413740 1 -- 172.21.15.107:0/1597427827 shutdown_connections 2024-03-08T21:01:17.374 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.390+0000 7fb9b4413740 1 -- 172.21.15.107:0/1597427827 wait complete. 2024-03-08T21:01:17.374 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.391+0000 7fb9b4413740 20 client.15531 trim_cache size 0 max 16384 2024-03-08T21:01:17.374 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:{ 2024-03-08T21:01:17.374 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "message": "", 2024-03-08T21:01:17.374 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "return_code": 0 2024-03-08T21:01:17.374 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:} 2024-03-08T21:01:17.375 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.391+0000 7fb9abe00640 1 -- 172.21.15.107:0/892881 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7fb9900a2a80 msgr2=0x7fb9900a4f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:01:17.375 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.391+0000 7fb9abe00640 1 --2- 172.21.15.107:0/892881 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7fb9900a2a80 0x7fb9900a4f40 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x558354725d40 tx=0x7fb99c005f50 comp rx=0 tx=0).stop 2024-03-08T21:01:17.375 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.391+0000 7fb9abe00640 1 -- 172.21.15.107:0/892881 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb9ac106410 msgr2=0x7fb9ac19db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:01:17.375 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.391+0000 7fb9abe00640 1 --2- 172.21.15.107:0/892881 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb9ac106410 0x7fb9ac19db60 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fb99800d740 tx=0x7fb99800dc00 comp rx=0 tx=0).stop 2024-03-08T21:01:17.375 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.391+0000 7fb9abe00640 1 -- 172.21.15.107:0/892881 shutdown_connections 2024-03-08T21:01:17.375 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.391+0000 7fb9abe00640 1 --2- 172.21.15.107:0/892881 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7fb9900a2a80 0x7fb9900a4f40 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:01:17.375 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.391+0000 7fb9abe00640 1 --2- 172.21.15.107:0/892881 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb9ac1976e0 0x7fb9ac197b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:01:17.376 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.391+0000 7fb9abe00640 1 --2- 172.21.15.107:0/892881 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb9ac106410 0x7fb9ac19db60 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:01:17.376 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.391+0000 7fb9abe00640 1 --2- 172.21.15.107:0/892881 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb9ac101340 0x7fb9ac19d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:01:17.376 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.391+0000 7fb9abe00640 1 -- 172.21.15.107:0/892881 >> 172.21.15.107:0/892881 conn(0x7fb9ac0fcc40 msgr2=0x7fb9ac0fef80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:01:17.376 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.391+0000 7fb9abe00640 1 -- 172.21.15.107:0/892881 shutdown_connections 2024-03-08T21:01:17.376 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:17.392+0000 7fb9abe00640 1 -- 172.21.15.107:0/892881 wait complete. 2024-03-08T21:01:17.378 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:01:17 smithi204 podman[69831]: 2024-03-08 21:01:17.133334796 +0000 UTC m=+0.468236594 container init 1633bce08122b382f994e613bbd2c64c80c2d7ffbef441b57ad2f7ac547c941c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=rishabh-2024mar8-7d63482, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-08T21:01:17.378 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:01:17 smithi204 podman[69831]: 2024-03-08 21:01:17.144265732 +0000 UTC m=+0.479167460 container start 1633bce08122b382f994e613bbd2c64c80c2d7ffbef441b57ad2f7ac547c941c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5, RELEASE=rishabh-2024mar8-7d63482, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-03-08T21:01:17.378 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:01:17 smithi204 ceph-mds[69849]: -- 172.21.15.204:0/2924497331 <== mon.1 v2:172.21.15.204:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55fe955c9a40 con 0x55fe9559cc00 2024-03-08T21:01:17.378 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:01:17 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[69845]: starting mds.c at 2024-03-08T21:01:17.378 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:01:17 smithi204 bash[69831]: 1633bce08122b382f994e613bbd2c64c80c2d7ffbef441b57ad2f7ac547c941c 2024-03-08T21:01:17.378 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:01:17 smithi204 systemd[1]: Started Ceph mds.c for dc20f0f8-dd8c-11ee-95c6-87774f69a715. 2024-03-08T21:01:17.878 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:01:17 smithi204 ceph-mon[34935]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 341 B/s wr, 7 op/s 2024-03-08T21:01:17.878 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:01:17 smithi204 ceph-mon[34935]: daemon mds.c restarted 2024-03-08T21:01:17.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:01:17 smithi107 ceph-mon[31381]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 341 B/s wr, 7 op/s 2024-03-08T21:01:17.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:01:17 smithi107 ceph-mon[31381]: daemon mds.c restarted 2024-03-08T21:01:18.014 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:01:17 smithi149 ceph-mon[35042]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 341 B/s wr, 7 op/s 2024-03-08T21:01:18.014 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:01:17 smithi149 ceph-mon[35042]: daemon mds.c restarted 2024-03-08T21:01:18.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:01:18 smithi107 ceph-mon[31381]: osdmap e90: 12 total, 12 up, 12 in 2024-03-08T21:01:18.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:01:18 smithi107 ceph-mon[31381]: mds.? [v2:172.21.15.204:6836/1631083588,v1:172.21.15.204:6837/1631083588] up:boot 2024-03-08T21:01:18.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:01:18 smithi107 ceph-mon[31381]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-03-08T21:01:18.967 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:01:18 smithi107 ceph-mon[31381]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-08T21:01:18.967 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:01:18 smithi107 ceph-mon[31381]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-08T21:01:18.967 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:01:18 smithi107 ceph-mon[31381]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 409 B/s wr, 8 op/s 2024-03-08T21:01:19.014 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:01:18 smithi149 ceph-mon[35042]: osdmap e90: 12 total, 12 up, 12 in 2024-03-08T21:01:19.014 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:01:18 smithi149 ceph-mon[35042]: mds.? [v2:172.21.15.204:6836/1631083588,v1:172.21.15.204:6837/1631083588] up:boot 2024-03-08T21:01:19.014 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:01:18 smithi149 ceph-mon[35042]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-03-08T21:01:19.014 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:01:18 smithi149 ceph-mon[35042]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-08T21:01:19.014 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:01:18 smithi149 ceph-mon[35042]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-08T21:01:19.015 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:01:18 smithi149 ceph-mon[35042]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 409 B/s wr, 8 op/s 2024-03-08T21:01:19.127 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:01:18 smithi204 ceph-mon[34935]: osdmap e90: 12 total, 12 up, 12 in 2024-03-08T21:01:19.127 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:01:18 smithi204 ceph-mon[34935]: mds.? [v2:172.21.15.204:6836/1631083588,v1:172.21.15.204:6837/1631083588] up:boot 2024-03-08T21:01:19.128 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:01:18 smithi204 ceph-mon[34935]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-03-08T21:01:19.128 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:01:18 smithi204 ceph-mon[34935]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-08T21:01:19.128 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:01:18 smithi204 ceph-mon[34935]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-08T21:01:19.128 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:01:18 smithi204 ceph-mon[34935]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 409 B/s wr, 8 op/s 2024-03-08T21:01:21.877 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:01:21 smithi204 ceph-mon[34935]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 307 B/s wr, 9 op/s 2024-03-08T21:01:21.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:01:21 smithi107 ceph-mon[31381]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 307 B/s wr, 9 op/s 2024-03-08T21:01:22.014 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:01:21 smithi149 ceph-mon[35042]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 307 B/s wr, 9 op/s 2024-03-08T21:01:23.855 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:01:23 smithi149 ceph-mon[35042]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 307 B/s wr, 9 op/s 2024-03-08T21:01:23.877 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:01:23 smithi204 ceph-mon[34935]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 307 B/s wr, 9 op/s 2024-03-08T21:01:23.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:01:23 smithi107 ceph-mon[31381]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 307 B/s wr, 9 op/s 2024-03-08T21:01:25.877 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:01:25 smithi204 ceph-mon[34935]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 307 B/s wr, 10 op/s 2024-03-08T21:01:25.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:01:25 smithi107 ceph-mon[31381]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 307 B/s wr, 10 op/s 2024-03-08T21:01:26.014 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:01:25 smithi149 ceph-mon[35042]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 307 B/s wr, 10 op/s 2024-03-08T21:01:27.877 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:01:27 smithi204 ceph-mon[34935]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 307 B/s wr, 10 op/s 2024-03-08T21:01:27.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:01:27 smithi107 ceph-mon[31381]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 307 B/s wr, 10 op/s 2024-03-08T21:01:28.014 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:01:27 smithi149 ceph-mon[35042]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 307 B/s wr, 10 op/s 2024-03-08T21:01:29.877 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:01:29 smithi204 ceph-mon[34935]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 282 B/s wr, 9 op/s 2024-03-08T21:01:29.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:01:29 smithi107 ceph-mon[31381]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 282 B/s wr, 9 op/s 2024-03-08T21:01:30.014 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:01:29 smithi149 ceph-mon[35042]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 282 B/s wr, 9 op/s 2024-03-08T21:01:30.511 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.529+0000 7f0e2fe00640 1 -- 172.21.15.107:0/2985389457 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0e30102bc0 msgr2=0x7f0e3010b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:01:30.511 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.529+0000 7f0e2fe00640 1 --2- 172.21.15.107:0/2985389457 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0e30102bc0 0x7f0e3010b0a0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f0e1c009900 tx=0x7f0e1c02bcd0 comp rx=0 tx=0).stop 2024-03-08T21:01:30.511 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.529+0000 7f0e2fe00640 1 -- 172.21.15.107:0/2985389457 shutdown_connections 2024-03-08T21:01:30.511 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.529+0000 7f0e2fe00640 1 --2- 172.21.15.107:0/2985389457 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0e3010b6c0 0x7f0e3010db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:01:30.511 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.529+0000 7f0e2fe00640 1 --2- 172.21.15.107:0/2985389457 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0e30102bc0 0x7f0e3010b0a0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:01:30.511 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.529+0000 7f0e2fe00640 1 --2- 172.21.15.107:0/2985389457 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0e301022a0 0x7f0e30102680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:01:30.511 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.529+0000 7f0e2fe00640 1 -- 172.21.15.107:0/2985389457 >> 172.21.15.107:0/2985389457 conn(0x7f0e300fbb00 msgr2=0x7f0e300fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:01:30.511 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.529+0000 7f0e2fe00640 1 -- 172.21.15.107:0/2985389457 shutdown_connections 2024-03-08T21:01:30.511 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.529+0000 7f0e2fe00640 1 -- 172.21.15.107:0/2985389457 wait complete. 2024-03-08T21:01:30.513 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.530+0000 7f0e2fe00640 1 Processor -- start 2024-03-08T21:01:30.513 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.530+0000 7f0e2fe00640 1 -- start start 2024-03-08T21:01:30.513 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.530+0000 7f0e2fe00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0e301022a0 0x7f0e3019f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:01:30.513 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.530+0000 7f0e2fe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0e30102bc0 0x7f0e3019fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:01:30.513 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.530+0000 7f0e2fe00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0e3010b6c0 0x7f0e30199960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:01:30.513 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.530+0000 7f0e2fe00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e301113a0 con 0x7f0e30102bc0 2024-03-08T21:01:30.513 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.530+0000 7f0e2fe00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e30111220 con 0x7f0e3010b6c0 2024-03-08T21:01:30.514 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.531+0000 7f0e2fe00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e30111520 con 0x7f0e301022a0 2024-03-08T21:01:30.514 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.531+0000 7f0e2e000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0e30102bc0 0x7f0e3019fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:01:30.514 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.531+0000 7f0e2e000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0e30102bc0 0x7f0e3019fca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:38258/0 (socket says 172.21.15.107:38258) 2024-03-08T21:01:30.514 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.531+0000 7f0e2e000640 1 -- 172.21.15.107:0/2577110218 learned_addr learned my addr 172.21.15.107:0/2577110218 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:01:30.514 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.531+0000 7f0e2ea00640 1 --2- 172.21.15.107:0/2577110218 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0e301022a0 0x7f0e3019f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:01:30.514 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.531+0000 7f0e2f400640 1 --2- 172.21.15.107:0/2577110218 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0e3010b6c0 0x7f0e30199960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:01:30.514 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.531+0000 7f0e2e000640 1 -- 172.21.15.107:0/2577110218 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0e301022a0 msgr2=0x7f0e3019f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:01:30.514 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.531+0000 7f0e2e000640 1 --2- 172.21.15.107:0/2577110218 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0e301022a0 0x7f0e3019f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:01:30.514 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.531+0000 7f0e2e000640 1 -- 172.21.15.107:0/2577110218 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0e3010b6c0 msgr2=0x7f0e30199960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:01:30.514 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.531+0000 7f0e2e000640 1 --2- 172.21.15.107:0/2577110218 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0e3010b6c0 0x7f0e30199960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:01:30.514 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.531+0000 7f0e2e000640 1 -- 172.21.15.107:0/2577110218 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e3019a0e0 con 0x7f0e30102bc0 2024-03-08T21:01:30.514 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.532+0000 7f0e2e000640 1 --2- 172.21.15.107:0/2577110218 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0e30102bc0 0x7f0e3019fca0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f0e1c000c00 tx=0x7f0e1c02c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:01:30.515 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.532+0000 7f0e27400640 1 -- 172.21.15.107:0/2577110218 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e1c043020 con 0x7f0e30102bc0 2024-03-08T21:01:30.515 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.532+0000 7f0e27400640 1 -- 172.21.15.107:0/2577110218 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f0e1c02dc60 con 0x7f0e30102bc0 2024-03-08T21:01:30.515 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.532+0000 7f0e2fe00640 1 -- 172.21.15.107:0/2577110218 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e3019a370 con 0x7f0e30102bc0 2024-03-08T21:01:30.515 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.532+0000 7f0e27400640 1 -- 172.21.15.107:0/2577110218 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e1c0039e0 con 0x7f0e30102bc0 2024-03-08T21:01:30.515 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.532+0000 7f0e2fe00640 1 -- 172.21.15.107:0/2577110218 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e301a6620 con 0x7f0e30102bc0 2024-03-08T21:01:30.519 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.537+0000 7f0e27400640 1 -- 172.21.15.107:0/2577110218 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 24) v1 ==== 137524+0+0 (secure 0 0 0) 0x7f0e1c046020 con 0x7f0e30102bc0 2024-03-08T21:01:30.519 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.537+0000 7f0e387de740 1 Processor -- start 2024-03-08T21:01:30.519 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.538+0000 7f0e387de740 1 -- start start 2024-03-08T21:01:30.520 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.538+0000 7f0e387de740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c9a4317e80 0x55c9a4318260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:01:30.520 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.538+0000 7f0e387de740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c9a43187a0 0x55c9a4321030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:01:30.520 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.538+0000 7f0e387de740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c9a4321570 0x55c9a4323960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:01:30.520 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.538+0000 7f0e387de740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9a427b3d0 con 0x55c9a4321570 2024-03-08T21:01:30.520 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.538+0000 7f0e387de740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9a42246b0 con 0x55c9a4317e80 2024-03-08T21:01:30.520 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.538+0000 7f0e387de740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9a4277470 con 0x55c9a43187a0 2024-03-08T21:01:30.520 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.539+0000 7f0e2f400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c9a4317e80 0x55c9a4318260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:01:30.521 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.539+0000 7f0e2ea00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c9a43187a0 0x55c9a4321030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:01:30.521 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.539+0000 7f0e2e000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c9a4321570 0x55c9a4323960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:01:30.521 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.539+0000 7f0e2f400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c9a4317e80 0x55c9a4318260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.107:47436/0 (socket says 172.21.15.107:47436) 2024-03-08T21:01:30.521 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.539+0000 7f0e2f400640 1 -- 172.21.15.107:0/2244772756 learned_addr learned my addr 172.21.15.107:0/2244772756 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:01:30.522 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.539+0000 7f0e2f400640 1 -- 172.21.15.107:0/2244772756 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c9a43187a0 msgr2=0x55c9a4321030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:01:30.522 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.539+0000 7f0e2f400640 1 --2- 172.21.15.107:0/2244772756 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c9a43187a0 0x55c9a4321030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:01:30.522 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.539+0000 7f0e2f400640 1 -- 172.21.15.107:0/2244772756 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c9a4321570 msgr2=0x55c9a4323960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:01:30.522 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.539+0000 7f0e2f400640 1 --2- 172.21.15.107:0/2244772756 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c9a4321570 0x55c9a4323960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:01:30.522 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.539+0000 7f0e2f400640 1 -- 172.21.15.107:0/2244772756 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c9a424ab50 con 0x55c9a4317e80 2024-03-08T21:01:30.523 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.539+0000 7f0e2e000640 1 --2- 172.21.15.107:0/2244772756 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c9a4321570 0x55c9a4323960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T21:01:30.523 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.540+0000 7f0e27400640 1 --2- 172.21.15.107:0/2577110218 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f0e140a2a80 0x7f0e140a4f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:01:30.523 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.540+0000 7f0e2f400640 1 --2- 172.21.15.107:0/2244772756 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c9a4317e80 0x55c9a4318260 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f0e28009e60 tx=0x7f0e28010740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=29157e31dfcd4e7 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:01:30.523 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.540+0000 7f0e27400640 1 -- 172.21.15.107:0/2577110218 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f0e1c0f1160 con 0x7f0e30102bc0 2024-03-08T21:01:30.524 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.540+0000 7f0e2ea00640 1 --2- 172.21.15.107:0/2577110218 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f0e140a2a80 0x7f0e140a4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:01:30.524 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.540+0000 7f0e1be00640 1 -- 172.21.15.107:0/2244772756 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e28016070 con 0x55c9a4317e80 2024-03-08T21:01:30.524 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.540+0000 7f0e1be00640 1 -- 172.21.15.107:0/2244772756 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f0e28004d20 con 0x55c9a4317e80 2024-03-08T21:01:30.524 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.541+0000 7f0e1be00640 1 -- 172.21.15.107:0/2244772756 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e28012700 con 0x55c9a4317e80 2024-03-08T21:01:30.526 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.541+0000 7f0e387de740 1 -- 172.21.15.107:0/2244772756 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c9a4317e80 msgr2=0x55c9a4318260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:01:30.527 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.541+0000 7f0e387de740 1 --2- 172.21.15.107:0/2244772756 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c9a4317e80 0x55c9a4318260 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f0e28009e60 tx=0x7f0e28010740 comp rx=0 tx=0).stop 2024-03-08T21:01:30.527 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.541+0000 7f0e2ea00640 1 --2- 172.21.15.107:0/2577110218 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f0e140a2a80 0x7f0e140a4f40 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x55c9a4317930 tx=0x7f0e2000b500 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:01:30.527 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.541+0000 7f0e387de740 1 -- 172.21.15.107:0/2244772756 shutdown_connections 2024-03-08T21:01:30.527 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.541+0000 7f0e387de740 1 --2- 172.21.15.107:0/2244772756 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c9a4321570 0x55c9a4323960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:01:30.527 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.541+0000 7f0e387de740 1 --2- 172.21.15.107:0/2244772756 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c9a43187a0 0x55c9a4321030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:01:30.527 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.541+0000 7f0e387de740 1 --2- 172.21.15.107:0/2244772756 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c9a4317e80 0x55c9a4318260 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:01:30.527 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.541+0000 7f0e387de740 1 -- 172.21.15.107:0/2244772756 >> 172.21.15.107:0/2244772756 conn(0x55c9a43127a0 msgr2=0x55c9a4312b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:01:30.527 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.541+0000 7f0e387de740 1 -- 172.21.15.107:0/2244772756 shutdown_connections 2024-03-08T21:01:30.527 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.542+0000 7f0e387de740 1 -- 172.21.15.107:0/2244772756 wait complete. 2024-03-08T21:01:30.527 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.542+0000 7f0e387de740 1 -- start start 2024-03-08T21:01:30.527 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.542+0000 7f0e387de740 1 Processor -- start 2024-03-08T21:01:30.527 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.543+0000 7f0e387de740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c9a4317e80 0x55c9a4320fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:01:30.527 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.543+0000 7f0e387de740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c9a43187a0 0x55c9a431e340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:01:30.527 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.543+0000 7f0e387de740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c9a4321570 0x55c9a431ea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:01:30.528 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.543+0000 7f0e387de740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9a43241a0 con 0x55c9a43187a0 2024-03-08T21:01:30.528 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.543+0000 7f0e387de740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9a42379f0 con 0x55c9a4321570 2024-03-08T21:01:30.528 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.543+0000 7f0e387de740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9a4324320 con 0x55c9a4317e80 2024-03-08T21:01:30.528 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.543+0000 7f0e387de740 10 client.0 _ll_register_callbacks cb 0x55c9a3f8fa00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T21:01:30.528 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.543+0000 7f0e2ea00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c9a4317e80 0x55c9a4320fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:01:30.528 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.543+0000 7f0e2ea00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c9a4317e80 0x55c9a4320fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.107:58988/0 (socket says 172.21.15.107:58988) 2024-03-08T21:01:30.528 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.543+0000 7f0e2ea00640 1 -- 172.21.15.107:0/612455342 learned_addr learned my addr 172.21.15.107:0/612455342 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:01:30.528 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.543+0000 7f0e2e000640 1 --2- 172.21.15.107:0/612455342 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c9a43187a0 0x55c9a431e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:01:30.528 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.543+0000 7f0e2f400640 1 --2- 172.21.15.107:0/612455342 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c9a4321570 0x55c9a431ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:01:30.528 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.544+0000 7f0e2e000640 1 -- 172.21.15.107:0/612455342 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c9a4317e80 msgr2=0x55c9a4320fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:01:30.528 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.544+0000 7f0e2e000640 1 --2- 172.21.15.107:0/612455342 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c9a4317e80 0x55c9a4320fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:01:30.529 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.544+0000 7f0e2e000640 1 -- 172.21.15.107:0/612455342 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c9a4321570 msgr2=0x55c9a431ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:01:30.529 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.544+0000 7f0e2e000640 1 --2- 172.21.15.107:0/612455342 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c9a4321570 0x55c9a431ea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:01:30.529 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.544+0000 7f0e2e000640 1 -- 172.21.15.107:0/612455342 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c9a3f90210 con 0x55c9a43187a0 2024-03-08T21:01:30.529 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.544+0000 7f0e2e000640 1 --2- 172.21.15.107:0/612455342 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c9a43187a0 0x55c9a431e340 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f0e1c0ee830 tx=0x7f0e1c0f4790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efc23e7ab42286fa server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:01:30.529 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.544+0000 7f0e18c00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-03-08T21:01:30.529 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.545+0000 7f0e18c00640 1 -- 172.21.15.107:0/612455342 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e1c0dd040 con 0x55c9a43187a0 2024-03-08T21:01:30.529 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.545+0000 7f0e18c00640 1 -- 172.21.15.107:0/612455342 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f0e1c0c5980 con 0x55c9a43187a0 2024-03-08T21:01:30.529 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.545+0000 7f0e387de740 1 -- 172.21.15.107:0/612455342 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0e28004d20 con 0x55c9a43187a0 2024-03-08T21:01:30.529 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.545+0000 7f0e18c00640 1 -- 172.21.15.107:0/612455342 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0e1c0bd440 con 0x55c9a43187a0 2024-03-08T21:01:30.529 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.545+0000 7f0e387de740 10 client.15543 fetch_fsmap learned FSMap version 30 2024-03-08T21:01:30.529 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.545+0000 7f0e387de740 1 -- 172.21.15.107:0/612455342 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55c9a405c620 con 0x55c9a43187a0 2024-03-08T21:01:30.531 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.546+0000 7f0e18c00640 1 -- 172.21.15.107:0/612455342 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f0e28004d20 con 0x55c9a43187a0 2024-03-08T21:01:30.531 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.546+0000 7f0e18c00640 1 -- 172.21.15.107:0/612455342 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 30) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f0e1c003b80 con 0x55c9a43187a0 2024-03-08T21:01:30.531 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.546+0000 7f0e387de740 10 client.15543 fetch_fsmap finished waiting for FSMap version 30 2024-03-08T21:01:30.532 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.546+0000 7f0e387de740 10 client.15543 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-08T21:01:30.532 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.546+0000 7f0e387de740 20 client.15543 populate_metadata read hostname 'smithi107' 2024-03-08T21:01:30.532 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.546+0000 7f0e387de740 1 --2- 172.21.15.107:0/612455342 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55c9a4331310 0x55c9a4333720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:01:30.532 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.546+0000 7f0e387de740 4 client.15543 mds_command: new command op to 34475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-08T21:01:30.532 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.546+0000 7f0e387de740 1 -- 172.21.15.107:0/612455342 --> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c9a4288910 con 0x55c9a4331310 2024-03-08T21:01:30.532 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.547+0000 7f0e2ea00640 1 --2- 172.21.15.107:0/612455342 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55c9a4331310 0x55c9a4333720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:01:30.532 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.548+0000 7f0e18c00640 10 client.15543 ms_handle_connect on v2:172.21.15.149:6838/55290596 2024-03-08T21:01:30.532 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.548+0000 7f0e2ea00640 1 --2- 172.21.15.107:0/612455342 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55c9a4331310 0x55c9a4333720 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f94db64554c01273 server_cookie=84280280acdacdbf in_seq=0 out_seq=0 2024-03-08T21:01:30.532 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.549+0000 7f0e18c00640 1 -- 172.21.15.107:0/612455342 <== mds.0 v2:172.21.15.149:6838/55290596 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c9a4288910 con 0x55c9a4331310 2024-03-08T21:01:30.532 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.549+0000 7f0e18c00640 10 client.15543 handle_command_reply: tid=0 2024-03-08T21:01:30.532 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.549+0000 7f0e387de740 1 client.15543 shutdown 2024-03-08T21:01:30.533 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.550+0000 7f0e387de740 1 -- 172.21.15.107:0/612455342 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c9a43187a0 msgr2=0x55c9a431e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:01:30.533 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.550+0000 7f0e387de740 1 --2- 172.21.15.107:0/612455342 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c9a43187a0 0x55c9a431e340 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f0e1c0ee830 tx=0x7f0e1c0f4790 comp rx=0 tx=0).stop 2024-03-08T21:01:30.533 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.550+0000 7f0e387de740 1 -- 172.21.15.107:0/612455342 shutdown_connections 2024-03-08T21:01:30.533 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.550+0000 7f0e387de740 1 --2- 172.21.15.107:0/612455342 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55c9a4331310 0x55c9a4333720 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:01:30.533 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.550+0000 7f0e387de740 1 --2- 172.21.15.107:0/612455342 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c9a4321570 0x55c9a431ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:01:30.533 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.550+0000 7f0e387de740 1 --2- 172.21.15.107:0/612455342 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c9a43187a0 0x55c9a431e340 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:01:30.533 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.550+0000 7f0e387de740 1 --2- 172.21.15.107:0/612455342 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c9a4317e80 0x55c9a4320fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:01:30.533 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.550+0000 7f0e18c00640 0 client.15543 ms_handle_reset on v2:172.21.15.149:6838/55290596 2024-03-08T21:01:30.533 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.550+0000 7f0e387de740 1 -- 172.21.15.107:0/612455342 >> 172.21.15.107:0/612455342 conn(0x55c9a43127a0 msgr2=0x55c9a43144f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:01:30.533 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.550+0000 7f0e387de740 1 -- 172.21.15.107:0/612455342 shutdown_connections 2024-03-08T21:01:30.533 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.550+0000 7f0e387de740 1 -- 172.21.15.107:0/612455342 wait complete. 2024-03-08T21:01:30.533 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.550+0000 7f0e387de740 20 client.15543 trim_cache size 0 max 16384 2024-03-08T21:01:30.546 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.565+0000 7f0e387de740 1 Processor -- start 2024-03-08T21:01:30.547 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.565+0000 7f0e387de740 1 -- start start 2024-03-08T21:01:30.547 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.565+0000 7f0e387de740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c9a4317e80 0x55c9a432d010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:01:30.547 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.565+0000 7f0e387de740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c9a43187a0 0x55c9a4328560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:01:30.547 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.565+0000 7f0e387de740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c9a4321570 0x55c9a4328aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:01:30.547 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.565+0000 7f0e387de740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e301aeb40 con 0x55c9a43187a0 2024-03-08T21:01:30.547 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.565+0000 7f0e387de740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e301b04c0 con 0x55c9a4317e80 2024-03-08T21:01:30.547 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.565+0000 7f0e387de740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e301af1a0 con 0x55c9a4321570 2024-03-08T21:01:30.548 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.566+0000 7f0e2e000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c9a43187a0 0x55c9a4328560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:01:30.548 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.566+0000 7f0e2e000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c9a43187a0 0x55c9a4328560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:38288/0 (socket says 172.21.15.107:38288) 2024-03-08T21:01:30.548 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.566+0000 7f0e2e000640 1 -- 172.21.15.107:0/3196201566 learned_addr learned my addr 172.21.15.107:0/3196201566 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:01:30.549 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.566+0000 7f0e2ea00640 1 --2- 172.21.15.107:0/3196201566 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c9a4317e80 0x55c9a432d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:01:30.549 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.566+0000 7f0e2e000640 1 -- 172.21.15.107:0/3196201566 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c9a4321570 msgr2=0x55c9a4328aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:01:30.549 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.566+0000 7f0e2e000640 1 --2- 172.21.15.107:0/3196201566 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c9a4321570 0x55c9a4328aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:01:30.549 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.566+0000 7f0e2e000640 1 -- 172.21.15.107:0/3196201566 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c9a4317e80 msgr2=0x55c9a432d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:01:30.549 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.566+0000 7f0e2e000640 1 --2- 172.21.15.107:0/3196201566 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c9a4317e80 0x55c9a432d010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:01:30.549 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.566+0000 7f0e2e000640 1 -- 172.21.15.107:0/3196201566 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c9a43185d0 con 0x55c9a43187a0 2024-03-08T21:01:30.549 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.566+0000 7f0e2e000640 1 --2- 172.21.15.107:0/3196201566 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c9a43187a0 0x55c9a4328560 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f0e1c005ec0 tx=0x7f0e1c005e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=647928918ac9dddd server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:01:30.549 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.567+0000 7f0e1b400640 1 -- 172.21.15.107:0/3196201566 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e1c0bd440 con 0x55c9a43187a0 2024-03-08T21:01:30.549 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.567+0000 7f0e1b400640 1 -- 172.21.15.107:0/3196201566 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f0e1c0c1800 con 0x55c9a43187a0 2024-03-08T21:01:30.549 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.567+0000 7f0e1b400640 1 -- 172.21.15.107:0/3196201566 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e1c0bcfd0 con 0x55c9a43187a0 2024-03-08T21:01:30.551 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.568+0000 7f0e387de740 1 -- 172.21.15.107:0/3196201566 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c9a43187a0 msgr2=0x55c9a4328560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:01:30.552 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.568+0000 7f0e387de740 1 --2- 172.21.15.107:0/3196201566 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c9a43187a0 0x55c9a4328560 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f0e1c005ec0 tx=0x7f0e1c005e20 comp rx=0 tx=0).stop 2024-03-08T21:01:30.552 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.568+0000 7f0e387de740 1 -- 172.21.15.107:0/3196201566 shutdown_connections 2024-03-08T21:01:30.552 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.568+0000 7f0e387de740 1 --2- 172.21.15.107:0/3196201566 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c9a4321570 0x55c9a4328aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:01:30.552 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.568+0000 7f0e387de740 1 --2- 172.21.15.107:0/3196201566 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c9a43187a0 0x55c9a4328560 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:01:30.552 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.568+0000 7f0e387de740 1 --2- 172.21.15.107:0/3196201566 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c9a4317e80 0x55c9a432d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:01:30.552 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.568+0000 7f0e387de740 1 -- 172.21.15.107:0/3196201566 >> 172.21.15.107:0/3196201566 conn(0x55c9a43127a0 msgr2=0x55c9a4310f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:01:30.552 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.568+0000 7f0e387de740 1 -- 172.21.15.107:0/3196201566 shutdown_connections 2024-03-08T21:01:30.552 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.568+0000 7f0e387de740 1 -- 172.21.15.107:0/3196201566 wait complete. 2024-03-08T21:01:30.552 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.568+0000 7f0e387de740 1 -- start start 2024-03-08T21:01:30.552 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.569+0000 7f0e387de740 1 Processor -- start 2024-03-08T21:01:30.552 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.569+0000 7f0e387de740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c9a4317e80 0x55c9a4327240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:01:30.552 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.569+0000 7f0e387de740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c9a43187a0 0x55c9a4327780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:01:30.552 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.569+0000 7f0e387de740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c9a4321570 0x55c9a4327ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:01:30.552 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.569+0000 7f0e387de740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e301afe60 con 0x55c9a43187a0 2024-03-08T21:01:30.553 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.569+0000 7f0e387de740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e301af800 con 0x55c9a4321570 2024-03-08T21:01:30.553 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.569+0000 7f0e387de740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e301acda0 con 0x55c9a4317e80 2024-03-08T21:01:30.553 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.569+0000 7f0e387de740 10 client.0 _ll_register_callbacks cb 0x7f0e28009600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T21:01:30.553 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.570+0000 7f0e2e000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c9a43187a0 0x55c9a4327780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:01:30.553 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.570+0000 7f0e2e000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c9a43187a0 0x55c9a4327780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:38294/0 (socket says 172.21.15.107:38294) 2024-03-08T21:01:30.553 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.570+0000 7f0e2e000640 1 -- 172.21.15.107:0/70560268 learned_addr learned my addr 172.21.15.107:0/70560268 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:01:30.553 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.570+0000 7f0e2ea00640 1 --2- 172.21.15.107:0/70560268 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c9a4317e80 0x55c9a4327240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:01:30.553 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.570+0000 7f0e2e000640 1 -- 172.21.15.107:0/70560268 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c9a4317e80 msgr2=0x55c9a4327240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:01:30.553 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.570+0000 7f0e2e000640 1 --2- 172.21.15.107:0/70560268 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c9a4317e80 0x55c9a4327240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:01:30.553 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.570+0000 7f0e2e000640 1 -- 172.21.15.107:0/70560268 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c9a4321570 msgr2=0x55c9a4327ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:01:30.553 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.570+0000 7f0e2e000640 1 --2- 172.21.15.107:0/70560268 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c9a4321570 0x55c9a4327ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:01:30.553 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.570+0000 7f0e2e000640 1 -- 172.21.15.107:0/70560268 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c9a4387b20 con 0x55c9a43187a0 2024-03-08T21:01:30.553 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.570+0000 7f0e2ea00640 1 --2- 172.21.15.107:0/70560268 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c9a4317e80 0x55c9a4327240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T21:01:30.553 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.570+0000 7f0e2e000640 1 --2- 172.21.15.107:0/70560268 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c9a43187a0 0x55c9a4327780 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f0e1c02c2b0 tx=0x7f0e1c0f10f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4ad9de271fe47ad server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:01:30.554 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.570+0000 7f0e18c00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-03-08T21:01:30.554 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.573+0000 7f0e18c00640 1 -- 172.21.15.107:0/70560268 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e1c0bd1b0 con 0x55c9a43187a0 2024-03-08T21:01:30.554 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.573+0000 7f0e18c00640 1 -- 172.21.15.107:0/70560268 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f0e1c0c5980 con 0x55c9a43187a0 2024-03-08T21:01:30.555 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.573+0000 7f0e387de740 1 -- 172.21.15.107:0/70560268 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0e1c0c1800 con 0x55c9a43187a0 2024-03-08T21:01:30.555 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.573+0000 7f0e18c00640 1 -- 172.21.15.107:0/70560268 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0e1c0bcfd0 con 0x55c9a43187a0 2024-03-08T21:01:30.555 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.573+0000 7f0e387de740 10 client.15549 fetch_fsmap learned FSMap version 30 2024-03-08T21:01:30.555 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.573+0000 7f0e387de740 1 -- 172.21.15.107:0/70560268 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55c9a4387b20 con 0x55c9a43187a0 2024-03-08T21:01:30.555 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.574+0000 7f0e18c00640 1 -- 172.21.15.107:0/70560268 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f0e1c0c1800 con 0x55c9a43187a0 2024-03-08T21:01:30.555 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.574+0000 7f0e18c00640 1 -- 172.21.15.107:0/70560268 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 30) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f0e1c0b93c0 con 0x55c9a43187a0 2024-03-08T21:01:30.556 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.574+0000 7f0e387de740 10 client.15549 fetch_fsmap finished waiting for FSMap version 30 2024-03-08T21:01:30.556 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.574+0000 7f0e387de740 10 client.15549 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-08T21:01:30.556 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.574+0000 7f0e387de740 20 client.15549 populate_metadata read hostname 'smithi107' 2024-03-08T21:01:30.556 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.574+0000 7f0e387de740 1 --2- 172.21.15.107:0/70560268 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55c9a4389080 0x55c9a438b470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:01:30.556 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.574+0000 7f0e387de740 4 client.15549 mds_command: new command op to 34475 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-08T21:01:30.556 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.574+0000 7f0e387de740 1 -- 172.21.15.107:0/70560268 --> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55c9a4282c60 con 0x55c9a4389080 2024-03-08T21:01:30.556 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.575+0000 7f0e2ea00640 1 --2- 172.21.15.107:0/70560268 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55c9a4389080 0x55c9a438b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:01:30.557 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.576+0000 7f0e2ea00640 1 --2- 172.21.15.107:0/70560268 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55c9a4389080 0x55c9a438b470 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3556f00ce7f8c4c5 server_cookie=97d2776c95f57223 in_seq=0 out_seq=0 2024-03-08T21:01:30.557 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.576+0000 7f0e18c00640 10 client.15549 ms_handle_connect on v2:172.21.15.149:6838/55290596 2024-03-08T21:01:30.567 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.585+0000 7f0e18c00640 1 -- 172.21.15.107:0/70560268 <== mds.0 v2:172.21.15.149:6838/55290596 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55c9a4282c60 con 0x55c9a4389080 2024-03-08T21:01:30.568 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.585+0000 7f0e18c00640 10 client.15549 handle_command_reply: tid=0 2024-03-08T21:01:30.568 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.585+0000 7f0e387de740 1 client.15549 shutdown 2024-03-08T21:01:30.568 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.585+0000 7f0e387de740 1 -- 172.21.15.107:0/70560268 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c9a43187a0 msgr2=0x55c9a4327780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:01:30.568 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.585+0000 7f0e387de740 1 --2- 172.21.15.107:0/70560268 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c9a43187a0 0x55c9a4327780 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f0e1c02c2b0 tx=0x7f0e1c0f10f0 comp rx=0 tx=0).stop 2024-03-08T21:01:30.568 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.585+0000 7f0e387de740 1 -- 172.21.15.107:0/70560268 shutdown_connections 2024-03-08T21:01:30.568 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.585+0000 7f0e387de740 1 --2- 172.21.15.107:0/70560268 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55c9a4389080 0x55c9a438b470 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:01:30.568 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.585+0000 7f0e387de740 1 --2- 172.21.15.107:0/70560268 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c9a4321570 0x55c9a4327ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:01:30.568 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.585+0000 7f0e387de740 1 --2- 172.21.15.107:0/70560268 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c9a43187a0 0x55c9a4327780 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:01:30.568 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.585+0000 7f0e387de740 1 --2- 172.21.15.107:0/70560268 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c9a4317e80 0x55c9a4327240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:01:30.568 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.585+0000 7f0e387de740 1 -- 172.21.15.107:0/70560268 >> 172.21.15.107:0/70560268 conn(0x55c9a43127a0 msgr2=0x55c9a4311800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:01:30.568 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.585+0000 7f0e18c00640 0 client.15549 ms_handle_reset on v2:172.21.15.149:6838/55290596 2024-03-08T21:01:30.569 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.586+0000 7f0e387de740 1 -- 172.21.15.107:0/70560268 shutdown_connections 2024-03-08T21:01:30.569 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.586+0000 7f0e387de740 1 -- 172.21.15.107:0/70560268 wait complete. 2024-03-08T21:01:30.569 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.586+0000 7f0e387de740 20 client.15549 trim_cache size 0 max 16384 2024-03-08T21:01:30.569 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:{ 2024-03-08T21:01:30.569 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "message": "", 2024-03-08T21:01:30.569 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "return_code": 0 2024-03-08T21:01:30.569 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:} 2024-03-08T21:01:30.570 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.587+0000 7f0e2fe00640 1 -- 172.21.15.107:0/2577110218 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f0e140a2a80 msgr2=0x7f0e140a4f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:01:30.570 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.587+0000 7f0e2fe00640 1 --2- 172.21.15.107:0/2577110218 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f0e140a2a80 0x7f0e140a4f40 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x55c9a4317930 tx=0x7f0e2000b500 comp rx=0 tx=0).stop 2024-03-08T21:01:30.570 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.587+0000 7f0e2fe00640 1 -- 172.21.15.107:0/2577110218 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0e30102bc0 msgr2=0x7f0e3019fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:01:30.571 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.587+0000 7f0e2fe00640 1 --2- 172.21.15.107:0/2577110218 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0e30102bc0 0x7f0e3019fca0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f0e1c000c00 tx=0x7f0e1c02c760 comp rx=0 tx=0).stop 2024-03-08T21:01:30.571 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.587+0000 7f0e2fe00640 1 -- 172.21.15.107:0/2577110218 shutdown_connections 2024-03-08T21:01:30.571 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.587+0000 7f0e2fe00640 1 --2- 172.21.15.107:0/2577110218 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f0e140a2a80 0x7f0e140a4f40 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:01:30.571 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.587+0000 7f0e2fe00640 1 --2- 172.21.15.107:0/2577110218 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0e3010b6c0 0x7f0e30199960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:01:30.572 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.587+0000 7f0e2fe00640 1 --2- 172.21.15.107:0/2577110218 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0e30102bc0 0x7f0e3019fca0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:01:30.572 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.587+0000 7f0e2fe00640 1 --2- 172.21.15.107:0/2577110218 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0e301022a0 0x7f0e3019f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:01:30.572 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.587+0000 7f0e2fe00640 1 -- 172.21.15.107:0/2577110218 >> 172.21.15.107:0/2577110218 conn(0x7f0e300fbb00 msgr2=0x7f0e30100f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:01:30.572 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.587+0000 7f0e2fe00640 1 -- 172.21.15.107:0/2577110218 shutdown_connections 2024-03-08T21:01:30.572 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:30.587+0000 7f0e2fe00640 1 -- 172.21.15.107:0/2577110218 wait complete. 2024-03-08T21:01:31.877 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:01:31 smithi204 ceph-mon[34935]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-08T21:01:31.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:01:31 smithi107 ceph-mon[31381]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-08T21:01:32.014 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:01:31 smithi149 ceph-mon[35042]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-08T21:01:33.877 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:01:33 smithi204 ceph-mon[34935]: pgmap v34: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-08T21:01:33.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:01:33 smithi107 ceph-mon[31381]: pgmap v34: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-08T21:01:34.013 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:01:33 smithi149 ceph-mon[35042]: pgmap v34: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-08T21:01:35.877 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:01:35 smithi204 ceph-mon[34935]: pgmap v35: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-08T21:01:35.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:01:35 smithi107 ceph-mon[31381]: pgmap v35: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-08T21:01:36.013 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:01:35 smithi149 ceph-mon[35042]: pgmap v35: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-08T21:01:37.877 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:01:37 smithi204 ceph-mon[34935]: pgmap v36: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:01:37.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:01:37 smithi107 ceph-mon[31381]: pgmap v36: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:01:38.014 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:01:37 smithi149 ceph-mon[35042]: pgmap v36: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:01:39.877 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:01:39 smithi204 ceph-mon[34935]: pgmap v37: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:01:39.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:01:39 smithi107 ceph-mon[31381]: pgmap v37: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:01:40.014 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:01:39 smithi149 ceph-mon[35042]: pgmap v37: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:01:41.876 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:01:41 smithi204 ceph-mon[34935]: pgmap v38: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-08T21:01:41.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:01:41 smithi107 ceph-mon[31381]: pgmap v38: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-08T21:01:42.013 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:01:41 smithi149 ceph-mon[35042]: pgmap v38: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-08T21:01:42.877 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:01:42 smithi204 ceph-mon[34935]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T21:01:42.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:01:42 smithi107 ceph-mon[31381]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T21:01:43.013 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:01:42 smithi149 ceph-mon[35042]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T21:01:43.752 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.771+0000 7f4a87e00640 1 -- 172.21.15.107:0/3072664804 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4a8810b6c0 msgr2=0x7f4a8810db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:01:43.753 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.771+0000 7f4a87e00640 1 --2- 172.21.15.107:0/3072664804 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4a8810b6c0 0x7f4a8810db60 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f4a80009ca0 tx=0x7f4a8002c190 comp rx=0 tx=0).stop 2024-03-08T21:01:43.753 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.771+0000 7f4a87e00640 1 -- 172.21.15.107:0/3072664804 shutdown_connections 2024-03-08T21:01:43.753 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.771+0000 7f4a87e00640 1 --2- 172.21.15.107:0/3072664804 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4a8810b6c0 0x7f4a8810db60 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:01:43.753 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.771+0000 7f4a87e00640 1 --2- 172.21.15.107:0/3072664804 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4a88102bc0 0x7f4a8810b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:01:43.753 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.771+0000 7f4a87e00640 1 --2- 172.21.15.107:0/3072664804 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4a881022a0 0x7f4a88102680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:01:43.753 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.771+0000 7f4a87e00640 1 -- 172.21.15.107:0/3072664804 >> 172.21.15.107:0/3072664804 conn(0x7f4a880fbb00 msgr2=0x7f4a880fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:01:43.753 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.771+0000 7f4a87e00640 1 -- 172.21.15.107:0/3072664804 shutdown_connections 2024-03-08T21:01:43.753 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.771+0000 7f4a87e00640 1 -- 172.21.15.107:0/3072664804 wait complete. 2024-03-08T21:01:43.754 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.772+0000 7f4a87e00640 1 Processor -- start 2024-03-08T21:01:43.754 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.773+0000 7f4a87e00640 1 -- start start 2024-03-08T21:01:43.755 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.773+0000 7f4a87e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4a881022a0 0x7f4a8819f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:01:43.755 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.773+0000 7f4a87e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4a88102bc0 0x7f4a8819fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:01:43.755 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.773+0000 7f4a87e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4a8810b6c0 0x7f4a881998d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:01:43.755 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.773+0000 7f4a87e00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a88111330 con 0x7f4a881022a0 2024-03-08T21:01:43.755 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.773+0000 7f4a87e00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a881111b0 con 0x7f4a8810b6c0 2024-03-08T21:01:43.755 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.773+0000 7f4a87e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a881114b0 con 0x7f4a88102bc0 2024-03-08T21:01:43.755 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.773+0000 7f4a86a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4a881022a0 0x7f4a8819f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:01:43.755 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.774+0000 7f4a86a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4a881022a0 0x7f4a8819f720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:43200/0 (socket says 172.21.15.107:43200) 2024-03-08T21:01:43.755 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.774+0000 7f4a86a00640 1 -- 172.21.15.107:0/3629519426 learned_addr learned my addr 172.21.15.107:0/3629519426 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:01:43.755 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.774+0000 7f4a87400640 1 --2- 172.21.15.107:0/3629519426 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4a8810b6c0 0x7f4a881998d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:01:43.756 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.774+0000 7f4a86000640 1 --2- 172.21.15.107:0/3629519426 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4a88102bc0 0x7f4a8819fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:01:43.756 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.774+0000 7f4a86a00640 1 -- 172.21.15.107:0/3629519426 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4a88102bc0 msgr2=0x7f4a8819fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:01:43.756 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.774+0000 7f4a86a00640 1 --2- 172.21.15.107:0/3629519426 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4a88102bc0 0x7f4a8819fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:01:43.756 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.774+0000 7f4a86a00640 1 -- 172.21.15.107:0/3629519426 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4a8810b6c0 msgr2=0x7f4a881998d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:01:43.757 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.774+0000 7f4a86a00640 1 --2- 172.21.15.107:0/3629519426 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4a8810b6c0 0x7f4a881998d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:01:43.757 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.774+0000 7f4a86a00640 1 -- 172.21.15.107:0/3629519426 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a8819a050 con 0x7f4a881022a0 2024-03-08T21:01:43.757 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.774+0000 7f4a87400640 1 --2- 172.21.15.107:0/3629519426 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4a8810b6c0 0x7f4a881998d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T21:01:43.757 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.774+0000 7f4a86a00640 1 --2- 172.21.15.107:0/3629519426 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4a881022a0 0x7f4a8819f720 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f4a7800e9f0 tx=0x7f4a7800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:01:43.757 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.775+0000 7f4a7f400640 1 -- 172.21.15.107:0/3629519426 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a7800cd80 con 0x7f4a881022a0 2024-03-08T21:01:43.757 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.775+0000 7f4a7f400640 1 -- 172.21.15.107:0/3629519426 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f4a780108e0 con 0x7f4a881022a0 2024-03-08T21:01:43.757 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.775+0000 7f4a7f400640 1 -- 172.21.15.107:0/3629519426 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a78018920 con 0x7f4a881022a0 2024-03-08T21:01:43.758 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.775+0000 7f4a87e00640 1 -- 172.21.15.107:0/3629519426 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a8819a340 con 0x7f4a881022a0 2024-03-08T21:01:43.758 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.775+0000 7f4a87e00640 1 -- 172.21.15.107:0/3629519426 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a881a65e0 con 0x7f4a881022a0 2024-03-08T21:01:43.761 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.780+0000 7f4a7f400640 1 -- 172.21.15.107:0/3629519426 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 24) v1 ==== 137524+0+0 (secure 0 0 0) 0x7f4a78010a90 con 0x7f4a881022a0 2024-03-08T21:01:43.762 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.780+0000 7f4a90588740 1 Processor -- start 2024-03-08T21:01:43.762 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.781+0000 7f4a90588740 1 -- start start 2024-03-08T21:01:43.762 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.781+0000 7f4a90588740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56072ba79e80 0x56072ba7a260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:01:43.762 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.781+0000 7f4a90588740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56072ba7a7a0 0x56072ba83030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:01:43.763 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.781+0000 7f4a90588740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56072ba83570 0x56072ba85960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:01:43.763 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.781+0000 7f4a90588740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x56072b9dd3d0 con 0x56072ba7a7a0 2024-03-08T21:01:43.763 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.781+0000 7f4a90588740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x56072b9866b0 con 0x56072ba79e80 2024-03-08T21:01:43.763 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.781+0000 7f4a90588740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56072b9d9470 con 0x56072ba83570 2024-03-08T21:01:43.764 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.782+0000 7f4a86a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56072ba7a7a0 0x56072ba83030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:01:43.764 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.782+0000 7f4a86a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56072ba7a7a0 0x56072ba83030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:43204/0 (socket says 172.21.15.107:43204) 2024-03-08T21:01:43.764 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.782+0000 7f4a86a00640 1 -- 172.21.15.107:0/434885480 learned_addr learned my addr 172.21.15.107:0/434885480 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:01:43.764 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.782+0000 7f4a87400640 1 --2- 172.21.15.107:0/434885480 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56072ba79e80 0x56072ba7a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:01:43.764 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.782+0000 7f4a86000640 1 --2- 172.21.15.107:0/434885480 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56072ba83570 0x56072ba85960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:01:43.764 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.783+0000 7f4a86a00640 1 -- 172.21.15.107:0/434885480 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56072ba83570 msgr2=0x56072ba85960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:01:43.764 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.783+0000 7f4a86a00640 1 --2- 172.21.15.107:0/434885480 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56072ba83570 0x56072ba85960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:01:43.764 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.783+0000 7f4a86a00640 1 -- 172.21.15.107:0/434885480 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56072ba79e80 msgr2=0x56072ba7a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:01:43.764 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.783+0000 7f4a86a00640 1 --2- 172.21.15.107:0/434885480 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56072ba79e80 0x56072ba7a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:01:43.764 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.783+0000 7f4a7f400640 1 --2- 172.21.15.107:0/3629519426 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f4a6c0a2b00 0x7f4a6c0a4fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:01:43.764 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.783+0000 7f4a86a00640 1 -- 172.21.15.107:0/434885480 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56072b9acb50 con 0x56072ba7a7a0 2024-03-08T21:01:43.765 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.783+0000 7f4a7f400640 1 -- 172.21.15.107:0/3629519426 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f4a780cf510 con 0x7f4a881022a0 2024-03-08T21:01:43.765 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.783+0000 7f4a87400640 1 --2- 172.21.15.107:0/3629519426 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f4a6c0a2b00 0x7f4a6c0a4fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:01:43.765 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.784+0000 7f4a86a00640 1 --2- 172.21.15.107:0/434885480 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56072ba7a7a0 0x56072ba83030 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f4a780cca70 tx=0x7f4a78097c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b951aac6dc01fdce server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:01:43.765 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.784+0000 7f4a87400640 1 --2- 172.21.15.107:0/3629519426 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f4a6c0a2b00 0x7f4a6c0a4fc0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f4a80009430 tx=0x7f4a80004a80 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:01:43.765 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.784+0000 7f4a73e00640 1 -- 172.21.15.107:0/434885480 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a7809b6e0 con 0x56072ba7a7a0 2024-03-08T21:01:43.766 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.784+0000 7f4a73e00640 1 -- 172.21.15.107:0/434885480 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f4a78084e50 con 0x56072ba7a7a0 2024-03-08T21:01:43.766 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.784+0000 7f4a73e00640 1 -- 172.21.15.107:0/434885480 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a7809b270 con 0x56072ba7a7a0 2024-03-08T21:01:43.766 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.785+0000 7f4a90588740 1 -- 172.21.15.107:0/434885480 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56072ba7a7a0 msgr2=0x56072ba83030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:01:43.766 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.785+0000 7f4a90588740 1 --2- 172.21.15.107:0/434885480 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56072ba7a7a0 0x56072ba83030 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f4a780cca70 tx=0x7f4a78097c70 comp rx=0 tx=0).stop 2024-03-08T21:01:43.766 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.785+0000 7f4a90588740 1 -- 172.21.15.107:0/434885480 shutdown_connections 2024-03-08T21:01:43.766 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.785+0000 7f4a90588740 1 --2- 172.21.15.107:0/434885480 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56072ba83570 0x56072ba85960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:01:43.767 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.785+0000 7f4a90588740 1 --2- 172.21.15.107:0/434885480 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56072ba7a7a0 0x56072ba83030 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:01:43.767 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.785+0000 7f4a90588740 1 --2- 172.21.15.107:0/434885480 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56072ba79e80 0x56072ba7a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:01:43.767 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.785+0000 7f4a90588740 1 -- 172.21.15.107:0/434885480 >> 172.21.15.107:0/434885480 conn(0x56072ba747a0 msgr2=0x56072ba74b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:01:43.767 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.786+0000 7f4a90588740 1 -- 172.21.15.107:0/434885480 shutdown_connections 2024-03-08T21:01:43.767 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.786+0000 7f4a90588740 1 -- 172.21.15.107:0/434885480 wait complete. 2024-03-08T21:01:43.767 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.786+0000 7f4a90588740 1 -- start start 2024-03-08T21:01:43.768 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.787+0000 7f4a90588740 1 Processor -- start 2024-03-08T21:01:43.769 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.787+0000 7f4a90588740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56072ba79e80 0x56072ba82fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:01:43.769 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.787+0000 7f4a90588740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56072ba83570 0x56072ba802c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:01:43.769 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.787+0000 7f4a90588740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56072ba80940 0x56072ba8f580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:01:43.769 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.787+0000 7f4a90588740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x56072ba860e0 con 0x56072ba79e80 2024-03-08T21:01:43.769 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.787+0000 7f4a90588740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x56072b9999f0 con 0x56072ba80940 2024-03-08T21:01:43.769 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.787+0000 7f4a90588740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56072ba86260 con 0x56072ba83570 2024-03-08T21:01:43.769 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.787+0000 7f4a90588740 10 client.0 _ll_register_callbacks cb 0x56072b6f1a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T21:01:43.769 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.788+0000 7f4a86a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56072ba79e80 0x56072ba82fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:01:43.770 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.788+0000 7f4a86a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56072ba79e80 0x56072ba82fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:43206/0 (socket says 172.21.15.107:43206) 2024-03-08T21:01:43.771 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.788+0000 7f4a86a00640 1 -- 172.21.15.107:0/3173414667 learned_addr learned my addr 172.21.15.107:0/3173414667 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:01:43.771 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.788+0000 7f4a87400640 1 --2- 172.21.15.107:0/3173414667 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56072ba80940 0x56072ba8f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:01:43.771 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.788+0000 7f4a86a00640 1 -- 172.21.15.107:0/3173414667 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56072ba83570 msgr2=0x56072ba802c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:01:43.771 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.788+0000 7f4a86a00640 1 --2- 172.21.15.107:0/3173414667 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56072ba83570 0x56072ba802c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:01:43.771 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.788+0000 7f4a86a00640 1 -- 172.21.15.107:0/3173414667 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56072ba80940 msgr2=0x56072ba8f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:01:43.771 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.788+0000 7f4a86a00640 1 --2- 172.21.15.107:0/3173414667 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56072ba80940 0x56072ba8f580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:01:43.771 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.788+0000 7f4a86a00640 1 -- 172.21.15.107:0/3173414667 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56072b6f2210 con 0x56072ba79e80 2024-03-08T21:01:43.771 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.789+0000 7f4a86a00640 1 --2- 172.21.15.107:0/3173414667 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56072ba79e80 0x56072ba82fd0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f4a78097d20 tx=0x7f4a780d2e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=356e11bcc4e591bd server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:01:43.771 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.789+0000 7f4a70c00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-03-08T21:01:43.771 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.789+0000 7f4a70c00640 1 -- 172.21.15.107:0/3173414667 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a7809b450 con 0x56072ba79e80 2024-03-08T21:01:43.771 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.789+0000 7f4a70c00640 1 -- 172.21.15.107:0/3173414667 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f4a7809bb70 con 0x56072ba79e80 2024-03-08T21:01:43.771 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.789+0000 7f4a90588740 1 -- 172.21.15.107:0/3173414667 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4a78084e50 con 0x56072ba79e80 2024-03-08T21:01:43.772 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.789+0000 7f4a70c00640 1 -- 172.21.15.107:0/3173414667 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4a7809b270 con 0x56072ba79e80 2024-03-08T21:01:43.772 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.790+0000 7f4a90588740 10 client.15561 fetch_fsmap learned FSMap version 30 2024-03-08T21:01:43.772 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.790+0000 7f4a90588740 1 -- 172.21.15.107:0/3173414667 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x56072b7be620 con 0x56072ba79e80 2024-03-08T21:01:43.772 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.790+0000 7f4a70c00640 1 -- 172.21.15.107:0/3173414667 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f4a78084e50 con 0x56072ba79e80 2024-03-08T21:01:43.773 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.790+0000 7f4a70c00640 1 -- 172.21.15.107:0/3173414667 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 30) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f4a780a3c20 con 0x56072ba79e80 2024-03-08T21:01:43.773 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.790+0000 7f4a90588740 10 client.15561 fetch_fsmap finished waiting for FSMap version 30 2024-03-08T21:01:43.773 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.790+0000 7f4a90588740 10 client.15561 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-08T21:01:43.773 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.790+0000 7f4a90588740 20 client.15561 populate_metadata read hostname 'smithi107' 2024-03-08T21:01:43.773 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.790+0000 7f4a90588740 1 --2- 172.21.15.107:0/3173414667 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x56072ba936d0 0x56072ba95ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:01:43.773 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.790+0000 7f4a90588740 4 client.15561 mds_command: new command op to 34475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-08T21:01:43.773 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.790+0000 7f4a90588740 1 -- 172.21.15.107:0/3173414667 --> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56072b9ea910 con 0x56072ba936d0 2024-03-08T21:01:43.773 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.791+0000 7f4a86000640 1 --2- 172.21.15.107:0/3173414667 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x56072ba936d0 0x56072ba95ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:01:43.774 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.792+0000 7f4a86000640 1 --2- 172.21.15.107:0/3173414667 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x56072ba936d0 0x56072ba95ac0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7146ed4c09f03868 server_cookie=2d6e336e862d1b57 in_seq=0 out_seq=0 2024-03-08T21:01:43.774 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.792+0000 7f4a70c00640 10 client.15561 ms_handle_connect on v2:172.21.15.149:6838/55290596 2024-03-08T21:01:43.775 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.794+0000 7f4a70c00640 1 -- 172.21.15.107:0/3173414667 <== mds.0 v2:172.21.15.149:6838/55290596 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56072b9ea910 con 0x56072ba936d0 2024-03-08T21:01:43.775 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.794+0000 7f4a70c00640 10 client.15561 handle_command_reply: tid=0 2024-03-08T21:01:43.776 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.794+0000 7f4a90588740 1 client.15561 shutdown 2024-03-08T21:01:43.776 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.794+0000 7f4a90588740 1 -- 172.21.15.107:0/3173414667 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56072ba79e80 msgr2=0x56072ba82fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:01:43.776 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.794+0000 7f4a90588740 1 --2- 172.21.15.107:0/3173414667 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56072ba79e80 0x56072ba82fd0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f4a78097d20 tx=0x7f4a780d2e70 comp rx=0 tx=0).stop 2024-03-08T21:01:43.776 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.794+0000 7f4a90588740 1 -- 172.21.15.107:0/3173414667 shutdown_connections 2024-03-08T21:01:43.776 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.794+0000 7f4a90588740 1 --2- 172.21.15.107:0/3173414667 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x56072ba936d0 0x56072ba95ac0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:01:43.776 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.794+0000 7f4a90588740 1 --2- 172.21.15.107:0/3173414667 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56072ba80940 0x56072ba8f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:01:43.776 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.794+0000 7f4a90588740 1 --2- 172.21.15.107:0/3173414667 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56072ba83570 0x56072ba802c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:01:43.776 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.794+0000 7f4a90588740 1 --2- 172.21.15.107:0/3173414667 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56072ba79e80 0x56072ba82fd0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:01:43.776 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.794+0000 7f4a90588740 1 -- 172.21.15.107:0/3173414667 >> 172.21.15.107:0/3173414667 conn(0x56072ba747a0 msgr2=0x56072ba764f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:01:43.776 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.794+0000 7f4a70c00640 0 client.15561 ms_handle_reset on v2:172.21.15.149:6838/55290596 2024-03-08T21:01:43.776 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.794+0000 7f4a90588740 1 -- 172.21.15.107:0/3173414667 shutdown_connections 2024-03-08T21:01:43.776 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.794+0000 7f4a90588740 1 -- 172.21.15.107:0/3173414667 wait complete. 2024-03-08T21:01:43.776 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.795+0000 7f4a90588740 20 client.15561 trim_cache size 0 max 16384 2024-03-08T21:01:43.803 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.821+0000 7f4a90588740 1 Processor -- start 2024-03-08T21:01:43.803 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.821+0000 7f4a90588740 1 -- start start 2024-03-08T21:01:43.803 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.822+0000 7f4a90588740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56072ba79e80 0x56072ba82b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:01:43.803 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.822+0000 7f4a90588740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56072ba80940 0x56072ba7fb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:01:43.803 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.822+0000 7f4a90588740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56072ba83570 0x56072ba80090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:01:43.803 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.822+0000 7f4a90588740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a881aee80 con 0x56072ba79e80 2024-03-08T21:01:43.803 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.822+0000 7f4a90588740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a881b0800 con 0x56072ba83570 2024-03-08T21:01:43.803 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.822+0000 7f4a90588740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a881af4e0 con 0x56072ba80940 2024-03-08T21:01:43.804 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.822+0000 7f4a86a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56072ba79e80 0x56072ba82b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:01:43.805 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.822+0000 7f4a86a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56072ba79e80 0x56072ba82b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:43220/0 (socket says 172.21.15.107:43220) 2024-03-08T21:01:43.805 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.822+0000 7f4a86a00640 1 -- 172.21.15.107:0/1348065308 learned_addr learned my addr 172.21.15.107:0/1348065308 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:01:43.805 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.822+0000 7f4a86000640 1 --2- 172.21.15.107:0/1348065308 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56072ba80940 0x56072ba7fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:01:43.805 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.822+0000 7f4a87400640 1 --2- 172.21.15.107:0/1348065308 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56072ba83570 0x56072ba80090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:01:43.805 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.823+0000 7f4a86a00640 1 -- 172.21.15.107:0/1348065308 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56072ba80940 msgr2=0x56072ba7fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:01:43.805 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.823+0000 7f4a86a00640 1 --2- 172.21.15.107:0/1348065308 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56072ba80940 0x56072ba7fb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:01:43.805 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.823+0000 7f4a86a00640 1 -- 172.21.15.107:0/1348065308 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56072ba83570 msgr2=0x56072ba80090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:01:43.805 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.823+0000 7f4a86a00640 1 --2- 172.21.15.107:0/1348065308 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56072ba83570 0x56072ba80090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:01:43.805 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.823+0000 7f4a86a00640 1 -- 172.21.15.107:0/1348065308 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56072ba7e810 con 0x56072ba79e80 2024-03-08T21:01:43.805 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.823+0000 7f4a86000640 1 --2- 172.21.15.107:0/1348065308 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56072ba80940 0x56072ba7fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T21:01:43.805 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.823+0000 7f4a86a00640 1 --2- 172.21.15.107:0/1348065308 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56072ba79e80 0x56072ba82b00 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f4a7809b8a0 tx=0x7f4a780a3fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f910d77f9d289c9e server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:01:43.805 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.823+0000 7f4a73400640 1 -- 172.21.15.107:0/1348065308 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a7809b6e0 con 0x56072ba79e80 2024-03-08T21:01:43.806 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.823+0000 7f4a73400640 1 -- 172.21.15.107:0/1348065308 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f4a78084e50 con 0x56072ba79e80 2024-03-08T21:01:43.806 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.823+0000 7f4a73400640 1 -- 172.21.15.107:0/1348065308 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a7809bb70 con 0x56072ba79e80 2024-03-08T21:01:43.807 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.824+0000 7f4a90588740 1 -- 172.21.15.107:0/1348065308 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56072ba79e80 msgr2=0x56072ba82b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:01:43.807 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.824+0000 7f4a90588740 1 --2- 172.21.15.107:0/1348065308 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56072ba79e80 0x56072ba82b00 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f4a7809b8a0 tx=0x7f4a780a3fc0 comp rx=0 tx=0).stop 2024-03-08T21:01:43.807 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.824+0000 7f4a90588740 1 -- 172.21.15.107:0/1348065308 shutdown_connections 2024-03-08T21:01:43.807 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.824+0000 7f4a90588740 1 --2- 172.21.15.107:0/1348065308 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56072ba83570 0x56072ba80090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:01:43.807 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.824+0000 7f4a90588740 1 --2- 172.21.15.107:0/1348065308 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56072ba80940 0x56072ba7fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:01:43.807 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.824+0000 7f4a90588740 1 --2- 172.21.15.107:0/1348065308 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56072ba79e80 0x56072ba82b00 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:01:43.807 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.824+0000 7f4a90588740 1 -- 172.21.15.107:0/1348065308 >> 172.21.15.107:0/1348065308 conn(0x56072ba747a0 msgr2=0x56072ba7eb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:01:43.807 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.825+0000 7f4a90588740 1 -- 172.21.15.107:0/1348065308 shutdown_connections 2024-03-08T21:01:43.807 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.825+0000 7f4a90588740 1 -- 172.21.15.107:0/1348065308 wait complete. 2024-03-08T21:01:43.807 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.825+0000 7f4a90588740 1 -- start start 2024-03-08T21:01:43.807 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.825+0000 7f4a90588740 1 Processor -- start 2024-03-08T21:01:43.809 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.826+0000 7f4a90588740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56072ba80940 0x56072ba8a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:01:43.809 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.826+0000 7f4a90588740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56072ba83570 0x56072ba88cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:01:43.809 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.826+0000 7f4a90588740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56072ba89390 0x56072bae9810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:01:43.809 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.826+0000 7f4a90588740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a881b01a0 con 0x56072ba80940 2024-03-08T21:01:43.810 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.826+0000 7f4a90588740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a881afb40 con 0x56072ba83570 2024-03-08T21:01:43.810 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.826+0000 7f4a90588740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a881ad120 con 0x56072ba89390 2024-03-08T21:01:43.810 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.826+0000 7f4a90588740 10 client.0 _ll_register_callbacks cb 0x7f4a780d2ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T21:01:43.810 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.827+0000 7f4a86a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56072ba80940 0x56072ba8a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:01:43.810 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.827+0000 7f4a86a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56072ba80940 0x56072ba8a9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:43236/0 (socket says 172.21.15.107:43236) 2024-03-08T21:01:43.810 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.827+0000 7f4a86a00640 1 -- 172.21.15.107:0/3941133906 learned_addr learned my addr 172.21.15.107:0/3941133906 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:01:43.810 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.827+0000 7f4a87400640 1 --2- 172.21.15.107:0/3941133906 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56072ba89390 0x56072bae9810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:01:43.810 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.827+0000 7f4a86000640 1 --2- 172.21.15.107:0/3941133906 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56072ba83570 0x56072ba88cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:01:43.810 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.827+0000 7f4a86a00640 1 -- 172.21.15.107:0/3941133906 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56072ba89390 msgr2=0x56072bae9810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:01:43.810 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.827+0000 7f4a86a00640 1 --2- 172.21.15.107:0/3941133906 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56072ba89390 0x56072bae9810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:01:43.810 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.827+0000 7f4a86a00640 1 -- 172.21.15.107:0/3941133906 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56072ba83570 msgr2=0x56072ba88cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:01:43.810 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.827+0000 7f4a86a00640 1 --2- 172.21.15.107:0/3941133906 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56072ba83570 0x56072ba88cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:01:43.810 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.827+0000 7f4a86a00640 1 -- 172.21.15.107:0/3941133906 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56072ba89ad0 con 0x56072ba80940 2024-03-08T21:01:43.811 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.828+0000 7f4a86a00640 1 --2- 172.21.15.107:0/3941133906 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56072ba80940 0x56072ba8a9a0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f4a780a3b80 tx=0x7f4a7809c080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82ca0452ae9dc90c server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:01:43.811 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.828+0000 7f4a70c00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-03-08T21:01:43.812 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.828+0000 7f4a70c00640 1 -- 172.21.15.107:0/3941133906 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a7809b6e0 con 0x56072ba80940 2024-03-08T21:01:43.812 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.828+0000 7f4a70c00640 1 -- 172.21.15.107:0/3941133906 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f4a780a3c20 con 0x56072ba80940 2024-03-08T21:01:43.812 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.828+0000 7f4a90588740 1 -- 172.21.15.107:0/3941133906 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4a78084e50 con 0x56072ba80940 2024-03-08T21:01:43.812 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.828+0000 7f4a70c00640 1 -- 172.21.15.107:0/3941133906 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4a7809b450 con 0x56072ba80940 2024-03-08T21:01:43.812 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.828+0000 7f4a90588740 10 client.15567 fetch_fsmap learned FSMap version 30 2024-03-08T21:01:43.812 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.828+0000 7f4a90588740 1 -- 172.21.15.107:0/3941133906 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x56072ba89ad0 con 0x56072ba80940 2024-03-08T21:01:43.812 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.829+0000 7f4a70c00640 1 -- 172.21.15.107:0/3941133906 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f4a78084e50 con 0x56072ba80940 2024-03-08T21:01:43.812 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.829+0000 7f4a70c00640 1 -- 172.21.15.107:0/3941133906 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 30) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f4a780cce40 con 0x56072ba80940 2024-03-08T21:01:43.812 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.829+0000 7f4a90588740 10 client.15567 fetch_fsmap finished waiting for FSMap version 30 2024-03-08T21:01:43.812 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.829+0000 7f4a90588740 10 client.15567 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-08T21:01:43.812 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.829+0000 7f4a90588740 20 client.15567 populate_metadata read hostname 'smithi107' 2024-03-08T21:01:43.813 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.829+0000 7f4a90588740 1 --2- 172.21.15.107:0/3941133906 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x56072baeafe0 0x56072baed3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:01:43.813 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.829+0000 7f4a90588740 4 client.15567 mds_command: new command op to 34475 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-08T21:01:43.813 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.829+0000 7f4a90588740 1 -- 172.21.15.107:0/3941133906 --> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x56072b9e4c60 con 0x56072baeafe0 2024-03-08T21:01:43.813 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.829+0000 7f4a86000640 1 --2- 172.21.15.107:0/3941133906 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x56072baeafe0 0x56072baed3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:01:43.813 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.830+0000 7f4a86000640 1 --2- 172.21.15.107:0/3941133906 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x56072baeafe0 0x56072baed3d0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=154523e4075a6360 server_cookie=674683c06fcf4953 in_seq=0 out_seq=0 2024-03-08T21:01:43.813 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.830+0000 7f4a70c00640 10 client.15567 ms_handle_connect on v2:172.21.15.149:6838/55290596 2024-03-08T21:01:43.822 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.840+0000 7f4a70c00640 1 -- 172.21.15.107:0/3941133906 <== mds.0 v2:172.21.15.149:6838/55290596 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x56072b9e4c60 con 0x56072baeafe0 2024-03-08T21:01:43.822 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.840+0000 7f4a70c00640 10 client.15567 handle_command_reply: tid=0 2024-03-08T21:01:43.823 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.840+0000 7f4a90588740 1 client.15567 shutdown 2024-03-08T21:01:43.823 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.840+0000 7f4a90588740 1 -- 172.21.15.107:0/3941133906 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56072ba80940 msgr2=0x56072ba8a9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:01:43.823 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.840+0000 7f4a90588740 1 --2- 172.21.15.107:0/3941133906 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56072ba80940 0x56072ba8a9a0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f4a780a3b80 tx=0x7f4a7809c080 comp rx=0 tx=0).stop 2024-03-08T21:01:43.823 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.840+0000 7f4a90588740 1 -- 172.21.15.107:0/3941133906 shutdown_connections 2024-03-08T21:01:43.823 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.840+0000 7f4a90588740 1 --2- 172.21.15.107:0/3941133906 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56072ba89390 0x56072bae9810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:01:43.823 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.840+0000 7f4a90588740 1 --2- 172.21.15.107:0/3941133906 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x56072baeafe0 0x56072baed3d0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:01:43.823 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.840+0000 7f4a90588740 1 --2- 172.21.15.107:0/3941133906 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56072ba83570 0x56072ba88cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:01:43.823 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.840+0000 7f4a90588740 1 --2- 172.21.15.107:0/3941133906 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56072ba80940 0x56072ba8a9a0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:01:43.823 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.840+0000 7f4a90588740 1 -- 172.21.15.107:0/3941133906 >> 172.21.15.107:0/3941133906 conn(0x56072ba747a0 msgr2=0x56072ba7ea20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:01:43.823 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.840+0000 7f4a70c00640 0 client.15567 ms_handle_reset on v2:172.21.15.149:6838/55290596 2024-03-08T21:01:43.823 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.840+0000 7f4a90588740 1 -- 172.21.15.107:0/3941133906 shutdown_connections 2024-03-08T21:01:43.823 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.840+0000 7f4a90588740 1 -- 172.21.15.107:0/3941133906 wait complete. 2024-03-08T21:01:43.823 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.841+0000 7f4a90588740 20 client.15567 trim_cache size 0 max 16384 2024-03-08T21:01:43.824 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:{ 2024-03-08T21:01:43.824 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "message": "", 2024-03-08T21:01:43.824 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "return_code": 0 2024-03-08T21:01:43.824 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:} 2024-03-08T21:01:43.825 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.841+0000 7f4a87e00640 1 -- 172.21.15.107:0/3629519426 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f4a6c0a2b00 msgr2=0x7f4a6c0a4fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:01:43.825 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.841+0000 7f4a87e00640 1 --2- 172.21.15.107:0/3629519426 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f4a6c0a2b00 0x7f4a6c0a4fc0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f4a80009430 tx=0x7f4a80004a80 comp rx=0 tx=0).stop 2024-03-08T21:01:43.825 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.841+0000 7f4a87e00640 1 -- 172.21.15.107:0/3629519426 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4a881022a0 msgr2=0x7f4a8819f720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:01:43.825 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.841+0000 7f4a87e00640 1 --2- 172.21.15.107:0/3629519426 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4a881022a0 0x7f4a8819f720 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f4a7800e9f0 tx=0x7f4a7800eeb0 comp rx=0 tx=0).stop 2024-03-08T21:01:43.825 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.841+0000 7f4a87e00640 1 -- 172.21.15.107:0/3629519426 shutdown_connections 2024-03-08T21:01:43.825 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.842+0000 7f4a87e00640 1 --2- 172.21.15.107:0/3629519426 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f4a6c0a2b00 0x7f4a6c0a4fc0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:01:43.825 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.842+0000 7f4a87e00640 1 --2- 172.21.15.107:0/3629519426 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4a8810b6c0 0x7f4a881998d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:01:43.825 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.842+0000 7f4a87e00640 1 --2- 172.21.15.107:0/3629519426 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4a88102bc0 0x7f4a8819fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:01:43.825 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.842+0000 7f4a87e00640 1 --2- 172.21.15.107:0/3629519426 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4a881022a0 0x7f4a8819f720 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:01:43.825 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.842+0000 7f4a87e00640 1 -- 172.21.15.107:0/3629519426 >> 172.21.15.107:0/3629519426 conn(0x7f4a880fbb00 msgr2=0x7f4a88101060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:01:43.825 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.842+0000 7f4a87e00640 1 -- 172.21.15.107:0/3629519426 shutdown_connections 2024-03-08T21:01:43.826 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:43.842+0000 7f4a87e00640 1 -- 172.21.15.107:0/3629519426 wait complete. 2024-03-08T21:01:43.866 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:01:43 smithi107 ceph-mon[31381]: pgmap v39: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:01:43.867 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:01:43 smithi107 ceph-mon[31381]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T21:01:43.867 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:01:43 smithi107 ceph-mon[31381]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T21:01:43.867 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:01:43 smithi107 ceph-mon[31381]: from='mgr.15375 ' entity='mgr.x' 2024-03-08T21:01:43.877 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:01:43 smithi204 ceph-mon[34935]: pgmap v39: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:01:43.877 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:01:43 smithi204 ceph-mon[34935]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T21:01:43.877 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:01:43 smithi204 ceph-mon[34935]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T21:01:43.877 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:01:43 smithi204 ceph-mon[34935]: from='mgr.15375 ' entity='mgr.x' 2024-03-08T21:01:44.014 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:01:43 smithi149 ceph-mon[35042]: pgmap v39: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:01:44.014 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:01:43 smithi149 ceph-mon[35042]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T21:01:44.014 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:01:43 smithi149 ceph-mon[35042]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T21:01:44.014 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:01:43 smithi149 ceph-mon[35042]: from='mgr.15375 ' entity='mgr.x' 2024-03-08T21:01:45.877 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:01:45 smithi204 ceph-mon[34935]: pgmap v40: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-08T21:01:45.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:01:45 smithi107 ceph-mon[31381]: pgmap v40: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-08T21:01:46.013 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:01:45 smithi149 ceph-mon[35042]: pgmap v40: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-08T21:01:47.878 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:01:47 smithi204 ceph-mon[34935]: pgmap v41: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-08T21:01:47.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:01:47 smithi107 ceph-mon[31381]: pgmap v41: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-08T21:01:48.013 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:01:47 smithi149 ceph-mon[35042]: pgmap v41: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-08T21:01:49.877 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:01:49 smithi204 ceph-mon[34935]: pgmap v42: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-08T21:01:49.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:01:49 smithi107 ceph-mon[31381]: pgmap v42: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-08T21:01:50.013 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:01:49 smithi149 ceph-mon[35042]: pgmap v42: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-08T21:01:51.877 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:01:51 smithi204 ceph-mon[34935]: pgmap v43: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-08T21:01:51.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:01:51 smithi107 ceph-mon[31381]: pgmap v43: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-08T21:01:52.013 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:01:51 smithi149 ceph-mon[35042]: pgmap v43: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-08T21:01:53.877 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:01:53 smithi204 ceph-mon[34935]: pgmap v44: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:01:53.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:01:53 smithi107 ceph-mon[31381]: pgmap v44: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:01:54.013 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:01:53 smithi149 ceph-mon[35042]: pgmap v44: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:01:54.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:01:54 smithi107 ceph-mon[31381]: pgmap v45: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-08T21:01:55.013 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:01:54 smithi149 ceph-mon[35042]: pgmap v45: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-08T21:01:55.127 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:01:54 smithi204 ceph-mon[34935]: pgmap v45: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-08T21:01:56.963 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:56.981+0000 7fb4be600640 1 -- 172.21.15.107:0/3763999240 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb4b8074040 msgr2=0x7fb4b810d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:01:56.963 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:56.981+0000 7fb4be600640 1 --2- 172.21.15.107:0/3763999240 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb4b8074040 0x7fb4b810d3d0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fb4a4009900 tx=0x7fb4a402bcd0 comp rx=0 tx=0).stop 2024-03-08T21:01:56.963 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:56.981+0000 7fb4be600640 1 -- 172.21.15.107:0/3763999240 shutdown_connections 2024-03-08T21:01:56.963 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:56.981+0000 7fb4be600640 1 --2- 172.21.15.107:0/3763999240 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb4b810d910 0x7fb4b810fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:01:56.963 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:56.981+0000 7fb4be600640 1 --2- 172.21.15.107:0/3763999240 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb4b8074040 0x7fb4b810d3d0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:01:56.963 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:56.981+0000 7fb4be600640 1 --2- 172.21.15.107:0/3763999240 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb4b8073720 0x7fb4b8073b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:01:56.963 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:56.981+0000 7fb4be600640 1 -- 172.21.15.107:0/3763999240 >> 172.21.15.107:0/3763999240 conn(0x7fb4b80fcc40 msgr2=0x7fb4b80ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:01:56.963 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:56.981+0000 7fb4be600640 1 -- 172.21.15.107:0/3763999240 shutdown_connections 2024-03-08T21:01:56.963 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:56.981+0000 7fb4be600640 1 -- 172.21.15.107:0/3763999240 wait complete. 2024-03-08T21:01:56.965 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:56.982+0000 7fb4be600640 1 Processor -- start 2024-03-08T21:01:56.966 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:56.982+0000 7fb4be600640 1 -- start start 2024-03-08T21:01:56.966 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:56.983+0000 7fb4be600640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb4b8073720 0x7fb4b819f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:01:56.966 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:56.983+0000 7fb4be600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb4b8074040 0x7fb4b819fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:01:56.966 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:56.983+0000 7fb4be600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb4b810d910 0x7fb4b81999d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:01:56.966 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:56.983+0000 7fb4be600640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4b81135f0 con 0x7fb4b8074040 2024-03-08T21:01:56.966 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:56.983+0000 7fb4be600640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4b8113470 con 0x7fb4b8073720 2024-03-08T21:01:56.966 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:56.983+0000 7fb4be600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4b8113770 con 0x7fb4b810d910 2024-03-08T21:01:56.966 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:56.983+0000 7fb4b7e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb4b8074040 0x7fb4b819fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:01:56.966 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:56.983+0000 7fb4b7e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb4b8074040 0x7fb4b819fd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:42990/0 (socket says 172.21.15.107:42990) 2024-03-08T21:01:56.966 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:56.983+0000 7fb4b7e00640 1 -- 172.21.15.107:0/3134550705 learned_addr learned my addr 172.21.15.107:0/3134550705 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:01:56.967 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:56.983+0000 7fb4bd200640 1 --2- 172.21.15.107:0/3134550705 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb4b8073720 0x7fb4b819f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:01:56.967 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:56.983+0000 7fb4b7e00640 1 -- 172.21.15.107:0/3134550705 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb4b810d910 msgr2=0x7fb4b81999d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:01:56.967 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:56.983+0000 7fb4bdc00640 1 --2- 172.21.15.107:0/3134550705 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb4b810d910 0x7fb4b81999d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:01:56.967 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:56.983+0000 7fb4b7e00640 1 --2- 172.21.15.107:0/3134550705 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb4b810d910 0x7fb4b81999d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:01:56.967 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:56.983+0000 7fb4b7e00640 1 -- 172.21.15.107:0/3134550705 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb4b8073720 msgr2=0x7fb4b819f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:01:56.967 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:56.983+0000 7fb4b7e00640 1 --2- 172.21.15.107:0/3134550705 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb4b8073720 0x7fb4b819f7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:01:56.967 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:56.983+0000 7fb4b7e00640 1 -- 172.21.15.107:0/3134550705 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4b819a150 con 0x7fb4b8074040 2024-03-08T21:01:56.967 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:56.983+0000 7fb4bd200640 1 --2- 172.21.15.107:0/3134550705 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb4b8073720 0x7fb4b819f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T21:01:56.967 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:56.983+0000 7fb4bdc00640 1 --2- 172.21.15.107:0/3134550705 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb4b810d910 0x7fb4b81999d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T21:01:56.967 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:56.984+0000 7fb4b7e00640 1 --2- 172.21.15.107:0/3134550705 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb4b8074040 0x7fb4b819fd10 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fb4a4005ab0 tx=0x7fb4a402db40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:01:56.967 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:56.984+0000 7fb4b5600640 1 -- 172.21.15.107:0/3134550705 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4a4043020 con 0x7fb4b8074040 2024-03-08T21:01:56.967 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:56.984+0000 7fb4b5600640 1 -- 172.21.15.107:0/3134550705 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fb4a40052b0 con 0x7fb4b8074040 2024-03-08T21:01:56.967 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:56.984+0000 7fb4be600640 1 -- 172.21.15.107:0/3134550705 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4b819a3e0 con 0x7fb4b8074040 2024-03-08T21:01:56.967 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:56.984+0000 7fb4b5600640 1 -- 172.21.15.107:0/3134550705 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4a4003c50 con 0x7fb4b8074040 2024-03-08T21:01:56.967 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:56.984+0000 7fb4be600640 1 -- 172.21.15.107:0/3134550705 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4b81a6690 con 0x7fb4b8074040 2024-03-08T21:01:56.969 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:56.987+0000 7fb4b5600640 1 -- 172.21.15.107:0/3134550705 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 24) v1 ==== 137524+0+0 (secure 0 0 0) 0x7fb4a4046020 con 0x7fb4b8074040 2024-03-08T21:01:56.969 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:56.987+0000 7fb4c2977740 1 Processor -- start 2024-03-08T21:01:56.969 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:56.987+0000 7fb4c2977740 1 -- start start 2024-03-08T21:01:56.969 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:56.988+0000 7fb4c2977740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561bb8518e80 0x561bb8519260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:01:56.969 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:56.988+0000 7fb4c2977740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561bb85197a0 0x561bb8522030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:01:56.969 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:56.988+0000 7fb4c2977740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561bb8522570 0x561bb8524960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:01:56.970 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:56.988+0000 7fb4c2977740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bb847c3d0 con 0x561bb85197a0 2024-03-08T21:01:56.970 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:56.988+0000 7fb4c2977740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bb84256b0 con 0x561bb8518e80 2024-03-08T21:01:56.970 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:56.988+0000 7fb4c2977740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bb8478470 con 0x561bb8522570 2024-03-08T21:01:56.970 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:56.988+0000 7fb4bd200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561bb85197a0 0x561bb8522030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:01:56.970 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:56.988+0000 7fb4bd200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561bb85197a0 0x561bb8522030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:42994/0 (socket says 172.21.15.107:42994) 2024-03-08T21:01:56.970 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:56.988+0000 7fb4bd200640 1 -- 172.21.15.107:0/763034538 learned_addr learned my addr 172.21.15.107:0/763034538 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:01:56.970 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:56.988+0000 7fb4bdc00640 1 --2- 172.21.15.107:0/763034538 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561bb8518e80 0x561bb8519260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:01:56.970 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:56.988+0000 7fb4b7e00640 1 --2- 172.21.15.107:0/763034538 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561bb8522570 0x561bb8524960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:01:56.974 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:56.988+0000 7fb4b5600640 1 --2- 172.21.15.107:0/3134550705 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7fb4a00a2b00 0x7fb4a00a4fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:01:56.974 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:56.989+0000 7fb4bd200640 1 -- 172.21.15.107:0/763034538 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561bb8522570 msgr2=0x561bb8524960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:01:56.974 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:56.989+0000 7fb4bd200640 1 --2- 172.21.15.107:0/763034538 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561bb8522570 0x561bb8524960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:01:56.974 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:56.989+0000 7fb4bd200640 1 -- 172.21.15.107:0/763034538 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561bb8518e80 msgr2=0x561bb8519260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:01:56.974 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:56.989+0000 7fb4bd200640 1 --2- 172.21.15.107:0/763034538 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561bb8518e80 0x561bb8519260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:01:56.975 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:56.989+0000 7fb4bd200640 1 -- 172.21.15.107:0/763034538 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561bb844bb50 con 0x561bb85197a0 2024-03-08T21:01:56.975 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:56.989+0000 7fb4b7e00640 1 --2- 172.21.15.107:0/763034538 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561bb8522570 0x561bb8524960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T21:01:56.975 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:56.989+0000 7fb4b5600640 1 -- 172.21.15.107:0/3134550705 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fb4a40f1380 con 0x7fb4b8074040 2024-03-08T21:01:56.975 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:56.989+0000 7fb4bdc00640 1 --2- 172.21.15.107:0/3134550705 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7fb4a00a2b00 0x7fb4a00a4fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:01:56.975 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:56.989+0000 7fb4bd200640 1 --2- 172.21.15.107:0/763034538 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561bb85197a0 0x561bb8522030 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fb4ac00bb80 tx=0x7fb4ac00d710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5095300a22b65a25 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:01:56.975 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:56.989+0000 7fb4aa000640 1 -- 172.21.15.107:0/763034538 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4ac017620 con 0x561bb85197a0 2024-03-08T21:01:56.975 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:56.989+0000 7fb4aa000640 1 -- 172.21.15.107:0/763034538 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fb4ac017c70 con 0x561bb85197a0 2024-03-08T21:01:56.975 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:56.989+0000 7fb4aa000640 1 -- 172.21.15.107:0/763034538 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4ac00fb00 con 0x561bb85197a0 2024-03-08T21:01:56.975 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:56.990+0000 7fb4c2977740 1 -- 172.21.15.107:0/763034538 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561bb85197a0 msgr2=0x561bb8522030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:01:56.975 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:56.990+0000 7fb4bdc00640 1 --2- 172.21.15.107:0/3134550705 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7fb4a00a2b00 0x7fb4a00a4fc0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fb4b000f4f0 tx=0x7fb4b0004b50 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:01:56.975 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:56.990+0000 7fb4c2977740 1 --2- 172.21.15.107:0/763034538 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561bb85197a0 0x561bb8522030 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fb4ac00bb80 tx=0x7fb4ac00d710 comp rx=0 tx=0).stop 2024-03-08T21:01:56.975 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:56.990+0000 7fb4c2977740 1 -- 172.21.15.107:0/763034538 shutdown_connections 2024-03-08T21:01:56.975 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:56.990+0000 7fb4c2977740 1 --2- 172.21.15.107:0/763034538 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561bb8522570 0x561bb8524960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:01:56.975 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:56.990+0000 7fb4c2977740 1 --2- 172.21.15.107:0/763034538 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561bb85197a0 0x561bb8522030 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:01:56.976 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:56.990+0000 7fb4c2977740 1 --2- 172.21.15.107:0/763034538 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561bb8518e80 0x561bb8519260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:01:56.976 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:56.990+0000 7fb4c2977740 1 -- 172.21.15.107:0/763034538 >> 172.21.15.107:0/763034538 conn(0x561bb85137a0 msgr2=0x561bb8513b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:01:56.976 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:56.990+0000 7fb4c2977740 1 -- 172.21.15.107:0/763034538 shutdown_connections 2024-03-08T21:01:56.976 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:56.990+0000 7fb4c2977740 1 -- 172.21.15.107:0/763034538 wait complete. 2024-03-08T21:01:56.976 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:56.991+0000 7fb4c2977740 1 -- start start 2024-03-08T21:01:56.976 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:56.991+0000 7fb4c2977740 1 Processor -- start 2024-03-08T21:01:56.976 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:56.991+0000 7fb4c2977740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561bb8518e80 0x561bb8521e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:01:56.976 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:56.991+0000 7fb4c2977740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561bb85197a0 0x561bb851f220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:01:56.976 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:56.992+0000 7fb4c2977740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561bb8522570 0x561bb851f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:01:56.976 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:56.992+0000 7fb4c2977740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bb85251a0 con 0x561bb8522570 2024-03-08T21:01:56.976 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:56.992+0000 7fb4c2977740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bb84389f0 con 0x561bb8518e80 2024-03-08T21:01:56.976 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:56.992+0000 7fb4c2977740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bb8525320 con 0x561bb85197a0 2024-03-08T21:01:56.976 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:56.992+0000 7fb4c2977740 10 client.0 _ll_register_callbacks cb 0x561bb8190a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T21:01:56.976 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:56.992+0000 7fb4bdc00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561bb8522570 0x561bb851f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:01:56.977 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:56.992+0000 7fb4bdc00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561bb8522570 0x561bb851f760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:42998/0 (socket says 172.21.15.107:42998) 2024-03-08T21:01:56.977 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:56.992+0000 7fb4bdc00640 1 -- 172.21.15.107:0/703437527 learned_addr learned my addr 172.21.15.107:0/703437527 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:01:56.977 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:56.992+0000 7fb4bd200640 1 --2- 172.21.15.107:0/703437527 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561bb8518e80 0x561bb8521e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:01:56.977 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:56.992+0000 7fb4b7e00640 1 --2- 172.21.15.107:0/703437527 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561bb85197a0 0x561bb851f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:01:56.977 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:56.992+0000 7fb4bdc00640 1 -- 172.21.15.107:0/703437527 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561bb85197a0 msgr2=0x561bb851f220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:01:56.977 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:56.992+0000 7fb4bdc00640 1 --2- 172.21.15.107:0/703437527 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561bb85197a0 0x561bb851f220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:01:56.977 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:56.992+0000 7fb4bdc00640 1 -- 172.21.15.107:0/703437527 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561bb8518e80 msgr2=0x561bb8521e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:01:56.977 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:56.992+0000 7fb4bdc00640 1 --2- 172.21.15.107:0/703437527 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561bb8518e80 0x561bb8521e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:01:56.977 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:56.992+0000 7fb4bdc00640 1 -- 172.21.15.107:0/703437527 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561bb8191210 con 0x561bb8522570 2024-03-08T21:01:56.977 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:56.993+0000 7fb4bdc00640 1 --2- 172.21.15.107:0/703437527 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561bb8522570 0x561bb851f760 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fb4b0011b20 tx=0x7fb4b0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aee6db5430fa22e0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:01:56.977 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:56.993+0000 7fb49ea00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-03-08T21:01:56.977 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:56.993+0000 7fb49ea00640 1 -- 172.21.15.107:0/703437527 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4b001a450 con 0x561bb8522570 2024-03-08T21:01:56.977 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:56.993+0000 7fb49ea00640 1 -- 172.21.15.107:0/703437527 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fb4b001a5f0 con 0x561bb8522570 2024-03-08T21:01:56.977 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:56.993+0000 7fb4c2977740 1 -- 172.21.15.107:0/703437527 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb4ac017c70 con 0x561bb8522570 2024-03-08T21:01:56.978 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:56.993+0000 7fb49ea00640 1 -- 172.21.15.107:0/703437527 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb4b001a830 con 0x561bb8522570 2024-03-08T21:01:56.978 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:56.993+0000 7fb4c2977740 10 client.15579 fetch_fsmap learned FSMap version 30 2024-03-08T21:01:56.978 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:56.993+0000 7fb4c2977740 1 -- 172.21.15.107:0/703437527 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x561bb825d620 con 0x561bb8522570 2024-03-08T21:01:56.980 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:56.994+0000 7fb49ea00640 1 -- 172.21.15.107:0/703437527 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fb4ac017c70 con 0x561bb8522570 2024-03-08T21:01:56.980 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:56.994+0000 7fb49ea00640 1 -- 172.21.15.107:0/703437527 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 30) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fb4b0023070 con 0x561bb8522570 2024-03-08T21:01:56.980 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:56.994+0000 7fb4c2977740 10 client.15579 fetch_fsmap finished waiting for FSMap version 30 2024-03-08T21:01:56.980 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:56.994+0000 7fb4c2977740 10 client.15579 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-08T21:01:56.980 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:56.994+0000 7fb4c2977740 20 client.15579 populate_metadata read hostname 'smithi107' 2024-03-08T21:01:56.980 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:56.994+0000 7fb4c2977740 1 --2- 172.21.15.107:0/703437527 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x561bb85323d0 0x561bb85347c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:01:56.980 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:56.994+0000 7fb4c2977740 4 client.15579 mds_command: new command op to 34475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-08T21:01:56.980 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:56.994+0000 7fb4c2977740 1 -- 172.21.15.107:0/703437527 --> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561bb8489910 con 0x561bb85323d0 2024-03-08T21:01:56.980 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:56.995+0000 7fb4bd200640 1 --2- 172.21.15.107:0/703437527 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x561bb85323d0 0x561bb85347c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:01:56.980 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:56.995+0000 7fb49ea00640 10 client.15579 ms_handle_connect on v2:172.21.15.149:6838/55290596 2024-03-08T21:01:56.981 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:56.995+0000 7fb4bd200640 1 --2- 172.21.15.107:0/703437527 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x561bb85323d0 0x561bb85347c0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a035504c1782cb7e server_cookie=49bac961650c6ce9 in_seq=0 out_seq=0 2024-03-08T21:01:56.981 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:56.997+0000 7fb49ea00640 1 -- 172.21.15.107:0/703437527 <== mds.0 v2:172.21.15.149:6838/55290596 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x561bb8489910 con 0x561bb85323d0 2024-03-08T21:01:56.981 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:56.997+0000 7fb49ea00640 10 client.15579 handle_command_reply: tid=0 2024-03-08T21:01:56.981 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:56.997+0000 7fb4c2977740 1 client.15579 shutdown 2024-03-08T21:01:56.981 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:56.997+0000 7fb4c2977740 1 -- 172.21.15.107:0/703437527 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561bb8522570 msgr2=0x561bb851f760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:01:56.981 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:56.997+0000 7fb4c2977740 1 --2- 172.21.15.107:0/703437527 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561bb8522570 0x561bb851f760 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fb4b0011b20 tx=0x7fb4b0004a40 comp rx=0 tx=0).stop 2024-03-08T21:01:56.981 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:56.997+0000 7fb4c2977740 1 -- 172.21.15.107:0/703437527 shutdown_connections 2024-03-08T21:01:56.981 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:56.997+0000 7fb4c2977740 1 --2- 172.21.15.107:0/703437527 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561bb8522570 0x561bb851f760 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:01:56.981 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:56.997+0000 7fb4c2977740 1 --2- 172.21.15.107:0/703437527 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561bb85197a0 0x561bb851f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:01:56.981 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:56.997+0000 7fb4c2977740 1 --2- 172.21.15.107:0/703437527 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x561bb85323d0 0x561bb85347c0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:01:56.981 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:56.997+0000 7fb4c2977740 1 --2- 172.21.15.107:0/703437527 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561bb8518e80 0x561bb8521e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:01:56.981 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:56.997+0000 7fb49ea00640 0 client.15579 ms_handle_reset on v2:172.21.15.149:6838/55290596 2024-03-08T21:01:56.981 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:56.997+0000 7fb4c2977740 1 -- 172.21.15.107:0/703437527 >> 172.21.15.107:0/703437527 conn(0x561bb85137a0 msgr2=0x561bb85154f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:01:56.981 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:56.998+0000 7fb4c2977740 1 -- 172.21.15.107:0/703437527 shutdown_connections 2024-03-08T21:01:56.982 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:56.998+0000 7fb4c2977740 1 -- 172.21.15.107:0/703437527 wait complete. 2024-03-08T21:01:56.982 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:56.998+0000 7fb4c2977740 20 client.15579 trim_cache size 0 max 16384 2024-03-08T21:01:56.999 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:57.012+0000 7fb4c2977740 1 Processor -- start 2024-03-08T21:01:56.999 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:57.012+0000 7fb4c2977740 1 -- start start 2024-03-08T21:01:56.999 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:57.012+0000 7fb4c2977740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561bb8518e80 0x561bb8512680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:01:56.999 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:57.012+0000 7fb4c2977740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561bb85197a0 0x561bb8521950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:01:56.999 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:57.012+0000 7fb4c2977740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561bb8522570 0x561bb8521e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:01:56.999 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:57.012+0000 7fb4c2977740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4b81aed90 con 0x561bb8522570 2024-03-08T21:01:56.999 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:57.012+0000 7fb4c2977740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4b81b0710 con 0x561bb85197a0 2024-03-08T21:01:57.000 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:57.012+0000 7fb4c2977740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4b81af3f0 con 0x561bb8518e80 2024-03-08T21:01:57.000 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:57.013+0000 7fb4bdc00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561bb8522570 0x561bb8521e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:01:57.000 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:57.013+0000 7fb4bdc00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561bb8522570 0x561bb8521e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:43010/0 (socket says 172.21.15.107:43010) 2024-03-08T21:01:57.000 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:57.013+0000 7fb4bdc00640 1 -- 172.21.15.107:0/1978377568 learned_addr learned my addr 172.21.15.107:0/1978377568 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:01:57.000 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:57.013+0000 7fb4b7e00640 1 --2- 172.21.15.107:0/1978377568 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561bb85197a0 0x561bb8521950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:01:57.000 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:57.013+0000 7fb4bdc00640 1 -- 172.21.15.107:0/1978377568 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561bb8518e80 msgr2=0x561bb8512680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:01:57.000 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:57.013+0000 7fb4bdc00640 1 --2- 172.21.15.107:0/1978377568 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561bb8518e80 0x561bb8512680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:01:57.000 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:57.013+0000 7fb4bdc00640 1 -- 172.21.15.107:0/1978377568 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561bb85197a0 msgr2=0x561bb8521950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:01:57.000 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:57.013+0000 7fb4bdc00640 1 --2- 172.21.15.107:0/1978377568 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561bb85197a0 0x561bb8521950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:01:57.000 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:57.013+0000 7fb4bdc00640 1 -- 172.21.15.107:0/1978377568 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561bb85135d0 con 0x561bb8522570 2024-03-08T21:01:57.000 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:57.013+0000 7fb4b7e00640 1 --2- 172.21.15.107:0/1978377568 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561bb85197a0 0x561bb8521950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T21:01:57.000 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:57.013+0000 7fb4bdc00640 1 --2- 172.21.15.107:0/1978377568 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561bb8522570 0x561bb8521e90 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fb4b0025bb0 tx=0x7fb4b002d410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65802a7ceccbef7e server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:01:57.000 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:57.014+0000 7fb4a9600640 1 -- 172.21.15.107:0/1978377568 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4b001a450 con 0x561bb8522570 2024-03-08T21:01:57.001 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:57.014+0000 7fb4a9600640 1 -- 172.21.15.107:0/1978377568 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fb4b001a5f0 con 0x561bb8522570 2024-03-08T21:01:57.001 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:57.014+0000 7fb4a9600640 1 -- 172.21.15.107:0/1978377568 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4b002d500 con 0x561bb8522570 2024-03-08T21:01:57.001 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:57.014+0000 7fb4c2977740 1 -- 172.21.15.107:0/1978377568 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561bb8522570 msgr2=0x561bb8521e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:01:57.001 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:57.014+0000 7fb4c2977740 1 --2- 172.21.15.107:0/1978377568 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561bb8522570 0x561bb8521e90 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fb4b0025bb0 tx=0x7fb4b002d410 comp rx=0 tx=0).stop 2024-03-08T21:01:57.001 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:57.014+0000 7fb4c2977740 1 -- 172.21.15.107:0/1978377568 shutdown_connections 2024-03-08T21:01:57.001 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:57.014+0000 7fb4c2977740 1 --2- 172.21.15.107:0/1978377568 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561bb8522570 0x561bb8521e90 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:01:57.001 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:57.014+0000 7fb4c2977740 1 --2- 172.21.15.107:0/1978377568 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561bb85197a0 0x561bb8521950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:01:57.001 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:57.014+0000 7fb4c2977740 1 --2- 172.21.15.107:0/1978377568 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561bb8518e80 0x561bb8512680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:01:57.001 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:57.014+0000 7fb4c2977740 1 -- 172.21.15.107:0/1978377568 >> 172.21.15.107:0/1978377568 conn(0x561bb85137a0 msgr2=0x561bb8511f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:01:57.001 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:57.015+0000 7fb4c2977740 1 -- 172.21.15.107:0/1978377568 shutdown_connections 2024-03-08T21:01:57.001 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:57.015+0000 7fb4c2977740 1 -- 172.21.15.107:0/1978377568 wait complete. 2024-03-08T21:01:57.001 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:57.015+0000 7fb4c2977740 1 -- start start 2024-03-08T21:01:57.001 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:57.015+0000 7fb4c2977740 1 Processor -- start 2024-03-08T21:01:57.001 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:57.016+0000 7fb4c2977740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561bb8518e80 0x561bb85280b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:01:57.002 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:57.016+0000 7fb4c2977740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561bb85197a0 0x561bb85285f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:01:57.002 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:57.016+0000 7fb4c2977740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561bb8522570 0x561bb8528b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:01:57.002 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:57.016+0000 7fb4c2977740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4b81b00b0 con 0x561bb8518e80 2024-03-08T21:01:57.002 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:57.016+0000 7fb4c2977740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4b81afa50 con 0x561bb8522570 2024-03-08T21:01:57.002 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:57.016+0000 7fb4c2977740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4b81ad030 con 0x561bb85197a0 2024-03-08T21:01:57.002 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:57.016+0000 7fb4c2977740 10 client.0 _ll_register_callbacks cb 0x7fb4ac00b340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T21:01:57.002 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:57.016+0000 7fb4bd200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561bb8518e80 0x561bb85280b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:01:57.002 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:57.016+0000 7fb4bd200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561bb8518e80 0x561bb85280b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:43016/0 (socket says 172.21.15.107:43016) 2024-03-08T21:01:57.002 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:57.016+0000 7fb4bd200640 1 -- 172.21.15.107:0/2439569432 learned_addr learned my addr 172.21.15.107:0/2439569432 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:01:57.002 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:57.016+0000 7fb4b7e00640 1 --2- 172.21.15.107:0/2439569432 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561bb85197a0 0x561bb85285f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:01:57.002 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:57.016+0000 7fb4bd200640 1 -- 172.21.15.107:0/2439569432 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561bb85197a0 msgr2=0x561bb85285f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:01:57.002 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:57.016+0000 7fb4bdc00640 1 --2- 172.21.15.107:0/2439569432 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561bb8522570 0x561bb8528b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:01:57.002 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:57.016+0000 7fb4bd200640 1 --2- 172.21.15.107:0/2439569432 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561bb85197a0 0x561bb85285f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:01:57.003 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:57.016+0000 7fb4bd200640 1 -- 172.21.15.107:0/2439569432 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561bb8522570 msgr2=0x561bb8528b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:01:57.003 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:57.016+0000 7fb4bd200640 1 --2- 172.21.15.107:0/2439569432 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561bb8522570 0x561bb8528b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:01:57.003 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:57.016+0000 7fb4bd200640 1 -- 172.21.15.107:0/2439569432 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561bb8588a20 con 0x561bb8518e80 2024-03-08T21:01:57.003 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:57.016+0000 7fb4b7e00640 1 --2- 172.21.15.107:0/2439569432 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561bb85197a0 0x561bb85285f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T21:01:57.003 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:57.017+0000 7fb4bd200640 1 --2- 172.21.15.107:0/2439569432 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561bb8518e80 0x561bb85280b0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fb4ac019040 tx=0x7fb4ac00b410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4ff1c601d46beb8 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:01:57.003 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:57.017+0000 7fb49ea00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-03-08T21:01:57.003 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:57.017+0000 7fb49ea00640 1 -- 172.21.15.107:0/2439569432 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4ac026d00 con 0x561bb8518e80 2024-03-08T21:01:57.003 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:57.017+0000 7fb4c2977740 1 -- 172.21.15.107:0/2439569432 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb4b001a5f0 con 0x561bb8518e80 2024-03-08T21:01:57.003 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:57.017+0000 7fb49ea00640 1 -- 172.21.15.107:0/2439569432 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fb4ac0262c0 con 0x561bb8518e80 2024-03-08T21:01:57.003 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:57.017+0000 7fb49ea00640 1 -- 172.21.15.107:0/2439569432 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb4ac0264c0 con 0x561bb8518e80 2024-03-08T21:01:57.003 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:57.017+0000 7fb4c2977740 10 client.15585 fetch_fsmap learned FSMap version 30 2024-03-08T21:01:57.003 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:57.017+0000 7fb4c2977740 1 -- 172.21.15.107:0/2439569432 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x561bb8589d00 con 0x561bb8518e80 2024-03-08T21:01:57.003 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:57.017+0000 7fb49ea00640 1 -- 172.21.15.107:0/2439569432 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fb4b001a5f0 con 0x561bb8518e80 2024-03-08T21:01:57.003 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:57.018+0000 7fb49ea00640 1 -- 172.21.15.107:0/2439569432 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 30) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fb4ac02d030 con 0x561bb8518e80 2024-03-08T21:01:57.003 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:57.018+0000 7fb4c2977740 10 client.15585 fetch_fsmap finished waiting for FSMap version 30 2024-03-08T21:01:57.004 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:57.018+0000 7fb4c2977740 10 client.15585 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-08T21:01:57.004 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:57.018+0000 7fb4c2977740 20 client.15585 populate_metadata read hostname 'smithi107' 2024-03-08T21:01:57.004 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:57.018+0000 7fb4c2977740 1 --2- 172.21.15.107:0/2439569432 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x561bb858a180 0x561bb858c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:01:57.004 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:57.018+0000 7fb4c2977740 4 client.15585 mds_command: new command op to 34475 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-08T21:01:57.004 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:57.018+0000 7fb4c2977740 1 -- 172.21.15.107:0/2439569432 --> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x561bb8483c60 con 0x561bb858a180 2024-03-08T21:01:57.004 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:57.018+0000 7fb4b7e00640 1 --2- 172.21.15.107:0/2439569432 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x561bb858a180 0x561bb858c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:01:57.004 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:57.019+0000 7fb49ea00640 10 client.15585 ms_handle_connect on v2:172.21.15.149:6838/55290596 2024-03-08T21:01:57.004 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:57.019+0000 7fb4b7e00640 1 --2- 172.21.15.107:0/2439569432 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x561bb858a180 0x561bb858c5a0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d3ef08b8949258b6 server_cookie=7c79f2fae298757b in_seq=0 out_seq=0 2024-03-08T21:01:57.011 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:57.029+0000 7fb49ea00640 1 -- 172.21.15.107:0/2439569432 <== mds.0 v2:172.21.15.149:6838/55290596 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x561bb8483c60 con 0x561bb858a180 2024-03-08T21:01:57.011 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:57.029+0000 7fb49ea00640 10 client.15585 handle_command_reply: tid=0 2024-03-08T21:01:57.011 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:57.029+0000 7fb4c2977740 1 client.15585 shutdown 2024-03-08T21:01:57.011 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:57.029+0000 7fb4c2977740 1 -- 172.21.15.107:0/2439569432 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561bb8518e80 msgr2=0x561bb85280b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:01:57.011 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:57.029+0000 7fb4c2977740 1 --2- 172.21.15.107:0/2439569432 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561bb8518e80 0x561bb85280b0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fb4ac019040 tx=0x7fb4ac00b410 comp rx=0 tx=0).stop 2024-03-08T21:01:57.011 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:57.029+0000 7fb4c2977740 1 -- 172.21.15.107:0/2439569432 shutdown_connections 2024-03-08T21:01:57.011 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:57.029+0000 7fb4c2977740 1 --2- 172.21.15.107:0/2439569432 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x561bb858a180 0x561bb858c5a0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:01:57.011 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:57.029+0000 7fb4c2977740 1 --2- 172.21.15.107:0/2439569432 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561bb8522570 0x561bb8528b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:01:57.011 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:57.029+0000 7fb4c2977740 1 --2- 172.21.15.107:0/2439569432 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561bb85197a0 0x561bb85285f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:01:57.011 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:57.029+0000 7fb49ea00640 0 client.15585 ms_handle_reset on v2:172.21.15.149:6838/55290596 2024-03-08T21:01:57.012 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:57.030+0000 7fb4c2977740 1 --2- 172.21.15.107:0/2439569432 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561bb8518e80 0x561bb85280b0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:01:57.012 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:57.030+0000 7fb4c2977740 1 -- 172.21.15.107:0/2439569432 >> 172.21.15.107:0/2439569432 conn(0x561bb85137a0 msgr2=0x561bb851d6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:01:57.012 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:57.030+0000 7fb4c2977740 1 -- 172.21.15.107:0/2439569432 shutdown_connections 2024-03-08T21:01:57.012 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:57.030+0000 7fb4c2977740 1 -- 172.21.15.107:0/2439569432 wait complete. 2024-03-08T21:01:57.012 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:57.030+0000 7fb4c2977740 20 client.15585 trim_cache size 0 max 16384 2024-03-08T21:01:57.012 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:{ 2024-03-08T21:01:57.012 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "message": "", 2024-03-08T21:01:57.012 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "return_code": 0 2024-03-08T21:01:57.012 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:} 2024-03-08T21:01:57.013 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:57.030+0000 7fb4be600640 1 -- 172.21.15.107:0/3134550705 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7fb4a00a2b00 msgr2=0x7fb4a00a4fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:01:57.013 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:57.030+0000 7fb4be600640 1 --2- 172.21.15.107:0/3134550705 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7fb4a00a2b00 0x7fb4a00a4fc0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fb4b000f4f0 tx=0x7fb4b0004b50 comp rx=0 tx=0).stop 2024-03-08T21:01:57.013 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:57.030+0000 7fb4be600640 1 -- 172.21.15.107:0/3134550705 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb4b8074040 msgr2=0x7fb4b819fd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:01:57.013 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:57.031+0000 7fb4be600640 1 --2- 172.21.15.107:0/3134550705 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb4b8074040 0x7fb4b819fd10 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fb4a4005ab0 tx=0x7fb4a402db40 comp rx=0 tx=0).stop 2024-03-08T21:01:57.013 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:57.031+0000 7fb4be600640 1 -- 172.21.15.107:0/3134550705 shutdown_connections 2024-03-08T21:01:57.013 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:57.031+0000 7fb4be600640 1 --2- 172.21.15.107:0/3134550705 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7fb4a00a2b00 0x7fb4a00a4fc0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:01:57.013 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:57.031+0000 7fb4be600640 1 --2- 172.21.15.107:0/3134550705 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb4b810d910 0x7fb4b81999d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:01:57.014 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:57.031+0000 7fb4be600640 1 --2- 172.21.15.107:0/3134550705 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb4b8074040 0x7fb4b819fd10 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:01:57.014 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:57.031+0000 7fb4be600640 1 --2- 172.21.15.107:0/3134550705 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb4b8073720 0x7fb4b819f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:01:57.014 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:57.031+0000 7fb4be600640 1 -- 172.21.15.107:0/3134550705 >> 172.21.15.107:0/3134550705 conn(0x7fb4b80fcc40 msgr2=0x7fb4b80fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:01:57.014 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:57.031+0000 7fb4be600640 1 -- 172.21.15.107:0/3134550705 shutdown_connections 2024-03-08T21:01:57.014 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:01:57.031+0000 7fb4be600640 1 -- 172.21.15.107:0/3134550705 wait complete. 2024-03-08T21:01:57.877 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:01:57 smithi204 ceph-mon[34935]: pgmap v46: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:01:57.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:01:57 smithi107 ceph-mon[31381]: pgmap v46: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:01:58.013 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:01:57 smithi149 ceph-mon[35042]: pgmap v46: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:01:58.362 INFO:tasks.workunit.client.0.smithi107.stderr:Updating files: 63% (7820/12271) Updating files: 64% (7854/12271) Updating files: 65% (7977/12271) Updating files: 66% (8099/12271) Updating files: 67% (8222/12271) Updating files: 68% (8345/12271) Updating files: 69% (8467/12271) Updating files: 70% (8590/12271) Updating files: 71% (8713/12271) Updating files: 72% (8836/12271) Updating files: 73% (8958/12271) Updating files: 74% (9081/12271) Updating files: 75% (9204/12271) Updating files: 76% (9326/12271) Updating files: 77% (9449/12271) Updating files: 78% (9572/12271) Updating files: 79% (9695/12271) Updating files: 80% (9817/12271) Updating files: 81% (9940/12271) Updating files: 82% (10063/12271) Updating files: 83% (10185/12271) Updating files: 84% (10308/12271) Updating files: 85% (10431/12271) Updating files: 86% (10554/12271) Updating files: 87% (10676/12271) Updating files: 88% (10799/12271) Updating files: 89% (10922/12271) Updating files: 90% (11044/12271) Updating files: 91% (11167/12271) Updating files: 92% (11290/12271) Updating files: 93% (11413/12271) Updating files: 94% (11535/12271) Updating files: 95% (11658/12271) Updating files: 96% (11781/12271) Updating files: 97% (11903/12271) Updating files: 98% (12026/12271) Updating files: 99% (12149/12271) Updating files: 100% (12271/12271) Updating files: 100% (12271/12271), done. 2024-03-08T21:01:58.897 INFO:tasks.workunit.client.0.smithi107.stderr:Note: switching to '7d63482aa89628d733e298984a762389b625c761'. 2024-03-08T21:01:58.898 INFO:tasks.workunit.client.0.smithi107.stderr: 2024-03-08T21:01:58.898 INFO:tasks.workunit.client.0.smithi107.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-03-08T21:01:58.898 INFO:tasks.workunit.client.0.smithi107.stderr:changes and commit them, and you can discard any commits you make in this 2024-03-08T21:01:58.898 INFO:tasks.workunit.client.0.smithi107.stderr:state without impacting any branches by switching back to a branch. 2024-03-08T21:01:58.898 INFO:tasks.workunit.client.0.smithi107.stderr: 2024-03-08T21:01:58.898 INFO:tasks.workunit.client.0.smithi107.stderr:If you want to create a new branch to retain commits you create, you may 2024-03-08T21:01:58.898 INFO:tasks.workunit.client.0.smithi107.stderr:do so (now or later) by using -c with the switch command. Example: 2024-03-08T21:01:58.898 INFO:tasks.workunit.client.0.smithi107.stderr: 2024-03-08T21:01:58.898 INFO:tasks.workunit.client.0.smithi107.stderr: git switch -c 2024-03-08T21:01:58.898 INFO:tasks.workunit.client.0.smithi107.stderr: 2024-03-08T21:01:58.898 INFO:tasks.workunit.client.0.smithi107.stderr:Or undo this operation with: 2024-03-08T21:01:58.898 INFO:tasks.workunit.client.0.smithi107.stderr: 2024-03-08T21:01:58.899 INFO:tasks.workunit.client.0.smithi107.stderr: git switch - 2024-03-08T21:01:58.899 INFO:tasks.workunit.client.0.smithi107.stderr: 2024-03-08T21:01:58.899 INFO:tasks.workunit.client.0.smithi107.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-03-08T21:01:58.899 INFO:tasks.workunit.client.0.smithi107.stderr: 2024-03-08T21:01:58.899 INFO:tasks.workunit.client.0.smithi107.stderr:HEAD is now at 7d63482aa89 Merge remote-tracking branch 'rishabh/fsshell-deprecation' into rishabh-2024mar8 2024-03-08T21:01:58.907 DEBUG:teuthology.orchestra.run.smithi107:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-03-08T21:01:59.010 INFO:tasks.workunit.client.0.smithi107.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-03-08T21:01:59.017 INFO:tasks.workunit.client.0.smithi107.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-03-08T21:01:59.017 INFO:tasks.workunit.client.0.smithi107.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-03-08T21:01:59.877 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:01:59 smithi204 ceph-mon[34935]: pgmap v47: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:01:59.947 INFO:tasks.workunit.client.0.smithi107.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-03-08T21:01:59.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:01:59 smithi107 ceph-mon[31381]: pgmap v47: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:02:00.013 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:01:59 smithi149 ceph-mon[35042]: pgmap v47: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:02:00.066 INFO:tasks.workunit.client.0.smithi107.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-03-08T21:02:00.146 INFO:tasks.workunit.client.0.smithi107.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-03-08T21:02:00.152 INFO:tasks.workunit.client.0.smithi107.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-03-08T21:02:00.152 INFO:tasks.workunit.client.0.smithi107.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-03-08T21:02:00.235 INFO:tasks.workunit.client.0.smithi107.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-03-08T21:02:00.245 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-03-08T21:02:00.245 DEBUG:teuthology.orchestra.run.smithi107:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-03-08T21:02:00.293 INFO:tasks.workunit:Running workunits matching direct_io on client.0... 2024-03-08T21:02:00.294 INFO:tasks.workunit:Running workunit direct_io/big.sh... 2024-03-08T21:02:00.295 DEBUG:teuthology.orchestra.run.smithi107:workunit test direct_io/big.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=7d63482aa89628d733e298984a762389b625c761 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io/big.sh 2024-03-08T21:02:00.425 INFO:tasks.workunit.client.0.smithi107.stdout:test large (16MB) dio write 2024-03-08T21:02:00.425 INFO:tasks.workunit.client.0.smithi107.stderr:+ echo 'test large (16MB) dio write' 2024-03-08T21:02:00.425 INFO:tasks.workunit.client.0.smithi107.stderr:+ dd if=/dev/zero of=foo.big bs=16M count=1 oflag=direct 2024-03-08T21:02:01.189 INFO:tasks.workunit.client.0.smithi107.stderr:1+0 records in 2024-03-08T21:02:01.189 INFO:tasks.workunit.client.0.smithi107.stderr:1+0 records out 2024-03-08T21:02:01.189 INFO:tasks.workunit.client.0.smithi107.stderr:16777216 bytes (17 MB, 16 MiB) copied, 0.754373 s, 22.2 MB/s 2024-03-08T21:02:01.194 INFO:tasks.workunit.client.0.smithi107.stdout:OK 2024-03-08T21:02:01.194 INFO:tasks.workunit.client.0.smithi107.stderr:+ echo OK 2024-03-08T21:02:01.196 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-03-08T21:02:01.196 DEBUG:teuthology.orchestra.run.smithi107:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-03-08T21:02:01.339 INFO:tasks.workunit:Running workunit direct_io/direct_io_test... 2024-03-08T21:02:01.339 DEBUG:teuthology.orchestra.run.smithi107:workunit test direct_io/direct_io_test> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=7d63482aa89628d733e298984a762389b625c761 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io/direct_io_test 2024-03-08T21:02:01.877 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:02:01 smithi204 ceph-mon[34935]: pgmap v48: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 511 B/s wr, 13 op/s 2024-03-08T21:02:01.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:02:01 smithi107 ceph-mon[31381]: pgmap v48: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 511 B/s wr, 13 op/s 2024-03-08T21:02:02.013 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:02:01 smithi149 ceph-mon[35042]: pgmap v48: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 511 B/s wr, 13 op/s 2024-03-08T21:02:03.877 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:02:03 smithi204 ceph-mon[34935]: pgmap v49: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:02:03.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:02:03 smithi107 ceph-mon[31381]: pgmap v49: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:02:04.013 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:02:03 smithi149 ceph-mon[35042]: pgmap v49: 97 pgs: 97 active+clean; 1.3 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:02:05.877 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:02:05 smithi204 ceph-mon[34935]: pgmap v50: 97 pgs: 97 active+clean; 1.4 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 1.4 MiB/s wr, 40 op/s 2024-03-08T21:02:05.877 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:02:05 smithi204 ceph-mon[34935]: mgrmap e25: x(active, since 92s), standbys: y, z 2024-03-08T21:02:05.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:02:05 smithi107 ceph-mon[31381]: pgmap v50: 97 pgs: 97 active+clean; 1.4 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 1.4 MiB/s wr, 40 op/s 2024-03-08T21:02:05.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:02:05 smithi107 ceph-mon[31381]: mgrmap e25: x(active, since 92s), standbys: y, z 2024-03-08T21:02:06.013 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:02:05 smithi149 ceph-mon[35042]: pgmap v50: 97 pgs: 97 active+clean; 1.4 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 1.4 MiB/s wr, 40 op/s 2024-03-08T21:02:06.013 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:02:05 smithi149 ceph-mon[35042]: mgrmap e25: x(active, since 92s), standbys: y, z 2024-03-08T21:02:07.877 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:02:07 smithi204 ceph-mon[34935]: pgmap v51: 97 pgs: 97 active+clean; 1.4 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 1.4 MiB/s wr, 36 op/s 2024-03-08T21:02:07.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:02:07 smithi107 ceph-mon[31381]: pgmap v51: 97 pgs: 97 active+clean; 1.4 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 1.4 MiB/s wr, 36 op/s 2024-03-08T21:02:08.013 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:02:07 smithi149 ceph-mon[35042]: pgmap v51: 97 pgs: 97 active+clean; 1.4 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 1.4 MiB/s wr, 36 op/s 2024-03-08T21:02:09.878 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:02:09 smithi204 ceph-mon[34935]: pgmap v52: 97 pgs: 97 active+clean; 1.4 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 1.4 MiB/s wr, 36 op/s 2024-03-08T21:02:09.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:02:09 smithi107 ceph-mon[31381]: pgmap v52: 97 pgs: 97 active+clean; 1.4 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 1.4 MiB/s wr, 36 op/s 2024-03-08T21:02:10.013 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:02:09 smithi149 ceph-mon[35042]: pgmap v52: 97 pgs: 97 active+clean; 1.4 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 1.4 MiB/s wr, 36 op/s 2024-03-08T21:02:10.156 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.173+0000 7fb7ce200640 1 -- 172.21.15.107:0/3707988751 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb7c8101340 msgr2=0x7fb7c8101720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:02:10.156 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.173+0000 7fb7ce200640 1 --2- 172.21.15.107:0/3707988751 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb7c8101340 0x7fb7c8101720 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fb7bc009900 tx=0x7fb7bc02bc30 comp rx=0 tx=0).stop 2024-03-08T21:02:10.156 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.174+0000 7fb7ce200640 1 -- 172.21.15.107:0/3707988751 shutdown_connections 2024-03-08T21:02:10.156 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.174+0000 7fb7ce200640 1 --2- 172.21.15.107:0/3707988751 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb7c8106410 0x7fb7c810dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:02:10.156 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.174+0000 7fb7ce200640 1 --2- 172.21.15.107:0/3707988751 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb7c8101cf0 0x7fb7c8105d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:02:10.156 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.174+0000 7fb7ce200640 1 --2- 172.21.15.107:0/3707988751 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb7c8101340 0x7fb7c8101720 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:02:10.157 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.174+0000 7fb7ce200640 1 -- 172.21.15.107:0/3707988751 >> 172.21.15.107:0/3707988751 conn(0x7fb7c80fcc20 msgr2=0x7fb7c80ff040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:02:10.157 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.174+0000 7fb7ce200640 1 -- 172.21.15.107:0/3707988751 shutdown_connections 2024-03-08T21:02:10.157 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.174+0000 7fb7ce200640 1 -- 172.21.15.107:0/3707988751 wait complete. 2024-03-08T21:02:10.159 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.175+0000 7fb7ce200640 1 Processor -- start 2024-03-08T21:02:10.159 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.175+0000 7fb7ce200640 1 -- start start 2024-03-08T21:02:10.159 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.176+0000 7fb7ce200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb7c8101340 0x7fb7c819f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:02:10.160 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.176+0000 7fb7ce200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb7c8106410 0x7fb7c819fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:02:10.160 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.176+0000 7fb7ce200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb7c81999c0 0x7fb7c8199e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:02:10.160 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.176+0000 7fb7ce200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7c81113a0 con 0x7fb7c8106410 2024-03-08T21:02:10.160 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.176+0000 7fb7ce200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7c8111220 con 0x7fb7c8101340 2024-03-08T21:02:10.160 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.176+0000 7fb7ce200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7c8111520 con 0x7fb7c81999c0 2024-03-08T21:02:10.160 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.176+0000 7fb7c7e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb7c8106410 0x7fb7c819fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:02:10.160 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.176+0000 7fb7c7e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb7c8106410 0x7fb7c819fd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:37524/0 (socket says 172.21.15.107:37524) 2024-03-08T21:02:10.160 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.176+0000 7fb7c7e00640 1 -- 172.21.15.107:0/4216652702 learned_addr learned my addr 172.21.15.107:0/4216652702 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:02:10.160 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.176+0000 7fb7cce00640 1 --2- 172.21.15.107:0/4216652702 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb7c8101340 0x7fb7c819f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:02:10.160 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.176+0000 7fb7cd800640 1 --2- 172.21.15.107:0/4216652702 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb7c81999c0 0x7fb7c8199e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:02:10.161 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.176+0000 7fb7c7e00640 1 -- 172.21.15.107:0/4216652702 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb7c81999c0 msgr2=0x7fb7c8199e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:02:10.161 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.176+0000 7fb7c7e00640 1 --2- 172.21.15.107:0/4216652702 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb7c81999c0 0x7fb7c8199e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:02:10.161 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.176+0000 7fb7c7e00640 1 -- 172.21.15.107:0/4216652702 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb7c8101340 msgr2=0x7fb7c819f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:02:10.161 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.176+0000 7fb7c7e00640 1 --2- 172.21.15.107:0/4216652702 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb7c8101340 0x7fb7c819f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:02:10.161 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.176+0000 7fb7c7e00640 1 -- 172.21.15.107:0/4216652702 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7c819a4e0 con 0x7fb7c8106410 2024-03-08T21:02:10.161 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.177+0000 7fb7cce00640 1 --2- 172.21.15.107:0/4216652702 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb7c8101340 0x7fb7c819f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T21:02:10.161 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.177+0000 7fb7c7e00640 1 --2- 172.21.15.107:0/4216652702 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb7c8106410 0x7fb7c819fd00 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fb7b400d770 tx=0x7fb7b400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:02:10.161 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.177+0000 7fb7c5600640 1 -- 172.21.15.107:0/4216652702 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7b4017070 con 0x7fb7c8106410 2024-03-08T21:02:10.161 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.177+0000 7fb7ce200640 1 -- 172.21.15.107:0/4216652702 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7c819a7d0 con 0x7fb7c8106410 2024-03-08T21:02:10.161 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.177+0000 7fb7c5600640 1 -- 172.21.15.107:0/4216652702 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fb7b4009c30 con 0x7fb7c8106410 2024-03-08T21:02:10.161 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.177+0000 7fb7c5600640 1 -- 172.21.15.107:0/4216652702 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7b4021060 con 0x7fb7c8106410 2024-03-08T21:02:10.161 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.177+0000 7fb7ce200640 1 -- 172.21.15.107:0/4216652702 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7c81a69d0 con 0x7fb7c8106410 2024-03-08T21:02:10.163 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.181+0000 7fb7d2404740 1 Processor -- start 2024-03-08T21:02:10.163 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.181+0000 7fb7d2404740 1 -- start start 2024-03-08T21:02:10.164 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.181+0000 7fb7c5600640 1 -- 172.21.15.107:0/4216652702 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) v1 ==== 137422+0+0 (secure 0 0 0) 0x7fb7b401a070 con 0x7fb7c8106410 2024-03-08T21:02:10.164 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.181+0000 7fb7d2404740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56369031ee80 0x56369031f260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:02:10.164 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.181+0000 7fb7d2404740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56369031f7a0 0x563690328030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:02:10.164 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.181+0000 7fb7d2404740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563690328570 0x56369032a960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:02:10.164 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.181+0000 7fb7d2404740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636902823d0 con 0x56369031f7a0 2024-03-08T21:02:10.164 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.181+0000 7fb7d2404740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x56369022b6b0 con 0x56369031ee80 2024-03-08T21:02:10.164 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.181+0000 7fb7d2404740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56369027e470 con 0x563690328570 2024-03-08T21:02:10.164 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.182+0000 7fb7cd800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56369031ee80 0x56369031f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:02:10.164 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.182+0000 7fb7c7e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563690328570 0x56369032a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:02:10.165 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.182+0000 7fb7cd800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56369031ee80 0x56369031f260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.107:53316/0 (socket says 172.21.15.107:53316) 2024-03-08T21:02:10.165 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.182+0000 7fb7cd800640 1 -- 172.21.15.107:0/3836592669 learned_addr learned my addr 172.21.15.107:0/3836592669 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:02:10.165 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.182+0000 7fb7cce00640 1 --2- 172.21.15.107:0/3836592669 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56369031f7a0 0x563690328030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:02:10.166 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.182+0000 7fb7cd800640 1 -- 172.21.15.107:0/3836592669 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563690328570 msgr2=0x56369032a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:02:10.166 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.182+0000 7fb7cd800640 1 --2- 172.21.15.107:0/3836592669 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563690328570 0x56369032a960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:02:10.166 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.182+0000 7fb7cd800640 1 -- 172.21.15.107:0/3836592669 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56369031f7a0 msgr2=0x563690328030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:02:10.166 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.182+0000 7fb7cd800640 1 --2- 172.21.15.107:0/3836592669 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56369031f7a0 0x563690328030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:02:10.166 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.183+0000 7fb7cd800640 1 -- 172.21.15.107:0/3836592669 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563690251b50 con 0x56369031ee80 2024-03-08T21:02:10.166 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.183+0000 7fb7cce00640 1 --2- 172.21.15.107:0/3836592669 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56369031f7a0 0x563690328030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T21:02:10.166 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.183+0000 7fb7c5600640 1 --2- 172.21.15.107:0/4216652702 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7fb7b00a2960 0x7fb7b00a4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:02:10.166 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.183+0000 7fb7c5600640 1 -- 172.21.15.107:0/4216652702 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fb7b40ceb60 con 0x7fb7c8106410 2024-03-08T21:02:10.167 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.183+0000 7fb7cd800640 1 --2- 172.21.15.107:0/3836592669 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56369031ee80 0x56369031f260 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fb7c0009e60 tx=0x7fb7c0010740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ae6394dfddfa68d2 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:02:10.167 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.183+0000 7fb7cce00640 1 --2- 172.21.15.107:0/4216652702 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7fb7b00a2960 0x7fb7b00a4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:02:10.167 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.183+0000 7fb7ba000640 1 -- 172.21.15.107:0/3836592669 <== mon.2 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7c0016070 con 0x56369031ee80 2024-03-08T21:02:10.167 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.184+0000 7fb7ba000640 1 -- 172.21.15.107:0/3836592669 <== mon.2 v2:172.21.15.149:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fb7c0004d20 con 0x56369031ee80 2024-03-08T21:02:10.167 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.184+0000 7fb7ba000640 1 -- 172.21.15.107:0/3836592669 <== mon.2 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7c00126b0 con 0x56369031ee80 2024-03-08T21:02:10.170 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.185+0000 7fb7cce00640 1 --2- 172.21.15.107:0/4216652702 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7fb7b00a2960 0x7fb7b00a4e20 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fb7c8107b70 tx=0x7fb7bc005b50 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:02:10.171 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.184+0000 7fb7d2404740 1 -- 172.21.15.107:0/3836592669 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56369031ee80 msgr2=0x56369031f260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:02:10.171 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.185+0000 7fb7d2404740 1 --2- 172.21.15.107:0/3836592669 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56369031ee80 0x56369031f260 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fb7c0009e60 tx=0x7fb7c0010740 comp rx=0 tx=0).stop 2024-03-08T21:02:10.171 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.185+0000 7fb7d2404740 1 -- 172.21.15.107:0/3836592669 shutdown_connections 2024-03-08T21:02:10.171 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.185+0000 7fb7d2404740 1 --2- 172.21.15.107:0/3836592669 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563690328570 0x56369032a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:02:10.171 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.185+0000 7fb7d2404740 1 --2- 172.21.15.107:0/3836592669 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56369031f7a0 0x563690328030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:02:10.171 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.185+0000 7fb7d2404740 1 --2- 172.21.15.107:0/3836592669 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56369031ee80 0x56369031f260 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:02:10.171 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.185+0000 7fb7d2404740 1 -- 172.21.15.107:0/3836592669 >> 172.21.15.107:0/3836592669 conn(0x5636903197a0 msgr2=0x563690319b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:02:10.171 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.185+0000 7fb7d2404740 1 -- 172.21.15.107:0/3836592669 shutdown_connections 2024-03-08T21:02:10.171 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.185+0000 7fb7d2404740 1 -- 172.21.15.107:0/3836592669 wait complete. 2024-03-08T21:02:10.171 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.185+0000 7fb7d2404740 1 -- start start 2024-03-08T21:02:10.171 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.185+0000 7fb7d2404740 1 Processor -- start 2024-03-08T21:02:10.171 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.186+0000 7fb7d2404740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56369031ee80 0x563690327f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:02:10.172 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.186+0000 7fb7d2404740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56369031f7a0 0x563690325310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:02:10.172 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.186+0000 7fb7d2404740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563690328570 0x563690325920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:02:10.172 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.186+0000 7fb7d2404740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x56369032b1a0 con 0x56369031f7a0 2024-03-08T21:02:10.172 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.186+0000 7fb7d2404740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x56369023e9f0 con 0x56369031ee80 2024-03-08T21:02:10.172 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.186+0000 7fb7d2404740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56369032b320 con 0x563690328570 2024-03-08T21:02:10.172 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.186+0000 7fb7d2404740 10 client.0 _ll_register_callbacks cb 0x56368ff96a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T21:02:10.172 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.186+0000 7fb7c7e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56369031f7a0 0x563690325310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:02:10.172 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.186+0000 7fb7c7e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56369031f7a0 0x563690325310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:37532/0 (socket says 172.21.15.107:37532) 2024-03-08T21:02:10.172 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.186+0000 7fb7c7e00640 1 -- 172.21.15.107:0/164868905 learned_addr learned my addr 172.21.15.107:0/164868905 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:02:10.172 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.186+0000 7fb7cd800640 1 --2- 172.21.15.107:0/164868905 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563690328570 0x563690325920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:02:10.172 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.187+0000 7fb7cce00640 1 --2- 172.21.15.107:0/164868905 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56369031ee80 0x563690327f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:02:10.172 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.187+0000 7fb7c7e00640 1 -- 172.21.15.107:0/164868905 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563690328570 msgr2=0x563690325920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:02:10.172 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.187+0000 7fb7c7e00640 1 --2- 172.21.15.107:0/164868905 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563690328570 0x563690325920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:02:10.173 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.187+0000 7fb7c7e00640 1 -- 172.21.15.107:0/164868905 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56369031ee80 msgr2=0x563690327f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:02:10.173 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.187+0000 7fb7c7e00640 1 --2- 172.21.15.107:0/164868905 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56369031ee80 0x563690327f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:02:10.173 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.187+0000 7fb7c7e00640 1 -- 172.21.15.107:0/164868905 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56368ff97210 con 0x56369031f7a0 2024-03-08T21:02:10.173 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.187+0000 7fb7cce00640 1 --2- 172.21.15.107:0/164868905 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56369031ee80 0x563690327f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T21:02:10.173 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.187+0000 7fb7c7e00640 1 --2- 172.21.15.107:0/164868905 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56369031f7a0 0x563690325310 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fb7b409aeb0 tx=0x7fb7b400b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd857f8eb8e27ce server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:02:10.173 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.187+0000 7fb7aea00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-03-08T21:02:10.173 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.187+0000 7fb7aea00640 1 -- 172.21.15.107:0/164868905 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7b409acf0 con 0x56369031f7a0 2024-03-08T21:02:10.173 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.187+0000 7fb7aea00640 1 -- 172.21.15.107:0/164868905 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fb7b4097820 con 0x56369031f7a0 2024-03-08T21:02:10.173 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.187+0000 7fb7d2404740 1 -- 172.21.15.107:0/164868905 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb7c0004d20 con 0x56369031f7a0 2024-03-08T21:02:10.173 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.187+0000 7fb7aea00640 1 -- 172.21.15.107:0/164868905 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb7b409ab10 con 0x56369031f7a0 2024-03-08T21:02:10.173 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.187+0000 7fb7d2404740 10 client.15597 fetch_fsmap learned FSMap version 30 2024-03-08T21:02:10.173 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.187+0000 7fb7d2404740 1 -- 172.21.15.107:0/164868905 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x563690063620 con 0x56369031f7a0 2024-03-08T21:02:10.173 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.188+0000 7fb7aea00640 1 -- 172.21.15.107:0/164868905 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fb7c0004d20 con 0x56369031f7a0 2024-03-08T21:02:10.173 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.188+0000 7fb7aea00640 1 -- 172.21.15.107:0/164868905 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 30) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fb7b4096f00 con 0x56369031f7a0 2024-03-08T21:02:10.173 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.188+0000 7fb7d2404740 10 client.15597 fetch_fsmap finished waiting for FSMap version 30 2024-03-08T21:02:10.174 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.188+0000 7fb7d2404740 10 client.15597 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-08T21:02:10.174 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.188+0000 7fb7d2404740 20 client.15597 populate_metadata read hostname 'smithi107' 2024-03-08T21:02:10.174 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.188+0000 7fb7d2404740 1 --2- 172.21.15.107:0/164868905 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x563690338490 0x56369033a880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:02:10.174 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.188+0000 7fb7d2404740 4 client.15597 mds_command: new command op to 34475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-08T21:02:10.174 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.188+0000 7fb7d2404740 1 -- 172.21.15.107:0/164868905 --> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56369028f910 con 0x563690338490 2024-03-08T21:02:10.174 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.189+0000 7fb7cce00640 1 --2- 172.21.15.107:0/164868905 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x563690338490 0x56369033a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:02:10.175 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.190+0000 7fb7cce00640 1 --2- 172.21.15.107:0/164868905 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x563690338490 0x56369033a880 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e1a563785bb8e247 server_cookie=53e5df8de1d9a5b4 in_seq=0 out_seq=0 2024-03-08T21:02:10.175 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.190+0000 7fb7aea00640 10 client.15597 ms_handle_connect on v2:172.21.15.149:6838/55290596 2024-03-08T21:02:10.175 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.191+0000 7fb7aea00640 1 -- 172.21.15.107:0/164868905 <== mds.0 v2:172.21.15.149:6838/55290596 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56369028f910 con 0x563690338490 2024-03-08T21:02:10.175 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.192+0000 7fb7aea00640 10 client.15597 handle_command_reply: tid=0 2024-03-08T21:02:10.175 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.192+0000 7fb7d2404740 1 client.15597 shutdown 2024-03-08T21:02:10.175 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.192+0000 7fb7d2404740 1 -- 172.21.15.107:0/164868905 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56369031f7a0 msgr2=0x563690325310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:02:10.175 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.192+0000 7fb7d2404740 1 --2- 172.21.15.107:0/164868905 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56369031f7a0 0x563690325310 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fb7b409aeb0 tx=0x7fb7b400b480 comp rx=0 tx=0).stop 2024-03-08T21:02:10.175 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.192+0000 7fb7d2404740 1 -- 172.21.15.107:0/164868905 shutdown_connections 2024-03-08T21:02:10.176 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.192+0000 7fb7d2404740 1 --2- 172.21.15.107:0/164868905 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x563690328570 0x563690325920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:02:10.176 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.192+0000 7fb7d2404740 1 --2- 172.21.15.107:0/164868905 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56369031f7a0 0x563690325310 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:02:10.176 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.192+0000 7fb7d2404740 1 --2- 172.21.15.107:0/164868905 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x563690338490 0x56369033a880 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:02:10.176 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.192+0000 7fb7d2404740 1 --2- 172.21.15.107:0/164868905 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56369031ee80 0x563690327f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:02:10.176 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.192+0000 7fb7d2404740 1 -- 172.21.15.107:0/164868905 >> 172.21.15.107:0/164868905 conn(0x5636903197a0 msgr2=0x56369031b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:02:10.176 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.192+0000 7fb7aea00640 0 client.15597 ms_handle_reset on v2:172.21.15.149:6838/55290596 2024-03-08T21:02:10.176 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.192+0000 7fb7d2404740 1 -- 172.21.15.107:0/164868905 shutdown_connections 2024-03-08T21:02:10.176 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.192+0000 7fb7d2404740 1 -- 172.21.15.107:0/164868905 wait complete. 2024-03-08T21:02:10.176 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.192+0000 7fb7d2404740 20 client.15597 trim_cache size 0 max 16384 2024-03-08T21:02:10.191 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.208+0000 7fb7d2404740 1 Processor -- start 2024-03-08T21:02:10.191 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.208+0000 7fb7d2404740 1 -- start start 2024-03-08T21:02:10.191 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.208+0000 7fb7d2404740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56369031ee80 0x563690327a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:02:10.191 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.208+0000 7fb7d2404740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56369031f7a0 0x563690327f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:02:10.191 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.209+0000 7fb7d2404740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563690328570 0x5636903225d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:02:10.191 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.209+0000 7fb7d2404740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7c81aeda0 con 0x56369031f7a0 2024-03-08T21:02:10.192 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.209+0000 7fb7d2404740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7c81b0720 con 0x563690328570 2024-03-08T21:02:10.192 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.209+0000 7fb7d2404740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7c81af400 con 0x56369031ee80 2024-03-08T21:02:10.192 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.209+0000 7fb7c7e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56369031f7a0 0x563690327f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:02:10.192 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.209+0000 7fb7c7e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56369031f7a0 0x563690327f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:37538/0 (socket says 172.21.15.107:37538) 2024-03-08T21:02:10.192 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.209+0000 7fb7c7e00640 1 -- 172.21.15.107:0/1127298842 learned_addr learned my addr 172.21.15.107:0/1127298842 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:02:10.192 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.209+0000 7fb7cce00640 1 --2- 172.21.15.107:0/1127298842 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56369031ee80 0x563690327a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:02:10.192 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.209+0000 7fb7cd800640 1 --2- 172.21.15.107:0/1127298842 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563690328570 0x5636903225d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:02:10.192 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.209+0000 7fb7c7e00640 1 -- 172.21.15.107:0/1127298842 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56369031ee80 msgr2=0x563690327a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:02:10.192 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.209+0000 7fb7c7e00640 1 --2- 172.21.15.107:0/1127298842 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56369031ee80 0x563690327a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:02:10.192 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.209+0000 7fb7c7e00640 1 -- 172.21.15.107:0/1127298842 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563690328570 msgr2=0x5636903225d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:02:10.192 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.209+0000 7fb7c7e00640 1 --2- 172.21.15.107:0/1127298842 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563690328570 0x5636903225d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:02:10.192 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.209+0000 7fb7c7e00640 1 -- 172.21.15.107:0/1127298842 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56369032acd0 con 0x56369031f7a0 2024-03-08T21:02:10.196 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.210+0000 7fb7c7e00640 1 --2- 172.21.15.107:0/1127298842 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56369031f7a0 0x563690327f80 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fb7b409b140 tx=0x7fb7b409b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0c9f36dd1f51edd server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:02:10.196 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.210+0000 7fb7b9600640 1 -- 172.21.15.107:0/1127298842 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7b40d2180 con 0x56369031f7a0 2024-03-08T21:02:10.196 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.210+0000 7fb7b9600640 1 -- 172.21.15.107:0/1127298842 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fb7b4009de0 con 0x56369031f7a0 2024-03-08T21:02:10.196 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.210+0000 7fb7b9600640 1 -- 172.21.15.107:0/1127298842 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7b40a3950 con 0x56369031f7a0 2024-03-08T21:02:10.196 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.211+0000 7fb7d2404740 1 -- 172.21.15.107:0/1127298842 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56369031f7a0 msgr2=0x563690327f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:02:10.196 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.211+0000 7fb7d2404740 1 --2- 172.21.15.107:0/1127298842 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56369031f7a0 0x563690327f80 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fb7b409b140 tx=0x7fb7b409b040 comp rx=0 tx=0).stop 2024-03-08T21:02:10.196 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.211+0000 7fb7d2404740 1 -- 172.21.15.107:0/1127298842 shutdown_connections 2024-03-08T21:02:10.196 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.211+0000 7fb7d2404740 1 --2- 172.21.15.107:0/1127298842 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563690328570 0x5636903225d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:02:10.196 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.211+0000 7fb7d2404740 1 --2- 172.21.15.107:0/1127298842 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56369031f7a0 0x563690327f80 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:02:10.197 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.211+0000 7fb7d2404740 1 --2- 172.21.15.107:0/1127298842 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56369031ee80 0x563690327a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:02:10.197 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.211+0000 7fb7d2404740 1 -- 172.21.15.107:0/1127298842 >> 172.21.15.107:0/1127298842 conn(0x5636903197a0 msgr2=0x56369033f150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:02:10.197 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.211+0000 7fb7d2404740 1 -- 172.21.15.107:0/1127298842 shutdown_connections 2024-03-08T21:02:10.197 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.211+0000 7fb7d2404740 1 -- 172.21.15.107:0/1127298842 wait complete. 2024-03-08T21:02:10.197 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.211+0000 7fb7d2404740 1 -- start start 2024-03-08T21:02:10.197 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.211+0000 7fb7d2404740 1 Processor -- start 2024-03-08T21:02:10.197 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.212+0000 7fb7d2404740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56369031ee80 0x56369032e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:02:10.197 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.212+0000 7fb7d2404740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563690328570 0x56369032ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:02:10.197 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.212+0000 7fb7d2404740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56369038c8d0 0x56369038ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:02:10.197 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.212+0000 7fb7d2404740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7c81b00c0 con 0x56369038c8d0 2024-03-08T21:02:10.197 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.212+0000 7fb7d2404740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7c81afa60 con 0x563690328570 2024-03-08T21:02:10.197 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.212+0000 7fb7d2404740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7c81ad040 con 0x56369031ee80 2024-03-08T21:02:10.197 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.212+0000 7fb7d2404740 10 client.0 _ll_register_callbacks cb 0x7fb7c0009600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T21:02:10.198 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.212+0000 7fb7cd800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56369038c8d0 0x56369038ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:02:10.198 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.212+0000 7fb7cd800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56369038c8d0 0x56369038ecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:37540/0 (socket says 172.21.15.107:37540) 2024-03-08T21:02:10.198 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.212+0000 7fb7cd800640 1 -- 172.21.15.107:0/2975162550 learned_addr learned my addr 172.21.15.107:0/2975162550 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:02:10.198 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.212+0000 7fb7c7e00640 1 --2- 172.21.15.107:0/2975162550 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563690328570 0x56369032ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:02:10.198 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.212+0000 7fb7cce00640 1 --2- 172.21.15.107:0/2975162550 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56369031ee80 0x56369032e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:02:10.198 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.213+0000 7fb7cd800640 1 -- 172.21.15.107:0/2975162550 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56369031ee80 msgr2=0x56369032e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:02:10.198 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.213+0000 7fb7cd800640 1 --2- 172.21.15.107:0/2975162550 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56369031ee80 0x56369032e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:02:10.198 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.213+0000 7fb7cd800640 1 -- 172.21.15.107:0/2975162550 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563690328570 msgr2=0x56369032ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:02:10.198 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.213+0000 7fb7cd800640 1 --2- 172.21.15.107:0/2975162550 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563690328570 0x56369032ebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:02:10.198 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.213+0000 7fb7cd800640 1 -- 172.21.15.107:0/2975162550 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56369038f300 con 0x56369038c8d0 2024-03-08T21:02:10.198 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.213+0000 7fb7cd800640 1 --2- 172.21.15.107:0/2975162550 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56369038c8d0 0x56369038ecc0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fb7c0009e60 tx=0x7fb7c0009430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de1d99685a124608 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:02:10.198 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.213+0000 7fb7aea00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-03-08T21:02:10.198 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.213+0000 7fb7aea00640 1 -- 172.21.15.107:0/2975162550 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7c0016040 con 0x56369038c8d0 2024-03-08T21:02:10.199 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.213+0000 7fb7aea00640 1 -- 172.21.15.107:0/2975162550 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fb7c0012660 con 0x56369038c8d0 2024-03-08T21:02:10.199 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.213+0000 7fb7d2404740 1 -- 172.21.15.107:0/2975162550 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb7b4009de0 con 0x56369038c8d0 2024-03-08T21:02:10.199 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.213+0000 7fb7aea00640 1 -- 172.21.15.107:0/2975162550 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb7c0030050 con 0x56369038c8d0 2024-03-08T21:02:10.199 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.213+0000 7fb7d2404740 10 client.15603 fetch_fsmap learned FSMap version 30 2024-03-08T21:02:10.199 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.213+0000 7fb7d2404740 1 -- 172.21.15.107:0/2975162550 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x563690390610 con 0x56369038c8d0 2024-03-08T21:02:10.199 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.214+0000 7fb7aea00640 1 -- 172.21.15.107:0/2975162550 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fb7b4009de0 con 0x56369038c8d0 2024-03-08T21:02:10.199 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.214+0000 7fb7aea00640 1 -- 172.21.15.107:0/2975162550 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 30) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fb7c002b030 con 0x56369038c8d0 2024-03-08T21:02:10.199 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.214+0000 7fb7d2404740 10 client.15603 fetch_fsmap finished waiting for FSMap version 30 2024-03-08T21:02:10.199 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.214+0000 7fb7d2404740 10 client.15603 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-08T21:02:10.199 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.214+0000 7fb7d2404740 20 client.15603 populate_metadata read hostname 'smithi107' 2024-03-08T21:02:10.199 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.214+0000 7fb7d2404740 1 --2- 172.21.15.107:0/2975162550 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x563690390a00 0x563690392e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:02:10.199 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.214+0000 7fb7d2404740 4 client.15603 mds_command: new command op to 34475 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-08T21:02:10.199 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.214+0000 7fb7d2404740 1 -- 172.21.15.107:0/2975162550 --> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x563690289c60 con 0x563690390a00 2024-03-08T21:02:10.200 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.215+0000 7fb7cce00640 1 --2- 172.21.15.107:0/2975162550 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x563690390a00 0x563690392e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:02:10.200 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.215+0000 7fb7aea00640 10 client.15603 ms_handle_connect on v2:172.21.15.149:6838/55290596 2024-03-08T21:02:10.200 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.215+0000 7fb7cce00640 1 --2- 172.21.15.107:0/2975162550 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x563690390a00 0x563690392e20 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3d3ffd28730ba955 server_cookie=a4ce2c2724336e83 in_seq=0 out_seq=0 2024-03-08T21:02:10.232 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.249+0000 7fb7aea00640 1 -- 172.21.15.107:0/2975162550 <== mds.0 v2:172.21.15.149:6838/55290596 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x563690289c60 con 0x563690390a00 2024-03-08T21:02:10.232 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.249+0000 7fb7aea00640 10 client.15603 handle_command_reply: tid=0 2024-03-08T21:02:10.232 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.249+0000 7fb7d2404740 1 client.15603 shutdown 2024-03-08T21:02:10.232 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.250+0000 7fb7d2404740 1 -- 172.21.15.107:0/2975162550 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56369038c8d0 msgr2=0x56369038ecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:02:10.232 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.250+0000 7fb7d2404740 1 --2- 172.21.15.107:0/2975162550 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56369038c8d0 0x56369038ecc0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fb7c0009e60 tx=0x7fb7c0009430 comp rx=0 tx=0).stop 2024-03-08T21:02:10.233 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.250+0000 7fb7d2404740 1 -- 172.21.15.107:0/2975162550 shutdown_connections 2024-03-08T21:02:10.233 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.250+0000 7fb7d2404740 1 --2- 172.21.15.107:0/2975162550 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56369038c8d0 0x56369038ecc0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:02:10.233 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.250+0000 7fb7d2404740 1 --2- 172.21.15.107:0/2975162550 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x563690390a00 0x563690392e20 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:02:10.233 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.250+0000 7fb7d2404740 1 --2- 172.21.15.107:0/2975162550 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x563690328570 0x56369032ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:02:10.233 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.250+0000 7fb7d2404740 1 --2- 172.21.15.107:0/2975162550 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56369031ee80 0x56369032e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:02:10.233 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.250+0000 7fb7aea00640 0 client.15603 ms_handle_reset on v2:172.21.15.149:6838/55290596 2024-03-08T21:02:10.233 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.250+0000 7fb7d2404740 1 -- 172.21.15.107:0/2975162550 >> 172.21.15.107:0/2975162550 conn(0x5636903197a0 msgr2=0x563690317f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:02:10.233 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.250+0000 7fb7d2404740 1 -- 172.21.15.107:0/2975162550 shutdown_connections 2024-03-08T21:02:10.233 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.250+0000 7fb7d2404740 1 -- 172.21.15.107:0/2975162550 wait complete. 2024-03-08T21:02:10.233 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.250+0000 7fb7d2404740 20 client.15603 trim_cache size 0 max 16384 2024-03-08T21:02:10.233 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:{ 2024-03-08T21:02:10.234 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "message": "", 2024-03-08T21:02:10.234 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "return_code": 0 2024-03-08T21:02:10.234 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:} 2024-03-08T21:02:10.235 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.251+0000 7fb7ce200640 1 -- 172.21.15.107:0/4216652702 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7fb7b00a2960 msgr2=0x7fb7b00a4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:02:10.235 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.251+0000 7fb7ce200640 1 --2- 172.21.15.107:0/4216652702 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7fb7b00a2960 0x7fb7b00a4e20 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fb7c8107b70 tx=0x7fb7bc005b50 comp rx=0 tx=0).stop 2024-03-08T21:02:10.235 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.251+0000 7fb7ce200640 1 -- 172.21.15.107:0/4216652702 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb7c8106410 msgr2=0x7fb7c819fd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:02:10.235 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.251+0000 7fb7ce200640 1 --2- 172.21.15.107:0/4216652702 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb7c8106410 0x7fb7c819fd00 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fb7b400d770 tx=0x7fb7b400dc30 comp rx=0 tx=0).stop 2024-03-08T21:02:10.235 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.251+0000 7fb7ce200640 1 -- 172.21.15.107:0/4216652702 shutdown_connections 2024-03-08T21:02:10.235 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.251+0000 7fb7ce200640 1 --2- 172.21.15.107:0/4216652702 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7fb7b00a2960 0x7fb7b00a4e20 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:02:10.235 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.251+0000 7fb7ce200640 1 --2- 172.21.15.107:0/4216652702 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb7c81999c0 0x7fb7c8199e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:02:10.235 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.251+0000 7fb7ce200640 1 --2- 172.21.15.107:0/4216652702 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb7c8106410 0x7fb7c819fd00 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:02:10.235 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.251+0000 7fb7ce200640 1 --2- 172.21.15.107:0/4216652702 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb7c8101340 0x7fb7c819f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:02:10.235 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.251+0000 7fb7ce200640 1 -- 172.21.15.107:0/4216652702 >> 172.21.15.107:0/4216652702 conn(0x7fb7c80fcc20 msgr2=0x7fb7c80fdf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:02:10.235 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.251+0000 7fb7ce200640 1 -- 172.21.15.107:0/4216652702 shutdown_connections 2024-03-08T21:02:10.235 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:10.252+0000 7fb7ce200640 1 -- 172.21.15.107:0/4216652702 wait complete. 2024-03-08T21:02:11.877 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:02:11 smithi204 ceph-mon[34935]: pgmap v53: 97 pgs: 97 active+clean; 1.8 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 560 KiB/s rd, 1.4 MiB/s wr, 176 op/s 2024-03-08T21:02:11.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:02:11 smithi107 ceph-mon[31381]: pgmap v53: 97 pgs: 97 active+clean; 1.8 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 560 KiB/s rd, 1.4 MiB/s wr, 176 op/s 2024-03-08T21:02:12.013 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:02:11 smithi149 ceph-mon[35042]: pgmap v53: 97 pgs: 97 active+clean; 1.8 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 560 KiB/s rd, 1.4 MiB/s wr, 176 op/s 2024-03-08T21:02:13.877 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:02:13 smithi204 ceph-mon[34935]: pgmap v54: 97 pgs: 97 active+clean; 1.8 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 558 KiB/s rd, 1.4 MiB/s wr, 171 op/s 2024-03-08T21:02:13.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:02:13 smithi107 ceph-mon[31381]: pgmap v54: 97 pgs: 97 active+clean; 1.8 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 558 KiB/s rd, 1.4 MiB/s wr, 171 op/s 2024-03-08T21:02:14.013 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:02:13 smithi149 ceph-mon[35042]: pgmap v54: 97 pgs: 97 active+clean; 1.8 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 558 KiB/s rd, 1.4 MiB/s wr, 171 op/s 2024-03-08T21:02:14.982 INFO:tasks.workunit.client.0.smithi107.stdout:.......... 2024-03-08T21:02:14.983 INFO:tasks.workunit.client.0.smithi107.stdout:do_o_direct_reads: SUCCESS 2024-03-08T21:02:14.992 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-03-08T21:02:14.992 DEBUG:teuthology.orchestra.run.smithi107:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-03-08T21:02:15.114 INFO:tasks.workunit:Running workunit direct_io/misc.sh... 2024-03-08T21:02:15.114 DEBUG:teuthology.orchestra.run.smithi107:workunit test direct_io/misc.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=7d63482aa89628d733e298984a762389b625c761 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io/misc.sh 2024-03-08T21:02:15.199 INFO:tasks.workunit.client.0.smithi107.stdout:test read from hole 2024-03-08T21:02:15.199 INFO:tasks.workunit.client.0.smithi107.stderr:+ echo 'test read from hole' 2024-03-08T21:02:15.199 INFO:tasks.workunit.client.0.smithi107.stderr:+ dd if=/dev/zero of=dd3 bs=1 seek=1048576 count=0 2024-03-08T21:02:15.207 INFO:tasks.workunit.client.0.smithi107.stderr:0+0 records in 2024-03-08T21:02:15.208 INFO:tasks.workunit.client.0.smithi107.stderr:0+0 records out 2024-03-08T21:02:15.208 INFO:tasks.workunit.client.0.smithi107.stderr:0 bytes copied, 0.000177834 s, 0.0 kB/s 2024-03-08T21:02:15.208 INFO:tasks.workunit.client.0.smithi107.stderr:+ dd if=dd3 of=/tmp/ddout1 skip=8 bs=512 count=2 iflag=direct 2024-03-08T21:02:15.221 INFO:tasks.workunit.client.0.smithi107.stderr:2+0 records in 2024-03-08T21:02:15.222 INFO:tasks.workunit.client.0.smithi107.stderr:2+0 records out 2024-03-08T21:02:15.222 INFO:tasks.workunit.client.0.smithi107.stderr:1024 bytes (1.0 kB, 1.0 KiB) copied, 0.0101074 s, 101 kB/s 2024-03-08T21:02:15.222 INFO:tasks.workunit.client.0.smithi107.stderr:+ dd if=/dev/zero of=/tmp/dd3 bs=512 count=2 2024-03-08T21:02:15.225 INFO:tasks.workunit.client.0.smithi107.stderr:2+0 records in 2024-03-08T21:02:15.225 INFO:tasks.workunit.client.0.smithi107.stderr:2+0 records out 2024-03-08T21:02:15.225 INFO:tasks.workunit.client.0.smithi107.stderr:1024 bytes (1.0 kB, 1.0 KiB) copied, 0.000124434 s, 8.2 MB/s 2024-03-08T21:02:15.225 INFO:tasks.workunit.client.0.smithi107.stderr:+ cmp /tmp/dd3 /tmp/ddout1 2024-03-08T21:02:15.228 INFO:tasks.workunit.client.0.smithi107.stderr:+ echo 'other thing' 2024-03-08T21:02:15.228 INFO:tasks.workunit.client.0.smithi107.stdout:other thing 2024-03-08T21:02:15.229 INFO:tasks.workunit.client.0.smithi107.stderr:+ dd if=/dev/urandom of=/tmp/dd10 bs=500 count=1 2024-03-08T21:02:15.231 INFO:tasks.workunit.client.0.smithi107.stderr:1+0 records in 2024-03-08T21:02:15.231 INFO:tasks.workunit.client.0.smithi107.stderr:1+0 records out 2024-03-08T21:02:15.231 INFO:tasks.workunit.client.0.smithi107.stderr:500 bytes copied, 0.000119284 s, 4.2 MB/s 2024-03-08T21:02:15.231 INFO:tasks.workunit.client.0.smithi107.stderr:+ dd if=/tmp/dd10 of=dd10 bs=512 seek=8388 count=1 2024-03-08T21:02:15.255 INFO:tasks.workunit.client.0.smithi107.stderr:0+1 records in 2024-03-08T21:02:15.255 INFO:tasks.workunit.client.0.smithi107.stderr:0+1 records out 2024-03-08T21:02:15.255 INFO:tasks.workunit.client.0.smithi107.stderr:500 bytes copied, 0.0180707 s, 27.7 kB/s 2024-03-08T21:02:15.256 INFO:tasks.workunit.client.0.smithi107.stderr:+ dd if=dd10 of=/tmp/dd10out bs=512 skip=8388 count=1 iflag=direct 2024-03-08T21:02:15.266 INFO:tasks.workunit.client.0.smithi107.stderr:0+1 records in 2024-03-08T21:02:15.266 INFO:tasks.workunit.client.0.smithi107.stderr:0+1 records out 2024-03-08T21:02:15.266 INFO:tasks.workunit.client.0.smithi107.stderr:500 bytes copied, 0.00762759 s, 65.6 kB/s 2024-03-08T21:02:15.267 INFO:tasks.workunit.client.0.smithi107.stderr:+ cmp /tmp/dd10 /tmp/dd10out 2024-03-08T21:02:15.269 INFO:tasks.workunit.client.0.smithi107.stdout:OK 2024-03-08T21:02:15.269 INFO:tasks.workunit.client.0.smithi107.stderr:+ echo OK 2024-03-08T21:02:15.270 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-03-08T21:02:15.270 DEBUG:teuthology.orchestra.run.smithi107:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-03-08T21:02:15.407 INFO:tasks.workunit:Running workunit direct_io/test_short_dio_read... 2024-03-08T21:02:15.408 DEBUG:teuthology.orchestra.run.smithi107:workunit test direct_io/test_short_dio_read> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=7d63482aa89628d733e298984a762389b625c761 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io/test_short_dio_read 2024-03-08T21:02:15.504 INFO:tasks.workunit.client.0.smithi107.stdout:writing first 3 bytes of 10k file 2024-03-08T21:02:15.504 INFO:tasks.workunit.client.0.smithi107.stdout:reading O_DIRECT 2024-03-08T21:02:15.504 INFO:tasks.workunit.client.0.smithi107.stdout:got 10000 2024-03-08T21:02:15.505 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-03-08T21:02:15.505 DEBUG:teuthology.orchestra.run.smithi107:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-03-08T21:02:15.639 INFO:tasks.workunit:Running workunit direct_io/test_sync_io... 2024-03-08T21:02:15.640 DEBUG:teuthology.orchestra.run.smithi107:workunit test direct_io/test_sync_io> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=7d63482aa89628d733e298984a762389b625c761 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io/test_sync_io 2024-03-08T21:02:15.877 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:02:15 smithi204 ceph-mon[34935]: pgmap v55: 97 pgs: 97 active+clean; 1.8 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.4 MiB/s wr, 443 op/s 2024-03-08T21:02:15.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:02:15 smithi107 ceph-mon[31381]: pgmap v55: 97 pgs: 97 active+clean; 1.8 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.4 MiB/s wr, 443 op/s 2024-03-08T21:02:16.013 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:02:15 smithi149 ceph-mon[35042]: pgmap v55: 97 pgs: 97 active+clean; 1.8 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.4 MiB/s wr, 443 op/s 2024-03-08T21:02:17.877 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:02:17 smithi204 ceph-mon[34935]: pgmap v56: 97 pgs: 97 active+clean; 1.8 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 39 KiB/s wr, 411 op/s 2024-03-08T21:02:17.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:02:17 smithi107 ceph-mon[31381]: pgmap v56: 97 pgs: 97 active+clean; 1.8 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 39 KiB/s wr, 411 op/s 2024-03-08T21:02:18.013 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:02:17 smithi149 ceph-mon[35042]: pgmap v56: 97 pgs: 97 active+clean; 1.8 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 39 KiB/s wr, 411 op/s 2024-03-08T21:02:18.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:02:18 smithi107 ceph-mon[31381]: pgmap v57: 97 pgs: 97 active+clean; 1.8 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 39 KiB/s wr, 411 op/s 2024-03-08T21:02:19.013 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:02:18 smithi149 ceph-mon[35042]: pgmap v57: 97 pgs: 97 active+clean; 1.8 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 39 KiB/s wr, 411 op/s 2024-03-08T21:02:19.126 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:02:18 smithi204 ceph-mon[34935]: pgmap v57: 97 pgs: 97 active+clean; 1.8 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 39 KiB/s wr, 411 op/s 2024-03-08T21:02:21.876 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:02:21 smithi204 ceph-mon[34935]: pgmap v58: 97 pgs: 97 active+clean; 1.5 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 45 KiB/s wr, 495 op/s 2024-03-08T21:02:21.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:02:21 smithi107 ceph-mon[31381]: pgmap v58: 97 pgs: 97 active+clean; 1.5 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 45 KiB/s wr, 495 op/s 2024-03-08T21:02:22.013 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:02:21 smithi149 ceph-mon[35042]: pgmap v58: 97 pgs: 97 active+clean; 1.5 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 45 KiB/s wr, 495 op/s 2024-03-08T21:02:23.381 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.399+0000 7fc81b200640 1 -- 172.21.15.107:0/1486004353 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc814100e00 msgr2=0x7fc81410d020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:02:23.381 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.399+0000 7fc81b200640 1 --2- 172.21.15.107:0/1486004353 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc814100e00 0x7fc81410d020 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fc80c009ca0 tx=0x7fc80c02c100 comp rx=0 tx=0).stop 2024-03-08T21:02:23.381 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.399+0000 7fc81b200640 1 -- 172.21.15.107:0/1486004353 shutdown_connections 2024-03-08T21:02:23.381 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.399+0000 7fc81b200640 1 --2- 172.21.15.107:0/1486004353 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc814100e00 0x7fc81410d020 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:02:23.382 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.399+0000 7fc81b200640 1 --2- 172.21.15.107:0/1486004353 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc814100440 0x7fc8141008c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:02:23.382 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.399+0000 7fc81b200640 1 --2- 172.21.15.107:0/1486004353 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc814105190 0x7fc814105570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:02:23.382 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.399+0000 7fc81b200640 1 -- 172.21.15.107:0/1486004353 >> 172.21.15.107:0/1486004353 conn(0x7fc814077040 msgr2=0x7fc814077450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:02:23.382 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.399+0000 7fc81b200640 1 -- 172.21.15.107:0/1486004353 shutdown_connections 2024-03-08T21:02:23.382 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.399+0000 7fc81b200640 1 -- 172.21.15.107:0/1486004353 wait complete. 2024-03-08T21:02:23.383 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.400+0000 7fc81b200640 1 Processor -- start 2024-03-08T21:02:23.383 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.400+0000 7fc81b200640 1 -- start start 2024-03-08T21:02:23.383 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.401+0000 7fc81b200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc814100440 0x7fc814101e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:02:23.383 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.401+0000 7fc81b200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc814100e00 0x7fc8141023a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:02:23.384 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.401+0000 7fc81b200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc814105190 0x7fc8141028e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:02:23.384 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.401+0000 7fc81b200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8141104e0 con 0x7fc814105190 2024-03-08T21:02:23.384 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.401+0000 7fc81b200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc814110360 con 0x7fc814100e00 2024-03-08T21:02:23.384 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.401+0000 7fc81b200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc814110660 con 0x7fc814100440 2024-03-08T21:02:23.384 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.401+0000 7fc819200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc814105190 0x7fc8141028e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:02:23.384 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.401+0000 7fc819200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc814105190 0x7fc8141028e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:42076/0 (socket says 172.21.15.107:42076) 2024-03-08T21:02:23.384 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.401+0000 7fc819200640 1 -- 172.21.15.107:0/1331715485 learned_addr learned my addr 172.21.15.107:0/1331715485 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:02:23.384 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.401+0000 7fc813e00640 1 --2- 172.21.15.107:0/1331715485 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc814100440 0x7fc814101e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:02:23.384 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.401+0000 7fc813400640 1 --2- 172.21.15.107:0/1331715485 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc814100e00 0x7fc8141023a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:02:23.385 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.402+0000 7fc819200640 1 -- 172.21.15.107:0/1331715485 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc814100440 msgr2=0x7fc814101e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:02:23.385 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.402+0000 7fc819200640 1 --2- 172.21.15.107:0/1331715485 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc814100440 0x7fc814101e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:02:23.385 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.402+0000 7fc819200640 1 -- 172.21.15.107:0/1331715485 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc814100e00 msgr2=0x7fc8141023a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:02:23.385 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.402+0000 7fc819200640 1 --2- 172.21.15.107:0/1331715485 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc814100e00 0x7fc8141023a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:02:23.385 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.402+0000 7fc819200640 1 -- 172.21.15.107:0/1331715485 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8141a5ef0 con 0x7fc814105190 2024-03-08T21:02:23.385 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.402+0000 7fc813400640 1 --2- 172.21.15.107:0/1331715485 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc814100e00 0x7fc8141023a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T21:02:23.385 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.402+0000 7fc819200640 1 --2- 172.21.15.107:0/1331715485 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc814105190 0x7fc8141028e0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fc80c02c710 tx=0x7fc80c035d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:02:23.386 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.402+0000 7fc810c00640 1 -- 172.21.15.107:0/1331715485 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc80c039040 con 0x7fc814105190 2024-03-08T21:02:23.386 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.403+0000 7fc810c00640 1 -- 172.21.15.107:0/1331715485 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fc80c044070 con 0x7fc814105190 2024-03-08T21:02:23.386 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.403+0000 7fc810c00640 1 -- 172.21.15.107:0/1331715485 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc80c0042a0 con 0x7fc814105190 2024-03-08T21:02:23.386 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.403+0000 7fc81b200640 1 -- 172.21.15.107:0/1331715485 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8141a6180 con 0x7fc814105190 2024-03-08T21:02:23.386 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.403+0000 7fc81b200640 1 -- 172.21.15.107:0/1331715485 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8141a6620 con 0x7fc814105190 2024-03-08T21:02:23.390 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.407+0000 7fc810c00640 1 -- 172.21.15.107:0/1331715485 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) v1 ==== 137422+0+0 (secure 0 0 0) 0x7fc80c003970 con 0x7fc814105190 2024-03-08T21:02:23.390 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.407+0000 7fc81df9e740 1 Processor -- start 2024-03-08T21:02:23.390 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.407+0000 7fc81df9e740 1 -- start start 2024-03-08T21:02:23.390 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.407+0000 7fc81df9e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e9129d4c80 0x55e9129d5060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:02:23.390 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.407+0000 7fc81df9e740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e9129d55a0 0x55e9129dde30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:02:23.390 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.407+0000 7fc81df9e740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e9129de370 0x55e9129e0760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:02:23.390 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.407+0000 7fc81df9e740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9129383d0 con 0x55e9129d4c80 2024-03-08T21:02:23.390 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.407+0000 7fc81df9e740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9128e16b0 con 0x55e9129d55a0 2024-03-08T21:02:23.390 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.407+0000 7fc81df9e740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e912934470 con 0x55e9129de370 2024-03-08T21:02:23.390 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.408+0000 7fc813400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e9129d4c80 0x55e9129d5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:02:23.390 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.408+0000 7fc813400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e9129d4c80 0x55e9129d5060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:42086/0 (socket says 172.21.15.107:42086) 2024-03-08T21:02:23.391 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.408+0000 7fc813400640 1 -- 172.21.15.107:0/1464085794 learned_addr learned my addr 172.21.15.107:0/1464085794 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:02:23.391 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.408+0000 7fc813e00640 1 --2- 172.21.15.107:0/1464085794 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e9129d55a0 0x55e9129dde30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:02:23.391 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.408+0000 7fc813400640 1 --2- 172.21.15.107:0/1464085794 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e9129de370 0x55e9129e0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:02:23.393 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.408+0000 7fc813400640 1 -- 172.21.15.107:0/1464085794 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e9129de370 msgr2=0x55e9129e0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:02:23.393 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.408+0000 7fc813400640 1 --2- 172.21.15.107:0/1464085794 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e9129de370 0x55e9129e0760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:02:23.393 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.408+0000 7fc813400640 1 -- 172.21.15.107:0/1464085794 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e9129d55a0 msgr2=0x55e9129dde30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:02:23.393 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.408+0000 7fc813400640 1 --2- 172.21.15.107:0/1464085794 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e9129d55a0 0x55e9129dde30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:02:23.393 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.408+0000 7fc813400640 1 -- 172.21.15.107:0/1464085794 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e912907b50 con 0x55e9129d4c80 2024-03-08T21:02:23.393 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.409+0000 7fc810c00640 1 --2- 172.21.15.107:0/1331715485 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7fc7f80a2a90 0x7fc7f80a4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:02:23.393 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.409+0000 7fc813400640 1 --2- 172.21.15.107:0/1464085794 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e9129d4c80 0x55e9129d5060 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fc800013560 tx=0x7fc8000097d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=721fd35246580aa8 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:02:23.393 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.409+0000 7fc810c00640 1 -- 172.21.15.107:0/1331715485 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fc80c0f1280 con 0x7fc814105190 2024-03-08T21:02:23.393 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.409+0000 7fc809600640 1 -- 172.21.15.107:0/1464085794 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc800011470 con 0x55e9129d4c80 2024-03-08T21:02:23.393 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.409+0000 7fc809600640 1 -- 172.21.15.107:0/1464085794 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fc800011ac0 con 0x55e9129d4c80 2024-03-08T21:02:23.393 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.409+0000 7fc809600640 1 -- 172.21.15.107:0/1464085794 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc80000f9e0 con 0x55e9129d4c80 2024-03-08T21:02:23.394 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.409+0000 7fc813e00640 1 --2- 172.21.15.107:0/1331715485 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7fc7f80a2a90 0x7fc7f80a4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:02:23.394 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.410+0000 7fc81df9e740 1 -- 172.21.15.107:0/1464085794 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e9129d4c80 msgr2=0x55e9129d5060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:02:23.394 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.410+0000 7fc81df9e740 1 --2- 172.21.15.107:0/1464085794 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e9129d4c80 0x55e9129d5060 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fc800013560 tx=0x7fc8000097d0 comp rx=0 tx=0).stop 2024-03-08T21:02:23.394 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.410+0000 7fc813e00640 1 --2- 172.21.15.107:0/1331715485 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7fc7f80a2a90 0x7fc7f80a4f50 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x55e9129d4730 tx=0x7fc80400b390 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:02:23.394 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.410+0000 7fc81df9e740 1 -- 172.21.15.107:0/1464085794 shutdown_connections 2024-03-08T21:02:23.394 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.410+0000 7fc81df9e740 1 --2- 172.21.15.107:0/1464085794 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e9129de370 0x55e9129e0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:02:23.394 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.410+0000 7fc81df9e740 1 --2- 172.21.15.107:0/1464085794 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e9129d55a0 0x55e9129dde30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:02:23.394 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.410+0000 7fc81df9e740 1 --2- 172.21.15.107:0/1464085794 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e9129d4c80 0x55e9129d5060 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:02:23.394 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.410+0000 7fc81df9e740 1 -- 172.21.15.107:0/1464085794 >> 172.21.15.107:0/1464085794 conn(0x55e9129cf5a0 msgr2=0x55e9129cf980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:02:23.394 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.410+0000 7fc81df9e740 1 -- 172.21.15.107:0/1464085794 shutdown_connections 2024-03-08T21:02:23.394 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.410+0000 7fc81df9e740 1 -- 172.21.15.107:0/1464085794 wait complete. 2024-03-08T21:02:23.394 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.411+0000 7fc81df9e740 1 -- start start 2024-03-08T21:02:23.394 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.411+0000 7fc81df9e740 1 Processor -- start 2024-03-08T21:02:23.397 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.412+0000 7fc81df9e740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e9129d4c80 0x55e9129ddd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:02:23.398 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.412+0000 7fc81df9e740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e9129d55a0 0x55e9129db0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:02:23.398 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.412+0000 7fc81df9e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e9129de370 0x55e9129db650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:02:23.398 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.412+0000 7fc81df9e740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9129e0fa0 con 0x55e9129de370 2024-03-08T21:02:23.398 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.412+0000 7fc81df9e740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9128f49f0 con 0x55e9129d55a0 2024-03-08T21:02:23.398 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.412+0000 7fc81df9e740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9129e1120 con 0x55e9129d4c80 2024-03-08T21:02:23.398 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.412+0000 7fc81df9e740 10 client.0 _ll_register_callbacks cb 0x55e91264ca00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T21:02:23.398 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.412+0000 7fc819200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e9129de370 0x55e9129db650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:02:23.398 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.412+0000 7fc819200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e9129de370 0x55e9129db650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:42100/0 (socket says 172.21.15.107:42100) 2024-03-08T21:02:23.398 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.412+0000 7fc819200640 1 -- 172.21.15.107:0/3929304697 learned_addr learned my addr 172.21.15.107:0/3929304697 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:02:23.398 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.412+0000 7fc813e00640 1 --2- 172.21.15.107:0/3929304697 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e9129d4c80 0x55e9129ddd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:02:23.398 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.412+0000 7fc813400640 1 --2- 172.21.15.107:0/3929304697 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e9129d55a0 0x55e9129db0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:02:23.399 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.412+0000 7fc819200640 1 -- 172.21.15.107:0/3929304697 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e9129d4c80 msgr2=0x55e9129ddd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:02:23.399 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.413+0000 7fc819200640 1 --2- 172.21.15.107:0/3929304697 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e9129d4c80 0x55e9129ddd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:02:23.399 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.413+0000 7fc819200640 1 -- 172.21.15.107:0/3929304697 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e9129d55a0 msgr2=0x55e9129db0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:02:23.399 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.413+0000 7fc819200640 1 --2- 172.21.15.107:0/3929304697 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e9129d55a0 0x55e9129db0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:02:23.399 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.413+0000 7fc819200640 1 -- 172.21.15.107:0/3929304697 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e91264d210 con 0x55e9129de370 2024-03-08T21:02:23.399 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.413+0000 7fc813e00640 1 --2- 172.21.15.107:0/3929304697 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e9129d4c80 0x55e9129ddd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T21:02:23.399 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.413+0000 7fc819200640 1 --2- 172.21.15.107:0/3929304697 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e9129de370 0x55e9129db650 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fc80c0ba2b0 tx=0x7fc80c0bd4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=298e842c70d5a84a server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:02:23.399 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.413+0000 7fc7fe000640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-03-08T21:02:23.399 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.413+0000 7fc7fe000640 1 -- 172.21.15.107:0/3929304697 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc80c034700 con 0x55e9129de370 2024-03-08T21:02:23.399 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.413+0000 7fc7fe000640 1 -- 172.21.15.107:0/3929304697 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fc80c0a80b0 con 0x55e9129de370 2024-03-08T21:02:23.399 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.413+0000 7fc81df9e740 1 -- 172.21.15.107:0/3929304697 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc800011ac0 con 0x55e9129de370 2024-03-08T21:02:23.399 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.413+0000 7fc7fe000640 1 -- 172.21.15.107:0/3929304697 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc80c0f1050 con 0x55e9129de370 2024-03-08T21:02:23.399 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.413+0000 7fc81df9e740 10 client.15615 fetch_fsmap learned FSMap version 30 2024-03-08T21:02:23.400 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.413+0000 7fc81df9e740 1 -- 172.21.15.107:0/3929304697 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55e912719620 con 0x55e9129de370 2024-03-08T21:02:23.400 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.414+0000 7fc7fe000640 1 -- 172.21.15.107:0/3929304697 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fc800011ac0 con 0x55e9129de370 2024-03-08T21:02:23.400 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.414+0000 7fc7fe000640 1 -- 172.21.15.107:0/3929304697 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 30) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fc80c03c030 con 0x55e9129de370 2024-03-08T21:02:23.400 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.414+0000 7fc81df9e740 10 client.15615 fetch_fsmap finished waiting for FSMap version 30 2024-03-08T21:02:23.400 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.414+0000 7fc81df9e740 10 client.15615 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-08T21:02:23.400 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.414+0000 7fc81df9e740 20 client.15615 populate_metadata read hostname 'smithi107' 2024-03-08T21:02:23.400 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.414+0000 7fc81df9e740 1 --2- 172.21.15.107:0/3929304697 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55e9129ee1a0 0x55e9129f05c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:02:23.400 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.414+0000 7fc81df9e740 4 client.15615 mds_command: new command op to 34475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-08T21:02:23.400 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.414+0000 7fc81df9e740 1 -- 172.21.15.107:0/3929304697 --> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e912945910 con 0x55e9129ee1a0 2024-03-08T21:02:23.400 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.415+0000 7fc813e00640 1 --2- 172.21.15.107:0/3929304697 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55e9129ee1a0 0x55e9129f05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:02:23.400 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.416+0000 7fc813e00640 1 --2- 172.21.15.107:0/3929304697 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55e9129ee1a0 0x55e9129f05c0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c9f004f431edaa28 server_cookie=67b472e0d0f3c903 in_seq=0 out_seq=0 2024-03-08T21:02:23.400 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.416+0000 7fc7fe000640 10 client.15615 ms_handle_connect on v2:172.21.15.149:6838/55290596 2024-03-08T21:02:23.401 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.417+0000 7fc7fe000640 1 -- 172.21.15.107:0/3929304697 <== mds.0 v2:172.21.15.149:6838/55290596 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e912945910 con 0x55e9129ee1a0 2024-03-08T21:02:23.402 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.417+0000 7fc7fe000640 10 client.15615 handle_command_reply: tid=0 2024-03-08T21:02:23.402 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.417+0000 7fc81df9e740 1 client.15615 shutdown 2024-03-08T21:02:23.402 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.418+0000 7fc81df9e740 1 -- 172.21.15.107:0/3929304697 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e9129de370 msgr2=0x55e9129db650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:02:23.402 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.418+0000 7fc81df9e740 1 --2- 172.21.15.107:0/3929304697 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e9129de370 0x55e9129db650 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fc80c0ba2b0 tx=0x7fc80c0bd4a0 comp rx=0 tx=0).stop 2024-03-08T21:02:23.402 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.418+0000 7fc81df9e740 1 -- 172.21.15.107:0/3929304697 shutdown_connections 2024-03-08T21:02:23.402 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.418+0000 7fc81df9e740 1 --2- 172.21.15.107:0/3929304697 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e9129de370 0x55e9129db650 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:02:23.402 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.418+0000 7fc81df9e740 1 --2- 172.21.15.107:0/3929304697 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55e9129ee1a0 0x55e9129f05c0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:02:23.402 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.418+0000 7fc81df9e740 1 --2- 172.21.15.107:0/3929304697 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e9129d55a0 0x55e9129db0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:02:23.402 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.418+0000 7fc81df9e740 1 --2- 172.21.15.107:0/3929304697 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e9129d4c80 0x55e9129ddd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:02:23.402 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.418+0000 7fc81df9e740 1 -- 172.21.15.107:0/3929304697 >> 172.21.15.107:0/3929304697 conn(0x55e9129cf5a0 msgr2=0x55e9129d12f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:02:23.402 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.418+0000 7fc7fe000640 0 client.15615 ms_handle_reset on v2:172.21.15.149:6838/55290596 2024-03-08T21:02:23.402 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.418+0000 7fc81df9e740 1 -- 172.21.15.107:0/3929304697 shutdown_connections 2024-03-08T21:02:23.402 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.418+0000 7fc81df9e740 1 -- 172.21.15.107:0/3929304697 wait complete. 2024-03-08T21:02:23.402 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.418+0000 7fc81df9e740 20 client.15615 trim_cache size 0 max 16384 2024-03-08T21:02:23.418 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.436+0000 7fc81df9e740 1 Processor -- start 2024-03-08T21:02:23.418 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.436+0000 7fc81df9e740 1 -- start start 2024-03-08T21:02:23.420 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.436+0000 7fc81df9e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e9129d4c80 0x55e9129f4f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:02:23.420 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.436+0000 7fc81df9e740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e9129d55a0 0x55e9129dd7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:02:23.420 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.436+0000 7fc81df9e740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e9129de370 0x55e9129ddd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:02:23.420 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.436+0000 7fc81df9e740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8141aecb0 con 0x55e9129d4c80 2024-03-08T21:02:23.420 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.436+0000 7fc81df9e740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8141b0630 con 0x55e9129d55a0 2024-03-08T21:02:23.420 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.436+0000 7fc81df9e740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8141af310 con 0x55e9129de370 2024-03-08T21:02:23.420 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.437+0000 7fc813e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e9129d4c80 0x55e9129f4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:02:23.420 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.437+0000 7fc813e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e9129d4c80 0x55e9129f4f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:42102/0 (socket says 172.21.15.107:42102) 2024-03-08T21:02:23.420 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.437+0000 7fc813e00640 1 -- 172.21.15.107:0/1099250810 learned_addr learned my addr 172.21.15.107:0/1099250810 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:02:23.420 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.437+0000 7fc813400640 1 --2- 172.21.15.107:0/1099250810 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e9129d55a0 0x55e9129dd7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:02:23.420 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.437+0000 7fc819200640 1 --2- 172.21.15.107:0/1099250810 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e9129de370 0x55e9129ddd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:02:23.420 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.437+0000 7fc813e00640 1 -- 172.21.15.107:0/1099250810 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e9129de370 msgr2=0x55e9129ddd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:02:23.420 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.437+0000 7fc813e00640 1 --2- 172.21.15.107:0/1099250810 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e9129de370 0x55e9129ddd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:02:23.421 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.437+0000 7fc813e00640 1 -- 172.21.15.107:0/1099250810 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e9129d55a0 msgr2=0x55e9129dd7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:02:23.421 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.437+0000 7fc813e00640 1 --2- 172.21.15.107:0/1099250810 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e9129d55a0 0x55e9129dd7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:02:23.421 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.437+0000 7fc813e00640 1 -- 172.21.15.107:0/1099250810 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e9129cfee0 con 0x55e9129d4c80 2024-03-08T21:02:23.421 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.437+0000 7fc819200640 1 --2- 172.21.15.107:0/1099250810 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e9129de370 0x55e9129ddd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T21:02:23.421 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.438+0000 7fc813e00640 1 --2- 172.21.15.107:0/1099250810 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e9129d4c80 0x55e9129f4f40 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x55e9129f0e90 tx=0x7fc804016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adcd3e024411b20b server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:02:23.421 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.438+0000 7fc808c00640 1 -- 172.21.15.107:0/1099250810 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc804020e60 con 0x55e9129d4c80 2024-03-08T21:02:23.421 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.438+0000 7fc808c00640 1 -- 172.21.15.107:0/1099250810 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fc804029860 con 0x55e9129d4c80 2024-03-08T21:02:23.421 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.438+0000 7fc808c00640 1 -- 172.21.15.107:0/1099250810 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8040047a0 con 0x55e9129d4c80 2024-03-08T21:02:23.424 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.439+0000 7fc81df9e740 1 -- 172.21.15.107:0/1099250810 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e9129d4c80 msgr2=0x55e9129f4f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:02:23.424 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.439+0000 7fc81df9e740 1 --2- 172.21.15.107:0/1099250810 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e9129d4c80 0x55e9129f4f40 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x55e9129f0e90 tx=0x7fc804016000 comp rx=0 tx=0).stop 2024-03-08T21:02:23.425 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.439+0000 7fc81df9e740 1 -- 172.21.15.107:0/1099250810 shutdown_connections 2024-03-08T21:02:23.425 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.439+0000 7fc81df9e740 1 --2- 172.21.15.107:0/1099250810 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e9129de370 0x55e9129ddd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:02:23.425 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.439+0000 7fc81df9e740 1 --2- 172.21.15.107:0/1099250810 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e9129d55a0 0x55e9129dd7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:02:23.425 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.439+0000 7fc81df9e740 1 --2- 172.21.15.107:0/1099250810 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e9129d4c80 0x55e9129f4f40 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:02:23.425 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.439+0000 7fc81df9e740 1 -- 172.21.15.107:0/1099250810 >> 172.21.15.107:0/1099250810 conn(0x55e9129cf5a0 msgr2=0x55e9129da940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:02:23.425 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.439+0000 7fc81df9e740 1 -- 172.21.15.107:0/1099250810 shutdown_connections 2024-03-08T21:02:23.425 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.439+0000 7fc81df9e740 1 -- 172.21.15.107:0/1099250810 wait complete. 2024-03-08T21:02:23.425 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.439+0000 7fc81df9e740 1 -- start start 2024-03-08T21:02:23.425 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.439+0000 7fc81df9e740 1 Processor -- start 2024-03-08T21:02:23.425 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.440+0000 7fc81df9e740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e9129d4c80 0x55e9129e3e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:02:23.425 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.440+0000 7fc81df9e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e9129d55a0 0x55e9129e4380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:02:23.425 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.440+0000 7fc81df9e740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e9129de370 0x55e9129e48c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:02:23.426 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.440+0000 7fc81df9e740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8141affd0 con 0x55e9129d55a0 2024-03-08T21:02:23.426 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.440+0000 7fc81df9e740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8141af970 con 0x55e9129de370 2024-03-08T21:02:23.426 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.440+0000 7fc81df9e740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8141acf50 con 0x55e9129d4c80 2024-03-08T21:02:23.426 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.440+0000 7fc81df9e740 10 client.0 _ll_register_callbacks cb 0x7fc8000096e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T21:02:23.426 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.440+0000 7fc813400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e9129d55a0 0x55e9129e4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:02:23.426 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.440+0000 7fc813400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e9129d55a0 0x55e9129e4380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:42110/0 (socket says 172.21.15.107:42110) 2024-03-08T21:02:23.426 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.440+0000 7fc813400640 1 -- 172.21.15.107:0/1423251994 learned_addr learned my addr 172.21.15.107:0/1423251994 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:02:23.426 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.441+0000 7fc813e00640 1 --2- 172.21.15.107:0/1423251994 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e9129d4c80 0x55e9129e3e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:02:23.426 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.441+0000 7fc819200640 1 --2- 172.21.15.107:0/1423251994 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e9129de370 0x55e9129e48c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:02:23.426 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.441+0000 7fc813400640 1 -- 172.21.15.107:0/1423251994 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e9129d4c80 msgr2=0x55e9129e3e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:02:23.426 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.441+0000 7fc813400640 1 --2- 172.21.15.107:0/1423251994 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e9129d4c80 0x55e9129e3e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:02:23.427 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.441+0000 7fc813400640 1 -- 172.21.15.107:0/1423251994 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e9129de370 msgr2=0x55e9129e48c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:02:23.427 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.441+0000 7fc813400640 1 --2- 172.21.15.107:0/1423251994 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e9129de370 0x55e9129e48c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:02:23.427 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.441+0000 7fc813400640 1 -- 172.21.15.107:0/1423251994 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e912a44800 con 0x55e9129d55a0 2024-03-08T21:02:23.427 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.441+0000 7fc819200640 1 --2- 172.21.15.107:0/1423251994 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e9129de370 0x55e9129e48c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T21:02:23.427 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.441+0000 7fc813e00640 1 --2- 172.21.15.107:0/1423251994 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e9129d4c80 0x55e9129e3e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T21:02:23.427 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.441+0000 7fc813400640 1 --2- 172.21.15.107:0/1423251994 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e9129d55a0 0x55e9129e4380 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fc80000f660 tx=0x7fc800013380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4084432a549b30ea server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:02:23.427 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.441+0000 7fc7fe000640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-03-08T21:02:23.427 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.441+0000 7fc7fe000640 1 -- 172.21.15.107:0/1423251994 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc80002c060 con 0x55e9129d55a0 2024-03-08T21:02:23.427 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.442+0000 7fc7fe000640 1 -- 172.21.15.107:0/1423251994 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fc800005560 con 0x55e9129d55a0 2024-03-08T21:02:23.427 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.442+0000 7fc81df9e740 1 -- 172.21.15.107:0/1423251994 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc804029860 con 0x55e9129d55a0 2024-03-08T21:02:23.427 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.442+0000 7fc7fe000640 1 -- 172.21.15.107:0/1423251994 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc80000f690 con 0x55e9129d55a0 2024-03-08T21:02:23.427 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.442+0000 7fc81df9e740 10 client.15621 fetch_fsmap learned FSMap version 30 2024-03-08T21:02:23.427 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.442+0000 7fc81df9e740 1 -- 172.21.15.107:0/1423251994 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55e912a45a80 con 0x55e9129d55a0 2024-03-08T21:02:23.427 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.442+0000 7fc7fe000640 1 -- 172.21.15.107:0/1423251994 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fc804029860 con 0x55e9129d55a0 2024-03-08T21:02:23.428 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.442+0000 7fc7fe000640 1 -- 172.21.15.107:0/1423251994 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 30) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fc8000039a0 con 0x55e9129d55a0 2024-03-08T21:02:23.428 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.442+0000 7fc81df9e740 10 client.15621 fetch_fsmap finished waiting for FSMap version 30 2024-03-08T21:02:23.428 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.442+0000 7fc81df9e740 10 client.15621 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-08T21:02:23.428 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.442+0000 7fc81df9e740 20 client.15621 populate_metadata read hostname 'smithi107' 2024-03-08T21:02:23.428 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.442+0000 7fc81df9e740 1 --2- 172.21.15.107:0/1423251994 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55e912a45fb0 0x55e912a483d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:02:23.428 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.442+0000 7fc81df9e740 4 client.15621 mds_command: new command op to 34475 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-08T21:02:23.428 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.442+0000 7fc81df9e740 1 -- 172.21.15.107:0/1423251994 --> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55e91293fc60 con 0x55e912a45fb0 2024-03-08T21:02:23.428 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.443+0000 7fc813e00640 1 --2- 172.21.15.107:0/1423251994 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55e912a45fb0 0x55e912a483d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:02:23.429 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.444+0000 7fc813e00640 1 --2- 172.21.15.107:0/1423251994 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55e912a45fb0 0x55e912a483d0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d242efb60806d4c6 server_cookie=33c70c9ad2fdae37 in_seq=0 out_seq=0 2024-03-08T21:02:23.429 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.444+0000 7fc7fe000640 10 client.15621 ms_handle_connect on v2:172.21.15.149:6838/55290596 2024-03-08T21:02:23.463 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.480+0000 7fc7fe000640 1 -- 172.21.15.107:0/1423251994 <== mds.0 v2:172.21.15.149:6838/55290596 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55e91293fc60 con 0x55e912a45fb0 2024-03-08T21:02:23.463 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.480+0000 7fc7fe000640 10 client.15621 handle_command_reply: tid=0 2024-03-08T21:02:23.463 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.481+0000 7fc81df9e740 1 client.15621 shutdown 2024-03-08T21:02:23.464 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.481+0000 7fc81df9e740 1 -- 172.21.15.107:0/1423251994 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e9129d55a0 msgr2=0x55e9129e4380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:02:23.464 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.481+0000 7fc81df9e740 1 --2- 172.21.15.107:0/1423251994 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e9129d55a0 0x55e9129e4380 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fc80000f660 tx=0x7fc800013380 comp rx=0 tx=0).stop 2024-03-08T21:02:23.464 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.481+0000 7fc81df9e740 1 -- 172.21.15.107:0/1423251994 shutdown_connections 2024-03-08T21:02:23.464 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.481+0000 7fc81df9e740 1 --2- 172.21.15.107:0/1423251994 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55e912a45fb0 0x55e912a483d0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:02:23.464 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.481+0000 7fc81df9e740 1 --2- 172.21.15.107:0/1423251994 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55e9129de370 0x55e9129e48c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:02:23.464 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.481+0000 7fc81df9e740 1 --2- 172.21.15.107:0/1423251994 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e9129d55a0 0x55e9129e4380 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:02:23.464 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.481+0000 7fc7fe000640 0 client.15621 ms_handle_reset on v2:172.21.15.149:6838/55290596 2024-03-08T21:02:23.464 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.481+0000 7fc81df9e740 1 --2- 172.21.15.107:0/1423251994 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55e9129d4c80 0x55e9129e3e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:02:23.464 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.481+0000 7fc81df9e740 1 -- 172.21.15.107:0/1423251994 >> 172.21.15.107:0/1423251994 conn(0x55e9129cf5a0 msgr2=0x55e9129d5c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:02:23.464 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.481+0000 7fc81df9e740 1 -- 172.21.15.107:0/1423251994 shutdown_connections 2024-03-08T21:02:23.464 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.481+0000 7fc81df9e740 1 -- 172.21.15.107:0/1423251994 wait complete. 2024-03-08T21:02:23.464 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.481+0000 7fc81df9e740 20 client.15621 trim_cache size 0 max 16384 2024-03-08T21:02:23.465 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:{ 2024-03-08T21:02:23.465 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "message": "", 2024-03-08T21:02:23.465 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "return_code": 0 2024-03-08T21:02:23.465 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:} 2024-03-08T21:02:23.465 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.482+0000 7fc81b200640 1 -- 172.21.15.107:0/1331715485 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7fc7f80a2a90 msgr2=0x7fc7f80a4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:02:23.466 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.482+0000 7fc81b200640 1 --2- 172.21.15.107:0/1331715485 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7fc7f80a2a90 0x7fc7f80a4f50 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x55e9129d4730 tx=0x7fc80400b390 comp rx=0 tx=0).stop 2024-03-08T21:02:23.466 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.482+0000 7fc81b200640 1 -- 172.21.15.107:0/1331715485 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc814105190 msgr2=0x7fc8141028e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:02:23.466 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.482+0000 7fc81b200640 1 --2- 172.21.15.107:0/1331715485 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc814105190 0x7fc8141028e0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fc80c02c710 tx=0x7fc80c035d60 comp rx=0 tx=0).stop 2024-03-08T21:02:23.466 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.482+0000 7fc81b200640 1 -- 172.21.15.107:0/1331715485 shutdown_connections 2024-03-08T21:02:23.466 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.482+0000 7fc81b200640 1 --2- 172.21.15.107:0/1331715485 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7fc7f80a2a90 0x7fc7f80a4f50 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:02:23.466 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.482+0000 7fc81b200640 1 --2- 172.21.15.107:0/1331715485 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc814105190 0x7fc8141028e0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:02:23.466 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.482+0000 7fc81b200640 1 --2- 172.21.15.107:0/1331715485 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc814100e00 0x7fc8141023a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:02:23.466 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.482+0000 7fc81b200640 1 --2- 172.21.15.107:0/1331715485 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc814100440 0x7fc814101e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:02:23.466 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.482+0000 7fc81b200640 1 -- 172.21.15.107:0/1331715485 >> 172.21.15.107:0/1331715485 conn(0x7fc814077040 msgr2=0x7fc814103fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:02:23.466 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.483+0000 7fc81b200640 1 -- 172.21.15.107:0/1331715485 shutdown_connections 2024-03-08T21:02:23.466 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:23.483+0000 7fc81b200640 1 -- 172.21.15.107:0/1331715485 wait complete. 2024-03-08T21:02:23.877 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:02:23 smithi204 ceph-mon[34935]: pgmap v59: 97 pgs: 97 active+clean; 1.5 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 11 KiB/s wr, 355 op/s 2024-03-08T21:02:23.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:02:23 smithi107 ceph-mon[31381]: pgmap v59: 97 pgs: 97 active+clean; 1.5 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 11 KiB/s wr, 355 op/s 2024-03-08T21:02:24.014 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:02:23 smithi149 ceph-mon[35042]: pgmap v59: 97 pgs: 97 active+clean; 1.5 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 11 KiB/s wr, 355 op/s 2024-03-08T21:02:25.845 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:02:25 smithi107 ceph-mon[31381]: pgmap v60: 97 pgs: 97 active+clean; 1.5 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 16 KiB/s wr, 363 op/s 2024-03-08T21:02:25.876 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:02:25 smithi204 ceph-mon[34935]: pgmap v60: 97 pgs: 97 active+clean; 1.5 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 16 KiB/s wr, 363 op/s 2024-03-08T21:02:26.013 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:02:25 smithi149 ceph-mon[35042]: pgmap v60: 97 pgs: 97 active+clean; 1.5 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 16 KiB/s wr, 363 op/s 2024-03-08T21:02:27.876 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:02:27 smithi204 ceph-mon[34935]: pgmap v61: 97 pgs: 97 active+clean; 1.5 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 319 KiB/s rd, 10 KiB/s wr, 91 op/s 2024-03-08T21:02:27.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:02:27 smithi107 ceph-mon[31381]: pgmap v61: 97 pgs: 97 active+clean; 1.5 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 319 KiB/s rd, 10 KiB/s wr, 91 op/s 2024-03-08T21:02:28.013 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:02:27 smithi149 ceph-mon[35042]: pgmap v61: 97 pgs: 97 active+clean; 1.5 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 319 KiB/s rd, 10 KiB/s wr, 91 op/s 2024-03-08T21:02:29.876 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:02:29 smithi204 ceph-mon[34935]: pgmap v62: 97 pgs: 97 active+clean; 1.5 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 319 KiB/s rd, 10 KiB/s wr, 91 op/s 2024-03-08T21:02:29.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:02:29 smithi107 ceph-mon[31381]: pgmap v62: 97 pgs: 97 active+clean; 1.5 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 319 KiB/s rd, 10 KiB/s wr, 91 op/s 2024-03-08T21:02:30.013 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:02:29 smithi149 ceph-mon[35042]: pgmap v62: 97 pgs: 97 active+clean; 1.5 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 319 KiB/s rd, 10 KiB/s wr, 91 op/s 2024-03-08T21:02:31.876 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:02:31 smithi204 ceph-mon[34935]: pgmap v63: 97 pgs: 97 active+clean; 1.5 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 322 KiB/s rd, 10 KiB/s wr, 95 op/s 2024-03-08T21:02:31.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:02:31 smithi107 ceph-mon[31381]: pgmap v63: 97 pgs: 97 active+clean; 1.5 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 322 KiB/s rd, 10 KiB/s wr, 95 op/s 2024-03-08T21:02:32.013 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:02:31 smithi149 ceph-mon[35042]: pgmap v63: 97 pgs: 97 active+clean; 1.5 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 322 KiB/s rd, 10 KiB/s wr, 95 op/s 2024-03-08T21:02:33.876 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:02:33 smithi204 ceph-mon[34935]: pgmap v64: 97 pgs: 97 active+clean; 1.5 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 4.4 KiB/s wr, 12 op/s 2024-03-08T21:02:33.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:02:33 smithi107 ceph-mon[31381]: pgmap v64: 97 pgs: 97 active+clean; 1.5 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 4.4 KiB/s wr, 12 op/s 2024-03-08T21:02:34.013 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:02:33 smithi149 ceph-mon[35042]: pgmap v64: 97 pgs: 97 active+clean; 1.5 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 4.4 KiB/s wr, 12 op/s 2024-03-08T21:02:35.876 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:02:35 smithi204 ceph-mon[34935]: pgmap v65: 97 pgs: 97 active+clean; 1.5 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 5.3 KiB/s wr, 17 op/s 2024-03-08T21:02:35.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:02:35 smithi107 ceph-mon[31381]: pgmap v65: 97 pgs: 97 active+clean; 1.5 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 5.3 KiB/s wr, 17 op/s 2024-03-08T21:02:36.013 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:02:35 smithi149 ceph-mon[35042]: pgmap v65: 97 pgs: 97 active+clean; 1.5 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 5.3 KiB/s wr, 17 op/s 2024-03-08T21:02:36.625 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.644+0000 7fc3c2400640 1 -- 172.21.15.107:0/1061907633 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc3bc109580 msgr2=0x7fc3bc109960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:02:36.625 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.644+0000 7fc3c2400640 1 --2- 172.21.15.107:0/1061907633 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc3bc109580 0x7fc3bc109960 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fc3ac009900 tx=0x7fc3ac02bc30 comp rx=0 tx=0).stop 2024-03-08T21:02:36.626 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.644+0000 7fc3c2400640 1 -- 172.21.15.107:0/1061907633 shutdown_connections 2024-03-08T21:02:36.626 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.644+0000 7fc3c2400640 1 --2- 172.21.15.107:0/1061907633 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc3bc103e80 0x7fc3bc110340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:02:36.626 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.644+0000 7fc3c2400640 1 --2- 172.21.15.107:0/1061907633 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc3bc1034c0 0x7fc3bc103940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:02:36.626 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.644+0000 7fc3c2400640 1 --2- 172.21.15.107:0/1061907633 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc3bc109580 0x7fc3bc109960 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:02:36.626 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.644+0000 7fc3c2400640 1 -- 172.21.15.107:0/1061907633 >> 172.21.15.107:0/1061907633 conn(0x7fc3bc0fef70 msgr2=0x7fc3bc101390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:02:36.626 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.644+0000 7fc3c2400640 1 -- 172.21.15.107:0/1061907633 shutdown_connections 2024-03-08T21:02:36.627 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.644+0000 7fc3c2400640 1 -- 172.21.15.107:0/1061907633 wait complete. 2024-03-08T21:02:36.628 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.645+0000 7fc3c2400640 1 Processor -- start 2024-03-08T21:02:36.629 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.645+0000 7fc3c2400640 1 -- start start 2024-03-08T21:02:36.629 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.646+0000 7fc3c2400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc3bc1034c0 0x7fc3bc19f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:02:36.629 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.646+0000 7fc3c2400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc3bc103e80 0x7fc3bc19fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:02:36.629 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.646+0000 7fc3c2400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc3bc109580 0x7fc3bc199940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:02:36.629 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.646+0000 7fc3c2400640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3bc113740 con 0x7fc3bc103e80 2024-03-08T21:02:36.629 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.646+0000 7fc3c2400640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3bc1135c0 con 0x7fc3bc1034c0 2024-03-08T21:02:36.629 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.646+0000 7fc3c2400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3bc1138c0 con 0x7fc3bc109580 2024-03-08T21:02:36.629 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.646+0000 7fc3baa00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc3bc103e80 0x7fc3bc19fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:02:36.629 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.646+0000 7fc3baa00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc3bc103e80 0x7fc3bc19fcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:40838/0 (socket says 172.21.15.107:40838) 2024-03-08T21:02:36.629 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.646+0000 7fc3baa00640 1 -- 172.21.15.107:0/2950389563 learned_addr learned my addr 172.21.15.107:0/2950389563 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:02:36.629 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.646+0000 7fc3bbe00640 1 --2- 172.21.15.107:0/2950389563 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc3bc109580 0x7fc3bc199940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:02:36.629 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.646+0000 7fc3bb400640 1 --2- 172.21.15.107:0/2950389563 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc3bc1034c0 0x7fc3bc19f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:02:36.630 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.646+0000 7fc3baa00640 1 -- 172.21.15.107:0/2950389563 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc3bc109580 msgr2=0x7fc3bc199940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:02:36.630 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.647+0000 7fc3baa00640 1 --2- 172.21.15.107:0/2950389563 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc3bc109580 0x7fc3bc199940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:02:36.630 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.647+0000 7fc3baa00640 1 -- 172.21.15.107:0/2950389563 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc3bc1034c0 msgr2=0x7fc3bc19f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:02:36.630 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.647+0000 7fc3baa00640 1 --2- 172.21.15.107:0/2950389563 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc3bc1034c0 0x7fc3bc19f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:02:36.630 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.647+0000 7fc3baa00640 1 -- 172.21.15.107:0/2950389563 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3bc19a0c0 con 0x7fc3bc103e80 2024-03-08T21:02:36.630 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.647+0000 7fc3baa00640 1 --2- 172.21.15.107:0/2950389563 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc3bc103e80 0x7fc3bc19fcd0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fc3a800d950 tx=0x7fc3a800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:02:36.630 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.647+0000 7fc3b3e00640 1 -- 172.21.15.107:0/2950389563 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3a8017070 con 0x7fc3bc103e80 2024-03-08T21:02:36.630 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.647+0000 7fc3c2400640 1 -- 172.21.15.107:0/2950389563 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3bc19a3b0 con 0x7fc3bc103e80 2024-03-08T21:02:36.630 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.647+0000 7fc3b3e00640 1 -- 172.21.15.107:0/2950389563 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fc3a8009c30 con 0x7fc3bc103e80 2024-03-08T21:02:36.630 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.647+0000 7fc3b3e00640 1 -- 172.21.15.107:0/2950389563 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3a800f620 con 0x7fc3bc103e80 2024-03-08T21:02:36.630 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.648+0000 7fc3c2400640 1 -- 172.21.15.107:0/2950389563 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3bc1a6650 con 0x7fc3bc103e80 2024-03-08T21:02:36.632 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.652+0000 7fc3b3e00640 1 -- 172.21.15.107:0/2950389563 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) v1 ==== 137422+0+0 (secure 0 0 0) 0x7fc3a80109a0 con 0x7fc3bc103e80 2024-03-08T21:02:36.632 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.652+0000 7fc3c5055740 1 Processor -- start 2024-03-08T21:02:36.632 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.652+0000 7fc3c5055740 1 -- start start 2024-03-08T21:02:36.633 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.652+0000 7fc3c5055740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557da3944c80 0x557da3945060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:02:36.633 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.652+0000 7fc3c5055740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557da39455a0 0x557da394de30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:02:36.634 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.652+0000 7fc3c5055740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557da394e370 0x557da3950760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:02:36.634 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.652+0000 7fc3c5055740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x557da38a83d0 con 0x557da394e370 2024-03-08T21:02:36.634 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.652+0000 7fc3c5055740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x557da38516b0 con 0x557da39455a0 2024-03-08T21:02:36.634 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.652+0000 7fc3c5055740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x557da38a4470 con 0x557da3944c80 2024-03-08T21:02:36.634 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.652+0000 7fc3baa00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557da394e370 0x557da3950760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:02:36.634 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.653+0000 7fc3baa00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557da394e370 0x557da3950760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:40854/0 (socket says 172.21.15.107:40854) 2024-03-08T21:02:36.634 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.653+0000 7fc3baa00640 1 -- 172.21.15.107:0/3600275243 learned_addr learned my addr 172.21.15.107:0/3600275243 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:02:36.634 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.653+0000 7fc3bbe00640 1 --2- 172.21.15.107:0/3600275243 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557da3944c80 0x557da3945060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:02:36.634 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.653+0000 7fc3bb400640 1 --2- 172.21.15.107:0/3600275243 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557da39455a0 0x557da394de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:02:36.636 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.653+0000 7fc3baa00640 1 -- 172.21.15.107:0/3600275243 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557da3944c80 msgr2=0x557da3945060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:02:36.636 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.653+0000 7fc3baa00640 1 --2- 172.21.15.107:0/3600275243 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557da3944c80 0x557da3945060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:02:36.636 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.653+0000 7fc3baa00640 1 -- 172.21.15.107:0/3600275243 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557da39455a0 msgr2=0x557da394de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:02:36.636 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.653+0000 7fc3baa00640 1 --2- 172.21.15.107:0/3600275243 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557da39455a0 0x557da394de30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:02:36.636 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.653+0000 7fc3baa00640 1 -- 172.21.15.107:0/3600275243 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557da3877b50 con 0x557da394e370 2024-03-08T21:02:36.637 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.653+0000 7fc3b3e00640 1 --2- 172.21.15.107:0/2950389563 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7fc3a00a2a10 0x7fc3a00a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:02:36.637 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.653+0000 7fc3bbe00640 1 --2- 172.21.15.107:0/3600275243 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557da3944c80 0x557da3945060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T21:02:36.637 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.653+0000 7fc3bb400640 1 --2- 172.21.15.107:0/3600275243 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557da39455a0 0x557da394de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T21:02:36.637 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.653+0000 7fc3b3e00640 1 -- 172.21.15.107:0/2950389563 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fc3a80ceb80 con 0x7fc3bc103e80 2024-03-08T21:02:36.637 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.654+0000 7fc3bbe00640 1 --2- 172.21.15.107:0/2950389563 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7fc3a00a2a10 0x7fc3a00a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:02:36.637 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.654+0000 7fc3baa00640 1 --2- 172.21.15.107:0/3600275243 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557da394e370 0x557da3950760 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fc3a80cc230 tx=0x7fc3a8097470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c19934295bfd65ec server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:02:36.637 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.654+0000 7fc3b0c00640 1 -- 172.21.15.107:0/3600275243 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3a8020070 con 0x557da394e370 2024-03-08T21:02:36.637 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.654+0000 7fc3b0c00640 1 -- 172.21.15.107:0/3600275243 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fc3a80d2e50 con 0x557da394e370 2024-03-08T21:02:36.637 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.654+0000 7fc3b0c00640 1 -- 172.21.15.107:0/3600275243 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3a809ab30 con 0x557da394e370 2024-03-08T21:02:36.637 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.655+0000 7fc3c5055740 1 -- 172.21.15.107:0/3600275243 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557da394e370 msgr2=0x557da3950760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:02:36.637 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.655+0000 7fc3c5055740 1 --2- 172.21.15.107:0/3600275243 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557da394e370 0x557da3950760 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fc3a80cc230 tx=0x7fc3a8097470 comp rx=0 tx=0).stop 2024-03-08T21:02:36.637 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.655+0000 7fc3bbe00640 1 --2- 172.21.15.107:0/2950389563 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7fc3a00a2a10 0x7fc3a00a4ed0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fc3b4007470 tx=0x7fc3b4007400 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:02:36.637 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.655+0000 7fc3c5055740 1 -- 172.21.15.107:0/3600275243 shutdown_connections 2024-03-08T21:02:36.637 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.655+0000 7fc3c5055740 1 --2- 172.21.15.107:0/3600275243 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557da394e370 0x557da3950760 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:02:36.638 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.655+0000 7fc3c5055740 1 --2- 172.21.15.107:0/3600275243 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557da39455a0 0x557da394de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:02:36.638 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.655+0000 7fc3c5055740 1 --2- 172.21.15.107:0/3600275243 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557da3944c80 0x557da3945060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:02:36.638 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.655+0000 7fc3c5055740 1 -- 172.21.15.107:0/3600275243 >> 172.21.15.107:0/3600275243 conn(0x557da393f5a0 msgr2=0x557da393f980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:02:36.638 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.655+0000 7fc3c5055740 1 -- 172.21.15.107:0/3600275243 shutdown_connections 2024-03-08T21:02:36.638 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.655+0000 7fc3c5055740 1 -- 172.21.15.107:0/3600275243 wait complete. 2024-03-08T21:02:36.638 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.656+0000 7fc3c5055740 1 -- start start 2024-03-08T21:02:36.638 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.656+0000 7fc3c5055740 1 Processor -- start 2024-03-08T21:02:36.641 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.656+0000 7fc3c5055740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557da3944c80 0x557da394dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:02:36.641 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.656+0000 7fc3c5055740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557da39455a0 0x557da394b050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:02:36.641 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.656+0000 7fc3c5055740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557da394e370 0x557da394b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:02:36.641 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.656+0000 7fc3c5055740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x557da3950fa0 con 0x557da39455a0 2024-03-08T21:02:36.641 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.656+0000 7fc3c5055740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x557da38649f0 con 0x557da394e370 2024-03-08T21:02:36.641 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.656+0000 7fc3c5055740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x557da3951120 con 0x557da3944c80 2024-03-08T21:02:36.641 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.656+0000 7fc3c5055740 10 client.0 _ll_register_callbacks cb 0x557da35bca00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T21:02:36.641 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.657+0000 7fc3baa00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557da39455a0 0x557da394b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:02:36.641 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.657+0000 7fc3baa00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557da39455a0 0x557da394b050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:40870/0 (socket says 172.21.15.107:40870) 2024-03-08T21:02:36.641 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.657+0000 7fc3baa00640 1 -- 172.21.15.107:0/1478712772 learned_addr learned my addr 172.21.15.107:0/1478712772 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:02:36.642 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.657+0000 7fc3bb400640 1 --2- 172.21.15.107:0/1478712772 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557da3944c80 0x557da394dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:02:36.642 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.657+0000 7fc3bbe00640 1 --2- 172.21.15.107:0/1478712772 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557da394e370 0x557da394b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:02:36.642 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.657+0000 7fc3baa00640 1 -- 172.21.15.107:0/1478712772 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557da3944c80 msgr2=0x557da394dcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:02:36.642 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.657+0000 7fc3baa00640 1 --2- 172.21.15.107:0/1478712772 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557da3944c80 0x557da394dcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:02:36.642 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.657+0000 7fc3baa00640 1 -- 172.21.15.107:0/1478712772 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557da394e370 msgr2=0x557da394b5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:02:36.642 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.657+0000 7fc3baa00640 1 --2- 172.21.15.107:0/1478712772 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557da394e370 0x557da394b5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:02:36.642 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.657+0000 7fc3baa00640 1 -- 172.21.15.107:0/1478712772 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557da35bd210 con 0x557da39455a0 2024-03-08T21:02:36.642 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.657+0000 7fc3baa00640 1 --2- 172.21.15.107:0/1478712772 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557da39455a0 0x557da394b050 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fc3a80d0040 tx=0x7fc3a800bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e197d169660efce3 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:02:36.642 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.657+0000 7fc3a5600640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-03-08T21:02:36.642 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.658+0000 7fc3a5600640 1 -- 172.21.15.107:0/1478712772 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3a8020040 con 0x557da39455a0 2024-03-08T21:02:36.642 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.658+0000 7fc3a5600640 1 -- 172.21.15.107:0/1478712772 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fc3a80d28b0 con 0x557da39455a0 2024-03-08T21:02:36.642 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.658+0000 7fc3c5055740 1 -- 172.21.15.107:0/1478712772 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc3a80d2e50 con 0x557da39455a0 2024-03-08T21:02:36.642 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.658+0000 7fc3a5600640 1 -- 172.21.15.107:0/1478712772 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc3a809ab30 con 0x557da39455a0 2024-03-08T21:02:36.643 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.658+0000 7fc3c5055740 10 client.15633 fetch_fsmap learned FSMap version 30 2024-03-08T21:02:36.643 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.658+0000 7fc3c5055740 1 -- 172.21.15.107:0/1478712772 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x557da3689620 con 0x557da39455a0 2024-03-08T21:02:36.643 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.658+0000 7fc3a5600640 1 -- 172.21.15.107:0/1478712772 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fc3a80d2e50 con 0x557da39455a0 2024-03-08T21:02:36.643 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.659+0000 7fc3a5600640 1 -- 172.21.15.107:0/1478712772 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 30) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fc3a80d5020 con 0x557da39455a0 2024-03-08T21:02:36.643 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.659+0000 7fc3c5055740 10 client.15633 fetch_fsmap finished waiting for FSMap version 30 2024-03-08T21:02:36.643 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.659+0000 7fc3c5055740 10 client.15633 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-08T21:02:36.643 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.659+0000 7fc3c5055740 20 client.15633 populate_metadata read hostname 'smithi107' 2024-03-08T21:02:36.643 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.659+0000 7fc3c5055740 1 --2- 172.21.15.107:0/1478712772 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x557da395e090 0x557da3960480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:02:36.643 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.659+0000 7fc3c5055740 4 client.15633 mds_command: new command op to 34475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-08T21:02:36.643 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.659+0000 7fc3c5055740 1 -- 172.21.15.107:0/1478712772 --> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557da38b5910 con 0x557da395e090 2024-03-08T21:02:36.643 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.660+0000 7fc3bb400640 1 --2- 172.21.15.107:0/1478712772 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x557da395e090 0x557da3960480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:02:36.644 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.660+0000 7fc3bb400640 1 --2- 172.21.15.107:0/1478712772 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x557da395e090 0x557da3960480 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7e29c54dd80bbc34 server_cookie=cad6162a2f3d46b0 in_seq=0 out_seq=0 2024-03-08T21:02:36.644 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.660+0000 7fc3a5600640 10 client.15633 ms_handle_connect on v2:172.21.15.149:6838/55290596 2024-03-08T21:02:36.645 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.662+0000 7fc3a5600640 1 -- 172.21.15.107:0/1478712772 <== mds.0 v2:172.21.15.149:6838/55290596 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x557da38b5910 con 0x557da395e090 2024-03-08T21:02:36.645 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.662+0000 7fc3a5600640 10 client.15633 handle_command_reply: tid=0 2024-03-08T21:02:36.645 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.662+0000 7fc3c5055740 1 client.15633 shutdown 2024-03-08T21:02:36.645 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.662+0000 7fc3c5055740 1 -- 172.21.15.107:0/1478712772 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557da39455a0 msgr2=0x557da394b050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:02:36.645 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.662+0000 7fc3c5055740 1 --2- 172.21.15.107:0/1478712772 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557da39455a0 0x557da394b050 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fc3a80d0040 tx=0x7fc3a800bf60 comp rx=0 tx=0).stop 2024-03-08T21:02:36.645 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.662+0000 7fc3c5055740 1 -- 172.21.15.107:0/1478712772 shutdown_connections 2024-03-08T21:02:36.645 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.662+0000 7fc3c5055740 1 --2- 172.21.15.107:0/1478712772 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x557da395e090 0x557da3960480 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:02:36.645 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.662+0000 7fc3c5055740 1 --2- 172.21.15.107:0/1478712772 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557da394e370 0x557da394b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:02:36.645 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.662+0000 7fc3c5055740 1 --2- 172.21.15.107:0/1478712772 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557da39455a0 0x557da394b050 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:02:36.645 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.662+0000 7fc3c5055740 1 --2- 172.21.15.107:0/1478712772 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557da3944c80 0x557da394dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:02:36.645 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.662+0000 7fc3a5600640 0 client.15633 ms_handle_reset on v2:172.21.15.149:6838/55290596 2024-03-08T21:02:36.645 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.662+0000 7fc3c5055740 1 -- 172.21.15.107:0/1478712772 >> 172.21.15.107:0/1478712772 conn(0x557da393f5a0 msgr2=0x557da39412f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:02:36.645 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.663+0000 7fc3c5055740 1 -- 172.21.15.107:0/1478712772 shutdown_connections 2024-03-08T21:02:36.645 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.663+0000 7fc3c5055740 1 -- 172.21.15.107:0/1478712772 wait complete. 2024-03-08T21:02:36.646 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.663+0000 7fc3c5055740 20 client.15633 trim_cache size 0 max 16384 2024-03-08T21:02:36.665 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.683+0000 7fc3c5055740 1 Processor -- start 2024-03-08T21:02:36.665 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.683+0000 7fc3c5055740 1 -- start start 2024-03-08T21:02:36.665 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.683+0000 7fc3c5055740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557da3944c80 0x557da3964af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:02:36.665 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.683+0000 7fc3c5055740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557da39455a0 0x557da39410b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:02:36.665 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.683+0000 7fc3c5055740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557da394e370 0x557da394d780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:02:36.666 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.683+0000 7fc3c5055740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3bc1aec40 con 0x557da394e370 2024-03-08T21:02:36.666 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.683+0000 7fc3c5055740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3bc1b05c0 con 0x557da3944c80 2024-03-08T21:02:36.666 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.683+0000 7fc3c5055740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3bc1af2a0 con 0x557da39455a0 2024-03-08T21:02:36.666 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.684+0000 7fc3bbe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557da394e370 0x557da394d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:02:36.666 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.684+0000 7fc3bbe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557da394e370 0x557da394d780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:40880/0 (socket says 172.21.15.107:40880) 2024-03-08T21:02:36.666 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.684+0000 7fc3bbe00640 1 -- 172.21.15.107:0/2645127453 learned_addr learned my addr 172.21.15.107:0/2645127453 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:02:36.666 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.684+0000 7fc3bb400640 1 --2- 172.21.15.107:0/2645127453 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557da3944c80 0x557da3964af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:02:36.666 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.684+0000 7fc3baa00640 1 --2- 172.21.15.107:0/2645127453 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557da39455a0 0x557da39410b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:02:36.666 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.684+0000 7fc3bbe00640 1 -- 172.21.15.107:0/2645127453 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557da39455a0 msgr2=0x557da39410b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:02:36.666 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.684+0000 7fc3bbe00640 1 --2- 172.21.15.107:0/2645127453 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557da39455a0 0x557da39410b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:02:36.666 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.684+0000 7fc3bbe00640 1 -- 172.21.15.107:0/2645127453 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557da3944c80 msgr2=0x557da3964af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:02:36.666 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.684+0000 7fc3bbe00640 1 --2- 172.21.15.107:0/2645127453 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557da3944c80 0x557da3964af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:02:36.667 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.684+0000 7fc3bbe00640 1 -- 172.21.15.107:0/2645127453 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557da393f3d0 con 0x557da394e370 2024-03-08T21:02:36.667 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.685+0000 7fc3bbe00640 1 --2- 172.21.15.107:0/2645127453 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557da394e370 0x557da394d780 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fc3b40118f0 tx=0x7fc3b4013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67b6b05d77108628 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:02:36.670 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.685+0000 7fc3a7e00640 1 -- 172.21.15.107:0/2645127453 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3b401eb40 con 0x557da394e370 2024-03-08T21:02:36.671 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.685+0000 7fc3a7e00640 1 -- 172.21.15.107:0/2645127453 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fc3b401ece0 con 0x557da394e370 2024-03-08T21:02:36.671 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.685+0000 7fc3a7e00640 1 -- 172.21.15.107:0/2645127453 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3b4027630 con 0x557da394e370 2024-03-08T21:02:36.671 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.686+0000 7fc3c5055740 1 -- 172.21.15.107:0/2645127453 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557da394e370 msgr2=0x557da394d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:02:36.671 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.686+0000 7fc3c5055740 1 --2- 172.21.15.107:0/2645127453 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557da394e370 0x557da394d780 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fc3b40118f0 tx=0x7fc3b4013000 comp rx=0 tx=0).stop 2024-03-08T21:02:36.671 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.686+0000 7fc3c5055740 1 -- 172.21.15.107:0/2645127453 shutdown_connections 2024-03-08T21:02:36.671 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.686+0000 7fc3c5055740 1 --2- 172.21.15.107:0/2645127453 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557da394e370 0x557da394d780 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:02:36.671 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.686+0000 7fc3c5055740 1 --2- 172.21.15.107:0/2645127453 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557da39455a0 0x557da39410b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:02:36.671 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.686+0000 7fc3c5055740 1 --2- 172.21.15.107:0/2645127453 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557da3944c80 0x557da3964af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:02:36.671 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.686+0000 7fc3c5055740 1 -- 172.21.15.107:0/2645127453 >> 172.21.15.107:0/2645127453 conn(0x557da393f5a0 msgr2=0x557da39409a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:02:36.671 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.686+0000 7fc3c5055740 1 -- 172.21.15.107:0/2645127453 shutdown_connections 2024-03-08T21:02:36.671 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.686+0000 7fc3c5055740 1 -- 172.21.15.107:0/2645127453 wait complete. 2024-03-08T21:02:36.671 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.686+0000 7fc3c5055740 1 -- start start 2024-03-08T21:02:36.671 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.687+0000 7fc3c5055740 1 Processor -- start 2024-03-08T21:02:36.672 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.687+0000 7fc3c5055740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557da3944c80 0x557da3954080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:02:36.672 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.687+0000 7fc3c5055740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557da39455a0 0x557da39545c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:02:36.672 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.687+0000 7fc3c5055740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557da394e370 0x557da39b4280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:02:36.672 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.687+0000 7fc3c5055740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3bc1aff60 con 0x557da394e370 2024-03-08T21:02:36.672 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.687+0000 7fc3c5055740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3bc1af900 con 0x557da3944c80 2024-03-08T21:02:36.672 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.687+0000 7fc3c5055740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3bc1acee0 con 0x557da39455a0 2024-03-08T21:02:36.672 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.687+0000 7fc3c5055740 10 client.0 _ll_register_callbacks cb 0x7fc3a80d25a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T21:02:36.672 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.687+0000 7fc3bbe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557da394e370 0x557da39b4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:02:36.672 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.688+0000 7fc3bbe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557da394e370 0x557da39b4280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:40896/0 (socket says 172.21.15.107:40896) 2024-03-08T21:02:36.672 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.688+0000 7fc3bbe00640 1 -- 172.21.15.107:0/3945702013 learned_addr learned my addr 172.21.15.107:0/3945702013 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:02:36.672 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.688+0000 7fc3baa00640 1 --2- 172.21.15.107:0/3945702013 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557da39455a0 0x557da39545c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:02:36.672 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.688+0000 7fc3bb400640 1 --2- 172.21.15.107:0/3945702013 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557da3944c80 0x557da3954080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:02:36.672 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.688+0000 7fc3bbe00640 1 -- 172.21.15.107:0/3945702013 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557da39455a0 msgr2=0x557da39545c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:02:36.672 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.688+0000 7fc3bbe00640 1 --2- 172.21.15.107:0/3945702013 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557da39455a0 0x557da39545c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:02:36.673 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.688+0000 7fc3bbe00640 1 -- 172.21.15.107:0/3945702013 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557da3944c80 msgr2=0x557da3954080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:02:36.673 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.688+0000 7fc3bbe00640 1 --2- 172.21.15.107:0/3945702013 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557da3944c80 0x557da3954080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:02:36.673 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.688+0000 7fc3bbe00640 1 -- 172.21.15.107:0/3945702013 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557da3954eb0 con 0x557da394e370 2024-03-08T21:02:36.673 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.688+0000 7fc3bb400640 1 --2- 172.21.15.107:0/3945702013 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557da3944c80 0x557da3954080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T21:02:36.673 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.688+0000 7fc3bbe00640 1 --2- 172.21.15.107:0/3945702013 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557da394e370 0x557da39b4280 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fc3b4010c30 tx=0x7fc3b401eb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fbeb2ae42905ac0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:02:36.673 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.688+0000 7fc3a5600640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-03-08T21:02:36.673 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.689+0000 7fc3a5600640 1 -- 172.21.15.107:0/3945702013 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3b4035020 con 0x557da394e370 2024-03-08T21:02:36.673 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.689+0000 7fc3a5600640 1 -- 172.21.15.107:0/3945702013 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fc3b4004c40 con 0x557da394e370 2024-03-08T21:02:36.673 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.689+0000 7fc3c5055740 1 -- 172.21.15.107:0/3945702013 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc3b401ece0 con 0x557da394e370 2024-03-08T21:02:36.673 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.689+0000 7fc3a5600640 1 -- 172.21.15.107:0/3945702013 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc3b4040050 con 0x557da394e370 2024-03-08T21:02:36.673 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.689+0000 7fc3c5055740 10 client.15639 fetch_fsmap learned FSMap version 30 2024-03-08T21:02:36.673 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.689+0000 7fc3c5055740 1 -- 172.21.15.107:0/3945702013 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x557da39b5900 con 0x557da394e370 2024-03-08T21:02:36.673 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.689+0000 7fc3a5600640 1 -- 172.21.15.107:0/3945702013 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fc3b401ece0 con 0x557da394e370 2024-03-08T21:02:36.673 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.689+0000 7fc3a5600640 1 -- 172.21.15.107:0/3945702013 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 30) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fc3b4030030 con 0x557da394e370 2024-03-08T21:02:36.673 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.689+0000 7fc3c5055740 10 client.15639 fetch_fsmap finished waiting for FSMap version 30 2024-03-08T21:02:36.674 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.689+0000 7fc3c5055740 10 client.15639 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-08T21:02:36.674 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.690+0000 7fc3c5055740 20 client.15639 populate_metadata read hostname 'smithi107' 2024-03-08T21:02:36.674 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.690+0000 7fc3c5055740 1 --2- 172.21.15.107:0/3945702013 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x557da39b5f40 0x557da39b8360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:02:36.674 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.690+0000 7fc3c5055740 4 client.15639 mds_command: new command op to 34475 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-08T21:02:36.674 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.690+0000 7fc3c5055740 1 -- 172.21.15.107:0/3945702013 --> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x557da38afc60 con 0x557da39b5f40 2024-03-08T21:02:36.674 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.690+0000 7fc3bb400640 1 --2- 172.21.15.107:0/3945702013 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x557da39b5f40 0x557da39b8360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:02:36.675 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.691+0000 7fc3bb400640 1 --2- 172.21.15.107:0/3945702013 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x557da39b5f40 0x557da39b8360 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1dda354eca5b361c server_cookie=1b3d027597e575b4 in_seq=0 out_seq=0 2024-03-08T21:02:36.675 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.691+0000 7fc3a5600640 10 client.15639 ms_handle_connect on v2:172.21.15.149:6838/55290596 2024-03-08T21:02:36.694 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.711+0000 7fc3a5600640 1 -- 172.21.15.107:0/3945702013 <== mds.0 v2:172.21.15.149:6838/55290596 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x557da38afc60 con 0x557da39b5f40 2024-03-08T21:02:36.694 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.711+0000 7fc3a5600640 10 client.15639 handle_command_reply: tid=0 2024-03-08T21:02:36.694 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.711+0000 7fc3c5055740 1 client.15639 shutdown 2024-03-08T21:02:36.695 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.711+0000 7fc3c5055740 1 -- 172.21.15.107:0/3945702013 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557da394e370 msgr2=0x557da39b4280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:02:36.695 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.711+0000 7fc3c5055740 1 --2- 172.21.15.107:0/3945702013 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557da394e370 0x557da39b4280 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fc3b4010c30 tx=0x7fc3b401eb70 comp rx=0 tx=0).stop 2024-03-08T21:02:36.695 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.712+0000 7fc3c5055740 1 -- 172.21.15.107:0/3945702013 shutdown_connections 2024-03-08T21:02:36.695 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.712+0000 7fc3c5055740 1 --2- 172.21.15.107:0/3945702013 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557da394e370 0x557da39b4280 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:02:36.695 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.712+0000 7fc3c5055740 1 --2- 172.21.15.107:0/3945702013 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557da39455a0 0x557da39545c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:02:36.695 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.712+0000 7fc3c5055740 1 --2- 172.21.15.107:0/3945702013 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x557da39b5f40 0x557da39b8360 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:02:36.695 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.712+0000 7fc3c5055740 1 --2- 172.21.15.107:0/3945702013 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557da3944c80 0x557da3954080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:02:36.695 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.712+0000 7fc3c5055740 1 -- 172.21.15.107:0/3945702013 >> 172.21.15.107:0/3945702013 conn(0x557da393f5a0 msgr2=0x557da393e6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:02:36.695 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.712+0000 7fc3a5600640 0 client.15639 ms_handle_reset on v2:172.21.15.149:6838/55290596 2024-03-08T21:02:36.695 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.712+0000 7fc3c5055740 1 -- 172.21.15.107:0/3945702013 shutdown_connections 2024-03-08T21:02:36.695 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.712+0000 7fc3c5055740 1 -- 172.21.15.107:0/3945702013 wait complete. 2024-03-08T21:02:36.695 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.712+0000 7fc3c5055740 20 client.15639 trim_cache size 0 max 16384 2024-03-08T21:02:36.696 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:{ 2024-03-08T21:02:36.696 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "message": "", 2024-03-08T21:02:36.696 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "return_code": 0 2024-03-08T21:02:36.696 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:} 2024-03-08T21:02:36.697 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.713+0000 7fc3c2400640 1 -- 172.21.15.107:0/2950389563 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7fc3a00a2a10 msgr2=0x7fc3a00a4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:02:36.697 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.713+0000 7fc3c2400640 1 --2- 172.21.15.107:0/2950389563 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7fc3a00a2a10 0x7fc3a00a4ed0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fc3b4007470 tx=0x7fc3b4007400 comp rx=0 tx=0).stop 2024-03-08T21:02:36.697 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.713+0000 7fc3c2400640 1 -- 172.21.15.107:0/2950389563 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc3bc103e80 msgr2=0x7fc3bc19fcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:02:36.697 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.713+0000 7fc3c2400640 1 --2- 172.21.15.107:0/2950389563 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc3bc103e80 0x7fc3bc19fcd0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fc3a800d950 tx=0x7fc3a800de10 comp rx=0 tx=0).stop 2024-03-08T21:02:36.697 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.713+0000 7fc3c2400640 1 -- 172.21.15.107:0/2950389563 shutdown_connections 2024-03-08T21:02:36.697 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.713+0000 7fc3c2400640 1 --2- 172.21.15.107:0/2950389563 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7fc3a00a2a10 0x7fc3a00a4ed0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:02:36.697 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.713+0000 7fc3c2400640 1 --2- 172.21.15.107:0/2950389563 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc3bc109580 0x7fc3bc199940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:02:36.697 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.713+0000 7fc3c2400640 1 --2- 172.21.15.107:0/2950389563 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc3bc103e80 0x7fc3bc19fcd0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:02:36.697 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.713+0000 7fc3c2400640 1 --2- 172.21.15.107:0/2950389563 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc3bc1034c0 0x7fc3bc19f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:02:36.698 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.713+0000 7fc3c2400640 1 -- 172.21.15.107:0/2950389563 >> 172.21.15.107:0/2950389563 conn(0x7fc3bc0fef70 msgr2=0x7fc3bc1004c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:02:36.698 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.713+0000 7fc3c2400640 1 -- 172.21.15.107:0/2950389563 shutdown_connections 2024-03-08T21:02:36.698 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:36.713+0000 7fc3c2400640 1 -- 172.21.15.107:0/2950389563 wait complete. 2024-03-08T21:02:37.876 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:02:37 smithi204 ceph-mon[34935]: pgmap v66: 97 pgs: 97 active+clean; 1.5 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 938 B/s wr, 8 op/s 2024-03-08T21:02:37.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:02:37 smithi107 ceph-mon[31381]: pgmap v66: 97 pgs: 97 active+clean; 1.5 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 938 B/s wr, 8 op/s 2024-03-08T21:02:38.013 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:02:37 smithi149 ceph-mon[35042]: pgmap v66: 97 pgs: 97 active+clean; 1.5 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 938 B/s wr, 8 op/s 2024-03-08T21:02:39.876 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:02:39 smithi204 ceph-mon[34935]: pgmap v67: 97 pgs: 97 active+clean; 1.5 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 938 B/s wr, 8 op/s 2024-03-08T21:02:39.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:02:39 smithi107 ceph-mon[31381]: pgmap v67: 97 pgs: 97 active+clean; 1.5 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 938 B/s wr, 8 op/s 2024-03-08T21:02:40.013 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:02:39 smithi149 ceph-mon[35042]: pgmap v67: 97 pgs: 97 active+clean; 1.5 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 938 B/s wr, 8 op/s 2024-03-08T21:02:41.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:02:41 smithi107 ceph-mon[31381]: pgmap v68: 97 pgs: 97 active+clean; 1.5 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 2.2 KiB/s wr, 14 op/s 2024-03-08T21:02:42.013 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:02:41 smithi149 ceph-mon[35042]: pgmap v68: 97 pgs: 97 active+clean; 1.5 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 2.2 KiB/s wr, 14 op/s 2024-03-08T21:02:42.126 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:02:41 smithi204 ceph-mon[34935]: pgmap v68: 97 pgs: 97 active+clean; 1.5 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 2.2 KiB/s wr, 14 op/s 2024-03-08T21:02:42.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:02:42 smithi107 ceph-mon[31381]: pgmap v69: 97 pgs: 97 active+clean; 1.5 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.2 KiB/s wr, 10 op/s 2024-03-08T21:02:43.013 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:02:42 smithi149 ceph-mon[35042]: pgmap v69: 97 pgs: 97 active+clean; 1.5 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.2 KiB/s wr, 10 op/s 2024-03-08T21:02:43.126 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:02:42 smithi204 ceph-mon[34935]: pgmap v69: 97 pgs: 97 active+clean; 1.5 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.2 KiB/s wr, 10 op/s 2024-03-08T21:02:43.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:02:43 smithi107 ceph-mon[31381]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T21:02:44.013 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:02:43 smithi149 ceph-mon[35042]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T21:02:44.126 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:02:43 smithi204 ceph-mon[34935]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T21:02:44.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:02:44 smithi107 ceph-mon[31381]: pgmap v70: 97 pgs: 97 active+clean; 1.5 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 2.2 KiB/s wr, 14 op/s 2024-03-08T21:02:45.013 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:02:44 smithi149 ceph-mon[35042]: pgmap v70: 97 pgs: 97 active+clean; 1.5 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 2.2 KiB/s wr, 14 op/s 2024-03-08T21:02:45.126 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:02:44 smithi204 ceph-mon[34935]: pgmap v70: 97 pgs: 97 active+clean; 1.5 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 2.2 KiB/s wr, 14 op/s 2024-03-08T21:02:45.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:02:45 smithi107 ceph-mon[31381]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T21:02:45.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:02:45 smithi107 ceph-mon[31381]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T21:02:45.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:02:45 smithi107 ceph-mon[31381]: from='mgr.15375 ' entity='mgr.x' 2024-03-08T21:02:46.013 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:02:45 smithi149 ceph-mon[35042]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T21:02:46.013 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:02:45 smithi149 ceph-mon[35042]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T21:02:46.013 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:02:45 smithi149 ceph-mon[35042]: from='mgr.15375 ' entity='mgr.x' 2024-03-08T21:02:46.126 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:02:45 smithi204 ceph-mon[34935]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T21:02:46.126 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:02:45 smithi204 ceph-mon[34935]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T21:02:46.127 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:02:45 smithi204 ceph-mon[34935]: from='mgr.15375 ' entity='mgr.x' 2024-03-08T21:02:46.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:02:46 smithi107 ceph-mon[31381]: pgmap v71: 97 pgs: 97 active+clean; 1.5 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.3 KiB/s wr, 10 op/s 2024-03-08T21:02:47.013 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:02:46 smithi149 ceph-mon[35042]: pgmap v71: 97 pgs: 97 active+clean; 1.5 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.3 KiB/s wr, 10 op/s 2024-03-08T21:02:47.126 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:02:46 smithi204 ceph-mon[34935]: pgmap v71: 97 pgs: 97 active+clean; 1.5 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.3 KiB/s wr, 10 op/s 2024-03-08T21:02:49.843 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.860+0000 7f88af200640 1 -- 172.21.15.107:0/3955362305 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f88b00736a0 msgr2=0x7f88b0073a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:02:49.843 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.860+0000 7f88af200640 1 --2- 172.21.15.107:0/3955362305 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f88b00736a0 0x7f88b0073a80 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f88a0009900 tx=0x7f88a002bcd0 comp rx=0 tx=0).stop 2024-03-08T21:02:49.843 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.860+0000 7f88af200640 1 -- 172.21.15.107:0/3955362305 shutdown_connections 2024-03-08T21:02:49.843 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.860+0000 7f88af200640 1 --2- 172.21.15.107:0/3955362305 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f88b010d9a0 0x7f88b010fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:02:49.843 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.860+0000 7f88af200640 1 --2- 172.21.15.107:0/3955362305 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f88b0073fc0 0x7f88b010d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:02:49.843 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.860+0000 7f88af200640 1 --2- 172.21.15.107:0/3955362305 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f88b00736a0 0x7f88b0073a80 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:02:49.843 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.860+0000 7f88af200640 1 -- 172.21.15.107:0/3955362305 >> 172.21.15.107:0/3955362305 conn(0x7f88b00fcc30 msgr2=0x7f88b00ff050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:02:49.843 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.860+0000 7f88af200640 1 -- 172.21.15.107:0/3955362305 shutdown_connections 2024-03-08T21:02:49.843 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.860+0000 7f88af200640 1 -- 172.21.15.107:0/3955362305 wait complete. 2024-03-08T21:02:49.843 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.861+0000 7f88af200640 1 Processor -- start 2024-03-08T21:02:49.844 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.861+0000 7f88af200640 1 -- start start 2024-03-08T21:02:49.844 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.862+0000 7f88af200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f88b00736a0 0x7f88b019f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:02:49.844 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.862+0000 7f88af200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f88b0073fc0 0x7f88b019fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:02:49.844 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.862+0000 7f88af200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f88b010d9a0 0x7f88b0199930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:02:49.844 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.862+0000 7f88af200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88b01135b0 con 0x7f88b010d9a0 2024-03-08T21:02:49.844 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.862+0000 7f88af200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88b0113430 con 0x7f88b00736a0 2024-03-08T21:02:49.844 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.862+0000 7f88af200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88b0113730 con 0x7f88b0073fc0 2024-03-08T21:02:49.844 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.862+0000 7f88ae800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f88b010d9a0 0x7f88b0199930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:02:49.844 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.862+0000 7f88ae800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f88b010d9a0 0x7f88b0199930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:43560/0 (socket says 172.21.15.107:43560) 2024-03-08T21:02:49.844 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.862+0000 7f88ae800640 1 -- 172.21.15.107:0/3866223091 learned_addr learned my addr 172.21.15.107:0/3866223091 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:02:49.844 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.862+0000 7f88ade00640 1 --2- 172.21.15.107:0/3866223091 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f88b00736a0 0x7f88b019f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:02:49.845 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.862+0000 7f88ad400640 1 --2- 172.21.15.107:0/3866223091 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f88b0073fc0 0x7f88b019fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:02:49.845 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.863+0000 7f88ae800640 1 -- 172.21.15.107:0/3866223091 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f88b0073fc0 msgr2=0x7f88b019fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:02:49.845 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.863+0000 7f88ae800640 1 --2- 172.21.15.107:0/3866223091 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f88b0073fc0 0x7f88b019fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:02:49.846 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.863+0000 7f88ae800640 1 -- 172.21.15.107:0/3866223091 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f88b00736a0 msgr2=0x7f88b019f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:02:49.846 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.863+0000 7f88ae800640 1 --2- 172.21.15.107:0/3866223091 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f88b00736a0 0x7f88b019f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:02:49.846 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.863+0000 7f88ae800640 1 -- 172.21.15.107:0/3866223091 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88b019a0b0 con 0x7f88b010d9a0 2024-03-08T21:02:49.846 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.863+0000 7f88ae800640 1 --2- 172.21.15.107:0/3866223091 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f88b010d9a0 0x7f88b0199930 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f88a800eaa0 tx=0x7f88a800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:02:49.846 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.863+0000 7f88a6a00640 1 -- 172.21.15.107:0/3866223091 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88a800cdb0 con 0x7f88b010d9a0 2024-03-08T21:02:49.846 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.863+0000 7f88a6a00640 1 -- 172.21.15.107:0/3866223091 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f88a8005020 con 0x7f88b010d9a0 2024-03-08T21:02:49.846 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.863+0000 7f88af200640 1 -- 172.21.15.107:0/3866223091 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88b019a3a0 con 0x7f88b010d9a0 2024-03-08T21:02:49.846 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.863+0000 7f88a6a00640 1 -- 172.21.15.107:0/3866223091 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88a80116d0 con 0x7f88b010d9a0 2024-03-08T21:02:49.846 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.864+0000 7f88af200640 1 -- 172.21.15.107:0/3866223091 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88b01a6640 con 0x7f88b010d9a0 2024-03-08T21:02:49.848 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.867+0000 7f88b776a740 1 Processor -- start 2024-03-08T21:02:49.848 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.868+0000 7f88a6a00640 1 -- 172.21.15.107:0/3866223091 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) v1 ==== 137422+0+0 (secure 0 0 0) 0x7f88a80051d0 con 0x7f88b010d9a0 2024-03-08T21:02:49.848 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.868+0000 7f88b776a740 1 -- start start 2024-03-08T21:02:49.851 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.868+0000 7f88b776a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55decb96be80 0x55decb96c260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:02:49.851 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.868+0000 7f88b776a740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55decb96c7a0 0x55decb975030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:02:49.851 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.868+0000 7f88b776a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55decb975570 0x55decb977960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:02:49.851 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.868+0000 7f88b776a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55decb8cf3d0 con 0x55decb96be80 2024-03-08T21:02:49.851 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.868+0000 7f88b776a740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55decb8786b0 con 0x55decb96c7a0 2024-03-08T21:02:49.851 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.868+0000 7f88b776a740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55decb8cb470 con 0x55decb975570 2024-03-08T21:02:49.851 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.869+0000 7f88ad400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55decb96be80 0x55decb96c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:02:49.851 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.869+0000 7f88ad400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55decb96be80 0x55decb96c260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:43572/0 (socket says 172.21.15.107:43572) 2024-03-08T21:02:49.852 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.869+0000 7f88ad400640 1 -- 172.21.15.107:0/3022073842 learned_addr learned my addr 172.21.15.107:0/3022073842 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:02:49.852 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.869+0000 7f88ade00640 1 --2- 172.21.15.107:0/3022073842 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55decb96c7a0 0x55decb975030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:02:49.852 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.869+0000 7f88ad400640 1 --2- 172.21.15.107:0/3022073842 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55decb975570 0x55decb977960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:02:49.852 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.869+0000 7f88a6a00640 1 --2- 172.21.15.107:0/3866223091 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f88940a2960 0x7f88940a4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:02:49.852 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.869+0000 7f88ad400640 1 -- 172.21.15.107:0/3022073842 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55decb975570 msgr2=0x55decb977960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:02:49.852 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.869+0000 7f88ad400640 1 --2- 172.21.15.107:0/3022073842 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55decb975570 0x55decb977960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:02:49.852 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.869+0000 7f88ad400640 1 -- 172.21.15.107:0/3022073842 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55decb96c7a0 msgr2=0x55decb975030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:02:49.852 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.869+0000 7f88ad400640 1 --2- 172.21.15.107:0/3022073842 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55decb96c7a0 0x55decb975030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:02:49.852 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.869+0000 7f88ad400640 1 -- 172.21.15.107:0/3022073842 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55decb89eb50 con 0x55decb96be80 2024-03-08T21:02:49.853 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.870+0000 7f88a6a00640 1 -- 172.21.15.107:0/3866223091 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f88a80cf040 con 0x7f88b010d9a0 2024-03-08T21:02:49.853 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.870+0000 7f88ae800640 1 --2- 172.21.15.107:0/3866223091 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f88940a2960 0x7f88940a4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:02:49.853 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.870+0000 7f88ad400640 1 --2- 172.21.15.107:0/3022073842 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55decb96be80 0x55decb96c260 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f889c0058c0 tx=0x7f889c00b690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=596df2411cc0edc4 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:02:49.853 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.870+0000 7f889b000640 1 -- 172.21.15.107:0/3022073842 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f889c019030 con 0x55decb96be80 2024-03-08T21:02:49.853 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.870+0000 7f889b000640 1 -- 172.21.15.107:0/3022073842 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f889c0118f0 con 0x55decb96be80 2024-03-08T21:02:49.853 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.870+0000 7f889b000640 1 -- 172.21.15.107:0/3022073842 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f889c01d820 con 0x55decb96be80 2024-03-08T21:02:49.857 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.871+0000 7f88b776a740 1 -- 172.21.15.107:0/3022073842 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55decb96be80 msgr2=0x55decb96c260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:02:49.857 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.871+0000 7f88b776a740 1 --2- 172.21.15.107:0/3022073842 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55decb96be80 0x55decb96c260 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f889c0058c0 tx=0x7f889c00b690 comp rx=0 tx=0).stop 2024-03-08T21:02:49.857 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.871+0000 7f88ae800640 1 --2- 172.21.15.107:0/3866223091 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f88940a2960 0x7f88940a4e20 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f88a800efd0 tx=0x7f88a8005830 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:02:49.858 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.871+0000 7f88b776a740 1 -- 172.21.15.107:0/3022073842 shutdown_connections 2024-03-08T21:02:49.858 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.871+0000 7f88b776a740 1 --2- 172.21.15.107:0/3022073842 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55decb975570 0x55decb977960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:02:49.858 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.871+0000 7f88b776a740 1 --2- 172.21.15.107:0/3022073842 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55decb96c7a0 0x55decb975030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:02:49.858 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.871+0000 7f88b776a740 1 --2- 172.21.15.107:0/3022073842 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55decb96be80 0x55decb96c260 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:02:49.858 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.871+0000 7f88b776a740 1 -- 172.21.15.107:0/3022073842 >> 172.21.15.107:0/3022073842 conn(0x55decb9667a0 msgr2=0x55decb966b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:02:49.858 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.871+0000 7f88b776a740 1 -- 172.21.15.107:0/3022073842 shutdown_connections 2024-03-08T21:02:49.858 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.871+0000 7f88b776a740 1 -- 172.21.15.107:0/3022073842 wait complete. 2024-03-08T21:02:49.858 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.872+0000 7f88b776a740 1 -- start start 2024-03-08T21:02:49.858 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.872+0000 7f88b776a740 1 Processor -- start 2024-03-08T21:02:49.858 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.872+0000 7f88b776a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55decb96be80 0x55decb974e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:02:49.858 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.872+0000 7f88b776a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55decb96c7a0 0x55decb9721f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:02:49.858 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.872+0000 7f88b776a740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55decb975570 0x55decb972730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:02:49.858 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.872+0000 7f88b776a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55decb9781a0 con 0x55decb96be80 2024-03-08T21:02:49.859 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.873+0000 7f88b776a740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55decb88b9f0 con 0x55decb975570 2024-03-08T21:02:49.859 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.873+0000 7f88b776a740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55decb978320 con 0x55decb96c7a0 2024-03-08T21:02:49.859 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.873+0000 7f88b776a740 10 client.0 _ll_register_callbacks cb 0x55decb5e3a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T21:02:49.859 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.873+0000 7f88ad400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55decb96be80 0x55decb974e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:02:49.859 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.873+0000 7f88ad400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55decb96be80 0x55decb974e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:43578/0 (socket says 172.21.15.107:43578) 2024-03-08T21:02:49.859 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.873+0000 7f88ad400640 1 -- 172.21.15.107:0/1749718081 learned_addr learned my addr 172.21.15.107:0/1749718081 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:02:49.859 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.873+0000 7f88ade00640 1 --2- 172.21.15.107:0/1749718081 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55decb96c7a0 0x55decb9721f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:02:49.859 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.873+0000 7f88ad400640 1 -- 172.21.15.107:0/1749718081 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55decb96c7a0 msgr2=0x55decb9721f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:02:49.859 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.873+0000 7f88ad400640 1 --2- 172.21.15.107:0/1749718081 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55decb96c7a0 0x55decb9721f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:02:49.859 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.873+0000 7f88ad400640 1 -- 172.21.15.107:0/1749718081 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55decb975570 msgr2=0x55decb972730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:02:49.859 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.873+0000 7f88ad400640 1 --2- 172.21.15.107:0/1749718081 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55decb975570 0x55decb972730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:02:49.859 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.873+0000 7f88ad400640 1 -- 172.21.15.107:0/1749718081 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55decb5e4210 con 0x55decb96be80 2024-03-08T21:02:49.859 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.874+0000 7f88ad400640 1 --2- 172.21.15.107:0/1749718081 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55decb96be80 0x55decb974e60 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f889c00bf00 tx=0x7f889c01d4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38b222ec0b5d99ac server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:02:49.859 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.874+0000 7f888f400640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-03-08T21:02:49.860 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.874+0000 7f888f400640 1 -- 172.21.15.107:0/1749718081 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f889c019050 con 0x55decb96be80 2024-03-08T21:02:49.860 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.874+0000 7f888f400640 1 -- 172.21.15.107:0/1749718081 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f889c005040 con 0x55decb96be80 2024-03-08T21:02:49.860 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.874+0000 7f88b776a740 1 -- 172.21.15.107:0/1749718081 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f889c0118f0 con 0x55decb96be80 2024-03-08T21:02:49.860 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.874+0000 7f888f400640 1 -- 172.21.15.107:0/1749718081 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f889c031050 con 0x55decb96be80 2024-03-08T21:02:49.860 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.874+0000 7f88b776a740 10 client.15651 fetch_fsmap learned FSMap version 30 2024-03-08T21:02:49.860 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.874+0000 7f88b776a740 1 -- 172.21.15.107:0/1749718081 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55decb6b0620 con 0x55decb96be80 2024-03-08T21:02:49.860 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.875+0000 7f888f400640 1 -- 172.21.15.107:0/1749718081 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f889c0118f0 con 0x55decb96be80 2024-03-08T21:02:49.860 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.875+0000 7f888f400640 1 -- 172.21.15.107:0/1749718081 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 30) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f889c004ad0 con 0x55decb96be80 2024-03-08T21:02:49.860 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.875+0000 7f88b776a740 10 client.15651 fetch_fsmap finished waiting for FSMap version 30 2024-03-08T21:02:49.860 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.875+0000 7f88b776a740 10 client.15651 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-08T21:02:49.860 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.875+0000 7f88b776a740 20 client.15651 populate_metadata read hostname 'smithi107' 2024-03-08T21:02:49.860 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.875+0000 7f88b776a740 1 --2- 172.21.15.107:0/1749718081 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55decb985290 0x55decb987680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:02:49.860 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.875+0000 7f88b776a740 4 client.15651 mds_command: new command op to 34475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-08T21:02:49.860 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.875+0000 7f88b776a740 1 -- 172.21.15.107:0/1749718081 --> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55decb8dc910 con 0x55decb985290 2024-03-08T21:02:49.861 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.876+0000 7f88ade00640 1 --2- 172.21.15.107:0/1749718081 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55decb985290 0x55decb987680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:02:49.861 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.877+0000 7f88ade00640 1 --2- 172.21.15.107:0/1749718081 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55decb985290 0x55decb987680 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b1baef5488842f6c server_cookie=dba90c46802911af in_seq=0 out_seq=0 2024-03-08T21:02:49.861 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.877+0000 7f888f400640 10 client.15651 ms_handle_connect on v2:172.21.15.149:6838/55290596 2024-03-08T21:02:49.862 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.878+0000 7f888f400640 1 -- 172.21.15.107:0/1749718081 <== mds.0 v2:172.21.15.149:6838/55290596 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55decb8dc910 con 0x55decb985290 2024-03-08T21:02:49.862 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.878+0000 7f888f400640 10 client.15651 handle_command_reply: tid=0 2024-03-08T21:02:49.862 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.878+0000 7f88b776a740 1 client.15651 shutdown 2024-03-08T21:02:49.863 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.879+0000 7f88b776a740 1 -- 172.21.15.107:0/1749718081 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55decb96be80 msgr2=0x55decb974e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:02:49.863 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.879+0000 7f88b776a740 1 --2- 172.21.15.107:0/1749718081 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55decb96be80 0x55decb974e60 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f889c00bf00 tx=0x7f889c01d4b0 comp rx=0 tx=0).stop 2024-03-08T21:02:49.863 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.879+0000 7f88b776a740 1 -- 172.21.15.107:0/1749718081 shutdown_connections 2024-03-08T21:02:49.863 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.879+0000 7f88b776a740 1 --2- 172.21.15.107:0/1749718081 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55decb985290 0x55decb987680 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:02:49.863 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.879+0000 7f88b776a740 1 --2- 172.21.15.107:0/1749718081 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55decb975570 0x55decb972730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:02:49.863 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.879+0000 7f88b776a740 1 --2- 172.21.15.107:0/1749718081 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55decb96c7a0 0x55decb9721f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:02:49.863 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.879+0000 7f88b776a740 1 --2- 172.21.15.107:0/1749718081 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55decb96be80 0x55decb974e60 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:02:49.863 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.879+0000 7f88b776a740 1 -- 172.21.15.107:0/1749718081 >> 172.21.15.107:0/1749718081 conn(0x55decb9667a0 msgr2=0x55decb9684f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:02:49.863 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.879+0000 7f888f400640 0 client.15651 ms_handle_reset on v2:172.21.15.149:6838/55290596 2024-03-08T21:02:49.863 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.879+0000 7f88b776a740 1 -- 172.21.15.107:0/1749718081 shutdown_connections 2024-03-08T21:02:49.863 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.879+0000 7f88b776a740 1 -- 172.21.15.107:0/1749718081 wait complete. 2024-03-08T21:02:49.863 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.879+0000 7f88b776a740 20 client.15651 trim_cache size 0 max 16384 2024-03-08T21:02:49.875 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.893+0000 7f88b776a740 1 Processor -- start 2024-03-08T21:02:49.875 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.893+0000 7f88b776a740 1 -- start start 2024-03-08T21:02:49.876 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.894+0000 7f88b776a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55decb96be80 0x55decb97c560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:02:49.876 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.894+0000 7f88b776a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55decb96c7a0 0x55decb97caa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:02:49.876 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.894+0000 7f88b776a740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55decb975570 0x55decb96d410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:02:49.876 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.894+0000 7f88b776a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88b01aecf0 con 0x55decb96c7a0 2024-03-08T21:02:49.876 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.894+0000 7f88b776a740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88b01b0670 con 0x55decb975570 2024-03-08T21:02:49.876 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.894+0000 7f88b776a740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88b01af350 con 0x55decb96be80 2024-03-08T21:02:49.876 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.894+0000 7f88ade00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55decb96c7a0 0x55decb97caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:02:49.876 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.894+0000 7f88ade00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55decb96c7a0 0x55decb97caa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:43586/0 (socket says 172.21.15.107:43586) 2024-03-08T21:02:49.876 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.894+0000 7f88ade00640 1 -- 172.21.15.107:0/3888767143 learned_addr learned my addr 172.21.15.107:0/3888767143 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:02:49.876 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.894+0000 7f88ad400640 1 --2- 172.21.15.107:0/3888767143 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55decb96be80 0x55decb97c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:02:49.876 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.894+0000 7f88ad400640 1 --2- 172.21.15.107:0/3888767143 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55decb975570 0x55decb96d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:02:49.876 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.895+0000 7f88ade00640 1 -- 172.21.15.107:0/3888767143 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55decb96be80 msgr2=0x55decb97c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:02:49.876 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.895+0000 7f88ade00640 1 --2- 172.21.15.107:0/3888767143 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55decb96be80 0x55decb97c560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:02:49.877 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.895+0000 7f88ade00640 1 -- 172.21.15.107:0/3888767143 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55decb975570 msgr2=0x55decb96d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:02:49.877 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.895+0000 7f88ade00640 1 --2- 172.21.15.107:0/3888767143 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55decb975570 0x55decb96d410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:02:49.877 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.895+0000 7f88ade00640 1 -- 172.21.15.107:0/3888767143 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55decb977d20 con 0x55decb96c7a0 2024-03-08T21:02:49.877 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.895+0000 7f88ade00640 1 --2- 172.21.15.107:0/3888767143 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55decb96c7a0 0x55decb97caa0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f88a0006010 tx=0x7f88a0004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ad9c30771c0fced server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:02:49.877 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:02:49 smithi204 ceph-mon[34935]: pgmap v72: 97 pgs: 97 active+clean; 1.5 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.3 KiB/s wr, 10 op/s 2024-03-08T21:02:49.880 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.895+0000 7f889a600640 1 -- 172.21.15.107:0/3888767143 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88a00043c0 con 0x55decb96c7a0 2024-03-08T21:02:49.880 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.895+0000 7f889a600640 1 -- 172.21.15.107:0/3888767143 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f88a002cd30 con 0x55decb96c7a0 2024-03-08T21:02:49.880 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.895+0000 7f889a600640 1 -- 172.21.15.107:0/3888767143 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88a00046c0 con 0x55decb96c7a0 2024-03-08T21:02:49.880 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.896+0000 7f88b776a740 1 -- 172.21.15.107:0/3888767143 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55decb96c7a0 msgr2=0x55decb97caa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:02:49.880 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.896+0000 7f88b776a740 1 --2- 172.21.15.107:0/3888767143 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55decb96c7a0 0x55decb97caa0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f88a0006010 tx=0x7f88a0004040 comp rx=0 tx=0).stop 2024-03-08T21:02:49.880 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.896+0000 7f88b776a740 1 -- 172.21.15.107:0/3888767143 shutdown_connections 2024-03-08T21:02:49.880 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.896+0000 7f88b776a740 1 --2- 172.21.15.107:0/3888767143 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55decb975570 0x55decb96d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:02:49.880 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.896+0000 7f88b776a740 1 --2- 172.21.15.107:0/3888767143 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55decb96c7a0 0x55decb97caa0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:02:49.880 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.896+0000 7f88b776a740 1 --2- 172.21.15.107:0/3888767143 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55decb96be80 0x55decb97c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:02:49.880 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.896+0000 7f88b776a740 1 -- 172.21.15.107:0/3888767143 >> 172.21.15.107:0/3888767143 conn(0x55decb9667a0 msgr2=0x55decb96cd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:02:49.880 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.896+0000 7f88b776a740 1 -- 172.21.15.107:0/3888767143 shutdown_connections 2024-03-08T21:02:49.880 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.897+0000 7f88b776a740 1 -- 172.21.15.107:0/3888767143 wait complete. 2024-03-08T21:02:49.881 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.897+0000 7f88b776a740 1 -- start start 2024-03-08T21:02:49.881 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.897+0000 7f88b776a740 1 Processor -- start 2024-03-08T21:02:49.881 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.898+0000 7f88b776a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55decb96be80 0x55decb9cf750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:02:49.881 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.898+0000 7f88b776a740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55decb96c7a0 0x55decb9cfc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:02:49.881 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.898+0000 7f88b776a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55decb975570 0x55decb9d4770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:02:49.881 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.898+0000 7f88b776a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88b01b0010 con 0x55decb975570 2024-03-08T21:02:49.881 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.898+0000 7f88b776a740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88b01af9b0 con 0x55decb96c7a0 2024-03-08T21:02:49.881 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.898+0000 7f88b776a740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88b01acf90 con 0x55decb96be80 2024-03-08T21:02:49.881 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.898+0000 7f88b776a740 10 client.0 _ll_register_callbacks cb 0x7f889c00b4f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T21:02:49.881 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.898+0000 7f88ad400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55decb975570 0x55decb9d4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:02:49.881 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.898+0000 7f88ad400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55decb975570 0x55decb9d4770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:43594/0 (socket says 172.21.15.107:43594) 2024-03-08T21:02:49.881 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.898+0000 7f88ad400640 1 -- 172.21.15.107:0/2159407695 learned_addr learned my addr 172.21.15.107:0/2159407695 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:02:49.881 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.898+0000 7f88ad400640 1 --2- 172.21.15.107:0/2159407695 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55decb96be80 0x55decb9cf750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:02:49.882 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.898+0000 7f88ade00640 1 --2- 172.21.15.107:0/2159407695 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55decb96c7a0 0x55decb9cfc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:02:49.882 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.899+0000 7f88ad400640 1 -- 172.21.15.107:0/2159407695 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55decb96be80 msgr2=0x55decb9cf750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:02:49.882 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.899+0000 7f88ad400640 1 --2- 172.21.15.107:0/2159407695 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55decb96be80 0x55decb9cf750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:02:49.882 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.899+0000 7f88ad400640 1 -- 172.21.15.107:0/2159407695 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55decb96c7a0 msgr2=0x55decb9cfc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:02:49.882 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.899+0000 7f88ad400640 1 --2- 172.21.15.107:0/2159407695 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55decb96c7a0 0x55decb9cfc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:02:49.882 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.899+0000 7f88ad400640 1 -- 172.21.15.107:0/2159407695 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55decb9d4e60 con 0x55decb975570 2024-03-08T21:02:49.882 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.899+0000 7f88ad400640 1 --2- 172.21.15.107:0/2159407695 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55decb975570 0x55decb9d4770 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f889c00b920 tx=0x7f889c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36830d9a51d8b960 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:02:49.882 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.899+0000 7f888f400640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-03-08T21:02:49.882 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.899+0000 7f888f400640 1 -- 172.21.15.107:0/2159407695 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f889c019050 con 0x55decb975570 2024-03-08T21:02:49.883 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.899+0000 7f888f400640 1 -- 172.21.15.107:0/2159407695 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f889c01da20 con 0x55decb975570 2024-03-08T21:02:49.883 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.899+0000 7f88b776a740 1 -- 172.21.15.107:0/2159407695 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f88a002cd30 con 0x55decb975570 2024-03-08T21:02:49.884 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.900+0000 7f888f400640 1 -- 172.21.15.107:0/2159407695 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f889c011650 con 0x55decb975570 2024-03-08T21:02:49.884 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.900+0000 7f88b776a740 10 client.15657 fetch_fsmap learned FSMap version 30 2024-03-08T21:02:49.884 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.900+0000 7f88b776a740 1 -- 172.21.15.107:0/2159407695 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55decb9dc2a0 con 0x55decb975570 2024-03-08T21:02:49.885 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.900+0000 7f888f400640 1 -- 172.21.15.107:0/2159407695 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f88a002cd30 con 0x55decb975570 2024-03-08T21:02:49.885 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.900+0000 7f888f400640 1 -- 172.21.15.107:0/2159407695 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 30) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f889c004fc0 con 0x55decb975570 2024-03-08T21:02:49.885 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.900+0000 7f88b776a740 10 client.15657 fetch_fsmap finished waiting for FSMap version 30 2024-03-08T21:02:49.885 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.900+0000 7f88b776a740 10 client.15657 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-08T21:02:49.885 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.900+0000 7f88b776a740 20 client.15657 populate_metadata read hostname 'smithi107' 2024-03-08T21:02:49.885 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.900+0000 7f88b776a740 1 --2- 172.21.15.107:0/2159407695 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55decb9dc800 0x55decb9debf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:02:49.885 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.900+0000 7f88b776a740 4 client.15657 mds_command: new command op to 34475 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-08T21:02:49.885 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.900+0000 7f88b776a740 1 -- 172.21.15.107:0/2159407695 --> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55decb8d6c60 con 0x55decb9dc800 2024-03-08T21:02:49.885 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.901+0000 7f88ade00640 1 --2- 172.21.15.107:0/2159407695 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55decb9dc800 0x55decb9debf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:02:49.885 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.902+0000 7f88ade00640 1 --2- 172.21.15.107:0/2159407695 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55decb9dc800 0x55decb9debf0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=60ab9fbd445b8f09 server_cookie=c680a38234de25ae in_seq=0 out_seq=0 2024-03-08T21:02:49.885 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.902+0000 7f888f400640 10 client.15657 ms_handle_connect on v2:172.21.15.149:6838/55290596 2024-03-08T21:02:49.892 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.911+0000 7f888f400640 1 -- 172.21.15.107:0/2159407695 <== mds.0 v2:172.21.15.149:6838/55290596 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55decb8d6c60 con 0x55decb9dc800 2024-03-08T21:02:49.892 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.911+0000 7f888f400640 10 client.15657 handle_command_reply: tid=0 2024-03-08T21:02:49.892 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.911+0000 7f88b776a740 1 client.15657 shutdown 2024-03-08T21:02:49.893 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.911+0000 7f88b776a740 1 -- 172.21.15.107:0/2159407695 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55decb975570 msgr2=0x55decb9d4770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:02:49.893 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.911+0000 7f88b776a740 1 --2- 172.21.15.107:0/2159407695 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55decb975570 0x55decb9d4770 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f889c00b920 tx=0x7f889c00b960 comp rx=0 tx=0).stop 2024-03-08T21:02:49.893 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.911+0000 7f88b776a740 1 -- 172.21.15.107:0/2159407695 shutdown_connections 2024-03-08T21:02:49.893 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.911+0000 7f88b776a740 1 --2- 172.21.15.107:0/2159407695 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55decb975570 0x55decb9d4770 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:02:49.893 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.911+0000 7f88b776a740 1 --2- 172.21.15.107:0/2159407695 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55decb9dc800 0x55decb9debf0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:02:49.893 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.911+0000 7f88b776a740 1 --2- 172.21.15.107:0/2159407695 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55decb96c7a0 0x55decb9cfc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:02:49.893 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.911+0000 7f88b776a740 1 --2- 172.21.15.107:0/2159407695 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55decb96be80 0x55decb9cf750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:02:49.893 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.911+0000 7f88b776a740 1 -- 172.21.15.107:0/2159407695 >> 172.21.15.107:0/2159407695 conn(0x55decb9667a0 msgr2=0x55decb96d590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:02:49.893 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.911+0000 7f888f400640 0 client.15657 ms_handle_reset on v2:172.21.15.149:6838/55290596 2024-03-08T21:02:49.893 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.911+0000 7f88b776a740 1 -- 172.21.15.107:0/2159407695 shutdown_connections 2024-03-08T21:02:49.893 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.912+0000 7f88b776a740 1 -- 172.21.15.107:0/2159407695 wait complete. 2024-03-08T21:02:49.893 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.912+0000 7f88b776a740 20 client.15657 trim_cache size 0 max 16384 2024-03-08T21:02:49.894 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:{ 2024-03-08T21:02:49.894 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "message": "", 2024-03-08T21:02:49.894 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "return_code": 0 2024-03-08T21:02:49.894 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:} 2024-03-08T21:02:49.895 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.912+0000 7f88af200640 1 -- 172.21.15.107:0/3866223091 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f88940a2960 msgr2=0x7f88940a4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:02:49.895 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.912+0000 7f88af200640 1 --2- 172.21.15.107:0/3866223091 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f88940a2960 0x7f88940a4e20 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f88a800efd0 tx=0x7f88a8005830 comp rx=0 tx=0).stop 2024-03-08T21:02:49.895 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.912+0000 7f88af200640 1 -- 172.21.15.107:0/3866223091 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f88b010d9a0 msgr2=0x7f88b0199930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:02:49.895 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.912+0000 7f88af200640 1 --2- 172.21.15.107:0/3866223091 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f88b010d9a0 0x7f88b0199930 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f88a800eaa0 tx=0x7f88a800ef60 comp rx=0 tx=0).stop 2024-03-08T21:02:49.895 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.913+0000 7f88af200640 1 -- 172.21.15.107:0/3866223091 shutdown_connections 2024-03-08T21:02:49.895 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.913+0000 7f88af200640 1 --2- 172.21.15.107:0/3866223091 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f88940a2960 0x7f88940a4e20 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:02:49.895 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.913+0000 7f88af200640 1 --2- 172.21.15.107:0/3866223091 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f88b010d9a0 0x7f88b0199930 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:02:49.895 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.913+0000 7f88af200640 1 --2- 172.21.15.107:0/3866223091 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f88b0073fc0 0x7f88b019fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:02:49.895 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.913+0000 7f88af200640 1 --2- 172.21.15.107:0/3866223091 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f88b00736a0 0x7f88b019f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:02:49.896 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.913+0000 7f88af200640 1 -- 172.21.15.107:0/3866223091 >> 172.21.15.107:0/3866223091 conn(0x7f88b00fcc30 msgr2=0x7f88b00fd940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:02:49.896 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.913+0000 7f88af200640 1 -- 172.21.15.107:0/3866223091 shutdown_connections 2024-03-08T21:02:49.896 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:02:49.913+0000 7f88af200640 1 -- 172.21.15.107:0/3866223091 wait complete. 2024-03-08T21:02:49.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:02:49 smithi107 ceph-mon[31381]: pgmap v72: 97 pgs: 97 active+clean; 1.5 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.3 KiB/s wr, 10 op/s 2024-03-08T21:02:50.013 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:02:49 smithi149 ceph-mon[35042]: pgmap v72: 97 pgs: 97 active+clean; 1.5 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.3 KiB/s wr, 10 op/s 2024-03-08T21:02:51.876 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:02:51 smithi204 ceph-mon[34935]: pgmap v73: 97 pgs: 97 active+clean; 1.5 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.3 KiB/s wr, 14 op/s 2024-03-08T21:02:51.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:02:51 smithi107 ceph-mon[31381]: pgmap v73: 97 pgs: 97 active+clean; 1.5 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.3 KiB/s wr, 14 op/s 2024-03-08T21:02:52.013 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:02:51 smithi149 ceph-mon[35042]: pgmap v73: 97 pgs: 97 active+clean; 1.5 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.3 KiB/s wr, 14 op/s 2024-03-08T21:02:53.878 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:02:53 smithi204 ceph-mon[34935]: pgmap v74: 97 pgs: 97 active+clean; 1.5 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-08T21:02:53.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:02:53 smithi107 ceph-mon[31381]: pgmap v74: 97 pgs: 97 active+clean; 1.5 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-08T21:02:54.013 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:02:53 smithi149 ceph-mon[35042]: pgmap v74: 97 pgs: 97 active+clean; 1.5 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-08T21:02:55.877 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:02:55 smithi204 ceph-mon[34935]: pgmap v75: 97 pgs: 97 active+clean; 1.5 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-08T21:02:55.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:02:55 smithi107 ceph-mon[31381]: pgmap v75: 97 pgs: 97 active+clean; 1.5 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-08T21:02:56.013 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:02:55 smithi149 ceph-mon[35042]: pgmap v75: 97 pgs: 97 active+clean; 1.5 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-08T21:02:57.876 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:02:57 smithi204 ceph-mon[34935]: pgmap v76: 97 pgs: 97 active+clean; 1.5 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:02:57.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:02:57 smithi107 ceph-mon[31381]: pgmap v76: 97 pgs: 97 active+clean; 1.5 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:02:58.013 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:02:57 smithi149 ceph-mon[35042]: pgmap v76: 97 pgs: 97 active+clean; 1.5 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:02:59.876 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:02:59 smithi204 ceph-mon[34935]: pgmap v77: 97 pgs: 97 active+clean; 1.5 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:02:59.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:02:59 smithi107 ceph-mon[31381]: pgmap v77: 97 pgs: 97 active+clean; 1.5 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:03:00.013 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:02:59 smithi149 ceph-mon[35042]: pgmap v77: 97 pgs: 97 active+clean; 1.5 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:03:01.876 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:03:01 smithi204 ceph-mon[34935]: pgmap v78: 97 pgs: 97 active+clean; 1.5 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-08T21:03:01.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:03:01 smithi107 ceph-mon[31381]: pgmap v78: 97 pgs: 97 active+clean; 1.5 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-08T21:03:02.013 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:03:01 smithi149 ceph-mon[35042]: pgmap v78: 97 pgs: 97 active+clean; 1.5 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-08T21:03:03.045 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:03.064+0000 7f60e9a00640 1 -- 172.21.15.107:0/4192635340 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f60e41009b0 msgr2=0x7f60e4104e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:03:03.045 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:03.064+0000 7f60e9a00640 1 --2- 172.21.15.107:0/4192635340 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f60e41009b0 0x7f60e4104e70 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f60cc009900 tx=0x7f60cc02bd20 comp rx=0 tx=0).stop 2024-03-08T21:03:03.045 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:03.064+0000 7f60e9a00640 1 -- 172.21.15.107:0/4192635340 shutdown_connections 2024-03-08T21:03:03.045 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:03.064+0000 7f60e9a00640 1 --2- 172.21.15.107:0/4192635340 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f60e4105490 0x7f60e4105920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:03:03.045 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:03.064+0000 7f60e9a00640 1 --2- 172.21.15.107:0/4192635340 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f60e41009b0 0x7f60e4104e70 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:03:03.045 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:03.064+0000 7f60e9a00640 1 --2- 172.21.15.107:0/4192635340 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f60e4100090 0x7f60e4100470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:03:03.045 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:03.064+0000 7f60e9a00640 1 -- 172.21.15.107:0/4192635340 >> 172.21.15.107:0/4192635340 conn(0x7f60e40fbb00 msgr2=0x7f60e40fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:03:03.046 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:03.064+0000 7f60e9a00640 1 -- 172.21.15.107:0/4192635340 shutdown_connections 2024-03-08T21:03:03.046 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:03.064+0000 7f60e9a00640 1 -- 172.21.15.107:0/4192635340 wait complete. 2024-03-08T21:03:03.048 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:03.065+0000 7f60e9a00640 1 Processor -- start 2024-03-08T21:03:03.048 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:03.065+0000 7f60e9a00640 1 -- start start 2024-03-08T21:03:03.048 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:03.066+0000 7f60e9a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f60e4100090 0x7f60e419df40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:03:03.048 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:03.066+0000 7f60e9a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f60e41009b0 0x7f60e419e480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:03:03.048 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:03.066+0000 7f60e9a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f60e4105490 0x7f60e41a2800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:03:03.048 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:03.066+0000 7f60e9a00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60e4117980 con 0x7f60e41009b0 2024-03-08T21:03:03.049 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:03.066+0000 7f60e9a00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60e4117800 con 0x7f60e4105490 2024-03-08T21:03:03.049 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:03.066+0000 7f60e9a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60e4117b00 con 0x7f60e4100090 2024-03-08T21:03:03.049 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:03.066+0000 7f60dbe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f60e41009b0 0x7f60e419e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:03:03.049 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:03.066+0000 7f60dbe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f60e41009b0 0x7f60e419e480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:50458/0 (socket says 172.21.15.107:50458) 2024-03-08T21:03:03.049 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:03.066+0000 7f60dbe00640 1 -- 172.21.15.107:0/1798082758 learned_addr learned my addr 172.21.15.107:0/1798082758 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:03:03.049 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:03.066+0000 7f60e3e00640 1 --2- 172.21.15.107:0/1798082758 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f60e4100090 0x7f60e419df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:03:03.049 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:03.066+0000 7f60e9000640 1 --2- 172.21.15.107:0/1798082758 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f60e4105490 0x7f60e41a2800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:03:03.049 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:03.066+0000 7f60dbe00640 1 -- 172.21.15.107:0/1798082758 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f60e4100090 msgr2=0x7f60e419df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:03:03.049 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:03.066+0000 7f60dbe00640 1 --2- 172.21.15.107:0/1798082758 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f60e4100090 0x7f60e419df40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:03:03.049 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:03.066+0000 7f60dbe00640 1 -- 172.21.15.107:0/1798082758 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f60e4105490 msgr2=0x7f60e41a2800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:03:03.049 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:03.066+0000 7f60dbe00640 1 --2- 172.21.15.107:0/1798082758 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f60e4105490 0x7f60e41a2800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:03:03.049 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:03.066+0000 7f60dbe00640 1 -- 172.21.15.107:0/1798082758 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60e41a2f30 con 0x7f60e41009b0 2024-03-08T21:03:03.049 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:03.067+0000 7f60e3e00640 1 --2- 172.21.15.107:0/1798082758 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f60e4100090 0x7f60e419df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T21:03:03.050 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:03.067+0000 7f60dbe00640 1 --2- 172.21.15.107:0/1798082758 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f60e41009b0 0x7f60e419e480 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f60cc005940 tx=0x7f60cc0045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:03:03.050 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:03.067+0000 7f60e1600640 1 -- 172.21.15.107:0/1798082758 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60cc043020 con 0x7f60e41009b0 2024-03-08T21:03:03.050 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:03.067+0000 7f60e1600640 1 -- 172.21.15.107:0/1798082758 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f60cc02db90 con 0x7f60e41009b0 2024-03-08T21:03:03.050 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:03.067+0000 7f60e9a00640 1 -- 172.21.15.107:0/1798082758 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60e41a31c0 con 0x7f60e41009b0 2024-03-08T21:03:03.050 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:03.067+0000 7f60e1600640 1 -- 172.21.15.107:0/1798082758 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60cc003a10 con 0x7f60e41009b0 2024-03-08T21:03:03.050 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:03.067+0000 7f60e9a00640 1 -- 172.21.15.107:0/1798082758 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60e41aaab0 con 0x7f60e41009b0 2024-03-08T21:03:03.051 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:03.071+0000 7f60e1600640 1 -- 172.21.15.107:0/1798082758 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) v1 ==== 137422+0+0 (secure 0 0 0) 0x7f60cc046020 con 0x7f60e41009b0 2024-03-08T21:03:03.052 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:03.071+0000 7f60eddca740 1 Processor -- start 2024-03-08T21:03:03.052 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:03.072+0000 7f60eddca740 1 -- start start 2024-03-08T21:03:03.053 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:03.072+0000 7f60eddca740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56282e4cde80 0x56282e4ce260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:03:03.054 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:03.072+0000 7f60eddca740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56282e4ce7a0 0x56282e4d7030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:03:03.054 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:03.072+0000 7f60eddca740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56282e4d7570 0x56282e4d9960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:03:03.054 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:03.072+0000 7f60eddca740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x56282e4313d0 con 0x56282e4d7570 2024-03-08T21:03:03.054 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:03.072+0000 7f60eddca740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x56282e3da6b0 con 0x56282e4ce7a0 2024-03-08T21:03:03.054 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:03.072+0000 7f60eddca740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56282e42d470 con 0x56282e4cde80 2024-03-08T21:03:03.054 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:03.072+0000 7f60e1600640 1 --2- 172.21.15.107:0/1798082758 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f60c80a2960 0x7f60c80a4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:03:03.054 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:03.072+0000 7f60dbe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56282e4d7570 0x56282e4d9960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:03:03.054 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:03.072+0000 7f60dbe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56282e4d7570 0x56282e4d9960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:50464/0 (socket says 172.21.15.107:50464) 2024-03-08T21:03:03.054 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:03.072+0000 7f60dbe00640 1 -- 172.21.15.107:0/1193511657 learned_addr learned my addr 172.21.15.107:0/1193511657 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:03:03.054 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:03.073+0000 7f60e1600640 1 -- 172.21.15.107:0/1798082758 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f60cc0f0f30 con 0x7f60e41009b0 2024-03-08T21:03:03.054 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:03.073+0000 7f60e3e00640 1 --2- 172.21.15.107:0/1193511657 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56282e4ce7a0 0x56282e4d7030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:03:03.055 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:03.073+0000 7f60e9000640 1 --2- 172.21.15.107:0/1193511657 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56282e4cde80 0x56282e4ce260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:03:03.055 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:03.073+0000 7f60dbe00640 1 -- 172.21.15.107:0/1193511657 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56282e4cde80 msgr2=0x56282e4ce260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:03:03.055 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:03.073+0000 7f60e9000640 1 --2- 172.21.15.107:0/1798082758 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f60c80a2960 0x7f60c80a4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:03:03.055 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:03.073+0000 7f60dbe00640 1 --2- 172.21.15.107:0/1193511657 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56282e4cde80 0x56282e4ce260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:03:03.055 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:03.073+0000 7f60dbe00640 1 -- 172.21.15.107:0/1193511657 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56282e4ce7a0 msgr2=0x56282e4d7030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:03:03.055 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:03.073+0000 7f60dbe00640 1 --2- 172.21.15.107:0/1193511657 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56282e4ce7a0 0x56282e4d7030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:03:03.055 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:03.073+0000 7f60dbe00640 1 -- 172.21.15.107:0/1193511657 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56282e400b50 con 0x56282e4d7570 2024-03-08T21:03:03.055 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:03.073+0000 7f60e3e00640 1 --2- 172.21.15.107:0/1193511657 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56282e4ce7a0 0x56282e4d7030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T21:03:03.055 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:03.073+0000 7f60dbe00640 1 --2- 172.21.15.107:0/1193511657 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56282e4d7570 0x56282e4d9960 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f60cc0bd330 tx=0x7f60cc090430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91b5f50e3b01a2b3 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:03:03.059 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:03.074+0000 7f60d9600640 1 -- 172.21.15.107:0/1193511657 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60cc02dd40 con 0x56282e4d7570 2024-03-08T21:03:03.059 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:03.074+0000 7f60d9600640 1 -- 172.21.15.107:0/1193511657 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f60cc088920 con 0x56282e4d7570 2024-03-08T21:03:03.059 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:03.074+0000 7f60d9600640 1 -- 172.21.15.107:0/1193511657 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60cc0b9ca0 con 0x56282e4d7570 2024-03-08T21:03:03.059 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:03.074+0000 7f60e9000640 1 --2- 172.21.15.107:0/1798082758 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f60c80a2960 0x7f60c80a4e20 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f60dc005cc0 tx=0x7f60dc004a40 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:03:03.059 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:03.075+0000 7f60eddca740 1 -- 172.21.15.107:0/1193511657 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56282e4d7570 msgr2=0x56282e4d9960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:03:03.060 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:03.075+0000 7f60eddca740 1 --2- 172.21.15.107:0/1193511657 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56282e4d7570 0x56282e4d9960 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f60cc0bd330 tx=0x7f60cc090430 comp rx=0 tx=0).stop 2024-03-08T21:03:03.060 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:03.075+0000 7f60eddca740 1 -- 172.21.15.107:0/1193511657 shutdown_connections 2024-03-08T21:03:03.060 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:03.075+0000 7f60eddca740 1 --2- 172.21.15.107:0/1193511657 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56282e4d7570 0x56282e4d9960 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:03:03.060 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:03.075+0000 7f60eddca740 1 --2- 172.21.15.107:0/1193511657 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56282e4ce7a0 0x56282e4d7030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:03:03.060 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:03.075+0000 7f60eddca740 1 --2- 172.21.15.107:0/1193511657 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56282e4cde80 0x56282e4ce260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:03:03.060 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:03.075+0000 7f60eddca740 1 -- 172.21.15.107:0/1193511657 >> 172.21.15.107:0/1193511657 conn(0x56282e4c87a0 msgr2=0x56282e4c8b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:03:03.060 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:03.075+0000 7f60eddca740 1 -- 172.21.15.107:0/1193511657 shutdown_connections 2024-03-08T21:03:03.060 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:03.075+0000 7f60eddca740 1 -- 172.21.15.107:0/1193511657 wait complete. 2024-03-08T21:03:03.060 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:03.075+0000 7f60eddca740 1 -- start start 2024-03-08T21:03:03.060 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:03.076+0000 7f60eddca740 1 Processor -- start 2024-03-08T21:03:03.060 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:03.076+0000 7f60eddca740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56282e4cde80 0x56282e4d6f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:03:03.060 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:03.076+0000 7f60eddca740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56282e4ce7a0 0x56282e4d42d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:03:03.060 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:03.076+0000 7f60eddca740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56282e4d7570 0x56282e4d48e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:03:03.061 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:03.076+0000 7f60eddca740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x56282e4d9f60 con 0x56282e4ce7a0 2024-03-08T21:03:03.061 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:03.076+0000 7f60eddca740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x56282e3ed9f0 con 0x56282e4cde80 2024-03-08T21:03:03.061 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:03.076+0000 7f60eddca740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56282e4da0e0 con 0x56282e4d7570 2024-03-08T21:03:03.061 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:03.076+0000 7f60eddca740 10 client.0 _ll_register_callbacks cb 0x56282e145a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T21:03:03.061 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:03.076+0000 7f60dbe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56282e4ce7a0 0x56282e4d42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:03:03.061 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:03.077+0000 7f60dbe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56282e4ce7a0 0x56282e4d42d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:50476/0 (socket says 172.21.15.107:50476) 2024-03-08T21:03:03.061 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:03.077+0000 7f60dbe00640 1 -- 172.21.15.107:0/1282866048 learned_addr learned my addr 172.21.15.107:0/1282866048 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:03:03.061 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:03.077+0000 7f60e3e00640 1 --2- 172.21.15.107:0/1282866048 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56282e4cde80 0x56282e4d6f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:03:03.061 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:03.077+0000 7f60e9000640 1 --2- 172.21.15.107:0/1282866048 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56282e4d7570 0x56282e4d48e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:03:03.061 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:03.077+0000 7f60dbe00640 1 -- 172.21.15.107:0/1282866048 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56282e4d7570 msgr2=0x56282e4d48e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:03:03.061 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:03.077+0000 7f60dbe00640 1 --2- 172.21.15.107:0/1282866048 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56282e4d7570 0x56282e4d48e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:03:03.062 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:03.077+0000 7f60dbe00640 1 -- 172.21.15.107:0/1282866048 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56282e4cde80 msgr2=0x56282e4d6f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:03:03.062 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:03.077+0000 7f60dbe00640 1 --2- 172.21.15.107:0/1282866048 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56282e4cde80 0x56282e4d6f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:03:03.062 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:03.077+0000 7f60dbe00640 1 -- 172.21.15.107:0/1282866048 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56282e146210 con 0x56282e4ce7a0 2024-03-08T21:03:03.062 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:03.077+0000 7f60dbe00640 1 --2- 172.21.15.107:0/1282866048 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56282e4ce7a0 0x56282e4d42d0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f60cc0b9a90 tx=0x7f60cc02ddb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a02bc56832adf98c server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:03:03.062 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:03.077+0000 7f60d2000640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-03-08T21:03:03.062 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:03.077+0000 7f60d2000640 1 -- 172.21.15.107:0/1282866048 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60cc0d5020 con 0x56282e4ce7a0 2024-03-08T21:03:03.062 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:03.078+0000 7f60d2000640 1 -- 172.21.15.107:0/1282866048 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f60cc0bd890 con 0x56282e4ce7a0 2024-03-08T21:03:03.062 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:03.078+0000 7f60eddca740 1 -- 172.21.15.107:0/1282866048 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f60cc088920 con 0x56282e4ce7a0 2024-03-08T21:03:03.062 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:03.078+0000 7f60d2000640 1 -- 172.21.15.107:0/1282866048 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f60cc0eea90 con 0x56282e4ce7a0 2024-03-08T21:03:03.062 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:03.078+0000 7f60eddca740 10 client.15669 fetch_fsmap learned FSMap version 30 2024-03-08T21:03:03.062 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:03.078+0000 7f60eddca740 1 -- 172.21.15.107:0/1282866048 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x56282e212620 con 0x56282e4ce7a0 2024-03-08T21:03:03.062 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:03.078+0000 7f60d2000640 1 -- 172.21.15.107:0/1282866048 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f60cc088920 con 0x56282e4ce7a0 2024-03-08T21:03:03.062 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:03.078+0000 7f60d2000640 1 -- 172.21.15.107:0/1282866048 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 30) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f60cc08f070 con 0x56282e4ce7a0 2024-03-08T21:03:03.062 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:03.078+0000 7f60eddca740 10 client.15669 fetch_fsmap finished waiting for FSMap version 30 2024-03-08T21:03:03.063 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:03.078+0000 7f60eddca740 10 client.15669 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-08T21:03:03.063 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:03.078+0000 7f60eddca740 20 client.15669 populate_metadata read hostname 'smithi107' 2024-03-08T21:03:03.063 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:03.078+0000 7f60eddca740 1 --2- 172.21.15.107:0/1282866048 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x56282e4e6fa0 0x56282e4e9390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:03:03.063 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:03.079+0000 7f60eddca740 4 client.15669 mds_command: new command op to 34475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-08T21:03:03.063 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:03.079+0000 7f60eddca740 1 -- 172.21.15.107:0/1282866048 --> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56282e43e910 con 0x56282e4e6fa0 2024-03-08T21:03:03.064 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:03.079+0000 7f60e3e00640 1 --2- 172.21.15.107:0/1282866048 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x56282e4e6fa0 0x56282e4e9390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:03:03.064 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:03.080+0000 7f60e3e00640 1 --2- 172.21.15.107:0/1282866048 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x56282e4e6fa0 0x56282e4e9390 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=99a6d041c6cc623b server_cookie=c67a40c1ee7ddd8 in_seq=0 out_seq=0 2024-03-08T21:03:03.064 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:03.080+0000 7f60d2000640 10 client.15669 ms_handle_connect on v2:172.21.15.149:6838/55290596 2024-03-08T21:03:03.064 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:03.082+0000 7f60d2000640 1 -- 172.21.15.107:0/1282866048 <== mds.0 v2:172.21.15.149:6838/55290596 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56282e43e910 con 0x56282e4e6fa0 2024-03-08T21:03:03.064 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:03.082+0000 7f60d2000640 10 client.15669 handle_command_reply: tid=0 2024-03-08T21:03:03.064 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:03.082+0000 7f60eddca740 1 client.15669 shutdown 2024-03-08T21:03:03.064 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:03.082+0000 7f60eddca740 1 -- 172.21.15.107:0/1282866048 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56282e4ce7a0 msgr2=0x56282e4d42d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:03:03.065 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:03.082+0000 7f60eddca740 1 --2- 172.21.15.107:0/1282866048 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56282e4ce7a0 0x56282e4d42d0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f60cc0b9a90 tx=0x7f60cc02ddb0 comp rx=0 tx=0).stop 2024-03-08T21:03:03.065 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:03.082+0000 7f60eddca740 1 -- 172.21.15.107:0/1282866048 shutdown_connections 2024-03-08T21:03:03.065 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:03.082+0000 7f60eddca740 1 --2- 172.21.15.107:0/1282866048 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56282e4d7570 0x56282e4d48e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:03:03.065 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:03.082+0000 7f60eddca740 1 --2- 172.21.15.107:0/1282866048 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56282e4ce7a0 0x56282e4d42d0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:03:03.065 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:03.082+0000 7f60eddca740 1 --2- 172.21.15.107:0/1282866048 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x56282e4e6fa0 0x56282e4e9390 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:03:03.065 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:03.082+0000 7f60eddca740 1 --2- 172.21.15.107:0/1282866048 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56282e4cde80 0x56282e4d6f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:03:03.065 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:03.082+0000 7f60d2000640 0 client.15669 ms_handle_reset on v2:172.21.15.149:6838/55290596 2024-03-08T21:03:03.065 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:03.082+0000 7f60eddca740 1 -- 172.21.15.107:0/1282866048 >> 172.21.15.107:0/1282866048 conn(0x56282e4c87a0 msgr2=0x56282e4ca4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:03:03.065 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:03.082+0000 7f60eddca740 1 -- 172.21.15.107:0/1282866048 shutdown_connections 2024-03-08T21:03:03.065 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:03.082+0000 7f60eddca740 1 -- 172.21.15.107:0/1282866048 wait complete. 2024-03-08T21:03:03.065 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:03.083+0000 7f60eddca740 20 client.15669 trim_cache size 0 max 16384 2024-03-08T21:03:03.083 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:03.101+0000 7f60eddca740 1 Processor -- start 2024-03-08T21:03:03.083 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:03.101+0000 7f60eddca740 1 -- start start 2024-03-08T21:03:03.083 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:03.102+0000 7f60eddca740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56282e4cde80 0x56282e4ed9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:03:03.083 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:03.102+0000 7f60eddca740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56282e4ce7a0 0x56282e4d3430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:03:03.083 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:03.102+0000 7f60eddca740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56282e4d7570 0x56282e4ca2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:03:03.084 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:03.102+0000 7f60eddca740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60e41b2f40 con 0x56282e4d7570 2024-03-08T21:03:03.084 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:03.102+0000 7f60eddca740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60e41b48c0 con 0x56282e4cde80 2024-03-08T21:03:03.084 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:03.102+0000 7f60eddca740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60e41b35a0 con 0x56282e4ce7a0 2024-03-08T21:03:03.084 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:03.102+0000 7f60e9000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56282e4d7570 0x56282e4ca2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:03:03.084 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:03.102+0000 7f60e9000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56282e4d7570 0x56282e4ca2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:50486/0 (socket says 172.21.15.107:50486) 2024-03-08T21:03:03.084 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:03.102+0000 7f60e9000640 1 -- 172.21.15.107:0/75494585 learned_addr learned my addr 172.21.15.107:0/75494585 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:03:03.084 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:03.102+0000 7f60dbe00640 1 --2- 172.21.15.107:0/75494585 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56282e4ce7a0 0x56282e4d3430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:03:03.084 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:03.102+0000 7f60e3e00640 1 --2- 172.21.15.107:0/75494585 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56282e4cde80 0x56282e4ed9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:03:03.084 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:03.103+0000 7f60e9000640 1 -- 172.21.15.107:0/75494585 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56282e4ce7a0 msgr2=0x56282e4d3430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:03:03.084 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:03.103+0000 7f60e9000640 1 --2- 172.21.15.107:0/75494585 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56282e4ce7a0 0x56282e4d3430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:03:03.084 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:03.103+0000 7f60e9000640 1 -- 172.21.15.107:0/75494585 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56282e4cde80 msgr2=0x56282e4ed9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:03:03.085 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:03.103+0000 7f60e9000640 1 --2- 172.21.15.107:0/75494585 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56282e4cde80 0x56282e4ed9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:03:03.085 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:03.103+0000 7f60e9000640 1 -- 172.21.15.107:0/75494585 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56282e4c4070 con 0x56282e4d7570 2024-03-08T21:03:03.085 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:03.103+0000 7f60dbe00640 1 --2- 172.21.15.107:0/75494585 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56282e4ce7a0 0x56282e4d3430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T21:03:03.087 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:03.103+0000 7f60e9000640 1 --2- 172.21.15.107:0/75494585 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56282e4d7570 0x56282e4ca2b0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f60dc00bee0 tx=0x7f60dc00e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47e60c2d00bbdc5b server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:03:03.087 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:03.103+0000 7f60d8c00640 1 -- 172.21.15.107:0/75494585 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60dc023070 con 0x56282e4d7570 2024-03-08T21:03:03.087 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:03.103+0000 7f60d8c00640 1 -- 172.21.15.107:0/75494585 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f60dc01fb20 con 0x56282e4d7570 2024-03-08T21:03:03.087 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:03.103+0000 7f60d8c00640 1 -- 172.21.15.107:0/75494585 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60dc017040 con 0x56282e4d7570 2024-03-08T21:03:03.087 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:03.104+0000 7f60eddca740 1 -- 172.21.15.107:0/75494585 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56282e4d7570 msgr2=0x56282e4ca2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:03:03.088 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:03.104+0000 7f60eddca740 1 --2- 172.21.15.107:0/75494585 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56282e4d7570 0x56282e4ca2b0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f60dc00bee0 tx=0x7f60dc00e000 comp rx=0 tx=0).stop 2024-03-08T21:03:03.088 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:03.104+0000 7f60eddca740 1 -- 172.21.15.107:0/75494585 shutdown_connections 2024-03-08T21:03:03.088 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:03.104+0000 7f60eddca740 1 --2- 172.21.15.107:0/75494585 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56282e4d7570 0x56282e4ca2b0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:03:03.088 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:03.104+0000 7f60eddca740 1 --2- 172.21.15.107:0/75494585 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56282e4ce7a0 0x56282e4d3430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:03:03.088 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:03.104+0000 7f60eddca740 1 --2- 172.21.15.107:0/75494585 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56282e4cde80 0x56282e4ed9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:03:03.088 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:03.104+0000 7f60eddca740 1 -- 172.21.15.107:0/75494585 >> 172.21.15.107:0/75494585 conn(0x56282e4c87a0 msgr2=0x56282e4c9ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:03:03.088 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:03.104+0000 7f60eddca740 1 -- 172.21.15.107:0/75494585 shutdown_connections 2024-03-08T21:03:03.088 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:03.104+0000 7f60eddca740 1 -- 172.21.15.107:0/75494585 wait complete. 2024-03-08T21:03:03.088 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:03.105+0000 7f60eddca740 1 -- start start 2024-03-08T21:03:03.088 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:03.105+0000 7f60eddca740 1 Processor -- start 2024-03-08T21:03:03.088 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:03.106+0000 7f60eddca740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56282e4cde80 0x56282e4dcbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:03:03.088 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:03.106+0000 7f60eddca740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56282e4ce7a0 0x56282e4dd0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:03:03.088 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:03.106+0000 7f60eddca740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56282e4d7570 0x56282e4dd630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:03:03.089 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:03.106+0000 7f60eddca740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60e41b4260 con 0x56282e4d7570 2024-03-08T21:03:03.089 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:03.106+0000 7f60eddca740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60e41b3c00 con 0x56282e4ce7a0 2024-03-08T21:03:03.089 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:03.106+0000 7f60eddca740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60e41b11e0 con 0x56282e4cde80 2024-03-08T21:03:03.089 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:03.106+0000 7f60eddca740 10 client.0 _ll_register_callbacks cb 0x7f60cc040f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T21:03:03.089 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:03.106+0000 7f60e9000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56282e4d7570 0x56282e4dd630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:03:03.089 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:03.106+0000 7f60e9000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56282e4d7570 0x56282e4dd630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:50502/0 (socket says 172.21.15.107:50502) 2024-03-08T21:03:03.089 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:03.106+0000 7f60e9000640 1 -- 172.21.15.107:0/1432053996 learned_addr learned my addr 172.21.15.107:0/1432053996 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:03:03.089 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:03.106+0000 7f60dbe00640 1 --2- 172.21.15.107:0/1432053996 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56282e4ce7a0 0x56282e4dd0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:03:03.089 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:03.106+0000 7f60e3e00640 1 --2- 172.21.15.107:0/1432053996 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56282e4cde80 0x56282e4dcbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:03:03.089 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:03.106+0000 7f60e9000640 1 -- 172.21.15.107:0/1432053996 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56282e4cde80 msgr2=0x56282e4dcbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:03:03.089 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:03.106+0000 7f60e9000640 1 --2- 172.21.15.107:0/1432053996 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56282e4cde80 0x56282e4dcbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:03:03.089 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:03.106+0000 7f60e9000640 1 -- 172.21.15.107:0/1432053996 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56282e4ce7a0 msgr2=0x56282e4dd0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:03:03.089 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:03.106+0000 7f60e9000640 1 --2- 172.21.15.107:0/1432053996 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56282e4ce7a0 0x56282e4dd0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:03:03.090 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:03.106+0000 7f60e9000640 1 -- 172.21.15.107:0/1432053996 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56282e53d170 con 0x56282e4d7570 2024-03-08T21:03:03.090 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:03.106+0000 7f60dbe00640 1 --2- 172.21.15.107:0/1432053996 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56282e4ce7a0 0x56282e4dd0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T21:03:03.090 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:03.107+0000 7f60e9000640 1 --2- 172.21.15.107:0/1432053996 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56282e4d7570 0x56282e4dd630 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f60dc00b900 tx=0x7f60dc010440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62b2669e52163127 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:03:03.090 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:03.107+0000 7f60d2000640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-03-08T21:03:03.091 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:03.107+0000 7f60d2000640 1 -- 172.21.15.107:0/1432053996 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60dc017030 con 0x56282e4d7570 2024-03-08T21:03:03.091 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:03.107+0000 7f60d2000640 1 -- 172.21.15.107:0/1432053996 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f60dc004cb0 con 0x56282e4d7570 2024-03-08T21:03:03.091 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:03.107+0000 7f60eddca740 1 -- 172.21.15.107:0/1432053996 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f60dc01fb20 con 0x56282e4d7570 2024-03-08T21:03:03.091 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:03.107+0000 7f60d2000640 1 -- 172.21.15.107:0/1432053996 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f60dc023040 con 0x56282e4d7570 2024-03-08T21:03:03.092 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:03.107+0000 7f60eddca740 10 client.15675 fetch_fsmap learned FSMap version 30 2024-03-08T21:03:03.092 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:03.107+0000 7f60eddca740 1 -- 172.21.15.107:0/1432053996 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x56282e53e3e0 con 0x56282e4d7570 2024-03-08T21:03:03.092 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:03.108+0000 7f60d2000640 1 -- 172.21.15.107:0/1432053996 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f60dc01fb20 con 0x56282e4d7570 2024-03-08T21:03:03.092 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:03.108+0000 7f60d2000640 1 -- 172.21.15.107:0/1432053996 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 30) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f60dc030070 con 0x56282e4d7570 2024-03-08T21:03:03.092 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:03.108+0000 7f60eddca740 10 client.15675 fetch_fsmap finished waiting for FSMap version 30 2024-03-08T21:03:03.092 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:03.108+0000 7f60eddca740 10 client.15675 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-08T21:03:03.092 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:03.108+0000 7f60eddca740 20 client.15675 populate_metadata read hostname 'smithi107' 2024-03-08T21:03:03.092 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:03.108+0000 7f60eddca740 1 --2- 172.21.15.107:0/1432053996 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x56282e53ea00 0x56282e540e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:03:03.092 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:03.108+0000 7f60eddca740 4 client.15675 mds_command: new command op to 34475 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-08T21:03:03.092 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:03.108+0000 7f60eddca740 1 -- 172.21.15.107:0/1432053996 --> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x56282e438c60 con 0x56282e53ea00 2024-03-08T21:03:03.092 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:03.109+0000 7f60e3e00640 1 --2- 172.21.15.107:0/1432053996 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x56282e53ea00 0x56282e540e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:03:03.092 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:03.109+0000 7f60e3e00640 1 --2- 172.21.15.107:0/1432053996 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x56282e53ea00 0x56282e540e20 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6ac13e91bee2404e server_cookie=64617c8be6eb1208 in_seq=0 out_seq=0 2024-03-08T21:03:03.092 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:03.109+0000 7f60d2000640 10 client.15675 ms_handle_connect on v2:172.21.15.149:6838/55290596 2024-03-08T21:03:03.876 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:03:03 smithi204 ceph-mon[34935]: pgmap v79: 97 pgs: 97 active+clean; 1.5 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:03:03.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:03:03 smithi107 ceph-mon[31381]: pgmap v79: 97 pgs: 97 active+clean; 1.5 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:03:04.013 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:03:03 smithi149 ceph-mon[35042]: pgmap v79: 97 pgs: 97 active+clean; 1.5 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:03:04.186 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:04.205+0000 7f60d2000640 1 -- 172.21.15.107:0/1432053996 <== mds.0 v2:172.21.15.149:6838/55290596 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x56282e438c60 con 0x56282e53ea00 2024-03-08T21:03:04.186 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:04.205+0000 7f60d2000640 10 client.15675 handle_command_reply: tid=0 2024-03-08T21:03:04.186 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:04.205+0000 7f60eddca740 1 client.15675 shutdown 2024-03-08T21:03:04.186 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:04.206+0000 7f60eddca740 1 -- 172.21.15.107:0/1432053996 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56282e4d7570 msgr2=0x56282e4dd630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:03:04.186 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:04.206+0000 7f60eddca740 1 --2- 172.21.15.107:0/1432053996 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56282e4d7570 0x56282e4dd630 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f60dc00b900 tx=0x7f60dc010440 comp rx=0 tx=0).stop 2024-03-08T21:03:04.187 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:04.206+0000 7f60eddca740 1 -- 172.21.15.107:0/1432053996 shutdown_connections 2024-03-08T21:03:04.187 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:04.206+0000 7f60eddca740 1 --2- 172.21.15.107:0/1432053996 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56282e4d7570 0x56282e4dd630 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:03:04.187 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:04.206+0000 7f60eddca740 1 --2- 172.21.15.107:0/1432053996 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x56282e53ea00 0x56282e540e20 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:03:04.188 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:04.206+0000 7f60eddca740 1 --2- 172.21.15.107:0/1432053996 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56282e4ce7a0 0x56282e4dd0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:03:04.188 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:04.206+0000 7f60eddca740 1 --2- 172.21.15.107:0/1432053996 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56282e4cde80 0x56282e4dcbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:03:04.188 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:04.206+0000 7f60d2000640 0 client.15675 ms_handle_reset on v2:172.21.15.149:6838/55290596 2024-03-08T21:03:04.188 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:04.206+0000 7f60eddca740 1 -- 172.21.15.107:0/1432053996 >> 172.21.15.107:0/1432053996 conn(0x56282e4c87a0 msgr2=0x56282e4d32b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:03:04.188 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:04.206+0000 7f60eddca740 1 -- 172.21.15.107:0/1432053996 shutdown_connections 2024-03-08T21:03:04.188 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:04.206+0000 7f60eddca740 1 -- 172.21.15.107:0/1432053996 wait complete. 2024-03-08T21:03:04.188 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:04.207+0000 7f60eddca740 20 client.15675 trim_cache size 0 max 16384 2024-03-08T21:03:04.188 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:{ 2024-03-08T21:03:04.188 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "message": "", 2024-03-08T21:03:04.188 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "return_code": 0 2024-03-08T21:03:04.189 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:} 2024-03-08T21:03:04.189 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:04.208+0000 7f60e9a00640 1 -- 172.21.15.107:0/1798082758 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f60c80a2960 msgr2=0x7f60c80a4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:03:04.189 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:04.208+0000 7f60e9a00640 1 --2- 172.21.15.107:0/1798082758 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f60c80a2960 0x7f60c80a4e20 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f60dc005cc0 tx=0x7f60dc004a40 comp rx=0 tx=0).stop 2024-03-08T21:03:04.189 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:04.208+0000 7f60e9a00640 1 -- 172.21.15.107:0/1798082758 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f60e41009b0 msgr2=0x7f60e419e480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:03:04.189 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:04.208+0000 7f60e9a00640 1 --2- 172.21.15.107:0/1798082758 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f60e41009b0 0x7f60e419e480 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f60cc005940 tx=0x7f60cc0045e0 comp rx=0 tx=0).stop 2024-03-08T21:03:04.189 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:04.208+0000 7f60e9a00640 1 -- 172.21.15.107:0/1798082758 shutdown_connections 2024-03-08T21:03:04.189 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:04.208+0000 7f60e9a00640 1 --2- 172.21.15.107:0/1798082758 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f60c80a2960 0x7f60c80a4e20 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:03:04.189 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:04.208+0000 7f60e9a00640 1 --2- 172.21.15.107:0/1798082758 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f60e4105490 0x7f60e41a2800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:03:04.190 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:04.208+0000 7f60e9a00640 1 --2- 172.21.15.107:0/1798082758 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f60e41009b0 0x7f60e419e480 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:03:04.190 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:04.208+0000 7f60e9a00640 1 --2- 172.21.15.107:0/1798082758 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f60e4100090 0x7f60e419df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:03:04.190 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:04.208+0000 7f60e9a00640 1 -- 172.21.15.107:0/1798082758 >> 172.21.15.107:0/1798082758 conn(0x7f60e40fbb00 msgr2=0x7f60e40fcbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:03:04.190 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:04.208+0000 7f60e9a00640 1 -- 172.21.15.107:0/1798082758 shutdown_connections 2024-03-08T21:03:04.190 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:04.208+0000 7f60e9a00640 1 -- 172.21.15.107:0/1798082758 wait complete. 2024-03-08T21:03:05.378 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:03:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[69845]: *** Caught signal (Segmentation fault) ** 2024-03-08T21:03:05.378 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:03:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[69845]: in thread 7f0222400700 thread_name:md_log_replay 2024-03-08T21:03:05.378 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:03:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[69845]: ceph version 19.0.0-1965-g7d63482a (7d63482aa89628d733e298984a762389b625c761) squid (dev) 2024-03-08T21:03:05.378 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:03:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[69845]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f0236a20d20] 2024-03-08T21:03:05.378 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:03:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[69845]: 2: /usr/bin/ceph-mds(+0x503eff) [0x55fe94443eff] 2024-03-08T21:03:05.379 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:03:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[69845]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x55fe94447773] 2024-03-08T21:03:05.379 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:03:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[69845]: 4: (MDLog::_replay_thread()+0x617) [0x55fe944a3587] 2024-03-08T21:03:05.379 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:03:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[69845]: 5: (MDLog::ReplayThread::entry()+0x11) [0x55fe944ac091] 2024-03-08T21:03:05.379 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:03:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[69845]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7f0236a161ca] 2024-03-08T21:03:05.379 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:03:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[69845]: 7: clone() 2024-03-08T21:03:05.379 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:03:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[69845]: 2024-03-08T21:03:05.031+0000 7f0222400700 -1 *** Caught signal (Segmentation fault) ** 2024-03-08T21:03:05.379 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:03:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[69845]: in thread 7f0222400700 thread_name:md_log_replay 2024-03-08T21:03:05.379 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:03:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[69845]: 2024-03-08T21:03:05.379 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:03:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[69845]: ceph version 19.0.0-1965-g7d63482a (7d63482aa89628d733e298984a762389b625c761) squid (dev) 2024-03-08T21:03:05.379 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:03:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[69845]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f0236a20d20] 2024-03-08T21:03:05.379 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:03:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[69845]: 2: /usr/bin/ceph-mds(+0x503eff) [0x55fe94443eff] 2024-03-08T21:03:05.379 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:03:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[69845]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x55fe94447773] 2024-03-08T21:03:05.379 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:03:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[69845]: 4: (MDLog::_replay_thread()+0x617) [0x55fe944a3587] 2024-03-08T21:03:05.379 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:03:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[69845]: 5: (MDLog::ReplayThread::entry()+0x11) [0x55fe944ac091] 2024-03-08T21:03:05.379 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:03:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[69845]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7f0236a161ca] 2024-03-08T21:03:05.380 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:03:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[69845]: 7: clone() 2024-03-08T21:03:05.380 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:03:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[69845]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-08T21:03:05.380 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:03:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[69845]: 2024-03-08T21:03:05.380 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:03:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[69845]: 0> 2024-03-08T21:03:05.031+0000 7f0222400700 -1 *** Caught signal (Segmentation fault) ** 2024-03-08T21:03:05.380 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:03:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[69845]: in thread 7f0222400700 thread_name:md_log_replay 2024-03-08T21:03:05.380 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:03:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[69845]: 2024-03-08T21:03:05.380 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:03:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[69845]: ceph version 19.0.0-1965-g7d63482a (7d63482aa89628d733e298984a762389b625c761) squid (dev) 2024-03-08T21:03:05.380 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:03:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[69845]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f0236a20d20] 2024-03-08T21:03:05.380 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:03:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[69845]: 2: /usr/bin/ceph-mds(+0x503eff) [0x55fe94443eff] 2024-03-08T21:03:05.381 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:03:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[69845]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x55fe94447773] 2024-03-08T21:03:05.381 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:03:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[69845]: 4: (MDLog::_replay_thread()+0x617) [0x55fe944a3587] 2024-03-08T21:03:05.381 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:03:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[69845]: 5: (MDLog::ReplayThread::entry()+0x11) [0x55fe944ac091] 2024-03-08T21:03:05.381 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:03:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[69845]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7f0236a161ca] 2024-03-08T21:03:05.381 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:03:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[69845]: 7: clone() 2024-03-08T21:03:05.381 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:03:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[69845]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-08T21:03:05.381 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:03:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[69845]: 2024-03-08T21:03:05.381 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:03:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[69845]: 0> 2024-03-08T21:03:05.031+0000 7f0222400700 -1 *** Caught signal (Segmentation fault) ** 2024-03-08T21:03:05.381 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:03:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[69845]: in thread 7f0222400700 thread_name:md_log_replay 2024-03-08T21:03:05.381 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:03:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[69845]: 2024-03-08T21:03:05.381 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:03:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[69845]: ceph version 19.0.0-1965-g7d63482a (7d63482aa89628d733e298984a762389b625c761) squid (dev) 2024-03-08T21:03:05.381 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:03:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[69845]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f0236a20d20] 2024-03-08T21:03:05.381 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:03:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[69845]: 2: /usr/bin/ceph-mds(+0x503eff) [0x55fe94443eff] 2024-03-08T21:03:05.381 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:03:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[69845]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x55fe94447773] 2024-03-08T21:03:05.382 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:03:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[69845]: 4: (MDLog::_replay_thread()+0x617) [0x55fe944a3587] 2024-03-08T21:03:05.382 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:03:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[69845]: 5: (MDLog::ReplayThread::entry()+0x11) [0x55fe944ac091] 2024-03-08T21:03:05.382 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:03:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[69845]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7f0236a161ca] 2024-03-08T21:03:05.382 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:03:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[69845]: 7: clone() 2024-03-08T21:03:05.382 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:03:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[69845]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-08T21:03:05.382 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:03:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[69845]: 2024-03-08T21:03:05.382 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:03:05 smithi204 podman[70160]: 2024-03-08 21:03:05.146410262 +0000 UTC m=+0.038762906 container died 1633bce08122b382f994e613bbd2c64c80c2d7ffbef441b57ad2f7ac547c941c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=rishabh-2024mar8-7d63482, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-03-08T21:03:05.876 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:03:05 smithi204 ceph-mon[34935]: pgmap v80: 97 pgs: 97 active+clean; 2.0 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 64 KiB/s wr, 27 op/s 2024-03-08T21:03:05.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:03:05 smithi107 ceph-mon[31381]: pgmap v80: 97 pgs: 97 active+clean; 2.0 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 64 KiB/s wr, 27 op/s 2024-03-08T21:03:06.013 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:03:05 smithi149 ceph-mon[35042]: pgmap v80: 97 pgs: 97 active+clean; 2.0 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 64 KiB/s wr, 27 op/s 2024-03-08T21:03:06.376 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:03:06 smithi204 podman[70160]: 2024-03-08 21:03:06.008115282 +0000 UTC m=+0.900467936 container remove 1633bce08122b382f994e613bbd2c64c80c2d7ffbef441b57ad2f7ac547c941c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, RELEASE=rishabh-2024mar8-7d63482, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-03-08T21:03:06.376 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:03:06 smithi204 systemd[1]: ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715@mds.c.service: Main process exited, code=exited, status=139/n/a 2024-03-08T21:03:06.876 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:03:06 smithi204 systemd[1]: ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715@mds.c.service: Failed with result 'exit-code'. 2024-03-08T21:03:06.876 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:03:06 smithi204 systemd[1]: ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715@mds.c.service: Consumed 1.894s CPU time. 2024-03-08T21:03:07.852 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:03:07 smithi204 ceph-mon[34935]: pgmap v81: 97 pgs: 97 active+clean; 2.0 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 64 KiB/s wr, 23 op/s 2024-03-08T21:03:07.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:03:07 smithi107 ceph-mon[31381]: pgmap v81: 97 pgs: 97 active+clean; 2.0 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 64 KiB/s wr, 23 op/s 2024-03-08T21:03:08.013 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:03:07 smithi149 ceph-mon[35042]: pgmap v81: 97 pgs: 97 active+clean; 2.0 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 64 KiB/s wr, 23 op/s 2024-03-08T21:03:09.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:03:09 smithi107 ceph-mon[31381]: pgmap v82: 97 pgs: 97 active+clean; 2.0 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 64 KiB/s wr, 23 op/s 2024-03-08T21:03:10.013 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:03:09 smithi149 ceph-mon[35042]: pgmap v82: 97 pgs: 97 active+clean; 2.0 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 64 KiB/s wr, 23 op/s 2024-03-08T21:03:10.126 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:03:09 smithi204 ceph-mon[34935]: pgmap v82: 97 pgs: 97 active+clean; 2.0 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 64 KiB/s wr, 23 op/s 2024-03-08T21:03:11.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:03:11 smithi107 ceph-mon[31381]: pgmap v83: 97 pgs: 97 active+clean; 2.0 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 65 KiB/s wr, 29 op/s 2024-03-08T21:03:12.013 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:03:11 smithi149 ceph-mon[35042]: pgmap v83: 97 pgs: 97 active+clean; 2.0 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 65 KiB/s wr, 29 op/s 2024-03-08T21:03:12.126 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:03:11 smithi204 ceph-mon[34935]: pgmap v83: 97 pgs: 97 active+clean; 2.0 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 65 KiB/s wr, 29 op/s 2024-03-08T21:03:12.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:03:12 smithi107 ceph-mon[31381]: pgmap v84: 97 pgs: 97 active+clean; 2.0 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 65 KiB/s wr, 25 op/s 2024-03-08T21:03:13.013 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:03:12 smithi149 ceph-mon[35042]: pgmap v84: 97 pgs: 97 active+clean; 2.0 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 65 KiB/s wr, 25 op/s 2024-03-08T21:03:13.126 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:03:12 smithi204 ceph-mon[34935]: pgmap v84: 97 pgs: 97 active+clean; 2.0 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 65 KiB/s wr, 25 op/s 2024-03-08T21:03:15.876 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:03:15 smithi204 ceph-mon[34935]: pgmap v85: 97 pgs: 97 active+clean; 2.0 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 65 KiB/s wr, 28 op/s 2024-03-08T21:03:15.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:03:15 smithi107 ceph-mon[31381]: pgmap v85: 97 pgs: 97 active+clean; 2.0 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 65 KiB/s wr, 28 op/s 2024-03-08T21:03:16.013 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:03:15 smithi149 ceph-mon[35042]: pgmap v85: 97 pgs: 97 active+clean; 2.0 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 65 KiB/s wr, 28 op/s 2024-03-08T21:03:17.008 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:03:16 smithi204 systemd[1]: ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715@mds.c.service: Scheduled restart job, restart counter is at 2. 2024-03-08T21:03:17.008 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:03:16 smithi204 systemd[1]: Stopped Ceph mds.c for dc20f0f8-dd8c-11ee-95c6-87774f69a715. 2024-03-08T21:03:17.008 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:03:16 smithi204 systemd[1]: ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715@mds.c.service: Consumed 1.894s CPU time. 2024-03-08T21:03:17.008 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:03:16 smithi204 systemd[1]: Starting Ceph mds.c for dc20f0f8-dd8c-11ee-95c6-87774f69a715... 2024-03-08T21:03:17.283 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:03:17 smithi204 podman[70266]: 2024-03-08 21:03:16.934777167 +0000 UTC m=+0.035019683 image pull a272146cb810b1fd4a53627f019472132e30966d5c25b01bfc44a001295f1249 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd 2024-03-08T21:03:17.283 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:03:17 smithi204 podman[70266]: 2024-03-08 21:03:17.057756796 +0000 UTC m=+0.157999348 container create d32e1fc0553409c051f342b3038b598a11012caf40e3811e5b7525ee00ca24fd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=rishabh-2024mar8-7d63482, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-03-08T21:03:17.345 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.363+0000 7fb7ff000640 1 -- 172.21.15.107:0/2111986760 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb7f8103f00 msgr2=0x7fb7f8107f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:03:17.345 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.363+0000 7fb7ff000640 1 --2- 172.21.15.107:0/2111986760 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb7f8103f00 0x7fb7f8107f40 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fb7e4009860 tx=0x7fb7e402bb90 comp rx=0 tx=0).stop 2024-03-08T21:03:17.345 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.363+0000 7fb7ff000640 1 -- 172.21.15.107:0/2111986760 shutdown_connections 2024-03-08T21:03:17.345 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.363+0000 7fb7ff000640 1 --2- 172.21.15.107:0/2111986760 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb7f8108620 0x7fb7f81101f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:03:17.345 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.363+0000 7fb7ff000640 1 --2- 172.21.15.107:0/2111986760 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb7f8103f00 0x7fb7f8107f40 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:03:17.345 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.363+0000 7fb7ff000640 1 --2- 172.21.15.107:0/2111986760 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb7f8103550 0x7fb7f8103930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:03:17.345 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.363+0000 7fb7ff000640 1 -- 172.21.15.107:0/2111986760 >> 172.21.15.107:0/2111986760 conn(0x7fb7f80fedd0 msgr2=0x7fb7f81011f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:03:17.345 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.363+0000 7fb7ff000640 1 -- 172.21.15.107:0/2111986760 shutdown_connections 2024-03-08T21:03:17.345 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.363+0000 7fb7ff000640 1 -- 172.21.15.107:0/2111986760 wait complete. 2024-03-08T21:03:17.347 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.364+0000 7fb7ff000640 1 Processor -- start 2024-03-08T21:03:17.347 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.364+0000 7fb7ff000640 1 -- start start 2024-03-08T21:03:17.347 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.365+0000 7fb7ff000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb7f8103550 0x7fb7f819f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:03:17.347 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.365+0000 7fb7ff000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb7f8108620 0x7fb7f819fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:03:17.347 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.365+0000 7fb7ff000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb7f8199980 0x7fb7f8199e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:03:17.347 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.365+0000 7fb7ff000640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7f8113560 con 0x7fb7f8103550 2024-03-08T21:03:17.347 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.365+0000 7fb7ff000640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7f81133e0 con 0x7fb7f8108620 2024-03-08T21:03:17.347 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.365+0000 7fb7ff000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7f81136e0 con 0x7fb7f8199980 2024-03-08T21:03:17.347 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.365+0000 7fb7fdc00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb7f8103550 0x7fb7f819f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:03:17.347 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.365+0000 7fb7fdc00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb7f8103550 0x7fb7f819f780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:56808/0 (socket says 172.21.15.107:56808) 2024-03-08T21:03:17.348 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.365+0000 7fb7fdc00640 1 -- 172.21.15.107:0/1991500072 learned_addr learned my addr 172.21.15.107:0/1991500072 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:03:17.348 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.365+0000 7fb7fe600640 1 --2- 172.21.15.107:0/1991500072 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb7f8199980 0x7fb7f8199e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:03:17.348 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.366+0000 7fb7fd200640 1 --2- 172.21.15.107:0/1991500072 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb7f8108620 0x7fb7f819fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:03:17.349 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.366+0000 7fb7fdc00640 1 -- 172.21.15.107:0/1991500072 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb7f8199980 msgr2=0x7fb7f8199e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:03:17.349 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.366+0000 7fb7fdc00640 1 --2- 172.21.15.107:0/1991500072 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb7f8199980 0x7fb7f8199e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:03:17.349 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.366+0000 7fb7fdc00640 1 -- 172.21.15.107:0/1991500072 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb7f8108620 msgr2=0x7fb7f819fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:03:17.349 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.366+0000 7fb7fdc00640 1 --2- 172.21.15.107:0/1991500072 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb7f8108620 0x7fb7f819fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:03:17.349 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.366+0000 7fb7fdc00640 1 -- 172.21.15.107:0/1991500072 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7f819a4a0 con 0x7fb7f8103550 2024-03-08T21:03:17.349 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.366+0000 7fb7fd200640 1 --2- 172.21.15.107:0/1991500072 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb7f8108620 0x7fb7f819fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T21:03:17.349 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.366+0000 7fb7fdc00640 1 --2- 172.21.15.107:0/1991500072 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb7f8103550 0x7fb7f819f780 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fb7ec00e9c0 tx=0x7fb7ec00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:03:17.349 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.366+0000 7fb7f6000640 1 -- 172.21.15.107:0/1991500072 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7ec00cde0 con 0x7fb7f8103550 2024-03-08T21:03:17.350 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.367+0000 7fb7f6000640 1 -- 172.21.15.107:0/1991500072 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fb7ec0035f0 con 0x7fb7f8103550 2024-03-08T21:03:17.350 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.367+0000 7fb7ff000640 1 -- 172.21.15.107:0/1991500072 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7f819a790 con 0x7fb7f8103550 2024-03-08T21:03:17.350 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.367+0000 7fb7f6000640 1 -- 172.21.15.107:0/1991500072 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7ec010690 con 0x7fb7f8103550 2024-03-08T21:03:17.350 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.367+0000 7fb7ff000640 1 -- 172.21.15.107:0/1991500072 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7f81a6990 con 0x7fb7f8103550 2024-03-08T21:03:17.352 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.370+0000 7fb80336d740 1 Processor -- start 2024-03-08T21:03:17.352 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.370+0000 7fb7f6000640 1 -- 172.21.15.107:0/1991500072 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) v1 ==== 137422+0+0 (secure 0 0 0) 0x7fb7ec0037a0 con 0x7fb7f8103550 2024-03-08T21:03:17.352 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.371+0000 7fb80336d740 1 -- start start 2024-03-08T21:03:17.353 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.371+0000 7fb80336d740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555ae4690e80 0x555ae4691260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:03:17.353 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.371+0000 7fb80336d740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555ae46917a0 0x555ae469a030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:03:17.353 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.371+0000 7fb80336d740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555ae469a570 0x555ae469c960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:03:17.353 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.371+0000 7fb80336d740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ae45f43d0 con 0x555ae469a570 2024-03-08T21:03:17.353 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.371+0000 7fb80336d740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ae459d6b0 con 0x555ae4690e80 2024-03-08T21:03:17.353 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.371+0000 7fb80336d740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ae45f0470 con 0x555ae46917a0 2024-03-08T21:03:17.355 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.372+0000 7fb7fd200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555ae469a570 0x555ae469c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:03:17.355 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.372+0000 7fb7fd200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555ae469a570 0x555ae469c960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:56820/0 (socket says 172.21.15.107:56820) 2024-03-08T21:03:17.355 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.372+0000 7fb7fd200640 1 -- 172.21.15.107:0/3306269991 learned_addr learned my addr 172.21.15.107:0/3306269991 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:03:17.355 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.372+0000 7fb7fe600640 1 --2- 172.21.15.107:0/3306269991 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555ae4690e80 0x555ae4691260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:03:17.355 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.372+0000 7fb7fdc00640 1 --2- 172.21.15.107:0/3306269991 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555ae46917a0 0x555ae469a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:03:17.355 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.372+0000 7fb7f6000640 1 --2- 172.21.15.107:0/1991500072 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7fb7e00a2960 0x7fb7e00a4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:03:17.355 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.372+0000 7fb7fd200640 1 -- 172.21.15.107:0/3306269991 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555ae46917a0 msgr2=0x555ae469a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:03:17.355 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.372+0000 7fb7fd200640 1 --2- 172.21.15.107:0/3306269991 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555ae46917a0 0x555ae469a030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:03:17.355 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.372+0000 7fb7fd200640 1 -- 172.21.15.107:0/3306269991 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555ae4690e80 msgr2=0x555ae4691260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:03:17.356 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.372+0000 7fb7fd200640 1 --2- 172.21.15.107:0/3306269991 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555ae4690e80 0x555ae4691260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:03:17.356 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.372+0000 7fb7fd200640 1 -- 172.21.15.107:0/3306269991 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555ae45c3b50 con 0x555ae469a570 2024-03-08T21:03:17.356 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.373+0000 7fb7f6000640 1 -- 172.21.15.107:0/1991500072 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fb7ec0cf250 con 0x7fb7f8103550 2024-03-08T21:03:17.356 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.373+0000 7fb7fe600640 1 --2- 172.21.15.107:0/1991500072 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7fb7e00a2960 0x7fb7e00a4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:03:17.356 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.373+0000 7fb7fd200640 1 --2- 172.21.15.107:0/3306269991 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555ae469a570 0x555ae469c960 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fb7e4000c00 tx=0x7fb7e4005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b63893101f9e7aba server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:03:17.356 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.373+0000 7fb7eaa00640 1 -- 172.21.15.107:0/3306269991 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7e4039040 con 0x555ae469a570 2024-03-08T21:03:17.356 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.373+0000 7fb7eaa00640 1 -- 172.21.15.107:0/3306269991 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fb7e4034700 con 0x555ae469a570 2024-03-08T21:03:17.356 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.373+0000 7fb7eaa00640 1 -- 172.21.15.107:0/3306269991 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7e4005310 con 0x555ae469a570 2024-03-08T21:03:17.360 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.374+0000 7fb80336d740 1 -- 172.21.15.107:0/3306269991 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555ae469a570 msgr2=0x555ae469c960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:03:17.360 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.374+0000 7fb80336d740 1 --2- 172.21.15.107:0/3306269991 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555ae469a570 0x555ae469c960 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fb7e4000c00 tx=0x7fb7e4005dc0 comp rx=0 tx=0).stop 2024-03-08T21:03:17.360 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.374+0000 7fb7fe600640 1 --2- 172.21.15.107:0/1991500072 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7fb7e00a2960 0x7fb7e00a4e20 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fb7f000a2c0 tx=0x7fb7f0004a40 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:03:17.360 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.374+0000 7fb80336d740 1 -- 172.21.15.107:0/3306269991 shutdown_connections 2024-03-08T21:03:17.361 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.374+0000 7fb80336d740 1 --2- 172.21.15.107:0/3306269991 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555ae469a570 0x555ae469c960 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:03:17.361 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.374+0000 7fb80336d740 1 --2- 172.21.15.107:0/3306269991 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555ae46917a0 0x555ae469a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:03:17.361 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.374+0000 7fb80336d740 1 --2- 172.21.15.107:0/3306269991 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555ae4690e80 0x555ae4691260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:03:17.361 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.374+0000 7fb80336d740 1 -- 172.21.15.107:0/3306269991 >> 172.21.15.107:0/3306269991 conn(0x555ae468b7a0 msgr2=0x555ae468bb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:03:17.361 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.374+0000 7fb80336d740 1 -- 172.21.15.107:0/3306269991 shutdown_connections 2024-03-08T21:03:17.361 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.374+0000 7fb80336d740 1 -- 172.21.15.107:0/3306269991 wait complete. 2024-03-08T21:03:17.361 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.375+0000 7fb80336d740 1 -- start start 2024-03-08T21:03:17.361 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.375+0000 7fb80336d740 1 Processor -- start 2024-03-08T21:03:17.361 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.375+0000 7fb80336d740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555ae4690e80 0x555ae4699ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:03:17.361 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.375+0000 7fb80336d740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555ae46917a0 0x555ae4697280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:03:17.361 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.375+0000 7fb80336d740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555ae469a570 0x555ae4697800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:03:17.361 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.376+0000 7fb80336d740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ae469d140 con 0x555ae4690e80 2024-03-08T21:03:17.361 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.376+0000 7fb80336d740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ae45b09f0 con 0x555ae46917a0 2024-03-08T21:03:17.362 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.376+0000 7fb80336d740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ae469d2c0 con 0x555ae469a570 2024-03-08T21:03:17.362 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.376+0000 7fb80336d740 10 client.0 _ll_register_callbacks cb 0x555ae4308a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T21:03:17.362 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.376+0000 7fb7fdc00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555ae4690e80 0x555ae4699ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:03:17.362 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.376+0000 7fb7fdc00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555ae4690e80 0x555ae4699ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:56824/0 (socket says 172.21.15.107:56824) 2024-03-08T21:03:17.362 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.376+0000 7fb7fdc00640 1 -- 172.21.15.107:0/1650302604 learned_addr learned my addr 172.21.15.107:0/1650302604 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:03:17.362 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.376+0000 7fb7fe600640 1 --2- 172.21.15.107:0/1650302604 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555ae469a570 0x555ae4697800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:03:17.362 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.376+0000 7fb7fd200640 1 --2- 172.21.15.107:0/1650302604 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555ae46917a0 0x555ae4697280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:03:17.362 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.376+0000 7fb7fdc00640 1 -- 172.21.15.107:0/1650302604 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555ae469a570 msgr2=0x555ae4697800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:03:17.362 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.376+0000 7fb7fdc00640 1 --2- 172.21.15.107:0/1650302604 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555ae469a570 0x555ae4697800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:03:17.362 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.376+0000 7fb7fdc00640 1 -- 172.21.15.107:0/1650302604 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555ae46917a0 msgr2=0x555ae4697280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:03:17.362 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.376+0000 7fb7fdc00640 1 --2- 172.21.15.107:0/1650302604 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555ae46917a0 0x555ae4697280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:03:17.362 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.376+0000 7fb7fdc00640 1 -- 172.21.15.107:0/1650302604 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555ae4309210 con 0x555ae4690e80 2024-03-08T21:03:17.362 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.376+0000 7fb7fd200640 1 --2- 172.21.15.107:0/1650302604 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555ae46917a0 0x555ae4697280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T21:03:17.363 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.377+0000 7fb7fdc00640 1 --2- 172.21.15.107:0/1650302604 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555ae4690e80 0x555ae4699ef0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fb7ec00c6a0 tx=0x7fb7ec09b380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea50700fe49c571c server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:03:17.363 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.377+0000 7fb7df400640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-03-08T21:03:17.363 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.377+0000 7fb7df400640 1 -- 172.21.15.107:0/1650302604 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7ec0ccdb0 con 0x555ae4690e80 2024-03-08T21:03:17.363 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.377+0000 7fb7df400640 1 -- 172.21.15.107:0/1650302604 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fb7ec09cb70 con 0x555ae4690e80 2024-03-08T21:03:17.363 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.377+0000 7fb80336d740 1 -- 172.21.15.107:0/1650302604 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb7e4034700 con 0x555ae4690e80 2024-03-08T21:03:17.363 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.377+0000 7fb7df400640 1 -- 172.21.15.107:0/1650302604 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb7ec009510 con 0x555ae4690e80 2024-03-08T21:03:17.363 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.377+0000 7fb80336d740 10 client.15690 fetch_fsmap learned FSMap version 30 2024-03-08T21:03:17.363 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.377+0000 7fb80336d740 1 -- 172.21.15.107:0/1650302604 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x555ae43d5620 con 0x555ae4690e80 2024-03-08T21:03:17.363 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.377+0000 7fb7df400640 1 -- 172.21.15.107:0/1650302604 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fb7e4034700 con 0x555ae4690e80 2024-03-08T21:03:17.363 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.378+0000 7fb7df400640 1 -- 172.21.15.107:0/1650302604 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 30) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fb7ec097590 con 0x555ae4690e80 2024-03-08T21:03:17.363 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.378+0000 7fb80336d740 10 client.15690 fetch_fsmap finished waiting for FSMap version 30 2024-03-08T21:03:17.363 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.378+0000 7fb80336d740 10 client.15690 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-08T21:03:17.363 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.378+0000 7fb80336d740 20 client.15690 populate_metadata read hostname 'smithi107' 2024-03-08T21:03:17.363 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.378+0000 7fb80336d740 1 --2- 172.21.15.107:0/1650302604 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x555ae46aa380 0x555ae46ac790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:03:17.364 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.378+0000 7fb80336d740 4 client.15690 mds_command: new command op to 34475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-08T21:03:17.364 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.378+0000 7fb80336d740 1 -- 172.21.15.107:0/1650302604 --> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555ae4601910 con 0x555ae46aa380 2024-03-08T21:03:17.364 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.379+0000 7fb7fd200640 1 --2- 172.21.15.107:0/1650302604 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x555ae46aa380 0x555ae46ac790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:03:17.365 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.379+0000 7fb7fd200640 1 --2- 172.21.15.107:0/1650302604 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x555ae46aa380 0x555ae46ac790 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e5c2053032c0cdaf server_cookie=abad24b139d5eb73 in_seq=0 out_seq=0 2024-03-08T21:03:17.365 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.379+0000 7fb7df400640 10 client.15690 ms_handle_connect on v2:172.21.15.149:6838/55290596 2024-03-08T21:03:17.365 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.381+0000 7fb7df400640 1 -- 172.21.15.107:0/1650302604 <== mds.0 v2:172.21.15.149:6838/55290596 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x555ae4601910 con 0x555ae46aa380 2024-03-08T21:03:17.365 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.381+0000 7fb7df400640 10 client.15690 handle_command_reply: tid=0 2024-03-08T21:03:17.365 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.381+0000 7fb80336d740 1 client.15690 shutdown 2024-03-08T21:03:17.365 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.381+0000 7fb80336d740 1 -- 172.21.15.107:0/1650302604 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555ae4690e80 msgr2=0x555ae4699ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:03:17.365 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.381+0000 7fb80336d740 1 --2- 172.21.15.107:0/1650302604 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555ae4690e80 0x555ae4699ef0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fb7ec00c6a0 tx=0x7fb7ec09b380 comp rx=0 tx=0).stop 2024-03-08T21:03:17.365 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.381+0000 7fb80336d740 1 -- 172.21.15.107:0/1650302604 shutdown_connections 2024-03-08T21:03:17.365 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.381+0000 7fb80336d740 1 --2- 172.21.15.107:0/1650302604 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555ae469a570 0x555ae4697800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:03:17.365 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.381+0000 7fb80336d740 1 --2- 172.21.15.107:0/1650302604 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x555ae46aa380 0x555ae46ac790 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:03:17.366 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.381+0000 7fb80336d740 1 --2- 172.21.15.107:0/1650302604 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555ae46917a0 0x555ae4697280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:03:17.366 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.381+0000 7fb80336d740 1 --2- 172.21.15.107:0/1650302604 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555ae4690e80 0x555ae4699ef0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:03:17.366 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.381+0000 7fb7df400640 0 client.15690 ms_handle_reset on v2:172.21.15.149:6838/55290596 2024-03-08T21:03:17.366 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.381+0000 7fb80336d740 1 -- 172.21.15.107:0/1650302604 >> 172.21.15.107:0/1650302604 conn(0x555ae468b7a0 msgr2=0x555ae468d4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:03:17.366 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.381+0000 7fb80336d740 1 -- 172.21.15.107:0/1650302604 shutdown_connections 2024-03-08T21:03:17.366 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.381+0000 7fb80336d740 1 -- 172.21.15.107:0/1650302604 wait complete. 2024-03-08T21:03:17.366 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.382+0000 7fb80336d740 20 client.15690 trim_cache size 0 max 16384 2024-03-08T21:03:17.384 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.402+0000 7fb80336d740 1 Processor -- start 2024-03-08T21:03:17.384 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.402+0000 7fb80336d740 1 -- start start 2024-03-08T21:03:17.384 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.403+0000 7fb80336d740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555ae4690e80 0x555ae4697220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:03:17.385 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.403+0000 7fb80336d740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555ae46917a0 0x555ae46999b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:03:17.385 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.403+0000 7fb80336d740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555ae469a570 0x555ae4699ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:03:17.385 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.403+0000 7fb80336d740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7f81aec60 con 0x555ae469a570 2024-03-08T21:03:17.385 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.403+0000 7fb80336d740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7f81b05e0 con 0x555ae4690e80 2024-03-08T21:03:17.385 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.403+0000 7fb80336d740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7f81af2c0 con 0x555ae46917a0 2024-03-08T21:03:17.387 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.403+0000 7fb7fe600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555ae469a570 0x555ae4699ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:03:17.387 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.403+0000 7fb7fe600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555ae469a570 0x555ae4699ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:56828/0 (socket says 172.21.15.107:56828) 2024-03-08T21:03:17.387 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.403+0000 7fb7fe600640 1 -- 172.21.15.107:0/2517926549 learned_addr learned my addr 172.21.15.107:0/2517926549 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:03:17.387 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.403+0000 7fb7fd200640 1 --2- 172.21.15.107:0/2517926549 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555ae46917a0 0x555ae46999b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:03:17.387 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.404+0000 7fb7fdc00640 1 --2- 172.21.15.107:0/2517926549 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555ae4690e80 0x555ae4697220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:03:17.388 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.404+0000 7fb7fe600640 1 -- 172.21.15.107:0/2517926549 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555ae46917a0 msgr2=0x555ae46999b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:03:17.388 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.404+0000 7fb7fe600640 1 --2- 172.21.15.107:0/2517926549 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555ae46917a0 0x555ae46999b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:03:17.388 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.404+0000 7fb7fe600640 1 -- 172.21.15.107:0/2517926549 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555ae4690e80 msgr2=0x555ae4697220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:03:17.388 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.404+0000 7fb7fe600640 1 --2- 172.21.15.107:0/2517926549 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555ae4690e80 0x555ae4697220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:03:17.388 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.404+0000 7fb7fe600640 1 -- 172.21.15.107:0/2517926549 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555ae4690640 con 0x555ae469a570 2024-03-08T21:03:17.388 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.404+0000 7fb7fd200640 1 --2- 172.21.15.107:0/2517926549 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555ae46917a0 0x555ae46999b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T21:03:17.388 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.404+0000 7fb7fdc00640 1 --2- 172.21.15.107:0/2517926549 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555ae4690e80 0x555ae4697220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T21:03:17.388 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.404+0000 7fb7fe600640 1 --2- 172.21.15.107:0/2517926549 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555ae469a570 0x555ae4699ef0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fb7f001e740 tx=0x7fb7f0015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae4f99211e60e687 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:03:17.388 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.404+0000 7fb7ea000640 1 -- 172.21.15.107:0/2517926549 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7f000f6e0 con 0x555ae469a570 2024-03-08T21:03:17.389 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.405+0000 7fb7ea000640 1 -- 172.21.15.107:0/2517926549 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fb7f000fd30 con 0x555ae469a570 2024-03-08T21:03:17.389 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.405+0000 7fb7ea000640 1 -- 172.21.15.107:0/2517926549 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7f001be30 con 0x555ae469a570 2024-03-08T21:03:17.389 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.405+0000 7fb80336d740 1 -- 172.21.15.107:0/2517926549 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555ae469a570 msgr2=0x555ae4699ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:03:17.389 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.405+0000 7fb80336d740 1 --2- 172.21.15.107:0/2517926549 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555ae469a570 0x555ae4699ef0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fb7f001e740 tx=0x7fb7f0015000 comp rx=0 tx=0).stop 2024-03-08T21:03:17.389 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.405+0000 7fb80336d740 1 -- 172.21.15.107:0/2517926549 shutdown_connections 2024-03-08T21:03:17.389 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.405+0000 7fb80336d740 1 --2- 172.21.15.107:0/2517926549 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555ae469a570 0x555ae4699ef0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:03:17.389 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.405+0000 7fb80336d740 1 --2- 172.21.15.107:0/2517926549 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555ae46917a0 0x555ae46999b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:03:17.389 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.405+0000 7fb80336d740 1 --2- 172.21.15.107:0/2517926549 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555ae4690e80 0x555ae4697220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:03:17.389 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.405+0000 7fb80336d740 1 -- 172.21.15.107:0/2517926549 >> 172.21.15.107:0/2517926549 conn(0x555ae468b7a0 msgr2=0x555ae4696b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:03:17.389 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.405+0000 7fb80336d740 1 -- 172.21.15.107:0/2517926549 shutdown_connections 2024-03-08T21:03:17.389 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.406+0000 7fb80336d740 1 -- 172.21.15.107:0/2517926549 wait complete. 2024-03-08T21:03:17.389 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.406+0000 7fb80336d740 1 -- start start 2024-03-08T21:03:17.395 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.406+0000 7fb80336d740 1 Processor -- start 2024-03-08T21:03:17.395 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.407+0000 7fb80336d740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555ae4690e80 0x555ae46a1800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:03:17.395 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.407+0000 7fb80336d740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555ae46917a0 0x555ae469fba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:03:17.395 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.407+0000 7fb80336d740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555ae469a570 0x555ae46a00e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:03:17.395 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.407+0000 7fb80336d740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7f81aff80 con 0x555ae4690e80 2024-03-08T21:03:17.395 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.407+0000 7fb80336d740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7f81af920 con 0x555ae469a570 2024-03-08T21:03:17.395 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.407+0000 7fb80336d740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7f81acf00 con 0x555ae46917a0 2024-03-08T21:03:17.395 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.407+0000 7fb80336d740 10 client.0 _ll_register_callbacks cb 0x7fb7e4037070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T21:03:17.396 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.407+0000 7fb7fdc00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555ae4690e80 0x555ae46a1800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:03:17.396 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.407+0000 7fb7fdc00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555ae4690e80 0x555ae46a1800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:56838/0 (socket says 172.21.15.107:56838) 2024-03-08T21:03:17.396 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.407+0000 7fb7fdc00640 1 -- 172.21.15.107:0/1641793991 learned_addr learned my addr 172.21.15.107:0/1641793991 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:03:17.396 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.407+0000 7fb7fd200640 1 --2- 172.21.15.107:0/1641793991 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555ae46917a0 0x555ae469fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:03:17.396 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.407+0000 7fb7fe600640 1 --2- 172.21.15.107:0/1641793991 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555ae469a570 0x555ae46a00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:03:17.396 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.407+0000 7fb7fdc00640 1 -- 172.21.15.107:0/1641793991 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555ae46917a0 msgr2=0x555ae469fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:03:17.396 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.407+0000 7fb7fdc00640 1 --2- 172.21.15.107:0/1641793991 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555ae46917a0 0x555ae469fba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:03:17.396 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.407+0000 7fb7fdc00640 1 -- 172.21.15.107:0/1641793991 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555ae469a570 msgr2=0x555ae46a00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:03:17.396 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.407+0000 7fb7fdc00640 1 --2- 172.21.15.107:0/1641793991 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555ae469a570 0x555ae46a00e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:03:17.396 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.407+0000 7fb7fdc00640 1 -- 172.21.15.107:0/1641793991 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555ae46a0890 con 0x555ae4690e80 2024-03-08T21:03:17.396 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.408+0000 7fb7fdc00640 1 --2- 172.21.15.107:0/1641793991 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555ae4690e80 0x555ae46a1800 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fb7ec0a3ab0 tx=0x7fb7ec0974e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b0cc8547c36dbfa server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:03:17.396 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.408+0000 7fb7df400640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-03-08T21:03:17.396 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.408+0000 7fb7df400640 1 -- 172.21.15.107:0/1641793991 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7ec097ae0 con 0x555ae4690e80 2024-03-08T21:03:17.397 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.408+0000 7fb7df400640 1 -- 172.21.15.107:0/1641793991 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fb7ec09ae90 con 0x555ae4690e80 2024-03-08T21:03:17.397 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.408+0000 7fb80336d740 1 -- 172.21.15.107:0/1641793991 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb7f000fd30 con 0x555ae4690e80 2024-03-08T21:03:17.397 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.408+0000 7fb7df400640 1 -- 172.21.15.107:0/1641793991 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb7ec09b1a0 con 0x555ae4690e80 2024-03-08T21:03:17.397 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.408+0000 7fb80336d740 10 client.15696 fetch_fsmap learned FSMap version 30 2024-03-08T21:03:17.397 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.408+0000 7fb80336d740 1 -- 172.21.15.107:0/1641793991 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x555ae47018b0 con 0x555ae4690e80 2024-03-08T21:03:17.397 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.408+0000 7fb7df400640 1 -- 172.21.15.107:0/1641793991 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fb7f000fd30 con 0x555ae4690e80 2024-03-08T21:03:17.397 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.409+0000 7fb7df400640 1 -- 172.21.15.107:0/1641793991 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 30) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fb7ec0a0a40 con 0x555ae4690e80 2024-03-08T21:03:17.397 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.409+0000 7fb80336d740 10 client.15696 fetch_fsmap finished waiting for FSMap version 30 2024-03-08T21:03:17.397 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.409+0000 7fb80336d740 10 client.15696 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-08T21:03:17.397 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.409+0000 7fb80336d740 20 client.15696 populate_metadata read hostname 'smithi107' 2024-03-08T21:03:17.397 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.409+0000 7fb80336d740 1 --2- 172.21.15.107:0/1641793991 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x555ae4701f60 0x555ae4704380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:03:17.397 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.409+0000 7fb80336d740 4 client.15696 mds_command: new command op to 34475 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-08T21:03:17.397 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.409+0000 7fb80336d740 1 -- 172.21.15.107:0/1641793991 --> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x555ae45fbc60 con 0x555ae4701f60 2024-03-08T21:03:17.397 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.409+0000 7fb7fd200640 1 --2- 172.21.15.107:0/1641793991 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x555ae4701f60 0x555ae4704380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:03:17.398 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.410+0000 7fb7fd200640 1 --2- 172.21.15.107:0/1641793991 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x555ae4701f60 0x555ae4704380 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7978e2cbf9027cd server_cookie=f988ee4c3389d9b6 in_seq=0 out_seq=0 2024-03-08T21:03:17.398 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.410+0000 7fb7df400640 10 client.15696 ms_handle_connect on v2:172.21.15.149:6838/55290596 2024-03-08T21:03:17.404 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.421+0000 7fb7df400640 1 -- 172.21.15.107:0/1641793991 <== mds.0 v2:172.21.15.149:6838/55290596 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x555ae45fbc60 con 0x555ae4701f60 2024-03-08T21:03:17.404 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.421+0000 7fb7df400640 10 client.15696 handle_command_reply: tid=0 2024-03-08T21:03:17.404 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.421+0000 7fb80336d740 1 client.15696 shutdown 2024-03-08T21:03:17.404 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.422+0000 7fb80336d740 1 -- 172.21.15.107:0/1641793991 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555ae4690e80 msgr2=0x555ae46a1800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:03:17.404 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.422+0000 7fb80336d740 1 --2- 172.21.15.107:0/1641793991 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555ae4690e80 0x555ae46a1800 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fb7ec0a3ab0 tx=0x7fb7ec0974e0 comp rx=0 tx=0).stop 2024-03-08T21:03:17.404 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.422+0000 7fb80336d740 1 -- 172.21.15.107:0/1641793991 shutdown_connections 2024-03-08T21:03:17.404 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.422+0000 7fb80336d740 1 --2- 172.21.15.107:0/1641793991 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x555ae4701f60 0x555ae4704380 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:03:17.404 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.422+0000 7fb80336d740 1 --2- 172.21.15.107:0/1641793991 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x555ae469a570 0x555ae46a00e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:03:17.404 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.422+0000 7fb7df400640 0 client.15696 ms_handle_reset on v2:172.21.15.149:6838/55290596 2024-03-08T21:03:17.404 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.422+0000 7fb80336d740 1 --2- 172.21.15.107:0/1641793991 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x555ae46917a0 0x555ae469fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:03:17.404 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.422+0000 7fb80336d740 1 --2- 172.21.15.107:0/1641793991 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555ae4690e80 0x555ae46a1800 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:03:17.404 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.422+0000 7fb80336d740 1 -- 172.21.15.107:0/1641793991 >> 172.21.15.107:0/1641793991 conn(0x555ae468b7a0 msgr2=0x555ae468cbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:03:17.404 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.422+0000 7fb80336d740 1 -- 172.21.15.107:0/1641793991 shutdown_connections 2024-03-08T21:03:17.405 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.422+0000 7fb80336d740 1 -- 172.21.15.107:0/1641793991 wait complete. 2024-03-08T21:03:17.405 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.422+0000 7fb80336d740 20 client.15696 trim_cache size 0 max 16384 2024-03-08T21:03:17.405 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:{ 2024-03-08T21:03:17.405 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "message": "", 2024-03-08T21:03:17.405 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "return_code": 0 2024-03-08T21:03:17.405 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:} 2024-03-08T21:03:17.406 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.423+0000 7fb7ff000640 1 -- 172.21.15.107:0/1991500072 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7fb7e00a2960 msgr2=0x7fb7e00a4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:03:17.406 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.423+0000 7fb7ff000640 1 --2- 172.21.15.107:0/1991500072 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7fb7e00a2960 0x7fb7e00a4e20 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fb7f000a2c0 tx=0x7fb7f0004a40 comp rx=0 tx=0).stop 2024-03-08T21:03:17.406 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.423+0000 7fb7ff000640 1 -- 172.21.15.107:0/1991500072 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb7f8103550 msgr2=0x7fb7f819f780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:03:17.406 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.423+0000 7fb7ff000640 1 --2- 172.21.15.107:0/1991500072 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb7f8103550 0x7fb7f819f780 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fb7ec00e9c0 tx=0x7fb7ec00ee80 comp rx=0 tx=0).stop 2024-03-08T21:03:17.406 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.423+0000 7fb7ff000640 1 -- 172.21.15.107:0/1991500072 shutdown_connections 2024-03-08T21:03:17.406 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.423+0000 7fb7ff000640 1 --2- 172.21.15.107:0/1991500072 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7fb7e00a2960 0x7fb7e00a4e20 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:03:17.406 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.423+0000 7fb7ff000640 1 --2- 172.21.15.107:0/1991500072 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb7f8199980 0x7fb7f8199e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:03:17.406 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.423+0000 7fb7ff000640 1 --2- 172.21.15.107:0/1991500072 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb7f8108620 0x7fb7f819fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:03:17.407 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.423+0000 7fb7ff000640 1 --2- 172.21.15.107:0/1991500072 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb7f8103550 0x7fb7f819f780 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:03:17.407 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.423+0000 7fb7ff000640 1 -- 172.21.15.107:0/1991500072 >> 172.21.15.107:0/1991500072 conn(0x7fb7f80fedd0 msgr2=0x7fb7f81001c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:03:17.407 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.423+0000 7fb7ff000640 1 -- 172.21.15.107:0/1991500072 shutdown_connections 2024-03-08T21:03:17.407 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:17.423+0000 7fb7ff000640 1 -- 172.21.15.107:0/1991500072 wait complete. 2024-03-08T21:03:17.602 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:03:17 smithi204 podman[70266]: 2024-03-08 21:03:17.302833021 +0000 UTC m=+0.403075556 container init d32e1fc0553409c051f342b3038b598a11012caf40e3811e5b7525ee00ca24fd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, RELEASE=rishabh-2024mar8-7d63482, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307) 2024-03-08T21:03:17.602 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:03:17 smithi204 podman[70266]: 2024-03-08 21:03:17.31268956 +0000 UTC m=+0.412932121 container start d32e1fc0553409c051f342b3038b598a11012caf40e3811e5b7525ee00ca24fd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, RELEASE=rishabh-2024mar8-7d63482, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-03-08T21:03:17.602 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:03:17 smithi204 ceph-mds[70285]: -- 172.21.15.204:0/4169658692 <== mon.1 v2:172.21.15.204:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x557bc5d6da40 con 0x557bc5d41000 2024-03-08T21:03:17.602 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:03:17 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[70281]: starting mds.c at 2024-03-08T21:03:17.602 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:03:17 smithi204 bash[70266]: d32e1fc0553409c051f342b3038b598a11012caf40e3811e5b7525ee00ca24fd 2024-03-08T21:03:17.602 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:03:17 smithi204 systemd[1]: Started Ceph mds.c for dc20f0f8-dd8c-11ee-95c6-87774f69a715. 2024-03-08T21:03:17.876 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:03:17 smithi204 ceph-mon[34935]: pgmap v86: 97 pgs: 97 active+clean; 2.0 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-03-08T21:03:17.876 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:03:17 smithi204 ceph-mon[34935]: daemon mds.c restarted 2024-03-08T21:03:17.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:03:17 smithi107 ceph-mon[31381]: pgmap v86: 97 pgs: 97 active+clean; 2.0 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-03-08T21:03:17.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:03:17 smithi107 ceph-mon[31381]: daemon mds.c restarted 2024-03-08T21:03:18.013 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:03:17 smithi149 ceph-mon[35042]: pgmap v86: 97 pgs: 97 active+clean; 2.0 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-03-08T21:03:18.013 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:03:17 smithi149 ceph-mon[35042]: daemon mds.c restarted 2024-03-08T21:03:18.876 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:03:18 smithi204 ceph-mon[34935]: osdmap e91: 12 total, 12 up, 12 in 2024-03-08T21:03:18.876 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:03:18 smithi204 ceph-mon[34935]: mds.? [v2:172.21.15.204:6836/215440450,v1:172.21.15.204:6837/215440450] up:boot 2024-03-08T21:03:18.876 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:03:18 smithi204 ceph-mon[34935]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-08T21:03:18.876 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:03:18 smithi204 ceph-mon[34935]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-03-08T21:03:18.877 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:03:18 smithi204 ceph-mon[34935]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-08T21:03:18.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:03:18 smithi107 ceph-mon[31381]: osdmap e91: 12 total, 12 up, 12 in 2024-03-08T21:03:18.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:03:18 smithi107 ceph-mon[31381]: mds.? [v2:172.21.15.204:6836/215440450,v1:172.21.15.204:6837/215440450] up:boot 2024-03-08T21:03:18.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:03:18 smithi107 ceph-mon[31381]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-08T21:03:18.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:03:18 smithi107 ceph-mon[31381]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-03-08T21:03:18.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:03:18 smithi107 ceph-mon[31381]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-08T21:03:19.013 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:03:18 smithi149 ceph-mon[35042]: osdmap e91: 12 total, 12 up, 12 in 2024-03-08T21:03:19.013 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:03:18 smithi149 ceph-mon[35042]: mds.? [v2:172.21.15.204:6836/215440450,v1:172.21.15.204:6837/215440450] up:boot 2024-03-08T21:03:19.013 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:03:18 smithi149 ceph-mon[35042]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-08T21:03:19.013 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:03:18 smithi149 ceph-mon[35042]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-03-08T21:03:19.013 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:03:18 smithi149 ceph-mon[35042]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-08T21:03:19.876 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:03:19 smithi204 ceph-mon[34935]: pgmap v88: 97 pgs: 97 active+clean; 2.0 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 2.0 KiB/s wr, 11 op/s 2024-03-08T21:03:19.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:03:19 smithi107 ceph-mon[31381]: pgmap v88: 97 pgs: 97 active+clean; 2.0 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 2.0 KiB/s wr, 11 op/s 2024-03-08T21:03:20.013 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:03:19 smithi149 ceph-mon[35042]: pgmap v88: 97 pgs: 97 active+clean; 2.0 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 2.0 KiB/s wr, 11 op/s 2024-03-08T21:03:21.876 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:03:21 smithi204 ceph-mon[34935]: pgmap v89: 97 pgs: 97 active+clean; 2.0 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 307 B/s wr, 9 op/s 2024-03-08T21:03:21.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:03:21 smithi107 ceph-mon[31381]: pgmap v89: 97 pgs: 97 active+clean; 2.0 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 307 B/s wr, 9 op/s 2024-03-08T21:03:22.013 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:03:21 smithi149 ceph-mon[35042]: pgmap v89: 97 pgs: 97 active+clean; 2.0 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 307 B/s wr, 9 op/s 2024-03-08T21:03:23.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:03:23 smithi107 ceph-mon[31381]: pgmap v90: 97 pgs: 97 active+clean; 2.0 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 307 B/s wr, 9 op/s 2024-03-08T21:03:24.012 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:03:23 smithi149 ceph-mon[35042]: pgmap v90: 97 pgs: 97 active+clean; 2.0 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 307 B/s wr, 9 op/s 2024-03-08T21:03:24.126 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:03:23 smithi204 ceph-mon[34935]: pgmap v90: 97 pgs: 97 active+clean; 2.0 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 307 B/s wr, 9 op/s 2024-03-08T21:03:25.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:03:25 smithi107 ceph-mon[31381]: pgmap v91: 97 pgs: 97 active+clean; 2.0 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 307 B/s wr, 10 op/s 2024-03-08T21:03:26.013 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:03:25 smithi149 ceph-mon[35042]: pgmap v91: 97 pgs: 97 active+clean; 2.0 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 307 B/s wr, 10 op/s 2024-03-08T21:03:26.126 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:03:25 smithi204 ceph-mon[34935]: pgmap v91: 97 pgs: 97 active+clean; 2.0 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 307 B/s wr, 10 op/s 2024-03-08T21:03:26.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:03:26 smithi107 ceph-mon[31381]: pgmap v92: 97 pgs: 97 active+clean; 2.0 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 307 B/s wr, 10 op/s 2024-03-08T21:03:27.013 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:03:26 smithi149 ceph-mon[35042]: pgmap v92: 97 pgs: 97 active+clean; 2.0 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 307 B/s wr, 10 op/s 2024-03-08T21:03:27.126 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:03:26 smithi204 ceph-mon[34935]: pgmap v92: 97 pgs: 97 active+clean; 2.0 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 307 B/s wr, 10 op/s 2024-03-08T21:03:29.876 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:03:29 smithi204 ceph-mon[34935]: pgmap v93: 97 pgs: 97 active+clean; 2.0 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 280 B/s wr, 9 op/s 2024-03-08T21:03:29.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:03:29 smithi107 ceph-mon[31381]: pgmap v93: 97 pgs: 97 active+clean; 2.0 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 280 B/s wr, 9 op/s 2024-03-08T21:03:30.013 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:03:29 smithi149 ceph-mon[35042]: pgmap v93: 97 pgs: 97 active+clean; 2.0 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 280 B/s wr, 9 op/s 2024-03-08T21:03:30.552 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.570+0000 7f7a68a00640 1 -- 172.21.15.107:0/393147227 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7a64105310 msgr2=0x7f7a6410cee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:03:30.553 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.570+0000 7f7a68a00640 1 --2- 172.21.15.107:0/393147227 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7a64105310 0x7f7a6410cee0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f7a5c009ca0 tx=0x7f7a5c02c0f0 comp rx=0 tx=0).stop 2024-03-08T21:03:30.553 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.570+0000 7f7a68a00640 1 -- 172.21.15.107:0/393147227 shutdown_connections 2024-03-08T21:03:30.553 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.570+0000 7f7a68a00640 1 --2- 172.21.15.107:0/393147227 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7a64105310 0x7f7a6410cee0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:03:30.553 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.570+0000 7f7a68a00640 1 --2- 172.21.15.107:0/393147227 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7a64100bf0 0x7f7a64104c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:03:30.553 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.570+0000 7f7a68a00640 1 --2- 172.21.15.107:0/393147227 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7a64100240 0x7f7a64100620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:03:30.553 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.570+0000 7f7a68a00640 1 -- 172.21.15.107:0/393147227 >> 172.21.15.107:0/393147227 conn(0x7f7a640fbb00 msgr2=0x7f7a640fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:03:30.553 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.571+0000 7f7a68a00640 1 -- 172.21.15.107:0/393147227 shutdown_connections 2024-03-08T21:03:30.553 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.571+0000 7f7a68a00640 1 -- 172.21.15.107:0/393147227 wait complete. 2024-03-08T21:03:30.555 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.572+0000 7f7a68a00640 1 Processor -- start 2024-03-08T21:03:30.555 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.572+0000 7f7a68a00640 1 -- start start 2024-03-08T21:03:30.555 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.572+0000 7f7a68a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7a64100240 0x7f7a64101cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:03:30.556 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.572+0000 7f7a68a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7a64105310 0x7f7a64102210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:03:30.556 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.572+0000 7f7a68a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7a64102750 0x7f7a641a5ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:03:30.556 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.572+0000 7f7a68a00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a64110280 con 0x7f7a64100240 2024-03-08T21:03:30.556 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.572+0000 7f7a68a00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a64110100 con 0x7f7a64105310 2024-03-08T21:03:30.556 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.572+0000 7f7a68a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a64110400 con 0x7f7a64102750 2024-03-08T21:03:30.556 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.572+0000 7f7a62800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7a64100240 0x7f7a64101cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:03:30.556 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.573+0000 7f7a62800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7a64100240 0x7f7a64101cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:38902/0 (socket says 172.21.15.107:38902) 2024-03-08T21:03:30.556 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.573+0000 7f7a62800640 1 -- 172.21.15.107:0/2801499111 learned_addr learned my addr 172.21.15.107:0/2801499111 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:03:30.556 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.573+0000 7f7a63200640 1 --2- 172.21.15.107:0/2801499111 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7a64102750 0x7f7a641a5ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:03:30.556 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.573+0000 7f7a61e00640 1 --2- 172.21.15.107:0/2801499111 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7a64105310 0x7f7a64102210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:03:30.556 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.573+0000 7f7a62800640 1 -- 172.21.15.107:0/2801499111 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7a64102750 msgr2=0x7f7a641a5ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:03:30.556 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.573+0000 7f7a62800640 1 --2- 172.21.15.107:0/2801499111 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7a64102750 0x7f7a641a5ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:03:30.556 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.573+0000 7f7a62800640 1 -- 172.21.15.107:0/2801499111 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7a64105310 msgr2=0x7f7a64102210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:03:30.556 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.573+0000 7f7a62800640 1 --2- 172.21.15.107:0/2801499111 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7a64105310 0x7f7a64102210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:03:30.557 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.573+0000 7f7a62800640 1 -- 172.21.15.107:0/2801499111 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a641a61c0 con 0x7f7a64100240 2024-03-08T21:03:30.557 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.573+0000 7f7a63200640 1 --2- 172.21.15.107:0/2801499111 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7a64102750 0x7f7a641a5ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T21:03:30.557 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.573+0000 7f7a62800640 1 --2- 172.21.15.107:0/2801499111 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7a64100240 0x7f7a64101cd0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f7a5400d770 tx=0x7f7a5400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:03:30.557 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.575+0000 7f7a5b400640 1 -- 172.21.15.107:0/2801499111 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a54017070 con 0x7f7a64100240 2024-03-08T21:03:30.557 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.575+0000 7f7a5b400640 1 -- 172.21.15.107:0/2801499111 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f7a540054d0 con 0x7f7a64100240 2024-03-08T21:03:30.557 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.575+0000 7f7a68a00640 1 -- 172.21.15.107:0/2801499111 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a641a64b0 con 0x7f7a64100240 2024-03-08T21:03:30.557 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.575+0000 7f7a5b400640 1 -- 172.21.15.107:0/2801499111 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a54004cf0 con 0x7f7a64100240 2024-03-08T21:03:30.557 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.575+0000 7f7a68a00640 1 -- 172.21.15.107:0/2801499111 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a641a6a00 con 0x7f7a64100240 2024-03-08T21:03:30.559 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.578+0000 7f7a6c22a740 1 Processor -- start 2024-03-08T21:03:30.559 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.579+0000 7f7a6c22a740 1 -- start start 2024-03-08T21:03:30.560 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.579+0000 7f7a5b400640 1 -- 172.21.15.107:0/2801499111 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) v1 ==== 137422+0+0 (secure 0 0 0) 0x7f7a54004e90 con 0x7f7a64100240 2024-03-08T21:03:30.560 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.579+0000 7f7a6c22a740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ce710afe80 0x55ce710b0260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:03:30.561 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.579+0000 7f7a6c22a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ce710b07a0 0x55ce710b9030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:03:30.561 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.579+0000 7f7a6c22a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ce710b9570 0x55ce710bb960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:03:30.561 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.579+0000 7f7a6c22a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce710133d0 con 0x55ce710b9570 2024-03-08T21:03:30.561 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.579+0000 7f7a6c22a740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce70fbc6b0 con 0x55ce710afe80 2024-03-08T21:03:30.561 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.579+0000 7f7a6c22a740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce7100f470 con 0x55ce710b07a0 2024-03-08T21:03:30.561 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.579+0000 7f7a61e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ce710b9570 0x55ce710bb960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:03:30.561 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.579+0000 7f7a61e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ce710b9570 0x55ce710bb960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:38906/0 (socket says 172.21.15.107:38906) 2024-03-08T21:03:30.561 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.579+0000 7f7a61e00640 1 -- 172.21.15.107:0/2125369076 learned_addr learned my addr 172.21.15.107:0/2125369076 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:03:30.561 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.579+0000 7f7a62800640 1 --2- 172.21.15.107:0/2125369076 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ce710b07a0 0x55ce710b9030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:03:30.561 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.579+0000 7f7a63200640 1 --2- 172.21.15.107:0/2125369076 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ce710afe80 0x55ce710b0260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:03:30.561 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.579+0000 7f7a61e00640 1 -- 172.21.15.107:0/2125369076 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ce710b07a0 msgr2=0x55ce710b9030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:03:30.561 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.579+0000 7f7a61e00640 1 --2- 172.21.15.107:0/2125369076 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ce710b07a0 0x55ce710b9030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:03:30.561 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.579+0000 7f7a61e00640 1 -- 172.21.15.107:0/2125369076 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ce710afe80 msgr2=0x55ce710b0260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:03:30.562 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.579+0000 7f7a61e00640 1 --2- 172.21.15.107:0/2125369076 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ce710afe80 0x55ce710b0260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:03:30.562 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.579+0000 7f7a61e00640 1 -- 172.21.15.107:0/2125369076 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ce70fe2b50 con 0x55ce710b9570 2024-03-08T21:03:30.562 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.579+0000 7f7a61e00640 1 --2- 172.21.15.107:0/2125369076 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ce710b9570 0x55ce710bb960 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f7a50009a90 tx=0x7f7a50009f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e1c34f1e33fe6f6 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:03:30.566 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.579+0000 7f7a5b400640 1 --2- 172.21.15.107:0/2801499111 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f7a480a2a10 0x7f7a480a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:03:30.566 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.579+0000 7f7a4f400640 1 -- 172.21.15.107:0/2125369076 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a50016070 con 0x55ce710b9570 2024-03-08T21:03:30.566 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.580+0000 7f7a4f400640 1 -- 172.21.15.107:0/2125369076 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f7a500054d0 con 0x55ce710b9570 2024-03-08T21:03:30.566 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.580+0000 7f7a4f400640 1 -- 172.21.15.107:0/2125369076 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a50004dd0 con 0x55ce710b9570 2024-03-08T21:03:30.566 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.580+0000 7f7a5b400640 1 -- 172.21.15.107:0/2801499111 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f7a540ce8e0 con 0x7f7a64100240 2024-03-08T21:03:30.566 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.580+0000 7f7a63200640 1 --2- 172.21.15.107:0/2801499111 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f7a480a2a10 0x7f7a480a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:03:30.567 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.582+0000 7f7a6c22a740 1 -- 172.21.15.107:0/2125369076 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ce710b9570 msgr2=0x55ce710bb960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:03:30.567 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.582+0000 7f7a6c22a740 1 --2- 172.21.15.107:0/2125369076 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ce710b9570 0x55ce710bb960 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f7a50009a90 tx=0x7f7a50009f50 comp rx=0 tx=0).stop 2024-03-08T21:03:30.567 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.582+0000 7f7a63200640 1 --2- 172.21.15.107:0/2801499111 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f7a480a2a10 0x7f7a480a4ed0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f7a5c004af0 tx=0x7f7a5c004a40 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:03:30.567 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.582+0000 7f7a6c22a740 1 -- 172.21.15.107:0/2125369076 shutdown_connections 2024-03-08T21:03:30.567 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.582+0000 7f7a6c22a740 1 --2- 172.21.15.107:0/2125369076 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ce710b9570 0x55ce710bb960 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:03:30.567 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.582+0000 7f7a6c22a740 1 --2- 172.21.15.107:0/2125369076 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ce710b07a0 0x55ce710b9030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:03:30.567 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.582+0000 7f7a6c22a740 1 --2- 172.21.15.107:0/2125369076 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ce710afe80 0x55ce710b0260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:03:30.567 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.582+0000 7f7a6c22a740 1 -- 172.21.15.107:0/2125369076 >> 172.21.15.107:0/2125369076 conn(0x55ce710aa7a0 msgr2=0x55ce710aab80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:03:30.567 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.582+0000 7f7a6c22a740 1 -- 172.21.15.107:0/2125369076 shutdown_connections 2024-03-08T21:03:30.567 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.582+0000 7f7a6c22a740 1 -- 172.21.15.107:0/2125369076 wait complete. 2024-03-08T21:03:30.567 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.582+0000 7f7a6c22a740 1 -- start start 2024-03-08T21:03:30.567 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.583+0000 7f7a6c22a740 1 Processor -- start 2024-03-08T21:03:30.568 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.583+0000 7f7a6c22a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ce710afe80 0x55ce710b8ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:03:30.568 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.583+0000 7f7a6c22a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ce710b07a0 0x55ce710b6260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:03:30.568 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.583+0000 7f7a6c22a740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ce710b9570 0x55ce710b67c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:03:30.568 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.583+0000 7f7a6c22a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce710bc1a0 con 0x55ce710afe80 2024-03-08T21:03:30.568 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.583+0000 7f7a6c22a740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce70fcf9f0 con 0x55ce710b9570 2024-03-08T21:03:30.568 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.583+0000 7f7a6c22a740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce710bc320 con 0x55ce710b07a0 2024-03-08T21:03:30.568 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.583+0000 7f7a6c22a740 10 client.0 _ll_register_callbacks cb 0x55ce70d27a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T21:03:30.568 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.583+0000 7f7a62800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ce710afe80 0x55ce710b8ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:03:30.568 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.583+0000 7f7a62800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ce710afe80 0x55ce710b8ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:38916/0 (socket says 172.21.15.107:38916) 2024-03-08T21:03:30.568 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.583+0000 7f7a62800640 1 -- 172.21.15.107:0/4076539961 learned_addr learned my addr 172.21.15.107:0/4076539961 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:03:30.568 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.583+0000 7f7a61e00640 1 --2- 172.21.15.107:0/4076539961 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ce710b07a0 0x55ce710b6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:03:30.568 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.583+0000 7f7a63200640 1 --2- 172.21.15.107:0/4076539961 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ce710b9570 0x55ce710b67c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:03:30.569 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.583+0000 7f7a62800640 1 -- 172.21.15.107:0/4076539961 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ce710b07a0 msgr2=0x55ce710b6260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:03:30.569 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.583+0000 7f7a62800640 1 --2- 172.21.15.107:0/4076539961 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ce710b07a0 0x55ce710b6260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:03:30.569 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.583+0000 7f7a62800640 1 -- 172.21.15.107:0/4076539961 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ce710b9570 msgr2=0x55ce710b67c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:03:30.569 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.583+0000 7f7a62800640 1 --2- 172.21.15.107:0/4076539961 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ce710b9570 0x55ce710b67c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:03:30.569 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.583+0000 7f7a62800640 1 -- 172.21.15.107:0/4076539961 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ce70d28210 con 0x55ce710afe80 2024-03-08T21:03:30.569 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.583+0000 7f7a63200640 1 --2- 172.21.15.107:0/4076539961 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ce710b9570 0x55ce710b67c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T21:03:30.569 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.583+0000 7f7a62800640 1 --2- 172.21.15.107:0/4076539961 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ce710afe80 0x55ce710b8ed0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f7a5400d740 tx=0x7f7a540971d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c47025032b5e95da server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:03:30.569 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.583+0000 7f7a43e00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-03-08T21:03:30.569 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.585+0000 7f7a43e00640 1 -- 172.21.15.107:0/4076539961 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a54020070 con 0x55ce710afe80 2024-03-08T21:03:30.569 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.585+0000 7f7a43e00640 1 -- 172.21.15.107:0/4076539961 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f7a5409a380 con 0x55ce710afe80 2024-03-08T21:03:30.569 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.585+0000 7f7a6c22a740 1 -- 172.21.15.107:0/4076539961 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7a500054d0 con 0x55ce710afe80 2024-03-08T21:03:30.569 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.585+0000 7f7a43e00640 1 -- 172.21.15.107:0/4076539961 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7a5409a890 con 0x55ce710afe80 2024-03-08T21:03:30.570 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.585+0000 7f7a6c22a740 10 client.15708 fetch_fsmap learned FSMap version 32 2024-03-08T21:03:30.570 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.585+0000 7f7a6c22a740 1 -- 172.21.15.107:0/4076539961 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55ce70df4620 con 0x55ce710afe80 2024-03-08T21:03:30.570 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.585+0000 7f7a43e00640 1 -- 172.21.15.107:0/4076539961 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f7a500054d0 con 0x55ce710afe80 2024-03-08T21:03:30.570 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.585+0000 7f7a43e00640 1 -- 172.21.15.107:0/4076539961 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 32) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f7a54099df0 con 0x55ce710afe80 2024-03-08T21:03:30.570 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.585+0000 7f7a6c22a740 10 client.15708 fetch_fsmap finished waiting for FSMap version 32 2024-03-08T21:03:30.570 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.585+0000 7f7a6c22a740 10 client.15708 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-08T21:03:30.570 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.585+0000 7f7a6c22a740 20 client.15708 populate_metadata read hostname 'smithi107' 2024-03-08T21:03:30.570 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.585+0000 7f7a6c22a740 1 --2- 172.21.15.107:0/4076539961 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55ce710c9370 0x55ce710cb7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:03:30.570 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.585+0000 7f7a6c22a740 4 client.15708 mds_command: new command op to 34475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-08T21:03:30.570 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.585+0000 7f7a6c22a740 1 -- 172.21.15.107:0/4076539961 --> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ce71020910 con 0x55ce710c9370 2024-03-08T21:03:30.571 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.585+0000 7f7a61e00640 1 --2- 172.21.15.107:0/4076539961 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55ce710c9370 0x55ce710cb7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:03:30.572 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.587+0000 7f7a61e00640 1 --2- 172.21.15.107:0/4076539961 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55ce710c9370 0x55ce710cb7a0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=421f6de23251b3c0 server_cookie=de5be0f5cab05daf in_seq=0 out_seq=0 2024-03-08T21:03:30.572 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.587+0000 7f7a43e00640 10 client.15708 ms_handle_connect on v2:172.21.15.149:6838/55290596 2024-03-08T21:03:30.572 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.588+0000 7f7a43e00640 1 -- 172.21.15.107:0/4076539961 <== mds.0 v2:172.21.15.149:6838/55290596 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ce71020910 con 0x55ce710c9370 2024-03-08T21:03:30.572 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.588+0000 7f7a43e00640 10 client.15708 handle_command_reply: tid=0 2024-03-08T21:03:30.572 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.588+0000 7f7a6c22a740 1 client.15708 shutdown 2024-03-08T21:03:30.572 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.588+0000 7f7a6c22a740 1 -- 172.21.15.107:0/4076539961 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ce710afe80 msgr2=0x55ce710b8ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:03:30.572 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.588+0000 7f7a6c22a740 1 --2- 172.21.15.107:0/4076539961 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ce710afe80 0x55ce710b8ed0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f7a5400d740 tx=0x7f7a540971d0 comp rx=0 tx=0).stop 2024-03-08T21:03:30.572 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.588+0000 7f7a6c22a740 1 -- 172.21.15.107:0/4076539961 shutdown_connections 2024-03-08T21:03:30.572 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.588+0000 7f7a6c22a740 1 --2- 172.21.15.107:0/4076539961 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55ce710c9370 0x55ce710cb7a0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:03:30.572 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.588+0000 7f7a6c22a740 1 --2- 172.21.15.107:0/4076539961 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ce710b9570 0x55ce710b67c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:03:30.572 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.588+0000 7f7a6c22a740 1 --2- 172.21.15.107:0/4076539961 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ce710b07a0 0x55ce710b6260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:03:30.572 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.588+0000 7f7a6c22a740 1 --2- 172.21.15.107:0/4076539961 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ce710afe80 0x55ce710b8ed0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:03:30.573 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.588+0000 7f7a6c22a740 1 -- 172.21.15.107:0/4076539961 >> 172.21.15.107:0/4076539961 conn(0x55ce710aa7a0 msgr2=0x55ce710ac4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:03:30.573 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.588+0000 7f7a43e00640 0 client.15708 ms_handle_reset on v2:172.21.15.149:6838/55290596 2024-03-08T21:03:30.573 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.588+0000 7f7a6c22a740 1 -- 172.21.15.107:0/4076539961 shutdown_connections 2024-03-08T21:03:30.573 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.588+0000 7f7a6c22a740 1 -- 172.21.15.107:0/4076539961 wait complete. 2024-03-08T21:03:30.573 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.588+0000 7f7a6c22a740 20 client.15708 trim_cache size 0 max 16384 2024-03-08T21:03:30.588 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.607+0000 7f7a6c22a740 1 Processor -- start 2024-03-08T21:03:30.588 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.607+0000 7f7a6c22a740 1 -- start start 2024-03-08T21:03:30.589 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.608+0000 7f7a6c22a740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ce710afe80 0x55ce710b8990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:03:30.589 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.608+0000 7f7a6c22a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ce710b07a0 0x55ce710b8ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:03:30.589 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.608+0000 7f7a6c22a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ce710b9570 0x55ce710b5af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:03:30.589 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.608+0000 7f7a6c22a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a641aecd0 con 0x55ce710b07a0 2024-03-08T21:03:30.589 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.608+0000 7f7a6c22a740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a641b0650 con 0x55ce710afe80 2024-03-08T21:03:30.589 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.608+0000 7f7a6c22a740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a641af330 con 0x55ce710b9570 2024-03-08T21:03:30.589 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.608+0000 7f7a61e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ce710b07a0 0x55ce710b8ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:03:30.589 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.608+0000 7f7a61e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ce710b07a0 0x55ce710b8ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:38928/0 (socket says 172.21.15.107:38928) 2024-03-08T21:03:30.589 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.608+0000 7f7a61e00640 1 -- 172.21.15.107:0/2643065967 learned_addr learned my addr 172.21.15.107:0/2643065967 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:03:30.591 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.608+0000 7f7a62800640 1 --2- 172.21.15.107:0/2643065967 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ce710afe80 0x55ce710b8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:03:30.591 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.608+0000 7f7a63200640 1 --2- 172.21.15.107:0/2643065967 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ce710b9570 0x55ce710b5af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:03:30.591 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.609+0000 7f7a61e00640 1 -- 172.21.15.107:0/2643065967 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ce710b9570 msgr2=0x55ce710b5af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:03:30.591 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.609+0000 7f7a61e00640 1 --2- 172.21.15.107:0/2643065967 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ce710b9570 0x55ce710b5af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:03:30.591 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.609+0000 7f7a61e00640 1 -- 172.21.15.107:0/2643065967 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ce710afe80 msgr2=0x55ce710b8990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:03:30.591 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.609+0000 7f7a61e00640 1 --2- 172.21.15.107:0/2643065967 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ce710afe80 0x55ce710b8990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:03:30.591 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.609+0000 7f7a61e00640 1 -- 172.21.15.107:0/2643065967 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ce710b05d0 con 0x55ce710b07a0 2024-03-08T21:03:30.591 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.609+0000 7f7a61e00640 1 --2- 172.21.15.107:0/2643065967 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ce710b07a0 0x55ce710b8ed0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f7a50009a90 tx=0x7f7a50009370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3df46ca67c415e7a server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:03:30.591 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.609+0000 7f7a4ea00640 1 -- 172.21.15.107:0/2643065967 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a50016040 con 0x55ce710b07a0 2024-03-08T21:03:30.592 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.609+0000 7f7a4ea00640 1 -- 172.21.15.107:0/2643065967 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f7a50005020 con 0x55ce710b07a0 2024-03-08T21:03:30.592 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.609+0000 7f7a4ea00640 1 -- 172.21.15.107:0/2643065967 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a50004620 con 0x55ce710b07a0 2024-03-08T21:03:30.592 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.610+0000 7f7a6c22a740 1 -- 172.21.15.107:0/2643065967 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ce710b07a0 msgr2=0x55ce710b8ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:03:30.592 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.610+0000 7f7a6c22a740 1 --2- 172.21.15.107:0/2643065967 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ce710b07a0 0x55ce710b8ed0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f7a50009a90 tx=0x7f7a50009370 comp rx=0 tx=0).stop 2024-03-08T21:03:30.592 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.610+0000 7f7a6c22a740 1 -- 172.21.15.107:0/2643065967 shutdown_connections 2024-03-08T21:03:30.592 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.610+0000 7f7a6c22a740 1 --2- 172.21.15.107:0/2643065967 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ce710b9570 0x55ce710b5af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:03:30.592 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.610+0000 7f7a6c22a740 1 --2- 172.21.15.107:0/2643065967 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ce710b07a0 0x55ce710b8ed0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:03:30.592 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.610+0000 7f7a6c22a740 1 --2- 172.21.15.107:0/2643065967 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ce710afe80 0x55ce710b8990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:03:30.592 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.610+0000 7f7a6c22a740 1 -- 172.21.15.107:0/2643065967 >> 172.21.15.107:0/2643065967 conn(0x55ce710aa7a0 msgr2=0x55ce710d0110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:03:30.592 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.611+0000 7f7a6c22a740 1 -- 172.21.15.107:0/2643065967 shutdown_connections 2024-03-08T21:03:30.592 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.611+0000 7f7a6c22a740 1 -- 172.21.15.107:0/2643065967 wait complete. 2024-03-08T21:03:30.595 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.611+0000 7f7a6c22a740 1 -- start start 2024-03-08T21:03:30.595 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.611+0000 7f7a6c22a740 1 Processor -- start 2024-03-08T21:03:30.595 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.612+0000 7f7a6c22a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ce710afe80 0x55ce710c09a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:03:30.595 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.612+0000 7f7a6c22a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ce710b07a0 0x55ce710bed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:03:30.595 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.612+0000 7f7a6c22a740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ce710b9570 0x55ce710bf3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:03:30.595 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.612+0000 7f7a6c22a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a641afff0 con 0x55ce710afe80 2024-03-08T21:03:30.595 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.612+0000 7f7a6c22a740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a641af990 con 0x55ce710b9570 2024-03-08T21:03:30.596 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.612+0000 7f7a6c22a740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a641acf70 con 0x55ce710b07a0 2024-03-08T21:03:30.596 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.612+0000 7f7a6c22a740 10 client.0 _ll_register_callbacks cb 0x7f7a500092a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T21:03:30.596 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.612+0000 7f7a62800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ce710afe80 0x55ce710c09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:03:30.596 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.612+0000 7f7a62800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ce710afe80 0x55ce710c09a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:38940/0 (socket says 172.21.15.107:38940) 2024-03-08T21:03:30.596 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.612+0000 7f7a62800640 1 -- 172.21.15.107:0/2670787250 learned_addr learned my addr 172.21.15.107:0/2670787250 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:03:30.596 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.612+0000 7f7a61e00640 1 --2- 172.21.15.107:0/2670787250 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ce710b07a0 0x55ce710bed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:03:30.596 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.613+0000 7f7a63200640 1 --2- 172.21.15.107:0/2670787250 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ce710b9570 0x55ce710bf3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:03:30.596 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.613+0000 7f7a62800640 1 -- 172.21.15.107:0/2670787250 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ce710b07a0 msgr2=0x55ce710bed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:03:30.596 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.613+0000 7f7a62800640 1 --2- 172.21.15.107:0/2670787250 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ce710b07a0 0x55ce710bed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:03:30.596 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.613+0000 7f7a62800640 1 -- 172.21.15.107:0/2670787250 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ce710b9570 msgr2=0x55ce710bf3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:03:30.596 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.613+0000 7f7a62800640 1 --2- 172.21.15.107:0/2670787250 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ce710b9570 0x55ce710bf3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:03:30.596 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.613+0000 7f7a62800640 1 -- 172.21.15.107:0/2670787250 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ce710bfbc0 con 0x55ce710afe80 2024-03-08T21:03:30.596 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.613+0000 7f7a62800640 1 --2- 172.21.15.107:0/2670787250 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ce710afe80 0x55ce710c09a0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f7a5409a1e0 tx=0x7f7a54097850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2058cb5cc9bbc2b7 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:03:30.597 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.613+0000 7f7a43e00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-03-08T21:03:30.597 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.613+0000 7f7a43e00640 1 -- 172.21.15.107:0/2670787250 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a54020040 con 0x55ce710afe80 2024-03-08T21:03:30.597 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.613+0000 7f7a43e00640 1 -- 172.21.15.107:0/2670787250 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f7a54096c80 con 0x55ce710afe80 2024-03-08T21:03:30.597 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.613+0000 7f7a6c22a740 1 -- 172.21.15.107:0/2670787250 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7a50005020 con 0x55ce710afe80 2024-03-08T21:03:30.597 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.613+0000 7f7a43e00640 1 -- 172.21.15.107:0/2670787250 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7a5407d070 con 0x55ce710afe80 2024-03-08T21:03:30.597 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.613+0000 7f7a6c22a740 10 client.15714 fetch_fsmap learned FSMap version 32 2024-03-08T21:03:30.597 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.613+0000 7f7a6c22a740 1 -- 172.21.15.107:0/2670787250 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55ce711209e0 con 0x55ce710afe80 2024-03-08T21:03:30.597 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.614+0000 7f7a43e00640 1 -- 172.21.15.107:0/2670787250 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f7a50005020 con 0x55ce710afe80 2024-03-08T21:03:30.597 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.614+0000 7f7a43e00640 1 -- 172.21.15.107:0/2670787250 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 32) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f7a54096e80 con 0x55ce710afe80 2024-03-08T21:03:30.597 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.614+0000 7f7a6c22a740 10 client.15714 fetch_fsmap finished waiting for FSMap version 32 2024-03-08T21:03:30.597 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.614+0000 7f7a6c22a740 10 client.15714 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-08T21:03:30.597 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.614+0000 7f7a6c22a740 20 client.15714 populate_metadata read hostname 'smithi107' 2024-03-08T21:03:30.597 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.614+0000 7f7a6c22a740 1 --2- 172.21.15.107:0/2670787250 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55ce71121090 0x55ce711234b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:03:30.597 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.614+0000 7f7a6c22a740 4 client.15714 mds_command: new command op to 34475 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-08T21:03:30.598 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.614+0000 7f7a6c22a740 1 -- 172.21.15.107:0/2670787250 --> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55ce7101ac60 con 0x55ce71121090 2024-03-08T21:03:30.598 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.615+0000 7f7a61e00640 1 --2- 172.21.15.107:0/2670787250 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55ce71121090 0x55ce711234b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:03:30.598 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.615+0000 7f7a43e00640 10 client.15714 ms_handle_connect on v2:172.21.15.149:6838/55290596 2024-03-08T21:03:30.598 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.615+0000 7f7a61e00640 1 --2- 172.21.15.107:0/2670787250 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55ce71121090 0x55ce711234b0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=27bd5c94e31dae22 server_cookie=127b551ff4364fdc in_seq=0 out_seq=0 2024-03-08T21:03:30.606 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.625+0000 7f7a43e00640 1 -- 172.21.15.107:0/2670787250 <== mds.0 v2:172.21.15.149:6838/55290596 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55ce7101ac60 con 0x55ce71121090 2024-03-08T21:03:30.606 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.625+0000 7f7a43e00640 10 client.15714 handle_command_reply: tid=0 2024-03-08T21:03:30.607 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.625+0000 7f7a6c22a740 1 client.15714 shutdown 2024-03-08T21:03:30.607 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.626+0000 7f7a6c22a740 1 -- 172.21.15.107:0/2670787250 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ce710afe80 msgr2=0x55ce710c09a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:03:30.607 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.626+0000 7f7a6c22a740 1 --2- 172.21.15.107:0/2670787250 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ce710afe80 0x55ce710c09a0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f7a5409a1e0 tx=0x7f7a54097850 comp rx=0 tx=0).stop 2024-03-08T21:03:30.607 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.626+0000 7f7a6c22a740 1 -- 172.21.15.107:0/2670787250 shutdown_connections 2024-03-08T21:03:30.607 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.626+0000 7f7a6c22a740 1 --2- 172.21.15.107:0/2670787250 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55ce71121090 0x55ce711234b0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:03:30.607 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.626+0000 7f7a6c22a740 1 --2- 172.21.15.107:0/2670787250 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ce710b9570 0x55ce710bf3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:03:30.607 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.626+0000 7f7a6c22a740 1 --2- 172.21.15.107:0/2670787250 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ce710b07a0 0x55ce710bed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:03:30.607 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.626+0000 7f7a43e00640 0 client.15714 ms_handle_reset on v2:172.21.15.149:6838/55290596 2024-03-08T21:03:30.607 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.626+0000 7f7a6c22a740 1 --2- 172.21.15.107:0/2670787250 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ce710afe80 0x55ce710c09a0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:03:30.607 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.626+0000 7f7a6c22a740 1 -- 172.21.15.107:0/2670787250 >> 172.21.15.107:0/2670787250 conn(0x55ce710aa7a0 msgr2=0x55ce710b5ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:03:30.607 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.626+0000 7f7a6c22a740 1 -- 172.21.15.107:0/2670787250 shutdown_connections 2024-03-08T21:03:30.607 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.626+0000 7f7a6c22a740 1 -- 172.21.15.107:0/2670787250 wait complete. 2024-03-08T21:03:30.607 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.626+0000 7f7a6c22a740 20 client.15714 trim_cache size 0 max 16384 2024-03-08T21:03:30.608 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:{ 2024-03-08T21:03:30.608 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "message": "", 2024-03-08T21:03:30.608 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "return_code": 0 2024-03-08T21:03:30.608 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:} 2024-03-08T21:03:30.608 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.627+0000 7f7a68a00640 1 -- 172.21.15.107:0/2801499111 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f7a480a2a10 msgr2=0x7f7a480a4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:03:30.609 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.627+0000 7f7a68a00640 1 --2- 172.21.15.107:0/2801499111 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f7a480a2a10 0x7f7a480a4ed0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f7a5c004af0 tx=0x7f7a5c004a40 comp rx=0 tx=0).stop 2024-03-08T21:03:30.609 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.627+0000 7f7a68a00640 1 -- 172.21.15.107:0/2801499111 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7a64100240 msgr2=0x7f7a64101cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:03:30.609 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.627+0000 7f7a68a00640 1 --2- 172.21.15.107:0/2801499111 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7a64100240 0x7f7a64101cd0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f7a5400d770 tx=0x7f7a5400dc30 comp rx=0 tx=0).stop 2024-03-08T21:03:30.609 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.627+0000 7f7a68a00640 1 -- 172.21.15.107:0/2801499111 shutdown_connections 2024-03-08T21:03:30.609 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.627+0000 7f7a68a00640 1 --2- 172.21.15.107:0/2801499111 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f7a480a2a10 0x7f7a480a4ed0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:03:30.609 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.627+0000 7f7a68a00640 1 --2- 172.21.15.107:0/2801499111 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7a64102750 0x7f7a641a5ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:03:30.609 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.627+0000 7f7a68a00640 1 --2- 172.21.15.107:0/2801499111 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7a64105310 0x7f7a64102210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:03:30.609 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.627+0000 7f7a68a00640 1 --2- 172.21.15.107:0/2801499111 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7a64100240 0x7f7a64101cd0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:03:30.609 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.627+0000 7f7a68a00640 1 -- 172.21.15.107:0/2801499111 >> 172.21.15.107:0/2801499111 conn(0x7f7a640fbb00 msgr2=0x7f7a64068cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:03:30.609 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.627+0000 7f7a68a00640 1 -- 172.21.15.107:0/2801499111 shutdown_connections 2024-03-08T21:03:30.609 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:30.627+0000 7f7a68a00640 1 -- 172.21.15.107:0/2801499111 wait complete. 2024-03-08T21:03:31.876 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:03:31 smithi204 ceph-mon[34935]: pgmap v94: 97 pgs: 97 active+clean; 2.0 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-08T21:03:31.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:03:31 smithi107 ceph-mon[31381]: pgmap v94: 97 pgs: 97 active+clean; 2.0 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-08T21:03:32.013 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:03:31 smithi149 ceph-mon[35042]: pgmap v94: 97 pgs: 97 active+clean; 2.0 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-08T21:03:33.876 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:03:33 smithi204 ceph-mon[34935]: pgmap v95: 97 pgs: 97 active+clean; 2.0 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-08T21:03:33.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:03:33 smithi107 ceph-mon[31381]: pgmap v95: 97 pgs: 97 active+clean; 2.0 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-08T21:03:34.013 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:03:33 smithi149 ceph-mon[35042]: pgmap v95: 97 pgs: 97 active+clean; 2.0 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-08T21:03:35.876 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:03:35 smithi204 ceph-mon[34935]: pgmap v96: 97 pgs: 97 active+clean; 2.0 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-08T21:03:35.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:03:35 smithi107 ceph-mon[31381]: pgmap v96: 97 pgs: 97 active+clean; 2.0 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-08T21:03:36.013 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:03:35 smithi149 ceph-mon[35042]: pgmap v96: 97 pgs: 97 active+clean; 2.0 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-08T21:03:37.876 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:03:37 smithi204 ceph-mon[34935]: pgmap v97: 97 pgs: 97 active+clean; 2.0 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:03:37.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:03:37 smithi107 ceph-mon[31381]: pgmap v97: 97 pgs: 97 active+clean; 2.0 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:03:38.012 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:03:37 smithi149 ceph-mon[35042]: pgmap v97: 97 pgs: 97 active+clean; 2.0 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:03:39.876 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:03:39 smithi204 ceph-mon[34935]: pgmap v98: 97 pgs: 97 active+clean; 2.0 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:03:39.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:03:39 smithi107 ceph-mon[31381]: pgmap v98: 97 pgs: 97 active+clean; 2.0 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:03:40.012 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:03:39 smithi149 ceph-mon[35042]: pgmap v98: 97 pgs: 97 active+clean; 2.0 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:03:41.876 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:03:41 smithi204 ceph-mon[34935]: pgmap v99: 97 pgs: 97 active+clean; 2.0 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-08T21:03:41.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:03:41 smithi107 ceph-mon[31381]: pgmap v99: 97 pgs: 97 active+clean; 2.0 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-08T21:03:42.012 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:03:41 smithi149 ceph-mon[35042]: pgmap v99: 97 pgs: 97 active+clean; 2.0 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-08T21:03:43.755 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.774+0000 7fa1bcc00640 1 -- 172.21.15.107:0/2983989516 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa1b8102320 msgr2=0x7fa1b8102700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:03:43.755 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.774+0000 7fa1bcc00640 1 --2- 172.21.15.107:0/2983989516 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa1b8102320 0x7fa1b8102700 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fa1a8009900 tx=0x7fa1a802bcd0 comp rx=0 tx=0).stop 2024-03-08T21:03:43.755 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.774+0000 7fa1bcc00640 1 -- 172.21.15.107:0/2983989516 shutdown_connections 2024-03-08T21:03:43.755 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.774+0000 7fa1bcc00640 1 --2- 172.21.15.107:0/2983989516 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa1b810b6f0 0x7fa1b810db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:03:43.755 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.774+0000 7fa1bcc00640 1 --2- 172.21.15.107:0/2983989516 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa1b8102c40 0x7fa1b810b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:03:43.755 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.774+0000 7fa1bcc00640 1 --2- 172.21.15.107:0/2983989516 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa1b8102320 0x7fa1b8102700 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:03:43.756 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.775+0000 7fa1bcc00640 1 -- 172.21.15.107:0/2983989516 >> 172.21.15.107:0/2983989516 conn(0x7fa1b80fbb00 msgr2=0x7fa1b80fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:03:43.756 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.775+0000 7fa1bcc00640 1 -- 172.21.15.107:0/2983989516 shutdown_connections 2024-03-08T21:03:43.756 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.775+0000 7fa1bcc00640 1 -- 172.21.15.107:0/2983989516 wait complete. 2024-03-08T21:03:43.757 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.776+0000 7fa1bcc00640 1 Processor -- start 2024-03-08T21:03:43.757 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.776+0000 7fa1bcc00640 1 -- start start 2024-03-08T21:03:43.757 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.776+0000 7fa1bcc00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa1b8102320 0x7fa1b8199da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:03:43.757 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.776+0000 7fa1bcc00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa1b8102c40 0x7fa1b819a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:03:43.757 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.776+0000 7fa1bcc00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa1b810b6f0 0x7fa1b819e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:03:43.757 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.776+0000 7fa1bcc00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1b81007a0 con 0x7fa1b8102320 2024-03-08T21:03:43.757 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.776+0000 7fa1bcc00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1b8100620 con 0x7fa1b810b6f0 2024-03-08T21:03:43.757 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.776+0000 7fa1bcc00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1b8100920 con 0x7fa1b8102c40 2024-03-08T21:03:43.757 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.776+0000 7fa1b7400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa1b8102320 0x7fa1b8199da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:03:43.757 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.777+0000 7fa1b7400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa1b8102320 0x7fa1b8199da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:36824/0 (socket says 172.21.15.107:36824) 2024-03-08T21:03:43.758 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.777+0000 7fa1b7400640 1 -- 172.21.15.107:0/804327120 learned_addr learned my addr 172.21.15.107:0/804327120 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:03:43.758 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.777+0000 7fa1b7e00640 1 --2- 172.21.15.107:0/804327120 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa1b810b6f0 0x7fa1b819e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:03:43.758 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.777+0000 7fa1b6a00640 1 --2- 172.21.15.107:0/804327120 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa1b8102c40 0x7fa1b819a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:03:43.759 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.777+0000 7fa1b7400640 1 -- 172.21.15.107:0/804327120 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa1b8102c40 msgr2=0x7fa1b819a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:03:43.759 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.777+0000 7fa1b7400640 1 --2- 172.21.15.107:0/804327120 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa1b8102c40 0x7fa1b819a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:03:43.759 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.777+0000 7fa1b7400640 1 -- 172.21.15.107:0/804327120 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa1b810b6f0 msgr2=0x7fa1b819e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:03:43.759 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.777+0000 7fa1b7400640 1 --2- 172.21.15.107:0/804327120 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa1b810b6f0 0x7fa1b819e410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:03:43.759 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.777+0000 7fa1b7400640 1 -- 172.21.15.107:0/804327120 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1b819eb40 con 0x7fa1b8102320 2024-03-08T21:03:43.759 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.777+0000 7fa1b7400640 1 --2- 172.21.15.107:0/804327120 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa1b8102320 0x7fa1b8199da0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fa1a8000c00 tx=0x7fa1a802c6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:03:43.759 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.778+0000 7fa1afe00640 1 -- 172.21.15.107:0/804327120 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1a8043020 con 0x7fa1b8102320 2024-03-08T21:03:43.759 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.778+0000 7fa1afe00640 1 -- 172.21.15.107:0/804327120 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa1a802dbe0 con 0x7fa1b8102320 2024-03-08T21:03:43.759 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.778+0000 7fa1bcc00640 1 -- 172.21.15.107:0/804327120 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1b819edd0 con 0x7fa1b8102320 2024-03-08T21:03:43.759 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.778+0000 7fa1afe00640 1 -- 172.21.15.107:0/804327120 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1a8003970 con 0x7fa1b8102320 2024-03-08T21:03:43.759 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.778+0000 7fa1bcc00640 1 -- 172.21.15.107:0/804327120 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1b81a66c0 con 0x7fa1b8102320 2024-03-08T21:03:43.761 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.781+0000 7fa1c0e50740 1 Processor -- start 2024-03-08T21:03:43.762 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.782+0000 7fa1c0e50740 1 -- start start 2024-03-08T21:03:43.763 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.782+0000 7fa1c0e50740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5572bc7e1e80 0x5572bc7e2260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:03:43.763 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.782+0000 7fa1c0e50740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5572bc7e27a0 0x5572bc7eb030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:03:43.763 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.782+0000 7fa1c0e50740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5572bc7eb570 0x5572bc7ed960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:03:43.763 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.782+0000 7fa1c0e50740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572bc7453d0 con 0x5572bc7e27a0 2024-03-08T21:03:43.763 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.782+0000 7fa1c0e50740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572bc6ee6b0 con 0x5572bc7eb570 2024-03-08T21:03:43.763 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.782+0000 7fa1c0e50740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572bc741470 con 0x5572bc7e1e80 2024-03-08T21:03:43.763 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.782+0000 7fa1afe00640 1 -- 172.21.15.107:0/804327120 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) v1 ==== 137422+0+0 (secure 0 0 0) 0x7fa1a8046020 con 0x7fa1b8102320 2024-03-08T21:03:43.763 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.782+0000 7fa1b7e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5572bc7e1e80 0x5572bc7e2260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:03:43.763 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.783+0000 7fa1b7e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5572bc7e1e80 0x5572bc7e2260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.107:57028/0 (socket says 172.21.15.107:57028) 2024-03-08T21:03:43.763 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.783+0000 7fa1b6a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5572bc7eb570 0x5572bc7ed960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:03:43.763 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.783+0000 7fa1b7e00640 1 -- 172.21.15.107:0/607789617 learned_addr learned my addr 172.21.15.107:0/607789617 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:03:43.765 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.783+0000 7fa1b7400640 1 --2- 172.21.15.107:0/607789617 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5572bc7e27a0 0x5572bc7eb030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:03:43.765 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.783+0000 7fa1b7e00640 1 -- 172.21.15.107:0/607789617 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5572bc7eb570 msgr2=0x5572bc7ed960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:03:43.765 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.783+0000 7fa1b7e00640 1 --2- 172.21.15.107:0/607789617 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5572bc7eb570 0x5572bc7ed960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:03:43.765 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.783+0000 7fa1b7e00640 1 -- 172.21.15.107:0/607789617 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5572bc7e27a0 msgr2=0x5572bc7eb030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:03:43.765 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.783+0000 7fa1b7e00640 1 --2- 172.21.15.107:0/607789617 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5572bc7e27a0 0x5572bc7eb030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:03:43.765 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.783+0000 7fa1b7e00640 1 -- 172.21.15.107:0/607789617 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5572bc714b50 con 0x5572bc7e1e80 2024-03-08T21:03:43.765 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.783+0000 7fa1b6a00640 1 --2- 172.21.15.107:0/607789617 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5572bc7eb570 0x5572bc7ed960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T21:03:43.765 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.783+0000 7fa1b7400640 1 --2- 172.21.15.107:0/607789617 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5572bc7e27a0 0x5572bc7eb030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T21:03:43.765 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.784+0000 7fa1b7e00640 1 --2- 172.21.15.107:0/607789617 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5572bc7e1e80 0x5572bc7e2260 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fa1b0009f40 tx=0x7fa1b0010860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5aec4c4233557696 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:03:43.765 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.784+0000 7fa1afe00640 1 --2- 172.21.15.107:0/804327120 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7fa19c0a2960 0x7fa19c0a4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:03:43.765 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.784+0000 7fa1a3e00640 1 -- 172.21.15.107:0/607789617 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1b0003b70 con 0x5572bc7e1e80 2024-03-08T21:03:43.765 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.784+0000 7fa1a3e00640 1 -- 172.21.15.107:0/607789617 <== mon.1 v2:172.21.15.204:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa1b0003d10 con 0x5572bc7e1e80 2024-03-08T21:03:43.765 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.784+0000 7fa1a3e00640 1 -- 172.21.15.107:0/607789617 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1b00118e0 con 0x5572bc7e1e80 2024-03-08T21:03:43.765 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.784+0000 7fa1afe00640 1 -- 172.21.15.107:0/804327120 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fa1a80f0d90 con 0x7fa1b8102320 2024-03-08T21:03:43.765 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.784+0000 7fa1b7400640 1 --2- 172.21.15.107:0/804327120 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7fa19c0a2960 0x7fa19c0a4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:03:43.767 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.785+0000 7fa1c0e50740 1 -- 172.21.15.107:0/607789617 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5572bc7e1e80 msgr2=0x5572bc7e2260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:03:43.767 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.785+0000 7fa1c0e50740 1 --2- 172.21.15.107:0/607789617 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5572bc7e1e80 0x5572bc7e2260 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fa1b0009f40 tx=0x7fa1b0010860 comp rx=0 tx=0).stop 2024-03-08T21:03:43.767 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.785+0000 7fa1b7400640 1 --2- 172.21.15.107:0/804327120 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7fa19c0a2960 0x7fa19c0a4e20 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fa1a80ee690 tx=0x7fa1a80f48e0 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:03:43.767 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.785+0000 7fa1c0e50740 1 -- 172.21.15.107:0/607789617 shutdown_connections 2024-03-08T21:03:43.767 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.785+0000 7fa1c0e50740 1 --2- 172.21.15.107:0/607789617 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5572bc7eb570 0x5572bc7ed960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:03:43.767 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.785+0000 7fa1c0e50740 1 --2- 172.21.15.107:0/607789617 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5572bc7e27a0 0x5572bc7eb030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:03:43.767 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.785+0000 7fa1c0e50740 1 --2- 172.21.15.107:0/607789617 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5572bc7e1e80 0x5572bc7e2260 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:03:43.767 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.785+0000 7fa1c0e50740 1 -- 172.21.15.107:0/607789617 >> 172.21.15.107:0/607789617 conn(0x5572bc7dc7a0 msgr2=0x5572bc7dcb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:03:43.767 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.786+0000 7fa1c0e50740 1 -- 172.21.15.107:0/607789617 shutdown_connections 2024-03-08T21:03:43.767 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.786+0000 7fa1c0e50740 1 -- 172.21.15.107:0/607789617 wait complete. 2024-03-08T21:03:43.767 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.786+0000 7fa1c0e50740 1 -- start start 2024-03-08T21:03:43.767 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.786+0000 7fa1c0e50740 1 Processor -- start 2024-03-08T21:03:43.770 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.787+0000 7fa1c0e50740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5572bc7e1e80 0x5572bc7eaf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:03:43.770 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.787+0000 7fa1c0e50740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5572bc7e27a0 0x5572bc7e8310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:03:43.770 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.787+0000 7fa1c0e50740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5572bc7eb570 0x5572bc7e8920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:03:43.771 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.787+0000 7fa1c0e50740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572bc7ee1a0 con 0x5572bc7eb570 2024-03-08T21:03:43.771 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.787+0000 7fa1c0e50740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572bc7019f0 con 0x5572bc7e1e80 2024-03-08T21:03:43.771 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.787+0000 7fa1c0e50740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572bc7ee320 con 0x5572bc7e27a0 2024-03-08T21:03:43.771 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.787+0000 7fa1c0e50740 10 client.0 _ll_register_callbacks cb 0x5572bc459a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T21:03:43.771 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.787+0000 7fa1b7e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5572bc7eb570 0x5572bc7e8920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:03:43.771 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.787+0000 7fa1b7e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5572bc7eb570 0x5572bc7e8920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:36836/0 (socket says 172.21.15.107:36836) 2024-03-08T21:03:43.771 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.787+0000 7fa1b7e00640 1 -- 172.21.15.107:0/2105150259 learned_addr learned my addr 172.21.15.107:0/2105150259 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:03:43.771 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.788+0000 7fa1b7e00640 1 --2- 172.21.15.107:0/2105150259 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5572bc7e1e80 0x5572bc7eaf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:03:43.771 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.788+0000 7fa1b6a00640 1 --2- 172.21.15.107:0/2105150259 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5572bc7e27a0 0x5572bc7e8310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:03:43.771 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.788+0000 7fa1b7e00640 1 -- 172.21.15.107:0/2105150259 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5572bc7e27a0 msgr2=0x5572bc7e8310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:03:43.771 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.788+0000 7fa1b7e00640 1 --2- 172.21.15.107:0/2105150259 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5572bc7e27a0 0x5572bc7e8310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:03:43.771 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.788+0000 7fa1b7e00640 1 -- 172.21.15.107:0/2105150259 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5572bc7e1e80 msgr2=0x5572bc7eaf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:03:43.771 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.788+0000 7fa1b7e00640 1 --2- 172.21.15.107:0/2105150259 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5572bc7e1e80 0x5572bc7eaf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:03:43.771 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.788+0000 7fa1b7e00640 1 -- 172.21.15.107:0/2105150259 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5572bc45a210 con 0x5572bc7eb570 2024-03-08T21:03:43.772 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.788+0000 7fa1b7e00640 1 --2- 172.21.15.107:0/2105150259 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5572bc7eb570 0x5572bc7e8920 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fa1b0004870 tx=0x7fa1b0011500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf2209892abaef7e server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:03:43.772 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.788+0000 7fa1a0c00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-03-08T21:03:43.772 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.788+0000 7fa1a0c00640 1 -- 172.21.15.107:0/2105150259 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1b0020060 con 0x5572bc7eb570 2024-03-08T21:03:43.772 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.789+0000 7fa1a0c00640 1 -- 172.21.15.107:0/2105150259 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fa1b0004550 con 0x5572bc7eb570 2024-03-08T21:03:43.772 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.789+0000 7fa1c0e50740 1 -- 172.21.15.107:0/2105150259 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa1b0003d10 con 0x5572bc7eb570 2024-03-08T21:03:43.772 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.789+0000 7fa1a0c00640 1 -- 172.21.15.107:0/2105150259 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa1b0004a40 con 0x5572bc7eb570 2024-03-08T21:03:43.772 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.789+0000 7fa1c0e50740 10 client.15726 fetch_fsmap learned FSMap version 32 2024-03-08T21:03:43.772 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.789+0000 7fa1c0e50740 1 -- 172.21.15.107:0/2105150259 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x5572bc526620 con 0x5572bc7eb570 2024-03-08T21:03:43.772 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.789+0000 7fa1a0c00640 1 -- 172.21.15.107:0/2105150259 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa1b0003d10 con 0x5572bc7eb570 2024-03-08T21:03:43.772 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.789+0000 7fa1a0c00640 1 -- 172.21.15.107:0/2105150259 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 32) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fa1b0005030 con 0x5572bc7eb570 2024-03-08T21:03:43.772 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.790+0000 7fa1c0e50740 10 client.15726 fetch_fsmap finished waiting for FSMap version 32 2024-03-08T21:03:43.772 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.790+0000 7fa1c0e50740 10 client.15726 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-08T21:03:43.772 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.790+0000 7fa1c0e50740 20 client.15726 populate_metadata read hostname 'smithi107' 2024-03-08T21:03:43.772 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.790+0000 7fa1c0e50740 1 --2- 172.21.15.107:0/2105150259 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x5572bc7fb3d0 0x5572bc7fd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:03:43.773 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.790+0000 7fa1c0e50740 4 client.15726 mds_command: new command op to 34475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-08T21:03:43.773 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.790+0000 7fa1c0e50740 1 -- 172.21.15.107:0/2105150259 --> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5572bc752910 con 0x5572bc7fb3d0 2024-03-08T21:03:43.773 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.790+0000 7fa1b6a00640 1 --2- 172.21.15.107:0/2105150259 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x5572bc7fb3d0 0x5572bc7fd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:03:43.774 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.791+0000 7fa1b6a00640 1 --2- 172.21.15.107:0/2105150259 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x5572bc7fb3d0 0x5572bc7fd7c0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=254fd080a0c4b037 server_cookie=1fef83accb9df96b in_seq=0 out_seq=0 2024-03-08T21:03:43.774 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.791+0000 7fa1a0c00640 10 client.15726 ms_handle_connect on v2:172.21.15.149:6838/55290596 2024-03-08T21:03:43.774 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.793+0000 7fa1a0c00640 1 -- 172.21.15.107:0/2105150259 <== mds.0 v2:172.21.15.149:6838/55290596 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5572bc752910 con 0x5572bc7fb3d0 2024-03-08T21:03:43.774 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.793+0000 7fa1a0c00640 10 client.15726 handle_command_reply: tid=0 2024-03-08T21:03:43.774 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.793+0000 7fa1c0e50740 1 client.15726 shutdown 2024-03-08T21:03:43.774 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.793+0000 7fa1c0e50740 1 -- 172.21.15.107:0/2105150259 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5572bc7eb570 msgr2=0x5572bc7e8920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:03:43.775 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.793+0000 7fa1c0e50740 1 --2- 172.21.15.107:0/2105150259 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5572bc7eb570 0x5572bc7e8920 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fa1b0004870 tx=0x7fa1b0011500 comp rx=0 tx=0).stop 2024-03-08T21:03:43.775 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.793+0000 7fa1c0e50740 1 -- 172.21.15.107:0/2105150259 shutdown_connections 2024-03-08T21:03:43.775 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.793+0000 7fa1c0e50740 1 --2- 172.21.15.107:0/2105150259 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5572bc7eb570 0x5572bc7e8920 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:03:43.775 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.793+0000 7fa1c0e50740 1 --2- 172.21.15.107:0/2105150259 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5572bc7e27a0 0x5572bc7e8310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:03:43.775 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.793+0000 7fa1c0e50740 1 --2- 172.21.15.107:0/2105150259 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x5572bc7fb3d0 0x5572bc7fd7c0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:03:43.775 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.793+0000 7fa1c0e50740 1 --2- 172.21.15.107:0/2105150259 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5572bc7e1e80 0x5572bc7eaf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:03:43.775 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.793+0000 7fa1c0e50740 1 -- 172.21.15.107:0/2105150259 >> 172.21.15.107:0/2105150259 conn(0x5572bc7dc7a0 msgr2=0x5572bc7de4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:03:43.775 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.793+0000 7fa1a0c00640 0 client.15726 ms_handle_reset on v2:172.21.15.149:6838/55290596 2024-03-08T21:03:43.775 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.793+0000 7fa1c0e50740 1 -- 172.21.15.107:0/2105150259 shutdown_connections 2024-03-08T21:03:43.775 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.793+0000 7fa1c0e50740 1 -- 172.21.15.107:0/2105150259 wait complete. 2024-03-08T21:03:43.775 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.793+0000 7fa1c0e50740 20 client.15726 trim_cache size 0 max 16384 2024-03-08T21:03:43.788 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.808+0000 7fa1c0e50740 1 Processor -- start 2024-03-08T21:03:43.788 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.808+0000 7fa1c0e50740 1 -- start start 2024-03-08T21:03:43.788 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.808+0000 7fa1c0e50740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5572bc7e1e80 0x5572bc7e7330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:03:43.788 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.808+0000 7fa1c0e50740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5572bc7e27a0 0x5572bc7eaa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:03:43.789 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.808+0000 7fa1c0e50740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5572bc7eb570 0x5572bc7eaf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:03:43.789 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.808+0000 7fa1c0e50740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1b81aed80 con 0x5572bc7e27a0 2024-03-08T21:03:43.789 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.808+0000 7fa1c0e50740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1b81b0700 con 0x5572bc7eb570 2024-03-08T21:03:43.789 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.808+0000 7fa1c0e50740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1b81af3e0 con 0x5572bc7e1e80 2024-03-08T21:03:43.790 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.808+0000 7fa1b6a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5572bc7e27a0 0x5572bc7eaa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:03:43.790 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.809+0000 7fa1b6a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5572bc7e27a0 0x5572bc7eaa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:36848/0 (socket says 172.21.15.107:36848) 2024-03-08T21:03:43.790 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.809+0000 7fa1b6a00640 1 -- 172.21.15.107:0/401623938 learned_addr learned my addr 172.21.15.107:0/401623938 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:03:43.790 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.809+0000 7fa1b7e00640 1 --2- 172.21.15.107:0/401623938 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5572bc7e1e80 0x5572bc7e7330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:03:43.790 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.809+0000 7fa1b6a00640 1 -- 172.21.15.107:0/401623938 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5572bc7e1e80 msgr2=0x5572bc7e7330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:03:43.790 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.809+0000 7fa1b6a00640 1 --2- 172.21.15.107:0/401623938 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5572bc7e1e80 0x5572bc7e7330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:03:43.790 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.809+0000 7fa1b7e00640 1 --2- 172.21.15.107:0/401623938 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5572bc7eb570 0x5572bc7eaf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:03:43.791 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.809+0000 7fa1b6a00640 1 -- 172.21.15.107:0/401623938 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5572bc7eb570 msgr2=0x5572bc7eaf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:03:43.791 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.809+0000 7fa1b6a00640 1 --2- 172.21.15.107:0/401623938 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5572bc7eb570 0x5572bc7eaf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:03:43.791 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.809+0000 7fa1b6a00640 1 -- 172.21.15.107:0/401623938 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5572bc7dbff0 con 0x5572bc7e27a0 2024-03-08T21:03:43.791 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.809+0000 7fa1b6a00640 1 --2- 172.21.15.107:0/401623938 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5572bc7e27a0 0x5572bc7eaa40 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x5572bc7dd930 tx=0x7fa1a4005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b31da146f01ab871 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:03:43.791 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.810+0000 7fa1a3400640 1 -- 172.21.15.107:0/401623938 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1a4004230 con 0x5572bc7e27a0 2024-03-08T21:03:43.791 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.810+0000 7fa1a3400640 1 -- 172.21.15.107:0/401623938 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa1a4023030 con 0x5572bc7e27a0 2024-03-08T21:03:43.791 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.810+0000 7fa1a3400640 1 -- 172.21.15.107:0/401623938 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1a4004cd0 con 0x5572bc7e27a0 2024-03-08T21:03:43.795 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.810+0000 7fa1c0e50740 1 -- 172.21.15.107:0/401623938 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5572bc7e27a0 msgr2=0x5572bc7eaa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:03:43.795 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.810+0000 7fa1c0e50740 1 --2- 172.21.15.107:0/401623938 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5572bc7e27a0 0x5572bc7eaa40 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x5572bc7dd930 tx=0x7fa1a4005f90 comp rx=0 tx=0).stop 2024-03-08T21:03:43.795 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.811+0000 7fa1c0e50740 1 -- 172.21.15.107:0/401623938 shutdown_connections 2024-03-08T21:03:43.795 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.811+0000 7fa1c0e50740 1 --2- 172.21.15.107:0/401623938 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5572bc7eb570 0x5572bc7eaf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:03:43.795 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.811+0000 7fa1c0e50740 1 --2- 172.21.15.107:0/401623938 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5572bc7e27a0 0x5572bc7eaa40 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:03:43.795 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.811+0000 7fa1c0e50740 1 --2- 172.21.15.107:0/401623938 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5572bc7e1e80 0x5572bc7e7330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:03:43.795 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.811+0000 7fa1c0e50740 1 -- 172.21.15.107:0/401623938 >> 172.21.15.107:0/401623938 conn(0x5572bc7dc7a0 msgr2=0x5572bc7e6c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:03:43.795 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.811+0000 7fa1c0e50740 1 -- 172.21.15.107:0/401623938 shutdown_connections 2024-03-08T21:03:43.795 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.811+0000 7fa1c0e50740 1 -- 172.21.15.107:0/401623938 wait complete. 2024-03-08T21:03:43.795 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.811+0000 7fa1c0e50740 1 -- start start 2024-03-08T21:03:43.795 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.811+0000 7fa1c0e50740 1 Processor -- start 2024-03-08T21:03:43.795 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.812+0000 7fa1c0e50740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5572bc7e1e80 0x5572bc849960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:03:43.795 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.812+0000 7fa1c0e50740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5572bc7e27a0 0x5572bc849ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:03:43.796 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.812+0000 7fa1c0e50740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5572bc7eb570 0x5572bc7f1f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:03:43.796 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.812+0000 7fa1c0e50740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1b81b00a0 con 0x5572bc7e1e80 2024-03-08T21:03:43.796 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.812+0000 7fa1c0e50740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1b81afa40 con 0x5572bc7e27a0 2024-03-08T21:03:43.796 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.812+0000 7fa1c0e50740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1b81ad020 con 0x5572bc7eb570 2024-03-08T21:03:43.796 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.812+0000 7fa1c0e50740 10 client.0 _ll_register_callbacks cb 0x7fa1b00099e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T21:03:43.796 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.812+0000 7fa1b7e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5572bc7e1e80 0x5572bc849960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:03:43.796 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.812+0000 7fa1b7e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5572bc7e1e80 0x5572bc849960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:36852/0 (socket says 172.21.15.107:36852) 2024-03-08T21:03:43.796 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.812+0000 7fa1b7e00640 1 -- 172.21.15.107:0/3211032528 learned_addr learned my addr 172.21.15.107:0/3211032528 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:03:43.796 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.812+0000 7fa1b6a00640 1 --2- 172.21.15.107:0/3211032528 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5572bc7e27a0 0x5572bc849ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:03:43.796 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.812+0000 7fa1b7e00640 1 -- 172.21.15.107:0/3211032528 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5572bc7eb570 msgr2=0x5572bc7f1f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:03:43.796 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.812+0000 7fa1b7e00640 1 --2- 172.21.15.107:0/3211032528 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5572bc7eb570 0x5572bc7f1f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:03:43.796 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.812+0000 7fa1b7e00640 1 -- 172.21.15.107:0/3211032528 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5572bc7e27a0 msgr2=0x5572bc849ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:03:43.796 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.812+0000 7fa1b7e00640 1 --2- 172.21.15.107:0/3211032528 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5572bc7e27a0 0x5572bc849ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:03:43.796 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.813+0000 7fa1b7e00640 1 -- 172.21.15.107:0/3211032528 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5572bc7f2640 con 0x5572bc7e1e80 2024-03-08T21:03:43.796 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.813+0000 7fa1b6a00640 1 --2- 172.21.15.107:0/3211032528 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5572bc7e27a0 0x5572bc849ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T21:03:43.797 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.813+0000 7fa1b7e00640 1 --2- 172.21.15.107:0/3211032528 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5572bc7e1e80 0x5572bc849960 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fa1b0005c40 tx=0x7fa1b0005320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec6fd41a91e0a3d2 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:03:43.797 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.813+0000 7fa1a0c00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-03-08T21:03:43.797 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.813+0000 7fa1a0c00640 1 -- 172.21.15.107:0/3211032528 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1b0038060 con 0x5572bc7e1e80 2024-03-08T21:03:43.797 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.813+0000 7fa1c0e50740 1 -- 172.21.15.107:0/3211032528 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa1a4023030 con 0x5572bc7e1e80 2024-03-08T21:03:43.797 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.813+0000 7fa1a0c00640 1 -- 172.21.15.107:0/3211032528 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fa1b0024040 con 0x5572bc7e1e80 2024-03-08T21:03:43.797 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.813+0000 7fa1a0c00640 1 -- 172.21.15.107:0/3211032528 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa1b0020040 con 0x5572bc7e1e80 2024-03-08T21:03:43.797 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.813+0000 7fa1c0e50740 10 client.15732 fetch_fsmap learned FSMap version 32 2024-03-08T21:03:43.797 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.813+0000 7fa1c0e50740 1 -- 172.21.15.107:0/3211032528 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x5572bc8521c0 con 0x5572bc7e1e80 2024-03-08T21:03:43.797 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.814+0000 7fa1a0c00640 1 -- 172.21.15.107:0/3211032528 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa1a4023030 con 0x5572bc7e1e80 2024-03-08T21:03:43.797 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.814+0000 7fa1a0c00640 1 -- 172.21.15.107:0/3211032528 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 32) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fa1b0011a20 con 0x5572bc7e1e80 2024-03-08T21:03:43.797 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.814+0000 7fa1c0e50740 10 client.15732 fetch_fsmap finished waiting for FSMap version 32 2024-03-08T21:03:43.797 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.814+0000 7fa1c0e50740 10 client.15732 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-08T21:03:43.797 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.814+0000 7fa1c0e50740 20 client.15732 populate_metadata read hostname 'smithi107' 2024-03-08T21:03:43.797 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.814+0000 7fa1c0e50740 1 --2- 172.21.15.107:0/3211032528 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x5572bc8527d0 0x5572bc854bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:03:43.798 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.814+0000 7fa1c0e50740 4 client.15732 mds_command: new command op to 34475 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-08T21:03:43.798 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.814+0000 7fa1c0e50740 1 -- 172.21.15.107:0/3211032528 --> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5572bc74cc60 con 0x5572bc8527d0 2024-03-08T21:03:43.798 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.815+0000 7fa1b6a00640 1 --2- 172.21.15.107:0/3211032528 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x5572bc8527d0 0x5572bc854bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:03:43.798 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.816+0000 7fa1b6a00640 1 --2- 172.21.15.107:0/3211032528 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x5572bc8527d0 0x5572bc854bc0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f0705bc0ff72118e server_cookie=7a72661bef5462f in_seq=0 out_seq=0 2024-03-08T21:03:43.798 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.816+0000 7fa1a0c00640 10 client.15732 ms_handle_connect on v2:172.21.15.149:6838/55290596 2024-03-08T21:03:43.805 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.824+0000 7fa1a0c00640 1 -- 172.21.15.107:0/3211032528 <== mds.0 v2:172.21.15.149:6838/55290596 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5572bc74cc60 con 0x5572bc8527d0 2024-03-08T21:03:43.806 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.824+0000 7fa1a0c00640 10 client.15732 handle_command_reply: tid=0 2024-03-08T21:03:43.806 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.824+0000 7fa1c0e50740 1 client.15732 shutdown 2024-03-08T21:03:43.806 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.825+0000 7fa1c0e50740 1 -- 172.21.15.107:0/3211032528 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5572bc7e1e80 msgr2=0x5572bc849960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:03:43.806 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.825+0000 7fa1c0e50740 1 --2- 172.21.15.107:0/3211032528 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5572bc7e1e80 0x5572bc849960 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fa1b0005c40 tx=0x7fa1b0005320 comp rx=0 tx=0).stop 2024-03-08T21:03:43.806 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.825+0000 7fa1c0e50740 1 -- 172.21.15.107:0/3211032528 shutdown_connections 2024-03-08T21:03:43.806 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.825+0000 7fa1c0e50740 1 --2- 172.21.15.107:0/3211032528 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5572bc7eb570 0x5572bc7f1f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:03:43.806 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.825+0000 7fa1c0e50740 1 --2- 172.21.15.107:0/3211032528 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x5572bc8527d0 0x5572bc854bc0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:03:43.806 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.825+0000 7fa1a0c00640 0 client.15732 ms_handle_reset on v2:172.21.15.149:6838/55290596 2024-03-08T21:03:43.806 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.825+0000 7fa1c0e50740 1 --2- 172.21.15.107:0/3211032528 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5572bc7e27a0 0x5572bc849ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:03:43.806 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.825+0000 7fa1c0e50740 1 --2- 172.21.15.107:0/3211032528 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5572bc7e1e80 0x5572bc849960 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:03:43.806 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.825+0000 7fa1c0e50740 1 -- 172.21.15.107:0/3211032528 >> 172.21.15.107:0/3211032528 conn(0x5572bc7dc7a0 msgr2=0x5572bc7daf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:03:43.806 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.825+0000 7fa1c0e50740 1 -- 172.21.15.107:0/3211032528 shutdown_connections 2024-03-08T21:03:43.806 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.825+0000 7fa1c0e50740 1 -- 172.21.15.107:0/3211032528 wait complete. 2024-03-08T21:03:43.807 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.825+0000 7fa1c0e50740 20 client.15732 trim_cache size 0 max 16384 2024-03-08T21:03:43.807 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:{ 2024-03-08T21:03:43.807 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "message": "", 2024-03-08T21:03:43.807 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "return_code": 0 2024-03-08T21:03:43.807 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:} 2024-03-08T21:03:43.808 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.826+0000 7fa1bcc00640 1 -- 172.21.15.107:0/804327120 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7fa19c0a2960 msgr2=0x7fa19c0a4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:03:43.809 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.826+0000 7fa1bcc00640 1 --2- 172.21.15.107:0/804327120 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7fa19c0a2960 0x7fa19c0a4e20 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fa1a80ee690 tx=0x7fa1a80f48e0 comp rx=0 tx=0).stop 2024-03-08T21:03:43.809 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.826+0000 7fa1bcc00640 1 -- 172.21.15.107:0/804327120 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa1b8102320 msgr2=0x7fa1b8199da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:03:43.809 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.826+0000 7fa1bcc00640 1 --2- 172.21.15.107:0/804327120 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa1b8102320 0x7fa1b8199da0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fa1a8000c00 tx=0x7fa1a802c6d0 comp rx=0 tx=0).stop 2024-03-08T21:03:43.809 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.826+0000 7fa1bcc00640 1 -- 172.21.15.107:0/804327120 shutdown_connections 2024-03-08T21:03:43.809 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.826+0000 7fa1bcc00640 1 --2- 172.21.15.107:0/804327120 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7fa19c0a2960 0x7fa19c0a4e20 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:03:43.809 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.826+0000 7fa1bcc00640 1 --2- 172.21.15.107:0/804327120 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa1b810b6f0 0x7fa1b819e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:03:43.809 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.826+0000 7fa1bcc00640 1 --2- 172.21.15.107:0/804327120 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa1b8102c40 0x7fa1b819a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:03:43.809 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.826+0000 7fa1bcc00640 1 --2- 172.21.15.107:0/804327120 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa1b8102320 0x7fa1b8199da0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:03:43.810 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.826+0000 7fa1bcc00640 1 -- 172.21.15.107:0/804327120 >> 172.21.15.107:0/804327120 conn(0x7fa1b80fbb00 msgr2=0x7fa1b80fdef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:03:43.810 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.826+0000 7fa1bcc00640 1 -- 172.21.15.107:0/804327120 shutdown_connections 2024-03-08T21:03:43.810 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:43.827+0000 7fa1bcc00640 1 -- 172.21.15.107:0/804327120 wait complete. 2024-03-08T21:03:43.876 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:03:43 smithi204 ceph-mon[34935]: pgmap v100: 97 pgs: 97 active+clean; 2.0 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:03:43.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:03:43 smithi107 ceph-mon[31381]: pgmap v100: 97 pgs: 97 active+clean; 2.0 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:03:44.012 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:03:43 smithi149 ceph-mon[35042]: pgmap v100: 97 pgs: 97 active+clean; 2.0 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:03:45.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:03:45 smithi107 ceph-mon[31381]: pgmap v101: 97 pgs: 97 active+clean; 2.0 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-08T21:03:45.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:03:45 smithi107 ceph-mon[31381]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T21:03:46.012 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:03:45 smithi149 ceph-mon[35042]: pgmap v101: 97 pgs: 97 active+clean; 2.0 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-08T21:03:46.012 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:03:45 smithi149 ceph-mon[35042]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T21:03:46.126 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:03:45 smithi204 ceph-mon[34935]: pgmap v101: 97 pgs: 97 active+clean; 2.0 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-08T21:03:46.126 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:03:45 smithi204 ceph-mon[34935]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T21:03:47.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:03:47 smithi107 ceph-mon[31381]: pgmap v102: 97 pgs: 97 active+clean; 2.0 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-08T21:03:47.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:03:47 smithi107 ceph-mon[31381]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T21:03:47.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:03:47 smithi107 ceph-mon[31381]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T21:03:47.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:03:47 smithi107 ceph-mon[31381]: from='mgr.15375 ' entity='mgr.x' 2024-03-08T21:03:48.013 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:03:47 smithi149 ceph-mon[35042]: pgmap v102: 97 pgs: 97 active+clean; 2.0 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-08T21:03:48.013 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:03:47 smithi149 ceph-mon[35042]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T21:03:48.013 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:03:47 smithi149 ceph-mon[35042]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T21:03:48.013 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:03:47 smithi149 ceph-mon[35042]: from='mgr.15375 ' entity='mgr.x' 2024-03-08T21:03:48.126 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:03:47 smithi204 ceph-mon[34935]: pgmap v102: 97 pgs: 97 active+clean; 2.0 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-08T21:03:48.126 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:03:47 smithi204 ceph-mon[34935]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T21:03:48.126 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:03:47 smithi204 ceph-mon[34935]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T21:03:48.126 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:03:47 smithi204 ceph-mon[34935]: from='mgr.15375 ' entity='mgr.x' 2024-03-08T21:03:48.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:03:48 smithi107 ceph-mon[31381]: pgmap v103: 97 pgs: 97 active+clean; 2.0 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-08T21:03:49.012 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:03:48 smithi149 ceph-mon[35042]: pgmap v103: 97 pgs: 97 active+clean; 2.0 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-08T21:03:49.126 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:03:48 smithi204 ceph-mon[34935]: pgmap v103: 97 pgs: 97 active+clean; 2.0 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-08T21:03:51.876 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:03:51 smithi204 ceph-mon[34935]: pgmap v104: 97 pgs: 97 active+clean; 2.0 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-08T21:03:51.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:03:51 smithi107 ceph-mon[31381]: pgmap v104: 97 pgs: 97 active+clean; 2.0 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-08T21:03:52.012 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:03:51 smithi149 ceph-mon[35042]: pgmap v104: 97 pgs: 97 active+clean; 2.0 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-08T21:03:53.876 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:03:53 smithi204 ceph-mon[34935]: pgmap v105: 97 pgs: 97 active+clean; 2.0 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:03:53.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:03:53 smithi107 ceph-mon[31381]: pgmap v105: 97 pgs: 97 active+clean; 2.0 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:03:54.012 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:03:53 smithi149 ceph-mon[35042]: pgmap v105: 97 pgs: 97 active+clean; 2.0 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:03:55.876 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:03:55 smithi204 ceph-mon[34935]: pgmap v106: 97 pgs: 97 active+clean; 2.0 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-08T21:03:55.964 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:03:55 smithi107 ceph-mon[31381]: pgmap v106: 97 pgs: 97 active+clean; 2.0 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-08T21:03:56.012 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:03:55 smithi149 ceph-mon[35042]: pgmap v106: 97 pgs: 97 active+clean; 2.0 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-08T21:03:56.952 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:56.971+0000 7f2e6fe00640 1 -- 172.21.15.107:0/733858962 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2e70101340 msgr2=0x7f2e70101720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:03:56.952 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:56.971+0000 7f2e6fe00640 1 --2- 172.21.15.107:0/733858962 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2e70101340 0x7f2e70101720 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f2e60009ca0 tx=0x7f2e6002c100 comp rx=0 tx=0).stop 2024-03-08T21:03:56.952 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:56.971+0000 7f2e6fe00640 1 -- 172.21.15.107:0/733858962 shutdown_connections 2024-03-08T21:03:56.952 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:56.971+0000 7f2e6fe00640 1 --2- 172.21.15.107:0/733858962 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2e70106410 0x7f2e7010dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:03:56.952 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:56.971+0000 7f2e6fe00640 1 --2- 172.21.15.107:0/733858962 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2e70101cf0 0x7f2e70105d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:03:56.952 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:56.971+0000 7f2e6fe00640 1 --2- 172.21.15.107:0/733858962 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2e70101340 0x7f2e70101720 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:03:56.953 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:56.971+0000 7f2e6fe00640 1 -- 172.21.15.107:0/733858962 >> 172.21.15.107:0/733858962 conn(0x7f2e700fcc40 msgr2=0x7f2e700ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:03:56.953 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:56.971+0000 7f2e6fe00640 1 -- 172.21.15.107:0/733858962 shutdown_connections 2024-03-08T21:03:56.953 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:56.971+0000 7f2e6fe00640 1 -- 172.21.15.107:0/733858962 wait complete. 2024-03-08T21:03:56.955 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:56.972+0000 7f2e6fe00640 1 Processor -- start 2024-03-08T21:03:56.955 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:56.973+0000 7f2e6fe00640 1 -- start start 2024-03-08T21:03:56.955 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:56.973+0000 7f2e6fe00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2e70101340 0x7f2e7019d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:03:56.955 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:56.973+0000 7f2e6fe00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2e70106410 0x7f2e7019dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:03:56.955 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:56.973+0000 7f2e6fe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2e70197740 0x7f2e70197bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:03:56.955 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:56.973+0000 7f2e6fe00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e70111380 con 0x7f2e70197740 2024-03-08T21:03:56.955 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:56.973+0000 7f2e6fe00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e70111200 con 0x7f2e70106410 2024-03-08T21:03:56.956 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:56.973+0000 7f2e6fe00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e70111500 con 0x7f2e70101340 2024-03-08T21:03:56.956 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:56.973+0000 7f2e6f400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2e70197740 0x7f2e70197bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:03:56.956 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:56.973+0000 7f2e6f400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2e70197740 0x7f2e70197bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46826/0 (socket says 172.21.15.107:46826) 2024-03-08T21:03:56.956 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:56.973+0000 7f2e6f400640 1 -- 172.21.15.107:0/4115678186 learned_addr learned my addr 172.21.15.107:0/4115678186 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:03:56.956 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:56.974+0000 7f2e6ea00640 1 --2- 172.21.15.107:0/4115678186 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2e70101340 0x7f2e7019d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:03:56.956 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:56.974+0000 7f2e66000640 1 --2- 172.21.15.107:0/4115678186 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2e70106410 0x7f2e7019dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:03:56.956 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:56.974+0000 7f2e6f400640 1 -- 172.21.15.107:0/4115678186 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2e70101340 msgr2=0x7f2e7019d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:03:56.956 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:56.974+0000 7f2e6f400640 1 --2- 172.21.15.107:0/4115678186 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2e70101340 0x7f2e7019d680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:03:56.956 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:56.974+0000 7f2e6f400640 1 -- 172.21.15.107:0/4115678186 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2e70106410 msgr2=0x7f2e7019dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:03:56.956 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:56.974+0000 7f2e6f400640 1 --2- 172.21.15.107:0/4115678186 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2e70106410 0x7f2e7019dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:03:56.956 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:56.974+0000 7f2e6f400640 1 -- 172.21.15.107:0/4115678186 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e70198340 con 0x7f2e70197740 2024-03-08T21:03:56.956 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:56.974+0000 7f2e6f400640 1 --2- 172.21.15.107:0/4115678186 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2e70197740 0x7f2e70197bc0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f2e6800d750 tx=0x7f2e6800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:03:56.957 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:56.974+0000 7f2e67e00640 1 -- 172.21.15.107:0/4115678186 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e68017070 con 0x7f2e70197740 2024-03-08T21:03:56.957 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:56.975+0000 7f2e67e00640 1 -- 172.21.15.107:0/4115678186 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f2e680054d0 con 0x7f2e70197740 2024-03-08T21:03:56.957 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:56.975+0000 7f2e67e00640 1 -- 172.21.15.107:0/4115678186 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e68004c60 con 0x7f2e70197740 2024-03-08T21:03:56.957 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:56.975+0000 7f2e6fe00640 1 -- 172.21.15.107:0/4115678186 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e701a4420 con 0x7f2e70197740 2024-03-08T21:03:56.957 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:56.975+0000 7f2e6fe00640 1 -- 172.21.15.107:0/4115678186 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e701a4820 con 0x7f2e70197740 2024-03-08T21:03:56.958 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:56.978+0000 7f2e67e00640 1 -- 172.21.15.107:0/4115678186 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) v1 ==== 137422+0+0 (secure 0 0 0) 0x7f2e68004e00 con 0x7f2e70197740 2024-03-08T21:03:56.959 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:56.979+0000 7f2e789cc740 1 Processor -- start 2024-03-08T21:03:56.959 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:56.979+0000 7f2e789cc740 1 -- start start 2024-03-08T21:03:56.961 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:56.979+0000 7f2e789cc740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dfa8c38290 0x55dfa8c38670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:03:56.961 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:56.979+0000 7f2e789cc740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dfa8c38bb0 0x55dfa8c41440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:03:56.961 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:56.979+0000 7f2e789cc740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dfa8c41980 0x55dfa8c43d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:03:56.961 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:56.979+0000 7f2e789cc740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dfa8b973d0 con 0x55dfa8c38290 2024-03-08T21:03:56.961 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:56.979+0000 7f2e789cc740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dfa8b406b0 con 0x55dfa8c41980 2024-03-08T21:03:56.961 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:56.979+0000 7f2e789cc740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dfa8b93470 con 0x55dfa8c38bb0 2024-03-08T21:03:56.961 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:56.980+0000 7f2e66000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dfa8c38290 0x55dfa8c38670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:03:56.961 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:56.980+0000 7f2e66000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dfa8c38290 0x55dfa8c38670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46838/0 (socket says 172.21.15.107:46838) 2024-03-08T21:03:56.961 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:56.980+0000 7f2e66000640 1 -- 172.21.15.107:0/795583615 learned_addr learned my addr 172.21.15.107:0/795583615 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:03:56.961 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:56.980+0000 7f2e67e00640 1 --2- 172.21.15.107:0/4115678186 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f2e540a2960 0x7f2e540a4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:03:56.961 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:56.980+0000 7f2e6ea00640 1 --2- 172.21.15.107:0/795583615 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dfa8c38bb0 0x55dfa8c41440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:03:56.962 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:56.980+0000 7f2e66000640 1 --2- 172.21.15.107:0/795583615 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dfa8c41980 0x55dfa8c43d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:03:56.962 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:56.980+0000 7f2e67e00640 1 -- 172.21.15.107:0/4115678186 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f2e680ce8b0 con 0x7f2e70197740 2024-03-08T21:03:56.962 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:56.980+0000 7f2e66000640 1 -- 172.21.15.107:0/795583615 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dfa8c38bb0 msgr2=0x55dfa8c41440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:03:56.962 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:56.980+0000 7f2e66000640 1 --2- 172.21.15.107:0/795583615 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dfa8c38bb0 0x55dfa8c41440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:03:56.962 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:56.980+0000 7f2e66000640 1 -- 172.21.15.107:0/795583615 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dfa8c41980 msgr2=0x55dfa8c43d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:03:56.962 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:56.980+0000 7f2e66000640 1 --2- 172.21.15.107:0/795583615 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dfa8c41980 0x55dfa8c43d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:03:56.962 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:56.980+0000 7f2e66000640 1 -- 172.21.15.107:0/795583615 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dfa8b66b50 con 0x55dfa8c38290 2024-03-08T21:03:56.962 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:56.980+0000 7f2e6f400640 1 --2- 172.21.15.107:0/4115678186 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f2e540a2960 0x7f2e540a4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:03:56.962 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:56.980+0000 7f2e6ea00640 1 --2- 172.21.15.107:0/795583615 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dfa8c38bb0 0x55dfa8c41440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T21:03:56.962 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:56.981+0000 7f2e66000640 1 --2- 172.21.15.107:0/795583615 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dfa8c38290 0x55dfa8c38670 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f2e58004650 tx=0x7f2e5800c620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f914a2f38f5d8684 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:03:56.966 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:56.981+0000 7f2e5fe00640 1 -- 172.21.15.107:0/795583615 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e58018030 con 0x55dfa8c38290 2024-03-08T21:03:56.966 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:56.981+0000 7f2e6f400640 1 --2- 172.21.15.107:0/4115678186 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f2e540a2960 0x7f2e540a4e20 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f2e68004770 tx=0x7f2e680046c0 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:03:56.966 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:56.981+0000 7f2e5fe00640 1 -- 172.21.15.107:0/795583615 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f2e580054d0 con 0x55dfa8c38290 2024-03-08T21:03:56.966 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:56.981+0000 7f2e5fe00640 1 -- 172.21.15.107:0/795583615 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e58010640 con 0x55dfa8c38290 2024-03-08T21:03:56.967 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:56.982+0000 7f2e789cc740 1 -- 172.21.15.107:0/795583615 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dfa8c38290 msgr2=0x55dfa8c38670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:03:56.967 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:56.982+0000 7f2e789cc740 1 --2- 172.21.15.107:0/795583615 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dfa8c38290 0x55dfa8c38670 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f2e58004650 tx=0x7f2e5800c620 comp rx=0 tx=0).stop 2024-03-08T21:03:56.967 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:56.982+0000 7f2e789cc740 1 -- 172.21.15.107:0/795583615 shutdown_connections 2024-03-08T21:03:56.967 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:56.982+0000 7f2e789cc740 1 --2- 172.21.15.107:0/795583615 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dfa8c41980 0x55dfa8c43d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:03:56.967 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:56.982+0000 7f2e789cc740 1 --2- 172.21.15.107:0/795583615 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dfa8c38bb0 0x55dfa8c41440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:03:56.967 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:56.982+0000 7f2e789cc740 1 --2- 172.21.15.107:0/795583615 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dfa8c38290 0x55dfa8c38670 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:03:56.967 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:56.982+0000 7f2e789cc740 1 -- 172.21.15.107:0/795583615 >> 172.21.15.107:0/795583615 conn(0x55dfa8c32bb0 msgr2=0x55dfa8c32f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:03:56.967 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:56.982+0000 7f2e789cc740 1 -- 172.21.15.107:0/795583615 shutdown_connections 2024-03-08T21:03:56.967 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:56.982+0000 7f2e789cc740 1 -- 172.21.15.107:0/795583615 wait complete. 2024-03-08T21:03:56.967 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:56.983+0000 7f2e789cc740 1 -- start start 2024-03-08T21:03:56.967 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:56.983+0000 7f2e789cc740 1 Processor -- start 2024-03-08T21:03:56.967 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:56.983+0000 7f2e789cc740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dfa8c38290 0x55dfa8c41240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:03:56.967 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:56.983+0000 7f2e789cc740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dfa8c38bb0 0x55dfa8c3e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:03:56.968 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:56.983+0000 7f2e789cc740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dfa8c41980 0x55dfa8c3eb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:03:56.968 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:56.983+0000 7f2e789cc740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dfa8c445b0 con 0x55dfa8c38bb0 2024-03-08T21:03:56.968 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:56.983+0000 7f2e789cc740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dfa8b539f0 con 0x55dfa8c38290 2024-03-08T21:03:56.968 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:56.983+0000 7f2e789cc740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dfa8c44730 con 0x55dfa8c41980 2024-03-08T21:03:56.968 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:56.983+0000 7f2e789cc740 10 client.0 _ll_register_callbacks cb 0x55dfa88aba00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T21:03:56.968 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:56.984+0000 7f2e6ea00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dfa8c38bb0 0x55dfa8c3e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:03:56.968 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:56.984+0000 7f2e6ea00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dfa8c38bb0 0x55dfa8c3e5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46850/0 (socket says 172.21.15.107:46850) 2024-03-08T21:03:56.968 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:56.984+0000 7f2e6ea00640 1 -- 172.21.15.107:0/2028762962 learned_addr learned my addr 172.21.15.107:0/2028762962 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:03:56.968 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:56.984+0000 7f2e66000640 1 --2- 172.21.15.107:0/2028762962 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dfa8c38290 0x55dfa8c41240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:03:56.968 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:56.984+0000 7f2e66000640 1 --2- 172.21.15.107:0/2028762962 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dfa8c41980 0x55dfa8c3eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:03:56.968 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:56.984+0000 7f2e6ea00640 1 -- 172.21.15.107:0/2028762962 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dfa8c41980 msgr2=0x55dfa8c3eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:03:56.968 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:56.984+0000 7f2e6ea00640 1 --2- 172.21.15.107:0/2028762962 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dfa8c41980 0x55dfa8c3eb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:03:56.968 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:56.984+0000 7f2e6ea00640 1 -- 172.21.15.107:0/2028762962 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dfa8c38290 msgr2=0x55dfa8c41240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:03:56.969 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:56.984+0000 7f2e6ea00640 1 --2- 172.21.15.107:0/2028762962 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dfa8c38290 0x55dfa8c41240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:03:56.969 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:56.984+0000 7f2e6ea00640 1 -- 172.21.15.107:0/2028762962 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55dfa88ac210 con 0x55dfa8c38bb0 2024-03-08T21:03:56.969 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:56.984+0000 7f2e66000640 1 --2- 172.21.15.107:0/2028762962 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dfa8c41980 0x55dfa8c3eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T21:03:56.969 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:56.985+0000 7f2e6ea00640 1 --2- 172.21.15.107:0/2028762962 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dfa8c38bb0 0x55dfa8c3e5d0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f2e60037040 tx=0x7f2e60009a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=383aab3dc696f6e6 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:03:56.969 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:56.985+0000 7f2e5cc00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-03-08T21:03:56.969 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:56.985+0000 7f2e5cc00640 1 -- 172.21.15.107:0/2028762962 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e60039070 con 0x55dfa8c38bb0 2024-03-08T21:03:56.969 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:56.985+0000 7f2e5cc00640 1 -- 172.21.15.107:0/2028762962 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f2e60003970 con 0x55dfa8c38bb0 2024-03-08T21:03:56.969 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:56.985+0000 7f2e789cc740 1 -- 172.21.15.107:0/2028762962 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2e580054d0 con 0x55dfa8c38bb0 2024-03-08T21:03:56.969 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:56.985+0000 7f2e5cc00640 1 -- 172.21.15.107:0/2028762962 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2e6004e050 con 0x55dfa8c38bb0 2024-03-08T21:03:56.969 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:56.985+0000 7f2e789cc740 10 client.15744 fetch_fsmap learned FSMap version 32 2024-03-08T21:03:56.969 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:56.985+0000 7f2e789cc740 1 -- 172.21.15.107:0/2028762962 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55dfa8978620 con 0x55dfa8c38bb0 2024-03-08T21:03:56.969 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:56.986+0000 7f2e5cc00640 1 -- 172.21.15.107:0/2028762962 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f2e580054d0 con 0x55dfa8c38bb0 2024-03-08T21:03:56.969 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:56.986+0000 7f2e5cc00640 1 -- 172.21.15.107:0/2028762962 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 32) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f2e60040480 con 0x55dfa8c38bb0 2024-03-08T21:03:56.969 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:56.986+0000 7f2e789cc740 10 client.15744 fetch_fsmap finished waiting for FSMap version 32 2024-03-08T21:03:56.970 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:56.986+0000 7f2e789cc740 10 client.15744 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-08T21:03:56.970 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:56.986+0000 7f2e789cc740 20 client.15744 populate_metadata read hostname 'smithi107' 2024-03-08T21:03:56.970 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:56.986+0000 7f2e789cc740 1 --2- 172.21.15.107:0/2028762962 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55dfa8c516e0 0x55dfa8c53ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:03:56.970 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:56.986+0000 7f2e789cc740 4 client.15744 mds_command: new command op to 34475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-08T21:03:56.970 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:56.986+0000 7f2e789cc740 1 -- 172.21.15.107:0/2028762962 --> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55dfa8ba4910 con 0x55dfa8c516e0 2024-03-08T21:03:56.971 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:56.987+0000 7f2e66000640 1 --2- 172.21.15.107:0/2028762962 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55dfa8c516e0 0x55dfa8c53ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:03:56.971 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:56.987+0000 7f2e66000640 1 --2- 172.21.15.107:0/2028762962 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55dfa8c516e0 0x55dfa8c53ad0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5c31572565dcc681 server_cookie=4fb929e2cc0477 in_seq=0 out_seq=0 2024-03-08T21:03:56.971 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:56.987+0000 7f2e5cc00640 10 client.15744 ms_handle_connect on v2:172.21.15.149:6838/55290596 2024-03-08T21:03:56.971 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:56.989+0000 7f2e5cc00640 1 -- 172.21.15.107:0/2028762962 <== mds.0 v2:172.21.15.149:6838/55290596 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55dfa8ba4910 con 0x55dfa8c516e0 2024-03-08T21:03:56.971 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:56.989+0000 7f2e5cc00640 10 client.15744 handle_command_reply: tid=0 2024-03-08T21:03:56.972 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:56.989+0000 7f2e789cc740 1 client.15744 shutdown 2024-03-08T21:03:56.972 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:56.989+0000 7f2e789cc740 1 -- 172.21.15.107:0/2028762962 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dfa8c38bb0 msgr2=0x55dfa8c3e5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:03:56.972 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:56.989+0000 7f2e789cc740 1 --2- 172.21.15.107:0/2028762962 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dfa8c38bb0 0x55dfa8c3e5d0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f2e60037040 tx=0x7f2e60009a10 comp rx=0 tx=0).stop 2024-03-08T21:03:56.972 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:56.989+0000 7f2e789cc740 1 -- 172.21.15.107:0/2028762962 shutdown_connections 2024-03-08T21:03:56.972 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:56.989+0000 7f2e789cc740 1 --2- 172.21.15.107:0/2028762962 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dfa8c41980 0x55dfa8c3eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:03:56.972 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:56.989+0000 7f2e789cc740 1 --2- 172.21.15.107:0/2028762962 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dfa8c38bb0 0x55dfa8c3e5d0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:03:56.972 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:56.989+0000 7f2e789cc740 1 --2- 172.21.15.107:0/2028762962 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55dfa8c516e0 0x55dfa8c53ad0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:03:56.972 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:56.989+0000 7f2e789cc740 1 --2- 172.21.15.107:0/2028762962 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dfa8c38290 0x55dfa8c41240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:03:56.972 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:56.989+0000 7f2e5cc00640 0 client.15744 ms_handle_reset on v2:172.21.15.149:6838/55290596 2024-03-08T21:03:56.972 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:56.989+0000 7f2e789cc740 1 -- 172.21.15.107:0/2028762962 >> 172.21.15.107:0/2028762962 conn(0x55dfa8c32bb0 msgr2=0x55dfa8c34900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:03:56.972 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:56.990+0000 7f2e789cc740 1 -- 172.21.15.107:0/2028762962 shutdown_connections 2024-03-08T21:03:56.972 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:56.990+0000 7f2e789cc740 1 -- 172.21.15.107:0/2028762962 wait complete. 2024-03-08T21:03:56.972 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:56.990+0000 7f2e789cc740 20 client.15744 trim_cache size 0 max 16384 2024-03-08T21:03:56.985 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:57.004+0000 7f2e789cc740 1 Processor -- start 2024-03-08T21:03:56.985 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:57.004+0000 7f2e789cc740 1 -- start start 2024-03-08T21:03:56.985 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:57.004+0000 7f2e789cc740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dfa8c38290 0x55dfa8c40d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:03:56.985 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:57.004+0000 7f2e789cc740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dfa8c38bb0 0x55dfa8c41240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:03:56.985 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:57.004+0000 7f2e789cc740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dfa8c41980 0x55dfa8c3b830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:03:56.985 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:57.004+0000 7f2e789cc740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e701aa860 con 0x55dfa8c41980 2024-03-08T21:03:56.985 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:57.004+0000 7f2e789cc740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e701ac1e0 con 0x55dfa8c38bb0 2024-03-08T21:03:56.985 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:57.004+0000 7f2e789cc740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e701aaec0 con 0x55dfa8c38290 2024-03-08T21:03:56.987 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:57.005+0000 7f2e66000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dfa8c41980 0x55dfa8c3b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:03:56.987 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:57.005+0000 7f2e66000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dfa8c41980 0x55dfa8c3b830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46860/0 (socket says 172.21.15.107:46860) 2024-03-08T21:03:56.987 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:57.005+0000 7f2e66000640 1 -- 172.21.15.107:0/55250509 learned_addr learned my addr 172.21.15.107:0/55250509 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:03:56.987 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:57.005+0000 7f2e66000640 1 --2- 172.21.15.107:0/55250509 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dfa8c38290 0x55dfa8c40d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:03:56.987 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:57.005+0000 7f2e6ea00640 1 --2- 172.21.15.107:0/55250509 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dfa8c38bb0 0x55dfa8c41240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:03:56.987 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:57.005+0000 7f2e66000640 1 -- 172.21.15.107:0/55250509 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dfa8c38290 msgr2=0x55dfa8c40d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:03:56.987 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:57.005+0000 7f2e66000640 1 --2- 172.21.15.107:0/55250509 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dfa8c38290 0x55dfa8c40d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:03:56.987 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:57.005+0000 7f2e66000640 1 -- 172.21.15.107:0/55250509 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dfa8c38bb0 msgr2=0x55dfa8c41240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:03:56.987 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:57.005+0000 7f2e66000640 1 --2- 172.21.15.107:0/55250509 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dfa8c38bb0 0x55dfa8c41240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:03:56.987 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:57.005+0000 7f2e66000640 1 -- 172.21.15.107:0/55250509 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dfa8c389e0 con 0x55dfa8c41980 2024-03-08T21:03:56.987 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:57.005+0000 7f2e6ea00640 1 --2- 172.21.15.107:0/55250509 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dfa8c38bb0 0x55dfa8c41240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T21:03:56.987 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:57.006+0000 7f2e66000640 1 --2- 172.21.15.107:0/55250509 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dfa8c41980 0x55dfa8c3b830 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f2e5800ceb0 tx=0x7f2e5800c790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ef07ab5204fd15c server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:03:56.988 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:57.006+0000 7f2e5f400640 1 -- 172.21.15.107:0/55250509 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e58018040 con 0x55dfa8c41980 2024-03-08T21:03:56.988 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:57.006+0000 7f2e5f400640 1 -- 172.21.15.107:0/55250509 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f2e58019040 con 0x55dfa8c41980 2024-03-08T21:03:56.988 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:57.006+0000 7f2e5f400640 1 -- 172.21.15.107:0/55250509 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e58005220 con 0x55dfa8c41980 2024-03-08T21:03:56.991 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:57.007+0000 7f2e789cc740 1 -- 172.21.15.107:0/55250509 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dfa8c41980 msgr2=0x55dfa8c3b830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:03:56.991 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:57.007+0000 7f2e789cc740 1 --2- 172.21.15.107:0/55250509 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dfa8c41980 0x55dfa8c3b830 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f2e5800ceb0 tx=0x7f2e5800c790 comp rx=0 tx=0).stop 2024-03-08T21:03:56.991 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:57.007+0000 7f2e789cc740 1 -- 172.21.15.107:0/55250509 shutdown_connections 2024-03-08T21:03:56.991 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:57.007+0000 7f2e789cc740 1 --2- 172.21.15.107:0/55250509 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dfa8c41980 0x55dfa8c3b830 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:03:56.991 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:57.007+0000 7f2e789cc740 1 --2- 172.21.15.107:0/55250509 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dfa8c38bb0 0x55dfa8c41240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:03:56.991 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:57.007+0000 7f2e789cc740 1 --2- 172.21.15.107:0/55250509 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dfa8c38290 0x55dfa8c40d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:03:56.991 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:57.007+0000 7f2e789cc740 1 -- 172.21.15.107:0/55250509 >> 172.21.15.107:0/55250509 conn(0x55dfa8c32bb0 msgr2=0x55dfa8c3cea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:03:56.991 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:57.007+0000 7f2e789cc740 1 -- 172.21.15.107:0/55250509 shutdown_connections 2024-03-08T21:03:56.991 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:57.007+0000 7f2e789cc740 1 -- 172.21.15.107:0/55250509 wait complete. 2024-03-08T21:03:56.991 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:57.008+0000 7f2e789cc740 1 -- start start 2024-03-08T21:03:56.991 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:57.008+0000 7f2e789cc740 1 Processor -- start 2024-03-08T21:03:56.991 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:57.008+0000 7f2e789cc740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dfa8c38290 0x55dfa8c47330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:03:56.992 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:57.008+0000 7f2e789cc740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dfa8c38bb0 0x55dfa8c47870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:03:56.992 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:57.008+0000 7f2e789cc740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dfa8c41980 0x55dfa8c47db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:03:56.992 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:57.008+0000 7f2e789cc740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e701abb80 con 0x55dfa8c38290 2024-03-08T21:03:56.992 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:57.008+0000 7f2e789cc740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e701ab520 con 0x55dfa8c41980 2024-03-08T21:03:56.992 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:57.008+0000 7f2e789cc740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e701a8b40 con 0x55dfa8c38bb0 2024-03-08T21:03:56.992 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:57.009+0000 7f2e789cc740 10 client.0 _ll_register_callbacks cb 0x7f2e5800c480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T21:03:56.992 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:57.009+0000 7f2e66000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dfa8c38290 0x55dfa8c47330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:03:56.992 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:57.009+0000 7f2e66000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dfa8c38290 0x55dfa8c47330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46868/0 (socket says 172.21.15.107:46868) 2024-03-08T21:03:56.992 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:57.009+0000 7f2e66000640 1 -- 172.21.15.107:0/1541622296 learned_addr learned my addr 172.21.15.107:0/1541622296 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:03:56.992 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:57.009+0000 7f2e6ea00640 1 --2- 172.21.15.107:0/1541622296 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dfa8c38bb0 0x55dfa8c47870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:03:56.992 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:57.009+0000 7f2e66000640 1 -- 172.21.15.107:0/1541622296 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dfa8c38bb0 msgr2=0x55dfa8c47870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:03:56.992 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:57.009+0000 7f2e66000640 1 --2- 172.21.15.107:0/1541622296 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dfa8c38bb0 0x55dfa8c47870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:03:56.992 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:57.009+0000 7f2e66000640 1 -- 172.21.15.107:0/1541622296 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dfa8c41980 msgr2=0x55dfa8c47db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:03:56.992 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:57.009+0000 7f2e66000640 1 --2- 172.21.15.107:0/1541622296 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dfa8c41980 0x55dfa8c47db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:03:56.992 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:57.009+0000 7f2e66000640 1 -- 172.21.15.107:0/1541622296 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55dfa8ca7bc0 con 0x55dfa8c38290 2024-03-08T21:03:56.993 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:57.009+0000 7f2e66000640 1 --2- 172.21.15.107:0/1541622296 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dfa8c38290 0x55dfa8c47330 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f2e58004620 tx=0x7f2e5800cfa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1ff811b32eee693 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:03:56.993 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:57.009+0000 7f2e5cc00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-03-08T21:03:56.993 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:57.010+0000 7f2e5cc00640 1 -- 172.21.15.107:0/1541622296 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e58018040 con 0x55dfa8c38290 2024-03-08T21:03:56.993 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:57.010+0000 7f2e5cc00640 1 -- 172.21.15.107:0/1541622296 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f2e58004b60 con 0x55dfa8c38290 2024-03-08T21:03:56.993 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:57.010+0000 7f2e789cc740 1 -- 172.21.15.107:0/1541622296 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2e58019040 con 0x55dfa8c38290 2024-03-08T21:03:56.993 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:57.010+0000 7f2e5cc00640 1 -- 172.21.15.107:0/1541622296 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2e58029320 con 0x55dfa8c38290 2024-03-08T21:03:56.993 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:57.010+0000 7f2e789cc740 10 client.15750 fetch_fsmap learned FSMap version 32 2024-03-08T21:03:56.993 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:57.010+0000 7f2e789cc740 1 -- 172.21.15.107:0/1541622296 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55dfa8ca8ed0 con 0x55dfa8c38290 2024-03-08T21:03:56.993 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:57.010+0000 7f2e5cc00640 1 -- 172.21.15.107:0/1541622296 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f2e58019040 con 0x55dfa8c38290 2024-03-08T21:03:56.993 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:57.011+0000 7f2e5cc00640 1 -- 172.21.15.107:0/1541622296 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 32) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f2e58010640 con 0x55dfa8c38290 2024-03-08T21:03:56.993 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:57.011+0000 7f2e789cc740 10 client.15750 fetch_fsmap finished waiting for FSMap version 32 2024-03-08T21:03:56.993 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:57.011+0000 7f2e789cc740 10 client.15750 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-08T21:03:56.993 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:57.011+0000 7f2e789cc740 20 client.15750 populate_metadata read hostname 'smithi107' 2024-03-08T21:03:56.993 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:57.011+0000 7f2e789cc740 1 --2- 172.21.15.107:0/1541622296 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55dfa8ca9350 0x55dfa8cab740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:03:56.993 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:57.011+0000 7f2e789cc740 4 client.15750 mds_command: new command op to 34475 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-08T21:03:56.994 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:57.011+0000 7f2e789cc740 1 -- 172.21.15.107:0/1541622296 --> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55dfa8b9ec60 con 0x55dfa8ca9350 2024-03-08T21:03:56.994 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:57.011+0000 7f2e6ea00640 1 --2- 172.21.15.107:0/1541622296 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55dfa8ca9350 0x55dfa8cab740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:03:56.994 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:57.012+0000 7f2e6ea00640 1 --2- 172.21.15.107:0/1541622296 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55dfa8ca9350 0x55dfa8cab740 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=494f2ca334bba1e9 server_cookie=313e69deac382ba0 in_seq=0 out_seq=0 2024-03-08T21:03:56.994 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:57.012+0000 7f2e5cc00640 10 client.15750 ms_handle_connect on v2:172.21.15.149:6838/55290596 2024-03-08T21:03:57.002 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:57.021+0000 7f2e5cc00640 1 -- 172.21.15.107:0/1541622296 <== mds.0 v2:172.21.15.149:6838/55290596 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55dfa8b9ec60 con 0x55dfa8ca9350 2024-03-08T21:03:57.003 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:57.021+0000 7f2e5cc00640 10 client.15750 handle_command_reply: tid=0 2024-03-08T21:03:57.003 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:57.022+0000 7f2e789cc740 1 client.15750 shutdown 2024-03-08T21:03:57.003 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:57.022+0000 7f2e789cc740 1 -- 172.21.15.107:0/1541622296 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dfa8c38290 msgr2=0x55dfa8c47330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:03:57.003 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:57.022+0000 7f2e789cc740 1 --2- 172.21.15.107:0/1541622296 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dfa8c38290 0x55dfa8c47330 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f2e58004620 tx=0x7f2e5800cfa0 comp rx=0 tx=0).stop 2024-03-08T21:03:57.003 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:57.022+0000 7f2e789cc740 1 -- 172.21.15.107:0/1541622296 shutdown_connections 2024-03-08T21:03:57.003 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:57.022+0000 7f2e789cc740 1 --2- 172.21.15.107:0/1541622296 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55dfa8ca9350 0x55dfa8cab740 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:03:57.003 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:57.022+0000 7f2e789cc740 1 --2- 172.21.15.107:0/1541622296 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dfa8c41980 0x55dfa8c47db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:03:57.003 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:57.022+0000 7f2e789cc740 1 --2- 172.21.15.107:0/1541622296 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dfa8c38bb0 0x55dfa8c47870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:03:57.003 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:57.022+0000 7f2e789cc740 1 --2- 172.21.15.107:0/1541622296 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dfa8c38290 0x55dfa8c47330 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:03:57.003 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:57.022+0000 7f2e789cc740 1 -- 172.21.15.107:0/1541622296 >> 172.21.15.107:0/1541622296 conn(0x55dfa8c32bb0 msgr2=0x55dfa8c2d800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:03:57.003 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:57.022+0000 7f2e5cc00640 0 client.15750 ms_handle_reset on v2:172.21.15.149:6838/55290596 2024-03-08T21:03:57.003 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:57.022+0000 7f2e789cc740 1 -- 172.21.15.107:0/1541622296 shutdown_connections 2024-03-08T21:03:57.003 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:57.022+0000 7f2e789cc740 1 -- 172.21.15.107:0/1541622296 wait complete. 2024-03-08T21:03:57.003 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:57.022+0000 7f2e789cc740 20 client.15750 trim_cache size 0 max 16384 2024-03-08T21:03:57.004 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:{ 2024-03-08T21:03:57.004 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "message": "", 2024-03-08T21:03:57.004 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "return_code": 0 2024-03-08T21:03:57.004 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:} 2024-03-08T21:03:57.005 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:57.023+0000 7f2e6fe00640 1 -- 172.21.15.107:0/4115678186 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f2e540a2960 msgr2=0x7f2e540a4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:03:57.005 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:57.023+0000 7f2e6fe00640 1 --2- 172.21.15.107:0/4115678186 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f2e540a2960 0x7f2e540a4e20 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f2e68004770 tx=0x7f2e680046c0 comp rx=0 tx=0).stop 2024-03-08T21:03:57.005 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:57.023+0000 7f2e6fe00640 1 -- 172.21.15.107:0/4115678186 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2e70197740 msgr2=0x7f2e70197bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:03:57.005 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:57.023+0000 7f2e6fe00640 1 --2- 172.21.15.107:0/4115678186 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2e70197740 0x7f2e70197bc0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f2e6800d750 tx=0x7f2e6800dc10 comp rx=0 tx=0).stop 2024-03-08T21:03:57.005 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:57.023+0000 7f2e6fe00640 1 -- 172.21.15.107:0/4115678186 shutdown_connections 2024-03-08T21:03:57.005 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:57.023+0000 7f2e6fe00640 1 --2- 172.21.15.107:0/4115678186 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f2e540a2960 0x7f2e540a4e20 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:03:57.005 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:57.023+0000 7f2e6fe00640 1 --2- 172.21.15.107:0/4115678186 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2e70197740 0x7f2e70197bc0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:03:57.005 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:57.023+0000 7f2e6fe00640 1 --2- 172.21.15.107:0/4115678186 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2e70106410 0x7f2e7019dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:03:57.005 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:57.023+0000 7f2e6fe00640 1 --2- 172.21.15.107:0/4115678186 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2e70101340 0x7f2e7019d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:03:57.005 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:57.023+0000 7f2e6fe00640 1 -- 172.21.15.107:0/4115678186 >> 172.21.15.107:0/4115678186 conn(0x7f2e700fcc40 msgr2=0x7f2e700fef60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:03:57.005 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:57.023+0000 7f2e6fe00640 1 -- 172.21.15.107:0/4115678186 shutdown_connections 2024-03-08T21:03:57.005 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:03:57.023+0000 7f2e6fe00640 1 -- 172.21.15.107:0/4115678186 wait complete. 2024-03-08T21:03:57.876 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:03:57 smithi204 ceph-mon[34935]: pgmap v107: 97 pgs: 97 active+clean; 2.0 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:03:57.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:03:57 smithi107 ceph-mon[31381]: pgmap v107: 97 pgs: 97 active+clean; 2.0 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:03:58.012 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:03:57 smithi149 ceph-mon[35042]: pgmap v107: 97 pgs: 97 active+clean; 2.0 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:03:59.876 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:03:59 smithi204 ceph-mon[34935]: pgmap v108: 97 pgs: 97 active+clean; 2.0 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:03:59.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:03:59 smithi107 ceph-mon[31381]: pgmap v108: 97 pgs: 97 active+clean; 2.0 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:04:00.012 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:03:59 smithi149 ceph-mon[35042]: pgmap v108: 97 pgs: 97 active+clean; 2.0 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:04:01.876 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:04:01 smithi204 ceph-mon[34935]: pgmap v109: 97 pgs: 97 active+clean; 2.0 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 511 B/s wr, 13 op/s 2024-03-08T21:04:01.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:04:01 smithi107 ceph-mon[31381]: pgmap v109: 97 pgs: 97 active+clean; 2.0 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 511 B/s wr, 13 op/s 2024-03-08T21:04:02.012 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:04:01 smithi149 ceph-mon[35042]: pgmap v109: 97 pgs: 97 active+clean; 2.0 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 511 B/s wr, 13 op/s 2024-03-08T21:04:03.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:04:03 smithi107 ceph-mon[31381]: pgmap v110: 97 pgs: 97 active+clean; 2.0 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:04:04.012 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:04:03 smithi149 ceph-mon[35042]: pgmap v110: 97 pgs: 97 active+clean; 2.0 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:04:04.126 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:04:03 smithi204 ceph-mon[34935]: pgmap v110: 97 pgs: 97 active+clean; 2.0 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:04:05.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:04:05 smithi107 ceph-mon[31381]: pgmap v111: 97 pgs: 97 active+clean; 2.6 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 76 KiB/s wr, 27 op/s 2024-03-08T21:04:06.012 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:04:05 smithi149 ceph-mon[35042]: pgmap v111: 97 pgs: 97 active+clean; 2.6 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 76 KiB/s wr, 27 op/s 2024-03-08T21:04:06.126 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:04:05 smithi204 ceph-mon[34935]: pgmap v111: 97 pgs: 97 active+clean; 2.6 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 76 KiB/s wr, 27 op/s 2024-03-08T21:04:07.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:04:07 smithi107 ceph-mon[31381]: pgmap v112: 97 pgs: 97 active+clean; 2.6 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 76 KiB/s wr, 23 op/s 2024-03-08T21:04:08.012 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:04:07 smithi149 ceph-mon[35042]: pgmap v112: 97 pgs: 97 active+clean; 2.6 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 76 KiB/s wr, 23 op/s 2024-03-08T21:04:08.125 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:04:07 smithi204 ceph-mon[34935]: pgmap v112: 97 pgs: 97 active+clean; 2.6 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 76 KiB/s wr, 23 op/s 2024-03-08T21:04:08.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:04:08 smithi107 ceph-mon[31381]: pgmap v113: 97 pgs: 97 active+clean; 2.6 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 76 KiB/s wr, 24 op/s 2024-03-08T21:04:09.012 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:04:08 smithi149 ceph-mon[35042]: pgmap v113: 97 pgs: 97 active+clean; 2.6 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 76 KiB/s wr, 24 op/s 2024-03-08T21:04:09.125 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:04:08 smithi204 ceph-mon[34935]: pgmap v113: 97 pgs: 97 active+clean; 2.6 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 76 KiB/s wr, 24 op/s 2024-03-08T21:04:10.150 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.168+0000 7f0e64c00640 1 -- 172.21.15.107:0/675680848 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0e60102bc0 msgr2=0x7f0e6010b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:04:10.150 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.168+0000 7f0e64c00640 1 --2- 172.21.15.107:0/675680848 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0e60102bc0 0x7f0e6010b0a0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f0e48009900 tx=0x7f0e4802bcd0 comp rx=0 tx=0).stop 2024-03-08T21:04:10.150 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.169+0000 7f0e64c00640 1 -- 172.21.15.107:0/675680848 shutdown_connections 2024-03-08T21:04:10.150 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.169+0000 7f0e64c00640 1 --2- 172.21.15.107:0/675680848 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0e6010b6c0 0x7f0e6010db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:04:10.150 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.169+0000 7f0e64c00640 1 --2- 172.21.15.107:0/675680848 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0e60102bc0 0x7f0e6010b0a0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:04:10.151 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.169+0000 7f0e64c00640 1 --2- 172.21.15.107:0/675680848 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0e601022a0 0x7f0e60102680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:04:10.151 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.169+0000 7f0e64c00640 1 -- 172.21.15.107:0/675680848 >> 172.21.15.107:0/675680848 conn(0x7f0e600fbb00 msgr2=0x7f0e600fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:04:10.151 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.169+0000 7f0e64c00640 1 -- 172.21.15.107:0/675680848 shutdown_connections 2024-03-08T21:04:10.151 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.169+0000 7f0e64c00640 1 -- 172.21.15.107:0/675680848 wait complete. 2024-03-08T21:04:10.152 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.170+0000 7f0e64c00640 1 Processor -- start 2024-03-08T21:04:10.152 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.170+0000 7f0e64c00640 1 -- start start 2024-03-08T21:04:10.152 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.170+0000 7f0e64c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0e601022a0 0x7f0e6019f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:04:10.152 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.170+0000 7f0e64c00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0e60102bc0 0x7f0e6019fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:04:10.152 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.170+0000 7f0e64c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0e6010b6c0 0x7f0e601998c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:04:10.152 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.170+0000 7f0e64c00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e60111380 con 0x7f0e6010b6c0 2024-03-08T21:04:10.152 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.170+0000 7f0e64c00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e60111200 con 0x7f0e60102bc0 2024-03-08T21:04:10.153 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.170+0000 7f0e64c00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e60111500 con 0x7f0e601022a0 2024-03-08T21:04:10.153 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.171+0000 7f0e5fe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0e6010b6c0 0x7f0e601998c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:04:10.153 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.171+0000 7f0e5fe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0e6010b6c0 0x7f0e601998c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:44846/0 (socket says 172.21.15.107:44846) 2024-03-08T21:04:10.153 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.171+0000 7f0e5fe00640 1 -- 172.21.15.107:0/2797162953 learned_addr learned my addr 172.21.15.107:0/2797162953 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:04:10.153 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.171+0000 7f0e5f400640 1 --2- 172.21.15.107:0/2797162953 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0e601022a0 0x7f0e6019f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:04:10.153 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.171+0000 7f0e57e00640 1 --2- 172.21.15.107:0/2797162953 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0e60102bc0 0x7f0e6019fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:04:10.153 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.171+0000 7f0e5fe00640 1 -- 172.21.15.107:0/2797162953 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0e601022a0 msgr2=0x7f0e6019f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:04:10.153 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.171+0000 7f0e5fe00640 1 --2- 172.21.15.107:0/2797162953 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0e601022a0 0x7f0e6019f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:04:10.153 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.171+0000 7f0e5fe00640 1 -- 172.21.15.107:0/2797162953 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0e60102bc0 msgr2=0x7f0e6019fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:04:10.153 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.171+0000 7f0e5fe00640 1 --2- 172.21.15.107:0/2797162953 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0e60102bc0 0x7f0e6019fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:04:10.153 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.171+0000 7f0e5fe00640 1 -- 172.21.15.107:0/2797162953 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e6019a040 con 0x7f0e6010b6c0 2024-03-08T21:04:10.154 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.172+0000 7f0e5fe00640 1 --2- 172.21.15.107:0/2797162953 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0e6010b6c0 0x7f0e601998c0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f0e5800cad0 tx=0x7f0e5800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:04:10.154 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.172+0000 7f0e5cc00640 1 -- 172.21.15.107:0/2797162953 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e5800ae20 con 0x7f0e6010b6c0 2024-03-08T21:04:10.154 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.172+0000 7f0e5cc00640 1 -- 172.21.15.107:0/2797162953 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f0e58005020 con 0x7f0e6010b6c0 2024-03-08T21:04:10.154 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.172+0000 7f0e64c00640 1 -- 172.21.15.107:0/2797162953 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e6019a330 con 0x7f0e6010b6c0 2024-03-08T21:04:10.154 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.172+0000 7f0e5cc00640 1 -- 172.21.15.107:0/2797162953 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e5800f6b0 con 0x7f0e6010b6c0 2024-03-08T21:04:10.154 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.172+0000 7f0e64c00640 1 -- 172.21.15.107:0/2797162953 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e601a65d0 con 0x7f0e6010b6c0 2024-03-08T21:04:10.156 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.175+0000 7f0e5cc00640 1 -- 172.21.15.107:0/2797162953 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) v1 ==== 137422+0+0 (secure 0 0 0) 0x7f0e5800f850 con 0x7f0e6010b6c0 2024-03-08T21:04:10.156 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.175+0000 7f0e68e11740 1 Processor -- start 2024-03-08T21:04:10.157 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.176+0000 7f0e68e11740 1 -- start start 2024-03-08T21:04:10.159 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.176+0000 7f0e68e11740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dc4ad74e80 0x55dc4ad75260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:04:10.159 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.176+0000 7f0e68e11740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc4ad757a0 0x55dc4ad7e030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:04:10.159 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.176+0000 7f0e68e11740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dc4ad7e570 0x55dc4ad80960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:04:10.159 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.176+0000 7f0e68e11740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc4acd83d0 con 0x55dc4ad7e570 2024-03-08T21:04:10.159 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.176+0000 7f0e68e11740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc4ac816b0 con 0x55dc4ad74e80 2024-03-08T21:04:10.159 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.176+0000 7f0e68e11740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc4acd4470 con 0x55dc4ad757a0 2024-03-08T21:04:10.159 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.176+0000 7f0e57e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dc4ad7e570 0x55dc4ad80960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:04:10.159 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.176+0000 7f0e57e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dc4ad7e570 0x55dc4ad80960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:44856/0 (socket says 172.21.15.107:44856) 2024-03-08T21:04:10.159 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.176+0000 7f0e57e00640 1 -- 172.21.15.107:0/1255918857 learned_addr learned my addr 172.21.15.107:0/1255918857 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:04:10.159 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.177+0000 7f0e5f400640 1 --2- 172.21.15.107:0/1255918857 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc4ad757a0 0x55dc4ad7e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:04:10.159 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.177+0000 7f0e57e00640 1 --2- 172.21.15.107:0/1255918857 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dc4ad74e80 0x55dc4ad75260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:04:10.160 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.177+0000 7f0e57e00640 1 -- 172.21.15.107:0/1255918857 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc4ad757a0 msgr2=0x55dc4ad7e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:04:10.160 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.177+0000 7f0e57e00640 1 --2- 172.21.15.107:0/1255918857 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc4ad757a0 0x55dc4ad7e030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:04:10.160 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.177+0000 7f0e57e00640 1 -- 172.21.15.107:0/1255918857 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dc4ad74e80 msgr2=0x55dc4ad75260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:04:10.160 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.177+0000 7f0e57e00640 1 --2- 172.21.15.107:0/1255918857 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dc4ad74e80 0x55dc4ad75260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:04:10.160 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.177+0000 7f0e57e00640 1 -- 172.21.15.107:0/1255918857 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dc4aca7b50 con 0x55dc4ad7e570 2024-03-08T21:04:10.160 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.177+0000 7f0e5cc00640 1 --2- 172.21.15.107:0/2797162953 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f0e440a2960 0x7f0e440a4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:04:10.160 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.177+0000 7f0e5cc00640 1 -- 172.21.15.107:0/2797162953 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f0e580cefa0 con 0x7f0e6010b6c0 2024-03-08T21:04:10.160 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.177+0000 7f0e57e00640 1 --2- 172.21.15.107:0/1255918857 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dc4ad7e570 0x55dc4ad80960 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f0e48009230 tx=0x7f0e480056d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1cf05bad5ca233a server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:04:10.160 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.177+0000 7f0e5f400640 1 --2- 172.21.15.107:0/2797162953 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f0e440a2960 0x7f0e440a4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:04:10.160 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.178+0000 7f0e4fe00640 1 -- 172.21.15.107:0/1255918857 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e48048070 con 0x55dc4ad7e570 2024-03-08T21:04:10.160 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.178+0000 7f0e4fe00640 1 -- 172.21.15.107:0/1255918857 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f0e4802cd30 con 0x55dc4ad7e570 2024-03-08T21:04:10.160 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.178+0000 7f0e4fe00640 1 -- 172.21.15.107:0/1255918857 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e48003e60 con 0x55dc4ad7e570 2024-03-08T21:04:10.164 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.179+0000 7f0e68e11740 1 -- 172.21.15.107:0/1255918857 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dc4ad7e570 msgr2=0x55dc4ad80960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:04:10.164 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.179+0000 7f0e5f400640 1 --2- 172.21.15.107:0/2797162953 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f0e440a2960 0x7f0e440a4e20 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x55dc4ad74930 tx=0x7f0e5000c4b0 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:04:10.164 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.179+0000 7f0e68e11740 1 --2- 172.21.15.107:0/1255918857 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dc4ad7e570 0x55dc4ad80960 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f0e48009230 tx=0x7f0e480056d0 comp rx=0 tx=0).stop 2024-03-08T21:04:10.164 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.179+0000 7f0e68e11740 1 -- 172.21.15.107:0/1255918857 shutdown_connections 2024-03-08T21:04:10.164 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.179+0000 7f0e68e11740 1 --2- 172.21.15.107:0/1255918857 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dc4ad7e570 0x55dc4ad80960 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:04:10.164 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.179+0000 7f0e68e11740 1 --2- 172.21.15.107:0/1255918857 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc4ad757a0 0x55dc4ad7e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:04:10.164 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.179+0000 7f0e68e11740 1 --2- 172.21.15.107:0/1255918857 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dc4ad74e80 0x55dc4ad75260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:04:10.164 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.179+0000 7f0e68e11740 1 -- 172.21.15.107:0/1255918857 >> 172.21.15.107:0/1255918857 conn(0x55dc4ad6f7a0 msgr2=0x55dc4ad6fb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:04:10.164 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.179+0000 7f0e68e11740 1 -- 172.21.15.107:0/1255918857 shutdown_connections 2024-03-08T21:04:10.164 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.179+0000 7f0e68e11740 1 -- 172.21.15.107:0/1255918857 wait complete. 2024-03-08T21:04:10.165 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.179+0000 7f0e68e11740 1 -- start start 2024-03-08T21:04:10.165 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.180+0000 7f0e68e11740 1 Processor -- start 2024-03-08T21:04:10.165 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.180+0000 7f0e68e11740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc4ad74e80 0x55dc4ad7df00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:04:10.165 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.180+0000 7f0e68e11740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dc4ad757a0 0x55dc4ad7b290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:04:10.165 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.180+0000 7f0e68e11740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dc4ad7e570 0x55dc4ad7b810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:04:10.165 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.180+0000 7f0e68e11740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc4ad81140 con 0x55dc4ad757a0 2024-03-08T21:04:10.165 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.180+0000 7f0e68e11740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc4ac949f0 con 0x55dc4ad7e570 2024-03-08T21:04:10.165 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.180+0000 7f0e68e11740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc4ad812c0 con 0x55dc4ad74e80 2024-03-08T21:04:10.165 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.180+0000 7f0e68e11740 10 client.0 _ll_register_callbacks cb 0x55dc4a9eca00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T21:04:10.165 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.180+0000 7f0e57e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dc4ad757a0 0x55dc4ad7b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:04:10.165 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.181+0000 7f0e57e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dc4ad757a0 0x55dc4ad7b290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:44872/0 (socket says 172.21.15.107:44872) 2024-03-08T21:04:10.165 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.181+0000 7f0e57e00640 1 -- 172.21.15.107:0/3852274199 learned_addr learned my addr 172.21.15.107:0/3852274199 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:04:10.165 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.181+0000 7f0e5f400640 1 --2- 172.21.15.107:0/3852274199 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc4ad74e80 0x55dc4ad7df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:04:10.165 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.181+0000 7f0e5fe00640 1 --2- 172.21.15.107:0/3852274199 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dc4ad7e570 0x55dc4ad7b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:04:10.166 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.181+0000 7f0e57e00640 1 -- 172.21.15.107:0/3852274199 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc4ad74e80 msgr2=0x55dc4ad7df00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:04:10.166 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.181+0000 7f0e57e00640 1 --2- 172.21.15.107:0/3852274199 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc4ad74e80 0x55dc4ad7df00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:04:10.166 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.181+0000 7f0e57e00640 1 -- 172.21.15.107:0/3852274199 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dc4ad7e570 msgr2=0x55dc4ad7b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:04:10.166 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.181+0000 7f0e57e00640 1 --2- 172.21.15.107:0/3852274199 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dc4ad7e570 0x55dc4ad7b810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:04:10.166 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.181+0000 7f0e57e00640 1 -- 172.21.15.107:0/3852274199 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55dc4a9ed210 con 0x55dc4ad757a0 2024-03-08T21:04:10.166 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.181+0000 7f0e5f400640 1 --2- 172.21.15.107:0/3852274199 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc4ad74e80 0x55dc4ad7df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T21:04:10.166 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.181+0000 7f0e57e00640 1 --2- 172.21.15.107:0/3852274199 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dc4ad757a0 0x55dc4ad7b290 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f0e4802c210 tx=0x7f0e48009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a17a7a825a20ca0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:04:10.166 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.181+0000 7f0e4cc00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-03-08T21:04:10.166 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.182+0000 7f0e4cc00640 1 -- 172.21.15.107:0/3852274199 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e48053060 con 0x55dc4ad757a0 2024-03-08T21:04:10.166 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.182+0000 7f0e4cc00640 1 -- 172.21.15.107:0/3852274199 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f0e480037f0 con 0x55dc4ad757a0 2024-03-08T21:04:10.166 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.182+0000 7f0e68e11740 1 -- 172.21.15.107:0/3852274199 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0e4802cd30 con 0x55dc4ad757a0 2024-03-08T21:04:10.166 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.182+0000 7f0e4cc00640 1 -- 172.21.15.107:0/3852274199 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0e48048070 con 0x55dc4ad757a0 2024-03-08T21:04:10.166 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.182+0000 7f0e68e11740 10 client.15762 fetch_fsmap learned FSMap version 32 2024-03-08T21:04:10.166 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.182+0000 7f0e68e11740 1 -- 172.21.15.107:0/3852274199 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55dc4aab9620 con 0x55dc4ad757a0 2024-03-08T21:04:10.167 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.182+0000 7f0e4cc00640 1 -- 172.21.15.107:0/3852274199 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f0e4802cd30 con 0x55dc4ad757a0 2024-03-08T21:04:10.167 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.183+0000 7f0e4cc00640 1 -- 172.21.15.107:0/3852274199 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 32) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f0e4804eb10 con 0x55dc4ad757a0 2024-03-08T21:04:10.167 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.183+0000 7f0e68e11740 10 client.15762 fetch_fsmap finished waiting for FSMap version 32 2024-03-08T21:04:10.167 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.183+0000 7f0e68e11740 10 client.15762 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-08T21:04:10.167 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.183+0000 7f0e68e11740 20 client.15762 populate_metadata read hostname 'smithi107' 2024-03-08T21:04:10.167 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.183+0000 7f0e68e11740 1 --2- 172.21.15.107:0/3852274199 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55dc4ad8e280 0x55dc4ad90670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:04:10.167 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.183+0000 7f0e68e11740 4 client.15762 mds_command: new command op to 34475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-08T21:04:10.167 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.183+0000 7f0e68e11740 1 -- 172.21.15.107:0/3852274199 --> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55dc4ace5910 con 0x55dc4ad8e280 2024-03-08T21:04:10.168 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.183+0000 7f0e5f400640 1 --2- 172.21.15.107:0/3852274199 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55dc4ad8e280 0x55dc4ad90670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:04:10.168 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.184+0000 7f0e4cc00640 10 client.15762 ms_handle_connect on v2:172.21.15.149:6838/55290596 2024-03-08T21:04:10.168 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.184+0000 7f0e5f400640 1 --2- 172.21.15.107:0/3852274199 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55dc4ad8e280 0x55dc4ad90670 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d48dffe49b015011 server_cookie=5b6450b8c04f64d2 in_seq=0 out_seq=0 2024-03-08T21:04:10.169 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.186+0000 7f0e4cc00640 1 -- 172.21.15.107:0/3852274199 <== mds.0 v2:172.21.15.149:6838/55290596 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55dc4ace5910 con 0x55dc4ad8e280 2024-03-08T21:04:10.169 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.186+0000 7f0e4cc00640 10 client.15762 handle_command_reply: tid=0 2024-03-08T21:04:10.169 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.186+0000 7f0e68e11740 1 client.15762 shutdown 2024-03-08T21:04:10.169 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.186+0000 7f0e68e11740 1 -- 172.21.15.107:0/3852274199 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dc4ad757a0 msgr2=0x55dc4ad7b290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:04:10.169 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.186+0000 7f0e68e11740 1 --2- 172.21.15.107:0/3852274199 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dc4ad757a0 0x55dc4ad7b290 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f0e4802c210 tx=0x7f0e48009260 comp rx=0 tx=0).stop 2024-03-08T21:04:10.169 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.186+0000 7f0e68e11740 1 -- 172.21.15.107:0/3852274199 shutdown_connections 2024-03-08T21:04:10.169 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.186+0000 7f0e68e11740 1 --2- 172.21.15.107:0/3852274199 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55dc4ad8e280 0x55dc4ad90670 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:04:10.169 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.186+0000 7f0e68e11740 1 --2- 172.21.15.107:0/3852274199 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dc4ad7e570 0x55dc4ad7b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:04:10.169 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.186+0000 7f0e68e11740 1 --2- 172.21.15.107:0/3852274199 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dc4ad757a0 0x55dc4ad7b290 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:04:10.169 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.186+0000 7f0e68e11740 1 --2- 172.21.15.107:0/3852274199 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc4ad74e80 0x55dc4ad7df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:04:10.169 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.186+0000 7f0e4cc00640 0 client.15762 ms_handle_reset on v2:172.21.15.149:6838/55290596 2024-03-08T21:04:10.169 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.186+0000 7f0e68e11740 1 -- 172.21.15.107:0/3852274199 >> 172.21.15.107:0/3852274199 conn(0x55dc4ad6f7a0 msgr2=0x55dc4ad714f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:04:10.169 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.186+0000 7f0e68e11740 1 -- 172.21.15.107:0/3852274199 shutdown_connections 2024-03-08T21:04:10.169 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.187+0000 7f0e68e11740 1 -- 172.21.15.107:0/3852274199 wait complete. 2024-03-08T21:04:10.170 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.187+0000 7f0e68e11740 20 client.15762 trim_cache size 0 max 16384 2024-03-08T21:04:10.187 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.206+0000 7f0e68e11740 1 Processor -- start 2024-03-08T21:04:10.187 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.206+0000 7f0e68e11740 1 -- start start 2024-03-08T21:04:10.188 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.206+0000 7f0e68e11740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dc4ad74e80 0x55dc4ad89dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:04:10.188 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.206+0000 7f0e68e11740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc4ad757a0 0x55dc4ad7d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:04:10.188 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.206+0000 7f0e68e11740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dc4ad7e570 0x55dc4ad7df00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:04:10.188 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.206+0000 7f0e68e11740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e601aec00 con 0x55dc4ad74e80 2024-03-08T21:04:10.188 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.206+0000 7f0e68e11740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e601b0580 con 0x55dc4ad7e570 2024-03-08T21:04:10.188 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.206+0000 7f0e68e11740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e601af260 con 0x55dc4ad757a0 2024-03-08T21:04:10.190 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.207+0000 7f0e5f400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dc4ad74e80 0x55dc4ad89dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:04:10.190 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.207+0000 7f0e5f400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dc4ad74e80 0x55dc4ad89dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:44878/0 (socket says 172.21.15.107:44878) 2024-03-08T21:04:10.190 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.207+0000 7f0e5f400640 1 -- 172.21.15.107:0/3521711856 learned_addr learned my addr 172.21.15.107:0/3521711856 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:04:10.190 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.207+0000 7f0e5fe00640 1 --2- 172.21.15.107:0/3521711856 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dc4ad7e570 0x55dc4ad7df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:04:10.190 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.207+0000 7f0e57e00640 1 --2- 172.21.15.107:0/3521711856 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc4ad757a0 0x55dc4ad7d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:04:10.190 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.207+0000 7f0e5f400640 1 -- 172.21.15.107:0/3521711856 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc4ad757a0 msgr2=0x55dc4ad7d9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:04:10.190 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.207+0000 7f0e5f400640 1 --2- 172.21.15.107:0/3521711856 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc4ad757a0 0x55dc4ad7d9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:04:10.190 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.207+0000 7f0e5f400640 1 -- 172.21.15.107:0/3521711856 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dc4ad7e570 msgr2=0x55dc4ad7df00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:04:10.190 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.207+0000 7f0e5f400640 1 --2- 172.21.15.107:0/3521711856 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dc4ad7e570 0x55dc4ad7df00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:04:10.190 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.207+0000 7f0e5f400640 1 -- 172.21.15.107:0/3521711856 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dc4ad7bb80 con 0x55dc4ad74e80 2024-03-08T21:04:10.190 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.207+0000 7f0e5f400640 1 --2- 172.21.15.107:0/3521711856 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dc4ad74e80 0x55dc4ad89dd0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f0e6019aa00 tx=0x7f0e50014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2e03757f124cc1c server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:04:10.191 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.208+0000 7f0e4f400640 1 -- 172.21.15.107:0/3521711856 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e5001ae90 con 0x55dc4ad74e80 2024-03-08T21:04:10.191 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.208+0000 7f0e4f400640 1 -- 172.21.15.107:0/3521711856 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f0e50026860 con 0x55dc4ad74e80 2024-03-08T21:04:10.191 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.208+0000 7f0e4f400640 1 -- 172.21.15.107:0/3521711856 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e50004a00 con 0x55dc4ad74e80 2024-03-08T21:04:10.191 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.208+0000 7f0e68e11740 1 -- 172.21.15.107:0/3521711856 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dc4ad74e80 msgr2=0x55dc4ad89dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:04:10.191 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.208+0000 7f0e68e11740 1 --2- 172.21.15.107:0/3521711856 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dc4ad74e80 0x55dc4ad89dd0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f0e6019aa00 tx=0x7f0e50014000 comp rx=0 tx=0).stop 2024-03-08T21:04:10.191 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.209+0000 7f0e68e11740 1 -- 172.21.15.107:0/3521711856 shutdown_connections 2024-03-08T21:04:10.191 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.209+0000 7f0e68e11740 1 --2- 172.21.15.107:0/3521711856 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dc4ad7e570 0x55dc4ad7df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:04:10.191 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.209+0000 7f0e68e11740 1 --2- 172.21.15.107:0/3521711856 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc4ad757a0 0x55dc4ad7d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:04:10.191 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.209+0000 7f0e68e11740 1 --2- 172.21.15.107:0/3521711856 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dc4ad74e80 0x55dc4ad89dd0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:04:10.191 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.209+0000 7f0e68e11740 1 -- 172.21.15.107:0/3521711856 >> 172.21.15.107:0/3521711856 conn(0x55dc4ad6f7a0 msgr2=0x55dc4ad6fb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:04:10.191 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.209+0000 7f0e68e11740 1 -- 172.21.15.107:0/3521711856 shutdown_connections 2024-03-08T21:04:10.191 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.209+0000 7f0e68e11740 1 -- 172.21.15.107:0/3521711856 wait complete. 2024-03-08T21:04:10.194 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.209+0000 7f0e68e11740 1 -- start start 2024-03-08T21:04:10.194 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.209+0000 7f0e68e11740 1 Processor -- start 2024-03-08T21:04:10.194 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.210+0000 7f0e68e11740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dc4ad74e80 0x55dc4ad92b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:04:10.194 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.210+0000 7f0e68e11740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc4ad757a0 0x55dc4ad90f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:04:10.194 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.210+0000 7f0e68e11740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dc4ad7e570 0x55dc4ad916a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:04:10.194 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.210+0000 7f0e68e11740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e601aff20 con 0x55dc4ad7e570 2024-03-08T21:04:10.194 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.210+0000 7f0e68e11740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e601af8c0 con 0x55dc4ad74e80 2024-03-08T21:04:10.194 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.210+0000 7f0e68e11740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e601acea0 con 0x55dc4ad757a0 2024-03-08T21:04:10.194 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.210+0000 7f0e68e11740 10 client.0 _ll_register_callbacks cb 0x7f0e600699d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T21:04:10.194 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.210+0000 7f0e5fe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dc4ad7e570 0x55dc4ad916a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:04:10.194 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.210+0000 7f0e5fe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dc4ad7e570 0x55dc4ad916a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:44882/0 (socket says 172.21.15.107:44882) 2024-03-08T21:04:10.195 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.210+0000 7f0e5fe00640 1 -- 172.21.15.107:0/208985665 learned_addr learned my addr 172.21.15.107:0/208985665 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:04:10.195 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.210+0000 7f0e57e00640 1 --2- 172.21.15.107:0/208985665 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc4ad757a0 0x55dc4ad90f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:04:10.195 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.210+0000 7f0e5f400640 1 --2- 172.21.15.107:0/208985665 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dc4ad74e80 0x55dc4ad92b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:04:10.195 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.210+0000 7f0e5fe00640 1 -- 172.21.15.107:0/208985665 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc4ad757a0 msgr2=0x55dc4ad90f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:04:10.195 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.210+0000 7f0e5fe00640 1 --2- 172.21.15.107:0/208985665 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc4ad757a0 0x55dc4ad90f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:04:10.195 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.210+0000 7f0e5fe00640 1 -- 172.21.15.107:0/208985665 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dc4ad74e80 msgr2=0x55dc4ad92b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:04:10.195 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.210+0000 7f0e5fe00640 1 --2- 172.21.15.107:0/208985665 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dc4ad74e80 0x55dc4ad92b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:04:10.195 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.211+0000 7f0e5fe00640 1 -- 172.21.15.107:0/208985665 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55dc4ade4360 con 0x55dc4ad7e570 2024-03-08T21:04:10.195 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.211+0000 7f0e57e00640 1 --2- 172.21.15.107:0/208985665 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc4ad757a0 0x55dc4ad90f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T21:04:10.195 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.211+0000 7f0e5f400640 1 --2- 172.21.15.107:0/208985665 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dc4ad74e80 0x55dc4ad92b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T21:04:10.195 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.211+0000 7f0e5fe00640 1 --2- 172.21.15.107:0/208985665 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dc4ad7e570 0x55dc4ad916a0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f0e58097f60 tx=0x7f0e5809bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ee6bc2d20fe1f6a server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:04:10.195 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.211+0000 7f0e4cc00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-03-08T21:04:10.195 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.211+0000 7f0e4cc00640 1 -- 172.21.15.107:0/208985665 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e580b4060 con 0x55dc4ad7e570 2024-03-08T21:04:10.195 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.211+0000 7f0e4cc00640 1 -- 172.21.15.107:0/208985665 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f0e580051d0 con 0x55dc4ad7e570 2024-03-08T21:04:10.196 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.211+0000 7f0e68e11740 1 -- 172.21.15.107:0/208985665 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0e50026860 con 0x55dc4ad7e570 2024-03-08T21:04:10.196 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.211+0000 7f0e4cc00640 1 -- 172.21.15.107:0/208985665 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0e5801d040 con 0x55dc4ad7e570 2024-03-08T21:04:10.196 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.211+0000 7f0e68e11740 10 client.15768 fetch_fsmap learned FSMap version 32 2024-03-08T21:04:10.196 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.211+0000 7f0e68e11740 1 -- 172.21.15.107:0/208985665 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55dc4ade54b0 con 0x55dc4ad7e570 2024-03-08T21:04:10.196 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.212+0000 7f0e4cc00640 1 -- 172.21.15.107:0/208985665 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f0e50026860 con 0x55dc4ad7e570 2024-03-08T21:04:10.196 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.212+0000 7f0e4cc00640 1 -- 172.21.15.107:0/208985665 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 32) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f0e5809f170 con 0x55dc4ad7e570 2024-03-08T21:04:10.196 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.212+0000 7f0e68e11740 10 client.15768 fetch_fsmap finished waiting for FSMap version 32 2024-03-08T21:04:10.196 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.212+0000 7f0e68e11740 10 client.15768 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-08T21:04:10.196 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.212+0000 7f0e68e11740 20 client.15768 populate_metadata read hostname 'smithi107' 2024-03-08T21:04:10.196 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.212+0000 7f0e68e11740 1 --2- 172.21.15.107:0/208985665 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55dc4ade5b30 0x55dc4ade7f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:04:10.196 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.212+0000 7f0e68e11740 4 client.15768 mds_command: new command op to 34475 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-08T21:04:10.196 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.212+0000 7f0e68e11740 1 -- 172.21.15.107:0/208985665 --> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55dc4acdfc60 con 0x55dc4ade5b30 2024-03-08T21:04:10.196 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.213+0000 7f0e5f400640 1 --2- 172.21.15.107:0/208985665 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55dc4ade5b30 0x55dc4ade7f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:04:10.197 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.213+0000 7f0e5f400640 1 --2- 172.21.15.107:0/208985665 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55dc4ade5b30 0x55dc4ade7f50 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f2d949ad639ae08 server_cookie=6f227643c27c7440 in_seq=0 out_seq=0 2024-03-08T21:04:10.197 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.213+0000 7f0e4cc00640 10 client.15768 ms_handle_connect on v2:172.21.15.149:6838/55290596 2024-03-08T21:04:10.218 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.236+0000 7f0e4cc00640 1 -- 172.21.15.107:0/208985665 <== mds.0 v2:172.21.15.149:6838/55290596 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55dc4acdfc60 con 0x55dc4ade5b30 2024-03-08T21:04:10.218 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.236+0000 7f0e4cc00640 10 client.15768 handle_command_reply: tid=0 2024-03-08T21:04:10.218 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.236+0000 7f0e68e11740 1 client.15768 shutdown 2024-03-08T21:04:10.218 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.236+0000 7f0e68e11740 1 -- 172.21.15.107:0/208985665 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dc4ad7e570 msgr2=0x55dc4ad916a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:04:10.218 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.236+0000 7f0e68e11740 1 --2- 172.21.15.107:0/208985665 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dc4ad7e570 0x55dc4ad916a0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f0e58097f60 tx=0x7f0e5809bd50 comp rx=0 tx=0).stop 2024-03-08T21:04:10.218 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.236+0000 7f0e68e11740 1 -- 172.21.15.107:0/208985665 shutdown_connections 2024-03-08T21:04:10.218 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.236+0000 7f0e68e11740 1 --2- 172.21.15.107:0/208985665 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dc4ad7e570 0x55dc4ad916a0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:04:10.218 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.236+0000 7f0e68e11740 1 --2- 172.21.15.107:0/208985665 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc4ad757a0 0x55dc4ad90f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:04:10.218 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.236+0000 7f0e68e11740 1 --2- 172.21.15.107:0/208985665 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55dc4ade5b30 0x55dc4ade7f50 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:04:10.218 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.236+0000 7f0e68e11740 1 --2- 172.21.15.107:0/208985665 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dc4ad74e80 0x55dc4ad92b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:04:10.218 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.236+0000 7f0e68e11740 1 -- 172.21.15.107:0/208985665 >> 172.21.15.107:0/208985665 conn(0x55dc4ad6f7a0 msgr2=0x55dc4ad89dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:04:10.219 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.236+0000 7f0e4cc00640 0 client.15768 ms_handle_reset on v2:172.21.15.149:6838/55290596 2024-03-08T21:04:10.219 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.236+0000 7f0e68e11740 1 -- 172.21.15.107:0/208985665 shutdown_connections 2024-03-08T21:04:10.219 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.236+0000 7f0e68e11740 1 -- 172.21.15.107:0/208985665 wait complete. 2024-03-08T21:04:10.219 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.237+0000 7f0e68e11740 20 client.15768 trim_cache size 0 max 16384 2024-03-08T21:04:10.219 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:{ 2024-03-08T21:04:10.219 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "message": "", 2024-03-08T21:04:10.219 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "return_code": 0 2024-03-08T21:04:10.219 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:} 2024-03-08T21:04:10.220 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.237+0000 7f0e64c00640 1 -- 172.21.15.107:0/2797162953 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f0e440a2960 msgr2=0x7f0e440a4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:04:10.220 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.237+0000 7f0e64c00640 1 --2- 172.21.15.107:0/2797162953 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f0e440a2960 0x7f0e440a4e20 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x55dc4ad74930 tx=0x7f0e5000c4b0 comp rx=0 tx=0).stop 2024-03-08T21:04:10.220 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.237+0000 7f0e64c00640 1 -- 172.21.15.107:0/2797162953 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0e6010b6c0 msgr2=0x7f0e601998c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:04:10.220 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.237+0000 7f0e64c00640 1 --2- 172.21.15.107:0/2797162953 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0e6010b6c0 0x7f0e601998c0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f0e5800cad0 tx=0x7f0e5800cf90 comp rx=0 tx=0).stop 2024-03-08T21:04:10.220 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.237+0000 7f0e64c00640 1 -- 172.21.15.107:0/2797162953 shutdown_connections 2024-03-08T21:04:10.220 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.237+0000 7f0e64c00640 1 --2- 172.21.15.107:0/2797162953 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f0e440a2960 0x7f0e440a4e20 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:04:10.220 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.237+0000 7f0e64c00640 1 --2- 172.21.15.107:0/2797162953 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0e6010b6c0 0x7f0e601998c0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:04:10.220 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.237+0000 7f0e64c00640 1 --2- 172.21.15.107:0/2797162953 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0e60102bc0 0x7f0e6019fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:04:10.220 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.237+0000 7f0e64c00640 1 --2- 172.21.15.107:0/2797162953 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0e601022a0 0x7f0e6019f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:04:10.221 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.237+0000 7f0e64c00640 1 -- 172.21.15.107:0/2797162953 >> 172.21.15.107:0/2797162953 conn(0x7f0e600fbb00 msgr2=0x7f0e600ffe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:04:10.221 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.238+0000 7f0e64c00640 1 -- 172.21.15.107:0/2797162953 shutdown_connections 2024-03-08T21:04:10.221 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:10.238+0000 7f0e64c00640 1 -- 172.21.15.107:0/2797162953 wait complete. 2024-03-08T21:04:11.876 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:04:11 smithi204 ceph-mon[34935]: pgmap v114: 97 pgs: 97 active+clean; 2.7 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 77 KiB/s wr, 27 op/s 2024-03-08T21:04:11.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:04:11 smithi107 ceph-mon[31381]: pgmap v114: 97 pgs: 97 active+clean; 2.7 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 77 KiB/s wr, 27 op/s 2024-03-08T21:04:12.012 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:04:11 smithi149 ceph-mon[35042]: pgmap v114: 97 pgs: 97 active+clean; 2.7 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 77 KiB/s wr, 27 op/s 2024-03-08T21:04:13.876 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:04:13 smithi204 ceph-mon[34935]: pgmap v115: 97 pgs: 97 active+clean; 2.7 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 77 KiB/s wr, 23 op/s 2024-03-08T21:04:13.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:04:13 smithi107 ceph-mon[31381]: pgmap v115: 97 pgs: 97 active+clean; 2.7 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 77 KiB/s wr, 23 op/s 2024-03-08T21:04:14.012 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:04:13 smithi149 ceph-mon[35042]: pgmap v115: 97 pgs: 97 active+clean; 2.7 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 77 KiB/s wr, 23 op/s 2024-03-08T21:04:15.876 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:04:15 smithi204 ceph-mon[34935]: pgmap v116: 97 pgs: 97 active+clean; 2.7 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 79 KiB/s wr, 30 op/s 2024-03-08T21:04:15.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:04:15 smithi107 ceph-mon[31381]: pgmap v116: 97 pgs: 97 active+clean; 2.7 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 79 KiB/s wr, 30 op/s 2024-03-08T21:04:16.012 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:04:15 smithi149 ceph-mon[35042]: pgmap v116: 97 pgs: 97 active+clean; 2.7 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 79 KiB/s wr, 30 op/s 2024-03-08T21:04:17.876 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:04:17 smithi204 ceph-mon[34935]: pgmap v117: 97 pgs: 97 active+clean; 2.7 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 4.0 KiB/s wr, 11 op/s 2024-03-08T21:04:17.964 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:04:17 smithi107 ceph-mon[31381]: pgmap v117: 97 pgs: 97 active+clean; 2.7 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 4.0 KiB/s wr, 11 op/s 2024-03-08T21:04:18.013 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:04:17 smithi149 ceph-mon[35042]: pgmap v117: 97 pgs: 97 active+clean; 2.7 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 4.0 KiB/s wr, 11 op/s 2024-03-08T21:04:19.876 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:04:19 smithi204 ceph-mon[34935]: pgmap v118: 97 pgs: 97 active+clean; 2.7 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 4.0 KiB/s wr, 11 op/s 2024-03-08T21:04:19.964 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:04:19 smithi107 ceph-mon[31381]: pgmap v118: 97 pgs: 97 active+clean; 2.7 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 4.0 KiB/s wr, 11 op/s 2024-03-08T21:04:20.012 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:04:19 smithi149 ceph-mon[35042]: pgmap v118: 97 pgs: 97 active+clean; 2.7 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 4.0 KiB/s wr, 11 op/s 2024-03-08T21:04:21.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:04:21 smithi107 ceph-mon[31381]: pgmap v119: 97 pgs: 97 active+clean; 2.7 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 4.0 KiB/s wr, 14 op/s 2024-03-08T21:04:22.012 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:04:21 smithi149 ceph-mon[35042]: pgmap v119: 97 pgs: 97 active+clean; 2.7 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 4.0 KiB/s wr, 14 op/s 2024-03-08T21:04:22.125 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:04:21 smithi204 ceph-mon[34935]: pgmap v119: 97 pgs: 97 active+clean; 2.7 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 4.0 KiB/s wr, 14 op/s 2024-03-08T21:04:23.370 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.388+0000 7fa93e000640 1 -- 172.21.15.107:0/2500143058 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa938108620 msgr2=0x7fa9381101f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:04:23.370 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.388+0000 7fa93e000640 1 --2- 172.21.15.107:0/2500143058 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa938108620 0x7fa9381101f0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fa930009ca0 tx=0x7fa93002c0f0 comp rx=0 tx=0).stop 2024-03-08T21:04:23.370 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.388+0000 7fa93e000640 1 -- 172.21.15.107:0/2500143058 shutdown_connections 2024-03-08T21:04:23.370 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.388+0000 7fa93e000640 1 --2- 172.21.15.107:0/2500143058 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa938108620 0x7fa9381101f0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:04:23.370 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.388+0000 7fa93e000640 1 --2- 172.21.15.107:0/2500143058 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa938103f00 0x7fa938107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:04:23.370 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.388+0000 7fa93e000640 1 --2- 172.21.15.107:0/2500143058 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa938103550 0x7fa938103930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:04:23.370 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.388+0000 7fa93e000640 1 -- 172.21.15.107:0/2500143058 >> 172.21.15.107:0/2500143058 conn(0x7fa9380fedd0 msgr2=0x7fa9381011f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:04:23.370 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.388+0000 7fa93e000640 1 -- 172.21.15.107:0/2500143058 shutdown_connections 2024-03-08T21:04:23.370 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.388+0000 7fa93e000640 1 -- 172.21.15.107:0/2500143058 wait complete. 2024-03-08T21:04:23.373 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.390+0000 7fa93e000640 1 Processor -- start 2024-03-08T21:04:23.373 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.390+0000 7fa93e000640 1 -- start start 2024-03-08T21:04:23.373 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.390+0000 7fa93e000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa938103550 0x7fa93819f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:04:23.373 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.390+0000 7fa93e000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa938108620 0x7fa93819fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:04:23.373 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.390+0000 7fa93e000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa938199990 0x7fa938199e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:04:23.373 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.390+0000 7fa93e000640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa938113570 con 0x7fa938108620 2024-03-08T21:04:23.373 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.390+0000 7fa93e000640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9381133f0 con 0x7fa938199990 2024-03-08T21:04:23.373 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.390+0000 7fa93e000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9381136f0 con 0x7fa938103550 2024-03-08T21:04:23.373 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.391+0000 7fa937e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa938108620 0x7fa93819fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:04:23.374 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.391+0000 7fa937e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa938108620 0x7fa93819fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:42806/0 (socket says 172.21.15.107:42806) 2024-03-08T21:04:23.374 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.391+0000 7fa937e00640 1 -- 172.21.15.107:0/778580945 learned_addr learned my addr 172.21.15.107:0/778580945 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:04:23.374 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.391+0000 7fa93cc00640 1 --2- 172.21.15.107:0/778580945 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa938103550 0x7fa93819f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:04:23.374 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.391+0000 7fa93d600640 1 --2- 172.21.15.107:0/778580945 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa938199990 0x7fa938199e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:04:23.374 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.391+0000 7fa937e00640 1 -- 172.21.15.107:0/778580945 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa938103550 msgr2=0x7fa93819f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:04:23.374 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.391+0000 7fa937e00640 1 --2- 172.21.15.107:0/778580945 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa938103550 0x7fa93819f7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:04:23.374 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.391+0000 7fa937e00640 1 -- 172.21.15.107:0/778580945 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa938199990 msgr2=0x7fa938199e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:04:23.374 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.391+0000 7fa937e00640 1 --2- 172.21.15.107:0/778580945 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa938199990 0x7fa938199e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:04:23.374 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.391+0000 7fa937e00640 1 -- 172.21.15.107:0/778580945 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa93819a4b0 con 0x7fa938108620 2024-03-08T21:04:23.374 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.391+0000 7fa93cc00640 1 --2- 172.21.15.107:0/778580945 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa938103550 0x7fa93819f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T21:04:23.374 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.391+0000 7fa937e00640 1 --2- 172.21.15.107:0/778580945 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa938108620 0x7fa93819fd20 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fa92400d770 tx=0x7fa92400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:04:23.374 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.392+0000 7fa935600640 1 -- 172.21.15.107:0/778580945 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa924017070 con 0x7fa938108620 2024-03-08T21:04:23.374 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.392+0000 7fa935600640 1 -- 172.21.15.107:0/778580945 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa924009c30 con 0x7fa938108620 2024-03-08T21:04:23.375 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.392+0000 7fa935600640 1 -- 172.21.15.107:0/778580945 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa92400f620 con 0x7fa938108620 2024-03-08T21:04:23.375 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.392+0000 7fa93e000640 1 -- 172.21.15.107:0/778580945 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa93819a7a0 con 0x7fa938108620 2024-03-08T21:04:23.375 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.392+0000 7fa93e000640 1 -- 172.21.15.107:0/778580945 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9381a69a0 con 0x7fa938108620 2024-03-08T21:04:23.376 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.395+0000 7fa935600640 1 -- 172.21.15.107:0/778580945 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) v1 ==== 137422+0+0 (secure 0 0 0) 0x7fa92401a070 con 0x7fa938108620 2024-03-08T21:04:23.376 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.395+0000 7fa942389740 1 Processor -- start 2024-03-08T21:04:23.377 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.396+0000 7fa942389740 1 -- start start 2024-03-08T21:04:23.377 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.396+0000 7fa942389740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f94832fe80 0x55f948330260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:04:23.378 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.396+0000 7fa942389740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f9483307a0 0x55f948339030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:04:23.378 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.396+0000 7fa942389740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f948339570 0x55f94833b960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:04:23.378 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.396+0000 7fa942389740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9482933d0 con 0x55f948339570 2024-03-08T21:04:23.378 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.396+0000 7fa942389740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f94823c6b0 con 0x55f94832fe80 2024-03-08T21:04:23.378 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.396+0000 7fa942389740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f94828f470 con 0x55f9483307a0 2024-03-08T21:04:23.378 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.396+0000 7fa937e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f948339570 0x55f94833b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:04:23.378 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.396+0000 7fa937e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f948339570 0x55f94833b960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:42808/0 (socket says 172.21.15.107:42808) 2024-03-08T21:04:23.378 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.396+0000 7fa937e00640 1 -- 172.21.15.107:0/2735693946 learned_addr learned my addr 172.21.15.107:0/2735693946 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:04:23.378 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.397+0000 7fa93d600640 1 --2- 172.21.15.107:0/2735693946 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f94832fe80 0x55f948330260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:04:23.378 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.397+0000 7fa93cc00640 1 --2- 172.21.15.107:0/2735693946 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f9483307a0 0x55f948339030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:04:23.380 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.397+0000 7fa937e00640 1 -- 172.21.15.107:0/2735693946 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f9483307a0 msgr2=0x55f948339030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:04:23.380 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.397+0000 7fa937e00640 1 --2- 172.21.15.107:0/2735693946 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f9483307a0 0x55f948339030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:04:23.380 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.397+0000 7fa937e00640 1 -- 172.21.15.107:0/2735693946 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f94832fe80 msgr2=0x55f948330260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:04:23.380 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.397+0000 7fa937e00640 1 --2- 172.21.15.107:0/2735693946 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f94832fe80 0x55f948330260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:04:23.380 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.397+0000 7fa937e00640 1 -- 172.21.15.107:0/2735693946 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f948262b50 con 0x55f948339570 2024-03-08T21:04:23.381 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.397+0000 7fa937e00640 1 --2- 172.21.15.107:0/2735693946 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f948339570 0x55f94833b960 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fa9240cc050 tx=0x7fa9240d74f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8817e6ae91f65115 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:04:23.381 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.398+0000 7fa929c00640 1 -- 172.21.15.107:0/2735693946 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9240e2070 con 0x55f948339570 2024-03-08T21:04:23.381 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.398+0000 7fa929c00640 1 -- 172.21.15.107:0/2735693946 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa9240d7e10 con 0x55f948339570 2024-03-08T21:04:23.381 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.398+0000 7fa929c00640 1 -- 172.21.15.107:0/2735693946 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9240dc760 con 0x55f948339570 2024-03-08T21:04:23.381 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.398+0000 7fa935600640 1 --2- 172.21.15.107:0/778580945 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7fa9200a2960 0x7fa9200a4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:04:23.381 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.398+0000 7fa935600640 1 -- 172.21.15.107:0/778580945 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fa9240cea30 con 0x7fa938108620 2024-03-08T21:04:23.381 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.398+0000 7fa93d600640 1 --2- 172.21.15.107:0/778580945 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7fa9200a2960 0x7fa9200a4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:04:23.381 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.399+0000 7fa942389740 1 -- 172.21.15.107:0/2735693946 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f948339570 msgr2=0x55f94833b960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:04:23.381 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.399+0000 7fa942389740 1 --2- 172.21.15.107:0/2735693946 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f948339570 0x55f94833b960 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fa9240cc050 tx=0x7fa9240d74f0 comp rx=0 tx=0).stop 2024-03-08T21:04:23.381 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.399+0000 7fa942389740 1 -- 172.21.15.107:0/2735693946 shutdown_connections 2024-03-08T21:04:23.381 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.399+0000 7fa942389740 1 --2- 172.21.15.107:0/2735693946 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f948339570 0x55f94833b960 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:04:23.381 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.399+0000 7fa942389740 1 --2- 172.21.15.107:0/2735693946 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f9483307a0 0x55f948339030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:04:23.381 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.399+0000 7fa942389740 1 --2- 172.21.15.107:0/2735693946 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f94832fe80 0x55f948330260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:04:23.381 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.399+0000 7fa942389740 1 -- 172.21.15.107:0/2735693946 >> 172.21.15.107:0/2735693946 conn(0x55f94832a7a0 msgr2=0x55f94832ab80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:04:23.381 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.399+0000 7fa942389740 1 -- 172.21.15.107:0/2735693946 shutdown_connections 2024-03-08T21:04:23.382 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.399+0000 7fa942389740 1 -- 172.21.15.107:0/2735693946 wait complete. 2024-03-08T21:04:23.382 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.399+0000 7fa942389740 1 -- start start 2024-03-08T21:04:23.382 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.399+0000 7fa93d600640 1 --2- 172.21.15.107:0/778580945 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7fa9200a2960 0x7fa9200a4e20 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fa930006010 tx=0x7fa930005f80 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:04:23.385 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.400+0000 7fa942389740 1 Processor -- start 2024-03-08T21:04:23.385 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.400+0000 7fa942389740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f94832fe80 0x55f948338ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:04:23.385 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.400+0000 7fa942389740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f9483307a0 0x55f948336250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:04:23.385 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.400+0000 7fa942389740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f948339570 0x55f9483367b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:04:23.385 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.400+0000 7fa942389740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f94833c1a0 con 0x55f9483307a0 2024-03-08T21:04:23.385 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.400+0000 7fa942389740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f94824f9f0 con 0x55f948339570 2024-03-08T21:04:23.385 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.400+0000 7fa942389740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f94833c320 con 0x55f94832fe80 2024-03-08T21:04:23.385 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.400+0000 7fa942389740 10 client.0 _ll_register_callbacks cb 0x55f947fa7a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T21:04:23.386 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.401+0000 7fa937e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f9483307a0 0x55f948336250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:04:23.386 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.401+0000 7fa937e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f9483307a0 0x55f948336250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:42818/0 (socket says 172.21.15.107:42818) 2024-03-08T21:04:23.386 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.401+0000 7fa937e00640 1 -- 172.21.15.107:0/3112754471 learned_addr learned my addr 172.21.15.107:0/3112754471 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:04:23.386 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.401+0000 7fa93cc00640 1 --2- 172.21.15.107:0/3112754471 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f94832fe80 0x55f948338ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:04:23.386 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.401+0000 7fa93d600640 1 --2- 172.21.15.107:0/3112754471 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f948339570 0x55f9483367b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:04:23.386 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.401+0000 7fa937e00640 1 -- 172.21.15.107:0/3112754471 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f94832fe80 msgr2=0x55f948338ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:04:23.386 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.401+0000 7fa937e00640 1 --2- 172.21.15.107:0/3112754471 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f94832fe80 0x55f948338ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:04:23.386 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.401+0000 7fa937e00640 1 -- 172.21.15.107:0/3112754471 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f948339570 msgr2=0x55f9483367b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:04:23.386 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.401+0000 7fa937e00640 1 --2- 172.21.15.107:0/3112754471 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f948339570 0x55f9483367b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:04:23.386 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.401+0000 7fa937e00640 1 -- 172.21.15.107:0/3112754471 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f947fa8210 con 0x55f9483307a0 2024-03-08T21:04:23.386 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.401+0000 7fa937e00640 1 --2- 172.21.15.107:0/3112754471 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f9483307a0 0x55f948336250 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fa9240979f0 tx=0x7fa9240cecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb18533be7a386a1 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:04:23.386 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.401+0000 7fa91e000640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-03-08T21:04:23.387 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.402+0000 7fa91e000640 1 -- 172.21.15.107:0/3112754471 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa924021030 con 0x55f9483307a0 2024-03-08T21:04:23.387 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.402+0000 7fa91e000640 1 -- 172.21.15.107:0/3112754471 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fa924097690 con 0x55f9483307a0 2024-03-08T21:04:23.387 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.402+0000 7fa942389740 1 -- 172.21.15.107:0/3112754471 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa9240d7e10 con 0x55f9483307a0 2024-03-08T21:04:23.387 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.402+0000 7fa91e000640 1 -- 172.21.15.107:0/3112754471 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa9240972c0 con 0x55f9483307a0 2024-03-08T21:04:23.387 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.402+0000 7fa942389740 10 client.15780 fetch_fsmap learned FSMap version 32 2024-03-08T21:04:23.387 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.402+0000 7fa942389740 1 -- 172.21.15.107:0/3112754471 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55f948074620 con 0x55f9483307a0 2024-03-08T21:04:23.387 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.402+0000 7fa91e000640 1 -- 172.21.15.107:0/3112754471 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa9240d7e10 con 0x55f9483307a0 2024-03-08T21:04:23.387 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.402+0000 7fa91e000640 1 -- 172.21.15.107:0/3112754471 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 32) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fa9240cc390 con 0x55f9483307a0 2024-03-08T21:04:23.387 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.403+0000 7fa942389740 10 client.15780 fetch_fsmap finished waiting for FSMap version 32 2024-03-08T21:04:23.387 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.403+0000 7fa942389740 10 client.15780 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-08T21:04:23.387 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.403+0000 7fa942389740 20 client.15780 populate_metadata read hostname 'smithi107' 2024-03-08T21:04:23.387 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.403+0000 7fa942389740 1 --2- 172.21.15.107:0/3112754471 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55f948349290 0x55f94834b680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:04:23.387 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.403+0000 7fa942389740 4 client.15780 mds_command: new command op to 34475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-08T21:04:23.387 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.403+0000 7fa942389740 1 -- 172.21.15.107:0/3112754471 --> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f9482a0910 con 0x55f948349290 2024-03-08T21:04:23.388 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.403+0000 7fa93cc00640 1 --2- 172.21.15.107:0/3112754471 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55f948349290 0x55f94834b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:04:23.388 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.404+0000 7fa93cc00640 1 --2- 172.21.15.107:0/3112754471 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55f948349290 0x55f94834b680 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e649692e220eafa6 server_cookie=8a792e6c09cb4a8 in_seq=0 out_seq=0 2024-03-08T21:04:23.388 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.404+0000 7fa91e000640 10 client.15780 ms_handle_connect on v2:172.21.15.149:6838/55290596 2024-03-08T21:04:23.389 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.406+0000 7fa91e000640 1 -- 172.21.15.107:0/3112754471 <== mds.0 v2:172.21.15.149:6838/55290596 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f9482a0910 con 0x55f948349290 2024-03-08T21:04:23.389 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.406+0000 7fa91e000640 10 client.15780 handle_command_reply: tid=0 2024-03-08T21:04:23.389 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.406+0000 7fa942389740 1 client.15780 shutdown 2024-03-08T21:04:23.389 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.406+0000 7fa942389740 1 -- 172.21.15.107:0/3112754471 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f9483307a0 msgr2=0x55f948336250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:04:23.389 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.406+0000 7fa942389740 1 --2- 172.21.15.107:0/3112754471 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f9483307a0 0x55f948336250 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fa9240979f0 tx=0x7fa9240cecb0 comp rx=0 tx=0).stop 2024-03-08T21:04:23.389 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.406+0000 7fa942389740 1 -- 172.21.15.107:0/3112754471 shutdown_connections 2024-03-08T21:04:23.389 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.406+0000 7fa942389740 1 --2- 172.21.15.107:0/3112754471 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55f948349290 0x55f94834b680 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:04:23.389 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.406+0000 7fa942389740 1 --2- 172.21.15.107:0/3112754471 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f948339570 0x55f9483367b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:04:23.389 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.406+0000 7fa942389740 1 --2- 172.21.15.107:0/3112754471 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f9483307a0 0x55f948336250 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:04:23.389 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.406+0000 7fa942389740 1 --2- 172.21.15.107:0/3112754471 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f94832fe80 0x55f948338ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:04:23.389 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.406+0000 7fa91e000640 0 client.15780 ms_handle_reset on v2:172.21.15.149:6838/55290596 2024-03-08T21:04:23.389 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.406+0000 7fa942389740 1 -- 172.21.15.107:0/3112754471 >> 172.21.15.107:0/3112754471 conn(0x55f94832a7a0 msgr2=0x55f94832c4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:04:23.390 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.406+0000 7fa942389740 1 -- 172.21.15.107:0/3112754471 shutdown_connections 2024-03-08T21:04:23.390 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.406+0000 7fa942389740 1 -- 172.21.15.107:0/3112754471 wait complete. 2024-03-08T21:04:23.390 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.406+0000 7fa942389740 20 client.15780 trim_cache size 0 max 16384 2024-03-08T21:04:23.408 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.426+0000 7fa942389740 1 Processor -- start 2024-03-08T21:04:23.408 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.427+0000 7fa942389740 1 -- start start 2024-03-08T21:04:23.408 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.427+0000 7fa942389740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f94832fe80 0x55f948338980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:04:23.408 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.427+0000 7fa942389740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f9483307a0 0x55f948338ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:04:23.408 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.427+0000 7fa942389740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f948339570 0x55f9483334b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:04:23.409 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.427+0000 7fa942389740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9381aed60 con 0x55f94832fe80 2024-03-08T21:04:23.409 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.427+0000 7fa942389740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9381b06e0 con 0x55f948339570 2024-03-08T21:04:23.409 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.427+0000 7fa942389740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9381af3c0 con 0x55f9483307a0 2024-03-08T21:04:23.409 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.427+0000 7fa93cc00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f94832fe80 0x55f948338980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:04:23.409 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.427+0000 7fa93cc00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f94832fe80 0x55f948338980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:42828/0 (socket says 172.21.15.107:42828) 2024-03-08T21:04:23.409 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.427+0000 7fa93cc00640 1 -- 172.21.15.107:0/3666118039 learned_addr learned my addr 172.21.15.107:0/3666118039 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:04:23.411 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.428+0000 7fa93cc00640 1 -- 172.21.15.107:0/3666118039 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f9483307a0 msgr2=0x55f948338ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:04:23.411 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.428+0000 7fa937e00640 1 --2- 172.21.15.107:0/3666118039 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f9483307a0 0x55f948338ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:04:23.411 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.428+0000 7fa93cc00640 1 --2- 172.21.15.107:0/3666118039 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f9483307a0 0x55f948338ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:04:23.411 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.428+0000 7fa93d600640 1 --2- 172.21.15.107:0/3666118039 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f948339570 0x55f9483334b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:04:23.411 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.428+0000 7fa93cc00640 1 -- 172.21.15.107:0/3666118039 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f948339570 msgr2=0x55f9483334b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:04:23.411 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.428+0000 7fa93cc00640 1 --2- 172.21.15.107:0/3666118039 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f948339570 0x55f9483334b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:04:23.411 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.428+0000 7fa93cc00640 1 -- 172.21.15.107:0/3666118039 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f948335200 con 0x55f94832fe80 2024-03-08T21:04:23.411 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.428+0000 7fa937e00640 1 --2- 172.21.15.107:0/3666118039 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f9483307a0 0x55f948338ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T21:04:23.411 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.428+0000 7fa93d600640 1 --2- 172.21.15.107:0/3666118039 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f948339570 0x55f9483334b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T21:04:23.411 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.428+0000 7fa93cc00640 1 --2- 172.21.15.107:0/3666118039 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f94832fe80 0x55f948338980 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x55f94832f930 tx=0x7fa92c012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55f0149631f055c1 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:04:23.412 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.428+0000 7fa929200640 1 -- 172.21.15.107:0/3666118039 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa92c019d70 con 0x55f94832fe80 2024-03-08T21:04:23.412 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.429+0000 7fa929200640 1 -- 172.21.15.107:0/3666118039 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa92c0196d0 con 0x55f94832fe80 2024-03-08T21:04:23.412 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.429+0000 7fa929200640 1 -- 172.21.15.107:0/3666118039 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa92c021b70 con 0x55f94832fe80 2024-03-08T21:04:23.412 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.429+0000 7fa942389740 1 -- 172.21.15.107:0/3666118039 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f94832fe80 msgr2=0x55f948338980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:04:23.412 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.429+0000 7fa942389740 1 --2- 172.21.15.107:0/3666118039 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f94832fe80 0x55f948338980 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x55f94832f930 tx=0x7fa92c012820 comp rx=0 tx=0).stop 2024-03-08T21:04:23.412 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.429+0000 7fa942389740 1 -- 172.21.15.107:0/3666118039 shutdown_connections 2024-03-08T21:04:23.412 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.429+0000 7fa942389740 1 --2- 172.21.15.107:0/3666118039 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f948339570 0x55f9483334b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:04:23.412 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.429+0000 7fa942389740 1 --2- 172.21.15.107:0/3666118039 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f9483307a0 0x55f948338ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:04:23.412 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.429+0000 7fa942389740 1 --2- 172.21.15.107:0/3666118039 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f94832fe80 0x55f948338980 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:04:23.412 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.429+0000 7fa942389740 1 -- 172.21.15.107:0/3666118039 >> 172.21.15.107:0/3666118039 conn(0x55f94832a7a0 msgr2=0x55f948329800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:04:23.412 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.430+0000 7fa942389740 1 -- 172.21.15.107:0/3666118039 shutdown_connections 2024-03-08T21:04:23.412 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.430+0000 7fa942389740 1 -- 172.21.15.107:0/3666118039 wait complete. 2024-03-08T21:04:23.412 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.430+0000 7fa942389740 1 -- start start 2024-03-08T21:04:23.417 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.430+0000 7fa942389740 1 Processor -- start 2024-03-08T21:04:23.417 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.431+0000 7fa942389740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f94832fe80 0x55f94834c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:04:23.417 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.431+0000 7fa942389740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f9483307a0 0x55f94834cee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:04:23.418 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.431+0000 7fa942389740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f948339570 0x55f9483993b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:04:23.418 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.431+0000 7fa942389740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9381b0080 con 0x55f948339570 2024-03-08T21:04:23.418 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.431+0000 7fa942389740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9381afa20 con 0x55f9483307a0 2024-03-08T21:04:23.418 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.431+0000 7fa942389740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9381ad000 con 0x55f94832fe80 2024-03-08T21:04:23.418 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.431+0000 7fa942389740 10 client.0 _ll_register_callbacks cb 0x7fa9240d2490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T21:04:23.418 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.431+0000 7fa93d600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f948339570 0x55f9483993b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:04:23.419 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.431+0000 7fa93d600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f948339570 0x55f9483993b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:42844/0 (socket says 172.21.15.107:42844) 2024-03-08T21:04:23.419 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.431+0000 7fa93d600640 1 -- 172.21.15.107:0/3333770255 learned_addr learned my addr 172.21.15.107:0/3333770255 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:04:23.419 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.431+0000 7fa93cc00640 1 --2- 172.21.15.107:0/3333770255 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f94832fe80 0x55f94834c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:04:23.419 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.431+0000 7fa937e00640 1 --2- 172.21.15.107:0/3333770255 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f9483307a0 0x55f94834cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:04:23.419 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.432+0000 7fa93d600640 1 -- 172.21.15.107:0/3333770255 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f94832fe80 msgr2=0x55f94834c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:04:23.419 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.432+0000 7fa93d600640 1 --2- 172.21.15.107:0/3333770255 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f94832fe80 0x55f94834c9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:04:23.420 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.432+0000 7fa93d600640 1 -- 172.21.15.107:0/3333770255 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f9483307a0 msgr2=0x55f94834cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:04:23.420 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.432+0000 7fa93d600640 1 --2- 172.21.15.107:0/3333770255 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f9483307a0 0x55f94834cee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:04:23.420 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.432+0000 7fa93d600640 1 -- 172.21.15.107:0/3333770255 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f948336b20 con 0x55f948339570 2024-03-08T21:04:23.420 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.432+0000 7fa93cc00640 1 --2- 172.21.15.107:0/3333770255 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f94832fe80 0x55f94834c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T21:04:23.420 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.432+0000 7fa93d600640 1 --2- 172.21.15.107:0/3333770255 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f948339570 0x55f9483993b0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fa930009ca0 tx=0x7fa93002c0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4505295417a53bad server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:04:23.420 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.432+0000 7fa91e000640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-03-08T21:04:23.421 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.432+0000 7fa91e000640 1 -- 172.21.15.107:0/3333770255 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa930050030 con 0x55f948339570 2024-03-08T21:04:23.421 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.433+0000 7fa91e000640 1 -- 172.21.15.107:0/3333770255 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fa930004040 con 0x55f948339570 2024-03-08T21:04:23.421 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.433+0000 7fa942389740 1 -- 172.21.15.107:0/3333770255 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa92c0196d0 con 0x55f948339570 2024-03-08T21:04:23.421 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.433+0000 7fa91e000640 1 -- 172.21.15.107:0/3333770255 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa930039040 con 0x55f948339570 2024-03-08T21:04:23.421 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.433+0000 7fa942389740 10 client.15786 fetch_fsmap learned FSMap version 32 2024-03-08T21:04:23.421 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.433+0000 7fa942389740 1 -- 172.21.15.107:0/3333770255 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55f9483a0b30 con 0x55f948339570 2024-03-08T21:04:23.422 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.433+0000 7fa91e000640 1 -- 172.21.15.107:0/3333770255 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa92c0196d0 con 0x55f948339570 2024-03-08T21:04:23.422 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.433+0000 7fa91e000640 1 -- 172.21.15.107:0/3333770255 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 32) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fa9300347f0 con 0x55f948339570 2024-03-08T21:04:23.422 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.434+0000 7fa942389740 10 client.15786 fetch_fsmap finished waiting for FSMap version 32 2024-03-08T21:04:23.422 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.434+0000 7fa942389740 10 client.15786 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-08T21:04:23.422 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.434+0000 7fa942389740 20 client.15786 populate_metadata read hostname 'smithi107' 2024-03-08T21:04:23.422 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.434+0000 7fa942389740 1 --2- 172.21.15.107:0/3333770255 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55f9483a12c0 0x55f9483a36e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:04:23.423 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.434+0000 7fa942389740 4 client.15786 mds_command: new command op to 34475 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-08T21:04:23.423 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.434+0000 7fa942389740 1 -- 172.21.15.107:0/3333770255 --> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55f94829ac60 con 0x55f9483a12c0 2024-03-08T21:04:23.423 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.434+0000 7fa93cc00640 1 --2- 172.21.15.107:0/3333770255 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55f9483a12c0 0x55f9483a36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:04:23.423 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.435+0000 7fa91e000640 10 client.15786 ms_handle_connect on v2:172.21.15.149:6838/55290596 2024-03-08T21:04:23.423 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.435+0000 7fa93cc00640 1 --2- 172.21.15.107:0/3333770255 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55f9483a12c0 0x55f9483a36e0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=504acbfe7c038c06 server_cookie=9669d33ba1b0e787 in_seq=0 out_seq=0 2024-03-08T21:04:23.429 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.446+0000 7fa91e000640 1 -- 172.21.15.107:0/3333770255 <== mds.0 v2:172.21.15.149:6838/55290596 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55f94829ac60 con 0x55f9483a12c0 2024-03-08T21:04:23.429 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.446+0000 7fa91e000640 10 client.15786 handle_command_reply: tid=0 2024-03-08T21:04:23.429 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.446+0000 7fa942389740 1 client.15786 shutdown 2024-03-08T21:04:23.429 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.446+0000 7fa942389740 1 -- 172.21.15.107:0/3333770255 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f948339570 msgr2=0x55f9483993b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:04:23.430 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.446+0000 7fa942389740 1 --2- 172.21.15.107:0/3333770255 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f948339570 0x55f9483993b0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fa930009ca0 tx=0x7fa93002c0f0 comp rx=0 tx=0).stop 2024-03-08T21:04:23.430 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.447+0000 7fa942389740 1 -- 172.21.15.107:0/3333770255 shutdown_connections 2024-03-08T21:04:23.430 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.447+0000 7fa942389740 1 --2- 172.21.15.107:0/3333770255 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f948339570 0x55f9483993b0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:04:23.430 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.447+0000 7fa942389740 1 --2- 172.21.15.107:0/3333770255 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55f9483a12c0 0x55f9483a36e0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:04:23.430 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.447+0000 7fa942389740 1 --2- 172.21.15.107:0/3333770255 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f9483307a0 0x55f94834cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:04:23.431 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.447+0000 7fa942389740 1 --2- 172.21.15.107:0/3333770255 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f94832fe80 0x55f94834c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:04:23.431 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.447+0000 7fa91e000640 0 client.15786 ms_handle_reset on v2:172.21.15.149:6838/55290596 2024-03-08T21:04:23.431 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.447+0000 7fa942389740 1 -- 172.21.15.107:0/3333770255 >> 172.21.15.107:0/3333770255 conn(0x55f94832a7a0 msgr2=0x55f948344570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:04:23.431 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.447+0000 7fa942389740 1 -- 172.21.15.107:0/3333770255 shutdown_connections 2024-03-08T21:04:23.431 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.447+0000 7fa942389740 1 -- 172.21.15.107:0/3333770255 wait complete. 2024-03-08T21:04:23.431 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.447+0000 7fa942389740 20 client.15786 trim_cache size 0 max 16384 2024-03-08T21:04:23.432 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:{ 2024-03-08T21:04:23.432 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "message": "", 2024-03-08T21:04:23.432 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "return_code": 0 2024-03-08T21:04:23.432 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:} 2024-03-08T21:04:23.434 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.448+0000 7fa93e000640 1 -- 172.21.15.107:0/778580945 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7fa9200a2960 msgr2=0x7fa9200a4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:04:23.434 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.448+0000 7fa93e000640 1 --2- 172.21.15.107:0/778580945 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7fa9200a2960 0x7fa9200a4e20 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fa930006010 tx=0x7fa930005f80 comp rx=0 tx=0).stop 2024-03-08T21:04:23.434 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.448+0000 7fa93e000640 1 -- 172.21.15.107:0/778580945 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa938108620 msgr2=0x7fa93819fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:04:23.434 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.448+0000 7fa93e000640 1 --2- 172.21.15.107:0/778580945 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa938108620 0x7fa93819fd20 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fa92400d770 tx=0x7fa92400dc30 comp rx=0 tx=0).stop 2024-03-08T21:04:23.434 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.448+0000 7fa93e000640 1 -- 172.21.15.107:0/778580945 shutdown_connections 2024-03-08T21:04:23.435 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.448+0000 7fa93e000640 1 --2- 172.21.15.107:0/778580945 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7fa9200a2960 0x7fa9200a4e20 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:04:23.435 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.448+0000 7fa93e000640 1 --2- 172.21.15.107:0/778580945 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa938199990 0x7fa938199e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:04:23.435 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.448+0000 7fa93e000640 1 --2- 172.21.15.107:0/778580945 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa938108620 0x7fa93819fd20 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:04:23.435 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.448+0000 7fa93e000640 1 --2- 172.21.15.107:0/778580945 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa938103550 0x7fa93819f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:04:23.435 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.448+0000 7fa93e000640 1 -- 172.21.15.107:0/778580945 >> 172.21.15.107:0/778580945 conn(0x7fa9380fedd0 msgr2=0x7fa938101110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:04:23.435 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.448+0000 7fa93e000640 1 -- 172.21.15.107:0/778580945 shutdown_connections 2024-03-08T21:04:23.435 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:23.448+0000 7fa93e000640 1 -- 172.21.15.107:0/778580945 wait complete. 2024-03-08T21:04:23.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:04:23 smithi107 ceph-mon[31381]: pgmap v120: 97 pgs: 97 active+clean; 2.7 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 2.3 KiB/s wr, 10 op/s 2024-03-08T21:04:24.012 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:04:23 smithi149 ceph-mon[35042]: pgmap v120: 97 pgs: 97 active+clean; 2.7 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 2.3 KiB/s wr, 10 op/s 2024-03-08T21:04:24.125 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:04:23 smithi204 ceph-mon[34935]: pgmap v120: 97 pgs: 97 active+clean; 2.7 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 2.3 KiB/s wr, 10 op/s 2024-03-08T21:04:25.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:04:25 smithi107 ceph-mon[31381]: pgmap v121: 97 pgs: 97 active+clean; 2.7 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 2.6 KiB/s wr, 15 op/s 2024-03-08T21:04:26.012 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:04:25 smithi149 ceph-mon[35042]: pgmap v121: 97 pgs: 97 active+clean; 2.7 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 2.6 KiB/s wr, 15 op/s 2024-03-08T21:04:26.125 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:04:25 smithi204 ceph-mon[34935]: pgmap v121: 97 pgs: 97 active+clean; 2.7 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 2.6 KiB/s wr, 15 op/s 2024-03-08T21:04:26.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:04:26 smithi107 ceph-mon[31381]: pgmap v122: 97 pgs: 97 active+clean; 2.7 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:04:27.012 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:04:26 smithi149 ceph-mon[35042]: pgmap v122: 97 pgs: 97 active+clean; 2.7 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:04:27.125 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:04:26 smithi204 ceph-mon[34935]: pgmap v122: 97 pgs: 97 active+clean; 2.7 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:04:29.876 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:04:29 smithi204 ceph-mon[34935]: pgmap v123: 97 pgs: 97 active+clean; 2.7 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 255 B/s wr, 9 op/s 2024-03-08T21:04:29.964 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:04:29 smithi107 ceph-mon[31381]: pgmap v123: 97 pgs: 97 active+clean; 2.7 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 255 B/s wr, 9 op/s 2024-03-08T21:04:30.012 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:04:29 smithi149 ceph-mon[35042]: pgmap v123: 97 pgs: 97 active+clean; 2.7 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 255 B/s wr, 9 op/s 2024-03-08T21:04:31.875 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:04:31 smithi204 ceph-mon[34935]: pgmap v124: 97 pgs: 97 active+clean; 2.7 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-08T21:04:31.964 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:04:31 smithi107 ceph-mon[31381]: pgmap v124: 97 pgs: 97 active+clean; 2.7 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-08T21:04:32.012 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:04:31 smithi149 ceph-mon[35042]: pgmap v124: 97 pgs: 97 active+clean; 2.7 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-08T21:04:33.875 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:04:33 smithi204 ceph-mon[34935]: pgmap v125: 97 pgs: 97 active+clean; 2.7 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:04:33.964 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:04:33 smithi107 ceph-mon[31381]: pgmap v125: 97 pgs: 97 active+clean; 2.7 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:04:34.012 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:04:33 smithi149 ceph-mon[35042]: pgmap v125: 97 pgs: 97 active+clean; 2.7 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:04:35.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:04:35 smithi107 ceph-mon[31381]: pgmap v126: 97 pgs: 97 active+clean; 2.7 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-08T21:04:36.012 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:04:35 smithi149 ceph-mon[35042]: pgmap v126: 97 pgs: 97 active+clean; 2.7 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-08T21:04:36.125 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:04:35 smithi204 ceph-mon[34935]: pgmap v126: 97 pgs: 97 active+clean; 2.7 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-08T21:04:36.578 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.597+0000 7fa1bea00640 1 -- 172.21.15.107:0/230100812 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa1b8101f00 msgr2=0x7fa1b8105f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:04:36.578 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.597+0000 7fa1bea00640 1 --2- 172.21.15.107:0/230100812 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa1b8101f00 0x7fa1b8105f40 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fa1a0009900 tx=0x7fa1a002bc30 comp rx=0 tx=0).stop 2024-03-08T21:04:36.578 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.597+0000 7fa1bea00640 1 -- 172.21.15.107:0/230100812 shutdown_connections 2024-03-08T21:04:36.578 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.597+0000 7fa1bea00640 1 --2- 172.21.15.107:0/230100812 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa1b8106620 0x7fa1b810e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:04:36.578 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.597+0000 7fa1bea00640 1 --2- 172.21.15.107:0/230100812 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa1b8101f00 0x7fa1b8105f40 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:04:36.578 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.597+0000 7fa1bea00640 1 --2- 172.21.15.107:0/230100812 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa1b8101550 0x7fa1b8101930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:04:36.578 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.597+0000 7fa1bea00640 1 -- 172.21.15.107:0/230100812 >> 172.21.15.107:0/230100812 conn(0x7fa1b80fcdb0 msgr2=0x7fa1b80ff1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:04:36.578 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.597+0000 7fa1bea00640 1 -- 172.21.15.107:0/230100812 shutdown_connections 2024-03-08T21:04:36.579 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.598+0000 7fa1bea00640 1 -- 172.21.15.107:0/230100812 wait complete. 2024-03-08T21:04:36.581 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.598+0000 7fa1bea00640 1 Processor -- start 2024-03-08T21:04:36.581 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.599+0000 7fa1bea00640 1 -- start start 2024-03-08T21:04:36.581 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.599+0000 7fa1bea00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa1b8101550 0x7fa1b80735f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:04:36.581 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.599+0000 7fa1bea00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa1b8106620 0x7fa1b8073b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:04:36.581 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.599+0000 7fa1bea00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa1b8074070 0x7fa1b81a5be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:04:36.581 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.599+0000 7fa1bea00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1b81115b0 con 0x7fa1b8074070 2024-03-08T21:04:36.581 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.599+0000 7fa1bea00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1b8111430 con 0x7fa1b8101550 2024-03-08T21:04:36.581 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.599+0000 7fa1bea00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1b8111730 con 0x7fa1b8106620 2024-03-08T21:04:36.581 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.599+0000 7fa1bca00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa1b8074070 0x7fa1b81a5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:04:36.582 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.599+0000 7fa1bca00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa1b8074070 0x7fa1b81a5be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:55174/0 (socket says 172.21.15.107:55174) 2024-03-08T21:04:36.582 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.599+0000 7fa1bca00640 1 -- 172.21.15.107:0/3077329998 learned_addr learned my addr 172.21.15.107:0/3077329998 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:04:36.582 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.599+0000 7fa1b7e00640 1 --2- 172.21.15.107:0/3077329998 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa1b8101550 0x7fa1b80735f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:04:36.582 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.599+0000 7fa1af400640 1 --2- 172.21.15.107:0/3077329998 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa1b8106620 0x7fa1b8073b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:04:36.582 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.600+0000 7fa1bca00640 1 -- 172.21.15.107:0/3077329998 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa1b8106620 msgr2=0x7fa1b8073b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:04:36.582 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.600+0000 7fa1bca00640 1 --2- 172.21.15.107:0/3077329998 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa1b8106620 0x7fa1b8073b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:04:36.582 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.600+0000 7fa1bca00640 1 -- 172.21.15.107:0/3077329998 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa1b8101550 msgr2=0x7fa1b80735f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:04:36.582 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.600+0000 7fa1bca00640 1 --2- 172.21.15.107:0/3077329998 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa1b8101550 0x7fa1b80735f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:04:36.582 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.600+0000 7fa1bca00640 1 -- 172.21.15.107:0/3077329998 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1b81a62e0 con 0x7fa1b8074070 2024-03-08T21:04:36.582 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.600+0000 7fa1bca00640 1 --2- 172.21.15.107:0/3077329998 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa1b8074070 0x7fa1b81a5be0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fa1b000cce0 tx=0x7fa1b000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:04:36.582 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.600+0000 7fa1b5600640 1 -- 172.21.15.107:0/3077329998 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1b000f450 con 0x7fa1b8074070 2024-03-08T21:04:36.582 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.600+0000 7fa1bea00640 1 -- 172.21.15.107:0/3077329998 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1b81a65d0 con 0x7fa1b8074070 2024-03-08T21:04:36.582 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.600+0000 7fa1b5600640 1 -- 172.21.15.107:0/3077329998 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa1b000faa0 con 0x7fa1b8074070 2024-03-08T21:04:36.583 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.600+0000 7fa1b5600640 1 -- 172.21.15.107:0/3077329998 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1b0018b70 con 0x7fa1b8074070 2024-03-08T21:04:36.583 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.600+0000 7fa1bea00640 1 -- 172.21.15.107:0/3077329998 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1b81a6af0 con 0x7fa1b8074070 2024-03-08T21:04:36.584 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.604+0000 7fa1c1641740 1 Processor -- start 2024-03-08T21:04:36.584 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.604+0000 7fa1b5600640 1 -- 172.21.15.107:0/3077329998 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) v1 ==== 137422+0+0 (secure 0 0 0) 0x7fa1b0018d10 con 0x7fa1b8074070 2024-03-08T21:04:36.584 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.604+0000 7fa1c1641740 1 -- start start 2024-03-08T21:04:36.585 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.605+0000 7fa1c1641740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5650747a3c80 0x5650747a4060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:04:36.585 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.605+0000 7fa1c1641740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5650747a45a0 0x5650747ace30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:04:36.585 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.605+0000 7fa1c1641740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5650747ad370 0x5650747af760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:04:36.586 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.605+0000 7fa1c1641740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650747073d0 con 0x5650747ad370 2024-03-08T21:04:36.586 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.605+0000 7fa1c1641740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650746b06b0 con 0x5650747a45a0 2024-03-08T21:04:36.586 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.605+0000 7fa1c1641740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x565074703470 con 0x5650747a3c80 2024-03-08T21:04:36.586 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.605+0000 7fa1af400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5650747ad370 0x5650747af760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:04:36.586 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.605+0000 7fa1af400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5650747ad370 0x5650747af760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:55176/0 (socket says 172.21.15.107:55176) 2024-03-08T21:04:36.586 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.605+0000 7fa1af400640 1 -- 172.21.15.107:0/3475547429 learned_addr learned my addr 172.21.15.107:0/3475547429 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:04:36.586 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.605+0000 7fa1af400640 1 --2- 172.21.15.107:0/3475547429 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5650747a3c80 0x5650747a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:04:36.586 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.606+0000 7fa1b7e00640 1 --2- 172.21.15.107:0/3475547429 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5650747a45a0 0x5650747ace30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:04:36.588 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.606+0000 7fa1af400640 1 -- 172.21.15.107:0/3475547429 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5650747a3c80 msgr2=0x5650747a4060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:04:36.588 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.606+0000 7fa1af400640 1 --2- 172.21.15.107:0/3475547429 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5650747a3c80 0x5650747a4060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:04:36.588 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.606+0000 7fa1af400640 1 -- 172.21.15.107:0/3475547429 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5650747a45a0 msgr2=0x5650747ace30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:04:36.588 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.606+0000 7fa1af400640 1 --2- 172.21.15.107:0/3475547429 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5650747a45a0 0x5650747ace30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:04:36.588 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.606+0000 7fa1af400640 1 -- 172.21.15.107:0/3475547429 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5650746d6b50 con 0x5650747ad370 2024-03-08T21:04:36.588 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.606+0000 7fa1b5600640 1 --2- 172.21.15.107:0/3077329998 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7fa19c0a2ac0 0x7fa19c0a4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:04:36.588 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.606+0000 7fa1b5600640 1 -- 172.21.15.107:0/3077329998 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fa1b00cf290 con 0x7fa1b8074070 2024-03-08T21:04:36.588 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.606+0000 7fa1af400640 1 --2- 172.21.15.107:0/3475547429 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5650747ad370 0x5650747af760 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fa1b806a920 tx=0x7fa1a0005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d35a7f4a674f5452 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:04:36.589 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.606+0000 7fa1b7e00640 1 --2- 172.21.15.107:0/3077329998 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7fa19c0a2ac0 0x7fa19c0a4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:04:36.589 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.607+0000 7fa1ad200640 1 -- 172.21.15.107:0/3475547429 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1a0049070 con 0x5650747ad370 2024-03-08T21:04:36.589 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.607+0000 7fa1ad200640 1 -- 172.21.15.107:0/3475547429 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa1a002c9e0 con 0x5650747ad370 2024-03-08T21:04:36.589 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.607+0000 7fa1ad200640 1 -- 172.21.15.107:0/3475547429 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1a00436b0 con 0x5650747ad370 2024-03-08T21:04:36.589 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.608+0000 7fa1c1641740 1 -- 172.21.15.107:0/3475547429 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5650747ad370 msgr2=0x5650747af760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:04:36.589 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.608+0000 7fa1c1641740 1 --2- 172.21.15.107:0/3475547429 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5650747ad370 0x5650747af760 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fa1b806a920 tx=0x7fa1a0005fd0 comp rx=0 tx=0).stop 2024-03-08T21:04:36.589 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.608+0000 7fa1b7e00640 1 --2- 172.21.15.107:0/3077329998 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7fa19c0a2ac0 0x7fa19c0a4f80 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x5650747a3730 tx=0x7fa1a800b4d0 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:04:36.589 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.608+0000 7fa1c1641740 1 -- 172.21.15.107:0/3475547429 shutdown_connections 2024-03-08T21:04:36.589 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.608+0000 7fa1c1641740 1 --2- 172.21.15.107:0/3475547429 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5650747ad370 0x5650747af760 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:04:36.589 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.608+0000 7fa1c1641740 1 --2- 172.21.15.107:0/3475547429 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5650747a45a0 0x5650747ace30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:04:36.589 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.608+0000 7fa1c1641740 1 --2- 172.21.15.107:0/3475547429 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5650747a3c80 0x5650747a4060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:04:36.589 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.608+0000 7fa1c1641740 1 -- 172.21.15.107:0/3475547429 >> 172.21.15.107:0/3475547429 conn(0x56507479e5a0 msgr2=0x56507479e980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:04:36.589 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.608+0000 7fa1c1641740 1 -- 172.21.15.107:0/3475547429 shutdown_connections 2024-03-08T21:04:36.590 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.608+0000 7fa1c1641740 1 -- 172.21.15.107:0/3475547429 wait complete. 2024-03-08T21:04:36.590 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.608+0000 7fa1c1641740 1 -- start start 2024-03-08T21:04:36.593 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.609+0000 7fa1c1641740 1 Processor -- start 2024-03-08T21:04:36.593 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.609+0000 7fa1c1641740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5650747a3c80 0x5650747acd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:04:36.594 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.609+0000 7fa1c1641740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5650747a45a0 0x5650747aa0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:04:36.594 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.609+0000 7fa1c1641740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5650747ad370 0x5650747aa6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:04:36.594 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.609+0000 7fa1c1641740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650747aff40 con 0x5650747a3c80 2024-03-08T21:04:36.594 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.609+0000 7fa1c1641740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650746c39f0 con 0x5650747ad370 2024-03-08T21:04:36.594 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.609+0000 7fa1c1641740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650747b00c0 con 0x5650747a45a0 2024-03-08T21:04:36.594 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.609+0000 7fa1c1641740 10 client.0 _ll_register_callbacks cb 0x56507441ba00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T21:04:36.594 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.609+0000 7fa1b7e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5650747a3c80 0x5650747acd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:04:36.594 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.609+0000 7fa1b7e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5650747a3c80 0x5650747acd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:55186/0 (socket says 172.21.15.107:55186) 2024-03-08T21:04:36.594 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.610+0000 7fa1b7e00640 1 -- 172.21.15.107:0/1201698654 learned_addr learned my addr 172.21.15.107:0/1201698654 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:04:36.594 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.610+0000 7fa1af400640 1 --2- 172.21.15.107:0/1201698654 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5650747a45a0 0x5650747aa0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:04:36.594 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.610+0000 7fa1bca00640 1 --2- 172.21.15.107:0/1201698654 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5650747ad370 0x5650747aa6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:04:36.595 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.610+0000 7fa1b7e00640 1 -- 172.21.15.107:0/1201698654 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5650747a45a0 msgr2=0x5650747aa0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:04:36.595 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.610+0000 7fa1b7e00640 1 --2- 172.21.15.107:0/1201698654 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5650747a45a0 0x5650747aa0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:04:36.595 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.610+0000 7fa1b7e00640 1 -- 172.21.15.107:0/1201698654 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5650747ad370 msgr2=0x5650747aa6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:04:36.595 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.610+0000 7fa1b7e00640 1 --2- 172.21.15.107:0/1201698654 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5650747ad370 0x5650747aa6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:04:36.595 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.610+0000 7fa1b7e00640 1 -- 172.21.15.107:0/1201698654 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56507441c210 con 0x5650747a3c80 2024-03-08T21:04:36.595 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.610+0000 7fa1b7e00640 1 --2- 172.21.15.107:0/1201698654 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5650747a3c80 0x5650747acd40 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fa1a8013fd0 tx=0x7fa1a80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c894ff1639c0c02 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:04:36.595 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.610+0000 7fa1a5600640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-03-08T21:04:36.595 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.611+0000 7fa1a5600640 1 -- 172.21.15.107:0/1201698654 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1a801de60 con 0x5650747a3c80 2024-03-08T21:04:36.595 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.611+0000 7fa1a5600640 1 -- 172.21.15.107:0/1201698654 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fa1a801e820 con 0x5650747a3c80 2024-03-08T21:04:36.595 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.611+0000 7fa1c1641740 1 -- 172.21.15.107:0/1201698654 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa1a002c9e0 con 0x5650747a3c80 2024-03-08T21:04:36.595 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.611+0000 7fa1a5600640 1 -- 172.21.15.107:0/1201698654 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa1a801ea20 con 0x5650747a3c80 2024-03-08T21:04:36.595 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.611+0000 7fa1c1641740 10 client.15798 fetch_fsmap learned FSMap version 32 2024-03-08T21:04:36.595 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.611+0000 7fa1c1641740 1 -- 172.21.15.107:0/1201698654 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x5650744e8620 con 0x5650747a3c80 2024-03-08T21:04:36.596 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.611+0000 7fa1a5600640 1 -- 172.21.15.107:0/1201698654 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa1a002c9e0 con 0x5650747a3c80 2024-03-08T21:04:36.596 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.612+0000 7fa1a5600640 1 -- 172.21.15.107:0/1201698654 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 32) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fa1a80313c0 con 0x5650747a3c80 2024-03-08T21:04:36.596 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.612+0000 7fa1c1641740 10 client.15798 fetch_fsmap finished waiting for FSMap version 32 2024-03-08T21:04:36.596 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.612+0000 7fa1c1641740 10 client.15798 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-08T21:04:36.596 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.612+0000 7fa1c1641740 20 client.15798 populate_metadata read hostname 'smithi107' 2024-03-08T21:04:36.596 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.612+0000 7fa1c1641740 1 --2- 172.21.15.107:0/1201698654 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x5650747bd260 0x5650747bf650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:04:36.596 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.612+0000 7fa1c1641740 4 client.15798 mds_command: new command op to 34475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-08T21:04:36.596 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.612+0000 7fa1c1641740 1 -- 172.21.15.107:0/1201698654 --> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x565074714910 con 0x5650747bd260 2024-03-08T21:04:36.596 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.612+0000 7fa1af400640 1 --2- 172.21.15.107:0/1201698654 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x5650747bd260 0x5650747bf650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:04:36.596 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.613+0000 7fa1af400640 1 --2- 172.21.15.107:0/1201698654 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x5650747bd260 0x5650747bf650 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=71c826254f61f44c server_cookie=ed529938de4c3b0c in_seq=0 out_seq=0 2024-03-08T21:04:36.596 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.613+0000 7fa1a5600640 10 client.15798 ms_handle_connect on v2:172.21.15.149:6838/55290596 2024-03-08T21:04:36.597 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.615+0000 7fa1a5600640 1 -- 172.21.15.107:0/1201698654 <== mds.0 v2:172.21.15.149:6838/55290596 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x565074714910 con 0x5650747bd260 2024-03-08T21:04:36.598 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.615+0000 7fa1a5600640 10 client.15798 handle_command_reply: tid=0 2024-03-08T21:04:36.598 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.615+0000 7fa1c1641740 1 client.15798 shutdown 2024-03-08T21:04:36.598 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.615+0000 7fa1c1641740 1 -- 172.21.15.107:0/1201698654 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5650747a3c80 msgr2=0x5650747acd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:04:36.598 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.615+0000 7fa1c1641740 1 --2- 172.21.15.107:0/1201698654 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5650747a3c80 0x5650747acd40 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fa1a8013fd0 tx=0x7fa1a80046c0 comp rx=0 tx=0).stop 2024-03-08T21:04:36.598 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.615+0000 7fa1c1641740 1 -- 172.21.15.107:0/1201698654 shutdown_connections 2024-03-08T21:04:36.598 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.615+0000 7fa1c1641740 1 --2- 172.21.15.107:0/1201698654 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x5650747bd260 0x5650747bf650 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:04:36.598 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.615+0000 7fa1c1641740 1 --2- 172.21.15.107:0/1201698654 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5650747ad370 0x5650747aa6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:04:36.598 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.615+0000 7fa1c1641740 1 --2- 172.21.15.107:0/1201698654 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5650747a45a0 0x5650747aa0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:04:36.598 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.615+0000 7fa1c1641740 1 --2- 172.21.15.107:0/1201698654 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5650747a3c80 0x5650747acd40 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:04:36.598 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.615+0000 7fa1c1641740 1 -- 172.21.15.107:0/1201698654 >> 172.21.15.107:0/1201698654 conn(0x56507479e5a0 msgr2=0x5650747a02f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:04:36.598 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.615+0000 7fa1a5600640 0 client.15798 ms_handle_reset on v2:172.21.15.149:6838/55290596 2024-03-08T21:04:36.598 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.615+0000 7fa1c1641740 1 -- 172.21.15.107:0/1201698654 shutdown_connections 2024-03-08T21:04:36.599 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.616+0000 7fa1c1641740 1 -- 172.21.15.107:0/1201698654 wait complete. 2024-03-08T21:04:36.599 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.616+0000 7fa1c1641740 20 client.15798 trim_cache size 0 max 16384 2024-03-08T21:04:36.614 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.634+0000 7fa1c1641740 1 Processor -- start 2024-03-08T21:04:36.614 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.634+0000 7fa1c1641740 1 -- start start 2024-03-08T21:04:36.615 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.634+0000 7fa1c1641740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5650747a3c80 0x5650747aa070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:04:36.615 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.634+0000 7fa1c1641740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5650747a45a0 0x5650747ac800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:04:36.616 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.634+0000 7fa1c1641740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5650747ad370 0x5650747acd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:04:36.616 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.634+0000 7fa1c1641740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1b81aed80 con 0x5650747a45a0 2024-03-08T21:04:36.616 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.634+0000 7fa1c1641740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1b81b0700 con 0x5650747a3c80 2024-03-08T21:04:36.616 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.634+0000 7fa1c1641740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1b81af3e0 con 0x5650747ad370 2024-03-08T21:04:36.616 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.635+0000 7fa1af400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5650747a45a0 0x5650747ac800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:04:36.616 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.635+0000 7fa1af400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5650747a45a0 0x5650747ac800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:55200/0 (socket says 172.21.15.107:55200) 2024-03-08T21:04:36.616 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.635+0000 7fa1af400640 1 -- 172.21.15.107:0/1748349259 learned_addr learned my addr 172.21.15.107:0/1748349259 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:04:36.616 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.635+0000 7fa1b7e00640 1 --2- 172.21.15.107:0/1748349259 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5650747a3c80 0x5650747aa070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:04:36.616 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.635+0000 7fa1bca00640 1 --2- 172.21.15.107:0/1748349259 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5650747ad370 0x5650747acd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:04:36.616 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.635+0000 7fa1af400640 1 -- 172.21.15.107:0/1748349259 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5650747ad370 msgr2=0x5650747acd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:04:36.616 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.635+0000 7fa1af400640 1 --2- 172.21.15.107:0/1748349259 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5650747ad370 0x5650747acd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:04:36.616 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.635+0000 7fa1af400640 1 -- 172.21.15.107:0/1748349259 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5650747a3c80 msgr2=0x5650747aa070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:04:36.616 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.635+0000 7fa1af400640 1 --2- 172.21.15.107:0/1748349259 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5650747a3c80 0x5650747aa070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:04:36.617 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.635+0000 7fa1af400640 1 -- 172.21.15.107:0/1748349259 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5650747aace0 con 0x5650747a45a0 2024-03-08T21:04:36.617 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.636+0000 7fa1af400640 1 --2- 172.21.15.107:0/1748349259 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5650747a45a0 0x5650747ac800 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fa1a0009850 tx=0x7fa1a0000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32b5bf7429826ad8 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:04:36.620 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.636+0000 7fa1a7e00640 1 -- 172.21.15.107:0/1748349259 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1a0049040 con 0x5650747a45a0 2024-03-08T21:04:36.620 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.636+0000 7fa1a7e00640 1 -- 172.21.15.107:0/1748349259 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa1a004a040 con 0x5650747a45a0 2024-03-08T21:04:36.620 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.636+0000 7fa1a7e00640 1 -- 172.21.15.107:0/1748349259 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1a0005040 con 0x5650747a45a0 2024-03-08T21:04:36.621 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.637+0000 7fa1c1641740 1 -- 172.21.15.107:0/1748349259 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5650747a45a0 msgr2=0x5650747ac800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:04:36.621 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.637+0000 7fa1c1641740 1 --2- 172.21.15.107:0/1748349259 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5650747a45a0 0x5650747ac800 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fa1a0009850 tx=0x7fa1a0000bc0 comp rx=0 tx=0).stop 2024-03-08T21:04:36.621 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.637+0000 7fa1c1641740 1 -- 172.21.15.107:0/1748349259 shutdown_connections 2024-03-08T21:04:36.621 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.637+0000 7fa1c1641740 1 --2- 172.21.15.107:0/1748349259 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5650747ad370 0x5650747acd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:04:36.621 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.637+0000 7fa1c1641740 1 --2- 172.21.15.107:0/1748349259 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5650747a45a0 0x5650747ac800 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:04:36.621 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.637+0000 7fa1c1641740 1 --2- 172.21.15.107:0/1748349259 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5650747a3c80 0x5650747aa070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:04:36.621 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.637+0000 7fa1c1641740 1 -- 172.21.15.107:0/1748349259 >> 172.21.15.107:0/1748349259 conn(0x56507479e5a0 msgr2=0x5650747a9960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:04:36.621 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.637+0000 7fa1c1641740 1 -- 172.21.15.107:0/1748349259 shutdown_connections 2024-03-08T21:04:36.621 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.637+0000 7fa1c1641740 1 -- 172.21.15.107:0/1748349259 wait complete. 2024-03-08T21:04:36.621 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.637+0000 7fa1c1641740 1 -- start start 2024-03-08T21:04:36.621 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.638+0000 7fa1c1641740 1 Processor -- start 2024-03-08T21:04:36.622 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.638+0000 7fa1c1641740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5650747a3c80 0x5650747b4380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:04:36.622 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.638+0000 7fa1c1641740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5650747a45a0 0x5650747b2720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:04:36.622 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.638+0000 7fa1c1641740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5650747ad370 0x5650747b2c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:04:36.622 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.638+0000 7fa1c1641740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1b81b00a0 con 0x5650747a45a0 2024-03-08T21:04:36.622 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.638+0000 7fa1c1641740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1b81afa40 con 0x5650747ad370 2024-03-08T21:04:36.622 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.638+0000 7fa1c1641740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1b81ad020 con 0x5650747a3c80 2024-03-08T21:04:36.622 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.638+0000 7fa1c1641740 10 client.0 _ll_register_callbacks cb 0x7fa1a0009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T21:04:36.622 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.638+0000 7fa1af400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5650747a45a0 0x5650747b2720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:04:36.622 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.638+0000 7fa1af400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5650747a45a0 0x5650747b2720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:55216/0 (socket says 172.21.15.107:55216) 2024-03-08T21:04:36.622 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.638+0000 7fa1af400640 1 -- 172.21.15.107:0/3615307332 learned_addr learned my addr 172.21.15.107:0/3615307332 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:04:36.622 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.639+0000 7fa1b7e00640 1 --2- 172.21.15.107:0/3615307332 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5650747a3c80 0x5650747b4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:04:36.622 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.639+0000 7fa1af400640 1 -- 172.21.15.107:0/3615307332 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5650747a3c80 msgr2=0x5650747b4380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:04:36.622 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.639+0000 7fa1af400640 1 --2- 172.21.15.107:0/3615307332 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5650747a3c80 0x5650747b4380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:04:36.623 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.639+0000 7fa1af400640 1 -- 172.21.15.107:0/3615307332 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5650747ad370 msgr2=0x5650747b2c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:04:36.623 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.639+0000 7fa1af400640 1 --2- 172.21.15.107:0/3615307332 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5650747ad370 0x5650747b2c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:04:36.623 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.639+0000 7fa1af400640 1 -- 172.21.15.107:0/3615307332 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5650747b31a0 con 0x5650747a45a0 2024-03-08T21:04:36.623 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.639+0000 7fa1b7e00640 1 --2- 172.21.15.107:0/3615307332 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5650747a3c80 0x5650747b4380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T21:04:36.623 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.639+0000 7fa1af400640 1 --2- 172.21.15.107:0/3615307332 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5650747a45a0 0x5650747b2720 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fa1a0001eb0 tx=0x7fa1a002c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36dfae79d2f198da server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:04:36.623 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.639+0000 7fa1a5600640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-03-08T21:04:36.623 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.639+0000 7fa1a5600640 1 -- 172.21.15.107:0/3615307332 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1a0059070 con 0x5650747a45a0 2024-03-08T21:04:36.623 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.640+0000 7fa1c1641740 1 -- 172.21.15.107:0/3615307332 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa1a004a040 con 0x5650747a45a0 2024-03-08T21:04:36.623 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.640+0000 7fa1a5600640 1 -- 172.21.15.107:0/3615307332 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fa1a002c510 con 0x5650747a45a0 2024-03-08T21:04:36.623 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.640+0000 7fa1a5600640 1 -- 172.21.15.107:0/3615307332 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa1a0049040 con 0x5650747a45a0 2024-03-08T21:04:36.623 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.640+0000 7fa1c1641740 10 client.15804 fetch_fsmap learned FSMap version 32 2024-03-08T21:04:36.623 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.640+0000 7fa1c1641740 1 -- 172.21.15.107:0/3615307332 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x565074814490 con 0x5650747a45a0 2024-03-08T21:04:36.623 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.640+0000 7fa1a5600640 1 -- 172.21.15.107:0/3615307332 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa1a004a040 con 0x5650747a45a0 2024-03-08T21:04:36.624 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.640+0000 7fa1a5600640 1 -- 172.21.15.107:0/3615307332 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 32) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fa1a0040040 con 0x5650747a45a0 2024-03-08T21:04:36.624 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.640+0000 7fa1c1641740 10 client.15804 fetch_fsmap finished waiting for FSMap version 32 2024-03-08T21:04:36.624 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.640+0000 7fa1c1641740 10 client.15804 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-08T21:04:36.624 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.640+0000 7fa1c1641740 20 client.15804 populate_metadata read hostname 'smithi107' 2024-03-08T21:04:36.624 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.640+0000 7fa1c1641740 1 --2- 172.21.15.107:0/3615307332 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x565074814660 0x565074816a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:04:36.624 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.640+0000 7fa1c1641740 4 client.15804 mds_command: new command op to 34475 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-08T21:04:36.624 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.640+0000 7fa1c1641740 1 -- 172.21.15.107:0/3615307332 --> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x56507470ec60 con 0x565074814660 2024-03-08T21:04:36.624 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.641+0000 7fa1b7e00640 1 --2- 172.21.15.107:0/3615307332 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x565074814660 0x565074816a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:04:36.625 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.642+0000 7fa1b7e00640 1 --2- 172.21.15.107:0/3615307332 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x565074814660 0x565074816a80 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=947108b7c02510ff server_cookie=18f9e37a97a3cc2e in_seq=0 out_seq=0 2024-03-08T21:04:36.625 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.642+0000 7fa1a5600640 10 client.15804 ms_handle_connect on v2:172.21.15.149:6838/55290596 2024-03-08T21:04:36.634 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.653+0000 7fa1a5600640 1 -- 172.21.15.107:0/3615307332 <== mds.0 v2:172.21.15.149:6838/55290596 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x56507470ec60 con 0x565074814660 2024-03-08T21:04:36.634 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.653+0000 7fa1a5600640 10 client.15804 handle_command_reply: tid=0 2024-03-08T21:04:36.634 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.653+0000 7fa1c1641740 1 client.15804 shutdown 2024-03-08T21:04:36.634 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.653+0000 7fa1c1641740 1 -- 172.21.15.107:0/3615307332 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5650747a45a0 msgr2=0x5650747b2720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:04:36.634 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.653+0000 7fa1c1641740 1 --2- 172.21.15.107:0/3615307332 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5650747a45a0 0x5650747b2720 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fa1a0001eb0 tx=0x7fa1a002c330 comp rx=0 tx=0).stop 2024-03-08T21:04:36.634 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.653+0000 7fa1c1641740 1 -- 172.21.15.107:0/3615307332 shutdown_connections 2024-03-08T21:04:36.634 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.653+0000 7fa1c1641740 1 --2- 172.21.15.107:0/3615307332 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x565074814660 0x565074816a80 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:04:36.634 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.653+0000 7fa1c1641740 1 --2- 172.21.15.107:0/3615307332 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5650747ad370 0x5650747b2c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:04:36.634 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.653+0000 7fa1c1641740 1 --2- 172.21.15.107:0/3615307332 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5650747a45a0 0x5650747b2720 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:04:36.634 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.653+0000 7fa1c1641740 1 --2- 172.21.15.107:0/3615307332 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5650747a3c80 0x5650747b4380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:04:36.634 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.653+0000 7fa1a5600640 0 client.15804 ms_handle_reset on v2:172.21.15.149:6838/55290596 2024-03-08T21:04:36.634 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.653+0000 7fa1c1641740 1 -- 172.21.15.107:0/3615307332 >> 172.21.15.107:0/3615307332 conn(0x56507479e5a0 msgr2=0x56507479f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:04:36.634 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.654+0000 7fa1c1641740 1 -- 172.21.15.107:0/3615307332 shutdown_connections 2024-03-08T21:04:36.635 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.654+0000 7fa1c1641740 1 -- 172.21.15.107:0/3615307332 wait complete. 2024-03-08T21:04:36.635 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.654+0000 7fa1c1641740 20 client.15804 trim_cache size 0 max 16384 2024-03-08T21:04:36.635 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:{ 2024-03-08T21:04:36.635 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "message": "", 2024-03-08T21:04:36.635 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "return_code": 0 2024-03-08T21:04:36.635 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:} 2024-03-08T21:04:36.636 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.654+0000 7fa1bea00640 1 -- 172.21.15.107:0/3077329998 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7fa19c0a2ac0 msgr2=0x7fa19c0a4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:04:36.636 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.654+0000 7fa1bea00640 1 --2- 172.21.15.107:0/3077329998 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7fa19c0a2ac0 0x7fa19c0a4f80 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x5650747a3730 tx=0x7fa1a800b4d0 comp rx=0 tx=0).stop 2024-03-08T21:04:36.636 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.654+0000 7fa1bea00640 1 -- 172.21.15.107:0/3077329998 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa1b8074070 msgr2=0x7fa1b81a5be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:04:36.636 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.654+0000 7fa1bea00640 1 --2- 172.21.15.107:0/3077329998 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa1b8074070 0x7fa1b81a5be0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fa1b000cce0 tx=0x7fa1b000a620 comp rx=0 tx=0).stop 2024-03-08T21:04:36.636 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.655+0000 7fa1bea00640 1 -- 172.21.15.107:0/3077329998 shutdown_connections 2024-03-08T21:04:36.636 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.655+0000 7fa1bea00640 1 --2- 172.21.15.107:0/3077329998 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7fa19c0a2ac0 0x7fa19c0a4f80 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:04:36.636 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.655+0000 7fa1bea00640 1 --2- 172.21.15.107:0/3077329998 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa1b8074070 0x7fa1b81a5be0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:04:36.636 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.655+0000 7fa1bea00640 1 --2- 172.21.15.107:0/3077329998 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa1b8106620 0x7fa1b8073b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:04:36.636 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.655+0000 7fa1bea00640 1 --2- 172.21.15.107:0/3077329998 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa1b8101550 0x7fa1b80735f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:04:36.636 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.655+0000 7fa1bea00640 1 -- 172.21.15.107:0/3077329998 >> 172.21.15.107:0/3077329998 conn(0x7fa1b80fcdb0 msgr2=0x7fa1b80ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:04:36.637 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.655+0000 7fa1bea00640 1 -- 172.21.15.107:0/3077329998 shutdown_connections 2024-03-08T21:04:36.637 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:36.655+0000 7fa1bea00640 1 -- 172.21.15.107:0/3077329998 wait complete. 2024-03-08T21:04:37.964 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:04:37 smithi107 ceph-mon[31381]: pgmap v127: 97 pgs: 97 active+clean; 2.7 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-08T21:04:38.012 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:04:37 smithi149 ceph-mon[35042]: pgmap v127: 97 pgs: 97 active+clean; 2.7 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-08T21:04:38.125 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:04:37 smithi204 ceph-mon[34935]: pgmap v127: 97 pgs: 97 active+clean; 2.7 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-08T21:04:39.964 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:04:39 smithi107 ceph-mon[31381]: pgmap v128: 97 pgs: 97 active+clean; 2.7 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 0 B/s wr, 9 op/s 2024-03-08T21:04:40.012 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:04:39 smithi149 ceph-mon[35042]: pgmap v128: 97 pgs: 97 active+clean; 2.7 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 0 B/s wr, 9 op/s 2024-03-08T21:04:40.125 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:04:39 smithi204 ceph-mon[34935]: pgmap v128: 97 pgs: 97 active+clean; 2.7 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 0 B/s wr, 9 op/s 2024-03-08T21:04:41.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:04:41 smithi107 ceph-mon[31381]: pgmap v129: 97 pgs: 97 active+clean; 2.7 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 255 B/s wr, 11 op/s 2024-03-08T21:04:42.012 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:04:41 smithi149 ceph-mon[35042]: pgmap v129: 97 pgs: 97 active+clean; 2.7 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 255 B/s wr, 11 op/s 2024-03-08T21:04:42.125 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:04:41 smithi204 ceph-mon[34935]: pgmap v129: 97 pgs: 97 active+clean; 2.7 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 255 B/s wr, 11 op/s 2024-03-08T21:04:42.964 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:04:42 smithi107 ceph-mon[31381]: pgmap v130: 97 pgs: 97 active+clean; 2.7 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:04:43.012 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:04:42 smithi149 ceph-mon[35042]: pgmap v130: 97 pgs: 97 active+clean; 2.7 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:04:43.125 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:04:42 smithi204 ceph-mon[34935]: pgmap v130: 97 pgs: 97 active+clean; 2.7 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:04:45.875 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:04:45 smithi204 ceph-mon[34935]: pgmap v131: 97 pgs: 97 active+clean; 2.7 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-08T21:04:45.964 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:04:45 smithi107 ceph-mon[31381]: pgmap v131: 97 pgs: 97 active+clean; 2.7 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-08T21:04:46.012 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:04:45 smithi149 ceph-mon[35042]: pgmap v131: 97 pgs: 97 active+clean; 2.7 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-08T21:04:47.876 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:04:47 smithi204 ceph-mon[34935]: pgmap v132: 97 pgs: 97 active+clean; 2.7 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:04:47.876 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:04:47 smithi204 ceph-mon[34935]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T21:04:47.964 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:04:47 smithi107 ceph-mon[31381]: pgmap v132: 97 pgs: 97 active+clean; 2.7 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:04:47.964 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:04:47 smithi107 ceph-mon[31381]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T21:04:48.012 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:04:47 smithi149 ceph-mon[35042]: pgmap v132: 97 pgs: 97 active+clean; 2.7 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:04:48.012 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:04:47 smithi149 ceph-mon[35042]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T21:04:48.964 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:04:48 smithi107 ceph-mon[31381]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T21:04:48.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:04:48 smithi107 ceph-mon[31381]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T21:04:48.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:04:48 smithi107 ceph-mon[31381]: from='mgr.15375 ' entity='mgr.x' 2024-03-08T21:04:49.012 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:04:48 smithi149 ceph-mon[35042]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T21:04:49.012 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:04:48 smithi149 ceph-mon[35042]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T21:04:49.013 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:04:48 smithi149 ceph-mon[35042]: from='mgr.15375 ' entity='mgr.x' 2024-03-08T21:04:49.126 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:04:48 smithi204 ceph-mon[34935]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T21:04:49.126 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:04:48 smithi204 ceph-mon[34935]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T21:04:49.126 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:04:48 smithi204 ceph-mon[34935]: from='mgr.15375 ' entity='mgr.x' 2024-03-08T21:04:49.779 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.798+0000 7f86d0a00640 1 -- 172.21.15.107:0/1207215734 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f86cc10d700 msgr2=0x7f86cc10dae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:04:49.779 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.798+0000 7f86d0a00640 1 --2- 172.21.15.107:0/1207215734 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f86cc10d700 0x7f86cc10dae0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f86bc009930 tx=0x7f86bc02bc30 comp rx=0 tx=0).stop 2024-03-08T21:04:49.780 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.798+0000 7f86d0a00640 1 -- 172.21.15.107:0/1207215734 shutdown_connections 2024-03-08T21:04:49.780 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.798+0000 7f86d0a00640 1 --2- 172.21.15.107:0/1207215734 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f86cc1081b0 0x7f86cc1145c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:04:49.780 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.798+0000 7f86d0a00640 1 --2- 172.21.15.107:0/1207215734 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f86cc107830 0x7f86cc107c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:04:49.780 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.798+0000 7f86d0a00640 1 --2- 172.21.15.107:0/1207215734 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f86cc10d700 0x7f86cc10dae0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:04:49.780 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.798+0000 7f86d0a00640 1 -- 172.21.15.107:0/1207215734 >> 172.21.15.107:0/1207215734 conn(0x7f86cc0770d0 msgr2=0x7f86cc106110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:04:49.780 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.799+0000 7f86d0a00640 1 -- 172.21.15.107:0/1207215734 shutdown_connections 2024-03-08T21:04:49.780 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.799+0000 7f86d0a00640 1 -- 172.21.15.107:0/1207215734 wait complete. 2024-03-08T21:04:49.780 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.799+0000 7f86d0a00640 1 Processor -- start 2024-03-08T21:04:49.780 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.800+0000 7f86d0a00640 1 -- start start 2024-03-08T21:04:49.782 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.800+0000 7f86d0a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f86cc107830 0x7f86cc19dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:04:49.782 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.800+0000 7f86d0a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f86cc1081b0 0x7f86cc19e420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:04:49.782 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.800+0000 7f86d0a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f86cc10d700 0x7f86cc1a27a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:04:49.782 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.800+0000 7f86d0a00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86cc117a00 con 0x7f86cc107830 2024-03-08T21:04:49.782 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.800+0000 7f86d0a00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86cc117880 con 0x7f86cc10d700 2024-03-08T21:04:49.782 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.800+0000 7f86d0a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86cc117b80 con 0x7f86cc1081b0 2024-03-08T21:04:49.783 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.800+0000 7f86ca800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f86cc107830 0x7f86cc19dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:04:49.783 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.800+0000 7f86ca800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f86cc107830 0x7f86cc19dee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:34676/0 (socket says 172.21.15.107:34676) 2024-03-08T21:04:49.783 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.800+0000 7f86ca800640 1 -- 172.21.15.107:0/293717533 learned_addr learned my addr 172.21.15.107:0/293717533 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:04:49.783 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.801+0000 7f86c3e00640 1 --2- 172.21.15.107:0/293717533 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f86cc1081b0 0x7f86cc19e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:04:49.783 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.801+0000 7f86ca800640 1 -- 172.21.15.107:0/293717533 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f86cc1081b0 msgr2=0x7f86cc19e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:04:49.783 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.801+0000 7f86ca800640 1 --2- 172.21.15.107:0/293717533 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f86cc1081b0 0x7f86cc19e420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:04:49.783 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.801+0000 7f86ca800640 1 -- 172.21.15.107:0/293717533 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f86cc10d700 msgr2=0x7f86cc1a27a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:04:49.784 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.801+0000 7f86ca800640 1 --2- 172.21.15.107:0/293717533 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f86cc10d700 0x7f86cc1a27a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:04:49.784 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.801+0000 7f86ca800640 1 -- 172.21.15.107:0/293717533 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86cc1a2ed0 con 0x7f86cc107830 2024-03-08T21:04:49.784 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.801+0000 7f86ca800640 1 --2- 172.21.15.107:0/293717533 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f86cc107830 0x7f86cc19dee0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f86bc000c00 tx=0x7f86bc02c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:04:49.784 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.801+0000 7f86c3400640 1 -- 172.21.15.107:0/293717533 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86bc043020 con 0x7f86cc107830 2024-03-08T21:04:49.784 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.802+0000 7f86d0a00640 1 -- 172.21.15.107:0/293717533 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86cc1a3160 con 0x7f86cc107830 2024-03-08T21:04:49.784 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.802+0000 7f86c3400640 1 -- 172.21.15.107:0/293717533 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f86bc02cc60 con 0x7f86cc107830 2024-03-08T21:04:49.784 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.802+0000 7f86c3400640 1 -- 172.21.15.107:0/293717533 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86bc003c30 con 0x7f86cc107830 2024-03-08T21:04:49.784 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.802+0000 7f86d0a00640 1 -- 172.21.15.107:0/293717533 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86cc1aaa50 con 0x7f86cc107830 2024-03-08T21:04:49.785 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.805+0000 7f86c3400640 1 -- 172.21.15.107:0/293717533 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) v1 ==== 137422+0+0 (secure 0 0 0) 0x7f86bc046020 con 0x7f86cc107830 2024-03-08T21:04:49.785 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.805+0000 7f86d41b4740 1 Processor -- start 2024-03-08T21:04:49.785 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.806+0000 7f86d41b4740 1 -- start start 2024-03-08T21:04:49.787 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.806+0000 7f86d41b4740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5579aa1b1e80 0x5579aa1b2260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:04:49.787 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.806+0000 7f86d41b4740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5579aa1b27a0 0x5579aa1bb030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:04:49.787 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.806+0000 7f86d41b4740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5579aa1bb570 0x5579aa1bd960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:04:49.787 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.806+0000 7f86d41b4740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579aa1153d0 con 0x5579aa1bb570 2024-03-08T21:04:49.788 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.806+0000 7f86d41b4740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579aa0be6b0 con 0x5579aa1b27a0 2024-03-08T21:04:49.788 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.806+0000 7f86d41b4740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579aa111470 con 0x5579aa1b1e80 2024-03-08T21:04:49.788 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.806+0000 7f86c3e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5579aa1bb570 0x5579aa1bd960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:04:49.788 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.806+0000 7f86c3e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5579aa1bb570 0x5579aa1bd960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:34680/0 (socket says 172.21.15.107:34680) 2024-03-08T21:04:49.788 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.807+0000 7f86c3e00640 1 -- 172.21.15.107:0/2009182796 learned_addr learned my addr 172.21.15.107:0/2009182796 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:04:49.788 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.807+0000 7f86cb200640 1 --2- 172.21.15.107:0/2009182796 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5579aa1b1e80 0x5579aa1b2260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:04:49.788 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.807+0000 7f86ca800640 1 --2- 172.21.15.107:0/2009182796 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5579aa1b27a0 0x5579aa1bb030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:04:49.788 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.807+0000 7f86c3400640 1 --2- 172.21.15.107:0/293717533 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f86b00a2960 0x7f86b00a4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:04:49.788 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.807+0000 7f86c3e00640 1 -- 172.21.15.107:0/2009182796 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5579aa1b1e80 msgr2=0x5579aa1b2260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:04:49.788 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.807+0000 7f86c3e00640 1 --2- 172.21.15.107:0/2009182796 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5579aa1b1e80 0x5579aa1b2260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:04:49.788 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.807+0000 7f86c3e00640 1 -- 172.21.15.107:0/2009182796 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5579aa1b27a0 msgr2=0x5579aa1bb030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:04:49.788 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.807+0000 7f86c3e00640 1 --2- 172.21.15.107:0/2009182796 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5579aa1b27a0 0x5579aa1bb030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:04:49.789 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.807+0000 7f86c3e00640 1 -- 172.21.15.107:0/2009182796 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5579aa0e4b50 con 0x5579aa1bb570 2024-03-08T21:04:49.789 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.807+0000 7f86c3400640 1 -- 172.21.15.107:0/293717533 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f86bc0f10c0 con 0x7f86cc107830 2024-03-08T21:04:49.789 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.807+0000 7f86cb200640 1 --2- 172.21.15.107:0/293717533 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f86b00a2960 0x7f86b00a4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:04:49.789 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.807+0000 7f86c3e00640 1 --2- 172.21.15.107:0/2009182796 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5579aa1bb570 0x5579aa1bd960 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f86b400bc80 tx=0x7f86b400f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c56ea657b11e36cc server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:04:49.798 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.808+0000 7f86bb400640 1 -- 172.21.15.107:0/2009182796 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86b4015070 con 0x5579aa1bb570 2024-03-08T21:04:49.798 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.808+0000 7f86bb400640 1 -- 172.21.15.107:0/2009182796 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f86b401a910 con 0x5579aa1bb570 2024-03-08T21:04:49.798 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.808+0000 7f86bb400640 1 -- 172.21.15.107:0/2009182796 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86b4011870 con 0x5579aa1bb570 2024-03-08T21:04:49.799 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.809+0000 7f86d41b4740 1 -- 172.21.15.107:0/2009182796 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5579aa1bb570 msgr2=0x5579aa1bd960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:04:49.799 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.809+0000 7f86cb200640 1 --2- 172.21.15.107:0/293717533 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f86b00a2960 0x7f86b00a4e20 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f86c400acc0 tx=0x7f86c4004a40 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:04:49.799 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.809+0000 7f86d41b4740 1 --2- 172.21.15.107:0/2009182796 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5579aa1bb570 0x5579aa1bd960 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f86b400bc80 tx=0x7f86b400f730 comp rx=0 tx=0).stop 2024-03-08T21:04:49.799 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.809+0000 7f86d41b4740 1 -- 172.21.15.107:0/2009182796 shutdown_connections 2024-03-08T21:04:49.799 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.809+0000 7f86d41b4740 1 --2- 172.21.15.107:0/2009182796 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5579aa1bb570 0x5579aa1bd960 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:04:49.800 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.809+0000 7f86d41b4740 1 --2- 172.21.15.107:0/2009182796 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5579aa1b27a0 0x5579aa1bb030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:04:49.800 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.809+0000 7f86d41b4740 1 --2- 172.21.15.107:0/2009182796 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5579aa1b1e80 0x5579aa1b2260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:04:49.800 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.809+0000 7f86d41b4740 1 -- 172.21.15.107:0/2009182796 >> 172.21.15.107:0/2009182796 conn(0x5579aa1ac7a0 msgr2=0x5579aa1acb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:04:49.800 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.809+0000 7f86d41b4740 1 -- 172.21.15.107:0/2009182796 shutdown_connections 2024-03-08T21:04:49.800 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.809+0000 7f86d41b4740 1 -- 172.21.15.107:0/2009182796 wait complete. 2024-03-08T21:04:49.800 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.809+0000 7f86d41b4740 1 -- start start 2024-03-08T21:04:49.801 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.810+0000 7f86d41b4740 1 Processor -- start 2024-03-08T21:04:49.801 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.810+0000 7f86d41b4740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5579aa1b1e80 0x5579aa1baf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:04:49.801 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.810+0000 7f86d41b4740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5579aa1b27a0 0x5579aa1b82a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:04:49.801 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.810+0000 7f86d41b4740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5579aa1bb570 0x5579aa1b8840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:04:49.801 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.810+0000 7f86d41b4740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579aa1be1a0 con 0x5579aa1bb570 2024-03-08T21:04:49.801 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.810+0000 7f86d41b4740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579aa0d19f0 con 0x5579aa1b1e80 2024-03-08T21:04:49.802 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.810+0000 7f86d41b4740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579aa1be320 con 0x5579aa1b27a0 2024-03-08T21:04:49.802 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.810+0000 7f86d41b4740 10 client.0 _ll_register_callbacks cb 0x5579a9e29a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T21:04:49.802 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.811+0000 7f86cb200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5579aa1bb570 0x5579aa1b8840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:04:49.802 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.811+0000 7f86cb200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5579aa1bb570 0x5579aa1b8840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:34696/0 (socket says 172.21.15.107:34696) 2024-03-08T21:04:49.802 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.811+0000 7f86cb200640 1 -- 172.21.15.107:0/3502675671 learned_addr learned my addr 172.21.15.107:0/3502675671 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:04:49.802 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.811+0000 7f86ca800640 1 --2- 172.21.15.107:0/3502675671 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5579aa1b1e80 0x5579aa1baf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:04:49.803 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.811+0000 7f86c3e00640 1 --2- 172.21.15.107:0/3502675671 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5579aa1b27a0 0x5579aa1b82a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:04:49.803 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.811+0000 7f86cb200640 1 -- 172.21.15.107:0/3502675671 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5579aa1b27a0 msgr2=0x5579aa1b82a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:04:49.803 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.811+0000 7f86cb200640 1 --2- 172.21.15.107:0/3502675671 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5579aa1b27a0 0x5579aa1b82a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:04:49.803 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.811+0000 7f86cb200640 1 -- 172.21.15.107:0/3502675671 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5579aa1b1e80 msgr2=0x5579aa1baf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:04:49.803 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.811+0000 7f86cb200640 1 --2- 172.21.15.107:0/3502675671 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5579aa1b1e80 0x5579aa1baf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:04:49.803 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.811+0000 7f86cb200640 1 -- 172.21.15.107:0/3502675671 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5579a9e2a210 con 0x5579aa1bb570 2024-03-08T21:04:49.804 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.811+0000 7f86cb200640 1 --2- 172.21.15.107:0/3502675671 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5579aa1bb570 0x5579aa1b8840 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f86c400a950 tx=0x7f86c400a3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=543337738c2f7eda server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:04:49.804 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.811+0000 7f86afe00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-03-08T21:04:49.804 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.812+0000 7f86afe00640 1 -- 172.21.15.107:0/3502675671 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86c4021070 con 0x5579aa1bb570 2024-03-08T21:04:49.804 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.812+0000 7f86afe00640 1 -- 172.21.15.107:0/3502675671 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f86c4010b50 con 0x5579aa1bb570 2024-03-08T21:04:49.804 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.812+0000 7f86d41b4740 1 -- 172.21.15.107:0/3502675671 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f86b401a910 con 0x5579aa1bb570 2024-03-08T21:04:49.804 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.812+0000 7f86afe00640 1 -- 172.21.15.107:0/3502675671 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f86c402d050 con 0x5579aa1bb570 2024-03-08T21:04:49.804 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.812+0000 7f86d41b4740 10 client.15816 fetch_fsmap learned FSMap version 32 2024-03-08T21:04:49.805 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.812+0000 7f86d41b4740 1 -- 172.21.15.107:0/3502675671 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x5579a9ef6620 con 0x5579aa1bb570 2024-03-08T21:04:49.805 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.812+0000 7f86afe00640 1 -- 172.21.15.107:0/3502675671 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f86b401a910 con 0x5579aa1bb570 2024-03-08T21:04:49.805 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.813+0000 7f86afe00640 1 -- 172.21.15.107:0/3502675671 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 32) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f86c401b3c0 con 0x5579aa1bb570 2024-03-08T21:04:49.805 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.813+0000 7f86d41b4740 10 client.15816 fetch_fsmap finished waiting for FSMap version 32 2024-03-08T21:04:49.805 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.813+0000 7f86d41b4740 10 client.15816 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-08T21:04:49.805 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.813+0000 7f86d41b4740 20 client.15816 populate_metadata read hostname 'smithi107' 2024-03-08T21:04:49.806 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.813+0000 7f86d41b4740 1 --2- 172.21.15.107:0/3502675671 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x5579aa1cb460 0x5579aa1cd870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:04:49.806 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.813+0000 7f86d41b4740 4 client.15816 mds_command: new command op to 34475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-08T21:04:49.806 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.813+0000 7f86d41b4740 1 -- 172.21.15.107:0/3502675671 --> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5579aa122910 con 0x5579aa1cb460 2024-03-08T21:04:49.806 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.814+0000 7f86ca800640 1 --2- 172.21.15.107:0/3502675671 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x5579aa1cb460 0x5579aa1cd870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:04:49.806 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.814+0000 7f86ca800640 1 --2- 172.21.15.107:0/3502675671 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x5579aa1cb460 0x5579aa1cd870 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f8b25fc5aa95d279 server_cookie=545be5c46925556e in_seq=0 out_seq=0 2024-03-08T21:04:49.806 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.814+0000 7f86afe00640 10 client.15816 ms_handle_connect on v2:172.21.15.149:6838/55290596 2024-03-08T21:04:49.807 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.816+0000 7f86afe00640 1 -- 172.21.15.107:0/3502675671 <== mds.0 v2:172.21.15.149:6838/55290596 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5579aa122910 con 0x5579aa1cb460 2024-03-08T21:04:49.807 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.816+0000 7f86afe00640 10 client.15816 handle_command_reply: tid=0 2024-03-08T21:04:49.807 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.816+0000 7f86d41b4740 1 client.15816 shutdown 2024-03-08T21:04:49.807 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.816+0000 7f86d41b4740 1 -- 172.21.15.107:0/3502675671 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5579aa1bb570 msgr2=0x5579aa1b8840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:04:49.807 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.816+0000 7f86d41b4740 1 --2- 172.21.15.107:0/3502675671 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5579aa1bb570 0x5579aa1b8840 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f86c400a950 tx=0x7f86c400a3c0 comp rx=0 tx=0).stop 2024-03-08T21:04:49.808 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.816+0000 7f86d41b4740 1 -- 172.21.15.107:0/3502675671 shutdown_connections 2024-03-08T21:04:49.808 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.816+0000 7f86d41b4740 1 --2- 172.21.15.107:0/3502675671 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5579aa1bb570 0x5579aa1b8840 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:04:49.808 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.816+0000 7f86d41b4740 1 --2- 172.21.15.107:0/3502675671 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5579aa1b27a0 0x5579aa1b82a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:04:49.808 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.816+0000 7f86d41b4740 1 --2- 172.21.15.107:0/3502675671 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x5579aa1cb460 0x5579aa1cd870 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:04:49.808 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.816+0000 7f86d41b4740 1 --2- 172.21.15.107:0/3502675671 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5579aa1b1e80 0x5579aa1baf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:04:49.808 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.816+0000 7f86d41b4740 1 -- 172.21.15.107:0/3502675671 >> 172.21.15.107:0/3502675671 conn(0x5579aa1ac7a0 msgr2=0x5579aa1ae4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:04:49.809 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.816+0000 7f86afe00640 0 client.15816 ms_handle_reset on v2:172.21.15.149:6838/55290596 2024-03-08T21:04:49.809 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.816+0000 7f86d41b4740 1 -- 172.21.15.107:0/3502675671 shutdown_connections 2024-03-08T21:04:49.809 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.816+0000 7f86d41b4740 1 -- 172.21.15.107:0/3502675671 wait complete. 2024-03-08T21:04:49.809 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.817+0000 7f86d41b4740 20 client.15816 trim_cache size 0 max 16384 2024-03-08T21:04:49.823 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.832+0000 7f86d41b4740 1 Processor -- start 2024-03-08T21:04:49.823 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.832+0000 7f86d41b4740 1 -- start start 2024-03-08T21:04:49.824 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.832+0000 7f86d41b4740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5579aa1b1e80 0x5579aa1ba9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:04:49.824 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.832+0000 7f86d41b4740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5579aa1b27a0 0x5579aa1baf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:04:49.824 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.832+0000 7f86d41b4740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5579aa1bb570 0x5579aa1b5500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:04:49.824 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.832+0000 7f86d41b4740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86cc1b3150 con 0x5579aa1b1e80 2024-03-08T21:04:49.824 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.832+0000 7f86d41b4740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86cc1b4ad0 con 0x5579aa1bb570 2024-03-08T21:04:49.824 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.832+0000 7f86d41b4740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86cc1b37b0 con 0x5579aa1b27a0 2024-03-08T21:04:49.825 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.833+0000 7f86ca800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5579aa1b1e80 0x5579aa1ba9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:04:49.825 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.833+0000 7f86ca800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5579aa1b1e80 0x5579aa1ba9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:34712/0 (socket says 172.21.15.107:34712) 2024-03-08T21:04:49.825 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.833+0000 7f86ca800640 1 -- 172.21.15.107:0/3573076151 learned_addr learned my addr 172.21.15.107:0/3573076151 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:04:49.825 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.833+0000 7f86cb200640 1 --2- 172.21.15.107:0/3573076151 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5579aa1bb570 0x5579aa1b5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:04:49.825 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.833+0000 7f86c3e00640 1 --2- 172.21.15.107:0/3573076151 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5579aa1b27a0 0x5579aa1baf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:04:49.826 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.833+0000 7f86ca800640 1 -- 172.21.15.107:0/3573076151 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5579aa1b27a0 msgr2=0x5579aa1baf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:04:49.826 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.833+0000 7f86ca800640 1 --2- 172.21.15.107:0/3573076151 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5579aa1b27a0 0x5579aa1baf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:04:49.826 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.833+0000 7f86ca800640 1 -- 172.21.15.107:0/3573076151 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5579aa1bb570 msgr2=0x5579aa1b5500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:04:49.826 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.833+0000 7f86ca800640 1 --2- 172.21.15.107:0/3573076151 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5579aa1bb570 0x5579aa1b5500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:04:49.826 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.833+0000 7f86ca800640 1 -- 172.21.15.107:0/3573076151 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5579aa1b67f0 con 0x5579aa1b1e80 2024-03-08T21:04:49.827 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.833+0000 7f86cb200640 1 --2- 172.21.15.107:0/3573076151 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5579aa1bb570 0x5579aa1b5500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T21:04:49.827 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.833+0000 7f86ca800640 1 --2- 172.21.15.107:0/3573076151 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5579aa1b1e80 0x5579aa1ba9d0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f86bc005ab0 tx=0x7f86bc0a8040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3678802ae86bc89a server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:04:49.827 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.834+0000 7f86baa00640 1 -- 172.21.15.107:0/3573076151 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86bc082e60 con 0x5579aa1b1e80 2024-03-08T21:04:49.827 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.834+0000 7f86baa00640 1 -- 172.21.15.107:0/3573076151 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f86bc0035f0 con 0x5579aa1b1e80 2024-03-08T21:04:49.827 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.834+0000 7f86baa00640 1 -- 172.21.15.107:0/3573076151 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86bc0c5fc0 con 0x5579aa1b1e80 2024-03-08T21:04:49.827 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.835+0000 7f86d41b4740 1 -- 172.21.15.107:0/3573076151 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5579aa1b1e80 msgr2=0x5579aa1ba9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:04:49.828 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.835+0000 7f86d41b4740 1 --2- 172.21.15.107:0/3573076151 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5579aa1b1e80 0x5579aa1ba9d0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f86bc005ab0 tx=0x7f86bc0a8040 comp rx=0 tx=0).stop 2024-03-08T21:04:49.828 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.835+0000 7f86d41b4740 1 -- 172.21.15.107:0/3573076151 shutdown_connections 2024-03-08T21:04:49.828 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.835+0000 7f86d41b4740 1 --2- 172.21.15.107:0/3573076151 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5579aa1bb570 0x5579aa1b5500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:04:49.828 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.835+0000 7f86d41b4740 1 --2- 172.21.15.107:0/3573076151 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5579aa1b27a0 0x5579aa1baf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:04:49.828 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.835+0000 7f86d41b4740 1 --2- 172.21.15.107:0/3573076151 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5579aa1b1e80 0x5579aa1ba9d0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:04:49.828 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.835+0000 7f86d41b4740 1 -- 172.21.15.107:0/3573076151 >> 172.21.15.107:0/3573076151 conn(0x5579aa1ac7a0 msgr2=0x5579aa1b6b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:04:49.829 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.835+0000 7f86d41b4740 1 -- 172.21.15.107:0/3573076151 shutdown_connections 2024-03-08T21:04:49.829 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.835+0000 7f86d41b4740 1 -- 172.21.15.107:0/3573076151 wait complete. 2024-03-08T21:04:49.829 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.835+0000 7f86d41b4740 1 -- start start 2024-03-08T21:04:49.829 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.835+0000 7f86d41b4740 1 Processor -- start 2024-03-08T21:04:49.829 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.836+0000 7f86d41b4740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5579aa1b1e80 0x5579aa1c1170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:04:49.829 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.836+0000 7f86d41b4740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5579aa1b27a0 0x5579aa1c16b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:04:49.830 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.836+0000 7f86d41b4740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5579aa1bb570 0x5579aa1c1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:04:49.830 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.836+0000 7f86d41b4740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86cc1b4470 con 0x5579aa1b27a0 2024-03-08T21:04:49.830 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.836+0000 7f86d41b4740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86cc1b3e10 con 0x5579aa1bb570 2024-03-08T21:04:49.830 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.836+0000 7f86d41b4740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86cc1b13f0 con 0x5579aa1b1e80 2024-03-08T21:04:49.831 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.836+0000 7f86d41b4740 10 client.0 _ll_register_callbacks cb 0x7f86b400b440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T21:04:49.831 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.836+0000 7f86c3e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5579aa1b27a0 0x5579aa1c16b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:04:49.831 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.836+0000 7f86c3e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5579aa1b27a0 0x5579aa1c16b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:34714/0 (socket says 172.21.15.107:34714) 2024-03-08T21:04:49.831 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.836+0000 7f86c3e00640 1 -- 172.21.15.107:0/2655321295 learned_addr learned my addr 172.21.15.107:0/2655321295 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:04:49.832 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.837+0000 7f86ca800640 1 --2- 172.21.15.107:0/2655321295 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5579aa1b1e80 0x5579aa1c1170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:04:49.832 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.837+0000 7f86cb200640 1 --2- 172.21.15.107:0/2655321295 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5579aa1bb570 0x5579aa1c1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:04:49.832 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.837+0000 7f86c3e00640 1 -- 172.21.15.107:0/2655321295 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5579aa1b1e80 msgr2=0x5579aa1c1170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:04:49.832 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.837+0000 7f86c3e00640 1 --2- 172.21.15.107:0/2655321295 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5579aa1b1e80 0x5579aa1c1170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:04:49.832 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.837+0000 7f86c3e00640 1 -- 172.21.15.107:0/2655321295 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5579aa1bb570 msgr2=0x5579aa1c1bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:04:49.833 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.837+0000 7f86c3e00640 1 --2- 172.21.15.107:0/2655321295 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5579aa1bb570 0x5579aa1c1bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:04:49.833 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.837+0000 7f86c3e00640 1 -- 172.21.15.107:0/2655321295 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5579aa221bc0 con 0x5579aa1b27a0 2024-03-08T21:04:49.833 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.837+0000 7f86c3e00640 1 --2- 172.21.15.107:0/2655321295 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5579aa1b27a0 0x5579aa1c16b0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x5579aa1ad930 tx=0x7f86b400b9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7828446356dcdd7 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:04:49.833 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.837+0000 7f86afe00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-03-08T21:04:49.833 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.837+0000 7f86afe00640 1 -- 172.21.15.107:0/2655321295 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86b4028060 con 0x5579aa1b27a0 2024-03-08T21:04:49.833 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.838+0000 7f86afe00640 1 -- 172.21.15.107:0/2655321295 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f86b4004230 con 0x5579aa1b27a0 2024-03-08T21:04:49.834 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.838+0000 7f86d41b4740 1 -- 172.21.15.107:0/2655321295 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5579aa11cc60 con 0x5579aa1b27a0 2024-03-08T21:04:49.834 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.838+0000 7f86afe00640 1 -- 172.21.15.107:0/2655321295 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f86b4015040 con 0x5579aa1b27a0 2024-03-08T21:04:49.834 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.838+0000 7f86d41b4740 10 client.15822 fetch_fsmap learned FSMap version 32 2024-03-08T21:04:49.834 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.838+0000 7f86d41b4740 1 -- 172.21.15.107:0/2655321295 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x5579aa222e00 con 0x5579aa1b27a0 2024-03-08T21:04:49.834 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.838+0000 7f86afe00640 1 -- 172.21.15.107:0/2655321295 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x5579aa11cc60 con 0x5579aa1b27a0 2024-03-08T21:04:49.835 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.838+0000 7f86afe00640 1 -- 172.21.15.107:0/2655321295 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 32) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f86b4004b70 con 0x5579aa1b27a0 2024-03-08T21:04:49.835 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.838+0000 7f86d41b4740 10 client.15822 fetch_fsmap finished waiting for FSMap version 32 2024-03-08T21:04:49.835 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.838+0000 7f86d41b4740 10 client.15822 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-08T21:04:49.835 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.838+0000 7f86d41b4740 20 client.15822 populate_metadata read hostname 'smithi107' 2024-03-08T21:04:49.835 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.838+0000 7f86d41b4740 1 --2- 172.21.15.107:0/2655321295 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x5579aa223190 0x5579aa225580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:04:49.835 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.838+0000 7f86d41b4740 4 client.15822 mds_command: new command op to 34475 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-08T21:04:49.836 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.838+0000 7f86d41b4740 1 -- 172.21.15.107:0/2655321295 --> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5579aa09b5d0 con 0x5579aa223190 2024-03-08T21:04:49.836 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.839+0000 7f86ca800640 1 --2- 172.21.15.107:0/2655321295 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x5579aa223190 0x5579aa225580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:04:49.836 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.840+0000 7f86ca800640 1 --2- 172.21.15.107:0/2655321295 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x5579aa223190 0x5579aa225580 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2adebf4160427cb5 server_cookie=30ea9ce3b21a79bc in_seq=0 out_seq=0 2024-03-08T21:04:49.836 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.840+0000 7f86afe00640 10 client.15822 ms_handle_connect on v2:172.21.15.149:6838/55290596 2024-03-08T21:04:49.839 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.851+0000 7f86afe00640 1 -- 172.21.15.107:0/2655321295 <== mds.0 v2:172.21.15.149:6838/55290596 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5579aa09b5d0 con 0x5579aa223190 2024-03-08T21:04:49.839 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.851+0000 7f86afe00640 10 client.15822 handle_command_reply: tid=0 2024-03-08T21:04:49.839 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.851+0000 7f86d41b4740 1 client.15822 shutdown 2024-03-08T21:04:49.839 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.851+0000 7f86d41b4740 1 -- 172.21.15.107:0/2655321295 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5579aa1b27a0 msgr2=0x5579aa1c16b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:04:49.839 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.851+0000 7f86d41b4740 1 --2- 172.21.15.107:0/2655321295 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5579aa1b27a0 0x5579aa1c16b0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x5579aa1ad930 tx=0x7f86b400b9f0 comp rx=0 tx=0).stop 2024-03-08T21:04:49.840 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.851+0000 7f86d41b4740 1 -- 172.21.15.107:0/2655321295 shutdown_connections 2024-03-08T21:04:49.840 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.851+0000 7f86d41b4740 1 --2- 172.21.15.107:0/2655321295 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x5579aa223190 0x5579aa225580 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:04:49.840 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.851+0000 7f86d41b4740 1 --2- 172.21.15.107:0/2655321295 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5579aa1bb570 0x5579aa1c1bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:04:49.840 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.851+0000 7f86d41b4740 1 --2- 172.21.15.107:0/2655321295 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5579aa1b27a0 0x5579aa1c16b0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:04:49.840 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.851+0000 7f86d41b4740 1 --2- 172.21.15.107:0/2655321295 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5579aa1b1e80 0x5579aa1c1170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:04:49.840 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.851+0000 7f86afe00640 0 client.15822 ms_handle_reset on v2:172.21.15.149:6838/55290596 2024-03-08T21:04:49.841 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.851+0000 7f86d41b4740 1 -- 172.21.15.107:0/2655321295 >> 172.21.15.107:0/2655321295 conn(0x5579aa1ac7a0 msgr2=0x5579aa1adbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:04:49.841 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.852+0000 7f86d41b4740 1 -- 172.21.15.107:0/2655321295 shutdown_connections 2024-03-08T21:04:49.841 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.852+0000 7f86d41b4740 1 -- 172.21.15.107:0/2655321295 wait complete. 2024-03-08T21:04:49.841 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.852+0000 7f86d41b4740 20 client.15822 trim_cache size 0 max 16384 2024-03-08T21:04:49.842 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:{ 2024-03-08T21:04:49.842 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "message": "", 2024-03-08T21:04:49.842 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "return_code": 0 2024-03-08T21:04:49.842 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:} 2024-03-08T21:04:49.843 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.852+0000 7f86d0a00640 1 -- 172.21.15.107:0/293717533 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f86b00a2960 msgr2=0x7f86b00a4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:04:49.844 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.852+0000 7f86d0a00640 1 --2- 172.21.15.107:0/293717533 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f86b00a2960 0x7f86b00a4e20 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f86c400acc0 tx=0x7f86c4004a40 comp rx=0 tx=0).stop 2024-03-08T21:04:49.844 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.852+0000 7f86d0a00640 1 -- 172.21.15.107:0/293717533 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f86cc107830 msgr2=0x7f86cc19dee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:04:49.844 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.852+0000 7f86d0a00640 1 --2- 172.21.15.107:0/293717533 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f86cc107830 0x7f86cc19dee0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f86bc000c00 tx=0x7f86bc02c860 comp rx=0 tx=0).stop 2024-03-08T21:04:49.844 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.852+0000 7f86d0a00640 1 -- 172.21.15.107:0/293717533 shutdown_connections 2024-03-08T21:04:49.844 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.852+0000 7f86d0a00640 1 --2- 172.21.15.107:0/293717533 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f86b00a2960 0x7f86b00a4e20 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:04:49.845 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.852+0000 7f86d0a00640 1 --2- 172.21.15.107:0/293717533 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f86cc10d700 0x7f86cc1a27a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:04:49.845 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.853+0000 7f86d0a00640 1 --2- 172.21.15.107:0/293717533 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f86cc1081b0 0x7f86cc19e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:04:49.845 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.853+0000 7f86d0a00640 1 --2- 172.21.15.107:0/293717533 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f86cc107830 0x7f86cc19dee0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:04:49.845 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.853+0000 7f86d0a00640 1 -- 172.21.15.107:0/293717533 >> 172.21.15.107:0/293717533 conn(0x7f86cc0770d0 msgr2=0x7f86cc078340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:04:49.845 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.853+0000 7f86d0a00640 1 -- 172.21.15.107:0/293717533 shutdown_connections 2024-03-08T21:04:49.846 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:04:49.853+0000 7f86d0a00640 1 -- 172.21.15.107:0/293717533 wait complete. 2024-03-08T21:04:49.964 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:04:49 smithi107 ceph-mon[31381]: pgmap v133: 97 pgs: 97 active+clean; 2.7 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 255 B/s wr, 10 op/s 2024-03-08T21:04:50.012 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:04:49 smithi149 ceph-mon[35042]: pgmap v133: 97 pgs: 97 active+clean; 2.7 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 255 B/s wr, 10 op/s 2024-03-08T21:04:50.125 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:04:49 smithi204 ceph-mon[34935]: pgmap v133: 97 pgs: 97 active+clean; 2.7 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 255 B/s wr, 10 op/s 2024-03-08T21:04:51.964 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:04:51 smithi107 ceph-mon[31381]: pgmap v134: 97 pgs: 97 active+clean; 2.7 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 255 B/s wr, 11 op/s 2024-03-08T21:04:52.012 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:04:51 smithi149 ceph-mon[35042]: pgmap v134: 97 pgs: 97 active+clean; 2.7 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 255 B/s wr, 11 op/s 2024-03-08T21:04:52.125 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:04:51 smithi204 ceph-mon[34935]: pgmap v134: 97 pgs: 97 active+clean; 2.7 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 255 B/s wr, 11 op/s 2024-03-08T21:04:53.964 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:04:53 smithi107 ceph-mon[31381]: pgmap v135: 97 pgs: 97 active+clean; 2.7 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-08T21:04:54.012 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:04:53 smithi149 ceph-mon[35042]: pgmap v135: 97 pgs: 97 active+clean; 2.7 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-08T21:04:54.125 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:04:53 smithi204 ceph-mon[34935]: pgmap v135: 97 pgs: 97 active+clean; 2.7 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-08T21:04:55.964 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:04:55 smithi107 ceph-mon[31381]: pgmap v136: 97 pgs: 97 active+clean; 2.7 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-08T21:04:56.012 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:04:55 smithi149 ceph-mon[35042]: pgmap v136: 97 pgs: 97 active+clean; 2.7 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-08T21:04:56.125 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:04:55 smithi204 ceph-mon[34935]: pgmap v136: 97 pgs: 97 active+clean; 2.7 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-08T21:04:56.964 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:04:56 smithi107 ceph-mon[31381]: pgmap v137: 97 pgs: 97 active+clean; 2.7 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:04:57.012 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:04:56 smithi149 ceph-mon[35042]: pgmap v137: 97 pgs: 97 active+clean; 2.7 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:04:57.125 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:04:56 smithi204 ceph-mon[34935]: pgmap v137: 97 pgs: 97 active+clean; 2.7 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:04:59.875 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:04:59 smithi204 ceph-mon[34935]: pgmap v138: 97 pgs: 97 active+clean; 2.7 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 255 B/s wr, 10 op/s 2024-03-08T21:04:59.964 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:04:59 smithi107 ceph-mon[31381]: pgmap v138: 97 pgs: 97 active+clean; 2.7 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 255 B/s wr, 10 op/s 2024-03-08T21:05:00.012 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:04:59 smithi149 ceph-mon[35042]: pgmap v138: 97 pgs: 97 active+clean; 2.7 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 255 B/s wr, 10 op/s 2024-03-08T21:05:01.964 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:05:01 smithi107 ceph-mon[31381]: pgmap v139: 97 pgs: 97 active+clean; 2.7 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 255 B/s wr, 11 op/s 2024-03-08T21:05:02.012 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:05:01 smithi149 ceph-mon[35042]: pgmap v139: 97 pgs: 97 active+clean; 2.7 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 255 B/s wr, 11 op/s 2024-03-08T21:05:02.125 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:05:01 smithi204 ceph-mon[34935]: pgmap v139: 97 pgs: 97 active+clean; 2.7 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 255 B/s wr, 11 op/s 2024-03-08T21:05:02.979 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:02.999+0000 7f1e21a00640 1 -- 172.21.15.107:0/777293129 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1e1c1078c0 msgr2=0x7f1e1c107d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:05:02.979 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:02.999+0000 7f1e21a00640 1 --2- 172.21.15.107:0/777293129 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1e1c1078c0 0x7f1e1c107d40 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f1e08009930 tx=0x7f1e0802bc30 comp rx=0 tx=0).stop 2024-03-08T21:05:02.979 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:02.999+0000 7f1e21a00640 1 -- 172.21.15.107:0/777293129 shutdown_connections 2024-03-08T21:05:02.979 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:02.999+0000 7f1e21a00640 1 --2- 172.21.15.107:0/777293129 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1e1c108280 0x7f1e1c114790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:05:02.979 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:02.999+0000 7f1e21a00640 1 --2- 172.21.15.107:0/777293129 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1e1c1078c0 0x7f1e1c107d40 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:05:02.979 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:02.999+0000 7f1e21a00640 1 --2- 172.21.15.107:0/777293129 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1e1c10d8d0 0x7f1e1c10dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:05:02.980 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:02.999+0000 7f1e21a00640 1 -- 172.21.15.107:0/777293129 >> 172.21.15.107:0/777293129 conn(0x7f1e1c075860 msgr2=0x7f1e1c077c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:05:02.980 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:02.999+0000 7f1e21a00640 1 -- 172.21.15.107:0/777293129 shutdown_connections 2024-03-08T21:05:02.980 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:02.999+0000 7f1e21a00640 1 -- 172.21.15.107:0/777293129 wait complete. 2024-03-08T21:05:02.981 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:03.000+0000 7f1e21a00640 1 Processor -- start 2024-03-08T21:05:02.981 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:03.001+0000 7f1e21a00640 1 -- start start 2024-03-08T21:05:02.981 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:03.001+0000 7f1e21a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1e1c1078c0 0x7f1e1c19e080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:05:02.981 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:03.001+0000 7f1e21a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1e1c108280 0x7f1e1c19e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:05:02.981 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:03.001+0000 7f1e21a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1e1c10d8d0 0x7f1e1c1a28f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:05:02.981 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:03.001+0000 7f1e21a00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e1c117be0 con 0x7f1e1c108280 2024-03-08T21:05:02.981 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:03.001+0000 7f1e21a00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e1c117a60 con 0x7f1e1c1078c0 2024-03-08T21:05:02.981 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:03.001+0000 7f1e21a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e1c117d60 con 0x7f1e1c10d8d0 2024-03-08T21:05:02.981 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:03.001+0000 7f1e1a000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1e1c108280 0x7f1e1c19e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:05:02.982 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:03.001+0000 7f1e1a000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1e1c108280 0x7f1e1c19e5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:50760/0 (socket says 172.21.15.107:50760) 2024-03-08T21:05:02.982 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:03.001+0000 7f1e1a000640 1 -- 172.21.15.107:0/859001874 learned_addr learned my addr 172.21.15.107:0/859001874 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:05:02.982 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:03.002+0000 7f1e1aa00640 1 --2- 172.21.15.107:0/859001874 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1e1c1078c0 0x7f1e1c19e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:05:02.983 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:03.002+0000 7f1e1b400640 1 --2- 172.21.15.107:0/859001874 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1e1c10d8d0 0x7f1e1c1a28f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:05:02.983 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:03.002+0000 7f1e1a000640 1 -- 172.21.15.107:0/859001874 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1e1c10d8d0 msgr2=0x7f1e1c1a28f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:05:02.983 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:03.002+0000 7f1e1a000640 1 --2- 172.21.15.107:0/859001874 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1e1c10d8d0 0x7f1e1c1a28f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:05:02.983 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:03.002+0000 7f1e1a000640 1 -- 172.21.15.107:0/859001874 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1e1c1078c0 msgr2=0x7f1e1c19e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:05:02.983 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:03.002+0000 7f1e1a000640 1 --2- 172.21.15.107:0/859001874 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1e1c1078c0 0x7f1e1c19e080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:05:02.983 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:03.002+0000 7f1e1a000640 1 -- 172.21.15.107:0/859001874 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e1c1a3020 con 0x7f1e1c108280 2024-03-08T21:05:02.983 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:03.002+0000 7f1e1a000640 1 --2- 172.21.15.107:0/859001874 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1e1c108280 0x7f1e1c19e5c0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f1e08000c00 tx=0x7f1e0802c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:05:02.983 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:03.003+0000 7f1e13400640 1 -- 172.21.15.107:0/859001874 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e08043020 con 0x7f1e1c108280 2024-03-08T21:05:02.984 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:03.003+0000 7f1e13400640 1 -- 172.21.15.107:0/859001874 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f1e0802cc60 con 0x7f1e1c108280 2024-03-08T21:05:02.984 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:03.003+0000 7f1e21a00640 1 -- 172.21.15.107:0/859001874 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e1c1a32b0 con 0x7f1e1c108280 2024-03-08T21:05:02.984 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:03.003+0000 7f1e13400640 1 -- 172.21.15.107:0/859001874 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e08003c30 con 0x7f1e1c108280 2024-03-08T21:05:02.984 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:03.003+0000 7f1e21a00640 1 -- 172.21.15.107:0/859001874 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e1c1aaba0 con 0x7f1e1c108280 2024-03-08T21:05:02.986 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:03.007+0000 7f1e2477f740 1 Processor -- start 2024-03-08T21:05:02.987 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:03.007+0000 7f1e2477f740 1 -- start start 2024-03-08T21:05:02.988 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:03.007+0000 7f1e13400640 1 -- 172.21.15.107:0/859001874 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) v1 ==== 137422+0+0 (secure 0 0 0) 0x7f1e08046020 con 0x7f1e1c108280 2024-03-08T21:05:02.988 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:03.007+0000 7f1e2477f740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5574ddc34c80 0x5574ddc35060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:05:02.988 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:03.007+0000 7f1e2477f740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5574ddc355a0 0x5574ddc3de30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:05:02.988 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:03.007+0000 7f1e2477f740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5574ddc3e370 0x5574ddc40760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:05:02.988 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:03.007+0000 7f1e2477f740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574ddb983d0 con 0x5574ddc355a0 2024-03-08T21:05:02.988 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:03.007+0000 7f1e2477f740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574ddb416b0 con 0x5574ddc3e370 2024-03-08T21:05:02.989 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:03.008+0000 7f1e2477f740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574ddb94470 con 0x5574ddc34c80 2024-03-08T21:05:02.989 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:03.008+0000 7f1e1aa00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5574ddc355a0 0x5574ddc3de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:05:02.989 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:03.008+0000 7f1e1aa00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5574ddc355a0 0x5574ddc3de30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:50766/0 (socket says 172.21.15.107:50766) 2024-03-08T21:05:02.989 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:03.008+0000 7f1e1aa00640 1 -- 172.21.15.107:0/1954352316 learned_addr learned my addr 172.21.15.107:0/1954352316 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:05:02.989 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:03.008+0000 7f1e1a000640 1 --2- 172.21.15.107:0/1954352316 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5574ddc3e370 0x5574ddc40760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:05:02.989 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:03.008+0000 7f1e1b400640 1 --2- 172.21.15.107:0/1954352316 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5574ddc34c80 0x5574ddc35060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:05:02.989 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:03.008+0000 7f1e1aa00640 1 -- 172.21.15.107:0/1954352316 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5574ddc34c80 msgr2=0x5574ddc35060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:05:02.989 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:03.008+0000 7f1e1aa00640 1 --2- 172.21.15.107:0/1954352316 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5574ddc34c80 0x5574ddc35060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:05:02.989 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:03.008+0000 7f1e1aa00640 1 -- 172.21.15.107:0/1954352316 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5574ddc3e370 msgr2=0x5574ddc40760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:05:02.989 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:03.008+0000 7f1e1aa00640 1 --2- 172.21.15.107:0/1954352316 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5574ddc3e370 0x5574ddc40760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:05:02.989 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:03.008+0000 7f1e1aa00640 1 -- 172.21.15.107:0/1954352316 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5574ddb67b50 con 0x5574ddc355a0 2024-03-08T21:05:02.989 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:03.008+0000 7f1e1a000640 1 --2- 172.21.15.107:0/1954352316 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5574ddc3e370 0x5574ddc40760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T21:05:02.992 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:03.009+0000 7f1e1aa00640 1 --2- 172.21.15.107:0/1954352316 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5574ddc355a0 0x5574ddc3de30 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f1e0c00bc10 tx=0x7f1e0c00f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41a23684facb1b5e server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:05:02.992 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:03.009+0000 7f1e07e00640 1 -- 172.21.15.107:0/1954352316 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e0c015070 con 0x5574ddc355a0 2024-03-08T21:05:02.992 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:03.009+0000 7f1e13400640 1 --2- 172.21.15.107:0/859001874 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f1e000a2a90 0x7f1e000a4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:05:02.993 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:03.009+0000 7f1e07e00640 1 -- 172.21.15.107:0/1954352316 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f1e0c01a910 con 0x5574ddc355a0 2024-03-08T21:05:02.993 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:03.009+0000 7f1e07e00640 1 -- 172.21.15.107:0/1954352316 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e0c0117c0 con 0x5574ddc355a0 2024-03-08T21:05:02.993 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:03.009+0000 7f1e13400640 1 -- 172.21.15.107:0/859001874 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f1e080f1040 con 0x7f1e1c108280 2024-03-08T21:05:02.993 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:03.010+0000 7f1e1b400640 1 --2- 172.21.15.107:0/859001874 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f1e000a2a90 0x7f1e000a4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:05:02.993 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:03.010+0000 7f1e2477f740 1 -- 172.21.15.107:0/1954352316 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5574ddc355a0 msgr2=0x5574ddc3de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:05:02.993 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:03.010+0000 7f1e2477f740 1 --2- 172.21.15.107:0/1954352316 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5574ddc355a0 0x5574ddc3de30 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f1e0c00bc10 tx=0x7f1e0c00f730 comp rx=0 tx=0).stop 2024-03-08T21:05:02.993 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:03.010+0000 7f1e1b400640 1 --2- 172.21.15.107:0/859001874 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f1e000a2a90 0x7f1e000a4f50 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f1e14005fd0 tx=0x7f1e14009580 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:05:02.993 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:03.010+0000 7f1e2477f740 1 -- 172.21.15.107:0/1954352316 shutdown_connections 2024-03-08T21:05:02.993 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:03.010+0000 7f1e2477f740 1 --2- 172.21.15.107:0/1954352316 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5574ddc3e370 0x5574ddc40760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:05:02.993 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:03.010+0000 7f1e2477f740 1 --2- 172.21.15.107:0/1954352316 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5574ddc355a0 0x5574ddc3de30 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:05:02.993 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:03.010+0000 7f1e2477f740 1 --2- 172.21.15.107:0/1954352316 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5574ddc34c80 0x5574ddc35060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:05:02.993 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:03.010+0000 7f1e2477f740 1 -- 172.21.15.107:0/1954352316 >> 172.21.15.107:0/1954352316 conn(0x5574ddc2f5a0 msgr2=0x5574ddc2f980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:05:02.993 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:03.011+0000 7f1e2477f740 1 -- 172.21.15.107:0/1954352316 shutdown_connections 2024-03-08T21:05:02.994 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:03.011+0000 7f1e2477f740 1 -- 172.21.15.107:0/1954352316 wait complete. 2024-03-08T21:05:02.994 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:03.011+0000 7f1e2477f740 1 -- start start 2024-03-08T21:05:02.994 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:03.011+0000 7f1e2477f740 1 Processor -- start 2024-03-08T21:05:02.994 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:03.012+0000 7f1e2477f740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5574ddc34c80 0x5574ddc3dd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:05:02.994 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:03.012+0000 7f1e2477f740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5574ddc355a0 0x5574ddc3b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:05:02.994 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:03.012+0000 7f1e2477f740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5574ddc3e370 0x5574ddc3b610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:05:02.994 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:03.012+0000 7f1e2477f740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574ddc40fa0 con 0x5574ddc3e370 2024-03-08T21:05:02.994 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:03.012+0000 7f1e2477f740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574ddb549f0 con 0x5574ddc355a0 2024-03-08T21:05:02.994 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:03.012+0000 7f1e2477f740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574ddc41120 con 0x5574ddc34c80 2024-03-08T21:05:02.994 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:03.012+0000 7f1e2477f740 10 client.0 _ll_register_callbacks cb 0x5574dd8aca00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T21:05:02.994 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:03.012+0000 7f1e1b400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5574ddc3e370 0x5574ddc3b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:05:02.994 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:03.012+0000 7f1e1b400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5574ddc3e370 0x5574ddc3b610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:50780/0 (socket says 172.21.15.107:50780) 2024-03-08T21:05:02.994 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:03.012+0000 7f1e1b400640 1 -- 172.21.15.107:0/171174986 learned_addr learned my addr 172.21.15.107:0/171174986 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:05:02.995 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:03.012+0000 7f1e1a000640 1 --2- 172.21.15.107:0/171174986 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5574ddc355a0 0x5574ddc3b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:05:02.995 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:03.012+0000 7f1e1aa00640 1 --2- 172.21.15.107:0/171174986 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5574ddc34c80 0x5574ddc3dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:05:02.995 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:03.013+0000 7f1e1b400640 1 -- 172.21.15.107:0/171174986 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5574ddc34c80 msgr2=0x5574ddc3dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:05:02.995 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:03.013+0000 7f1e1b400640 1 --2- 172.21.15.107:0/171174986 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5574ddc34c80 0x5574ddc3dd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:05:02.995 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:03.013+0000 7f1e1b400640 1 -- 172.21.15.107:0/171174986 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5574ddc355a0 msgr2=0x5574ddc3b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:05:02.995 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:03.013+0000 7f1e1b400640 1 --2- 172.21.15.107:0/171174986 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5574ddc355a0 0x5574ddc3b090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:05:02.995 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:03.013+0000 7f1e1b400640 1 -- 172.21.15.107:0/171174986 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5574dd8ad210 con 0x5574ddc3e370 2024-03-08T21:05:02.995 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:03.013+0000 7f1e1a000640 1 --2- 172.21.15.107:0/171174986 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5574ddc355a0 0x5574ddc3b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T21:05:02.996 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:03.013+0000 7f1e1b400640 1 --2- 172.21.15.107:0/171174986 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5574ddc3e370 0x5574ddc3b610 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f1e14012890 tx=0x7f1e14004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f86752d72d4609bb server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:05:02.997 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:03.013+0000 7f1e04c00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-03-08T21:05:02.997 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:03.013+0000 7f1e04c00640 1 -- 172.21.15.107:0/171174986 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e14004d20 con 0x5574ddc3e370 2024-03-08T21:05:02.997 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:03.013+0000 7f1e04c00640 1 -- 172.21.15.107:0/171174986 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f1e1400eae0 con 0x5574ddc3e370 2024-03-08T21:05:02.997 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:03.014+0000 7f1e2477f740 1 -- 172.21.15.107:0/171174986 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1e0c01a910 con 0x5574ddc3e370 2024-03-08T21:05:02.997 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:03.014+0000 7f1e04c00640 1 -- 172.21.15.107:0/171174986 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1e14012e10 con 0x5574ddc3e370 2024-03-08T21:05:02.997 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:03.014+0000 7f1e2477f740 10 client.15834 fetch_fsmap learned FSMap version 32 2024-03-08T21:05:02.997 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:03.014+0000 7f1e2477f740 1 -- 172.21.15.107:0/171174986 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x5574dd979620 con 0x5574ddc3e370 2024-03-08T21:05:02.997 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:03.014+0000 7f1e04c00640 1 -- 172.21.15.107:0/171174986 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f1e0c01a910 con 0x5574ddc3e370 2024-03-08T21:05:02.997 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:03.014+0000 7f1e04c00640 1 -- 172.21.15.107:0/171174986 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 32) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f1e1400f040 con 0x5574ddc3e370 2024-03-08T21:05:02.997 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:03.014+0000 7f1e2477f740 10 client.15834 fetch_fsmap finished waiting for FSMap version 32 2024-03-08T21:05:02.997 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:03.014+0000 7f1e2477f740 10 client.15834 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-08T21:05:02.997 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:03.015+0000 7f1e2477f740 20 client.15834 populate_metadata read hostname 'smithi107' 2024-03-08T21:05:02.997 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:03.015+0000 7f1e2477f740 1 --2- 172.21.15.107:0/171174986 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x5574ddc4e240 0x5574ddc50630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:05:02.997 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:03.015+0000 7f1e2477f740 4 client.15834 mds_command: new command op to 34475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-08T21:05:02.998 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:03.015+0000 7f1e2477f740 1 -- 172.21.15.107:0/171174986 --> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5574ddba5910 con 0x5574ddc4e240 2024-03-08T21:05:02.998 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:03.015+0000 7f1e1aa00640 1 --2- 172.21.15.107:0/171174986 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x5574ddc4e240 0x5574ddc50630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:05:02.998 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:03.016+0000 7f1e1aa00640 1 --2- 172.21.15.107:0/171174986 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x5574ddc4e240 0x5574ddc50630 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=602e46e77baa5eac server_cookie=d4962acfcb543aad in_seq=0 out_seq=0 2024-03-08T21:05:02.998 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:03.016+0000 7f1e04c00640 10 client.15834 ms_handle_connect on v2:172.21.15.149:6838/55290596 2024-03-08T21:05:02.999 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:03.018+0000 7f1e04c00640 1 -- 172.21.15.107:0/171174986 <== mds.0 v2:172.21.15.149:6838/55290596 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5574ddba5910 con 0x5574ddc4e240 2024-03-08T21:05:02.999 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:03.018+0000 7f1e04c00640 10 client.15834 handle_command_reply: tid=0 2024-03-08T21:05:02.999 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:03.018+0000 7f1e2477f740 1 client.15834 shutdown 2024-03-08T21:05:02.999 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:03.018+0000 7f1e2477f740 1 -- 172.21.15.107:0/171174986 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5574ddc3e370 msgr2=0x5574ddc3b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:05:02.999 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:03.018+0000 7f1e2477f740 1 --2- 172.21.15.107:0/171174986 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5574ddc3e370 0x5574ddc3b610 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f1e14012890 tx=0x7f1e14004a40 comp rx=0 tx=0).stop 2024-03-08T21:05:02.999 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:03.018+0000 7f1e2477f740 1 -- 172.21.15.107:0/171174986 shutdown_connections 2024-03-08T21:05:02.999 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:03.018+0000 7f1e2477f740 1 --2- 172.21.15.107:0/171174986 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5574ddc3e370 0x5574ddc3b610 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:05:02.999 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:03.018+0000 7f1e2477f740 1 --2- 172.21.15.107:0/171174986 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x5574ddc4e240 0x5574ddc50630 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:05:02.999 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:03.018+0000 7f1e2477f740 1 --2- 172.21.15.107:0/171174986 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5574ddc355a0 0x5574ddc3b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:05:02.999 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:03.018+0000 7f1e2477f740 1 --2- 172.21.15.107:0/171174986 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5574ddc34c80 0x5574ddc3dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:05:03.000 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:03.018+0000 7f1e04c00640 0 client.15834 ms_handle_reset on v2:172.21.15.149:6838/55290596 2024-03-08T21:05:03.000 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:03.018+0000 7f1e2477f740 1 -- 172.21.15.107:0/171174986 >> 172.21.15.107:0/171174986 conn(0x5574ddc2f5a0 msgr2=0x5574ddc312f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:05:03.000 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:03.018+0000 7f1e2477f740 1 -- 172.21.15.107:0/171174986 shutdown_connections 2024-03-08T21:05:03.000 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:03.018+0000 7f1e2477f740 1 -- 172.21.15.107:0/171174986 wait complete. 2024-03-08T21:05:03.000 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:03.019+0000 7f1e2477f740 20 client.15834 trim_cache size 0 max 16384 2024-03-08T21:05:03.015 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:03.034+0000 7f1e2477f740 1 Processor -- start 2024-03-08T21:05:03.015 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:03.034+0000 7f1e2477f740 1 -- start start 2024-03-08T21:05:03.015 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:03.034+0000 7f1e2477f740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5574ddc34c80 0x5574ddc310d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:05:03.015 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:03.035+0000 7f1e2477f740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5574ddc355a0 0x5574ddc3d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:05:03.015 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:03.035+0000 7f1e2477f740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5574ddc3e370 0x5574ddc3dd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:05:03.015 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:03.035+0000 7f1e2477f740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e1c1b31c0 con 0x5574ddc34c80 2024-03-08T21:05:03.015 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:03.035+0000 7f1e2477f740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e1c1b4b40 con 0x5574ddc355a0 2024-03-08T21:05:03.015 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:03.035+0000 7f1e2477f740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e1c1b3820 con 0x5574ddc3e370 2024-03-08T21:05:03.016 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:03.035+0000 7f1e1aa00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5574ddc34c80 0x5574ddc310d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:05:03.016 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:03.035+0000 7f1e1aa00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5574ddc34c80 0x5574ddc310d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:50784/0 (socket says 172.21.15.107:50784) 2024-03-08T21:05:03.016 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:03.035+0000 7f1e1aa00640 1 -- 172.21.15.107:0/3490500844 learned_addr learned my addr 172.21.15.107:0/3490500844 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:05:03.016 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:03.035+0000 7f1e1a000640 1 --2- 172.21.15.107:0/3490500844 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5574ddc355a0 0x5574ddc3d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:05:03.016 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:03.035+0000 7f1e1b400640 1 --2- 172.21.15.107:0/3490500844 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5574ddc3e370 0x5574ddc3dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:05:03.016 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:03.035+0000 7f1e1aa00640 1 -- 172.21.15.107:0/3490500844 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5574ddc3e370 msgr2=0x5574ddc3dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:05:03.016 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:03.035+0000 7f1e1aa00640 1 --2- 172.21.15.107:0/3490500844 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5574ddc3e370 0x5574ddc3dd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:05:03.016 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:03.035+0000 7f1e1aa00640 1 -- 172.21.15.107:0/3490500844 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5574ddc355a0 msgr2=0x5574ddc3d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:05:03.016 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:03.035+0000 7f1e1aa00640 1 --2- 172.21.15.107:0/3490500844 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5574ddc355a0 0x5574ddc3d7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:05:03.016 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:03.035+0000 7f1e1aa00640 1 -- 172.21.15.107:0/3490500844 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5574ddc2ae60 con 0x5574ddc34c80 2024-03-08T21:05:03.016 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:03.035+0000 7f1e1b400640 1 --2- 172.21.15.107:0/3490500844 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5574ddc3e370 0x5574ddc3dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T21:05:03.018 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:03.036+0000 7f1e1aa00640 1 --2- 172.21.15.107:0/3490500844 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5574ddc34c80 0x5574ddc310d0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f1e0c00bb30 tx=0x7f1e0c005690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef9343181bea74d1 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:05:03.018 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:03.036+0000 7f1e07400640 1 -- 172.21.15.107:0/3490500844 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e0c015050 con 0x5574ddc34c80 2024-03-08T21:05:03.018 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:03.036+0000 7f1e07400640 1 -- 172.21.15.107:0/3490500844 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f1e0c02c070 con 0x5574ddc34c80 2024-03-08T21:05:03.018 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:03.036+0000 7f1e07400640 1 -- 172.21.15.107:0/3490500844 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e0c027860 con 0x5574ddc34c80 2024-03-08T21:05:03.019 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:03.037+0000 7f1e2477f740 1 -- 172.21.15.107:0/3490500844 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5574ddc34c80 msgr2=0x5574ddc310d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:05:03.019 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:03.037+0000 7f1e2477f740 1 --2- 172.21.15.107:0/3490500844 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5574ddc34c80 0x5574ddc310d0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f1e0c00bb30 tx=0x7f1e0c005690 comp rx=0 tx=0).stop 2024-03-08T21:05:03.019 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:03.037+0000 7f1e2477f740 1 -- 172.21.15.107:0/3490500844 shutdown_connections 2024-03-08T21:05:03.019 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:03.037+0000 7f1e2477f740 1 --2- 172.21.15.107:0/3490500844 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5574ddc3e370 0x5574ddc3dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:05:03.019 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:03.037+0000 7f1e2477f740 1 --2- 172.21.15.107:0/3490500844 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5574ddc355a0 0x5574ddc3d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:05:03.019 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:03.037+0000 7f1e2477f740 1 --2- 172.21.15.107:0/3490500844 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5574ddc34c80 0x5574ddc310d0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:05:03.019 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:03.037+0000 7f1e2477f740 1 -- 172.21.15.107:0/3490500844 >> 172.21.15.107:0/3490500844 conn(0x5574ddc2f5a0 msgr2=0x5574ddc309c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:05:03.019 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:03.037+0000 7f1e2477f740 1 -- 172.21.15.107:0/3490500844 shutdown_connections 2024-03-08T21:05:03.019 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:03.037+0000 7f1e2477f740 1 -- 172.21.15.107:0/3490500844 wait complete. 2024-03-08T21:05:03.019 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:03.037+0000 7f1e2477f740 1 -- start start 2024-03-08T21:05:03.019 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:03.038+0000 7f1e2477f740 1 Processor -- start 2024-03-08T21:05:03.019 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:03.038+0000 7f1e2477f740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5574ddc34c80 0x5574ddc44710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:05:03.019 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:03.038+0000 7f1e2477f740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5574ddc355a0 0x5574ddc44c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:05:03.020 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:03.038+0000 7f1e2477f740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5574ddc3e370 0x5574ddc9e860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:05:03.020 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:03.038+0000 7f1e2477f740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e1c1b44e0 con 0x5574ddc3e370 2024-03-08T21:05:03.020 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:03.038+0000 7f1e2477f740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e1c1b3e80 con 0x5574ddc34c80 2024-03-08T21:05:03.020 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:03.038+0000 7f1e2477f740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e1c1b1460 con 0x5574ddc355a0 2024-03-08T21:05:03.020 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:03.038+0000 7f1e2477f740 10 client.0 _ll_register_callbacks cb 0x7f1e0c00b3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T21:05:03.020 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:03.038+0000 7f1e1b400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5574ddc3e370 0x5574ddc9e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:05:03.020 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:03.039+0000 7f1e1b400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5574ddc3e370 0x5574ddc9e860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:50800/0 (socket says 172.21.15.107:50800) 2024-03-08T21:05:03.020 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:03.039+0000 7f1e1b400640 1 -- 172.21.15.107:0/1734764862 learned_addr learned my addr 172.21.15.107:0/1734764862 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:05:03.020 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:03.039+0000 7f1e1a000640 1 --2- 172.21.15.107:0/1734764862 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5574ddc355a0 0x5574ddc44c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:05:03.022 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:03.039+0000 7f1e1aa00640 1 --2- 172.21.15.107:0/1734764862 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5574ddc34c80 0x5574ddc44710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:05:03.022 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:03.039+0000 7f1e1b400640 1 -- 172.21.15.107:0/1734764862 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5574ddc355a0 msgr2=0x5574ddc44c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:05:03.022 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:03.039+0000 7f1e1b400640 1 --2- 172.21.15.107:0/1734764862 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5574ddc355a0 0x5574ddc44c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:05:03.022 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:03.039+0000 7f1e1b400640 1 -- 172.21.15.107:0/1734764862 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5574ddc34c80 msgr2=0x5574ddc44710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:05:03.022 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:03.039+0000 7f1e1b400640 1 --2- 172.21.15.107:0/1734764862 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5574ddc34c80 0x5574ddc44710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:05:03.022 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:03.039+0000 7f1e1b400640 1 -- 172.21.15.107:0/1734764862 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5574ddc9ee50 con 0x5574ddc3e370 2024-03-08T21:05:03.022 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:03.039+0000 7f1e1b400640 1 --2- 172.21.15.107:0/1734764862 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5574ddc3e370 0x5574ddc9e860 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f1e14012010 tx=0x7f1e1400ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d21bc3e8f14b4b0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:05:03.022 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:03.039+0000 7f1e04c00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-03-08T21:05:03.022 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:03.040+0000 7f1e04c00640 1 -- 172.21.15.107:0/1734764862 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e1403e060 con 0x5574ddc3e370 2024-03-08T21:05:03.023 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:03.040+0000 7f1e04c00640 1 -- 172.21.15.107:0/1734764862 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f1e1401f990 con 0x5574ddc3e370 2024-03-08T21:05:03.023 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:03.040+0000 7f1e2477f740 1 -- 172.21.15.107:0/1734764862 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1e0c02c070 con 0x5574ddc3e370 2024-03-08T21:05:03.023 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:03.040+0000 7f1e04c00640 1 -- 172.21.15.107:0/1734764862 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1e14033040 con 0x5574ddc3e370 2024-03-08T21:05:03.023 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:03.040+0000 7f1e2477f740 10 client.15840 fetch_fsmap learned FSMap version 32 2024-03-08T21:05:03.023 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:03.040+0000 7f1e2477f740 1 -- 172.21.15.107:0/1734764862 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x5574ddca6150 con 0x5574ddc3e370 2024-03-08T21:05:03.023 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:03.040+0000 7f1e04c00640 1 -- 172.21.15.107:0/1734764862 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f1e0c02c070 con 0x5574ddc3e370 2024-03-08T21:05:03.023 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:03.040+0000 7f1e04c00640 1 -- 172.21.15.107:0/1734764862 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 32) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f1e14039920 con 0x5574ddc3e370 2024-03-08T21:05:03.023 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:03.041+0000 7f1e2477f740 10 client.15840 fetch_fsmap finished waiting for FSMap version 32 2024-03-08T21:05:03.023 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:03.041+0000 7f1e2477f740 10 client.15840 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-08T21:05:03.023 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:03.041+0000 7f1e2477f740 20 client.15840 populate_metadata read hostname 'smithi107' 2024-03-08T21:05:03.023 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:03.041+0000 7f1e2477f740 1 --2- 172.21.15.107:0/1734764862 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x5574ddca67c0 0x5574ddca8bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:05:03.023 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:03.041+0000 7f1e2477f740 4 client.15840 mds_command: new command op to 34475 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-08T21:05:03.023 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:03.041+0000 7f1e2477f740 1 -- 172.21.15.107:0/1734764862 --> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5574ddb9fc60 con 0x5574ddca67c0 2024-03-08T21:05:03.023 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:03.041+0000 7f1e1aa00640 1 --2- 172.21.15.107:0/1734764862 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x5574ddca67c0 0x5574ddca8bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:05:03.024 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:03.042+0000 7f1e1aa00640 1 --2- 172.21.15.107:0/1734764862 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x5574ddca67c0 0x5574ddca8bb0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1c0af4fa36875e54 server_cookie=b6f6186bd1a51c15 in_seq=0 out_seq=0 2024-03-08T21:05:03.024 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:03.042+0000 7f1e04c00640 10 client.15840 ms_handle_connect on v2:172.21.15.149:6838/55290596 2024-03-08T21:05:03.927 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:05:03 smithi204 ceph-mon[34935]: pgmap v140: 97 pgs: 97 active+clean; 2.7 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:05:03.964 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:05:03 smithi107 ceph-mon[31381]: pgmap v140: 97 pgs: 97 active+clean; 2.7 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:05:04.012 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:05:03 smithi149 ceph-mon[35042]: pgmap v140: 97 pgs: 97 active+clean; 2.7 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:05:04.185 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:04.205+0000 7f1e04c00640 1 -- 172.21.15.107:0/1734764862 <== mds.0 v2:172.21.15.149:6838/55290596 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5574ddb9fc60 con 0x5574ddca67c0 2024-03-08T21:05:04.186 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:04.205+0000 7f1e04c00640 10 client.15840 handle_command_reply: tid=0 2024-03-08T21:05:04.186 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:04.206+0000 7f1e2477f740 1 client.15840 shutdown 2024-03-08T21:05:04.187 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:04.206+0000 7f1e2477f740 1 -- 172.21.15.107:0/1734764862 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5574ddc3e370 msgr2=0x5574ddc9e860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:05:04.187 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:04.206+0000 7f1e2477f740 1 --2- 172.21.15.107:0/1734764862 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5574ddc3e370 0x5574ddc9e860 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f1e14012010 tx=0x7f1e1400ed70 comp rx=0 tx=0).stop 2024-03-08T21:05:04.187 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:04.206+0000 7f1e2477f740 1 -- 172.21.15.107:0/1734764862 shutdown_connections 2024-03-08T21:05:04.187 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:04.206+0000 7f1e2477f740 1 --2- 172.21.15.107:0/1734764862 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5574ddc3e370 0x5574ddc9e860 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:05:04.187 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:04.206+0000 7f1e2477f740 1 --2- 172.21.15.107:0/1734764862 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5574ddc355a0 0x5574ddc44c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:05:04.187 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:04.206+0000 7f1e2477f740 1 --2- 172.21.15.107:0/1734764862 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x5574ddca67c0 0x5574ddca8bb0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:05:04.187 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:04.206+0000 7f1e2477f740 1 --2- 172.21.15.107:0/1734764862 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5574ddc34c80 0x5574ddc44710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:05:04.187 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:04.206+0000 7f1e2477f740 1 -- 172.21.15.107:0/1734764862 >> 172.21.15.107:0/1734764862 conn(0x5574ddc2f5a0 msgr2=0x5574ddc2de30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:05:04.187 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:04.207+0000 7f1e04c00640 0 client.15840 ms_handle_reset on v2:172.21.15.149:6838/55290596 2024-03-08T21:05:04.188 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:04.207+0000 7f1e2477f740 1 -- 172.21.15.107:0/1734764862 shutdown_connections 2024-03-08T21:05:04.188 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:04.207+0000 7f1e2477f740 1 -- 172.21.15.107:0/1734764862 wait complete. 2024-03-08T21:05:04.188 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:04.207+0000 7f1e2477f740 20 client.15840 trim_cache size 0 max 16384 2024-03-08T21:05:04.188 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:{ 2024-03-08T21:05:04.188 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "message": "", 2024-03-08T21:05:04.188 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "return_code": 0 2024-03-08T21:05:04.188 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:} 2024-03-08T21:05:04.189 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:04.208+0000 7f1e21a00640 1 -- 172.21.15.107:0/859001874 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f1e000a2a90 msgr2=0x7f1e000a4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:05:04.189 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:04.208+0000 7f1e21a00640 1 --2- 172.21.15.107:0/859001874 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f1e000a2a90 0x7f1e000a4f50 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f1e14005fd0 tx=0x7f1e14009580 comp rx=0 tx=0).stop 2024-03-08T21:05:04.189 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:04.208+0000 7f1e21a00640 1 -- 172.21.15.107:0/859001874 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1e1c108280 msgr2=0x7f1e1c19e5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:05:04.189 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:04.208+0000 7f1e21a00640 1 --2- 172.21.15.107:0/859001874 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1e1c108280 0x7f1e1c19e5c0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f1e08000c00 tx=0x7f1e0802c860 comp rx=0 tx=0).stop 2024-03-08T21:05:04.189 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:04.209+0000 7f1e21a00640 1 -- 172.21.15.107:0/859001874 shutdown_connections 2024-03-08T21:05:04.189 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:04.209+0000 7f1e21a00640 1 --2- 172.21.15.107:0/859001874 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f1e000a2a90 0x7f1e000a4f50 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:05:04.190 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:04.209+0000 7f1e21a00640 1 --2- 172.21.15.107:0/859001874 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1e1c10d8d0 0x7f1e1c1a28f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:05:04.190 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:04.209+0000 7f1e21a00640 1 --2- 172.21.15.107:0/859001874 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1e1c108280 0x7f1e1c19e5c0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:05:04.190 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:04.209+0000 7f1e21a00640 1 --2- 172.21.15.107:0/859001874 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1e1c1078c0 0x7f1e1c19e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:05:04.190 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:04.209+0000 7f1e21a00640 1 -- 172.21.15.107:0/859001874 >> 172.21.15.107:0/859001874 conn(0x7f1e1c075860 msgr2=0x7f1e1c075f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:05:04.190 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:04.209+0000 7f1e21a00640 1 -- 172.21.15.107:0/859001874 shutdown_connections 2024-03-08T21:05:04.190 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:04.209+0000 7f1e21a00640 1 -- 172.21.15.107:0/859001874 wait complete. 2024-03-08T21:05:05.377 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:05:04 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[70281]: *** Caught signal (Segmentation fault) ** 2024-03-08T21:05:05.377 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:05:04 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[70281]: in thread 7f355c800700 thread_name:md_log_replay 2024-03-08T21:05:05.378 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:05:04 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[70281]: ceph version 19.0.0-1965-g7d63482a (7d63482aa89628d733e298984a762389b625c761) squid (dev) 2024-03-08T21:05:05.378 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:05:04 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[70281]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f3570e51d20] 2024-03-08T21:05:05.378 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:05:04 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[70281]: 2: /usr/bin/ceph-mds(+0x503eff) [0x557bc385eeff] 2024-03-08T21:05:05.378 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:05:04 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[70281]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x557bc3862773] 2024-03-08T21:05:05.378 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:05:04 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[70281]: 4: (MDLog::_replay_thread()+0x617) [0x557bc38be587] 2024-03-08T21:05:05.378 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:05:04 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[70281]: 5: (MDLog::ReplayThread::entry()+0x11) [0x557bc38c7091] 2024-03-08T21:05:05.378 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:05:04 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[70281]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7f3570e471ca] 2024-03-08T21:05:05.378 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:05:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[70281]: 7: clone() 2024-03-08T21:05:05.378 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:05:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[70281]: 2024-03-08T21:05:04.999+0000 7f355c800700 -1 *** Caught signal (Segmentation fault) ** 2024-03-08T21:05:05.378 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:05:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[70281]: in thread 7f355c800700 thread_name:md_log_replay 2024-03-08T21:05:05.378 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:05:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[70281]: 2024-03-08T21:05:05.378 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:05:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[70281]: ceph version 19.0.0-1965-g7d63482a (7d63482aa89628d733e298984a762389b625c761) squid (dev) 2024-03-08T21:05:05.378 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:05:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[70281]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f3570e51d20] 2024-03-08T21:05:05.379 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:05:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[70281]: 2: /usr/bin/ceph-mds(+0x503eff) [0x557bc385eeff] 2024-03-08T21:05:05.379 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:05:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[70281]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x557bc3862773] 2024-03-08T21:05:05.379 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:05:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[70281]: 4: (MDLog::_replay_thread()+0x617) [0x557bc38be587] 2024-03-08T21:05:05.379 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:05:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[70281]: 5: (MDLog::ReplayThread::entry()+0x11) [0x557bc38c7091] 2024-03-08T21:05:05.379 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:05:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[70281]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7f3570e471ca] 2024-03-08T21:05:05.379 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:05:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[70281]: 7: clone() 2024-03-08T21:05:05.379 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:05:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[70281]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-08T21:05:05.379 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:05:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[70281]: 2024-03-08T21:05:05.379 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:05:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[70281]: 0> 2024-03-08T21:05:04.999+0000 7f355c800700 -1 *** Caught signal (Segmentation fault) ** 2024-03-08T21:05:05.379 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:05:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[70281]: in thread 7f355c800700 thread_name:md_log_replay 2024-03-08T21:05:05.379 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:05:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[70281]: 2024-03-08T21:05:05.379 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:05:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[70281]: ceph version 19.0.0-1965-g7d63482a (7d63482aa89628d733e298984a762389b625c761) squid (dev) 2024-03-08T21:05:05.379 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:05:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[70281]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f3570e51d20] 2024-03-08T21:05:05.379 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:05:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[70281]: 2: /usr/bin/ceph-mds(+0x503eff) [0x557bc385eeff] 2024-03-08T21:05:05.380 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:05:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[70281]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x557bc3862773] 2024-03-08T21:05:05.380 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:05:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[70281]: 4: (MDLog::_replay_thread()+0x617) [0x557bc38be587] 2024-03-08T21:05:05.380 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:05:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[70281]: 5: (MDLog::ReplayThread::entry()+0x11) [0x557bc38c7091] 2024-03-08T21:05:05.380 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:05:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[70281]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7f3570e471ca] 2024-03-08T21:05:05.380 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:05:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[70281]: 7: clone() 2024-03-08T21:05:05.380 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:05:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[70281]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-08T21:05:05.380 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:05:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[70281]: 2024-03-08T21:05:05.380 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:05:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[70281]: 0> 2024-03-08T21:05:04.999+0000 7f355c800700 -1 *** Caught signal (Segmentation fault) ** 2024-03-08T21:05:05.380 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:05:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[70281]: in thread 7f355c800700 thread_name:md_log_replay 2024-03-08T21:05:05.380 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:05:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[70281]: 2024-03-08T21:05:05.381 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:05:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[70281]: ceph version 19.0.0-1965-g7d63482a (7d63482aa89628d733e298984a762389b625c761) squid (dev) 2024-03-08T21:05:05.381 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:05:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[70281]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f3570e51d20] 2024-03-08T21:05:05.381 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:05:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[70281]: 2: /usr/bin/ceph-mds(+0x503eff) [0x557bc385eeff] 2024-03-08T21:05:05.381 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:05:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[70281]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x557bc3862773] 2024-03-08T21:05:05.381 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:05:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[70281]: 4: (MDLog::_replay_thread()+0x617) [0x557bc38be587] 2024-03-08T21:05:05.381 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:05:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[70281]: 5: (MDLog::ReplayThread::entry()+0x11) [0x557bc38c7091] 2024-03-08T21:05:05.381 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:05:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[70281]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7f3570e471ca] 2024-03-08T21:05:05.381 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:05:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[70281]: 7: clone() 2024-03-08T21:05:05.381 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:05:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[70281]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-08T21:05:05.381 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:05:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[70281]: 2024-03-08T21:05:05.381 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:05:05 smithi204 podman[70588]: 2024-03-08 21:05:05.111333583 +0000 UTC m=+0.038632666 container died d32e1fc0553409c051f342b3038b598a11012caf40e3811e5b7525ee00ca24fd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=rishabh-2024mar8-7d63482, org.label-schema.schema-version=1.0) 2024-03-08T21:05:05.745 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:05:05 smithi204 ceph-mon[34935]: pgmap v141: 97 pgs: 97 active+clean; 3.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 76 KiB/s wr, 27 op/s 2024-03-08T21:05:05.964 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:05:05 smithi107 ceph-mon[31381]: pgmap v141: 97 pgs: 97 active+clean; 3.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 76 KiB/s wr, 27 op/s 2024-03-08T21:05:06.012 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:05:05 smithi149 ceph-mon[35042]: pgmap v141: 97 pgs: 97 active+clean; 3.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 76 KiB/s wr, 27 op/s 2024-03-08T21:05:06.125 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:05:05 smithi204 podman[70588]: 2024-03-08 21:05:05.76360385 +0000 UTC m=+0.690902990 container remove d32e1fc0553409c051f342b3038b598a11012caf40e3811e5b7525ee00ca24fd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=rishabh-2024mar8-7d63482, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2) 2024-03-08T21:05:06.125 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:05:05 smithi204 systemd[1]: ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715@mds.c.service: Main process exited, code=exited, status=139/n/a 2024-03-08T21:05:06.625 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:05:06 smithi204 systemd[1]: ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715@mds.c.service: Failed with result 'exit-code'. 2024-03-08T21:05:06.625 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:05:06 smithi204 systemd[1]: ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715@mds.c.service: Consumed 1.845s CPU time. 2024-03-08T21:05:07.964 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:05:07 smithi107 ceph-mon[31381]: pgmap v142: 97 pgs: 97 active+clean; 3.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 76 KiB/s wr, 23 op/s 2024-03-08T21:05:08.012 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:05:07 smithi149 ceph-mon[35042]: pgmap v142: 97 pgs: 97 active+clean; 3.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 76 KiB/s wr, 23 op/s 2024-03-08T21:05:08.125 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:05:07 smithi204 ceph-mon[34935]: pgmap v142: 97 pgs: 97 active+clean; 3.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 76 KiB/s wr, 23 op/s 2024-03-08T21:05:09.964 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:05:09 smithi107 ceph-mon[31381]: pgmap v143: 97 pgs: 97 active+clean; 3.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 79 KiB/s wr, 29 op/s 2024-03-08T21:05:10.012 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:05:09 smithi149 ceph-mon[35042]: pgmap v143: 97 pgs: 97 active+clean; 3.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 79 KiB/s wr, 29 op/s 2024-03-08T21:05:10.125 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:05:09 smithi204 ceph-mon[34935]: pgmap v143: 97 pgs: 97 active+clean; 3.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 79 KiB/s wr, 29 op/s 2024-03-08T21:05:10.964 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:05:10 smithi107 ceph-mon[31381]: pgmap v144: 97 pgs: 97 active+clean; 3.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 79 KiB/s wr, 27 op/s 2024-03-08T21:05:11.013 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:05:10 smithi149 ceph-mon[35042]: pgmap v144: 97 pgs: 97 active+clean; 3.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 79 KiB/s wr, 27 op/s 2024-03-08T21:05:11.125 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:05:10 smithi204 ceph-mon[34935]: pgmap v144: 97 pgs: 97 active+clean; 3.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 79 KiB/s wr, 27 op/s 2024-03-08T21:05:13.964 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:05:13 smithi107 ceph-mon[31381]: pgmap v145: 97 pgs: 97 active+clean; 3.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 79 KiB/s wr, 25 op/s 2024-03-08T21:05:14.012 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:05:13 smithi149 ceph-mon[35042]: pgmap v145: 97 pgs: 97 active+clean; 3.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 79 KiB/s wr, 25 op/s 2024-03-08T21:05:14.125 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:05:13 smithi204 ceph-mon[34935]: pgmap v145: 97 pgs: 97 active+clean; 3.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 79 KiB/s wr, 25 op/s 2024-03-08T21:05:15.964 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:05:15 smithi107 ceph-mon[31381]: pgmap v146: 97 pgs: 97 active+clean; 3.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 79 KiB/s wr, 28 op/s 2024-03-08T21:05:16.012 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:05:15 smithi149 ceph-mon[35042]: pgmap v146: 97 pgs: 97 active+clean; 3.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 79 KiB/s wr, 28 op/s 2024-03-08T21:05:16.125 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:05:15 smithi204 ceph-mon[34935]: pgmap v146: 97 pgs: 97 active+clean; 3.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 79 KiB/s wr, 28 op/s 2024-03-08T21:05:16.768 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:05:16 smithi204 systemd[1]: ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715@mds.c.service: Scheduled restart job, restart counter is at 3. 2024-03-08T21:05:16.768 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:05:16 smithi204 systemd[1]: Stopped Ceph mds.c for dc20f0f8-dd8c-11ee-95c6-87774f69a715. 2024-03-08T21:05:16.769 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:05:16 smithi204 systemd[1]: ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715@mds.c.service: Consumed 1.845s CPU time. 2024-03-08T21:05:16.769 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:05:16 smithi204 systemd[1]: Starting Ceph mds.c for dc20f0f8-dd8c-11ee-95c6-87774f69a715... 2024-03-08T21:05:17.115 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:05:16 smithi204 podman[70680]: 2024-03-08 21:05:16.694475981 +0000 UTC m=+0.035279263 image pull a272146cb810b1fd4a53627f019472132e30966d5c25b01bfc44a001295f1249 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd 2024-03-08T21:05:17.115 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:05:16 smithi204 podman[70680]: 2024-03-08 21:05:16.822682898 +0000 UTC m=+0.163486209 container create bdc11ef2f39f583a2df227a3d6e704b46acfad788012076752a4bf7ab88301c5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=rishabh-2024mar8-7d63482, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD) 2024-03-08T21:05:17.354 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.373+0000 7f5f99000640 1 -- 172.21.15.107:0/2834632745 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5f9410b6c0 msgr2=0x7f5f9410db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:05:17.354 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.373+0000 7f5f99000640 1 --2- 172.21.15.107:0/2834632745 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5f9410b6c0 0x7f5f9410db60 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f5f8c009900 tx=0x7f5f8c02bcc0 comp rx=0 tx=0).stop 2024-03-08T21:05:17.354 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.373+0000 7f5f99000640 1 -- 172.21.15.107:0/2834632745 shutdown_connections 2024-03-08T21:05:17.354 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.373+0000 7f5f99000640 1 --2- 172.21.15.107:0/2834632745 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5f9410b6c0 0x7f5f9410db60 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:05:17.354 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.373+0000 7f5f99000640 1 --2- 172.21.15.107:0/2834632745 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5f94102bc0 0x7f5f9410b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:05:17.354 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.373+0000 7f5f99000640 1 --2- 172.21.15.107:0/2834632745 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5f941022a0 0x7f5f94102680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:05:17.355 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.373+0000 7f5f99000640 1 -- 172.21.15.107:0/2834632745 >> 172.21.15.107:0/2834632745 conn(0x7f5f940fbb00 msgr2=0x7f5f940fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:05:17.355 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.373+0000 7f5f99000640 1 -- 172.21.15.107:0/2834632745 shutdown_connections 2024-03-08T21:05:17.355 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.373+0000 7f5f99000640 1 -- 172.21.15.107:0/2834632745 wait complete. 2024-03-08T21:05:17.357 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.374+0000 7f5f99000640 1 Processor -- start 2024-03-08T21:05:17.357 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.374+0000 7f5f99000640 1 -- start start 2024-03-08T21:05:17.357 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.374+0000 7f5f99000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5f941022a0 0x7f5f9419d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:05:17.357 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.374+0000 7f5f99000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5f94102bc0 0x7f5f9419dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:05:17.357 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.374+0000 7f5f99000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5f9410b6c0 0x7f5f941977f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:05:17.357 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.374+0000 7f5f99000640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f94111400 con 0x7f5f941022a0 2024-03-08T21:05:17.357 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.374+0000 7f5f99000640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f94111280 con 0x7f5f94102bc0 2024-03-08T21:05:17.357 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.374+0000 7f5f99000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f94111580 con 0x7f5f9410b6c0 2024-03-08T21:05:17.357 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.375+0000 7f5f93400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5f941022a0 0x7f5f9419d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:05:17.357 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.375+0000 7f5f93400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5f941022a0 0x7f5f9419d680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:34622/0 (socket says 172.21.15.107:34622) 2024-03-08T21:05:17.357 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.375+0000 7f5f93400640 1 -- 172.21.15.107:0/2745896971 learned_addr learned my addr 172.21.15.107:0/2745896971 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:05:17.358 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.375+0000 7f5f93e00640 1 --2- 172.21.15.107:0/2745896971 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5f9410b6c0 0x7f5f941977f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:05:17.358 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.375+0000 7f5f93400640 1 -- 172.21.15.107:0/2745896971 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5f9410b6c0 msgr2=0x7f5f941977f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:05:17.358 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.375+0000 7f5f93400640 1 --2- 172.21.15.107:0/2745896971 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5f9410b6c0 0x7f5f941977f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:05:17.358 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.375+0000 7f5f93400640 1 -- 172.21.15.107:0/2745896971 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5f94102bc0 msgr2=0x7f5f9419dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:05:17.358 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.375+0000 7f5f92a00640 1 --2- 172.21.15.107:0/2745896971 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5f94102bc0 0x7f5f9419dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:05:17.358 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.375+0000 7f5f93400640 1 --2- 172.21.15.107:0/2745896971 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5f94102bc0 0x7f5f9419dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:05:17.358 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.375+0000 7f5f93400640 1 -- 172.21.15.107:0/2745896971 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f94197fc0 con 0x7f5f941022a0 2024-03-08T21:05:17.358 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.375+0000 7f5f93e00640 1 --2- 172.21.15.107:0/2745896971 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5f9410b6c0 0x7f5f941977f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T21:05:17.358 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.375+0000 7f5f92a00640 1 --2- 172.21.15.107:0/2745896971 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5f94102bc0 0x7f5f9419dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T21:05:17.358 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.376+0000 7f5f93400640 1 --2- 172.21.15.107:0/2745896971 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5f941022a0 0x7f5f9419d680 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f5f8400ba50 tx=0x7f5f8400bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:05:17.358 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.376+0000 7f5f8be00640 1 -- 172.21.15.107:0/2745896971 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f84019070 con 0x7f5f941022a0 2024-03-08T21:05:17.358 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.376+0000 7f5f8be00640 1 -- 172.21.15.107:0/2745896971 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f5f8400cbe0 con 0x7f5f941022a0 2024-03-08T21:05:17.358 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.376+0000 7f5f8be00640 1 -- 172.21.15.107:0/2745896971 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f840075d0 con 0x7f5f941022a0 2024-03-08T21:05:17.358 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.376+0000 7f5f99000640 1 -- 172.21.15.107:0/2745896971 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f941982b0 con 0x7f5f941022a0 2024-03-08T21:05:17.359 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.376+0000 7f5f99000640 1 -- 172.21.15.107:0/2745896971 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f941a44d0 con 0x7f5f941022a0 2024-03-08T21:05:17.360 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.380+0000 7f5f8be00640 1 -- 172.21.15.107:0/2745896971 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) v1 ==== 137422+0+0 (secure 0 0 0) 0x7f5f8401c070 con 0x7f5f941022a0 2024-03-08T21:05:17.360 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.380+0000 7f5f9d3b1740 1 Processor -- start 2024-03-08T21:05:17.361 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.380+0000 7f5f9d3b1740 1 -- start start 2024-03-08T21:05:17.363 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.381+0000 7f5f9d3b1740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55700fdd1e80 0x55700fdd2260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:05:17.363 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.381+0000 7f5f9d3b1740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55700fdd27a0 0x55700fddb030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:05:17.363 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.381+0000 7f5f9d3b1740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55700fddb570 0x55700fddd960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:05:17.363 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.381+0000 7f5f9d3b1740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55700fd353d0 con 0x55700fdd1e80 2024-03-08T21:05:17.363 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.381+0000 7f5f9d3b1740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55700fcde6b0 con 0x55700fddb570 2024-03-08T21:05:17.363 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.381+0000 7f5f9d3b1740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55700fd31470 con 0x55700fdd27a0 2024-03-08T21:05:17.363 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.381+0000 7f5f93e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55700fdd1e80 0x55700fdd2260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:05:17.363 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.381+0000 7f5f93e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55700fdd1e80 0x55700fdd2260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:34632/0 (socket says 172.21.15.107:34632) 2024-03-08T21:05:17.363 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.381+0000 7f5f93e00640 1 -- 172.21.15.107:0/920553998 learned_addr learned my addr 172.21.15.107:0/920553998 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:05:17.363 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.381+0000 7f5f8be00640 1 --2- 172.21.15.107:0/2745896971 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f5f780a29e0 0x7f5f780a4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:05:17.363 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.382+0000 7f5f93400640 1 --2- 172.21.15.107:0/920553998 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55700fdd27a0 0x55700fddb030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:05:17.364 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.382+0000 7f5f8be00640 1 -- 172.21.15.107:0/2745896971 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f5f840ce430 con 0x7f5f941022a0 2024-03-08T21:05:17.364 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.382+0000 7f5f92a00640 1 --2- 172.21.15.107:0/920553998 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55700fddb570 0x55700fddd960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:05:17.364 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.382+0000 7f5f93e00640 1 --2- 172.21.15.107:0/2745896971 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f5f780a29e0 0x7f5f780a4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:05:17.364 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.382+0000 7f5f93e00640 1 -- 172.21.15.107:0/920553998 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55700fdd27a0 msgr2=0x55700fddb030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:05:17.364 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.382+0000 7f5f93e00640 1 --2- 172.21.15.107:0/920553998 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55700fdd27a0 0x55700fddb030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:05:17.364 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.382+0000 7f5f93e00640 1 -- 172.21.15.107:0/920553998 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55700fddb570 msgr2=0x55700fddd960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:05:17.364 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.382+0000 7f5f93e00640 1 --2- 172.21.15.107:0/920553998 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55700fddb570 0x55700fddd960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:05:17.364 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.382+0000 7f5f93e00640 1 -- 172.21.15.107:0/920553998 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55700fd04b50 con 0x55700fdd1e80 2024-03-08T21:05:17.366 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.383+0000 7f5f93e00640 1 --2- 172.21.15.107:0/920553998 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55700fdd1e80 0x55700fdd2260 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f5f8c0096c0 tx=0x7f5f8c004560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48f4abdbcaac5161 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:05:17.366 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.383+0000 7f5f93e00640 1 --2- 172.21.15.107:0/2745896971 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f5f780a29e0 0x7f5f780a4ea0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f5f8c045000 tx=0x7f5f8c048000 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:05:17.366 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.383+0000 7f5f88c00640 1 -- 172.21.15.107:0/920553998 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f8c005100 con 0x55700fdd1e80 2024-03-08T21:05:17.366 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.383+0000 7f5f88c00640 1 -- 172.21.15.107:0/920553998 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f5f8c0046c0 con 0x55700fdd1e80 2024-03-08T21:05:17.366 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.383+0000 7f5f88c00640 1 -- 172.21.15.107:0/920553998 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f8c0037f0 con 0x55700fdd1e80 2024-03-08T21:05:17.366 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.384+0000 7f5f9d3b1740 1 -- 172.21.15.107:0/920553998 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55700fdd1e80 msgr2=0x55700fdd2260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:05:17.366 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.384+0000 7f5f9d3b1740 1 --2- 172.21.15.107:0/920553998 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55700fdd1e80 0x55700fdd2260 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f5f8c0096c0 tx=0x7f5f8c004560 comp rx=0 tx=0).stop 2024-03-08T21:05:17.366 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.384+0000 7f5f9d3b1740 1 -- 172.21.15.107:0/920553998 shutdown_connections 2024-03-08T21:05:17.366 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.384+0000 7f5f9d3b1740 1 --2- 172.21.15.107:0/920553998 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55700fddb570 0x55700fddd960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:05:17.367 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.384+0000 7f5f9d3b1740 1 --2- 172.21.15.107:0/920553998 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55700fdd27a0 0x55700fddb030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:05:17.367 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.384+0000 7f5f9d3b1740 1 --2- 172.21.15.107:0/920553998 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55700fdd1e80 0x55700fdd2260 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:05:17.367 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.384+0000 7f5f9d3b1740 1 -- 172.21.15.107:0/920553998 >> 172.21.15.107:0/920553998 conn(0x55700fdcc7a0 msgr2=0x55700fdccb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:05:17.367 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.384+0000 7f5f9d3b1740 1 -- 172.21.15.107:0/920553998 shutdown_connections 2024-03-08T21:05:17.367 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.384+0000 7f5f9d3b1740 1 -- 172.21.15.107:0/920553998 wait complete. 2024-03-08T21:05:17.367 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.385+0000 7f5f9d3b1740 1 -- start start 2024-03-08T21:05:17.367 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.385+0000 7f5f9d3b1740 1 Processor -- start 2024-03-08T21:05:17.370 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.386+0000 7f5f9d3b1740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55700fdd1e80 0x55700fddaf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:05:17.370 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.386+0000 7f5f9d3b1740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55700fdd27a0 0x55700fdd8310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:05:17.370 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.386+0000 7f5f9d3b1740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55700fddb570 0x55700fdd8920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:05:17.370 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.386+0000 7f5f9d3b1740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55700fdde140 con 0x55700fdd27a0 2024-03-08T21:05:17.371 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.386+0000 7f5f9d3b1740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55700fcf19f0 con 0x55700fdd1e80 2024-03-08T21:05:17.371 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.386+0000 7f5f9d3b1740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55700fdde2c0 con 0x55700fddb570 2024-03-08T21:05:17.371 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.386+0000 7f5f9d3b1740 10 client.0 _ll_register_callbacks cb 0x55700fa49a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T21:05:17.371 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.386+0000 7f5f92a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55700fdd27a0 0x55700fdd8310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:05:17.371 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.386+0000 7f5f92a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55700fdd27a0 0x55700fdd8310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:34634/0 (socket says 172.21.15.107:34634) 2024-03-08T21:05:17.371 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.386+0000 7f5f92a00640 1 -- 172.21.15.107:0/3223610587 learned_addr learned my addr 172.21.15.107:0/3223610587 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:05:17.371 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.386+0000 7f5f93400640 1 --2- 172.21.15.107:0/3223610587 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55700fdd1e80 0x55700fddaf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:05:17.371 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.386+0000 7f5f93e00640 1 --2- 172.21.15.107:0/3223610587 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55700fddb570 0x55700fdd8920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:05:17.371 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.386+0000 7f5f92a00640 1 -- 172.21.15.107:0/3223610587 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55700fddb570 msgr2=0x55700fdd8920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:05:17.371 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.386+0000 7f5f92a00640 1 --2- 172.21.15.107:0/3223610587 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55700fddb570 0x55700fdd8920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:05:17.371 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.387+0000 7f5f92a00640 1 -- 172.21.15.107:0/3223610587 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55700fdd1e80 msgr2=0x55700fddaf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:05:17.371 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.387+0000 7f5f92a00640 1 --2- 172.21.15.107:0/3223610587 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55700fdd1e80 0x55700fddaf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:05:17.371 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.387+0000 7f5f92a00640 1 -- 172.21.15.107:0/3223610587 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55700fa4a210 con 0x55700fdd27a0 2024-03-08T21:05:17.372 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.387+0000 7f5f93400640 1 --2- 172.21.15.107:0/3223610587 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55700fdd1e80 0x55700fddaf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T21:05:17.372 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.387+0000 7f5f92a00640 1 --2- 172.21.15.107:0/3223610587 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55700fdd27a0 0x55700fdd8310 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f5f800106a0 tx=0x7f5f80010b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8075b3c75d4258a server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:05:17.372 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.387+0000 7f5f7d600640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-03-08T21:05:17.372 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.387+0000 7f5f7d600640 1 -- 172.21.15.107:0/3223610587 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f80010ea0 con 0x55700fdd27a0 2024-03-08T21:05:17.372 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.387+0000 7f5f9d3b1740 1 -- 172.21.15.107:0/3223610587 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5f8c0046c0 con 0x55700fdd27a0 2024-03-08T21:05:17.372 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.387+0000 7f5f7d600640 1 -- 172.21.15.107:0/3223610587 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f5f80004220 con 0x55700fdd27a0 2024-03-08T21:05:17.372 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.388+0000 7f5f7d600640 1 -- 172.21.15.107:0/3223610587 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5f80023050 con 0x55700fdd27a0 2024-03-08T21:05:17.372 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.388+0000 7f5f9d3b1740 10 client.15855 fetch_fsmap learned FSMap version 32 2024-03-08T21:05:17.372 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.388+0000 7f5f9d3b1740 1 -- 172.21.15.107:0/3223610587 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55700fb16620 con 0x55700fdd27a0 2024-03-08T21:05:17.372 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.388+0000 7f5f7d600640 1 -- 172.21.15.107:0/3223610587 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f5f8c0046c0 con 0x55700fdd27a0 2024-03-08T21:05:17.372 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.388+0000 7f5f7d600640 1 -- 172.21.15.107:0/3223610587 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 32) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f5f8000f040 con 0x55700fdd27a0 2024-03-08T21:05:17.372 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.388+0000 7f5f9d3b1740 10 client.15855 fetch_fsmap finished waiting for FSMap version 32 2024-03-08T21:05:17.372 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.388+0000 7f5f9d3b1740 10 client.15855 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-08T21:05:17.373 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.388+0000 7f5f9d3b1740 20 client.15855 populate_metadata read hostname 'smithi107' 2024-03-08T21:05:17.373 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.388+0000 7f5f9d3b1740 1 --2- 172.21.15.107:0/3223610587 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55700fdeb480 0x55700fded890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:05:17.373 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.388+0000 7f5f9d3b1740 4 client.15855 mds_command: new command op to 34475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-08T21:05:17.373 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.388+0000 7f5f9d3b1740 1 -- 172.21.15.107:0/3223610587 --> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55700fd42910 con 0x55700fdeb480 2024-03-08T21:05:17.373 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.389+0000 7f5f93400640 1 --2- 172.21.15.107:0/3223610587 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55700fdeb480 0x55700fded890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:05:17.373 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.390+0000 7f5f7d600640 10 client.15855 ms_handle_connect on v2:172.21.15.149:6838/55290596 2024-03-08T21:05:17.373 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.390+0000 7f5f93400640 1 --2- 172.21.15.107:0/3223610587 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55700fdeb480 0x55700fded890 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4ea9bc1b18538817 server_cookie=3155142b4beaccfa in_seq=0 out_seq=0 2024-03-08T21:05:17.374 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.391+0000 7f5f7d600640 1 -- 172.21.15.107:0/3223610587 <== mds.0 v2:172.21.15.149:6838/55290596 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55700fd42910 con 0x55700fdeb480 2024-03-08T21:05:17.374 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.391+0000 7f5f7d600640 10 client.15855 handle_command_reply: tid=0 2024-03-08T21:05:17.374 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.392+0000 7f5f9d3b1740 1 client.15855 shutdown 2024-03-08T21:05:17.374 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.392+0000 7f5f9d3b1740 1 -- 172.21.15.107:0/3223610587 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55700fdd27a0 msgr2=0x55700fdd8310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:05:17.374 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.392+0000 7f5f9d3b1740 1 --2- 172.21.15.107:0/3223610587 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55700fdd27a0 0x55700fdd8310 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f5f800106a0 tx=0x7f5f80010b60 comp rx=0 tx=0).stop 2024-03-08T21:05:17.375 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.392+0000 7f5f9d3b1740 1 -- 172.21.15.107:0/3223610587 shutdown_connections 2024-03-08T21:05:17.375 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.392+0000 7f5f9d3b1740 1 --2- 172.21.15.107:0/3223610587 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55700fddb570 0x55700fdd8920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:05:17.375 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.392+0000 7f5f9d3b1740 1 --2- 172.21.15.107:0/3223610587 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55700fdd27a0 0x55700fdd8310 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:05:17.375 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.392+0000 7f5f9d3b1740 1 --2- 172.21.15.107:0/3223610587 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55700fdeb480 0x55700fded890 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:05:17.375 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.392+0000 7f5f9d3b1740 1 --2- 172.21.15.107:0/3223610587 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55700fdd1e80 0x55700fddaf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:05:17.375 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.392+0000 7f5f9d3b1740 1 -- 172.21.15.107:0/3223610587 >> 172.21.15.107:0/3223610587 conn(0x55700fdcc7a0 msgr2=0x55700fdce4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:05:17.375 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.392+0000 7f5f7d600640 0 client.15855 ms_handle_reset on v2:172.21.15.149:6838/55290596 2024-03-08T21:05:17.375 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.392+0000 7f5f9d3b1740 1 -- 172.21.15.107:0/3223610587 shutdown_connections 2024-03-08T21:05:17.375 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.392+0000 7f5f9d3b1740 1 -- 172.21.15.107:0/3223610587 wait complete. 2024-03-08T21:05:17.375 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.392+0000 7f5f9d3b1740 20 client.15855 trim_cache size 0 max 16384 2024-03-08T21:05:17.376 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:05:17 smithi204 podman[70680]: 2024-03-08 21:05:17.135536596 +0000 UTC m=+0.476339925 container init bdc11ef2f39f583a2df227a3d6e704b46acfad788012076752a4bf7ab88301c5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, RELEASE=rishabh-2024mar8-7d63482, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-08T21:05:17.376 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:05:17 smithi204 podman[70680]: 2024-03-08 21:05:17.145297075 +0000 UTC m=+0.486100421 container start bdc11ef2f39f583a2df227a3d6e704b46acfad788012076752a4bf7ab88301c5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=rishabh-2024mar8-7d63482, ceph=True) 2024-03-08T21:05:17.376 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:05:17 smithi204 ceph-mds[70698]: -- 172.21.15.204:0/1596946025 <== mon.1 v2:172.21.15.204:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x561fe33e3a40 con 0x561fe33b7000 2024-03-08T21:05:17.376 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:05:17 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[70694]: starting mds.c at 2024-03-08T21:05:17.376 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:05:17 smithi204 bash[70680]: bdc11ef2f39f583a2df227a3d6e704b46acfad788012076752a4bf7ab88301c5 2024-03-08T21:05:17.376 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:05:17 smithi204 systemd[1]: Started Ceph mds.c for dc20f0f8-dd8c-11ee-95c6-87774f69a715. 2024-03-08T21:05:17.394 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.413+0000 7f5f9d3b1740 1 Processor -- start 2024-03-08T21:05:17.394 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.413+0000 7f5f9d3b1740 1 -- start start 2024-03-08T21:05:17.394 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.413+0000 7f5f9d3b1740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55700fdd1e80 0x55700fddaa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:05:17.394 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.413+0000 7f5f9d3b1740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55700fdd27a0 0x55700fddaf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:05:17.394 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.413+0000 7f5f9d3b1740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55700fddb570 0x55700fdd55d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:05:17.394 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.413+0000 7f5f9d3b1740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f941ac940 con 0x55700fddb570 2024-03-08T21:05:17.394 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.413+0000 7f5f9d3b1740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f941ae2c0 con 0x55700fdd1e80 2024-03-08T21:05:17.394 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.413+0000 7f5f9d3b1740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f941acfa0 con 0x55700fdd27a0 2024-03-08T21:05:17.396 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.414+0000 7f5f93e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55700fddb570 0x55700fdd55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:05:17.396 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.414+0000 7f5f93e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55700fddb570 0x55700fdd55d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:34644/0 (socket says 172.21.15.107:34644) 2024-03-08T21:05:17.396 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.414+0000 7f5f93e00640 1 -- 172.21.15.107:0/2478606999 learned_addr learned my addr 172.21.15.107:0/2478606999 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:05:17.396 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.414+0000 7f5f93400640 1 --2- 172.21.15.107:0/2478606999 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55700fdd1e80 0x55700fddaa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:05:17.396 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.414+0000 7f5f92a00640 1 --2- 172.21.15.107:0/2478606999 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55700fdd27a0 0x55700fddaf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:05:17.396 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.414+0000 7f5f93e00640 1 -- 172.21.15.107:0/2478606999 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55700fdd27a0 msgr2=0x55700fddaf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:05:17.396 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.414+0000 7f5f93e00640 1 --2- 172.21.15.107:0/2478606999 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55700fdd27a0 0x55700fddaf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:05:17.396 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.414+0000 7f5f93e00640 1 -- 172.21.15.107:0/2478606999 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55700fdd1e80 msgr2=0x55700fddaa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:05:17.396 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.414+0000 7f5f93e00640 1 --2- 172.21.15.107:0/2478606999 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55700fdd1e80 0x55700fddaa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:05:17.396 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.414+0000 7f5f93e00640 1 -- 172.21.15.107:0/2478606999 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55700fde8240 con 0x55700fddb570 2024-03-08T21:05:17.396 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.415+0000 7f5f93e00640 1 --2- 172.21.15.107:0/2478606999 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55700fddb570 0x55700fdd55d0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f5f8c0096c0 tx=0x7f5f8c02df90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6aab51cc0062bea1 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:05:17.396 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.415+0000 7f5f7fe00640 1 -- 172.21.15.107:0/2478606999 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f8c05a070 con 0x55700fddb570 2024-03-08T21:05:17.397 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.415+0000 7f5f7fe00640 1 -- 172.21.15.107:0/2478606999 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f5f8c0046c0 con 0x55700fddb570 2024-03-08T21:05:17.397 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.415+0000 7f5f7fe00640 1 -- 172.21.15.107:0/2478606999 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f8c051040 con 0x55700fddb570 2024-03-08T21:05:17.399 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.416+0000 7f5f9d3b1740 1 -- 172.21.15.107:0/2478606999 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55700fddb570 msgr2=0x55700fdd55d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:05:17.400 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.416+0000 7f5f9d3b1740 1 --2- 172.21.15.107:0/2478606999 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55700fddb570 0x55700fdd55d0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f5f8c0096c0 tx=0x7f5f8c02df90 comp rx=0 tx=0).stop 2024-03-08T21:05:17.400 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.416+0000 7f5f9d3b1740 1 -- 172.21.15.107:0/2478606999 shutdown_connections 2024-03-08T21:05:17.400 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.416+0000 7f5f9d3b1740 1 --2- 172.21.15.107:0/2478606999 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55700fddb570 0x55700fdd55d0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:05:17.400 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.416+0000 7f5f9d3b1740 1 --2- 172.21.15.107:0/2478606999 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55700fdd27a0 0x55700fddaf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:05:17.400 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.416+0000 7f5f9d3b1740 1 --2- 172.21.15.107:0/2478606999 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55700fdd1e80 0x55700fddaa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:05:17.400 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.416+0000 7f5f9d3b1740 1 -- 172.21.15.107:0/2478606999 >> 172.21.15.107:0/2478606999 conn(0x55700fdcc7a0 msgr2=0x55700fdf2230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:05:17.400 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.416+0000 7f5f9d3b1740 1 -- 172.21.15.107:0/2478606999 shutdown_connections 2024-03-08T21:05:17.400 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.416+0000 7f5f9d3b1740 1 -- 172.21.15.107:0/2478606999 wait complete. 2024-03-08T21:05:17.400 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.416+0000 7f5f9d3b1740 1 -- start start 2024-03-08T21:05:17.400 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.417+0000 7f5f9d3b1740 1 Processor -- start 2024-03-08T21:05:17.400 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.417+0000 7f5f9d3b1740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55700fdd1e80 0x55700fde1250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:05:17.400 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.417+0000 7f5f9d3b1740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55700fdd27a0 0x55700fde1790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:05:17.400 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.417+0000 7f5f9d3b1740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55700fddb570 0x55700fde1d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:05:17.401 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.417+0000 7f5f9d3b1740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f941adc60 con 0x55700fdd27a0 2024-03-08T21:05:17.401 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.417+0000 7f5f9d3b1740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f941ad600 con 0x55700fdd1e80 2024-03-08T21:05:17.401 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.417+0000 7f5f9d3b1740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f941aabe0 con 0x55700fddb570 2024-03-08T21:05:17.401 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.417+0000 7f5f9d3b1740 10 client.0 _ll_register_callbacks cb 0x7f5f8c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T21:05:17.401 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.418+0000 7f5f92a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55700fdd27a0 0x55700fde1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:05:17.401 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.418+0000 7f5f92a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55700fdd27a0 0x55700fde1790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:34654/0 (socket says 172.21.15.107:34654) 2024-03-08T21:05:17.401 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.418+0000 7f5f92a00640 1 -- 172.21.15.107:0/625119292 learned_addr learned my addr 172.21.15.107:0/625119292 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:05:17.401 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.418+0000 7f5f93400640 1 --2- 172.21.15.107:0/625119292 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55700fdd1e80 0x55700fde1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:05:17.401 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.418+0000 7f5f93e00640 1 --2- 172.21.15.107:0/625119292 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55700fddb570 0x55700fde1d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:05:17.401 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.418+0000 7f5f92a00640 1 -- 172.21.15.107:0/625119292 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55700fddb570 msgr2=0x55700fde1d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:05:17.401 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.418+0000 7f5f92a00640 1 --2- 172.21.15.107:0/625119292 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55700fddb570 0x55700fde1d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:05:17.401 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.418+0000 7f5f92a00640 1 -- 172.21.15.107:0/625119292 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55700fdd1e80 msgr2=0x55700fde1250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:05:17.401 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.418+0000 7f5f92a00640 1 --2- 172.21.15.107:0/625119292 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55700fdd1e80 0x55700fde1250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:05:17.402 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.418+0000 7f5f92a00640 1 -- 172.21.15.107:0/625119292 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55700fe460c0 con 0x55700fdd27a0 2024-03-08T21:05:17.402 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.418+0000 7f5f93e00640 1 --2- 172.21.15.107:0/625119292 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55700fddb570 0x55700fde1d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T21:05:17.402 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.418+0000 7f5f92a00640 1 --2- 172.21.15.107:0/625119292 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55700fdd27a0 0x55700fde1790 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x55700fdd7b70 tx=0x7f5f80010300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1fb315ebce34a57 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:05:17.402 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.418+0000 7f5f7d600640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-03-08T21:05:17.402 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.419+0000 7f5f7d600640 1 -- 172.21.15.107:0/625119292 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f80010ea0 con 0x55700fdd27a0 2024-03-08T21:05:17.402 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.419+0000 7f5f7d600640 1 -- 172.21.15.107:0/625119292 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f5f8000e8d0 con 0x55700fdd27a0 2024-03-08T21:05:17.402 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.419+0000 7f5f9d3b1740 1 -- 172.21.15.107:0/625119292 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5f8c0046c0 con 0x55700fdd27a0 2024-03-08T21:05:17.402 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.419+0000 7f5f7d600640 1 -- 172.21.15.107:0/625119292 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5f8000e6c0 con 0x55700fdd27a0 2024-03-08T21:05:17.402 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.419+0000 7f5f9d3b1740 10 client.15861 fetch_fsmap learned FSMap version 32 2024-03-08T21:05:17.402 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.419+0000 7f5f9d3b1740 1 -- 172.21.15.107:0/625119292 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55700fe47400 con 0x55700fdd27a0 2024-03-08T21:05:17.403 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.419+0000 7f5f7d600640 1 -- 172.21.15.107:0/625119292 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f5f8c0046c0 con 0x55700fdd27a0 2024-03-08T21:05:17.403 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.420+0000 7f5f7d600640 1 -- 172.21.15.107:0/625119292 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 32) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f5f800041c0 con 0x55700fdd27a0 2024-03-08T21:05:17.403 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.420+0000 7f5f9d3b1740 10 client.15861 fetch_fsmap finished waiting for FSMap version 32 2024-03-08T21:05:17.403 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.420+0000 7f5f9d3b1740 10 client.15861 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-08T21:05:17.403 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.420+0000 7f5f9d3b1740 20 client.15861 populate_metadata read hostname 'smithi107' 2024-03-08T21:05:17.403 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.420+0000 7f5f9d3b1740 1 --2- 172.21.15.107:0/625119292 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55700fe47b00 0x55700fe49f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:05:17.404 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.420+0000 7f5f9d3b1740 4 client.15861 mds_command: new command op to 34475 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-08T21:05:17.404 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.420+0000 7f5f9d3b1740 1 -- 172.21.15.107:0/625119292 --> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55700fd3cc60 con 0x55700fe47b00 2024-03-08T21:05:17.404 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.420+0000 7f5f93400640 1 --2- 172.21.15.107:0/625119292 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55700fe47b00 0x55700fe49f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:05:17.404 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.421+0000 7f5f93400640 1 --2- 172.21.15.107:0/625119292 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55700fe47b00 0x55700fe49f20 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c6f1e8af4c250c6 server_cookie=bbc2191518f40e80 in_seq=0 out_seq=0 2024-03-08T21:05:17.404 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.421+0000 7f5f7d600640 10 client.15861 ms_handle_connect on v2:172.21.15.149:6838/55290596 2024-03-08T21:05:17.414 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.433+0000 7f5f7d600640 1 -- 172.21.15.107:0/625119292 <== mds.0 v2:172.21.15.149:6838/55290596 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55700fd3cc60 con 0x55700fe47b00 2024-03-08T21:05:17.414 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.433+0000 7f5f7d600640 10 client.15861 handle_command_reply: tid=0 2024-03-08T21:05:17.414 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.433+0000 7f5f9d3b1740 1 client.15861 shutdown 2024-03-08T21:05:17.414 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.433+0000 7f5f9d3b1740 1 -- 172.21.15.107:0/625119292 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55700fdd27a0 msgr2=0x55700fde1790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:05:17.414 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.433+0000 7f5f9d3b1740 1 --2- 172.21.15.107:0/625119292 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55700fdd27a0 0x55700fde1790 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x55700fdd7b70 tx=0x7f5f80010300 comp rx=0 tx=0).stop 2024-03-08T21:05:17.414 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.433+0000 7f5f9d3b1740 1 -- 172.21.15.107:0/625119292 shutdown_connections 2024-03-08T21:05:17.414 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.433+0000 7f5f9d3b1740 1 --2- 172.21.15.107:0/625119292 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55700fddb570 0x55700fde1d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:05:17.414 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.433+0000 7f5f9d3b1740 1 --2- 172.21.15.107:0/625119292 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55700fdd27a0 0x55700fde1790 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:05:17.415 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.433+0000 7f5f9d3b1740 1 --2- 172.21.15.107:0/625119292 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55700fe47b00 0x55700fe49f20 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:05:17.415 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.433+0000 7f5f9d3b1740 1 --2- 172.21.15.107:0/625119292 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55700fdd1e80 0x55700fde1250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:05:17.415 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.433+0000 7f5f9d3b1740 1 -- 172.21.15.107:0/625119292 >> 172.21.15.107:0/625119292 conn(0x55700fdcc7a0 msgr2=0x55700fdcaf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:05:17.415 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.433+0000 7f5f7d600640 0 client.15861 ms_handle_reset on v2:172.21.15.149:6838/55290596 2024-03-08T21:05:17.415 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.433+0000 7f5f9d3b1740 1 -- 172.21.15.107:0/625119292 shutdown_connections 2024-03-08T21:05:17.415 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.433+0000 7f5f9d3b1740 1 -- 172.21.15.107:0/625119292 wait complete. 2024-03-08T21:05:17.415 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.433+0000 7f5f9d3b1740 20 client.15861 trim_cache size 0 max 16384 2024-03-08T21:05:17.415 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:{ 2024-03-08T21:05:17.415 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "message": "", 2024-03-08T21:05:17.415 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "return_code": 0 2024-03-08T21:05:17.416 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:} 2024-03-08T21:05:17.416 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.434+0000 7f5f99000640 1 -- 172.21.15.107:0/2745896971 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f5f780a29e0 msgr2=0x7f5f780a4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:05:17.416 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.434+0000 7f5f99000640 1 --2- 172.21.15.107:0/2745896971 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f5f780a29e0 0x7f5f780a4ea0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f5f8c045000 tx=0x7f5f8c048000 comp rx=0 tx=0).stop 2024-03-08T21:05:17.416 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.434+0000 7f5f99000640 1 -- 172.21.15.107:0/2745896971 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5f941022a0 msgr2=0x7f5f9419d680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:05:17.416 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.434+0000 7f5f99000640 1 --2- 172.21.15.107:0/2745896971 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5f941022a0 0x7f5f9419d680 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f5f8400ba50 tx=0x7f5f8400bf10 comp rx=0 tx=0).stop 2024-03-08T21:05:17.417 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.434+0000 7f5f99000640 1 -- 172.21.15.107:0/2745896971 shutdown_connections 2024-03-08T21:05:17.417 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.434+0000 7f5f99000640 1 --2- 172.21.15.107:0/2745896971 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f5f780a29e0 0x7f5f780a4ea0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:05:17.417 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.434+0000 7f5f99000640 1 --2- 172.21.15.107:0/2745896971 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5f9410b6c0 0x7f5f941977f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:05:17.417 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.434+0000 7f5f99000640 1 --2- 172.21.15.107:0/2745896971 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5f94102bc0 0x7f5f9419dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:05:17.417 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.434+0000 7f5f99000640 1 --2- 172.21.15.107:0/2745896971 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5f941022a0 0x7f5f9419d680 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:05:17.417 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.434+0000 7f5f99000640 1 -- 172.21.15.107:0/2745896971 >> 172.21.15.107:0/2745896971 conn(0x7f5f940fbb00 msgr2=0x7f5f940fd2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:05:17.417 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.435+0000 7f5f99000640 1 -- 172.21.15.107:0/2745896971 shutdown_connections 2024-03-08T21:05:17.417 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:17.435+0000 7f5f99000640 1 -- 172.21.15.107:0/2745896971 wait complete. 2024-03-08T21:05:17.964 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:05:17 smithi107 ceph-mon[31381]: pgmap v147: 97 pgs: 97 active+clean; 3.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 2.3 KiB/s wr, 9 op/s 2024-03-08T21:05:17.964 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:05:17 smithi107 ceph-mon[31381]: daemon mds.c restarted 2024-03-08T21:05:18.012 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:05:17 smithi149 ceph-mon[35042]: pgmap v147: 97 pgs: 97 active+clean; 3.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 2.3 KiB/s wr, 9 op/s 2024-03-08T21:05:18.012 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:05:17 smithi149 ceph-mon[35042]: daemon mds.c restarted 2024-03-08T21:05:18.126 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:05:17 smithi204 ceph-mon[34935]: pgmap v147: 97 pgs: 97 active+clean; 3.3 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 2.3 KiB/s wr, 9 op/s 2024-03-08T21:05:18.126 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:05:17 smithi204 ceph-mon[34935]: daemon mds.c restarted 2024-03-08T21:05:18.964 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:05:18 smithi107 ceph-mon[31381]: osdmap e92: 12 total, 12 up, 12 in 2024-03-08T21:05:18.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:05:18 smithi107 ceph-mon[31381]: mds.? [v2:172.21.15.204:6836/2511971929,v1:172.21.15.204:6837/2511971929] up:boot 2024-03-08T21:05:18.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:05:18 smithi107 ceph-mon[31381]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-08T21:05:18.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:05:18 smithi107 ceph-mon[31381]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-03-08T21:05:18.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:05:18 smithi107 ceph-mon[31381]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-08T21:05:19.012 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:05:18 smithi149 ceph-mon[35042]: osdmap e92: 12 total, 12 up, 12 in 2024-03-08T21:05:19.012 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:05:18 smithi149 ceph-mon[35042]: mds.? [v2:172.21.15.204:6836/2511971929,v1:172.21.15.204:6837/2511971929] up:boot 2024-03-08T21:05:19.012 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:05:18 smithi149 ceph-mon[35042]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-08T21:05:19.013 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:05:18 smithi149 ceph-mon[35042]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-03-08T21:05:19.013 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:05:18 smithi149 ceph-mon[35042]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-08T21:05:19.126 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:05:18 smithi204 ceph-mon[34935]: osdmap e92: 12 total, 12 up, 12 in 2024-03-08T21:05:19.126 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:05:18 smithi204 ceph-mon[34935]: mds.? [v2:172.21.15.204:6836/2511971929,v1:172.21.15.204:6837/2511971929] up:boot 2024-03-08T21:05:19.126 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:05:18 smithi204 ceph-mon[34935]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-08T21:05:19.126 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:05:18 smithi204 ceph-mon[34935]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-03-08T21:05:19.126 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:05:18 smithi204 ceph-mon[34935]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-08T21:05:19.964 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:05:19 smithi107 ceph-mon[31381]: pgmap v149: 97 pgs: 97 active+clean; 3.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 307 B/s wr, 9 op/s 2024-03-08T21:05:20.012 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:05:19 smithi149 ceph-mon[35042]: pgmap v149: 97 pgs: 97 active+clean; 3.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 307 B/s wr, 9 op/s 2024-03-08T21:05:20.125 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:05:19 smithi204 ceph-mon[34935]: pgmap v149: 97 pgs: 97 active+clean; 3.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 307 B/s wr, 9 op/s 2024-03-08T21:05:21.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:05:21 smithi107 ceph-mon[31381]: pgmap v150: 97 pgs: 97 active+clean; 3.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 307 B/s wr, 9 op/s 2024-03-08T21:05:22.014 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:05:21 smithi149 ceph-mon[35042]: pgmap v150: 97 pgs: 97 active+clean; 3.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 307 B/s wr, 9 op/s 2024-03-08T21:05:22.126 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:05:21 smithi204 ceph-mon[34935]: pgmap v150: 97 pgs: 97 active+clean; 3.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 307 B/s wr, 9 op/s 2024-03-08T21:05:22.964 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:05:22 smithi107 ceph-mon[31381]: pgmap v151: 97 pgs: 97 active+clean; 3.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 307 B/s wr, 9 op/s 2024-03-08T21:05:23.012 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:05:22 smithi149 ceph-mon[35042]: pgmap v151: 97 pgs: 97 active+clean; 3.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 307 B/s wr, 9 op/s 2024-03-08T21:05:23.125 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:05:22 smithi204 ceph-mon[34935]: pgmap v151: 97 pgs: 97 active+clean; 3.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 307 B/s wr, 9 op/s 2024-03-08T21:05:25.964 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:05:25 smithi107 ceph-mon[31381]: pgmap v152: 97 pgs: 97 active+clean; 3.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 307 B/s wr, 10 op/s 2024-03-08T21:05:26.012 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:05:25 smithi149 ceph-mon[35042]: pgmap v152: 97 pgs: 97 active+clean; 3.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 307 B/s wr, 10 op/s 2024-03-08T21:05:26.125 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:05:25 smithi204 ceph-mon[34935]: pgmap v152: 97 pgs: 97 active+clean; 3.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 307 B/s wr, 10 op/s 2024-03-08T21:05:27.964 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:05:27 smithi107 ceph-mon[31381]: pgmap v153: 97 pgs: 97 active+clean; 3.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 307 B/s wr, 10 op/s 2024-03-08T21:05:28.012 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:05:27 smithi149 ceph-mon[35042]: pgmap v153: 97 pgs: 97 active+clean; 3.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 307 B/s wr, 10 op/s 2024-03-08T21:05:28.125 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:05:27 smithi204 ceph-mon[34935]: pgmap v153: 97 pgs: 97 active+clean; 3.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 307 B/s wr, 10 op/s 2024-03-08T21:05:29.964 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:05:29 smithi107 ceph-mon[31381]: pgmap v154: 97 pgs: 97 active+clean; 3.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 281 B/s wr, 14 op/s 2024-03-08T21:05:30.012 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:05:29 smithi149 ceph-mon[35042]: pgmap v154: 97 pgs: 97 active+clean; 3.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 281 B/s wr, 14 op/s 2024-03-08T21:05:30.125 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:05:29 smithi204 ceph-mon[34935]: pgmap v154: 97 pgs: 97 active+clean; 3.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 281 B/s wr, 14 op/s 2024-03-08T21:05:30.557 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.576+0000 7f4733200640 1 -- 172.21.15.107:0/3818878844 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f472c101eb0 msgr2=0x7f472c105ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:05:30.557 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.576+0000 7f4733200640 1 --2- 172.21.15.107:0/3818878844 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f472c101eb0 0x7f472c105ef0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f4714009900 tx=0x7f471402bc30 comp rx=0 tx=0).stop 2024-03-08T21:05:30.557 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.576+0000 7f4733200640 1 -- 172.21.15.107:0/3818878844 shutdown_connections 2024-03-08T21:05:30.557 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.576+0000 7f4733200640 1 --2- 172.21.15.107:0/3818878844 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f472c1065d0 0x7f472c10e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:05:30.557 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.576+0000 7f4733200640 1 --2- 172.21.15.107:0/3818878844 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f472c101eb0 0x7f472c105ef0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:05:30.557 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.576+0000 7f4733200640 1 --2- 172.21.15.107:0/3818878844 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f472c101500 0x7f472c1018e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:05:30.557 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.576+0000 7f4733200640 1 -- 172.21.15.107:0/3818878844 >> 172.21.15.107:0/3818878844 conn(0x7f472c0fcd80 msgr2=0x7f472c0ff1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:05:30.558 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.576+0000 7f4733200640 1 -- 172.21.15.107:0/3818878844 shutdown_connections 2024-03-08T21:05:30.558 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.576+0000 7f4733200640 1 -- 172.21.15.107:0/3818878844 wait complete. 2024-03-08T21:05:30.559 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.577+0000 7f4733200640 1 Processor -- start 2024-03-08T21:05:30.559 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.577+0000 7f4733200640 1 -- start start 2024-03-08T21:05:30.559 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.578+0000 7f4733200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f472c101500 0x7f472c0735f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:05:30.560 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.578+0000 7f4733200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f472c1065d0 0x7f472c073b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:05:30.560 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.578+0000 7f4733200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f472c074070 0x7f472c1a5c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:05:30.560 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.578+0000 7f4733200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f472c111510 con 0x7f472c1065d0 2024-03-08T21:05:30.560 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.578+0000 7f4733200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f472c111390 con 0x7f472c101500 2024-03-08T21:05:30.560 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.578+0000 7f4733200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f472c111690 con 0x7f472c074070 2024-03-08T21:05:30.560 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.578+0000 7f472b400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f472c1065d0 0x7f472c073b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:05:30.560 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.578+0000 7f472b400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f472c1065d0 0x7f472c073b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:45590/0 (socket says 172.21.15.107:45590) 2024-03-08T21:05:30.560 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.578+0000 7f472b400640 1 -- 172.21.15.107:0/643329111 learned_addr learned my addr 172.21.15.107:0/643329111 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:05:30.560 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.578+0000 7f472be00640 1 --2- 172.21.15.107:0/643329111 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f472c101500 0x7f472c0735f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:05:30.560 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.579+0000 7f4731200640 1 --2- 172.21.15.107:0/643329111 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f472c074070 0x7f472c1a5c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:05:30.560 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.579+0000 7f472b400640 1 -- 172.21.15.107:0/643329111 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f472c074070 msgr2=0x7f472c1a5c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:05:30.560 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.579+0000 7f472b400640 1 --2- 172.21.15.107:0/643329111 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f472c074070 0x7f472c1a5c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:05:30.560 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.579+0000 7f472b400640 1 -- 172.21.15.107:0/643329111 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f472c101500 msgr2=0x7f472c0735f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:05:30.561 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.579+0000 7f472b400640 1 --2- 172.21.15.107:0/643329111 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f472c101500 0x7f472c0735f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:05:30.561 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.579+0000 7f472b400640 1 -- 172.21.15.107:0/643329111 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f472c1a6320 con 0x7f472c1065d0 2024-03-08T21:05:30.561 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.579+0000 7f472b400640 1 --2- 172.21.15.107:0/643329111 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f472c1065d0 0x7f472c073b30 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f4714036040 tx=0x7f47140045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:05:30.561 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.579+0000 7f4728c00640 1 -- 172.21.15.107:0/643329111 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4714043020 con 0x7f472c1065d0 2024-03-08T21:05:30.562 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.579+0000 7f4728c00640 1 -- 172.21.15.107:0/643329111 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f471402ca70 con 0x7f472c1065d0 2024-03-08T21:05:30.562 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.579+0000 7f4733200640 1 -- 172.21.15.107:0/643329111 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f472c1a65b0 con 0x7f472c1065d0 2024-03-08T21:05:30.562 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.579+0000 7f4728c00640 1 -- 172.21.15.107:0/643329111 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4714003ae0 con 0x7f472c1065d0 2024-03-08T21:05:30.562 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.580+0000 7f4733200640 1 -- 172.21.15.107:0/643329111 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f472c1a6b50 con 0x7f472c1065d0 2024-03-08T21:05:30.563 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.583+0000 7f4728c00640 1 -- 172.21.15.107:0/643329111 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) v1 ==== 137422+0+0 (secure 0 0 0) 0x7f4714046020 con 0x7f472c1065d0 2024-03-08T21:05:30.564 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.583+0000 7f4735fd4740 1 Processor -- start 2024-03-08T21:05:30.564 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.584+0000 7f4735fd4740 1 -- start start 2024-03-08T21:05:30.565 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.584+0000 7f4735fd4740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561e6f3bcc80 0x561e6f3bd060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:05:30.565 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.584+0000 7f4735fd4740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561e6f3bd5a0 0x561e6f3c5e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:05:30.565 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.584+0000 7f4735fd4740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561e6f3c6370 0x561e6f3c8760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:05:30.565 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.584+0000 7f4735fd4740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e6f3203d0 con 0x561e6f3bcc80 2024-03-08T21:05:30.565 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.584+0000 7f4735fd4740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e6f2c96b0 con 0x561e6f3c6370 2024-03-08T21:05:30.566 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.584+0000 7f4735fd4740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e6f31c470 con 0x561e6f3bd5a0 2024-03-08T21:05:30.566 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.584+0000 7f4728c00640 1 --2- 172.21.15.107:0/643329111 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f47100a2ac0 0x7f47100a4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:05:30.566 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.585+0000 7f4731200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561e6f3bcc80 0x561e6f3bd060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:05:30.566 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.585+0000 7f4731200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561e6f3bcc80 0x561e6f3bd060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:45598/0 (socket says 172.21.15.107:45598) 2024-03-08T21:05:30.566 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.585+0000 7f4731200640 1 -- 172.21.15.107:0/3863852454 learned_addr learned my addr 172.21.15.107:0/3863852454 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:05:30.566 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.585+0000 7f4728c00640 1 -- 172.21.15.107:0/643329111 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f47140f0ea0 con 0x7f472c1065d0 2024-03-08T21:05:30.566 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.585+0000 7f472be00640 1 --2- 172.21.15.107:0/3863852454 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561e6f3bd5a0 0x561e6f3c5e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:05:30.566 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.585+0000 7f4731200640 1 --2- 172.21.15.107:0/643329111 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f47100a2ac0 0x7f47100a4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:05:30.566 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.585+0000 7f472b400640 1 --2- 172.21.15.107:0/3863852454 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561e6f3c6370 0x561e6f3c8760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:05:30.568 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.585+0000 7f4731200640 1 -- 172.21.15.107:0/3863852454 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561e6f3bd5a0 msgr2=0x561e6f3c5e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:05:30.568 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.585+0000 7f4731200640 1 --2- 172.21.15.107:0/3863852454 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561e6f3bd5a0 0x561e6f3c5e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:05:30.568 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.585+0000 7f4731200640 1 -- 172.21.15.107:0/3863852454 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561e6f3c6370 msgr2=0x561e6f3c8760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:05:30.568 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.585+0000 7f4731200640 1 --2- 172.21.15.107:0/3863852454 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561e6f3c6370 0x561e6f3c8760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:05:30.568 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.585+0000 7f4731200640 1 -- 172.21.15.107:0/3863852454 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561e6f2efb50 con 0x561e6f3bcc80 2024-03-08T21:05:30.568 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.586+0000 7f4731200640 1 --2- 172.21.15.107:0/3863852454 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561e6f3bcc80 0x561e6f3bd060 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f4724014e00 tx=0x7f47240117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de274a6b5273f9f0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:05:30.568 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.586+0000 7f4731200640 1 --2- 172.21.15.107:0/643329111 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f47100a2ac0 0x7f47100a4f80 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f4724009fd0 tx=0x7f472401c000 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:05:30.568 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.586+0000 7f4721600640 1 -- 172.21.15.107:0/3863852454 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f472400ccb0 con 0x561e6f3bcc80 2024-03-08T21:05:30.568 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.586+0000 7f4721600640 1 -- 172.21.15.107:0/3863852454 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f4724009c40 con 0x561e6f3bcc80 2024-03-08T21:05:30.568 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.586+0000 7f4721600640 1 -- 172.21.15.107:0/3863852454 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47240045a0 con 0x561e6f3bcc80 2024-03-08T21:05:30.569 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.587+0000 7f4735fd4740 1 -- 172.21.15.107:0/3863852454 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561e6f3bcc80 msgr2=0x561e6f3bd060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:05:30.569 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.587+0000 7f4735fd4740 1 --2- 172.21.15.107:0/3863852454 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561e6f3bcc80 0x561e6f3bd060 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f4724014e00 tx=0x7f47240117b0 comp rx=0 tx=0).stop 2024-03-08T21:05:30.569 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.587+0000 7f4735fd4740 1 -- 172.21.15.107:0/3863852454 shutdown_connections 2024-03-08T21:05:30.569 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.587+0000 7f4735fd4740 1 --2- 172.21.15.107:0/3863852454 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561e6f3c6370 0x561e6f3c8760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:05:30.569 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.587+0000 7f4735fd4740 1 --2- 172.21.15.107:0/3863852454 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561e6f3bd5a0 0x561e6f3c5e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:05:30.569 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.587+0000 7f4735fd4740 1 --2- 172.21.15.107:0/3863852454 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561e6f3bcc80 0x561e6f3bd060 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:05:30.569 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.587+0000 7f4735fd4740 1 -- 172.21.15.107:0/3863852454 >> 172.21.15.107:0/3863852454 conn(0x561e6f3b75a0 msgr2=0x561e6f3b7980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:05:30.569 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.587+0000 7f4735fd4740 1 -- 172.21.15.107:0/3863852454 shutdown_connections 2024-03-08T21:05:30.569 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.587+0000 7f4735fd4740 1 -- 172.21.15.107:0/3863852454 wait complete. 2024-03-08T21:05:30.573 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.588+0000 7f4735fd4740 1 -- start start 2024-03-08T21:05:30.573 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.588+0000 7f4735fd4740 1 Processor -- start 2024-03-08T21:05:30.573 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.588+0000 7f4735fd4740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561e6f3bcc80 0x561e6f3c5e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:05:30.573 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.588+0000 7f4735fd4740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561e6f3bd5a0 0x561e6f3c3190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:05:30.573 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.588+0000 7f4735fd4740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561e6f3c6370 0x561e6f3c3880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:05:30.573 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.588+0000 7f4735fd4740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e6f3c8f40 con 0x561e6f3bd5a0 2024-03-08T21:05:30.573 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.588+0000 7f4735fd4740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e6f2dc9f0 con 0x561e6f3bcc80 2024-03-08T21:05:30.573 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.588+0000 7f4735fd4740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e6f3c90c0 con 0x561e6f3c6370 2024-03-08T21:05:30.573 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.589+0000 7f4735fd4740 10 client.0 _ll_register_callbacks cb 0x561e6f034a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T21:05:30.573 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.589+0000 7f472b400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561e6f3bd5a0 0x561e6f3c3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:05:30.573 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.589+0000 7f472b400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561e6f3bd5a0 0x561e6f3c3190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:45604/0 (socket says 172.21.15.107:45604) 2024-03-08T21:05:30.574 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.589+0000 7f472b400640 1 -- 172.21.15.107:0/754788829 learned_addr learned my addr 172.21.15.107:0/754788829 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:05:30.574 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.589+0000 7f472be00640 1 --2- 172.21.15.107:0/754788829 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561e6f3bcc80 0x561e6f3c5e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:05:30.574 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.589+0000 7f4731200640 1 --2- 172.21.15.107:0/754788829 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561e6f3c6370 0x561e6f3c3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:05:30.574 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.589+0000 7f472b400640 1 -- 172.21.15.107:0/754788829 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561e6f3c6370 msgr2=0x561e6f3c3880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:05:30.574 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.589+0000 7f472b400640 1 --2- 172.21.15.107:0/754788829 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561e6f3c6370 0x561e6f3c3880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:05:30.574 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.589+0000 7f472b400640 1 -- 172.21.15.107:0/754788829 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561e6f3bcc80 msgr2=0x561e6f3c5e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:05:30.574 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.589+0000 7f472b400640 1 --2- 172.21.15.107:0/754788829 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561e6f3bcc80 0x561e6f3c5e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:05:30.574 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.589+0000 7f472b400640 1 -- 172.21.15.107:0/754788829 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561e6f035210 con 0x561e6f3bd5a0 2024-03-08T21:05:30.574 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.589+0000 7f4731200640 1 --2- 172.21.15.107:0/754788829 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561e6f3c6370 0x561e6f3c3880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T21:05:30.574 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.589+0000 7f472b400640 1 --2- 172.21.15.107:0/754788829 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561e6f3bd5a0 0x561e6f3c3190 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f4714000c00 tx=0x7f4714009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95c450ea4029c874 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:05:30.574 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.589+0000 7f471a000640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-03-08T21:05:30.574 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.590+0000 7f471a000640 1 -- 172.21.15.107:0/754788829 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47140d6070 con 0x561e6f3bd5a0 2024-03-08T21:05:30.575 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.590+0000 7f471a000640 1 -- 172.21.15.107:0/754788829 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f47140bd860 con 0x561e6f3bd5a0 2024-03-08T21:05:30.575 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.590+0000 7f4735fd4740 1 -- 172.21.15.107:0/754788829 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561e6f32d910 con 0x561e6f3bd5a0 2024-03-08T21:05:30.575 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.590+0000 7f471a000640 1 -- 172.21.15.107:0/754788829 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f471402cc20 con 0x561e6f3bd5a0 2024-03-08T21:05:30.575 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.590+0000 7f4735fd4740 10 client.15873 fetch_fsmap learned FSMap version 34 2024-03-08T21:05:30.575 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.590+0000 7f4735fd4740 1 -- 172.21.15.107:0/754788829 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x561e6f101620 con 0x561e6f3bd5a0 2024-03-08T21:05:30.575 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.590+0000 7f471a000640 1 -- 172.21.15.107:0/754788829 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x561e6f32d910 con 0x561e6f3bd5a0 2024-03-08T21:05:30.575 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.590+0000 7f471a000640 1 -- 172.21.15.107:0/754788829 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 34) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f47140c5910 con 0x561e6f3bd5a0 2024-03-08T21:05:30.575 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.591+0000 7f4735fd4740 10 client.15873 fetch_fsmap finished waiting for FSMap version 34 2024-03-08T21:05:30.575 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.591+0000 7f4735fd4740 10 client.15873 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-08T21:05:30.575 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.591+0000 7f4735fd4740 20 client.15873 populate_metadata read hostname 'smithi107' 2024-03-08T21:05:30.575 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.591+0000 7f4735fd4740 1 --2- 172.21.15.107:0/754788829 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x561e6f3d6180 0x561e6f3d85f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:05:30.575 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.591+0000 7f4735fd4740 4 client.15873 mds_command: new command op to 34475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-08T21:05:30.575 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.591+0000 7f4735fd4740 1 -- 172.21.15.107:0/754788829 --> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561e6f327c60 con 0x561e6f3d6180 2024-03-08T21:05:30.575 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.591+0000 7f472be00640 1 --2- 172.21.15.107:0/754788829 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x561e6f3d6180 0x561e6f3d85f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:05:30.576 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.592+0000 7f471a000640 10 client.15873 ms_handle_connect on v2:172.21.15.149:6838/55290596 2024-03-08T21:05:30.576 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.592+0000 7f472be00640 1 --2- 172.21.15.107:0/754788829 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x561e6f3d6180 0x561e6f3d85f0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=888d01dca0b8b6d9 server_cookie=df7de875885d09d9 in_seq=0 out_seq=0 2024-03-08T21:05:30.577 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.594+0000 7f471a000640 1 -- 172.21.15.107:0/754788829 <== mds.0 v2:172.21.15.149:6838/55290596 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x561e6f327c60 con 0x561e6f3d6180 2024-03-08T21:05:30.577 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.594+0000 7f471a000640 10 client.15873 handle_command_reply: tid=0 2024-03-08T21:05:30.577 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.594+0000 7f4735fd4740 1 client.15873 shutdown 2024-03-08T21:05:30.577 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.594+0000 7f4735fd4740 1 -- 172.21.15.107:0/754788829 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561e6f3bd5a0 msgr2=0x561e6f3c3190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:05:30.577 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.594+0000 7f4735fd4740 1 --2- 172.21.15.107:0/754788829 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561e6f3bd5a0 0x561e6f3c3190 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f4714000c00 tx=0x7f4714009260 comp rx=0 tx=0).stop 2024-03-08T21:05:30.577 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.594+0000 7f4735fd4740 1 -- 172.21.15.107:0/754788829 shutdown_connections 2024-03-08T21:05:30.577 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.594+0000 7f4735fd4740 1 --2- 172.21.15.107:0/754788829 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561e6f3c6370 0x561e6f3c3880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:05:30.577 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.594+0000 7f4735fd4740 1 --2- 172.21.15.107:0/754788829 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561e6f3bd5a0 0x561e6f3c3190 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:05:30.577 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.594+0000 7f4735fd4740 1 --2- 172.21.15.107:0/754788829 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x561e6f3d6180 0x561e6f3d85f0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:05:30.578 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.594+0000 7f4735fd4740 1 --2- 172.21.15.107:0/754788829 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561e6f3bcc80 0x561e6f3c5e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:05:30.578 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.594+0000 7f4735fd4740 1 -- 172.21.15.107:0/754788829 >> 172.21.15.107:0/754788829 conn(0x561e6f3b75a0 msgr2=0x561e6f3b92f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:05:30.578 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.594+0000 7f471a000640 0 client.15873 ms_handle_reset on v2:172.21.15.149:6838/55290596 2024-03-08T21:05:30.578 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.594+0000 7f4735fd4740 1 -- 172.21.15.107:0/754788829 shutdown_connections 2024-03-08T21:05:30.578 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.594+0000 7f4735fd4740 1 -- 172.21.15.107:0/754788829 wait complete. 2024-03-08T21:05:30.578 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.594+0000 7f4735fd4740 20 client.15873 trim_cache size 0 max 16384 2024-03-08T21:05:30.592 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.612+0000 7f4735fd4740 1 Processor -- start 2024-03-08T21:05:30.592 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.612+0000 7f4735fd4740 1 -- start start 2024-03-08T21:05:30.594 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.612+0000 7f4735fd4740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561e6f3bcc80 0x561e6f3c58c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:05:30.594 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.612+0000 7f4735fd4740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561e6f3bd5a0 0x561e6f3c5e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:05:30.594 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.612+0000 7f4735fd4740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561e6f3c6370 0x561e6f3c0470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:05:30.594 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.612+0000 7f4735fd4740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f472c1aee40 con 0x561e6f3bd5a0 2024-03-08T21:05:30.594 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.612+0000 7f4735fd4740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f472c1b07c0 con 0x561e6f3bcc80 2024-03-08T21:05:30.594 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.612+0000 7f4735fd4740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f472c1af4a0 con 0x561e6f3c6370 2024-03-08T21:05:30.594 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.612+0000 7f472b400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561e6f3bd5a0 0x561e6f3c5e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:05:30.595 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.613+0000 7f472b400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561e6f3bd5a0 0x561e6f3c5e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:45606/0 (socket says 172.21.15.107:45606) 2024-03-08T21:05:30.595 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.613+0000 7f472b400640 1 -- 172.21.15.107:0/1701075292 learned_addr learned my addr 172.21.15.107:0/1701075292 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:05:30.595 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.613+0000 7f472be00640 1 --2- 172.21.15.107:0/1701075292 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561e6f3bcc80 0x561e6f3c58c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:05:30.595 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.613+0000 7f4731200640 1 --2- 172.21.15.107:0/1701075292 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561e6f3c6370 0x561e6f3c0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:05:30.595 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.613+0000 7f472b400640 1 -- 172.21.15.107:0/1701075292 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561e6f3c6370 msgr2=0x561e6f3c0470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:05:30.595 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.613+0000 7f472b400640 1 --2- 172.21.15.107:0/1701075292 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561e6f3c6370 0x561e6f3c0470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:05:30.595 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.613+0000 7f472b400640 1 -- 172.21.15.107:0/1701075292 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561e6f3bcc80 msgr2=0x561e6f3c58c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:05:30.595 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.613+0000 7f472b400640 1 --2- 172.21.15.107:0/1701075292 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561e6f3bcc80 0x561e6f3c58c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:05:30.595 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.613+0000 7f472b400640 1 -- 172.21.15.107:0/1701075292 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561e6f3dd820 con 0x561e6f3bd5a0 2024-03-08T21:05:30.595 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.613+0000 7f4731200640 1 --2- 172.21.15.107:0/1701075292 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561e6f3c6370 0x561e6f3c0470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T21:05:30.595 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.613+0000 7f472be00640 1 --2- 172.21.15.107:0/1701075292 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561e6f3bcc80 0x561e6f3c58c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T21:05:30.595 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.613+0000 7f472b400640 1 --2- 172.21.15.107:0/1701075292 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561e6f3bd5a0 0x561e6f3c5e00 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f47140f0010 tx=0x7f471402bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fca8dcccf1d08f8f server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:05:30.595 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.613+0000 7f4720c00640 1 -- 172.21.15.107:0/1701075292 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47140d6040 con 0x561e6f3bd5a0 2024-03-08T21:05:30.595 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.613+0000 7f4720c00640 1 -- 172.21.15.107:0/1701075292 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f47140bd600 con 0x561e6f3bd5a0 2024-03-08T21:05:30.595 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.614+0000 7f4720c00640 1 -- 172.21.15.107:0/1701075292 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47140b9040 con 0x561e6f3bd5a0 2024-03-08T21:05:30.598 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.614+0000 7f4735fd4740 1 -- 172.21.15.107:0/1701075292 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561e6f3bd5a0 msgr2=0x561e6f3c5e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:05:30.599 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.614+0000 7f4735fd4740 1 --2- 172.21.15.107:0/1701075292 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561e6f3bd5a0 0x561e6f3c5e00 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f47140f0010 tx=0x7f471402bc30 comp rx=0 tx=0).stop 2024-03-08T21:05:30.599 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.614+0000 7f4735fd4740 1 -- 172.21.15.107:0/1701075292 shutdown_connections 2024-03-08T21:05:30.599 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.614+0000 7f4735fd4740 1 --2- 172.21.15.107:0/1701075292 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561e6f3c6370 0x561e6f3c0470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:05:30.599 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.614+0000 7f4735fd4740 1 --2- 172.21.15.107:0/1701075292 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561e6f3bd5a0 0x561e6f3c5e00 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:05:30.599 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.614+0000 7f4735fd4740 1 --2- 172.21.15.107:0/1701075292 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561e6f3bcc80 0x561e6f3c58c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:05:30.599 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.614+0000 7f4735fd4740 1 -- 172.21.15.107:0/1701075292 >> 172.21.15.107:0/1701075292 conn(0x561e6f3b75a0 msgr2=0x561e6f3c1ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:05:30.599 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.614+0000 7f4735fd4740 1 -- 172.21.15.107:0/1701075292 shutdown_connections 2024-03-08T21:05:30.599 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.615+0000 7f4735fd4740 1 -- 172.21.15.107:0/1701075292 wait complete. 2024-03-08T21:05:30.599 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.615+0000 7f4735fd4740 1 -- start start 2024-03-08T21:05:30.599 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.615+0000 7f4735fd4740 1 Processor -- start 2024-03-08T21:05:30.599 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.616+0000 7f4735fd4740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561e6f3bcc80 0x561e6f3cd6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:05:30.599 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.616+0000 7f4735fd4740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561e6f3bd5a0 0x561e6f3cbb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:05:30.600 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.616+0000 7f4735fd4740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561e6f3c6370 0x561e6f3cc0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:05:30.600 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.616+0000 7f4735fd4740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f472c1b0160 con 0x561e6f3c6370 2024-03-08T21:05:30.600 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.616+0000 7f4735fd4740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f472c1afb00 con 0x561e6f3bcc80 2024-03-08T21:05:30.600 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.616+0000 7f4735fd4740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f472c1ad100 con 0x561e6f3bd5a0 2024-03-08T21:05:30.600 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.616+0000 7f4735fd4740 10 client.0 _ll_register_callbacks cb 0x7f4724009e00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T21:05:30.600 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.616+0000 7f4731200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561e6f3c6370 0x561e6f3cc0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:05:30.600 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.616+0000 7f4731200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561e6f3c6370 0x561e6f3cc0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:45620/0 (socket says 172.21.15.107:45620) 2024-03-08T21:05:30.600 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.616+0000 7f4731200640 1 -- 172.21.15.107:0/3293417456 learned_addr learned my addr 172.21.15.107:0/3293417456 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:05:30.600 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.616+0000 7f472be00640 1 --2- 172.21.15.107:0/3293417456 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561e6f3bcc80 0x561e6f3cd6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:05:30.600 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.616+0000 7f472b400640 1 --2- 172.21.15.107:0/3293417456 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561e6f3bd5a0 0x561e6f3cbb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:05:30.600 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.616+0000 7f4731200640 1 -- 172.21.15.107:0/3293417456 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561e6f3bd5a0 msgr2=0x561e6f3cbb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:05:30.600 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.616+0000 7f4731200640 1 --2- 172.21.15.107:0/3293417456 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561e6f3bd5a0 0x561e6f3cbb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:05:30.600 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.616+0000 7f4731200640 1 -- 172.21.15.107:0/3293417456 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561e6f3bcc80 msgr2=0x561e6f3cd6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:05:30.600 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.616+0000 7f4731200640 1 --2- 172.21.15.107:0/3293417456 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561e6f3bcc80 0x561e6f3cd6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:05:30.601 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.616+0000 7f4731200640 1 -- 172.21.15.107:0/3293417456 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561e6f3cc8d0 con 0x561e6f3c6370 2024-03-08T21:05:30.601 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.616+0000 7f472b400640 1 --2- 172.21.15.107:0/3293417456 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561e6f3bd5a0 0x561e6f3cbb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T21:05:30.601 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.617+0000 7f4731200640 1 --2- 172.21.15.107:0/3293417456 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561e6f3c6370 0x561e6f3cc0a0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f4724009600 tx=0x7f4724014a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=850c39c237194c18 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:05:30.601 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.617+0000 7f471a000640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-03-08T21:05:30.601 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.617+0000 7f471a000640 1 -- 172.21.15.107:0/3293417456 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4724003d80 con 0x561e6f3c6370 2024-03-08T21:05:30.601 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.617+0000 7f4735fd4740 1 -- 172.21.15.107:0/3293417456 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f47140bd600 con 0x561e6f3c6370 2024-03-08T21:05:30.601 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.617+0000 7f471a000640 1 -- 172.21.15.107:0/3293417456 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f4724005330 con 0x561e6f3c6370 2024-03-08T21:05:30.601 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.617+0000 7f471a000640 1 -- 172.21.15.107:0/3293417456 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4724005530 con 0x561e6f3c6370 2024-03-08T21:05:30.601 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.617+0000 7f4735fd4740 10 client.15879 fetch_fsmap learned FSMap version 34 2024-03-08T21:05:30.601 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.617+0000 7f4735fd4740 1 -- 172.21.15.107:0/3293417456 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x561e6f42d770 con 0x561e6f3c6370 2024-03-08T21:05:30.601 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.618+0000 7f471a000640 1 -- 172.21.15.107:0/3293417456 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f47140bd600 con 0x561e6f3c6370 2024-03-08T21:05:30.601 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.618+0000 7f471a000640 1 -- 172.21.15.107:0/3293417456 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 34) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f472401d070 con 0x561e6f3c6370 2024-03-08T21:05:30.601 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.618+0000 7f4735fd4740 10 client.15879 fetch_fsmap finished waiting for FSMap version 34 2024-03-08T21:05:30.602 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.618+0000 7f4735fd4740 10 client.15879 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-08T21:05:30.602 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.618+0000 7f4735fd4740 20 client.15879 populate_metadata read hostname 'smithi107' 2024-03-08T21:05:30.602 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.618+0000 7f4735fd4740 1 --2- 172.21.15.107:0/3293417456 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x561e6f42ddb0 0x561e6f4301d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:05:30.602 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.618+0000 7f4735fd4740 4 client.15879 mds_command: new command op to 34475 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-08T21:05:30.602 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.618+0000 7f4735fd4740 1 -- 172.21.15.107:0/3293417456 --> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x561e6f2a65d0 con 0x561e6f42ddb0 2024-03-08T21:05:30.602 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.619+0000 7f472be00640 1 --2- 172.21.15.107:0/3293417456 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x561e6f42ddb0 0x561e6f4301d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:05:30.602 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.619+0000 7f471a000640 10 client.15879 ms_handle_connect on v2:172.21.15.149:6838/55290596 2024-03-08T21:05:30.603 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.619+0000 7f472be00640 1 --2- 172.21.15.107:0/3293417456 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x561e6f42ddb0 0x561e6f4301d0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bb16ca40be1886e3 server_cookie=42b8d66cdf28cbe6 in_seq=0 out_seq=0 2024-03-08T21:05:30.622 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.641+0000 7f471a000640 1 -- 172.21.15.107:0/3293417456 <== mds.0 v2:172.21.15.149:6838/55290596 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x561e6f2a65d0 con 0x561e6f42ddb0 2024-03-08T21:05:30.622 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.641+0000 7f471a000640 10 client.15879 handle_command_reply: tid=0 2024-03-08T21:05:30.622 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.641+0000 7f4735fd4740 1 client.15879 shutdown 2024-03-08T21:05:30.623 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.642+0000 7f4735fd4740 1 -- 172.21.15.107:0/3293417456 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561e6f3c6370 msgr2=0x561e6f3cc0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:05:30.623 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.642+0000 7f4735fd4740 1 --2- 172.21.15.107:0/3293417456 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561e6f3c6370 0x561e6f3cc0a0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f4724009600 tx=0x7f4724014a00 comp rx=0 tx=0).stop 2024-03-08T21:05:30.623 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.642+0000 7f4735fd4740 1 -- 172.21.15.107:0/3293417456 shutdown_connections 2024-03-08T21:05:30.623 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.642+0000 7f4735fd4740 1 --2- 172.21.15.107:0/3293417456 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561e6f3c6370 0x561e6f3cc0a0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:05:30.623 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.642+0000 7f4735fd4740 1 --2- 172.21.15.107:0/3293417456 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561e6f3bd5a0 0x561e6f3cbb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:05:30.623 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.642+0000 7f4735fd4740 1 --2- 172.21.15.107:0/3293417456 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x561e6f42ddb0 0x561e6f4301d0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:05:30.623 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.642+0000 7f4735fd4740 1 --2- 172.21.15.107:0/3293417456 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561e6f3bcc80 0x561e6f3cd6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:05:30.624 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.642+0000 7f4735fd4740 1 -- 172.21.15.107:0/3293417456 >> 172.21.15.107:0/3293417456 conn(0x561e6f3b75a0 msgr2=0x561e6f3b5e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:05:30.624 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.642+0000 7f471a000640 0 client.15879 ms_handle_reset on v2:172.21.15.149:6838/55290596 2024-03-08T21:05:30.624 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.642+0000 7f4735fd4740 1 -- 172.21.15.107:0/3293417456 shutdown_connections 2024-03-08T21:05:30.624 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.642+0000 7f4735fd4740 1 -- 172.21.15.107:0/3293417456 wait complete. 2024-03-08T21:05:30.624 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.642+0000 7f4735fd4740 20 client.15879 trim_cache size 0 max 16384 2024-03-08T21:05:30.624 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:{ 2024-03-08T21:05:30.624 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "message": "", 2024-03-08T21:05:30.624 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "return_code": 0 2024-03-08T21:05:30.624 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:} 2024-03-08T21:05:30.625 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.643+0000 7f4733200640 1 -- 172.21.15.107:0/643329111 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f47100a2ac0 msgr2=0x7f47100a4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:05:30.625 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.643+0000 7f4733200640 1 --2- 172.21.15.107:0/643329111 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f47100a2ac0 0x7f47100a4f80 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f4724009fd0 tx=0x7f472401c000 comp rx=0 tx=0).stop 2024-03-08T21:05:30.625 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.643+0000 7f4733200640 1 -- 172.21.15.107:0/643329111 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f472c1065d0 msgr2=0x7f472c073b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:05:30.625 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.643+0000 7f4733200640 1 --2- 172.21.15.107:0/643329111 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f472c1065d0 0x7f472c073b30 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f4714036040 tx=0x7f47140045e0 comp rx=0 tx=0).stop 2024-03-08T21:05:30.625 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.643+0000 7f4733200640 1 -- 172.21.15.107:0/643329111 shutdown_connections 2024-03-08T21:05:30.626 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.643+0000 7f4733200640 1 --2- 172.21.15.107:0/643329111 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f47100a2ac0 0x7f47100a4f80 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:05:30.626 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.643+0000 7f4733200640 1 --2- 172.21.15.107:0/643329111 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f472c074070 0x7f472c1a5c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:05:30.626 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.643+0000 7f4733200640 1 --2- 172.21.15.107:0/643329111 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f472c1065d0 0x7f472c073b30 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:05:30.626 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.643+0000 7f4733200640 1 --2- 172.21.15.107:0/643329111 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f472c101500 0x7f472c0735f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:05:30.626 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.643+0000 7f4733200640 1 -- 172.21.15.107:0/643329111 >> 172.21.15.107:0/643329111 conn(0x7f472c0fcd80 msgr2=0x7f472c0fe130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:05:30.626 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.643+0000 7f4733200640 1 -- 172.21.15.107:0/643329111 shutdown_connections 2024-03-08T21:05:30.626 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:30.644+0000 7f4733200640 1 -- 172.21.15.107:0/643329111 wait complete. 2024-03-08T21:05:31.964 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:05:31 smithi107 ceph-mon[31381]: pgmap v155: 97 pgs: 97 active+clean; 3.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-08T21:05:32.012 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:05:31 smithi149 ceph-mon[35042]: pgmap v155: 97 pgs: 97 active+clean; 3.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-08T21:05:32.125 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:05:31 smithi204 ceph-mon[34935]: pgmap v155: 97 pgs: 97 active+clean; 3.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-08T21:05:33.928 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:05:33 smithi204 ceph-mon[34935]: pgmap v156: 97 pgs: 97 active+clean; 3.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-08T21:05:33.964 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:05:33 smithi107 ceph-mon[31381]: pgmap v156: 97 pgs: 97 active+clean; 3.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-08T21:05:34.012 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:05:33 smithi149 ceph-mon[35042]: pgmap v156: 97 pgs: 97 active+clean; 3.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-08T21:05:34.963 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:05:34 smithi204 ceph-mon[34935]: pgmap v157: 97 pgs: 97 active+clean; 3.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 4.2 KiB/s wr, 14 op/s 2024-03-08T21:05:34.964 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:05:34 smithi107 ceph-mon[31381]: pgmap v157: 97 pgs: 97 active+clean; 3.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 4.2 KiB/s wr, 14 op/s 2024-03-08T21:05:35.012 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:05:34 smithi149 ceph-mon[35042]: pgmap v157: 97 pgs: 97 active+clean; 3.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 4.2 KiB/s wr, 14 op/s 2024-03-08T21:05:37.964 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:05:37 smithi107 ceph-mon[31381]: pgmap v158: 97 pgs: 97 active+clean; 3.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 4.2 KiB/s wr, 10 op/s 2024-03-08T21:05:38.012 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:05:37 smithi149 ceph-mon[35042]: pgmap v158: 97 pgs: 97 active+clean; 3.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 4.2 KiB/s wr, 10 op/s 2024-03-08T21:05:38.125 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:05:37 smithi204 ceph-mon[34935]: pgmap v158: 97 pgs: 97 active+clean; 3.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 4.2 KiB/s wr, 10 op/s 2024-03-08T21:05:39.964 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:05:39 smithi107 ceph-mon[31381]: pgmap v159: 97 pgs: 97 active+clean; 3.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 4.2 KiB/s wr, 14 op/s 2024-03-08T21:05:40.012 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:05:39 smithi149 ceph-mon[35042]: pgmap v159: 97 pgs: 97 active+clean; 3.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 4.2 KiB/s wr, 14 op/s 2024-03-08T21:05:40.125 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:05:39 smithi204 ceph-mon[34935]: pgmap v159: 97 pgs: 97 active+clean; 3.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 4.2 KiB/s wr, 14 op/s 2024-03-08T21:05:41.964 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:05:41 smithi107 ceph-mon[31381]: pgmap v160: 97 pgs: 97 active+clean; 3.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 4.2 KiB/s wr, 10 op/s 2024-03-08T21:05:42.012 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:05:41 smithi149 ceph-mon[35042]: pgmap v160: 97 pgs: 97 active+clean; 3.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 4.2 KiB/s wr, 10 op/s 2024-03-08T21:05:42.125 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:05:41 smithi204 ceph-mon[34935]: pgmap v160: 97 pgs: 97 active+clean; 3.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 4.2 KiB/s wr, 10 op/s 2024-03-08T21:05:43.776 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.792+0000 7f0a86a00640 1 -- 172.21.15.107:0/3014973401 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0a80101340 msgr2=0x7f0a80101720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:05:43.776 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.792+0000 7f0a86a00640 1 --2- 172.21.15.107:0/3014973401 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0a80101340 0x7f0a80101720 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f0a74009ca0 tx=0x7f0a7402c100 comp rx=0 tx=0).stop 2024-03-08T21:05:43.776 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.793+0000 7f0a86a00640 1 -- 172.21.15.107:0/3014973401 shutdown_connections 2024-03-08T21:05:43.776 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.793+0000 7f0a86a00640 1 --2- 172.21.15.107:0/3014973401 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0a80106410 0x7f0a8010dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:05:43.776 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.793+0000 7f0a86a00640 1 --2- 172.21.15.107:0/3014973401 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0a80101cf0 0x7f0a80105d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:05:43.776 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.793+0000 7f0a86a00640 1 --2- 172.21.15.107:0/3014973401 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0a80101340 0x7f0a80101720 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:05:43.776 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.793+0000 7f0a86a00640 1 -- 172.21.15.107:0/3014973401 >> 172.21.15.107:0/3014973401 conn(0x7f0a800fcc40 msgr2=0x7f0a800ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:05:43.777 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.793+0000 7f0a86a00640 1 -- 172.21.15.107:0/3014973401 shutdown_connections 2024-03-08T21:05:43.777 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.793+0000 7f0a86a00640 1 -- 172.21.15.107:0/3014973401 wait complete. 2024-03-08T21:05:43.777 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.794+0000 7f0a86a00640 1 Processor -- start 2024-03-08T21:05:43.777 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.794+0000 7f0a86a00640 1 -- start start 2024-03-08T21:05:43.777 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.794+0000 7f0a86a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0a80101340 0x7f0a8019f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:05:43.777 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.794+0000 7f0a86a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0a80106410 0x7f0a8019fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:05:43.777 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.794+0000 7f0a86a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0a801998b0 0x7f0a80199d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:05:43.777 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.794+0000 7f0a86a00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a80111350 con 0x7f0a80101340 2024-03-08T21:05:43.777 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.794+0000 7f0a86a00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a801111d0 con 0x7f0a801998b0 2024-03-08T21:05:43.778 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.794+0000 7f0a86a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a801114d0 con 0x7f0a80106410 2024-03-08T21:05:43.778 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.795+0000 7f0a85600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0a80101340 0x7f0a8019f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:05:43.778 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.795+0000 7f0a85600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0a80101340 0x7f0a8019f700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:35932/0 (socket says 172.21.15.107:35932) 2024-03-08T21:05:43.778 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.795+0000 7f0a85600640 1 -- 172.21.15.107:0/58449224 learned_addr learned my addr 172.21.15.107:0/58449224 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:05:43.778 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.795+0000 7f0a85600640 1 -- 172.21.15.107:0/58449224 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0a80106410 msgr2=0x7f0a8019fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:05:43.778 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.795+0000 7f0a85600640 1 --2- 172.21.15.107:0/58449224 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0a80106410 0x7f0a8019fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:05:43.778 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.795+0000 7f0a85600640 1 -- 172.21.15.107:0/58449224 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0a801998b0 msgr2=0x7f0a80199d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:05:43.779 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.795+0000 7f0a85600640 1 --2- 172.21.15.107:0/58449224 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0a801998b0 0x7f0a80199d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:05:43.779 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.795+0000 7f0a85600640 1 -- 172.21.15.107:0/58449224 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a8019a3d0 con 0x7f0a80101340 2024-03-08T21:05:43.779 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.795+0000 7f0a85600640 1 --2- 172.21.15.107:0/58449224 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0a80101340 0x7f0a8019f700 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f0a74004af0 tx=0x7f0a74035ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:05:43.779 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.796+0000 7f0a7e000640 1 -- 172.21.15.107:0/58449224 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a74039040 con 0x7f0a80101340 2024-03-08T21:05:43.779 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.796+0000 7f0a7e000640 1 -- 172.21.15.107:0/58449224 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f0a74044070 con 0x7f0a80101340 2024-03-08T21:05:43.779 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.796+0000 7f0a7e000640 1 -- 172.21.15.107:0/58449224 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a740043c0 con 0x7f0a80101340 2024-03-08T21:05:43.779 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.796+0000 7f0a86a00640 1 -- 172.21.15.107:0/58449224 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a8019a660 con 0x7f0a80101340 2024-03-08T21:05:43.779 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.796+0000 7f0a86a00640 1 -- 172.21.15.107:0/58449224 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a801a6800 con 0x7f0a80101340 2024-03-08T21:05:43.783 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.800+0000 7f0a8ac25740 1 Processor -- start 2024-03-08T21:05:43.783 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.800+0000 7f0a8ac25740 1 -- start start 2024-03-08T21:05:43.783 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.800+0000 7f0a7e000640 1 -- 172.21.15.107:0/58449224 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) v1 ==== 137422+0+0 (secure 0 0 0) 0x7f0a740326c0 con 0x7f0a80101340 2024-03-08T21:05:43.783 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.800+0000 7f0a8ac25740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56371f1d9e80 0x56371f1da260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:05:43.784 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.800+0000 7f0a8ac25740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56371f1da7a0 0x56371f1e3030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:05:43.784 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.800+0000 7f0a8ac25740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56371f1e3570 0x56371f1e5960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:05:43.784 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.800+0000 7f0a8ac25740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x56371f13d3d0 con 0x56371f1e3570 2024-03-08T21:05:43.784 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.800+0000 7f0a8ac25740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x56371f0e66b0 con 0x56371f1d9e80 2024-03-08T21:05:43.784 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.800+0000 7f0a8ac25740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56371f139470 con 0x56371f1da7a0 2024-03-08T21:05:43.784 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.801+0000 7f0a84c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56371f1e3570 0x56371f1e5960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:05:43.784 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.801+0000 7f0a84c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56371f1e3570 0x56371f1e5960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:35940/0 (socket says 172.21.15.107:35940) 2024-03-08T21:05:43.784 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.801+0000 7f0a84c00640 1 -- 172.21.15.107:0/3081366867 learned_addr learned my addr 172.21.15.107:0/3081366867 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:05:43.784 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.801+0000 7f0a86000640 1 --2- 172.21.15.107:0/3081366867 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56371f1d9e80 0x56371f1da260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:05:43.784 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.801+0000 7f0a85600640 1 --2- 172.21.15.107:0/3081366867 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56371f1da7a0 0x56371f1e3030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:05:43.785 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.801+0000 7f0a84c00640 1 -- 172.21.15.107:0/3081366867 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56371f1da7a0 msgr2=0x56371f1e3030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:05:43.785 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.801+0000 7f0a84c00640 1 --2- 172.21.15.107:0/3081366867 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56371f1da7a0 0x56371f1e3030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:05:43.785 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.801+0000 7f0a84c00640 1 -- 172.21.15.107:0/3081366867 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56371f1d9e80 msgr2=0x56371f1da260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:05:43.785 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.801+0000 7f0a84c00640 1 --2- 172.21.15.107:0/3081366867 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56371f1d9e80 0x56371f1da260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:05:43.785 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.801+0000 7f0a84c00640 1 -- 172.21.15.107:0/3081366867 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56371f10cb50 con 0x56371f1e3570 2024-03-08T21:05:43.785 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.802+0000 7f0a7e000640 1 --2- 172.21.15.107:0/58449224 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f0a680a2960 0x7f0a680a4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:05:43.785 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.802+0000 7f0a84c00640 1 --2- 172.21.15.107:0/3081366867 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56371f1e3570 0x56371f1e5960 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f0a6c00ba70 tx=0x7f0a6c00bf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7aa224edebc63cd5 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:05:43.785 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.802+0000 7f0a7e000640 1 -- 172.21.15.107:0/58449224 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f0a740f1230 con 0x7f0a80101340 2024-03-08T21:05:43.786 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.802+0000 7f0a86000640 1 --2- 172.21.15.107:0/58449224 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f0a680a2960 0x7f0a680a4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:05:43.786 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.802+0000 7f0a72600640 1 -- 172.21.15.107:0/3081366867 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a6c016070 con 0x56371f1e3570 2024-03-08T21:05:43.786 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.802+0000 7f0a72600640 1 -- 172.21.15.107:0/3081366867 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f0a6c0054d0 con 0x56371f1e3570 2024-03-08T21:05:43.786 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.802+0000 7f0a72600640 1 -- 172.21.15.107:0/3081366867 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a6c004e60 con 0x56371f1e3570 2024-03-08T21:05:43.786 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.803+0000 7f0a8ac25740 1 -- 172.21.15.107:0/3081366867 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56371f1e3570 msgr2=0x56371f1e5960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:05:43.786 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.803+0000 7f0a86000640 1 --2- 172.21.15.107:0/58449224 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f0a680a2960 0x7f0a680a4e20 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f0a78005fd0 tx=0x7f0a78005de0 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:05:43.787 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.803+0000 7f0a8ac25740 1 --2- 172.21.15.107:0/3081366867 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56371f1e3570 0x56371f1e5960 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f0a6c00ba70 tx=0x7f0a6c00bf30 comp rx=0 tx=0).stop 2024-03-08T21:05:43.787 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.803+0000 7f0a8ac25740 1 -- 172.21.15.107:0/3081366867 shutdown_connections 2024-03-08T21:05:43.787 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.803+0000 7f0a8ac25740 1 --2- 172.21.15.107:0/3081366867 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56371f1e3570 0x56371f1e5960 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:05:43.787 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.803+0000 7f0a8ac25740 1 --2- 172.21.15.107:0/3081366867 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56371f1da7a0 0x56371f1e3030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:05:43.787 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.803+0000 7f0a8ac25740 1 --2- 172.21.15.107:0/3081366867 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56371f1d9e80 0x56371f1da260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:05:43.787 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.803+0000 7f0a8ac25740 1 -- 172.21.15.107:0/3081366867 >> 172.21.15.107:0/3081366867 conn(0x56371f1d47a0 msgr2=0x56371f1d4b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:05:43.788 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.803+0000 7f0a8ac25740 1 -- 172.21.15.107:0/3081366867 shutdown_connections 2024-03-08T21:05:43.788 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.803+0000 7f0a8ac25740 1 -- 172.21.15.107:0/3081366867 wait complete. 2024-03-08T21:05:43.788 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.804+0000 7f0a8ac25740 1 -- start start 2024-03-08T21:05:43.796 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.804+0000 7f0a8ac25740 1 Processor -- start 2024-03-08T21:05:43.796 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.805+0000 7f0a8ac25740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56371f1d9e80 0x56371f1e2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:05:43.796 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.805+0000 7f0a8ac25740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56371f1da7a0 0x56371f1e0280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:05:43.796 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.805+0000 7f0a8ac25740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56371f1e3570 0x56371f1e0800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:05:43.796 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.805+0000 7f0a8ac25740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x56371f1e61a0 con 0x56371f1d9e80 2024-03-08T21:05:43.797 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.805+0000 7f0a8ac25740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x56371f0f99f0 con 0x56371f1e3570 2024-03-08T21:05:43.797 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.805+0000 7f0a8ac25740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56371f1e6320 con 0x56371f1da7a0 2024-03-08T21:05:43.797 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.805+0000 7f0a8ac25740 10 client.0 _ll_register_callbacks cb 0x56371ee51a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T21:05:43.797 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.805+0000 7f0a85600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56371f1d9e80 0x56371f1e2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:05:43.797 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.805+0000 7f0a85600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56371f1d9e80 0x56371f1e2ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:35950/0 (socket says 172.21.15.107:35950) 2024-03-08T21:05:43.798 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.805+0000 7f0a85600640 1 -- 172.21.15.107:0/62226677 learned_addr learned my addr 172.21.15.107:0/62226677 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:05:43.798 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.805+0000 7f0a84c00640 1 --2- 172.21.15.107:0/62226677 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56371f1da7a0 0x56371f1e0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:05:43.798 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.805+0000 7f0a86000640 1 --2- 172.21.15.107:0/62226677 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56371f1e3570 0x56371f1e0800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:05:43.798 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.805+0000 7f0a85600640 1 -- 172.21.15.107:0/62226677 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56371f1da7a0 msgr2=0x56371f1e0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:05:43.798 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.805+0000 7f0a85600640 1 --2- 172.21.15.107:0/62226677 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56371f1da7a0 0x56371f1e0280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:05:43.798 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.805+0000 7f0a85600640 1 -- 172.21.15.107:0/62226677 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56371f1e3570 msgr2=0x56371f1e0800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:05:43.799 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.805+0000 7f0a85600640 1 --2- 172.21.15.107:0/62226677 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56371f1e3570 0x56371f1e0800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:05:43.799 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.805+0000 7f0a85600640 1 -- 172.21.15.107:0/62226677 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56371ee52210 con 0x56371f1d9e80 2024-03-08T21:05:43.799 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.806+0000 7f0a85600640 1 --2- 172.21.15.107:0/62226677 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56371f1d9e80 0x56371f1e2ef0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f0a74005e30 tx=0x7f0a74009800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d0dea3284dff6bd server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:05:43.799 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.806+0000 7f0a66a00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-03-08T21:05:43.799 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.806+0000 7f0a66a00640 1 -- 172.21.15.107:0/62226677 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a740bd2f0 con 0x56371f1d9e80 2024-03-08T21:05:43.800 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.806+0000 7f0a66a00640 1 -- 172.21.15.107:0/62226677 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f0a740c5ea0 con 0x56371f1d9e80 2024-03-08T21:05:43.800 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.806+0000 7f0a8ac25740 1 -- 172.21.15.107:0/62226677 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0a6c0054d0 con 0x56371f1d9e80 2024-03-08T21:05:43.800 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.806+0000 7f0a66a00640 1 -- 172.21.15.107:0/62226677 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0a740be090 con 0x56371f1d9e80 2024-03-08T21:05:43.800 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.806+0000 7f0a8ac25740 10 client.15891 fetch_fsmap learned FSMap version 34 2024-03-08T21:05:43.800 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.806+0000 7f0a8ac25740 1 -- 172.21.15.107:0/62226677 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x56371ef1e620 con 0x56371f1d9e80 2024-03-08T21:05:43.800 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.806+0000 7f0a66a00640 1 -- 172.21.15.107:0/62226677 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f0a6c0054d0 con 0x56371f1d9e80 2024-03-08T21:05:43.801 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.807+0000 7f0a66a00640 1 -- 172.21.15.107:0/62226677 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 34) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f0a7403c030 con 0x56371f1d9e80 2024-03-08T21:05:43.801 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.807+0000 7f0a8ac25740 10 client.15891 fetch_fsmap finished waiting for FSMap version 34 2024-03-08T21:05:43.801 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.807+0000 7f0a8ac25740 10 client.15891 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-08T21:05:43.801 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.807+0000 7f0a8ac25740 20 client.15891 populate_metadata read hostname 'smithi107' 2024-03-08T21:05:43.801 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.807+0000 7f0a8ac25740 1 --2- 172.21.15.107:0/62226677 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x56371f1f3380 0x56371f1f57f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:05:43.801 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.807+0000 7f0a8ac25740 4 client.15891 mds_command: new command op to 34475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-08T21:05:43.802 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.807+0000 7f0a8ac25740 1 -- 172.21.15.107:0/62226677 --> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56371f14a910 con 0x56371f1f3380 2024-03-08T21:05:43.802 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.808+0000 7f0a84c00640 1 --2- 172.21.15.107:0/62226677 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x56371f1f3380 0x56371f1f57f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:05:43.802 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.808+0000 7f0a84c00640 1 --2- 172.21.15.107:0/62226677 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x56371f1f3380 0x56371f1f57f0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d8d7faf533dd9a62 server_cookie=aa27e0d4979676a7 in_seq=0 out_seq=0 2024-03-08T21:05:43.802 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.808+0000 7f0a66a00640 10 client.15891 ms_handle_connect on v2:172.21.15.149:6838/55290596 2024-03-08T21:05:43.802 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.810+0000 7f0a66a00640 1 -- 172.21.15.107:0/62226677 <== mds.0 v2:172.21.15.149:6838/55290596 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56371f14a910 con 0x56371f1f3380 2024-03-08T21:05:43.803 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.810+0000 7f0a66a00640 10 client.15891 handle_command_reply: tid=0 2024-03-08T21:05:43.803 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.810+0000 7f0a8ac25740 1 client.15891 shutdown 2024-03-08T21:05:43.803 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.810+0000 7f0a8ac25740 1 -- 172.21.15.107:0/62226677 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56371f1d9e80 msgr2=0x56371f1e2ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:05:43.803 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.810+0000 7f0a8ac25740 1 --2- 172.21.15.107:0/62226677 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56371f1d9e80 0x56371f1e2ef0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f0a74005e30 tx=0x7f0a74009800 comp rx=0 tx=0).stop 2024-03-08T21:05:43.803 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.811+0000 7f0a8ac25740 1 -- 172.21.15.107:0/62226677 shutdown_connections 2024-03-08T21:05:43.803 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.811+0000 7f0a8ac25740 1 --2- 172.21.15.107:0/62226677 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x56371f1f3380 0x56371f1f57f0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:05:43.804 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.811+0000 7f0a8ac25740 1 --2- 172.21.15.107:0/62226677 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56371f1e3570 0x56371f1e0800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:05:43.804 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.811+0000 7f0a8ac25740 1 --2- 172.21.15.107:0/62226677 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56371f1da7a0 0x56371f1e0280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:05:43.804 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.811+0000 7f0a8ac25740 1 --2- 172.21.15.107:0/62226677 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56371f1d9e80 0x56371f1e2ef0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:05:43.804 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.811+0000 7f0a66a00640 0 client.15891 ms_handle_reset on v2:172.21.15.149:6838/55290596 2024-03-08T21:05:43.804 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.811+0000 7f0a8ac25740 1 -- 172.21.15.107:0/62226677 >> 172.21.15.107:0/62226677 conn(0x56371f1d47a0 msgr2=0x56371f1d64f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:05:43.804 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.811+0000 7f0a8ac25740 1 -- 172.21.15.107:0/62226677 shutdown_connections 2024-03-08T21:05:43.805 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.811+0000 7f0a8ac25740 1 -- 172.21.15.107:0/62226677 wait complete. 2024-03-08T21:05:43.805 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.811+0000 7f0a8ac25740 20 client.15891 trim_cache size 0 max 16384 2024-03-08T21:05:43.812 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.826+0000 7f0a8ac25740 1 Processor -- start 2024-03-08T21:05:43.812 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.826+0000 7f0a8ac25740 1 -- start start 2024-03-08T21:05:43.812 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.826+0000 7f0a8ac25740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56371f1d9e80 0x56371f1fa160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:05:43.812 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.826+0000 7f0a8ac25740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56371f1da7a0 0x56371f1e29b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:05:43.812 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.826+0000 7f0a8ac25740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56371f1e3570 0x56371f1e2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:05:43.812 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.826+0000 7f0a8ac25740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a801aebc0 con 0x56371f1e3570 2024-03-08T21:05:43.813 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.826+0000 7f0a8ac25740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a801b0540 con 0x56371f1da7a0 2024-03-08T21:05:43.813 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.826+0000 7f0a8ac25740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a801af220 con 0x56371f1d9e80 2024-03-08T21:05:43.813 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.826+0000 7f0a86000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56371f1e3570 0x56371f1e2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:05:43.813 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.826+0000 7f0a86000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56371f1e3570 0x56371f1e2ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:35954/0 (socket says 172.21.15.107:35954) 2024-03-08T21:05:43.813 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.826+0000 7f0a86000640 1 -- 172.21.15.107:0/3367481869 learned_addr learned my addr 172.21.15.107:0/3367481869 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:05:43.813 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.827+0000 7f0a85600640 1 --2- 172.21.15.107:0/3367481869 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56371f1d9e80 0x56371f1fa160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:05:43.813 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.827+0000 7f0a84c00640 1 --2- 172.21.15.107:0/3367481869 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56371f1da7a0 0x56371f1e29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:05:43.813 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.827+0000 7f0a86000640 1 -- 172.21.15.107:0/3367481869 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56371f1d9e80 msgr2=0x56371f1fa160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:05:43.813 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.827+0000 7f0a86000640 1 --2- 172.21.15.107:0/3367481869 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56371f1d9e80 0x56371f1fa160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:05:43.813 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.827+0000 7f0a86000640 1 -- 172.21.15.107:0/3367481869 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56371f1da7a0 msgr2=0x56371f1e29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:05:43.813 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.827+0000 7f0a86000640 1 --2- 172.21.15.107:0/3367481869 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56371f1da7a0 0x56371f1e29b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:05:43.813 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.827+0000 7f0a86000640 1 -- 172.21.15.107:0/3367481869 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56371f1d9640 con 0x56371f1e3570 2024-03-08T21:05:43.813 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.827+0000 7f0a85600640 1 --2- 172.21.15.107:0/3367481869 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56371f1d9e80 0x56371f1fa160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T21:05:43.814 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.827+0000 7f0a86000640 1 --2- 172.21.15.107:0/3367481869 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56371f1e3570 0x56371f1e2ef0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f0a7800ed80 tx=0x7f0a78011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e345cf246b3b8b4 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:05:43.814 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.828+0000 7f0a71c00640 1 -- 172.21.15.107:0/3367481869 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a78026430 con 0x56371f1e3570 2024-03-08T21:05:43.814 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.828+0000 7f0a71c00640 1 -- 172.21.15.107:0/3367481869 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f0a7801d8e0 con 0x56371f1e3570 2024-03-08T21:05:43.814 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.828+0000 7f0a71c00640 1 -- 172.21.15.107:0/3367481869 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a7801c8c0 con 0x56371f1e3570 2024-03-08T21:05:43.814 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.828+0000 7f0a8ac25740 1 -- 172.21.15.107:0/3367481869 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56371f1e3570 msgr2=0x56371f1e2ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:05:43.814 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.828+0000 7f0a8ac25740 1 --2- 172.21.15.107:0/3367481869 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56371f1e3570 0x56371f1e2ef0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f0a7800ed80 tx=0x7f0a78011000 comp rx=0 tx=0).stop 2024-03-08T21:05:43.814 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.829+0000 7f0a8ac25740 1 -- 172.21.15.107:0/3367481869 shutdown_connections 2024-03-08T21:05:43.814 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.829+0000 7f0a8ac25740 1 --2- 172.21.15.107:0/3367481869 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56371f1e3570 0x56371f1e2ef0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:05:43.814 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.829+0000 7f0a8ac25740 1 --2- 172.21.15.107:0/3367481869 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56371f1da7a0 0x56371f1e29b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:05:43.814 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.829+0000 7f0a8ac25740 1 --2- 172.21.15.107:0/3367481869 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56371f1d9e80 0x56371f1fa160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:05:43.814 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.829+0000 7f0a8ac25740 1 -- 172.21.15.107:0/3367481869 >> 172.21.15.107:0/3367481869 conn(0x56371f1d47a0 msgr2=0x56371f1e0050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:05:43.814 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.829+0000 7f0a8ac25740 1 -- 172.21.15.107:0/3367481869 shutdown_connections 2024-03-08T21:05:43.815 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.829+0000 7f0a8ac25740 1 -- 172.21.15.107:0/3367481869 wait complete. 2024-03-08T21:05:43.815 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.829+0000 7f0a8ac25740 1 -- start start 2024-03-08T21:05:43.815 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.829+0000 7f0a8ac25740 1 Processor -- start 2024-03-08T21:05:43.815 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.830+0000 7f0a8ac25740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56371f1d9e80 0x56371f1e90f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:05:43.815 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.830+0000 7f0a8ac25740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56371f1da7a0 0x56371f1e9630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:05:43.815 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.830+0000 7f0a8ac25740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56371f1e3570 0x56371f1e9b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:05:43.815 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.830+0000 7f0a8ac25740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a801afee0 con 0x56371f1da7a0 2024-03-08T21:05:43.815 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.830+0000 7f0a8ac25740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a801af880 con 0x56371f1e3570 2024-03-08T21:05:43.815 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.830+0000 7f0a8ac25740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a801ace60 con 0x56371f1d9e80 2024-03-08T21:05:43.815 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.830+0000 7f0a8ac25740 10 client.0 _ll_register_callbacks cb 0x7f0a6c00b260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T21:05:43.815 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.830+0000 7f0a84c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56371f1da7a0 0x56371f1e9630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:05:43.815 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.830+0000 7f0a84c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56371f1da7a0 0x56371f1e9630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:35958/0 (socket says 172.21.15.107:35958) 2024-03-08T21:05:43.815 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.830+0000 7f0a84c00640 1 -- 172.21.15.107:0/740584880 learned_addr learned my addr 172.21.15.107:0/740584880 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:05:43.816 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.830+0000 7f0a85600640 1 --2- 172.21.15.107:0/740584880 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56371f1d9e80 0x56371f1e90f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:05:43.816 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.830+0000 7f0a86000640 1 --2- 172.21.15.107:0/740584880 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56371f1e3570 0x56371f1e9b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:05:43.816 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.831+0000 7f0a84c00640 1 -- 172.21.15.107:0/740584880 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56371f1d9e80 msgr2=0x56371f1e90f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:05:43.816 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.831+0000 7f0a84c00640 1 --2- 172.21.15.107:0/740584880 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56371f1d9e80 0x56371f1e90f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:05:43.816 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.831+0000 7f0a84c00640 1 -- 172.21.15.107:0/740584880 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56371f1e3570 msgr2=0x56371f1e9b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:05:43.816 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.831+0000 7f0a84c00640 1 --2- 172.21.15.107:0/740584880 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56371f1e3570 0x56371f1e9b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:05:43.816 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.831+0000 7f0a84c00640 1 -- 172.21.15.107:0/740584880 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56371f2499f0 con 0x56371f1da7a0 2024-03-08T21:05:43.816 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.831+0000 7f0a85600640 1 --2- 172.21.15.107:0/740584880 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56371f1d9e80 0x56371f1e90f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T21:05:43.816 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.831+0000 7f0a86000640 1 --2- 172.21.15.107:0/740584880 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56371f1e3570 0x56371f1e9b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T21:05:43.816 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.831+0000 7f0a84c00640 1 --2- 172.21.15.107:0/740584880 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56371f1da7a0 0x56371f1e9630 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x56371f1d5930 tx=0x7f0a6c0049c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcf405cf7d87fa6 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:05:43.816 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.831+0000 7f0a66a00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-03-08T21:05:43.816 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.831+0000 7f0a66a00640 1 -- 172.21.15.107:0/740584880 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a6c016040 con 0x56371f1da7a0 2024-03-08T21:05:43.817 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.831+0000 7f0a66a00640 1 -- 172.21.15.107:0/740584880 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f0a6c0263b0 con 0x56371f1da7a0 2024-03-08T21:05:43.817 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.831+0000 7f0a8ac25740 1 -- 172.21.15.107:0/740584880 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0a7801d8e0 con 0x56371f1da7a0 2024-03-08T21:05:43.817 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.832+0000 7f0a66a00640 1 -- 172.21.15.107:0/740584880 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0a6c038050 con 0x56371f1da7a0 2024-03-08T21:05:43.817 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.832+0000 7f0a8ac25740 10 client.15897 fetch_fsmap learned FSMap version 34 2024-03-08T21:05:43.817 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.832+0000 7f0a8ac25740 1 -- 172.21.15.107:0/740584880 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x56371f24ad60 con 0x56371f1da7a0 2024-03-08T21:05:43.817 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.832+0000 7f0a66a00640 1 -- 172.21.15.107:0/740584880 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f0a7801d8e0 con 0x56371f1da7a0 2024-03-08T21:05:43.817 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.832+0000 7f0a66a00640 1 -- 172.21.15.107:0/740584880 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 34) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f0a6c010af0 con 0x56371f1da7a0 2024-03-08T21:05:43.817 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.832+0000 7f0a8ac25740 10 client.15897 fetch_fsmap finished waiting for FSMap version 34 2024-03-08T21:05:43.817 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.832+0000 7f0a8ac25740 10 client.15897 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-08T21:05:43.817 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.832+0000 7f0a8ac25740 20 client.15897 populate_metadata read hostname 'smithi107' 2024-03-08T21:05:43.817 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.832+0000 7f0a8ac25740 1 --2- 172.21.15.107:0/740584880 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x56371f24b1b0 0x56371f24d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:05:43.817 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.832+0000 7f0a8ac25740 4 client.15897 mds_command: new command op to 34475 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-08T21:05:43.817 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.832+0000 7f0a8ac25740 1 -- 172.21.15.107:0/740584880 --> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x56371f144c60 con 0x56371f24b1b0 2024-03-08T21:05:43.818 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.833+0000 7f0a85600640 1 --2- 172.21.15.107:0/740584880 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x56371f24b1b0 0x56371f24d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:05:43.818 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.834+0000 7f0a85600640 1 --2- 172.21.15.107:0/740584880 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x56371f24b1b0 0x56371f24d5d0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=337fee54d9f11bb2 server_cookie=3234a0eff1920c2c in_seq=0 out_seq=0 2024-03-08T21:05:43.818 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.834+0000 7f0a66a00640 10 client.15897 ms_handle_connect on v2:172.21.15.149:6838/55290596 2024-03-08T21:05:43.826 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.845+0000 7f0a66a00640 1 -- 172.21.15.107:0/740584880 <== mds.0 v2:172.21.15.149:6838/55290596 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x56371f144c60 con 0x56371f24b1b0 2024-03-08T21:05:43.826 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.845+0000 7f0a66a00640 10 client.15897 handle_command_reply: tid=0 2024-03-08T21:05:43.826 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.845+0000 7f0a8ac25740 1 client.15897 shutdown 2024-03-08T21:05:43.826 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.846+0000 7f0a8ac25740 1 -- 172.21.15.107:0/740584880 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56371f1da7a0 msgr2=0x56371f1e9630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:05:43.827 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.846+0000 7f0a8ac25740 1 --2- 172.21.15.107:0/740584880 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56371f1da7a0 0x56371f1e9630 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x56371f1d5930 tx=0x7f0a6c0049c0 comp rx=0 tx=0).stop 2024-03-08T21:05:43.827 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.846+0000 7f0a8ac25740 1 -- 172.21.15.107:0/740584880 shutdown_connections 2024-03-08T21:05:43.827 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.846+0000 7f0a8ac25740 1 --2- 172.21.15.107:0/740584880 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x56371f24b1b0 0x56371f24d5d0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:05:43.827 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.846+0000 7f0a8ac25740 1 --2- 172.21.15.107:0/740584880 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56371f1e3570 0x56371f1e9b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:05:43.827 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.846+0000 7f0a8ac25740 1 --2- 172.21.15.107:0/740584880 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56371f1da7a0 0x56371f1e9630 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:05:43.827 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.846+0000 7f0a8ac25740 1 --2- 172.21.15.107:0/740584880 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56371f1d9e80 0x56371f1e90f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:05:43.827 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.846+0000 7f0a8ac25740 1 -- 172.21.15.107:0/740584880 >> 172.21.15.107:0/740584880 conn(0x56371f1d47a0 msgr2=0x56371f1d5bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:05:43.827 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.846+0000 7f0a66a00640 0 client.15897 ms_handle_reset on v2:172.21.15.149:6838/55290596 2024-03-08T21:05:43.827 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.846+0000 7f0a8ac25740 1 -- 172.21.15.107:0/740584880 shutdown_connections 2024-03-08T21:05:43.827 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.846+0000 7f0a8ac25740 1 -- 172.21.15.107:0/740584880 wait complete. 2024-03-08T21:05:43.827 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.846+0000 7f0a8ac25740 20 client.15897 trim_cache size 0 max 16384 2024-03-08T21:05:43.828 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:{ 2024-03-08T21:05:43.828 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "message": "", 2024-03-08T21:05:43.828 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "return_code": 0 2024-03-08T21:05:43.828 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:} 2024-03-08T21:05:43.828 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.847+0000 7f0a86a00640 1 -- 172.21.15.107:0/58449224 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f0a680a2960 msgr2=0x7f0a680a4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:05:43.829 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.847+0000 7f0a86a00640 1 --2- 172.21.15.107:0/58449224 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f0a680a2960 0x7f0a680a4e20 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f0a78005fd0 tx=0x7f0a78005de0 comp rx=0 tx=0).stop 2024-03-08T21:05:43.829 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.847+0000 7f0a86a00640 1 -- 172.21.15.107:0/58449224 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0a80101340 msgr2=0x7f0a8019f700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:05:43.829 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.847+0000 7f0a86a00640 1 --2- 172.21.15.107:0/58449224 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0a80101340 0x7f0a8019f700 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f0a74004af0 tx=0x7f0a74035ce0 comp rx=0 tx=0).stop 2024-03-08T21:05:43.829 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.847+0000 7f0a86a00640 1 -- 172.21.15.107:0/58449224 shutdown_connections 2024-03-08T21:05:43.829 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.847+0000 7f0a86a00640 1 --2- 172.21.15.107:0/58449224 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f0a680a2960 0x7f0a680a4e20 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:05:43.829 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.847+0000 7f0a86a00640 1 --2- 172.21.15.107:0/58449224 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0a801998b0 0x7f0a80199d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:05:43.829 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.847+0000 7f0a86a00640 1 --2- 172.21.15.107:0/58449224 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0a80106410 0x7f0a8019fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:05:43.829 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.847+0000 7f0a86a00640 1 --2- 172.21.15.107:0/58449224 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0a80101340 0x7f0a8019f700 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:05:43.829 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.847+0000 7f0a86a00640 1 -- 172.21.15.107:0/58449224 >> 172.21.15.107:0/58449224 conn(0x7f0a800fcc40 msgr2=0x7f0a800fef60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:05:43.829 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.847+0000 7f0a86a00640 1 -- 172.21.15.107:0/58449224 shutdown_connections 2024-03-08T21:05:43.829 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:43.847+0000 7f0a86a00640 1 -- 172.21.15.107:0/58449224 wait complete. 2024-03-08T21:05:43.964 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:05:43 smithi107 ceph-mon[31381]: pgmap v161: 97 pgs: 97 active+clean; 3.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 4.2 KiB/s wr, 10 op/s 2024-03-08T21:05:44.012 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:05:43 smithi149 ceph-mon[35042]: pgmap v161: 97 pgs: 97 active+clean; 3.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 4.2 KiB/s wr, 10 op/s 2024-03-08T21:05:44.125 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:05:43 smithi204 ceph-mon[34935]: pgmap v161: 97 pgs: 97 active+clean; 3.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 4.2 KiB/s wr, 10 op/s 2024-03-08T21:05:45.375 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:05:45 smithi204 ceph-mon[34935]: pgmap v162: 97 pgs: 97 active+clean; 3.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 4.2 KiB/s wr, 14 op/s 2024-03-08T21:05:45.464 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:05:45 smithi107 ceph-mon[31381]: pgmap v162: 97 pgs: 97 active+clean; 3.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 4.2 KiB/s wr, 14 op/s 2024-03-08T21:05:45.512 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:05:45 smithi149 ceph-mon[35042]: pgmap v162: 97 pgs: 97 active+clean; 3.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 4.2 KiB/s wr, 14 op/s 2024-03-08T21:05:47.964 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:05:47 smithi107 ceph-mon[31381]: pgmap v163: 97 pgs: 97 active+clean; 3.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-08T21:05:48.012 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:05:47 smithi149 ceph-mon[35042]: pgmap v163: 97 pgs: 97 active+clean; 3.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-08T21:05:48.125 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:05:47 smithi204 ceph-mon[34935]: pgmap v163: 97 pgs: 97 active+clean; 3.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-08T21:05:49.964 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:05:49 smithi107 ceph-mon[31381]: pgmap v164: 97 pgs: 97 active+clean; 3.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-08T21:05:49.964 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:05:49 smithi107 ceph-mon[31381]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T21:05:50.012 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:05:49 smithi149 ceph-mon[35042]: pgmap v164: 97 pgs: 97 active+clean; 3.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-08T21:05:50.012 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:05:49 smithi149 ceph-mon[35042]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T21:05:50.125 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:05:49 smithi204 ceph-mon[34935]: pgmap v164: 97 pgs: 97 active+clean; 3.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-08T21:05:50.125 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:05:49 smithi204 ceph-mon[34935]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T21:05:50.964 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:05:50 smithi107 ceph-mon[31381]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T21:05:50.964 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:05:50 smithi107 ceph-mon[31381]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T21:05:50.964 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:05:50 smithi107 ceph-mon[31381]: from='mgr.15375 ' entity='mgr.x' 2024-03-08T21:05:51.012 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:05:50 smithi149 ceph-mon[35042]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T21:05:51.012 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:05:50 smithi149 ceph-mon[35042]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T21:05:51.012 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:05:50 smithi149 ceph-mon[35042]: from='mgr.15375 ' entity='mgr.x' 2024-03-08T21:05:51.125 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:05:50 smithi204 ceph-mon[34935]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T21:05:51.125 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:05:50 smithi204 ceph-mon[34935]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T21:05:51.125 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:05:50 smithi204 ceph-mon[34935]: from='mgr.15375 ' entity='mgr.x' 2024-03-08T21:05:51.964 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:05:51 smithi107 ceph-mon[31381]: pgmap v165: 97 pgs: 97 active+clean; 3.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:05:52.012 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:05:51 smithi149 ceph-mon[35042]: pgmap v165: 97 pgs: 97 active+clean; 3.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:05:52.125 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:05:51 smithi204 ceph-mon[34935]: pgmap v165: 97 pgs: 97 active+clean; 3.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:05:53.964 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:05:53 smithi107 ceph-mon[31381]: pgmap v166: 97 pgs: 97 active+clean; 3.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:05:54.012 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:05:53 smithi149 ceph-mon[35042]: pgmap v166: 97 pgs: 97 active+clean; 3.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:05:54.125 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:05:53 smithi204 ceph-mon[34935]: pgmap v166: 97 pgs: 97 active+clean; 3.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:05:54.964 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:05:54 smithi107 ceph-mon[31381]: pgmap v167: 97 pgs: 97 active+clean; 3.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-08T21:05:55.012 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:05:54 smithi149 ceph-mon[35042]: pgmap v167: 97 pgs: 97 active+clean; 3.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-08T21:05:55.125 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:05:54 smithi204 ceph-mon[34935]: pgmap v167: 97 pgs: 97 active+clean; 3.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-08T21:05:56.981 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.002+0000 7fe4e4a00640 1 -- 172.21.15.107:0/2742303466 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe4e010b6c0 msgr2=0x7fe4e010db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:05:56.982 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.002+0000 7fe4e4a00640 1 --2- 172.21.15.107:0/2742303466 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe4e010b6c0 0x7fe4e010db60 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fe4d8009ca0 tx=0x7fe4d802c190 comp rx=0 tx=0).stop 2024-03-08T21:05:56.982 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.002+0000 7fe4e4a00640 1 -- 172.21.15.107:0/2742303466 shutdown_connections 2024-03-08T21:05:56.982 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.002+0000 7fe4e4a00640 1 --2- 172.21.15.107:0/2742303466 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe4e010b6c0 0x7fe4e010db60 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:05:56.982 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.002+0000 7fe4e4a00640 1 --2- 172.21.15.107:0/2742303466 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe4e0102bc0 0x7fe4e010b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:05:56.982 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.002+0000 7fe4e4a00640 1 --2- 172.21.15.107:0/2742303466 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe4e01022a0 0x7fe4e0102680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:05:56.982 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.002+0000 7fe4e4a00640 1 -- 172.21.15.107:0/2742303466 >> 172.21.15.107:0/2742303466 conn(0x7fe4e00fbb00 msgr2=0x7fe4e00fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:05:56.982 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.002+0000 7fe4e4a00640 1 -- 172.21.15.107:0/2742303466 shutdown_connections 2024-03-08T21:05:56.982 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.002+0000 7fe4e4a00640 1 -- 172.21.15.107:0/2742303466 wait complete. 2024-03-08T21:05:56.984 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.003+0000 7fe4e4a00640 1 Processor -- start 2024-03-08T21:05:56.985 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.003+0000 7fe4e4a00640 1 -- start start 2024-03-08T21:05:56.985 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.004+0000 7fe4e4a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe4e01022a0 0x7fe4e019f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:05:56.985 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.004+0000 7fe4e4a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe4e0102bc0 0x7fe4e019fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:05:56.985 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.004+0000 7fe4e4a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe4e010b6c0 0x7fe4e01998d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:05:56.985 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.004+0000 7fe4e4a00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4e0111350 con 0x7fe4e0102bc0 2024-03-08T21:05:56.985 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.004+0000 7fe4e4a00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4e01111d0 con 0x7fe4e010b6c0 2024-03-08T21:05:56.985 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.004+0000 7fe4e4a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4e01114d0 con 0x7fe4e01022a0 2024-03-08T21:05:56.985 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.004+0000 7fe4dde00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe4e0102bc0 0x7fe4e019fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:05:56.985 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.004+0000 7fe4dde00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe4e0102bc0 0x7fe4e019fc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:45882/0 (socket says 172.21.15.107:45882) 2024-03-08T21:05:56.985 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.004+0000 7fe4dde00640 1 -- 172.21.15.107:0/3542073519 learned_addr learned my addr 172.21.15.107:0/3542073519 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:05:56.985 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.004+0000 7fe4de800640 1 --2- 172.21.15.107:0/3542073519 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe4e01022a0 0x7fe4e019f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:05:56.985 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.004+0000 7fe4df200640 1 --2- 172.21.15.107:0/3542073519 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe4e010b6c0 0x7fe4e01998d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:05:56.985 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.004+0000 7fe4dde00640 1 -- 172.21.15.107:0/3542073519 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe4e01022a0 msgr2=0x7fe4e019f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:05:56.986 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.004+0000 7fe4dde00640 1 --2- 172.21.15.107:0/3542073519 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe4e01022a0 0x7fe4e019f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:05:56.986 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.004+0000 7fe4dde00640 1 -- 172.21.15.107:0/3542073519 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe4e010b6c0 msgr2=0x7fe4e01998d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:05:56.986 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.004+0000 7fe4dde00640 1 --2- 172.21.15.107:0/3542073519 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe4e010b6c0 0x7fe4e01998d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:05:56.986 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.004+0000 7fe4dde00640 1 -- 172.21.15.107:0/3542073519 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4e019a050 con 0x7fe4e0102bc0 2024-03-08T21:05:56.986 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.005+0000 7fe4dde00640 1 --2- 172.21.15.107:0/3542073519 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe4e0102bc0 0x7fe4e019fc60 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fe4cc00d950 tx=0x7fe4cc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:05:56.986 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.005+0000 7fe4d7400640 1 -- 172.21.15.107:0/3542073519 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4cc017030 con 0x7fe4e0102bc0 2024-03-08T21:05:56.986 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.005+0000 7fe4d7400640 1 -- 172.21.15.107:0/3542073519 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fe4cc004020 con 0x7fe4e0102bc0 2024-03-08T21:05:56.986 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.005+0000 7fe4e4a00640 1 -- 172.21.15.107:0/3542073519 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4e019a340 con 0x7fe4e0102bc0 2024-03-08T21:05:56.986 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.005+0000 7fe4d7400640 1 -- 172.21.15.107:0/3542073519 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4cc0055e0 con 0x7fe4e0102bc0 2024-03-08T21:05:56.986 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.005+0000 7fe4e4a00640 1 -- 172.21.15.107:0/3542073519 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4e01a65e0 con 0x7fe4e0102bc0 2024-03-08T21:05:56.988 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.009+0000 7fe4d7400640 1 -- 172.21.15.107:0/3542073519 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) v1 ==== 137422+0+0 (secure 0 0 0) 0x7fe4cc0041d0 con 0x7fe4e0102bc0 2024-03-08T21:05:56.988 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.009+0000 7fe4e81ef740 1 Processor -- start 2024-03-08T21:05:56.988 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.009+0000 7fe4e81ef740 1 -- start start 2024-03-08T21:05:56.997 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.010+0000 7fe4e81ef740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d38e64fe80 0x55d38e650260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:05:56.997 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.010+0000 7fe4e81ef740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d38e6507a0 0x55d38e659030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:05:56.997 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.010+0000 7fe4e81ef740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d38e659570 0x55d38e65b960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:05:56.998 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.010+0000 7fe4e81ef740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d38e5b33d0 con 0x55d38e659570 2024-03-08T21:05:56.998 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.010+0000 7fe4e81ef740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d38e55c6b0 con 0x55d38e64fe80 2024-03-08T21:05:56.998 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.010+0000 7fe4e81ef740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d38e5af470 con 0x55d38e6507a0 2024-03-08T21:05:56.998 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.010+0000 7fe4dde00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d38e659570 0x55d38e65b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:05:56.998 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.010+0000 7fe4dde00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d38e659570 0x55d38e65b960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:45894/0 (socket says 172.21.15.107:45894) 2024-03-08T21:05:56.998 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.010+0000 7fe4dde00640 1 -- 172.21.15.107:0/2053398286 learned_addr learned my addr 172.21.15.107:0/2053398286 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:05:56.998 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.010+0000 7fe4df200640 1 --2- 172.21.15.107:0/2053398286 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d38e64fe80 0x55d38e650260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:05:56.998 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.010+0000 7fe4de800640 1 --2- 172.21.15.107:0/2053398286 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d38e6507a0 0x55d38e659030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:05:56.998 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.010+0000 7fe4d7400640 1 --2- 172.21.15.107:0/3542073519 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7fe4c40a29e0 0x7fe4c40a4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:05:56.998 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.011+0000 7fe4dde00640 1 -- 172.21.15.107:0/2053398286 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d38e6507a0 msgr2=0x55d38e659030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:05:56.998 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.011+0000 7fe4dde00640 1 --2- 172.21.15.107:0/2053398286 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d38e6507a0 0x55d38e659030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:05:56.998 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.011+0000 7fe4dde00640 1 -- 172.21.15.107:0/2053398286 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d38e64fe80 msgr2=0x55d38e650260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:05:56.999 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.011+0000 7fe4dde00640 1 --2- 172.21.15.107:0/2053398286 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d38e64fe80 0x55d38e650260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:05:56.999 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.011+0000 7fe4dde00640 1 -- 172.21.15.107:0/2053398286 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d38e582b50 con 0x55d38e659570 2024-03-08T21:05:56.999 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.011+0000 7fe4de800640 1 --2- 172.21.15.107:0/2053398286 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d38e6507a0 0x55d38e659030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T21:05:56.999 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.011+0000 7fe4d7400640 1 -- 172.21.15.107:0/3542073519 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fe4cc0ce7c0 con 0x7fe4e0102bc0 2024-03-08T21:05:56.999 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.011+0000 7fe4df200640 1 --2- 172.21.15.107:0/3542073519 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7fe4c40a29e0 0x7fe4c40a4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:05:56.999 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.011+0000 7fe4dde00640 1 --2- 172.21.15.107:0/2053398286 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d38e659570 0x55d38e65b960 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fe4cc0cbde0 tx=0x7fe4cc021000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bc41e3aa229aea9 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:05:56.999 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.012+0000 7fe4cb400640 1 -- 172.21.15.107:0/2053398286 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4cc09ab80 con 0x55d38e659570 2024-03-08T21:05:56.999 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.012+0000 7fe4cb400640 1 -- 172.21.15.107:0/2053398286 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fe4cc04f070 con 0x55d38e659570 2024-03-08T21:05:56.999 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.012+0000 7fe4cb400640 1 -- 172.21.15.107:0/2053398286 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4cc09a710 con 0x55d38e659570 2024-03-08T21:05:56.999 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.012+0000 7fe4e81ef740 1 -- 172.21.15.107:0/2053398286 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d38e659570 msgr2=0x55d38e65b960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:05:56.999 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.012+0000 7fe4df200640 1 --2- 172.21.15.107:0/3542073519 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7fe4c40a29e0 0x7fe4c40a4ea0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fe4d8006010 tx=0x7fe4d802c190 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:05:56.999 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.012+0000 7fe4e81ef740 1 --2- 172.21.15.107:0/2053398286 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d38e659570 0x55d38e65b960 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fe4cc0cbde0 tx=0x7fe4cc021000 comp rx=0 tx=0).stop 2024-03-08T21:05:56.999 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.013+0000 7fe4e81ef740 1 -- 172.21.15.107:0/2053398286 shutdown_connections 2024-03-08T21:05:57.000 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.013+0000 7fe4e81ef740 1 --2- 172.21.15.107:0/2053398286 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d38e659570 0x55d38e65b960 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:05:57.000 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.013+0000 7fe4e81ef740 1 --2- 172.21.15.107:0/2053398286 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d38e6507a0 0x55d38e659030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:05:57.000 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.013+0000 7fe4e81ef740 1 --2- 172.21.15.107:0/2053398286 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d38e64fe80 0x55d38e650260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:05:57.000 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.013+0000 7fe4e81ef740 1 -- 172.21.15.107:0/2053398286 >> 172.21.15.107:0/2053398286 conn(0x55d38e64a7a0 msgr2=0x55d38e64ab80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:05:57.000 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.013+0000 7fe4e81ef740 1 -- 172.21.15.107:0/2053398286 shutdown_connections 2024-03-08T21:05:57.000 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.013+0000 7fe4e81ef740 1 -- 172.21.15.107:0/2053398286 wait complete. 2024-03-08T21:05:57.000 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.013+0000 7fe4e81ef740 1 -- start start 2024-03-08T21:05:57.000 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.013+0000 7fe4e81ef740 1 Processor -- start 2024-03-08T21:05:57.000 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.014+0000 7fe4e81ef740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d38e64fe80 0x55d38e658e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:05:57.001 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.014+0000 7fe4e81ef740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d38e6507a0 0x55d38e6561f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:05:57.001 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.014+0000 7fe4e81ef740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d38e659570 0x55d38e656730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:05:57.001 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.014+0000 7fe4e81ef740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d38e65c140 con 0x55d38e659570 2024-03-08T21:05:57.001 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.014+0000 7fe4e81ef740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d38e56f9f0 con 0x55d38e6507a0 2024-03-08T21:05:57.001 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.014+0000 7fe4e81ef740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d38e65c2c0 con 0x55d38e64fe80 2024-03-08T21:05:57.001 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.014+0000 7fe4e81ef740 10 client.0 _ll_register_callbacks cb 0x55d38e2c7a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T21:05:57.001 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.014+0000 7fe4df200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d38e659570 0x55d38e656730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:05:57.001 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.014+0000 7fe4df200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d38e659570 0x55d38e656730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:45900/0 (socket says 172.21.15.107:45900) 2024-03-08T21:05:57.001 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.014+0000 7fe4df200640 1 -- 172.21.15.107:0/4148306556 learned_addr learned my addr 172.21.15.107:0/4148306556 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:05:57.001 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.014+0000 7fe4de800640 1 --2- 172.21.15.107:0/4148306556 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d38e64fe80 0x55d38e658e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:05:57.001 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.015+0000 7fe4dde00640 1 --2- 172.21.15.107:0/4148306556 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d38e6507a0 0x55d38e6561f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:05:57.001 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.015+0000 7fe4df200640 1 -- 172.21.15.107:0/4148306556 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d38e64fe80 msgr2=0x55d38e658e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:05:57.001 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.015+0000 7fe4df200640 1 --2- 172.21.15.107:0/4148306556 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d38e64fe80 0x55d38e658e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:05:57.002 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.015+0000 7fe4df200640 1 -- 172.21.15.107:0/4148306556 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d38e6507a0 msgr2=0x55d38e6561f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:05:57.002 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.015+0000 7fe4df200640 1 --2- 172.21.15.107:0/4148306556 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d38e6507a0 0x55d38e6561f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:05:57.002 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.015+0000 7fe4df200640 1 -- 172.21.15.107:0/4148306556 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d38e2c8210 con 0x55d38e659570 2024-03-08T21:05:57.002 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.015+0000 7fe4df200640 1 --2- 172.21.15.107:0/4148306556 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d38e659570 0x55d38e656730 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fe4d8005fe0 tx=0x7fe4d80095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e03c86c0ca772d4a server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:05:57.002 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.015+0000 7fe4bfe00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-03-08T21:05:57.002 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.016+0000 7fe4bfe00640 1 -- 172.21.15.107:0/4148306556 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4d8053060 con 0x55d38e659570 2024-03-08T21:05:57.002 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.016+0000 7fe4bfe00640 1 -- 172.21.15.107:0/4148306556 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fe4d8034850 con 0x55d38e659570 2024-03-08T21:05:57.003 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.016+0000 7fe4e81ef740 1 -- 172.21.15.107:0/4148306556 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d38e5c0910 con 0x55d38e659570 2024-03-08T21:05:57.003 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.016+0000 7fe4bfe00640 1 -- 172.21.15.107:0/4148306556 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe4d8039040 con 0x55d38e659570 2024-03-08T21:05:57.003 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.016+0000 7fe4e81ef740 10 client.15909 fetch_fsmap learned FSMap version 34 2024-03-08T21:05:57.003 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.016+0000 7fe4e81ef740 1 -- 172.21.15.107:0/4148306556 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55d38e394620 con 0x55d38e659570 2024-03-08T21:05:57.003 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.016+0000 7fe4bfe00640 1 -- 172.21.15.107:0/4148306556 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x55d38e5c0910 con 0x55d38e659570 2024-03-08T21:05:57.003 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.017+0000 7fe4bfe00640 1 -- 172.21.15.107:0/4148306556 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 34) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fe4d804eb30 con 0x55d38e659570 2024-03-08T21:05:57.003 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.017+0000 7fe4e81ef740 10 client.15909 fetch_fsmap finished waiting for FSMap version 34 2024-03-08T21:05:57.003 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.017+0000 7fe4e81ef740 10 client.15909 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-08T21:05:57.003 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.017+0000 7fe4e81ef740 20 client.15909 populate_metadata read hostname 'smithi107' 2024-03-08T21:05:57.003 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.017+0000 7fe4e81ef740 1 --2- 172.21.15.107:0/4148306556 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55d38e6692b0 0x55d38e66b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:05:57.003 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.017+0000 7fe4e81ef740 4 client.15909 mds_command: new command op to 34475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-08T21:05:57.003 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.017+0000 7fe4e81ef740 1 -- 172.21.15.107:0/4148306556 --> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d38e5bac60 con 0x55d38e6692b0 2024-03-08T21:05:57.003 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.017+0000 7fe4de800640 1 --2- 172.21.15.107:0/4148306556 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55d38e6692b0 0x55d38e66b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:05:57.005 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.018+0000 7fe4de800640 1 --2- 172.21.15.107:0/4148306556 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55d38e6692b0 0x55d38e66b6a0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=755bd378e9c3cc9a server_cookie=7cf4dc046b5faf38 in_seq=0 out_seq=0 2024-03-08T21:05:57.005 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.018+0000 7fe4bfe00640 10 client.15909 ms_handle_connect on v2:172.21.15.149:6838/55290596 2024-03-08T21:05:57.005 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.020+0000 7fe4bfe00640 1 -- 172.21.15.107:0/4148306556 <== mds.0 v2:172.21.15.149:6838/55290596 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55d38e5bac60 con 0x55d38e6692b0 2024-03-08T21:05:57.005 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.020+0000 7fe4bfe00640 10 client.15909 handle_command_reply: tid=0 2024-03-08T21:05:57.005 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.020+0000 7fe4e81ef740 1 client.15909 shutdown 2024-03-08T21:05:57.005 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.020+0000 7fe4e81ef740 1 -- 172.21.15.107:0/4148306556 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d38e659570 msgr2=0x55d38e656730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:05:57.005 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.020+0000 7fe4e81ef740 1 --2- 172.21.15.107:0/4148306556 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d38e659570 0x55d38e656730 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fe4d8005fe0 tx=0x7fe4d80095c0 comp rx=0 tx=0).stop 2024-03-08T21:05:57.005 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.020+0000 7fe4e81ef740 1 -- 172.21.15.107:0/4148306556 shutdown_connections 2024-03-08T21:05:57.006 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.020+0000 7fe4e81ef740 1 --2- 172.21.15.107:0/4148306556 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d38e659570 0x55d38e656730 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:05:57.006 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.020+0000 7fe4e81ef740 1 --2- 172.21.15.107:0/4148306556 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55d38e6692b0 0x55d38e66b6a0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:05:57.006 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.020+0000 7fe4e81ef740 1 --2- 172.21.15.107:0/4148306556 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d38e6507a0 0x55d38e6561f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:05:57.006 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.020+0000 7fe4e81ef740 1 --2- 172.21.15.107:0/4148306556 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d38e64fe80 0x55d38e658e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:05:57.006 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.020+0000 7fe4e81ef740 1 -- 172.21.15.107:0/4148306556 >> 172.21.15.107:0/4148306556 conn(0x55d38e64a7a0 msgr2=0x55d38e64c4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:05:57.006 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.020+0000 7fe4bfe00640 0 client.15909 ms_handle_reset on v2:172.21.15.149:6838/55290596 2024-03-08T21:05:57.006 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.021+0000 7fe4e81ef740 1 -- 172.21.15.107:0/4148306556 shutdown_connections 2024-03-08T21:05:57.006 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.021+0000 7fe4e81ef740 1 -- 172.21.15.107:0/4148306556 wait complete. 2024-03-08T21:05:57.006 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.021+0000 7fe4e81ef740 20 client.15909 trim_cache size 0 max 16384 2024-03-08T21:05:57.020 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.039+0000 7fe4e81ef740 1 Processor -- start 2024-03-08T21:05:57.020 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.040+0000 7fe4e81ef740 1 -- start start 2024-03-08T21:05:57.020 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.040+0000 7fe4e81ef740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d38e64fe80 0x55d38e649680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:05:57.020 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.040+0000 7fe4e81ef740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d38e6507a0 0x55d38e658920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:05:57.020 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.040+0000 7fe4e81ef740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d38e659570 0x55d38e658e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:05:57.020 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.040+0000 7fe4e81ef740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4e01aed70 con 0x55d38e64fe80 2024-03-08T21:05:57.021 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.040+0000 7fe4e81ef740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4e01b06f0 con 0x55d38e6507a0 2024-03-08T21:05:57.021 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.040+0000 7fe4e81ef740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4e01af3d0 con 0x55d38e659570 2024-03-08T21:05:57.021 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.040+0000 7fe4de800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d38e64fe80 0x55d38e649680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:05:57.021 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.040+0000 7fe4de800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d38e64fe80 0x55d38e649680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:45914/0 (socket says 172.21.15.107:45914) 2024-03-08T21:05:57.021 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.040+0000 7fe4de800640 1 -- 172.21.15.107:0/2479851382 learned_addr learned my addr 172.21.15.107:0/2479851382 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:05:57.021 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.041+0000 7fe4df200640 1 --2- 172.21.15.107:0/2479851382 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d38e659570 0x55d38e658e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:05:57.021 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.041+0000 7fe4dde00640 1 --2- 172.21.15.107:0/2479851382 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d38e6507a0 0x55d38e658920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:05:57.021 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.041+0000 7fe4de800640 1 -- 172.21.15.107:0/2479851382 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d38e659570 msgr2=0x55d38e658e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:05:57.021 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.041+0000 7fe4de800640 1 --2- 172.21.15.107:0/2479851382 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d38e659570 0x55d38e658e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:05:57.021 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.041+0000 7fe4de800640 1 -- 172.21.15.107:0/2479851382 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d38e6507a0 msgr2=0x55d38e658920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:05:57.021 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.041+0000 7fe4de800640 1 --2- 172.21.15.107:0/2479851382 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d38e6507a0 0x55d38e658920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:05:57.021 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.041+0000 7fe4de800640 1 -- 172.21.15.107:0/2479851382 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d38e670860 con 0x55d38e64fe80 2024-03-08T21:05:57.025 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.041+0000 7fe4de800640 1 --2- 172.21.15.107:0/2479851382 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d38e64fe80 0x55d38e649680 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fe4d000bd40 tx=0x7fe4d000b800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=326bc2a337c0a4c8 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:05:57.026 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.041+0000 7fe4caa00640 1 -- 172.21.15.107:0/2479851382 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4d001d420 con 0x55d38e64fe80 2024-03-08T21:05:57.026 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.042+0000 7fe4caa00640 1 -- 172.21.15.107:0/2479851382 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fe4d0005020 con 0x55d38e64fe80 2024-03-08T21:05:57.026 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.042+0000 7fe4caa00640 1 -- 172.21.15.107:0/2479851382 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4d0004bc0 con 0x55d38e64fe80 2024-03-08T21:05:57.026 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.042+0000 7fe4e81ef740 1 -- 172.21.15.107:0/2479851382 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d38e64fe80 msgr2=0x55d38e649680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:05:57.026 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.042+0000 7fe4e81ef740 1 --2- 172.21.15.107:0/2479851382 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d38e64fe80 0x55d38e649680 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fe4d000bd40 tx=0x7fe4d000b800 comp rx=0 tx=0).stop 2024-03-08T21:05:57.026 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.042+0000 7fe4e81ef740 1 -- 172.21.15.107:0/2479851382 shutdown_connections 2024-03-08T21:05:57.026 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.042+0000 7fe4e81ef740 1 --2- 172.21.15.107:0/2479851382 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d38e659570 0x55d38e658e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:05:57.026 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.042+0000 7fe4e81ef740 1 --2- 172.21.15.107:0/2479851382 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d38e6507a0 0x55d38e658920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:05:57.026 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.042+0000 7fe4e81ef740 1 --2- 172.21.15.107:0/2479851382 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d38e64fe80 0x55d38e649680 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:05:57.026 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.042+0000 7fe4e81ef740 1 -- 172.21.15.107:0/2479851382 >> 172.21.15.107:0/2479851382 conn(0x55d38e64a7a0 msgr2=0x55d38e648f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:05:57.026 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.043+0000 7fe4e81ef740 1 -- 172.21.15.107:0/2479851382 shutdown_connections 2024-03-08T21:05:57.027 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.043+0000 7fe4e81ef740 1 -- 172.21.15.107:0/2479851382 wait complete. 2024-03-08T21:05:57.027 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.043+0000 7fe4e81ef740 1 -- start start 2024-03-08T21:05:57.027 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.043+0000 7fe4e81ef740 1 Processor -- start 2024-03-08T21:05:57.027 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.044+0000 7fe4e81ef740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d38e64fe80 0x55d38e6609f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:05:57.027 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.044+0000 7fe4e81ef740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d38e6507a0 0x55d38e65ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:05:57.027 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.044+0000 7fe4e81ef740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d38e659570 0x55d38e65f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:05:57.027 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.044+0000 7fe4e81ef740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4e01b0090 con 0x55d38e6507a0 2024-03-08T21:05:57.027 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.044+0000 7fe4e81ef740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4e01afa30 con 0x55d38e659570 2024-03-08T21:05:57.027 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.044+0000 7fe4e81ef740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4e01ad010 con 0x55d38e64fe80 2024-03-08T21:05:57.027 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.044+0000 7fe4e81ef740 10 client.0 _ll_register_callbacks cb 0x7fe4cc0d4400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T21:05:57.027 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.044+0000 7fe4dde00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d38e6507a0 0x55d38e65ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:05:57.027 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.044+0000 7fe4dde00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d38e6507a0 0x55d38e65ed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:45922/0 (socket says 172.21.15.107:45922) 2024-03-08T21:05:57.028 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.044+0000 7fe4dde00640 1 -- 172.21.15.107:0/435233029 learned_addr learned my addr 172.21.15.107:0/435233029 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:05:57.028 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.045+0000 7fe4df200640 1 --2- 172.21.15.107:0/435233029 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d38e659570 0x55d38e65f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:05:57.028 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.045+0000 7fe4dde00640 1 -- 172.21.15.107:0/435233029 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d38e64fe80 msgr2=0x55d38e6609f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:05:57.028 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.045+0000 7fe4dde00640 1 --2- 172.21.15.107:0/435233029 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d38e64fe80 0x55d38e6609f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:05:57.028 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.045+0000 7fe4dde00640 1 -- 172.21.15.107:0/435233029 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d38e659570 msgr2=0x55d38e65f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:05:57.028 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.045+0000 7fe4dde00640 1 --2- 172.21.15.107:0/435233029 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d38e659570 0x55d38e65f4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:05:57.028 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.045+0000 7fe4dde00640 1 -- 172.21.15.107:0/435233029 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d38e6bf760 con 0x55d38e6507a0 2024-03-08T21:05:57.028 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.045+0000 7fe4dde00640 1 --2- 172.21.15.107:0/435233029 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d38e6507a0 0x55d38e65ed90 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fe4cc0a3020 tx=0x7fe4cc084930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=696e3e63e3cce94c server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:05:57.028 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.045+0000 7fe4bfe00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-03-08T21:05:57.028 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.045+0000 7fe4bfe00640 1 -- 172.21.15.107:0/435233029 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4cc09a8f0 con 0x55d38e6507a0 2024-03-08T21:05:57.028 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.045+0000 7fe4bfe00640 1 -- 172.21.15.107:0/435233029 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fe4cc096b80 con 0x55d38e6507a0 2024-03-08T21:05:57.028 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.045+0000 7fe4e81ef740 1 -- 172.21.15.107:0/435233029 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe4d0005020 con 0x55d38e6507a0 2024-03-08T21:05:57.028 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.045+0000 7fe4bfe00640 1 -- 172.21.15.107:0/435233029 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe4cc09a710 con 0x55d38e6507a0 2024-03-08T21:05:57.028 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.045+0000 7fe4e81ef740 10 client.15915 fetch_fsmap learned FSMap version 34 2024-03-08T21:05:57.029 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.046+0000 7fe4e81ef740 1 -- 172.21.15.107:0/435233029 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55d38e6c0900 con 0x55d38e6507a0 2024-03-08T21:05:57.029 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.046+0000 7fe4bfe00640 1 -- 172.21.15.107:0/435233029 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fe4d0005020 con 0x55d38e6507a0 2024-03-08T21:05:57.029 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.046+0000 7fe4bfe00640 1 -- 172.21.15.107:0/435233029 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 34) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fe4cc097420 con 0x55d38e6507a0 2024-03-08T21:05:57.029 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.046+0000 7fe4e81ef740 10 client.15915 fetch_fsmap finished waiting for FSMap version 34 2024-03-08T21:05:57.029 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.046+0000 7fe4e81ef740 10 client.15915 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-08T21:05:57.029 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.046+0000 7fe4e81ef740 20 client.15915 populate_metadata read hostname 'smithi107' 2024-03-08T21:05:57.029 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.046+0000 7fe4e81ef740 1 --2- 172.21.15.107:0/435233029 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55d38e6c0ca0 0x55d38e6c3090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:05:57.029 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.046+0000 7fe4e81ef740 4 client.15915 mds_command: new command op to 34475 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-08T21:05:57.029 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.046+0000 7fe4e81ef740 1 -- 172.21.15.107:0/435233029 --> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55d38e5395d0 con 0x55d38e6c0ca0 2024-03-08T21:05:57.030 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.047+0000 7fe4de800640 1 --2- 172.21.15.107:0/435233029 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55d38e6c0ca0 0x55d38e6c3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:05:57.030 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.048+0000 7fe4de800640 1 --2- 172.21.15.107:0/435233029 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55d38e6c0ca0 0x55d38e6c3090 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9dcf2f8da33bd677 server_cookie=36cf29c2da4196e3 in_seq=0 out_seq=0 2024-03-08T21:05:57.030 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.048+0000 7fe4bfe00640 10 client.15915 ms_handle_connect on v2:172.21.15.149:6838/55290596 2024-03-08T21:05:57.037 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.057+0000 7fe4bfe00640 1 -- 172.21.15.107:0/435233029 <== mds.0 v2:172.21.15.149:6838/55290596 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55d38e5395d0 con 0x55d38e6c0ca0 2024-03-08T21:05:57.037 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.057+0000 7fe4bfe00640 10 client.15915 handle_command_reply: tid=0 2024-03-08T21:05:57.037 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.057+0000 7fe4e81ef740 1 client.15915 shutdown 2024-03-08T21:05:57.037 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.057+0000 7fe4e81ef740 1 -- 172.21.15.107:0/435233029 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d38e6507a0 msgr2=0x55d38e65ed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:05:57.037 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.057+0000 7fe4e81ef740 1 --2- 172.21.15.107:0/435233029 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d38e6507a0 0x55d38e65ed90 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fe4cc0a3020 tx=0x7fe4cc084930 comp rx=0 tx=0).stop 2024-03-08T21:05:57.037 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.057+0000 7fe4e81ef740 1 -- 172.21.15.107:0/435233029 shutdown_connections 2024-03-08T21:05:57.037 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.057+0000 7fe4e81ef740 1 --2- 172.21.15.107:0/435233029 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55d38e6c0ca0 0x55d38e6c3090 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:05:57.037 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.057+0000 7fe4e81ef740 1 --2- 172.21.15.107:0/435233029 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55d38e659570 0x55d38e65f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:05:57.037 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.057+0000 7fe4e81ef740 1 --2- 172.21.15.107:0/435233029 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d38e6507a0 0x55d38e65ed90 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:05:57.038 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.057+0000 7fe4bfe00640 0 client.15915 ms_handle_reset on v2:172.21.15.149:6838/55290596 2024-03-08T21:05:57.038 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.057+0000 7fe4e81ef740 1 --2- 172.21.15.107:0/435233029 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55d38e64fe80 0x55d38e6609f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:05:57.038 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.057+0000 7fe4e81ef740 1 -- 172.21.15.107:0/435233029 >> 172.21.15.107:0/435233029 conn(0x55d38e64a7a0 msgr2=0x55d38e649800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:05:57.038 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.057+0000 7fe4e81ef740 1 -- 172.21.15.107:0/435233029 shutdown_connections 2024-03-08T21:05:57.038 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.058+0000 7fe4e81ef740 1 -- 172.21.15.107:0/435233029 wait complete. 2024-03-08T21:05:57.038 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.058+0000 7fe4e81ef740 20 client.15915 trim_cache size 0 max 16384 2024-03-08T21:05:57.039 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:{ 2024-03-08T21:05:57.039 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "message": "", 2024-03-08T21:05:57.039 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "return_code": 0 2024-03-08T21:05:57.039 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:} 2024-03-08T21:05:57.040 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.058+0000 7fe4e4a00640 1 -- 172.21.15.107:0/3542073519 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7fe4c40a29e0 msgr2=0x7fe4c40a4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:05:57.040 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.058+0000 7fe4e4a00640 1 --2- 172.21.15.107:0/3542073519 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7fe4c40a29e0 0x7fe4c40a4ea0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fe4d8006010 tx=0x7fe4d802c190 comp rx=0 tx=0).stop 2024-03-08T21:05:57.040 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.058+0000 7fe4e4a00640 1 -- 172.21.15.107:0/3542073519 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe4e0102bc0 msgr2=0x7fe4e019fc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:05:57.040 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.058+0000 7fe4e4a00640 1 --2- 172.21.15.107:0/3542073519 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe4e0102bc0 0x7fe4e019fc60 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fe4cc00d950 tx=0x7fe4cc00de10 comp rx=0 tx=0).stop 2024-03-08T21:05:57.040 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.059+0000 7fe4e4a00640 1 -- 172.21.15.107:0/3542073519 shutdown_connections 2024-03-08T21:05:57.040 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.059+0000 7fe4e4a00640 1 --2- 172.21.15.107:0/3542073519 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7fe4c40a29e0 0x7fe4c40a4ea0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:05:57.040 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.059+0000 7fe4e4a00640 1 --2- 172.21.15.107:0/3542073519 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe4e010b6c0 0x7fe4e01998d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:05:57.040 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.059+0000 7fe4e4a00640 1 --2- 172.21.15.107:0/3542073519 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe4e0102bc0 0x7fe4e019fc60 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:05:57.040 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.059+0000 7fe4e4a00640 1 --2- 172.21.15.107:0/3542073519 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe4e01022a0 0x7fe4e019f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:05:57.040 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.059+0000 7fe4e4a00640 1 -- 172.21.15.107:0/3542073519 >> 172.21.15.107:0/3542073519 conn(0x7fe4e00fbb00 msgr2=0x7fe4e0100f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:05:57.040 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.059+0000 7fe4e4a00640 1 -- 172.21.15.107:0/3542073519 shutdown_connections 2024-03-08T21:05:57.041 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:05:57.059+0000 7fe4e4a00640 1 -- 172.21.15.107:0/3542073519 wait complete. 2024-03-08T21:05:57.964 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:05:57 smithi107 ceph-mon[31381]: pgmap v168: 97 pgs: 97 active+clean; 3.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:05:58.012 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:05:57 smithi149 ceph-mon[35042]: pgmap v168: 97 pgs: 97 active+clean; 3.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:05:58.125 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:05:57 smithi204 ceph-mon[34935]: pgmap v168: 97 pgs: 97 active+clean; 3.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:05:59.964 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:05:59 smithi107 ceph-mon[31381]: pgmap v169: 97 pgs: 97 active+clean; 3.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 511 B/s wr, 13 op/s 2024-03-08T21:06:00.012 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:05:59 smithi149 ceph-mon[35042]: pgmap v169: 97 pgs: 97 active+clean; 3.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 511 B/s wr, 13 op/s 2024-03-08T21:06:00.125 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:05:59 smithi204 ceph-mon[34935]: pgmap v169: 97 pgs: 97 active+clean; 3.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 511 B/s wr, 13 op/s 2024-03-08T21:06:01.964 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:06:01 smithi107 ceph-mon[31381]: pgmap v170: 97 pgs: 97 active+clean; 3.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:06:02.012 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:06:01 smithi149 ceph-mon[35042]: pgmap v170: 97 pgs: 97 active+clean; 3.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:06:02.125 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:06:01 smithi204 ceph-mon[34935]: pgmap v170: 97 pgs: 97 active+clean; 3.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:06:02.964 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:06:02 smithi107 ceph-mon[31381]: pgmap v171: 97 pgs: 97 active+clean; 3.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:06:03.012 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:06:02 smithi149 ceph-mon[35042]: pgmap v171: 97 pgs: 97 active+clean; 3.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:06:03.027 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:06:02 smithi204 ceph-mon[34935]: pgmap v171: 97 pgs: 97 active+clean; 3.3 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:06:05.964 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:06:05 smithi107 ceph-mon[31381]: pgmap v172: 97 pgs: 97 active+clean; 4.0 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 76 KiB/s wr, 27 op/s 2024-03-08T21:06:06.011 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:06:05 smithi149 ceph-mon[35042]: pgmap v172: 97 pgs: 97 active+clean; 4.0 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 76 KiB/s wr, 27 op/s 2024-03-08T21:06:06.125 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:06:05 smithi204 ceph-mon[34935]: pgmap v172: 97 pgs: 97 active+clean; 4.0 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 76 KiB/s wr, 27 op/s 2024-03-08T21:06:07.964 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:06:07 smithi107 ceph-mon[31381]: pgmap v173: 97 pgs: 97 active+clean; 4.0 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 76 KiB/s wr, 23 op/s 2024-03-08T21:06:08.012 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:06:07 smithi149 ceph-mon[35042]: pgmap v173: 97 pgs: 97 active+clean; 4.0 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 76 KiB/s wr, 23 op/s 2024-03-08T21:06:08.125 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:06:07 smithi204 ceph-mon[34935]: pgmap v173: 97 pgs: 97 active+clean; 4.0 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 76 KiB/s wr, 23 op/s 2024-03-08T21:06:09.964 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:06:09 smithi107 ceph-mon[31381]: pgmap v174: 97 pgs: 97 active+clean; 4.0 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 78 KiB/s wr, 27 op/s 2024-03-08T21:06:10.011 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:06:09 smithi149 ceph-mon[35042]: pgmap v174: 97 pgs: 97 active+clean; 4.0 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 78 KiB/s wr, 27 op/s 2024-03-08T21:06:10.125 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:06:09 smithi204 ceph-mon[34935]: pgmap v174: 97 pgs: 97 active+clean; 4.0 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 78 KiB/s wr, 27 op/s 2024-03-08T21:06:10.184 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.203+0000 7f3274600640 1 -- 172.21.15.107:0/2704312560 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f326c103ea0 msgr2=0x7f326c110360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:06:10.184 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.203+0000 7f3274600640 1 --2- 172.21.15.107:0/2704312560 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f326c103ea0 0x7f326c110360 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f3264009ca0 tx=0x7f326402c0f0 comp rx=0 tx=0).stop 2024-03-08T21:06:10.184 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.203+0000 7f3274600640 1 -- 172.21.15.107:0/2704312560 shutdown_connections 2024-03-08T21:06:10.184 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.203+0000 7f3274600640 1 --2- 172.21.15.107:0/2704312560 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f326c103ea0 0x7f326c110360 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:06:10.184 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.203+0000 7f3274600640 1 --2- 172.21.15.107:0/2704312560 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f326c1034e0 0x7f326c103960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:06:10.185 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.203+0000 7f3274600640 1 --2- 172.21.15.107:0/2704312560 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f326c1095a0 0x7f326c109980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:06:10.185 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.203+0000 7f3274600640 1 -- 172.21.15.107:0/2704312560 >> 172.21.15.107:0/2704312560 conn(0x7f326c0fefb0 msgr2=0x7f326c1013d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:06:10.185 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.204+0000 7f3274600640 1 -- 172.21.15.107:0/2704312560 shutdown_connections 2024-03-08T21:06:10.185 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.204+0000 7f3274600640 1 -- 172.21.15.107:0/2704312560 wait complete. 2024-03-08T21:06:10.186 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.205+0000 7f3274600640 1 Processor -- start 2024-03-08T21:06:10.186 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.205+0000 7f3274600640 1 -- start start 2024-03-08T21:06:10.186 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.205+0000 7f3274600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f326c1034e0 0x7f326c19f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:06:10.186 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.205+0000 7f3274600640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f326c103ea0 0x7f326c19fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:06:10.186 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.205+0000 7f3274600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f326c1095a0 0x7f326c199930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:06:10.186 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.205+0000 7f3274600640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f326c113730 con 0x7f326c1095a0 2024-03-08T21:06:10.186 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.205+0000 7f3274600640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f326c1135b0 con 0x7f326c103ea0 2024-03-08T21:06:10.186 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.205+0000 7f3274600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f326c1138b0 con 0x7f326c1034e0 2024-03-08T21:06:10.186 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.206+0000 7f3272600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f326c1095a0 0x7f326c199930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:06:10.187 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.206+0000 7f3272600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f326c1095a0 0x7f326c199930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:32956/0 (socket says 172.21.15.107:32956) 2024-03-08T21:06:10.187 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.206+0000 7f3272600640 1 -- 172.21.15.107:0/3610442278 learned_addr learned my addr 172.21.15.107:0/3610442278 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:06:10.188 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.206+0000 7f3271c00640 1 --2- 172.21.15.107:0/3610442278 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f326c1034e0 0x7f326c19f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:06:10.188 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.206+0000 7f3271200640 1 --2- 172.21.15.107:0/3610442278 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f326c103ea0 0x7f326c19fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:06:10.188 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.206+0000 7f3272600640 1 -- 172.21.15.107:0/3610442278 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f326c1034e0 msgr2=0x7f326c19f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:06:10.188 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.206+0000 7f3272600640 1 --2- 172.21.15.107:0/3610442278 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f326c1034e0 0x7f326c19f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:06:10.188 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.206+0000 7f3272600640 1 -- 172.21.15.107:0/3610442278 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f326c103ea0 msgr2=0x7f326c19fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:06:10.188 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.206+0000 7f3272600640 1 --2- 172.21.15.107:0/3610442278 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f326c103ea0 0x7f326c19fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:06:10.188 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.206+0000 7f3272600640 1 -- 172.21.15.107:0/3610442278 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f326c19a0b0 con 0x7f326c1095a0 2024-03-08T21:06:10.189 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.206+0000 7f3271c00640 1 --2- 172.21.15.107:0/3610442278 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f326c1034e0 0x7f326c19f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T21:06:10.189 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.206+0000 7f3271200640 1 --2- 172.21.15.107:0/3610442278 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f326c103ea0 0x7f326c19fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T21:06:10.189 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.207+0000 7f3272600640 1 --2- 172.21.15.107:0/3610442278 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f326c1095a0 0x7f326c199930 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f326402c700 tx=0x7f3264035d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:06:10.189 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.207+0000 7f326a000640 1 -- 172.21.15.107:0/3610442278 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3264039040 con 0x7f326c1095a0 2024-03-08T21:06:10.189 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.207+0000 7f326a000640 1 -- 172.21.15.107:0/3610442278 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f3264044070 con 0x7f326c1095a0 2024-03-08T21:06:10.189 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.207+0000 7f3274600640 1 -- 172.21.15.107:0/3610442278 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f326c19a340 con 0x7f326c1095a0 2024-03-08T21:06:10.189 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.207+0000 7f326a000640 1 -- 172.21.15.107:0/3610442278 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32640042a0 con 0x7f326c1095a0 2024-03-08T21:06:10.189 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.207+0000 7f3274600640 1 -- 172.21.15.107:0/3610442278 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f326c1a6640 con 0x7f326c1095a0 2024-03-08T21:06:10.190 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.210+0000 7f32771fc740 1 Processor -- start 2024-03-08T21:06:10.191 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.211+0000 7f326a000640 1 -- 172.21.15.107:0/3610442278 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) v1 ==== 137422+0+0 (secure 0 0 0) 0x7f3264004440 con 0x7f326c1095a0 2024-03-08T21:06:10.191 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.211+0000 7f32771fc740 1 -- start start 2024-03-08T21:06:10.192 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.211+0000 7f32771fc740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c42ea38c80 0x55c42ea39060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:06:10.192 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.211+0000 7f32771fc740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c42ea395a0 0x55c42ea41e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:06:10.193 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.211+0000 7f32771fc740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c42ea42370 0x55c42ea44760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:06:10.193 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.211+0000 7f32771fc740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c42e99c3d0 con 0x55c42ea38c80 2024-03-08T21:06:10.193 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.211+0000 7f32771fc740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c42e9456b0 con 0x55c42ea395a0 2024-03-08T21:06:10.193 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.211+0000 7f32771fc740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c42e998470 con 0x55c42ea42370 2024-03-08T21:06:10.193 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.212+0000 7f3271200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c42ea38c80 0x55c42ea39060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:06:10.193 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.212+0000 7f3271200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c42ea38c80 0x55c42ea39060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:32972/0 (socket says 172.21.15.107:32972) 2024-03-08T21:06:10.193 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.212+0000 7f3271200640 1 -- 172.21.15.107:0/1211078912 learned_addr learned my addr 172.21.15.107:0/1211078912 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:06:10.193 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.212+0000 7f3271c00640 1 --2- 172.21.15.107:0/1211078912 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c42ea395a0 0x55c42ea41e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:06:10.193 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.212+0000 7f3271200640 1 --2- 172.21.15.107:0/1211078912 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c42ea42370 0x55c42ea44760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:06:10.193 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.212+0000 7f3271200640 1 -- 172.21.15.107:0/1211078912 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c42ea42370 msgr2=0x55c42ea44760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:06:10.193 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.212+0000 7f3271200640 1 --2- 172.21.15.107:0/1211078912 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c42ea42370 0x55c42ea44760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:06:10.193 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.212+0000 7f3271200640 1 -- 172.21.15.107:0/1211078912 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c42ea395a0 msgr2=0x55c42ea41e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:06:10.193 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.212+0000 7f3271200640 1 --2- 172.21.15.107:0/1211078912 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c42ea395a0 0x55c42ea41e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:06:10.194 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.212+0000 7f3271200640 1 -- 172.21.15.107:0/1211078912 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c42e96bb50 con 0x55c42ea38c80 2024-03-08T21:06:10.195 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.213+0000 7f326a000640 1 --2- 172.21.15.107:0/3610442278 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f32540a2a90 0x7f32540a4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:06:10.195 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.213+0000 7f3271200640 1 --2- 172.21.15.107:0/1211078912 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c42ea38c80 0x55c42ea39060 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f3258009fd0 tx=0x7f32580095f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3e22c3e9c0a8eb server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:06:10.195 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.213+0000 7f326a000640 1 -- 172.21.15.107:0/3610442278 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f32640f13e0 con 0x7f326c1095a0 2024-03-08T21:06:10.195 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.213+0000 7f3271c00640 1 --2- 172.21.15.107:0/3610442278 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f32540a2a90 0x7f32540a4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:06:10.195 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.213+0000 7f325ea00640 1 -- 172.21.15.107:0/1211078912 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32580134c0 con 0x55c42ea38c80 2024-03-08T21:06:10.196 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.213+0000 7f325ea00640 1 -- 172.21.15.107:0/1211078912 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f325801b070 con 0x55c42ea38c80 2024-03-08T21:06:10.196 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.213+0000 7f325ea00640 1 -- 172.21.15.107:0/1211078912 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3258012710 con 0x55c42ea38c80 2024-03-08T21:06:10.196 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.214+0000 7f32771fc740 1 -- 172.21.15.107:0/1211078912 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c42ea38c80 msgr2=0x55c42ea39060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:06:10.196 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.214+0000 7f3271c00640 1 --2- 172.21.15.107:0/3610442278 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f32540a2a90 0x7f32540a4f50 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f3260005fd0 tx=0x7f32600094c0 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:06:10.196 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.214+0000 7f32771fc740 1 --2- 172.21.15.107:0/1211078912 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c42ea38c80 0x55c42ea39060 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f3258009fd0 tx=0x7f32580095f0 comp rx=0 tx=0).stop 2024-03-08T21:06:10.196 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.214+0000 7f32771fc740 1 -- 172.21.15.107:0/1211078912 shutdown_connections 2024-03-08T21:06:10.196 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.214+0000 7f32771fc740 1 --2- 172.21.15.107:0/1211078912 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c42ea42370 0x55c42ea44760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:06:10.196 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.214+0000 7f32771fc740 1 --2- 172.21.15.107:0/1211078912 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c42ea395a0 0x55c42ea41e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:06:10.196 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.214+0000 7f32771fc740 1 --2- 172.21.15.107:0/1211078912 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c42ea38c80 0x55c42ea39060 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:06:10.196 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.214+0000 7f32771fc740 1 -- 172.21.15.107:0/1211078912 >> 172.21.15.107:0/1211078912 conn(0x55c42ea335a0 msgr2=0x55c42ea33980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:06:10.196 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.214+0000 7f32771fc740 1 -- 172.21.15.107:0/1211078912 shutdown_connections 2024-03-08T21:06:10.196 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.214+0000 7f32771fc740 1 -- 172.21.15.107:0/1211078912 wait complete. 2024-03-08T21:06:10.196 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.215+0000 7f32771fc740 1 -- start start 2024-03-08T21:06:10.196 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.215+0000 7f32771fc740 1 Processor -- start 2024-03-08T21:06:10.199 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.216+0000 7f32771fc740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c42ea38c80 0x55c42ea41c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:06:10.200 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.216+0000 7f32771fc740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c42ea395a0 0x55c42ea3efd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:06:10.200 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.216+0000 7f32771fc740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c42ea42370 0x55c42ea3f510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:06:10.200 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.216+0000 7f32771fc740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c42ea44f40 con 0x55c42ea38c80 2024-03-08T21:06:10.200 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.216+0000 7f32771fc740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c42e9589f0 con 0x55c42ea42370 2024-03-08T21:06:10.200 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.216+0000 7f32771fc740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c42ea450c0 con 0x55c42ea395a0 2024-03-08T21:06:10.200 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.216+0000 7f32771fc740 10 client.0 _ll_register_callbacks cb 0x55c42e6b0a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T21:06:10.200 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.216+0000 7f3271c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c42ea38c80 0x55c42ea41c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:06:10.200 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.216+0000 7f3271c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c42ea38c80 0x55c42ea41c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:32988/0 (socket says 172.21.15.107:32988) 2024-03-08T21:06:10.201 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.216+0000 7f3271c00640 1 -- 172.21.15.107:0/2134941594 learned_addr learned my addr 172.21.15.107:0/2134941594 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:06:10.201 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.216+0000 7f3271200640 1 --2- 172.21.15.107:0/2134941594 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c42ea395a0 0x55c42ea3efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:06:10.201 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.216+0000 7f3272600640 1 --2- 172.21.15.107:0/2134941594 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c42ea42370 0x55c42ea3f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:06:10.201 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.216+0000 7f3271c00640 1 -- 172.21.15.107:0/2134941594 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c42ea395a0 msgr2=0x55c42ea3efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:06:10.201 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.217+0000 7f3271c00640 1 --2- 172.21.15.107:0/2134941594 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c42ea395a0 0x55c42ea3efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:06:10.201 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.217+0000 7f3271c00640 1 -- 172.21.15.107:0/2134941594 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c42ea42370 msgr2=0x55c42ea3f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:06:10.201 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.217+0000 7f3271c00640 1 --2- 172.21.15.107:0/2134941594 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c42ea42370 0x55c42ea3f510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:06:10.201 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.217+0000 7f3271c00640 1 -- 172.21.15.107:0/2134941594 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c42e6b1210 con 0x55c42ea38c80 2024-03-08T21:06:10.201 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.217+0000 7f3272600640 1 --2- 172.21.15.107:0/2134941594 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c42ea42370 0x55c42ea3f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T21:06:10.201 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.217+0000 7f3271200640 1 --2- 172.21.15.107:0/2134941594 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c42ea395a0 0x55c42ea3efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T21:06:10.201 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.217+0000 7f3271c00640 1 --2- 172.21.15.107:0/2134941594 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c42ea38c80 0x55c42ea41c40 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f3260009fd0 tx=0x7f32600046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=539cd305eb526e19 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:06:10.201 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.217+0000 7f3253400640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-03-08T21:06:10.201 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.217+0000 7f3253400640 1 -- 172.21.15.107:0/2134941594 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f326001f9b0 con 0x55c42ea38c80 2024-03-08T21:06:10.202 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.218+0000 7f3253400640 1 -- 172.21.15.107:0/2134941594 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f326001fb50 con 0x55c42ea38c80 2024-03-08T21:06:10.202 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.218+0000 7f32771fc740 1 -- 172.21.15.107:0/2134941594 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f325801b070 con 0x55c42ea38c80 2024-03-08T21:06:10.202 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.218+0000 7f3253400640 1 -- 172.21.15.107:0/2134941594 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f326001fd50 con 0x55c42ea38c80 2024-03-08T21:06:10.202 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.218+0000 7f32771fc740 10 client.15927 fetch_fsmap learned FSMap version 34 2024-03-08T21:06:10.202 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.218+0000 7f32771fc740 1 -- 172.21.15.107:0/2134941594 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55c42e77d620 con 0x55c42ea38c80 2024-03-08T21:06:10.202 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.218+0000 7f3253400640 1 -- 172.21.15.107:0/2134941594 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f325801b070 con 0x55c42ea38c80 2024-03-08T21:06:10.202 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.218+0000 7f3253400640 1 -- 172.21.15.107:0/2134941594 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 34) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f326000f040 con 0x55c42ea38c80 2024-03-08T21:06:10.202 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.219+0000 7f32771fc740 10 client.15927 fetch_fsmap finished waiting for FSMap version 34 2024-03-08T21:06:10.202 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.219+0000 7f32771fc740 10 client.15927 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-08T21:06:10.202 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.219+0000 7f32771fc740 20 client.15927 populate_metadata read hostname 'smithi107' 2024-03-08T21:06:10.202 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.219+0000 7f32771fc740 1 --2- 172.21.15.107:0/2134941594 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55c42ea52170 0x55c42ea54560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:06:10.202 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.219+0000 7f32771fc740 4 client.15927 mds_command: new command op to 34475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-08T21:06:10.202 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.219+0000 7f32771fc740 1 -- 172.21.15.107:0/2134941594 --> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c42e9a9910 con 0x55c42ea52170 2024-03-08T21:06:10.202 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.219+0000 7f3271200640 1 --2- 172.21.15.107:0/2134941594 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55c42ea52170 0x55c42ea54560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:06:10.204 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.220+0000 7f3271200640 1 --2- 172.21.15.107:0/2134941594 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55c42ea52170 0x55c42ea54560 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=674208b2238b4b9d server_cookie=d35e0c9039a0fc92 in_seq=0 out_seq=0 2024-03-08T21:06:10.204 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.220+0000 7f3253400640 10 client.15927 ms_handle_connect on v2:172.21.15.149:6838/55290596 2024-03-08T21:06:10.204 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.222+0000 7f3253400640 1 -- 172.21.15.107:0/2134941594 <== mds.0 v2:172.21.15.149:6838/55290596 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c42e9a9910 con 0x55c42ea52170 2024-03-08T21:06:10.204 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.222+0000 7f3253400640 10 client.15927 handle_command_reply: tid=0 2024-03-08T21:06:10.204 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.222+0000 7f32771fc740 1 client.15927 shutdown 2024-03-08T21:06:10.204 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.222+0000 7f32771fc740 1 -- 172.21.15.107:0/2134941594 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c42ea38c80 msgr2=0x55c42ea41c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:06:10.204 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.222+0000 7f32771fc740 1 --2- 172.21.15.107:0/2134941594 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c42ea38c80 0x55c42ea41c40 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f3260009fd0 tx=0x7f32600046c0 comp rx=0 tx=0).stop 2024-03-08T21:06:10.204 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.222+0000 7f32771fc740 1 -- 172.21.15.107:0/2134941594 shutdown_connections 2024-03-08T21:06:10.204 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.222+0000 7f32771fc740 1 --2- 172.21.15.107:0/2134941594 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55c42ea52170 0x55c42ea54560 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:06:10.204 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.222+0000 7f32771fc740 1 --2- 172.21.15.107:0/2134941594 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c42ea42370 0x55c42ea3f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:06:10.204 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.222+0000 7f32771fc740 1 --2- 172.21.15.107:0/2134941594 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c42ea395a0 0x55c42ea3efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:06:10.204 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.222+0000 7f3253400640 0 client.15927 ms_handle_reset on v2:172.21.15.149:6838/55290596 2024-03-08T21:06:10.204 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.222+0000 7f32771fc740 1 --2- 172.21.15.107:0/2134941594 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c42ea38c80 0x55c42ea41c40 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:06:10.205 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.222+0000 7f32771fc740 1 -- 172.21.15.107:0/2134941594 >> 172.21.15.107:0/2134941594 conn(0x55c42ea335a0 msgr2=0x55c42ea352f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:06:10.205 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.222+0000 7f32771fc740 1 -- 172.21.15.107:0/2134941594 shutdown_connections 2024-03-08T21:06:10.205 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.222+0000 7f32771fc740 1 -- 172.21.15.107:0/2134941594 wait complete. 2024-03-08T21:06:10.205 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.222+0000 7f32771fc740 20 client.15927 trim_cache size 0 max 16384 2024-03-08T21:06:10.222 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.240+0000 7f32771fc740 1 Processor -- start 2024-03-08T21:06:10.222 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.240+0000 7f32771fc740 1 -- start start 2024-03-08T21:06:10.222 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.241+0000 7f32771fc740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c42ea38c80 0x55c42ea41700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:06:10.222 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.241+0000 7f32771fc740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c42ea395a0 0x55c42ea41c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:06:10.222 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.241+0000 7f32771fc740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c42ea42370 0x55c42ea3c230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:06:10.222 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.241+0000 7f32771fc740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f326c1aed40 con 0x55c42ea42370 2024-03-08T21:06:10.222 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.241+0000 7f32771fc740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f326c1b06c0 con 0x55c42ea38c80 2024-03-08T21:06:10.223 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.241+0000 7f32771fc740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f326c1af3a0 con 0x55c42ea395a0 2024-03-08T21:06:10.223 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.241+0000 7f3272600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c42ea42370 0x55c42ea3c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:06:10.223 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.241+0000 7f3272600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c42ea42370 0x55c42ea3c230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:33004/0 (socket says 172.21.15.107:33004) 2024-03-08T21:06:10.223 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.241+0000 7f3272600640 1 -- 172.21.15.107:0/1530534283 learned_addr learned my addr 172.21.15.107:0/1530534283 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:06:10.223 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.241+0000 7f3271200640 1 --2- 172.21.15.107:0/1530534283 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c42ea395a0 0x55c42ea41c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:06:10.223 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.241+0000 7f3271c00640 1 --2- 172.21.15.107:0/1530534283 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c42ea38c80 0x55c42ea41700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:06:10.223 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.242+0000 7f3272600640 1 -- 172.21.15.107:0/1530534283 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c42ea395a0 msgr2=0x55c42ea41c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:06:10.223 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.242+0000 7f3272600640 1 --2- 172.21.15.107:0/1530534283 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c42ea395a0 0x55c42ea41c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:06:10.223 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.242+0000 7f3272600640 1 -- 172.21.15.107:0/1530534283 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c42ea38c80 msgr2=0x55c42ea41700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:06:10.223 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.242+0000 7f3272600640 1 --2- 172.21.15.107:0/1530534283 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c42ea38c80 0x55c42ea41700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:06:10.223 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.242+0000 7f3272600640 1 -- 172.21.15.107:0/1530534283 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c42ea393d0 con 0x55c42ea42370 2024-03-08T21:06:10.223 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.242+0000 7f3272600640 1 --2- 172.21.15.107:0/1530534283 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c42ea42370 0x55c42ea3c230 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f32640eece0 tx=0x7f3264034700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74285531b742f5d9 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:06:10.227 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.242+0000 7f325e000640 1 -- 172.21.15.107:0/1530534283 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32640bdb40 con 0x55c42ea42370 2024-03-08T21:06:10.227 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.242+0000 7f325e000640 1 -- 172.21.15.107:0/1530534283 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f32640c1c90 con 0x55c42ea42370 2024-03-08T21:06:10.227 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.242+0000 7f325e000640 1 -- 172.21.15.107:0/1530534283 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32640ba1b0 con 0x55c42ea42370 2024-03-08T21:06:10.227 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.243+0000 7f32771fc740 1 -- 172.21.15.107:0/1530534283 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c42ea42370 msgr2=0x55c42ea3c230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:06:10.227 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.243+0000 7f32771fc740 1 --2- 172.21.15.107:0/1530534283 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c42ea42370 0x55c42ea3c230 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f32640eece0 tx=0x7f3264034700 comp rx=0 tx=0).stop 2024-03-08T21:06:10.227 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.243+0000 7f32771fc740 1 -- 172.21.15.107:0/1530534283 shutdown_connections 2024-03-08T21:06:10.227 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.243+0000 7f32771fc740 1 --2- 172.21.15.107:0/1530534283 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c42ea42370 0x55c42ea3c230 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:06:10.227 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.243+0000 7f32771fc740 1 --2- 172.21.15.107:0/1530534283 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c42ea395a0 0x55c42ea41c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:06:10.227 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.243+0000 7f32771fc740 1 --2- 172.21.15.107:0/1530534283 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c42ea38c80 0x55c42ea41700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:06:10.227 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.243+0000 7f32771fc740 1 -- 172.21.15.107:0/1530534283 >> 172.21.15.107:0/1530534283 conn(0x55c42ea335a0 msgr2=0x55c42ea326c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:06:10.228 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.243+0000 7f32771fc740 1 -- 172.21.15.107:0/1530534283 shutdown_connections 2024-03-08T21:06:10.228 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.243+0000 7f32771fc740 1 -- 172.21.15.107:0/1530534283 wait complete. 2024-03-08T21:06:10.228 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.244+0000 7f32771fc740 1 -- start start 2024-03-08T21:06:10.228 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.244+0000 7f32771fc740 1 Processor -- start 2024-03-08T21:06:10.228 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.244+0000 7f32771fc740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c42ea38c80 0x55c42ea47cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:06:10.228 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.244+0000 7f32771fc740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c42ea395a0 0x55c42ea48230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:06:10.228 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.244+0000 7f32771fc740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c42ea42370 0x55c42ea48770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:06:10.228 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.244+0000 7f32771fc740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f326c1b0060 con 0x55c42ea38c80 2024-03-08T21:06:10.228 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.244+0000 7f32771fc740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f326c1afa00 con 0x55c42ea395a0 2024-03-08T21:06:10.228 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.244+0000 7f32771fc740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f326c1acfe0 con 0x55c42ea42370 2024-03-08T21:06:10.228 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.244+0000 7f32771fc740 10 client.0 _ll_register_callbacks cb 0x7f3258009470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T21:06:10.228 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.245+0000 7f3271c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c42ea38c80 0x55c42ea47cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:06:10.228 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.245+0000 7f3271c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c42ea38c80 0x55c42ea47cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:33012/0 (socket says 172.21.15.107:33012) 2024-03-08T21:06:10.229 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.245+0000 7f3271c00640 1 -- 172.21.15.107:0/3752774730 learned_addr learned my addr 172.21.15.107:0/3752774730 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:06:10.229 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.245+0000 7f3271200640 1 --2- 172.21.15.107:0/3752774730 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c42ea395a0 0x55c42ea48230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:06:10.229 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.245+0000 7f3271c00640 1 -- 172.21.15.107:0/3752774730 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c42ea42370 msgr2=0x55c42ea48770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:06:10.229 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.245+0000 7f3271c00640 1 --2- 172.21.15.107:0/3752774730 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c42ea42370 0x55c42ea48770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:06:10.229 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.245+0000 7f3271c00640 1 -- 172.21.15.107:0/3752774730 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c42ea395a0 msgr2=0x55c42ea48230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:06:10.229 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.245+0000 7f3271c00640 1 --2- 172.21.15.107:0/3752774730 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c42ea395a0 0x55c42ea48230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:06:10.229 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.245+0000 7f3271c00640 1 -- 172.21.15.107:0/3752774730 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c42eaa8680 con 0x55c42ea38c80 2024-03-08T21:06:10.229 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.245+0000 7f3271c00640 1 --2- 172.21.15.107:0/3752774730 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c42ea38c80 0x55c42ea47cf0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f3260009fd0 tx=0x7f326001d950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b876ad2e0d1973 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:06:10.229 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.245+0000 7f3253400640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-03-08T21:06:10.229 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.246+0000 7f3253400640 1 -- 172.21.15.107:0/3752774730 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f326003c070 con 0x55c42ea38c80 2024-03-08T21:06:10.229 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.246+0000 7f3253400640 1 -- 172.21.15.107:0/3752774730 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f326001dd70 con 0x55c42ea38c80 2024-03-08T21:06:10.229 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.246+0000 7f32771fc740 1 -- 172.21.15.107:0/3752774730 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f32640c1c90 con 0x55c42ea38c80 2024-03-08T21:06:10.229 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.246+0000 7f3253400640 1 -- 172.21.15.107:0/3752774730 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3260044050 con 0x55c42ea38c80 2024-03-08T21:06:10.229 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.246+0000 7f32771fc740 10 client.15933 fetch_fsmap learned FSMap version 34 2024-03-08T21:06:10.230 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.246+0000 7f32771fc740 1 -- 172.21.15.107:0/3752774730 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55c42eaa9990 con 0x55c42ea38c80 2024-03-08T21:06:10.230 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.246+0000 7f3253400640 1 -- 172.21.15.107:0/3752774730 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f32640c1c90 con 0x55c42ea38c80 2024-03-08T21:06:10.230 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.246+0000 7f3253400640 1 -- 172.21.15.107:0/3752774730 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 34) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f326003f030 con 0x55c42ea38c80 2024-03-08T21:06:10.230 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.246+0000 7f32771fc740 10 client.15933 fetch_fsmap finished waiting for FSMap version 34 2024-03-08T21:06:10.230 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.246+0000 7f32771fc740 10 client.15933 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-08T21:06:10.230 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.246+0000 7f32771fc740 20 client.15933 populate_metadata read hostname 'smithi107' 2024-03-08T21:06:10.230 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.247+0000 7f32771fc740 1 --2- 172.21.15.107:0/3752774730 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55c42eaaa110 0x55c42eaac530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:06:10.230 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.247+0000 7f32771fc740 4 client.15933 mds_command: new command op to 34475 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-08T21:06:10.230 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.247+0000 7f32771fc740 1 -- 172.21.15.107:0/3752774730 --> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55c42e9a3c60 con 0x55c42eaaa110 2024-03-08T21:06:10.231 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.247+0000 7f3271200640 1 --2- 172.21.15.107:0/3752774730 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55c42eaaa110 0x55c42eaac530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:06:10.231 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.248+0000 7f3271200640 1 --2- 172.21.15.107:0/3752774730 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55c42eaaa110 0x55c42eaac530 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f8629d9c432c75a8 server_cookie=9c1740d504e7e6a in_seq=0 out_seq=0 2024-03-08T21:06:10.231 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.248+0000 7f3253400640 10 client.15933 ms_handle_connect on v2:172.21.15.149:6838/55290596 2024-03-08T21:06:10.253 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.272+0000 7f3253400640 1 -- 172.21.15.107:0/3752774730 <== mds.0 v2:172.21.15.149:6838/55290596 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55c42e9a3c60 con 0x55c42eaaa110 2024-03-08T21:06:10.253 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.272+0000 7f3253400640 10 client.15933 handle_command_reply: tid=0 2024-03-08T21:06:10.253 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.272+0000 7f32771fc740 1 client.15933 shutdown 2024-03-08T21:06:10.253 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.272+0000 7f32771fc740 1 -- 172.21.15.107:0/3752774730 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c42ea38c80 msgr2=0x55c42ea47cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:06:10.253 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.272+0000 7f32771fc740 1 --2- 172.21.15.107:0/3752774730 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c42ea38c80 0x55c42ea47cf0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f3260009fd0 tx=0x7f326001d950 comp rx=0 tx=0).stop 2024-03-08T21:06:10.253 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.272+0000 7f32771fc740 1 -- 172.21.15.107:0/3752774730 shutdown_connections 2024-03-08T21:06:10.253 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.272+0000 7f32771fc740 1 --2- 172.21.15.107:0/3752774730 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c42ea42370 0x55c42ea48770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:06:10.254 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.272+0000 7f32771fc740 1 --2- 172.21.15.107:0/3752774730 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55c42eaaa110 0x55c42eaac530 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:06:10.254 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.272+0000 7f32771fc740 1 --2- 172.21.15.107:0/3752774730 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c42ea395a0 0x55c42ea48230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:06:10.254 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.272+0000 7f32771fc740 1 --2- 172.21.15.107:0/3752774730 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c42ea38c80 0x55c42ea47cf0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:06:10.254 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.272+0000 7f3253400640 0 client.15933 ms_handle_reset on v2:172.21.15.149:6838/55290596 2024-03-08T21:06:10.254 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.272+0000 7f32771fc740 1 -- 172.21.15.107:0/3752774730 >> 172.21.15.107:0/3752774730 conn(0x55c42ea335a0 msgr2=0x55c42ea39c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:06:10.254 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.272+0000 7f32771fc740 1 -- 172.21.15.107:0/3752774730 shutdown_connections 2024-03-08T21:06:10.254 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.272+0000 7f32771fc740 1 -- 172.21.15.107:0/3752774730 wait complete. 2024-03-08T21:06:10.254 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.273+0000 7f32771fc740 20 client.15933 trim_cache size 0 max 16384 2024-03-08T21:06:10.254 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:{ 2024-03-08T21:06:10.255 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "message": "", 2024-03-08T21:06:10.255 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "return_code": 0 2024-03-08T21:06:10.255 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:} 2024-03-08T21:06:10.255 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.273+0000 7f3274600640 1 -- 172.21.15.107:0/3610442278 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f32540a2a90 msgr2=0x7f32540a4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:06:10.255 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.273+0000 7f3274600640 1 --2- 172.21.15.107:0/3610442278 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f32540a2a90 0x7f32540a4f50 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f3260005fd0 tx=0x7f32600094c0 comp rx=0 tx=0).stop 2024-03-08T21:06:10.255 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.273+0000 7f3274600640 1 -- 172.21.15.107:0/3610442278 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f326c1095a0 msgr2=0x7f326c199930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:06:10.255 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.273+0000 7f3274600640 1 --2- 172.21.15.107:0/3610442278 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f326c1095a0 0x7f326c199930 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f326402c700 tx=0x7f3264035d60 comp rx=0 tx=0).stop 2024-03-08T21:06:10.256 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.273+0000 7f3274600640 1 -- 172.21.15.107:0/3610442278 shutdown_connections 2024-03-08T21:06:10.256 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.274+0000 7f3274600640 1 --2- 172.21.15.107:0/3610442278 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f32540a2a90 0x7f32540a4f50 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:06:10.256 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.274+0000 7f3274600640 1 --2- 172.21.15.107:0/3610442278 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f326c1095a0 0x7f326c199930 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:06:10.256 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.274+0000 7f3274600640 1 --2- 172.21.15.107:0/3610442278 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f326c103ea0 0x7f326c19fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:06:10.256 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.274+0000 7f3274600640 1 --2- 172.21.15.107:0/3610442278 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f326c1034e0 0x7f326c19f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:06:10.256 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.274+0000 7f3274600640 1 -- 172.21.15.107:0/3610442278 >> 172.21.15.107:0/3610442278 conn(0x7f326c0fefb0 msgr2=0x7f326c1006f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:06:10.256 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.274+0000 7f3274600640 1 -- 172.21.15.107:0/3610442278 shutdown_connections 2024-03-08T21:06:10.256 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:10.274+0000 7f3274600640 1 -- 172.21.15.107:0/3610442278 wait complete. 2024-03-08T21:06:10.964 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:06:10 smithi107 ceph-mon[31381]: pgmap v175: 97 pgs: 97 active+clean; 4.0 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 78 KiB/s wr, 23 op/s 2024-03-08T21:06:11.011 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:06:10 smithi149 ceph-mon[35042]: pgmap v175: 97 pgs: 97 active+clean; 4.0 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 78 KiB/s wr, 23 op/s 2024-03-08T21:06:11.125 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:06:10 smithi204 ceph-mon[34935]: pgmap v175: 97 pgs: 97 active+clean; 4.0 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 78 KiB/s wr, 23 op/s 2024-03-08T21:06:13.964 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:06:13 smithi107 ceph-mon[31381]: pgmap v176: 97 pgs: 97 active+clean; 4.0 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 78 KiB/s wr, 23 op/s 2024-03-08T21:06:14.011 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:06:13 smithi149 ceph-mon[35042]: pgmap v176: 97 pgs: 97 active+clean; 4.0 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 78 KiB/s wr, 23 op/s 2024-03-08T21:06:14.125 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:06:13 smithi204 ceph-mon[34935]: pgmap v176: 97 pgs: 97 active+clean; 4.0 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 78 KiB/s wr, 23 op/s 2024-03-08T21:06:15.964 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:06:15 smithi107 ceph-mon[31381]: pgmap v177: 97 pgs: 97 active+clean; 4.0 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 81 KiB/s wr, 30 op/s 2024-03-08T21:06:16.011 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:06:15 smithi149 ceph-mon[35042]: pgmap v177: 97 pgs: 97 active+clean; 4.0 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 81 KiB/s wr, 30 op/s 2024-03-08T21:06:16.125 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:06:15 smithi204 ceph-mon[34935]: pgmap v177: 97 pgs: 97 active+clean; 4.0 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 81 KiB/s wr, 30 op/s 2024-03-08T21:06:17.012 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:06:16 smithi149 ceph-mon[35042]: pgmap v178: 97 pgs: 97 active+clean; 4.0 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 5.0 KiB/s wr, 11 op/s 2024-03-08T21:06:17.125 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:06:16 smithi204 ceph-mon[34935]: pgmap v178: 97 pgs: 97 active+clean; 4.0 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 5.0 KiB/s wr, 11 op/s 2024-03-08T21:06:17.214 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:06:16 smithi107 ceph-mon[31381]: pgmap v178: 97 pgs: 97 active+clean; 4.0 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 5.0 KiB/s wr, 11 op/s 2024-03-08T21:06:19.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:06:19 smithi107 ceph-mon[31381]: pgmap v179: 97 pgs: 97 active+clean; 4.0 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 5.0 KiB/s wr, 15 op/s 2024-03-08T21:06:20.011 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:06:19 smithi149 ceph-mon[35042]: pgmap v179: 97 pgs: 97 active+clean; 4.0 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 5.0 KiB/s wr, 15 op/s 2024-03-08T21:06:20.125 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:06:19 smithi204 ceph-mon[34935]: pgmap v179: 97 pgs: 97 active+clean; 4.0 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 5.0 KiB/s wr, 15 op/s 2024-03-08T21:06:21.964 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:06:21 smithi107 ceph-mon[31381]: pgmap v180: 97 pgs: 97 active+clean; 4.0 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 2.8 KiB/s wr, 10 op/s 2024-03-08T21:06:22.011 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:06:21 smithi149 ceph-mon[35042]: pgmap v180: 97 pgs: 97 active+clean; 4.0 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 2.8 KiB/s wr, 10 op/s 2024-03-08T21:06:22.125 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:06:21 smithi204 ceph-mon[34935]: pgmap v180: 97 pgs: 97 active+clean; 4.0 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 2.8 KiB/s wr, 10 op/s 2024-03-08T21:06:23.011 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:06:22 smithi149 ceph-mon[35042]: pgmap v181: 97 pgs: 97 active+clean; 4.0 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 2.8 KiB/s wr, 10 op/s 2024-03-08T21:06:23.125 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:06:22 smithi204 ceph-mon[34935]: pgmap v181: 97 pgs: 97 active+clean; 4.0 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 2.8 KiB/s wr, 10 op/s 2024-03-08T21:06:23.214 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:06:22 smithi107 ceph-mon[31381]: pgmap v181: 97 pgs: 97 active+clean; 4.0 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 2.8 KiB/s wr, 10 op/s 2024-03-08T21:06:23.400 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.419+0000 7fd7c0a00640 1 -- 172.21.15.107:0/723382988 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd7bc101510 msgr2=0x7fd7bc1018f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:06:23.400 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.419+0000 7fd7c0a00640 1 --2- 172.21.15.107:0/723382988 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd7bc101510 0x7fd7bc1018f0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fd7ac009900 tx=0x7fd7ac02bc30 comp rx=0 tx=0).stop 2024-03-08T21:06:23.400 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.419+0000 7fd7c0a00640 1 -- 172.21.15.107:0/723382988 shutdown_connections 2024-03-08T21:06:23.400 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.419+0000 7fd7c0a00640 1 --2- 172.21.15.107:0/723382988 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd7bc1065e0 0x7fd7bc10e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:06:23.400 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.419+0000 7fd7c0a00640 1 --2- 172.21.15.107:0/723382988 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd7bc101ec0 0x7fd7bc105f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:06:23.400 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.419+0000 7fd7c0a00640 1 --2- 172.21.15.107:0/723382988 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd7bc101510 0x7fd7bc1018f0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:06:23.400 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.419+0000 7fd7c0a00640 1 -- 172.21.15.107:0/723382988 >> 172.21.15.107:0/723382988 conn(0x7fd7bc0fcdb0 msgr2=0x7fd7bc0ff1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:06:23.401 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.420+0000 7fd7c0a00640 1 -- 172.21.15.107:0/723382988 shutdown_connections 2024-03-08T21:06:23.401 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.420+0000 7fd7c0a00640 1 -- 172.21.15.107:0/723382988 wait complete. 2024-03-08T21:06:23.402 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.421+0000 7fd7c0a00640 1 Processor -- start 2024-03-08T21:06:23.402 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.421+0000 7fd7c0a00640 1 -- start start 2024-03-08T21:06:23.403 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.421+0000 7fd7c0a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd7bc101510 0x7fd7bc19f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:06:23.403 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.421+0000 7fd7c0a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd7bc1065e0 0x7fd7bc19fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:06:23.403 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.421+0000 7fd7c0a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd7bc1999c0 0x7fd7bc199e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:06:23.403 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.421+0000 7fd7c0a00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7bc111570 con 0x7fd7bc101510 2024-03-08T21:06:23.403 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.421+0000 7fd7c0a00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7bc1113f0 con 0x7fd7bc1065e0 2024-03-08T21:06:23.403 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.421+0000 7fd7c0a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7bc1116f0 con 0x7fd7bc1999c0 2024-03-08T21:06:23.403 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.421+0000 7fd7b9e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd7bc101510 0x7fd7bc19f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:06:23.403 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.422+0000 7fd7b9e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd7bc101510 0x7fd7bc19f810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:54630/0 (socket says 172.21.15.107:54630) 2024-03-08T21:06:23.403 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.422+0000 7fd7b9e00640 1 -- 172.21.15.107:0/3241451875 learned_addr learned my addr 172.21.15.107:0/3241451875 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:06:23.403 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.422+0000 7fd7b9400640 1 --2- 172.21.15.107:0/3241451875 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd7bc1065e0 0x7fd7bc19fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:06:23.403 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.422+0000 7fd7b9e00640 1 -- 172.21.15.107:0/3241451875 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd7bc1999c0 msgr2=0x7fd7bc199e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:06:23.403 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.422+0000 7fd7b9e00640 1 --2- 172.21.15.107:0/3241451875 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd7bc1999c0 0x7fd7bc199e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:06:23.403 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.422+0000 7fd7b9e00640 1 -- 172.21.15.107:0/3241451875 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd7bc1065e0 msgr2=0x7fd7bc19fd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:06:23.403 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.422+0000 7fd7b9e00640 1 --2- 172.21.15.107:0/3241451875 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd7bc1065e0 0x7fd7bc19fd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:06:23.404 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.422+0000 7fd7b9e00640 1 -- 172.21.15.107:0/3241451875 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7bc19a4e0 con 0x7fd7bc101510 2024-03-08T21:06:23.404 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.422+0000 7fd7b9400640 1 --2- 172.21.15.107:0/3241451875 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd7bc1065e0 0x7fd7bc19fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T21:06:23.404 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.422+0000 7fd7b9e00640 1 --2- 172.21.15.107:0/3241451875 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd7bc101510 0x7fd7bc19f810 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fd7ac005ab0 tx=0x7fd7ac02ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:06:23.404 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.423+0000 7fd7b2a00640 1 -- 172.21.15.107:0/3241451875 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7ac043020 con 0x7fd7bc101510 2024-03-08T21:06:23.404 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.423+0000 7fd7b2a00640 1 -- 172.21.15.107:0/3241451875 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fd7ac02cc80 con 0x7fd7bc101510 2024-03-08T21:06:23.405 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.423+0000 7fd7b2a00640 1 -- 172.21.15.107:0/3241451875 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7ac003d70 con 0x7fd7bc101510 2024-03-08T21:06:23.405 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.423+0000 7fd7c0a00640 1 -- 172.21.15.107:0/3241451875 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7bc19a770 con 0x7fd7bc101510 2024-03-08T21:06:23.405 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.423+0000 7fd7c0a00640 1 -- 172.21.15.107:0/3241451875 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7bc1a6970 con 0x7fd7bc101510 2024-03-08T21:06:23.407 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.426+0000 7fd7b2a00640 1 -- 172.21.15.107:0/3241451875 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) v1 ==== 137422+0+0 (secure 0 0 0) 0x7fd7ac046020 con 0x7fd7bc101510 2024-03-08T21:06:23.407 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.427+0000 7fd7c35fa740 1 Processor -- start 2024-03-08T21:06:23.407 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.427+0000 7fd7c35fa740 1 -- start start 2024-03-08T21:06:23.408 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.427+0000 7fd7c35fa740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5618ebe81c80 0x5618ebe82060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:06:23.408 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.427+0000 7fd7c35fa740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5618ebe825a0 0x5618ebe8ae30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:06:23.409 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.427+0000 7fd7c35fa740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5618ebe8b370 0x5618ebe8d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:06:23.409 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.427+0000 7fd7c35fa740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618ebde53d0 con 0x5618ebe81c80 2024-03-08T21:06:23.409 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.427+0000 7fd7c35fa740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618ebd8e6b0 con 0x5618ebe825a0 2024-03-08T21:06:23.409 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.427+0000 7fd7c35fa740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618ebde1470 con 0x5618ebe8b370 2024-03-08T21:06:23.409 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.427+0000 7fd7ba800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5618ebe81c80 0x5618ebe82060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:06:23.409 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.428+0000 7fd7ba800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5618ebe81c80 0x5618ebe82060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:54644/0 (socket says 172.21.15.107:54644) 2024-03-08T21:06:23.409 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.428+0000 7fd7ba800640 1 -- 172.21.15.107:0/2032806070 learned_addr learned my addr 172.21.15.107:0/2032806070 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:06:23.409 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.428+0000 7fd7b9400640 1 --2- 172.21.15.107:0/2032806070 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5618ebe8b370 0x5618ebe8d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:06:23.409 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.428+0000 7fd7b9e00640 1 --2- 172.21.15.107:0/2032806070 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5618ebe825a0 0x5618ebe8ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:06:23.409 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.428+0000 7fd7ba800640 1 -- 172.21.15.107:0/2032806070 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5618ebe8b370 msgr2=0x5618ebe8d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:06:23.409 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.428+0000 7fd7ba800640 1 --2- 172.21.15.107:0/2032806070 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5618ebe8b370 0x5618ebe8d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:06:23.409 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.428+0000 7fd7ba800640 1 -- 172.21.15.107:0/2032806070 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5618ebe825a0 msgr2=0x5618ebe8ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:06:23.409 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.428+0000 7fd7ba800640 1 --2- 172.21.15.107:0/2032806070 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5618ebe825a0 0x5618ebe8ae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:06:23.410 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.428+0000 7fd7ba800640 1 -- 172.21.15.107:0/2032806070 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5618ebdb4b50 con 0x5618ebe81c80 2024-03-08T21:06:23.410 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.428+0000 7fd7b9e00640 1 --2- 172.21.15.107:0/2032806070 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5618ebe825a0 0x5618ebe8ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T21:06:23.410 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.428+0000 7fd7b2a00640 1 --2- 172.21.15.107:0/3241451875 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7fd7a00a2a90 0x7fd7a00a4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:06:23.410 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.428+0000 7fd7ba800640 1 --2- 172.21.15.107:0/2032806070 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5618ebe81c80 0x5618ebe82060 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fd7b4007bb0 tx=0x7fd7b400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=267d5da147271e51 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:06:23.410 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.428+0000 7fd7b2a00640 1 -- 172.21.15.107:0/3241451875 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fd7ac0f1190 con 0x7fd7bc101510 2024-03-08T21:06:23.410 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.429+0000 7fd7b9e00640 1 --2- 172.21.15.107:0/3241451875 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7fd7a00a2a90 0x7fd7a00a4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:06:23.413 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.429+0000 7fd7a7000640 1 -- 172.21.15.107:0/2032806070 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7b4014070 con 0x5618ebe81c80 2024-03-08T21:06:23.413 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.429+0000 7fd7a7000640 1 -- 172.21.15.107:0/2032806070 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fd7b400fb00 con 0x5618ebe81c80 2024-03-08T21:06:23.413 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.429+0000 7fd7a7000640 1 -- 172.21.15.107:0/2032806070 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7b400e9b0 con 0x5618ebe81c80 2024-03-08T21:06:23.413 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.430+0000 7fd7c35fa740 1 -- 172.21.15.107:0/2032806070 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5618ebe81c80 msgr2=0x5618ebe82060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:06:23.413 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.430+0000 7fd7b9e00640 1 --2- 172.21.15.107:0/3241451875 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7fd7a00a2a90 0x7fd7a00a4f50 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fd7ac0eea90 tx=0x7fd7ac009260 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:06:23.413 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.430+0000 7fd7c35fa740 1 --2- 172.21.15.107:0/2032806070 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5618ebe81c80 0x5618ebe82060 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fd7b4007bb0 tx=0x7fd7b400c6a0 comp rx=0 tx=0).stop 2024-03-08T21:06:23.413 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.430+0000 7fd7c35fa740 1 -- 172.21.15.107:0/2032806070 shutdown_connections 2024-03-08T21:06:23.413 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.430+0000 7fd7c35fa740 1 --2- 172.21.15.107:0/2032806070 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5618ebe8b370 0x5618ebe8d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:06:23.414 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.430+0000 7fd7c35fa740 1 --2- 172.21.15.107:0/2032806070 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5618ebe825a0 0x5618ebe8ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:06:23.414 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.430+0000 7fd7c35fa740 1 --2- 172.21.15.107:0/2032806070 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5618ebe81c80 0x5618ebe82060 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:06:23.414 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.430+0000 7fd7c35fa740 1 -- 172.21.15.107:0/2032806070 >> 172.21.15.107:0/2032806070 conn(0x5618ebe7c5a0 msgr2=0x5618ebe7c980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:06:23.414 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.430+0000 7fd7c35fa740 1 -- 172.21.15.107:0/2032806070 shutdown_connections 2024-03-08T21:06:23.414 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.430+0000 7fd7c35fa740 1 -- 172.21.15.107:0/2032806070 wait complete. 2024-03-08T21:06:23.414 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.430+0000 7fd7c35fa740 1 -- start start 2024-03-08T21:06:23.414 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.431+0000 7fd7c35fa740 1 Processor -- start 2024-03-08T21:06:23.414 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.431+0000 7fd7c35fa740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5618ebe81c80 0x5618ebe8ad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:06:23.414 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.431+0000 7fd7c35fa740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5618ebe825a0 0x5618ebe88100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:06:23.414 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.431+0000 7fd7c35fa740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5618ebe8b370 0x5618ebe88710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:06:23.414 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.431+0000 7fd7c35fa740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618ebe8dfa0 con 0x5618ebe81c80 2024-03-08T21:06:23.414 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.431+0000 7fd7c35fa740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618ebda19f0 con 0x5618ebe8b370 2024-03-08T21:06:23.415 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.431+0000 7fd7c35fa740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618ebe8e120 con 0x5618ebe825a0 2024-03-08T21:06:23.415 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.431+0000 7fd7c35fa740 10 client.0 _ll_register_callbacks cb 0x5618ebaf9a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T21:06:23.415 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.432+0000 7fd7ba800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5618ebe81c80 0x5618ebe8ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:06:23.415 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.432+0000 7fd7ba800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5618ebe81c80 0x5618ebe8ad70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:54650/0 (socket says 172.21.15.107:54650) 2024-03-08T21:06:23.415 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.432+0000 7fd7ba800640 1 -- 172.21.15.107:0/2514082138 learned_addr learned my addr 172.21.15.107:0/2514082138 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:06:23.415 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.432+0000 7fd7b9400640 1 --2- 172.21.15.107:0/2514082138 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5618ebe825a0 0x5618ebe88100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:06:23.415 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.432+0000 7fd7ba800640 1 --2- 172.21.15.107:0/2514082138 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5618ebe8b370 0x5618ebe88710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:06:23.415 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.432+0000 7fd7ba800640 1 -- 172.21.15.107:0/2514082138 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5618ebe825a0 msgr2=0x5618ebe88100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:06:23.415 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.432+0000 7fd7ba800640 1 --2- 172.21.15.107:0/2514082138 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5618ebe825a0 0x5618ebe88100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:06:23.415 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.432+0000 7fd7ba800640 1 -- 172.21.15.107:0/2514082138 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5618ebe8b370 msgr2=0x5618ebe88710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:06:23.415 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.432+0000 7fd7ba800640 1 --2- 172.21.15.107:0/2514082138 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5618ebe8b370 0x5618ebe88710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:06:23.415 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.432+0000 7fd7ba800640 1 -- 172.21.15.107:0/2514082138 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5618ebafa210 con 0x5618ebe81c80 2024-03-08T21:06:23.415 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.433+0000 7fd7ba800640 1 --2- 172.21.15.107:0/2514082138 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5618ebe81c80 0x5618ebe8ad70 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fd7b400e650 tx=0x7fd7b4007ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5e696af33c67712 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:06:23.415 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.433+0000 7fd79b400640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-03-08T21:06:23.415 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.433+0000 7fd79b400640 1 -- 172.21.15.107:0/2514082138 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7b4014070 con 0x5618ebe81c80 2024-03-08T21:06:23.416 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.433+0000 7fd79b400640 1 -- 172.21.15.107:0/2514082138 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fd7b400f860 con 0x5618ebe81c80 2024-03-08T21:06:23.416 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.433+0000 7fd7c35fa740 1 -- 172.21.15.107:0/2514082138 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd7b400fb00 con 0x5618ebe81c80 2024-03-08T21:06:23.418 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.433+0000 7fd79b400640 1 -- 172.21.15.107:0/2514082138 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd7b4022040 con 0x5618ebe81c80 2024-03-08T21:06:23.419 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.433+0000 7fd7c35fa740 10 client.15945 fetch_fsmap learned FSMap version 34 2024-03-08T21:06:23.419 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.433+0000 7fd7c35fa740 1 -- 172.21.15.107:0/2514082138 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x5618ebbc6620 con 0x5618ebe81c80 2024-03-08T21:06:23.419 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.434+0000 7fd79b400640 1 -- 172.21.15.107:0/2514082138 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fd7b400fb00 con 0x5618ebe81c80 2024-03-08T21:06:23.419 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.434+0000 7fd79b400640 1 -- 172.21.15.107:0/2514082138 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 34) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fd7b4004040 con 0x5618ebe81c80 2024-03-08T21:06:23.419 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.434+0000 7fd7c35fa740 10 client.15945 fetch_fsmap finished waiting for FSMap version 34 2024-03-08T21:06:23.419 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.434+0000 7fd7c35fa740 10 client.15945 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-08T21:06:23.419 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.434+0000 7fd7c35fa740 20 client.15945 populate_metadata read hostname 'smithi107' 2024-03-08T21:06:23.419 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.434+0000 7fd7c35fa740 1 --2- 172.21.15.107:0/2514082138 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x5618ebe9b0f0 0x5618ebe9d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:06:23.419 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.434+0000 7fd7c35fa740 4 client.15945 mds_command: new command op to 34475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-08T21:06:23.419 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.434+0000 7fd7c35fa740 1 -- 172.21.15.107:0/2514082138 --> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5618ebdf2910 con 0x5618ebe9b0f0 2024-03-08T21:06:23.419 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.435+0000 7fd7b9400640 1 --2- 172.21.15.107:0/2514082138 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x5618ebe9b0f0 0x5618ebe9d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:06:23.419 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.436+0000 7fd7b9400640 1 --2- 172.21.15.107:0/2514082138 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x5618ebe9b0f0 0x5618ebe9d4e0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd8b5a9f19aaa039 server_cookie=8bfa0837687d21ab in_seq=0 out_seq=0 2024-03-08T21:06:23.420 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.436+0000 7fd79b400640 10 client.15945 ms_handle_connect on v2:172.21.15.149:6838/55290596 2024-03-08T21:06:23.420 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.437+0000 7fd79b400640 1 -- 172.21.15.107:0/2514082138 <== mds.0 v2:172.21.15.149:6838/55290596 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5618ebdf2910 con 0x5618ebe9b0f0 2024-03-08T21:06:23.420 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.437+0000 7fd79b400640 10 client.15945 handle_command_reply: tid=0 2024-03-08T21:06:23.420 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.437+0000 7fd7c35fa740 1 client.15945 shutdown 2024-03-08T21:06:23.420 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.438+0000 7fd7c35fa740 1 -- 172.21.15.107:0/2514082138 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5618ebe81c80 msgr2=0x5618ebe8ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:06:23.420 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.438+0000 7fd7c35fa740 1 --2- 172.21.15.107:0/2514082138 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5618ebe81c80 0x5618ebe8ad70 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fd7b400e650 tx=0x7fd7b4007ef0 comp rx=0 tx=0).stop 2024-03-08T21:06:23.420 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.438+0000 7fd7c35fa740 1 -- 172.21.15.107:0/2514082138 shutdown_connections 2024-03-08T21:06:23.420 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.438+0000 7fd7c35fa740 1 --2- 172.21.15.107:0/2514082138 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x5618ebe9b0f0 0x5618ebe9d4e0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:06:23.420 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.438+0000 7fd7c35fa740 1 --2- 172.21.15.107:0/2514082138 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5618ebe8b370 0x5618ebe88710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:06:23.420 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.438+0000 7fd7c35fa740 1 --2- 172.21.15.107:0/2514082138 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5618ebe825a0 0x5618ebe88100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:06:23.420 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.438+0000 7fd7c35fa740 1 --2- 172.21.15.107:0/2514082138 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5618ebe81c80 0x5618ebe8ad70 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:06:23.420 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.438+0000 7fd7c35fa740 1 -- 172.21.15.107:0/2514082138 >> 172.21.15.107:0/2514082138 conn(0x5618ebe7c5a0 msgr2=0x5618ebe7e2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:06:23.420 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.438+0000 7fd79b400640 0 client.15945 ms_handle_reset on v2:172.21.15.149:6838/55290596 2024-03-08T21:06:23.420 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.438+0000 7fd7c35fa740 1 -- 172.21.15.107:0/2514082138 shutdown_connections 2024-03-08T21:06:23.421 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.438+0000 7fd7c35fa740 1 -- 172.21.15.107:0/2514082138 wait complete. 2024-03-08T21:06:23.421 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.438+0000 7fd7c35fa740 20 client.15945 trim_cache size 0 max 16384 2024-03-08T21:06:23.438 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.458+0000 7fd7c35fa740 1 Processor -- start 2024-03-08T21:06:23.439 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.458+0000 7fd7c35fa740 1 -- start start 2024-03-08T21:06:23.439 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.458+0000 7fd7c35fa740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5618ebe81c80 0x5618ebe7e100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:06:23.439 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.458+0000 7fd7c35fa740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5618ebe825a0 0x5618ebe8a830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:06:23.439 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.458+0000 7fd7c35fa740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5618ebe8b370 0x5618ebe8ad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:06:23.439 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.458+0000 7fd7c35fa740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7bc1aedf0 con 0x5618ebe825a0 2024-03-08T21:06:23.439 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.458+0000 7fd7c35fa740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7bc1b0770 con 0x5618ebe8b370 2024-03-08T21:06:23.439 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.458+0000 7fd7c35fa740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7bc1af450 con 0x5618ebe81c80 2024-03-08T21:06:23.441 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.459+0000 7fd7b9400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5618ebe825a0 0x5618ebe8a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:06:23.441 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.459+0000 7fd7b9400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5618ebe825a0 0x5618ebe8a830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:54664/0 (socket says 172.21.15.107:54664) 2024-03-08T21:06:23.441 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.459+0000 7fd7b9400640 1 -- 172.21.15.107:0/3524991390 learned_addr learned my addr 172.21.15.107:0/3524991390 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:06:23.441 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.459+0000 7fd7ba800640 1 --2- 172.21.15.107:0/3524991390 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5618ebe8b370 0x5618ebe8ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:06:23.441 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.459+0000 7fd7ba800640 1 --2- 172.21.15.107:0/3524991390 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5618ebe81c80 0x5618ebe7e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:06:23.441 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.459+0000 7fd7b9400640 1 -- 172.21.15.107:0/3524991390 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5618ebe81c80 msgr2=0x5618ebe7e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:06:23.442 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.459+0000 7fd7b9400640 1 --2- 172.21.15.107:0/3524991390 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5618ebe81c80 0x5618ebe7e100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:06:23.442 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.459+0000 7fd7b9400640 1 -- 172.21.15.107:0/3524991390 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5618ebe8b370 msgr2=0x5618ebe8ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:06:23.442 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.459+0000 7fd7b9400640 1 --2- 172.21.15.107:0/3524991390 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5618ebe8b370 0x5618ebe8ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:06:23.442 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.459+0000 7fd7b9400640 1 -- 172.21.15.107:0/3524991390 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5618ebe823d0 con 0x5618ebe825a0 2024-03-08T21:06:23.442 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.460+0000 7fd7b9400640 1 --2- 172.21.15.107:0/3524991390 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5618ebe825a0 0x5618ebe8a830 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fd7bc19b1b0 tx=0x7fd7a800d6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b63ad108c602813 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:06:23.442 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.460+0000 7fd7a6600640 1 -- 172.21.15.107:0/3524991390 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7a8012980 con 0x5618ebe825a0 2024-03-08T21:06:23.442 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.460+0000 7fd7a6600640 1 -- 172.21.15.107:0/3524991390 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fd7a801c030 con 0x5618ebe825a0 2024-03-08T21:06:23.442 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.460+0000 7fd7a6600640 1 -- 172.21.15.107:0/3524991390 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7a8010690 con 0x5618ebe825a0 2024-03-08T21:06:23.442 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.461+0000 7fd7c35fa740 1 -- 172.21.15.107:0/3524991390 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5618ebe825a0 msgr2=0x5618ebe8a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:06:23.442 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.461+0000 7fd7c35fa740 1 --2- 172.21.15.107:0/3524991390 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5618ebe825a0 0x5618ebe8a830 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fd7bc19b1b0 tx=0x7fd7a800d6b0 comp rx=0 tx=0).stop 2024-03-08T21:06:23.442 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.461+0000 7fd7c35fa740 1 -- 172.21.15.107:0/3524991390 shutdown_connections 2024-03-08T21:06:23.442 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.461+0000 7fd7c35fa740 1 --2- 172.21.15.107:0/3524991390 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5618ebe8b370 0x5618ebe8ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:06:23.442 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.461+0000 7fd7c35fa740 1 --2- 172.21.15.107:0/3524991390 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5618ebe825a0 0x5618ebe8a830 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:06:23.442 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.461+0000 7fd7c35fa740 1 --2- 172.21.15.107:0/3524991390 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5618ebe81c80 0x5618ebe7e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:06:23.443 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.461+0000 7fd7c35fa740 1 -- 172.21.15.107:0/3524991390 >> 172.21.15.107:0/3524991390 conn(0x5618ebe7c5a0 msgr2=0x5618ebe7d9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:06:23.443 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.461+0000 7fd7c35fa740 1 -- 172.21.15.107:0/3524991390 shutdown_connections 2024-03-08T21:06:23.443 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.461+0000 7fd7c35fa740 1 -- 172.21.15.107:0/3524991390 wait complete. 2024-03-08T21:06:23.443 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.461+0000 7fd7c35fa740 1 -- start start 2024-03-08T21:06:23.445 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.461+0000 7fd7c35fa740 1 Processor -- start 2024-03-08T21:06:23.445 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.462+0000 7fd7c35fa740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5618ebe81c80 0x5618ebe92030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:06:23.445 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.462+0000 7fd7c35fa740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5618ebe825a0 0x5618ebe92570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:06:23.445 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.462+0000 7fd7c35fa740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5618ebe8b370 0x5618ebe904d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:06:23.445 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.462+0000 7fd7c35fa740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7bc1b0110 con 0x5618ebe81c80 2024-03-08T21:06:23.446 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.462+0000 7fd7c35fa740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7bc1afab0 con 0x5618ebe825a0 2024-03-08T21:06:23.446 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.462+0000 7fd7c35fa740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7bc1ad090 con 0x5618ebe8b370 2024-03-08T21:06:23.446 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.462+0000 7fd7c35fa740 10 client.0 _ll_register_callbacks cb 0x7fd7b4007360 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T21:06:23.446 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.462+0000 7fd7ba800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5618ebe81c80 0x5618ebe92030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:06:23.446 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.462+0000 7fd7ba800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5618ebe81c80 0x5618ebe92030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:54670/0 (socket says 172.21.15.107:54670) 2024-03-08T21:06:23.446 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.462+0000 7fd7ba800640 1 -- 172.21.15.107:0/1377800157 learned_addr learned my addr 172.21.15.107:0/1377800157 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:06:23.446 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.463+0000 7fd7ba800640 1 -- 172.21.15.107:0/1377800157 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5618ebe8b370 msgr2=0x5618ebe904d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:06:23.446 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.463+0000 7fd7ba800640 1 --2- 172.21.15.107:0/1377800157 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5618ebe8b370 0x5618ebe904d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:06:23.446 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.463+0000 7fd7ba800640 1 -- 172.21.15.107:0/1377800157 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5618ebe825a0 msgr2=0x5618ebe92570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:06:23.446 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.463+0000 7fd7ba800640 1 --2- 172.21.15.107:0/1377800157 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5618ebe825a0 0x5618ebe92570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:06:23.446 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.463+0000 7fd7ba800640 1 -- 172.21.15.107:0/1377800157 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5618ebe90c60 con 0x5618ebe81c80 2024-03-08T21:06:23.446 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.463+0000 7fd7ba800640 1 --2- 172.21.15.107:0/1377800157 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5618ebe81c80 0x5618ebe92030 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fd7b4004460 tx=0x7fd7b4007440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc080ea1b9d49304 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:06:23.447 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.463+0000 7fd79b400640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-03-08T21:06:23.447 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.463+0000 7fd79b400640 1 -- 172.21.15.107:0/1377800157 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7b4033040 con 0x5618ebe81c80 2024-03-08T21:06:23.447 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.463+0000 7fd79b400640 1 -- 172.21.15.107:0/1377800157 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fd7b4007660 con 0x5618ebe81c80 2024-03-08T21:06:23.447 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.463+0000 7fd7c35fa740 1 -- 172.21.15.107:0/1377800157 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd7a801c030 con 0x5618ebe81c80 2024-03-08T21:06:23.447 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.464+0000 7fd79b400640 1 -- 172.21.15.107:0/1377800157 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd7b4014040 con 0x5618ebe81c80 2024-03-08T21:06:23.447 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.464+0000 7fd7c35fa740 10 client.15951 fetch_fsmap learned FSMap version 34 2024-03-08T21:06:23.447 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.464+0000 7fd7c35fa740 1 -- 172.21.15.107:0/1377800157 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x5618ebef1db0 con 0x5618ebe81c80 2024-03-08T21:06:23.447 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.464+0000 7fd79b400640 1 -- 172.21.15.107:0/1377800157 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fd7a801c030 con 0x5618ebe81c80 2024-03-08T21:06:23.447 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.464+0000 7fd79b400640 1 -- 172.21.15.107:0/1377800157 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 34) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fd7b40040a0 con 0x5618ebe81c80 2024-03-08T21:06:23.447 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.464+0000 7fd7c35fa740 10 client.15951 fetch_fsmap finished waiting for FSMap version 34 2024-03-08T21:06:23.447 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.464+0000 7fd7c35fa740 10 client.15951 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-08T21:06:23.447 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.464+0000 7fd7c35fa740 20 client.15951 populate_metadata read hostname 'smithi107' 2024-03-08T21:06:23.447 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.464+0000 7fd7c35fa740 1 --2- 172.21.15.107:0/1377800157 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x5618ebef2460 0x5618ebef48a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:06:23.447 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.465+0000 7fd7c35fa740 4 client.15951 mds_command: new command op to 34475 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-08T21:06:23.448 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.465+0000 7fd7c35fa740 1 -- 172.21.15.107:0/1377800157 --> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5618ebdecc60 con 0x5618ebef2460 2024-03-08T21:06:23.448 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.465+0000 7fd7b9400640 1 --2- 172.21.15.107:0/1377800157 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x5618ebef2460 0x5618ebef48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:06:23.448 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.466+0000 7fd79b400640 10 client.15951 ms_handle_connect on v2:172.21.15.149:6838/55290596 2024-03-08T21:06:23.448 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.466+0000 7fd7b9400640 1 --2- 172.21.15.107:0/1377800157 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x5618ebef2460 0x5618ebef48a0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e6ccb3a9f71fac05 server_cookie=506a00aa7cfcc9de in_seq=0 out_seq=0 2024-03-08T21:06:23.457 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.477+0000 7fd79b400640 1 -- 172.21.15.107:0/1377800157 <== mds.0 v2:172.21.15.149:6838/55290596 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5618ebdecc60 con 0x5618ebef2460 2024-03-08T21:06:23.457 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.477+0000 7fd79b400640 10 client.15951 handle_command_reply: tid=0 2024-03-08T21:06:23.458 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.477+0000 7fd7c35fa740 1 client.15951 shutdown 2024-03-08T21:06:23.458 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.477+0000 7fd7c35fa740 1 -- 172.21.15.107:0/1377800157 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5618ebe81c80 msgr2=0x5618ebe92030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:06:23.458 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.477+0000 7fd7c35fa740 1 --2- 172.21.15.107:0/1377800157 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5618ebe81c80 0x5618ebe92030 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fd7b4004460 tx=0x7fd7b4007440 comp rx=0 tx=0).stop 2024-03-08T21:06:23.458 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.477+0000 7fd7c35fa740 1 -- 172.21.15.107:0/1377800157 shutdown_connections 2024-03-08T21:06:23.458 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.477+0000 7fd7c35fa740 1 --2- 172.21.15.107:0/1377800157 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5618ebe8b370 0x5618ebe904d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:06:23.458 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.477+0000 7fd7c35fa740 1 --2- 172.21.15.107:0/1377800157 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x5618ebef2460 0x5618ebef48a0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:06:23.459 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.477+0000 7fd7c35fa740 1 --2- 172.21.15.107:0/1377800157 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5618ebe825a0 0x5618ebe92570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:06:23.459 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.477+0000 7fd7c35fa740 1 --2- 172.21.15.107:0/1377800157 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5618ebe81c80 0x5618ebe92030 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:06:23.459 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.477+0000 7fd79b400640 0 client.15951 ms_handle_reset on v2:172.21.15.149:6838/55290596 2024-03-08T21:06:23.459 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.477+0000 7fd7c35fa740 1 -- 172.21.15.107:0/1377800157 >> 172.21.15.107:0/1377800157 conn(0x5618ebe7c5a0 msgr2=0x5618ebe87ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:06:23.459 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.477+0000 7fd7c35fa740 1 -- 172.21.15.107:0/1377800157 shutdown_connections 2024-03-08T21:06:23.459 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.477+0000 7fd7c35fa740 1 -- 172.21.15.107:0/1377800157 wait complete. 2024-03-08T21:06:23.459 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.477+0000 7fd7c35fa740 20 client.15951 trim_cache size 0 max 16384 2024-03-08T21:06:23.459 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:{ 2024-03-08T21:06:23.459 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "message": "", 2024-03-08T21:06:23.460 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "return_code": 0 2024-03-08T21:06:23.460 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:} 2024-03-08T21:06:23.460 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.478+0000 7fd7c0a00640 1 -- 172.21.15.107:0/3241451875 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7fd7a00a2a90 msgr2=0x7fd7a00a4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:06:23.460 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.478+0000 7fd7c0a00640 1 --2- 172.21.15.107:0/3241451875 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7fd7a00a2a90 0x7fd7a00a4f50 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fd7ac0eea90 tx=0x7fd7ac009260 comp rx=0 tx=0).stop 2024-03-08T21:06:23.461 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.478+0000 7fd7c0a00640 1 -- 172.21.15.107:0/3241451875 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd7bc101510 msgr2=0x7fd7bc19f810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:06:23.461 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.478+0000 7fd7c0a00640 1 --2- 172.21.15.107:0/3241451875 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd7bc101510 0x7fd7bc19f810 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fd7ac005ab0 tx=0x7fd7ac02ca90 comp rx=0 tx=0).stop 2024-03-08T21:06:23.461 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.478+0000 7fd7c0a00640 1 -- 172.21.15.107:0/3241451875 shutdown_connections 2024-03-08T21:06:23.461 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.478+0000 7fd7c0a00640 1 --2- 172.21.15.107:0/3241451875 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7fd7a00a2a90 0x7fd7a00a4f50 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:06:23.461 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.478+0000 7fd7c0a00640 1 --2- 172.21.15.107:0/3241451875 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd7bc1999c0 0x7fd7bc199e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:06:23.461 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.478+0000 7fd7c0a00640 1 --2- 172.21.15.107:0/3241451875 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd7bc1065e0 0x7fd7bc19fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:06:23.461 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.478+0000 7fd7c0a00640 1 --2- 172.21.15.107:0/3241451875 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd7bc101510 0x7fd7bc19f810 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:06:23.461 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.478+0000 7fd7c0a00640 1 -- 172.21.15.107:0/3241451875 >> 172.21.15.107:0/3241451875 conn(0x7fd7bc0fcdb0 msgr2=0x7fd7bc10caf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:06:23.461 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.479+0000 7fd7c0a00640 1 -- 172.21.15.107:0/3241451875 shutdown_connections 2024-03-08T21:06:23.461 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:23.479+0000 7fd7c0a00640 1 -- 172.21.15.107:0/3241451875 wait complete. 2024-03-08T21:06:25.964 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:06:25 smithi107 ceph-mon[31381]: pgmap v182: 97 pgs: 97 active+clean; 4.0 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 3.1 KiB/s wr, 15 op/s 2024-03-08T21:06:26.011 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:06:25 smithi149 ceph-mon[35042]: pgmap v182: 97 pgs: 97 active+clean; 4.0 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 3.1 KiB/s wr, 15 op/s 2024-03-08T21:06:26.125 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:06:25 smithi204 ceph-mon[34935]: pgmap v182: 97 pgs: 97 active+clean; 4.0 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 3.1 KiB/s wr, 15 op/s 2024-03-08T21:06:27.964 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:06:27 smithi107 ceph-mon[31381]: pgmap v183: 97 pgs: 97 active+clean; 4.0 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:06:28.011 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:06:27 smithi149 ceph-mon[35042]: pgmap v183: 97 pgs: 97 active+clean; 4.0 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:06:28.125 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:06:27 smithi204 ceph-mon[34935]: pgmap v183: 97 pgs: 97 active+clean; 4.0 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:06:28.964 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:06:28 smithi107 ceph-mon[31381]: pgmap v184: 97 pgs: 97 active+clean; 4.0 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-08T21:06:29.011 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:06:28 smithi149 ceph-mon[35042]: pgmap v184: 97 pgs: 97 active+clean; 4.0 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-08T21:06:29.125 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:06:28 smithi204 ceph-mon[34935]: pgmap v184: 97 pgs: 97 active+clean; 4.0 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-08T21:06:31.964 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:06:31 smithi107 ceph-mon[31381]: pgmap v185: 97 pgs: 97 active+clean; 4.0 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:06:32.011 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:06:31 smithi149 ceph-mon[35042]: pgmap v185: 97 pgs: 97 active+clean; 4.0 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:06:32.125 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:06:31 smithi204 ceph-mon[34935]: pgmap v185: 97 pgs: 97 active+clean; 4.0 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:06:33.928 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:06:33 smithi204 ceph-mon[34935]: pgmap v186: 97 pgs: 97 active+clean; 4.0 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:06:33.964 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:06:33 smithi107 ceph-mon[31381]: pgmap v186: 97 pgs: 97 active+clean; 4.0 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:06:34.011 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:06:33 smithi149 ceph-mon[35042]: pgmap v186: 97 pgs: 97 active+clean; 4.0 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:06:34.964 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:06:34 smithi107 ceph-mon[31381]: pgmap v187: 97 pgs: 97 active+clean; 4.0 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-08T21:06:35.011 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:06:34 smithi149 ceph-mon[35042]: pgmap v187: 97 pgs: 97 active+clean; 4.0 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-08T21:06:35.125 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:06:34 smithi204 ceph-mon[34935]: pgmap v187: 97 pgs: 97 active+clean; 4.0 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-08T21:06:36.617 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.638+0000 7f3190800640 1 -- 172.21.15.107:0/3806280816 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f31881021d0 msgr2=0x7f3188068970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:06:36.618 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.638+0000 7f3190800640 1 --2- 172.21.15.107:0/3806280816 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f31881021d0 0x7f3188068970 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f317c009900 tx=0x7f317c02bcc0 comp rx=0 tx=0).stop 2024-03-08T21:06:36.618 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.638+0000 7f3190800640 1 -- 172.21.15.107:0/3806280816 shutdown_connections 2024-03-08T21:06:36.618 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.638+0000 7f3190800640 1 --2- 172.21.15.107:0/3806280816 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f318810db40 0x7f318810ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:06:36.618 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.638+0000 7f3190800640 1 --2- 172.21.15.107:0/3806280816 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3188068eb0 0x7f31880692f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:06:36.618 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.638+0000 7f3190800640 1 --2- 172.21.15.107:0/3806280816 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f31881021d0 0x7f3188068970 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:06:36.618 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.638+0000 7f3190800640 1 -- 172.21.15.107:0/3806280816 >> 172.21.15.107:0/3806280816 conn(0x7f3188077040 msgr2=0x7f3188077450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:06:36.618 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.638+0000 7f3190800640 1 -- 172.21.15.107:0/3806280816 shutdown_connections 2024-03-08T21:06:36.618 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.638+0000 7f3190800640 1 -- 172.21.15.107:0/3806280816 wait complete. 2024-03-08T21:06:36.620 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.639+0000 7f3190800640 1 Processor -- start 2024-03-08T21:06:36.620 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.640+0000 7f3190800640 1 -- start start 2024-03-08T21:06:36.620 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.640+0000 7f3190800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3188068eb0 0x7f318819f900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:06:36.620 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.640+0000 7f3190800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f31881021d0 0x7f318819fe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:06:36.620 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.640+0000 7f3190800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f318810db40 0x7f3188199b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:06:36.620 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.640+0000 7f3190800640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3188113840 con 0x7f31881021d0 2024-03-08T21:06:36.620 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.640+0000 7f3190800640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31881136c0 con 0x7f318810db40 2024-03-08T21:06:36.620 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.640+0000 7f3190800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31881139c0 con 0x7f3188068eb0 2024-03-08T21:06:36.620 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.640+0000 7f318d400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f31881021d0 0x7f318819fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:06:36.620 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.640+0000 7f318d400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f31881021d0 0x7f318819fe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:43288/0 (socket says 172.21.15.107:43288) 2024-03-08T21:06:36.620 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.640+0000 7f318d400640 1 -- 172.21.15.107:0/2826158700 learned_addr learned my addr 172.21.15.107:0/2826158700 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:06:36.620 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.641+0000 7f318de00640 1 --2- 172.21.15.107:0/2826158700 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3188068eb0 0x7f318819f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:06:36.620 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.641+0000 7f318e800640 1 --2- 172.21.15.107:0/2826158700 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f318810db40 0x7f3188199b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:06:36.622 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.641+0000 7f318d400640 1 -- 172.21.15.107:0/2826158700 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3188068eb0 msgr2=0x7f318819f900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:06:36.622 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.641+0000 7f318d400640 1 --2- 172.21.15.107:0/2826158700 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3188068eb0 0x7f318819f900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:06:36.622 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.641+0000 7f318d400640 1 -- 172.21.15.107:0/2826158700 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f318810db40 msgr2=0x7f3188199b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:06:36.622 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.641+0000 7f318d400640 1 --2- 172.21.15.107:0/2826158700 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f318810db40 0x7f3188199b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:06:36.622 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.641+0000 7f318d400640 1 -- 172.21.15.107:0/2826158700 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f318819a280 con 0x7f31881021d0 2024-03-08T21:06:36.622 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.641+0000 7f318d400640 1 --2- 172.21.15.107:0/2826158700 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f31881021d0 0x7f318819fe40 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f317800d770 tx=0x7f317800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:06:36.622 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.642+0000 7f3182a00640 1 -- 172.21.15.107:0/2826158700 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3178017070 con 0x7f31881021d0 2024-03-08T21:06:36.622 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.642+0000 7f3182a00640 1 -- 172.21.15.107:0/2826158700 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f31780054d0 con 0x7f31881021d0 2024-03-08T21:06:36.622 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.642+0000 7f3190800640 1 -- 172.21.15.107:0/2826158700 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f318819a570 con 0x7f31881021d0 2024-03-08T21:06:36.622 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.642+0000 7f3182a00640 1 -- 172.21.15.107:0/2826158700 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3178004cf0 con 0x7f31881021d0 2024-03-08T21:06:36.622 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.642+0000 7f3190800640 1 -- 172.21.15.107:0/2826158700 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31881a67c0 con 0x7f31881021d0 2024-03-08T21:06:36.624 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.645+0000 7f3182a00640 1 -- 172.21.15.107:0/2826158700 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) v1 ==== 137422+0+0 (secure 0 0 0) 0x7f3178004e90 con 0x7f31881021d0 2024-03-08T21:06:36.625 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.646+0000 7f3193625740 1 Processor -- start 2024-03-08T21:06:36.625 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.646+0000 7f3193625740 1 -- start start 2024-03-08T21:06:36.627 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.647+0000 7f3193625740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5628f24f3c80 0x5628f24f4060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:06:36.627 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.647+0000 7f3193625740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5628f24f45a0 0x5628f24fce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:06:36.627 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.647+0000 7f3193625740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5628f24fd370 0x5628f24ff760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:06:36.627 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.647+0000 7f3193625740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628f24573d0 con 0x5628f24fd370 2024-03-08T21:06:36.627 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.647+0000 7f3193625740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628f24006b0 con 0x5628f24f45a0 2024-03-08T21:06:36.627 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.647+0000 7f3193625740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628f2453470 con 0x5628f24f3c80 2024-03-08T21:06:36.627 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.647+0000 7f318d400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5628f24fd370 0x5628f24ff760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:06:36.627 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.647+0000 7f318d400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5628f24fd370 0x5628f24ff760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:43300/0 (socket says 172.21.15.107:43300) 2024-03-08T21:06:36.627 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.647+0000 7f318d400640 1 -- 172.21.15.107:0/1783440252 learned_addr learned my addr 172.21.15.107:0/1783440252 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:06:36.627 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.647+0000 7f318e800640 1 --2- 172.21.15.107:0/1783440252 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5628f24f3c80 0x5628f24f4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:06:36.628 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.647+0000 7f318de00640 1 --2- 172.21.15.107:0/1783440252 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5628f24f45a0 0x5628f24fce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:06:36.628 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.647+0000 7f3182a00640 1 --2- 172.21.15.107:0/2826158700 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f31700a2a90 0x7f31700a4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:06:36.628 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.648+0000 7f3182a00640 1 -- 172.21.15.107:0/2826158700 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f31780ce860 con 0x7f31881021d0 2024-03-08T21:06:36.628 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.648+0000 7f318d400640 1 -- 172.21.15.107:0/1783440252 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5628f24f3c80 msgr2=0x5628f24f4060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:06:36.628 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.648+0000 7f318d400640 1 --2- 172.21.15.107:0/1783440252 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5628f24f3c80 0x5628f24f4060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:06:36.628 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.648+0000 7f318d400640 1 -- 172.21.15.107:0/1783440252 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5628f24f45a0 msgr2=0x5628f24fce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:06:36.628 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.648+0000 7f318d400640 1 --2- 172.21.15.107:0/1783440252 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5628f24f45a0 0x5628f24fce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:06:36.628 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.648+0000 7f318d400640 1 -- 172.21.15.107:0/1783440252 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5628f2426b50 con 0x5628f24fd370 2024-03-08T21:06:36.628 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.648+0000 7f318e800640 1 --2- 172.21.15.107:0/2826158700 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f31700a2a90 0x7f31700a4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:06:36.632 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.648+0000 7f318d400640 1 --2- 172.21.15.107:0/1783440252 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5628f24fd370 0x5628f24ff760 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f31780cbf40 tx=0x7f3178005680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a0d4513d22f9028 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:06:36.632 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.648+0000 7f3177000640 1 -- 172.21.15.107:0/1783440252 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3178020070 con 0x5628f24fd370 2024-03-08T21:06:36.632 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.649+0000 7f3177000640 1 -- 172.21.15.107:0/1783440252 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f31780d0e50 con 0x5628f24fd370 2024-03-08T21:06:36.632 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.649+0000 7f3177000640 1 -- 172.21.15.107:0/1783440252 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f317809f000 con 0x5628f24fd370 2024-03-08T21:06:36.632 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.649+0000 7f3193625740 1 -- 172.21.15.107:0/1783440252 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5628f24fd370 msgr2=0x5628f24ff760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:06:36.632 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.649+0000 7f3193625740 1 --2- 172.21.15.107:0/1783440252 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5628f24fd370 0x5628f24ff760 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f31780cbf40 tx=0x7f3178005680 comp rx=0 tx=0).stop 2024-03-08T21:06:36.632 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.649+0000 7f318e800640 1 --2- 172.21.15.107:0/2826158700 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f31700a2a90 0x7f31700a4f50 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f318806a810 tx=0x7f318400c040 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:06:36.633 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.649+0000 7f3193625740 1 -- 172.21.15.107:0/1783440252 shutdown_connections 2024-03-08T21:06:36.633 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.649+0000 7f3193625740 1 --2- 172.21.15.107:0/1783440252 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5628f24fd370 0x5628f24ff760 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:06:36.633 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.649+0000 7f3193625740 1 --2- 172.21.15.107:0/1783440252 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5628f24f45a0 0x5628f24fce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:06:36.633 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.649+0000 7f3193625740 1 --2- 172.21.15.107:0/1783440252 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5628f24f3c80 0x5628f24f4060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:06:36.633 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.650+0000 7f3193625740 1 -- 172.21.15.107:0/1783440252 >> 172.21.15.107:0/1783440252 conn(0x5628f24ee5a0 msgr2=0x5628f24ee980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:06:36.633 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.650+0000 7f3193625740 1 -- 172.21.15.107:0/1783440252 shutdown_connections 2024-03-08T21:06:36.633 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.650+0000 7f3193625740 1 -- 172.21.15.107:0/1783440252 wait complete. 2024-03-08T21:06:36.633 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.650+0000 7f3193625740 1 -- start start 2024-03-08T21:06:36.633 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.650+0000 7f3193625740 1 Processor -- start 2024-03-08T21:06:36.633 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.651+0000 7f3193625740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5628f24f3c80 0x5628f24fccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:06:36.633 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.651+0000 7f3193625740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5628f24f45a0 0x5628f24fa040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:06:36.633 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.651+0000 7f3193625740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5628f24fd370 0x5628f24fa580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:06:36.633 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.651+0000 7f3193625740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628f24fff40 con 0x5628f24f3c80 2024-03-08T21:06:36.634 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.651+0000 7f3193625740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628f24139f0 con 0x5628f24fd370 2024-03-08T21:06:36.634 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.651+0000 7f3193625740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628f25000c0 con 0x5628f24f45a0 2024-03-08T21:06:36.634 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.651+0000 7f3193625740 10 client.0 _ll_register_callbacks cb 0x5628f216ba00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T21:06:36.634 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.651+0000 7f318de00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5628f24f3c80 0x5628f24fccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:06:36.634 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.651+0000 7f318de00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5628f24f3c80 0x5628f24fccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:43306/0 (socket says 172.21.15.107:43306) 2024-03-08T21:06:36.634 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.651+0000 7f318de00640 1 -- 172.21.15.107:0/4211720710 learned_addr learned my addr 172.21.15.107:0/4211720710 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:06:36.634 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.651+0000 7f318d400640 1 --2- 172.21.15.107:0/4211720710 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5628f24f45a0 0x5628f24fa040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:06:36.634 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.651+0000 7f318e800640 1 --2- 172.21.15.107:0/4211720710 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5628f24fd370 0x5628f24fa580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:06:36.634 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.652+0000 7f318de00640 1 -- 172.21.15.107:0/4211720710 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5628f24f45a0 msgr2=0x5628f24fa040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:06:36.634 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.652+0000 7f318de00640 1 --2- 172.21.15.107:0/4211720710 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5628f24f45a0 0x5628f24fa040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:06:36.634 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.652+0000 7f318de00640 1 -- 172.21.15.107:0/4211720710 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5628f24fd370 msgr2=0x5628f24fa580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:06:36.634 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.652+0000 7f318de00640 1 --2- 172.21.15.107:0/4211720710 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5628f24fd370 0x5628f24fa580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:06:36.634 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.652+0000 7f318de00640 1 -- 172.21.15.107:0/4211720710 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5628f216c210 con 0x5628f24f3c80 2024-03-08T21:06:36.635 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.652+0000 7f318de00640 1 --2- 172.21.15.107:0/4211720710 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5628f24f3c80 0x5628f24fccb0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f317c006010 tx=0x7f317c02dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dc196a98a0f9fda server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:06:36.635 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.652+0000 7f316b400640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-03-08T21:06:36.635 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.652+0000 7f316b400640 1 -- 172.21.15.107:0/4211720710 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f317c044030 con 0x5628f24f3c80 2024-03-08T21:06:36.635 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.652+0000 7f3193625740 1 -- 172.21.15.107:0/4211720710 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f31780d0e50 con 0x5628f24f3c80 2024-03-08T21:06:36.635 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.652+0000 7f316b400640 1 -- 172.21.15.107:0/4211720710 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f317c03a040 con 0x5628f24f3c80 2024-03-08T21:06:36.635 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.653+0000 7f316b400640 1 -- 172.21.15.107:0/4211720710 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f317c005250 con 0x5628f24f3c80 2024-03-08T21:06:36.635 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.653+0000 7f3193625740 10 client.15963 fetch_fsmap learned FSMap version 34 2024-03-08T21:06:36.635 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.653+0000 7f3193625740 1 -- 172.21.15.107:0/4211720710 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x5628f2238620 con 0x5628f24f3c80 2024-03-08T21:06:36.635 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.653+0000 7f316b400640 1 -- 172.21.15.107:0/4211720710 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f31780d0e50 con 0x5628f24f3c80 2024-03-08T21:06:36.637 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.653+0000 7f316b400640 1 -- 172.21.15.107:0/4211720710 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 34) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f317c0035f0 con 0x5628f24f3c80 2024-03-08T21:06:36.637 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.653+0000 7f3193625740 10 client.15963 fetch_fsmap finished waiting for FSMap version 34 2024-03-08T21:06:36.637 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.653+0000 7f3193625740 10 client.15963 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-08T21:06:36.637 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.653+0000 7f3193625740 20 client.15963 populate_metadata read hostname 'smithi107' 2024-03-08T21:06:36.638 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.653+0000 7f3193625740 1 --2- 172.21.15.107:0/4211720710 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x5628f250d1d0 0x5628f250f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:06:36.638 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.653+0000 7f3193625740 4 client.15963 mds_command: new command op to 34475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-08T21:06:36.638 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.653+0000 7f3193625740 1 -- 172.21.15.107:0/4211720710 --> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5628f2464910 con 0x5628f250d1d0 2024-03-08T21:06:36.638 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.654+0000 7f318d400640 1 --2- 172.21.15.107:0/4211720710 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x5628f250d1d0 0x5628f250f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:06:36.638 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.655+0000 7f318d400640 1 --2- 172.21.15.107:0/4211720710 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x5628f250d1d0 0x5628f250f5c0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=65002a53c7b6a67b server_cookie=8a60dcd991057cdb in_seq=0 out_seq=0 2024-03-08T21:06:36.638 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.655+0000 7f316b400640 10 client.15963 ms_handle_connect on v2:172.21.15.149:6838/55290596 2024-03-08T21:06:36.638 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.656+0000 7f316b400640 1 -- 172.21.15.107:0/4211720710 <== mds.0 v2:172.21.15.149:6838/55290596 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5628f2464910 con 0x5628f250d1d0 2024-03-08T21:06:36.638 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.656+0000 7f316b400640 10 client.15963 handle_command_reply: tid=0 2024-03-08T21:06:36.638 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.657+0000 7f3193625740 1 client.15963 shutdown 2024-03-08T21:06:36.638 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.657+0000 7f3193625740 1 -- 172.21.15.107:0/4211720710 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5628f24f3c80 msgr2=0x5628f24fccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:06:36.638 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.657+0000 7f3193625740 1 --2- 172.21.15.107:0/4211720710 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5628f24f3c80 0x5628f24fccb0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f317c006010 tx=0x7f317c02dab0 comp rx=0 tx=0).stop 2024-03-08T21:06:36.638 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.657+0000 7f3193625740 1 -- 172.21.15.107:0/4211720710 shutdown_connections 2024-03-08T21:06:36.639 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.657+0000 7f3193625740 1 --2- 172.21.15.107:0/4211720710 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x5628f250d1d0 0x5628f250f5c0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:06:36.639 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.657+0000 7f3193625740 1 --2- 172.21.15.107:0/4211720710 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5628f24fd370 0x5628f24fa580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:06:36.639 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.657+0000 7f3193625740 1 --2- 172.21.15.107:0/4211720710 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5628f24f45a0 0x5628f24fa040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:06:36.639 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.657+0000 7f3193625740 1 --2- 172.21.15.107:0/4211720710 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5628f24f3c80 0x5628f24fccb0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:06:36.639 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.657+0000 7f316b400640 0 client.15963 ms_handle_reset on v2:172.21.15.149:6838/55290596 2024-03-08T21:06:36.639 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.657+0000 7f3193625740 1 -- 172.21.15.107:0/4211720710 >> 172.21.15.107:0/4211720710 conn(0x5628f24ee5a0 msgr2=0x5628f24f02f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:06:36.639 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.657+0000 7f3193625740 1 -- 172.21.15.107:0/4211720710 shutdown_connections 2024-03-08T21:06:36.639 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.657+0000 7f3193625740 1 -- 172.21.15.107:0/4211720710 wait complete. 2024-03-08T21:06:36.639 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.657+0000 7f3193625740 20 client.15963 trim_cache size 0 max 16384 2024-03-08T21:06:36.654 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.675+0000 7f3193625740 1 Processor -- start 2024-03-08T21:06:36.654 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.675+0000 7f3193625740 1 -- start start 2024-03-08T21:06:36.654 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.675+0000 7f3193625740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5628f24f3c80 0x5628f24fc770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:06:36.654 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.675+0000 7f3193625740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5628f24f45a0 0x5628f24fccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:06:36.654 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.675+0000 7f3193625740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5628f24fd370 0x5628f24f72c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:06:36.654 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.675+0000 7f3193625740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31881aecf0 con 0x5628f24f3c80 2024-03-08T21:06:36.654 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.675+0000 7f3193625740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31881b0670 con 0x5628f24fd370 2024-03-08T21:06:36.654 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.675+0000 7f3193625740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31881af350 con 0x5628f24f45a0 2024-03-08T21:06:36.656 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.675+0000 7f318de00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5628f24f3c80 0x5628f24fc770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:06:36.656 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.675+0000 7f318de00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5628f24f3c80 0x5628f24fc770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:43314/0 (socket says 172.21.15.107:43314) 2024-03-08T21:06:36.656 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.675+0000 7f318de00640 1 -- 172.21.15.107:0/3834991004 learned_addr learned my addr 172.21.15.107:0/3834991004 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:06:36.656 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.676+0000 7f318d400640 1 --2- 172.21.15.107:0/3834991004 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5628f24f45a0 0x5628f24fccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:06:36.656 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.676+0000 7f318e800640 1 --2- 172.21.15.107:0/3834991004 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5628f24fd370 0x5628f24f72c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:06:36.656 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.676+0000 7f318de00640 1 -- 172.21.15.107:0/3834991004 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5628f24f45a0 msgr2=0x5628f24fccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:06:36.656 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.676+0000 7f318de00640 1 --2- 172.21.15.107:0/3834991004 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5628f24f45a0 0x5628f24fccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:06:36.656 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.676+0000 7f318de00640 1 -- 172.21.15.107:0/3834991004 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5628f24fd370 msgr2=0x5628f24f72c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:06:36.656 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.676+0000 7f318de00640 1 --2- 172.21.15.107:0/3834991004 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5628f24fd370 0x5628f24f72c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:06:36.656 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.676+0000 7f318de00640 1 -- 172.21.15.107:0/3834991004 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5628f25147f0 con 0x5628f24f3c80 2024-03-08T21:06:36.656 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.676+0000 7f318de00640 1 --2- 172.21.15.107:0/3834991004 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5628f24f3c80 0x5628f24fc770 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f317c000c00 tx=0x7f317c004440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f49743e49b7b5a93 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:06:36.657 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.676+0000 7f3176600640 1 -- 172.21.15.107:0/3834991004 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f317c044070 con 0x5628f24f3c80 2024-03-08T21:06:36.657 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.677+0000 7f3176600640 1 -- 172.21.15.107:0/3834991004 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f317c003c00 con 0x5628f24f3c80 2024-03-08T21:06:36.657 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.677+0000 7f3176600640 1 -- 172.21.15.107:0/3834991004 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f317c004920 con 0x5628f24f3c80 2024-03-08T21:06:36.658 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.677+0000 7f3193625740 1 -- 172.21.15.107:0/3834991004 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5628f24f3c80 msgr2=0x5628f24fc770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:06:36.658 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.677+0000 7f3193625740 1 --2- 172.21.15.107:0/3834991004 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5628f24f3c80 0x5628f24fc770 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f317c000c00 tx=0x7f317c004440 comp rx=0 tx=0).stop 2024-03-08T21:06:36.659 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.677+0000 7f3193625740 1 -- 172.21.15.107:0/3834991004 shutdown_connections 2024-03-08T21:06:36.659 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.677+0000 7f3193625740 1 --2- 172.21.15.107:0/3834991004 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5628f24fd370 0x5628f24f72c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:06:36.659 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.677+0000 7f3193625740 1 --2- 172.21.15.107:0/3834991004 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5628f24f45a0 0x5628f24fccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:06:36.659 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.677+0000 7f3193625740 1 --2- 172.21.15.107:0/3834991004 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5628f24f3c80 0x5628f24fc770 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:06:36.659 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.677+0000 7f3193625740 1 -- 172.21.15.107:0/3834991004 >> 172.21.15.107:0/3834991004 conn(0x5628f24ee5a0 msgr2=0x5628f24f8910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:06:36.659 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.678+0000 7f3193625740 1 -- 172.21.15.107:0/3834991004 shutdown_connections 2024-03-08T21:06:36.659 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.678+0000 7f3193625740 1 -- 172.21.15.107:0/3834991004 wait complete. 2024-03-08T21:06:36.659 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.678+0000 7f3193625740 1 -- start start 2024-03-08T21:06:36.659 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.678+0000 7f3193625740 1 Processor -- start 2024-03-08T21:06:36.659 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.679+0000 7f3193625740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5628f24f3c80 0x5628f2504290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:06:36.659 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.679+0000 7f3193625740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5628f24f45a0 0x5628f25047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:06:36.659 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.679+0000 7f3193625740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5628f24fd370 0x5628f25028a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:06:36.659 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.679+0000 7f3193625740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31881b0010 con 0x5628f24fd370 2024-03-08T21:06:36.660 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.679+0000 7f3193625740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31881af9b0 con 0x5628f24f45a0 2024-03-08T21:06:36.660 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.679+0000 7f3193625740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31881acf90 con 0x5628f24f3c80 2024-03-08T21:06:36.660 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.679+0000 7f3193625740 10 client.0 _ll_register_callbacks cb 0x7f31780971d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T21:06:36.660 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.679+0000 7f318e800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5628f24fd370 0x5628f25028a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:06:36.660 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.679+0000 7f318e800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5628f24fd370 0x5628f25028a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:43330/0 (socket says 172.21.15.107:43330) 2024-03-08T21:06:36.660 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.679+0000 7f318e800640 1 -- 172.21.15.107:0/365185343 learned_addr learned my addr 172.21.15.107:0/365185343 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:06:36.660 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.679+0000 7f318de00640 1 --2- 172.21.15.107:0/365185343 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5628f24f3c80 0x5628f2504290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:06:36.660 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.680+0000 7f318d400640 1 --2- 172.21.15.107:0/365185343 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5628f24f45a0 0x5628f25047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:06:36.661 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.680+0000 7f318e800640 1 -- 172.21.15.107:0/365185343 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5628f24f3c80 msgr2=0x5628f2504290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:06:36.661 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.680+0000 7f318e800640 1 --2- 172.21.15.107:0/365185343 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5628f24f3c80 0x5628f2504290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:06:36.661 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.680+0000 7f318e800640 1 -- 172.21.15.107:0/365185343 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5628f24f45a0 msgr2=0x5628f25047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:06:36.661 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.680+0000 7f318e800640 1 --2- 172.21.15.107:0/365185343 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5628f24f45a0 0x5628f25047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:06:36.661 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.680+0000 7f318e800640 1 -- 172.21.15.107:0/365185343 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5628f25030d0 con 0x5628f24fd370 2024-03-08T21:06:36.661 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.680+0000 7f318e800640 1 --2- 172.21.15.107:0/365185343 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5628f24fd370 0x5628f25028a0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f3184007e60 tx=0x7f31840096c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86c1380253858c58 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:06:36.661 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.680+0000 7f316b400640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-03-08T21:06:36.663 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.683+0000 7f316b400640 1 -- 172.21.15.107:0/365185343 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3184028060 con 0x5628f24fd370 2024-03-08T21:06:36.663 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.683+0000 7f316b400640 1 -- 172.21.15.107:0/365185343 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f3184017800 con 0x5628f24fd370 2024-03-08T21:06:36.663 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.683+0000 7f3193625740 1 -- 172.21.15.107:0/365185343 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f317c003c00 con 0x5628f24fd370 2024-03-08T21:06:36.663 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.683+0000 7f316b400640 1 -- 172.21.15.107:0/365185343 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3184018e10 con 0x5628f24fd370 2024-03-08T21:06:36.663 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.683+0000 7f3193625740 10 client.15969 fetch_fsmap learned FSMap version 34 2024-03-08T21:06:36.663 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.683+0000 7f3193625740 1 -- 172.21.15.107:0/365185343 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x5628f25643c0 con 0x5628f24fd370 2024-03-08T21:06:36.663 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.683+0000 7f316b400640 1 -- 172.21.15.107:0/365185343 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f317c003c00 con 0x5628f24fd370 2024-03-08T21:06:36.663 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.684+0000 7f316b400640 1 -- 172.21.15.107:0/365185343 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 34) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f3184023ba0 con 0x5628f24fd370 2024-03-08T21:06:36.663 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.684+0000 7f3193625740 10 client.15969 fetch_fsmap finished waiting for FSMap version 34 2024-03-08T21:06:36.663 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.684+0000 7f3193625740 10 client.15969 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-08T21:06:36.664 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.684+0000 7f3193625740 20 client.15969 populate_metadata read hostname 'smithi107' 2024-03-08T21:06:36.664 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.684+0000 7f3193625740 1 --2- 172.21.15.107:0/365185343 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x5628f2564b50 0x5628f2566f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:06:36.664 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.684+0000 7f3193625740 4 client.15969 mds_command: new command op to 34475 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-08T21:06:36.664 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.684+0000 7f3193625740 1 -- 172.21.15.107:0/365185343 --> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5628f245ec60 con 0x5628f2564b50 2024-03-08T21:06:36.664 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.684+0000 7f318de00640 1 --2- 172.21.15.107:0/365185343 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x5628f2564b50 0x5628f2566f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:06:36.664 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.685+0000 7f318de00640 1 --2- 172.21.15.107:0/365185343 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x5628f2564b50 0x5628f2566f70 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b0c82d2188f93b0 server_cookie=41107d05bebf3d6d in_seq=0 out_seq=0 2024-03-08T21:06:36.664 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.685+0000 7f316b400640 10 client.15969 ms_handle_connect on v2:172.21.15.149:6838/55290596 2024-03-08T21:06:36.675 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.695+0000 7f316b400640 1 -- 172.21.15.107:0/365185343 <== mds.0 v2:172.21.15.149:6838/55290596 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5628f245ec60 con 0x5628f2564b50 2024-03-08T21:06:36.675 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.695+0000 7f316b400640 10 client.15969 handle_command_reply: tid=0 2024-03-08T21:06:36.675 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.695+0000 7f3193625740 1 client.15969 shutdown 2024-03-08T21:06:36.675 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.695+0000 7f3193625740 1 -- 172.21.15.107:0/365185343 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5628f24fd370 msgr2=0x5628f25028a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:06:36.675 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.695+0000 7f3193625740 1 --2- 172.21.15.107:0/365185343 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5628f24fd370 0x5628f25028a0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f3184007e60 tx=0x7f31840096c0 comp rx=0 tx=0).stop 2024-03-08T21:06:36.675 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.695+0000 7f3193625740 1 -- 172.21.15.107:0/365185343 shutdown_connections 2024-03-08T21:06:36.675 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.695+0000 7f3193625740 1 --2- 172.21.15.107:0/365185343 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5628f24fd370 0x5628f25028a0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:06:36.675 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.695+0000 7f3193625740 1 --2- 172.21.15.107:0/365185343 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x5628f2564b50 0x5628f2566f70 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:06:36.675 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.695+0000 7f3193625740 1 --2- 172.21.15.107:0/365185343 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5628f24f45a0 0x5628f25047d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:06:36.675 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.695+0000 7f3193625740 1 --2- 172.21.15.107:0/365185343 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5628f24f3c80 0x5628f2504290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:06:36.675 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.695+0000 7f3193625740 1 -- 172.21.15.107:0/365185343 >> 172.21.15.107:0/365185343 conn(0x5628f24ee5a0 msgr2=0x5628f24f8da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:06:36.675 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.695+0000 7f316b400640 0 client.15969 ms_handle_reset on v2:172.21.15.149:6838/55290596 2024-03-08T21:06:36.676 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.695+0000 7f3193625740 1 -- 172.21.15.107:0/365185343 shutdown_connections 2024-03-08T21:06:36.676 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.695+0000 7f3193625740 1 -- 172.21.15.107:0/365185343 wait complete. 2024-03-08T21:06:36.676 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.695+0000 7f3193625740 20 client.15969 trim_cache size 0 max 16384 2024-03-08T21:06:36.676 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:{ 2024-03-08T21:06:36.676 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "message": "", 2024-03-08T21:06:36.676 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "return_code": 0 2024-03-08T21:06:36.676 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:} 2024-03-08T21:06:36.677 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.696+0000 7f3190800640 1 -- 172.21.15.107:0/2826158700 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f31700a2a90 msgr2=0x7f31700a4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:06:36.677 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.696+0000 7f3190800640 1 --2- 172.21.15.107:0/2826158700 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f31700a2a90 0x7f31700a4f50 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f318806a810 tx=0x7f318400c040 comp rx=0 tx=0).stop 2024-03-08T21:06:36.677 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.696+0000 7f3190800640 1 -- 172.21.15.107:0/2826158700 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f31881021d0 msgr2=0x7f318819fe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:06:36.677 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.696+0000 7f3190800640 1 --2- 172.21.15.107:0/2826158700 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f31881021d0 0x7f318819fe40 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f317800d770 tx=0x7f317800dc30 comp rx=0 tx=0).stop 2024-03-08T21:06:36.677 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.696+0000 7f3190800640 1 -- 172.21.15.107:0/2826158700 shutdown_connections 2024-03-08T21:06:36.677 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.696+0000 7f3190800640 1 --2- 172.21.15.107:0/2826158700 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f31700a2a90 0x7f31700a4f50 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:06:36.677 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.696+0000 7f3190800640 1 --2- 172.21.15.107:0/2826158700 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f318810db40 0x7f3188199b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:06:36.677 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.696+0000 7f3190800640 1 --2- 172.21.15.107:0/2826158700 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f31881021d0 0x7f318819fe40 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:06:36.677 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.696+0000 7f3190800640 1 --2- 172.21.15.107:0/2826158700 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3188068eb0 0x7f318819f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:06:36.677 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.696+0000 7f3190800640 1 -- 172.21.15.107:0/2826158700 >> 172.21.15.107:0/2826158700 conn(0x7f3188077040 msgr2=0x7f31880ff430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:06:36.677 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.697+0000 7f3190800640 1 -- 172.21.15.107:0/2826158700 shutdown_connections 2024-03-08T21:06:36.677 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:36.697+0000 7f3190800640 1 -- 172.21.15.107:0/2826158700 wait complete. 2024-03-08T21:06:37.964 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:06:37 smithi107 ceph-mon[31381]: pgmap v188: 97 pgs: 97 active+clean; 4.0 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-08T21:06:38.011 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:06:37 smithi149 ceph-mon[35042]: pgmap v188: 97 pgs: 97 active+clean; 4.0 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-08T21:06:38.124 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:06:37 smithi204 ceph-mon[34935]: pgmap v188: 97 pgs: 97 active+clean; 4.0 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-08T21:06:39.964 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:06:39 smithi107 ceph-mon[31381]: pgmap v189: 97 pgs: 97 active+clean; 4.0 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-08T21:06:40.011 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:06:39 smithi149 ceph-mon[35042]: pgmap v189: 97 pgs: 97 active+clean; 4.0 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-08T21:06:40.124 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:06:39 smithi204 ceph-mon[34935]: pgmap v189: 97 pgs: 97 active+clean; 4.0 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-08T21:06:40.963 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:06:40 smithi107 ceph-mon[31381]: pgmap v190: 97 pgs: 97 active+clean; 4.0 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:06:41.014 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:06:40 smithi149 ceph-mon[35042]: pgmap v190: 97 pgs: 97 active+clean; 4.0 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:06:41.124 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:06:40 smithi204 ceph-mon[34935]: pgmap v190: 97 pgs: 97 active+clean; 4.0 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:06:43.964 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:06:43 smithi107 ceph-mon[31381]: pgmap v191: 97 pgs: 97 active+clean; 4.0 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:06:44.011 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:06:43 smithi149 ceph-mon[35042]: pgmap v191: 97 pgs: 97 active+clean; 4.0 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:06:44.125 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:06:43 smithi204 ceph-mon[34935]: pgmap v191: 97 pgs: 97 active+clean; 4.0 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:06:44.963 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:06:44 smithi107 ceph-mon[31381]: pgmap v192: 97 pgs: 97 active+clean; 4.0 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-08T21:06:45.011 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:06:44 smithi149 ceph-mon[35042]: pgmap v192: 97 pgs: 97 active+clean; 4.0 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-08T21:06:45.125 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:06:44 smithi204 ceph-mon[34935]: pgmap v192: 97 pgs: 97 active+clean; 4.0 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-08T21:06:47.964 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:06:47 smithi107 ceph-mon[31381]: pgmap v193: 97 pgs: 97 active+clean; 4.0 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:06:48.011 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:06:47 smithi149 ceph-mon[35042]: pgmap v193: 97 pgs: 97 active+clean; 4.0 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:06:48.125 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:06:47 smithi204 ceph-mon[34935]: pgmap v193: 97 pgs: 97 active+clean; 4.0 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:06:48.964 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:06:48 smithi107 ceph-mon[31381]: pgmap v194: 97 pgs: 97 active+clean; 4.0 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-08T21:06:49.011 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:06:48 smithi149 ceph-mon[35042]: pgmap v194: 97 pgs: 97 active+clean; 4.0 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-08T21:06:49.124 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:06:48 smithi204 ceph-mon[34935]: pgmap v194: 97 pgs: 97 active+clean; 4.0 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-08T21:06:49.830 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.851+0000 7f32f3200640 1 -- 172.21.15.107:0/3036556241 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f32f4101340 msgr2=0x7f32f4101720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:06:49.831 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.851+0000 7f32f3200640 1 --2- 172.21.15.107:0/3036556241 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f32f4101340 0x7f32f4101720 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f32e4009ca0 tx=0x7f32e402c100 comp rx=0 tx=0).stop 2024-03-08T21:06:49.831 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.851+0000 7f32f3200640 1 -- 172.21.15.107:0/3036556241 shutdown_connections 2024-03-08T21:06:49.831 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.851+0000 7f32f3200640 1 --2- 172.21.15.107:0/3036556241 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f32f4106410 0x7f32f410dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:06:49.831 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.851+0000 7f32f3200640 1 --2- 172.21.15.107:0/3036556241 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f32f4101cf0 0x7f32f4105d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:06:49.831 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.851+0000 7f32f3200640 1 --2- 172.21.15.107:0/3036556241 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f32f4101340 0x7f32f4101720 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:06:49.831 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.851+0000 7f32f3200640 1 -- 172.21.15.107:0/3036556241 >> 172.21.15.107:0/3036556241 conn(0x7f32f40fcc00 msgr2=0x7f32f40ff020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:06:49.831 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.851+0000 7f32f3200640 1 -- 172.21.15.107:0/3036556241 shutdown_connections 2024-03-08T21:06:49.831 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.851+0000 7f32f3200640 1 -- 172.21.15.107:0/3036556241 wait complete. 2024-03-08T21:06:49.834 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.852+0000 7f32f3200640 1 Processor -- start 2024-03-08T21:06:49.834 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.853+0000 7f32f3200640 1 -- start start 2024-03-08T21:06:49.834 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.853+0000 7f32f3200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f32f4101340 0x7f32f419f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:06:49.834 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.853+0000 7f32f3200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f32f4106410 0x7f32f419fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:06:49.834 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.853+0000 7f32f3200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f32f4199910 0x7f32f4199dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:06:49.834 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.853+0000 7f32f3200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32f4111350 con 0x7f32f4106410 2024-03-08T21:06:49.834 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.853+0000 7f32f3200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32f41111d0 con 0x7f32f4101340 2024-03-08T21:06:49.834 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.853+0000 7f32f3200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32f41114d0 con 0x7f32f4199910 2024-03-08T21:06:49.834 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.853+0000 7f32e9400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f32f4106410 0x7f32f419fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:06:49.835 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.853+0000 7f32e9400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f32f4106410 0x7f32f419fc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:43112/0 (socket says 172.21.15.107:43112) 2024-03-08T21:06:49.835 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.853+0000 7f32e9400640 1 -- 172.21.15.107:0/3290987396 learned_addr learned my addr 172.21.15.107:0/3290987396 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:06:49.835 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.854+0000 7f32f1e00640 1 --2- 172.21.15.107:0/3290987396 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f32f4101340 0x7f32f419f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:06:49.835 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.854+0000 7f32f2800640 1 --2- 172.21.15.107:0/3290987396 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f32f4199910 0x7f32f4199dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:06:49.835 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.854+0000 7f32e9400640 1 -- 172.21.15.107:0/3290987396 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f32f4199910 msgr2=0x7f32f4199dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:06:49.835 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.854+0000 7f32e9400640 1 --2- 172.21.15.107:0/3290987396 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f32f4199910 0x7f32f4199dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:06:49.835 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.854+0000 7f32e9400640 1 -- 172.21.15.107:0/3290987396 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f32f4101340 msgr2=0x7f32f419f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:06:49.835 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.854+0000 7f32e9400640 1 --2- 172.21.15.107:0/3290987396 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f32f4101340 0x7f32f419f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:06:49.835 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.854+0000 7f32e9400640 1 -- 172.21.15.107:0/3290987396 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32f419a430 con 0x7f32f4106410 2024-03-08T21:06:49.835 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.854+0000 7f32f1e00640 1 --2- 172.21.15.107:0/3290987396 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f32f4101340 0x7f32f419f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T21:06:49.835 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.854+0000 7f32e9400640 1 --2- 172.21.15.107:0/3290987396 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f32f4106410 0x7f32f419fc50 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f32dc00c970 tx=0x7f32dc00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:06:49.835 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.855+0000 7f32eb400640 1 -- 172.21.15.107:0/3290987396 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32dc014070 con 0x7f32f4106410 2024-03-08T21:06:49.835 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.855+0000 7f32eb400640 1 -- 172.21.15.107:0/3290987396 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f32dc004440 con 0x7f32f4106410 2024-03-08T21:06:49.835 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.855+0000 7f32eb400640 1 -- 172.21.15.107:0/3290987396 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32dc003e50 con 0x7f32f4106410 2024-03-08T21:06:49.835 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.855+0000 7f32f3200640 1 -- 172.21.15.107:0/3290987396 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32f419a720 con 0x7f32f4106410 2024-03-08T21:06:49.836 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.855+0000 7f32f3200640 1 -- 172.21.15.107:0/3290987396 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32f41a6970 con 0x7f32f4106410 2024-03-08T21:06:49.838 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.859+0000 7f32fbfbb740 1 Processor -- start 2024-03-08T21:06:49.839 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.859+0000 7f32fbfbb740 1 -- start start 2024-03-08T21:06:49.839 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.860+0000 7f32fbfbb740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565237d65e80 0x565237d66260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:06:49.840 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.860+0000 7f32fbfbb740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x565237d667a0 0x565237d6f030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:06:49.840 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.860+0000 7f32fbfbb740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x565237d6f570 0x565237d71960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:06:49.840 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.860+0000 7f32fbfbb740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x565237cc93d0 con 0x565237d667a0 2024-03-08T21:06:49.840 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.860+0000 7f32fbfbb740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x565237c726b0 con 0x565237d6f570 2024-03-08T21:06:49.840 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.860+0000 7f32fbfbb740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x565237cc5470 con 0x565237d65e80 2024-03-08T21:06:49.840 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.860+0000 7f32eb400640 1 -- 172.21.15.107:0/3290987396 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) v1 ==== 137422+0+0 (secure 0 0 0) 0x7f32dc017070 con 0x7f32f4106410 2024-03-08T21:06:49.840 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.860+0000 7f32f1e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x565237d667a0 0x565237d6f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:06:49.840 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.860+0000 7f32f1e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x565237d667a0 0x565237d6f030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:43124/0 (socket says 172.21.15.107:43124) 2024-03-08T21:06:49.840 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.860+0000 7f32f1e00640 1 -- 172.21.15.107:0/906545153 learned_addr learned my addr 172.21.15.107:0/906545153 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:06:49.840 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.860+0000 7f32f2800640 1 --2- 172.21.15.107:0/906545153 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565237d65e80 0x565237d66260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:06:49.840 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.860+0000 7f32e9400640 1 --2- 172.21.15.107:0/906545153 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x565237d6f570 0x565237d71960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:06:49.840 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.860+0000 7f32f1e00640 1 -- 172.21.15.107:0/906545153 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565237d65e80 msgr2=0x565237d66260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:06:49.840 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.860+0000 7f32f1e00640 1 --2- 172.21.15.107:0/906545153 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565237d65e80 0x565237d66260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:06:49.840 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.860+0000 7f32f1e00640 1 -- 172.21.15.107:0/906545153 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x565237d6f570 msgr2=0x565237d71960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:06:49.841 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.860+0000 7f32f1e00640 1 --2- 172.21.15.107:0/906545153 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x565237d6f570 0x565237d71960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:06:49.841 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.860+0000 7f32f1e00640 1 -- 172.21.15.107:0/906545153 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x565237c98b50 con 0x565237d667a0 2024-03-08T21:06:49.841 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.861+0000 7f32e9400640 1 --2- 172.21.15.107:0/906545153 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x565237d6f570 0x565237d71960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T21:06:49.842 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.861+0000 7f32f1e00640 1 --2- 172.21.15.107:0/906545153 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x565237d667a0 0x565237d6f030 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f32e4000f80 tx=0x7f32e4009930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c8587c03312350f server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:06:49.842 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.861+0000 7f32e3400640 1 -- 172.21.15.107:0/906545153 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32e4039050 con 0x565237d667a0 2024-03-08T21:06:49.843 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.861+0000 7f32e3400640 1 -- 172.21.15.107:0/906545153 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f32e4045070 con 0x565237d667a0 2024-03-08T21:06:49.843 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.861+0000 7f32e3400640 1 -- 172.21.15.107:0/906545153 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32e4004240 con 0x565237d667a0 2024-03-08T21:06:49.843 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.862+0000 7f32eb400640 1 --2- 172.21.15.107:0/3290987396 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f32d80a2b80 0x7f32d80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:06:49.843 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.862+0000 7f32eb400640 1 -- 172.21.15.107:0/3290987396 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f32dc0ceda0 con 0x7f32f4106410 2024-03-08T21:06:49.843 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.862+0000 7f32f2800640 1 --2- 172.21.15.107:0/3290987396 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f32d80a2b80 0x7f32d80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:06:49.843 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.862+0000 7f32fbfbb740 1 -- 172.21.15.107:0/906545153 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x565237d667a0 msgr2=0x565237d6f030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:06:49.843 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.862+0000 7f32fbfbb740 1 --2- 172.21.15.107:0/906545153 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x565237d667a0 0x565237d6f030 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f32e4000f80 tx=0x7f32e4009930 comp rx=0 tx=0).stop 2024-03-08T21:06:49.843 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.863+0000 7f32fbfbb740 1 -- 172.21.15.107:0/906545153 shutdown_connections 2024-03-08T21:06:49.843 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.863+0000 7f32fbfbb740 1 --2- 172.21.15.107:0/906545153 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x565237d6f570 0x565237d71960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:06:49.843 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.863+0000 7f32fbfbb740 1 --2- 172.21.15.107:0/906545153 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x565237d667a0 0x565237d6f030 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:06:49.843 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.863+0000 7f32fbfbb740 1 --2- 172.21.15.107:0/906545153 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565237d65e80 0x565237d66260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:06:49.843 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.863+0000 7f32fbfbb740 1 -- 172.21.15.107:0/906545153 >> 172.21.15.107:0/906545153 conn(0x565237d607a0 msgr2=0x565237d60b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:06:49.843 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.863+0000 7f32fbfbb740 1 -- 172.21.15.107:0/906545153 shutdown_connections 2024-03-08T21:06:49.844 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.863+0000 7f32f2800640 1 --2- 172.21.15.107:0/3290987396 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f32d80a2b80 0x7f32d80a5040 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f32ec00da80 tx=0x7f32ec005cc0 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:06:49.844 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.863+0000 7f32fbfbb740 1 -- 172.21.15.107:0/906545153 wait complete. 2024-03-08T21:06:49.844 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.863+0000 7f32fbfbb740 1 -- start start 2024-03-08T21:06:49.844 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.864+0000 7f32fbfbb740 1 Processor -- start 2024-03-08T21:06:49.847 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.864+0000 7f32fbfbb740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x565237d65e80 0x565237d6ef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:06:49.847 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.864+0000 7f32fbfbb740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565237d667a0 0x565237d6c290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:06:49.847 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.864+0000 7f32fbfbb740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x565237d6f570 0x565237d6c810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:06:49.847 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.864+0000 7f32fbfbb740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x565237d72140 con 0x565237d6f570 2024-03-08T21:06:49.847 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.864+0000 7f32fbfbb740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x565237c859f0 con 0x565237d65e80 2024-03-08T21:06:49.847 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.864+0000 7f32fbfbb740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x565237d722c0 con 0x565237d667a0 2024-03-08T21:06:49.847 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.864+0000 7f32fbfbb740 10 client.0 _ll_register_callbacks cb 0x5652379dda00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T21:06:49.848 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.864+0000 7f32f2800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x565237d6f570 0x565237d6c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:06:49.848 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.865+0000 7f32f2800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x565237d6f570 0x565237d6c810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:43126/0 (socket says 172.21.15.107:43126) 2024-03-08T21:06:49.848 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.865+0000 7f32f2800640 1 -- 172.21.15.107:0/3471327417 learned_addr learned my addr 172.21.15.107:0/3471327417 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:06:49.848 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.865+0000 7f32e9400640 1 --2- 172.21.15.107:0/3471327417 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565237d667a0 0x565237d6c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:06:49.848 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.865+0000 7f32f1e00640 1 --2- 172.21.15.107:0/3471327417 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x565237d65e80 0x565237d6ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:06:49.848 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.865+0000 7f32f2800640 1 -- 172.21.15.107:0/3471327417 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565237d667a0 msgr2=0x565237d6c290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:06:49.848 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.865+0000 7f32f2800640 1 --2- 172.21.15.107:0/3471327417 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565237d667a0 0x565237d6c290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:06:49.848 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.865+0000 7f32f2800640 1 -- 172.21.15.107:0/3471327417 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x565237d65e80 msgr2=0x565237d6ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:06:49.848 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.865+0000 7f32f2800640 1 --2- 172.21.15.107:0/3471327417 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x565237d65e80 0x565237d6ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:06:49.848 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.865+0000 7f32f2800640 1 -- 172.21.15.107:0/3471327417 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5652379de210 con 0x565237d6f570 2024-03-08T21:06:49.848 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.865+0000 7f32f1e00640 1 --2- 172.21.15.107:0/3471327417 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x565237d65e80 0x565237d6ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T21:06:49.848 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.865+0000 7f32f2800640 1 --2- 172.21.15.107:0/3471327417 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x565237d6f570 0x565237d6c810 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f32ec0109b0 tx=0x7f32ec0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b05da86e6845fd8a server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:06:49.849 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.865+0000 7f32d7e00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-03-08T21:06:49.849 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.866+0000 7f32d7e00640 1 -- 172.21.15.107:0/3471327417 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32ec01bcd0 con 0x565237d6f570 2024-03-08T21:06:49.849 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.866+0000 7f32fbfbb740 1 -- 172.21.15.107:0/3471327417 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f32e4045070 con 0x565237d6f570 2024-03-08T21:06:49.849 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.866+0000 7f32d7e00640 1 -- 172.21.15.107:0/3471327417 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f32ec026480 con 0x565237d6f570 2024-03-08T21:06:49.849 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.866+0000 7f32d7e00640 1 -- 172.21.15.107:0/3471327417 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f32ec026680 con 0x565237d6f570 2024-03-08T21:06:49.849 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.866+0000 7f32fbfbb740 10 client.15981 fetch_fsmap learned FSMap version 34 2024-03-08T21:06:49.849 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.866+0000 7f32fbfbb740 1 -- 172.21.15.107:0/3471327417 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x565237aaa620 con 0x565237d6f570 2024-03-08T21:06:49.849 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.866+0000 7f32d7e00640 1 -- 172.21.15.107:0/3471327417 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f32e4045070 con 0x565237d6f570 2024-03-08T21:06:49.849 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.867+0000 7f32d7e00640 1 -- 172.21.15.107:0/3471327417 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 34) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f32ec01d420 con 0x565237d6f570 2024-03-08T21:06:49.849 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.867+0000 7f32fbfbb740 10 client.15981 fetch_fsmap finished waiting for FSMap version 34 2024-03-08T21:06:49.849 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.867+0000 7f32fbfbb740 10 client.15981 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-08T21:06:49.849 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.867+0000 7f32fbfbb740 20 client.15981 populate_metadata read hostname 'smithi107' 2024-03-08T21:06:49.849 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.867+0000 7f32fbfbb740 1 --2- 172.21.15.107:0/3471327417 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x565237d7f300 0x565237d816f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:06:49.849 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.867+0000 7f32fbfbb740 4 client.15981 mds_command: new command op to 34475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-08T21:06:49.850 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.867+0000 7f32fbfbb740 1 -- 172.21.15.107:0/3471327417 --> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x565237cd6910 con 0x565237d7f300 2024-03-08T21:06:49.850 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.867+0000 7f32f1e00640 1 --2- 172.21.15.107:0/3471327417 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x565237d7f300 0x565237d816f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:06:49.850 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.868+0000 7f32f1e00640 1 --2- 172.21.15.107:0/3471327417 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x565237d7f300 0x565237d816f0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=82702120155f4497 server_cookie=1c81408fe166b884 in_seq=0 out_seq=0 2024-03-08T21:06:49.850 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.868+0000 7f32d7e00640 10 client.15981 ms_handle_connect on v2:172.21.15.149:6838/55290596 2024-03-08T21:06:49.851 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.870+0000 7f32d7e00640 1 -- 172.21.15.107:0/3471327417 <== mds.0 v2:172.21.15.149:6838/55290596 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x565237cd6910 con 0x565237d7f300 2024-03-08T21:06:49.851 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.870+0000 7f32d7e00640 10 client.15981 handle_command_reply: tid=0 2024-03-08T21:06:49.851 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.870+0000 7f32fbfbb740 1 client.15981 shutdown 2024-03-08T21:06:49.851 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.870+0000 7f32fbfbb740 1 -- 172.21.15.107:0/3471327417 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x565237d6f570 msgr2=0x565237d6c810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:06:49.851 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.870+0000 7f32fbfbb740 1 --2- 172.21.15.107:0/3471327417 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x565237d6f570 0x565237d6c810 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f32ec0109b0 tx=0x7f32ec0046c0 comp rx=0 tx=0).stop 2024-03-08T21:06:49.851 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.870+0000 7f32fbfbb740 1 -- 172.21.15.107:0/3471327417 shutdown_connections 2024-03-08T21:06:49.851 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.870+0000 7f32fbfbb740 1 --2- 172.21.15.107:0/3471327417 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x565237d6f570 0x565237d6c810 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:06:49.852 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.870+0000 7f32fbfbb740 1 --2- 172.21.15.107:0/3471327417 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565237d667a0 0x565237d6c290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:06:49.852 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.870+0000 7f32fbfbb740 1 --2- 172.21.15.107:0/3471327417 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x565237d7f300 0x565237d816f0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:06:49.852 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.870+0000 7f32fbfbb740 1 --2- 172.21.15.107:0/3471327417 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x565237d65e80 0x565237d6ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:06:49.852 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.870+0000 7f32fbfbb740 1 -- 172.21.15.107:0/3471327417 >> 172.21.15.107:0/3471327417 conn(0x565237d607a0 msgr2=0x565237d624f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:06:49.852 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.870+0000 7f32d7e00640 0 client.15981 ms_handle_reset on v2:172.21.15.149:6838/55290596 2024-03-08T21:06:49.852 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.870+0000 7f32fbfbb740 1 -- 172.21.15.107:0/3471327417 shutdown_connections 2024-03-08T21:06:49.852 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.871+0000 7f32fbfbb740 1 -- 172.21.15.107:0/3471327417 wait complete. 2024-03-08T21:06:49.852 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.871+0000 7f32fbfbb740 20 client.15981 trim_cache size 0 max 16384 2024-03-08T21:06:49.865 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.885+0000 7f32fbfbb740 1 Processor -- start 2024-03-08T21:06:49.865 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.885+0000 7f32fbfbb740 1 -- start start 2024-03-08T21:06:49.865 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.886+0000 7f32fbfbb740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565237d65e80 0x565237d6e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:06:49.865 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.886+0000 7f32fbfbb740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x565237d667a0 0x565237d6ef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:06:49.865 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.886+0000 7f32fbfbb740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x565237d6f570 0x565237d694f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:06:49.865 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.886+0000 7f32fbfbb740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32f41aece0 con 0x565237d6f570 2024-03-08T21:06:49.865 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.886+0000 7f32fbfbb740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32f41b0660 con 0x565237d667a0 2024-03-08T21:06:49.865 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.886+0000 7f32fbfbb740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32f41af340 con 0x565237d65e80 2024-03-08T21:06:49.867 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.886+0000 7f32f2800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x565237d6f570 0x565237d694f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:06:49.867 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.886+0000 7f32f2800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x565237d6f570 0x565237d694f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:43128/0 (socket says 172.21.15.107:43128) 2024-03-08T21:06:49.867 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.886+0000 7f32f2800640 1 -- 172.21.15.107:0/214579789 learned_addr learned my addr 172.21.15.107:0/214579789 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:06:49.867 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.886+0000 7f32e9400640 1 --2- 172.21.15.107:0/214579789 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x565237d667a0 0x565237d6ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:06:49.867 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.886+0000 7f32f1e00640 1 --2- 172.21.15.107:0/214579789 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565237d65e80 0x565237d6e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:06:49.867 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.886+0000 7f32f2800640 1 -- 172.21.15.107:0/214579789 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565237d65e80 msgr2=0x565237d6e9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:06:49.868 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.887+0000 7f32f2800640 1 --2- 172.21.15.107:0/214579789 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565237d65e80 0x565237d6e9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:06:49.868 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.887+0000 7f32f2800640 1 -- 172.21.15.107:0/214579789 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x565237d667a0 msgr2=0x565237d6ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:06:49.868 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.887+0000 7f32f2800640 1 --2- 172.21.15.107:0/214579789 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x565237d667a0 0x565237d6ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:06:49.868 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.887+0000 7f32f2800640 1 -- 172.21.15.107:0/214579789 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x565237d65620 con 0x565237d6f570 2024-03-08T21:06:49.868 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.887+0000 7f32f1e00640 1 --2- 172.21.15.107:0/214579789 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565237d65e80 0x565237d6e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T21:06:49.868 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.887+0000 7f32f2800640 1 --2- 172.21.15.107:0/214579789 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x565237d6f570 0x565237d694f0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f32ec00e010 tx=0x7f32ec00ff80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29ea0922e0543b69 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:06:49.868 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.887+0000 7f32e2a00640 1 -- 172.21.15.107:0/214579789 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32ec01d420 con 0x565237d6f570 2024-03-08T21:06:49.868 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.887+0000 7f32e2a00640 1 -- 172.21.15.107:0/214579789 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f32ec01d5c0 con 0x565237d6f570 2024-03-08T21:06:49.868 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.887+0000 7f32e2a00640 1 -- 172.21.15.107:0/214579789 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32ec026e20 con 0x565237d6f570 2024-03-08T21:06:49.868 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.888+0000 7f32fbfbb740 1 -- 172.21.15.107:0/214579789 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x565237d6f570 msgr2=0x565237d694f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:06:49.868 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.888+0000 7f32fbfbb740 1 --2- 172.21.15.107:0/214579789 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x565237d6f570 0x565237d694f0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f32ec00e010 tx=0x7f32ec00ff80 comp rx=0 tx=0).stop 2024-03-08T21:06:49.868 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.888+0000 7f32fbfbb740 1 -- 172.21.15.107:0/214579789 shutdown_connections 2024-03-08T21:06:49.868 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.888+0000 7f32fbfbb740 1 --2- 172.21.15.107:0/214579789 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x565237d6f570 0x565237d694f0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:06:49.868 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.888+0000 7f32fbfbb740 1 --2- 172.21.15.107:0/214579789 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x565237d667a0 0x565237d6ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:06:49.869 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.888+0000 7f32fbfbb740 1 --2- 172.21.15.107:0/214579789 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565237d65e80 0x565237d6e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:06:49.869 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.888+0000 7f32fbfbb740 1 -- 172.21.15.107:0/214579789 >> 172.21.15.107:0/214579789 conn(0x565237d607a0 msgr2=0x565237d6ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:06:49.869 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.888+0000 7f32fbfbb740 1 -- 172.21.15.107:0/214579789 shutdown_connections 2024-03-08T21:06:49.869 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.888+0000 7f32fbfbb740 1 -- 172.21.15.107:0/214579789 wait complete. 2024-03-08T21:06:49.871 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.889+0000 7f32fbfbb740 1 -- start start 2024-03-08T21:06:49.871 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.889+0000 7f32fbfbb740 1 Processor -- start 2024-03-08T21:06:49.871 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.889+0000 7f32fbfbb740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565237d65e80 0x565237d74ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:06:49.871 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.889+0000 7f32fbfbb740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x565237d667a0 0x565237d753e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:06:49.872 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.889+0000 7f32fbfbb740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x565237d6f570 0x565237d75920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:06:49.872 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.889+0000 7f32fbfbb740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32f41b0000 con 0x565237d6f570 2024-03-08T21:06:49.872 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.889+0000 7f32fbfbb740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32f41af9a0 con 0x565237d667a0 2024-03-08T21:06:49.872 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.889+0000 7f32fbfbb740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32f41acf80 con 0x565237d65e80 2024-03-08T21:06:49.872 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.889+0000 7f32fbfbb740 10 client.0 _ll_register_callbacks cb 0x7f32e4037070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T21:06:49.872 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.890+0000 7f32f2800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x565237d6f570 0x565237d75920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:06:49.872 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.890+0000 7f32f2800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x565237d6f570 0x565237d75920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:43140/0 (socket says 172.21.15.107:43140) 2024-03-08T21:06:49.872 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.890+0000 7f32f2800640 1 -- 172.21.15.107:0/102722788 learned_addr learned my addr 172.21.15.107:0/102722788 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:06:49.872 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.890+0000 7f32e9400640 1 --2- 172.21.15.107:0/102722788 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x565237d667a0 0x565237d753e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:06:49.872 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.890+0000 7f32f2800640 1 -- 172.21.15.107:0/102722788 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565237d65e80 msgr2=0x565237d74ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:06:49.872 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.890+0000 7f32f2800640 1 --2- 172.21.15.107:0/102722788 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565237d65e80 0x565237d74ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:06:49.872 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.890+0000 7f32f2800640 1 -- 172.21.15.107:0/102722788 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x565237d667a0 msgr2=0x565237d753e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:06:49.872 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.890+0000 7f32f2800640 1 --2- 172.21.15.107:0/102722788 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x565237d667a0 0x565237d753e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:06:49.872 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.890+0000 7f32f2800640 1 -- 172.21.15.107:0/102722788 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x565237dd57b0 con 0x565237d6f570 2024-03-08T21:06:49.873 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.890+0000 7f32f2800640 1 --2- 172.21.15.107:0/102722788 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x565237d6f570 0x565237d75920 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f32ec010fd0 tx=0x7f32ec01b710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41c69995a4012fc8 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:06:49.873 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.890+0000 7f32d7e00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-03-08T21:06:49.873 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.891+0000 7f32d7e00640 1 -- 172.21.15.107:0/102722788 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32ec01d9e0 con 0x565237d6f570 2024-03-08T21:06:49.873 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.891+0000 7f32d7e00640 1 -- 172.21.15.107:0/102722788 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f32ec01db80 con 0x565237d6f570 2024-03-08T21:06:49.873 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.891+0000 7f32fbfbb740 1 -- 172.21.15.107:0/102722788 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f32ec01d5c0 con 0x565237d6f570 2024-03-08T21:06:49.873 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.891+0000 7f32d7e00640 1 -- 172.21.15.107:0/102722788 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f32ec026ba0 con 0x565237d6f570 2024-03-08T21:06:49.873 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.891+0000 7f32fbfbb740 10 client.15987 fetch_fsmap learned FSMap version 34 2024-03-08T21:06:49.873 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.891+0000 7f32fbfbb740 1 -- 172.21.15.107:0/102722788 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x565237dd6990 con 0x565237d6f570 2024-03-08T21:06:49.873 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.891+0000 7f32d7e00640 1 -- 172.21.15.107:0/102722788 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f32ec01d5c0 con 0x565237d6f570 2024-03-08T21:06:49.873 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.891+0000 7f32d7e00640 1 -- 172.21.15.107:0/102722788 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 34) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f32ec049030 con 0x565237d6f570 2024-03-08T21:06:49.873 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.892+0000 7f32fbfbb740 10 client.15987 fetch_fsmap finished waiting for FSMap version 34 2024-03-08T21:06:49.873 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.892+0000 7f32fbfbb740 10 client.15987 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-08T21:06:49.873 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.892+0000 7f32fbfbb740 20 client.15987 populate_metadata read hostname 'smithi107' 2024-03-08T21:06:49.873 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.892+0000 7f32fbfbb740 1 --2- 172.21.15.107:0/102722788 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x565237dd6fa0 0x565237dd93c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:06:49.874 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.892+0000 7f32fbfbb740 4 client.15987 mds_command: new command op to 34475 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-08T21:06:49.874 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.892+0000 7f32fbfbb740 1 -- 172.21.15.107:0/102722788 --> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x565237cd0c60 con 0x565237dd6fa0 2024-03-08T21:06:49.874 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.892+0000 7f32f1e00640 1 --2- 172.21.15.107:0/102722788 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x565237dd6fa0 0x565237dd93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:06:49.874 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.893+0000 7f32f1e00640 1 --2- 172.21.15.107:0/102722788 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x565237dd6fa0 0x565237dd93c0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ca3fb65f90c0f60d server_cookie=adc6282c21d8e028 in_seq=0 out_seq=0 2024-03-08T21:06:49.874 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.893+0000 7f32d7e00640 10 client.15987 ms_handle_connect on v2:172.21.15.149:6838/55290596 2024-03-08T21:06:49.883 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.904+0000 7f32d7e00640 1 -- 172.21.15.107:0/102722788 <== mds.0 v2:172.21.15.149:6838/55290596 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x565237cd0c60 con 0x565237dd6fa0 2024-03-08T21:06:49.883 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.904+0000 7f32d7e00640 10 client.15987 handle_command_reply: tid=0 2024-03-08T21:06:49.883 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.904+0000 7f32fbfbb740 1 client.15987 shutdown 2024-03-08T21:06:49.883 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.904+0000 7f32fbfbb740 1 -- 172.21.15.107:0/102722788 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x565237d6f570 msgr2=0x565237d75920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:06:49.884 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.904+0000 7f32fbfbb740 1 --2- 172.21.15.107:0/102722788 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x565237d6f570 0x565237d75920 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f32ec010fd0 tx=0x7f32ec01b710 comp rx=0 tx=0).stop 2024-03-08T21:06:49.884 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.904+0000 7f32fbfbb740 1 -- 172.21.15.107:0/102722788 shutdown_connections 2024-03-08T21:06:49.884 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.904+0000 7f32fbfbb740 1 --2- 172.21.15.107:0/102722788 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x565237d6f570 0x565237d75920 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:06:49.884 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.904+0000 7f32fbfbb740 1 --2- 172.21.15.107:0/102722788 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x565237dd6fa0 0x565237dd93c0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:06:49.884 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.904+0000 7f32fbfbb740 1 --2- 172.21.15.107:0/102722788 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x565237d667a0 0x565237d753e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:06:49.884 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.904+0000 7f32fbfbb740 1 --2- 172.21.15.107:0/102722788 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x565237d65e80 0x565237d74ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:06:49.884 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.904+0000 7f32fbfbb740 1 -- 172.21.15.107:0/102722788 >> 172.21.15.107:0/102722788 conn(0x565237d607a0 msgr2=0x565237d5ef70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:06:49.884 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.904+0000 7f32d7e00640 0 client.15987 ms_handle_reset on v2:172.21.15.149:6838/55290596 2024-03-08T21:06:49.884 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.904+0000 7f32fbfbb740 1 -- 172.21.15.107:0/102722788 shutdown_connections 2024-03-08T21:06:49.884 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.904+0000 7f32fbfbb740 1 -- 172.21.15.107:0/102722788 wait complete. 2024-03-08T21:06:49.884 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.904+0000 7f32fbfbb740 20 client.15987 trim_cache size 0 max 16384 2024-03-08T21:06:49.885 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:{ 2024-03-08T21:06:49.885 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "message": "", 2024-03-08T21:06:49.885 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "return_code": 0 2024-03-08T21:06:49.885 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:} 2024-03-08T21:06:49.885 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.905+0000 7f32f3200640 1 -- 172.21.15.107:0/3290987396 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f32d80a2b80 msgr2=0x7f32d80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:06:49.885 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.905+0000 7f32f3200640 1 --2- 172.21.15.107:0/3290987396 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f32d80a2b80 0x7f32d80a5040 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f32ec00da80 tx=0x7f32ec005cc0 comp rx=0 tx=0).stop 2024-03-08T21:06:49.886 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.905+0000 7f32f3200640 1 -- 172.21.15.107:0/3290987396 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f32f4106410 msgr2=0x7f32f419fc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:06:49.886 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.905+0000 7f32f3200640 1 --2- 172.21.15.107:0/3290987396 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f32f4106410 0x7f32f419fc50 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f32dc00c970 tx=0x7f32dc00ce30 comp rx=0 tx=0).stop 2024-03-08T21:06:49.886 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.905+0000 7f32f3200640 1 -- 172.21.15.107:0/3290987396 shutdown_connections 2024-03-08T21:06:49.886 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.905+0000 7f32f3200640 1 --2- 172.21.15.107:0/3290987396 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f32d80a2b80 0x7f32d80a5040 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:06:49.886 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.905+0000 7f32f3200640 1 --2- 172.21.15.107:0/3290987396 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f32f4199910 0x7f32f4199dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:06:49.886 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.905+0000 7f32f3200640 1 --2- 172.21.15.107:0/3290987396 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f32f4106410 0x7f32f419fc50 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:06:49.886 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.905+0000 7f32f3200640 1 --2- 172.21.15.107:0/3290987396 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f32f4101340 0x7f32f419f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:06:49.886 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.905+0000 7f32f3200640 1 -- 172.21.15.107:0/3290987396 >> 172.21.15.107:0/3290987396 conn(0x7f32f40fcc00 msgr2=0x7f32f40fef40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:06:49.886 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.905+0000 7f32f3200640 1 -- 172.21.15.107:0/3290987396 shutdown_connections 2024-03-08T21:06:49.886 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:06:49.905+0000 7f32f3200640 1 -- 172.21.15.107:0/3290987396 wait complete. 2024-03-08T21:06:51.964 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:06:51 smithi107 ceph-mon[31381]: pgmap v195: 97 pgs: 97 active+clean; 4.0 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-08T21:06:51.964 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:06:51 smithi107 ceph-mon[31381]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T21:06:52.011 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:06:51 smithi149 ceph-mon[35042]: pgmap v195: 97 pgs: 97 active+clean; 4.0 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-08T21:06:52.011 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:06:51 smithi149 ceph-mon[35042]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T21:06:52.124 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:06:51 smithi204 ceph-mon[34935]: pgmap v195: 97 pgs: 97 active+clean; 4.0 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-08T21:06:52.125 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:06:51 smithi204 ceph-mon[34935]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T21:06:52.964 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:06:52 smithi107 ceph-mon[31381]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T21:06:52.964 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:06:52 smithi107 ceph-mon[31381]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T21:06:52.964 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:06:52 smithi107 ceph-mon[31381]: from='mgr.15375 ' entity='mgr.x' 2024-03-08T21:06:52.964 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:06:52 smithi107 ceph-mon[31381]: pgmap v196: 97 pgs: 97 active+clean; 4.0 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-08T21:06:53.011 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:06:52 smithi149 ceph-mon[35042]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T21:06:53.011 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:06:52 smithi149 ceph-mon[35042]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T21:06:53.011 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:06:52 smithi149 ceph-mon[35042]: from='mgr.15375 ' entity='mgr.x' 2024-03-08T21:06:53.011 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:06:52 smithi149 ceph-mon[35042]: pgmap v196: 97 pgs: 97 active+clean; 4.0 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-08T21:06:53.125 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:06:52 smithi204 ceph-mon[34935]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T21:06:53.125 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:06:52 smithi204 ceph-mon[34935]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T21:06:53.125 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:06:52 smithi204 ceph-mon[34935]: from='mgr.15375 ' entity='mgr.x' 2024-03-08T21:06:53.125 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:06:52 smithi204 ceph-mon[34935]: pgmap v196: 97 pgs: 97 active+clean; 4.0 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-08T21:06:55.964 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:06:55 smithi107 ceph-mon[31381]: pgmap v197: 97 pgs: 97 active+clean; 4.0 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-08T21:06:56.011 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:06:55 smithi149 ceph-mon[35042]: pgmap v197: 97 pgs: 97 active+clean; 4.0 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-08T21:06:56.124 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:06:55 smithi204 ceph-mon[34935]: pgmap v197: 97 pgs: 97 active+clean; 4.0 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-08T21:06:56.963 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:06:56 smithi107 ceph-mon[31381]: pgmap v198: 97 pgs: 97 active+clean; 4.0 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:06:57.011 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:06:56 smithi149 ceph-mon[35042]: pgmap v198: 97 pgs: 97 active+clean; 4.0 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:06:57.124 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:06:56 smithi204 ceph-mon[34935]: pgmap v198: 97 pgs: 97 active+clean; 4.0 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:06:59.011 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:06:58 smithi149 ceph-mon[35042]: pgmap v199: 97 pgs: 97 active+clean; 4.0 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-08T21:06:59.124 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:06:58 smithi204 ceph-mon[34935]: pgmap v199: 97 pgs: 97 active+clean; 4.0 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-08T21:06:59.213 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:06:58 smithi107 ceph-mon[31381]: pgmap v199: 97 pgs: 97 active+clean; 4.0 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-08T21:07:01.011 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:07:00 smithi149 ceph-mon[35042]: pgmap v200: 97 pgs: 97 active+clean; 4.0 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:07:01.124 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:07:00 smithi204 ceph-mon[34935]: pgmap v200: 97 pgs: 97 active+clean; 4.0 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:07:01.213 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:07:00 smithi107 ceph-mon[31381]: pgmap v200: 97 pgs: 97 active+clean; 4.0 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:07:03.011 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:07:02 smithi149 ceph-mon[35042]: pgmap v201: 97 pgs: 97 active+clean; 4.0 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:07:03.029 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:03.049+0000 7ffa7d000640 1 -- 172.21.15.107:0/3607527135 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffa78073fe0 msgr2=0x7ffa7810d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:07:03.029 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:03.049+0000 7ffa7d000640 1 --2- 172.21.15.107:0/3607527135 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffa78073fe0 0x7ffa7810d450 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7ffa64009900 tx=0x7ffa6402bcd0 comp rx=0 tx=0).stop 2024-03-08T21:07:03.029 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:03.049+0000 7ffa7d000640 1 -- 172.21.15.107:0/3607527135 shutdown_connections 2024-03-08T21:07:03.029 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:03.049+0000 7ffa7d000640 1 --2- 172.21.15.107:0/3607527135 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffa7810d990 0x7ffa7810fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:07:03.029 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:03.049+0000 7ffa7d000640 1 --2- 172.21.15.107:0/3607527135 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffa78073fe0 0x7ffa7810d450 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:07:03.029 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:03.049+0000 7ffa7d000640 1 --2- 172.21.15.107:0/3607527135 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffa780736c0 0x7ffa78073aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:07:03.029 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:03.049+0000 7ffa7d000640 1 -- 172.21.15.107:0/3607527135 >> 172.21.15.107:0/3607527135 conn(0x7ffa780fcc00 msgr2=0x7ffa780ff020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:07:03.029 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:03.049+0000 7ffa7d000640 1 -- 172.21.15.107:0/3607527135 shutdown_connections 2024-03-08T21:07:03.029 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:03.049+0000 7ffa7d000640 1 -- 172.21.15.107:0/3607527135 wait complete. 2024-03-08T21:07:03.032 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:03.050+0000 7ffa7d000640 1 Processor -- start 2024-03-08T21:07:03.032 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:03.051+0000 7ffa7d000640 1 -- start start 2024-03-08T21:07:03.032 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:03.051+0000 7ffa7d000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffa780736c0 0x7ffa7819f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:07:03.032 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:03.051+0000 7ffa7d000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffa78073fe0 0x7ffa7819fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:07:03.032 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:03.051+0000 7ffa7d000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffa7810d990 0x7ffa78199980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:07:03.032 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:03.051+0000 7ffa7d000640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa781135c0 con 0x7ffa780736c0 2024-03-08T21:07:03.032 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:03.051+0000 7ffa7d000640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa78113440 con 0x7ffa78073fe0 2024-03-08T21:07:03.032 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:03.051+0000 7ffa7d000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa78113740 con 0x7ffa7810d990 2024-03-08T21:07:03.032 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:03.051+0000 7ffa77400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffa780736c0 0x7ffa7819f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:07:03.032 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:03.051+0000 7ffa77400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffa780736c0 0x7ffa7819f7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:54404/0 (socket says 172.21.15.107:54404) 2024-03-08T21:07:03.033 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:03.051+0000 7ffa77400640 1 -- 172.21.15.107:0/1889276392 learned_addr learned my addr 172.21.15.107:0/1889276392 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:07:03.033 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:03.052+0000 7ffa77400640 1 -- 172.21.15.107:0/1889276392 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffa7810d990 msgr2=0x7ffa78199980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:07:03.033 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:03.052+0000 7ffa77400640 1 --2- 172.21.15.107:0/1889276392 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffa7810d990 0x7ffa78199980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:07:03.033 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:03.052+0000 7ffa76a00640 1 --2- 172.21.15.107:0/1889276392 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffa78073fe0 0x7ffa7819fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:07:03.033 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:03.052+0000 7ffa77400640 1 -- 172.21.15.107:0/1889276392 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffa78073fe0 msgr2=0x7ffa7819fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:07:03.033 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:03.052+0000 7ffa77400640 1 --2- 172.21.15.107:0/1889276392 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffa78073fe0 0x7ffa7819fd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:07:03.033 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:03.052+0000 7ffa77400640 1 -- 172.21.15.107:0/1889276392 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa7819a100 con 0x7ffa780736c0 2024-03-08T21:07:03.033 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:03.052+0000 7ffa76a00640 1 --2- 172.21.15.107:0/1889276392 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffa78073fe0 0x7ffa7819fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T21:07:03.033 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:03.052+0000 7ffa77400640 1 --2- 172.21.15.107:0/1889276392 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffa780736c0 0x7ffa7819f7d0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7ffa6800d950 tx=0x7ffa6800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:07:03.033 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:03.052+0000 7ffa6fe00640 1 -- 172.21.15.107:0/1889276392 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa68017070 con 0x7ffa780736c0 2024-03-08T21:07:03.033 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:03.053+0000 7ffa6fe00640 1 -- 172.21.15.107:0/1889276392 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7ffa68009c30 con 0x7ffa780736c0 2024-03-08T21:07:03.033 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:03.053+0000 7ffa7d000640 1 -- 172.21.15.107:0/1889276392 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa7819a3f0 con 0x7ffa780736c0 2024-03-08T21:07:03.033 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:03.053+0000 7ffa6fe00640 1 -- 172.21.15.107:0/1889276392 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa6800f620 con 0x7ffa780736c0 2024-03-08T21:07:03.033 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:03.053+0000 7ffa7d000640 1 -- 172.21.15.107:0/1889276392 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa781a6690 con 0x7ffa780736c0 2024-03-08T21:07:03.035 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:03.056+0000 7ffa6fe00640 1 -- 172.21.15.107:0/1889276392 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) v1 ==== 137422+0+0 (secure 0 0 0) 0x7ffa680109a0 con 0x7ffa780736c0 2024-03-08T21:07:03.035 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:03.057+0000 7ffa8142f740 1 Processor -- start 2024-03-08T21:07:03.037 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:03.057+0000 7ffa8142f740 1 -- start start 2024-03-08T21:07:03.037 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:03.057+0000 7ffa8142f740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a32de8e80 0x557a32de9260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:07:03.037 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:03.057+0000 7ffa8142f740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557a32de97a0 0x557a32df2030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:07:03.037 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:03.057+0000 7ffa8142f740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557a32df2570 0x557a32df4960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:07:03.037 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:03.057+0000 7ffa8142f740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a32d4c3d0 con 0x557a32df2570 2024-03-08T21:07:03.037 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:03.057+0000 7ffa8142f740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a32cf56b0 con 0x557a32de97a0 2024-03-08T21:07:03.038 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:03.057+0000 7ffa8142f740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a32d48470 con 0x557a32de8e80 2024-03-08T21:07:03.038 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:03.058+0000 7ffa76a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557a32df2570 0x557a32df4960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:07:03.038 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:03.058+0000 7ffa76a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557a32df2570 0x557a32df4960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:54406/0 (socket says 172.21.15.107:54406) 2024-03-08T21:07:03.038 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:03.058+0000 7ffa76a00640 1 -- 172.21.15.107:0/3816061844 learned_addr learned my addr 172.21.15.107:0/3816061844 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:07:03.038 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:03.058+0000 7ffa6fe00640 1 --2- 172.21.15.107:0/1889276392 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7ffa5c0a2960 0x7ffa5c0a4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:07:03.038 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:03.058+0000 7ffa77400640 1 --2- 172.21.15.107:0/3816061844 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557a32de97a0 0x557a32df2030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:07:03.038 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:03.058+0000 7ffa77e00640 1 --2- 172.21.15.107:0/3816061844 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a32de8e80 0x557a32de9260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:07:03.038 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:03.058+0000 7ffa6fe00640 1 -- 172.21.15.107:0/1889276392 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8541+0+0 (secure 0 0 0) 0x7ffa680cebe0 con 0x7ffa780736c0 2024-03-08T21:07:03.038 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:03.058+0000 7ffa77e00640 1 --2- 172.21.15.107:0/1889276392 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7ffa5c0a2960 0x7ffa5c0a4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:07:03.038 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:03.058+0000 7ffa76a00640 1 -- 172.21.15.107:0/3816061844 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a32de8e80 msgr2=0x557a32de9260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:07:03.038 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:03.058+0000 7ffa76a00640 1 --2- 172.21.15.107:0/3816061844 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a32de8e80 0x557a32de9260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:07:03.038 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:03.058+0000 7ffa76a00640 1 -- 172.21.15.107:0/3816061844 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557a32de97a0 msgr2=0x557a32df2030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:07:03.038 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:03.058+0000 7ffa76a00640 1 --2- 172.21.15.107:0/3816061844 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557a32de97a0 0x557a32df2030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:07:03.039 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:03.058+0000 7ffa76a00640 1 -- 172.21.15.107:0/3816061844 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557a32d1bb50 con 0x557a32df2570 2024-03-08T21:07:03.039 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:03.058+0000 7ffa77400640 1 --2- 172.21.15.107:0/3816061844 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557a32de97a0 0x557a32df2030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T21:07:03.043 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:03.059+0000 7ffa76a00640 1 --2- 172.21.15.107:0/3816061844 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557a32df2570 0x557a32df4960 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7ffa64006010 tx=0x7ffa6402c6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5dc96863aca4bfb5 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:07:03.043 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:03.059+0000 7ffa77e00640 1 --2- 172.21.15.107:0/1889276392 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7ffa5c0a2960 0x7ffa5c0a4e20 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ffa70005c40 tx=0x7ffa70009200 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:07:03.043 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:03.059+0000 7ffa6cc00640 1 -- 172.21.15.107:0/3816061844 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa64041020 con 0x557a32df2570 2024-03-08T21:07:03.044 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:03.059+0000 7ffa6cc00640 1 -- 172.21.15.107:0/3816061844 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7ffa64005040 con 0x557a32df2570 2024-03-08T21:07:03.044 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:03.059+0000 7ffa6cc00640 1 -- 172.21.15.107:0/3816061844 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa6402cc20 con 0x557a32df2570 2024-03-08T21:07:03.044 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:03.060+0000 7ffa8142f740 1 -- 172.21.15.107:0/3816061844 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557a32df2570 msgr2=0x557a32df4960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:07:03.044 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:03.060+0000 7ffa8142f740 1 --2- 172.21.15.107:0/3816061844 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557a32df2570 0x557a32df4960 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7ffa64006010 tx=0x7ffa6402c6f0 comp rx=0 tx=0).stop 2024-03-08T21:07:03.044 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:03.060+0000 7ffa8142f740 1 -- 172.21.15.107:0/3816061844 shutdown_connections 2024-03-08T21:07:03.044 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:03.060+0000 7ffa8142f740 1 --2- 172.21.15.107:0/3816061844 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557a32df2570 0x557a32df4960 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:07:03.044 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:03.060+0000 7ffa8142f740 1 --2- 172.21.15.107:0/3816061844 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557a32de97a0 0x557a32df2030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:07:03.044 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:03.060+0000 7ffa8142f740 1 --2- 172.21.15.107:0/3816061844 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a32de8e80 0x557a32de9260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:07:03.044 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:03.060+0000 7ffa8142f740 1 -- 172.21.15.107:0/3816061844 >> 172.21.15.107:0/3816061844 conn(0x557a32de37a0 msgr2=0x557a32de3b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:07:03.044 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:03.060+0000 7ffa8142f740 1 -- 172.21.15.107:0/3816061844 shutdown_connections 2024-03-08T21:07:03.044 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:03.060+0000 7ffa8142f740 1 -- 172.21.15.107:0/3816061844 wait complete. 2024-03-08T21:07:03.044 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:03.061+0000 7ffa8142f740 1 -- start start 2024-03-08T21:07:03.044 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:03.061+0000 7ffa8142f740 1 Processor -- start 2024-03-08T21:07:03.045 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:03.062+0000 7ffa8142f740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a32de8e80 0x557a32df1e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:07:03.045 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:03.062+0000 7ffa8142f740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557a32de97a0 0x557a32def1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:07:03.045 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:03.062+0000 7ffa8142f740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557a32df2570 0x557a32def6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:07:03.045 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:03.062+0000 7ffa8142f740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a32df5140 con 0x557a32df2570 2024-03-08T21:07:03.045 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:03.062+0000 7ffa8142f740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a32d089f0 con 0x557a32de97a0 2024-03-08T21:07:03.045 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:03.062+0000 7ffa8142f740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a32df52c0 con 0x557a32de8e80 2024-03-08T21:07:03.045 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:03.062+0000 7ffa8142f740 10 client.0 _ll_register_callbacks cb 0x557a32a60a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T21:07:03.045 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:03.062+0000 7ffa77e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557a32df2570 0x557a32def6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:07:03.045 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:03.062+0000 7ffa77e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557a32df2570 0x557a32def6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:54412/0 (socket says 172.21.15.107:54412) 2024-03-08T21:07:03.045 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:03.062+0000 7ffa77e00640 1 -- 172.21.15.107:0/2329007784 learned_addr learned my addr 172.21.15.107:0/2329007784 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:07:03.045 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:03.062+0000 7ffa77400640 1 --2- 172.21.15.107:0/2329007784 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a32de8e80 0x557a32df1e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:07:03.045 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:03.062+0000 7ffa76a00640 1 --2- 172.21.15.107:0/2329007784 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557a32de97a0 0x557a32def1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:07:03.045 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:03.062+0000 7ffa77e00640 1 -- 172.21.15.107:0/2329007784 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a32de8e80 msgr2=0x557a32df1e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:07:03.045 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:03.062+0000 7ffa77e00640 1 --2- 172.21.15.107:0/2329007784 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a32de8e80 0x557a32df1e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:07:03.045 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:03.062+0000 7ffa77e00640 1 -- 172.21.15.107:0/2329007784 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557a32de97a0 msgr2=0x557a32def1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:07:03.046 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:03.062+0000 7ffa77e00640 1 --2- 172.21.15.107:0/2329007784 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557a32de97a0 0x557a32def1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:07:03.046 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:03.063+0000 7ffa77e00640 1 -- 172.21.15.107:0/2329007784 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557a32a61210 con 0x557a32df2570 2024-03-08T21:07:03.046 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:03.063+0000 7ffa76a00640 1 --2- 172.21.15.107:0/2329007784 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557a32de97a0 0x557a32def1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T21:07:03.046 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:03.063+0000 7ffa77400640 1 --2- 172.21.15.107:0/2329007784 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a32de8e80 0x557a32df1e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T21:07:03.046 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:03.063+0000 7ffa77e00640 1 --2- 172.21.15.107:0/2329007784 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557a32df2570 0x557a32def6e0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7ffa7001c880 tx=0x7ffa70004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e724f724bb2ef7b server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:07:03.046 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:03.063+0000 7ffa61600640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-03-08T21:07:03.046 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:03.063+0000 7ffa61600640 1 -- 172.21.15.107:0/2329007784 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa7000db30 con 0x557a32df2570 2024-03-08T21:07:03.046 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:03.063+0000 7ffa61600640 1 -- 172.21.15.107:0/2329007784 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8541+0+0 (secure 0 0 0) 0x7ffa7000dcd0 con 0x557a32df2570 2024-03-08T21:07:03.046 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:03.063+0000 7ffa8142f740 1 -- 172.21.15.107:0/2329007784 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ffa64005040 con 0x557a32df2570 2024-03-08T21:07:03.046 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:03.064+0000 7ffa61600640 1 -- 172.21.15.107:0/2329007784 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffa70004b20 con 0x557a32df2570 2024-03-08T21:07:03.047 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:03.064+0000 7ffa8142f740 10 client.15999 fetch_fsmap learned FSMap version 34 2024-03-08T21:07:03.047 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:03.064+0000 7ffa8142f740 1 -- 172.21.15.107:0/2329007784 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x557a32b2d620 con 0x557a32df2570 2024-03-08T21:07:03.047 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:03.064+0000 7ffa61600640 1 -- 172.21.15.107:0/2329007784 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7ffa64005040 con 0x557a32df2570 2024-03-08T21:07:03.047 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:03.064+0000 7ffa61600640 1 -- 172.21.15.107:0/2329007784 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 34) v1 ==== 7882+0+0 (secure 0 0 0) 0x7ffa7001d440 con 0x557a32df2570 2024-03-08T21:07:03.047 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:03.064+0000 7ffa8142f740 10 client.15999 fetch_fsmap finished waiting for FSMap version 34 2024-03-08T21:07:03.047 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:03.064+0000 7ffa8142f740 10 client.15999 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-08T21:07:03.047 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:03.064+0000 7ffa8142f740 20 client.15999 populate_metadata read hostname 'smithi107' 2024-03-08T21:07:03.047 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:03.064+0000 7ffa8142f740 1 --2- 172.21.15.107:0/2329007784 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x557a32e02360 0x557a32e047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:07:03.047 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:03.064+0000 7ffa8142f740 4 client.15999 mds_command: new command op to 34475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-08T21:07:03.047 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:03.064+0000 7ffa8142f740 1 -- 172.21.15.107:0/2329007784 --> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557a32d59910 con 0x557a32e02360 2024-03-08T21:07:03.048 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:03.065+0000 7ffa77400640 1 --2- 172.21.15.107:0/2329007784 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x557a32e02360 0x557a32e047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:07:03.049 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:03.066+0000 7ffa77400640 1 --2- 172.21.15.107:0/2329007784 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x557a32e02360 0x557a32e047c0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f8e2e2258b5260c5 server_cookie=ed964ac415b7ae7f in_seq=0 out_seq=0 2024-03-08T21:07:03.049 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:03.066+0000 7ffa61600640 10 client.15999 ms_handle_connect on v2:172.21.15.149:6838/55290596 2024-03-08T21:07:03.049 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:03.067+0000 7ffa61600640 1 -- 172.21.15.107:0/2329007784 <== mds.0 v2:172.21.15.149:6838/55290596 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x557a32d59910 con 0x557a32e02360 2024-03-08T21:07:03.049 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:03.067+0000 7ffa61600640 10 client.15999 handle_command_reply: tid=0 2024-03-08T21:07:03.049 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:03.067+0000 7ffa8142f740 1 client.15999 shutdown 2024-03-08T21:07:03.049 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:03.068+0000 7ffa8142f740 1 -- 172.21.15.107:0/2329007784 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557a32df2570 msgr2=0x557a32def6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:07:03.049 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:03.068+0000 7ffa8142f740 1 --2- 172.21.15.107:0/2329007784 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557a32df2570 0x557a32def6e0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7ffa7001c880 tx=0x7ffa70004a40 comp rx=0 tx=0).stop 2024-03-08T21:07:03.049 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:03.068+0000 7ffa8142f740 1 -- 172.21.15.107:0/2329007784 shutdown_connections 2024-03-08T21:07:03.049 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:03.068+0000 7ffa8142f740 1 --2- 172.21.15.107:0/2329007784 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557a32df2570 0x557a32def6e0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:07:03.049 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:03.068+0000 7ffa8142f740 1 --2- 172.21.15.107:0/2329007784 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x557a32e02360 0x557a32e047c0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:07:03.049 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:03.068+0000 7ffa8142f740 1 --2- 172.21.15.107:0/2329007784 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557a32de97a0 0x557a32def1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:07:03.049 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:03.068+0000 7ffa8142f740 1 --2- 172.21.15.107:0/2329007784 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a32de8e80 0x557a32df1e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:07:03.049 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:03.068+0000 7ffa8142f740 1 -- 172.21.15.107:0/2329007784 >> 172.21.15.107:0/2329007784 conn(0x557a32de37a0 msgr2=0x557a32de54f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:07:03.049 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:03.068+0000 7ffa61600640 0 client.15999 ms_handle_reset on v2:172.21.15.149:6838/55290596 2024-03-08T21:07:03.050 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:03.068+0000 7ffa8142f740 1 -- 172.21.15.107:0/2329007784 shutdown_connections 2024-03-08T21:07:03.050 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:03.068+0000 7ffa8142f740 1 -- 172.21.15.107:0/2329007784 wait complete. 2024-03-08T21:07:03.050 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:03.068+0000 7ffa8142f740 20 client.15999 trim_cache size 0 max 16384 2024-03-08T21:07:03.062 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:03.083+0000 7ffa8142f740 1 Processor -- start 2024-03-08T21:07:03.063 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:03.083+0000 7ffa8142f740 1 -- start start 2024-03-08T21:07:03.063 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:03.083+0000 7ffa8142f740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557a32de8e80 0x557a32dee300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:07:03.063 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:03.083+0000 7ffa8142f740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557a32de97a0 0x557a32de2680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:07:03.063 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:03.083+0000 7ffa8142f740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a32df2570 0x557a32df18d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:07:03.063 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:03.083+0000 7ffa8142f740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa781aecc0 con 0x557a32de97a0 2024-03-08T21:07:03.063 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:03.083+0000 7ffa8142f740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa781b0640 con 0x557a32de8e80 2024-03-08T21:07:03.063 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:03.083+0000 7ffa8142f740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa781af320 con 0x557a32df2570 2024-03-08T21:07:03.063 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:03.083+0000 7ffa76a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557a32de97a0 0x557a32de2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:07:03.063 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:03.083+0000 7ffa76a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557a32de97a0 0x557a32de2680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:54422/0 (socket says 172.21.15.107:54422) 2024-03-08T21:07:03.063 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:03.083+0000 7ffa76a00640 1 -- 172.21.15.107:0/1529871239 learned_addr learned my addr 172.21.15.107:0/1529871239 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:07:03.063 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:03.084+0000 7ffa77e00640 1 --2- 172.21.15.107:0/1529871239 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a32df2570 0x557a32df18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:07:03.063 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:03.084+0000 7ffa77400640 1 --2- 172.21.15.107:0/1529871239 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557a32de8e80 0x557a32dee300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:07:03.064 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:03.084+0000 7ffa76a00640 1 -- 172.21.15.107:0/1529871239 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a32df2570 msgr2=0x557a32df18d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:07:03.064 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:03.084+0000 7ffa76a00640 1 --2- 172.21.15.107:0/1529871239 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a32df2570 0x557a32df18d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:07:03.064 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:03.084+0000 7ffa76a00640 1 -- 172.21.15.107:0/1529871239 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557a32de8e80 msgr2=0x557a32dee300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:07:03.064 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:03.084+0000 7ffa76a00640 1 --2- 172.21.15.107:0/1529871239 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557a32de8e80 0x557a32dee300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:07:03.064 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:03.084+0000 7ffa76a00640 1 -- 172.21.15.107:0/1529871239 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557a32ddf070 con 0x557a32de97a0 2024-03-08T21:07:03.066 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:03.084+0000 7ffa76a00640 1 --2- 172.21.15.107:0/1529871239 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557a32de97a0 0x557a32de2680 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7ffa64042000 tx=0x7ffa64000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b72d803f3bca59f1 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:07:03.066 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:03.084+0000 7ffa63e00640 1 -- 172.21.15.107:0/1529871239 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa64041040 con 0x557a32de97a0 2024-03-08T21:07:03.067 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:03.085+0000 7ffa63e00640 1 -- 172.21.15.107:0/1529871239 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7ffa6404b070 con 0x557a32de97a0 2024-03-08T21:07:03.067 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:03.085+0000 7ffa63e00640 1 -- 172.21.15.107:0/1529871239 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa64003c00 con 0x557a32de97a0 2024-03-08T21:07:03.067 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:03.085+0000 7ffa8142f740 1 -- 172.21.15.107:0/1529871239 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557a32de97a0 msgr2=0x557a32de2680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:07:03.067 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:03.085+0000 7ffa8142f740 1 --2- 172.21.15.107:0/1529871239 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557a32de97a0 0x557a32de2680 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7ffa64042000 tx=0x7ffa64000bc0 comp rx=0 tx=0).stop 2024-03-08T21:07:03.067 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:03.085+0000 7ffa8142f740 1 -- 172.21.15.107:0/1529871239 shutdown_connections 2024-03-08T21:07:03.067 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:03.085+0000 7ffa8142f740 1 --2- 172.21.15.107:0/1529871239 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a32df2570 0x557a32df18d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:07:03.067 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:03.085+0000 7ffa8142f740 1 --2- 172.21.15.107:0/1529871239 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557a32de97a0 0x557a32de2680 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:07:03.067 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:03.085+0000 7ffa8142f740 1 --2- 172.21.15.107:0/1529871239 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557a32de8e80 0x557a32dee300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:07:03.067 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:03.085+0000 7ffa8142f740 1 -- 172.21.15.107:0/1529871239 >> 172.21.15.107:0/1529871239 conn(0x557a32de37a0 msgr2=0x557a32de1f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:07:03.067 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:03.085+0000 7ffa8142f740 1 -- 172.21.15.107:0/1529871239 shutdown_connections 2024-03-08T21:07:03.067 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:03.085+0000 7ffa8142f740 1 -- 172.21.15.107:0/1529871239 wait complete. 2024-03-08T21:07:03.067 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:03.086+0000 7ffa8142f740 1 -- start start 2024-03-08T21:07:03.068 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:03.086+0000 7ffa8142f740 1 Processor -- start 2024-03-08T21:07:03.068 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:03.087+0000 7ffa8142f740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557a32de8e80 0x557a32df8880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:07:03.068 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:03.087+0000 7ffa8142f740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a32de97a0 0x557a32df8dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:07:03.068 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:03.087+0000 7ffa8142f740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557a32df2570 0x557a32e52a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:07:03.068 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:03.087+0000 7ffa8142f740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa781affe0 con 0x557a32df2570 2024-03-08T21:07:03.068 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:03.087+0000 7ffa8142f740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa781af980 con 0x557a32de8e80 2024-03-08T21:07:03.068 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:03.087+0000 7ffa8142f740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa781acf60 con 0x557a32de97a0 2024-03-08T21:07:03.068 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:03.087+0000 7ffa8142f740 10 client.0 _ll_register_callbacks cb 0x7ffa64009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T21:07:03.068 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:03.087+0000 7ffa77e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557a32df2570 0x557a32e52a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:07:03.068 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:03.087+0000 7ffa77e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557a32df2570 0x557a32e52a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:54438/0 (socket says 172.21.15.107:54438) 2024-03-08T21:07:03.068 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:03.087+0000 7ffa77e00640 1 -- 172.21.15.107:0/3010254661 learned_addr learned my addr 172.21.15.107:0/3010254661 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:07:03.068 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:03.087+0000 7ffa77400640 1 --2- 172.21.15.107:0/3010254661 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557a32de8e80 0x557a32df8880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:07:03.068 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:03.087+0000 7ffa76a00640 1 --2- 172.21.15.107:0/3010254661 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a32de97a0 0x557a32df8dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:07:03.068 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:03.087+0000 7ffa77e00640 1 -- 172.21.15.107:0/3010254661 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a32de97a0 msgr2=0x557a32df8dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:07:03.069 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:03.087+0000 7ffa77e00640 1 --2- 172.21.15.107:0/3010254661 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a32de97a0 0x557a32df8dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:07:03.069 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:03.087+0000 7ffa77e00640 1 -- 172.21.15.107:0/3010254661 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557a32de8e80 msgr2=0x557a32df8880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:07:03.069 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:03.087+0000 7ffa77e00640 1 --2- 172.21.15.107:0/3010254661 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557a32de8e80 0x557a32df8880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:07:03.069 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:03.087+0000 7ffa77e00640 1 -- 172.21.15.107:0/3010254661 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557a32e53080 con 0x557a32df2570 2024-03-08T21:07:03.069 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:03.087+0000 7ffa76a00640 1 --2- 172.21.15.107:0/3010254661 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a32de97a0 0x557a32df8dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T21:07:03.069 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:03.088+0000 7ffa77e00640 1 --2- 172.21.15.107:0/3010254661 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557a32df2570 0x557a32e52a10 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7ffa700092a0 tx=0x7ffa70012ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42c0275b9fb07e04 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:07:03.069 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:03.088+0000 7ffa61600640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-03-08T21:07:03.071 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:03.088+0000 7ffa61600640 1 -- 172.21.15.107:0/3010254661 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa7000db30 con 0x557a32df2570 2024-03-08T21:07:03.071 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:03.088+0000 7ffa61600640 1 -- 172.21.15.107:0/3010254661 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8541+0+0 (secure 0 0 0) 0x7ffa7000dcd0 con 0x557a32df2570 2024-03-08T21:07:03.071 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:03.088+0000 7ffa8142f740 1 -- 172.21.15.107:0/3010254661 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ffa6404b070 con 0x557a32df2570 2024-03-08T21:07:03.071 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:03.088+0000 7ffa61600640 1 -- 172.21.15.107:0/3010254661 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffa7003d420 con 0x557a32df2570 2024-03-08T21:07:03.071 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:03.088+0000 7ffa8142f740 10 client.16005 fetch_fsmap learned FSMap version 34 2024-03-08T21:07:03.071 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:03.088+0000 7ffa8142f740 1 -- 172.21.15.107:0/3010254661 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x557a32e5a3f0 con 0x557a32df2570 2024-03-08T21:07:03.071 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:03.089+0000 7ffa61600640 1 -- 172.21.15.107:0/3010254661 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7ffa6404b070 con 0x557a32df2570 2024-03-08T21:07:03.071 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:03.089+0000 7ffa61600640 1 -- 172.21.15.107:0/3010254661 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 34) v1 ==== 7882+0+0 (secure 0 0 0) 0x7ffa7002d410 con 0x557a32df2570 2024-03-08T21:07:03.071 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:03.089+0000 7ffa8142f740 10 client.16005 fetch_fsmap finished waiting for FSMap version 34 2024-03-08T21:07:03.071 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:03.089+0000 7ffa8142f740 10 client.16005 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-08T21:07:03.071 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:03.089+0000 7ffa8142f740 20 client.16005 populate_metadata read hostname 'smithi107' 2024-03-08T21:07:03.071 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:03.089+0000 7ffa8142f740 1 --2- 172.21.15.107:0/3010254661 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x557a32e5aa10 0x557a32e5ce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:07:03.071 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:03.089+0000 7ffa8142f740 4 client.16005 mds_command: new command op to 34475 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-08T21:07:03.072 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:03.089+0000 7ffa8142f740 1 -- 172.21.15.107:0/3010254661 --> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x557a32d53c60 con 0x557a32e5aa10 2024-03-08T21:07:03.072 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:03.090+0000 7ffa77400640 1 --2- 172.21.15.107:0/3010254661 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x557a32e5aa10 0x557a32e5ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:07:03.072 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:03.090+0000 7ffa77400640 1 --2- 172.21.15.107:0/3010254661 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x557a32e5aa10 0x557a32e5ce30 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7112e8875c7649e6 server_cookie=d834ec93145f09c1 in_seq=0 out_seq=0 2024-03-08T21:07:03.072 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:03.090+0000 7ffa61600640 10 client.16005 ms_handle_connect on v2:172.21.15.149:6838/55290596 2024-03-08T21:07:03.124 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:07:02 smithi204 ceph-mon[34935]: pgmap v201: 97 pgs: 97 active+clean; 4.0 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:07:03.213 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:07:02 smithi107 ceph-mon[31381]: pgmap v201: 97 pgs: 97 active+clean; 4.0 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:07:04.177 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:04.198+0000 7ffa61600640 1 -- 172.21.15.107:0/3010254661 <== mds.0 v2:172.21.15.149:6838/55290596 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x557a32d53c60 con 0x557a32e5aa10 2024-03-08T21:07:04.177 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:04.198+0000 7ffa61600640 10 client.16005 handle_command_reply: tid=0 2024-03-08T21:07:04.177 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:04.198+0000 7ffa8142f740 1 client.16005 shutdown 2024-03-08T21:07:04.177 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:04.198+0000 7ffa8142f740 1 -- 172.21.15.107:0/3010254661 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557a32df2570 msgr2=0x557a32e52a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:07:04.177 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:04.198+0000 7ffa8142f740 1 --2- 172.21.15.107:0/3010254661 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557a32df2570 0x557a32e52a10 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7ffa700092a0 tx=0x7ffa70012ea0 comp rx=0 tx=0).stop 2024-03-08T21:07:04.178 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:04.199+0000 7ffa8142f740 1 -- 172.21.15.107:0/3010254661 shutdown_connections 2024-03-08T21:07:04.179 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:04.199+0000 7ffa8142f740 1 --2- 172.21.15.107:0/3010254661 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557a32df2570 0x557a32e52a10 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:07:04.179 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:04.199+0000 7ffa8142f740 1 --2- 172.21.15.107:0/3010254661 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557a32de97a0 0x557a32df8dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:07:04.179 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:04.199+0000 7ffa8142f740 1 --2- 172.21.15.107:0/3010254661 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x557a32e5aa10 0x557a32e5ce30 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:07:04.179 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:04.199+0000 7ffa8142f740 1 --2- 172.21.15.107:0/3010254661 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557a32de8e80 0x557a32df8880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:07:04.179 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:04.199+0000 7ffa61600640 0 client.16005 ms_handle_reset on v2:172.21.15.149:6838/55290596 2024-03-08T21:07:04.179 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:04.199+0000 7ffa8142f740 1 -- 172.21.15.107:0/3010254661 >> 172.21.15.107:0/3010254661 conn(0x557a32de37a0 msgr2=0x557a32de1f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:07:04.179 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:04.199+0000 7ffa8142f740 1 -- 172.21.15.107:0/3010254661 shutdown_connections 2024-03-08T21:07:04.179 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:04.199+0000 7ffa8142f740 1 -- 172.21.15.107:0/3010254661 wait complete. 2024-03-08T21:07:04.179 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:04.200+0000 7ffa8142f740 20 client.16005 trim_cache size 0 max 16384 2024-03-08T21:07:04.179 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:{ 2024-03-08T21:07:04.180 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "message": "", 2024-03-08T21:07:04.180 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "return_code": 0 2024-03-08T21:07:04.180 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:} 2024-03-08T21:07:04.181 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:04.201+0000 7ffa7d000640 1 -- 172.21.15.107:0/1889276392 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7ffa5c0a2960 msgr2=0x7ffa5c0a4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:07:04.181 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:04.201+0000 7ffa7d000640 1 --2- 172.21.15.107:0/1889276392 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7ffa5c0a2960 0x7ffa5c0a4e20 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ffa70005c40 tx=0x7ffa70009200 comp rx=0 tx=0).stop 2024-03-08T21:07:04.181 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:04.201+0000 7ffa7d000640 1 -- 172.21.15.107:0/1889276392 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffa780736c0 msgr2=0x7ffa7819f7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:07:04.181 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:04.201+0000 7ffa7d000640 1 --2- 172.21.15.107:0/1889276392 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffa780736c0 0x7ffa7819f7d0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7ffa6800d950 tx=0x7ffa6800de10 comp rx=0 tx=0).stop 2024-03-08T21:07:04.181 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:04.202+0000 7ffa7d000640 1 -- 172.21.15.107:0/1889276392 shutdown_connections 2024-03-08T21:07:04.181 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:04.202+0000 7ffa7d000640 1 --2- 172.21.15.107:0/1889276392 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7ffa5c0a2960 0x7ffa5c0a4e20 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:07:04.181 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:04.202+0000 7ffa7d000640 1 --2- 172.21.15.107:0/1889276392 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ffa7810d990 0x7ffa78199980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:07:04.181 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:04.202+0000 7ffa7d000640 1 --2- 172.21.15.107:0/1889276392 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffa78073fe0 0x7ffa7819fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:07:04.181 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:04.202+0000 7ffa7d000640 1 --2- 172.21.15.107:0/1889276392 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffa780736c0 0x7ffa7819f7d0 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:07:04.181 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:04.202+0000 7ffa7d000640 1 -- 172.21.15.107:0/1889276392 >> 172.21.15.107:0/1889276392 conn(0x7ffa780fcc00 msgr2=0x7ffa780fd970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:07:04.181 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:04.202+0000 7ffa7d000640 1 -- 172.21.15.107:0/1889276392 shutdown_connections 2024-03-08T21:07:04.181 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:04.202+0000 7ffa7d000640 1 -- 172.21.15.107:0/1889276392 wait complete. 2024-03-08T21:07:05.374 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:07:05 smithi204 ceph-mon[34935]: pgmap v202: 97 pgs: 97 active+clean; 4.7 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 103 KiB/s wr, 32 op/s 2024-03-08T21:07:05.375 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:07:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[70694]: *** Caught signal (Segmentation fault) ** 2024-03-08T21:07:05.375 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:07:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[70694]: in thread 7f41cd800700 thread_name:md_log_replay 2024-03-08T21:07:05.375 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:07:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[70694]: ceph version 19.0.0-1965-g7d63482a (7d63482aa89628d733e298984a762389b625c761) squid (dev) 2024-03-08T21:07:05.375 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:07:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[70694]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f41e1e0dd20] 2024-03-08T21:07:05.375 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:07:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[70694]: 2: /usr/bin/ceph-mds(+0x503eff) [0x561fe2658eff] 2024-03-08T21:07:05.376 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:07:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[70694]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x561fe265c773] 2024-03-08T21:07:05.376 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:07:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[70694]: 4: (MDLog::_replay_thread()+0x617) [0x561fe26b8587] 2024-03-08T21:07:05.376 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:07:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[70694]: 5: (MDLog::ReplayThread::entry()+0x11) [0x561fe26c1091] 2024-03-08T21:07:05.376 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:07:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[70694]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7f41e1e031ca] 2024-03-08T21:07:05.376 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:07:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[70694]: 7: clone() 2024-03-08T21:07:05.376 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:07:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[70694]: 2024-03-08T21:07:05.055+0000 7f41cd800700 -1 *** Caught signal (Segmentation fault) ** 2024-03-08T21:07:05.376 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:07:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[70694]: in thread 7f41cd800700 thread_name:md_log_replay 2024-03-08T21:07:05.376 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:07:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[70694]: 2024-03-08T21:07:05.376 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:07:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[70694]: ceph version 19.0.0-1965-g7d63482a (7d63482aa89628d733e298984a762389b625c761) squid (dev) 2024-03-08T21:07:05.376 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:07:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[70694]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f41e1e0dd20] 2024-03-08T21:07:05.376 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:07:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[70694]: 2: /usr/bin/ceph-mds(+0x503eff) [0x561fe2658eff] 2024-03-08T21:07:05.377 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:07:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[70694]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x561fe265c773] 2024-03-08T21:07:05.377 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:07:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[70694]: 4: (MDLog::_replay_thread()+0x617) [0x561fe26b8587] 2024-03-08T21:07:05.377 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:07:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[70694]: 5: (MDLog::ReplayThread::entry()+0x11) [0x561fe26c1091] 2024-03-08T21:07:05.377 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:07:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[70694]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7f41e1e031ca] 2024-03-08T21:07:05.377 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:07:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[70694]: 7: clone() 2024-03-08T21:07:05.377 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:07:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[70694]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-08T21:07:05.377 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:07:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[70694]: 2024-03-08T21:07:05.377 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:07:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[70694]: 0> 2024-03-08T21:07:05.055+0000 7f41cd800700 -1 *** Caught signal (Segmentation fault) ** 2024-03-08T21:07:05.377 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:07:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[70694]: in thread 7f41cd800700 thread_name:md_log_replay 2024-03-08T21:07:05.377 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:07:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[70694]: 2024-03-08T21:07:05.377 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:07:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[70694]: ceph version 19.0.0-1965-g7d63482a (7d63482aa89628d733e298984a762389b625c761) squid (dev) 2024-03-08T21:07:05.377 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:07:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[70694]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f41e1e0dd20] 2024-03-08T21:07:05.377 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:07:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[70694]: 2: /usr/bin/ceph-mds(+0x503eff) [0x561fe2658eff] 2024-03-08T21:07:05.378 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:07:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[70694]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x561fe265c773] 2024-03-08T21:07:05.378 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:07:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[70694]: 4: (MDLog::_replay_thread()+0x617) [0x561fe26b8587] 2024-03-08T21:07:05.378 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:07:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[70694]: 5: (MDLog::ReplayThread::entry()+0x11) [0x561fe26c1091] 2024-03-08T21:07:05.378 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:07:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[70694]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7f41e1e031ca] 2024-03-08T21:07:05.378 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:07:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[70694]: 7: clone() 2024-03-08T21:07:05.378 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:07:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[70694]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-08T21:07:05.378 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:07:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[70694]: 2024-03-08T21:07:05.378 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:07:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[70694]: 0> 2024-03-08T21:07:05.055+0000 7f41cd800700 -1 *** Caught signal (Segmentation fault) ** 2024-03-08T21:07:05.378 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:07:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[70694]: in thread 7f41cd800700 thread_name:md_log_replay 2024-03-08T21:07:05.379 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:07:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[70694]: 2024-03-08T21:07:05.379 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:07:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[70694]: ceph version 19.0.0-1965-g7d63482a (7d63482aa89628d733e298984a762389b625c761) squid (dev) 2024-03-08T21:07:05.379 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:07:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[70694]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f41e1e0dd20] 2024-03-08T21:07:05.379 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:07:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[70694]: 2: /usr/bin/ceph-mds(+0x503eff) [0x561fe2658eff] 2024-03-08T21:07:05.379 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:07:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[70694]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x561fe265c773] 2024-03-08T21:07:05.379 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:07:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[70694]: 4: (MDLog::_replay_thread()+0x617) [0x561fe26b8587] 2024-03-08T21:07:05.379 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:07:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[70694]: 5: (MDLog::ReplayThread::entry()+0x11) [0x561fe26c1091] 2024-03-08T21:07:05.379 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:07:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[70694]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7f41e1e031ca] 2024-03-08T21:07:05.379 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:07:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[70694]: 7: clone() 2024-03-08T21:07:05.379 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:07:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[70694]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-08T21:07:05.379 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:07:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[70694]: 2024-03-08T21:07:05.379 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:07:05 smithi204 podman[71013]: 2024-03-08 21:07:05.170491934 +0000 UTC m=+0.041426232 container died bdc11ef2f39f583a2df227a3d6e704b46acfad788012076752a4bf7ab88301c5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c, io.buildah.version=1.33.5, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=rishabh-2024mar8-7d63482, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-08T21:07:05.463 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:07:05 smithi107 ceph-mon[31381]: pgmap v202: 97 pgs: 97 active+clean; 4.7 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 103 KiB/s wr, 32 op/s 2024-03-08T21:07:05.511 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:07:05 smithi149 ceph-mon[35042]: pgmap v202: 97 pgs: 97 active+clean; 4.7 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 103 KiB/s wr, 32 op/s 2024-03-08T21:07:06.133 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:07:05 smithi204 podman[71013]: 2024-03-08 21:07:05.980393474 +0000 UTC m=+0.851327815 container remove bdc11ef2f39f583a2df227a3d6e704b46acfad788012076752a4bf7ab88301c5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=rishabh-2024mar8-7d63482, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-08T21:07:06.133 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:07:05 smithi204 systemd[1]: ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715@mds.c.service: Main process exited, code=exited, status=139/n/a 2024-03-08T21:07:06.437 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:07:06 smithi204 ceph-mon[34935]: osdmap e93: 12 total, 12 up, 12 in 2024-03-08T21:07:06.463 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:07:06 smithi107 ceph-mon[31381]: osdmap e93: 12 total, 12 up, 12 in 2024-03-08T21:07:06.511 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:07:06 smithi149 ceph-mon[35042]: osdmap e93: 12 total, 12 up, 12 in 2024-03-08T21:07:06.875 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:07:06 smithi204 systemd[1]: ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715@mds.c.service: Failed with result 'exit-code'. 2024-03-08T21:07:06.875 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:07:06 smithi204 systemd[1]: ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715@mds.c.service: Consumed 1.811s CPU time. 2024-03-08T21:07:07.464 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:07:07 smithi107 ceph-mon[31381]: pgmap v204: 97 pgs: 97 active+clean; 4.7 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 124 KiB/s wr, 33 op/s 2024-03-08T21:07:07.511 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:07:07 smithi149 ceph-mon[35042]: pgmap v204: 97 pgs: 97 active+clean; 4.7 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 124 KiB/s wr, 33 op/s 2024-03-08T21:07:07.624 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:07:07 smithi204 ceph-mon[34935]: pgmap v204: 97 pgs: 97 active+clean; 4.7 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 124 KiB/s wr, 33 op/s 2024-03-08T21:07:09.011 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:07:08 smithi149 ceph-mon[35042]: pgmap v205: 97 pgs: 97 active+clean; 4.7 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 124 KiB/s wr, 33 op/s 2024-03-08T21:07:09.124 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:07:08 smithi204 ceph-mon[34935]: pgmap v205: 97 pgs: 97 active+clean; 4.7 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 124 KiB/s wr, 33 op/s 2024-03-08T21:07:09.213 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:07:08 smithi107 ceph-mon[31381]: pgmap v205: 97 pgs: 97 active+clean; 4.7 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 124 KiB/s wr, 33 op/s 2024-03-08T21:07:10.463 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:07:10 smithi107 ceph-mon[31381]: osdmap e94: 12 total, 12 up, 12 in 2024-03-08T21:07:10.511 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:07:10 smithi149 ceph-mon[35042]: osdmap e94: 12 total, 12 up, 12 in 2024-03-08T21:07:10.624 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:07:10 smithi204 ceph-mon[34935]: osdmap e94: 12 total, 12 up, 12 in 2024-03-08T21:07:11.464 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:07:11 smithi107 ceph-mon[31381]: pgmap v207: 97 pgs: 97 active+clean; 4.7 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 155 KiB/s wr, 41 op/s 2024-03-08T21:07:11.511 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:07:11 smithi149 ceph-mon[35042]: pgmap v207: 97 pgs: 97 active+clean; 4.7 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 155 KiB/s wr, 41 op/s 2024-03-08T21:07:11.624 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:07:11 smithi204 ceph-mon[34935]: pgmap v207: 97 pgs: 97 active+clean; 4.7 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 155 KiB/s wr, 41 op/s 2024-03-08T21:07:13.011 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:07:12 smithi149 ceph-mon[35042]: pgmap v208: 97 pgs: 97 active+clean; 4.7 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 383 B/s wr, 6 op/s 2024-03-08T21:07:13.124 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:07:12 smithi204 ceph-mon[34935]: pgmap v208: 97 pgs: 97 active+clean; 4.7 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 383 B/s wr, 6 op/s 2024-03-08T21:07:13.214 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:07:12 smithi107 ceph-mon[31381]: pgmap v208: 97 pgs: 97 active+clean; 4.7 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 383 B/s wr, 6 op/s 2024-03-08T21:07:15.011 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:07:14 smithi149 ceph-mon[35042]: pgmap v209: 97 pgs: 97 active+clean; 4.7 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 431 B/s wr, 9 op/s 2024-03-08T21:07:15.124 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:07:14 smithi204 ceph-mon[34935]: pgmap v209: 97 pgs: 97 active+clean; 4.7 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 431 B/s wr, 9 op/s 2024-03-08T21:07:15.214 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:07:14 smithi107 ceph-mon[31381]: pgmap v209: 97 pgs: 97 active+clean; 4.7 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 431 B/s wr, 9 op/s 2024-03-08T21:07:17.011 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:07:16 smithi149 ceph-mon[35042]: pgmap v210: 97 pgs: 97 active+clean; 4.7 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 409 B/s wr, 8 op/s 2024-03-08T21:07:17.015 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:07:16 smithi107 ceph-mon[31381]: pgmap v210: 97 pgs: 97 active+clean; 4.7 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 409 B/s wr, 8 op/s 2024-03-08T21:07:17.025 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:07:16 smithi204 ceph-mon[34935]: pgmap v210: 97 pgs: 97 active+clean; 4.7 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 409 B/s wr, 8 op/s 2024-03-08T21:07:17.025 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:07:16 smithi204 systemd[1]: ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715@mds.c.service: Scheduled restart job, restart counter is at 4. 2024-03-08T21:07:17.025 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:07:16 smithi204 systemd[1]: Stopped Ceph mds.c for dc20f0f8-dd8c-11ee-95c6-87774f69a715. 2024-03-08T21:07:17.025 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:07:16 smithi204 systemd[1]: ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715@mds.c.service: Consumed 1.811s CPU time. 2024-03-08T21:07:17.025 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:07:16 smithi204 systemd[1]: Starting Ceph mds.c for dc20f0f8-dd8c-11ee-95c6-87774f69a715... 2024-03-08T21:07:17.323 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:07:17 smithi204 podman[71108]: 2024-03-08 21:07:16.956325645 +0000 UTC m=+0.041997369 image pull a272146cb810b1fd4a53627f019472132e30966d5c25b01bfc44a001295f1249 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd 2024-03-08T21:07:17.324 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:07:17 smithi204 podman[71108]: 2024-03-08 21:07:17.073274243 +0000 UTC m=+0.158946017 container create ac85e479b71fcf12a3e9715687d6094abecdd1de10db12aa559758bfc17e216a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, RELEASE=rishabh-2024mar8-7d63482, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-08T21:07:17.334 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.353+0000 7ff5d4600640 1 -- 172.21.15.107:0/1917434923 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff5cc073fb0 msgr2=0x7ff5cc111d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:07:17.334 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.353+0000 7ff5d4600640 1 --2- 172.21.15.107:0/1917434923 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff5cc073fb0 0x7ff5cc111d70 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7ff5b8009900 tx=0x7ff5b802bcd0 comp rx=0 tx=0).stop 2024-03-08T21:07:17.334 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.354+0000 7ff5d4600640 1 -- 172.21.15.107:0/1917434923 shutdown_connections 2024-03-08T21:07:17.334 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.354+0000 7ff5d4600640 1 --2- 172.21.15.107:0/1917434923 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff5cc1122b0 0x7ff5cc1146a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:07:17.334 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.354+0000 7ff5d4600640 1 --2- 172.21.15.107:0/1917434923 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff5cc073fb0 0x7ff5cc111d70 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:07:17.334 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.354+0000 7ff5d4600640 1 --2- 172.21.15.107:0/1917434923 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff5cc073690 0x7ff5cc073a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:07:17.334 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.354+0000 7ff5d4600640 1 -- 172.21.15.107:0/1917434923 >> 172.21.15.107:0/1917434923 conn(0x7ff5cc0fcdb0 msgr2=0x7ff5cc0ff1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:07:17.334 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.354+0000 7ff5d4600640 1 -- 172.21.15.107:0/1917434923 shutdown_connections 2024-03-08T21:07:17.334 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.354+0000 7ff5d4600640 1 -- 172.21.15.107:0/1917434923 wait complete. 2024-03-08T21:07:17.336 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.355+0000 7ff5d4600640 1 Processor -- start 2024-03-08T21:07:17.336 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.355+0000 7ff5d4600640 1 -- start start 2024-03-08T21:07:17.336 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.355+0000 7ff5d4600640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff5cc073690 0x7ff5cc19e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:07:17.336 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.355+0000 7ff5d4600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff5cc073fb0 0x7ff5cc19e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:07:17.336 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.355+0000 7ff5d4600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff5cc1122b0 0x7ff5cc1a2950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:07:17.336 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.355+0000 7ff5d4600640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5cc117bf0 con 0x7ff5cc073fb0 2024-03-08T21:07:17.336 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.355+0000 7ff5d4600640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5cc117a70 con 0x7ff5cc073690 2024-03-08T21:07:17.336 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.355+0000 7ff5d4600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5cc117d70 con 0x7ff5cc1122b0 2024-03-08T21:07:17.336 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.356+0000 7ff5d1200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff5cc073fb0 0x7ff5cc19e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:07:17.337 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.356+0000 7ff5d1200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff5cc073fb0 0x7ff5cc19e5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:34990/0 (socket says 172.21.15.107:34990) 2024-03-08T21:07:17.337 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.356+0000 7ff5d1200640 1 -- 172.21.15.107:0/3156296698 learned_addr learned my addr 172.21.15.107:0/3156296698 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:07:17.337 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.356+0000 7ff5d1c00640 1 --2- 172.21.15.107:0/3156296698 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff5cc073690 0x7ff5cc19e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:07:17.337 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.356+0000 7ff5d2600640 1 --2- 172.21.15.107:0/3156296698 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff5cc1122b0 0x7ff5cc1a2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:07:17.337 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.356+0000 7ff5d1200640 1 -- 172.21.15.107:0/3156296698 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff5cc1122b0 msgr2=0x7ff5cc1a2950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:07:17.337 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.356+0000 7ff5d1200640 1 --2- 172.21.15.107:0/3156296698 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff5cc1122b0 0x7ff5cc1a2950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:07:17.337 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.356+0000 7ff5d1200640 1 -- 172.21.15.107:0/3156296698 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff5cc073690 msgr2=0x7ff5cc19e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:07:17.337 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.356+0000 7ff5d1200640 1 --2- 172.21.15.107:0/3156296698 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff5cc073690 0x7ff5cc19e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:07:17.337 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.356+0000 7ff5d1200640 1 -- 172.21.15.107:0/3156296698 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5cc1a3080 con 0x7ff5cc073fb0 2024-03-08T21:07:17.338 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.357+0000 7ff5d1200640 1 --2- 172.21.15.107:0/3156296698 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff5cc073fb0 0x7ff5cc19e5d0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7ff5b8005ab0 tx=0x7ff5b802db40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:07:17.338 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.357+0000 7ff5ca000640 1 -- 172.21.15.107:0/3156296698 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5b8043020 con 0x7ff5cc073fb0 2024-03-08T21:07:17.338 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.357+0000 7ff5ca000640 1 -- 172.21.15.107:0/3156296698 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7ff5b80052b0 con 0x7ff5cc073fb0 2024-03-08T21:07:17.338 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.357+0000 7ff5ca000640 1 -- 172.21.15.107:0/3156296698 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5b8003c70 con 0x7ff5cc073fb0 2024-03-08T21:07:17.338 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.357+0000 7ff5d4600640 1 -- 172.21.15.107:0/3156296698 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5cc1a3310 con 0x7ff5cc073fb0 2024-03-08T21:07:17.338 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.357+0000 7ff5d4600640 1 -- 172.21.15.107:0/3156296698 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5cc1aac00 con 0x7ff5cc073fb0 2024-03-08T21:07:17.341 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.360+0000 7ff5d7363740 1 Processor -- start 2024-03-08T21:07:17.341 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.360+0000 7ff5d7363740 1 -- start start 2024-03-08T21:07:17.341 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.361+0000 7ff5ca000640 1 -- 172.21.15.107:0/3156296698 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) v1 ==== 137422+0+0 (secure 0 0 0) 0x7ff5b8046020 con 0x7ff5cc073fb0 2024-03-08T21:07:17.341 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.361+0000 7ff5d7363740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55894fb18c80 0x55894fb19060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:07:17.341 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.361+0000 7ff5d7363740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55894fb195a0 0x55894fb21e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:07:17.341 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.361+0000 7ff5d7363740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55894fb22370 0x55894fb24760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:07:17.341 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.361+0000 7ff5d7363740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55894fa7c3d0 con 0x55894fb195a0 2024-03-08T21:07:17.341 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.361+0000 7ff5d7363740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55894fa256b0 con 0x55894fb22370 2024-03-08T21:07:17.342 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.361+0000 7ff5d7363740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55894fa78470 con 0x55894fb18c80 2024-03-08T21:07:17.342 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.361+0000 7ff5d1c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55894fb195a0 0x55894fb21e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:07:17.342 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.361+0000 7ff5d1c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55894fb195a0 0x55894fb21e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:34996/0 (socket says 172.21.15.107:34996) 2024-03-08T21:07:17.342 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.361+0000 7ff5d1c00640 1 -- 172.21.15.107:0/1552758757 learned_addr learned my addr 172.21.15.107:0/1552758757 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:07:17.342 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.361+0000 7ff5d2600640 1 --2- 172.21.15.107:0/1552758757 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55894fb18c80 0x55894fb19060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:07:17.344 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.362+0000 7ff5d1200640 1 --2- 172.21.15.107:0/1552758757 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55894fb22370 0x55894fb24760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:07:17.344 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.362+0000 7ff5d1c00640 1 -- 172.21.15.107:0/1552758757 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55894fb18c80 msgr2=0x55894fb19060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:07:17.344 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.362+0000 7ff5d1c00640 1 --2- 172.21.15.107:0/1552758757 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55894fb18c80 0x55894fb19060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:07:17.344 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.362+0000 7ff5d1c00640 1 -- 172.21.15.107:0/1552758757 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55894fb22370 msgr2=0x55894fb24760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:07:17.344 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.362+0000 7ff5d1c00640 1 --2- 172.21.15.107:0/1552758757 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55894fb22370 0x55894fb24760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:07:17.344 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.362+0000 7ff5d1c00640 1 -- 172.21.15.107:0/1552758757 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55894fa4bb50 con 0x55894fb195a0 2024-03-08T21:07:17.344 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.362+0000 7ff5d1c00640 1 --2- 172.21.15.107:0/1552758757 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55894fb195a0 0x55894fb21e30 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7ff5bc00ae90 tx=0x7ff5bc00f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8e8e18bfebe90eb server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:07:17.345 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.362+0000 7ff5c2a00640 1 -- 172.21.15.107:0/1552758757 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5bc011850 con 0x55894fb195a0 2024-03-08T21:07:17.345 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.362+0000 7ff5c2a00640 1 -- 172.21.15.107:0/1552758757 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7ff5bc01a480 con 0x55894fb195a0 2024-03-08T21:07:17.345 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.362+0000 7ff5c2a00640 1 -- 172.21.15.107:0/1552758757 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5bc010610 con 0x55894fb195a0 2024-03-08T21:07:17.345 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.362+0000 7ff5ca000640 1 --2- 172.21.15.107:0/3156296698 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7ff5b00a2a10 0x7ff5b00a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:07:17.345 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.363+0000 7ff5ca000640 1 -- 172.21.15.107:0/3156296698 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 8585+0+0 (secure 0 0 0) 0x7ff5b80f1140 con 0x7ff5cc073fb0 2024-03-08T21:07:17.345 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.363+0000 7ff5d2600640 1 --2- 172.21.15.107:0/3156296698 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7ff5b00a2a10 0x7ff5b00a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:07:17.345 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.363+0000 7ff5d7363740 1 -- 172.21.15.107:0/1552758757 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55894fb195a0 msgr2=0x55894fb21e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:07:17.345 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.363+0000 7ff5d7363740 1 --2- 172.21.15.107:0/1552758757 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55894fb195a0 0x55894fb21e30 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7ff5bc00ae90 tx=0x7ff5bc00f730 comp rx=0 tx=0).stop 2024-03-08T21:07:17.345 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.363+0000 7ff5d7363740 1 -- 172.21.15.107:0/1552758757 shutdown_connections 2024-03-08T21:07:17.345 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.363+0000 7ff5d7363740 1 --2- 172.21.15.107:0/1552758757 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55894fb22370 0x55894fb24760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:07:17.345 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.363+0000 7ff5d7363740 1 --2- 172.21.15.107:0/1552758757 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55894fb195a0 0x55894fb21e30 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:07:17.345 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.363+0000 7ff5d7363740 1 --2- 172.21.15.107:0/1552758757 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55894fb18c80 0x55894fb19060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:07:17.345 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.363+0000 7ff5d7363740 1 -- 172.21.15.107:0/1552758757 >> 172.21.15.107:0/1552758757 conn(0x55894fb135a0 msgr2=0x55894fb13980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:07:17.345 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.364+0000 7ff5d7363740 1 -- 172.21.15.107:0/1552758757 shutdown_connections 2024-03-08T21:07:17.346 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.364+0000 7ff5d7363740 1 -- 172.21.15.107:0/1552758757 wait complete. 2024-03-08T21:07:17.346 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.364+0000 7ff5d2600640 1 --2- 172.21.15.107:0/3156296698 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7ff5b00a2a10 0x7ff5b00a4ed0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ff5cc06a9a0 tx=0x7ff5c4008040 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:07:17.346 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.364+0000 7ff5d7363740 1 -- start start 2024-03-08T21:07:17.346 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.364+0000 7ff5d7363740 1 Processor -- start 2024-03-08T21:07:17.349 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.365+0000 7ff5d7363740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55894fb18c80 0x55894fb21cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:07:17.349 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.365+0000 7ff5d7363740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55894fb195a0 0x55894fb1f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:07:17.349 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.365+0000 7ff5d7363740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55894fb22370 0x55894fb1f680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:07:17.349 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.365+0000 7ff5d7363740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55894fb24fa0 con 0x55894fb18c80 2024-03-08T21:07:17.349 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.365+0000 7ff5d7363740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55894fa389f0 con 0x55894fb22370 2024-03-08T21:07:17.349 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.365+0000 7ff5d7363740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55894fb25120 con 0x55894fb195a0 2024-03-08T21:07:17.349 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.365+0000 7ff5d7363740 10 client.0 _ll_register_callbacks cb 0x55894f790a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T21:07:17.350 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.365+0000 7ff5d1c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55894fb18c80 0x55894fb21cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:07:17.350 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.365+0000 7ff5d1c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55894fb18c80 0x55894fb21cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:35002/0 (socket says 172.21.15.107:35002) 2024-03-08T21:07:17.350 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.365+0000 7ff5d1c00640 1 -- 172.21.15.107:0/2878728409 learned_addr learned my addr 172.21.15.107:0/2878728409 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:07:17.350 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.365+0000 7ff5d1200640 1 --2- 172.21.15.107:0/2878728409 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55894fb195a0 0x55894fb1f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:07:17.350 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.365+0000 7ff5d2600640 1 --2- 172.21.15.107:0/2878728409 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55894fb22370 0x55894fb1f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:07:17.350 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.366+0000 7ff5d1c00640 1 -- 172.21.15.107:0/2878728409 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55894fb195a0 msgr2=0x55894fb1f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:07:17.350 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.366+0000 7ff5d1c00640 1 --2- 172.21.15.107:0/2878728409 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55894fb195a0 0x55894fb1f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:07:17.350 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.366+0000 7ff5d1c00640 1 -- 172.21.15.107:0/2878728409 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55894fb22370 msgr2=0x55894fb1f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:07:17.350 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.366+0000 7ff5d1c00640 1 --2- 172.21.15.107:0/2878728409 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55894fb22370 0x55894fb1f680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:07:17.350 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.366+0000 7ff5d1c00640 1 -- 172.21.15.107:0/2878728409 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55894f791210 con 0x55894fb18c80 2024-03-08T21:07:17.350 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.366+0000 7ff5d2600640 1 --2- 172.21.15.107:0/2878728409 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55894fb22370 0x55894fb1f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T21:07:17.350 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.366+0000 7ff5d1200640 1 --2- 172.21.15.107:0/2878728409 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55894fb195a0 0x55894fb1f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T21:07:17.351 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.366+0000 7ff5d1c00640 1 --2- 172.21.15.107:0/2878728409 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55894fb18c80 0x55894fb21cc0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7ff5bc000f80 tx=0x7ff5bc00ac00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3460d57531398d7e server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:07:17.351 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.366+0000 7ff5b7400640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-03-08T21:07:17.351 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.366+0000 7ff5b7400640 1 -- 172.21.15.107:0/2878728409 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5bc028060 con 0x55894fb18c80 2024-03-08T21:07:17.351 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.366+0000 7ff5b7400640 1 -- 172.21.15.107:0/2878728409 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8585+0+0 (secure 0 0 0) 0x7ff5bc004a40 con 0x55894fb18c80 2024-03-08T21:07:17.351 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.366+0000 7ff5d7363740 1 -- 172.21.15.107:0/2878728409 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff5bc01a480 con 0x55894fb18c80 2024-03-08T21:07:17.351 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.367+0000 7ff5b7400640 1 -- 172.21.15.107:0/2878728409 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff5bc01a980 con 0x55894fb18c80 2024-03-08T21:07:17.351 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.367+0000 7ff5d7363740 10 client.16017 fetch_fsmap learned FSMap version 34 2024-03-08T21:07:17.351 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.367+0000 7ff5d7363740 1 -- 172.21.15.107:0/2878728409 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55894f85d620 con 0x55894fb18c80 2024-03-08T21:07:17.351 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.367+0000 7ff5b7400640 1 -- 172.21.15.107:0/2878728409 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7ff5bc01a480 con 0x55894fb18c80 2024-03-08T21:07:17.351 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.367+0000 7ff5b7400640 1 -- 172.21.15.107:0/2878728409 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 34) v1 ==== 7882+0+0 (secure 0 0 0) 0x7ff5bc005040 con 0x55894fb18c80 2024-03-08T21:07:17.351 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.367+0000 7ff5d7363740 10 client.16017 fetch_fsmap finished waiting for FSMap version 34 2024-03-08T21:07:17.351 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.367+0000 7ff5d7363740 10 client.16017 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-08T21:07:17.351 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.367+0000 7ff5d7363740 20 client.16017 populate_metadata read hostname 'smithi107' 2024-03-08T21:07:17.351 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.367+0000 7ff5d7363740 1 --2- 172.21.15.107:0/2878728409 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55894fb32160 0x55894fb34550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:07:17.352 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.367+0000 7ff5d7363740 4 client.16017 mds_command: new command op to 34475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-08T21:07:17.352 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.367+0000 7ff5d7363740 1 -- 172.21.15.107:0/2878728409 --> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55894fa89910 con 0x55894fb32160 2024-03-08T21:07:17.352 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.368+0000 7ff5d1200640 1 --2- 172.21.15.107:0/2878728409 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55894fb32160 0x55894fb34550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:07:17.352 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.369+0000 7ff5d1200640 1 --2- 172.21.15.107:0/2878728409 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55894fb32160 0x55894fb34550 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3f8a168e077676e server_cookie=d7d89de43d6cff72 in_seq=0 out_seq=0 2024-03-08T21:07:17.352 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.369+0000 7ff5b7400640 10 client.16017 ms_handle_connect on v2:172.21.15.149:6838/55290596 2024-03-08T21:07:17.353 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.371+0000 7ff5b7400640 1 -- 172.21.15.107:0/2878728409 <== mds.0 v2:172.21.15.149:6838/55290596 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x7ff5b8082e50 con 0x55894fb32160 2024-03-08T21:07:17.353 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.371+0000 7ff5b7400640 10 client.16017 handle_command_reply: tid=0 2024-03-08T21:07:17.353 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.371+0000 7ff5d7363740 1 client.16017 shutdown 2024-03-08T21:07:17.353 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.371+0000 7ff5d7363740 1 -- 172.21.15.107:0/2878728409 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55894fb18c80 msgr2=0x55894fb21cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:07:17.353 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.371+0000 7ff5d7363740 1 --2- 172.21.15.107:0/2878728409 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55894fb18c80 0x55894fb21cc0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7ff5bc000f80 tx=0x7ff5bc00ac00 comp rx=0 tx=0).stop 2024-03-08T21:07:17.353 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.371+0000 7ff5d7363740 1 -- 172.21.15.107:0/2878728409 shutdown_connections 2024-03-08T21:07:17.354 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.371+0000 7ff5d7363740 1 --2- 172.21.15.107:0/2878728409 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55894fb32160 0x55894fb34550 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:07:17.354 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.371+0000 7ff5d7363740 1 --2- 172.21.15.107:0/2878728409 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55894fb22370 0x55894fb1f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:07:17.354 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.371+0000 7ff5d7363740 1 --2- 172.21.15.107:0/2878728409 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55894fb195a0 0x55894fb1f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:07:17.354 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.371+0000 7ff5d7363740 1 --2- 172.21.15.107:0/2878728409 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55894fb18c80 0x55894fb21cc0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:07:17.354 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.371+0000 7ff5b7400640 0 client.16017 ms_handle_reset on v2:172.21.15.149:6838/55290596 2024-03-08T21:07:17.354 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.371+0000 7ff5d7363740 1 -- 172.21.15.107:0/2878728409 >> 172.21.15.107:0/2878728409 conn(0x55894fb135a0 msgr2=0x55894fb152f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:07:17.354 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.371+0000 7ff5d7363740 1 -- 172.21.15.107:0/2878728409 shutdown_connections 2024-03-08T21:07:17.354 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.371+0000 7ff5d7363740 1 -- 172.21.15.107:0/2878728409 wait complete. 2024-03-08T21:07:17.354 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.371+0000 7ff5d7363740 20 client.16017 trim_cache size 0 max 16384 2024-03-08T21:07:17.370 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.391+0000 7ff5d7363740 1 Processor -- start 2024-03-08T21:07:17.370 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.391+0000 7ff5d7363740 1 -- start start 2024-03-08T21:07:17.372 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.391+0000 7ff5d7363740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55894fb18c80 0x55894fb1ed10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:07:17.372 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.391+0000 7ff5d7363740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55894fb195a0 0x55894fb217f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:07:17.372 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.391+0000 7ff5d7363740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55894fb22370 0x55894fb1c320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:07:17.372 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.391+0000 7ff5d7363740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5cc1b3320 con 0x55894fb195a0 2024-03-08T21:07:17.372 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.391+0000 7ff5d7363740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5cc1b4ca0 con 0x55894fb22370 2024-03-08T21:07:17.372 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.391+0000 7ff5d7363740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5cc1b3980 con 0x55894fb18c80 2024-03-08T21:07:17.372 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.391+0000 7ff5d1200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55894fb195a0 0x55894fb217f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:07:17.372 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.391+0000 7ff5d1200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55894fb195a0 0x55894fb217f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:35004/0 (socket says 172.21.15.107:35004) 2024-03-08T21:07:17.372 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.391+0000 7ff5d1200640 1 -- 172.21.15.107:0/1466262850 learned_addr learned my addr 172.21.15.107:0/1466262850 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:07:17.372 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.392+0000 7ff5d1c00640 1 --2- 172.21.15.107:0/1466262850 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55894fb18c80 0x55894fb1ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:07:17.373 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.392+0000 7ff5d2600640 1 --2- 172.21.15.107:0/1466262850 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55894fb22370 0x55894fb1c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:07:17.373 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.392+0000 7ff5d1200640 1 -- 172.21.15.107:0/1466262850 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55894fb18c80 msgr2=0x55894fb1ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:07:17.373 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.392+0000 7ff5d1200640 1 --2- 172.21.15.107:0/1466262850 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55894fb18c80 0x55894fb1ed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:07:17.373 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.392+0000 7ff5d1200640 1 -- 172.21.15.107:0/1466262850 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55894fb22370 msgr2=0x55894fb1c320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:07:17.373 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.392+0000 7ff5d1200640 1 --2- 172.21.15.107:0/1466262850 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55894fb22370 0x55894fb1c320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:07:17.373 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.392+0000 7ff5d1200640 1 -- 172.21.15.107:0/1466262850 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55894fb1f9f0 con 0x55894fb195a0 2024-03-08T21:07:17.373 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.392+0000 7ff5d1200640 1 --2- 172.21.15.107:0/1466262850 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55894fb195a0 0x55894fb217f0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7ff5b80bd510 tx=0x7ff5b80f3590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19061dc75d020d7d server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:07:17.375 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.392+0000 7ff5c2000640 1 -- 172.21.15.107:0/1466262850 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5b80035f0 con 0x55894fb195a0 2024-03-08T21:07:17.375 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.392+0000 7ff5c2000640 1 -- 172.21.15.107:0/1466262850 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x55894fa89910 con 0x55894fb195a0 2024-03-08T21:07:17.375 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.392+0000 7ff5c2000640 1 -- 172.21.15.107:0/1466262850 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5b80c5fb0 con 0x55894fb195a0 2024-03-08T21:07:17.376 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.393+0000 7ff5d7363740 1 -- 172.21.15.107:0/1466262850 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55894fb195a0 msgr2=0x55894fb217f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:07:17.376 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.393+0000 7ff5d7363740 1 --2- 172.21.15.107:0/1466262850 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55894fb195a0 0x55894fb217f0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7ff5b80bd510 tx=0x7ff5b80f3590 comp rx=0 tx=0).stop 2024-03-08T21:07:17.376 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.393+0000 7ff5d7363740 1 -- 172.21.15.107:0/1466262850 shutdown_connections 2024-03-08T21:07:17.376 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.393+0000 7ff5d7363740 1 --2- 172.21.15.107:0/1466262850 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55894fb22370 0x55894fb1c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:07:17.376 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.393+0000 7ff5d7363740 1 --2- 172.21.15.107:0/1466262850 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55894fb195a0 0x55894fb217f0 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:07:17.376 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.393+0000 7ff5d7363740 1 --2- 172.21.15.107:0/1466262850 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55894fb18c80 0x55894fb1ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:07:17.376 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.393+0000 7ff5d7363740 1 -- 172.21.15.107:0/1466262850 >> 172.21.15.107:0/1466262850 conn(0x55894fb135a0 msgr2=0x55894fb1e600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:07:17.376 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.393+0000 7ff5d7363740 1 -- 172.21.15.107:0/1466262850 shutdown_connections 2024-03-08T21:07:17.376 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.393+0000 7ff5d7363740 1 -- 172.21.15.107:0/1466262850 wait complete. 2024-03-08T21:07:17.376 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.394+0000 7ff5d7363740 1 -- start start 2024-03-08T21:07:17.376 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.394+0000 7ff5d7363740 1 Processor -- start 2024-03-08T21:07:17.376 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.395+0000 7ff5d7363740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55894fb18c80 0x55894fb297e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:07:17.376 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.395+0000 7ff5d7363740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55894fb195a0 0x55894fb27b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:07:17.377 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.395+0000 7ff5d7363740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55894fb22370 0x55894fb281b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:07:17.377 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.395+0000 7ff5d7363740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5cc1b4640 con 0x55894fb195a0 2024-03-08T21:07:17.377 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.395+0000 7ff5d7363740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5cc1b3fe0 con 0x55894fb18c80 2024-03-08T21:07:17.377 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.395+0000 7ff5d7363740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5cc1b15e0 con 0x55894fb22370 2024-03-08T21:07:17.377 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.395+0000 7ff5d7363740 10 client.0 _ll_register_callbacks cb 0x7ff5bc00a630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T21:07:17.377 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.395+0000 7ff5d1200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55894fb195a0 0x55894fb27b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:07:17.377 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.395+0000 7ff5d1200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55894fb195a0 0x55894fb27b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:35018/0 (socket says 172.21.15.107:35018) 2024-03-08T21:07:17.377 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.395+0000 7ff5d1200640 1 -- 172.21.15.107:0/3970824812 learned_addr learned my addr 172.21.15.107:0/3970824812 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:07:17.377 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.395+0000 7ff5d1c00640 1 --2- 172.21.15.107:0/3970824812 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55894fb18c80 0x55894fb297e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:07:17.377 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.395+0000 7ff5d2600640 1 --2- 172.21.15.107:0/3970824812 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55894fb22370 0x55894fb281b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:07:17.377 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.395+0000 7ff5d1200640 1 -- 172.21.15.107:0/3970824812 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55894fb22370 msgr2=0x55894fb281b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:07:17.377 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.395+0000 7ff5d1200640 1 --2- 172.21.15.107:0/3970824812 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55894fb22370 0x55894fb281b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:07:17.377 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.395+0000 7ff5d1200640 1 -- 172.21.15.107:0/3970824812 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55894fb18c80 msgr2=0x55894fb297e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:07:17.377 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.395+0000 7ff5d1200640 1 --2- 172.21.15.107:0/3970824812 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55894fb18c80 0x55894fb297e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:07:17.378 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.395+0000 7ff5d1200640 1 -- 172.21.15.107:0/3970824812 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55894fb28980 con 0x55894fb195a0 2024-03-08T21:07:17.379 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.396+0000 7ff5d1200640 1 --2- 172.21.15.107:0/3970824812 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55894fb195a0 0x55894fb27b80 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7ff5b8000c00 tx=0x7ff5b80035f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30027756a3c8770c server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:07:17.379 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.396+0000 7ff5b7400640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-03-08T21:07:17.379 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.396+0000 7ff5b7400640 1 -- 172.21.15.107:0/3970824812 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5b803c350 con 0x55894fb195a0 2024-03-08T21:07:17.379 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.396+0000 7ff5b7400640 1 -- 172.21.15.107:0/3970824812 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8585+0+0 (secure 0 0 0) 0x7ff5b80b9e80 con 0x55894fb195a0 2024-03-08T21:07:17.379 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.396+0000 7ff5d7363740 1 -- 172.21.15.107:0/3970824812 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55894fa89910 con 0x55894fb195a0 2024-03-08T21:07:17.380 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.396+0000 7ff5b7400640 1 -- 172.21.15.107:0/3970824812 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff5b80bdf10 con 0x55894fb195a0 2024-03-08T21:07:17.380 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.396+0000 7ff5d7363740 10 client.16023 fetch_fsmap learned FSMap version 34 2024-03-08T21:07:17.380 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.396+0000 7ff5d7363740 1 -- 172.21.15.107:0/3970824812 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55894fb89410 con 0x55894fb195a0 2024-03-08T21:07:17.380 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.397+0000 7ff5b7400640 1 -- 172.21.15.107:0/3970824812 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x55894fa89910 con 0x55894fb195a0 2024-03-08T21:07:17.380 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.397+0000 7ff5b7400640 1 -- 172.21.15.107:0/3970824812 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 34) v1 ==== 7882+0+0 (secure 0 0 0) 0x7ff5b80eea70 con 0x55894fb195a0 2024-03-08T21:07:17.380 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.397+0000 7ff5d7363740 10 client.16023 fetch_fsmap finished waiting for FSMap version 34 2024-03-08T21:07:17.380 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.397+0000 7ff5d7363740 10 client.16023 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-08T21:07:17.380 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.397+0000 7ff5d7363740 20 client.16023 populate_metadata read hostname 'smithi107' 2024-03-08T21:07:17.380 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.397+0000 7ff5d7363740 1 --2- 172.21.15.107:0/3970824812 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55894fb89870 0x55894fb8bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:07:17.380 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.397+0000 7ff5d7363740 4 client.16023 mds_command: new command op to 34475 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-08T21:07:17.380 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.397+0000 7ff5d7363740 1 -- 172.21.15.107:0/3970824812 --> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55894fa83c60 con 0x55894fb89870 2024-03-08T21:07:17.380 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.397+0000 7ff5d1c00640 1 --2- 172.21.15.107:0/3970824812 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55894fb89870 0x55894fb8bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:07:17.380 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.398+0000 7ff5b7400640 10 client.16023 ms_handle_connect on v2:172.21.15.149:6838/55290596 2024-03-08T21:07:17.380 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.398+0000 7ff5d1c00640 1 --2- 172.21.15.107:0/3970824812 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55894fb89870 0x55894fb8bc90 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a4c9d8b2ec3eb151 server_cookie=27928555ea9638c0 in_seq=0 out_seq=0 2024-03-08T21:07:17.394 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.414+0000 7ff5b7400640 1 -- 172.21.15.107:0/3970824812 <== mds.0 v2:172.21.15.149:6838/55290596 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55894fa83c60 con 0x55894fb89870 2024-03-08T21:07:17.394 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.414+0000 7ff5b7400640 10 client.16023 handle_command_reply: tid=0 2024-03-08T21:07:17.395 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.414+0000 7ff5d7363740 1 client.16023 shutdown 2024-03-08T21:07:17.395 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.414+0000 7ff5d7363740 1 -- 172.21.15.107:0/3970824812 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55894fb195a0 msgr2=0x55894fb27b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:07:17.395 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.414+0000 7ff5d7363740 1 --2- 172.21.15.107:0/3970824812 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55894fb195a0 0x55894fb27b80 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7ff5b8000c00 tx=0x7ff5b80035f0 comp rx=0 tx=0).stop 2024-03-08T21:07:17.395 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.414+0000 7ff5d7363740 1 -- 172.21.15.107:0/3970824812 shutdown_connections 2024-03-08T21:07:17.395 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.414+0000 7ff5d7363740 1 --2- 172.21.15.107:0/3970824812 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55894fb22370 0x55894fb281b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:07:17.395 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.414+0000 7ff5d7363740 1 --2- 172.21.15.107:0/3970824812 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55894fb195a0 0x55894fb27b80 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:07:17.395 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.414+0000 7ff5d7363740 1 --2- 172.21.15.107:0/3970824812 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55894fb89870 0x55894fb8bc90 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:07:17.395 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.414+0000 7ff5d7363740 1 --2- 172.21.15.107:0/3970824812 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55894fb18c80 0x55894fb297e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:07:17.395 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.414+0000 7ff5d7363740 1 -- 172.21.15.107:0/3970824812 >> 172.21.15.107:0/3970824812 conn(0x55894fb135a0 msgr2=0x55894fb126c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:07:17.395 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.414+0000 7ff5b7400640 0 client.16023 ms_handle_reset on v2:172.21.15.149:6838/55290596 2024-03-08T21:07:17.395 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.414+0000 7ff5d7363740 1 -- 172.21.15.107:0/3970824812 shutdown_connections 2024-03-08T21:07:17.395 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.414+0000 7ff5d7363740 1 -- 172.21.15.107:0/3970824812 wait complete. 2024-03-08T21:07:17.395 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.415+0000 7ff5d7363740 20 client.16023 trim_cache size 0 max 16384 2024-03-08T21:07:17.396 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:{ 2024-03-08T21:07:17.396 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "message": "", 2024-03-08T21:07:17.396 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "return_code": 0 2024-03-08T21:07:17.396 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:} 2024-03-08T21:07:17.396 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.415+0000 7ff5d4600640 1 -- 172.21.15.107:0/3156296698 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7ff5b00a2a10 msgr2=0x7ff5b00a4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:07:17.397 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.415+0000 7ff5d4600640 1 --2- 172.21.15.107:0/3156296698 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7ff5b00a2a10 0x7ff5b00a4ed0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ff5cc06a9a0 tx=0x7ff5c4008040 comp rx=0 tx=0).stop 2024-03-08T21:07:17.397 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.415+0000 7ff5d4600640 1 -- 172.21.15.107:0/3156296698 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff5cc073fb0 msgr2=0x7ff5cc19e5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:07:17.397 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.415+0000 7ff5d4600640 1 --2- 172.21.15.107:0/3156296698 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff5cc073fb0 0x7ff5cc19e5d0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7ff5b8005ab0 tx=0x7ff5b802db40 comp rx=0 tx=0).stop 2024-03-08T21:07:17.397 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.415+0000 7ff5d4600640 1 -- 172.21.15.107:0/3156296698 shutdown_connections 2024-03-08T21:07:17.397 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.415+0000 7ff5d4600640 1 --2- 172.21.15.107:0/3156296698 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7ff5b00a2a10 0x7ff5b00a4ed0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:07:17.397 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.415+0000 7ff5d4600640 1 --2- 172.21.15.107:0/3156296698 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff5cc1122b0 0x7ff5cc1a2950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:07:17.397 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.415+0000 7ff5d4600640 1 --2- 172.21.15.107:0/3156296698 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff5cc073fb0 0x7ff5cc19e5d0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:07:17.397 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.415+0000 7ff5d4600640 1 --2- 172.21.15.107:0/3156296698 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff5cc073690 0x7ff5cc19e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:07:17.397 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.415+0000 7ff5d4600640 1 -- 172.21.15.107:0/3156296698 >> 172.21.15.107:0/3156296698 conn(0x7ff5cc0fcdb0 msgr2=0x7ff5cc0fe4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:07:17.397 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.416+0000 7ff5d4600640 1 -- 172.21.15.107:0/3156296698 shutdown_connections 2024-03-08T21:07:17.397 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:17.416+0000 7ff5d4600640 1 -- 172.21.15.107:0/3156296698 wait complete. 2024-03-08T21:07:17.625 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:07:17 smithi204 podman[71108]: 2024-03-08 21:07:17.344607833 +0000 UTC m=+0.430279603 container init ac85e479b71fcf12a3e9715687d6094abecdd1de10db12aa559758bfc17e216a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=rishabh-2024mar8-7d63482, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-08T21:07:17.625 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:07:17 smithi204 podman[71108]: 2024-03-08 21:07:17.354817003 +0000 UTC m=+0.440488747 container start ac85e479b71fcf12a3e9715687d6094abecdd1de10db12aa559758bfc17e216a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=rishabh-2024mar8-7d63482, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-03-08T21:07:17.625 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:07:17 smithi204 ceph-mds[71126]: -- 172.21.15.204:0/3753388441 <== mon.1 v2:172.21.15.204:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x558bb28d1a40 con 0x558bb28a4800 2024-03-08T21:07:17.625 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:07:17 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[71122]: starting mds.c at 2024-03-08T21:07:17.625 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:07:17 smithi204 bash[71108]: ac85e479b71fcf12a3e9715687d6094abecdd1de10db12aa559758bfc17e216a 2024-03-08T21:07:17.625 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:07:17 smithi204 systemd[1]: Started Ceph mds.c for dc20f0f8-dd8c-11ee-95c6-87774f69a715. 2024-03-08T21:07:18.011 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:07:17 smithi149 ceph-mon[35042]: daemon mds.c restarted 2024-03-08T21:07:18.124 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:07:17 smithi204 ceph-mon[34935]: daemon mds.c restarted 2024-03-08T21:07:18.213 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:07:17 smithi107 ceph-mon[31381]: daemon mds.c restarted 2024-03-08T21:07:19.011 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:07:18 smithi149 ceph-mon[35042]: osdmap e95: 12 total, 12 up, 12 in 2024-03-08T21:07:19.011 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:07:18 smithi149 ceph-mon[35042]: mds.? [v2:172.21.15.204:6836/2711192050,v1:172.21.15.204:6837/2711192050] up:boot 2024-03-08T21:07:19.012 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:07:18 smithi149 ceph-mon[35042]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-08T21:07:19.012 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:07:18 smithi149 ceph-mon[35042]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-03-08T21:07:19.012 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:07:18 smithi149 ceph-mon[35042]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-08T21:07:19.012 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:07:18 smithi149 ceph-mon[35042]: pgmap v212: 97 pgs: 97 active+clean; 4.7 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 538 B/s wr, 10 op/s 2024-03-08T21:07:19.125 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:07:18 smithi204 ceph-mon[34935]: osdmap e95: 12 total, 12 up, 12 in 2024-03-08T21:07:19.125 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:07:18 smithi204 ceph-mon[34935]: mds.? [v2:172.21.15.204:6836/2711192050,v1:172.21.15.204:6837/2711192050] up:boot 2024-03-08T21:07:19.125 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:07:18 smithi204 ceph-mon[34935]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-08T21:07:19.125 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:07:18 smithi204 ceph-mon[34935]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-03-08T21:07:19.125 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:07:18 smithi204 ceph-mon[34935]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-08T21:07:19.125 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:07:18 smithi204 ceph-mon[34935]: pgmap v212: 97 pgs: 97 active+clean; 4.7 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 538 B/s wr, 10 op/s 2024-03-08T21:07:19.214 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:07:18 smithi107 ceph-mon[31381]: osdmap e95: 12 total, 12 up, 12 in 2024-03-08T21:07:19.214 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:07:18 smithi107 ceph-mon[31381]: mds.? [v2:172.21.15.204:6836/2711192050,v1:172.21.15.204:6837/2711192050] up:boot 2024-03-08T21:07:19.214 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:07:18 smithi107 ceph-mon[31381]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-08T21:07:19.214 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:07:18 smithi107 ceph-mon[31381]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-03-08T21:07:19.214 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:07:18 smithi107 ceph-mon[31381]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-08T21:07:19.214 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:07:18 smithi107 ceph-mon[31381]: pgmap v212: 97 pgs: 97 active+clean; 4.7 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 538 B/s wr, 10 op/s 2024-03-08T21:07:21.011 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:07:20 smithi149 ceph-mon[35042]: pgmap v213: 97 pgs: 97 active+clean; 4.7 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 511 B/s wr, 9 op/s 2024-03-08T21:07:21.124 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:07:20 smithi204 ceph-mon[34935]: pgmap v213: 97 pgs: 97 active+clean; 4.7 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 511 B/s wr, 9 op/s 2024-03-08T21:07:21.213 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:07:20 smithi107 ceph-mon[31381]: pgmap v213: 97 pgs: 97 active+clean; 4.7 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 511 B/s wr, 9 op/s 2024-03-08T21:07:23.011 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:07:22 smithi149 ceph-mon[35042]: pgmap v214: 97 pgs: 97 active+clean; 4.7 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 511 B/s wr, 9 op/s 2024-03-08T21:07:23.124 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:07:22 smithi204 ceph-mon[34935]: pgmap v214: 97 pgs: 97 active+clean; 4.7 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 511 B/s wr, 9 op/s 2024-03-08T21:07:23.214 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:07:22 smithi107 ceph-mon[31381]: pgmap v214: 97 pgs: 97 active+clean; 4.7 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 511 B/s wr, 9 op/s 2024-03-08T21:07:25.011 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:07:24 smithi149 ceph-mon[35042]: pgmap v215: 97 pgs: 97 active+clean; 4.7 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 409 B/s wr, 10 op/s 2024-03-08T21:07:25.124 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:07:24 smithi204 ceph-mon[34935]: pgmap v215: 97 pgs: 97 active+clean; 4.7 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 409 B/s wr, 10 op/s 2024-03-08T21:07:25.213 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:07:24 smithi107 ceph-mon[31381]: pgmap v215: 97 pgs: 97 active+clean; 4.7 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 409 B/s wr, 10 op/s 2024-03-08T21:07:27.011 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:07:26 smithi149 ceph-mon[35042]: pgmap v216: 97 pgs: 97 active+clean; 4.7 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 409 B/s wr, 10 op/s 2024-03-08T21:07:27.124 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:07:26 smithi204 ceph-mon[34935]: pgmap v216: 97 pgs: 97 active+clean; 4.7 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 409 B/s wr, 10 op/s 2024-03-08T21:07:27.213 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:07:26 smithi107 ceph-mon[31381]: pgmap v216: 97 pgs: 97 active+clean; 4.7 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 409 B/s wr, 10 op/s 2024-03-08T21:07:29.011 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:07:28 smithi149 ceph-mon[35042]: pgmap v217: 97 pgs: 97 active+clean; 4.7 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 375 B/s wr, 14 op/s 2024-03-08T21:07:29.124 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:07:28 smithi204 ceph-mon[34935]: pgmap v217: 97 pgs: 97 active+clean; 4.7 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 375 B/s wr, 14 op/s 2024-03-08T21:07:29.213 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:07:28 smithi107 ceph-mon[31381]: pgmap v217: 97 pgs: 97 active+clean; 4.7 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 375 B/s wr, 14 op/s 2024-03-08T21:07:30.557 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.577+0000 7f89ed800640 1 -- 172.21.15.107:0/1891386239 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f89e810b8a0 msgr2=0x7f89e810dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:07:30.557 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.577+0000 7f89ed800640 1 --2- 172.21.15.107:0/1891386239 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f89e810b8a0 0x7f89e810dd40 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f89e0009be0 tx=0x7f89e002c010 comp rx=0 tx=0).stop 2024-03-08T21:07:30.557 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.577+0000 7f89ed800640 1 -- 172.21.15.107:0/1891386239 shutdown_connections 2024-03-08T21:07:30.557 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.577+0000 7f89ed800640 1 --2- 172.21.15.107:0/1891386239 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f89e810b8a0 0x7f89e810dd40 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:07:30.557 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.577+0000 7f89ed800640 1 --2- 172.21.15.107:0/1891386239 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f89e8102e80 0x7f89e810b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:07:30.557 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.577+0000 7f89ed800640 1 --2- 172.21.15.107:0/1891386239 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f89e8102560 0x7f89e8102940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:07:30.557 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.577+0000 7f89ed800640 1 -- 172.21.15.107:0/1891386239 >> 172.21.15.107:0/1891386239 conn(0x7f89e8077310 msgr2=0x7f89e8077720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:07:30.558 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.577+0000 7f89ed800640 1 -- 172.21.15.107:0/1891386239 shutdown_connections 2024-03-08T21:07:30.558 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.577+0000 7f89ed800640 1 -- 172.21.15.107:0/1891386239 wait complete. 2024-03-08T21:07:30.560 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.578+0000 7f89ed800640 1 Processor -- start 2024-03-08T21:07:30.560 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.578+0000 7f89ed800640 1 -- start start 2024-03-08T21:07:30.560 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.578+0000 7f89ed800640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f89e8102560 0x7f89e819f890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:07:30.560 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.578+0000 7f89ed800640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f89e8102e80 0x7f89e819fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:07:30.560 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.578+0000 7f89ed800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f89e810b8a0 0x7f89e8199a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:07:30.560 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.578+0000 7f89ed800640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89e8111580 con 0x7f89e810b8a0 2024-03-08T21:07:30.560 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.578+0000 7f89ed800640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89e8111400 con 0x7f89e8102560 2024-03-08T21:07:30.560 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.578+0000 7f89ed800640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89e8111700 con 0x7f89e8102e80 2024-03-08T21:07:30.561 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.579+0000 7f89e7400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f89e810b8a0 0x7f89e8199a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:07:30.561 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.579+0000 7f89e7400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f89e810b8a0 0x7f89e8199a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:33796/0 (socket says 172.21.15.107:33796) 2024-03-08T21:07:30.561 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.579+0000 7f89e7400640 1 -- 172.21.15.107:0/2107143583 learned_addr learned my addr 172.21.15.107:0/2107143583 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:07:30.561 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.579+0000 7f89e6a00640 1 --2- 172.21.15.107:0/2107143583 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f89e8102560 0x7f89e819f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:07:30.561 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.579+0000 7f89dfe00640 1 --2- 172.21.15.107:0/2107143583 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f89e8102e80 0x7f89e819fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:07:30.561 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.579+0000 7f89e7400640 1 -- 172.21.15.107:0/2107143583 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f89e8102e80 msgr2=0x7f89e819fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:07:30.561 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.579+0000 7f89e7400640 1 --2- 172.21.15.107:0/2107143583 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f89e8102e80 0x7f89e819fdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:07:30.561 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.579+0000 7f89e7400640 1 -- 172.21.15.107:0/2107143583 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f89e8102560 msgr2=0x7f89e819f890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:07:30.561 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.579+0000 7f89e7400640 1 --2- 172.21.15.107:0/2107143583 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f89e8102560 0x7f89e819f890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:07:30.561 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.579+0000 7f89e7400640 1 -- 172.21.15.107:0/2107143583 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89e819a1c0 con 0x7f89e810b8a0 2024-03-08T21:07:30.561 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.579+0000 7f89dfe00640 1 --2- 172.21.15.107:0/2107143583 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f89e8102e80 0x7f89e819fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T21:07:30.561 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.580+0000 7f89e7400640 1 --2- 172.21.15.107:0/2107143583 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f89e810b8a0 0x7f89e8199a40 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f89e0036040 tx=0x7f89e0004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:07:30.563 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.580+0000 7f89df400640 1 -- 172.21.15.107:0/2107143583 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89e0043020 con 0x7f89e810b8a0 2024-03-08T21:07:30.563 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.580+0000 7f89df400640 1 -- 172.21.15.107:0/2107143583 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f89e0004620 con 0x7f89e810b8a0 2024-03-08T21:07:30.563 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.580+0000 7f89ed800640 1 -- 172.21.15.107:0/2107143583 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89e819a450 con 0x7f89e810b8a0 2024-03-08T21:07:30.563 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.580+0000 7f89df400640 1 -- 172.21.15.107:0/2107143583 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89e0034690 con 0x7f89e810b8a0 2024-03-08T21:07:30.563 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.580+0000 7f89ed800640 1 -- 172.21.15.107:0/2107143583 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89e81a6750 con 0x7f89e810b8a0 2024-03-08T21:07:30.564 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.584+0000 7f89f05a4740 1 Processor -- start 2024-03-08T21:07:30.564 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.585+0000 7f89df400640 1 -- 172.21.15.107:0/2107143583 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) v1 ==== 137422+0+0 (secure 0 0 0) 0x7f89e0046020 con 0x7f89e810b8a0 2024-03-08T21:07:30.564 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.585+0000 7f89f05a4740 1 -- start start 2024-03-08T21:07:30.565 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.585+0000 7f89f05a4740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b2d0d4fc80 0x55b2d0d50060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:07:30.566 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.585+0000 7f89f05a4740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b2d0d505a0 0x55b2d0d58e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:07:30.566 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.585+0000 7f89f05a4740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b2d0d59370 0x55b2d0d5b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:07:30.566 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.585+0000 7f89f05a4740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2d0cb33d0 con 0x55b2d0d59370 2024-03-08T21:07:30.566 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.585+0000 7f89f05a4740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2d0c5c6b0 con 0x55b2d0d505a0 2024-03-08T21:07:30.566 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.585+0000 7f89f05a4740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2d0caf470 con 0x55b2d0d4fc80 2024-03-08T21:07:30.566 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.586+0000 7f89dfe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b2d0d59370 0x55b2d0d5b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:07:30.566 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.586+0000 7f89dfe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b2d0d59370 0x55b2d0d5b760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:33800/0 (socket says 172.21.15.107:33800) 2024-03-08T21:07:30.566 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.586+0000 7f89dfe00640 1 -- 172.21.15.107:0/2579244374 learned_addr learned my addr 172.21.15.107:0/2579244374 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:07:30.566 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.586+0000 7f89e6a00640 1 --2- 172.21.15.107:0/2579244374 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b2d0d505a0 0x55b2d0d58e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:07:30.566 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.586+0000 7f89dfe00640 1 --2- 172.21.15.107:0/2579244374 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b2d0d4fc80 0x55b2d0d50060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:07:30.568 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.586+0000 7f89df400640 1 --2- 172.21.15.107:0/2107143583 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f89cc0a2a10 0x7f89cc0a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:07:30.568 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.586+0000 7f89dfe00640 1 -- 172.21.15.107:0/2579244374 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b2d0d4fc80 msgr2=0x55b2d0d50060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:07:30.568 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.586+0000 7f89dfe00640 1 --2- 172.21.15.107:0/2579244374 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b2d0d4fc80 0x55b2d0d50060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:07:30.568 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.586+0000 7f89dfe00640 1 -- 172.21.15.107:0/2579244374 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b2d0d505a0 msgr2=0x55b2d0d58e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:07:30.568 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.586+0000 7f89dfe00640 1 --2- 172.21.15.107:0/2579244374 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b2d0d505a0 0x55b2d0d58e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:07:30.568 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.586+0000 7f89dfe00640 1 -- 172.21.15.107:0/2579244374 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b2d0c82b50 con 0x55b2d0d59370 2024-03-08T21:07:30.569 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.587+0000 7f89df400640 1 -- 172.21.15.107:0/2107143583 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f89e00f1530 con 0x7f89e810b8a0 2024-03-08T21:07:30.569 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.587+0000 7f89e7400640 1 --2- 172.21.15.107:0/2107143583 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f89cc0a2a10 0x7f89cc0a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:07:30.569 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.587+0000 7f89dfe00640 1 --2- 172.21.15.107:0/2579244374 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b2d0d59370 0x55b2d0d5b760 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f89d0013460 tx=0x7f89d00095b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=308a587bd345bcef server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:07:30.569 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.587+0000 7f89d7e00640 1 -- 172.21.15.107:0/2579244374 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89d0019020 con 0x55b2d0d59370 2024-03-08T21:07:30.569 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.587+0000 7f89d7e00640 1 -- 172.21.15.107:0/2579244374 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f89d0004230 con 0x55b2d0d59370 2024-03-08T21:07:30.569 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.587+0000 7f89d7e00640 1 -- 172.21.15.107:0/2579244374 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89d0011700 con 0x55b2d0d59370 2024-03-08T21:07:30.569 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.588+0000 7f89f05a4740 1 -- 172.21.15.107:0/2579244374 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b2d0d59370 msgr2=0x55b2d0d5b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:07:30.569 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.588+0000 7f89e7400640 1 --2- 172.21.15.107:0/2107143583 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f89cc0a2a10 0x7f89cc0a4ed0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f89e002c5f0 tx=0x7f89e002c010 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:07:30.569 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.588+0000 7f89f05a4740 1 --2- 172.21.15.107:0/2579244374 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b2d0d59370 0x55b2d0d5b760 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f89d0013460 tx=0x7f89d00095b0 comp rx=0 tx=0).stop 2024-03-08T21:07:30.569 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.588+0000 7f89f05a4740 1 -- 172.21.15.107:0/2579244374 shutdown_connections 2024-03-08T21:07:30.569 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.588+0000 7f89f05a4740 1 --2- 172.21.15.107:0/2579244374 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b2d0d59370 0x55b2d0d5b760 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:07:30.569 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.588+0000 7f89f05a4740 1 --2- 172.21.15.107:0/2579244374 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b2d0d505a0 0x55b2d0d58e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:07:30.569 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.588+0000 7f89f05a4740 1 --2- 172.21.15.107:0/2579244374 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b2d0d4fc80 0x55b2d0d50060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:07:30.570 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.588+0000 7f89f05a4740 1 -- 172.21.15.107:0/2579244374 >> 172.21.15.107:0/2579244374 conn(0x55b2d0d4a5a0 msgr2=0x55b2d0d4a980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:07:30.570 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.588+0000 7f89f05a4740 1 -- 172.21.15.107:0/2579244374 shutdown_connections 2024-03-08T21:07:30.570 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.588+0000 7f89f05a4740 1 -- 172.21.15.107:0/2579244374 wait complete. 2024-03-08T21:07:30.572 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.589+0000 7f89f05a4740 1 -- start start 2024-03-08T21:07:30.572 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.589+0000 7f89f05a4740 1 Processor -- start 2024-03-08T21:07:30.572 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.590+0000 7f89f05a4740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b2d0d4fc80 0x55b2d0d58cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:07:30.572 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.590+0000 7f89f05a4740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b2d0d505a0 0x55b2d0d56050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:07:30.573 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.590+0000 7f89f05a4740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b2d0d59370 0x55b2d0d565b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:07:30.573 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.590+0000 7f89f05a4740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2d0d5bfa0 con 0x55b2d0d59370 2024-03-08T21:07:30.573 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.590+0000 7f89f05a4740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2d0c6f9f0 con 0x55b2d0d4fc80 2024-03-08T21:07:30.573 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.590+0000 7f89f05a4740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2d0d5c120 con 0x55b2d0d505a0 2024-03-08T21:07:30.573 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.590+0000 7f89f05a4740 10 client.0 _ll_register_callbacks cb 0x55b2d09c7a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T21:07:30.573 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.590+0000 7f89dfe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b2d0d59370 0x55b2d0d565b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:07:30.573 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.590+0000 7f89dfe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b2d0d59370 0x55b2d0d565b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:33812/0 (socket says 172.21.15.107:33812) 2024-03-08T21:07:30.573 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.590+0000 7f89dfe00640 1 -- 172.21.15.107:0/1936851913 learned_addr learned my addr 172.21.15.107:0/1936851913 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:07:30.573 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.590+0000 7f89dfe00640 1 --2- 172.21.15.107:0/1936851913 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b2d0d4fc80 0x55b2d0d58cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:07:30.573 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.591+0000 7f89dfe00640 1 -- 172.21.15.107:0/1936851913 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b2d0d505a0 msgr2=0x55b2d0d56050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:07:30.573 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.591+0000 7f89dfe00640 1 --2- 172.21.15.107:0/1936851913 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b2d0d505a0 0x55b2d0d56050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:07:30.573 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.591+0000 7f89dfe00640 1 -- 172.21.15.107:0/1936851913 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b2d0d4fc80 msgr2=0x55b2d0d58cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:07:30.573 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.591+0000 7f89dfe00640 1 --2- 172.21.15.107:0/1936851913 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b2d0d4fc80 0x55b2d0d58cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:07:30.574 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.591+0000 7f89dfe00640 1 -- 172.21.15.107:0/1936851913 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b2d09c8210 con 0x55b2d0d59370 2024-03-08T21:07:30.574 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.591+0000 7f89dfe00640 1 --2- 172.21.15.107:0/1936851913 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b2d0d59370 0x55b2d0d565b0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f89d0009250 tx=0x7f89d0003ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fe7076a5196c87 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:07:30.574 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.591+0000 7f89d4c00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-03-08T21:07:30.574 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.591+0000 7f89d4c00640 1 -- 172.21.15.107:0/1936851913 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89d0019030 con 0x55b2d0d59370 2024-03-08T21:07:30.574 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.591+0000 7f89f05a4740 1 -- 172.21.15.107:0/1936851913 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f89d0004230 con 0x55b2d0d59370 2024-03-08T21:07:30.574 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.591+0000 7f89d4c00640 1 -- 172.21.15.107:0/1936851913 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f89d00037f0 con 0x55b2d0d59370 2024-03-08T21:07:30.574 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.592+0000 7f89d4c00640 1 -- 172.21.15.107:0/1936851913 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f89d00118e0 con 0x55b2d0d59370 2024-03-08T21:07:30.574 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.592+0000 7f89f05a4740 10 client.16041 fetch_fsmap learned FSMap version 36 2024-03-08T21:07:30.574 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.592+0000 7f89f05a4740 1 -- 172.21.15.107:0/1936851913 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x55b2d0a94620 con 0x55b2d0d59370 2024-03-08T21:07:30.574 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.592+0000 7f89d4c00640 1 -- 172.21.15.107:0/1936851913 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f89d0004230 con 0x55b2d0d59370 2024-03-08T21:07:30.574 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.592+0000 7f89d4c00640 1 -- 172.21.15.107:0/1936851913 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 36) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f89d00329a0 con 0x55b2d0d59370 2024-03-08T21:07:30.574 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.592+0000 7f89f05a4740 10 client.16041 fetch_fsmap finished waiting for FSMap version 36 2024-03-08T21:07:30.574 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.592+0000 7f89f05a4740 10 client.16041 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-08T21:07:30.575 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.592+0000 7f89f05a4740 20 client.16041 populate_metadata read hostname 'smithi107' 2024-03-08T21:07:30.575 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.592+0000 7f89f05a4740 1 --2- 172.21.15.107:0/1936851913 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55b2d0d691b0 0x55b2d0d6b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:07:30.575 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.592+0000 7f89f05a4740 4 client.16041 mds_command: new command op to 34475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-08T21:07:30.575 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.592+0000 7f89f05a4740 1 -- 172.21.15.107:0/1936851913 --> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b2d0cc0910 con 0x55b2d0d691b0 2024-03-08T21:07:30.576 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.593+0000 7f89e6a00640 1 --2- 172.21.15.107:0/1936851913 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55b2d0d691b0 0x55b2d0d6b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:07:30.576 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.594+0000 7f89e6a00640 1 --2- 172.21.15.107:0/1936851913 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55b2d0d691b0 0x55b2d0d6b5a0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=757ab743414244c3 server_cookie=8be127a6a66204a7 in_seq=0 out_seq=0 2024-03-08T21:07:30.576 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.594+0000 7f89d4c00640 10 client.16041 ms_handle_connect on v2:172.21.15.149:6838/55290596 2024-03-08T21:07:30.576 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.595+0000 7f89d4c00640 1 -- 172.21.15.107:0/1936851913 <== mds.0 v2:172.21.15.149:6838/55290596 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55b2d0cc0910 con 0x55b2d0d691b0 2024-03-08T21:07:30.576 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.595+0000 7f89d4c00640 10 client.16041 handle_command_reply: tid=0 2024-03-08T21:07:30.576 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.596+0000 7f89f05a4740 1 client.16041 shutdown 2024-03-08T21:07:30.576 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.596+0000 7f89f05a4740 1 -- 172.21.15.107:0/1936851913 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b2d0d59370 msgr2=0x55b2d0d565b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:07:30.577 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.596+0000 7f89f05a4740 1 --2- 172.21.15.107:0/1936851913 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b2d0d59370 0x55b2d0d565b0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f89d0009250 tx=0x7f89d0003ee0 comp rx=0 tx=0).stop 2024-03-08T21:07:30.577 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.596+0000 7f89f05a4740 1 -- 172.21.15.107:0/1936851913 shutdown_connections 2024-03-08T21:07:30.577 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.596+0000 7f89f05a4740 1 --2- 172.21.15.107:0/1936851913 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b2d0d59370 0x55b2d0d565b0 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:07:30.577 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.596+0000 7f89f05a4740 1 --2- 172.21.15.107:0/1936851913 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b2d0d505a0 0x55b2d0d56050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:07:30.577 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.596+0000 7f89f05a4740 1 --2- 172.21.15.107:0/1936851913 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55b2d0d691b0 0x55b2d0d6b5a0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:07:30.577 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.596+0000 7f89f05a4740 1 --2- 172.21.15.107:0/1936851913 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b2d0d4fc80 0x55b2d0d58cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:07:30.577 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.596+0000 7f89f05a4740 1 -- 172.21.15.107:0/1936851913 >> 172.21.15.107:0/1936851913 conn(0x55b2d0d4a5a0 msgr2=0x55b2d0d4c2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:07:30.577 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.596+0000 7f89d4c00640 0 client.16041 ms_handle_reset on v2:172.21.15.149:6838/55290596 2024-03-08T21:07:30.577 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.596+0000 7f89f05a4740 1 -- 172.21.15.107:0/1936851913 shutdown_connections 2024-03-08T21:07:30.577 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.596+0000 7f89f05a4740 1 -- 172.21.15.107:0/1936851913 wait complete. 2024-03-08T21:07:30.577 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.596+0000 7f89f05a4740 20 client.16041 trim_cache size 0 max 16384 2024-03-08T21:07:30.591 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.611+0000 7f89f05a4740 1 Processor -- start 2024-03-08T21:07:30.591 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.612+0000 7f89f05a4740 1 -- start start 2024-03-08T21:07:30.592 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.612+0000 7f89f05a4740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b2d0d4fc80 0x55b2d0d4c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:07:30.593 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.612+0000 7f89f05a4740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b2d0d505a0 0x55b2d0d58780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:07:30.593 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.612+0000 7f89f05a4740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b2d0d59370 0x55b2d0d58cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:07:30.593 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.612+0000 7f89f05a4740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89e81aee00 con 0x55b2d0d505a0 2024-03-08T21:07:30.593 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.612+0000 7f89f05a4740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89e81b0780 con 0x55b2d0d4fc80 2024-03-08T21:07:30.593 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.612+0000 7f89f05a4740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89e81af460 con 0x55b2d0d59370 2024-03-08T21:07:30.593 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.612+0000 7f89e6a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b2d0d505a0 0x55b2d0d58780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:07:30.593 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.612+0000 7f89e6a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b2d0d505a0 0x55b2d0d58780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:33828/0 (socket says 172.21.15.107:33828) 2024-03-08T21:07:30.593 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.612+0000 7f89e6a00640 1 -- 172.21.15.107:0/403634499 learned_addr learned my addr 172.21.15.107:0/403634499 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:07:30.593 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.613+0000 7f89dfe00640 1 --2- 172.21.15.107:0/403634499 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b2d0d4fc80 0x55b2d0d4c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:07:30.593 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.613+0000 7f89e6a00640 1 -- 172.21.15.107:0/403634499 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b2d0d59370 msgr2=0x55b2d0d58cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:07:30.593 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.613+0000 7f89e6a00640 1 --2- 172.21.15.107:0/403634499 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b2d0d59370 0x55b2d0d58cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:07:30.593 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.613+0000 7f89e6a00640 1 -- 172.21.15.107:0/403634499 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b2d0d4fc80 msgr2=0x55b2d0d4c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:07:30.593 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.613+0000 7f89e6a00640 1 --2- 172.21.15.107:0/403634499 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b2d0d4fc80 0x55b2d0d4c0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:07:30.594 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.613+0000 7f89e6a00640 1 -- 172.21.15.107:0/403634499 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b2d0d4a3d0 con 0x55b2d0d505a0 2024-03-08T21:07:30.594 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.613+0000 7f89e6a00640 1 --2- 172.21.15.107:0/403634499 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b2d0d505a0 0x55b2d0d58780 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f89d8009c20 tx=0x7f89d80097c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=154ded5408f8edc9 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:07:30.595 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.613+0000 7f89d7400640 1 -- 172.21.15.107:0/403634499 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89d80142d0 con 0x55b2d0d505a0 2024-03-08T21:07:30.595 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.613+0000 7f89d7400640 1 -- 172.21.15.107:0/403634499 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f89d801e030 con 0x55b2d0d505a0 2024-03-08T21:07:30.595 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.613+0000 7f89d7400640 1 -- 172.21.15.107:0/403634499 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89d801a510 con 0x55b2d0d505a0 2024-03-08T21:07:30.595 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.614+0000 7f89f05a4740 1 -- 172.21.15.107:0/403634499 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b2d0d505a0 msgr2=0x55b2d0d58780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:07:30.595 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.614+0000 7f89f05a4740 1 --2- 172.21.15.107:0/403634499 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b2d0d505a0 0x55b2d0d58780 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f89d8009c20 tx=0x7f89d80097c0 comp rx=0 tx=0).stop 2024-03-08T21:07:30.595 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.614+0000 7f89f05a4740 1 -- 172.21.15.107:0/403634499 shutdown_connections 2024-03-08T21:07:30.595 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.614+0000 7f89f05a4740 1 --2- 172.21.15.107:0/403634499 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b2d0d59370 0x55b2d0d58cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:07:30.595 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.614+0000 7f89f05a4740 1 --2- 172.21.15.107:0/403634499 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b2d0d505a0 0x55b2d0d58780 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:07:30.595 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.614+0000 7f89f05a4740 1 --2- 172.21.15.107:0/403634499 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b2d0d4fc80 0x55b2d0d4c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:07:30.595 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.614+0000 7f89f05a4740 1 -- 172.21.15.107:0/403634499 >> 172.21.15.107:0/403634499 conn(0x55b2d0d4a5a0 msgr2=0x55b2d0d4b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:07:30.596 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.614+0000 7f89f05a4740 1 -- 172.21.15.107:0/403634499 shutdown_connections 2024-03-08T21:07:30.596 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.614+0000 7f89f05a4740 1 -- 172.21.15.107:0/403634499 wait complete. 2024-03-08T21:07:30.596 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.615+0000 7f89f05a4740 1 -- start start 2024-03-08T21:07:30.596 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.615+0000 7f89f05a4740 1 Processor -- start 2024-03-08T21:07:30.598 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.616+0000 7f89f05a4740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b2d0d4fc80 0x55b2d0d5f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:07:30.598 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.616+0000 7f89f05a4740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b2d0d505a0 0x55b2d0d5f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:07:30.598 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.616+0000 7f89f05a4740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b2d0d59370 0x55b2d0d5fb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:07:30.598 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.616+0000 7f89f05a4740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89e81b0120 con 0x55b2d0d4fc80 2024-03-08T21:07:30.599 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.616+0000 7f89f05a4740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89e81afac0 con 0x55b2d0d505a0 2024-03-08T21:07:30.599 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.616+0000 7f89f05a4740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89e81ad0a0 con 0x55b2d0d59370 2024-03-08T21:07:30.599 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.616+0000 7f89f05a4740 10 client.0 _ll_register_callbacks cb 0x7f89d0009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T21:07:30.599 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.616+0000 7f89dfe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b2d0d4fc80 0x55b2d0d5f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:07:30.599 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.616+0000 7f89dfe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b2d0d4fc80 0x55b2d0d5f0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:33830/0 (socket says 172.21.15.107:33830) 2024-03-08T21:07:30.599 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.616+0000 7f89dfe00640 1 -- 172.21.15.107:0/3957933986 learned_addr learned my addr 172.21.15.107:0/3957933986 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:07:30.599 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.616+0000 7f89e6a00640 1 --2- 172.21.15.107:0/3957933986 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b2d0d505a0 0x55b2d0d5f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:07:30.599 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.616+0000 7f89dfe00640 1 -- 172.21.15.107:0/3957933986 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b2d0d59370 msgr2=0x55b2d0d5fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:07:30.599 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.616+0000 7f89dfe00640 1 --2- 172.21.15.107:0/3957933986 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b2d0d59370 0x55b2d0d5fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:07:30.599 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.616+0000 7f89dfe00640 1 -- 172.21.15.107:0/3957933986 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b2d0d505a0 msgr2=0x55b2d0d5f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:07:30.599 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.616+0000 7f89dfe00640 1 --2- 172.21.15.107:0/3957933986 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b2d0d505a0 0x55b2d0d5f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:07:30.599 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.616+0000 7f89dfe00640 1 -- 172.21.15.107:0/3957933986 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b2d0d5b760 con 0x55b2d0d4fc80 2024-03-08T21:07:30.599 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.616+0000 7f89e6a00640 1 --2- 172.21.15.107:0/3957933986 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b2d0d505a0 0x55b2d0d5f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T21:07:30.599 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.617+0000 7f89dfe00640 1 --2- 172.21.15.107:0/3957933986 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b2d0d4fc80 0x55b2d0d5f0a0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f89d0001eb0 tx=0x7f89d00324c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a6f2004c1839647 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:07:30.600 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.617+0000 7f89d4c00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-03-08T21:07:30.600 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.617+0000 7f89d4c00640 1 -- 172.21.15.107:0/3957933986 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89d0019070 con 0x55b2d0d4fc80 2024-03-08T21:07:30.600 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.617+0000 7f89f05a4740 1 -- 172.21.15.107:0/3957933986 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f89d801e030 con 0x55b2d0d4fc80 2024-03-08T21:07:30.600 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.617+0000 7f89d4c00640 1 -- 172.21.15.107:0/3957933986 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f89d002e030 con 0x55b2d0d4fc80 2024-03-08T21:07:30.600 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.617+0000 7f89d4c00640 1 -- 172.21.15.107:0/3957933986 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f89d0009770 con 0x55b2d0d4fc80 2024-03-08T21:07:30.600 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.617+0000 7f89f05a4740 10 client.16047 fetch_fsmap learned FSMap version 36 2024-03-08T21:07:30.600 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.617+0000 7f89f05a4740 1 -- 172.21.15.107:0/3957933986 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x55b2d0dc0790 con 0x55b2d0d4fc80 2024-03-08T21:07:30.600 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.618+0000 7f89d4c00640 1 -- 172.21.15.107:0/3957933986 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f89d801e030 con 0x55b2d0d4fc80 2024-03-08T21:07:30.600 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.618+0000 7f89d4c00640 1 -- 172.21.15.107:0/3957933986 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 36) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f89d0005070 con 0x55b2d0d4fc80 2024-03-08T21:07:30.600 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.618+0000 7f89f05a4740 10 client.16047 fetch_fsmap finished waiting for FSMap version 36 2024-03-08T21:07:30.600 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.618+0000 7f89f05a4740 10 client.16047 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-08T21:07:30.600 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.618+0000 7f89f05a4740 20 client.16047 populate_metadata read hostname 'smithi107' 2024-03-08T21:07:30.600 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.618+0000 7f89f05a4740 1 --2- 172.21.15.107:0/3957933986 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55b2d0dc0d80 0x55b2d0dc31a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:07:30.600 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.618+0000 7f89f05a4740 4 client.16047 mds_command: new command op to 34475 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-08T21:07:30.601 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.618+0000 7f89f05a4740 1 -- 172.21.15.107:0/3957933986 --> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55b2d0cbac60 con 0x55b2d0dc0d80 2024-03-08T21:07:30.601 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.619+0000 7f89e6a00640 1 --2- 172.21.15.107:0/3957933986 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55b2d0dc0d80 0x55b2d0dc31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:07:30.601 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.620+0000 7f89d4c00640 10 client.16047 ms_handle_connect on v2:172.21.15.149:6838/55290596 2024-03-08T21:07:30.601 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.620+0000 7f89e6a00640 1 --2- 172.21.15.107:0/3957933986 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55b2d0dc0d80 0x55b2d0dc31a0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=920376c97acbd410 server_cookie=1cb86da3ad6afc85 in_seq=0 out_seq=0 2024-03-08T21:07:30.610 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.630+0000 7f89d4c00640 1 -- 172.21.15.107:0/3957933986 <== mds.0 v2:172.21.15.149:6838/55290596 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55b2d0cbac60 con 0x55b2d0dc0d80 2024-03-08T21:07:30.610 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.630+0000 7f89d4c00640 10 client.16047 handle_command_reply: tid=0 2024-03-08T21:07:30.610 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.630+0000 7f89f05a4740 1 client.16047 shutdown 2024-03-08T21:07:30.610 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.630+0000 7f89f05a4740 1 -- 172.21.15.107:0/3957933986 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b2d0d4fc80 msgr2=0x55b2d0d5f0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:07:30.610 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.630+0000 7f89f05a4740 1 --2- 172.21.15.107:0/3957933986 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b2d0d4fc80 0x55b2d0d5f0a0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f89d0001eb0 tx=0x7f89d00324c0 comp rx=0 tx=0).stop 2024-03-08T21:07:30.610 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.630+0000 7f89f05a4740 1 -- 172.21.15.107:0/3957933986 shutdown_connections 2024-03-08T21:07:30.610 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.630+0000 7f89f05a4740 1 --2- 172.21.15.107:0/3957933986 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b2d0d59370 0x55b2d0d5fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:07:30.611 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.630+0000 7f89f05a4740 1 --2- 172.21.15.107:0/3957933986 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55b2d0dc0d80 0x55b2d0dc31a0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:07:30.611 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.630+0000 7f89f05a4740 1 --2- 172.21.15.107:0/3957933986 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b2d0d505a0 0x55b2d0d5f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:07:30.611 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.630+0000 7f89f05a4740 1 --2- 172.21.15.107:0/3957933986 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b2d0d4fc80 0x55b2d0d5f0a0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:07:30.611 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.630+0000 7f89f05a4740 1 -- 172.21.15.107:0/3957933986 >> 172.21.15.107:0/3957933986 conn(0x55b2d0d4a5a0 msgr2=0x55b2d0d4c250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:07:30.611 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.630+0000 7f89d4c00640 0 client.16047 ms_handle_reset on v2:172.21.15.149:6838/55290596 2024-03-08T21:07:30.611 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.630+0000 7f89f05a4740 1 -- 172.21.15.107:0/3957933986 shutdown_connections 2024-03-08T21:07:30.611 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.630+0000 7f89f05a4740 1 -- 172.21.15.107:0/3957933986 wait complete. 2024-03-08T21:07:30.611 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.630+0000 7f89f05a4740 20 client.16047 trim_cache size 0 max 16384 2024-03-08T21:07:30.611 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:{ 2024-03-08T21:07:30.611 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "message": "", 2024-03-08T21:07:30.611 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "return_code": 0 2024-03-08T21:07:30.611 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:} 2024-03-08T21:07:30.612 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.631+0000 7f89ed800640 1 -- 172.21.15.107:0/2107143583 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f89cc0a2a10 msgr2=0x7f89cc0a4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:07:30.612 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.631+0000 7f89ed800640 1 --2- 172.21.15.107:0/2107143583 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f89cc0a2a10 0x7f89cc0a4ed0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f89e002c5f0 tx=0x7f89e002c010 comp rx=0 tx=0).stop 2024-03-08T21:07:30.612 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.631+0000 7f89ed800640 1 -- 172.21.15.107:0/2107143583 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f89e810b8a0 msgr2=0x7f89e8199a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:07:30.612 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.631+0000 7f89ed800640 1 --2- 172.21.15.107:0/2107143583 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f89e810b8a0 0x7f89e8199a40 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f89e0036040 tx=0x7f89e0004270 comp rx=0 tx=0).stop 2024-03-08T21:07:30.612 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.631+0000 7f89ed800640 1 -- 172.21.15.107:0/2107143583 shutdown_connections 2024-03-08T21:07:30.612 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.631+0000 7f89ed800640 1 --2- 172.21.15.107:0/2107143583 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f89cc0a2a10 0x7f89cc0a4ed0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:07:30.613 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.631+0000 7f89ed800640 1 --2- 172.21.15.107:0/2107143583 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f89e810b8a0 0x7f89e8199a40 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:07:30.613 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.631+0000 7f89ed800640 1 --2- 172.21.15.107:0/2107143583 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f89e8102e80 0x7f89e819fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:07:30.613 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.631+0000 7f89ed800640 1 --2- 172.21.15.107:0/2107143583 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f89e8102560 0x7f89e819f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:07:30.613 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.631+0000 7f89ed800640 1 -- 172.21.15.107:0/2107143583 >> 172.21.15.107:0/2107143583 conn(0x7f89e8077310 msgr2=0x7f89e80fe700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:07:30.613 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.631+0000 7f89ed800640 1 -- 172.21.15.107:0/2107143583 shutdown_connections 2024-03-08T21:07:30.613 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:30.631+0000 7f89ed800640 1 -- 172.21.15.107:0/2107143583 wait complete. 2024-03-08T21:07:31.011 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:07:30 smithi149 ceph-mon[35042]: pgmap v218: 97 pgs: 97 active+clean; 4.7 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-08T21:07:31.124 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:07:30 smithi204 ceph-mon[34935]: pgmap v218: 97 pgs: 97 active+clean; 4.7 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-08T21:07:31.213 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:07:30 smithi107 ceph-mon[31381]: pgmap v218: 97 pgs: 97 active+clean; 4.7 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-08T21:07:33.011 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:07:32 smithi149 ceph-mon[35042]: pgmap v219: 97 pgs: 97 active+clean; 4.7 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-08T21:07:33.124 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:07:32 smithi204 ceph-mon[34935]: pgmap v219: 97 pgs: 97 active+clean; 4.7 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-08T21:07:33.213 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:07:32 smithi107 ceph-mon[31381]: pgmap v219: 97 pgs: 97 active+clean; 4.7 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-03-08T21:07:35.011 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:07:34 smithi149 ceph-mon[35042]: pgmap v220: 97 pgs: 97 active+clean; 4.7 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-08T21:07:35.124 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:07:34 smithi204 ceph-mon[34935]: pgmap v220: 97 pgs: 97 active+clean; 4.7 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-08T21:07:35.214 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:07:34 smithi107 ceph-mon[31381]: pgmap v220: 97 pgs: 97 active+clean; 4.7 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-08T21:07:37.011 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:07:36 smithi149 ceph-mon[35042]: pgmap v221: 97 pgs: 97 active+clean; 4.7 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:07:37.126 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:07:36 smithi204 ceph-mon[34935]: pgmap v221: 97 pgs: 97 active+clean; 4.7 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:07:37.213 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:07:36 smithi107 ceph-mon[31381]: pgmap v221: 97 pgs: 97 active+clean; 4.7 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:07:38.874 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:07:38 smithi204 ceph-mon[34935]: pgmap v222: 97 pgs: 97 active+clean; 4.7 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-08T21:07:39.011 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:07:38 smithi149 ceph-mon[35042]: pgmap v222: 97 pgs: 97 active+clean; 4.7 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-08T21:07:39.213 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:07:38 smithi107 ceph-mon[31381]: pgmap v222: 97 pgs: 97 active+clean; 4.7 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-08T21:07:41.011 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:07:40 smithi149 ceph-mon[35042]: pgmap v223: 97 pgs: 97 active+clean; 4.7 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:07:41.124 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:07:40 smithi204 ceph-mon[34935]: pgmap v223: 97 pgs: 97 active+clean; 4.7 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:07:41.214 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:07:40 smithi107 ceph-mon[31381]: pgmap v223: 97 pgs: 97 active+clean; 4.7 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:07:43.011 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:07:42 smithi149 ceph-mon[35042]: pgmap v224: 97 pgs: 97 active+clean; 4.7 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:07:43.124 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:07:42 smithi204 ceph-mon[34935]: pgmap v224: 97 pgs: 97 active+clean; 4.7 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:07:43.213 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:07:42 smithi107 ceph-mon[31381]: pgmap v224: 97 pgs: 97 active+clean; 4.7 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:07:43.762 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.783+0000 7f6763e00640 1 -- 172.21.15.107:0/3343439204 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6764103f00 msgr2=0x7f6764107f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:07:43.762 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.783+0000 7f6763e00640 1 --2- 172.21.15.107:0/3343439204 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6764103f00 0x7f6764107f40 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f6750009900 tx=0x7f675002bc30 comp rx=0 tx=0).stop 2024-03-08T21:07:43.762 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.783+0000 7f6763e00640 1 -- 172.21.15.107:0/3343439204 shutdown_connections 2024-03-08T21:07:43.762 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.783+0000 7f6763e00640 1 --2- 172.21.15.107:0/3343439204 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6764108620 0x7f67641101f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:07:43.762 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.783+0000 7f6763e00640 1 --2- 172.21.15.107:0/3343439204 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6764103f00 0x7f6764107f40 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:07:43.762 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.783+0000 7f6763e00640 1 --2- 172.21.15.107:0/3343439204 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6764103550 0x7f6764103930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:07:43.762 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.783+0000 7f6763e00640 1 -- 172.21.15.107:0/3343439204 >> 172.21.15.107:0/3343439204 conn(0x7f67640fedd0 msgr2=0x7f67641011f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:07:43.763 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.783+0000 7f6763e00640 1 -- 172.21.15.107:0/3343439204 shutdown_connections 2024-03-08T21:07:43.763 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.783+0000 7f6763e00640 1 -- 172.21.15.107:0/3343439204 wait complete. 2024-03-08T21:07:43.764 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.784+0000 7f6763e00640 1 Processor -- start 2024-03-08T21:07:43.764 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.784+0000 7f6763e00640 1 -- start start 2024-03-08T21:07:43.764 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.785+0000 7f6763e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6764103550 0x7f676419f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:07:43.764 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.785+0000 7f6763e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6764108620 0x7f676419fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:07:43.764 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.785+0000 7f6763e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6764199940 0x7f6764199df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:07:43.764 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.785+0000 7f6763e00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6764113560 con 0x7f6764108620 2024-03-08T21:07:43.764 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.785+0000 7f6763e00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67641133e0 con 0x7f6764103550 2024-03-08T21:07:43.765 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.785+0000 7f6763e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67641136e0 con 0x7f6764199940 2024-03-08T21:07:43.765 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.785+0000 7f6762000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6764108620 0x7f676419fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:07:43.765 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.785+0000 7f6762000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6764108620 0x7f676419fcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:59454/0 (socket says 172.21.15.107:59454) 2024-03-08T21:07:43.765 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.785+0000 7f6762000640 1 -- 172.21.15.107:0/1955630822 learned_addr learned my addr 172.21.15.107:0/1955630822 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:07:43.765 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.785+0000 7f6762a00640 1 --2- 172.21.15.107:0/1955630822 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6764103550 0x7f676419f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:07:43.765 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.785+0000 7f6763400640 1 --2- 172.21.15.107:0/1955630822 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6764199940 0x7f6764199df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:07:43.765 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.785+0000 7f6762000640 1 -- 172.21.15.107:0/1955630822 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6764199940 msgr2=0x7f6764199df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:07:43.765 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.785+0000 7f6762000640 1 --2- 172.21.15.107:0/1955630822 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6764199940 0x7f6764199df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:07:43.765 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.785+0000 7f6762000640 1 -- 172.21.15.107:0/1955630822 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6764103550 msgr2=0x7f676419f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:07:43.765 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.785+0000 7f6762000640 1 --2- 172.21.15.107:0/1955630822 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6764103550 0x7f676419f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:07:43.765 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.785+0000 7f6762000640 1 -- 172.21.15.107:0/1955630822 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f676419a460 con 0x7f6764108620 2024-03-08T21:07:43.765 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.785+0000 7f6763400640 1 --2- 172.21.15.107:0/1955630822 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6764199940 0x7f6764199df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T21:07:43.766 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.786+0000 7f6762000640 1 --2- 172.21.15.107:0/1955630822 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6764108620 0x7f676419fcd0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f6750036040 tx=0x7f675002c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:07:43.766 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.786+0000 7f6768a00640 1 -- 172.21.15.107:0/1955630822 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6750043020 con 0x7f6764108620 2024-03-08T21:07:43.766 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.786+0000 7f6768a00640 1 -- 172.21.15.107:0/1955630822 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f675002cbe0 con 0x7f6764108620 2024-03-08T21:07:43.767 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.786+0000 7f6763e00640 1 -- 172.21.15.107:0/1955630822 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f676419a6f0 con 0x7f6764108620 2024-03-08T21:07:43.767 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.786+0000 7f6768a00640 1 -- 172.21.15.107:0/1955630822 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6750003bf0 con 0x7f6764108620 2024-03-08T21:07:43.767 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.786+0000 7f6763e00640 1 -- 172.21.15.107:0/1955630822 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67641a6940 con 0x7f6764108620 2024-03-08T21:07:43.769 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.790+0000 7f676cc07740 1 Processor -- start 2024-03-08T21:07:43.769 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.790+0000 7f6768a00640 1 -- 172.21.15.107:0/1955630822 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) v1 ==== 137422+0+0 (secure 0 0 0) 0x7f6750046020 con 0x7f6764108620 2024-03-08T21:07:43.769 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.791+0000 7f676cc07740 1 -- start start 2024-03-08T21:07:43.770 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.791+0000 7f676cc07740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bc71257e80 0x55bc71258260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:07:43.770 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.791+0000 7f676cc07740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bc712587a0 0x55bc71261030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:07:43.770 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.791+0000 7f676cc07740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bc71261570 0x55bc71263960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:07:43.770 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.791+0000 7f676cc07740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc711bb3d0 con 0x55bc712587a0 2024-03-08T21:07:43.770 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.791+0000 7f676cc07740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc711646b0 con 0x55bc71257e80 2024-03-08T21:07:43.770 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.791+0000 7f676cc07740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc711b7470 con 0x55bc71261570 2024-03-08T21:07:43.770 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.791+0000 7f6762a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bc712587a0 0x55bc71261030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:07:43.771 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.791+0000 7f6762a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bc712587a0 0x55bc71261030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:59460/0 (socket says 172.21.15.107:59460) 2024-03-08T21:07:43.771 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.791+0000 7f6762a00640 1 -- 172.21.15.107:0/1776408928 learned_addr learned my addr 172.21.15.107:0/1776408928 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:07:43.771 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.791+0000 7f6763400640 1 --2- 172.21.15.107:0/1776408928 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bc71257e80 0x55bc71258260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:07:43.771 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.792+0000 7f6762000640 1 --2- 172.21.15.107:0/1776408928 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bc71261570 0x55bc71263960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:07:43.772 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.792+0000 7f6762a00640 1 -- 172.21.15.107:0/1776408928 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bc71261570 msgr2=0x55bc71263960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:07:43.772 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.792+0000 7f6762a00640 1 --2- 172.21.15.107:0/1776408928 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bc71261570 0x55bc71263960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:07:43.772 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.792+0000 7f6762a00640 1 -- 172.21.15.107:0/1776408928 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bc71257e80 msgr2=0x55bc71258260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:07:43.772 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.792+0000 7f6762a00640 1 --2- 172.21.15.107:0/1776408928 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bc71257e80 0x55bc71258260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:07:43.772 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.792+0000 7f6762a00640 1 -- 172.21.15.107:0/1776408928 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bc7118ab50 con 0x55bc712587a0 2024-03-08T21:07:43.772 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.792+0000 7f6768a00640 1 --2- 172.21.15.107:0/1955630822 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f67480a2960 0x7f67480a4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:07:43.772 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.792+0000 7f6768a00640 1 -- 172.21.15.107:0/1955630822 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f67500f10e0 con 0x7f6764108620 2024-03-08T21:07:43.772 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.792+0000 7f6762a00640 1 --2- 172.21.15.107:0/1776408928 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bc712587a0 0x55bc71261030 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f675400bc90 tx=0x7f675400f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71964ea13fb11bf1 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:07:43.773 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.792+0000 7f6763400640 1 --2- 172.21.15.107:0/1955630822 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f67480a2960 0x7f67480a4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:07:43.773 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.793+0000 7f674fe00640 1 -- 172.21.15.107:0/1776408928 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6754015070 con 0x55bc712587a0 2024-03-08T21:07:43.773 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.793+0000 7f674fe00640 1 -- 172.21.15.107:0/1776408928 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f675401a910 con 0x55bc712587a0 2024-03-08T21:07:43.773 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.793+0000 7f674fe00640 1 -- 172.21.15.107:0/1776408928 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67540118a0 con 0x55bc712587a0 2024-03-08T21:07:43.777 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.794+0000 7f676cc07740 1 -- 172.21.15.107:0/1776408928 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bc712587a0 msgr2=0x55bc71261030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:07:43.777 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.794+0000 7f6763400640 1 --2- 172.21.15.107:0/1955630822 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f67480a2960 0x7f67480a4e20 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f675c005fd0 tx=0x7f675c009680 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:07:43.777 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.794+0000 7f676cc07740 1 --2- 172.21.15.107:0/1776408928 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bc712587a0 0x55bc71261030 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f675400bc90 tx=0x7f675400f730 comp rx=0 tx=0).stop 2024-03-08T21:07:43.777 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.794+0000 7f676cc07740 1 -- 172.21.15.107:0/1776408928 shutdown_connections 2024-03-08T21:07:43.777 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.794+0000 7f676cc07740 1 --2- 172.21.15.107:0/1776408928 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bc71261570 0x55bc71263960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:07:43.777 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.794+0000 7f676cc07740 1 --2- 172.21.15.107:0/1776408928 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bc712587a0 0x55bc71261030 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:07:43.777 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.794+0000 7f676cc07740 1 --2- 172.21.15.107:0/1776408928 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bc71257e80 0x55bc71258260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:07:43.777 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.794+0000 7f676cc07740 1 -- 172.21.15.107:0/1776408928 >> 172.21.15.107:0/1776408928 conn(0x55bc712527a0 msgr2=0x55bc71252b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:07:43.777 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.794+0000 7f676cc07740 1 -- 172.21.15.107:0/1776408928 shutdown_connections 2024-03-08T21:07:43.777 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.794+0000 7f676cc07740 1 -- 172.21.15.107:0/1776408928 wait complete. 2024-03-08T21:07:43.777 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.795+0000 7f676cc07740 1 -- start start 2024-03-08T21:07:43.778 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.795+0000 7f676cc07740 1 Processor -- start 2024-03-08T21:07:43.778 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.795+0000 7f676cc07740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bc71257e80 0x55bc71260ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:07:43.778 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.795+0000 7f676cc07740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bc712587a0 0x55bc7125e230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:07:43.778 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.795+0000 7f676cc07740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bc71261570 0x55bc7125e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:07:43.778 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.795+0000 7f676cc07740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc712641a0 con 0x55bc712587a0 2024-03-08T21:07:43.778 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.795+0000 7f676cc07740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc711779f0 con 0x55bc71261570 2024-03-08T21:07:43.778 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.795+0000 7f676cc07740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc71264320 con 0x55bc71257e80 2024-03-08T21:07:43.778 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.796+0000 7f676cc07740 10 client.0 _ll_register_callbacks cb 0x55bc70ecfa00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T21:07:43.778 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.796+0000 7f6762000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bc712587a0 0x55bc7125e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:07:43.778 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.796+0000 7f6762000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bc712587a0 0x55bc7125e230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:59464/0 (socket says 172.21.15.107:59464) 2024-03-08T21:07:43.778 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.796+0000 7f6762000640 1 -- 172.21.15.107:0/3989899312 learned_addr learned my addr 172.21.15.107:0/3989899312 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:07:43.778 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.796+0000 7f6762a00640 1 --2- 172.21.15.107:0/3989899312 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bc71257e80 0x55bc71260ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:07:43.779 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.796+0000 7f6763400640 1 --2- 172.21.15.107:0/3989899312 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bc71261570 0x55bc7125e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:07:43.779 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.796+0000 7f6762000640 1 -- 172.21.15.107:0/3989899312 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bc71257e80 msgr2=0x55bc71260ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:07:43.779 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.796+0000 7f6762000640 1 --2- 172.21.15.107:0/3989899312 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bc71257e80 0x55bc71260ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:07:43.779 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.796+0000 7f6762000640 1 -- 172.21.15.107:0/3989899312 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bc71261570 msgr2=0x55bc7125e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:07:43.779 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.796+0000 7f6762000640 1 --2- 172.21.15.107:0/3989899312 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bc71261570 0x55bc7125e770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:07:43.779 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.796+0000 7f6762000640 1 -- 172.21.15.107:0/3989899312 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bc70ed0210 con 0x55bc712587a0 2024-03-08T21:07:43.779 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.796+0000 7f6763400640 1 --2- 172.21.15.107:0/3989899312 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bc71261570 0x55bc7125e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T21:07:43.779 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.797+0000 7f6762000640 1 --2- 172.21.15.107:0/3989899312 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bc712587a0 0x55bc7125e230 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f6750005ec0 tx=0x7f6750009560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42babcfa58afd94f server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:07:43.779 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.797+0000 7f674cc00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-03-08T21:07:43.779 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.797+0000 7f674cc00640 1 -- 172.21.15.107:0/3989899312 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67500bd380 con 0x55bc712587a0 2024-03-08T21:07:43.779 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.797+0000 7f674cc00640 1 -- 172.21.15.107:0/3989899312 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f67500a80e0 con 0x55bc712587a0 2024-03-08T21:07:43.779 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.797+0000 7f676cc07740 1 -- 172.21.15.107:0/3989899312 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f675401a910 con 0x55bc712587a0 2024-03-08T21:07:43.779 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.797+0000 7f674cc00640 1 -- 172.21.15.107:0/3989899312 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f67500bd1a0 con 0x55bc712587a0 2024-03-08T21:07:43.779 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.797+0000 7f676cc07740 10 client.16059 fetch_fsmap learned FSMap version 36 2024-03-08T21:07:43.779 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.797+0000 7f676cc07740 1 -- 172.21.15.107:0/3989899312 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x55bc70f9c620 con 0x55bc712587a0 2024-03-08T21:07:43.780 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.797+0000 7f674cc00640 1 -- 172.21.15.107:0/3989899312 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f675401a910 con 0x55bc712587a0 2024-03-08T21:07:43.780 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.798+0000 7f674cc00640 1 -- 172.21.15.107:0/3989899312 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 36) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f67500eeaa0 con 0x55bc712587a0 2024-03-08T21:07:43.780 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.798+0000 7f676cc07740 10 client.16059 fetch_fsmap finished waiting for FSMap version 36 2024-03-08T21:07:43.780 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.798+0000 7f676cc07740 10 client.16059 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-08T21:07:43.780 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.798+0000 7f676cc07740 20 client.16059 populate_metadata read hostname 'smithi107' 2024-03-08T21:07:43.780 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.798+0000 7f676cc07740 1 --2- 172.21.15.107:0/3989899312 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55bc712712f0 0x55bc71273720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:07:43.780 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.798+0000 7f676cc07740 4 client.16059 mds_command: new command op to 34475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-08T21:07:43.780 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.798+0000 7f676cc07740 1 -- 172.21.15.107:0/3989899312 --> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bc711c8910 con 0x55bc712712f0 2024-03-08T21:07:43.780 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.798+0000 7f6762a00640 1 --2- 172.21.15.107:0/3989899312 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55bc712712f0 0x55bc71273720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:07:43.781 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.799+0000 7f6762a00640 1 --2- 172.21.15.107:0/3989899312 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55bc712712f0 0x55bc71273720 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=124e9245ff906eaa server_cookie=284a7f204496cbe0 in_seq=0 out_seq=0 2024-03-08T21:07:43.781 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.799+0000 7f674cc00640 10 client.16059 ms_handle_connect on v2:172.21.15.149:6838/55290596 2024-03-08T21:07:43.781 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.801+0000 7f674cc00640 1 -- 172.21.15.107:0/3989899312 <== mds.0 v2:172.21.15.149:6838/55290596 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55bc711c8910 con 0x55bc712712f0 2024-03-08T21:07:43.782 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.801+0000 7f674cc00640 10 client.16059 handle_command_reply: tid=0 2024-03-08T21:07:43.782 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.801+0000 7f676cc07740 1 client.16059 shutdown 2024-03-08T21:07:43.782 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.801+0000 7f676cc07740 1 -- 172.21.15.107:0/3989899312 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bc712587a0 msgr2=0x55bc7125e230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:07:43.782 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.801+0000 7f676cc07740 1 --2- 172.21.15.107:0/3989899312 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bc712587a0 0x55bc7125e230 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f6750005ec0 tx=0x7f6750009560 comp rx=0 tx=0).stop 2024-03-08T21:07:43.782 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.801+0000 7f676cc07740 1 -- 172.21.15.107:0/3989899312 shutdown_connections 2024-03-08T21:07:43.782 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.801+0000 7f676cc07740 1 --2- 172.21.15.107:0/3989899312 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55bc712712f0 0x55bc71273720 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:07:43.782 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.801+0000 7f676cc07740 1 --2- 172.21.15.107:0/3989899312 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bc71261570 0x55bc7125e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:07:43.782 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.801+0000 7f676cc07740 1 --2- 172.21.15.107:0/3989899312 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bc712587a0 0x55bc7125e230 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:07:43.782 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.801+0000 7f676cc07740 1 --2- 172.21.15.107:0/3989899312 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bc71257e80 0x55bc71260ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:07:43.782 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.801+0000 7f674cc00640 0 client.16059 ms_handle_reset on v2:172.21.15.149:6838/55290596 2024-03-08T21:07:43.782 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.801+0000 7f676cc07740 1 -- 172.21.15.107:0/3989899312 >> 172.21.15.107:0/3989899312 conn(0x55bc712527a0 msgr2=0x55bc712544f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:07:43.782 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.801+0000 7f676cc07740 1 -- 172.21.15.107:0/3989899312 shutdown_connections 2024-03-08T21:07:43.782 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.801+0000 7f676cc07740 1 -- 172.21.15.107:0/3989899312 wait complete. 2024-03-08T21:07:43.783 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.802+0000 7f676cc07740 20 client.16059 trim_cache size 0 max 16384 2024-03-08T21:07:43.794 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.816+0000 7f676cc07740 1 Processor -- start 2024-03-08T21:07:43.795 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.816+0000 7f676cc07740 1 -- start start 2024-03-08T21:07:43.796 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.816+0000 7f676cc07740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bc71257e80 0x55bc71260960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:07:43.796 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.816+0000 7f676cc07740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bc712587a0 0x55bc71260ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:07:43.796 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.816+0000 7f676cc07740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bc71261570 0x55bc7125b490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:07:43.797 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.816+0000 7f676cc07740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67641aec60 con 0x55bc71257e80 2024-03-08T21:07:43.797 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.816+0000 7f676cc07740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67641b05e0 con 0x55bc712587a0 2024-03-08T21:07:43.797 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.816+0000 7f676cc07740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67641af2c0 con 0x55bc71261570 2024-03-08T21:07:43.797 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.817+0000 7f6762a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bc71257e80 0x55bc71260960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:07:43.797 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.817+0000 7f6762a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bc71257e80 0x55bc71260960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:59478/0 (socket says 172.21.15.107:59478) 2024-03-08T21:07:43.797 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.817+0000 7f6762a00640 1 -- 172.21.15.107:0/2463335802 learned_addr learned my addr 172.21.15.107:0/2463335802 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:07:43.797 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.817+0000 7f6762000640 1 --2- 172.21.15.107:0/2463335802 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bc712587a0 0x55bc71260ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:07:43.797 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.817+0000 7f6763400640 1 --2- 172.21.15.107:0/2463335802 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bc71261570 0x55bc7125b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:07:43.797 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.817+0000 7f6762a00640 1 -- 172.21.15.107:0/2463335802 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bc71261570 msgr2=0x55bc7125b490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:07:43.797 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.817+0000 7f6762a00640 1 --2- 172.21.15.107:0/2463335802 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bc71261570 0x55bc7125b490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:07:43.797 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.817+0000 7f6762a00640 1 -- 172.21.15.107:0/2463335802 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bc712587a0 msgr2=0x55bc71260ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:07:43.797 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.817+0000 7f6762a00640 1 --2- 172.21.15.107:0/2463335802 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bc712587a0 0x55bc71260ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:07:43.797 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.817+0000 7f6762a00640 1 -- 172.21.15.107:0/2463335802 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bc7125ea80 con 0x55bc71257e80 2024-03-08T21:07:43.798 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.817+0000 7f6763400640 1 --2- 172.21.15.107:0/2463335802 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bc71261570 0x55bc7125b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T21:07:43.798 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.817+0000 7f6762000640 1 --2- 172.21.15.107:0/2463335802 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bc712587a0 0x55bc71260ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T21:07:43.798 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.818+0000 7f6762a00640 1 --2- 172.21.15.107:0/2463335802 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bc71257e80 0x55bc71260960 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x55bc71257930 tx=0x7f675400b5a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=197a8b9912df568 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:07:43.798 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.818+0000 7f674f400640 1 -- 172.21.15.107:0/2463335802 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6754015040 con 0x55bc71257e80 2024-03-08T21:07:43.798 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.818+0000 7f674f400640 1 -- 172.21.15.107:0/2463335802 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f6754005450 con 0x55bc71257e80 2024-03-08T21:07:43.798 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.818+0000 7f674f400640 1 -- 172.21.15.107:0/2463335802 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67540274d0 con 0x55bc71257e80 2024-03-08T21:07:43.801 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.819+0000 7f676cc07740 1 -- 172.21.15.107:0/2463335802 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bc71257e80 msgr2=0x55bc71260960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:07:43.801 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.819+0000 7f676cc07740 1 --2- 172.21.15.107:0/2463335802 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bc71257e80 0x55bc71260960 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x55bc71257930 tx=0x7f675400b5a0 comp rx=0 tx=0).stop 2024-03-08T21:07:43.801 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.819+0000 7f676cc07740 1 -- 172.21.15.107:0/2463335802 shutdown_connections 2024-03-08T21:07:43.801 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.819+0000 7f676cc07740 1 --2- 172.21.15.107:0/2463335802 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bc71261570 0x55bc7125b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:07:43.802 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.819+0000 7f676cc07740 1 --2- 172.21.15.107:0/2463335802 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bc712587a0 0x55bc71260ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:07:43.802 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.819+0000 7f676cc07740 1 --2- 172.21.15.107:0/2463335802 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bc71257e80 0x55bc71260960 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:07:43.802 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.819+0000 7f676cc07740 1 -- 172.21.15.107:0/2463335802 >> 172.21.15.107:0/2463335802 conn(0x55bc712527a0 msgr2=0x55bc7125cb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:07:43.802 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.819+0000 7f676cc07740 1 -- 172.21.15.107:0/2463335802 shutdown_connections 2024-03-08T21:07:43.802 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.819+0000 7f676cc07740 1 -- 172.21.15.107:0/2463335802 wait complete. 2024-03-08T21:07:43.802 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.819+0000 7f676cc07740 1 -- start start 2024-03-08T21:07:43.802 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.819+0000 7f676cc07740 1 Processor -- start 2024-03-08T21:07:43.802 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.820+0000 7f676cc07740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bc71257e80 0x55bc712675c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:07:43.802 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.820+0000 7f676cc07740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bc712587a0 0x55bc71267b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:07:43.802 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.820+0000 7f676cc07740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bc71261570 0x55bc712c7850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:07:43.802 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.820+0000 7f676cc07740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67641aff80 con 0x55bc71261570 2024-03-08T21:07:43.802 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.820+0000 7f676cc07740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67641af920 con 0x55bc71257e80 2024-03-08T21:07:43.803 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.820+0000 7f676cc07740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67641acf00 con 0x55bc712587a0 2024-03-08T21:07:43.803 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.820+0000 7f676cc07740 10 client.0 _ll_register_callbacks cb 0x7f675400b440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T21:07:43.803 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.820+0000 7f6763400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bc71261570 0x55bc712c7850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:07:43.803 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.820+0000 7f6763400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bc71261570 0x55bc712c7850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:59492/0 (socket says 172.21.15.107:59492) 2024-03-08T21:07:43.803 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.820+0000 7f6763400640 1 -- 172.21.15.107:0/2170441847 learned_addr learned my addr 172.21.15.107:0/2170441847 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:07:43.803 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.820+0000 7f6762000640 1 --2- 172.21.15.107:0/2170441847 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bc712587a0 0x55bc71267b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:07:43.803 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.820+0000 7f6762a00640 1 --2- 172.21.15.107:0/2170441847 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bc71257e80 0x55bc712675c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:07:43.803 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.821+0000 7f6763400640 1 -- 172.21.15.107:0/2170441847 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bc712587a0 msgr2=0x55bc71267b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:07:43.803 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.821+0000 7f6763400640 1 --2- 172.21.15.107:0/2170441847 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bc712587a0 0x55bc71267b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:07:43.803 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.821+0000 7f6763400640 1 -- 172.21.15.107:0/2170441847 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bc71257e80 msgr2=0x55bc712675c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:07:43.803 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.821+0000 7f6763400640 1 --2- 172.21.15.107:0/2170441847 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bc71257e80 0x55bc712675c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:07:43.803 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.821+0000 7f6763400640 1 -- 172.21.15.107:0/2170441847 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bc712c7d90 con 0x55bc71261570 2024-03-08T21:07:43.803 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.821+0000 7f6763400640 1 --2- 172.21.15.107:0/2170441847 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bc71261570 0x55bc712c7850 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f675c009fd0 tx=0x7f675c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9013bae3be5952f server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:07:43.804 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.821+0000 7f674cc00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-03-08T21:07:43.804 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.821+0000 7f674cc00640 1 -- 172.21.15.107:0/2170441847 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f675c01f8f0 con 0x55bc71261570 2024-03-08T21:07:43.804 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.821+0000 7f674cc00640 1 -- 172.21.15.107:0/2170441847 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f675c01fa90 con 0x55bc71261570 2024-03-08T21:07:43.804 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.821+0000 7f676cc07740 1 -- 172.21.15.107:0/2170441847 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6754005450 con 0x55bc71261570 2024-03-08T21:07:43.804 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.822+0000 7f674cc00640 1 -- 172.21.15.107:0/2170441847 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f675c01fcd0 con 0x55bc71261570 2024-03-08T21:07:43.804 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.822+0000 7f676cc07740 10 client.16065 fetch_fsmap learned FSMap version 36 2024-03-08T21:07:43.804 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.822+0000 7f676cc07740 1 -- 172.21.15.107:0/2170441847 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x55bc712c9010 con 0x55bc71261570 2024-03-08T21:07:43.804 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.822+0000 7f674cc00640 1 -- 172.21.15.107:0/2170441847 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f6754005450 con 0x55bc71261570 2024-03-08T21:07:43.804 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.822+0000 7f674cc00640 1 -- 172.21.15.107:0/2170441847 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 36) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f675c026070 con 0x55bc71261570 2024-03-08T21:07:43.804 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.822+0000 7f676cc07740 10 client.16065 fetch_fsmap finished waiting for FSMap version 36 2024-03-08T21:07:43.804 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.822+0000 7f676cc07740 10 client.16065 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-08T21:07:43.804 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.822+0000 7f676cc07740 20 client.16065 populate_metadata read hostname 'smithi107' 2024-03-08T21:07:43.804 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.822+0000 7f676cc07740 1 --2- 172.21.15.107:0/2170441847 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55bc712c97c0 0x55bc712cbbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:07:43.805 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.822+0000 7f676cc07740 4 client.16065 mds_command: new command op to 34475 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-08T21:07:43.805 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.822+0000 7f676cc07740 1 -- 172.21.15.107:0/2170441847 --> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55bc711c2c60 con 0x55bc712c97c0 2024-03-08T21:07:43.805 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.823+0000 7f6762a00640 1 --2- 172.21.15.107:0/2170441847 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55bc712c97c0 0x55bc712cbbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:07:43.805 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.824+0000 7f674cc00640 10 client.16065 ms_handle_connect on v2:172.21.15.149:6838/55290596 2024-03-08T21:07:43.805 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.824+0000 7f6762a00640 1 --2- 172.21.15.107:0/2170441847 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55bc712c97c0 0x55bc712cbbe0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1d15052e29baaa6d server_cookie=d3989f354399cf80 in_seq=0 out_seq=0 2024-03-08T21:07:43.814 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.835+0000 7f674cc00640 1 -- 172.21.15.107:0/2170441847 <== mds.0 v2:172.21.15.149:6838/55290596 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55bc711c2c60 con 0x55bc712c97c0 2024-03-08T21:07:43.814 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.835+0000 7f674cc00640 10 client.16065 handle_command_reply: tid=0 2024-03-08T21:07:43.814 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.835+0000 7f676cc07740 1 client.16065 shutdown 2024-03-08T21:07:43.814 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.835+0000 7f676cc07740 1 -- 172.21.15.107:0/2170441847 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bc71261570 msgr2=0x55bc712c7850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:07:43.814 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.835+0000 7f676cc07740 1 --2- 172.21.15.107:0/2170441847 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bc71261570 0x55bc712c7850 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f675c009fd0 tx=0x7f675c004a40 comp rx=0 tx=0).stop 2024-03-08T21:07:43.814 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.835+0000 7f676cc07740 1 -- 172.21.15.107:0/2170441847 shutdown_connections 2024-03-08T21:07:43.814 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.835+0000 7f676cc07740 1 --2- 172.21.15.107:0/2170441847 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bc71261570 0x55bc712c7850 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:07:43.814 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.835+0000 7f676cc07740 1 --2- 172.21.15.107:0/2170441847 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bc712587a0 0x55bc71267b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:07:43.814 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.835+0000 7f676cc07740 1 --2- 172.21.15.107:0/2170441847 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55bc712c97c0 0x55bc712cbbe0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:07:43.815 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.835+0000 7f676cc07740 1 --2- 172.21.15.107:0/2170441847 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bc71257e80 0x55bc712675c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:07:43.815 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.835+0000 7f674cc00640 0 client.16065 ms_handle_reset on v2:172.21.15.149:6838/55290596 2024-03-08T21:07:43.815 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.835+0000 7f676cc07740 1 -- 172.21.15.107:0/2170441847 >> 172.21.15.107:0/2170441847 conn(0x55bc712527a0 msgr2=0x55bc71250f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:07:43.815 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.835+0000 7f676cc07740 1 -- 172.21.15.107:0/2170441847 shutdown_connections 2024-03-08T21:07:43.815 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.835+0000 7f676cc07740 1 -- 172.21.15.107:0/2170441847 wait complete. 2024-03-08T21:07:43.815 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.835+0000 7f676cc07740 20 client.16065 trim_cache size 0 max 16384 2024-03-08T21:07:43.815 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:{ 2024-03-08T21:07:43.815 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "message": "", 2024-03-08T21:07:43.815 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "return_code": 0 2024-03-08T21:07:43.815 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:} 2024-03-08T21:07:43.816 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.836+0000 7f6763e00640 1 -- 172.21.15.107:0/1955630822 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f67480a2960 msgr2=0x7f67480a4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:07:43.816 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.836+0000 7f6763e00640 1 --2- 172.21.15.107:0/1955630822 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f67480a2960 0x7f67480a4e20 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f675c005fd0 tx=0x7f675c009680 comp rx=0 tx=0).stop 2024-03-08T21:07:43.816 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.836+0000 7f6763e00640 1 -- 172.21.15.107:0/1955630822 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6764108620 msgr2=0x7f676419fcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:07:43.816 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.836+0000 7f6763e00640 1 --2- 172.21.15.107:0/1955630822 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6764108620 0x7f676419fcd0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f6750036040 tx=0x7f675002c7e0 comp rx=0 tx=0).stop 2024-03-08T21:07:43.816 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.836+0000 7f6763e00640 1 -- 172.21.15.107:0/1955630822 shutdown_connections 2024-03-08T21:07:43.816 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.836+0000 7f6763e00640 1 --2- 172.21.15.107:0/1955630822 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f67480a2960 0x7f67480a4e20 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:07:43.816 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.836+0000 7f6763e00640 1 --2- 172.21.15.107:0/1955630822 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6764199940 0x7f6764199df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:07:43.816 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.836+0000 7f6763e00640 1 --2- 172.21.15.107:0/1955630822 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6764108620 0x7f676419fcd0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:07:43.817 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.836+0000 7f6763e00640 1 --2- 172.21.15.107:0/1955630822 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6764103550 0x7f676419f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:07:43.817 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.836+0000 7f6763e00640 1 -- 172.21.15.107:0/1955630822 >> 172.21.15.107:0/1955630822 conn(0x7f67640fedd0 msgr2=0x7f6764100170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:07:43.817 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.836+0000 7f6763e00640 1 -- 172.21.15.107:0/1955630822 shutdown_connections 2024-03-08T21:07:43.817 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:43.836+0000 7f6763e00640 1 -- 172.21.15.107:0/1955630822 wait complete. 2024-03-08T21:07:45.011 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:07:44 smithi149 ceph-mon[35042]: pgmap v225: 97 pgs: 97 active+clean; 4.7 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-08T21:07:45.124 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:07:44 smithi204 ceph-mon[34935]: pgmap v225: 97 pgs: 97 active+clean; 4.7 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-08T21:07:45.213 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:07:44 smithi107 ceph-mon[31381]: pgmap v225: 97 pgs: 97 active+clean; 4.7 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-08T21:07:47.011 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:07:46 smithi149 ceph-mon[35042]: pgmap v226: 97 pgs: 97 active+clean; 4.7 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-08T21:07:47.015 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:07:46 smithi107 ceph-mon[31381]: pgmap v226: 97 pgs: 97 active+clean; 4.7 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-08T21:07:47.124 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:07:46 smithi204 ceph-mon[34935]: pgmap v226: 97 pgs: 97 active+clean; 4.7 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-08T21:07:48.874 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:07:48 smithi204 ceph-mon[34935]: pgmap v227: 97 pgs: 97 active+clean; 4.7 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-08T21:07:49.011 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:07:48 smithi149 ceph-mon[35042]: pgmap v227: 97 pgs: 97 active+clean; 4.7 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-08T21:07:49.213 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:07:48 smithi107 ceph-mon[31381]: pgmap v227: 97 pgs: 97 active+clean; 4.7 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-08T21:07:51.011 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:07:50 smithi149 ceph-mon[35042]: pgmap v228: 97 pgs: 97 active+clean; 4.7 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:07:51.124 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:07:50 smithi204 ceph-mon[34935]: pgmap v228: 97 pgs: 97 active+clean; 4.7 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:07:51.213 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:07:50 smithi107 ceph-mon[31381]: pgmap v228: 97 pgs: 97 active+clean; 4.7 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:07:52.874 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:07:52 smithi204 ceph-mon[34935]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T21:07:52.963 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:07:52 smithi107 ceph-mon[31381]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T21:07:53.011 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:07:52 smithi149 ceph-mon[35042]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T21:07:53.874 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:07:53 smithi204 ceph-mon[34935]: pgmap v229: 97 pgs: 97 active+clean; 4.7 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:07:53.963 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:07:53 smithi107 ceph-mon[31381]: pgmap v229: 97 pgs: 97 active+clean; 4.7 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:07:54.011 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:07:53 smithi149 ceph-mon[35042]: pgmap v229: 97 pgs: 97 active+clean; 4.7 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:07:54.874 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:07:54 smithi204 ceph-mon[34935]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T21:07:54.874 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:07:54 smithi204 ceph-mon[34935]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T21:07:54.874 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:07:54 smithi204 ceph-mon[34935]: from='mgr.15375 ' entity='mgr.x' 2024-03-08T21:07:54.963 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:07:54 smithi107 ceph-mon[31381]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T21:07:54.964 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:07:54 smithi107 ceph-mon[31381]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T21:07:54.964 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:07:54 smithi107 ceph-mon[31381]: from='mgr.15375 ' entity='mgr.x' 2024-03-08T21:07:55.011 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:07:54 smithi149 ceph-mon[35042]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T21:07:55.011 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:07:54 smithi149 ceph-mon[35042]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T21:07:55.011 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:07:54 smithi149 ceph-mon[35042]: from='mgr.15375 ' entity='mgr.x' 2024-03-08T21:07:55.874 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:07:55 smithi204 ceph-mon[34935]: pgmap v230: 97 pgs: 97 active+clean; 4.7 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-08T21:07:55.963 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:07:55 smithi107 ceph-mon[31381]: pgmap v230: 97 pgs: 97 active+clean; 4.7 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-08T21:07:56.011 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:07:55 smithi149 ceph-mon[35042]: pgmap v230: 97 pgs: 97 active+clean; 4.7 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-08T21:07:56.974 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:56.995+0000 7f3c15600640 1 -- 172.21.15.107:0/372421497 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3c10107400 msgr2=0x7f3c101077e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:07:56.974 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:56.995+0000 7f3c15600640 1 --2- 172.21.15.107:0/372421497 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3c10107400 0x7f3c101077e0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f3c00009900 tx=0x7f3c0002bc30 comp rx=0 tx=0).stop 2024-03-08T21:07:56.975 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:56.996+0000 7f3c15600640 1 -- 172.21.15.107:0/372421497 shutdown_connections 2024-03-08T21:07:56.975 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:56.996+0000 7f3c15600640 1 --2- 172.21.15.107:0/372421497 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3c10069310 0x7f3c1010e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:07:56.975 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:56.996+0000 7f3c15600640 1 --2- 172.21.15.107:0/372421497 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3c10068970 0x7f3c10068dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:07:56.975 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:56.996+0000 7f3c15600640 1 --2- 172.21.15.107:0/372421497 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3c10107400 0x7f3c101077e0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:07:56.975 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:56.996+0000 7f3c15600640 1 -- 172.21.15.107:0/372421497 >> 172.21.15.107:0/372421497 conn(0x7f3c10077040 msgr2=0x7f3c10077450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:07:56.975 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:56.996+0000 7f3c15600640 1 -- 172.21.15.107:0/372421497 shutdown_connections 2024-03-08T21:07:56.975 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:56.996+0000 7f3c15600640 1 -- 172.21.15.107:0/372421497 wait complete. 2024-03-08T21:07:56.976 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:56.997+0000 7f3c15600640 1 Processor -- start 2024-03-08T21:07:56.976 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:56.997+0000 7f3c15600640 1 -- start start 2024-03-08T21:07:56.976 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:56.997+0000 7f3c15600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3c10068970 0x7f3c1019f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:07:56.976 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:56.997+0000 7f3c15600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3c10069310 0x7f3c1019fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:07:56.976 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:56.997+0000 7f3c15600640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3c10107400 0x7f3c101999f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:07:56.977 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:56.997+0000 7f3c15600640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c10111500 con 0x7f3c10068970 2024-03-08T21:07:56.977 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:56.997+0000 7f3c15600640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c10111380 con 0x7f3c10107400 2024-03-08T21:07:56.977 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:56.997+0000 7f3c15600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c10111680 con 0x7f3c10069310 2024-03-08T21:07:56.977 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:56.998+0000 7f3c0ea00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3c10068970 0x7f3c1019f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:07:56.977 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:56.998+0000 7f3c0ea00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3c10068970 0x7f3c1019f840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:36828/0 (socket says 172.21.15.107:36828) 2024-03-08T21:07:56.977 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:56.998+0000 7f3c0ea00640 1 -- 172.21.15.107:0/3649454478 learned_addr learned my addr 172.21.15.107:0/3649454478 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:07:56.978 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:56.998+0000 7f3c0e000640 1 --2- 172.21.15.107:0/3649454478 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3c10069310 0x7f3c1019fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:07:56.978 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:56.998+0000 7f3c0f400640 1 --2- 172.21.15.107:0/3649454478 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3c10107400 0x7f3c101999f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:07:56.978 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:56.998+0000 7f3c0ea00640 1 -- 172.21.15.107:0/3649454478 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3c10069310 msgr2=0x7f3c1019fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:07:56.979 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:56.998+0000 7f3c0ea00640 1 --2- 172.21.15.107:0/3649454478 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3c10069310 0x7f3c1019fd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:07:56.979 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:56.998+0000 7f3c0ea00640 1 -- 172.21.15.107:0/3649454478 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3c10107400 msgr2=0x7f3c101999f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:07:56.979 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:56.998+0000 7f3c0ea00640 1 --2- 172.21.15.107:0/3649454478 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3c10107400 0x7f3c101999f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:07:56.979 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:56.998+0000 7f3c0ea00640 1 -- 172.21.15.107:0/3649454478 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c1019a170 con 0x7f3c10068970 2024-03-08T21:07:56.979 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:56.998+0000 7f3c0e000640 1 --2- 172.21.15.107:0/3649454478 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3c10069310 0x7f3c1019fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T21:07:56.979 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:56.998+0000 7f3c0f400640 1 --2- 172.21.15.107:0/3649454478 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3c10107400 0x7f3c101999f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T21:07:56.979 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:56.999+0000 7f3c0ea00640 1 --2- 172.21.15.107:0/3649454478 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3c10068970 0x7f3c1019f840 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f3c00005940 tx=0x7f3c0002c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:07:56.979 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:56.999+0000 7f3c07400640 1 -- 172.21.15.107:0/3649454478 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c00043020 con 0x7f3c10068970 2024-03-08T21:07:56.979 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:56.999+0000 7f3c07400640 1 -- 172.21.15.107:0/3649454478 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f3c0002cbe0 con 0x7f3c10068970 2024-03-08T21:07:56.979 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:56.999+0000 7f3c07400640 1 -- 172.21.15.107:0/3649454478 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c00003bf0 con 0x7f3c10068970 2024-03-08T21:07:56.979 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:56.999+0000 7f3c15600640 1 -- 172.21.15.107:0/3649454478 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c1019a400 con 0x7f3c10068970 2024-03-08T21:07:56.979 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:56.999+0000 7f3c15600640 1 -- 172.21.15.107:0/3649454478 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c101a6700 con 0x7f3c10068970 2024-03-08T21:07:56.981 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:57.002+0000 7f3c07400640 1 -- 172.21.15.107:0/3649454478 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) v1 ==== 137422+0+0 (secure 0 0 0) 0x7f3c00046020 con 0x7f3c10068970 2024-03-08T21:07:56.982 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:57.003+0000 7f3c183bf740 1 Processor -- start 2024-03-08T21:07:56.982 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:57.003+0000 7f3c183bf740 1 -- start start 2024-03-08T21:07:56.982 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:57.004+0000 7f3c183bf740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561055400c80 0x561055401060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:07:56.982 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:57.004+0000 7f3c183bf740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5610554015a0 0x561055409e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:07:56.984 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:57.004+0000 7f3c183bf740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56105540a370 0x56105540c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:07:56.984 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:57.004+0000 7f3c183bf740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610553643d0 con 0x5610554015a0 2024-03-08T21:07:56.984 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:57.004+0000 7f3c183bf740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x56105530d6b0 con 0x56105540a370 2024-03-08T21:07:56.984 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:57.004+0000 7f3c183bf740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x561055360470 con 0x561055400c80 2024-03-08T21:07:56.984 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:57.004+0000 7f3c0ea00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5610554015a0 0x561055409e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:07:56.984 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:57.004+0000 7f3c0ea00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5610554015a0 0x561055409e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:36832/0 (socket says 172.21.15.107:36832) 2024-03-08T21:07:56.984 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:57.004+0000 7f3c0ea00640 1 -- 172.21.15.107:0/4188615120 learned_addr learned my addr 172.21.15.107:0/4188615120 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:07:56.984 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:57.004+0000 7f3c0f400640 1 --2- 172.21.15.107:0/4188615120 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561055400c80 0x561055401060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:07:56.984 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:57.005+0000 7f3c0e000640 1 --2- 172.21.15.107:0/4188615120 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56105540a370 0x56105540c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:07:56.984 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:57.005+0000 7f3c0ea00640 1 -- 172.21.15.107:0/4188615120 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561055400c80 msgr2=0x561055401060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:07:56.985 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:57.005+0000 7f3c0ea00640 1 --2- 172.21.15.107:0/4188615120 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561055400c80 0x561055401060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:07:56.985 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:57.005+0000 7f3c0ea00640 1 -- 172.21.15.107:0/4188615120 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56105540a370 msgr2=0x56105540c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:07:56.985 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:57.005+0000 7f3c0ea00640 1 --2- 172.21.15.107:0/4188615120 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56105540a370 0x56105540c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:07:56.985 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:57.005+0000 7f3c07400640 1 --2- 172.21.15.107:0/3649454478 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f3bf40a2a90 0x7f3bf40a4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:07:56.985 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:57.005+0000 7f3c0ea00640 1 -- 172.21.15.107:0/4188615120 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561055333b50 con 0x5610554015a0 2024-03-08T21:07:56.985 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:57.005+0000 7f3c0f400640 1 --2- 172.21.15.107:0/3649454478 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f3bf40a2a90 0x7f3bf40a4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:07:56.985 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:57.005+0000 7f3c07400640 1 -- 172.21.15.107:0/3649454478 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f3c000f1090 con 0x7f3c10068970 2024-03-08T21:07:56.985 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:57.005+0000 7f3c0ea00640 1 --2- 172.21.15.107:0/4188615120 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5610554015a0 0x561055409e30 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f3c000ee7c0 tx=0x7f3c000b9af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5050131e30ef39a6 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:07:56.989 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:57.006+0000 7f3bfb400640 1 -- 172.21.15.107:0/4188615120 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c000bd2d0 con 0x5610554015a0 2024-03-08T21:07:56.989 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:57.006+0000 7f3bfb400640 1 -- 172.21.15.107:0/4188615120 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f3c000c0460 con 0x5610554015a0 2024-03-08T21:07:56.989 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:57.006+0000 7f3bfb400640 1 -- 172.21.15.107:0/4188615120 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c000c06b0 con 0x5610554015a0 2024-03-08T21:07:56.989 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:57.007+0000 7f3c183bf740 1 -- 172.21.15.107:0/4188615120 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5610554015a0 msgr2=0x561055409e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:07:56.989 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:57.007+0000 7f3c0f400640 1 --2- 172.21.15.107:0/3649454478 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f3bf40a2a90 0x7f3bf40a4f50 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f3c08005fd0 tx=0x7f3c08009780 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:07:56.990 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:57.007+0000 7f3c183bf740 1 --2- 172.21.15.107:0/4188615120 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5610554015a0 0x561055409e30 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f3c000ee7c0 tx=0x7f3c000b9af0 comp rx=0 tx=0).stop 2024-03-08T21:07:56.990 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:57.007+0000 7f3c183bf740 1 -- 172.21.15.107:0/4188615120 shutdown_connections 2024-03-08T21:07:56.990 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:57.007+0000 7f3c183bf740 1 --2- 172.21.15.107:0/4188615120 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56105540a370 0x56105540c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:07:56.990 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:57.007+0000 7f3c183bf740 1 --2- 172.21.15.107:0/4188615120 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5610554015a0 0x561055409e30 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:07:56.990 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:57.007+0000 7f3c183bf740 1 --2- 172.21.15.107:0/4188615120 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561055400c80 0x561055401060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:07:56.990 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:57.007+0000 7f3c183bf740 1 -- 172.21.15.107:0/4188615120 >> 172.21.15.107:0/4188615120 conn(0x5610553fb5a0 msgr2=0x5610553fb980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:07:56.990 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:57.007+0000 7f3c183bf740 1 -- 172.21.15.107:0/4188615120 shutdown_connections 2024-03-08T21:07:56.990 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:57.007+0000 7f3c183bf740 1 -- 172.21.15.107:0/4188615120 wait complete. 2024-03-08T21:07:56.990 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:57.007+0000 7f3c183bf740 1 -- start start 2024-03-08T21:07:56.990 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:57.008+0000 7f3c183bf740 1 Processor -- start 2024-03-08T21:07:56.990 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:57.008+0000 7f3c183bf740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561055400c80 0x561055409d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:07:56.990 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:57.008+0000 7f3c183bf740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5610554015a0 0x5610554070a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:07:56.991 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:57.008+0000 7f3c183bf740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56105540a370 0x561055407640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:07:56.991 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:57.008+0000 7f3c183bf740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x56105540cd60 con 0x561055400c80 2024-03-08T21:07:56.991 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:57.008+0000 7f3c183bf740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610553209f0 con 0x56105540a370 2024-03-08T21:07:56.991 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:57.008+0000 7f3c183bf740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56105540cee0 con 0x5610554015a0 2024-03-08T21:07:56.991 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:57.008+0000 7f3c183bf740 10 client.0 _ll_register_callbacks cb 0x561055078a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T21:07:56.991 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:57.008+0000 7f3c0ea00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561055400c80 0x561055409d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:07:56.991 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:57.009+0000 7f3c0ea00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561055400c80 0x561055409d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:36846/0 (socket says 172.21.15.107:36846) 2024-03-08T21:07:56.991 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:57.009+0000 7f3c0ea00640 1 -- 172.21.15.107:0/593062996 learned_addr learned my addr 172.21.15.107:0/593062996 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:07:56.991 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:57.009+0000 7f3c0e000640 1 --2- 172.21.15.107:0/593062996 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5610554015a0 0x5610554070a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:07:56.991 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:57.009+0000 7f3c0f400640 1 --2- 172.21.15.107:0/593062996 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56105540a370 0x561055407640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:07:56.991 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:57.009+0000 7f3c0ea00640 1 -- 172.21.15.107:0/593062996 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5610554015a0 msgr2=0x5610554070a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:07:56.991 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:57.009+0000 7f3c0ea00640 1 --2- 172.21.15.107:0/593062996 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5610554015a0 0x5610554070a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:07:56.991 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:57.009+0000 7f3c0ea00640 1 -- 172.21.15.107:0/593062996 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56105540a370 msgr2=0x561055407640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:07:56.991 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:57.009+0000 7f3c0ea00640 1 --2- 172.21.15.107:0/593062996 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56105540a370 0x561055407640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:07:56.992 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:57.009+0000 7f3c0ea00640 1 -- 172.21.15.107:0/593062996 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561055079210 con 0x561055400c80 2024-03-08T21:07:56.992 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:57.009+0000 7f3c0f400640 1 --2- 172.21.15.107:0/593062996 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56105540a370 0x561055407640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T21:07:56.992 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:57.009+0000 7f3c0ea00640 1 --2- 172.21.15.107:0/593062996 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561055400c80 0x561055409d10 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f3c00005ec0 tx=0x7f3c0002bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b32e0230566e3881 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:07:56.992 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:57.009+0000 7f3befe00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-03-08T21:07:56.992 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:57.009+0000 7f3befe00640 1 -- 172.21.15.107:0/593062996 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c000bd0f0 con 0x561055400c80 2024-03-08T21:07:56.992 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:57.010+0000 7f3befe00640 1 -- 172.21.15.107:0/593062996 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f3c000bba90 con 0x561055400c80 2024-03-08T21:07:56.992 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:57.010+0000 7f3c183bf740 1 -- 172.21.15.107:0/593062996 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561055371910 con 0x561055400c80 2024-03-08T21:07:56.992 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:57.010+0000 7f3befe00640 1 -- 172.21.15.107:0/593062996 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3c000f4c30 con 0x561055400c80 2024-03-08T21:07:56.992 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:57.010+0000 7f3c183bf740 10 client.16077 fetch_fsmap learned FSMap version 36 2024-03-08T21:07:56.992 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:57.010+0000 7f3c183bf740 1 -- 172.21.15.107:0/593062996 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x561055145620 con 0x561055400c80 2024-03-08T21:07:56.992 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:57.010+0000 7f3befe00640 1 -- 172.21.15.107:0/593062996 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x561055371910 con 0x561055400c80 2024-03-08T21:07:56.992 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:57.010+0000 7f3befe00640 1 -- 172.21.15.107:0/593062996 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 36) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f3c000ee9f0 con 0x561055400c80 2024-03-08T21:07:56.992 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:57.010+0000 7f3c183bf740 10 client.16077 fetch_fsmap finished waiting for FSMap version 36 2024-03-08T21:07:56.993 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:57.010+0000 7f3c183bf740 10 client.16077 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-08T21:07:56.993 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:57.010+0000 7f3c183bf740 20 client.16077 populate_metadata read hostname 'smithi107' 2024-03-08T21:07:56.993 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:57.010+0000 7f3c183bf740 1 --2- 172.21.15.107:0/593062996 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x561055419e00 0x56105541c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:07:56.993 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:57.010+0000 7f3c183bf740 4 client.16077 mds_command: new command op to 34475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-08T21:07:56.993 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:57.010+0000 7f3c183bf740 1 -- 172.21.15.107:0/593062996 --> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56105536bc60 con 0x561055419e00 2024-03-08T21:07:56.994 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:57.011+0000 7f3c0e000640 1 --2- 172.21.15.107:0/593062996 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x561055419e00 0x56105541c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:07:56.994 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:57.012+0000 7f3c0e000640 1 --2- 172.21.15.107:0/593062996 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x561055419e00 0x56105541c1f0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=84984c7d432c66af server_cookie=8a0ddfee819c1f98 in_seq=0 out_seq=0 2024-03-08T21:07:56.994 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:57.012+0000 7f3befe00640 10 client.16077 ms_handle_connect on v2:172.21.15.149:6838/55290596 2024-03-08T21:07:56.994 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:57.014+0000 7f3befe00640 1 -- 172.21.15.107:0/593062996 <== mds.0 v2:172.21.15.149:6838/55290596 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56105536bc60 con 0x561055419e00 2024-03-08T21:07:56.994 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:57.014+0000 7f3befe00640 10 client.16077 handle_command_reply: tid=0 2024-03-08T21:07:56.995 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:57.014+0000 7f3c183bf740 1 client.16077 shutdown 2024-03-08T21:07:56.995 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:57.014+0000 7f3c183bf740 1 -- 172.21.15.107:0/593062996 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561055400c80 msgr2=0x561055409d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:07:56.995 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:57.014+0000 7f3c183bf740 1 --2- 172.21.15.107:0/593062996 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561055400c80 0x561055409d10 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f3c00005ec0 tx=0x7f3c0002bc30 comp rx=0 tx=0).stop 2024-03-08T21:07:56.995 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:57.014+0000 7f3c183bf740 1 -- 172.21.15.107:0/593062996 shutdown_connections 2024-03-08T21:07:56.995 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:57.014+0000 7f3c183bf740 1 --2- 172.21.15.107:0/593062996 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x561055419e00 0x56105541c1f0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:07:56.995 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:57.014+0000 7f3c183bf740 1 --2- 172.21.15.107:0/593062996 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56105540a370 0x561055407640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:07:56.995 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:57.014+0000 7f3c183bf740 1 --2- 172.21.15.107:0/593062996 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5610554015a0 0x5610554070a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:07:56.995 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:57.014+0000 7f3c183bf740 1 --2- 172.21.15.107:0/593062996 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561055400c80 0x561055409d10 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:07:56.995 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:57.014+0000 7f3befe00640 0 client.16077 ms_handle_reset on v2:172.21.15.149:6838/55290596 2024-03-08T21:07:56.995 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:57.014+0000 7f3c183bf740 1 -- 172.21.15.107:0/593062996 >> 172.21.15.107:0/593062996 conn(0x5610553fb5a0 msgr2=0x5610553fd2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:07:56.995 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:57.014+0000 7f3c183bf740 1 -- 172.21.15.107:0/593062996 shutdown_connections 2024-03-08T21:07:56.995 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:57.014+0000 7f3c183bf740 1 -- 172.21.15.107:0/593062996 wait complete. 2024-03-08T21:07:56.995 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:57.014+0000 7f3c183bf740 20 client.16077 trim_cache size 0 max 16384 2024-03-08T21:07:57.011 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:07:56 smithi149 ceph-mon[35042]: pgmap v231: 97 pgs: 97 active+clean; 4.7 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:07:57.012 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:57.033+0000 7f3c183bf740 1 Processor -- start 2024-03-08T21:07:57.012 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:57.033+0000 7f3c183bf740 1 -- start start 2024-03-08T21:07:57.012 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:57.034+0000 7f3c183bf740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561055400c80 0x561055407040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:07:57.012 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:57.034+0000 7f3c183bf740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5610554015a0 0x5610554097d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:07:57.012 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:57.034+0000 7f3c183bf740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56105540a370 0x561055409d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:07:57.013 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:57.034+0000 7f3c183bf740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c101aee10 con 0x56105540a370 2024-03-08T21:07:57.013 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:57.034+0000 7f3c183bf740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c101b0790 con 0x5610554015a0 2024-03-08T21:07:57.013 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:57.034+0000 7f3c183bf740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c101af470 con 0x561055400c80 2024-03-08T21:07:57.014 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:57.034+0000 7f3c0f400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56105540a370 0x561055409d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:07:57.014 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:57.034+0000 7f3c0f400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56105540a370 0x561055409d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:36858/0 (socket says 172.21.15.107:36858) 2024-03-08T21:07:57.014 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:57.034+0000 7f3c0f400640 1 -- 172.21.15.107:0/3523805435 learned_addr learned my addr 172.21.15.107:0/3523805435 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:07:57.014 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:57.034+0000 7f3c0ea00640 1 --2- 172.21.15.107:0/3523805435 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561055400c80 0x561055407040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:07:57.014 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:57.034+0000 7f3c0e000640 1 --2- 172.21.15.107:0/3523805435 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5610554015a0 0x5610554097d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:07:57.014 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:57.035+0000 7f3c0f400640 1 -- 172.21.15.107:0/3523805435 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561055400c80 msgr2=0x561055407040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:07:57.015 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:57.035+0000 7f3c0f400640 1 --2- 172.21.15.107:0/3523805435 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561055400c80 0x561055407040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:07:57.015 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:57.035+0000 7f3c0f400640 1 -- 172.21.15.107:0/3523805435 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5610554015a0 msgr2=0x5610554097d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:07:57.015 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:57.035+0000 7f3c0f400640 1 --2- 172.21.15.107:0/3523805435 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5610554015a0 0x5610554097d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:07:57.015 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:57.035+0000 7f3c0f400640 1 -- 172.21.15.107:0/3523805435 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561055421040 con 0x56105540a370 2024-03-08T21:07:57.015 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:57.035+0000 7f3c0f400640 1 --2- 172.21.15.107:0/3523805435 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56105540a370 0x561055409d10 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f3c08010660 tx=0x7f3c08015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f9d4d76221c69d6 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:07:57.015 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:57.035+0000 7f3bfaa00640 1 -- 172.21.15.107:0/3523805435 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c0802b070 con 0x56105540a370 2024-03-08T21:07:57.015 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:57.035+0000 7f3bfaa00640 1 -- 172.21.15.107:0/3523805435 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f3c0801ec00 con 0x56105540a370 2024-03-08T21:07:57.015 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:57.035+0000 7f3bfaa00640 1 -- 172.21.15.107:0/3523805435 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c0801cdb0 con 0x56105540a370 2024-03-08T21:07:57.018 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:57.036+0000 7f3c183bf740 1 -- 172.21.15.107:0/3523805435 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56105540a370 msgr2=0x561055409d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:07:57.018 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:57.036+0000 7f3c183bf740 1 --2- 172.21.15.107:0/3523805435 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56105540a370 0x561055409d10 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f3c08010660 tx=0x7f3c08015000 comp rx=0 tx=0).stop 2024-03-08T21:07:57.018 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:57.036+0000 7f3c183bf740 1 -- 172.21.15.107:0/3523805435 shutdown_connections 2024-03-08T21:07:57.018 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:57.036+0000 7f3c183bf740 1 --2- 172.21.15.107:0/3523805435 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56105540a370 0x561055409d10 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:07:57.018 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:57.036+0000 7f3c183bf740 1 --2- 172.21.15.107:0/3523805435 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5610554015a0 0x5610554097d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:07:57.018 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:57.036+0000 7f3c183bf740 1 --2- 172.21.15.107:0/3523805435 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561055400c80 0x561055407040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:07:57.018 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:57.036+0000 7f3c183bf740 1 -- 172.21.15.107:0/3523805435 >> 172.21.15.107:0/3523805435 conn(0x5610553fb5a0 msgr2=0x561055406930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:07:57.018 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:57.036+0000 7f3c183bf740 1 -- 172.21.15.107:0/3523805435 shutdown_connections 2024-03-08T21:07:57.018 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:57.036+0000 7f3c183bf740 1 -- 172.21.15.107:0/3523805435 wait complete. 2024-03-08T21:07:57.018 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:57.037+0000 7f3c183bf740 1 -- start start 2024-03-08T21:07:57.018 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:57.037+0000 7f3c183bf740 1 Processor -- start 2024-03-08T21:07:57.019 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:57.037+0000 7f3c183bf740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561055400c80 0x561055411280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:07:57.019 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:57.037+0000 7f3c183bf740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5610554015a0 0x56105540f670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:07:57.019 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:57.037+0000 7f3c183bf740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56105540a370 0x56105540fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:07:57.019 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:57.037+0000 7f3c183bf740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c101b0130 con 0x56105540a370 2024-03-08T21:07:57.019 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:57.038+0000 7f3c183bf740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c101afad0 con 0x5610554015a0 2024-03-08T21:07:57.019 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:57.038+0000 7f3c183bf740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c101ad0b0 con 0x561055400c80 2024-03-08T21:07:57.019 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:57.038+0000 7f3c183bf740 10 client.0 _ll_register_callbacks cb 0x7f3c000f4a10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T21:07:57.019 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:57.038+0000 7f3c0f400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56105540a370 0x56105540fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:07:57.019 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:57.038+0000 7f3c0f400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56105540a370 0x56105540fbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:36860/0 (socket says 172.21.15.107:36860) 2024-03-08T21:07:57.019 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:57.038+0000 7f3c0f400640 1 -- 172.21.15.107:0/3718806858 learned_addr learned my addr 172.21.15.107:0/3718806858 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:07:57.019 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:57.038+0000 7f3c0ea00640 1 --2- 172.21.15.107:0/3718806858 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561055400c80 0x561055411280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:07:57.019 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:57.038+0000 7f3c0e000640 1 --2- 172.21.15.107:0/3718806858 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5610554015a0 0x56105540f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:07:57.019 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:57.038+0000 7f3c0f400640 1 -- 172.21.15.107:0/3718806858 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561055400c80 msgr2=0x561055411280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:07:57.020 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:57.038+0000 7f3c0f400640 1 --2- 172.21.15.107:0/3718806858 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561055400c80 0x561055411280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:07:57.020 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:57.038+0000 7f3c0f400640 1 -- 172.21.15.107:0/3718806858 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5610554015a0 msgr2=0x56105540f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:07:57.020 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:57.038+0000 7f3c0f400640 1 --2- 172.21.15.107:0/3718806858 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5610554015a0 0x56105540f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:07:57.020 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:57.038+0000 7f3c0f400640 1 -- 172.21.15.107:0/3718806858 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561055410430 con 0x56105540a370 2024-03-08T21:07:57.020 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:57.039+0000 7f3c0f400640 1 --2- 172.21.15.107:0/3718806858 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56105540a370 0x56105540fbb0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f3c08029040 tx=0x7f3c0801c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=291f552eeba65a2 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:07:57.020 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:57.039+0000 7f3befe00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-03-08T21:07:57.020 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:57.039+0000 7f3befe00640 1 -- 172.21.15.107:0/3718806858 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c08033070 con 0x56105540a370 2024-03-08T21:07:57.020 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:57.039+0000 7f3befe00640 1 -- 172.21.15.107:0/3718806858 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f3c0801cd30 con 0x56105540a370 2024-03-08T21:07:57.020 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:57.039+0000 7f3c183bf740 1 -- 172.21.15.107:0/3718806858 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3c0801ec00 con 0x56105540a370 2024-03-08T21:07:57.020 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:57.039+0000 7f3befe00640 1 -- 172.21.15.107:0/3718806858 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3c0803f050 con 0x56105540a370 2024-03-08T21:07:57.020 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:57.039+0000 7f3c183bf740 10 client.16083 fetch_fsmap learned FSMap version 36 2024-03-08T21:07:57.020 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:57.039+0000 7f3c183bf740 1 -- 172.21.15.107:0/3718806858 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x561055470fa0 con 0x56105540a370 2024-03-08T21:07:57.021 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:57.040+0000 7f3befe00640 1 -- 172.21.15.107:0/3718806858 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f3c0801ec00 con 0x56105540a370 2024-03-08T21:07:57.021 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:57.040+0000 7f3befe00640 1 -- 172.21.15.107:0/3718806858 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 36) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f3c0802e050 con 0x56105540a370 2024-03-08T21:07:57.022 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:57.040+0000 7f3c183bf740 10 client.16083 fetch_fsmap finished waiting for FSMap version 36 2024-03-08T21:07:57.022 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:57.040+0000 7f3c183bf740 10 client.16083 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-08T21:07:57.022 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:57.040+0000 7f3c183bf740 20 client.16083 populate_metadata read hostname 'smithi107' 2024-03-08T21:07:57.022 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:57.040+0000 7f3c183bf740 1 --2- 172.21.15.107:0/3718806858 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x561055471590 0x5610554739b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:07:57.022 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:57.040+0000 7f3c183bf740 4 client.16083 mds_command: new command op to 34475 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-08T21:07:57.022 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:57.040+0000 7f3c183bf740 1 -- 172.21.15.107:0/3718806858 --> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5610552ea5d0 con 0x561055471590 2024-03-08T21:07:57.022 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:57.041+0000 7f3c0ea00640 1 --2- 172.21.15.107:0/3718806858 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x561055471590 0x5610554739b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:07:57.022 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:57.041+0000 7f3c0ea00640 1 --2- 172.21.15.107:0/3718806858 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x561055471590 0x5610554739b0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9c4916e513da7ab2 server_cookie=1687ee5a92438e6c in_seq=0 out_seq=0 2024-03-08T21:07:57.022 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:57.041+0000 7f3befe00640 10 client.16083 ms_handle_connect on v2:172.21.15.149:6838/55290596 2024-03-08T21:07:57.030 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:57.050+0000 7f3befe00640 1 -- 172.21.15.107:0/3718806858 <== mds.0 v2:172.21.15.149:6838/55290596 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5610552ea5d0 con 0x561055471590 2024-03-08T21:07:57.030 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:57.050+0000 7f3befe00640 10 client.16083 handle_command_reply: tid=0 2024-03-08T21:07:57.030 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:57.051+0000 7f3c183bf740 1 client.16083 shutdown 2024-03-08T21:07:57.030 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:57.051+0000 7f3c183bf740 1 -- 172.21.15.107:0/3718806858 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56105540a370 msgr2=0x56105540fbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:07:57.030 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:57.051+0000 7f3c183bf740 1 --2- 172.21.15.107:0/3718806858 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56105540a370 0x56105540fbb0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f3c08029040 tx=0x7f3c0801c4a0 comp rx=0 tx=0).stop 2024-03-08T21:07:57.030 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:57.051+0000 7f3c183bf740 1 -- 172.21.15.107:0/3718806858 shutdown_connections 2024-03-08T21:07:57.030 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:57.051+0000 7f3c183bf740 1 --2- 172.21.15.107:0/3718806858 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56105540a370 0x56105540fbb0 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:07:57.030 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:57.051+0000 7f3c183bf740 1 --2- 172.21.15.107:0/3718806858 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x561055471590 0x5610554739b0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:07:57.030 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:57.051+0000 7f3c183bf740 1 --2- 172.21.15.107:0/3718806858 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5610554015a0 0x56105540f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:07:57.030 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:57.051+0000 7f3c183bf740 1 --2- 172.21.15.107:0/3718806858 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561055400c80 0x561055411280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:07:57.030 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:57.051+0000 7f3c183bf740 1 -- 172.21.15.107:0/3718806858 >> 172.21.15.107:0/3718806858 conn(0x5610553fb5a0 msgr2=0x5610553fc9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:07:57.031 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:57.051+0000 7f3befe00640 0 client.16083 ms_handle_reset on v2:172.21.15.149:6838/55290596 2024-03-08T21:07:57.031 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:57.051+0000 7f3c183bf740 1 -- 172.21.15.107:0/3718806858 shutdown_connections 2024-03-08T21:07:57.031 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:57.051+0000 7f3c183bf740 1 -- 172.21.15.107:0/3718806858 wait complete. 2024-03-08T21:07:57.031 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:57.051+0000 7f3c183bf740 20 client.16083 trim_cache size 0 max 16384 2024-03-08T21:07:57.031 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:{ 2024-03-08T21:07:57.031 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "message": "", 2024-03-08T21:07:57.031 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "return_code": 0 2024-03-08T21:07:57.031 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:} 2024-03-08T21:07:57.032 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:57.052+0000 7f3c15600640 1 -- 172.21.15.107:0/3649454478 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f3bf40a2a90 msgr2=0x7f3bf40a4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:07:57.032 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:57.052+0000 7f3c15600640 1 --2- 172.21.15.107:0/3649454478 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f3bf40a2a90 0x7f3bf40a4f50 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f3c08005fd0 tx=0x7f3c08009780 comp rx=0 tx=0).stop 2024-03-08T21:07:57.032 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:57.052+0000 7f3c15600640 1 -- 172.21.15.107:0/3649454478 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3c10068970 msgr2=0x7f3c1019f840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:07:57.032 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:57.052+0000 7f3c15600640 1 --2- 172.21.15.107:0/3649454478 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3c10068970 0x7f3c1019f840 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f3c00005940 tx=0x7f3c0002c7e0 comp rx=0 tx=0).stop 2024-03-08T21:07:57.032 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:57.052+0000 7f3c15600640 1 -- 172.21.15.107:0/3649454478 shutdown_connections 2024-03-08T21:07:57.032 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:57.052+0000 7f3c15600640 1 --2- 172.21.15.107:0/3649454478 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f3bf40a2a90 0x7f3bf40a4f50 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:07:57.032 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:57.052+0000 7f3c15600640 1 --2- 172.21.15.107:0/3649454478 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3c10107400 0x7f3c101999f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:07:57.032 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:57.052+0000 7f3c15600640 1 --2- 172.21.15.107:0/3649454478 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3c10069310 0x7f3c1019fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:07:57.032 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:57.052+0000 7f3c15600640 1 --2- 172.21.15.107:0/3649454478 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3c10068970 0x7f3c1019f840 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:07:57.033 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:57.052+0000 7f3c15600640 1 -- 172.21.15.107:0/3649454478 >> 172.21.15.107:0/3649454478 conn(0x7f3c10077040 msgr2=0x7f3c1010c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:07:57.033 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:57.052+0000 7f3c15600640 1 -- 172.21.15.107:0/3649454478 shutdown_connections 2024-03-08T21:07:57.033 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:07:57.052+0000 7f3c15600640 1 -- 172.21.15.107:0/3649454478 wait complete. 2024-03-08T21:07:57.124 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:07:56 smithi204 ceph-mon[34935]: pgmap v231: 97 pgs: 97 active+clean; 4.7 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:07:57.213 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:07:56 smithi107 ceph-mon[31381]: pgmap v231: 97 pgs: 97 active+clean; 4.7 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:07:59.011 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:07:58 smithi149 ceph-mon[35042]: pgmap v232: 97 pgs: 97 active+clean; 4.7 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 511 B/s wr, 13 op/s 2024-03-08T21:07:59.124 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:07:58 smithi204 ceph-mon[34935]: pgmap v232: 97 pgs: 97 active+clean; 4.7 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 511 B/s wr, 13 op/s 2024-03-08T21:07:59.213 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:07:58 smithi107 ceph-mon[31381]: pgmap v232: 97 pgs: 97 active+clean; 4.7 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 511 B/s wr, 13 op/s 2024-03-08T21:08:01.011 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:08:00 smithi149 ceph-mon[35042]: pgmap v233: 97 pgs: 97 active+clean; 4.7 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:08:01.124 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:08:00 smithi204 ceph-mon[34935]: pgmap v233: 97 pgs: 97 active+clean; 4.7 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:08:01.213 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:08:00 smithi107 ceph-mon[31381]: pgmap v233: 97 pgs: 97 active+clean; 4.7 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:08:03.011 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:08:02 smithi149 ceph-mon[35042]: pgmap v234: 97 pgs: 97 active+clean; 4.7 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:08:03.124 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:08:02 smithi204 ceph-mon[34935]: pgmap v234: 97 pgs: 97 active+clean; 4.7 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:08:03.213 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:08:02 smithi107 ceph-mon[31381]: pgmap v234: 97 pgs: 97 active+clean; 4.7 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:08:05.011 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:08:04 smithi149 ceph-mon[35042]: pgmap v235: 97 pgs: 97 active+clean; 5.3 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 101 KiB/s wr, 29 op/s 2024-03-08T21:08:05.124 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:08:04 smithi204 ceph-mon[34935]: pgmap v235: 97 pgs: 97 active+clean; 5.3 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 101 KiB/s wr, 29 op/s 2024-03-08T21:08:05.214 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:08:04 smithi107 ceph-mon[31381]: pgmap v235: 97 pgs: 97 active+clean; 5.3 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 101 KiB/s wr, 29 op/s 2024-03-08T21:08:07.011 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:08:06 smithi149 ceph-mon[35042]: pgmap v236: 97 pgs: 97 active+clean; 5.3 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 101 KiB/s wr, 25 op/s 2024-03-08T21:08:07.124 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:08:06 smithi204 ceph-mon[34935]: pgmap v236: 97 pgs: 97 active+clean; 5.3 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 101 KiB/s wr, 25 op/s 2024-03-08T21:08:07.213 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:08:06 smithi107 ceph-mon[31381]: pgmap v236: 97 pgs: 97 active+clean; 5.3 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 101 KiB/s wr, 25 op/s 2024-03-08T21:08:09.011 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:08:08 smithi149 ceph-mon[35042]: pgmap v237: 97 pgs: 97 active+clean; 5.3 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 101 KiB/s wr, 30 op/s 2024-03-08T21:08:09.124 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:08:08 smithi204 ceph-mon[34935]: pgmap v237: 97 pgs: 97 active+clean; 5.3 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 101 KiB/s wr, 30 op/s 2024-03-08T21:08:09.213 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:08:08 smithi107 ceph-mon[31381]: pgmap v237: 97 pgs: 97 active+clean; 5.3 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 101 KiB/s wr, 30 op/s 2024-03-08T21:08:10.183 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.203+0000 7eff6b200640 1 -- 172.21.15.107:0/1660917678 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7eff641034e0 msgr2=0x7eff64103960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:08:10.183 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.203+0000 7eff6b200640 1 --2- 172.21.15.107:0/1660917678 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7eff641034e0 0x7eff64103960 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7eff50009900 tx=0x7eff5002bc30 comp rx=0 tx=0).stop 2024-03-08T21:08:10.183 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.203+0000 7eff6b200640 1 -- 172.21.15.107:0/1660917678 shutdown_connections 2024-03-08T21:08:10.183 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.203+0000 7eff6b200640 1 --2- 172.21.15.107:0/1660917678 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7eff64103ea0 0x7eff64110360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:08:10.183 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.203+0000 7eff6b200640 1 --2- 172.21.15.107:0/1660917678 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7eff641034e0 0x7eff64103960 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:08:10.183 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.203+0000 7eff6b200640 1 --2- 172.21.15.107:0/1660917678 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7eff641095a0 0x7eff64109980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:08:10.183 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.203+0000 7eff6b200640 1 -- 172.21.15.107:0/1660917678 >> 172.21.15.107:0/1660917678 conn(0x7eff640fefb0 msgr2=0x7eff641013d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:08:10.183 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.204+0000 7eff6b200640 1 -- 172.21.15.107:0/1660917678 shutdown_connections 2024-03-08T21:08:10.183 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.204+0000 7eff6b200640 1 -- 172.21.15.107:0/1660917678 wait complete. 2024-03-08T21:08:10.183 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.204+0000 7eff6b200640 1 Processor -- start 2024-03-08T21:08:10.186 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.205+0000 7eff6b200640 1 -- start start 2024-03-08T21:08:10.186 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.205+0000 7eff6b200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7eff641034e0 0x7eff6419f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:08:10.186 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.205+0000 7eff6b200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7eff64103ea0 0x7eff6419fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:08:10.186 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.205+0000 7eff6b200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7eff641095a0 0x7eff641999f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:08:10.186 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.205+0000 7eff6b200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff64113730 con 0x7eff641034e0 2024-03-08T21:08:10.186 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.205+0000 7eff6b200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff641135b0 con 0x7eff641095a0 2024-03-08T21:08:10.186 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.205+0000 7eff6b200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff641138b0 con 0x7eff64103ea0 2024-03-08T21:08:10.186 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.205+0000 7eff63e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7eff641034e0 0x7eff6419f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:08:10.186 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.205+0000 7eff63e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7eff641034e0 0x7eff6419f7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:57564/0 (socket says 172.21.15.107:57564) 2024-03-08T21:08:10.187 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.205+0000 7eff63e00640 1 -- 172.21.15.107:0/3535062909 learned_addr learned my addr 172.21.15.107:0/3535062909 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:08:10.187 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.206+0000 7eff63400640 1 --2- 172.21.15.107:0/3535062909 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7eff64103ea0 0x7eff6419fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:08:10.187 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.206+0000 7eff69200640 1 --2- 172.21.15.107:0/3535062909 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7eff641095a0 0x7eff641999f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:08:10.187 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.206+0000 7eff63e00640 1 -- 172.21.15.107:0/3535062909 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7eff64103ea0 msgr2=0x7eff6419fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:08:10.187 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.206+0000 7eff63e00640 1 --2- 172.21.15.107:0/3535062909 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7eff64103ea0 0x7eff6419fd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:08:10.187 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.206+0000 7eff63e00640 1 -- 172.21.15.107:0/3535062909 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7eff641095a0 msgr2=0x7eff641999f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:08:10.187 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.206+0000 7eff63e00640 1 --2- 172.21.15.107:0/3535062909 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7eff641095a0 0x7eff641999f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:08:10.187 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.206+0000 7eff63e00640 1 -- 172.21.15.107:0/3535062909 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff6419a170 con 0x7eff641034e0 2024-03-08T21:08:10.187 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.206+0000 7eff69200640 1 --2- 172.21.15.107:0/3535062909 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7eff641095a0 0x7eff641999f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T21:08:10.187 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.206+0000 7eff63e00640 1 --2- 172.21.15.107:0/3535062909 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7eff641034e0 0x7eff6419f7f0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7eff5400ea80 tx=0x7eff5400ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:08:10.187 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.206+0000 7eff60c00640 1 -- 172.21.15.107:0/3535062909 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff5400ce40 con 0x7eff641034e0 2024-03-08T21:08:10.187 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.207+0000 7eff6b200640 1 -- 172.21.15.107:0/3535062909 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff6419a460 con 0x7eff641034e0 2024-03-08T21:08:10.187 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.207+0000 7eff60c00640 1 -- 172.21.15.107:0/3535062909 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7eff54005020 con 0x7eff641034e0 2024-03-08T21:08:10.188 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.207+0000 7eff60c00640 1 -- 172.21.15.107:0/3535062909 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff540116d0 con 0x7eff641034e0 2024-03-08T21:08:10.188 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.207+0000 7eff6b200640 1 -- 172.21.15.107:0/3535062909 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff641a66b0 con 0x7eff641034e0 2024-03-08T21:08:10.189 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.210+0000 7eff60c00640 1 -- 172.21.15.107:0/3535062909 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) v1 ==== 137422+0+0 (secure 0 0 0) 0x7eff540051d0 con 0x7eff641034e0 2024-03-08T21:08:10.189 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.210+0000 7eff6de4a740 1 Processor -- start 2024-03-08T21:08:10.189 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.211+0000 7eff6de4a740 1 -- start start 2024-03-08T21:08:10.190 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.211+0000 7eff6de4a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5650fad6ec80 0x5650fad6f060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:08:10.190 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.211+0000 7eff6de4a740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5650fad6f5a0 0x5650fad77e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:08:10.190 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.211+0000 7eff6de4a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5650fad78370 0x5650fad7a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:08:10.190 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.211+0000 7eff6de4a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650facd23d0 con 0x5650fad6ec80 2024-03-08T21:08:10.190 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.211+0000 7eff6de4a740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650fac7b6b0 con 0x5650fad6f5a0 2024-03-08T21:08:10.190 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.211+0000 7eff6de4a740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650facce470 con 0x5650fad78370 2024-03-08T21:08:10.190 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.211+0000 7eff69200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5650fad6ec80 0x5650fad6f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:08:10.190 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.211+0000 7eff69200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5650fad6ec80 0x5650fad6f060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:57572/0 (socket says 172.21.15.107:57572) 2024-03-08T21:08:10.190 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.211+0000 7eff69200640 1 -- 172.21.15.107:0/1612897355 learned_addr learned my addr 172.21.15.107:0/1612897355 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:08:10.191 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.212+0000 7eff63e00640 1 --2- 172.21.15.107:0/1612897355 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5650fad6f5a0 0x5650fad77e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:08:10.191 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.212+0000 7eff63400640 1 --2- 172.21.15.107:0/1612897355 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5650fad78370 0x5650fad7a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:08:10.192 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.212+0000 7eff69200640 1 -- 172.21.15.107:0/1612897355 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5650fad78370 msgr2=0x5650fad7a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:08:10.192 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.212+0000 7eff69200640 1 --2- 172.21.15.107:0/1612897355 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5650fad78370 0x5650fad7a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:08:10.192 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.212+0000 7eff69200640 1 -- 172.21.15.107:0/1612897355 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5650fad6f5a0 msgr2=0x5650fad77e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:08:10.192 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.212+0000 7eff69200640 1 --2- 172.21.15.107:0/1612897355 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5650fad6f5a0 0x5650fad77e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:08:10.192 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.212+0000 7eff69200640 1 -- 172.21.15.107:0/1612897355 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5650faca1b50 con 0x5650fad6ec80 2024-03-08T21:08:10.192 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.212+0000 7eff60c00640 1 --2- 172.21.15.107:0/3535062909 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7eff480a2a10 0x7eff480a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:08:10.192 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.212+0000 7eff69200640 1 --2- 172.21.15.107:0/1612897355 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5650fad6ec80 0x5650fad6f060 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7eff5c00bae0 tx=0x7eff5c00d340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94cbfd9365bc2b4f server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:08:10.192 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.212+0000 7eff60c00640 1 -- 172.21.15.107:0/3535062909 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 8629+0+0 (secure 0 0 0) 0x7eff540cf260 con 0x7eff641034e0 2024-03-08T21:08:10.192 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.212+0000 7eff63e00640 1 --2- 172.21.15.107:0/3535062909 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7eff480a2a10 0x7eff480a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:08:10.192 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.213+0000 7eff59600640 1 -- 172.21.15.107:0/1612897355 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff5c017420 con 0x5650fad6ec80 2024-03-08T21:08:10.192 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.213+0000 7eff59600640 1 -- 172.21.15.107:0/1612897355 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7eff5c017a70 con 0x5650fad6ec80 2024-03-08T21:08:10.192 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.213+0000 7eff59600640 1 -- 172.21.15.107:0/1612897355 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff5c00f9d0 con 0x5650fad6ec80 2024-03-08T21:08:10.193 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.214+0000 7eff6de4a740 1 -- 172.21.15.107:0/1612897355 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5650fad6ec80 msgr2=0x5650fad6f060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:08:10.193 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.214+0000 7eff63e00640 1 --2- 172.21.15.107:0/3535062909 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7eff480a2a10 0x7eff480a4ed0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7eff540cc9f0 tx=0x7eff54005810 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:08:10.193 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.214+0000 7eff6de4a740 1 --2- 172.21.15.107:0/1612897355 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5650fad6ec80 0x5650fad6f060 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7eff5c00bae0 tx=0x7eff5c00d340 comp rx=0 tx=0).stop 2024-03-08T21:08:10.194 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.214+0000 7eff6de4a740 1 -- 172.21.15.107:0/1612897355 shutdown_connections 2024-03-08T21:08:10.194 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.214+0000 7eff6de4a740 1 --2- 172.21.15.107:0/1612897355 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5650fad78370 0x5650fad7a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:08:10.194 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.214+0000 7eff6de4a740 1 --2- 172.21.15.107:0/1612897355 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5650fad6f5a0 0x5650fad77e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:08:10.194 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.214+0000 7eff6de4a740 1 --2- 172.21.15.107:0/1612897355 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5650fad6ec80 0x5650fad6f060 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:08:10.194 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.214+0000 7eff6de4a740 1 -- 172.21.15.107:0/1612897355 >> 172.21.15.107:0/1612897355 conn(0x5650fad695a0 msgr2=0x5650fad69980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:08:10.194 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.214+0000 7eff6de4a740 1 -- 172.21.15.107:0/1612897355 shutdown_connections 2024-03-08T21:08:10.194 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.214+0000 7eff6de4a740 1 -- 172.21.15.107:0/1612897355 wait complete. 2024-03-08T21:08:10.194 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.215+0000 7eff6de4a740 1 -- start start 2024-03-08T21:08:10.194 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.215+0000 7eff6de4a740 1 Processor -- start 2024-03-08T21:08:10.196 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.215+0000 7eff6de4a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5650fad6ec80 0x5650fad77c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:08:10.197 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.215+0000 7eff6de4a740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5650fad6f5a0 0x5650fad75010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:08:10.197 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.215+0000 7eff6de4a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5650fad78370 0x5650fad75550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:08:10.197 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.215+0000 7eff6de4a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650fad7afa0 con 0x5650fad6ec80 2024-03-08T21:08:10.197 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.215+0000 7eff6de4a740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650fac8e9f0 con 0x5650fad6f5a0 2024-03-08T21:08:10.197 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.215+0000 7eff6de4a740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650fad7b120 con 0x5650fad78370 2024-03-08T21:08:10.197 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.216+0000 7eff6de4a740 10 client.0 _ll_register_callbacks cb 0x5650fa9e6a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T21:08:10.197 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.216+0000 7eff69200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5650fad6ec80 0x5650fad77c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:08:10.197 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.216+0000 7eff69200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5650fad6ec80 0x5650fad77c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:57578/0 (socket says 172.21.15.107:57578) 2024-03-08T21:08:10.197 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.216+0000 7eff69200640 1 -- 172.21.15.107:0/1838549568 learned_addr learned my addr 172.21.15.107:0/1838549568 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:08:10.197 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.216+0000 7eff63400640 1 --2- 172.21.15.107:0/1838549568 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5650fad6f5a0 0x5650fad75010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:08:10.197 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.216+0000 7eff69200640 1 -- 172.21.15.107:0/1838549568 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5650fad78370 msgr2=0x5650fad75550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:08:10.197 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.216+0000 7eff69200640 1 --2- 172.21.15.107:0/1838549568 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5650fad78370 0x5650fad75550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:08:10.198 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.216+0000 7eff69200640 1 -- 172.21.15.107:0/1838549568 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5650fad6f5a0 msgr2=0x5650fad75010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:08:10.198 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.216+0000 7eff69200640 1 --2- 172.21.15.107:0/1838549568 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5650fad6f5a0 0x5650fad75010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:08:10.198 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.216+0000 7eff69200640 1 -- 172.21.15.107:0/1838549568 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5650fa9e7210 con 0x5650fad6ec80 2024-03-08T21:08:10.198 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.216+0000 7eff69200640 1 --2- 172.21.15.107:0/1838549568 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5650fad6ec80 0x5650fad77c10 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7eff5c00f690 tx=0x7eff5c017ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b6f4b52c1b85152 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:08:10.198 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.216+0000 7eff4e000640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-03-08T21:08:10.198 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.217+0000 7eff4e000640 1 -- 172.21.15.107:0/1838549568 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff5c026070 con 0x5650fad6ec80 2024-03-08T21:08:10.198 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.217+0000 7eff4e000640 1 -- 172.21.15.107:0/1838549568 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 8629+0+0 (secure 0 0 0) 0x7eff5c00f730 con 0x5650fad6ec80 2024-03-08T21:08:10.198 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.217+0000 7eff6de4a740 1 -- 172.21.15.107:0/1838549568 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7eff5c017a70 con 0x5650fad6ec80 2024-03-08T21:08:10.198 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.217+0000 7eff4e000640 1 -- 172.21.15.107:0/1838549568 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7eff5c032050 con 0x5650fad6ec80 2024-03-08T21:08:10.198 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.217+0000 7eff6de4a740 10 client.16095 fetch_fsmap learned FSMap version 36 2024-03-08T21:08:10.198 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.217+0000 7eff6de4a740 1 -- 172.21.15.107:0/1838549568 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x5650faab3620 con 0x5650fad6ec80 2024-03-08T21:08:10.198 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.218+0000 7eff4e000640 1 -- 172.21.15.107:0/1838549568 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7eff5c017a70 con 0x5650fad6ec80 2024-03-08T21:08:10.199 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.218+0000 7eff4e000640 1 -- 172.21.15.107:0/1838549568 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 36) v1 ==== 7882+0+0 (secure 0 0 0) 0x7eff5c022930 con 0x5650fad6ec80 2024-03-08T21:08:10.199 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.218+0000 7eff6de4a740 10 client.16095 fetch_fsmap finished waiting for FSMap version 36 2024-03-08T21:08:10.199 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.218+0000 7eff6de4a740 10 client.16095 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-08T21:08:10.199 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.218+0000 7eff6de4a740 20 client.16095 populate_metadata read hostname 'smithi107' 2024-03-08T21:08:10.199 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.218+0000 7eff6de4a740 1 --2- 172.21.15.107:0/1838549568 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x5650fad880c0 0x5650fad8a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:08:10.199 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.218+0000 7eff6de4a740 4 client.16095 mds_command: new command op to 34475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-08T21:08:10.199 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.218+0000 7eff6de4a740 1 -- 172.21.15.107:0/1838549568 --> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5650facdf910 con 0x5650fad880c0 2024-03-08T21:08:10.200 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.219+0000 7eff63400640 1 --2- 172.21.15.107:0/1838549568 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x5650fad880c0 0x5650fad8a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:08:10.200 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.220+0000 7eff63400640 1 --2- 172.21.15.107:0/1838549568 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x5650fad880c0 0x5650fad8a4b0 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=588c942f08784403 server_cookie=33d60e2b8521138b in_seq=0 out_seq=0 2024-03-08T21:08:10.200 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.220+0000 7eff4e000640 10 client.16095 ms_handle_connect on v2:172.21.15.149:6838/55290596 2024-03-08T21:08:10.201 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.221+0000 7eff4e000640 1 -- 172.21.15.107:0/1838549568 <== mds.0 v2:172.21.15.149:6838/55290596 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5650facdf910 con 0x5650fad880c0 2024-03-08T21:08:10.201 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.221+0000 7eff4e000640 10 client.16095 handle_command_reply: tid=0 2024-03-08T21:08:10.201 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.221+0000 7eff6de4a740 1 client.16095 shutdown 2024-03-08T21:08:10.201 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.221+0000 7eff6de4a740 1 -- 172.21.15.107:0/1838549568 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5650fad6ec80 msgr2=0x5650fad77c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:08:10.201 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.221+0000 7eff6de4a740 1 --2- 172.21.15.107:0/1838549568 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5650fad6ec80 0x5650fad77c10 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7eff5c00f690 tx=0x7eff5c017ea0 comp rx=0 tx=0).stop 2024-03-08T21:08:10.201 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.222+0000 7eff6de4a740 1 -- 172.21.15.107:0/1838549568 shutdown_connections 2024-03-08T21:08:10.201 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.222+0000 7eff6de4a740 1 --2- 172.21.15.107:0/1838549568 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5650fad78370 0x5650fad75550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:08:10.201 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.222+0000 7eff6de4a740 1 --2- 172.21.15.107:0/1838549568 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x5650fad880c0 0x5650fad8a4b0 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:08:10.201 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.222+0000 7eff6de4a740 1 --2- 172.21.15.107:0/1838549568 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5650fad6f5a0 0x5650fad75010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:08:10.201 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.222+0000 7eff6de4a740 1 --2- 172.21.15.107:0/1838549568 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5650fad6ec80 0x5650fad77c10 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:08:10.201 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.222+0000 7eff4e000640 0 client.16095 ms_handle_reset on v2:172.21.15.149:6838/55290596 2024-03-08T21:08:10.201 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.222+0000 7eff6de4a740 1 -- 172.21.15.107:0/1838549568 >> 172.21.15.107:0/1838549568 conn(0x5650fad695a0 msgr2=0x5650fad6b2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:08:10.201 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.222+0000 7eff6de4a740 1 -- 172.21.15.107:0/1838549568 shutdown_connections 2024-03-08T21:08:10.201 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.222+0000 7eff6de4a740 1 -- 172.21.15.107:0/1838549568 wait complete. 2024-03-08T21:08:10.202 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.222+0000 7eff6de4a740 20 client.16095 trim_cache size 0 max 16384 2024-03-08T21:08:10.216 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.238+0000 7eff6de4a740 1 Processor -- start 2024-03-08T21:08:10.216 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.238+0000 7eff6de4a740 1 -- start start 2024-03-08T21:08:10.218 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.238+0000 7eff6de4a740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5650fad6ec80 0x5650fad748f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:08:10.218 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.238+0000 7eff6de4a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5650fad6f5a0 0x5650fad68540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:08:10.218 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.238+0000 7eff6de4a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5650fad78370 0x5650fad776d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:08:10.218 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.238+0000 7eff6de4a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff641aef40 con 0x5650fad6f5a0 2024-03-08T21:08:10.218 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.238+0000 7eff6de4a740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff641b08c0 con 0x5650fad6ec80 2024-03-08T21:08:10.219 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.238+0000 7eff6de4a740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff641af5a0 con 0x5650fad78370 2024-03-08T21:08:10.219 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.238+0000 7eff63400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5650fad6f5a0 0x5650fad68540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:08:10.219 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.238+0000 7eff63400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5650fad6f5a0 0x5650fad68540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:57582/0 (socket says 172.21.15.107:57582) 2024-03-08T21:08:10.219 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.239+0000 7eff63400640 1 -- 172.21.15.107:0/357657429 learned_addr learned my addr 172.21.15.107:0/357657429 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:08:10.219 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.239+0000 7eff69200640 1 --2- 172.21.15.107:0/357657429 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5650fad6ec80 0x5650fad748f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:08:10.219 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.239+0000 7eff63400640 1 -- 172.21.15.107:0/357657429 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5650fad78370 msgr2=0x5650fad776d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:08:10.219 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.239+0000 7eff63400640 1 --2- 172.21.15.107:0/357657429 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5650fad78370 0x5650fad776d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:08:10.219 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.239+0000 7eff63400640 1 -- 172.21.15.107:0/357657429 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5650fad6ec80 msgr2=0x5650fad748f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:08:10.219 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.239+0000 7eff63400640 1 --2- 172.21.15.107:0/357657429 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5650fad6ec80 0x5650fad748f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:08:10.219 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.239+0000 7eff63400640 1 -- 172.21.15.107:0/357657429 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5650fad64c10 con 0x5650fad6f5a0 2024-03-08T21:08:10.220 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.239+0000 7eff69200640 1 --2- 172.21.15.107:0/357657429 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5650fad6ec80 0x5650fad748f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T21:08:10.220 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.239+0000 7eff63400640 1 --2- 172.21.15.107:0/357657429 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5650fad6f5a0 0x5650fad68540 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7eff50009900 tx=0x7eff500058b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8efea34da352a6 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:08:10.220 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.240+0000 7eff58c00640 1 -- 172.21.15.107:0/357657429 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff5002ca80 con 0x5650fad6f5a0 2024-03-08T21:08:10.220 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.240+0000 7eff58c00640 1 -- 172.21.15.107:0/357657429 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7eff5002cc20 con 0x5650fad6f5a0 2024-03-08T21:08:10.220 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.240+0000 7eff58c00640 1 -- 172.21.15.107:0/357657429 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff500048c0 con 0x5650fad6f5a0 2024-03-08T21:08:10.224 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.240+0000 7eff6de4a740 1 -- 172.21.15.107:0/357657429 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5650fad6f5a0 msgr2=0x5650fad68540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:08:10.224 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.240+0000 7eff6de4a740 1 --2- 172.21.15.107:0/357657429 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5650fad6f5a0 0x5650fad68540 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7eff50009900 tx=0x7eff500058b0 comp rx=0 tx=0).stop 2024-03-08T21:08:10.224 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.241+0000 7eff6de4a740 1 -- 172.21.15.107:0/357657429 shutdown_connections 2024-03-08T21:08:10.224 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.241+0000 7eff6de4a740 1 --2- 172.21.15.107:0/357657429 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5650fad78370 0x5650fad776d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:08:10.224 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.241+0000 7eff6de4a740 1 --2- 172.21.15.107:0/357657429 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5650fad6f5a0 0x5650fad68540 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:08:10.224 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.241+0000 7eff6de4a740 1 --2- 172.21.15.107:0/357657429 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5650fad6ec80 0x5650fad748f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:08:10.225 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.241+0000 7eff6de4a740 1 -- 172.21.15.107:0/357657429 >> 172.21.15.107:0/357657429 conn(0x5650fad695a0 msgr2=0x5650fad67e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:08:10.225 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.241+0000 7eff6de4a740 1 -- 172.21.15.107:0/357657429 shutdown_connections 2024-03-08T21:08:10.225 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.241+0000 7eff6de4a740 1 -- 172.21.15.107:0/357657429 wait complete. 2024-03-08T21:08:10.225 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.241+0000 7eff6de4a740 1 -- start start 2024-03-08T21:08:10.225 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.241+0000 7eff6de4a740 1 Processor -- start 2024-03-08T21:08:10.225 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.242+0000 7eff6de4a740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5650fad6ec80 0x5650fad7f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:08:10.225 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.242+0000 7eff6de4a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5650fad6f5a0 0x5650fad7da50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:08:10.225 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.242+0000 7eff6de4a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5650fad78370 0x5650fad7df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:08:10.225 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.242+0000 7eff6de4a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff641b0260 con 0x5650fad6f5a0 2024-03-08T21:08:10.225 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.242+0000 7eff6de4a740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff641afc00 con 0x5650fad6ec80 2024-03-08T21:08:10.225 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.242+0000 7eff6de4a740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff641ad200 con 0x5650fad78370 2024-03-08T21:08:10.226 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.242+0000 7eff6de4a740 10 client.0 _ll_register_callbacks cb 0x7eff5c00b2c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T21:08:10.226 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.242+0000 7eff63400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5650fad6f5a0 0x5650fad7da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:08:10.226 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.242+0000 7eff63400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5650fad6f5a0 0x5650fad7da50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:57590/0 (socket says 172.21.15.107:57590) 2024-03-08T21:08:10.226 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.242+0000 7eff63400640 1 -- 172.21.15.107:0/2403865069 learned_addr learned my addr 172.21.15.107:0/2403865069 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:08:10.226 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.242+0000 7eff69200640 1 --2- 172.21.15.107:0/2403865069 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5650fad6ec80 0x5650fad7f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:08:10.226 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.242+0000 7eff69200640 1 --2- 172.21.15.107:0/2403865069 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5650fad78370 0x5650fad7df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:08:10.226 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.242+0000 7eff63400640 1 -- 172.21.15.107:0/2403865069 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5650fad78370 msgr2=0x5650fad7df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:08:10.226 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.243+0000 7eff63400640 1 --2- 172.21.15.107:0/2403865069 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5650fad78370 0x5650fad7df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:08:10.226 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.243+0000 7eff63400640 1 -- 172.21.15.107:0/2403865069 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5650fad6ec80 msgr2=0x5650fad7f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:08:10.227 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.243+0000 7eff63400640 1 --2- 172.21.15.107:0/2403865069 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5650fad6ec80 0x5650fad7f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:08:10.227 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.243+0000 7eff63400640 1 -- 172.21.15.107:0/2403865069 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5650fad7e7a0 con 0x5650fad6f5a0 2024-03-08T21:08:10.227 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.243+0000 7eff69200640 1 --2- 172.21.15.107:0/2403865069 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5650fad78370 0x5650fad7df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T21:08:10.227 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.243+0000 7eff63400640 1 --2- 172.21.15.107:0/2403865069 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5650fad6f5a0 0x5650fad7da50 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7eff500098d0 tx=0x7eff5002cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31231e99acd01a92 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:08:10.227 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.243+0000 7eff4e000640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-03-08T21:08:10.227 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.243+0000 7eff4e000640 1 -- 172.21.15.107:0/2403865069 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff500037f0 con 0x5650fad6f5a0 2024-03-08T21:08:10.227 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.243+0000 7eff4e000640 1 -- 172.21.15.107:0/2403865069 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 8629+0+0 (secure 0 0 0) 0x7eff50003990 con 0x5650fad6f5a0 2024-03-08T21:08:10.227 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.243+0000 7eff6de4a740 1 -- 172.21.15.107:0/2403865069 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7eff5002cc20 con 0x5650fad6f5a0 2024-03-08T21:08:10.227 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.243+0000 7eff4e000640 1 -- 172.21.15.107:0/2403865069 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7eff50004490 con 0x5650fad6f5a0 2024-03-08T21:08:10.227 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.244+0000 7eff6de4a740 10 client.16101 fetch_fsmap learned FSMap version 36 2024-03-08T21:08:10.227 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.244+0000 7eff6de4a740 1 -- 172.21.15.107:0/2403865069 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x5650faddf170 con 0x5650fad6f5a0 2024-03-08T21:08:10.227 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.244+0000 7eff4e000640 1 -- 172.21.15.107:0/2403865069 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7eff5002cc20 con 0x5650fad6f5a0 2024-03-08T21:08:10.228 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.244+0000 7eff4e000640 1 -- 172.21.15.107:0/2403865069 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 36) v1 ==== 7882+0+0 (secure 0 0 0) 0x7eff5c00dd20 con 0x5650fad6f5a0 2024-03-08T21:08:10.228 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.244+0000 7eff6de4a740 10 client.16101 fetch_fsmap finished waiting for FSMap version 36 2024-03-08T21:08:10.228 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.244+0000 7eff6de4a740 10 client.16101 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-08T21:08:10.228 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.244+0000 7eff6de4a740 20 client.16101 populate_metadata read hostname 'smithi107' 2024-03-08T21:08:10.228 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.244+0000 7eff6de4a740 1 --2- 172.21.15.107:0/2403865069 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x5650faddf790 0x5650fade1bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:08:10.228 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.244+0000 7eff6de4a740 4 client.16101 mds_command: new command op to 34475 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-08T21:08:10.228 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.244+0000 7eff6de4a740 1 -- 172.21.15.107:0/2403865069 --> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5650facd9c60 con 0x5650faddf790 2024-03-08T21:08:10.228 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.245+0000 7eff69200640 1 --2- 172.21.15.107:0/2403865069 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x5650faddf790 0x5650fade1bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:08:10.228 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.245+0000 7eff4e000640 10 client.16101 ms_handle_connect on v2:172.21.15.149:6838/55290596 2024-03-08T21:08:10.228 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.245+0000 7eff69200640 1 --2- 172.21.15.107:0/2403865069 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x5650faddf790 0x5650fade1bb0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3a5ad819dad884ac server_cookie=cc76d6b3ea3b0b0 in_seq=0 out_seq=0 2024-03-08T21:08:10.251 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.272+0000 7eff4e000640 1 -- 172.21.15.107:0/2403865069 <== mds.0 v2:172.21.15.149:6838/55290596 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5650facd9c60 con 0x5650faddf790 2024-03-08T21:08:10.251 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.272+0000 7eff4e000640 10 client.16101 handle_command_reply: tid=0 2024-03-08T21:08:10.251 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.272+0000 7eff6de4a740 1 client.16101 shutdown 2024-03-08T21:08:10.251 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.272+0000 7eff6de4a740 1 -- 172.21.15.107:0/2403865069 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5650fad6f5a0 msgr2=0x5650fad7da50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:08:10.252 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.272+0000 7eff6de4a740 1 --2- 172.21.15.107:0/2403865069 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5650fad6f5a0 0x5650fad7da50 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7eff500098d0 tx=0x7eff5002cb90 comp rx=0 tx=0).stop 2024-03-08T21:08:10.252 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.273+0000 7eff6de4a740 1 -- 172.21.15.107:0/2403865069 shutdown_connections 2024-03-08T21:08:10.252 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.273+0000 7eff6de4a740 1 --2- 172.21.15.107:0/2403865069 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5650fad78370 0x5650fad7df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:08:10.252 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.273+0000 7eff6de4a740 1 --2- 172.21.15.107:0/2403865069 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5650fad6f5a0 0x5650fad7da50 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:08:10.252 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.273+0000 7eff6de4a740 1 --2- 172.21.15.107:0/2403865069 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x5650faddf790 0x5650fade1bb0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:08:10.252 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.273+0000 7eff6de4a740 1 --2- 172.21.15.107:0/2403865069 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5650fad6ec80 0x5650fad7f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:08:10.252 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.273+0000 7eff4e000640 0 client.16101 ms_handle_reset on v2:172.21.15.149:6838/55290596 2024-03-08T21:08:10.252 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.273+0000 7eff6de4a740 1 -- 172.21.15.107:0/2403865069 >> 172.21.15.107:0/2403865069 conn(0x5650fad695a0 msgr2=0x5650fad686c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:08:10.252 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.273+0000 7eff6de4a740 1 -- 172.21.15.107:0/2403865069 shutdown_connections 2024-03-08T21:08:10.252 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.273+0000 7eff6de4a740 1 -- 172.21.15.107:0/2403865069 wait complete. 2024-03-08T21:08:10.252 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.273+0000 7eff6de4a740 20 client.16101 trim_cache size 0 max 16384 2024-03-08T21:08:10.253 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:{ 2024-03-08T21:08:10.253 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "message": "", 2024-03-08T21:08:10.253 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "return_code": 0 2024-03-08T21:08:10.253 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:} 2024-03-08T21:08:10.253 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.274+0000 7eff6b200640 1 -- 172.21.15.107:0/3535062909 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7eff480a2a10 msgr2=0x7eff480a4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:08:10.254 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.274+0000 7eff6b200640 1 --2- 172.21.15.107:0/3535062909 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7eff480a2a10 0x7eff480a4ed0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7eff540cc9f0 tx=0x7eff54005810 comp rx=0 tx=0).stop 2024-03-08T21:08:10.254 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.274+0000 7eff6b200640 1 -- 172.21.15.107:0/3535062909 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7eff641034e0 msgr2=0x7eff6419f7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:08:10.254 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.274+0000 7eff6b200640 1 --2- 172.21.15.107:0/3535062909 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7eff641034e0 0x7eff6419f7f0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7eff5400ea80 tx=0x7eff5400ef40 comp rx=0 tx=0).stop 2024-03-08T21:08:10.254 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.274+0000 7eff6b200640 1 -- 172.21.15.107:0/3535062909 shutdown_connections 2024-03-08T21:08:10.254 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.274+0000 7eff6b200640 1 --2- 172.21.15.107:0/3535062909 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7eff480a2a10 0x7eff480a4ed0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:08:10.254 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.274+0000 7eff6b200640 1 --2- 172.21.15.107:0/3535062909 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7eff641095a0 0x7eff641999f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:08:10.254 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.274+0000 7eff6b200640 1 --2- 172.21.15.107:0/3535062909 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7eff64103ea0 0x7eff6419fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:08:10.254 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.274+0000 7eff6b200640 1 --2- 172.21.15.107:0/3535062909 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7eff641034e0 0x7eff6419f7f0 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:08:10.254 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.274+0000 7eff6b200640 1 -- 172.21.15.107:0/3535062909 >> 172.21.15.107:0/3535062909 conn(0x7eff640fefb0 msgr2=0x7eff64100750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:08:10.254 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.274+0000 7eff6b200640 1 -- 172.21.15.107:0/3535062909 shutdown_connections 2024-03-08T21:08:10.254 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:10.274+0000 7eff6b200640 1 -- 172.21.15.107:0/3535062909 wait complete. 2024-03-08T21:08:11.011 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:08:10 smithi149 ceph-mon[35042]: osdmap e96: 12 total, 12 up, 12 in 2024-03-08T21:08:11.011 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:08:10 smithi149 ceph-mon[35042]: pgmap v239: 97 pgs: 97 active+clean; 5.3 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 121 KiB/s wr, 30 op/s 2024-03-08T21:08:11.124 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:08:10 smithi204 ceph-mon[34935]: osdmap e96: 12 total, 12 up, 12 in 2024-03-08T21:08:11.124 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:08:10 smithi204 ceph-mon[34935]: pgmap v239: 97 pgs: 97 active+clean; 5.3 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 121 KiB/s wr, 30 op/s 2024-03-08T21:08:11.213 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:08:10 smithi107 ceph-mon[31381]: osdmap e96: 12 total, 12 up, 12 in 2024-03-08T21:08:11.213 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:08:10 smithi107 ceph-mon[31381]: pgmap v239: 97 pgs: 97 active+clean; 5.3 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 121 KiB/s wr, 30 op/s 2024-03-08T21:08:13.011 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:08:12 smithi149 ceph-mon[35042]: pgmap v240: 97 pgs: 97 active+clean; 5.3 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 121 KiB/s wr, 30 op/s 2024-03-08T21:08:13.124 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:08:12 smithi204 ceph-mon[34935]: pgmap v240: 97 pgs: 97 active+clean; 5.3 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 121 KiB/s wr, 30 op/s 2024-03-08T21:08:13.213 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:08:12 smithi107 ceph-mon[31381]: pgmap v240: 97 pgs: 97 active+clean; 5.3 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 121 KiB/s wr, 30 op/s 2024-03-08T21:08:15.011 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:08:14 smithi149 ceph-mon[35042]: pgmap v241: 97 pgs: 97 active+clean; 5.3 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 3.7 KiB/s wr, 13 op/s 2024-03-08T21:08:15.124 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:08:14 smithi204 ceph-mon[34935]: pgmap v241: 97 pgs: 97 active+clean; 5.3 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 3.7 KiB/s wr, 13 op/s 2024-03-08T21:08:15.213 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:08:14 smithi107 ceph-mon[31381]: pgmap v241: 97 pgs: 97 active+clean; 5.3 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 3.7 KiB/s wr, 13 op/s 2024-03-08T21:08:17.011 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:08:16 smithi149 ceph-mon[35042]: pgmap v242: 97 pgs: 97 active+clean; 5.3 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 3.7 KiB/s wr, 13 op/s 2024-03-08T21:08:17.015 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:08:16 smithi107 ceph-mon[31381]: pgmap v242: 97 pgs: 97 active+clean; 5.3 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 3.7 KiB/s wr, 13 op/s 2024-03-08T21:08:17.124 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:08:16 smithi204 ceph-mon[34935]: pgmap v242: 97 pgs: 97 active+clean; 5.3 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 3.7 KiB/s wr, 13 op/s 2024-03-08T21:08:19.011 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:08:18 smithi149 ceph-mon[35042]: pgmap v243: 97 pgs: 97 active+clean; 5.3 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 3.6 KiB/s wr, 13 op/s 2024-03-08T21:08:19.124 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:08:18 smithi204 ceph-mon[34935]: pgmap v243: 97 pgs: 97 active+clean; 5.3 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 3.6 KiB/s wr, 13 op/s 2024-03-08T21:08:19.213 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:08:18 smithi107 ceph-mon[31381]: pgmap v243: 97 pgs: 97 active+clean; 5.3 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 3.6 KiB/s wr, 13 op/s 2024-03-08T21:08:20.463 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:08:20 smithi107 ceph-mon[31381]: osdmap e97: 12 total, 12 up, 12 in 2024-03-08T21:08:20.511 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:08:20 smithi149 ceph-mon[35042]: osdmap e97: 12 total, 12 up, 12 in 2024-03-08T21:08:20.624 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:08:20 smithi204 ceph-mon[34935]: osdmap e97: 12 total, 12 up, 12 in 2024-03-08T21:08:21.463 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:08:21 smithi107 ceph-mon[31381]: pgmap v245: 97 pgs: 97 active+clean; 5.3 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 3.6 KiB/s wr, 13 op/s 2024-03-08T21:08:21.511 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:08:21 smithi149 ceph-mon[35042]: pgmap v245: 97 pgs: 97 active+clean; 5.3 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 3.6 KiB/s wr, 13 op/s 2024-03-08T21:08:21.624 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:08:21 smithi204 ceph-mon[34935]: pgmap v245: 97 pgs: 97 active+clean; 5.3 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 3.6 KiB/s wr, 13 op/s 2024-03-08T21:08:23.011 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:08:22 smithi149 ceph-mon[35042]: pgmap v246: 97 pgs: 97 active+clean; 5.3 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 3.6 KiB/s wr, 13 op/s 2024-03-08T21:08:23.124 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:08:22 smithi204 ceph-mon[34935]: pgmap v246: 97 pgs: 97 active+clean; 5.3 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 3.6 KiB/s wr, 13 op/s 2024-03-08T21:08:23.213 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:08:22 smithi107 ceph-mon[31381]: pgmap v246: 97 pgs: 97 active+clean; 5.3 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 3.6 KiB/s wr, 13 op/s 2024-03-08T21:08:23.404 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.421+0000 7faae2600640 1 -- 172.21.15.107:0/3484601775 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faadc073fe0 msgr2=0x7faadc10d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:08:23.404 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.421+0000 7faae2600640 1 --2- 172.21.15.107:0/3484601775 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faadc073fe0 0x7faadc10d600 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7faac8009900 tx=0x7faac802bcd0 comp rx=0 tx=0).stop 2024-03-08T21:08:23.404 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.421+0000 7faae2600640 1 -- 172.21.15.107:0/3484601775 shutdown_connections 2024-03-08T21:08:23.404 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.421+0000 7faae2600640 1 --2- 172.21.15.107:0/3484601775 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faadc10db40 0x7faadc10ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:08:23.404 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.421+0000 7faae2600640 1 --2- 172.21.15.107:0/3484601775 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faadc073fe0 0x7faadc10d600 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:08:23.404 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.421+0000 7faae2600640 1 --2- 172.21.15.107:0/3484601775 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faadc0736c0 0x7faadc073aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:08:23.404 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.421+0000 7faae2600640 1 -- 172.21.15.107:0/3484601775 >> 172.21.15.107:0/3484601775 conn(0x7faadc0fcdb0 msgr2=0x7faadc0ff1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:08:23.405 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.421+0000 7faae2600640 1 -- 172.21.15.107:0/3484601775 shutdown_connections 2024-03-08T21:08:23.405 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.421+0000 7faae2600640 1 -- 172.21.15.107:0/3484601775 wait complete. 2024-03-08T21:08:23.405 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.422+0000 7faae2600640 1 Processor -- start 2024-03-08T21:08:23.405 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.422+0000 7faae2600640 1 -- start start 2024-03-08T21:08:23.405 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.423+0000 7faae2600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faadc0736c0 0x7faadc19f860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:08:23.405 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.423+0000 7faae2600640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faadc073fe0 0x7faadc19fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:08:23.405 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.423+0000 7faae2600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faadc10db40 0x7faadc199a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:08:23.405 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.423+0000 7faae2600640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faadc1137a0 con 0x7faadc0736c0 2024-03-08T21:08:23.405 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.423+0000 7faae2600640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faadc113620 con 0x7faadc073fe0 2024-03-08T21:08:23.406 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.423+0000 7faae2600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faadc113920 con 0x7faadc10db40 2024-03-08T21:08:23.406 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.423+0000 7faadb400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faadc0736c0 0x7faadc19f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:08:23.406 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.423+0000 7faadb400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faadc0736c0 0x7faadc19f860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:55158/0 (socket says 172.21.15.107:55158) 2024-03-08T21:08:23.406 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.423+0000 7faadb400640 1 -- 172.21.15.107:0/1258811641 learned_addr learned my addr 172.21.15.107:0/1258811641 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:08:23.406 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.423+0000 7faadaa00640 1 --2- 172.21.15.107:0/1258811641 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faadc073fe0 0x7faadc19fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:08:23.406 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.423+0000 7faadbe00640 1 --2- 172.21.15.107:0/1258811641 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faadc10db40 0x7faadc199a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:08:23.406 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.424+0000 7faadb400640 1 -- 172.21.15.107:0/1258811641 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faadc10db40 msgr2=0x7faadc199a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:08:23.406 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.424+0000 7faadb400640 1 --2- 172.21.15.107:0/1258811641 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faadc10db40 0x7faadc199a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:08:23.406 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.424+0000 7faadb400640 1 -- 172.21.15.107:0/1258811641 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faadc073fe0 msgr2=0x7faadc19fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:08:23.406 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.424+0000 7faadb400640 1 --2- 172.21.15.107:0/1258811641 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faadc073fe0 0x7faadc19fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:08:23.406 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.424+0000 7faadb400640 1 -- 172.21.15.107:0/1258811641 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faadc19a1e0 con 0x7faadc0736c0 2024-03-08T21:08:23.406 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.424+0000 7faadb400640 1 --2- 172.21.15.107:0/1258811641 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faadc0736c0 0x7faadc19f860 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7faacc00d950 tx=0x7faacc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:08:23.406 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.424+0000 7faad3e00640 1 -- 172.21.15.107:0/1258811641 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faacc017070 con 0x7faadc0736c0 2024-03-08T21:08:23.406 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.424+0000 7faad3e00640 1 -- 172.21.15.107:0/1258811641 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7faacc009c30 con 0x7faadc0736c0 2024-03-08T21:08:23.407 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.424+0000 7faae2600640 1 -- 172.21.15.107:0/1258811641 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faadc19a4d0 con 0x7faadc0736c0 2024-03-08T21:08:23.407 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.424+0000 7faad3e00640 1 -- 172.21.15.107:0/1258811641 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faacc00f620 con 0x7faadc0736c0 2024-03-08T21:08:23.407 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.424+0000 7faae2600640 1 -- 172.21.15.107:0/1258811641 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faadc1a6720 con 0x7faadc0736c0 2024-03-08T21:08:23.407 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.428+0000 7faad3e00640 1 -- 172.21.15.107:0/1258811641 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) v1 ==== 137422+0+0 (secure 0 0 0) 0x7faacc0109a0 con 0x7faadc0736c0 2024-03-08T21:08:23.408 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.428+0000 7faae5378740 1 Processor -- start 2024-03-08T21:08:23.408 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.429+0000 7faae5378740 1 -- start start 2024-03-08T21:08:23.409 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.429+0000 7faae5378740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5628250c7c80 0x5628250c8060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:08:23.409 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.429+0000 7faae5378740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5628250c85a0 0x5628250d0e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:08:23.409 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.429+0000 7faae5378740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5628250d1370 0x5628250d3760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:08:23.409 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.429+0000 7faae5378740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x56282502b3d0 con 0x5628250c7c80 2024-03-08T21:08:23.409 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.429+0000 7faae5378740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x562824fd46b0 con 0x5628250d1370 2024-03-08T21:08:23.409 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.429+0000 7faae5378740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x562825027470 con 0x5628250c85a0 2024-03-08T21:08:23.409 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.429+0000 7faad3e00640 1 --2- 172.21.15.107:0/1258811641 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7faac00a2a90 0x7faac00a4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:08:23.409 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.429+0000 7faadbe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5628250c7c80 0x5628250c8060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:08:23.410 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.429+0000 7faad3e00640 1 -- 172.21.15.107:0/1258811641 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 8585+0+0 (secure 0 0 0) 0x7faacc0ceb30 con 0x7faadc0736c0 2024-03-08T21:08:23.410 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.429+0000 7faadbe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5628250c7c80 0x5628250c8060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:55172/0 (socket says 172.21.15.107:55172) 2024-03-08T21:08:23.411 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.430+0000 7faadbe00640 1 -- 172.21.15.107:0/4157000789 learned_addr learned my addr 172.21.15.107:0/4157000789 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:08:23.411 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.430+0000 7faadbe00640 1 --2- 172.21.15.107:0/1258811641 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7faac00a2a90 0x7faac00a4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:08:23.411 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.430+0000 7faadb400640 1 --2- 172.21.15.107:0/4157000789 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5628250c85a0 0x5628250d0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:08:23.411 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.430+0000 7faadaa00640 1 --2- 172.21.15.107:0/4157000789 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5628250d1370 0x5628250d3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:08:23.411 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.430+0000 7faadbe00640 1 -- 172.21.15.107:0/4157000789 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5628250c85a0 msgr2=0x5628250d0e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:08:23.411 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.430+0000 7faadbe00640 1 --2- 172.21.15.107:0/4157000789 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5628250c85a0 0x5628250d0e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:08:23.411 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.430+0000 7faadbe00640 1 -- 172.21.15.107:0/4157000789 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5628250d1370 msgr2=0x5628250d3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:08:23.411 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.430+0000 7faadbe00640 1 --2- 172.21.15.107:0/4157000789 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5628250d1370 0x5628250d3760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:08:23.411 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.430+0000 7faadbe00640 1 -- 172.21.15.107:0/4157000789 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562824ffab50 con 0x5628250c7c80 2024-03-08T21:08:23.411 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.431+0000 7faadbe00640 1 --2- 172.21.15.107:0/4157000789 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5628250c7c80 0x5628250c8060 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7faad4013da0 tx=0x7faad40107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70ecc38c4c832831 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:08:23.411 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.431+0000 7faadbe00640 1 --2- 172.21.15.107:0/1258811641 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7faac00a2a90 0x7faac00a4f50 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7faad4010ca0 tx=0x7faad4015000 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:08:23.411 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.431+0000 7faad0c00640 1 -- 172.21.15.107:0/4157000789 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faad4021070 con 0x5628250c7c80 2024-03-08T21:08:23.411 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.431+0000 7faad0c00640 1 -- 172.21.15.107:0/4157000789 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7faad400bb40 con 0x5628250c7c80 2024-03-08T21:08:23.411 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.431+0000 7faad0c00640 1 -- 172.21.15.107:0/4157000789 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faad400ee60 con 0x5628250c7c80 2024-03-08T21:08:23.414 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.432+0000 7faae5378740 1 -- 172.21.15.107:0/4157000789 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5628250c7c80 msgr2=0x5628250c8060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:08:23.415 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.432+0000 7faae5378740 1 --2- 172.21.15.107:0/4157000789 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5628250c7c80 0x5628250c8060 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7faad4013da0 tx=0x7faad40107a0 comp rx=0 tx=0).stop 2024-03-08T21:08:23.415 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.432+0000 7faae5378740 1 -- 172.21.15.107:0/4157000789 shutdown_connections 2024-03-08T21:08:23.415 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.432+0000 7faae5378740 1 --2- 172.21.15.107:0/4157000789 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5628250d1370 0x5628250d3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:08:23.415 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.432+0000 7faae5378740 1 --2- 172.21.15.107:0/4157000789 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5628250c85a0 0x5628250d0e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:08:23.415 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.432+0000 7faae5378740 1 --2- 172.21.15.107:0/4157000789 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5628250c7c80 0x5628250c8060 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:08:23.415 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.432+0000 7faae5378740 1 -- 172.21.15.107:0/4157000789 >> 172.21.15.107:0/4157000789 conn(0x5628250c25a0 msgr2=0x5628250c2980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:08:23.415 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.432+0000 7faae5378740 1 -- 172.21.15.107:0/4157000789 shutdown_connections 2024-03-08T21:08:23.415 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.432+0000 7faae5378740 1 -- 172.21.15.107:0/4157000789 wait complete. 2024-03-08T21:08:23.415 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.432+0000 7faae5378740 1 -- start start 2024-03-08T21:08:23.415 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.433+0000 7faae5378740 1 Processor -- start 2024-03-08T21:08:23.415 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.433+0000 7faae5378740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5628250c7c80 0x5628250d0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:08:23.415 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.433+0000 7faae5378740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5628250c85a0 0x5628250ce190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:08:23.415 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.433+0000 7faae5378740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5628250d1370 0x5628250ce880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:08:23.416 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.433+0000 7faae5378740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628250d3f40 con 0x5628250c85a0 2024-03-08T21:08:23.416 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.433+0000 7faae5378740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x562824fe79f0 con 0x5628250c7c80 2024-03-08T21:08:23.416 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.433+0000 7faae5378740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628250d40c0 con 0x5628250d1370 2024-03-08T21:08:23.416 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.433+0000 7faae5378740 10 client.0 _ll_register_callbacks cb 0x562824d3fa00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T21:08:23.416 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.434+0000 7faadaa00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5628250c85a0 0x5628250ce190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:08:23.416 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.434+0000 7faadaa00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5628250c85a0 0x5628250ce190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:55182/0 (socket says 172.21.15.107:55182) 2024-03-08T21:08:23.416 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.434+0000 7faadaa00640 1 -- 172.21.15.107:0/2632157380 learned_addr learned my addr 172.21.15.107:0/2632157380 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:08:23.416 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.434+0000 7faadb400640 1 --2- 172.21.15.107:0/2632157380 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5628250c7c80 0x5628250d0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:08:23.416 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.434+0000 7faadbe00640 1 --2- 172.21.15.107:0/2632157380 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5628250d1370 0x5628250ce880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:08:23.416 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.434+0000 7faadaa00640 1 -- 172.21.15.107:0/2632157380 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5628250d1370 msgr2=0x5628250ce880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:08:23.416 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.434+0000 7faadaa00640 1 --2- 172.21.15.107:0/2632157380 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5628250d1370 0x5628250ce880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:08:23.416 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.434+0000 7faadaa00640 1 -- 172.21.15.107:0/2632157380 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5628250c7c80 msgr2=0x5628250d0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:08:23.416 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.434+0000 7faadaa00640 1 --2- 172.21.15.107:0/2632157380 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5628250c7c80 0x5628250d0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:08:23.416 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.434+0000 7faadaa00640 1 -- 172.21.15.107:0/2632157380 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562824d40210 con 0x5628250c85a0 2024-03-08T21:08:23.417 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.434+0000 7faadaa00640 1 --2- 172.21.15.107:0/2632157380 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5628250c85a0 0x5628250ce190 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7faac8000c00 tx=0x7faac802cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37bb5d49fce691db server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:08:23.417 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.434+0000 7faac5600640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-03-08T21:08:23.417 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.435+0000 7faac5600640 1 -- 172.21.15.107:0/2632157380 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faac8042070 con 0x5628250c85a0 2024-03-08T21:08:23.417 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.435+0000 7faac5600640 1 -- 172.21.15.107:0/2632157380 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 8585+0+0 (secure 0 0 0) 0x7faac8005230 con 0x5628250c85a0 2024-03-08T21:08:23.417 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.435+0000 7faae5378740 1 -- 172.21.15.107:0/2632157380 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562825038910 con 0x5628250c85a0 2024-03-08T21:08:23.417 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.435+0000 7faac5600640 1 -- 172.21.15.107:0/2632157380 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7faac804e050 con 0x5628250c85a0 2024-03-08T21:08:23.417 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.435+0000 7faae5378740 10 client.16113 fetch_fsmap learned FSMap version 36 2024-03-08T21:08:23.417 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.435+0000 7faae5378740 1 -- 172.21.15.107:0/2632157380 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x562824e0c620 con 0x5628250c85a0 2024-03-08T21:08:23.417 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.435+0000 7faac5600640 1 -- 172.21.15.107:0/2632157380 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x562825038910 con 0x5628250c85a0 2024-03-08T21:08:23.419 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.436+0000 7faac5600640 1 -- 172.21.15.107:0/2632157380 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 36) v1 ==== 7882+0+0 (secure 0 0 0) 0x7faac80035f0 con 0x5628250c85a0 2024-03-08T21:08:23.419 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.436+0000 7faae5378740 10 client.16113 fetch_fsmap finished waiting for FSMap version 36 2024-03-08T21:08:23.419 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.436+0000 7faae5378740 10 client.16113 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-08T21:08:23.419 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.436+0000 7faae5378740 20 client.16113 populate_metadata read hostname 'smithi107' 2024-03-08T21:08:23.420 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.436+0000 7faae5378740 1 --2- 172.21.15.107:0/2632157380 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x5628250e1230 0x5628250e3620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:08:23.420 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.436+0000 7faae5378740 4 client.16113 mds_command: new command op to 34475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-08T21:08:23.420 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.436+0000 7faae5378740 1 -- 172.21.15.107:0/2632157380 --> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562825032c60 con 0x5628250e1230 2024-03-08T21:08:23.420 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.436+0000 7faadb400640 1 --2- 172.21.15.107:0/2632157380 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x5628250e1230 0x5628250e3620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:08:23.420 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.437+0000 7faadb400640 1 --2- 172.21.15.107:0/2632157380 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x5628250e1230 0x5628250e3620 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2a77bb4e0b75bd68 server_cookie=e41062894f2c52ac in_seq=0 out_seq=0 2024-03-08T21:08:23.420 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.437+0000 7faac5600640 10 client.16113 ms_handle_connect on v2:172.21.15.149:6838/55290596 2024-03-08T21:08:23.420 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.439+0000 7faac5600640 1 -- 172.21.15.107:0/2632157380 <== mds.0 v2:172.21.15.149:6838/55290596 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x562825032c60 con 0x5628250e1230 2024-03-08T21:08:23.420 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.439+0000 7faac5600640 10 client.16113 handle_command_reply: tid=0 2024-03-08T21:08:23.420 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.439+0000 7faae5378740 1 client.16113 shutdown 2024-03-08T21:08:23.420 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.439+0000 7faae5378740 1 -- 172.21.15.107:0/2632157380 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5628250c85a0 msgr2=0x5628250ce190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:08:23.420 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.439+0000 7faae5378740 1 --2- 172.21.15.107:0/2632157380 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5628250c85a0 0x5628250ce190 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7faac8000c00 tx=0x7faac802cab0 comp rx=0 tx=0).stop 2024-03-08T21:08:23.420 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.439+0000 7faae5378740 1 -- 172.21.15.107:0/2632157380 shutdown_connections 2024-03-08T21:08:23.420 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.439+0000 7faae5378740 1 --2- 172.21.15.107:0/2632157380 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5628250d1370 0x5628250ce880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:08:23.420 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.439+0000 7faae5378740 1 --2- 172.21.15.107:0/2632157380 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5628250c85a0 0x5628250ce190 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:08:23.421 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.439+0000 7faae5378740 1 --2- 172.21.15.107:0/2632157380 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x5628250e1230 0x5628250e3620 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:08:23.421 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.439+0000 7faae5378740 1 --2- 172.21.15.107:0/2632157380 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5628250c7c80 0x5628250d0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:08:23.421 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.439+0000 7faac5600640 0 client.16113 ms_handle_reset on v2:172.21.15.149:6838/55290596 2024-03-08T21:08:23.421 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.439+0000 7faae5378740 1 -- 172.21.15.107:0/2632157380 >> 172.21.15.107:0/2632157380 conn(0x5628250c25a0 msgr2=0x5628250c42f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:08:23.421 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.440+0000 7faae5378740 1 -- 172.21.15.107:0/2632157380 shutdown_connections 2024-03-08T21:08:23.421 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.440+0000 7faae5378740 1 -- 172.21.15.107:0/2632157380 wait complete. 2024-03-08T21:08:23.421 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.440+0000 7faae5378740 20 client.16113 trim_cache size 0 max 16384 2024-03-08T21:08:23.435 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.456+0000 7faae5378740 1 Processor -- start 2024-03-08T21:08:23.435 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.456+0000 7faae5378740 1 -- start start 2024-03-08T21:08:23.436 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.456+0000 7faae5378740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5628250c7c80 0x5628250c1540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:08:23.436 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.456+0000 7faae5378740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5628250c85a0 0x5628250d08c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:08:23.436 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.456+0000 7faae5378740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5628250d1370 0x5628250d0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:08:23.436 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.456+0000 7faae5378740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faadc1aeda0 con 0x5628250c85a0 2024-03-08T21:08:23.436 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.456+0000 7faae5378740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faadc1b0720 con 0x5628250c7c80 2024-03-08T21:08:23.436 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.456+0000 7faae5378740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faadc1af400 con 0x5628250d1370 2024-03-08T21:08:23.437 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.456+0000 7faadaa00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5628250c85a0 0x5628250d08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:08:23.437 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.457+0000 7faadaa00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5628250c85a0 0x5628250d08c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:55190/0 (socket says 172.21.15.107:55190) 2024-03-08T21:08:23.437 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.457+0000 7faadaa00640 1 -- 172.21.15.107:0/4114833245 learned_addr learned my addr 172.21.15.107:0/4114833245 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:08:23.437 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.457+0000 7faadb400640 1 --2- 172.21.15.107:0/4114833245 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5628250c7c80 0x5628250c1540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:08:23.438 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.457+0000 7faadbe00640 1 --2- 172.21.15.107:0/4114833245 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5628250d1370 0x5628250d0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:08:23.438 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.457+0000 7faadaa00640 1 -- 172.21.15.107:0/4114833245 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5628250d1370 msgr2=0x5628250d0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:08:23.438 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.457+0000 7faadaa00640 1 --2- 172.21.15.107:0/4114833245 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5628250d1370 0x5628250d0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:08:23.438 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.457+0000 7faadaa00640 1 -- 172.21.15.107:0/4114833245 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5628250c7c80 msgr2=0x5628250c1540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:08:23.438 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.457+0000 7faadaa00640 1 --2- 172.21.15.107:0/4114833245 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5628250c7c80 0x5628250c1540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:08:23.438 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.457+0000 7faadaa00640 1 -- 172.21.15.107:0/4114833245 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5628250d3ad0 con 0x5628250c85a0 2024-03-08T21:08:23.438 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.457+0000 7faadaa00640 1 --2- 172.21.15.107:0/4114833245 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5628250c85a0 0x5628250d08c0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7faac802c210 tx=0x7faac80043e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87c8b90676b62634 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:08:23.438 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.458+0000 7faac7e00640 1 -- 172.21.15.107:0/4114833245 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faac8042070 con 0x5628250c85a0 2024-03-08T21:08:23.438 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.458+0000 7faac7e00640 1 -- 172.21.15.107:0/4114833245 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7faac802c670 con 0x5628250c85a0 2024-03-08T21:08:23.438 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.458+0000 7faac7e00640 1 -- 172.21.15.107:0/4114833245 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faac80046c0 con 0x5628250c85a0 2024-03-08T21:08:23.441 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.459+0000 7faae5378740 1 -- 172.21.15.107:0/4114833245 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5628250c85a0 msgr2=0x5628250d08c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:08:23.441 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.459+0000 7faae5378740 1 --2- 172.21.15.107:0/4114833245 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5628250c85a0 0x5628250d08c0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7faac802c210 tx=0x7faac80043e0 comp rx=0 tx=0).stop 2024-03-08T21:08:23.441 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.459+0000 7faae5378740 1 -- 172.21.15.107:0/4114833245 shutdown_connections 2024-03-08T21:08:23.441 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.459+0000 7faae5378740 1 --2- 172.21.15.107:0/4114833245 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5628250d1370 0x5628250d0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:08:23.441 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.459+0000 7faae5378740 1 --2- 172.21.15.107:0/4114833245 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5628250c85a0 0x5628250d08c0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:08:23.441 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.459+0000 7faae5378740 1 --2- 172.21.15.107:0/4114833245 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5628250c7c80 0x5628250c1540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:08:23.441 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.459+0000 7faae5378740 1 -- 172.21.15.107:0/4114833245 >> 172.21.15.107:0/4114833245 conn(0x5628250c25a0 msgr2=0x5628250c0e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:08:23.441 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.459+0000 7faae5378740 1 -- 172.21.15.107:0/4114833245 shutdown_connections 2024-03-08T21:08:23.442 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.459+0000 7faae5378740 1 -- 172.21.15.107:0/4114833245 wait complete. 2024-03-08T21:08:23.442 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.459+0000 7faae5378740 1 -- start start 2024-03-08T21:08:23.442 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.459+0000 7faae5378740 1 Processor -- start 2024-03-08T21:08:23.442 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.460+0000 7faae5378740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5628250c7c80 0x5628250d6fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:08:23.442 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.460+0000 7faae5378740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5628250c85a0 0x5628250d74e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:08:23.442 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.460+0000 7faae5378740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5628250d1370 0x562825137710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:08:23.442 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.460+0000 7faae5378740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faadc1b00c0 con 0x5628250d1370 2024-03-08T21:08:23.442 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.460+0000 7faae5378740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faadc1afa60 con 0x5628250c85a0 2024-03-08T21:08:23.442 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.460+0000 7faae5378740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faadc1ad060 con 0x5628250c7c80 2024-03-08T21:08:23.442 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.460+0000 7faae5378740 10 client.0 _ll_register_callbacks cb 0x7faad400bd80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T21:08:23.442 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.460+0000 7faadbe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5628250d1370 0x562825137710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:08:23.442 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.460+0000 7faadbe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5628250d1370 0x562825137710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:55202/0 (socket says 172.21.15.107:55202) 2024-03-08T21:08:23.442 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.460+0000 7faadbe00640 1 -- 172.21.15.107:0/3340197263 learned_addr learned my addr 172.21.15.107:0/3340197263 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:08:23.443 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.461+0000 7faadb400640 1 --2- 172.21.15.107:0/3340197263 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5628250c7c80 0x5628250d6fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:08:23.443 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.461+0000 7faadaa00640 1 --2- 172.21.15.107:0/3340197263 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5628250c85a0 0x5628250d74e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:08:23.443 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.461+0000 7faadbe00640 1 -- 172.21.15.107:0/3340197263 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5628250c7c80 msgr2=0x5628250d6fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:08:23.443 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.461+0000 7faadbe00640 1 --2- 172.21.15.107:0/3340197263 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5628250c7c80 0x5628250d6fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:08:23.443 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.461+0000 7faadbe00640 1 -- 172.21.15.107:0/3340197263 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5628250c85a0 msgr2=0x5628250d74e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:08:23.443 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.461+0000 7faadbe00640 1 --2- 172.21.15.107:0/3340197263 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5628250c85a0 0x5628250d74e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:08:23.443 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.461+0000 7faadbe00640 1 -- 172.21.15.107:0/3340197263 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5628250e3990 con 0x5628250d1370 2024-03-08T21:08:23.443 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.461+0000 7faadaa00640 1 --2- 172.21.15.107:0/3340197263 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5628250c85a0 0x5628250d74e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T21:08:23.443 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.461+0000 7faadbe00640 1 --2- 172.21.15.107:0/3340197263 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5628250d1370 0x562825137710 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7faad4013d70 tx=0x7faad4003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=635e32aa6d29514f server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:08:23.443 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.461+0000 7faac5600640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-03-08T21:08:23.443 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.461+0000 7faac5600640 1 -- 172.21.15.107:0/3340197263 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faad4035060 con 0x5628250d1370 2024-03-08T21:08:23.443 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.461+0000 7faac5600640 1 -- 172.21.15.107:0/3340197263 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 8585+0+0 (secure 0 0 0) 0x7faad40052c0 con 0x5628250d1370 2024-03-08T21:08:23.443 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.461+0000 7faae5378740 1 -- 172.21.15.107:0/3340197263 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7faac802c670 con 0x5628250d1370 2024-03-08T21:08:23.443 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.462+0000 7faac5600640 1 -- 172.21.15.107:0/3340197263 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7faad4021040 con 0x5628250d1370 2024-03-08T21:08:23.443 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.462+0000 7faae5378740 10 client.16119 fetch_fsmap learned FSMap version 36 2024-03-08T21:08:23.444 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.462+0000 7faae5378740 1 -- 172.21.15.107:0/3340197263 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x562825138b50 con 0x5628250d1370 2024-03-08T21:08:23.444 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.462+0000 7faac5600640 1 -- 172.21.15.107:0/3340197263 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7faac802c670 con 0x5628250d1370 2024-03-08T21:08:23.445 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.462+0000 7faac5600640 1 -- 172.21.15.107:0/3340197263 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 36) v1 ==== 7882+0+0 (secure 0 0 0) 0x7faad402f070 con 0x5628250d1370 2024-03-08T21:08:23.445 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.462+0000 7faae5378740 10 client.16119 fetch_fsmap finished waiting for FSMap version 36 2024-03-08T21:08:23.445 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.462+0000 7faae5378740 10 client.16119 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-08T21:08:23.445 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.462+0000 7faae5378740 20 client.16119 populate_metadata read hostname 'smithi107' 2024-03-08T21:08:23.445 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.462+0000 7faae5378740 1 --2- 172.21.15.107:0/3340197263 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x562825138ff0 0x56282513b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:08:23.445 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.462+0000 7faae5378740 4 client.16119 mds_command: new command op to 34475 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-08T21:08:23.445 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.462+0000 7faae5378740 1 -- 172.21.15.107:0/3340197263 --> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x562824fb15d0 con 0x562825138ff0 2024-03-08T21:08:23.445 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.463+0000 7faadb400640 1 --2- 172.21.15.107:0/3340197263 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x562825138ff0 0x56282513b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:08:23.445 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.464+0000 7faadb400640 1 --2- 172.21.15.107:0/3340197263 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x562825138ff0 0x56282513b3e0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=96e696f49de92681 server_cookie=644c56190e215d5a in_seq=0 out_seq=0 2024-03-08T21:08:23.445 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.464+0000 7faac5600640 10 client.16119 ms_handle_connect on v2:172.21.15.149:6838/55290596 2024-03-08T21:08:23.455 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.475+0000 7faac5600640 1 -- 172.21.15.107:0/3340197263 <== mds.0 v2:172.21.15.149:6838/55290596 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x562824fb15d0 con 0x562825138ff0 2024-03-08T21:08:23.455 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.475+0000 7faac5600640 10 client.16119 handle_command_reply: tid=0 2024-03-08T21:08:23.455 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.475+0000 7faae5378740 1 client.16119 shutdown 2024-03-08T21:08:23.455 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.475+0000 7faae5378740 1 -- 172.21.15.107:0/3340197263 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5628250d1370 msgr2=0x562825137710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:08:23.455 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.475+0000 7faae5378740 1 --2- 172.21.15.107:0/3340197263 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5628250d1370 0x562825137710 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7faad4013d70 tx=0x7faad4003ba0 comp rx=0 tx=0).stop 2024-03-08T21:08:23.455 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.475+0000 7faae5378740 1 -- 172.21.15.107:0/3340197263 shutdown_connections 2024-03-08T21:08:23.455 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.475+0000 7faae5378740 1 --2- 172.21.15.107:0/3340197263 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5628250d1370 0x562825137710 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:08:23.455 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.475+0000 7faae5378740 1 --2- 172.21.15.107:0/3340197263 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x562825138ff0 0x56282513b3e0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:08:23.455 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.475+0000 7faae5378740 1 --2- 172.21.15.107:0/3340197263 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5628250c85a0 0x5628250d74e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:08:23.455 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.475+0000 7faae5378740 1 --2- 172.21.15.107:0/3340197263 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5628250c7c80 0x5628250d6fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:08:23.455 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.475+0000 7faae5378740 1 -- 172.21.15.107:0/3340197263 >> 172.21.15.107:0/3340197263 conn(0x5628250c25a0 msgr2=0x5628250cc680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:08:23.455 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.475+0000 7faac5600640 0 client.16119 ms_handle_reset on v2:172.21.15.149:6838/55290596 2024-03-08T21:08:23.455 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.475+0000 7faae5378740 1 -- 172.21.15.107:0/3340197263 shutdown_connections 2024-03-08T21:08:23.456 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.475+0000 7faae5378740 1 -- 172.21.15.107:0/3340197263 wait complete. 2024-03-08T21:08:23.456 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.475+0000 7faae5378740 20 client.16119 trim_cache size 0 max 16384 2024-03-08T21:08:23.456 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:{ 2024-03-08T21:08:23.456 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "message": "", 2024-03-08T21:08:23.456 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "return_code": 0 2024-03-08T21:08:23.456 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:} 2024-03-08T21:08:23.457 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.476+0000 7faae2600640 1 -- 172.21.15.107:0/1258811641 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7faac00a2a90 msgr2=0x7faac00a4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:08:23.457 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.476+0000 7faae2600640 1 --2- 172.21.15.107:0/1258811641 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7faac00a2a90 0x7faac00a4f50 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7faad4010ca0 tx=0x7faad4015000 comp rx=0 tx=0).stop 2024-03-08T21:08:23.457 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.476+0000 7faae2600640 1 -- 172.21.15.107:0/1258811641 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faadc0736c0 msgr2=0x7faadc19f860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:08:23.457 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.476+0000 7faae2600640 1 --2- 172.21.15.107:0/1258811641 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faadc0736c0 0x7faadc19f860 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7faacc00d950 tx=0x7faacc00de10 comp rx=0 tx=0).stop 2024-03-08T21:08:23.457 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.476+0000 7faae2600640 1 -- 172.21.15.107:0/1258811641 shutdown_connections 2024-03-08T21:08:23.457 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.476+0000 7faae2600640 1 --2- 172.21.15.107:0/1258811641 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7faac00a2a90 0x7faac00a4f50 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:08:23.457 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.476+0000 7faae2600640 1 --2- 172.21.15.107:0/1258811641 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faadc10db40 0x7faadc199a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:08:23.457 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.476+0000 7faae2600640 1 --2- 172.21.15.107:0/1258811641 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faadc073fe0 0x7faadc19fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:08:23.457 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.476+0000 7faae2600640 1 --2- 172.21.15.107:0/1258811641 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faadc0736c0 0x7faadc19f860 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:08:23.457 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.476+0000 7faae2600640 1 -- 172.21.15.107:0/1258811641 >> 172.21.15.107:0/1258811641 conn(0x7faadc0fcdb0 msgr2=0x7faadc0fe4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:08:23.458 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.476+0000 7faae2600640 1 -- 172.21.15.107:0/1258811641 shutdown_connections 2024-03-08T21:08:23.458 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:23.476+0000 7faae2600640 1 -- 172.21.15.107:0/1258811641 wait complete. 2024-03-08T21:08:25.011 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:08:24 smithi149 ceph-mon[35042]: pgmap v247: 97 pgs: 97 active+clean; 5.3 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 307 B/s wr, 10 op/s 2024-03-08T21:08:25.124 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:08:24 smithi204 ceph-mon[34935]: pgmap v247: 97 pgs: 97 active+clean; 5.3 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 307 B/s wr, 10 op/s 2024-03-08T21:08:25.213 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:08:24 smithi107 ceph-mon[31381]: pgmap v247: 97 pgs: 97 active+clean; 5.3 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 307 B/s wr, 10 op/s 2024-03-08T21:08:27.011 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:08:26 smithi149 ceph-mon[35042]: pgmap v248: 97 pgs: 97 active+clean; 5.3 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 307 B/s wr, 10 op/s 2024-03-08T21:08:27.124 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:08:26 smithi204 ceph-mon[34935]: pgmap v248: 97 pgs: 97 active+clean; 5.3 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 307 B/s wr, 10 op/s 2024-03-08T21:08:27.213 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:08:26 smithi107 ceph-mon[31381]: pgmap v248: 97 pgs: 97 active+clean; 5.3 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 307 B/s wr, 10 op/s 2024-03-08T21:08:29.010 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:08:28 smithi149 ceph-mon[35042]: pgmap v249: 97 pgs: 97 active+clean; 5.3 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 307 B/s wr, 10 op/s 2024-03-08T21:08:29.124 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:08:28 smithi204 ceph-mon[34935]: pgmap v249: 97 pgs: 97 active+clean; 5.3 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 307 B/s wr, 10 op/s 2024-03-08T21:08:29.213 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:08:28 smithi107 ceph-mon[31381]: pgmap v249: 97 pgs: 97 active+clean; 5.3 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 307 B/s wr, 10 op/s 2024-03-08T21:08:31.010 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:08:30 smithi149 ceph-mon[35042]: pgmap v250: 97 pgs: 97 active+clean; 5.3 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 267 B/s wr, 9 op/s 2024-03-08T21:08:31.124 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:08:30 smithi204 ceph-mon[34935]: pgmap v250: 97 pgs: 97 active+clean; 5.3 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 267 B/s wr, 9 op/s 2024-03-08T21:08:31.213 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:08:30 smithi107 ceph-mon[31381]: pgmap v250: 97 pgs: 97 active+clean; 5.3 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 267 B/s wr, 9 op/s 2024-03-08T21:08:33.010 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:08:32 smithi149 ceph-mon[35042]: pgmap v251: 97 pgs: 97 active+clean; 5.3 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:08:33.124 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:08:32 smithi204 ceph-mon[34935]: pgmap v251: 97 pgs: 97 active+clean; 5.3 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:08:33.213 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:08:32 smithi107 ceph-mon[31381]: pgmap v251: 97 pgs: 97 active+clean; 5.3 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:08:35.010 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:08:34 smithi149 ceph-mon[35042]: pgmap v252: 97 pgs: 97 active+clean; 5.3 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-08T21:08:35.124 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:08:34 smithi204 ceph-mon[34935]: pgmap v252: 97 pgs: 97 active+clean; 5.3 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-08T21:08:35.213 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:08:34 smithi107 ceph-mon[31381]: pgmap v252: 97 pgs: 97 active+clean; 5.3 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-08T21:08:36.605 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.625+0000 7f30ed200640 1 -- 172.21.15.107:0/488899603 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f30e80ffeb0 msgr2=0x7f30e8100290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:08:36.605 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.625+0000 7f30ed200640 1 --2- 172.21.15.107:0/488899603 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f30e80ffeb0 0x7f30e8100290 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f30d8009ca0 tx=0x7f30d802c0f0 comp rx=0 tx=0).stop 2024-03-08T21:08:36.605 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.625+0000 7f30ed200640 1 -- 172.21.15.107:0/488899603 shutdown_connections 2024-03-08T21:08:36.605 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.625+0000 7f30ed200640 1 --2- 172.21.15.107:0/488899603 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f30e810da60 0x7f30e810fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:08:36.605 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.625+0000 7f30ed200640 1 --2- 172.21.15.107:0/488899603 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f30e8100860 0x7f30e810d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:08:36.606 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.625+0000 7f30ed200640 1 --2- 172.21.15.107:0/488899603 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f30e80ffeb0 0x7f30e8100290 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:08:36.606 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.625+0000 7f30ed200640 1 -- 172.21.15.107:0/488899603 >> 172.21.15.107:0/488899603 conn(0x7f30e80fbb00 msgr2=0x7f30e80fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:08:36.606 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.625+0000 7f30ed200640 1 -- 172.21.15.107:0/488899603 shutdown_connections 2024-03-08T21:08:36.606 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.625+0000 7f30ed200640 1 -- 172.21.15.107:0/488899603 wait complete. 2024-03-08T21:08:36.608 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.626+0000 7f30ed200640 1 Processor -- start 2024-03-08T21:08:36.608 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.626+0000 7f30ed200640 1 -- start start 2024-03-08T21:08:36.608 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.627+0000 7f30ed200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f30e80ffeb0 0x7f30e819f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:08:36.609 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.627+0000 7f30ed200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f30e8100860 0x7f30e819fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:08:36.609 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.627+0000 7f30ed200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f30e810da60 0x7f30e8199950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:08:36.609 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.627+0000 7f30ed200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30e8113590 con 0x7f30e80ffeb0 2024-03-08T21:08:36.609 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.627+0000 7f30ed200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30e8113410 con 0x7f30e810da60 2024-03-08T21:08:36.609 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.627+0000 7f30ed200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30e8113710 con 0x7f30e8100860 2024-03-08T21:08:36.609 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.627+0000 7f30e7400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f30e80ffeb0 0x7f30e819f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:08:36.609 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.627+0000 7f30e7400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f30e80ffeb0 0x7f30e819f7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:38984/0 (socket says 172.21.15.107:38984) 2024-03-08T21:08:36.609 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.627+0000 7f30e7400640 1 -- 172.21.15.107:0/3199290249 learned_addr learned my addr 172.21.15.107:0/3199290249 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:08:36.609 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.627+0000 7f30e7e00640 1 --2- 172.21.15.107:0/3199290249 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f30e810da60 0x7f30e8199950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:08:36.609 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.627+0000 7f30dea00640 1 --2- 172.21.15.107:0/3199290249 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f30e8100860 0x7f30e819fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:08:36.609 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.627+0000 7f30e7400640 1 -- 172.21.15.107:0/3199290249 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f30e8100860 msgr2=0x7f30e819fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:08:36.610 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.627+0000 7f30e7400640 1 --2- 172.21.15.107:0/3199290249 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f30e8100860 0x7f30e819fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:08:36.610 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.627+0000 7f30e7400640 1 -- 172.21.15.107:0/3199290249 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f30e810da60 msgr2=0x7f30e8199950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:08:36.610 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.627+0000 7f30e7400640 1 --2- 172.21.15.107:0/3199290249 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f30e810da60 0x7f30e8199950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:08:36.610 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.627+0000 7f30e7400640 1 -- 172.21.15.107:0/3199290249 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30e819a0d0 con 0x7f30e80ffeb0 2024-03-08T21:08:36.610 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.628+0000 7f30e7400640 1 --2- 172.21.15.107:0/3199290249 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f30e80ffeb0 0x7f30e819f7a0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f30d8005e30 tx=0x7f30d8035d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:08:36.610 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.628+0000 7f30e4c00640 1 -- 172.21.15.107:0/3199290249 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30d8039040 con 0x7f30e80ffeb0 2024-03-08T21:08:36.610 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.628+0000 7f30e4c00640 1 -- 172.21.15.107:0/3199290249 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f30d8044070 con 0x7f30e80ffeb0 2024-03-08T21:08:36.610 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.628+0000 7f30e4c00640 1 -- 172.21.15.107:0/3199290249 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30d8004460 con 0x7f30e80ffeb0 2024-03-08T21:08:36.610 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.628+0000 7f30ed200640 1 -- 172.21.15.107:0/3199290249 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30e819a360 con 0x7f30e80ffeb0 2024-03-08T21:08:36.611 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.628+0000 7f30ed200640 1 -- 172.21.15.107:0/3199290249 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30e81a6660 con 0x7f30e80ffeb0 2024-03-08T21:08:36.614 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.632+0000 7f30e4c00640 1 -- 172.21.15.107:0/3199290249 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) v1 ==== 137422+0+0 (secure 0 0 0) 0x7f30d80326f0 con 0x7f30e80ffeb0 2024-03-08T21:08:36.614 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.632+0000 7f30f162a740 1 Processor -- start 2024-03-08T21:08:36.614 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.632+0000 7f30f162a740 1 -- start start 2024-03-08T21:08:36.614 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.632+0000 7f30f162a740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5576b1282e80 0x5576b1283260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:08:36.614 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.633+0000 7f30f162a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5576b12837a0 0x5576b128c030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:08:36.614 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.633+0000 7f30f162a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5576b128c570 0x5576b128e960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:08:36.614 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.633+0000 7f30f162a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576b11e63d0 con 0x5576b12837a0 2024-03-08T21:08:36.614 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.633+0000 7f30f162a740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576b118f6b0 con 0x5576b1282e80 2024-03-08T21:08:36.614 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.633+0000 7f30f162a740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576b11e2470 con 0x5576b128c570 2024-03-08T21:08:36.615 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.633+0000 7f30e7400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5576b12837a0 0x5576b128c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:08:36.615 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.633+0000 7f30e7400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5576b12837a0 0x5576b128c030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:38992/0 (socket says 172.21.15.107:38992) 2024-03-08T21:08:36.615 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.633+0000 7f30e7400640 1 -- 172.21.15.107:0/126735645 learned_addr learned my addr 172.21.15.107:0/126735645 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:08:36.615 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.633+0000 7f30e7e00640 1 --2- 172.21.15.107:0/126735645 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5576b1282e80 0x5576b1283260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:08:36.615 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.633+0000 7f30dea00640 1 --2- 172.21.15.107:0/126735645 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5576b128c570 0x5576b128e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:08:36.615 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.633+0000 7f30e7400640 1 -- 172.21.15.107:0/126735645 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5576b128c570 msgr2=0x5576b128e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:08:36.615 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.633+0000 7f30e7400640 1 --2- 172.21.15.107:0/126735645 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5576b128c570 0x5576b128e960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:08:36.615 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.633+0000 7f30e7400640 1 -- 172.21.15.107:0/126735645 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5576b1282e80 msgr2=0x5576b1283260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:08:36.615 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.633+0000 7f30e7400640 1 --2- 172.21.15.107:0/126735645 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5576b1282e80 0x5576b1283260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:08:36.615 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.634+0000 7f30e7400640 1 -- 172.21.15.107:0/126735645 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5576b11b5b50 con 0x5576b12837a0 2024-03-08T21:08:36.616 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.634+0000 7f30e4c00640 1 --2- 172.21.15.107:0/3199290249 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f30cc0a29e0 0x7f30cc0a4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:08:36.616 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.634+0000 7f30e4c00640 1 -- 172.21.15.107:0/3199290249 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f30d80f1410 con 0x7f30e80ffeb0 2024-03-08T21:08:36.616 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.634+0000 7f30e7e00640 1 --2- 172.21.15.107:0/3199290249 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f30cc0a29e0 0x7f30cc0a4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:08:36.616 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.634+0000 7f30e7400640 1 --2- 172.21.15.107:0/126735645 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5576b12837a0 0x5576b128c030 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f30d80eebd0 tx=0x7f30d80f7550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21db31a473c3eb6e server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:08:36.616 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.634+0000 7f30dcc00640 1 -- 172.21.15.107:0/126735645 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30d80b9de0 con 0x5576b12837a0 2024-03-08T21:08:36.616 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.634+0000 7f30dcc00640 1 -- 172.21.15.107:0/126735645 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f30d80098e0 con 0x5576b12837a0 2024-03-08T21:08:36.616 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.634+0000 7f30dcc00640 1 -- 172.21.15.107:0/126735645 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30d80bd4a0 con 0x5576b12837a0 2024-03-08T21:08:36.621 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.635+0000 7f30f162a740 1 -- 172.21.15.107:0/126735645 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5576b12837a0 msgr2=0x5576b128c030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:08:36.621 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.635+0000 7f30e7e00640 1 --2- 172.21.15.107:0/3199290249 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f30cc0a29e0 0x7f30cc0a4ea0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f30e0005fd0 tx=0x7f30e000b530 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:08:36.621 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.635+0000 7f30f162a740 1 --2- 172.21.15.107:0/126735645 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5576b12837a0 0x5576b128c030 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f30d80eebd0 tx=0x7f30d80f7550 comp rx=0 tx=0).stop 2024-03-08T21:08:36.621 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.635+0000 7f30f162a740 1 -- 172.21.15.107:0/126735645 shutdown_connections 2024-03-08T21:08:36.621 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.635+0000 7f30f162a740 1 --2- 172.21.15.107:0/126735645 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5576b128c570 0x5576b128e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:08:36.621 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.635+0000 7f30f162a740 1 --2- 172.21.15.107:0/126735645 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5576b12837a0 0x5576b128c030 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:08:36.621 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.635+0000 7f30f162a740 1 --2- 172.21.15.107:0/126735645 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5576b1282e80 0x5576b1283260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:08:36.621 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.635+0000 7f30f162a740 1 -- 172.21.15.107:0/126735645 >> 172.21.15.107:0/126735645 conn(0x5576b127d7a0 msgr2=0x5576b127db80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:08:36.621 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.636+0000 7f30f162a740 1 -- 172.21.15.107:0/126735645 shutdown_connections 2024-03-08T21:08:36.621 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.636+0000 7f30f162a740 1 -- 172.21.15.107:0/126735645 wait complete. 2024-03-08T21:08:36.622 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.636+0000 7f30f162a740 1 -- start start 2024-03-08T21:08:36.622 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.636+0000 7f30f162a740 1 Processor -- start 2024-03-08T21:08:36.622 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.637+0000 7f30f162a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5576b1282e80 0x5576b128bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:08:36.622 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.637+0000 7f30f162a740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5576b12837a0 0x5576b1289270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:08:36.622 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.637+0000 7f30f162a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5576b128c570 0x5576b12897d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:08:36.622 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.637+0000 7f30f162a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576b128ef60 con 0x5576b1282e80 2024-03-08T21:08:36.622 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.637+0000 7f30f162a740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576b11a29f0 con 0x5576b12837a0 2024-03-08T21:08:36.622 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.637+0000 7f30f162a740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576b128f0e0 con 0x5576b128c570 2024-03-08T21:08:36.622 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.637+0000 7f30f162a740 10 client.0 _ll_register_callbacks cb 0x5576b0efaa00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T21:08:36.622 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.637+0000 7f30e7400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5576b1282e80 0x5576b128bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:08:36.622 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.637+0000 7f30e7400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5576b1282e80 0x5576b128bee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:39002/0 (socket says 172.21.15.107:39002) 2024-03-08T21:08:36.622 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.637+0000 7f30e7400640 1 -- 172.21.15.107:0/2878032720 learned_addr learned my addr 172.21.15.107:0/2878032720 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:08:36.622 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.637+0000 7f30dea00640 1 --2- 172.21.15.107:0/2878032720 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5576b12837a0 0x5576b1289270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:08:36.623 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.637+0000 7f30e7e00640 1 --2- 172.21.15.107:0/2878032720 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5576b128c570 0x5576b12897d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:08:36.623 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.637+0000 7f30e7400640 1 -- 172.21.15.107:0/2878032720 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5576b128c570 msgr2=0x5576b12897d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:08:36.623 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.637+0000 7f30e7400640 1 --2- 172.21.15.107:0/2878032720 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5576b128c570 0x5576b12897d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:08:36.623 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.638+0000 7f30e7400640 1 -- 172.21.15.107:0/2878032720 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5576b12837a0 msgr2=0x5576b1289270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:08:36.623 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.638+0000 7f30e7400640 1 --2- 172.21.15.107:0/2878032720 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5576b12837a0 0x5576b1289270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:08:36.623 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.638+0000 7f30e7400640 1 -- 172.21.15.107:0/2878032720 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5576b0efb210 con 0x5576b1282e80 2024-03-08T21:08:36.623 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.638+0000 7f30e7e00640 1 --2- 172.21.15.107:0/2878032720 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5576b128c570 0x5576b12897d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T21:08:36.623 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.638+0000 7f30e7400640 1 --2- 172.21.15.107:0/2878032720 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5576b1282e80 0x5576b128bee0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f30d80eece0 tx=0x7f30d8000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18cd0de819641827 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:08:36.623 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.638+0000 7f30d5600640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-03-08T21:08:36.623 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.638+0000 7f30d5600640 1 -- 172.21.15.107:0/2878032720 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30d80bd680 con 0x5576b1282e80 2024-03-08T21:08:36.623 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.638+0000 7f30d5600640 1 -- 172.21.15.107:0/2878032720 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f30d80333a0 con 0x5576b1282e80 2024-03-08T21:08:36.623 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.638+0000 7f30f162a740 1 -- 172.21.15.107:0/2878032720 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f30d80098e0 con 0x5576b1282e80 2024-03-08T21:08:36.623 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.638+0000 7f30d5600640 1 -- 172.21.15.107:0/2878032720 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f30d80bd910 con 0x5576b1282e80 2024-03-08T21:08:36.623 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.638+0000 7f30f162a740 10 client.16131 fetch_fsmap learned FSMap version 36 2024-03-08T21:08:36.624 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.638+0000 7f30f162a740 1 -- 172.21.15.107:0/2878032720 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x5576b0fc7620 con 0x5576b1282e80 2024-03-08T21:08:36.624 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.639+0000 7f30d5600640 1 -- 172.21.15.107:0/2878032720 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f30d80098e0 con 0x5576b1282e80 2024-03-08T21:08:36.624 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.639+0000 7f30d5600640 1 -- 172.21.15.107:0/2878032720 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 36) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f30d803c030 con 0x5576b1282e80 2024-03-08T21:08:36.624 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.639+0000 7f30f162a740 10 client.16131 fetch_fsmap finished waiting for FSMap version 36 2024-03-08T21:08:36.624 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.639+0000 7f30f162a740 10 client.16131 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-08T21:08:36.624 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.639+0000 7f30f162a740 20 client.16131 populate_metadata read hostname 'smithi107' 2024-03-08T21:08:36.624 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.639+0000 7f30f162a740 1 --2- 172.21.15.107:0/2878032720 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x5576b129bfc0 0x5576b129e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:08:36.624 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.639+0000 7f30f162a740 4 client.16131 mds_command: new command op to 34475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-08T21:08:36.624 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.639+0000 7f30f162a740 1 -- 172.21.15.107:0/2878032720 --> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5576b11f3910 con 0x5576b129bfc0 2024-03-08T21:08:36.624 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.640+0000 7f30dea00640 1 --2- 172.21.15.107:0/2878032720 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x5576b129bfc0 0x5576b129e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:08:36.624 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.641+0000 7f30dea00640 1 --2- 172.21.15.107:0/2878032720 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x5576b129bfc0 0x5576b129e3b0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=89cb219c46ece3fa server_cookie=7f0465367711efc9 in_seq=0 out_seq=0 2024-03-08T21:08:36.624 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.641+0000 7f30d5600640 10 client.16131 ms_handle_connect on v2:172.21.15.149:6838/55290596 2024-03-08T21:08:36.626 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.642+0000 7f30d5600640 1 -- 172.21.15.107:0/2878032720 <== mds.0 v2:172.21.15.149:6838/55290596 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5576b11f3910 con 0x5576b129bfc0 2024-03-08T21:08:36.626 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.642+0000 7f30d5600640 10 client.16131 handle_command_reply: tid=0 2024-03-08T21:08:36.626 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.642+0000 7f30f162a740 1 client.16131 shutdown 2024-03-08T21:08:36.626 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.643+0000 7f30f162a740 1 -- 172.21.15.107:0/2878032720 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5576b1282e80 msgr2=0x5576b128bee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:08:36.626 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.643+0000 7f30f162a740 1 --2- 172.21.15.107:0/2878032720 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5576b1282e80 0x5576b128bee0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f30d80eece0 tx=0x7f30d8000f40 comp rx=0 tx=0).stop 2024-03-08T21:08:36.626 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.643+0000 7f30f162a740 1 -- 172.21.15.107:0/2878032720 shutdown_connections 2024-03-08T21:08:36.626 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.643+0000 7f30f162a740 1 --2- 172.21.15.107:0/2878032720 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5576b128c570 0x5576b12897d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:08:36.626 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.643+0000 7f30f162a740 1 --2- 172.21.15.107:0/2878032720 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x5576b129bfc0 0x5576b129e3b0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:08:36.626 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.643+0000 7f30f162a740 1 --2- 172.21.15.107:0/2878032720 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5576b12837a0 0x5576b1289270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:08:36.626 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.643+0000 7f30f162a740 1 --2- 172.21.15.107:0/2878032720 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5576b1282e80 0x5576b128bee0 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:08:36.627 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.643+0000 7f30f162a740 1 -- 172.21.15.107:0/2878032720 >> 172.21.15.107:0/2878032720 conn(0x5576b127d7a0 msgr2=0x5576b127f4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:08:36.627 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.643+0000 7f30d5600640 0 client.16131 ms_handle_reset on v2:172.21.15.149:6838/55290596 2024-03-08T21:08:36.627 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.643+0000 7f30f162a740 1 -- 172.21.15.107:0/2878032720 shutdown_connections 2024-03-08T21:08:36.627 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.643+0000 7f30f162a740 1 -- 172.21.15.107:0/2878032720 wait complete. 2024-03-08T21:08:36.627 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.643+0000 7f30f162a740 20 client.16131 trim_cache size 0 max 16384 2024-03-08T21:08:36.638 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.658+0000 7f30f162a740 1 Processor -- start 2024-03-08T21:08:36.638 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.658+0000 7f30f162a740 1 -- start start 2024-03-08T21:08:36.638 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.658+0000 7f30f162a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5576b1282e80 0x5576b1289210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:08:36.638 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.658+0000 7f30f162a740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5576b12837a0 0x5576b128b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:08:36.638 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.658+0000 7f30f162a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5576b128c570 0x5576b128bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:08:36.638 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.658+0000 7f30f162a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30e81aede0 con 0x5576b128c570 2024-03-08T21:08:36.638 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.658+0000 7f30f162a740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30e81b0760 con 0x5576b12837a0 2024-03-08T21:08:36.638 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.658+0000 7f30f162a740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30e81af440 con 0x5576b1282e80 2024-03-08T21:08:36.639 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.658+0000 7f30e7e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5576b128c570 0x5576b128bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:08:36.639 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.658+0000 7f30e7e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5576b128c570 0x5576b128bee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:39010/0 (socket says 172.21.15.107:39010) 2024-03-08T21:08:36.639 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.658+0000 7f30e7e00640 1 -- 172.21.15.107:0/846560819 learned_addr learned my addr 172.21.15.107:0/846560819 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:08:36.639 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.659+0000 7f30e7400640 1 --2- 172.21.15.107:0/846560819 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5576b1282e80 0x5576b1289210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:08:36.639 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.659+0000 7f30dea00640 1 --2- 172.21.15.107:0/846560819 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5576b12837a0 0x5576b128b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:08:36.641 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.659+0000 7f30e7e00640 1 -- 172.21.15.107:0/846560819 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5576b1282e80 msgr2=0x5576b1289210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:08:36.641 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.659+0000 7f30e7e00640 1 --2- 172.21.15.107:0/846560819 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5576b1282e80 0x5576b1289210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:08:36.641 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.659+0000 7f30e7e00640 1 -- 172.21.15.107:0/846560819 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5576b12837a0 msgr2=0x5576b128b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:08:36.641 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.659+0000 7f30e7e00640 1 --2- 172.21.15.107:0/846560819 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5576b12837a0 0x5576b128b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:08:36.641 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.659+0000 7f30e7e00640 1 -- 172.21.15.107:0/846560819 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5576b128e960 con 0x5576b128c570 2024-03-08T21:08:36.641 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.659+0000 7f30dea00640 1 --2- 172.21.15.107:0/846560819 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5576b12837a0 0x5576b128b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T21:08:36.642 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.659+0000 7f30e7e00640 1 --2- 172.21.15.107:0/846560819 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5576b128c570 0x5576b128bee0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f30e0011fb0 tx=0x7f30e0014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b7a8b3706aab6eb server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:08:36.642 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.660+0000 7f30d7e00640 1 -- 172.21.15.107:0/846560819 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30e0028670 con 0x5576b128c570 2024-03-08T21:08:36.642 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.660+0000 7f30d7e00640 1 -- 172.21.15.107:0/846560819 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f30e0023070 con 0x5576b128c570 2024-03-08T21:08:36.642 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.660+0000 7f30d7e00640 1 -- 172.21.15.107:0/846560819 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30e001fc00 con 0x5576b128c570 2024-03-08T21:08:36.642 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.661+0000 7f30f162a740 1 -- 172.21.15.107:0/846560819 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5576b128c570 msgr2=0x5576b128bee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:08:36.642 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.661+0000 7f30f162a740 1 --2- 172.21.15.107:0/846560819 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5576b128c570 0x5576b128bee0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f30e0011fb0 tx=0x7f30e0014000 comp rx=0 tx=0).stop 2024-03-08T21:08:36.642 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.661+0000 7f30f162a740 1 -- 172.21.15.107:0/846560819 shutdown_connections 2024-03-08T21:08:36.642 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.661+0000 7f30f162a740 1 --2- 172.21.15.107:0/846560819 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5576b128c570 0x5576b128bee0 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:08:36.642 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.661+0000 7f30f162a740 1 --2- 172.21.15.107:0/846560819 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5576b12837a0 0x5576b128b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:08:36.642 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.661+0000 7f30f162a740 1 --2- 172.21.15.107:0/846560819 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5576b1282e80 0x5576b1289210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:08:36.643 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.661+0000 7f30f162a740 1 -- 172.21.15.107:0/846560819 >> 172.21.15.107:0/846560819 conn(0x5576b127d7a0 msgr2=0x5576b1288b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:08:36.643 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.661+0000 7f30f162a740 1 -- 172.21.15.107:0/846560819 shutdown_connections 2024-03-08T21:08:36.643 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.661+0000 7f30f162a740 1 -- 172.21.15.107:0/846560819 wait complete. 2024-03-08T21:08:36.643 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.661+0000 7f30f162a740 1 -- start start 2024-03-08T21:08:36.643 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.662+0000 7f30f162a740 1 Processor -- start 2024-03-08T21:08:36.646 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.662+0000 7f30f162a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5576b1282e80 0x5576b1291d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:08:36.646 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.662+0000 7f30f162a740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5576b12837a0 0x5576b12922d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:08:36.646 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.662+0000 7f30f162a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5576b128c570 0x5576b1292810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:08:36.647 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.662+0000 7f30f162a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30e81b0100 con 0x5576b128c570 2024-03-08T21:08:36.647 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.662+0000 7f30f162a740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30e81afaa0 con 0x5576b12837a0 2024-03-08T21:08:36.647 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.662+0000 7f30f162a740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30e81ad0a0 con 0x5576b1282e80 2024-03-08T21:08:36.647 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.662+0000 7f30f162a740 10 client.0 _ll_register_callbacks cb 0x7f30d80f4e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T21:08:36.647 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.663+0000 7f30e7e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5576b128c570 0x5576b1292810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:08:36.647 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.663+0000 7f30e7e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5576b128c570 0x5576b1292810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:39018/0 (socket says 172.21.15.107:39018) 2024-03-08T21:08:36.647 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.663+0000 7f30e7e00640 1 -- 172.21.15.107:0/137667402 learned_addr learned my addr 172.21.15.107:0/137667402 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:08:36.647 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.663+0000 7f30e7400640 1 --2- 172.21.15.107:0/137667402 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5576b1282e80 0x5576b1291d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:08:36.647 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.663+0000 7f30dea00640 1 --2- 172.21.15.107:0/137667402 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5576b12837a0 0x5576b12922d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:08:36.647 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.663+0000 7f30e7e00640 1 -- 172.21.15.107:0/137667402 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5576b1282e80 msgr2=0x5576b1291d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:08:36.648 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.663+0000 7f30e7e00640 1 --2- 172.21.15.107:0/137667402 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5576b1282e80 0x5576b1291d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:08:36.648 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.663+0000 7f30e7e00640 1 -- 172.21.15.107:0/137667402 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5576b12837a0 msgr2=0x5576b12922d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:08:36.648 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.663+0000 7f30e7e00640 1 --2- 172.21.15.107:0/137667402 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5576b12837a0 0x5576b12922d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:08:36.648 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.663+0000 7f30e7e00640 1 -- 172.21.15.107:0/137667402 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5576b12a33f0 con 0x5576b128c570 2024-03-08T21:08:36.648 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.663+0000 7f30e7e00640 1 --2- 172.21.15.107:0/137667402 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5576b128c570 0x5576b1292810 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f30e001f820 tx=0x7f30e00056d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a22827b8ebf1985a server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:08:36.648 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.663+0000 7f30d5600640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-03-08T21:08:36.648 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.664+0000 7f30d5600640 1 -- 172.21.15.107:0/137667402 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30e001fb90 con 0x5576b128c570 2024-03-08T21:08:36.648 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.664+0000 7f30d5600640 1 -- 172.21.15.107:0/137667402 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f30e001fd30 con 0x5576b128c570 2024-03-08T21:08:36.648 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.664+0000 7f30f162a740 1 -- 172.21.15.107:0/137667402 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f30e0023070 con 0x5576b128c570 2024-03-08T21:08:36.648 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.664+0000 7f30d5600640 1 -- 172.21.15.107:0/137667402 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f30e0028470 con 0x5576b128c570 2024-03-08T21:08:36.649 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.664+0000 7f30f162a740 10 client.16137 fetch_fsmap learned FSMap version 36 2024-03-08T21:08:36.649 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.664+0000 7f30f162a740 1 -- 172.21.15.107:0/137667402 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x5576b12f35d0 con 0x5576b128c570 2024-03-08T21:08:36.649 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.664+0000 7f30d5600640 1 -- 172.21.15.107:0/137667402 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f30e0023070 con 0x5576b128c570 2024-03-08T21:08:36.649 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.664+0000 7f30d5600640 1 -- 172.21.15.107:0/137667402 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 36) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f30e0038030 con 0x5576b128c570 2024-03-08T21:08:36.649 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.665+0000 7f30f162a740 10 client.16137 fetch_fsmap finished waiting for FSMap version 36 2024-03-08T21:08:36.649 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.665+0000 7f30f162a740 10 client.16137 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-08T21:08:36.649 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.665+0000 7f30f162a740 20 client.16137 populate_metadata read hostname 'smithi107' 2024-03-08T21:08:36.649 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.665+0000 7f30f162a740 1 --2- 172.21.15.107:0/137667402 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x5576b12f3c60 0x5576b12f6080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:08:36.649 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.665+0000 7f30f162a740 4 client.16137 mds_command: new command op to 34475 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-08T21:08:36.649 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.665+0000 7f30f162a740 1 -- 172.21.15.107:0/137667402 --> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5576b11edc60 con 0x5576b12f3c60 2024-03-08T21:08:36.649 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.665+0000 7f30e7400640 1 --2- 172.21.15.107:0/137667402 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x5576b12f3c60 0x5576b12f6080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:08:36.650 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.666+0000 7f30e7400640 1 --2- 172.21.15.107:0/137667402 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x5576b12f3c60 0x5576b12f6080 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=246fe381c9c3b09b server_cookie=7a74a906983936e1 in_seq=0 out_seq=0 2024-03-08T21:08:36.650 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.666+0000 7f30d5600640 10 client.16137 ms_handle_connect on v2:172.21.15.149:6838/55290596 2024-03-08T21:08:36.655 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.675+0000 7f30d5600640 1 -- 172.21.15.107:0/137667402 <== mds.0 v2:172.21.15.149:6838/55290596 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5576b11edc60 con 0x5576b12f3c60 2024-03-08T21:08:36.655 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.675+0000 7f30d5600640 10 client.16137 handle_command_reply: tid=0 2024-03-08T21:08:36.655 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.675+0000 7f30f162a740 1 client.16137 shutdown 2024-03-08T21:08:36.655 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.675+0000 7f30f162a740 1 -- 172.21.15.107:0/137667402 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5576b128c570 msgr2=0x5576b1292810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:08:36.655 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.675+0000 7f30f162a740 1 --2- 172.21.15.107:0/137667402 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5576b128c570 0x5576b1292810 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f30e001f820 tx=0x7f30e00056d0 comp rx=0 tx=0).stop 2024-03-08T21:08:36.655 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.675+0000 7f30f162a740 1 -- 172.21.15.107:0/137667402 shutdown_connections 2024-03-08T21:08:36.655 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.675+0000 7f30f162a740 1 --2- 172.21.15.107:0/137667402 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5576b128c570 0x5576b1292810 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:08:36.656 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.676+0000 7f30f162a740 1 --2- 172.21.15.107:0/137667402 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x5576b12f3c60 0x5576b12f6080 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:08:36.656 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.676+0000 7f30f162a740 1 --2- 172.21.15.107:0/137667402 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5576b12837a0 0x5576b12922d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:08:36.656 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.676+0000 7f30f162a740 1 --2- 172.21.15.107:0/137667402 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5576b1282e80 0x5576b1291d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:08:36.656 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.676+0000 7f30f162a740 1 -- 172.21.15.107:0/137667402 >> 172.21.15.107:0/137667402 conn(0x5576b127d7a0 msgr2=0x5576b127ebf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:08:36.656 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.676+0000 7f30d5600640 0 client.16137 ms_handle_reset on v2:172.21.15.149:6838/55290596 2024-03-08T21:08:36.656 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.676+0000 7f30f162a740 1 -- 172.21.15.107:0/137667402 shutdown_connections 2024-03-08T21:08:36.656 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.676+0000 7f30f162a740 1 -- 172.21.15.107:0/137667402 wait complete. 2024-03-08T21:08:36.656 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.676+0000 7f30f162a740 20 client.16137 trim_cache size 0 max 16384 2024-03-08T21:08:36.657 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:{ 2024-03-08T21:08:36.657 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "message": "", 2024-03-08T21:08:36.657 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "return_code": 0 2024-03-08T21:08:36.657 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:} 2024-03-08T21:08:36.657 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.677+0000 7f30ed200640 1 -- 172.21.15.107:0/3199290249 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f30cc0a29e0 msgr2=0x7f30cc0a4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:08:36.657 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.677+0000 7f30ed200640 1 --2- 172.21.15.107:0/3199290249 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f30cc0a29e0 0x7f30cc0a4ea0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f30e0005fd0 tx=0x7f30e000b530 comp rx=0 tx=0).stop 2024-03-08T21:08:36.658 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.677+0000 7f30ed200640 1 -- 172.21.15.107:0/3199290249 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f30e80ffeb0 msgr2=0x7f30e819f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:08:36.658 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.677+0000 7f30ed200640 1 --2- 172.21.15.107:0/3199290249 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f30e80ffeb0 0x7f30e819f7a0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f30d8005e30 tx=0x7f30d8035d60 comp rx=0 tx=0).stop 2024-03-08T21:08:36.658 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.677+0000 7f30ed200640 1 -- 172.21.15.107:0/3199290249 shutdown_connections 2024-03-08T21:08:36.658 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.677+0000 7f30ed200640 1 --2- 172.21.15.107:0/3199290249 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f30cc0a29e0 0x7f30cc0a4ea0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:08:36.658 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.677+0000 7f30ed200640 1 --2- 172.21.15.107:0/3199290249 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f30e810da60 0x7f30e8199950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:08:36.658 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.677+0000 7f30ed200640 1 --2- 172.21.15.107:0/3199290249 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f30e8100860 0x7f30e819fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:08:36.658 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.677+0000 7f30ed200640 1 --2- 172.21.15.107:0/3199290249 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f30e80ffeb0 0x7f30e819f7a0 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:08:36.658 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.677+0000 7f30ed200640 1 -- 172.21.15.107:0/3199290249 >> 172.21.15.107:0/3199290249 conn(0x7f30e80fbb00 msgr2=0x7f30e80fdef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:08:36.658 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.677+0000 7f30ed200640 1 -- 172.21.15.107:0/3199290249 shutdown_connections 2024-03-08T21:08:36.658 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:36.677+0000 7f30ed200640 1 -- 172.21.15.107:0/3199290249 wait complete. 2024-03-08T21:08:37.010 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:08:36 smithi149 ceph-mon[35042]: pgmap v253: 97 pgs: 97 active+clean; 5.3 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-08T21:08:37.125 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:08:36 smithi204 ceph-mon[34935]: pgmap v253: 97 pgs: 97 active+clean; 5.3 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-08T21:08:37.213 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:08:36 smithi107 ceph-mon[31381]: pgmap v253: 97 pgs: 97 active+clean; 5.3 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-08T21:08:39.010 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:08:38 smithi149 ceph-mon[35042]: pgmap v254: 97 pgs: 97 active+clean; 5.3 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-08T21:08:39.124 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:08:38 smithi204 ceph-mon[34935]: pgmap v254: 97 pgs: 97 active+clean; 5.3 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-08T21:08:39.213 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:08:38 smithi107 ceph-mon[31381]: pgmap v254: 97 pgs: 97 active+clean; 5.3 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-08T21:08:41.010 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:08:40 smithi149 ceph-mon[35042]: pgmap v255: 97 pgs: 97 active+clean; 5.3 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:08:41.124 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:08:40 smithi204 ceph-mon[34935]: pgmap v255: 97 pgs: 97 active+clean; 5.3 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:08:41.213 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:08:40 smithi107 ceph-mon[31381]: pgmap v255: 97 pgs: 97 active+clean; 5.3 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:08:43.011 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:08:42 smithi149 ceph-mon[35042]: pgmap v256: 97 pgs: 97 active+clean; 5.3 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:08:43.124 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:08:42 smithi204 ceph-mon[34935]: pgmap v256: 97 pgs: 97 active+clean; 5.3 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:08:43.213 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:08:42 smithi107 ceph-mon[31381]: pgmap v256: 97 pgs: 97 active+clean; 5.3 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:08:45.010 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:08:44 smithi149 ceph-mon[35042]: pgmap v257: 97 pgs: 97 active+clean; 5.3 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-08T21:08:45.124 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:08:44 smithi204 ceph-mon[34935]: pgmap v257: 97 pgs: 97 active+clean; 5.3 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-08T21:08:45.213 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:08:44 smithi107 ceph-mon[31381]: pgmap v257: 97 pgs: 97 active+clean; 5.3 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-08T21:08:47.011 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:08:46 smithi149 ceph-mon[35042]: pgmap v258: 97 pgs: 97 active+clean; 5.3 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:08:47.015 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:08:46 smithi107 ceph-mon[31381]: pgmap v258: 97 pgs: 97 active+clean; 5.3 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:08:47.124 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:08:46 smithi204 ceph-mon[34935]: pgmap v258: 97 pgs: 97 active+clean; 5.3 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:08:49.010 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:08:48 smithi149 ceph-mon[35042]: pgmap v259: 97 pgs: 97 active+clean; 5.7 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 35 KiB/s wr, 12 op/s 2024-03-08T21:08:49.124 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:08:48 smithi204 ceph-mon[34935]: pgmap v259: 97 pgs: 97 active+clean; 5.7 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 35 KiB/s wr, 12 op/s 2024-03-08T21:08:49.213 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:08:48 smithi107 ceph-mon[31381]: pgmap v259: 97 pgs: 97 active+clean; 5.7 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 35 KiB/s wr, 12 op/s 2024-03-08T21:08:49.812 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.833+0000 7f175b200640 1 -- 172.21.15.107:0/524751931 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f175c1022a0 msgr2=0x7f175c102680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:08:49.812 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.833+0000 7f175b200640 1 --2- 172.21.15.107:0/524751931 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f175c1022a0 0x7f175c102680 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f174c009900 tx=0x7f174c02bcd0 comp rx=0 tx=0).stop 2024-03-08T21:08:49.812 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.833+0000 7f175b200640 1 -- 172.21.15.107:0/524751931 shutdown_connections 2024-03-08T21:08:49.812 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.833+0000 7f175b200640 1 --2- 172.21.15.107:0/524751931 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f175c10b6c0 0x7f175c10db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:08:49.812 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.833+0000 7f175b200640 1 --2- 172.21.15.107:0/524751931 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f175c102bc0 0x7f175c10b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:08:49.812 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.833+0000 7f175b200640 1 --2- 172.21.15.107:0/524751931 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f175c1022a0 0x7f175c102680 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:08:49.813 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.833+0000 7f175b200640 1 -- 172.21.15.107:0/524751931 >> 172.21.15.107:0/524751931 conn(0x7f175c0fbb00 msgr2=0x7f175c0fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:08:49.813 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.833+0000 7f175b200640 1 -- 172.21.15.107:0/524751931 shutdown_connections 2024-03-08T21:08:49.813 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.833+0000 7f175b200640 1 -- 172.21.15.107:0/524751931 wait complete. 2024-03-08T21:08:49.813 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.835+0000 7f175b200640 1 Processor -- start 2024-03-08T21:08:49.813 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.835+0000 7f175b200640 1 -- start start 2024-03-08T21:08:49.815 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.836+0000 7f175b200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f175c1022a0 0x7f175c19f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:08:49.815 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.836+0000 7f175b200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f175c102bc0 0x7f175c19fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:08:49.815 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.836+0000 7f175b200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f175c10b6c0 0x7f175c199990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:08:49.815 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.836+0000 7f175b200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f175c111350 con 0x7f175c10b6c0 2024-03-08T21:08:49.815 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.836+0000 7f175b200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f175c1111d0 con 0x7f175c102bc0 2024-03-08T21:08:49.815 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.836+0000 7f175b200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f175c1114d0 con 0x7f175c1022a0 2024-03-08T21:08:49.815 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.836+0000 7f175a800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f175c10b6c0 0x7f175c199990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:08:49.815 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.836+0000 7f175a800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f175c10b6c0 0x7f175c199990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:33466/0 (socket says 172.21.15.107:33466) 2024-03-08T21:08:49.815 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.836+0000 7f175a800640 1 -- 172.21.15.107:0/3498046590 learned_addr learned my addr 172.21.15.107:0/3498046590 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:08:49.815 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.836+0000 7f1759e00640 1 --2- 172.21.15.107:0/3498046590 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f175c1022a0 0x7f175c19f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:08:49.816 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.837+0000 7f1759400640 1 --2- 172.21.15.107:0/3498046590 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f175c102bc0 0x7f175c19fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:08:49.817 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.837+0000 7f175a800640 1 -- 172.21.15.107:0/3498046590 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f175c1022a0 msgr2=0x7f175c19f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:08:49.817 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.837+0000 7f175a800640 1 --2- 172.21.15.107:0/3498046590 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f175c1022a0 0x7f175c19f7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:08:49.817 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.837+0000 7f175a800640 1 -- 172.21.15.107:0/3498046590 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f175c102bc0 msgr2=0x7f175c19fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:08:49.817 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.837+0000 7f175a800640 1 --2- 172.21.15.107:0/3498046590 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f175c102bc0 0x7f175c19fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:08:49.817 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.837+0000 7f175a800640 1 -- 172.21.15.107:0/3498046590 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f175c19a110 con 0x7f175c10b6c0 2024-03-08T21:08:49.817 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.837+0000 7f175a800640 1 --2- 172.21.15.107:0/3498046590 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f175c10b6c0 0x7f175c199990 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f175400eaa0 tx=0x7f175400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:08:49.817 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.838+0000 7f1752a00640 1 -- 172.21.15.107:0/3498046590 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f175400cdb0 con 0x7f175c10b6c0 2024-03-08T21:08:49.817 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.838+0000 7f1752a00640 1 -- 172.21.15.107:0/3498046590 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f1754005020 con 0x7f175c10b6c0 2024-03-08T21:08:49.817 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.838+0000 7f175b200640 1 -- 172.21.15.107:0/3498046590 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f175c19a400 con 0x7f175c10b6c0 2024-03-08T21:08:49.817 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.838+0000 7f1752a00640 1 -- 172.21.15.107:0/3498046590 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17540116d0 con 0x7f175c10b6c0 2024-03-08T21:08:49.818 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.838+0000 7f175b200640 1 -- 172.21.15.107:0/3498046590 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f175c1a66a0 con 0x7f175c10b6c0 2024-03-08T21:08:49.820 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.842+0000 7f1763be6740 1 Processor -- start 2024-03-08T21:08:49.821 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.843+0000 7f1752a00640 1 -- 172.21.15.107:0/3498046590 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) v1 ==== 137422+0+0 (secure 0 0 0) 0x7f17540051d0 con 0x7f175c10b6c0 2024-03-08T21:08:49.821 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.843+0000 7f1763be6740 1 -- start start 2024-03-08T21:08:49.823 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.844+0000 7f1763be6740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55656d5bbe80 0x55656d5bc260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:08:49.823 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.844+0000 7f1763be6740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55656d5bc7a0 0x55656d5c5030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:08:49.823 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.844+0000 7f1763be6740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55656d5c5570 0x55656d5c7960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:08:49.824 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.844+0000 7f1763be6740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55656d51f3d0 con 0x55656d5bc7a0 2024-03-08T21:08:49.824 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.844+0000 7f1763be6740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55656d4c86b0 con 0x55656d5bbe80 2024-03-08T21:08:49.824 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.844+0000 7f1763be6740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55656d51b470 con 0x55656d5c5570 2024-03-08T21:08:49.824 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.844+0000 7f1759e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55656d5bc7a0 0x55656d5c5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:08:49.824 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.844+0000 7f1759e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55656d5bc7a0 0x55656d5c5030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:33478/0 (socket says 172.21.15.107:33478) 2024-03-08T21:08:49.824 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.844+0000 7f1759e00640 1 -- 172.21.15.107:0/335097234 learned_addr learned my addr 172.21.15.107:0/335097234 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:08:49.824 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.844+0000 7f1759400640 1 --2- 172.21.15.107:0/335097234 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55656d5bbe80 0x55656d5bc260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:08:49.824 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.845+0000 7f1759400640 1 --2- 172.21.15.107:0/335097234 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55656d5c5570 0x55656d5c7960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:08:49.824 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.845+0000 7f1759e00640 1 -- 172.21.15.107:0/335097234 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55656d5c5570 msgr2=0x55656d5c7960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:08:49.824 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.845+0000 7f1759e00640 1 --2- 172.21.15.107:0/335097234 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55656d5c5570 0x55656d5c7960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:08:49.824 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.845+0000 7f1759e00640 1 -- 172.21.15.107:0/335097234 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55656d5bbe80 msgr2=0x55656d5bc260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:08:49.824 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.845+0000 7f1759e00640 1 --2- 172.21.15.107:0/335097234 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55656d5bbe80 0x55656d5bc260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:08:49.825 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.845+0000 7f1759e00640 1 -- 172.21.15.107:0/335097234 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55656d4eeb50 con 0x55656d5bc7a0 2024-03-08T21:08:49.825 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.845+0000 7f1759400640 1 --2- 172.21.15.107:0/335097234 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55656d5bbe80 0x55656d5bc260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T21:08:49.825 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.845+0000 7f1752a00640 1 --2- 172.21.15.107:0/3498046590 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f17400a2960 0x7f17400a4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:08:49.825 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.845+0000 7f1759e00640 1 --2- 172.21.15.107:0/335097234 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55656d5bc7a0 0x55656d5c5030 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f174c009900 tx=0x7f174c02c6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb0c48be5aca5637 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:08:49.825 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.845+0000 7f1752a00640 1 -- 172.21.15.107:0/3498046590 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f17540cefd0 con 0x7f175c10b6c0 2024-03-08T21:08:49.829 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.845+0000 7f1759400640 1 --2- 172.21.15.107:0/3498046590 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f17400a2960 0x7f17400a4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:08:49.829 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.846+0000 7f1747400640 1 -- 172.21.15.107:0/335097234 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f174c0040c0 con 0x55656d5bc7a0 2024-03-08T21:08:49.829 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.846+0000 7f1747400640 1 -- 172.21.15.107:0/335097234 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f174c0054f0 con 0x55656d5bc7a0 2024-03-08T21:08:49.829 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.846+0000 7f1747400640 1 -- 172.21.15.107:0/335097234 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f174c0046c0 con 0x55656d5bc7a0 2024-03-08T21:08:49.829 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.847+0000 7f1763be6740 1 -- 172.21.15.107:0/335097234 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55656d5bc7a0 msgr2=0x55656d5c5030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:08:49.829 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.847+0000 7f1759400640 1 --2- 172.21.15.107:0/3498046590 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f17400a2960 0x7f17400a4e20 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f17480058c0 tx=0x7f174800c4b0 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:08:49.829 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.847+0000 7f1763be6740 1 --2- 172.21.15.107:0/335097234 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55656d5bc7a0 0x55656d5c5030 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f174c009900 tx=0x7f174c02c6b0 comp rx=0 tx=0).stop 2024-03-08T21:08:49.829 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.847+0000 7f1763be6740 1 -- 172.21.15.107:0/335097234 shutdown_connections 2024-03-08T21:08:49.829 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.847+0000 7f1763be6740 1 --2- 172.21.15.107:0/335097234 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55656d5c5570 0x55656d5c7960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:08:49.830 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.847+0000 7f1763be6740 1 --2- 172.21.15.107:0/335097234 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55656d5bc7a0 0x55656d5c5030 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:08:49.830 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.847+0000 7f1763be6740 1 --2- 172.21.15.107:0/335097234 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55656d5bbe80 0x55656d5bc260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:08:49.830 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.847+0000 7f1763be6740 1 -- 172.21.15.107:0/335097234 >> 172.21.15.107:0/335097234 conn(0x55656d5b67a0 msgr2=0x55656d5b6b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:08:49.830 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.847+0000 7f1763be6740 1 -- 172.21.15.107:0/335097234 shutdown_connections 2024-03-08T21:08:49.830 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.847+0000 7f1763be6740 1 -- 172.21.15.107:0/335097234 wait complete. 2024-03-08T21:08:49.830 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.847+0000 7f1763be6740 1 -- start start 2024-03-08T21:08:49.830 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.848+0000 7f1763be6740 1 Processor -- start 2024-03-08T21:08:49.830 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.848+0000 7f1763be6740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55656d5bbe80 0x55656d5c4e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:08:49.830 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.848+0000 7f1763be6740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55656d5bc7a0 0x55656d5c21d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:08:49.830 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.848+0000 7f1763be6740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55656d5c5570 0x55656d5c2710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:08:49.830 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.848+0000 7f1763be6740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55656d5c8140 con 0x55656d5bbe80 2024-03-08T21:08:49.830 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.848+0000 7f1763be6740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55656d4db9f0 con 0x55656d5bc7a0 2024-03-08T21:08:49.831 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.848+0000 7f1763be6740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55656d5c82c0 con 0x55656d5c5570 2024-03-08T21:08:49.831 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.848+0000 7f1763be6740 10 client.0 _ll_register_callbacks cb 0x55656d233a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T21:08:49.831 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.848+0000 7f1759e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55656d5bbe80 0x55656d5c4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:08:49.831 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.848+0000 7f1759e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55656d5bbe80 0x55656d5c4e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:33494/0 (socket says 172.21.15.107:33494) 2024-03-08T21:08:49.831 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.848+0000 7f1759e00640 1 -- 172.21.15.107:0/2499451709 learned_addr learned my addr 172.21.15.107:0/2499451709 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:08:49.831 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.849+0000 7f1759e00640 1 -- 172.21.15.107:0/2499451709 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55656d5c5570 msgr2=0x55656d5c2710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:08:49.831 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.849+0000 7f1759e00640 1 --2- 172.21.15.107:0/2499451709 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55656d5c5570 0x55656d5c2710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:08:49.831 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.849+0000 7f1759400640 1 --2- 172.21.15.107:0/2499451709 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55656d5bc7a0 0x55656d5c21d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:08:49.831 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.849+0000 7f1759e00640 1 -- 172.21.15.107:0/2499451709 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55656d5bc7a0 msgr2=0x55656d5c21d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:08:49.831 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.849+0000 7f1759e00640 1 --2- 172.21.15.107:0/2499451709 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55656d5bc7a0 0x55656d5c21d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:08:49.831 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.849+0000 7f1759e00640 1 -- 172.21.15.107:0/2499451709 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55656d234210 con 0x55656d5bbe80 2024-03-08T21:08:49.831 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.849+0000 7f1759400640 1 --2- 172.21.15.107:0/2499451709 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55656d5bc7a0 0x55656d5c21d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T21:08:49.832 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.849+0000 7f1759e00640 1 --2- 172.21.15.107:0/2499451709 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55656d5bbe80 0x55656d5c4e40 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f174c02c210 tx=0x7f174c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=331fe2b6983ba3fe server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:08:49.832 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.849+0000 7f173be00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-03-08T21:08:49.832 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.849+0000 7f173be00640 1 -- 172.21.15.107:0/2499451709 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f174c003a00 con 0x55656d5bbe80 2024-03-08T21:08:49.832 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.849+0000 7f173be00640 1 -- 172.21.15.107:0/2499451709 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f174c003ba0 con 0x55656d5bbe80 2024-03-08T21:08:49.832 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.850+0000 7f1763be6740 1 -- 172.21.15.107:0/2499451709 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f174c0054f0 con 0x55656d5bbe80 2024-03-08T21:08:49.832 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.850+0000 7f173be00640 1 -- 172.21.15.107:0/2499451709 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f174c0044a0 con 0x55656d5bbe80 2024-03-08T21:08:49.832 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.850+0000 7f1763be6740 10 client.16149 fetch_fsmap learned FSMap version 36 2024-03-08T21:08:49.832 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.850+0000 7f1763be6740 1 -- 172.21.15.107:0/2499451709 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x55656d300620 con 0x55656d5bbe80 2024-03-08T21:08:49.832 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.850+0000 7f173be00640 1 -- 172.21.15.107:0/2499451709 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f174c0054f0 con 0x55656d5bbe80 2024-03-08T21:08:49.832 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.850+0000 7f173be00640 1 -- 172.21.15.107:0/2499451709 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 36) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f174c03f430 con 0x55656d5bbe80 2024-03-08T21:08:49.832 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.850+0000 7f1763be6740 10 client.16149 fetch_fsmap finished waiting for FSMap version 36 2024-03-08T21:08:49.832 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.850+0000 7f1763be6740 10 client.16149 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-08T21:08:49.833 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.850+0000 7f1763be6740 20 client.16149 populate_metadata read hostname 'smithi107' 2024-03-08T21:08:49.833 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.851+0000 7f1763be6740 1 --2- 172.21.15.107:0/2499451709 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55656d5d51d0 0x55656d5d75c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:08:49.833 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.851+0000 7f1763be6740 4 client.16149 mds_command: new command op to 34475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-08T21:08:49.833 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.851+0000 7f1763be6740 1 -- 172.21.15.107:0/2499451709 --> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55656d52c910 con 0x55656d5d51d0 2024-03-08T21:08:49.834 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.851+0000 7f1759400640 1 --2- 172.21.15.107:0/2499451709 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55656d5d51d0 0x55656d5d75c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:08:49.835 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.852+0000 7f1759400640 1 --2- 172.21.15.107:0/2499451709 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55656d5d51d0 0x55656d5d75c0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a3bcdca87984b75a server_cookie=f0c761f420c5f5a5 in_seq=0 out_seq=0 2024-03-08T21:08:49.835 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.852+0000 7f173be00640 10 client.16149 ms_handle_connect on v2:172.21.15.149:6838/55290596 2024-03-08T21:08:49.835 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.854+0000 7f173be00640 1 -- 172.21.15.107:0/2499451709 <== mds.0 v2:172.21.15.149:6838/55290596 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55656d52c910 con 0x55656d5d51d0 2024-03-08T21:08:49.835 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.854+0000 7f173be00640 10 client.16149 handle_command_reply: tid=0 2024-03-08T21:08:49.835 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.854+0000 7f1763be6740 1 client.16149 shutdown 2024-03-08T21:08:49.835 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.854+0000 7f1763be6740 1 -- 172.21.15.107:0/2499451709 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55656d5bbe80 msgr2=0x55656d5c4e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:08:49.835 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.854+0000 7f1763be6740 1 --2- 172.21.15.107:0/2499451709 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55656d5bbe80 0x55656d5c4e40 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f174c02c210 tx=0x7f174c004270 comp rx=0 tx=0).stop 2024-03-08T21:08:49.835 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.854+0000 7f1763be6740 1 -- 172.21.15.107:0/2499451709 shutdown_connections 2024-03-08T21:08:49.835 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.854+0000 7f1763be6740 1 --2- 172.21.15.107:0/2499451709 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55656d5c5570 0x55656d5c2710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:08:49.835 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.854+0000 7f1763be6740 1 --2- 172.21.15.107:0/2499451709 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55656d5d51d0 0x55656d5d75c0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:08:49.835 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.854+0000 7f1763be6740 1 --2- 172.21.15.107:0/2499451709 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55656d5bc7a0 0x55656d5c21d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:08:49.836 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.854+0000 7f1763be6740 1 --2- 172.21.15.107:0/2499451709 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55656d5bbe80 0x55656d5c4e40 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:08:49.836 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.854+0000 7f1763be6740 1 -- 172.21.15.107:0/2499451709 >> 172.21.15.107:0/2499451709 conn(0x55656d5b67a0 msgr2=0x55656d5b84f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:08:49.836 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.854+0000 7f173be00640 0 client.16149 ms_handle_reset on v2:172.21.15.149:6838/55290596 2024-03-08T21:08:49.836 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.854+0000 7f1763be6740 1 -- 172.21.15.107:0/2499451709 shutdown_connections 2024-03-08T21:08:49.836 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.854+0000 7f1763be6740 1 -- 172.21.15.107:0/2499451709 wait complete. 2024-03-08T21:08:49.836 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.854+0000 7f1763be6740 20 client.16149 trim_cache size 0 max 16384 2024-03-08T21:08:49.854 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.874+0000 7f1763be6740 1 Processor -- start 2024-03-08T21:08:49.854 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.874+0000 7f1763be6740 1 -- start start 2024-03-08T21:08:49.854 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.874+0000 7f1763be6740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55656d5bbe80 0x55656d5c4900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:08:49.854 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.874+0000 7f1763be6740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55656d5bc7a0 0x55656d5c4e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:08:49.855 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.874+0000 7f1763be6740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55656d5c5570 0x55656d5bf430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:08:49.855 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.874+0000 7f1763be6740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f175c1aedb0 con 0x55656d5c5570 2024-03-08T21:08:49.855 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.874+0000 7f1763be6740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f175c1b0730 con 0x55656d5bbe80 2024-03-08T21:08:49.855 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.874+0000 7f1763be6740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f175c1af410 con 0x55656d5bc7a0 2024-03-08T21:08:49.855 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.875+0000 7f175a800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55656d5c5570 0x55656d5bf430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:08:49.855 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.875+0000 7f175a800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55656d5c5570 0x55656d5bf430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:33502/0 (socket says 172.21.15.107:33502) 2024-03-08T21:08:49.855 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.875+0000 7f175a800640 1 -- 172.21.15.107:0/813167856 learned_addr learned my addr 172.21.15.107:0/813167856 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:08:49.855 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.875+0000 7f1759400640 1 --2- 172.21.15.107:0/813167856 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55656d5bc7a0 0x55656d5c4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:08:49.856 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.875+0000 7f1759e00640 1 --2- 172.21.15.107:0/813167856 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55656d5bbe80 0x55656d5c4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:08:49.856 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.875+0000 7f175a800640 1 -- 172.21.15.107:0/813167856 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55656d5bc7a0 msgr2=0x55656d5c4e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:08:49.856 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.875+0000 7f175a800640 1 --2- 172.21.15.107:0/813167856 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55656d5bc7a0 0x55656d5c4e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:08:49.856 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.875+0000 7f175a800640 1 -- 172.21.15.107:0/813167856 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55656d5bbe80 msgr2=0x55656d5c4900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:08:49.856 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.875+0000 7f175a800640 1 --2- 172.21.15.107:0/813167856 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55656d5bbe80 0x55656d5c4900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:08:49.856 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.875+0000 7f175a800640 1 -- 172.21.15.107:0/813167856 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55656d5dc4d0 con 0x55656d5c5570 2024-03-08T21:08:49.856 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.875+0000 7f175a800640 1 --2- 172.21.15.107:0/813167856 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55656d5c5570 0x55656d5bf430 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f17540cc790 tx=0x7f175409b130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7551c8f3437a7179 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:08:49.856 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.876+0000 7f1746a00640 1 -- 172.21.15.107:0/813167856 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f175401f040 con 0x55656d5c5570 2024-03-08T21:08:49.856 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.876+0000 7f1746a00640 1 -- 172.21.15.107:0/813167856 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f175409b5f0 con 0x55656d5c5570 2024-03-08T21:08:49.857 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.876+0000 7f1746a00640 1 -- 172.21.15.107:0/813167856 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1754097c60 con 0x55656d5c5570 2024-03-08T21:08:49.861 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.876+0000 7f1763be6740 1 -- 172.21.15.107:0/813167856 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55656d5c5570 msgr2=0x55656d5bf430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:08:49.861 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.876+0000 7f1763be6740 1 --2- 172.21.15.107:0/813167856 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55656d5c5570 0x55656d5bf430 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f17540cc790 tx=0x7f175409b130 comp rx=0 tx=0).stop 2024-03-08T21:08:49.861 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.876+0000 7f1763be6740 1 -- 172.21.15.107:0/813167856 shutdown_connections 2024-03-08T21:08:49.861 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.876+0000 7f1763be6740 1 --2- 172.21.15.107:0/813167856 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55656d5c5570 0x55656d5bf430 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:08:49.861 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.876+0000 7f1763be6740 1 --2- 172.21.15.107:0/813167856 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55656d5bc7a0 0x55656d5c4e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:08:49.861 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.876+0000 7f1763be6740 1 --2- 172.21.15.107:0/813167856 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55656d5bbe80 0x55656d5c4900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:08:49.861 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.876+0000 7f1763be6740 1 -- 172.21.15.107:0/813167856 >> 172.21.15.107:0/813167856 conn(0x55656d5b67a0 msgr2=0x55656d5c0aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:08:49.861 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.877+0000 7f1763be6740 1 -- 172.21.15.107:0/813167856 shutdown_connections 2024-03-08T21:08:49.861 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.877+0000 7f1763be6740 1 -- 172.21.15.107:0/813167856 wait complete. 2024-03-08T21:08:49.861 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.877+0000 7f1763be6740 1 -- start start 2024-03-08T21:08:49.861 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.877+0000 7f1763be6740 1 Processor -- start 2024-03-08T21:08:49.861 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.878+0000 7f1763be6740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55656d5bbe80 0x55656d5cc750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:08:49.862 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.878+0000 7f1763be6740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55656d5bc7a0 0x55656d5caaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:08:49.862 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.878+0000 7f1763be6740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55656d5c5570 0x55656d5cb030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:08:49.862 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.878+0000 7f1763be6740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f175c1b00d0 con 0x55656d5c5570 2024-03-08T21:08:49.862 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.878+0000 7f1763be6740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f175c1afa70 con 0x55656d5bbe80 2024-03-08T21:08:49.862 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.878+0000 7f1763be6740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f175c1ad070 con 0x55656d5bc7a0 2024-03-08T21:08:49.862 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.878+0000 7f1763be6740 10 client.0 _ll_register_callbacks cb 0x7f174c005e40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T21:08:49.862 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.878+0000 7f175a800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55656d5c5570 0x55656d5cb030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:08:49.862 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.878+0000 7f175a800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55656d5c5570 0x55656d5cb030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:33508/0 (socket says 172.21.15.107:33508) 2024-03-08T21:08:49.862 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.878+0000 7f175a800640 1 -- 172.21.15.107:0/2214219897 learned_addr learned my addr 172.21.15.107:0/2214219897 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:08:49.862 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.878+0000 7f1759400640 1 --2- 172.21.15.107:0/2214219897 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55656d5bc7a0 0x55656d5caaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:08:49.862 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.878+0000 7f1759e00640 1 --2- 172.21.15.107:0/2214219897 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55656d5bbe80 0x55656d5cc750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:08:49.862 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.879+0000 7f175a800640 1 -- 172.21.15.107:0/2214219897 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55656d5bc7a0 msgr2=0x55656d5caaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:08:49.862 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.879+0000 7f175a800640 1 --2- 172.21.15.107:0/2214219897 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55656d5bc7a0 0x55656d5caaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:08:49.863 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.879+0000 7f175a800640 1 -- 172.21.15.107:0/2214219897 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55656d5bbe80 msgr2=0x55656d5cc750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:08:49.863 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.879+0000 7f175a800640 1 --2- 172.21.15.107:0/2214219897 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55656d5bbe80 0x55656d5cc750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:08:49.863 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.879+0000 7f175a800640 1 -- 172.21.15.107:0/2214219897 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55656d5cb6a0 con 0x55656d5c5570 2024-03-08T21:08:49.863 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.879+0000 7f175a800640 1 --2- 172.21.15.107:0/2214219897 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55656d5c5570 0x55656d5cb030 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f175400efd0 tx=0x7f1754005790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=447aa87f8faf4032 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:08:49.863 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.879+0000 7f173be00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-03-08T21:08:49.863 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.879+0000 7f173be00640 1 -- 172.21.15.107:0/2214219897 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17540b9060 con 0x55656d5c5570 2024-03-08T21:08:49.863 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.879+0000 7f173be00640 1 -- 172.21.15.107:0/2214219897 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f17540b4260 con 0x55656d5c5570 2024-03-08T21:08:49.863 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.879+0000 7f1763be6740 1 -- 172.21.15.107:0/2214219897 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f175409b5f0 con 0x55656d5c5570 2024-03-08T21:08:49.863 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.880+0000 7f173be00640 1 -- 172.21.15.107:0/2214219897 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1754081050 con 0x55656d5c5570 2024-03-08T21:08:49.863 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.880+0000 7f1763be6740 10 client.16155 fetch_fsmap learned FSMap version 36 2024-03-08T21:08:49.863 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.880+0000 7f1763be6740 1 -- 172.21.15.107:0/2214219897 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x55656d62c270 con 0x55656d5c5570 2024-03-08T21:08:49.863 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.880+0000 7f173be00640 1 -- 172.21.15.107:0/2214219897 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f175409b5f0 con 0x55656d5c5570 2024-03-08T21:08:49.863 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.880+0000 7f173be00640 1 -- 172.21.15.107:0/2214219897 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 36) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f175400f040 con 0x55656d5c5570 2024-03-08T21:08:49.863 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.880+0000 7f1763be6740 10 client.16155 fetch_fsmap finished waiting for FSMap version 36 2024-03-08T21:08:49.864 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.880+0000 7f1763be6740 10 client.16155 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-08T21:08:49.864 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.880+0000 7f1763be6740 20 client.16155 populate_metadata read hostname 'smithi107' 2024-03-08T21:08:49.864 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.880+0000 7f1763be6740 1 --2- 172.21.15.107:0/2214219897 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55656d62c870 0x55656d62ec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:08:49.864 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.880+0000 7f1763be6740 4 client.16155 mds_command: new command op to 34475 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-08T21:08:49.864 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.880+0000 7f1763be6740 1 -- 172.21.15.107:0/2214219897 --> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55656d526c60 con 0x55656d62c870 2024-03-08T21:08:49.864 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.881+0000 7f1759e00640 1 --2- 172.21.15.107:0/2214219897 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55656d62c870 0x55656d62ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:08:49.864 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.882+0000 7f1759e00640 1 --2- 172.21.15.107:0/2214219897 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55656d62c870 0x55656d62ec90 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8a283e30877724c7 server_cookie=53520cc31240d56e in_seq=0 out_seq=0 2024-03-08T21:08:49.864 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.882+0000 7f173be00640 10 client.16155 ms_handle_connect on v2:172.21.15.149:6838/55290596 2024-03-08T21:08:49.872 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.893+0000 7f173be00640 1 -- 172.21.15.107:0/2214219897 <== mds.0 v2:172.21.15.149:6838/55290596 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55656d526c60 con 0x55656d62c870 2024-03-08T21:08:49.872 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.893+0000 7f173be00640 10 client.16155 handle_command_reply: tid=0 2024-03-08T21:08:49.872 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.893+0000 7f1763be6740 1 client.16155 shutdown 2024-03-08T21:08:49.873 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.894+0000 7f1763be6740 1 -- 172.21.15.107:0/2214219897 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55656d5c5570 msgr2=0x55656d5cb030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:08:49.873 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.894+0000 7f1763be6740 1 --2- 172.21.15.107:0/2214219897 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55656d5c5570 0x55656d5cb030 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f175400efd0 tx=0x7f1754005790 comp rx=0 tx=0).stop 2024-03-08T21:08:49.873 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.894+0000 7f1763be6740 1 -- 172.21.15.107:0/2214219897 shutdown_connections 2024-03-08T21:08:49.873 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.894+0000 7f1763be6740 1 --2- 172.21.15.107:0/2214219897 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55656d5c5570 0x55656d5cb030 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:08:49.873 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.894+0000 7f1763be6740 1 --2- 172.21.15.107:0/2214219897 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55656d5bc7a0 0x55656d5caaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:08:49.873 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.894+0000 7f1763be6740 1 --2- 172.21.15.107:0/2214219897 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55656d62c870 0x55656d62ec90 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:08:49.873 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.894+0000 7f1763be6740 1 --2- 172.21.15.107:0/2214219897 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55656d5bbe80 0x55656d5cc750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:08:49.873 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.894+0000 7f1763be6740 1 -- 172.21.15.107:0/2214219897 >> 172.21.15.107:0/2214219897 conn(0x55656d5b67a0 msgr2=0x55656d5bcfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:08:49.873 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.894+0000 7f173be00640 0 client.16155 ms_handle_reset on v2:172.21.15.149:6838/55290596 2024-03-08T21:08:49.873 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.894+0000 7f1763be6740 1 -- 172.21.15.107:0/2214219897 shutdown_connections 2024-03-08T21:08:49.873 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.894+0000 7f1763be6740 1 -- 172.21.15.107:0/2214219897 wait complete. 2024-03-08T21:08:49.874 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.894+0000 7f1763be6740 20 client.16155 trim_cache size 0 max 16384 2024-03-08T21:08:49.874 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:{ 2024-03-08T21:08:49.874 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "message": "", 2024-03-08T21:08:49.874 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "return_code": 0 2024-03-08T21:08:49.874 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:} 2024-03-08T21:08:49.875 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.895+0000 7f175b200640 1 -- 172.21.15.107:0/3498046590 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f17400a2960 msgr2=0x7f17400a4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:08:49.875 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.895+0000 7f175b200640 1 --2- 172.21.15.107:0/3498046590 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f17400a2960 0x7f17400a4e20 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f17480058c0 tx=0x7f174800c4b0 comp rx=0 tx=0).stop 2024-03-08T21:08:49.875 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.895+0000 7f175b200640 1 -- 172.21.15.107:0/3498046590 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f175c10b6c0 msgr2=0x7f175c199990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:08:49.875 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.895+0000 7f175b200640 1 --2- 172.21.15.107:0/3498046590 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f175c10b6c0 0x7f175c199990 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f175400eaa0 tx=0x7f175400ef60 comp rx=0 tx=0).stop 2024-03-08T21:08:49.875 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.895+0000 7f175b200640 1 -- 172.21.15.107:0/3498046590 shutdown_connections 2024-03-08T21:08:49.875 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.895+0000 7f175b200640 1 --2- 172.21.15.107:0/3498046590 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f17400a2960 0x7f17400a4e20 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:08:49.875 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.895+0000 7f175b200640 1 --2- 172.21.15.107:0/3498046590 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f175c10b6c0 0x7f175c199990 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:08:49.875 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.895+0000 7f175b200640 1 --2- 172.21.15.107:0/3498046590 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f175c102bc0 0x7f175c19fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:08:49.875 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.895+0000 7f175b200640 1 --2- 172.21.15.107:0/3498046590 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f175c1022a0 0x7f175c19f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:08:49.876 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.895+0000 7f175b200640 1 -- 172.21.15.107:0/3498046590 >> 172.21.15.107:0/3498046590 conn(0x7f175c0fbb00 msgr2=0x7f175c100f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:08:49.876 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.895+0000 7f175b200640 1 -- 172.21.15.107:0/3498046590 shutdown_connections 2024-03-08T21:08:49.876 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:08:49.895+0000 7f175b200640 1 -- 172.21.15.107:0/3498046590 wait complete. 2024-03-08T21:08:51.010 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:08:50 smithi149 ceph-mon[35042]: pgmap v260: 97 pgs: 97 active+clean; 5.7 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 35 KiB/s wr, 8 op/s 2024-03-08T21:08:51.124 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:08:50 smithi204 ceph-mon[34935]: pgmap v260: 97 pgs: 97 active+clean; 5.7 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 35 KiB/s wr, 8 op/s 2024-03-08T21:08:51.213 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:08:50 smithi107 ceph-mon[31381]: pgmap v260: 97 pgs: 97 active+clean; 5.7 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 35 KiB/s wr, 8 op/s 2024-03-08T21:08:53.011 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:08:52 smithi149 ceph-mon[35042]: pgmap v261: 97 pgs: 97 active+clean; 5.7 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 35 KiB/s wr, 8 op/s 2024-03-08T21:08:53.124 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:08:52 smithi204 ceph-mon[34935]: pgmap v261: 97 pgs: 97 active+clean; 5.7 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 35 KiB/s wr, 8 op/s 2024-03-08T21:08:53.213 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:08:52 smithi107 ceph-mon[31381]: pgmap v261: 97 pgs: 97 active+clean; 5.7 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 35 KiB/s wr, 8 op/s 2024-03-08T21:08:54.712 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:08:54 smithi107 ceph-mon[31381]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T21:08:54.760 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:08:54 smithi149 ceph-mon[35042]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T21:08:54.873 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:08:54 smithi204 ceph-mon[34935]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T21:08:55.713 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:08:55 smithi107 ceph-mon[31381]: pgmap v262: 97 pgs: 97 active+clean; 5.7 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 35 KiB/s wr, 12 op/s 2024-03-08T21:08:55.713 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:08:55 smithi107 ceph-mon[31381]: from='mgr.15375 ' entity='mgr.x' 2024-03-08T21:08:55.713 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:08:55 smithi107 ceph-mon[31381]: from='mgr.15375 ' entity='mgr.x' 2024-03-08T21:08:55.760 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:08:55 smithi149 ceph-mon[35042]: pgmap v262: 97 pgs: 97 active+clean; 5.7 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 35 KiB/s wr, 12 op/s 2024-03-08T21:08:55.761 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:08:55 smithi149 ceph-mon[35042]: from='mgr.15375 ' entity='mgr.x' 2024-03-08T21:08:55.761 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:08:55 smithi149 ceph-mon[35042]: from='mgr.15375 ' entity='mgr.x' 2024-03-08T21:08:55.874 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:08:55 smithi204 ceph-mon[34935]: pgmap v262: 97 pgs: 97 active+clean; 5.7 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 35 KiB/s wr, 12 op/s 2024-03-08T21:08:55.874 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:08:55 smithi204 ceph-mon[34935]: from='mgr.15375 ' entity='mgr.x' 2024-03-08T21:08:55.874 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:08:55 smithi204 ceph-mon[34935]: from='mgr.15375 ' entity='mgr.x' 2024-03-08T21:08:56.873 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:08:56 smithi204 ceph-mon[34935]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T21:08:56.873 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:08:56 smithi204 ceph-mon[34935]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T21:08:56.874 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:08:56 smithi204 ceph-mon[34935]: from='mgr.15375 ' entity='mgr.x' 2024-03-08T21:08:56.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:08:56 smithi107 ceph-mon[31381]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T21:08:56.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:08:56 smithi107 ceph-mon[31381]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T21:08:56.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:08:56 smithi107 ceph-mon[31381]: from='mgr.15375 ' entity='mgr.x' 2024-03-08T21:08:57.010 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:08:56 smithi149 ceph-mon[35042]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T21:08:57.011 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:08:56 smithi149 ceph-mon[35042]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T21:08:57.011 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:08:56 smithi149 ceph-mon[35042]: from='mgr.15375 ' entity='mgr.x' 2024-03-08T21:08:57.874 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:08:57 smithi204 ceph-mon[34935]: pgmap v263: 97 pgs: 97 active+clean; 5.7 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 35 KiB/s wr, 8 op/s 2024-03-08T21:08:57.963 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:08:57 smithi107 ceph-mon[31381]: pgmap v263: 97 pgs: 97 active+clean; 5.7 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 35 KiB/s wr, 8 op/s 2024-03-08T21:08:58.010 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:08:57 smithi149 ceph-mon[35042]: pgmap v263: 97 pgs: 97 active+clean; 5.7 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 35 KiB/s wr, 8 op/s 2024-03-08T21:08:59.010 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:08:58 smithi149 ceph-mon[35042]: pgmap v264: 97 pgs: 97 active+clean; 5.7 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 35 KiB/s wr, 12 op/s 2024-03-08T21:08:59.123 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:08:58 smithi204 ceph-mon[34935]: pgmap v264: 97 pgs: 97 active+clean; 5.7 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 35 KiB/s wr, 12 op/s 2024-03-08T21:08:59.213 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:08:58 smithi107 ceph-mon[31381]: pgmap v264: 97 pgs: 97 active+clean; 5.7 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 35 KiB/s wr, 12 op/s 2024-03-08T21:09:01.011 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:09:00 smithi149 ceph-mon[35042]: pgmap v265: 97 pgs: 97 active+clean; 5.7 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:09:01.124 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:09:00 smithi204 ceph-mon[34935]: pgmap v265: 97 pgs: 97 active+clean; 5.7 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:09:01.213 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:09:00 smithi107 ceph-mon[31381]: pgmap v265: 97 pgs: 97 active+clean; 5.7 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:09:03.010 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:09:02 smithi149 ceph-mon[35042]: pgmap v266: 97 pgs: 97 active+clean; 5.7 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:09:03.018 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:03.039+0000 7fd647200640 1 -- 172.21.15.107:0/2349851945 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd648108620 msgr2=0x7fd6481101f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:09:03.018 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:03.039+0000 7fd647200640 1 --2- 172.21.15.107:0/2349851945 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd648108620 0x7fd6481101f0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fd640009ca0 tx=0x7fd64002c0f0 comp rx=0 tx=0).stop 2024-03-08T21:09:03.018 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:03.039+0000 7fd647200640 1 -- 172.21.15.107:0/2349851945 shutdown_connections 2024-03-08T21:09:03.018 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:03.039+0000 7fd647200640 1 --2- 172.21.15.107:0/2349851945 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd648108620 0x7fd6481101f0 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:09:03.018 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:03.039+0000 7fd647200640 1 --2- 172.21.15.107:0/2349851945 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd648103f00 0x7fd648107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:09:03.018 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:03.039+0000 7fd647200640 1 --2- 172.21.15.107:0/2349851945 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd648103550 0x7fd648103930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:09:03.018 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:03.039+0000 7fd647200640 1 -- 172.21.15.107:0/2349851945 >> 172.21.15.107:0/2349851945 conn(0x7fd6480fedd0 msgr2=0x7fd6481011f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:09:03.018 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:03.039+0000 7fd647200640 1 -- 172.21.15.107:0/2349851945 shutdown_connections 2024-03-08T21:09:03.019 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:03.039+0000 7fd647200640 1 -- 172.21.15.107:0/2349851945 wait complete. 2024-03-08T21:09:03.021 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:03.040+0000 7fd647200640 1 Processor -- start 2024-03-08T21:09:03.021 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:03.040+0000 7fd647200640 1 -- start start 2024-03-08T21:09:03.021 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:03.041+0000 7fd647200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd648103550 0x7fd64819f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:09:03.022 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:03.041+0000 7fd647200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd648108620 0x7fd64819fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:09:03.022 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:03.041+0000 7fd647200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd6481999c0 0x7fd648199e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:09:03.022 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:03.041+0000 7fd647200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd648113590 con 0x7fd6481999c0 2024-03-08T21:09:03.022 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:03.041+0000 7fd647200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd648113410 con 0x7fd648108620 2024-03-08T21:09:03.022 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:03.041+0000 7fd647200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd648113710 con 0x7fd648103550 2024-03-08T21:09:03.022 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:03.041+0000 7fd646800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd6481999c0 0x7fd648199e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:09:03.023 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:03.041+0000 7fd646800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd6481999c0 0x7fd648199e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:40374/0 (socket says 172.21.15.107:40374) 2024-03-08T21:09:03.023 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:03.041+0000 7fd646800640 1 -- 172.21.15.107:0/1504724516 learned_addr learned my addr 172.21.15.107:0/1504724516 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:09:03.023 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:03.041+0000 7fd645e00640 1 --2- 172.21.15.107:0/1504724516 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd648103550 0x7fd64819f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:09:03.023 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:03.041+0000 7fd645400640 1 --2- 172.21.15.107:0/1504724516 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd648108620 0x7fd64819fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:09:03.023 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:03.042+0000 7fd646800640 1 -- 172.21.15.107:0/1504724516 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd648103550 msgr2=0x7fd64819f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:09:03.023 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:03.042+0000 7fd646800640 1 --2- 172.21.15.107:0/1504724516 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd648103550 0x7fd64819f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:09:03.023 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:03.042+0000 7fd646800640 1 -- 172.21.15.107:0/1504724516 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd648108620 msgr2=0x7fd64819fd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:09:03.023 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:03.042+0000 7fd646800640 1 --2- 172.21.15.107:0/1504724516 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd648108620 0x7fd64819fd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:09:03.024 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:03.042+0000 7fd646800640 1 -- 172.21.15.107:0/1504724516 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd64819a4e0 con 0x7fd6481999c0 2024-03-08T21:09:03.024 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:03.042+0000 7fd645400640 1 --2- 172.21.15.107:0/1504724516 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd648108620 0x7fd64819fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T21:09:03.024 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:03.042+0000 7fd646800640 1 --2- 172.21.15.107:0/1504724516 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd6481999c0 0x7fd648199e70 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fd64002c700 tx=0x7fd640035d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:09:03.024 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:03.042+0000 7fd63ea00640 1 -- 172.21.15.107:0/1504724516 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd640039040 con 0x7fd6481999c0 2024-03-08T21:09:03.024 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:03.042+0000 7fd63ea00640 1 -- 172.21.15.107:0/1504724516 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fd640044070 con 0x7fd6481999c0 2024-03-08T21:09:03.024 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:03.042+0000 7fd63ea00640 1 -- 172.21.15.107:0/1504724516 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6400042a0 con 0x7fd6481999c0 2024-03-08T21:09:03.024 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:03.042+0000 7fd647200640 1 -- 172.21.15.107:0/1504724516 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd64819a770 con 0x7fd6481999c0 2024-03-08T21:09:03.024 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:03.042+0000 7fd647200640 1 -- 172.21.15.107:0/1504724516 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6481a6910 con 0x7fd6481999c0 2024-03-08T21:09:03.027 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:03.046+0000 7fd64fa81740 1 Processor -- start 2024-03-08T21:09:03.027 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:03.046+0000 7fd63ea00640 1 -- 172.21.15.107:0/1504724516 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) v1 ==== 137422+0+0 (secure 0 0 0) 0x7fd640003970 con 0x7fd6481999c0 2024-03-08T21:09:03.027 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:03.046+0000 7fd64fa81740 1 -- start start 2024-03-08T21:09:03.027 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:03.047+0000 7fd64fa81740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562b25916e80 0x562b25917260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:09:03.027 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:03.047+0000 7fd64fa81740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562b259177a0 0x562b25920030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:09:03.027 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:03.047+0000 7fd64fa81740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b25920570 0x562b25922960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:09:03.027 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:03.047+0000 7fd64fa81740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b2587a3d0 con 0x562b259177a0 2024-03-08T21:09:03.028 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:03.047+0000 7fd64fa81740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b258236b0 con 0x562b25916e80 2024-03-08T21:09:03.028 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:03.047+0000 7fd64fa81740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b25876470 con 0x562b25920570 2024-03-08T21:09:03.028 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:03.047+0000 7fd645e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562b259177a0 0x562b25920030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:09:03.028 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:03.047+0000 7fd645e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562b259177a0 0x562b25920030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:40376/0 (socket says 172.21.15.107:40376) 2024-03-08T21:09:03.028 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:03.047+0000 7fd645e00640 1 -- 172.21.15.107:0/1911586907 learned_addr learned my addr 172.21.15.107:0/1911586907 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:09:03.028 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:03.048+0000 7fd645400640 1 --2- 172.21.15.107:0/1911586907 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562b25916e80 0x562b25917260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:09:03.028 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:03.048+0000 7fd645400640 1 --2- 172.21.15.107:0/1911586907 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b25920570 0x562b25922960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:09:03.028 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:03.048+0000 7fd645e00640 1 -- 172.21.15.107:0/1911586907 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b25920570 msgr2=0x562b25922960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:09:03.028 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:03.048+0000 7fd645e00640 1 --2- 172.21.15.107:0/1911586907 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b25920570 0x562b25922960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:09:03.028 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:03.048+0000 7fd645e00640 1 -- 172.21.15.107:0/1911586907 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562b25916e80 msgr2=0x562b25917260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:09:03.028 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:03.048+0000 7fd645e00640 1 --2- 172.21.15.107:0/1911586907 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562b25916e80 0x562b25917260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:09:03.029 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:03.048+0000 7fd645e00640 1 -- 172.21.15.107:0/1911586907 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562b25849b50 con 0x562b259177a0 2024-03-08T21:09:03.029 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:03.048+0000 7fd645400640 1 --2- 172.21.15.107:0/1911586907 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b25920570 0x562b25922960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T21:09:03.029 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:03.048+0000 7fd63ea00640 1 --2- 172.21.15.107:0/1504724516 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7fd62c0a29e0 0x7fd62c0a4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:09:03.029 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:03.048+0000 7fd63ea00640 1 -- 172.21.15.107:0/1504724516 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fd6400f1040 con 0x7fd6481999c0 2024-03-08T21:09:03.029 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:03.048+0000 7fd645e00640 1 --2- 172.21.15.107:0/1911586907 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562b259177a0 0x562b25920030 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fd638009c20 tx=0x7fd6380117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc43c28238222197 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:09:03.029 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:03.048+0000 7fd645400640 1 --2- 172.21.15.107:0/1504724516 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7fd62c0a29e0 0x7fd62c0a4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:09:03.029 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:03.049+0000 7fd633400640 1 -- 172.21.15.107:0/1911586907 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd638004030 con 0x562b259177a0 2024-03-08T21:09:03.029 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:03.049+0000 7fd633400640 1 -- 172.21.15.107:0/1911586907 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fd6380035f0 con 0x562b259177a0 2024-03-08T21:09:03.029 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:03.049+0000 7fd633400640 1 -- 172.21.15.107:0/1911586907 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6380126b0 con 0x562b259177a0 2024-03-08T21:09:03.034 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:03.049+0000 7fd64fa81740 1 -- 172.21.15.107:0/1911586907 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562b259177a0 msgr2=0x562b25920030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:09:03.034 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:03.049+0000 7fd645400640 1 --2- 172.21.15.107:0/1504724516 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7fd62c0a29e0 0x7fd62c0a4ea0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x562b25912bc0 tx=0x7fd634014040 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:09:03.034 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:03.049+0000 7fd64fa81740 1 --2- 172.21.15.107:0/1911586907 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562b259177a0 0x562b25920030 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fd638009c20 tx=0x7fd6380117b0 comp rx=0 tx=0).stop 2024-03-08T21:09:03.034 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:03.050+0000 7fd64fa81740 1 -- 172.21.15.107:0/1911586907 shutdown_connections 2024-03-08T21:09:03.035 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:03.050+0000 7fd64fa81740 1 --2- 172.21.15.107:0/1911586907 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b25920570 0x562b25922960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:09:03.035 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:03.050+0000 7fd64fa81740 1 --2- 172.21.15.107:0/1911586907 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562b259177a0 0x562b25920030 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:09:03.035 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:03.050+0000 7fd64fa81740 1 --2- 172.21.15.107:0/1911586907 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562b25916e80 0x562b25917260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:09:03.035 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:03.050+0000 7fd64fa81740 1 -- 172.21.15.107:0/1911586907 >> 172.21.15.107:0/1911586907 conn(0x562b259117a0 msgr2=0x562b25911b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:09:03.035 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:03.050+0000 7fd64fa81740 1 -- 172.21.15.107:0/1911586907 shutdown_connections 2024-03-08T21:09:03.035 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:03.050+0000 7fd64fa81740 1 -- 172.21.15.107:0/1911586907 wait complete. 2024-03-08T21:09:03.035 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:03.050+0000 7fd64fa81740 1 -- start start 2024-03-08T21:09:03.035 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:03.050+0000 7fd64fa81740 1 Processor -- start 2024-03-08T21:09:03.036 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:03.051+0000 7fd64fa81740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562b25916e80 0x562b2591fe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:09:03.036 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:03.051+0000 7fd64fa81740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562b259177a0 0x562b2591d220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:09:03.036 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:03.051+0000 7fd64fa81740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b25920570 0x562b2591d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:09:03.036 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:03.051+0000 7fd64fa81740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b259231a0 con 0x562b25916e80 2024-03-08T21:09:03.036 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:03.051+0000 7fd64fa81740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b258369f0 con 0x562b259177a0 2024-03-08T21:09:03.036 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:03.051+0000 7fd64fa81740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b25923320 con 0x562b25920570 2024-03-08T21:09:03.036 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:03.051+0000 7fd64fa81740 10 client.0 _ll_register_callbacks cb 0x562b2558ea00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T21:09:03.036 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:03.051+0000 7fd645e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562b25916e80 0x562b2591fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:09:03.036 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:03.051+0000 7fd645e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562b25916e80 0x562b2591fe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:40380/0 (socket says 172.21.15.107:40380) 2024-03-08T21:09:03.036 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:03.051+0000 7fd645e00640 1 -- 172.21.15.107:0/2697951094 learned_addr learned my addr 172.21.15.107:0/2697951094 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:09:03.036 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:03.052+0000 7fd646800640 1 --2- 172.21.15.107:0/2697951094 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b25920570 0x562b2591d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:09:03.036 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:03.052+0000 7fd645400640 1 --2- 172.21.15.107:0/2697951094 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562b259177a0 0x562b2591d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:09:03.036 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:03.052+0000 7fd645e00640 1 -- 172.21.15.107:0/2697951094 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b25920570 msgr2=0x562b2591d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:09:03.037 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:03.052+0000 7fd645e00640 1 --2- 172.21.15.107:0/2697951094 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b25920570 0x562b2591d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:09:03.037 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:03.052+0000 7fd645e00640 1 -- 172.21.15.107:0/2697951094 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562b259177a0 msgr2=0x562b2591d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:09:03.037 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:03.052+0000 7fd645e00640 1 --2- 172.21.15.107:0/2697951094 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562b259177a0 0x562b2591d220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:09:03.037 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:03.052+0000 7fd645e00640 1 -- 172.21.15.107:0/2697951094 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562b2558f210 con 0x562b25916e80 2024-03-08T21:09:03.037 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:03.052+0000 7fd645400640 1 --2- 172.21.15.107:0/2697951094 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562b259177a0 0x562b2591d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T21:09:03.037 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:03.052+0000 7fd645e00640 1 --2- 172.21.15.107:0/2697951094 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562b25916e80 0x562b2591fe90 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fd638005d80 tx=0x7fd638003f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c87ab24f3cb955f7 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:09:03.037 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:03.052+0000 7fd627e00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-03-08T21:09:03.037 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:03.052+0000 7fd627e00640 1 -- 172.21.15.107:0/2697951094 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd638028030 con 0x562b25916e80 2024-03-08T21:09:03.037 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:03.053+0000 7fd627e00640 1 -- 172.21.15.107:0/2697951094 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fd638004000 con 0x562b25916e80 2024-03-08T21:09:03.037 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:03.053+0000 7fd64fa81740 1 -- 172.21.15.107:0/2697951094 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd6380035f0 con 0x562b25916e80 2024-03-08T21:09:03.037 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:03.053+0000 7fd627e00640 1 -- 172.21.15.107:0/2697951094 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd63801f040 con 0x562b25916e80 2024-03-08T21:09:03.037 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:03.053+0000 7fd64fa81740 10 client.16167 fetch_fsmap learned FSMap version 36 2024-03-08T21:09:03.037 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:03.053+0000 7fd64fa81740 1 -- 172.21.15.107:0/2697951094 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x562b2565b620 con 0x562b25916e80 2024-03-08T21:09:03.038 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:03.053+0000 7fd627e00640 1 -- 172.21.15.107:0/2697951094 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fd6380035f0 con 0x562b25916e80 2024-03-08T21:09:03.038 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:03.054+0000 7fd627e00640 1 -- 172.21.15.107:0/2697951094 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 36) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fd638024410 con 0x562b25916e80 2024-03-08T21:09:03.038 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:03.054+0000 7fd64fa81740 10 client.16167 fetch_fsmap finished waiting for FSMap version 36 2024-03-08T21:09:03.038 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:03.054+0000 7fd64fa81740 10 client.16167 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-08T21:09:03.038 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:03.054+0000 7fd64fa81740 20 client.16167 populate_metadata read hostname 'smithi107' 2024-03-08T21:09:03.038 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:03.054+0000 7fd64fa81740 1 --2- 172.21.15.107:0/2697951094 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x562b25930260 0x562b25932650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:09:03.038 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:03.054+0000 7fd64fa81740 4 client.16167 mds_command: new command op to 34475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-08T21:09:03.038 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:03.054+0000 7fd64fa81740 1 -- 172.21.15.107:0/2697951094 --> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562b25887910 con 0x562b25930260 2024-03-08T21:09:03.038 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:03.054+0000 7fd645400640 1 --2- 172.21.15.107:0/2697951094 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x562b25930260 0x562b25932650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:09:03.038 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:03.055+0000 7fd645400640 1 --2- 172.21.15.107:0/2697951094 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x562b25930260 0x562b25932650 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5952a8dc9e4f491b server_cookie=40e94860a3799693 in_seq=0 out_seq=0 2024-03-08T21:09:03.038 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:03.055+0000 7fd627e00640 10 client.16167 ms_handle_connect on v2:172.21.15.149:6838/55290596 2024-03-08T21:09:03.040 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:03.057+0000 7fd627e00640 1 -- 172.21.15.107:0/2697951094 <== mds.0 v2:172.21.15.149:6838/55290596 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x562b25887910 con 0x562b25930260 2024-03-08T21:09:03.040 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:03.057+0000 7fd627e00640 10 client.16167 handle_command_reply: tid=0 2024-03-08T21:09:03.040 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:03.057+0000 7fd64fa81740 1 client.16167 shutdown 2024-03-08T21:09:03.040 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:03.057+0000 7fd64fa81740 1 -- 172.21.15.107:0/2697951094 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562b25916e80 msgr2=0x562b2591fe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:09:03.040 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:03.057+0000 7fd64fa81740 1 --2- 172.21.15.107:0/2697951094 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562b25916e80 0x562b2591fe90 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fd638005d80 tx=0x7fd638003f00 comp rx=0 tx=0).stop 2024-03-08T21:09:03.040 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:03.057+0000 7fd64fa81740 1 -- 172.21.15.107:0/2697951094 shutdown_connections 2024-03-08T21:09:03.040 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:03.057+0000 7fd64fa81740 1 --2- 172.21.15.107:0/2697951094 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b25920570 0x562b2591d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:09:03.040 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:03.057+0000 7fd64fa81740 1 --2- 172.21.15.107:0/2697951094 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x562b25930260 0x562b25932650 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:09:03.040 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:03.057+0000 7fd64fa81740 1 --2- 172.21.15.107:0/2697951094 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562b259177a0 0x562b2591d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:09:03.041 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:03.057+0000 7fd627e00640 0 client.16167 ms_handle_reset on v2:172.21.15.149:6838/55290596 2024-03-08T21:09:03.041 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:03.057+0000 7fd64fa81740 1 --2- 172.21.15.107:0/2697951094 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562b25916e80 0x562b2591fe90 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:09:03.041 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:03.057+0000 7fd64fa81740 1 -- 172.21.15.107:0/2697951094 >> 172.21.15.107:0/2697951094 conn(0x562b259117a0 msgr2=0x562b259134f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:09:03.041 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:03.058+0000 7fd64fa81740 1 -- 172.21.15.107:0/2697951094 shutdown_connections 2024-03-08T21:09:03.041 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:03.058+0000 7fd64fa81740 1 -- 172.21.15.107:0/2697951094 wait complete. 2024-03-08T21:09:03.041 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:03.058+0000 7fd64fa81740 20 client.16167 trim_cache size 0 max 16384 2024-03-08T21:09:03.056 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:03.077+0000 7fd64fa81740 1 Processor -- start 2024-03-08T21:09:03.056 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:03.077+0000 7fd64fa81740 1 -- start start 2024-03-08T21:09:03.056 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:03.077+0000 7fd64fa81740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b25916e80 0x562b25910680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:09:03.056 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:03.077+0000 7fd64fa81740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562b259177a0 0x562b2591f950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:09:03.056 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:03.077+0000 7fd64fa81740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562b25920570 0x562b2591fe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:09:03.056 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:03.077+0000 7fd64fa81740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6481aee00 con 0x562b259177a0 2024-03-08T21:09:03.057 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:03.077+0000 7fd64fa81740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6481b0780 con 0x562b25920570 2024-03-08T21:09:03.057 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:03.077+0000 7fd64fa81740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6481af460 con 0x562b25916e80 2024-03-08T21:09:03.057 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:03.078+0000 7fd645400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562b259177a0 0x562b2591f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:09:03.057 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:03.078+0000 7fd645400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562b259177a0 0x562b2591f950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:40390/0 (socket says 172.21.15.107:40390) 2024-03-08T21:09:03.057 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:03.078+0000 7fd645400640 1 -- 172.21.15.107:0/674809251 learned_addr learned my addr 172.21.15.107:0/674809251 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:09:03.059 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:03.078+0000 7fd645e00640 1 --2- 172.21.15.107:0/674809251 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b25916e80 0x562b25910680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:09:03.059 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:03.078+0000 7fd645400640 1 -- 172.21.15.107:0/674809251 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b25916e80 msgr2=0x562b25910680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:09:03.059 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:03.078+0000 7fd645400640 1 --2- 172.21.15.107:0/674809251 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b25916e80 0x562b25910680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:09:03.059 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:03.078+0000 7fd645400640 1 -- 172.21.15.107:0/674809251 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562b25920570 msgr2=0x562b2591fe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:09:03.059 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:03.078+0000 7fd645400640 1 --2- 172.21.15.107:0/674809251 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562b25920570 0x562b2591fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:09:03.059 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:03.078+0000 7fd645400640 1 -- 172.21.15.107:0/674809251 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562b259232c0 con 0x562b259177a0 2024-03-08T21:09:03.059 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:03.079+0000 7fd645400640 1 --2- 172.21.15.107:0/674809251 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562b259177a0 0x562b2591f950 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fd648109b90 tx=0x7fd6340046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e11c29207b3b72b server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:09:03.059 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:03.079+0000 7fd632a00640 1 -- 172.21.15.107:0/674809251 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd63402a070 con 0x562b259177a0 2024-03-08T21:09:03.060 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:03.079+0000 7fd632a00640 1 -- 172.21.15.107:0/674809251 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fd634026bc0 con 0x562b259177a0 2024-03-08T21:09:03.060 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:03.079+0000 7fd632a00640 1 -- 172.21.15.107:0/674809251 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6340035f0 con 0x562b259177a0 2024-03-08T21:09:03.060 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:03.079+0000 7fd64fa81740 1 -- 172.21.15.107:0/674809251 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562b259177a0 msgr2=0x562b2591f950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:09:03.060 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:03.080+0000 7fd64fa81740 1 --2- 172.21.15.107:0/674809251 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562b259177a0 0x562b2591f950 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fd648109b90 tx=0x7fd6340046c0 comp rx=0 tx=0).stop 2024-03-08T21:09:03.060 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:03.080+0000 7fd64fa81740 1 -- 172.21.15.107:0/674809251 shutdown_connections 2024-03-08T21:09:03.060 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:03.080+0000 7fd64fa81740 1 --2- 172.21.15.107:0/674809251 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562b25920570 0x562b2591fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:09:03.060 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:03.080+0000 7fd64fa81740 1 --2- 172.21.15.107:0/674809251 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562b259177a0 0x562b2591f950 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:09:03.060 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:03.080+0000 7fd64fa81740 1 --2- 172.21.15.107:0/674809251 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b25916e80 0x562b25910680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:09:03.060 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:03.080+0000 7fd64fa81740 1 -- 172.21.15.107:0/674809251 >> 172.21.15.107:0/674809251 conn(0x562b259117a0 msgr2=0x562b2590ff70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:09:03.060 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:03.080+0000 7fd64fa81740 1 -- 172.21.15.107:0/674809251 shutdown_connections 2024-03-08T21:09:03.060 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:03.080+0000 7fd64fa81740 1 -- 172.21.15.107:0/674809251 wait complete. 2024-03-08T21:09:03.060 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:03.080+0000 7fd64fa81740 1 -- start start 2024-03-08T21:09:03.061 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:03.081+0000 7fd64fa81740 1 Processor -- start 2024-03-08T21:09:03.064 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:03.081+0000 7fd64fa81740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562b25916e80 0x562b259278d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:09:03.065 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:03.081+0000 7fd64fa81740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b259177a0 0x562b25925c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:09:03.065 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:03.081+0000 7fd64fa81740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562b25920570 0x562b259261b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:09:03.065 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:03.081+0000 7fd64fa81740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6481b0120 con 0x562b25916e80 2024-03-08T21:09:03.065 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:03.081+0000 7fd64fa81740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6481afac0 con 0x562b25920570 2024-03-08T21:09:03.065 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:03.081+0000 7fd64fa81740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6481ad0c0 con 0x562b259177a0 2024-03-08T21:09:03.065 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:03.081+0000 7fd64fa81740 10 client.0 _ll_register_callbacks cb 0x7fd638009670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T21:09:03.065 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:03.081+0000 7fd645e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562b25916e80 0x562b259278d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:09:03.065 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:03.081+0000 7fd645e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562b25916e80 0x562b259278d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:40402/0 (socket says 172.21.15.107:40402) 2024-03-08T21:09:03.066 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:03.081+0000 7fd645e00640 1 -- 172.21.15.107:0/3158955917 learned_addr learned my addr 172.21.15.107:0/3158955917 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:09:03.066 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:03.082+0000 7fd645400640 1 --2- 172.21.15.107:0/3158955917 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b259177a0 0x562b25925c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:09:03.066 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:03.082+0000 7fd645e00640 1 -- 172.21.15.107:0/3158955917 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b259177a0 msgr2=0x562b25925c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:09:03.066 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:03.082+0000 7fd645e00640 1 --2- 172.21.15.107:0/3158955917 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b259177a0 0x562b25925c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:09:03.066 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:03.082+0000 7fd645e00640 1 -- 172.21.15.107:0/3158955917 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562b25920570 msgr2=0x562b259261b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:09:03.066 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:03.082+0000 7fd645e00640 1 --2- 172.21.15.107:0/3158955917 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562b25920570 0x562b259261b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:09:03.066 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:03.082+0000 7fd645e00640 1 -- 172.21.15.107:0/3158955917 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562b259269e0 con 0x562b25916e80 2024-03-08T21:09:03.066 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:03.082+0000 7fd645e00640 1 --2- 172.21.15.107:0/3158955917 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562b25916e80 0x562b259278d0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fd638024d30 tx=0x7fd638009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ce3e8feba1c9878 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:09:03.066 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:03.082+0000 7fd627e00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-03-08T21:09:03.066 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:03.082+0000 7fd627e00640 1 -- 172.21.15.107:0/3158955917 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd63803b060 con 0x562b25916e80 2024-03-08T21:09:03.067 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:03.082+0000 7fd627e00640 1 -- 172.21.15.107:0/3158955917 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fd638024410 con 0x562b25916e80 2024-03-08T21:09:03.067 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:03.082+0000 7fd64fa81740 1 -- 172.21.15.107:0/3158955917 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd634026bc0 con 0x562b25916e80 2024-03-08T21:09:03.067 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:03.083+0000 7fd627e00640 1 -- 172.21.15.107:0/3158955917 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd638028040 con 0x562b25916e80 2024-03-08T21:09:03.067 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:03.083+0000 7fd64fa81740 10 client.16173 fetch_fsmap learned FSMap version 36 2024-03-08T21:09:03.067 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:03.083+0000 7fd64fa81740 1 -- 172.21.15.107:0/3158955917 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x562b25987440 con 0x562b25916e80 2024-03-08T21:09:03.067 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:03.083+0000 7fd627e00640 1 -- 172.21.15.107:0/3158955917 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fd634026bc0 con 0x562b25916e80 2024-03-08T21:09:03.067 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:03.083+0000 7fd627e00640 1 -- 172.21.15.107:0/3158955917 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 36) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fd638012a70 con 0x562b25916e80 2024-03-08T21:09:03.067 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:03.083+0000 7fd64fa81740 10 client.16173 fetch_fsmap finished waiting for FSMap version 36 2024-03-08T21:09:03.067 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:03.083+0000 7fd64fa81740 10 client.16173 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-08T21:09:03.067 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:03.083+0000 7fd64fa81740 20 client.16173 populate_metadata read hostname 'smithi107' 2024-03-08T21:09:03.067 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:03.083+0000 7fd64fa81740 1 --2- 172.21.15.107:0/3158955917 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x562b25987b40 0x562b25989f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:09:03.067 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:03.083+0000 7fd64fa81740 4 client.16173 mds_command: new command op to 34475 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-08T21:09:03.067 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:03.083+0000 7fd64fa81740 1 -- 172.21.15.107:0/3158955917 --> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x562b25881c60 con 0x562b25987b40 2024-03-08T21:09:03.068 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:03.084+0000 7fd645400640 1 --2- 172.21.15.107:0/3158955917 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x562b25987b40 0x562b25989f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:09:03.068 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:03.085+0000 7fd645400640 1 --2- 172.21.15.107:0/3158955917 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x562b25987b40 0x562b25989f60 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=71c415bb39cef7a4 server_cookie=e1d624eedcd0723a in_seq=0 out_seq=0 2024-03-08T21:09:03.068 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:03.085+0000 7fd627e00640 10 client.16173 ms_handle_connect on v2:172.21.15.149:6838/55290596 2024-03-08T21:09:03.124 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:09:02 smithi204 ceph-mon[34935]: pgmap v266: 97 pgs: 97 active+clean; 5.7 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:09:03.213 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:09:02 smithi107 ceph-mon[31381]: pgmap v266: 97 pgs: 97 active+clean; 5.7 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:09:04.180 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:04.201+0000 7fd627e00640 1 -- 172.21.15.107:0/3158955917 <== mds.0 v2:172.21.15.149:6838/55290596 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x562b25881c60 con 0x562b25987b40 2024-03-08T21:09:04.180 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:04.202+0000 7fd627e00640 10 client.16173 handle_command_reply: tid=0 2024-03-08T21:09:04.180 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:04.202+0000 7fd64fa81740 1 client.16173 shutdown 2024-03-08T21:09:04.180 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:04.202+0000 7fd64fa81740 1 -- 172.21.15.107:0/3158955917 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562b25916e80 msgr2=0x562b259278d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:09:04.180 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:04.202+0000 7fd64fa81740 1 --2- 172.21.15.107:0/3158955917 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562b25916e80 0x562b259278d0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fd638024d30 tx=0x7fd638009b20 comp rx=0 tx=0).stop 2024-03-08T21:09:04.181 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:04.202+0000 7fd64fa81740 1 -- 172.21.15.107:0/3158955917 shutdown_connections 2024-03-08T21:09:04.181 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:04.202+0000 7fd64fa81740 1 --2- 172.21.15.107:0/3158955917 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x562b25987b40 0x562b25989f60 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:09:04.181 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:04.202+0000 7fd64fa81740 1 --2- 172.21.15.107:0/3158955917 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562b25920570 0x562b259261b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:09:04.181 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:04.202+0000 7fd64fa81740 1 --2- 172.21.15.107:0/3158955917 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562b259177a0 0x562b25925c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:09:04.181 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:04.202+0000 7fd64fa81740 1 --2- 172.21.15.107:0/3158955917 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562b25916e80 0x562b259278d0 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:09:04.182 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:04.202+0000 7fd64fa81740 1 -- 172.21.15.107:0/3158955917 >> 172.21.15.107:0/3158955917 conn(0x562b259117a0 msgr2=0x562b25912bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:09:04.182 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:04.202+0000 7fd627e00640 0 client.16173 ms_handle_reset on v2:172.21.15.149:6838/55290596 2024-03-08T21:09:04.182 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:04.203+0000 7fd64fa81740 1 -- 172.21.15.107:0/3158955917 shutdown_connections 2024-03-08T21:09:04.182 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:04.203+0000 7fd64fa81740 1 -- 172.21.15.107:0/3158955917 wait complete. 2024-03-08T21:09:04.182 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:04.203+0000 7fd64fa81740 20 client.16173 trim_cache size 0 max 16384 2024-03-08T21:09:04.182 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:{ 2024-03-08T21:09:04.182 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "message": "", 2024-03-08T21:09:04.182 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "return_code": 0 2024-03-08T21:09:04.182 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:} 2024-03-08T21:09:04.183 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:04.204+0000 7fd647200640 1 -- 172.21.15.107:0/1504724516 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7fd62c0a29e0 msgr2=0x7fd62c0a4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:09:04.183 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:04.204+0000 7fd647200640 1 --2- 172.21.15.107:0/1504724516 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7fd62c0a29e0 0x7fd62c0a4ea0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x562b25912bc0 tx=0x7fd634014040 comp rx=0 tx=0).stop 2024-03-08T21:09:04.183 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:04.204+0000 7fd647200640 1 -- 172.21.15.107:0/1504724516 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd6481999c0 msgr2=0x7fd648199e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:09:04.183 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:04.204+0000 7fd647200640 1 --2- 172.21.15.107:0/1504724516 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd6481999c0 0x7fd648199e70 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fd64002c700 tx=0x7fd640035d60 comp rx=0 tx=0).stop 2024-03-08T21:09:04.183 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:04.204+0000 7fd647200640 1 -- 172.21.15.107:0/1504724516 shutdown_connections 2024-03-08T21:09:04.183 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:04.204+0000 7fd647200640 1 --2- 172.21.15.107:0/1504724516 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7fd62c0a29e0 0x7fd62c0a4ea0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:09:04.183 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:04.204+0000 7fd647200640 1 --2- 172.21.15.107:0/1504724516 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd6481999c0 0x7fd648199e70 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:09:04.183 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:04.204+0000 7fd647200640 1 --2- 172.21.15.107:0/1504724516 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd648108620 0x7fd64819fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:09:04.184 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:04.204+0000 7fd647200640 1 --2- 172.21.15.107:0/1504724516 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd648103550 0x7fd64819f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:09:04.184 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:04.204+0000 7fd647200640 1 -- 172.21.15.107:0/1504724516 >> 172.21.15.107:0/1504724516 conn(0x7fd6480fedd0 msgr2=0x7fd648101130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:09:04.184 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:04.205+0000 7fd647200640 1 -- 172.21.15.107:0/1504724516 shutdown_connections 2024-03-08T21:09:04.184 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:04.205+0000 7fd647200640 1 -- 172.21.15.107:0/1504724516 wait complete. 2024-03-08T21:09:05.511 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:09:05 smithi149 ceph-mon[35042]: osdmap e98: 12 total, 12 up, 12 in 2024-03-08T21:09:05.511 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:09:05 smithi149 ceph-mon[35042]: pgmap v268: 97 pgs: 97 active+clean; 6.0 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 81 KiB/s wr, 33 op/s 2024-03-08T21:09:05.604 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:09:05 smithi204 ceph-mon[34935]: osdmap e98: 12 total, 12 up, 12 in 2024-03-08T21:09:05.604 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:09:05 smithi204 ceph-mon[34935]: pgmap v268: 97 pgs: 97 active+clean; 6.0 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 81 KiB/s wr, 33 op/s 2024-03-08T21:09:05.605 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:09:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[71122]: *** Caught signal (Segmentation fault) ** 2024-03-08T21:09:05.605 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:09:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[71122]: in thread 7f4033200700 thread_name:md_log_replay 2024-03-08T21:09:05.605 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:09:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[71122]: ceph version 19.0.0-1965-g7d63482a (7d63482aa89628d733e298984a762389b625c761) squid (dev) 2024-03-08T21:09:05.605 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:09:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[71122]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f4047815d20] 2024-03-08T21:09:05.605 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:09:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[71122]: 2: /usr/bin/ceph-mds(+0x503eff) [0x558bb0407eff] 2024-03-08T21:09:05.605 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:09:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[71122]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x558bb040b773] 2024-03-08T21:09:05.605 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:09:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[71122]: 4: (MDLog::_replay_thread()+0x617) [0x558bb0467587] 2024-03-08T21:09:05.605 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:09:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[71122]: 5: (MDLog::ReplayThread::entry()+0x11) [0x558bb0470091] 2024-03-08T21:09:05.606 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:09:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[71122]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7f404780b1ca] 2024-03-08T21:09:05.606 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:09:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[71122]: 7: clone() 2024-03-08T21:09:05.606 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:09:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[71122]: 2024-03-08T21:09:05.169+0000 7f4033200700 -1 *** Caught signal (Segmentation fault) ** 2024-03-08T21:09:05.606 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:09:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[71122]: in thread 7f4033200700 thread_name:md_log_replay 2024-03-08T21:09:05.606 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:09:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[71122]: 2024-03-08T21:09:05.606 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:09:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[71122]: ceph version 19.0.0-1965-g7d63482a (7d63482aa89628d733e298984a762389b625c761) squid (dev) 2024-03-08T21:09:05.606 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:09:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[71122]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f4047815d20] 2024-03-08T21:09:05.606 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:09:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[71122]: 2: /usr/bin/ceph-mds(+0x503eff) [0x558bb0407eff] 2024-03-08T21:09:05.606 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:09:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[71122]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x558bb040b773] 2024-03-08T21:09:05.606 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:09:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[71122]: 4: (MDLog::_replay_thread()+0x617) [0x558bb0467587] 2024-03-08T21:09:05.606 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:09:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[71122]: 5: (MDLog::ReplayThread::entry()+0x11) [0x558bb0470091] 2024-03-08T21:09:05.606 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:09:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[71122]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7f404780b1ca] 2024-03-08T21:09:05.606 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:09:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[71122]: 7: clone() 2024-03-08T21:09:05.607 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:09:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[71122]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-08T21:09:05.607 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:09:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[71122]: 2024-03-08T21:09:05.607 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:09:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[71122]: 0> 2024-03-08T21:09:05.169+0000 7f4033200700 -1 *** Caught signal (Segmentation fault) ** 2024-03-08T21:09:05.607 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:09:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[71122]: in thread 7f4033200700 thread_name:md_log_replay 2024-03-08T21:09:05.607 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:09:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[71122]: 2024-03-08T21:09:05.607 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:09:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[71122]: ceph version 19.0.0-1965-g7d63482a (7d63482aa89628d733e298984a762389b625c761) squid (dev) 2024-03-08T21:09:05.607 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:09:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[71122]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f4047815d20] 2024-03-08T21:09:05.607 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:09:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[71122]: 2: /usr/bin/ceph-mds(+0x503eff) [0x558bb0407eff] 2024-03-08T21:09:05.607 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:09:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[71122]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x558bb040b773] 2024-03-08T21:09:05.607 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:09:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[71122]: 4: (MDLog::_replay_thread()+0x617) [0x558bb0467587] 2024-03-08T21:09:05.608 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:09:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[71122]: 5: (MDLog::ReplayThread::entry()+0x11) [0x558bb0470091] 2024-03-08T21:09:05.608 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:09:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[71122]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7f404780b1ca] 2024-03-08T21:09:05.608 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:09:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[71122]: 7: clone() 2024-03-08T21:09:05.608 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:09:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[71122]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-08T21:09:05.608 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:09:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[71122]: 2024-03-08T21:09:05.608 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:09:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[71122]: 0> 2024-03-08T21:09:05.169+0000 7f4033200700 -1 *** Caught signal (Segmentation fault) ** 2024-03-08T21:09:05.608 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:09:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[71122]: in thread 7f4033200700 thread_name:md_log_replay 2024-03-08T21:09:05.608 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:09:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[71122]: 2024-03-08T21:09:05.608 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:09:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[71122]: ceph version 19.0.0-1965-g7d63482a (7d63482aa89628d733e298984a762389b625c761) squid (dev) 2024-03-08T21:09:05.608 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:09:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[71122]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f4047815d20] 2024-03-08T21:09:05.608 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:09:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[71122]: 2: /usr/bin/ceph-mds(+0x503eff) [0x558bb0407eff] 2024-03-08T21:09:05.608 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:09:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[71122]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x558bb040b773] 2024-03-08T21:09:05.608 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:09:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[71122]: 4: (MDLog::_replay_thread()+0x617) [0x558bb0467587] 2024-03-08T21:09:05.609 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:09:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[71122]: 5: (MDLog::ReplayThread::entry()+0x11) [0x558bb0470091] 2024-03-08T21:09:05.609 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:09:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[71122]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7f404780b1ca] 2024-03-08T21:09:05.609 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:09:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[71122]: 7: clone() 2024-03-08T21:09:05.609 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:09:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[71122]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-08T21:09:05.609 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:09:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[71122]: 2024-03-08T21:09:05.609 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:09:05 smithi204 podman[71444]: 2024-03-08 21:09:05.274567468 +0000 UTC m=+0.035466543 container died ac85e479b71fcf12a3e9715687d6094abecdd1de10db12aa559758bfc17e216a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=rishabh-2024mar8-7d63482, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-03-08T21:09:05.712 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:09:05 smithi107 ceph-mon[31381]: osdmap e98: 12 total, 12 up, 12 in 2024-03-08T21:09:05.713 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:09:05 smithi107 ceph-mon[31381]: pgmap v268: 97 pgs: 97 active+clean; 6.0 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 81 KiB/s wr, 33 op/s 2024-03-08T21:09:06.373 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:09:06 smithi204 podman[71444]: 2024-03-08 21:09:06.105624783 +0000 UTC m=+0.866523895 container remove ac85e479b71fcf12a3e9715687d6094abecdd1de10db12aa559758bfc17e216a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=rishabh-2024mar8-7d63482, io.buildah.version=1.33.5, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-08T21:09:06.374 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:09:06 smithi204 systemd[1]: ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715@mds.c.service: Main process exited, code=exited, status=139/n/a 2024-03-08T21:09:06.746 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:09:06 smithi204 systemd[1]: ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715@mds.c.service: Failed with result 'exit-code'. 2024-03-08T21:09:06.746 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:09:06 smithi204 systemd[1]: ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715@mds.c.service: Consumed 1.828s CPU time. 2024-03-08T21:09:07.010 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:09:06 smithi149 ceph-mon[35042]: pgmap v269: 97 pgs: 97 active+clean; 6.0 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 81 KiB/s wr, 33 op/s 2024-03-08T21:09:07.124 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:09:06 smithi204 ceph-mon[34935]: pgmap v269: 97 pgs: 97 active+clean; 6.0 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 81 KiB/s wr, 33 op/s 2024-03-08T21:09:07.213 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:09:06 smithi107 ceph-mon[31381]: pgmap v269: 97 pgs: 97 active+clean; 6.0 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 81 KiB/s wr, 33 op/s 2024-03-08T21:09:09.010 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:09:08 smithi149 ceph-mon[35042]: pgmap v270: 97 pgs: 97 active+clean; 6.0 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 82 KiB/s wr, 33 op/s 2024-03-08T21:09:09.124 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:09:08 smithi204 ceph-mon[34935]: pgmap v270: 97 pgs: 97 active+clean; 6.0 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 82 KiB/s wr, 33 op/s 2024-03-08T21:09:09.213 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:09:08 smithi107 ceph-mon[31381]: pgmap v270: 97 pgs: 97 active+clean; 6.0 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 82 KiB/s wr, 33 op/s 2024-03-08T21:09:10.463 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:09:10 smithi107 ceph-mon[31381]: osdmap e99: 12 total, 12 up, 12 in 2024-03-08T21:09:10.510 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:09:10 smithi149 ceph-mon[35042]: osdmap e99: 12 total, 12 up, 12 in 2024-03-08T21:09:10.624 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:09:10 smithi204 ceph-mon[34935]: osdmap e99: 12 total, 12 up, 12 in 2024-03-08T21:09:11.463 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:09:11 smithi107 ceph-mon[31381]: pgmap v272: 97 pgs: 97 active+clean; 6.0 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 102 KiB/s wr, 41 op/s 2024-03-08T21:09:11.510 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:09:11 smithi149 ceph-mon[35042]: pgmap v272: 97 pgs: 97 active+clean; 6.0 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 102 KiB/s wr, 41 op/s 2024-03-08T21:09:11.624 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:09:11 smithi204 ceph-mon[34935]: pgmap v272: 97 pgs: 97 active+clean; 6.0 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 102 KiB/s wr, 41 op/s 2024-03-08T21:09:13.010 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:09:12 smithi149 ceph-mon[35042]: pgmap v273: 97 pgs: 97 active+clean; 6.0 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 363 B/s wr, 5 op/s 2024-03-08T21:09:13.124 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:09:12 smithi204 ceph-mon[34935]: pgmap v273: 97 pgs: 97 active+clean; 6.0 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 363 B/s wr, 5 op/s 2024-03-08T21:09:13.213 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:09:12 smithi107 ceph-mon[31381]: pgmap v273: 97 pgs: 97 active+clean; 6.0 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 363 B/s wr, 5 op/s 2024-03-08T21:09:15.010 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:09:14 smithi149 ceph-mon[35042]: pgmap v274: 97 pgs: 97 active+clean; 6.0 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 409 B/s wr, 8 op/s 2024-03-08T21:09:15.123 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:09:14 smithi204 ceph-mon[34935]: pgmap v274: 97 pgs: 97 active+clean; 6.0 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 409 B/s wr, 8 op/s 2024-03-08T21:09:15.213 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:09:14 smithi107 ceph-mon[31381]: pgmap v274: 97 pgs: 97 active+clean; 6.0 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 409 B/s wr, 8 op/s 2024-03-08T21:09:17.010 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:09:16 smithi149 ceph-mon[35042]: pgmap v275: 97 pgs: 97 active+clean; 6.0 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 409 B/s wr, 8 op/s 2024-03-08T21:09:17.014 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:09:16 smithi107 ceph-mon[31381]: pgmap v275: 97 pgs: 97 active+clean; 6.0 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 409 B/s wr, 8 op/s 2024-03-08T21:09:17.018 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:09:16 smithi204 systemd[1]: ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715@mds.c.service: Scheduled restart job, restart counter is at 5. 2024-03-08T21:09:17.018 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:09:16 smithi204 systemd[1]: Stopped Ceph mds.c for dc20f0f8-dd8c-11ee-95c6-87774f69a715. 2024-03-08T21:09:17.019 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:09:16 smithi204 systemd[1]: ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715@mds.c.service: Consumed 1.828s CPU time. 2024-03-08T21:09:17.019 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:09:16 smithi204 systemd[1]: Starting Ceph mds.c for dc20f0f8-dd8c-11ee-95c6-87774f69a715... 2024-03-08T21:09:17.019 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:09:16 smithi204 ceph-mon[34935]: pgmap v275: 97 pgs: 97 active+clean; 6.0 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 409 B/s wr, 8 op/s 2024-03-08T21:09:17.333 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:09:17 smithi204 podman[71537]: 2024-03-08 21:09:16.946642096 +0000 UTC m=+0.034747077 image pull a272146cb810b1fd4a53627f019472132e30966d5c25b01bfc44a001295f1249 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd 2024-03-08T21:09:17.333 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:09:17 smithi204 podman[71537]: 2024-03-08 21:09:17.073410437 +0000 UTC m=+0.161515451 container create e8449560c340b0836ff42cc1649f3bb380947f5beb57f560f203f985e87ef28f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c, RELEASE=rishabh-2024mar8-7d63482, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-03-08T21:09:17.342 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.363+0000 7f76e5600640 1 -- 172.21.15.107:0/238171138 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f76e0101360 msgr2=0x7f76e0101740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:09:17.342 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.363+0000 7f76e5600640 1 --2- 172.21.15.107:0/238171138 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f76e0101360 0x7f76e0101740 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f76d0009900 tx=0x7f76d002bc30 comp rx=0 tx=0).stop 2024-03-08T21:09:17.342 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.363+0000 7f76e5600640 1 -- 172.21.15.107:0/238171138 shutdown_connections 2024-03-08T21:09:17.342 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.363+0000 7f76e5600640 1 --2- 172.21.15.107:0/238171138 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f76e0106430 0x7f76e010e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:09:17.342 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.363+0000 7f76e5600640 1 --2- 172.21.15.107:0/238171138 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f76e0101d10 0x7f76e0105d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:09:17.342 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.363+0000 7f76e5600640 1 --2- 172.21.15.107:0/238171138 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f76e0101360 0x7f76e0101740 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:09:17.342 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.363+0000 7f76e5600640 1 -- 172.21.15.107:0/238171138 >> 172.21.15.107:0/238171138 conn(0x7f76e00fcc00 msgr2=0x7f76e00ff020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:09:17.343 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.363+0000 7f76e5600640 1 -- 172.21.15.107:0/238171138 shutdown_connections 2024-03-08T21:09:17.343 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.363+0000 7f76e5600640 1 -- 172.21.15.107:0/238171138 wait complete. 2024-03-08T21:09:17.343 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.364+0000 7f76e5600640 1 Processor -- start 2024-03-08T21:09:17.343 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.364+0000 7f76e5600640 1 -- start start 2024-03-08T21:09:17.344 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.365+0000 7f76e5600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f76e0101360 0x7f76e019d610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:09:17.344 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.365+0000 7f76e5600640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f76e0106430 0x7f76e019db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:09:17.344 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.365+0000 7f76e5600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f76e0197780 0x7f76e0197c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:09:17.344 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.365+0000 7f76e5600640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76e01113a0 con 0x7f76e0101360 2024-03-08T21:09:17.345 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.365+0000 7f76e5600640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76e0111220 con 0x7f76e0106430 2024-03-08T21:09:17.345 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.365+0000 7f76e5600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76e0111520 con 0x7f76e0197780 2024-03-08T21:09:17.345 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.365+0000 7f76dfe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f76e0101360 0x7f76e019d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:09:17.345 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.365+0000 7f76dfe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f76e0101360 0x7f76e019d610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:57514/0 (socket says 172.21.15.107:57514) 2024-03-08T21:09:17.345 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.365+0000 7f76dfe00640 1 -- 172.21.15.107:0/2050497024 learned_addr learned my addr 172.21.15.107:0/2050497024 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:09:17.345 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.365+0000 7f76df400640 1 --2- 172.21.15.107:0/2050497024 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f76e0106430 0x7f76e019db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:09:17.345 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.365+0000 7f76e4c00640 1 --2- 172.21.15.107:0/2050497024 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f76e0197780 0x7f76e0197c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:09:17.345 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.365+0000 7f76dfe00640 1 -- 172.21.15.107:0/2050497024 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f76e0197780 msgr2=0x7f76e0197c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:09:17.345 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.365+0000 7f76dfe00640 1 --2- 172.21.15.107:0/2050497024 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f76e0197780 0x7f76e0197c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:09:17.345 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.365+0000 7f76dfe00640 1 -- 172.21.15.107:0/2050497024 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f76e0106430 msgr2=0x7f76e019db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:09:17.345 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.365+0000 7f76dfe00640 1 --2- 172.21.15.107:0/2050497024 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f76e0106430 0x7f76e019db50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:09:17.345 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.365+0000 7f76dfe00640 1 -- 172.21.15.107:0/2050497024 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76e0198310 con 0x7f76e0101360 2024-03-08T21:09:17.345 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.365+0000 7f76df400640 1 --2- 172.21.15.107:0/2050497024 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f76e0106430 0x7f76e019db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T21:09:17.346 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.366+0000 7f76dfe00640 1 --2- 172.21.15.107:0/2050497024 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f76e0101360 0x7f76e019d610 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f76d0005940 tx=0x7f76d002c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:09:17.346 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.366+0000 7f76dcc00640 1 -- 172.21.15.107:0/2050497024 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76d0043020 con 0x7f76e0101360 2024-03-08T21:09:17.346 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.366+0000 7f76e5600640 1 -- 172.21.15.107:0/2050497024 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76e01985a0 con 0x7f76e0101360 2024-03-08T21:09:17.346 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.366+0000 7f76dcc00640 1 -- 172.21.15.107:0/2050497024 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f76d002cbe0 con 0x7f76e0101360 2024-03-08T21:09:17.347 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.366+0000 7f76dcc00640 1 -- 172.21.15.107:0/2050497024 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76d0003bf0 con 0x7f76e0101360 2024-03-08T21:09:17.347 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.366+0000 7f76e5600640 1 -- 172.21.15.107:0/2050497024 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76e01a4780 con 0x7f76e0101360 2024-03-08T21:09:17.348 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.370+0000 7f76dcc00640 1 -- 172.21.15.107:0/2050497024 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) v1 ==== 137422+0+0 (secure 0 0 0) 0x7f76d0046020 con 0x7f76e0101360 2024-03-08T21:09:17.349 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.370+0000 7f76e99be740 1 Processor -- start 2024-03-08T21:09:17.349 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.370+0000 7f76e99be740 1 -- start start 2024-03-08T21:09:17.351 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.371+0000 7f76e99be740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fb95a0be80 0x55fb95a0c260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:09:17.351 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.371+0000 7f76e99be740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fb95a0c7a0 0x55fb95a15030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:09:17.351 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.371+0000 7f76e99be740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fb95a15570 0x55fb95a17960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:09:17.351 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.371+0000 7f76e99be740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb9596f3d0 con 0x55fb95a0be80 2024-03-08T21:09:17.351 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.371+0000 7f76e99be740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb959186b0 con 0x55fb95a0c7a0 2024-03-08T21:09:17.351 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.371+0000 7f76e99be740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb9596b470 con 0x55fb95a15570 2024-03-08T21:09:17.351 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.371+0000 7f76dcc00640 1 --2- 172.21.15.107:0/2050497024 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f76c40a2960 0x7f76c40a4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:09:17.351 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.371+0000 7f76e4c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fb95a0be80 0x55fb95a0c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:09:17.351 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.371+0000 7f76dcc00640 1 -- 172.21.15.107:0/2050497024 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f76d00f1070 con 0x7f76e0101360 2024-03-08T21:09:17.351 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.371+0000 7f76e4c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fb95a0be80 0x55fb95a0c260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:57516/0 (socket says 172.21.15.107:57516) 2024-03-08T21:09:17.351 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.371+0000 7f76e4c00640 1 -- 172.21.15.107:0/168856543 learned_addr learned my addr 172.21.15.107:0/168856543 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:09:17.351 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.372+0000 7f76e4c00640 1 --2- 172.21.15.107:0/2050497024 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f76c40a2960 0x7f76c40a4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:09:17.351 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.372+0000 7f76dfe00640 1 --2- 172.21.15.107:0/168856543 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fb95a0c7a0 0x55fb95a15030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:09:17.352 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.372+0000 7f76df400640 1 --2- 172.21.15.107:0/168856543 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fb95a15570 0x55fb95a17960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:09:17.353 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.372+0000 7f76e4c00640 1 -- 172.21.15.107:0/168856543 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fb95a15570 msgr2=0x55fb95a17960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:09:17.353 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.372+0000 7f76e4c00640 1 --2- 172.21.15.107:0/168856543 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fb95a15570 0x55fb95a17960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:09:17.354 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.372+0000 7f76e4c00640 1 -- 172.21.15.107:0/168856543 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fb95a0c7a0 msgr2=0x55fb95a15030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:09:17.354 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.372+0000 7f76e4c00640 1 --2- 172.21.15.107:0/168856543 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fb95a0c7a0 0x55fb95a15030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:09:17.354 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.372+0000 7f76e4c00640 1 -- 172.21.15.107:0/168856543 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fb9593eb50 con 0x55fb95a0be80 2024-03-08T21:09:17.354 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.372+0000 7f76e4c00640 1 --2- 172.21.15.107:0/168856543 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fb95a0be80 0x55fb95a0c260 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f76d8014cf0 tx=0x7f76d8011750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=957aeac01baa805f server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:09:17.354 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.373+0000 7f76e4c00640 1 --2- 172.21.15.107:0/2050497024 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f76c40a2960 0x7f76c40a4e20 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f76d8009fd0 tx=0x7f76d8016000 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:09:17.354 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.373+0000 7f76d5200640 1 -- 172.21.15.107:0/168856543 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76d8009ab0 con 0x55fb95a0be80 2024-03-08T21:09:17.354 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.373+0000 7f76d5200640 1 -- 172.21.15.107:0/168856543 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f76d8014640 con 0x55fb95a0be80 2024-03-08T21:09:17.354 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.373+0000 7f76d5200640 1 -- 172.21.15.107:0/168856543 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76d8005020 con 0x55fb95a0be80 2024-03-08T21:09:17.354 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.374+0000 7f76e99be740 1 -- 172.21.15.107:0/168856543 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fb95a0be80 msgr2=0x55fb95a0c260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:09:17.354 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.374+0000 7f76e99be740 1 --2- 172.21.15.107:0/168856543 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fb95a0be80 0x55fb95a0c260 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f76d8014cf0 tx=0x7f76d8011750 comp rx=0 tx=0).stop 2024-03-08T21:09:17.354 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.374+0000 7f76e99be740 1 -- 172.21.15.107:0/168856543 shutdown_connections 2024-03-08T21:09:17.354 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.374+0000 7f76e99be740 1 --2- 172.21.15.107:0/168856543 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fb95a15570 0x55fb95a17960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:09:17.354 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.374+0000 7f76e99be740 1 --2- 172.21.15.107:0/168856543 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fb95a0c7a0 0x55fb95a15030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:09:17.355 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.374+0000 7f76e99be740 1 --2- 172.21.15.107:0/168856543 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fb95a0be80 0x55fb95a0c260 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:09:17.355 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.374+0000 7f76e99be740 1 -- 172.21.15.107:0/168856543 >> 172.21.15.107:0/168856543 conn(0x55fb95a067a0 msgr2=0x55fb95a06b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:09:17.355 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.374+0000 7f76e99be740 1 -- 172.21.15.107:0/168856543 shutdown_connections 2024-03-08T21:09:17.355 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.374+0000 7f76e99be740 1 -- 172.21.15.107:0/168856543 wait complete. 2024-03-08T21:09:17.355 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.374+0000 7f76e99be740 1 -- start start 2024-03-08T21:09:17.355 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.375+0000 7f76e99be740 1 Processor -- start 2024-03-08T21:09:17.357 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.375+0000 7f76e99be740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fb95a0be80 0x55fb95a14fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:09:17.358 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.375+0000 7f76e99be740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fb95a0c7a0 0x55fb95a12370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:09:17.358 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.375+0000 7f76e99be740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fb95a15570 0x55fb95a12ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:09:17.358 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.375+0000 7f76e99be740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb95a18140 con 0x55fb95a15570 2024-03-08T21:09:17.358 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.375+0000 7f76e99be740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb9592b9f0 con 0x55fb95a0be80 2024-03-08T21:09:17.358 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.375+0000 7f76e99be740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb95a182c0 con 0x55fb95a0c7a0 2024-03-08T21:09:17.358 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.375+0000 7f76e99be740 10 client.0 _ll_register_callbacks cb 0x55fb95683a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T21:09:17.358 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.375+0000 7f76e4c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fb95a15570 0x55fb95a12ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:09:17.358 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.375+0000 7f76e4c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fb95a15570 0x55fb95a12ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:57518/0 (socket says 172.21.15.107:57518) 2024-03-08T21:09:17.358 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.375+0000 7f76e4c00640 1 -- 172.21.15.107:0/2291459203 learned_addr learned my addr 172.21.15.107:0/2291459203 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:09:17.358 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.376+0000 7f76df400640 1 --2- 172.21.15.107:0/2291459203 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fb95a0c7a0 0x55fb95a12370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:09:17.358 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.376+0000 7f76dfe00640 1 --2- 172.21.15.107:0/2291459203 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fb95a0be80 0x55fb95a14fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:09:17.359 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.376+0000 7f76e4c00640 1 -- 172.21.15.107:0/2291459203 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fb95a0c7a0 msgr2=0x55fb95a12370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:09:17.359 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.376+0000 7f76e4c00640 1 --2- 172.21.15.107:0/2291459203 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fb95a0c7a0 0x55fb95a12370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:09:17.359 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.376+0000 7f76e4c00640 1 -- 172.21.15.107:0/2291459203 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fb95a0be80 msgr2=0x55fb95a14fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:09:17.359 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.376+0000 7f76e4c00640 1 --2- 172.21.15.107:0/2291459203 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fb95a0be80 0x55fb95a14fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:09:17.359 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.376+0000 7f76e4c00640 1 -- 172.21.15.107:0/2291459203 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fb95684210 con 0x55fb95a15570 2024-03-08T21:09:17.359 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.376+0000 7f76e4c00640 1 --2- 172.21.15.107:0/2291459203 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fb95a15570 0x55fb95a12ab0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f76d8014830 tx=0x7f76d8014400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3197b8d4f99e44e server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:09:17.359 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.376+0000 7f76c9600640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-03-08T21:09:17.359 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.377+0000 7f76c9600640 1 -- 172.21.15.107:0/2291459203 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76d800cd50 con 0x55fb95a15570 2024-03-08T21:09:17.359 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.377+0000 7f76c9600640 1 -- 172.21.15.107:0/2291459203 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f76d8003970 con 0x55fb95a15570 2024-03-08T21:09:17.359 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.377+0000 7f76e99be740 1 -- 172.21.15.107:0/2291459203 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f76d8014640 con 0x55fb95a15570 2024-03-08T21:09:17.359 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.377+0000 7f76c9600640 1 -- 172.21.15.107:0/2291459203 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f76d8003b70 con 0x55fb95a15570 2024-03-08T21:09:17.359 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.377+0000 7f76e99be740 10 client.16185 fetch_fsmap learned FSMap version 36 2024-03-08T21:09:17.359 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.377+0000 7f76e99be740 1 -- 172.21.15.107:0/2291459203 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x55fb95750620 con 0x55fb95a15570 2024-03-08T21:09:17.359 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.377+0000 7f76c9600640 1 -- 172.21.15.107:0/2291459203 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f76d8014640 con 0x55fb95a15570 2024-03-08T21:09:17.360 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.377+0000 7f76c9600640 1 -- 172.21.15.107:0/2291459203 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 36) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f76d801d070 con 0x55fb95a15570 2024-03-08T21:09:17.360 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.378+0000 7f76e99be740 10 client.16185 fetch_fsmap finished waiting for FSMap version 36 2024-03-08T21:09:17.360 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.378+0000 7f76e99be740 10 client.16185 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-08T21:09:17.360 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.378+0000 7f76e99be740 20 client.16185 populate_metadata read hostname 'smithi107' 2024-03-08T21:09:17.360 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.378+0000 7f76e99be740 1 --2- 172.21.15.107:0/2291459203 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55fb95a25480 0x55fb95a27870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:09:17.360 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.378+0000 7f76e99be740 4 client.16185 mds_command: new command op to 34475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-08T21:09:17.360 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.378+0000 7f76e99be740 1 -- 172.21.15.107:0/2291459203 --> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fb9597c910 con 0x55fb95a25480 2024-03-08T21:09:17.360 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.378+0000 7f76dfe00640 1 --2- 172.21.15.107:0/2291459203 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55fb95a25480 0x55fb95a27870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:09:17.361 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.379+0000 7f76dfe00640 1 --2- 172.21.15.107:0/2291459203 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55fb95a25480 0x55fb95a27870 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5a23f36973f4bc0e server_cookie=a469efa6bc442f77 in_seq=0 out_seq=0 2024-03-08T21:09:17.361 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.379+0000 7f76c9600640 10 client.16185 ms_handle_connect on v2:172.21.15.149:6838/55290596 2024-03-08T21:09:17.361 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.381+0000 7f76c9600640 1 -- 172.21.15.107:0/2291459203 <== mds.0 v2:172.21.15.149:6838/55290596 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55fb9597c910 con 0x55fb95a25480 2024-03-08T21:09:17.361 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.381+0000 7f76c9600640 10 client.16185 handle_command_reply: tid=0 2024-03-08T21:09:17.362 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.381+0000 7f76e99be740 1 client.16185 shutdown 2024-03-08T21:09:17.362 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.381+0000 7f76e99be740 1 -- 172.21.15.107:0/2291459203 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fb95a15570 msgr2=0x55fb95a12ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:09:17.362 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.381+0000 7f76e99be740 1 --2- 172.21.15.107:0/2291459203 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fb95a15570 0x55fb95a12ab0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f76d8014830 tx=0x7f76d8014400 comp rx=0 tx=0).stop 2024-03-08T21:09:17.362 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.381+0000 7f76e99be740 1 -- 172.21.15.107:0/2291459203 shutdown_connections 2024-03-08T21:09:17.362 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.381+0000 7f76e99be740 1 --2- 172.21.15.107:0/2291459203 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fb95a15570 0x55fb95a12ab0 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:09:17.362 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.381+0000 7f76e99be740 1 --2- 172.21.15.107:0/2291459203 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fb95a0c7a0 0x55fb95a12370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:09:17.362 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.381+0000 7f76e99be740 1 --2- 172.21.15.107:0/2291459203 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55fb95a25480 0x55fb95a27870 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:09:17.362 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.381+0000 7f76e99be740 1 --2- 172.21.15.107:0/2291459203 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fb95a0be80 0x55fb95a14fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:09:17.362 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.381+0000 7f76c9600640 0 client.16185 ms_handle_reset on v2:172.21.15.149:6838/55290596 2024-03-08T21:09:17.362 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.381+0000 7f76e99be740 1 -- 172.21.15.107:0/2291459203 >> 172.21.15.107:0/2291459203 conn(0x55fb95a067a0 msgr2=0x55fb95a084f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:09:17.362 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.381+0000 7f76e99be740 1 -- 172.21.15.107:0/2291459203 shutdown_connections 2024-03-08T21:09:17.362 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.382+0000 7f76e99be740 1 -- 172.21.15.107:0/2291459203 wait complete. 2024-03-08T21:09:17.362 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.382+0000 7f76e99be740 20 client.16185 trim_cache size 0 max 16384 2024-03-08T21:09:17.379 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.400+0000 7f76e99be740 1 Processor -- start 2024-03-08T21:09:17.379 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.400+0000 7f76e99be740 1 -- start start 2024-03-08T21:09:17.381 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.401+0000 7f76e99be740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fb95a0be80 0x55fb95a05680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:09:17.381 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.401+0000 7f76e99be740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fb95a0c7a0 0x55fb95a1c500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:09:17.381 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.401+0000 7f76e99be740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fb95a15570 0x55fb95a1ca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:09:17.381 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.401+0000 7f76e99be740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76e01acc30 con 0x55fb95a15570 2024-03-08T21:09:17.381 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.401+0000 7f76e99be740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76e01ae5b0 con 0x55fb95a0be80 2024-03-08T21:09:17.381 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.401+0000 7f76e99be740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76e01ad290 con 0x55fb95a0c7a0 2024-03-08T21:09:17.382 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.401+0000 7f76e4c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fb95a15570 0x55fb95a1ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:09:17.382 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.401+0000 7f76e4c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fb95a15570 0x55fb95a1ca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:57530/0 (socket says 172.21.15.107:57530) 2024-03-08T21:09:17.382 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.401+0000 7f76e4c00640 1 -- 172.21.15.107:0/1926998699 learned_addr learned my addr 172.21.15.107:0/1926998699 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:09:17.382 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.401+0000 7f76df400640 1 --2- 172.21.15.107:0/1926998699 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fb95a0c7a0 0x55fb95a1c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:09:17.382 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.401+0000 7f76dfe00640 1 --2- 172.21.15.107:0/1926998699 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fb95a0be80 0x55fb95a05680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:09:17.382 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.402+0000 7f76e4c00640 1 -- 172.21.15.107:0/1926998699 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fb95a0c7a0 msgr2=0x55fb95a1c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:09:17.382 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.402+0000 7f76e4c00640 1 --2- 172.21.15.107:0/1926998699 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fb95a0c7a0 0x55fb95a1c500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:09:17.382 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.402+0000 7f76e4c00640 1 -- 172.21.15.107:0/1926998699 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fb95a0be80 msgr2=0x55fb95a05680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:09:17.382 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.402+0000 7f76e4c00640 1 --2- 172.21.15.107:0/1926998699 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fb95a0be80 0x55fb95a05680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:09:17.382 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.402+0000 7f76e4c00640 1 -- 172.21.15.107:0/1926998699 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fb95a02070 con 0x55fb95a15570 2024-03-08T21:09:17.382 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.402+0000 7f76dfe00640 1 --2- 172.21.15.107:0/1926998699 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fb95a0be80 0x55fb95a05680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T21:09:17.382 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.402+0000 7f76e4c00640 1 --2- 172.21.15.107:0/1926998699 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fb95a15570 0x55fb95a1ca40 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f76d8014cc0 tx=0x7f76d8009800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55c29b025f9474ca server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:09:17.382 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.402+0000 7f76cbe00640 1 -- 172.21.15.107:0/1926998699 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76d8014860 con 0x55fb95a15570 2024-03-08T21:09:17.383 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.402+0000 7f76cbe00640 1 -- 172.21.15.107:0/1926998699 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f76d8009ab0 con 0x55fb95a15570 2024-03-08T21:09:17.383 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.402+0000 7f76cbe00640 1 -- 172.21.15.107:0/1926998699 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76d80054e0 con 0x55fb95a15570 2024-03-08T21:09:17.386 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.403+0000 7f76e99be740 1 -- 172.21.15.107:0/1926998699 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fb95a15570 msgr2=0x55fb95a1ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:09:17.386 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.403+0000 7f76e99be740 1 --2- 172.21.15.107:0/1926998699 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fb95a15570 0x55fb95a1ca40 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f76d8014cc0 tx=0x7f76d8009800 comp rx=0 tx=0).stop 2024-03-08T21:09:17.386 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.403+0000 7f76e99be740 1 -- 172.21.15.107:0/1926998699 shutdown_connections 2024-03-08T21:09:17.386 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.403+0000 7f76e99be740 1 --2- 172.21.15.107:0/1926998699 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fb95a15570 0x55fb95a1ca40 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:09:17.386 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.403+0000 7f76e99be740 1 --2- 172.21.15.107:0/1926998699 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fb95a0c7a0 0x55fb95a1c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:09:17.386 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.403+0000 7f76e99be740 1 --2- 172.21.15.107:0/1926998699 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fb95a0be80 0x55fb95a05680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:09:17.387 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.403+0000 7f76e99be740 1 -- 172.21.15.107:0/1926998699 >> 172.21.15.107:0/1926998699 conn(0x55fb95a067a0 msgr2=0x55fb95a04f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:09:17.387 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.403+0000 7f76e99be740 1 -- 172.21.15.107:0/1926998699 shutdown_connections 2024-03-08T21:09:17.387 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.403+0000 7f76e99be740 1 -- 172.21.15.107:0/1926998699 wait complete. 2024-03-08T21:09:17.387 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.404+0000 7f76e99be740 1 -- start start 2024-03-08T21:09:17.387 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.404+0000 7f76e99be740 1 Processor -- start 2024-03-08T21:09:17.387 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.404+0000 7f76e99be740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fb95a0be80 0x55fb95a1b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:09:17.387 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.404+0000 7f76e99be740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fb95a0c7a0 0x55fb95a1bc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:09:17.387 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.404+0000 7f76e99be740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fb95a15570 0x55fb95a7fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:09:17.387 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.404+0000 7f76e99be740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76e01adf50 con 0x55fb95a0be80 2024-03-08T21:09:17.387 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.404+0000 7f76e99be740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76e01ad8f0 con 0x55fb95a15570 2024-03-08T21:09:17.387 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.404+0000 7f76e99be740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76e01aaef0 con 0x55fb95a0c7a0 2024-03-08T21:09:17.387 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.404+0000 7f76e99be740 10 client.0 _ll_register_callbacks cb 0x7f76d8009d10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T21:09:17.387 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.405+0000 7f76dfe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fb95a0be80 0x55fb95a1b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:09:17.388 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.405+0000 7f76dfe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fb95a0be80 0x55fb95a1b6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:57540/0 (socket says 172.21.15.107:57540) 2024-03-08T21:09:17.388 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.405+0000 7f76dfe00640 1 -- 172.21.15.107:0/2729992097 learned_addr learned my addr 172.21.15.107:0/2729992097 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:09:17.388 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.405+0000 7f76df400640 1 --2- 172.21.15.107:0/2729992097 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fb95a0c7a0 0x55fb95a1bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:09:17.388 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.405+0000 7f76e4c00640 1 --2- 172.21.15.107:0/2729992097 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fb95a15570 0x55fb95a7fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:09:17.388 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.405+0000 7f76dfe00640 1 -- 172.21.15.107:0/2729992097 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fb95a0c7a0 msgr2=0x55fb95a1bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:09:17.388 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.405+0000 7f76dfe00640 1 --2- 172.21.15.107:0/2729992097 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fb95a0c7a0 0x55fb95a1bc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:09:17.388 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.405+0000 7f76dfe00640 1 -- 172.21.15.107:0/2729992097 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fb95a15570 msgr2=0x55fb95a7fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:09:17.388 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.405+0000 7f76dfe00640 1 --2- 172.21.15.107:0/2729992097 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fb95a15570 0x55fb95a7fa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:09:17.388 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.405+0000 7f76dfe00640 1 -- 172.21.15.107:0/2729992097 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fb95a80070 con 0x55fb95a0be80 2024-03-08T21:09:17.388 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.405+0000 7f76dfe00640 1 --2- 172.21.15.107:0/2729992097 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fb95a0be80 0x55fb95a1b6d0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f76d00ee940 tx=0x7f76d00f0040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53a167fa44de9a4b server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:09:17.388 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.405+0000 7f76c9600640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-03-08T21:09:17.388 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.406+0000 7f76c9600640 1 -- 172.21.15.107:0/2729992097 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76d00c5f40 con 0x55fb95a0be80 2024-03-08T21:09:17.388 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.406+0000 7f76c9600640 1 -- 172.21.15.107:0/2729992097 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f76d00bbaa0 con 0x55fb95a0be80 2024-03-08T21:09:17.388 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.406+0000 7f76e99be740 1 -- 172.21.15.107:0/2729992097 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f76d8009ab0 con 0x55fb95a0be80 2024-03-08T21:09:17.389 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.406+0000 7f76c9600640 1 -- 172.21.15.107:0/2729992097 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f76d00a80b0 con 0x55fb95a0be80 2024-03-08T21:09:17.389 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.406+0000 7f76e99be740 10 client.16191 fetch_fsmap learned FSMap version 36 2024-03-08T21:09:17.389 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.406+0000 7f76e99be740 1 -- 172.21.15.107:0/2729992097 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x55fb95a81350 con 0x55fb95a0be80 2024-03-08T21:09:17.389 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.406+0000 7f76c9600640 1 -- 172.21.15.107:0/2729992097 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f76d8009ab0 con 0x55fb95a0be80 2024-03-08T21:09:17.389 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.406+0000 7f76c9600640 1 -- 172.21.15.107:0/2729992097 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 36) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f76d00a6ce0 con 0x55fb95a0be80 2024-03-08T21:09:17.389 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.406+0000 7f76e99be740 10 client.16191 fetch_fsmap finished waiting for FSMap version 36 2024-03-08T21:09:17.389 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.407+0000 7f76e99be740 10 client.16191 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-08T21:09:17.389 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.407+0000 7f76e99be740 20 client.16191 populate_metadata read hostname 'smithi107' 2024-03-08T21:09:17.389 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.407+0000 7f76e99be740 1 --2- 172.21.15.107:0/2729992097 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55fb95a81a00 0x55fb95a83e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:09:17.389 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.407+0000 7f76e99be740 4 client.16191 mds_command: new command op to 34475 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-08T21:09:17.389 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.407+0000 7f76e99be740 1 -- 172.21.15.107:0/2729992097 --> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55fb95976c60 con 0x55fb95a81a00 2024-03-08T21:09:17.389 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.407+0000 7f76df400640 1 --2- 172.21.15.107:0/2729992097 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55fb95a81a00 0x55fb95a83e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:09:17.390 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.408+0000 7f76c9600640 10 client.16191 ms_handle_connect on v2:172.21.15.149:6838/55290596 2024-03-08T21:09:17.390 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.408+0000 7f76df400640 1 --2- 172.21.15.107:0/2729992097 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55fb95a81a00 0x55fb95a83e20 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bac7f13dce4142b4 server_cookie=89ef96bc79666105 in_seq=0 out_seq=0 2024-03-08T21:09:17.403 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.423+0000 7f76c9600640 1 -- 172.21.15.107:0/2729992097 <== mds.0 v2:172.21.15.149:6838/55290596 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55fb95976c60 con 0x55fb95a81a00 2024-03-08T21:09:17.403 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.423+0000 7f76c9600640 10 client.16191 handle_command_reply: tid=0 2024-03-08T21:09:17.403 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.423+0000 7f76e99be740 1 client.16191 shutdown 2024-03-08T21:09:17.403 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.423+0000 7f76e99be740 1 -- 172.21.15.107:0/2729992097 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fb95a0be80 msgr2=0x55fb95a1b6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:09:17.403 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.423+0000 7f76e99be740 1 --2- 172.21.15.107:0/2729992097 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fb95a0be80 0x55fb95a1b6d0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f76d00ee940 tx=0x7f76d00f0040 comp rx=0 tx=0).stop 2024-03-08T21:09:17.403 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.424+0000 7f76e99be740 1 -- 172.21.15.107:0/2729992097 shutdown_connections 2024-03-08T21:09:17.403 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.424+0000 7f76e99be740 1 --2- 172.21.15.107:0/2729992097 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55fb95a81a00 0x55fb95a83e20 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:09:17.404 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.424+0000 7f76e99be740 1 --2- 172.21.15.107:0/2729992097 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fb95a15570 0x55fb95a7fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:09:17.404 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.424+0000 7f76e99be740 1 --2- 172.21.15.107:0/2729992097 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fb95a0c7a0 0x55fb95a1bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:09:17.404 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.424+0000 7f76c9600640 0 client.16191 ms_handle_reset on v2:172.21.15.149:6838/55290596 2024-03-08T21:09:17.404 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.424+0000 7f76e99be740 1 --2- 172.21.15.107:0/2729992097 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fb95a0be80 0x55fb95a1b6d0 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:09:17.404 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.424+0000 7f76e99be740 1 -- 172.21.15.107:0/2729992097 >> 172.21.15.107:0/2729992097 conn(0x55fb95a067a0 msgr2=0x55fb95a08480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:09:17.404 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.424+0000 7f76e99be740 1 -- 172.21.15.107:0/2729992097 shutdown_connections 2024-03-08T21:09:17.404 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.424+0000 7f76e99be740 1 -- 172.21.15.107:0/2729992097 wait complete. 2024-03-08T21:09:17.404 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.424+0000 7f76e99be740 20 client.16191 trim_cache size 0 max 16384 2024-03-08T21:09:17.404 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:{ 2024-03-08T21:09:17.405 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "message": "", 2024-03-08T21:09:17.405 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "return_code": 0 2024-03-08T21:09:17.405 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:} 2024-03-08T21:09:17.405 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.425+0000 7f76e5600640 1 -- 172.21.15.107:0/2050497024 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f76c40a2960 msgr2=0x7f76c40a4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:09:17.405 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.425+0000 7f76e5600640 1 --2- 172.21.15.107:0/2050497024 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f76c40a2960 0x7f76c40a4e20 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f76d8009fd0 tx=0x7f76d8016000 comp rx=0 tx=0).stop 2024-03-08T21:09:17.405 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.425+0000 7f76e5600640 1 -- 172.21.15.107:0/2050497024 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f76e0101360 msgr2=0x7f76e019d610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:09:17.406 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.425+0000 7f76e5600640 1 --2- 172.21.15.107:0/2050497024 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f76e0101360 0x7f76e019d610 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f76d0005940 tx=0x7f76d002c7e0 comp rx=0 tx=0).stop 2024-03-08T21:09:17.406 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.425+0000 7f76e5600640 1 -- 172.21.15.107:0/2050497024 shutdown_connections 2024-03-08T21:09:17.406 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.425+0000 7f76e5600640 1 --2- 172.21.15.107:0/2050497024 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f76c40a2960 0x7f76c40a4e20 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:09:17.406 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.425+0000 7f76e5600640 1 --2- 172.21.15.107:0/2050497024 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f76e0197780 0x7f76e0197c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:09:17.406 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.425+0000 7f76e5600640 1 --2- 172.21.15.107:0/2050497024 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f76e0106430 0x7f76e019db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:09:17.406 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.425+0000 7f76e5600640 1 --2- 172.21.15.107:0/2050497024 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f76e0101360 0x7f76e019d610 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:09:17.406 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.425+0000 7f76e5600640 1 -- 172.21.15.107:0/2050497024 >> 172.21.15.107:0/2050497024 conn(0x7f76e00fcc00 msgr2=0x7f76e00fe020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:09:17.406 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.425+0000 7f76e5600640 1 -- 172.21.15.107:0/2050497024 shutdown_connections 2024-03-08T21:09:17.406 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:17.425+0000 7f76e5600640 1 -- 172.21.15.107:0/2050497024 wait complete. 2024-03-08T21:09:17.624 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:09:17 smithi204 podman[71537]: 2024-03-08 21:09:17.354964122 +0000 UTC m=+0.443069146 container init e8449560c340b0836ff42cc1649f3bb380947f5beb57f560f203f985e87ef28f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=rishabh-2024mar8-7d63482, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, GIT_CLEAN=True) 2024-03-08T21:09:17.624 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:09:17 smithi204 podman[71537]: 2024-03-08 21:09:17.365728819 +0000 UTC m=+0.453833840 container start e8449560c340b0836ff42cc1649f3bb380947f5beb57f560f203f985e87ef28f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, RELEASE=rishabh-2024mar8-7d63482, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-08T21:09:17.624 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:09:17 smithi204 ceph-mds[71555]: -- 172.21.15.204:0/3053196806 <== mon.1 v2:172.21.15.204:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55770d3c9a40 con 0x55770d39cc00 2024-03-08T21:09:17.624 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:09:17 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[71551]: starting mds.c at 2024-03-08T21:09:17.624 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:09:17 smithi204 bash[71537]: e8449560c340b0836ff42cc1649f3bb380947f5beb57f560f203f985e87ef28f 2024-03-08T21:09:17.625 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:09:17 smithi204 systemd[1]: Started Ceph mds.c for dc20f0f8-dd8c-11ee-95c6-87774f69a715. 2024-03-08T21:09:18.010 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:09:17 smithi149 ceph-mon[35042]: daemon mds.c restarted 2024-03-08T21:09:18.124 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:09:17 smithi204 ceph-mon[34935]: daemon mds.c restarted 2024-03-08T21:09:18.213 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:09:17 smithi107 ceph-mon[31381]: daemon mds.c restarted 2024-03-08T21:09:19.010 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:09:18 smithi149 ceph-mon[35042]: osdmap e100: 12 total, 12 up, 12 in 2024-03-08T21:09:19.011 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:09:18 smithi149 ceph-mon[35042]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-08T21:09:19.011 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:09:18 smithi149 ceph-mon[35042]: mds.? [v2:172.21.15.204:6836/1694030516,v1:172.21.15.204:6837/1694030516] up:boot 2024-03-08T21:09:19.011 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:09:18 smithi149 ceph-mon[35042]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-03-08T21:09:19.011 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:09:18 smithi149 ceph-mon[35042]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-08T21:09:19.011 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:09:18 smithi149 ceph-mon[35042]: pgmap v277: 97 pgs: 97 active+clean; 6.0 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 646 B/s wr, 10 op/s 2024-03-08T21:09:19.124 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:09:18 smithi204 ceph-mon[34935]: osdmap e100: 12 total, 12 up, 12 in 2024-03-08T21:09:19.124 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:09:18 smithi204 ceph-mon[34935]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-08T21:09:19.124 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:09:18 smithi204 ceph-mon[34935]: mds.? [v2:172.21.15.204:6836/1694030516,v1:172.21.15.204:6837/1694030516] up:boot 2024-03-08T21:09:19.124 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:09:18 smithi204 ceph-mon[34935]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-03-08T21:09:19.124 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:09:18 smithi204 ceph-mon[34935]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-08T21:09:19.124 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:09:18 smithi204 ceph-mon[34935]: pgmap v277: 97 pgs: 97 active+clean; 6.0 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 646 B/s wr, 10 op/s 2024-03-08T21:09:19.213 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:09:18 smithi107 ceph-mon[31381]: osdmap e100: 12 total, 12 up, 12 in 2024-03-08T21:09:19.213 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:09:18 smithi107 ceph-mon[31381]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-08T21:09:19.213 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:09:18 smithi107 ceph-mon[31381]: mds.? [v2:172.21.15.204:6836/1694030516,v1:172.21.15.204:6837/1694030516] up:boot 2024-03-08T21:09:19.213 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:09:18 smithi107 ceph-mon[31381]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-03-08T21:09:19.213 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:09:18 smithi107 ceph-mon[31381]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-08T21:09:19.213 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:09:18 smithi107 ceph-mon[31381]: pgmap v277: 97 pgs: 97 active+clean; 6.0 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 646 B/s wr, 10 op/s 2024-03-08T21:09:21.010 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:09:20 smithi149 ceph-mon[35042]: pgmap v278: 97 pgs: 97 active+clean; 6.0 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 614 B/s wr, 9 op/s 2024-03-08T21:09:21.123 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:09:20 smithi204 ceph-mon[34935]: pgmap v278: 97 pgs: 97 active+clean; 6.0 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 614 B/s wr, 9 op/s 2024-03-08T21:09:21.213 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:09:20 smithi107 ceph-mon[31381]: pgmap v278: 97 pgs: 97 active+clean; 6.0 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 614 B/s wr, 9 op/s 2024-03-08T21:09:23.010 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:09:22 smithi149 ceph-mon[35042]: pgmap v279: 97 pgs: 97 active+clean; 6.0 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 614 B/s wr, 9 op/s 2024-03-08T21:09:23.123 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:09:22 smithi204 ceph-mon[34935]: pgmap v279: 97 pgs: 97 active+clean; 6.0 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 614 B/s wr, 9 op/s 2024-03-08T21:09:23.213 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:09:22 smithi107 ceph-mon[31381]: pgmap v279: 97 pgs: 97 active+clean; 6.0 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 614 B/s wr, 9 op/s 2024-03-08T21:09:25.010 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:09:24 smithi149 ceph-mon[35042]: pgmap v280: 97 pgs: 97 active+clean; 6.0 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 511 B/s wr, 10 op/s 2024-03-08T21:09:25.123 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:09:24 smithi204 ceph-mon[34935]: pgmap v280: 97 pgs: 97 active+clean; 6.0 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 511 B/s wr, 10 op/s 2024-03-08T21:09:25.212 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:09:24 smithi107 ceph-mon[31381]: pgmap v280: 97 pgs: 97 active+clean; 6.0 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 511 B/s wr, 10 op/s 2024-03-08T21:09:27.010 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:09:26 smithi149 ceph-mon[35042]: pgmap v281: 97 pgs: 97 active+clean; 6.0 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 511 B/s wr, 10 op/s 2024-03-08T21:09:27.124 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:09:26 smithi204 ceph-mon[34935]: pgmap v281: 97 pgs: 97 active+clean; 6.0 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 511 B/s wr, 10 op/s 2024-03-08T21:09:27.213 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:09:26 smithi107 ceph-mon[31381]: pgmap v281: 97 pgs: 97 active+clean; 6.0 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 511 B/s wr, 10 op/s 2024-03-08T21:09:29.010 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:09:28 smithi149 ceph-mon[35042]: pgmap v282: 97 pgs: 97 active+clean; 6.0 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 468 B/s wr, 14 op/s 2024-03-08T21:09:29.124 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:09:28 smithi204 ceph-mon[34935]: pgmap v282: 97 pgs: 97 active+clean; 6.0 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 468 B/s wr, 14 op/s 2024-03-08T21:09:29.213 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:09:28 smithi107 ceph-mon[31381]: pgmap v282: 97 pgs: 97 active+clean; 6.0 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 468 B/s wr, 14 op/s 2024-03-08T21:09:30.552 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.572+0000 7faacca00640 1 -- 172.21.15.107:0/1615836809 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faac8105310 msgr2=0x7faac810cee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:09:30.552 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.572+0000 7faacca00640 1 --2- 172.21.15.107:0/1615836809 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faac8105310 0x7faac810cee0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7faac0009ca0 tx=0x7faac002c0f0 comp rx=0 tx=0).stop 2024-03-08T21:09:30.552 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.572+0000 7faacca00640 1 -- 172.21.15.107:0/1615836809 shutdown_connections 2024-03-08T21:09:30.552 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.572+0000 7faacca00640 1 --2- 172.21.15.107:0/1615836809 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faac8105310 0x7faac810cee0 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:09:30.552 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.572+0000 7faacca00640 1 --2- 172.21.15.107:0/1615836809 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faac8100bf0 0x7faac8104c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:09:30.552 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.572+0000 7faacca00640 1 --2- 172.21.15.107:0/1615836809 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faac8100240 0x7faac8100620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:09:30.552 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.572+0000 7faacca00640 1 -- 172.21.15.107:0/1615836809 >> 172.21.15.107:0/1615836809 conn(0x7faac80fbb00 msgr2=0x7faac80fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:09:30.553 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.573+0000 7faacca00640 1 -- 172.21.15.107:0/1615836809 shutdown_connections 2024-03-08T21:09:30.553 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.573+0000 7faacca00640 1 -- 172.21.15.107:0/1615836809 wait complete. 2024-03-08T21:09:30.554 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.574+0000 7faacca00640 1 Processor -- start 2024-03-08T21:09:30.554 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.574+0000 7faacca00640 1 -- start start 2024-03-08T21:09:30.554 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.574+0000 7faacca00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faac8100240 0x7faac8101cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:09:30.554 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.574+0000 7faacca00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faac8105310 0x7faac8102210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:09:30.554 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.574+0000 7faacca00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faac8102750 0x7faac81a5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:09:30.554 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.574+0000 7faacca00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faac8110260 con 0x7faac8102750 2024-03-08T21:09:30.554 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.574+0000 7faacca00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faac81100e0 con 0x7faac8105310 2024-03-08T21:09:30.554 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.574+0000 7faacca00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faac81103e0 con 0x7faac8100240 2024-03-08T21:09:30.555 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.575+0000 7faac7200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faac8102750 0x7faac81a5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:09:30.555 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.575+0000 7faac7200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faac8102750 0x7faac81a5ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:59056/0 (socket says 172.21.15.107:59056) 2024-03-08T21:09:30.555 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.575+0000 7faac7200640 1 -- 172.21.15.107:0/788944308 learned_addr learned my addr 172.21.15.107:0/788944308 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:09:30.555 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.575+0000 7faac6800640 1 --2- 172.21.15.107:0/788944308 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faac8100240 0x7faac8101cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:09:30.555 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.575+0000 7faac5e00640 1 --2- 172.21.15.107:0/788944308 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faac8105310 0x7faac8102210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:09:30.555 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.575+0000 7faac7200640 1 -- 172.21.15.107:0/788944308 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faac8100240 msgr2=0x7faac8101cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:09:30.555 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.575+0000 7faac7200640 1 --2- 172.21.15.107:0/788944308 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faac8100240 0x7faac8101cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:09:30.555 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.575+0000 7faac7200640 1 -- 172.21.15.107:0/788944308 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faac8105310 msgr2=0x7faac8102210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:09:30.555 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.575+0000 7faac7200640 1 --2- 172.21.15.107:0/788944308 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faac8105310 0x7faac8102210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:09:30.555 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.575+0000 7faac7200640 1 -- 172.21.15.107:0/788944308 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faac81a61e0 con 0x7faac8102750 2024-03-08T21:09:30.556 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.575+0000 7faac7200640 1 --2- 172.21.15.107:0/788944308 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faac8102750 0x7faac81a5ae0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7faac002c700 tx=0x7faac0035d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:09:30.556 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.576+0000 7faabf400640 1 -- 172.21.15.107:0/788944308 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faac0039040 con 0x7faac8102750 2024-03-08T21:09:30.556 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.576+0000 7faabf400640 1 -- 172.21.15.107:0/788944308 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7faac0044070 con 0x7faac8102750 2024-03-08T21:09:30.556 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.576+0000 7faacca00640 1 -- 172.21.15.107:0/788944308 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faac81a6470 con 0x7faac8102750 2024-03-08T21:09:30.556 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.576+0000 7faabf400640 1 -- 172.21.15.107:0/788944308 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faac0004330 con 0x7faac8102750 2024-03-08T21:09:30.556 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.576+0000 7faacca00640 1 -- 172.21.15.107:0/788944308 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faac81a6960 con 0x7faac8102750 2024-03-08T21:09:30.559 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.580+0000 7faad0185740 1 Processor -- start 2024-03-08T21:09:30.559 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.581+0000 7faabf400640 1 -- 172.21.15.107:0/788944308 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) v1 ==== 137422+0+0 (secure 0 0 0) 0x7faac00044d0 con 0x7faac8102750 2024-03-08T21:09:30.560 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.581+0000 7faad0185740 1 -- start start 2024-03-08T21:09:30.560 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.581+0000 7faad0185740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56124cb88e80 0x56124cb89260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:09:30.561 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.581+0000 7faad0185740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56124cb897a0 0x56124cb9a840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:09:30.561 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.581+0000 7faad0185740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56124cb9ad80 0x56124cb9d170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:09:30.561 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.581+0000 7faad0185740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x56124caec3d0 con 0x56124cb88e80 2024-03-08T21:09:30.561 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.581+0000 7faad0185740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x56124ca956b0 con 0x56124cb897a0 2024-03-08T21:09:30.561 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.581+0000 7faad0185740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56124cae8470 con 0x56124cb9ad80 2024-03-08T21:09:30.561 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.582+0000 7faac5e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56124cb88e80 0x56124cb89260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:09:30.562 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.582+0000 7faac5e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56124cb88e80 0x56124cb89260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:59058/0 (socket says 172.21.15.107:59058) 2024-03-08T21:09:30.563 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.582+0000 7faac5e00640 1 -- 172.21.15.107:0/1444365320 learned_addr learned my addr 172.21.15.107:0/1444365320 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:09:30.563 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.582+0000 7faac6800640 1 --2- 172.21.15.107:0/1444365320 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56124cb897a0 0x56124cb9a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:09:30.563 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.582+0000 7faac5e00640 1 --2- 172.21.15.107:0/1444365320 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56124cb9ad80 0x56124cb9d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:09:30.563 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.583+0000 7faabf400640 1 --2- 172.21.15.107:0/788944308 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7faaac0a2990 0x7faaac0a4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:09:30.563 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.583+0000 7faac5e00640 1 -- 172.21.15.107:0/1444365320 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56124cb9ad80 msgr2=0x56124cb9d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:09:30.563 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.583+0000 7faac5e00640 1 --2- 172.21.15.107:0/1444365320 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56124cb9ad80 0x56124cb9d170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:09:30.563 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.583+0000 7faac5e00640 1 -- 172.21.15.107:0/1444365320 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56124cb897a0 msgr2=0x56124cb9a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:09:30.563 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.583+0000 7faac5e00640 1 --2- 172.21.15.107:0/1444365320 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56124cb897a0 0x56124cb9a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:09:30.563 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.583+0000 7faac5e00640 1 -- 172.21.15.107:0/1444365320 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56124cabbb50 con 0x56124cb88e80 2024-03-08T21:09:30.563 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.583+0000 7faabf400640 1 -- 172.21.15.107:0/788944308 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 8541+0+0 (secure 0 0 0) 0x7faac00f11a0 con 0x7faac8102750 2024-03-08T21:09:30.563 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.583+0000 7faac7200640 1 --2- 172.21.15.107:0/788944308 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7faaac0a2990 0x7faaac0a4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:09:30.563 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.583+0000 7faac5e00640 1 --2- 172.21.15.107:0/1444365320 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56124cb88e80 0x56124cb89260 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7faab400dfd0 tx=0x7faab400d5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=635fb92eb570011e server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:09:30.564 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.584+0000 7faab3400640 1 -- 172.21.15.107:0/1444365320 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faab4004030 con 0x56124cb88e80 2024-03-08T21:09:30.564 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.584+0000 7faab3400640 1 -- 172.21.15.107:0/1444365320 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7faab40041d0 con 0x56124cb88e80 2024-03-08T21:09:30.564 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.584+0000 7faab3400640 1 -- 172.21.15.107:0/1444365320 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faab40106c0 con 0x56124cb88e80 2024-03-08T21:09:30.568 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.584+0000 7faad0185740 1 -- 172.21.15.107:0/1444365320 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56124cb88e80 msgr2=0x56124cb89260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:09:30.568 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.584+0000 7faac7200640 1 --2- 172.21.15.107:0/788944308 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7faaac0a2990 0x7faaac0a4e50 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7faac00bd890 tx=0x7faac00095c0 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:09:30.568 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.584+0000 7faad0185740 1 --2- 172.21.15.107:0/1444365320 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56124cb88e80 0x56124cb89260 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7faab400dfd0 tx=0x7faab400d5d0 comp rx=0 tx=0).stop 2024-03-08T21:09:30.568 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.585+0000 7faad0185740 1 -- 172.21.15.107:0/1444365320 shutdown_connections 2024-03-08T21:09:30.568 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.585+0000 7faad0185740 1 --2- 172.21.15.107:0/1444365320 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56124cb9ad80 0x56124cb9d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:09:30.568 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.585+0000 7faad0185740 1 --2- 172.21.15.107:0/1444365320 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56124cb897a0 0x56124cb9a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:09:30.568 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.585+0000 7faad0185740 1 --2- 172.21.15.107:0/1444365320 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56124cb88e80 0x56124cb89260 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:09:30.568 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.585+0000 7faad0185740 1 -- 172.21.15.107:0/1444365320 >> 172.21.15.107:0/1444365320 conn(0x56124cb837a0 msgr2=0x56124cb83b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:09:30.569 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.585+0000 7faad0185740 1 -- 172.21.15.107:0/1444365320 shutdown_connections 2024-03-08T21:09:30.569 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.585+0000 7faad0185740 1 -- 172.21.15.107:0/1444365320 wait complete. 2024-03-08T21:09:30.569 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.585+0000 7faad0185740 1 -- start start 2024-03-08T21:09:30.569 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.586+0000 7faad0185740 1 Processor -- start 2024-03-08T21:09:30.569 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.586+0000 7faad0185740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56124cb88e80 0x56124cb9a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:09:30.569 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.586+0000 7faad0185740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56124cb897a0 0x56124cb97b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:09:30.569 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.586+0000 7faad0185740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56124cb9ad80 0x56124cb98280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:09:30.569 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.586+0000 7faad0185740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x56124cb9d9b0 con 0x56124cb897a0 2024-03-08T21:09:30.569 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.586+0000 7faad0185740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x56124caa89f0 con 0x56124cb88e80 2024-03-08T21:09:30.569 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.586+0000 7faad0185740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56124cb9db30 con 0x56124cb9ad80 2024-03-08T21:09:30.570 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.586+0000 7faad0185740 10 client.0 _ll_register_callbacks cb 0x56124c800a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T21:09:30.570 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.587+0000 7faac6800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56124cb897a0 0x56124cb97b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:09:30.570 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.587+0000 7faac6800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56124cb897a0 0x56124cb97b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:59064/0 (socket says 172.21.15.107:59064) 2024-03-08T21:09:30.570 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.587+0000 7faac6800640 1 -- 172.21.15.107:0/971307591 learned_addr learned my addr 172.21.15.107:0/971307591 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:09:30.570 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.587+0000 7faac5e00640 1 --2- 172.21.15.107:0/971307591 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56124cb88e80 0x56124cb9a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:09:30.570 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.587+0000 7faac5e00640 1 --2- 172.21.15.107:0/971307591 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56124cb9ad80 0x56124cb98280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:09:30.570 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.587+0000 7faac6800640 1 -- 172.21.15.107:0/971307591 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56124cb9ad80 msgr2=0x56124cb98280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:09:30.570 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.587+0000 7faac6800640 1 --2- 172.21.15.107:0/971307591 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56124cb9ad80 0x56124cb98280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:09:30.571 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.587+0000 7faac6800640 1 -- 172.21.15.107:0/971307591 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56124cb88e80 msgr2=0x56124cb9a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:09:30.571 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.587+0000 7faac6800640 1 --2- 172.21.15.107:0/971307591 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56124cb88e80 0x56124cb9a7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:09:30.571 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.587+0000 7faac6800640 1 -- 172.21.15.107:0/971307591 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56124c801210 con 0x56124cb897a0 2024-03-08T21:09:30.571 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.588+0000 7faac6800640 1 --2- 172.21.15.107:0/971307591 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56124cb897a0 0x56124cb97b40 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7faab800bee0 tx=0x7faab8012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4bd12389efd8820 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:09:30.571 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.588+0000 7faaa7e00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-03-08T21:09:30.571 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.588+0000 7faaa7e00640 1 -- 172.21.15.107:0/971307591 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faab8004230 con 0x56124cb897a0 2024-03-08T21:09:30.571 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.588+0000 7faaa7e00640 1 -- 172.21.15.107:0/971307591 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 8541+0+0 (secure 0 0 0) 0x7faab80043d0 con 0x56124cb897a0 2024-03-08T21:09:30.571 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.588+0000 7faad0185740 1 -- 172.21.15.107:0/971307591 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7faab40041d0 con 0x56124cb897a0 2024-03-08T21:09:30.571 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.588+0000 7faaa7e00640 1 -- 172.21.15.107:0/971307591 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7faab80106a0 con 0x56124cb897a0 2024-03-08T21:09:30.571 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.588+0000 7faad0185740 10 client.16209 fetch_fsmap learned FSMap version 38 2024-03-08T21:09:30.572 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.588+0000 7faad0185740 1 -- 172.21.15.107:0/971307591 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x56124c8cd620 con 0x56124cb897a0 2024-03-08T21:09:30.572 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.588+0000 7faaa7e00640 1 -- 172.21.15.107:0/971307591 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7faab40041d0 con 0x56124cb897a0 2024-03-08T21:09:30.572 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.589+0000 7faaa7e00640 1 -- 172.21.15.107:0/971307591 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 38) v1 ==== 7882+0+0 (secure 0 0 0) 0x7faab8003b50 con 0x56124cb897a0 2024-03-08T21:09:30.572 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.589+0000 7faad0185740 10 client.16209 fetch_fsmap finished waiting for FSMap version 38 2024-03-08T21:09:30.572 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.589+0000 7faad0185740 10 client.16209 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-08T21:09:30.572 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.589+0000 7faad0185740 20 client.16209 populate_metadata read hostname 'smithi107' 2024-03-08T21:09:30.572 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.589+0000 7faad0185740 1 --2- 172.21.15.107:0/971307591 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x56124cbaad20 0x56124cbad110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:09:30.572 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.589+0000 7faad0185740 4 client.16209 mds_command: new command op to 34475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-08T21:09:30.572 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.589+0000 7faad0185740 1 -- 172.21.15.107:0/971307591 --> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56124caf9910 con 0x56124cbaad20 2024-03-08T21:09:30.574 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.590+0000 7faac5e00640 1 --2- 172.21.15.107:0/971307591 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x56124cbaad20 0x56124cbad110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:09:30.574 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.591+0000 7faaa7e00640 10 client.16209 ms_handle_connect on v2:172.21.15.149:6838/55290596 2024-03-08T21:09:30.574 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.591+0000 7faac5e00640 1 --2- 172.21.15.107:0/971307591 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x56124cbaad20 0x56124cbad110 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d9ae4a731c7e0faa server_cookie=f4aaa9078dd77fc5 in_seq=0 out_seq=0 2024-03-08T21:09:30.574 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.592+0000 7faaa7e00640 1 -- 172.21.15.107:0/971307591 <== mds.0 v2:172.21.15.149:6838/55290596 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56124caf9910 con 0x56124cbaad20 2024-03-08T21:09:30.574 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.592+0000 7faaa7e00640 10 client.16209 handle_command_reply: tid=0 2024-03-08T21:09:30.574 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.592+0000 7faad0185740 1 client.16209 shutdown 2024-03-08T21:09:30.574 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.592+0000 7faad0185740 1 -- 172.21.15.107:0/971307591 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56124cb897a0 msgr2=0x56124cb97b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:09:30.575 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.592+0000 7faad0185740 1 --2- 172.21.15.107:0/971307591 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56124cb897a0 0x56124cb97b40 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7faab800bee0 tx=0x7faab8012820 comp rx=0 tx=0).stop 2024-03-08T21:09:30.575 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.593+0000 7faad0185740 1 -- 172.21.15.107:0/971307591 shutdown_connections 2024-03-08T21:09:30.575 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.593+0000 7faad0185740 1 --2- 172.21.15.107:0/971307591 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56124cb9ad80 0x56124cb98280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:09:30.575 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.593+0000 7faad0185740 1 --2- 172.21.15.107:0/971307591 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56124cb897a0 0x56124cb97b40 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:09:30.575 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.593+0000 7faad0185740 1 --2- 172.21.15.107:0/971307591 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x56124cbaad20 0x56124cbad110 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:09:30.575 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.593+0000 7faad0185740 1 --2- 172.21.15.107:0/971307591 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56124cb88e80 0x56124cb9a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:09:30.575 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.593+0000 7faaa7e00640 0 client.16209 ms_handle_reset on v2:172.21.15.149:6838/55290596 2024-03-08T21:09:30.575 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.593+0000 7faad0185740 1 -- 172.21.15.107:0/971307591 >> 172.21.15.107:0/971307591 conn(0x56124cb837a0 msgr2=0x56124cb854f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:09:30.575 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.593+0000 7faad0185740 1 -- 172.21.15.107:0/971307591 shutdown_connections 2024-03-08T21:09:30.575 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.593+0000 7faad0185740 1 -- 172.21.15.107:0/971307591 wait complete. 2024-03-08T21:09:30.575 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.593+0000 7faad0185740 20 client.16209 trim_cache size 0 max 16384 2024-03-08T21:09:30.588 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.609+0000 7faad0185740 1 Processor -- start 2024-03-08T21:09:30.588 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.609+0000 7faad0185740 1 -- start start 2024-03-08T21:09:30.588 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.609+0000 7faad0185740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56124cb88e80 0x56124cb82680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:09:30.588 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.609+0000 7faad0185740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56124cb897a0 0x56124cba1d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:09:30.588 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.609+0000 7faad0185740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56124cb9ad80 0x56124cba22b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:09:30.588 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.609+0000 7faad0185740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faac81aa900 con 0x56124cb9ad80 2024-03-08T21:09:30.589 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.609+0000 7faad0185740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faac81ac280 con 0x56124cb88e80 2024-03-08T21:09:30.589 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.609+0000 7faad0185740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faac81aaf60 con 0x56124cb897a0 2024-03-08T21:09:30.594 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.609+0000 7faac5e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56124cb9ad80 0x56124cba22b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:09:30.594 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.610+0000 7faac5e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56124cb9ad80 0x56124cba22b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:59080/0 (socket says 172.21.15.107:59080) 2024-03-08T21:09:30.594 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.610+0000 7faac5e00640 1 -- 172.21.15.107:0/4110190772 learned_addr learned my addr 172.21.15.107:0/4110190772 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:09:30.595 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.610+0000 7faac5e00640 1 --2- 172.21.15.107:0/4110190772 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56124cb88e80 0x56124cb82680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:09:30.595 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.610+0000 7faac6800640 1 --2- 172.21.15.107:0/4110190772 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56124cb897a0 0x56124cba1d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:09:30.595 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.610+0000 7faac5e00640 1 -- 172.21.15.107:0/4110190772 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56124cb897a0 msgr2=0x56124cba1d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:09:30.595 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.610+0000 7faac5e00640 1 --2- 172.21.15.107:0/4110190772 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56124cb897a0 0x56124cba1d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:09:30.595 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.610+0000 7faac5e00640 1 -- 172.21.15.107:0/4110190772 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56124cb88e80 msgr2=0x56124cb82680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:09:30.595 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.610+0000 7faac5e00640 1 --2- 172.21.15.107:0/4110190772 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56124cb88e80 0x56124cb82680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:09:30.595 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.610+0000 7faac5e00640 1 -- 172.21.15.107:0/4110190772 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56124cb883d0 con 0x56124cb9ad80 2024-03-08T21:09:30.595 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.610+0000 7faac5e00640 1 --2- 172.21.15.107:0/4110190772 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56124cb9ad80 0x56124cba22b0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7faab400dfd0 tx=0x7faab4015040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=581193a873a54924 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:09:30.595 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.610+0000 7faab2a00640 1 -- 172.21.15.107:0/4110190772 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faab4004400 con 0x56124cb9ad80 2024-03-08T21:09:30.595 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.611+0000 7faab2a00640 1 -- 172.21.15.107:0/4110190772 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7faab4005040 con 0x56124cb9ad80 2024-03-08T21:09:30.595 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.611+0000 7faab2a00640 1 -- 172.21.15.107:0/4110190772 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faab4027140 con 0x56124cb9ad80 2024-03-08T21:09:30.596 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.611+0000 7faad0185740 1 -- 172.21.15.107:0/4110190772 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56124cb9ad80 msgr2=0x56124cba22b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:09:30.596 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.611+0000 7faad0185740 1 --2- 172.21.15.107:0/4110190772 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56124cb9ad80 0x56124cba22b0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7faab400dfd0 tx=0x7faab4015040 comp rx=0 tx=0).stop 2024-03-08T21:09:30.596 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.611+0000 7faad0185740 1 -- 172.21.15.107:0/4110190772 shutdown_connections 2024-03-08T21:09:30.596 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.611+0000 7faad0185740 1 --2- 172.21.15.107:0/4110190772 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56124cb9ad80 0x56124cba22b0 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:09:30.596 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.611+0000 7faad0185740 1 --2- 172.21.15.107:0/4110190772 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56124cb897a0 0x56124cba1d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:09:30.596 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.611+0000 7faad0185740 1 --2- 172.21.15.107:0/4110190772 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56124cb88e80 0x56124cb82680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:09:30.596 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.611+0000 7faad0185740 1 -- 172.21.15.107:0/4110190772 >> 172.21.15.107:0/4110190772 conn(0x56124cb837a0 msgr2=0x56124cb81f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:09:30.596 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.611+0000 7faad0185740 1 -- 172.21.15.107:0/4110190772 shutdown_connections 2024-03-08T21:09:30.596 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.611+0000 7faad0185740 1 -- 172.21.15.107:0/4110190772 wait complete. 2024-03-08T21:09:30.596 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.612+0000 7faad0185740 1 -- start start 2024-03-08T21:09:30.597 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.612+0000 7faad0185740 1 Processor -- start 2024-03-08T21:09:30.597 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.613+0000 7faad0185740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56124cb88e80 0x56124cba0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:09:30.597 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.613+0000 7faad0185740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56124cb897a0 0x56124cba1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:09:30.597 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.613+0000 7faad0185740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56124cb9ad80 0x56124cc011b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:09:30.597 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.613+0000 7faad0185740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faac81abc20 con 0x56124cb897a0 2024-03-08T21:09:30.597 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.613+0000 7faad0185740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faac81ab5c0 con 0x56124cb88e80 2024-03-08T21:09:30.597 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.613+0000 7faad0185740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faac81a8bc0 con 0x56124cb9ad80 2024-03-08T21:09:30.597 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.613+0000 7faad0185740 10 client.0 _ll_register_callbacks cb 0x7faab400d430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T21:09:30.597 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.613+0000 7faac6800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56124cb897a0 0x56124cba1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:09:30.598 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.613+0000 7faac6800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56124cb897a0 0x56124cba1340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:59084/0 (socket says 172.21.15.107:59084) 2024-03-08T21:09:30.598 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.613+0000 7faac6800640 1 -- 172.21.15.107:0/3254017069 learned_addr learned my addr 172.21.15.107:0/3254017069 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:09:30.598 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.613+0000 7faac5e00640 1 --2- 172.21.15.107:0/3254017069 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56124cb88e80 0x56124cba0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:09:30.598 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.613+0000 7faac5e00640 1 --2- 172.21.15.107:0/3254017069 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56124cb9ad80 0x56124cc011b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:09:30.598 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.614+0000 7faac6800640 1 -- 172.21.15.107:0/3254017069 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56124cb9ad80 msgr2=0x56124cc011b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:09:30.598 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.614+0000 7faac6800640 1 --2- 172.21.15.107:0/3254017069 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56124cb9ad80 0x56124cc011b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:09:30.598 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.614+0000 7faac6800640 1 -- 172.21.15.107:0/3254017069 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56124cb88e80 msgr2=0x56124cba0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:09:30.598 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.614+0000 7faac6800640 1 --2- 172.21.15.107:0/3254017069 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56124cb88e80 0x56124cba0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:09:30.598 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.614+0000 7faac6800640 1 -- 172.21.15.107:0/3254017069 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56124cc016f0 con 0x56124cb897a0 2024-03-08T21:09:30.598 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.614+0000 7faac6800640 1 --2- 172.21.15.107:0/3254017069 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56124cb897a0 0x56124cba1340 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7faab8005fa0 tx=0x7faab801ba20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=def1524aeee19fda server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:09:30.599 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.614+0000 7faaa7e00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-03-08T21:09:30.599 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.614+0000 7faaa7e00640 1 -- 172.21.15.107:0/3254017069 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faab8028040 con 0x56124cb897a0 2024-03-08T21:09:30.599 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.614+0000 7faaa7e00640 1 -- 172.21.15.107:0/3254017069 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 8541+0+0 (secure 0 0 0) 0x7faab8003f70 con 0x56124cb897a0 2024-03-08T21:09:30.599 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.614+0000 7faad0185740 1 -- 172.21.15.107:0/3254017069 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7faab4005040 con 0x56124cb897a0 2024-03-08T21:09:30.599 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.615+0000 7faaa7e00640 1 -- 172.21.15.107:0/3254017069 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7faab801be10 con 0x56124cb897a0 2024-03-08T21:09:30.599 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.615+0000 7faad0185740 10 client.16215 fetch_fsmap learned FSMap version 38 2024-03-08T21:09:30.599 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.615+0000 7faad0185740 1 -- 172.21.15.107:0/3254017069 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x56124cc02930 con 0x56124cb897a0 2024-03-08T21:09:30.599 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.615+0000 7faaa7e00640 1 -- 172.21.15.107:0/3254017069 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7faab4005040 con 0x56124cb897a0 2024-03-08T21:09:30.599 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.615+0000 7faaa7e00640 1 -- 172.21.15.107:0/3254017069 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 38) v1 ==== 7882+0+0 (secure 0 0 0) 0x7faab80108f0 con 0x56124cb897a0 2024-03-08T21:09:30.599 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.615+0000 7faad0185740 10 client.16215 fetch_fsmap finished waiting for FSMap version 38 2024-03-08T21:09:30.599 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.615+0000 7faad0185740 10 client.16215 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-08T21:09:30.600 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.615+0000 7faad0185740 20 client.16215 populate_metadata read hostname 'smithi107' 2024-03-08T21:09:30.600 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.615+0000 7faad0185740 1 --2- 172.21.15.107:0/3254017069 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x56124cc03020 0x56124cc05440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:09:30.600 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.615+0000 7faad0185740 4 client.16215 mds_command: new command op to 34475 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-08T21:09:30.600 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.615+0000 7faad0185740 1 -- 172.21.15.107:0/3254017069 --> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x56124caf3c60 con 0x56124cc03020 2024-03-08T21:09:30.600 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.616+0000 7faac5e00640 1 --2- 172.21.15.107:0/3254017069 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x56124cc03020 0x56124cc05440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:09:30.601 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.617+0000 7faac5e00640 1 --2- 172.21.15.107:0/3254017069 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x56124cc03020 0x56124cc05440 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3789c8538f575ee7 server_cookie=3f681d301b0905dc in_seq=0 out_seq=0 2024-03-08T21:09:30.601 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.617+0000 7faaa7e00640 10 client.16215 ms_handle_connect on v2:172.21.15.149:6838/55290596 2024-03-08T21:09:30.605 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.626+0000 7faaa7e00640 1 -- 172.21.15.107:0/3254017069 <== mds.0 v2:172.21.15.149:6838/55290596 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x56124caf3c60 con 0x56124cc03020 2024-03-08T21:09:30.606 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.626+0000 7faaa7e00640 10 client.16215 handle_command_reply: tid=0 2024-03-08T21:09:30.606 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.626+0000 7faad0185740 1 client.16215 shutdown 2024-03-08T21:09:30.606 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.626+0000 7faad0185740 1 -- 172.21.15.107:0/3254017069 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56124cb897a0 msgr2=0x56124cba1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:09:30.606 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.626+0000 7faad0185740 1 --2- 172.21.15.107:0/3254017069 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56124cb897a0 0x56124cba1340 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7faab8005fa0 tx=0x7faab801ba20 comp rx=0 tx=0).stop 2024-03-08T21:09:30.606 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.626+0000 7faad0185740 1 -- 172.21.15.107:0/3254017069 shutdown_connections 2024-03-08T21:09:30.606 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.626+0000 7faad0185740 1 --2- 172.21.15.107:0/3254017069 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56124cb9ad80 0x56124cc011b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:09:30.606 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.626+0000 7faad0185740 1 --2- 172.21.15.107:0/3254017069 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56124cb897a0 0x56124cba1340 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:09:30.606 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.626+0000 7faad0185740 1 --2- 172.21.15.107:0/3254017069 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x56124cc03020 0x56124cc05440 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:09:30.606 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.626+0000 7faad0185740 1 --2- 172.21.15.107:0/3254017069 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56124cb88e80 0x56124cba0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:09:30.606 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.626+0000 7faad0185740 1 -- 172.21.15.107:0/3254017069 >> 172.21.15.107:0/3254017069 conn(0x56124cb837a0 msgr2=0x56124cb82800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:09:30.606 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.626+0000 7faaa7e00640 0 client.16215 ms_handle_reset on v2:172.21.15.149:6838/55290596 2024-03-08T21:09:30.607 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.627+0000 7faad0185740 1 -- 172.21.15.107:0/3254017069 shutdown_connections 2024-03-08T21:09:30.607 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.627+0000 7faad0185740 1 -- 172.21.15.107:0/3254017069 wait complete. 2024-03-08T21:09:30.607 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.627+0000 7faad0185740 20 client.16215 trim_cache size 0 max 16384 2024-03-08T21:09:30.607 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:{ 2024-03-08T21:09:30.607 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "message": "", 2024-03-08T21:09:30.607 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "return_code": 0 2024-03-08T21:09:30.607 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:} 2024-03-08T21:09:30.608 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.627+0000 7faacca00640 1 -- 172.21.15.107:0/788944308 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7faaac0a2990 msgr2=0x7faaac0a4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:09:30.608 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.628+0000 7faacca00640 1 --2- 172.21.15.107:0/788944308 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7faaac0a2990 0x7faaac0a4e50 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7faac00bd890 tx=0x7faac00095c0 comp rx=0 tx=0).stop 2024-03-08T21:09:30.608 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.628+0000 7faacca00640 1 -- 172.21.15.107:0/788944308 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faac8102750 msgr2=0x7faac81a5ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:09:30.608 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.628+0000 7faacca00640 1 --2- 172.21.15.107:0/788944308 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faac8102750 0x7faac81a5ae0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7faac002c700 tx=0x7faac0035d60 comp rx=0 tx=0).stop 2024-03-08T21:09:30.608 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.628+0000 7faacca00640 1 -- 172.21.15.107:0/788944308 shutdown_connections 2024-03-08T21:09:30.608 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.628+0000 7faacca00640 1 --2- 172.21.15.107:0/788944308 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7faaac0a2990 0x7faaac0a4e50 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:09:30.608 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.628+0000 7faacca00640 1 --2- 172.21.15.107:0/788944308 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faac8102750 0x7faac81a5ae0 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:09:30.608 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.628+0000 7faacca00640 1 --2- 172.21.15.107:0/788944308 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faac8105310 0x7faac8102210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:09:30.608 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.628+0000 7faacca00640 1 --2- 172.21.15.107:0/788944308 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7faac8100240 0x7faac8101cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:09:30.608 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.628+0000 7faacca00640 1 -- 172.21.15.107:0/788944308 >> 172.21.15.107:0/788944308 conn(0x7faac80fbb00 msgr2=0x7faac8068d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:09:30.609 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.628+0000 7faacca00640 1 -- 172.21.15.107:0/788944308 shutdown_connections 2024-03-08T21:09:30.609 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:30.628+0000 7faacca00640 1 -- 172.21.15.107:0/788944308 wait complete. 2024-03-08T21:09:31.010 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:09:30 smithi149 ceph-mon[35042]: pgmap v283: 97 pgs: 97 active+clean; 6.0 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-08T21:09:31.123 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:09:30 smithi204 ceph-mon[34935]: pgmap v283: 97 pgs: 97 active+clean; 6.0 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-08T21:09:31.212 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:09:30 smithi107 ceph-mon[31381]: pgmap v283: 97 pgs: 97 active+clean; 6.0 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-08T21:09:33.010 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:09:32 smithi149 ceph-mon[35042]: pgmap v284: 97 pgs: 97 active+clean; 6.0 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-08T21:09:33.123 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:09:32 smithi204 ceph-mon[34935]: pgmap v284: 97 pgs: 97 active+clean; 6.0 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-08T21:09:33.212 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:09:32 smithi107 ceph-mon[31381]: pgmap v284: 97 pgs: 97 active+clean; 6.0 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-08T21:09:35.010 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:09:34 smithi149 ceph-mon[35042]: pgmap v285: 97 pgs: 97 active+clean; 6.0 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-08T21:09:35.124 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:09:34 smithi204 ceph-mon[34935]: pgmap v285: 97 pgs: 97 active+clean; 6.0 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-08T21:09:35.212 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:09:34 smithi107 ceph-mon[31381]: pgmap v285: 97 pgs: 97 active+clean; 6.0 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-08T21:09:37.010 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:09:36 smithi149 ceph-mon[35042]: pgmap v286: 97 pgs: 97 active+clean; 6.0 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:09:37.123 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:09:36 smithi204 ceph-mon[34935]: pgmap v286: 97 pgs: 97 active+clean; 6.0 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:09:37.212 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:09:36 smithi107 ceph-mon[31381]: pgmap v286: 97 pgs: 97 active+clean; 6.0 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:09:39.010 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:09:38 smithi149 ceph-mon[35042]: pgmap v287: 97 pgs: 97 active+clean; 6.0 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-08T21:09:39.124 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:09:38 smithi204 ceph-mon[34935]: pgmap v287: 97 pgs: 97 active+clean; 6.0 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-08T21:09:39.213 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:09:38 smithi107 ceph-mon[31381]: pgmap v287: 97 pgs: 97 active+clean; 6.0 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-08T21:09:41.010 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:09:40 smithi149 ceph-mon[35042]: pgmap v288: 97 pgs: 97 active+clean; 6.0 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:09:41.123 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:09:40 smithi204 ceph-mon[34935]: pgmap v288: 97 pgs: 97 active+clean; 6.0 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:09:41.213 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:09:40 smithi107 ceph-mon[31381]: pgmap v288: 97 pgs: 97 active+clean; 6.0 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:09:43.010 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:09:42 smithi149 ceph-mon[35042]: pgmap v289: 97 pgs: 97 active+clean; 6.0 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:09:43.124 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:09:42 smithi204 ceph-mon[34935]: pgmap v289: 97 pgs: 97 active+clean; 6.0 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:09:43.212 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:09:42 smithi107 ceph-mon[31381]: pgmap v289: 97 pgs: 97 active+clean; 6.0 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:09:43.758 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.780+0000 7f5a2ee00640 1 -- 172.21.15.107:0/1561378330 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5a28102bc0 msgr2=0x7f5a2810b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:09:43.758 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.780+0000 7f5a2ee00640 1 --2- 172.21.15.107:0/1561378330 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5a28102bc0 0x7f5a2810b0a0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f5a14009900 tx=0x7f5a1402bcd0 comp rx=0 tx=0).stop 2024-03-08T21:09:43.758 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.780+0000 7f5a2ee00640 1 -- 172.21.15.107:0/1561378330 shutdown_connections 2024-03-08T21:09:43.758 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.780+0000 7f5a2ee00640 1 --2- 172.21.15.107:0/1561378330 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5a2810b6c0 0x7f5a2810db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:09:43.759 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.780+0000 7f5a2ee00640 1 --2- 172.21.15.107:0/1561378330 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5a28102bc0 0x7f5a2810b0a0 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:09:43.759 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.780+0000 7f5a2ee00640 1 --2- 172.21.15.107:0/1561378330 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5a281022a0 0x7f5a28102680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:09:43.759 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.780+0000 7f5a2ee00640 1 -- 172.21.15.107:0/1561378330 >> 172.21.15.107:0/1561378330 conn(0x7f5a280fbb00 msgr2=0x7f5a280fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:09:43.759 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.780+0000 7f5a2ee00640 1 -- 172.21.15.107:0/1561378330 shutdown_connections 2024-03-08T21:09:43.759 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.780+0000 7f5a2ee00640 1 -- 172.21.15.107:0/1561378330 wait complete. 2024-03-08T21:09:43.761 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.781+0000 7f5a2ee00640 1 Processor -- start 2024-03-08T21:09:43.761 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.781+0000 7f5a2ee00640 1 -- start start 2024-03-08T21:09:43.761 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.781+0000 7f5a2ee00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5a281022a0 0x7f5a2819f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:09:43.761 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.781+0000 7f5a2ee00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5a28102bc0 0x7f5a2819fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:09:43.761 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.781+0000 7f5a2ee00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5a2810b6c0 0x7f5a281999a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:09:43.761 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.781+0000 7f5a2ee00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a28111380 con 0x7f5a28102bc0 2024-03-08T21:09:43.761 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.781+0000 7f5a2ee00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a28111200 con 0x7f5a281022a0 2024-03-08T21:09:43.762 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.781+0000 7f5a2ee00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a28111500 con 0x7f5a2810b6c0 2024-03-08T21:09:43.762 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.782+0000 7f5a2d000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5a28102bc0 0x7f5a2819fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:09:43.762 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.782+0000 7f5a2d000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5a28102bc0 0x7f5a2819fd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:42534/0 (socket says 172.21.15.107:42534) 2024-03-08T21:09:43.762 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.782+0000 7f5a2d000640 1 -- 172.21.15.107:0/3905748044 learned_addr learned my addr 172.21.15.107:0/3905748044 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:09:43.762 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.782+0000 7f5a2da00640 1 --2- 172.21.15.107:0/3905748044 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5a281022a0 0x7f5a2819f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:09:43.762 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.782+0000 7f5a2e400640 1 --2- 172.21.15.107:0/3905748044 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5a2810b6c0 0x7f5a281999a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:09:43.762 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.782+0000 7f5a2d000640 1 -- 172.21.15.107:0/3905748044 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5a2810b6c0 msgr2=0x7f5a281999a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:09:43.762 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.782+0000 7f5a2d000640 1 --2- 172.21.15.107:0/3905748044 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5a2810b6c0 0x7f5a281999a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:09:43.762 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.782+0000 7f5a2d000640 1 -- 172.21.15.107:0/3905748044 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5a281022a0 msgr2=0x7f5a2819f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:09:43.762 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.782+0000 7f5a2d000640 1 --2- 172.21.15.107:0/3905748044 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5a281022a0 0x7f5a2819f7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:09:43.762 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.782+0000 7f5a2d000640 1 -- 172.21.15.107:0/3905748044 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a2819a120 con 0x7f5a28102bc0 2024-03-08T21:09:43.762 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.782+0000 7f5a2e400640 1 --2- 172.21.15.107:0/3905748044 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5a2810b6c0 0x7f5a281999a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T21:09:43.762 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.783+0000 7f5a2d000640 1 --2- 172.21.15.107:0/3905748044 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5a28102bc0 0x7f5a2819fd30 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f5a14000c00 tx=0x7f5a1402c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:09:43.762 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.783+0000 7f5a26000640 1 -- 172.21.15.107:0/3905748044 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a14043020 con 0x7f5a28102bc0 2024-03-08T21:09:43.763 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.783+0000 7f5a26000640 1 -- 172.21.15.107:0/3905748044 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f5a1402dc60 con 0x7f5a28102bc0 2024-03-08T21:09:43.763 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.783+0000 7f5a2ee00640 1 -- 172.21.15.107:0/3905748044 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a2819a3b0 con 0x7f5a28102bc0 2024-03-08T21:09:43.763 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.783+0000 7f5a26000640 1 -- 172.21.15.107:0/3905748044 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a140039e0 con 0x7f5a28102bc0 2024-03-08T21:09:43.763 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.783+0000 7f5a2ee00640 1 -- 172.21.15.107:0/3905748044 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a281a66b0 con 0x7f5a28102bc0 2024-03-08T21:09:43.765 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.787+0000 7f5a26000640 1 -- 172.21.15.107:0/3905748044 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) v1 ==== 137422+0+0 (secure 0 0 0) 0x7f5a14046020 con 0x7f5a28102bc0 2024-03-08T21:09:43.765 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.787+0000 7f5a331cb740 1 Processor -- start 2024-03-08T21:09:43.765 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.787+0000 7f5a331cb740 1 -- start start 2024-03-08T21:09:43.767 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.787+0000 7f5a331cb740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561019756e80 0x561019757260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:09:43.767 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.787+0000 7f5a331cb740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5610197577a0 0x561019760030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:09:43.767 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.787+0000 7f5a331cb740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561019760570 0x561019762960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:09:43.767 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.787+0000 7f5a331cb740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610196ba3d0 con 0x561019760570 2024-03-08T21:09:43.767 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.787+0000 7f5a331cb740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610196636b0 con 0x5610197577a0 2024-03-08T21:09:43.767 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.787+0000 7f5a331cb740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610196b6470 con 0x561019756e80 2024-03-08T21:09:43.767 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.788+0000 7f5a2d000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561019760570 0x561019762960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:09:43.767 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.788+0000 7f5a2d000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561019760570 0x561019762960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:42548/0 (socket says 172.21.15.107:42548) 2024-03-08T21:09:43.767 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.788+0000 7f5a2d000640 1 -- 172.21.15.107:0/1375575156 learned_addr learned my addr 172.21.15.107:0/1375575156 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:09:43.767 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.788+0000 7f5a2e400640 1 --2- 172.21.15.107:0/1375575156 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561019756e80 0x561019757260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:09:43.767 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.788+0000 7f5a2da00640 1 --2- 172.21.15.107:0/1375575156 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5610197577a0 0x561019760030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:09:43.767 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.788+0000 7f5a26000640 1 --2- 172.21.15.107:0/3905748044 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f5a100a29e0 0x7f5a100a4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:09:43.767 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.788+0000 7f5a2d000640 1 -- 172.21.15.107:0/1375575156 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561019756e80 msgr2=0x561019757260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:09:43.768 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.788+0000 7f5a2d000640 1 --2- 172.21.15.107:0/1375575156 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561019756e80 0x561019757260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:09:43.768 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.788+0000 7f5a2d000640 1 -- 172.21.15.107:0/1375575156 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5610197577a0 msgr2=0x561019760030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:09:43.768 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.788+0000 7f5a2d000640 1 --2- 172.21.15.107:0/1375575156 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5610197577a0 0x561019760030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:09:43.768 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.788+0000 7f5a2d000640 1 -- 172.21.15.107:0/1375575156 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561019689b50 con 0x561019760570 2024-03-08T21:09:43.768 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.789+0000 7f5a2e400640 1 --2- 172.21.15.107:0/1375575156 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561019756e80 0x561019757260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T21:09:43.768 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.789+0000 7f5a26000640 1 -- 172.21.15.107:0/3905748044 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f5a140f0e70 con 0x7f5a28102bc0 2024-03-08T21:09:43.768 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.789+0000 7f5a2e400640 1 --2- 172.21.15.107:0/3905748044 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f5a100a29e0 0x7f5a100a4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:09:43.768 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.789+0000 7f5a2d000640 1 --2- 172.21.15.107:0/1375575156 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561019760570 0x561019762960 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f5a140ee5a0 tx=0x7f5a140e7fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24ec1e02ff9cdf0a server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:09:43.772 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.789+0000 7f5a1aa00640 1 -- 172.21.15.107:0/1375575156 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a140bd1d0 con 0x561019760570 2024-03-08T21:09:43.772 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.789+0000 7f5a1aa00640 1 -- 172.21.15.107:0/1375575156 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f5a140eb8e0 con 0x561019760570 2024-03-08T21:09:43.772 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.789+0000 7f5a1aa00640 1 -- 172.21.15.107:0/1375575156 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a140f4a10 con 0x561019760570 2024-03-08T21:09:43.772 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.790+0000 7f5a331cb740 1 -- 172.21.15.107:0/1375575156 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561019760570 msgr2=0x561019762960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:09:43.772 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.790+0000 7f5a2e400640 1 --2- 172.21.15.107:0/3905748044 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f5a100a29e0 0x7f5a100a4ea0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f5a20005fd0 tx=0x7f5a20004a40 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:09:43.772 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.790+0000 7f5a331cb740 1 --2- 172.21.15.107:0/1375575156 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561019760570 0x561019762960 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f5a140ee5a0 tx=0x7f5a140e7fc0 comp rx=0 tx=0).stop 2024-03-08T21:09:43.772 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.790+0000 7f5a331cb740 1 -- 172.21.15.107:0/1375575156 shutdown_connections 2024-03-08T21:09:43.772 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.790+0000 7f5a331cb740 1 --2- 172.21.15.107:0/1375575156 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561019760570 0x561019762960 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:09:43.772 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.790+0000 7f5a331cb740 1 --2- 172.21.15.107:0/1375575156 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5610197577a0 0x561019760030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:09:43.773 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.790+0000 7f5a331cb740 1 --2- 172.21.15.107:0/1375575156 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561019756e80 0x561019757260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:09:43.773 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.790+0000 7f5a331cb740 1 -- 172.21.15.107:0/1375575156 >> 172.21.15.107:0/1375575156 conn(0x5610197517a0 msgr2=0x561019751b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:09:43.773 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.790+0000 7f5a331cb740 1 -- 172.21.15.107:0/1375575156 shutdown_connections 2024-03-08T21:09:43.773 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.790+0000 7f5a331cb740 1 -- 172.21.15.107:0/1375575156 wait complete. 2024-03-08T21:09:43.773 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.791+0000 7f5a331cb740 1 -- start start 2024-03-08T21:09:43.773 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.791+0000 7f5a331cb740 1 Processor -- start 2024-03-08T21:09:43.773 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.791+0000 7f5a331cb740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561019756e80 0x56101975ff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:09:43.773 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.791+0000 7f5a331cb740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5610197577a0 0x56101975d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:09:43.773 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.791+0000 7f5a331cb740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561019760570 0x56101975d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:09:43.773 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.791+0000 7f5a331cb740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610196ba3d0 con 0x5610197577a0 2024-03-08T21:09:43.773 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.791+0000 7f5a331cb740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610196769f0 con 0x561019756e80 2024-03-08T21:09:43.773 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.791+0000 7f5a331cb740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x561019762f60 con 0x561019760570 2024-03-08T21:09:43.774 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.792+0000 7f5a331cb740 10 client.0 _ll_register_callbacks cb 0x5610193cea00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T21:09:43.774 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.792+0000 7f5a2d000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5610197577a0 0x56101975d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:09:43.774 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.792+0000 7f5a2d000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5610197577a0 0x56101975d2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:42556/0 (socket says 172.21.15.107:42556) 2024-03-08T21:09:43.774 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.792+0000 7f5a2d000640 1 -- 172.21.15.107:0/2474792794 learned_addr learned my addr 172.21.15.107:0/2474792794 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:09:43.774 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.792+0000 7f5a2da00640 1 --2- 172.21.15.107:0/2474792794 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561019756e80 0x56101975ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:09:43.774 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.792+0000 7f5a2e400640 1 --2- 172.21.15.107:0/2474792794 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561019760570 0x56101975d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:09:43.774 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.792+0000 7f5a2d000640 1 -- 172.21.15.107:0/2474792794 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561019760570 msgr2=0x56101975d8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:09:43.774 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.792+0000 7f5a2d000640 1 --2- 172.21.15.107:0/2474792794 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561019760570 0x56101975d8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:09:43.774 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.792+0000 7f5a2d000640 1 -- 172.21.15.107:0/2474792794 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561019756e80 msgr2=0x56101975ff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:09:43.774 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.792+0000 7f5a2d000640 1 --2- 172.21.15.107:0/2474792794 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561019756e80 0x56101975ff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:09:43.774 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.792+0000 7f5a2d000640 1 -- 172.21.15.107:0/2474792794 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5610193cf210 con 0x5610197577a0 2024-03-08T21:09:43.774 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.793+0000 7f5a2d000640 1 --2- 172.21.15.107:0/2474792794 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5610197577a0 0x56101975d2c0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f5a14005ab0 tx=0x7f5a14005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6347bd7468e7f01 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:09:43.774 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.793+0000 7f5a0f400640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-03-08T21:09:43.775 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.793+0000 7f5a0f400640 1 -- 172.21.15.107:0/2474792794 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a140c48f0 con 0x5610197577a0 2024-03-08T21:09:43.775 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.793+0000 7f5a0f400640 1 -- 172.21.15.107:0/2474792794 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f5a140bb990 con 0x5610197577a0 2024-03-08T21:09:43.775 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.793+0000 7f5a331cb740 1 -- 172.21.15.107:0/2474792794 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5610196c7910 con 0x5610197577a0 2024-03-08T21:09:43.775 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.793+0000 7f5a0f400640 1 -- 172.21.15.107:0/2474792794 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5a140bcff0 con 0x5610197577a0 2024-03-08T21:09:43.775 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.793+0000 7f5a331cb740 10 client.16227 fetch_fsmap learned FSMap version 38 2024-03-08T21:09:43.775 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.793+0000 7f5a331cb740 1 -- 172.21.15.107:0/2474792794 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x56101949b620 con 0x5610197577a0 2024-03-08T21:09:43.775 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.793+0000 7f5a0f400640 1 -- 172.21.15.107:0/2474792794 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x5610196c7910 con 0x5610197577a0 2024-03-08T21:09:43.775 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.794+0000 7f5a0f400640 1 -- 172.21.15.107:0/2474792794 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 38) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f5a140ed7a0 con 0x5610197577a0 2024-03-08T21:09:43.775 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.794+0000 7f5a331cb740 10 client.16227 fetch_fsmap finished waiting for FSMap version 38 2024-03-08T21:09:43.775 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.794+0000 7f5a331cb740 10 client.16227 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-08T21:09:43.775 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.794+0000 7f5a331cb740 20 client.16227 populate_metadata read hostname 'smithi107' 2024-03-08T21:09:43.775 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.794+0000 7f5a331cb740 1 --2- 172.21.15.107:0/2474792794 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x56101976ff30 0x561019772320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:09:43.775 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.794+0000 7f5a331cb740 4 client.16227 mds_command: new command op to 34475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-08T21:09:43.776 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.794+0000 7f5a331cb740 1 -- 172.21.15.107:0/2474792794 --> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5610196c1c60 con 0x56101976ff30 2024-03-08T21:09:43.777 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.795+0000 7f5a2da00640 1 --2- 172.21.15.107:0/2474792794 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x56101976ff30 0x561019772320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:09:43.777 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.795+0000 7f5a2da00640 1 --2- 172.21.15.107:0/2474792794 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x56101976ff30 0x561019772320 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=755fe5e2005ea59 server_cookie=512993513c68a3e9 in_seq=0 out_seq=0 2024-03-08T21:09:43.777 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.795+0000 7f5a0f400640 10 client.16227 ms_handle_connect on v2:172.21.15.149:6838/55290596 2024-03-08T21:09:43.777 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.797+0000 7f5a0f400640 1 -- 172.21.15.107:0/2474792794 <== mds.0 v2:172.21.15.149:6838/55290596 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5610196c1c60 con 0x56101976ff30 2024-03-08T21:09:43.777 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.797+0000 7f5a0f400640 10 client.16227 handle_command_reply: tid=0 2024-03-08T21:09:43.777 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.797+0000 7f5a331cb740 1 client.16227 shutdown 2024-03-08T21:09:43.777 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.797+0000 7f5a331cb740 1 -- 172.21.15.107:0/2474792794 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5610197577a0 msgr2=0x56101975d2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:09:43.777 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.797+0000 7f5a331cb740 1 --2- 172.21.15.107:0/2474792794 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5610197577a0 0x56101975d2c0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f5a14005ab0 tx=0x7f5a14005fd0 comp rx=0 tx=0).stop 2024-03-08T21:09:43.778 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.798+0000 7f5a331cb740 1 -- 172.21.15.107:0/2474792794 shutdown_connections 2024-03-08T21:09:43.778 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.798+0000 7f5a331cb740 1 --2- 172.21.15.107:0/2474792794 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561019760570 0x56101975d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:09:43.778 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.798+0000 7f5a331cb740 1 --2- 172.21.15.107:0/2474792794 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5610197577a0 0x56101975d2c0 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:09:43.778 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.798+0000 7f5a331cb740 1 --2- 172.21.15.107:0/2474792794 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x56101976ff30 0x561019772320 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:09:43.778 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.798+0000 7f5a331cb740 1 --2- 172.21.15.107:0/2474792794 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561019756e80 0x56101975ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:09:43.778 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.798+0000 7f5a331cb740 1 -- 172.21.15.107:0/2474792794 >> 172.21.15.107:0/2474792794 conn(0x5610197517a0 msgr2=0x5610197534f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:09:43.778 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.798+0000 7f5a0f400640 0 client.16227 ms_handle_reset on v2:172.21.15.149:6838/55290596 2024-03-08T21:09:43.778 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.798+0000 7f5a331cb740 1 -- 172.21.15.107:0/2474792794 shutdown_connections 2024-03-08T21:09:43.778 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.798+0000 7f5a331cb740 1 -- 172.21.15.107:0/2474792794 wait complete. 2024-03-08T21:09:43.778 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.798+0000 7f5a331cb740 20 client.16227 trim_cache size 0 max 16384 2024-03-08T21:09:43.791 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.813+0000 7f5a331cb740 1 Processor -- start 2024-03-08T21:09:43.791 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.813+0000 7f5a331cb740 1 -- start start 2024-03-08T21:09:43.791 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.813+0000 7f5a331cb740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561019756e80 0x56101975f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:09:43.791 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.813+0000 7f5a331cb740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5610197577a0 0x56101975ff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:09:43.791 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.813+0000 7f5a331cb740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561019760570 0x56101975a560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:09:43.791 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.813+0000 7f5a331cb740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a281aed80 con 0x561019760570 2024-03-08T21:09:43.792 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.813+0000 7f5a331cb740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a281b0700 con 0x561019756e80 2024-03-08T21:09:43.792 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.813+0000 7f5a331cb740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a281af3e0 con 0x5610197577a0 2024-03-08T21:09:43.793 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.814+0000 7f5a2e400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561019760570 0x56101975a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:09:43.793 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.814+0000 7f5a2e400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561019760570 0x56101975a560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:42558/0 (socket says 172.21.15.107:42558) 2024-03-08T21:09:43.793 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.814+0000 7f5a2e400640 1 -- 172.21.15.107:0/4123733980 learned_addr learned my addr 172.21.15.107:0/4123733980 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:09:43.793 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.814+0000 7f5a2da00640 1 --2- 172.21.15.107:0/4123733980 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561019756e80 0x56101975f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:09:43.793 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.814+0000 7f5a2d000640 1 --2- 172.21.15.107:0/4123733980 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5610197577a0 0x56101975ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:09:43.793 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.814+0000 7f5a2e400640 1 -- 172.21.15.107:0/4123733980 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5610197577a0 msgr2=0x56101975ff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:09:43.793 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.814+0000 7f5a2e400640 1 --2- 172.21.15.107:0/4123733980 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5610197577a0 0x56101975ff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:09:43.793 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.814+0000 7f5a2e400640 1 -- 172.21.15.107:0/4123733980 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561019756e80 msgr2=0x56101975f9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:09:43.793 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.814+0000 7f5a2e400640 1 --2- 172.21.15.107:0/4123733980 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561019756e80 0x56101975f9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:09:43.794 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.814+0000 7f5a2e400640 1 -- 172.21.15.107:0/4123733980 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56101975b830 con 0x561019760570 2024-03-08T21:09:43.794 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.815+0000 7f5a2e400640 1 --2- 172.21.15.107:0/4123733980 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561019760570 0x56101975a560 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f5a2001e6a0 tx=0x7f5a20014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=701ccd09a540dc7d server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:09:43.794 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.815+0000 7f5a1a000640 1 -- 172.21.15.107:0/4123733980 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a2001e310 con 0x561019760570 2024-03-08T21:09:43.794 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.815+0000 7f5a1a000640 1 -- 172.21.15.107:0/4123733980 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f5a20012a60 con 0x561019760570 2024-03-08T21:09:43.794 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.815+0000 7f5a1a000640 1 -- 172.21.15.107:0/4123733980 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a2001ac40 con 0x561019760570 2024-03-08T21:09:43.795 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.816+0000 7f5a331cb740 1 -- 172.21.15.107:0/4123733980 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561019760570 msgr2=0x56101975a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:09:43.795 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.816+0000 7f5a331cb740 1 --2- 172.21.15.107:0/4123733980 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561019760570 0x56101975a560 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f5a2001e6a0 tx=0x7f5a20014000 comp rx=0 tx=0).stop 2024-03-08T21:09:43.795 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.816+0000 7f5a331cb740 1 -- 172.21.15.107:0/4123733980 shutdown_connections 2024-03-08T21:09:43.796 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.816+0000 7f5a331cb740 1 --2- 172.21.15.107:0/4123733980 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561019760570 0x56101975a560 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:09:43.796 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.816+0000 7f5a331cb740 1 --2- 172.21.15.107:0/4123733980 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5610197577a0 0x56101975ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:09:43.796 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.816+0000 7f5a331cb740 1 --2- 172.21.15.107:0/4123733980 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561019756e80 0x56101975f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:09:43.796 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.816+0000 7f5a331cb740 1 -- 172.21.15.107:0/4123733980 >> 172.21.15.107:0/4123733980 conn(0x5610197517a0 msgr2=0x561019751b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:09:43.796 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.816+0000 7f5a331cb740 1 -- 172.21.15.107:0/4123733980 shutdown_connections 2024-03-08T21:09:43.796 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.816+0000 7f5a331cb740 1 -- 172.21.15.107:0/4123733980 wait complete. 2024-03-08T21:09:43.796 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.816+0000 7f5a331cb740 1 -- start start 2024-03-08T21:09:43.796 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.817+0000 7f5a331cb740 1 Processor -- start 2024-03-08T21:09:43.796 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.817+0000 7f5a331cb740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561019756e80 0x5610197653b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:09:43.796 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.817+0000 7f5a331cb740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5610197577a0 0x5610197658f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:09:43.796 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.817+0000 7f5a331cb740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561019760570 0x5610197bf2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:09:43.796 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.817+0000 7f5a331cb740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a281b00a0 con 0x5610197577a0 2024-03-08T21:09:43.796 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.817+0000 7f5a331cb740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a281afa40 con 0x561019760570 2024-03-08T21:09:43.796 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.817+0000 7f5a331cb740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a281ad020 con 0x561019756e80 2024-03-08T21:09:43.797 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.817+0000 7f5a331cb740 10 client.0 _ll_register_callbacks cb 0x7f5a140f47f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T21:09:43.797 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.818+0000 7f5a2d000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5610197577a0 0x5610197658f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:09:43.797 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.818+0000 7f5a2d000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5610197577a0 0x5610197658f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:42560/0 (socket says 172.21.15.107:42560) 2024-03-08T21:09:43.797 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.818+0000 7f5a2d000640 1 -- 172.21.15.107:0/2824519651 learned_addr learned my addr 172.21.15.107:0/2824519651 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:09:43.798 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.818+0000 7f5a2da00640 1 --2- 172.21.15.107:0/2824519651 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561019756e80 0x5610197653b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:09:43.798 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.818+0000 7f5a2d000640 1 -- 172.21.15.107:0/2824519651 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561019756e80 msgr2=0x5610197653b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:09:43.799 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.818+0000 7f5a2e400640 1 --2- 172.21.15.107:0/2824519651 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561019760570 0x5610197bf2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:09:43.799 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.818+0000 7f5a2d000640 1 --2- 172.21.15.107:0/2824519651 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561019756e80 0x5610197653b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:09:43.799 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.818+0000 7f5a2d000640 1 -- 172.21.15.107:0/2824519651 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561019760570 msgr2=0x5610197bf2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:09:43.799 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.818+0000 7f5a2d000640 1 --2- 172.21.15.107:0/2824519651 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561019760570 0x5610197bf2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:09:43.799 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.818+0000 7f5a2d000640 1 -- 172.21.15.107:0/2824519651 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5610197bf880 con 0x5610197577a0 2024-03-08T21:09:43.799 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.818+0000 7f5a2da00640 1 --2- 172.21.15.107:0/2824519651 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561019756e80 0x5610197653b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T21:09:43.799 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.818+0000 7f5a2e400640 1 --2- 172.21.15.107:0/2824519651 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561019760570 0x5610197bf2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T21:09:43.799 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.818+0000 7f5a2d000640 1 --2- 172.21.15.107:0/2824519651 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5610197577a0 0x5610197658f0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f5a140b9730 tx=0x7f5a140c18a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0b1f8e505d2d9d0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:09:43.799 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.818+0000 7f5a0f400640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-03-08T21:09:43.799 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.819+0000 7f5a0f400640 1 -- 172.21.15.107:0/2824519651 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a140bd1d0 con 0x5610197577a0 2024-03-08T21:09:43.799 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.819+0000 7f5a0f400640 1 -- 172.21.15.107:0/2824519651 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f5a140bb990 con 0x5610197577a0 2024-03-08T21:09:43.799 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.819+0000 7f5a331cb740 1 -- 172.21.15.107:0/2824519651 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5a20012a60 con 0x5610197577a0 2024-03-08T21:09:43.799 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.819+0000 7f5a0f400640 1 -- 172.21.15.107:0/2824519651 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5a140c9050 con 0x5610197577a0 2024-03-08T21:09:43.799 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.819+0000 7f5a331cb740 10 client.16233 fetch_fsmap learned FSMap version 38 2024-03-08T21:09:43.800 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.819+0000 7f5a331cb740 1 -- 172.21.15.107:0/2824519651 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x5610197bf880 con 0x5610197577a0 2024-03-08T21:09:43.800 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.819+0000 7f5a0f400640 1 -- 172.21.15.107:0/2824519651 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f5a20012a60 con 0x5610197577a0 2024-03-08T21:09:43.800 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.819+0000 7f5a0f400640 1 -- 172.21.15.107:0/2824519651 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 38) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f5a140ee220 con 0x5610197577a0 2024-03-08T21:09:43.800 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.819+0000 7f5a331cb740 10 client.16233 fetch_fsmap finished waiting for FSMap version 38 2024-03-08T21:09:43.800 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.819+0000 7f5a331cb740 10 client.16233 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-08T21:09:43.800 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.820+0000 7f5a331cb740 20 client.16233 populate_metadata read hostname 'smithi107' 2024-03-08T21:09:43.800 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.820+0000 7f5a331cb740 1 --2- 172.21.15.107:0/2824519651 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x5610197c6f90 0x5610197c93b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:09:43.800 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.820+0000 7f5a331cb740 4 client.16233 mds_command: new command op to 34475 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-08T21:09:43.800 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.820+0000 7f5a331cb740 1 -- 172.21.15.107:0/2824519651 --> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5610196405d0 con 0x5610197c6f90 2024-03-08T21:09:43.800 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.820+0000 7f5a2da00640 1 --2- 172.21.15.107:0/2824519651 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x5610197c6f90 0x5610197c93b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:09:43.801 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.821+0000 7f5a2da00640 1 --2- 172.21.15.107:0/2824519651 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x5610197c6f90 0x5610197c93b0 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c9e4a1b45ad97b42 server_cookie=3c52853aaa08753b in_seq=0 out_seq=0 2024-03-08T21:09:43.801 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.821+0000 7f5a0f400640 10 client.16233 ms_handle_connect on v2:172.21.15.149:6838/55290596 2024-03-08T21:09:43.809 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.830+0000 7f5a0f400640 1 -- 172.21.15.107:0/2824519651 <== mds.0 v2:172.21.15.149:6838/55290596 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5610196405d0 con 0x5610197c6f90 2024-03-08T21:09:43.809 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.830+0000 7f5a0f400640 10 client.16233 handle_command_reply: tid=0 2024-03-08T21:09:43.809 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.830+0000 7f5a331cb740 1 client.16233 shutdown 2024-03-08T21:09:43.809 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.831+0000 7f5a331cb740 1 -- 172.21.15.107:0/2824519651 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5610197577a0 msgr2=0x5610197658f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:09:43.809 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.831+0000 7f5a331cb740 1 --2- 172.21.15.107:0/2824519651 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5610197577a0 0x5610197658f0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f5a140b9730 tx=0x7f5a140c18a0 comp rx=0 tx=0).stop 2024-03-08T21:09:43.809 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.831+0000 7f5a331cb740 1 -- 172.21.15.107:0/2824519651 shutdown_connections 2024-03-08T21:09:43.809 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.831+0000 7f5a331cb740 1 --2- 172.21.15.107:0/2824519651 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x5610197c6f90 0x5610197c93b0 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:09:43.809 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.831+0000 7f5a331cb740 1 --2- 172.21.15.107:0/2824519651 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561019760570 0x5610197bf2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:09:43.810 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.831+0000 7f5a331cb740 1 --2- 172.21.15.107:0/2824519651 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5610197577a0 0x5610197658f0 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:09:43.810 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.831+0000 7f5a331cb740 1 --2- 172.21.15.107:0/2824519651 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561019756e80 0x5610197653b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:09:43.810 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.831+0000 7f5a0f400640 0 client.16233 ms_handle_reset on v2:172.21.15.149:6838/55290596 2024-03-08T21:09:43.810 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.831+0000 7f5a331cb740 1 -- 172.21.15.107:0/2824519651 >> 172.21.15.107:0/2824519651 conn(0x5610197517a0 msgr2=0x56101975bbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:09:43.810 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.831+0000 7f5a331cb740 1 -- 172.21.15.107:0/2824519651 shutdown_connections 2024-03-08T21:09:43.810 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.831+0000 7f5a331cb740 1 -- 172.21.15.107:0/2824519651 wait complete. 2024-03-08T21:09:43.810 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.831+0000 7f5a331cb740 20 client.16233 trim_cache size 0 max 16384 2024-03-08T21:09:43.810 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:{ 2024-03-08T21:09:43.810 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "message": "", 2024-03-08T21:09:43.811 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "return_code": 0 2024-03-08T21:09:43.811 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:} 2024-03-08T21:09:43.811 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.832+0000 7f5a2ee00640 1 -- 172.21.15.107:0/3905748044 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f5a100a29e0 msgr2=0x7f5a100a4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:09:43.811 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.832+0000 7f5a2ee00640 1 --2- 172.21.15.107:0/3905748044 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f5a100a29e0 0x7f5a100a4ea0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f5a20005fd0 tx=0x7f5a20004a40 comp rx=0 tx=0).stop 2024-03-08T21:09:43.811 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.832+0000 7f5a2ee00640 1 -- 172.21.15.107:0/3905748044 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5a28102bc0 msgr2=0x7f5a2819fd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:09:43.811 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.832+0000 7f5a2ee00640 1 --2- 172.21.15.107:0/3905748044 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5a28102bc0 0x7f5a2819fd30 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f5a14000c00 tx=0x7f5a1402c760 comp rx=0 tx=0).stop 2024-03-08T21:09:43.812 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.832+0000 7f5a2ee00640 1 -- 172.21.15.107:0/3905748044 shutdown_connections 2024-03-08T21:09:43.812 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.832+0000 7f5a2ee00640 1 --2- 172.21.15.107:0/3905748044 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f5a100a29e0 0x7f5a100a4ea0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:09:43.812 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.832+0000 7f5a2ee00640 1 --2- 172.21.15.107:0/3905748044 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5a2810b6c0 0x7f5a281999a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:09:43.812 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.832+0000 7f5a2ee00640 1 --2- 172.21.15.107:0/3905748044 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5a28102bc0 0x7f5a2819fd30 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:09:43.812 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.832+0000 7f5a2ee00640 1 --2- 172.21.15.107:0/3905748044 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5a281022a0 0x7f5a2819f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:09:43.812 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.832+0000 7f5a2ee00640 1 -- 172.21.15.107:0/3905748044 >> 172.21.15.107:0/3905748044 conn(0x7f5a280fbb00 msgr2=0x7f5a28100fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:09:43.812 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.832+0000 7f5a2ee00640 1 -- 172.21.15.107:0/3905748044 shutdown_connections 2024-03-08T21:09:43.812 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:43.832+0000 7f5a2ee00640 1 -- 172.21.15.107:0/3905748044 wait complete. 2024-03-08T21:09:45.010 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:09:44 smithi149 ceph-mon[35042]: pgmap v290: 97 pgs: 97 active+clean; 6.0 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-08T21:09:45.123 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:09:44 smithi204 ceph-mon[34935]: pgmap v290: 97 pgs: 97 active+clean; 6.0 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-08T21:09:45.212 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:09:44 smithi107 ceph-mon[31381]: pgmap v290: 97 pgs: 97 active+clean; 6.0 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-08T21:09:47.010 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:09:46 smithi149 ceph-mon[35042]: pgmap v291: 97 pgs: 97 active+clean; 6.0 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-08T21:09:47.014 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:09:46 smithi107 ceph-mon[31381]: pgmap v291: 97 pgs: 97 active+clean; 6.0 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-08T21:09:47.123 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:09:46 smithi204 ceph-mon[34935]: pgmap v291: 97 pgs: 97 active+clean; 6.0 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-08T21:09:49.010 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:09:48 smithi149 ceph-mon[35042]: pgmap v292: 97 pgs: 97 active+clean; 6.0 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-08T21:09:49.123 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:09:48 smithi204 ceph-mon[34935]: pgmap v292: 97 pgs: 97 active+clean; 6.0 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-08T21:09:49.212 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:09:48 smithi107 ceph-mon[31381]: pgmap v292: 97 pgs: 97 active+clean; 6.0 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-08T21:09:51.010 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:09:50 smithi149 ceph-mon[35042]: pgmap v293: 97 pgs: 97 active+clean; 6.0 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:09:51.124 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:09:50 smithi204 ceph-mon[34935]: pgmap v293: 97 pgs: 97 active+clean; 6.0 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:09:51.212 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:09:50 smithi107 ceph-mon[31381]: pgmap v293: 97 pgs: 97 active+clean; 6.0 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:09:53.010 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:09:52 smithi149 ceph-mon[35042]: pgmap v294: 97 pgs: 97 active+clean; 6.0 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:09:53.123 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:09:52 smithi204 ceph-mon[34935]: pgmap v294: 97 pgs: 97 active+clean; 6.0 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:09:53.212 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:09:52 smithi107 ceph-mon[31381]: pgmap v294: 97 pgs: 97 active+clean; 6.0 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:09:55.010 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:09:54 smithi149 ceph-mon[35042]: pgmap v295: 97 pgs: 97 active+clean; 6.0 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-08T21:09:55.123 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:09:54 smithi204 ceph-mon[34935]: pgmap v295: 97 pgs: 97 active+clean; 6.0 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-08T21:09:55.212 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:09:54 smithi107 ceph-mon[31381]: pgmap v295: 97 pgs: 97 active+clean; 6.0 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-08T21:09:56.959 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:56.980+0000 7fdb47200640 1 -- 172.21.15.107:0/3981843687 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdb481009b0 msgr2=0x7fdb48104e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:09:56.959 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:56.980+0000 7fdb47200640 1 --2- 172.21.15.107:0/3981843687 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdb481009b0 0x7fdb48104e70 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fdb30009900 tx=0x7fdb3002bd10 comp rx=0 tx=0).stop 2024-03-08T21:09:56.959 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:56.981+0000 7fdb47200640 1 -- 172.21.15.107:0/3981843687 shutdown_connections 2024-03-08T21:09:56.959 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:56.981+0000 7fdb47200640 1 --2- 172.21.15.107:0/3981843687 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdb48105490 0x7fdb48105920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:09:56.959 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:56.981+0000 7fdb47200640 1 --2- 172.21.15.107:0/3981843687 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdb481009b0 0x7fdb48104e70 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:09:56.959 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:56.981+0000 7fdb47200640 1 --2- 172.21.15.107:0/3981843687 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdb48100090 0x7fdb48100470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:09:56.959 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:56.981+0000 7fdb47200640 1 -- 172.21.15.107:0/3981843687 >> 172.21.15.107:0/3981843687 conn(0x7fdb480fbb00 msgr2=0x7fdb480fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:09:56.959 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:56.981+0000 7fdb47200640 1 -- 172.21.15.107:0/3981843687 shutdown_connections 2024-03-08T21:09:56.960 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:56.981+0000 7fdb47200640 1 -- 172.21.15.107:0/3981843687 wait complete. 2024-03-08T21:09:56.960 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:56.982+0000 7fdb47200640 1 Processor -- start 2024-03-08T21:09:56.960 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:56.982+0000 7fdb47200640 1 -- start start 2024-03-08T21:09:56.961 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:56.983+0000 7fdb47200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdb48100090 0x7fdb4819de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:09:56.961 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:56.983+0000 7fdb47200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdb481009b0 0x7fdb4819e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:09:56.962 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:56.983+0000 7fdb47200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdb48105490 0x7fdb481a2740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:09:56.962 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:56.983+0000 7fdb47200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb48117980 con 0x7fdb48100090 2024-03-08T21:09:56.962 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:56.983+0000 7fdb47200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb48117800 con 0x7fdb481009b0 2024-03-08T21:09:56.962 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:56.983+0000 7fdb47200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb48117b00 con 0x7fdb48105490 2024-03-08T21:09:56.962 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:56.983+0000 7fdb45e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdb48100090 0x7fdb4819de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:09:56.962 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:56.983+0000 7fdb45e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdb48100090 0x7fdb4819de80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:55072/0 (socket says 172.21.15.107:55072) 2024-03-08T21:09:56.962 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:56.983+0000 7fdb45e00640 1 -- 172.21.15.107:0/3772391726 learned_addr learned my addr 172.21.15.107:0/3772391726 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:09:56.962 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:56.984+0000 7fdb3fe00640 1 --2- 172.21.15.107:0/3772391726 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdb481009b0 0x7fdb4819e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:09:56.962 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:56.984+0000 7fdb45e00640 1 -- 172.21.15.107:0/3772391726 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdb48105490 msgr2=0x7fdb481a2740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:09:56.962 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:56.984+0000 7fdb46800640 1 --2- 172.21.15.107:0/3772391726 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdb48105490 0x7fdb481a2740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:09:56.962 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:56.984+0000 7fdb45e00640 1 --2- 172.21.15.107:0/3772391726 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdb48105490 0x7fdb481a2740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:09:56.962 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:56.984+0000 7fdb45e00640 1 -- 172.21.15.107:0/3772391726 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdb481009b0 msgr2=0x7fdb4819e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:09:56.962 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:56.984+0000 7fdb45e00640 1 --2- 172.21.15.107:0/3772391726 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdb481009b0 0x7fdb4819e3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:09:56.963 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:56.984+0000 7fdb45e00640 1 -- 172.21.15.107:0/3772391726 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb481a2e70 con 0x7fdb48100090 2024-03-08T21:09:56.963 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:56.984+0000 7fdb3fe00640 1 --2- 172.21.15.107:0/3772391726 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdb481009b0 0x7fdb4819e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T21:09:56.963 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:56.984+0000 7fdb46800640 1 --2- 172.21.15.107:0/3772391726 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdb48105490 0x7fdb481a2740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T21:09:56.963 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:56.984+0000 7fdb45e00640 1 --2- 172.21.15.107:0/3772391726 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdb48100090 0x7fdb4819de80 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fdb3800c920 tx=0x7fdb3800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:09:56.964 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:56.984+0000 7fdb3ea00640 1 -- 172.21.15.107:0/3772391726 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb38007c70 con 0x7fdb48100090 2024-03-08T21:09:56.964 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:56.984+0000 7fdb3ea00640 1 -- 172.21.15.107:0/3772391726 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fdb38007e10 con 0x7fdb48100090 2024-03-08T21:09:56.964 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:56.984+0000 7fdb47200640 1 -- 172.21.15.107:0/3772391726 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb481a3160 con 0x7fdb48100090 2024-03-08T21:09:56.964 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:56.984+0000 7fdb3ea00640 1 -- 172.21.15.107:0/3772391726 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb38018820 con 0x7fdb48100090 2024-03-08T21:09:56.964 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:56.985+0000 7fdb47200640 1 -- 172.21.15.107:0/3772391726 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb481aa9f0 con 0x7fdb48100090 2024-03-08T21:09:56.966 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:56.988+0000 7fdb4f7be740 1 Processor -- start 2024-03-08T21:09:56.966 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:56.989+0000 7fdb3ea00640 1 -- 172.21.15.107:0/3772391726 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) v1 ==== 137422+0+0 (secure 0 0 0) 0x7fdb380189c0 con 0x7fdb48100090 2024-03-08T21:09:56.966 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:56.989+0000 7fdb4f7be740 1 -- start start 2024-03-08T21:09:56.968 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:56.989+0000 7fdb4f7be740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55890ed4ee80 0x55890ed4f260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:09:56.968 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:56.989+0000 7fdb4f7be740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55890ed4f7a0 0x55890ed58030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:09:56.968 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:56.989+0000 7fdb4f7be740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55890ed58570 0x55890ed5a960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:09:56.968 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:56.989+0000 7fdb4f7be740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55890ecb23d0 con 0x55890ed4f7a0 2024-03-08T21:09:56.968 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:56.989+0000 7fdb4f7be740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55890ec5b6b0 con 0x55890ed58570 2024-03-08T21:09:56.968 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:56.989+0000 7fdb4f7be740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55890ecae470 con 0x55890ed4ee80 2024-03-08T21:09:56.969 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:56.989+0000 7fdb45e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55890ed4f7a0 0x55890ed58030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:09:56.969 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:56.990+0000 7fdb45e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55890ed4f7a0 0x55890ed58030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:55088/0 (socket says 172.21.15.107:55088) 2024-03-08T21:09:56.969 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:56.990+0000 7fdb45e00640 1 -- 172.21.15.107:0/4162661876 learned_addr learned my addr 172.21.15.107:0/4162661876 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:09:56.969 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:56.990+0000 7fdb46800640 1 --2- 172.21.15.107:0/4162661876 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55890ed4ee80 0x55890ed4f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:09:56.969 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:56.990+0000 7fdb3fe00640 1 --2- 172.21.15.107:0/4162661876 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55890ed58570 0x55890ed5a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:09:56.969 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:56.990+0000 7fdb45e00640 1 -- 172.21.15.107:0/4162661876 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55890ed4ee80 msgr2=0x55890ed4f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:09:56.969 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:56.990+0000 7fdb45e00640 1 --2- 172.21.15.107:0/4162661876 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55890ed4ee80 0x55890ed4f260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:09:56.969 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:56.990+0000 7fdb45e00640 1 -- 172.21.15.107:0/4162661876 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55890ed58570 msgr2=0x55890ed5a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:09:56.969 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:56.990+0000 7fdb45e00640 1 --2- 172.21.15.107:0/4162661876 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55890ed58570 0x55890ed5a960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:09:56.969 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:56.990+0000 7fdb45e00640 1 -- 172.21.15.107:0/4162661876 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55890ec81b50 con 0x55890ed4f7a0 2024-03-08T21:09:56.969 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:56.990+0000 7fdb46800640 1 --2- 172.21.15.107:0/4162661876 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55890ed4ee80 0x55890ed4f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T21:09:56.969 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:56.990+0000 7fdb3fe00640 1 --2- 172.21.15.107:0/4162661876 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55890ed58570 0x55890ed5a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T21:09:56.969 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:56.990+0000 7fdb3ea00640 1 --2- 172.21.15.107:0/3772391726 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7fdb2c0a29e0 0x7fdb2c0a4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:09:56.969 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:56.991+0000 7fdb45e00640 1 --2- 172.21.15.107:0/4162661876 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55890ed4f7a0 0x55890ed58030 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fdb380cc690 tx=0x7fdb380d7540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ce374c75e83901b server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:09:56.971 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:56.991+0000 7fdb46800640 1 --2- 172.21.15.107:0/3772391726 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7fdb2c0a29e0 0x7fdb2c0a4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:09:56.971 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:56.991+0000 7fdb3ea00640 1 -- 172.21.15.107:0/3772391726 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fdb380cefe0 con 0x7fdb48100090 2024-03-08T21:09:56.971 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:56.991+0000 7fdb37000640 1 -- 172.21.15.107:0/4162661876 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb380978d0 con 0x55890ed4f7a0 2024-03-08T21:09:56.971 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:56.991+0000 7fdb37000640 1 -- 172.21.15.107:0/4162661876 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fdb3800fb00 con 0x55890ed4f7a0 2024-03-08T21:09:56.971 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:56.991+0000 7fdb37000640 1 -- 172.21.15.107:0/4162661876 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb38018cb0 con 0x55890ed4f7a0 2024-03-08T21:09:56.971 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:56.992+0000 7fdb4f7be740 1 -- 172.21.15.107:0/4162661876 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55890ed4f7a0 msgr2=0x55890ed58030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:09:56.971 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:56.992+0000 7fdb46800640 1 --2- 172.21.15.107:0/3772391726 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7fdb2c0a29e0 0x7fdb2c0a4ea0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fdb40005fd0 tx=0x7fdb40008450 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:09:56.971 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:56.992+0000 7fdb4f7be740 1 --2- 172.21.15.107:0/4162661876 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55890ed4f7a0 0x55890ed58030 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fdb380cc690 tx=0x7fdb380d7540 comp rx=0 tx=0).stop 2024-03-08T21:09:56.971 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:56.992+0000 7fdb4f7be740 1 -- 172.21.15.107:0/4162661876 shutdown_connections 2024-03-08T21:09:56.971 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:56.992+0000 7fdb4f7be740 1 --2- 172.21.15.107:0/4162661876 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55890ed58570 0x55890ed5a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:09:56.972 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:56.992+0000 7fdb4f7be740 1 --2- 172.21.15.107:0/4162661876 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55890ed4f7a0 0x55890ed58030 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:09:56.972 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:56.992+0000 7fdb4f7be740 1 --2- 172.21.15.107:0/4162661876 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55890ed4ee80 0x55890ed4f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:09:56.972 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:56.992+0000 7fdb4f7be740 1 -- 172.21.15.107:0/4162661876 >> 172.21.15.107:0/4162661876 conn(0x55890ed497a0 msgr2=0x55890ed49b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:09:56.972 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:56.992+0000 7fdb4f7be740 1 -- 172.21.15.107:0/4162661876 shutdown_connections 2024-03-08T21:09:56.972 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:56.992+0000 7fdb4f7be740 1 -- 172.21.15.107:0/4162661876 wait complete. 2024-03-08T21:09:56.972 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:56.993+0000 7fdb4f7be740 1 -- start start 2024-03-08T21:09:56.972 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:56.993+0000 7fdb4f7be740 1 Processor -- start 2024-03-08T21:09:56.975 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:56.994+0000 7fdb4f7be740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55890ed4ee80 0x55890ed57f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:09:56.975 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:56.994+0000 7fdb4f7be740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55890ed4f7a0 0x55890ed552c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:09:56.975 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:56.994+0000 7fdb4f7be740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55890ed58570 0x55890ed558d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:09:56.975 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:56.994+0000 7fdb4f7be740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55890ed5b1a0 con 0x55890ed4f7a0 2024-03-08T21:09:56.975 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:56.994+0000 7fdb4f7be740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55890ec6e9f0 con 0x55890ed4ee80 2024-03-08T21:09:56.975 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:56.994+0000 7fdb4f7be740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55890ed5b320 con 0x55890ed58570 2024-03-08T21:09:56.975 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:56.994+0000 7fdb4f7be740 10 client.0 _ll_register_callbacks cb 0x55890e9c6a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T21:09:56.975 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:56.994+0000 7fdb3fe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55890ed4f7a0 0x55890ed552c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:09:56.975 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:56.994+0000 7fdb3fe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55890ed4f7a0 0x55890ed552c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:55096/0 (socket says 172.21.15.107:55096) 2024-03-08T21:09:56.976 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:56.994+0000 7fdb3fe00640 1 -- 172.21.15.107:0/4159026397 learned_addr learned my addr 172.21.15.107:0/4159026397 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:09:56.976 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:56.994+0000 7fdb45e00640 1 --2- 172.21.15.107:0/4159026397 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55890ed4ee80 0x55890ed57f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:09:56.976 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:56.994+0000 7fdb46800640 1 --2- 172.21.15.107:0/4159026397 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55890ed58570 0x55890ed558d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:09:56.976 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:56.994+0000 7fdb3fe00640 1 -- 172.21.15.107:0/4159026397 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55890ed58570 msgr2=0x55890ed558d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:09:56.976 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:56.994+0000 7fdb3fe00640 1 --2- 172.21.15.107:0/4159026397 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55890ed58570 0x55890ed558d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:09:56.976 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:56.995+0000 7fdb3fe00640 1 -- 172.21.15.107:0/4159026397 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55890ed4ee80 msgr2=0x55890ed57f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:09:56.976 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:56.995+0000 7fdb3fe00640 1 --2- 172.21.15.107:0/4159026397 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55890ed4ee80 0x55890ed57f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:09:56.976 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:56.995+0000 7fdb3fe00640 1 -- 172.21.15.107:0/4159026397 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55890e9c7210 con 0x55890ed4f7a0 2024-03-08T21:09:56.976 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:56.995+0000 7fdb3fe00640 1 --2- 172.21.15.107:0/4159026397 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55890ed4f7a0 0x55890ed552c0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fdb30009230 tx=0x7fdb30005270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd14d8b312edf1a7 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:09:56.976 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:56.995+0000 7fdb2b400640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-03-08T21:09:56.976 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:56.995+0000 7fdb2b400640 1 -- 172.21.15.107:0/4159026397 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb30043070 con 0x55890ed4f7a0 2024-03-08T21:09:56.976 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:56.995+0000 7fdb2b400640 1 -- 172.21.15.107:0/4159026397 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fdb3002c860 con 0x55890ed4f7a0 2024-03-08T21:09:56.976 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:56.995+0000 7fdb4f7be740 1 -- 172.21.15.107:0/4159026397 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55890ecbf910 con 0x55890ed4f7a0 2024-03-08T21:09:56.976 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:56.996+0000 7fdb2b400640 1 -- 172.21.15.107:0/4159026397 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdb3004f050 con 0x55890ed4f7a0 2024-03-08T21:09:56.977 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:56.996+0000 7fdb4f7be740 10 client.16245 fetch_fsmap learned FSMap version 38 2024-03-08T21:09:56.977 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:56.996+0000 7fdb4f7be740 1 -- 172.21.15.107:0/4159026397 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x55890ea93620 con 0x55890ed4f7a0 2024-03-08T21:09:56.977 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:56.996+0000 7fdb2b400640 1 -- 172.21.15.107:0/4159026397 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x55890ecbf910 con 0x55890ed4f7a0 2024-03-08T21:09:56.977 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:56.996+0000 7fdb2b400640 1 -- 172.21.15.107:0/4159026397 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 38) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fdb30004a40 con 0x55890ed4f7a0 2024-03-08T21:09:56.977 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:56.996+0000 7fdb4f7be740 10 client.16245 fetch_fsmap finished waiting for FSMap version 38 2024-03-08T21:09:56.977 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:56.996+0000 7fdb4f7be740 10 client.16245 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-08T21:09:56.977 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:56.996+0000 7fdb4f7be740 20 client.16245 populate_metadata read hostname 'smithi107' 2024-03-08T21:09:56.977 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:56.996+0000 7fdb4f7be740 1 --2- 172.21.15.107:0/4159026397 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55890ed68410 0x55890ed6a800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:09:56.977 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:56.996+0000 7fdb4f7be740 4 client.16245 mds_command: new command op to 34475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-08T21:09:56.977 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:56.996+0000 7fdb4f7be740 1 -- 172.21.15.107:0/4159026397 --> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55890ecb9c60 con 0x55890ed68410 2024-03-08T21:09:56.977 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:56.997+0000 7fdb45e00640 1 --2- 172.21.15.107:0/4159026397 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55890ed68410 0x55890ed6a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:09:56.979 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:56.998+0000 7fdb2b400640 10 client.16245 ms_handle_connect on v2:172.21.15.149:6838/55290596 2024-03-08T21:09:56.979 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:56.998+0000 7fdb45e00640 1 --2- 172.21.15.107:0/4159026397 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55890ed68410 0x55890ed6a800 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=afbad81de89c941e server_cookie=beefb33359fff86e in_seq=0 out_seq=0 2024-03-08T21:09:56.979 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:56.999+0000 7fdb2b400640 1 -- 172.21.15.107:0/4159026397 <== mds.0 v2:172.21.15.149:6838/55290596 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55890ecb9c60 con 0x55890ed68410 2024-03-08T21:09:56.979 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:56.999+0000 7fdb2b400640 10 client.16245 handle_command_reply: tid=0 2024-03-08T21:09:56.979 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:57.000+0000 7fdb4f7be740 1 client.16245 shutdown 2024-03-08T21:09:56.979 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:57.000+0000 7fdb4f7be740 1 -- 172.21.15.107:0/4159026397 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55890ed4f7a0 msgr2=0x55890ed552c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:09:56.979 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:57.000+0000 7fdb4f7be740 1 --2- 172.21.15.107:0/4159026397 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55890ed4f7a0 0x55890ed552c0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fdb30009230 tx=0x7fdb30005270 comp rx=0 tx=0).stop 2024-03-08T21:09:56.979 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:57.000+0000 7fdb4f7be740 1 -- 172.21.15.107:0/4159026397 shutdown_connections 2024-03-08T21:09:56.979 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:57.000+0000 7fdb4f7be740 1 --2- 172.21.15.107:0/4159026397 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55890ed58570 0x55890ed558d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:09:56.979 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:57.000+0000 7fdb4f7be740 1 --2- 172.21.15.107:0/4159026397 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55890ed4f7a0 0x55890ed552c0 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:09:56.979 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:57.000+0000 7fdb4f7be740 1 --2- 172.21.15.107:0/4159026397 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55890ed68410 0x55890ed6a800 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:09:56.979 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:57.000+0000 7fdb4f7be740 1 --2- 172.21.15.107:0/4159026397 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55890ed4ee80 0x55890ed57f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:09:56.979 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:57.000+0000 7fdb4f7be740 1 -- 172.21.15.107:0/4159026397 >> 172.21.15.107:0/4159026397 conn(0x55890ed497a0 msgr2=0x55890ed4b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:09:56.980 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:57.000+0000 7fdb2b400640 0 client.16245 ms_handle_reset on v2:172.21.15.149:6838/55290596 2024-03-08T21:09:56.980 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:57.000+0000 7fdb4f7be740 1 -- 172.21.15.107:0/4159026397 shutdown_connections 2024-03-08T21:09:56.980 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:57.000+0000 7fdb4f7be740 1 -- 172.21.15.107:0/4159026397 wait complete. 2024-03-08T21:09:56.980 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:57.000+0000 7fdb4f7be740 20 client.16245 trim_cache size 0 max 16384 2024-03-08T21:09:56.999 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:57.020+0000 7fdb4f7be740 1 Processor -- start 2024-03-08T21:09:56.999 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:57.020+0000 7fdb4f7be740 1 -- start start 2024-03-08T21:09:56.999 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:57.020+0000 7fdb4f7be740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55890ed4ee80 0x55890ed579f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:09:56.999 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:57.020+0000 7fdb4f7be740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55890ed4f7a0 0x55890ed57f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:09:56.999 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:57.020+0000 7fdb4f7be740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55890ed58570 0x55890ed52540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:09:56.999 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:57.020+0000 7fdb4f7be740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb481b30a0 con 0x55890ed4ee80 2024-03-08T21:09:56.999 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:57.020+0000 7fdb4f7be740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb481b4a20 con 0x55890ed4f7a0 2024-03-08T21:09:56.999 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:57.020+0000 7fdb4f7be740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb481b3700 con 0x55890ed58570 2024-03-08T21:09:57.000 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:57.021+0000 7fdb45e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55890ed4ee80 0x55890ed579f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:09:57.000 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:57.021+0000 7fdb45e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55890ed4ee80 0x55890ed579f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:55108/0 (socket says 172.21.15.107:55108) 2024-03-08T21:09:57.000 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:57.021+0000 7fdb45e00640 1 -- 172.21.15.107:0/3651606723 learned_addr learned my addr 172.21.15.107:0/3651606723 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:09:57.000 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:57.021+0000 7fdb3fe00640 1 --2- 172.21.15.107:0/3651606723 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55890ed4f7a0 0x55890ed57f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:09:57.000 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:57.021+0000 7fdb46800640 1 --2- 172.21.15.107:0/3651606723 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55890ed58570 0x55890ed52540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:09:57.001 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:57.021+0000 7fdb45e00640 1 -- 172.21.15.107:0/3651606723 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55890ed58570 msgr2=0x55890ed52540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:09:57.001 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:57.021+0000 7fdb45e00640 1 --2- 172.21.15.107:0/3651606723 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55890ed58570 0x55890ed52540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:09:57.001 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:57.021+0000 7fdb45e00640 1 -- 172.21.15.107:0/3651606723 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55890ed4f7a0 msgr2=0x55890ed57f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:09:57.001 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:57.021+0000 7fdb45e00640 1 --2- 172.21.15.107:0/3651606723 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55890ed4f7a0 0x55890ed57f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:09:57.002 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:57.021+0000 7fdb45e00640 1 -- 172.21.15.107:0/3651606723 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55890ed56230 con 0x55890ed4ee80 2024-03-08T21:09:57.002 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:57.021+0000 7fdb3fe00640 1 --2- 172.21.15.107:0/3651606723 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55890ed4f7a0 0x55890ed57f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T21:09:57.002 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:57.022+0000 7fdb45e00640 1 --2- 172.21.15.107:0/3651606723 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55890ed4ee80 0x55890ed579f0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fdb38097a90 tx=0x7fdb38000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=554c16f417bedc9f server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:09:57.002 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:57.022+0000 7fdb36600640 1 -- 172.21.15.107:0/3651606723 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb380b3030 con 0x55890ed4ee80 2024-03-08T21:09:57.002 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:57.022+0000 7fdb36600640 1 -- 172.21.15.107:0/3651606723 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fdb3800de20 con 0x55890ed4ee80 2024-03-08T21:09:57.002 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:57.022+0000 7fdb36600640 1 -- 172.21.15.107:0/3651606723 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb3809af90 con 0x55890ed4ee80 2024-03-08T21:09:57.002 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:57.023+0000 7fdb4f7be740 1 -- 172.21.15.107:0/3651606723 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55890ed4ee80 msgr2=0x55890ed579f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:09:57.002 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:57.023+0000 7fdb4f7be740 1 --2- 172.21.15.107:0/3651606723 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55890ed4ee80 0x55890ed579f0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fdb38097a90 tx=0x7fdb38000bc0 comp rx=0 tx=0).stop 2024-03-08T21:09:57.002 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:57.023+0000 7fdb4f7be740 1 -- 172.21.15.107:0/3651606723 shutdown_connections 2024-03-08T21:09:57.002 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:57.023+0000 7fdb4f7be740 1 --2- 172.21.15.107:0/3651606723 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55890ed58570 0x55890ed52540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:09:57.003 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:57.023+0000 7fdb4f7be740 1 --2- 172.21.15.107:0/3651606723 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55890ed4f7a0 0x55890ed57f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:09:57.003 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:57.023+0000 7fdb4f7be740 1 --2- 172.21.15.107:0/3651606723 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55890ed4ee80 0x55890ed579f0 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:09:57.003 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:57.023+0000 7fdb4f7be740 1 -- 172.21.15.107:0/3651606723 >> 172.21.15.107:0/3651606723 conn(0x55890ed497a0 msgr2=0x55890ed6f180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:09:57.003 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:57.023+0000 7fdb4f7be740 1 -- 172.21.15.107:0/3651606723 shutdown_connections 2024-03-08T21:09:57.003 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:57.023+0000 7fdb4f7be740 1 -- 172.21.15.107:0/3651606723 wait complete. 2024-03-08T21:09:57.006 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:57.024+0000 7fdb4f7be740 1 -- start start 2024-03-08T21:09:57.006 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:57.024+0000 7fdb4f7be740 1 Processor -- start 2024-03-08T21:09:57.006 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:57.025+0000 7fdb4f7be740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55890ed4ee80 0x55890ed5d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:09:57.006 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:57.025+0000 7fdb4f7be740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55890ed4f7a0 0x55890ed5dd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:09:57.007 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:57.025+0000 7fdb4f7be740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55890ed58570 0x55890edb7940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:09:57.007 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:57.025+0000 7fdb4f7be740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb481b43c0 con 0x55890ed4f7a0 2024-03-08T21:09:57.007 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:57.025+0000 7fdb4f7be740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb481b3d60 con 0x55890ed58570 2024-03-08T21:09:57.007 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:57.025+0000 7fdb4f7be740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb481b1340 con 0x55890ed4ee80 2024-03-08T21:09:57.007 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:57.025+0000 7fdb4f7be740 10 client.0 _ll_register_callbacks cb 0x7fdb380d2ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T21:09:57.007 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:57.025+0000 7fdb3fe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55890ed4f7a0 0x55890ed5dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:09:57.007 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:57.025+0000 7fdb3fe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55890ed4f7a0 0x55890ed5dd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:55112/0 (socket says 172.21.15.107:55112) 2024-03-08T21:09:57.007 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:57.025+0000 7fdb3fe00640 1 -- 172.21.15.107:0/4141062557 learned_addr learned my addr 172.21.15.107:0/4141062557 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:09:57.007 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:57.025+0000 7fdb45e00640 1 --2- 172.21.15.107:0/4141062557 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55890ed4ee80 0x55890ed5d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:09:57.008 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:57.025+0000 7fdb46800640 1 --2- 172.21.15.107:0/4141062557 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55890ed58570 0x55890edb7940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:09:57.008 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:57.026+0000 7fdb3fe00640 1 -- 172.21.15.107:0/4141062557 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55890ed4ee80 msgr2=0x55890ed5d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:09:57.008 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:57.026+0000 7fdb3fe00640 1 --2- 172.21.15.107:0/4141062557 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55890ed4ee80 0x55890ed5d7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:09:57.008 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:57.026+0000 7fdb3fe00640 1 -- 172.21.15.107:0/4141062557 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55890ed58570 msgr2=0x55890edb7940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:09:57.008 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:57.026+0000 7fdb3fe00640 1 --2- 172.21.15.107:0/4141062557 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55890ed58570 0x55890edb7940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:09:57.008 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:57.026+0000 7fdb3fe00640 1 -- 172.21.15.107:0/4141062557 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55890edb7f80 con 0x55890ed4f7a0 2024-03-08T21:09:57.008 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:57.026+0000 7fdb3fe00640 1 --2- 172.21.15.107:0/4141062557 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55890ed4f7a0 0x55890ed5dd30 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x55890ed56890 tx=0x7fdb30005670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab91d362b7553f26 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:09:57.008 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:57.026+0000 7fdb2b400640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-03-08T21:09:57.008 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:57.026+0000 7fdb2b400640 1 -- 172.21.15.107:0/4141062557 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb3004c070 con 0x55890ed4f7a0 2024-03-08T21:09:57.008 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:57.026+0000 7fdb2b400640 1 -- 172.21.15.107:0/4141062557 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fdb30003db0 con 0x55890ed4f7a0 2024-03-08T21:09:57.009 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:57.026+0000 7fdb4f7be740 1 -- 172.21.15.107:0/4141062557 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdb3800de20 con 0x55890ed4f7a0 2024-03-08T21:09:57.009 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:57.027+0000 7fdb2b400640 1 -- 172.21.15.107:0/4141062557 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdb30043070 con 0x55890ed4f7a0 2024-03-08T21:09:57.009 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:57.027+0000 7fdb4f7be740 10 client.16251 fetch_fsmap learned FSMap version 38 2024-03-08T21:09:57.009 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:57.027+0000 7fdb4f7be740 1 -- 172.21.15.107:0/4141062557 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x55890edbf340 con 0x55890ed4f7a0 2024-03-08T21:09:57.009 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:57.027+0000 7fdb2b400640 1 -- 172.21.15.107:0/4141062557 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fdb3800de20 con 0x55890ed4f7a0 2024-03-08T21:09:57.009 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:57.027+0000 7fdb2b400640 1 -- 172.21.15.107:0/4141062557 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 38) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fdb3002c860 con 0x55890ed4f7a0 2024-03-08T21:09:57.009 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:57.027+0000 7fdb4f7be740 10 client.16251 fetch_fsmap finished waiting for FSMap version 38 2024-03-08T21:09:57.009 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:57.027+0000 7fdb4f7be740 10 client.16251 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-08T21:09:57.009 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:57.027+0000 7fdb4f7be740 20 client.16251 populate_metadata read hostname 'smithi107' 2024-03-08T21:09:57.010 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:57.027+0000 7fdb4f7be740 1 --2- 172.21.15.107:0/4141062557 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55890edbfa90 0x55890edc1eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:09:57.010 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:57.027+0000 7fdb4f7be740 4 client.16251 mds_command: new command op to 34475 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-08T21:09:57.010 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:57.027+0000 7fdb4f7be740 1 -- 172.21.15.107:0/4141062557 --> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55890ec385d0 con 0x55890edbfa90 2024-03-08T21:09:57.010 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:57.028+0000 7fdb45e00640 1 --2- 172.21.15.107:0/4141062557 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55890edbfa90 0x55890edc1eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:09:57.010 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:57.029+0000 7fdb45e00640 1 --2- 172.21.15.107:0/4141062557 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55890edbfa90 0x55890edc1eb0 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d73a35b000500222 server_cookie=e28a76f756ea82e8 in_seq=0 out_seq=0 2024-03-08T21:09:57.010 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:57.029+0000 7fdb2b400640 10 client.16251 ms_handle_connect on v2:172.21.15.149:6838/55290596 2024-03-08T21:09:57.015 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:57.037+0000 7fdb2b400640 1 -- 172.21.15.107:0/4141062557 <== mds.0 v2:172.21.15.149:6838/55290596 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55890ec385d0 con 0x55890edbfa90 2024-03-08T21:09:57.015 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:57.037+0000 7fdb2b400640 10 client.16251 handle_command_reply: tid=0 2024-03-08T21:09:57.016 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:57.037+0000 7fdb4f7be740 1 client.16251 shutdown 2024-03-08T21:09:57.016 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:57.037+0000 7fdb4f7be740 1 -- 172.21.15.107:0/4141062557 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55890ed4f7a0 msgr2=0x55890ed5dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:09:57.016 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:57.037+0000 7fdb4f7be740 1 --2- 172.21.15.107:0/4141062557 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55890ed4f7a0 0x55890ed5dd30 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x55890ed56890 tx=0x7fdb30005670 comp rx=0 tx=0).stop 2024-03-08T21:09:57.016 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:57.037+0000 7fdb4f7be740 1 -- 172.21.15.107:0/4141062557 shutdown_connections 2024-03-08T21:09:57.016 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:57.037+0000 7fdb4f7be740 1 --2- 172.21.15.107:0/4141062557 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55890edbfa90 0x55890edc1eb0 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:09:57.016 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:57.037+0000 7fdb4f7be740 1 --2- 172.21.15.107:0/4141062557 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55890ed58570 0x55890edb7940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:09:57.016 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:57.037+0000 7fdb4f7be740 1 --2- 172.21.15.107:0/4141062557 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55890ed4f7a0 0x55890ed5dd30 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:09:57.016 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:57.037+0000 7fdb4f7be740 1 --2- 172.21.15.107:0/4141062557 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55890ed4ee80 0x55890ed5d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:09:57.016 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:57.037+0000 7fdb4f7be740 1 -- 172.21.15.107:0/4141062557 >> 172.21.15.107:0/4141062557 conn(0x55890ed497a0 msgr2=0x55890ed53b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:09:57.016 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:57.037+0000 7fdb2b400640 0 client.16251 ms_handle_reset on v2:172.21.15.149:6838/55290596 2024-03-08T21:09:57.016 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:57.038+0000 7fdb4f7be740 1 -- 172.21.15.107:0/4141062557 shutdown_connections 2024-03-08T21:09:57.016 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:57.038+0000 7fdb4f7be740 1 -- 172.21.15.107:0/4141062557 wait complete. 2024-03-08T21:09:57.016 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:57.038+0000 7fdb4f7be740 20 client.16251 trim_cache size 0 max 16384 2024-03-08T21:09:57.017 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:{ 2024-03-08T21:09:57.017 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "message": "", 2024-03-08T21:09:57.017 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "return_code": 0 2024-03-08T21:09:57.017 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:} 2024-03-08T21:09:57.018 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:57.039+0000 7fdb47200640 1 -- 172.21.15.107:0/3772391726 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7fdb2c0a29e0 msgr2=0x7fdb2c0a4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:09:57.018 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:57.039+0000 7fdb47200640 1 --2- 172.21.15.107:0/3772391726 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7fdb2c0a29e0 0x7fdb2c0a4ea0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fdb40005fd0 tx=0x7fdb40008450 comp rx=0 tx=0).stop 2024-03-08T21:09:57.018 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:57.039+0000 7fdb47200640 1 -- 172.21.15.107:0/3772391726 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdb48100090 msgr2=0x7fdb4819de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:09:57.018 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:57.039+0000 7fdb47200640 1 --2- 172.21.15.107:0/3772391726 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdb48100090 0x7fdb4819de80 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fdb3800c920 tx=0x7fdb3800cde0 comp rx=0 tx=0).stop 2024-03-08T21:09:57.018 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:57.039+0000 7fdb47200640 1 -- 172.21.15.107:0/3772391726 shutdown_connections 2024-03-08T21:09:57.018 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:57.039+0000 7fdb47200640 1 --2- 172.21.15.107:0/3772391726 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7fdb2c0a29e0 0x7fdb2c0a4ea0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:09:57.018 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:57.039+0000 7fdb47200640 1 --2- 172.21.15.107:0/3772391726 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdb48105490 0x7fdb481a2740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:09:57.018 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:57.039+0000 7fdb47200640 1 --2- 172.21.15.107:0/3772391726 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdb481009b0 0x7fdb4819e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:09:57.018 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:57.039+0000 7fdb47200640 1 --2- 172.21.15.107:0/3772391726 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdb48100090 0x7fdb4819de80 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:09:57.018 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:57.039+0000 7fdb47200640 1 -- 172.21.15.107:0/3772391726 >> 172.21.15.107:0/3772391726 conn(0x7fdb480fbb00 msgr2=0x7fdb480fcb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:09:57.019 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:57.039+0000 7fdb47200640 1 -- 172.21.15.107:0/3772391726 shutdown_connections 2024-03-08T21:09:57.019 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:09:57.039+0000 7fdb47200640 1 -- 172.21.15.107:0/3772391726 wait complete. 2024-03-08T21:09:57.123 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:09:56 smithi204 ceph-mon[34935]: pgmap v296: 97 pgs: 97 active+clean; 6.0 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:09:57.212 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:09:56 smithi107 ceph-mon[31381]: pgmap v296: 97 pgs: 97 active+clean; 6.0 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:09:57.260 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:09:56 smithi149 ceph-mon[35042]: pgmap v296: 97 pgs: 97 active+clean; 6.0 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:09:58.123 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:09:57 smithi204 ceph-mon[34935]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T21:09:58.212 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:09:57 smithi107 ceph-mon[31381]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T21:09:58.260 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:09:57 smithi149 ceph-mon[35042]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T21:09:59.125 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:09:58 smithi204 ceph-mon[34935]: from='mgr.15375 ' entity='mgr.x' 2024-03-08T21:09:59.126 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:09:58 smithi204 ceph-mon[34935]: from='mgr.15375 ' entity='mgr.x' 2024-03-08T21:09:59.126 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:09:58 smithi204 ceph-mon[34935]: pgmap v297: 97 pgs: 97 active+clean; 6.0 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 511 B/s wr, 13 op/s 2024-03-08T21:09:59.212 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:09:58 smithi107 ceph-mon[31381]: from='mgr.15375 ' entity='mgr.x' 2024-03-08T21:09:59.213 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:09:58 smithi107 ceph-mon[31381]: from='mgr.15375 ' entity='mgr.x' 2024-03-08T21:09:59.213 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:09:58 smithi107 ceph-mon[31381]: pgmap v297: 97 pgs: 97 active+clean; 6.0 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 511 B/s wr, 13 op/s 2024-03-08T21:09:59.260 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:09:58 smithi149 ceph-mon[35042]: from='mgr.15375 ' entity='mgr.x' 2024-03-08T21:09:59.260 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:09:58 smithi149 ceph-mon[35042]: from='mgr.15375 ' entity='mgr.x' 2024-03-08T21:09:59.260 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:09:58 smithi149 ceph-mon[35042]: pgmap v297: 97 pgs: 97 active+clean; 6.0 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 511 B/s wr, 13 op/s 2024-03-08T21:10:00.123 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:09:59 smithi204 ceph-mon[34935]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T21:10:00.124 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:09:59 smithi204 ceph-mon[34935]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T21:10:00.124 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:09:59 smithi204 ceph-mon[34935]: from='mgr.15375 ' entity='mgr.x' 2024-03-08T21:10:00.212 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:09:59 smithi107 ceph-mon[31381]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T21:10:00.213 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:09:59 smithi107 ceph-mon[31381]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T21:10:00.213 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:09:59 smithi107 ceph-mon[31381]: from='mgr.15375 ' entity='mgr.x' 2024-03-08T21:10:00.260 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:09:59 smithi149 ceph-mon[35042]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T21:10:00.260 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:09:59 smithi149 ceph-mon[35042]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T21:10:00.261 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:09:59 smithi149 ceph-mon[35042]: from='mgr.15375 ' entity='mgr.x' 2024-03-08T21:10:01.123 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:10:00 smithi204 ceph-mon[34935]: overall HEALTH_OK 2024-03-08T21:10:01.124 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:10:00 smithi204 ceph-mon[34935]: pgmap v298: 97 pgs: 97 active+clean; 6.0 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:10:01.212 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:10:00 smithi107 ceph-mon[31381]: overall HEALTH_OK 2024-03-08T21:10:01.213 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:10:00 smithi107 ceph-mon[31381]: pgmap v298: 97 pgs: 97 active+clean; 6.0 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:10:01.260 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:10:00 smithi149 ceph-mon[35042]: overall HEALTH_OK 2024-03-08T21:10:01.260 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:10:00 smithi149 ceph-mon[35042]: pgmap v298: 97 pgs: 97 active+clean; 6.0 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:10:03.123 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:10:02 smithi204 ceph-mon[34935]: pgmap v299: 97 pgs: 97 active+clean; 6.0 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:10:03.212 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:10:02 smithi107 ceph-mon[31381]: pgmap v299: 97 pgs: 97 active+clean; 6.0 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:10:03.260 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:10:02 smithi149 ceph-mon[35042]: pgmap v299: 97 pgs: 97 active+clean; 6.0 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:10:05.123 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:10:04 smithi204 ceph-mon[34935]: pgmap v300: 97 pgs: 97 active+clean; 6.7 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 101 KiB/s wr, 30 op/s 2024-03-08T21:10:05.212 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:10:04 smithi107 ceph-mon[31381]: pgmap v300: 97 pgs: 97 active+clean; 6.7 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 101 KiB/s wr, 30 op/s 2024-03-08T21:10:05.260 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:10:04 smithi149 ceph-mon[35042]: pgmap v300: 97 pgs: 97 active+clean; 6.7 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 101 KiB/s wr, 30 op/s 2024-03-08T21:10:07.123 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:10:06 smithi204 ceph-mon[34935]: pgmap v301: 97 pgs: 97 active+clean; 6.7 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 101 KiB/s wr, 25 op/s 2024-03-08T21:10:07.212 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:10:06 smithi107 ceph-mon[31381]: pgmap v301: 97 pgs: 97 active+clean; 6.7 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 101 KiB/s wr, 25 op/s 2024-03-08T21:10:07.260 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:10:06 smithi149 ceph-mon[35042]: pgmap v301: 97 pgs: 97 active+clean; 6.7 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 101 KiB/s wr, 25 op/s 2024-03-08T21:10:09.123 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:10:08 smithi204 ceph-mon[34935]: pgmap v302: 97 pgs: 97 active+clean; 6.7 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 101 KiB/s wr, 30 op/s 2024-03-08T21:10:09.212 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:10:08 smithi107 ceph-mon[31381]: pgmap v302: 97 pgs: 97 active+clean; 6.7 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 101 KiB/s wr, 30 op/s 2024-03-08T21:10:09.260 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:10:08 smithi149 ceph-mon[35042]: pgmap v302: 97 pgs: 97 active+clean; 6.7 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 101 KiB/s wr, 30 op/s 2024-03-08T21:10:10.171 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.193+0000 7fb4f1200640 1 -- 172.21.15.107:0/3402643413 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb4ec105190 msgr2=0x7fb4ec105570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:10:10.171 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.193+0000 7fb4f1200640 1 --2- 172.21.15.107:0/3402643413 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb4ec105190 0x7fb4ec105570 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fb4dc009900 tx=0x7fb4dc02bc50 comp rx=0 tx=0).stop 2024-03-08T21:10:10.171 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.193+0000 7fb4f1200640 1 -- 172.21.15.107:0/3402643413 shutdown_connections 2024-03-08T21:10:10.171 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.193+0000 7fb4f1200640 1 --2- 172.21.15.107:0/3402643413 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb4ec100e00 0x7fb4ec10d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:10:10.171 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.193+0000 7fb4f1200640 1 --2- 172.21.15.107:0/3402643413 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb4ec100440 0x7fb4ec1008c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:10:10.171 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.193+0000 7fb4f1200640 1 --2- 172.21.15.107:0/3402643413 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb4ec105190 0x7fb4ec105570 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:10:10.171 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.193+0000 7fb4f1200640 1 -- 172.21.15.107:0/3402643413 >> 172.21.15.107:0/3402643413 conn(0x7fb4ec077040 msgr2=0x7fb4ec077450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:10:10.172 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.193+0000 7fb4f1200640 1 -- 172.21.15.107:0/3402643413 shutdown_connections 2024-03-08T21:10:10.172 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.193+0000 7fb4f1200640 1 -- 172.21.15.107:0/3402643413 wait complete. 2024-03-08T21:10:10.173 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.194+0000 7fb4f1200640 1 Processor -- start 2024-03-08T21:10:10.173 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.194+0000 7fb4f1200640 1 -- start start 2024-03-08T21:10:10.173 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.195+0000 7fb4f1200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb4ec100440 0x7fb4ec19d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:10:10.173 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.195+0000 7fb4f1200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb4ec100e00 0x7fb4ec19dc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:10:10.174 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.195+0000 7fb4f1200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb4ec105190 0x7fb4ec197840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:10:10.174 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.195+0000 7fb4f1200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4ec110530 con 0x7fb4ec100e00 2024-03-08T21:10:10.174 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.195+0000 7fb4f1200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4ec1103b0 con 0x7fb4ec100440 2024-03-08T21:10:10.174 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.195+0000 7fb4f1200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4ec1106b0 con 0x7fb4ec105190 2024-03-08T21:10:10.174 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.195+0000 7fb4e9400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb4ec100e00 0x7fb4ec19dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:10:10.174 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.195+0000 7fb4e9400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb4ec100e00 0x7fb4ec19dc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:37302/0 (socket says 172.21.15.107:37302) 2024-03-08T21:10:10.174 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.195+0000 7fb4e9400640 1 -- 172.21.15.107:0/423197130 learned_addr learned my addr 172.21.15.107:0/423197130 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:10:10.174 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.195+0000 7fb4e9e00640 1 --2- 172.21.15.107:0/423197130 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb4ec100440 0x7fb4ec19d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:10:10.174 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.195+0000 7fb4ea800640 1 --2- 172.21.15.107:0/423197130 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb4ec105190 0x7fb4ec197840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:10:10.174 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.196+0000 7fb4e9400640 1 -- 172.21.15.107:0/423197130 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb4ec105190 msgr2=0x7fb4ec197840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:10:10.174 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.196+0000 7fb4e9400640 1 --2- 172.21.15.107:0/423197130 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb4ec105190 0x7fb4ec197840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:10:10.174 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.196+0000 7fb4e9400640 1 -- 172.21.15.107:0/423197130 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb4ec100440 msgr2=0x7fb4ec19d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:10:10.175 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.196+0000 7fb4e9400640 1 --2- 172.21.15.107:0/423197130 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb4ec100440 0x7fb4ec19d6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:10:10.175 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.196+0000 7fb4e9400640 1 -- 172.21.15.107:0/423197130 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4ec198010 con 0x7fb4ec100e00 2024-03-08T21:10:10.175 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.196+0000 7fb4e9400640 1 --2- 172.21.15.107:0/423197130 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb4ec100e00 0x7fb4ec19dc10 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fb4d800c920 tx=0x7fb4d800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:10:10.175 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.196+0000 7fb4e2a00640 1 -- 172.21.15.107:0/423197130 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4d8007c70 con 0x7fb4ec100e00 2024-03-08T21:10:10.176 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.196+0000 7fb4e2a00640 1 -- 172.21.15.107:0/423197130 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fb4d8007e10 con 0x7fb4ec100e00 2024-03-08T21:10:10.176 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.196+0000 7fb4e2a00640 1 -- 172.21.15.107:0/423197130 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4d8018820 con 0x7fb4ec100e00 2024-03-08T21:10:10.176 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.196+0000 7fb4f1200640 1 -- 172.21.15.107:0/423197130 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4ec198300 con 0x7fb4ec100e00 2024-03-08T21:10:10.176 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.197+0000 7fb4f1200640 1 -- 172.21.15.107:0/423197130 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4ec1a4520 con 0x7fb4ec100e00 2024-03-08T21:10:10.178 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.200+0000 7fb4e2a00640 1 -- 172.21.15.107:0/423197130 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) v1 ==== 137422+0+0 (secure 0 0 0) 0x7fb4d80189c0 con 0x7fb4ec100e00 2024-03-08T21:10:10.178 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.201+0000 7fb4f3fa2740 1 Processor -- start 2024-03-08T21:10:10.179 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.201+0000 7fb4f3fa2740 1 -- start start 2024-03-08T21:10:10.179 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.201+0000 7fb4f3fa2740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56353f98cc80 0x56353f98d060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:10:10.179 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.201+0000 7fb4f3fa2740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56353f98d5a0 0x56353f995e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:10:10.179 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.201+0000 7fb4f3fa2740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56353f996370 0x56353f998760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:10:10.179 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.201+0000 7fb4f3fa2740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x56353f8f03d0 con 0x56353f996370 2024-03-08T21:10:10.179 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.201+0000 7fb4f3fa2740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x56353f8996b0 con 0x56353f98cc80 2024-03-08T21:10:10.179 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.201+0000 7fb4f3fa2740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56353f8ec470 con 0x56353f98d5a0 2024-03-08T21:10:10.181 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.202+0000 7fb4e9400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56353f996370 0x56353f998760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:10:10.181 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.202+0000 7fb4e9400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56353f996370 0x56353f998760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:37308/0 (socket says 172.21.15.107:37308) 2024-03-08T21:10:10.181 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.202+0000 7fb4e9400640 1 -- 172.21.15.107:0/2999563935 learned_addr learned my addr 172.21.15.107:0/2999563935 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:10:10.181 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.202+0000 7fb4ea800640 1 --2- 172.21.15.107:0/2999563935 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56353f98cc80 0x56353f98d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:10:10.181 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.202+0000 7fb4e9e00640 1 --2- 172.21.15.107:0/2999563935 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56353f98d5a0 0x56353f995e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:10:10.181 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.202+0000 7fb4e2a00640 1 --2- 172.21.15.107:0/423197130 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7fb4d00a2a90 0x7fb4d00a4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:10:10.181 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.202+0000 7fb4e9400640 1 -- 172.21.15.107:0/2999563935 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56353f98d5a0 msgr2=0x56353f995e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:10:10.181 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.202+0000 7fb4e9400640 1 --2- 172.21.15.107:0/2999563935 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56353f98d5a0 0x56353f995e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:10:10.182 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.202+0000 7fb4e9400640 1 -- 172.21.15.107:0/2999563935 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56353f98cc80 msgr2=0x56353f98d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:10:10.182 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.202+0000 7fb4e9400640 1 --2- 172.21.15.107:0/2999563935 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56353f98cc80 0x56353f98d060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:10:10.182 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.202+0000 7fb4e9400640 1 -- 172.21.15.107:0/2999563935 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56353f8bfb50 con 0x56353f996370 2024-03-08T21:10:10.182 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.202+0000 7fb4e9e00640 1 --2- 172.21.15.107:0/2999563935 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56353f98d5a0 0x56353f995e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T21:10:10.182 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.203+0000 7fb4ea800640 1 --2- 172.21.15.107:0/423197130 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7fb4d00a2a90 0x7fb4d00a4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:10:10.182 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.203+0000 7fb4e2a00640 1 -- 172.21.15.107:0/423197130 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 8629+0+0 (secure 0 0 0) 0x7fb4d80cefb0 con 0x7fb4ec100e00 2024-03-08T21:10:10.182 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.203+0000 7fb4e9400640 1 --2- 172.21.15.107:0/2999563935 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56353f996370 0x56353f998760 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fb4d80cc600 tx=0x7fb4d8020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fece466335b3e00 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:10:10.182 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.203+0000 7fb4d7400640 1 -- 172.21.15.107:0/2999563935 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4d809af00 con 0x56353f996370 2024-03-08T21:10:10.182 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.203+0000 7fb4d7400640 1 -- 172.21.15.107:0/2999563935 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fb4d800fb00 con 0x56353f996370 2024-03-08T21:10:10.182 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.203+0000 7fb4d7400640 1 -- 172.21.15.107:0/2999563935 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4d80131b0 con 0x56353f996370 2024-03-08T21:10:10.184 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.204+0000 7fb4f3fa2740 1 -- 172.21.15.107:0/2999563935 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56353f996370 msgr2=0x56353f998760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:10:10.184 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.204+0000 7fb4ea800640 1 --2- 172.21.15.107:0/423197130 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7fb4d00a2a90 0x7fb4d00a4f50 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fb4e4007f20 tx=0x7fb4e4004a40 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:10:10.185 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.204+0000 7fb4f3fa2740 1 --2- 172.21.15.107:0/2999563935 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56353f996370 0x56353f998760 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fb4d80cc600 tx=0x7fb4d8020000 comp rx=0 tx=0).stop 2024-03-08T21:10:10.185 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.204+0000 7fb4f3fa2740 1 -- 172.21.15.107:0/2999563935 shutdown_connections 2024-03-08T21:10:10.185 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.204+0000 7fb4f3fa2740 1 --2- 172.21.15.107:0/2999563935 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56353f996370 0x56353f998760 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:10:10.185 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.204+0000 7fb4f3fa2740 1 --2- 172.21.15.107:0/2999563935 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56353f98d5a0 0x56353f995e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:10:10.185 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.204+0000 7fb4f3fa2740 1 --2- 172.21.15.107:0/2999563935 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56353f98cc80 0x56353f98d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:10:10.185 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.204+0000 7fb4f3fa2740 1 -- 172.21.15.107:0/2999563935 >> 172.21.15.107:0/2999563935 conn(0x56353f9875a0 msgr2=0x56353f987980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:10:10.185 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.204+0000 7fb4f3fa2740 1 -- 172.21.15.107:0/2999563935 shutdown_connections 2024-03-08T21:10:10.185 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.204+0000 7fb4f3fa2740 1 -- 172.21.15.107:0/2999563935 wait complete. 2024-03-08T21:10:10.185 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.205+0000 7fb4f3fa2740 1 -- start start 2024-03-08T21:10:10.185 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.205+0000 7fb4f3fa2740 1 Processor -- start 2024-03-08T21:10:10.185 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.206+0000 7fb4f3fa2740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56353f98cc80 0x56353f99ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:10:10.185 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.206+0000 7fb4f3fa2740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56353f98d5a0 0x56353f9a0520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:10:10.185 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.206+0000 7fb4f3fa2740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56353f996370 0x56353f99a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:10:10.186 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.206+0000 7fb4f3fa2740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x56353f998f40 con 0x56353f98cc80 2024-03-08T21:10:10.186 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.206+0000 7fb4f3fa2740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x56353f8ac9f0 con 0x56353f98d5a0 2024-03-08T21:10:10.186 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.206+0000 7fb4f3fa2740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56353f9990c0 con 0x56353f996370 2024-03-08T21:10:10.186 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.206+0000 7fb4f3fa2740 10 client.0 _ll_register_callbacks cb 0x56353f604a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T21:10:10.186 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.206+0000 7fb4e9e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56353f98cc80 0x56353f99ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:10:10.186 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.206+0000 7fb4e9e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56353f98cc80 0x56353f99ffe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:37320/0 (socket says 172.21.15.107:37320) 2024-03-08T21:10:10.186 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.206+0000 7fb4e9e00640 1 -- 172.21.15.107:0/512428607 learned_addr learned my addr 172.21.15.107:0/512428607 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:10:10.186 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.206+0000 7fb4e9400640 1 --2- 172.21.15.107:0/512428607 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56353f98d5a0 0x56353f9a0520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:10:10.186 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.206+0000 7fb4ea800640 1 --2- 172.21.15.107:0/512428607 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56353f996370 0x56353f99a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:10:10.186 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.206+0000 7fb4e9e00640 1 -- 172.21.15.107:0/512428607 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56353f996370 msgr2=0x56353f99a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:10:10.186 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.206+0000 7fb4e9e00640 1 --2- 172.21.15.107:0/512428607 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56353f996370 0x56353f99a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:10:10.186 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.206+0000 7fb4e9e00640 1 -- 172.21.15.107:0/512428607 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56353f98d5a0 msgr2=0x56353f9a0520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:10:10.186 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.206+0000 7fb4e9e00640 1 --2- 172.21.15.107:0/512428607 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56353f98d5a0 0x56353f9a0520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:10:10.187 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.206+0000 7fb4e9e00640 1 -- 172.21.15.107:0/512428607 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56353f605210 con 0x56353f98cc80 2024-03-08T21:10:10.187 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.207+0000 7fb4e9e00640 1 --2- 172.21.15.107:0/512428607 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56353f98cc80 0x56353f99ffe0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fb4dc000c00 tx=0x7fb4dc02ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42317d05f5493e95 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:10:10.187 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.207+0000 7fb4cbe00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-03-08T21:10:10.188 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.207+0000 7fb4cbe00640 1 -- 172.21.15.107:0/512428607 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4dc02cc20 con 0x56353f98cc80 2024-03-08T21:10:10.188 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.207+0000 7fb4cbe00640 1 -- 172.21.15.107:0/512428607 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 8629+0+0 (secure 0 0 0) 0x7fb4dc005250 con 0x56353f98cc80 2024-03-08T21:10:10.188 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.207+0000 7fb4f3fa2740 1 -- 172.21.15.107:0/512428607 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56353f8fd910 con 0x56353f98cc80 2024-03-08T21:10:10.188 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.208+0000 7fb4cbe00640 1 -- 172.21.15.107:0/512428607 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb4dc005450 con 0x56353f98cc80 2024-03-08T21:10:10.188 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.208+0000 7fb4f3fa2740 10 client.16263 fetch_fsmap learned FSMap version 38 2024-03-08T21:10:10.189 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.208+0000 7fb4f3fa2740 1 -- 172.21.15.107:0/512428607 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x56353f6d1620 con 0x56353f98cc80 2024-03-08T21:10:10.189 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.208+0000 7fb4cbe00640 1 -- 172.21.15.107:0/512428607 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x56353f8fd910 con 0x56353f98cc80 2024-03-08T21:10:10.189 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.208+0000 7fb4cbe00640 1 -- 172.21.15.107:0/512428607 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 38) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fb4dc003720 con 0x56353f98cc80 2024-03-08T21:10:10.189 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.208+0000 7fb4f3fa2740 10 client.16263 fetch_fsmap finished waiting for FSMap version 38 2024-03-08T21:10:10.189 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.208+0000 7fb4f3fa2740 10 client.16263 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-08T21:10:10.189 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.208+0000 7fb4f3fa2740 20 client.16263 populate_metadata read hostname 'smithi107' 2024-03-08T21:10:10.189 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.208+0000 7fb4f3fa2740 1 --2- 172.21.15.107:0/512428607 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x56353f9aa5a0 0x56353f9ac990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:10:10.189 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.208+0000 7fb4f3fa2740 4 client.16263 mds_command: new command op to 34475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-08T21:10:10.189 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.208+0000 7fb4f3fa2740 1 -- 172.21.15.107:0/512428607 --> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56353f8f7c60 con 0x56353f9aa5a0 2024-03-08T21:10:10.189 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.209+0000 7fb4e9400640 1 --2- 172.21.15.107:0/512428607 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x56353f9aa5a0 0x56353f9ac990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:10:10.189 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.210+0000 7fb4e9400640 1 --2- 172.21.15.107:0/512428607 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x56353f9aa5a0 0x56353f9ac990 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1f8d0c09cbeefae8 server_cookie=38fcef25e77e50bd in_seq=0 out_seq=0 2024-03-08T21:10:10.189 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.210+0000 7fb4cbe00640 10 client.16263 ms_handle_connect on v2:172.21.15.149:6838/55290596 2024-03-08T21:10:10.191 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.211+0000 7fb4cbe00640 1 -- 172.21.15.107:0/512428607 <== mds.0 v2:172.21.15.149:6838/55290596 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56353f8f7c60 con 0x56353f9aa5a0 2024-03-08T21:10:10.191 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.211+0000 7fb4cbe00640 10 client.16263 handle_command_reply: tid=0 2024-03-08T21:10:10.191 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.212+0000 7fb4f3fa2740 1 client.16263 shutdown 2024-03-08T21:10:10.191 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.212+0000 7fb4f3fa2740 1 -- 172.21.15.107:0/512428607 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56353f98cc80 msgr2=0x56353f99ffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:10:10.191 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.212+0000 7fb4f3fa2740 1 --2- 172.21.15.107:0/512428607 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56353f98cc80 0x56353f99ffe0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fb4dc000c00 tx=0x7fb4dc02ca30 comp rx=0 tx=0).stop 2024-03-08T21:10:10.191 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.212+0000 7fb4f3fa2740 1 -- 172.21.15.107:0/512428607 shutdown_connections 2024-03-08T21:10:10.191 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.212+0000 7fb4f3fa2740 1 --2- 172.21.15.107:0/512428607 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56353f996370 0x56353f99a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:10:10.191 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.212+0000 7fb4f3fa2740 1 --2- 172.21.15.107:0/512428607 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x56353f9aa5a0 0x56353f9ac990 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:10:10.191 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.212+0000 7fb4f3fa2740 1 --2- 172.21.15.107:0/512428607 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56353f98d5a0 0x56353f9a0520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:10:10.191 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.212+0000 7fb4f3fa2740 1 --2- 172.21.15.107:0/512428607 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56353f98cc80 0x56353f99ffe0 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:10:10.191 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.212+0000 7fb4f3fa2740 1 -- 172.21.15.107:0/512428607 >> 172.21.15.107:0/512428607 conn(0x56353f9875a0 msgr2=0x56353f9892f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:10:10.191 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.212+0000 7fb4cbe00640 0 client.16263 ms_handle_reset on v2:172.21.15.149:6838/55290596 2024-03-08T21:10:10.191 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.212+0000 7fb4f3fa2740 1 -- 172.21.15.107:0/512428607 shutdown_connections 2024-03-08T21:10:10.192 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.212+0000 7fb4f3fa2740 1 -- 172.21.15.107:0/512428607 wait complete. 2024-03-08T21:10:10.192 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.212+0000 7fb4f3fa2740 20 client.16263 trim_cache size 0 max 16384 2024-03-08T21:10:10.205 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.226+0000 7fb4f3fa2740 1 Processor -- start 2024-03-08T21:10:10.205 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.227+0000 7fb4f3fa2740 1 -- start start 2024-03-08T21:10:10.205 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.227+0000 7fb4f3fa2740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56353f98cc80 0x56353f99a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:10:10.205 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.227+0000 7fb4f3fa2740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56353f98d5a0 0x56353f99fb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:10:10.205 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.227+0000 7fb4f3fa2740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56353f996370 0x56353f9a0080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:10:10.205 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.227+0000 7fb4f3fa2740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4ec1aaa80 con 0x56353f98cc80 2024-03-08T21:10:10.205 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.227+0000 7fb4f3fa2740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4ec1ac400 con 0x56353f98d5a0 2024-03-08T21:10:10.206 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.227+0000 7fb4f3fa2740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4ec1ab0e0 con 0x56353f996370 2024-03-08T21:10:10.206 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.227+0000 7fb4e9e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56353f98cc80 0x56353f99a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:10:10.206 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.227+0000 7fb4e9e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56353f98cc80 0x56353f99a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:37326/0 (socket says 172.21.15.107:37326) 2024-03-08T21:10:10.206 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.227+0000 7fb4e9e00640 1 -- 172.21.15.107:0/2444123813 learned_addr learned my addr 172.21.15.107:0/2444123813 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:10:10.207 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.228+0000 7fb4e9400640 1 --2- 172.21.15.107:0/2444123813 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56353f98d5a0 0x56353f99fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:10:10.207 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.228+0000 7fb4e9e00640 1 -- 172.21.15.107:0/2444123813 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56353f996370 msgr2=0x56353f9a0080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:10:10.207 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.228+0000 7fb4e9e00640 1 --2- 172.21.15.107:0/2444123813 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56353f996370 0x56353f9a0080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:10:10.208 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.228+0000 7fb4e9e00640 1 -- 172.21.15.107:0/2444123813 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56353f98d5a0 msgr2=0x56353f99fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:10:10.208 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.228+0000 7fb4e9e00640 1 --2- 172.21.15.107:0/2444123813 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56353f98d5a0 0x56353f99fb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:10:10.208 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.228+0000 7fb4e9e00640 1 -- 172.21.15.107:0/2444123813 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56353f99b740 con 0x56353f98cc80 2024-03-08T21:10:10.208 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.228+0000 7fb4e9e00640 1 --2- 172.21.15.107:0/2444123813 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56353f98cc80 0x56353f99a2f0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fb4dc02c740 tx=0x7fb4dc02bc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69cd6fa83586f355 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:10:10.208 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.228+0000 7fb4d6a00640 1 -- 172.21.15.107:0/2444123813 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4dc009520 con 0x56353f98cc80 2024-03-08T21:10:10.208 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.228+0000 7fb4d6a00640 1 -- 172.21.15.107:0/2444123813 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fb4dc003e10 con 0x56353f98cc80 2024-03-08T21:10:10.208 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.228+0000 7fb4d6a00640 1 -- 172.21.15.107:0/2444123813 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4dc004bb0 con 0x56353f98cc80 2024-03-08T21:10:10.208 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.229+0000 7fb4f3fa2740 1 -- 172.21.15.107:0/2444123813 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56353f98cc80 msgr2=0x56353f99a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:10:10.208 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.229+0000 7fb4f3fa2740 1 --2- 172.21.15.107:0/2444123813 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56353f98cc80 0x56353f99a2f0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fb4dc02c740 tx=0x7fb4dc02bc50 comp rx=0 tx=0).stop 2024-03-08T21:10:10.208 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.229+0000 7fb4f3fa2740 1 -- 172.21.15.107:0/2444123813 shutdown_connections 2024-03-08T21:10:10.208 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.229+0000 7fb4f3fa2740 1 --2- 172.21.15.107:0/2444123813 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56353f996370 0x56353f9a0080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:10:10.208 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.229+0000 7fb4f3fa2740 1 --2- 172.21.15.107:0/2444123813 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56353f98d5a0 0x56353f99fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:10:10.208 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.229+0000 7fb4f3fa2740 1 --2- 172.21.15.107:0/2444123813 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56353f98cc80 0x56353f99a2f0 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:10:10.208 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.229+0000 7fb4f3fa2740 1 -- 172.21.15.107:0/2444123813 >> 172.21.15.107:0/2444123813 conn(0x56353f9875a0 msgr2=0x56353f992a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:10:10.209 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.229+0000 7fb4f3fa2740 1 -- 172.21.15.107:0/2444123813 shutdown_connections 2024-03-08T21:10:10.209 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.229+0000 7fb4f3fa2740 1 -- 172.21.15.107:0/2444123813 wait complete. 2024-03-08T21:10:10.209 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.230+0000 7fb4f3fa2740 1 -- start start 2024-03-08T21:10:10.209 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.230+0000 7fb4f3fa2740 1 Processor -- start 2024-03-08T21:10:10.211 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.230+0000 7fb4f3fa2740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56353f98cc80 0x56353f9fade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:10:10.211 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.230+0000 7fb4f3fa2740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56353f98d5a0 0x56353f9fb320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:10:10.211 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.230+0000 7fb4f3fa2740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56353f996370 0x56353f9f51f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:10:10.211 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.230+0000 7fb4f3fa2740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4ec1abda0 con 0x56353f98cc80 2024-03-08T21:10:10.212 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.230+0000 7fb4f3fa2740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4ec1ab740 con 0x56353f98d5a0 2024-03-08T21:10:10.212 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.230+0000 7fb4f3fa2740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4ec1a8d40 con 0x56353f996370 2024-03-08T21:10:10.212 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.230+0000 7fb4f3fa2740 10 client.0 _ll_register_callbacks cb 0x7fb4d80d29d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T21:10:10.212 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.231+0000 7fb4e9e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56353f98cc80 0x56353f9fade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:10:10.212 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.231+0000 7fb4e9e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56353f98cc80 0x56353f9fade0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:37334/0 (socket says 172.21.15.107:37334) 2024-03-08T21:10:10.212 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.231+0000 7fb4e9e00640 1 -- 172.21.15.107:0/3544131683 learned_addr learned my addr 172.21.15.107:0/3544131683 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:10:10.212 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.231+0000 7fb4e9400640 1 --2- 172.21.15.107:0/3544131683 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56353f98d5a0 0x56353f9fb320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:10:10.212 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.231+0000 7fb4e9e00640 1 -- 172.21.15.107:0/3544131683 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56353f996370 msgr2=0x56353f9f51f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:10:10.212 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.231+0000 7fb4ea800640 1 --2- 172.21.15.107:0/3544131683 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56353f996370 0x56353f9f51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:10:10.212 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.231+0000 7fb4e9e00640 1 --2- 172.21.15.107:0/3544131683 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56353f996370 0x56353f9f51f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:10:10.212 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.231+0000 7fb4e9e00640 1 -- 172.21.15.107:0/3544131683 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56353f98d5a0 msgr2=0x56353f9fb320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:10:10.212 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.231+0000 7fb4e9e00640 1 --2- 172.21.15.107:0/3544131683 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56353f98d5a0 0x56353f9fb320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:10:10.212 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.231+0000 7fb4e9e00640 1 -- 172.21.15.107:0/3544131683 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56353f9f58e0 con 0x56353f98cc80 2024-03-08T21:10:10.213 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.231+0000 7fb4e9400640 1 --2- 172.21.15.107:0/3544131683 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56353f98d5a0 0x56353f9fb320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T21:10:10.213 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.231+0000 7fb4e9e00640 1 --2- 172.21.15.107:0/3544131683 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56353f98cc80 0x56353f9fade0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fb4dc003de0 tx=0x7fb4dc009610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e519a277578f9888 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:10:10.213 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.231+0000 7fb4cbe00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-03-08T21:10:10.213 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.232+0000 7fb4cbe00640 1 -- 172.21.15.107:0/3544131683 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4dc05d020 con 0x56353f98cc80 2024-03-08T21:10:10.213 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.232+0000 7fb4f3fa2740 1 -- 172.21.15.107:0/3544131683 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb4dc003e10 con 0x56353f98cc80 2024-03-08T21:10:10.213 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.232+0000 7fb4cbe00640 1 -- 172.21.15.107:0/3544131683 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 8629+0+0 (secure 0 0 0) 0x7fb4dc0048c0 con 0x56353f98cc80 2024-03-08T21:10:10.213 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.232+0000 7fb4cbe00640 1 -- 172.21.15.107:0/3544131683 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb4dc068050 con 0x56353f98cc80 2024-03-08T21:10:10.213 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.232+0000 7fb4f3fa2740 10 client.16269 fetch_fsmap learned FSMap version 38 2024-03-08T21:10:10.213 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.232+0000 7fb4f3fa2740 1 -- 172.21.15.107:0/3544131683 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x56353fa025f0 con 0x56353f98cc80 2024-03-08T21:10:10.213 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.232+0000 7fb4cbe00640 1 -- 172.21.15.107:0/3544131683 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fb4dc003e10 con 0x56353f98cc80 2024-03-08T21:10:10.213 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.232+0000 7fb4cbe00640 1 -- 172.21.15.107:0/3544131683 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 38) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fb4dc034a40 con 0x56353f98cc80 2024-03-08T21:10:10.213 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.233+0000 7fb4f3fa2740 10 client.16269 fetch_fsmap finished waiting for FSMap version 38 2024-03-08T21:10:10.213 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.233+0000 7fb4f3fa2740 10 client.16269 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-08T21:10:10.214 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.233+0000 7fb4f3fa2740 20 client.16269 populate_metadata read hostname 'smithi107' 2024-03-08T21:10:10.214 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.233+0000 7fb4f3fa2740 1 --2- 172.21.15.107:0/3544131683 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x56353fa02ba0 0x56353fa04fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:10:10.214 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.233+0000 7fb4f3fa2740 4 client.16269 mds_command: new command op to 34475 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-08T21:10:10.214 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.233+0000 7fb4f3fa2740 1 -- 172.21.15.107:0/3544131683 --> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x56353f8765d0 con 0x56353fa02ba0 2024-03-08T21:10:10.214 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.233+0000 7fb4e9400640 1 --2- 172.21.15.107:0/3544131683 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x56353fa02ba0 0x56353fa04fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:10:10.214 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.234+0000 7fb4e9400640 1 --2- 172.21.15.107:0/3544131683 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x56353fa02ba0 0x56353fa04fc0 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a5fed22be137ba88 server_cookie=7f0565fb7ac0f7bc in_seq=0 out_seq=0 2024-03-08T21:10:10.214 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.234+0000 7fb4cbe00640 10 client.16269 ms_handle_connect on v2:172.21.15.149:6838/55290596 2024-03-08T21:10:10.253 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.275+0000 7fb4cbe00640 1 -- 172.21.15.107:0/3544131683 <== mds.0 v2:172.21.15.149:6838/55290596 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x56353f8765d0 con 0x56353fa02ba0 2024-03-08T21:10:10.253 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.275+0000 7fb4cbe00640 10 client.16269 handle_command_reply: tid=0 2024-03-08T21:10:10.254 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.275+0000 7fb4f3fa2740 1 client.16269 shutdown 2024-03-08T21:10:10.254 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.275+0000 7fb4f3fa2740 1 -- 172.21.15.107:0/3544131683 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56353f98cc80 msgr2=0x56353f9fade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:10:10.254 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.275+0000 7fb4f3fa2740 1 --2- 172.21.15.107:0/3544131683 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56353f98cc80 0x56353f9fade0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fb4dc003de0 tx=0x7fb4dc009610 comp rx=0 tx=0).stop 2024-03-08T21:10:10.254 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.275+0000 7fb4f3fa2740 1 -- 172.21.15.107:0/3544131683 shutdown_connections 2024-03-08T21:10:10.254 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.275+0000 7fb4f3fa2740 1 --2- 172.21.15.107:0/3544131683 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56353f996370 0x56353f9f51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:10:10.254 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.275+0000 7fb4f3fa2740 1 --2- 172.21.15.107:0/3544131683 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x56353fa02ba0 0x56353fa04fc0 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:10:10.254 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.275+0000 7fb4f3fa2740 1 --2- 172.21.15.107:0/3544131683 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56353f98d5a0 0x56353f9fb320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:10:10.254 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.275+0000 7fb4f3fa2740 1 --2- 172.21.15.107:0/3544131683 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56353f98cc80 0x56353f9fade0 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:10:10.254 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.275+0000 7fb4cbe00640 0 client.16269 ms_handle_reset on v2:172.21.15.149:6838/55290596 2024-03-08T21:10:10.254 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.275+0000 7fb4f3fa2740 1 -- 172.21.15.107:0/3544131683 >> 172.21.15.107:0/3544131683 conn(0x56353f9875a0 msgr2=0x56353f985e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:10:10.254 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.275+0000 7fb4f3fa2740 1 -- 172.21.15.107:0/3544131683 shutdown_connections 2024-03-08T21:10:10.254 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.275+0000 7fb4f3fa2740 1 -- 172.21.15.107:0/3544131683 wait complete. 2024-03-08T21:10:10.255 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.276+0000 7fb4f3fa2740 20 client.16269 trim_cache size 0 max 16384 2024-03-08T21:10:10.255 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:{ 2024-03-08T21:10:10.255 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "message": "", 2024-03-08T21:10:10.255 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "return_code": 0 2024-03-08T21:10:10.255 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:} 2024-03-08T21:10:10.256 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.276+0000 7fb4f1200640 1 -- 172.21.15.107:0/423197130 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7fb4d00a2a90 msgr2=0x7fb4d00a4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:10:10.257 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.276+0000 7fb4f1200640 1 --2- 172.21.15.107:0/423197130 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7fb4d00a2a90 0x7fb4d00a4f50 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fb4e4007f20 tx=0x7fb4e4004a40 comp rx=0 tx=0).stop 2024-03-08T21:10:10.257 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.276+0000 7fb4f1200640 1 -- 172.21.15.107:0/423197130 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb4ec100e00 msgr2=0x7fb4ec19dc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:10:10.257 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.276+0000 7fb4f1200640 1 --2- 172.21.15.107:0/423197130 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb4ec100e00 0x7fb4ec19dc10 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fb4d800c920 tx=0x7fb4d800cde0 comp rx=0 tx=0).stop 2024-03-08T21:10:10.257 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.277+0000 7fb4f1200640 1 -- 172.21.15.107:0/423197130 shutdown_connections 2024-03-08T21:10:10.257 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.277+0000 7fb4f1200640 1 --2- 172.21.15.107:0/423197130 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7fb4d00a2a90 0x7fb4d00a4f50 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:10:10.257 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.277+0000 7fb4f1200640 1 --2- 172.21.15.107:0/423197130 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb4ec105190 0x7fb4ec197840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:10:10.257 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.277+0000 7fb4f1200640 1 --2- 172.21.15.107:0/423197130 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb4ec100e00 0x7fb4ec19dc10 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:10:10.258 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.277+0000 7fb4f1200640 1 --2- 172.21.15.107:0/423197130 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb4ec100440 0x7fb4ec19d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:10:10.258 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.277+0000 7fb4f1200640 1 -- 172.21.15.107:0/423197130 >> 172.21.15.107:0/423197130 conn(0x7fb4ec077040 msgr2=0x7fb4ec103fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:10:10.258 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.277+0000 7fb4f1200640 1 -- 172.21.15.107:0/423197130 shutdown_connections 2024-03-08T21:10:10.258 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:10.277+0000 7fb4f1200640 1 -- 172.21.15.107:0/423197130 wait complete. 2024-03-08T21:10:11.123 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:10:10 smithi204 ceph-mon[34935]: osdmap e101: 12 total, 12 up, 12 in 2024-03-08T21:10:11.124 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:10:10 smithi204 ceph-mon[34935]: pgmap v304: 97 pgs: 97 active+clean; 6.7 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 121 KiB/s wr, 30 op/s 2024-03-08T21:10:11.212 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:10:10 smithi107 ceph-mon[31381]: osdmap e101: 12 total, 12 up, 12 in 2024-03-08T21:10:11.213 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:10:10 smithi107 ceph-mon[31381]: pgmap v304: 97 pgs: 97 active+clean; 6.7 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 121 KiB/s wr, 30 op/s 2024-03-08T21:10:11.260 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:10:10 smithi149 ceph-mon[35042]: osdmap e101: 12 total, 12 up, 12 in 2024-03-08T21:10:11.260 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:10:10 smithi149 ceph-mon[35042]: pgmap v304: 97 pgs: 97 active+clean; 6.7 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 121 KiB/s wr, 30 op/s 2024-03-08T21:10:13.123 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:10:12 smithi204 ceph-mon[34935]: pgmap v305: 97 pgs: 97 active+clean; 6.7 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 121 KiB/s wr, 30 op/s 2024-03-08T21:10:13.212 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:10:12 smithi107 ceph-mon[31381]: pgmap v305: 97 pgs: 97 active+clean; 6.7 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 121 KiB/s wr, 30 op/s 2024-03-08T21:10:13.260 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:10:12 smithi149 ceph-mon[35042]: pgmap v305: 97 pgs: 97 active+clean; 6.7 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 121 KiB/s wr, 30 op/s 2024-03-08T21:10:15.063 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:10:14 smithi149 ceph-mon[35042]: pgmap v306: 97 pgs: 97 active+clean; 6.7 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 3.7 KiB/s wr, 13 op/s 2024-03-08T21:10:15.123 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:10:14 smithi204 ceph-mon[34935]: pgmap v306: 97 pgs: 97 active+clean; 6.7 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 3.7 KiB/s wr, 13 op/s 2024-03-08T21:10:15.212 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:10:14 smithi107 ceph-mon[31381]: pgmap v306: 97 pgs: 97 active+clean; 6.7 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 3.7 KiB/s wr, 13 op/s 2024-03-08T21:10:17.016 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:10:16 smithi107 ceph-mon[31381]: pgmap v307: 97 pgs: 97 active+clean; 6.7 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 3.7 KiB/s wr, 13 op/s 2024-03-08T21:10:17.123 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:10:16 smithi204 ceph-mon[34935]: pgmap v307: 97 pgs: 97 active+clean; 6.7 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 3.7 KiB/s wr, 13 op/s 2024-03-08T21:10:17.210 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:10:16 smithi149 ceph-mon[35042]: pgmap v307: 97 pgs: 97 active+clean; 6.7 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 3.7 KiB/s wr, 13 op/s 2024-03-08T21:10:20.462 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:10:20 smithi107 ceph-mon[31381]: pgmap v308: 97 pgs: 97 active+clean; 6.7 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 6.7 KiB/s wr, 13 op/s 2024-03-08T21:10:20.463 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:10:20 smithi107 ceph-mon[31381]: osdmap e102: 12 total, 12 up, 12 in 2024-03-08T21:10:20.510 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:10:20 smithi149 ceph-mon[35042]: pgmap v308: 97 pgs: 97 active+clean; 6.7 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 6.7 KiB/s wr, 13 op/s 2024-03-08T21:10:20.511 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:10:20 smithi149 ceph-mon[35042]: osdmap e102: 12 total, 12 up, 12 in 2024-03-08T21:10:20.623 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:10:20 smithi204 ceph-mon[34935]: pgmap v308: 97 pgs: 97 active+clean; 6.7 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 6.7 KiB/s wr, 13 op/s 2024-03-08T21:10:20.623 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:10:20 smithi204 ceph-mon[34935]: osdmap e102: 12 total, 12 up, 12 in 2024-03-08T21:10:22.462 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:10:22 smithi107 ceph-mon[31381]: pgmap v310: 97 pgs: 97 active+clean; 6.7 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 6.7 KiB/s wr, 13 op/s 2024-03-08T21:10:22.510 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:10:22 smithi149 ceph-mon[35042]: pgmap v310: 97 pgs: 97 active+clean; 6.7 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 6.7 KiB/s wr, 13 op/s 2024-03-08T21:10:22.623 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:10:22 smithi204 ceph-mon[34935]: pgmap v310: 97 pgs: 97 active+clean; 6.7 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 6.7 KiB/s wr, 13 op/s 2024-03-08T21:10:23.413 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.433+0000 7f5563200640 1 -- 172.21.15.107:0/79577974 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f556410b6c0 msgr2=0x7f556410db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:10:23.413 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.433+0000 7f5563200640 1 --2- 172.21.15.107:0/79577974 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f556410b6c0 0x7f556410db60 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f555c009ca0 tx=0x7f555c02c180 comp rx=0 tx=0).stop 2024-03-08T21:10:23.413 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.434+0000 7f5563200640 1 -- 172.21.15.107:0/79577974 shutdown_connections 2024-03-08T21:10:23.413 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.434+0000 7f5563200640 1 --2- 172.21.15.107:0/79577974 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f556410b6c0 0x7f556410db60 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:10:23.413 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.434+0000 7f5563200640 1 --2- 172.21.15.107:0/79577974 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5564102bc0 0x7f556410b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:10:23.413 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.434+0000 7f5563200640 1 --2- 172.21.15.107:0/79577974 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f55641022a0 0x7f5564102680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:10:23.413 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.434+0000 7f5563200640 1 -- 172.21.15.107:0/79577974 >> 172.21.15.107:0/79577974 conn(0x7f55640fbb00 msgr2=0x7f55640fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:10:23.413 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.434+0000 7f5563200640 1 -- 172.21.15.107:0/79577974 shutdown_connections 2024-03-08T21:10:23.413 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.434+0000 7f5563200640 1 -- 172.21.15.107:0/79577974 wait complete. 2024-03-08T21:10:23.415 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.435+0000 7f5563200640 1 Processor -- start 2024-03-08T21:10:23.415 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.435+0000 7f5563200640 1 -- start start 2024-03-08T21:10:23.416 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.435+0000 7f5563200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f55641022a0 0x7f556419f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:10:23.416 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.435+0000 7f5563200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5564102bc0 0x7f556419fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:10:23.416 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.435+0000 7f5563200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f556410b6c0 0x7f55641999d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:10:23.416 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.435+0000 7f5563200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5564111350 con 0x7f5564102bc0 2024-03-08T21:10:23.416 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.435+0000 7f5563200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55641111d0 con 0x7f556410b6c0 2024-03-08T21:10:23.416 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.435+0000 7f5563200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55641114d0 con 0x7f55641022a0 2024-03-08T21:10:23.416 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.436+0000 7f5561400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5564102bc0 0x7f556419fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:10:23.416 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.436+0000 7f5561400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5564102bc0 0x7f556419fc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:35434/0 (socket says 172.21.15.107:35434) 2024-03-08T21:10:23.416 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.436+0000 7f5561400640 1 -- 172.21.15.107:0/1897194609 learned_addr learned my addr 172.21.15.107:0/1897194609 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:10:23.416 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.436+0000 7f5562800640 1 --2- 172.21.15.107:0/1897194609 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f556410b6c0 0x7f55641999d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:10:23.416 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.436+0000 7f5561e00640 1 --2- 172.21.15.107:0/1897194609 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f55641022a0 0x7f556419f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:10:23.417 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.436+0000 7f5561400640 1 -- 172.21.15.107:0/1897194609 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f55641022a0 msgr2=0x7f556419f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:10:23.417 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.436+0000 7f5561400640 1 --2- 172.21.15.107:0/1897194609 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f55641022a0 0x7f556419f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:10:23.417 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.436+0000 7f5561400640 1 -- 172.21.15.107:0/1897194609 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f556410b6c0 msgr2=0x7f55641999d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:10:23.417 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.436+0000 7f5561400640 1 --2- 172.21.15.107:0/1897194609 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f556410b6c0 0x7f55641999d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:10:23.417 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.436+0000 7f5561400640 1 -- 172.21.15.107:0/1897194609 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f556419a150 con 0x7f5564102bc0 2024-03-08T21:10:23.417 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.436+0000 7f5561400640 1 --2- 172.21.15.107:0/1897194609 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5564102bc0 0x7f556419fc70 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f555000e9f0 tx=0x7f555000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:10:23.417 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.437+0000 7f555aa00640 1 -- 172.21.15.107:0/1897194609 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f555000ccc0 con 0x7f5564102bc0 2024-03-08T21:10:23.417 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.437+0000 7f555aa00640 1 -- 172.21.15.107:0/1897194609 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f555000ce60 con 0x7f5564102bc0 2024-03-08T21:10:23.417 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.437+0000 7f5563200640 1 -- 172.21.15.107:0/1897194609 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f556419a440 con 0x7f5564102bc0 2024-03-08T21:10:23.417 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.437+0000 7f555aa00640 1 -- 172.21.15.107:0/1897194609 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5550010430 con 0x7f5564102bc0 2024-03-08T21:10:23.417 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.437+0000 7f5563200640 1 -- 172.21.15.107:0/1897194609 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55641a65f0 con 0x7f5564102bc0 2024-03-08T21:10:23.420 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.441+0000 7f556b5db740 1 Processor -- start 2024-03-08T21:10:23.420 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.441+0000 7f556b5db740 1 -- start start 2024-03-08T21:10:23.420 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.441+0000 7f555aa00640 1 -- 172.21.15.107:0/1897194609 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) v1 ==== 137422+0+0 (secure 0 0 0) 0x7f5550005020 con 0x7f5564102bc0 2024-03-08T21:10:23.420 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.441+0000 7f556b5db740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557006527e80 0x557006528260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:10:23.420 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.441+0000 7f556b5db740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5570065287a0 0x557006531030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:10:23.420 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.441+0000 7f556b5db740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557006531570 0x557006533960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:10:23.420 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.441+0000 7f556b5db740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55700648b3d0 con 0x557006531570 2024-03-08T21:10:23.420 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.441+0000 7f556b5db740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5570064346b0 con 0x557006527e80 2024-03-08T21:10:23.420 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.441+0000 7f556b5db740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x557006487470 con 0x5570065287a0 2024-03-08T21:10:23.422 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.442+0000 7f5561400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557006531570 0x557006533960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:10:23.422 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.442+0000 7f5561400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557006531570 0x557006533960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:35438/0 (socket says 172.21.15.107:35438) 2024-03-08T21:10:23.422 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.442+0000 7f5561400640 1 -- 172.21.15.107:0/4264511139 learned_addr learned my addr 172.21.15.107:0/4264511139 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:10:23.422 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.442+0000 7f5562800640 1 --2- 172.21.15.107:0/4264511139 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557006527e80 0x557006528260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:10:23.422 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.442+0000 7f5561e00640 1 --2- 172.21.15.107:0/4264511139 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5570065287a0 0x557006531030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:10:23.422 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.442+0000 7f5561400640 1 -- 172.21.15.107:0/4264511139 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5570065287a0 msgr2=0x557006531030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:10:23.423 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.442+0000 7f5561400640 1 --2- 172.21.15.107:0/4264511139 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5570065287a0 0x557006531030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:10:23.423 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.442+0000 7f5561400640 1 -- 172.21.15.107:0/4264511139 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557006527e80 msgr2=0x557006528260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:10:23.423 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.442+0000 7f5561400640 1 --2- 172.21.15.107:0/4264511139 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557006527e80 0x557006528260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:10:23.423 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.442+0000 7f5561400640 1 -- 172.21.15.107:0/4264511139 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55700645ab50 con 0x557006531570 2024-03-08T21:10:23.423 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.443+0000 7f5561400640 1 --2- 172.21.15.107:0/4264511139 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557006531570 0x557006533960 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f55500ccae0 tx=0x7f55500d5950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=825ed6d1b7599e54 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:10:23.423 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.443+0000 7f554f000640 1 -- 172.21.15.107:0/4264511139 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55500e2070 con 0x557006531570 2024-03-08T21:10:23.423 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.443+0000 7f555aa00640 1 --2- 172.21.15.107:0/1897194609 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f55480a2960 0x7f55480a4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:10:23.423 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.443+0000 7f554f000640 1 -- 172.21.15.107:0/4264511139 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f5550099b20 con 0x557006531570 2024-03-08T21:10:23.423 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.443+0000 7f554f000640 1 -- 172.21.15.107:0/4264511139 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f555009a780 con 0x557006531570 2024-03-08T21:10:23.423 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.443+0000 7f555aa00640 1 -- 172.21.15.107:0/1897194609 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f55500cf460 con 0x7f5564102bc0 2024-03-08T21:10:23.423 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.443+0000 7f5562800640 1 --2- 172.21.15.107:0/1897194609 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f55480a2960 0x7f55480a4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:10:23.427 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.444+0000 7f556b5db740 1 -- 172.21.15.107:0/4264511139 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557006531570 msgr2=0x557006533960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:10:23.427 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.444+0000 7f556b5db740 1 --2- 172.21.15.107:0/4264511139 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557006531570 0x557006533960 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f55500ccae0 tx=0x7f55500d5950 comp rx=0 tx=0).stop 2024-03-08T21:10:23.427 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.444+0000 7f556b5db740 1 -- 172.21.15.107:0/4264511139 shutdown_connections 2024-03-08T21:10:23.427 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.444+0000 7f556b5db740 1 --2- 172.21.15.107:0/4264511139 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557006531570 0x557006533960 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:10:23.427 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.444+0000 7f556b5db740 1 --2- 172.21.15.107:0/4264511139 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5570065287a0 0x557006531030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:10:23.428 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.444+0000 7f556b5db740 1 --2- 172.21.15.107:0/4264511139 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557006527e80 0x557006528260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:10:23.428 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.444+0000 7f556b5db740 1 -- 172.21.15.107:0/4264511139 >> 172.21.15.107:0/4264511139 conn(0x5570065227a0 msgr2=0x557006522b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:10:23.428 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.444+0000 7f556b5db740 1 -- 172.21.15.107:0/4264511139 shutdown_connections 2024-03-08T21:10:23.428 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.444+0000 7f5562800640 1 --2- 172.21.15.107:0/1897194609 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f55480a2960 0x7f55480a4e20 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f555c009430 tx=0x7f555c02e040 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:10:23.428 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.444+0000 7f556b5db740 1 -- 172.21.15.107:0/4264511139 wait complete. 2024-03-08T21:10:23.428 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.445+0000 7f556b5db740 1 -- start start 2024-03-08T21:10:23.428 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.445+0000 7f556b5db740 1 Processor -- start 2024-03-08T21:10:23.428 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.445+0000 7f556b5db740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557006527e80 0x557006530e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:10:23.428 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.445+0000 7f556b5db740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5570065287a0 0x55700652e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:10:23.428 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.445+0000 7f556b5db740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557006531570 0x55700652e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:10:23.428 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.445+0000 7f556b5db740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x5570065341a0 con 0x5570065287a0 2024-03-08T21:10:23.429 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.446+0000 7f556b5db740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5570064479f0 con 0x557006527e80 2024-03-08T21:10:23.429 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.446+0000 7f556b5db740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x557006534320 con 0x557006531570 2024-03-08T21:10:23.429 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.446+0000 7f556b5db740 10 client.0 _ll_register_callbacks cb 0x55700619fa00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T21:10:23.429 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.446+0000 7f5561400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5570065287a0 0x55700652e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:10:23.429 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.446+0000 7f5561400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5570065287a0 0x55700652e220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:35448/0 (socket says 172.21.15.107:35448) 2024-03-08T21:10:23.429 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.446+0000 7f5561400640 1 -- 172.21.15.107:0/3107289903 learned_addr learned my addr 172.21.15.107:0/3107289903 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:10:23.429 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.446+0000 7f5561e00640 1 --2- 172.21.15.107:0/3107289903 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557006527e80 0x557006530e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:10:23.429 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.446+0000 7f5562800640 1 --2- 172.21.15.107:0/3107289903 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557006531570 0x55700652e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:10:23.429 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.446+0000 7f5561400640 1 -- 172.21.15.107:0/3107289903 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557006531570 msgr2=0x55700652e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:10:23.429 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.446+0000 7f5561400640 1 --2- 172.21.15.107:0/3107289903 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557006531570 0x55700652e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:10:23.429 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.446+0000 7f5561400640 1 -- 172.21.15.107:0/3107289903 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557006527e80 msgr2=0x557006530e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:10:23.429 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.446+0000 7f5561400640 1 --2- 172.21.15.107:0/3107289903 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557006527e80 0x557006530e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:10:23.429 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.446+0000 7f5561400640 1 -- 172.21.15.107:0/3107289903 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5570061a0210 con 0x5570065287a0 2024-03-08T21:10:23.429 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.447+0000 7f5561400640 1 --2- 172.21.15.107:0/3107289903 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5570065287a0 0x55700652e220 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f555000e9c0 tx=0x7f55500056f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42b0a4852b40e873 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:10:23.430 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.447+0000 7f5543400640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-03-08T21:10:23.430 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.447+0000 7f5543400640 1 -- 172.21.15.107:0/3107289903 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5550099960 con 0x5570065287a0 2024-03-08T21:10:23.430 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.447+0000 7f5543400640 1 -- 172.21.15.107:0/3107289903 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f5550099d80 con 0x5570065287a0 2024-03-08T21:10:23.430 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.447+0000 7f556b5db740 1 -- 172.21.15.107:0/3107289903 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5550099b20 con 0x5570065287a0 2024-03-08T21:10:23.430 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.447+0000 7f5543400640 1 -- 172.21.15.107:0/3107289903 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5550097d20 con 0x5570065287a0 2024-03-08T21:10:23.430 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.447+0000 7f556b5db740 10 client.16281 fetch_fsmap learned FSMap version 38 2024-03-08T21:10:23.430 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.447+0000 7f556b5db740 1 -- 172.21.15.107:0/3107289903 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x55700626c620 con 0x5570065287a0 2024-03-08T21:10:23.430 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.448+0000 7f5543400640 1 -- 172.21.15.107:0/3107289903 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f5550099b20 con 0x5570065287a0 2024-03-08T21:10:23.430 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.448+0000 7f5543400640 1 -- 172.21.15.107:0/3107289903 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 38) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f55500980f0 con 0x5570065287a0 2024-03-08T21:10:23.431 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.448+0000 7f556b5db740 10 client.16281 fetch_fsmap finished waiting for FSMap version 38 2024-03-08T21:10:23.431 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.448+0000 7f556b5db740 10 client.16281 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-08T21:10:23.431 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.448+0000 7f556b5db740 20 client.16281 populate_metadata read hostname 'smithi107' 2024-03-08T21:10:23.431 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.448+0000 7f556b5db740 1 --2- 172.21.15.107:0/3107289903 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x557006541240 0x557006543630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:10:23.431 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.448+0000 7f556b5db740 4 client.16281 mds_command: new command op to 34475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-08T21:10:23.431 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.448+0000 7f556b5db740 1 -- 172.21.15.107:0/3107289903 --> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557006498910 con 0x557006541240 2024-03-08T21:10:23.431 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.449+0000 7f5561e00640 1 --2- 172.21.15.107:0/3107289903 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x557006541240 0x557006543630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:10:23.432 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.450+0000 7f5561e00640 1 --2- 172.21.15.107:0/3107289903 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x557006541240 0x557006543630 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=90dc960b408da1f6 server_cookie=14a66e0b226bc703 in_seq=0 out_seq=0 2024-03-08T21:10:23.432 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.450+0000 7f5543400640 10 client.16281 ms_handle_connect on v2:172.21.15.149:6838/55290596 2024-03-08T21:10:23.433 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.451+0000 7f5543400640 1 -- 172.21.15.107:0/3107289903 <== mds.0 v2:172.21.15.149:6838/55290596 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x557006498910 con 0x557006541240 2024-03-08T21:10:23.433 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.451+0000 7f5543400640 10 client.16281 handle_command_reply: tid=0 2024-03-08T21:10:23.433 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.451+0000 7f556b5db740 1 client.16281 shutdown 2024-03-08T21:10:23.433 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.451+0000 7f556b5db740 1 -- 172.21.15.107:0/3107289903 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5570065287a0 msgr2=0x55700652e220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:10:23.433 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.451+0000 7f556b5db740 1 --2- 172.21.15.107:0/3107289903 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5570065287a0 0x55700652e220 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f555000e9c0 tx=0x7f55500056f0 comp rx=0 tx=0).stop 2024-03-08T21:10:23.433 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.451+0000 7f556b5db740 1 -- 172.21.15.107:0/3107289903 shutdown_connections 2024-03-08T21:10:23.433 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.451+0000 7f556b5db740 1 --2- 172.21.15.107:0/3107289903 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557006531570 0x55700652e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:10:23.433 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.451+0000 7f556b5db740 1 --2- 172.21.15.107:0/3107289903 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5570065287a0 0x55700652e220 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:10:23.433 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.451+0000 7f556b5db740 1 --2- 172.21.15.107:0/3107289903 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x557006541240 0x557006543630 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:10:23.433 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.452+0000 7f556b5db740 1 --2- 172.21.15.107:0/3107289903 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557006527e80 0x557006530e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:10:23.433 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.452+0000 7f556b5db740 1 -- 172.21.15.107:0/3107289903 >> 172.21.15.107:0/3107289903 conn(0x5570065227a0 msgr2=0x5570065244f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:10:23.434 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.452+0000 7f5543400640 0 client.16281 ms_handle_reset on v2:172.21.15.149:6838/55290596 2024-03-08T21:10:23.434 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.452+0000 7f556b5db740 1 -- 172.21.15.107:0/3107289903 shutdown_connections 2024-03-08T21:10:23.434 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.452+0000 7f556b5db740 1 -- 172.21.15.107:0/3107289903 wait complete. 2024-03-08T21:10:23.434 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.452+0000 7f556b5db740 20 client.16281 trim_cache size 0 max 16384 2024-03-08T21:10:23.447 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.468+0000 7f556b5db740 1 Processor -- start 2024-03-08T21:10:23.447 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.468+0000 7f556b5db740 1 -- start start 2024-03-08T21:10:23.447 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.468+0000 7f556b5db740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557006527e80 0x55700653cc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:10:23.447 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.468+0000 7f556b5db740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5570065287a0 0x557006530950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:10:23.447 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.468+0000 7f556b5db740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557006531570 0x557006530e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:10:23.447 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.468+0000 7f556b5db740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55641aed40 con 0x557006531570 2024-03-08T21:10:23.447 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.468+0000 7f556b5db740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55641b06c0 con 0x557006527e80 2024-03-08T21:10:23.448 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.468+0000 7f556b5db740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55641af3a0 con 0x5570065287a0 2024-03-08T21:10:23.448 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.468+0000 7f5562800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557006531570 0x557006530e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:10:23.448 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.469+0000 7f5562800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557006531570 0x557006530e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:35464/0 (socket says 172.21.15.107:35464) 2024-03-08T21:10:23.448 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.469+0000 7f5562800640 1 -- 172.21.15.107:0/3814052530 learned_addr learned my addr 172.21.15.107:0/3814052530 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:10:23.449 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.469+0000 7f5561400640 1 --2- 172.21.15.107:0/3814052530 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5570065287a0 0x557006530950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:10:23.449 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.469+0000 7f5561e00640 1 --2- 172.21.15.107:0/3814052530 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557006527e80 0x55700653cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:10:23.449 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.469+0000 7f5562800640 1 -- 172.21.15.107:0/3814052530 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5570065287a0 msgr2=0x557006530950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:10:23.450 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.469+0000 7f5562800640 1 --2- 172.21.15.107:0/3814052530 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5570065287a0 0x557006530950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:10:23.450 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.469+0000 7f5562800640 1 -- 172.21.15.107:0/3814052530 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557006527e80 msgr2=0x55700653cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:10:23.450 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.469+0000 7f5562800640 1 --2- 172.21.15.107:0/3814052530 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557006527e80 0x55700653cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:10:23.450 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.469+0000 7f5562800640 1 -- 172.21.15.107:0/3814052530 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5570065285d0 con 0x557006531570 2024-03-08T21:10:23.450 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.469+0000 7f5562800640 1 --2- 172.21.15.107:0/3814052530 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557006531570 0x557006530e90 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f555c009ca0 tx=0x7f555c03f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfc89f9319a9aec server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:10:23.450 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.470+0000 7f554e600640 1 -- 172.21.15.107:0/3814052530 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f555c039040 con 0x557006531570 2024-03-08T21:10:23.450 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.470+0000 7f554e600640 1 -- 172.21.15.107:0/3814052530 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f555c052070 con 0x557006531570 2024-03-08T21:10:23.450 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.470+0000 7f554e600640 1 -- 172.21.15.107:0/3814052530 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f555c04d450 con 0x557006531570 2024-03-08T21:10:23.450 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.471+0000 7f556b5db740 1 -- 172.21.15.107:0/3814052530 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557006531570 msgr2=0x557006530e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:10:23.450 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.471+0000 7f556b5db740 1 --2- 172.21.15.107:0/3814052530 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557006531570 0x557006530e90 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f555c009ca0 tx=0x7f555c03f000 comp rx=0 tx=0).stop 2024-03-08T21:10:23.450 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.471+0000 7f556b5db740 1 -- 172.21.15.107:0/3814052530 shutdown_connections 2024-03-08T21:10:23.450 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.471+0000 7f556b5db740 1 --2- 172.21.15.107:0/3814052530 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557006531570 0x557006530e90 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:10:23.450 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.471+0000 7f556b5db740 1 --2- 172.21.15.107:0/3814052530 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5570065287a0 0x557006530950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:10:23.451 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.471+0000 7f556b5db740 1 --2- 172.21.15.107:0/3814052530 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x557006527e80 0x55700653cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:10:23.451 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.471+0000 7f556b5db740 1 -- 172.21.15.107:0/3814052530 >> 172.21.15.107:0/3814052530 conn(0x5570065227a0 msgr2=0x55700653c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:10:23.451 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.471+0000 7f556b5db740 1 -- 172.21.15.107:0/3814052530 shutdown_connections 2024-03-08T21:10:23.451 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.471+0000 7f556b5db740 1 -- 172.21.15.107:0/3814052530 wait complete. 2024-03-08T21:10:23.453 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.471+0000 7f556b5db740 1 -- start start 2024-03-08T21:10:23.454 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.471+0000 7f556b5db740 1 Processor -- start 2024-03-08T21:10:23.454 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.472+0000 7f556b5db740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557006527e80 0x557006544040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:10:23.454 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.472+0000 7f556b5db740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5570065287a0 0x557006544580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:10:23.454 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.472+0000 7f556b5db740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557006531570 0x557006544ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:10:23.454 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.472+0000 7f556b5db740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55641b0060 con 0x557006527e80 2024-03-08T21:10:23.459 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.472+0000 7f556b5db740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55641afa00 con 0x5570065287a0 2024-03-08T21:10:23.459 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.472+0000 7f556b5db740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55641ad000 con 0x557006531570 2024-03-08T21:10:23.459 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.472+0000 7f556b5db740 10 client.0 _ll_register_callbacks cb 0x7f55500d2e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T21:10:23.459 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.472+0000 7f5561e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557006527e80 0x557006544040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:10:23.459 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.472+0000 7f5561e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557006527e80 0x557006544040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:35466/0 (socket says 172.21.15.107:35466) 2024-03-08T21:10:23.459 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.472+0000 7f5561e00640 1 -- 172.21.15.107:0/3291267097 learned_addr learned my addr 172.21.15.107:0/3291267097 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:10:23.459 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.473+0000 7f5562800640 1 --2- 172.21.15.107:0/3291267097 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557006531570 0x557006544ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:10:23.459 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.473+0000 7f5561e00640 1 -- 172.21.15.107:0/3291267097 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557006531570 msgr2=0x557006544ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:10:23.459 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.473+0000 7f5561e00640 1 --2- 172.21.15.107:0/3291267097 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557006531570 0x557006544ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:10:23.459 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.473+0000 7f5561e00640 1 -- 172.21.15.107:0/3291267097 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5570065287a0 msgr2=0x557006544580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:10:23.460 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.473+0000 7f5561e00640 1 --2- 172.21.15.107:0/3291267097 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5570065287a0 0x557006544580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:10:23.460 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.473+0000 7f5561e00640 1 -- 172.21.15.107:0/3291267097 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557006597350 con 0x557006527e80 2024-03-08T21:10:23.460 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.473+0000 7f5561e00640 1 --2- 172.21.15.107:0/3291267097 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557006527e80 0x557006544040 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x557006534370 tx=0x7f5554009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c30047c182d0dff server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:10:23.460 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.473+0000 7f5543400640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-03-08T21:10:23.460 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.474+0000 7f5543400640 1 -- 172.21.15.107:0/3291267097 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f555401d720 con 0x557006527e80 2024-03-08T21:10:23.460 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.474+0000 7f5543400640 1 -- 172.21.15.107:0/3291267097 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f5554025410 con 0x557006527e80 2024-03-08T21:10:23.460 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.474+0000 7f556b5db740 1 -- 172.21.15.107:0/3291267097 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f555c052070 con 0x557006527e80 2024-03-08T21:10:23.460 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.474+0000 7f5543400640 1 -- 172.21.15.107:0/3291267097 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f555401d8c0 con 0x557006527e80 2024-03-08T21:10:23.460 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.474+0000 7f556b5db740 10 client.16287 fetch_fsmap learned FSMap version 38 2024-03-08T21:10:23.460 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.474+0000 7f556b5db740 1 -- 172.21.15.107:0/3291267097 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x5570065986c0 con 0x557006527e80 2024-03-08T21:10:23.460 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.474+0000 7f5543400640 1 -- 172.21.15.107:0/3291267097 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f555c052070 con 0x557006527e80 2024-03-08T21:10:23.460 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.474+0000 7f5543400640 1 -- 172.21.15.107:0/3291267097 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 38) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f55540117b0 con 0x557006527e80 2024-03-08T21:10:23.460 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.474+0000 7f556b5db740 10 client.16287 fetch_fsmap finished waiting for FSMap version 38 2024-03-08T21:10:23.461 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.474+0000 7f556b5db740 10 client.16287 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-08T21:10:23.461 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.474+0000 7f556b5db740 20 client.16287 populate_metadata read hostname 'smithi107' 2024-03-08T21:10:23.461 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.474+0000 7f556b5db740 1 --2- 172.21.15.107:0/3291267097 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x557006598da0 0x55700659b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:10:23.461 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.474+0000 7f556b5db740 4 client.16287 mds_command: new command op to 34475 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-08T21:10:23.461 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.474+0000 7f556b5db740 1 -- 172.21.15.107:0/3291267097 --> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x557006492c60 con 0x557006598da0 2024-03-08T21:10:23.461 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.475+0000 7f5561400640 1 --2- 172.21.15.107:0/3291267097 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x557006598da0 0x55700659b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:10:23.461 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.476+0000 7f5561400640 1 --2- 172.21.15.107:0/3291267097 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x557006598da0 0x55700659b1c0 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=652b586aecc7fff3 server_cookie=9cd5f61752c995b6 in_seq=0 out_seq=0 2024-03-08T21:10:23.461 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.476+0000 7f5543400640 10 client.16287 ms_handle_connect on v2:172.21.15.149:6838/55290596 2024-03-08T21:10:23.479 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.500+0000 7f5543400640 1 -- 172.21.15.107:0/3291267097 <== mds.0 v2:172.21.15.149:6838/55290596 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x557006492c60 con 0x557006598da0 2024-03-08T21:10:23.480 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.500+0000 7f5543400640 10 client.16287 handle_command_reply: tid=0 2024-03-08T21:10:23.480 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.500+0000 7f556b5db740 1 client.16287 shutdown 2024-03-08T21:10:23.480 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.500+0000 7f556b5db740 1 -- 172.21.15.107:0/3291267097 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557006527e80 msgr2=0x557006544040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:10:23.480 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.500+0000 7f556b5db740 1 --2- 172.21.15.107:0/3291267097 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557006527e80 0x557006544040 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x557006534370 tx=0x7f5554009f90 comp rx=0 tx=0).stop 2024-03-08T21:10:23.480 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.501+0000 7f556b5db740 1 -- 172.21.15.107:0/3291267097 shutdown_connections 2024-03-08T21:10:23.480 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.501+0000 7f556b5db740 1 --2- 172.21.15.107:0/3291267097 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x557006531570 0x557006544ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:10:23.480 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.501+0000 7f556b5db740 1 --2- 172.21.15.107:0/3291267097 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x557006598da0 0x55700659b1c0 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:10:23.480 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.501+0000 7f556b5db740 1 --2- 172.21.15.107:0/3291267097 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5570065287a0 0x557006544580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:10:23.480 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.501+0000 7f556b5db740 1 --2- 172.21.15.107:0/3291267097 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557006527e80 0x557006544040 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:10:23.480 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.501+0000 7f556b5db740 1 -- 172.21.15.107:0/3291267097 >> 172.21.15.107:0/3291267097 conn(0x5570065227a0 msgr2=0x557006521800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:10:23.480 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.501+0000 7f5543400640 0 client.16287 ms_handle_reset on v2:172.21.15.149:6838/55290596 2024-03-08T21:10:23.480 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.501+0000 7f556b5db740 1 -- 172.21.15.107:0/3291267097 shutdown_connections 2024-03-08T21:10:23.480 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.501+0000 7f556b5db740 1 -- 172.21.15.107:0/3291267097 wait complete. 2024-03-08T21:10:23.481 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.501+0000 7f556b5db740 20 client.16287 trim_cache size 0 max 16384 2024-03-08T21:10:23.481 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:{ 2024-03-08T21:10:23.481 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "message": "", 2024-03-08T21:10:23.481 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "return_code": 0 2024-03-08T21:10:23.481 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:} 2024-03-08T21:10:23.482 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.502+0000 7f5563200640 1 -- 172.21.15.107:0/1897194609 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f55480a2960 msgr2=0x7f55480a4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:10:23.482 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.502+0000 7f5563200640 1 --2- 172.21.15.107:0/1897194609 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f55480a2960 0x7f55480a4e20 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f555c009430 tx=0x7f555c02e040 comp rx=0 tx=0).stop 2024-03-08T21:10:23.482 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.502+0000 7f5563200640 1 -- 172.21.15.107:0/1897194609 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5564102bc0 msgr2=0x7f556419fc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:10:23.482 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.502+0000 7f5563200640 1 --2- 172.21.15.107:0/1897194609 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5564102bc0 0x7f556419fc70 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f555000e9f0 tx=0x7f555000eeb0 comp rx=0 tx=0).stop 2024-03-08T21:10:23.482 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.502+0000 7f5563200640 1 -- 172.21.15.107:0/1897194609 shutdown_connections 2024-03-08T21:10:23.482 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.502+0000 7f5563200640 1 --2- 172.21.15.107:0/1897194609 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f55480a2960 0x7f55480a4e20 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:10:23.483 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.502+0000 7f5563200640 1 --2- 172.21.15.107:0/1897194609 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f556410b6c0 0x7f55641999d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:10:23.483 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.502+0000 7f5563200640 1 --2- 172.21.15.107:0/1897194609 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5564102bc0 0x7f556419fc70 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:10:23.483 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.502+0000 7f5563200640 1 --2- 172.21.15.107:0/1897194609 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f55641022a0 0x7f556419f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:10:23.483 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.502+0000 7f5563200640 1 -- 172.21.15.107:0/1897194609 >> 172.21.15.107:0/1897194609 conn(0x7f55640fbb00 msgr2=0x7f5564101090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:10:23.483 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.502+0000 7f5563200640 1 -- 172.21.15.107:0/1897194609 shutdown_connections 2024-03-08T21:10:23.483 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:23.502+0000 7f5563200640 1 -- 172.21.15.107:0/1897194609 wait complete. 2024-03-08T21:10:24.462 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:10:24 smithi107 ceph-mon[31381]: pgmap v311: 97 pgs: 97 active+clean; 6.7 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 6.7 KiB/s wr, 13 op/s 2024-03-08T21:10:24.510 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:10:24 smithi149 ceph-mon[35042]: pgmap v311: 97 pgs: 97 active+clean; 6.7 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 6.7 KiB/s wr, 13 op/s 2024-03-08T21:10:24.623 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:10:24 smithi204 ceph-mon[34935]: pgmap v311: 97 pgs: 97 active+clean; 6.7 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 6.7 KiB/s wr, 13 op/s 2024-03-08T21:10:26.510 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:10:26 smithi149 ceph-mon[35042]: pgmap v312: 97 pgs: 97 active+clean; 6.7 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 8.8 KiB/s wr, 12 op/s 2024-03-08T21:10:26.623 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:10:26 smithi204 ceph-mon[34935]: pgmap v312: 97 pgs: 97 active+clean; 6.7 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 8.8 KiB/s wr, 12 op/s 2024-03-08T21:10:26.712 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:10:26 smithi107 ceph-mon[31381]: pgmap v312: 97 pgs: 97 active+clean; 6.7 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 8.8 KiB/s wr, 12 op/s 2024-03-08T21:10:28.510 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:10:28 smithi149 ceph-mon[35042]: pgmap v313: 97 pgs: 97 active+clean; 6.7 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 8.8 KiB/s wr, 12 op/s 2024-03-08T21:10:28.623 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:10:28 smithi204 ceph-mon[34935]: pgmap v313: 97 pgs: 97 active+clean; 6.7 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 8.8 KiB/s wr, 12 op/s 2024-03-08T21:10:28.712 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:10:28 smithi107 ceph-mon[31381]: pgmap v313: 97 pgs: 97 active+clean; 6.7 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 8.8 KiB/s wr, 12 op/s 2024-03-08T21:10:30.510 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:10:30 smithi149 ceph-mon[35042]: pgmap v314: 97 pgs: 97 active+clean; 6.7 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 5.7 KiB/s wr, 12 op/s 2024-03-08T21:10:30.623 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:10:30 smithi204 ceph-mon[34935]: pgmap v314: 97 pgs: 97 active+clean; 6.7 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 5.7 KiB/s wr, 12 op/s 2024-03-08T21:10:30.714 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:10:30 smithi107 ceph-mon[31381]: pgmap v314: 97 pgs: 97 active+clean; 6.7 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 5.7 KiB/s wr, 12 op/s 2024-03-08T21:10:32.510 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:10:32 smithi149 ceph-mon[35042]: pgmap v315: 97 pgs: 97 active+clean; 6.7 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 5.0 KiB/s wr, 11 op/s 2024-03-08T21:10:32.623 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:10:32 smithi204 ceph-mon[34935]: pgmap v315: 97 pgs: 97 active+clean; 6.7 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 5.0 KiB/s wr, 11 op/s 2024-03-08T21:10:32.712 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:10:32 smithi107 ceph-mon[31381]: pgmap v315: 97 pgs: 97 active+clean; 6.7 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 5.0 KiB/s wr, 11 op/s 2024-03-08T21:10:34.510 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:10:34 smithi149 ceph-mon[35042]: pgmap v316: 97 pgs: 97 active+clean; 6.7 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 4.7 KiB/s wr, 10 op/s 2024-03-08T21:10:34.623 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:10:34 smithi204 ceph-mon[34935]: pgmap v316: 97 pgs: 97 active+clean; 6.7 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 4.7 KiB/s wr, 10 op/s 2024-03-08T21:10:34.712 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:10:34 smithi107 ceph-mon[31381]: pgmap v316: 97 pgs: 97 active+clean; 6.7 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 4.7 KiB/s wr, 10 op/s 2024-03-08T21:10:36.510 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:10:36 smithi149 ceph-mon[35042]: pgmap v317: 97 pgs: 97 active+clean; 7.1 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 33 KiB/s wr, 14 op/s 2024-03-08T21:10:36.623 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:10:36 smithi204 ceph-mon[34935]: pgmap v317: 97 pgs: 97 active+clean; 7.1 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 33 KiB/s wr, 14 op/s 2024-03-08T21:10:36.634 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.655+0000 7f7171a00640 1 -- 172.21.15.107:0/3648341854 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f716c10d990 msgr2=0x7f716c10fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:10:36.634 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.655+0000 7f7171a00640 1 --2- 172.21.15.107:0/3648341854 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f716c10d990 0x7f716c10fda0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f7164009900 tx=0x7f716402bcc0 comp rx=0 tx=0).stop 2024-03-08T21:10:36.634 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.655+0000 7f7171a00640 1 -- 172.21.15.107:0/3648341854 shutdown_connections 2024-03-08T21:10:36.634 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.655+0000 7f7171a00640 1 --2- 172.21.15.107:0/3648341854 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f716c10d990 0x7f716c10fda0 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:10:36.634 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.655+0000 7f7171a00640 1 --2- 172.21.15.107:0/3648341854 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f716c073fe0 0x7f716c10d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:10:36.634 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.655+0000 7f7171a00640 1 --2- 172.21.15.107:0/3648341854 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f716c0736c0 0x7f716c073aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:10:36.634 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.655+0000 7f7171a00640 1 -- 172.21.15.107:0/3648341854 >> 172.21.15.107:0/3648341854 conn(0x7f716c0fcc00 msgr2=0x7f716c0ff020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:10:36.634 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.655+0000 7f7171a00640 1 -- 172.21.15.107:0/3648341854 shutdown_connections 2024-03-08T21:10:36.634 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.655+0000 7f7171a00640 1 -- 172.21.15.107:0/3648341854 wait complete. 2024-03-08T21:10:36.635 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.656+0000 7f7171a00640 1 Processor -- start 2024-03-08T21:10:36.635 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.656+0000 7f7171a00640 1 -- start start 2024-03-08T21:10:36.636 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.657+0000 7f7171a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f716c0736c0 0x7f716c19f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:10:36.636 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.657+0000 7f7171a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f716c073fe0 0x7f716c19fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:10:36.636 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.657+0000 7f7171a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f716c10d990 0x7f716c1998f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:10:36.636 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.657+0000 7f7171a00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f716c1135b0 con 0x7f716c073fe0 2024-03-08T21:10:36.636 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.657+0000 7f7171a00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f716c113430 con 0x7f716c0736c0 2024-03-08T21:10:36.636 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.657+0000 7f7171a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f716c113730 con 0x7f716c10d990 2024-03-08T21:10:36.637 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.657+0000 7f716b400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f716c073fe0 0x7f716c19fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:10:36.637 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.657+0000 7f716b400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f716c073fe0 0x7f716c19fc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:57194/0 (socket says 172.21.15.107:57194) 2024-03-08T21:10:36.637 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.657+0000 7f716b400640 1 -- 172.21.15.107:0/3774495291 learned_addr learned my addr 172.21.15.107:0/3774495291 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:10:36.637 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.657+0000 7f716be00640 1 --2- 172.21.15.107:0/3774495291 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f716c0736c0 0x7f716c19f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:10:36.637 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.657+0000 7f7171000640 1 --2- 172.21.15.107:0/3774495291 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f716c10d990 0x7f716c1998f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:10:36.637 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.658+0000 7f716b400640 1 -- 172.21.15.107:0/3774495291 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f716c10d990 msgr2=0x7f716c1998f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:10:36.637 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.658+0000 7f716b400640 1 --2- 172.21.15.107:0/3774495291 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f716c10d990 0x7f716c1998f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:10:36.637 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.658+0000 7f716b400640 1 -- 172.21.15.107:0/3774495291 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f716c0736c0 msgr2=0x7f716c19f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:10:36.637 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.658+0000 7f716b400640 1 --2- 172.21.15.107:0/3774495291 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f716c0736c0 0x7f716c19f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:10:36.637 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.658+0000 7f716b400640 1 -- 172.21.15.107:0/3774495291 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f716c19a070 con 0x7f716c073fe0 2024-03-08T21:10:36.638 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.658+0000 7f716b400640 1 --2- 172.21.15.107:0/3774495291 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f716c073fe0 0x7f716c19fc80 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f715800d950 tx=0x7f715800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:10:36.638 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.658+0000 7f7168c00640 1 -- 172.21.15.107:0/3774495291 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7158017070 con 0x7f716c073fe0 2024-03-08T21:10:36.638 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.658+0000 7f7168c00640 1 -- 172.21.15.107:0/3774495291 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f7158009c30 con 0x7f716c073fe0 2024-03-08T21:10:36.638 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.658+0000 7f7168c00640 1 -- 172.21.15.107:0/3774495291 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f715800f620 con 0x7f716c073fe0 2024-03-08T21:10:36.638 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.658+0000 7f7171a00640 1 -- 172.21.15.107:0/3774495291 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f716c19a360 con 0x7f716c073fe0 2024-03-08T21:10:36.638 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.659+0000 7f7171a00640 1 -- 172.21.15.107:0/3774495291 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f716c1a6600 con 0x7f716c073fe0 2024-03-08T21:10:36.641 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.662+0000 7f7175d6c740 1 Processor -- start 2024-03-08T21:10:36.641 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.662+0000 7f7175d6c740 1 -- start start 2024-03-08T21:10:36.641 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.663+0000 7f7175d6c740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bedcc3de80 0x55bedcc3e260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:10:36.642 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.663+0000 7f7175d6c740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bedcc3e7a0 0x55bedcc47030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:10:36.642 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.663+0000 7f7175d6c740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bedcc47570 0x55bedcc49960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:10:36.642 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.663+0000 7f7175d6c740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bedcba13d0 con 0x55bedcc47570 2024-03-08T21:10:36.642 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.663+0000 7f7175d6c740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bedcb4a6b0 con 0x55bedcc3e7a0 2024-03-08T21:10:36.642 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.663+0000 7f7175d6c740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bedcb9d470 con 0x55bedcc3de80 2024-03-08T21:10:36.642 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.663+0000 7f7168c00640 1 -- 172.21.15.107:0/3774495291 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) v1 ==== 137422+0+0 (secure 0 0 0) 0x7f71580109a0 con 0x7f716c073fe0 2024-03-08T21:10:36.643 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.663+0000 7f716b400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bedcc47570 0x55bedcc49960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:10:36.644 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.663+0000 7f7171000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bedcc3de80 0x55bedcc3e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:10:36.644 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.663+0000 7f716b400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bedcc47570 0x55bedcc49960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:57204/0 (socket says 172.21.15.107:57204) 2024-03-08T21:10:36.644 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.663+0000 7f716b400640 1 -- 172.21.15.107:0/481101612 learned_addr learned my addr 172.21.15.107:0/481101612 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:10:36.644 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.664+0000 7f716be00640 1 --2- 172.21.15.107:0/481101612 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bedcc3e7a0 0x55bedcc47030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:10:36.644 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.664+0000 7f716b400640 1 -- 172.21.15.107:0/481101612 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bedcc3de80 msgr2=0x55bedcc3e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:10:36.644 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.664+0000 7f716b400640 1 --2- 172.21.15.107:0/481101612 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bedcc3de80 0x55bedcc3e260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:10:36.644 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.664+0000 7f716b400640 1 -- 172.21.15.107:0/481101612 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bedcc3e7a0 msgr2=0x55bedcc47030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:10:36.644 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.664+0000 7f716b400640 1 --2- 172.21.15.107:0/481101612 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bedcc3e7a0 0x55bedcc47030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:10:36.644 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.664+0000 7f716b400640 1 -- 172.21.15.107:0/481101612 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bedcb70b50 con 0x55bedcc47570 2024-03-08T21:10:36.644 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.664+0000 7f716be00640 1 --2- 172.21.15.107:0/481101612 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bedcc3e7a0 0x55bedcc47030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T21:10:36.644 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.664+0000 7f716b400640 1 --2- 172.21.15.107:0/481101612 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bedcc47570 0x55bedcc49960 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f71580cc120 tx=0x7f71580d75e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7b5ce0e8302bb81 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:10:36.644 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.665+0000 7f7161600640 1 -- 172.21.15.107:0/481101612 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71580d4050 con 0x55bedcc47570 2024-03-08T21:10:36.644 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.665+0000 7f7161600640 1 -- 172.21.15.107:0/481101612 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f71580e0070 con 0x55bedcc47570 2024-03-08T21:10:36.644 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.665+0000 7f7161600640 1 -- 172.21.15.107:0/481101612 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71580da6c0 con 0x55bedcc47570 2024-03-08T21:10:36.645 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.665+0000 7f7168c00640 1 --2- 172.21.15.107:0/3774495291 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f71500a29e0 0x7f71500a4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:10:36.645 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.665+0000 7f7168c00640 1 -- 172.21.15.107:0/3774495291 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f71580ceaa0 con 0x7f716c073fe0 2024-03-08T21:10:36.645 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.665+0000 7f7171000640 1 --2- 172.21.15.107:0/3774495291 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f71500a29e0 0x7f71500a4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:10:36.647 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.666+0000 7f7175d6c740 1 -- 172.21.15.107:0/481101612 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bedcc47570 msgr2=0x55bedcc49960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:10:36.647 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.666+0000 7f7175d6c740 1 --2- 172.21.15.107:0/481101612 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bedcc47570 0x55bedcc49960 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f71580cc120 tx=0x7f71580d75e0 comp rx=0 tx=0).stop 2024-03-08T21:10:36.648 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.666+0000 7f7175d6c740 1 -- 172.21.15.107:0/481101612 shutdown_connections 2024-03-08T21:10:36.648 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.666+0000 7f7175d6c740 1 --2- 172.21.15.107:0/481101612 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bedcc47570 0x55bedcc49960 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:10:36.648 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.666+0000 7f7175d6c740 1 --2- 172.21.15.107:0/481101612 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bedcc3e7a0 0x55bedcc47030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:10:36.648 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.666+0000 7f7175d6c740 1 --2- 172.21.15.107:0/481101612 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bedcc3de80 0x55bedcc3e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:10:36.648 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.666+0000 7f7175d6c740 1 -- 172.21.15.107:0/481101612 >> 172.21.15.107:0/481101612 conn(0x55bedcc387a0 msgr2=0x55bedcc38b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:10:36.648 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.666+0000 7f7175d6c740 1 -- 172.21.15.107:0/481101612 shutdown_connections 2024-03-08T21:10:36.648 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.666+0000 7f7171000640 1 --2- 172.21.15.107:0/3774495291 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f71500a29e0 0x7f71500a4ea0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f7164036040 tx=0x7f7164005b50 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:10:36.648 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.666+0000 7f7175d6c740 1 -- 172.21.15.107:0/481101612 wait complete. 2024-03-08T21:10:36.648 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.667+0000 7f7175d6c740 1 -- start start 2024-03-08T21:10:36.648 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.667+0000 7f7175d6c740 1 Processor -- start 2024-03-08T21:10:36.648 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.667+0000 7f7175d6c740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bedcc3de80 0x55bedcc46f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:10:36.648 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.667+0000 7f7175d6c740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bedcc3e7a0 0x55bedcc442b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:10:36.648 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.668+0000 7f7175d6c740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bedcc47570 0x55bedcc44850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:10:36.648 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.668+0000 7f7175d6c740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bedcc4a140 con 0x55bedcc3e7a0 2024-03-08T21:10:36.649 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.668+0000 7f7175d6c740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bedcb5d9f0 con 0x55bedcc47570 2024-03-08T21:10:36.649 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.668+0000 7f7175d6c740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bedcc4a2c0 con 0x55bedcc3de80 2024-03-08T21:10:36.649 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.668+0000 7f7175d6c740 10 client.0 _ll_register_callbacks cb 0x55bedc8b5a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T21:10:36.649 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.668+0000 7f716b400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bedcc3e7a0 0x55bedcc442b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:10:36.649 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.668+0000 7f716b400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bedcc3e7a0 0x55bedcc442b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:57216/0 (socket says 172.21.15.107:57216) 2024-03-08T21:10:36.649 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.668+0000 7f716b400640 1 -- 172.21.15.107:0/2121191251 learned_addr learned my addr 172.21.15.107:0/2121191251 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:10:36.649 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.668+0000 7f716be00640 1 --2- 172.21.15.107:0/2121191251 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bedcc3de80 0x55bedcc46f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:10:36.649 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.668+0000 7f7171000640 1 --2- 172.21.15.107:0/2121191251 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bedcc47570 0x55bedcc44850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:10:36.649 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.668+0000 7f716b400640 1 -- 172.21.15.107:0/2121191251 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bedcc3de80 msgr2=0x55bedcc46f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:10:36.649 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.668+0000 7f716b400640 1 --2- 172.21.15.107:0/2121191251 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bedcc3de80 0x55bedcc46f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:10:36.649 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.668+0000 7f716b400640 1 -- 172.21.15.107:0/2121191251 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bedcc47570 msgr2=0x55bedcc44850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:10:36.649 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.668+0000 7f716b400640 1 --2- 172.21.15.107:0/2121191251 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bedcc47570 0x55bedcc44850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:10:36.649 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.668+0000 7f716b400640 1 -- 172.21.15.107:0/2121191251 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bedc8b6210 con 0x55bedcc3e7a0 2024-03-08T21:10:36.650 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.669+0000 7f716b400640 1 --2- 172.21.15.107:0/2121191251 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bedcc3e7a0 0x55bedcc442b0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f71580cc260 tx=0x7f7158010f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7d1efa290523637 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:10:36.650 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.669+0000 7f7156000640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-03-08T21:10:36.650 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.669+0000 7f7156000640 1 -- 172.21.15.107:0/2121191251 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7158020070 con 0x55bedcc3e7a0 2024-03-08T21:10:36.650 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.669+0000 7f7156000640 1 -- 172.21.15.107:0/2121191251 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f71580cc430 con 0x55bedcc3e7a0 2024-03-08T21:10:36.650 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.669+0000 7f7175d6c740 1 -- 172.21.15.107:0/2121191251 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f71580e0070 con 0x55bedcc3e7a0 2024-03-08T21:10:36.651 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.669+0000 7f7156000640 1 -- 172.21.15.107:0/2121191251 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7158097360 con 0x55bedcc3e7a0 2024-03-08T21:10:36.651 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.669+0000 7f7175d6c740 10 client.16299 fetch_fsmap learned FSMap version 38 2024-03-08T21:10:36.651 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.669+0000 7f7175d6c740 1 -- 172.21.15.107:0/2121191251 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x55bedc982620 con 0x55bedcc3e7a0 2024-03-08T21:10:36.651 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.670+0000 7f7156000640 1 -- 172.21.15.107:0/2121191251 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f71580e0070 con 0x55bedcc3e7a0 2024-03-08T21:10:36.651 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.670+0000 7f7156000640 1 -- 172.21.15.107:0/2121191251 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 38) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f71580da070 con 0x55bedcc3e7a0 2024-03-08T21:10:36.651 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.670+0000 7f7175d6c740 10 client.16299 fetch_fsmap finished waiting for FSMap version 38 2024-03-08T21:10:36.651 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.670+0000 7f7175d6c740 10 client.16299 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-08T21:10:36.652 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.670+0000 7f7175d6c740 20 client.16299 populate_metadata read hostname 'smithi107' 2024-03-08T21:10:36.652 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.670+0000 7f7175d6c740 1 --2- 172.21.15.107:0/2121191251 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55bedcc57270 0x55bedcc59660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:10:36.652 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.670+0000 7f7175d6c740 4 client.16299 mds_command: new command op to 34475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-08T21:10:36.652 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.670+0000 7f7175d6c740 1 -- 172.21.15.107:0/2121191251 --> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bedcbae910 con 0x55bedcc57270 2024-03-08T21:10:36.652 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.671+0000 7f716be00640 1 --2- 172.21.15.107:0/2121191251 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55bedcc57270 0x55bedcc59660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:10:36.652 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.672+0000 7f7156000640 10 client.16299 ms_handle_connect on v2:172.21.15.149:6838/55290596 2024-03-08T21:10:36.652 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.672+0000 7f716be00640 1 --2- 172.21.15.107:0/2121191251 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55bedcc57270 0x55bedcc59660 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7fbae14b0d508d97 server_cookie=b6864721f939be3e in_seq=0 out_seq=0 2024-03-08T21:10:36.653 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.673+0000 7f7156000640 1 -- 172.21.15.107:0/2121191251 <== mds.0 v2:172.21.15.149:6838/55290596 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55bedcbae910 con 0x55bedcc57270 2024-03-08T21:10:36.653 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.673+0000 7f7156000640 10 client.16299 handle_command_reply: tid=0 2024-03-08T21:10:36.653 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.673+0000 7f7175d6c740 1 client.16299 shutdown 2024-03-08T21:10:36.653 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.674+0000 7f7175d6c740 1 -- 172.21.15.107:0/2121191251 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bedcc3e7a0 msgr2=0x55bedcc442b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:10:36.653 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.674+0000 7f7175d6c740 1 --2- 172.21.15.107:0/2121191251 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bedcc3e7a0 0x55bedcc442b0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f71580cc260 tx=0x7f7158010f90 comp rx=0 tx=0).stop 2024-03-08T21:10:36.653 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.674+0000 7f7175d6c740 1 -- 172.21.15.107:0/2121191251 shutdown_connections 2024-03-08T21:10:36.653 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.674+0000 7f7175d6c740 1 --2- 172.21.15.107:0/2121191251 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55bedcc57270 0x55bedcc59660 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:10:36.653 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.674+0000 7f7175d6c740 1 --2- 172.21.15.107:0/2121191251 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bedcc47570 0x55bedcc44850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:10:36.653 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.674+0000 7f7175d6c740 1 --2- 172.21.15.107:0/2121191251 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bedcc3e7a0 0x55bedcc442b0 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:10:36.654 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.674+0000 7f7175d6c740 1 --2- 172.21.15.107:0/2121191251 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bedcc3de80 0x55bedcc46f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:10:36.654 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.674+0000 7f7156000640 0 client.16299 ms_handle_reset on v2:172.21.15.149:6838/55290596 2024-03-08T21:10:36.654 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.674+0000 7f7175d6c740 1 -- 172.21.15.107:0/2121191251 >> 172.21.15.107:0/2121191251 conn(0x55bedcc387a0 msgr2=0x55bedcc3a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:10:36.654 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.674+0000 7f7175d6c740 1 -- 172.21.15.107:0/2121191251 shutdown_connections 2024-03-08T21:10:36.654 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.674+0000 7f7175d6c740 1 -- 172.21.15.107:0/2121191251 wait complete. 2024-03-08T21:10:36.654 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.674+0000 7f7175d6c740 20 client.16299 trim_cache size 0 max 16384 2024-03-08T21:10:36.672 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.693+0000 7f7175d6c740 1 Processor -- start 2024-03-08T21:10:36.672 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.693+0000 7f7175d6c740 1 -- start start 2024-03-08T21:10:36.673 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.693+0000 7f7175d6c740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bedcc3de80 0x55bedcc3a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:10:36.673 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.693+0000 7f7175d6c740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bedcc3e7a0 0x55bedcc469e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:10:36.673 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.694+0000 7f7175d6c740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bedcc47570 0x55bedcc46f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:10:36.673 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.694+0000 7f7175d6c740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f716c1aee00 con 0x55bedcc47570 2024-03-08T21:10:36.673 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.694+0000 7f7175d6c740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f716c1b0780 con 0x55bedcc3de80 2024-03-08T21:10:36.673 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.694+0000 7f7175d6c740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f716c1af460 con 0x55bedcc3e7a0 2024-03-08T21:10:36.673 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.694+0000 7f7171000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bedcc47570 0x55bedcc46f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:10:36.674 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.694+0000 7f7171000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bedcc47570 0x55bedcc46f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:57224/0 (socket says 172.21.15.107:57224) 2024-03-08T21:10:36.674 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.694+0000 7f7171000640 1 -- 172.21.15.107:0/2824071489 learned_addr learned my addr 172.21.15.107:0/2824071489 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:10:36.674 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.694+0000 7f716be00640 1 --2- 172.21.15.107:0/2824071489 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bedcc3de80 0x55bedcc3a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:10:36.674 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.694+0000 7f716b400640 1 --2- 172.21.15.107:0/2824071489 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bedcc3e7a0 0x55bedcc469e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:10:36.674 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.694+0000 7f7171000640 1 -- 172.21.15.107:0/2824071489 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bedcc3e7a0 msgr2=0x55bedcc469e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:10:36.674 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.694+0000 7f7171000640 1 --2- 172.21.15.107:0/2824071489 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bedcc3e7a0 0x55bedcc469e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:10:36.674 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.694+0000 7f7171000640 1 -- 172.21.15.107:0/2824071489 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bedcc3de80 msgr2=0x55bedcc3a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:10:36.674 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.694+0000 7f7171000640 1 --2- 172.21.15.107:0/2824071489 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bedcc3de80 0x55bedcc3a2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:10:36.674 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.694+0000 7f7171000640 1 -- 172.21.15.107:0/2824071489 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bedcc44bc0 con 0x55bedcc47570 2024-03-08T21:10:36.674 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.694+0000 7f716b400640 1 --2- 172.21.15.107:0/2824071489 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bedcc3e7a0 0x55bedcc469e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T21:10:36.674 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.695+0000 7f7171000640 1 --2- 172.21.15.107:0/2824071489 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bedcc47570 0x55bedcc46f20 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f716c074610 tx=0x7f716403f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c513ba030fe6a882 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:10:36.675 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.695+0000 7f7160c00640 1 -- 172.21.15.107:0/2824071489 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f716402ca70 con 0x55bedcc47570 2024-03-08T21:10:36.676 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.695+0000 7f7160c00640 1 -- 172.21.15.107:0/2824071489 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f7164051070 con 0x55bedcc47570 2024-03-08T21:10:36.676 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.695+0000 7f7160c00640 1 -- 172.21.15.107:0/2824071489 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7164004a80 con 0x55bedcc47570 2024-03-08T21:10:36.676 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.696+0000 7f7175d6c740 1 -- 172.21.15.107:0/2824071489 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bedcc47570 msgr2=0x55bedcc46f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:10:36.676 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.696+0000 7f7175d6c740 1 --2- 172.21.15.107:0/2824071489 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bedcc47570 0x55bedcc46f20 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f716c074610 tx=0x7f716403f000 comp rx=0 tx=0).stop 2024-03-08T21:10:36.676 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.696+0000 7f7175d6c740 1 -- 172.21.15.107:0/2824071489 shutdown_connections 2024-03-08T21:10:36.676 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.696+0000 7f7175d6c740 1 --2- 172.21.15.107:0/2824071489 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bedcc47570 0x55bedcc46f20 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:10:36.676 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.696+0000 7f7175d6c740 1 --2- 172.21.15.107:0/2824071489 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bedcc3e7a0 0x55bedcc469e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:10:36.676 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.696+0000 7f7175d6c740 1 --2- 172.21.15.107:0/2824071489 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bedcc3de80 0x55bedcc3a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:10:36.676 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.696+0000 7f7175d6c740 1 -- 172.21.15.107:0/2824071489 >> 172.21.15.107:0/2824071489 conn(0x55bedcc387a0 msgr2=0x55bedcc39ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:10:36.676 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.696+0000 7f7175d6c740 1 -- 172.21.15.107:0/2824071489 shutdown_connections 2024-03-08T21:10:36.676 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.696+0000 7f7175d6c740 1 -- 172.21.15.107:0/2824071489 wait complete. 2024-03-08T21:10:36.676 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.697+0000 7f7175d6c740 1 -- start start 2024-03-08T21:10:36.676 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.697+0000 7f7175d6c740 1 Processor -- start 2024-03-08T21:10:36.679 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.697+0000 7f7175d6c740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bedcc3de80 0x55bedcc5a970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:10:36.679 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.698+0000 7f7175d6c740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bedcc3e7a0 0x55bedcc5aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:10:36.679 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.698+0000 7f7175d6c740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bedcc47570 0x55bedccafba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:10:36.679 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.698+0000 7f7175d6c740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f716c1b0120 con 0x55bedcc3de80 2024-03-08T21:10:36.680 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.698+0000 7f7175d6c740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f716c1afac0 con 0x55bedcc3e7a0 2024-03-08T21:10:36.680 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.698+0000 7f7175d6c740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f716c1ad0c0 con 0x55bedcc47570 2024-03-08T21:10:36.680 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.698+0000 7f7175d6c740 10 client.0 _ll_register_callbacks cb 0x7f71580d24c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T21:10:36.680 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.698+0000 7f716be00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bedcc3de80 0x55bedcc5a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:10:36.680 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.698+0000 7f716be00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bedcc3de80 0x55bedcc5a970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:57228/0 (socket says 172.21.15.107:57228) 2024-03-08T21:10:36.680 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.698+0000 7f716be00640 1 -- 172.21.15.107:0/2321078224 learned_addr learned my addr 172.21.15.107:0/2321078224 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:10:36.680 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.698+0000 7f716b400640 1 --2- 172.21.15.107:0/2321078224 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bedcc3e7a0 0x55bedcc5aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:10:36.680 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.698+0000 7f7171000640 1 --2- 172.21.15.107:0/2321078224 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bedcc47570 0x55bedccafba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:10:36.680 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.698+0000 7f716be00640 1 -- 172.21.15.107:0/2321078224 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bedcc47570 msgr2=0x55bedccafba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:10:36.680 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.698+0000 7f716be00640 1 --2- 172.21.15.107:0/2321078224 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bedcc47570 0x55bedccafba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:10:36.680 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.698+0000 7f716be00640 1 -- 172.21.15.107:0/2321078224 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bedcc3e7a0 msgr2=0x55bedcc5aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:10:36.680 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.698+0000 7f716be00640 1 --2- 172.21.15.107:0/2321078224 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bedcc3e7a0 0x55bedcc5aeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:10:36.680 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.698+0000 7f716be00640 1 -- 172.21.15.107:0/2321078224 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bedccb0190 con 0x55bedcc3de80 2024-03-08T21:10:36.681 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.698+0000 7f716b400640 1 --2- 172.21.15.107:0/2321078224 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bedcc3e7a0 0x55bedcc5aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T21:10:36.681 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.699+0000 7f716be00640 1 --2- 172.21.15.107:0/2321078224 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bedcc3de80 0x55bedcc5a970 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f716c19aa30 tx=0x7f715c009f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59a8294af7beb33e server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:10:36.681 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.699+0000 7f7156000640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-03-08T21:10:36.681 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.699+0000 7f7156000640 1 -- 172.21.15.107:0/2321078224 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f715c01a8b0 con 0x55bedcc3de80 2024-03-08T21:10:36.681 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.699+0000 7f7156000640 1 -- 172.21.15.107:0/2321078224 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f715c004ab0 con 0x55bedcc3de80 2024-03-08T21:10:36.681 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.699+0000 7f7175d6c740 1 -- 172.21.15.107:0/2321078224 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7164051070 con 0x55bedcc3de80 2024-03-08T21:10:36.681 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.699+0000 7f7156000640 1 -- 172.21.15.107:0/2321078224 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f715c004cb0 con 0x55bedcc3de80 2024-03-08T21:10:36.681 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.699+0000 7f7175d6c740 10 client.16305 fetch_fsmap learned FSMap version 38 2024-03-08T21:10:36.681 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.699+0000 7f7175d6c740 1 -- 172.21.15.107:0/2321078224 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x55bedccb7510 con 0x55bedcc3de80 2024-03-08T21:10:36.681 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.700+0000 7f7156000640 1 -- 172.21.15.107:0/2321078224 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f7164051070 con 0x55bedcc3de80 2024-03-08T21:10:36.681 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.700+0000 7f7156000640 1 -- 172.21.15.107:0/2321078224 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 38) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f715c028030 con 0x55bedcc3de80 2024-03-08T21:10:36.681 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.700+0000 7f7175d6c740 10 client.16305 fetch_fsmap finished waiting for FSMap version 38 2024-03-08T21:10:36.681 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.700+0000 7f7175d6c740 10 client.16305 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-08T21:10:36.681 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.700+0000 7f7175d6c740 20 client.16305 populate_metadata read hostname 'smithi107' 2024-03-08T21:10:36.681 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.700+0000 7f7175d6c740 1 --2- 172.21.15.107:0/2321078224 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55bedccb7c10 0x55bedccba030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:10:36.682 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.700+0000 7f7175d6c740 4 client.16305 mds_command: new command op to 34475 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-08T21:10:36.682 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.700+0000 7f7175d6c740 1 -- 172.21.15.107:0/2321078224 --> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55bedcba8c60 con 0x55bedccb7c10 2024-03-08T21:10:36.682 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.701+0000 7f716b400640 1 --2- 172.21.15.107:0/2321078224 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55bedccb7c10 0x55bedccba030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:10:36.682 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.702+0000 7f716b400640 1 --2- 172.21.15.107:0/2321078224 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55bedccb7c10 0x55bedccba030 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e8e2bc37d7de8f8f server_cookie=d5b477009f131357 in_seq=0 out_seq=0 2024-03-08T21:10:36.682 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.702+0000 7f7156000640 10 client.16305 ms_handle_connect on v2:172.21.15.149:6838/55290596 2024-03-08T21:10:36.712 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.732+0000 7f7156000640 1 -- 172.21.15.107:0/2321078224 <== mds.0 v2:172.21.15.149:6838/55290596 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55bedcba8c60 con 0x55bedccb7c10 2024-03-08T21:10:36.712 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.732+0000 7f7156000640 10 client.16305 handle_command_reply: tid=0 2024-03-08T21:10:36.712 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.732+0000 7f7175d6c740 1 client.16305 shutdown 2024-03-08T21:10:36.712 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.732+0000 7f7175d6c740 1 -- 172.21.15.107:0/2321078224 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bedcc3de80 msgr2=0x55bedcc5a970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:10:36.712 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.732+0000 7f7175d6c740 1 --2- 172.21.15.107:0/2321078224 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bedcc3de80 0x55bedcc5a970 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f716c19aa30 tx=0x7f715c009f50 comp rx=0 tx=0).stop 2024-03-08T21:10:36.712 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.733+0000 7f7175d6c740 1 -- 172.21.15.107:0/2321078224 shutdown_connections 2024-03-08T21:10:36.712 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.733+0000 7f7175d6c740 1 --2- 172.21.15.107:0/2321078224 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55bedcc47570 0x55bedccafba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:10:36.712 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.733+0000 7f7175d6c740 1 --2- 172.21.15.107:0/2321078224 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55bedccb7c10 0x55bedccba030 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:10:36.712 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.733+0000 7f7175d6c740 1 --2- 172.21.15.107:0/2321078224 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55bedcc3e7a0 0x55bedcc5aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:10:36.712 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.733+0000 7f7175d6c740 1 --2- 172.21.15.107:0/2321078224 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bedcc3de80 0x55bedcc5a970 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:10:36.712 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.733+0000 7f7175d6c740 1 -- 172.21.15.107:0/2321078224 >> 172.21.15.107:0/2321078224 conn(0x55bedcc387a0 msgr2=0x55bedcc3a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:10:36.712 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.733+0000 7f7156000640 0 client.16305 ms_handle_reset on v2:172.21.15.149:6838/55290596 2024-03-08T21:10:36.713 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.733+0000 7f7175d6c740 1 -- 172.21.15.107:0/2321078224 shutdown_connections 2024-03-08T21:10:36.713 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.733+0000 7f7175d6c740 1 -- 172.21.15.107:0/2321078224 wait complete. 2024-03-08T21:10:36.713 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:10:36 smithi107 ceph-mon[31381]: pgmap v317: 97 pgs: 97 active+clean; 7.1 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 33 KiB/s wr, 14 op/s 2024-03-08T21:10:36.713 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.733+0000 7f7175d6c740 20 client.16305 trim_cache size 0 max 16384 2024-03-08T21:10:36.714 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:{ 2024-03-08T21:10:36.714 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "message": "", 2024-03-08T21:10:36.714 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "return_code": 0 2024-03-08T21:10:36.714 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:} 2024-03-08T21:10:36.714 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.734+0000 7f7171a00640 1 -- 172.21.15.107:0/3774495291 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f71500a29e0 msgr2=0x7f71500a4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:10:36.715 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.734+0000 7f7171a00640 1 --2- 172.21.15.107:0/3774495291 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f71500a29e0 0x7f71500a4ea0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f7164036040 tx=0x7f7164005b50 comp rx=0 tx=0).stop 2024-03-08T21:10:36.715 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.734+0000 7f7171a00640 1 -- 172.21.15.107:0/3774495291 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f716c073fe0 msgr2=0x7f716c19fc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:10:36.715 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.734+0000 7f7171a00640 1 --2- 172.21.15.107:0/3774495291 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f716c073fe0 0x7f716c19fc80 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f715800d950 tx=0x7f715800de10 comp rx=0 tx=0).stop 2024-03-08T21:10:36.715 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.734+0000 7f7171a00640 1 -- 172.21.15.107:0/3774495291 shutdown_connections 2024-03-08T21:10:36.715 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.734+0000 7f7171a00640 1 --2- 172.21.15.107:0/3774495291 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f71500a29e0 0x7f71500a4ea0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:10:36.715 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.734+0000 7f7171a00640 1 --2- 172.21.15.107:0/3774495291 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f716c10d990 0x7f716c1998f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:10:36.715 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.734+0000 7f7171a00640 1 --2- 172.21.15.107:0/3774495291 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f716c073fe0 0x7f716c19fc80 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:10:36.715 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.734+0000 7f7171a00640 1 --2- 172.21.15.107:0/3774495291 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f716c0736c0 0x7f716c19f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:10:36.715 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.734+0000 7f7171a00640 1 -- 172.21.15.107:0/3774495291 >> 172.21.15.107:0/3774495291 conn(0x7f716c0fcc00 msgr2=0x7f716c0fd910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:10:36.715 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.734+0000 7f7171a00640 1 -- 172.21.15.107:0/3774495291 shutdown_connections 2024-03-08T21:10:36.715 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:36.734+0000 7f7171a00640 1 -- 172.21.15.107:0/3774495291 wait complete. 2024-03-08T21:10:38.510 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:10:38 smithi149 ceph-mon[35042]: pgmap v318: 97 pgs: 97 active+clean; 7.1 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 28 KiB/s wr, 8 op/s 2024-03-08T21:10:38.623 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:10:38 smithi204 ceph-mon[34935]: pgmap v318: 97 pgs: 97 active+clean; 7.1 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 28 KiB/s wr, 8 op/s 2024-03-08T21:10:38.712 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:10:38 smithi107 ceph-mon[31381]: pgmap v318: 97 pgs: 97 active+clean; 7.1 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 28 KiB/s wr, 8 op/s 2024-03-08T21:10:40.623 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:10:40 smithi204 ceph-mon[34935]: pgmap v319: 97 pgs: 97 active+clean; 7.1 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 33 KiB/s wr, 15 op/s 2024-03-08T21:10:40.712 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:10:40 smithi107 ceph-mon[31381]: pgmap v319: 97 pgs: 97 active+clean; 7.1 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 33 KiB/s wr, 15 op/s 2024-03-08T21:10:40.760 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:10:40 smithi149 ceph-mon[35042]: pgmap v319: 97 pgs: 97 active+clean; 7.1 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 33 KiB/s wr, 15 op/s 2024-03-08T21:10:42.623 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:10:42 smithi204 ceph-mon[34935]: pgmap v320: 97 pgs: 97 active+clean; 7.1 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 33 KiB/s wr, 11 op/s 2024-03-08T21:10:42.712 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:10:42 smithi107 ceph-mon[31381]: pgmap v320: 97 pgs: 97 active+clean; 7.1 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 33 KiB/s wr, 11 op/s 2024-03-08T21:10:42.760 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:10:42 smithi149 ceph-mon[35042]: pgmap v320: 97 pgs: 97 active+clean; 7.1 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 33 KiB/s wr, 11 op/s 2024-03-08T21:10:44.623 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:10:44 smithi204 ceph-mon[34935]: pgmap v321: 97 pgs: 97 active+clean; 7.1 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 33 KiB/s wr, 11 op/s 2024-03-08T21:10:44.712 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:10:44 smithi107 ceph-mon[31381]: pgmap v321: 97 pgs: 97 active+clean; 7.1 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 33 KiB/s wr, 11 op/s 2024-03-08T21:10:44.760 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:10:44 smithi149 ceph-mon[35042]: pgmap v321: 97 pgs: 97 active+clean; 7.1 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 33 KiB/s wr, 11 op/s 2024-03-08T21:10:46.563 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:10:46 smithi149 ceph-mon[35042]: pgmap v322: 97 pgs: 97 active+clean; 7.1 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 33 KiB/s wr, 15 op/s 2024-03-08T21:10:46.623 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:10:46 smithi204 ceph-mon[34935]: pgmap v322: 97 pgs: 97 active+clean; 7.1 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 33 KiB/s wr, 15 op/s 2024-03-08T21:10:46.712 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:10:46 smithi107 ceph-mon[31381]: pgmap v322: 97 pgs: 97 active+clean; 7.1 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 33 KiB/s wr, 15 op/s 2024-03-08T21:10:48.623 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:10:48 smithi204 ceph-mon[34935]: pgmap v323: 97 pgs: 97 active+clean; 7.1 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 5.2 KiB/s wr, 10 op/s 2024-03-08T21:10:48.712 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:10:48 smithi107 ceph-mon[31381]: pgmap v323: 97 pgs: 97 active+clean; 7.1 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 5.2 KiB/s wr, 10 op/s 2024-03-08T21:10:48.760 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:10:48 smithi149 ceph-mon[35042]: pgmap v323: 97 pgs: 97 active+clean; 7.1 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 5.2 KiB/s wr, 10 op/s 2024-03-08T21:10:49.860 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.882+0000 7fd5a7e00640 1 -- 172.21.15.107:0/2431656314 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd5a8100830 msgr2=0x7fd5a810d4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:10:49.861 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.882+0000 7fd5a7e00640 1 --2- 172.21.15.107:0/2431656314 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd5a8100830 0x7fd5a810d4f0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fd590009900 tx=0x7fd59002bc30 comp rx=0 tx=0).stop 2024-03-08T21:10:49.861 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.883+0000 7fd5a7e00640 1 -- 172.21.15.107:0/2431656314 shutdown_connections 2024-03-08T21:10:49.861 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.883+0000 7fd5a7e00640 1 --2- 172.21.15.107:0/2431656314 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd5a810da30 0x7fd5a810fe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:10:49.861 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.883+0000 7fd5a7e00640 1 --2- 172.21.15.107:0/2431656314 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd5a8100830 0x7fd5a810d4f0 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:10:49.861 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.883+0000 7fd5a7e00640 1 --2- 172.21.15.107:0/2431656314 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd5a80ffe80 0x7fd5a8100260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:10:49.861 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.883+0000 7fd5a7e00640 1 -- 172.21.15.107:0/2431656314 >> 172.21.15.107:0/2431656314 conn(0x7fd5a80fbb00 msgr2=0x7fd5a80fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:10:49.861 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.883+0000 7fd5a7e00640 1 -- 172.21.15.107:0/2431656314 shutdown_connections 2024-03-08T21:10:49.861 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.883+0000 7fd5a7e00640 1 -- 172.21.15.107:0/2431656314 wait complete. 2024-03-08T21:10:49.862 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.884+0000 7fd5a7e00640 1 Processor -- start 2024-03-08T21:10:49.862 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.884+0000 7fd5a7e00640 1 -- start start 2024-03-08T21:10:49.862 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.884+0000 7fd5a7e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd5a80ffe80 0x7fd5a819f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:10:49.863 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.884+0000 7fd5a7e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd5a8100830 0x7fd5a819fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:10:49.863 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.884+0000 7fd5a7e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd5a810da30 0x7fd5a81998b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:10:49.863 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.884+0000 7fd5a7e00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5a81135b0 con 0x7fd5a80ffe80 2024-03-08T21:10:49.863 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.884+0000 7fd5a7e00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5a8113430 con 0x7fd5a810da30 2024-03-08T21:10:49.863 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.884+0000 7fd5a7e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5a8113730 con 0x7fd5a8100830 2024-03-08T21:10:49.863 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.885+0000 7fd5a6a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd5a80ffe80 0x7fd5a819f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:10:49.863 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.885+0000 7fd5a6a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd5a80ffe80 0x7fd5a819f700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:39426/0 (socket says 172.21.15.107:39426) 2024-03-08T21:10:49.863 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.885+0000 7fd5a6a00640 1 -- 172.21.15.107:0/2782920502 learned_addr learned my addr 172.21.15.107:0/2782920502 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:10:49.863 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.885+0000 7fd5a6000640 1 --2- 172.21.15.107:0/2782920502 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd5a8100830 0x7fd5a819fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:10:49.863 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.885+0000 7fd5a7400640 1 --2- 172.21.15.107:0/2782920502 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd5a810da30 0x7fd5a81998b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:10:49.864 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.885+0000 7fd5a6a00640 1 -- 172.21.15.107:0/2782920502 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd5a8100830 msgr2=0x7fd5a819fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:10:49.864 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.885+0000 7fd5a6a00640 1 --2- 172.21.15.107:0/2782920502 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd5a8100830 0x7fd5a819fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:10:49.865 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.885+0000 7fd5a6a00640 1 -- 172.21.15.107:0/2782920502 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd5a810da30 msgr2=0x7fd5a81998b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:10:49.865 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.885+0000 7fd5a6a00640 1 --2- 172.21.15.107:0/2782920502 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd5a810da30 0x7fd5a81998b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:10:49.865 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.885+0000 7fd5a6a00640 1 -- 172.21.15.107:0/2782920502 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5a819a030 con 0x7fd5a80ffe80 2024-03-08T21:10:49.865 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.885+0000 7fd5a7400640 1 --2- 172.21.15.107:0/2782920502 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd5a810da30 0x7fd5a81998b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T21:10:49.865 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.886+0000 7fd5a6a00640 1 --2- 172.21.15.107:0/2782920502 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd5a80ffe80 0x7fd5a819f700 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fd59800d950 tx=0x7fd59800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:10:49.865 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.886+0000 7fd59f400640 1 -- 172.21.15.107:0/2782920502 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd598017070 con 0x7fd5a80ffe80 2024-03-08T21:10:49.865 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.886+0000 7fd59f400640 1 -- 172.21.15.107:0/2782920502 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fd598009c30 con 0x7fd5a80ffe80 2024-03-08T21:10:49.865 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.886+0000 7fd59f400640 1 -- 172.21.15.107:0/2782920502 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd59800f620 con 0x7fd5a80ffe80 2024-03-08T21:10:49.865 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.886+0000 7fd5a7e00640 1 -- 172.21.15.107:0/2782920502 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5a819a320 con 0x7fd5a80ffe80 2024-03-08T21:10:49.865 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.886+0000 7fd5a7e00640 1 -- 172.21.15.107:0/2782920502 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5a81a65c0 con 0x7fd5a80ffe80 2024-03-08T21:10:49.867 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.890+0000 7fd59f400640 1 -- 172.21.15.107:0/2782920502 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) v1 ==== 137422+0+0 (secure 0 0 0) 0x7fd5980109a0 con 0x7fd5a80ffe80 2024-03-08T21:10:49.868 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.890+0000 7fd5b057a740 1 Processor -- start 2024-03-08T21:10:49.868 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.890+0000 7fd5b057a740 1 -- start start 2024-03-08T21:10:49.869 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.891+0000 7fd5b057a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561245e8ae80 0x561245e8b260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:10:49.869 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.891+0000 7fd5b057a740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561245e8b7a0 0x561245e94030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:10:49.869 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.891+0000 7fd5b057a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561245e94570 0x561245e96960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:10:49.869 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.891+0000 7fd5b057a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x561245dee3d0 con 0x561245e94570 2024-03-08T21:10:49.869 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.891+0000 7fd5b057a740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x561245d976b0 con 0x561245e8b7a0 2024-03-08T21:10:49.870 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.891+0000 7fd5b057a740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x561245dea470 con 0x561245e8ae80 2024-03-08T21:10:49.870 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.891+0000 7fd5a6000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561245e94570 0x561245e96960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:10:49.870 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.891+0000 7fd5a6000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561245e94570 0x561245e96960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:39436/0 (socket says 172.21.15.107:39436) 2024-03-08T21:10:49.870 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.891+0000 7fd5a6000640 1 -- 172.21.15.107:0/2321412805 learned_addr learned my addr 172.21.15.107:0/2321412805 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:10:49.870 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.891+0000 7fd5a7400640 1 --2- 172.21.15.107:0/2321412805 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561245e8ae80 0x561245e8b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:10:49.870 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.892+0000 7fd5a6a00640 1 --2- 172.21.15.107:0/2321412805 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561245e8b7a0 0x561245e94030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:10:49.870 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.892+0000 7fd5a6000640 1 -- 172.21.15.107:0/2321412805 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561245e8ae80 msgr2=0x561245e8b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:10:49.870 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.892+0000 7fd5a6000640 1 --2- 172.21.15.107:0/2321412805 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561245e8ae80 0x561245e8b260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:10:49.870 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.892+0000 7fd5a6000640 1 -- 172.21.15.107:0/2321412805 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561245e8b7a0 msgr2=0x561245e94030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:10:49.870 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.892+0000 7fd5a6000640 1 --2- 172.21.15.107:0/2321412805 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561245e8b7a0 0x561245e94030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:10:49.870 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.892+0000 7fd5a6000640 1 -- 172.21.15.107:0/2321412805 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561245dbdb50 con 0x561245e94570 2024-03-08T21:10:49.870 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.892+0000 7fd5a7400640 1 --2- 172.21.15.107:0/2321412805 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561245e8ae80 0x561245e8b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T21:10:49.872 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.892+0000 7fd59f400640 1 --2- 172.21.15.107:0/2782920502 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7fd58c0a2960 0x7fd58c0a4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:10:49.872 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.892+0000 7fd5a6000640 1 --2- 172.21.15.107:0/2321412805 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561245e94570 0x561245e96960 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fd590009230 tx=0x7fd59002c720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e8d4cb9a31bbcc2 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:10:49.872 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.892+0000 7fd59f400640 1 -- 172.21.15.107:0/2782920502 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fd5980ceb60 con 0x7fd5a80ffe80 2024-03-08T21:10:49.872 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.893+0000 7fd5a7400640 1 --2- 172.21.15.107:0/2782920502 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7fd58c0a2960 0x7fd58c0a4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:10:49.872 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.893+0000 7fd597e00640 1 -- 172.21.15.107:0/2321412805 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd590045020 con 0x561245e94570 2024-03-08T21:10:49.872 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.893+0000 7fd597e00640 1 -- 172.21.15.107:0/2321412805 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fd59002c7e0 con 0x561245e94570 2024-03-08T21:10:49.873 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.893+0000 7fd597e00640 1 -- 172.21.15.107:0/2321412805 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5900046c0 con 0x561245e94570 2024-03-08T21:10:49.873 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.894+0000 7fd5b057a740 1 -- 172.21.15.107:0/2321412805 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561245e94570 msgr2=0x561245e96960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:10:49.873 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.894+0000 7fd5b057a740 1 --2- 172.21.15.107:0/2321412805 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561245e94570 0x561245e96960 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fd590009230 tx=0x7fd59002c720 comp rx=0 tx=0).stop 2024-03-08T21:10:49.873 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.894+0000 7fd5a7400640 1 --2- 172.21.15.107:0/2782920502 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7fd58c0a2960 0x7fd58c0a4e20 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fd5a8102060 tx=0x7fd5a0007910 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:10:49.873 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.894+0000 7fd5b057a740 1 -- 172.21.15.107:0/2321412805 shutdown_connections 2024-03-08T21:10:49.873 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.894+0000 7fd5b057a740 1 --2- 172.21.15.107:0/2321412805 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561245e94570 0x561245e96960 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:10:49.873 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.894+0000 7fd5b057a740 1 --2- 172.21.15.107:0/2321412805 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561245e8b7a0 0x561245e94030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:10:49.873 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.894+0000 7fd5b057a740 1 --2- 172.21.15.107:0/2321412805 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561245e8ae80 0x561245e8b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:10:49.873 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.894+0000 7fd5b057a740 1 -- 172.21.15.107:0/2321412805 >> 172.21.15.107:0/2321412805 conn(0x561245e857a0 msgr2=0x561245e85b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:10:49.873 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.894+0000 7fd5b057a740 1 -- 172.21.15.107:0/2321412805 shutdown_connections 2024-03-08T21:10:49.873 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.894+0000 7fd5b057a740 1 -- 172.21.15.107:0/2321412805 wait complete. 2024-03-08T21:10:49.873 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.895+0000 7fd5b057a740 1 -- start start 2024-03-08T21:10:49.873 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.895+0000 7fd5b057a740 1 Processor -- start 2024-03-08T21:10:49.876 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.895+0000 7fd5b057a740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561245e8ae80 0x561245e93f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:10:49.876 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.896+0000 7fd5b057a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561245e8b7a0 0x561245e912a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:10:49.877 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.896+0000 7fd5b057a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561245e94570 0x561245e91840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:10:49.877 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.896+0000 7fd5b057a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x561245e97140 con 0x561245e94570 2024-03-08T21:10:49.877 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.896+0000 7fd5b057a740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x561245daa9f0 con 0x561245e8ae80 2024-03-08T21:10:49.877 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.896+0000 7fd5b057a740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x561245e972c0 con 0x561245e8b7a0 2024-03-08T21:10:49.877 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.896+0000 7fd5b057a740 10 client.0 _ll_register_callbacks cb 0x561245b02a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T21:10:49.877 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.896+0000 7fd5a7400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561245e94570 0x561245e91840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:10:49.877 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.896+0000 7fd5a7400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561245e94570 0x561245e91840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:39442/0 (socket says 172.21.15.107:39442) 2024-03-08T21:10:49.877 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.896+0000 7fd5a7400640 1 -- 172.21.15.107:0/2486876910 learned_addr learned my addr 172.21.15.107:0/2486876910 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:10:49.877 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.896+0000 7fd5a6a00640 1 --2- 172.21.15.107:0/2486876910 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561245e8ae80 0x561245e93f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:10:49.877 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.896+0000 7fd5a6000640 1 --2- 172.21.15.107:0/2486876910 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561245e8b7a0 0x561245e912a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:10:49.877 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.896+0000 7fd5a7400640 1 -- 172.21.15.107:0/2486876910 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561245e8b7a0 msgr2=0x561245e912a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:10:49.877 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.896+0000 7fd5a7400640 1 --2- 172.21.15.107:0/2486876910 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561245e8b7a0 0x561245e912a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:10:49.878 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.896+0000 7fd5a7400640 1 -- 172.21.15.107:0/2486876910 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561245e8ae80 msgr2=0x561245e93f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:10:49.878 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.896+0000 7fd5a7400640 1 --2- 172.21.15.107:0/2486876910 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561245e8ae80 0x561245e93f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:10:49.878 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.897+0000 7fd5a7400640 1 -- 172.21.15.107:0/2486876910 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561245b03210 con 0x561245e94570 2024-03-08T21:10:49.878 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.897+0000 7fd5a7400640 1 --2- 172.21.15.107:0/2486876910 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561245e94570 0x561245e91840 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fd5a0011e10 tx=0x7fd5a0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d4f0133a67682d server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:10:49.878 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.897+0000 7fd594c00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-03-08T21:10:49.878 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.897+0000 7fd594c00640 1 -- 172.21.15.107:0/2486876910 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5a0003b70 con 0x561245e94570 2024-03-08T21:10:49.878 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.897+0000 7fd594c00640 1 -- 172.21.15.107:0/2486876910 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fd5a0003d10 con 0x561245e94570 2024-03-08T21:10:49.878 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.897+0000 7fd5b057a740 1 -- 172.21.15.107:0/2486876910 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd59002c7e0 con 0x561245e94570 2024-03-08T21:10:49.878 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.898+0000 7fd594c00640 1 -- 172.21.15.107:0/2486876910 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd5a0011e40 con 0x561245e94570 2024-03-08T21:10:49.878 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.898+0000 7fd5b057a740 10 client.16317 fetch_fsmap learned FSMap version 38 2024-03-08T21:10:49.878 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.898+0000 7fd5b057a740 1 -- 172.21.15.107:0/2486876910 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x561245bcf620 con 0x561245e94570 2024-03-08T21:10:49.878 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.898+0000 7fd594c00640 1 -- 172.21.15.107:0/2486876910 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fd59002c7e0 con 0x561245e94570 2024-03-08T21:10:49.878 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.898+0000 7fd594c00640 1 -- 172.21.15.107:0/2486876910 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 38) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fd5a001e7f0 con 0x561245e94570 2024-03-08T21:10:49.879 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.898+0000 7fd5b057a740 10 client.16317 fetch_fsmap finished waiting for FSMap version 38 2024-03-08T21:10:49.879 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.898+0000 7fd5b057a740 10 client.16317 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-08T21:10:49.879 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.898+0000 7fd5b057a740 20 client.16317 populate_metadata read hostname 'smithi107' 2024-03-08T21:10:49.879 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.898+0000 7fd5b057a740 1 --2- 172.21.15.107:0/2486876910 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x561245ea4440 0x561245ea6830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:10:49.879 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.898+0000 7fd5b057a740 4 client.16317 mds_command: new command op to 34475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-08T21:10:49.879 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.898+0000 7fd5b057a740 1 -- 172.21.15.107:0/2486876910 --> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561245dfb910 con 0x561245ea4440 2024-03-08T21:10:49.879 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.899+0000 7fd5a6a00640 1 --2- 172.21.15.107:0/2486876910 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x561245ea4440 0x561245ea6830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:10:49.880 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.900+0000 7fd594c00640 10 client.16317 ms_handle_connect on v2:172.21.15.149:6838/55290596 2024-03-08T21:10:49.880 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.900+0000 7fd5a6a00640 1 --2- 172.21.15.107:0/2486876910 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x561245ea4440 0x561245ea6830 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d77c0011bde4a8ba server_cookie=2c59178468ca4a97 in_seq=0 out_seq=0 2024-03-08T21:10:49.880 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.901+0000 7fd594c00640 1 -- 172.21.15.107:0/2486876910 <== mds.0 v2:172.21.15.149:6838/55290596 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x561245dfb910 con 0x561245ea4440 2024-03-08T21:10:49.880 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.901+0000 7fd594c00640 10 client.16317 handle_command_reply: tid=0 2024-03-08T21:10:49.881 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.901+0000 7fd5b057a740 1 client.16317 shutdown 2024-03-08T21:10:49.881 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.902+0000 7fd5b057a740 1 -- 172.21.15.107:0/2486876910 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561245e94570 msgr2=0x561245e91840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:10:49.881 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.902+0000 7fd5b057a740 1 --2- 172.21.15.107:0/2486876910 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561245e94570 0x561245e91840 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fd5a0011e10 tx=0x7fd5a0004a40 comp rx=0 tx=0).stop 2024-03-08T21:10:49.881 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.902+0000 7fd5b057a740 1 -- 172.21.15.107:0/2486876910 shutdown_connections 2024-03-08T21:10:49.881 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.902+0000 7fd5b057a740 1 --2- 172.21.15.107:0/2486876910 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561245e94570 0x561245e91840 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:10:49.881 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.902+0000 7fd5b057a740 1 --2- 172.21.15.107:0/2486876910 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561245e8b7a0 0x561245e912a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:10:49.881 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.902+0000 7fd5b057a740 1 --2- 172.21.15.107:0/2486876910 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x561245ea4440 0x561245ea6830 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:10:49.881 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.902+0000 7fd5b057a740 1 --2- 172.21.15.107:0/2486876910 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561245e8ae80 0x561245e93f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:10:49.881 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.902+0000 7fd5b057a740 1 -- 172.21.15.107:0/2486876910 >> 172.21.15.107:0/2486876910 conn(0x561245e857a0 msgr2=0x561245e874f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:10:49.881 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.902+0000 7fd594c00640 0 client.16317 ms_handle_reset on v2:172.21.15.149:6838/55290596 2024-03-08T21:10:49.881 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.902+0000 7fd5b057a740 1 -- 172.21.15.107:0/2486876910 shutdown_connections 2024-03-08T21:10:49.881 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.902+0000 7fd5b057a740 1 -- 172.21.15.107:0/2486876910 wait complete. 2024-03-08T21:10:49.881 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.902+0000 7fd5b057a740 20 client.16317 trim_cache size 0 max 16384 2024-03-08T21:10:49.897 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.918+0000 7fd5b057a740 1 Processor -- start 2024-03-08T21:10:49.897 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.918+0000 7fd5b057a740 1 -- start start 2024-03-08T21:10:49.897 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.918+0000 7fd5b057a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561245e8ae80 0x561245e939d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:10:49.897 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.918+0000 7fd5b057a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561245e8b7a0 0x561245e93f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:10:49.897 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.918+0000 7fd5b057a740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561245e94570 0x561245e8fb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:10:49.897 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.918+0000 7fd5b057a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5a81aec60 con 0x561245e8ae80 2024-03-08T21:10:49.897 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.918+0000 7fd5b057a740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5a81b05e0 con 0x561245e94570 2024-03-08T21:10:49.897 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.918+0000 7fd5b057a740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5a81af2c0 con 0x561245e8b7a0 2024-03-08T21:10:49.897 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.919+0000 7fd5a6a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561245e8ae80 0x561245e939d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:10:49.897 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.919+0000 7fd5a6a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561245e8ae80 0x561245e939d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:39444/0 (socket says 172.21.15.107:39444) 2024-03-08T21:10:49.897 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.919+0000 7fd5a6a00640 1 -- 172.21.15.107:0/2433750148 learned_addr learned my addr 172.21.15.107:0/2433750148 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:10:49.898 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.919+0000 7fd5a6000640 1 --2- 172.21.15.107:0/2433750148 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561245e8b7a0 0x561245e93f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:10:49.898 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.919+0000 7fd5a6a00640 1 -- 172.21.15.107:0/2433750148 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561245e8b7a0 msgr2=0x561245e93f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:10:49.898 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.919+0000 7fd5a6a00640 1 --2- 172.21.15.107:0/2433750148 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561245e8b7a0 0x561245e93f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:10:49.898 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.919+0000 7fd5a7400640 1 --2- 172.21.15.107:0/2433750148 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561245e94570 0x561245e8fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:10:49.898 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.919+0000 7fd5a6a00640 1 -- 172.21.15.107:0/2433750148 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561245e94570 msgr2=0x561245e8fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:10:49.898 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.919+0000 7fd5a6a00640 1 --2- 172.21.15.107:0/2433750148 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561245e94570 0x561245e8fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:10:49.898 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.919+0000 7fd5a6a00640 1 -- 172.21.15.107:0/2433750148 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561245eaba60 con 0x561245e8ae80 2024-03-08T21:10:49.898 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.919+0000 7fd5a6000640 1 --2- 172.21.15.107:0/2433750148 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561245e8b7a0 0x561245e93f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T21:10:49.898 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.919+0000 7fd5a7400640 1 --2- 172.21.15.107:0/2433750148 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561245e94570 0x561245e8fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T21:10:49.901 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.920+0000 7fd5a6a00640 1 --2- 172.21.15.107:0/2433750148 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561245e8ae80 0x561245e939d0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fd59809b0b0 tx=0x7fd5980bb040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4ac521847d4b90a server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:10:49.901 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.920+0000 7fd597400640 1 -- 172.21.15.107:0/2433750148 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd598020040 con 0x561245e8ae80 2024-03-08T21:10:49.901 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.920+0000 7fd597400640 1 -- 172.21.15.107:0/2433750148 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fd5980bb0b0 con 0x561245e8ae80 2024-03-08T21:10:49.901 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.920+0000 7fd597400640 1 -- 172.21.15.107:0/2433750148 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5980a38c0 con 0x561245e8ae80 2024-03-08T21:10:49.901 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.921+0000 7fd5b057a740 1 -- 172.21.15.107:0/2433750148 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561245e8ae80 msgr2=0x561245e939d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:10:49.901 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.921+0000 7fd5b057a740 1 --2- 172.21.15.107:0/2433750148 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561245e8ae80 0x561245e939d0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fd59809b0b0 tx=0x7fd5980bb040 comp rx=0 tx=0).stop 2024-03-08T21:10:49.901 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.921+0000 7fd5b057a740 1 -- 172.21.15.107:0/2433750148 shutdown_connections 2024-03-08T21:10:49.901 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.921+0000 7fd5b057a740 1 --2- 172.21.15.107:0/2433750148 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561245e94570 0x561245e8fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:10:49.901 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.921+0000 7fd5b057a740 1 --2- 172.21.15.107:0/2433750148 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561245e8b7a0 0x561245e93f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:10:49.901 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.921+0000 7fd5b057a740 1 --2- 172.21.15.107:0/2433750148 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561245e8ae80 0x561245e939d0 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:10:49.902 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.921+0000 7fd5b057a740 1 -- 172.21.15.107:0/2433750148 >> 172.21.15.107:0/2433750148 conn(0x561245e857a0 msgr2=0x561245e91260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:10:49.902 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.921+0000 7fd5b057a740 1 -- 172.21.15.107:0/2433750148 shutdown_connections 2024-03-08T21:10:49.902 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.921+0000 7fd5b057a740 1 -- 172.21.15.107:0/2433750148 wait complete. 2024-03-08T21:10:49.902 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.921+0000 7fd5b057a740 1 -- start start 2024-03-08T21:10:49.902 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.921+0000 7fd5b057a740 1 Processor -- start 2024-03-08T21:10:49.902 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.922+0000 7fd5b057a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561245e8ae80 0x561245e9b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:10:49.902 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.922+0000 7fd5b057a740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561245e8b7a0 0x561245e99a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:10:49.902 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.922+0000 7fd5b057a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561245e94570 0x561245e99fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:10:49.902 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.922+0000 7fd5b057a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5a81aff80 con 0x561245e94570 2024-03-08T21:10:49.902 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.922+0000 7fd5b057a740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5a81af920 con 0x561245e8b7a0 2024-03-08T21:10:49.902 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.922+0000 7fd5b057a740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5a81acf20 con 0x561245e8ae80 2024-03-08T21:10:49.902 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.922+0000 7fd5b057a740 10 client.0 _ll_register_callbacks cb 0x7fd590005e20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T21:10:49.902 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.922+0000 7fd5a7400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561245e94570 0x561245e99fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:10:49.903 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.922+0000 7fd5a7400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561245e94570 0x561245e99fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:39452/0 (socket says 172.21.15.107:39452) 2024-03-08T21:10:49.903 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.922+0000 7fd5a7400640 1 -- 172.21.15.107:0/452129411 learned_addr learned my addr 172.21.15.107:0/452129411 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:10:49.903 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.923+0000 7fd5a6a00640 1 --2- 172.21.15.107:0/452129411 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561245e8ae80 0x561245e9b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:10:49.903 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.923+0000 7fd5a6000640 1 --2- 172.21.15.107:0/452129411 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561245e8b7a0 0x561245e99a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:10:49.903 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.923+0000 7fd5a7400640 1 -- 172.21.15.107:0/452129411 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561245e8ae80 msgr2=0x561245e9b6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:10:49.903 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.923+0000 7fd5a7400640 1 --2- 172.21.15.107:0/452129411 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561245e8ae80 0x561245e9b6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:10:49.903 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.923+0000 7fd5a7400640 1 -- 172.21.15.107:0/452129411 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561245e8b7a0 msgr2=0x561245e99a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:10:49.903 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.923+0000 7fd5a7400640 1 --2- 172.21.15.107:0/452129411 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561245e8b7a0 0x561245e99a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:10:49.903 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.923+0000 7fd5a7400640 1 -- 172.21.15.107:0/452129411 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561245e9a610 con 0x561245e94570 2024-03-08T21:10:49.903 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.923+0000 7fd5a6000640 1 --2- 172.21.15.107:0/452129411 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561245e8b7a0 0x561245e99a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T21:10:49.903 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.923+0000 7fd5a7400640 1 --2- 172.21.15.107:0/452129411 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561245e94570 0x561245e99fd0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fd5a001ece0 tx=0x7fd5a001e6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76f82a1ae149bf15 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:10:49.903 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.923+0000 7fd594c00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-03-08T21:10:49.904 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.923+0000 7fd594c00640 1 -- 172.21.15.107:0/452129411 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5a0032040 con 0x561245e94570 2024-03-08T21:10:49.904 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.923+0000 7fd5b057a740 1 -- 172.21.15.107:0/452129411 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd5980bb0b0 con 0x561245e94570 2024-03-08T21:10:49.904 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.923+0000 7fd594c00640 1 -- 172.21.15.107:0/452129411 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fd5a0005040 con 0x561245e94570 2024-03-08T21:10:49.905 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.924+0000 7fd594c00640 1 -- 172.21.15.107:0/452129411 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd5a0043050 con 0x561245e94570 2024-03-08T21:10:49.905 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.924+0000 7fd5b057a740 10 client.16323 fetch_fsmap learned FSMap version 38 2024-03-08T21:10:49.905 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.924+0000 7fd5b057a740 1 -- 172.21.15.107:0/452129411 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x561245efb730 con 0x561245e94570 2024-03-08T21:10:49.905 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.924+0000 7fd594c00640 1 -- 172.21.15.107:0/452129411 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fd5980bb0b0 con 0x561245e94570 2024-03-08T21:10:49.905 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.924+0000 7fd594c00640 1 -- 172.21.15.107:0/452129411 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 38) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fd5a003e030 con 0x561245e94570 2024-03-08T21:10:49.905 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.924+0000 7fd5b057a740 10 client.16323 fetch_fsmap finished waiting for FSMap version 38 2024-03-08T21:10:49.905 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.924+0000 7fd5b057a740 10 client.16323 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-08T21:10:49.905 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.924+0000 7fd5b057a740 20 client.16323 populate_metadata read hostname 'smithi107' 2024-03-08T21:10:49.905 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.924+0000 7fd5b057a740 1 --2- 172.21.15.107:0/452129411 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x561245efbe20 0x561245efe240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:10:49.906 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.924+0000 7fd5b057a740 4 client.16323 mds_command: new command op to 34475 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-08T21:10:49.906 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.924+0000 7fd5b057a740 1 -- 172.21.15.107:0/452129411 --> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x561245df5c60 con 0x561245efbe20 2024-03-08T21:10:49.906 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.925+0000 7fd5a6a00640 1 --2- 172.21.15.107:0/452129411 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x561245efbe20 0x561245efe240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:10:49.906 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.926+0000 7fd5a6a00640 1 --2- 172.21.15.107:0/452129411 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x561245efbe20 0x561245efe240 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=badf40d642bc209 server_cookie=c33c6e735c001efe in_seq=0 out_seq=0 2024-03-08T21:10:49.906 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.926+0000 7fd594c00640 10 client.16323 ms_handle_connect on v2:172.21.15.149:6838/55290596 2024-03-08T21:10:49.925 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.946+0000 7fd594c00640 1 -- 172.21.15.107:0/452129411 <== mds.0 v2:172.21.15.149:6838/55290596 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x561245df5c60 con 0x561245efbe20 2024-03-08T21:10:49.925 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.946+0000 7fd594c00640 10 client.16323 handle_command_reply: tid=0 2024-03-08T21:10:49.925 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.946+0000 7fd5b057a740 1 client.16323 shutdown 2024-03-08T21:10:49.925 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.947+0000 7fd5b057a740 1 -- 172.21.15.107:0/452129411 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561245e94570 msgr2=0x561245e99fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:10:49.925 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.947+0000 7fd5b057a740 1 --2- 172.21.15.107:0/452129411 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561245e94570 0x561245e99fd0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fd5a001ece0 tx=0x7fd5a001e6d0 comp rx=0 tx=0).stop 2024-03-08T21:10:49.925 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.947+0000 7fd5b057a740 1 -- 172.21.15.107:0/452129411 shutdown_connections 2024-03-08T21:10:49.925 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.947+0000 7fd5b057a740 1 --2- 172.21.15.107:0/452129411 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561245e94570 0x561245e99fd0 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:10:49.925 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.947+0000 7fd5b057a740 1 --2- 172.21.15.107:0/452129411 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x561245efbe20 0x561245efe240 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:10:49.925 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.947+0000 7fd5b057a740 1 --2- 172.21.15.107:0/452129411 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561245e8b7a0 0x561245e99a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:10:49.925 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.947+0000 7fd5b057a740 1 --2- 172.21.15.107:0/452129411 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561245e8ae80 0x561245e9b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:10:49.925 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.947+0000 7fd5b057a740 1 -- 172.21.15.107:0/452129411 >> 172.21.15.107:0/452129411 conn(0x561245e857a0 msgr2=0x561245e86bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:10:49.926 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.947+0000 7fd594c00640 0 client.16323 ms_handle_reset on v2:172.21.15.149:6838/55290596 2024-03-08T21:10:49.926 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.947+0000 7fd5b057a740 1 -- 172.21.15.107:0/452129411 shutdown_connections 2024-03-08T21:10:49.926 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.947+0000 7fd5b057a740 1 -- 172.21.15.107:0/452129411 wait complete. 2024-03-08T21:10:49.926 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.947+0000 7fd5b057a740 20 client.16323 trim_cache size 0 max 16384 2024-03-08T21:10:49.926 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:{ 2024-03-08T21:10:49.926 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "message": "", 2024-03-08T21:10:49.926 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "return_code": 0 2024-03-08T21:10:49.926 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:} 2024-03-08T21:10:49.927 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.948+0000 7fd5a7e00640 1 -- 172.21.15.107:0/2782920502 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7fd58c0a2960 msgr2=0x7fd58c0a4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:10:49.927 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.948+0000 7fd5a7e00640 1 --2- 172.21.15.107:0/2782920502 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7fd58c0a2960 0x7fd58c0a4e20 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fd5a8102060 tx=0x7fd5a0007910 comp rx=0 tx=0).stop 2024-03-08T21:10:49.927 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.948+0000 7fd5a7e00640 1 -- 172.21.15.107:0/2782920502 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd5a80ffe80 msgr2=0x7fd5a819f700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:10:49.927 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.948+0000 7fd5a7e00640 1 --2- 172.21.15.107:0/2782920502 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd5a80ffe80 0x7fd5a819f700 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fd59800d950 tx=0x7fd59800de10 comp rx=0 tx=0).stop 2024-03-08T21:10:49.927 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.948+0000 7fd5a7e00640 1 -- 172.21.15.107:0/2782920502 shutdown_connections 2024-03-08T21:10:49.927 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.948+0000 7fd5a7e00640 1 --2- 172.21.15.107:0/2782920502 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7fd58c0a2960 0x7fd58c0a4e20 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:10:49.927 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.948+0000 7fd5a7e00640 1 --2- 172.21.15.107:0/2782920502 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd5a810da30 0x7fd5a81998b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:10:49.928 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.948+0000 7fd5a7e00640 1 --2- 172.21.15.107:0/2782920502 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd5a8100830 0x7fd5a819fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:10:49.928 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.948+0000 7fd5a7e00640 1 --2- 172.21.15.107:0/2782920502 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd5a80ffe80 0x7fd5a819f700 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:10:49.928 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.948+0000 7fd5a7e00640 1 -- 172.21.15.107:0/2782920502 >> 172.21.15.107:0/2782920502 conn(0x7fd5a80fbb00 msgr2=0x7fd5a810ed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:10:49.928 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.948+0000 7fd5a7e00640 1 -- 172.21.15.107:0/2782920502 shutdown_connections 2024-03-08T21:10:49.928 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:10:49.948+0000 7fd5a7e00640 1 -- 172.21.15.107:0/2782920502 wait complete. 2024-03-08T21:10:50.623 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:10:50 smithi204 ceph-mon[34935]: pgmap v324: 97 pgs: 97 active+clean; 7.1 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 5.2 KiB/s wr, 15 op/s 2024-03-08T21:10:50.712 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:10:50 smithi107 ceph-mon[31381]: pgmap v324: 97 pgs: 97 active+clean; 7.1 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 5.2 KiB/s wr, 15 op/s 2024-03-08T21:10:50.760 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:10:50 smithi149 ceph-mon[35042]: pgmap v324: 97 pgs: 97 active+clean; 7.1 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 5.2 KiB/s wr, 15 op/s 2024-03-08T21:10:52.623 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:10:52 smithi204 ceph-mon[34935]: pgmap v325: 97 pgs: 97 active+clean; 7.1 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-08T21:10:52.712 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:10:52 smithi107 ceph-mon[31381]: pgmap v325: 97 pgs: 97 active+clean; 7.1 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-08T21:10:52.760 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:10:52 smithi149 ceph-mon[35042]: pgmap v325: 97 pgs: 97 active+clean; 7.1 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-08T21:10:54.623 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:10:54 smithi204 ceph-mon[34935]: pgmap v326: 97 pgs: 97 active+clean; 7.1 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-08T21:10:54.712 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:10:54 smithi107 ceph-mon[31381]: pgmap v326: 97 pgs: 97 active+clean; 7.1 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-08T21:10:54.760 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:10:54 smithi149 ceph-mon[35042]: pgmap v326: 97 pgs: 97 active+clean; 7.1 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-08T21:10:56.623 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:10:56 smithi204 ceph-mon[34935]: pgmap v327: 97 pgs: 97 active+clean; 7.1 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 2.2 KiB/s wr, 14 op/s 2024-03-08T21:10:56.712 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:10:56 smithi107 ceph-mon[31381]: pgmap v327: 97 pgs: 97 active+clean; 7.1 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 2.2 KiB/s wr, 14 op/s 2024-03-08T21:10:56.760 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:10:56 smithi149 ceph-mon[35042]: pgmap v327: 97 pgs: 97 active+clean; 7.1 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 2.2 KiB/s wr, 14 op/s 2024-03-08T21:10:58.623 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:10:58 smithi204 ceph-mon[34935]: pgmap v328: 97 pgs: 97 active+clean; 7.1 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.2 KiB/s wr, 10 op/s 2024-03-08T21:10:58.712 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:10:58 smithi107 ceph-mon[31381]: pgmap v328: 97 pgs: 97 active+clean; 7.1 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.2 KiB/s wr, 10 op/s 2024-03-08T21:10:58.760 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:10:58 smithi149 ceph-mon[35042]: pgmap v328: 97 pgs: 97 active+clean; 7.1 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.2 KiB/s wr, 10 op/s 2024-03-08T21:11:00.623 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:11:00 smithi204 ceph-mon[34935]: pgmap v329: 97 pgs: 97 active+clean; 7.1 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 2.2 KiB/s wr, 14 op/s 2024-03-08T21:11:00.623 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:11:00 smithi204 ceph-mon[34935]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T21:11:00.712 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:11:00 smithi107 ceph-mon[31381]: pgmap v329: 97 pgs: 97 active+clean; 7.1 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 2.2 KiB/s wr, 14 op/s 2024-03-08T21:11:00.712 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:11:00 smithi107 ceph-mon[31381]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T21:11:00.759 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:11:00 smithi149 ceph-mon[35042]: pgmap v329: 97 pgs: 97 active+clean; 7.1 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 2.2 KiB/s wr, 14 op/s 2024-03-08T21:11:00.759 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:11:00 smithi149 ceph-mon[35042]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T21:11:01.873 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:11:01 smithi204 ceph-mon[34935]: from='mgr.15375 ' entity='mgr.x' 2024-03-08T21:11:01.873 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:11:01 smithi204 ceph-mon[34935]: from='mgr.15375 ' entity='mgr.x' 2024-03-08T21:11:01.962 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:11:01 smithi107 ceph-mon[31381]: from='mgr.15375 ' entity='mgr.x' 2024-03-08T21:11:01.962 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:11:01 smithi107 ceph-mon[31381]: from='mgr.15375 ' entity='mgr.x' 2024-03-08T21:11:02.009 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:11:01 smithi149 ceph-mon[35042]: from='mgr.15375 ' entity='mgr.x' 2024-03-08T21:11:02.010 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:11:01 smithi149 ceph-mon[35042]: from='mgr.15375 ' entity='mgr.x' 2024-03-08T21:11:02.873 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:11:02 smithi204 ceph-mon[34935]: pgmap v330: 97 pgs: 97 active+clean; 7.1 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.2 KiB/s wr, 10 op/s 2024-03-08T21:11:02.873 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:11:02 smithi204 ceph-mon[34935]: from='mgr.15375 ' entity='mgr.x' 2024-03-08T21:11:02.873 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:11:02 smithi204 ceph-mon[34935]: from='mgr.15375 ' entity='mgr.x' 2024-03-08T21:11:02.873 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:11:02 smithi204 ceph-mon[34935]: from='mgr.15375 ' entity='mgr.x' 2024-03-08T21:11:02.962 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:11:02 smithi107 ceph-mon[31381]: pgmap v330: 97 pgs: 97 active+clean; 7.1 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.2 KiB/s wr, 10 op/s 2024-03-08T21:11:02.962 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:11:02 smithi107 ceph-mon[31381]: from='mgr.15375 ' entity='mgr.x' 2024-03-08T21:11:02.962 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:11:02 smithi107 ceph-mon[31381]: from='mgr.15375 ' entity='mgr.x' 2024-03-08T21:11:02.962 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:11:02 smithi107 ceph-mon[31381]: from='mgr.15375 ' entity='mgr.x' 2024-03-08T21:11:03.010 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:11:02 smithi149 ceph-mon[35042]: pgmap v330: 97 pgs: 97 active+clean; 7.1 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.2 KiB/s wr, 10 op/s 2024-03-08T21:11:03.010 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:11:02 smithi149 ceph-mon[35042]: from='mgr.15375 ' entity='mgr.x' 2024-03-08T21:11:03.010 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:11:02 smithi149 ceph-mon[35042]: from='mgr.15375 ' entity='mgr.x' 2024-03-08T21:11:03.011 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:11:02 smithi149 ceph-mon[35042]: from='mgr.15375 ' entity='mgr.x' 2024-03-08T21:11:03.078 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:03.101+0000 7f8d83e00640 1 -- 172.21.15.107:0/2352216611 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8d84106410 msgr2=0x7f8d8410dfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:11:03.079 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:03.101+0000 7f8d83e00640 1 --2- 172.21.15.107:0/2352216611 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8d84106410 0x7f8d8410dfe0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f8d7c009ca0 tx=0x7f8d7c02c0f0 comp rx=0 tx=0).stop 2024-03-08T21:11:03.079 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:03.101+0000 7f8d83e00640 1 -- 172.21.15.107:0/2352216611 shutdown_connections 2024-03-08T21:11:03.079 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:03.101+0000 7f8d83e00640 1 --2- 172.21.15.107:0/2352216611 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8d84106410 0x7f8d8410dfe0 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:11:03.079 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:03.101+0000 7f8d83e00640 1 --2- 172.21.15.107:0/2352216611 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8d84101cf0 0x7f8d84105d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:11:03.079 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:03.101+0000 7f8d83e00640 1 --2- 172.21.15.107:0/2352216611 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8d84101340 0x7f8d84101720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:11:03.079 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:03.101+0000 7f8d83e00640 1 -- 172.21.15.107:0/2352216611 >> 172.21.15.107:0/2352216611 conn(0x7f8d840fcc40 msgr2=0x7f8d840ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:11:03.079 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:03.101+0000 7f8d83e00640 1 -- 172.21.15.107:0/2352216611 shutdown_connections 2024-03-08T21:11:03.079 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:03.101+0000 7f8d83e00640 1 -- 172.21.15.107:0/2352216611 wait complete. 2024-03-08T21:11:03.079 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:03.102+0000 7f8d83e00640 1 Processor -- start 2024-03-08T21:11:03.080 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:03.102+0000 7f8d83e00640 1 -- start start 2024-03-08T21:11:03.081 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:03.103+0000 7f8d83e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8d84101340 0x7f8d8419f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:11:03.081 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:03.103+0000 7f8d83e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8d84106410 0x7f8d8419fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:11:03.081 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:03.103+0000 7f8d83e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8d841998c0 0x7f8d84199d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:11:03.081 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:03.103+0000 7f8d83e00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d84111360 con 0x7f8d84106410 2024-03-08T21:11:03.081 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:03.103+0000 7f8d83e00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d841111e0 con 0x7f8d84101340 2024-03-08T21:11:03.082 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:03.103+0000 7f8d83e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d841114e0 con 0x7f8d841998c0 2024-03-08T21:11:03.082 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:03.103+0000 7f8d82000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8d84106410 0x7f8d8419fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:11:03.082 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:03.103+0000 7f8d82000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8d84106410 0x7f8d8419fc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:39420/0 (socket says 172.21.15.107:39420) 2024-03-08T21:11:03.082 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:03.103+0000 7f8d82000640 1 -- 172.21.15.107:0/3540534031 learned_addr learned my addr 172.21.15.107:0/3540534031 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:11:03.082 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:03.103+0000 7f8d83400640 1 --2- 172.21.15.107:0/3540534031 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8d841998c0 0x7f8d84199d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:11:03.082 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:03.103+0000 7f8d82a00640 1 --2- 172.21.15.107:0/3540534031 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8d84101340 0x7f8d8419f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:11:03.082 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:03.104+0000 7f8d82000640 1 -- 172.21.15.107:0/3540534031 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8d841998c0 msgr2=0x7f8d84199d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:11:03.082 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:03.104+0000 7f8d82000640 1 --2- 172.21.15.107:0/3540534031 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8d841998c0 0x7f8d84199d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:11:03.082 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:03.104+0000 7f8d82000640 1 -- 172.21.15.107:0/3540534031 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8d84101340 msgr2=0x7f8d8419f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:11:03.082 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:03.104+0000 7f8d82000640 1 --2- 172.21.15.107:0/3540534031 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8d84101340 0x7f8d8419f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:11:03.082 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:03.104+0000 7f8d82000640 1 -- 172.21.15.107:0/3540534031 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d8419a3e0 con 0x7f8d84106410 2024-03-08T21:11:03.082 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:03.104+0000 7f8d82a00640 1 --2- 172.21.15.107:0/3540534031 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8d84101340 0x7f8d8419f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T21:11:03.082 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:03.104+0000 7f8d82000640 1 --2- 172.21.15.107:0/3540534031 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8d84106410 0x7f8d8419fc50 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f8d70005fd0 tx=0x7f8d7000dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:11:03.083 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:03.104+0000 7f8d7b400640 1 -- 172.21.15.107:0/3540534031 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d70017070 con 0x7f8d84106410 2024-03-08T21:11:03.083 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:03.104+0000 7f8d7b400640 1 -- 172.21.15.107:0/3540534031 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f8d70009c30 con 0x7f8d84106410 2024-03-08T21:11:03.083 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:03.104+0000 7f8d83e00640 1 -- 172.21.15.107:0/3540534031 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d8419a6d0 con 0x7f8d84106410 2024-03-08T21:11:03.083 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:03.104+0000 7f8d7b400640 1 -- 172.21.15.107:0/3540534031 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d70003d40 con 0x7f8d84106410 2024-03-08T21:11:03.083 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:03.104+0000 7f8d83e00640 1 -- 172.21.15.107:0/3540534031 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d841a6920 con 0x7f8d84106410 2024-03-08T21:11:03.085 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:03.108+0000 7f8d8c9e5740 1 Processor -- start 2024-03-08T21:11:03.085 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:03.108+0000 7f8d8c9e5740 1 -- start start 2024-03-08T21:11:03.087 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:03.109+0000 7f8d8c9e5740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f512624e80 0x55f512625260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:11:03.087 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:03.109+0000 7f8d8c9e5740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f5126257a0 0x55f51262e030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:11:03.087 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:03.109+0000 7f8d8c9e5740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f51262e570 0x55f512630960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:11:03.087 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:03.109+0000 7f8d8c9e5740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5125883d0 con 0x55f512624e80 2024-03-08T21:11:03.087 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:03.109+0000 7f8d8c9e5740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5125316b0 con 0x55f51262e570 2024-03-08T21:11:03.087 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:03.109+0000 7f8d8c9e5740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f512584470 con 0x55f5126257a0 2024-03-08T21:11:03.087 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:03.109+0000 7f8d83400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f512624e80 0x55f512625260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:11:03.087 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:03.109+0000 7f8d83400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f512624e80 0x55f512625260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:39428/0 (socket says 172.21.15.107:39428) 2024-03-08T21:11:03.087 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:03.109+0000 7f8d83400640 1 -- 172.21.15.107:0/45914896 learned_addr learned my addr 172.21.15.107:0/45914896 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:11:03.087 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:03.109+0000 7f8d7b400640 1 -- 172.21.15.107:0/3540534031 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) v1 ==== 137422+0+0 (secure 0 0 0) 0x7f8d70003ee0 con 0x7f8d84106410 2024-03-08T21:11:03.087 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:03.109+0000 7f8d82a00640 1 --2- 172.21.15.107:0/45914896 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f5126257a0 0x55f51262e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:11:03.089 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:03.110+0000 7f8d83400640 1 -- 172.21.15.107:0/45914896 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f5126257a0 msgr2=0x55f51262e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:11:03.089 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:03.110+0000 7f8d83400640 1 --2- 172.21.15.107:0/45914896 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f5126257a0 0x55f51262e030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:11:03.089 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:03.110+0000 7f8d83400640 1 -- 172.21.15.107:0/45914896 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f51262e570 msgr2=0x55f512630960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:11:03.089 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:03.110+0000 7f8d83400640 1 --2- 172.21.15.107:0/45914896 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f51262e570 0x55f512630960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:11:03.089 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:03.110+0000 7f8d83400640 1 -- 172.21.15.107:0/45914896 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f512557b50 con 0x55f512624e80 2024-03-08T21:11:03.089 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:03.110+0000 7f8d82a00640 1 --2- 172.21.15.107:0/45914896 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f5126257a0 0x55f51262e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T21:11:03.089 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:03.110+0000 7f8d83400640 1 --2- 172.21.15.107:0/45914896 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f512624e80 0x55f512625260 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f8d7c02c700 tx=0x7f8d7c005f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbc1ca32107f69f2 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:11:03.089 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:03.110+0000 7f8d6fe00640 1 -- 172.21.15.107:0/45914896 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d7c043070 con 0x55f512624e80 2024-03-08T21:11:03.089 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:03.110+0000 7f8d6fe00640 1 -- 172.21.15.107:0/45914896 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f8d7c0098a0 con 0x55f512624e80 2024-03-08T21:11:03.089 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:03.110+0000 7f8d6fe00640 1 -- 172.21.15.107:0/45914896 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d7c039040 con 0x55f512624e80 2024-03-08T21:11:03.089 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:03.111+0000 7f8d7b400640 1 --2- 172.21.15.107:0/3540534031 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f8d680a2960 0x7f8d680a4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:11:03.090 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:03.111+0000 7f8d7b400640 1 -- 172.21.15.107:0/3540534031 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f8d700ceb40 con 0x7f8d84106410 2024-03-08T21:11:03.090 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:03.111+0000 7f8d82a00640 1 --2- 172.21.15.107:0/3540534031 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f8d680a2960 0x7f8d680a4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:11:03.090 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:03.112+0000 7f8d8c9e5740 1 -- 172.21.15.107:0/45914896 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f512624e80 msgr2=0x55f512625260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:11:03.090 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:03.112+0000 7f8d8c9e5740 1 --2- 172.21.15.107:0/45914896 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f512624e80 0x55f512625260 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f8d7c02c700 tx=0x7f8d7c005f80 comp rx=0 tx=0).stop 2024-03-08T21:11:03.091 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:03.112+0000 7f8d8c9e5740 1 -- 172.21.15.107:0/45914896 shutdown_connections 2024-03-08T21:11:03.091 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:03.112+0000 7f8d8c9e5740 1 --2- 172.21.15.107:0/45914896 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f51262e570 0x55f512630960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:11:03.091 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:03.112+0000 7f8d8c9e5740 1 --2- 172.21.15.107:0/45914896 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f5126257a0 0x55f51262e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:11:03.091 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:03.112+0000 7f8d8c9e5740 1 --2- 172.21.15.107:0/45914896 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f512624e80 0x55f512625260 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:11:03.092 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:03.112+0000 7f8d8c9e5740 1 -- 172.21.15.107:0/45914896 >> 172.21.15.107:0/45914896 conn(0x55f51261f7a0 msgr2=0x55f51261fb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:11:03.092 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:03.112+0000 7f8d8c9e5740 1 -- 172.21.15.107:0/45914896 shutdown_connections 2024-03-08T21:11:03.092 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:03.112+0000 7f8d8c9e5740 1 -- 172.21.15.107:0/45914896 wait complete. 2024-03-08T21:11:03.092 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:03.112+0000 7f8d82a00640 1 --2- 172.21.15.107:0/3540534031 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f8d680a2960 0x7f8d680a4e20 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x55f512624930 tx=0x7f8d74005f90 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:11:03.092 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:03.113+0000 7f8d8c9e5740 1 -- start start 2024-03-08T21:11:03.092 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:03.113+0000 7f8d8c9e5740 1 Processor -- start 2024-03-08T21:11:03.092 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:03.114+0000 7f8d8c9e5740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f512624e80 0x55f51262df40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:11:03.092 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:03.114+0000 7f8d8c9e5740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f5126257a0 0x55f51262b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:11:03.092 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:03.114+0000 7f8d8c9e5740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f51262e570 0x55f51262b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:11:03.092 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:03.114+0000 7f8d8c9e5740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f512631140 con 0x55f51262e570 2024-03-08T21:11:03.092 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:03.114+0000 7f8d8c9e5740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5125449f0 con 0x55f5126257a0 2024-03-08T21:11:03.092 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:03.114+0000 7f8d8c9e5740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5126312c0 con 0x55f512624e80 2024-03-08T21:11:03.092 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:03.114+0000 7f8d8c9e5740 10 client.0 _ll_register_callbacks cb 0x55f51229ca00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T21:11:03.095 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:03.114+0000 7f8d82a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f512624e80 0x55f51262df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:11:03.095 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:03.114+0000 7f8d83400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f51262e570 0x55f51262b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:11:03.095 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:03.114+0000 7f8d83400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f51262e570 0x55f51262b8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:39434/0 (socket says 172.21.15.107:39434) 2024-03-08T21:11:03.095 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:03.114+0000 7f8d82a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f512624e80 0x55f51262df40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.107:60306/0 (socket says 172.21.15.107:60306) 2024-03-08T21:11:03.095 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:03.114+0000 7f8d83400640 1 -- 172.21.15.107:0/589314186 learned_addr learned my addr 172.21.15.107:0/589314186 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:11:03.095 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:03.114+0000 7f8d82000640 1 --2- 172.21.15.107:0/589314186 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f5126257a0 0x55f51262b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:11:03.095 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:03.115+0000 7f8d82a00640 1 -- 172.21.15.107:0/589314186 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f5126257a0 msgr2=0x55f51262b2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:11:03.095 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:03.115+0000 7f8d82a00640 1 --2- 172.21.15.107:0/589314186 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f5126257a0 0x55f51262b2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:11:03.095 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:03.115+0000 7f8d82a00640 1 -- 172.21.15.107:0/589314186 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f51262e570 msgr2=0x55f51262b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:11:03.095 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:03.115+0000 7f8d82a00640 1 --2- 172.21.15.107:0/589314186 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f51262e570 0x55f51262b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:11:03.095 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:03.115+0000 7f8d82a00640 1 -- 172.21.15.107:0/589314186 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f51229d210 con 0x55f512624e80 2024-03-08T21:11:03.096 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:03.115+0000 7f8d82a00640 1 --2- 172.21.15.107:0/589314186 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f512624e80 0x55f51262df40 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f8d74011c30 tx=0x7f8d740046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3d307d43001e406e server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:11:03.096 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:03.115+0000 7f8d6cc00640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-03-08T21:11:03.096 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:03.115+0000 7f8d6cc00640 1 -- 172.21.15.107:0/589314186 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d7402b060 con 0x55f512624e80 2024-03-08T21:11:03.096 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:03.116+0000 7f8d8c9e5740 1 -- 172.21.15.107:0/589314186 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8d7c0098a0 con 0x55f512624e80 2024-03-08T21:11:03.096 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:03.116+0000 7f8d6cc00640 1 -- 172.21.15.107:0/589314186 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f8d7401ec50 con 0x55f512624e80 2024-03-08T21:11:03.096 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:03.116+0000 7f8d6cc00640 1 -- 172.21.15.107:0/589314186 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8d7401ee50 con 0x55f512624e80 2024-03-08T21:11:03.096 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:03.116+0000 7f8d8c9e5740 10 client.25489 fetch_fsmap learned FSMap version 38 2024-03-08T21:11:03.096 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:03.116+0000 7f8d8c9e5740 1 -- 172.21.15.107:0/589314186 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x55f512369620 con 0x55f512624e80 2024-03-08T21:11:03.096 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:03.116+0000 7f8d6cc00640 1 -- 172.21.15.107:0/589314186 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f8d7c0098a0 con 0x55f512624e80 2024-03-08T21:11:03.096 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:03.116+0000 7f8d6cc00640 1 -- 172.21.15.107:0/589314186 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 38) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f8d7401a040 con 0x55f512624e80 2024-03-08T21:11:03.096 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:03.117+0000 7f8d8c9e5740 10 client.25489 fetch_fsmap finished waiting for FSMap version 38 2024-03-08T21:11:03.096 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:03.117+0000 7f8d8c9e5740 10 client.25489 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-08T21:11:03.096 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:03.117+0000 7f8d8c9e5740 20 client.25489 populate_metadata read hostname 'smithi107' 2024-03-08T21:11:03.096 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:03.117+0000 7f8d8c9e5740 1 --2- 172.21.15.107:0/589314186 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55f51263e460 0x55f512640850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:11:03.097 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:03.117+0000 7f8d8c9e5740 4 client.25489 mds_command: new command op to 34475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-08T21:11:03.097 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:03.117+0000 7f8d8c9e5740 1 -- 172.21.15.107:0/589314186 --> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f512595910 con 0x55f51263e460 2024-03-08T21:11:03.097 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:03.117+0000 7f8d82000640 1 --2- 172.21.15.107:0/589314186 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55f51263e460 0x55f512640850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:11:03.098 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:03.118+0000 7f8d82000640 1 --2- 172.21.15.107:0/589314186 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55f51263e460 0x55f512640850 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e2779a8caad0336 server_cookie=e3bfbe95f3f7951c in_seq=0 out_seq=0 2024-03-08T21:11:03.098 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:03.118+0000 7f8d6cc00640 10 client.25489 ms_handle_connect on v2:172.21.15.149:6838/55290596 2024-03-08T21:11:03.098 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:03.120+0000 7f8d6cc00640 1 -- 172.21.15.107:0/589314186 <== mds.0 v2:172.21.15.149:6838/55290596 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f512595910 con 0x55f51263e460 2024-03-08T21:11:03.098 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:03.120+0000 7f8d6cc00640 10 client.25489 handle_command_reply: tid=0 2024-03-08T21:11:03.098 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:03.120+0000 7f8d8c9e5740 1 client.25489 shutdown 2024-03-08T21:11:03.099 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:03.120+0000 7f8d8c9e5740 1 -- 172.21.15.107:0/589314186 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f512624e80 msgr2=0x55f51262df40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:11:03.099 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:03.120+0000 7f8d8c9e5740 1 --2- 172.21.15.107:0/589314186 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f512624e80 0x55f51262df40 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f8d74011c30 tx=0x7f8d740046c0 comp rx=0 tx=0).stop 2024-03-08T21:11:03.099 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:03.120+0000 7f8d8c9e5740 1 -- 172.21.15.107:0/589314186 shutdown_connections 2024-03-08T21:11:03.099 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:03.120+0000 7f8d8c9e5740 1 --2- 172.21.15.107:0/589314186 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f51262e570 0x55f51262b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:11:03.099 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:03.120+0000 7f8d8c9e5740 1 --2- 172.21.15.107:0/589314186 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55f51263e460 0x55f512640850 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:11:03.099 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:03.120+0000 7f8d8c9e5740 1 --2- 172.21.15.107:0/589314186 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f5126257a0 0x55f51262b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:11:03.099 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:03.120+0000 7f8d8c9e5740 1 --2- 172.21.15.107:0/589314186 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f512624e80 0x55f51262df40 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:11:03.099 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:03.120+0000 7f8d6cc00640 0 client.25489 ms_handle_reset on v2:172.21.15.149:6838/55290596 2024-03-08T21:11:03.099 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:03.120+0000 7f8d8c9e5740 1 -- 172.21.15.107:0/589314186 >> 172.21.15.107:0/589314186 conn(0x55f51261f7a0 msgr2=0x55f5126214f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:11:03.099 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:03.121+0000 7f8d8c9e5740 1 -- 172.21.15.107:0/589314186 shutdown_connections 2024-03-08T21:11:03.099 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:03.121+0000 7f8d8c9e5740 1 -- 172.21.15.107:0/589314186 wait complete. 2024-03-08T21:11:03.099 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:03.121+0000 7f8d8c9e5740 20 client.25489 trim_cache size 0 max 16384 2024-03-08T21:11:03.130 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:03.150+0000 7f8d8c9e5740 1 Processor -- start 2024-03-08T21:11:03.131 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:03.151+0000 7f8d8c9e5740 1 -- start start 2024-03-08T21:11:03.131 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:03.151+0000 7f8d8c9e5740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f512624e80 0x55f51262da00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:11:03.131 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:03.151+0000 7f8d8c9e5740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f5126257a0 0x55f51262df40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:11:03.131 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:03.151+0000 7f8d8c9e5740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f51262e570 0x55f512635500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:11:03.131 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:03.151+0000 7f8d8c9e5740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d841aed20 con 0x55f5126257a0 2024-03-08T21:11:03.131 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:03.151+0000 7f8d8c9e5740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d841b06a0 con 0x55f512624e80 2024-03-08T21:11:03.131 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:03.151+0000 7f8d8c9e5740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d841af380 con 0x55f51262e570 2024-03-08T21:11:03.132 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:03.151+0000 7f8d82000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f5126257a0 0x55f51262df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:11:03.132 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:03.151+0000 7f8d82000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f5126257a0 0x55f51262df40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:39446/0 (socket says 172.21.15.107:39446) 2024-03-08T21:11:03.132 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:03.151+0000 7f8d82000640 1 -- 172.21.15.107:0/1783658093 learned_addr learned my addr 172.21.15.107:0/1783658093 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:11:03.132 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:03.152+0000 7f8d83400640 1 --2- 172.21.15.107:0/1783658093 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f51262e570 0x55f512635500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:11:03.132 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:03.152+0000 7f8d82a00640 1 --2- 172.21.15.107:0/1783658093 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f512624e80 0x55f51262da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:11:03.132 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:03.152+0000 7f8d82000640 1 -- 172.21.15.107:0/1783658093 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f51262e570 msgr2=0x55f512635500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:11:03.132 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:03.152+0000 7f8d82000640 1 --2- 172.21.15.107:0/1783658093 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f51262e570 0x55f512635500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:11:03.133 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:03.152+0000 7f8d82000640 1 -- 172.21.15.107:0/1783658093 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f512624e80 msgr2=0x55f51262da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:11:03.133 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:03.152+0000 7f8d82000640 1 --2- 172.21.15.107:0/1783658093 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f512624e80 0x55f51262da00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:11:03.133 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:03.152+0000 7f8d82000640 1 -- 172.21.15.107:0/1783658093 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f512624640 con 0x55f5126257a0 2024-03-08T21:11:03.133 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:03.152+0000 7f8d82000640 1 --2- 172.21.15.107:0/1783658093 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f5126257a0 0x55f51262df40 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f8d70097560 tx=0x7f8d700bb040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=439f33328338f00f server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:11:03.133 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:03.153+0000 7f8d6f400640 1 -- 172.21.15.107:0/1783658093 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d7009b100 con 0x55f5126257a0 2024-03-08T21:11:03.133 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:03.153+0000 7f8d6f400640 1 -- 172.21.15.107:0/1783658093 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f8d7009f250 con 0x55f5126257a0 2024-03-08T21:11:03.133 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:03.153+0000 7f8d6f400640 1 -- 172.21.15.107:0/1783658093 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d700041d0 con 0x55f5126257a0 2024-03-08T21:11:03.136 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:03.154+0000 7f8d8c9e5740 1 -- 172.21.15.107:0/1783658093 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f5126257a0 msgr2=0x55f51262df40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:11:03.136 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:03.154+0000 7f8d8c9e5740 1 --2- 172.21.15.107:0/1783658093 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f5126257a0 0x55f51262df40 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f8d70097560 tx=0x7f8d700bb040 comp rx=0 tx=0).stop 2024-03-08T21:11:03.136 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:03.154+0000 7f8d8c9e5740 1 -- 172.21.15.107:0/1783658093 shutdown_connections 2024-03-08T21:11:03.136 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:03.154+0000 7f8d8c9e5740 1 --2- 172.21.15.107:0/1783658093 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f51262e570 0x55f512635500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:11:03.137 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:03.154+0000 7f8d8c9e5740 1 --2- 172.21.15.107:0/1783658093 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f5126257a0 0x55f51262df40 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:11:03.137 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:03.154+0000 7f8d8c9e5740 1 --2- 172.21.15.107:0/1783658093 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f512624e80 0x55f51262da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:11:03.137 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:03.154+0000 7f8d8c9e5740 1 -- 172.21.15.107:0/1783658093 >> 172.21.15.107:0/1783658093 conn(0x55f51261f7a0 msgr2=0x55f512625e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:11:03.137 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:03.154+0000 7f8d8c9e5740 1 -- 172.21.15.107:0/1783658093 shutdown_connections 2024-03-08T21:11:03.137 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:03.154+0000 7f8d8c9e5740 1 -- 172.21.15.107:0/1783658093 wait complete. 2024-03-08T21:11:03.137 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:03.154+0000 7f8d8c9e5740 1 -- start start 2024-03-08T21:11:03.137 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:03.155+0000 7f8d8c9e5740 1 Processor -- start 2024-03-08T21:11:03.137 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:03.155+0000 7f8d8c9e5740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f512624e80 0x55f512634730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:11:03.137 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:03.155+0000 7f8d8c9e5740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f5126257a0 0x55f512634c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:11:03.137 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:03.155+0000 7f8d8c9e5740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f51262e570 0x55f51268e860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:11:03.137 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:03.155+0000 7f8d8c9e5740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d841b0040 con 0x55f512624e80 2024-03-08T21:11:03.137 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:03.155+0000 7f8d8c9e5740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d841af9e0 con 0x55f51262e570 2024-03-08T21:11:03.138 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:03.155+0000 7f8d8c9e5740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d841acfe0 con 0x55f5126257a0 2024-03-08T21:11:03.138 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:03.155+0000 7f8d8c9e5740 10 client.0 _ll_register_callbacks cb 0x7f8d74005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T21:11:03.138 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:03.155+0000 7f8d82a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f512624e80 0x55f512634730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:11:03.138 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:03.156+0000 7f8d82000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f5126257a0 0x55f512634c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:11:03.138 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:03.156+0000 7f8d82000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f5126257a0 0x55f512634c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.204:3300/0 says I am v2:172.21.15.107:60318/0 (socket says 172.21.15.107:60318) 2024-03-08T21:11:03.138 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:03.156+0000 7f8d82000640 1 -- 172.21.15.107:0/3153727981 learned_addr learned my addr 172.21.15.107:0/3153727981 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:11:03.138 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:03.156+0000 7f8d83400640 1 --2- 172.21.15.107:0/3153727981 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f51262e570 0x55f51268e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:11:03.138 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:03.156+0000 7f8d82000640 1 -- 172.21.15.107:0/3153727981 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f51262e570 msgr2=0x55f51268e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:11:03.138 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:03.156+0000 7f8d82000640 1 --2- 172.21.15.107:0/3153727981 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f51262e570 0x55f51268e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:11:03.138 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:03.156+0000 7f8d82000640 1 -- 172.21.15.107:0/3153727981 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f512624e80 msgr2=0x55f512634730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:11:03.138 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:03.156+0000 7f8d82000640 1 --2- 172.21.15.107:0/3153727981 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f512624e80 0x55f512634730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:11:03.138 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:03.156+0000 7f8d82000640 1 -- 172.21.15.107:0/3153727981 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f51268ee50 con 0x55f5126257a0 2024-03-08T21:11:03.138 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:03.156+0000 7f8d82a00640 1 --2- 172.21.15.107:0/3153727981 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f512624e80 0x55f512634730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-08T21:11:03.139 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:03.156+0000 7f8d82000640 1 --2- 172.21.15.107:0/3153727981 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f5126257a0 0x55f512634c70 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f8d700bb010 tx=0x7f8d7000bf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=260428419dcdd1ba server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:11:03.139 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:03.156+0000 7f8d6cc00640 10 client.0 ms_handle_connect on v2:172.21.15.204:3300/0 2024-03-08T21:11:03.139 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:03.157+0000 7f8d6cc00640 1 -- 172.21.15.107:0/3153727981 <== mon.1 v2:172.21.15.204:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d7009b100 con 0x55f5126257a0 2024-03-08T21:11:03.139 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:03.157+0000 7f8d8c9e5740 1 -- 172.21.15.107:0/3153727981 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8d7009f250 con 0x55f5126257a0 2024-03-08T21:11:03.139 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:03.157+0000 7f8d6cc00640 1 -- 172.21.15.107:0/3153727981 <== mon.1 v2:172.21.15.204:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f8d700cc470 con 0x55f5126257a0 2024-03-08T21:11:03.139 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:03.157+0000 7f8d6cc00640 1 -- 172.21.15.107:0/3153727981 <== mon.1 v2:172.21.15.204:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8d700cc670 con 0x55f5126257a0 2024-03-08T21:11:03.139 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:03.157+0000 7f8d8c9e5740 10 client.25495 fetch_fsmap learned FSMap version 38 2024-03-08T21:11:03.139 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:03.157+0000 7f8d8c9e5740 1 -- 172.21.15.107:0/3153727981 --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x55f5126961e0 con 0x55f5126257a0 2024-03-08T21:11:03.139 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:03.157+0000 7f8d6cc00640 1 -- 172.21.15.107:0/3153727981 <== mon.1 v2:172.21.15.204:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f8d7009f250 con 0x55f5126257a0 2024-03-08T21:11:03.139 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:03.157+0000 7f8d6cc00640 1 -- 172.21.15.107:0/3153727981 <== mon.1 v2:172.21.15.204:3300/0 5 ==== fsmap(e 38) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f8d70096e50 con 0x55f5126257a0 2024-03-08T21:11:03.139 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:03.157+0000 7f8d8c9e5740 10 client.25495 fetch_fsmap finished waiting for FSMap version 38 2024-03-08T21:11:03.139 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:03.157+0000 7f8d8c9e5740 10 client.25495 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-08T21:11:03.139 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:03.157+0000 7f8d8c9e5740 20 client.25495 populate_metadata read hostname 'smithi107' 2024-03-08T21:11:03.139 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:03.157+0000 7f8d8c9e5740 1 --2- 172.21.15.107:0/3153727981 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55f5126968f0 0x55f512698d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:11:03.140 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:03.158+0000 7f8d8c9e5740 4 client.25495 mds_command: new command op to 34475 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-08T21:11:03.140 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:03.158+0000 7f8d8c9e5740 1 -- 172.21.15.107:0/3153727981 --> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55f51258fc60 con 0x55f5126968f0 2024-03-08T21:11:03.140 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:03.158+0000 7f8d82a00640 1 --2- 172.21.15.107:0/3153727981 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55f5126968f0 0x55f512698d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:11:03.140 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:03.159+0000 7f8d82a00640 1 --2- 172.21.15.107:0/3153727981 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55f5126968f0 0x55f512698d10 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c58ba75c9106115c server_cookie=1ed82a7288357f89 in_seq=0 out_seq=0 2024-03-08T21:11:03.140 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:03.159+0000 7f8d6cc00640 10 client.25495 ms_handle_connect on v2:172.21.15.149:6838/55290596 2024-03-08T21:11:03.962 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:11:03 smithi107 ceph-mon[31381]: from='mgr.15375 ' entity='mgr.x' 2024-03-08T21:11:03.962 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:11:03 smithi107 ceph-mon[31381]: from='mgr.15375 ' entity='mgr.x' 2024-03-08T21:11:03.962 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:11:03 smithi107 ceph-mon[31381]: from='mgr.15375 ' entity='mgr.x' 2024-03-08T21:11:04.010 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:11:03 smithi149 ceph-mon[35042]: from='mgr.15375 ' entity='mgr.x' 2024-03-08T21:11:04.010 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:11:03 smithi149 ceph-mon[35042]: from='mgr.15375 ' entity='mgr.x' 2024-03-08T21:11:04.010 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:11:03 smithi149 ceph-mon[35042]: from='mgr.15375 ' entity='mgr.x' 2024-03-08T21:11:04.123 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:11:03 smithi204 ceph-mon[34935]: from='mgr.15375 ' entity='mgr.x' 2024-03-08T21:11:04.123 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:11:03 smithi204 ceph-mon[34935]: from='mgr.15375 ' entity='mgr.x' 2024-03-08T21:11:04.123 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:11:03 smithi204 ceph-mon[34935]: from='mgr.15375 ' entity='mgr.x' 2024-03-08T21:11:04.184 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:04.205+0000 7f8d6cc00640 1 -- 172.21.15.107:0/3153727981 <== mds.0 v2:172.21.15.149:6838/55290596 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55f51258fc60 con 0x55f5126968f0 2024-03-08T21:11:04.184 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:04.205+0000 7f8d6cc00640 10 client.25495 handle_command_reply: tid=0 2024-03-08T21:11:04.184 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:04.205+0000 7f8d8c9e5740 1 client.25495 shutdown 2024-03-08T21:11:04.184 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:04.205+0000 7f8d8c9e5740 1 -- 172.21.15.107:0/3153727981 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f5126257a0 msgr2=0x55f512634c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:11:04.184 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:04.205+0000 7f8d8c9e5740 1 --2- 172.21.15.107:0/3153727981 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f5126257a0 0x55f512634c70 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f8d700bb010 tx=0x7f8d7000bf60 comp rx=0 tx=0).stop 2024-03-08T21:11:04.185 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:04.205+0000 7f8d8c9e5740 1 -- 172.21.15.107:0/3153727981 shutdown_connections 2024-03-08T21:11:04.185 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:04.205+0000 7f8d8c9e5740 1 --2- 172.21.15.107:0/3153727981 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55f5126968f0 0x55f512698d10 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:11:04.185 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:04.205+0000 7f8d8c9e5740 1 --2- 172.21.15.107:0/3153727981 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f51262e570 0x55f51268e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:11:04.185 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:04.205+0000 7f8d8c9e5740 1 --2- 172.21.15.107:0/3153727981 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f5126257a0 0x55f512634c70 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:11:04.185 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:04.205+0000 7f8d8c9e5740 1 --2- 172.21.15.107:0/3153727981 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f512624e80 0x55f512634730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:11:04.186 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:04.205+0000 7f8d8c9e5740 1 -- 172.21.15.107:0/3153727981 >> 172.21.15.107:0/3153727981 conn(0x55f51261f7a0 msgr2=0x55f5126266c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:11:04.186 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:04.205+0000 7f8d6cc00640 0 client.25495 ms_handle_reset on v2:172.21.15.149:6838/55290596 2024-03-08T21:11:04.186 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:04.206+0000 7f8d8c9e5740 1 -- 172.21.15.107:0/3153727981 shutdown_connections 2024-03-08T21:11:04.186 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:04.206+0000 7f8d8c9e5740 1 -- 172.21.15.107:0/3153727981 wait complete. 2024-03-08T21:11:04.186 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:04.206+0000 7f8d8c9e5740 20 client.25495 trim_cache size 0 max 16384 2024-03-08T21:11:04.187 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:{ 2024-03-08T21:11:04.187 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "message": "", 2024-03-08T21:11:04.187 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "return_code": 0 2024-03-08T21:11:04.187 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:} 2024-03-08T21:11:04.189 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:04.208+0000 7f8d83e00640 1 -- 172.21.15.107:0/3540534031 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f8d680a2960 msgr2=0x7f8d680a4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:11:04.189 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:04.208+0000 7f8d83e00640 1 --2- 172.21.15.107:0/3540534031 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f8d680a2960 0x7f8d680a4e20 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x55f512624930 tx=0x7f8d74005f90 comp rx=0 tx=0).stop 2024-03-08T21:11:04.189 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:04.208+0000 7f8d83e00640 1 -- 172.21.15.107:0/3540534031 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8d84106410 msgr2=0x7f8d8419fc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:11:04.189 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:04.208+0000 7f8d83e00640 1 --2- 172.21.15.107:0/3540534031 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8d84106410 0x7f8d8419fc50 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f8d70005fd0 tx=0x7f8d7000dde0 comp rx=0 tx=0).stop 2024-03-08T21:11:04.189 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:04.208+0000 7f8d83e00640 1 -- 172.21.15.107:0/3540534031 shutdown_connections 2024-03-08T21:11:04.190 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:04.208+0000 7f8d83e00640 1 --2- 172.21.15.107:0/3540534031 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f8d680a2960 0x7f8d680a4e20 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:11:04.190 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:04.208+0000 7f8d83e00640 1 --2- 172.21.15.107:0/3540534031 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f8d841998c0 0x7f8d84199d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:11:04.190 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:04.208+0000 7f8d83e00640 1 --2- 172.21.15.107:0/3540534031 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8d84106410 0x7f8d8419fc50 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:11:04.190 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:04.208+0000 7f8d83e00640 1 --2- 172.21.15.107:0/3540534031 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8d84101340 0x7f8d8419f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:11:04.190 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:04.208+0000 7f8d83e00640 1 -- 172.21.15.107:0/3540534031 >> 172.21.15.107:0/3540534031 conn(0x7f8d840fcc40 msgr2=0x7f8d8410dcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:11:04.190 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:04.208+0000 7f8d83e00640 1 -- 172.21.15.107:0/3540534031 shutdown_connections 2024-03-08T21:11:04.191 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:04.208+0000 7f8d83e00640 1 -- 172.21.15.107:0/3540534031 wait complete. 2024-03-08T21:11:04.963 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:11:04 smithi107 ceph-mon[31381]: pgmap v331: 97 pgs: 97 active+clean; 7.1 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.2 KiB/s wr, 10 op/s 2024-03-08T21:11:04.964 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:11:04 smithi107 ceph-mon[31381]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi149", "name": "osd_memory_target"}]: dispatch 2024-03-08T21:11:04.964 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:11:04 smithi107 ceph-mon[31381]: from='mgr.15375 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi149", "name": "osd_memory_target"}]: dispatch 2024-03-08T21:11:04.964 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:11:04 smithi107 ceph-mon[31381]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-08T21:11:04.964 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:11:04 smithi107 ceph-mon[31381]: from='mgr.15375 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-08T21:11:04.964 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:11:04 smithi107 ceph-mon[31381]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-08T21:11:04.964 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:11:04 smithi107 ceph-mon[31381]: from='mgr.15375 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-08T21:11:04.964 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:11:04 smithi107 ceph-mon[31381]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-08T21:11:04.964 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:11:04 smithi107 ceph-mon[31381]: from='mgr.15375 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-08T21:11:04.964 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:11:04 smithi107 ceph-mon[31381]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-08T21:11:04.964 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:11:04 smithi107 ceph-mon[31381]: from='mgr.15375 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-08T21:11:04.964 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:11:04 smithi107 ceph-mon[31381]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-08T21:11:04.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:11:04 smithi107 ceph-mon[31381]: from='mgr.15375 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-08T21:11:04.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:11:04 smithi107 ceph-mon[31381]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-08T21:11:04.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:11:04 smithi107 ceph-mon[31381]: from='mgr.15375 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-08T21:11:04.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:11:04 smithi107 ceph-mon[31381]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-08T21:11:04.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:11:04 smithi107 ceph-mon[31381]: from='mgr.15375 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-08T21:11:04.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:11:04 smithi107 ceph-mon[31381]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-08T21:11:04.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:11:04 smithi107 ceph-mon[31381]: from='mgr.15375 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-08T21:11:04.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:11:04 smithi107 ceph-mon[31381]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T21:11:04.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:11:04 smithi107 ceph-mon[31381]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T21:11:05.010 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:11:04 smithi149 ceph-mon[35042]: pgmap v331: 97 pgs: 97 active+clean; 7.1 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.2 KiB/s wr, 10 op/s 2024-03-08T21:11:05.011 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:11:04 smithi149 ceph-mon[35042]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi149", "name": "osd_memory_target"}]: dispatch 2024-03-08T21:11:05.011 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:11:04 smithi149 ceph-mon[35042]: from='mgr.15375 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi149", "name": "osd_memory_target"}]: dispatch 2024-03-08T21:11:05.011 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:11:04 smithi149 ceph-mon[35042]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-08T21:11:05.011 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:11:04 smithi149 ceph-mon[35042]: from='mgr.15375 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-08T21:11:05.011 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:11:04 smithi149 ceph-mon[35042]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-08T21:11:05.011 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:11:04 smithi149 ceph-mon[35042]: from='mgr.15375 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-08T21:11:05.011 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:11:04 smithi149 ceph-mon[35042]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-08T21:11:05.011 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:11:04 smithi149 ceph-mon[35042]: from='mgr.15375 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-08T21:11:05.011 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:11:04 smithi149 ceph-mon[35042]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-08T21:11:05.011 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:11:04 smithi149 ceph-mon[35042]: from='mgr.15375 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-08T21:11:05.011 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:11:04 smithi149 ceph-mon[35042]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-08T21:11:05.011 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:11:04 smithi149 ceph-mon[35042]: from='mgr.15375 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-08T21:11:05.012 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:11:04 smithi149 ceph-mon[35042]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-08T21:11:05.012 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:11:04 smithi149 ceph-mon[35042]: from='mgr.15375 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-08T21:11:05.012 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:11:04 smithi149 ceph-mon[35042]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-08T21:11:05.012 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:11:04 smithi149 ceph-mon[35042]: from='mgr.15375 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-08T21:11:05.012 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:11:04 smithi149 ceph-mon[35042]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-08T21:11:05.012 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:11:04 smithi149 ceph-mon[35042]: from='mgr.15375 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-08T21:11:05.012 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:11:04 smithi149 ceph-mon[35042]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T21:11:05.012 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:11:04 smithi149 ceph-mon[35042]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T21:11:05.124 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:11:04 smithi204 ceph-mon[34935]: pgmap v331: 97 pgs: 97 active+clean; 7.1 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.2 KiB/s wr, 10 op/s 2024-03-08T21:11:05.124 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:11:04 smithi204 ceph-mon[34935]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi149", "name": "osd_memory_target"}]: dispatch 2024-03-08T21:11:05.124 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:11:04 smithi204 ceph-mon[34935]: from='mgr.15375 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi149", "name": "osd_memory_target"}]: dispatch 2024-03-08T21:11:05.124 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:11:04 smithi204 ceph-mon[34935]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-08T21:11:05.124 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:11:04 smithi204 ceph-mon[34935]: from='mgr.15375 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-08T21:11:05.124 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:11:04 smithi204 ceph-mon[34935]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-08T21:11:05.124 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:11:04 smithi204 ceph-mon[34935]: from='mgr.15375 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-08T21:11:05.124 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:11:04 smithi204 ceph-mon[34935]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-08T21:11:05.124 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:11:04 smithi204 ceph-mon[34935]: from='mgr.15375 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-08T21:11:05.125 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:11:04 smithi204 ceph-mon[34935]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-08T21:11:05.125 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:11:04 smithi204 ceph-mon[34935]: from='mgr.15375 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-08T21:11:05.125 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:11:04 smithi204 ceph-mon[34935]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-08T21:11:05.125 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:11:04 smithi204 ceph-mon[34935]: from='mgr.15375 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-08T21:11:05.125 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:11:04 smithi204 ceph-mon[34935]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-08T21:11:05.125 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:11:04 smithi204 ceph-mon[34935]: from='mgr.15375 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-08T21:11:05.125 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:11:04 smithi204 ceph-mon[34935]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-08T21:11:05.125 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:11:04 smithi204 ceph-mon[34935]: from='mgr.15375 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-08T21:11:05.125 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:11:04 smithi204 ceph-mon[34935]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-08T21:11:05.125 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:11:04 smithi204 ceph-mon[34935]: from='mgr.15375 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-08T21:11:05.125 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:11:04 smithi204 ceph-mon[34935]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T21:11:05.125 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:11:04 smithi204 ceph-mon[34935]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T21:11:05.604 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:11:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[71551]: *** Caught signal (Segmentation fault) ** 2024-03-08T21:11:05.604 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:11:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[71551]: in thread 7f18c3c00700 thread_name:md_log_replay 2024-03-08T21:11:05.604 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:11:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[71551]: ceph version 19.0.0-1965-g7d63482a (7d63482aa89628d733e298984a762389b625c761) squid (dev) 2024-03-08T21:11:05.604 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:11:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[71551]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f18d81b0d20] 2024-03-08T21:11:05.605 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:11:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[71551]: 2: /usr/bin/ceph-mds(+0x503ee2) [0x55770b7d7ee2] 2024-03-08T21:11:05.605 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:11:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[71551]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x55770b7db773] 2024-03-08T21:11:05.605 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:11:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[71551]: 4: (MDLog::_replay_thread()+0x617) [0x55770b837587] 2024-03-08T21:11:05.605 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:11:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[71551]: 5: (MDLog::ReplayThread::entry()+0x11) [0x55770b840091] 2024-03-08T21:11:05.605 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:11:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[71551]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7f18d81a61ca] 2024-03-08T21:11:05.605 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:11:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[71551]: 7: clone() 2024-03-08T21:11:05.605 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:11:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[71551]: 2024-03-08T21:11:05.183+0000 7f18c3c00700 -1 *** Caught signal (Segmentation fault) ** 2024-03-08T21:11:05.605 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:11:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[71551]: in thread 7f18c3c00700 thread_name:md_log_replay 2024-03-08T21:11:05.605 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:11:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[71551]: 2024-03-08T21:11:05.605 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:11:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[71551]: ceph version 19.0.0-1965-g7d63482a (7d63482aa89628d733e298984a762389b625c761) squid (dev) 2024-03-08T21:11:05.605 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:11:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[71551]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f18d81b0d20] 2024-03-08T21:11:05.605 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:11:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[71551]: 2: /usr/bin/ceph-mds(+0x503ee2) [0x55770b7d7ee2] 2024-03-08T21:11:05.605 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:11:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[71551]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x55770b7db773] 2024-03-08T21:11:05.605 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:11:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[71551]: 4: (MDLog::_replay_thread()+0x617) [0x55770b837587] 2024-03-08T21:11:05.605 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:11:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[71551]: 5: (MDLog::ReplayThread::entry()+0x11) [0x55770b840091] 2024-03-08T21:11:05.606 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:11:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[71551]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7f18d81a61ca] 2024-03-08T21:11:05.606 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:11:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[71551]: 7: clone() 2024-03-08T21:11:05.606 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:11:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[71551]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-08T21:11:05.606 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:11:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[71551]: 2024-03-08T21:11:05.606 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:11:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[71551]: 0> 2024-03-08T21:11:05.183+0000 7f18c3c00700 -1 *** Caught signal (Segmentation fault) ** 2024-03-08T21:11:05.606 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:11:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[71551]: in thread 7f18c3c00700 thread_name:md_log_replay 2024-03-08T21:11:05.606 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:11:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[71551]: 2024-03-08T21:11:05.606 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:11:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[71551]: ceph version 19.0.0-1965-g7d63482a (7d63482aa89628d733e298984a762389b625c761) squid (dev) 2024-03-08T21:11:05.606 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:11:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[71551]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f18d81b0d20] 2024-03-08T21:11:05.606 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:11:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[71551]: 2: /usr/bin/ceph-mds(+0x503ee2) [0x55770b7d7ee2] 2024-03-08T21:11:05.607 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:11:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[71551]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x55770b7db773] 2024-03-08T21:11:05.607 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:11:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[71551]: 4: (MDLog::_replay_thread()+0x617) [0x55770b837587] 2024-03-08T21:11:05.607 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:11:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[71551]: 5: (MDLog::ReplayThread::entry()+0x11) [0x55770b840091] 2024-03-08T21:11:05.607 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:11:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[71551]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7f18d81a61ca] 2024-03-08T21:11:05.607 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:11:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[71551]: 7: clone() 2024-03-08T21:11:05.607 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:11:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[71551]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-08T21:11:05.607 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:11:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[71551]: 2024-03-08T21:11:05.607 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:11:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[71551]: 0> 2024-03-08T21:11:05.183+0000 7f18c3c00700 -1 *** Caught signal (Segmentation fault) ** 2024-03-08T21:11:05.607 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:11:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[71551]: in thread 7f18c3c00700 thread_name:md_log_replay 2024-03-08T21:11:05.607 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:11:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[71551]: 2024-03-08T21:11:05.607 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:11:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[71551]: ceph version 19.0.0-1965-g7d63482a (7d63482aa89628d733e298984a762389b625c761) squid (dev) 2024-03-08T21:11:05.607 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:11:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[71551]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f18d81b0d20] 2024-03-08T21:11:05.607 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:11:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[71551]: 2: /usr/bin/ceph-mds(+0x503ee2) [0x55770b7d7ee2] 2024-03-08T21:11:05.608 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:11:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[71551]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x55770b7db773] 2024-03-08T21:11:05.608 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:11:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[71551]: 4: (MDLog::_replay_thread()+0x617) [0x55770b837587] 2024-03-08T21:11:05.608 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:11:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[71551]: 5: (MDLog::ReplayThread::entry()+0x11) [0x55770b840091] 2024-03-08T21:11:05.608 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:11:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[71551]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7f18d81a61ca] 2024-03-08T21:11:05.608 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:11:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[71551]: 7: clone() 2024-03-08T21:11:05.608 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:11:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[71551]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-08T21:11:05.608 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:11:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c[71551]: 2024-03-08T21:11:05.608 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:11:05 smithi204 podman[72297]: 2024-03-08 21:11:05.296423721 +0000 UTC m=+0.041880695 container died e8449560c340b0836ff42cc1649f3bb380947f5beb57f560f203f985e87ef28f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=rishabh-2024mar8-7d63482, io.buildah.version=1.33.5) 2024-03-08T21:11:05.873 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:11:05 smithi204 ceph-mon[34935]: Adjusting osd_memory_target on smithi107 to 176.7M 2024-03-08T21:11:05.873 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:11:05 smithi204 ceph-mon[34935]: Unable to set osd_memory_target on smithi107 to 185328640: error parsing value: Value '185328640' is below minimum 939524096 2024-03-08T21:11:05.873 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:11:05 smithi204 ceph-mon[34935]: Adjusting osd_memory_target on smithi204 to 176.7M 2024-03-08T21:11:05.874 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:11:05 smithi204 ceph-mon[34935]: Unable to set osd_memory_target on smithi204 to 185360896: error parsing value: Value '185360896' is below minimum 939524096 2024-03-08T21:11:05.874 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:11:05 smithi204 ceph-mon[34935]: osdmap e103: 12 total, 12 up, 12 in 2024-03-08T21:11:05.874 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:11:05 smithi204 ceph-mon[34935]: from='mgr.15375 ' entity='mgr.x' 2024-03-08T21:11:05.962 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:11:05 smithi107 ceph-mon[31381]: Adjusting osd_memory_target on smithi107 to 176.7M 2024-03-08T21:11:05.962 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:11:05 smithi107 ceph-mon[31381]: Unable to set osd_memory_target on smithi107 to 185328640: error parsing value: Value '185328640' is below minimum 939524096 2024-03-08T21:11:05.962 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:11:05 smithi107 ceph-mon[31381]: Adjusting osd_memory_target on smithi204 to 176.7M 2024-03-08T21:11:05.963 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:11:05 smithi107 ceph-mon[31381]: Unable to set osd_memory_target on smithi204 to 185360896: error parsing value: Value '185360896' is below minimum 939524096 2024-03-08T21:11:05.963 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:11:05 smithi107 ceph-mon[31381]: osdmap e103: 12 total, 12 up, 12 in 2024-03-08T21:11:05.963 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:11:05 smithi107 ceph-mon[31381]: from='mgr.15375 ' entity='mgr.x' 2024-03-08T21:11:06.010 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:11:05 smithi149 ceph-mon[35042]: Adjusting osd_memory_target on smithi107 to 176.7M 2024-03-08T21:11:06.010 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:11:05 smithi149 ceph-mon[35042]: Unable to set osd_memory_target on smithi107 to 185328640: error parsing value: Value '185328640' is below minimum 939524096 2024-03-08T21:11:06.010 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:11:05 smithi149 ceph-mon[35042]: Adjusting osd_memory_target on smithi204 to 176.7M 2024-03-08T21:11:06.010 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:11:05 smithi149 ceph-mon[35042]: Unable to set osd_memory_target on smithi204 to 185360896: error parsing value: Value '185360896' is below minimum 939524096 2024-03-08T21:11:06.010 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:11:05 smithi149 ceph-mon[35042]: osdmap e103: 12 total, 12 up, 12 in 2024-03-08T21:11:06.010 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:11:05 smithi149 ceph-mon[35042]: from='mgr.15375 ' entity='mgr.x' 2024-03-08T21:11:06.373 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:11:06 smithi204 podman[72297]: 2024-03-08 21:11:06.092643379 +0000 UTC m=+0.838100340 container remove e8449560c340b0836ff42cc1649f3bb380947f5beb57f560f203f985e87ef28f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-c, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=rishabh-2024mar8-7d63482, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=) 2024-03-08T21:11:06.373 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:11:06 smithi204 systemd[1]: ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715@mds.c.service: Main process exited, code=exited, status=139/n/a 2024-03-08T21:11:06.668 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:11:06 smithi204 systemd[1]: ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715@mds.c.service: Failed with result 'exit-code'. 2024-03-08T21:11:06.668 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:11:06 smithi204 systemd[1]: ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715@mds.c.service: Consumed 1.823s CPU time. 2024-03-08T21:11:06.962 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:11:06 smithi107 ceph-mon[31381]: pgmap v333: 97 pgs: 97 active+clean; 7.4 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 92 KiB/s wr, 33 op/s 2024-03-08T21:11:07.010 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:11:06 smithi149 ceph-mon[35042]: pgmap v333: 97 pgs: 97 active+clean; 7.4 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 92 KiB/s wr, 33 op/s 2024-03-08T21:11:07.123 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:11:06 smithi204 ceph-mon[34935]: pgmap v333: 97 pgs: 97 active+clean; 7.4 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 92 KiB/s wr, 33 op/s 2024-03-08T21:11:08.962 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:11:08 smithi107 ceph-mon[31381]: pgmap v334: 97 pgs: 97 active+clean; 7.4 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 92 KiB/s wr, 33 op/s 2024-03-08T21:11:09.010 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:11:08 smithi149 ceph-mon[35042]: pgmap v334: 97 pgs: 97 active+clean; 7.4 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 92 KiB/s wr, 33 op/s 2024-03-08T21:11:09.081 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:11:08 smithi204 ceph-mon[34935]: pgmap v334: 97 pgs: 97 active+clean; 7.4 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 92 KiB/s wr, 33 op/s 2024-03-08T21:11:10.962 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:11:10 smithi107 ceph-mon[31381]: pgmap v335: 97 pgs: 97 active+clean; 7.4 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 93 KiB/s wr, 33 op/s 2024-03-08T21:11:11.010 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:11:10 smithi149 ceph-mon[35042]: pgmap v335: 97 pgs: 97 active+clean; 7.4 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 93 KiB/s wr, 33 op/s 2024-03-08T21:11:11.123 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:11:10 smithi204 ceph-mon[34935]: pgmap v335: 97 pgs: 97 active+clean; 7.4 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 93 KiB/s wr, 33 op/s 2024-03-08T21:11:12.963 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:11:12 smithi107 ceph-mon[31381]: pgmap v336: 97 pgs: 97 active+clean; 7.4 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 93 KiB/s wr, 33 op/s 2024-03-08T21:11:13.010 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:11:12 smithi149 ceph-mon[35042]: pgmap v336: 97 pgs: 97 active+clean; 7.4 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 93 KiB/s wr, 33 op/s 2024-03-08T21:11:13.123 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:11:12 smithi204 ceph-mon[34935]: pgmap v336: 97 pgs: 97 active+clean; 7.4 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 93 KiB/s wr, 33 op/s 2024-03-08T21:11:14.962 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:11:14 smithi107 ceph-mon[31381]: pgmap v337: 97 pgs: 97 active+clean; 7.4 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 93 KiB/s wr, 33 op/s 2024-03-08T21:11:14.962 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:11:14 smithi107 ceph-mon[31381]: osdmap e104: 12 total, 12 up, 12 in 2024-03-08T21:11:15.010 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:11:14 smithi149 ceph-mon[35042]: pgmap v337: 97 pgs: 97 active+clean; 7.4 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 93 KiB/s wr, 33 op/s 2024-03-08T21:11:15.010 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:11:14 smithi149 ceph-mon[35042]: osdmap e104: 12 total, 12 up, 12 in 2024-03-08T21:11:15.123 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:11:14 smithi204 ceph-mon[34935]: pgmap v337: 97 pgs: 97 active+clean; 7.4 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 93 KiB/s wr, 33 op/s 2024-03-08T21:11:15.123 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:11:14 smithi204 ceph-mon[34935]: osdmap e104: 12 total, 12 up, 12 in 2024-03-08T21:11:16.962 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:11:16 smithi107 ceph-mon[31381]: pgmap v339: 97 pgs: 97 active+clean; 7.4 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 409 B/s wr, 8 op/s 2024-03-08T21:11:17.009 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:11:16 smithi149 ceph-mon[35042]: pgmap v339: 97 pgs: 97 active+clean; 7.4 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 409 B/s wr, 8 op/s 2024-03-08T21:11:17.123 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:11:16 smithi204 ceph-mon[34935]: pgmap v339: 97 pgs: 97 active+clean; 7.4 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 409 B/s wr, 8 op/s 2024-03-08T21:11:17.123 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:11:16 smithi204 systemd[1]: ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715@mds.c.service: Scheduled restart job, restart counter is at 6. 2024-03-08T21:11:17.123 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:11:16 smithi204 systemd[1]: Stopped Ceph mds.c for dc20f0f8-dd8c-11ee-95c6-87774f69a715. 2024-03-08T21:11:17.123 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:11:16 smithi204 systemd[1]: ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715@mds.c.service: Consumed 1.823s CPU time. 2024-03-08T21:11:17.124 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:11:16 smithi204 systemd[1]: ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715@mds.c.service: Start request repeated too quickly. 2024-03-08T21:11:17.124 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:11:16 smithi204 systemd[1]: ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715@mds.c.service: Failed with result 'exit-code'. 2024-03-08T21:11:17.124 INFO:journalctl@ceph.mds.c.smithi204.stdout:Mar 08 21:11:16 smithi204 systemd[1]: Failed to start Ceph mds.c for dc20f0f8-dd8c-11ee-95c6-87774f69a715. 2024-03-08T21:11:17.343 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.364+0000 7f6f31000640 1 -- 172.21.15.107:0/2775425423 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6f2c108620 msgr2=0x7f6f2c1101f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:11:17.343 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.364+0000 7f6f31000640 1 --2- 172.21.15.107:0/2775425423 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6f2c108620 0x7f6f2c1101f0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f6f24009ca0 tx=0x7f6f2402c100 comp rx=0 tx=0).stop 2024-03-08T21:11:17.343 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.364+0000 7f6f31000640 1 -- 172.21.15.107:0/2775425423 shutdown_connections 2024-03-08T21:11:17.343 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.365+0000 7f6f31000640 1 --2- 172.21.15.107:0/2775425423 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6f2c108620 0x7f6f2c1101f0 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:11:17.343 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.365+0000 7f6f31000640 1 --2- 172.21.15.107:0/2775425423 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6f2c103f00 0x7f6f2c107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:11:17.344 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.365+0000 7f6f31000640 1 --2- 172.21.15.107:0/2775425423 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6f2c103550 0x7f6f2c103930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:11:17.344 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.365+0000 7f6f31000640 1 -- 172.21.15.107:0/2775425423 >> 172.21.15.107:0/2775425423 conn(0x7f6f2c0fedd0 msgr2=0x7f6f2c1011f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:11:17.344 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.365+0000 7f6f31000640 1 -- 172.21.15.107:0/2775425423 shutdown_connections 2024-03-08T21:11:17.344 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.365+0000 7f6f31000640 1 -- 172.21.15.107:0/2775425423 wait complete. 2024-03-08T21:11:17.344 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.366+0000 7f6f31000640 1 Processor -- start 2024-03-08T21:11:17.344 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.366+0000 7f6f31000640 1 -- start start 2024-03-08T21:11:17.346 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.366+0000 7f6f31000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6f2c103550 0x7f6f2c19f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:11:17.346 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.366+0000 7f6f31000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6f2c108620 0x7f6f2c19fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:11:17.346 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.366+0000 7f6f31000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6f2c199990 0x7f6f2c199e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:11:17.346 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.366+0000 7f6f31000640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f2c113570 con 0x7f6f2c108620 2024-03-08T21:11:17.346 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.366+0000 7f6f31000640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f2c1133f0 con 0x7f6f2c199990 2024-03-08T21:11:17.346 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.366+0000 7f6f31000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f2c1136f0 con 0x7f6f2c103550 2024-03-08T21:11:17.346 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.367+0000 7f6f23e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6f2c108620 0x7f6f2c19fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:11:17.346 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.367+0000 7f6f23e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6f2c108620 0x7f6f2c19fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:36214/0 (socket says 172.21.15.107:36214) 2024-03-08T21:11:17.347 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.367+0000 7f6f23e00640 1 -- 172.21.15.107:0/3093599342 learned_addr learned my addr 172.21.15.107:0/3093599342 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:11:17.347 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.367+0000 7f6f2b400640 1 --2- 172.21.15.107:0/3093599342 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6f2c103550 0x7f6f2c19f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:11:17.347 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.367+0000 7f6f2be00640 1 --2- 172.21.15.107:0/3093599342 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6f2c199990 0x7f6f2c199e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:11:17.347 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.367+0000 7f6f23e00640 1 -- 172.21.15.107:0/3093599342 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6f2c103550 msgr2=0x7f6f2c19f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:11:17.347 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.367+0000 7f6f23e00640 1 --2- 172.21.15.107:0/3093599342 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6f2c103550 0x7f6f2c19f7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:11:17.347 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.367+0000 7f6f23e00640 1 -- 172.21.15.107:0/3093599342 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6f2c199990 msgr2=0x7f6f2c199e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:11:17.347 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.367+0000 7f6f23e00640 1 --2- 172.21.15.107:0/3093599342 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6f2c199990 0x7f6f2c199e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:11:17.347 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.367+0000 7f6f23e00640 1 -- 172.21.15.107:0/3093599342 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f2c19a4b0 con 0x7f6f2c108620 2024-03-08T21:11:17.347 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.367+0000 7f6f2b400640 1 --2- 172.21.15.107:0/3093599342 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6f2c103550 0x7f6f2c19f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T21:11:17.347 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.368+0000 7f6f23e00640 1 --2- 172.21.15.107:0/3093599342 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6f2c108620 0x7f6f2c19fd20 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f6f1400d770 tx=0x7f6f1400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:11:17.347 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.368+0000 7f6f28c00640 1 -- 172.21.15.107:0/3093599342 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f1401c070 con 0x7f6f2c108620 2024-03-08T21:11:17.347 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.368+0000 7f6f28c00640 1 -- 172.21.15.107:0/3093599342 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f6f14009cd0 con 0x7f6f2c108620 2024-03-08T21:11:17.347 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.368+0000 7f6f28c00640 1 -- 172.21.15.107:0/3093599342 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f14020c20 con 0x7f6f2c108620 2024-03-08T21:11:17.347 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.368+0000 7f6f31000640 1 -- 172.21.15.107:0/3093599342 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f2c19a7a0 con 0x7f6f2c108620 2024-03-08T21:11:17.348 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.368+0000 7f6f31000640 1 -- 172.21.15.107:0/3093599342 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f2c075a80 con 0x7f6f2c108620 2024-03-08T21:11:17.350 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.372+0000 7f6f28c00640 1 -- 172.21.15.107:0/3093599342 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) v1 ==== 137422+0+0 (secure 0 0 0) 0x7f6f14009820 con 0x7f6f2c108620 2024-03-08T21:11:17.350 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.372+0000 7f6f35200740 1 Processor -- start 2024-03-08T21:11:17.350 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.373+0000 7f6f35200740 1 -- start start 2024-03-08T21:11:17.352 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.373+0000 7f6f35200740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5639a65ede80 0x5639a65ee260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:11:17.352 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.373+0000 7f6f35200740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5639a65ee7a0 0x5639a65f7030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:11:17.352 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.373+0000 7f6f35200740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5639a65f7570 0x5639a65f9960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:11:17.352 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.373+0000 7f6f35200740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639a65513d0 con 0x5639a65ee7a0 2024-03-08T21:11:17.352 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.373+0000 7f6f35200740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639a64fa6b0 con 0x5639a65ede80 2024-03-08T21:11:17.352 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.373+0000 7f6f35200740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639a654d470 con 0x5639a65f7570 2024-03-08T21:11:17.352 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.373+0000 7f6f2b400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5639a65ee7a0 0x5639a65f7030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:11:17.352 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.373+0000 7f6f2b400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5639a65ee7a0 0x5639a65f7030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:36224/0 (socket says 172.21.15.107:36224) 2024-03-08T21:11:17.353 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.373+0000 7f6f2b400640 1 -- 172.21.15.107:0/3498127384 learned_addr learned my addr 172.21.15.107:0/3498127384 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:11:17.353 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.374+0000 7f6f2be00640 1 --2- 172.21.15.107:0/3498127384 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5639a65ede80 0x5639a65ee260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:11:17.353 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.374+0000 7f6f23e00640 1 --2- 172.21.15.107:0/3498127384 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5639a65f7570 0x5639a65f9960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:11:17.353 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.374+0000 7f6f2b400640 1 -- 172.21.15.107:0/3498127384 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5639a65f7570 msgr2=0x5639a65f9960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:11:17.353 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.374+0000 7f6f2b400640 1 --2- 172.21.15.107:0/3498127384 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5639a65f7570 0x5639a65f9960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:11:17.353 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.374+0000 7f6f2b400640 1 -- 172.21.15.107:0/3498127384 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5639a65ede80 msgr2=0x5639a65ee260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:11:17.353 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.374+0000 7f6f2b400640 1 --2- 172.21.15.107:0/3498127384 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5639a65ede80 0x5639a65ee260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:11:17.353 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.374+0000 7f6f2b400640 1 -- 172.21.15.107:0/3498127384 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5639a6520b50 con 0x5639a65ee7a0 2024-03-08T21:11:17.353 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.374+0000 7f6f28c00640 1 --2- 172.21.15.107:0/3093599342 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f6f100a2960 0x7f6f100a4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:11:17.353 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.375+0000 7f6f2b400640 1 --2- 172.21.15.107:0/3498127384 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5639a65ee7a0 0x5639a65f7030 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f6f1c00bcc0 tx=0x7f6f1c00f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2905265f7f053 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:11:17.353 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.375+0000 7f6f28c00640 1 -- 172.21.15.107:0/3093599342 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f6f140d6dd0 con 0x7f6f2c108620 2024-03-08T21:11:17.353 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.375+0000 7f6f2be00640 1 --2- 172.21.15.107:0/3093599342 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f6f100a2960 0x7f6f100a4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:11:17.357 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.375+0000 7f6f20c00640 1 -- 172.21.15.107:0/3498127384 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f1c005020 con 0x5639a65ee7a0 2024-03-08T21:11:17.357 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.375+0000 7f6f20c00640 1 -- 172.21.15.107:0/3498127384 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f6f1c0047a0 con 0x5639a65ee7a0 2024-03-08T21:11:17.357 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.375+0000 7f6f20c00640 1 -- 172.21.15.107:0/3498127384 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f1c0116a0 con 0x5639a65ee7a0 2024-03-08T21:11:17.357 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.376+0000 7f6f35200740 1 -- 172.21.15.107:0/3498127384 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5639a65ee7a0 msgr2=0x5639a65f7030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:11:17.357 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.376+0000 7f6f2be00640 1 --2- 172.21.15.107:0/3093599342 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f6f100a2960 0x7f6f100a4e20 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f6f24005e30 tx=0x7f6f2402e040 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:11:17.357 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.376+0000 7f6f35200740 1 --2- 172.21.15.107:0/3498127384 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5639a65ee7a0 0x5639a65f7030 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f6f1c00bcc0 tx=0x7f6f1c00f730 comp rx=0 tx=0).stop 2024-03-08T21:11:17.357 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.376+0000 7f6f35200740 1 -- 172.21.15.107:0/3498127384 shutdown_connections 2024-03-08T21:11:17.357 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.376+0000 7f6f35200740 1 --2- 172.21.15.107:0/3498127384 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5639a65f7570 0x5639a65f9960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:11:17.358 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.376+0000 7f6f35200740 1 --2- 172.21.15.107:0/3498127384 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5639a65ee7a0 0x5639a65f7030 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:11:17.358 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.376+0000 7f6f35200740 1 --2- 172.21.15.107:0/3498127384 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5639a65ede80 0x5639a65ee260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:11:17.358 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.376+0000 7f6f35200740 1 -- 172.21.15.107:0/3498127384 >> 172.21.15.107:0/3498127384 conn(0x5639a65e87a0 msgr2=0x5639a65e8b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:11:17.358 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.376+0000 7f6f35200740 1 -- 172.21.15.107:0/3498127384 shutdown_connections 2024-03-08T21:11:17.358 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.376+0000 7f6f35200740 1 -- 172.21.15.107:0/3498127384 wait complete. 2024-03-08T21:11:17.358 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.377+0000 7f6f35200740 1 -- start start 2024-03-08T21:11:17.358 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.377+0000 7f6f35200740 1 Processor -- start 2024-03-08T21:11:17.358 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.377+0000 7f6f35200740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5639a65ede80 0x5639a65f6f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:11:17.358 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.377+0000 7f6f35200740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5639a65ee7a0 0x5639a65f42d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:11:17.358 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.377+0000 7f6f35200740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5639a65f7570 0x5639a65f48e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:11:17.358 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.377+0000 7f6f35200740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639a65fa1a0 con 0x5639a65ee7a0 2024-03-08T21:11:17.358 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.377+0000 7f6f35200740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639a650d9f0 con 0x5639a65f7570 2024-03-08T21:11:17.359 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.377+0000 7f6f35200740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639a65fa320 con 0x5639a65ede80 2024-03-08T21:11:17.359 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.377+0000 7f6f35200740 10 client.0 _ll_register_callbacks cb 0x5639a6265a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T21:11:17.359 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.378+0000 7f6f23e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5639a65ee7a0 0x5639a65f42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:11:17.359 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.378+0000 7f6f23e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5639a65ee7a0 0x5639a65f42d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:36240/0 (socket says 172.21.15.107:36240) 2024-03-08T21:11:17.359 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.378+0000 7f6f23e00640 1 -- 172.21.15.107:0/3719814402 learned_addr learned my addr 172.21.15.107:0/3719814402 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:11:17.359 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.378+0000 7f6f2b400640 1 --2- 172.21.15.107:0/3719814402 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5639a65ede80 0x5639a65f6f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:11:17.359 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.378+0000 7f6f2be00640 1 --2- 172.21.15.107:0/3719814402 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5639a65f7570 0x5639a65f48e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:11:17.359 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.378+0000 7f6f23e00640 1 -- 172.21.15.107:0/3719814402 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5639a65ede80 msgr2=0x5639a65f6f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:11:17.359 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.378+0000 7f6f23e00640 1 --2- 172.21.15.107:0/3719814402 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5639a65ede80 0x5639a65f6f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:11:17.359 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.378+0000 7f6f23e00640 1 -- 172.21.15.107:0/3719814402 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5639a65f7570 msgr2=0x5639a65f48e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:11:17.359 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.378+0000 7f6f23e00640 1 --2- 172.21.15.107:0/3719814402 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5639a65f7570 0x5639a65f48e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:11:17.359 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.378+0000 7f6f23e00640 1 -- 172.21.15.107:0/3719814402 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5639a6266210 con 0x5639a65ee7a0 2024-03-08T21:11:17.359 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.378+0000 7f6f2be00640 1 --2- 172.21.15.107:0/3719814402 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5639a65f7570 0x5639a65f48e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T21:11:17.359 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.378+0000 7f6f23e00640 1 --2- 172.21.15.107:0/3719814402 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5639a65ee7a0 0x5639a65f42d0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f6f1409f960 tx=0x7f6f1409f1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e62629128e10150f server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:11:17.360 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.378+0000 7f6f19600640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-03-08T21:11:17.360 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.379+0000 7f6f19600640 1 -- 172.21.15.107:0/3719814402 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f140a3040 con 0x5639a65ee7a0 2024-03-08T21:11:17.360 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.379+0000 7f6f19600640 1 -- 172.21.15.107:0/3719814402 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f6f1409fb70 con 0x5639a65ee7a0 2024-03-08T21:11:17.360 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.379+0000 7f6f35200740 1 -- 172.21.15.107:0/3719814402 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6f1c0047a0 con 0x5639a65ee7a0 2024-03-08T21:11:17.360 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.379+0000 7f6f19600640 1 -- 172.21.15.107:0/3719814402 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6f140a2e60 con 0x5639a65ee7a0 2024-03-08T21:11:17.360 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.379+0000 7f6f35200740 10 client.16353 fetch_fsmap learned FSMap version 38 2024-03-08T21:11:17.360 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.379+0000 7f6f35200740 1 -- 172.21.15.107:0/3719814402 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x5639a6332620 con 0x5639a65ee7a0 2024-03-08T21:11:17.360 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.379+0000 7f6f19600640 1 -- 172.21.15.107:0/3719814402 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f6f1c0047a0 con 0x5639a65ee7a0 2024-03-08T21:11:17.360 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.379+0000 7f6f19600640 1 -- 172.21.15.107:0/3719814402 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 38) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f6f140a7160 con 0x5639a65ee7a0 2024-03-08T21:11:17.360 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.379+0000 7f6f35200740 10 client.16353 fetch_fsmap finished waiting for FSMap version 38 2024-03-08T21:11:17.360 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.379+0000 7f6f35200740 10 client.16353 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-08T21:11:17.360 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.379+0000 7f6f35200740 20 client.16353 populate_metadata read hostname 'smithi107' 2024-03-08T21:11:17.360 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.379+0000 7f6f35200740 1 --2- 172.21.15.107:0/3719814402 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x5639a6607430 0x5639a6609840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:11:17.360 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.380+0000 7f6f35200740 4 client.16353 mds_command: new command op to 34475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-08T21:11:17.361 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.380+0000 7f6f35200740 1 -- 172.21.15.107:0/3719814402 --> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5639a655e910 con 0x5639a6607430 2024-03-08T21:11:17.362 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.380+0000 7f6f2b400640 1 --2- 172.21.15.107:0/3719814402 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x5639a6607430 0x5639a6609840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:11:17.362 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.381+0000 7f6f19600640 10 client.16353 ms_handle_connect on v2:172.21.15.149:6838/55290596 2024-03-08T21:11:17.362 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.381+0000 7f6f2b400640 1 --2- 172.21.15.107:0/3719814402 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x5639a6607430 0x5639a6609840 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a2448c38fb94671d server_cookie=cea870a66e592819 in_seq=0 out_seq=0 2024-03-08T21:11:17.362 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.383+0000 7f6f19600640 1 -- 172.21.15.107:0/3719814402 <== mds.0 v2:172.21.15.149:6838/55290596 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5639a655e910 con 0x5639a6607430 2024-03-08T21:11:17.362 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.383+0000 7f6f19600640 10 client.16353 handle_command_reply: tid=0 2024-03-08T21:11:17.362 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.383+0000 7f6f35200740 1 client.16353 shutdown 2024-03-08T21:11:17.362 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.383+0000 7f6f35200740 1 -- 172.21.15.107:0/3719814402 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5639a65ee7a0 msgr2=0x5639a65f42d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:11:17.362 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.383+0000 7f6f35200740 1 --2- 172.21.15.107:0/3719814402 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5639a65ee7a0 0x5639a65f42d0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f6f1409f960 tx=0x7f6f1409f1a0 comp rx=0 tx=0).stop 2024-03-08T21:11:17.362 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.383+0000 7f6f35200740 1 -- 172.21.15.107:0/3719814402 shutdown_connections 2024-03-08T21:11:17.363 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.383+0000 7f6f35200740 1 --2- 172.21.15.107:0/3719814402 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x5639a6607430 0x5639a6609840 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:11:17.363 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.383+0000 7f6f35200740 1 --2- 172.21.15.107:0/3719814402 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5639a65f7570 0x5639a65f48e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:11:17.363 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.383+0000 7f6f35200740 1 --2- 172.21.15.107:0/3719814402 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5639a65ee7a0 0x5639a65f42d0 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:11:17.363 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.383+0000 7f6f35200740 1 --2- 172.21.15.107:0/3719814402 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5639a65ede80 0x5639a65f6f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:11:17.363 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.383+0000 7f6f19600640 0 client.16353 ms_handle_reset on v2:172.21.15.149:6838/55290596 2024-03-08T21:11:17.363 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.383+0000 7f6f35200740 1 -- 172.21.15.107:0/3719814402 >> 172.21.15.107:0/3719814402 conn(0x5639a65e87a0 msgr2=0x5639a65ea4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:11:17.363 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.383+0000 7f6f35200740 1 -- 172.21.15.107:0/3719814402 shutdown_connections 2024-03-08T21:11:17.363 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.383+0000 7f6f35200740 1 -- 172.21.15.107:0/3719814402 wait complete. 2024-03-08T21:11:17.363 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.384+0000 7f6f35200740 20 client.16353 trim_cache size 0 max 16384 2024-03-08T21:11:17.382 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.403+0000 7f6f35200740 1 Processor -- start 2024-03-08T21:11:17.382 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.403+0000 7f6f35200740 1 -- start start 2024-03-08T21:11:17.382 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.403+0000 7f6f35200740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5639a65ede80 0x5639a65e7680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:11:17.382 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.403+0000 7f6f35200740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5639a65ee7a0 0x5639a65f6a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:11:17.382 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.403+0000 7f6f35200740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5639a65f7570 0x5639a65f6f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:11:17.382 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.403+0000 7f6f35200740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f2c1aa9f0 con 0x5639a65f7570 2024-03-08T21:11:17.382 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.403+0000 7f6f35200740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f2c1ac370 con 0x5639a65ede80 2024-03-08T21:11:17.382 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.403+0000 7f6f35200740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f2c1ab050 con 0x5639a65ee7a0 2024-03-08T21:11:17.382 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.404+0000 7f6f2be00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5639a65f7570 0x5639a65f6f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:11:17.383 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.404+0000 7f6f2be00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5639a65f7570 0x5639a65f6f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:36244/0 (socket says 172.21.15.107:36244) 2024-03-08T21:11:17.383 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.404+0000 7f6f2be00640 1 -- 172.21.15.107:0/3520261923 learned_addr learned my addr 172.21.15.107:0/3520261923 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:11:17.383 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.404+0000 7f6f23e00640 1 --2- 172.21.15.107:0/3520261923 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5639a65ee7a0 0x5639a65f6a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:11:17.383 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.404+0000 7f6f2b400640 1 --2- 172.21.15.107:0/3520261923 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5639a65ede80 0x5639a65e7680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:11:17.383 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.404+0000 7f6f2be00640 1 -- 172.21.15.107:0/3520261923 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5639a65ee7a0 msgr2=0x5639a65f6a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:11:17.383 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.404+0000 7f6f2be00640 1 --2- 172.21.15.107:0/3520261923 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5639a65ee7a0 0x5639a65f6a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:11:17.383 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.404+0000 7f6f2be00640 1 -- 172.21.15.107:0/3520261923 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5639a65ede80 msgr2=0x5639a65e7680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:11:17.383 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.404+0000 7f6f2be00640 1 --2- 172.21.15.107:0/3520261923 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5639a65ede80 0x5639a65e7680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:11:17.383 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.404+0000 7f6f2be00640 1 -- 172.21.15.107:0/3520261923 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5639a65f4c50 con 0x5639a65f7570 2024-03-08T21:11:17.383 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.405+0000 7f6f2be00640 1 --2- 172.21.15.107:0/3520261923 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5639a65f7570 0x5639a65f6f40 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f6f24004af0 tx=0x7f6f24040000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20a4d50e48ac64d4 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:11:17.386 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.405+0000 7f6f1be00640 1 -- 172.21.15.107:0/3520261923 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f24046070 con 0x5639a65f7570 2024-03-08T21:11:17.386 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.405+0000 7f6f1be00640 1 -- 172.21.15.107:0/3520261923 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f6f24052070 con 0x5639a65f7570 2024-03-08T21:11:17.386 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.405+0000 7f6f1be00640 1 -- 172.21.15.107:0/3520261923 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f24039040 con 0x5639a65f7570 2024-03-08T21:11:17.386 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.406+0000 7f6f35200740 1 -- 172.21.15.107:0/3520261923 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5639a65f7570 msgr2=0x5639a65f6f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:11:17.386 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.406+0000 7f6f35200740 1 --2- 172.21.15.107:0/3520261923 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5639a65f7570 0x5639a65f6f40 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f6f24004af0 tx=0x7f6f24040000 comp rx=0 tx=0).stop 2024-03-08T21:11:17.386 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.406+0000 7f6f35200740 1 -- 172.21.15.107:0/3520261923 shutdown_connections 2024-03-08T21:11:17.387 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.406+0000 7f6f35200740 1 --2- 172.21.15.107:0/3520261923 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5639a65f7570 0x5639a65f6f40 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:11:17.387 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.406+0000 7f6f35200740 1 --2- 172.21.15.107:0/3520261923 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5639a65ee7a0 0x5639a65f6a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:11:17.387 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.406+0000 7f6f35200740 1 --2- 172.21.15.107:0/3520261923 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5639a65ede80 0x5639a65e7680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:11:17.387 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.406+0000 7f6f35200740 1 -- 172.21.15.107:0/3520261923 >> 172.21.15.107:0/3520261923 conn(0x5639a65e87a0 msgr2=0x5639a65e6f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:11:17.387 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.406+0000 7f6f35200740 1 -- 172.21.15.107:0/3520261923 shutdown_connections 2024-03-08T21:11:17.387 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.406+0000 7f6f35200740 1 -- 172.21.15.107:0/3520261923 wait complete. 2024-03-08T21:11:17.387 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.407+0000 7f6f35200740 1 -- start start 2024-03-08T21:11:17.387 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.407+0000 7f6f35200740 1 Processor -- start 2024-03-08T21:11:17.387 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.407+0000 7f6f35200740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5639a65ede80 0x5639a65fd8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:11:17.387 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.407+0000 7f6f35200740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5639a65ee7a0 0x5639a65fde10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:11:17.387 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.407+0000 7f6f35200740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5639a65f7570 0x5639a6657a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:11:17.387 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.407+0000 7f6f35200740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f2c1abd10 con 0x5639a65ee7a0 2024-03-08T21:11:17.388 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.407+0000 7f6f35200740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f2c1ab6b0 con 0x5639a65ede80 2024-03-08T21:11:17.388 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.407+0000 7f6f35200740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f2c1a8cb0 con 0x5639a65f7570 2024-03-08T21:11:17.388 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.408+0000 7f6f35200740 10 client.0 _ll_register_callbacks cb 0x7f6f1c00b480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T21:11:17.388 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.408+0000 7f6f23e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5639a65ee7a0 0x5639a65fde10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:11:17.388 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.408+0000 7f6f23e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5639a65ee7a0 0x5639a65fde10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:36252/0 (socket says 172.21.15.107:36252) 2024-03-08T21:11:17.388 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.408+0000 7f6f23e00640 1 -- 172.21.15.107:0/537053437 learned_addr learned my addr 172.21.15.107:0/537053437 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:11:17.388 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.408+0000 7f6f2b400640 1 --2- 172.21.15.107:0/537053437 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5639a65ede80 0x5639a65fd8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:11:17.388 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.408+0000 7f6f2be00640 1 --2- 172.21.15.107:0/537053437 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5639a65f7570 0x5639a6657a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:11:17.388 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.408+0000 7f6f23e00640 1 -- 172.21.15.107:0/537053437 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5639a65f7570 msgr2=0x5639a6657a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:11:17.388 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.408+0000 7f6f23e00640 1 --2- 172.21.15.107:0/537053437 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5639a65f7570 0x5639a6657a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:11:17.389 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.408+0000 7f6f23e00640 1 -- 172.21.15.107:0/537053437 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5639a65ede80 msgr2=0x5639a65fd8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:11:17.389 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.408+0000 7f6f23e00640 1 --2- 172.21.15.107:0/537053437 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5639a65ede80 0x5639a65fd8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:11:17.389 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.408+0000 7f6f23e00640 1 -- 172.21.15.107:0/537053437 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5639a66580d0 con 0x5639a65ee7a0 2024-03-08T21:11:17.389 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.409+0000 7f6f23e00640 1 --2- 172.21.15.107:0/537053437 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5639a65ee7a0 0x5639a65fde10 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f6f1409fed0 tx=0x7f6f140d4990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95c336e32f86ad6d server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:11:17.389 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.409+0000 7f6f19600640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-03-08T21:11:17.389 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.409+0000 7f6f19600640 1 -- 172.21.15.107:0/537053437 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f140a3040 con 0x5639a65ee7a0 2024-03-08T21:11:17.389 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.409+0000 7f6f19600640 1 -- 172.21.15.107:0/537053437 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f6f1409f4a0 con 0x5639a65ee7a0 2024-03-08T21:11:17.389 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.409+0000 7f6f35200740 1 -- 172.21.15.107:0/537053437 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6f24052070 con 0x5639a65ee7a0 2024-03-08T21:11:17.390 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.409+0000 7f6f19600640 1 -- 172.21.15.107:0/537053437 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6f14009b10 con 0x5639a65ee7a0 2024-03-08T21:11:17.390 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.409+0000 7f6f35200740 10 client.16359 fetch_fsmap learned FSMap version 38 2024-03-08T21:11:17.390 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.409+0000 7f6f35200740 1 -- 172.21.15.107:0/537053437 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x5639a665f3f0 con 0x5639a65ee7a0 2024-03-08T21:11:17.390 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.409+0000 7f6f19600640 1 -- 172.21.15.107:0/537053437 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f6f24052070 con 0x5639a65ee7a0 2024-03-08T21:11:17.390 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.410+0000 7f6f19600640 1 -- 172.21.15.107:0/537053437 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 38) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f6f140a23c0 con 0x5639a65ee7a0 2024-03-08T21:11:17.391 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.410+0000 7f6f35200740 10 client.16359 fetch_fsmap finished waiting for FSMap version 38 2024-03-08T21:11:17.391 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.410+0000 7f6f35200740 10 client.16359 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-08T21:11:17.391 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.410+0000 7f6f35200740 20 client.16359 populate_metadata read hostname 'smithi107' 2024-03-08T21:11:17.391 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.410+0000 7f6f35200740 1 --2- 172.21.15.107:0/537053437 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x5639a665fb20 0x5639a6661f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:11:17.391 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.410+0000 7f6f35200740 4 client.16359 mds_command: new command op to 34475 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-08T21:11:17.391 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.410+0000 7f6f35200740 1 -- 172.21.15.107:0/537053437 --> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5639a6558c60 con 0x5639a665fb20 2024-03-08T21:11:17.391 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.410+0000 7f6f2b400640 1 --2- 172.21.15.107:0/537053437 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x5639a665fb20 0x5639a6661f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:11:17.391 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.411+0000 7f6f2b400640 1 --2- 172.21.15.107:0/537053437 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x5639a665fb20 0x5639a6661f40 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3def30e4cd2a1406 server_cookie=e701102dd199b4bd in_seq=0 out_seq=0 2024-03-08T21:11:17.391 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.411+0000 7f6f19600640 10 client.16359 ms_handle_connect on v2:172.21.15.149:6838/55290596 2024-03-08T21:11:17.405 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.427+0000 7f6f19600640 1 -- 172.21.15.107:0/537053437 <== mds.0 v2:172.21.15.149:6838/55290596 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5639a6558c60 con 0x5639a665fb20 2024-03-08T21:11:17.405 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.427+0000 7f6f19600640 10 client.16359 handle_command_reply: tid=0 2024-03-08T21:11:17.405 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.427+0000 7f6f35200740 1 client.16359 shutdown 2024-03-08T21:11:17.405 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.427+0000 7f6f35200740 1 -- 172.21.15.107:0/537053437 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5639a65ee7a0 msgr2=0x5639a65fde10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:11:17.405 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.427+0000 7f6f35200740 1 --2- 172.21.15.107:0/537053437 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5639a65ee7a0 0x5639a65fde10 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f6f1409fed0 tx=0x7f6f140d4990 comp rx=0 tx=0).stop 2024-03-08T21:11:17.405 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.427+0000 7f6f35200740 1 -- 172.21.15.107:0/537053437 shutdown_connections 2024-03-08T21:11:17.405 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.427+0000 7f6f35200740 1 --2- 172.21.15.107:0/537053437 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5639a65f7570 0x5639a6657a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:11:17.405 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.427+0000 7f6f35200740 1 --2- 172.21.15.107:0/537053437 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5639a65ee7a0 0x5639a65fde10 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:11:17.405 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.427+0000 7f6f35200740 1 --2- 172.21.15.107:0/537053437 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x5639a665fb20 0x5639a6661f40 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:11:17.405 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.427+0000 7f6f35200740 1 --2- 172.21.15.107:0/537053437 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5639a65ede80 0x5639a65fd8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:11:17.406 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.427+0000 7f6f35200740 1 -- 172.21.15.107:0/537053437 >> 172.21.15.107:0/537053437 conn(0x5639a65e87a0 msgr2=0x5639a65e7800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:11:17.406 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.427+0000 7f6f19600640 0 client.16359 ms_handle_reset on v2:172.21.15.149:6838/55290596 2024-03-08T21:11:17.406 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.427+0000 7f6f35200740 1 -- 172.21.15.107:0/537053437 shutdown_connections 2024-03-08T21:11:17.406 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.427+0000 7f6f35200740 1 -- 172.21.15.107:0/537053437 wait complete. 2024-03-08T21:11:17.406 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.427+0000 7f6f35200740 20 client.16359 trim_cache size 0 max 16384 2024-03-08T21:11:17.406 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:{ 2024-03-08T21:11:17.406 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "message": "", 2024-03-08T21:11:17.406 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "return_code": 0 2024-03-08T21:11:17.406 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:} 2024-03-08T21:11:17.407 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.428+0000 7f6f31000640 1 -- 172.21.15.107:0/3093599342 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f6f100a2960 msgr2=0x7f6f100a4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:11:17.407 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.428+0000 7f6f31000640 1 --2- 172.21.15.107:0/3093599342 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f6f100a2960 0x7f6f100a4e20 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f6f24005e30 tx=0x7f6f2402e040 comp rx=0 tx=0).stop 2024-03-08T21:11:17.407 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.428+0000 7f6f31000640 1 -- 172.21.15.107:0/3093599342 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6f2c108620 msgr2=0x7f6f2c19fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:11:17.407 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.428+0000 7f6f31000640 1 --2- 172.21.15.107:0/3093599342 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6f2c108620 0x7f6f2c19fd20 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f6f1400d770 tx=0x7f6f1400dc30 comp rx=0 tx=0).stop 2024-03-08T21:11:17.407 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.428+0000 7f6f31000640 1 -- 172.21.15.107:0/3093599342 shutdown_connections 2024-03-08T21:11:17.407 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.428+0000 7f6f31000640 1 --2- 172.21.15.107:0/3093599342 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f6f100a2960 0x7f6f100a4e20 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:11:17.408 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.428+0000 7f6f31000640 1 --2- 172.21.15.107:0/3093599342 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6f2c199990 0x7f6f2c199e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:11:17.408 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.428+0000 7f6f31000640 1 --2- 172.21.15.107:0/3093599342 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6f2c108620 0x7f6f2c19fd20 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:11:17.408 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.428+0000 7f6f31000640 1 --2- 172.21.15.107:0/3093599342 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f6f2c103550 0x7f6f2c19f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:11:17.408 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.428+0000 7f6f31000640 1 -- 172.21.15.107:0/3093599342 >> 172.21.15.107:0/3093599342 conn(0x7f6f2c0fedd0 msgr2=0x7f6f2c101110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:11:17.408 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.428+0000 7f6f31000640 1 -- 172.21.15.107:0/3093599342 shutdown_connections 2024-03-08T21:11:17.408 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:17.429+0000 7f6f31000640 1 -- 172.21.15.107:0/3093599342 wait complete. 2024-03-08T21:11:18.760 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:11:18 smithi149 ceph-mon[35042]: pgmap v340: 97 pgs: 97 active+clean; 7.4 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 409 B/s wr, 8 op/s 2024-03-08T21:11:18.873 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:11:18 smithi204 ceph-mon[34935]: pgmap v340: 97 pgs: 97 active+clean; 7.4 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 409 B/s wr, 8 op/s 2024-03-08T21:11:18.962 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:11:18 smithi107 ceph-mon[31381]: pgmap v340: 97 pgs: 97 active+clean; 7.4 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 409 B/s wr, 8 op/s 2024-03-08T21:11:20.010 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:11:19 smithi149 ceph-mon[35042]: MDS daemon mds.c is removed because it is dead or otherwise unavailable. 2024-03-08T21:11:20.010 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:11:19 smithi149 ceph-mon[35042]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2024-03-08T21:11:20.123 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:11:19 smithi204 ceph-mon[34935]: MDS daemon mds.c is removed because it is dead or otherwise unavailable. 2024-03-08T21:11:20.123 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:11:19 smithi204 ceph-mon[34935]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2024-03-08T21:11:20.153 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:11:19 smithi107 ceph-mon[31381]: MDS daemon mds.c is removed because it is dead or otherwise unavailable. 2024-03-08T21:11:20.153 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:11:19 smithi107 ceph-mon[31381]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2024-03-08T21:11:21.010 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:11:20 smithi149 ceph-mon[35042]: pgmap v341: 97 pgs: 97 active+clean; 7.4 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 511 B/s wr, 8 op/s 2024-03-08T21:11:21.123 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:11:20 smithi204 ceph-mon[34935]: pgmap v341: 97 pgs: 97 active+clean; 7.4 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 511 B/s wr, 8 op/s 2024-03-08T21:11:21.212 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:11:20 smithi107 ceph-mon[31381]: pgmap v341: 97 pgs: 97 active+clean; 7.4 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 511 B/s wr, 8 op/s 2024-03-08T21:11:21.962 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:11:21 smithi107 ceph-mon[31381]: pgmap v342: 97 pgs: 97 active+clean; 7.4 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 511 B/s wr, 8 op/s 2024-03-08T21:11:22.009 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:11:21 smithi149 ceph-mon[35042]: pgmap v342: 97 pgs: 97 active+clean; 7.4 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 511 B/s wr, 8 op/s 2024-03-08T21:11:22.123 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:11:21 smithi204 ceph-mon[34935]: pgmap v342: 97 pgs: 97 active+clean; 7.4 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 511 B/s wr, 8 op/s 2024-03-08T21:11:24.123 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:11:23 smithi204 ceph-mon[34935]: pgmap v343: 97 pgs: 97 active+clean; 7.4 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 511 B/s wr, 8 op/s 2024-03-08T21:11:24.212 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:11:23 smithi107 ceph-mon[31381]: pgmap v343: 97 pgs: 97 active+clean; 7.4 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 511 B/s wr, 8 op/s 2024-03-08T21:11:24.260 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:11:23 smithi149 ceph-mon[35042]: pgmap v343: 97 pgs: 97 active+clean; 7.4 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 511 B/s wr, 8 op/s 2024-03-08T21:11:26.123 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:11:25 smithi204 ceph-mon[34935]: pgmap v344: 97 pgs: 97 active+clean; 7.4 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 389 B/s wr, 9 op/s 2024-03-08T21:11:26.212 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:11:25 smithi107 ceph-mon[31381]: pgmap v344: 97 pgs: 97 active+clean; 7.4 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 389 B/s wr, 9 op/s 2024-03-08T21:11:26.261 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:11:25 smithi149 ceph-mon[35042]: pgmap v344: 97 pgs: 97 active+clean; 7.4 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 389 B/s wr, 9 op/s 2024-03-08T21:11:28.123 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:11:27 smithi204 ceph-mon[34935]: pgmap v345: 97 pgs: 97 active+clean; 7.4 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 341 B/s wr, 8 op/s 2024-03-08T21:11:28.212 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:11:27 smithi107 ceph-mon[31381]: pgmap v345: 97 pgs: 97 active+clean; 7.4 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 341 B/s wr, 8 op/s 2024-03-08T21:11:28.260 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:11:27 smithi149 ceph-mon[35042]: pgmap v345: 97 pgs: 97 active+clean; 7.4 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 341 B/s wr, 8 op/s 2024-03-08T21:11:30.123 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:11:29 smithi204 ceph-mon[34935]: pgmap v346: 97 pgs: 97 active+clean; 7.4 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 341 B/s wr, 12 op/s 2024-03-08T21:11:30.212 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:11:29 smithi107 ceph-mon[31381]: pgmap v346: 97 pgs: 97 active+clean; 7.4 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 341 B/s wr, 12 op/s 2024-03-08T21:11:30.260 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:11:29 smithi149 ceph-mon[35042]: pgmap v346: 97 pgs: 97 active+clean; 7.4 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 341 B/s wr, 12 op/s 2024-03-08T21:11:30.558 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.579+0000 7f03bb000640 1 -- 172.21.15.107:0/676507257 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f03b410da60 msgr2=0x7f03b410fe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:11:30.558 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.579+0000 7f03bb000640 1 --2- 172.21.15.107:0/676507257 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f03b410da60 0x7f03b410fe50 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f03ac009ca0 tx=0x7f03ac02c100 comp rx=0 tx=0).stop 2024-03-08T21:11:30.558 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.579+0000 7f03bb000640 1 -- 172.21.15.107:0/676507257 shutdown_connections 2024-03-08T21:11:30.558 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.579+0000 7f03bb000640 1 --2- 172.21.15.107:0/676507257 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f03b410da60 0x7f03b410fe50 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:11:30.558 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.579+0000 7f03bb000640 1 --2- 172.21.15.107:0/676507257 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f03b4100860 0x7f03b410d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:11:30.558 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.579+0000 7f03bb000640 1 --2- 172.21.15.107:0/676507257 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f03b40ffeb0 0x7f03b4100290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:11:30.558 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.579+0000 7f03bb000640 1 -- 172.21.15.107:0/676507257 >> 172.21.15.107:0/676507257 conn(0x7f03b40fbb00 msgr2=0x7f03b40fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:11:30.559 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.579+0000 7f03bb000640 1 -- 172.21.15.107:0/676507257 shutdown_connections 2024-03-08T21:11:30.559 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.579+0000 7f03bb000640 1 -- 172.21.15.107:0/676507257 wait complete. 2024-03-08T21:11:30.561 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.580+0000 7f03bb000640 1 Processor -- start 2024-03-08T21:11:30.561 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.581+0000 7f03bb000640 1 -- start start 2024-03-08T21:11:30.561 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.581+0000 7f03bb000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f03b40ffeb0 0x7f03b419f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:11:30.561 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.581+0000 7f03bb000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f03b4100860 0x7f03b419fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:11:30.561 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.581+0000 7f03bb000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f03b410da60 0x7f03b4199960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:11:30.561 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.581+0000 7f03bb000640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03b41135c0 con 0x7f03b4100860 2024-03-08T21:11:30.561 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.581+0000 7f03bb000640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03b4113440 con 0x7f03b410da60 2024-03-08T21:11:30.561 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.581+0000 7f03bb000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03b4113740 con 0x7f03b40ffeb0 2024-03-08T21:11:30.562 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.581+0000 7f03b9200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f03b4100860 0x7f03b419fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:11:30.562 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.581+0000 7f03b9200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f03b4100860 0x7f03b419fcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:58186/0 (socket says 172.21.15.107:58186) 2024-03-08T21:11:30.562 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.581+0000 7f03b9200640 1 -- 172.21.15.107:0/246394846 learned_addr learned my addr 172.21.15.107:0/246394846 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:11:30.562 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.582+0000 7f03b9c00640 1 --2- 172.21.15.107:0/246394846 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f03b40ffeb0 0x7f03b419f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:11:30.562 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.582+0000 7f03ba600640 1 --2- 172.21.15.107:0/246394846 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f03b410da60 0x7f03b4199960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:11:30.562 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.582+0000 7f03b9200640 1 -- 172.21.15.107:0/246394846 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f03b40ffeb0 msgr2=0x7f03b419f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:11:30.562 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.582+0000 7f03b9200640 1 --2- 172.21.15.107:0/246394846 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f03b40ffeb0 0x7f03b419f7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:11:30.562 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.582+0000 7f03b9200640 1 -- 172.21.15.107:0/246394846 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f03b410da60 msgr2=0x7f03b4199960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:11:30.562 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.582+0000 7f03b9200640 1 --2- 172.21.15.107:0/246394846 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f03b410da60 0x7f03b4199960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:11:30.562 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.582+0000 7f03b9200640 1 -- 172.21.15.107:0/246394846 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03b419a0e0 con 0x7f03b4100860 2024-03-08T21:11:30.562 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.582+0000 7f03b9200640 1 --2- 172.21.15.107:0/246394846 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f03b4100860 0x7f03b419fcf0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f03a000b4a0 tx=0x7f03a000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:11:30.562 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.582+0000 7f03b2000640 1 -- 172.21.15.107:0/246394846 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03a0014070 con 0x7f03b4100860 2024-03-08T21:11:30.562 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.583+0000 7f03bb000640 1 -- 172.21.15.107:0/246394846 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03b419a3d0 con 0x7f03b4100860 2024-03-08T21:11:30.563 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.583+0000 7f03b2000640 1 -- 172.21.15.107:0/246394846 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f03a0010ac0 con 0x7f03b4100860 2024-03-08T21:11:30.563 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.583+0000 7f03b2000640 1 -- 172.21.15.107:0/246394846 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03a0017c30 con 0x7f03b4100860 2024-03-08T21:11:30.563 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.583+0000 7f03bb000640 1 -- 172.21.15.107:0/246394846 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03b41a6670 con 0x7f03b4100860 2024-03-08T21:11:30.565 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.586+0000 7f03b2000640 1 -- 172.21.15.107:0/246394846 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) v1 ==== 137422+0+0 (secure 0 0 0) 0x7f03a0017070 con 0x7f03b4100860 2024-03-08T21:11:30.565 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.586+0000 7f03bf3c4740 1 Processor -- start 2024-03-08T21:11:30.565 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.587+0000 7f03bf3c4740 1 -- start start 2024-03-08T21:11:30.566 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.587+0000 7f03bf3c4740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a0d0d4ee80 0x55a0d0d4f260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:11:30.566 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.587+0000 7f03bf3c4740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a0d0d4f7a0 0x55a0d0d58030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:11:30.566 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.587+0000 7f03bf3c4740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a0d0d58570 0x55a0d0d5a960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:11:30.566 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.587+0000 7f03bf3c4740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a0d0cb23d0 con 0x55a0d0d58570 2024-03-08T21:11:30.566 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.587+0000 7f03bf3c4740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a0d0c5b6b0 con 0x55a0d0d4ee80 2024-03-08T21:11:30.566 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.587+0000 7f03bf3c4740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a0d0cae470 con 0x55a0d0d4f7a0 2024-03-08T21:11:30.566 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.587+0000 7f03b9200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a0d0d58570 0x55a0d0d5a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:11:30.566 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.588+0000 7f03b9200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a0d0d58570 0x55a0d0d5a960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:58192/0 (socket says 172.21.15.107:58192) 2024-03-08T21:11:30.566 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.588+0000 7f03b9200640 1 -- 172.21.15.107:0/3410142914 learned_addr learned my addr 172.21.15.107:0/3410142914 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:11:30.566 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.588+0000 7f03ba600640 1 --2- 172.21.15.107:0/3410142914 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a0d0d4ee80 0x55a0d0d4f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:11:30.567 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.588+0000 7f03b9c00640 1 --2- 172.21.15.107:0/3410142914 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a0d0d4f7a0 0x55a0d0d58030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:11:30.569 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.588+0000 7f03b9200640 1 -- 172.21.15.107:0/3410142914 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a0d0d4f7a0 msgr2=0x55a0d0d58030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:11:30.569 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.588+0000 7f03b9200640 1 --2- 172.21.15.107:0/3410142914 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a0d0d4f7a0 0x55a0d0d58030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:11:30.569 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.588+0000 7f03b9200640 1 -- 172.21.15.107:0/3410142914 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a0d0d4ee80 msgr2=0x55a0d0d4f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:11:30.569 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.588+0000 7f03b9200640 1 --2- 172.21.15.107:0/3410142914 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a0d0d4ee80 0x55a0d0d4f260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:11:30.569 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.588+0000 7f03b9200640 1 -- 172.21.15.107:0/3410142914 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a0d0c81b50 con 0x55a0d0d58570 2024-03-08T21:11:30.569 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.588+0000 7f03b2000640 1 --2- 172.21.15.107:0/246394846 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f039c0a2960 0x7f039c0a4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:11:30.569 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.589+0000 7f03b9200640 1 --2- 172.21.15.107:0/3410142914 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a0d0d58570 0x55a0d0d5a960 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f03a00cbcd0 tx=0x7f03a00d47b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d021f8028bbe5e34 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:11:30.569 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.589+0000 7f03ba600640 1 --2- 172.21.15.107:0/246394846 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f039c0a2960 0x7f039c0a4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:11:30.569 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.589+0000 7f03b2000640 1 -- 172.21.15.107:0/246394846 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f03a00ce5a0 con 0x7f03b4100860 2024-03-08T21:11:30.569 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.589+0000 7f03a6a00640 1 -- 172.21.15.107:0/3410142914 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03a0096f10 con 0x55a0d0d58570 2024-03-08T21:11:30.569 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.589+0000 7f03a6a00640 1 -- 172.21.15.107:0/3410142914 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f03a009b370 con 0x55a0d0d58570 2024-03-08T21:11:30.570 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.589+0000 7f03a6a00640 1 -- 172.21.15.107:0/3410142914 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03a009a5d0 con 0x55a0d0d58570 2024-03-08T21:11:30.570 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.590+0000 7f03bf3c4740 1 -- 172.21.15.107:0/3410142914 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a0d0d58570 msgr2=0x55a0d0d5a960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:11:30.570 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.590+0000 7f03ba600640 1 --2- 172.21.15.107:0/246394846 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f039c0a2960 0x7f039c0a4e20 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f03ac009bf0 tx=0x7f03ac0093a0 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:11:30.570 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.590+0000 7f03bf3c4740 1 --2- 172.21.15.107:0/3410142914 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a0d0d58570 0x55a0d0d5a960 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f03a00cbcd0 tx=0x7f03a00d47b0 comp rx=0 tx=0).stop 2024-03-08T21:11:30.570 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.590+0000 7f03bf3c4740 1 -- 172.21.15.107:0/3410142914 shutdown_connections 2024-03-08T21:11:30.570 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.590+0000 7f03bf3c4740 1 --2- 172.21.15.107:0/3410142914 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a0d0d58570 0x55a0d0d5a960 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:11:30.570 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.590+0000 7f03bf3c4740 1 --2- 172.21.15.107:0/3410142914 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a0d0d4f7a0 0x55a0d0d58030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:11:30.570 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.590+0000 7f03bf3c4740 1 --2- 172.21.15.107:0/3410142914 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a0d0d4ee80 0x55a0d0d4f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:11:30.570 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.590+0000 7f03bf3c4740 1 -- 172.21.15.107:0/3410142914 >> 172.21.15.107:0/3410142914 conn(0x55a0d0d497a0 msgr2=0x55a0d0d49b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:11:30.570 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.590+0000 7f03bf3c4740 1 -- 172.21.15.107:0/3410142914 shutdown_connections 2024-03-08T21:11:30.570 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.590+0000 7f03bf3c4740 1 -- 172.21.15.107:0/3410142914 wait complete. 2024-03-08T21:11:30.574 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.591+0000 7f03bf3c4740 1 -- start start 2024-03-08T21:11:30.574 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.591+0000 7f03bf3c4740 1 Processor -- start 2024-03-08T21:11:30.574 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.591+0000 7f03bf3c4740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a0d0d4ee80 0x55a0d0d57f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:11:30.574 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.592+0000 7f03bf3c4740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a0d0d4f7a0 0x55a0d0d55290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:11:30.574 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.592+0000 7f03bf3c4740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a0d0d58570 0x55a0d0d55810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:11:30.574 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.592+0000 7f03bf3c4740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a0d0d5b1a0 con 0x55a0d0d4ee80 2024-03-08T21:11:30.574 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.592+0000 7f03bf3c4740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a0d0c6e9f0 con 0x55a0d0d58570 2024-03-08T21:11:30.574 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.592+0000 7f03bf3c4740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a0d0d5b320 con 0x55a0d0d4f7a0 2024-03-08T21:11:30.574 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.592+0000 7f03bf3c4740 10 client.0 _ll_register_callbacks cb 0x55a0d09c6a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T21:11:30.574 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.592+0000 7f03b9c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a0d0d4ee80 0x55a0d0d57f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:11:30.575 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.592+0000 7f03b9c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a0d0d4ee80 0x55a0d0d57f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:58194/0 (socket says 172.21.15.107:58194) 2024-03-08T21:11:30.575 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.592+0000 7f03b9c00640 1 -- 172.21.15.107:0/592407293 learned_addr learned my addr 172.21.15.107:0/592407293 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:11:30.575 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.592+0000 7f03b9200640 1 --2- 172.21.15.107:0/592407293 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a0d0d4f7a0 0x55a0d0d55290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:11:30.575 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.592+0000 7f03ba600640 1 --2- 172.21.15.107:0/592407293 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a0d0d58570 0x55a0d0d55810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:11:30.575 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.592+0000 7f03b9c00640 1 -- 172.21.15.107:0/592407293 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a0d0d4f7a0 msgr2=0x55a0d0d55290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:11:30.575 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.592+0000 7f03b9c00640 1 --2- 172.21.15.107:0/592407293 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a0d0d4f7a0 0x55a0d0d55290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:11:30.575 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.592+0000 7f03b9c00640 1 -- 172.21.15.107:0/592407293 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a0d0d58570 msgr2=0x55a0d0d55810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:11:30.575 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.592+0000 7f03b9c00640 1 --2- 172.21.15.107:0/592407293 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a0d0d58570 0x55a0d0d55810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:11:30.575 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.592+0000 7f03b9c00640 1 -- 172.21.15.107:0/592407293 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a0d09c7210 con 0x55a0d0d4ee80 2024-03-08T21:11:30.575 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.592+0000 7f03ba600640 1 --2- 172.21.15.107:0/592407293 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a0d0d58570 0x55a0d0d55810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T21:11:30.575 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.593+0000 7f03b9c00640 1 --2- 172.21.15.107:0/592407293 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a0d0d4ee80 0x55a0d0d57f00 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f03a80117c0 tx=0x7f03a8011c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1a627bc206017e1 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:11:30.575 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.593+0000 7f039b400640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-03-08T21:11:30.575 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.593+0000 7f039b400640 1 -- 172.21.15.107:0/592407293 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03a801e060 con 0x55a0d0d4ee80 2024-03-08T21:11:30.576 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.593+0000 7f039b400640 1 -- 172.21.15.107:0/592407293 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f03a800fdb0 con 0x55a0d0d4ee80 2024-03-08T21:11:30.576 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.593+0000 7f03bf3c4740 1 -- 172.21.15.107:0/592407293 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f03a009b370 con 0x55a0d0d4ee80 2024-03-08T21:11:30.576 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.593+0000 7f039b400640 1 -- 172.21.15.107:0/592407293 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f03a8005020 con 0x55a0d0d4ee80 2024-03-08T21:11:30.576 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.593+0000 7f03bf3c4740 10 client.16371 fetch_fsmap learned FSMap version 39 2024-03-08T21:11:30.576 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.593+0000 7f03bf3c4740 1 -- 172.21.15.107:0/592407293 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x55a0d0a93620 con 0x55a0d0d4ee80 2024-03-08T21:11:30.576 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.594+0000 7f039b400640 1 -- 172.21.15.107:0/592407293 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f03a009b370 con 0x55a0d0d4ee80 2024-03-08T21:11:30.576 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.594+0000 7f039b400640 1 -- 172.21.15.107:0/592407293 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 39) v1 ==== 7307+0+0 (secure 0 0 0) 0x7f03a8018030 con 0x55a0d0d4ee80 2024-03-08T21:11:30.576 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.594+0000 7f03bf3c4740 10 client.16371 fetch_fsmap finished waiting for FSMap version 39 2024-03-08T21:11:30.576 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.594+0000 7f03bf3c4740 10 client.16371 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-08T21:11:30.576 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.594+0000 7f03bf3c4740 20 client.16371 populate_metadata read hostname 'smithi107' 2024-03-08T21:11:30.576 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.594+0000 7f03bf3c4740 1 --2- 172.21.15.107:0/592407293 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55a0d0d68480 0x55a0d0d6a870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:11:30.577 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.594+0000 7f03bf3c4740 4 client.16371 mds_command: new command op to 34475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-08T21:11:30.577 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.594+0000 7f03bf3c4740 1 -- 172.21.15.107:0/592407293 --> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a0d0cbf910 con 0x55a0d0d68480 2024-03-08T21:11:30.577 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.595+0000 7f03b9200640 1 --2- 172.21.15.107:0/592407293 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55a0d0d68480 0x55a0d0d6a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:11:30.577 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.596+0000 7f03b9200640 1 --2- 172.21.15.107:0/592407293 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55a0d0d68480 0x55a0d0d6a870 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3a10ebfff1a8f6dc server_cookie=69ca46d596b96ef9 in_seq=0 out_seq=0 2024-03-08T21:11:30.577 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.596+0000 7f039b400640 10 client.16371 ms_handle_connect on v2:172.21.15.149:6838/55290596 2024-03-08T21:11:30.578 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.597+0000 7f039b400640 1 -- 172.21.15.107:0/592407293 <== mds.0 v2:172.21.15.149:6838/55290596 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55a0d0cbf910 con 0x55a0d0d68480 2024-03-08T21:11:30.578 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.597+0000 7f039b400640 10 client.16371 handle_command_reply: tid=0 2024-03-08T21:11:30.578 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.597+0000 7f03bf3c4740 1 client.16371 shutdown 2024-03-08T21:11:30.578 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.597+0000 7f03bf3c4740 1 -- 172.21.15.107:0/592407293 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a0d0d4ee80 msgr2=0x55a0d0d57f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:11:30.578 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.597+0000 7f03bf3c4740 1 --2- 172.21.15.107:0/592407293 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a0d0d4ee80 0x55a0d0d57f00 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f03a80117c0 tx=0x7f03a8011c80 comp rx=0 tx=0).stop 2024-03-08T21:11:30.578 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.598+0000 7f03bf3c4740 1 -- 172.21.15.107:0/592407293 shutdown_connections 2024-03-08T21:11:30.578 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.598+0000 7f03bf3c4740 1 --2- 172.21.15.107:0/592407293 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55a0d0d68480 0x55a0d0d6a870 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:11:30.579 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.598+0000 7f03bf3c4740 1 --2- 172.21.15.107:0/592407293 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a0d0d58570 0x55a0d0d55810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:11:30.579 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.598+0000 7f03bf3c4740 1 --2- 172.21.15.107:0/592407293 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a0d0d4f7a0 0x55a0d0d55290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:11:30.579 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.598+0000 7f039b400640 0 client.16371 ms_handle_reset on v2:172.21.15.149:6838/55290596 2024-03-08T21:11:30.579 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.598+0000 7f03bf3c4740 1 --2- 172.21.15.107:0/592407293 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a0d0d4ee80 0x55a0d0d57f00 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:11:30.579 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.598+0000 7f03bf3c4740 1 -- 172.21.15.107:0/592407293 >> 172.21.15.107:0/592407293 conn(0x55a0d0d497a0 msgr2=0x55a0d0d4b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:11:30.579 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.598+0000 7f03bf3c4740 1 -- 172.21.15.107:0/592407293 shutdown_connections 2024-03-08T21:11:30.579 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.598+0000 7f03bf3c4740 1 -- 172.21.15.107:0/592407293 wait complete. 2024-03-08T21:11:30.579 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.598+0000 7f03bf3c4740 20 client.16371 trim_cache size 0 max 16384 2024-03-08T21:11:30.595 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.617+0000 7f03bf3c4740 1 Processor -- start 2024-03-08T21:11:30.595 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.617+0000 7f03bf3c4740 1 -- start start 2024-03-08T21:11:30.595 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.617+0000 7f03bf3c4740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a0d0d4ee80 0x55a0d0d48680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:11:30.595 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.617+0000 7f03bf3c4740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a0d0d4f7a0 0x55a0d0d579c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:11:30.595 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.617+0000 7f03bf3c4740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a0d0d58570 0x55a0d0d57f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:11:30.596 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.617+0000 7f03bf3c4740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03b41b0740 con 0x55a0d0d4ee80 2024-03-08T21:11:30.596 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.617+0000 7f03bf3c4740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03b41ae760 con 0x55a0d0d4f7a0 2024-03-08T21:11:30.596 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.617+0000 7f03bf3c4740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03b41b00e0 con 0x55a0d0d58570 2024-03-08T21:11:30.597 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.617+0000 7f03b9c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a0d0d4ee80 0x55a0d0d48680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:11:30.597 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.617+0000 7f03b9c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a0d0d4ee80 0x55a0d0d48680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:58204/0 (socket says 172.21.15.107:58204) 2024-03-08T21:11:30.597 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.617+0000 7f03b9c00640 1 -- 172.21.15.107:0/1546028986 learned_addr learned my addr 172.21.15.107:0/1546028986 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:11:30.597 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.618+0000 7f03ba600640 1 --2- 172.21.15.107:0/1546028986 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a0d0d58570 0x55a0d0d57f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:11:30.597 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.618+0000 7f03b9200640 1 --2- 172.21.15.107:0/1546028986 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a0d0d4f7a0 0x55a0d0d579c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:11:30.597 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.618+0000 7f03b9c00640 1 -- 172.21.15.107:0/1546028986 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a0d0d58570 msgr2=0x55a0d0d57f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:11:30.597 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.618+0000 7f03b9c00640 1 --2- 172.21.15.107:0/1546028986 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a0d0d58570 0x55a0d0d57f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:11:30.597 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.618+0000 7f03b9c00640 1 -- 172.21.15.107:0/1546028986 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a0d0d4f7a0 msgr2=0x55a0d0d579c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:11:30.597 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.618+0000 7f03b9c00640 1 --2- 172.21.15.107:0/1546028986 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a0d0d4f7a0 0x55a0d0d579c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:11:30.598 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.618+0000 7f03b9c00640 1 -- 172.21.15.107:0/1546028986 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a0d0d4f5d0 con 0x55a0d0d4ee80 2024-03-08T21:11:30.598 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.618+0000 7f03ba600640 1 --2- 172.21.15.107:0/1546028986 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a0d0d58570 0x55a0d0d57f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T21:11:30.598 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.618+0000 7f03b9c00640 1 --2- 172.21.15.107:0/1546028986 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a0d0d4ee80 0x55a0d0d48680 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f03a8000c00 tx=0x7f03a8005fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16fb9ac9650be7 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:11:30.598 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.619+0000 7f03a6000640 1 -- 172.21.15.107:0/1546028986 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03a800fbb0 con 0x55a0d0d4ee80 2024-03-08T21:11:30.598 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.619+0000 7f03a6000640 1 -- 172.21.15.107:0/1546028986 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f03a802e070 con 0x55a0d0d4ee80 2024-03-08T21:11:30.598 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.619+0000 7f03a6000640 1 -- 172.21.15.107:0/1546028986 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03a80055b0 con 0x55a0d0d4ee80 2024-03-08T21:11:30.600 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.619+0000 7f03bf3c4740 1 -- 172.21.15.107:0/1546028986 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a0d0d4ee80 msgr2=0x55a0d0d48680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:11:30.600 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.619+0000 7f03bf3c4740 1 --2- 172.21.15.107:0/1546028986 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a0d0d4ee80 0x55a0d0d48680 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f03a8000c00 tx=0x7f03a8005fc0 comp rx=0 tx=0).stop 2024-03-08T21:11:30.600 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.620+0000 7f03bf3c4740 1 -- 172.21.15.107:0/1546028986 shutdown_connections 2024-03-08T21:11:30.600 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.620+0000 7f03bf3c4740 1 --2- 172.21.15.107:0/1546028986 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a0d0d58570 0x55a0d0d57f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:11:30.600 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.620+0000 7f03bf3c4740 1 --2- 172.21.15.107:0/1546028986 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a0d0d4f7a0 0x55a0d0d579c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:11:30.600 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.620+0000 7f03bf3c4740 1 --2- 172.21.15.107:0/1546028986 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a0d0d4ee80 0x55a0d0d48680 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:11:30.600 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.620+0000 7f03bf3c4740 1 -- 172.21.15.107:0/1546028986 >> 172.21.15.107:0/1546028986 conn(0x55a0d0d497a0 msgr2=0x55a0d0d47f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:11:30.600 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.620+0000 7f03bf3c4740 1 -- 172.21.15.107:0/1546028986 shutdown_connections 2024-03-08T21:11:30.601 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.620+0000 7f03bf3c4740 1 -- 172.21.15.107:0/1546028986 wait complete. 2024-03-08T21:11:30.601 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.620+0000 7f03bf3c4740 1 -- start start 2024-03-08T21:11:30.601 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.620+0000 7f03bf3c4740 1 Processor -- start 2024-03-08T21:11:30.601 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.621+0000 7f03bf3c4740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a0d0d4ee80 0x55a0d0d5e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:11:30.601 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.621+0000 7f03bf3c4740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a0d0d4f7a0 0x55a0d0d5e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:11:30.601 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.621+0000 7f03bf3c4740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a0d0d58570 0x55a0d0d5ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:11:30.601 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.621+0000 7f03bf3c4740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03b41af420 con 0x55a0d0d4ee80 2024-03-08T21:11:30.601 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.621+0000 7f03bf3c4740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03b41aedc0 con 0x55a0d0d4f7a0 2024-03-08T21:11:30.601 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.621+0000 7f03bf3c4740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03b41afa80 con 0x55a0d0d58570 2024-03-08T21:11:30.601 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.621+0000 7f03bf3c4740 10 client.0 _ll_register_callbacks cb 0x7f03a00d1f20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T21:11:30.601 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.621+0000 7f03b9c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a0d0d4ee80 0x55a0d0d5e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:11:30.601 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.621+0000 7f03b9c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a0d0d4ee80 0x55a0d0d5e170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:58216/0 (socket says 172.21.15.107:58216) 2024-03-08T21:11:30.601 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.621+0000 7f03b9c00640 1 -- 172.21.15.107:0/3769500999 learned_addr learned my addr 172.21.15.107:0/3769500999 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:11:30.601 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.621+0000 7f03b9c00640 1 -- 172.21.15.107:0/3769500999 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a0d0d58570 msgr2=0x55a0d0d5ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:11:30.601 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.621+0000 7f03b9c00640 1 --2- 172.21.15.107:0/3769500999 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a0d0d58570 0x55a0d0d5ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:11:30.601 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.621+0000 7f03b9200640 1 --2- 172.21.15.107:0/3769500999 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a0d0d4f7a0 0x55a0d0d5e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:11:30.602 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.622+0000 7f03b9c00640 1 -- 172.21.15.107:0/3769500999 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a0d0d4f7a0 msgr2=0x55a0d0d5e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:11:30.602 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.622+0000 7f03b9c00640 1 --2- 172.21.15.107:0/3769500999 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a0d0d4f7a0 0x55a0d0d5e6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:11:30.602 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.622+0000 7f03b9c00640 1 -- 172.21.15.107:0/3769500999 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a0d0dbec00 con 0x55a0d0d4ee80 2024-03-08T21:11:30.602 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.622+0000 7f03b9200640 1 --2- 172.21.15.107:0/3769500999 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a0d0d4f7a0 0x55a0d0d5e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T21:11:30.602 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.622+0000 7f03b9c00640 1 --2- 172.21.15.107:0/3769500999 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a0d0d4ee80 0x55a0d0d5e170 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f03a800c5a0 tx=0x7f03a800e850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ddd9759468255b9 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:11:30.602 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.622+0000 7f039b400640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-03-08T21:11:30.603 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.622+0000 7f039b400640 1 -- 172.21.15.107:0/3769500999 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03a800ebd0 con 0x55a0d0d4ee80 2024-03-08T21:11:30.604 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.622+0000 7f039b400640 1 -- 172.21.15.107:0/3769500999 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f03a800ed70 con 0x55a0d0d4ee80 2024-03-08T21:11:30.604 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.622+0000 7f03bf3c4740 1 -- 172.21.15.107:0/3769500999 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f03a802e070 con 0x55a0d0d4ee80 2024-03-08T21:11:30.604 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.623+0000 7f039b400640 1 -- 172.21.15.107:0/3769500999 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f03a80051c0 con 0x55a0d0d4ee80 2024-03-08T21:11:30.604 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.623+0000 7f03bf3c4740 10 client.16377 fetch_fsmap learned FSMap version 39 2024-03-08T21:11:30.604 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.623+0000 7f03bf3c4740 1 -- 172.21.15.107:0/3769500999 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x55a0d0dbfed0 con 0x55a0d0d4ee80 2024-03-08T21:11:30.604 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.623+0000 7f039b400640 1 -- 172.21.15.107:0/3769500999 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f03a802e070 con 0x55a0d0d4ee80 2024-03-08T21:11:30.604 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.623+0000 7f039b400640 1 -- 172.21.15.107:0/3769500999 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 39) v1 ==== 7307+0+0 (secure 0 0 0) 0x7f03a8018040 con 0x55a0d0d4ee80 2024-03-08T21:11:30.604 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.623+0000 7f03bf3c4740 10 client.16377 fetch_fsmap finished waiting for FSMap version 39 2024-03-08T21:11:30.604 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.623+0000 7f03bf3c4740 10 client.16377 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-08T21:11:30.604 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.623+0000 7f03bf3c4740 20 client.16377 populate_metadata read hostname 'smithi107' 2024-03-08T21:11:30.604 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.623+0000 7f03bf3c4740 1 --2- 172.21.15.107:0/3769500999 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55a0d0dc0430 0x55a0d0dc2840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:11:30.604 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.623+0000 7f03bf3c4740 4 client.16377 mds_command: new command op to 34475 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-08T21:11:30.604 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.623+0000 7f03bf3c4740 1 -- 172.21.15.107:0/3769500999 --> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55a0d0cb9c60 con 0x55a0d0dc0430 2024-03-08T21:11:30.604 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.624+0000 7f03b9200640 1 --2- 172.21.15.107:0/3769500999 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55a0d0dc0430 0x55a0d0dc2840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:11:30.604 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.625+0000 7f03b9200640 1 --2- 172.21.15.107:0/3769500999 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55a0d0dc0430 0x55a0d0dc2840 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b6ee0a6d3027d4fc server_cookie=c866b04079e35232 in_seq=0 out_seq=0 2024-03-08T21:11:30.605 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.625+0000 7f039b400640 10 client.16377 ms_handle_connect on v2:172.21.15.149:6838/55290596 2024-03-08T21:11:30.612 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.634+0000 7f039b400640 1 -- 172.21.15.107:0/3769500999 <== mds.0 v2:172.21.15.149:6838/55290596 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55a0d0cb9c60 con 0x55a0d0dc0430 2024-03-08T21:11:30.612 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.634+0000 7f039b400640 10 client.16377 handle_command_reply: tid=0 2024-03-08T21:11:30.612 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.634+0000 7f03bf3c4740 1 client.16377 shutdown 2024-03-08T21:11:30.613 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.634+0000 7f03bf3c4740 1 -- 172.21.15.107:0/3769500999 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a0d0d4ee80 msgr2=0x55a0d0d5e170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:11:30.613 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.634+0000 7f03bf3c4740 1 --2- 172.21.15.107:0/3769500999 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a0d0d4ee80 0x55a0d0d5e170 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f03a800c5a0 tx=0x7f03a800e850 comp rx=0 tx=0).stop 2024-03-08T21:11:30.613 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.634+0000 7f03bf3c4740 1 -- 172.21.15.107:0/3769500999 shutdown_connections 2024-03-08T21:11:30.613 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.634+0000 7f03bf3c4740 1 --2- 172.21.15.107:0/3769500999 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55a0d0d58570 0x55a0d0d5ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:11:30.613 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.634+0000 7f03bf3c4740 1 --2- 172.21.15.107:0/3769500999 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55a0d0dc0430 0x55a0d0dc2840 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:11:30.613 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.634+0000 7f03bf3c4740 1 --2- 172.21.15.107:0/3769500999 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55a0d0d4f7a0 0x55a0d0d5e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:11:30.613 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.634+0000 7f03bf3c4740 1 --2- 172.21.15.107:0/3769500999 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a0d0d4ee80 0x55a0d0d5e170 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:11:30.613 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.634+0000 7f03bf3c4740 1 -- 172.21.15.107:0/3769500999 >> 172.21.15.107:0/3769500999 conn(0x55a0d0d497a0 msgr2=0x55a0d0d48800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:11:30.613 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.634+0000 7f039b400640 0 client.16377 ms_handle_reset on v2:172.21.15.149:6838/55290596 2024-03-08T21:11:30.614 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.634+0000 7f03bf3c4740 1 -- 172.21.15.107:0/3769500999 shutdown_connections 2024-03-08T21:11:30.614 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.634+0000 7f03bf3c4740 1 -- 172.21.15.107:0/3769500999 wait complete. 2024-03-08T21:11:30.614 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.635+0000 7f03bf3c4740 20 client.16377 trim_cache size 0 max 16384 2024-03-08T21:11:30.614 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:{ 2024-03-08T21:11:30.614 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "message": "", 2024-03-08T21:11:30.614 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "return_code": 0 2024-03-08T21:11:30.614 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:} 2024-03-08T21:11:30.615 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.635+0000 7f03bb000640 1 -- 172.21.15.107:0/246394846 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f039c0a2960 msgr2=0x7f039c0a4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:11:30.615 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.635+0000 7f03bb000640 1 --2- 172.21.15.107:0/246394846 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f039c0a2960 0x7f039c0a4e20 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f03ac009bf0 tx=0x7f03ac0093a0 comp rx=0 tx=0).stop 2024-03-08T21:11:30.615 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.635+0000 7f03bb000640 1 -- 172.21.15.107:0/246394846 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f03b4100860 msgr2=0x7f03b419fcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:11:30.615 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.635+0000 7f03bb000640 1 --2- 172.21.15.107:0/246394846 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f03b4100860 0x7f03b419fcf0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f03a000b4a0 tx=0x7f03a000b960 comp rx=0 tx=0).stop 2024-03-08T21:11:30.615 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.636+0000 7f03bb000640 1 -- 172.21.15.107:0/246394846 shutdown_connections 2024-03-08T21:11:30.615 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.636+0000 7f03bb000640 1 --2- 172.21.15.107:0/246394846 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f039c0a2960 0x7f039c0a4e20 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:11:30.615 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.636+0000 7f03bb000640 1 --2- 172.21.15.107:0/246394846 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f03b410da60 0x7f03b4199960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:11:30.615 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.636+0000 7f03bb000640 1 --2- 172.21.15.107:0/246394846 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f03b4100860 0x7f03b419fcf0 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:11:30.615 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.636+0000 7f03bb000640 1 --2- 172.21.15.107:0/246394846 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f03b40ffeb0 0x7f03b419f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:11:30.615 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.636+0000 7f03bb000640 1 -- 172.21.15.107:0/246394846 >> 172.21.15.107:0/246394846 conn(0x7f03b40fbb00 msgr2=0x7f03b40fdef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:11:30.616 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.636+0000 7f03bb000640 1 -- 172.21.15.107:0/246394846 shutdown_connections 2024-03-08T21:11:30.616 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:30.636+0000 7f03bb000640 1 -- 172.21.15.107:0/246394846 wait complete. 2024-03-08T21:11:31.962 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:11:31 smithi107 ceph-mon[31381]: pgmap v347: 97 pgs: 97 active+clean; 7.4 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 8 op/s 2024-03-08T21:11:32.123 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:11:31 smithi204 ceph-mon[34935]: pgmap v347: 97 pgs: 97 active+clean; 7.4 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 8 op/s 2024-03-08T21:11:32.259 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:11:31 smithi149 ceph-mon[35042]: pgmap v347: 97 pgs: 97 active+clean; 7.4 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 8 op/s 2024-03-08T21:11:34.123 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:11:33 smithi204 ceph-mon[34935]: pgmap v348: 97 pgs: 97 active+clean; 7.4 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 8 op/s 2024-03-08T21:11:34.212 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:11:33 smithi107 ceph-mon[31381]: pgmap v348: 97 pgs: 97 active+clean; 7.4 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 8 op/s 2024-03-08T21:11:34.259 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:11:33 smithi149 ceph-mon[35042]: pgmap v348: 97 pgs: 97 active+clean; 7.4 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 8 op/s 2024-03-08T21:11:36.123 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:11:35 smithi204 ceph-mon[34935]: pgmap v349: 97 pgs: 97 active+clean; 7.4 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 255 B/s wr, 13 op/s 2024-03-08T21:11:36.212 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:11:35 smithi107 ceph-mon[31381]: pgmap v349: 97 pgs: 97 active+clean; 7.4 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 255 B/s wr, 13 op/s 2024-03-08T21:11:36.259 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:11:35 smithi149 ceph-mon[35042]: pgmap v349: 97 pgs: 97 active+clean; 7.4 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 255 B/s wr, 13 op/s 2024-03-08T21:11:38.123 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:11:37 smithi204 ceph-mon[34935]: pgmap v350: 97 pgs: 97 active+clean; 7.4 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:11:38.212 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:11:37 smithi107 ceph-mon[31381]: pgmap v350: 97 pgs: 97 active+clean; 7.4 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:11:38.259 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:11:37 smithi149 ceph-mon[35042]: pgmap v350: 97 pgs: 97 active+clean; 7.4 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:11:40.123 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:11:39 smithi204 ceph-mon[34935]: pgmap v351: 97 pgs: 97 active+clean; 7.4 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-08T21:11:40.212 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:11:39 smithi107 ceph-mon[31381]: pgmap v351: 97 pgs: 97 active+clean; 7.4 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-08T21:11:40.259 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:11:39 smithi149 ceph-mon[35042]: pgmap v351: 97 pgs: 97 active+clean; 7.4 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-08T21:11:42.123 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:11:41 smithi204 ceph-mon[34935]: pgmap v352: 97 pgs: 97 active+clean; 7.4 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:11:42.212 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:11:41 smithi107 ceph-mon[31381]: pgmap v352: 97 pgs: 97 active+clean; 7.4 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:11:42.259 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:11:41 smithi149 ceph-mon[35042]: pgmap v352: 97 pgs: 97 active+clean; 7.4 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:11:43.762 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.783+0000 7f3cb6c00640 1 -- 172.21.15.107:0/3597674495 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3cb0069a90 msgr2=0x7f3cb0069ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:11:43.762 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.783+0000 7f3cb6c00640 1 --2- 172.21.15.107:0/3597674495 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3cb0069a90 0x7f3cb0069ed0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f3c9c009930 tx=0x7f3c9c02bc30 comp rx=0 tx=0).stop 2024-03-08T21:11:43.762 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.783+0000 7f3cb6c00640 1 -- 172.21.15.107:0/3597674495 shutdown_connections 2024-03-08T21:11:43.762 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.783+0000 7f3cb6c00640 1 --2- 172.21.15.107:0/3597674495 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3cb006a410 0x7f3cb010e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:11:43.762 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.783+0000 7f3cb6c00640 1 --2- 172.21.15.107:0/3597674495 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3cb0069a90 0x7f3cb0069ed0 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:11:43.762 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.783+0000 7f3cb6c00640 1 --2- 172.21.15.107:0/3597674495 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3cb0107360 0x7f3cb0107740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:11:43.763 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.783+0000 7f3cb6c00640 1 -- 172.21.15.107:0/3597674495 >> 172.21.15.107:0/3597674495 conn(0x7f3cb00771d0 msgr2=0x7f3cb00775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:11:43.763 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.783+0000 7f3cb6c00640 1 -- 172.21.15.107:0/3597674495 shutdown_connections 2024-03-08T21:11:43.763 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.783+0000 7f3cb6c00640 1 -- 172.21.15.107:0/3597674495 wait complete. 2024-03-08T21:11:43.764 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.784+0000 7f3cb6c00640 1 Processor -- start 2024-03-08T21:11:43.764 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.785+0000 7f3cb6c00640 1 -- start start 2024-03-08T21:11:43.765 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.785+0000 7f3cb6c00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3cb0069a90 0x7f3cb01005a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:11:43.765 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.785+0000 7f3cb6c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3cb006a410 0x7f3cb0100ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:11:43.765 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.785+0000 7f3cb6c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3cb0107360 0x7f3cb0101020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:11:43.765 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.785+0000 7f3cb6c00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cb0111530 con 0x7f3cb006a410 2024-03-08T21:11:43.765 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.785+0000 7f3cb6c00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cb01113b0 con 0x7f3cb0069a90 2024-03-08T21:11:43.765 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.785+0000 7f3cb6c00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cb01116b0 con 0x7f3cb0107360 2024-03-08T21:11:43.765 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.785+0000 7f3caf400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3cb006a410 0x7f3cb0100ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:11:43.765 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.786+0000 7f3caf400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3cb006a410 0x7f3cb0100ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:44450/0 (socket says 172.21.15.107:44450) 2024-03-08T21:11:43.765 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.786+0000 7f3caf400640 1 -- 172.21.15.107:0/453267869 learned_addr learned my addr 172.21.15.107:0/453267869 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:11:43.765 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.786+0000 7f3cafe00640 1 --2- 172.21.15.107:0/453267869 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3cb0069a90 0x7f3cb01005a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:11:43.765 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.786+0000 7f3cb4c00640 1 --2- 172.21.15.107:0/453267869 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3cb0107360 0x7f3cb0101020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:11:43.765 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.786+0000 7f3caf400640 1 -- 172.21.15.107:0/453267869 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3cb0107360 msgr2=0x7f3cb0101020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:11:43.765 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.786+0000 7f3caf400640 1 --2- 172.21.15.107:0/453267869 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3cb0107360 0x7f3cb0101020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:11:43.765 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.786+0000 7f3caf400640 1 -- 172.21.15.107:0/453267869 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3cb0069a90 msgr2=0x7f3cb01005a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:11:43.765 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.786+0000 7f3caf400640 1 --2- 172.21.15.107:0/453267869 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3cb0069a90 0x7f3cb01005a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:11:43.766 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.786+0000 7f3caf400640 1 -- 172.21.15.107:0/453267869 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cb01a5f30 con 0x7f3cb006a410 2024-03-08T21:11:43.766 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.786+0000 7f3caf400640 1 --2- 172.21.15.107:0/453267869 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3cb006a410 0x7f3cb0100ae0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f3c9c009230 tx=0x7f3c9c0045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:11:43.766 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.787+0000 7f3cacc00640 1 -- 172.21.15.107:0/453267869 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c9c043020 con 0x7f3cb006a410 2024-03-08T21:11:43.766 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.787+0000 7f3cb6c00640 1 -- 172.21.15.107:0/453267869 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3cb01a61c0 con 0x7f3cb006a410 2024-03-08T21:11:43.767 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.787+0000 7f3cacc00640 1 -- 172.21.15.107:0/453267869 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f3c9c02cb70 con 0x7f3cb006a410 2024-03-08T21:11:43.767 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.787+0000 7f3cacc00640 1 -- 172.21.15.107:0/453267869 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c9c003b50 con 0x7f3cb006a410 2024-03-08T21:11:43.767 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.787+0000 7f3cb6c00640 1 -- 172.21.15.107:0/453267869 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3cb01a6710 con 0x7f3cb006a410 2024-03-08T21:11:43.769 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.791+0000 7f3cb9836740 1 Processor -- start 2024-03-08T21:11:43.769 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.791+0000 7f3cb9836740 1 -- start start 2024-03-08T21:11:43.771 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.791+0000 7f3cb9836740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55678995dc80 0x55678995e060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:11:43.771 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.791+0000 7f3cacc00640 1 -- 172.21.15.107:0/453267869 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) v1 ==== 137422+0+0 (secure 0 0 0) 0x7f3c9c046020 con 0x7f3cb006a410 2024-03-08T21:11:43.771 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.791+0000 7f3cb9836740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55678995e5a0 0x556789966e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:11:43.771 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.791+0000 7f3cb9836740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556789967370 0x556789969760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:11:43.771 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.791+0000 7f3cb9836740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567898c13d0 con 0x55678995e5a0 2024-03-08T21:11:43.772 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.791+0000 7f3cb9836740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55678986a6b0 con 0x55678995dc80 2024-03-08T21:11:43.772 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.791+0000 7f3cb9836740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567898bd470 con 0x556789967370 2024-03-08T21:11:43.772 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.792+0000 7f3cafe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55678995e5a0 0x556789966e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:11:43.772 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.792+0000 7f3cafe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55678995e5a0 0x556789966e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:44464/0 (socket says 172.21.15.107:44464) 2024-03-08T21:11:43.772 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.792+0000 7f3cafe00640 1 -- 172.21.15.107:0/2467482955 learned_addr learned my addr 172.21.15.107:0/2467482955 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:11:43.772 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.792+0000 7f3cb4c00640 1 --2- 172.21.15.107:0/2467482955 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55678995dc80 0x55678995e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:11:43.772 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.792+0000 7f3caf400640 1 --2- 172.21.15.107:0/2467482955 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556789967370 0x556789969760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:11:43.772 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.792+0000 7f3cafe00640 1 -- 172.21.15.107:0/2467482955 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556789967370 msgr2=0x556789969760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:11:43.772 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.792+0000 7f3cafe00640 1 --2- 172.21.15.107:0/2467482955 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556789967370 0x556789969760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:11:43.772 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.792+0000 7f3cafe00640 1 -- 172.21.15.107:0/2467482955 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55678995dc80 msgr2=0x55678995e060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:11:43.772 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.792+0000 7f3cafe00640 1 --2- 172.21.15.107:0/2467482955 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55678995dc80 0x55678995e060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:11:43.773 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.792+0000 7f3cafe00640 1 -- 172.21.15.107:0/2467482955 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556789890b50 con 0x55678995e5a0 2024-03-08T21:11:43.773 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.792+0000 7f3cb4c00640 1 --2- 172.21.15.107:0/2467482955 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55678995dc80 0x55678995e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T21:11:43.773 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.793+0000 7f3cafe00640 1 --2- 172.21.15.107:0/2467482955 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55678995e5a0 0x556789966e30 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f3ca000bc60 tx=0x7f3ca000f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5501a33168fcde9a server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:11:43.773 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.793+0000 7f3ca5200640 1 -- 172.21.15.107:0/2467482955 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ca0015070 con 0x55678995e5a0 2024-03-08T21:11:43.773 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.793+0000 7f3ca5200640 1 -- 172.21.15.107:0/2467482955 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f3ca00118f0 con 0x55678995e5a0 2024-03-08T21:11:43.773 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.793+0000 7f3ca5200640 1 -- 172.21.15.107:0/2467482955 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ca0010810 con 0x55678995e5a0 2024-03-08T21:11:43.773 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.793+0000 7f3cacc00640 1 --2- 172.21.15.107:0/453267869 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f3c940a2a10 0x7f3c940a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:11:43.778 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.793+0000 7f3cacc00640 1 -- 172.21.15.107:0/453267869 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f3c9c0f1070 con 0x7f3cb006a410 2024-03-08T21:11:43.778 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.794+0000 7f3cb4c00640 1 --2- 172.21.15.107:0/453267869 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f3c940a2a10 0x7f3c940a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:11:43.778 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.794+0000 7f3cb9836740 1 -- 172.21.15.107:0/2467482955 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55678995e5a0 msgr2=0x556789966e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:11:43.778 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.794+0000 7f3cb9836740 1 --2- 172.21.15.107:0/2467482955 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55678995e5a0 0x556789966e30 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f3ca000bc60 tx=0x7f3ca000f730 comp rx=0 tx=0).stop 2024-03-08T21:11:43.778 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.794+0000 7f3cb9836740 1 -- 172.21.15.107:0/2467482955 shutdown_connections 2024-03-08T21:11:43.778 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.794+0000 7f3cb9836740 1 --2- 172.21.15.107:0/2467482955 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556789967370 0x556789969760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:11:43.779 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.794+0000 7f3cb9836740 1 --2- 172.21.15.107:0/2467482955 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55678995e5a0 0x556789966e30 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:11:43.779 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.794+0000 7f3cb9836740 1 --2- 172.21.15.107:0/2467482955 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55678995dc80 0x55678995e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:11:43.779 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.794+0000 7f3cb9836740 1 -- 172.21.15.107:0/2467482955 >> 172.21.15.107:0/2467482955 conn(0x5567899585a0 msgr2=0x556789958980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:11:43.779 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.794+0000 7f3cb4c00640 1 --2- 172.21.15.107:0/453267869 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f3c940a2a10 0x7f3c940a4ed0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f3ca8005fd0 tx=0x7f3ca80094e0 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:11:43.779 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.794+0000 7f3cb9836740 1 -- 172.21.15.107:0/2467482955 shutdown_connections 2024-03-08T21:11:43.779 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.795+0000 7f3cb9836740 1 -- 172.21.15.107:0/2467482955 wait complete. 2024-03-08T21:11:43.779 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.795+0000 7f3cb9836740 1 -- start start 2024-03-08T21:11:43.779 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.795+0000 7f3cb9836740 1 Processor -- start 2024-03-08T21:11:43.779 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.796+0000 7f3cb9836740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55678995dc80 0x556789966cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:11:43.779 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.796+0000 7f3cb9836740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55678995e5a0 0x556789964060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:11:43.779 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.796+0000 7f3cb9836740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556789967370 0x5567899645c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:11:43.779 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.796+0000 7f3cb9836740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x556789969fa0 con 0x55678995e5a0 2024-03-08T21:11:43.780 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.796+0000 7f3cb9836740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55678987d9f0 con 0x556789967370 2024-03-08T21:11:43.780 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.796+0000 7f3cb9836740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55678996a120 con 0x55678995dc80 2024-03-08T21:11:43.780 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.796+0000 7f3cb9836740 10 client.0 _ll_register_callbacks cb 0x5567895d5a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T21:11:43.780 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.796+0000 7f3caf400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55678995e5a0 0x556789964060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:11:43.780 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.796+0000 7f3caf400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55678995e5a0 0x556789964060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:44480/0 (socket says 172.21.15.107:44480) 2024-03-08T21:11:43.780 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.796+0000 7f3caf400640 1 -- 172.21.15.107:0/4153857058 learned_addr learned my addr 172.21.15.107:0/4153857058 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:11:43.780 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.796+0000 7f3cafe00640 1 --2- 172.21.15.107:0/4153857058 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55678995dc80 0x556789966cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:11:43.780 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.796+0000 7f3cb4c00640 1 --2- 172.21.15.107:0/4153857058 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556789967370 0x5567899645c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:11:43.780 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.796+0000 7f3caf400640 1 -- 172.21.15.107:0/4153857058 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55678995dc80 msgr2=0x556789966cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:11:43.780 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.796+0000 7f3caf400640 1 --2- 172.21.15.107:0/4153857058 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55678995dc80 0x556789966cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:11:43.780 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.796+0000 7f3caf400640 1 -- 172.21.15.107:0/4153857058 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556789967370 msgr2=0x5567899645c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:11:43.781 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.796+0000 7f3caf400640 1 --2- 172.21.15.107:0/4153857058 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556789967370 0x5567899645c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:11:43.781 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.796+0000 7f3caf400640 1 -- 172.21.15.107:0/4153857058 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5567895d6210 con 0x55678995e5a0 2024-03-08T21:11:43.781 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.796+0000 7f3cb4c00640 1 --2- 172.21.15.107:0/4153857058 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556789967370 0x5567899645c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T21:11:43.781 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.797+0000 7f3caf400640 1 --2- 172.21.15.107:0/4153857058 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55678995e5a0 0x556789964060 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f3c9c0ee8f0 tx=0x7f3c9c009620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec796223f8528f1b server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:11:43.781 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.797+0000 7f3c99600640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-03-08T21:11:43.781 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.797+0000 7f3c99600640 1 -- 172.21.15.107:0/4153857058 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c9c0bd100 con 0x55678995e5a0 2024-03-08T21:11:43.781 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.797+0000 7f3c99600640 1 -- 172.21.15.107:0/4153857058 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f3c9c0b9e10 con 0x55678995e5a0 2024-03-08T21:11:43.781 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.797+0000 7f3cb9836740 1 -- 172.21.15.107:0/4153857058 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3ca00118f0 con 0x55678995e5a0 2024-03-08T21:11:43.781 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.797+0000 7f3c99600640 1 -- 172.21.15.107:0/4153857058 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3c9c0bdea0 con 0x55678995e5a0 2024-03-08T21:11:43.781 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.797+0000 7f3cb9836740 10 client.16389 fetch_fsmap learned FSMap version 39 2024-03-08T21:11:43.781 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.797+0000 7f3cb9836740 1 -- 172.21.15.107:0/4153857058 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x5567896a2620 con 0x55678995e5a0 2024-03-08T21:11:43.782 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.797+0000 7f3c99600640 1 -- 172.21.15.107:0/4153857058 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f3ca00118f0 con 0x55678995e5a0 2024-03-08T21:11:43.782 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.798+0000 7f3c99600640 1 -- 172.21.15.107:0/4153857058 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 39) v1 ==== 7307+0+0 (secure 0 0 0) 0x7f3c9c0a6ce0 con 0x55678995e5a0 2024-03-08T21:11:43.782 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.798+0000 7f3cb9836740 10 client.16389 fetch_fsmap finished waiting for FSMap version 39 2024-03-08T21:11:43.782 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.798+0000 7f3cb9836740 10 client.16389 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-08T21:11:43.782 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.798+0000 7f3cb9836740 20 client.16389 populate_metadata read hostname 'smithi107' 2024-03-08T21:11:43.782 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.798+0000 7f3cb9836740 1 --2- 172.21.15.107:0/4153857058 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x5567899770d0 0x556789979500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:11:43.782 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.798+0000 7f3cb9836740 4 client.16389 mds_command: new command op to 34475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-08T21:11:43.782 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.798+0000 7f3cb9836740 1 -- 172.21.15.107:0/4153857058 --> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5567898ce910 con 0x5567899770d0 2024-03-08T21:11:43.782 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.798+0000 7f3cafe00640 1 --2- 172.21.15.107:0/4153857058 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x5567899770d0 0x556789979500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:11:43.782 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.799+0000 7f3cafe00640 1 --2- 172.21.15.107:0/4153857058 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x5567899770d0 0x556789979500 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6215a62d6e4cbcdc server_cookie=ca3916082ac9d63b in_seq=0 out_seq=0 2024-03-08T21:11:43.782 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.799+0000 7f3c99600640 10 client.16389 ms_handle_connect on v2:172.21.15.149:6838/55290596 2024-03-08T21:11:43.783 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.801+0000 7f3c99600640 1 -- 172.21.15.107:0/4153857058 <== mds.0 v2:172.21.15.149:6838/55290596 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5567898ce910 con 0x5567899770d0 2024-03-08T21:11:43.784 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.801+0000 7f3c99600640 10 client.16389 handle_command_reply: tid=0 2024-03-08T21:11:43.784 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.801+0000 7f3cb9836740 1 client.16389 shutdown 2024-03-08T21:11:43.784 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.801+0000 7f3cb9836740 1 -- 172.21.15.107:0/4153857058 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55678995e5a0 msgr2=0x556789964060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:11:43.784 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.801+0000 7f3cb9836740 1 --2- 172.21.15.107:0/4153857058 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55678995e5a0 0x556789964060 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f3c9c0ee8f0 tx=0x7f3c9c009620 comp rx=0 tx=0).stop 2024-03-08T21:11:43.784 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.801+0000 7f3cb9836740 1 -- 172.21.15.107:0/4153857058 shutdown_connections 2024-03-08T21:11:43.784 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.801+0000 7f3cb9836740 1 --2- 172.21.15.107:0/4153857058 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x5567899770d0 0x556789979500 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:11:43.784 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.801+0000 7f3cb9836740 1 --2- 172.21.15.107:0/4153857058 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x556789967370 0x5567899645c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:11:43.784 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.801+0000 7f3cb9836740 1 --2- 172.21.15.107:0/4153857058 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55678995e5a0 0x556789964060 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:11:43.784 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.801+0000 7f3c99600640 0 client.16389 ms_handle_reset on v2:172.21.15.149:6838/55290596 2024-03-08T21:11:43.785 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.801+0000 7f3cb9836740 1 --2- 172.21.15.107:0/4153857058 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55678995dc80 0x556789966cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:11:43.785 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.801+0000 7f3cb9836740 1 -- 172.21.15.107:0/4153857058 >> 172.21.15.107:0/4153857058 conn(0x5567899585a0 msgr2=0x55678995a2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:11:43.785 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.801+0000 7f3cb9836740 1 -- 172.21.15.107:0/4153857058 shutdown_connections 2024-03-08T21:11:43.785 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.802+0000 7f3cb9836740 1 -- 172.21.15.107:0/4153857058 wait complete. 2024-03-08T21:11:43.785 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.802+0000 7f3cb9836740 20 client.16389 trim_cache size 0 max 16384 2024-03-08T21:11:43.801 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.822+0000 7f3cb9836740 1 Processor -- start 2024-03-08T21:11:43.802 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.822+0000 7f3cb9836740 1 -- start start 2024-03-08T21:11:43.802 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.822+0000 7f3cb9836740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55678995dc80 0x556789957540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:11:43.802 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.822+0000 7f3cb9836740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55678995e5a0 0x556789966790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:11:43.802 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.822+0000 7f3cb9836740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556789967370 0x556789966cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:11:43.802 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.822+0000 7f3cb9836740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cb01b0750 con 0x55678995e5a0 2024-03-08T21:11:43.802 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.822+0000 7f3cb9836740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cb01ae770 con 0x55678995dc80 2024-03-08T21:11:43.802 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.822+0000 7f3cb9836740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cb01b00f0 con 0x556789967370 2024-03-08T21:11:43.802 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.822+0000 7f3caf400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55678995e5a0 0x556789966790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:11:43.802 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.822+0000 7f3caf400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55678995e5a0 0x556789966790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:44494/0 (socket says 172.21.15.107:44494) 2024-03-08T21:11:43.802 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.822+0000 7f3caf400640 1 -- 172.21.15.107:0/1229472340 learned_addr learned my addr 172.21.15.107:0/1229472340 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:11:43.802 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.823+0000 7f3cafe00640 1 --2- 172.21.15.107:0/1229472340 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55678995dc80 0x556789957540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:11:43.803 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.823+0000 7f3cb4c00640 1 --2- 172.21.15.107:0/1229472340 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556789967370 0x556789966cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:11:43.803 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.823+0000 7f3caf400640 1 -- 172.21.15.107:0/1229472340 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556789967370 msgr2=0x556789966cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:11:43.803 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.823+0000 7f3caf400640 1 --2- 172.21.15.107:0/1229472340 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556789967370 0x556789966cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:11:43.803 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.823+0000 7f3caf400640 1 -- 172.21.15.107:0/1229472340 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55678995dc80 msgr2=0x556789957540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:11:43.803 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.823+0000 7f3caf400640 1 --2- 172.21.15.107:0/1229472340 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55678995dc80 0x556789957540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:11:43.803 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.823+0000 7f3caf400640 1 -- 172.21.15.107:0/1229472340 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55678995e3d0 con 0x55678995e5a0 2024-03-08T21:11:43.803 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.823+0000 7f3cb4c00640 1 --2- 172.21.15.107:0/1229472340 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556789967370 0x556789966cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T21:11:43.803 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.823+0000 7f3caf400640 1 --2- 172.21.15.107:0/1229472340 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55678995e5a0 0x556789966790 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f3c9c0b9c00 tx=0x7f3c9c005cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e72c3ebef6e99f79 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:11:43.803 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.823+0000 7f3c9be00640 1 -- 172.21.15.107:0/1229472340 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c9c0bd100 con 0x55678995e5a0 2024-03-08T21:11:43.803 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.824+0000 7f3c9be00640 1 -- 172.21.15.107:0/1229472340 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f3c9c005500 con 0x55678995e5a0 2024-03-08T21:11:43.803 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.824+0000 7f3c9be00640 1 -- 172.21.15.107:0/1229472340 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c9c0c5ab0 con 0x55678995e5a0 2024-03-08T21:11:43.807 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.824+0000 7f3cb9836740 1 -- 172.21.15.107:0/1229472340 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55678995e5a0 msgr2=0x556789966790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:11:43.807 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.824+0000 7f3cb9836740 1 --2- 172.21.15.107:0/1229472340 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55678995e5a0 0x556789966790 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f3c9c0b9c00 tx=0x7f3c9c005cb0 comp rx=0 tx=0).stop 2024-03-08T21:11:43.807 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.824+0000 7f3cb9836740 1 -- 172.21.15.107:0/1229472340 shutdown_connections 2024-03-08T21:11:43.807 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.824+0000 7f3cb9836740 1 --2- 172.21.15.107:0/1229472340 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556789967370 0x556789966cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:11:43.807 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.824+0000 7f3cb9836740 1 --2- 172.21.15.107:0/1229472340 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55678995e5a0 0x556789966790 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:11:43.807 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.824+0000 7f3cb9836740 1 --2- 172.21.15.107:0/1229472340 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55678995dc80 0x556789957540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:11:43.807 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.824+0000 7f3cb9836740 1 -- 172.21.15.107:0/1229472340 >> 172.21.15.107:0/1229472340 conn(0x5567899585a0 msgr2=0x556789956e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:11:43.808 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.824+0000 7f3cb9836740 1 -- 172.21.15.107:0/1229472340 shutdown_connections 2024-03-08T21:11:43.808 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.825+0000 7f3cb9836740 1 -- 172.21.15.107:0/1229472340 wait complete. 2024-03-08T21:11:43.808 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.825+0000 7f3cb9836740 1 -- start start 2024-03-08T21:11:43.808 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.825+0000 7f3cb9836740 1 Processor -- start 2024-03-08T21:11:43.808 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.825+0000 7f3cb9836740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55678995dc80 0x55678996d190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:11:43.808 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.826+0000 7f3cb9836740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55678995e5a0 0x55678996d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:11:43.808 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.826+0000 7f3cb9836740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556789967370 0x5567899cd630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:11:43.808 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.826+0000 7f3cb9836740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cb01af430 con 0x55678995e5a0 2024-03-08T21:11:43.808 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.826+0000 7f3cb9836740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cb01aedd0 con 0x55678995dc80 2024-03-08T21:11:43.809 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.826+0000 7f3cb9836740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cb01afa90 con 0x556789967370 2024-03-08T21:11:43.809 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.826+0000 7f3cb9836740 10 client.0 _ll_register_callbacks cb 0x7f3ca000b420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T21:11:43.809 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.826+0000 7f3caf400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55678995e5a0 0x55678996d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:11:43.809 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.826+0000 7f3caf400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55678995e5a0 0x55678996d6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:44506/0 (socket says 172.21.15.107:44506) 2024-03-08T21:11:43.809 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.826+0000 7f3caf400640 1 -- 172.21.15.107:0/84886199 learned_addr learned my addr 172.21.15.107:0/84886199 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:11:43.809 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.826+0000 7f3cafe00640 1 --2- 172.21.15.107:0/84886199 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55678995dc80 0x55678996d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:11:43.809 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.826+0000 7f3cb4c00640 1 --2- 172.21.15.107:0/84886199 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556789967370 0x5567899cd630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:11:43.809 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.826+0000 7f3caf400640 1 -- 172.21.15.107:0/84886199 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556789967370 msgr2=0x5567899cd630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:11:43.809 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.826+0000 7f3caf400640 1 --2- 172.21.15.107:0/84886199 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556789967370 0x5567899cd630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:11:43.809 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.826+0000 7f3caf400640 1 -- 172.21.15.107:0/84886199 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55678995dc80 msgr2=0x55678996d190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:11:43.809 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.826+0000 7f3caf400640 1 --2- 172.21.15.107:0/84886199 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55678995dc80 0x55678996d190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:11:43.810 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.826+0000 7f3caf400640 1 -- 172.21.15.107:0/84886199 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5567899cdb70 con 0x55678995e5a0 2024-03-08T21:11:43.810 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.826+0000 7f3cafe00640 1 --2- 172.21.15.107:0/84886199 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55678995dc80 0x55678996d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T21:11:43.810 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.827+0000 7f3caf400640 1 --2- 172.21.15.107:0/84886199 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55678995e5a0 0x55678996d6d0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f3c9c0bd730 tx=0x7f3c9c0eeb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ebf4de8c82c2823 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:11:43.810 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.827+0000 7f3c99600640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-03-08T21:11:43.810 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.827+0000 7f3c99600640 1 -- 172.21.15.107:0/84886199 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c9c0bd100 con 0x55678995e5a0 2024-03-08T21:11:43.810 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.827+0000 7f3c99600640 1 -- 172.21.15.107:0/84886199 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f3c9c0d1b80 con 0x55678995e5a0 2024-03-08T21:11:43.810 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.827+0000 7f3cb9836740 1 -- 172.21.15.107:0/84886199 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3c9c005500 con 0x55678995e5a0 2024-03-08T21:11:43.810 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.827+0000 7f3c99600640 1 -- 172.21.15.107:0/84886199 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3c9c0bd7a0 con 0x55678995e5a0 2024-03-08T21:11:43.810 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.827+0000 7f3cb9836740 10 client.16395 fetch_fsmap learned FSMap version 39 2024-03-08T21:11:43.810 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.827+0000 7f3cb9836740 1 -- 172.21.15.107:0/84886199 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x5567899cdb70 con 0x55678995e5a0 2024-03-08T21:11:43.811 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.828+0000 7f3c99600640 1 -- 172.21.15.107:0/84886199 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f3c9c005500 con 0x55678995e5a0 2024-03-08T21:11:43.811 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.828+0000 7f3c99600640 1 -- 172.21.15.107:0/84886199 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 39) v1 ==== 7307+0+0 (secure 0 0 0) 0x7f3c9c0a6ce0 con 0x55678995e5a0 2024-03-08T21:11:43.811 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.828+0000 7f3cb9836740 10 client.16395 fetch_fsmap finished waiting for FSMap version 39 2024-03-08T21:11:43.811 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.828+0000 7f3cb9836740 10 client.16395 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-08T21:11:43.811 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.828+0000 7f3cb9836740 20 client.16395 populate_metadata read hostname 'smithi107' 2024-03-08T21:11:43.811 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.828+0000 7f3cb9836740 1 --2- 172.21.15.107:0/84886199 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x5567899cefd0 0x5567899d13f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:11:43.811 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.828+0000 7f3cb9836740 4 client.16395 mds_command: new command op to 34475 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-08T21:11:43.811 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.828+0000 7f3cb9836740 1 -- 172.21.15.107:0/84886199 --> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5567898c8c60 con 0x5567899cefd0 2024-03-08T21:11:43.811 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.828+0000 7f3cafe00640 1 --2- 172.21.15.107:0/84886199 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x5567899cefd0 0x5567899d13f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:11:43.812 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.829+0000 7f3cafe00640 1 --2- 172.21.15.107:0/84886199 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x5567899cefd0 0x5567899d13f0 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d6134447bd8600e4 server_cookie=c14c6c3a7b7c8f2d in_seq=0 out_seq=0 2024-03-08T21:11:43.812 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.829+0000 7f3c99600640 10 client.16395 ms_handle_connect on v2:172.21.15.149:6838/55290596 2024-03-08T21:11:43.818 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.839+0000 7f3c99600640 1 -- 172.21.15.107:0/84886199 <== mds.0 v2:172.21.15.149:6838/55290596 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5567898c8c60 con 0x5567899cefd0 2024-03-08T21:11:43.818 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.839+0000 7f3c99600640 10 client.16395 handle_command_reply: tid=0 2024-03-08T21:11:43.818 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.839+0000 7f3cb9836740 1 client.16395 shutdown 2024-03-08T21:11:43.818 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.839+0000 7f3cb9836740 1 -- 172.21.15.107:0/84886199 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55678995e5a0 msgr2=0x55678996d6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:11:43.819 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.839+0000 7f3cb9836740 1 --2- 172.21.15.107:0/84886199 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55678995e5a0 0x55678996d6d0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f3c9c0bd730 tx=0x7f3c9c0eeb70 comp rx=0 tx=0).stop 2024-03-08T21:11:43.819 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.839+0000 7f3cb9836740 1 -- 172.21.15.107:0/84886199 shutdown_connections 2024-03-08T21:11:43.819 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.839+0000 7f3cb9836740 1 --2- 172.21.15.107:0/84886199 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x556789967370 0x5567899cd630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:11:43.819 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.839+0000 7f3cb9836740 1 --2- 172.21.15.107:0/84886199 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55678995e5a0 0x55678996d6d0 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:11:43.819 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.839+0000 7f3cb9836740 1 --2- 172.21.15.107:0/84886199 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x5567899cefd0 0x5567899d13f0 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:11:43.819 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.839+0000 7f3cb9836740 1 --2- 172.21.15.107:0/84886199 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55678995dc80 0x55678996d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:11:43.819 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.839+0000 7f3cb9836740 1 -- 172.21.15.107:0/84886199 >> 172.21.15.107:0/84886199 conn(0x5567899585a0 msgr2=0x5567899576c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:11:43.819 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.839+0000 7f3c99600640 0 client.16395 ms_handle_reset on v2:172.21.15.149:6838/55290596 2024-03-08T21:11:43.819 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.840+0000 7f3cb9836740 1 -- 172.21.15.107:0/84886199 shutdown_connections 2024-03-08T21:11:43.819 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.840+0000 7f3cb9836740 1 -- 172.21.15.107:0/84886199 wait complete. 2024-03-08T21:11:43.820 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.840+0000 7f3cb9836740 20 client.16395 trim_cache size 0 max 16384 2024-03-08T21:11:43.820 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:{ 2024-03-08T21:11:43.820 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "message": "", 2024-03-08T21:11:43.820 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "return_code": 0 2024-03-08T21:11:43.820 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:} 2024-03-08T21:11:43.821 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.840+0000 7f3cb6c00640 1 -- 172.21.15.107:0/453267869 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f3c940a2a10 msgr2=0x7f3c940a4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:11:43.821 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.840+0000 7f3cb6c00640 1 --2- 172.21.15.107:0/453267869 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f3c940a2a10 0x7f3c940a4ed0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f3ca8005fd0 tx=0x7f3ca80094e0 comp rx=0 tx=0).stop 2024-03-08T21:11:43.821 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.840+0000 7f3cb6c00640 1 -- 172.21.15.107:0/453267869 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3cb006a410 msgr2=0x7f3cb0100ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:11:43.821 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.840+0000 7f3cb6c00640 1 --2- 172.21.15.107:0/453267869 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3cb006a410 0x7f3cb0100ae0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f3c9c009230 tx=0x7f3c9c0045e0 comp rx=0 tx=0).stop 2024-03-08T21:11:43.821 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.841+0000 7f3cb6c00640 1 -- 172.21.15.107:0/453267869 shutdown_connections 2024-03-08T21:11:43.822 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.841+0000 7f3cb6c00640 1 --2- 172.21.15.107:0/453267869 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f3c940a2a10 0x7f3c940a4ed0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:11:43.822 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.841+0000 7f3cb6c00640 1 --2- 172.21.15.107:0/453267869 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3cb0107360 0x7f3cb0101020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:11:43.822 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.841+0000 7f3cb6c00640 1 --2- 172.21.15.107:0/453267869 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3cb006a410 0x7f3cb0100ae0 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:11:43.822 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.841+0000 7f3cb6c00640 1 --2- 172.21.15.107:0/453267869 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3cb0069a90 0x7f3cb01005a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:11:43.822 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.841+0000 7f3cb6c00640 1 -- 172.21.15.107:0/453267869 >> 172.21.15.107:0/453267869 conn(0x7f3cb00771d0 msgr2=0x7f3cb00ff1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:11:43.822 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.841+0000 7f3cb6c00640 1 -- 172.21.15.107:0/453267869 shutdown_connections 2024-03-08T21:11:43.822 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:43.841+0000 7f3cb6c00640 1 -- 172.21.15.107:0/453267869 wait complete. 2024-03-08T21:11:44.123 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:11:43 smithi204 ceph-mon[34935]: pgmap v353: 97 pgs: 97 active+clean; 7.4 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:11:44.212 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:11:43 smithi107 ceph-mon[31381]: pgmap v353: 97 pgs: 97 active+clean; 7.4 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:11:44.259 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:11:43 smithi149 ceph-mon[35042]: pgmap v353: 97 pgs: 97 active+clean; 7.4 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:11:46.123 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:11:45 smithi204 ceph-mon[34935]: pgmap v354: 97 pgs: 97 active+clean; 7.4 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-08T21:11:46.212 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:11:45 smithi107 ceph-mon[31381]: pgmap v354: 97 pgs: 97 active+clean; 7.4 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-08T21:11:46.259 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:11:45 smithi149 ceph-mon[35042]: pgmap v354: 97 pgs: 97 active+clean; 7.4 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-08T21:11:48.123 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:11:47 smithi204 ceph-mon[34935]: pgmap v355: 97 pgs: 97 active+clean; 7.4 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-08T21:11:48.212 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:11:47 smithi107 ceph-mon[31381]: pgmap v355: 97 pgs: 97 active+clean; 7.4 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-08T21:11:48.259 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:11:47 smithi149 ceph-mon[35042]: pgmap v355: 97 pgs: 97 active+clean; 7.4 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-08T21:11:50.123 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:11:49 smithi204 ceph-mon[34935]: pgmap v356: 97 pgs: 97 active+clean; 7.4 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-08T21:11:50.212 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:11:49 smithi107 ceph-mon[31381]: pgmap v356: 97 pgs: 97 active+clean; 7.4 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-08T21:11:50.259 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:11:49 smithi149 ceph-mon[35042]: pgmap v356: 97 pgs: 97 active+clean; 7.4 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-08T21:11:52.212 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:11:51 smithi107 ceph-mon[31381]: pgmap v357: 97 pgs: 97 active+clean; 7.4 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:11:52.259 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:11:51 smithi149 ceph-mon[35042]: pgmap v357: 97 pgs: 97 active+clean; 7.4 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:11:52.373 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:11:51 smithi204 ceph-mon[34935]: pgmap v357: 97 pgs: 97 active+clean; 7.4 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:11:54.212 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:11:53 smithi107 ceph-mon[31381]: pgmap v358: 97 pgs: 97 active+clean; 7.4 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:11:54.259 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:11:53 smithi149 ceph-mon[35042]: pgmap v358: 97 pgs: 97 active+clean; 7.4 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:11:54.373 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:11:53 smithi204 ceph-mon[34935]: pgmap v358: 97 pgs: 97 active+clean; 7.4 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:11:56.212 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:11:55 smithi107 ceph-mon[31381]: pgmap v359: 97 pgs: 97 active+clean; 7.4 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-08T21:11:56.259 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:11:55 smithi149 ceph-mon[35042]: pgmap v359: 97 pgs: 97 active+clean; 7.4 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-08T21:11:56.373 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:11:55 smithi204 ceph-mon[34935]: pgmap v359: 97 pgs: 97 active+clean; 7.4 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-08T21:11:56.975 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:56.997+0000 7f06abe00640 1 -- 172.21.15.107:0/1015479287 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f06ac103f00 msgr2=0x7f06ac107f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:11:56.975 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:56.997+0000 7f06abe00640 1 --2- 172.21.15.107:0/1015479287 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f06ac103f00 0x7f06ac107f40 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f0698009900 tx=0x7f069802bc30 comp rx=0 tx=0).stop 2024-03-08T21:11:56.975 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:56.997+0000 7f06abe00640 1 -- 172.21.15.107:0/1015479287 shutdown_connections 2024-03-08T21:11:56.975 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:56.997+0000 7f06abe00640 1 --2- 172.21.15.107:0/1015479287 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f06ac108620 0x7f06ac1101f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:11:56.975 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:56.997+0000 7f06abe00640 1 --2- 172.21.15.107:0/1015479287 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f06ac103f00 0x7f06ac107f40 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:11:56.975 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:56.997+0000 7f06abe00640 1 --2- 172.21.15.107:0/1015479287 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f06ac103550 0x7f06ac103930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:11:56.975 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:56.997+0000 7f06abe00640 1 -- 172.21.15.107:0/1015479287 >> 172.21.15.107:0/1015479287 conn(0x7f06ac0fedd0 msgr2=0x7f06ac1011f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:11:56.975 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:56.997+0000 7f06abe00640 1 -- 172.21.15.107:0/1015479287 shutdown_connections 2024-03-08T21:11:56.975 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:56.997+0000 7f06abe00640 1 -- 172.21.15.107:0/1015479287 wait complete. 2024-03-08T21:11:56.976 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:56.999+0000 7f06abe00640 1 Processor -- start 2024-03-08T21:11:56.977 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:56.999+0000 7f06abe00640 1 -- start start 2024-03-08T21:11:56.977 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:56.999+0000 7f06abe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f06ac103550 0x7f06ac19f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:11:56.977 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:56.999+0000 7f06abe00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f06ac108620 0x7f06ac19fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:11:56.977 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:56.999+0000 7f06abe00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f06ac1999f0 0x7f06ac199ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:11:56.977 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:56.999+0000 7f06abe00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06ac1135b0 con 0x7f06ac103550 2024-03-08T21:11:56.977 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:56.999+0000 7f06abe00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06ac113430 con 0x7f06ac1999f0 2024-03-08T21:11:56.977 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:56.999+0000 7f06abe00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06ac113730 con 0x7f06ac108620 2024-03-08T21:11:56.977 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:57.000+0000 7f06aaa00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f06ac103550 0x7f06ac19f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:11:56.977 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:57.000+0000 7f06aaa00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f06ac103550 0x7f06ac19f7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:42706/0 (socket says 172.21.15.107:42706) 2024-03-08T21:11:56.977 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:57.000+0000 7f06aaa00640 1 -- 172.21.15.107:0/1687852599 learned_addr learned my addr 172.21.15.107:0/1687852599 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:11:56.979 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:57.000+0000 7f06ab400640 1 --2- 172.21.15.107:0/1687852599 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f06ac1999f0 0x7f06ac199ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:11:56.979 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:57.000+0000 7f06aa000640 1 --2- 172.21.15.107:0/1687852599 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f06ac108620 0x7f06ac19fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:11:56.979 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:57.000+0000 7f06aaa00640 1 -- 172.21.15.107:0/1687852599 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f06ac108620 msgr2=0x7f06ac19fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:11:56.979 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:57.000+0000 7f06aaa00640 1 --2- 172.21.15.107:0/1687852599 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f06ac108620 0x7f06ac19fd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:11:56.979 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:57.000+0000 7f06aaa00640 1 -- 172.21.15.107:0/1687852599 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f06ac1999f0 msgr2=0x7f06ac199ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:11:56.979 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:57.000+0000 7f06aaa00640 1 --2- 172.21.15.107:0/1687852599 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f06ac1999f0 0x7f06ac199ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:11:56.979 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:57.000+0000 7f06aaa00640 1 -- 172.21.15.107:0/1687852599 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06ac19a510 con 0x7f06ac103550 2024-03-08T21:11:56.979 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:57.000+0000 7f06aa000640 1 --2- 172.21.15.107:0/1687852599 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f06ac108620 0x7f06ac19fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T21:11:56.979 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:57.000+0000 7f06aaa00640 1 --2- 172.21.15.107:0/1687852599 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f06ac103550 0x7f06ac19f7f0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f069c00d950 tx=0x7f069c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:11:56.980 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:57.001+0000 7f06a3400640 1 -- 172.21.15.107:0/1687852599 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f069c017070 con 0x7f06ac103550 2024-03-08T21:11:56.980 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:57.001+0000 7f06a3400640 1 -- 172.21.15.107:0/1687852599 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f069c009c30 con 0x7f06ac103550 2024-03-08T21:11:56.980 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:57.001+0000 7f06abe00640 1 -- 172.21.15.107:0/1687852599 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06ac19a800 con 0x7f06ac103550 2024-03-08T21:11:56.980 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:57.001+0000 7f06a3400640 1 -- 172.21.15.107:0/1687852599 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f069c00f620 con 0x7f06ac103550 2024-03-08T21:11:56.980 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:57.001+0000 7f06abe00640 1 -- 172.21.15.107:0/1687852599 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06ac1a6a00 con 0x7f06ac103550 2024-03-08T21:11:56.981 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:57.004+0000 7f06b49a6740 1 Processor -- start 2024-03-08T21:11:56.981 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:57.004+0000 7f06b49a6740 1 -- start start 2024-03-08T21:11:56.983 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:57.005+0000 7f06b49a6740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56111016fe80 0x561110170260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:11:56.983 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:57.005+0000 7f06b49a6740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5611101707a0 0x561110179030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:11:56.983 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:57.005+0000 7f06b49a6740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561110179570 0x56111017b960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:11:56.983 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:57.005+0000 7f06a3400640 1 -- 172.21.15.107:0/1687852599 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) v1 ==== 137422+0+0 (secure 0 0 0) 0x7f069c0109a0 con 0x7f06ac103550 2024-03-08T21:11:56.983 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:57.005+0000 7f06b49a6740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611100d33d0 con 0x5611101707a0 2024-03-08T21:11:56.983 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:57.005+0000 7f06b49a6740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x56111007c6b0 con 0x56111016fe80 2024-03-08T21:11:56.983 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:57.005+0000 7f06b49a6740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611100cf470 con 0x561110179570 2024-03-08T21:11:56.983 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:57.005+0000 7f06aaa00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5611101707a0 0x561110179030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:11:56.983 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:57.005+0000 7f06aaa00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5611101707a0 0x561110179030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:42720/0 (socket says 172.21.15.107:42720) 2024-03-08T21:11:56.983 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:57.005+0000 7f06aaa00640 1 -- 172.21.15.107:0/3482592229 learned_addr learned my addr 172.21.15.107:0/3482592229 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:11:56.983 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:57.005+0000 7f06ab400640 1 --2- 172.21.15.107:0/3482592229 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56111016fe80 0x561110170260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:11:56.983 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:57.005+0000 7f06aa000640 1 --2- 172.21.15.107:0/3482592229 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561110179570 0x56111017b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:11:56.984 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:57.006+0000 7f06aaa00640 1 -- 172.21.15.107:0/3482592229 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561110179570 msgr2=0x56111017b960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:11:56.984 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:57.006+0000 7f06aaa00640 1 --2- 172.21.15.107:0/3482592229 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561110179570 0x56111017b960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:11:56.984 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:57.006+0000 7f06aaa00640 1 -- 172.21.15.107:0/3482592229 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56111016fe80 msgr2=0x561110170260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:11:56.984 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:57.006+0000 7f06aaa00640 1 --2- 172.21.15.107:0/3482592229 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56111016fe80 0x561110170260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:11:56.984 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:57.006+0000 7f06aaa00640 1 -- 172.21.15.107:0/3482592229 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5611100a2b50 con 0x5611101707a0 2024-03-08T21:11:56.988 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:57.006+0000 7f06aaa00640 1 --2- 172.21.15.107:0/3482592229 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5611101707a0 0x561110179030 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f069c0cc230 tx=0x7f069c0d76f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=427f66676a3b9994 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:11:56.988 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:57.006+0000 7f0697e00640 1 -- 172.21.15.107:0/3482592229 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f069c0e1600 con 0x5611101707a0 2024-03-08T21:11:56.988 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:57.006+0000 7f0697e00640 1 -- 172.21.15.107:0/3482592229 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f069c0d98b0 con 0x5611101707a0 2024-03-08T21:11:56.988 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:57.007+0000 7f0697e00640 1 -- 172.21.15.107:0/3482592229 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f069c0d5880 con 0x5611101707a0 2024-03-08T21:11:56.988 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:57.007+0000 7f06a3400640 1 --2- 172.21.15.107:0/1687852599 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f06900a2960 0x7f06900a4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:11:56.988 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:57.007+0000 7f06a3400640 1 -- 172.21.15.107:0/1687852599 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f069c0cebb0 con 0x7f06ac103550 2024-03-08T21:11:56.988 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:57.007+0000 7f06ab400640 1 --2- 172.21.15.107:0/1687852599 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f06900a2960 0x7f06900a4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:11:56.988 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:57.007+0000 7f06b49a6740 1 -- 172.21.15.107:0/3482592229 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5611101707a0 msgr2=0x561110179030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:11:56.989 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:57.007+0000 7f06b49a6740 1 --2- 172.21.15.107:0/3482592229 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5611101707a0 0x561110179030 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f069c0cc230 tx=0x7f069c0d76f0 comp rx=0 tx=0).stop 2024-03-08T21:11:56.989 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:57.008+0000 7f06b49a6740 1 -- 172.21.15.107:0/3482592229 shutdown_connections 2024-03-08T21:11:56.989 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:57.008+0000 7f06b49a6740 1 --2- 172.21.15.107:0/3482592229 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561110179570 0x56111017b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:11:56.989 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:57.008+0000 7f06b49a6740 1 --2- 172.21.15.107:0/3482592229 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5611101707a0 0x561110179030 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:11:56.989 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:57.008+0000 7f06b49a6740 1 --2- 172.21.15.107:0/3482592229 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56111016fe80 0x561110170260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:11:56.989 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:57.008+0000 7f06b49a6740 1 -- 172.21.15.107:0/3482592229 >> 172.21.15.107:0/3482592229 conn(0x56111016a7a0 msgr2=0x56111016ab80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:11:56.989 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:57.008+0000 7f06b49a6740 1 -- 172.21.15.107:0/3482592229 shutdown_connections 2024-03-08T21:11:56.989 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:57.008+0000 7f06b49a6740 1 -- 172.21.15.107:0/3482592229 wait complete. 2024-03-08T21:11:56.989 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:57.008+0000 7f06ab400640 1 --2- 172.21.15.107:0/1687852599 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f06900a2960 0x7f06900a4e20 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f06a4005fd0 tx=0x7f06a40096e0 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:11:56.989 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:57.008+0000 7f06b49a6740 1 -- start start 2024-03-08T21:11:56.989 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:57.009+0000 7f06b49a6740 1 Processor -- start 2024-03-08T21:11:56.989 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:57.009+0000 7f06b49a6740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56111016fe80 0x561110178f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:11:56.989 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:57.009+0000 7f06b49a6740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5611101707a0 0x5611101762e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:11:56.990 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:57.009+0000 7f06b49a6740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561110179570 0x5611101768f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:11:56.990 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:57.009+0000 7f06b49a6740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x56111017c1a0 con 0x561110179570 2024-03-08T21:11:56.990 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:57.009+0000 7f06b49a6740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x56111008f9f0 con 0x56111016fe80 2024-03-08T21:11:56.990 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:57.009+0000 7f06b49a6740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56111017c320 con 0x5611101707a0 2024-03-08T21:11:56.990 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:57.009+0000 7f06b49a6740 10 client.0 _ll_register_callbacks cb 0x56110fde7a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T21:11:56.990 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:57.009+0000 7f06ab400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561110179570 0x5611101768f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:11:56.990 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:57.010+0000 7f06ab400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561110179570 0x5611101768f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:42734/0 (socket says 172.21.15.107:42734) 2024-03-08T21:11:56.990 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:57.010+0000 7f06ab400640 1 -- 172.21.15.107:0/526225092 learned_addr learned my addr 172.21.15.107:0/526225092 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:11:56.990 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:57.010+0000 7f06aaa00640 1 --2- 172.21.15.107:0/526225092 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56111016fe80 0x561110178f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:11:56.990 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:57.010+0000 7f06aa000640 1 --2- 172.21.15.107:0/526225092 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5611101707a0 0x5611101762e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:11:56.990 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:57.010+0000 7f06ab400640 1 -- 172.21.15.107:0/526225092 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5611101707a0 msgr2=0x5611101762e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:11:56.990 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:57.010+0000 7f06ab400640 1 --2- 172.21.15.107:0/526225092 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5611101707a0 0x5611101762e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:11:56.990 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:57.010+0000 7f06ab400640 1 -- 172.21.15.107:0/526225092 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56111016fe80 msgr2=0x561110178f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:11:56.991 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:57.010+0000 7f06ab400640 1 --2- 172.21.15.107:0/526225092 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56111016fe80 0x561110178f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:11:56.991 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:57.010+0000 7f06ab400640 1 -- 172.21.15.107:0/526225092 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56110fde8210 con 0x561110179570 2024-03-08T21:11:56.991 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:57.010+0000 7f06aa000640 1 --2- 172.21.15.107:0/526225092 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5611101707a0 0x5611101762e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T21:11:56.991 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:57.010+0000 7f06aaa00640 1 --2- 172.21.15.107:0/526225092 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56111016fe80 0x561110178f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T21:11:56.991 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:57.010+0000 7f06ab400640 1 --2- 172.21.15.107:0/526225092 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561110179570 0x5611101768f0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f06a401d870 tx=0x7f06a4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3951e7eafb5b0906 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:11:56.991 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:57.010+0000 7f0694c00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-03-08T21:11:56.991 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:57.011+0000 7f0694c00640 1 -- 172.21.15.107:0/526225092 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06a401e820 con 0x561110179570 2024-03-08T21:11:56.991 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:57.011+0000 7f0694c00640 1 -- 172.21.15.107:0/526225092 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f06a401e9c0 con 0x561110179570 2024-03-08T21:11:56.991 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:57.011+0000 7f06b49a6740 1 -- 172.21.15.107:0/526225092 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f069c0d98b0 con 0x561110179570 2024-03-08T21:11:56.991 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:57.011+0000 7f0694c00640 1 -- 172.21.15.107:0/526225092 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f06a401ebc0 con 0x561110179570 2024-03-08T21:11:56.991 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:57.011+0000 7f06b49a6740 10 client.16407 fetch_fsmap learned FSMap version 39 2024-03-08T21:11:56.991 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:57.011+0000 7f06b49a6740 1 -- 172.21.15.107:0/526225092 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x56110feb4620 con 0x561110179570 2024-03-08T21:11:56.993 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:57.011+0000 7f0694c00640 1 -- 172.21.15.107:0/526225092 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f069c0d98b0 con 0x561110179570 2024-03-08T21:11:56.993 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:57.012+0000 7f0694c00640 1 -- 172.21.15.107:0/526225092 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 39) v1 ==== 7307+0+0 (secure 0 0 0) 0x7f06a400f040 con 0x561110179570 2024-03-08T21:11:56.993 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:57.012+0000 7f06b49a6740 10 client.16407 fetch_fsmap finished waiting for FSMap version 39 2024-03-08T21:11:56.993 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:57.012+0000 7f06b49a6740 10 client.16407 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-08T21:11:56.993 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:57.012+0000 7f06b49a6740 20 client.16407 populate_metadata read hostname 'smithi107' 2024-03-08T21:11:56.993 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:57.012+0000 7f06b49a6740 1 --2- 172.21.15.107:0/526225092 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x5611101894a0 0x56111018b890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:11:56.994 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:57.012+0000 7f06b49a6740 4 client.16407 mds_command: new command op to 34475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-08T21:11:56.994 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:57.012+0000 7f06b49a6740 1 -- 172.21.15.107:0/526225092 --> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5611100e0910 con 0x5611101894a0 2024-03-08T21:11:56.994 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:57.012+0000 7f06aaa00640 1 --2- 172.21.15.107:0/526225092 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x5611101894a0 0x56111018b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:11:56.994 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:57.013+0000 7f06aaa00640 1 --2- 172.21.15.107:0/526225092 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x5611101894a0 0x56111018b890 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fe1a53b8ec77b542 server_cookie=75f59d49c87e16a2 in_seq=0 out_seq=0 2024-03-08T21:11:56.994 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:57.013+0000 7f0694c00640 10 client.16407 ms_handle_connect on v2:172.21.15.149:6838/55290596 2024-03-08T21:11:56.994 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:57.015+0000 7f0694c00640 1 -- 172.21.15.107:0/526225092 <== mds.0 v2:172.21.15.149:6838/55290596 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5611100e0910 con 0x5611101894a0 2024-03-08T21:11:56.994 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:57.015+0000 7f0694c00640 10 client.16407 handle_command_reply: tid=0 2024-03-08T21:11:56.994 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:57.015+0000 7f06b49a6740 1 client.16407 shutdown 2024-03-08T21:11:56.994 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:57.015+0000 7f06b49a6740 1 -- 172.21.15.107:0/526225092 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561110179570 msgr2=0x5611101768f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:11:56.994 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:57.015+0000 7f06b49a6740 1 --2- 172.21.15.107:0/526225092 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561110179570 0x5611101768f0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f06a401d870 tx=0x7f06a4004a40 comp rx=0 tx=0).stop 2024-03-08T21:11:56.994 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:57.015+0000 7f06b49a6740 1 -- 172.21.15.107:0/526225092 shutdown_connections 2024-03-08T21:11:56.994 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:57.015+0000 7f06b49a6740 1 --2- 172.21.15.107:0/526225092 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561110179570 0x5611101768f0 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:11:56.994 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:57.015+0000 7f06b49a6740 1 --2- 172.21.15.107:0/526225092 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5611101707a0 0x5611101762e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:11:56.995 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:57.015+0000 7f06b49a6740 1 --2- 172.21.15.107:0/526225092 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x5611101894a0 0x56111018b890 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:11:56.995 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:57.015+0000 7f06b49a6740 1 --2- 172.21.15.107:0/526225092 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56111016fe80 0x561110178f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:11:56.995 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:57.015+0000 7f0694c00640 0 client.16407 ms_handle_reset on v2:172.21.15.149:6838/55290596 2024-03-08T21:11:56.995 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:57.015+0000 7f06b49a6740 1 -- 172.21.15.107:0/526225092 >> 172.21.15.107:0/526225092 conn(0x56111016a7a0 msgr2=0x56111016c4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:11:56.995 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:57.015+0000 7f06b49a6740 1 -- 172.21.15.107:0/526225092 shutdown_connections 2024-03-08T21:11:56.995 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:57.015+0000 7f06b49a6740 1 -- 172.21.15.107:0/526225092 wait complete. 2024-03-08T21:11:56.995 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:57.016+0000 7f06b49a6740 20 client.16407 trim_cache size 0 max 16384 2024-03-08T21:11:57.008 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:57.031+0000 7f06b49a6740 1 Processor -- start 2024-03-08T21:11:57.008 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:57.031+0000 7f06b49a6740 1 -- start start 2024-03-08T21:11:57.010 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:57.032+0000 7f06b49a6740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56111016fe80 0x56111016c700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:11:57.010 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:57.032+0000 7f06b49a6740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5611101707a0 0x561110178a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:11:57.010 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:57.032+0000 7f06b49a6740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561110179570 0x561110178f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:11:57.010 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:57.032+0000 7f06b49a6740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06ac1b0790 con 0x5611101707a0 2024-03-08T21:11:57.010 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:57.032+0000 7f06b49a6740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06ac1ae7b0 con 0x561110179570 2024-03-08T21:11:57.011 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:57.032+0000 7f06b49a6740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06ac1b0130 con 0x56111016fe80 2024-03-08T21:11:57.011 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:57.032+0000 7f06aa000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5611101707a0 0x561110178a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:11:57.011 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:57.032+0000 7f06aa000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5611101707a0 0x561110178a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:42750/0 (socket says 172.21.15.107:42750) 2024-03-08T21:11:57.011 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:57.032+0000 7f06aa000640 1 -- 172.21.15.107:0/1929586541 learned_addr learned my addr 172.21.15.107:0/1929586541 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:11:57.011 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:57.032+0000 7f06aaa00640 1 --2- 172.21.15.107:0/1929586541 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56111016fe80 0x56111016c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:11:57.011 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:57.032+0000 7f06ab400640 1 --2- 172.21.15.107:0/1929586541 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561110179570 0x561110178f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:11:57.011 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:57.032+0000 7f06aa000640 1 -- 172.21.15.107:0/1929586541 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56111016fe80 msgr2=0x56111016c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:11:57.011 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:57.032+0000 7f06aa000640 1 --2- 172.21.15.107:0/1929586541 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56111016fe80 0x56111016c700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:11:57.011 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:57.032+0000 7f06aa000640 1 -- 172.21.15.107:0/1929586541 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561110179570 msgr2=0x561110178f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:11:57.011 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:57.032+0000 7f06aa000640 1 --2- 172.21.15.107:0/1929586541 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561110179570 0x561110178f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:11:57.011 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:57.032+0000 7f06aa000640 1 -- 172.21.15.107:0/1929586541 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5611101705d0 con 0x5611101707a0 2024-03-08T21:11:57.011 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:57.032+0000 7f06ab400640 1 --2- 172.21.15.107:0/1929586541 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561110179570 0x561110178f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T21:11:57.011 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:57.033+0000 7f06aa000640 1 --2- 172.21.15.107:0/1929586541 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5611101707a0 0x561110178a10 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f069802c170 tx=0x7f069802c680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e84242830b4f73a server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:11:57.011 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:57.033+0000 7f0697400640 1 -- 172.21.15.107:0/1929586541 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0698045070 con 0x5611101707a0 2024-03-08T21:11:57.011 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:57.033+0000 7f0697400640 1 -- 172.21.15.107:0/1929586541 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f069802c7e0 con 0x5611101707a0 2024-03-08T21:11:57.012 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:57.033+0000 7f0697400640 1 -- 172.21.15.107:0/1929586541 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06980046c0 con 0x5611101707a0 2024-03-08T21:11:57.014 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:57.034+0000 7f06b49a6740 1 -- 172.21.15.107:0/1929586541 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5611101707a0 msgr2=0x561110178a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:11:57.014 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:57.034+0000 7f06b49a6740 1 --2- 172.21.15.107:0/1929586541 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5611101707a0 0x561110178a10 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f069802c170 tx=0x7f069802c680 comp rx=0 tx=0).stop 2024-03-08T21:11:57.014 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:57.034+0000 7f06b49a6740 1 -- 172.21.15.107:0/1929586541 shutdown_connections 2024-03-08T21:11:57.015 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:57.034+0000 7f06b49a6740 1 --2- 172.21.15.107:0/1929586541 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x561110179570 0x561110178f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:11:57.015 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:57.034+0000 7f06b49a6740 1 --2- 172.21.15.107:0/1929586541 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5611101707a0 0x561110178a10 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:11:57.015 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:57.034+0000 7f06b49a6740 1 --2- 172.21.15.107:0/1929586541 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56111016fe80 0x56111016c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:11:57.015 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:57.034+0000 7f06b49a6740 1 -- 172.21.15.107:0/1929586541 >> 172.21.15.107:0/1929586541 conn(0x56111016a7a0 msgr2=0x5611101762a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:11:57.015 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:57.034+0000 7f06b49a6740 1 -- 172.21.15.107:0/1929586541 shutdown_connections 2024-03-08T21:11:57.015 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:57.034+0000 7f06b49a6740 1 -- 172.21.15.107:0/1929586541 wait complete. 2024-03-08T21:11:57.015 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:57.035+0000 7f06b49a6740 1 -- start start 2024-03-08T21:11:57.015 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:57.035+0000 7f06b49a6740 1 Processor -- start 2024-03-08T21:11:57.015 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:57.035+0000 7f06b49a6740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56111016fe80 0x561110180aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:11:57.015 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:57.035+0000 7f06b49a6740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5611101707a0 0x56111017ee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:11:57.015 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:57.035+0000 7f06b49a6740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561110179570 0x56111017f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:11:57.015 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:57.035+0000 7f06b49a6740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06ac1af470 con 0x561110179570 2024-03-08T21:11:57.015 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:57.035+0000 7f06b49a6740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06ac1aee10 con 0x56111016fe80 2024-03-08T21:11:57.015 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:57.035+0000 7f06b49a6740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06ac1afad0 con 0x5611101707a0 2024-03-08T21:11:57.016 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:57.035+0000 7f06b49a6740 10 client.0 _ll_register_callbacks cb 0x7f069c0d25d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T21:11:57.016 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:57.036+0000 7f06ab400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561110179570 0x56111017f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:11:57.016 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:57.036+0000 7f06ab400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561110179570 0x56111017f5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:42756/0 (socket says 172.21.15.107:42756) 2024-03-08T21:11:57.016 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:57.036+0000 7f06ab400640 1 -- 172.21.15.107:0/3085998822 learned_addr learned my addr 172.21.15.107:0/3085998822 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:11:57.016 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:57.036+0000 7f06aaa00640 1 --2- 172.21.15.107:0/3085998822 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56111016fe80 0x561110180aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:11:57.016 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:57.036+0000 7f06aa000640 1 --2- 172.21.15.107:0/3085998822 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5611101707a0 0x56111017ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:11:57.016 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:57.036+0000 7f06ab400640 1 -- 172.21.15.107:0/3085998822 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5611101707a0 msgr2=0x56111017ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:11:57.016 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:57.036+0000 7f06ab400640 1 --2- 172.21.15.107:0/3085998822 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5611101707a0 0x56111017ee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:11:57.016 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:57.036+0000 7f06ab400640 1 -- 172.21.15.107:0/3085998822 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56111016fe80 msgr2=0x561110180aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:11:57.016 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:57.036+0000 7f06ab400640 1 --2- 172.21.15.107:0/3085998822 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56111016fe80 0x561110180aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:11:57.016 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:57.036+0000 7f06ab400640 1 -- 172.21.15.107:0/3085998822 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5611101df950 con 0x561110179570 2024-03-08T21:11:57.016 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:57.036+0000 7f06aa000640 1 --2- 172.21.15.107:0/3085998822 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5611101707a0 0x56111017ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T21:11:57.016 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:57.037+0000 7f06ab400640 1 --2- 172.21.15.107:0/3085998822 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561110179570 0x56111017f5d0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f06a4013fd0 tx=0x7f06a401de20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=939124e6edd942af server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:11:57.016 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:57.037+0000 7f0694c00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-03-08T21:11:57.017 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:57.037+0000 7f0694c00640 1 -- 172.21.15.107:0/3085998822 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06a403d020 con 0x561110179570 2024-03-08T21:11:57.017 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:57.037+0000 7f0694c00640 1 -- 172.21.15.107:0/3085998822 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f06a401e8e0 con 0x561110179570 2024-03-08T21:11:57.017 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:57.037+0000 7f06b49a6740 1 -- 172.21.15.107:0/3085998822 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f069802c7e0 con 0x561110179570 2024-03-08T21:11:57.017 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:57.037+0000 7f0694c00640 1 -- 172.21.15.107:0/3085998822 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f06a4048050 con 0x561110179570 2024-03-08T21:11:57.017 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:57.037+0000 7f06b49a6740 10 client.16413 fetch_fsmap learned FSMap version 39 2024-03-08T21:11:57.017 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:57.037+0000 7f06b49a6740 1 -- 172.21.15.107:0/3085998822 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x5611101e0b30 con 0x561110179570 2024-03-08T21:11:57.017 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:57.038+0000 7f0694c00640 1 -- 172.21.15.107:0/3085998822 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f069802c7e0 con 0x561110179570 2024-03-08T21:11:57.018 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:57.038+0000 7f0694c00640 1 -- 172.21.15.107:0/3085998822 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 39) v1 ==== 7307+0+0 (secure 0 0 0) 0x7f06a4038050 con 0x561110179570 2024-03-08T21:11:57.018 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:57.038+0000 7f06b49a6740 10 client.16413 fetch_fsmap finished waiting for FSMap version 39 2024-03-08T21:11:57.018 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:57.038+0000 7f06b49a6740 10 client.16413 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-08T21:11:57.018 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:57.038+0000 7f06b49a6740 20 client.16413 populate_metadata read hostname 'smithi107' 2024-03-08T21:11:57.018 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:57.038+0000 7f06b49a6740 1 --2- 172.21.15.107:0/3085998822 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x5611101e1280 0x5611101e36a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:11:57.018 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:57.038+0000 7f06b49a6740 4 client.16413 mds_command: new command op to 34475 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-08T21:11:57.018 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:57.038+0000 7f06b49a6740 1 -- 172.21.15.107:0/3085998822 --> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5611100dac60 con 0x5611101e1280 2024-03-08T21:11:57.018 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:57.038+0000 7f06aaa00640 1 --2- 172.21.15.107:0/3085998822 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x5611101e1280 0x5611101e36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:11:57.018 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:57.039+0000 7f06aaa00640 1 --2- 172.21.15.107:0/3085998822 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x5611101e1280 0x5611101e36a0 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bfb3d868f1bd1b2e server_cookie=3db2b641c8ed37aa in_seq=0 out_seq=0 2024-03-08T21:11:57.019 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:57.039+0000 7f0694c00640 10 client.16413 ms_handle_connect on v2:172.21.15.149:6838/55290596 2024-03-08T21:11:57.026 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:57.049+0000 7f0694c00640 1 -- 172.21.15.107:0/3085998822 <== mds.0 v2:172.21.15.149:6838/55290596 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5611100dac60 con 0x5611101e1280 2024-03-08T21:11:57.026 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:57.049+0000 7f0694c00640 10 client.16413 handle_command_reply: tid=0 2024-03-08T21:11:57.026 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:57.049+0000 7f06b49a6740 1 client.16413 shutdown 2024-03-08T21:11:57.026 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:57.049+0000 7f06b49a6740 1 -- 172.21.15.107:0/3085998822 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561110179570 msgr2=0x56111017f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:11:57.026 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:57.049+0000 7f06b49a6740 1 --2- 172.21.15.107:0/3085998822 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561110179570 0x56111017f5d0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f06a4013fd0 tx=0x7f06a401de20 comp rx=0 tx=0).stop 2024-03-08T21:11:57.026 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:57.049+0000 7f06b49a6740 1 -- 172.21.15.107:0/3085998822 shutdown_connections 2024-03-08T21:11:57.027 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:57.049+0000 7f06b49a6740 1 --2- 172.21.15.107:0/3085998822 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561110179570 0x56111017f5d0 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:11:57.027 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:57.049+0000 7f06b49a6740 1 --2- 172.21.15.107:0/3085998822 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5611101707a0 0x56111017ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:11:57.027 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:57.049+0000 7f06b49a6740 1 --2- 172.21.15.107:0/3085998822 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x5611101e1280 0x5611101e36a0 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:11:57.027 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:57.049+0000 7f06b49a6740 1 --2- 172.21.15.107:0/3085998822 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56111016fe80 0x561110180aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:11:57.027 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:57.049+0000 7f0694c00640 0 client.16413 ms_handle_reset on v2:172.21.15.149:6838/55290596 2024-03-08T21:11:57.027 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:57.049+0000 7f06b49a6740 1 -- 172.21.15.107:0/3085998822 >> 172.21.15.107:0/3085998822 conn(0x56111016a7a0 msgr2=0x561110173630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:11:57.028 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:57.049+0000 7f06b49a6740 1 -- 172.21.15.107:0/3085998822 shutdown_connections 2024-03-08T21:11:57.028 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:57.049+0000 7f06b49a6740 1 -- 172.21.15.107:0/3085998822 wait complete. 2024-03-08T21:11:57.028 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:57.050+0000 7f06b49a6740 20 client.16413 trim_cache size 0 max 16384 2024-03-08T21:11:57.028 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:{ 2024-03-08T21:11:57.028 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "message": "", 2024-03-08T21:11:57.028 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "return_code": 0 2024-03-08T21:11:57.028 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:} 2024-03-08T21:11:57.029 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:57.050+0000 7f06abe00640 1 -- 172.21.15.107:0/1687852599 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f06900a2960 msgr2=0x7f06900a4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:11:57.029 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:57.050+0000 7f06abe00640 1 --2- 172.21.15.107:0/1687852599 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f06900a2960 0x7f06900a4e20 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f06a4005fd0 tx=0x7f06a40096e0 comp rx=0 tx=0).stop 2024-03-08T21:11:57.029 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:57.050+0000 7f06abe00640 1 -- 172.21.15.107:0/1687852599 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f06ac103550 msgr2=0x7f06ac19f7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:11:57.029 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:57.050+0000 7f06abe00640 1 --2- 172.21.15.107:0/1687852599 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f06ac103550 0x7f06ac19f7f0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f069c00d950 tx=0x7f069c00de10 comp rx=0 tx=0).stop 2024-03-08T21:11:57.029 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:57.051+0000 7f06abe00640 1 -- 172.21.15.107:0/1687852599 shutdown_connections 2024-03-08T21:11:57.029 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:57.051+0000 7f06abe00640 1 --2- 172.21.15.107:0/1687852599 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f06900a2960 0x7f06900a4e20 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:11:57.030 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:57.051+0000 7f06abe00640 1 --2- 172.21.15.107:0/1687852599 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f06ac1999f0 0x7f06ac199ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:11:57.030 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:57.051+0000 7f06abe00640 1 --2- 172.21.15.107:0/1687852599 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f06ac108620 0x7f06ac19fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:11:57.030 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:57.051+0000 7f06abe00640 1 --2- 172.21.15.107:0/1687852599 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f06ac103550 0x7f06ac19f7f0 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:11:57.030 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:57.051+0000 7f06abe00640 1 -- 172.21.15.107:0/1687852599 >> 172.21.15.107:0/1687852599 conn(0x7f06ac0fedd0 msgr2=0x7f06ac100140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:11:57.030 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:57.051+0000 7f06abe00640 1 -- 172.21.15.107:0/1687852599 shutdown_connections 2024-03-08T21:11:57.030 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:11:57.051+0000 7f06abe00640 1 -- 172.21.15.107:0/1687852599 wait complete. 2024-03-08T21:11:58.212 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:11:57 smithi107 ceph-mon[31381]: pgmap v360: 97 pgs: 97 active+clean; 7.4 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:11:58.259 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:11:57 smithi149 ceph-mon[35042]: pgmap v360: 97 pgs: 97 active+clean; 7.4 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:11:58.373 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:11:57 smithi204 ceph-mon[34935]: pgmap v360: 97 pgs: 97 active+clean; 7.4 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:12:00.212 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:11:59 smithi107 ceph-mon[31381]: pgmap v361: 97 pgs: 97 active+clean; 7.4 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 511 B/s wr, 13 op/s 2024-03-08T21:12:00.259 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:11:59 smithi149 ceph-mon[35042]: pgmap v361: 97 pgs: 97 active+clean; 7.4 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 511 B/s wr, 13 op/s 2024-03-08T21:12:00.373 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:11:59 smithi204 ceph-mon[34935]: pgmap v361: 97 pgs: 97 active+clean; 7.4 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 511 B/s wr, 13 op/s 2024-03-08T21:12:02.212 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:12:01 smithi107 ceph-mon[31381]: pgmap v362: 97 pgs: 97 active+clean; 7.4 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:12:02.259 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:12:01 smithi149 ceph-mon[35042]: pgmap v362: 97 pgs: 97 active+clean; 7.4 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:12:02.373 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:12:01 smithi204 ceph-mon[34935]: pgmap v362: 97 pgs: 97 active+clean; 7.4 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:12:04.212 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:12:03 smithi107 ceph-mon[31381]: pgmap v363: 97 pgs: 97 active+clean; 7.4 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:12:04.259 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:12:03 smithi149 ceph-mon[35042]: pgmap v363: 97 pgs: 97 active+clean; 7.4 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:12:04.373 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:12:03 smithi204 ceph-mon[34935]: pgmap v363: 97 pgs: 97 active+clean; 7.4 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:12:06.212 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:12:05 smithi107 ceph-mon[31381]: pgmap v364: 97 pgs: 97 active+clean; 8.1 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 104 KiB/s wr, 30 op/s 2024-03-08T21:12:06.212 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:12:05 smithi107 ceph-mon[31381]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T21:12:06.259 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:12:05 smithi149 ceph-mon[35042]: pgmap v364: 97 pgs: 97 active+clean; 8.1 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 104 KiB/s wr, 30 op/s 2024-03-08T21:12:06.259 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:12:05 smithi149 ceph-mon[35042]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T21:12:06.373 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:12:05 smithi204 ceph-mon[34935]: pgmap v364: 97 pgs: 97 active+clean; 8.1 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 104 KiB/s wr, 30 op/s 2024-03-08T21:12:06.373 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:12:05 smithi204 ceph-mon[34935]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T21:12:07.212 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:12:06 smithi107 ceph-mon[31381]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T21:12:07.212 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:12:06 smithi107 ceph-mon[31381]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T21:12:07.212 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:12:06 smithi107 ceph-mon[31381]: from='mgr.15375 ' entity='mgr.x' 2024-03-08T21:12:07.259 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:12:06 smithi149 ceph-mon[35042]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T21:12:07.259 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:12:06 smithi149 ceph-mon[35042]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T21:12:07.259 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:12:06 smithi149 ceph-mon[35042]: from='mgr.15375 ' entity='mgr.x' 2024-03-08T21:12:07.275 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:12:06 smithi204 ceph-mon[34935]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T21:12:07.275 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:12:06 smithi204 ceph-mon[34935]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T21:12:07.275 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:12:06 smithi204 ceph-mon[34935]: from='mgr.15375 ' entity='mgr.x' 2024-03-08T21:12:08.759 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:12:08 smithi149 ceph-mon[35042]: pgmap v365: 97 pgs: 97 active+clean; 8.1 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 104 KiB/s wr, 25 op/s 2024-03-08T21:12:08.873 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:12:08 smithi204 ceph-mon[34935]: pgmap v365: 97 pgs: 97 active+clean; 8.1 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 104 KiB/s wr, 25 op/s 2024-03-08T21:12:08.962 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:12:08 smithi107 ceph-mon[31381]: pgmap v365: 97 pgs: 97 active+clean; 8.1 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 104 KiB/s wr, 25 op/s 2024-03-08T21:12:10.174 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.197+0000 7fbbb6c00640 1 -- 172.21.15.107:0/3743237880 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbbb0101cf0 msgr2=0x7fbbb0105d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:12:10.175 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.197+0000 7fbbb6c00640 1 --2- 172.21.15.107:0/3743237880 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbbb0101cf0 0x7fbbb0105d30 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fbb9c009900 tx=0x7fbb9c02bc30 comp rx=0 tx=0).stop 2024-03-08T21:12:10.175 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.197+0000 7fbbb6c00640 1 -- 172.21.15.107:0/3743237880 shutdown_connections 2024-03-08T21:12:10.175 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.197+0000 7fbbb6c00640 1 --2- 172.21.15.107:0/3743237880 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbbb0106410 0x7fbbb010dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:12:10.175 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.197+0000 7fbbb6c00640 1 --2- 172.21.15.107:0/3743237880 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbbb0101cf0 0x7fbbb0105d30 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:12:10.175 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.197+0000 7fbbb6c00640 1 --2- 172.21.15.107:0/3743237880 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbbb0101340 0x7fbbb0101720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:12:10.175 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.197+0000 7fbbb6c00640 1 -- 172.21.15.107:0/3743237880 >> 172.21.15.107:0/3743237880 conn(0x7fbbb00fcc40 msgr2=0x7fbbb00ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:12:10.175 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.197+0000 7fbbb6c00640 1 -- 172.21.15.107:0/3743237880 shutdown_connections 2024-03-08T21:12:10.175 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.197+0000 7fbbb6c00640 1 -- 172.21.15.107:0/3743237880 wait complete. 2024-03-08T21:12:10.176 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.198+0000 7fbbb6c00640 1 Processor -- start 2024-03-08T21:12:10.176 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.199+0000 7fbbb6c00640 1 -- start start 2024-03-08T21:12:10.176 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.199+0000 7fbbb6c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbbb0101340 0x7fbbb019d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:12:10.177 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.199+0000 7fbbb6c00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbbb0106410 0x7fbbb019dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:12:10.177 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.199+0000 7fbbb6c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbbb0197750 0x7fbbb0197bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:12:10.177 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.199+0000 7fbbb6c00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbb0111350 con 0x7fbbb0101340 2024-03-08T21:12:10.177 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.199+0000 7fbbb6c00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbb01111d0 con 0x7fbbb0106410 2024-03-08T21:12:10.177 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.199+0000 7fbbb6c00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbb01114d0 con 0x7fbbb0197750 2024-03-08T21:12:10.177 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.199+0000 7fbbb5800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbbb0101340 0x7fbbb019d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:12:10.177 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.199+0000 7fbbb5800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbbb0101340 0x7fbbb019d690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:33388/0 (socket says 172.21.15.107:33388) 2024-03-08T21:12:10.177 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.199+0000 7fbbb5800640 1 -- 172.21.15.107:0/3895535140 learned_addr learned my addr 172.21.15.107:0/3895535140 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:12:10.177 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.200+0000 7fbbb6200640 1 --2- 172.21.15.107:0/3895535140 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbbb0197750 0x7fbbb0197bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:12:10.178 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.200+0000 7fbbb5800640 1 -- 172.21.15.107:0/3895535140 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbbb0197750 msgr2=0x7fbbb0197bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:12:10.178 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.200+0000 7fbbb4e00640 1 --2- 172.21.15.107:0/3895535140 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbbb0106410 0x7fbbb019dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:12:10.178 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.200+0000 7fbbb5800640 1 --2- 172.21.15.107:0/3895535140 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbbb0197750 0x7fbbb0197bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:12:10.179 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.200+0000 7fbbb5800640 1 -- 172.21.15.107:0/3895535140 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbbb0106410 msgr2=0x7fbbb019dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:12:10.179 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.200+0000 7fbbb5800640 1 --2- 172.21.15.107:0/3895535140 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbbb0106410 0x7fbbb019dbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:12:10.179 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.200+0000 7fbbb5800640 1 -- 172.21.15.107:0/3895535140 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbb0198350 con 0x7fbbb0101340 2024-03-08T21:12:10.179 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.200+0000 7fbbb6200640 1 --2- 172.21.15.107:0/3895535140 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbbb0197750 0x7fbbb0197bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T21:12:10.179 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.200+0000 7fbbb5800640 1 --2- 172.21.15.107:0/3895535140 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbbb0101340 0x7fbbb019d690 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fbba400e9f0 tx=0x7fbba400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:12:10.179 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.201+0000 7fbbae000640 1 -- 172.21.15.107:0/3895535140 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbba400cd80 con 0x7fbbb0101340 2024-03-08T21:12:10.179 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.201+0000 7fbbae000640 1 -- 172.21.15.107:0/3895535140 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fbba40054d0 con 0x7fbbb0101340 2024-03-08T21:12:10.179 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.201+0000 7fbbb6c00640 1 -- 172.21.15.107:0/3895535140 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbb01a4430 con 0x7fbbb0101340 2024-03-08T21:12:10.179 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.201+0000 7fbbae000640 1 -- 172.21.15.107:0/3895535140 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbba4010490 con 0x7fbbb0101340 2024-03-08T21:12:10.179 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.201+0000 7fbbb6c00640 1 -- 172.21.15.107:0/3895535140 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbb01a4830 con 0x7fbbb0101340 2024-03-08T21:12:10.181 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.204+0000 7fbbbaf92740 1 Processor -- start 2024-03-08T21:12:10.182 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.204+0000 7fbbbaf92740 1 -- start start 2024-03-08T21:12:10.182 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.205+0000 7fbbbaf92740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559f168d18d0 0x559f168d1cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:12:10.182 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.205+0000 7fbbbaf92740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559f168d2280 0x559f168d2700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:12:10.182 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.205+0000 7fbbbaf92740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559f168e3c10 0x559f168e6000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:12:10.182 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.205+0000 7fbbbaf92740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f168353d0 con 0x559f168e3c10 2024-03-08T21:12:10.182 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.205+0000 7fbbbaf92740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f167de6b0 con 0x559f168d2280 2024-03-08T21:12:10.182 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.205+0000 7fbbbaf92740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f16831470 con 0x559f168d18d0 2024-03-08T21:12:10.184 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.205+0000 7fbbae000640 1 -- 172.21.15.107:0/3895535140 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) v1 ==== 137422+0+0 (secure 0 0 0) 0x7fbba4005020 con 0x7fbbb0101340 2024-03-08T21:12:10.184 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.205+0000 7fbbb4e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559f168e3c10 0x559f168e6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:12:10.184 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.205+0000 7fbbb4e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559f168e3c10 0x559f168e6000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:33400/0 (socket says 172.21.15.107:33400) 2024-03-08T21:12:10.184 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.205+0000 7fbbb4e00640 1 -- 172.21.15.107:0/1339518742 learned_addr learned my addr 172.21.15.107:0/1339518742 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:12:10.184 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.205+0000 7fbbb6200640 1 --2- 172.21.15.107:0/1339518742 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559f168d18d0 0x559f168d1cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:12:10.184 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.206+0000 7fbbb4e00640 1 -- 172.21.15.107:0/1339518742 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559f168d18d0 msgr2=0x559f168d1cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:12:10.184 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.206+0000 7fbbb4e00640 1 --2- 172.21.15.107:0/1339518742 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559f168d18d0 0x559f168d1cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:12:10.185 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.206+0000 7fbbb4e00640 1 -- 172.21.15.107:0/1339518742 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559f168d2280 msgr2=0x559f168d2700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:12:10.185 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.206+0000 7fbbb4e00640 1 --2- 172.21.15.107:0/1339518742 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559f168d2280 0x559f168d2700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:12:10.185 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.206+0000 7fbbb4e00640 1 -- 172.21.15.107:0/1339518742 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559f16804b50 con 0x559f168e3c10 2024-03-08T21:12:10.185 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.206+0000 7fbbb4e00640 1 --2- 172.21.15.107:0/1339518742 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559f168e3c10 0x559f168e6000 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7fbb9c006010 tx=0x7fbb9c005dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c7abe680aeba785 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:12:10.185 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.206+0000 7fbba2a00640 1 -- 172.21.15.107:0/1339518742 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb9c043020 con 0x559f168e3c10 2024-03-08T21:12:10.185 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.206+0000 7fbba2a00640 1 -- 172.21.15.107:0/1339518742 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fbb9c02c7e0 con 0x559f168e3c10 2024-03-08T21:12:10.185 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.207+0000 7fbba2a00640 1 -- 172.21.15.107:0/1339518742 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb9c005510 con 0x559f168e3c10 2024-03-08T21:12:10.185 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.207+0000 7fbbae000640 1 --2- 172.21.15.107:0/3895535140 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7fbb980a29e0 0x7fbb980a4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:12:10.185 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.207+0000 7fbbae000640 1 -- 172.21.15.107:0/3895535140 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 8629+0+0 (secure 0 0 0) 0x7fbba40cf350 con 0x7fbbb0101340 2024-03-08T21:12:10.185 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.207+0000 7fbbb6200640 1 --2- 172.21.15.107:0/3895535140 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7fbb980a29e0 0x7fbb980a4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:12:10.189 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.207+0000 7fbbbaf92740 1 -- 172.21.15.107:0/1339518742 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559f168e3c10 msgr2=0x559f168e6000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:12:10.189 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.207+0000 7fbbbaf92740 1 --2- 172.21.15.107:0/1339518742 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559f168e3c10 0x559f168e6000 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7fbb9c006010 tx=0x7fbb9c005dd0 comp rx=0 tx=0).stop 2024-03-08T21:12:10.189 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.208+0000 7fbbbaf92740 1 -- 172.21.15.107:0/1339518742 shutdown_connections 2024-03-08T21:12:10.189 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.208+0000 7fbbbaf92740 1 --2- 172.21.15.107:0/1339518742 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559f168e3c10 0x559f168e6000 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:12:10.189 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.208+0000 7fbbbaf92740 1 --2- 172.21.15.107:0/1339518742 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559f168d2280 0x559f168d2700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:12:10.189 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.208+0000 7fbbbaf92740 1 --2- 172.21.15.107:0/1339518742 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559f168d18d0 0x559f168d1cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:12:10.189 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.208+0000 7fbbbaf92740 1 -- 172.21.15.107:0/1339518742 >> 172.21.15.107:0/1339518742 conn(0x559f168cc7a0 msgr2=0x559f168ccb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:12:10.189 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.208+0000 7fbbbaf92740 1 -- 172.21.15.107:0/1339518742 shutdown_connections 2024-03-08T21:12:10.190 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.208+0000 7fbbb6200640 1 --2- 172.21.15.107:0/3895535140 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7fbb980a29e0 0x7fbb980a4ea0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fbba8005fd0 tx=0x7fbba8009530 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:12:10.190 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.208+0000 7fbbbaf92740 1 -- 172.21.15.107:0/1339518742 wait complete. 2024-03-08T21:12:10.190 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.208+0000 7fbbbaf92740 1 -- start start 2024-03-08T21:12:10.190 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.209+0000 7fbbbaf92740 1 Processor -- start 2024-03-08T21:12:10.190 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.209+0000 7fbbbaf92740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559f168d18d0 0x559f168d69a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:12:10.190 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.209+0000 7fbbbaf92740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559f168d2280 0x559f168d3d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:12:10.190 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.209+0000 7fbbbaf92740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559f168e3c10 0x559f168d4270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:12:10.190 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.209+0000 7fbbbaf92740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f168e68e0 con 0x559f168d18d0 2024-03-08T21:12:10.190 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.209+0000 7fbbbaf92740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f167f19f0 con 0x559f168e3c10 2024-03-08T21:12:10.190 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.209+0000 7fbbbaf92740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f168e6a60 con 0x559f168d2280 2024-03-08T21:12:10.190 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.209+0000 7fbbbaf92740 10 client.0 _ll_register_callbacks cb 0x559f16549a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T21:12:10.191 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.210+0000 7fbbb5800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559f168d18d0 0x559f168d69a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:12:10.191 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.210+0000 7fbbb5800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559f168d18d0 0x559f168d69a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:33404/0 (socket says 172.21.15.107:33404) 2024-03-08T21:12:10.191 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.210+0000 7fbbb5800640 1 -- 172.21.15.107:0/2365589939 learned_addr learned my addr 172.21.15.107:0/2365589939 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:12:10.191 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.210+0000 7fbbb6200640 1 --2- 172.21.15.107:0/2365589939 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559f168e3c10 0x559f168d4270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:12:10.191 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.210+0000 7fbbb4e00640 1 --2- 172.21.15.107:0/2365589939 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559f168d2280 0x559f168d3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:12:10.191 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.210+0000 7fbbb5800640 1 -- 172.21.15.107:0/2365589939 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559f168d2280 msgr2=0x559f168d3d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:12:10.191 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.210+0000 7fbbb5800640 1 --2- 172.21.15.107:0/2365589939 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559f168d2280 0x559f168d3d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:12:10.191 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.210+0000 7fbbb5800640 1 -- 172.21.15.107:0/2365589939 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559f168e3c10 msgr2=0x559f168d4270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:12:10.191 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.210+0000 7fbbb5800640 1 --2- 172.21.15.107:0/2365589939 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559f168e3c10 0x559f168d4270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:12:10.191 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.210+0000 7fbbb5800640 1 -- 172.21.15.107:0/2365589939 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559f1654a210 con 0x559f168d18d0 2024-03-08T21:12:10.191 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.210+0000 7fbbb5800640 1 --2- 172.21.15.107:0/2365589939 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559f168d18d0 0x559f168d69a0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7fbba40ccae0 tx=0x7fbba409b360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4eb50bf965a0c36 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:12:10.191 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.210+0000 7fbb97400640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-03-08T21:12:10.191 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.211+0000 7fbb97400640 1 -- 172.21.15.107:0/2365589939 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbba409c040 con 0x559f168d18d0 2024-03-08T21:12:10.192 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.211+0000 7fbb97400640 1 -- 172.21.15.107:0/2365589939 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 8629+0+0 (secure 0 0 0) 0x7fbba4097fb0 con 0x559f168d18d0 2024-03-08T21:12:10.192 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.211+0000 7fbbbaf92740 1 -- 172.21.15.107:0/2365589939 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbb9c02c7e0 con 0x559f168d18d0 2024-03-08T21:12:10.192 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.211+0000 7fbb97400640 1 -- 172.21.15.107:0/2365589939 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbba409b940 con 0x559f168d18d0 2024-03-08T21:12:10.192 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.211+0000 7fbbbaf92740 10 client.16425 fetch_fsmap learned FSMap version 39 2024-03-08T21:12:10.192 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.211+0000 7fbbbaf92740 1 -- 172.21.15.107:0/2365589939 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x559f16616620 con 0x559f168d18d0 2024-03-08T21:12:10.192 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.211+0000 7fbb97400640 1 -- 172.21.15.107:0/2365589939 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fbb9c02c7e0 con 0x559f168d18d0 2024-03-08T21:12:10.192 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.211+0000 7fbb97400640 1 -- 172.21.15.107:0/2365589939 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 39) v1 ==== 7307+0+0 (secure 0 0 0) 0x7fbba40860c0 con 0x559f168d18d0 2024-03-08T21:12:10.192 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.211+0000 7fbbbaf92740 10 client.16425 fetch_fsmap finished waiting for FSMap version 39 2024-03-08T21:12:10.192 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.211+0000 7fbbbaf92740 10 client.16425 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-08T21:12:10.192 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.212+0000 7fbbbaf92740 20 client.16425 populate_metadata read hostname 'smithi107' 2024-03-08T21:12:10.192 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.212+0000 7fbbbaf92740 1 --2- 172.21.15.107:0/2365589939 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x559f168f3b90 0x559f168f5f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:12:10.192 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.212+0000 7fbbbaf92740 4 client.16425 mds_command: new command op to 34475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-08T21:12:10.192 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.212+0000 7fbbbaf92740 1 -- 172.21.15.107:0/2365589939 --> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559f16842910 con 0x559f168f3b90 2024-03-08T21:12:10.192 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.212+0000 7fbbb4e00640 1 --2- 172.21.15.107:0/2365589939 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x559f168f3b90 0x559f168f5f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:12:10.194 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.213+0000 7fbbb4e00640 1 --2- 172.21.15.107:0/2365589939 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x559f168f3b90 0x559f168f5f80 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d59b3a55adbc4a75 server_cookie=f7751447713c4322 in_seq=0 out_seq=0 2024-03-08T21:12:10.194 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.213+0000 7fbb97400640 10 client.16425 ms_handle_connect on v2:172.21.15.149:6838/55290596 2024-03-08T21:12:10.194 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.216+0000 7fbb97400640 1 -- 172.21.15.107:0/2365589939 <== mds.0 v2:172.21.15.149:6838/55290596 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x559f16842910 con 0x559f168f3b90 2024-03-08T21:12:10.194 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.216+0000 7fbb97400640 10 client.16425 handle_command_reply: tid=0 2024-03-08T21:12:10.194 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.216+0000 7fbbbaf92740 1 client.16425 shutdown 2024-03-08T21:12:10.194 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.216+0000 7fbbbaf92740 1 -- 172.21.15.107:0/2365589939 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559f168d18d0 msgr2=0x559f168d69a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:12:10.194 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.216+0000 7fbbbaf92740 1 --2- 172.21.15.107:0/2365589939 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559f168d18d0 0x559f168d69a0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7fbba40ccae0 tx=0x7fbba409b360 comp rx=0 tx=0).stop 2024-03-08T21:12:10.194 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.216+0000 7fbbbaf92740 1 -- 172.21.15.107:0/2365589939 shutdown_connections 2024-03-08T21:12:10.194 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.216+0000 7fbbbaf92740 1 --2- 172.21.15.107:0/2365589939 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x559f168f3b90 0x559f168f5f80 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:12:10.195 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.216+0000 7fbbbaf92740 1 --2- 172.21.15.107:0/2365589939 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559f168e3c10 0x559f168d4270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:12:10.195 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.216+0000 7fbbbaf92740 1 --2- 172.21.15.107:0/2365589939 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559f168d2280 0x559f168d3d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:12:10.195 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.216+0000 7fbb97400640 0 client.16425 ms_handle_reset on v2:172.21.15.149:6838/55290596 2024-03-08T21:12:10.195 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.216+0000 7fbbbaf92740 1 --2- 172.21.15.107:0/2365589939 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559f168d18d0 0x559f168d69a0 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:12:10.195 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.216+0000 7fbbbaf92740 1 -- 172.21.15.107:0/2365589939 >> 172.21.15.107:0/2365589939 conn(0x559f168cc7a0 msgr2=0x559f168ce4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:12:10.195 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.216+0000 7fbbbaf92740 1 -- 172.21.15.107:0/2365589939 shutdown_connections 2024-03-08T21:12:10.195 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.216+0000 7fbbbaf92740 1 -- 172.21.15.107:0/2365589939 wait complete. 2024-03-08T21:12:10.195 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.217+0000 7fbbbaf92740 20 client.16425 trim_cache size 0 max 16384 2024-03-08T21:12:10.217 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.239+0000 7fbbbaf92740 1 Processor -- start 2024-03-08T21:12:10.217 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.240+0000 7fbbbaf92740 1 -- start start 2024-03-08T21:12:10.217 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.240+0000 7fbbbaf92740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559f168d18d0 0x559f168d6460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:12:10.217 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.240+0000 7fbbbaf92740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559f168d2280 0x559f168d69a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:12:10.217 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.240+0000 7fbbbaf92740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559f168e3c10 0x559f168ead40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:12:10.217 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.240+0000 7fbbbaf92740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbb01ac260 con 0x559f168e3c10 2024-03-08T21:12:10.217 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.240+0000 7fbbbaf92740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbb01aa280 con 0x559f168d2280 2024-03-08T21:12:10.217 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.240+0000 7fbbbaf92740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbb01abc00 con 0x559f168d18d0 2024-03-08T21:12:10.228 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.240+0000 7fbbb6200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559f168e3c10 0x559f168ead40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:12:10.229 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.240+0000 7fbbb6200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559f168e3c10 0x559f168ead40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:33420/0 (socket says 172.21.15.107:33420) 2024-03-08T21:12:10.229 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.240+0000 7fbbb6200640 1 -- 172.21.15.107:0/3233663850 learned_addr learned my addr 172.21.15.107:0/3233663850 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:12:10.229 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.241+0000 7fbbb5800640 1 --2- 172.21.15.107:0/3233663850 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559f168d18d0 0x559f168d6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:12:10.229 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.241+0000 7fbbb4e00640 1 --2- 172.21.15.107:0/3233663850 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559f168d2280 0x559f168d69a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:12:10.229 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.241+0000 7fbbb6200640 1 -- 172.21.15.107:0/3233663850 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559f168d18d0 msgr2=0x559f168d6460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:12:10.230 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.241+0000 7fbbb6200640 1 --2- 172.21.15.107:0/3233663850 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559f168d18d0 0x559f168d6460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:12:10.230 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.241+0000 7fbbb6200640 1 -- 172.21.15.107:0/3233663850 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559f168d2280 msgr2=0x559f168d69a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:12:10.230 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.241+0000 7fbbb6200640 1 --2- 172.21.15.107:0/3233663850 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559f168d2280 0x559f168d69a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:12:10.230 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.241+0000 7fbbb6200640 1 -- 172.21.15.107:0/3233663850 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559f168c6be0 con 0x559f168e3c10 2024-03-08T21:12:10.230 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.241+0000 7fbbb5800640 1 --2- 172.21.15.107:0/3233663850 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559f168d18d0 0x559f168d6460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T21:12:10.231 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.241+0000 7fbbb6200640 1 --2- 172.21.15.107:0/3233663850 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559f168e3c10 0x559f168ead40 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fbba8012860 tx=0x7fbba8014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=418650d3ac30e172 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:12:10.231 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.241+0000 7fbba2000640 1 -- 172.21.15.107:0/3233663850 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbba800f950 con 0x559f168e3c10 2024-03-08T21:12:10.231 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.242+0000 7fbba2000640 1 -- 172.21.15.107:0/3233663850 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fbba8020430 con 0x559f168e3c10 2024-03-08T21:12:10.231 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.242+0000 7fbba2000640 1 -- 172.21.15.107:0/3233663850 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbba801f620 con 0x559f168e3c10 2024-03-08T21:12:10.231 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.242+0000 7fbbbaf92740 1 -- 172.21.15.107:0/3233663850 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559f168e3c10 msgr2=0x559f168ead40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:12:10.231 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.242+0000 7fbbbaf92740 1 --2- 172.21.15.107:0/3233663850 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559f168e3c10 0x559f168ead40 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fbba8012860 tx=0x7fbba8014000 comp rx=0 tx=0).stop 2024-03-08T21:12:10.231 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.242+0000 7fbbbaf92740 1 -- 172.21.15.107:0/3233663850 shutdown_connections 2024-03-08T21:12:10.231 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.242+0000 7fbbbaf92740 1 --2- 172.21.15.107:0/3233663850 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559f168e3c10 0x559f168ead40 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:12:10.232 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.242+0000 7fbbbaf92740 1 --2- 172.21.15.107:0/3233663850 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559f168d2280 0x559f168d69a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:12:10.232 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.242+0000 7fbbbaf92740 1 --2- 172.21.15.107:0/3233663850 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559f168d18d0 0x559f168d6460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:12:10.232 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.242+0000 7fbbbaf92740 1 -- 172.21.15.107:0/3233663850 >> 172.21.15.107:0/3233663850 conn(0x559f168cc7a0 msgr2=0x559f168fa8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:12:10.232 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.243+0000 7fbbbaf92740 1 -- 172.21.15.107:0/3233663850 shutdown_connections 2024-03-08T21:12:10.232 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.243+0000 7fbbbaf92740 1 -- 172.21.15.107:0/3233663850 wait complete. 2024-03-08T21:12:10.232 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.243+0000 7fbbbaf92740 1 -- start start 2024-03-08T21:12:10.233 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.243+0000 7fbbbaf92740 1 Processor -- start 2024-03-08T21:12:10.233 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.244+0000 7fbbbaf92740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559f168d18d0 0x559f168e9a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:12:10.233 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.244+0000 7fbbbaf92740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559f168d2280 0x559f168e9f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:12:10.233 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.244+0000 7fbbbaf92740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559f168e3c10 0x559f168ea4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:12:10.233 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.244+0000 7fbbbaf92740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbb01aaf40 con 0x559f168d2280 2024-03-08T21:12:10.233 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.244+0000 7fbbbaf92740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbb01aa8e0 con 0x559f168e3c10 2024-03-08T21:12:10.234 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.244+0000 7fbbbaf92740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbb01ab5a0 con 0x559f168d18d0 2024-03-08T21:12:10.234 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.244+0000 7fbbbaf92740 10 client.0 _ll_register_callbacks cb 0x7fbba4005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T21:12:10.234 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.244+0000 7fbbb4e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559f168d2280 0x559f168e9f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:12:10.234 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.244+0000 7fbbb4e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559f168d2280 0x559f168e9f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:33430/0 (socket says 172.21.15.107:33430) 2024-03-08T21:12:10.234 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.244+0000 7fbbb4e00640 1 -- 172.21.15.107:0/4113723477 learned_addr learned my addr 172.21.15.107:0/4113723477 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:12:10.235 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.244+0000 7fbbb5800640 1 --2- 172.21.15.107:0/4113723477 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559f168d18d0 0x559f168e9a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:12:10.235 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.244+0000 7fbbb6200640 1 --2- 172.21.15.107:0/4113723477 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559f168e3c10 0x559f168ea4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:12:10.235 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.244+0000 7fbbb4e00640 1 -- 172.21.15.107:0/4113723477 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559f168d18d0 msgr2=0x559f168e9a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:12:10.235 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.244+0000 7fbbb4e00640 1 --2- 172.21.15.107:0/4113723477 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559f168d18d0 0x559f168e9a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:12:10.235 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.244+0000 7fbbb4e00640 1 -- 172.21.15.107:0/4113723477 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559f168e3c10 msgr2=0x559f168ea4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:12:10.235 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.244+0000 7fbbb4e00640 1 --2- 172.21.15.107:0/4113723477 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559f168e3c10 0x559f168ea4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:12:10.236 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.244+0000 7fbbb4e00640 1 -- 172.21.15.107:0/4113723477 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559f1694a310 con 0x559f168d2280 2024-03-08T21:12:10.236 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.244+0000 7fbbb6200640 1 --2- 172.21.15.107:0/4113723477 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559f168e3c10 0x559f168ea4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T21:12:10.236 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.245+0000 7fbbb4e00640 1 --2- 172.21.15.107:0/4113723477 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559f168d2280 0x559f168e9f60 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fbb9c006010 tx=0x7fbb9c02c6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb7a9cddefb9bfab server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:12:10.236 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.245+0000 7fbb97400640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-03-08T21:12:10.236 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.245+0000 7fbb97400640 1 -- 172.21.15.107:0/4113723477 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb9c043040 con 0x559f168d2280 2024-03-08T21:12:10.236 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.245+0000 7fbb97400640 1 -- 172.21.15.107:0/4113723477 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 8629+0+0 (secure 0 0 0) 0x7fbb9c004450 con 0x559f168d2280 2024-03-08T21:12:10.237 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.245+0000 7fbbbaf92740 1 -- 172.21.15.107:0/4113723477 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbba8020430 con 0x559f168d2280 2024-03-08T21:12:10.237 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.245+0000 7fbb97400640 1 -- 172.21.15.107:0/4113723477 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbb9c02c2b0 con 0x559f168d2280 2024-03-08T21:12:10.237 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.245+0000 7fbbbaf92740 10 client.16431 fetch_fsmap learned FSMap version 39 2024-03-08T21:12:10.237 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.245+0000 7fbbbaf92740 1 -- 172.21.15.107:0/4113723477 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x559f1694b620 con 0x559f168d2280 2024-03-08T21:12:10.237 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.246+0000 7fbb97400640 1 -- 172.21.15.107:0/4113723477 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fbba8020430 con 0x559f168d2280 2024-03-08T21:12:10.237 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.246+0000 7fbb97400640 1 -- 172.21.15.107:0/4113723477 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 39) v1 ==== 7307+0+0 (secure 0 0 0) 0x7fbb9c004aa0 con 0x559f168d2280 2024-03-08T21:12:10.238 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.246+0000 7fbbbaf92740 10 client.16431 fetch_fsmap finished waiting for FSMap version 39 2024-03-08T21:12:10.238 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.246+0000 7fbbbaf92740 10 client.16431 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-08T21:12:10.238 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.246+0000 7fbbbaf92740 20 client.16431 populate_metadata read hostname 'smithi107' 2024-03-08T21:12:10.238 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.246+0000 7fbbbaf92740 1 --2- 172.21.15.107:0/4113723477 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x559f1694b9c0 0x559f1694ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:12:10.238 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.246+0000 7fbbbaf92740 4 client.16431 mds_command: new command op to 34475 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-08T21:12:10.238 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.246+0000 7fbbbaf92740 1 -- 172.21.15.107:0/4113723477 --> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x559f1683cc60 con 0x559f1694b9c0 2024-03-08T21:12:10.239 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.247+0000 7fbbb5800640 1 --2- 172.21.15.107:0/4113723477 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x559f1694b9c0 0x559f1694ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:12:10.239 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.248+0000 7fbbb5800640 1 --2- 172.21.15.107:0/4113723477 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x559f1694b9c0 0x559f1694ddb0 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ae187366d621d08b server_cookie=89e7d8078092a1ab in_seq=0 out_seq=0 2024-03-08T21:12:10.239 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.248+0000 7fbb97400640 10 client.16431 ms_handle_connect on v2:172.21.15.149:6838/55290596 2024-03-08T21:12:10.253 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.275+0000 7fbb97400640 1 -- 172.21.15.107:0/4113723477 <== mds.0 v2:172.21.15.149:6838/55290596 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x559f1683cc60 con 0x559f1694b9c0 2024-03-08T21:12:10.253 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.275+0000 7fbb97400640 10 client.16431 handle_command_reply: tid=0 2024-03-08T21:12:10.254 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.275+0000 7fbbbaf92740 1 client.16431 shutdown 2024-03-08T21:12:10.254 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.276+0000 7fbbbaf92740 1 -- 172.21.15.107:0/4113723477 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559f168d2280 msgr2=0x559f168e9f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:12:10.254 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.276+0000 7fbbbaf92740 1 --2- 172.21.15.107:0/4113723477 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559f168d2280 0x559f168e9f60 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fbb9c006010 tx=0x7fbb9c02c6d0 comp rx=0 tx=0).stop 2024-03-08T21:12:10.254 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.276+0000 7fbbbaf92740 1 -- 172.21.15.107:0/4113723477 shutdown_connections 2024-03-08T21:12:10.254 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.276+0000 7fbbbaf92740 1 --2- 172.21.15.107:0/4113723477 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x559f1694b9c0 0x559f1694ddb0 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:12:10.254 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.276+0000 7fbbbaf92740 1 --2- 172.21.15.107:0/4113723477 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559f168e3c10 0x559f168ea4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:12:10.254 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.276+0000 7fbbbaf92740 1 --2- 172.21.15.107:0/4113723477 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559f168d2280 0x559f168e9f60 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:12:10.254 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.276+0000 7fbbbaf92740 1 --2- 172.21.15.107:0/4113723477 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x559f168d18d0 0x559f168e9a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:12:10.254 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.276+0000 7fbbbaf92740 1 -- 172.21.15.107:0/4113723477 >> 172.21.15.107:0/4113723477 conn(0x559f168cc7a0 msgr2=0x559f168d3cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:12:10.254 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.276+0000 7fbb97400640 0 client.16431 ms_handle_reset on v2:172.21.15.149:6838/55290596 2024-03-08T21:12:10.254 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.276+0000 7fbbbaf92740 1 -- 172.21.15.107:0/4113723477 shutdown_connections 2024-03-08T21:12:10.254 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.276+0000 7fbbbaf92740 1 -- 172.21.15.107:0/4113723477 wait complete. 2024-03-08T21:12:10.254 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.276+0000 7fbbbaf92740 20 client.16431 trim_cache size 0 max 16384 2024-03-08T21:12:10.255 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:{ 2024-03-08T21:12:10.255 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "message": "", 2024-03-08T21:12:10.255 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "return_code": 0 2024-03-08T21:12:10.255 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:} 2024-03-08T21:12:10.256 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.277+0000 7fbbb6c00640 1 -- 172.21.15.107:0/3895535140 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7fbb980a29e0 msgr2=0x7fbb980a4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:12:10.256 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.277+0000 7fbbb6c00640 1 --2- 172.21.15.107:0/3895535140 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7fbb980a29e0 0x7fbb980a4ea0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fbba8005fd0 tx=0x7fbba8009530 comp rx=0 tx=0).stop 2024-03-08T21:12:10.256 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.277+0000 7fbbb6c00640 1 -- 172.21.15.107:0/3895535140 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbbb0101340 msgr2=0x7fbbb019d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:12:10.256 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.277+0000 7fbbb6c00640 1 --2- 172.21.15.107:0/3895535140 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbbb0101340 0x7fbbb019d690 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fbba400e9f0 tx=0x7fbba400edf0 comp rx=0 tx=0).stop 2024-03-08T21:12:10.256 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.277+0000 7fbbb6c00640 1 -- 172.21.15.107:0/3895535140 shutdown_connections 2024-03-08T21:12:10.256 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.277+0000 7fbbb6c00640 1 --2- 172.21.15.107:0/3895535140 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7fbb980a29e0 0x7fbb980a4ea0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:12:10.256 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.277+0000 7fbbb6c00640 1 --2- 172.21.15.107:0/3895535140 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fbbb0197750 0x7fbbb0197bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:12:10.256 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.277+0000 7fbbb6c00640 1 --2- 172.21.15.107:0/3895535140 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbbb0106410 0x7fbbb019dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:12:10.256 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.277+0000 7fbbb6c00640 1 --2- 172.21.15.107:0/3895535140 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbbb0101340 0x7fbbb019d690 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:12:10.256 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.277+0000 7fbbb6c00640 1 -- 172.21.15.107:0/3895535140 >> 172.21.15.107:0/3895535140 conn(0x7fbbb00fcc40 msgr2=0x7fbbb00fe080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:12:10.257 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.278+0000 7fbbb6c00640 1 -- 172.21.15.107:0/3895535140 shutdown_connections 2024-03-08T21:12:10.257 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:10.278+0000 7fbbb6c00640 1 -- 172.21.15.107:0/3895535140 wait complete. 2024-03-08T21:12:10.759 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:12:10 smithi149 ceph-mon[35042]: pgmap v366: 97 pgs: 97 active+clean; 8.1 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 104 KiB/s wr, 30 op/s 2024-03-08T21:12:10.759 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:12:10 smithi149 ceph-mon[35042]: osdmap e105: 12 total, 12 up, 12 in 2024-03-08T21:12:10.873 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:12:10 smithi204 ceph-mon[34935]: pgmap v366: 97 pgs: 97 active+clean; 8.1 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 104 KiB/s wr, 30 op/s 2024-03-08T21:12:10.873 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:12:10 smithi204 ceph-mon[34935]: osdmap e105: 12 total, 12 up, 12 in 2024-03-08T21:12:10.962 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:12:10 smithi107 ceph-mon[31381]: pgmap v366: 97 pgs: 97 active+clean; 8.1 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 104 KiB/s wr, 30 op/s 2024-03-08T21:12:10.962 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:12:10 smithi107 ceph-mon[31381]: osdmap e105: 12 total, 12 up, 12 in 2024-03-08T21:12:12.759 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:12:12 smithi149 ceph-mon[35042]: pgmap v368: 97 pgs: 97 active+clean; 8.1 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 124 KiB/s wr, 30 op/s 2024-03-08T21:12:12.874 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:12:12 smithi204 ceph-mon[34935]: pgmap v368: 97 pgs: 97 active+clean; 8.1 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 124 KiB/s wr, 30 op/s 2024-03-08T21:12:12.962 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:12:12 smithi107 ceph-mon[31381]: pgmap v368: 97 pgs: 97 active+clean; 8.1 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 124 KiB/s wr, 30 op/s 2024-03-08T21:12:14.872 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:12:14 smithi204 ceph-mon[34935]: pgmap v369: 97 pgs: 97 active+clean; 8.1 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 124 KiB/s wr, 30 op/s 2024-03-08T21:12:14.962 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:12:14 smithi107 ceph-mon[31381]: pgmap v369: 97 pgs: 97 active+clean; 8.1 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 124 KiB/s wr, 30 op/s 2024-03-08T21:12:15.217 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:12:14 smithi149 ceph-mon[35042]: pgmap v369: 97 pgs: 97 active+clean; 8.1 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 124 KiB/s wr, 30 op/s 2024-03-08T21:12:16.812 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:12:16 smithi149 ceph-mon[35042]: pgmap v370: 97 pgs: 97 active+clean; 8.1 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 3.7 KiB/s wr, 13 op/s 2024-03-08T21:12:16.872 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:12:16 smithi204 ceph-mon[34935]: pgmap v370: 97 pgs: 97 active+clean; 8.1 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 3.7 KiB/s wr, 13 op/s 2024-03-08T21:12:16.961 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:12:16 smithi107 ceph-mon[31381]: pgmap v370: 97 pgs: 97 active+clean; 8.1 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 3.7 KiB/s wr, 13 op/s 2024-03-08T21:12:18.872 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:12:18 smithi204 ceph-mon[34935]: pgmap v371: 97 pgs: 97 active+clean; 8.1 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 3.7 KiB/s wr, 13 op/s 2024-03-08T21:12:18.962 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:12:18 smithi107 ceph-mon[31381]: pgmap v371: 97 pgs: 97 active+clean; 8.1 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 3.7 KiB/s wr, 13 op/s 2024-03-08T21:12:19.009 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:12:18 smithi149 ceph-mon[35042]: pgmap v371: 97 pgs: 97 active+clean; 8.1 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 3.7 KiB/s wr, 13 op/s 2024-03-08T21:12:20.509 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:12:20 smithi149 ceph-mon[35042]: pgmap v372: 97 pgs: 97 active+clean; 8.1 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 3.6 KiB/s wr, 13 op/s 2024-03-08T21:12:20.509 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:12:20 smithi149 ceph-mon[35042]: osdmap e106: 12 total, 12 up, 12 in 2024-03-08T21:12:20.623 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:12:20 smithi204 ceph-mon[34935]: pgmap v372: 97 pgs: 97 active+clean; 8.1 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 3.6 KiB/s wr, 13 op/s 2024-03-08T21:12:20.623 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:12:20 smithi204 ceph-mon[34935]: osdmap e106: 12 total, 12 up, 12 in 2024-03-08T21:12:20.712 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:12:20 smithi107 ceph-mon[31381]: pgmap v372: 97 pgs: 97 active+clean; 8.1 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 3.6 KiB/s wr, 13 op/s 2024-03-08T21:12:20.712 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:12:20 smithi107 ceph-mon[31381]: osdmap e106: 12 total, 12 up, 12 in 2024-03-08T21:12:22.509 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:12:22 smithi149 ceph-mon[35042]: pgmap v374: 97 pgs: 97 active+clean; 8.1 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 3.6 KiB/s wr, 13 op/s 2024-03-08T21:12:22.622 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:12:22 smithi204 ceph-mon[34935]: pgmap v374: 97 pgs: 97 active+clean; 8.1 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 3.6 KiB/s wr, 13 op/s 2024-03-08T21:12:22.712 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:12:22 smithi107 ceph-mon[31381]: pgmap v374: 97 pgs: 97 active+clean; 8.1 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 3.6 KiB/s wr, 13 op/s 2024-03-08T21:12:23.402 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.424+0000 7fd2adc00640 1 -- 172.21.15.107:0/3293110825 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd2a80ffeb0 msgr2=0x7fd2a8100290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:12:23.403 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.424+0000 7fd2adc00640 1 --2- 172.21.15.107:0/3293110825 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd2a80ffeb0 0x7fd2a8100290 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fd298009900 tx=0x7fd29802bc40 comp rx=0 tx=0).stop 2024-03-08T21:12:23.403 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.424+0000 7fd2adc00640 1 -- 172.21.15.107:0/3293110825 shutdown_connections 2024-03-08T21:12:23.403 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.424+0000 7fd2adc00640 1 --2- 172.21.15.107:0/3293110825 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd2a810da60 0x7fd2a810fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:12:23.403 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.424+0000 7fd2adc00640 1 --2- 172.21.15.107:0/3293110825 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd2a8100860 0x7fd2a810d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:12:23.403 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.424+0000 7fd2adc00640 1 --2- 172.21.15.107:0/3293110825 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd2a80ffeb0 0x7fd2a8100290 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:12:23.403 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.424+0000 7fd2adc00640 1 -- 172.21.15.107:0/3293110825 >> 172.21.15.107:0/3293110825 conn(0x7fd2a80fbb00 msgr2=0x7fd2a80fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:12:23.403 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.424+0000 7fd2adc00640 1 -- 172.21.15.107:0/3293110825 shutdown_connections 2024-03-08T21:12:23.403 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.424+0000 7fd2adc00640 1 -- 172.21.15.107:0/3293110825 wait complete. 2024-03-08T21:12:23.405 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.425+0000 7fd2adc00640 1 Processor -- start 2024-03-08T21:12:23.405 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.425+0000 7fd2adc00640 1 -- start start 2024-03-08T21:12:23.405 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.426+0000 7fd2adc00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd2a80ffeb0 0x7fd2a819f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:12:23.406 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.426+0000 7fd2adc00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd2a8100860 0x7fd2a819fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:12:23.406 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.426+0000 7fd2adc00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd2a810da60 0x7fd2a8199940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:12:23.406 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.426+0000 7fd2adc00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2a81135e0 con 0x7fd2a80ffeb0 2024-03-08T21:12:23.406 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.426+0000 7fd2adc00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2a8113460 con 0x7fd2a810da60 2024-03-08T21:12:23.406 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.426+0000 7fd2adc00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2a8113760 con 0x7fd2a8100860 2024-03-08T21:12:23.406 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.426+0000 7fd2a7e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd2a80ffeb0 0x7fd2a819f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:12:23.406 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.426+0000 7fd2a7e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd2a80ffeb0 0x7fd2a819f740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:58172/0 (socket says 172.21.15.107:58172) 2024-03-08T21:12:23.406 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.426+0000 7fd2a7e00640 1 -- 172.21.15.107:0/1462996248 learned_addr learned my addr 172.21.15.107:0/1462996248 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:12:23.406 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.426+0000 7fd2a7400640 1 --2- 172.21.15.107:0/1462996248 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd2a8100860 0x7fd2a819fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:12:23.406 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.426+0000 7fd2ad200640 1 --2- 172.21.15.107:0/1462996248 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd2a810da60 0x7fd2a8199940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:12:23.406 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.426+0000 7fd2a7e00640 1 -- 172.21.15.107:0/1462996248 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd2a8100860 msgr2=0x7fd2a819fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:12:23.406 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.426+0000 7fd2a7e00640 1 --2- 172.21.15.107:0/1462996248 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd2a8100860 0x7fd2a819fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:12:23.407 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.426+0000 7fd2a7e00640 1 -- 172.21.15.107:0/1462996248 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd2a810da60 msgr2=0x7fd2a8199940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:12:23.407 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.426+0000 7fd2a7e00640 1 --2- 172.21.15.107:0/1462996248 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd2a810da60 0x7fd2a8199940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:12:23.407 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.426+0000 7fd2a7e00640 1 -- 172.21.15.107:0/1462996248 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2a819a0c0 con 0x7fd2a80ffeb0 2024-03-08T21:12:23.407 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.426+0000 7fd2a7400640 1 --2- 172.21.15.107:0/1462996248 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd2a8100860 0x7fd2a819fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T21:12:23.407 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.427+0000 7fd2ad200640 1 --2- 172.21.15.107:0/1462996248 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd2a810da60 0x7fd2a8199940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T21:12:23.407 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.427+0000 7fd2a7e00640 1 --2- 172.21.15.107:0/1462996248 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd2a80ffeb0 0x7fd2a819f740 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fd298005ab0 tx=0x7fd29802caa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:12:23.407 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.427+0000 7fd2a4c00640 1 -- 172.21.15.107:0/1462996248 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd298043020 con 0x7fd2a80ffeb0 2024-03-08T21:12:23.407 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.427+0000 7fd2a4c00640 1 -- 172.21.15.107:0/1462996248 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fd29802cc90 con 0x7fd2a80ffeb0 2024-03-08T21:12:23.407 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.427+0000 7fd2adc00640 1 -- 172.21.15.107:0/1462996248 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2a819a350 con 0x7fd2a80ffeb0 2024-03-08T21:12:23.407 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.427+0000 7fd2a4c00640 1 -- 172.21.15.107:0/1462996248 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd298003d70 con 0x7fd2a80ffeb0 2024-03-08T21:12:23.407 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.427+0000 7fd2adc00640 1 -- 172.21.15.107:0/1462996248 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2a81a6600 con 0x7fd2a80ffeb0 2024-03-08T21:12:23.409 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.431+0000 7fd2a4c00640 1 -- 172.21.15.107:0/1462996248 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) v1 ==== 137422+0+0 (secure 0 0 0) 0x7fd298046020 con 0x7fd2a80ffeb0 2024-03-08T21:12:23.409 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.431+0000 7fd2b1f9a740 1 Processor -- start 2024-03-08T21:12:23.409 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.431+0000 7fd2b1f9a740 1 -- start start 2024-03-08T21:12:23.411 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.432+0000 7fd2b1f9a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f84d224e80 0x55f84d225260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:12:23.411 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.432+0000 7fd2b1f9a740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f84d2257a0 0x55f84d22e030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:12:23.411 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.432+0000 7fd2b1f9a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f84d22e570 0x55f84d230960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:12:23.411 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.432+0000 7fd2b1f9a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f84d1883d0 con 0x55f84d224e80 2024-03-08T21:12:23.411 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.432+0000 7fd2b1f9a740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f84d1316b0 con 0x55f84d2257a0 2024-03-08T21:12:23.412 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.432+0000 7fd2b1f9a740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f84d184470 con 0x55f84d22e570 2024-03-08T21:12:23.412 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.432+0000 7fd2ad200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f84d224e80 0x55f84d225260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:12:23.412 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.432+0000 7fd2ad200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f84d224e80 0x55f84d225260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:58178/0 (socket says 172.21.15.107:58178) 2024-03-08T21:12:23.412 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.432+0000 7fd2ad200640 1 -- 172.21.15.107:0/941498862 learned_addr learned my addr 172.21.15.107:0/941498862 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:12:23.412 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.432+0000 7fd2a7e00640 1 --2- 172.21.15.107:0/941498862 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f84d2257a0 0x55f84d22e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:12:23.412 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.432+0000 7fd2a7400640 1 --2- 172.21.15.107:0/941498862 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f84d22e570 0x55f84d230960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:12:23.412 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.433+0000 7fd2ad200640 1 -- 172.21.15.107:0/941498862 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f84d22e570 msgr2=0x55f84d230960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:12:23.412 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.433+0000 7fd2ad200640 1 --2- 172.21.15.107:0/941498862 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f84d22e570 0x55f84d230960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:12:23.412 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.433+0000 7fd2ad200640 1 -- 172.21.15.107:0/941498862 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f84d2257a0 msgr2=0x55f84d22e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:12:23.412 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.433+0000 7fd2ad200640 1 --2- 172.21.15.107:0/941498862 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f84d2257a0 0x55f84d22e030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:12:23.412 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.433+0000 7fd2ad200640 1 -- 172.21.15.107:0/941498862 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f84d157b50 con 0x55f84d224e80 2024-03-08T21:12:23.412 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.433+0000 7fd2ad200640 1 --2- 172.21.15.107:0/941498862 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f84d224e80 0x55f84d225260 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fd2a0008cf0 tx=0x7fd2a000d710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=204a57b3605e9729 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:12:23.412 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.433+0000 7fd2a4c00640 1 --2- 172.21.15.107:0/1462996248 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7fd28c0a2960 0x7fd28c0a4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:12:23.412 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.433+0000 7fd29d600640 1 -- 172.21.15.107:0/941498862 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2a0018680 con 0x55f84d224e80 2024-03-08T21:12:23.413 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.433+0000 7fd2a4c00640 1 -- 172.21.15.107:0/1462996248 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fd2980f1240 con 0x7fd2a80ffeb0 2024-03-08T21:12:23.413 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.433+0000 7fd29d600640 1 -- 172.21.15.107:0/941498862 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fd2a0018cd0 con 0x55f84d224e80 2024-03-08T21:12:23.413 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.433+0000 7fd29d600640 1 -- 172.21.15.107:0/941498862 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2a000fe20 con 0x55f84d224e80 2024-03-08T21:12:23.413 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.433+0000 7fd2a7e00640 1 --2- 172.21.15.107:0/1462996248 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7fd28c0a2960 0x7fd28c0a4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:12:23.415 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.434+0000 7fd2b1f9a740 1 -- 172.21.15.107:0/941498862 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f84d224e80 msgr2=0x55f84d225260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:12:23.416 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.434+0000 7fd2b1f9a740 1 --2- 172.21.15.107:0/941498862 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f84d224e80 0x55f84d225260 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fd2a0008cf0 tx=0x7fd2a000d710 comp rx=0 tx=0).stop 2024-03-08T21:12:23.416 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.434+0000 7fd2a7e00640 1 --2- 172.21.15.107:0/1462996248 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7fd28c0a2960 0x7fd28c0a4e20 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fd2980eeb10 tx=0x7fd298009260 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:12:23.416 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.434+0000 7fd2b1f9a740 1 -- 172.21.15.107:0/941498862 shutdown_connections 2024-03-08T21:12:23.416 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.434+0000 7fd2b1f9a740 1 --2- 172.21.15.107:0/941498862 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f84d22e570 0x55f84d230960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:12:23.416 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.434+0000 7fd2b1f9a740 1 --2- 172.21.15.107:0/941498862 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f84d2257a0 0x55f84d22e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:12:23.416 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.434+0000 7fd2b1f9a740 1 --2- 172.21.15.107:0/941498862 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f84d224e80 0x55f84d225260 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:12:23.416 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.434+0000 7fd2b1f9a740 1 -- 172.21.15.107:0/941498862 >> 172.21.15.107:0/941498862 conn(0x55f84d21f7a0 msgr2=0x55f84d21fb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:12:23.416 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.435+0000 7fd2b1f9a740 1 -- 172.21.15.107:0/941498862 shutdown_connections 2024-03-08T21:12:23.416 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.435+0000 7fd2b1f9a740 1 -- 172.21.15.107:0/941498862 wait complete. 2024-03-08T21:12:23.416 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.435+0000 7fd2b1f9a740 1 -- start start 2024-03-08T21:12:23.416 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.435+0000 7fd2b1f9a740 1 Processor -- start 2024-03-08T21:12:23.416 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.436+0000 7fd2b1f9a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f84d224e80 0x55f84d22de70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:12:23.417 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.436+0000 7fd2b1f9a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f84d2257a0 0x55f84d22b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:12:23.417 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.436+0000 7fd2b1f9a740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f84d22e570 0x55f84d22b740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:12:23.417 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.436+0000 7fd2b1f9a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f84d2311a0 con 0x55f84d224e80 2024-03-08T21:12:23.417 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.436+0000 7fd2b1f9a740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f84d1449f0 con 0x55f84d22e570 2024-03-08T21:12:23.417 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.436+0000 7fd2b1f9a740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f84d231320 con 0x55f84d2257a0 2024-03-08T21:12:23.417 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.436+0000 7fd2b1f9a740 10 client.0 _ll_register_callbacks cb 0x55f84ce9ca00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T21:12:23.417 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.436+0000 7fd2ad200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f84d224e80 0x55f84d22de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:12:23.417 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.436+0000 7fd2ad200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f84d224e80 0x55f84d22de70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:58196/0 (socket says 172.21.15.107:58196) 2024-03-08T21:12:23.417 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.436+0000 7fd2ad200640 1 -- 172.21.15.107:0/2739544514 learned_addr learned my addr 172.21.15.107:0/2739544514 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:12:23.417 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.436+0000 7fd2a7400640 1 --2- 172.21.15.107:0/2739544514 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f84d2257a0 0x55f84d22b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:12:23.417 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.437+0000 7fd2ad200640 1 -- 172.21.15.107:0/2739544514 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f84d2257a0 msgr2=0x55f84d22b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:12:23.417 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.437+0000 7fd2ad200640 1 --2- 172.21.15.107:0/2739544514 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f84d2257a0 0x55f84d22b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:12:23.417 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.437+0000 7fd2ad200640 1 -- 172.21.15.107:0/2739544514 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f84d22e570 msgr2=0x55f84d22b740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:12:23.417 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.437+0000 7fd2ad200640 1 --2- 172.21.15.107:0/2739544514 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f84d22e570 0x55f84d22b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:12:23.418 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.437+0000 7fd2ad200640 1 -- 172.21.15.107:0/2739544514 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f84ce9d210 con 0x55f84d224e80 2024-03-08T21:12:23.418 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.437+0000 7fd2ad200640 1 --2- 172.21.15.107:0/2739544514 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f84d224e80 0x55f84d22de70 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fd2a000f7b0 tx=0x7fd2a0008a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8c813042b944c11 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:12:23.418 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.437+0000 7fd292000640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-03-08T21:12:23.418 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.437+0000 7fd292000640 1 -- 172.21.15.107:0/2739544514 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2a0018690 con 0x55f84d224e80 2024-03-08T21:12:23.418 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.437+0000 7fd292000640 1 -- 172.21.15.107:0/2739544514 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fd2a0018830 con 0x55f84d224e80 2024-03-08T21:12:23.418 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.437+0000 7fd2b1f9a740 1 -- 172.21.15.107:0/2739544514 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd2a0018cd0 con 0x55f84d224e80 2024-03-08T21:12:23.418 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.438+0000 7fd292000640 1 -- 172.21.15.107:0/2739544514 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd2a0018a30 con 0x55f84d224e80 2024-03-08T21:12:23.418 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.438+0000 7fd2b1f9a740 10 client.16443 fetch_fsmap learned FSMap version 39 2024-03-08T21:12:23.418 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.438+0000 7fd2b1f9a740 1 -- 172.21.15.107:0/2739544514 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x55f84cf69620 con 0x55f84d224e80 2024-03-08T21:12:23.420 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.438+0000 7fd292000640 1 -- 172.21.15.107:0/2739544514 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fd2a0018cd0 con 0x55f84d224e80 2024-03-08T21:12:23.420 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.438+0000 7fd292000640 1 -- 172.21.15.107:0/2739544514 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 39) v1 ==== 7307+0+0 (secure 0 0 0) 0x7fd2a002b030 con 0x55f84d224e80 2024-03-08T21:12:23.420 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.438+0000 7fd2b1f9a740 10 client.16443 fetch_fsmap finished waiting for FSMap version 39 2024-03-08T21:12:23.420 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.438+0000 7fd2b1f9a740 10 client.16443 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-08T21:12:23.421 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.439+0000 7fd2b1f9a740 20 client.16443 populate_metadata read hostname 'smithi107' 2024-03-08T21:12:23.421 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.439+0000 7fd2b1f9a740 1 --2- 172.21.15.107:0/2739544514 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55f84d23e290 0x55f84d240680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:12:23.421 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.439+0000 7fd2b1f9a740 4 client.16443 mds_command: new command op to 34475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-08T21:12:23.421 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.439+0000 7fd2b1f9a740 1 -- 172.21.15.107:0/2739544514 --> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f84d195910 con 0x55f84d23e290 2024-03-08T21:12:23.421 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.439+0000 7fd2a7400640 1 --2- 172.21.15.107:0/2739544514 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55f84d23e290 0x55f84d240680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:12:23.421 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.440+0000 7fd2a7400640 1 --2- 172.21.15.107:0/2739544514 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55f84d23e290 0x55f84d240680 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1aaaa55a87f178aa server_cookie=76283e593f8b93d1 in_seq=0 out_seq=0 2024-03-08T21:12:23.421 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.440+0000 7fd292000640 10 client.16443 ms_handle_connect on v2:172.21.15.149:6838/55290596 2024-03-08T21:12:23.421 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.442+0000 7fd292000640 1 -- 172.21.15.107:0/2739544514 <== mds.0 v2:172.21.15.149:6838/55290596 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f84d195910 con 0x55f84d23e290 2024-03-08T21:12:23.421 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.442+0000 7fd292000640 10 client.16443 handle_command_reply: tid=0 2024-03-08T21:12:23.421 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.442+0000 7fd2b1f9a740 1 client.16443 shutdown 2024-03-08T21:12:23.421 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.442+0000 7fd2b1f9a740 1 -- 172.21.15.107:0/2739544514 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f84d224e80 msgr2=0x55f84d22de70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:12:23.421 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.442+0000 7fd2b1f9a740 1 --2- 172.21.15.107:0/2739544514 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f84d224e80 0x55f84d22de70 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fd2a000f7b0 tx=0x7fd2a0008a60 comp rx=0 tx=0).stop 2024-03-08T21:12:23.421 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.442+0000 7fd2b1f9a740 1 -- 172.21.15.107:0/2739544514 shutdown_connections 2024-03-08T21:12:23.422 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.442+0000 7fd2b1f9a740 1 --2- 172.21.15.107:0/2739544514 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55f84d23e290 0x55f84d240680 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:12:23.422 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.442+0000 7fd2b1f9a740 1 --2- 172.21.15.107:0/2739544514 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f84d22e570 0x55f84d22b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:12:23.422 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.442+0000 7fd2b1f9a740 1 --2- 172.21.15.107:0/2739544514 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f84d2257a0 0x55f84d22b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:12:23.422 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.442+0000 7fd2b1f9a740 1 --2- 172.21.15.107:0/2739544514 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f84d224e80 0x55f84d22de70 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:12:23.422 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.442+0000 7fd2b1f9a740 1 -- 172.21.15.107:0/2739544514 >> 172.21.15.107:0/2739544514 conn(0x55f84d21f7a0 msgr2=0x55f84d2214f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:12:23.422 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.442+0000 7fd292000640 0 client.16443 ms_handle_reset on v2:172.21.15.149:6838/55290596 2024-03-08T21:12:23.422 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.442+0000 7fd2b1f9a740 1 -- 172.21.15.107:0/2739544514 shutdown_connections 2024-03-08T21:12:23.422 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.442+0000 7fd2b1f9a740 1 -- 172.21.15.107:0/2739544514 wait complete. 2024-03-08T21:12:23.422 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.442+0000 7fd2b1f9a740 20 client.16443 trim_cache size 0 max 16384 2024-03-08T21:12:23.435 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.457+0000 7fd2b1f9a740 1 Processor -- start 2024-03-08T21:12:23.435 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.457+0000 7fd2b1f9a740 1 -- start start 2024-03-08T21:12:23.435 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.457+0000 7fd2b1f9a740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f84d224e80 0x55f84d22d930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:12:23.435 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.457+0000 7fd2b1f9a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f84d2257a0 0x55f84d22de70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:12:23.435 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.457+0000 7fd2b1f9a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f84d22e570 0x55f84d228460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:12:23.435 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.457+0000 7fd2b1f9a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2a81b0640 con 0x55f84d22e570 2024-03-08T21:12:23.435 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.457+0000 7fd2b1f9a740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2a81ae660 con 0x55f84d224e80 2024-03-08T21:12:23.436 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.457+0000 7fd2b1f9a740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2a81affe0 con 0x55f84d2257a0 2024-03-08T21:12:23.437 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.458+0000 7fd2ad200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f84d22e570 0x55f84d228460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:12:23.437 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.458+0000 7fd2ad200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f84d22e570 0x55f84d228460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:58198/0 (socket says 172.21.15.107:58198) 2024-03-08T21:12:23.437 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.458+0000 7fd2ad200640 1 -- 172.21.15.107:0/4035913047 learned_addr learned my addr 172.21.15.107:0/4035913047 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:12:23.437 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.458+0000 7fd2ad200640 1 --2- 172.21.15.107:0/4035913047 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f84d224e80 0x55f84d22d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:12:23.437 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.458+0000 7fd2a7400640 1 --2- 172.21.15.107:0/4035913047 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f84d2257a0 0x55f84d22de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:12:23.437 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.458+0000 7fd2ad200640 1 -- 172.21.15.107:0/4035913047 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f84d2257a0 msgr2=0x55f84d22de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:12:23.437 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.458+0000 7fd2ad200640 1 --2- 172.21.15.107:0/4035913047 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f84d2257a0 0x55f84d22de70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:12:23.437 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.458+0000 7fd2ad200640 1 -- 172.21.15.107:0/4035913047 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f84d224e80 msgr2=0x55f84d22d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:12:23.437 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.458+0000 7fd2ad200640 1 --2- 172.21.15.107:0/4035913047 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f84d224e80 0x55f84d22d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:12:23.437 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.458+0000 7fd2ad200640 1 -- 172.21.15.107:0/4035913047 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f84d220530 con 0x55f84d22e570 2024-03-08T21:12:23.437 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.459+0000 7fd2ad200640 1 --2- 172.21.15.107:0/4035913047 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f84d22e570 0x55f84d228460 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fd2a000fef0 tx=0x7fd2a0005c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abb631d25e2bdbac server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:12:23.438 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.459+0000 7fd29cc00640 1 -- 172.21.15.107:0/4035913047 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2a003c070 con 0x55f84d22e570 2024-03-08T21:12:23.438 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.459+0000 7fd29cc00640 1 -- 172.21.15.107:0/4035913047 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fd2a000f940 con 0x55f84d22e570 2024-03-08T21:12:23.439 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.459+0000 7fd29cc00640 1 -- 172.21.15.107:0/4035913047 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2a0018bd0 con 0x55f84d22e570 2024-03-08T21:12:23.439 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.460+0000 7fd2b1f9a740 1 -- 172.21.15.107:0/4035913047 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f84d22e570 msgr2=0x55f84d228460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:12:23.439 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.460+0000 7fd2b1f9a740 1 --2- 172.21.15.107:0/4035913047 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f84d22e570 0x55f84d228460 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fd2a000fef0 tx=0x7fd2a0005c80 comp rx=0 tx=0).stop 2024-03-08T21:12:23.439 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.460+0000 7fd2b1f9a740 1 -- 172.21.15.107:0/4035913047 shutdown_connections 2024-03-08T21:12:23.439 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.460+0000 7fd2b1f9a740 1 --2- 172.21.15.107:0/4035913047 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f84d22e570 0x55f84d228460 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:12:23.439 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.460+0000 7fd2b1f9a740 1 --2- 172.21.15.107:0/4035913047 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f84d2257a0 0x55f84d22de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:12:23.439 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.460+0000 7fd2b1f9a740 1 --2- 172.21.15.107:0/4035913047 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f84d224e80 0x55f84d22d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:12:23.439 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.460+0000 7fd2b1f9a740 1 -- 172.21.15.107:0/4035913047 >> 172.21.15.107:0/4035913047 conn(0x55f84d21f7a0 msgr2=0x55f84d221480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:12:23.439 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.460+0000 7fd2b1f9a740 1 -- 172.21.15.107:0/4035913047 shutdown_connections 2024-03-08T21:12:23.439 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.460+0000 7fd2b1f9a740 1 -- 172.21.15.107:0/4035913047 wait complete. 2024-03-08T21:12:23.439 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.461+0000 7fd2b1f9a740 1 -- start start 2024-03-08T21:12:23.442 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.461+0000 7fd2b1f9a740 1 Processor -- start 2024-03-08T21:12:23.442 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.461+0000 7fd2b1f9a740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f84d224e80 0x55f84d2357a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:12:23.442 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.461+0000 7fd2b1f9a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f84d2257a0 0x55f84d233b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:12:23.442 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.461+0000 7fd2b1f9a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f84d22e570 0x55f84d234080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:12:23.442 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.461+0000 7fd2b1f9a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2a81af320 con 0x55f84d2257a0 2024-03-08T21:12:23.442 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.461+0000 7fd2b1f9a740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2a81aecc0 con 0x55f84d224e80 2024-03-08T21:12:23.442 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.461+0000 7fd2b1f9a740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2a81af980 con 0x55f84d22e570 2024-03-08T21:12:23.442 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.461+0000 7fd2b1f9a740 10 client.0 _ll_register_callbacks cb 0x7fd2a00084b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T21:12:23.442 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.462+0000 7fd2a7400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f84d2257a0 0x55f84d233b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:12:23.443 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.462+0000 7fd2a7400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f84d2257a0 0x55f84d233b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:58214/0 (socket says 172.21.15.107:58214) 2024-03-08T21:12:23.443 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.462+0000 7fd2a7400640 1 -- 172.21.15.107:0/348556813 learned_addr learned my addr 172.21.15.107:0/348556813 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:12:23.443 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.462+0000 7fd2ad200640 1 --2- 172.21.15.107:0/348556813 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f84d224e80 0x55f84d2357a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:12:23.443 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.462+0000 7fd2a7400640 1 -- 172.21.15.107:0/348556813 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f84d22e570 msgr2=0x55f84d234080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:12:23.443 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.462+0000 7fd2ad200640 1 --2- 172.21.15.107:0/348556813 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f84d22e570 0x55f84d234080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:12:23.443 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.462+0000 7fd2a7400640 1 --2- 172.21.15.107:0/348556813 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f84d22e570 0x55f84d234080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:12:23.443 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.462+0000 7fd2a7400640 1 -- 172.21.15.107:0/348556813 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f84d224e80 msgr2=0x55f84d2357a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:12:23.443 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.462+0000 7fd2a7400640 1 --2- 172.21.15.107:0/348556813 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f84d224e80 0x55f84d2357a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:12:23.443 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.462+0000 7fd2a7400640 1 -- 172.21.15.107:0/348556813 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f84d2346f0 con 0x55f84d2257a0 2024-03-08T21:12:23.443 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.462+0000 7fd2a7400640 1 --2- 172.21.15.107:0/348556813 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f84d2257a0 0x55f84d233b40 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fd2a810e3e0 tx=0x7fd294018590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28834cb76636c066 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:12:23.443 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.462+0000 7fd292000640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-03-08T21:12:23.443 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.463+0000 7fd292000640 1 -- 172.21.15.107:0/348556813 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd294025060 con 0x55f84d2257a0 2024-03-08T21:12:23.443 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.463+0000 7fd292000640 1 -- 172.21.15.107:0/348556813 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fd294010900 con 0x55f84d2257a0 2024-03-08T21:12:23.443 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.463+0000 7fd2b1f9a740 1 -- 172.21.15.107:0/348556813 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd2a000f940 con 0x55f84d2257a0 2024-03-08T21:12:23.444 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.463+0000 7fd292000640 1 -- 172.21.15.107:0/348556813 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd294010b20 con 0x55f84d2257a0 2024-03-08T21:12:23.444 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.463+0000 7fd2b1f9a740 10 client.16449 fetch_fsmap learned FSMap version 39 2024-03-08T21:12:23.444 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.463+0000 7fd2b1f9a740 1 -- 172.21.15.107:0/348556813 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x55f84d295660 con 0x55f84d2257a0 2024-03-08T21:12:23.444 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.464+0000 7fd292000640 1 -- 172.21.15.107:0/348556813 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fd2a000f940 con 0x55f84d2257a0 2024-03-08T21:12:23.444 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.464+0000 7fd292000640 1 -- 172.21.15.107:0/348556813 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 39) v1 ==== 7307+0+0 (secure 0 0 0) 0x7fd29400e3b0 con 0x55f84d2257a0 2024-03-08T21:12:23.444 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.464+0000 7fd2b1f9a740 10 client.16449 fetch_fsmap finished waiting for FSMap version 39 2024-03-08T21:12:23.444 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.464+0000 7fd2b1f9a740 10 client.16449 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-08T21:12:23.444 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.464+0000 7fd2b1f9a740 20 client.16449 populate_metadata read hostname 'smithi107' 2024-03-08T21:12:23.444 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.464+0000 7fd2b1f9a740 1 --2- 172.21.15.107:0/348556813 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55f84d295d20 0x55f84d298140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:12:23.444 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.464+0000 7fd2b1f9a740 4 client.16449 mds_command: new command op to 34475 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-08T21:12:23.444 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.464+0000 7fd2b1f9a740 1 -- 172.21.15.107:0/348556813 --> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55f84d18fc60 con 0x55f84d295d20 2024-03-08T21:12:23.446 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.464+0000 7fd2ad200640 1 --2- 172.21.15.107:0/348556813 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55f84d295d20 0x55f84d298140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:12:23.446 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.465+0000 7fd2ad200640 1 --2- 172.21.15.107:0/348556813 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55f84d295d20 0x55f84d298140 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4900e88d1144b13d server_cookie=1f0e67d8594e2f97 in_seq=0 out_seq=0 2024-03-08T21:12:23.446 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.465+0000 7fd292000640 10 client.16449 ms_handle_connect on v2:172.21.15.149:6838/55290596 2024-03-08T21:12:23.454 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.475+0000 7fd292000640 1 -- 172.21.15.107:0/348556813 <== mds.0 v2:172.21.15.149:6838/55290596 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55f84d18fc60 con 0x55f84d295d20 2024-03-08T21:12:23.454 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.475+0000 7fd292000640 10 client.16449 handle_command_reply: tid=0 2024-03-08T21:12:23.454 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.475+0000 7fd2b1f9a740 1 client.16449 shutdown 2024-03-08T21:12:23.454 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.475+0000 7fd2b1f9a740 1 -- 172.21.15.107:0/348556813 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f84d2257a0 msgr2=0x55f84d233b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:12:23.454 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.475+0000 7fd2b1f9a740 1 --2- 172.21.15.107:0/348556813 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f84d2257a0 0x55f84d233b40 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fd2a810e3e0 tx=0x7fd294018590 comp rx=0 tx=0).stop 2024-03-08T21:12:23.454 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.475+0000 7fd2b1f9a740 1 -- 172.21.15.107:0/348556813 shutdown_connections 2024-03-08T21:12:23.454 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.476+0000 7fd2b1f9a740 1 --2- 172.21.15.107:0/348556813 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55f84d22e570 0x55f84d234080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:12:23.454 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.476+0000 7fd2b1f9a740 1 --2- 172.21.15.107:0/348556813 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f84d2257a0 0x55f84d233b40 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:12:23.454 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.476+0000 7fd2b1f9a740 1 --2- 172.21.15.107:0/348556813 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55f84d295d20 0x55f84d298140 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:12:23.454 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.476+0000 7fd2b1f9a740 1 --2- 172.21.15.107:0/348556813 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f84d224e80 0x55f84d2357a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:12:23.454 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.476+0000 7fd292000640 0 client.16449 ms_handle_reset on v2:172.21.15.149:6838/55290596 2024-03-08T21:12:23.455 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.476+0000 7fd2b1f9a740 1 -- 172.21.15.107:0/348556813 >> 172.21.15.107:0/348556813 conn(0x55f84d21f7a0 msgr2=0x55f84d21df70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:12:23.455 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.476+0000 7fd2b1f9a740 1 -- 172.21.15.107:0/348556813 shutdown_connections 2024-03-08T21:12:23.455 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.476+0000 7fd2b1f9a740 1 -- 172.21.15.107:0/348556813 wait complete. 2024-03-08T21:12:23.455 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.476+0000 7fd2b1f9a740 20 client.16449 trim_cache size 0 max 16384 2024-03-08T21:12:23.455 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:{ 2024-03-08T21:12:23.455 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "message": "", 2024-03-08T21:12:23.455 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "return_code": 0 2024-03-08T21:12:23.455 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:} 2024-03-08T21:12:23.456 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.476+0000 7fd2adc00640 1 -- 172.21.15.107:0/1462996248 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7fd28c0a2960 msgr2=0x7fd28c0a4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:12:23.456 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.477+0000 7fd2adc00640 1 --2- 172.21.15.107:0/1462996248 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7fd28c0a2960 0x7fd28c0a4e20 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fd2980eeb10 tx=0x7fd298009260 comp rx=0 tx=0).stop 2024-03-08T21:12:23.456 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.477+0000 7fd2adc00640 1 -- 172.21.15.107:0/1462996248 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd2a80ffeb0 msgr2=0x7fd2a819f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:12:23.456 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.477+0000 7fd2adc00640 1 --2- 172.21.15.107:0/1462996248 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd2a80ffeb0 0x7fd2a819f740 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fd298005ab0 tx=0x7fd29802caa0 comp rx=0 tx=0).stop 2024-03-08T21:12:23.456 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.477+0000 7fd2adc00640 1 -- 172.21.15.107:0/1462996248 shutdown_connections 2024-03-08T21:12:23.456 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.477+0000 7fd2adc00640 1 --2- 172.21.15.107:0/1462996248 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7fd28c0a2960 0x7fd28c0a4e20 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:12:23.456 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.477+0000 7fd2adc00640 1 --2- 172.21.15.107:0/1462996248 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd2a810da60 0x7fd2a8199940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:12:23.456 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.477+0000 7fd2adc00640 1 --2- 172.21.15.107:0/1462996248 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fd2a8100860 0x7fd2a819fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:12:23.456 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.477+0000 7fd2adc00640 1 --2- 172.21.15.107:0/1462996248 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd2a80ffeb0 0x7fd2a819f740 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:12:23.457 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.477+0000 7fd2adc00640 1 -- 172.21.15.107:0/1462996248 >> 172.21.15.107:0/1462996248 conn(0x7fd2a80fbb00 msgr2=0x7fd2a80fdef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:12:23.457 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.477+0000 7fd2adc00640 1 -- 172.21.15.107:0/1462996248 shutdown_connections 2024-03-08T21:12:23.457 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:23.477+0000 7fd2adc00640 1 -- 172.21.15.107:0/1462996248 wait complete. 2024-03-08T21:12:24.622 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:12:24 smithi204 ceph-mon[34935]: pgmap v375: 97 pgs: 97 active+clean; 8.1 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 3.6 KiB/s wr, 13 op/s 2024-03-08T21:12:24.711 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:12:24 smithi107 ceph-mon[31381]: pgmap v375: 97 pgs: 97 active+clean; 8.1 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 3.6 KiB/s wr, 13 op/s 2024-03-08T21:12:24.759 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:12:24 smithi149 ceph-mon[35042]: pgmap v375: 97 pgs: 97 active+clean; 8.1 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 3.6 KiB/s wr, 13 op/s 2024-03-08T21:12:26.622 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:12:26 smithi204 ceph-mon[34935]: pgmap v376: 97 pgs: 97 active+clean; 8.1 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 307 B/s wr, 10 op/s 2024-03-08T21:12:26.711 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:12:26 smithi107 ceph-mon[31381]: pgmap v376: 97 pgs: 97 active+clean; 8.1 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 307 B/s wr, 10 op/s 2024-03-08T21:12:26.759 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:12:26 smithi149 ceph-mon[35042]: pgmap v376: 97 pgs: 97 active+clean; 8.1 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 307 B/s wr, 10 op/s 2024-03-08T21:12:28.622 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:12:28 smithi204 ceph-mon[34935]: pgmap v377: 97 pgs: 97 active+clean; 8.1 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 307 B/s wr, 10 op/s 2024-03-08T21:12:28.711 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:12:28 smithi107 ceph-mon[31381]: pgmap v377: 97 pgs: 97 active+clean; 8.1 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 307 B/s wr, 10 op/s 2024-03-08T21:12:28.759 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:12:28 smithi149 ceph-mon[35042]: pgmap v377: 97 pgs: 97 active+clean; 8.1 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 307 B/s wr, 10 op/s 2024-03-08T21:12:30.622 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:12:30 smithi204 ceph-mon[34935]: pgmap v378: 97 pgs: 97 active+clean; 8.1 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 307 B/s wr, 10 op/s 2024-03-08T21:12:30.712 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:12:30 smithi107 ceph-mon[31381]: pgmap v378: 97 pgs: 97 active+clean; 8.1 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 307 B/s wr, 10 op/s 2024-03-08T21:12:30.759 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:12:30 smithi149 ceph-mon[35042]: pgmap v378: 97 pgs: 97 active+clean; 8.1 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 307 B/s wr, 10 op/s 2024-03-08T21:12:32.622 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:12:32 smithi204 ceph-mon[34935]: pgmap v379: 97 pgs: 97 active+clean; 8.1 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 266 B/s wr, 9 op/s 2024-03-08T21:12:32.711 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:12:32 smithi107 ceph-mon[31381]: pgmap v379: 97 pgs: 97 active+clean; 8.1 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 266 B/s wr, 9 op/s 2024-03-08T21:12:32.759 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:12:32 smithi149 ceph-mon[35042]: pgmap v379: 97 pgs: 97 active+clean; 8.1 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 266 B/s wr, 9 op/s 2024-03-08T21:12:34.622 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:12:34 smithi204 ceph-mon[34935]: pgmap v380: 97 pgs: 97 active+clean; 8.1 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:12:34.711 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:12:34 smithi107 ceph-mon[31381]: pgmap v380: 97 pgs: 97 active+clean; 8.1 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:12:34.759 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:12:34 smithi149 ceph-mon[35042]: pgmap v380: 97 pgs: 97 active+clean; 8.1 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:12:36.600 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.621+0000 7f3d2a200640 1 -- 172.21.15.107:0/2088088716 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3d241066b0 msgr2=0x7f3d2410e1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:12:36.600 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.621+0000 7f3d2a200640 1 --2- 172.21.15.107:0/2088088716 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3d241066b0 0x7f3d2410e1f0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f3d1c009ca0 tx=0x7f3d1c02c0f0 comp rx=0 tx=0).stop 2024-03-08T21:12:36.600 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.621+0000 7f3d2a200640 1 -- 172.21.15.107:0/2088088716 shutdown_connections 2024-03-08T21:12:36.600 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.621+0000 7f3d2a200640 1 --2- 172.21.15.107:0/2088088716 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3d241066b0 0x7f3d2410e1f0 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:12:36.601 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.621+0000 7f3d2a200640 1 --2- 172.21.15.107:0/2088088716 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3d2406a710 0x7f3d24105f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:12:36.601 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.621+0000 7f3d2a200640 1 --2- 172.21.15.107:0/2088088716 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3d24069d60 0x7f3d2406a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:12:36.601 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.621+0000 7f3d2a200640 1 -- 172.21.15.107:0/2088088716 >> 172.21.15.107:0/2088088716 conn(0x7f3d24076fc0 msgr2=0x7f3d240773d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:12:36.601 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.621+0000 7f3d2a200640 1 -- 172.21.15.107:0/2088088716 shutdown_connections 2024-03-08T21:12:36.601 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.622+0000 7f3d2a200640 1 -- 172.21.15.107:0/2088088716 wait complete. 2024-03-08T21:12:36.601 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.622+0000 7f3d2a200640 1 Processor -- start 2024-03-08T21:12:36.603 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.623+0000 7f3d2a200640 1 -- start start 2024-03-08T21:12:36.604 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.623+0000 7f3d2a200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3d24069d60 0x7f3d2419f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:12:36.604 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.623+0000 7f3d2a200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3d241066b0 0x7f3d2419fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:12:36.604 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.623+0000 7f3d2a200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3d24199a10 0x7f3d24199ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:12:36.604 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.623+0000 7f3d2a200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d24111590 con 0x7f3d24069d60 2024-03-08T21:12:36.604 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.623+0000 7f3d2a200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d24111410 con 0x7f3d241066b0 2024-03-08T21:12:36.604 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.623+0000 7f3d2a200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d24111710 con 0x7f3d24199a10 2024-03-08T21:12:36.604 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.623+0000 7f3d23400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3d24069d60 0x7f3d2419f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:12:36.604 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.623+0000 7f3d23400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3d24069d60 0x7f3d2419f810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:50332/0 (socket says 172.21.15.107:50332) 2024-03-08T21:12:36.604 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.623+0000 7f3d23400640 1 -- 172.21.15.107:0/1483725237 learned_addr learned my addr 172.21.15.107:0/1483725237 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:12:36.604 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.624+0000 7f3d1aa00640 1 --2- 172.21.15.107:0/1483725237 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3d241066b0 0x7f3d2419fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:12:36.605 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.624+0000 7f3d23e00640 1 --2- 172.21.15.107:0/1483725237 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3d24199a10 0x7f3d24199ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:12:36.605 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.624+0000 7f3d23400640 1 -- 172.21.15.107:0/1483725237 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3d24199a10 msgr2=0x7f3d24199ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:12:36.605 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.624+0000 7f3d23400640 1 --2- 172.21.15.107:0/1483725237 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3d24199a10 0x7f3d24199ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:12:36.605 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.624+0000 7f3d23400640 1 -- 172.21.15.107:0/1483725237 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3d241066b0 msgr2=0x7f3d2419fd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:12:36.605 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.624+0000 7f3d23400640 1 --2- 172.21.15.107:0/1483725237 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3d241066b0 0x7f3d2419fd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:12:36.605 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.624+0000 7f3d23400640 1 -- 172.21.15.107:0/1483725237 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d2419a530 con 0x7f3d24069d60 2024-03-08T21:12:36.605 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.624+0000 7f3d23400640 1 --2- 172.21.15.107:0/1483725237 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3d24069d60 0x7f3d2419f810 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f3d1400c890 tx=0x7f3d1400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:12:36.605 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.624+0000 7f3d20c00640 1 -- 172.21.15.107:0/1483725237 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d14004220 con 0x7f3d24069d60 2024-03-08T21:12:36.605 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.625+0000 7f3d2a200640 1 -- 172.21.15.107:0/1483725237 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d2419a820 con 0x7f3d24069d60 2024-03-08T21:12:36.606 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.625+0000 7f3d20c00640 1 -- 172.21.15.107:0/1483725237 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f3d140043c0 con 0x7f3d24069d60 2024-03-08T21:12:36.606 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.625+0000 7f3d20c00640 1 -- 172.21.15.107:0/1483725237 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d14004a00 con 0x7f3d24069d60 2024-03-08T21:12:36.606 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.625+0000 7f3d2a200640 1 -- 172.21.15.107:0/1483725237 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d241a6a20 con 0x7f3d24069d60 2024-03-08T21:12:36.608 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.628+0000 7f3d20c00640 1 -- 172.21.15.107:0/1483725237 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) v1 ==== 137422+0+0 (secure 0 0 0) 0x7f3d14004ba0 con 0x7f3d24069d60 2024-03-08T21:12:36.608 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.628+0000 7f3d2cfd9740 1 Processor -- start 2024-03-08T21:12:36.609 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.628+0000 7f3d2cfd9740 1 -- start start 2024-03-08T21:12:36.609 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.629+0000 7f3d2cfd9740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56266db03c80 0x56266db04060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:12:36.609 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.629+0000 7f3d2cfd9740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56266db045a0 0x56266db0ce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:12:36.609 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.629+0000 7f3d2cfd9740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56266db0d370 0x56266db0f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:12:36.609 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.629+0000 7f3d2cfd9740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x56266da673d0 con 0x56266db03c80 2024-03-08T21:12:36.609 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.629+0000 7f3d2cfd9740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x56266da106b0 con 0x56266db045a0 2024-03-08T21:12:36.609 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.629+0000 7f3d2cfd9740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56266da63470 con 0x56266db0d370 2024-03-08T21:12:36.609 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.629+0000 7f3d23e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56266db03c80 0x56266db04060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:12:36.609 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.629+0000 7f3d23e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56266db03c80 0x56266db04060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:50348/0 (socket says 172.21.15.107:50348) 2024-03-08T21:12:36.609 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.629+0000 7f3d23e00640 1 -- 172.21.15.107:0/920991395 learned_addr learned my addr 172.21.15.107:0/920991395 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:12:36.610 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.629+0000 7f3d23400640 1 --2- 172.21.15.107:0/920991395 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56266db045a0 0x56266db0ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:12:36.610 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.629+0000 7f3d1aa00640 1 --2- 172.21.15.107:0/920991395 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56266db0d370 0x56266db0f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:12:36.610 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.630+0000 7f3d23e00640 1 -- 172.21.15.107:0/920991395 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56266db0d370 msgr2=0x56266db0f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:12:36.610 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.630+0000 7f3d23e00640 1 --2- 172.21.15.107:0/920991395 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56266db0d370 0x56266db0f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:12:36.610 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.630+0000 7f3d23e00640 1 -- 172.21.15.107:0/920991395 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56266db045a0 msgr2=0x56266db0ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:12:36.610 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.630+0000 7f3d23e00640 1 --2- 172.21.15.107:0/920991395 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56266db045a0 0x56266db0ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:12:36.610 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.630+0000 7f3d23e00640 1 -- 172.21.15.107:0/920991395 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56266da36b50 con 0x56266db03c80 2024-03-08T21:12:36.610 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.630+0000 7f3d20c00640 1 --2- 172.21.15.107:0/1483725237 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f3d040a2a10 0x7f3d040a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:12:36.610 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.630+0000 7f3d20c00640 1 -- 172.21.15.107:0/1483725237 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f3d140cec30 con 0x7f3d24069d60 2024-03-08T21:12:36.610 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.630+0000 7f3d23e00640 1 --2- 172.21.15.107:0/920991395 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56266db03c80 0x56266db04060 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f3d1c009ca0 tx=0x7f3d1c009800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b3c998a8216f12f server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:12:36.611 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.630+0000 7f3d23400640 1 --2- 172.21.15.107:0/1483725237 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f3d040a2a10 0x7f3d040a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:12:36.611 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.630+0000 7f3d13e00640 1 -- 172.21.15.107:0/920991395 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d1c043020 con 0x56266db03c80 2024-03-08T21:12:36.611 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.631+0000 7f3d13e00640 1 -- 172.21.15.107:0/920991395 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f3d1c032db0 con 0x56266db03c80 2024-03-08T21:12:36.611 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.631+0000 7f3d13e00640 1 -- 172.21.15.107:0/920991395 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d1c039040 con 0x56266db03c80 2024-03-08T21:12:36.615 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.632+0000 7f3d2cfd9740 1 -- 172.21.15.107:0/920991395 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56266db03c80 msgr2=0x56266db04060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:12:36.615 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.632+0000 7f3d23400640 1 --2- 172.21.15.107:0/1483725237 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f3d040a2a10 0x7f3d040a4ed0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f3d140cc3f0 tx=0x7f3d14000bc0 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:12:36.615 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.632+0000 7f3d2cfd9740 1 --2- 172.21.15.107:0/920991395 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56266db03c80 0x56266db04060 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f3d1c009ca0 tx=0x7f3d1c009800 comp rx=0 tx=0).stop 2024-03-08T21:12:36.615 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.632+0000 7f3d2cfd9740 1 -- 172.21.15.107:0/920991395 shutdown_connections 2024-03-08T21:12:36.615 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.632+0000 7f3d2cfd9740 1 --2- 172.21.15.107:0/920991395 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56266db0d370 0x56266db0f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:12:36.615 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.632+0000 7f3d2cfd9740 1 --2- 172.21.15.107:0/920991395 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56266db045a0 0x56266db0ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:12:36.615 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.632+0000 7f3d2cfd9740 1 --2- 172.21.15.107:0/920991395 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56266db03c80 0x56266db04060 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:12:36.615 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.632+0000 7f3d2cfd9740 1 -- 172.21.15.107:0/920991395 >> 172.21.15.107:0/920991395 conn(0x56266dafe5a0 msgr2=0x56266dafe980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:12:36.616 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.632+0000 7f3d2cfd9740 1 -- 172.21.15.107:0/920991395 shutdown_connections 2024-03-08T21:12:36.616 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.632+0000 7f3d2cfd9740 1 -- 172.21.15.107:0/920991395 wait complete. 2024-03-08T21:12:36.616 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.632+0000 7f3d2cfd9740 1 -- start start 2024-03-08T21:12:36.616 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.633+0000 7f3d2cfd9740 1 Processor -- start 2024-03-08T21:12:36.616 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.633+0000 7f3d2cfd9740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56266db03c80 0x56266db0cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:12:36.616 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.633+0000 7f3d2cfd9740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56266db045a0 0x56266db0a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:12:36.616 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.633+0000 7f3d2cfd9740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56266db0d370 0x56266db0a650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:12:36.616 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.633+0000 7f3d2cfd9740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x56266db0ff40 con 0x56266db0d370 2024-03-08T21:12:36.616 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.633+0000 7f3d2cfd9740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x56266da239f0 con 0x56266db045a0 2024-03-08T21:12:36.616 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.633+0000 7f3d2cfd9740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x56266db100c0 con 0x56266db03c80 2024-03-08T21:12:36.616 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.633+0000 7f3d2cfd9740 10 client.0 _ll_register_callbacks cb 0x56266d77ba00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T21:12:36.617 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.634+0000 7f3d23e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56266db0d370 0x56266db0a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:12:36.617 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.634+0000 7f3d23e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56266db0d370 0x56266db0a650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:50354/0 (socket says 172.21.15.107:50354) 2024-03-08T21:12:36.617 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.634+0000 7f3d23e00640 1 -- 172.21.15.107:0/3287139769 learned_addr learned my addr 172.21.15.107:0/3287139769 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:12:36.617 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.634+0000 7f3d23e00640 1 --2- 172.21.15.107:0/3287139769 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56266db03c80 0x56266db0cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:12:36.617 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.634+0000 7f3d1aa00640 1 --2- 172.21.15.107:0/3287139769 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56266db045a0 0x56266db0a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:12:36.617 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.634+0000 7f3d23e00640 1 -- 172.21.15.107:0/3287139769 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56266db03c80 msgr2=0x56266db0cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:12:36.617 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.634+0000 7f3d23e00640 1 --2- 172.21.15.107:0/3287139769 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56266db03c80 0x56266db0cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:12:36.617 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.634+0000 7f3d23e00640 1 -- 172.21.15.107:0/3287139769 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56266db045a0 msgr2=0x56266db0a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:12:36.617 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.634+0000 7f3d23e00640 1 --2- 172.21.15.107:0/3287139769 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56266db045a0 0x56266db0a0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:12:36.617 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.634+0000 7f3d23e00640 1 -- 172.21.15.107:0/3287139769 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56266d77c210 con 0x56266db0d370 2024-03-08T21:12:36.617 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.634+0000 7f3d23e00640 1 --2- 172.21.15.107:0/3287139769 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56266db0d370 0x56266db0a650 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f3d1c009bf0 tx=0x7f3d1c003e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46d5524e22c903cd server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:12:36.617 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.634+0000 7f3d10c00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-03-08T21:12:36.618 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.635+0000 7f3d10c00640 1 -- 172.21.15.107:0/3287139769 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d1c043040 con 0x56266db0d370 2024-03-08T21:12:36.618 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.635+0000 7f3d10c00640 1 -- 172.21.15.107:0/3287139769 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f3d1c032900 con 0x56266db0d370 2024-03-08T21:12:36.618 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.635+0000 7f3d2cfd9740 1 -- 172.21.15.107:0/3287139769 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3d1c032db0 con 0x56266db0d370 2024-03-08T21:12:36.618 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.635+0000 7f3d10c00640 1 -- 172.21.15.107:0/3287139769 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3d1c039040 con 0x56266db0d370 2024-03-08T21:12:36.618 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.635+0000 7f3d2cfd9740 10 client.16461 fetch_fsmap learned FSMap version 39 2024-03-08T21:12:36.618 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.635+0000 7f3d2cfd9740 1 -- 172.21.15.107:0/3287139769 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x56266d848620 con 0x56266db0d370 2024-03-08T21:12:36.618 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.635+0000 7f3d10c00640 1 -- 172.21.15.107:0/3287139769 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f3d1c032db0 con 0x56266db0d370 2024-03-08T21:12:36.618 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.636+0000 7f3d10c00640 1 -- 172.21.15.107:0/3287139769 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 39) v1 ==== 7307+0+0 (secure 0 0 0) 0x7f3d1c03f980 con 0x56266db0d370 2024-03-08T21:12:36.618 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.636+0000 7f3d2cfd9740 10 client.16461 fetch_fsmap finished waiting for FSMap version 39 2024-03-08T21:12:36.618 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.636+0000 7f3d2cfd9740 10 client.16461 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-08T21:12:36.618 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.636+0000 7f3d2cfd9740 20 client.16461 populate_metadata read hostname 'smithi107' 2024-03-08T21:12:36.619 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.636+0000 7f3d2cfd9740 1 --2- 172.21.15.107:0/3287139769 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x56266db1d030 0x56266db1f420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:12:36.619 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.636+0000 7f3d2cfd9740 4 client.16461 mds_command: new command op to 34475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-08T21:12:36.619 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.636+0000 7f3d2cfd9740 1 -- 172.21.15.107:0/3287139769 --> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56266da74910 con 0x56266db1d030 2024-03-08T21:12:36.619 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.637+0000 7f3d1aa00640 1 --2- 172.21.15.107:0/3287139769 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x56266db1d030 0x56266db1f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:12:36.620 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.638+0000 7f3d1aa00640 1 --2- 172.21.15.107:0/3287139769 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x56266db1d030 0x56266db1f420 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=46ded08a812076d4 server_cookie=b492315701848e8f in_seq=0 out_seq=0 2024-03-08T21:12:36.620 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.638+0000 7f3d10c00640 10 client.16461 ms_handle_connect on v2:172.21.15.149:6838/55290596 2024-03-08T21:12:36.620 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.639+0000 7f3d10c00640 1 -- 172.21.15.107:0/3287139769 <== mds.0 v2:172.21.15.149:6838/55290596 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56266da74910 con 0x56266db1d030 2024-03-08T21:12:36.620 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.639+0000 7f3d10c00640 10 client.16461 handle_command_reply: tid=0 2024-03-08T21:12:36.620 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.639+0000 7f3d2cfd9740 1 client.16461 shutdown 2024-03-08T21:12:36.621 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.640+0000 7f3d2cfd9740 1 -- 172.21.15.107:0/3287139769 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56266db0d370 msgr2=0x56266db0a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:12:36.621 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.640+0000 7f3d2cfd9740 1 --2- 172.21.15.107:0/3287139769 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56266db0d370 0x56266db0a650 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f3d1c009bf0 tx=0x7f3d1c003e20 comp rx=0 tx=0).stop 2024-03-08T21:12:36.621 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.640+0000 7f3d2cfd9740 1 -- 172.21.15.107:0/3287139769 shutdown_connections 2024-03-08T21:12:36.621 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.640+0000 7f3d2cfd9740 1 --2- 172.21.15.107:0/3287139769 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56266db0d370 0x56266db0a650 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:12:36.621 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.640+0000 7f3d2cfd9740 1 --2- 172.21.15.107:0/3287139769 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x56266db1d030 0x56266db1f420 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:12:36.621 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.640+0000 7f3d2cfd9740 1 --2- 172.21.15.107:0/3287139769 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56266db045a0 0x56266db0a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:12:36.621 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.640+0000 7f3d2cfd9740 1 --2- 172.21.15.107:0/3287139769 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56266db03c80 0x56266db0cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:12:36.621 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.640+0000 7f3d10c00640 0 client.16461 ms_handle_reset on v2:172.21.15.149:6838/55290596 2024-03-08T21:12:36.621 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.640+0000 7f3d2cfd9740 1 -- 172.21.15.107:0/3287139769 >> 172.21.15.107:0/3287139769 conn(0x56266dafe5a0 msgr2=0x56266db002f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:12:36.621 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.640+0000 7f3d2cfd9740 1 -- 172.21.15.107:0/3287139769 shutdown_connections 2024-03-08T21:12:36.621 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.640+0000 7f3d2cfd9740 1 -- 172.21.15.107:0/3287139769 wait complete. 2024-03-08T21:12:36.621 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.640+0000 7f3d2cfd9740 20 client.16461 trim_cache size 0 max 16384 2024-03-08T21:12:36.622 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:12:36 smithi204 ceph-mon[34935]: pgmap v381: 97 pgs: 97 active+clean; 8.1 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-08T21:12:36.637 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.660+0000 7f3d2cfd9740 1 Processor -- start 2024-03-08T21:12:36.638 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.660+0000 7f3d2cfd9740 1 -- start start 2024-03-08T21:12:36.639 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.660+0000 7f3d2cfd9740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56266db03c80 0x56266db23aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:12:36.639 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.660+0000 7f3d2cfd9740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56266db045a0 0x56266db0c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:12:36.640 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.660+0000 7f3d2cfd9740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56266db0d370 0x56266db0cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:12:36.640 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.660+0000 7f3d2cfd9740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d241b0770 con 0x56266db045a0 2024-03-08T21:12:36.640 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.660+0000 7f3d2cfd9740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d241ae790 con 0x56266db0d370 2024-03-08T21:12:36.640 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.660+0000 7f3d2cfd9740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d241b0110 con 0x56266db03c80 2024-03-08T21:12:36.640 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.660+0000 7f3d1aa00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56266db045a0 0x56266db0c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:12:36.640 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.661+0000 7f3d1aa00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56266db045a0 0x56266db0c7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:50364/0 (socket says 172.21.15.107:50364) 2024-03-08T21:12:36.640 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.661+0000 7f3d1aa00640 1 -- 172.21.15.107:0/452065752 learned_addr learned my addr 172.21.15.107:0/452065752 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:12:36.640 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.661+0000 7f3d23e00640 1 --2- 172.21.15.107:0/452065752 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56266db03c80 0x56266db23aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:12:36.640 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.661+0000 7f3d1aa00640 1 -- 172.21.15.107:0/452065752 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56266db03c80 msgr2=0x56266db23aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:12:36.641 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.661+0000 7f3d1aa00640 1 --2- 172.21.15.107:0/452065752 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56266db03c80 0x56266db23aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:12:36.641 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.661+0000 7f3d1aa00640 1 -- 172.21.15.107:0/452065752 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56266db0d370 msgr2=0x56266db0cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:12:36.641 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.661+0000 7f3d1aa00640 1 --2- 172.21.15.107:0/452065752 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56266db0d370 0x56266db0cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:12:36.641 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.661+0000 7f3d1aa00640 1 -- 172.21.15.107:0/452065752 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56266dafeee0 con 0x56266db045a0 2024-03-08T21:12:36.641 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.661+0000 7f3d1aa00640 1 --2- 172.21.15.107:0/452065752 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56266db045a0 0x56266db0c7e0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f3d0c005f00 tx=0x7f3d0c005f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec7a1bef579b666f server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:12:36.641 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.661+0000 7f3d13400640 1 -- 172.21.15.107:0/452065752 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d0c014aa0 con 0x56266db045a0 2024-03-08T21:12:36.641 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.662+0000 7f3d13400640 1 -- 172.21.15.107:0/452065752 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f3d0c01e070 con 0x56266db045a0 2024-03-08T21:12:36.641 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.662+0000 7f3d13400640 1 -- 172.21.15.107:0/452065752 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d0c01a810 con 0x56266db045a0 2024-03-08T21:12:36.645 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.662+0000 7f3d2cfd9740 1 -- 172.21.15.107:0/452065752 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56266db045a0 msgr2=0x56266db0c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:12:36.645 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.662+0000 7f3d2cfd9740 1 --2- 172.21.15.107:0/452065752 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56266db045a0 0x56266db0c7e0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f3d0c005f00 tx=0x7f3d0c005f30 comp rx=0 tx=0).stop 2024-03-08T21:12:36.645 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.662+0000 7f3d2cfd9740 1 -- 172.21.15.107:0/452065752 shutdown_connections 2024-03-08T21:12:36.645 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.662+0000 7f3d2cfd9740 1 --2- 172.21.15.107:0/452065752 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56266db0d370 0x56266db0cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:12:36.645 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.662+0000 7f3d2cfd9740 1 --2- 172.21.15.107:0/452065752 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56266db045a0 0x56266db0c7e0 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:12:36.645 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.662+0000 7f3d2cfd9740 1 --2- 172.21.15.107:0/452065752 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56266db03c80 0x56266db23aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:12:36.645 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.662+0000 7f3d2cfd9740 1 -- 172.21.15.107:0/452065752 >> 172.21.15.107:0/452065752 conn(0x56266dafe5a0 msgr2=0x56266daff760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:12:36.645 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.663+0000 7f3d2cfd9740 1 -- 172.21.15.107:0/452065752 shutdown_connections 2024-03-08T21:12:36.646 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.663+0000 7f3d2cfd9740 1 -- 172.21.15.107:0/452065752 wait complete. 2024-03-08T21:12:36.646 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.663+0000 7f3d2cfd9740 1 -- start start 2024-03-08T21:12:36.646 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.663+0000 7f3d2cfd9740 1 Processor -- start 2024-03-08T21:12:36.646 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.664+0000 7f3d2cfd9740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56266db03c80 0x56266db14760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:12:36.646 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.664+0000 7f3d2cfd9740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56266db045a0 0x56266db12b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:12:36.646 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.664+0000 7f3d2cfd9740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56266db0d370 0x56266db13180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:12:36.646 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.664+0000 7f3d2cfd9740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d241af450 con 0x56266db045a0 2024-03-08T21:12:36.646 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.664+0000 7f3d2cfd9740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d241aedf0 con 0x56266db03c80 2024-03-08T21:12:36.646 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.664+0000 7f3d2cfd9740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d241afab0 con 0x56266db0d370 2024-03-08T21:12:36.646 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.664+0000 7f3d2cfd9740 10 client.0 _ll_register_callbacks cb 0x7f3d1c037070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T21:12:36.646 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.664+0000 7f3d1aa00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56266db045a0 0x56266db12b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:12:36.647 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.664+0000 7f3d1aa00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56266db045a0 0x56266db12b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:50370/0 (socket says 172.21.15.107:50370) 2024-03-08T21:12:36.647 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.664+0000 7f3d1aa00640 1 -- 172.21.15.107:0/3039276433 learned_addr learned my addr 172.21.15.107:0/3039276433 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:12:36.647 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.664+0000 7f3d23e00640 1 --2- 172.21.15.107:0/3039276433 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56266db03c80 0x56266db14760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:12:36.647 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.664+0000 7f3d23e00640 1 --2- 172.21.15.107:0/3039276433 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56266db0d370 0x56266db13180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:12:36.647 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.664+0000 7f3d1aa00640 1 -- 172.21.15.107:0/3039276433 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56266db0d370 msgr2=0x56266db13180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:12:36.647 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.664+0000 7f3d1aa00640 1 --2- 172.21.15.107:0/3039276433 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56266db0d370 0x56266db13180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:12:36.647 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.664+0000 7f3d1aa00640 1 -- 172.21.15.107:0/3039276433 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56266db03c80 msgr2=0x56266db14760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:12:36.647 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.664+0000 7f3d1aa00640 1 --2- 172.21.15.107:0/3039276433 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56266db03c80 0x56266db14760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:12:36.647 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.664+0000 7f3d1aa00640 1 -- 172.21.15.107:0/3039276433 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56266db73230 con 0x56266db045a0 2024-03-08T21:12:36.647 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.664+0000 7f3d23e00640 1 --2- 172.21.15.107:0/3039276433 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56266db03c80 0x56266db14760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T21:12:36.647 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.665+0000 7f3d1aa00640 1 --2- 172.21.15.107:0/3039276433 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56266db045a0 0x56266db12b00 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f3d0c006010 tx=0x7f3d0c00fef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22b9d96fa3fd8fa6 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:12:36.648 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.665+0000 7f3d10c00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-03-08T21:12:36.648 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.665+0000 7f3d10c00640 1 -- 172.21.15.107:0/3039276433 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d0c01a450 con 0x56266db045a0 2024-03-08T21:12:36.648 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.665+0000 7f3d10c00640 1 -- 172.21.15.107:0/3039276433 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f3d0c01a5f0 con 0x56266db045a0 2024-03-08T21:12:36.648 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.665+0000 7f3d2cfd9740 1 -- 172.21.15.107:0/3039276433 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3d0c01e070 con 0x56266db045a0 2024-03-08T21:12:36.648 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.665+0000 7f3d10c00640 1 -- 172.21.15.107:0/3039276433 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3d0c01a7f0 con 0x56266db045a0 2024-03-08T21:12:36.648 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.665+0000 7f3d2cfd9740 10 client.16467 fetch_fsmap learned FSMap version 39 2024-03-08T21:12:36.648 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.665+0000 7f3d2cfd9740 1 -- 172.21.15.107:0/3039276433 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x56266db743b0 con 0x56266db045a0 2024-03-08T21:12:36.648 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.666+0000 7f3d10c00640 1 -- 172.21.15.107:0/3039276433 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f3d0c01e070 con 0x56266db045a0 2024-03-08T21:12:36.648 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.666+0000 7f3d10c00640 1 -- 172.21.15.107:0/3039276433 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 39) v1 ==== 7307+0+0 (secure 0 0 0) 0x7f3d0c012450 con 0x56266db045a0 2024-03-08T21:12:36.648 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.666+0000 7f3d2cfd9740 10 client.16467 fetch_fsmap finished waiting for FSMap version 39 2024-03-08T21:12:36.649 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.666+0000 7f3d2cfd9740 10 client.16467 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-08T21:12:36.649 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.666+0000 7f3d2cfd9740 20 client.16467 populate_metadata read hostname 'smithi107' 2024-03-08T21:12:36.649 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.666+0000 7f3d2cfd9740 1 --2- 172.21.15.107:0/3039276433 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x56266db74ab0 0x56266db76ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:12:36.649 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.666+0000 7f3d2cfd9740 4 client.16467 mds_command: new command op to 34475 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-08T21:12:36.649 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.666+0000 7f3d2cfd9740 1 -- 172.21.15.107:0/3039276433 --> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x56266da6ec60 con 0x56266db74ab0 2024-03-08T21:12:36.649 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.667+0000 7f3d23e00640 1 --2- 172.21.15.107:0/3039276433 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x56266db74ab0 0x56266db76ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:12:36.649 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.668+0000 7f3d23e00640 1 --2- 172.21.15.107:0/3039276433 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x56266db74ab0 0x56266db76ed0 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aad62829787794ec server_cookie=f992bbf9ba9f1667 in_seq=0 out_seq=0 2024-03-08T21:12:36.650 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.668+0000 7f3d10c00640 10 client.16467 ms_handle_connect on v2:172.21.15.149:6838/55290596 2024-03-08T21:12:36.656 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.677+0000 7f3d10c00640 1 -- 172.21.15.107:0/3039276433 <== mds.0 v2:172.21.15.149:6838/55290596 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x7f3d1c04c040 con 0x56266db74ab0 2024-03-08T21:12:36.656 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.677+0000 7f3d10c00640 10 client.16467 handle_command_reply: tid=0 2024-03-08T21:12:36.656 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.677+0000 7f3d2cfd9740 1 client.16467 shutdown 2024-03-08T21:12:36.656 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.677+0000 7f3d2cfd9740 1 -- 172.21.15.107:0/3039276433 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56266db045a0 msgr2=0x56266db12b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:12:36.656 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.677+0000 7f3d2cfd9740 1 --2- 172.21.15.107:0/3039276433 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56266db045a0 0x56266db12b00 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f3d0c006010 tx=0x7f3d0c00fef0 comp rx=0 tx=0).stop 2024-03-08T21:12:36.656 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.677+0000 7f3d2cfd9740 1 -- 172.21.15.107:0/3039276433 shutdown_connections 2024-03-08T21:12:36.656 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.677+0000 7f3d2cfd9740 1 --2- 172.21.15.107:0/3039276433 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56266db0d370 0x56266db13180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:12:36.656 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.677+0000 7f3d2cfd9740 1 --2- 172.21.15.107:0/3039276433 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56266db045a0 0x56266db12b00 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:12:36.656 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.677+0000 7f3d2cfd9740 1 --2- 172.21.15.107:0/3039276433 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x56266db74ab0 0x56266db76ed0 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:12:36.656 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.677+0000 7f3d2cfd9740 1 --2- 172.21.15.107:0/3039276433 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56266db03c80 0x56266db14760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:12:36.656 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.677+0000 7f3d2cfd9740 1 -- 172.21.15.107:0/3039276433 >> 172.21.15.107:0/3039276433 conn(0x56266dafe5a0 msgr2=0x56266db04b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:12:36.657 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.677+0000 7f3d10c00640 0 client.16467 ms_handle_reset on v2:172.21.15.149:6838/55290596 2024-03-08T21:12:36.657 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.678+0000 7f3d2cfd9740 1 -- 172.21.15.107:0/3039276433 shutdown_connections 2024-03-08T21:12:36.657 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.678+0000 7f3d2cfd9740 1 -- 172.21.15.107:0/3039276433 wait complete. 2024-03-08T21:12:36.657 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.678+0000 7f3d2cfd9740 20 client.16467 trim_cache size 0 max 16384 2024-03-08T21:12:36.657 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:{ 2024-03-08T21:12:36.657 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "message": "", 2024-03-08T21:12:36.657 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "return_code": 0 2024-03-08T21:12:36.657 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:} 2024-03-08T21:12:36.658 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.678+0000 7f3d2a200640 1 -- 172.21.15.107:0/1483725237 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f3d040a2a10 msgr2=0x7f3d040a4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:12:36.658 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.678+0000 7f3d2a200640 1 --2- 172.21.15.107:0/1483725237 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f3d040a2a10 0x7f3d040a4ed0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f3d140cc3f0 tx=0x7f3d14000bc0 comp rx=0 tx=0).stop 2024-03-08T21:12:36.658 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.678+0000 7f3d2a200640 1 -- 172.21.15.107:0/1483725237 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3d24069d60 msgr2=0x7f3d2419f810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:12:36.658 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.678+0000 7f3d2a200640 1 --2- 172.21.15.107:0/1483725237 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3d24069d60 0x7f3d2419f810 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f3d1400c890 tx=0x7f3d1400cd50 comp rx=0 tx=0).stop 2024-03-08T21:12:36.659 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.678+0000 7f3d2a200640 1 -- 172.21.15.107:0/1483725237 shutdown_connections 2024-03-08T21:12:36.659 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.678+0000 7f3d2a200640 1 --2- 172.21.15.107:0/1483725237 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f3d040a2a10 0x7f3d040a4ed0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:12:36.659 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.678+0000 7f3d2a200640 1 --2- 172.21.15.107:0/1483725237 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f3d24199a10 0x7f3d24199ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:12:36.659 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.678+0000 7f3d2a200640 1 --2- 172.21.15.107:0/1483725237 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3d241066b0 0x7f3d2419fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:12:36.659 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.679+0000 7f3d2a200640 1 --2- 172.21.15.107:0/1483725237 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3d24069d60 0x7f3d2419f810 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:12:36.659 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.679+0000 7f3d2a200640 1 -- 172.21.15.107:0/1483725237 >> 172.21.15.107:0/1483725237 conn(0x7f3d24076fc0 msgr2=0x7f3d2410c5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:12:36.659 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.679+0000 7f3d2a200640 1 -- 172.21.15.107:0/1483725237 shutdown_connections 2024-03-08T21:12:36.659 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:36.679+0000 7f3d2a200640 1 -- 172.21.15.107:0/1483725237 wait complete. 2024-03-08T21:12:36.711 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:12:36 smithi107 ceph-mon[31381]: pgmap v381: 97 pgs: 97 active+clean; 8.1 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-08T21:12:36.759 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:12:36 smithi149 ceph-mon[35042]: pgmap v381: 97 pgs: 97 active+clean; 8.1 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-08T21:12:38.625 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:12:38 smithi204 ceph-mon[34935]: pgmap v382: 97 pgs: 97 active+clean; 8.1 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-08T21:12:38.711 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:12:38 smithi107 ceph-mon[31381]: pgmap v382: 97 pgs: 97 active+clean; 8.1 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-08T21:12:38.759 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:12:38 smithi149 ceph-mon[35042]: pgmap v382: 97 pgs: 97 active+clean; 8.1 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-08T21:12:40.623 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:12:40 smithi204 ceph-mon[34935]: pgmap v383: 97 pgs: 97 active+clean; 8.1 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-08T21:12:40.711 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:12:40 smithi107 ceph-mon[31381]: pgmap v383: 97 pgs: 97 active+clean; 8.1 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-08T21:12:40.759 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:12:40 smithi149 ceph-mon[35042]: pgmap v383: 97 pgs: 97 active+clean; 8.1 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-08T21:12:42.622 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:12:42 smithi204 ceph-mon[34935]: pgmap v384: 97 pgs: 97 active+clean; 8.1 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:12:42.711 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:12:42 smithi107 ceph-mon[31381]: pgmap v384: 97 pgs: 97 active+clean; 8.1 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:12:42.759 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:12:42 smithi149 ceph-mon[35042]: pgmap v384: 97 pgs: 97 active+clean; 8.1 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:12:44.623 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:12:44 smithi204 ceph-mon[34935]: pgmap v385: 97 pgs: 97 active+clean; 8.1 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:12:44.711 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:12:44 smithi107 ceph-mon[31381]: pgmap v385: 97 pgs: 97 active+clean; 8.1 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:12:44.759 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:12:44 smithi149 ceph-mon[35042]: pgmap v385: 97 pgs: 97 active+clean; 8.1 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:12:46.623 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:12:46 smithi204 ceph-mon[34935]: pgmap v386: 97 pgs: 97 active+clean; 8.1 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-08T21:12:46.712 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:12:46 smithi107 ceph-mon[31381]: pgmap v386: 97 pgs: 97 active+clean; 8.1 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-08T21:12:46.759 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:12:46 smithi149 ceph-mon[35042]: pgmap v386: 97 pgs: 97 active+clean; 8.1 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-08T21:12:48.622 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:12:48 smithi204 ceph-mon[34935]: pgmap v387: 97 pgs: 97 active+clean; 8.1 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:12:48.711 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:12:48 smithi107 ceph-mon[31381]: pgmap v387: 97 pgs: 97 active+clean; 8.1 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:12:48.759 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:12:48 smithi149 ceph-mon[35042]: pgmap v387: 97 pgs: 97 active+clean; 8.1 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:12:49.808 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.831+0000 7f8741c00640 1 -- 172.21.15.107:0/2256431554 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f873c1073e0 msgr2=0x7f873c1077c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:12:49.808 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.831+0000 7f8741c00640 1 --2- 172.21.15.107:0/2256431554 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f873c1073e0 0x7f873c1077c0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f872c009ca0 tx=0x7f872c02c190 comp rx=0 tx=0).stop 2024-03-08T21:12:49.808 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.831+0000 7f8741c00640 1 -- 172.21.15.107:0/2256431554 shutdown_connections 2024-03-08T21:12:49.808 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.831+0000 7f8741c00640 1 --2- 172.21.15.107:0/2256431554 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f873c069310 0x7f873c10e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:12:49.808 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.831+0000 7f8741c00640 1 --2- 172.21.15.107:0/2256431554 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f873c068970 0x7f873c068dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:12:49.808 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.831+0000 7f8741c00640 1 --2- 172.21.15.107:0/2256431554 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f873c1073e0 0x7f873c1077c0 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:12:49.808 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.831+0000 7f8741c00640 1 -- 172.21.15.107:0/2256431554 >> 172.21.15.107:0/2256431554 conn(0x7f873c077040 msgr2=0x7f873c077450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:12:49.808 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.831+0000 7f8741c00640 1 -- 172.21.15.107:0/2256431554 shutdown_connections 2024-03-08T21:12:49.809 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.831+0000 7f8741c00640 1 -- 172.21.15.107:0/2256431554 wait complete. 2024-03-08T21:12:49.810 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.832+0000 7f8741c00640 1 Processor -- start 2024-03-08T21:12:49.810 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.832+0000 7f8741c00640 1 -- start start 2024-03-08T21:12:49.810 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.832+0000 7f8741c00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f873c068970 0x7f873c19f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:12:49.810 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.832+0000 7f8741c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f873c069310 0x7f873c19fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:12:49.810 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.832+0000 7f8741c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f873c1073e0 0x7f873c1999c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:12:49.810 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.832+0000 7f8741c00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f873c111510 con 0x7f873c069310 2024-03-08T21:12:49.810 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.832+0000 7f8741c00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f873c111390 con 0x7f873c068970 2024-03-08T21:12:49.810 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.832+0000 7f8741c00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f873c111690 con 0x7f873c1073e0 2024-03-08T21:12:49.811 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.833+0000 7f873a000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f873c069310 0x7f873c19fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:12:49.811 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.833+0000 7f873a000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f873c069310 0x7f873c19fd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:55740/0 (socket says 172.21.15.107:55740) 2024-03-08T21:12:49.811 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.833+0000 7f873a000640 1 -- 172.21.15.107:0/2371667095 learned_addr learned my addr 172.21.15.107:0/2371667095 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:12:49.811 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.833+0000 7f873aa00640 1 --2- 172.21.15.107:0/2371667095 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f873c068970 0x7f873c19f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:12:49.811 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.833+0000 7f873b400640 1 --2- 172.21.15.107:0/2371667095 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f873c1073e0 0x7f873c1999c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:12:49.811 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.833+0000 7f873a000640 1 -- 172.21.15.107:0/2371667095 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f873c1073e0 msgr2=0x7f873c1999c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:12:49.811 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.833+0000 7f873a000640 1 --2- 172.21.15.107:0/2371667095 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f873c1073e0 0x7f873c1999c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:12:49.811 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.833+0000 7f873a000640 1 -- 172.21.15.107:0/2371667095 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f873c068970 msgr2=0x7f873c19f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:12:49.811 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.833+0000 7f873a000640 1 --2- 172.21.15.107:0/2371667095 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f873c068970 0x7f873c19f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:12:49.811 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.833+0000 7f873a000640 1 -- 172.21.15.107:0/2371667095 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f873c19a140 con 0x7f873c069310 2024-03-08T21:12:49.812 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.834+0000 7f873a000640 1 --2- 172.21.15.107:0/2371667095 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f873c069310 0x7f873c19fd50 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f872800d950 tx=0x7f872800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:12:49.812 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.834+0000 7f8733400640 1 -- 172.21.15.107:0/2371667095 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8728017070 con 0x7f873c069310 2024-03-08T21:12:49.812 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.834+0000 7f8733400640 1 -- 172.21.15.107:0/2371667095 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f8728009c30 con 0x7f873c069310 2024-03-08T21:12:49.812 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.834+0000 7f8733400640 1 -- 172.21.15.107:0/2371667095 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f872800f620 con 0x7f873c069310 2024-03-08T21:12:49.812 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.834+0000 7f8741c00640 1 -- 172.21.15.107:0/2371667095 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f873c19a430 con 0x7f873c069310 2024-03-08T21:12:49.812 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.834+0000 7f8741c00640 1 -- 172.21.15.107:0/2371667095 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f873c1a66d0 con 0x7f873c069310 2024-03-08T21:12:49.815 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.838+0000 7f87449b1740 1 Processor -- start 2024-03-08T21:12:49.815 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.838+0000 7f87449b1740 1 -- start start 2024-03-08T21:12:49.816 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.839+0000 7f8733400640 1 -- 172.21.15.107:0/2371667095 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) v1 ==== 137422+0+0 (secure 0 0 0) 0x7f87280109a0 con 0x7f873c069310 2024-03-08T21:12:49.817 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.839+0000 7f87449b1740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b65c0a6c80 0x55b65c0a7060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:12:49.817 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.839+0000 7f87449b1740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b65c0a75a0 0x55b65c0afe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:12:49.817 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.839+0000 7f87449b1740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b65c0b0370 0x55b65c0b2760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:12:49.817 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.839+0000 7f87449b1740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b65c00a3d0 con 0x55b65c0a6c80 2024-03-08T21:12:49.817 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.839+0000 7f87449b1740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b65bfb36b0 con 0x55b65c0a75a0 2024-03-08T21:12:49.817 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.839+0000 7f87449b1740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b65c006470 con 0x55b65c0b0370 2024-03-08T21:12:49.817 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.839+0000 7f873b400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b65c0a6c80 0x55b65c0a7060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:12:49.817 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.839+0000 7f873b400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b65c0a6c80 0x55b65c0a7060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:55744/0 (socket says 172.21.15.107:55744) 2024-03-08T21:12:49.817 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.839+0000 7f873b400640 1 -- 172.21.15.107:0/1270022799 learned_addr learned my addr 172.21.15.107:0/1270022799 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:12:49.817 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.839+0000 7f873aa00640 1 --2- 172.21.15.107:0/1270022799 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b65c0a75a0 0x55b65c0afe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:12:49.817 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.840+0000 7f873b400640 1 -- 172.21.15.107:0/1270022799 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b65c0b0370 msgr2=0x55b65c0b2760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:12:49.817 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.840+0000 7f873a000640 1 --2- 172.21.15.107:0/1270022799 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b65c0b0370 0x55b65c0b2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:12:49.817 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.840+0000 7f873b400640 1 --2- 172.21.15.107:0/1270022799 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b65c0b0370 0x55b65c0b2760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:12:49.817 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.840+0000 7f873b400640 1 -- 172.21.15.107:0/1270022799 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b65c0a75a0 msgr2=0x55b65c0afe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:12:49.818 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.840+0000 7f873b400640 1 --2- 172.21.15.107:0/1270022799 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b65c0a75a0 0x55b65c0afe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:12:49.818 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.840+0000 7f873b400640 1 -- 172.21.15.107:0/1270022799 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b65bfd9b50 con 0x55b65c0a6c80 2024-03-08T21:12:49.818 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.840+0000 7f873a000640 1 --2- 172.21.15.107:0/1270022799 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b65c0b0370 0x55b65c0b2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T21:12:49.822 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.840+0000 7f873b400640 1 --2- 172.21.15.107:0/1270022799 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b65c0a6c80 0x55b65c0a7060 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f873400bc70 tx=0x7f873400f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a768636bcf0f787b server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:12:49.822 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.840+0000 7f8733400640 1 --2- 172.21.15.107:0/2371667095 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f87200a2a10 0x7f87200a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:12:49.822 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.840+0000 7f8727e00640 1 -- 172.21.15.107:0/1270022799 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8734015070 con 0x55b65c0a6c80 2024-03-08T21:12:49.822 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.841+0000 7f8727e00640 1 -- 172.21.15.107:0/1270022799 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f873401a910 con 0x55b65c0a6c80 2024-03-08T21:12:49.822 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.841+0000 7f8727e00640 1 -- 172.21.15.107:0/1270022799 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8734011870 con 0x55b65c0a6c80 2024-03-08T21:12:49.822 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.841+0000 7f8733400640 1 -- 172.21.15.107:0/2371667095 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f87280cebb0 con 0x7f873c069310 2024-03-08T21:12:49.822 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.841+0000 7f873aa00640 1 --2- 172.21.15.107:0/2371667095 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f87200a2a10 0x7f87200a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:12:49.822 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.841+0000 7f87449b1740 1 -- 172.21.15.107:0/1270022799 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b65c0a6c80 msgr2=0x55b65c0a7060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:12:49.822 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.841+0000 7f87449b1740 1 --2- 172.21.15.107:0/1270022799 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b65c0a6c80 0x55b65c0a7060 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f873400bc70 tx=0x7f873400f730 comp rx=0 tx=0).stop 2024-03-08T21:12:49.822 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.842+0000 7f87449b1740 1 -- 172.21.15.107:0/1270022799 shutdown_connections 2024-03-08T21:12:49.822 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.842+0000 7f87449b1740 1 --2- 172.21.15.107:0/1270022799 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b65c0b0370 0x55b65c0b2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:12:49.823 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.842+0000 7f87449b1740 1 --2- 172.21.15.107:0/1270022799 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b65c0a75a0 0x55b65c0afe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:12:49.823 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.842+0000 7f87449b1740 1 --2- 172.21.15.107:0/1270022799 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b65c0a6c80 0x55b65c0a7060 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:12:49.823 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.842+0000 7f87449b1740 1 -- 172.21.15.107:0/1270022799 >> 172.21.15.107:0/1270022799 conn(0x55b65c0a15a0 msgr2=0x55b65c0a1980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:12:49.823 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.842+0000 7f873aa00640 1 --2- 172.21.15.107:0/2371667095 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f87200a2a10 0x7f87200a4ed0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f872c02c7a0 tx=0x7f872c02e040 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:12:49.823 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.842+0000 7f87449b1740 1 -- 172.21.15.107:0/1270022799 shutdown_connections 2024-03-08T21:12:49.823 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.842+0000 7f87449b1740 1 -- 172.21.15.107:0/1270022799 wait complete. 2024-03-08T21:12:49.823 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.842+0000 7f87449b1740 1 -- start start 2024-03-08T21:12:49.823 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.842+0000 7f87449b1740 1 Processor -- start 2024-03-08T21:12:49.823 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.843+0000 7f87449b1740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b65c0a6c80 0x55b65c0afe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:12:49.823 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.843+0000 7f87449b1740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b65c0a75a0 0x55b65c0ad190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:12:49.823 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.843+0000 7f87449b1740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b65c0b0370 0x55b65c0ad880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:12:49.823 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.843+0000 7f87449b1740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b65c0b2fa0 con 0x55b65c0a75a0 2024-03-08T21:12:49.823 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.843+0000 7f87449b1740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b65bfc69f0 con 0x55b65c0b0370 2024-03-08T21:12:49.823 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.843+0000 7f87449b1740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b65c0b3120 con 0x55b65c0a6c80 2024-03-08T21:12:49.824 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.843+0000 7f87449b1740 10 client.0 _ll_register_callbacks cb 0x55b65bd1ea00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T21:12:49.824 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.843+0000 7f873a000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b65c0a75a0 0x55b65c0ad190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:12:49.824 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.843+0000 7f873a000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b65c0a75a0 0x55b65c0ad190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:55750/0 (socket says 172.21.15.107:55750) 2024-03-08T21:12:49.824 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.844+0000 7f873a000640 1 -- 172.21.15.107:0/3125461262 learned_addr learned my addr 172.21.15.107:0/3125461262 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:12:49.824 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.844+0000 7f873aa00640 1 --2- 172.21.15.107:0/3125461262 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b65c0a6c80 0x55b65c0afe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:12:49.824 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.844+0000 7f873b400640 1 --2- 172.21.15.107:0/3125461262 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b65c0b0370 0x55b65c0ad880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:12:49.824 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.844+0000 7f873a000640 1 -- 172.21.15.107:0/3125461262 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b65c0a6c80 msgr2=0x55b65c0afe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:12:49.824 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.844+0000 7f873a000640 1 --2- 172.21.15.107:0/3125461262 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b65c0a6c80 0x55b65c0afe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:12:49.824 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.844+0000 7f873a000640 1 -- 172.21.15.107:0/3125461262 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b65c0b0370 msgr2=0x55b65c0ad880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:12:49.824 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.844+0000 7f873a000640 1 --2- 172.21.15.107:0/3125461262 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b65c0b0370 0x55b65c0ad880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:12:49.824 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.844+0000 7f873a000640 1 -- 172.21.15.107:0/3125461262 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b65bd1f210 con 0x55b65c0a75a0 2024-03-08T21:12:49.824 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.844+0000 7f873b400640 1 --2- 172.21.15.107:0/3125461262 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b65c0b0370 0x55b65c0ad880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T21:12:49.824 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.844+0000 7f873a000640 1 --2- 172.21.15.107:0/3125461262 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b65c0a75a0 0x55b65c0ad190 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f8728000c00 tx=0x7f8728009de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e79c3c9e66326107 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:12:49.824 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.844+0000 7f8724c00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-03-08T21:12:49.825 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.844+0000 7f8724c00640 1 -- 172.21.15.107:0/3125461262 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8728020040 con 0x55b65c0a75a0 2024-03-08T21:12:49.825 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.844+0000 7f8724c00640 1 -- 172.21.15.107:0/3125461262 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f8728097840 con 0x55b65c0a75a0 2024-03-08T21:12:49.825 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.844+0000 7f87449b1740 1 -- 172.21.15.107:0/3125461262 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f873401a910 con 0x55b65c0a75a0 2024-03-08T21:12:49.825 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.845+0000 7f8724c00640 1 -- 172.21.15.107:0/3125461262 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f872809f320 con 0x55b65c0a75a0 2024-03-08T21:12:49.825 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.845+0000 7f87449b1740 10 client.16479 fetch_fsmap learned FSMap version 39 2024-03-08T21:12:49.825 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.845+0000 7f87449b1740 1 -- 172.21.15.107:0/3125461262 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x55b65bdeb620 con 0x55b65c0a75a0 2024-03-08T21:12:49.825 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.845+0000 7f8724c00640 1 -- 172.21.15.107:0/3125461262 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f873401a910 con 0x55b65c0a75a0 2024-03-08T21:12:49.825 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.845+0000 7f8724c00640 1 -- 172.21.15.107:0/3125461262 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 39) v1 ==== 7307+0+0 (secure 0 0 0) 0x7f87280970b0 con 0x55b65c0a75a0 2024-03-08T21:12:49.827 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.845+0000 7f87449b1740 10 client.16479 fetch_fsmap finished waiting for FSMap version 39 2024-03-08T21:12:49.827 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.845+0000 7f87449b1740 10 client.16479 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-08T21:12:49.827 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.845+0000 7f87449b1740 20 client.16479 populate_metadata read hostname 'smithi107' 2024-03-08T21:12:49.827 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.845+0000 7f87449b1740 1 --2- 172.21.15.107:0/3125461262 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55b65c0c0250 0x55b65c0c2640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:12:49.827 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.845+0000 7f87449b1740 4 client.16479 mds_command: new command op to 34475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-08T21:12:49.827 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.845+0000 7f87449b1740 1 -- 172.21.15.107:0/3125461262 --> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b65c017910 con 0x55b65c0c0250 2024-03-08T21:12:49.827 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.846+0000 7f873aa00640 1 --2- 172.21.15.107:0/3125461262 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55b65c0c0250 0x55b65c0c2640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:12:49.827 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.847+0000 7f873aa00640 1 --2- 172.21.15.107:0/3125461262 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55b65c0c0250 0x55b65c0c2640 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=46ac474e964ad5b server_cookie=d3e9b35ad858d32a in_seq=0 out_seq=0 2024-03-08T21:12:49.828 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.847+0000 7f8724c00640 10 client.16479 ms_handle_connect on v2:172.21.15.149:6838/55290596 2024-03-08T21:12:49.828 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.849+0000 7f8724c00640 1 -- 172.21.15.107:0/3125461262 <== mds.0 v2:172.21.15.149:6838/55290596 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55b65c017910 con 0x55b65c0c0250 2024-03-08T21:12:49.828 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.849+0000 7f8724c00640 10 client.16479 handle_command_reply: tid=0 2024-03-08T21:12:49.828 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.849+0000 7f87449b1740 1 client.16479 shutdown 2024-03-08T21:12:49.828 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.849+0000 7f87449b1740 1 -- 172.21.15.107:0/3125461262 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b65c0a75a0 msgr2=0x55b65c0ad190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:12:49.828 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.849+0000 7f87449b1740 1 --2- 172.21.15.107:0/3125461262 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b65c0a75a0 0x55b65c0ad190 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f8728000c00 tx=0x7f8728009de0 comp rx=0 tx=0).stop 2024-03-08T21:12:49.828 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.849+0000 7f87449b1740 1 -- 172.21.15.107:0/3125461262 shutdown_connections 2024-03-08T21:12:49.828 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.849+0000 7f87449b1740 1 --2- 172.21.15.107:0/3125461262 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55b65c0c0250 0x55b65c0c2640 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:12:49.828 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.849+0000 7f87449b1740 1 --2- 172.21.15.107:0/3125461262 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b65c0b0370 0x55b65c0ad880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:12:49.828 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.849+0000 7f87449b1740 1 --2- 172.21.15.107:0/3125461262 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b65c0a75a0 0x55b65c0ad190 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:12:49.828 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.849+0000 7f87449b1740 1 --2- 172.21.15.107:0/3125461262 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b65c0a6c80 0x55b65c0afe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:12:49.828 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.849+0000 7f8724c00640 0 client.16479 ms_handle_reset on v2:172.21.15.149:6838/55290596 2024-03-08T21:12:49.828 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.849+0000 7f87449b1740 1 -- 172.21.15.107:0/3125461262 >> 172.21.15.107:0/3125461262 conn(0x55b65c0a15a0 msgr2=0x55b65c0a32f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:12:49.828 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.849+0000 7f87449b1740 1 -- 172.21.15.107:0/3125461262 shutdown_connections 2024-03-08T21:12:49.829 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.849+0000 7f87449b1740 1 -- 172.21.15.107:0/3125461262 wait complete. 2024-03-08T21:12:49.829 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.849+0000 7f87449b1740 20 client.16479 trim_cache size 0 max 16384 2024-03-08T21:12:49.840 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.863+0000 7f87449b1740 1 Processor -- start 2024-03-08T21:12:49.840 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.864+0000 7f87449b1740 1 -- start start 2024-03-08T21:12:49.842 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.864+0000 7f87449b1740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b65c0a6c80 0x55b65c0af8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:12:49.842 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.864+0000 7f87449b1740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b65c0a75a0 0x55b65c0afe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:12:49.842 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.864+0000 7f87449b1740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b65c0b0370 0x55b65c0aba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:12:49.842 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.864+0000 7f87449b1740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f873c1b06d0 con 0x55b65c0b0370 2024-03-08T21:12:49.843 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.864+0000 7f87449b1740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f873c1ae6f0 con 0x55b65c0a6c80 2024-03-08T21:12:49.843 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.864+0000 7f87449b1740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f873c1b0070 con 0x55b65c0a75a0 2024-03-08T21:12:49.843 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.864+0000 7f873b400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b65c0b0370 0x55b65c0aba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:12:49.843 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.864+0000 7f873b400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b65c0b0370 0x55b65c0aba80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:55762/0 (socket says 172.21.15.107:55762) 2024-03-08T21:12:49.843 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.864+0000 7f873b400640 1 -- 172.21.15.107:0/998840538 learned_addr learned my addr 172.21.15.107:0/998840538 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:12:49.843 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.865+0000 7f873a000640 1 --2- 172.21.15.107:0/998840538 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b65c0a75a0 0x55b65c0afe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:12:49.843 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.865+0000 7f873aa00640 1 --2- 172.21.15.107:0/998840538 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b65c0a6c80 0x55b65c0af8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:12:49.843 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.865+0000 7f873b400640 1 -- 172.21.15.107:0/998840538 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b65c0a75a0 msgr2=0x55b65c0afe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:12:49.843 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.865+0000 7f873b400640 1 --2- 172.21.15.107:0/998840538 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b65c0a75a0 0x55b65c0afe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:12:49.843 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.865+0000 7f873b400640 1 -- 172.21.15.107:0/998840538 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b65c0a6c80 msgr2=0x55b65c0af8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:12:49.843 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.865+0000 7f873b400640 1 --2- 172.21.15.107:0/998840538 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b65c0a6c80 0x55b65c0af8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:12:49.843 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.865+0000 7f873b400640 1 -- 172.21.15.107:0/998840538 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b65c0b2ad0 con 0x55b65c0b0370 2024-03-08T21:12:49.843 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.865+0000 7f873aa00640 1 --2- 172.21.15.107:0/998840538 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b65c0a6c80 0x55b65c0af8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T21:12:49.843 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.865+0000 7f873b400640 1 --2- 172.21.15.107:0/998840538 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b65c0b0370 0x55b65c0aba80 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f8734006010 tx=0x7f873400b9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=997f6ee0149515be server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:12:49.844 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.865+0000 7f8727400640 1 -- 172.21.15.107:0/998840538 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8734015040 con 0x55b65c0b0370 2024-03-08T21:12:49.844 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.866+0000 7f8727400640 1 -- 172.21.15.107:0/998840538 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f873400bd60 con 0x55b65c0b0370 2024-03-08T21:12:49.844 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.866+0000 7f8727400640 1 -- 172.21.15.107:0/998840538 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8734004da0 con 0x55b65c0b0370 2024-03-08T21:12:49.849 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.866+0000 7f87449b1740 1 -- 172.21.15.107:0/998840538 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b65c0b0370 msgr2=0x55b65c0aba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:12:49.849 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.866+0000 7f87449b1740 1 --2- 172.21.15.107:0/998840538 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b65c0b0370 0x55b65c0aba80 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f8734006010 tx=0x7f873400b9f0 comp rx=0 tx=0).stop 2024-03-08T21:12:49.849 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.866+0000 7f87449b1740 1 -- 172.21.15.107:0/998840538 shutdown_connections 2024-03-08T21:12:49.849 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.866+0000 7f87449b1740 1 --2- 172.21.15.107:0/998840538 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b65c0b0370 0x55b65c0aba80 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:12:49.850 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.866+0000 7f87449b1740 1 --2- 172.21.15.107:0/998840538 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b65c0a75a0 0x55b65c0afe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:12:49.850 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.866+0000 7f87449b1740 1 --2- 172.21.15.107:0/998840538 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b65c0a6c80 0x55b65c0af8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:12:49.850 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.866+0000 7f87449b1740 1 -- 172.21.15.107:0/998840538 >> 172.21.15.107:0/998840538 conn(0x55b65c0a15a0 msgr2=0x55b65c0aca20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:12:49.850 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.866+0000 7f87449b1740 1 -- 172.21.15.107:0/998840538 shutdown_connections 2024-03-08T21:12:49.850 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.867+0000 7f87449b1740 1 -- 172.21.15.107:0/998840538 wait complete. 2024-03-08T21:12:49.850 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.867+0000 7f87449b1740 1 -- start start 2024-03-08T21:12:49.850 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.867+0000 7f87449b1740 1 Processor -- start 2024-03-08T21:12:49.850 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.868+0000 7f87449b1740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b65c0a6c80 0x55b65c0b5f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:12:49.850 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.868+0000 7f87449b1740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b65c0a75a0 0x55b65c0b64a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:12:49.850 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.868+0000 7f87449b1740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b65c0b0370 0x55b65c0b69e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:12:49.850 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.868+0000 7f87449b1740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f873c1af3b0 con 0x55b65c0a6c80 2024-03-08T21:12:49.850 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.868+0000 7f87449b1740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f873c1aed50 con 0x55b65c0b0370 2024-03-08T21:12:49.851 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.868+0000 7f87449b1740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f873c1afa10 con 0x55b65c0a75a0 2024-03-08T21:12:49.851 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.868+0000 7f87449b1740 10 client.0 _ll_register_callbacks cb 0x7f873c06ac40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T21:12:49.851 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.868+0000 7f873aa00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b65c0a6c80 0x55b65c0b5f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:12:49.851 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.868+0000 7f873aa00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b65c0a6c80 0x55b65c0b5f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:55776/0 (socket says 172.21.15.107:55776) 2024-03-08T21:12:49.851 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.868+0000 7f873aa00640 1 -- 172.21.15.107:0/2058311154 learned_addr learned my addr 172.21.15.107:0/2058311154 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:12:49.851 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.868+0000 7f873a000640 1 --2- 172.21.15.107:0/2058311154 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b65c0a75a0 0x55b65c0b64a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:12:49.851 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.868+0000 7f873b400640 1 --2- 172.21.15.107:0/2058311154 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b65c0b0370 0x55b65c0b69e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:12:49.851 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.868+0000 7f873aa00640 1 -- 172.21.15.107:0/2058311154 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b65c0a75a0 msgr2=0x55b65c0b64a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:12:49.851 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.868+0000 7f873aa00640 1 --2- 172.21.15.107:0/2058311154 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b65c0a75a0 0x55b65c0b64a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:12:49.851 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.868+0000 7f873aa00640 1 -- 172.21.15.107:0/2058311154 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b65c0b0370 msgr2=0x55b65c0b69e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:12:49.851 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.868+0000 7f873aa00640 1 --2- 172.21.15.107:0/2058311154 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b65c0b0370 0x55b65c0b69e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:12:49.851 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.868+0000 7f873aa00640 1 -- 172.21.15.107:0/2058311154 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b65c116900 con 0x55b65c0a6c80 2024-03-08T21:12:49.851 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.868+0000 7f873a000640 1 --2- 172.21.15.107:0/2058311154 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b65c0a75a0 0x55b65c0b64a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T21:12:49.851 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.869+0000 7f873aa00640 1 --2- 172.21.15.107:0/2058311154 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b65c0a6c80 0x55b65c0b5f60 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f872c02e010 tx=0x7f872c02c190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66ab4eef56ee252 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:12:49.852 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.869+0000 7f8724c00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-03-08T21:12:49.852 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.869+0000 7f8724c00640 1 -- 172.21.15.107:0/2058311154 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f872c053020 con 0x55b65c0a6c80 2024-03-08T21:12:49.852 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.869+0000 7f8724c00640 1 -- 172.21.15.107:0/2058311154 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f872c004710 con 0x55b65c0a6c80 2024-03-08T21:12:49.852 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.869+0000 7f87449b1740 1 -- 172.21.15.107:0/2058311154 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f873400bd60 con 0x55b65c0a6c80 2024-03-08T21:12:49.852 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.869+0000 7f8724c00640 1 -- 172.21.15.107:0/2058311154 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f872c05e050 con 0x55b65c0a6c80 2024-03-08T21:12:49.852 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.869+0000 7f87449b1740 10 client.16485 fetch_fsmap learned FSMap version 39 2024-03-08T21:12:49.852 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.869+0000 7f87449b1740 1 -- 172.21.15.107:0/2058311154 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x55b65c117b80 con 0x55b65c0a6c80 2024-03-08T21:12:49.852 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.870+0000 7f8724c00640 1 -- 172.21.15.107:0/2058311154 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f873400bd60 con 0x55b65c0a6c80 2024-03-08T21:12:49.852 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.870+0000 7f8724c00640 1 -- 172.21.15.107:0/2058311154 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 39) v1 ==== 7307+0+0 (secure 0 0 0) 0x7f872c032790 con 0x55b65c0a6c80 2024-03-08T21:12:49.852 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.870+0000 7f87449b1740 10 client.16485 fetch_fsmap finished waiting for FSMap version 39 2024-03-08T21:12:49.852 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.870+0000 7f87449b1740 10 client.16485 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-08T21:12:49.852 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.870+0000 7f87449b1740 20 client.16485 populate_metadata read hostname 'smithi107' 2024-03-08T21:12:49.852 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.870+0000 7f87449b1740 1 --2- 172.21.15.107:0/2058311154 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55b65c1182b0 0x55b65c11a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:12:49.852 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.870+0000 7f87449b1740 4 client.16485 mds_command: new command op to 34475 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-08T21:12:49.853 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.870+0000 7f87449b1740 1 -- 172.21.15.107:0/2058311154 --> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55b65c011c60 con 0x55b65c1182b0 2024-03-08T21:12:49.853 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.871+0000 7f873a000640 1 --2- 172.21.15.107:0/2058311154 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55b65c1182b0 0x55b65c11a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:12:49.853 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.871+0000 7f8724c00640 10 client.16485 ms_handle_connect on v2:172.21.15.149:6838/55290596 2024-03-08T21:12:49.853 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.871+0000 7f873a000640 1 --2- 172.21.15.107:0/2058311154 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55b65c1182b0 0x55b65c11a6d0 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3378fd5a4a9f43c2 server_cookie=f22d2d4a9f400195 in_seq=0 out_seq=0 2024-03-08T21:12:49.858 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.880+0000 7f8724c00640 1 -- 172.21.15.107:0/2058311154 <== mds.0 v2:172.21.15.149:6838/55290596 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55b65c011c60 con 0x55b65c1182b0 2024-03-08T21:12:49.858 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.880+0000 7f8724c00640 10 client.16485 handle_command_reply: tid=0 2024-03-08T21:12:49.858 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.881+0000 7f87449b1740 1 client.16485 shutdown 2024-03-08T21:12:49.858 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.881+0000 7f87449b1740 1 -- 172.21.15.107:0/2058311154 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b65c0a6c80 msgr2=0x55b65c0b5f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:12:49.859 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.881+0000 7f87449b1740 1 --2- 172.21.15.107:0/2058311154 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b65c0a6c80 0x55b65c0b5f60 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f872c02e010 tx=0x7f872c02c190 comp rx=0 tx=0).stop 2024-03-08T21:12:49.859 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.881+0000 7f87449b1740 1 -- 172.21.15.107:0/2058311154 shutdown_connections 2024-03-08T21:12:49.859 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.881+0000 7f87449b1740 1 --2- 172.21.15.107:0/2058311154 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55b65c1182b0 0x55b65c11a6d0 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:12:49.859 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.881+0000 7f87449b1740 1 --2- 172.21.15.107:0/2058311154 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55b65c0b0370 0x55b65c0b69e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:12:49.859 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.881+0000 7f87449b1740 1 --2- 172.21.15.107:0/2058311154 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55b65c0a75a0 0x55b65c0b64a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:12:49.859 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.881+0000 7f87449b1740 1 --2- 172.21.15.107:0/2058311154 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b65c0a6c80 0x55b65c0b5f60 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:12:49.859 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.881+0000 7f8724c00640 0 client.16485 ms_handle_reset on v2:172.21.15.149:6838/55290596 2024-03-08T21:12:49.859 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.881+0000 7f87449b1740 1 -- 172.21.15.107:0/2058311154 >> 172.21.15.107:0/2058311154 conn(0x55b65c0a15a0 msgr2=0x55b65c0aba40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:12:49.859 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.881+0000 7f87449b1740 1 -- 172.21.15.107:0/2058311154 shutdown_connections 2024-03-08T21:12:49.859 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.881+0000 7f87449b1740 1 -- 172.21.15.107:0/2058311154 wait complete. 2024-03-08T21:12:49.859 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.881+0000 7f87449b1740 20 client.16485 trim_cache size 0 max 16384 2024-03-08T21:12:49.860 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:{ 2024-03-08T21:12:49.860 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "message": "", 2024-03-08T21:12:49.860 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "return_code": 0 2024-03-08T21:12:49.860 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:} 2024-03-08T21:12:49.860 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.882+0000 7f8741c00640 1 -- 172.21.15.107:0/2371667095 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f87200a2a10 msgr2=0x7f87200a4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:12:49.860 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.882+0000 7f8741c00640 1 --2- 172.21.15.107:0/2371667095 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f87200a2a10 0x7f87200a4ed0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f872c02c7a0 tx=0x7f872c02e040 comp rx=0 tx=0).stop 2024-03-08T21:12:49.860 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.882+0000 7f8741c00640 1 -- 172.21.15.107:0/2371667095 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f873c069310 msgr2=0x7f873c19fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:12:49.861 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.882+0000 7f8741c00640 1 --2- 172.21.15.107:0/2371667095 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f873c069310 0x7f873c19fd50 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f872800d950 tx=0x7f872800de10 comp rx=0 tx=0).stop 2024-03-08T21:12:49.861 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.882+0000 7f8741c00640 1 -- 172.21.15.107:0/2371667095 shutdown_connections 2024-03-08T21:12:49.861 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.882+0000 7f8741c00640 1 --2- 172.21.15.107:0/2371667095 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f87200a2a10 0x7f87200a4ed0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:12:49.861 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.882+0000 7f8741c00640 1 --2- 172.21.15.107:0/2371667095 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f873c1073e0 0x7f873c1999c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:12:49.861 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.882+0000 7f8741c00640 1 --2- 172.21.15.107:0/2371667095 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f873c069310 0x7f873c19fd50 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:12:49.861 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.882+0000 7f8741c00640 1 --2- 172.21.15.107:0/2371667095 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f873c068970 0x7f873c19f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:12:49.861 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.882+0000 7f8741c00640 1 -- 172.21.15.107:0/2371667095 >> 172.21.15.107:0/2371667095 conn(0x7f873c077040 msgr2=0x7f873c10c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:12:49.861 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.882+0000 7f8741c00640 1 -- 172.21.15.107:0/2371667095 shutdown_connections 2024-03-08T21:12:49.861 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:12:49.882+0000 7f8741c00640 1 -- 172.21.15.107:0/2371667095 wait complete. 2024-03-08T21:12:50.622 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:12:50 smithi204 ceph-mon[34935]: pgmap v388: 97 pgs: 97 active+clean; 8.1 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-08T21:12:50.711 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:12:50 smithi107 ceph-mon[31381]: pgmap v388: 97 pgs: 97 active+clean; 8.1 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-08T21:12:50.759 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:12:50 smithi149 ceph-mon[35042]: pgmap v388: 97 pgs: 97 active+clean; 8.1 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-08T21:12:52.622 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:12:52 smithi204 ceph-mon[34935]: pgmap v389: 97 pgs: 97 active+clean; 8.1 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-08T21:12:52.711 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:12:52 smithi107 ceph-mon[31381]: pgmap v389: 97 pgs: 97 active+clean; 8.1 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-08T21:12:52.759 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:12:52 smithi149 ceph-mon[35042]: pgmap v389: 97 pgs: 97 active+clean; 8.1 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-08T21:12:54.622 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:12:54 smithi204 ceph-mon[34935]: pgmap v390: 97 pgs: 97 active+clean; 8.1 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-08T21:12:54.711 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:12:54 smithi107 ceph-mon[31381]: pgmap v390: 97 pgs: 97 active+clean; 8.1 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-08T21:12:54.759 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:12:54 smithi149 ceph-mon[35042]: pgmap v390: 97 pgs: 97 active+clean; 8.1 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-08T21:12:56.622 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:12:56 smithi204 ceph-mon[34935]: pgmap v391: 97 pgs: 97 active+clean; 8.1 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-08T21:12:56.711 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:12:56 smithi107 ceph-mon[31381]: pgmap v391: 97 pgs: 97 active+clean; 8.1 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-08T21:12:56.759 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:12:56 smithi149 ceph-mon[35042]: pgmap v391: 97 pgs: 97 active+clean; 8.1 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-08T21:12:58.622 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:12:58 smithi204 ceph-mon[34935]: pgmap v392: 97 pgs: 97 active+clean; 8.1 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:12:58.711 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:12:58 smithi107 ceph-mon[31381]: pgmap v392: 97 pgs: 97 active+clean; 8.1 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:12:58.759 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:12:58 smithi149 ceph-mon[35042]: pgmap v392: 97 pgs: 97 active+clean; 8.1 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:13:00.712 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:13:00 smithi107 ceph-mon[31381]: pgmap v393: 97 pgs: 97 active+clean; 8.1 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-08T21:13:00.759 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:13:00 smithi149 ceph-mon[35042]: pgmap v393: 97 pgs: 97 active+clean; 8.1 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-08T21:13:00.872 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:13:00 smithi204 ceph-mon[34935]: pgmap v393: 97 pgs: 97 active+clean; 8.1 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-08T21:13:02.711 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:13:02 smithi107 ceph-mon[31381]: pgmap v394: 97 pgs: 97 active+clean; 8.1 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:13:02.759 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:13:02 smithi149 ceph-mon[35042]: pgmap v394: 97 pgs: 97 active+clean; 8.1 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:13:02.872 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:13:02 smithi204 ceph-mon[34935]: pgmap v394: 97 pgs: 97 active+clean; 8.1 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:13:03.005 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:03.028+0000 7f2b19400640 1 -- 172.21.15.107:0/1872342310 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2b141034e0 msgr2=0x7f2b14103960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:13:03.005 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:03.028+0000 7f2b19400640 1 --2- 172.21.15.107:0/1872342310 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2b141034e0 0x7f2b14103960 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f2b00009900 tx=0x7f2b0002bc30 comp rx=0 tx=0).stop 2024-03-08T21:13:03.005 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:03.028+0000 7f2b19400640 1 -- 172.21.15.107:0/1872342310 shutdown_connections 2024-03-08T21:13:03.005 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:03.028+0000 7f2b19400640 1 --2- 172.21.15.107:0/1872342310 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2b14103ea0 0x7f2b14110360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:13:03.005 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:03.028+0000 7f2b19400640 1 --2- 172.21.15.107:0/1872342310 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2b141034e0 0x7f2b14103960 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:13:03.005 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:03.028+0000 7f2b19400640 1 --2- 172.21.15.107:0/1872342310 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2b141095a0 0x7f2b14109980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:13:03.005 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:03.028+0000 7f2b19400640 1 -- 172.21.15.107:0/1872342310 >> 172.21.15.107:0/1872342310 conn(0x7f2b140fefb0 msgr2=0x7f2b141013d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:13:03.005 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:03.028+0000 7f2b19400640 1 -- 172.21.15.107:0/1872342310 shutdown_connections 2024-03-08T21:13:03.005 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:03.028+0000 7f2b19400640 1 -- 172.21.15.107:0/1872342310 wait complete. 2024-03-08T21:13:03.007 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:03.029+0000 7f2b19400640 1 Processor -- start 2024-03-08T21:13:03.007 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:03.029+0000 7f2b19400640 1 -- start start 2024-03-08T21:13:03.007 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:03.029+0000 7f2b19400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2b141034e0 0x7f2b1419f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:13:03.007 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:03.029+0000 7f2b19400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2b14103ea0 0x7f2b1419fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:13:03.007 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:03.030+0000 7f2b19400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2b141095a0 0x7f2b141999b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:13:03.007 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:03.030+0000 7f2b19400640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b14113710 con 0x7f2b14103ea0 2024-03-08T21:13:03.008 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:03.030+0000 7f2b19400640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b14113590 con 0x7f2b141095a0 2024-03-08T21:13:03.008 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:03.030+0000 7f2b19400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b14113890 con 0x7f2b141034e0 2024-03-08T21:13:03.008 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:03.030+0000 7f2b11e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2b14103ea0 0x7f2b1419fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:13:03.008 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:03.030+0000 7f2b11e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2b14103ea0 0x7f2b1419fd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:39776/0 (socket says 172.21.15.107:39776) 2024-03-08T21:13:03.008 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:03.030+0000 7f2b11e00640 1 -- 172.21.15.107:0/300920049 learned_addr learned my addr 172.21.15.107:0/300920049 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:13:03.008 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:03.030+0000 7f2b12800640 1 --2- 172.21.15.107:0/300920049 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2b141034e0 0x7f2b1419f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:13:03.008 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:03.030+0000 7f2b13200640 1 --2- 172.21.15.107:0/300920049 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2b141095a0 0x7f2b141999b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:13:03.008 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:03.030+0000 7f2b11e00640 1 -- 172.21.15.107:0/300920049 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2b141034e0 msgr2=0x7f2b1419f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:13:03.008 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:03.030+0000 7f2b11e00640 1 --2- 172.21.15.107:0/300920049 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2b141034e0 0x7f2b1419f800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:13:03.008 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:03.030+0000 7f2b11e00640 1 -- 172.21.15.107:0/300920049 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2b141095a0 msgr2=0x7f2b141999b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:13:03.008 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:03.030+0000 7f2b11e00640 1 --2- 172.21.15.107:0/300920049 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2b141095a0 0x7f2b141999b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:13:03.008 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:03.030+0000 7f2b11e00640 1 -- 172.21.15.107:0/300920049 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b1419a130 con 0x7f2b14103ea0 2024-03-08T21:13:03.009 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:03.031+0000 7f2b11e00640 1 --2- 172.21.15.107:0/300920049 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2b14103ea0 0x7f2b1419fd40 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f2b00006010 tx=0x7f2b00004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:13:03.009 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:03.031+0000 7f2b0b400640 1 -- 172.21.15.107:0/300920049 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b00043020 con 0x7f2b14103ea0 2024-03-08T21:13:03.009 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:03.031+0000 7f2b19400640 1 -- 172.21.15.107:0/300920049 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b1419a3c0 con 0x7f2b14103ea0 2024-03-08T21:13:03.009 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:03.031+0000 7f2b0b400640 1 -- 172.21.15.107:0/300920049 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f2b00004440 con 0x7f2b14103ea0 2024-03-08T21:13:03.009 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:03.031+0000 7f2b0b400640 1 -- 172.21.15.107:0/300920049 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b00003b50 con 0x7f2b14103ea0 2024-03-08T21:13:03.009 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:03.031+0000 7f2b19400640 1 -- 172.21.15.107:0/300920049 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b141a66c0 con 0x7f2b14103ea0 2024-03-08T21:13:03.012 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:03.035+0000 7f2b0b400640 1 -- 172.21.15.107:0/300920049 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) v1 ==== 137422+0+0 (secure 0 0 0) 0x7f2b00046020 con 0x7f2b14103ea0 2024-03-08T21:13:03.012 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:03.035+0000 7f2b1c18e740 1 Processor -- start 2024-03-08T21:13:03.012 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:03.035+0000 7f2b1c18e740 1 -- start start 2024-03-08T21:13:03.014 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:03.036+0000 7f2b1c18e740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55586809ac80 0x55586809b060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:13:03.014 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:03.036+0000 7f2b1c18e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55586809b5a0 0x5558680a3e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:13:03.015 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:03.036+0000 7f2b1c18e740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5558680a4370 0x5558680a6760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:13:03.015 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:03.036+0000 7f2b1c18e740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x555867ffe3d0 con 0x55586809b5a0 2024-03-08T21:13:03.015 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:03.036+0000 7f2b1c18e740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x555867fa76b0 con 0x5558680a4370 2024-03-08T21:13:03.015 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:03.036+0000 7f2b1c18e740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x555867ffa470 con 0x55586809ac80 2024-03-08T21:13:03.015 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:03.036+0000 7f2b12800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55586809b5a0 0x5558680a3e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:13:03.015 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:03.036+0000 7f2b12800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55586809b5a0 0x5558680a3e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:39782/0 (socket says 172.21.15.107:39782) 2024-03-08T21:13:03.015 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:03.036+0000 7f2b12800640 1 -- 172.21.15.107:0/1498760839 learned_addr learned my addr 172.21.15.107:0/1498760839 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:13:03.015 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:03.036+0000 7f2b13200640 1 --2- 172.21.15.107:0/1498760839 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55586809ac80 0x55586809b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:13:03.015 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:03.037+0000 7f2b11e00640 1 --2- 172.21.15.107:0/1498760839 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5558680a4370 0x5558680a6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:13:03.015 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:03.037+0000 7f2b12800640 1 -- 172.21.15.107:0/1498760839 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55586809ac80 msgr2=0x55586809b060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:13:03.015 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:03.037+0000 7f2b12800640 1 --2- 172.21.15.107:0/1498760839 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55586809ac80 0x55586809b060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:13:03.015 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:03.037+0000 7f2b12800640 1 -- 172.21.15.107:0/1498760839 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5558680a4370 msgr2=0x5558680a6760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:13:03.015 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:03.037+0000 7f2b12800640 1 --2- 172.21.15.107:0/1498760839 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5558680a4370 0x5558680a6760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:13:03.016 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:03.037+0000 7f2b12800640 1 -- 172.21.15.107:0/1498760839 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555867fcdb50 con 0x55586809b5a0 2024-03-08T21:13:03.016 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:03.037+0000 7f2b12800640 1 --2- 172.21.15.107:0/1498760839 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55586809b5a0 0x5558680a3e30 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f2b0400bc90 tx=0x7f2b0400f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4965360401e6fb0a server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:13:03.016 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:03.037+0000 7f2b0b400640 1 --2- 172.21.15.107:0/300920049 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f2af80a2a90 0x7f2af80a4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:13:03.016 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:03.037+0000 7f2b0b400640 1 -- 172.21.15.107:0/300920049 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f2b000f0960 con 0x7f2b14103ea0 2024-03-08T21:13:03.016 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:03.037+0000 7f2b13200640 1 --2- 172.21.15.107:0/300920049 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f2af80a2a90 0x7f2af80a4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:13:03.016 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:03.038+0000 7f2aff400640 1 -- 172.21.15.107:0/1498760839 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b04015070 con 0x55586809b5a0 2024-03-08T21:13:03.016 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:03.038+0000 7f2aff400640 1 -- 172.21.15.107:0/1498760839 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f2b0401a910 con 0x55586809b5a0 2024-03-08T21:13:03.016 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:03.038+0000 7f2aff400640 1 -- 172.21.15.107:0/1498760839 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b04011900 con 0x55586809b5a0 2024-03-08T21:13:03.019 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:03.039+0000 7f2b1c18e740 1 -- 172.21.15.107:0/1498760839 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55586809b5a0 msgr2=0x5558680a3e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:13:03.020 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:03.039+0000 7f2b13200640 1 --2- 172.21.15.107:0/300920049 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f2af80a2a90 0x7f2af80a4f50 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f2b0c005fd0 tx=0x7f2b0c009580 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:13:03.020 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:03.039+0000 7f2b1c18e740 1 --2- 172.21.15.107:0/1498760839 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55586809b5a0 0x5558680a3e30 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f2b0400bc90 tx=0x7f2b0400f730 comp rx=0 tx=0).stop 2024-03-08T21:13:03.020 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:03.039+0000 7f2b1c18e740 1 -- 172.21.15.107:0/1498760839 shutdown_connections 2024-03-08T21:13:03.020 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:03.039+0000 7f2b1c18e740 1 --2- 172.21.15.107:0/1498760839 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5558680a4370 0x5558680a6760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:13:03.020 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:03.039+0000 7f2b1c18e740 1 --2- 172.21.15.107:0/1498760839 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55586809b5a0 0x5558680a3e30 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:13:03.020 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:03.039+0000 7f2b1c18e740 1 --2- 172.21.15.107:0/1498760839 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55586809ac80 0x55586809b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:13:03.020 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:03.039+0000 7f2b1c18e740 1 -- 172.21.15.107:0/1498760839 >> 172.21.15.107:0/1498760839 conn(0x5558680955a0 msgr2=0x555868095980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:13:03.020 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:03.039+0000 7f2b1c18e740 1 -- 172.21.15.107:0/1498760839 shutdown_connections 2024-03-08T21:13:03.020 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:03.039+0000 7f2b1c18e740 1 -- 172.21.15.107:0/1498760839 wait complete. 2024-03-08T21:13:03.020 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:03.039+0000 7f2b1c18e740 1 -- start start 2024-03-08T21:13:03.020 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:03.040+0000 7f2b1c18e740 1 Processor -- start 2024-03-08T21:13:03.020 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:03.040+0000 7f2b1c18e740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55586809ac80 0x5558680a3da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:13:03.020 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:03.040+0000 7f2b1c18e740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55586809b5a0 0x5558680a1130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:13:03.021 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:03.040+0000 7f2b1c18e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5558680a4370 0x5558680a1870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:13:03.021 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:03.040+0000 7f2b1c18e740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558680a6fa0 con 0x5558680a4370 2024-03-08T21:13:03.021 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:03.040+0000 7f2b1c18e740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x555867fba9f0 con 0x55586809ac80 2024-03-08T21:13:03.021 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:03.040+0000 7f2b1c18e740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558680a7120 con 0x55586809b5a0 2024-03-08T21:13:03.021 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:03.040+0000 7f2b1c18e740 10 client.0 _ll_register_callbacks cb 0x555867d12a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T21:13:03.021 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:03.041+0000 7f2b13200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5558680a4370 0x5558680a1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:13:03.021 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:03.041+0000 7f2b13200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5558680a4370 0x5558680a1870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:39784/0 (socket says 172.21.15.107:39784) 2024-03-08T21:13:03.021 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:03.041+0000 7f2b13200640 1 -- 172.21.15.107:0/2436712507 learned_addr learned my addr 172.21.15.107:0/2436712507 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:13:03.021 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:03.041+0000 7f2b11e00640 1 --2- 172.21.15.107:0/2436712507 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55586809b5a0 0x5558680a1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:13:03.021 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:03.041+0000 7f2b12800640 1 --2- 172.21.15.107:0/2436712507 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55586809ac80 0x5558680a3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:13:03.021 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:03.041+0000 7f2b13200640 1 -- 172.21.15.107:0/2436712507 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55586809b5a0 msgr2=0x5558680a1130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:13:03.021 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:03.041+0000 7f2b13200640 1 --2- 172.21.15.107:0/2436712507 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55586809b5a0 0x5558680a1130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:13:03.021 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:03.041+0000 7f2b13200640 1 -- 172.21.15.107:0/2436712507 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55586809ac80 msgr2=0x5558680a3da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:13:03.022 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:03.041+0000 7f2b13200640 1 --2- 172.21.15.107:0/2436712507 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55586809ac80 0x5558680a3da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:13:03.022 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:03.041+0000 7f2b13200640 1 -- 172.21.15.107:0/2436712507 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555867d13210 con 0x5558680a4370 2024-03-08T21:13:03.022 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:03.041+0000 7f2b13200640 1 --2- 172.21.15.107:0/2436712507 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5558680a4370 0x5558680a1870 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f2b0c012890 tx=0x7f2b0c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1ee93e184a5e763 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:13:03.022 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:03.041+0000 7f2af3e00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-03-08T21:13:03.022 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:03.042+0000 7f2af3e00640 1 -- 172.21.15.107:0/2436712507 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b0c004d20 con 0x5558680a4370 2024-03-08T21:13:03.022 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:03.042+0000 7f2af3e00640 1 -- 172.21.15.107:0/2436712507 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f2b0c00eae0 con 0x5558680a4370 2024-03-08T21:13:03.022 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:03.042+0000 7f2b1c18e740 1 -- 172.21.15.107:0/2436712507 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2b0401a910 con 0x5558680a4370 2024-03-08T21:13:03.022 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:03.042+0000 7f2af3e00640 1 -- 172.21.15.107:0/2436712507 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2b0c012e10 con 0x5558680a4370 2024-03-08T21:13:03.022 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:03.042+0000 7f2b1c18e740 10 client.16497 fetch_fsmap learned FSMap version 39 2024-03-08T21:13:03.022 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:03.042+0000 7f2b1c18e740 1 -- 172.21.15.107:0/2436712507 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x555867ddf620 con 0x5558680a4370 2024-03-08T21:13:03.022 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:03.042+0000 7f2af3e00640 1 -- 172.21.15.107:0/2436712507 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f2b0401a910 con 0x5558680a4370 2024-03-08T21:13:03.022 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:03.043+0000 7f2af3e00640 1 -- 172.21.15.107:0/2436712507 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 39) v1 ==== 7307+0+0 (secure 0 0 0) 0x7f2b0c00f040 con 0x5558680a4370 2024-03-08T21:13:03.022 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:03.043+0000 7f2b1c18e740 10 client.16497 fetch_fsmap finished waiting for FSMap version 39 2024-03-08T21:13:03.023 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:03.043+0000 7f2b1c18e740 10 client.16497 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-08T21:13:03.023 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:03.043+0000 7f2b1c18e740 20 client.16497 populate_metadata read hostname 'smithi107' 2024-03-08T21:13:03.023 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:03.043+0000 7f2b1c18e740 1 --2- 172.21.15.107:0/2436712507 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x5558680b42d0 0x5558680b66c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:13:03.023 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:03.043+0000 7f2b1c18e740 4 client.16497 mds_command: new command op to 34475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-08T21:13:03.023 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:03.043+0000 7f2b1c18e740 1 -- 172.21.15.107:0/2436712507 --> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55586800b910 con 0x5558680b42d0 2024-03-08T21:13:03.024 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:03.043+0000 7f2b12800640 1 --2- 172.21.15.107:0/2436712507 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x5558680b42d0 0x5558680b66c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:13:03.024 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:03.044+0000 7f2b12800640 1 --2- 172.21.15.107:0/2436712507 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x5558680b42d0 0x5558680b66c0 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4cd60d683ebd6d79 server_cookie=f3ab432a12e3b438 in_seq=0 out_seq=0 2024-03-08T21:13:03.024 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:03.044+0000 7f2af3e00640 10 client.16497 ms_handle_connect on v2:172.21.15.149:6838/55290596 2024-03-08T21:13:03.024 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:03.046+0000 7f2af3e00640 1 -- 172.21.15.107:0/2436712507 <== mds.0 v2:172.21.15.149:6838/55290596 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55586800b910 con 0x5558680b42d0 2024-03-08T21:13:03.024 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:03.046+0000 7f2af3e00640 10 client.16497 handle_command_reply: tid=0 2024-03-08T21:13:03.024 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:03.046+0000 7f2b1c18e740 1 client.16497 shutdown 2024-03-08T21:13:03.025 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:03.046+0000 7f2b1c18e740 1 -- 172.21.15.107:0/2436712507 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5558680a4370 msgr2=0x5558680a1870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:13:03.025 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:03.046+0000 7f2b1c18e740 1 --2- 172.21.15.107:0/2436712507 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5558680a4370 0x5558680a1870 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f2b0c012890 tx=0x7f2b0c004a40 comp rx=0 tx=0).stop 2024-03-08T21:13:03.025 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:03.046+0000 7f2b1c18e740 1 -- 172.21.15.107:0/2436712507 shutdown_connections 2024-03-08T21:13:03.025 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:03.046+0000 7f2b1c18e740 1 --2- 172.21.15.107:0/2436712507 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5558680a4370 0x5558680a1870 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:13:03.025 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:03.046+0000 7f2b1c18e740 1 --2- 172.21.15.107:0/2436712507 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55586809b5a0 0x5558680a1130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:13:03.025 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:03.046+0000 7f2b1c18e740 1 --2- 172.21.15.107:0/2436712507 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x5558680b42d0 0x5558680b66c0 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:13:03.025 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:03.046+0000 7f2b1c18e740 1 --2- 172.21.15.107:0/2436712507 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55586809ac80 0x5558680a3da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:13:03.025 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:03.046+0000 7f2b1c18e740 1 -- 172.21.15.107:0/2436712507 >> 172.21.15.107:0/2436712507 conn(0x5558680955a0 msgr2=0x5558680972f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:13:03.025 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:03.046+0000 7f2af3e00640 0 client.16497 ms_handle_reset on v2:172.21.15.149:6838/55290596 2024-03-08T21:13:03.025 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:03.047+0000 7f2b1c18e740 1 -- 172.21.15.107:0/2436712507 shutdown_connections 2024-03-08T21:13:03.025 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:03.047+0000 7f2b1c18e740 1 -- 172.21.15.107:0/2436712507 wait complete. 2024-03-08T21:13:03.025 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:03.047+0000 7f2b1c18e740 20 client.16497 trim_cache size 0 max 16384 2024-03-08T21:13:03.040 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:03.062+0000 7f2b1c18e740 1 Processor -- start 2024-03-08T21:13:03.040 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:03.062+0000 7f2b1c18e740 1 -- start start 2024-03-08T21:13:03.040 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:03.062+0000 7f2b1c18e740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55586809ac80 0x5558680a3860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:13:03.040 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:03.062+0000 7f2b1c18e740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55586809b5a0 0x5558680a3da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:13:03.040 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:03.062+0000 7f2b1c18e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5558680a4370 0x55586809e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:13:03.040 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:03.062+0000 7f2b1c18e740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b141b0710 con 0x5558680a4370 2024-03-08T21:13:03.040 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:03.062+0000 7f2b1c18e740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b141ae730 con 0x55586809ac80 2024-03-08T21:13:03.040 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:03.062+0000 7f2b1c18e740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b141b00b0 con 0x55586809b5a0 2024-03-08T21:13:03.040 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:03.062+0000 7f2b13200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5558680a4370 0x55586809e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:13:03.041 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:03.062+0000 7f2b13200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5558680a4370 0x55586809e3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:39796/0 (socket says 172.21.15.107:39796) 2024-03-08T21:13:03.041 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:03.062+0000 7f2b13200640 1 -- 172.21.15.107:0/2712979951 learned_addr learned my addr 172.21.15.107:0/2712979951 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:13:03.041 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:03.063+0000 7f2b11e00640 1 --2- 172.21.15.107:0/2712979951 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55586809b5a0 0x5558680a3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:13:03.041 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:03.063+0000 7f2b12800640 1 --2- 172.21.15.107:0/2712979951 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55586809ac80 0x5558680a3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:13:03.041 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:03.063+0000 7f2b13200640 1 -- 172.21.15.107:0/2712979951 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55586809b5a0 msgr2=0x5558680a3da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:13:03.041 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:03.063+0000 7f2b13200640 1 --2- 172.21.15.107:0/2712979951 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55586809b5a0 0x5558680a3da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:13:03.041 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:03.063+0000 7f2b13200640 1 -- 172.21.15.107:0/2712979951 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55586809ac80 msgr2=0x5558680a3860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:13:03.041 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:03.063+0000 7f2b13200640 1 --2- 172.21.15.107:0/2712979951 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55586809ac80 0x5558680a3860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:13:03.041 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:03.063+0000 7f2b13200640 1 -- 172.21.15.107:0/2712979951 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5558680bb8d0 con 0x5558680a4370 2024-03-08T21:13:03.041 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:03.063+0000 7f2b12800640 1 --2- 172.21.15.107:0/2712979951 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55586809ac80 0x5558680a3860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T21:13:03.041 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:03.063+0000 7f2b13200640 1 --2- 172.21.15.107:0/2712979951 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5558680a4370 0x55586809e3d0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f2b0c000f80 tx=0x7f2b0c005b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24abb1f475b23717 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:13:03.045 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:03.064+0000 7f2afea00640 1 -- 172.21.15.107:0/2712979951 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b0c03e070 con 0x5558680a4370 2024-03-08T21:13:03.045 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:03.064+0000 7f2afea00640 1 -- 172.21.15.107:0/2712979951 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f2b0c01f990 con 0x5558680a4370 2024-03-08T21:13:03.045 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:03.064+0000 7f2afea00640 1 -- 172.21.15.107:0/2712979951 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b0c033040 con 0x5558680a4370 2024-03-08T21:13:03.045 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:03.064+0000 7f2b1c18e740 1 -- 172.21.15.107:0/2712979951 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5558680a4370 msgr2=0x55586809e3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:13:03.045 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:03.064+0000 7f2b1c18e740 1 --2- 172.21.15.107:0/2712979951 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5558680a4370 0x55586809e3d0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f2b0c000f80 tx=0x7f2b0c005b30 comp rx=0 tx=0).stop 2024-03-08T21:13:03.045 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:03.065+0000 7f2b1c18e740 1 -- 172.21.15.107:0/2712979951 shutdown_connections 2024-03-08T21:13:03.045 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:03.065+0000 7f2b1c18e740 1 --2- 172.21.15.107:0/2712979951 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5558680a4370 0x55586809e3d0 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:13:03.046 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:03.065+0000 7f2b1c18e740 1 --2- 172.21.15.107:0/2712979951 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55586809b5a0 0x5558680a3da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:13:03.046 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:03.065+0000 7f2b1c18e740 1 --2- 172.21.15.107:0/2712979951 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55586809ac80 0x5558680a3860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:13:03.046 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:03.065+0000 7f2b1c18e740 1 -- 172.21.15.107:0/2712979951 >> 172.21.15.107:0/2712979951 conn(0x5558680955a0 msgr2=0x55586809fa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:13:03.046 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:03.065+0000 7f2b1c18e740 1 -- 172.21.15.107:0/2712979951 shutdown_connections 2024-03-08T21:13:03.046 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:03.065+0000 7f2b1c18e740 1 -- 172.21.15.107:0/2712979951 wait complete. 2024-03-08T21:13:03.046 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:03.065+0000 7f2b1c18e740 1 -- start start 2024-03-08T21:13:03.046 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:03.065+0000 7f2b1c18e740 1 Processor -- start 2024-03-08T21:13:03.046 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:03.066+0000 7f2b1c18e740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55586809ac80 0x5558680a9dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:13:03.046 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:03.066+0000 7f2b1c18e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55586809b5a0 0x5558680aa300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:13:03.046 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:03.066+0000 7f2b1c18e740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5558680a4370 0x5558680aa840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:13:03.046 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:03.066+0000 7f2b1c18e740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b141af3f0 con 0x55586809b5a0 2024-03-08T21:13:03.046 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:03.066+0000 7f2b1c18e740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b141aed90 con 0x5558680a4370 2024-03-08T21:13:03.047 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:03.066+0000 7f2b1c18e740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b141afa50 con 0x55586809ac80 2024-03-08T21:13:03.047 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:03.066+0000 7f2b1c18e740 10 client.0 _ll_register_callbacks cb 0x7f2b0400b440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T21:13:03.047 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:03.066+0000 7f2b11e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55586809b5a0 0x5558680aa300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:13:03.047 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:03.066+0000 7f2b11e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55586809b5a0 0x5558680aa300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:39798/0 (socket says 172.21.15.107:39798) 2024-03-08T21:13:03.047 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:03.066+0000 7f2b11e00640 1 -- 172.21.15.107:0/3543220524 learned_addr learned my addr 172.21.15.107:0/3543220524 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:13:03.047 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:03.066+0000 7f2b13200640 1 --2- 172.21.15.107:0/3543220524 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5558680a4370 0x5558680aa840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:13:03.047 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:03.066+0000 7f2b12800640 1 --2- 172.21.15.107:0/3543220524 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55586809ac80 0x5558680a9dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:13:03.047 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:03.066+0000 7f2b11e00640 1 -- 172.21.15.107:0/3543220524 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55586809ac80 msgr2=0x5558680a9dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:13:03.047 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:03.066+0000 7f2b11e00640 1 --2- 172.21.15.107:0/3543220524 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55586809ac80 0x5558680a9dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:13:03.047 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:03.066+0000 7f2b11e00640 1 -- 172.21.15.107:0/3543220524 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5558680a4370 msgr2=0x5558680aa840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:13:03.047 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:03.066+0000 7f2b11e00640 1 --2- 172.21.15.107:0/3543220524 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5558680a4370 0x5558680aa840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:13:03.047 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:03.066+0000 7f2b11e00640 1 -- 172.21.15.107:0/3543220524 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55586810a800 con 0x55586809b5a0 2024-03-08T21:13:03.047 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:03.066+0000 7f2b13200640 1 --2- 172.21.15.107:0/3543220524 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5558680a4370 0x5558680aa840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T21:13:03.048 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:03.067+0000 7f2b11e00640 1 --2- 172.21.15.107:0/3543220524 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55586809b5a0 0x5558680aa300 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f2b000b94f0 tx=0x7f2b000bcf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fa65d7327eb3b22 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:13:03.048 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:03.067+0000 7f2af3e00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-03-08T21:13:03.048 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:03.067+0000 7f2af3e00640 1 -- 172.21.15.107:0/3543220524 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b000bd790 con 0x55586809b5a0 2024-03-08T21:13:03.048 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:03.067+0000 7f2af3e00640 1 -- 172.21.15.107:0/3543220524 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f2b000ee2f0 con 0x55586809b5a0 2024-03-08T21:13:03.048 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:03.067+0000 7f2b1c18e740 1 -- 172.21.15.107:0/3543220524 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2b0c01f990 con 0x55586809b5a0 2024-03-08T21:13:03.048 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:03.067+0000 7f2af3e00640 1 -- 172.21.15.107:0/3543220524 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2b000ee4f0 con 0x55586809b5a0 2024-03-08T21:13:03.048 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:03.067+0000 7f2b1c18e740 10 client.16503 fetch_fsmap learned FSMap version 39 2024-03-08T21:13:03.048 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:03.067+0000 7f2b1c18e740 1 -- 172.21.15.107:0/3543220524 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x55586810bae0 con 0x55586809b5a0 2024-03-08T21:13:03.048 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:03.068+0000 7f2af3e00640 1 -- 172.21.15.107:0/3543220524 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f2b0c01f990 con 0x55586809b5a0 2024-03-08T21:13:03.048 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:03.068+0000 7f2af3e00640 1 -- 172.21.15.107:0/3543220524 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 39) v1 ==== 7307+0+0 (secure 0 0 0) 0x7f2b000c5430 con 0x55586809b5a0 2024-03-08T21:13:03.048 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:03.068+0000 7f2b1c18e740 10 client.16503 fetch_fsmap finished waiting for FSMap version 39 2024-03-08T21:13:03.048 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:03.068+0000 7f2b1c18e740 10 client.16503 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-08T21:13:03.048 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:03.068+0000 7f2b1c18e740 20 client.16503 populate_metadata read hostname 'smithi107' 2024-03-08T21:13:03.048 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:03.068+0000 7f2b1c18e740 1 --2- 172.21.15.107:0/3543220524 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55586810c2b0 0x55586810e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:13:03.049 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:03.068+0000 7f2b1c18e740 4 client.16503 mds_command: new command op to 34475 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-08T21:13:03.049 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:03.068+0000 7f2b1c18e740 1 -- 172.21.15.107:0/3543220524 --> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x555868005c60 con 0x55586810c2b0 2024-03-08T21:13:03.049 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:03.068+0000 7f2b12800640 1 --2- 172.21.15.107:0/3543220524 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55586810c2b0 0x55586810e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:13:03.049 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:03.069+0000 7f2b12800640 1 --2- 172.21.15.107:0/3543220524 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55586810c2b0 0x55586810e6d0 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2a83250c65deca05 server_cookie=d02924d1b9bf3f11 in_seq=0 out_seq=0 2024-03-08T21:13:03.049 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:03.069+0000 7f2af3e00640 10 client.16503 ms_handle_connect on v2:172.21.15.149:6838/55290596 2024-03-08T21:13:04.185 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:04.208+0000 7f2af3e00640 1 -- 172.21.15.107:0/3543220524 <== mds.0 v2:172.21.15.149:6838/55290596 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x555868005c60 con 0x55586810c2b0 2024-03-08T21:13:04.186 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:04.208+0000 7f2af3e00640 10 client.16503 handle_command_reply: tid=0 2024-03-08T21:13:04.186 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:04.208+0000 7f2b1c18e740 1 client.16503 shutdown 2024-03-08T21:13:04.186 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:04.209+0000 7f2b1c18e740 1 -- 172.21.15.107:0/3543220524 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55586809b5a0 msgr2=0x5558680aa300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:13:04.186 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:04.209+0000 7f2b1c18e740 1 --2- 172.21.15.107:0/3543220524 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55586809b5a0 0x5558680aa300 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f2b000b94f0 tx=0x7f2b000bcf20 comp rx=0 tx=0).stop 2024-03-08T21:13:04.187 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:04.209+0000 7f2b1c18e740 1 -- 172.21.15.107:0/3543220524 shutdown_connections 2024-03-08T21:13:04.187 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:04.209+0000 7f2b1c18e740 1 --2- 172.21.15.107:0/3543220524 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55586810c2b0 0x55586810e6d0 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:13:04.187 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:04.209+0000 7f2b1c18e740 1 --2- 172.21.15.107:0/3543220524 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5558680a4370 0x5558680aa840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:13:04.187 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:04.209+0000 7f2b1c18e740 1 --2- 172.21.15.107:0/3543220524 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55586809b5a0 0x5558680aa300 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:13:04.187 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:04.209+0000 7f2b1c18e740 1 --2- 172.21.15.107:0/3543220524 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55586809ac80 0x5558680a9dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:13:04.187 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:04.209+0000 7f2af3e00640 0 client.16503 ms_handle_reset on v2:172.21.15.149:6838/55290596 2024-03-08T21:13:04.187 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:04.209+0000 7f2b1c18e740 1 -- 172.21.15.107:0/3543220524 >> 172.21.15.107:0/3543220524 conn(0x5558680955a0 msgr2=0x555868093e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:13:04.187 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:04.210+0000 7f2b1c18e740 1 -- 172.21.15.107:0/3543220524 shutdown_connections 2024-03-08T21:13:04.187 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:04.210+0000 7f2b1c18e740 1 -- 172.21.15.107:0/3543220524 wait complete. 2024-03-08T21:13:04.187 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:04.210+0000 7f2b1c18e740 20 client.16503 trim_cache size 0 max 16384 2024-03-08T21:13:04.188 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:{ 2024-03-08T21:13:04.188 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "message": "", 2024-03-08T21:13:04.188 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "return_code": 0 2024-03-08T21:13:04.188 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:} 2024-03-08T21:13:04.188 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:04.211+0000 7f2b19400640 1 -- 172.21.15.107:0/300920049 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f2af80a2a90 msgr2=0x7f2af80a4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:13:04.189 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:04.211+0000 7f2b19400640 1 --2- 172.21.15.107:0/300920049 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f2af80a2a90 0x7f2af80a4f50 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f2b0c005fd0 tx=0x7f2b0c009580 comp rx=0 tx=0).stop 2024-03-08T21:13:04.189 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:04.211+0000 7f2b19400640 1 -- 172.21.15.107:0/300920049 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2b14103ea0 msgr2=0x7f2b1419fd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:13:04.189 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:04.211+0000 7f2b19400640 1 --2- 172.21.15.107:0/300920049 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2b14103ea0 0x7f2b1419fd40 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f2b00006010 tx=0x7f2b00004040 comp rx=0 tx=0).stop 2024-03-08T21:13:04.189 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:04.211+0000 7f2b19400640 1 -- 172.21.15.107:0/300920049 shutdown_connections 2024-03-08T21:13:04.189 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:04.211+0000 7f2b19400640 1 --2- 172.21.15.107:0/300920049 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f2af80a2a90 0x7f2af80a4f50 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:13:04.189 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:04.211+0000 7f2b19400640 1 --2- 172.21.15.107:0/300920049 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2b141095a0 0x7f2b141999b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:13:04.189 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:04.211+0000 7f2b19400640 1 --2- 172.21.15.107:0/300920049 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2b14103ea0 0x7f2b1419fd40 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:13:04.189 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:04.211+0000 7f2b19400640 1 --2- 172.21.15.107:0/300920049 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2b141034e0 0x7f2b1419f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:13:04.189 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:04.211+0000 7f2b19400640 1 -- 172.21.15.107:0/300920049 >> 172.21.15.107:0/300920049 conn(0x7f2b140fefb0 msgr2=0x7f2b14100850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:13:04.189 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:04.211+0000 7f2b19400640 1 -- 172.21.15.107:0/300920049 shutdown_connections 2024-03-08T21:13:04.189 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:04.211+0000 7f2b19400640 1 -- 172.21.15.107:0/300920049 wait complete. 2024-03-08T21:13:04.697 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:13:04 smithi107 ceph-mon[31381]: pgmap v395: 97 pgs: 97 active+clean; 8.1 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:13:04.698 INFO:journalctl@ceph.mds.d.smithi107.stdout:Mar 08 21:13:04 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-d[72883]: *** Caught signal (Segmentation fault) ** 2024-03-08T21:13:04.698 INFO:journalctl@ceph.mds.d.smithi107.stdout:Mar 08 21:13:04 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-d[72883]: in thread 7f56d0800700 thread_name:md_log_replay 2024-03-08T21:13:04.698 INFO:journalctl@ceph.mds.d.smithi107.stdout:Mar 08 21:13:04 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-d[72883]: ceph version 19.0.0-1965-g7d63482a (7d63482aa89628d733e298984a762389b625c761) squid (dev) 2024-03-08T21:13:04.698 INFO:journalctl@ceph.mds.d.smithi107.stdout:Mar 08 21:13:04 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-d[72883]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f56e4f10d20] 2024-03-08T21:13:04.698 INFO:journalctl@ceph.mds.d.smithi107.stdout:Mar 08 21:13:04 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-d[72883]: 2: /usr/bin/ceph-mds(+0x503eff) [0x55eb954c9eff] 2024-03-08T21:13:04.698 INFO:journalctl@ceph.mds.d.smithi107.stdout:Mar 08 21:13:04 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-d[72883]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x55eb954cd773] 2024-03-08T21:13:04.698 INFO:journalctl@ceph.mds.d.smithi107.stdout:Mar 08 21:13:04 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-d[72883]: 4: (MDLog::_replay_thread()+0x617) [0x55eb95529587] 2024-03-08T21:13:04.698 INFO:journalctl@ceph.mds.d.smithi107.stdout:Mar 08 21:13:04 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-d[72883]: 5: (MDLog::ReplayThread::entry()+0x11) [0x55eb95532091] 2024-03-08T21:13:04.698 INFO:journalctl@ceph.mds.d.smithi107.stdout:Mar 08 21:13:04 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-d[72883]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7f56e4f061ca] 2024-03-08T21:13:04.698 INFO:journalctl@ceph.mds.d.smithi107.stdout:Mar 08 21:13:04 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-d[72883]: 7: clone() 2024-03-08T21:13:04.699 INFO:journalctl@ceph.mds.d.smithi107.stdout:Mar 08 21:13:04 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-d[72883]: 2024-03-08T21:13:04.596+0000 7f56d0800700 -1 *** Caught signal (Segmentation fault) ** 2024-03-08T21:13:04.699 INFO:journalctl@ceph.mds.d.smithi107.stdout:Mar 08 21:13:04 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-d[72883]: in thread 7f56d0800700 thread_name:md_log_replay 2024-03-08T21:13:04.699 INFO:journalctl@ceph.mds.d.smithi107.stdout:Mar 08 21:13:04 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-d[72883]: 2024-03-08T21:13:04.699 INFO:journalctl@ceph.mds.d.smithi107.stdout:Mar 08 21:13:04 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-d[72883]: ceph version 19.0.0-1965-g7d63482a (7d63482aa89628d733e298984a762389b625c761) squid (dev) 2024-03-08T21:13:04.699 INFO:journalctl@ceph.mds.d.smithi107.stdout:Mar 08 21:13:04 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-d[72883]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f56e4f10d20] 2024-03-08T21:13:04.699 INFO:journalctl@ceph.mds.d.smithi107.stdout:Mar 08 21:13:04 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-d[72883]: 2: /usr/bin/ceph-mds(+0x503eff) [0x55eb954c9eff] 2024-03-08T21:13:04.699 INFO:journalctl@ceph.mds.d.smithi107.stdout:Mar 08 21:13:04 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-d[72883]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x55eb954cd773] 2024-03-08T21:13:04.699 INFO:journalctl@ceph.mds.d.smithi107.stdout:Mar 08 21:13:04 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-d[72883]: 4: (MDLog::_replay_thread()+0x617) [0x55eb95529587] 2024-03-08T21:13:04.699 INFO:journalctl@ceph.mds.d.smithi107.stdout:Mar 08 21:13:04 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-d[72883]: 5: (MDLog::ReplayThread::entry()+0x11) [0x55eb95532091] 2024-03-08T21:13:04.699 INFO:journalctl@ceph.mds.d.smithi107.stdout:Mar 08 21:13:04 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-d[72883]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7f56e4f061ca] 2024-03-08T21:13:04.699 INFO:journalctl@ceph.mds.d.smithi107.stdout:Mar 08 21:13:04 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-d[72883]: 7: clone() 2024-03-08T21:13:04.699 INFO:journalctl@ceph.mds.d.smithi107.stdout:Mar 08 21:13:04 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-d[72883]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-08T21:13:04.699 INFO:journalctl@ceph.mds.d.smithi107.stdout:Mar 08 21:13:04 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-d[72883]: 2024-03-08T21:13:04.700 INFO:journalctl@ceph.mds.d.smithi107.stdout:Mar 08 21:13:04 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-d[72883]: 0> 2024-03-08T21:13:04.596+0000 7f56d0800700 -1 *** Caught signal (Segmentation fault) ** 2024-03-08T21:13:04.700 INFO:journalctl@ceph.mds.d.smithi107.stdout:Mar 08 21:13:04 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-d[72883]: in thread 7f56d0800700 thread_name:md_log_replay 2024-03-08T21:13:04.700 INFO:journalctl@ceph.mds.d.smithi107.stdout:Mar 08 21:13:04 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-d[72883]: 2024-03-08T21:13:04.700 INFO:journalctl@ceph.mds.d.smithi107.stdout:Mar 08 21:13:04 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-d[72883]: ceph version 19.0.0-1965-g7d63482a (7d63482aa89628d733e298984a762389b625c761) squid (dev) 2024-03-08T21:13:04.700 INFO:journalctl@ceph.mds.d.smithi107.stdout:Mar 08 21:13:04 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-d[72883]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f56e4f10d20] 2024-03-08T21:13:04.700 INFO:journalctl@ceph.mds.d.smithi107.stdout:Mar 08 21:13:04 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-d[72883]: 2: /usr/bin/ceph-mds(+0x503eff) [0x55eb954c9eff] 2024-03-08T21:13:04.700 INFO:journalctl@ceph.mds.d.smithi107.stdout:Mar 08 21:13:04 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-d[72883]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x55eb954cd773] 2024-03-08T21:13:04.700 INFO:journalctl@ceph.mds.d.smithi107.stdout:Mar 08 21:13:04 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-d[72883]: 4: (MDLog::_replay_thread()+0x617) [0x55eb95529587] 2024-03-08T21:13:04.700 INFO:journalctl@ceph.mds.d.smithi107.stdout:Mar 08 21:13:04 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-d[72883]: 5: (MDLog::ReplayThread::entry()+0x11) [0x55eb95532091] 2024-03-08T21:13:04.701 INFO:journalctl@ceph.mds.d.smithi107.stdout:Mar 08 21:13:04 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-d[72883]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7f56e4f061ca] 2024-03-08T21:13:04.701 INFO:journalctl@ceph.mds.d.smithi107.stdout:Mar 08 21:13:04 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-d[72883]: 7: clone() 2024-03-08T21:13:04.701 INFO:journalctl@ceph.mds.d.smithi107.stdout:Mar 08 21:13:04 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-d[72883]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-08T21:13:04.701 INFO:journalctl@ceph.mds.d.smithi107.stdout:Mar 08 21:13:04 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-d[72883]: 2024-03-08T21:13:04.701 INFO:journalctl@ceph.mds.d.smithi107.stdout:Mar 08 21:13:04 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-d[72883]: 0> 2024-03-08T21:13:04.596+0000 7f56d0800700 -1 *** Caught signal (Segmentation fault) ** 2024-03-08T21:13:04.701 INFO:journalctl@ceph.mds.d.smithi107.stdout:Mar 08 21:13:04 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-d[72883]: in thread 7f56d0800700 thread_name:md_log_replay 2024-03-08T21:13:04.701 INFO:journalctl@ceph.mds.d.smithi107.stdout:Mar 08 21:13:04 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-d[72883]: 2024-03-08T21:13:04.701 INFO:journalctl@ceph.mds.d.smithi107.stdout:Mar 08 21:13:04 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-d[72883]: ceph version 19.0.0-1965-g7d63482a (7d63482aa89628d733e298984a762389b625c761) squid (dev) 2024-03-08T21:13:04.701 INFO:journalctl@ceph.mds.d.smithi107.stdout:Mar 08 21:13:04 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-d[72883]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f56e4f10d20] 2024-03-08T21:13:04.701 INFO:journalctl@ceph.mds.d.smithi107.stdout:Mar 08 21:13:04 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-d[72883]: 2: /usr/bin/ceph-mds(+0x503eff) [0x55eb954c9eff] 2024-03-08T21:13:04.701 INFO:journalctl@ceph.mds.d.smithi107.stdout:Mar 08 21:13:04 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-d[72883]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x55eb954cd773] 2024-03-08T21:13:04.701 INFO:journalctl@ceph.mds.d.smithi107.stdout:Mar 08 21:13:04 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-d[72883]: 4: (MDLog::_replay_thread()+0x617) [0x55eb95529587] 2024-03-08T21:13:04.701 INFO:journalctl@ceph.mds.d.smithi107.stdout:Mar 08 21:13:04 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-d[72883]: 5: (MDLog::ReplayThread::entry()+0x11) [0x55eb95532091] 2024-03-08T21:13:04.702 INFO:journalctl@ceph.mds.d.smithi107.stdout:Mar 08 21:13:04 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-d[72883]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7f56e4f061ca] 2024-03-08T21:13:04.702 INFO:journalctl@ceph.mds.d.smithi107.stdout:Mar 08 21:13:04 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-d[72883]: 7: clone() 2024-03-08T21:13:04.702 INFO:journalctl@ceph.mds.d.smithi107.stdout:Mar 08 21:13:04 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-d[72883]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-03-08T21:13:04.702 INFO:journalctl@ceph.mds.d.smithi107.stdout:Mar 08 21:13:04 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-d[72883]: 2024-03-08T21:13:04.702 INFO:journalctl@ceph.mds.d.smithi107.stdout:Mar 08 21:13:04 smithi107 podman[91998]: 2024-03-08 21:13:04.720511266 +0000 UTC m=+0.040341425 container died d1bfb094648089fee45743483afde5b3705a8deeee706e70572d47baebad5962 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-d, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=rishabh-2024mar8-7d63482, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307) 2024-03-08T21:13:04.759 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:13:04 smithi149 ceph-mon[35042]: pgmap v395: 97 pgs: 97 active+clean; 8.1 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:13:04.872 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:13:04 smithi204 ceph-mon[34935]: pgmap v395: 97 pgs: 97 active+clean; 8.1 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:13:05.711 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:13:05 smithi107 ceph-mon[31381]: osdmap e107: 12 total, 12 up, 12 in 2024-03-08T21:13:05.711 INFO:journalctl@ceph.mds.d.smithi107.stdout:Mar 08 21:13:05 smithi107 podman[91998]: 2024-03-08 21:13:05.617673818 +0000 UTC m=+0.937504023 container remove d1bfb094648089fee45743483afde5b3705a8deeee706e70572d47baebad5962 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-d, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, RELEASE=rishabh-2024mar8-7d63482, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-03-08T21:13:05.711 INFO:journalctl@ceph.mds.d.smithi107.stdout:Mar 08 21:13:05 smithi107 systemd[1]: ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715@mds.d.service: Main process exited, code=exited, status=139/n/a 2024-03-08T21:13:05.759 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:13:05 smithi149 ceph-mon[35042]: osdmap e107: 12 total, 12 up, 12 in 2024-03-08T21:13:05.872 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:13:05 smithi204 ceph-mon[34935]: osdmap e107: 12 total, 12 up, 12 in 2024-03-08T21:13:06.417 INFO:journalctl@ceph.mds.d.smithi107.stdout:Mar 08 21:13:06 smithi107 systemd[1]: ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715@mds.d.service: Failed with result 'exit-code'. 2024-03-08T21:13:06.418 INFO:journalctl@ceph.mds.d.smithi107.stdout:Mar 08 21:13:06 smithi107 systemd[1]: ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715@mds.d.service: Consumed 2.260s CPU time. 2024-03-08T21:13:06.711 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:13:06 smithi107 ceph-mon[31381]: pgmap v397: 97 pgs: 97 active+clean; 8.8 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 124 KiB/s wr, 33 op/s 2024-03-08T21:13:06.759 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:13:06 smithi149 ceph-mon[35042]: pgmap v397: 97 pgs: 97 active+clean; 8.8 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 124 KiB/s wr, 33 op/s 2024-03-08T21:13:06.872 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:13:06 smithi204 ceph-mon[34935]: pgmap v397: 97 pgs: 97 active+clean; 8.8 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 124 KiB/s wr, 33 op/s 2024-03-08T21:13:07.711 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:13:07 smithi107 ceph-mon[31381]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T21:13:07.759 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:13:07 smithi149 ceph-mon[35042]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T21:13:07.873 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:13:07 smithi204 ceph-mon[34935]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T21:13:08.711 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:13:08 smithi107 ceph-mon[31381]: pgmap v398: 97 pgs: 97 active+clean; 8.8 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 124 KiB/s wr, 33 op/s 2024-03-08T21:13:08.712 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:13:08 smithi107 ceph-mon[31381]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T21:13:08.712 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:13:08 smithi107 ceph-mon[31381]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T21:13:08.712 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:13:08 smithi107 ceph-mon[31381]: from='mgr.15375 ' entity='mgr.x' 2024-03-08T21:13:08.759 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:13:08 smithi149 ceph-mon[35042]: pgmap v398: 97 pgs: 97 active+clean; 8.8 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 124 KiB/s wr, 33 op/s 2024-03-08T21:13:08.759 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:13:08 smithi149 ceph-mon[35042]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T21:13:08.759 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:13:08 smithi149 ceph-mon[35042]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T21:13:08.759 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:13:08 smithi149 ceph-mon[35042]: from='mgr.15375 ' entity='mgr.x' 2024-03-08T21:13:08.775 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:13:08 smithi204 ceph-mon[34935]: pgmap v398: 97 pgs: 97 active+clean; 8.8 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 124 KiB/s wr, 33 op/s 2024-03-08T21:13:08.775 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:13:08 smithi204 ceph-mon[34935]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T21:13:08.775 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:13:08 smithi204 ceph-mon[34935]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T21:13:08.775 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:13:08 smithi204 ceph-mon[34935]: from='mgr.15375 ' entity='mgr.x' 2024-03-08T21:13:10.711 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:13:10 smithi107 ceph-mon[31381]: pgmap v399: 97 pgs: 97 active+clean; 8.8 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 124 KiB/s wr, 32 op/s 2024-03-08T21:13:10.759 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:13:10 smithi149 ceph-mon[35042]: pgmap v399: 97 pgs: 97 active+clean; 8.8 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 124 KiB/s wr, 32 op/s 2024-03-08T21:13:10.872 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:13:10 smithi204 ceph-mon[34935]: pgmap v399: 97 pgs: 97 active+clean; 8.8 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 124 KiB/s wr, 32 op/s 2024-03-08T21:13:12.711 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:13:12 smithi107 ceph-mon[31381]: pgmap v400: 97 pgs: 97 active+clean; 8.8 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 124 KiB/s wr, 32 op/s 2024-03-08T21:13:12.759 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:13:12 smithi149 ceph-mon[35042]: pgmap v400: 97 pgs: 97 active+clean; 8.8 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 124 KiB/s wr, 32 op/s 2024-03-08T21:13:12.872 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:13:12 smithi204 ceph-mon[34935]: pgmap v400: 97 pgs: 97 active+clean; 8.8 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 124 KiB/s wr, 32 op/s 2024-03-08T21:13:14.711 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:13:14 smithi107 ceph-mon[31381]: pgmap v401: 97 pgs: 97 active+clean; 8.8 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 124 KiB/s wr, 32 op/s 2024-03-08T21:13:14.712 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:13:14 smithi107 ceph-mon[31381]: osdmap e108: 12 total, 12 up, 12 in 2024-03-08T21:13:14.759 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:13:14 smithi149 ceph-mon[35042]: pgmap v401: 97 pgs: 97 active+clean; 8.8 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 124 KiB/s wr, 32 op/s 2024-03-08T21:13:14.759 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:13:14 smithi149 ceph-mon[35042]: osdmap e108: 12 total, 12 up, 12 in 2024-03-08T21:13:14.872 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:13:14 smithi204 ceph-mon[34935]: pgmap v401: 97 pgs: 97 active+clean; 8.8 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 124 KiB/s wr, 32 op/s 2024-03-08T21:13:14.872 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:13:14 smithi204 ceph-mon[34935]: osdmap e108: 12 total, 12 up, 12 in 2024-03-08T21:13:16.620 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:13:16 smithi107 ceph-mon[31381]: pgmap v403: 97 pgs: 97 active+clean; 8.8 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 409 B/s wr, 8 op/s 2024-03-08T21:13:16.620 INFO:journalctl@ceph.mds.d.smithi107.stdout:Mar 08 21:13:16 smithi107 systemd[1]: ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715@mds.d.service: Scheduled restart job, restart counter is at 1. 2024-03-08T21:13:16.620 INFO:journalctl@ceph.mds.d.smithi107.stdout:Mar 08 21:13:16 smithi107 systemd[1]: Stopped Ceph mds.d for dc20f0f8-dd8c-11ee-95c6-87774f69a715. 2024-03-08T21:13:16.620 INFO:journalctl@ceph.mds.d.smithi107.stdout:Mar 08 21:13:16 smithi107 systemd[1]: ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715@mds.d.service: Consumed 2.260s CPU time. 2024-03-08T21:13:16.620 INFO:journalctl@ceph.mds.d.smithi107.stdout:Mar 08 21:13:16 smithi107 systemd[1]: Starting Ceph mds.d for dc20f0f8-dd8c-11ee-95c6-87774f69a715... 2024-03-08T21:13:16.759 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:13:16 smithi149 ceph-mon[35042]: pgmap v403: 97 pgs: 97 active+clean; 8.8 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 409 B/s wr, 8 op/s 2024-03-08T21:13:16.874 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:13:16 smithi204 ceph-mon[34935]: pgmap v403: 97 pgs: 97 active+clean; 8.8 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 409 B/s wr, 8 op/s 2024-03-08T21:13:16.962 INFO:journalctl@ceph.mds.d.smithi107.stdout:Mar 08 21:13:16 smithi107 podman[92219]: 2024-03-08 21:13:16.549636391 +0000 UTC m=+0.039017345 image pull a272146cb810b1fd4a53627f019472132e30966d5c25b01bfc44a001295f1249 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd 2024-03-08T21:13:16.962 INFO:journalctl@ceph.mds.d.smithi107.stdout:Mar 08 21:13:16 smithi107 podman[92219]: 2024-03-08 21:13:16.684570163 +0000 UTC m=+0.173951137 container create 4591026eeab2fd936f64158ced5c82a9e6430b80aea54d4523d7fc9b1184554f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-d, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=rishabh-2024mar8-7d63482, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307) 2024-03-08T21:13:17.344 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.367+0000 7f5d97e00640 1 -- 172.21.15.107:0/3488485096 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5d9810d980 msgr2=0x7f5d9810fd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:13:17.344 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.367+0000 7f5d97e00640 1 --2- 172.21.15.107:0/3488485096 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5d9810d980 0x7f5d9810fd90 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f5d90009900 tx=0x7f5d9002bcd0 comp rx=0 tx=0).stop 2024-03-08T21:13:17.344 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.367+0000 7f5d97e00640 1 -- 172.21.15.107:0/3488485096 shutdown_connections 2024-03-08T21:13:17.344 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.367+0000 7f5d97e00640 1 --2- 172.21.15.107:0/3488485096 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5d9810d980 0x7f5d9810fd90 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:13:17.344 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.367+0000 7f5d97e00640 1 --2- 172.21.15.107:0/3488485096 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5d98073fe0 0x7f5d9810d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:13:17.345 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.367+0000 7f5d97e00640 1 --2- 172.21.15.107:0/3488485096 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5d980736c0 0x7f5d98073aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:13:17.345 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.367+0000 7f5d97e00640 1 -- 172.21.15.107:0/3488485096 >> 172.21.15.107:0/3488485096 conn(0x7f5d980fcc30 msgr2=0x7f5d980ff050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:13:17.345 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.367+0000 7f5d97e00640 1 -- 172.21.15.107:0/3488485096 shutdown_connections 2024-03-08T21:13:17.345 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.367+0000 7f5d97e00640 1 -- 172.21.15.107:0/3488485096 wait complete. 2024-03-08T21:13:17.347 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.368+0000 7f5d97e00640 1 Processor -- start 2024-03-08T21:13:17.347 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.368+0000 7f5d97e00640 1 -- start start 2024-03-08T21:13:17.347 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.369+0000 7f5d97e00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5d980736c0 0x7f5d9819f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:13:17.347 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.369+0000 7f5d97e00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5d98073fe0 0x7f5d9819fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:13:17.347 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.369+0000 7f5d97e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5d9810d980 0x7f5d981999a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:13:17.347 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.369+0000 7f5d97e00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d981135c0 con 0x7f5d9810d980 2024-03-08T21:13:17.347 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.369+0000 7f5d97e00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d98113440 con 0x7f5d980736c0 2024-03-08T21:13:17.348 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.369+0000 7f5d97e00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d98113740 con 0x7f5d98073fe0 2024-03-08T21:13:17.348 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.369+0000 7f5d97400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5d9810d980 0x7f5d981999a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:13:17.348 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.369+0000 7f5d97400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5d9810d980 0x7f5d981999a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:35180/0 (socket says 172.21.15.107:35180) 2024-03-08T21:13:17.348 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.369+0000 7f5d97400640 1 -- 172.21.15.107:0/4060481076 learned_addr learned my addr 172.21.15.107:0/4060481076 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:13:17.348 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.369+0000 7f5d96a00640 1 --2- 172.21.15.107:0/4060481076 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5d980736c0 0x7f5d9819f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:13:17.348 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.369+0000 7f5d96000640 1 --2- 172.21.15.107:0/4060481076 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5d98073fe0 0x7f5d9819fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:13:17.348 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.369+0000 7f5d97400640 1 -- 172.21.15.107:0/4060481076 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5d98073fe0 msgr2=0x7f5d9819fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:13:17.348 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.369+0000 7f5d97400640 1 --2- 172.21.15.107:0/4060481076 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5d98073fe0 0x7f5d9819fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:13:17.348 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.369+0000 7f5d97400640 1 -- 172.21.15.107:0/4060481076 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5d980736c0 msgr2=0x7f5d9819f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:13:17.348 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.369+0000 7f5d97400640 1 --2- 172.21.15.107:0/4060481076 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5d980736c0 0x7f5d9819f7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:13:17.348 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.369+0000 7f5d97400640 1 -- 172.21.15.107:0/4060481076 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d9819a120 con 0x7f5d9810d980 2024-03-08T21:13:17.348 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.370+0000 7f5d97400640 1 --2- 172.21.15.107:0/4060481076 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5d9810d980 0x7f5d981999a0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f5d9002c210 tx=0x7f5d900045c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:13:17.348 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.370+0000 7f5d9ca00640 1 -- 172.21.15.107:0/4060481076 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d90043020 con 0x7f5d9810d980 2024-03-08T21:13:17.348 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.370+0000 7f5d9ca00640 1 -- 172.21.15.107:0/4060481076 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f5d9002dce0 con 0x7f5d9810d980 2024-03-08T21:13:17.349 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.370+0000 7f5d97e00640 1 -- 172.21.15.107:0/4060481076 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d9819a3b0 con 0x7f5d9810d980 2024-03-08T21:13:17.349 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.370+0000 7f5d9ca00640 1 -- 172.21.15.107:0/4060481076 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d90003a10 con 0x7f5d9810d980 2024-03-08T21:13:17.349 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.370+0000 7f5d97e00640 1 -- 172.21.15.107:0/4060481076 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d981a6660 con 0x7f5d9810d980 2024-03-08T21:13:17.351 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.374+0000 7f5d9ca00640 1 -- 172.21.15.107:0/4060481076 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) v1 ==== 137422+0+0 (secure 0 0 0) 0x7f5d90046020 con 0x7f5d9810d980 2024-03-08T21:13:17.351 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.374+0000 7f5da0c10740 1 Processor -- start 2024-03-08T21:13:17.351 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.374+0000 7f5da0c10740 1 -- start start 2024-03-08T21:13:17.353 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.375+0000 7f5da0c10740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ea66bc9e80 0x55ea66bca260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:13:17.353 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.375+0000 7f5da0c10740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea66bca7a0 0x55ea66bd3030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:13:17.353 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.375+0000 7f5da0c10740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ea66bd3570 0x55ea66bd5960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:13:17.353 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.375+0000 7f5da0c10740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea66b2d3d0 con 0x55ea66bd3570 2024-03-08T21:13:17.353 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.375+0000 7f5da0c10740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea66ad66b0 con 0x55ea66bc9e80 2024-03-08T21:13:17.353 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.375+0000 7f5da0c10740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea66b29470 con 0x55ea66bca7a0 2024-03-08T21:13:17.353 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.375+0000 7f5d96000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ea66bd3570 0x55ea66bd5960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:13:17.353 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.375+0000 7f5d96000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ea66bd3570 0x55ea66bd5960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:35186/0 (socket says 172.21.15.107:35186) 2024-03-08T21:13:17.353 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.375+0000 7f5d96000640 1 -- 172.21.15.107:0/1082353305 learned_addr learned my addr 172.21.15.107:0/1082353305 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:13:17.353 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.375+0000 7f5d96000640 1 --2- 172.21.15.107:0/1082353305 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ea66bc9e80 0x55ea66bca260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:13:17.354 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.375+0000 7f5d96a00640 1 --2- 172.21.15.107:0/1082353305 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea66bca7a0 0x55ea66bd3030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:13:17.354 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.375+0000 7f5d96000640 1 -- 172.21.15.107:0/1082353305 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea66bca7a0 msgr2=0x55ea66bd3030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:13:17.354 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.375+0000 7f5d96000640 1 --2- 172.21.15.107:0/1082353305 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea66bca7a0 0x55ea66bd3030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:13:17.354 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.375+0000 7f5d96000640 1 -- 172.21.15.107:0/1082353305 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ea66bc9e80 msgr2=0x55ea66bca260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:13:17.354 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.375+0000 7f5d96000640 1 --2- 172.21.15.107:0/1082353305 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ea66bc9e80 0x55ea66bca260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:13:17.354 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.375+0000 7f5d96000640 1 -- 172.21.15.107:0/1082353305 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ea66afcb50 con 0x55ea66bd3570 2024-03-08T21:13:17.354 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.376+0000 7f5d9ca00640 1 --2- 172.21.15.107:0/4060481076 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f5d7c0a2960 0x7f5d7c0a4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:13:17.354 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.376+0000 7f5d96000640 1 --2- 172.21.15.107:0/1082353305 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ea66bd3570 0x55ea66bd5960 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f5d84013550 tx=0x7f5d8400c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=974508768d4b7c76 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:13:17.354 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.376+0000 7f5d9ca00640 1 -- 172.21.15.107:0/4060481076 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f5d900f0fe0 con 0x7f5d9810d980 2024-03-08T21:13:17.354 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.376+0000 7f5d96a00640 1 --2- 172.21.15.107:0/4060481076 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f5d7c0a2960 0x7f5d7c0a4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:13:17.354 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.376+0000 7f5d83e00640 1 -- 172.21.15.107:0/1082353305 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d84019030 con 0x55ea66bd3570 2024-03-08T21:13:17.354 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.376+0000 7f5d83e00640 1 -- 172.21.15.107:0/1082353305 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f5d840054d0 con 0x55ea66bd3570 2024-03-08T21:13:17.354 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.376+0000 7f5d83e00640 1 -- 172.21.15.107:0/1082353305 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d84011440 con 0x55ea66bd3570 2024-03-08T21:13:17.357 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.377+0000 7f5da0c10740 1 -- 172.21.15.107:0/1082353305 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ea66bd3570 msgr2=0x55ea66bd5960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:13:17.358 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.377+0000 7f5d96a00640 1 --2- 172.21.15.107:0/4060481076 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f5d7c0a2960 0x7f5d7c0a4e20 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x55ea66bc9930 tx=0x7f5d88009580 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:13:17.358 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.377+0000 7f5da0c10740 1 --2- 172.21.15.107:0/1082353305 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ea66bd3570 0x55ea66bd5960 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f5d84013550 tx=0x7f5d8400c430 comp rx=0 tx=0).stop 2024-03-08T21:13:17.358 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.377+0000 7f5da0c10740 1 -- 172.21.15.107:0/1082353305 shutdown_connections 2024-03-08T21:13:17.358 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.377+0000 7f5da0c10740 1 --2- 172.21.15.107:0/1082353305 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ea66bd3570 0x55ea66bd5960 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:13:17.358 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.377+0000 7f5da0c10740 1 --2- 172.21.15.107:0/1082353305 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea66bca7a0 0x55ea66bd3030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:13:17.358 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.377+0000 7f5da0c10740 1 --2- 172.21.15.107:0/1082353305 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ea66bc9e80 0x55ea66bca260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:13:17.358 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.377+0000 7f5da0c10740 1 -- 172.21.15.107:0/1082353305 >> 172.21.15.107:0/1082353305 conn(0x55ea66bc47a0 msgr2=0x55ea66bc4b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:13:17.358 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.378+0000 7f5da0c10740 1 -- 172.21.15.107:0/1082353305 shutdown_connections 2024-03-08T21:13:17.358 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.378+0000 7f5da0c10740 1 -- 172.21.15.107:0/1082353305 wait complete. 2024-03-08T21:13:17.358 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.378+0000 7f5da0c10740 1 -- start start 2024-03-08T21:13:17.358 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.378+0000 7f5da0c10740 1 Processor -- start 2024-03-08T21:13:17.358 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.379+0000 7f5da0c10740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ea66bc9e80 0x55ea66bd2f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:13:17.359 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.379+0000 7f5da0c10740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ea66bca7a0 0x55ea66bd02f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:13:17.359 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.379+0000 7f5da0c10740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea66bd3570 0x55ea66bd0900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:13:17.359 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.379+0000 7f5da0c10740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea66bd61a0 con 0x55ea66bca7a0 2024-03-08T21:13:17.359 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.379+0000 7f5da0c10740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea66ae99f0 con 0x55ea66bc9e80 2024-03-08T21:13:17.359 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.379+0000 7f5da0c10740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea66bd6320 con 0x55ea66bd3570 2024-03-08T21:13:17.359 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.379+0000 7f5da0c10740 10 client.0 _ll_register_callbacks cb 0x55ea66841a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T21:13:17.359 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.379+0000 7f5d96000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ea66bca7a0 0x55ea66bd02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:13:17.359 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.379+0000 7f5d96000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ea66bca7a0 0x55ea66bd02f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:35198/0 (socket says 172.21.15.107:35198) 2024-03-08T21:13:17.359 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.379+0000 7f5d96000640 1 -- 172.21.15.107:0/3096082001 learned_addr learned my addr 172.21.15.107:0/3096082001 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:13:17.359 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.379+0000 7f5d96a00640 1 --2- 172.21.15.107:0/3096082001 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ea66bc9e80 0x55ea66bd2f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:13:17.359 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.380+0000 7f5d97400640 1 --2- 172.21.15.107:0/3096082001 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea66bd3570 0x55ea66bd0900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:13:17.359 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.380+0000 7f5d96000640 1 -- 172.21.15.107:0/3096082001 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea66bd3570 msgr2=0x55ea66bd0900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:13:17.359 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.380+0000 7f5d96000640 1 --2- 172.21.15.107:0/3096082001 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea66bd3570 0x55ea66bd0900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:13:17.360 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.380+0000 7f5d96000640 1 -- 172.21.15.107:0/3096082001 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ea66bc9e80 msgr2=0x55ea66bd2f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:13:17.360 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.380+0000 7f5d96000640 1 --2- 172.21.15.107:0/3096082001 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ea66bc9e80 0x55ea66bd2f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:13:17.360 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.380+0000 7f5d96000640 1 -- 172.21.15.107:0/3096082001 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ea66842210 con 0x55ea66bca7a0 2024-03-08T21:13:17.360 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.380+0000 7f5d97400640 1 --2- 172.21.15.107:0/3096082001 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea66bd3570 0x55ea66bd0900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T21:13:17.360 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.380+0000 7f5d96a00640 1 --2- 172.21.15.107:0/3096082001 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ea66bc9e80 0x55ea66bd2f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T21:13:17.360 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.380+0000 7f5d96000640 1 --2- 172.21.15.107:0/3096082001 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ea66bca7a0 0x55ea66bd02f0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f5d840057c0 tx=0x7f5d8400ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2321b6f4685e65dd server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:13:17.360 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.380+0000 7f5d80c00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-03-08T21:13:17.360 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.380+0000 7f5d80c00640 1 -- 172.21.15.107:0/3096082001 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d84004030 con 0x55ea66bca7a0 2024-03-08T21:13:17.360 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.381+0000 7f5da0c10740 1 -- 172.21.15.107:0/3096082001 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5d840054d0 con 0x55ea66bca7a0 2024-03-08T21:13:17.360 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.381+0000 7f5d80c00640 1 -- 172.21.15.107:0/3096082001 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f5d840041d0 con 0x55ea66bca7a0 2024-03-08T21:13:17.360 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.381+0000 7f5d80c00640 1 -- 172.21.15.107:0/3096082001 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5d840043d0 con 0x55ea66bca7a0 2024-03-08T21:13:17.360 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.381+0000 7f5da0c10740 10 client.16515 fetch_fsmap learned FSMap version 39 2024-03-08T21:13:17.360 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.381+0000 7f5da0c10740 1 -- 172.21.15.107:0/3096082001 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x55ea6690e620 con 0x55ea66bca7a0 2024-03-08T21:13:17.362 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.381+0000 7f5d80c00640 1 -- 172.21.15.107:0/3096082001 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f5d840054d0 con 0x55ea66bca7a0 2024-03-08T21:13:17.362 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.381+0000 7f5d80c00640 1 -- 172.21.15.107:0/3096082001 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 39) v1 ==== 7307+0+0 (secure 0 0 0) 0x7f5d84026020 con 0x55ea66bca7a0 2024-03-08T21:13:17.362 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.382+0000 7f5da0c10740 10 client.16515 fetch_fsmap finished waiting for FSMap version 39 2024-03-08T21:13:17.363 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.382+0000 7f5da0c10740 10 client.16515 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-08T21:13:17.363 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.382+0000 7f5da0c10740 20 client.16515 populate_metadata read hostname 'smithi107' 2024-03-08T21:13:17.363 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.382+0000 7f5da0c10740 1 --2- 172.21.15.107:0/3096082001 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55ea66be33c0 0x55ea66be57b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:13:17.363 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.382+0000 7f5da0c10740 4 client.16515 mds_command: new command op to 34475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-08T21:13:17.363 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.382+0000 7f5da0c10740 1 -- 172.21.15.107:0/3096082001 --> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ea66b3a910 con 0x55ea66be33c0 2024-03-08T21:13:17.363 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.382+0000 7f5d96a00640 1 --2- 172.21.15.107:0/3096082001 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55ea66be33c0 0x55ea66be57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:13:17.363 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.383+0000 7f5d96a00640 1 --2- 172.21.15.107:0/3096082001 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55ea66be33c0 0x55ea66be57b0 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=44117f11a115fbfa server_cookie=7a3f675bc839ba3c in_seq=0 out_seq=0 2024-03-08T21:13:17.363 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.383+0000 7f5d80c00640 10 client.16515 ms_handle_connect on v2:172.21.15.149:6838/55290596 2024-03-08T21:13:17.363 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.385+0000 7f5d80c00640 1 -- 172.21.15.107:0/3096082001 <== mds.0 v2:172.21.15.149:6838/55290596 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ea66b3a910 con 0x55ea66be33c0 2024-03-08T21:13:17.363 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.385+0000 7f5d80c00640 10 client.16515 handle_command_reply: tid=0 2024-03-08T21:13:17.363 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.385+0000 7f5da0c10740 1 client.16515 shutdown 2024-03-08T21:13:17.363 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.385+0000 7f5da0c10740 1 -- 172.21.15.107:0/3096082001 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ea66bca7a0 msgr2=0x55ea66bd02f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:13:17.364 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.385+0000 7f5da0c10740 1 --2- 172.21.15.107:0/3096082001 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ea66bca7a0 0x55ea66bd02f0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f5d840057c0 tx=0x7f5d8400ced0 comp rx=0 tx=0).stop 2024-03-08T21:13:17.364 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.385+0000 7f5da0c10740 1 -- 172.21.15.107:0/3096082001 shutdown_connections 2024-03-08T21:13:17.364 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.385+0000 7f5da0c10740 1 --2- 172.21.15.107:0/3096082001 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea66bd3570 0x55ea66bd0900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:13:17.364 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.385+0000 7f5da0c10740 1 --2- 172.21.15.107:0/3096082001 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ea66bca7a0 0x55ea66bd02f0 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:13:17.364 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.385+0000 7f5da0c10740 1 --2- 172.21.15.107:0/3096082001 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55ea66be33c0 0x55ea66be57b0 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:13:17.364 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.385+0000 7f5da0c10740 1 --2- 172.21.15.107:0/3096082001 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ea66bc9e80 0x55ea66bd2f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:13:17.364 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.385+0000 7f5da0c10740 1 -- 172.21.15.107:0/3096082001 >> 172.21.15.107:0/3096082001 conn(0x55ea66bc47a0 msgr2=0x55ea66bc64f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:13:17.364 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.385+0000 7f5d80c00640 0 client.16515 ms_handle_reset on v2:172.21.15.149:6838/55290596 2024-03-08T21:13:17.364 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.385+0000 7f5da0c10740 1 -- 172.21.15.107:0/3096082001 shutdown_connections 2024-03-08T21:13:17.364 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.385+0000 7f5da0c10740 1 -- 172.21.15.107:0/3096082001 wait complete. 2024-03-08T21:13:17.364 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.386+0000 7f5da0c10740 20 client.16515 trim_cache size 0 max 16384 2024-03-08T21:13:17.376 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.400+0000 7f5da0c10740 1 Processor -- start 2024-03-08T21:13:17.376 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.400+0000 7f5da0c10740 1 -- start start 2024-03-08T21:13:17.377 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.400+0000 7f5da0c10740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ea66bc9e80 0x55ea66bd2a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:13:17.378 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.400+0000 7f5da0c10740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea66bca7a0 0x55ea66bd2f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:13:17.378 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.400+0000 7f5da0c10740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ea66bd3570 0x55ea66bcebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:13:17.378 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.400+0000 7f5da0c10740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d981b0660 con 0x55ea66bd3570 2024-03-08T21:13:17.378 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.400+0000 7f5da0c10740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d981ae680 con 0x55ea66bc9e80 2024-03-08T21:13:17.378 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.400+0000 7f5da0c10740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d981b0000 con 0x55ea66bca7a0 2024-03-08T21:13:17.378 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.401+0000 7f5d97400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ea66bd3570 0x55ea66bcebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:13:17.378 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.401+0000 7f5d97400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ea66bd3570 0x55ea66bcebc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:35200/0 (socket says 172.21.15.107:35200) 2024-03-08T21:13:17.378 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.401+0000 7f5d97400640 1 -- 172.21.15.107:0/1232158158 learned_addr learned my addr 172.21.15.107:0/1232158158 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:13:17.378 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.401+0000 7f5d96000640 1 --2- 172.21.15.107:0/1232158158 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea66bca7a0 0x55ea66bd2f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:13:17.378 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.401+0000 7f5d96a00640 1 --2- 172.21.15.107:0/1232158158 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ea66bc9e80 0x55ea66bd2a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:13:17.378 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.401+0000 7f5d97400640 1 -- 172.21.15.107:0/1232158158 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea66bca7a0 msgr2=0x55ea66bd2f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:13:17.378 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.401+0000 7f5d97400640 1 --2- 172.21.15.107:0/1232158158 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea66bca7a0 0x55ea66bd2f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:13:17.378 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.401+0000 7f5d97400640 1 -- 172.21.15.107:0/1232158158 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ea66bc9e80 msgr2=0x55ea66bd2a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:13:17.379 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.401+0000 7f5d97400640 1 --2- 172.21.15.107:0/1232158158 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ea66bc9e80 0x55ea66bd2a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:13:17.379 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.401+0000 7f5d97400640 1 -- 172.21.15.107:0/1232158158 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ea66bea950 con 0x55ea66bd3570 2024-03-08T21:13:17.380 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.401+0000 7f5d97400640 1 --2- 172.21.15.107:0/1232158158 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ea66bd3570 0x55ea66bcebc0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f5d90005ab0 tx=0x7f5d900092e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc6cd648291fd8ed server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:13:17.380 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.402+0000 7f5d83400640 1 -- 172.21.15.107:0/1232158158 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d900d6070 con 0x55ea66bd3570 2024-03-08T21:13:17.380 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.402+0000 7f5d83400640 1 -- 172.21.15.107:0/1232158158 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f5d900bde10 con 0x55ea66bd3570 2024-03-08T21:13:17.380 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.402+0000 7f5d83400640 1 -- 172.21.15.107:0/1232158158 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d900b9d80 con 0x55ea66bd3570 2024-03-08T21:13:17.380 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.402+0000 7f5da0c10740 1 -- 172.21.15.107:0/1232158158 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ea66bd3570 msgr2=0x55ea66bcebc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:13:17.380 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.402+0000 7f5da0c10740 1 --2- 172.21.15.107:0/1232158158 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ea66bd3570 0x55ea66bcebc0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f5d90005ab0 tx=0x7f5d900092e0 comp rx=0 tx=0).stop 2024-03-08T21:13:17.380 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.403+0000 7f5da0c10740 1 -- 172.21.15.107:0/1232158158 shutdown_connections 2024-03-08T21:13:17.380 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.403+0000 7f5da0c10740 1 --2- 172.21.15.107:0/1232158158 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ea66bd3570 0x55ea66bcebc0 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:13:17.380 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.403+0000 7f5da0c10740 1 --2- 172.21.15.107:0/1232158158 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea66bca7a0 0x55ea66bd2f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:13:17.380 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.403+0000 7f5da0c10740 1 --2- 172.21.15.107:0/1232158158 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ea66bc9e80 0x55ea66bd2a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:13:17.380 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.403+0000 7f5da0c10740 1 -- 172.21.15.107:0/1232158158 >> 172.21.15.107:0/1232158158 conn(0x55ea66bc47a0 msgr2=0x55ea66bc4b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:13:17.381 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.403+0000 7f5da0c10740 1 -- 172.21.15.107:0/1232158158 shutdown_connections 2024-03-08T21:13:17.381 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.403+0000 7f5da0c10740 1 -- 172.21.15.107:0/1232158158 wait complete. 2024-03-08T21:13:17.381 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.403+0000 7f5da0c10740 1 -- start start 2024-03-08T21:13:17.381 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.403+0000 7f5da0c10740 1 Processor -- start 2024-03-08T21:13:17.383 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.404+0000 7f5da0c10740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ea66bc9e80 0x55ea66bda850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:13:17.383 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.404+0000 7f5da0c10740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea66bca7a0 0x55ea66bd8c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:13:17.383 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.404+0000 7f5da0c10740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ea66bd3570 0x55ea66bd9160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:13:17.383 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.404+0000 7f5da0c10740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d981af340 con 0x55ea66bd3570 2024-03-08T21:13:17.383 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.404+0000 7f5da0c10740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d981aece0 con 0x55ea66bc9e80 2024-03-08T21:13:17.384 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.404+0000 7f5da0c10740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d981af9a0 con 0x55ea66bca7a0 2024-03-08T21:13:17.384 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.404+0000 7f5da0c10740 10 client.0 _ll_register_callbacks cb 0x7f5d8400c640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T21:13:17.384 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.404+0000 7f5d97400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ea66bd3570 0x55ea66bd9160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:13:17.384 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.404+0000 7f5d97400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ea66bd3570 0x55ea66bd9160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:35212/0 (socket says 172.21.15.107:35212) 2024-03-08T21:13:17.384 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.404+0000 7f5d97400640 1 -- 172.21.15.107:0/3923436819 learned_addr learned my addr 172.21.15.107:0/3923436819 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:13:17.384 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.404+0000 7f5d96a00640 1 --2- 172.21.15.107:0/3923436819 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ea66bc9e80 0x55ea66bda850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:13:17.384 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.405+0000 7f5d96000640 1 --2- 172.21.15.107:0/3923436819 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea66bca7a0 0x55ea66bd8c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:13:17.384 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.405+0000 7f5d97400640 1 -- 172.21.15.107:0/3923436819 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea66bca7a0 msgr2=0x55ea66bd8c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:13:17.384 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.405+0000 7f5d97400640 1 --2- 172.21.15.107:0/3923436819 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea66bca7a0 0x55ea66bd8c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:13:17.384 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.405+0000 7f5d97400640 1 -- 172.21.15.107:0/3923436819 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ea66bc9e80 msgr2=0x55ea66bda850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:13:17.384 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.405+0000 7f5d97400640 1 --2- 172.21.15.107:0/3923436819 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ea66bc9e80 0x55ea66bda850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:13:17.384 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.405+0000 7f5d97400640 1 -- 172.21.15.107:0/3923436819 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ea66bd9960 con 0x55ea66bd3570 2024-03-08T21:13:17.384 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.405+0000 7f5d97400640 1 --2- 172.21.15.107:0/3923436819 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ea66bd3570 0x55ea66bd9160 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f5d90006010 tx=0x7f5d900ba030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acde19cb1f691430 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:13:17.384 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.405+0000 7f5d80c00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-03-08T21:13:17.384 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.405+0000 7f5d80c00640 1 -- 172.21.15.107:0/3923436819 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d900d6040 con 0x55ea66bd3570 2024-03-08T21:13:17.385 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.405+0000 7f5d80c00640 1 -- 172.21.15.107:0/3923436819 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f5d900bba10 con 0x55ea66bd3570 2024-03-08T21:13:17.385 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.405+0000 7f5da0c10740 1 -- 172.21.15.107:0/3923436819 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5d900bde10 con 0x55ea66bd3570 2024-03-08T21:13:17.385 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.406+0000 7f5d80c00640 1 -- 172.21.15.107:0/3923436819 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5d900c1860 con 0x55ea66bd3570 2024-03-08T21:13:17.385 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.406+0000 7f5da0c10740 10 client.16521 fetch_fsmap learned FSMap version 39 2024-03-08T21:13:17.385 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.406+0000 7f5da0c10740 1 -- 172.21.15.107:0/3923436819 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x55ea66c3a830 con 0x55ea66bd3570 2024-03-08T21:13:17.385 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.406+0000 7f5d80c00640 1 -- 172.21.15.107:0/3923436819 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f5d900bde10 con 0x55ea66bd3570 2024-03-08T21:13:17.385 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.406+0000 7f5d80c00640 1 -- 172.21.15.107:0/3923436819 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 39) v1 ==== 7307+0+0 (secure 0 0 0) 0x7f5d90088020 con 0x55ea66bd3570 2024-03-08T21:13:17.385 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.406+0000 7f5da0c10740 10 client.16521 fetch_fsmap finished waiting for FSMap version 39 2024-03-08T21:13:17.385 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.406+0000 7f5da0c10740 10 client.16521 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-08T21:13:17.385 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.406+0000 7f5da0c10740 20 client.16521 populate_metadata read hostname 'smithi107' 2024-03-08T21:13:17.385 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.406+0000 7f5da0c10740 1 --2- 172.21.15.107:0/3923436819 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55ea66c3af00 0x55ea66c3d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:13:17.385 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.406+0000 7f5da0c10740 4 client.16521 mds_command: new command op to 34475 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-08T21:13:17.385 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.406+0000 7f5da0c10740 1 -- 172.21.15.107:0/3923436819 --> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55ea66b34c60 con 0x55ea66c3af00 2024-03-08T21:13:17.385 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.407+0000 7f5d96a00640 1 --2- 172.21.15.107:0/3923436819 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55ea66c3af00 0x55ea66c3d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:13:17.386 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.408+0000 7f5d96a00640 1 --2- 172.21.15.107:0/3923436819 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55ea66c3af00 0x55ea66c3d2f0 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e951dc4855678e9 server_cookie=6a1e965953c8b5d2 in_seq=0 out_seq=0 2024-03-08T21:13:17.386 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.408+0000 7f5d80c00640 10 client.16521 ms_handle_connect on v2:172.21.15.149:6838/55290596 2024-03-08T21:13:17.401 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.424+0000 7f5d80c00640 1 -- 172.21.15.107:0/3923436819 <== mds.0 v2:172.21.15.149:6838/55290596 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55ea66b34c60 con 0x55ea66c3af00 2024-03-08T21:13:17.402 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.424+0000 7f5d80c00640 10 client.16521 handle_command_reply: tid=0 2024-03-08T21:13:17.402 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.424+0000 7f5da0c10740 1 client.16521 shutdown 2024-03-08T21:13:17.402 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.424+0000 7f5da0c10740 1 -- 172.21.15.107:0/3923436819 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ea66bd3570 msgr2=0x55ea66bd9160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:13:17.402 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.424+0000 7f5da0c10740 1 --2- 172.21.15.107:0/3923436819 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ea66bd3570 0x55ea66bd9160 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f5d90006010 tx=0x7f5d900ba030 comp rx=0 tx=0).stop 2024-03-08T21:13:17.402 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.424+0000 7f5da0c10740 1 -- 172.21.15.107:0/3923436819 shutdown_connections 2024-03-08T21:13:17.402 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.425+0000 7f5da0c10740 1 --2- 172.21.15.107:0/3923436819 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ea66bd3570 0x55ea66bd9160 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:13:17.402 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.425+0000 7f5da0c10740 1 --2- 172.21.15.107:0/3923436819 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55ea66bca7a0 0x55ea66bd8c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:13:17.402 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.425+0000 7f5da0c10740 1 --2- 172.21.15.107:0/3923436819 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55ea66c3af00 0x55ea66c3d2f0 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:13:17.402 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.425+0000 7f5da0c10740 1 --2- 172.21.15.107:0/3923436819 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55ea66bc9e80 0x55ea66bda850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:13:17.402 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.425+0000 7f5da0c10740 1 -- 172.21.15.107:0/3923436819 >> 172.21.15.107:0/3923436819 conn(0x55ea66bc47a0 msgr2=0x55ea66bcea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:13:17.402 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.425+0000 7f5d80c00640 0 client.16521 ms_handle_reset on v2:172.21.15.149:6838/55290596 2024-03-08T21:13:17.402 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.425+0000 7f5da0c10740 1 -- 172.21.15.107:0/3923436819 shutdown_connections 2024-03-08T21:13:17.402 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.425+0000 7f5da0c10740 1 -- 172.21.15.107:0/3923436819 wait complete. 2024-03-08T21:13:17.402 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.425+0000 7f5da0c10740 20 client.16521 trim_cache size 0 max 16384 2024-03-08T21:13:17.403 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:{ 2024-03-08T21:13:17.403 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "message": "", 2024-03-08T21:13:17.403 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "return_code": 0 2024-03-08T21:13:17.403 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:} 2024-03-08T21:13:17.404 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.426+0000 7f5d97e00640 1 -- 172.21.15.107:0/4060481076 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f5d7c0a2960 msgr2=0x7f5d7c0a4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:13:17.404 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.426+0000 7f5d97e00640 1 --2- 172.21.15.107:0/4060481076 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f5d7c0a2960 0x7f5d7c0a4e20 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x55ea66bc9930 tx=0x7f5d88009580 comp rx=0 tx=0).stop 2024-03-08T21:13:17.404 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.426+0000 7f5d97e00640 1 -- 172.21.15.107:0/4060481076 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5d9810d980 msgr2=0x7f5d981999a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:13:17.404 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.426+0000 7f5d97e00640 1 --2- 172.21.15.107:0/4060481076 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5d9810d980 0x7f5d981999a0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f5d9002c210 tx=0x7f5d900045c0 comp rx=0 tx=0).stop 2024-03-08T21:13:17.404 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.426+0000 7f5d97e00640 1 -- 172.21.15.107:0/4060481076 shutdown_connections 2024-03-08T21:13:17.404 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.426+0000 7f5d97e00640 1 --2- 172.21.15.107:0/4060481076 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f5d7c0a2960 0x7f5d7c0a4e20 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:13:17.404 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.426+0000 7f5d97e00640 1 --2- 172.21.15.107:0/4060481076 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5d9810d980 0x7f5d981999a0 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:13:17.404 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.426+0000 7f5d97e00640 1 --2- 172.21.15.107:0/4060481076 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5d98073fe0 0x7f5d9819fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:13:17.404 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.426+0000 7f5d97e00640 1 --2- 172.21.15.107:0/4060481076 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5d980736c0 0x7f5d9819f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:13:17.404 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.426+0000 7f5d97e00640 1 -- 172.21.15.107:0/4060481076 >> 172.21.15.107:0/4060481076 conn(0x7f5d980fcc30 msgr2=0x7f5d980fd9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:13:17.404 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.426+0000 7f5d97e00640 1 -- 172.21.15.107:0/4060481076 shutdown_connections 2024-03-08T21:13:17.404 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:17.426+0000 7f5d97e00640 1 -- 172.21.15.107:0/4060481076 wait complete. 2024-03-08T21:13:18.212 INFO:journalctl@ceph.mds.d.smithi107.stdout:Mar 08 21:13:17 smithi107 podman[92219]: 2024-03-08 21:13:17.896767301 +0000 UTC m=+1.386148285 container init 4591026eeab2fd936f64158ced5c82a9e6430b80aea54d4523d7fc9b1184554f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-d, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, RELEASE=rishabh-2024mar8-7d63482, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS) 2024-03-08T21:13:18.212 INFO:journalctl@ceph.mds.d.smithi107.stdout:Mar 08 21:13:17 smithi107 podman[92219]: 2024-03-08 21:13:17.908080761 +0000 UTC m=+1.397461761 container start 4591026eeab2fd936f64158ced5c82a9e6430b80aea54d4523d7fc9b1184554f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-d, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=rishabh-2024mar8-7d63482, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307) 2024-03-08T21:13:18.212 INFO:journalctl@ceph.mds.d.smithi107.stdout:Mar 08 21:13:17 smithi107 ceph-mds[92285]: -- 172.21.15.107:0/3095758104 <== mon.0 v2:172.21.15.107:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x559e5c4d1a40 con 0x559e5c4a4c00 2024-03-08T21:13:18.213 INFO:journalctl@ceph.mds.d.smithi107.stdout:Mar 08 21:13:17 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-d[92234]: starting mds.d at 2024-03-08T21:13:18.213 INFO:journalctl@ceph.mds.d.smithi107.stdout:Mar 08 21:13:18 smithi107 bash[92219]: 4591026eeab2fd936f64158ced5c82a9e6430b80aea54d4523d7fc9b1184554f 2024-03-08T21:13:18.213 INFO:journalctl@ceph.mds.d.smithi107.stdout:Mar 08 21:13:18 smithi107 systemd[1]: Started Ceph mds.d for dc20f0f8-dd8c-11ee-95c6-87774f69a715. 2024-03-08T21:13:18.759 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:13:18 smithi149 ceph-mon[35042]: pgmap v404: 97 pgs: 97 active+clean; 8.8 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 409 B/s wr, 8 op/s 2024-03-08T21:13:18.759 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:13:18 smithi149 ceph-mon[35042]: daemon mds.d restarted 2024-03-08T21:13:18.872 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:13:18 smithi204 ceph-mon[34935]: pgmap v404: 97 pgs: 97 active+clean; 8.8 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 409 B/s wr, 8 op/s 2024-03-08T21:13:18.872 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:13:18 smithi204 ceph-mon[34935]: daemon mds.d restarted 2024-03-08T21:13:18.961 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:13:18 smithi107 ceph-mon[31381]: pgmap v404: 97 pgs: 97 active+clean; 8.8 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 409 B/s wr, 8 op/s 2024-03-08T21:13:18.961 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:13:18 smithi107 ceph-mon[31381]: daemon mds.d restarted 2024-03-08T21:13:19.596 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:13:19 smithi107 ceph-mon[31381]: osdmap e109: 12 total, 12 up, 12 in 2024-03-08T21:13:19.596 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:13:19 smithi107 ceph-mon[31381]: mds.? [v2:172.21.15.107:6836/2838045512,v1:172.21.15.107:6837/2838045512] up:boot 2024-03-08T21:13:19.596 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:13:19 smithi107 ceph-mon[31381]: fsmap cephfs:5 4 up:standby-replay 2 up:standby 5 up:active 2024-03-08T21:13:19.596 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:13:19 smithi107 ceph-mon[31381]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-08T21:13:19.596 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:13:19 smithi107 ceph-mon[31381]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2024-03-08T21:13:19.759 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:13:19 smithi149 ceph-mon[35042]: osdmap e109: 12 total, 12 up, 12 in 2024-03-08T21:13:19.759 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:13:19 smithi149 ceph-mon[35042]: mds.? [v2:172.21.15.107:6836/2838045512,v1:172.21.15.107:6837/2838045512] up:boot 2024-03-08T21:13:19.759 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:13:19 smithi149 ceph-mon[35042]: fsmap cephfs:5 4 up:standby-replay 2 up:standby 5 up:active 2024-03-08T21:13:19.759 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:13:19 smithi149 ceph-mon[35042]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-08T21:13:19.759 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:13:19 smithi149 ceph-mon[35042]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2024-03-08T21:13:19.872 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:13:19 smithi204 ceph-mon[34935]: osdmap e109: 12 total, 12 up, 12 in 2024-03-08T21:13:19.872 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:13:19 smithi204 ceph-mon[34935]: mds.? [v2:172.21.15.107:6836/2838045512,v1:172.21.15.107:6837/2838045512] up:boot 2024-03-08T21:13:19.872 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:13:19 smithi204 ceph-mon[34935]: fsmap cephfs:5 4 up:standby-replay 2 up:standby 5 up:active 2024-03-08T21:13:19.872 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:13:19 smithi204 ceph-mon[34935]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-08T21:13:19.873 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:13:19 smithi204 ceph-mon[34935]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2024-03-08T21:13:20.759 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:13:20 smithi149 ceph-mon[35042]: pgmap v406: 97 pgs: 97 active+clean; 8.8 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 639 B/s wr, 11 op/s 2024-03-08T21:13:20.872 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:13:20 smithi204 ceph-mon[34935]: pgmap v406: 97 pgs: 97 active+clean; 8.8 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 639 B/s wr, 11 op/s 2024-03-08T21:13:20.924 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:13:20 smithi107 ceph-mon[31381]: pgmap v406: 97 pgs: 97 active+clean; 8.8 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 639 B/s wr, 11 op/s 2024-03-08T21:13:22.759 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:13:22 smithi149 ceph-mon[35042]: pgmap v407: 97 pgs: 97 active+clean; 8.8 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 639 B/s wr, 11 op/s 2024-03-08T21:13:22.872 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:13:22 smithi204 ceph-mon[34935]: pgmap v407: 97 pgs: 97 active+clean; 8.8 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 639 B/s wr, 11 op/s 2024-03-08T21:13:22.961 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:13:22 smithi107 ceph-mon[31381]: pgmap v407: 97 pgs: 97 active+clean; 8.8 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 639 B/s wr, 11 op/s 2024-03-08T21:13:24.872 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:13:24 smithi204 ceph-mon[34935]: pgmap v408: 97 pgs: 97 active+clean; 8.8 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 480 B/s wr, 6 op/s 2024-03-08T21:13:24.961 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:13:24 smithi107 ceph-mon[31381]: pgmap v408: 97 pgs: 97 active+clean; 8.8 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 480 B/s wr, 6 op/s 2024-03-08T21:13:25.009 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:13:24 smithi149 ceph-mon[35042]: pgmap v408: 97 pgs: 97 active+clean; 8.8 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 480 B/s wr, 6 op/s 2024-03-08T21:13:26.872 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:13:26 smithi204 ceph-mon[34935]: pgmap v409: 97 pgs: 97 active+clean; 8.8 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 409 B/s wr, 10 op/s 2024-03-08T21:13:26.961 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:13:26 smithi107 ceph-mon[31381]: pgmap v409: 97 pgs: 97 active+clean; 8.8 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 409 B/s wr, 10 op/s 2024-03-08T21:13:27.009 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:13:26 smithi149 ceph-mon[35042]: pgmap v409: 97 pgs: 97 active+clean; 8.8 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 409 B/s wr, 10 op/s 2024-03-08T21:13:28.872 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:13:28 smithi204 ceph-mon[34935]: pgmap v410: 97 pgs: 97 active+clean; 8.8 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 409 B/s wr, 10 op/s 2024-03-08T21:13:28.961 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:13:28 smithi107 ceph-mon[31381]: pgmap v410: 97 pgs: 97 active+clean; 8.8 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 409 B/s wr, 10 op/s 2024-03-08T21:13:29.009 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:13:28 smithi149 ceph-mon[35042]: pgmap v410: 97 pgs: 97 active+clean; 8.8 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 409 B/s wr, 10 op/s 2024-03-08T21:13:30.554 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.576+0000 7f82c6600640 1 -- 172.21.15.107:0/2711841281 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f82c0103ea0 msgr2=0x7f82c0110360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:13:30.555 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.576+0000 7f82c6600640 1 --2- 172.21.15.107:0/2711841281 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f82c0103ea0 0x7f82c0110360 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f82b8009ca0 tx=0x7f82b802c0f0 comp rx=0 tx=0).stop 2024-03-08T21:13:30.555 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.577+0000 7f82c6600640 1 -- 172.21.15.107:0/2711841281 shutdown_connections 2024-03-08T21:13:30.555 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.577+0000 7f82c6600640 1 --2- 172.21.15.107:0/2711841281 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f82c0103ea0 0x7f82c0110360 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:13:30.555 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.577+0000 7f82c6600640 1 --2- 172.21.15.107:0/2711841281 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f82c01034e0 0x7f82c0103960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:13:30.555 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.577+0000 7f82c6600640 1 --2- 172.21.15.107:0/2711841281 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f82c01095a0 0x7f82c0109980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:13:30.555 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.577+0000 7f82c6600640 1 -- 172.21.15.107:0/2711841281 >> 172.21.15.107:0/2711841281 conn(0x7f82c00fefb0 msgr2=0x7f82c01013d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:13:30.555 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.577+0000 7f82c6600640 1 -- 172.21.15.107:0/2711841281 shutdown_connections 2024-03-08T21:13:30.555 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.577+0000 7f82c6600640 1 -- 172.21.15.107:0/2711841281 wait complete. 2024-03-08T21:13:30.557 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.578+0000 7f82c6600640 1 Processor -- start 2024-03-08T21:13:30.557 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.578+0000 7f82c6600640 1 -- start start 2024-03-08T21:13:30.557 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.578+0000 7f82c6600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f82c01034e0 0x7f82c019f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:13:30.557 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.578+0000 7f82c6600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f82c0103ea0 0x7f82c019fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:13:30.558 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.578+0000 7f82c6600640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f82c01095a0 0x7f82c0199990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:13:30.558 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.578+0000 7f82c6600640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82c0113710 con 0x7f82c0103ea0 2024-03-08T21:13:30.558 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.578+0000 7f82c6600640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82c0113590 con 0x7f82c01095a0 2024-03-08T21:13:30.558 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.578+0000 7f82c6600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82c0113890 con 0x7f82c01034e0 2024-03-08T21:13:30.558 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.578+0000 7f82bea00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f82c0103ea0 0x7f82c019fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:13:30.558 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.578+0000 7f82bea00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f82c0103ea0 0x7f82c019fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:37212/0 (socket says 172.21.15.107:37212) 2024-03-08T21:13:30.558 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.578+0000 7f82bea00640 1 -- 172.21.15.107:0/1299186103 learned_addr learned my addr 172.21.15.107:0/1299186103 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:13:30.558 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.579+0000 7f82bf400640 1 --2- 172.21.15.107:0/1299186103 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f82c01034e0 0x7f82c019f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:13:30.558 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.579+0000 7f82bfe00640 1 --2- 172.21.15.107:0/1299186103 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f82c01095a0 0x7f82c0199990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:13:30.558 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.579+0000 7f82bea00640 1 -- 172.21.15.107:0/1299186103 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f82c01034e0 msgr2=0x7f82c019f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:13:30.558 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.579+0000 7f82bea00640 1 --2- 172.21.15.107:0/1299186103 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f82c01034e0 0x7f82c019f7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:13:30.558 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.579+0000 7f82bea00640 1 -- 172.21.15.107:0/1299186103 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f82c01095a0 msgr2=0x7f82c0199990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:13:30.559 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.579+0000 7f82bea00640 1 --2- 172.21.15.107:0/1299186103 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f82c01095a0 0x7f82c0199990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:13:30.559 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.579+0000 7f82bea00640 1 -- 172.21.15.107:0/1299186103 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82c019a110 con 0x7f82c0103ea0 2024-03-08T21:13:30.559 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.579+0000 7f82bfe00640 1 --2- 172.21.15.107:0/1299186103 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f82c01095a0 0x7f82c0199990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T21:13:30.559 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.579+0000 7f82bea00640 1 --2- 172.21.15.107:0/1299186103 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f82c0103ea0 0x7f82c019fd20 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f82ac00e9f0 tx=0x7f82ac00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:13:30.559 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.580+0000 7f82b7e00640 1 -- 172.21.15.107:0/1299186103 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82ac00cd80 con 0x7f82c0103ea0 2024-03-08T21:13:30.559 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.580+0000 7f82b7e00640 1 -- 172.21.15.107:0/1299186103 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f82ac0108e0 con 0x7f82c0103ea0 2024-03-08T21:13:30.559 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.580+0000 7f82c6600640 1 -- 172.21.15.107:0/1299186103 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82c019a400 con 0x7f82c0103ea0 2024-03-08T21:13:30.559 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.580+0000 7f82b7e00640 1 -- 172.21.15.107:0/1299186103 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82ac018920 con 0x7f82c0103ea0 2024-03-08T21:13:30.559 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.580+0000 7f82c6600640 1 -- 172.21.15.107:0/1299186103 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82c01a66a0 con 0x7f82c0103ea0 2024-03-08T21:13:30.561 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.584+0000 7f82c93ab740 1 Processor -- start 2024-03-08T21:13:30.561 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.584+0000 7f82c93ab740 1 -- start start 2024-03-08T21:13:30.563 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.584+0000 7f82c93ab740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562f4ea67c80 0x562f4ea68060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:13:30.563 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.584+0000 7f82c93ab740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562f4ea685a0 0x562f4ea70e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:13:30.563 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.585+0000 7f82c93ab740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562f4ea71370 0x562f4ea73760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:13:30.563 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.585+0000 7f82c93ab740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f4e9cb3d0 con 0x562f4ea67c80 2024-03-08T21:13:30.563 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.585+0000 7f82c93ab740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f4e9746b0 con 0x562f4ea71370 2024-03-08T21:13:30.563 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.585+0000 7f82c93ab740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f4e9c7470 con 0x562f4ea685a0 2024-03-08T21:13:30.563 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.585+0000 7f82bfe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562f4ea67c80 0x562f4ea68060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:13:30.563 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.585+0000 7f82bfe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562f4ea67c80 0x562f4ea68060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:37224/0 (socket says 172.21.15.107:37224) 2024-03-08T21:13:30.563 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.585+0000 7f82bfe00640 1 -- 172.21.15.107:0/1197762682 learned_addr learned my addr 172.21.15.107:0/1197762682 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:13:30.564 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.585+0000 7f82bf400640 1 --2- 172.21.15.107:0/1197762682 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562f4ea685a0 0x562f4ea70e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:13:30.564 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.585+0000 7f82b7e00640 1 -- 172.21.15.107:0/1299186103 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) v1 ==== 137422+0+0 (secure 0 0 0) 0x7f82ac010a90 con 0x7f82c0103ea0 2024-03-08T21:13:30.564 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.585+0000 7f82bfe00640 1 -- 172.21.15.107:0/1197762682 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562f4ea685a0 msgr2=0x562f4ea70e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:13:30.564 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.585+0000 7f82bfe00640 1 --2- 172.21.15.107:0/1197762682 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562f4ea685a0 0x562f4ea70e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:13:30.564 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.585+0000 7f82bfe00640 1 -- 172.21.15.107:0/1197762682 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562f4ea71370 msgr2=0x562f4ea73760 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-08T21:13:30.564 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.585+0000 7f82bfe00640 1 --2- 172.21.15.107:0/1197762682 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562f4ea71370 0x562f4ea73760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:13:30.564 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.585+0000 7f82bfe00640 1 -- 172.21.15.107:0/1197762682 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562f4e99ab50 con 0x562f4ea67c80 2024-03-08T21:13:30.566 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.586+0000 7f82bfe00640 1 --2- 172.21.15.107:0/1197762682 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562f4ea67c80 0x562f4ea68060 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f82b8037040 tx=0x7f82b8009800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1330fa189c2329eb server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:13:30.566 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.586+0000 7f82b4c00640 1 -- 172.21.15.107:0/1197762682 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82b8039050 con 0x562f4ea67c80 2024-03-08T21:13:30.566 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.586+0000 7f82b4c00640 1 -- 172.21.15.107:0/1197762682 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f82b8045070 con 0x562f4ea67c80 2024-03-08T21:13:30.566 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.586+0000 7f82b4c00640 1 -- 172.21.15.107:0/1197762682 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82b8004040 con 0x562f4ea67c80 2024-03-08T21:13:30.566 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.587+0000 7f82c93ab740 1 -- 172.21.15.107:0/1197762682 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562f4ea67c80 msgr2=0x562f4ea68060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:13:30.566 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.587+0000 7f82c93ab740 1 --2- 172.21.15.107:0/1197762682 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562f4ea67c80 0x562f4ea68060 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f82b8037040 tx=0x7f82b8009800 comp rx=0 tx=0).stop 2024-03-08T21:13:30.566 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.587+0000 7f82b7e00640 1 --2- 172.21.15.107:0/1299186103 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f82a40a2a10 0x7f82a40a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:13:30.566 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.587+0000 7f82c93ab740 1 -- 172.21.15.107:0/1197762682 shutdown_connections 2024-03-08T21:13:30.566 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.587+0000 7f82c93ab740 1 --2- 172.21.15.107:0/1197762682 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562f4ea71370 0x562f4ea73760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:13:30.566 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.587+0000 7f82c93ab740 1 --2- 172.21.15.107:0/1197762682 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562f4ea685a0 0x562f4ea70e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:13:30.566 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.587+0000 7f82c93ab740 1 --2- 172.21.15.107:0/1197762682 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562f4ea67c80 0x562f4ea68060 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:13:30.567 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.587+0000 7f82c93ab740 1 -- 172.21.15.107:0/1197762682 >> 172.21.15.107:0/1197762682 conn(0x562f4ea625a0 msgr2=0x562f4ea62980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:13:30.567 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.587+0000 7f82b7e00640 1 -- 172.21.15.107:0/1299186103 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f82ac0cf2d0 con 0x7f82c0103ea0 2024-03-08T21:13:30.567 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.587+0000 7f82c93ab740 1 -- 172.21.15.107:0/1197762682 shutdown_connections 2024-03-08T21:13:30.567 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.587+0000 7f82bfe00640 1 --2- 172.21.15.107:0/1299186103 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f82a40a2a10 0x7f82a40a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:13:30.567 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.587+0000 7f82c93ab740 1 -- 172.21.15.107:0/1197762682 wait complete. 2024-03-08T21:13:30.567 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.588+0000 7f82c93ab740 1 -- start start 2024-03-08T21:13:30.567 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.588+0000 7f82c93ab740 1 Processor -- start 2024-03-08T21:13:30.567 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.588+0000 7f82bfe00640 1 --2- 172.21.15.107:0/1299186103 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f82a40a2a10 0x7f82a40a4ed0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f82b8009600 tx=0x7f82b802c0f0 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:13:30.570 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.589+0000 7f82c93ab740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562f4ea67c80 0x562f4ea70c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:13:30.570 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.589+0000 7f82c93ab740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562f4ea685a0 0x562f4ea6e030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:13:30.570 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.589+0000 7f82c93ab740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562f4ea71370 0x562f4ea6e570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:13:30.570 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.589+0000 7f82c93ab740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f4ea73f40 con 0x562f4ea685a0 2024-03-08T21:13:30.570 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.589+0000 7f82c93ab740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f4e9879f0 con 0x562f4ea67c80 2024-03-08T21:13:30.570 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.589+0000 7f82c93ab740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f4ea740c0 con 0x562f4ea71370 2024-03-08T21:13:30.570 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.589+0000 7f82c93ab740 10 client.0 _ll_register_callbacks cb 0x562f4e6dfa00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T21:13:30.570 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.589+0000 7f82bea00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562f4ea685a0 0x562f4ea6e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:13:30.570 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.589+0000 7f82bea00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562f4ea685a0 0x562f4ea6e030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:37238/0 (socket says 172.21.15.107:37238) 2024-03-08T21:13:30.570 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.589+0000 7f82bea00640 1 -- 172.21.15.107:0/1529363118 learned_addr learned my addr 172.21.15.107:0/1529363118 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:13:30.571 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.589+0000 7f82bf400640 1 --2- 172.21.15.107:0/1529363118 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562f4ea67c80 0x562f4ea70c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:13:30.571 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.589+0000 7f82bfe00640 1 --2- 172.21.15.107:0/1529363118 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562f4ea71370 0x562f4ea6e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:13:30.571 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.590+0000 7f82bea00640 1 -- 172.21.15.107:0/1529363118 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562f4ea71370 msgr2=0x562f4ea6e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:13:30.571 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.590+0000 7f82bea00640 1 --2- 172.21.15.107:0/1529363118 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562f4ea71370 0x562f4ea6e570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:13:30.571 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.590+0000 7f82bea00640 1 -- 172.21.15.107:0/1529363118 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562f4ea67c80 msgr2=0x562f4ea70c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:13:30.571 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.590+0000 7f82bea00640 1 --2- 172.21.15.107:0/1529363118 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562f4ea67c80 0x562f4ea70c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:13:30.571 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.590+0000 7f82bea00640 1 -- 172.21.15.107:0/1529363118 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562f4e6e0210 con 0x562f4ea685a0 2024-03-08T21:13:30.571 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.590+0000 7f82bea00640 1 --2- 172.21.15.107:0/1529363118 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562f4ea685a0 0x562f4ea6e030 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f82ac0ccac0 tx=0x7f82ac09b550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ac7066f75d544a0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:13:30.571 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.590+0000 7f82a9600640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-03-08T21:13:30.571 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.590+0000 7f82a9600640 1 -- 172.21.15.107:0/1529363118 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82ac084e60 con 0x562f4ea685a0 2024-03-08T21:13:30.571 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.590+0000 7f82a9600640 1 -- 172.21.15.107:0/1529363118 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(109..109 src has 1..109) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f82ac086070 con 0x562f4ea685a0 2024-03-08T21:13:30.571 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.590+0000 7f82c93ab740 1 -- 172.21.15.107:0/1529363118 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f82b8045070 con 0x562f4ea685a0 2024-03-08T21:13:30.571 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.591+0000 7f82a9600640 1 -- 172.21.15.107:0/1529363118 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f82ac0ccea0 con 0x562f4ea685a0 2024-03-08T21:13:30.572 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.591+0000 7f82c93ab740 10 client.16539 fetch_fsmap learned FSMap version 41 2024-03-08T21:13:30.572 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.591+0000 7f82c93ab740 1 -- 172.21.15.107:0/1529363118 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x562f4e7ac620 con 0x562f4ea685a0 2024-03-08T21:13:30.572 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.591+0000 7f82a9600640 1 -- 172.21.15.107:0/1529363118 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f82b8045070 con 0x562f4ea685a0 2024-03-08T21:13:30.572 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.591+0000 7f82a9600640 1 -- 172.21.15.107:0/1529363118 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 41) v1 ==== 7307+0+0 (secure 0 0 0) 0x7f82ac09ad70 con 0x562f4ea685a0 2024-03-08T21:13:30.572 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.591+0000 7f82c93ab740 10 client.16539 fetch_fsmap finished waiting for FSMap version 41 2024-03-08T21:13:30.572 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.591+0000 7f82c93ab740 10 client.16539 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-08T21:13:30.572 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.591+0000 7f82c93ab740 20 client.16539 populate_metadata read hostname 'smithi107' 2024-03-08T21:13:30.572 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.591+0000 7f82c93ab740 1 --2- 172.21.15.107:0/1529363118 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x562f4ea811b0 0x562f4ea835a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:13:30.572 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.591+0000 7f82c93ab740 4 client.16539 mds_command: new command op to 34475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-08T21:13:30.572 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.591+0000 7f82c93ab740 1 -- 172.21.15.107:0/1529363118 --> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562f4e9d8910 con 0x562f4ea811b0 2024-03-08T21:13:30.572 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.592+0000 7f82bf400640 1 --2- 172.21.15.107:0/1529363118 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x562f4ea811b0 0x562f4ea835a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:13:30.573 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.593+0000 7f82bf400640 1 --2- 172.21.15.107:0/1529363118 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x562f4ea811b0 0x562f4ea835a0 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6414784067753166 server_cookie=1c8aa55bb2b23f86 in_seq=0 out_seq=0 2024-03-08T21:13:30.574 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.593+0000 7f82a9600640 10 client.16539 ms_handle_connect on v2:172.21.15.149:6838/55290596 2024-03-08T21:13:30.574 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.594+0000 7f82a9600640 1 -- 172.21.15.107:0/1529363118 <== mds.0 v2:172.21.15.149:6838/55290596 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x562f4e9d8910 con 0x562f4ea811b0 2024-03-08T21:13:30.574 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.594+0000 7f82a9600640 10 client.16539 handle_command_reply: tid=0 2024-03-08T21:13:30.574 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.594+0000 7f82c93ab740 1 client.16539 shutdown 2024-03-08T21:13:30.574 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.595+0000 7f82c93ab740 1 -- 172.21.15.107:0/1529363118 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562f4ea685a0 msgr2=0x562f4ea6e030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:13:30.574 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.595+0000 7f82c93ab740 1 --2- 172.21.15.107:0/1529363118 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562f4ea685a0 0x562f4ea6e030 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f82ac0ccac0 tx=0x7f82ac09b550 comp rx=0 tx=0).stop 2024-03-08T21:13:30.574 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.595+0000 7f82c93ab740 1 -- 172.21.15.107:0/1529363118 shutdown_connections 2024-03-08T21:13:30.574 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.595+0000 7f82c93ab740 1 --2- 172.21.15.107:0/1529363118 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562f4ea71370 0x562f4ea6e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:13:30.574 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.595+0000 7f82c93ab740 1 --2- 172.21.15.107:0/1529363118 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562f4ea685a0 0x562f4ea6e030 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:13:30.574 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.595+0000 7f82c93ab740 1 --2- 172.21.15.107:0/1529363118 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x562f4ea811b0 0x562f4ea835a0 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:13:30.574 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.595+0000 7f82c93ab740 1 --2- 172.21.15.107:0/1529363118 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562f4ea67c80 0x562f4ea70c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:13:30.574 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.595+0000 7f82c93ab740 1 -- 172.21.15.107:0/1529363118 >> 172.21.15.107:0/1529363118 conn(0x562f4ea625a0 msgr2=0x562f4ea642f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:13:30.575 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.595+0000 7f82a9600640 0 client.16539 ms_handle_reset on v2:172.21.15.149:6838/55290596 2024-03-08T21:13:30.575 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.595+0000 7f82c93ab740 1 -- 172.21.15.107:0/1529363118 shutdown_connections 2024-03-08T21:13:30.575 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.595+0000 7f82c93ab740 1 -- 172.21.15.107:0/1529363118 wait complete. 2024-03-08T21:13:30.575 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.595+0000 7f82c93ab740 20 client.16539 trim_cache size 0 max 16384 2024-03-08T21:13:30.594 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.616+0000 7f82c93ab740 1 Processor -- start 2024-03-08T21:13:30.594 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.616+0000 7f82c93ab740 1 -- start start 2024-03-08T21:13:30.594 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.616+0000 7f82c93ab740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562f4ea67c80 0x562f4ea640d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:13:30.594 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.616+0000 7f82c93ab740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562f4ea685a0 0x562f4ea6b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:13:30.594 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.616+0000 7f82c93ab740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562f4ea71370 0x562f4ea6b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:13:30.594 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.616+0000 7f82c93ab740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82c01b05d0 con 0x562f4ea71370 2024-03-08T21:13:30.594 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.616+0000 7f82c93ab740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82c01ae5f0 con 0x562f4ea685a0 2024-03-08T21:13:30.594 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.616+0000 7f82c93ab740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82c01aff70 con 0x562f4ea67c80 2024-03-08T21:13:30.596 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.617+0000 7f82bfe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562f4ea71370 0x562f4ea6b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:13:30.596 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.617+0000 7f82bfe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562f4ea71370 0x562f4ea6b7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:37246/0 (socket says 172.21.15.107:37246) 2024-03-08T21:13:30.596 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.617+0000 7f82bfe00640 1 -- 172.21.15.107:0/4189989950 learned_addr learned my addr 172.21.15.107:0/4189989950 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:13:30.596 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.617+0000 7f82bf400640 1 --2- 172.21.15.107:0/4189989950 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562f4ea67c80 0x562f4ea640d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:13:30.596 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.617+0000 7f82bea00640 1 --2- 172.21.15.107:0/4189989950 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562f4ea685a0 0x562f4ea6b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:13:30.596 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.617+0000 7f82bfe00640 1 -- 172.21.15.107:0/4189989950 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562f4ea67c80 msgr2=0x562f4ea640d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:13:30.596 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.617+0000 7f82bfe00640 1 --2- 172.21.15.107:0/4189989950 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562f4ea67c80 0x562f4ea640d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:13:30.597 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.617+0000 7f82bfe00640 1 -- 172.21.15.107:0/4189989950 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562f4ea685a0 msgr2=0x562f4ea6b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:13:30.597 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.617+0000 7f82bfe00640 1 --2- 172.21.15.107:0/4189989950 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562f4ea685a0 0x562f4ea6b2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:13:30.597 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.617+0000 7f82bfe00640 1 -- 172.21.15.107:0/4189989950 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562f4ea67160 con 0x562f4ea71370 2024-03-08T21:13:30.597 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.617+0000 7f82bea00640 1 --2- 172.21.15.107:0/4189989950 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562f4ea685a0 0x562f4ea6b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T21:13:30.597 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.617+0000 7f82bfe00640 1 --2- 172.21.15.107:0/4189989950 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562f4ea71370 0x562f4ea6b7f0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f82b80039e0 tx=0x7f82b80323d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50066b53eac85af7 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:13:30.597 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.618+0000 7f82abe00640 1 -- 172.21.15.107:0/4189989950 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82b8040450 con 0x562f4ea71370 2024-03-08T21:13:30.597 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.618+0000 7f82abe00640 1 -- 172.21.15.107:0/4189989950 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f82b8045040 con 0x562f4ea71370 2024-03-08T21:13:30.597 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.618+0000 7f82abe00640 1 -- 172.21.15.107:0/4189989950 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82b8034730 con 0x562f4ea71370 2024-03-08T21:13:30.601 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.619+0000 7f82c93ab740 1 -- 172.21.15.107:0/4189989950 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562f4ea71370 msgr2=0x562f4ea6b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:13:30.601 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.619+0000 7f82c93ab740 1 --2- 172.21.15.107:0/4189989950 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562f4ea71370 0x562f4ea6b7f0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f82b80039e0 tx=0x7f82b80323d0 comp rx=0 tx=0).stop 2024-03-08T21:13:30.601 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.619+0000 7f82c93ab740 1 -- 172.21.15.107:0/4189989950 shutdown_connections 2024-03-08T21:13:30.601 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.619+0000 7f82c93ab740 1 --2- 172.21.15.107:0/4189989950 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562f4ea71370 0x562f4ea6b7f0 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:13:30.601 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.619+0000 7f82c93ab740 1 --2- 172.21.15.107:0/4189989950 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562f4ea685a0 0x562f4ea6b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:13:30.602 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.619+0000 7f82c93ab740 1 --2- 172.21.15.107:0/4189989950 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562f4ea67c80 0x562f4ea640d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:13:30.602 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.619+0000 7f82c93ab740 1 -- 172.21.15.107:0/4189989950 >> 172.21.15.107:0/4189989950 conn(0x562f4ea625a0 msgr2=0x562f4ea639c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:13:30.602 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.619+0000 7f82c93ab740 1 -- 172.21.15.107:0/4189989950 shutdown_connections 2024-03-08T21:13:30.602 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.619+0000 7f82c93ab740 1 -- 172.21.15.107:0/4189989950 wait complete. 2024-03-08T21:13:30.602 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.620+0000 7f82c93ab740 1 -- start start 2024-03-08T21:13:30.602 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.620+0000 7f82c93ab740 1 Processor -- start 2024-03-08T21:13:30.602 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.620+0000 7f82c93ab740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562f4ea67c80 0x562f4ea76ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:13:30.602 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.620+0000 7f82c93ab740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562f4ea685a0 0x562f4ea771e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:13:30.602 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.620+0000 7f82c93ab740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562f4ea71370 0x562f4ea77720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:13:30.602 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.620+0000 7f82c93ab740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82c01af2b0 con 0x562f4ea67c80 2024-03-08T21:13:30.602 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.620+0000 7f82c93ab740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82c01aec50 con 0x562f4ea71370 2024-03-08T21:13:30.602 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.620+0000 7f82c93ab740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82c01af910 con 0x562f4ea685a0 2024-03-08T21:13:30.603 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.620+0000 7f82c93ab740 10 client.0 _ll_register_callbacks cb 0x7f82b8037070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T21:13:30.603 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.621+0000 7f82bf400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562f4ea67c80 0x562f4ea76ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:13:30.603 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.621+0000 7f82bf400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562f4ea67c80 0x562f4ea76ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:37252/0 (socket says 172.21.15.107:37252) 2024-03-08T21:13:30.603 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.621+0000 7f82bf400640 1 -- 172.21.15.107:0/1258024771 learned_addr learned my addr 172.21.15.107:0/1258024771 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:13:30.603 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.621+0000 7f82bfe00640 1 --2- 172.21.15.107:0/1258024771 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562f4ea71370 0x562f4ea77720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:13:30.603 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.621+0000 7f82bf400640 1 -- 172.21.15.107:0/1258024771 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562f4ea685a0 msgr2=0x562f4ea771e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:13:30.603 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.621+0000 7f82bf400640 1 --2- 172.21.15.107:0/1258024771 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562f4ea685a0 0x562f4ea771e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:13:30.603 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.621+0000 7f82bf400640 1 -- 172.21.15.107:0/1258024771 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562f4ea71370 msgr2=0x562f4ea77720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:13:30.603 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.621+0000 7f82bf400640 1 --2- 172.21.15.107:0/1258024771 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562f4ea71370 0x562f4ea77720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:13:30.603 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.621+0000 7f82bf400640 1 -- 172.21.15.107:0/1258024771 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562f4ead75e0 con 0x562f4ea67c80 2024-03-08T21:13:30.603 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.621+0000 7f82bfe00640 1 --2- 172.21.15.107:0/1258024771 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562f4ea71370 0x562f4ea77720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T21:13:30.603 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.621+0000 7f82bf400640 1 --2- 172.21.15.107:0/1258024771 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562f4ea67c80 0x562f4ea76ca0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f82b0004700 tx=0x7f82b0012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bdf6b45427aa55e server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:13:30.604 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.621+0000 7f82a9600640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-03-08T21:13:30.604 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.622+0000 7f82a9600640 1 -- 172.21.15.107:0/1258024771 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82b0004440 con 0x562f4ea67c80 2024-03-08T21:13:30.604 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.622+0000 7f82a9600640 1 -- 172.21.15.107:0/1258024771 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(109..109 src has 1..109) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f82b001d310 con 0x562f4ea67c80 2024-03-08T21:13:30.604 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.622+0000 7f82c93ab740 1 -- 172.21.15.107:0/1258024771 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f82b8045040 con 0x562f4ea67c80 2024-03-08T21:13:30.604 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.622+0000 7f82a9600640 1 -- 172.21.15.107:0/1258024771 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f82b002d050 con 0x562f4ea67c80 2024-03-08T21:13:30.604 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.622+0000 7f82c93ab740 10 client.16545 fetch_fsmap learned FSMap version 41 2024-03-08T21:13:30.604 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.622+0000 7f82c93ab740 1 -- 172.21.15.107:0/1258024771 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x562f4ead8850 con 0x562f4ea67c80 2024-03-08T21:13:30.604 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.622+0000 7f82a9600640 1 -- 172.21.15.107:0/1258024771 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f82b8045040 con 0x562f4ea67c80 2024-03-08T21:13:30.604 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.622+0000 7f82a9600640 1 -- 172.21.15.107:0/1258024771 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 41) v1 ==== 7307+0+0 (secure 0 0 0) 0x7f82b0003cb0 con 0x562f4ea67c80 2024-03-08T21:13:30.604 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.622+0000 7f82c93ab740 10 client.16545 fetch_fsmap finished waiting for FSMap version 41 2024-03-08T21:13:30.604 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.622+0000 7f82c93ab740 10 client.16545 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-08T21:13:30.604 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.622+0000 7f82c93ab740 20 client.16545 populate_metadata read hostname 'smithi107' 2024-03-08T21:13:30.604 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.623+0000 7f82c93ab740 1 --2- 172.21.15.107:0/1258024771 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x562f4ead8f60 0x562f4eadb380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:13:30.605 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.623+0000 7f82c93ab740 4 client.16545 mds_command: new command op to 34475 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-08T21:13:30.605 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.623+0000 7f82c93ab740 1 -- 172.21.15.107:0/1258024771 --> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x562f4e9d2c60 con 0x562f4ead8f60 2024-03-08T21:13:30.605 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.623+0000 7f82bea00640 1 --2- 172.21.15.107:0/1258024771 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x562f4ead8f60 0x562f4eadb380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:13:30.605 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.624+0000 7f82bea00640 1 --2- 172.21.15.107:0/1258024771 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x562f4ead8f60 0x562f4eadb380 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=801b243feb6747cd server_cookie=5350c22e68b74900 in_seq=0 out_seq=0 2024-03-08T21:13:30.605 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.624+0000 7f82a9600640 10 client.16545 ms_handle_connect on v2:172.21.15.149:6838/55290596 2024-03-08T21:13:30.612 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.634+0000 7f82a9600640 1 -- 172.21.15.107:0/1258024771 <== mds.0 v2:172.21.15.149:6838/55290596 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x562f4e9d2c60 con 0x562f4ead8f60 2024-03-08T21:13:30.612 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.634+0000 7f82a9600640 10 client.16545 handle_command_reply: tid=0 2024-03-08T21:13:30.612 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.634+0000 7f82c93ab740 1 client.16545 shutdown 2024-03-08T21:13:30.613 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.634+0000 7f82c93ab740 1 -- 172.21.15.107:0/1258024771 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562f4ea67c80 msgr2=0x562f4ea76ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:13:30.613 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.634+0000 7f82c93ab740 1 --2- 172.21.15.107:0/1258024771 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562f4ea67c80 0x562f4ea76ca0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f82b0004700 tx=0x7f82b0012820 comp rx=0 tx=0).stop 2024-03-08T21:13:30.613 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.634+0000 7f82c93ab740 1 -- 172.21.15.107:0/1258024771 shutdown_connections 2024-03-08T21:13:30.613 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.634+0000 7f82c93ab740 1 --2- 172.21.15.107:0/1258024771 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x562f4ead8f60 0x562f4eadb380 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:13:30.613 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.634+0000 7f82c93ab740 1 --2- 172.21.15.107:0/1258024771 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562f4ea71370 0x562f4ea77720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:13:30.613 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.634+0000 7f82a9600640 0 client.16545 ms_handle_reset on v2:172.21.15.149:6838/55290596 2024-03-08T21:13:30.613 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.634+0000 7f82c93ab740 1 --2- 172.21.15.107:0/1258024771 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x562f4ea685a0 0x562f4ea771e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:13:30.613 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.634+0000 7f82c93ab740 1 --2- 172.21.15.107:0/1258024771 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562f4ea67c80 0x562f4ea76ca0 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:13:30.613 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.634+0000 7f82c93ab740 1 -- 172.21.15.107:0/1258024771 >> 172.21.15.107:0/1258024771 conn(0x562f4ea625a0 msgr2=0x562f4ea6ff70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:13:30.613 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.634+0000 7f82c93ab740 1 -- 172.21.15.107:0/1258024771 shutdown_connections 2024-03-08T21:13:30.613 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.634+0000 7f82c93ab740 1 -- 172.21.15.107:0/1258024771 wait complete. 2024-03-08T21:13:30.613 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.635+0000 7f82c93ab740 20 client.16545 trim_cache size 0 max 16384 2024-03-08T21:13:30.614 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:{ 2024-03-08T21:13:30.614 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "message": "", 2024-03-08T21:13:30.614 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "return_code": 0 2024-03-08T21:13:30.614 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:} 2024-03-08T21:13:30.615 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.635+0000 7f82c6600640 1 -- 172.21.15.107:0/1299186103 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f82a40a2a10 msgr2=0x7f82a40a4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:13:30.615 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.635+0000 7f82c6600640 1 --2- 172.21.15.107:0/1299186103 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f82a40a2a10 0x7f82a40a4ed0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f82b8009600 tx=0x7f82b802c0f0 comp rx=0 tx=0).stop 2024-03-08T21:13:30.615 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.635+0000 7f82c6600640 1 -- 172.21.15.107:0/1299186103 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f82c0103ea0 msgr2=0x7f82c019fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:13:30.615 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.635+0000 7f82c6600640 1 --2- 172.21.15.107:0/1299186103 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f82c0103ea0 0x7f82c019fd20 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f82ac00e9f0 tx=0x7f82ac00eeb0 comp rx=0 tx=0).stop 2024-03-08T21:13:30.615 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.635+0000 7f82c6600640 1 -- 172.21.15.107:0/1299186103 shutdown_connections 2024-03-08T21:13:30.615 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.635+0000 7f82c6600640 1 --2- 172.21.15.107:0/1299186103 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f82a40a2a10 0x7f82a40a4ed0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:13:30.615 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.635+0000 7f82c6600640 1 --2- 172.21.15.107:0/1299186103 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f82c01095a0 0x7f82c0199990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:13:30.615 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.635+0000 7f82c6600640 1 --2- 172.21.15.107:0/1299186103 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f82c0103ea0 0x7f82c019fd20 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:13:30.615 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.635+0000 7f82c6600640 1 --2- 172.21.15.107:0/1299186103 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f82c01034e0 0x7f82c019f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:13:30.615 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.636+0000 7f82c6600640 1 -- 172.21.15.107:0/1299186103 >> 172.21.15.107:0/1299186103 conn(0x7f82c00fefb0 msgr2=0x7f82c01006f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:13:30.615 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.636+0000 7f82c6600640 1 -- 172.21.15.107:0/1299186103 shutdown_connections 2024-03-08T21:13:30.615 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:30.636+0000 7f82c6600640 1 -- 172.21.15.107:0/1299186103 wait complete. 2024-03-08T21:13:30.711 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:13:30 smithi107 ceph-mon[31381]: pgmap v411: 97 pgs: 97 active+clean; 8.8 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 397 B/s wr, 15 op/s 2024-03-08T21:13:30.872 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:13:30 smithi204 ceph-mon[34935]: pgmap v411: 97 pgs: 97 active+clean; 8.8 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 397 B/s wr, 15 op/s 2024-03-08T21:13:31.009 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:13:30 smithi149 ceph-mon[35042]: pgmap v411: 97 pgs: 97 active+clean; 8.8 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 397 B/s wr, 15 op/s 2024-03-08T21:13:32.872 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:13:32 smithi204 ceph-mon[34935]: pgmap v412: 97 pgs: 97 active+clean; 8.8 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-08T21:13:32.961 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:13:32 smithi107 ceph-mon[31381]: pgmap v412: 97 pgs: 97 active+clean; 8.8 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-08T21:13:33.009 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:13:32 smithi149 ceph-mon[35042]: pgmap v412: 97 pgs: 97 active+clean; 8.8 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-08T21:13:34.872 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:13:34 smithi204 ceph-mon[34935]: pgmap v413: 97 pgs: 97 active+clean; 8.8 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-08T21:13:34.961 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:13:34 smithi107 ceph-mon[31381]: pgmap v413: 97 pgs: 97 active+clean; 8.8 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-08T21:13:35.009 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:13:34 smithi149 ceph-mon[35042]: pgmap v413: 97 pgs: 97 active+clean; 8.8 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-08T21:13:36.872 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:13:36 smithi204 ceph-mon[34935]: pgmap v414: 97 pgs: 97 active+clean; 8.8 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-08T21:13:36.961 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:13:36 smithi107 ceph-mon[31381]: pgmap v414: 97 pgs: 97 active+clean; 8.8 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-08T21:13:37.009 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:13:36 smithi149 ceph-mon[35042]: pgmap v414: 97 pgs: 97 active+clean; 8.8 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-08T21:13:38.872 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:13:38 smithi204 ceph-mon[34935]: pgmap v415: 97 pgs: 97 active+clean; 8.8 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:13:38.961 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:13:38 smithi107 ceph-mon[31381]: pgmap v415: 97 pgs: 97 active+clean; 8.8 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:13:39.009 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:13:38 smithi149 ceph-mon[35042]: pgmap v415: 97 pgs: 97 active+clean; 8.8 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:13:40.872 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:13:40 smithi204 ceph-mon[34935]: pgmap v416: 97 pgs: 97 active+clean; 8.8 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-08T21:13:40.961 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:13:40 smithi107 ceph-mon[31381]: pgmap v416: 97 pgs: 97 active+clean; 8.8 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-08T21:13:41.009 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:13:40 smithi149 ceph-mon[35042]: pgmap v416: 97 pgs: 97 active+clean; 8.8 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-08T21:13:42.872 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:13:42 smithi204 ceph-mon[34935]: pgmap v417: 97 pgs: 97 active+clean; 8.8 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:13:42.961 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:13:42 smithi107 ceph-mon[31381]: pgmap v417: 97 pgs: 97 active+clean; 8.8 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:13:43.009 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:13:42 smithi149 ceph-mon[35042]: pgmap v417: 97 pgs: 97 active+clean; 8.8 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:13:43.772 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.794+0000 7f959ec00640 1 -- 172.21.15.107:0/2543460170 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f95980ffeb0 msgr2=0x7f9598100290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:13:43.772 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.794+0000 7f959ec00640 1 --2- 172.21.15.107:0/2543460170 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f95980ffeb0 0x7f9598100290 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f958c009900 tx=0x7f958c02bc50 comp rx=0 tx=0).stop 2024-03-08T21:13:43.772 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.794+0000 7f959ec00640 1 -- 172.21.15.107:0/2543460170 shutdown_connections 2024-03-08T21:13:43.772 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.794+0000 7f959ec00640 1 --2- 172.21.15.107:0/2543460170 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f959810da60 0x7f959810fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:13:43.772 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.794+0000 7f959ec00640 1 --2- 172.21.15.107:0/2543460170 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9598100860 0x7f959810d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:13:43.772 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.794+0000 7f959ec00640 1 --2- 172.21.15.107:0/2543460170 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f95980ffeb0 0x7f9598100290 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:13:43.772 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.794+0000 7f959ec00640 1 -- 172.21.15.107:0/2543460170 >> 172.21.15.107:0/2543460170 conn(0x7f95980fbb00 msgr2=0x7f95980fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:13:43.772 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.794+0000 7f959ec00640 1 -- 172.21.15.107:0/2543460170 shutdown_connections 2024-03-08T21:13:43.772 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.794+0000 7f959ec00640 1 -- 172.21.15.107:0/2543460170 wait complete. 2024-03-08T21:13:43.773 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.795+0000 7f959ec00640 1 Processor -- start 2024-03-08T21:13:43.773 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.795+0000 7f959ec00640 1 -- start start 2024-03-08T21:13:43.773 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.796+0000 7f959ec00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f95980ffeb0 0x7f959819f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:13:43.773 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.796+0000 7f959ec00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9598100860 0x7f959819fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:13:43.773 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.796+0000 7f959ec00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f959810da60 0x7f95981999f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:13:43.774 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.796+0000 7f959ec00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9598113650 con 0x7f959810da60 2024-03-08T21:13:43.774 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.796+0000 7f959ec00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95981134d0 con 0x7f95980ffeb0 2024-03-08T21:13:43.774 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.796+0000 7f959ec00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95981137d0 con 0x7f9598100860 2024-03-08T21:13:43.775 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.796+0000 7f959e200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f959810da60 0x7f95981999f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:13:43.775 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.796+0000 7f959e200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f959810da60 0x7f95981999f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:42076/0 (socket says 172.21.15.107:42076) 2024-03-08T21:13:43.775 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.796+0000 7f959e200640 1 -- 172.21.15.107:0/812493306 learned_addr learned my addr 172.21.15.107:0/812493306 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:13:43.775 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.796+0000 7f959d800640 1 --2- 172.21.15.107:0/812493306 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f95980ffeb0 0x7f959819f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:13:43.776 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.796+0000 7f959ce00640 1 --2- 172.21.15.107:0/812493306 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9598100860 0x7f959819fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:13:43.776 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.797+0000 7f959e200640 1 -- 172.21.15.107:0/812493306 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9598100860 msgr2=0x7f959819fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:13:43.776 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.797+0000 7f959e200640 1 --2- 172.21.15.107:0/812493306 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9598100860 0x7f959819fd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:13:43.776 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.797+0000 7f959e200640 1 -- 172.21.15.107:0/812493306 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f95980ffeb0 msgr2=0x7f959819f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:13:43.776 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.797+0000 7f959e200640 1 --2- 172.21.15.107:0/812493306 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f95980ffeb0 0x7f959819f7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:13:43.776 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.797+0000 7f959e200640 1 -- 172.21.15.107:0/812493306 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f959819a170 con 0x7f959810da60 2024-03-08T21:13:43.776 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.797+0000 7f959ce00640 1 --2- 172.21.15.107:0/812493306 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9598100860 0x7f959819fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T21:13:43.776 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.797+0000 7f959d800640 1 --2- 172.21.15.107:0/812493306 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f95980ffeb0 0x7f959819f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T21:13:43.776 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.797+0000 7f959e200640 1 --2- 172.21.15.107:0/812493306 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f959810da60 0x7f95981999f0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f959000b9e0 tx=0x7f959000bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:13:43.776 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.797+0000 7f9596000640 1 -- 172.21.15.107:0/812493306 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9590019070 con 0x7f959810da60 2024-03-08T21:13:43.776 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.797+0000 7f959ec00640 1 -- 172.21.15.107:0/812493306 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f959819a460 con 0x7f959810da60 2024-03-08T21:13:43.776 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.797+0000 7f9596000640 1 -- 172.21.15.107:0/812493306 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f9590014440 con 0x7f959810da60 2024-03-08T21:13:43.776 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.797+0000 7f9596000640 1 -- 172.21.15.107:0/812493306 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95900125a0 con 0x7f959810da60 2024-03-08T21:13:43.776 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.798+0000 7f959ec00640 1 -- 172.21.15.107:0/812493306 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95981a66b0 con 0x7f959810da60 2024-03-08T21:13:43.778 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.801+0000 7f9596000640 1 -- 172.21.15.107:0/812493306 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) v1 ==== 137422+0+0 (secure 0 0 0) 0x7f959001c070 con 0x7f959810da60 2024-03-08T21:13:43.779 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.801+0000 7f95a2fd2740 1 Processor -- start 2024-03-08T21:13:43.779 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.802+0000 7f95a2fd2740 1 -- start start 2024-03-08T21:13:43.779 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.802+0000 7f95a2fd2740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559836413e80 0x559836414260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:13:43.779 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.802+0000 7f95a2fd2740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5598364147a0 0x55983641d030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:13:43.780 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.802+0000 7f95a2fd2740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55983641d570 0x55983641f960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:13:43.780 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.802+0000 7f95a2fd2740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598363773d0 con 0x559836413e80 2024-03-08T21:13:43.780 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.802+0000 7f95a2fd2740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598363206b0 con 0x55983641d570 2024-03-08T21:13:43.780 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.802+0000 7f95a2fd2740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x559836373470 con 0x5598364147a0 2024-03-08T21:13:43.781 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.802+0000 7f959ce00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559836413e80 0x559836414260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:13:43.781 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.802+0000 7f959ce00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559836413e80 0x559836414260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:42090/0 (socket says 172.21.15.107:42090) 2024-03-08T21:13:43.781 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.802+0000 7f959ce00640 1 -- 172.21.15.107:0/2673446765 learned_addr learned my addr 172.21.15.107:0/2673446765 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:13:43.781 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.803+0000 7f959d800640 1 --2- 172.21.15.107:0/2673446765 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5598364147a0 0x55983641d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:13:43.781 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.803+0000 7f9596000640 1 --2- 172.21.15.107:0/812493306 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f95800a2960 0x7f95800a4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:13:43.781 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.803+0000 7f959ce00640 1 -- 172.21.15.107:0/2673446765 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5598364147a0 msgr2=0x55983641d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:13:43.781 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.803+0000 7f959ce00640 1 --2- 172.21.15.107:0/2673446765 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5598364147a0 0x55983641d030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:13:43.782 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.803+0000 7f959ce00640 1 -- 172.21.15.107:0/2673446765 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55983641d570 msgr2=0x55983641f960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:13:43.782 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.803+0000 7f959ce00640 1 --2- 172.21.15.107:0/2673446765 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55983641d570 0x55983641f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:13:43.782 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.803+0000 7f959ce00640 1 -- 172.21.15.107:0/2673446765 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559836346b50 con 0x559836413e80 2024-03-08T21:13:43.782 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.803+0000 7f959e200640 1 --2- 172.21.15.107:0/812493306 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f95800a2960 0x7f95800a4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:13:43.782 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.803+0000 7f9596000640 1 -- 172.21.15.107:0/812493306 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f95900ce370 con 0x7f959810da60 2024-03-08T21:13:43.782 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.803+0000 7f959ce00640 1 --2- 172.21.15.107:0/2673446765 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559836413e80 0x559836414260 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f958400cca0 tx=0x7f9584010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c477b9c80ed1bd server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:13:43.782 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.804+0000 7f958aa00640 1 -- 172.21.15.107:0/2673446765 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9584016070 con 0x559836413e80 2024-03-08T21:13:43.782 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.804+0000 7f958aa00640 1 -- 172.21.15.107:0/2673446765 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f9584004220 con 0x559836413e80 2024-03-08T21:13:43.782 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.804+0000 7f958aa00640 1 -- 172.21.15.107:0/2673446765 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95840127a0 con 0x559836413e80 2024-03-08T21:13:43.785 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.804+0000 7f95a2fd2740 1 -- 172.21.15.107:0/2673446765 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559836413e80 msgr2=0x559836414260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:13:43.785 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.804+0000 7f959e200640 1 --2- 172.21.15.107:0/812493306 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f95800a2960 0x7f95800a4e20 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f9590096ed0 tx=0x7f959009d040 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:13:43.785 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.804+0000 7f95a2fd2740 1 --2- 172.21.15.107:0/2673446765 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559836413e80 0x559836414260 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f958400cca0 tx=0x7f9584010740 comp rx=0 tx=0).stop 2024-03-08T21:13:43.785 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.805+0000 7f95a2fd2740 1 -- 172.21.15.107:0/2673446765 shutdown_connections 2024-03-08T21:13:43.785 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.805+0000 7f95a2fd2740 1 --2- 172.21.15.107:0/2673446765 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55983641d570 0x55983641f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:13:43.785 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.805+0000 7f95a2fd2740 1 --2- 172.21.15.107:0/2673446765 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5598364147a0 0x55983641d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:13:43.785 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.805+0000 7f95a2fd2740 1 --2- 172.21.15.107:0/2673446765 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559836413e80 0x559836414260 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:13:43.785 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.805+0000 7f95a2fd2740 1 -- 172.21.15.107:0/2673446765 >> 172.21.15.107:0/2673446765 conn(0x55983640e7a0 msgr2=0x55983640eb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:13:43.785 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.805+0000 7f95a2fd2740 1 -- 172.21.15.107:0/2673446765 shutdown_connections 2024-03-08T21:13:43.785 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.805+0000 7f95a2fd2740 1 -- 172.21.15.107:0/2673446765 wait complete. 2024-03-08T21:13:43.786 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.805+0000 7f95a2fd2740 1 -- start start 2024-03-08T21:13:43.786 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.805+0000 7f95a2fd2740 1 Processor -- start 2024-03-08T21:13:43.786 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.806+0000 7f95a2fd2740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559836413e80 0x55983641cf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:13:43.786 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.806+0000 7f95a2fd2740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5598364147a0 0x55983641a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:13:43.786 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.806+0000 7f95a2fd2740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55983641d570 0x55983641a840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:13:43.786 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.806+0000 7f95a2fd2740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598364201a0 con 0x559836413e80 2024-03-08T21:13:43.786 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.806+0000 7f95a2fd2740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598363339f0 con 0x5598364147a0 2024-03-08T21:13:43.786 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.806+0000 7f95a2fd2740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x559836420320 con 0x55983641d570 2024-03-08T21:13:43.786 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.806+0000 7f95a2fd2740 10 client.0 _ll_register_callbacks cb 0x55983608ba00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T21:13:43.786 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.806+0000 7f959ce00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559836413e80 0x55983641cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:13:43.786 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.806+0000 7f959ce00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559836413e80 0x55983641cf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:42106/0 (socket says 172.21.15.107:42106) 2024-03-08T21:13:43.786 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.806+0000 7f959ce00640 1 -- 172.21.15.107:0/2933553325 learned_addr learned my addr 172.21.15.107:0/2933553325 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:13:43.786 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.807+0000 7f959d800640 1 --2- 172.21.15.107:0/2933553325 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5598364147a0 0x55983641a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:13:43.787 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.807+0000 7f959ce00640 1 -- 172.21.15.107:0/2933553325 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55983641d570 msgr2=0x55983641a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:13:43.787 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.807+0000 7f959ce00640 1 --2- 172.21.15.107:0/2933553325 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55983641d570 0x55983641a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:13:43.787 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.807+0000 7f959ce00640 1 -- 172.21.15.107:0/2933553325 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5598364147a0 msgr2=0x55983641a2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:13:43.787 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.807+0000 7f959ce00640 1 --2- 172.21.15.107:0/2933553325 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5598364147a0 0x55983641a2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:13:43.787 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.807+0000 7f959ce00640 1 -- 172.21.15.107:0/2933553325 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55983608c210 con 0x559836413e80 2024-03-08T21:13:43.787 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.807+0000 7f959d800640 1 --2- 172.21.15.107:0/2933553325 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5598364147a0 0x55983641a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T21:13:43.787 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.807+0000 7f959ce00640 1 --2- 172.21.15.107:0/2933553325 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559836413e80 0x55983641cf10 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f958400c450 tx=0x7f9584003de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1861863c73e21da6 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:13:43.787 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.807+0000 7f957f400640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-03-08T21:13:43.787 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.807+0000 7f957f400640 1 -- 172.21.15.107:0/2933553325 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9584016040 con 0x559836413e80 2024-03-08T21:13:43.787 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.807+0000 7f957f400640 1 -- 172.21.15.107:0/2933553325 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(109..109 src has 1..109) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f9584005260 con 0x559836413e80 2024-03-08T21:13:43.787 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.807+0000 7f95a2fd2740 1 -- 172.21.15.107:0/2933553325 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9584004220 con 0x559836413e80 2024-03-08T21:13:43.789 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.808+0000 7f957f400640 1 -- 172.21.15.107:0/2933553325 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9584030050 con 0x559836413e80 2024-03-08T21:13:43.789 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.808+0000 7f95a2fd2740 10 client.16557 fetch_fsmap learned FSMap version 41 2024-03-08T21:13:43.789 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.808+0000 7f95a2fd2740 1 -- 172.21.15.107:0/2933553325 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x559836158620 con 0x559836413e80 2024-03-08T21:13:43.789 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.808+0000 7f957f400640 1 -- 172.21.15.107:0/2933553325 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f9584004220 con 0x559836413e80 2024-03-08T21:13:43.789 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.808+0000 7f957f400640 1 -- 172.21.15.107:0/2933553325 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 41) v1 ==== 7307+0+0 (secure 0 0 0) 0x7f9584012560 con 0x559836413e80 2024-03-08T21:13:43.789 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.808+0000 7f95a2fd2740 10 client.16557 fetch_fsmap finished waiting for FSMap version 41 2024-03-08T21:13:43.789 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.808+0000 7f95a2fd2740 10 client.16557 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-08T21:13:43.789 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.808+0000 7f95a2fd2740 20 client.16557 populate_metadata read hostname 'smithi107' 2024-03-08T21:13:43.789 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.808+0000 7f95a2fd2740 1 --2- 172.21.15.107:0/2933553325 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55983642d340 0x55983642f730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:13:43.789 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.809+0000 7f95a2fd2740 4 client.16557 mds_command: new command op to 34475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-08T21:13:43.790 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.809+0000 7f95a2fd2740 1 -- 172.21.15.107:0/2933553325 --> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559836384910 con 0x55983642d340 2024-03-08T21:13:43.790 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.809+0000 7f959d800640 1 --2- 172.21.15.107:0/2933553325 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55983642d340 0x55983642f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:13:43.790 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.810+0000 7f959d800640 1 --2- 172.21.15.107:0/2933553325 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55983642d340 0x55983642f730 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2edf2de1c0fe5b0f server_cookie=a18cd0b3c182e149 in_seq=0 out_seq=0 2024-03-08T21:13:43.790 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.810+0000 7f957f400640 10 client.16557 ms_handle_connect on v2:172.21.15.149:6838/55290596 2024-03-08T21:13:43.791 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.812+0000 7f957f400640 1 -- 172.21.15.107:0/2933553325 <== mds.0 v2:172.21.15.149:6838/55290596 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x559836384910 con 0x55983642d340 2024-03-08T21:13:43.791 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.812+0000 7f957f400640 10 client.16557 handle_command_reply: tid=0 2024-03-08T21:13:43.791 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.812+0000 7f95a2fd2740 1 client.16557 shutdown 2024-03-08T21:13:43.791 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.812+0000 7f95a2fd2740 1 -- 172.21.15.107:0/2933553325 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559836413e80 msgr2=0x55983641cf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:13:43.791 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.812+0000 7f95a2fd2740 1 --2- 172.21.15.107:0/2933553325 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559836413e80 0x55983641cf10 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f958400c450 tx=0x7f9584003de0 comp rx=0 tx=0).stop 2024-03-08T21:13:43.791 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.812+0000 7f95a2fd2740 1 -- 172.21.15.107:0/2933553325 shutdown_connections 2024-03-08T21:13:43.791 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.812+0000 7f95a2fd2740 1 --2- 172.21.15.107:0/2933553325 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55983641d570 0x55983641a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:13:43.791 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.812+0000 7f95a2fd2740 1 --2- 172.21.15.107:0/2933553325 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55983642d340 0x55983642f730 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:13:43.792 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.812+0000 7f95a2fd2740 1 --2- 172.21.15.107:0/2933553325 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5598364147a0 0x55983641a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:13:43.792 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.812+0000 7f95a2fd2740 1 --2- 172.21.15.107:0/2933553325 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559836413e80 0x55983641cf10 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:13:43.792 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.812+0000 7f957f400640 0 client.16557 ms_handle_reset on v2:172.21.15.149:6838/55290596 2024-03-08T21:13:43.792 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.812+0000 7f95a2fd2740 1 -- 172.21.15.107:0/2933553325 >> 172.21.15.107:0/2933553325 conn(0x55983640e7a0 msgr2=0x5598364104f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:13:43.792 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.812+0000 7f95a2fd2740 1 -- 172.21.15.107:0/2933553325 shutdown_connections 2024-03-08T21:13:43.792 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.812+0000 7f95a2fd2740 1 -- 172.21.15.107:0/2933553325 wait complete. 2024-03-08T21:13:43.792 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.813+0000 7f95a2fd2740 20 client.16557 trim_cache size 0 max 16384 2024-03-08T21:13:43.806 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.827+0000 7f95a2fd2740 1 Processor -- start 2024-03-08T21:13:43.806 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.828+0000 7f95a2fd2740 1 -- start start 2024-03-08T21:13:43.806 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.828+0000 7f95a2fd2740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559836413e80 0x5598364191e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:13:43.806 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.828+0000 7f95a2fd2740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5598364147a0 0x55983641c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:13:43.806 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.828+0000 7f95a2fd2740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55983641d570 0x55983641cf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:13:43.806 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.828+0000 7f95a2fd2740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95981b05b0 con 0x55983641d570 2024-03-08T21:13:43.806 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.828+0000 7f95a2fd2740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95981ae5d0 con 0x559836413e80 2024-03-08T21:13:43.807 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.828+0000 7f95a2fd2740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95981aff50 con 0x5598364147a0 2024-03-08T21:13:43.807 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.828+0000 7f959ce00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55983641d570 0x55983641cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:13:43.807 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.828+0000 7f959ce00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55983641d570 0x55983641cf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:42120/0 (socket says 172.21.15.107:42120) 2024-03-08T21:13:43.807 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.828+0000 7f959ce00640 1 -- 172.21.15.107:0/2150227025 learned_addr learned my addr 172.21.15.107:0/2150227025 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:13:43.807 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.828+0000 7f959d800640 1 --2- 172.21.15.107:0/2150227025 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5598364147a0 0x55983641c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:13:43.807 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.829+0000 7f959ce00640 1 --2- 172.21.15.107:0/2150227025 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559836413e80 0x5598364191e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:13:43.809 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.829+0000 7f959ce00640 1 -- 172.21.15.107:0/2150227025 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5598364147a0 msgr2=0x55983641c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:13:43.809 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.829+0000 7f959ce00640 1 --2- 172.21.15.107:0/2150227025 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5598364147a0 0x55983641c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:13:43.809 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.829+0000 7f959ce00640 1 -- 172.21.15.107:0/2150227025 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559836413e80 msgr2=0x5598364191e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:13:43.809 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.829+0000 7f959ce00640 1 --2- 172.21.15.107:0/2150227025 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559836413e80 0x5598364191e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:13:43.809 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.829+0000 7f959ce00640 1 -- 172.21.15.107:0/2150227025 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5598364145d0 con 0x55983641d570 2024-03-08T21:13:43.809 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.829+0000 7f959ce00640 1 --2- 172.21.15.107:0/2150227025 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55983641d570 0x55983641cf10 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f9584005ab0 tx=0x7f9584010f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f457542b515a082 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:13:43.809 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.829+0000 7f958a000640 1 -- 172.21.15.107:0/2150227025 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9584005260 con 0x55983641d570 2024-03-08T21:13:43.809 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.830+0000 7f958a000640 1 -- 172.21.15.107:0/2150227025 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f9584023070 con 0x55983641d570 2024-03-08T21:13:43.809 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.830+0000 7f958a000640 1 -- 172.21.15.107:0/2150227025 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9584012480 con 0x55983641d570 2024-03-08T21:13:43.809 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.830+0000 7f95a2fd2740 1 -- 172.21.15.107:0/2150227025 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55983641d570 msgr2=0x55983641cf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:13:43.809 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.830+0000 7f95a2fd2740 1 --2- 172.21.15.107:0/2150227025 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55983641d570 0x55983641cf10 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f9584005ab0 tx=0x7f9584010f70 comp rx=0 tx=0).stop 2024-03-08T21:13:43.810 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.830+0000 7f95a2fd2740 1 -- 172.21.15.107:0/2150227025 shutdown_connections 2024-03-08T21:13:43.810 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.831+0000 7f95a2fd2740 1 --2- 172.21.15.107:0/2150227025 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55983641d570 0x55983641cf10 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:13:43.810 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.831+0000 7f95a2fd2740 1 --2- 172.21.15.107:0/2150227025 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5598364147a0 0x55983641c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:13:43.810 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.831+0000 7f95a2fd2740 1 --2- 172.21.15.107:0/2150227025 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x559836413e80 0x5598364191e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:13:43.810 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.831+0000 7f95a2fd2740 1 -- 172.21.15.107:0/2150227025 >> 172.21.15.107:0/2150227025 conn(0x55983640e7a0 msgr2=0x55983641a070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:13:43.810 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.831+0000 7f95a2fd2740 1 -- 172.21.15.107:0/2150227025 shutdown_connections 2024-03-08T21:13:43.810 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.831+0000 7f95a2fd2740 1 -- 172.21.15.107:0/2150227025 wait complete. 2024-03-08T21:13:43.810 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.831+0000 7f95a2fd2740 1 -- start start 2024-03-08T21:13:43.810 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.831+0000 7f95a2fd2740 1 Processor -- start 2024-03-08T21:13:43.813 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.832+0000 7f95a2fd2740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559836413e80 0x559836422f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:13:43.813 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.832+0000 7f95a2fd2740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5598364147a0 0x559836423450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:13:43.813 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.832+0000 7f95a2fd2740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55983641d570 0x559836423990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:13:43.813 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.832+0000 7f95a2fd2740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95981af290 con 0x559836413e80 2024-03-08T21:13:43.813 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.832+0000 7f95a2fd2740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95981aec30 con 0x55983641d570 2024-03-08T21:13:43.813 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.832+0000 7f95a2fd2740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95981af8f0 con 0x5598364147a0 2024-03-08T21:13:43.813 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.832+0000 7f95a2fd2740 10 client.0 _ll_register_callbacks cb 0x7f958400c480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T21:13:43.813 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.832+0000 7f959ce00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559836413e80 0x559836422f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:13:43.813 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.832+0000 7f959ce00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559836413e80 0x559836422f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:42134/0 (socket says 172.21.15.107:42134) 2024-03-08T21:13:43.813 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.832+0000 7f959ce00640 1 -- 172.21.15.107:0/4137618518 learned_addr learned my addr 172.21.15.107:0/4137618518 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:13:43.813 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.832+0000 7f959ce00640 1 -- 172.21.15.107:0/4137618518 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5598364147a0 msgr2=0x559836423450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:13:43.814 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.832+0000 7f959ce00640 1 --2- 172.21.15.107:0/4137618518 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5598364147a0 0x559836423450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:13:43.814 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.832+0000 7f959ce00640 1 -- 172.21.15.107:0/4137618518 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55983641d570 msgr2=0x559836423990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:13:43.814 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.832+0000 7f959ce00640 1 --2- 172.21.15.107:0/4137618518 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55983641d570 0x559836423990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:13:43.814 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.832+0000 7f959ce00640 1 -- 172.21.15.107:0/4137618518 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55983642faa0 con 0x559836413e80 2024-03-08T21:13:43.814 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.833+0000 7f959ce00640 1 --2- 172.21.15.107:0/4137618518 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559836413e80 0x559836422f10 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f95840058c0 tx=0x7f9584003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=800dd16c162b1c1d server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:13:43.814 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.833+0000 7f957f400640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-03-08T21:13:43.814 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.833+0000 7f957f400640 1 -- 172.21.15.107:0/4137618518 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f958401e030 con 0x559836413e80 2024-03-08T21:13:43.814 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.833+0000 7f957f400640 1 -- 172.21.15.107:0/4137618518 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(109..109 src has 1..109) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f9584004100 con 0x559836413e80 2024-03-08T21:13:43.814 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.833+0000 7f95a2fd2740 1 -- 172.21.15.107:0/4137618518 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9584023070 con 0x559836413e80 2024-03-08T21:13:43.814 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.834+0000 7f957f400640 1 -- 172.21.15.107:0/4137618518 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9584005260 con 0x559836413e80 2024-03-08T21:13:43.814 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.834+0000 7f95a2fd2740 10 client.16563 fetch_fsmap learned FSMap version 41 2024-03-08T21:13:43.814 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.834+0000 7f95a2fd2740 1 -- 172.21.15.107:0/4137618518 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x5598364848b0 con 0x559836413e80 2024-03-08T21:13:43.814 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.834+0000 7f957f400640 1 -- 172.21.15.107:0/4137618518 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f9584023070 con 0x559836413e80 2024-03-08T21:13:43.814 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.834+0000 7f957f400640 1 -- 172.21.15.107:0/4137618518 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 41) v1 ==== 7307+0+0 (secure 0 0 0) 0x7f9584033040 con 0x559836413e80 2024-03-08T21:13:43.815 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.834+0000 7f95a2fd2740 10 client.16563 fetch_fsmap finished waiting for FSMap version 41 2024-03-08T21:13:43.815 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.834+0000 7f95a2fd2740 10 client.16563 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-08T21:13:43.815 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.834+0000 7f95a2fd2740 20 client.16563 populate_metadata read hostname 'smithi107' 2024-03-08T21:13:43.815 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.834+0000 7f95a2fd2740 1 --2- 172.21.15.107:0/4137618518 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x559836484e40 0x559836487260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:13:43.815 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.834+0000 7f95a2fd2740 4 client.16563 mds_command: new command op to 34475 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-08T21:13:43.815 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.834+0000 7f95a2fd2740 1 -- 172.21.15.107:0/4137618518 --> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55983637ec60 con 0x559836484e40 2024-03-08T21:13:43.815 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.835+0000 7f959d800640 1 --2- 172.21.15.107:0/4137618518 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x559836484e40 0x559836487260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:13:43.815 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.836+0000 7f957f400640 10 client.16563 ms_handle_connect on v2:172.21.15.149:6838/55290596 2024-03-08T21:13:43.816 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.836+0000 7f959d800640 1 --2- 172.21.15.107:0/4137618518 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x559836484e40 0x559836487260 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3041e34e2a5d5358 server_cookie=9d6238eec18c014 in_seq=0 out_seq=0 2024-03-08T21:13:43.823 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.845+0000 7f957f400640 1 -- 172.21.15.107:0/4137618518 <== mds.0 v2:172.21.15.149:6838/55290596 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55983637ec60 con 0x559836484e40 2024-03-08T21:13:43.824 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.845+0000 7f957f400640 10 client.16563 handle_command_reply: tid=0 2024-03-08T21:13:43.824 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.845+0000 7f95a2fd2740 1 client.16563 shutdown 2024-03-08T21:13:43.824 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.845+0000 7f95a2fd2740 1 -- 172.21.15.107:0/4137618518 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559836413e80 msgr2=0x559836422f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:13:43.824 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.845+0000 7f95a2fd2740 1 --2- 172.21.15.107:0/4137618518 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559836413e80 0x559836422f10 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f95840058c0 tx=0x7f9584003970 comp rx=0 tx=0).stop 2024-03-08T21:13:43.824 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.846+0000 7f95a2fd2740 1 -- 172.21.15.107:0/4137618518 shutdown_connections 2024-03-08T21:13:43.824 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.846+0000 7f95a2fd2740 1 --2- 172.21.15.107:0/4137618518 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x559836484e40 0x559836487260 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:13:43.824 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.846+0000 7f95a2fd2740 1 --2- 172.21.15.107:0/4137618518 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55983641d570 0x559836423990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:13:43.824 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.846+0000 7f95a2fd2740 1 --2- 172.21.15.107:0/4137618518 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5598364147a0 0x559836423450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:13:43.824 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.846+0000 7f95a2fd2740 1 --2- 172.21.15.107:0/4137618518 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559836413e80 0x559836422f10 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:13:43.824 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.846+0000 7f957f400640 0 client.16563 ms_handle_reset on v2:172.21.15.149:6838/55290596 2024-03-08T21:13:43.824 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.846+0000 7f95a2fd2740 1 -- 172.21.15.107:0/4137618518 >> 172.21.15.107:0/4137618518 conn(0x55983640e7a0 msgr2=0x55983640cf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:13:43.824 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.846+0000 7f95a2fd2740 1 -- 172.21.15.107:0/4137618518 shutdown_connections 2024-03-08T21:13:43.824 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.846+0000 7f95a2fd2740 1 -- 172.21.15.107:0/4137618518 wait complete. 2024-03-08T21:13:43.825 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.846+0000 7f95a2fd2740 20 client.16563 trim_cache size 0 max 16384 2024-03-08T21:13:43.825 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:{ 2024-03-08T21:13:43.825 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "message": "", 2024-03-08T21:13:43.825 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "return_code": 0 2024-03-08T21:13:43.825 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:} 2024-03-08T21:13:43.826 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.847+0000 7f959ec00640 1 -- 172.21.15.107:0/812493306 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f95800a2960 msgr2=0x7f95800a4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:13:43.826 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.847+0000 7f959ec00640 1 --2- 172.21.15.107:0/812493306 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f95800a2960 0x7f95800a4e20 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f9590096ed0 tx=0x7f959009d040 comp rx=0 tx=0).stop 2024-03-08T21:13:43.826 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.847+0000 7f959ec00640 1 -- 172.21.15.107:0/812493306 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f959810da60 msgr2=0x7f95981999f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:13:43.826 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.847+0000 7f959ec00640 1 --2- 172.21.15.107:0/812493306 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f959810da60 0x7f95981999f0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f959000b9e0 tx=0x7f959000bea0 comp rx=0 tx=0).stop 2024-03-08T21:13:43.826 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.847+0000 7f959ec00640 1 -- 172.21.15.107:0/812493306 shutdown_connections 2024-03-08T21:13:43.826 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.847+0000 7f959ec00640 1 --2- 172.21.15.107:0/812493306 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f95800a2960 0x7f95800a4e20 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:13:43.826 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.847+0000 7f959ec00640 1 --2- 172.21.15.107:0/812493306 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f959810da60 0x7f95981999f0 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:13:43.826 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.847+0000 7f959ec00640 1 --2- 172.21.15.107:0/812493306 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9598100860 0x7f959819fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:13:43.826 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.847+0000 7f959ec00640 1 --2- 172.21.15.107:0/812493306 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f95980ffeb0 0x7f959819f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:13:43.827 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.847+0000 7f959ec00640 1 -- 172.21.15.107:0/812493306 >> 172.21.15.107:0/812493306 conn(0x7f95980fbb00 msgr2=0x7f95980fdef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:13:43.827 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.847+0000 7f959ec00640 1 -- 172.21.15.107:0/812493306 shutdown_connections 2024-03-08T21:13:43.827 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:43.847+0000 7f959ec00640 1 -- 172.21.15.107:0/812493306 wait complete. 2024-03-08T21:13:44.872 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:13:44 smithi204 ceph-mon[34935]: pgmap v418: 97 pgs: 97 active+clean; 8.8 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:13:44.961 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:13:44 smithi107 ceph-mon[31381]: pgmap v418: 97 pgs: 97 active+clean; 8.8 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:13:45.008 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:13:44 smithi149 ceph-mon[35042]: pgmap v418: 97 pgs: 97 active+clean; 8.8 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:13:46.872 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:13:46 smithi204 ceph-mon[34935]: pgmap v419: 97 pgs: 97 active+clean; 8.8 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-08T21:13:46.961 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:13:46 smithi107 ceph-mon[31381]: pgmap v419: 97 pgs: 97 active+clean; 8.8 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-08T21:13:47.008 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:13:46 smithi149 ceph-mon[35042]: pgmap v419: 97 pgs: 97 active+clean; 8.8 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-08T21:13:48.872 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:13:48 smithi204 ceph-mon[34935]: pgmap v420: 97 pgs: 97 active+clean; 8.8 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-08T21:13:48.961 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:13:48 smithi107 ceph-mon[31381]: pgmap v420: 97 pgs: 97 active+clean; 8.8 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-08T21:13:49.009 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:13:48 smithi149 ceph-mon[35042]: pgmap v420: 97 pgs: 97 active+clean; 8.8 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-08T21:13:50.872 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:13:50 smithi204 ceph-mon[34935]: pgmap v421: 97 pgs: 97 active+clean; 8.8 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-08T21:13:50.961 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:13:50 smithi107 ceph-mon[31381]: pgmap v421: 97 pgs: 97 active+clean; 8.8 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-08T21:13:51.009 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:13:50 smithi149 ceph-mon[35042]: pgmap v421: 97 pgs: 97 active+clean; 8.8 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-08T21:13:52.872 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:13:52 smithi204 ceph-mon[34935]: pgmap v422: 97 pgs: 97 active+clean; 8.8 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:13:52.961 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:13:52 smithi107 ceph-mon[31381]: pgmap v422: 97 pgs: 97 active+clean; 8.8 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:13:53.009 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:13:52 smithi149 ceph-mon[35042]: pgmap v422: 97 pgs: 97 active+clean; 8.8 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:13:54.872 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:13:54 smithi204 ceph-mon[34935]: pgmap v423: 97 pgs: 97 active+clean; 8.8 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:13:54.961 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:13:54 smithi107 ceph-mon[31381]: pgmap v423: 97 pgs: 97 active+clean; 8.8 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:13:55.008 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:13:54 smithi149 ceph-mon[35042]: pgmap v423: 97 pgs: 97 active+clean; 8.8 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:13:56.872 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:13:56 smithi204 ceph-mon[34935]: pgmap v424: 97 pgs: 97 active+clean; 8.8 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-08T21:13:56.961 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:13:56 smithi107 ceph-mon[31381]: pgmap v424: 97 pgs: 97 active+clean; 8.8 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-08T21:13:56.984 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.007+0000 7f5a7b200640 1 -- 172.21.15.107:0/3978893808 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5a7c1022a0 msgr2=0x7f5a7c102680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:13:56.984 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.007+0000 7f5a7b200640 1 --2- 172.21.15.107:0/3978893808 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5a7c1022a0 0x7f5a7c102680 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f5a6c009900 tx=0x7f5a6c02bcd0 comp rx=0 tx=0).stop 2024-03-08T21:13:56.984 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.007+0000 7f5a7b200640 1 -- 172.21.15.107:0/3978893808 shutdown_connections 2024-03-08T21:13:56.985 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.007+0000 7f5a7b200640 1 --2- 172.21.15.107:0/3978893808 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5a7c10b6c0 0x7f5a7c10db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:13:56.985 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.007+0000 7f5a7b200640 1 --2- 172.21.15.107:0/3978893808 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5a7c102bc0 0x7f5a7c10b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:13:56.985 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.007+0000 7f5a7b200640 1 --2- 172.21.15.107:0/3978893808 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5a7c1022a0 0x7f5a7c102680 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:13:56.985 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.007+0000 7f5a7b200640 1 -- 172.21.15.107:0/3978893808 >> 172.21.15.107:0/3978893808 conn(0x7f5a7c0fbb00 msgr2=0x7f5a7c0fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:13:56.985 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.007+0000 7f5a7b200640 1 -- 172.21.15.107:0/3978893808 shutdown_connections 2024-03-08T21:13:56.985 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.007+0000 7f5a7b200640 1 -- 172.21.15.107:0/3978893808 wait complete. 2024-03-08T21:13:56.987 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.009+0000 7f5a7b200640 1 Processor -- start 2024-03-08T21:13:56.987 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.009+0000 7f5a7b200640 1 -- start start 2024-03-08T21:13:56.987 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.009+0000 7f5a7b200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5a7c1022a0 0x7f5a7c0ffef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:13:56.987 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.009+0000 7f5a7b200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5a7c102bc0 0x7f5a7c100430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:13:56.987 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.009+0000 7f5a7b200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5a7c10b6c0 0x7f5a7c100970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:13:56.987 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.009+0000 7f5a7b200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a7c1113a0 con 0x7f5a7c1022a0 2024-03-08T21:13:56.987 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.009+0000 7f5a7b200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a7c111220 con 0x7f5a7c102bc0 2024-03-08T21:13:56.988 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.009+0000 7f5a7b200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a7c111520 con 0x7f5a7c10b6c0 2024-03-08T21:13:56.988 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.009+0000 7f5a79e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5a7c1022a0 0x7f5a7c0ffef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:13:56.988 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.010+0000 7f5a79e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5a7c1022a0 0x7f5a7c0ffef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:42096/0 (socket says 172.21.15.107:42096) 2024-03-08T21:13:56.988 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.010+0000 7f5a79e00640 1 -- 172.21.15.107:0/3712612902 learned_addr learned my addr 172.21.15.107:0/3712612902 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:13:56.988 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.010+0000 7f5a79400640 1 --2- 172.21.15.107:0/3712612902 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5a7c102bc0 0x7f5a7c100430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:13:56.988 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.010+0000 7f5a7a800640 1 --2- 172.21.15.107:0/3712612902 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5a7c10b6c0 0x7f5a7c100970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:13:56.988 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.010+0000 7f5a79e00640 1 -- 172.21.15.107:0/3712612902 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5a7c10b6c0 msgr2=0x7f5a7c100970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:13:56.988 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.010+0000 7f5a79e00640 1 --2- 172.21.15.107:0/3712612902 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5a7c10b6c0 0x7f5a7c100970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:13:56.988 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.010+0000 7f5a79e00640 1 -- 172.21.15.107:0/3712612902 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5a7c102bc0 msgr2=0x7f5a7c100430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:13:56.988 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.010+0000 7f5a79e00640 1 --2- 172.21.15.107:0/3712612902 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5a7c102bc0 0x7f5a7c100430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:13:56.988 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.010+0000 7f5a79e00640 1 -- 172.21.15.107:0/3712612902 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a7c1a5e80 con 0x7f5a7c1022a0 2024-03-08T21:13:56.988 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.010+0000 7f5a7a800640 1 --2- 172.21.15.107:0/3712612902 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5a7c10b6c0 0x7f5a7c100970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T21:13:56.988 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.010+0000 7f5a79e00640 1 --2- 172.21.15.107:0/3712612902 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5a7c1022a0 0x7f5a7c0ffef0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f5a6c000c00 tx=0x7f5a6c02d890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:13:56.989 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.011+0000 7f5a72a00640 1 -- 172.21.15.107:0/3712612902 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a6c043020 con 0x7f5a7c1022a0 2024-03-08T21:13:56.989 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.011+0000 7f5a72a00640 1 -- 172.21.15.107:0/3712612902 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f5a6c02dd50 con 0x7f5a7c1022a0 2024-03-08T21:13:56.989 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.011+0000 7f5a7b200640 1 -- 172.21.15.107:0/3712612902 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a7c1a6110 con 0x7f5a7c1022a0 2024-03-08T21:13:56.989 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.011+0000 7f5a72a00640 1 -- 172.21.15.107:0/3712612902 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a6c003ab0 con 0x7f5a7c1022a0 2024-03-08T21:13:56.989 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.011+0000 7f5a7b200640 1 -- 172.21.15.107:0/3712612902 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a7c1a6660 con 0x7f5a7c1022a0 2024-03-08T21:13:56.990 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.014+0000 7f5a72a00640 1 -- 172.21.15.107:0/3712612902 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) v1 ==== 137422+0+0 (secure 0 0 0) 0x7f5a6c046020 con 0x7f5a7c1022a0 2024-03-08T21:13:56.993 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.015+0000 7f5a83810740 1 Processor -- start 2024-03-08T21:13:56.993 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.015+0000 7f5a83810740 1 -- start start 2024-03-08T21:13:56.993 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.016+0000 7f5a83810740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dc74bade80 0x55dc74bae260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:13:56.993 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.016+0000 7f5a83810740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dc74bae7a0 0x55dc74bb7030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:13:56.993 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.016+0000 7f5a83810740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc74bb7570 0x55dc74bb9960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:13:56.993 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.016+0000 7f5a83810740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc74b113d0 con 0x55dc74bade80 2024-03-08T21:13:56.993 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.016+0000 7f5a83810740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc74aba6b0 con 0x55dc74bae7a0 2024-03-08T21:13:56.993 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.016+0000 7f5a83810740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc74b0d470 con 0x55dc74bb7570 2024-03-08T21:13:56.993 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.016+0000 7f5a7a800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dc74bade80 0x55dc74bae260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:13:56.993 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.016+0000 7f5a7a800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dc74bade80 0x55dc74bae260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:42100/0 (socket says 172.21.15.107:42100) 2024-03-08T21:13:56.993 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.016+0000 7f5a7a800640 1 -- 172.21.15.107:0/1585820755 learned_addr learned my addr 172.21.15.107:0/1585820755 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:13:56.993 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.016+0000 7f5a72a00640 1 --2- 172.21.15.107:0/3712612902 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f5a600a2960 0x7f5a600a4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:13:56.994 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.016+0000 7f5a72a00640 1 -- 172.21.15.107:0/3712612902 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f5a6c0f0fa0 con 0x7f5a7c1022a0 2024-03-08T21:13:56.994 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.016+0000 7f5a79400640 1 --2- 172.21.15.107:0/1585820755 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc74bb7570 0x55dc74bb9960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:13:56.994 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.016+0000 7f5a79e00640 1 --2- 172.21.15.107:0/1585820755 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dc74bae7a0 0x55dc74bb7030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:13:56.995 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.017+0000 7f5a7a800640 1 --2- 172.21.15.107:0/3712612902 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f5a600a2960 0x7f5a600a4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:13:56.995 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.017+0000 7f5a7a800640 1 -- 172.21.15.107:0/1585820755 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc74bb7570 msgr2=0x55dc74bb9960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:13:56.995 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.017+0000 7f5a7a800640 1 --2- 172.21.15.107:0/1585820755 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc74bb7570 0x55dc74bb9960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:13:56.995 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.017+0000 7f5a7a800640 1 -- 172.21.15.107:0/1585820755 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dc74bae7a0 msgr2=0x55dc74bb7030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:13:56.995 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.017+0000 7f5a7a800640 1 --2- 172.21.15.107:0/1585820755 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dc74bae7a0 0x55dc74bb7030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:13:56.995 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.017+0000 7f5a7a800640 1 -- 172.21.15.107:0/1585820755 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dc74ae0b50 con 0x55dc74bade80 2024-03-08T21:13:56.995 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.017+0000 7f5a7a800640 1 --2- 172.21.15.107:0/1585820755 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dc74bade80 0x55dc74bae260 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f5a74014930 tx=0x7f5a74014d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af1a1712f75c254b server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:13:56.995 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.018+0000 7f5a7a800640 1 --2- 172.21.15.107:0/3712612902 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f5a600a2960 0x7f5a600a4e20 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f5a7400b000 tx=0x7f5a7401a000 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:13:56.995 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.018+0000 7f5a67000640 1 -- 172.21.15.107:0/1585820755 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a74021070 con 0x55dc74bade80 2024-03-08T21:13:56.995 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.018+0000 7f5a67000640 1 -- 172.21.15.107:0/1585820755 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f5a7400fe50 con 0x55dc74bade80 2024-03-08T21:13:56.995 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.018+0000 7f5a67000640 1 -- 172.21.15.107:0/1585820755 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a7401caa0 con 0x55dc74bade80 2024-03-08T21:13:56.998 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.019+0000 7f5a83810740 1 -- 172.21.15.107:0/1585820755 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dc74bade80 msgr2=0x55dc74bae260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:13:56.998 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.019+0000 7f5a83810740 1 --2- 172.21.15.107:0/1585820755 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dc74bade80 0x55dc74bae260 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f5a74014930 tx=0x7f5a74014d30 comp rx=0 tx=0).stop 2024-03-08T21:13:56.998 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.019+0000 7f5a83810740 1 -- 172.21.15.107:0/1585820755 shutdown_connections 2024-03-08T21:13:56.999 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.019+0000 7f5a83810740 1 --2- 172.21.15.107:0/1585820755 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc74bb7570 0x55dc74bb9960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:13:56.999 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.019+0000 7f5a83810740 1 --2- 172.21.15.107:0/1585820755 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dc74bae7a0 0x55dc74bb7030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:13:56.999 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.019+0000 7f5a83810740 1 --2- 172.21.15.107:0/1585820755 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dc74bade80 0x55dc74bae260 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:13:56.999 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.019+0000 7f5a83810740 1 -- 172.21.15.107:0/1585820755 >> 172.21.15.107:0/1585820755 conn(0x55dc74ba87a0 msgr2=0x55dc74ba8b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:13:56.999 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.019+0000 7f5a83810740 1 -- 172.21.15.107:0/1585820755 shutdown_connections 2024-03-08T21:13:56.999 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.019+0000 7f5a83810740 1 -- 172.21.15.107:0/1585820755 wait complete. 2024-03-08T21:13:56.999 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.019+0000 7f5a83810740 1 -- start start 2024-03-08T21:13:56.999 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.020+0000 7f5a83810740 1 Processor -- start 2024-03-08T21:13:56.999 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.020+0000 7f5a83810740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc74bade80 0x55dc74bb7000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:13:56.999 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.020+0000 7f5a83810740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dc74bae7a0 0x55dc74bb4390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:13:56.999 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.020+0000 7f5a83810740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dc74bb7570 0x55dc74bb4a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:13:56.999 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.020+0000 7f5a83810740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc74bba140 con 0x55dc74bae7a0 2024-03-08T21:13:57.000 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.020+0000 7f5a83810740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc74acd9f0 con 0x55dc74bb7570 2024-03-08T21:13:57.000 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.020+0000 7f5a83810740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc74bba2c0 con 0x55dc74bade80 2024-03-08T21:13:57.000 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.020+0000 7f5a83810740 10 client.0 _ll_register_callbacks cb 0x55dc74825a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T21:13:57.000 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.020+0000 7f5a79400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dc74bae7a0 0x55dc74bb4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:13:57.000 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.021+0000 7f5a79400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dc74bae7a0 0x55dc74bb4390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:42108/0 (socket says 172.21.15.107:42108) 2024-03-08T21:13:57.000 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.021+0000 7f5a79400640 1 -- 172.21.15.107:0/2606373651 learned_addr learned my addr 172.21.15.107:0/2606373651 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:13:57.000 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.021+0000 7f5a79e00640 1 --2- 172.21.15.107:0/2606373651 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc74bade80 0x55dc74bb7000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:13:57.000 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.021+0000 7f5a7a800640 1 --2- 172.21.15.107:0/2606373651 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dc74bb7570 0x55dc74bb4a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:13:57.000 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.021+0000 7f5a79400640 1 -- 172.21.15.107:0/2606373651 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc74bade80 msgr2=0x55dc74bb7000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:13:57.000 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.021+0000 7f5a79400640 1 --2- 172.21.15.107:0/2606373651 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc74bade80 0x55dc74bb7000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:13:57.000 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.021+0000 7f5a79400640 1 -- 172.21.15.107:0/2606373651 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dc74bb7570 msgr2=0x55dc74bb4a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:13:57.000 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.021+0000 7f5a79400640 1 --2- 172.21.15.107:0/2606373651 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dc74bb7570 0x55dc74bb4a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:13:57.000 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.021+0000 7f5a79400640 1 -- 172.21.15.107:0/2606373651 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55dc74826210 con 0x55dc74bae7a0 2024-03-08T21:13:57.000 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.021+0000 7f5a79400640 1 --2- 172.21.15.107:0/2606373651 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dc74bae7a0 0x55dc74bb4390 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f5a6800be70 tx=0x7f5a68012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b123133a26fcffa7 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:13:57.001 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.021+0000 7f5a5b400640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-03-08T21:13:57.001 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.022+0000 7f5a5b400640 1 -- 172.21.15.107:0/2606373651 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a6801f060 con 0x55dc74bae7a0 2024-03-08T21:13:57.001 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.022+0000 7f5a5b400640 1 -- 172.21.15.107:0/2606373651 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(109..109 src has 1..109) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f5a68010960 con 0x55dc74bae7a0 2024-03-08T21:13:57.001 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.022+0000 7f5a83810740 1 -- 172.21.15.107:0/2606373651 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5a7400fe50 con 0x55dc74bae7a0 2024-03-08T21:13:57.001 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.022+0000 7f5a5b400640 1 -- 172.21.15.107:0/2606373651 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5a68010b60 con 0x55dc74bae7a0 2024-03-08T21:13:57.001 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.022+0000 7f5a83810740 10 client.16575 fetch_fsmap learned FSMap version 41 2024-03-08T21:13:57.001 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.022+0000 7f5a83810740 1 -- 172.21.15.107:0/2606373651 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x55dc748f2620 con 0x55dc74bae7a0 2024-03-08T21:13:57.002 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.022+0000 7f5a5b400640 1 -- 172.21.15.107:0/2606373651 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f5a7400fe50 con 0x55dc74bae7a0 2024-03-08T21:13:57.002 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.023+0000 7f5a5b400640 1 -- 172.21.15.107:0/2606373651 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 41) v1 ==== 7307+0+0 (secure 0 0 0) 0x7f5a68019030 con 0x55dc74bae7a0 2024-03-08T21:13:57.002 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.023+0000 7f5a83810740 10 client.16575 fetch_fsmap finished waiting for FSMap version 41 2024-03-08T21:13:57.002 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.023+0000 7f5a83810740 10 client.16575 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-08T21:13:57.002 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.023+0000 7f5a83810740 20 client.16575 populate_metadata read hostname 'smithi107' 2024-03-08T21:13:57.002 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.023+0000 7f5a83810740 1 --2- 172.21.15.107:0/2606373651 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55dc74bc74e0 0x55dc74bc98d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:13:57.002 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.023+0000 7f5a83810740 4 client.16575 mds_command: new command op to 34475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-08T21:13:57.002 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.023+0000 7f5a83810740 1 -- 172.21.15.107:0/2606373651 --> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55dc74b1e910 con 0x55dc74bc74e0 2024-03-08T21:13:57.003 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.024+0000 7f5a79e00640 1 --2- 172.21.15.107:0/2606373651 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55dc74bc74e0 0x55dc74bc98d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:13:57.003 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.024+0000 7f5a79e00640 1 --2- 172.21.15.107:0/2606373651 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55dc74bc74e0 0x55dc74bc98d0 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9186f0d21e40bbd server_cookie=43deb85496b5d224 in_seq=0 out_seq=0 2024-03-08T21:13:57.003 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.024+0000 7f5a5b400640 10 client.16575 ms_handle_connect on v2:172.21.15.149:6838/55290596 2024-03-08T21:13:57.004 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.026+0000 7f5a5b400640 1 -- 172.21.15.107:0/2606373651 <== mds.0 v2:172.21.15.149:6838/55290596 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55dc74b1e910 con 0x55dc74bc74e0 2024-03-08T21:13:57.004 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.026+0000 7f5a5b400640 10 client.16575 handle_command_reply: tid=0 2024-03-08T21:13:57.004 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.026+0000 7f5a83810740 1 client.16575 shutdown 2024-03-08T21:13:57.004 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.026+0000 7f5a83810740 1 -- 172.21.15.107:0/2606373651 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dc74bae7a0 msgr2=0x55dc74bb4390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:13:57.005 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.026+0000 7f5a83810740 1 --2- 172.21.15.107:0/2606373651 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dc74bae7a0 0x55dc74bb4390 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f5a6800be70 tx=0x7f5a68012820 comp rx=0 tx=0).stop 2024-03-08T21:13:57.005 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.026+0000 7f5a83810740 1 -- 172.21.15.107:0/2606373651 shutdown_connections 2024-03-08T21:13:57.005 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.026+0000 7f5a83810740 1 --2- 172.21.15.107:0/2606373651 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55dc74bc74e0 0x55dc74bc98d0 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:13:57.005 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.026+0000 7f5a83810740 1 --2- 172.21.15.107:0/2606373651 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dc74bb7570 0x55dc74bb4a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:13:57.005 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.026+0000 7f5a83810740 1 --2- 172.21.15.107:0/2606373651 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dc74bae7a0 0x55dc74bb4390 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:13:57.005 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.026+0000 7f5a83810740 1 --2- 172.21.15.107:0/2606373651 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc74bade80 0x55dc74bb7000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:13:57.005 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.026+0000 7f5a5b400640 0 client.16575 ms_handle_reset on v2:172.21.15.149:6838/55290596 2024-03-08T21:13:57.005 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.026+0000 7f5a83810740 1 -- 172.21.15.107:0/2606373651 >> 172.21.15.107:0/2606373651 conn(0x55dc74ba87a0 msgr2=0x55dc74baa4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:13:57.005 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.027+0000 7f5a83810740 1 -- 172.21.15.107:0/2606373651 shutdown_connections 2024-03-08T21:13:57.005 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.027+0000 7f5a83810740 1 -- 172.21.15.107:0/2606373651 wait complete. 2024-03-08T21:13:57.005 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.027+0000 7f5a83810740 20 client.16575 trim_cache size 0 max 16384 2024-03-08T21:13:57.008 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:13:56 smithi149 ceph-mon[35042]: pgmap v424: 97 pgs: 97 active+clean; 8.8 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-08T21:13:57.025 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.048+0000 7f5a83810740 1 Processor -- start 2024-03-08T21:13:57.025 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.048+0000 7f5a83810740 1 -- start start 2024-03-08T21:13:57.025 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.048+0000 7f5a83810740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dc74bade80 0x55dc74bb4330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:13:57.026 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.048+0000 7f5a83810740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc74bae7a0 0x55dc74bb6ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:13:57.026 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.048+0000 7f5a83810740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dc74bb7570 0x55dc74bb7000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:13:57.026 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.048+0000 7f5a83810740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a7c1b0670 con 0x55dc74bade80 2024-03-08T21:13:57.026 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.048+0000 7f5a83810740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a7c1ae690 con 0x55dc74bb7570 2024-03-08T21:13:57.026 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.048+0000 7f5a83810740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a7c1b0010 con 0x55dc74bae7a0 2024-03-08T21:13:57.026 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.048+0000 7f5a79e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dc74bade80 0x55dc74bb4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:13:57.026 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.048+0000 7f5a79e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dc74bade80 0x55dc74bb4330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:42112/0 (socket says 172.21.15.107:42112) 2024-03-08T21:13:57.026 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.049+0000 7f5a79e00640 1 -- 172.21.15.107:0/3797710726 learned_addr learned my addr 172.21.15.107:0/3797710726 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:13:57.026 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.049+0000 7f5a79400640 1 --2- 172.21.15.107:0/3797710726 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc74bae7a0 0x55dc74bb6ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:13:57.026 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.049+0000 7f5a79e00640 1 -- 172.21.15.107:0/3797710726 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc74bae7a0 msgr2=0x55dc74bb6ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:13:57.026 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.049+0000 7f5a79e00640 1 --2- 172.21.15.107:0/3797710726 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc74bae7a0 0x55dc74bb6ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:13:57.026 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.049+0000 7f5a7a800640 1 --2- 172.21.15.107:0/3797710726 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dc74bb7570 0x55dc74bb7000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:13:57.026 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.049+0000 7f5a79e00640 1 -- 172.21.15.107:0/3797710726 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dc74bb7570 msgr2=0x55dc74bb7000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:13:57.027 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.049+0000 7f5a79e00640 1 --2- 172.21.15.107:0/3797710726 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dc74bb7570 0x55dc74bb7000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:13:57.027 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.049+0000 7f5a79e00640 1 -- 172.21.15.107:0/3797710726 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dc74bb4df0 con 0x55dc74bade80 2024-03-08T21:13:57.027 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.049+0000 7f5a7a800640 1 --2- 172.21.15.107:0/3797710726 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dc74bb7570 0x55dc74bb7000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T21:13:57.030 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.049+0000 7f5a79e00640 1 --2- 172.21.15.107:0/3797710726 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dc74bade80 0x55dc74bb4330 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f5a6c0ee7f0 tx=0x7f5a6c0f33f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fff716806df41b56 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:13:57.030 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.050+0000 7f5a66600640 1 -- 172.21.15.107:0/3797710726 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a6c0bd6a0 con 0x55dc74bade80 2024-03-08T21:13:57.030 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.050+0000 7f5a66600640 1 -- 172.21.15.107:0/3797710726 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f5a6c0c17f0 con 0x55dc74bade80 2024-03-08T21:13:57.030 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.050+0000 7f5a66600640 1 -- 172.21.15.107:0/3797710726 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a6c0ee900 con 0x55dc74bade80 2024-03-08T21:13:57.030 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.050+0000 7f5a83810740 1 -- 172.21.15.107:0/3797710726 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dc74bade80 msgr2=0x55dc74bb4330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:13:57.030 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.050+0000 7f5a83810740 1 --2- 172.21.15.107:0/3797710726 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dc74bade80 0x55dc74bb4330 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f5a6c0ee7f0 tx=0x7f5a6c0f33f0 comp rx=0 tx=0).stop 2024-03-08T21:13:57.030 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.050+0000 7f5a83810740 1 -- 172.21.15.107:0/3797710726 shutdown_connections 2024-03-08T21:13:57.030 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.050+0000 7f5a83810740 1 --2- 172.21.15.107:0/3797710726 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dc74bb7570 0x55dc74bb7000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:13:57.031 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.050+0000 7f5a83810740 1 --2- 172.21.15.107:0/3797710726 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc74bae7a0 0x55dc74bb6ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:13:57.031 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.050+0000 7f5a83810740 1 --2- 172.21.15.107:0/3797710726 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dc74bade80 0x55dc74bb4330 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:13:57.031 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.051+0000 7f5a83810740 1 -- 172.21.15.107:0/3797710726 >> 172.21.15.107:0/3797710726 conn(0x55dc74ba87a0 msgr2=0x55dc74bb3c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:13:57.031 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.051+0000 7f5a83810740 1 -- 172.21.15.107:0/3797710726 shutdown_connections 2024-03-08T21:13:57.031 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.051+0000 7f5a83810740 1 -- 172.21.15.107:0/3797710726 wait complete. 2024-03-08T21:13:57.031 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.051+0000 7f5a83810740 1 -- start start 2024-03-08T21:13:57.031 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.051+0000 7f5a83810740 1 Processor -- start 2024-03-08T21:13:57.031 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.052+0000 7f5a83810740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dc74bade80 0x55dc74bbce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:13:57.031 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.052+0000 7f5a83810740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dc74bae7a0 0x55dc74bbd370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:13:57.031 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.052+0000 7f5a83810740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc74bb7570 0x55dc74bbd8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:13:57.031 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.052+0000 7f5a83810740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a7c1af350 con 0x55dc74bae7a0 2024-03-08T21:13:57.031 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.052+0000 7f5a83810740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a7c1aecf0 con 0x55dc74bade80 2024-03-08T21:13:57.031 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.052+0000 7f5a83810740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a7c1af9b0 con 0x55dc74bb7570 2024-03-08T21:13:57.032 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.052+0000 7f5a83810740 10 client.0 _ll_register_callbacks cb 0x7f5a740085f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T21:13:57.032 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.052+0000 7f5a79400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dc74bae7a0 0x55dc74bbd370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:13:57.032 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.052+0000 7f5a79400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dc74bae7a0 0x55dc74bbd370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:42118/0 (socket says 172.21.15.107:42118) 2024-03-08T21:13:57.032 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.052+0000 7f5a79400640 1 -- 172.21.15.107:0/3651457213 learned_addr learned my addr 172.21.15.107:0/3651457213 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:13:57.032 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.052+0000 7f5a79e00640 1 --2- 172.21.15.107:0/3651457213 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dc74bade80 0x55dc74bbce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:13:57.032 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.052+0000 7f5a7a800640 1 --2- 172.21.15.107:0/3651457213 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc74bb7570 0x55dc74bbd8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:13:57.032 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.053+0000 7f5a79400640 1 -- 172.21.15.107:0/3651457213 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc74bb7570 msgr2=0x55dc74bbd8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:13:57.032 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.053+0000 7f5a79400640 1 --2- 172.21.15.107:0/3651457213 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc74bb7570 0x55dc74bbd8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:13:57.032 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.053+0000 7f5a79400640 1 -- 172.21.15.107:0/3651457213 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dc74bade80 msgr2=0x55dc74bbce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:13:57.032 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.053+0000 7f5a79400640 1 --2- 172.21.15.107:0/3651457213 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dc74bade80 0x55dc74bbce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:13:57.032 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.053+0000 7f5a79400640 1 -- 172.21.15.107:0/3651457213 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55dc74c1d9b0 con 0x55dc74bae7a0 2024-03-08T21:13:57.032 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.053+0000 7f5a79e00640 1 --2- 172.21.15.107:0/3651457213 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dc74bade80 0x55dc74bbce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T21:13:57.032 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.053+0000 7f5a7a800640 1 --2- 172.21.15.107:0/3651457213 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc74bb7570 0x55dc74bbd8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T21:13:57.032 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.053+0000 7f5a79400640 1 --2- 172.21.15.107:0/3651457213 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dc74bae7a0 0x55dc74bbd370 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f5a68022ec0 tx=0x7f5a68010990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2406fd0dfd83f84d server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:13:57.033 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.053+0000 7f5a5b400640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-03-08T21:13:57.033 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.053+0000 7f5a5b400640 1 -- 172.21.15.107:0/3651457213 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a68031060 con 0x55dc74bae7a0 2024-03-08T21:13:57.033 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.053+0000 7f5a83810740 1 -- 172.21.15.107:0/3651457213 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5a6c0c17f0 con 0x55dc74bae7a0 2024-03-08T21:13:57.033 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.053+0000 7f5a5b400640 1 -- 172.21.15.107:0/3651457213 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(109..109 src has 1..109) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f5a68005450 con 0x55dc74bae7a0 2024-03-08T21:13:57.033 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.053+0000 7f5a5b400640 1 -- 172.21.15.107:0/3651457213 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5a68005670 con 0x55dc74bae7a0 2024-03-08T21:13:57.033 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.053+0000 7f5a83810740 10 client.16581 fetch_fsmap learned FSMap version 41 2024-03-08T21:13:57.033 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.053+0000 7f5a83810740 1 -- 172.21.15.107:0/3651457213 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x55dc74c1eba0 con 0x55dc74bae7a0 2024-03-08T21:13:57.033 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.054+0000 7f5a5b400640 1 -- 172.21.15.107:0/3651457213 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f5a6c0c17f0 con 0x55dc74bae7a0 2024-03-08T21:13:57.034 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.054+0000 7f5a5b400640 1 -- 172.21.15.107:0/3651457213 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 41) v1 ==== 7307+0+0 (secure 0 0 0) 0x7f5a68019040 con 0x55dc74bae7a0 2024-03-08T21:13:57.034 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.054+0000 7f5a83810740 10 client.16581 fetch_fsmap finished waiting for FSMap version 41 2024-03-08T21:13:57.034 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.054+0000 7f5a83810740 10 client.16581 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-08T21:13:57.034 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.054+0000 7f5a83810740 20 client.16581 populate_metadata read hostname 'smithi107' 2024-03-08T21:13:57.034 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.054+0000 7f5a83810740 1 --2- 172.21.15.107:0/3651457213 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55dc74c1f1b0 0x55dc74c215a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:13:57.034 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.054+0000 7f5a83810740 4 client.16581 mds_command: new command op to 34475 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-08T21:13:57.035 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.054+0000 7f5a83810740 1 -- 172.21.15.107:0/3651457213 --> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55dc74b18c60 con 0x55dc74c1f1b0 2024-03-08T21:13:57.035 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.055+0000 7f5a79e00640 1 --2- 172.21.15.107:0/3651457213 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55dc74c1f1b0 0x55dc74c215a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:13:57.035 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.056+0000 7f5a5b400640 10 client.16581 ms_handle_connect on v2:172.21.15.149:6838/55290596 2024-03-08T21:13:57.035 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.056+0000 7f5a79e00640 1 --2- 172.21.15.107:0/3651457213 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55dc74c1f1b0 0x55dc74c215a0 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=16c6bd63485d0247 server_cookie=94c24da0ede637ac in_seq=0 out_seq=0 2024-03-08T21:13:57.043 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.065+0000 7f5a5b400640 1 -- 172.21.15.107:0/3651457213 <== mds.0 v2:172.21.15.149:6838/55290596 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55dc74b18c60 con 0x55dc74c1f1b0 2024-03-08T21:13:57.043 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.065+0000 7f5a5b400640 10 client.16581 handle_command_reply: tid=0 2024-03-08T21:13:57.043 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.065+0000 7f5a83810740 1 client.16581 shutdown 2024-03-08T21:13:57.043 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.066+0000 7f5a83810740 1 -- 172.21.15.107:0/3651457213 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dc74bae7a0 msgr2=0x55dc74bbd370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:13:57.043 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.066+0000 7f5a83810740 1 --2- 172.21.15.107:0/3651457213 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dc74bae7a0 0x55dc74bbd370 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f5a68022ec0 tx=0x7f5a68010990 comp rx=0 tx=0).stop 2024-03-08T21:13:57.043 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.066+0000 7f5a83810740 1 -- 172.21.15.107:0/3651457213 shutdown_connections 2024-03-08T21:13:57.043 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.066+0000 7f5a83810740 1 --2- 172.21.15.107:0/3651457213 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55dc74bb7570 0x55dc74bbd8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:13:57.043 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.066+0000 7f5a83810740 1 --2- 172.21.15.107:0/3651457213 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dc74bae7a0 0x55dc74bbd370 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:13:57.043 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.066+0000 7f5a83810740 1 --2- 172.21.15.107:0/3651457213 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55dc74c1f1b0 0x55dc74c215a0 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:13:57.043 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.066+0000 7f5a83810740 1 --2- 172.21.15.107:0/3651457213 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55dc74bade80 0x55dc74bbce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:13:57.044 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.066+0000 7f5a83810740 1 -- 172.21.15.107:0/3651457213 >> 172.21.15.107:0/3651457213 conn(0x55dc74ba87a0 msgr2=0x55dc74ba9a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:13:57.044 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.066+0000 7f5a5b400640 0 client.16581 ms_handle_reset on v2:172.21.15.149:6838/55290596 2024-03-08T21:13:57.044 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.066+0000 7f5a83810740 1 -- 172.21.15.107:0/3651457213 shutdown_connections 2024-03-08T21:13:57.044 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.066+0000 7f5a83810740 1 -- 172.21.15.107:0/3651457213 wait complete. 2024-03-08T21:13:57.044 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.066+0000 7f5a83810740 20 client.16581 trim_cache size 0 max 16384 2024-03-08T21:13:57.044 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:{ 2024-03-08T21:13:57.044 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "message": "", 2024-03-08T21:13:57.044 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "return_code": 0 2024-03-08T21:13:57.044 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:} 2024-03-08T21:13:57.045 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.067+0000 7f5a7b200640 1 -- 172.21.15.107:0/3712612902 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f5a600a2960 msgr2=0x7f5a600a4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:13:57.045 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.067+0000 7f5a7b200640 1 --2- 172.21.15.107:0/3712612902 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f5a600a2960 0x7f5a600a4e20 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f5a7400b000 tx=0x7f5a7401a000 comp rx=0 tx=0).stop 2024-03-08T21:13:57.045 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.067+0000 7f5a7b200640 1 -- 172.21.15.107:0/3712612902 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5a7c1022a0 msgr2=0x7f5a7c0ffef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:13:57.045 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.067+0000 7f5a7b200640 1 --2- 172.21.15.107:0/3712612902 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5a7c1022a0 0x7f5a7c0ffef0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f5a6c000c00 tx=0x7f5a6c02d890 comp rx=0 tx=0).stop 2024-03-08T21:13:57.045 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.067+0000 7f5a7b200640 1 -- 172.21.15.107:0/3712612902 shutdown_connections 2024-03-08T21:13:57.046 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.067+0000 7f5a7b200640 1 --2- 172.21.15.107:0/3712612902 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f5a600a2960 0x7f5a600a4e20 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:13:57.046 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.067+0000 7f5a7b200640 1 --2- 172.21.15.107:0/3712612902 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f5a7c10b6c0 0x7f5a7c100970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:13:57.046 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.067+0000 7f5a7b200640 1 --2- 172.21.15.107:0/3712612902 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5a7c102bc0 0x7f5a7c100430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:13:57.046 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.067+0000 7f5a7b200640 1 --2- 172.21.15.107:0/3712612902 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5a7c1022a0 0x7f5a7c0ffef0 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:13:57.046 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.067+0000 7f5a7b200640 1 -- 172.21.15.107:0/3712612902 >> 172.21.15.107:0/3712612902 conn(0x7f5a7c0fbb00 msgr2=0x7f5a7c0fd400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:13:57.046 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.067+0000 7f5a7b200640 1 -- 172.21.15.107:0/3712612902 shutdown_connections 2024-03-08T21:13:57.046 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:13:57.067+0000 7f5a7b200640 1 -- 172.21.15.107:0/3712612902 wait complete. 2024-03-08T21:13:58.961 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:13:58 smithi107 ceph-mon[31381]: pgmap v425: 97 pgs: 97 active+clean; 8.8 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:13:59.008 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:13:58 smithi149 ceph-mon[35042]: pgmap v425: 97 pgs: 97 active+clean; 8.8 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:13:59.122 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:13:58 smithi204 ceph-mon[34935]: pgmap v425: 97 pgs: 97 active+clean; 8.8 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:14:00.961 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:14:00 smithi107 ceph-mon[31381]: pgmap v426: 97 pgs: 97 active+clean; 8.8 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 511 B/s wr, 13 op/s 2024-03-08T21:14:01.009 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:14:00 smithi149 ceph-mon[35042]: pgmap v426: 97 pgs: 97 active+clean; 8.8 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 511 B/s wr, 13 op/s 2024-03-08T21:14:01.122 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:14:00 smithi204 ceph-mon[34935]: pgmap v426: 97 pgs: 97 active+clean; 8.8 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 511 B/s wr, 13 op/s 2024-03-08T21:14:02.961 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:14:02 smithi107 ceph-mon[31381]: pgmap v427: 97 pgs: 97 active+clean; 8.8 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:14:03.009 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:14:02 smithi149 ceph-mon[35042]: pgmap v427: 97 pgs: 97 active+clean; 8.8 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:14:03.122 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:14:02 smithi204 ceph-mon[34935]: pgmap v427: 97 pgs: 97 active+clean; 8.8 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:14:04.961 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:14:04 smithi107 ceph-mon[31381]: pgmap v428: 97 pgs: 97 active+clean; 8.8 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:14:05.008 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:14:04 smithi149 ceph-mon[35042]: pgmap v428: 97 pgs: 97 active+clean; 8.8 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:14:05.122 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:14:04 smithi204 ceph-mon[34935]: pgmap v428: 97 pgs: 97 active+clean; 8.8 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:14:06.961 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:14:06 smithi107 ceph-mon[31381]: pgmap v429: 97 pgs: 97 active+clean; 9.5 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 104 KiB/s wr, 29 op/s 2024-03-08T21:14:07.008 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:14:06 smithi149 ceph-mon[35042]: pgmap v429: 97 pgs: 97 active+clean; 9.5 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 104 KiB/s wr, 29 op/s 2024-03-08T21:14:07.122 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:14:06 smithi204 ceph-mon[34935]: pgmap v429: 97 pgs: 97 active+clean; 9.5 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 104 KiB/s wr, 29 op/s 2024-03-08T21:14:08.924 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:14:08 smithi204 ceph-mon[34935]: pgmap v430: 97 pgs: 97 active+clean; 9.5 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 104 KiB/s wr, 25 op/s 2024-03-08T21:14:08.960 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:14:08 smithi107 ceph-mon[31381]: pgmap v430: 97 pgs: 97 active+clean; 9.5 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 104 KiB/s wr, 25 op/s 2024-03-08T21:14:09.009 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:14:08 smithi149 ceph-mon[35042]: pgmap v430: 97 pgs: 97 active+clean; 9.5 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 104 KiB/s wr, 25 op/s 2024-03-08T21:14:09.961 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:14:09 smithi107 ceph-mon[31381]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T21:14:10.009 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:14:09 smithi149 ceph-mon[35042]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T21:14:10.122 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:14:09 smithi204 ceph-mon[34935]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T21:14:10.193 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.216+0000 7ff2f5600640 1 -- 172.21.15.107:0/3707075221 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff2f0101cf0 msgr2=0x7ff2f0105d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:14:10.193 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.216+0000 7ff2f5600640 1 --2- 172.21.15.107:0/3707075221 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff2f0101cf0 0x7ff2f0105d30 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7ff2dc009900 tx=0x7ff2dc02bc30 comp rx=0 tx=0).stop 2024-03-08T21:14:10.193 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.216+0000 7ff2f5600640 1 -- 172.21.15.107:0/3707075221 shutdown_connections 2024-03-08T21:14:10.193 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.216+0000 7ff2f5600640 1 --2- 172.21.15.107:0/3707075221 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff2f0106410 0x7ff2f010dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:14:10.193 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.216+0000 7ff2f5600640 1 --2- 172.21.15.107:0/3707075221 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff2f0101cf0 0x7ff2f0105d30 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:14:10.193 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.216+0000 7ff2f5600640 1 --2- 172.21.15.107:0/3707075221 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff2f0101340 0x7ff2f0101720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:14:10.194 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.216+0000 7ff2f5600640 1 -- 172.21.15.107:0/3707075221 >> 172.21.15.107:0/3707075221 conn(0x7ff2f00fcc40 msgr2=0x7ff2f00ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:14:10.194 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.217+0000 7ff2f5600640 1 -- 172.21.15.107:0/3707075221 shutdown_connections 2024-03-08T21:14:10.194 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.217+0000 7ff2f5600640 1 -- 172.21.15.107:0/3707075221 wait complete. 2024-03-08T21:14:10.194 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.218+0000 7ff2f5600640 1 Processor -- start 2024-03-08T21:14:10.194 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.218+0000 7ff2f5600640 1 -- start start 2024-03-08T21:14:10.195 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.218+0000 7ff2f5600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff2f0101340 0x7ff2f019f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:14:10.195 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.218+0000 7ff2f5600640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff2f0106410 0x7ff2f019fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:14:10.195 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.218+0000 7ff2f5600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff2f0199980 0x7ff2f0199e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:14:10.196 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.218+0000 7ff2f5600640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2f01113a0 con 0x7ff2f0101340 2024-03-08T21:14:10.196 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.218+0000 7ff2f5600640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2f0111220 con 0x7ff2f0106410 2024-03-08T21:14:10.196 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.218+0000 7ff2f5600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2f0111520 con 0x7ff2f0199980 2024-03-08T21:14:10.196 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.219+0000 7ff2efe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff2f0101340 0x7ff2f019f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:14:10.196 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.219+0000 7ff2efe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff2f0101340 0x7ff2f019f780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46792/0 (socket says 172.21.15.107:46792) 2024-03-08T21:14:10.196 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.219+0000 7ff2efe00640 1 -- 172.21.15.107:0/400407988 learned_addr learned my addr 172.21.15.107:0/400407988 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:14:10.196 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.219+0000 7ff2ef400640 1 --2- 172.21.15.107:0/400407988 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff2f0106410 0x7ff2f019fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:14:10.196 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.219+0000 7ff2f4c00640 1 --2- 172.21.15.107:0/400407988 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff2f0199980 0x7ff2f0199e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:14:10.197 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.219+0000 7ff2efe00640 1 -- 172.21.15.107:0/400407988 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff2f0199980 msgr2=0x7ff2f0199e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:14:10.197 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.219+0000 7ff2efe00640 1 --2- 172.21.15.107:0/400407988 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff2f0199980 0x7ff2f0199e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:14:10.198 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.219+0000 7ff2efe00640 1 -- 172.21.15.107:0/400407988 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff2f0106410 msgr2=0x7ff2f019fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:14:10.198 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.219+0000 7ff2efe00640 1 --2- 172.21.15.107:0/400407988 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff2f0106410 0x7ff2f019fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:14:10.198 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.219+0000 7ff2efe00640 1 -- 172.21.15.107:0/400407988 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2f019a4a0 con 0x7ff2f0101340 2024-03-08T21:14:10.198 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.220+0000 7ff2efe00640 1 --2- 172.21.15.107:0/400407988 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff2f0101340 0x7ff2f019f780 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7ff2e000e9f0 tx=0x7ff2e000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:14:10.198 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.220+0000 7ff2ecc00640 1 -- 172.21.15.107:0/400407988 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2e000cd80 con 0x7ff2f0101340 2024-03-08T21:14:10.198 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.220+0000 7ff2f5600640 1 -- 172.21.15.107:0/400407988 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2f019a790 con 0x7ff2f0101340 2024-03-08T21:14:10.198 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.220+0000 7ff2ecc00640 1 -- 172.21.15.107:0/400407988 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7ff2e00108e0 con 0x7ff2f0101340 2024-03-08T21:14:10.198 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.220+0000 7ff2ecc00640 1 -- 172.21.15.107:0/400407988 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2e0018920 con 0x7ff2f0101340 2024-03-08T21:14:10.198 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.220+0000 7ff2f5600640 1 -- 172.21.15.107:0/400407988 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2f01a69e0 con 0x7ff2f0101340 2024-03-08T21:14:10.201 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.223+0000 7ff2f99c3740 1 Processor -- start 2024-03-08T21:14:10.201 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.224+0000 7ff2ecc00640 1 -- 172.21.15.107:0/400407988 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) v1 ==== 137422+0+0 (secure 0 0 0) 0x7ff2e0010a90 con 0x7ff2f0101340 2024-03-08T21:14:10.201 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.224+0000 7ff2f99c3740 1 -- start start 2024-03-08T21:14:10.201 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.224+0000 7ff2f99c3740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561814834e80 0x561814835260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:14:10.201 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.224+0000 7ff2f99c3740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5618148357a0 0x56181483e030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:14:10.201 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.224+0000 7ff2f99c3740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56181483e570 0x561814840960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:14:10.201 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.224+0000 7ff2f99c3740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618147983d0 con 0x5618148357a0 2024-03-08T21:14:10.202 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.224+0000 7ff2f99c3740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618147416b0 con 0x56181483e570 2024-03-08T21:14:10.202 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.224+0000 7ff2f99c3740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x561814794470 con 0x561814834e80 2024-03-08T21:14:10.202 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.224+0000 7ff2efe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5618148357a0 0x56181483e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:14:10.202 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.225+0000 7ff2efe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5618148357a0 0x56181483e030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46804/0 (socket says 172.21.15.107:46804) 2024-03-08T21:14:10.202 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.225+0000 7ff2efe00640 1 -- 172.21.15.107:0/90996503 learned_addr learned my addr 172.21.15.107:0/90996503 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:14:10.202 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.225+0000 7ff2ef400640 1 --2- 172.21.15.107:0/90996503 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56181483e570 0x561814840960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:14:10.202 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.225+0000 7ff2f4c00640 1 --2- 172.21.15.107:0/90996503 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561814834e80 0x561814835260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:14:10.204 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.225+0000 7ff2efe00640 1 -- 172.21.15.107:0/90996503 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561814834e80 msgr2=0x561814835260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:14:10.204 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.225+0000 7ff2efe00640 1 --2- 172.21.15.107:0/90996503 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561814834e80 0x561814835260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:14:10.204 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.225+0000 7ff2efe00640 1 -- 172.21.15.107:0/90996503 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56181483e570 msgr2=0x561814840960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:14:10.204 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.225+0000 7ff2efe00640 1 --2- 172.21.15.107:0/90996503 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56181483e570 0x561814840960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:14:10.204 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.225+0000 7ff2efe00640 1 -- 172.21.15.107:0/90996503 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561814767b50 con 0x5618148357a0 2024-03-08T21:14:10.204 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.226+0000 7ff2efe00640 1 --2- 172.21.15.107:0/90996503 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5618148357a0 0x56181483e030 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7ff2e00cc8a0 tx=0x7ff2e00d7930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa0b26fe7c3153af server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:14:10.204 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.226+0000 7ff2ecc00640 1 --2- 172.21.15.107:0/400407988 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7ff2d40a2960 0x7ff2d40a4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:14:10.204 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.226+0000 7ff2ecc00640 1 -- 172.21.15.107:0/400407988 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 8629+0+0 (secure 0 0 0) 0x7ff2e00cf2e0 con 0x7ff2f0101340 2024-03-08T21:14:10.205 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.226+0000 7ff2e5200640 1 -- 172.21.15.107:0/90996503 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2e00e2070 con 0x5618148357a0 2024-03-08T21:14:10.205 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.226+0000 7ff2f4c00640 1 --2- 172.21.15.107:0/400407988 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7ff2d40a2960 0x7ff2d40a4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:14:10.205 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.226+0000 7ff2e5200640 1 -- 172.21.15.107:0/90996503 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7ff2e0010de0 con 0x5618148357a0 2024-03-08T21:14:10.205 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.226+0000 7ff2e5200640 1 -- 172.21.15.107:0/90996503 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2e009b1d0 con 0x5618148357a0 2024-03-08T21:14:10.205 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.227+0000 7ff2f99c3740 1 -- 172.21.15.107:0/90996503 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5618148357a0 msgr2=0x56181483e030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:14:10.205 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.227+0000 7ff2f4c00640 1 --2- 172.21.15.107:0/400407988 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7ff2d40a2960 0x7ff2d40a4e20 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7ff2e80092f0 tx=0x7ff2e8009280 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:14:10.205 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.227+0000 7ff2f99c3740 1 --2- 172.21.15.107:0/90996503 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5618148357a0 0x56181483e030 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7ff2e00cc8a0 tx=0x7ff2e00d7930 comp rx=0 tx=0).stop 2024-03-08T21:14:10.205 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.227+0000 7ff2f99c3740 1 -- 172.21.15.107:0/90996503 shutdown_connections 2024-03-08T21:14:10.205 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.227+0000 7ff2f99c3740 1 --2- 172.21.15.107:0/90996503 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56181483e570 0x561814840960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:14:10.205 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.227+0000 7ff2f99c3740 1 --2- 172.21.15.107:0/90996503 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5618148357a0 0x56181483e030 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:14:10.205 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.227+0000 7ff2f99c3740 1 --2- 172.21.15.107:0/90996503 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561814834e80 0x561814835260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:14:10.205 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.227+0000 7ff2f99c3740 1 -- 172.21.15.107:0/90996503 >> 172.21.15.107:0/90996503 conn(0x56181482f7a0 msgr2=0x56181482fb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:14:10.205 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.227+0000 7ff2f99c3740 1 -- 172.21.15.107:0/90996503 shutdown_connections 2024-03-08T21:14:10.205 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.227+0000 7ff2f99c3740 1 -- 172.21.15.107:0/90996503 wait complete. 2024-03-08T21:14:10.205 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.228+0000 7ff2f99c3740 1 -- start start 2024-03-08T21:14:10.209 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.228+0000 7ff2f99c3740 1 Processor -- start 2024-03-08T21:14:10.209 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.229+0000 7ff2f99c3740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561814834e80 0x56181483dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:14:10.209 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.229+0000 7ff2f99c3740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5618148357a0 0x56181483b350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:14:10.209 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.229+0000 7ff2f99c3740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56181483e570 0x56181483ba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:14:10.209 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.229+0000 7ff2f99c3740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618148411a0 con 0x5618148357a0 2024-03-08T21:14:10.209 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.229+0000 7ff2f99c3740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618147549f0 con 0x56181483e570 2024-03-08T21:14:10.209 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.229+0000 7ff2f99c3740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x561814841320 con 0x561814834e80 2024-03-08T21:14:10.209 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.229+0000 7ff2f99c3740 10 client.0 _ll_register_callbacks cb 0x5618144aca00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T21:14:10.210 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.229+0000 7ff2ef400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5618148357a0 0x56181483b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:14:10.210 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.229+0000 7ff2ef400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5618148357a0 0x56181483b350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46806/0 (socket says 172.21.15.107:46806) 2024-03-08T21:14:10.210 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.229+0000 7ff2ef400640 1 -- 172.21.15.107:0/3080145199 learned_addr learned my addr 172.21.15.107:0/3080145199 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:14:10.210 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.229+0000 7ff2f4c00640 1 --2- 172.21.15.107:0/3080145199 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56181483e570 0x56181483ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:14:10.210 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.229+0000 7ff2efe00640 1 --2- 172.21.15.107:0/3080145199 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561814834e80 0x56181483dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:14:10.210 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.230+0000 7ff2ef400640 1 -- 172.21.15.107:0/3080145199 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561814834e80 msgr2=0x56181483dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:14:10.210 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.230+0000 7ff2ef400640 1 --2- 172.21.15.107:0/3080145199 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561814834e80 0x56181483dfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:14:10.210 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.230+0000 7ff2ef400640 1 -- 172.21.15.107:0/3080145199 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56181483e570 msgr2=0x56181483ba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:14:10.210 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.230+0000 7ff2ef400640 1 --2- 172.21.15.107:0/3080145199 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56181483e570 0x56181483ba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:14:10.211 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.230+0000 7ff2ef400640 1 -- 172.21.15.107:0/3080145199 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5618144ad210 con 0x5618148357a0 2024-03-08T21:14:10.211 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.230+0000 7ff2efe00640 1 --2- 172.21.15.107:0/3080145199 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561814834e80 0x56181483dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T21:14:10.211 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.230+0000 7ff2ef400640 1 --2- 172.21.15.107:0/3080145199 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5618148357a0 0x56181483b350 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7ff2f006a740 tx=0x7ff2dc02c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e18af9a14fed4506 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:14:10.211 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.230+0000 7ff2d9600640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-03-08T21:14:10.211 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.230+0000 7ff2d9600640 1 -- 172.21.15.107:0/3080145199 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2dc02c5d0 con 0x5618148357a0 2024-03-08T21:14:10.211 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.230+0000 7ff2d9600640 1 -- 172.21.15.107:0/3080145199 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 8629+0+0 (secure 0 0 0) 0x7ff2dc005030 con 0x5618148357a0 2024-03-08T21:14:10.211 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.230+0000 7ff2f99c3740 1 -- 172.21.15.107:0/3080145199 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5618147a5910 con 0x5618148357a0 2024-03-08T21:14:10.211 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.231+0000 7ff2d9600640 1 -- 172.21.15.107:0/3080145199 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff2dc005230 con 0x5618148357a0 2024-03-08T21:14:10.211 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.231+0000 7ff2f99c3740 10 client.16593 fetch_fsmap learned FSMap version 41 2024-03-08T21:14:10.211 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.231+0000 7ff2f99c3740 1 -- 172.21.15.107:0/3080145199 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x561814579620 con 0x5618148357a0 2024-03-08T21:14:10.211 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.231+0000 7ff2d9600640 1 -- 172.21.15.107:0/3080145199 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x5618147a5910 con 0x5618148357a0 2024-03-08T21:14:10.211 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.231+0000 7ff2d9600640 1 -- 172.21.15.107:0/3080145199 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 41) v1 ==== 7307+0+0 (secure 0 0 0) 0x7ff2dc0035f0 con 0x5618148357a0 2024-03-08T21:14:10.211 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.231+0000 7ff2f99c3740 10 client.16593 fetch_fsmap finished waiting for FSMap version 41 2024-03-08T21:14:10.211 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.231+0000 7ff2f99c3740 10 client.16593 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-08T21:14:10.212 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.231+0000 7ff2f99c3740 20 client.16593 populate_metadata read hostname 'smithi107' 2024-03-08T21:14:10.212 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.231+0000 7ff2f99c3740 1 --2- 172.21.15.107:0/3080145199 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x56181484e4a0 0x561814850890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:14:10.212 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.231+0000 7ff2f99c3740 4 client.16593 mds_command: new command op to 34475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-08T21:14:10.212 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.231+0000 7ff2f99c3740 1 -- 172.21.15.107:0/3080145199 --> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56181479fc60 con 0x56181484e4a0 2024-03-08T21:14:10.212 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.232+0000 7ff2efe00640 1 --2- 172.21.15.107:0/3080145199 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x56181484e4a0 0x561814850890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:14:10.212 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.232+0000 7ff2d9600640 10 client.16593 ms_handle_connect on v2:172.21.15.149:6838/55290596 2024-03-08T21:14:10.212 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.232+0000 7ff2efe00640 1 --2- 172.21.15.107:0/3080145199 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x56181484e4a0 0x561814850890 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d275ed3aae7eeac7 server_cookie=45dd0756cca6b0c3 in_seq=0 out_seq=0 2024-03-08T21:14:10.213 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.233+0000 7ff2d9600640 1 -- 172.21.15.107:0/3080145199 <== mds.0 v2:172.21.15.149:6838/55290596 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56181479fc60 con 0x56181484e4a0 2024-03-08T21:14:10.213 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.233+0000 7ff2d9600640 10 client.16593 handle_command_reply: tid=0 2024-03-08T21:14:10.213 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.233+0000 7ff2f99c3740 1 client.16593 shutdown 2024-03-08T21:14:10.213 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.234+0000 7ff2f99c3740 1 -- 172.21.15.107:0/3080145199 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5618148357a0 msgr2=0x56181483b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:14:10.213 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.234+0000 7ff2f99c3740 1 --2- 172.21.15.107:0/3080145199 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5618148357a0 0x56181483b350 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7ff2f006a740 tx=0x7ff2dc02c9f0 comp rx=0 tx=0).stop 2024-03-08T21:14:10.213 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.234+0000 7ff2f99c3740 1 -- 172.21.15.107:0/3080145199 shutdown_connections 2024-03-08T21:14:10.214 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.234+0000 7ff2f99c3740 1 --2- 172.21.15.107:0/3080145199 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x56181484e4a0 0x561814850890 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:14:10.214 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.234+0000 7ff2f99c3740 1 --2- 172.21.15.107:0/3080145199 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56181483e570 0x56181483ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:14:10.214 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.234+0000 7ff2f99c3740 1 --2- 172.21.15.107:0/3080145199 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5618148357a0 0x56181483b350 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:14:10.214 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.234+0000 7ff2f99c3740 1 --2- 172.21.15.107:0/3080145199 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x561814834e80 0x56181483dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:14:10.214 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.234+0000 7ff2d9600640 0 client.16593 ms_handle_reset on v2:172.21.15.149:6838/55290596 2024-03-08T21:14:10.214 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.234+0000 7ff2f99c3740 1 -- 172.21.15.107:0/3080145199 >> 172.21.15.107:0/3080145199 conn(0x56181482f7a0 msgr2=0x5618148314f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:14:10.214 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.234+0000 7ff2f99c3740 1 -- 172.21.15.107:0/3080145199 shutdown_connections 2024-03-08T21:14:10.214 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.234+0000 7ff2f99c3740 1 -- 172.21.15.107:0/3080145199 wait complete. 2024-03-08T21:14:10.214 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.234+0000 7ff2f99c3740 20 client.16593 trim_cache size 0 max 16384 2024-03-08T21:14:10.231 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.253+0000 7ff2f99c3740 1 Processor -- start 2024-03-08T21:14:10.232 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.253+0000 7ff2f99c3740 1 -- start start 2024-03-08T21:14:10.232 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.254+0000 7ff2f99c3740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561814834e80 0x56181483da80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:14:10.232 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.254+0000 7ff2f99c3740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5618148357a0 0x56181483dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:14:10.232 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.254+0000 7ff2f99c3740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56181483e570 0x5618148385b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:14:10.232 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.254+0000 7ff2f99c3740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2f01b0790 con 0x561814834e80 2024-03-08T21:14:10.232 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.254+0000 7ff2f99c3740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2f01ae7b0 con 0x56181483e570 2024-03-08T21:14:10.232 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.254+0000 7ff2f99c3740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2f01b0130 con 0x5618148357a0 2024-03-08T21:14:10.232 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.254+0000 7ff2efe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561814834e80 0x56181483da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:14:10.232 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.254+0000 7ff2efe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561814834e80 0x56181483da80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46810/0 (socket says 172.21.15.107:46810) 2024-03-08T21:14:10.232 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.254+0000 7ff2efe00640 1 -- 172.21.15.107:0/2022557053 learned_addr learned my addr 172.21.15.107:0/2022557053 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:14:10.232 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.254+0000 7ff2f4c00640 1 --2- 172.21.15.107:0/2022557053 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56181483e570 0x5618148385b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:14:10.232 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.254+0000 7ff2ef400640 1 --2- 172.21.15.107:0/2022557053 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5618148357a0 0x56181483dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:14:10.233 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.255+0000 7ff2efe00640 1 -- 172.21.15.107:0/2022557053 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5618148357a0 msgr2=0x56181483dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:14:10.233 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.255+0000 7ff2efe00640 1 --2- 172.21.15.107:0/2022557053 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5618148357a0 0x56181483dfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:14:10.233 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.255+0000 7ff2efe00640 1 -- 172.21.15.107:0/2022557053 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56181483e570 msgr2=0x5618148385b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:14:10.233 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.255+0000 7ff2efe00640 1 --2- 172.21.15.107:0/2022557053 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56181483e570 0x5618148385b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:14:10.233 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.255+0000 7ff2efe00640 1 -- 172.21.15.107:0/2022557053 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561814840cd0 con 0x561814834e80 2024-03-08T21:14:10.233 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.255+0000 7ff2efe00640 1 --2- 172.21.15.107:0/2022557053 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561814834e80 0x56181483da80 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7ff2e0098240 tx=0x7ff2e00d2d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9405f34c553adcfe server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:14:10.233 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.255+0000 7ff2dbe00640 1 -- 172.21.15.107:0/2022557053 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2e0097b10 con 0x561814834e80 2024-03-08T21:14:10.233 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.255+0000 7ff2dbe00640 1 -- 172.21.15.107:0/2022557053 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7ff2e009bf70 con 0x561814834e80 2024-03-08T21:14:10.233 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.255+0000 7ff2dbe00640 1 -- 172.21.15.107:0/2022557053 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2e009b3b0 con 0x561814834e80 2024-03-08T21:14:10.236 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.256+0000 7ff2f99c3740 1 -- 172.21.15.107:0/2022557053 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561814834e80 msgr2=0x56181483da80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:14:10.237 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.256+0000 7ff2f99c3740 1 --2- 172.21.15.107:0/2022557053 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561814834e80 0x56181483da80 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7ff2e0098240 tx=0x7ff2e00d2d90 comp rx=0 tx=0).stop 2024-03-08T21:14:10.237 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.256+0000 7ff2f99c3740 1 -- 172.21.15.107:0/2022557053 shutdown_connections 2024-03-08T21:14:10.237 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.256+0000 7ff2f99c3740 1 --2- 172.21.15.107:0/2022557053 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x56181483e570 0x5618148385b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:14:10.237 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.256+0000 7ff2f99c3740 1 --2- 172.21.15.107:0/2022557053 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5618148357a0 0x56181483dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:14:10.237 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.256+0000 7ff2f99c3740 1 --2- 172.21.15.107:0/2022557053 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561814834e80 0x56181483da80 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:14:10.237 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.256+0000 7ff2f99c3740 1 -- 172.21.15.107:0/2022557053 >> 172.21.15.107:0/2022557053 conn(0x56181482f7a0 msgr2=0x56181482fb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:14:10.237 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.256+0000 7ff2f99c3740 1 -- 172.21.15.107:0/2022557053 shutdown_connections 2024-03-08T21:14:10.237 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.256+0000 7ff2f99c3740 1 -- 172.21.15.107:0/2022557053 wait complete. 2024-03-08T21:14:10.237 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.257+0000 7ff2f99c3740 1 -- start start 2024-03-08T21:14:10.237 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.257+0000 7ff2f99c3740 1 Processor -- start 2024-03-08T21:14:10.237 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.257+0000 7ff2f99c3740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561814834e80 0x561814845960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:14:10.238 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.257+0000 7ff2f99c3740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5618148357a0 0x561814843d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:14:10.238 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.257+0000 7ff2f99c3740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56181483e570 0x561814844260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:14:10.238 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.257+0000 7ff2f99c3740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2f01af470 con 0x561814834e80 2024-03-08T21:14:10.238 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.257+0000 7ff2f99c3740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2f01aee10 con 0x5618148357a0 2024-03-08T21:14:10.238 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.257+0000 7ff2f99c3740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2f01afad0 con 0x56181483e570 2024-03-08T21:14:10.238 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.257+0000 7ff2f99c3740 10 client.0 _ll_register_callbacks cb 0x7ff2e00d2c60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T21:14:10.238 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.258+0000 7ff2efe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561814834e80 0x561814845960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:14:10.238 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.258+0000 7ff2efe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561814834e80 0x561814845960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46820/0 (socket says 172.21.15.107:46820) 2024-03-08T21:14:10.238 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.258+0000 7ff2efe00640 1 -- 172.21.15.107:0/764122070 learned_addr learned my addr 172.21.15.107:0/764122070 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:14:10.238 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.258+0000 7ff2ef400640 1 --2- 172.21.15.107:0/764122070 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5618148357a0 0x561814843d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:14:10.239 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.258+0000 7ff2efe00640 1 -- 172.21.15.107:0/764122070 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56181483e570 msgr2=0x561814844260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:14:10.239 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.258+0000 7ff2efe00640 1 --2- 172.21.15.107:0/764122070 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56181483e570 0x561814844260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:14:10.239 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.258+0000 7ff2efe00640 1 -- 172.21.15.107:0/764122070 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5618148357a0 msgr2=0x561814843d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:14:10.239 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.258+0000 7ff2efe00640 1 --2- 172.21.15.107:0/764122070 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5618148357a0 0x561814843d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:14:10.239 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.258+0000 7ff2efe00640 1 -- 172.21.15.107:0/764122070 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561814844a90 con 0x561814834e80 2024-03-08T21:14:10.239 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.258+0000 7ff2efe00640 1 --2- 172.21.15.107:0/764122070 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561814834e80 0x561814845960 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7ff2e00a3ae0 tx=0x7ff2e00974d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a2171000a7efca3 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:14:10.239 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.258+0000 7ff2d9600640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-03-08T21:14:10.239 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.259+0000 7ff2d9600640 1 -- 172.21.15.107:0/764122070 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2e009b640 con 0x561814834e80 2024-03-08T21:14:10.239 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.259+0000 7ff2d9600640 1 -- 172.21.15.107:0/764122070 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 8629+0+0 (secure 0 0 0) 0x7ff2e0097ee0 con 0x561814834e80 2024-03-08T21:14:10.239 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.259+0000 7ff2f99c3740 1 -- 172.21.15.107:0/764122070 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff2e009bf70 con 0x561814834e80 2024-03-08T21:14:10.239 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.259+0000 7ff2d9600640 1 -- 172.21.15.107:0/764122070 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff2e009b3b0 con 0x561814834e80 2024-03-08T21:14:10.239 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.259+0000 7ff2f99c3740 10 client.16599 fetch_fsmap learned FSMap version 41 2024-03-08T21:14:10.240 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.259+0000 7ff2f99c3740 1 -- 172.21.15.107:0/764122070 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x5618148a5a40 con 0x561814834e80 2024-03-08T21:14:10.240 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.259+0000 7ff2d9600640 1 -- 172.21.15.107:0/764122070 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7ff2e009bf70 con 0x561814834e80 2024-03-08T21:14:10.240 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.259+0000 7ff2d9600640 1 -- 172.21.15.107:0/764122070 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 41) v1 ==== 7307+0+0 (secure 0 0 0) 0x7ff2e00a6030 con 0x561814834e80 2024-03-08T21:14:10.240 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.259+0000 7ff2f99c3740 10 client.16599 fetch_fsmap finished waiting for FSMap version 41 2024-03-08T21:14:10.240 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.259+0000 7ff2f99c3740 10 client.16599 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-08T21:14:10.240 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.259+0000 7ff2f99c3740 20 client.16599 populate_metadata read hostname 'smithi107' 2024-03-08T21:14:10.240 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.259+0000 7ff2f99c3740 1 --2- 172.21.15.107:0/764122070 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x5618148a5c70 0x5618148a8060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:14:10.240 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.259+0000 7ff2f99c3740 4 client.16599 mds_command: new command op to 34475 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-08T21:14:10.240 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.259+0000 7ff2f99c3740 1 -- 172.21.15.107:0/764122070 --> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x56181471e5d0 con 0x5618148a5c70 2024-03-08T21:14:10.240 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.260+0000 7ff2ef400640 1 --2- 172.21.15.107:0/764122070 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x5618148a5c70 0x5618148a8060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:14:10.240 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.260+0000 7ff2d9600640 10 client.16599 ms_handle_connect on v2:172.21.15.149:6838/55290596 2024-03-08T21:14:10.240 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.260+0000 7ff2ef400640 1 --2- 172.21.15.107:0/764122070 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x5618148a5c70 0x5618148a8060 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e744db1316265d4 server_cookie=5e5835dc6910ce9c in_seq=0 out_seq=0 2024-03-08T21:14:10.255 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.278+0000 7ff2d9600640 1 -- 172.21.15.107:0/764122070 <== mds.0 v2:172.21.15.149:6838/55290596 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x56181471e5d0 con 0x5618148a5c70 2024-03-08T21:14:10.255 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.278+0000 7ff2d9600640 10 client.16599 handle_command_reply: tid=0 2024-03-08T21:14:10.255 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.278+0000 7ff2f99c3740 1 client.16599 shutdown 2024-03-08T21:14:10.255 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.278+0000 7ff2f99c3740 1 -- 172.21.15.107:0/764122070 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561814834e80 msgr2=0x561814845960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:14:10.255 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.278+0000 7ff2f99c3740 1 --2- 172.21.15.107:0/764122070 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561814834e80 0x561814845960 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7ff2e00a3ae0 tx=0x7ff2e00974d0 comp rx=0 tx=0).stop 2024-03-08T21:14:10.255 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.278+0000 7ff2f99c3740 1 -- 172.21.15.107:0/764122070 shutdown_connections 2024-03-08T21:14:10.255 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.278+0000 7ff2f99c3740 1 --2- 172.21.15.107:0/764122070 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x56181483e570 0x561814844260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:14:10.255 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.278+0000 7ff2f99c3740 1 --2- 172.21.15.107:0/764122070 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x5618148a5c70 0x5618148a8060 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:14:10.255 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.278+0000 7ff2f99c3740 1 --2- 172.21.15.107:0/764122070 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5618148357a0 0x561814843d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:14:10.256 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.278+0000 7ff2f99c3740 1 --2- 172.21.15.107:0/764122070 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561814834e80 0x561814845960 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:14:10.256 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.278+0000 7ff2d9600640 0 client.16599 ms_handle_reset on v2:172.21.15.149:6838/55290596 2024-03-08T21:14:10.256 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.278+0000 7ff2f99c3740 1 -- 172.21.15.107:0/764122070 >> 172.21.15.107:0/764122070 conn(0x56181482f7a0 msgr2=0x56181482df70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:14:10.256 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.278+0000 7ff2f99c3740 1 -- 172.21.15.107:0/764122070 shutdown_connections 2024-03-08T21:14:10.256 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.278+0000 7ff2f99c3740 1 -- 172.21.15.107:0/764122070 wait complete. 2024-03-08T21:14:10.256 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.279+0000 7ff2f99c3740 20 client.16599 trim_cache size 0 max 16384 2024-03-08T21:14:10.256 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:{ 2024-03-08T21:14:10.256 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "message": "", 2024-03-08T21:14:10.256 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "return_code": 0 2024-03-08T21:14:10.256 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:} 2024-03-08T21:14:10.257 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.279+0000 7ff2f5600640 1 -- 172.21.15.107:0/400407988 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7ff2d40a2960 msgr2=0x7ff2d40a4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:14:10.257 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.279+0000 7ff2f5600640 1 --2- 172.21.15.107:0/400407988 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7ff2d40a2960 0x7ff2d40a4e20 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7ff2e80092f0 tx=0x7ff2e8009280 comp rx=0 tx=0).stop 2024-03-08T21:14:10.257 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.279+0000 7ff2f5600640 1 -- 172.21.15.107:0/400407988 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff2f0101340 msgr2=0x7ff2f019f780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:14:10.257 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.279+0000 7ff2f5600640 1 --2- 172.21.15.107:0/400407988 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff2f0101340 0x7ff2f019f780 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7ff2e000e9f0 tx=0x7ff2e000eeb0 comp rx=0 tx=0).stop 2024-03-08T21:14:10.257 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.280+0000 7ff2f5600640 1 -- 172.21.15.107:0/400407988 shutdown_connections 2024-03-08T21:14:10.258 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.280+0000 7ff2f5600640 1 --2- 172.21.15.107:0/400407988 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7ff2d40a2960 0x7ff2d40a4e20 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:14:10.258 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.280+0000 7ff2f5600640 1 --2- 172.21.15.107:0/400407988 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7ff2f0199980 0x7ff2f0199e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:14:10.258 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.280+0000 7ff2f5600640 1 --2- 172.21.15.107:0/400407988 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff2f0106410 0x7ff2f019fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:14:10.258 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.280+0000 7ff2f5600640 1 --2- 172.21.15.107:0/400407988 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff2f0101340 0x7ff2f019f780 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:14:10.258 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.280+0000 7ff2f5600640 1 -- 172.21.15.107:0/400407988 >> 172.21.15.107:0/400407988 conn(0x7ff2f00fcc40 msgr2=0x7ff2f00fefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:14:10.258 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.280+0000 7ff2f5600640 1 -- 172.21.15.107:0/400407988 shutdown_connections 2024-03-08T21:14:10.258 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:10.280+0000 7ff2f5600640 1 -- 172.21.15.107:0/400407988 wait complete. 2024-03-08T21:14:10.961 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:14:10 smithi107 ceph-mon[31381]: pgmap v431: 97 pgs: 97 active+clean; 9.5 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 104 KiB/s wr, 30 op/s 2024-03-08T21:14:10.961 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:14:10 smithi107 ceph-mon[31381]: osdmap e110: 12 total, 12 up, 12 in 2024-03-08T21:14:10.961 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:14:10 smithi107 ceph-mon[31381]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T21:14:10.961 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:14:10 smithi107 ceph-mon[31381]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T21:14:10.961 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:14:10 smithi107 ceph-mon[31381]: from='mgr.15375 ' entity='mgr.x' 2024-03-08T21:14:11.009 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:14:10 smithi149 ceph-mon[35042]: pgmap v431: 97 pgs: 97 active+clean; 9.5 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 104 KiB/s wr, 30 op/s 2024-03-08T21:14:11.009 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:14:10 smithi149 ceph-mon[35042]: osdmap e110: 12 total, 12 up, 12 in 2024-03-08T21:14:11.009 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:14:10 smithi149 ceph-mon[35042]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T21:14:11.009 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:14:10 smithi149 ceph-mon[35042]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T21:14:11.009 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:14:10 smithi149 ceph-mon[35042]: from='mgr.15375 ' entity='mgr.x' 2024-03-08T21:14:11.122 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:14:10 smithi204 ceph-mon[34935]: pgmap v431: 97 pgs: 97 active+clean; 9.5 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 104 KiB/s wr, 30 op/s 2024-03-08T21:14:11.122 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:14:10 smithi204 ceph-mon[34935]: osdmap e110: 12 total, 12 up, 12 in 2024-03-08T21:14:11.122 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:14:10 smithi204 ceph-mon[34935]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T21:14:11.123 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:14:10 smithi204 ceph-mon[34935]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T21:14:11.123 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:14:10 smithi204 ceph-mon[34935]: from='mgr.15375 ' entity='mgr.x' 2024-03-08T21:14:12.961 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:14:12 smithi107 ceph-mon[31381]: pgmap v433: 97 pgs: 97 active+clean; 9.5 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 124 KiB/s wr, 30 op/s 2024-03-08T21:14:13.009 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:14:12 smithi149 ceph-mon[35042]: pgmap v433: 97 pgs: 97 active+clean; 9.5 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 124 KiB/s wr, 30 op/s 2024-03-08T21:14:13.122 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:14:12 smithi204 ceph-mon[34935]: pgmap v433: 97 pgs: 97 active+clean; 9.5 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 124 KiB/s wr, 30 op/s 2024-03-08T21:14:14.961 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:14:14 smithi107 ceph-mon[31381]: pgmap v434: 97 pgs: 97 active+clean; 9.5 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 124 KiB/s wr, 30 op/s 2024-03-08T21:14:15.008 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:14:14 smithi149 ceph-mon[35042]: pgmap v434: 97 pgs: 97 active+clean; 9.5 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 124 KiB/s wr, 30 op/s 2024-03-08T21:14:15.122 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:14:14 smithi204 ceph-mon[34935]: pgmap v434: 97 pgs: 97 active+clean; 9.5 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 124 KiB/s wr, 30 op/s 2024-03-08T21:14:16.961 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:14:16 smithi107 ceph-mon[31381]: pgmap v435: 97 pgs: 97 active+clean; 9.5 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 3.7 KiB/s wr, 13 op/s 2024-03-08T21:14:17.009 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:14:16 smithi149 ceph-mon[35042]: pgmap v435: 97 pgs: 97 active+clean; 9.5 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 3.7 KiB/s wr, 13 op/s 2024-03-08T21:14:17.122 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:14:16 smithi204 ceph-mon[34935]: pgmap v435: 97 pgs: 97 active+clean; 9.5 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 3.7 KiB/s wr, 13 op/s 2024-03-08T21:14:18.961 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:14:18 smithi107 ceph-mon[31381]: pgmap v436: 97 pgs: 97 active+clean; 9.5 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 3.7 KiB/s wr, 13 op/s 2024-03-08T21:14:19.008 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:14:18 smithi149 ceph-mon[35042]: pgmap v436: 97 pgs: 97 active+clean; 9.5 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 3.7 KiB/s wr, 13 op/s 2024-03-08T21:14:19.122 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:14:18 smithi204 ceph-mon[34935]: pgmap v436: 97 pgs: 97 active+clean; 9.5 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 3.7 KiB/s wr, 13 op/s 2024-03-08T21:14:20.622 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:14:20 smithi204 ceph-mon[34935]: pgmap v437: 97 pgs: 97 active+clean; 9.5 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 3.6 KiB/s wr, 13 op/s 2024-03-08T21:14:20.622 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:14:20 smithi204 ceph-mon[34935]: osdmap e111: 12 total, 12 up, 12 in 2024-03-08T21:14:20.711 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:14:20 smithi107 ceph-mon[31381]: pgmap v437: 97 pgs: 97 active+clean; 9.5 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 3.6 KiB/s wr, 13 op/s 2024-03-08T21:14:20.711 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:14:20 smithi107 ceph-mon[31381]: osdmap e111: 12 total, 12 up, 12 in 2024-03-08T21:14:20.759 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:14:20 smithi149 ceph-mon[35042]: pgmap v437: 97 pgs: 97 active+clean; 9.5 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 3.6 KiB/s wr, 13 op/s 2024-03-08T21:14:20.759 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:14:20 smithi149 ceph-mon[35042]: osdmap e111: 12 total, 12 up, 12 in 2024-03-08T21:14:22.622 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:14:22 smithi204 ceph-mon[34935]: pgmap v439: 97 pgs: 97 active+clean; 9.5 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 3.6 KiB/s wr, 13 op/s 2024-03-08T21:14:22.711 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:14:22 smithi107 ceph-mon[31381]: pgmap v439: 97 pgs: 97 active+clean; 9.5 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 3.6 KiB/s wr, 13 op/s 2024-03-08T21:14:22.759 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:14:22 smithi149 ceph-mon[35042]: pgmap v439: 97 pgs: 97 active+clean; 9.5 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 3.6 KiB/s wr, 13 op/s 2024-03-08T21:14:23.413 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.436+0000 7fb102c00640 1 -- 172.21.15.107:0/1505372897 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb0fc106410 msgr2=0x7fb0fc10dfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:14:23.413 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.436+0000 7fb102c00640 1 --2- 172.21.15.107:0/1505372897 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb0fc106410 0x7fb0fc10dfe0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fb0f4009ca0 tx=0x7fb0f402c0f0 comp rx=0 tx=0).stop 2024-03-08T21:14:23.413 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.436+0000 7fb102c00640 1 -- 172.21.15.107:0/1505372897 shutdown_connections 2024-03-08T21:14:23.413 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.436+0000 7fb102c00640 1 --2- 172.21.15.107:0/1505372897 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb0fc106410 0x7fb0fc10dfe0 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:14:23.413 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.436+0000 7fb102c00640 1 --2- 172.21.15.107:0/1505372897 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb0fc101cf0 0x7fb0fc105d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:14:23.413 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.436+0000 7fb102c00640 1 --2- 172.21.15.107:0/1505372897 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb0fc101340 0x7fb0fc101720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:14:23.413 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.436+0000 7fb102c00640 1 -- 172.21.15.107:0/1505372897 >> 172.21.15.107:0/1505372897 conn(0x7fb0fc0fcc40 msgr2=0x7fb0fc0ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:14:23.413 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.436+0000 7fb102c00640 1 -- 172.21.15.107:0/1505372897 shutdown_connections 2024-03-08T21:14:23.413 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.436+0000 7fb102c00640 1 -- 172.21.15.107:0/1505372897 wait complete. 2024-03-08T21:14:23.415 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.437+0000 7fb102c00640 1 Processor -- start 2024-03-08T21:14:23.415 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.438+0000 7fb102c00640 1 -- start start 2024-03-08T21:14:23.415 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.438+0000 7fb102c00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb0fc101340 0x7fb0fc19f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:14:23.415 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.438+0000 7fb102c00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb0fc106410 0x7fb0fc19fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:14:23.415 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.438+0000 7fb102c00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb0fc199920 0x7fb0fc199dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:14:23.416 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.438+0000 7fb102c00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0fc111380 con 0x7fb0fc106410 2024-03-08T21:14:23.416 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.438+0000 7fb102c00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0fc111200 con 0x7fb0fc101340 2024-03-08T21:14:23.416 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.438+0000 7fb102c00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0fc111500 con 0x7fb0fc199920 2024-03-08T21:14:23.416 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.438+0000 7fb100e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb0fc106410 0x7fb0fc19fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:14:23.416 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.438+0000 7fb100e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb0fc106410 0x7fb0fc19fcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:44960/0 (socket says 172.21.15.107:44960) 2024-03-08T21:14:23.416 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.438+0000 7fb100e00640 1 -- 172.21.15.107:0/703781157 learned_addr learned my addr 172.21.15.107:0/703781157 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:14:23.416 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.439+0000 7fb101800640 1 --2- 172.21.15.107:0/703781157 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb0fc101340 0x7fb0fc19f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:14:23.416 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.439+0000 7fb102200640 1 --2- 172.21.15.107:0/703781157 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb0fc199920 0x7fb0fc199dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:14:23.416 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.439+0000 7fb100e00640 1 -- 172.21.15.107:0/703781157 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb0fc199920 msgr2=0x7fb0fc199dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:14:23.416 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.439+0000 7fb100e00640 1 --2- 172.21.15.107:0/703781157 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb0fc199920 0x7fb0fc199dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:14:23.416 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.439+0000 7fb100e00640 1 -- 172.21.15.107:0/703781157 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb0fc101340 msgr2=0x7fb0fc19f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:14:23.416 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.439+0000 7fb100e00640 1 --2- 172.21.15.107:0/703781157 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb0fc101340 0x7fb0fc19f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:14:23.416 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.439+0000 7fb100e00640 1 -- 172.21.15.107:0/703781157 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0fc19a440 con 0x7fb0fc106410 2024-03-08T21:14:23.416 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.439+0000 7fb102200640 1 --2- 172.21.15.107:0/703781157 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb0fc199920 0x7fb0fc199dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T21:14:23.417 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.439+0000 7fb101800640 1 --2- 172.21.15.107:0/703781157 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb0fc101340 0x7fb0fc19f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T21:14:23.417 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.439+0000 7fb100e00640 1 --2- 172.21.15.107:0/703781157 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb0fc106410 0x7fb0fc19fcb0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fb0e800e9f0 tx=0x7fb0e800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:14:23.417 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.439+0000 7fb0fa000640 1 -- 172.21.15.107:0/703781157 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0e800ccc0 con 0x7fb0fc106410 2024-03-08T21:14:23.417 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.439+0000 7fb0fa000640 1 -- 172.21.15.107:0/703781157 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fb0e80054d0 con 0x7fb0fc106410 2024-03-08T21:14:23.417 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.439+0000 7fb102c00640 1 -- 172.21.15.107:0/703781157 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0fc19a730 con 0x7fb0fc106410 2024-03-08T21:14:23.418 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.440+0000 7fb0fa000640 1 -- 172.21.15.107:0/703781157 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0e8010430 con 0x7fb0fc106410 2024-03-08T21:14:23.418 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.440+0000 7fb102c00640 1 -- 172.21.15.107:0/703781157 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0fc1a6930 con 0x7fb0fc106410 2024-03-08T21:14:23.420 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.444+0000 7fb106f88740 1 Processor -- start 2024-03-08T21:14:23.420 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.444+0000 7fb106f88740 1 -- start start 2024-03-08T21:14:23.421 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.444+0000 7fb106f88740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55923b889e80 0x55923b88a260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:14:23.421 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.444+0000 7fb106f88740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55923b88a7a0 0x55923b893030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:14:23.421 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.445+0000 7fb106f88740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55923b893570 0x55923b895960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:14:23.421 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.445+0000 7fb106f88740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55923b7ed3d0 con 0x55923b893570 2024-03-08T21:14:23.422 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.445+0000 7fb106f88740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55923b7966b0 con 0x55923b889e80 2024-03-08T21:14:23.422 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.445+0000 7fb106f88740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55923b7e9470 con 0x55923b88a7a0 2024-03-08T21:14:23.422 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.445+0000 7fb0fa000640 1 -- 172.21.15.107:0/703781157 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) v1 ==== 137422+0+0 (secure 0 0 0) 0x7fb0e8005020 con 0x7fb0fc106410 2024-03-08T21:14:23.422 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.445+0000 7fb100e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55923b893570 0x55923b895960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:14:23.422 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.445+0000 7fb100e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55923b893570 0x55923b895960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:44970/0 (socket says 172.21.15.107:44970) 2024-03-08T21:14:23.422 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.445+0000 7fb100e00640 1 -- 172.21.15.107:0/3386013404 learned_addr learned my addr 172.21.15.107:0/3386013404 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:14:23.422 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.445+0000 7fb102200640 1 --2- 172.21.15.107:0/3386013404 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55923b889e80 0x55923b88a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:14:23.422 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.445+0000 7fb101800640 1 --2- 172.21.15.107:0/3386013404 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55923b88a7a0 0x55923b893030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:14:23.423 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.446+0000 7fb100e00640 1 -- 172.21.15.107:0/3386013404 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55923b88a7a0 msgr2=0x55923b893030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:14:23.423 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.446+0000 7fb100e00640 1 --2- 172.21.15.107:0/3386013404 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55923b88a7a0 0x55923b893030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:14:23.424 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.446+0000 7fb100e00640 1 -- 172.21.15.107:0/3386013404 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55923b889e80 msgr2=0x55923b88a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:14:23.424 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.446+0000 7fb100e00640 1 --2- 172.21.15.107:0/3386013404 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55923b889e80 0x55923b88a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:14:23.424 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.446+0000 7fb100e00640 1 -- 172.21.15.107:0/3386013404 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55923b7bcb50 con 0x55923b893570 2024-03-08T21:14:23.424 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.446+0000 7fb100e00640 1 --2- 172.21.15.107:0/3386013404 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55923b893570 0x55923b895960 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7fb0e80cc9d0 tx=0x7fb0e80d74b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5a80082fad5f664 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:14:23.424 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.446+0000 7fb0eea00640 1 -- 172.21.15.107:0/3386013404 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0e80d1310 con 0x55923b893570 2024-03-08T21:14:23.424 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.446+0000 7fb0fa000640 1 --2- 172.21.15.107:0/703781157 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7fb0e40a2960 0x7fb0e40a4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:14:23.424 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.446+0000 7fb0eea00640 1 -- 172.21.15.107:0/3386013404 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fb0e80d14b0 con 0x55923b893570 2024-03-08T21:14:23.424 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.446+0000 7fb0eea00640 1 -- 172.21.15.107:0/3386013404 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0e8097c10 con 0x55923b893570 2024-03-08T21:14:23.424 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.447+0000 7fb102200640 1 --2- 172.21.15.107:0/703781157 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7fb0e40a2960 0x7fb0e40a4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:14:23.424 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.447+0000 7fb0fa000640 1 -- 172.21.15.107:0/703781157 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fb0e80cf3b0 con 0x7fb0fc106410 2024-03-08T21:14:23.424 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.447+0000 7fb106f88740 1 -- 172.21.15.107:0/3386013404 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55923b893570 msgr2=0x55923b895960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:14:23.424 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.447+0000 7fb106f88740 1 --2- 172.21.15.107:0/3386013404 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55923b893570 0x55923b895960 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7fb0e80cc9d0 tx=0x7fb0e80d74b0 comp rx=0 tx=0).stop 2024-03-08T21:14:23.428 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.448+0000 7fb106f88740 1 -- 172.21.15.107:0/3386013404 shutdown_connections 2024-03-08T21:14:23.428 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.448+0000 7fb106f88740 1 --2- 172.21.15.107:0/3386013404 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55923b893570 0x55923b895960 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:14:23.428 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.448+0000 7fb106f88740 1 --2- 172.21.15.107:0/3386013404 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55923b88a7a0 0x55923b893030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:14:23.428 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.448+0000 7fb106f88740 1 --2- 172.21.15.107:0/3386013404 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55923b889e80 0x55923b88a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:14:23.428 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.448+0000 7fb106f88740 1 -- 172.21.15.107:0/3386013404 >> 172.21.15.107:0/3386013404 conn(0x55923b8847a0 msgr2=0x55923b884b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:14:23.428 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.448+0000 7fb102200640 1 --2- 172.21.15.107:0/703781157 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7fb0e40a2960 0x7fb0e40a4e20 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fb0f4009430 tx=0x7fb0f4004a80 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:14:23.428 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.448+0000 7fb106f88740 1 -- 172.21.15.107:0/3386013404 shutdown_connections 2024-03-08T21:14:23.428 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.448+0000 7fb106f88740 1 -- 172.21.15.107:0/3386013404 wait complete. 2024-03-08T21:14:23.429 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.448+0000 7fb106f88740 1 -- start start 2024-03-08T21:14:23.429 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.448+0000 7fb106f88740 1 Processor -- start 2024-03-08T21:14:23.429 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.449+0000 7fb106f88740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55923b889e80 0x55923b892e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:14:23.429 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.449+0000 7fb106f88740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55923b88a7a0 0x55923b890190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:14:23.429 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.449+0000 7fb106f88740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55923b893570 0x55923b8906d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:14:23.429 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.449+0000 7fb106f88740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55923b8961a0 con 0x55923b889e80 2024-03-08T21:14:23.429 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.449+0000 7fb106f88740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55923b7a99f0 con 0x55923b88a7a0 2024-03-08T21:14:23.429 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.449+0000 7fb106f88740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55923b896320 con 0x55923b893570 2024-03-08T21:14:23.429 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.449+0000 7fb106f88740 10 client.0 _ll_register_callbacks cb 0x55923b501a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T21:14:23.429 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.449+0000 7fb101800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55923b889e80 0x55923b892e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:14:23.429 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.449+0000 7fb101800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55923b889e80 0x55923b892e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:44972/0 (socket says 172.21.15.107:44972) 2024-03-08T21:14:23.429 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.449+0000 7fb101800640 1 -- 172.21.15.107:0/4206032063 learned_addr learned my addr 172.21.15.107:0/4206032063 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:14:23.429 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.450+0000 7fb102200640 1 --2- 172.21.15.107:0/4206032063 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55923b893570 0x55923b8906d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:14:23.429 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.450+0000 7fb100e00640 1 --2- 172.21.15.107:0/4206032063 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55923b88a7a0 0x55923b890190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:14:23.430 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.450+0000 7fb101800640 1 -- 172.21.15.107:0/4206032063 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55923b893570 msgr2=0x55923b8906d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:14:23.430 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.450+0000 7fb101800640 1 --2- 172.21.15.107:0/4206032063 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55923b893570 0x55923b8906d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:14:23.430 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.450+0000 7fb101800640 1 -- 172.21.15.107:0/4206032063 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55923b88a7a0 msgr2=0x55923b890190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:14:23.430 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.450+0000 7fb101800640 1 --2- 172.21.15.107:0/4206032063 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55923b88a7a0 0x55923b890190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:14:23.430 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.450+0000 7fb101800640 1 -- 172.21.15.107:0/4206032063 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55923b502210 con 0x55923b889e80 2024-03-08T21:14:23.430 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.450+0000 7fb101800640 1 --2- 172.21.15.107:0/4206032063 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55923b889e80 0x55923b892e00 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7fb0f0013710 tx=0x7fb0f0013b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89dca8eb443b6de2 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:14:23.430 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.450+0000 7fb0e3400640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-03-08T21:14:23.430 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.450+0000 7fb0e3400640 1 -- 172.21.15.107:0/4206032063 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0f0011dd0 con 0x55923b889e80 2024-03-08T21:14:23.430 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.450+0000 7fb0e3400640 1 -- 172.21.15.107:0/4206032063 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fb0f0004020 con 0x55923b889e80 2024-03-08T21:14:23.430 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.450+0000 7fb106f88740 1 -- 172.21.15.107:0/4206032063 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb0e80d14b0 con 0x55923b889e80 2024-03-08T21:14:23.430 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.451+0000 7fb0e3400640 1 -- 172.21.15.107:0/4206032063 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb0f0004220 con 0x55923b889e80 2024-03-08T21:14:23.430 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.451+0000 7fb106f88740 10 client.16611 fetch_fsmap learned FSMap version 41 2024-03-08T21:14:23.430 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.451+0000 7fb106f88740 1 -- 172.21.15.107:0/4206032063 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x55923b5ce620 con 0x55923b889e80 2024-03-08T21:14:23.431 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.451+0000 7fb0e3400640 1 -- 172.21.15.107:0/4206032063 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fb0e80d14b0 con 0x55923b889e80 2024-03-08T21:14:23.431 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.451+0000 7fb0e3400640 1 -- 172.21.15.107:0/4206032063 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 41) v1 ==== 7307+0+0 (secure 0 0 0) 0x7fb0f0021030 con 0x55923b889e80 2024-03-08T21:14:23.431 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.451+0000 7fb106f88740 10 client.16611 fetch_fsmap finished waiting for FSMap version 41 2024-03-08T21:14:23.431 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.451+0000 7fb106f88740 10 client.16611 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-08T21:14:23.431 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.451+0000 7fb106f88740 20 client.16611 populate_metadata read hostname 'smithi107' 2024-03-08T21:14:23.431 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.451+0000 7fb106f88740 1 --2- 172.21.15.107:0/4206032063 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55923b8a34d0 0x55923b8a5940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:14:23.431 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.451+0000 7fb106f88740 4 client.16611 mds_command: new command op to 34475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-08T21:14:23.431 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.451+0000 7fb106f88740 1 -- 172.21.15.107:0/4206032063 --> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55923b7fa910 con 0x55923b8a34d0 2024-03-08T21:14:23.432 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.452+0000 7fb100e00640 1 --2- 172.21.15.107:0/4206032063 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55923b8a34d0 0x55923b8a5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:14:23.433 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.453+0000 7fb100e00640 1 --2- 172.21.15.107:0/4206032063 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55923b8a34d0 0x55923b8a5940 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2586d252d699ad23 server_cookie=364e3183b5f19729 in_seq=0 out_seq=0 2024-03-08T21:14:23.433 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.453+0000 7fb0e3400640 10 client.16611 ms_handle_connect on v2:172.21.15.149:6838/55290596 2024-03-08T21:14:23.433 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.455+0000 7fb0e3400640 1 -- 172.21.15.107:0/4206032063 <== mds.0 v2:172.21.15.149:6838/55290596 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55923b7fa910 con 0x55923b8a34d0 2024-03-08T21:14:23.433 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.455+0000 7fb0e3400640 10 client.16611 handle_command_reply: tid=0 2024-03-08T21:14:23.433 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.455+0000 7fb106f88740 1 client.16611 shutdown 2024-03-08T21:14:23.433 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.455+0000 7fb106f88740 1 -- 172.21.15.107:0/4206032063 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55923b889e80 msgr2=0x55923b892e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:14:23.433 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.455+0000 7fb106f88740 1 --2- 172.21.15.107:0/4206032063 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55923b889e80 0x55923b892e00 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7fb0f0013710 tx=0x7fb0f0013b10 comp rx=0 tx=0).stop 2024-03-08T21:14:23.433 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.455+0000 7fb106f88740 1 -- 172.21.15.107:0/4206032063 shutdown_connections 2024-03-08T21:14:23.433 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.455+0000 7fb106f88740 1 --2- 172.21.15.107:0/4206032063 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55923b893570 0x55923b8906d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:14:23.433 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.455+0000 7fb106f88740 1 --2- 172.21.15.107:0/4206032063 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55923b8a34d0 0x55923b8a5940 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:14:23.433 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.455+0000 7fb106f88740 1 --2- 172.21.15.107:0/4206032063 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55923b88a7a0 0x55923b890190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:14:23.433 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.455+0000 7fb106f88740 1 --2- 172.21.15.107:0/4206032063 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55923b889e80 0x55923b892e00 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:14:23.433 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.455+0000 7fb106f88740 1 -- 172.21.15.107:0/4206032063 >> 172.21.15.107:0/4206032063 conn(0x55923b8847a0 msgr2=0x55923b8864f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:14:23.434 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.455+0000 7fb0e3400640 0 client.16611 ms_handle_reset on v2:172.21.15.149:6838/55290596 2024-03-08T21:14:23.434 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.455+0000 7fb106f88740 1 -- 172.21.15.107:0/4206032063 shutdown_connections 2024-03-08T21:14:23.434 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.455+0000 7fb106f88740 1 -- 172.21.15.107:0/4206032063 wait complete. 2024-03-08T21:14:23.434 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.455+0000 7fb106f88740 20 client.16611 trim_cache size 0 max 16384 2024-03-08T21:14:23.453 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.476+0000 7fb106f88740 1 Processor -- start 2024-03-08T21:14:23.453 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.476+0000 7fb106f88740 1 -- start start 2024-03-08T21:14:23.453 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.476+0000 7fb106f88740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55923b889e80 0x55923b8928c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:14:23.453 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.476+0000 7fb106f88740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55923b88a7a0 0x55923b892e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:14:23.453 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.476+0000 7fb106f88740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55923b893570 0x55923b89a550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:14:23.453 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.476+0000 7fb106f88740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0fc1b0770 con 0x55923b889e80 2024-03-08T21:14:23.453 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.476+0000 7fb106f88740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0fc1ae790 con 0x55923b88a7a0 2024-03-08T21:14:23.453 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.476+0000 7fb106f88740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0fc1b0110 con 0x55923b893570 2024-03-08T21:14:23.454 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.477+0000 7fb101800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55923b889e80 0x55923b8928c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:14:23.454 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.477+0000 7fb101800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55923b889e80 0x55923b8928c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:44976/0 (socket says 172.21.15.107:44976) 2024-03-08T21:14:23.454 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.477+0000 7fb101800640 1 -- 172.21.15.107:0/2369391745 learned_addr learned my addr 172.21.15.107:0/2369391745 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:14:23.460 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.477+0000 7fb102200640 1 --2- 172.21.15.107:0/2369391745 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55923b893570 0x55923b89a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:14:23.461 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.477+0000 7fb100e00640 1 --2- 172.21.15.107:0/2369391745 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55923b88a7a0 0x55923b892e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:14:23.461 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.477+0000 7fb101800640 1 -- 172.21.15.107:0/2369391745 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55923b893570 msgr2=0x55923b89a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:14:23.461 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.477+0000 7fb101800640 1 --2- 172.21.15.107:0/2369391745 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55923b893570 0x55923b89a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:14:23.461 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.477+0000 7fb101800640 1 -- 172.21.15.107:0/2369391745 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55923b88a7a0 msgr2=0x55923b892e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:14:23.461 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.477+0000 7fb101800640 1 --2- 172.21.15.107:0/2369391745 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55923b88a7a0 0x55923b892e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:14:23.461 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.477+0000 7fb101800640 1 -- 172.21.15.107:0/2369391745 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55923b8aab70 con 0x55923b889e80 2024-03-08T21:14:23.461 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.477+0000 7fb102200640 1 --2- 172.21.15.107:0/2369391745 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55923b893570 0x55923b89a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T21:14:23.461 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.478+0000 7fb101800640 1 --2- 172.21.15.107:0/2369391745 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55923b889e80 0x55923b8928c0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fb0f00136e0 tx=0x7fb0f0005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4cf70284a5d99ce server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:14:23.461 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.478+0000 7fb0ee000640 1 -- 172.21.15.107:0/2369391745 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0f002f070 con 0x55923b889e80 2024-03-08T21:14:23.461 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.478+0000 7fb0ee000640 1 -- 172.21.15.107:0/2369391745 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fb0f0022bf0 con 0x55923b889e80 2024-03-08T21:14:23.461 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.478+0000 7fb0ee000640 1 -- 172.21.15.107:0/2369391745 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0f002a850 con 0x55923b889e80 2024-03-08T21:14:23.461 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.479+0000 7fb106f88740 1 -- 172.21.15.107:0/2369391745 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55923b889e80 msgr2=0x55923b8928c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:14:23.462 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.479+0000 7fb106f88740 1 --2- 172.21.15.107:0/2369391745 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55923b889e80 0x55923b8928c0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fb0f00136e0 tx=0x7fb0f0005740 comp rx=0 tx=0).stop 2024-03-08T21:14:23.462 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.479+0000 7fb106f88740 1 -- 172.21.15.107:0/2369391745 shutdown_connections 2024-03-08T21:14:23.462 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.479+0000 7fb106f88740 1 --2- 172.21.15.107:0/2369391745 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55923b893570 0x55923b89a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:14:23.462 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.479+0000 7fb106f88740 1 --2- 172.21.15.107:0/2369391745 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55923b88a7a0 0x55923b892e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:14:23.462 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.479+0000 7fb106f88740 1 --2- 172.21.15.107:0/2369391745 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55923b889e80 0x55923b8928c0 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:14:23.462 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.479+0000 7fb106f88740 1 -- 172.21.15.107:0/2369391745 >> 172.21.15.107:0/2369391745 conn(0x55923b8847a0 msgr2=0x55923b88ea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:14:23.462 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.479+0000 7fb106f88740 1 -- 172.21.15.107:0/2369391745 shutdown_connections 2024-03-08T21:14:23.462 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.479+0000 7fb106f88740 1 -- 172.21.15.107:0/2369391745 wait complete. 2024-03-08T21:14:23.462 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.479+0000 7fb106f88740 1 -- start start 2024-03-08T21:14:23.462 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.480+0000 7fb106f88740 1 Processor -- start 2024-03-08T21:14:23.462 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.480+0000 7fb106f88740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55923b889e80 0x55923b89a310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:14:23.462 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.480+0000 7fb106f88740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55923b88a7a0 0x55923b89a850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:14:23.462 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.480+0000 7fb106f88740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55923b893570 0x55923b898760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:14:23.463 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.480+0000 7fb106f88740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0fc1af450 con 0x55923b889e80 2024-03-08T21:14:23.463 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.480+0000 7fb106f88740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0fc1aedf0 con 0x55923b88a7a0 2024-03-08T21:14:23.463 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.480+0000 7fb106f88740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0fc1afab0 con 0x55923b893570 2024-03-08T21:14:23.463 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.480+0000 7fb106f88740 10 client.0 _ll_register_callbacks cb 0x7fb0e80d2d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T21:14:23.463 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.480+0000 7fb101800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55923b889e80 0x55923b89a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:14:23.463 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.481+0000 7fb101800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55923b889e80 0x55923b89a310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:44982/0 (socket says 172.21.15.107:44982) 2024-03-08T21:14:23.463 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.481+0000 7fb101800640 1 -- 172.21.15.107:0/2809987731 learned_addr learned my addr 172.21.15.107:0/2809987731 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:14:23.463 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.481+0000 7fb100e00640 1 --2- 172.21.15.107:0/2809987731 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55923b88a7a0 0x55923b89a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:14:23.463 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.481+0000 7fb102200640 1 --2- 172.21.15.107:0/2809987731 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55923b893570 0x55923b898760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:14:23.463 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.481+0000 7fb101800640 1 -- 172.21.15.107:0/2809987731 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55923b893570 msgr2=0x55923b898760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:14:23.463 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.481+0000 7fb101800640 1 --2- 172.21.15.107:0/2809987731 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55923b893570 0x55923b898760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:14:23.463 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.481+0000 7fb101800640 1 -- 172.21.15.107:0/2809987731 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55923b88a7a0 msgr2=0x55923b89a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:14:23.463 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.481+0000 7fb101800640 1 --2- 172.21.15.107:0/2809987731 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55923b88a7a0 0x55923b89a850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:14:23.463 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.481+0000 7fb101800640 1 -- 172.21.15.107:0/2809987731 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55923b898f40 con 0x55923b889e80 2024-03-08T21:14:23.463 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.481+0000 7fb101800640 1 --2- 172.21.15.107:0/2809987731 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55923b889e80 0x55923b89a310 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fb0f0025040 tx=0x7fb0f00058d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0011efdd3315f17 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:14:23.464 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.481+0000 7fb0e3400640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-03-08T21:14:23.464 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.482+0000 7fb0e3400640 1 -- 172.21.15.107:0/2809987731 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0f002f040 con 0x55923b889e80 2024-03-08T21:14:23.464 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.482+0000 7fb0e3400640 1 -- 172.21.15.107:0/2809987731 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fb0f0011dd0 con 0x55923b889e80 2024-03-08T21:14:23.464 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.482+0000 7fb106f88740 1 -- 172.21.15.107:0/2809987731 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb0f0022bf0 con 0x55923b889e80 2024-03-08T21:14:23.464 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.482+0000 7fb0e3400640 1 -- 172.21.15.107:0/2809987731 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb0f0041050 con 0x55923b889e80 2024-03-08T21:14:23.464 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.482+0000 7fb106f88740 10 client.16617 fetch_fsmap learned FSMap version 41 2024-03-08T21:14:23.464 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.482+0000 7fb106f88740 1 -- 172.21.15.107:0/2809987731 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x55923b8fa4e0 con 0x55923b889e80 2024-03-08T21:14:23.464 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.482+0000 7fb0e3400640 1 -- 172.21.15.107:0/2809987731 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fb0f0022bf0 con 0x55923b889e80 2024-03-08T21:14:23.464 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.482+0000 7fb0e3400640 1 -- 172.21.15.107:0/2809987731 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 41) v1 ==== 7307+0+0 (secure 0 0 0) 0x7fb0f001a070 con 0x55923b889e80 2024-03-08T21:14:23.464 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.482+0000 7fb106f88740 10 client.16617 fetch_fsmap finished waiting for FSMap version 41 2024-03-08T21:14:23.464 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.482+0000 7fb106f88740 10 client.16617 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-08T21:14:23.464 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.482+0000 7fb106f88740 20 client.16617 populate_metadata read hostname 'smithi107' 2024-03-08T21:14:23.464 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.482+0000 7fb106f88740 1 --2- 172.21.15.107:0/2809987731 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55923b8faa90 0x55923b8fce80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:14:23.464 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.482+0000 7fb106f88740 4 client.16617 mds_command: new command op to 34475 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-08T21:14:23.464 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.483+0000 7fb106f88740 1 -- 172.21.15.107:0/2809987731 --> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55923b7f4c60 con 0x55923b8faa90 2024-03-08T21:14:23.465 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.483+0000 7fb100e00640 1 --2- 172.21.15.107:0/2809987731 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55923b8faa90 0x55923b8fce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:14:23.465 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.484+0000 7fb100e00640 1 --2- 172.21.15.107:0/2809987731 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55923b8faa90 0x55923b8fce80 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eaa3d9ecabe4c6ae server_cookie=40a8e7fe9f8b93fd in_seq=0 out_seq=0 2024-03-08T21:14:23.465 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.484+0000 7fb0e3400640 10 client.16617 ms_handle_connect on v2:172.21.15.149:6838/55290596 2024-03-08T21:14:23.471 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.494+0000 7fb0e3400640 1 -- 172.21.15.107:0/2809987731 <== mds.0 v2:172.21.15.149:6838/55290596 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55923b7f4c60 con 0x55923b8faa90 2024-03-08T21:14:23.471 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.494+0000 7fb0e3400640 10 client.16617 handle_command_reply: tid=0 2024-03-08T21:14:23.471 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.494+0000 7fb106f88740 1 client.16617 shutdown 2024-03-08T21:14:23.471 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.494+0000 7fb106f88740 1 -- 172.21.15.107:0/2809987731 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55923b889e80 msgr2=0x55923b89a310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:14:23.471 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.494+0000 7fb106f88740 1 --2- 172.21.15.107:0/2809987731 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55923b889e80 0x55923b89a310 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fb0f0025040 tx=0x7fb0f00058d0 comp rx=0 tx=0).stop 2024-03-08T21:14:23.471 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.494+0000 7fb106f88740 1 -- 172.21.15.107:0/2809987731 shutdown_connections 2024-03-08T21:14:23.471 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.494+0000 7fb106f88740 1 --2- 172.21.15.107:0/2809987731 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55923b893570 0x55923b898760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:14:23.472 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.494+0000 7fb106f88740 1 --2- 172.21.15.107:0/2809987731 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55923b8faa90 0x55923b8fce80 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:14:23.472 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.494+0000 7fb106f88740 1 --2- 172.21.15.107:0/2809987731 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55923b88a7a0 0x55923b89a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:14:23.472 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.494+0000 7fb106f88740 1 --2- 172.21.15.107:0/2809987731 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55923b889e80 0x55923b89a310 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:14:23.472 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.494+0000 7fb106f88740 1 -- 172.21.15.107:0/2809987731 >> 172.21.15.107:0/2809987731 conn(0x55923b8847a0 msgr2=0x55923b885bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:14:23.472 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.494+0000 7fb0e3400640 0 client.16617 ms_handle_reset on v2:172.21.15.149:6838/55290596 2024-03-08T21:14:23.472 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.494+0000 7fb106f88740 1 -- 172.21.15.107:0/2809987731 shutdown_connections 2024-03-08T21:14:23.472 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.494+0000 7fb106f88740 1 -- 172.21.15.107:0/2809987731 wait complete. 2024-03-08T21:14:23.472 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.495+0000 7fb106f88740 20 client.16617 trim_cache size 0 max 16384 2024-03-08T21:14:23.472 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:{ 2024-03-08T21:14:23.472 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "message": "", 2024-03-08T21:14:23.473 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "return_code": 0 2024-03-08T21:14:23.473 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:} 2024-03-08T21:14:23.473 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.495+0000 7fb102c00640 1 -- 172.21.15.107:0/703781157 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7fb0e40a2960 msgr2=0x7fb0e40a4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:14:23.473 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.495+0000 7fb102c00640 1 --2- 172.21.15.107:0/703781157 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7fb0e40a2960 0x7fb0e40a4e20 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fb0f4009430 tx=0x7fb0f4004a80 comp rx=0 tx=0).stop 2024-03-08T21:14:23.473 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.495+0000 7fb102c00640 1 -- 172.21.15.107:0/703781157 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb0fc106410 msgr2=0x7fb0fc19fcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:14:23.474 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.495+0000 7fb102c00640 1 --2- 172.21.15.107:0/703781157 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb0fc106410 0x7fb0fc19fcb0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fb0e800e9f0 tx=0x7fb0e800eeb0 comp rx=0 tx=0).stop 2024-03-08T21:14:23.474 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.495+0000 7fb102c00640 1 -- 172.21.15.107:0/703781157 shutdown_connections 2024-03-08T21:14:23.474 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.495+0000 7fb102c00640 1 --2- 172.21.15.107:0/703781157 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7fb0e40a2960 0x7fb0e40a4e20 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:14:23.474 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.495+0000 7fb102c00640 1 --2- 172.21.15.107:0/703781157 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fb0fc199920 0x7fb0fc199dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:14:23.474 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.495+0000 7fb102c00640 1 --2- 172.21.15.107:0/703781157 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb0fc106410 0x7fb0fc19fcb0 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:14:23.474 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.495+0000 7fb102c00640 1 --2- 172.21.15.107:0/703781157 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb0fc101340 0x7fb0fc19f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:14:23.474 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.495+0000 7fb102c00640 1 -- 172.21.15.107:0/703781157 >> 172.21.15.107:0/703781157 conn(0x7fb0fc0fcc40 msgr2=0x7fb0fc0fef90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:14:23.474 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.496+0000 7fb102c00640 1 -- 172.21.15.107:0/703781157 shutdown_connections 2024-03-08T21:14:23.474 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:23.496+0000 7fb102c00640 1 -- 172.21.15.107:0/703781157 wait complete. 2024-03-08T21:14:24.622 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:14:24 smithi204 ceph-mon[34935]: pgmap v440: 97 pgs: 97 active+clean; 9.5 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 3.6 KiB/s wr, 13 op/s 2024-03-08T21:14:24.711 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:14:24 smithi107 ceph-mon[31381]: pgmap v440: 97 pgs: 97 active+clean; 9.5 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 3.6 KiB/s wr, 13 op/s 2024-03-08T21:14:24.758 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:14:24 smithi149 ceph-mon[35042]: pgmap v440: 97 pgs: 97 active+clean; 9.5 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 3.6 KiB/s wr, 13 op/s 2024-03-08T21:14:26.622 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:14:26 smithi204 ceph-mon[34935]: pgmap v441: 97 pgs: 97 active+clean; 9.5 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 307 B/s wr, 10 op/s 2024-03-08T21:14:26.711 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:14:26 smithi107 ceph-mon[31381]: pgmap v441: 97 pgs: 97 active+clean; 9.5 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 307 B/s wr, 10 op/s 2024-03-08T21:14:26.759 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:14:26 smithi149 ceph-mon[35042]: pgmap v441: 97 pgs: 97 active+clean; 9.5 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 307 B/s wr, 10 op/s 2024-03-08T21:14:28.622 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:14:28 smithi204 ceph-mon[34935]: pgmap v442: 97 pgs: 97 active+clean; 9.5 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 307 B/s wr, 10 op/s 2024-03-08T21:14:28.711 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:14:28 smithi107 ceph-mon[31381]: pgmap v442: 97 pgs: 97 active+clean; 9.5 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 307 B/s wr, 10 op/s 2024-03-08T21:14:28.758 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:14:28 smithi149 ceph-mon[35042]: pgmap v442: 97 pgs: 97 active+clean; 9.5 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 307 B/s wr, 10 op/s 2024-03-08T21:14:30.622 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:14:30 smithi204 ceph-mon[34935]: pgmap v443: 97 pgs: 97 active+clean; 9.5 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 307 B/s wr, 10 op/s 2024-03-08T21:14:30.711 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:14:30 smithi107 ceph-mon[31381]: pgmap v443: 97 pgs: 97 active+clean; 9.5 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 307 B/s wr, 10 op/s 2024-03-08T21:14:30.758 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:14:30 smithi149 ceph-mon[35042]: pgmap v443: 97 pgs: 97 active+clean; 9.5 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 307 B/s wr, 10 op/s 2024-03-08T21:14:32.622 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:14:32 smithi204 ceph-mon[34935]: pgmap v444: 97 pgs: 97 active+clean; 9.5 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 266 B/s wr, 9 op/s 2024-03-08T21:14:32.711 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:14:32 smithi107 ceph-mon[31381]: pgmap v444: 97 pgs: 97 active+clean; 9.5 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 266 B/s wr, 9 op/s 2024-03-08T21:14:32.758 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:14:32 smithi149 ceph-mon[35042]: pgmap v444: 97 pgs: 97 active+clean; 9.5 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 266 B/s wr, 9 op/s 2024-03-08T21:14:34.622 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:14:34 smithi204 ceph-mon[34935]: pgmap v445: 97 pgs: 97 active+clean; 9.5 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:14:34.711 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:14:34 smithi107 ceph-mon[31381]: pgmap v445: 97 pgs: 97 active+clean; 9.5 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:14:34.758 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:14:34 smithi149 ceph-mon[35042]: pgmap v445: 97 pgs: 97 active+clean; 9.5 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:14:36.621 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.644+0000 7fef02000640 1 -- 172.21.15.107:0/3441744850 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feefc101510 msgr2=0x7feefc1018f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:14:36.622 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.644+0000 7fef02000640 1 --2- 172.21.15.107:0/3441744850 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feefc101510 0x7feefc1018f0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7feeec009ca0 tx=0x7feeec02c0f0 comp rx=0 tx=0).stop 2024-03-08T21:14:36.622 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.644+0000 7fef02000640 1 -- 172.21.15.107:0/3441744850 shutdown_connections 2024-03-08T21:14:36.622 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.644+0000 7fef02000640 1 --2- 172.21.15.107:0/3441744850 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7feefc1065e0 0x7feefc10e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:14:36.622 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.644+0000 7fef02000640 1 --2- 172.21.15.107:0/3441744850 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7feefc101ec0 0x7feefc105f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:14:36.622 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.644+0000 7fef02000640 1 --2- 172.21.15.107:0/3441744850 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feefc101510 0x7feefc1018f0 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:14:36.622 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.644+0000 7fef02000640 1 -- 172.21.15.107:0/3441744850 >> 172.21.15.107:0/3441744850 conn(0x7feefc0fcdb0 msgr2=0x7feefc0ff1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:14:36.622 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.644+0000 7fef02000640 1 -- 172.21.15.107:0/3441744850 shutdown_connections 2024-03-08T21:14:36.622 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.644+0000 7fef02000640 1 -- 172.21.15.107:0/3441744850 wait complete. 2024-03-08T21:14:36.623 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:14:36 smithi204 ceph-mon[34935]: pgmap v446: 97 pgs: 97 active+clean; 9.5 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-08T21:14:36.625 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.645+0000 7fef02000640 1 Processor -- start 2024-03-08T21:14:36.625 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.645+0000 7fef02000640 1 -- start start 2024-03-08T21:14:36.625 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.646+0000 7fef02000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7feefc101510 0x7feefc0735f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:14:36.626 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.646+0000 7fef02000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feefc1065e0 0x7feefc073b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:14:36.626 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.646+0000 7fef02000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7feefc074070 0x7feefc1a5c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:14:36.626 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.646+0000 7fef02000640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feefc111540 con 0x7feefc1065e0 2024-03-08T21:14:36.626 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.646+0000 7fef02000640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feefc1113c0 con 0x7feefc074070 2024-03-08T21:14:36.626 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.646+0000 7fef02000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feefc1116c0 con 0x7feefc101510 2024-03-08T21:14:36.626 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.646+0000 7feef2a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feefc1065e0 0x7feefc073b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:14:36.626 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.646+0000 7feef2a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feefc1065e0 0x7feefc073b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:33232/0 (socket says 172.21.15.107:33232) 2024-03-08T21:14:36.626 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.646+0000 7feef2a00640 1 -- 172.21.15.107:0/4216082659 learned_addr learned my addr 172.21.15.107:0/4216082659 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:14:36.626 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.646+0000 7feefb400640 1 --2- 172.21.15.107:0/4216082659 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7feefc101510 0x7feefc0735f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:14:36.626 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.646+0000 7feefbe00640 1 --2- 172.21.15.107:0/4216082659 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7feefc074070 0x7feefc1a5c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:14:36.627 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.647+0000 7feef2a00640 1 -- 172.21.15.107:0/4216082659 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7feefc101510 msgr2=0x7feefc0735f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:14:36.627 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.647+0000 7feef2a00640 1 --2- 172.21.15.107:0/4216082659 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7feefc101510 0x7feefc0735f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:14:36.627 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.647+0000 7feef2a00640 1 -- 172.21.15.107:0/4216082659 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7feefc074070 msgr2=0x7feefc1a5c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:14:36.627 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.647+0000 7feef2a00640 1 --2- 172.21.15.107:0/4216082659 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7feefc074070 0x7feefc1a5c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:14:36.627 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.647+0000 7feef2a00640 1 -- 172.21.15.107:0/4216082659 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feefc1a6330 con 0x7feefc1065e0 2024-03-08T21:14:36.627 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.647+0000 7feefbe00640 1 --2- 172.21.15.107:0/4216082659 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7feefc074070 0x7feefc1a5c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T21:14:36.627 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.647+0000 7feef2a00640 1 --2- 172.21.15.107:0/4216082659 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feefc1065e0 0x7feefc073b30 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7feee400e9f0 tx=0x7feee400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:14:36.627 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.647+0000 7feef8c00640 1 -- 172.21.15.107:0/4216082659 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feee400ccc0 con 0x7feefc1065e0 2024-03-08T21:14:36.627 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.647+0000 7feef8c00640 1 -- 172.21.15.107:0/4216082659 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7feee40054d0 con 0x7feefc1065e0 2024-03-08T21:14:36.627 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.647+0000 7fef02000640 1 -- 172.21.15.107:0/4216082659 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feefc1a6620 con 0x7feefc1065e0 2024-03-08T21:14:36.627 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.647+0000 7feef8c00640 1 -- 172.21.15.107:0/4216082659 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feee4010430 con 0x7feefc1065e0 2024-03-08T21:14:36.628 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.648+0000 7fef02000640 1 -- 172.21.15.107:0/4216082659 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feefc1a6b20 con 0x7feefc1065e0 2024-03-08T21:14:36.629 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.651+0000 7fef04d69740 1 Processor -- start 2024-03-08T21:14:36.629 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.651+0000 7feef8c00640 1 -- 172.21.15.107:0/4216082659 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) v1 ==== 137422+0+0 (secure 0 0 0) 0x7feee4005020 con 0x7feefc1065e0 2024-03-08T21:14:36.629 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.651+0000 7fef04d69740 1 -- start start 2024-03-08T21:14:36.629 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.651+0000 7fef04d69740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5576d2f59c80 0x5576d2f5a060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:14:36.630 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.651+0000 7fef04d69740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5576d2f5a5a0 0x5576d2f62e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:14:36.630 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.651+0000 7fef04d69740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5576d2f63370 0x5576d2f65760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:14:36.630 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.651+0000 7fef04d69740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576d2ebd3d0 con 0x5576d2f5a5a0 2024-03-08T21:14:36.630 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.651+0000 7fef04d69740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576d2e666b0 con 0x5576d2f59c80 2024-03-08T21:14:36.630 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.651+0000 7fef04d69740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576d2eb9470 con 0x5576d2f63370 2024-03-08T21:14:36.630 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.652+0000 7feefb400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5576d2f5a5a0 0x5576d2f62e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:14:36.630 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.652+0000 7feefb400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5576d2f5a5a0 0x5576d2f62e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:33234/0 (socket says 172.21.15.107:33234) 2024-03-08T21:14:36.630 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.652+0000 7feefb400640 1 -- 172.21.15.107:0/642215958 learned_addr learned my addr 172.21.15.107:0/642215958 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:14:36.630 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.652+0000 7feefbe00640 1 --2- 172.21.15.107:0/642215958 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5576d2f59c80 0x5576d2f5a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:14:36.630 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.652+0000 7feef2a00640 1 --2- 172.21.15.107:0/642215958 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5576d2f63370 0x5576d2f65760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:14:36.632 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.652+0000 7feefb400640 1 -- 172.21.15.107:0/642215958 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5576d2f63370 msgr2=0x5576d2f65760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:14:36.632 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.652+0000 7feefb400640 1 --2- 172.21.15.107:0/642215958 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5576d2f63370 0x5576d2f65760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:14:36.633 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.652+0000 7feefb400640 1 -- 172.21.15.107:0/642215958 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5576d2f59c80 msgr2=0x5576d2f5a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:14:36.633 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.652+0000 7feefb400640 1 --2- 172.21.15.107:0/642215958 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5576d2f59c80 0x5576d2f5a060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:14:36.633 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.652+0000 7feefb400640 1 -- 172.21.15.107:0/642215958 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5576d2e8cb50 con 0x5576d2f5a5a0 2024-03-08T21:14:36.633 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.652+0000 7feefbe00640 1 --2- 172.21.15.107:0/642215958 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5576d2f59c80 0x5576d2f5a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T21:14:36.633 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.653+0000 7feef8c00640 1 --2- 172.21.15.107:0/4216082659 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7feee00a2a40 0x7feee00a4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:14:36.633 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.653+0000 7feefb400640 1 --2- 172.21.15.107:0/642215958 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5576d2f5a5a0 0x5576d2f62e30 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7feeec02c700 tx=0x7feeec032340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=623e36bebc00fb95 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:14:36.633 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.653+0000 7feef8c00640 1 -- 172.21.15.107:0/4216082659 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 8585+0+0 (secure 0 0 0) 0x7feee40cf320 con 0x7feefc1065e0 2024-03-08T21:14:36.633 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.653+0000 7feefbe00640 1 --2- 172.21.15.107:0/4216082659 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7feee00a2a40 0x7feee00a4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:14:36.633 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.653+0000 7feeebe00640 1 -- 172.21.15.107:0/642215958 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feeec043070 con 0x5576d2f5a5a0 2024-03-08T21:14:36.633 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.654+0000 7feeebe00640 1 -- 172.21.15.107:0/642215958 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7feeec032db0 con 0x5576d2f5a5a0 2024-03-08T21:14:36.633 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.654+0000 7feeebe00640 1 -- 172.21.15.107:0/642215958 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feeec039040 con 0x5576d2f5a5a0 2024-03-08T21:14:36.633 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.654+0000 7fef04d69740 1 -- 172.21.15.107:0/642215958 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5576d2f5a5a0 msgr2=0x5576d2f62e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:14:36.634 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.654+0000 7feefbe00640 1 --2- 172.21.15.107:0/4216082659 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7feee00a2a40 0x7feee00a4f00 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7feef4009d40 tx=0x7feef4005f90 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:14:36.634 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.654+0000 7fef04d69740 1 --2- 172.21.15.107:0/642215958 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5576d2f5a5a0 0x5576d2f62e30 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7feeec02c700 tx=0x7feeec032340 comp rx=0 tx=0).stop 2024-03-08T21:14:36.634 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.655+0000 7fef04d69740 1 -- 172.21.15.107:0/642215958 shutdown_connections 2024-03-08T21:14:36.634 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.655+0000 7fef04d69740 1 --2- 172.21.15.107:0/642215958 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5576d2f63370 0x5576d2f65760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:14:36.634 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.655+0000 7fef04d69740 1 --2- 172.21.15.107:0/642215958 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5576d2f5a5a0 0x5576d2f62e30 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:14:36.634 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.655+0000 7fef04d69740 1 --2- 172.21.15.107:0/642215958 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5576d2f59c80 0x5576d2f5a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:14:36.634 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.655+0000 7fef04d69740 1 -- 172.21.15.107:0/642215958 >> 172.21.15.107:0/642215958 conn(0x5576d2f545a0 msgr2=0x5576d2f54980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:14:36.634 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.655+0000 7fef04d69740 1 -- 172.21.15.107:0/642215958 shutdown_connections 2024-03-08T21:14:36.634 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.655+0000 7fef04d69740 1 -- 172.21.15.107:0/642215958 wait complete. 2024-03-08T21:14:36.634 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.655+0000 7fef04d69740 1 -- start start 2024-03-08T21:14:36.634 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.655+0000 7fef04d69740 1 Processor -- start 2024-03-08T21:14:36.637 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.656+0000 7fef04d69740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5576d2f59c80 0x5576d2f62cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:14:36.637 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.656+0000 7fef04d69740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5576d2f5a5a0 0x5576d2f60050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:14:36.638 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.656+0000 7fef04d69740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5576d2f63370 0x5576d2f605b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:14:36.638 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.656+0000 7fef04d69740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576d2f65f40 con 0x5576d2f5a5a0 2024-03-08T21:14:36.638 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.656+0000 7fef04d69740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576d2e799f0 con 0x5576d2f59c80 2024-03-08T21:14:36.638 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.656+0000 7fef04d69740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576d2f660c0 con 0x5576d2f63370 2024-03-08T21:14:36.638 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.656+0000 7fef04d69740 10 client.0 _ll_register_callbacks cb 0x5576d2bd1a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T21:14:36.638 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.656+0000 7feef2a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5576d2f5a5a0 0x5576d2f60050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:14:36.638 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.656+0000 7feef2a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5576d2f5a5a0 0x5576d2f60050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:33238/0 (socket says 172.21.15.107:33238) 2024-03-08T21:14:36.638 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.656+0000 7feef2a00640 1 -- 172.21.15.107:0/2123132996 learned_addr learned my addr 172.21.15.107:0/2123132996 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:14:36.638 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.657+0000 7feefbe00640 1 --2- 172.21.15.107:0/2123132996 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5576d2f63370 0x5576d2f605b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:14:36.638 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.657+0000 7feefb400640 1 --2- 172.21.15.107:0/2123132996 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5576d2f59c80 0x5576d2f62cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:14:36.639 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.657+0000 7feef2a00640 1 -- 172.21.15.107:0/2123132996 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5576d2f63370 msgr2=0x5576d2f605b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:14:36.639 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.657+0000 7feef2a00640 1 --2- 172.21.15.107:0/2123132996 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5576d2f63370 0x5576d2f605b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:14:36.639 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.657+0000 7feef2a00640 1 -- 172.21.15.107:0/2123132996 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5576d2f59c80 msgr2=0x5576d2f62cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:14:36.639 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.657+0000 7feef2a00640 1 --2- 172.21.15.107:0/2123132996 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5576d2f59c80 0x5576d2f62cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:14:36.639 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.657+0000 7feef2a00640 1 -- 172.21.15.107:0/2123132996 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5576d2bd2210 con 0x5576d2f5a5a0 2024-03-08T21:14:36.639 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.657+0000 7feef2a00640 1 --2- 172.21.15.107:0/2123132996 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5576d2f5a5a0 0x5576d2f60050 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7feee40cca30 tx=0x7feee409b420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fe36ee0797f192 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:14:36.639 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.657+0000 7feee8c00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-03-08T21:14:36.639 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.657+0000 7feee8c00640 1 -- 172.21.15.107:0/2123132996 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feee40cce60 con 0x5576d2f5a5a0 2024-03-08T21:14:36.639 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.657+0000 7feee8c00640 1 -- 172.21.15.107:0/2123132996 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 8585+0+0 (secure 0 0 0) 0x7feee409bb40 con 0x5576d2f5a5a0 2024-03-08T21:14:36.639 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.658+0000 7fef04d69740 1 -- 172.21.15.107:0/2123132996 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7feeec032db0 con 0x5576d2f5a5a0 2024-03-08T21:14:36.639 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.658+0000 7feee8c00640 1 -- 172.21.15.107:0/2123132996 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7feee40b1e60 con 0x5576d2f5a5a0 2024-03-08T21:14:36.639 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.658+0000 7fef04d69740 10 client.16629 fetch_fsmap learned FSMap version 41 2024-03-08T21:14:36.640 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.658+0000 7fef04d69740 1 -- 172.21.15.107:0/2123132996 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x5576d2c9e620 con 0x5576d2f5a5a0 2024-03-08T21:14:36.640 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.658+0000 7feee8c00640 1 -- 172.21.15.107:0/2123132996 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7feeec032db0 con 0x5576d2f5a5a0 2024-03-08T21:14:36.640 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.658+0000 7feee8c00640 1 -- 172.21.15.107:0/2123132996 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 41) v1 ==== 7307+0+0 (secure 0 0 0) 0x7feee40a3bf0 con 0x5576d2f5a5a0 2024-03-08T21:14:36.640 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.658+0000 7fef04d69740 10 client.16629 fetch_fsmap finished waiting for FSMap version 41 2024-03-08T21:14:36.640 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.658+0000 7fef04d69740 10 client.16629 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-08T21:14:36.640 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.658+0000 7fef04d69740 20 client.16629 populate_metadata read hostname 'smithi107' 2024-03-08T21:14:36.640 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.658+0000 7fef04d69740 1 --2- 172.21.15.107:0/2123132996 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x5576d2f73210 0x5576d2f75600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:14:36.640 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.658+0000 7fef04d69740 4 client.16629 mds_command: new command op to 34475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-08T21:14:36.640 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.658+0000 7fef04d69740 1 -- 172.21.15.107:0/2123132996 --> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5576d2eca910 con 0x5576d2f73210 2024-03-08T21:14:36.640 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.659+0000 7feefb400640 1 --2- 172.21.15.107:0/2123132996 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x5576d2f73210 0x5576d2f75600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:14:36.640 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.660+0000 7feee8c00640 10 client.16629 ms_handle_connect on v2:172.21.15.149:6838/55290596 2024-03-08T21:14:36.640 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.660+0000 7feefb400640 1 --2- 172.21.15.107:0/2123132996 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x5576d2f73210 0x5576d2f75600 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9003fabe2a781430 server_cookie=8443f53502215389 in_seq=0 out_seq=0 2024-03-08T21:14:36.642 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.662+0000 7feee8c00640 1 -- 172.21.15.107:0/2123132996 <== mds.0 v2:172.21.15.149:6838/55290596 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5576d2eca910 con 0x5576d2f73210 2024-03-08T21:14:36.642 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.662+0000 7feee8c00640 10 client.16629 handle_command_reply: tid=0 2024-03-08T21:14:36.642 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.662+0000 7fef04d69740 1 client.16629 shutdown 2024-03-08T21:14:36.642 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.662+0000 7fef04d69740 1 -- 172.21.15.107:0/2123132996 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5576d2f5a5a0 msgr2=0x5576d2f60050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:14:36.642 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.662+0000 7fef04d69740 1 --2- 172.21.15.107:0/2123132996 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5576d2f5a5a0 0x5576d2f60050 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7feee40cca30 tx=0x7feee409b420 comp rx=0 tx=0).stop 2024-03-08T21:14:36.642 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.662+0000 7fef04d69740 1 -- 172.21.15.107:0/2123132996 shutdown_connections 2024-03-08T21:14:36.642 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.662+0000 7fef04d69740 1 --2- 172.21.15.107:0/2123132996 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5576d2f63370 0x5576d2f605b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:14:36.642 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.662+0000 7fef04d69740 1 --2- 172.21.15.107:0/2123132996 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5576d2f5a5a0 0x5576d2f60050 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:14:36.643 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.662+0000 7fef04d69740 1 --2- 172.21.15.107:0/2123132996 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x5576d2f73210 0x5576d2f75600 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:14:36.643 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.662+0000 7fef04d69740 1 --2- 172.21.15.107:0/2123132996 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5576d2f59c80 0x5576d2f62cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:14:36.643 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.662+0000 7fef04d69740 1 -- 172.21.15.107:0/2123132996 >> 172.21.15.107:0/2123132996 conn(0x5576d2f545a0 msgr2=0x5576d2f562f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:14:36.643 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.662+0000 7feee8c00640 0 client.16629 ms_handle_reset on v2:172.21.15.149:6838/55290596 2024-03-08T21:14:36.643 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.662+0000 7fef04d69740 1 -- 172.21.15.107:0/2123132996 shutdown_connections 2024-03-08T21:14:36.643 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.662+0000 7fef04d69740 1 -- 172.21.15.107:0/2123132996 wait complete. 2024-03-08T21:14:36.643 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.662+0000 7fef04d69740 20 client.16629 trim_cache size 0 max 16384 2024-03-08T21:14:36.657 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.679+0000 7fef04d69740 1 Processor -- start 2024-03-08T21:14:36.658 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.679+0000 7fef04d69740 1 -- start start 2024-03-08T21:14:36.658 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.679+0000 7fef04d69740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5576d2f59c80 0x5576d2f62780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:14:36.658 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.679+0000 7fef04d69740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5576d2f5a5a0 0x5576d2f62cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:14:36.658 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.679+0000 7fef04d69740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5576d2f63370 0x5576d2f6a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:14:36.658 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.679+0000 7fef04d69740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feefc1b07a0 con 0x5576d2f59c80 2024-03-08T21:14:36.658 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.679+0000 7fef04d69740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feefc1ae7c0 con 0x5576d2f5a5a0 2024-03-08T21:14:36.658 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.679+0000 7fef04d69740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feefc1b0140 con 0x5576d2f63370 2024-03-08T21:14:36.658 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.679+0000 7feefb400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5576d2f59c80 0x5576d2f62780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:14:36.658 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.679+0000 7feefb400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5576d2f59c80 0x5576d2f62780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:33242/0 (socket says 172.21.15.107:33242) 2024-03-08T21:14:36.658 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.679+0000 7feefb400640 1 -- 172.21.15.107:0/4282674141 learned_addr learned my addr 172.21.15.107:0/4282674141 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:14:36.659 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.680+0000 7feef2a00640 1 --2- 172.21.15.107:0/4282674141 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5576d2f5a5a0 0x5576d2f62cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:14:36.659 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.680+0000 7feefbe00640 1 --2- 172.21.15.107:0/4282674141 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5576d2f63370 0x5576d2f6a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:14:36.659 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.680+0000 7feefb400640 1 -- 172.21.15.107:0/4282674141 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5576d2f63370 msgr2=0x5576d2f6a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:14:36.659 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.680+0000 7feefb400640 1 --2- 172.21.15.107:0/4282674141 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5576d2f63370 0x5576d2f6a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:14:36.659 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.680+0000 7feefb400640 1 -- 172.21.15.107:0/4282674141 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5576d2f5a5a0 msgr2=0x5576d2f62cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:14:36.659 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.680+0000 7feefb400640 1 --2- 172.21.15.107:0/4282674141 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5576d2f5a5a0 0x5576d2f62cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:14:36.659 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.680+0000 7feefb400640 1 -- 172.21.15.107:0/4282674141 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5576d2f5bc40 con 0x5576d2f59c80 2024-03-08T21:14:36.659 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.680+0000 7feef2a00640 1 --2- 172.21.15.107:0/4282674141 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5576d2f5a5a0 0x5576d2f62cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T21:14:36.659 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.680+0000 7feefb400640 1 --2- 172.21.15.107:0/4282674141 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5576d2f59c80 0x5576d2f62780 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7feeec005cc0 tx=0x7feeec02e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0b643579928a45e server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:14:36.659 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.680+0000 7feeeb400640 1 -- 172.21.15.107:0/4282674141 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feeec052070 con 0x5576d2f59c80 2024-03-08T21:14:36.659 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.681+0000 7feeeb400640 1 -- 172.21.15.107:0/4282674141 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7feeec032db0 con 0x5576d2f59c80 2024-03-08T21:14:36.660 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.681+0000 7feeeb400640 1 -- 172.21.15.107:0/4282674141 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feeec039040 con 0x5576d2f59c80 2024-03-08T21:14:36.663 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.681+0000 7fef04d69740 1 -- 172.21.15.107:0/4282674141 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5576d2f59c80 msgr2=0x5576d2f62780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:14:36.663 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.681+0000 7fef04d69740 1 --2- 172.21.15.107:0/4282674141 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5576d2f59c80 0x5576d2f62780 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7feeec005cc0 tx=0x7feeec02e040 comp rx=0 tx=0).stop 2024-03-08T21:14:36.663 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.681+0000 7fef04d69740 1 -- 172.21.15.107:0/4282674141 shutdown_connections 2024-03-08T21:14:36.663 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.681+0000 7fef04d69740 1 --2- 172.21.15.107:0/4282674141 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5576d2f63370 0x5576d2f6a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:14:36.663 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.681+0000 7fef04d69740 1 --2- 172.21.15.107:0/4282674141 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5576d2f5a5a0 0x5576d2f62cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:14:36.664 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.681+0000 7fef04d69740 1 --2- 172.21.15.107:0/4282674141 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5576d2f59c80 0x5576d2f62780 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:14:36.664 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.681+0000 7fef04d69740 1 -- 172.21.15.107:0/4282674141 >> 172.21.15.107:0/4282674141 conn(0x5576d2f545a0 msgr2=0x5576d2f79eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:14:36.664 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.682+0000 7fef04d69740 1 -- 172.21.15.107:0/4282674141 shutdown_connections 2024-03-08T21:14:36.664 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.682+0000 7fef04d69740 1 -- 172.21.15.107:0/4282674141 wait complete. 2024-03-08T21:14:36.664 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.682+0000 7fef04d69740 1 -- start start 2024-03-08T21:14:36.664 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.682+0000 7fef04d69740 1 Processor -- start 2024-03-08T21:14:36.664 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.683+0000 7fef04d69740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5576d2f59c80 0x5576d2f696b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:14:36.664 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.683+0000 7fef04d69740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5576d2f5a5a0 0x5576d2f69bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:14:36.664 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.683+0000 7fef04d69740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5576d2f63370 0x5576d2fc3760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:14:36.664 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.683+0000 7fef04d69740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feefc1af480 con 0x5576d2f63370 2024-03-08T21:14:36.664 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.683+0000 7fef04d69740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feefc1aee20 con 0x5576d2f5a5a0 2024-03-08T21:14:36.664 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.683+0000 7fef04d69740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feefc1afae0 con 0x5576d2f59c80 2024-03-08T21:14:36.665 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.683+0000 7fef04d69740 10 client.0 _ll_register_callbacks cb 0x7feef4005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T21:14:36.665 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.683+0000 7feefbe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5576d2f63370 0x5576d2fc3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:14:36.665 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.683+0000 7feefbe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5576d2f63370 0x5576d2fc3760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:33254/0 (socket says 172.21.15.107:33254) 2024-03-08T21:14:36.665 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.683+0000 7feefbe00640 1 -- 172.21.15.107:0/1787913987 learned_addr learned my addr 172.21.15.107:0/1787913987 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:14:36.665 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.683+0000 7feefb400640 1 --2- 172.21.15.107:0/1787913987 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5576d2f59c80 0x5576d2f696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:14:36.665 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.683+0000 7feef2a00640 1 --2- 172.21.15.107:0/1787913987 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5576d2f5a5a0 0x5576d2f69bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:14:36.665 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.684+0000 7feefbe00640 1 -- 172.21.15.107:0/1787913987 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5576d2f59c80 msgr2=0x5576d2f696b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:14:36.665 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.684+0000 7feefbe00640 1 --2- 172.21.15.107:0/1787913987 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5576d2f59c80 0x5576d2f696b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:14:36.665 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.684+0000 7feefbe00640 1 -- 172.21.15.107:0/1787913987 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5576d2f5a5a0 msgr2=0x5576d2f69bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:14:36.665 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.684+0000 7feefbe00640 1 --2- 172.21.15.107:0/1787913987 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5576d2f5a5a0 0x5576d2f69bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:14:36.665 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.684+0000 7feefbe00640 1 -- 172.21.15.107:0/1787913987 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5576d2fc3d50 con 0x5576d2f63370 2024-03-08T21:14:36.666 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.684+0000 7feefbe00640 1 --2- 172.21.15.107:0/1787913987 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5576d2f63370 0x5576d2fc3760 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7feef401fb20 tx=0x7feef40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8ae13fb568df1c4 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:14:36.666 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.684+0000 7feee8c00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-03-08T21:14:36.666 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.684+0000 7feee8c00640 1 -- 172.21.15.107:0/1787913987 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feef402d060 con 0x5576d2f63370 2024-03-08T21:14:36.666 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.684+0000 7feee8c00640 1 -- 172.21.15.107:0/1787913987 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 8585+0+0 (secure 0 0 0) 0x7feef4011b60 con 0x5576d2f63370 2024-03-08T21:14:36.666 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.684+0000 7fef04d69740 1 -- 172.21.15.107:0/1787913987 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7feeec032db0 con 0x5576d2f63370 2024-03-08T21:14:36.666 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.685+0000 7feee8c00640 1 -- 172.21.15.107:0/1787913987 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7feef4009e30 con 0x5576d2f63370 2024-03-08T21:14:36.666 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.685+0000 7fef04d69740 10 client.16635 fetch_fsmap learned FSMap version 41 2024-03-08T21:14:36.666 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.685+0000 7fef04d69740 1 -- 172.21.15.107:0/1787913987 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x5576d2fcb0b0 con 0x5576d2f63370 2024-03-08T21:14:36.666 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.685+0000 7feee8c00640 1 -- 172.21.15.107:0/1787913987 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7feeec032db0 con 0x5576d2f63370 2024-03-08T21:14:36.666 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.685+0000 7feee8c00640 1 -- 172.21.15.107:0/1787913987 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 41) v1 ==== 7307+0+0 (secure 0 0 0) 0x7feef4027070 con 0x5576d2f63370 2024-03-08T21:14:36.666 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.685+0000 7fef04d69740 10 client.16635 fetch_fsmap finished waiting for FSMap version 41 2024-03-08T21:14:36.667 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.685+0000 7fef04d69740 10 client.16635 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-08T21:14:36.667 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.685+0000 7fef04d69740 20 client.16635 populate_metadata read hostname 'smithi107' 2024-03-08T21:14:36.667 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.685+0000 7fef04d69740 1 --2- 172.21.15.107:0/1787913987 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x5576d2fcb7e0 0x5576d2fcdc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:14:36.667 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.685+0000 7fef04d69740 4 client.16635 mds_command: new command op to 34475 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-08T21:14:36.667 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.685+0000 7fef04d69740 1 -- 172.21.15.107:0/1787913987 --> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5576d2ec4c60 con 0x5576d2fcb7e0 2024-03-08T21:14:36.667 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.686+0000 7feefb400640 1 --2- 172.21.15.107:0/1787913987 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x5576d2fcb7e0 0x5576d2fcdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:14:36.667 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.687+0000 7feefb400640 1 --2- 172.21.15.107:0/1787913987 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x5576d2fcb7e0 0x5576d2fcdc00 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b8c64037dc2e02e9 server_cookie=25b1169283be27cf in_seq=0 out_seq=0 2024-03-08T21:14:36.667 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.687+0000 7feee8c00640 10 client.16635 ms_handle_connect on v2:172.21.15.149:6838/55290596 2024-03-08T21:14:36.675 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.697+0000 7feee8c00640 1 -- 172.21.15.107:0/1787913987 <== mds.0 v2:172.21.15.149:6838/55290596 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5576d2ec4c60 con 0x5576d2fcb7e0 2024-03-08T21:14:36.675 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.697+0000 7feee8c00640 10 client.16635 handle_command_reply: tid=0 2024-03-08T21:14:36.675 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.697+0000 7fef04d69740 1 client.16635 shutdown 2024-03-08T21:14:36.675 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.697+0000 7fef04d69740 1 -- 172.21.15.107:0/1787913987 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5576d2f63370 msgr2=0x5576d2fc3760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:14:36.675 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.697+0000 7fef04d69740 1 --2- 172.21.15.107:0/1787913987 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5576d2f63370 0x5576d2fc3760 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7feef401fb20 tx=0x7feef40046c0 comp rx=0 tx=0).stop 2024-03-08T21:14:36.675 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.697+0000 7fef04d69740 1 -- 172.21.15.107:0/1787913987 shutdown_connections 2024-03-08T21:14:36.675 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.697+0000 7fef04d69740 1 --2- 172.21.15.107:0/1787913987 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5576d2f63370 0x5576d2fc3760 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:14:36.675 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.697+0000 7fef04d69740 1 --2- 172.21.15.107:0/1787913987 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x5576d2fcb7e0 0x5576d2fcdc00 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:14:36.676 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.697+0000 7fef04d69740 1 --2- 172.21.15.107:0/1787913987 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x5576d2f5a5a0 0x5576d2f69bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:14:36.676 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.697+0000 7fef04d69740 1 --2- 172.21.15.107:0/1787913987 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x5576d2f59c80 0x5576d2f696b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:14:36.676 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.697+0000 7fef04d69740 1 -- 172.21.15.107:0/1787913987 >> 172.21.15.107:0/1787913987 conn(0x5576d2f545a0 msgr2=0x5576d2f5e8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:14:36.676 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.697+0000 7feee8c00640 0 client.16635 ms_handle_reset on v2:172.21.15.149:6838/55290596 2024-03-08T21:14:36.676 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.697+0000 7fef04d69740 1 -- 172.21.15.107:0/1787913987 shutdown_connections 2024-03-08T21:14:36.676 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.698+0000 7fef04d69740 1 -- 172.21.15.107:0/1787913987 wait complete. 2024-03-08T21:14:36.676 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.698+0000 7fef04d69740 20 client.16635 trim_cache size 0 max 16384 2024-03-08T21:14:36.676 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:{ 2024-03-08T21:14:36.676 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "message": "", 2024-03-08T21:14:36.677 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "return_code": 0 2024-03-08T21:14:36.677 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:} 2024-03-08T21:14:36.677 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.698+0000 7fef02000640 1 -- 172.21.15.107:0/4216082659 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7feee00a2a40 msgr2=0x7feee00a4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:14:36.677 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.698+0000 7fef02000640 1 --2- 172.21.15.107:0/4216082659 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7feee00a2a40 0x7feee00a4f00 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7feef4009d40 tx=0x7feef4005f90 comp rx=0 tx=0).stop 2024-03-08T21:14:36.677 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.698+0000 7fef02000640 1 -- 172.21.15.107:0/4216082659 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feefc1065e0 msgr2=0x7feefc073b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:14:36.677 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.698+0000 7fef02000640 1 --2- 172.21.15.107:0/4216082659 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feefc1065e0 0x7feefc073b30 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7feee400e9f0 tx=0x7feee400eeb0 comp rx=0 tx=0).stop 2024-03-08T21:14:36.677 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.699+0000 7fef02000640 1 -- 172.21.15.107:0/4216082659 shutdown_connections 2024-03-08T21:14:36.678 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.699+0000 7fef02000640 1 --2- 172.21.15.107:0/4216082659 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7feee00a2a40 0x7feee00a4f00 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:14:36.678 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.699+0000 7fef02000640 1 --2- 172.21.15.107:0/4216082659 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7feefc074070 0x7feefc1a5c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:14:36.678 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.699+0000 7fef02000640 1 --2- 172.21.15.107:0/4216082659 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feefc1065e0 0x7feefc073b30 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:14:36.678 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.699+0000 7fef02000640 1 --2- 172.21.15.107:0/4216082659 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7feefc101510 0x7feefc0735f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:14:36.678 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.699+0000 7fef02000640 1 -- 172.21.15.107:0/4216082659 >> 172.21.15.107:0/4216082659 conn(0x7feefc0fcdb0 msgr2=0x7feefc0ff100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:14:36.678 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.699+0000 7fef02000640 1 -- 172.21.15.107:0/4216082659 shutdown_connections 2024-03-08T21:14:36.678 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:36.699+0000 7fef02000640 1 -- 172.21.15.107:0/4216082659 wait complete. 2024-03-08T21:14:36.710 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:14:36 smithi107 ceph-mon[31381]: pgmap v446: 97 pgs: 97 active+clean; 9.5 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-08T21:14:36.758 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:14:36 smithi149 ceph-mon[35042]: pgmap v446: 97 pgs: 97 active+clean; 9.5 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-08T21:14:38.622 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:14:38 smithi204 ceph-mon[34935]: pgmap v447: 97 pgs: 97 active+clean; 9.5 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-08T21:14:38.711 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:14:38 smithi107 ceph-mon[31381]: pgmap v447: 97 pgs: 97 active+clean; 9.5 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-08T21:14:38.758 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:14:38 smithi149 ceph-mon[35042]: pgmap v447: 97 pgs: 97 active+clean; 9.5 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-08T21:14:40.622 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:14:40 smithi204 ceph-mon[34935]: pgmap v448: 97 pgs: 97 active+clean; 9.5 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-08T21:14:40.711 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:14:40 smithi107 ceph-mon[31381]: pgmap v448: 97 pgs: 97 active+clean; 9.5 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-08T21:14:40.758 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:14:40 smithi149 ceph-mon[35042]: pgmap v448: 97 pgs: 97 active+clean; 9.5 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-08T21:14:42.622 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:14:42 smithi204 ceph-mon[34935]: pgmap v449: 97 pgs: 97 active+clean; 9.5 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:14:42.711 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:14:42 smithi107 ceph-mon[31381]: pgmap v449: 97 pgs: 97 active+clean; 9.5 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:14:42.759 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:14:42 smithi149 ceph-mon[35042]: pgmap v449: 97 pgs: 97 active+clean; 9.5 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:14:44.711 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:14:44 smithi107 ceph-mon[31381]: pgmap v450: 97 pgs: 97 active+clean; 9.5 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:14:44.758 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:14:44 smithi149 ceph-mon[35042]: pgmap v450: 97 pgs: 97 active+clean; 9.5 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:14:44.872 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:14:44 smithi204 ceph-mon[34935]: pgmap v450: 97 pgs: 97 active+clean; 9.5 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:14:46.711 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:14:46 smithi107 ceph-mon[31381]: pgmap v451: 97 pgs: 97 active+clean; 9.5 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-08T21:14:46.758 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:14:46 smithi149 ceph-mon[35042]: pgmap v451: 97 pgs: 97 active+clean; 9.5 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-08T21:14:46.872 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:14:46 smithi204 ceph-mon[34935]: pgmap v451: 97 pgs: 97 active+clean; 9.5 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-08T21:14:48.711 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:14:48 smithi107 ceph-mon[31381]: pgmap v452: 97 pgs: 97 active+clean; 9.5 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:14:48.758 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:14:48 smithi149 ceph-mon[35042]: pgmap v452: 97 pgs: 97 active+clean; 9.5 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:14:48.872 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:14:48 smithi204 ceph-mon[34935]: pgmap v452: 97 pgs: 97 active+clean; 9.5 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:14:49.821 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.844+0000 7f0f50a00640 1 -- 172.21.15.107:0/2873752246 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0f4c108620 msgr2=0x7f0f4c1101f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:14:49.821 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.844+0000 7f0f50a00640 1 --2- 172.21.15.107:0/2873752246 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0f4c108620 0x7f0f4c1101f0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f0f44009ca0 tx=0x7f0f4402c0f0 comp rx=0 tx=0).stop 2024-03-08T21:14:49.821 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.844+0000 7f0f50a00640 1 -- 172.21.15.107:0/2873752246 shutdown_connections 2024-03-08T21:14:49.821 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.844+0000 7f0f50a00640 1 --2- 172.21.15.107:0/2873752246 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0f4c108620 0x7f0f4c1101f0 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:14:49.821 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.844+0000 7f0f50a00640 1 --2- 172.21.15.107:0/2873752246 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0f4c103f00 0x7f0f4c107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:14:49.821 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.844+0000 7f0f50a00640 1 --2- 172.21.15.107:0/2873752246 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0f4c103550 0x7f0f4c103930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:14:49.821 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.844+0000 7f0f50a00640 1 -- 172.21.15.107:0/2873752246 >> 172.21.15.107:0/2873752246 conn(0x7f0f4c0fedd0 msgr2=0x7f0f4c1011f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:14:49.821 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.844+0000 7f0f50a00640 1 -- 172.21.15.107:0/2873752246 shutdown_connections 2024-03-08T21:14:49.821 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.844+0000 7f0f50a00640 1 -- 172.21.15.107:0/2873752246 wait complete. 2024-03-08T21:14:49.822 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.846+0000 7f0f50a00640 1 Processor -- start 2024-03-08T21:14:49.822 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.846+0000 7f0f50a00640 1 -- start start 2024-03-08T21:14:49.823 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.847+0000 7f0f50a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0f4c103550 0x7f0f4c19f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:14:49.824 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.847+0000 7f0f50a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0f4c108620 0x7f0f4c19fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:14:49.824 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.847+0000 7f0f50a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0f4c199910 0x7f0f4c199dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:14:49.824 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.847+0000 7f0f50a00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f4c113570 con 0x7f0f4c199910 2024-03-08T21:14:49.824 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.847+0000 7f0f50a00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f4c1133f0 con 0x7f0f4c108620 2024-03-08T21:14:49.824 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.847+0000 7f0f50a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f4c1136f0 con 0x7f0f4c103550 2024-03-08T21:14:49.824 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.847+0000 7f0f4b200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0f4c199910 0x7f0f4c199dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:14:49.824 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.847+0000 7f0f4b200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0f4c199910 0x7f0f4c199dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:38770/0 (socket says 172.21.15.107:38770) 2024-03-08T21:14:49.824 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.847+0000 7f0f4b200640 1 -- 172.21.15.107:0/3131660654 learned_addr learned my addr 172.21.15.107:0/3131660654 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:14:49.824 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.847+0000 7f0f4a800640 1 --2- 172.21.15.107:0/3131660654 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0f4c103550 0x7f0f4c19f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:14:49.824 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.848+0000 7f0f49e00640 1 --2- 172.21.15.107:0/3131660654 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0f4c108620 0x7f0f4c19fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:14:49.825 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.848+0000 7f0f4b200640 1 -- 172.21.15.107:0/3131660654 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0f4c103550 msgr2=0x7f0f4c19f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:14:49.825 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.848+0000 7f0f4b200640 1 --2- 172.21.15.107:0/3131660654 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0f4c103550 0x7f0f4c19f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:14:49.825 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.848+0000 7f0f4b200640 1 -- 172.21.15.107:0/3131660654 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0f4c108620 msgr2=0x7f0f4c19fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:14:49.825 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.848+0000 7f0f4b200640 1 --2- 172.21.15.107:0/3131660654 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0f4c108620 0x7f0f4c19fca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:14:49.826 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.848+0000 7f0f4b200640 1 -- 172.21.15.107:0/3131660654 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f4c19a430 con 0x7f0f4c199910 2024-03-08T21:14:49.826 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.848+0000 7f0f4b200640 1 --2- 172.21.15.107:0/3131660654 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0f4c199910 0x7f0f4c199dc0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f0f4402c700 tx=0x7f0f44035d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:14:49.826 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.849+0000 7f0f43400640 1 -- 172.21.15.107:0/3131660654 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f44039040 con 0x7f0f4c199910 2024-03-08T21:14:49.826 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.849+0000 7f0f50a00640 1 -- 172.21.15.107:0/3131660654 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f4c19a6c0 con 0x7f0f4c199910 2024-03-08T21:14:49.826 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.849+0000 7f0f43400640 1 -- 172.21.15.107:0/3131660654 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f0f44044070 con 0x7f0f4c199910 2024-03-08T21:14:49.826 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.849+0000 7f0f43400640 1 -- 172.21.15.107:0/3131660654 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f44004330 con 0x7f0f4c199910 2024-03-08T21:14:49.826 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.849+0000 7f0f50a00640 1 -- 172.21.15.107:0/3131660654 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f4c1a68b0 con 0x7f0f4c199910 2024-03-08T21:14:49.828 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.852+0000 7f0f5407e740 1 Processor -- start 2024-03-08T21:14:49.829 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.853+0000 7f0f43400640 1 -- 172.21.15.107:0/3131660654 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) v1 ==== 137422+0+0 (secure 0 0 0) 0x7f0f440044d0 con 0x7f0f4c199910 2024-03-08T21:14:49.829 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.853+0000 7f0f5407e740 1 -- start start 2024-03-08T21:14:49.832 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.853+0000 7f0f5407e740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c5ee540e80 0x55c5ee541260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:14:49.832 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.854+0000 7f0f5407e740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c5ee5417a0 0x55c5ee54a030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:14:49.832 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.854+0000 7f0f5407e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c5ee54a570 0x55c5ee54c960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:14:49.832 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.854+0000 7f0f5407e740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5ee4a43d0 con 0x55c5ee54a570 2024-03-08T21:14:49.832 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.854+0000 7f0f5407e740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5ee44d6b0 con 0x55c5ee540e80 2024-03-08T21:14:49.832 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.854+0000 7f0f5407e740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5ee4a0470 con 0x55c5ee5417a0 2024-03-08T21:14:49.832 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.854+0000 7f0f49e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c5ee54a570 0x55c5ee54c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:14:49.832 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.854+0000 7f0f49e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c5ee54a570 0x55c5ee54c960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:38780/0 (socket says 172.21.15.107:38780) 2024-03-08T21:14:49.833 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.854+0000 7f0f49e00640 1 -- 172.21.15.107:0/3375315398 learned_addr learned my addr 172.21.15.107:0/3375315398 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:14:49.833 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.854+0000 7f0f49e00640 1 --2- 172.21.15.107:0/3375315398 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c5ee540e80 0x55c5ee541260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:14:49.833 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.854+0000 7f0f4a800640 1 --2- 172.21.15.107:0/3375315398 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c5ee5417a0 0x55c5ee54a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:14:49.833 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.855+0000 7f0f49e00640 1 -- 172.21.15.107:0/3375315398 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c5ee5417a0 msgr2=0x55c5ee54a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:14:49.833 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.855+0000 7f0f49e00640 1 --2- 172.21.15.107:0/3375315398 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c5ee5417a0 0x55c5ee54a030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:14:49.833 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.855+0000 7f0f49e00640 1 -- 172.21.15.107:0/3375315398 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c5ee540e80 msgr2=0x55c5ee541260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:14:49.833 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.855+0000 7f0f49e00640 1 --2- 172.21.15.107:0/3375315398 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c5ee540e80 0x55c5ee541260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:14:49.833 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.855+0000 7f0f49e00640 1 -- 172.21.15.107:0/3375315398 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c5ee473b50 con 0x55c5ee54a570 2024-03-08T21:14:49.833 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.855+0000 7f0f4a800640 1 --2- 172.21.15.107:0/3375315398 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c5ee5417a0 0x55c5ee54a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T21:14:49.833 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.855+0000 7f0f43400640 1 --2- 172.21.15.107:0/3131660654 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f0f300a2960 0x7f0f300a4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:14:49.833 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.855+0000 7f0f43400640 1 -- 172.21.15.107:0/3131660654 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f0f440f1150 con 0x7f0f4c199910 2024-03-08T21:14:49.833 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.855+0000 7f0f49e00640 1 --2- 172.21.15.107:0/3375315398 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c5ee54a570 0x55c5ee54c960 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f0f38014920 tx=0x7f0f3800f440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6ce51778b570f75 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:14:49.833 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.855+0000 7f0f4a800640 1 --2- 172.21.15.107:0/3131660654 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f0f300a2960 0x7f0f300a4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:14:49.833 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.855+0000 7f0f37400640 1 -- 172.21.15.107:0/3375315398 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f38004030 con 0x55c5ee54a570 2024-03-08T21:14:49.834 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.855+0000 7f0f37400640 1 -- 172.21.15.107:0/3375315398 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f0f380041d0 con 0x55c5ee54a570 2024-03-08T21:14:49.834 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.856+0000 7f0f37400640 1 -- 172.21.15.107:0/3375315398 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f380035f0 con 0x55c5ee54a570 2024-03-08T21:14:49.837 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.856+0000 7f0f5407e740 1 -- 172.21.15.107:0/3375315398 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c5ee54a570 msgr2=0x55c5ee54c960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:14:49.837 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.856+0000 7f0f4a800640 1 --2- 172.21.15.107:0/3131660654 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f0f300a2960 0x7f0f300a4e20 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x55c5ee540930 tx=0x7f0f3c00b390 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:14:49.838 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.856+0000 7f0f5407e740 1 --2- 172.21.15.107:0/3375315398 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c5ee54a570 0x55c5ee54c960 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f0f38014920 tx=0x7f0f3800f440 comp rx=0 tx=0).stop 2024-03-08T21:14:49.838 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.857+0000 7f0f5407e740 1 -- 172.21.15.107:0/3375315398 shutdown_connections 2024-03-08T21:14:49.838 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.857+0000 7f0f5407e740 1 --2- 172.21.15.107:0/3375315398 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c5ee54a570 0x55c5ee54c960 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:14:49.838 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.857+0000 7f0f5407e740 1 --2- 172.21.15.107:0/3375315398 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c5ee5417a0 0x55c5ee54a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:14:49.838 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.857+0000 7f0f5407e740 1 --2- 172.21.15.107:0/3375315398 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c5ee540e80 0x55c5ee541260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:14:49.838 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.857+0000 7f0f5407e740 1 -- 172.21.15.107:0/3375315398 >> 172.21.15.107:0/3375315398 conn(0x55c5ee53b7a0 msgr2=0x55c5ee53bb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:14:49.838 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.857+0000 7f0f5407e740 1 -- 172.21.15.107:0/3375315398 shutdown_connections 2024-03-08T21:14:49.838 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.857+0000 7f0f5407e740 1 -- 172.21.15.107:0/3375315398 wait complete. 2024-03-08T21:14:49.838 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.857+0000 7f0f5407e740 1 -- start start 2024-03-08T21:14:49.838 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.857+0000 7f0f5407e740 1 Processor -- start 2024-03-08T21:14:49.838 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.858+0000 7f0f5407e740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c5ee540e80 0x55c5ee549f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:14:49.838 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.858+0000 7f0f5407e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c5ee5417a0 0x55c5ee5472f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:14:49.839 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.858+0000 7f0f5407e740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c5ee54a570 0x55c5ee547900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:14:49.839 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.858+0000 7f0f5407e740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5ee54d1a0 con 0x55c5ee5417a0 2024-03-08T21:14:49.839 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.858+0000 7f0f5407e740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5ee4609f0 con 0x55c5ee540e80 2024-03-08T21:14:49.839 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.858+0000 7f0f5407e740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5ee54d320 con 0x55c5ee54a570 2024-03-08T21:14:49.839 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.858+0000 7f0f5407e740 10 client.0 _ll_register_callbacks cb 0x55c5ee1b8a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T21:14:49.839 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.858+0000 7f0f49e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c5ee5417a0 0x55c5ee5472f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:14:49.839 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.858+0000 7f0f49e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c5ee5417a0 0x55c5ee5472f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:38790/0 (socket says 172.21.15.107:38790) 2024-03-08T21:14:49.839 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.858+0000 7f0f49e00640 1 -- 172.21.15.107:0/3326088921 learned_addr learned my addr 172.21.15.107:0/3326088921 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:14:49.839 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.859+0000 7f0f4a800640 1 --2- 172.21.15.107:0/3326088921 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c5ee540e80 0x55c5ee549f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:14:49.839 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.859+0000 7f0f4b200640 1 --2- 172.21.15.107:0/3326088921 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c5ee54a570 0x55c5ee547900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:14:49.839 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.859+0000 7f0f49e00640 1 -- 172.21.15.107:0/3326088921 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c5ee54a570 msgr2=0x55c5ee547900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:14:49.839 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.859+0000 7f0f49e00640 1 --2- 172.21.15.107:0/3326088921 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c5ee54a570 0x55c5ee547900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:14:49.839 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.859+0000 7f0f49e00640 1 -- 172.21.15.107:0/3326088921 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c5ee540e80 msgr2=0x55c5ee549f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:14:49.839 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.859+0000 7f0f49e00640 1 --2- 172.21.15.107:0/3326088921 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c5ee540e80 0x55c5ee549f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:14:49.840 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.859+0000 7f0f49e00640 1 -- 172.21.15.107:0/3326088921 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c5ee1b9210 con 0x55c5ee5417a0 2024-03-08T21:14:49.840 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.859+0000 7f0f49e00640 1 --2- 172.21.15.107:0/3326088921 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c5ee5417a0 0x55c5ee5472f0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f0f38017040 tx=0x7f0f3800fcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3506759022076550 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:14:49.840 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.859+0000 7f0f2be00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-03-08T21:14:49.840 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.859+0000 7f0f2be00640 1 -- 172.21.15.107:0/3326088921 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f3800c8d0 con 0x55c5ee5417a0 2024-03-08T21:14:49.840 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.860+0000 7f0f2be00640 1 -- 172.21.15.107:0/3326088921 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f0f3800ca70 con 0x55c5ee5417a0 2024-03-08T21:14:49.840 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.860+0000 7f0f5407e740 1 -- 172.21.15.107:0/3326088921 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0f380041d0 con 0x55c5ee5417a0 2024-03-08T21:14:49.840 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.860+0000 7f0f2be00640 1 -- 172.21.15.107:0/3326088921 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0f38012450 con 0x55c5ee5417a0 2024-03-08T21:14:49.840 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.860+0000 7f0f5407e740 10 client.16647 fetch_fsmap learned FSMap version 41 2024-03-08T21:14:49.840 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.860+0000 7f0f5407e740 1 -- 172.21.15.107:0/3326088921 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x55c5ee285620 con 0x55c5ee5417a0 2024-03-08T21:14:49.840 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.860+0000 7f0f2be00640 1 -- 172.21.15.107:0/3326088921 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f0f380041d0 con 0x55c5ee5417a0 2024-03-08T21:14:49.840 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.860+0000 7f0f2be00640 1 -- 172.21.15.107:0/3326088921 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 41) v1 ==== 7307+0+0 (secure 0 0 0) 0x7f0f38003800 con 0x55c5ee5417a0 2024-03-08T21:14:49.840 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.860+0000 7f0f5407e740 10 client.16647 fetch_fsmap finished waiting for FSMap version 41 2024-03-08T21:14:49.840 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.860+0000 7f0f5407e740 10 client.16647 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-08T21:14:49.841 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.861+0000 7f0f5407e740 20 client.16647 populate_metadata read hostname 'smithi107' 2024-03-08T21:14:49.841 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.861+0000 7f0f5407e740 1 --2- 172.21.15.107:0/3326088921 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55c5ee55a360 0x55c5ee55c750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:14:49.841 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.861+0000 7f0f5407e740 4 client.16647 mds_command: new command op to 34475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-08T21:14:49.841 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.861+0000 7f0f5407e740 1 -- 172.21.15.107:0/3326088921 --> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c5ee4b1910 con 0x55c5ee55a360 2024-03-08T21:14:49.841 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.861+0000 7f0f4a800640 1 --2- 172.21.15.107:0/3326088921 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55c5ee55a360 0x55c5ee55c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:14:49.842 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.862+0000 7f0f4a800640 1 --2- 172.21.15.107:0/3326088921 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55c5ee55a360 0x55c5ee55c750 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e8840b6dc80816ca server_cookie=cf611f33250d82f3 in_seq=0 out_seq=0 2024-03-08T21:14:49.842 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.862+0000 7f0f2be00640 10 client.16647 ms_handle_connect on v2:172.21.15.149:6838/55290596 2024-03-08T21:14:49.842 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.864+0000 7f0f2be00640 1 -- 172.21.15.107:0/3326088921 <== mds.0 v2:172.21.15.149:6838/55290596 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c5ee4b1910 con 0x55c5ee55a360 2024-03-08T21:14:49.842 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.864+0000 7f0f2be00640 10 client.16647 handle_command_reply: tid=0 2024-03-08T21:14:49.842 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.864+0000 7f0f5407e740 1 client.16647 shutdown 2024-03-08T21:14:49.842 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.864+0000 7f0f5407e740 1 -- 172.21.15.107:0/3326088921 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c5ee5417a0 msgr2=0x55c5ee5472f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:14:49.843 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.864+0000 7f0f5407e740 1 --2- 172.21.15.107:0/3326088921 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c5ee5417a0 0x55c5ee5472f0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f0f38017040 tx=0x7f0f3800fcc0 comp rx=0 tx=0).stop 2024-03-08T21:14:49.843 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.864+0000 7f0f5407e740 1 -- 172.21.15.107:0/3326088921 shutdown_connections 2024-03-08T21:14:49.843 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.864+0000 7f0f5407e740 1 --2- 172.21.15.107:0/3326088921 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c5ee54a570 0x55c5ee547900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:14:49.843 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.864+0000 7f0f5407e740 1 --2- 172.21.15.107:0/3326088921 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c5ee5417a0 0x55c5ee5472f0 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:14:49.843 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.864+0000 7f0f5407e740 1 --2- 172.21.15.107:0/3326088921 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55c5ee55a360 0x55c5ee55c750 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:14:49.843 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.864+0000 7f0f5407e740 1 --2- 172.21.15.107:0/3326088921 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c5ee540e80 0x55c5ee549f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:14:49.843 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.864+0000 7f0f5407e740 1 -- 172.21.15.107:0/3326088921 >> 172.21.15.107:0/3326088921 conn(0x55c5ee53b7a0 msgr2=0x55c5ee53d4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:14:49.843 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.864+0000 7f0f2be00640 0 client.16647 ms_handle_reset on v2:172.21.15.149:6838/55290596 2024-03-08T21:14:49.843 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.865+0000 7f0f5407e740 1 -- 172.21.15.107:0/3326088921 shutdown_connections 2024-03-08T21:14:49.843 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.865+0000 7f0f5407e740 1 -- 172.21.15.107:0/3326088921 wait complete. 2024-03-08T21:14:49.843 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.865+0000 7f0f5407e740 20 client.16647 trim_cache size 0 max 16384 2024-03-08T21:14:49.863 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.886+0000 7f0f5407e740 1 Processor -- start 2024-03-08T21:14:49.863 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.886+0000 7f0f5407e740 1 -- start start 2024-03-08T21:14:49.863 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.886+0000 7f0f5407e740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c5ee540e80 0x55c5ee549a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:14:49.863 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.886+0000 7f0f5407e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c5ee54a570 0x55c5ee549f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:14:49.863 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.886+0000 7f0f5407e740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c5ee556fb0 0x55c5ee544570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:14:49.863 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.886+0000 7f0f5407e740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f4c1b06d0 con 0x55c5ee54a570 2024-03-08T21:14:49.863 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.886+0000 7f0f5407e740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f4c1ae6f0 con 0x55c5ee540e80 2024-03-08T21:14:49.863 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.886+0000 7f0f5407e740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f4c1b0070 con 0x55c5ee556fb0 2024-03-08T21:14:49.864 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.886+0000 7f0f49e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c5ee54a570 0x55c5ee549f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:14:49.864 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.886+0000 7f0f49e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c5ee54a570 0x55c5ee549f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:38792/0 (socket says 172.21.15.107:38792) 2024-03-08T21:14:49.864 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.886+0000 7f0f49e00640 1 -- 172.21.15.107:0/1052002591 learned_addr learned my addr 172.21.15.107:0/1052002591 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:14:49.864 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.887+0000 7f0f4a800640 1 --2- 172.21.15.107:0/1052002591 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c5ee540e80 0x55c5ee549a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:14:49.864 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.887+0000 7f0f4b200640 1 --2- 172.21.15.107:0/1052002591 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c5ee556fb0 0x55c5ee544570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:14:49.864 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.887+0000 7f0f49e00640 1 -- 172.21.15.107:0/1052002591 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c5ee556fb0 msgr2=0x55c5ee544570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:14:49.864 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.887+0000 7f0f49e00640 1 --2- 172.21.15.107:0/1052002591 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c5ee556fb0 0x55c5ee544570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:14:49.864 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.887+0000 7f0f49e00640 1 -- 172.21.15.107:0/1052002591 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c5ee540e80 msgr2=0x55c5ee549a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:14:49.864 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.887+0000 7f0f49e00640 1 --2- 172.21.15.107:0/1052002591 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c5ee540e80 0x55c5ee549a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:14:49.864 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.887+0000 7f0f49e00640 1 -- 172.21.15.107:0/1052002591 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c5ee5481a0 con 0x55c5ee54a570 2024-03-08T21:14:49.867 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.887+0000 7f0f49e00640 1 --2- 172.21.15.107:0/1052002591 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c5ee54a570 0x55c5ee549f60 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f0f38003bf0 tx=0x7f0f38005c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b3ef78c98e1135c server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:14:49.867 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.887+0000 7f0f36a00640 1 -- 172.21.15.107:0/1052002591 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f3800fe60 con 0x55c5ee54a570 2024-03-08T21:14:49.867 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.888+0000 7f0f36a00640 1 -- 172.21.15.107:0/1052002591 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f0f38003de0 con 0x55c5ee54a570 2024-03-08T21:14:49.867 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.888+0000 7f0f36a00640 1 -- 172.21.15.107:0/1052002591 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f38005650 con 0x55c5ee54a570 2024-03-08T21:14:49.867 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.888+0000 7f0f5407e740 1 -- 172.21.15.107:0/1052002591 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c5ee54a570 msgr2=0x55c5ee549f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:14:49.867 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.888+0000 7f0f5407e740 1 --2- 172.21.15.107:0/1052002591 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c5ee54a570 0x55c5ee549f60 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f0f38003bf0 tx=0x7f0f38005c50 comp rx=0 tx=0).stop 2024-03-08T21:14:49.867 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.888+0000 7f0f5407e740 1 -- 172.21.15.107:0/1052002591 shutdown_connections 2024-03-08T21:14:49.867 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.888+0000 7f0f5407e740 1 --2- 172.21.15.107:0/1052002591 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c5ee556fb0 0x55c5ee544570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:14:49.868 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.888+0000 7f0f5407e740 1 --2- 172.21.15.107:0/1052002591 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c5ee54a570 0x55c5ee549f60 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:14:49.868 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.888+0000 7f0f5407e740 1 --2- 172.21.15.107:0/1052002591 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c5ee540e80 0x55c5ee549a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:14:49.868 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.889+0000 7f0f5407e740 1 -- 172.21.15.107:0/1052002591 >> 172.21.15.107:0/1052002591 conn(0x55c5ee53b7a0 msgr2=0x55c5ee545bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:14:49.868 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.889+0000 7f0f5407e740 1 -- 172.21.15.107:0/1052002591 shutdown_connections 2024-03-08T21:14:49.868 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.889+0000 7f0f5407e740 1 -- 172.21.15.107:0/1052002591 wait complete. 2024-03-08T21:14:49.868 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.889+0000 7f0f5407e740 1 -- start start 2024-03-08T21:14:49.868 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.889+0000 7f0f5407e740 1 Processor -- start 2024-03-08T21:14:49.868 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.890+0000 7f0f5407e740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c5ee540e80 0x55c5ee5518b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:14:49.868 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.890+0000 7f0f5407e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c5ee54a570 0x55c5ee54fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:14:49.868 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.890+0000 7f0f5407e740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c5ee556fb0 0x55c5ee550230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:14:49.868 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.890+0000 7f0f5407e740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f4c1af3b0 con 0x55c5ee54a570 2024-03-08T21:14:49.868 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.890+0000 7f0f5407e740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f4c1aed50 con 0x55c5ee556fb0 2024-03-08T21:14:49.868 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.890+0000 7f0f5407e740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f4c1afa10 con 0x55c5ee540e80 2024-03-08T21:14:49.868 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.890+0000 7f0f5407e740 10 client.0 _ll_register_callbacks cb 0x7f0f3800fa00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T21:14:49.869 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.890+0000 7f0f49e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c5ee54a570 0x55c5ee54fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:14:49.869 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.890+0000 7f0f49e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c5ee54a570 0x55c5ee54fcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:38808/0 (socket says 172.21.15.107:38808) 2024-03-08T21:14:49.869 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.890+0000 7f0f49e00640 1 -- 172.21.15.107:0/2981748692 learned_addr learned my addr 172.21.15.107:0/2981748692 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:14:49.869 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.890+0000 7f0f4a800640 1 --2- 172.21.15.107:0/2981748692 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c5ee540e80 0x55c5ee5518b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:14:49.869 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.891+0000 7f0f4b200640 1 --2- 172.21.15.107:0/2981748692 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c5ee556fb0 0x55c5ee550230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:14:49.869 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.891+0000 7f0f49e00640 1 -- 172.21.15.107:0/2981748692 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c5ee540e80 msgr2=0x55c5ee5518b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:14:49.869 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.891+0000 7f0f49e00640 1 --2- 172.21.15.107:0/2981748692 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c5ee540e80 0x55c5ee5518b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:14:49.869 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.891+0000 7f0f49e00640 1 -- 172.21.15.107:0/2981748692 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c5ee556fb0 msgr2=0x55c5ee550230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:14:49.869 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.891+0000 7f0f49e00640 1 --2- 172.21.15.107:0/2981748692 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c5ee556fb0 0x55c5ee550230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:14:49.869 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.891+0000 7f0f49e00640 1 -- 172.21.15.107:0/2981748692 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c5ee550a40 con 0x55c5ee54a570 2024-03-08T21:14:49.869 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.891+0000 7f0f49e00640 1 --2- 172.21.15.107:0/2981748692 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c5ee54a570 0x55c5ee54fcf0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f0f38014920 tx=0x7f0f38003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c4026fcde8010f2 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:14:49.869 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.891+0000 7f0f2be00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-03-08T21:14:49.871 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.891+0000 7f0f2be00640 1 -- 172.21.15.107:0/2981748692 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f380037f0 con 0x55c5ee54a570 2024-03-08T21:14:49.871 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.891+0000 7f0f2be00640 1 -- 172.21.15.107:0/2981748692 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f0f380052d0 con 0x55c5ee54a570 2024-03-08T21:14:49.871 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.891+0000 7f0f5407e740 1 -- 172.21.15.107:0/2981748692 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0f38003de0 con 0x55c5ee54a570 2024-03-08T21:14:49.871 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.892+0000 7f0f2be00640 1 -- 172.21.15.107:0/2981748692 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0f38005510 con 0x55c5ee54a570 2024-03-08T21:14:49.871 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.892+0000 7f0f5407e740 10 client.16653 fetch_fsmap learned FSMap version 41 2024-03-08T21:14:49.871 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.892+0000 7f0f5407e740 1 -- 172.21.15.107:0/2981748692 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x55c5ee5b1740 con 0x55c5ee54a570 2024-03-08T21:14:49.871 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.892+0000 7f0f2be00640 1 -- 172.21.15.107:0/2981748692 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f0f38003de0 con 0x55c5ee54a570 2024-03-08T21:14:49.872 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.892+0000 7f0f2be00640 1 -- 172.21.15.107:0/2981748692 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 41) v1 ==== 7307+0+0 (secure 0 0 0) 0x7f0f38047030 con 0x55c5ee54a570 2024-03-08T21:14:49.872 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.892+0000 7f0f5407e740 10 client.16653 fetch_fsmap finished waiting for FSMap version 41 2024-03-08T21:14:49.872 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.892+0000 7f0f5407e740 10 client.16653 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-08T21:14:49.872 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.892+0000 7f0f5407e740 20 client.16653 populate_metadata read hostname 'smithi107' 2024-03-08T21:14:49.872 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.892+0000 7f0f5407e740 1 --2- 172.21.15.107:0/2981748692 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55c5ee5b1df0 0x55c5ee5b4210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:14:49.872 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.892+0000 7f0f5407e740 4 client.16653 mds_command: new command op to 34475 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-08T21:14:49.872 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.892+0000 7f0f5407e740 1 -- 172.21.15.107:0/2981748692 --> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55c5ee4abc60 con 0x55c5ee5b1df0 2024-03-08T21:14:49.872 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.893+0000 7f0f4a800640 1 --2- 172.21.15.107:0/2981748692 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55c5ee5b1df0 0x55c5ee5b4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:14:49.872 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.894+0000 7f0f4a800640 1 --2- 172.21.15.107:0/2981748692 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55c5ee5b1df0 0x55c5ee5b4210 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ce84a7d0143b5673 server_cookie=f80782c08a181db2 in_seq=0 out_seq=0 2024-03-08T21:14:49.872 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.894+0000 7f0f2be00640 10 client.16653 ms_handle_connect on v2:172.21.15.149:6838/55290596 2024-03-08T21:14:49.881 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.904+0000 7f0f2be00640 1 -- 172.21.15.107:0/2981748692 <== mds.0 v2:172.21.15.149:6838/55290596 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55c5ee4abc60 con 0x55c5ee5b1df0 2024-03-08T21:14:49.881 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.904+0000 7f0f2be00640 10 client.16653 handle_command_reply: tid=0 2024-03-08T21:14:49.881 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.904+0000 7f0f5407e740 1 client.16653 shutdown 2024-03-08T21:14:49.881 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.904+0000 7f0f5407e740 1 -- 172.21.15.107:0/2981748692 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c5ee54a570 msgr2=0x55c5ee54fcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:14:49.881 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.904+0000 7f0f5407e740 1 --2- 172.21.15.107:0/2981748692 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c5ee54a570 0x55c5ee54fcf0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f0f38014920 tx=0x7f0f38003f90 comp rx=0 tx=0).stop 2024-03-08T21:14:49.881 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.904+0000 7f0f5407e740 1 -- 172.21.15.107:0/2981748692 shutdown_connections 2024-03-08T21:14:49.881 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.904+0000 7f0f5407e740 1 --2- 172.21.15.107:0/2981748692 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55c5ee5b1df0 0x55c5ee5b4210 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:14:49.881 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.904+0000 7f0f5407e740 1 --2- 172.21.15.107:0/2981748692 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55c5ee556fb0 0x55c5ee550230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:14:49.881 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.904+0000 7f0f5407e740 1 --2- 172.21.15.107:0/2981748692 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c5ee54a570 0x55c5ee54fcf0 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:14:49.881 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.904+0000 7f0f5407e740 1 --2- 172.21.15.107:0/2981748692 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55c5ee540e80 0x55c5ee5518b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:14:49.881 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.904+0000 7f0f5407e740 1 -- 172.21.15.107:0/2981748692 >> 172.21.15.107:0/2981748692 conn(0x55c5ee53b7a0 msgr2=0x55c5ee539f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:14:49.882 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.904+0000 7f0f2be00640 0 client.16653 ms_handle_reset on v2:172.21.15.149:6838/55290596 2024-03-08T21:14:49.882 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.904+0000 7f0f5407e740 1 -- 172.21.15.107:0/2981748692 shutdown_connections 2024-03-08T21:14:49.882 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.904+0000 7f0f5407e740 1 -- 172.21.15.107:0/2981748692 wait complete. 2024-03-08T21:14:49.882 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.905+0000 7f0f5407e740 20 client.16653 trim_cache size 0 max 16384 2024-03-08T21:14:49.882 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:{ 2024-03-08T21:14:49.882 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "message": "", 2024-03-08T21:14:49.882 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "return_code": 0 2024-03-08T21:14:49.882 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:} 2024-03-08T21:14:49.883 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.905+0000 7f0f50a00640 1 -- 172.21.15.107:0/3131660654 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f0f300a2960 msgr2=0x7f0f300a4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:14:49.883 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.905+0000 7f0f50a00640 1 --2- 172.21.15.107:0/3131660654 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f0f300a2960 0x7f0f300a4e20 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x55c5ee540930 tx=0x7f0f3c00b390 comp rx=0 tx=0).stop 2024-03-08T21:14:49.883 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.905+0000 7f0f50a00640 1 -- 172.21.15.107:0/3131660654 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0f4c199910 msgr2=0x7f0f4c199dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:14:49.883 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.905+0000 7f0f50a00640 1 --2- 172.21.15.107:0/3131660654 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0f4c199910 0x7f0f4c199dc0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f0f4402c700 tx=0x7f0f44035d60 comp rx=0 tx=0).stop 2024-03-08T21:14:49.883 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.905+0000 7f0f50a00640 1 -- 172.21.15.107:0/3131660654 shutdown_connections 2024-03-08T21:14:49.883 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.905+0000 7f0f50a00640 1 --2- 172.21.15.107:0/3131660654 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f0f300a2960 0x7f0f300a4e20 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:14:49.883 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.905+0000 7f0f50a00640 1 --2- 172.21.15.107:0/3131660654 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0f4c199910 0x7f0f4c199dc0 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:14:49.884 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.905+0000 7f0f50a00640 1 --2- 172.21.15.107:0/3131660654 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0f4c108620 0x7f0f4c19fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:14:49.884 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.905+0000 7f0f50a00640 1 --2- 172.21.15.107:0/3131660654 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f0f4c103550 0x7f0f4c19f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:14:49.884 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.905+0000 7f0f50a00640 1 -- 172.21.15.107:0/3131660654 >> 172.21.15.107:0/3131660654 conn(0x7f0f4c0fedd0 msgr2=0x7f0f4c100180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:14:49.884 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.906+0000 7f0f50a00640 1 -- 172.21.15.107:0/3131660654 shutdown_connections 2024-03-08T21:14:49.884 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:14:49.906+0000 7f0f50a00640 1 -- 172.21.15.107:0/3131660654 wait complete. 2024-03-08T21:14:50.711 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:14:50 smithi107 ceph-mon[31381]: pgmap v453: 97 pgs: 97 active+clean; 9.5 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-08T21:14:50.758 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:14:50 smithi149 ceph-mon[35042]: pgmap v453: 97 pgs: 97 active+clean; 9.5 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-08T21:14:50.872 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:14:50 smithi204 ceph-mon[34935]: pgmap v453: 97 pgs: 97 active+clean; 9.5 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-08T21:14:52.711 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:14:52 smithi107 ceph-mon[31381]: pgmap v454: 97 pgs: 97 active+clean; 9.5 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-08T21:14:52.758 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:14:52 smithi149 ceph-mon[35042]: pgmap v454: 97 pgs: 97 active+clean; 9.5 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-08T21:14:52.872 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:14:52 smithi204 ceph-mon[34935]: pgmap v454: 97 pgs: 97 active+clean; 9.5 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-08T21:14:54.711 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:14:54 smithi107 ceph-mon[31381]: pgmap v455: 97 pgs: 97 active+clean; 9.5 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-08T21:14:54.758 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:14:54 smithi149 ceph-mon[35042]: pgmap v455: 97 pgs: 97 active+clean; 9.5 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-08T21:14:54.872 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:14:54 smithi204 ceph-mon[34935]: pgmap v455: 97 pgs: 97 active+clean; 9.5 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-08T21:14:56.711 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:14:56 smithi107 ceph-mon[31381]: pgmap v456: 97 pgs: 97 active+clean; 9.5 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-08T21:14:56.758 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:14:56 smithi149 ceph-mon[35042]: pgmap v456: 97 pgs: 97 active+clean; 9.5 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-08T21:14:56.872 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:14:56 smithi204 ceph-mon[34935]: pgmap v456: 97 pgs: 97 active+clean; 9.5 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-03-08T21:14:57.571 INFO:tasks.workunit.client.0.smithi107.stdout:writing pattern 2024-03-08T21:14:57.571 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 0 offset 4190208 len 1024 2024-03-08T21:14:57.571 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 0 offset 4190208 len 1024 2024-03-08T21:14:57.571 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 0 offset 4190208 len 1024 2024-03-08T21:14:57.571 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 0 offset 4190208 len 1024 2024-03-08T21:14:57.571 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 0 offset 4190208 len 2048 2024-03-08T21:14:57.571 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 0 offset 4190208 len 2048 2024-03-08T21:14:57.572 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 0 offset 4190208 len 2048 2024-03-08T21:14:57.572 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 0 offset 4190208 len 2048 2024-03-08T21:14:57.572 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 0 offset 4190208 len 4096 2024-03-08T21:14:57.572 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 0 offset 4190208 len 4096 2024-03-08T21:14:57.572 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 0 offset 4190208 len 4096 2024-03-08T21:14:57.572 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 0 offset 4190208 len 4096 2024-03-08T21:14:57.572 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 0 offset 4190208 len 8192 2024-03-08T21:14:57.572 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 0 offset 4190208 len 8192 2024-03-08T21:14:57.572 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 0 offset 4190208 len 8192 2024-03-08T21:14:57.572 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 0 offset 4190208 len 8192 2024-03-08T21:14:57.572 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 0 offset 4190208 len 16384 2024-03-08T21:14:57.572 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 0 offset 4190208 len 16384 2024-03-08T21:14:57.572 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 0 offset 4190208 len 16384 2024-03-08T21:14:57.573 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 0 offset 4190208 len 16384 2024-03-08T21:14:57.573 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 0 offset 4190720 len 1024 2024-03-08T21:14:57.573 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 0 offset 4190720 len 1024 2024-03-08T21:14:57.573 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 0 offset 4190720 len 1024 2024-03-08T21:14:57.573 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 0 offset 4190720 len 1024 2024-03-08T21:14:57.573 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 0 offset 4190720 len 2048 2024-03-08T21:14:57.573 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 0 offset 4190720 len 2048 2024-03-08T21:14:57.573 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 0 offset 4190720 len 2048 2024-03-08T21:14:57.573 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 0 offset 4190720 len 2048 2024-03-08T21:14:57.573 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 0 offset 4190720 len 4096 2024-03-08T21:14:57.573 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 0 offset 4190720 len 4096 2024-03-08T21:14:57.573 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 0 offset 4190720 len 4096 2024-03-08T21:14:57.574 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 0 offset 4190720 len 4096 2024-03-08T21:14:57.574 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 0 offset 4190720 len 8192 2024-03-08T21:14:57.574 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 0 offset 4190720 len 8192 2024-03-08T21:14:57.574 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 0 offset 4190720 len 8192 2024-03-08T21:14:57.574 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 0 offset 4190720 len 8192 2024-03-08T21:14:57.574 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 0 offset 4190720 len 16384 2024-03-08T21:14:57.574 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 0 offset 4190720 len 16384 2024-03-08T21:14:57.574 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 0 offset 4190720 len 16384 2024-03-08T21:14:57.574 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 0 offset 4190720 len 16384 2024-03-08T21:14:57.574 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 0 offset 4191232 len 1024 2024-03-08T21:14:57.574 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 0 offset 4191232 len 1024 2024-03-08T21:14:57.574 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 0 offset 4191232 len 1024 2024-03-08T21:14:57.574 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 0 offset 4191232 len 1024 2024-03-08T21:14:57.575 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 0 offset 4191232 len 2048 2024-03-08T21:14:57.575 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 0 offset 4191232 len 2048 2024-03-08T21:14:57.575 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 0 offset 4191232 len 2048 2024-03-08T21:14:57.575 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 0 offset 4191232 len 2048 2024-03-08T21:14:57.575 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 0 offset 4191232 len 4096 2024-03-08T21:14:57.575 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 0 offset 4191232 len 4096 2024-03-08T21:14:57.575 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 0 offset 4191232 len 4096 2024-03-08T21:14:57.575 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 0 offset 4191232 len 4096 2024-03-08T21:14:57.575 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 0 offset 4191232 len 8192 2024-03-08T21:14:57.575 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 0 offset 4191232 len 8192 2024-03-08T21:14:57.575 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 0 offset 4191232 len 8192 2024-03-08T21:14:57.575 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 0 offset 4191232 len 8192 2024-03-08T21:14:57.575 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 0 offset 4191232 len 16384 2024-03-08T21:14:57.575 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 0 offset 4191232 len 16384 2024-03-08T21:14:57.576 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 0 offset 4191232 len 16384 2024-03-08T21:14:57.576 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 0 offset 4191232 len 16384 2024-03-08T21:14:57.576 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 0 offset 4191744 len 1024 2024-03-08T21:14:57.576 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 0 offset 4191744 len 1024 2024-03-08T21:14:57.576 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 0 offset 4191744 len 1024 2024-03-08T21:14:57.576 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 0 offset 4191744 len 1024 2024-03-08T21:14:57.576 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 0 offset 4191744 len 2048 2024-03-08T21:14:57.576 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 0 offset 4191744 len 2048 2024-03-08T21:14:57.576 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 0 offset 4191744 len 2048 2024-03-08T21:14:57.576 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 0 offset 4191744 len 2048 2024-03-08T21:14:57.576 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 0 offset 4191744 len 4096 2024-03-08T21:14:57.576 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 0 offset 4191744 len 4096 2024-03-08T21:14:57.576 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 0 offset 4191744 len 4096 2024-03-08T21:14:57.576 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 0 offset 4191744 len 4096 2024-03-08T21:14:57.577 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 0 offset 4191744 len 8192 2024-03-08T21:14:57.577 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 0 offset 4191744 len 8192 2024-03-08T21:14:57.577 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 0 offset 4191744 len 8192 2024-03-08T21:14:57.577 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 0 offset 4191744 len 8192 2024-03-08T21:14:57.577 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 0 offset 4191744 len 16384 2024-03-08T21:14:57.577 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 0 offset 4191744 len 16384 2024-03-08T21:14:57.577 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 0 offset 4191744 len 16384 2024-03-08T21:14:57.577 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 0 offset 4191744 len 16384 2024-03-08T21:14:57.577 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 0 offset 4192256 len 1024 2024-03-08T21:14:57.577 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 0 offset 4192256 len 1024 2024-03-08T21:14:57.577 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 0 offset 4192256 len 1024 2024-03-08T21:14:57.578 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 0 offset 4192256 len 1024 2024-03-08T21:14:57.578 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 0 offset 4192256 len 2048 2024-03-08T21:14:57.578 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 0 offset 4192256 len 2048 2024-03-08T21:14:57.578 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 0 offset 4192256 len 2048 2024-03-08T21:14:57.672 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 0 offset 4192256 len 2048 2024-03-08T21:14:57.672 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 0 offset 4192256 len 4096 2024-03-08T21:14:57.672 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 0 offset 4192256 len 4096 2024-03-08T21:14:57.672 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 0 offset 4192256 len 4096 2024-03-08T21:14:57.672 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 0 offset 4192256 len 4096 2024-03-08T21:14:57.672 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 0 offset 4192256 len 8192 2024-03-08T21:14:57.672 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 0 offset 4192256 len 8192 2024-03-08T21:14:57.672 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 0 offset 4192256 len 8192 2024-03-08T21:14:57.672 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 0 offset 4192256 len 8192 2024-03-08T21:14:57.672 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 0 offset 4192256 len 16384 2024-03-08T21:14:57.672 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 0 offset 4192256 len 16384 2024-03-08T21:14:57.673 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 0 offset 4192256 len 16384 2024-03-08T21:14:57.673 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 0 offset 4192256 len 16384 2024-03-08T21:14:57.673 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 0 offset 4192768 len 1024 2024-03-08T21:14:57.673 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 0 offset 4192768 len 1024 2024-03-08T21:14:57.673 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 0 offset 4192768 len 1024 2024-03-08T21:14:57.673 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 0 offset 4192768 len 1024 2024-03-08T21:14:57.673 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 0 offset 4192768 len 2048 2024-03-08T21:14:57.673 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 0 offset 4192768 len 2048 2024-03-08T21:14:57.673 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 0 offset 4192768 len 2048 2024-03-08T21:14:57.673 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 0 offset 4192768 len 2048 2024-03-08T21:14:57.673 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 0 offset 4192768 len 4096 2024-03-08T21:14:57.673 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 0 offset 4192768 len 4096 2024-03-08T21:14:57.673 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 0 offset 4192768 len 4096 2024-03-08T21:14:57.673 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 0 offset 4192768 len 4096 2024-03-08T21:14:57.674 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 0 offset 4192768 len 8192 2024-03-08T21:14:57.674 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 0 offset 4192768 len 8192 2024-03-08T21:14:57.674 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 0 offset 4192768 len 8192 2024-03-08T21:14:57.674 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 0 offset 4192768 len 8192 2024-03-08T21:14:57.674 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 0 offset 4192768 len 16384 2024-03-08T21:14:57.674 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 0 offset 4192768 len 16384 2024-03-08T21:14:57.674 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 0 offset 4192768 len 16384 2024-03-08T21:14:57.674 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 0 offset 4192768 len 16384 2024-03-08T21:14:57.674 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 0 offset 4193280 len 1024 2024-03-08T21:14:57.674 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 0 offset 4193280 len 1024 2024-03-08T21:14:57.674 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 0 offset 4193280 len 1024 2024-03-08T21:14:57.675 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 0 offset 4193280 len 1024 2024-03-08T21:14:57.675 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 0 offset 4193280 len 2048 2024-03-08T21:14:57.675 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 0 offset 4193280 len 2048 2024-03-08T21:14:57.675 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 0 offset 4193280 len 2048 2024-03-08T21:14:57.675 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 0 offset 4193280 len 2048 2024-03-08T21:14:57.675 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 0 offset 4193280 len 4096 2024-03-08T21:14:57.675 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 0 offset 4193280 len 4096 2024-03-08T21:14:57.675 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 0 offset 4193280 len 4096 2024-03-08T21:14:57.675 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 0 offset 4193280 len 4096 2024-03-08T21:14:57.675 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 0 offset 4193280 len 8192 2024-03-08T21:14:57.675 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 0 offset 4193280 len 8192 2024-03-08T21:14:57.675 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 0 offset 4193280 len 8192 2024-03-08T21:14:57.675 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 0 offset 4193280 len 8192 2024-03-08T21:14:57.675 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 0 offset 4193280 len 16384 2024-03-08T21:14:57.675 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 0 offset 4193280 len 16384 2024-03-08T21:14:57.676 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 0 offset 4193280 len 16384 2024-03-08T21:14:57.676 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 0 offset 4193280 len 16384 2024-03-08T21:14:57.676 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 0 offset 4193792 len 1024 2024-03-08T21:14:57.676 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 0 offset 4193792 len 1024 2024-03-08T21:14:57.676 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 0 offset 4193792 len 1024 2024-03-08T21:14:57.676 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 0 offset 4193792 len 1024 2024-03-08T21:14:57.676 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 0 offset 4193792 len 2048 2024-03-08T21:14:57.676 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 0 offset 4193792 len 2048 2024-03-08T21:14:57.676 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 0 offset 4193792 len 2048 2024-03-08T21:14:57.676 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 0 offset 4193792 len 2048 2024-03-08T21:14:57.676 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 0 offset 4193792 len 4096 2024-03-08T21:14:57.676 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 0 offset 4193792 len 4096 2024-03-08T21:14:57.676 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 0 offset 4193792 len 4096 2024-03-08T21:14:57.676 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 0 offset 4193792 len 4096 2024-03-08T21:14:57.676 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 0 offset 4193792 len 8192 2024-03-08T21:14:57.677 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 0 offset 4193792 len 8192 2024-03-08T21:14:57.677 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 0 offset 4193792 len 8192 2024-03-08T21:14:57.677 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 0 offset 4193792 len 8192 2024-03-08T21:14:57.677 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 0 offset 4193792 len 16384 2024-03-08T21:14:57.677 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 0 offset 4193792 len 16384 2024-03-08T21:14:57.677 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 0 offset 4193792 len 16384 2024-03-08T21:14:57.677 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 0 offset 4193792 len 16384 2024-03-08T21:14:57.677 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 0 offset 4194304 len 1024 2024-03-08T21:14:57.677 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 0 offset 4194304 len 1024 2024-03-08T21:14:57.677 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 0 offset 4194304 len 1024 2024-03-08T21:14:57.677 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 0 offset 4194304 len 1024 2024-03-08T21:14:57.678 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 0 offset 4194304 len 2048 2024-03-08T21:14:57.678 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 0 offset 4194304 len 2048 2024-03-08T21:14:57.678 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 0 offset 4194304 len 2048 2024-03-08T21:14:57.678 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 0 offset 4194304 len 2048 2024-03-08T21:14:57.678 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 0 offset 4194304 len 4096 2024-03-08T21:14:57.678 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 0 offset 4194304 len 4096 2024-03-08T21:14:57.678 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 0 offset 4194304 len 4096 2024-03-08T21:14:57.678 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 0 offset 4194304 len 4096 2024-03-08T21:14:57.678 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 0 offset 4194304 len 8192 2024-03-08T21:14:57.678 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 0 offset 4194304 len 8192 2024-03-08T21:14:57.679 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 0 offset 4194304 len 8192 2024-03-08T21:14:57.768 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 0 offset 4194304 len 8192 2024-03-08T21:14:57.768 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 0 offset 4194304 len 16384 2024-03-08T21:14:57.768 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 0 offset 4194304 len 16384 2024-03-08T21:14:57.768 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 0 offset 4194304 len 16384 2024-03-08T21:14:57.768 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 0 offset 4194304 len 16384 2024-03-08T21:14:57.768 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 0 offset 4194816 len 1024 2024-03-08T21:14:57.768 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 0 offset 4194816 len 1024 2024-03-08T21:14:57.769 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 0 offset 4194816 len 1024 2024-03-08T21:14:57.769 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 0 offset 4194816 len 1024 2024-03-08T21:14:57.769 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 0 offset 4194816 len 2048 2024-03-08T21:14:57.769 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 0 offset 4194816 len 2048 2024-03-08T21:14:57.769 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 0 offset 4194816 len 2048 2024-03-08T21:14:57.769 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 0 offset 4194816 len 2048 2024-03-08T21:14:57.769 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 0 offset 4194816 len 4096 2024-03-08T21:14:57.769 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 0 offset 4194816 len 4096 2024-03-08T21:14:57.769 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 0 offset 4194816 len 4096 2024-03-08T21:14:57.769 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 0 offset 4194816 len 4096 2024-03-08T21:14:57.769 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 0 offset 4194816 len 8192 2024-03-08T21:14:57.769 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 0 offset 4194816 len 8192 2024-03-08T21:14:57.769 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 0 offset 4194816 len 8192 2024-03-08T21:14:57.769 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 0 offset 4194816 len 8192 2024-03-08T21:14:57.770 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 0 offset 4194816 len 16384 2024-03-08T21:14:57.770 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 0 offset 4194816 len 16384 2024-03-08T21:14:57.770 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 0 offset 4194816 len 16384 2024-03-08T21:14:57.770 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 0 offset 4194816 len 16384 2024-03-08T21:14:57.770 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 0 offset 4195328 len 1024 2024-03-08T21:14:57.770 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 0 offset 4195328 len 1024 2024-03-08T21:14:57.770 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 0 offset 4195328 len 1024 2024-03-08T21:14:57.770 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 0 offset 4195328 len 1024 2024-03-08T21:14:57.770 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 0 offset 4195328 len 2048 2024-03-08T21:14:57.770 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 0 offset 4195328 len 2048 2024-03-08T21:14:57.770 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 0 offset 4195328 len 2048 2024-03-08T21:14:57.770 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 0 offset 4195328 len 2048 2024-03-08T21:14:57.771 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 0 offset 4195328 len 4096 2024-03-08T21:14:57.771 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 0 offset 4195328 len 4096 2024-03-08T21:14:57.771 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 0 offset 4195328 len 4096 2024-03-08T21:14:57.771 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 0 offset 4195328 len 4096 2024-03-08T21:14:57.771 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 0 offset 4195328 len 8192 2024-03-08T21:14:57.771 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 0 offset 4195328 len 8192 2024-03-08T21:14:57.771 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 0 offset 4195328 len 8192 2024-03-08T21:14:57.771 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 0 offset 4195328 len 8192 2024-03-08T21:14:57.771 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 0 offset 4195328 len 16384 2024-03-08T21:14:57.771 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 0 offset 4195328 len 16384 2024-03-08T21:14:57.771 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 0 offset 4195328 len 16384 2024-03-08T21:14:57.771 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 0 offset 4195328 len 16384 2024-03-08T21:14:57.771 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 0 offset 4195840 len 1024 2024-03-08T21:14:57.771 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 0 offset 4195840 len 1024 2024-03-08T21:14:57.771 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 0 offset 4195840 len 1024 2024-03-08T21:14:57.772 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 0 offset 4195840 len 1024 2024-03-08T21:14:57.772 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 0 offset 4195840 len 2048 2024-03-08T21:14:57.772 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 0 offset 4195840 len 2048 2024-03-08T21:14:57.772 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 0 offset 4195840 len 2048 2024-03-08T21:14:57.772 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 0 offset 4195840 len 2048 2024-03-08T21:14:57.772 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 0 offset 4195840 len 4096 2024-03-08T21:14:57.772 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 0 offset 4195840 len 4096 2024-03-08T21:14:57.772 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 0 offset 4195840 len 4096 2024-03-08T21:14:57.772 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 0 offset 4195840 len 4096 2024-03-08T21:14:57.772 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 0 offset 4195840 len 8192 2024-03-08T21:14:57.772 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 0 offset 4195840 len 8192 2024-03-08T21:14:57.772 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 0 offset 4195840 len 8192 2024-03-08T21:14:57.772 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 0 offset 4195840 len 8192 2024-03-08T21:14:57.772 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 0 offset 4195840 len 16384 2024-03-08T21:14:57.772 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 0 offset 4195840 len 16384 2024-03-08T21:14:57.773 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 0 offset 4195840 len 16384 2024-03-08T21:14:57.773 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 0 offset 4195840 len 16384 2024-03-08T21:14:57.773 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 0 offset 4196352 len 1024 2024-03-08T21:14:57.773 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 0 offset 4196352 len 1024 2024-03-08T21:14:57.773 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 0 offset 4196352 len 1024 2024-03-08T21:14:57.773 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 0 offset 4196352 len 1024 2024-03-08T21:14:57.773 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 0 offset 4196352 len 2048 2024-03-08T21:14:57.773 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 0 offset 4196352 len 2048 2024-03-08T21:14:57.773 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 0 offset 4196352 len 2048 2024-03-08T21:14:57.773 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 0 offset 4196352 len 2048 2024-03-08T21:14:57.773 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 0 offset 4196352 len 4096 2024-03-08T21:14:57.773 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 0 offset 4196352 len 4096 2024-03-08T21:14:57.773 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 0 offset 4196352 len 4096 2024-03-08T21:14:57.773 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 0 offset 4196352 len 4096 2024-03-08T21:14:57.774 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 0 offset 4196352 len 8192 2024-03-08T21:14:57.774 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 0 offset 4196352 len 8192 2024-03-08T21:14:57.774 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 0 offset 4196352 len 8192 2024-03-08T21:14:57.774 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 0 offset 4196352 len 8192 2024-03-08T21:14:57.774 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 0 offset 4196352 len 16384 2024-03-08T21:14:57.774 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 0 offset 4196352 len 16384 2024-03-08T21:14:57.774 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 0 offset 4196352 len 16384 2024-03-08T21:14:57.774 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 0 offset 4196352 len 16384 2024-03-08T21:14:57.774 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 0 offset 4196864 len 1024 2024-03-08T21:14:57.774 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 0 offset 4196864 len 1024 2024-03-08T21:14:57.864 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 0 offset 4196864 len 1024 2024-03-08T21:14:57.864 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 0 offset 4196864 len 1024 2024-03-08T21:14:57.864 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 0 offset 4196864 len 2048 2024-03-08T21:14:57.865 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 0 offset 4196864 len 2048 2024-03-08T21:14:57.865 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 0 offset 4196864 len 2048 2024-03-08T21:14:57.865 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 0 offset 4196864 len 2048 2024-03-08T21:14:57.865 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 0 offset 4196864 len 4096 2024-03-08T21:14:57.865 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 0 offset 4196864 len 4096 2024-03-08T21:14:57.865 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 0 offset 4196864 len 4096 2024-03-08T21:14:57.865 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 0 offset 4196864 len 4096 2024-03-08T21:14:57.865 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 0 offset 4196864 len 8192 2024-03-08T21:14:57.865 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 0 offset 4196864 len 8192 2024-03-08T21:14:57.865 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 0 offset 4196864 len 8192 2024-03-08T21:14:57.865 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 0 offset 4196864 len 8192 2024-03-08T21:14:57.866 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 0 offset 4196864 len 16384 2024-03-08T21:14:57.866 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 0 offset 4196864 len 16384 2024-03-08T21:14:57.866 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 0 offset 4196864 len 16384 2024-03-08T21:14:57.866 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 0 offset 4196864 len 16384 2024-03-08T21:14:57.866 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 0 offset 4197376 len 1024 2024-03-08T21:14:57.866 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 0 offset 4197376 len 1024 2024-03-08T21:14:57.866 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 0 offset 4197376 len 1024 2024-03-08T21:14:57.866 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 0 offset 4197376 len 1024 2024-03-08T21:14:57.867 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 0 offset 4197376 len 2048 2024-03-08T21:14:57.867 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 0 offset 4197376 len 2048 2024-03-08T21:14:57.867 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 0 offset 4197376 len 2048 2024-03-08T21:14:57.867 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 0 offset 4197376 len 2048 2024-03-08T21:14:57.867 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 0 offset 4197376 len 4096 2024-03-08T21:14:57.867 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 0 offset 4197376 len 4096 2024-03-08T21:14:57.867 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 0 offset 4197376 len 4096 2024-03-08T21:14:57.867 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 0 offset 4197376 len 4096 2024-03-08T21:14:57.867 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 0 offset 4197376 len 8192 2024-03-08T21:14:57.867 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 0 offset 4197376 len 8192 2024-03-08T21:14:57.868 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 0 offset 4197376 len 8192 2024-03-08T21:14:57.868 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 0 offset 4197376 len 8192 2024-03-08T21:14:57.868 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 0 offset 4197376 len 16384 2024-03-08T21:14:57.868 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 0 offset 4197376 len 16384 2024-03-08T21:14:57.868 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 0 offset 4197376 len 16384 2024-03-08T21:14:57.868 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 0 offset 4197376 len 16384 2024-03-08T21:14:57.868 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 0 offset 4197888 len 1024 2024-03-08T21:14:57.868 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 0 offset 4197888 len 1024 2024-03-08T21:14:57.868 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 0 offset 4197888 len 1024 2024-03-08T21:14:57.868 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 0 offset 4197888 len 1024 2024-03-08T21:14:57.869 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 0 offset 4197888 len 2048 2024-03-08T21:14:57.869 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 0 offset 4197888 len 2048 2024-03-08T21:14:57.869 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 0 offset 4197888 len 2048 2024-03-08T21:14:57.869 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 0 offset 4197888 len 2048 2024-03-08T21:14:57.869 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 0 offset 4197888 len 4096 2024-03-08T21:14:57.869 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 0 offset 4197888 len 4096 2024-03-08T21:14:57.869 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 0 offset 4197888 len 4096 2024-03-08T21:14:57.869 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 0 offset 4197888 len 4096 2024-03-08T21:14:57.869 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 0 offset 4197888 len 8192 2024-03-08T21:14:57.869 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 0 offset 4197888 len 8192 2024-03-08T21:14:57.869 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 0 offset 4197888 len 8192 2024-03-08T21:14:57.869 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 0 offset 4197888 len 8192 2024-03-08T21:14:57.869 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 0 offset 4197888 len 16384 2024-03-08T21:14:57.869 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 0 offset 4197888 len 16384 2024-03-08T21:14:57.870 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 0 offset 4197888 len 16384 2024-03-08T21:14:57.870 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 0 offset 4197888 len 16384 2024-03-08T21:14:57.870 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 512 offset 4190208 len 1024 2024-03-08T21:14:57.870 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 512 offset 4190208 len 1024 2024-03-08T21:14:57.870 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 512 offset 4190208 len 1024 2024-03-08T21:14:57.870 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 512 offset 4190208 len 1024 2024-03-08T21:14:57.870 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 512 offset 4190208 len 2048 2024-03-08T21:14:57.870 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 512 offset 4190208 len 2048 2024-03-08T21:14:57.870 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 512 offset 4190208 len 2048 2024-03-08T21:14:57.870 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 512 offset 4190208 len 2048 2024-03-08T21:14:57.870 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 512 offset 4190208 len 4096 2024-03-08T21:14:57.870 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 512 offset 4190208 len 4096 2024-03-08T21:14:57.870 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 512 offset 4190208 len 4096 2024-03-08T21:14:57.871 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 512 offset 4190208 len 4096 2024-03-08T21:14:57.871 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 512 offset 4190208 len 8192 2024-03-08T21:14:57.871 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 512 offset 4190208 len 8192 2024-03-08T21:14:57.871 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 512 offset 4190208 len 8192 2024-03-08T21:14:57.871 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 512 offset 4190208 len 8192 2024-03-08T21:14:57.871 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 512 offset 4190208 len 16384 2024-03-08T21:14:57.871 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 512 offset 4190208 len 16384 2024-03-08T21:14:57.871 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 512 offset 4190208 len 16384 2024-03-08T21:14:57.871 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 512 offset 4190208 len 16384 2024-03-08T21:14:57.871 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 512 offset 4190720 len 1024 2024-03-08T21:14:57.871 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 512 offset 4190720 len 1024 2024-03-08T21:14:57.871 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 512 offset 4190720 len 1024 2024-03-08T21:14:57.871 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 512 offset 4190720 len 1024 2024-03-08T21:14:57.871 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 512 offset 4190720 len 2048 2024-03-08T21:14:57.872 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 512 offset 4190720 len 2048 2024-03-08T21:14:57.872 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 512 offset 4190720 len 2048 2024-03-08T21:14:57.872 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 512 offset 4190720 len 2048 2024-03-08T21:14:57.872 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 512 offset 4190720 len 4096 2024-03-08T21:14:57.964 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 512 offset 4190720 len 4096 2024-03-08T21:14:57.964 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 512 offset 4190720 len 4096 2024-03-08T21:14:57.964 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 512 offset 4190720 len 4096 2024-03-08T21:14:57.964 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 512 offset 4190720 len 8192 2024-03-08T21:14:57.965 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 512 offset 4190720 len 8192 2024-03-08T21:14:57.965 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 512 offset 4190720 len 8192 2024-03-08T21:14:57.965 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 512 offset 4190720 len 8192 2024-03-08T21:14:57.965 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 512 offset 4190720 len 16384 2024-03-08T21:14:57.965 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 512 offset 4190720 len 16384 2024-03-08T21:14:57.965 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 512 offset 4190720 len 16384 2024-03-08T21:14:57.965 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 512 offset 4190720 len 16384 2024-03-08T21:14:57.965 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 512 offset 4191232 len 1024 2024-03-08T21:14:57.966 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 512 offset 4191232 len 1024 2024-03-08T21:14:57.966 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 512 offset 4191232 len 1024 2024-03-08T21:14:57.966 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 512 offset 4191232 len 1024 2024-03-08T21:14:57.966 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 512 offset 4191232 len 2048 2024-03-08T21:14:57.966 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 512 offset 4191232 len 2048 2024-03-08T21:14:57.966 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 512 offset 4191232 len 2048 2024-03-08T21:14:57.966 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 512 offset 4191232 len 2048 2024-03-08T21:14:57.966 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 512 offset 4191232 len 4096 2024-03-08T21:14:57.966 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 512 offset 4191232 len 4096 2024-03-08T21:14:57.966 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 512 offset 4191232 len 4096 2024-03-08T21:14:57.966 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 512 offset 4191232 len 4096 2024-03-08T21:14:57.966 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 512 offset 4191232 len 8192 2024-03-08T21:14:57.967 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 512 offset 4191232 len 8192 2024-03-08T21:14:57.967 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 512 offset 4191232 len 8192 2024-03-08T21:14:57.967 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 512 offset 4191232 len 8192 2024-03-08T21:14:57.967 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 512 offset 4191232 len 16384 2024-03-08T21:14:57.967 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 512 offset 4191232 len 16384 2024-03-08T21:14:57.967 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 512 offset 4191232 len 16384 2024-03-08T21:14:57.967 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 512 offset 4191232 len 16384 2024-03-08T21:14:57.968 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 512 offset 4191744 len 1024 2024-03-08T21:14:57.968 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 512 offset 4191744 len 1024 2024-03-08T21:14:57.968 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 512 offset 4191744 len 1024 2024-03-08T21:14:57.968 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 512 offset 4191744 len 1024 2024-03-08T21:14:57.968 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 512 offset 4191744 len 2048 2024-03-08T21:14:57.968 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 512 offset 4191744 len 2048 2024-03-08T21:14:57.968 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 512 offset 4191744 len 2048 2024-03-08T21:14:57.968 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 512 offset 4191744 len 2048 2024-03-08T21:14:57.968 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 512 offset 4191744 len 4096 2024-03-08T21:14:57.968 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 512 offset 4191744 len 4096 2024-03-08T21:14:57.968 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 512 offset 4191744 len 4096 2024-03-08T21:14:57.968 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 512 offset 4191744 len 4096 2024-03-08T21:14:57.969 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 512 offset 4191744 len 8192 2024-03-08T21:14:57.969 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 512 offset 4191744 len 8192 2024-03-08T21:14:57.969 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 512 offset 4191744 len 8192 2024-03-08T21:14:57.969 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 512 offset 4191744 len 8192 2024-03-08T21:14:57.969 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 512 offset 4191744 len 16384 2024-03-08T21:14:57.969 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 512 offset 4191744 len 16384 2024-03-08T21:14:57.969 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 512 offset 4191744 len 16384 2024-03-08T21:14:57.969 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 512 offset 4191744 len 16384 2024-03-08T21:14:57.969 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 512 offset 4192256 len 1024 2024-03-08T21:14:57.969 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 512 offset 4192256 len 1024 2024-03-08T21:14:57.969 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 512 offset 4192256 len 1024 2024-03-08T21:14:57.969 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 512 offset 4192256 len 1024 2024-03-08T21:14:57.969 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 512 offset 4192256 len 2048 2024-03-08T21:14:57.970 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 512 offset 4192256 len 2048 2024-03-08T21:14:57.970 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 512 offset 4192256 len 2048 2024-03-08T21:14:57.970 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 512 offset 4192256 len 2048 2024-03-08T21:14:57.970 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 512 offset 4192256 len 4096 2024-03-08T21:14:57.970 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 512 offset 4192256 len 4096 2024-03-08T21:14:57.970 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 512 offset 4192256 len 4096 2024-03-08T21:14:57.970 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 512 offset 4192256 len 4096 2024-03-08T21:14:57.970 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 512 offset 4192256 len 8192 2024-03-08T21:14:57.970 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 512 offset 4192256 len 8192 2024-03-08T21:14:57.970 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 512 offset 4192256 len 8192 2024-03-08T21:14:57.970 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 512 offset 4192256 len 8192 2024-03-08T21:14:57.970 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 512 offset 4192256 len 16384 2024-03-08T21:14:57.970 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 512 offset 4192256 len 16384 2024-03-08T21:14:57.971 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 512 offset 4192256 len 16384 2024-03-08T21:14:57.971 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 512 offset 4192256 len 16384 2024-03-08T21:14:57.971 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 512 offset 4192768 len 1024 2024-03-08T21:14:57.971 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 512 offset 4192768 len 1024 2024-03-08T21:14:57.971 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 512 offset 4192768 len 1024 2024-03-08T21:14:57.971 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 512 offset 4192768 len 1024 2024-03-08T21:14:57.971 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 512 offset 4192768 len 2048 2024-03-08T21:14:57.971 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 512 offset 4192768 len 2048 2024-03-08T21:14:57.971 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 512 offset 4192768 len 2048 2024-03-08T21:14:57.971 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 512 offset 4192768 len 2048 2024-03-08T21:14:57.971 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 512 offset 4192768 len 4096 2024-03-08T21:14:57.972 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 512 offset 4192768 len 4096 2024-03-08T21:14:57.972 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 512 offset 4192768 len 4096 2024-03-08T21:14:57.972 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 512 offset 4192768 len 4096 2024-03-08T21:14:57.972 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 512 offset 4192768 len 8192 2024-03-08T21:14:58.059 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 512 offset 4192768 len 8192 2024-03-08T21:14:58.060 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 512 offset 4192768 len 8192 2024-03-08T21:14:58.060 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 512 offset 4192768 len 8192 2024-03-08T21:14:58.060 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 512 offset 4192768 len 16384 2024-03-08T21:14:58.060 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 512 offset 4192768 len 16384 2024-03-08T21:14:58.060 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 512 offset 4192768 len 16384 2024-03-08T21:14:58.060 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 512 offset 4192768 len 16384 2024-03-08T21:14:58.060 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 512 offset 4193280 len 1024 2024-03-08T21:14:58.060 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 512 offset 4193280 len 1024 2024-03-08T21:14:58.060 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 512 offset 4193280 len 1024 2024-03-08T21:14:58.060 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 512 offset 4193280 len 1024 2024-03-08T21:14:58.060 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 512 offset 4193280 len 2048 2024-03-08T21:14:58.061 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 512 offset 4193280 len 2048 2024-03-08T21:14:58.061 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 512 offset 4193280 len 2048 2024-03-08T21:14:58.061 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 512 offset 4193280 len 2048 2024-03-08T21:14:58.061 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 512 offset 4193280 len 4096 2024-03-08T21:14:58.061 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 512 offset 4193280 len 4096 2024-03-08T21:14:58.061 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 512 offset 4193280 len 4096 2024-03-08T21:14:58.061 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 512 offset 4193280 len 4096 2024-03-08T21:14:58.061 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 512 offset 4193280 len 8192 2024-03-08T21:14:58.061 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 512 offset 4193280 len 8192 2024-03-08T21:14:58.061 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 512 offset 4193280 len 8192 2024-03-08T21:14:58.061 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 512 offset 4193280 len 8192 2024-03-08T21:14:58.061 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 512 offset 4193280 len 16384 2024-03-08T21:14:58.061 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 512 offset 4193280 len 16384 2024-03-08T21:14:58.061 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 512 offset 4193280 len 16384 2024-03-08T21:14:58.061 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 512 offset 4193280 len 16384 2024-03-08T21:14:58.062 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 512 offset 4193792 len 1024 2024-03-08T21:14:58.062 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 512 offset 4193792 len 1024 2024-03-08T21:14:58.062 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 512 offset 4193792 len 1024 2024-03-08T21:14:58.062 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 512 offset 4193792 len 1024 2024-03-08T21:14:58.062 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 512 offset 4193792 len 2048 2024-03-08T21:14:58.062 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 512 offset 4193792 len 2048 2024-03-08T21:14:58.062 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 512 offset 4193792 len 2048 2024-03-08T21:14:58.062 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 512 offset 4193792 len 2048 2024-03-08T21:14:58.062 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 512 offset 4193792 len 4096 2024-03-08T21:14:58.062 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 512 offset 4193792 len 4096 2024-03-08T21:14:58.062 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 512 offset 4193792 len 4096 2024-03-08T21:14:58.062 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 512 offset 4193792 len 4096 2024-03-08T21:14:58.062 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 512 offset 4193792 len 8192 2024-03-08T21:14:58.063 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 512 offset 4193792 len 8192 2024-03-08T21:14:58.063 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 512 offset 4193792 len 8192 2024-03-08T21:14:58.063 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 512 offset 4193792 len 8192 2024-03-08T21:14:58.063 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 512 offset 4193792 len 16384 2024-03-08T21:14:58.063 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 512 offset 4193792 len 16384 2024-03-08T21:14:58.063 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 512 offset 4193792 len 16384 2024-03-08T21:14:58.063 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 512 offset 4193792 len 16384 2024-03-08T21:14:58.063 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 512 offset 4194304 len 1024 2024-03-08T21:14:58.063 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 512 offset 4194304 len 1024 2024-03-08T21:14:58.063 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 512 offset 4194304 len 1024 2024-03-08T21:14:58.063 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 512 offset 4194304 len 1024 2024-03-08T21:14:58.063 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 512 offset 4194304 len 2048 2024-03-08T21:14:58.063 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 512 offset 4194304 len 2048 2024-03-08T21:14:58.063 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 512 offset 4194304 len 2048 2024-03-08T21:14:58.063 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 512 offset 4194304 len 2048 2024-03-08T21:14:58.064 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 512 offset 4194304 len 4096 2024-03-08T21:14:58.064 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 512 offset 4194304 len 4096 2024-03-08T21:14:58.064 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 512 offset 4194304 len 4096 2024-03-08T21:14:58.064 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 512 offset 4194304 len 4096 2024-03-08T21:14:58.064 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 512 offset 4194304 len 8192 2024-03-08T21:14:58.064 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 512 offset 4194304 len 8192 2024-03-08T21:14:58.064 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 512 offset 4194304 len 8192 2024-03-08T21:14:58.064 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 512 offset 4194304 len 8192 2024-03-08T21:14:58.064 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 512 offset 4194304 len 16384 2024-03-08T21:14:58.064 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 512 offset 4194304 len 16384 2024-03-08T21:14:58.064 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 512 offset 4194304 len 16384 2024-03-08T21:14:58.064 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 512 offset 4194304 len 16384 2024-03-08T21:14:58.064 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 512 offset 4194816 len 1024 2024-03-08T21:14:58.064 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 512 offset 4194816 len 1024 2024-03-08T21:14:58.065 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 512 offset 4194816 len 1024 2024-03-08T21:14:58.065 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 512 offset 4194816 len 1024 2024-03-08T21:14:58.065 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 512 offset 4194816 len 2048 2024-03-08T21:14:58.065 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 512 offset 4194816 len 2048 2024-03-08T21:14:58.065 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 512 offset 4194816 len 2048 2024-03-08T21:14:58.065 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 512 offset 4194816 len 2048 2024-03-08T21:14:58.065 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 512 offset 4194816 len 4096 2024-03-08T21:14:58.065 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 512 offset 4194816 len 4096 2024-03-08T21:14:58.065 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 512 offset 4194816 len 4096 2024-03-08T21:14:58.065 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 512 offset 4194816 len 4096 2024-03-08T21:14:58.065 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 512 offset 4194816 len 8192 2024-03-08T21:14:58.065 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 512 offset 4194816 len 8192 2024-03-08T21:14:58.065 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 512 offset 4194816 len 8192 2024-03-08T21:14:58.065 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 512 offset 4194816 len 8192 2024-03-08T21:14:58.066 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 512 offset 4194816 len 16384 2024-03-08T21:14:58.155 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 512 offset 4194816 len 16384 2024-03-08T21:14:58.155 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 512 offset 4194816 len 16384 2024-03-08T21:14:58.155 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 512 offset 4194816 len 16384 2024-03-08T21:14:58.155 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 512 offset 4195328 len 1024 2024-03-08T21:14:58.155 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 512 offset 4195328 len 1024 2024-03-08T21:14:58.155 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 512 offset 4195328 len 1024 2024-03-08T21:14:58.156 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 512 offset 4195328 len 1024 2024-03-08T21:14:58.156 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 512 offset 4195328 len 2048 2024-03-08T21:14:58.156 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 512 offset 4195328 len 2048 2024-03-08T21:14:58.156 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 512 offset 4195328 len 2048 2024-03-08T21:14:58.156 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 512 offset 4195328 len 2048 2024-03-08T21:14:58.156 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 512 offset 4195328 len 4096 2024-03-08T21:14:58.156 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 512 offset 4195328 len 4096 2024-03-08T21:14:58.156 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 512 offset 4195328 len 4096 2024-03-08T21:14:58.156 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 512 offset 4195328 len 4096 2024-03-08T21:14:58.156 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 512 offset 4195328 len 8192 2024-03-08T21:14:58.156 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 512 offset 4195328 len 8192 2024-03-08T21:14:58.156 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 512 offset 4195328 len 8192 2024-03-08T21:14:58.156 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 512 offset 4195328 len 8192 2024-03-08T21:14:58.156 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 512 offset 4195328 len 16384 2024-03-08T21:14:58.157 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 512 offset 4195328 len 16384 2024-03-08T21:14:58.157 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 512 offset 4195328 len 16384 2024-03-08T21:14:58.157 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 512 offset 4195328 len 16384 2024-03-08T21:14:58.157 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 512 offset 4195840 len 1024 2024-03-08T21:14:58.157 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 512 offset 4195840 len 1024 2024-03-08T21:14:58.157 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 512 offset 4195840 len 1024 2024-03-08T21:14:58.157 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 512 offset 4195840 len 1024 2024-03-08T21:14:58.157 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 512 offset 4195840 len 2048 2024-03-08T21:14:58.157 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 512 offset 4195840 len 2048 2024-03-08T21:14:58.157 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 512 offset 4195840 len 2048 2024-03-08T21:14:58.157 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 512 offset 4195840 len 2048 2024-03-08T21:14:58.157 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 512 offset 4195840 len 4096 2024-03-08T21:14:58.157 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 512 offset 4195840 len 4096 2024-03-08T21:14:58.157 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 512 offset 4195840 len 4096 2024-03-08T21:14:58.157 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 512 offset 4195840 len 4096 2024-03-08T21:14:58.158 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 512 offset 4195840 len 8192 2024-03-08T21:14:58.158 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 512 offset 4195840 len 8192 2024-03-08T21:14:58.158 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 512 offset 4195840 len 8192 2024-03-08T21:14:58.158 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 512 offset 4195840 len 8192 2024-03-08T21:14:58.158 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 512 offset 4195840 len 16384 2024-03-08T21:14:58.158 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 512 offset 4195840 len 16384 2024-03-08T21:14:58.158 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 512 offset 4195840 len 16384 2024-03-08T21:14:58.158 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 512 offset 4195840 len 16384 2024-03-08T21:14:58.158 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 512 offset 4196352 len 1024 2024-03-08T21:14:58.158 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 512 offset 4196352 len 1024 2024-03-08T21:14:58.158 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 512 offset 4196352 len 1024 2024-03-08T21:14:58.158 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 512 offset 4196352 len 1024 2024-03-08T21:14:58.158 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 512 offset 4196352 len 2048 2024-03-08T21:14:58.158 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 512 offset 4196352 len 2048 2024-03-08T21:14:58.159 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 512 offset 4196352 len 2048 2024-03-08T21:14:58.159 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 512 offset 4196352 len 2048 2024-03-08T21:14:58.159 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 512 offset 4196352 len 4096 2024-03-08T21:14:58.159 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 512 offset 4196352 len 4096 2024-03-08T21:14:58.159 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 512 offset 4196352 len 4096 2024-03-08T21:14:58.159 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 512 offset 4196352 len 4096 2024-03-08T21:14:58.159 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 512 offset 4196352 len 8192 2024-03-08T21:14:58.159 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 512 offset 4196352 len 8192 2024-03-08T21:14:58.159 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 512 offset 4196352 len 8192 2024-03-08T21:14:58.159 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 512 offset 4196352 len 8192 2024-03-08T21:14:58.159 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 512 offset 4196352 len 16384 2024-03-08T21:14:58.159 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 512 offset 4196352 len 16384 2024-03-08T21:14:58.159 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 512 offset 4196352 len 16384 2024-03-08T21:14:58.159 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 512 offset 4196352 len 16384 2024-03-08T21:14:58.159 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 512 offset 4196864 len 1024 2024-03-08T21:14:58.160 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 512 offset 4196864 len 1024 2024-03-08T21:14:58.160 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 512 offset 4196864 len 1024 2024-03-08T21:14:58.160 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 512 offset 4196864 len 1024 2024-03-08T21:14:58.160 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 512 offset 4196864 len 2048 2024-03-08T21:14:58.160 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 512 offset 4196864 len 2048 2024-03-08T21:14:58.160 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 512 offset 4196864 len 2048 2024-03-08T21:14:58.160 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 512 offset 4196864 len 2048 2024-03-08T21:14:58.160 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 512 offset 4196864 len 4096 2024-03-08T21:14:58.160 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 512 offset 4196864 len 4096 2024-03-08T21:14:58.160 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 512 offset 4196864 len 4096 2024-03-08T21:14:58.160 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 512 offset 4196864 len 4096 2024-03-08T21:14:58.160 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 512 offset 4196864 len 8192 2024-03-08T21:14:58.160 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 512 offset 4196864 len 8192 2024-03-08T21:14:58.160 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 512 offset 4196864 len 8192 2024-03-08T21:14:58.161 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 512 offset 4196864 len 8192 2024-03-08T21:14:58.161 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 512 offset 4196864 len 16384 2024-03-08T21:14:58.161 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 512 offset 4196864 len 16384 2024-03-08T21:14:58.161 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 512 offset 4196864 len 16384 2024-03-08T21:14:58.161 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 512 offset 4196864 len 16384 2024-03-08T21:14:58.161 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 512 offset 4197376 len 1024 2024-03-08T21:14:58.253 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 512 offset 4197376 len 1024 2024-03-08T21:14:58.253 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 512 offset 4197376 len 1024 2024-03-08T21:14:58.253 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 512 offset 4197376 len 1024 2024-03-08T21:14:58.253 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 512 offset 4197376 len 2048 2024-03-08T21:14:58.253 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 512 offset 4197376 len 2048 2024-03-08T21:14:58.253 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 512 offset 4197376 len 2048 2024-03-08T21:14:58.253 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 512 offset 4197376 len 2048 2024-03-08T21:14:58.254 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 512 offset 4197376 len 4096 2024-03-08T21:14:58.254 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 512 offset 4197376 len 4096 2024-03-08T21:14:58.254 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 512 offset 4197376 len 4096 2024-03-08T21:14:58.254 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 512 offset 4197376 len 4096 2024-03-08T21:14:58.254 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 512 offset 4197376 len 8192 2024-03-08T21:14:58.254 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 512 offset 4197376 len 8192 2024-03-08T21:14:58.254 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 512 offset 4197376 len 8192 2024-03-08T21:14:58.254 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 512 offset 4197376 len 8192 2024-03-08T21:14:58.254 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 512 offset 4197376 len 16384 2024-03-08T21:14:58.254 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 512 offset 4197376 len 16384 2024-03-08T21:14:58.254 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 512 offset 4197376 len 16384 2024-03-08T21:14:58.254 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 512 offset 4197376 len 16384 2024-03-08T21:14:58.255 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 512 offset 4197888 len 1024 2024-03-08T21:14:58.255 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 512 offset 4197888 len 1024 2024-03-08T21:14:58.255 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 512 offset 4197888 len 1024 2024-03-08T21:14:58.255 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 512 offset 4197888 len 1024 2024-03-08T21:14:58.255 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 512 offset 4197888 len 2048 2024-03-08T21:14:58.255 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 512 offset 4197888 len 2048 2024-03-08T21:14:58.255 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 512 offset 4197888 len 2048 2024-03-08T21:14:58.255 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 512 offset 4197888 len 2048 2024-03-08T21:14:58.255 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 512 offset 4197888 len 4096 2024-03-08T21:14:58.255 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 512 offset 4197888 len 4096 2024-03-08T21:14:58.255 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 512 offset 4197888 len 4096 2024-03-08T21:14:58.255 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 512 offset 4197888 len 4096 2024-03-08T21:14:58.256 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 512 offset 4197888 len 8192 2024-03-08T21:14:58.256 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 512 offset 4197888 len 8192 2024-03-08T21:14:58.256 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 512 offset 4197888 len 8192 2024-03-08T21:14:58.256 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 512 offset 4197888 len 8192 2024-03-08T21:14:58.256 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 512 offset 4197888 len 16384 2024-03-08T21:14:58.256 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 512 offset 4197888 len 16384 2024-03-08T21:14:58.256 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 512 offset 4197888 len 16384 2024-03-08T21:14:58.256 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 512 offset 4197888 len 16384 2024-03-08T21:14:58.256 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 1024 offset 4190208 len 1024 2024-03-08T21:14:58.256 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1024 offset 4190208 len 1024 2024-03-08T21:14:58.256 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 1024 offset 4190208 len 1024 2024-03-08T21:14:58.256 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1024 offset 4190208 len 1024 2024-03-08T21:14:58.256 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 1024 offset 4190208 len 2048 2024-03-08T21:14:58.256 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1024 offset 4190208 len 2048 2024-03-08T21:14:58.257 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 1024 offset 4190208 len 2048 2024-03-08T21:14:58.257 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1024 offset 4190208 len 2048 2024-03-08T21:14:58.257 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 1024 offset 4190208 len 4096 2024-03-08T21:14:58.257 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1024 offset 4190208 len 4096 2024-03-08T21:14:58.257 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 1024 offset 4190208 len 4096 2024-03-08T21:14:58.257 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1024 offset 4190208 len 4096 2024-03-08T21:14:58.257 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 1024 offset 4190208 len 8192 2024-03-08T21:14:58.257 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1024 offset 4190208 len 8192 2024-03-08T21:14:58.257 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 1024 offset 4190208 len 8192 2024-03-08T21:14:58.257 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1024 offset 4190208 len 8192 2024-03-08T21:14:58.257 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 1024 offset 4190208 len 16384 2024-03-08T21:14:58.257 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1024 offset 4190208 len 16384 2024-03-08T21:14:58.257 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 1024 offset 4190208 len 16384 2024-03-08T21:14:58.258 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1024 offset 4190208 len 16384 2024-03-08T21:14:58.258 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 1024 offset 4190720 len 1024 2024-03-08T21:14:58.258 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1024 offset 4190720 len 1024 2024-03-08T21:14:58.258 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 1024 offset 4190720 len 1024 2024-03-08T21:14:58.258 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1024 offset 4190720 len 1024 2024-03-08T21:14:58.258 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 1024 offset 4190720 len 2048 2024-03-08T21:14:58.258 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1024 offset 4190720 len 2048 2024-03-08T21:14:58.258 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 1024 offset 4190720 len 2048 2024-03-08T21:14:58.258 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1024 offset 4190720 len 2048 2024-03-08T21:14:58.258 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 1024 offset 4190720 len 4096 2024-03-08T21:14:58.258 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1024 offset 4190720 len 4096 2024-03-08T21:14:58.259 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 1024 offset 4190720 len 4096 2024-03-08T21:14:58.259 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1024 offset 4190720 len 4096 2024-03-08T21:14:58.259 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 1024 offset 4190720 len 8192 2024-03-08T21:14:58.259 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1024 offset 4190720 len 8192 2024-03-08T21:14:58.259 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 1024 offset 4190720 len 8192 2024-03-08T21:14:58.259 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1024 offset 4190720 len 8192 2024-03-08T21:14:58.259 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 1024 offset 4190720 len 16384 2024-03-08T21:14:58.259 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1024 offset 4190720 len 16384 2024-03-08T21:14:58.259 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 1024 offset 4190720 len 16384 2024-03-08T21:14:58.259 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1024 offset 4190720 len 16384 2024-03-08T21:14:58.259 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 1024 offset 4191232 len 1024 2024-03-08T21:14:58.259 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1024 offset 4191232 len 1024 2024-03-08T21:14:58.260 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 1024 offset 4191232 len 1024 2024-03-08T21:14:58.260 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1024 offset 4191232 len 1024 2024-03-08T21:14:58.355 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 1024 offset 4191232 len 2048 2024-03-08T21:14:58.355 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1024 offset 4191232 len 2048 2024-03-08T21:14:58.355 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 1024 offset 4191232 len 2048 2024-03-08T21:14:58.355 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1024 offset 4191232 len 2048 2024-03-08T21:14:58.355 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 1024 offset 4191232 len 4096 2024-03-08T21:14:58.356 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1024 offset 4191232 len 4096 2024-03-08T21:14:58.356 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 1024 offset 4191232 len 4096 2024-03-08T21:14:58.356 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1024 offset 4191232 len 4096 2024-03-08T21:14:58.356 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 1024 offset 4191232 len 8192 2024-03-08T21:14:58.356 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1024 offset 4191232 len 8192 2024-03-08T21:14:58.356 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 1024 offset 4191232 len 8192 2024-03-08T21:14:58.356 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1024 offset 4191232 len 8192 2024-03-08T21:14:58.356 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 1024 offset 4191232 len 16384 2024-03-08T21:14:58.356 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1024 offset 4191232 len 16384 2024-03-08T21:14:58.356 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 1024 offset 4191232 len 16384 2024-03-08T21:14:58.356 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1024 offset 4191232 len 16384 2024-03-08T21:14:58.356 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 1024 offset 4191744 len 1024 2024-03-08T21:14:58.356 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1024 offset 4191744 len 1024 2024-03-08T21:14:58.356 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 1024 offset 4191744 len 1024 2024-03-08T21:14:58.357 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1024 offset 4191744 len 1024 2024-03-08T21:14:58.357 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 1024 offset 4191744 len 2048 2024-03-08T21:14:58.357 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1024 offset 4191744 len 2048 2024-03-08T21:14:58.357 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 1024 offset 4191744 len 2048 2024-03-08T21:14:58.357 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1024 offset 4191744 len 2048 2024-03-08T21:14:58.357 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 1024 offset 4191744 len 4096 2024-03-08T21:14:58.357 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1024 offset 4191744 len 4096 2024-03-08T21:14:58.357 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 1024 offset 4191744 len 4096 2024-03-08T21:14:58.357 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1024 offset 4191744 len 4096 2024-03-08T21:14:58.357 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 1024 offset 4191744 len 8192 2024-03-08T21:14:58.357 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1024 offset 4191744 len 8192 2024-03-08T21:14:58.357 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 1024 offset 4191744 len 8192 2024-03-08T21:14:58.357 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1024 offset 4191744 len 8192 2024-03-08T21:14:58.358 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 1024 offset 4191744 len 16384 2024-03-08T21:14:58.358 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1024 offset 4191744 len 16384 2024-03-08T21:14:58.358 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 1024 offset 4191744 len 16384 2024-03-08T21:14:58.358 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1024 offset 4191744 len 16384 2024-03-08T21:14:58.358 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 1024 offset 4192256 len 1024 2024-03-08T21:14:58.358 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1024 offset 4192256 len 1024 2024-03-08T21:14:58.358 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 1024 offset 4192256 len 1024 2024-03-08T21:14:58.358 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1024 offset 4192256 len 1024 2024-03-08T21:14:58.358 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 1024 offset 4192256 len 2048 2024-03-08T21:14:58.358 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1024 offset 4192256 len 2048 2024-03-08T21:14:58.359 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 1024 offset 4192256 len 2048 2024-03-08T21:14:58.359 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1024 offset 4192256 len 2048 2024-03-08T21:14:58.359 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 1024 offset 4192256 len 4096 2024-03-08T21:14:58.359 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1024 offset 4192256 len 4096 2024-03-08T21:14:58.359 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 1024 offset 4192256 len 4096 2024-03-08T21:14:58.359 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1024 offset 4192256 len 4096 2024-03-08T21:14:58.359 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 1024 offset 4192256 len 8192 2024-03-08T21:14:58.359 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1024 offset 4192256 len 8192 2024-03-08T21:14:58.359 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 1024 offset 4192256 len 8192 2024-03-08T21:14:58.359 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1024 offset 4192256 len 8192 2024-03-08T21:14:58.359 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 1024 offset 4192256 len 16384 2024-03-08T21:14:58.359 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1024 offset 4192256 len 16384 2024-03-08T21:14:58.359 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 1024 offset 4192256 len 16384 2024-03-08T21:14:58.359 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1024 offset 4192256 len 16384 2024-03-08T21:14:58.360 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 1024 offset 4192768 len 1024 2024-03-08T21:14:58.360 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1024 offset 4192768 len 1024 2024-03-08T21:14:58.360 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 1024 offset 4192768 len 1024 2024-03-08T21:14:58.360 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1024 offset 4192768 len 1024 2024-03-08T21:14:58.360 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 1024 offset 4192768 len 2048 2024-03-08T21:14:58.360 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1024 offset 4192768 len 2048 2024-03-08T21:14:58.360 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 1024 offset 4192768 len 2048 2024-03-08T21:14:58.360 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1024 offset 4192768 len 2048 2024-03-08T21:14:58.360 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 1024 offset 4192768 len 4096 2024-03-08T21:14:58.360 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1024 offset 4192768 len 4096 2024-03-08T21:14:58.360 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 1024 offset 4192768 len 4096 2024-03-08T21:14:58.360 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1024 offset 4192768 len 4096 2024-03-08T21:14:58.360 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 1024 offset 4192768 len 8192 2024-03-08T21:14:58.360 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1024 offset 4192768 len 8192 2024-03-08T21:14:58.361 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 1024 offset 4192768 len 8192 2024-03-08T21:14:58.361 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1024 offset 4192768 len 8192 2024-03-08T21:14:58.361 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 1024 offset 4192768 len 16384 2024-03-08T21:14:58.361 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1024 offset 4192768 len 16384 2024-03-08T21:14:58.361 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 1024 offset 4192768 len 16384 2024-03-08T21:14:58.361 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1024 offset 4192768 len 16384 2024-03-08T21:14:58.361 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 1024 offset 4193280 len 1024 2024-03-08T21:14:58.361 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1024 offset 4193280 len 1024 2024-03-08T21:14:58.362 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 1024 offset 4193280 len 1024 2024-03-08T21:14:58.362 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1024 offset 4193280 len 1024 2024-03-08T21:14:58.362 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 1024 offset 4193280 len 2048 2024-03-08T21:14:58.362 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1024 offset 4193280 len 2048 2024-03-08T21:14:58.362 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 1024 offset 4193280 len 2048 2024-03-08T21:14:58.449 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1024 offset 4193280 len 2048 2024-03-08T21:14:58.449 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 1024 offset 4193280 len 4096 2024-03-08T21:14:58.449 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1024 offset 4193280 len 4096 2024-03-08T21:14:58.449 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 1024 offset 4193280 len 4096 2024-03-08T21:14:58.449 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1024 offset 4193280 len 4096 2024-03-08T21:14:58.449 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 1024 offset 4193280 len 8192 2024-03-08T21:14:58.449 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1024 offset 4193280 len 8192 2024-03-08T21:14:58.449 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 1024 offset 4193280 len 8192 2024-03-08T21:14:58.449 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1024 offset 4193280 len 8192 2024-03-08T21:14:58.449 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 1024 offset 4193280 len 16384 2024-03-08T21:14:58.449 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1024 offset 4193280 len 16384 2024-03-08T21:14:58.450 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 1024 offset 4193280 len 16384 2024-03-08T21:14:58.450 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1024 offset 4193280 len 16384 2024-03-08T21:14:58.450 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 1024 offset 4193792 len 1024 2024-03-08T21:14:58.450 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1024 offset 4193792 len 1024 2024-03-08T21:14:58.450 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 1024 offset 4193792 len 1024 2024-03-08T21:14:58.450 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1024 offset 4193792 len 1024 2024-03-08T21:14:58.450 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 1024 offset 4193792 len 2048 2024-03-08T21:14:58.450 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1024 offset 4193792 len 2048 2024-03-08T21:14:58.450 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 1024 offset 4193792 len 2048 2024-03-08T21:14:58.450 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1024 offset 4193792 len 2048 2024-03-08T21:14:58.450 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 1024 offset 4193792 len 4096 2024-03-08T21:14:58.450 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1024 offset 4193792 len 4096 2024-03-08T21:14:58.450 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 1024 offset 4193792 len 4096 2024-03-08T21:14:58.451 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1024 offset 4193792 len 4096 2024-03-08T21:14:58.451 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 1024 offset 4193792 len 8192 2024-03-08T21:14:58.451 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1024 offset 4193792 len 8192 2024-03-08T21:14:58.451 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 1024 offset 4193792 len 8192 2024-03-08T21:14:58.451 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1024 offset 4193792 len 8192 2024-03-08T21:14:58.451 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 1024 offset 4193792 len 16384 2024-03-08T21:14:58.451 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1024 offset 4193792 len 16384 2024-03-08T21:14:58.451 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 1024 offset 4193792 len 16384 2024-03-08T21:14:58.451 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1024 offset 4193792 len 16384 2024-03-08T21:14:58.451 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 1024 offset 4194304 len 1024 2024-03-08T21:14:58.451 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1024 offset 4194304 len 1024 2024-03-08T21:14:58.451 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 1024 offset 4194304 len 1024 2024-03-08T21:14:58.452 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1024 offset 4194304 len 1024 2024-03-08T21:14:58.452 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 1024 offset 4194304 len 2048 2024-03-08T21:14:58.452 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1024 offset 4194304 len 2048 2024-03-08T21:14:58.452 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 1024 offset 4194304 len 2048 2024-03-08T21:14:58.452 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1024 offset 4194304 len 2048 2024-03-08T21:14:58.452 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 1024 offset 4194304 len 4096 2024-03-08T21:14:58.452 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1024 offset 4194304 len 4096 2024-03-08T21:14:58.452 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 1024 offset 4194304 len 4096 2024-03-08T21:14:58.452 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1024 offset 4194304 len 4096 2024-03-08T21:14:58.452 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 1024 offset 4194304 len 8192 2024-03-08T21:14:58.452 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1024 offset 4194304 len 8192 2024-03-08T21:14:58.452 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 1024 offset 4194304 len 8192 2024-03-08T21:14:58.452 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1024 offset 4194304 len 8192 2024-03-08T21:14:58.452 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 1024 offset 4194304 len 16384 2024-03-08T21:14:58.452 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1024 offset 4194304 len 16384 2024-03-08T21:14:58.453 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 1024 offset 4194304 len 16384 2024-03-08T21:14:58.453 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1024 offset 4194304 len 16384 2024-03-08T21:14:58.453 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 1024 offset 4194816 len 1024 2024-03-08T21:14:58.453 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1024 offset 4194816 len 1024 2024-03-08T21:14:58.453 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 1024 offset 4194816 len 1024 2024-03-08T21:14:58.453 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1024 offset 4194816 len 1024 2024-03-08T21:14:58.453 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 1024 offset 4194816 len 2048 2024-03-08T21:14:58.453 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1024 offset 4194816 len 2048 2024-03-08T21:14:58.453 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 1024 offset 4194816 len 2048 2024-03-08T21:14:58.453 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1024 offset 4194816 len 2048 2024-03-08T21:14:58.453 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 1024 offset 4194816 len 4096 2024-03-08T21:14:58.453 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1024 offset 4194816 len 4096 2024-03-08T21:14:58.453 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 1024 offset 4194816 len 4096 2024-03-08T21:14:58.454 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1024 offset 4194816 len 4096 2024-03-08T21:14:58.454 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 1024 offset 4194816 len 8192 2024-03-08T21:14:58.454 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1024 offset 4194816 len 8192 2024-03-08T21:14:58.454 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 1024 offset 4194816 len 8192 2024-03-08T21:14:58.454 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1024 offset 4194816 len 8192 2024-03-08T21:14:58.454 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 1024 offset 4194816 len 16384 2024-03-08T21:14:58.454 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1024 offset 4194816 len 16384 2024-03-08T21:14:58.454 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 1024 offset 4194816 len 16384 2024-03-08T21:14:58.454 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1024 offset 4194816 len 16384 2024-03-08T21:14:58.454 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 1024 offset 4195328 len 1024 2024-03-08T21:14:58.454 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1024 offset 4195328 len 1024 2024-03-08T21:14:58.454 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 1024 offset 4195328 len 1024 2024-03-08T21:14:58.454 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1024 offset 4195328 len 1024 2024-03-08T21:14:58.455 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 1024 offset 4195328 len 2048 2024-03-08T21:14:58.455 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1024 offset 4195328 len 2048 2024-03-08T21:14:58.455 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 1024 offset 4195328 len 2048 2024-03-08T21:14:58.455 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1024 offset 4195328 len 2048 2024-03-08T21:14:58.455 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 1024 offset 4195328 len 4096 2024-03-08T21:14:58.540 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1024 offset 4195328 len 4096 2024-03-08T21:14:58.540 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 1024 offset 4195328 len 4096 2024-03-08T21:14:58.540 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1024 offset 4195328 len 4096 2024-03-08T21:14:58.540 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 1024 offset 4195328 len 8192 2024-03-08T21:14:58.540 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1024 offset 4195328 len 8192 2024-03-08T21:14:58.540 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 1024 offset 4195328 len 8192 2024-03-08T21:14:58.540 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1024 offset 4195328 len 8192 2024-03-08T21:14:58.540 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 1024 offset 4195328 len 16384 2024-03-08T21:14:58.541 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1024 offset 4195328 len 16384 2024-03-08T21:14:58.541 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 1024 offset 4195328 len 16384 2024-03-08T21:14:58.541 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1024 offset 4195328 len 16384 2024-03-08T21:14:58.541 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 1024 offset 4195840 len 1024 2024-03-08T21:14:58.541 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1024 offset 4195840 len 1024 2024-03-08T21:14:58.541 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 1024 offset 4195840 len 1024 2024-03-08T21:14:58.541 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1024 offset 4195840 len 1024 2024-03-08T21:14:58.541 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 1024 offset 4195840 len 2048 2024-03-08T21:14:58.541 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1024 offset 4195840 len 2048 2024-03-08T21:14:58.541 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 1024 offset 4195840 len 2048 2024-03-08T21:14:58.541 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1024 offset 4195840 len 2048 2024-03-08T21:14:58.541 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 1024 offset 4195840 len 4096 2024-03-08T21:14:58.541 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1024 offset 4195840 len 4096 2024-03-08T21:14:58.541 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 1024 offset 4195840 len 4096 2024-03-08T21:14:58.542 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1024 offset 4195840 len 4096 2024-03-08T21:14:58.542 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 1024 offset 4195840 len 8192 2024-03-08T21:14:58.542 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1024 offset 4195840 len 8192 2024-03-08T21:14:58.542 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 1024 offset 4195840 len 8192 2024-03-08T21:14:58.542 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1024 offset 4195840 len 8192 2024-03-08T21:14:58.542 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 1024 offset 4195840 len 16384 2024-03-08T21:14:58.542 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1024 offset 4195840 len 16384 2024-03-08T21:14:58.542 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 1024 offset 4195840 len 16384 2024-03-08T21:14:58.542 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1024 offset 4195840 len 16384 2024-03-08T21:14:58.542 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 1024 offset 4196352 len 1024 2024-03-08T21:14:58.542 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1024 offset 4196352 len 1024 2024-03-08T21:14:58.542 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 1024 offset 4196352 len 1024 2024-03-08T21:14:58.542 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1024 offset 4196352 len 1024 2024-03-08T21:14:58.543 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 1024 offset 4196352 len 2048 2024-03-08T21:14:58.543 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1024 offset 4196352 len 2048 2024-03-08T21:14:58.543 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 1024 offset 4196352 len 2048 2024-03-08T21:14:58.543 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1024 offset 4196352 len 2048 2024-03-08T21:14:58.543 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 1024 offset 4196352 len 4096 2024-03-08T21:14:58.543 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1024 offset 4196352 len 4096 2024-03-08T21:14:58.543 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 1024 offset 4196352 len 4096 2024-03-08T21:14:58.543 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1024 offset 4196352 len 4096 2024-03-08T21:14:58.543 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 1024 offset 4196352 len 8192 2024-03-08T21:14:58.543 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1024 offset 4196352 len 8192 2024-03-08T21:14:58.544 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 1024 offset 4196352 len 8192 2024-03-08T21:14:58.544 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1024 offset 4196352 len 8192 2024-03-08T21:14:58.544 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 1024 offset 4196352 len 16384 2024-03-08T21:14:58.544 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1024 offset 4196352 len 16384 2024-03-08T21:14:58.544 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 1024 offset 4196352 len 16384 2024-03-08T21:14:58.544 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1024 offset 4196352 len 16384 2024-03-08T21:14:58.544 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 1024 offset 4196864 len 1024 2024-03-08T21:14:58.544 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1024 offset 4196864 len 1024 2024-03-08T21:14:58.544 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 1024 offset 4196864 len 1024 2024-03-08T21:14:58.544 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1024 offset 4196864 len 1024 2024-03-08T21:14:58.544 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 1024 offset 4196864 len 2048 2024-03-08T21:14:58.544 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1024 offset 4196864 len 2048 2024-03-08T21:14:58.544 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 1024 offset 4196864 len 2048 2024-03-08T21:14:58.545 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1024 offset 4196864 len 2048 2024-03-08T21:14:58.545 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 1024 offset 4196864 len 4096 2024-03-08T21:14:58.545 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1024 offset 4196864 len 4096 2024-03-08T21:14:58.545 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 1024 offset 4196864 len 4096 2024-03-08T21:14:58.545 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1024 offset 4196864 len 4096 2024-03-08T21:14:58.545 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 1024 offset 4196864 len 8192 2024-03-08T21:14:58.545 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1024 offset 4196864 len 8192 2024-03-08T21:14:58.545 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 1024 offset 4196864 len 8192 2024-03-08T21:14:58.545 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1024 offset 4196864 len 8192 2024-03-08T21:14:58.545 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 1024 offset 4196864 len 16384 2024-03-08T21:14:58.545 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1024 offset 4196864 len 16384 2024-03-08T21:14:58.545 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 1024 offset 4196864 len 16384 2024-03-08T21:14:58.545 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1024 offset 4196864 len 16384 2024-03-08T21:14:58.545 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 1024 offset 4197376 len 1024 2024-03-08T21:14:58.545 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1024 offset 4197376 len 1024 2024-03-08T21:14:58.546 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 1024 offset 4197376 len 1024 2024-03-08T21:14:58.546 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1024 offset 4197376 len 1024 2024-03-08T21:14:58.546 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 1024 offset 4197376 len 2048 2024-03-08T21:14:58.546 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1024 offset 4197376 len 2048 2024-03-08T21:14:58.546 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 1024 offset 4197376 len 2048 2024-03-08T21:14:58.546 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1024 offset 4197376 len 2048 2024-03-08T21:14:58.546 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 1024 offset 4197376 len 4096 2024-03-08T21:14:58.546 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1024 offset 4197376 len 4096 2024-03-08T21:14:58.546 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 1024 offset 4197376 len 4096 2024-03-08T21:14:58.546 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1024 offset 4197376 len 4096 2024-03-08T21:14:58.634 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 1024 offset 4197376 len 8192 2024-03-08T21:14:58.634 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1024 offset 4197376 len 8192 2024-03-08T21:14:58.634 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 1024 offset 4197376 len 8192 2024-03-08T21:14:58.634 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1024 offset 4197376 len 8192 2024-03-08T21:14:58.634 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 1024 offset 4197376 len 16384 2024-03-08T21:14:58.634 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1024 offset 4197376 len 16384 2024-03-08T21:14:58.634 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 1024 offset 4197376 len 16384 2024-03-08T21:14:58.634 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1024 offset 4197376 len 16384 2024-03-08T21:14:58.634 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 1024 offset 4197888 len 1024 2024-03-08T21:14:58.634 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1024 offset 4197888 len 1024 2024-03-08T21:14:58.635 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 1024 offset 4197888 len 1024 2024-03-08T21:14:58.635 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1024 offset 4197888 len 1024 2024-03-08T21:14:58.635 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 1024 offset 4197888 len 2048 2024-03-08T21:14:58.635 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1024 offset 4197888 len 2048 2024-03-08T21:14:58.635 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 1024 offset 4197888 len 2048 2024-03-08T21:14:58.635 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1024 offset 4197888 len 2048 2024-03-08T21:14:58.635 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 1024 offset 4197888 len 4096 2024-03-08T21:14:58.635 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1024 offset 4197888 len 4096 2024-03-08T21:14:58.635 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 1024 offset 4197888 len 4096 2024-03-08T21:14:58.635 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1024 offset 4197888 len 4096 2024-03-08T21:14:58.635 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 1024 offset 4197888 len 8192 2024-03-08T21:14:58.635 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1024 offset 4197888 len 8192 2024-03-08T21:14:58.635 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 1024 offset 4197888 len 8192 2024-03-08T21:14:58.635 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1024 offset 4197888 len 8192 2024-03-08T21:14:58.635 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 1024 offset 4197888 len 16384 2024-03-08T21:14:58.636 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1024 offset 4197888 len 16384 2024-03-08T21:14:58.636 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 1024 offset 4197888 len 16384 2024-03-08T21:14:58.636 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1024 offset 4197888 len 16384 2024-03-08T21:14:58.636 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 1536 offset 4190208 len 1024 2024-03-08T21:14:58.636 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1536 offset 4190208 len 1024 2024-03-08T21:14:58.636 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 1536 offset 4190208 len 1024 2024-03-08T21:14:58.636 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1536 offset 4190208 len 1024 2024-03-08T21:14:58.636 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 1536 offset 4190208 len 2048 2024-03-08T21:14:58.636 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1536 offset 4190208 len 2048 2024-03-08T21:14:58.636 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 1536 offset 4190208 len 2048 2024-03-08T21:14:58.636 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1536 offset 4190208 len 2048 2024-03-08T21:14:58.636 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 1536 offset 4190208 len 4096 2024-03-08T21:14:58.636 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1536 offset 4190208 len 4096 2024-03-08T21:14:58.636 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 1536 offset 4190208 len 4096 2024-03-08T21:14:58.637 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1536 offset 4190208 len 4096 2024-03-08T21:14:58.637 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 1536 offset 4190208 len 8192 2024-03-08T21:14:58.637 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1536 offset 4190208 len 8192 2024-03-08T21:14:58.637 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 1536 offset 4190208 len 8192 2024-03-08T21:14:58.637 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1536 offset 4190208 len 8192 2024-03-08T21:14:58.637 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 1536 offset 4190208 len 16384 2024-03-08T21:14:58.637 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1536 offset 4190208 len 16384 2024-03-08T21:14:58.637 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 1536 offset 4190208 len 16384 2024-03-08T21:14:58.637 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1536 offset 4190208 len 16384 2024-03-08T21:14:58.637 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 1536 offset 4190720 len 1024 2024-03-08T21:14:58.637 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1536 offset 4190720 len 1024 2024-03-08T21:14:58.637 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 1536 offset 4190720 len 1024 2024-03-08T21:14:58.637 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1536 offset 4190720 len 1024 2024-03-08T21:14:58.637 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 1536 offset 4190720 len 2048 2024-03-08T21:14:58.638 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1536 offset 4190720 len 2048 2024-03-08T21:14:58.638 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 1536 offset 4190720 len 2048 2024-03-08T21:14:58.638 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1536 offset 4190720 len 2048 2024-03-08T21:14:58.638 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 1536 offset 4190720 len 4096 2024-03-08T21:14:58.638 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1536 offset 4190720 len 4096 2024-03-08T21:14:58.638 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 1536 offset 4190720 len 4096 2024-03-08T21:14:58.638 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1536 offset 4190720 len 4096 2024-03-08T21:14:58.638 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 1536 offset 4190720 len 8192 2024-03-08T21:14:58.638 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1536 offset 4190720 len 8192 2024-03-08T21:14:58.638 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 1536 offset 4190720 len 8192 2024-03-08T21:14:58.638 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1536 offset 4190720 len 8192 2024-03-08T21:14:58.638 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 1536 offset 4190720 len 16384 2024-03-08T21:14:58.638 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1536 offset 4190720 len 16384 2024-03-08T21:14:58.638 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 1536 offset 4190720 len 16384 2024-03-08T21:14:58.638 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1536 offset 4190720 len 16384 2024-03-08T21:14:58.639 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 1536 offset 4191232 len 1024 2024-03-08T21:14:58.639 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1536 offset 4191232 len 1024 2024-03-08T21:14:58.639 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 1536 offset 4191232 len 1024 2024-03-08T21:14:58.639 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1536 offset 4191232 len 1024 2024-03-08T21:14:58.639 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 1536 offset 4191232 len 2048 2024-03-08T21:14:58.639 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1536 offset 4191232 len 2048 2024-03-08T21:14:58.639 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 1536 offset 4191232 len 2048 2024-03-08T21:14:58.639 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1536 offset 4191232 len 2048 2024-03-08T21:14:58.639 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 1536 offset 4191232 len 4096 2024-03-08T21:14:58.639 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1536 offset 4191232 len 4096 2024-03-08T21:14:58.639 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 1536 offset 4191232 len 4096 2024-03-08T21:14:58.639 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1536 offset 4191232 len 4096 2024-03-08T21:14:58.639 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 1536 offset 4191232 len 8192 2024-03-08T21:14:58.639 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1536 offset 4191232 len 8192 2024-03-08T21:14:58.711 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:14:58 smithi107 ceph-mon[31381]: pgmap v457: 97 pgs: 97 active+clean; 9.5 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:14:58.727 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 1536 offset 4191232 len 8192 2024-03-08T21:14:58.727 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1536 offset 4191232 len 8192 2024-03-08T21:14:58.728 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 1536 offset 4191232 len 16384 2024-03-08T21:14:58.728 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1536 offset 4191232 len 16384 2024-03-08T21:14:58.728 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 1536 offset 4191232 len 16384 2024-03-08T21:14:58.728 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1536 offset 4191232 len 16384 2024-03-08T21:14:58.728 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 1536 offset 4191744 len 1024 2024-03-08T21:14:58.728 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1536 offset 4191744 len 1024 2024-03-08T21:14:58.728 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 1536 offset 4191744 len 1024 2024-03-08T21:14:58.728 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1536 offset 4191744 len 1024 2024-03-08T21:14:58.728 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 1536 offset 4191744 len 2048 2024-03-08T21:14:58.728 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1536 offset 4191744 len 2048 2024-03-08T21:14:58.728 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 1536 offset 4191744 len 2048 2024-03-08T21:14:58.728 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1536 offset 4191744 len 2048 2024-03-08T21:14:58.728 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 1536 offset 4191744 len 4096 2024-03-08T21:14:58.728 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1536 offset 4191744 len 4096 2024-03-08T21:14:58.729 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 1536 offset 4191744 len 4096 2024-03-08T21:14:58.729 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1536 offset 4191744 len 4096 2024-03-08T21:14:58.729 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 1536 offset 4191744 len 8192 2024-03-08T21:14:58.729 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1536 offset 4191744 len 8192 2024-03-08T21:14:58.729 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 1536 offset 4191744 len 8192 2024-03-08T21:14:58.729 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1536 offset 4191744 len 8192 2024-03-08T21:14:58.729 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 1536 offset 4191744 len 16384 2024-03-08T21:14:58.729 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1536 offset 4191744 len 16384 2024-03-08T21:14:58.729 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 1536 offset 4191744 len 16384 2024-03-08T21:14:58.729 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1536 offset 4191744 len 16384 2024-03-08T21:14:58.729 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 1536 offset 4192256 len 1024 2024-03-08T21:14:58.729 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1536 offset 4192256 len 1024 2024-03-08T21:14:58.729 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 1536 offset 4192256 len 1024 2024-03-08T21:14:58.729 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1536 offset 4192256 len 1024 2024-03-08T21:14:58.730 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 1536 offset 4192256 len 2048 2024-03-08T21:14:58.730 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1536 offset 4192256 len 2048 2024-03-08T21:14:58.730 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 1536 offset 4192256 len 2048 2024-03-08T21:14:58.730 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1536 offset 4192256 len 2048 2024-03-08T21:14:58.730 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 1536 offset 4192256 len 4096 2024-03-08T21:14:58.730 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1536 offset 4192256 len 4096 2024-03-08T21:14:58.730 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 1536 offset 4192256 len 4096 2024-03-08T21:14:58.730 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1536 offset 4192256 len 4096 2024-03-08T21:14:58.730 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 1536 offset 4192256 len 8192 2024-03-08T21:14:58.730 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1536 offset 4192256 len 8192 2024-03-08T21:14:58.730 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 1536 offset 4192256 len 8192 2024-03-08T21:14:58.730 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1536 offset 4192256 len 8192 2024-03-08T21:14:58.730 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 1536 offset 4192256 len 16384 2024-03-08T21:14:58.730 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1536 offset 4192256 len 16384 2024-03-08T21:14:58.731 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 1536 offset 4192256 len 16384 2024-03-08T21:14:58.731 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1536 offset 4192256 len 16384 2024-03-08T21:14:58.731 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 1536 offset 4192768 len 1024 2024-03-08T21:14:58.731 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1536 offset 4192768 len 1024 2024-03-08T21:14:58.731 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 1536 offset 4192768 len 1024 2024-03-08T21:14:58.731 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1536 offset 4192768 len 1024 2024-03-08T21:14:58.731 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 1536 offset 4192768 len 2048 2024-03-08T21:14:58.731 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1536 offset 4192768 len 2048 2024-03-08T21:14:58.731 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 1536 offset 4192768 len 2048 2024-03-08T21:14:58.731 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1536 offset 4192768 len 2048 2024-03-08T21:14:58.731 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 1536 offset 4192768 len 4096 2024-03-08T21:14:58.731 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1536 offset 4192768 len 4096 2024-03-08T21:14:58.731 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 1536 offset 4192768 len 4096 2024-03-08T21:14:58.731 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1536 offset 4192768 len 4096 2024-03-08T21:14:58.731 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 1536 offset 4192768 len 8192 2024-03-08T21:14:58.732 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1536 offset 4192768 len 8192 2024-03-08T21:14:58.732 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 1536 offset 4192768 len 8192 2024-03-08T21:14:58.732 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1536 offset 4192768 len 8192 2024-03-08T21:14:58.732 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 1536 offset 4192768 len 16384 2024-03-08T21:14:58.732 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1536 offset 4192768 len 16384 2024-03-08T21:14:58.732 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 1536 offset 4192768 len 16384 2024-03-08T21:14:58.732 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1536 offset 4192768 len 16384 2024-03-08T21:14:58.732 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 1536 offset 4193280 len 1024 2024-03-08T21:14:58.732 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1536 offset 4193280 len 1024 2024-03-08T21:14:58.732 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 1536 offset 4193280 len 1024 2024-03-08T21:14:58.732 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1536 offset 4193280 len 1024 2024-03-08T21:14:58.732 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 1536 offset 4193280 len 2048 2024-03-08T21:14:58.732 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1536 offset 4193280 len 2048 2024-03-08T21:14:58.732 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 1536 offset 4193280 len 2048 2024-03-08T21:14:58.732 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1536 offset 4193280 len 2048 2024-03-08T21:14:58.733 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 1536 offset 4193280 len 4096 2024-03-08T21:14:58.733 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1536 offset 4193280 len 4096 2024-03-08T21:14:58.733 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 1536 offset 4193280 len 4096 2024-03-08T21:14:58.733 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1536 offset 4193280 len 4096 2024-03-08T21:14:58.733 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 1536 offset 4193280 len 8192 2024-03-08T21:14:58.733 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1536 offset 4193280 len 8192 2024-03-08T21:14:58.733 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 1536 offset 4193280 len 8192 2024-03-08T21:14:58.733 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1536 offset 4193280 len 8192 2024-03-08T21:14:58.758 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:14:58 smithi149 ceph-mon[35042]: pgmap v457: 97 pgs: 97 active+clean; 9.5 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:14:58.824 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 1536 offset 4193280 len 16384 2024-03-08T21:14:58.824 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1536 offset 4193280 len 16384 2024-03-08T21:14:58.824 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 1536 offset 4193280 len 16384 2024-03-08T21:14:58.824 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1536 offset 4193280 len 16384 2024-03-08T21:14:58.824 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 1536 offset 4193792 len 1024 2024-03-08T21:14:58.824 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1536 offset 4193792 len 1024 2024-03-08T21:14:58.824 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 1536 offset 4193792 len 1024 2024-03-08T21:14:58.824 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1536 offset 4193792 len 1024 2024-03-08T21:14:58.825 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 1536 offset 4193792 len 2048 2024-03-08T21:14:58.825 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1536 offset 4193792 len 2048 2024-03-08T21:14:58.825 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 1536 offset 4193792 len 2048 2024-03-08T21:14:58.825 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1536 offset 4193792 len 2048 2024-03-08T21:14:58.825 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 1536 offset 4193792 len 4096 2024-03-08T21:14:58.825 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1536 offset 4193792 len 4096 2024-03-08T21:14:58.825 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 1536 offset 4193792 len 4096 2024-03-08T21:14:58.825 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1536 offset 4193792 len 4096 2024-03-08T21:14:58.825 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 1536 offset 4193792 len 8192 2024-03-08T21:14:58.825 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1536 offset 4193792 len 8192 2024-03-08T21:14:58.825 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 1536 offset 4193792 len 8192 2024-03-08T21:14:58.825 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1536 offset 4193792 len 8192 2024-03-08T21:14:58.825 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 1536 offset 4193792 len 16384 2024-03-08T21:14:58.825 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1536 offset 4193792 len 16384 2024-03-08T21:14:58.825 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 1536 offset 4193792 len 16384 2024-03-08T21:14:58.826 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1536 offset 4193792 len 16384 2024-03-08T21:14:58.826 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 1536 offset 4194304 len 1024 2024-03-08T21:14:58.826 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1536 offset 4194304 len 1024 2024-03-08T21:14:58.826 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 1536 offset 4194304 len 1024 2024-03-08T21:14:58.826 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1536 offset 4194304 len 1024 2024-03-08T21:14:58.826 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 1536 offset 4194304 len 2048 2024-03-08T21:14:58.826 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1536 offset 4194304 len 2048 2024-03-08T21:14:58.826 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 1536 offset 4194304 len 2048 2024-03-08T21:14:58.826 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1536 offset 4194304 len 2048 2024-03-08T21:14:58.826 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 1536 offset 4194304 len 4096 2024-03-08T21:14:58.826 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1536 offset 4194304 len 4096 2024-03-08T21:14:58.826 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 1536 offset 4194304 len 4096 2024-03-08T21:14:58.826 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1536 offset 4194304 len 4096 2024-03-08T21:14:58.826 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 1536 offset 4194304 len 8192 2024-03-08T21:14:58.827 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1536 offset 4194304 len 8192 2024-03-08T21:14:58.827 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 1536 offset 4194304 len 8192 2024-03-08T21:14:58.827 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1536 offset 4194304 len 8192 2024-03-08T21:14:58.827 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 1536 offset 4194304 len 16384 2024-03-08T21:14:58.827 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1536 offset 4194304 len 16384 2024-03-08T21:14:58.827 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 1536 offset 4194304 len 16384 2024-03-08T21:14:58.827 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1536 offset 4194304 len 16384 2024-03-08T21:14:58.827 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 1536 offset 4194816 len 1024 2024-03-08T21:14:58.827 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1536 offset 4194816 len 1024 2024-03-08T21:14:58.827 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 1536 offset 4194816 len 1024 2024-03-08T21:14:58.827 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1536 offset 4194816 len 1024 2024-03-08T21:14:58.827 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 1536 offset 4194816 len 2048 2024-03-08T21:14:58.827 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1536 offset 4194816 len 2048 2024-03-08T21:14:58.827 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 1536 offset 4194816 len 2048 2024-03-08T21:14:58.827 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1536 offset 4194816 len 2048 2024-03-08T21:14:58.828 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 1536 offset 4194816 len 4096 2024-03-08T21:14:58.828 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1536 offset 4194816 len 4096 2024-03-08T21:14:58.828 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 1536 offset 4194816 len 4096 2024-03-08T21:14:58.828 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1536 offset 4194816 len 4096 2024-03-08T21:14:58.828 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 1536 offset 4194816 len 8192 2024-03-08T21:14:58.828 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1536 offset 4194816 len 8192 2024-03-08T21:14:58.828 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 1536 offset 4194816 len 8192 2024-03-08T21:14:58.828 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1536 offset 4194816 len 8192 2024-03-08T21:14:58.828 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 1536 offset 4194816 len 16384 2024-03-08T21:14:58.828 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1536 offset 4194816 len 16384 2024-03-08T21:14:58.828 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 1536 offset 4194816 len 16384 2024-03-08T21:14:58.828 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1536 offset 4194816 len 16384 2024-03-08T21:14:58.828 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 1536 offset 4195328 len 1024 2024-03-08T21:14:58.828 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1536 offset 4195328 len 1024 2024-03-08T21:14:58.828 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 1536 offset 4195328 len 1024 2024-03-08T21:14:58.828 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1536 offset 4195328 len 1024 2024-03-08T21:14:58.829 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 1536 offset 4195328 len 2048 2024-03-08T21:14:58.829 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1536 offset 4195328 len 2048 2024-03-08T21:14:58.829 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 1536 offset 4195328 len 2048 2024-03-08T21:14:58.829 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1536 offset 4195328 len 2048 2024-03-08T21:14:58.829 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 1536 offset 4195328 len 4096 2024-03-08T21:14:58.829 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1536 offset 4195328 len 4096 2024-03-08T21:14:58.829 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 1536 offset 4195328 len 4096 2024-03-08T21:14:58.829 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1536 offset 4195328 len 4096 2024-03-08T21:14:58.829 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 1536 offset 4195328 len 8192 2024-03-08T21:14:58.829 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1536 offset 4195328 len 8192 2024-03-08T21:14:58.829 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 1536 offset 4195328 len 8192 2024-03-08T21:14:58.829 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1536 offset 4195328 len 8192 2024-03-08T21:14:58.829 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 1536 offset 4195328 len 16384 2024-03-08T21:14:58.829 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1536 offset 4195328 len 16384 2024-03-08T21:14:58.830 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 1536 offset 4195328 len 16384 2024-03-08T21:14:58.872 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:14:58 smithi204 ceph-mon[34935]: pgmap v457: 97 pgs: 97 active+clean; 9.5 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-03-08T21:14:58.913 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1536 offset 4195328 len 16384 2024-03-08T21:14:58.913 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 1536 offset 4195840 len 1024 2024-03-08T21:14:58.913 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1536 offset 4195840 len 1024 2024-03-08T21:14:58.913 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 1536 offset 4195840 len 1024 2024-03-08T21:14:58.913 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1536 offset 4195840 len 1024 2024-03-08T21:14:58.914 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 1536 offset 4195840 len 2048 2024-03-08T21:14:58.914 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1536 offset 4195840 len 2048 2024-03-08T21:14:58.914 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 1536 offset 4195840 len 2048 2024-03-08T21:14:58.914 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1536 offset 4195840 len 2048 2024-03-08T21:14:58.914 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 1536 offset 4195840 len 4096 2024-03-08T21:14:58.915 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1536 offset 4195840 len 4096 2024-03-08T21:14:58.915 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 1536 offset 4195840 len 4096 2024-03-08T21:14:58.915 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1536 offset 4195840 len 4096 2024-03-08T21:14:58.915 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 1536 offset 4195840 len 8192 2024-03-08T21:14:58.915 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1536 offset 4195840 len 8192 2024-03-08T21:14:58.915 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 1536 offset 4195840 len 8192 2024-03-08T21:14:58.915 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1536 offset 4195840 len 8192 2024-03-08T21:14:58.915 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 1536 offset 4195840 len 16384 2024-03-08T21:14:58.916 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1536 offset 4195840 len 16384 2024-03-08T21:14:58.916 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 1536 offset 4195840 len 16384 2024-03-08T21:14:58.916 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1536 offset 4195840 len 16384 2024-03-08T21:14:58.916 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 1536 offset 4196352 len 1024 2024-03-08T21:14:58.916 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1536 offset 4196352 len 1024 2024-03-08T21:14:58.916 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 1536 offset 4196352 len 1024 2024-03-08T21:14:58.916 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1536 offset 4196352 len 1024 2024-03-08T21:14:58.917 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 1536 offset 4196352 len 2048 2024-03-08T21:14:58.917 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1536 offset 4196352 len 2048 2024-03-08T21:14:58.917 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 1536 offset 4196352 len 2048 2024-03-08T21:14:58.917 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1536 offset 4196352 len 2048 2024-03-08T21:14:58.917 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 1536 offset 4196352 len 4096 2024-03-08T21:14:58.917 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1536 offset 4196352 len 4096 2024-03-08T21:14:58.917 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 1536 offset 4196352 len 4096 2024-03-08T21:14:58.918 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1536 offset 4196352 len 4096 2024-03-08T21:14:58.918 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 1536 offset 4196352 len 8192 2024-03-08T21:14:58.918 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1536 offset 4196352 len 8192 2024-03-08T21:14:58.918 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 1536 offset 4196352 len 8192 2024-03-08T21:14:58.918 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1536 offset 4196352 len 8192 2024-03-08T21:14:58.918 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 1536 offset 4196352 len 16384 2024-03-08T21:14:58.919 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1536 offset 4196352 len 16384 2024-03-08T21:14:58.919 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 1536 offset 4196352 len 16384 2024-03-08T21:14:58.919 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1536 offset 4196352 len 16384 2024-03-08T21:14:58.919 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 1536 offset 4196864 len 1024 2024-03-08T21:14:58.919 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1536 offset 4196864 len 1024 2024-03-08T21:14:58.919 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 1536 offset 4196864 len 1024 2024-03-08T21:14:58.919 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1536 offset 4196864 len 1024 2024-03-08T21:14:58.920 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 1536 offset 4196864 len 2048 2024-03-08T21:14:58.920 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1536 offset 4196864 len 2048 2024-03-08T21:14:58.920 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 1536 offset 4196864 len 2048 2024-03-08T21:14:58.920 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1536 offset 4196864 len 2048 2024-03-08T21:14:58.920 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 1536 offset 4196864 len 4096 2024-03-08T21:14:58.921 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1536 offset 4196864 len 4096 2024-03-08T21:14:58.921 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 1536 offset 4196864 len 4096 2024-03-08T21:14:58.921 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1536 offset 4196864 len 4096 2024-03-08T21:14:58.921 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 1536 offset 4196864 len 8192 2024-03-08T21:14:58.921 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1536 offset 4196864 len 8192 2024-03-08T21:14:58.921 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 1536 offset 4196864 len 8192 2024-03-08T21:14:58.921 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1536 offset 4196864 len 8192 2024-03-08T21:14:58.922 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 1536 offset 4196864 len 16384 2024-03-08T21:14:58.922 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1536 offset 4196864 len 16384 2024-03-08T21:14:58.922 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 1536 offset 4196864 len 16384 2024-03-08T21:14:58.922 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1536 offset 4196864 len 16384 2024-03-08T21:14:58.922 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 1536 offset 4197376 len 1024 2024-03-08T21:14:58.922 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1536 offset 4197376 len 1024 2024-03-08T21:14:58.923 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 1536 offset 4197376 len 1024 2024-03-08T21:14:58.923 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1536 offset 4197376 len 1024 2024-03-08T21:14:58.923 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 1536 offset 4197376 len 2048 2024-03-08T21:14:58.923 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1536 offset 4197376 len 2048 2024-03-08T21:14:58.923 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 1536 offset 4197376 len 2048 2024-03-08T21:14:58.923 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1536 offset 4197376 len 2048 2024-03-08T21:14:58.923 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 1536 offset 4197376 len 4096 2024-03-08T21:14:58.924 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1536 offset 4197376 len 4096 2024-03-08T21:14:58.924 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 1536 offset 4197376 len 4096 2024-03-08T21:14:58.924 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1536 offset 4197376 len 4096 2024-03-08T21:14:58.924 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 1536 offset 4197376 len 8192 2024-03-08T21:14:58.924 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1536 offset 4197376 len 8192 2024-03-08T21:14:58.924 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 1536 offset 4197376 len 8192 2024-03-08T21:14:58.925 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1536 offset 4197376 len 8192 2024-03-08T21:14:58.925 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 1536 offset 4197376 len 16384 2024-03-08T21:14:58.925 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1536 offset 4197376 len 16384 2024-03-08T21:14:58.925 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 1536 offset 4197376 len 16384 2024-03-08T21:14:58.925 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1536 offset 4197376 len 16384 2024-03-08T21:14:58.925 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 1536 offset 4197888 len 1024 2024-03-08T21:14:59.008 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1536 offset 4197888 len 1024 2024-03-08T21:14:59.009 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 1536 offset 4197888 len 1024 2024-03-08T21:14:59.009 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1536 offset 4197888 len 1024 2024-03-08T21:14:59.009 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 1536 offset 4197888 len 2048 2024-03-08T21:14:59.009 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1536 offset 4197888 len 2048 2024-03-08T21:14:59.009 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 1536 offset 4197888 len 2048 2024-03-08T21:14:59.009 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1536 offset 4197888 len 2048 2024-03-08T21:14:59.010 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 1536 offset 4197888 len 4096 2024-03-08T21:14:59.010 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1536 offset 4197888 len 4096 2024-03-08T21:14:59.010 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 1536 offset 4197888 len 4096 2024-03-08T21:14:59.010 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1536 offset 4197888 len 4096 2024-03-08T21:14:59.010 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 1536 offset 4197888 len 8192 2024-03-08T21:14:59.010 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1536 offset 4197888 len 8192 2024-03-08T21:14:59.011 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 1536 offset 4197888 len 8192 2024-03-08T21:14:59.011 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1536 offset 4197888 len 8192 2024-03-08T21:14:59.011 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 1536 offset 4197888 len 16384 2024-03-08T21:14:59.011 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1536 offset 4197888 len 16384 2024-03-08T21:14:59.011 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 1536 offset 4197888 len 16384 2024-03-08T21:14:59.011 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 1536 offset 4197888 len 16384 2024-03-08T21:14:59.012 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 2048 offset 4190208 len 1024 2024-03-08T21:14:59.012 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2048 offset 4190208 len 1024 2024-03-08T21:14:59.012 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 2048 offset 4190208 len 1024 2024-03-08T21:14:59.012 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2048 offset 4190208 len 1024 2024-03-08T21:14:59.012 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 2048 offset 4190208 len 2048 2024-03-08T21:14:59.012 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2048 offset 4190208 len 2048 2024-03-08T21:14:59.013 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 2048 offset 4190208 len 2048 2024-03-08T21:14:59.013 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2048 offset 4190208 len 2048 2024-03-08T21:14:59.013 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 2048 offset 4190208 len 4096 2024-03-08T21:14:59.013 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2048 offset 4190208 len 4096 2024-03-08T21:14:59.013 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 2048 offset 4190208 len 4096 2024-03-08T21:14:59.013 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2048 offset 4190208 len 4096 2024-03-08T21:14:59.014 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 2048 offset 4190208 len 8192 2024-03-08T21:14:59.014 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2048 offset 4190208 len 8192 2024-03-08T21:14:59.014 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 2048 offset 4190208 len 8192 2024-03-08T21:14:59.014 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2048 offset 4190208 len 8192 2024-03-08T21:14:59.014 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 2048 offset 4190208 len 16384 2024-03-08T21:14:59.014 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2048 offset 4190208 len 16384 2024-03-08T21:14:59.015 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 2048 offset 4190208 len 16384 2024-03-08T21:14:59.015 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2048 offset 4190208 len 16384 2024-03-08T21:14:59.015 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 2048 offset 4190720 len 1024 2024-03-08T21:14:59.015 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2048 offset 4190720 len 1024 2024-03-08T21:14:59.015 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 2048 offset 4190720 len 1024 2024-03-08T21:14:59.015 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2048 offset 4190720 len 1024 2024-03-08T21:14:59.016 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 2048 offset 4190720 len 2048 2024-03-08T21:14:59.016 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2048 offset 4190720 len 2048 2024-03-08T21:14:59.016 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 2048 offset 4190720 len 2048 2024-03-08T21:14:59.016 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2048 offset 4190720 len 2048 2024-03-08T21:14:59.016 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 2048 offset 4190720 len 4096 2024-03-08T21:14:59.016 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2048 offset 4190720 len 4096 2024-03-08T21:14:59.016 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 2048 offset 4190720 len 4096 2024-03-08T21:14:59.017 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2048 offset 4190720 len 4096 2024-03-08T21:14:59.017 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 2048 offset 4190720 len 8192 2024-03-08T21:14:59.017 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2048 offset 4190720 len 8192 2024-03-08T21:14:59.017 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 2048 offset 4190720 len 8192 2024-03-08T21:14:59.017 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2048 offset 4190720 len 8192 2024-03-08T21:14:59.017 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 2048 offset 4190720 len 16384 2024-03-08T21:14:59.018 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2048 offset 4190720 len 16384 2024-03-08T21:14:59.018 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 2048 offset 4190720 len 16384 2024-03-08T21:14:59.018 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2048 offset 4190720 len 16384 2024-03-08T21:14:59.018 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 2048 offset 4191232 len 1024 2024-03-08T21:14:59.018 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2048 offset 4191232 len 1024 2024-03-08T21:14:59.018 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 2048 offset 4191232 len 1024 2024-03-08T21:14:59.019 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2048 offset 4191232 len 1024 2024-03-08T21:14:59.019 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 2048 offset 4191232 len 2048 2024-03-08T21:14:59.019 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2048 offset 4191232 len 2048 2024-03-08T21:14:59.019 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 2048 offset 4191232 len 2048 2024-03-08T21:14:59.019 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2048 offset 4191232 len 2048 2024-03-08T21:14:59.019 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 2048 offset 4191232 len 4096 2024-03-08T21:14:59.020 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2048 offset 4191232 len 4096 2024-03-08T21:14:59.020 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 2048 offset 4191232 len 4096 2024-03-08T21:14:59.020 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2048 offset 4191232 len 4096 2024-03-08T21:14:59.020 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 2048 offset 4191232 len 8192 2024-03-08T21:14:59.020 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2048 offset 4191232 len 8192 2024-03-08T21:14:59.020 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 2048 offset 4191232 len 8192 2024-03-08T21:14:59.020 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2048 offset 4191232 len 8192 2024-03-08T21:14:59.021 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 2048 offset 4191232 len 16384 2024-03-08T21:14:59.021 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2048 offset 4191232 len 16384 2024-03-08T21:14:59.021 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 2048 offset 4191232 len 16384 2024-03-08T21:14:59.021 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2048 offset 4191232 len 16384 2024-03-08T21:14:59.021 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 2048 offset 4191744 len 1024 2024-03-08T21:14:59.021 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2048 offset 4191744 len 1024 2024-03-08T21:14:59.022 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 2048 offset 4191744 len 1024 2024-03-08T21:14:59.022 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2048 offset 4191744 len 1024 2024-03-08T21:14:59.102 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 2048 offset 4191744 len 2048 2024-03-08T21:14:59.102 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2048 offset 4191744 len 2048 2024-03-08T21:14:59.102 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 2048 offset 4191744 len 2048 2024-03-08T21:14:59.102 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2048 offset 4191744 len 2048 2024-03-08T21:14:59.103 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 2048 offset 4191744 len 4096 2024-03-08T21:14:59.103 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2048 offset 4191744 len 4096 2024-03-08T21:14:59.103 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 2048 offset 4191744 len 4096 2024-03-08T21:14:59.103 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2048 offset 4191744 len 4096 2024-03-08T21:14:59.103 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 2048 offset 4191744 len 8192 2024-03-08T21:14:59.103 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2048 offset 4191744 len 8192 2024-03-08T21:14:59.103 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 2048 offset 4191744 len 8192 2024-03-08T21:14:59.103 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2048 offset 4191744 len 8192 2024-03-08T21:14:59.103 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 2048 offset 4191744 len 16384 2024-03-08T21:14:59.104 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2048 offset 4191744 len 16384 2024-03-08T21:14:59.104 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 2048 offset 4191744 len 16384 2024-03-08T21:14:59.104 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2048 offset 4191744 len 16384 2024-03-08T21:14:59.104 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 2048 offset 4192256 len 1024 2024-03-08T21:14:59.104 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2048 offset 4192256 len 1024 2024-03-08T21:14:59.104 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 2048 offset 4192256 len 1024 2024-03-08T21:14:59.104 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2048 offset 4192256 len 1024 2024-03-08T21:14:59.104 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 2048 offset 4192256 len 2048 2024-03-08T21:14:59.105 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2048 offset 4192256 len 2048 2024-03-08T21:14:59.105 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 2048 offset 4192256 len 2048 2024-03-08T21:14:59.105 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2048 offset 4192256 len 2048 2024-03-08T21:14:59.105 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 2048 offset 4192256 len 4096 2024-03-08T21:14:59.105 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2048 offset 4192256 len 4096 2024-03-08T21:14:59.105 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 2048 offset 4192256 len 4096 2024-03-08T21:14:59.105 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2048 offset 4192256 len 4096 2024-03-08T21:14:59.105 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 2048 offset 4192256 len 8192 2024-03-08T21:14:59.106 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2048 offset 4192256 len 8192 2024-03-08T21:14:59.106 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 2048 offset 4192256 len 8192 2024-03-08T21:14:59.106 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2048 offset 4192256 len 8192 2024-03-08T21:14:59.106 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 2048 offset 4192256 len 16384 2024-03-08T21:14:59.106 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2048 offset 4192256 len 16384 2024-03-08T21:14:59.106 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 2048 offset 4192256 len 16384 2024-03-08T21:14:59.106 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2048 offset 4192256 len 16384 2024-03-08T21:14:59.106 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 2048 offset 4192768 len 1024 2024-03-08T21:14:59.107 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2048 offset 4192768 len 1024 2024-03-08T21:14:59.107 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 2048 offset 4192768 len 1024 2024-03-08T21:14:59.107 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2048 offset 4192768 len 1024 2024-03-08T21:14:59.107 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 2048 offset 4192768 len 2048 2024-03-08T21:14:59.107 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2048 offset 4192768 len 2048 2024-03-08T21:14:59.107 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 2048 offset 4192768 len 2048 2024-03-08T21:14:59.107 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2048 offset 4192768 len 2048 2024-03-08T21:14:59.107 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 2048 offset 4192768 len 4096 2024-03-08T21:14:59.108 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2048 offset 4192768 len 4096 2024-03-08T21:14:59.108 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 2048 offset 4192768 len 4096 2024-03-08T21:14:59.108 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2048 offset 4192768 len 4096 2024-03-08T21:14:59.108 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 2048 offset 4192768 len 8192 2024-03-08T21:14:59.108 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2048 offset 4192768 len 8192 2024-03-08T21:14:59.108 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 2048 offset 4192768 len 8192 2024-03-08T21:14:59.108 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2048 offset 4192768 len 8192 2024-03-08T21:14:59.108 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 2048 offset 4192768 len 16384 2024-03-08T21:14:59.108 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2048 offset 4192768 len 16384 2024-03-08T21:14:59.109 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 2048 offset 4192768 len 16384 2024-03-08T21:14:59.109 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2048 offset 4192768 len 16384 2024-03-08T21:14:59.109 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 2048 offset 4193280 len 1024 2024-03-08T21:14:59.109 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2048 offset 4193280 len 1024 2024-03-08T21:14:59.109 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 2048 offset 4193280 len 1024 2024-03-08T21:14:59.109 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2048 offset 4193280 len 1024 2024-03-08T21:14:59.109 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 2048 offset 4193280 len 2048 2024-03-08T21:14:59.109 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2048 offset 4193280 len 2048 2024-03-08T21:14:59.110 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 2048 offset 4193280 len 2048 2024-03-08T21:14:59.110 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2048 offset 4193280 len 2048 2024-03-08T21:14:59.110 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 2048 offset 4193280 len 4096 2024-03-08T21:14:59.110 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2048 offset 4193280 len 4096 2024-03-08T21:14:59.110 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 2048 offset 4193280 len 4096 2024-03-08T21:14:59.110 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2048 offset 4193280 len 4096 2024-03-08T21:14:59.110 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 2048 offset 4193280 len 8192 2024-03-08T21:14:59.110 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2048 offset 4193280 len 8192 2024-03-08T21:14:59.111 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 2048 offset 4193280 len 8192 2024-03-08T21:14:59.111 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2048 offset 4193280 len 8192 2024-03-08T21:14:59.111 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 2048 offset 4193280 len 16384 2024-03-08T21:14:59.111 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2048 offset 4193280 len 16384 2024-03-08T21:14:59.111 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 2048 offset 4193280 len 16384 2024-03-08T21:14:59.111 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2048 offset 4193280 len 16384 2024-03-08T21:14:59.111 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 2048 offset 4193792 len 1024 2024-03-08T21:14:59.111 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2048 offset 4193792 len 1024 2024-03-08T21:14:59.111 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 2048 offset 4193792 len 1024 2024-03-08T21:14:59.112 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2048 offset 4193792 len 1024 2024-03-08T21:14:59.112 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 2048 offset 4193792 len 2048 2024-03-08T21:14:59.112 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2048 offset 4193792 len 2048 2024-03-08T21:14:59.187 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 2048 offset 4193792 len 2048 2024-03-08T21:14:59.187 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2048 offset 4193792 len 2048 2024-03-08T21:14:59.187 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 2048 offset 4193792 len 4096 2024-03-08T21:14:59.187 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2048 offset 4193792 len 4096 2024-03-08T21:14:59.187 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 2048 offset 4193792 len 4096 2024-03-08T21:14:59.187 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2048 offset 4193792 len 4096 2024-03-08T21:14:59.187 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 2048 offset 4193792 len 8192 2024-03-08T21:14:59.187 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2048 offset 4193792 len 8192 2024-03-08T21:14:59.187 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 2048 offset 4193792 len 8192 2024-03-08T21:14:59.187 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2048 offset 4193792 len 8192 2024-03-08T21:14:59.188 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 2048 offset 4193792 len 16384 2024-03-08T21:14:59.188 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2048 offset 4193792 len 16384 2024-03-08T21:14:59.188 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 2048 offset 4193792 len 16384 2024-03-08T21:14:59.188 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2048 offset 4193792 len 16384 2024-03-08T21:14:59.188 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 2048 offset 4194304 len 1024 2024-03-08T21:14:59.188 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2048 offset 4194304 len 1024 2024-03-08T21:14:59.188 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 2048 offset 4194304 len 1024 2024-03-08T21:14:59.188 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2048 offset 4194304 len 1024 2024-03-08T21:14:59.188 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 2048 offset 4194304 len 2048 2024-03-08T21:14:59.188 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2048 offset 4194304 len 2048 2024-03-08T21:14:59.188 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 2048 offset 4194304 len 2048 2024-03-08T21:14:59.188 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2048 offset 4194304 len 2048 2024-03-08T21:14:59.188 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 2048 offset 4194304 len 4096 2024-03-08T21:14:59.188 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2048 offset 4194304 len 4096 2024-03-08T21:14:59.188 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 2048 offset 4194304 len 4096 2024-03-08T21:14:59.189 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2048 offset 4194304 len 4096 2024-03-08T21:14:59.189 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 2048 offset 4194304 len 8192 2024-03-08T21:14:59.189 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2048 offset 4194304 len 8192 2024-03-08T21:14:59.189 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 2048 offset 4194304 len 8192 2024-03-08T21:14:59.189 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2048 offset 4194304 len 8192 2024-03-08T21:14:59.189 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 2048 offset 4194304 len 16384 2024-03-08T21:14:59.189 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2048 offset 4194304 len 16384 2024-03-08T21:14:59.189 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 2048 offset 4194304 len 16384 2024-03-08T21:14:59.189 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2048 offset 4194304 len 16384 2024-03-08T21:14:59.189 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 2048 offset 4194816 len 1024 2024-03-08T21:14:59.189 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2048 offset 4194816 len 1024 2024-03-08T21:14:59.189 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 2048 offset 4194816 len 1024 2024-03-08T21:14:59.189 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2048 offset 4194816 len 1024 2024-03-08T21:14:59.189 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 2048 offset 4194816 len 2048 2024-03-08T21:14:59.189 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2048 offset 4194816 len 2048 2024-03-08T21:14:59.190 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 2048 offset 4194816 len 2048 2024-03-08T21:14:59.190 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2048 offset 4194816 len 2048 2024-03-08T21:14:59.190 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 2048 offset 4194816 len 4096 2024-03-08T21:14:59.190 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2048 offset 4194816 len 4096 2024-03-08T21:14:59.190 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 2048 offset 4194816 len 4096 2024-03-08T21:14:59.190 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2048 offset 4194816 len 4096 2024-03-08T21:14:59.190 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 2048 offset 4194816 len 8192 2024-03-08T21:14:59.190 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2048 offset 4194816 len 8192 2024-03-08T21:14:59.190 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 2048 offset 4194816 len 8192 2024-03-08T21:14:59.190 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2048 offset 4194816 len 8192 2024-03-08T21:14:59.190 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 2048 offset 4194816 len 16384 2024-03-08T21:14:59.190 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2048 offset 4194816 len 16384 2024-03-08T21:14:59.190 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 2048 offset 4194816 len 16384 2024-03-08T21:14:59.190 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2048 offset 4194816 len 16384 2024-03-08T21:14:59.190 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 2048 offset 4195328 len 1024 2024-03-08T21:14:59.191 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2048 offset 4195328 len 1024 2024-03-08T21:14:59.191 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 2048 offset 4195328 len 1024 2024-03-08T21:14:59.191 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2048 offset 4195328 len 1024 2024-03-08T21:14:59.191 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 2048 offset 4195328 len 2048 2024-03-08T21:14:59.191 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2048 offset 4195328 len 2048 2024-03-08T21:14:59.191 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 2048 offset 4195328 len 2048 2024-03-08T21:14:59.191 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2048 offset 4195328 len 2048 2024-03-08T21:14:59.191 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 2048 offset 4195328 len 4096 2024-03-08T21:14:59.191 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2048 offset 4195328 len 4096 2024-03-08T21:14:59.191 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 2048 offset 4195328 len 4096 2024-03-08T21:14:59.191 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2048 offset 4195328 len 4096 2024-03-08T21:14:59.191 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 2048 offset 4195328 len 8192 2024-03-08T21:14:59.191 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2048 offset 4195328 len 8192 2024-03-08T21:14:59.191 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 2048 offset 4195328 len 8192 2024-03-08T21:14:59.191 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2048 offset 4195328 len 8192 2024-03-08T21:14:59.191 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 2048 offset 4195328 len 16384 2024-03-08T21:14:59.192 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2048 offset 4195328 len 16384 2024-03-08T21:14:59.192 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 2048 offset 4195328 len 16384 2024-03-08T21:14:59.192 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2048 offset 4195328 len 16384 2024-03-08T21:14:59.192 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 2048 offset 4195840 len 1024 2024-03-08T21:14:59.192 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2048 offset 4195840 len 1024 2024-03-08T21:14:59.192 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 2048 offset 4195840 len 1024 2024-03-08T21:14:59.192 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2048 offset 4195840 len 1024 2024-03-08T21:14:59.192 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 2048 offset 4195840 len 2048 2024-03-08T21:14:59.192 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2048 offset 4195840 len 2048 2024-03-08T21:14:59.192 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 2048 offset 4195840 len 2048 2024-03-08T21:14:59.192 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2048 offset 4195840 len 2048 2024-03-08T21:14:59.279 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 2048 offset 4195840 len 4096 2024-03-08T21:14:59.280 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2048 offset 4195840 len 4096 2024-03-08T21:14:59.280 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 2048 offset 4195840 len 4096 2024-03-08T21:14:59.280 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2048 offset 4195840 len 4096 2024-03-08T21:14:59.280 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 2048 offset 4195840 len 8192 2024-03-08T21:14:59.280 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2048 offset 4195840 len 8192 2024-03-08T21:14:59.280 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 2048 offset 4195840 len 8192 2024-03-08T21:14:59.280 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2048 offset 4195840 len 8192 2024-03-08T21:14:59.280 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 2048 offset 4195840 len 16384 2024-03-08T21:14:59.280 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2048 offset 4195840 len 16384 2024-03-08T21:14:59.281 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 2048 offset 4195840 len 16384 2024-03-08T21:14:59.281 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2048 offset 4195840 len 16384 2024-03-08T21:14:59.281 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 2048 offset 4196352 len 1024 2024-03-08T21:14:59.281 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2048 offset 4196352 len 1024 2024-03-08T21:14:59.281 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 2048 offset 4196352 len 1024 2024-03-08T21:14:59.281 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2048 offset 4196352 len 1024 2024-03-08T21:14:59.281 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 2048 offset 4196352 len 2048 2024-03-08T21:14:59.281 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2048 offset 4196352 len 2048 2024-03-08T21:14:59.281 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 2048 offset 4196352 len 2048 2024-03-08T21:14:59.281 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2048 offset 4196352 len 2048 2024-03-08T21:14:59.282 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 2048 offset 4196352 len 4096 2024-03-08T21:14:59.282 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2048 offset 4196352 len 4096 2024-03-08T21:14:59.282 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 2048 offset 4196352 len 4096 2024-03-08T21:14:59.282 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2048 offset 4196352 len 4096 2024-03-08T21:14:59.282 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 2048 offset 4196352 len 8192 2024-03-08T21:14:59.282 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2048 offset 4196352 len 8192 2024-03-08T21:14:59.282 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 2048 offset 4196352 len 8192 2024-03-08T21:14:59.282 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2048 offset 4196352 len 8192 2024-03-08T21:14:59.282 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 2048 offset 4196352 len 16384 2024-03-08T21:14:59.282 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2048 offset 4196352 len 16384 2024-03-08T21:14:59.282 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 2048 offset 4196352 len 16384 2024-03-08T21:14:59.282 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2048 offset 4196352 len 16384 2024-03-08T21:14:59.283 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 2048 offset 4196864 len 1024 2024-03-08T21:14:59.283 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2048 offset 4196864 len 1024 2024-03-08T21:14:59.283 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 2048 offset 4196864 len 1024 2024-03-08T21:14:59.283 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2048 offset 4196864 len 1024 2024-03-08T21:14:59.283 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 2048 offset 4196864 len 2048 2024-03-08T21:14:59.283 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2048 offset 4196864 len 2048 2024-03-08T21:14:59.283 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 2048 offset 4196864 len 2048 2024-03-08T21:14:59.283 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2048 offset 4196864 len 2048 2024-03-08T21:14:59.283 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 2048 offset 4196864 len 4096 2024-03-08T21:14:59.283 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2048 offset 4196864 len 4096 2024-03-08T21:14:59.283 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 2048 offset 4196864 len 4096 2024-03-08T21:14:59.283 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2048 offset 4196864 len 4096 2024-03-08T21:14:59.283 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 2048 offset 4196864 len 8192 2024-03-08T21:14:59.283 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2048 offset 4196864 len 8192 2024-03-08T21:14:59.283 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 2048 offset 4196864 len 8192 2024-03-08T21:14:59.284 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2048 offset 4196864 len 8192 2024-03-08T21:14:59.284 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 2048 offset 4196864 len 16384 2024-03-08T21:14:59.284 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2048 offset 4196864 len 16384 2024-03-08T21:14:59.285 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 2048 offset 4196864 len 16384 2024-03-08T21:14:59.285 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2048 offset 4196864 len 16384 2024-03-08T21:14:59.286 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 2048 offset 4197376 len 1024 2024-03-08T21:14:59.286 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2048 offset 4197376 len 1024 2024-03-08T21:14:59.286 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 2048 offset 4197376 len 1024 2024-03-08T21:14:59.286 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2048 offset 4197376 len 1024 2024-03-08T21:14:59.286 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 2048 offset 4197376 len 2048 2024-03-08T21:14:59.286 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2048 offset 4197376 len 2048 2024-03-08T21:14:59.286 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 2048 offset 4197376 len 2048 2024-03-08T21:14:59.286 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2048 offset 4197376 len 2048 2024-03-08T21:14:59.286 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 2048 offset 4197376 len 4096 2024-03-08T21:14:59.286 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2048 offset 4197376 len 4096 2024-03-08T21:14:59.286 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 2048 offset 4197376 len 4096 2024-03-08T21:14:59.287 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2048 offset 4197376 len 4096 2024-03-08T21:14:59.287 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 2048 offset 4197376 len 8192 2024-03-08T21:14:59.287 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2048 offset 4197376 len 8192 2024-03-08T21:14:59.287 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 2048 offset 4197376 len 8192 2024-03-08T21:14:59.287 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2048 offset 4197376 len 8192 2024-03-08T21:14:59.287 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 2048 offset 4197376 len 16384 2024-03-08T21:14:59.287 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2048 offset 4197376 len 16384 2024-03-08T21:14:59.287 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 2048 offset 4197376 len 16384 2024-03-08T21:14:59.287 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2048 offset 4197376 len 16384 2024-03-08T21:14:59.287 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 2048 offset 4197888 len 1024 2024-03-08T21:14:59.287 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2048 offset 4197888 len 1024 2024-03-08T21:14:59.288 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 2048 offset 4197888 len 1024 2024-03-08T21:14:59.288 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2048 offset 4197888 len 1024 2024-03-08T21:14:59.288 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 2048 offset 4197888 len 2048 2024-03-08T21:14:59.288 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2048 offset 4197888 len 2048 2024-03-08T21:14:59.288 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 2048 offset 4197888 len 2048 2024-03-08T21:14:59.288 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2048 offset 4197888 len 2048 2024-03-08T21:14:59.288 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 2048 offset 4197888 len 4096 2024-03-08T21:14:59.288 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2048 offset 4197888 len 4096 2024-03-08T21:14:59.288 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 2048 offset 4197888 len 4096 2024-03-08T21:14:59.369 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2048 offset 4197888 len 4096 2024-03-08T21:14:59.369 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 2048 offset 4197888 len 8192 2024-03-08T21:14:59.369 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2048 offset 4197888 len 8192 2024-03-08T21:14:59.370 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 2048 offset 4197888 len 8192 2024-03-08T21:14:59.370 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2048 offset 4197888 len 8192 2024-03-08T21:14:59.370 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 2048 offset 4197888 len 16384 2024-03-08T21:14:59.370 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2048 offset 4197888 len 16384 2024-03-08T21:14:59.370 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 2048 offset 4197888 len 16384 2024-03-08T21:14:59.370 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2048 offset 4197888 len 16384 2024-03-08T21:14:59.370 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 2560 offset 4190208 len 1024 2024-03-08T21:14:59.370 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2560 offset 4190208 len 1024 2024-03-08T21:14:59.370 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 2560 offset 4190208 len 1024 2024-03-08T21:14:59.370 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2560 offset 4190208 len 1024 2024-03-08T21:14:59.371 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 2560 offset 4190208 len 2048 2024-03-08T21:14:59.371 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2560 offset 4190208 len 2048 2024-03-08T21:14:59.371 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 2560 offset 4190208 len 2048 2024-03-08T21:14:59.371 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2560 offset 4190208 len 2048 2024-03-08T21:14:59.371 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 2560 offset 4190208 len 4096 2024-03-08T21:14:59.371 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2560 offset 4190208 len 4096 2024-03-08T21:14:59.371 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 2560 offset 4190208 len 4096 2024-03-08T21:14:59.371 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2560 offset 4190208 len 4096 2024-03-08T21:14:59.371 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 2560 offset 4190208 len 8192 2024-03-08T21:14:59.371 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2560 offset 4190208 len 8192 2024-03-08T21:14:59.372 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 2560 offset 4190208 len 8192 2024-03-08T21:14:59.372 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2560 offset 4190208 len 8192 2024-03-08T21:14:59.372 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 2560 offset 4190208 len 16384 2024-03-08T21:14:59.372 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2560 offset 4190208 len 16384 2024-03-08T21:14:59.372 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 2560 offset 4190208 len 16384 2024-03-08T21:14:59.372 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2560 offset 4190208 len 16384 2024-03-08T21:14:59.372 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 2560 offset 4190720 len 1024 2024-03-08T21:14:59.372 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2560 offset 4190720 len 1024 2024-03-08T21:14:59.372 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 2560 offset 4190720 len 1024 2024-03-08T21:14:59.372 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2560 offset 4190720 len 1024 2024-03-08T21:14:59.372 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 2560 offset 4190720 len 2048 2024-03-08T21:14:59.372 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2560 offset 4190720 len 2048 2024-03-08T21:14:59.373 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 2560 offset 4190720 len 2048 2024-03-08T21:14:59.373 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2560 offset 4190720 len 2048 2024-03-08T21:14:59.373 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 2560 offset 4190720 len 4096 2024-03-08T21:14:59.373 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2560 offset 4190720 len 4096 2024-03-08T21:14:59.373 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 2560 offset 4190720 len 4096 2024-03-08T21:14:59.373 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2560 offset 4190720 len 4096 2024-03-08T21:14:59.373 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 2560 offset 4190720 len 8192 2024-03-08T21:14:59.373 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2560 offset 4190720 len 8192 2024-03-08T21:14:59.373 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 2560 offset 4190720 len 8192 2024-03-08T21:14:59.373 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2560 offset 4190720 len 8192 2024-03-08T21:14:59.373 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 2560 offset 4190720 len 16384 2024-03-08T21:14:59.373 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2560 offset 4190720 len 16384 2024-03-08T21:14:59.373 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 2560 offset 4190720 len 16384 2024-03-08T21:14:59.373 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2560 offset 4190720 len 16384 2024-03-08T21:14:59.373 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 2560 offset 4191232 len 1024 2024-03-08T21:14:59.374 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2560 offset 4191232 len 1024 2024-03-08T21:14:59.374 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 2560 offset 4191232 len 1024 2024-03-08T21:14:59.374 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2560 offset 4191232 len 1024 2024-03-08T21:14:59.374 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 2560 offset 4191232 len 2048 2024-03-08T21:14:59.374 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2560 offset 4191232 len 2048 2024-03-08T21:14:59.374 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 2560 offset 4191232 len 2048 2024-03-08T21:14:59.374 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2560 offset 4191232 len 2048 2024-03-08T21:14:59.374 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 2560 offset 4191232 len 4096 2024-03-08T21:14:59.374 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2560 offset 4191232 len 4096 2024-03-08T21:14:59.374 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 2560 offset 4191232 len 4096 2024-03-08T21:14:59.374 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2560 offset 4191232 len 4096 2024-03-08T21:14:59.374 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 2560 offset 4191232 len 8192 2024-03-08T21:14:59.374 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2560 offset 4191232 len 8192 2024-03-08T21:14:59.374 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 2560 offset 4191232 len 8192 2024-03-08T21:14:59.375 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2560 offset 4191232 len 8192 2024-03-08T21:14:59.375 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 2560 offset 4191232 len 16384 2024-03-08T21:14:59.375 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2560 offset 4191232 len 16384 2024-03-08T21:14:59.375 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 2560 offset 4191232 len 16384 2024-03-08T21:14:59.375 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2560 offset 4191232 len 16384 2024-03-08T21:14:59.375 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 2560 offset 4191744 len 1024 2024-03-08T21:14:59.375 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2560 offset 4191744 len 1024 2024-03-08T21:14:59.375 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 2560 offset 4191744 len 1024 2024-03-08T21:14:59.375 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2560 offset 4191744 len 1024 2024-03-08T21:14:59.375 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 2560 offset 4191744 len 2048 2024-03-08T21:14:59.375 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2560 offset 4191744 len 2048 2024-03-08T21:14:59.375 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 2560 offset 4191744 len 2048 2024-03-08T21:14:59.375 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2560 offset 4191744 len 2048 2024-03-08T21:14:59.375 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 2560 offset 4191744 len 4096 2024-03-08T21:14:59.375 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2560 offset 4191744 len 4096 2024-03-08T21:14:59.376 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 2560 offset 4191744 len 4096 2024-03-08T21:14:59.376 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2560 offset 4191744 len 4096 2024-03-08T21:14:59.376 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 2560 offset 4191744 len 8192 2024-03-08T21:14:59.464 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2560 offset 4191744 len 8192 2024-03-08T21:14:59.465 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 2560 offset 4191744 len 8192 2024-03-08T21:14:59.465 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2560 offset 4191744 len 8192 2024-03-08T21:14:59.465 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 2560 offset 4191744 len 16384 2024-03-08T21:14:59.465 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2560 offset 4191744 len 16384 2024-03-08T21:14:59.465 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 2560 offset 4191744 len 16384 2024-03-08T21:14:59.465 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2560 offset 4191744 len 16384 2024-03-08T21:14:59.465 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 2560 offset 4192256 len 1024 2024-03-08T21:14:59.465 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2560 offset 4192256 len 1024 2024-03-08T21:14:59.465 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 2560 offset 4192256 len 1024 2024-03-08T21:14:59.465 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2560 offset 4192256 len 1024 2024-03-08T21:14:59.465 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 2560 offset 4192256 len 2048 2024-03-08T21:14:59.466 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2560 offset 4192256 len 2048 2024-03-08T21:14:59.466 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 2560 offset 4192256 len 2048 2024-03-08T21:14:59.466 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2560 offset 4192256 len 2048 2024-03-08T21:14:59.466 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 2560 offset 4192256 len 4096 2024-03-08T21:14:59.466 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2560 offset 4192256 len 4096 2024-03-08T21:14:59.466 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 2560 offset 4192256 len 4096 2024-03-08T21:14:59.466 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2560 offset 4192256 len 4096 2024-03-08T21:14:59.466 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 2560 offset 4192256 len 8192 2024-03-08T21:14:59.466 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2560 offset 4192256 len 8192 2024-03-08T21:14:59.467 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 2560 offset 4192256 len 8192 2024-03-08T21:14:59.467 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2560 offset 4192256 len 8192 2024-03-08T21:14:59.467 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 2560 offset 4192256 len 16384 2024-03-08T21:14:59.467 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2560 offset 4192256 len 16384 2024-03-08T21:14:59.467 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 2560 offset 4192256 len 16384 2024-03-08T21:14:59.467 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2560 offset 4192256 len 16384 2024-03-08T21:14:59.467 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 2560 offset 4192768 len 1024 2024-03-08T21:14:59.467 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2560 offset 4192768 len 1024 2024-03-08T21:14:59.467 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 2560 offset 4192768 len 1024 2024-03-08T21:14:59.467 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2560 offset 4192768 len 1024 2024-03-08T21:14:59.467 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 2560 offset 4192768 len 2048 2024-03-08T21:14:59.467 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2560 offset 4192768 len 2048 2024-03-08T21:14:59.467 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 2560 offset 4192768 len 2048 2024-03-08T21:14:59.467 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2560 offset 4192768 len 2048 2024-03-08T21:14:59.468 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 2560 offset 4192768 len 4096 2024-03-08T21:14:59.468 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2560 offset 4192768 len 4096 2024-03-08T21:14:59.468 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 2560 offset 4192768 len 4096 2024-03-08T21:14:59.468 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2560 offset 4192768 len 4096 2024-03-08T21:14:59.468 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 2560 offset 4192768 len 8192 2024-03-08T21:14:59.468 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2560 offset 4192768 len 8192 2024-03-08T21:14:59.468 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 2560 offset 4192768 len 8192 2024-03-08T21:14:59.468 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2560 offset 4192768 len 8192 2024-03-08T21:14:59.468 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 2560 offset 4192768 len 16384 2024-03-08T21:14:59.468 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2560 offset 4192768 len 16384 2024-03-08T21:14:59.468 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 2560 offset 4192768 len 16384 2024-03-08T21:14:59.468 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2560 offset 4192768 len 16384 2024-03-08T21:14:59.468 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 2560 offset 4193280 len 1024 2024-03-08T21:14:59.468 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2560 offset 4193280 len 1024 2024-03-08T21:14:59.469 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 2560 offset 4193280 len 1024 2024-03-08T21:14:59.469 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2560 offset 4193280 len 1024 2024-03-08T21:14:59.469 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 2560 offset 4193280 len 2048 2024-03-08T21:14:59.469 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2560 offset 4193280 len 2048 2024-03-08T21:14:59.469 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 2560 offset 4193280 len 2048 2024-03-08T21:14:59.469 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2560 offset 4193280 len 2048 2024-03-08T21:14:59.469 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 2560 offset 4193280 len 4096 2024-03-08T21:14:59.469 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2560 offset 4193280 len 4096 2024-03-08T21:14:59.469 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 2560 offset 4193280 len 4096 2024-03-08T21:14:59.469 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2560 offset 4193280 len 4096 2024-03-08T21:14:59.469 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 2560 offset 4193280 len 8192 2024-03-08T21:14:59.469 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2560 offset 4193280 len 8192 2024-03-08T21:14:59.470 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 2560 offset 4193280 len 8192 2024-03-08T21:14:59.470 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2560 offset 4193280 len 8192 2024-03-08T21:14:59.470 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 2560 offset 4193280 len 16384 2024-03-08T21:14:59.470 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2560 offset 4193280 len 16384 2024-03-08T21:14:59.470 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 2560 offset 4193280 len 16384 2024-03-08T21:14:59.470 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2560 offset 4193280 len 16384 2024-03-08T21:14:59.470 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 2560 offset 4193792 len 1024 2024-03-08T21:14:59.470 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2560 offset 4193792 len 1024 2024-03-08T21:14:59.470 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 2560 offset 4193792 len 1024 2024-03-08T21:14:59.470 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2560 offset 4193792 len 1024 2024-03-08T21:14:59.470 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 2560 offset 4193792 len 2048 2024-03-08T21:14:59.470 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2560 offset 4193792 len 2048 2024-03-08T21:14:59.471 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 2560 offset 4193792 len 2048 2024-03-08T21:14:59.471 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2560 offset 4193792 len 2048 2024-03-08T21:14:59.471 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 2560 offset 4193792 len 4096 2024-03-08T21:14:59.471 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2560 offset 4193792 len 4096 2024-03-08T21:14:59.471 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 2560 offset 4193792 len 4096 2024-03-08T21:14:59.471 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2560 offset 4193792 len 4096 2024-03-08T21:14:59.471 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 2560 offset 4193792 len 8192 2024-03-08T21:14:59.471 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2560 offset 4193792 len 8192 2024-03-08T21:14:59.471 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 2560 offset 4193792 len 8192 2024-03-08T21:14:59.471 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2560 offset 4193792 len 8192 2024-03-08T21:14:59.556 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 2560 offset 4193792 len 16384 2024-03-08T21:14:59.556 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2560 offset 4193792 len 16384 2024-03-08T21:14:59.556 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 2560 offset 4193792 len 16384 2024-03-08T21:14:59.556 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2560 offset 4193792 len 16384 2024-03-08T21:14:59.557 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 2560 offset 4194304 len 1024 2024-03-08T21:14:59.557 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2560 offset 4194304 len 1024 2024-03-08T21:14:59.557 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 2560 offset 4194304 len 1024 2024-03-08T21:14:59.557 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2560 offset 4194304 len 1024 2024-03-08T21:14:59.557 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 2560 offset 4194304 len 2048 2024-03-08T21:14:59.557 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2560 offset 4194304 len 2048 2024-03-08T21:14:59.557 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 2560 offset 4194304 len 2048 2024-03-08T21:14:59.557 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2560 offset 4194304 len 2048 2024-03-08T21:14:59.557 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 2560 offset 4194304 len 4096 2024-03-08T21:14:59.557 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2560 offset 4194304 len 4096 2024-03-08T21:14:59.558 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 2560 offset 4194304 len 4096 2024-03-08T21:14:59.558 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2560 offset 4194304 len 4096 2024-03-08T21:14:59.558 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 2560 offset 4194304 len 8192 2024-03-08T21:14:59.558 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2560 offset 4194304 len 8192 2024-03-08T21:14:59.558 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 2560 offset 4194304 len 8192 2024-03-08T21:14:59.558 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2560 offset 4194304 len 8192 2024-03-08T21:14:59.558 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 2560 offset 4194304 len 16384 2024-03-08T21:14:59.558 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2560 offset 4194304 len 16384 2024-03-08T21:14:59.558 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 2560 offset 4194304 len 16384 2024-03-08T21:14:59.558 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2560 offset 4194304 len 16384 2024-03-08T21:14:59.558 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 2560 offset 4194816 len 1024 2024-03-08T21:14:59.558 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2560 offset 4194816 len 1024 2024-03-08T21:14:59.558 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 2560 offset 4194816 len 1024 2024-03-08T21:14:59.559 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2560 offset 4194816 len 1024 2024-03-08T21:14:59.559 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 2560 offset 4194816 len 2048 2024-03-08T21:14:59.559 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2560 offset 4194816 len 2048 2024-03-08T21:14:59.559 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 2560 offset 4194816 len 2048 2024-03-08T21:14:59.559 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2560 offset 4194816 len 2048 2024-03-08T21:14:59.559 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 2560 offset 4194816 len 4096 2024-03-08T21:14:59.559 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2560 offset 4194816 len 4096 2024-03-08T21:14:59.559 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 2560 offset 4194816 len 4096 2024-03-08T21:14:59.559 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2560 offset 4194816 len 4096 2024-03-08T21:14:59.559 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 2560 offset 4194816 len 8192 2024-03-08T21:14:59.560 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2560 offset 4194816 len 8192 2024-03-08T21:14:59.560 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 2560 offset 4194816 len 8192 2024-03-08T21:14:59.560 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2560 offset 4194816 len 8192 2024-03-08T21:14:59.560 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 2560 offset 4194816 len 16384 2024-03-08T21:14:59.560 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2560 offset 4194816 len 16384 2024-03-08T21:14:59.560 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 2560 offset 4194816 len 16384 2024-03-08T21:14:59.560 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2560 offset 4194816 len 16384 2024-03-08T21:14:59.560 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 2560 offset 4195328 len 1024 2024-03-08T21:14:59.561 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2560 offset 4195328 len 1024 2024-03-08T21:14:59.561 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 2560 offset 4195328 len 1024 2024-03-08T21:14:59.561 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2560 offset 4195328 len 1024 2024-03-08T21:14:59.561 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 2560 offset 4195328 len 2048 2024-03-08T21:14:59.561 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2560 offset 4195328 len 2048 2024-03-08T21:14:59.561 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 2560 offset 4195328 len 2048 2024-03-08T21:14:59.561 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2560 offset 4195328 len 2048 2024-03-08T21:14:59.561 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 2560 offset 4195328 len 4096 2024-03-08T21:14:59.561 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2560 offset 4195328 len 4096 2024-03-08T21:14:59.562 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 2560 offset 4195328 len 4096 2024-03-08T21:14:59.562 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2560 offset 4195328 len 4096 2024-03-08T21:14:59.562 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 2560 offset 4195328 len 8192 2024-03-08T21:14:59.562 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2560 offset 4195328 len 8192 2024-03-08T21:14:59.562 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 2560 offset 4195328 len 8192 2024-03-08T21:14:59.562 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2560 offset 4195328 len 8192 2024-03-08T21:14:59.562 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 2560 offset 4195328 len 16384 2024-03-08T21:14:59.562 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2560 offset 4195328 len 16384 2024-03-08T21:14:59.562 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 2560 offset 4195328 len 16384 2024-03-08T21:14:59.562 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2560 offset 4195328 len 16384 2024-03-08T21:14:59.563 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 2560 offset 4195840 len 1024 2024-03-08T21:14:59.563 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2560 offset 4195840 len 1024 2024-03-08T21:14:59.563 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 2560 offset 4195840 len 1024 2024-03-08T21:14:59.563 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2560 offset 4195840 len 1024 2024-03-08T21:14:59.563 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 2560 offset 4195840 len 2048 2024-03-08T21:14:59.563 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2560 offset 4195840 len 2048 2024-03-08T21:14:59.563 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 2560 offset 4195840 len 2048 2024-03-08T21:14:59.563 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2560 offset 4195840 len 2048 2024-03-08T21:14:59.563 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 2560 offset 4195840 len 4096 2024-03-08T21:14:59.563 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2560 offset 4195840 len 4096 2024-03-08T21:14:59.563 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 2560 offset 4195840 len 4096 2024-03-08T21:14:59.563 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2560 offset 4195840 len 4096 2024-03-08T21:14:59.564 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 2560 offset 4195840 len 8192 2024-03-08T21:14:59.564 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2560 offset 4195840 len 8192 2024-03-08T21:14:59.564 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 2560 offset 4195840 len 8192 2024-03-08T21:14:59.564 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2560 offset 4195840 len 8192 2024-03-08T21:14:59.564 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 2560 offset 4195840 len 16384 2024-03-08T21:14:59.564 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2560 offset 4195840 len 16384 2024-03-08T21:14:59.642 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 2560 offset 4195840 len 16384 2024-03-08T21:14:59.642 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2560 offset 4195840 len 16384 2024-03-08T21:14:59.643 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 2560 offset 4196352 len 1024 2024-03-08T21:14:59.643 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2560 offset 4196352 len 1024 2024-03-08T21:14:59.643 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 2560 offset 4196352 len 1024 2024-03-08T21:14:59.643 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2560 offset 4196352 len 1024 2024-03-08T21:14:59.643 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 2560 offset 4196352 len 2048 2024-03-08T21:14:59.643 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2560 offset 4196352 len 2048 2024-03-08T21:14:59.643 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 2560 offset 4196352 len 2048 2024-03-08T21:14:59.643 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2560 offset 4196352 len 2048 2024-03-08T21:14:59.643 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 2560 offset 4196352 len 4096 2024-03-08T21:14:59.643 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2560 offset 4196352 len 4096 2024-03-08T21:14:59.643 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 2560 offset 4196352 len 4096 2024-03-08T21:14:59.643 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2560 offset 4196352 len 4096 2024-03-08T21:14:59.643 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 2560 offset 4196352 len 8192 2024-03-08T21:14:59.643 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2560 offset 4196352 len 8192 2024-03-08T21:14:59.644 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 2560 offset 4196352 len 8192 2024-03-08T21:14:59.644 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2560 offset 4196352 len 8192 2024-03-08T21:14:59.644 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 2560 offset 4196352 len 16384 2024-03-08T21:14:59.644 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2560 offset 4196352 len 16384 2024-03-08T21:14:59.644 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 2560 offset 4196352 len 16384 2024-03-08T21:14:59.644 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2560 offset 4196352 len 16384 2024-03-08T21:14:59.644 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 2560 offset 4196864 len 1024 2024-03-08T21:14:59.644 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2560 offset 4196864 len 1024 2024-03-08T21:14:59.644 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 2560 offset 4196864 len 1024 2024-03-08T21:14:59.644 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2560 offset 4196864 len 1024 2024-03-08T21:14:59.644 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 2560 offset 4196864 len 2048 2024-03-08T21:14:59.644 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2560 offset 4196864 len 2048 2024-03-08T21:14:59.644 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 2560 offset 4196864 len 2048 2024-03-08T21:14:59.644 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2560 offset 4196864 len 2048 2024-03-08T21:14:59.644 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 2560 offset 4196864 len 4096 2024-03-08T21:14:59.645 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2560 offset 4196864 len 4096 2024-03-08T21:14:59.645 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 2560 offset 4196864 len 4096 2024-03-08T21:14:59.645 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2560 offset 4196864 len 4096 2024-03-08T21:14:59.645 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 2560 offset 4196864 len 8192 2024-03-08T21:14:59.645 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2560 offset 4196864 len 8192 2024-03-08T21:14:59.645 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 2560 offset 4196864 len 8192 2024-03-08T21:14:59.645 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2560 offset 4196864 len 8192 2024-03-08T21:14:59.645 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 2560 offset 4196864 len 16384 2024-03-08T21:14:59.645 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2560 offset 4196864 len 16384 2024-03-08T21:14:59.645 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 2560 offset 4196864 len 16384 2024-03-08T21:14:59.645 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2560 offset 4196864 len 16384 2024-03-08T21:14:59.645 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 2560 offset 4197376 len 1024 2024-03-08T21:14:59.645 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2560 offset 4197376 len 1024 2024-03-08T21:14:59.645 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 2560 offset 4197376 len 1024 2024-03-08T21:14:59.645 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2560 offset 4197376 len 1024 2024-03-08T21:14:59.646 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 2560 offset 4197376 len 2048 2024-03-08T21:14:59.646 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2560 offset 4197376 len 2048 2024-03-08T21:14:59.646 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 2560 offset 4197376 len 2048 2024-03-08T21:14:59.646 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2560 offset 4197376 len 2048 2024-03-08T21:14:59.646 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 2560 offset 4197376 len 4096 2024-03-08T21:14:59.646 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2560 offset 4197376 len 4096 2024-03-08T21:14:59.646 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 2560 offset 4197376 len 4096 2024-03-08T21:14:59.646 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2560 offset 4197376 len 4096 2024-03-08T21:14:59.646 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 2560 offset 4197376 len 8192 2024-03-08T21:14:59.646 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2560 offset 4197376 len 8192 2024-03-08T21:14:59.646 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 2560 offset 4197376 len 8192 2024-03-08T21:14:59.646 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2560 offset 4197376 len 8192 2024-03-08T21:14:59.646 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 2560 offset 4197376 len 16384 2024-03-08T21:14:59.646 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2560 offset 4197376 len 16384 2024-03-08T21:14:59.647 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 2560 offset 4197376 len 16384 2024-03-08T21:14:59.647 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2560 offset 4197376 len 16384 2024-03-08T21:14:59.647 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 2560 offset 4197888 len 1024 2024-03-08T21:14:59.647 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2560 offset 4197888 len 1024 2024-03-08T21:14:59.647 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 2560 offset 4197888 len 1024 2024-03-08T21:14:59.647 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2560 offset 4197888 len 1024 2024-03-08T21:14:59.647 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 2560 offset 4197888 len 2048 2024-03-08T21:14:59.647 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2560 offset 4197888 len 2048 2024-03-08T21:14:59.647 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 2560 offset 4197888 len 2048 2024-03-08T21:14:59.647 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2560 offset 4197888 len 2048 2024-03-08T21:14:59.647 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 2560 offset 4197888 len 4096 2024-03-08T21:14:59.647 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2560 offset 4197888 len 4096 2024-03-08T21:14:59.647 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 2560 offset 4197888 len 4096 2024-03-08T21:14:59.647 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2560 offset 4197888 len 4096 2024-03-08T21:14:59.648 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 2560 offset 4197888 len 8192 2024-03-08T21:14:59.648 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2560 offset 4197888 len 8192 2024-03-08T21:14:59.648 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 2560 offset 4197888 len 8192 2024-03-08T21:14:59.648 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2560 offset 4197888 len 8192 2024-03-08T21:14:59.648 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 2560 offset 4197888 len 16384 2024-03-08T21:14:59.648 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2560 offset 4197888 len 16384 2024-03-08T21:14:59.648 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 2560 offset 4197888 len 16384 2024-03-08T21:14:59.648 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 2560 offset 4197888 len 16384 2024-03-08T21:14:59.735 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 3072 offset 4190208 len 1024 2024-03-08T21:14:59.736 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3072 offset 4190208 len 1024 2024-03-08T21:14:59.736 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 3072 offset 4190208 len 1024 2024-03-08T21:14:59.736 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3072 offset 4190208 len 1024 2024-03-08T21:14:59.736 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 3072 offset 4190208 len 2048 2024-03-08T21:14:59.736 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3072 offset 4190208 len 2048 2024-03-08T21:14:59.736 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 3072 offset 4190208 len 2048 2024-03-08T21:14:59.736 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3072 offset 4190208 len 2048 2024-03-08T21:14:59.736 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 3072 offset 4190208 len 4096 2024-03-08T21:14:59.736 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3072 offset 4190208 len 4096 2024-03-08T21:14:59.736 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 3072 offset 4190208 len 4096 2024-03-08T21:14:59.736 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3072 offset 4190208 len 4096 2024-03-08T21:14:59.737 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 3072 offset 4190208 len 8192 2024-03-08T21:14:59.737 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3072 offset 4190208 len 8192 2024-03-08T21:14:59.737 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 3072 offset 4190208 len 8192 2024-03-08T21:14:59.737 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3072 offset 4190208 len 8192 2024-03-08T21:14:59.737 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 3072 offset 4190208 len 16384 2024-03-08T21:14:59.737 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3072 offset 4190208 len 16384 2024-03-08T21:14:59.737 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 3072 offset 4190208 len 16384 2024-03-08T21:14:59.737 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3072 offset 4190208 len 16384 2024-03-08T21:14:59.737 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 3072 offset 4190720 len 1024 2024-03-08T21:14:59.737 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3072 offset 4190720 len 1024 2024-03-08T21:14:59.737 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 3072 offset 4190720 len 1024 2024-03-08T21:14:59.737 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3072 offset 4190720 len 1024 2024-03-08T21:14:59.737 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 3072 offset 4190720 len 2048 2024-03-08T21:14:59.737 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3072 offset 4190720 len 2048 2024-03-08T21:14:59.738 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 3072 offset 4190720 len 2048 2024-03-08T21:14:59.738 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3072 offset 4190720 len 2048 2024-03-08T21:14:59.738 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 3072 offset 4190720 len 4096 2024-03-08T21:14:59.738 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3072 offset 4190720 len 4096 2024-03-08T21:14:59.738 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 3072 offset 4190720 len 4096 2024-03-08T21:14:59.738 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3072 offset 4190720 len 4096 2024-03-08T21:14:59.738 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 3072 offset 4190720 len 8192 2024-03-08T21:14:59.738 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3072 offset 4190720 len 8192 2024-03-08T21:14:59.738 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 3072 offset 4190720 len 8192 2024-03-08T21:14:59.738 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3072 offset 4190720 len 8192 2024-03-08T21:14:59.738 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 3072 offset 4190720 len 16384 2024-03-08T21:14:59.738 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3072 offset 4190720 len 16384 2024-03-08T21:14:59.738 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 3072 offset 4190720 len 16384 2024-03-08T21:14:59.738 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3072 offset 4190720 len 16384 2024-03-08T21:14:59.739 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 3072 offset 4191232 len 1024 2024-03-08T21:14:59.739 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3072 offset 4191232 len 1024 2024-03-08T21:14:59.739 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 3072 offset 4191232 len 1024 2024-03-08T21:14:59.739 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3072 offset 4191232 len 1024 2024-03-08T21:14:59.739 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 3072 offset 4191232 len 2048 2024-03-08T21:14:59.739 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3072 offset 4191232 len 2048 2024-03-08T21:14:59.739 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 3072 offset 4191232 len 2048 2024-03-08T21:14:59.739 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3072 offset 4191232 len 2048 2024-03-08T21:14:59.739 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 3072 offset 4191232 len 4096 2024-03-08T21:14:59.739 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3072 offset 4191232 len 4096 2024-03-08T21:14:59.739 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 3072 offset 4191232 len 4096 2024-03-08T21:14:59.739 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3072 offset 4191232 len 4096 2024-03-08T21:14:59.739 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 3072 offset 4191232 len 8192 2024-03-08T21:14:59.739 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3072 offset 4191232 len 8192 2024-03-08T21:14:59.740 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 3072 offset 4191232 len 8192 2024-03-08T21:14:59.740 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3072 offset 4191232 len 8192 2024-03-08T21:14:59.740 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 3072 offset 4191232 len 16384 2024-03-08T21:14:59.740 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3072 offset 4191232 len 16384 2024-03-08T21:14:59.740 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 3072 offset 4191232 len 16384 2024-03-08T21:14:59.740 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3072 offset 4191232 len 16384 2024-03-08T21:14:59.740 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 3072 offset 4191744 len 1024 2024-03-08T21:14:59.740 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3072 offset 4191744 len 1024 2024-03-08T21:14:59.740 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 3072 offset 4191744 len 1024 2024-03-08T21:14:59.740 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3072 offset 4191744 len 1024 2024-03-08T21:14:59.740 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 3072 offset 4191744 len 2048 2024-03-08T21:14:59.740 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3072 offset 4191744 len 2048 2024-03-08T21:14:59.740 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 3072 offset 4191744 len 2048 2024-03-08T21:14:59.740 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3072 offset 4191744 len 2048 2024-03-08T21:14:59.740 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 3072 offset 4191744 len 4096 2024-03-08T21:14:59.741 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3072 offset 4191744 len 4096 2024-03-08T21:14:59.741 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 3072 offset 4191744 len 4096 2024-03-08T21:14:59.741 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3072 offset 4191744 len 4096 2024-03-08T21:14:59.741 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 3072 offset 4191744 len 8192 2024-03-08T21:14:59.741 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3072 offset 4191744 len 8192 2024-03-08T21:14:59.741 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 3072 offset 4191744 len 8192 2024-03-08T21:14:59.741 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3072 offset 4191744 len 8192 2024-03-08T21:14:59.741 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 3072 offset 4191744 len 16384 2024-03-08T21:14:59.741 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3072 offset 4191744 len 16384 2024-03-08T21:14:59.741 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 3072 offset 4191744 len 16384 2024-03-08T21:14:59.741 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3072 offset 4191744 len 16384 2024-03-08T21:14:59.741 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 3072 offset 4192256 len 1024 2024-03-08T21:14:59.741 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3072 offset 4192256 len 1024 2024-03-08T21:14:59.742 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 3072 offset 4192256 len 1024 2024-03-08T21:14:59.829 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3072 offset 4192256 len 1024 2024-03-08T21:14:59.829 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 3072 offset 4192256 len 2048 2024-03-08T21:14:59.829 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3072 offset 4192256 len 2048 2024-03-08T21:14:59.829 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 3072 offset 4192256 len 2048 2024-03-08T21:14:59.829 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3072 offset 4192256 len 2048 2024-03-08T21:14:59.830 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 3072 offset 4192256 len 4096 2024-03-08T21:14:59.830 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3072 offset 4192256 len 4096 2024-03-08T21:14:59.830 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 3072 offset 4192256 len 4096 2024-03-08T21:14:59.830 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3072 offset 4192256 len 4096 2024-03-08T21:14:59.830 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 3072 offset 4192256 len 8192 2024-03-08T21:14:59.830 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3072 offset 4192256 len 8192 2024-03-08T21:14:59.830 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 3072 offset 4192256 len 8192 2024-03-08T21:14:59.830 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3072 offset 4192256 len 8192 2024-03-08T21:14:59.830 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 3072 offset 4192256 len 16384 2024-03-08T21:14:59.830 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3072 offset 4192256 len 16384 2024-03-08T21:14:59.830 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 3072 offset 4192256 len 16384 2024-03-08T21:14:59.830 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3072 offset 4192256 len 16384 2024-03-08T21:14:59.830 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 3072 offset 4192768 len 1024 2024-03-08T21:14:59.830 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3072 offset 4192768 len 1024 2024-03-08T21:14:59.831 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 3072 offset 4192768 len 1024 2024-03-08T21:14:59.831 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3072 offset 4192768 len 1024 2024-03-08T21:14:59.831 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 3072 offset 4192768 len 2048 2024-03-08T21:14:59.831 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3072 offset 4192768 len 2048 2024-03-08T21:14:59.831 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 3072 offset 4192768 len 2048 2024-03-08T21:14:59.831 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3072 offset 4192768 len 2048 2024-03-08T21:14:59.831 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 3072 offset 4192768 len 4096 2024-03-08T21:14:59.831 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3072 offset 4192768 len 4096 2024-03-08T21:14:59.831 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 3072 offset 4192768 len 4096 2024-03-08T21:14:59.831 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3072 offset 4192768 len 4096 2024-03-08T21:14:59.831 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 3072 offset 4192768 len 8192 2024-03-08T21:14:59.831 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3072 offset 4192768 len 8192 2024-03-08T21:14:59.831 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 3072 offset 4192768 len 8192 2024-03-08T21:14:59.832 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3072 offset 4192768 len 8192 2024-03-08T21:14:59.832 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 3072 offset 4192768 len 16384 2024-03-08T21:14:59.832 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3072 offset 4192768 len 16384 2024-03-08T21:14:59.832 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 3072 offset 4192768 len 16384 2024-03-08T21:14:59.832 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3072 offset 4192768 len 16384 2024-03-08T21:14:59.832 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 3072 offset 4193280 len 1024 2024-03-08T21:14:59.832 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3072 offset 4193280 len 1024 2024-03-08T21:14:59.832 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 3072 offset 4193280 len 1024 2024-03-08T21:14:59.832 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3072 offset 4193280 len 1024 2024-03-08T21:14:59.832 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 3072 offset 4193280 len 2048 2024-03-08T21:14:59.832 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3072 offset 4193280 len 2048 2024-03-08T21:14:59.832 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 3072 offset 4193280 len 2048 2024-03-08T21:14:59.832 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3072 offset 4193280 len 2048 2024-03-08T21:14:59.832 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 3072 offset 4193280 len 4096 2024-03-08T21:14:59.832 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3072 offset 4193280 len 4096 2024-03-08T21:14:59.833 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 3072 offset 4193280 len 4096 2024-03-08T21:14:59.833 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3072 offset 4193280 len 4096 2024-03-08T21:14:59.833 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 3072 offset 4193280 len 8192 2024-03-08T21:14:59.833 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3072 offset 4193280 len 8192 2024-03-08T21:14:59.833 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 3072 offset 4193280 len 8192 2024-03-08T21:14:59.833 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3072 offset 4193280 len 8192 2024-03-08T21:14:59.833 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 3072 offset 4193280 len 16384 2024-03-08T21:14:59.833 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3072 offset 4193280 len 16384 2024-03-08T21:14:59.833 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 3072 offset 4193280 len 16384 2024-03-08T21:14:59.833 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3072 offset 4193280 len 16384 2024-03-08T21:14:59.833 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 3072 offset 4193792 len 1024 2024-03-08T21:14:59.833 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3072 offset 4193792 len 1024 2024-03-08T21:14:59.833 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 3072 offset 4193792 len 1024 2024-03-08T21:14:59.833 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3072 offset 4193792 len 1024 2024-03-08T21:14:59.833 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 3072 offset 4193792 len 2048 2024-03-08T21:14:59.834 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3072 offset 4193792 len 2048 2024-03-08T21:14:59.834 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 3072 offset 4193792 len 2048 2024-03-08T21:14:59.834 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3072 offset 4193792 len 2048 2024-03-08T21:14:59.834 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 3072 offset 4193792 len 4096 2024-03-08T21:14:59.834 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3072 offset 4193792 len 4096 2024-03-08T21:14:59.834 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 3072 offset 4193792 len 4096 2024-03-08T21:14:59.834 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3072 offset 4193792 len 4096 2024-03-08T21:14:59.834 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 3072 offset 4193792 len 8192 2024-03-08T21:14:59.834 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3072 offset 4193792 len 8192 2024-03-08T21:14:59.834 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 3072 offset 4193792 len 8192 2024-03-08T21:14:59.834 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3072 offset 4193792 len 8192 2024-03-08T21:14:59.834 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 3072 offset 4193792 len 16384 2024-03-08T21:14:59.834 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3072 offset 4193792 len 16384 2024-03-08T21:14:59.834 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 3072 offset 4193792 len 16384 2024-03-08T21:14:59.835 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3072 offset 4193792 len 16384 2024-03-08T21:14:59.835 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 3072 offset 4194304 len 1024 2024-03-08T21:14:59.835 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3072 offset 4194304 len 1024 2024-03-08T21:14:59.835 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 3072 offset 4194304 len 1024 2024-03-08T21:14:59.835 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3072 offset 4194304 len 1024 2024-03-08T21:14:59.835 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 3072 offset 4194304 len 2048 2024-03-08T21:14:59.921 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3072 offset 4194304 len 2048 2024-03-08T21:14:59.922 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 3072 offset 4194304 len 2048 2024-03-08T21:14:59.922 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3072 offset 4194304 len 2048 2024-03-08T21:14:59.922 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 3072 offset 4194304 len 4096 2024-03-08T21:14:59.922 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3072 offset 4194304 len 4096 2024-03-08T21:14:59.922 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 3072 offset 4194304 len 4096 2024-03-08T21:14:59.922 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3072 offset 4194304 len 4096 2024-03-08T21:14:59.922 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 3072 offset 4194304 len 8192 2024-03-08T21:14:59.922 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3072 offset 4194304 len 8192 2024-03-08T21:14:59.922 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 3072 offset 4194304 len 8192 2024-03-08T21:14:59.922 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3072 offset 4194304 len 8192 2024-03-08T21:14:59.923 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 3072 offset 4194304 len 16384 2024-03-08T21:14:59.923 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3072 offset 4194304 len 16384 2024-03-08T21:14:59.923 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 3072 offset 4194304 len 16384 2024-03-08T21:14:59.923 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3072 offset 4194304 len 16384 2024-03-08T21:14:59.923 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 3072 offset 4194816 len 1024 2024-03-08T21:14:59.923 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3072 offset 4194816 len 1024 2024-03-08T21:14:59.923 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 3072 offset 4194816 len 1024 2024-03-08T21:14:59.923 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3072 offset 4194816 len 1024 2024-03-08T21:14:59.923 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 3072 offset 4194816 len 2048 2024-03-08T21:14:59.923 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3072 offset 4194816 len 2048 2024-03-08T21:14:59.923 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 3072 offset 4194816 len 2048 2024-03-08T21:14:59.923 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3072 offset 4194816 len 2048 2024-03-08T21:14:59.923 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 3072 offset 4194816 len 4096 2024-03-08T21:14:59.923 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3072 offset 4194816 len 4096 2024-03-08T21:14:59.923 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 3072 offset 4194816 len 4096 2024-03-08T21:14:59.924 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3072 offset 4194816 len 4096 2024-03-08T21:14:59.924 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 3072 offset 4194816 len 8192 2024-03-08T21:14:59.924 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3072 offset 4194816 len 8192 2024-03-08T21:14:59.924 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 3072 offset 4194816 len 8192 2024-03-08T21:14:59.924 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3072 offset 4194816 len 8192 2024-03-08T21:14:59.924 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 3072 offset 4194816 len 16384 2024-03-08T21:14:59.924 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3072 offset 4194816 len 16384 2024-03-08T21:14:59.924 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 3072 offset 4194816 len 16384 2024-03-08T21:14:59.924 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3072 offset 4194816 len 16384 2024-03-08T21:14:59.924 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 3072 offset 4195328 len 1024 2024-03-08T21:14:59.924 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3072 offset 4195328 len 1024 2024-03-08T21:14:59.924 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 3072 offset 4195328 len 1024 2024-03-08T21:14:59.924 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3072 offset 4195328 len 1024 2024-03-08T21:14:59.924 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 3072 offset 4195328 len 2048 2024-03-08T21:14:59.924 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3072 offset 4195328 len 2048 2024-03-08T21:14:59.925 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 3072 offset 4195328 len 2048 2024-03-08T21:14:59.925 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3072 offset 4195328 len 2048 2024-03-08T21:14:59.925 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 3072 offset 4195328 len 4096 2024-03-08T21:14:59.925 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3072 offset 4195328 len 4096 2024-03-08T21:14:59.925 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 3072 offset 4195328 len 4096 2024-03-08T21:14:59.925 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3072 offset 4195328 len 4096 2024-03-08T21:14:59.925 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 3072 offset 4195328 len 8192 2024-03-08T21:14:59.925 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3072 offset 4195328 len 8192 2024-03-08T21:14:59.925 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 3072 offset 4195328 len 8192 2024-03-08T21:14:59.925 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3072 offset 4195328 len 8192 2024-03-08T21:14:59.925 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 3072 offset 4195328 len 16384 2024-03-08T21:14:59.925 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3072 offset 4195328 len 16384 2024-03-08T21:14:59.925 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 3072 offset 4195328 len 16384 2024-03-08T21:14:59.925 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3072 offset 4195328 len 16384 2024-03-08T21:14:59.925 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 3072 offset 4195840 len 1024 2024-03-08T21:14:59.926 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3072 offset 4195840 len 1024 2024-03-08T21:14:59.926 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 3072 offset 4195840 len 1024 2024-03-08T21:14:59.926 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3072 offset 4195840 len 1024 2024-03-08T21:14:59.926 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 3072 offset 4195840 len 2048 2024-03-08T21:14:59.926 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3072 offset 4195840 len 2048 2024-03-08T21:14:59.926 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 3072 offset 4195840 len 2048 2024-03-08T21:14:59.926 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3072 offset 4195840 len 2048 2024-03-08T21:14:59.926 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 3072 offset 4195840 len 4096 2024-03-08T21:14:59.926 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3072 offset 4195840 len 4096 2024-03-08T21:14:59.926 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 3072 offset 4195840 len 4096 2024-03-08T21:14:59.927 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3072 offset 4195840 len 4096 2024-03-08T21:14:59.927 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 3072 offset 4195840 len 8192 2024-03-08T21:14:59.927 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3072 offset 4195840 len 8192 2024-03-08T21:14:59.927 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 3072 offset 4195840 len 8192 2024-03-08T21:14:59.927 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3072 offset 4195840 len 8192 2024-03-08T21:14:59.927 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 3072 offset 4195840 len 16384 2024-03-08T21:14:59.927 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3072 offset 4195840 len 16384 2024-03-08T21:14:59.927 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 3072 offset 4195840 len 16384 2024-03-08T21:14:59.927 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3072 offset 4195840 len 16384 2024-03-08T21:14:59.927 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 3072 offset 4196352 len 1024 2024-03-08T21:14:59.927 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3072 offset 4196352 len 1024 2024-03-08T21:14:59.928 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 3072 offset 4196352 len 1024 2024-03-08T21:14:59.928 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3072 offset 4196352 len 1024 2024-03-08T21:14:59.928 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 3072 offset 4196352 len 2048 2024-03-08T21:14:59.928 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3072 offset 4196352 len 2048 2024-03-08T21:14:59.928 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 3072 offset 4196352 len 2048 2024-03-08T21:14:59.928 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3072 offset 4196352 len 2048 2024-03-08T21:15:00.015 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 3072 offset 4196352 len 4096 2024-03-08T21:15:00.015 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3072 offset 4196352 len 4096 2024-03-08T21:15:00.015 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 3072 offset 4196352 len 4096 2024-03-08T21:15:00.015 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3072 offset 4196352 len 4096 2024-03-08T21:15:00.016 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 3072 offset 4196352 len 8192 2024-03-08T21:15:00.016 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3072 offset 4196352 len 8192 2024-03-08T21:15:00.016 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 3072 offset 4196352 len 8192 2024-03-08T21:15:00.016 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3072 offset 4196352 len 8192 2024-03-08T21:15:00.016 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 3072 offset 4196352 len 16384 2024-03-08T21:15:00.016 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3072 offset 4196352 len 16384 2024-03-08T21:15:00.016 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 3072 offset 4196352 len 16384 2024-03-08T21:15:00.016 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3072 offset 4196352 len 16384 2024-03-08T21:15:00.016 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 3072 offset 4196864 len 1024 2024-03-08T21:15:00.016 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3072 offset 4196864 len 1024 2024-03-08T21:15:00.016 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 3072 offset 4196864 len 1024 2024-03-08T21:15:00.016 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3072 offset 4196864 len 1024 2024-03-08T21:15:00.017 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 3072 offset 4196864 len 2048 2024-03-08T21:15:00.017 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3072 offset 4196864 len 2048 2024-03-08T21:15:00.017 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 3072 offset 4196864 len 2048 2024-03-08T21:15:00.017 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3072 offset 4196864 len 2048 2024-03-08T21:15:00.017 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 3072 offset 4196864 len 4096 2024-03-08T21:15:00.017 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3072 offset 4196864 len 4096 2024-03-08T21:15:00.017 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 3072 offset 4196864 len 4096 2024-03-08T21:15:00.017 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3072 offset 4196864 len 4096 2024-03-08T21:15:00.017 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 3072 offset 4196864 len 8192 2024-03-08T21:15:00.017 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3072 offset 4196864 len 8192 2024-03-08T21:15:00.017 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 3072 offset 4196864 len 8192 2024-03-08T21:15:00.017 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3072 offset 4196864 len 8192 2024-03-08T21:15:00.017 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 3072 offset 4196864 len 16384 2024-03-08T21:15:00.017 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3072 offset 4196864 len 16384 2024-03-08T21:15:00.017 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 3072 offset 4196864 len 16384 2024-03-08T21:15:00.018 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3072 offset 4196864 len 16384 2024-03-08T21:15:00.018 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 3072 offset 4197376 len 1024 2024-03-08T21:15:00.018 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3072 offset 4197376 len 1024 2024-03-08T21:15:00.018 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 3072 offset 4197376 len 1024 2024-03-08T21:15:00.018 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3072 offset 4197376 len 1024 2024-03-08T21:15:00.018 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 3072 offset 4197376 len 2048 2024-03-08T21:15:00.018 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3072 offset 4197376 len 2048 2024-03-08T21:15:00.018 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 3072 offset 4197376 len 2048 2024-03-08T21:15:00.018 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3072 offset 4197376 len 2048 2024-03-08T21:15:00.019 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 3072 offset 4197376 len 4096 2024-03-08T21:15:00.019 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3072 offset 4197376 len 4096 2024-03-08T21:15:00.019 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 3072 offset 4197376 len 4096 2024-03-08T21:15:00.019 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3072 offset 4197376 len 4096 2024-03-08T21:15:00.019 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 3072 offset 4197376 len 8192 2024-03-08T21:15:00.019 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3072 offset 4197376 len 8192 2024-03-08T21:15:00.019 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 3072 offset 4197376 len 8192 2024-03-08T21:15:00.019 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3072 offset 4197376 len 8192 2024-03-08T21:15:00.019 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 3072 offset 4197376 len 16384 2024-03-08T21:15:00.019 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3072 offset 4197376 len 16384 2024-03-08T21:15:00.019 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 3072 offset 4197376 len 16384 2024-03-08T21:15:00.019 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3072 offset 4197376 len 16384 2024-03-08T21:15:00.019 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 3072 offset 4197888 len 1024 2024-03-08T21:15:00.020 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3072 offset 4197888 len 1024 2024-03-08T21:15:00.020 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 3072 offset 4197888 len 1024 2024-03-08T21:15:00.020 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3072 offset 4197888 len 1024 2024-03-08T21:15:00.020 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 3072 offset 4197888 len 2048 2024-03-08T21:15:00.020 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3072 offset 4197888 len 2048 2024-03-08T21:15:00.020 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 3072 offset 4197888 len 2048 2024-03-08T21:15:00.020 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3072 offset 4197888 len 2048 2024-03-08T21:15:00.020 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 3072 offset 4197888 len 4096 2024-03-08T21:15:00.020 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3072 offset 4197888 len 4096 2024-03-08T21:15:00.021 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 3072 offset 4197888 len 4096 2024-03-08T21:15:00.021 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3072 offset 4197888 len 4096 2024-03-08T21:15:00.021 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 3072 offset 4197888 len 8192 2024-03-08T21:15:00.021 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3072 offset 4197888 len 8192 2024-03-08T21:15:00.021 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 3072 offset 4197888 len 8192 2024-03-08T21:15:00.021 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3072 offset 4197888 len 8192 2024-03-08T21:15:00.021 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 3072 offset 4197888 len 16384 2024-03-08T21:15:00.021 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3072 offset 4197888 len 16384 2024-03-08T21:15:00.021 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 3072 offset 4197888 len 16384 2024-03-08T21:15:00.021 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3072 offset 4197888 len 16384 2024-03-08T21:15:00.021 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 3584 offset 4190208 len 1024 2024-03-08T21:15:00.021 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3584 offset 4190208 len 1024 2024-03-08T21:15:00.021 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 3584 offset 4190208 len 1024 2024-03-08T21:15:00.021 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3584 offset 4190208 len 1024 2024-03-08T21:15:00.022 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 3584 offset 4190208 len 2048 2024-03-08T21:15:00.022 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3584 offset 4190208 len 2048 2024-03-08T21:15:00.022 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 3584 offset 4190208 len 2048 2024-03-08T21:15:00.022 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3584 offset 4190208 len 2048 2024-03-08T21:15:00.022 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 3584 offset 4190208 len 4096 2024-03-08T21:15:00.022 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3584 offset 4190208 len 4096 2024-03-08T21:15:00.116 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 3584 offset 4190208 len 4096 2024-03-08T21:15:00.117 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3584 offset 4190208 len 4096 2024-03-08T21:15:00.117 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 3584 offset 4190208 len 8192 2024-03-08T21:15:00.117 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3584 offset 4190208 len 8192 2024-03-08T21:15:00.117 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 3584 offset 4190208 len 8192 2024-03-08T21:15:00.117 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3584 offset 4190208 len 8192 2024-03-08T21:15:00.117 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 3584 offset 4190208 len 16384 2024-03-08T21:15:00.117 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3584 offset 4190208 len 16384 2024-03-08T21:15:00.117 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 3584 offset 4190208 len 16384 2024-03-08T21:15:00.117 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3584 offset 4190208 len 16384 2024-03-08T21:15:00.117 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 3584 offset 4190720 len 1024 2024-03-08T21:15:00.117 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3584 offset 4190720 len 1024 2024-03-08T21:15:00.117 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 3584 offset 4190720 len 1024 2024-03-08T21:15:00.118 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3584 offset 4190720 len 1024 2024-03-08T21:15:00.118 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 3584 offset 4190720 len 2048 2024-03-08T21:15:00.118 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3584 offset 4190720 len 2048 2024-03-08T21:15:00.118 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 3584 offset 4190720 len 2048 2024-03-08T21:15:00.118 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3584 offset 4190720 len 2048 2024-03-08T21:15:00.118 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 3584 offset 4190720 len 4096 2024-03-08T21:15:00.118 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3584 offset 4190720 len 4096 2024-03-08T21:15:00.118 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 3584 offset 4190720 len 4096 2024-03-08T21:15:00.118 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3584 offset 4190720 len 4096 2024-03-08T21:15:00.118 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 3584 offset 4190720 len 8192 2024-03-08T21:15:00.118 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3584 offset 4190720 len 8192 2024-03-08T21:15:00.118 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 3584 offset 4190720 len 8192 2024-03-08T21:15:00.118 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3584 offset 4190720 len 8192 2024-03-08T21:15:00.119 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 3584 offset 4190720 len 16384 2024-03-08T21:15:00.119 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3584 offset 4190720 len 16384 2024-03-08T21:15:00.119 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 3584 offset 4190720 len 16384 2024-03-08T21:15:00.119 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3584 offset 4190720 len 16384 2024-03-08T21:15:00.119 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 3584 offset 4191232 len 1024 2024-03-08T21:15:00.119 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3584 offset 4191232 len 1024 2024-03-08T21:15:00.119 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 3584 offset 4191232 len 1024 2024-03-08T21:15:00.119 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3584 offset 4191232 len 1024 2024-03-08T21:15:00.119 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 3584 offset 4191232 len 2048 2024-03-08T21:15:00.119 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3584 offset 4191232 len 2048 2024-03-08T21:15:00.119 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 3584 offset 4191232 len 2048 2024-03-08T21:15:00.119 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3584 offset 4191232 len 2048 2024-03-08T21:15:00.119 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 3584 offset 4191232 len 4096 2024-03-08T21:15:00.119 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3584 offset 4191232 len 4096 2024-03-08T21:15:00.120 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 3584 offset 4191232 len 4096 2024-03-08T21:15:00.120 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3584 offset 4191232 len 4096 2024-03-08T21:15:00.120 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 3584 offset 4191232 len 8192 2024-03-08T21:15:00.120 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3584 offset 4191232 len 8192 2024-03-08T21:15:00.120 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 3584 offset 4191232 len 8192 2024-03-08T21:15:00.120 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3584 offset 4191232 len 8192 2024-03-08T21:15:00.120 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 3584 offset 4191232 len 16384 2024-03-08T21:15:00.120 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3584 offset 4191232 len 16384 2024-03-08T21:15:00.120 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 3584 offset 4191232 len 16384 2024-03-08T21:15:00.120 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3584 offset 4191232 len 16384 2024-03-08T21:15:00.120 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 3584 offset 4191744 len 1024 2024-03-08T21:15:00.120 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3584 offset 4191744 len 1024 2024-03-08T21:15:00.120 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 3584 offset 4191744 len 1024 2024-03-08T21:15:00.120 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3584 offset 4191744 len 1024 2024-03-08T21:15:00.121 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 3584 offset 4191744 len 2048 2024-03-08T21:15:00.121 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3584 offset 4191744 len 2048 2024-03-08T21:15:00.121 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 3584 offset 4191744 len 2048 2024-03-08T21:15:00.121 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3584 offset 4191744 len 2048 2024-03-08T21:15:00.121 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 3584 offset 4191744 len 4096 2024-03-08T21:15:00.121 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3584 offset 4191744 len 4096 2024-03-08T21:15:00.121 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 3584 offset 4191744 len 4096 2024-03-08T21:15:00.121 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3584 offset 4191744 len 4096 2024-03-08T21:15:00.121 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 3584 offset 4191744 len 8192 2024-03-08T21:15:00.121 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3584 offset 4191744 len 8192 2024-03-08T21:15:00.121 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 3584 offset 4191744 len 8192 2024-03-08T21:15:00.121 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3584 offset 4191744 len 8192 2024-03-08T21:15:00.121 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 3584 offset 4191744 len 16384 2024-03-08T21:15:00.121 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3584 offset 4191744 len 16384 2024-03-08T21:15:00.121 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 3584 offset 4191744 len 16384 2024-03-08T21:15:00.122 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3584 offset 4191744 len 16384 2024-03-08T21:15:00.122 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 3584 offset 4192256 len 1024 2024-03-08T21:15:00.122 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3584 offset 4192256 len 1024 2024-03-08T21:15:00.122 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 3584 offset 4192256 len 1024 2024-03-08T21:15:00.122 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3584 offset 4192256 len 1024 2024-03-08T21:15:00.122 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 3584 offset 4192256 len 2048 2024-03-08T21:15:00.122 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3584 offset 4192256 len 2048 2024-03-08T21:15:00.122 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 3584 offset 4192256 len 2048 2024-03-08T21:15:00.122 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3584 offset 4192256 len 2048 2024-03-08T21:15:00.122 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 3584 offset 4192256 len 4096 2024-03-08T21:15:00.122 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3584 offset 4192256 len 4096 2024-03-08T21:15:00.122 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 3584 offset 4192256 len 4096 2024-03-08T21:15:00.122 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3584 offset 4192256 len 4096 2024-03-08T21:15:00.122 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 3584 offset 4192256 len 8192 2024-03-08T21:15:00.201 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3584 offset 4192256 len 8192 2024-03-08T21:15:00.201 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 3584 offset 4192256 len 8192 2024-03-08T21:15:00.201 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3584 offset 4192256 len 8192 2024-03-08T21:15:00.201 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 3584 offset 4192256 len 16384 2024-03-08T21:15:00.201 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3584 offset 4192256 len 16384 2024-03-08T21:15:00.201 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 3584 offset 4192256 len 16384 2024-03-08T21:15:00.201 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3584 offset 4192256 len 16384 2024-03-08T21:15:00.202 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 3584 offset 4192768 len 1024 2024-03-08T21:15:00.202 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3584 offset 4192768 len 1024 2024-03-08T21:15:00.202 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 3584 offset 4192768 len 1024 2024-03-08T21:15:00.202 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3584 offset 4192768 len 1024 2024-03-08T21:15:00.202 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 3584 offset 4192768 len 2048 2024-03-08T21:15:00.202 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3584 offset 4192768 len 2048 2024-03-08T21:15:00.202 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 3584 offset 4192768 len 2048 2024-03-08T21:15:00.202 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3584 offset 4192768 len 2048 2024-03-08T21:15:00.202 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 3584 offset 4192768 len 4096 2024-03-08T21:15:00.202 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3584 offset 4192768 len 4096 2024-03-08T21:15:00.203 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 3584 offset 4192768 len 4096 2024-03-08T21:15:00.203 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3584 offset 4192768 len 4096 2024-03-08T21:15:00.203 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 3584 offset 4192768 len 8192 2024-03-08T21:15:00.203 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3584 offset 4192768 len 8192 2024-03-08T21:15:00.203 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 3584 offset 4192768 len 8192 2024-03-08T21:15:00.203 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3584 offset 4192768 len 8192 2024-03-08T21:15:00.203 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 3584 offset 4192768 len 16384 2024-03-08T21:15:00.203 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3584 offset 4192768 len 16384 2024-03-08T21:15:00.203 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 3584 offset 4192768 len 16384 2024-03-08T21:15:00.203 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3584 offset 4192768 len 16384 2024-03-08T21:15:00.203 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 3584 offset 4193280 len 1024 2024-03-08T21:15:00.203 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3584 offset 4193280 len 1024 2024-03-08T21:15:00.203 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 3584 offset 4193280 len 1024 2024-03-08T21:15:00.204 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3584 offset 4193280 len 1024 2024-03-08T21:15:00.204 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 3584 offset 4193280 len 2048 2024-03-08T21:15:00.204 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3584 offset 4193280 len 2048 2024-03-08T21:15:00.204 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 3584 offset 4193280 len 2048 2024-03-08T21:15:00.204 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3584 offset 4193280 len 2048 2024-03-08T21:15:00.204 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 3584 offset 4193280 len 4096 2024-03-08T21:15:00.204 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3584 offset 4193280 len 4096 2024-03-08T21:15:00.204 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 3584 offset 4193280 len 4096 2024-03-08T21:15:00.204 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3584 offset 4193280 len 4096 2024-03-08T21:15:00.204 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 3584 offset 4193280 len 8192 2024-03-08T21:15:00.204 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3584 offset 4193280 len 8192 2024-03-08T21:15:00.205 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 3584 offset 4193280 len 8192 2024-03-08T21:15:00.205 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3584 offset 4193280 len 8192 2024-03-08T21:15:00.205 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 3584 offset 4193280 len 16384 2024-03-08T21:15:00.205 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3584 offset 4193280 len 16384 2024-03-08T21:15:00.205 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 3584 offset 4193280 len 16384 2024-03-08T21:15:00.205 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3584 offset 4193280 len 16384 2024-03-08T21:15:00.205 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 3584 offset 4193792 len 1024 2024-03-08T21:15:00.205 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3584 offset 4193792 len 1024 2024-03-08T21:15:00.205 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 3584 offset 4193792 len 1024 2024-03-08T21:15:00.205 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3584 offset 4193792 len 1024 2024-03-08T21:15:00.205 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 3584 offset 4193792 len 2048 2024-03-08T21:15:00.205 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3584 offset 4193792 len 2048 2024-03-08T21:15:00.205 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 3584 offset 4193792 len 2048 2024-03-08T21:15:00.206 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3584 offset 4193792 len 2048 2024-03-08T21:15:00.206 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 3584 offset 4193792 len 4096 2024-03-08T21:15:00.206 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3584 offset 4193792 len 4096 2024-03-08T21:15:00.206 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 3584 offset 4193792 len 4096 2024-03-08T21:15:00.206 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3584 offset 4193792 len 4096 2024-03-08T21:15:00.206 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 3584 offset 4193792 len 8192 2024-03-08T21:15:00.206 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3584 offset 4193792 len 8192 2024-03-08T21:15:00.206 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 3584 offset 4193792 len 8192 2024-03-08T21:15:00.206 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3584 offset 4193792 len 8192 2024-03-08T21:15:00.206 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 3584 offset 4193792 len 16384 2024-03-08T21:15:00.206 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3584 offset 4193792 len 16384 2024-03-08T21:15:00.207 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 3584 offset 4193792 len 16384 2024-03-08T21:15:00.207 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3584 offset 4193792 len 16384 2024-03-08T21:15:00.207 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 3584 offset 4194304 len 1024 2024-03-08T21:15:00.207 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3584 offset 4194304 len 1024 2024-03-08T21:15:00.207 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 3584 offset 4194304 len 1024 2024-03-08T21:15:00.207 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3584 offset 4194304 len 1024 2024-03-08T21:15:00.207 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 3584 offset 4194304 len 2048 2024-03-08T21:15:00.207 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3584 offset 4194304 len 2048 2024-03-08T21:15:00.207 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 3584 offset 4194304 len 2048 2024-03-08T21:15:00.207 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3584 offset 4194304 len 2048 2024-03-08T21:15:00.207 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 3584 offset 4194304 len 4096 2024-03-08T21:15:00.207 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3584 offset 4194304 len 4096 2024-03-08T21:15:00.207 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 3584 offset 4194304 len 4096 2024-03-08T21:15:00.208 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3584 offset 4194304 len 4096 2024-03-08T21:15:00.208 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 3584 offset 4194304 len 8192 2024-03-08T21:15:00.208 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3584 offset 4194304 len 8192 2024-03-08T21:15:00.208 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 3584 offset 4194304 len 8192 2024-03-08T21:15:00.263 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3584 offset 4194304 len 8192 2024-03-08T21:15:00.263 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 3584 offset 4194304 len 16384 2024-03-08T21:15:00.264 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3584 offset 4194304 len 16384 2024-03-08T21:15:00.264 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 3584 offset 4194304 len 16384 2024-03-08T21:15:00.264 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3584 offset 4194304 len 16384 2024-03-08T21:15:00.264 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 3584 offset 4194816 len 1024 2024-03-08T21:15:00.264 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3584 offset 4194816 len 1024 2024-03-08T21:15:00.264 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 3584 offset 4194816 len 1024 2024-03-08T21:15:00.264 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3584 offset 4194816 len 1024 2024-03-08T21:15:00.264 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 3584 offset 4194816 len 2048 2024-03-08T21:15:00.264 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3584 offset 4194816 len 2048 2024-03-08T21:15:00.264 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 3584 offset 4194816 len 2048 2024-03-08T21:15:00.264 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3584 offset 4194816 len 2048 2024-03-08T21:15:00.264 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 3584 offset 4194816 len 4096 2024-03-08T21:15:00.264 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3584 offset 4194816 len 4096 2024-03-08T21:15:00.264 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 3584 offset 4194816 len 4096 2024-03-08T21:15:00.265 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3584 offset 4194816 len 4096 2024-03-08T21:15:00.265 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 3584 offset 4194816 len 8192 2024-03-08T21:15:00.265 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3584 offset 4194816 len 8192 2024-03-08T21:15:00.265 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 3584 offset 4194816 len 8192 2024-03-08T21:15:00.265 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3584 offset 4194816 len 8192 2024-03-08T21:15:00.265 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 3584 offset 4194816 len 16384 2024-03-08T21:15:00.265 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3584 offset 4194816 len 16384 2024-03-08T21:15:00.265 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 3584 offset 4194816 len 16384 2024-03-08T21:15:00.265 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3584 offset 4194816 len 16384 2024-03-08T21:15:00.265 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 3584 offset 4195328 len 1024 2024-03-08T21:15:00.265 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3584 offset 4195328 len 1024 2024-03-08T21:15:00.265 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 3584 offset 4195328 len 1024 2024-03-08T21:15:00.265 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3584 offset 4195328 len 1024 2024-03-08T21:15:00.266 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 3584 offset 4195328 len 2048 2024-03-08T21:15:00.266 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3584 offset 4195328 len 2048 2024-03-08T21:15:00.266 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 3584 offset 4195328 len 2048 2024-03-08T21:15:00.266 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3584 offset 4195328 len 2048 2024-03-08T21:15:00.266 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 3584 offset 4195328 len 4096 2024-03-08T21:15:00.266 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3584 offset 4195328 len 4096 2024-03-08T21:15:00.266 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 3584 offset 4195328 len 4096 2024-03-08T21:15:00.266 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3584 offset 4195328 len 4096 2024-03-08T21:15:00.266 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 3584 offset 4195328 len 8192 2024-03-08T21:15:00.266 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3584 offset 4195328 len 8192 2024-03-08T21:15:00.266 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 3584 offset 4195328 len 8192 2024-03-08T21:15:00.266 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3584 offset 4195328 len 8192 2024-03-08T21:15:00.266 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 3584 offset 4195328 len 16384 2024-03-08T21:15:00.266 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3584 offset 4195328 len 16384 2024-03-08T21:15:00.267 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 3584 offset 4195328 len 16384 2024-03-08T21:15:00.267 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3584 offset 4195328 len 16384 2024-03-08T21:15:00.267 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 3584 offset 4195840 len 1024 2024-03-08T21:15:00.267 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3584 offset 4195840 len 1024 2024-03-08T21:15:00.267 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 3584 offset 4195840 len 1024 2024-03-08T21:15:00.267 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3584 offset 4195840 len 1024 2024-03-08T21:15:00.267 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 3584 offset 4195840 len 2048 2024-03-08T21:15:00.267 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3584 offset 4195840 len 2048 2024-03-08T21:15:00.267 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 3584 offset 4195840 len 2048 2024-03-08T21:15:00.267 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3584 offset 4195840 len 2048 2024-03-08T21:15:00.267 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 3584 offset 4195840 len 4096 2024-03-08T21:15:00.267 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3584 offset 4195840 len 4096 2024-03-08T21:15:00.267 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 3584 offset 4195840 len 4096 2024-03-08T21:15:00.267 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3584 offset 4195840 len 4096 2024-03-08T21:15:00.267 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 3584 offset 4195840 len 8192 2024-03-08T21:15:00.268 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3584 offset 4195840 len 8192 2024-03-08T21:15:00.268 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 3584 offset 4195840 len 8192 2024-03-08T21:15:00.268 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3584 offset 4195840 len 8192 2024-03-08T21:15:00.268 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 3584 offset 4195840 len 16384 2024-03-08T21:15:00.268 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3584 offset 4195840 len 16384 2024-03-08T21:15:00.268 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 3584 offset 4195840 len 16384 2024-03-08T21:15:00.268 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3584 offset 4195840 len 16384 2024-03-08T21:15:00.268 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 3584 offset 4196352 len 1024 2024-03-08T21:15:00.268 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3584 offset 4196352 len 1024 2024-03-08T21:15:00.268 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 3584 offset 4196352 len 1024 2024-03-08T21:15:00.268 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3584 offset 4196352 len 1024 2024-03-08T21:15:00.268 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 3584 offset 4196352 len 2048 2024-03-08T21:15:00.268 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3584 offset 4196352 len 2048 2024-03-08T21:15:00.268 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 3584 offset 4196352 len 2048 2024-03-08T21:15:00.268 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3584 offset 4196352 len 2048 2024-03-08T21:15:00.269 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 3584 offset 4196352 len 4096 2024-03-08T21:15:00.269 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3584 offset 4196352 len 4096 2024-03-08T21:15:00.269 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 3584 offset 4196352 len 4096 2024-03-08T21:15:00.269 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3584 offset 4196352 len 4096 2024-03-08T21:15:00.269 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 3584 offset 4196352 len 8192 2024-03-08T21:15:00.269 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3584 offset 4196352 len 8192 2024-03-08T21:15:00.269 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 3584 offset 4196352 len 8192 2024-03-08T21:15:00.269 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3584 offset 4196352 len 8192 2024-03-08T21:15:00.269 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 3584 offset 4196352 len 16384 2024-03-08T21:15:00.618 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3584 offset 4196352 len 16384 2024-03-08T21:15:00.618 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 3584 offset 4196352 len 16384 2024-03-08T21:15:00.618 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3584 offset 4196352 len 16384 2024-03-08T21:15:00.618 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 3584 offset 4196864 len 1024 2024-03-08T21:15:00.618 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3584 offset 4196864 len 1024 2024-03-08T21:15:00.618 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 3584 offset 4196864 len 1024 2024-03-08T21:15:00.618 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3584 offset 4196864 len 1024 2024-03-08T21:15:00.619 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 3584 offset 4196864 len 2048 2024-03-08T21:15:00.619 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3584 offset 4196864 len 2048 2024-03-08T21:15:00.619 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 3584 offset 4196864 len 2048 2024-03-08T21:15:00.619 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3584 offset 4196864 len 2048 2024-03-08T21:15:00.619 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 3584 offset 4196864 len 4096 2024-03-08T21:15:00.619 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3584 offset 4196864 len 4096 2024-03-08T21:15:00.619 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 3584 offset 4196864 len 4096 2024-03-08T21:15:00.619 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3584 offset 4196864 len 4096 2024-03-08T21:15:00.619 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 3584 offset 4196864 len 8192 2024-03-08T21:15:00.619 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3584 offset 4196864 len 8192 2024-03-08T21:15:00.619 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 3584 offset 4196864 len 8192 2024-03-08T21:15:00.619 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3584 offset 4196864 len 8192 2024-03-08T21:15:00.619 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 3584 offset 4196864 len 16384 2024-03-08T21:15:00.619 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3584 offset 4196864 len 16384 2024-03-08T21:15:00.620 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 3584 offset 4196864 len 16384 2024-03-08T21:15:00.620 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3584 offset 4196864 len 16384 2024-03-08T21:15:00.620 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 3584 offset 4197376 len 1024 2024-03-08T21:15:00.620 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3584 offset 4197376 len 1024 2024-03-08T21:15:00.620 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 3584 offset 4197376 len 1024 2024-03-08T21:15:00.620 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3584 offset 4197376 len 1024 2024-03-08T21:15:00.620 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 3584 offset 4197376 len 2048 2024-03-08T21:15:00.620 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3584 offset 4197376 len 2048 2024-03-08T21:15:00.620 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 3584 offset 4197376 len 2048 2024-03-08T21:15:00.620 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3584 offset 4197376 len 2048 2024-03-08T21:15:00.620 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 3584 offset 4197376 len 4096 2024-03-08T21:15:00.620 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3584 offset 4197376 len 4096 2024-03-08T21:15:00.620 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 3584 offset 4197376 len 4096 2024-03-08T21:15:00.620 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3584 offset 4197376 len 4096 2024-03-08T21:15:00.621 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 3584 offset 4197376 len 8192 2024-03-08T21:15:00.621 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3584 offset 4197376 len 8192 2024-03-08T21:15:00.621 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 3584 offset 4197376 len 8192 2024-03-08T21:15:00.621 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3584 offset 4197376 len 8192 2024-03-08T21:15:00.621 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 3584 offset 4197376 len 16384 2024-03-08T21:15:00.621 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3584 offset 4197376 len 16384 2024-03-08T21:15:00.621 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 3584 offset 4197376 len 16384 2024-03-08T21:15:00.621 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3584 offset 4197376 len 16384 2024-03-08T21:15:00.621 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 3584 offset 4197888 len 1024 2024-03-08T21:15:00.621 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3584 offset 4197888 len 1024 2024-03-08T21:15:00.621 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 3584 offset 4197888 len 1024 2024-03-08T21:15:00.621 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3584 offset 4197888 len 1024 2024-03-08T21:15:00.621 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 3584 offset 4197888 len 2048 2024-03-08T21:15:00.621 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3584 offset 4197888 len 2048 2024-03-08T21:15:00.621 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 3584 offset 4197888 len 2048 2024-03-08T21:15:00.622 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3584 offset 4197888 len 2048 2024-03-08T21:15:00.622 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 3584 offset 4197888 len 4096 2024-03-08T21:15:00.622 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3584 offset 4197888 len 4096 2024-03-08T21:15:00.622 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 3584 offset 4197888 len 4096 2024-03-08T21:15:00.622 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3584 offset 4197888 len 4096 2024-03-08T21:15:00.622 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 3584 offset 4197888 len 8192 2024-03-08T21:15:00.622 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3584 offset 4197888 len 8192 2024-03-08T21:15:00.622 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 3584 offset 4197888 len 8192 2024-03-08T21:15:00.622 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3584 offset 4197888 len 8192 2024-03-08T21:15:00.622 INFO:tasks.workunit.client.0.smithi107.stdout:read_direct buf_align 3584 offset 4197888 len 16384 2024-03-08T21:15:00.622 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3584 offset 4197888 len 16384 2024-03-08T21:15:00.622 INFO:tasks.workunit.client.0.smithi107.stdout:read_sync buf_align 3584 offset 4197888 len 16384 2024-03-08T21:15:00.622 INFO:tasks.workunit.client.0.smithi107.stdout:read_file buf_align 3584 offset 4197888 len 16384 2024-03-08T21:15:00.622 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 0 offset 4190720 len 1024 2024-03-08T21:15:00.622 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 0 offset 4190720 len 1024 2024-03-08T21:15:00.623 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 0 offset 4190720 len 1024 2024-03-08T21:15:00.623 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 0 offset 4190720 len 1024 2024-03-08T21:15:00.623 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 0 offset 4190720 len 2048 2024-03-08T21:15:00.623 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 0 offset 4190720 len 2048 2024-03-08T21:15:00.623 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 0 offset 4190720 len 2048 2024-03-08T21:15:00.623 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 0 offset 4190720 len 2048 2024-03-08T21:15:00.623 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 0 offset 4190720 len 4096 2024-03-08T21:15:00.623 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 0 offset 4190720 len 4096 2024-03-08T21:15:00.623 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 0 offset 4190720 len 4096 2024-03-08T21:15:00.623 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 0 offset 4190720 len 4096 2024-03-08T21:15:00.623 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 0 offset 4190720 len 8192 2024-03-08T21:15:00.623 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 0 offset 4190720 len 8192 2024-03-08T21:15:00.623 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 0 offset 4190720 len 8192 2024-03-08T21:15:00.623 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 0 offset 4190720 len 8192 2024-03-08T21:15:00.623 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 0 offset 4190720 len 16384 2024-03-08T21:15:00.624 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 0 offset 4190720 len 16384 2024-03-08T21:15:00.624 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 0 offset 4190720 len 16384 2024-03-08T21:15:00.624 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 0 offset 4190720 len 16384 2024-03-08T21:15:00.624 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 0 offset 4191232 len 1024 2024-03-08T21:15:00.713 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:15:00 smithi107 ceph-mon[31381]: pgmap v458: 97 pgs: 97 active+clean; 10 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 53 KiB/s wr, 41 op/s 2024-03-08T21:15:00.758 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:15:00 smithi149 ceph-mon[35042]: pgmap v458: 97 pgs: 97 active+clean; 10 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 53 KiB/s wr, 41 op/s 2024-03-08T21:15:00.872 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:15:00 smithi204 ceph-mon[34935]: pgmap v458: 97 pgs: 97 active+clean; 10 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 53 KiB/s wr, 41 op/s 2024-03-08T21:15:01.786 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 0 offset 4191232 len 1024 2024-03-08T21:15:01.786 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 0 offset 4191232 len 1024 2024-03-08T21:15:01.786 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 0 offset 4191232 len 1024 2024-03-08T21:15:01.786 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 0 offset 4191232 len 2048 2024-03-08T21:15:01.787 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 0 offset 4191232 len 2048 2024-03-08T21:15:01.787 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 0 offset 4191232 len 2048 2024-03-08T21:15:01.787 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 0 offset 4191232 len 2048 2024-03-08T21:15:01.787 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 0 offset 4191232 len 4096 2024-03-08T21:15:01.787 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 0 offset 4191232 len 4096 2024-03-08T21:15:01.787 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 0 offset 4191232 len 4096 2024-03-08T21:15:01.787 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 0 offset 4191232 len 4096 2024-03-08T21:15:01.787 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 0 offset 4191232 len 8192 2024-03-08T21:15:01.787 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 0 offset 4191232 len 8192 2024-03-08T21:15:01.787 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 0 offset 4191232 len 8192 2024-03-08T21:15:01.787 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 0 offset 4191232 len 8192 2024-03-08T21:15:01.787 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 0 offset 4191232 len 16384 2024-03-08T21:15:01.787 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 0 offset 4191232 len 16384 2024-03-08T21:15:01.787 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 0 offset 4191232 len 16384 2024-03-08T21:15:01.788 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 0 offset 4191232 len 16384 2024-03-08T21:15:01.788 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 0 offset 4191744 len 1024 2024-03-08T21:15:01.788 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 0 offset 4191744 len 1024 2024-03-08T21:15:01.788 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 0 offset 4191744 len 1024 2024-03-08T21:15:01.788 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 0 offset 4191744 len 1024 2024-03-08T21:15:01.788 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 0 offset 4191744 len 2048 2024-03-08T21:15:01.788 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 0 offset 4191744 len 2048 2024-03-08T21:15:01.788 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 0 offset 4191744 len 2048 2024-03-08T21:15:01.788 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 0 offset 4191744 len 2048 2024-03-08T21:15:01.788 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 0 offset 4191744 len 4096 2024-03-08T21:15:01.788 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 0 offset 4191744 len 4096 2024-03-08T21:15:01.788 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 0 offset 4191744 len 4096 2024-03-08T21:15:01.789 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 0 offset 4191744 len 4096 2024-03-08T21:15:01.789 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 0 offset 4191744 len 8192 2024-03-08T21:15:01.789 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 0 offset 4191744 len 8192 2024-03-08T21:15:01.789 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 0 offset 4191744 len 8192 2024-03-08T21:15:01.789 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 0 offset 4191744 len 8192 2024-03-08T21:15:01.789 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 0 offset 4191744 len 16384 2024-03-08T21:15:01.789 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 0 offset 4191744 len 16384 2024-03-08T21:15:01.789 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 0 offset 4191744 len 16384 2024-03-08T21:15:01.789 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 0 offset 4191744 len 16384 2024-03-08T21:15:01.789 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 0 offset 4192256 len 1024 2024-03-08T21:15:01.789 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 0 offset 4192256 len 1024 2024-03-08T21:15:01.789 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 0 offset 4192256 len 1024 2024-03-08T21:15:01.789 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 0 offset 4192256 len 1024 2024-03-08T21:15:01.789 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 0 offset 4192256 len 2048 2024-03-08T21:15:01.789 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 0 offset 4192256 len 2048 2024-03-08T21:15:01.790 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 0 offset 4192256 len 2048 2024-03-08T21:15:01.790 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 0 offset 4192256 len 2048 2024-03-08T21:15:01.790 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 0 offset 4192256 len 4096 2024-03-08T21:15:01.790 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 0 offset 4192256 len 4096 2024-03-08T21:15:01.790 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 0 offset 4192256 len 4096 2024-03-08T21:15:01.790 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 0 offset 4192256 len 4096 2024-03-08T21:15:01.790 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 0 offset 4192256 len 8192 2024-03-08T21:15:01.790 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 0 offset 4192256 len 8192 2024-03-08T21:15:01.790 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 0 offset 4192256 len 8192 2024-03-08T21:15:01.790 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 0 offset 4192256 len 8192 2024-03-08T21:15:01.790 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 0 offset 4192256 len 16384 2024-03-08T21:15:01.791 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 0 offset 4192256 len 16384 2024-03-08T21:15:01.791 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 0 offset 4192256 len 16384 2024-03-08T21:15:01.791 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 0 offset 4192256 len 16384 2024-03-08T21:15:01.791 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 0 offset 4192768 len 1024 2024-03-08T21:15:01.791 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 0 offset 4192768 len 1024 2024-03-08T21:15:01.791 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 0 offset 4192768 len 1024 2024-03-08T21:15:01.791 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 0 offset 4192768 len 1024 2024-03-08T21:15:01.791 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 0 offset 4192768 len 2048 2024-03-08T21:15:01.791 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 0 offset 4192768 len 2048 2024-03-08T21:15:01.791 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 0 offset 4192768 len 2048 2024-03-08T21:15:01.791 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 0 offset 4192768 len 2048 2024-03-08T21:15:01.791 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 0 offset 4192768 len 4096 2024-03-08T21:15:01.792 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 0 offset 4192768 len 4096 2024-03-08T21:15:01.792 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 0 offset 4192768 len 4096 2024-03-08T21:15:01.792 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 0 offset 4192768 len 4096 2024-03-08T21:15:01.792 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 0 offset 4192768 len 8192 2024-03-08T21:15:01.792 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 0 offset 4192768 len 8192 2024-03-08T21:15:01.792 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 0 offset 4192768 len 8192 2024-03-08T21:15:01.792 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 0 offset 4192768 len 8192 2024-03-08T21:15:01.792 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 0 offset 4192768 len 16384 2024-03-08T21:15:01.792 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 0 offset 4192768 len 16384 2024-03-08T21:15:01.792 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 0 offset 4192768 len 16384 2024-03-08T21:15:01.792 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 0 offset 4192768 len 16384 2024-03-08T21:15:01.792 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 0 offset 4193280 len 1024 2024-03-08T21:15:01.792 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 0 offset 4193280 len 1024 2024-03-08T21:15:01.792 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 0 offset 4193280 len 1024 2024-03-08T21:15:01.793 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 0 offset 4193280 len 1024 2024-03-08T21:15:01.793 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 0 offset 4193280 len 2048 2024-03-08T21:15:01.793 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 0 offset 4193280 len 2048 2024-03-08T21:15:01.793 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 0 offset 4193280 len 2048 2024-03-08T21:15:02.711 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:15:02 smithi107 ceph-mon[31381]: pgmap v459: 97 pgs: 97 active+clean; 10 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 53 KiB/s wr, 37 op/s 2024-03-08T21:15:02.762 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:15:02 smithi149 ceph-mon[35042]: pgmap v459: 97 pgs: 97 active+clean; 10 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 53 KiB/s wr, 37 op/s 2024-03-08T21:15:02.872 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:15:02 smithi204 ceph-mon[34935]: pgmap v459: 97 pgs: 97 active+clean; 10 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 53 KiB/s wr, 37 op/s 2024-03-08T21:15:03.031 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.054+0000 7fcc0da00640 1 -- 172.21.15.107:0/1227532924 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcc08108640 msgr2=0x7fcc08110210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:15:03.031 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.054+0000 7fcc0da00640 1 --2- 172.21.15.107:0/1227532924 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcc08108640 0x7fcc08110210 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7fcc00009ca0 tx=0x7fcc0002c0f0 comp rx=0 tx=0).stop 2024-03-08T21:15:03.031 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.054+0000 7fcc0da00640 1 -- 172.21.15.107:0/1227532924 shutdown_connections 2024-03-08T21:15:03.031 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.054+0000 7fcc0da00640 1 --2- 172.21.15.107:0/1227532924 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcc08108640 0x7fcc08110210 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:15:03.031 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.054+0000 7fcc0da00640 1 --2- 172.21.15.107:0/1227532924 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcc08103f20 0x7fcc08107f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:15:03.031 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.054+0000 7fcc0da00640 1 --2- 172.21.15.107:0/1227532924 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcc08103570 0x7fcc08103950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:15:03.031 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.054+0000 7fcc0da00640 1 -- 172.21.15.107:0/1227532924 >> 172.21.15.107:0/1227532924 conn(0x7fcc080fedd0 msgr2=0x7fcc081011f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:15:03.031 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.054+0000 7fcc0da00640 1 -- 172.21.15.107:0/1227532924 shutdown_connections 2024-03-08T21:15:03.031 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.054+0000 7fcc0da00640 1 -- 172.21.15.107:0/1227532924 wait complete. 2024-03-08T21:15:03.034 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.055+0000 7fcc0da00640 1 Processor -- start 2024-03-08T21:15:03.034 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.056+0000 7fcc0da00640 1 -- start start 2024-03-08T21:15:03.034 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.056+0000 7fcc0da00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcc08103570 0x7fcc0819f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:15:03.034 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.056+0000 7fcc0da00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcc08108640 0x7fcc0819fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:15:03.034 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.056+0000 7fcc0da00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcc08199930 0x7fcc08199de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:15:03.034 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.056+0000 7fcc0da00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc08113590 con 0x7fcc08199930 2024-03-08T21:15:03.034 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.056+0000 7fcc0da00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc08113410 con 0x7fcc08108640 2024-03-08T21:15:03.034 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.056+0000 7fcc0da00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc08113710 con 0x7fcc08103570 2024-03-08T21:15:03.034 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.056+0000 7fcc0d000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcc08199930 0x7fcc08199de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:15:03.034 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.056+0000 7fcc0d000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcc08199930 0x7fcc08199de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:54628/0 (socket says 172.21.15.107:54628) 2024-03-08T21:15:03.034 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.056+0000 7fcc0d000640 1 -- 172.21.15.107:0/70481697 learned_addr learned my addr 172.21.15.107:0/70481697 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:15:03.035 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.056+0000 7fcc07e00640 1 --2- 172.21.15.107:0/70481697 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcc08103570 0x7fcc0819f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:15:03.035 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.056+0000 7fcbff400640 1 --2- 172.21.15.107:0/70481697 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcc08108640 0x7fcc0819fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:15:03.035 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.057+0000 7fcc0d000640 1 -- 172.21.15.107:0/70481697 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcc08103570 msgr2=0x7fcc0819f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:15:03.035 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.057+0000 7fcc0d000640 1 --2- 172.21.15.107:0/70481697 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcc08103570 0x7fcc0819f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:15:03.035 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.057+0000 7fcc0d000640 1 -- 172.21.15.107:0/70481697 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcc08108640 msgr2=0x7fcc0819fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:15:03.035 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.057+0000 7fcc0d000640 1 --2- 172.21.15.107:0/70481697 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcc08108640 0x7fcc0819fc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:15:03.035 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.057+0000 7fcc0d000640 1 -- 172.21.15.107:0/70481697 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc0819a450 con 0x7fcc08199930 2024-03-08T21:15:03.035 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.057+0000 7fcc07e00640 1 --2- 172.21.15.107:0/70481697 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcc08103570 0x7fcc0819f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T21:15:03.035 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.057+0000 7fcc0d000640 1 --2- 172.21.15.107:0/70481697 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcc08199930 0x7fcc08199de0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7fcc0002c700 tx=0x7fcc00035d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:15:03.035 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.057+0000 7fcc05600640 1 -- 172.21.15.107:0/70481697 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc00039040 con 0x7fcc08199930 2024-03-08T21:15:03.035 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.057+0000 7fcc0da00640 1 -- 172.21.15.107:0/70481697 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc0819a6e0 con 0x7fcc08199930 2024-03-08T21:15:03.035 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.057+0000 7fcc05600640 1 -- 172.21.15.107:0/70481697 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fcc00044070 con 0x7fcc08199930 2024-03-08T21:15:03.036 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.057+0000 7fcc05600640 1 -- 172.21.15.107:0/70481697 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc00004330 con 0x7fcc08199930 2024-03-08T21:15:03.036 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.057+0000 7fcc0da00640 1 -- 172.21.15.107:0/70481697 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc081a6890 con 0x7fcc08199930 2024-03-08T21:15:03.037 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.061+0000 7fcc11d6a740 1 Processor -- start 2024-03-08T21:15:03.038 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.062+0000 7fcc05600640 1 -- 172.21.15.107:0/70481697 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) v1 ==== 137422+0+0 (secure 0 0 0) 0x7fcc000044d0 con 0x7fcc08199930 2024-03-08T21:15:03.040 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.062+0000 7fcc11d6a740 1 -- start start 2024-03-08T21:15:03.040 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.062+0000 7fcc11d6a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fdca6f9e80 0x55fdca6fa260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:15:03.040 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.062+0000 7fcc11d6a740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fdca6fa7a0 0x55fdca703030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:15:03.040 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.062+0000 7fcc11d6a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fdca703570 0x55fdca705960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:15:03.040 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.062+0000 7fcc11d6a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fdca65d3d0 con 0x55fdca703570 2024-03-08T21:15:03.040 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.062+0000 7fcc11d6a740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fdca6066b0 con 0x55fdca6fa7a0 2024-03-08T21:15:03.040 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.062+0000 7fcc11d6a740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fdca659470 con 0x55fdca6f9e80 2024-03-08T21:15:03.040 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.063+0000 7fcbff400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fdca703570 0x55fdca705960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:15:03.040 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.063+0000 7fcbff400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fdca703570 0x55fdca705960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:54640/0 (socket says 172.21.15.107:54640) 2024-03-08T21:15:03.041 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.063+0000 7fcbff400640 1 -- 172.21.15.107:0/4245563064 learned_addr learned my addr 172.21.15.107:0/4245563064 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:15:03.041 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.063+0000 7fcc07e00640 1 --2- 172.21.15.107:0/4245563064 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fdca6fa7a0 0x55fdca703030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:15:03.041 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.063+0000 7fcbff400640 1 --2- 172.21.15.107:0/4245563064 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fdca6f9e80 0x55fdca6fa260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:15:03.041 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.063+0000 7fcbff400640 1 -- 172.21.15.107:0/4245563064 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fdca6f9e80 msgr2=0x55fdca6fa260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:15:03.041 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.063+0000 7fcbff400640 1 --2- 172.21.15.107:0/4245563064 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fdca6f9e80 0x55fdca6fa260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:15:03.041 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.063+0000 7fcbff400640 1 -- 172.21.15.107:0/4245563064 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fdca6fa7a0 msgr2=0x55fdca703030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:15:03.041 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.063+0000 7fcbff400640 1 --2- 172.21.15.107:0/4245563064 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fdca6fa7a0 0x55fdca703030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:15:03.041 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.063+0000 7fcbff400640 1 -- 172.21.15.107:0/4245563064 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fdca62cb50 con 0x55fdca703570 2024-03-08T21:15:03.041 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.063+0000 7fcbff400640 1 --2- 172.21.15.107:0/4245563064 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fdca703570 0x55fdca705960 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fcbf00134b0 tx=0x7fcbf000c5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71d4628495d004ce server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:15:03.041 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.064+0000 7fcc05600640 1 --2- 172.21.15.107:0/70481697 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7fcbec0a2960 0x7fcbec0a4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:15:03.041 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.064+0000 7fcbfd600640 1 -- 172.21.15.107:0/4245563064 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbf0019030 con 0x55fdca703570 2024-03-08T21:15:03.041 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.064+0000 7fcbfd600640 1 -- 172.21.15.107:0/4245563064 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fcbf0011950 con 0x55fdca703570 2024-03-08T21:15:03.041 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.064+0000 7fcbfd600640 1 -- 172.21.15.107:0/4245563064 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbf000f800 con 0x55fdca703570 2024-03-08T21:15:03.041 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.064+0000 7fcc05600640 1 -- 172.21.15.107:0/70481697 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fcc000f1410 con 0x7fcc08199930 2024-03-08T21:15:03.042 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.064+0000 7fcc07e00640 1 --2- 172.21.15.107:0/70481697 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7fcbec0a2960 0x7fcbec0a4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:15:03.045 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.065+0000 7fcc11d6a740 1 -- 172.21.15.107:0/4245563064 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fdca703570 msgr2=0x55fdca705960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:15:03.045 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.065+0000 7fcc11d6a740 1 --2- 172.21.15.107:0/4245563064 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fdca703570 0x55fdca705960 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fcbf00134b0 tx=0x7fcbf000c5d0 comp rx=0 tx=0).stop 2024-03-08T21:15:03.045 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.065+0000 7fcc07e00640 1 --2- 172.21.15.107:0/70481697 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7fcbec0a2960 0x7fcbec0a4e20 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fcbf8005fd0 tx=0x7fcbf8009350 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:15:03.045 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.065+0000 7fcc11d6a740 1 -- 172.21.15.107:0/4245563064 shutdown_connections 2024-03-08T21:15:03.045 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.065+0000 7fcc11d6a740 1 --2- 172.21.15.107:0/4245563064 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fdca703570 0x55fdca705960 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:15:03.045 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.065+0000 7fcc11d6a740 1 --2- 172.21.15.107:0/4245563064 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fdca6fa7a0 0x55fdca703030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:15:03.045 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.065+0000 7fcc11d6a740 1 --2- 172.21.15.107:0/4245563064 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fdca6f9e80 0x55fdca6fa260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:15:03.045 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.065+0000 7fcc11d6a740 1 -- 172.21.15.107:0/4245563064 >> 172.21.15.107:0/4245563064 conn(0x55fdca6f47a0 msgr2=0x55fdca6f4b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:15:03.045 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.065+0000 7fcc11d6a740 1 -- 172.21.15.107:0/4245563064 shutdown_connections 2024-03-08T21:15:03.045 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.065+0000 7fcc11d6a740 1 -- 172.21.15.107:0/4245563064 wait complete. 2024-03-08T21:15:03.045 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.065+0000 7fcc11d6a740 1 -- start start 2024-03-08T21:15:03.045 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.066+0000 7fcc11d6a740 1 Processor -- start 2024-03-08T21:15:03.045 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.066+0000 7fcc11d6a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fdca6f9e80 0x55fdca702fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:15:03.045 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.066+0000 7fcc11d6a740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fdca6fa7a0 0x55fdca700330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:15:03.046 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.066+0000 7fcc11d6a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fdca703570 0x55fdca700a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:15:03.046 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.066+0000 7fcc11d6a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fdca7061a0 con 0x55fdca703570 2024-03-08T21:15:03.046 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.066+0000 7fcc11d6a740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fdca6199f0 con 0x55fdca6fa7a0 2024-03-08T21:15:03.046 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.066+0000 7fcc11d6a740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fdca706320 con 0x55fdca6f9e80 2024-03-08T21:15:03.046 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.066+0000 7fcc11d6a740 10 client.0 _ll_register_callbacks cb 0x55fdca371a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T21:15:03.046 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.066+0000 7fcc0d000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fdca703570 0x55fdca700a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:15:03.046 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.066+0000 7fcc0d000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fdca703570 0x55fdca700a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:54646/0 (socket says 172.21.15.107:54646) 2024-03-08T21:15:03.046 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.066+0000 7fcc0d000640 1 -- 172.21.15.107:0/4026351509 learned_addr learned my addr 172.21.15.107:0/4026351509 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:15:03.046 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.067+0000 7fcbff400640 1 --2- 172.21.15.107:0/4026351509 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fdca6fa7a0 0x55fdca700330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:15:03.046 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.067+0000 7fcc07e00640 1 --2- 172.21.15.107:0/4026351509 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fdca6f9e80 0x55fdca702fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:15:03.046 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.067+0000 7fcc0d000640 1 -- 172.21.15.107:0/4026351509 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fdca6f9e80 msgr2=0x55fdca702fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:15:03.046 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.067+0000 7fcc0d000640 1 --2- 172.21.15.107:0/4026351509 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fdca6f9e80 0x55fdca702fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:15:03.046 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.067+0000 7fcc0d000640 1 -- 172.21.15.107:0/4026351509 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fdca6fa7a0 msgr2=0x55fdca700330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:15:03.047 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.067+0000 7fcc0d000640 1 --2- 172.21.15.107:0/4026351509 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fdca6fa7a0 0x55fdca700330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:15:03.047 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.067+0000 7fcc0d000640 1 -- 172.21.15.107:0/4026351509 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fdca372210 con 0x55fdca703570 2024-03-08T21:15:03.047 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.067+0000 7fcbff400640 1 --2- 172.21.15.107:0/4026351509 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fdca6fa7a0 0x55fdca700330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T21:15:03.047 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.067+0000 7fcc0d000640 1 --2- 172.21.15.107:0/4026351509 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fdca703570 0x55fdca700a70 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7fcc00009dc0 tx=0x7fcc000bd740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a923af44270e78e server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:15:03.047 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.067+0000 7fcbf6000640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-03-08T21:15:03.047 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.067+0000 7fcbf6000640 1 -- 172.21.15.107:0/4026351509 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc000be240 con 0x55fdca703570 2024-03-08T21:15:03.047 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.067+0000 7fcbf6000640 1 -- 172.21.15.107:0/4026351509 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fcc000ba1b0 con 0x55fdca703570 2024-03-08T21:15:03.047 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.067+0000 7fcc11d6a740 1 -- 172.21.15.107:0/4026351509 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcbf0011950 con 0x55fdca703570 2024-03-08T21:15:03.047 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.068+0000 7fcbf6000640 1 -- 172.21.15.107:0/4026351509 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcc000bdb40 con 0x55fdca703570 2024-03-08T21:15:03.047 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.068+0000 7fcc11d6a740 10 client.16665 fetch_fsmap learned FSMap version 41 2024-03-08T21:15:03.047 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.068+0000 7fcc11d6a740 1 -- 172.21.15.107:0/4026351509 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x55fdca43e620 con 0x55fdca703570 2024-03-08T21:15:03.047 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.068+0000 7fcbf6000640 1 -- 172.21.15.107:0/4026351509 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fcbf0011950 con 0x55fdca703570 2024-03-08T21:15:03.047 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.068+0000 7fcbf6000640 1 -- 172.21.15.107:0/4026351509 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 41) v1 ==== 7307+0+0 (secure 0 0 0) 0x7fcc0003c030 con 0x55fdca703570 2024-03-08T21:15:03.047 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.068+0000 7fcc11d6a740 10 client.16665 fetch_fsmap finished waiting for FSMap version 41 2024-03-08T21:15:03.047 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.068+0000 7fcc11d6a740 10 client.16665 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-08T21:15:03.048 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.068+0000 7fcc11d6a740 20 client.16665 populate_metadata read hostname 'smithi107' 2024-03-08T21:15:03.048 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.068+0000 7fcc11d6a740 1 --2- 172.21.15.107:0/4026351509 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55fdca713440 0x55fdca7158a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:15:03.048 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.068+0000 7fcc11d6a740 4 client.16665 mds_command: new command op to 34475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-08T21:15:03.048 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.068+0000 7fcc11d6a740 1 -- 172.21.15.107:0/4026351509 --> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fdca66a910 con 0x55fdca713440 2024-03-08T21:15:03.048 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.069+0000 7fcc07e00640 1 --2- 172.21.15.107:0/4026351509 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55fdca713440 0x55fdca7158a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:15:03.048 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.070+0000 7fcc07e00640 1 --2- 172.21.15.107:0/4026351509 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55fdca713440 0x55fdca7158a0 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=df9bc5f6d9581441 server_cookie=12e6d9719803583 in_seq=0 out_seq=0 2024-03-08T21:15:03.048 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.070+0000 7fcbf6000640 10 client.16665 ms_handle_connect on v2:172.21.15.149:6838/55290596 2024-03-08T21:15:03.050 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.073+0000 7fcbf6000640 1 -- 172.21.15.107:0/4026351509 <== mds.0 v2:172.21.15.149:6838/55290596 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55fdca66a910 con 0x55fdca713440 2024-03-08T21:15:03.050 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.073+0000 7fcbf6000640 10 client.16665 handle_command_reply: tid=0 2024-03-08T21:15:03.050 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.073+0000 7fcc11d6a740 1 client.16665 shutdown 2024-03-08T21:15:03.050 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.073+0000 7fcc11d6a740 1 -- 172.21.15.107:0/4026351509 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fdca703570 msgr2=0x55fdca700a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:15:03.050 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.073+0000 7fcc11d6a740 1 --2- 172.21.15.107:0/4026351509 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fdca703570 0x55fdca700a70 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7fcc00009dc0 tx=0x7fcc000bd740 comp rx=0 tx=0).stop 2024-03-08T21:15:03.050 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.073+0000 7fcc11d6a740 1 -- 172.21.15.107:0/4026351509 shutdown_connections 2024-03-08T21:15:03.050 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.073+0000 7fcc11d6a740 1 --2- 172.21.15.107:0/4026351509 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fdca703570 0x55fdca700a70 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:15:03.050 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.073+0000 7fcc11d6a740 1 --2- 172.21.15.107:0/4026351509 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55fdca713440 0x55fdca7158a0 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:15:03.050 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.073+0000 7fcc11d6a740 1 --2- 172.21.15.107:0/4026351509 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fdca6fa7a0 0x55fdca700330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:15:03.050 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.073+0000 7fcc11d6a740 1 --2- 172.21.15.107:0/4026351509 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fdca6f9e80 0x55fdca702fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:15:03.050 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.073+0000 7fcc11d6a740 1 -- 172.21.15.107:0/4026351509 >> 172.21.15.107:0/4026351509 conn(0x55fdca6f47a0 msgr2=0x55fdca6f64f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:15:03.050 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.073+0000 7fcbf6000640 0 client.16665 ms_handle_reset on v2:172.21.15.149:6838/55290596 2024-03-08T21:15:03.050 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.074+0000 7fcc11d6a740 1 -- 172.21.15.107:0/4026351509 shutdown_connections 2024-03-08T21:15:03.051 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.074+0000 7fcc11d6a740 1 -- 172.21.15.107:0/4026351509 wait complete. 2024-03-08T21:15:03.051 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.074+0000 7fcc11d6a740 20 client.16665 trim_cache size 0 max 16384 2024-03-08T21:15:03.065 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.088+0000 7fcc11d6a740 1 Processor -- start 2024-03-08T21:15:03.065 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.088+0000 7fcc11d6a740 1 -- start start 2024-03-08T21:15:03.065 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.088+0000 7fcc11d6a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fdca6f9e80 0x55fdca702a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:15:03.065 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.088+0000 7fcc11d6a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fdca6fa7a0 0x55fdca702fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:15:03.065 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.088+0000 7fcc11d6a740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fdca703570 0x55fdca6ffbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:15:03.065 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.088+0000 7fcc11d6a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc081b0610 con 0x55fdca6fa7a0 2024-03-08T21:15:03.065 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.088+0000 7fcc11d6a740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc081ae630 con 0x55fdca703570 2024-03-08T21:15:03.065 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.088+0000 7fcc11d6a740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc081affb0 con 0x55fdca6f9e80 2024-03-08T21:15:03.066 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.089+0000 7fcbff400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fdca6fa7a0 0x55fdca702fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:15:03.066 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.089+0000 7fcbff400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fdca6fa7a0 0x55fdca702fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:54662/0 (socket says 172.21.15.107:54662) 2024-03-08T21:15:03.066 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.089+0000 7fcbff400640 1 -- 172.21.15.107:0/2510794608 learned_addr learned my addr 172.21.15.107:0/2510794608 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:15:03.066 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.089+0000 7fcc0d000640 1 --2- 172.21.15.107:0/2510794608 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fdca703570 0x55fdca6ffbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:15:03.066 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.089+0000 7fcc07e00640 1 --2- 172.21.15.107:0/2510794608 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fdca6f9e80 0x55fdca702a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:15:03.066 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.089+0000 7fcbff400640 1 -- 172.21.15.107:0/2510794608 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fdca6f9e80 msgr2=0x55fdca702a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:15:03.066 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.089+0000 7fcbff400640 1 --2- 172.21.15.107:0/2510794608 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fdca6f9e80 0x55fdca702a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:15:03.066 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.089+0000 7fcbff400640 1 -- 172.21.15.107:0/2510794608 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fdca703570 msgr2=0x55fdca6ffbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:15:03.066 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.089+0000 7fcbff400640 1 --2- 172.21.15.107:0/2510794608 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fdca703570 0x55fdca6ffbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:15:03.066 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.089+0000 7fcbff400640 1 -- 172.21.15.107:0/2510794608 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fdca71aad0 con 0x55fdca6fa7a0 2024-03-08T21:15:03.070 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.089+0000 7fcbff400640 1 --2- 172.21.15.107:0/2510794608 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fdca6fa7a0 0x55fdca702fa0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7fcbf00058c0 tx=0x7fcbf000c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75ea7d869405dd93 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:15:03.070 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.090+0000 7fcbfcc00640 1 -- 172.21.15.107:0/2510794608 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbf0019040 con 0x55fdca6fa7a0 2024-03-08T21:15:03.070 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.090+0000 7fcbfcc00640 1 -- 172.21.15.107:0/2510794608 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fcbf00054f0 con 0x55fdca6fa7a0 2024-03-08T21:15:03.070 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.090+0000 7fcbfcc00640 1 -- 172.21.15.107:0/2510794608 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbf0004c60 con 0x55fdca6fa7a0 2024-03-08T21:15:03.070 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.090+0000 7fcc11d6a740 1 -- 172.21.15.107:0/2510794608 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fdca6fa7a0 msgr2=0x55fdca702fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:15:03.070 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.090+0000 7fcc11d6a740 1 --2- 172.21.15.107:0/2510794608 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fdca6fa7a0 0x55fdca702fa0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7fcbf00058c0 tx=0x7fcbf000c890 comp rx=0 tx=0).stop 2024-03-08T21:15:03.070 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.091+0000 7fcc11d6a740 1 -- 172.21.15.107:0/2510794608 shutdown_connections 2024-03-08T21:15:03.070 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.091+0000 7fcc11d6a740 1 --2- 172.21.15.107:0/2510794608 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fdca703570 0x55fdca6ffbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:15:03.070 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.091+0000 7fcc11d6a740 1 --2- 172.21.15.107:0/2510794608 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fdca6fa7a0 0x55fdca702fa0 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:15:03.070 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.091+0000 7fcc11d6a740 1 --2- 172.21.15.107:0/2510794608 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fdca6f9e80 0x55fdca702a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:15:03.070 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.091+0000 7fcc11d6a740 1 -- 172.21.15.107:0/2510794608 >> 172.21.15.107:0/2510794608 conn(0x55fdca6f47a0 msgr2=0x55fdca6fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:15:03.071 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.091+0000 7fcc11d6a740 1 -- 172.21.15.107:0/2510794608 shutdown_connections 2024-03-08T21:15:03.071 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.091+0000 7fcc11d6a740 1 -- 172.21.15.107:0/2510794608 wait complete. 2024-03-08T21:15:03.071 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.091+0000 7fcc11d6a740 1 -- start start 2024-03-08T21:15:03.071 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.091+0000 7fcc11d6a740 1 Processor -- start 2024-03-08T21:15:03.071 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.092+0000 7fcc11d6a740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fdca6f9e80 0x55fdca708730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:15:03.071 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.092+0000 7fcc11d6a740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fdca6fa7a0 0x55fdca708c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:15:03.071 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.092+0000 7fcc11d6a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fdca703570 0x55fdca7689c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:15:03.071 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.092+0000 7fcc11d6a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc081af2f0 con 0x55fdca703570 2024-03-08T21:15:03.071 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.092+0000 7fcc11d6a740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc081aec90 con 0x55fdca6f9e80 2024-03-08T21:15:03.071 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.092+0000 7fcc11d6a740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc081af950 con 0x55fdca6fa7a0 2024-03-08T21:15:03.071 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.092+0000 7fcc11d6a740 10 client.0 _ll_register_callbacks cb 0x7fcbf000c470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T21:15:03.071 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.092+0000 7fcc0d000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fdca703570 0x55fdca7689c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:15:03.071 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.092+0000 7fcc0d000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fdca703570 0x55fdca7689c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:54670/0 (socket says 172.21.15.107:54670) 2024-03-08T21:15:03.071 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.092+0000 7fcc0d000640 1 -- 172.21.15.107:0/3122773862 learned_addr learned my addr 172.21.15.107:0/3122773862 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:15:03.071 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.092+0000 7fcc07e00640 1 --2- 172.21.15.107:0/3122773862 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fdca6f9e80 0x55fdca708730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:15:03.072 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.092+0000 7fcbff400640 1 --2- 172.21.15.107:0/3122773862 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fdca6fa7a0 0x55fdca708c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:15:03.072 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.092+0000 7fcc0d000640 1 -- 172.21.15.107:0/3122773862 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fdca6fa7a0 msgr2=0x55fdca708c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:15:03.072 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.092+0000 7fcc0d000640 1 --2- 172.21.15.107:0/3122773862 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fdca6fa7a0 0x55fdca708c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:15:03.072 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.092+0000 7fcc0d000640 1 -- 172.21.15.107:0/3122773862 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fdca6f9e80 msgr2=0x55fdca708730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:15:03.072 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.092+0000 7fcc0d000640 1 --2- 172.21.15.107:0/3122773862 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fdca6f9e80 0x55fdca708730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:15:03.072 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.092+0000 7fcc0d000640 1 -- 172.21.15.107:0/3122773862 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fdca768f50 con 0x55fdca703570 2024-03-08T21:15:03.072 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.092+0000 7fcbff400640 1 --2- 172.21.15.107:0/3122773862 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fdca6fa7a0 0x55fdca708c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T21:15:03.072 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.093+0000 7fcc0d000640 1 --2- 172.21.15.107:0/3122773862 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fdca703570 0x55fdca7689c0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7fcc000bd840 tx=0x7fcc000d3f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91073ea4a128a95b server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:15:03.072 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.093+0000 7fcbf6000640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-03-08T21:15:03.072 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.093+0000 7fcbf6000640 1 -- 172.21.15.107:0/3122773862 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc000b9de0 con 0x55fdca703570 2024-03-08T21:15:03.072 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.093+0000 7fcbf6000640 1 -- 172.21.15.107:0/3122773862 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fcc000bdec0 con 0x55fdca703570 2024-03-08T21:15:03.072 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.093+0000 7fcc11d6a740 1 -- 172.21.15.107:0/3122773862 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcbf00054f0 con 0x55fdca703570 2024-03-08T21:15:03.072 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.093+0000 7fcbf6000640 1 -- 172.21.15.107:0/3122773862 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcc000be240 con 0x55fdca703570 2024-03-08T21:15:03.073 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.093+0000 7fcc11d6a740 10 client.16671 fetch_fsmap learned FSMap version 41 2024-03-08T21:15:03.073 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.093+0000 7fcc11d6a740 1 -- 172.21.15.107:0/3122773862 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x55fdca76a260 con 0x55fdca703570 2024-03-08T21:15:03.073 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.093+0000 7fcbf6000640 1 -- 172.21.15.107:0/3122773862 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fcbf00054f0 con 0x55fdca703570 2024-03-08T21:15:03.073 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.093+0000 7fcbf6000640 1 -- 172.21.15.107:0/3122773862 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 41) v1 ==== 7307+0+0 (secure 0 0 0) 0x7fcc0002cca0 con 0x55fdca703570 2024-03-08T21:15:03.073 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.094+0000 7fcc11d6a740 10 client.16671 fetch_fsmap finished waiting for FSMap version 41 2024-03-08T21:15:03.073 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.094+0000 7fcc11d6a740 10 client.16671 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-08T21:15:03.073 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.094+0000 7fcc11d6a740 20 client.16671 populate_metadata read hostname 'smithi107' 2024-03-08T21:15:03.073 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.094+0000 7fcc11d6a740 1 --2- 172.21.15.107:0/3122773862 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55fdca76a850 0x55fdca76cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:15:03.073 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.094+0000 7fcc11d6a740 4 client.16671 mds_command: new command op to 34475 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-08T21:15:03.073 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.094+0000 7fcc11d6a740 1 -- 172.21.15.107:0/3122773862 --> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55fdca664c60 con 0x55fdca76a850 2024-03-08T21:15:03.074 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.094+0000 7fcc07e00640 1 --2- 172.21.15.107:0/3122773862 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55fdca76a850 0x55fdca76cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:15:03.074 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.095+0000 7fcc07e00640 1 --2- 172.21.15.107:0/3122773862 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55fdca76a850 0x55fdca76cc70 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e429d5bcdc7afa29 server_cookie=c28b9318a8a166a0 in_seq=0 out_seq=0 2024-03-08T21:15:03.074 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.095+0000 7fcbf6000640 10 client.16671 ms_handle_connect on v2:172.21.15.149:6838/55290596 2024-03-08T21:15:03.132 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 0 offset 4193280 len 2048 2024-03-08T21:15:03.132 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 0 offset 4193280 len 4096 2024-03-08T21:15:03.132 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 0 offset 4193280 len 4096 2024-03-08T21:15:03.132 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 0 offset 4193280 len 4096 2024-03-08T21:15:03.132 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 0 offset 4193280 len 4096 2024-03-08T21:15:03.132 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 0 offset 4193280 len 8192 2024-03-08T21:15:03.132 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 0 offset 4193280 len 8192 2024-03-08T21:15:03.132 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 0 offset 4193280 len 8192 2024-03-08T21:15:03.132 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 0 offset 4193280 len 8192 2024-03-08T21:15:03.132 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 0 offset 4193280 len 16384 2024-03-08T21:15:03.132 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 0 offset 4193280 len 16384 2024-03-08T21:15:03.133 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 0 offset 4193280 len 16384 2024-03-08T21:15:03.133 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 0 offset 4193280 len 16384 2024-03-08T21:15:03.133 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 0 offset 4193792 len 1024 2024-03-08T21:15:03.133 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 0 offset 4193792 len 1024 2024-03-08T21:15:03.133 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 0 offset 4193792 len 1024 2024-03-08T21:15:03.133 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 0 offset 4193792 len 1024 2024-03-08T21:15:03.133 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 0 offset 4193792 len 2048 2024-03-08T21:15:03.133 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 0 offset 4193792 len 2048 2024-03-08T21:15:03.133 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 0 offset 4193792 len 2048 2024-03-08T21:15:03.133 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 0 offset 4193792 len 2048 2024-03-08T21:15:03.133 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 0 offset 4193792 len 4096 2024-03-08T21:15:03.133 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 0 offset 4193792 len 4096 2024-03-08T21:15:03.133 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 0 offset 4193792 len 4096 2024-03-08T21:15:03.133 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 0 offset 4193792 len 4096 2024-03-08T21:15:03.133 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 0 offset 4193792 len 8192 2024-03-08T21:15:03.134 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 0 offset 4193792 len 8192 2024-03-08T21:15:03.134 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 0 offset 4193792 len 8192 2024-03-08T21:15:03.134 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 0 offset 4193792 len 8192 2024-03-08T21:15:03.134 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 0 offset 4193792 len 16384 2024-03-08T21:15:03.134 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 0 offset 4193792 len 16384 2024-03-08T21:15:03.134 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 0 offset 4193792 len 16384 2024-03-08T21:15:03.134 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 0 offset 4193792 len 16384 2024-03-08T21:15:03.134 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 0 offset 4194304 len 1024 2024-03-08T21:15:03.134 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 0 offset 4194304 len 1024 2024-03-08T21:15:03.134 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 0 offset 4194304 len 1024 2024-03-08T21:15:03.134 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 0 offset 4194304 len 1024 2024-03-08T21:15:03.134 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 0 offset 4194304 len 2048 2024-03-08T21:15:03.134 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 0 offset 4194304 len 2048 2024-03-08T21:15:03.135 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 0 offset 4194304 len 2048 2024-03-08T21:15:03.135 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 0 offset 4194304 len 2048 2024-03-08T21:15:03.135 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 0 offset 4194304 len 4096 2024-03-08T21:15:03.135 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 0 offset 4194304 len 4096 2024-03-08T21:15:03.135 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 0 offset 4194304 len 4096 2024-03-08T21:15:03.135 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 0 offset 4194304 len 4096 2024-03-08T21:15:03.135 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 0 offset 4194304 len 8192 2024-03-08T21:15:03.135 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 0 offset 4194304 len 8192 2024-03-08T21:15:03.135 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 0 offset 4194304 len 8192 2024-03-08T21:15:03.135 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 0 offset 4194304 len 8192 2024-03-08T21:15:03.135 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 0 offset 4194304 len 16384 2024-03-08T21:15:03.135 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 0 offset 4194304 len 16384 2024-03-08T21:15:03.135 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 0 offset 4194304 len 16384 2024-03-08T21:15:03.135 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 0 offset 4194304 len 16384 2024-03-08T21:15:03.135 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 0 offset 4194816 len 1024 2024-03-08T21:15:03.135 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 0 offset 4194816 len 1024 2024-03-08T21:15:03.136 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 0 offset 4194816 len 1024 2024-03-08T21:15:03.136 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 0 offset 4194816 len 1024 2024-03-08T21:15:03.136 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 0 offset 4194816 len 2048 2024-03-08T21:15:03.136 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 0 offset 4194816 len 2048 2024-03-08T21:15:03.136 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 0 offset 4194816 len 2048 2024-03-08T21:15:03.136 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 0 offset 4194816 len 2048 2024-03-08T21:15:03.136 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 0 offset 4194816 len 4096 2024-03-08T21:15:03.136 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 0 offset 4194816 len 4096 2024-03-08T21:15:03.136 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 0 offset 4194816 len 4096 2024-03-08T21:15:03.136 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 0 offset 4194816 len 4096 2024-03-08T21:15:03.136 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 0 offset 4194816 len 8192 2024-03-08T21:15:03.136 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 0 offset 4194816 len 8192 2024-03-08T21:15:03.136 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 0 offset 4194816 len 8192 2024-03-08T21:15:03.136 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 0 offset 4194816 len 8192 2024-03-08T21:15:03.136 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 0 offset 4194816 len 16384 2024-03-08T21:15:03.137 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 0 offset 4194816 len 16384 2024-03-08T21:15:03.137 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 0 offset 4194816 len 16384 2024-03-08T21:15:03.137 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 0 offset 4194816 len 16384 2024-03-08T21:15:03.137 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 0 offset 4195328 len 1024 2024-03-08T21:15:03.137 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 0 offset 4195328 len 1024 2024-03-08T21:15:03.137 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 0 offset 4195328 len 1024 2024-03-08T21:15:03.137 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 0 offset 4195328 len 1024 2024-03-08T21:15:03.137 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 0 offset 4195328 len 2048 2024-03-08T21:15:03.137 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 0 offset 4195328 len 2048 2024-03-08T21:15:03.137 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 0 offset 4195328 len 2048 2024-03-08T21:15:03.137 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 0 offset 4195328 len 2048 2024-03-08T21:15:03.137 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 0 offset 4195328 len 4096 2024-03-08T21:15:03.137 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 0 offset 4195328 len 4096 2024-03-08T21:15:03.137 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 0 offset 4195328 len 4096 2024-03-08T21:15:03.137 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 0 offset 4195328 len 4096 2024-03-08T21:15:03.152 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.176+0000 7fcbf6000640 1 -- 172.21.15.107:0/3122773862 <== mds.0 v2:172.21.15.149:6838/55290596 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55fdca664c60 con 0x55fdca76a850 2024-03-08T21:15:03.152 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.176+0000 7fcbf6000640 10 client.16671 handle_command_reply: tid=0 2024-03-08T21:15:03.152 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.176+0000 7fcc11d6a740 1 client.16671 shutdown 2024-03-08T21:15:03.153 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.176+0000 7fcc11d6a740 1 -- 172.21.15.107:0/3122773862 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fdca703570 msgr2=0x55fdca7689c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:15:03.153 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.176+0000 7fcc11d6a740 1 --2- 172.21.15.107:0/3122773862 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fdca703570 0x55fdca7689c0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7fcc000bd840 tx=0x7fcc000d3f90 comp rx=0 tx=0).stop 2024-03-08T21:15:03.153 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.176+0000 7fcc11d6a740 1 -- 172.21.15.107:0/3122773862 shutdown_connections 2024-03-08T21:15:03.153 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.176+0000 7fcc11d6a740 1 --2- 172.21.15.107:0/3122773862 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fdca703570 0x55fdca7689c0 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:15:03.153 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.176+0000 7fcc11d6a740 1 --2- 172.21.15.107:0/3122773862 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55fdca6fa7a0 0x55fdca708c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:15:03.153 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.176+0000 7fcc11d6a740 1 --2- 172.21.15.107:0/3122773862 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55fdca76a850 0x55fdca76cc70 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:15:03.153 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.177+0000 7fcc11d6a740 1 --2- 172.21.15.107:0/3122773862 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55fdca6f9e80 0x55fdca708730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:15:03.153 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.177+0000 7fcbf6000640 0 client.16671 ms_handle_reset on v2:172.21.15.149:6838/55290596 2024-03-08T21:15:03.153 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.177+0000 7fcc11d6a740 1 -- 172.21.15.107:0/3122773862 >> 172.21.15.107:0/3122773862 conn(0x55fdca6f47a0 msgr2=0x55fdca6ffb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:15:03.153 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.177+0000 7fcc11d6a740 1 -- 172.21.15.107:0/3122773862 shutdown_connections 2024-03-08T21:15:03.153 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.177+0000 7fcc11d6a740 1 -- 172.21.15.107:0/3122773862 wait complete. 2024-03-08T21:15:03.154 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.177+0000 7fcc11d6a740 20 client.16671 trim_cache size 0 max 16384 2024-03-08T21:15:03.154 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:{ 2024-03-08T21:15:03.154 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "message": "", 2024-03-08T21:15:03.154 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "return_code": 0 2024-03-08T21:15:03.154 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:} 2024-03-08T21:15:03.155 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.178+0000 7fcc0da00640 1 -- 172.21.15.107:0/70481697 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7fcbec0a2960 msgr2=0x7fcbec0a4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:15:03.155 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.178+0000 7fcc0da00640 1 --2- 172.21.15.107:0/70481697 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7fcbec0a2960 0x7fcbec0a4e20 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fcbf8005fd0 tx=0x7fcbf8009350 comp rx=0 tx=0).stop 2024-03-08T21:15:03.155 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.178+0000 7fcc0da00640 1 -- 172.21.15.107:0/70481697 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcc08199930 msgr2=0x7fcc08199de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:15:03.155 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.178+0000 7fcc0da00640 1 --2- 172.21.15.107:0/70481697 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcc08199930 0x7fcc08199de0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7fcc0002c700 tx=0x7fcc00035d60 comp rx=0 tx=0).stop 2024-03-08T21:15:03.155 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.179+0000 7fcc0da00640 1 -- 172.21.15.107:0/70481697 shutdown_connections 2024-03-08T21:15:03.155 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.179+0000 7fcc0da00640 1 --2- 172.21.15.107:0/70481697 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7fcbec0a2960 0x7fcbec0a4e20 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:15:03.155 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.179+0000 7fcc0da00640 1 --2- 172.21.15.107:0/70481697 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcc08199930 0x7fcc08199de0 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:15:03.155 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.179+0000 7fcc0da00640 1 --2- 172.21.15.107:0/70481697 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcc08108640 0x7fcc0819fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:15:03.155 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.179+0000 7fcc0da00640 1 --2- 172.21.15.107:0/70481697 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fcc08103570 0x7fcc0819f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:15:03.156 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.179+0000 7fcc0da00640 1 -- 172.21.15.107:0/70481697 >> 172.21.15.107:0/70481697 conn(0x7fcc080fedd0 msgr2=0x7fcc08101180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:15:03.156 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.179+0000 7fcc0da00640 1 -- 172.21.15.107:0/70481697 shutdown_connections 2024-03-08T21:15:03.156 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:03.179+0000 7fcc0da00640 1 -- 172.21.15.107:0/70481697 wait complete. 2024-03-08T21:15:04.438 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 0 offset 4195328 len 8192 2024-03-08T21:15:04.438 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 0 offset 4195328 len 8192 2024-03-08T21:15:04.438 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 0 offset 4195328 len 8192 2024-03-08T21:15:04.439 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 0 offset 4195328 len 8192 2024-03-08T21:15:04.439 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 0 offset 4195328 len 16384 2024-03-08T21:15:04.439 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 0 offset 4195328 len 16384 2024-03-08T21:15:04.439 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 0 offset 4195328 len 16384 2024-03-08T21:15:04.439 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 0 offset 4195328 len 16384 2024-03-08T21:15:04.439 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 0 offset 4195840 len 1024 2024-03-08T21:15:04.439 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 0 offset 4195840 len 1024 2024-03-08T21:15:04.439 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 0 offset 4195840 len 1024 2024-03-08T21:15:04.439 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 0 offset 4195840 len 1024 2024-03-08T21:15:04.439 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 0 offset 4195840 len 2048 2024-03-08T21:15:04.439 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 0 offset 4195840 len 2048 2024-03-08T21:15:04.439 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 0 offset 4195840 len 2048 2024-03-08T21:15:04.439 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 0 offset 4195840 len 2048 2024-03-08T21:15:04.440 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 0 offset 4195840 len 4096 2024-03-08T21:15:04.440 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 0 offset 4195840 len 4096 2024-03-08T21:15:04.440 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 0 offset 4195840 len 4096 2024-03-08T21:15:04.440 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 0 offset 4195840 len 4096 2024-03-08T21:15:04.440 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 0 offset 4195840 len 8192 2024-03-08T21:15:04.440 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 0 offset 4195840 len 8192 2024-03-08T21:15:04.440 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 0 offset 4195840 len 8192 2024-03-08T21:15:04.440 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 0 offset 4195840 len 8192 2024-03-08T21:15:04.440 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 0 offset 4195840 len 16384 2024-03-08T21:15:04.440 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 0 offset 4195840 len 16384 2024-03-08T21:15:04.440 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 0 offset 4195840 len 16384 2024-03-08T21:15:04.440 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 0 offset 4195840 len 16384 2024-03-08T21:15:04.440 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 0 offset 4196352 len 1024 2024-03-08T21:15:04.440 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 0 offset 4196352 len 1024 2024-03-08T21:15:04.440 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 0 offset 4196352 len 1024 2024-03-08T21:15:04.441 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 0 offset 4196352 len 1024 2024-03-08T21:15:04.441 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 0 offset 4196352 len 2048 2024-03-08T21:15:04.441 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 0 offset 4196352 len 2048 2024-03-08T21:15:04.441 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 0 offset 4196352 len 2048 2024-03-08T21:15:04.441 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 0 offset 4196352 len 2048 2024-03-08T21:15:04.441 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 0 offset 4196352 len 4096 2024-03-08T21:15:04.441 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 0 offset 4196352 len 4096 2024-03-08T21:15:04.441 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 0 offset 4196352 len 4096 2024-03-08T21:15:04.441 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 0 offset 4196352 len 4096 2024-03-08T21:15:04.441 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 0 offset 4196352 len 8192 2024-03-08T21:15:04.441 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 0 offset 4196352 len 8192 2024-03-08T21:15:04.441 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 0 offset 4196352 len 8192 2024-03-08T21:15:04.441 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 0 offset 4196352 len 8192 2024-03-08T21:15:04.441 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 0 offset 4196352 len 16384 2024-03-08T21:15:04.441 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 0 offset 4196352 len 16384 2024-03-08T21:15:04.442 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 0 offset 4196352 len 16384 2024-03-08T21:15:04.442 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 0 offset 4196352 len 16384 2024-03-08T21:15:04.442 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 0 offset 4196864 len 1024 2024-03-08T21:15:04.442 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 0 offset 4196864 len 1024 2024-03-08T21:15:04.442 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 0 offset 4196864 len 1024 2024-03-08T21:15:04.442 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 0 offset 4196864 len 1024 2024-03-08T21:15:04.442 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 0 offset 4196864 len 2048 2024-03-08T21:15:04.442 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 0 offset 4196864 len 2048 2024-03-08T21:15:04.442 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 0 offset 4196864 len 2048 2024-03-08T21:15:04.442 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 0 offset 4196864 len 2048 2024-03-08T21:15:04.442 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 0 offset 4196864 len 4096 2024-03-08T21:15:04.442 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 0 offset 4196864 len 4096 2024-03-08T21:15:04.442 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 0 offset 4196864 len 4096 2024-03-08T21:15:04.442 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 0 offset 4196864 len 4096 2024-03-08T21:15:04.443 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 0 offset 4196864 len 8192 2024-03-08T21:15:04.443 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 0 offset 4196864 len 8192 2024-03-08T21:15:04.443 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 0 offset 4196864 len 8192 2024-03-08T21:15:04.443 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 0 offset 4196864 len 8192 2024-03-08T21:15:04.443 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 0 offset 4196864 len 16384 2024-03-08T21:15:04.443 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 0 offset 4196864 len 16384 2024-03-08T21:15:04.443 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 0 offset 4196864 len 16384 2024-03-08T21:15:04.443 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 0 offset 4196864 len 16384 2024-03-08T21:15:04.443 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 0 offset 4197376 len 1024 2024-03-08T21:15:04.443 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 0 offset 4197376 len 1024 2024-03-08T21:15:04.443 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 0 offset 4197376 len 1024 2024-03-08T21:15:04.443 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 0 offset 4197376 len 1024 2024-03-08T21:15:04.443 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 0 offset 4197376 len 2048 2024-03-08T21:15:04.443 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 0 offset 4197376 len 2048 2024-03-08T21:15:04.443 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 0 offset 4197376 len 2048 2024-03-08T21:15:04.443 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 0 offset 4197376 len 2048 2024-03-08T21:15:04.444 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 0 offset 4197376 len 4096 2024-03-08T21:15:04.444 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 0 offset 4197376 len 4096 2024-03-08T21:15:04.444 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 0 offset 4197376 len 4096 2024-03-08T21:15:04.444 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 0 offset 4197376 len 4096 2024-03-08T21:15:04.444 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 0 offset 4197376 len 8192 2024-03-08T21:15:04.444 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 0 offset 4197376 len 8192 2024-03-08T21:15:04.444 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 0 offset 4197376 len 8192 2024-03-08T21:15:04.444 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 0 offset 4197376 len 8192 2024-03-08T21:15:04.444 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 0 offset 4197376 len 16384 2024-03-08T21:15:04.444 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 0 offset 4197376 len 16384 2024-03-08T21:15:04.710 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:15:04 smithi107 ceph-mon[31381]: pgmap v460: 97 pgs: 97 active+clean; 10 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 53 KiB/s wr, 37 op/s 2024-03-08T21:15:04.758 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:15:04 smithi149 ceph-mon[35042]: pgmap v460: 97 pgs: 97 active+clean; 10 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 53 KiB/s wr, 37 op/s 2024-03-08T21:15:04.872 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:15:04 smithi204 ceph-mon[34935]: pgmap v460: 97 pgs: 97 active+clean; 10 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 53 KiB/s wr, 37 op/s 2024-03-08T21:15:05.702 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 0 offset 4197376 len 16384 2024-03-08T21:15:05.703 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 0 offset 4197376 len 16384 2024-03-08T21:15:05.703 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 0 offset 4197888 len 1024 2024-03-08T21:15:05.703 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 0 offset 4197888 len 1024 2024-03-08T21:15:05.703 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 0 offset 4197888 len 1024 2024-03-08T21:15:05.703 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 0 offset 4197888 len 1024 2024-03-08T21:15:05.703 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 0 offset 4197888 len 2048 2024-03-08T21:15:05.703 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 0 offset 4197888 len 2048 2024-03-08T21:15:05.703 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 0 offset 4197888 len 2048 2024-03-08T21:15:05.703 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 0 offset 4197888 len 2048 2024-03-08T21:15:05.703 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 0 offset 4197888 len 4096 2024-03-08T21:15:05.703 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 0 offset 4197888 len 4096 2024-03-08T21:15:05.703 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 0 offset 4197888 len 4096 2024-03-08T21:15:05.703 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 0 offset 4197888 len 4096 2024-03-08T21:15:05.703 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 0 offset 4197888 len 8192 2024-03-08T21:15:05.703 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 0 offset 4197888 len 8192 2024-03-08T21:15:05.704 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 0 offset 4197888 len 8192 2024-03-08T21:15:05.704 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 0 offset 4197888 len 8192 2024-03-08T21:15:05.704 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 0 offset 4197888 len 16384 2024-03-08T21:15:05.704 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 0 offset 4197888 len 16384 2024-03-08T21:15:05.704 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 0 offset 4197888 len 16384 2024-03-08T21:15:05.704 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 0 offset 4197888 len 16384 2024-03-08T21:15:05.704 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 512 offset 4190720 len 1024 2024-03-08T21:15:05.704 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 512 offset 4190720 len 1024 2024-03-08T21:15:05.704 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 512 offset 4190720 len 1024 2024-03-08T21:15:05.704 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 512 offset 4190720 len 1024 2024-03-08T21:15:05.704 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 512 offset 4190720 len 2048 2024-03-08T21:15:05.704 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 512 offset 4190720 len 2048 2024-03-08T21:15:05.704 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 512 offset 4190720 len 2048 2024-03-08T21:15:05.704 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 512 offset 4190720 len 2048 2024-03-08T21:15:05.704 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 512 offset 4190720 len 4096 2024-03-08T21:15:05.704 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 512 offset 4190720 len 4096 2024-03-08T21:15:05.705 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 512 offset 4190720 len 4096 2024-03-08T21:15:05.705 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 512 offset 4190720 len 4096 2024-03-08T21:15:05.705 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 512 offset 4190720 len 8192 2024-03-08T21:15:05.705 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 512 offset 4190720 len 8192 2024-03-08T21:15:05.705 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 512 offset 4190720 len 8192 2024-03-08T21:15:05.705 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 512 offset 4190720 len 8192 2024-03-08T21:15:05.705 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 512 offset 4190720 len 16384 2024-03-08T21:15:05.705 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 512 offset 4190720 len 16384 2024-03-08T21:15:05.705 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 512 offset 4190720 len 16384 2024-03-08T21:15:05.705 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 512 offset 4190720 len 16384 2024-03-08T21:15:05.705 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 512 offset 4191232 len 1024 2024-03-08T21:15:05.705 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 512 offset 4191232 len 1024 2024-03-08T21:15:05.705 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 512 offset 4191232 len 1024 2024-03-08T21:15:05.705 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 512 offset 4191232 len 1024 2024-03-08T21:15:05.705 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 512 offset 4191232 len 2048 2024-03-08T21:15:05.705 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 512 offset 4191232 len 2048 2024-03-08T21:15:05.706 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 512 offset 4191232 len 2048 2024-03-08T21:15:05.706 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 512 offset 4191232 len 2048 2024-03-08T21:15:05.706 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 512 offset 4191232 len 4096 2024-03-08T21:15:05.706 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 512 offset 4191232 len 4096 2024-03-08T21:15:05.706 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 512 offset 4191232 len 4096 2024-03-08T21:15:05.706 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 512 offset 4191232 len 4096 2024-03-08T21:15:05.706 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 512 offset 4191232 len 8192 2024-03-08T21:15:05.706 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 512 offset 4191232 len 8192 2024-03-08T21:15:05.706 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 512 offset 4191232 len 8192 2024-03-08T21:15:05.706 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 512 offset 4191232 len 8192 2024-03-08T21:15:05.706 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 512 offset 4191232 len 16384 2024-03-08T21:15:05.706 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 512 offset 4191232 len 16384 2024-03-08T21:15:05.707 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 512 offset 4191232 len 16384 2024-03-08T21:15:05.707 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 512 offset 4191232 len 16384 2024-03-08T21:15:05.707 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 512 offset 4191744 len 1024 2024-03-08T21:15:05.707 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 512 offset 4191744 len 1024 2024-03-08T21:15:05.707 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 512 offset 4191744 len 1024 2024-03-08T21:15:05.707 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 512 offset 4191744 len 1024 2024-03-08T21:15:05.707 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 512 offset 4191744 len 2048 2024-03-08T21:15:05.707 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 512 offset 4191744 len 2048 2024-03-08T21:15:05.707 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 512 offset 4191744 len 2048 2024-03-08T21:15:05.707 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 512 offset 4191744 len 2048 2024-03-08T21:15:05.707 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 512 offset 4191744 len 4096 2024-03-08T21:15:05.707 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 512 offset 4191744 len 4096 2024-03-08T21:15:05.707 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 512 offset 4191744 len 4096 2024-03-08T21:15:05.707 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 512 offset 4191744 len 4096 2024-03-08T21:15:05.707 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 512 offset 4191744 len 8192 2024-03-08T21:15:05.708 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 512 offset 4191744 len 8192 2024-03-08T21:15:05.708 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 512 offset 4191744 len 8192 2024-03-08T21:15:05.708 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 512 offset 4191744 len 8192 2024-03-08T21:15:05.708 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 512 offset 4191744 len 16384 2024-03-08T21:15:05.708 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 512 offset 4191744 len 16384 2024-03-08T21:15:05.708 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 512 offset 4191744 len 16384 2024-03-08T21:15:05.708 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 512 offset 4191744 len 16384 2024-03-08T21:15:05.708 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 512 offset 4192256 len 1024 2024-03-08T21:15:05.708 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 512 offset 4192256 len 1024 2024-03-08T21:15:05.758 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:15:05 smithi149 ceph-mon[35042]: osdmap e112: 12 total, 12 up, 12 in 2024-03-08T21:15:05.871 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:15:05 smithi204 ceph-mon[34935]: osdmap e112: 12 total, 12 up, 12 in 2024-03-08T21:15:05.960 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:15:05 smithi107 ceph-mon[31381]: osdmap e112: 12 total, 12 up, 12 in 2024-03-08T21:15:06.758 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:15:06 smithi149 ceph-mon[35042]: pgmap v462: 97 pgs: 97 active+clean; 74 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 732 KiB/s rd, 525 KiB/s wr, 181 op/s 2024-03-08T21:15:06.872 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:15:06 smithi204 ceph-mon[34935]: pgmap v462: 97 pgs: 97 active+clean; 74 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 732 KiB/s rd, 525 KiB/s wr, 181 op/s 2024-03-08T21:15:06.960 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:15:06 smithi107 ceph-mon[31381]: pgmap v462: 97 pgs: 97 active+clean; 74 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 732 KiB/s rd, 525 KiB/s wr, 181 op/s 2024-03-08T21:15:06.961 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 512 offset 4192256 len 1024 2024-03-08T21:15:06.961 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 512 offset 4192256 len 1024 2024-03-08T21:15:06.961 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 512 offset 4192256 len 2048 2024-03-08T21:15:06.961 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 512 offset 4192256 len 2048 2024-03-08T21:15:06.962 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 512 offset 4192256 len 2048 2024-03-08T21:15:06.962 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 512 offset 4192256 len 2048 2024-03-08T21:15:06.962 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 512 offset 4192256 len 4096 2024-03-08T21:15:06.962 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 512 offset 4192256 len 4096 2024-03-08T21:15:06.962 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 512 offset 4192256 len 4096 2024-03-08T21:15:06.962 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 512 offset 4192256 len 4096 2024-03-08T21:15:06.962 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 512 offset 4192256 len 8192 2024-03-08T21:15:06.962 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 512 offset 4192256 len 8192 2024-03-08T21:15:06.962 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 512 offset 4192256 len 8192 2024-03-08T21:15:06.962 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 512 offset 4192256 len 8192 2024-03-08T21:15:06.962 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 512 offset 4192256 len 16384 2024-03-08T21:15:06.962 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 512 offset 4192256 len 16384 2024-03-08T21:15:06.962 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 512 offset 4192256 len 16384 2024-03-08T21:15:06.963 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 512 offset 4192256 len 16384 2024-03-08T21:15:06.963 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 512 offset 4192768 len 1024 2024-03-08T21:15:06.963 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 512 offset 4192768 len 1024 2024-03-08T21:15:06.963 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 512 offset 4192768 len 1024 2024-03-08T21:15:06.963 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 512 offset 4192768 len 1024 2024-03-08T21:15:06.963 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 512 offset 4192768 len 2048 2024-03-08T21:15:06.963 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 512 offset 4192768 len 2048 2024-03-08T21:15:06.963 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 512 offset 4192768 len 2048 2024-03-08T21:15:06.963 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 512 offset 4192768 len 2048 2024-03-08T21:15:06.963 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 512 offset 4192768 len 4096 2024-03-08T21:15:06.963 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 512 offset 4192768 len 4096 2024-03-08T21:15:06.963 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 512 offset 4192768 len 4096 2024-03-08T21:15:06.963 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 512 offset 4192768 len 4096 2024-03-08T21:15:06.963 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 512 offset 4192768 len 8192 2024-03-08T21:15:06.963 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 512 offset 4192768 len 8192 2024-03-08T21:15:06.964 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 512 offset 4192768 len 8192 2024-03-08T21:15:06.964 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 512 offset 4192768 len 8192 2024-03-08T21:15:06.964 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 512 offset 4192768 len 16384 2024-03-08T21:15:06.964 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 512 offset 4192768 len 16384 2024-03-08T21:15:06.964 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 512 offset 4192768 len 16384 2024-03-08T21:15:06.964 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 512 offset 4192768 len 16384 2024-03-08T21:15:06.964 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 512 offset 4193280 len 1024 2024-03-08T21:15:06.964 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 512 offset 4193280 len 1024 2024-03-08T21:15:06.964 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 512 offset 4193280 len 1024 2024-03-08T21:15:06.964 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 512 offset 4193280 len 1024 2024-03-08T21:15:06.964 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 512 offset 4193280 len 2048 2024-03-08T21:15:06.964 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 512 offset 4193280 len 2048 2024-03-08T21:15:06.964 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 512 offset 4193280 len 2048 2024-03-08T21:15:06.964 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 512 offset 4193280 len 2048 2024-03-08T21:15:06.965 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 512 offset 4193280 len 4096 2024-03-08T21:15:06.965 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 512 offset 4193280 len 4096 2024-03-08T21:15:06.965 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 512 offset 4193280 len 4096 2024-03-08T21:15:06.965 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 512 offset 4193280 len 4096 2024-03-08T21:15:06.965 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 512 offset 4193280 len 8192 2024-03-08T21:15:06.965 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 512 offset 4193280 len 8192 2024-03-08T21:15:06.965 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 512 offset 4193280 len 8192 2024-03-08T21:15:06.965 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 512 offset 4193280 len 8192 2024-03-08T21:15:06.965 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 512 offset 4193280 len 16384 2024-03-08T21:15:06.965 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 512 offset 4193280 len 16384 2024-03-08T21:15:06.965 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 512 offset 4193280 len 16384 2024-03-08T21:15:06.965 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 512 offset 4193280 len 16384 2024-03-08T21:15:06.965 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 512 offset 4193792 len 1024 2024-03-08T21:15:06.965 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 512 offset 4193792 len 1024 2024-03-08T21:15:06.966 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 512 offset 4193792 len 1024 2024-03-08T21:15:06.966 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 512 offset 4193792 len 1024 2024-03-08T21:15:06.966 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 512 offset 4193792 len 2048 2024-03-08T21:15:06.966 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 512 offset 4193792 len 2048 2024-03-08T21:15:06.966 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 512 offset 4193792 len 2048 2024-03-08T21:15:06.966 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 512 offset 4193792 len 2048 2024-03-08T21:15:06.966 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 512 offset 4193792 len 4096 2024-03-08T21:15:06.966 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 512 offset 4193792 len 4096 2024-03-08T21:15:06.966 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 512 offset 4193792 len 4096 2024-03-08T21:15:06.966 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 512 offset 4193792 len 4096 2024-03-08T21:15:06.966 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 512 offset 4193792 len 8192 2024-03-08T21:15:06.966 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 512 offset 4193792 len 8192 2024-03-08T21:15:06.966 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 512 offset 4193792 len 8192 2024-03-08T21:15:06.966 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 512 offset 4193792 len 8192 2024-03-08T21:15:06.966 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 512 offset 4193792 len 16384 2024-03-08T21:15:06.967 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 512 offset 4193792 len 16384 2024-03-08T21:15:06.967 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 512 offset 4193792 len 16384 2024-03-08T21:15:06.967 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 512 offset 4193792 len 16384 2024-03-08T21:15:06.967 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 512 offset 4194304 len 1024 2024-03-08T21:15:06.967 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 512 offset 4194304 len 1024 2024-03-08T21:15:06.967 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 512 offset 4194304 len 1024 2024-03-08T21:15:06.967 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 512 offset 4194304 len 1024 2024-03-08T21:15:08.283 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 512 offset 4194304 len 2048 2024-03-08T21:15:08.283 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 512 offset 4194304 len 2048 2024-03-08T21:15:08.283 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 512 offset 4194304 len 2048 2024-03-08T21:15:08.283 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 512 offset 4194304 len 2048 2024-03-08T21:15:08.283 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 512 offset 4194304 len 4096 2024-03-08T21:15:08.284 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 512 offset 4194304 len 4096 2024-03-08T21:15:08.284 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 512 offset 4194304 len 4096 2024-03-08T21:15:08.284 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 512 offset 4194304 len 4096 2024-03-08T21:15:08.284 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 512 offset 4194304 len 8192 2024-03-08T21:15:08.284 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 512 offset 4194304 len 8192 2024-03-08T21:15:08.284 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 512 offset 4194304 len 8192 2024-03-08T21:15:08.284 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 512 offset 4194304 len 8192 2024-03-08T21:15:08.284 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 512 offset 4194304 len 16384 2024-03-08T21:15:08.284 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 512 offset 4194304 len 16384 2024-03-08T21:15:08.284 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 512 offset 4194304 len 16384 2024-03-08T21:15:08.284 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 512 offset 4194304 len 16384 2024-03-08T21:15:08.284 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 512 offset 4194816 len 1024 2024-03-08T21:15:08.284 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 512 offset 4194816 len 1024 2024-03-08T21:15:08.284 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 512 offset 4194816 len 1024 2024-03-08T21:15:08.284 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 512 offset 4194816 len 1024 2024-03-08T21:15:08.285 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 512 offset 4194816 len 2048 2024-03-08T21:15:08.285 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 512 offset 4194816 len 2048 2024-03-08T21:15:08.285 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 512 offset 4194816 len 2048 2024-03-08T21:15:08.285 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 512 offset 4194816 len 2048 2024-03-08T21:15:08.285 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 512 offset 4194816 len 4096 2024-03-08T21:15:08.285 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 512 offset 4194816 len 4096 2024-03-08T21:15:08.285 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 512 offset 4194816 len 4096 2024-03-08T21:15:08.285 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 512 offset 4194816 len 4096 2024-03-08T21:15:08.285 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 512 offset 4194816 len 8192 2024-03-08T21:15:08.285 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 512 offset 4194816 len 8192 2024-03-08T21:15:08.285 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 512 offset 4194816 len 8192 2024-03-08T21:15:08.285 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 512 offset 4194816 len 8192 2024-03-08T21:15:08.285 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 512 offset 4194816 len 16384 2024-03-08T21:15:08.286 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 512 offset 4194816 len 16384 2024-03-08T21:15:08.286 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 512 offset 4194816 len 16384 2024-03-08T21:15:08.286 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 512 offset 4194816 len 16384 2024-03-08T21:15:08.286 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 512 offset 4195328 len 1024 2024-03-08T21:15:08.286 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 512 offset 4195328 len 1024 2024-03-08T21:15:08.286 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 512 offset 4195328 len 1024 2024-03-08T21:15:08.286 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 512 offset 4195328 len 1024 2024-03-08T21:15:08.286 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 512 offset 4195328 len 2048 2024-03-08T21:15:08.286 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 512 offset 4195328 len 2048 2024-03-08T21:15:08.286 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 512 offset 4195328 len 2048 2024-03-08T21:15:08.286 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 512 offset 4195328 len 2048 2024-03-08T21:15:08.286 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 512 offset 4195328 len 4096 2024-03-08T21:15:08.286 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 512 offset 4195328 len 4096 2024-03-08T21:15:08.286 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 512 offset 4195328 len 4096 2024-03-08T21:15:08.287 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 512 offset 4195328 len 4096 2024-03-08T21:15:08.287 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 512 offset 4195328 len 8192 2024-03-08T21:15:08.287 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 512 offset 4195328 len 8192 2024-03-08T21:15:08.287 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 512 offset 4195328 len 8192 2024-03-08T21:15:08.287 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 512 offset 4195328 len 8192 2024-03-08T21:15:08.287 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 512 offset 4195328 len 16384 2024-03-08T21:15:08.287 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 512 offset 4195328 len 16384 2024-03-08T21:15:08.287 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 512 offset 4195328 len 16384 2024-03-08T21:15:08.287 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 512 offset 4195328 len 16384 2024-03-08T21:15:08.287 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 512 offset 4195840 len 1024 2024-03-08T21:15:08.287 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 512 offset 4195840 len 1024 2024-03-08T21:15:08.287 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 512 offset 4195840 len 1024 2024-03-08T21:15:08.287 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 512 offset 4195840 len 1024 2024-03-08T21:15:08.287 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 512 offset 4195840 len 2048 2024-03-08T21:15:08.288 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 512 offset 4195840 len 2048 2024-03-08T21:15:08.288 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 512 offset 4195840 len 2048 2024-03-08T21:15:08.288 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 512 offset 4195840 len 2048 2024-03-08T21:15:08.288 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 512 offset 4195840 len 4096 2024-03-08T21:15:08.288 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 512 offset 4195840 len 4096 2024-03-08T21:15:08.288 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 512 offset 4195840 len 4096 2024-03-08T21:15:08.288 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 512 offset 4195840 len 4096 2024-03-08T21:15:08.288 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 512 offset 4195840 len 8192 2024-03-08T21:15:08.288 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 512 offset 4195840 len 8192 2024-03-08T21:15:08.288 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 512 offset 4195840 len 8192 2024-03-08T21:15:08.288 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 512 offset 4195840 len 8192 2024-03-08T21:15:08.288 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 512 offset 4195840 len 16384 2024-03-08T21:15:08.288 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 512 offset 4195840 len 16384 2024-03-08T21:15:08.288 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 512 offset 4195840 len 16384 2024-03-08T21:15:08.288 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 512 offset 4195840 len 16384 2024-03-08T21:15:08.289 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 512 offset 4196352 len 1024 2024-03-08T21:15:08.289 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 512 offset 4196352 len 1024 2024-03-08T21:15:08.289 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 512 offset 4196352 len 1024 2024-03-08T21:15:08.289 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 512 offset 4196352 len 1024 2024-03-08T21:15:08.289 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 512 offset 4196352 len 2048 2024-03-08T21:15:08.289 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 512 offset 4196352 len 2048 2024-03-08T21:15:08.759 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:15:08 smithi149 ceph-mon[35042]: pgmap v463: 97 pgs: 97 active+clean; 74 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 732 KiB/s rd, 525 KiB/s wr, 181 op/s 2024-03-08T21:15:08.872 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:15:08 smithi204 ceph-mon[34935]: pgmap v463: 97 pgs: 97 active+clean; 74 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 732 KiB/s rd, 525 KiB/s wr, 181 op/s 2024-03-08T21:15:08.961 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:15:08 smithi107 ceph-mon[31381]: pgmap v463: 97 pgs: 97 active+clean; 74 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 732 KiB/s rd, 525 KiB/s wr, 181 op/s 2024-03-08T21:15:09.570 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 512 offset 4196352 len 2048 2024-03-08T21:15:09.571 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 512 offset 4196352 len 2048 2024-03-08T21:15:09.571 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 512 offset 4196352 len 4096 2024-03-08T21:15:09.571 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 512 offset 4196352 len 4096 2024-03-08T21:15:09.571 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 512 offset 4196352 len 4096 2024-03-08T21:15:09.571 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 512 offset 4196352 len 4096 2024-03-08T21:15:09.571 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 512 offset 4196352 len 8192 2024-03-08T21:15:09.571 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 512 offset 4196352 len 8192 2024-03-08T21:15:09.571 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 512 offset 4196352 len 8192 2024-03-08T21:15:09.571 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 512 offset 4196352 len 8192 2024-03-08T21:15:09.571 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 512 offset 4196352 len 16384 2024-03-08T21:15:09.571 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 512 offset 4196352 len 16384 2024-03-08T21:15:09.572 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 512 offset 4196352 len 16384 2024-03-08T21:15:09.572 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 512 offset 4196352 len 16384 2024-03-08T21:15:09.572 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 512 offset 4196864 len 1024 2024-03-08T21:15:09.572 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 512 offset 4196864 len 1024 2024-03-08T21:15:09.572 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 512 offset 4196864 len 1024 2024-03-08T21:15:09.572 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 512 offset 4196864 len 1024 2024-03-08T21:15:09.572 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 512 offset 4196864 len 2048 2024-03-08T21:15:09.572 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 512 offset 4196864 len 2048 2024-03-08T21:15:09.572 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 512 offset 4196864 len 2048 2024-03-08T21:15:09.572 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 512 offset 4196864 len 2048 2024-03-08T21:15:09.572 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 512 offset 4196864 len 4096 2024-03-08T21:15:09.572 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 512 offset 4196864 len 4096 2024-03-08T21:15:09.572 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 512 offset 4196864 len 4096 2024-03-08T21:15:09.572 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 512 offset 4196864 len 4096 2024-03-08T21:15:09.572 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 512 offset 4196864 len 8192 2024-03-08T21:15:09.573 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 512 offset 4196864 len 8192 2024-03-08T21:15:09.573 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 512 offset 4196864 len 8192 2024-03-08T21:15:09.573 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 512 offset 4196864 len 8192 2024-03-08T21:15:09.573 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 512 offset 4196864 len 16384 2024-03-08T21:15:09.573 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 512 offset 4196864 len 16384 2024-03-08T21:15:09.573 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 512 offset 4196864 len 16384 2024-03-08T21:15:09.573 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 512 offset 4196864 len 16384 2024-03-08T21:15:09.573 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 512 offset 4197376 len 1024 2024-03-08T21:15:09.573 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 512 offset 4197376 len 1024 2024-03-08T21:15:09.573 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 512 offset 4197376 len 1024 2024-03-08T21:15:09.573 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 512 offset 4197376 len 1024 2024-03-08T21:15:09.573 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 512 offset 4197376 len 2048 2024-03-08T21:15:09.573 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 512 offset 4197376 len 2048 2024-03-08T21:15:09.573 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 512 offset 4197376 len 2048 2024-03-08T21:15:09.574 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 512 offset 4197376 len 2048 2024-03-08T21:15:09.574 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 512 offset 4197376 len 4096 2024-03-08T21:15:09.574 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 512 offset 4197376 len 4096 2024-03-08T21:15:09.574 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 512 offset 4197376 len 4096 2024-03-08T21:15:09.574 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 512 offset 4197376 len 4096 2024-03-08T21:15:09.574 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 512 offset 4197376 len 8192 2024-03-08T21:15:09.574 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 512 offset 4197376 len 8192 2024-03-08T21:15:09.574 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 512 offset 4197376 len 8192 2024-03-08T21:15:09.574 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 512 offset 4197376 len 8192 2024-03-08T21:15:09.574 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 512 offset 4197376 len 16384 2024-03-08T21:15:09.574 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 512 offset 4197376 len 16384 2024-03-08T21:15:09.574 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 512 offset 4197376 len 16384 2024-03-08T21:15:09.574 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 512 offset 4197376 len 16384 2024-03-08T21:15:09.575 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 512 offset 4197888 len 1024 2024-03-08T21:15:09.575 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 512 offset 4197888 len 1024 2024-03-08T21:15:09.575 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 512 offset 4197888 len 1024 2024-03-08T21:15:09.575 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 512 offset 4197888 len 1024 2024-03-08T21:15:09.575 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 512 offset 4197888 len 2048 2024-03-08T21:15:09.575 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 512 offset 4197888 len 2048 2024-03-08T21:15:09.575 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 512 offset 4197888 len 2048 2024-03-08T21:15:09.575 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 512 offset 4197888 len 2048 2024-03-08T21:15:09.575 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 512 offset 4197888 len 4096 2024-03-08T21:15:09.575 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 512 offset 4197888 len 4096 2024-03-08T21:15:09.575 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 512 offset 4197888 len 4096 2024-03-08T21:15:09.575 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 512 offset 4197888 len 4096 2024-03-08T21:15:09.575 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 512 offset 4197888 len 8192 2024-03-08T21:15:09.575 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 512 offset 4197888 len 8192 2024-03-08T21:15:09.575 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 512 offset 4197888 len 8192 2024-03-08T21:15:09.576 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 512 offset 4197888 len 8192 2024-03-08T21:15:09.576 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 512 offset 4197888 len 16384 2024-03-08T21:15:09.576 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 512 offset 4197888 len 16384 2024-03-08T21:15:09.576 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 512 offset 4197888 len 16384 2024-03-08T21:15:09.576 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 512 offset 4197888 len 16384 2024-03-08T21:15:09.576 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 1024 offset 4190720 len 1024 2024-03-08T21:15:09.576 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1024 offset 4190720 len 1024 2024-03-08T21:15:09.576 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 1024 offset 4190720 len 1024 2024-03-08T21:15:09.576 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1024 offset 4190720 len 1024 2024-03-08T21:15:09.576 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 1024 offset 4190720 len 2048 2024-03-08T21:15:09.576 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1024 offset 4190720 len 2048 2024-03-08T21:15:09.576 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 1024 offset 4190720 len 2048 2024-03-08T21:15:09.576 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1024 offset 4190720 len 2048 2024-03-08T21:15:09.576 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 1024 offset 4190720 len 4096 2024-03-08T21:15:10.635 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1024 offset 4190720 len 4096 2024-03-08T21:15:10.635 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 1024 offset 4190720 len 4096 2024-03-08T21:15:10.635 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1024 offset 4190720 len 4096 2024-03-08T21:15:10.635 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 1024 offset 4190720 len 8192 2024-03-08T21:15:10.635 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1024 offset 4190720 len 8192 2024-03-08T21:15:10.635 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 1024 offset 4190720 len 8192 2024-03-08T21:15:10.635 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1024 offset 4190720 len 8192 2024-03-08T21:15:10.635 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 1024 offset 4190720 len 16384 2024-03-08T21:15:10.635 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1024 offset 4190720 len 16384 2024-03-08T21:15:10.635 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 1024 offset 4190720 len 16384 2024-03-08T21:15:10.635 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1024 offset 4190720 len 16384 2024-03-08T21:15:10.636 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 1024 offset 4191232 len 1024 2024-03-08T21:15:10.636 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1024 offset 4191232 len 1024 2024-03-08T21:15:10.636 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 1024 offset 4191232 len 1024 2024-03-08T21:15:10.636 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1024 offset 4191232 len 1024 2024-03-08T21:15:10.636 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 1024 offset 4191232 len 2048 2024-03-08T21:15:10.636 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1024 offset 4191232 len 2048 2024-03-08T21:15:10.636 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 1024 offset 4191232 len 2048 2024-03-08T21:15:10.636 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1024 offset 4191232 len 2048 2024-03-08T21:15:10.636 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 1024 offset 4191232 len 4096 2024-03-08T21:15:10.636 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1024 offset 4191232 len 4096 2024-03-08T21:15:10.636 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 1024 offset 4191232 len 4096 2024-03-08T21:15:10.636 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1024 offset 4191232 len 4096 2024-03-08T21:15:10.636 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 1024 offset 4191232 len 8192 2024-03-08T21:15:10.637 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1024 offset 4191232 len 8192 2024-03-08T21:15:10.637 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 1024 offset 4191232 len 8192 2024-03-08T21:15:10.637 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1024 offset 4191232 len 8192 2024-03-08T21:15:10.637 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 1024 offset 4191232 len 16384 2024-03-08T21:15:10.637 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1024 offset 4191232 len 16384 2024-03-08T21:15:10.637 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 1024 offset 4191232 len 16384 2024-03-08T21:15:10.637 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1024 offset 4191232 len 16384 2024-03-08T21:15:10.637 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 1024 offset 4191744 len 1024 2024-03-08T21:15:10.637 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1024 offset 4191744 len 1024 2024-03-08T21:15:10.637 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 1024 offset 4191744 len 1024 2024-03-08T21:15:10.637 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1024 offset 4191744 len 1024 2024-03-08T21:15:10.637 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 1024 offset 4191744 len 2048 2024-03-08T21:15:10.637 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1024 offset 4191744 len 2048 2024-03-08T21:15:10.637 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 1024 offset 4191744 len 2048 2024-03-08T21:15:10.637 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1024 offset 4191744 len 2048 2024-03-08T21:15:10.638 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 1024 offset 4191744 len 4096 2024-03-08T21:15:10.638 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1024 offset 4191744 len 4096 2024-03-08T21:15:10.638 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 1024 offset 4191744 len 4096 2024-03-08T21:15:10.638 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1024 offset 4191744 len 4096 2024-03-08T21:15:10.638 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 1024 offset 4191744 len 8192 2024-03-08T21:15:10.638 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1024 offset 4191744 len 8192 2024-03-08T21:15:10.638 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 1024 offset 4191744 len 8192 2024-03-08T21:15:10.638 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1024 offset 4191744 len 8192 2024-03-08T21:15:10.638 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 1024 offset 4191744 len 16384 2024-03-08T21:15:10.638 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1024 offset 4191744 len 16384 2024-03-08T21:15:10.638 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 1024 offset 4191744 len 16384 2024-03-08T21:15:10.638 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1024 offset 4191744 len 16384 2024-03-08T21:15:10.638 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 1024 offset 4192256 len 1024 2024-03-08T21:15:10.638 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1024 offset 4192256 len 1024 2024-03-08T21:15:10.639 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 1024 offset 4192256 len 1024 2024-03-08T21:15:10.639 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1024 offset 4192256 len 1024 2024-03-08T21:15:10.639 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 1024 offset 4192256 len 2048 2024-03-08T21:15:10.639 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1024 offset 4192256 len 2048 2024-03-08T21:15:10.639 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 1024 offset 4192256 len 2048 2024-03-08T21:15:10.639 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1024 offset 4192256 len 2048 2024-03-08T21:15:10.639 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 1024 offset 4192256 len 4096 2024-03-08T21:15:10.639 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1024 offset 4192256 len 4096 2024-03-08T21:15:10.639 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 1024 offset 4192256 len 4096 2024-03-08T21:15:10.639 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1024 offset 4192256 len 4096 2024-03-08T21:15:10.639 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 1024 offset 4192256 len 8192 2024-03-08T21:15:10.639 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1024 offset 4192256 len 8192 2024-03-08T21:15:10.639 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 1024 offset 4192256 len 8192 2024-03-08T21:15:10.639 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1024 offset 4192256 len 8192 2024-03-08T21:15:10.639 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 1024 offset 4192256 len 16384 2024-03-08T21:15:10.640 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1024 offset 4192256 len 16384 2024-03-08T21:15:10.640 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 1024 offset 4192256 len 16384 2024-03-08T21:15:10.640 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1024 offset 4192256 len 16384 2024-03-08T21:15:10.640 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 1024 offset 4192768 len 1024 2024-03-08T21:15:10.640 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1024 offset 4192768 len 1024 2024-03-08T21:15:10.640 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 1024 offset 4192768 len 1024 2024-03-08T21:15:10.640 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1024 offset 4192768 len 1024 2024-03-08T21:15:10.640 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 1024 offset 4192768 len 2048 2024-03-08T21:15:10.640 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1024 offset 4192768 len 2048 2024-03-08T21:15:10.640 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 1024 offset 4192768 len 2048 2024-03-08T21:15:10.640 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1024 offset 4192768 len 2048 2024-03-08T21:15:10.640 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 1024 offset 4192768 len 4096 2024-03-08T21:15:10.758 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:15:10 smithi149 ceph-mon[35042]: pgmap v464: 97 pgs: 97 active+clean; 22 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1003 KiB/s wr, 231 op/s 2024-03-08T21:15:10.872 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:15:10 smithi204 ceph-mon[34935]: pgmap v464: 97 pgs: 97 active+clean; 22 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1003 KiB/s wr, 231 op/s 2024-03-08T21:15:10.961 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:15:10 smithi107 ceph-mon[31381]: pgmap v464: 97 pgs: 97 active+clean; 22 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1003 KiB/s wr, 231 op/s 2024-03-08T21:15:11.597 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1024 offset 4192768 len 4096 2024-03-08T21:15:11.597 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 1024 offset 4192768 len 4096 2024-03-08T21:15:11.597 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1024 offset 4192768 len 4096 2024-03-08T21:15:11.597 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 1024 offset 4192768 len 8192 2024-03-08T21:15:11.597 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1024 offset 4192768 len 8192 2024-03-08T21:15:11.597 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 1024 offset 4192768 len 8192 2024-03-08T21:15:11.597 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1024 offset 4192768 len 8192 2024-03-08T21:15:11.597 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 1024 offset 4192768 len 16384 2024-03-08T21:15:11.597 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1024 offset 4192768 len 16384 2024-03-08T21:15:11.597 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 1024 offset 4192768 len 16384 2024-03-08T21:15:11.597 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1024 offset 4192768 len 16384 2024-03-08T21:15:11.598 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 1024 offset 4193280 len 1024 2024-03-08T21:15:11.598 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1024 offset 4193280 len 1024 2024-03-08T21:15:11.598 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 1024 offset 4193280 len 1024 2024-03-08T21:15:11.598 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1024 offset 4193280 len 1024 2024-03-08T21:15:11.598 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 1024 offset 4193280 len 2048 2024-03-08T21:15:11.598 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1024 offset 4193280 len 2048 2024-03-08T21:15:11.598 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 1024 offset 4193280 len 2048 2024-03-08T21:15:11.598 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1024 offset 4193280 len 2048 2024-03-08T21:15:11.598 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 1024 offset 4193280 len 4096 2024-03-08T21:15:11.598 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1024 offset 4193280 len 4096 2024-03-08T21:15:11.598 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 1024 offset 4193280 len 4096 2024-03-08T21:15:11.598 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1024 offset 4193280 len 4096 2024-03-08T21:15:11.598 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 1024 offset 4193280 len 8192 2024-03-08T21:15:11.599 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1024 offset 4193280 len 8192 2024-03-08T21:15:11.599 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 1024 offset 4193280 len 8192 2024-03-08T21:15:11.599 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1024 offset 4193280 len 8192 2024-03-08T21:15:11.599 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 1024 offset 4193280 len 16384 2024-03-08T21:15:11.599 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1024 offset 4193280 len 16384 2024-03-08T21:15:11.599 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 1024 offset 4193280 len 16384 2024-03-08T21:15:11.599 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1024 offset 4193280 len 16384 2024-03-08T21:15:11.599 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 1024 offset 4193792 len 1024 2024-03-08T21:15:11.599 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1024 offset 4193792 len 1024 2024-03-08T21:15:11.599 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 1024 offset 4193792 len 1024 2024-03-08T21:15:11.599 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1024 offset 4193792 len 1024 2024-03-08T21:15:11.599 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 1024 offset 4193792 len 2048 2024-03-08T21:15:11.599 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1024 offset 4193792 len 2048 2024-03-08T21:15:11.599 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 1024 offset 4193792 len 2048 2024-03-08T21:15:11.600 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1024 offset 4193792 len 2048 2024-03-08T21:15:11.600 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 1024 offset 4193792 len 4096 2024-03-08T21:15:11.600 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1024 offset 4193792 len 4096 2024-03-08T21:15:11.600 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 1024 offset 4193792 len 4096 2024-03-08T21:15:11.600 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1024 offset 4193792 len 4096 2024-03-08T21:15:11.600 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 1024 offset 4193792 len 8192 2024-03-08T21:15:11.600 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1024 offset 4193792 len 8192 2024-03-08T21:15:11.600 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 1024 offset 4193792 len 8192 2024-03-08T21:15:11.600 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1024 offset 4193792 len 8192 2024-03-08T21:15:11.600 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 1024 offset 4193792 len 16384 2024-03-08T21:15:11.600 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1024 offset 4193792 len 16384 2024-03-08T21:15:11.600 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 1024 offset 4193792 len 16384 2024-03-08T21:15:11.600 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1024 offset 4193792 len 16384 2024-03-08T21:15:11.600 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 1024 offset 4194304 len 1024 2024-03-08T21:15:11.601 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1024 offset 4194304 len 1024 2024-03-08T21:15:11.601 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 1024 offset 4194304 len 1024 2024-03-08T21:15:11.601 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1024 offset 4194304 len 1024 2024-03-08T21:15:11.601 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 1024 offset 4194304 len 2048 2024-03-08T21:15:11.601 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1024 offset 4194304 len 2048 2024-03-08T21:15:11.601 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 1024 offset 4194304 len 2048 2024-03-08T21:15:11.601 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1024 offset 4194304 len 2048 2024-03-08T21:15:11.601 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 1024 offset 4194304 len 4096 2024-03-08T21:15:11.601 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1024 offset 4194304 len 4096 2024-03-08T21:15:11.601 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 1024 offset 4194304 len 4096 2024-03-08T21:15:11.601 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1024 offset 4194304 len 4096 2024-03-08T21:15:11.601 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 1024 offset 4194304 len 8192 2024-03-08T21:15:11.601 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1024 offset 4194304 len 8192 2024-03-08T21:15:11.601 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 1024 offset 4194304 len 8192 2024-03-08T21:15:11.601 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1024 offset 4194304 len 8192 2024-03-08T21:15:11.602 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 1024 offset 4194304 len 16384 2024-03-08T21:15:11.602 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1024 offset 4194304 len 16384 2024-03-08T21:15:11.602 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 1024 offset 4194304 len 16384 2024-03-08T21:15:11.602 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1024 offset 4194304 len 16384 2024-03-08T21:15:11.602 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 1024 offset 4194816 len 1024 2024-03-08T21:15:11.602 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1024 offset 4194816 len 1024 2024-03-08T21:15:11.602 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 1024 offset 4194816 len 1024 2024-03-08T21:15:11.602 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1024 offset 4194816 len 1024 2024-03-08T21:15:11.602 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 1024 offset 4194816 len 2048 2024-03-08T21:15:11.602 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1024 offset 4194816 len 2048 2024-03-08T21:15:11.602 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 1024 offset 4194816 len 2048 2024-03-08T21:15:11.602 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1024 offset 4194816 len 2048 2024-03-08T21:15:11.602 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 1024 offset 4194816 len 4096 2024-03-08T21:15:11.603 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1024 offset 4194816 len 4096 2024-03-08T21:15:11.758 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:15:11 smithi149 ceph-mon[35042]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T21:15:11.871 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:15:11 smithi204 ceph-mon[34935]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T21:15:11.961 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:15:11 smithi107 ceph-mon[31381]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T21:15:12.758 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:15:12 smithi149 ceph-mon[35042]: pgmap v465: 97 pgs: 97 active+clean; 22 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1003 KiB/s wr, 231 op/s 2024-03-08T21:15:12.759 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:15:12 smithi149 ceph-mon[35042]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T21:15:12.759 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:15:12 smithi149 ceph-mon[35042]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T21:15:12.759 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:15:12 smithi149 ceph-mon[35042]: from='mgr.15375 ' entity='mgr.x' 2024-03-08T21:15:12.806 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 1024 offset 4194816 len 4096 2024-03-08T21:15:12.806 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1024 offset 4194816 len 4096 2024-03-08T21:15:12.806 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 1024 offset 4194816 len 8192 2024-03-08T21:15:12.806 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1024 offset 4194816 len 8192 2024-03-08T21:15:12.806 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 1024 offset 4194816 len 8192 2024-03-08T21:15:12.807 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1024 offset 4194816 len 8192 2024-03-08T21:15:12.807 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 1024 offset 4194816 len 16384 2024-03-08T21:15:12.807 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1024 offset 4194816 len 16384 2024-03-08T21:15:12.807 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 1024 offset 4194816 len 16384 2024-03-08T21:15:12.807 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1024 offset 4194816 len 16384 2024-03-08T21:15:12.807 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 1024 offset 4195328 len 1024 2024-03-08T21:15:12.807 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1024 offset 4195328 len 1024 2024-03-08T21:15:12.807 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 1024 offset 4195328 len 1024 2024-03-08T21:15:12.807 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1024 offset 4195328 len 1024 2024-03-08T21:15:12.807 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 1024 offset 4195328 len 2048 2024-03-08T21:15:12.807 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1024 offset 4195328 len 2048 2024-03-08T21:15:12.807 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 1024 offset 4195328 len 2048 2024-03-08T21:15:12.807 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1024 offset 4195328 len 2048 2024-03-08T21:15:12.807 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 1024 offset 4195328 len 4096 2024-03-08T21:15:12.808 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1024 offset 4195328 len 4096 2024-03-08T21:15:12.808 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 1024 offset 4195328 len 4096 2024-03-08T21:15:12.808 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1024 offset 4195328 len 4096 2024-03-08T21:15:12.808 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 1024 offset 4195328 len 8192 2024-03-08T21:15:12.808 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1024 offset 4195328 len 8192 2024-03-08T21:15:12.808 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 1024 offset 4195328 len 8192 2024-03-08T21:15:12.808 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1024 offset 4195328 len 8192 2024-03-08T21:15:12.808 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 1024 offset 4195328 len 16384 2024-03-08T21:15:12.808 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1024 offset 4195328 len 16384 2024-03-08T21:15:12.808 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 1024 offset 4195328 len 16384 2024-03-08T21:15:12.808 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1024 offset 4195328 len 16384 2024-03-08T21:15:12.808 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 1024 offset 4195840 len 1024 2024-03-08T21:15:12.808 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1024 offset 4195840 len 1024 2024-03-08T21:15:12.808 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 1024 offset 4195840 len 1024 2024-03-08T21:15:12.809 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1024 offset 4195840 len 1024 2024-03-08T21:15:12.809 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 1024 offset 4195840 len 2048 2024-03-08T21:15:12.809 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1024 offset 4195840 len 2048 2024-03-08T21:15:12.809 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 1024 offset 4195840 len 2048 2024-03-08T21:15:12.809 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1024 offset 4195840 len 2048 2024-03-08T21:15:12.809 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 1024 offset 4195840 len 4096 2024-03-08T21:15:12.809 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1024 offset 4195840 len 4096 2024-03-08T21:15:12.809 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 1024 offset 4195840 len 4096 2024-03-08T21:15:12.809 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1024 offset 4195840 len 4096 2024-03-08T21:15:12.809 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 1024 offset 4195840 len 8192 2024-03-08T21:15:12.809 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1024 offset 4195840 len 8192 2024-03-08T21:15:12.809 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 1024 offset 4195840 len 8192 2024-03-08T21:15:12.809 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1024 offset 4195840 len 8192 2024-03-08T21:15:12.810 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 1024 offset 4195840 len 16384 2024-03-08T21:15:12.810 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1024 offset 4195840 len 16384 2024-03-08T21:15:12.810 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 1024 offset 4195840 len 16384 2024-03-08T21:15:12.810 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1024 offset 4195840 len 16384 2024-03-08T21:15:12.810 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 1024 offset 4196352 len 1024 2024-03-08T21:15:12.810 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1024 offset 4196352 len 1024 2024-03-08T21:15:12.810 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 1024 offset 4196352 len 1024 2024-03-08T21:15:12.810 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1024 offset 4196352 len 1024 2024-03-08T21:15:12.810 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 1024 offset 4196352 len 2048 2024-03-08T21:15:12.810 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1024 offset 4196352 len 2048 2024-03-08T21:15:12.810 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 1024 offset 4196352 len 2048 2024-03-08T21:15:12.810 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1024 offset 4196352 len 2048 2024-03-08T21:15:12.810 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 1024 offset 4196352 len 4096 2024-03-08T21:15:12.811 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1024 offset 4196352 len 4096 2024-03-08T21:15:12.811 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 1024 offset 4196352 len 4096 2024-03-08T21:15:12.811 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1024 offset 4196352 len 4096 2024-03-08T21:15:12.811 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 1024 offset 4196352 len 8192 2024-03-08T21:15:12.811 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1024 offset 4196352 len 8192 2024-03-08T21:15:12.811 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 1024 offset 4196352 len 8192 2024-03-08T21:15:12.811 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1024 offset 4196352 len 8192 2024-03-08T21:15:12.811 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 1024 offset 4196352 len 16384 2024-03-08T21:15:12.811 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1024 offset 4196352 len 16384 2024-03-08T21:15:12.811 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 1024 offset 4196352 len 16384 2024-03-08T21:15:12.811 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1024 offset 4196352 len 16384 2024-03-08T21:15:12.811 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 1024 offset 4196864 len 1024 2024-03-08T21:15:12.811 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1024 offset 4196864 len 1024 2024-03-08T21:15:12.812 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 1024 offset 4196864 len 1024 2024-03-08T21:15:12.812 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1024 offset 4196864 len 1024 2024-03-08T21:15:12.812 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 1024 offset 4196864 len 2048 2024-03-08T21:15:12.812 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1024 offset 4196864 len 2048 2024-03-08T21:15:12.812 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 1024 offset 4196864 len 2048 2024-03-08T21:15:12.812 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1024 offset 4196864 len 2048 2024-03-08T21:15:12.812 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 1024 offset 4196864 len 4096 2024-03-08T21:15:12.812 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1024 offset 4196864 len 4096 2024-03-08T21:15:12.812 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 1024 offset 4196864 len 4096 2024-03-08T21:15:12.872 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:15:12 smithi204 ceph-mon[34935]: pgmap v465: 97 pgs: 97 active+clean; 22 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1003 KiB/s wr, 231 op/s 2024-03-08T21:15:12.872 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:15:12 smithi204 ceph-mon[34935]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T21:15:12.872 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:15:12 smithi204 ceph-mon[34935]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T21:15:12.872 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:15:12 smithi204 ceph-mon[34935]: from='mgr.15375 ' entity='mgr.x' 2024-03-08T21:15:12.961 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:15:12 smithi107 ceph-mon[31381]: pgmap v465: 97 pgs: 97 active+clean; 22 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1003 KiB/s wr, 231 op/s 2024-03-08T21:15:12.961 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:15:12 smithi107 ceph-mon[31381]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T21:15:12.961 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:15:12 smithi107 ceph-mon[31381]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T21:15:12.961 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:15:12 smithi107 ceph-mon[31381]: from='mgr.15375 ' entity='mgr.x' 2024-03-08T21:15:13.950 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1024 offset 4196864 len 4096 2024-03-08T21:15:13.950 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 1024 offset 4196864 len 8192 2024-03-08T21:15:13.951 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1024 offset 4196864 len 8192 2024-03-08T21:15:13.951 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 1024 offset 4196864 len 8192 2024-03-08T21:15:13.951 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1024 offset 4196864 len 8192 2024-03-08T21:15:13.951 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 1024 offset 4196864 len 16384 2024-03-08T21:15:13.951 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1024 offset 4196864 len 16384 2024-03-08T21:15:13.951 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 1024 offset 4196864 len 16384 2024-03-08T21:15:13.951 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1024 offset 4196864 len 16384 2024-03-08T21:15:13.951 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 1024 offset 4197376 len 1024 2024-03-08T21:15:13.951 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1024 offset 4197376 len 1024 2024-03-08T21:15:13.951 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 1024 offset 4197376 len 1024 2024-03-08T21:15:13.951 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1024 offset 4197376 len 1024 2024-03-08T21:15:13.951 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 1024 offset 4197376 len 2048 2024-03-08T21:15:13.951 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1024 offset 4197376 len 2048 2024-03-08T21:15:13.951 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 1024 offset 4197376 len 2048 2024-03-08T21:15:13.952 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1024 offset 4197376 len 2048 2024-03-08T21:15:13.952 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 1024 offset 4197376 len 4096 2024-03-08T21:15:13.952 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1024 offset 4197376 len 4096 2024-03-08T21:15:13.952 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 1024 offset 4197376 len 4096 2024-03-08T21:15:13.952 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1024 offset 4197376 len 4096 2024-03-08T21:15:13.952 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 1024 offset 4197376 len 8192 2024-03-08T21:15:13.952 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1024 offset 4197376 len 8192 2024-03-08T21:15:13.952 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 1024 offset 4197376 len 8192 2024-03-08T21:15:13.952 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1024 offset 4197376 len 8192 2024-03-08T21:15:13.952 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 1024 offset 4197376 len 16384 2024-03-08T21:15:13.952 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1024 offset 4197376 len 16384 2024-03-08T21:15:13.952 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 1024 offset 4197376 len 16384 2024-03-08T21:15:13.952 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1024 offset 4197376 len 16384 2024-03-08T21:15:13.952 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 1024 offset 4197888 len 1024 2024-03-08T21:15:13.953 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1024 offset 4197888 len 1024 2024-03-08T21:15:13.953 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 1024 offset 4197888 len 1024 2024-03-08T21:15:13.953 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1024 offset 4197888 len 1024 2024-03-08T21:15:13.953 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 1024 offset 4197888 len 2048 2024-03-08T21:15:13.953 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1024 offset 4197888 len 2048 2024-03-08T21:15:13.953 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 1024 offset 4197888 len 2048 2024-03-08T21:15:13.953 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1024 offset 4197888 len 2048 2024-03-08T21:15:13.953 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 1024 offset 4197888 len 4096 2024-03-08T21:15:13.953 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1024 offset 4197888 len 4096 2024-03-08T21:15:13.953 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 1024 offset 4197888 len 4096 2024-03-08T21:15:13.953 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1024 offset 4197888 len 4096 2024-03-08T21:15:13.953 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 1024 offset 4197888 len 8192 2024-03-08T21:15:13.953 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1024 offset 4197888 len 8192 2024-03-08T21:15:13.953 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 1024 offset 4197888 len 8192 2024-03-08T21:15:13.953 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1024 offset 4197888 len 8192 2024-03-08T21:15:13.954 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 1024 offset 4197888 len 16384 2024-03-08T21:15:13.954 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1024 offset 4197888 len 16384 2024-03-08T21:15:13.954 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 1024 offset 4197888 len 16384 2024-03-08T21:15:13.954 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1024 offset 4197888 len 16384 2024-03-08T21:15:13.954 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 1536 offset 4190720 len 1024 2024-03-08T21:15:13.954 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1536 offset 4190720 len 1024 2024-03-08T21:15:13.954 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 1536 offset 4190720 len 1024 2024-03-08T21:15:13.954 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1536 offset 4190720 len 1024 2024-03-08T21:15:13.954 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 1536 offset 4190720 len 2048 2024-03-08T21:15:13.954 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1536 offset 4190720 len 2048 2024-03-08T21:15:13.954 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 1536 offset 4190720 len 2048 2024-03-08T21:15:13.954 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1536 offset 4190720 len 2048 2024-03-08T21:15:13.954 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 1536 offset 4190720 len 4096 2024-03-08T21:15:13.954 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1536 offset 4190720 len 4096 2024-03-08T21:15:13.955 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 1536 offset 4190720 len 4096 2024-03-08T21:15:13.955 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1536 offset 4190720 len 4096 2024-03-08T21:15:13.955 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 1536 offset 4190720 len 8192 2024-03-08T21:15:13.955 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1536 offset 4190720 len 8192 2024-03-08T21:15:13.955 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 1536 offset 4190720 len 8192 2024-03-08T21:15:13.955 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1536 offset 4190720 len 8192 2024-03-08T21:15:13.955 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 1536 offset 4190720 len 16384 2024-03-08T21:15:13.955 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1536 offset 4190720 len 16384 2024-03-08T21:15:13.955 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 1536 offset 4190720 len 16384 2024-03-08T21:15:13.955 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1536 offset 4190720 len 16384 2024-03-08T21:15:13.955 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 1536 offset 4191232 len 1024 2024-03-08T21:15:13.955 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1536 offset 4191232 len 1024 2024-03-08T21:15:13.955 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 1536 offset 4191232 len 1024 2024-03-08T21:15:13.955 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1536 offset 4191232 len 1024 2024-03-08T21:15:13.955 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 1536 offset 4191232 len 2048 2024-03-08T21:15:13.956 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1536 offset 4191232 len 2048 2024-03-08T21:15:13.956 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 1536 offset 4191232 len 2048 2024-03-08T21:15:13.956 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1536 offset 4191232 len 2048 2024-03-08T21:15:13.956 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 1536 offset 4191232 len 4096 2024-03-08T21:15:13.956 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1536 offset 4191232 len 4096 2024-03-08T21:15:13.956 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 1536 offset 4191232 len 4096 2024-03-08T21:15:13.956 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1536 offset 4191232 len 4096 2024-03-08T21:15:14.758 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:15:14 smithi149 ceph-mon[35042]: pgmap v466: 97 pgs: 97 active+clean; 22 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1003 KiB/s wr, 231 op/s 2024-03-08T21:15:14.758 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:15:14 smithi149 ceph-mon[35042]: osdmap e113: 12 total, 12 up, 12 in 2024-03-08T21:15:14.872 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:15:14 smithi204 ceph-mon[34935]: pgmap v466: 97 pgs: 97 active+clean; 22 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1003 KiB/s wr, 231 op/s 2024-03-08T21:15:14.872 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:15:14 smithi204 ceph-mon[34935]: osdmap e113: 12 total, 12 up, 12 in 2024-03-08T21:15:14.961 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:15:14 smithi107 ceph-mon[31381]: pgmap v466: 97 pgs: 97 active+clean; 22 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1003 KiB/s wr, 231 op/s 2024-03-08T21:15:14.962 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:15:14 smithi107 ceph-mon[31381]: osdmap e113: 12 total, 12 up, 12 in 2024-03-08T21:15:15.071 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 1536 offset 4191232 len 8192 2024-03-08T21:15:15.071 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1536 offset 4191232 len 8192 2024-03-08T21:15:15.071 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 1536 offset 4191232 len 8192 2024-03-08T21:15:15.071 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1536 offset 4191232 len 8192 2024-03-08T21:15:15.071 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 1536 offset 4191232 len 16384 2024-03-08T21:15:15.071 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1536 offset 4191232 len 16384 2024-03-08T21:15:15.072 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 1536 offset 4191232 len 16384 2024-03-08T21:15:15.072 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1536 offset 4191232 len 16384 2024-03-08T21:15:15.072 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 1536 offset 4191744 len 1024 2024-03-08T21:15:15.072 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1536 offset 4191744 len 1024 2024-03-08T21:15:15.072 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 1536 offset 4191744 len 1024 2024-03-08T21:15:15.072 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1536 offset 4191744 len 1024 2024-03-08T21:15:15.072 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 1536 offset 4191744 len 2048 2024-03-08T21:15:15.072 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1536 offset 4191744 len 2048 2024-03-08T21:15:15.072 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 1536 offset 4191744 len 2048 2024-03-08T21:15:15.072 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1536 offset 4191744 len 2048 2024-03-08T21:15:15.072 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 1536 offset 4191744 len 4096 2024-03-08T21:15:15.072 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1536 offset 4191744 len 4096 2024-03-08T21:15:15.072 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 1536 offset 4191744 len 4096 2024-03-08T21:15:15.072 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1536 offset 4191744 len 4096 2024-03-08T21:15:15.073 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 1536 offset 4191744 len 8192 2024-03-08T21:15:15.073 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1536 offset 4191744 len 8192 2024-03-08T21:15:15.073 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 1536 offset 4191744 len 8192 2024-03-08T21:15:15.073 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1536 offset 4191744 len 8192 2024-03-08T21:15:15.073 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 1536 offset 4191744 len 16384 2024-03-08T21:15:15.073 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1536 offset 4191744 len 16384 2024-03-08T21:15:15.073 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 1536 offset 4191744 len 16384 2024-03-08T21:15:15.073 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1536 offset 4191744 len 16384 2024-03-08T21:15:15.073 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 1536 offset 4192256 len 1024 2024-03-08T21:15:15.073 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1536 offset 4192256 len 1024 2024-03-08T21:15:15.073 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 1536 offset 4192256 len 1024 2024-03-08T21:15:15.073 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1536 offset 4192256 len 1024 2024-03-08T21:15:15.073 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 1536 offset 4192256 len 2048 2024-03-08T21:15:15.073 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1536 offset 4192256 len 2048 2024-03-08T21:15:15.074 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 1536 offset 4192256 len 2048 2024-03-08T21:15:15.074 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1536 offset 4192256 len 2048 2024-03-08T21:15:15.074 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 1536 offset 4192256 len 4096 2024-03-08T21:15:15.074 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1536 offset 4192256 len 4096 2024-03-08T21:15:15.074 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 1536 offset 4192256 len 4096 2024-03-08T21:15:15.074 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1536 offset 4192256 len 4096 2024-03-08T21:15:15.074 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 1536 offset 4192256 len 8192 2024-03-08T21:15:15.074 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1536 offset 4192256 len 8192 2024-03-08T21:15:15.074 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 1536 offset 4192256 len 8192 2024-03-08T21:15:15.074 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1536 offset 4192256 len 8192 2024-03-08T21:15:15.074 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 1536 offset 4192256 len 16384 2024-03-08T21:15:15.074 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1536 offset 4192256 len 16384 2024-03-08T21:15:15.074 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 1536 offset 4192256 len 16384 2024-03-08T21:15:15.075 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1536 offset 4192256 len 16384 2024-03-08T21:15:15.075 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 1536 offset 4192768 len 1024 2024-03-08T21:15:15.075 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1536 offset 4192768 len 1024 2024-03-08T21:15:15.075 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 1536 offset 4192768 len 1024 2024-03-08T21:15:15.075 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1536 offset 4192768 len 1024 2024-03-08T21:15:15.075 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 1536 offset 4192768 len 2048 2024-03-08T21:15:15.075 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1536 offset 4192768 len 2048 2024-03-08T21:15:15.075 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 1536 offset 4192768 len 2048 2024-03-08T21:15:15.075 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1536 offset 4192768 len 2048 2024-03-08T21:15:15.075 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 1536 offset 4192768 len 4096 2024-03-08T21:15:15.075 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1536 offset 4192768 len 4096 2024-03-08T21:15:15.075 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 1536 offset 4192768 len 4096 2024-03-08T21:15:15.075 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1536 offset 4192768 len 4096 2024-03-08T21:15:15.075 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 1536 offset 4192768 len 8192 2024-03-08T21:15:15.076 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1536 offset 4192768 len 8192 2024-03-08T21:15:15.076 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 1536 offset 4192768 len 8192 2024-03-08T21:15:15.076 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1536 offset 4192768 len 8192 2024-03-08T21:15:15.076 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 1536 offset 4192768 len 16384 2024-03-08T21:15:15.076 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1536 offset 4192768 len 16384 2024-03-08T21:15:15.076 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 1536 offset 4192768 len 16384 2024-03-08T21:15:15.076 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1536 offset 4192768 len 16384 2024-03-08T21:15:15.076 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 1536 offset 4193280 len 1024 2024-03-08T21:15:15.076 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1536 offset 4193280 len 1024 2024-03-08T21:15:15.076 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 1536 offset 4193280 len 1024 2024-03-08T21:15:15.076 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1536 offset 4193280 len 1024 2024-03-08T21:15:15.076 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 1536 offset 4193280 len 2048 2024-03-08T21:15:15.076 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1536 offset 4193280 len 2048 2024-03-08T21:15:15.076 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 1536 offset 4193280 len 2048 2024-03-08T21:15:15.076 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1536 offset 4193280 len 2048 2024-03-08T21:15:15.077 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 1536 offset 4193280 len 4096 2024-03-08T21:15:15.077 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1536 offset 4193280 len 4096 2024-03-08T21:15:15.077 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 1536 offset 4193280 len 4096 2024-03-08T21:15:15.077 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1536 offset 4193280 len 4096 2024-03-08T21:15:15.077 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 1536 offset 4193280 len 8192 2024-03-08T21:15:16.269 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1536 offset 4193280 len 8192 2024-03-08T21:15:16.269 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 1536 offset 4193280 len 8192 2024-03-08T21:15:16.269 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1536 offset 4193280 len 8192 2024-03-08T21:15:16.269 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 1536 offset 4193280 len 16384 2024-03-08T21:15:16.269 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1536 offset 4193280 len 16384 2024-03-08T21:15:16.269 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 1536 offset 4193280 len 16384 2024-03-08T21:15:16.269 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1536 offset 4193280 len 16384 2024-03-08T21:15:16.269 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 1536 offset 4193792 len 1024 2024-03-08T21:15:16.269 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1536 offset 4193792 len 1024 2024-03-08T21:15:16.269 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 1536 offset 4193792 len 1024 2024-03-08T21:15:16.270 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1536 offset 4193792 len 1024 2024-03-08T21:15:16.270 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 1536 offset 4193792 len 2048 2024-03-08T21:15:16.270 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1536 offset 4193792 len 2048 2024-03-08T21:15:16.270 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 1536 offset 4193792 len 2048 2024-03-08T21:15:16.270 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1536 offset 4193792 len 2048 2024-03-08T21:15:16.270 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 1536 offset 4193792 len 4096 2024-03-08T21:15:16.270 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1536 offset 4193792 len 4096 2024-03-08T21:15:16.270 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 1536 offset 4193792 len 4096 2024-03-08T21:15:16.270 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1536 offset 4193792 len 4096 2024-03-08T21:15:16.270 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 1536 offset 4193792 len 8192 2024-03-08T21:15:16.270 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1536 offset 4193792 len 8192 2024-03-08T21:15:16.270 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 1536 offset 4193792 len 8192 2024-03-08T21:15:16.271 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1536 offset 4193792 len 8192 2024-03-08T21:15:16.271 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 1536 offset 4193792 len 16384 2024-03-08T21:15:16.271 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1536 offset 4193792 len 16384 2024-03-08T21:15:16.271 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 1536 offset 4193792 len 16384 2024-03-08T21:15:16.271 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1536 offset 4193792 len 16384 2024-03-08T21:15:16.271 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 1536 offset 4194304 len 1024 2024-03-08T21:15:16.271 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1536 offset 4194304 len 1024 2024-03-08T21:15:16.271 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 1536 offset 4194304 len 1024 2024-03-08T21:15:16.271 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1536 offset 4194304 len 1024 2024-03-08T21:15:16.271 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 1536 offset 4194304 len 2048 2024-03-08T21:15:16.271 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1536 offset 4194304 len 2048 2024-03-08T21:15:16.271 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 1536 offset 4194304 len 2048 2024-03-08T21:15:16.271 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1536 offset 4194304 len 2048 2024-03-08T21:15:16.271 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 1536 offset 4194304 len 4096 2024-03-08T21:15:16.272 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1536 offset 4194304 len 4096 2024-03-08T21:15:16.272 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 1536 offset 4194304 len 4096 2024-03-08T21:15:16.272 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1536 offset 4194304 len 4096 2024-03-08T21:15:16.272 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 1536 offset 4194304 len 8192 2024-03-08T21:15:16.272 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1536 offset 4194304 len 8192 2024-03-08T21:15:16.272 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 1536 offset 4194304 len 8192 2024-03-08T21:15:16.272 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1536 offset 4194304 len 8192 2024-03-08T21:15:16.272 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 1536 offset 4194304 len 16384 2024-03-08T21:15:16.272 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1536 offset 4194304 len 16384 2024-03-08T21:15:16.272 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 1536 offset 4194304 len 16384 2024-03-08T21:15:16.272 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1536 offset 4194304 len 16384 2024-03-08T21:15:16.272 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 1536 offset 4194816 len 1024 2024-03-08T21:15:16.272 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1536 offset 4194816 len 1024 2024-03-08T21:15:16.272 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 1536 offset 4194816 len 1024 2024-03-08T21:15:16.273 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1536 offset 4194816 len 1024 2024-03-08T21:15:16.273 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 1536 offset 4194816 len 2048 2024-03-08T21:15:16.273 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1536 offset 4194816 len 2048 2024-03-08T21:15:16.273 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 1536 offset 4194816 len 2048 2024-03-08T21:15:16.273 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1536 offset 4194816 len 2048 2024-03-08T21:15:16.273 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 1536 offset 4194816 len 4096 2024-03-08T21:15:16.273 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1536 offset 4194816 len 4096 2024-03-08T21:15:16.273 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 1536 offset 4194816 len 4096 2024-03-08T21:15:16.273 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1536 offset 4194816 len 4096 2024-03-08T21:15:16.273 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 1536 offset 4194816 len 8192 2024-03-08T21:15:16.273 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1536 offset 4194816 len 8192 2024-03-08T21:15:16.273 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 1536 offset 4194816 len 8192 2024-03-08T21:15:16.273 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1536 offset 4194816 len 8192 2024-03-08T21:15:16.273 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 1536 offset 4194816 len 16384 2024-03-08T21:15:16.274 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1536 offset 4194816 len 16384 2024-03-08T21:15:16.274 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 1536 offset 4194816 len 16384 2024-03-08T21:15:16.274 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1536 offset 4194816 len 16384 2024-03-08T21:15:16.274 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 1536 offset 4195328 len 1024 2024-03-08T21:15:16.274 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1536 offset 4195328 len 1024 2024-03-08T21:15:16.274 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 1536 offset 4195328 len 1024 2024-03-08T21:15:16.274 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1536 offset 4195328 len 1024 2024-03-08T21:15:16.274 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 1536 offset 4195328 len 2048 2024-03-08T21:15:16.274 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1536 offset 4195328 len 2048 2024-03-08T21:15:16.274 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 1536 offset 4195328 len 2048 2024-03-08T21:15:16.274 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1536 offset 4195328 len 2048 2024-03-08T21:15:16.274 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 1536 offset 4195328 len 4096 2024-03-08T21:15:16.274 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1536 offset 4195328 len 4096 2024-03-08T21:15:16.274 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 1536 offset 4195328 len 4096 2024-03-08T21:15:16.274 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1536 offset 4195328 len 4096 2024-03-08T21:15:16.275 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 1536 offset 4195328 len 8192 2024-03-08T21:15:16.322 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.345+0000 7fdd91400640 1 -- 172.21.15.107:0/1420601169 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdd8c100070 msgr2=0x7fdd8c100450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:15:16.322 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.345+0000 7fdd91400640 1 --2- 172.21.15.107:0/1420601169 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdd8c100070 0x7fdd8c100450 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7fdd7c009930 tx=0x7fdd7c02bc20 comp rx=0 tx=0).stop 2024-03-08T21:15:16.322 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.345+0000 7fdd91400640 1 -- 172.21.15.107:0/1420601169 shutdown_connections 2024-03-08T21:15:16.322 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.345+0000 7fdd91400640 1 --2- 172.21.15.107:0/1420601169 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdd8c105580 0x7fdd8c105a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:15:16.322 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.345+0000 7fdd91400640 1 --2- 172.21.15.107:0/1420601169 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdd8c100990 0x7fdd8c104e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:15:16.322 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.345+0000 7fdd91400640 1 --2- 172.21.15.107:0/1420601169 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdd8c100070 0x7fdd8c100450 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:15:16.322 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.345+0000 7fdd91400640 1 -- 172.21.15.107:0/1420601169 >> 172.21.15.107:0/1420601169 conn(0x7fdd8c0fbac0 msgr2=0x7fdd8c0fdee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:15:16.322 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.345+0000 7fdd91400640 1 -- 172.21.15.107:0/1420601169 shutdown_connections 2024-03-08T21:15:16.322 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.345+0000 7fdd91400640 1 -- 172.21.15.107:0/1420601169 wait complete. 2024-03-08T21:15:16.324 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.346+0000 7fdd91400640 1 Processor -- start 2024-03-08T21:15:16.324 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.347+0000 7fdd91400640 1 -- start start 2024-03-08T21:15:16.324 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.347+0000 7fdd91400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdd8c100070 0x7fdd8c19dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:15:16.324 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.347+0000 7fdd91400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdd8c100990 0x7fdd8c19e510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:15:16.325 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.347+0000 7fdd91400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdd8c105580 0x7fdd8c1a2890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:15:16.325 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.347+0000 7fdd91400640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd8c1179d0 con 0x7fdd8c100070 2024-03-08T21:15:16.325 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.347+0000 7fdd91400640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd8c117850 con 0x7fdd8c105580 2024-03-08T21:15:16.325 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.347+0000 7fdd91400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd8c117b50 con 0x7fdd8c100990 2024-03-08T21:15:16.325 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.347+0000 7fdd8be00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdd8c100070 0x7fdd8c19dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:15:16.325 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.347+0000 7fdd8be00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdd8c100070 0x7fdd8c19dfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:33788/0 (socket says 172.21.15.107:33788) 2024-03-08T21:15:16.325 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.347+0000 7fdd8be00640 1 -- 172.21.15.107:0/2428440482 learned_addr learned my addr 172.21.15.107:0/2428440482 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:15:16.325 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.348+0000 7fdd8b400640 1 --2- 172.21.15.107:0/2428440482 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdd8c100990 0x7fdd8c19e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:15:16.325 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.348+0000 7fdd8be00640 1 -- 172.21.15.107:0/2428440482 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdd8c100990 msgr2=0x7fdd8c19e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:15:16.325 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.348+0000 7fdd8be00640 1 --2- 172.21.15.107:0/2428440482 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdd8c100990 0x7fdd8c19e510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:15:16.325 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.348+0000 7fdd8be00640 1 -- 172.21.15.107:0/2428440482 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdd8c105580 msgr2=0x7fdd8c1a2890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:15:16.325 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.348+0000 7fdd8be00640 1 --2- 172.21.15.107:0/2428440482 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdd8c105580 0x7fdd8c1a2890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:15:16.325 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.348+0000 7fdd8be00640 1 -- 172.21.15.107:0/2428440482 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd8c1a2fc0 con 0x7fdd8c100070 2024-03-08T21:15:16.325 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.348+0000 7fdd8be00640 1 --2- 172.21.15.107:0/2428440482 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdd8c100070 0x7fdd8c19dfd0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7fdd7c005ec0 tx=0x7fdd7c02ca80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:15:16.326 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.348+0000 7fdd88c00640 1 -- 172.21.15.107:0/2428440482 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd7c043020 con 0x7fdd8c100070 2024-03-08T21:15:16.326 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.348+0000 7fdd88c00640 1 -- 172.21.15.107:0/2428440482 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fdd7c02cc70 con 0x7fdd8c100070 2024-03-08T21:15:16.326 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.348+0000 7fdd88c00640 1 -- 172.21.15.107:0/2428440482 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd7c003c90 con 0x7fdd8c100070 2024-03-08T21:15:16.326 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.348+0000 7fdd91400640 1 -- 172.21.15.107:0/2428440482 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd8c1a3250 con 0x7fdd8c100070 2024-03-08T21:15:16.326 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.348+0000 7fdd91400640 1 -- 172.21.15.107:0/2428440482 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd8c1aab40 con 0x7fdd8c100070 2024-03-08T21:15:16.328 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.352+0000 7fdd88c00640 1 -- 172.21.15.107:0/2428440482 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) v1 ==== 137422+0+0 (secure 0 0 0) 0x7fdd7c046020 con 0x7fdd8c100070 2024-03-08T21:15:16.328 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.352+0000 7fdd9564f740 1 Processor -- start 2024-03-08T21:15:16.328 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.352+0000 7fdd9564f740 1 -- start start 2024-03-08T21:15:16.329 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.352+0000 7fdd9564f740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564280e24e80 0x564280e25260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:15:16.329 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.352+0000 7fdd9564f740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564280e257a0 0x564280e2e030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:15:16.329 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.352+0000 7fdd9564f740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564280e2e570 0x564280e30960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:15:16.329 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.352+0000 7fdd9564f740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x564280d883d0 con 0x564280e257a0 2024-03-08T21:15:16.329 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.352+0000 7fdd9564f740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x564280d316b0 con 0x564280e2e570 2024-03-08T21:15:16.329 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.352+0000 7fdd9564f740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x564280d84470 con 0x564280e24e80 2024-03-08T21:15:16.329 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.353+0000 7fdd8be00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564280e257a0 0x564280e2e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:15:16.330 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.353+0000 7fdd8be00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564280e257a0 0x564280e2e030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:33796/0 (socket says 172.21.15.107:33796) 2024-03-08T21:15:16.330 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.353+0000 7fdd8be00640 1 -- 172.21.15.107:0/3743225529 learned_addr learned my addr 172.21.15.107:0/3743225529 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:15:16.330 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.353+0000 7fdd8b400640 1 --2- 172.21.15.107:0/3743225529 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564280e2e570 0x564280e30960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:15:16.330 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.353+0000 7fdd90a00640 1 --2- 172.21.15.107:0/3743225529 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564280e24e80 0x564280e25260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:15:16.331 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.353+0000 7fdd8be00640 1 -- 172.21.15.107:0/3743225529 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564280e24e80 msgr2=0x564280e25260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:15:16.331 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.353+0000 7fdd88c00640 1 --2- 172.21.15.107:0/2428440482 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7fdd700a29e0 0x7fdd700a4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:15:16.331 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.353+0000 7fdd8be00640 1 --2- 172.21.15.107:0/3743225529 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564280e24e80 0x564280e25260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:15:16.331 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.353+0000 7fdd8be00640 1 -- 172.21.15.107:0/3743225529 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564280e2e570 msgr2=0x564280e30960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:15:16.331 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.353+0000 7fdd8be00640 1 --2- 172.21.15.107:0/3743225529 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564280e2e570 0x564280e30960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:15:16.331 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.353+0000 7fdd8be00640 1 -- 172.21.15.107:0/3743225529 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564280d57b50 con 0x564280e257a0 2024-03-08T21:15:16.331 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.354+0000 7fdd90a00640 1 --2- 172.21.15.107:0/2428440482 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7fdd700a29e0 0x7fdd700a4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:15:16.331 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.354+0000 7fdd88c00640 1 -- 172.21.15.107:0/2428440482 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fdd7c0f0fa0 con 0x7fdd8c100070 2024-03-08T21:15:16.331 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.354+0000 7fdd8be00640 1 --2- 172.21.15.107:0/3743225529 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564280e257a0 0x564280e2e030 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7fdd7c0ee730 tx=0x7fdd7c0b9a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae3fc3b0ff94e417 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:15:16.331 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.354+0000 7fdd81200640 1 -- 172.21.15.107:0/3743225529 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd7c0bd210 con 0x564280e257a0 2024-03-08T21:15:16.332 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.354+0000 7fdd81200640 1 -- 172.21.15.107:0/3743225529 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fdd7c0035f0 con 0x564280e257a0 2024-03-08T21:15:16.332 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.354+0000 7fdd81200640 1 -- 172.21.15.107:0/3743225529 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd7c0a80d0 con 0x564280e257a0 2024-03-08T21:15:16.335 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.355+0000 7fdd9564f740 1 -- 172.21.15.107:0/3743225529 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564280e257a0 msgr2=0x564280e2e030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:15:16.335 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.355+0000 7fdd90a00640 1 --2- 172.21.15.107:0/2428440482 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7fdd700a29e0 0x7fdd700a4ea0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fdd84007cc0 tx=0x7fdd84007440 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:15:16.335 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.355+0000 7fdd9564f740 1 --2- 172.21.15.107:0/3743225529 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564280e257a0 0x564280e2e030 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7fdd7c0ee730 tx=0x7fdd7c0b9a30 comp rx=0 tx=0).stop 2024-03-08T21:15:16.335 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.355+0000 7fdd9564f740 1 -- 172.21.15.107:0/3743225529 shutdown_connections 2024-03-08T21:15:16.335 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.355+0000 7fdd9564f740 1 --2- 172.21.15.107:0/3743225529 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564280e2e570 0x564280e30960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:15:16.336 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.355+0000 7fdd9564f740 1 --2- 172.21.15.107:0/3743225529 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564280e257a0 0x564280e2e030 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:15:16.336 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.355+0000 7fdd9564f740 1 --2- 172.21.15.107:0/3743225529 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564280e24e80 0x564280e25260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:15:16.336 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.355+0000 7fdd9564f740 1 -- 172.21.15.107:0/3743225529 >> 172.21.15.107:0/3743225529 conn(0x564280e1f7a0 msgr2=0x564280e1fb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:15:16.336 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.356+0000 7fdd9564f740 1 -- 172.21.15.107:0/3743225529 shutdown_connections 2024-03-08T21:15:16.336 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.356+0000 7fdd9564f740 1 -- 172.21.15.107:0/3743225529 wait complete. 2024-03-08T21:15:16.336 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.356+0000 7fdd9564f740 1 -- start start 2024-03-08T21:15:16.336 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.356+0000 7fdd9564f740 1 Processor -- start 2024-03-08T21:15:16.336 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.357+0000 7fdd9564f740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564280e24e80 0x564280e2df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:15:16.336 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.357+0000 7fdd9564f740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564280e257a0 0x564280e2b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:15:16.336 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.357+0000 7fdd9564f740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564280e2e570 0x564280e2b840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:15:16.336 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.357+0000 7fdd9564f740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x564280e30f60 con 0x564280e257a0 2024-03-08T21:15:16.336 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.357+0000 7fdd9564f740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x564280d449f0 con 0x564280e24e80 2024-03-08T21:15:16.336 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.357+0000 7fdd9564f740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x564280e310e0 con 0x564280e2e570 2024-03-08T21:15:16.337 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.357+0000 7fdd9564f740 10 client.0 _ll_register_callbacks cb 0x564280a9ca00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T21:15:16.337 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.357+0000 7fdd8b400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564280e257a0 0x564280e2b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:15:16.337 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.357+0000 7fdd8b400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564280e257a0 0x564280e2b2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:33806/0 (socket says 172.21.15.107:33806) 2024-03-08T21:15:16.337 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.357+0000 7fdd8b400640 1 -- 172.21.15.107:0/4007843159 learned_addr learned my addr 172.21.15.107:0/4007843159 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:15:16.337 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.357+0000 7fdd90a00640 1 --2- 172.21.15.107:0/4007843159 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564280e2e570 0x564280e2b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:15:16.337 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.357+0000 7fdd8be00640 1 --2- 172.21.15.107:0/4007843159 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564280e24e80 0x564280e2df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:15:16.337 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.358+0000 7fdd8b400640 1 -- 172.21.15.107:0/4007843159 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564280e2e570 msgr2=0x564280e2b840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:15:16.337 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.358+0000 7fdd8b400640 1 --2- 172.21.15.107:0/4007843159 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564280e2e570 0x564280e2b840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:15:16.337 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.358+0000 7fdd8b400640 1 -- 172.21.15.107:0/4007843159 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564280e24e80 msgr2=0x564280e2df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:15:16.337 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.358+0000 7fdd8b400640 1 --2- 172.21.15.107:0/4007843159 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564280e24e80 0x564280e2df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:15:16.337 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.358+0000 7fdd8b400640 1 -- 172.21.15.107:0/4007843159 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564280a9d210 con 0x564280e257a0 2024-03-08T21:15:16.337 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.358+0000 7fdd8b400640 1 --2- 172.21.15.107:0/4007843159 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564280e257a0 0x564280e2b2a0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7fdd7800f4f0 tx=0x7fdd7800f9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b482cd871d1765f server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:15:16.337 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.358+0000 7fdd75600640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-03-08T21:15:16.337 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.358+0000 7fdd75600640 1 -- 172.21.15.107:0/4007843159 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd78013850 con 0x564280e257a0 2024-03-08T21:15:16.338 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.358+0000 7fdd75600640 1 -- 172.21.15.107:0/4007843159 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fdd780139f0 con 0x564280e257a0 2024-03-08T21:15:16.338 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.358+0000 7fdd9564f740 1 -- 172.21.15.107:0/4007843159 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564280d95910 con 0x564280e257a0 2024-03-08T21:15:16.338 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.358+0000 7fdd75600640 1 -- 172.21.15.107:0/4007843159 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdd78013c30 con 0x564280e257a0 2024-03-08T21:15:16.338 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.358+0000 7fdd9564f740 10 client.16683 fetch_fsmap learned FSMap version 41 2024-03-08T21:15:16.338 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.359+0000 7fdd9564f740 1 -- 172.21.15.107:0/4007843159 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x564280b69620 con 0x564280e257a0 2024-03-08T21:15:16.338 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.359+0000 7fdd75600640 1 -- 172.21.15.107:0/4007843159 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x564280d95910 con 0x564280e257a0 2024-03-08T21:15:16.338 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.359+0000 7fdd75600640 1 -- 172.21.15.107:0/4007843159 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 41) v1 ==== 7307+0+0 (secure 0 0 0) 0x7fdd78006600 con 0x564280e257a0 2024-03-08T21:15:16.338 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.359+0000 7fdd9564f740 10 client.16683 fetch_fsmap finished waiting for FSMap version 41 2024-03-08T21:15:16.338 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.359+0000 7fdd9564f740 10 client.16683 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-08T21:15:16.338 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.359+0000 7fdd9564f740 20 client.16683 populate_metadata read hostname 'smithi107' 2024-03-08T21:15:16.338 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.359+0000 7fdd9564f740 1 --2- 172.21.15.107:0/4007843159 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x564280e3e170 0x564280e405a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:15:16.338 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.359+0000 7fdd9564f740 4 client.16683 mds_command: new command op to 34475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-08T21:15:16.338 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.359+0000 7fdd9564f740 1 -- 172.21.15.107:0/4007843159 --> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564280d8fc60 con 0x564280e3e170 2024-03-08T21:15:16.340 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.360+0000 7fdd8be00640 1 --2- 172.21.15.107:0/4007843159 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x564280e3e170 0x564280e405a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:15:16.340 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.361+0000 7fdd8be00640 1 --2- 172.21.15.107:0/4007843159 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x564280e3e170 0x564280e405a0 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=37c2531a8eb351a7 server_cookie=fc3dde9292cb645e in_seq=0 out_seq=0 2024-03-08T21:15:16.340 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.361+0000 7fdd75600640 10 client.16683 ms_handle_connect on v2:172.21.15.149:6838/55290596 2024-03-08T21:15:16.340 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.363+0000 7fdd75600640 1 -- 172.21.15.107:0/4007843159 <== mds.0 v2:172.21.15.149:6838/55290596 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x564280d8fc60 con 0x564280e3e170 2024-03-08T21:15:16.340 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.363+0000 7fdd75600640 10 client.16683 handle_command_reply: tid=0 2024-03-08T21:15:16.341 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.363+0000 7fdd9564f740 1 client.16683 shutdown 2024-03-08T21:15:16.341 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.363+0000 7fdd9564f740 1 -- 172.21.15.107:0/4007843159 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564280e257a0 msgr2=0x564280e2b2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:15:16.341 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.363+0000 7fdd9564f740 1 --2- 172.21.15.107:0/4007843159 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564280e257a0 0x564280e2b2a0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7fdd7800f4f0 tx=0x7fdd7800f9b0 comp rx=0 tx=0).stop 2024-03-08T21:15:16.341 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.363+0000 7fdd9564f740 1 -- 172.21.15.107:0/4007843159 shutdown_connections 2024-03-08T21:15:16.341 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.363+0000 7fdd9564f740 1 --2- 172.21.15.107:0/4007843159 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564280e2e570 0x564280e2b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:15:16.341 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.363+0000 7fdd9564f740 1 --2- 172.21.15.107:0/4007843159 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564280e257a0 0x564280e2b2a0 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:15:16.341 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.363+0000 7fdd9564f740 1 --2- 172.21.15.107:0/4007843159 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x564280e3e170 0x564280e405a0 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:15:16.341 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.363+0000 7fdd9564f740 1 --2- 172.21.15.107:0/4007843159 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564280e24e80 0x564280e2df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:15:16.341 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.363+0000 7fdd75600640 0 client.16683 ms_handle_reset on v2:172.21.15.149:6838/55290596 2024-03-08T21:15:16.341 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.363+0000 7fdd9564f740 1 -- 172.21.15.107:0/4007843159 >> 172.21.15.107:0/4007843159 conn(0x564280e1f7a0 msgr2=0x564280e214f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:15:16.341 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.363+0000 7fdd9564f740 1 -- 172.21.15.107:0/4007843159 shutdown_connections 2024-03-08T21:15:16.341 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.363+0000 7fdd9564f740 1 -- 172.21.15.107:0/4007843159 wait complete. 2024-03-08T21:15:16.341 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.363+0000 7fdd9564f740 20 client.16683 trim_cache size 0 max 16384 2024-03-08T21:15:16.356 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.379+0000 7fdd9564f740 1 Processor -- start 2024-03-08T21:15:16.356 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.379+0000 7fdd9564f740 1 -- start start 2024-03-08T21:15:16.356 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.379+0000 7fdd9564f740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564280e24e80 0x564280e2d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:15:16.356 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.379+0000 7fdd9564f740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564280e257a0 0x564280e2df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:15:16.356 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.379+0000 7fdd9564f740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564280e2e570 0x564280e28500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:15:16.356 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.379+0000 7fdd9564f740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd8c1b4b30 con 0x564280e2e570 2024-03-08T21:15:16.357 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.379+0000 7fdd9564f740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd8c1b2b50 con 0x564280e24e80 2024-03-08T21:15:16.357 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.379+0000 7fdd9564f740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd8c1b44d0 con 0x564280e257a0 2024-03-08T21:15:16.357 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.379+0000 7fdd90a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564280e2e570 0x564280e28500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:15:16.357 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.380+0000 7fdd90a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564280e2e570 0x564280e28500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:33810/0 (socket says 172.21.15.107:33810) 2024-03-08T21:15:16.357 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.380+0000 7fdd90a00640 1 -- 172.21.15.107:0/2569547544 learned_addr learned my addr 172.21.15.107:0/2569547544 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:15:16.357 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.380+0000 7fdd8b400640 1 --2- 172.21.15.107:0/2569547544 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564280e257a0 0x564280e2df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:15:16.357 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.380+0000 7fdd8be00640 1 --2- 172.21.15.107:0/2569547544 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564280e24e80 0x564280e2d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:15:16.357 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.380+0000 7fdd90a00640 1 -- 172.21.15.107:0/2569547544 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564280e257a0 msgr2=0x564280e2df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:15:16.357 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.380+0000 7fdd90a00640 1 --2- 172.21.15.107:0/2569547544 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564280e257a0 0x564280e2df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:15:16.357 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.380+0000 7fdd90a00640 1 -- 172.21.15.107:0/2569547544 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564280e24e80 msgr2=0x564280e2d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:15:16.357 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.380+0000 7fdd90a00640 1 --2- 172.21.15.107:0/2569547544 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564280e24e80 0x564280e2d9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:15:16.357 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.380+0000 7fdd90a00640 1 -- 172.21.15.107:0/2569547544 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564280e255d0 con 0x564280e2e570 2024-03-08T21:15:16.357 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.380+0000 7fdd90a00640 1 --2- 172.21.15.107:0/2569547544 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564280e2e570 0x564280e28500 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7fdd840114f0 tx=0x7fdd84011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1872df6dd9c2639 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:15:16.361 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.381+0000 7fdd77e00640 1 -- 172.21.15.107:0/2569547544 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd84023070 con 0x564280e2e570 2024-03-08T21:15:16.362 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.381+0000 7fdd77e00640 1 -- 172.21.15.107:0/2569547544 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fdd8401fdb0 con 0x564280e2e570 2024-03-08T21:15:16.362 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.381+0000 7fdd77e00640 1 -- 172.21.15.107:0/2569547544 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd8401e3c0 con 0x564280e2e570 2024-03-08T21:15:16.362 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.381+0000 7fdd9564f740 1 -- 172.21.15.107:0/2569547544 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564280e2e570 msgr2=0x564280e28500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:15:16.362 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.381+0000 7fdd9564f740 1 --2- 172.21.15.107:0/2569547544 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564280e2e570 0x564280e28500 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7fdd840114f0 tx=0x7fdd84011000 comp rx=0 tx=0).stop 2024-03-08T21:15:16.362 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.381+0000 7fdd9564f740 1 -- 172.21.15.107:0/2569547544 shutdown_connections 2024-03-08T21:15:16.362 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.381+0000 7fdd9564f740 1 --2- 172.21.15.107:0/2569547544 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564280e2e570 0x564280e28500 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:15:16.362 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.381+0000 7fdd9564f740 1 --2- 172.21.15.107:0/2569547544 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564280e257a0 0x564280e2df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:15:16.362 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.381+0000 7fdd9564f740 1 --2- 172.21.15.107:0/2569547544 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564280e24e80 0x564280e2d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:15:16.362 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.381+0000 7fdd9564f740 1 -- 172.21.15.107:0/2569547544 >> 172.21.15.107:0/2569547544 conn(0x564280e1f7a0 msgr2=0x564280e1fb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:15:16.362 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.382+0000 7fdd9564f740 1 -- 172.21.15.107:0/2569547544 shutdown_connections 2024-03-08T21:15:16.363 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.382+0000 7fdd9564f740 1 -- 172.21.15.107:0/2569547544 wait complete. 2024-03-08T21:15:16.363 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.382+0000 7fdd9564f740 1 -- start start 2024-03-08T21:15:16.363 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.382+0000 7fdd9564f740 1 Processor -- start 2024-03-08T21:15:16.363 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.383+0000 7fdd9564f740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564280e24e80 0x564280e35510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:15:16.363 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.383+0000 7fdd9564f740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564280e257a0 0x564280e33900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:15:16.363 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.383+0000 7fdd9564f740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564280e2e570 0x564280e33e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:15:16.363 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.383+0000 7fdd9564f740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd8c1b3810 con 0x564280e24e80 2024-03-08T21:15:16.363 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.383+0000 7fdd9564f740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd8c1b31b0 con 0x564280e2e570 2024-03-08T21:15:16.363 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.383+0000 7fdd9564f740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd8c1b3e70 con 0x564280e257a0 2024-03-08T21:15:16.363 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.383+0000 7fdd9564f740 10 client.0 _ll_register_callbacks cb 0x7fdd7c0f4aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T21:15:16.363 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.383+0000 7fdd8be00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564280e24e80 0x564280e35510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:15:16.363 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.383+0000 7fdd8be00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564280e24e80 0x564280e35510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:33820/0 (socket says 172.21.15.107:33820) 2024-03-08T21:15:16.363 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.383+0000 7fdd8be00640 1 -- 172.21.15.107:0/2938542922 learned_addr learned my addr 172.21.15.107:0/2938542922 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:15:16.363 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.383+0000 7fdd90a00640 1 --2- 172.21.15.107:0/2938542922 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564280e2e570 0x564280e33e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:15:16.364 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.383+0000 7fdd8b400640 1 --2- 172.21.15.107:0/2938542922 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564280e257a0 0x564280e33900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:15:16.364 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.383+0000 7fdd8be00640 1 -- 172.21.15.107:0/2938542922 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564280e257a0 msgr2=0x564280e33900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:15:16.364 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.383+0000 7fdd8be00640 1 --2- 172.21.15.107:0/2938542922 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564280e257a0 0x564280e33900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:15:16.364 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.383+0000 7fdd8be00640 1 -- 172.21.15.107:0/2938542922 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564280e2e570 msgr2=0x564280e33e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:15:16.364 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.383+0000 7fdd8be00640 1 --2- 172.21.15.107:0/2938542922 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564280e2e570 0x564280e33e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:15:16.364 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.383+0000 7fdd8be00640 1 -- 172.21.15.107:0/2938542922 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564280e345b0 con 0x564280e24e80 2024-03-08T21:15:16.364 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.383+0000 7fdd8b400640 1 --2- 172.21.15.107:0/2938542922 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564280e257a0 0x564280e33900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T21:15:16.364 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.384+0000 7fdd8be00640 1 --2- 172.21.15.107:0/2938542922 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564280e24e80 0x564280e35510 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7fdd7c006010 tx=0x7fdd7c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a22e429d8d788d93 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:15:16.364 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.384+0000 7fdd75600640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-03-08T21:15:16.364 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.384+0000 7fdd75600640 1 -- 172.21.15.107:0/2938542922 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd7c0bd030 con 0x564280e24e80 2024-03-08T21:15:16.364 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.384+0000 7fdd75600640 1 -- 172.21.15.107:0/2938542922 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fdd7c0dd040 con 0x564280e24e80 2024-03-08T21:15:16.364 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.384+0000 7fdd9564f740 1 -- 172.21.15.107:0/2938542922 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdd8401fdb0 con 0x564280e24e80 2024-03-08T21:15:16.364 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.384+0000 7fdd75600640 1 -- 172.21.15.107:0/2938542922 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdd7c0a80d0 con 0x564280e24e80 2024-03-08T21:15:16.364 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.384+0000 7fdd9564f740 10 client.16689 fetch_fsmap learned FSMap version 41 2024-03-08T21:15:16.365 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.384+0000 7fdd9564f740 1 -- 172.21.15.107:0/2938542922 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x564280e95680 con 0x564280e24e80 2024-03-08T21:15:16.365 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.385+0000 7fdd75600640 1 -- 172.21.15.107:0/2938542922 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fdd8401fdb0 con 0x564280e24e80 2024-03-08T21:15:16.365 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.385+0000 7fdd75600640 1 -- 172.21.15.107:0/2938542922 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 41) v1 ==== 7307+0+0 (secure 0 0 0) 0x7fdd7c0b9d40 con 0x564280e24e80 2024-03-08T21:15:16.365 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.385+0000 7fdd9564f740 10 client.16689 fetch_fsmap finished waiting for FSMap version 41 2024-03-08T21:15:16.365 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.385+0000 7fdd9564f740 10 client.16689 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-08T21:15:16.365 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.385+0000 7fdd9564f740 20 client.16689 populate_metadata read hostname 'smithi107' 2024-03-08T21:15:16.365 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.385+0000 7fdd9564f740 1 --2- 172.21.15.107:0/2938542922 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x564280e95b50 0x564280e97f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:15:16.365 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.385+0000 7fdd9564f740 4 client.16689 mds_command: new command op to 34475 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-08T21:15:16.365 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.385+0000 7fdd9564f740 1 -- 172.21.15.107:0/2938542922 --> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x564280d0e5d0 con 0x564280e95b50 2024-03-08T21:15:16.365 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.386+0000 7fdd8b400640 1 --2- 172.21.15.107:0/2938542922 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x564280e95b50 0x564280e97f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:15:16.365 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.387+0000 7fdd8b400640 1 --2- 172.21.15.107:0/2938542922 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x564280e95b50 0x564280e97f40 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6fee3d5344412fb2 server_cookie=f51d647b64ee0bc3 in_seq=0 out_seq=0 2024-03-08T21:15:16.366 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.387+0000 7fdd75600640 10 client.16689 ms_handle_connect on v2:172.21.15.149:6838/55290596 2024-03-08T21:15:16.437 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.460+0000 7fdd75600640 1 -- 172.21.15.107:0/2938542922 <== mds.0 v2:172.21.15.149:6838/55290596 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x564280d0e5d0 con 0x564280e95b50 2024-03-08T21:15:16.437 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.460+0000 7fdd75600640 10 client.16689 handle_command_reply: tid=0 2024-03-08T21:15:16.437 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.460+0000 7fdd9564f740 1 client.16689 shutdown 2024-03-08T21:15:16.437 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.460+0000 7fdd9564f740 1 -- 172.21.15.107:0/2938542922 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564280e24e80 msgr2=0x564280e35510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:15:16.437 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.460+0000 7fdd9564f740 1 --2- 172.21.15.107:0/2938542922 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564280e24e80 0x564280e35510 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7fdd7c006010 tx=0x7fdd7c000bc0 comp rx=0 tx=0).stop 2024-03-08T21:15:16.437 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.460+0000 7fdd9564f740 1 -- 172.21.15.107:0/2938542922 shutdown_connections 2024-03-08T21:15:16.437 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.460+0000 7fdd9564f740 1 --2- 172.21.15.107:0/2938542922 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x564280e95b50 0x564280e97f40 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:15:16.437 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.460+0000 7fdd9564f740 1 --2- 172.21.15.107:0/2938542922 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x564280e2e570 0x564280e33e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:15:16.437 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.460+0000 7fdd9564f740 1 --2- 172.21.15.107:0/2938542922 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x564280e257a0 0x564280e33900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:15:16.438 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.460+0000 7fdd9564f740 1 --2- 172.21.15.107:0/2938542922 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564280e24e80 0x564280e35510 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:15:16.438 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.460+0000 7fdd9564f740 1 -- 172.21.15.107:0/2938542922 >> 172.21.15.107:0/2938542922 conn(0x564280e1f7a0 msgr2=0x564280e29fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:15:16.438 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.460+0000 7fdd75600640 0 client.16689 ms_handle_reset on v2:172.21.15.149:6838/55290596 2024-03-08T21:15:16.438 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.460+0000 7fdd9564f740 1 -- 172.21.15.107:0/2938542922 shutdown_connections 2024-03-08T21:15:16.438 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.460+0000 7fdd9564f740 1 -- 172.21.15.107:0/2938542922 wait complete. 2024-03-08T21:15:16.438 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.461+0000 7fdd9564f740 20 client.16689 trim_cache size 0 max 16384 2024-03-08T21:15:16.438 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:{ 2024-03-08T21:15:16.438 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "message": "", 2024-03-08T21:15:16.438 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "return_code": 0 2024-03-08T21:15:16.439 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:} 2024-03-08T21:15:16.439 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.461+0000 7fdd91400640 1 -- 172.21.15.107:0/2428440482 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7fdd700a29e0 msgr2=0x7fdd700a4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:15:16.439 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.461+0000 7fdd91400640 1 --2- 172.21.15.107:0/2428440482 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7fdd700a29e0 0x7fdd700a4ea0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fdd84007cc0 tx=0x7fdd84007440 comp rx=0 tx=0).stop 2024-03-08T21:15:16.440 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.461+0000 7fdd91400640 1 -- 172.21.15.107:0/2428440482 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdd8c100070 msgr2=0x7fdd8c19dfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:15:16.440 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.461+0000 7fdd91400640 1 --2- 172.21.15.107:0/2428440482 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdd8c100070 0x7fdd8c19dfd0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7fdd7c005ec0 tx=0x7fdd7c02ca80 comp rx=0 tx=0).stop 2024-03-08T21:15:16.440 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.462+0000 7fdd91400640 1 -- 172.21.15.107:0/2428440482 shutdown_connections 2024-03-08T21:15:16.440 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.462+0000 7fdd91400640 1 --2- 172.21.15.107:0/2428440482 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7fdd700a29e0 0x7fdd700a4ea0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:15:16.440 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.462+0000 7fdd91400640 1 --2- 172.21.15.107:0/2428440482 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdd8c105580 0x7fdd8c1a2890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:15:16.440 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.462+0000 7fdd91400640 1 --2- 172.21.15.107:0/2428440482 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fdd8c100990 0x7fdd8c19e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:15:16.440 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.462+0000 7fdd91400640 1 --2- 172.21.15.107:0/2428440482 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdd8c100070 0x7fdd8c19dfd0 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:15:16.440 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.462+0000 7fdd91400640 1 -- 172.21.15.107:0/2428440482 >> 172.21.15.107:0/2428440482 conn(0x7fdd8c0fbac0 msgr2=0x7fdd8c0fcb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:15:16.440 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.462+0000 7fdd91400640 1 -- 172.21.15.107:0/2428440482 shutdown_connections 2024-03-08T21:15:16.440 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:16.462+0000 7fdd91400640 1 -- 172.21.15.107:0/2428440482 wait complete. 2024-03-08T21:15:16.872 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:15:16 smithi204 ceph-mon[34935]: pgmap v468: 97 pgs: 97 active+clean; 43 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 185 op/s 2024-03-08T21:15:16.960 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:15:16 smithi107 ceph-mon[31381]: pgmap v468: 97 pgs: 97 active+clean; 43 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 185 op/s 2024-03-08T21:15:17.008 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:15:16 smithi149 ceph-mon[35042]: pgmap v468: 97 pgs: 97 active+clean; 43 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 185 op/s 2024-03-08T21:15:17.428 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1536 offset 4195328 len 8192 2024-03-08T21:15:17.429 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 1536 offset 4195328 len 8192 2024-03-08T21:15:17.429 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1536 offset 4195328 len 8192 2024-03-08T21:15:17.429 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 1536 offset 4195328 len 16384 2024-03-08T21:15:17.429 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1536 offset 4195328 len 16384 2024-03-08T21:15:17.429 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 1536 offset 4195328 len 16384 2024-03-08T21:15:17.429 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1536 offset 4195328 len 16384 2024-03-08T21:15:17.429 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 1536 offset 4195840 len 1024 2024-03-08T21:15:17.429 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1536 offset 4195840 len 1024 2024-03-08T21:15:17.429 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 1536 offset 4195840 len 1024 2024-03-08T21:15:17.429 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1536 offset 4195840 len 1024 2024-03-08T21:15:17.429 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 1536 offset 4195840 len 2048 2024-03-08T21:15:17.429 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1536 offset 4195840 len 2048 2024-03-08T21:15:17.430 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 1536 offset 4195840 len 2048 2024-03-08T21:15:17.430 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1536 offset 4195840 len 2048 2024-03-08T21:15:17.430 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 1536 offset 4195840 len 4096 2024-03-08T21:15:17.430 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1536 offset 4195840 len 4096 2024-03-08T21:15:17.430 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 1536 offset 4195840 len 4096 2024-03-08T21:15:17.430 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1536 offset 4195840 len 4096 2024-03-08T21:15:17.430 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 1536 offset 4195840 len 8192 2024-03-08T21:15:17.430 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1536 offset 4195840 len 8192 2024-03-08T21:15:17.430 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 1536 offset 4195840 len 8192 2024-03-08T21:15:17.430 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1536 offset 4195840 len 8192 2024-03-08T21:15:17.430 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 1536 offset 4195840 len 16384 2024-03-08T21:15:17.430 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1536 offset 4195840 len 16384 2024-03-08T21:15:17.430 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 1536 offset 4195840 len 16384 2024-03-08T21:15:17.430 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1536 offset 4195840 len 16384 2024-03-08T21:15:17.431 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 1536 offset 4196352 len 1024 2024-03-08T21:15:17.431 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1536 offset 4196352 len 1024 2024-03-08T21:15:17.431 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 1536 offset 4196352 len 1024 2024-03-08T21:15:17.431 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1536 offset 4196352 len 1024 2024-03-08T21:15:17.431 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 1536 offset 4196352 len 2048 2024-03-08T21:15:17.431 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1536 offset 4196352 len 2048 2024-03-08T21:15:17.431 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 1536 offset 4196352 len 2048 2024-03-08T21:15:17.431 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1536 offset 4196352 len 2048 2024-03-08T21:15:17.431 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 1536 offset 4196352 len 4096 2024-03-08T21:15:17.431 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1536 offset 4196352 len 4096 2024-03-08T21:15:17.431 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 1536 offset 4196352 len 4096 2024-03-08T21:15:17.431 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1536 offset 4196352 len 4096 2024-03-08T21:15:17.431 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 1536 offset 4196352 len 8192 2024-03-08T21:15:17.432 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1536 offset 4196352 len 8192 2024-03-08T21:15:17.432 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 1536 offset 4196352 len 8192 2024-03-08T21:15:17.432 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1536 offset 4196352 len 8192 2024-03-08T21:15:17.432 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 1536 offset 4196352 len 16384 2024-03-08T21:15:17.432 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1536 offset 4196352 len 16384 2024-03-08T21:15:17.432 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 1536 offset 4196352 len 16384 2024-03-08T21:15:17.432 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1536 offset 4196352 len 16384 2024-03-08T21:15:17.432 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 1536 offset 4196864 len 1024 2024-03-08T21:15:17.432 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1536 offset 4196864 len 1024 2024-03-08T21:15:17.432 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 1536 offset 4196864 len 1024 2024-03-08T21:15:17.432 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1536 offset 4196864 len 1024 2024-03-08T21:15:17.432 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 1536 offset 4196864 len 2048 2024-03-08T21:15:17.432 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1536 offset 4196864 len 2048 2024-03-08T21:15:17.432 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 1536 offset 4196864 len 2048 2024-03-08T21:15:17.432 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1536 offset 4196864 len 2048 2024-03-08T21:15:17.433 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 1536 offset 4196864 len 4096 2024-03-08T21:15:17.433 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1536 offset 4196864 len 4096 2024-03-08T21:15:17.433 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 1536 offset 4196864 len 4096 2024-03-08T21:15:17.433 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1536 offset 4196864 len 4096 2024-03-08T21:15:17.433 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 1536 offset 4196864 len 8192 2024-03-08T21:15:17.433 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1536 offset 4196864 len 8192 2024-03-08T21:15:17.433 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 1536 offset 4196864 len 8192 2024-03-08T21:15:17.433 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1536 offset 4196864 len 8192 2024-03-08T21:15:17.433 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 1536 offset 4196864 len 16384 2024-03-08T21:15:17.433 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1536 offset 4196864 len 16384 2024-03-08T21:15:17.433 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 1536 offset 4196864 len 16384 2024-03-08T21:15:17.433 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1536 offset 4196864 len 16384 2024-03-08T21:15:17.433 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 1536 offset 4197376 len 1024 2024-03-08T21:15:17.433 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1536 offset 4197376 len 1024 2024-03-08T21:15:17.434 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 1536 offset 4197376 len 1024 2024-03-08T21:15:17.434 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1536 offset 4197376 len 1024 2024-03-08T21:15:17.434 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 1536 offset 4197376 len 2048 2024-03-08T21:15:17.434 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1536 offset 4197376 len 2048 2024-03-08T21:15:17.434 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 1536 offset 4197376 len 2048 2024-03-08T21:15:17.434 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1536 offset 4197376 len 2048 2024-03-08T21:15:17.434 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 1536 offset 4197376 len 4096 2024-03-08T21:15:17.434 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1536 offset 4197376 len 4096 2024-03-08T21:15:17.434 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 1536 offset 4197376 len 4096 2024-03-08T21:15:17.434 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1536 offset 4197376 len 4096 2024-03-08T21:15:17.434 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 1536 offset 4197376 len 8192 2024-03-08T21:15:17.434 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1536 offset 4197376 len 8192 2024-03-08T21:15:18.511 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 1536 offset 4197376 len 8192 2024-03-08T21:15:18.511 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1536 offset 4197376 len 8192 2024-03-08T21:15:18.511 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 1536 offset 4197376 len 16384 2024-03-08T21:15:18.511 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1536 offset 4197376 len 16384 2024-03-08T21:15:18.511 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 1536 offset 4197376 len 16384 2024-03-08T21:15:18.511 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1536 offset 4197376 len 16384 2024-03-08T21:15:18.511 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 1536 offset 4197888 len 1024 2024-03-08T21:15:18.511 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1536 offset 4197888 len 1024 2024-03-08T21:15:18.511 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 1536 offset 4197888 len 1024 2024-03-08T21:15:18.511 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1536 offset 4197888 len 1024 2024-03-08T21:15:18.512 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 1536 offset 4197888 len 2048 2024-03-08T21:15:18.512 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1536 offset 4197888 len 2048 2024-03-08T21:15:18.512 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 1536 offset 4197888 len 2048 2024-03-08T21:15:18.512 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1536 offset 4197888 len 2048 2024-03-08T21:15:18.512 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 1536 offset 4197888 len 4096 2024-03-08T21:15:18.512 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1536 offset 4197888 len 4096 2024-03-08T21:15:18.512 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 1536 offset 4197888 len 4096 2024-03-08T21:15:18.512 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1536 offset 4197888 len 4096 2024-03-08T21:15:18.512 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 1536 offset 4197888 len 8192 2024-03-08T21:15:18.512 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1536 offset 4197888 len 8192 2024-03-08T21:15:18.512 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 1536 offset 4197888 len 8192 2024-03-08T21:15:18.512 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1536 offset 4197888 len 8192 2024-03-08T21:15:18.512 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 1536 offset 4197888 len 16384 2024-03-08T21:15:18.512 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1536 offset 4197888 len 16384 2024-03-08T21:15:18.512 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 1536 offset 4197888 len 16384 2024-03-08T21:15:18.512 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 1536 offset 4197888 len 16384 2024-03-08T21:15:18.513 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 2048 offset 4190720 len 1024 2024-03-08T21:15:18.513 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2048 offset 4190720 len 1024 2024-03-08T21:15:18.513 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 2048 offset 4190720 len 1024 2024-03-08T21:15:18.513 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2048 offset 4190720 len 1024 2024-03-08T21:15:18.513 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 2048 offset 4190720 len 2048 2024-03-08T21:15:18.513 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2048 offset 4190720 len 2048 2024-03-08T21:15:18.513 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 2048 offset 4190720 len 2048 2024-03-08T21:15:18.513 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2048 offset 4190720 len 2048 2024-03-08T21:15:18.513 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 2048 offset 4190720 len 4096 2024-03-08T21:15:18.513 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2048 offset 4190720 len 4096 2024-03-08T21:15:18.513 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 2048 offset 4190720 len 4096 2024-03-08T21:15:18.513 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2048 offset 4190720 len 4096 2024-03-08T21:15:18.513 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 2048 offset 4190720 len 8192 2024-03-08T21:15:18.513 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2048 offset 4190720 len 8192 2024-03-08T21:15:18.513 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 2048 offset 4190720 len 8192 2024-03-08T21:15:18.514 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2048 offset 4190720 len 8192 2024-03-08T21:15:18.514 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 2048 offset 4190720 len 16384 2024-03-08T21:15:18.514 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2048 offset 4190720 len 16384 2024-03-08T21:15:18.514 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 2048 offset 4190720 len 16384 2024-03-08T21:15:18.514 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2048 offset 4190720 len 16384 2024-03-08T21:15:18.514 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 2048 offset 4191232 len 1024 2024-03-08T21:15:18.514 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2048 offset 4191232 len 1024 2024-03-08T21:15:18.514 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 2048 offset 4191232 len 1024 2024-03-08T21:15:18.514 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2048 offset 4191232 len 1024 2024-03-08T21:15:18.514 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 2048 offset 4191232 len 2048 2024-03-08T21:15:18.514 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2048 offset 4191232 len 2048 2024-03-08T21:15:18.514 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 2048 offset 4191232 len 2048 2024-03-08T21:15:18.514 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2048 offset 4191232 len 2048 2024-03-08T21:15:18.515 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 2048 offset 4191232 len 4096 2024-03-08T21:15:18.515 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2048 offset 4191232 len 4096 2024-03-08T21:15:18.515 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 2048 offset 4191232 len 4096 2024-03-08T21:15:18.515 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2048 offset 4191232 len 4096 2024-03-08T21:15:18.515 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 2048 offset 4191232 len 8192 2024-03-08T21:15:18.515 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2048 offset 4191232 len 8192 2024-03-08T21:15:18.515 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 2048 offset 4191232 len 8192 2024-03-08T21:15:18.515 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2048 offset 4191232 len 8192 2024-03-08T21:15:18.515 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 2048 offset 4191232 len 16384 2024-03-08T21:15:18.515 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2048 offset 4191232 len 16384 2024-03-08T21:15:18.515 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 2048 offset 4191232 len 16384 2024-03-08T21:15:18.515 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2048 offset 4191232 len 16384 2024-03-08T21:15:18.515 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 2048 offset 4191744 len 1024 2024-03-08T21:15:18.515 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2048 offset 4191744 len 1024 2024-03-08T21:15:18.516 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 2048 offset 4191744 len 1024 2024-03-08T21:15:18.516 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2048 offset 4191744 len 1024 2024-03-08T21:15:18.516 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 2048 offset 4191744 len 2048 2024-03-08T21:15:18.516 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2048 offset 4191744 len 2048 2024-03-08T21:15:18.516 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 2048 offset 4191744 len 2048 2024-03-08T21:15:18.516 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2048 offset 4191744 len 2048 2024-03-08T21:15:18.516 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 2048 offset 4191744 len 4096 2024-03-08T21:15:18.516 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2048 offset 4191744 len 4096 2024-03-08T21:15:18.516 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 2048 offset 4191744 len 4096 2024-03-08T21:15:18.516 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2048 offset 4191744 len 4096 2024-03-08T21:15:18.516 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 2048 offset 4191744 len 8192 2024-03-08T21:15:18.516 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2048 offset 4191744 len 8192 2024-03-08T21:15:18.516 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 2048 offset 4191744 len 8192 2024-03-08T21:15:18.766 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:15:18 smithi107 ceph-mon[31381]: pgmap v469: 97 pgs: 97 active+clean; 43 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 185 op/s 2024-03-08T21:15:18.871 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:15:18 smithi204 ceph-mon[34935]: pgmap v469: 97 pgs: 97 active+clean; 43 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 185 op/s 2024-03-08T21:15:19.008 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:15:18 smithi149 ceph-mon[35042]: pgmap v469: 97 pgs: 97 active+clean; 43 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 185 op/s 2024-03-08T21:15:19.695 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2048 offset 4191744 len 8192 2024-03-08T21:15:19.695 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 2048 offset 4191744 len 16384 2024-03-08T21:15:19.695 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2048 offset 4191744 len 16384 2024-03-08T21:15:19.695 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 2048 offset 4191744 len 16384 2024-03-08T21:15:19.695 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2048 offset 4191744 len 16384 2024-03-08T21:15:19.695 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 2048 offset 4192256 len 1024 2024-03-08T21:15:19.696 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2048 offset 4192256 len 1024 2024-03-08T21:15:19.696 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 2048 offset 4192256 len 1024 2024-03-08T21:15:19.696 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2048 offset 4192256 len 1024 2024-03-08T21:15:19.696 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 2048 offset 4192256 len 2048 2024-03-08T21:15:19.696 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2048 offset 4192256 len 2048 2024-03-08T21:15:19.696 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 2048 offset 4192256 len 2048 2024-03-08T21:15:19.696 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2048 offset 4192256 len 2048 2024-03-08T21:15:19.696 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 2048 offset 4192256 len 4096 2024-03-08T21:15:19.696 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2048 offset 4192256 len 4096 2024-03-08T21:15:19.696 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 2048 offset 4192256 len 4096 2024-03-08T21:15:19.696 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2048 offset 4192256 len 4096 2024-03-08T21:15:19.696 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 2048 offset 4192256 len 8192 2024-03-08T21:15:19.696 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2048 offset 4192256 len 8192 2024-03-08T21:15:19.696 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 2048 offset 4192256 len 8192 2024-03-08T21:15:19.696 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2048 offset 4192256 len 8192 2024-03-08T21:15:19.697 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 2048 offset 4192256 len 16384 2024-03-08T21:15:19.697 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2048 offset 4192256 len 16384 2024-03-08T21:15:19.697 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 2048 offset 4192256 len 16384 2024-03-08T21:15:19.697 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2048 offset 4192256 len 16384 2024-03-08T21:15:19.697 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 2048 offset 4192768 len 1024 2024-03-08T21:15:19.697 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2048 offset 4192768 len 1024 2024-03-08T21:15:19.697 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 2048 offset 4192768 len 1024 2024-03-08T21:15:19.697 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2048 offset 4192768 len 1024 2024-03-08T21:15:19.697 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 2048 offset 4192768 len 2048 2024-03-08T21:15:19.697 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2048 offset 4192768 len 2048 2024-03-08T21:15:19.697 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 2048 offset 4192768 len 2048 2024-03-08T21:15:19.697 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2048 offset 4192768 len 2048 2024-03-08T21:15:19.697 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 2048 offset 4192768 len 4096 2024-03-08T21:15:19.697 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2048 offset 4192768 len 4096 2024-03-08T21:15:19.698 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 2048 offset 4192768 len 4096 2024-03-08T21:15:19.698 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2048 offset 4192768 len 4096 2024-03-08T21:15:19.698 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 2048 offset 4192768 len 8192 2024-03-08T21:15:19.698 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2048 offset 4192768 len 8192 2024-03-08T21:15:19.698 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 2048 offset 4192768 len 8192 2024-03-08T21:15:19.698 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2048 offset 4192768 len 8192 2024-03-08T21:15:19.698 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 2048 offset 4192768 len 16384 2024-03-08T21:15:19.698 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2048 offset 4192768 len 16384 2024-03-08T21:15:19.698 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 2048 offset 4192768 len 16384 2024-03-08T21:15:19.698 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2048 offset 4192768 len 16384 2024-03-08T21:15:19.698 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 2048 offset 4193280 len 1024 2024-03-08T21:15:19.698 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2048 offset 4193280 len 1024 2024-03-08T21:15:19.698 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 2048 offset 4193280 len 1024 2024-03-08T21:15:19.699 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2048 offset 4193280 len 1024 2024-03-08T21:15:19.699 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 2048 offset 4193280 len 2048 2024-03-08T21:15:19.699 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2048 offset 4193280 len 2048 2024-03-08T21:15:19.699 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 2048 offset 4193280 len 2048 2024-03-08T21:15:19.699 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2048 offset 4193280 len 2048 2024-03-08T21:15:19.699 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 2048 offset 4193280 len 4096 2024-03-08T21:15:19.699 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2048 offset 4193280 len 4096 2024-03-08T21:15:19.699 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 2048 offset 4193280 len 4096 2024-03-08T21:15:19.699 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2048 offset 4193280 len 4096 2024-03-08T21:15:19.699 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 2048 offset 4193280 len 8192 2024-03-08T21:15:19.699 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2048 offset 4193280 len 8192 2024-03-08T21:15:19.699 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 2048 offset 4193280 len 8192 2024-03-08T21:15:19.699 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2048 offset 4193280 len 8192 2024-03-08T21:15:19.699 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 2048 offset 4193280 len 16384 2024-03-08T21:15:19.699 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2048 offset 4193280 len 16384 2024-03-08T21:15:19.700 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 2048 offset 4193280 len 16384 2024-03-08T21:15:19.700 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2048 offset 4193280 len 16384 2024-03-08T21:15:19.700 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 2048 offset 4193792 len 1024 2024-03-08T21:15:19.700 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2048 offset 4193792 len 1024 2024-03-08T21:15:19.700 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 2048 offset 4193792 len 1024 2024-03-08T21:15:19.700 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2048 offset 4193792 len 1024 2024-03-08T21:15:19.700 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 2048 offset 4193792 len 2048 2024-03-08T21:15:19.700 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2048 offset 4193792 len 2048 2024-03-08T21:15:19.700 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 2048 offset 4193792 len 2048 2024-03-08T21:15:19.700 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2048 offset 4193792 len 2048 2024-03-08T21:15:19.700 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 2048 offset 4193792 len 4096 2024-03-08T21:15:19.700 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2048 offset 4193792 len 4096 2024-03-08T21:15:19.700 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 2048 offset 4193792 len 4096 2024-03-08T21:15:19.700 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2048 offset 4193792 len 4096 2024-03-08T21:15:19.701 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 2048 offset 4193792 len 8192 2024-03-08T21:15:19.701 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2048 offset 4193792 len 8192 2024-03-08T21:15:19.701 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 2048 offset 4193792 len 8192 2024-03-08T21:15:19.701 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2048 offset 4193792 len 8192 2024-03-08T21:15:20.872 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:15:20 smithi204 ceph-mon[34935]: pgmap v470: 97 pgs: 97 active+clean; 143 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 209 op/s 2024-03-08T21:15:20.955 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 2048 offset 4193792 len 16384 2024-03-08T21:15:20.955 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2048 offset 4193792 len 16384 2024-03-08T21:15:20.955 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 2048 offset 4193792 len 16384 2024-03-08T21:15:20.955 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2048 offset 4193792 len 16384 2024-03-08T21:15:20.955 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 2048 offset 4194304 len 1024 2024-03-08T21:15:20.955 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2048 offset 4194304 len 1024 2024-03-08T21:15:20.955 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 2048 offset 4194304 len 1024 2024-03-08T21:15:20.956 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2048 offset 4194304 len 1024 2024-03-08T21:15:20.956 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 2048 offset 4194304 len 2048 2024-03-08T21:15:20.956 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2048 offset 4194304 len 2048 2024-03-08T21:15:20.956 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 2048 offset 4194304 len 2048 2024-03-08T21:15:20.956 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2048 offset 4194304 len 2048 2024-03-08T21:15:20.956 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 2048 offset 4194304 len 4096 2024-03-08T21:15:20.956 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2048 offset 4194304 len 4096 2024-03-08T21:15:20.956 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 2048 offset 4194304 len 4096 2024-03-08T21:15:20.956 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2048 offset 4194304 len 4096 2024-03-08T21:15:20.956 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 2048 offset 4194304 len 8192 2024-03-08T21:15:20.956 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2048 offset 4194304 len 8192 2024-03-08T21:15:20.956 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 2048 offset 4194304 len 8192 2024-03-08T21:15:20.956 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2048 offset 4194304 len 8192 2024-03-08T21:15:20.956 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 2048 offset 4194304 len 16384 2024-03-08T21:15:20.957 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2048 offset 4194304 len 16384 2024-03-08T21:15:20.957 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 2048 offset 4194304 len 16384 2024-03-08T21:15:20.957 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2048 offset 4194304 len 16384 2024-03-08T21:15:20.957 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 2048 offset 4194816 len 1024 2024-03-08T21:15:20.957 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2048 offset 4194816 len 1024 2024-03-08T21:15:20.957 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 2048 offset 4194816 len 1024 2024-03-08T21:15:20.957 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2048 offset 4194816 len 1024 2024-03-08T21:15:20.957 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 2048 offset 4194816 len 2048 2024-03-08T21:15:20.957 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2048 offset 4194816 len 2048 2024-03-08T21:15:20.957 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 2048 offset 4194816 len 2048 2024-03-08T21:15:20.957 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2048 offset 4194816 len 2048 2024-03-08T21:15:20.957 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 2048 offset 4194816 len 4096 2024-03-08T21:15:20.957 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2048 offset 4194816 len 4096 2024-03-08T21:15:20.957 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 2048 offset 4194816 len 4096 2024-03-08T21:15:20.957 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2048 offset 4194816 len 4096 2024-03-08T21:15:20.958 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 2048 offset 4194816 len 8192 2024-03-08T21:15:20.958 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2048 offset 4194816 len 8192 2024-03-08T21:15:20.958 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 2048 offset 4194816 len 8192 2024-03-08T21:15:20.958 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2048 offset 4194816 len 8192 2024-03-08T21:15:20.958 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 2048 offset 4194816 len 16384 2024-03-08T21:15:20.958 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2048 offset 4194816 len 16384 2024-03-08T21:15:20.958 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 2048 offset 4194816 len 16384 2024-03-08T21:15:20.958 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2048 offset 4194816 len 16384 2024-03-08T21:15:20.958 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 2048 offset 4195328 len 1024 2024-03-08T21:15:20.958 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2048 offset 4195328 len 1024 2024-03-08T21:15:20.958 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 2048 offset 4195328 len 1024 2024-03-08T21:15:20.958 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2048 offset 4195328 len 1024 2024-03-08T21:15:20.958 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 2048 offset 4195328 len 2048 2024-03-08T21:15:20.959 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2048 offset 4195328 len 2048 2024-03-08T21:15:20.959 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 2048 offset 4195328 len 2048 2024-03-08T21:15:20.959 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2048 offset 4195328 len 2048 2024-03-08T21:15:20.959 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 2048 offset 4195328 len 4096 2024-03-08T21:15:20.959 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2048 offset 4195328 len 4096 2024-03-08T21:15:20.959 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 2048 offset 4195328 len 4096 2024-03-08T21:15:20.959 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2048 offset 4195328 len 4096 2024-03-08T21:15:20.959 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 2048 offset 4195328 len 8192 2024-03-08T21:15:20.959 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2048 offset 4195328 len 8192 2024-03-08T21:15:20.959 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 2048 offset 4195328 len 8192 2024-03-08T21:15:20.959 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2048 offset 4195328 len 8192 2024-03-08T21:15:20.959 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 2048 offset 4195328 len 16384 2024-03-08T21:15:20.959 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2048 offset 4195328 len 16384 2024-03-08T21:15:20.959 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 2048 offset 4195328 len 16384 2024-03-08T21:15:20.960 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2048 offset 4195328 len 16384 2024-03-08T21:15:20.960 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 2048 offset 4195840 len 1024 2024-03-08T21:15:20.960 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2048 offset 4195840 len 1024 2024-03-08T21:15:20.960 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 2048 offset 4195840 len 1024 2024-03-08T21:15:20.960 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2048 offset 4195840 len 1024 2024-03-08T21:15:20.960 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 2048 offset 4195840 len 2048 2024-03-08T21:15:20.960 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2048 offset 4195840 len 2048 2024-03-08T21:15:20.960 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 2048 offset 4195840 len 2048 2024-03-08T21:15:20.960 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2048 offset 4195840 len 2048 2024-03-08T21:15:20.960 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 2048 offset 4195840 len 4096 2024-03-08T21:15:20.960 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2048 offset 4195840 len 4096 2024-03-08T21:15:20.960 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 2048 offset 4195840 len 4096 2024-03-08T21:15:20.960 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2048 offset 4195840 len 4096 2024-03-08T21:15:20.960 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 2048 offset 4195840 len 8192 2024-03-08T21:15:20.961 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2048 offset 4195840 len 8192 2024-03-08T21:15:20.961 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 2048 offset 4195840 len 8192 2024-03-08T21:15:20.961 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2048 offset 4195840 len 8192 2024-03-08T21:15:20.961 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:15:20 smithi107 ceph-mon[31381]: pgmap v470: 97 pgs: 97 active+clean; 143 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 209 op/s 2024-03-08T21:15:21.008 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:15:20 smithi149 ceph-mon[35042]: pgmap v470: 97 pgs: 97 active+clean; 143 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 209 op/s 2024-03-08T21:15:22.176 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 2048 offset 4195840 len 16384 2024-03-08T21:15:22.177 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2048 offset 4195840 len 16384 2024-03-08T21:15:22.177 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 2048 offset 4195840 len 16384 2024-03-08T21:15:22.177 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2048 offset 4195840 len 16384 2024-03-08T21:15:22.177 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 2048 offset 4196352 len 1024 2024-03-08T21:15:22.177 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2048 offset 4196352 len 1024 2024-03-08T21:15:22.177 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 2048 offset 4196352 len 1024 2024-03-08T21:15:22.177 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2048 offset 4196352 len 1024 2024-03-08T21:15:22.177 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 2048 offset 4196352 len 2048 2024-03-08T21:15:22.177 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2048 offset 4196352 len 2048 2024-03-08T21:15:22.177 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 2048 offset 4196352 len 2048 2024-03-08T21:15:22.177 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2048 offset 4196352 len 2048 2024-03-08T21:15:22.177 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 2048 offset 4196352 len 4096 2024-03-08T21:15:22.178 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2048 offset 4196352 len 4096 2024-03-08T21:15:22.178 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 2048 offset 4196352 len 4096 2024-03-08T21:15:22.178 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2048 offset 4196352 len 4096 2024-03-08T21:15:22.178 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 2048 offset 4196352 len 8192 2024-03-08T21:15:22.178 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2048 offset 4196352 len 8192 2024-03-08T21:15:22.178 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 2048 offset 4196352 len 8192 2024-03-08T21:15:22.178 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2048 offset 4196352 len 8192 2024-03-08T21:15:22.178 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 2048 offset 4196352 len 16384 2024-03-08T21:15:22.178 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2048 offset 4196352 len 16384 2024-03-08T21:15:22.178 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 2048 offset 4196352 len 16384 2024-03-08T21:15:22.178 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2048 offset 4196352 len 16384 2024-03-08T21:15:22.178 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 2048 offset 4196864 len 1024 2024-03-08T21:15:22.178 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2048 offset 4196864 len 1024 2024-03-08T21:15:22.178 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 2048 offset 4196864 len 1024 2024-03-08T21:15:22.179 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2048 offset 4196864 len 1024 2024-03-08T21:15:22.179 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 2048 offset 4196864 len 2048 2024-03-08T21:15:22.179 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2048 offset 4196864 len 2048 2024-03-08T21:15:22.179 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 2048 offset 4196864 len 2048 2024-03-08T21:15:22.179 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2048 offset 4196864 len 2048 2024-03-08T21:15:22.179 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 2048 offset 4196864 len 4096 2024-03-08T21:15:22.179 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2048 offset 4196864 len 4096 2024-03-08T21:15:22.179 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 2048 offset 4196864 len 4096 2024-03-08T21:15:22.179 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2048 offset 4196864 len 4096 2024-03-08T21:15:22.179 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 2048 offset 4196864 len 8192 2024-03-08T21:15:22.179 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2048 offset 4196864 len 8192 2024-03-08T21:15:22.179 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 2048 offset 4196864 len 8192 2024-03-08T21:15:22.179 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2048 offset 4196864 len 8192 2024-03-08T21:15:22.179 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 2048 offset 4196864 len 16384 2024-03-08T21:15:22.179 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2048 offset 4196864 len 16384 2024-03-08T21:15:22.180 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 2048 offset 4196864 len 16384 2024-03-08T21:15:22.180 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2048 offset 4196864 len 16384 2024-03-08T21:15:22.180 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 2048 offset 4197376 len 1024 2024-03-08T21:15:22.180 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2048 offset 4197376 len 1024 2024-03-08T21:15:22.180 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 2048 offset 4197376 len 1024 2024-03-08T21:15:22.180 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2048 offset 4197376 len 1024 2024-03-08T21:15:22.180 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 2048 offset 4197376 len 2048 2024-03-08T21:15:22.180 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2048 offset 4197376 len 2048 2024-03-08T21:15:22.180 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 2048 offset 4197376 len 2048 2024-03-08T21:15:22.180 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2048 offset 4197376 len 2048 2024-03-08T21:15:22.180 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 2048 offset 4197376 len 4096 2024-03-08T21:15:22.180 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2048 offset 4197376 len 4096 2024-03-08T21:15:22.180 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 2048 offset 4197376 len 4096 2024-03-08T21:15:22.180 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2048 offset 4197376 len 4096 2024-03-08T21:15:22.181 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 2048 offset 4197376 len 8192 2024-03-08T21:15:22.181 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2048 offset 4197376 len 8192 2024-03-08T21:15:22.181 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 2048 offset 4197376 len 8192 2024-03-08T21:15:22.181 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2048 offset 4197376 len 8192 2024-03-08T21:15:22.181 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 2048 offset 4197376 len 16384 2024-03-08T21:15:22.181 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2048 offset 4197376 len 16384 2024-03-08T21:15:22.181 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 2048 offset 4197376 len 16384 2024-03-08T21:15:22.181 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2048 offset 4197376 len 16384 2024-03-08T21:15:22.181 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 2048 offset 4197888 len 1024 2024-03-08T21:15:22.181 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2048 offset 4197888 len 1024 2024-03-08T21:15:22.181 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 2048 offset 4197888 len 1024 2024-03-08T21:15:22.181 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2048 offset 4197888 len 1024 2024-03-08T21:15:22.181 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 2048 offset 4197888 len 2048 2024-03-08T21:15:22.181 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2048 offset 4197888 len 2048 2024-03-08T21:15:22.181 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 2048 offset 4197888 len 2048 2024-03-08T21:15:22.182 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2048 offset 4197888 len 2048 2024-03-08T21:15:22.182 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 2048 offset 4197888 len 4096 2024-03-08T21:15:22.182 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2048 offset 4197888 len 4096 2024-03-08T21:15:22.182 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 2048 offset 4197888 len 4096 2024-03-08T21:15:22.182 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2048 offset 4197888 len 4096 2024-03-08T21:15:22.182 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 2048 offset 4197888 len 8192 2024-03-08T21:15:22.182 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2048 offset 4197888 len 8192 2024-03-08T21:15:22.182 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 2048 offset 4197888 len 8192 2024-03-08T21:15:22.182 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2048 offset 4197888 len 8192 2024-03-08T21:15:22.182 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 2048 offset 4197888 len 16384 2024-03-08T21:15:22.872 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:15:22 smithi204 ceph-mon[34935]: pgmap v471: 97 pgs: 97 active+clean; 143 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 209 op/s 2024-03-08T21:15:22.961 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:15:22 smithi107 ceph-mon[31381]: pgmap v471: 97 pgs: 97 active+clean; 143 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 209 op/s 2024-03-08T21:15:23.008 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:15:22 smithi149 ceph-mon[35042]: pgmap v471: 97 pgs: 97 active+clean; 143 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 209 op/s 2024-03-08T21:15:23.299 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2048 offset 4197888 len 16384 2024-03-08T21:15:23.299 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 2048 offset 4197888 len 16384 2024-03-08T21:15:23.299 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2048 offset 4197888 len 16384 2024-03-08T21:15:23.300 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 2560 offset 4190720 len 1024 2024-03-08T21:15:23.300 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2560 offset 4190720 len 1024 2024-03-08T21:15:23.300 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 2560 offset 4190720 len 1024 2024-03-08T21:15:23.300 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2560 offset 4190720 len 1024 2024-03-08T21:15:23.300 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 2560 offset 4190720 len 2048 2024-03-08T21:15:23.300 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2560 offset 4190720 len 2048 2024-03-08T21:15:23.300 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 2560 offset 4190720 len 2048 2024-03-08T21:15:23.300 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2560 offset 4190720 len 2048 2024-03-08T21:15:23.300 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 2560 offset 4190720 len 4096 2024-03-08T21:15:23.300 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2560 offset 4190720 len 4096 2024-03-08T21:15:23.300 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 2560 offset 4190720 len 4096 2024-03-08T21:15:23.300 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2560 offset 4190720 len 4096 2024-03-08T21:15:23.300 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 2560 offset 4190720 len 8192 2024-03-08T21:15:23.301 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2560 offset 4190720 len 8192 2024-03-08T21:15:23.301 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 2560 offset 4190720 len 8192 2024-03-08T21:15:23.301 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2560 offset 4190720 len 8192 2024-03-08T21:15:23.301 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 2560 offset 4190720 len 16384 2024-03-08T21:15:23.301 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2560 offset 4190720 len 16384 2024-03-08T21:15:23.301 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 2560 offset 4190720 len 16384 2024-03-08T21:15:23.301 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2560 offset 4190720 len 16384 2024-03-08T21:15:23.301 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 2560 offset 4191232 len 1024 2024-03-08T21:15:23.301 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2560 offset 4191232 len 1024 2024-03-08T21:15:23.301 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 2560 offset 4191232 len 1024 2024-03-08T21:15:23.301 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2560 offset 4191232 len 1024 2024-03-08T21:15:23.301 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 2560 offset 4191232 len 2048 2024-03-08T21:15:23.301 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2560 offset 4191232 len 2048 2024-03-08T21:15:23.301 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 2560 offset 4191232 len 2048 2024-03-08T21:15:23.302 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2560 offset 4191232 len 2048 2024-03-08T21:15:23.302 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 2560 offset 4191232 len 4096 2024-03-08T21:15:23.302 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2560 offset 4191232 len 4096 2024-03-08T21:15:23.302 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 2560 offset 4191232 len 4096 2024-03-08T21:15:23.302 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2560 offset 4191232 len 4096 2024-03-08T21:15:23.302 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 2560 offset 4191232 len 8192 2024-03-08T21:15:23.302 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2560 offset 4191232 len 8192 2024-03-08T21:15:23.302 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 2560 offset 4191232 len 8192 2024-03-08T21:15:23.302 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2560 offset 4191232 len 8192 2024-03-08T21:15:23.302 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 2560 offset 4191232 len 16384 2024-03-08T21:15:23.302 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2560 offset 4191232 len 16384 2024-03-08T21:15:23.302 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 2560 offset 4191232 len 16384 2024-03-08T21:15:23.302 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2560 offset 4191232 len 16384 2024-03-08T21:15:23.302 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 2560 offset 4191744 len 1024 2024-03-08T21:15:23.303 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2560 offset 4191744 len 1024 2024-03-08T21:15:23.303 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 2560 offset 4191744 len 1024 2024-03-08T21:15:23.303 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2560 offset 4191744 len 1024 2024-03-08T21:15:23.303 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 2560 offset 4191744 len 2048 2024-03-08T21:15:23.303 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2560 offset 4191744 len 2048 2024-03-08T21:15:23.303 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 2560 offset 4191744 len 2048 2024-03-08T21:15:23.303 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2560 offset 4191744 len 2048 2024-03-08T21:15:23.303 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 2560 offset 4191744 len 4096 2024-03-08T21:15:23.303 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2560 offset 4191744 len 4096 2024-03-08T21:15:23.303 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 2560 offset 4191744 len 4096 2024-03-08T21:15:23.303 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2560 offset 4191744 len 4096 2024-03-08T21:15:23.303 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 2560 offset 4191744 len 8192 2024-03-08T21:15:23.303 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2560 offset 4191744 len 8192 2024-03-08T21:15:23.303 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 2560 offset 4191744 len 8192 2024-03-08T21:15:23.303 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2560 offset 4191744 len 8192 2024-03-08T21:15:23.304 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 2560 offset 4191744 len 16384 2024-03-08T21:15:23.304 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2560 offset 4191744 len 16384 2024-03-08T21:15:23.304 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 2560 offset 4191744 len 16384 2024-03-08T21:15:23.304 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2560 offset 4191744 len 16384 2024-03-08T21:15:23.304 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 2560 offset 4192256 len 1024 2024-03-08T21:15:23.304 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2560 offset 4192256 len 1024 2024-03-08T21:15:23.304 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 2560 offset 4192256 len 1024 2024-03-08T21:15:23.304 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2560 offset 4192256 len 1024 2024-03-08T21:15:23.304 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 2560 offset 4192256 len 2048 2024-03-08T21:15:23.304 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2560 offset 4192256 len 2048 2024-03-08T21:15:23.304 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 2560 offset 4192256 len 2048 2024-03-08T21:15:23.304 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2560 offset 4192256 len 2048 2024-03-08T21:15:23.304 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 2560 offset 4192256 len 4096 2024-03-08T21:15:23.304 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2560 offset 4192256 len 4096 2024-03-08T21:15:23.305 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 2560 offset 4192256 len 4096 2024-03-08T21:15:23.305 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2560 offset 4192256 len 4096 2024-03-08T21:15:23.305 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 2560 offset 4192256 len 8192 2024-03-08T21:15:23.305 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2560 offset 4192256 len 8192 2024-03-08T21:15:23.305 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 2560 offset 4192256 len 8192 2024-03-08T21:15:23.305 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2560 offset 4192256 len 8192 2024-03-08T21:15:23.305 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 2560 offset 4192256 len 16384 2024-03-08T21:15:23.305 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2560 offset 4192256 len 16384 2024-03-08T21:15:24.488 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 2560 offset 4192256 len 16384 2024-03-08T21:15:24.488 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2560 offset 4192256 len 16384 2024-03-08T21:15:24.488 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 2560 offset 4192768 len 1024 2024-03-08T21:15:24.488 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2560 offset 4192768 len 1024 2024-03-08T21:15:24.488 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 2560 offset 4192768 len 1024 2024-03-08T21:15:24.488 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2560 offset 4192768 len 1024 2024-03-08T21:15:24.488 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 2560 offset 4192768 len 2048 2024-03-08T21:15:24.489 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2560 offset 4192768 len 2048 2024-03-08T21:15:24.489 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 2560 offset 4192768 len 2048 2024-03-08T21:15:24.489 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2560 offset 4192768 len 2048 2024-03-08T21:15:24.489 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 2560 offset 4192768 len 4096 2024-03-08T21:15:24.489 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2560 offset 4192768 len 4096 2024-03-08T21:15:24.489 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 2560 offset 4192768 len 4096 2024-03-08T21:15:24.489 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2560 offset 4192768 len 4096 2024-03-08T21:15:24.489 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 2560 offset 4192768 len 8192 2024-03-08T21:15:24.489 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2560 offset 4192768 len 8192 2024-03-08T21:15:24.489 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 2560 offset 4192768 len 8192 2024-03-08T21:15:24.489 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2560 offset 4192768 len 8192 2024-03-08T21:15:24.490 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 2560 offset 4192768 len 16384 2024-03-08T21:15:24.490 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2560 offset 4192768 len 16384 2024-03-08T21:15:24.490 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 2560 offset 4192768 len 16384 2024-03-08T21:15:24.490 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2560 offset 4192768 len 16384 2024-03-08T21:15:24.490 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 2560 offset 4193280 len 1024 2024-03-08T21:15:24.490 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2560 offset 4193280 len 1024 2024-03-08T21:15:24.490 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 2560 offset 4193280 len 1024 2024-03-08T21:15:24.490 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2560 offset 4193280 len 1024 2024-03-08T21:15:24.490 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 2560 offset 4193280 len 2048 2024-03-08T21:15:24.490 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2560 offset 4193280 len 2048 2024-03-08T21:15:24.490 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 2560 offset 4193280 len 2048 2024-03-08T21:15:24.490 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2560 offset 4193280 len 2048 2024-03-08T21:15:24.491 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 2560 offset 4193280 len 4096 2024-03-08T21:15:24.491 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2560 offset 4193280 len 4096 2024-03-08T21:15:24.491 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 2560 offset 4193280 len 4096 2024-03-08T21:15:24.491 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2560 offset 4193280 len 4096 2024-03-08T21:15:24.491 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 2560 offset 4193280 len 8192 2024-03-08T21:15:24.491 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2560 offset 4193280 len 8192 2024-03-08T21:15:24.491 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 2560 offset 4193280 len 8192 2024-03-08T21:15:24.491 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2560 offset 4193280 len 8192 2024-03-08T21:15:24.491 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 2560 offset 4193280 len 16384 2024-03-08T21:15:24.491 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2560 offset 4193280 len 16384 2024-03-08T21:15:24.491 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 2560 offset 4193280 len 16384 2024-03-08T21:15:24.492 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2560 offset 4193280 len 16384 2024-03-08T21:15:24.492 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 2560 offset 4193792 len 1024 2024-03-08T21:15:24.492 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2560 offset 4193792 len 1024 2024-03-08T21:15:24.492 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 2560 offset 4193792 len 1024 2024-03-08T21:15:24.492 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2560 offset 4193792 len 1024 2024-03-08T21:15:24.492 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 2560 offset 4193792 len 2048 2024-03-08T21:15:24.492 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2560 offset 4193792 len 2048 2024-03-08T21:15:24.492 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 2560 offset 4193792 len 2048 2024-03-08T21:15:24.492 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2560 offset 4193792 len 2048 2024-03-08T21:15:24.492 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 2560 offset 4193792 len 4096 2024-03-08T21:15:24.492 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2560 offset 4193792 len 4096 2024-03-08T21:15:24.492 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 2560 offset 4193792 len 4096 2024-03-08T21:15:24.492 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2560 offset 4193792 len 4096 2024-03-08T21:15:24.492 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 2560 offset 4193792 len 8192 2024-03-08T21:15:24.492 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2560 offset 4193792 len 8192 2024-03-08T21:15:24.493 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 2560 offset 4193792 len 8192 2024-03-08T21:15:24.493 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2560 offset 4193792 len 8192 2024-03-08T21:15:24.493 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 2560 offset 4193792 len 16384 2024-03-08T21:15:24.493 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2560 offset 4193792 len 16384 2024-03-08T21:15:24.493 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 2560 offset 4193792 len 16384 2024-03-08T21:15:24.493 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2560 offset 4193792 len 16384 2024-03-08T21:15:24.493 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 2560 offset 4194304 len 1024 2024-03-08T21:15:24.493 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2560 offset 4194304 len 1024 2024-03-08T21:15:24.493 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 2560 offset 4194304 len 1024 2024-03-08T21:15:24.493 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2560 offset 4194304 len 1024 2024-03-08T21:15:24.493 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 2560 offset 4194304 len 2048 2024-03-08T21:15:24.494 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2560 offset 4194304 len 2048 2024-03-08T21:15:24.494 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 2560 offset 4194304 len 2048 2024-03-08T21:15:24.494 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2560 offset 4194304 len 2048 2024-03-08T21:15:24.494 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 2560 offset 4194304 len 4096 2024-03-08T21:15:24.494 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2560 offset 4194304 len 4096 2024-03-08T21:15:24.494 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 2560 offset 4194304 len 4096 2024-03-08T21:15:24.494 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2560 offset 4194304 len 4096 2024-03-08T21:15:24.494 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 2560 offset 4194304 len 8192 2024-03-08T21:15:24.494 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2560 offset 4194304 len 8192 2024-03-08T21:15:24.494 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 2560 offset 4194304 len 8192 2024-03-08T21:15:24.494 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2560 offset 4194304 len 8192 2024-03-08T21:15:24.494 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 2560 offset 4194304 len 16384 2024-03-08T21:15:24.494 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2560 offset 4194304 len 16384 2024-03-08T21:15:24.495 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 2560 offset 4194304 len 16384 2024-03-08T21:15:24.875 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:15:24 smithi204 ceph-mon[34935]: pgmap v472: 97 pgs: 97 active+clean; 143 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 209 op/s 2024-03-08T21:15:24.961 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:15:24 smithi107 ceph-mon[31381]: pgmap v472: 97 pgs: 97 active+clean; 143 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 209 op/s 2024-03-08T21:15:25.008 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:15:24 smithi149 ceph-mon[35042]: pgmap v472: 97 pgs: 97 active+clean; 143 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 209 op/s 2024-03-08T21:15:25.722 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2560 offset 4194304 len 16384 2024-03-08T21:15:25.722 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 2560 offset 4194816 len 1024 2024-03-08T21:15:25.723 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2560 offset 4194816 len 1024 2024-03-08T21:15:25.723 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 2560 offset 4194816 len 1024 2024-03-08T21:15:25.723 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2560 offset 4194816 len 1024 2024-03-08T21:15:25.723 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 2560 offset 4194816 len 2048 2024-03-08T21:15:25.723 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2560 offset 4194816 len 2048 2024-03-08T21:15:25.723 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 2560 offset 4194816 len 2048 2024-03-08T21:15:25.723 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2560 offset 4194816 len 2048 2024-03-08T21:15:25.723 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 2560 offset 4194816 len 4096 2024-03-08T21:15:25.723 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2560 offset 4194816 len 4096 2024-03-08T21:15:25.723 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 2560 offset 4194816 len 4096 2024-03-08T21:15:25.723 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2560 offset 4194816 len 4096 2024-03-08T21:15:25.723 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 2560 offset 4194816 len 8192 2024-03-08T21:15:25.724 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2560 offset 4194816 len 8192 2024-03-08T21:15:25.724 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 2560 offset 4194816 len 8192 2024-03-08T21:15:25.724 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2560 offset 4194816 len 8192 2024-03-08T21:15:25.724 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 2560 offset 4194816 len 16384 2024-03-08T21:15:25.724 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2560 offset 4194816 len 16384 2024-03-08T21:15:25.724 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 2560 offset 4194816 len 16384 2024-03-08T21:15:25.724 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2560 offset 4194816 len 16384 2024-03-08T21:15:25.724 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 2560 offset 4195328 len 1024 2024-03-08T21:15:25.724 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2560 offset 4195328 len 1024 2024-03-08T21:15:25.724 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 2560 offset 4195328 len 1024 2024-03-08T21:15:25.724 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2560 offset 4195328 len 1024 2024-03-08T21:15:25.724 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 2560 offset 4195328 len 2048 2024-03-08T21:15:25.724 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2560 offset 4195328 len 2048 2024-03-08T21:15:25.724 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 2560 offset 4195328 len 2048 2024-03-08T21:15:25.725 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2560 offset 4195328 len 2048 2024-03-08T21:15:25.725 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 2560 offset 4195328 len 4096 2024-03-08T21:15:25.725 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2560 offset 4195328 len 4096 2024-03-08T21:15:25.725 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 2560 offset 4195328 len 4096 2024-03-08T21:15:25.725 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2560 offset 4195328 len 4096 2024-03-08T21:15:25.725 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 2560 offset 4195328 len 8192 2024-03-08T21:15:25.725 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2560 offset 4195328 len 8192 2024-03-08T21:15:25.725 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 2560 offset 4195328 len 8192 2024-03-08T21:15:25.725 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2560 offset 4195328 len 8192 2024-03-08T21:15:25.725 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 2560 offset 4195328 len 16384 2024-03-08T21:15:25.725 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2560 offset 4195328 len 16384 2024-03-08T21:15:25.725 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 2560 offset 4195328 len 16384 2024-03-08T21:15:25.725 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2560 offset 4195328 len 16384 2024-03-08T21:15:25.725 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 2560 offset 4195840 len 1024 2024-03-08T21:15:25.725 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2560 offset 4195840 len 1024 2024-03-08T21:15:25.726 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 2560 offset 4195840 len 1024 2024-03-08T21:15:25.726 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2560 offset 4195840 len 1024 2024-03-08T21:15:25.726 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 2560 offset 4195840 len 2048 2024-03-08T21:15:25.726 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2560 offset 4195840 len 2048 2024-03-08T21:15:25.726 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 2560 offset 4195840 len 2048 2024-03-08T21:15:25.726 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2560 offset 4195840 len 2048 2024-03-08T21:15:25.726 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 2560 offset 4195840 len 4096 2024-03-08T21:15:25.726 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2560 offset 4195840 len 4096 2024-03-08T21:15:25.726 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 2560 offset 4195840 len 4096 2024-03-08T21:15:25.726 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2560 offset 4195840 len 4096 2024-03-08T21:15:25.726 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 2560 offset 4195840 len 8192 2024-03-08T21:15:25.726 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2560 offset 4195840 len 8192 2024-03-08T21:15:25.726 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 2560 offset 4195840 len 8192 2024-03-08T21:15:25.727 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2560 offset 4195840 len 8192 2024-03-08T21:15:25.727 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 2560 offset 4195840 len 16384 2024-03-08T21:15:25.727 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2560 offset 4195840 len 16384 2024-03-08T21:15:25.727 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 2560 offset 4195840 len 16384 2024-03-08T21:15:25.727 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2560 offset 4195840 len 16384 2024-03-08T21:15:25.727 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 2560 offset 4196352 len 1024 2024-03-08T21:15:25.727 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2560 offset 4196352 len 1024 2024-03-08T21:15:25.727 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 2560 offset 4196352 len 1024 2024-03-08T21:15:25.727 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2560 offset 4196352 len 1024 2024-03-08T21:15:25.727 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 2560 offset 4196352 len 2048 2024-03-08T21:15:25.727 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2560 offset 4196352 len 2048 2024-03-08T21:15:25.727 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 2560 offset 4196352 len 2048 2024-03-08T21:15:25.727 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2560 offset 4196352 len 2048 2024-03-08T21:15:25.727 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 2560 offset 4196352 len 4096 2024-03-08T21:15:25.727 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2560 offset 4196352 len 4096 2024-03-08T21:15:25.728 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 2560 offset 4196352 len 4096 2024-03-08T21:15:25.728 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2560 offset 4196352 len 4096 2024-03-08T21:15:25.728 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 2560 offset 4196352 len 8192 2024-03-08T21:15:25.728 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2560 offset 4196352 len 8192 2024-03-08T21:15:25.728 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 2560 offset 4196352 len 8192 2024-03-08T21:15:25.728 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2560 offset 4196352 len 8192 2024-03-08T21:15:25.728 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 2560 offset 4196352 len 16384 2024-03-08T21:15:25.728 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2560 offset 4196352 len 16384 2024-03-08T21:15:25.728 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 2560 offset 4196352 len 16384 2024-03-08T21:15:25.728 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2560 offset 4196352 len 16384 2024-03-08T21:15:26.871 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:15:26 smithi204 ceph-mon[34935]: pgmap v473: 97 pgs: 97 active+clean; 224 MiB data, 469 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 196 op/s 2024-03-08T21:15:26.929 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 2560 offset 4196864 len 1024 2024-03-08T21:15:26.929 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2560 offset 4196864 len 1024 2024-03-08T21:15:26.929 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 2560 offset 4196864 len 1024 2024-03-08T21:15:26.930 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2560 offset 4196864 len 1024 2024-03-08T21:15:26.930 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 2560 offset 4196864 len 2048 2024-03-08T21:15:26.930 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2560 offset 4196864 len 2048 2024-03-08T21:15:26.930 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 2560 offset 4196864 len 2048 2024-03-08T21:15:26.930 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2560 offset 4196864 len 2048 2024-03-08T21:15:26.930 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 2560 offset 4196864 len 4096 2024-03-08T21:15:26.930 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2560 offset 4196864 len 4096 2024-03-08T21:15:26.930 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 2560 offset 4196864 len 4096 2024-03-08T21:15:26.930 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2560 offset 4196864 len 4096 2024-03-08T21:15:26.930 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 2560 offset 4196864 len 8192 2024-03-08T21:15:26.930 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2560 offset 4196864 len 8192 2024-03-08T21:15:26.930 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 2560 offset 4196864 len 8192 2024-03-08T21:15:26.930 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2560 offset 4196864 len 8192 2024-03-08T21:15:26.930 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 2560 offset 4196864 len 16384 2024-03-08T21:15:26.931 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2560 offset 4196864 len 16384 2024-03-08T21:15:26.931 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 2560 offset 4196864 len 16384 2024-03-08T21:15:26.931 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2560 offset 4196864 len 16384 2024-03-08T21:15:26.931 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 2560 offset 4197376 len 1024 2024-03-08T21:15:26.931 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2560 offset 4197376 len 1024 2024-03-08T21:15:26.931 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 2560 offset 4197376 len 1024 2024-03-08T21:15:26.931 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2560 offset 4197376 len 1024 2024-03-08T21:15:26.931 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 2560 offset 4197376 len 2048 2024-03-08T21:15:26.931 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2560 offset 4197376 len 2048 2024-03-08T21:15:26.931 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 2560 offset 4197376 len 2048 2024-03-08T21:15:26.931 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2560 offset 4197376 len 2048 2024-03-08T21:15:26.931 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 2560 offset 4197376 len 4096 2024-03-08T21:15:26.931 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2560 offset 4197376 len 4096 2024-03-08T21:15:26.932 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 2560 offset 4197376 len 4096 2024-03-08T21:15:26.932 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2560 offset 4197376 len 4096 2024-03-08T21:15:26.932 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 2560 offset 4197376 len 8192 2024-03-08T21:15:26.932 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2560 offset 4197376 len 8192 2024-03-08T21:15:26.932 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 2560 offset 4197376 len 8192 2024-03-08T21:15:26.932 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2560 offset 4197376 len 8192 2024-03-08T21:15:26.932 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 2560 offset 4197376 len 16384 2024-03-08T21:15:26.932 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2560 offset 4197376 len 16384 2024-03-08T21:15:26.932 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 2560 offset 4197376 len 16384 2024-03-08T21:15:26.932 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2560 offset 4197376 len 16384 2024-03-08T21:15:26.932 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 2560 offset 4197888 len 1024 2024-03-08T21:15:26.932 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2560 offset 4197888 len 1024 2024-03-08T21:15:26.932 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 2560 offset 4197888 len 1024 2024-03-08T21:15:26.933 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2560 offset 4197888 len 1024 2024-03-08T21:15:26.933 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 2560 offset 4197888 len 2048 2024-03-08T21:15:26.933 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2560 offset 4197888 len 2048 2024-03-08T21:15:26.933 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 2560 offset 4197888 len 2048 2024-03-08T21:15:26.933 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2560 offset 4197888 len 2048 2024-03-08T21:15:26.933 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 2560 offset 4197888 len 4096 2024-03-08T21:15:26.933 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2560 offset 4197888 len 4096 2024-03-08T21:15:26.933 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 2560 offset 4197888 len 4096 2024-03-08T21:15:26.933 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2560 offset 4197888 len 4096 2024-03-08T21:15:26.933 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 2560 offset 4197888 len 8192 2024-03-08T21:15:26.933 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2560 offset 4197888 len 8192 2024-03-08T21:15:26.933 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 2560 offset 4197888 len 8192 2024-03-08T21:15:26.933 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2560 offset 4197888 len 8192 2024-03-08T21:15:26.933 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 2560 offset 4197888 len 16384 2024-03-08T21:15:26.933 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2560 offset 4197888 len 16384 2024-03-08T21:15:26.934 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 2560 offset 4197888 len 16384 2024-03-08T21:15:26.934 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 2560 offset 4197888 len 16384 2024-03-08T21:15:26.934 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 3072 offset 4190720 len 1024 2024-03-08T21:15:26.934 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3072 offset 4190720 len 1024 2024-03-08T21:15:26.934 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 3072 offset 4190720 len 1024 2024-03-08T21:15:26.934 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3072 offset 4190720 len 1024 2024-03-08T21:15:26.934 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 3072 offset 4190720 len 2048 2024-03-08T21:15:26.934 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3072 offset 4190720 len 2048 2024-03-08T21:15:26.934 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 3072 offset 4190720 len 2048 2024-03-08T21:15:26.934 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3072 offset 4190720 len 2048 2024-03-08T21:15:26.934 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 3072 offset 4190720 len 4096 2024-03-08T21:15:26.934 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3072 offset 4190720 len 4096 2024-03-08T21:15:26.934 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 3072 offset 4190720 len 4096 2024-03-08T21:15:26.935 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3072 offset 4190720 len 4096 2024-03-08T21:15:26.935 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 3072 offset 4190720 len 8192 2024-03-08T21:15:26.935 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3072 offset 4190720 len 8192 2024-03-08T21:15:26.935 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 3072 offset 4190720 len 8192 2024-03-08T21:15:26.935 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3072 offset 4190720 len 8192 2024-03-08T21:15:26.935 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 3072 offset 4190720 len 16384 2024-03-08T21:15:26.935 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3072 offset 4190720 len 16384 2024-03-08T21:15:26.935 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 3072 offset 4190720 len 16384 2024-03-08T21:15:26.935 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3072 offset 4190720 len 16384 2024-03-08T21:15:26.960 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:15:26 smithi107 ceph-mon[31381]: pgmap v473: 97 pgs: 97 active+clean; 224 MiB data, 469 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 196 op/s 2024-03-08T21:15:27.008 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:15:26 smithi149 ceph-mon[35042]: pgmap v473: 97 pgs: 97 active+clean; 224 MiB data, 469 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 196 op/s 2024-03-08T21:15:27.998 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 3072 offset 4191232 len 1024 2024-03-08T21:15:27.998 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3072 offset 4191232 len 1024 2024-03-08T21:15:27.998 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 3072 offset 4191232 len 1024 2024-03-08T21:15:27.999 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3072 offset 4191232 len 1024 2024-03-08T21:15:27.999 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 3072 offset 4191232 len 2048 2024-03-08T21:15:27.999 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3072 offset 4191232 len 2048 2024-03-08T21:15:27.999 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 3072 offset 4191232 len 2048 2024-03-08T21:15:27.999 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3072 offset 4191232 len 2048 2024-03-08T21:15:27.999 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 3072 offset 4191232 len 4096 2024-03-08T21:15:27.999 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3072 offset 4191232 len 4096 2024-03-08T21:15:27.999 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 3072 offset 4191232 len 4096 2024-03-08T21:15:27.999 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3072 offset 4191232 len 4096 2024-03-08T21:15:27.999 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 3072 offset 4191232 len 8192 2024-03-08T21:15:27.999 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3072 offset 4191232 len 8192 2024-03-08T21:15:27.999 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 3072 offset 4191232 len 8192 2024-03-08T21:15:27.999 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3072 offset 4191232 len 8192 2024-03-08T21:15:27.999 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 3072 offset 4191232 len 16384 2024-03-08T21:15:28.000 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3072 offset 4191232 len 16384 2024-03-08T21:15:28.000 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 3072 offset 4191232 len 16384 2024-03-08T21:15:28.000 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3072 offset 4191232 len 16384 2024-03-08T21:15:28.000 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 3072 offset 4191744 len 1024 2024-03-08T21:15:28.000 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3072 offset 4191744 len 1024 2024-03-08T21:15:28.000 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 3072 offset 4191744 len 1024 2024-03-08T21:15:28.000 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3072 offset 4191744 len 1024 2024-03-08T21:15:28.000 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 3072 offset 4191744 len 2048 2024-03-08T21:15:28.000 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3072 offset 4191744 len 2048 2024-03-08T21:15:28.000 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 3072 offset 4191744 len 2048 2024-03-08T21:15:28.000 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3072 offset 4191744 len 2048 2024-03-08T21:15:28.000 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 3072 offset 4191744 len 4096 2024-03-08T21:15:28.000 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3072 offset 4191744 len 4096 2024-03-08T21:15:28.000 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 3072 offset 4191744 len 4096 2024-03-08T21:15:28.000 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3072 offset 4191744 len 4096 2024-03-08T21:15:28.001 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 3072 offset 4191744 len 8192 2024-03-08T21:15:28.001 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3072 offset 4191744 len 8192 2024-03-08T21:15:28.001 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 3072 offset 4191744 len 8192 2024-03-08T21:15:28.001 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3072 offset 4191744 len 8192 2024-03-08T21:15:28.001 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 3072 offset 4191744 len 16384 2024-03-08T21:15:28.001 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3072 offset 4191744 len 16384 2024-03-08T21:15:28.001 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 3072 offset 4191744 len 16384 2024-03-08T21:15:28.001 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3072 offset 4191744 len 16384 2024-03-08T21:15:28.001 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 3072 offset 4192256 len 1024 2024-03-08T21:15:28.001 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3072 offset 4192256 len 1024 2024-03-08T21:15:28.001 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 3072 offset 4192256 len 1024 2024-03-08T21:15:28.001 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3072 offset 4192256 len 1024 2024-03-08T21:15:28.001 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 3072 offset 4192256 len 2048 2024-03-08T21:15:28.001 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3072 offset 4192256 len 2048 2024-03-08T21:15:28.001 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 3072 offset 4192256 len 2048 2024-03-08T21:15:28.002 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3072 offset 4192256 len 2048 2024-03-08T21:15:28.002 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 3072 offset 4192256 len 4096 2024-03-08T21:15:28.002 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3072 offset 4192256 len 4096 2024-03-08T21:15:28.002 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 3072 offset 4192256 len 4096 2024-03-08T21:15:28.002 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3072 offset 4192256 len 4096 2024-03-08T21:15:28.002 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 3072 offset 4192256 len 8192 2024-03-08T21:15:28.002 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3072 offset 4192256 len 8192 2024-03-08T21:15:28.002 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 3072 offset 4192256 len 8192 2024-03-08T21:15:28.002 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3072 offset 4192256 len 8192 2024-03-08T21:15:28.002 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 3072 offset 4192256 len 16384 2024-03-08T21:15:28.002 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3072 offset 4192256 len 16384 2024-03-08T21:15:28.002 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 3072 offset 4192256 len 16384 2024-03-08T21:15:28.002 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3072 offset 4192256 len 16384 2024-03-08T21:15:28.002 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 3072 offset 4192768 len 1024 2024-03-08T21:15:28.002 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3072 offset 4192768 len 1024 2024-03-08T21:15:28.003 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 3072 offset 4192768 len 1024 2024-03-08T21:15:28.003 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3072 offset 4192768 len 1024 2024-03-08T21:15:28.003 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 3072 offset 4192768 len 2048 2024-03-08T21:15:28.003 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3072 offset 4192768 len 2048 2024-03-08T21:15:28.003 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 3072 offset 4192768 len 2048 2024-03-08T21:15:28.003 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3072 offset 4192768 len 2048 2024-03-08T21:15:28.003 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 3072 offset 4192768 len 4096 2024-03-08T21:15:28.003 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3072 offset 4192768 len 4096 2024-03-08T21:15:28.003 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 3072 offset 4192768 len 4096 2024-03-08T21:15:28.003 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3072 offset 4192768 len 4096 2024-03-08T21:15:28.003 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 3072 offset 4192768 len 8192 2024-03-08T21:15:28.003 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3072 offset 4192768 len 8192 2024-03-08T21:15:28.003 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 3072 offset 4192768 len 8192 2024-03-08T21:15:28.003 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3072 offset 4192768 len 8192 2024-03-08T21:15:28.003 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 3072 offset 4192768 len 16384 2024-03-08T21:15:28.003 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3072 offset 4192768 len 16384 2024-03-08T21:15:28.004 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 3072 offset 4192768 len 16384 2024-03-08T21:15:28.004 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3072 offset 4192768 len 16384 2024-03-08T21:15:28.004 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 3072 offset 4193280 len 1024 2024-03-08T21:15:28.872 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:15:28 smithi204 ceph-mon[34935]: pgmap v474: 97 pgs: 97 active+clean; 224 MiB data, 469 MiB used, 1.0 TiB / 1.0 TiB avail; 974 KiB/s rd, 1001 KiB/s wr, 171 op/s 2024-03-08T21:15:28.960 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:15:28 smithi107 ceph-mon[31381]: pgmap v474: 97 pgs: 97 active+clean; 224 MiB data, 469 MiB used, 1.0 TiB / 1.0 TiB avail; 974 KiB/s rd, 1001 KiB/s wr, 171 op/s 2024-03-08T21:15:29.008 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:15:28 smithi149 ceph-mon[35042]: pgmap v474: 97 pgs: 97 active+clean; 224 MiB data, 469 MiB used, 1.0 TiB / 1.0 TiB avail; 974 KiB/s rd, 1001 KiB/s wr, 171 op/s 2024-03-08T21:15:29.294 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3072 offset 4193280 len 1024 2024-03-08T21:15:29.294 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 3072 offset 4193280 len 1024 2024-03-08T21:15:29.294 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3072 offset 4193280 len 1024 2024-03-08T21:15:29.294 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 3072 offset 4193280 len 2048 2024-03-08T21:15:29.294 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3072 offset 4193280 len 2048 2024-03-08T21:15:29.294 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 3072 offset 4193280 len 2048 2024-03-08T21:15:29.294 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3072 offset 4193280 len 2048 2024-03-08T21:15:29.295 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 3072 offset 4193280 len 4096 2024-03-08T21:15:29.295 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3072 offset 4193280 len 4096 2024-03-08T21:15:29.295 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 3072 offset 4193280 len 4096 2024-03-08T21:15:29.295 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3072 offset 4193280 len 4096 2024-03-08T21:15:29.295 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 3072 offset 4193280 len 8192 2024-03-08T21:15:29.295 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3072 offset 4193280 len 8192 2024-03-08T21:15:29.295 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 3072 offset 4193280 len 8192 2024-03-08T21:15:29.295 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3072 offset 4193280 len 8192 2024-03-08T21:15:29.295 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 3072 offset 4193280 len 16384 2024-03-08T21:15:29.295 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3072 offset 4193280 len 16384 2024-03-08T21:15:29.295 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 3072 offset 4193280 len 16384 2024-03-08T21:15:29.295 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3072 offset 4193280 len 16384 2024-03-08T21:15:29.295 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 3072 offset 4193792 len 1024 2024-03-08T21:15:29.295 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3072 offset 4193792 len 1024 2024-03-08T21:15:29.296 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 3072 offset 4193792 len 1024 2024-03-08T21:15:29.296 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3072 offset 4193792 len 1024 2024-03-08T21:15:29.296 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 3072 offset 4193792 len 2048 2024-03-08T21:15:29.296 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3072 offset 4193792 len 2048 2024-03-08T21:15:29.296 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 3072 offset 4193792 len 2048 2024-03-08T21:15:29.296 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3072 offset 4193792 len 2048 2024-03-08T21:15:29.296 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 3072 offset 4193792 len 4096 2024-03-08T21:15:29.296 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3072 offset 4193792 len 4096 2024-03-08T21:15:29.296 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 3072 offset 4193792 len 4096 2024-03-08T21:15:29.296 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3072 offset 4193792 len 4096 2024-03-08T21:15:29.296 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 3072 offset 4193792 len 8192 2024-03-08T21:15:29.296 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3072 offset 4193792 len 8192 2024-03-08T21:15:29.296 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 3072 offset 4193792 len 8192 2024-03-08T21:15:29.296 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3072 offset 4193792 len 8192 2024-03-08T21:15:29.297 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 3072 offset 4193792 len 16384 2024-03-08T21:15:29.297 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3072 offset 4193792 len 16384 2024-03-08T21:15:29.297 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 3072 offset 4193792 len 16384 2024-03-08T21:15:29.297 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3072 offset 4193792 len 16384 2024-03-08T21:15:29.297 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 3072 offset 4194304 len 1024 2024-03-08T21:15:29.297 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3072 offset 4194304 len 1024 2024-03-08T21:15:29.297 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 3072 offset 4194304 len 1024 2024-03-08T21:15:29.297 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3072 offset 4194304 len 1024 2024-03-08T21:15:29.297 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 3072 offset 4194304 len 2048 2024-03-08T21:15:29.297 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3072 offset 4194304 len 2048 2024-03-08T21:15:29.297 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 3072 offset 4194304 len 2048 2024-03-08T21:15:29.297 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3072 offset 4194304 len 2048 2024-03-08T21:15:29.297 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 3072 offset 4194304 len 4096 2024-03-08T21:15:29.297 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3072 offset 4194304 len 4096 2024-03-08T21:15:29.298 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 3072 offset 4194304 len 4096 2024-03-08T21:15:29.298 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3072 offset 4194304 len 4096 2024-03-08T21:15:29.298 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 3072 offset 4194304 len 8192 2024-03-08T21:15:29.298 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3072 offset 4194304 len 8192 2024-03-08T21:15:29.298 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 3072 offset 4194304 len 8192 2024-03-08T21:15:29.298 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3072 offset 4194304 len 8192 2024-03-08T21:15:29.298 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 3072 offset 4194304 len 16384 2024-03-08T21:15:29.298 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3072 offset 4194304 len 16384 2024-03-08T21:15:29.298 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 3072 offset 4194304 len 16384 2024-03-08T21:15:29.298 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3072 offset 4194304 len 16384 2024-03-08T21:15:29.299 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 3072 offset 4194816 len 1024 2024-03-08T21:15:29.299 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3072 offset 4194816 len 1024 2024-03-08T21:15:29.299 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 3072 offset 4194816 len 1024 2024-03-08T21:15:29.299 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3072 offset 4194816 len 1024 2024-03-08T21:15:29.299 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 3072 offset 4194816 len 2048 2024-03-08T21:15:29.299 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3072 offset 4194816 len 2048 2024-03-08T21:15:29.299 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 3072 offset 4194816 len 2048 2024-03-08T21:15:29.299 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3072 offset 4194816 len 2048 2024-03-08T21:15:29.299 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 3072 offset 4194816 len 4096 2024-03-08T21:15:29.299 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3072 offset 4194816 len 4096 2024-03-08T21:15:29.299 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 3072 offset 4194816 len 4096 2024-03-08T21:15:29.299 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3072 offset 4194816 len 4096 2024-03-08T21:15:29.299 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 3072 offset 4194816 len 8192 2024-03-08T21:15:29.299 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3072 offset 4194816 len 8192 2024-03-08T21:15:29.300 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 3072 offset 4194816 len 8192 2024-03-08T21:15:29.300 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3072 offset 4194816 len 8192 2024-03-08T21:15:29.300 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 3072 offset 4194816 len 16384 2024-03-08T21:15:29.300 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3072 offset 4194816 len 16384 2024-03-08T21:15:29.300 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 3072 offset 4194816 len 16384 2024-03-08T21:15:29.300 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3072 offset 4194816 len 16384 2024-03-08T21:15:29.300 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 3072 offset 4195328 len 1024 2024-03-08T21:15:29.300 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3072 offset 4195328 len 1024 2024-03-08T21:15:29.583 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.605+0000 7f1c8fe00640 1 -- 172.21.15.107:0/1866924666 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1c901081b0 msgr2=0x7f1c901145c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:15:29.583 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.605+0000 7f1c8fe00640 1 --2- 172.21.15.107:0/1866924666 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1c901081b0 0x7f1c901145c0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f1c88009cd0 tx=0x7f1c8802c0f0 comp rx=0 tx=0).stop 2024-03-08T21:15:29.583 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.606+0000 7f1c8fe00640 1 -- 172.21.15.107:0/1866924666 shutdown_connections 2024-03-08T21:15:29.583 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.606+0000 7f1c8fe00640 1 --2- 172.21.15.107:0/1866924666 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1c901081b0 0x7f1c901145c0 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:15:29.583 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.606+0000 7f1c8fe00640 1 --2- 172.21.15.107:0/1866924666 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1c90107830 0x7f1c90107c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:15:29.583 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.606+0000 7f1c8fe00640 1 --2- 172.21.15.107:0/1866924666 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1c9010d700 0x7f1c9010dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:15:29.583 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.606+0000 7f1c8fe00640 1 -- 172.21.15.107:0/1866924666 >> 172.21.15.107:0/1866924666 conn(0x7f1c900770d0 msgr2=0x7f1c90106110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:15:29.583 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.606+0000 7f1c8fe00640 1 -- 172.21.15.107:0/1866924666 shutdown_connections 2024-03-08T21:15:29.584 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.606+0000 7f1c8fe00640 1 -- 172.21.15.107:0/1866924666 wait complete. 2024-03-08T21:15:29.585 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.607+0000 7f1c8fe00640 1 Processor -- start 2024-03-08T21:15:29.585 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.607+0000 7f1c8fe00640 1 -- start start 2024-03-08T21:15:29.585 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.607+0000 7f1c8fe00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1c90107830 0x7f1c9019df60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:15:29.585 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.607+0000 7f1c8fe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1c901081b0 0x7f1c9019e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:15:29.586 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.607+0000 7f1c8fe00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1c9010d700 0x7f1c901a2820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:15:29.586 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.607+0000 7f1c8fe00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c901179b0 con 0x7f1c901081b0 2024-03-08T21:15:29.586 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.607+0000 7f1c8fe00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c90117830 con 0x7f1c9010d700 2024-03-08T21:15:29.586 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.607+0000 7f1c8fe00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c90117b30 con 0x7f1c90107830 2024-03-08T21:15:29.586 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.608+0000 7f1c87e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1c901081b0 0x7f1c9019e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:15:29.586 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.608+0000 7f1c87e00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1c901081b0 0x7f1c9019e4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:34514/0 (socket says 172.21.15.107:34514) 2024-03-08T21:15:29.586 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.608+0000 7f1c87e00640 1 -- 172.21.15.107:0/977376194 learned_addr learned my addr 172.21.15.107:0/977376194 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:15:29.586 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.608+0000 7f1c8f400640 1 --2- 172.21.15.107:0/977376194 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1c9010d700 0x7f1c901a2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:15:29.586 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.608+0000 7f1c8ea00640 1 --2- 172.21.15.107:0/977376194 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1c90107830 0x7f1c9019df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:15:29.586 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.608+0000 7f1c87e00640 1 -- 172.21.15.107:0/977376194 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1c90107830 msgr2=0x7f1c9019df60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:15:29.586 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.608+0000 7f1c87e00640 1 --2- 172.21.15.107:0/977376194 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1c90107830 0x7f1c9019df60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:15:29.586 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.608+0000 7f1c87e00640 1 -- 172.21.15.107:0/977376194 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1c9010d700 msgr2=0x7f1c901a2820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:15:29.586 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.608+0000 7f1c87e00640 1 --2- 172.21.15.107:0/977376194 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1c9010d700 0x7f1c901a2820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:15:29.586 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.608+0000 7f1c87e00640 1 -- 172.21.15.107:0/977376194 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c901a2f50 con 0x7f1c901081b0 2024-03-08T21:15:29.587 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.608+0000 7f1c8ea00640 1 --2- 172.21.15.107:0/977376194 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1c90107830 0x7f1c9019df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T21:15:29.587 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.608+0000 7f1c8f400640 1 --2- 172.21.15.107:0/977376194 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1c9010d700 0x7f1c901a2820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T21:15:29.588 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.609+0000 7f1c87e00640 1 --2- 172.21.15.107:0/977376194 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1c901081b0 0x7f1c9019e4a0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f1c7800d770 tx=0x7f1c7800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:15:29.588 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.609+0000 7f1c87400640 1 -- 172.21.15.107:0/977376194 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c78017070 con 0x7f1c901081b0 2024-03-08T21:15:29.588 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.609+0000 7f1c87400640 1 -- 172.21.15.107:0/977376194 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f1c780054d0 con 0x7f1c901081b0 2024-03-08T21:15:29.588 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.609+0000 7f1c8fe00640 1 -- 172.21.15.107:0/977376194 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c901a3240 con 0x7f1c901081b0 2024-03-08T21:15:29.588 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.609+0000 7f1c87400640 1 -- 172.21.15.107:0/977376194 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c78004cf0 con 0x7f1c901081b0 2024-03-08T21:15:29.588 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.609+0000 7f1c8fe00640 1 -- 172.21.15.107:0/977376194 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c901aaad0 con 0x7f1c901081b0 2024-03-08T21:15:29.590 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.612+0000 7f1c87400640 1 -- 172.21.15.107:0/977376194 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) v1 ==== 137422+0+0 (secure 0 0 0) 0x7f1c78004e90 con 0x7f1c901081b0 2024-03-08T21:15:29.590 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.613+0000 7f1c989c0740 1 Processor -- start 2024-03-08T21:15:29.590 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.613+0000 7f1c989c0740 1 -- start start 2024-03-08T21:15:29.591 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.613+0000 7f1c989c0740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55daeb292e80 0x55daeb293260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:15:29.591 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.613+0000 7f1c989c0740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55daeb2937a0 0x55daeb29c030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:15:29.591 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.613+0000 7f1c989c0740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55daeb29c570 0x55daeb29e960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:15:29.592 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.613+0000 7f1c989c0740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55daeb1f63d0 con 0x55daeb292e80 2024-03-08T21:15:29.592 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.613+0000 7f1c989c0740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55daeb19f6b0 con 0x55daeb2937a0 2024-03-08T21:15:29.592 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.613+0000 7f1c989c0740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55daeb1f2470 con 0x55daeb29c570 2024-03-08T21:15:29.592 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.614+0000 7f1c8f400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55daeb292e80 0x55daeb293260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:15:29.592 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.614+0000 7f1c8f400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55daeb292e80 0x55daeb293260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:34520/0 (socket says 172.21.15.107:34520) 2024-03-08T21:15:29.592 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.614+0000 7f1c8f400640 1 -- 172.21.15.107:0/1622594416 learned_addr learned my addr 172.21.15.107:0/1622594416 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:15:29.592 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.614+0000 7f1c8ea00640 1 --2- 172.21.15.107:0/1622594416 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55daeb2937a0 0x55daeb29c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:15:29.592 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.614+0000 7f1c87e00640 1 --2- 172.21.15.107:0/1622594416 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55daeb29c570 0x55daeb29e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:15:29.592 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.614+0000 7f1c8f400640 1 -- 172.21.15.107:0/1622594416 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55daeb29c570 msgr2=0x55daeb29e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:15:29.592 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.614+0000 7f1c8f400640 1 --2- 172.21.15.107:0/1622594416 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55daeb29c570 0x55daeb29e960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:15:29.592 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.614+0000 7f1c8f400640 1 -- 172.21.15.107:0/1622594416 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55daeb2937a0 msgr2=0x55daeb29c030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:15:29.592 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.614+0000 7f1c8f400640 1 --2- 172.21.15.107:0/1622594416 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55daeb2937a0 0x55daeb29c030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:15:29.592 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.614+0000 7f1c8f400640 1 -- 172.21.15.107:0/1622594416 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55daeb1c5b50 con 0x55daeb292e80 2024-03-08T21:15:29.592 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.614+0000 7f1c87400640 1 --2- 172.21.15.107:0/977376194 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f1c740a2960 0x7f1c740a4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:15:29.593 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.614+0000 7f1c87400640 1 -- 172.21.15.107:0/977376194 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f1c780cea20 con 0x7f1c901081b0 2024-03-08T21:15:29.593 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.614+0000 7f1c87e00640 1 --2- 172.21.15.107:0/977376194 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f1c740a2960 0x7f1c740a4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:15:29.593 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.615+0000 7f1c8f400640 1 --2- 172.21.15.107:0/1622594416 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55daeb292e80 0x55daeb293260 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f1c88009ca0 tx=0x7f1c880323c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3aa6aad13d975485 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:15:29.596 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.615+0000 7f1c7fe00640 1 -- 172.21.15.107:0/1622594416 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c88043020 con 0x55daeb292e80 2024-03-08T21:15:29.596 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.615+0000 7f1c7fe00640 1 -- 172.21.15.107:0/1622594416 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f1c88032db0 con 0x55daeb292e80 2024-03-08T21:15:29.597 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.615+0000 7f1c7fe00640 1 -- 172.21.15.107:0/1622594416 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c88039040 con 0x55daeb292e80 2024-03-08T21:15:29.597 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.616+0000 7f1c989c0740 1 -- 172.21.15.107:0/1622594416 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55daeb292e80 msgr2=0x55daeb293260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:15:29.597 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.616+0000 7f1c989c0740 1 --2- 172.21.15.107:0/1622594416 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55daeb292e80 0x55daeb293260 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f1c88009ca0 tx=0x7f1c880323c0 comp rx=0 tx=0).stop 2024-03-08T21:15:29.597 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.616+0000 7f1c87e00640 1 --2- 172.21.15.107:0/977376194 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f1c740a2960 0x7f1c740a4e20 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x55daeb28e930 tx=0x7f1c780bb040 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:15:29.597 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.616+0000 7f1c989c0740 1 -- 172.21.15.107:0/1622594416 shutdown_connections 2024-03-08T21:15:29.597 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.616+0000 7f1c989c0740 1 --2- 172.21.15.107:0/1622594416 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55daeb29c570 0x55daeb29e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:15:29.597 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.616+0000 7f1c989c0740 1 --2- 172.21.15.107:0/1622594416 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55daeb2937a0 0x55daeb29c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:15:29.597 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.616+0000 7f1c989c0740 1 --2- 172.21.15.107:0/1622594416 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55daeb292e80 0x55daeb293260 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:15:29.597 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.616+0000 7f1c989c0740 1 -- 172.21.15.107:0/1622594416 >> 172.21.15.107:0/1622594416 conn(0x55daeb28d7a0 msgr2=0x55daeb28db80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:15:29.597 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.616+0000 7f1c989c0740 1 -- 172.21.15.107:0/1622594416 shutdown_connections 2024-03-08T21:15:29.597 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.616+0000 7f1c989c0740 1 -- 172.21.15.107:0/1622594416 wait complete. 2024-03-08T21:15:29.598 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.616+0000 7f1c989c0740 1 -- start start 2024-03-08T21:15:29.598 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.617+0000 7f1c989c0740 1 Processor -- start 2024-03-08T21:15:29.598 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.617+0000 7f1c989c0740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55daeb292e80 0x55daeb29be80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:15:29.598 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.617+0000 7f1c989c0740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55daeb2937a0 0x55daeb299210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:15:29.598 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.617+0000 7f1c989c0740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55daeb29c570 0x55daeb299750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:15:29.598 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.617+0000 7f1c989c0740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55daeb29f140 con 0x55daeb29c570 2024-03-08T21:15:29.598 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.617+0000 7f1c989c0740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x55daeb1b29f0 con 0x55daeb292e80 2024-03-08T21:15:29.598 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.617+0000 7f1c989c0740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x55daeb29f2c0 con 0x55daeb2937a0 2024-03-08T21:15:29.598 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.617+0000 7f1c989c0740 10 client.0 _ll_register_callbacks cb 0x55daeaf0aa00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T21:15:29.598 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.617+0000 7f1c8f400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55daeb29c570 0x55daeb299750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:15:29.598 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.618+0000 7f1c8f400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55daeb29c570 0x55daeb299750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:34530/0 (socket says 172.21.15.107:34530) 2024-03-08T21:15:29.598 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.618+0000 7f1c8f400640 1 -- 172.21.15.107:0/2013241349 learned_addr learned my addr 172.21.15.107:0/2013241349 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:15:29.598 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.618+0000 7f1c8ea00640 1 --2- 172.21.15.107:0/2013241349 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55daeb2937a0 0x55daeb299210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:15:29.599 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.618+0000 7f1c8f400640 1 --2- 172.21.15.107:0/2013241349 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55daeb292e80 0x55daeb29be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:15:29.599 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.618+0000 7f1c8f400640 1 -- 172.21.15.107:0/2013241349 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55daeb2937a0 msgr2=0x55daeb299210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:15:29.599 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.618+0000 7f1c8f400640 1 --2- 172.21.15.107:0/2013241349 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55daeb2937a0 0x55daeb299210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:15:29.599 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.618+0000 7f1c8f400640 1 -- 172.21.15.107:0/2013241349 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55daeb292e80 msgr2=0x55daeb29be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:15:29.599 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.618+0000 7f1c8f400640 1 --2- 172.21.15.107:0/2013241349 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55daeb292e80 0x55daeb29be80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:15:29.599 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.618+0000 7f1c8f400640 1 -- 172.21.15.107:0/2013241349 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55daeaf0b210 con 0x55daeb29c570 2024-03-08T21:15:29.599 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.618+0000 7f1c8f400640 1 --2- 172.21.15.107:0/2013241349 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55daeb29c570 0x55daeb299750 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f1c88009a20 tx=0x7f1c8802e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6acceb10bb871a10 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:15:29.599 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.618+0000 7f1c7cc00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-03-08T21:15:29.599 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.618+0000 7f1c7cc00640 1 -- 172.21.15.107:0/2013241349 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c88050020 con 0x55daeb29c570 2024-03-08T21:15:29.599 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.618+0000 7f1c7cc00640 1 -- 172.21.15.107:0/2013241349 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f1c88004240 con 0x55daeb29c570 2024-03-08T21:15:29.599 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.618+0000 7f1c989c0740 1 -- 172.21.15.107:0/2013241349 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1c88032db0 con 0x55daeb29c570 2024-03-08T21:15:29.599 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.619+0000 7f1c7cc00640 1 -- 172.21.15.107:0/2013241349 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1c8805b050 con 0x55daeb29c570 2024-03-08T21:15:29.599 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.619+0000 7f1c989c0740 10 client.16701 fetch_fsmap learned FSMap version 41 2024-03-08T21:15:29.600 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.619+0000 7f1c989c0740 1 -- 172.21.15.107:0/2013241349 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x55daeafd7620 con 0x55daeb29c570 2024-03-08T21:15:29.600 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.619+0000 7f1c7cc00640 1 -- 172.21.15.107:0/2013241349 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f1c88032db0 con 0x55daeb29c570 2024-03-08T21:15:29.600 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.619+0000 7f1c7cc00640 1 -- 172.21.15.107:0/2013241349 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 41) v1 ==== 7307+0+0 (secure 0 0 0) 0x7f1c8803f8a0 con 0x55daeb29c570 2024-03-08T21:15:29.600 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.619+0000 7f1c989c0740 10 client.16701 fetch_fsmap finished waiting for FSMap version 41 2024-03-08T21:15:29.600 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.619+0000 7f1c989c0740 10 client.16701 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-08T21:15:29.600 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.619+0000 7f1c989c0740 20 client.16701 populate_metadata read hostname 'smithi107' 2024-03-08T21:15:29.600 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.619+0000 7f1c989c0740 1 --2- 172.21.15.107:0/2013241349 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55daeb2ac1a0 0x55daeb2ae590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:15:29.600 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.619+0000 7f1c989c0740 4 client.16701 mds_command: new command op to 34475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-08T21:15:29.600 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.619+0000 7f1c989c0740 1 -- 172.21.15.107:0/2013241349 --> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55daeb203910 con 0x55daeb2ac1a0 2024-03-08T21:15:29.602 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.620+0000 7f1c8ea00640 1 --2- 172.21.15.107:0/2013241349 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55daeb2ac1a0 0x55daeb2ae590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:15:29.602 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.621+0000 7f1c8ea00640 1 --2- 172.21.15.107:0/2013241349 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55daeb2ac1a0 0x55daeb2ae590 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6dded317c86fc43c server_cookie=b6dda3ba46dc8c2d in_seq=0 out_seq=0 2024-03-08T21:15:29.602 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.621+0000 7f1c7cc00640 10 client.16701 ms_handle_connect on v2:172.21.15.149:6838/55290596 2024-03-08T21:15:29.602 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.624+0000 7f1c7cc00640 1 -- 172.21.15.107:0/2013241349 <== mds.0 v2:172.21.15.149:6838/55290596 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55daeb203910 con 0x55daeb2ac1a0 2024-03-08T21:15:29.602 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.624+0000 7f1c7cc00640 10 client.16701 handle_command_reply: tid=0 2024-03-08T21:15:29.602 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.624+0000 7f1c989c0740 1 client.16701 shutdown 2024-03-08T21:15:29.602 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.624+0000 7f1c989c0740 1 -- 172.21.15.107:0/2013241349 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55daeb29c570 msgr2=0x55daeb299750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:15:29.602 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.624+0000 7f1c989c0740 1 --2- 172.21.15.107:0/2013241349 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55daeb29c570 0x55daeb299750 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f1c88009a20 tx=0x7f1c8802e040 comp rx=0 tx=0).stop 2024-03-08T21:15:29.602 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.624+0000 7f1c989c0740 1 -- 172.21.15.107:0/2013241349 shutdown_connections 2024-03-08T21:15:29.602 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.624+0000 7f1c989c0740 1 --2- 172.21.15.107:0/2013241349 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55daeb29c570 0x55daeb299750 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:15:29.602 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.624+0000 7f1c989c0740 1 --2- 172.21.15.107:0/2013241349 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55daeb2937a0 0x55daeb299210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:15:29.602 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.624+0000 7f1c989c0740 1 --2- 172.21.15.107:0/2013241349 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55daeb2ac1a0 0x55daeb2ae590 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:15:29.603 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.624+0000 7f1c989c0740 1 --2- 172.21.15.107:0/2013241349 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55daeb292e80 0x55daeb29be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:15:29.603 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.624+0000 7f1c989c0740 1 -- 172.21.15.107:0/2013241349 >> 172.21.15.107:0/2013241349 conn(0x55daeb28d7a0 msgr2=0x55daeb28f4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:15:29.603 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.624+0000 7f1c7cc00640 0 client.16701 ms_handle_reset on v2:172.21.15.149:6838/55290596 2024-03-08T21:15:29.603 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.624+0000 7f1c989c0740 1 -- 172.21.15.107:0/2013241349 shutdown_connections 2024-03-08T21:15:29.603 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.624+0000 7f1c989c0740 1 -- 172.21.15.107:0/2013241349 wait complete. 2024-03-08T21:15:29.603 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.624+0000 7f1c989c0740 20 client.16701 trim_cache size 0 max 16384 2024-03-08T21:15:29.618 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.640+0000 7f1c989c0740 1 Processor -- start 2024-03-08T21:15:29.618 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.640+0000 7f1c989c0740 1 -- start start 2024-03-08T21:15:29.618 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.640+0000 7f1c989c0740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55daeb292e80 0x55daeb2981f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:15:29.618 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.640+0000 7f1c989c0740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55daeb2937a0 0x55daeb29b940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:15:29.618 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.640+0000 7f1c989c0740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55daeb29c570 0x55daeb29be80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:15:29.618 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.640+0000 7f1c989c0740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c901b49e0 con 0x55daeb2937a0 2024-03-08T21:15:29.618 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.640+0000 7f1c989c0740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c901b2a00 con 0x55daeb29c570 2024-03-08T21:15:29.618 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.640+0000 7f1c989c0740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c901b4380 con 0x55daeb292e80 2024-03-08T21:15:29.618 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.640+0000 7f1c8ea00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55daeb2937a0 0x55daeb29b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:15:29.618 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.640+0000 7f1c8ea00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55daeb2937a0 0x55daeb29b940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:34542/0 (socket says 172.21.15.107:34542) 2024-03-08T21:15:29.618 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.640+0000 7f1c8ea00640 1 -- 172.21.15.107:0/1886072789 learned_addr learned my addr 172.21.15.107:0/1886072789 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:15:29.619 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.641+0000 7f1c8ea00640 1 -- 172.21.15.107:0/1886072789 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55daeb292e80 msgr2=0x55daeb2981f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:15:29.619 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.641+0000 7f1c8f400640 1 --2- 172.21.15.107:0/1886072789 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55daeb292e80 0x55daeb2981f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:15:29.619 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.641+0000 7f1c8ea00640 1 --2- 172.21.15.107:0/1886072789 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55daeb292e80 0x55daeb2981f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:15:29.619 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.641+0000 7f1c8ea00640 1 -- 172.21.15.107:0/1886072789 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55daeb29c570 msgr2=0x55daeb29be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:15:29.619 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.641+0000 7f1c8ea00640 1 --2- 172.21.15.107:0/1886072789 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55daeb29c570 0x55daeb29be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:15:29.619 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.641+0000 7f1c8ea00640 1 -- 172.21.15.107:0/1886072789 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55daeb29e960 con 0x55daeb2937a0 2024-03-08T21:15:29.619 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.641+0000 7f1c8ea00640 1 --2- 172.21.15.107:0/1886072789 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55daeb2937a0 0x55daeb29b940 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f1c80005d80 tx=0x7f1c80005d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b883b0b53980bf6c server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:15:29.623 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.641+0000 7f1c7f400640 1 -- 172.21.15.107:0/1886072789 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c800149a0 con 0x55daeb2937a0 2024-03-08T21:15:29.623 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.641+0000 7f1c7f400640 1 -- 172.21.15.107:0/1886072789 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f1c80005030 con 0x55daeb2937a0 2024-03-08T21:15:29.623 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.641+0000 7f1c7f400640 1 -- 172.21.15.107:0/1886072789 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c8001a820 con 0x55daeb2937a0 2024-03-08T21:15:29.623 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.642+0000 7f1c989c0740 1 -- 172.21.15.107:0/1886072789 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55daeb2937a0 msgr2=0x55daeb29b940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:15:29.623 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.642+0000 7f1c989c0740 1 --2- 172.21.15.107:0/1886072789 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55daeb2937a0 0x55daeb29b940 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f1c80005d80 tx=0x7f1c80005d10 comp rx=0 tx=0).stop 2024-03-08T21:15:29.623 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.642+0000 7f1c989c0740 1 -- 172.21.15.107:0/1886072789 shutdown_connections 2024-03-08T21:15:29.623 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.642+0000 7f1c989c0740 1 --2- 172.21.15.107:0/1886072789 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55daeb29c570 0x55daeb29be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:15:29.623 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.642+0000 7f1c989c0740 1 --2- 172.21.15.107:0/1886072789 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55daeb2937a0 0x55daeb29b940 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:15:29.623 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.642+0000 7f1c989c0740 1 --2- 172.21.15.107:0/1886072789 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55daeb292e80 0x55daeb2981f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:15:29.623 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.642+0000 7f1c989c0740 1 -- 172.21.15.107:0/1886072789 >> 172.21.15.107:0/1886072789 conn(0x55daeb28d7a0 msgr2=0x55daeb297ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:15:29.623 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.642+0000 7f1c989c0740 1 -- 172.21.15.107:0/1886072789 shutdown_connections 2024-03-08T21:15:29.623 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.642+0000 7f1c989c0740 1 -- 172.21.15.107:0/1886072789 wait complete. 2024-03-08T21:15:29.624 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.643+0000 7f1c989c0740 1 -- start start 2024-03-08T21:15:29.624 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.643+0000 7f1c989c0740 1 Processor -- start 2024-03-08T21:15:29.624 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.643+0000 7f1c989c0740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55daeb292e80 0x55daeb2a3520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:15:29.624 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.643+0000 7f1c989c0740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55daeb2937a0 0x55daeb2a1a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:15:29.624 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.643+0000 7f1c989c0740 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55daeb29c570 0x55daeb2a1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:15:29.624 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.643+0000 7f1c989c0740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c901b36c0 con 0x55daeb2937a0 2024-03-08T21:15:29.624 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.643+0000 7f1c989c0740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c901b3060 con 0x55daeb292e80 2024-03-08T21:15:29.624 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.643+0000 7f1c989c0740 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c901b3d20 con 0x55daeb29c570 2024-03-08T21:15:29.624 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.643+0000 7f1c989c0740 10 client.0 _ll_register_callbacks cb 0x7f1c88037070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-08T21:15:29.624 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.643+0000 7f1c8ea00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55daeb2937a0 0x55daeb2a1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:15:29.624 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.644+0000 7f1c8ea00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55daeb2937a0 0x55daeb2a1a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:34554/0 (socket says 172.21.15.107:34554) 2024-03-08T21:15:29.624 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.644+0000 7f1c8ea00640 1 -- 172.21.15.107:0/2418909835 learned_addr learned my addr 172.21.15.107:0/2418909835 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:15:29.624 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.644+0000 7f1c8f400640 1 --2- 172.21.15.107:0/2418909835 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55daeb292e80 0x55daeb2a3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:15:29.624 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.644+0000 7f1c8f400640 1 --2- 172.21.15.107:0/2418909835 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55daeb29c570 0x55daeb2a1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:15:29.624 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.644+0000 7f1c8ea00640 1 -- 172.21.15.107:0/2418909835 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55daeb29c570 msgr2=0x55daeb2a1fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:15:29.625 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.644+0000 7f1c8ea00640 1 --2- 172.21.15.107:0/2418909835 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55daeb29c570 0x55daeb2a1fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:15:29.625 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.644+0000 7f1c8ea00640 1 -- 172.21.15.107:0/2418909835 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55daeb292e80 msgr2=0x55daeb2a3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-08T21:15:29.625 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.644+0000 7f1c8ea00640 1 --2- 172.21.15.107:0/2418909835 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55daeb292e80 0x55daeb2a3520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:15:29.625 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.644+0000 7f1c8ea00640 1 -- 172.21.15.107:0/2418909835 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55daeb2a24e0 con 0x55daeb2937a0 2024-03-08T21:15:29.625 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.644+0000 7f1c8f400640 1 --2- 172.21.15.107:0/2418909835 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55daeb292e80 0x55daeb2a3520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T21:15:29.625 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.644+0000 7f1c8ea00640 1 --2- 172.21.15.107:0/2418909835 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55daeb2937a0 0x55daeb2a1a60 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f1c8000b7e0 tx=0x7f1c8001a3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=581202492c61a6af server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:15:29.625 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.644+0000 7f1c7cc00640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-03-08T21:15:29.625 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.645+0000 7f1c7cc00640 1 -- 172.21.15.107:0/2418909835 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c8001a7a0 con 0x55daeb2937a0 2024-03-08T21:15:29.625 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.645+0000 7f1c7cc00640 1 -- 172.21.15.107:0/2418909835 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f1c8001a940 con 0x55daeb2937a0 2024-03-08T21:15:29.625 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.645+0000 7f1c989c0740 1 -- 172.21.15.107:0/2418909835 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1c80005030 con 0x55daeb2937a0 2024-03-08T21:15:29.625 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.645+0000 7f1c7cc00640 1 -- 172.21.15.107:0/2418909835 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1c800051e0 con 0x55daeb2937a0 2024-03-08T21:15:29.625 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.645+0000 7f1c989c0740 10 client.16707 fetch_fsmap learned FSMap version 41 2024-03-08T21:15:29.625 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.645+0000 7f1c989c0740 1 -- 172.21.15.107:0/2418909835 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x55daeb303050 con 0x55daeb2937a0 2024-03-08T21:15:29.625 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.645+0000 7f1c7cc00640 1 -- 172.21.15.107:0/2418909835 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f1c80005030 con 0x55daeb2937a0 2024-03-08T21:15:29.626 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.645+0000 7f1c7cc00640 1 -- 172.21.15.107:0/2418909835 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 41) v1 ==== 7307+0+0 (secure 0 0 0) 0x7f1c80032030 con 0x55daeb2937a0 2024-03-08T21:15:29.626 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.645+0000 7f1c989c0740 10 client.16707 fetch_fsmap finished waiting for FSMap version 41 2024-03-08T21:15:29.626 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.645+0000 7f1c989c0740 10 client.16707 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-03-08T21:15:29.626 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.645+0000 7f1c989c0740 20 client.16707 populate_metadata read hostname 'smithi107' 2024-03-08T21:15:29.626 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.646+0000 7f1c989c0740 1 --2- 172.21.15.107:0/2418909835 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55daeb303740 0x55daeb305b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:15:29.626 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.646+0000 7f1c989c0740 4 client.16707 mds_command: new command op to 34475 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-08T21:15:29.626 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.646+0000 7f1c989c0740 1 -- 172.21.15.107:0/2418909835 --> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55daeb1fdc60 con 0x55daeb303740 2024-03-08T21:15:29.627 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.646+0000 7f1c8f400640 1 --2- 172.21.15.107:0/2418909835 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55daeb303740 0x55daeb305b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:15:29.627 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.647+0000 7f1c8f400640 1 --2- 172.21.15.107:0/2418909835 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55daeb303740 0x55daeb305b60 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7ba34d144da78c7e server_cookie=df7d9a37603c7047 in_seq=0 out_seq=0 2024-03-08T21:15:29.627 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.647+0000 7f1c7cc00640 10 client.16707 ms_handle_connect on v2:172.21.15.149:6838/55290596 2024-03-08T21:15:29.694 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.716+0000 7f1c7cc00640 1 -- 172.21.15.107:0/2418909835 <== mds.0 v2:172.21.15.149:6838/55290596 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55daeb1fdc60 con 0x55daeb303740 2024-03-08T21:15:29.694 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.716+0000 7f1c7cc00640 10 client.16707 handle_command_reply: tid=0 2024-03-08T21:15:29.694 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.717+0000 7f1c989c0740 1 client.16707 shutdown 2024-03-08T21:15:29.695 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.718+0000 7f1c989c0740 1 -- 172.21.15.107:0/2418909835 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55daeb2937a0 msgr2=0x55daeb2a1a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:15:29.695 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.718+0000 7f1c989c0740 1 --2- 172.21.15.107:0/2418909835 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55daeb2937a0 0x55daeb2a1a60 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f1c8000b7e0 tx=0x7f1c8001a3c0 comp rx=0 tx=0).stop 2024-03-08T21:15:29.695 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.718+0000 7f1c989c0740 1 -- 172.21.15.107:0/2418909835 shutdown_connections 2024-03-08T21:15:29.695 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.718+0000 7f1c989c0740 1 --2- 172.21.15.107:0/2418909835 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x55daeb29c570 0x55daeb2a1fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:15:29.695 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.718+0000 7f1c989c0740 1 --2- 172.21.15.107:0/2418909835 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55daeb2937a0 0x55daeb2a1a60 secure :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f1c8000b7e0 tx=0x7f1c8001a3c0 comp rx=0 tx=0).stop 2024-03-08T21:15:29.695 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.718+0000 7f1c989c0740 1 --2- 172.21.15.107:0/2418909835 >> [v2:172.21.15.149:6838/55290596,v1:172.21.15.149:6839/55290596] conn(0x55daeb303740 0x55daeb305b60 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:15:29.696 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.718+0000 7f1c989c0740 1 --2- 172.21.15.107:0/2418909835 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55daeb292e80 0x55daeb2a3520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:15:29.696 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.718+0000 7f1c989c0740 1 -- 172.21.15.107:0/2418909835 >> 172.21.15.107:0/2418909835 conn(0x55daeb28d7a0 msgr2=0x55daeb28bf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:15:29.696 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.718+0000 7f1c7cc00640 0 client.16707 ms_handle_reset on v2:172.21.15.149:6838/55290596 2024-03-08T21:15:29.698 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.721+0000 7f1c989c0740 1 -- 172.21.15.107:0/2418909835 shutdown_connections 2024-03-08T21:15:29.699 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.722+0000 7f1c989c0740 1 -- 172.21.15.107:0/2418909835 wait complete. 2024-03-08T21:15:29.699 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.722+0000 7f1c989c0740 20 client.16707 trim_cache size 0 max 16384 2024-03-08T21:15:29.699 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:{ 2024-03-08T21:15:29.699 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "message": "", 2024-03-08T21:15:29.699 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout: "return_code": 0 2024-03-08T21:15:29.699 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stdout:} 2024-03-08T21:15:29.700 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.723+0000 7f1c8fe00640 1 -- 172.21.15.107:0/977376194 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f1c740a2960 msgr2=0x7f1c740a4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:15:29.700 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.723+0000 7f1c8fe00640 1 --2- 172.21.15.107:0/977376194 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f1c740a2960 0x7f1c740a4e20 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x55daeb28e930 tx=0x7f1c780bb040 comp rx=0 tx=0).stop 2024-03-08T21:15:29.700 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.723+0000 7f1c8fe00640 1 -- 172.21.15.107:0/977376194 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1c901081b0 msgr2=0x7f1c9019e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:15:29.700 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.723+0000 7f1c8fe00640 1 --2- 172.21.15.107:0/977376194 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1c901081b0 0x7f1c9019e4a0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f1c7800d770 tx=0x7f1c7800dc30 comp rx=0 tx=0).stop 2024-03-08T21:15:29.700 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.723+0000 7f1c8fe00640 1 -- 172.21.15.107:0/977376194 shutdown_connections 2024-03-08T21:15:29.700 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.723+0000 7f1c8fe00640 1 --2- 172.21.15.107:0/977376194 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f1c740a2960 0x7f1c740a4e20 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:15:29.700 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.723+0000 7f1c8fe00640 1 --2- 172.21.15.107:0/977376194 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1c9010d700 0x7f1c901a2820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:15:29.700 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.723+0000 7f1c8fe00640 1 --2- 172.21.15.107:0/977376194 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1c901081b0 0x7f1c9019e4a0 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:15:29.701 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.723+0000 7f1c8fe00640 1 --2- 172.21.15.107:0/977376194 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1c90107830 0x7f1c9019df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:15:29.701 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.723+0000 7f1c8fe00640 1 -- 172.21.15.107:0/977376194 >> 172.21.15.107:0/977376194 conn(0x7f1c900770d0 msgr2=0x7f1c90077790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:15:29.701 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.723+0000 7f1c8fe00640 1 -- 172.21.15.107:0/977376194 shutdown_connections 2024-03-08T21:15:29.701 INFO:teuthology.task.background_exec.ubuntu@smithi107.front.sepia.ceph.com.smithi107.stderr:2024-03-08T21:15:29.724+0000 7f1c8fe00640 1 -- 172.21.15.107:0/977376194 wait complete. 2024-03-08T21:15:30.399 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 3072 offset 4195328 len 1024 2024-03-08T21:15:30.399 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3072 offset 4195328 len 1024 2024-03-08T21:15:30.399 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 3072 offset 4195328 len 2048 2024-03-08T21:15:30.399 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3072 offset 4195328 len 2048 2024-03-08T21:15:30.399 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 3072 offset 4195328 len 2048 2024-03-08T21:15:30.399 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3072 offset 4195328 len 2048 2024-03-08T21:15:30.399 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 3072 offset 4195328 len 4096 2024-03-08T21:15:30.399 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3072 offset 4195328 len 4096 2024-03-08T21:15:30.400 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 3072 offset 4195328 len 4096 2024-03-08T21:15:30.400 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3072 offset 4195328 len 4096 2024-03-08T21:15:30.400 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 3072 offset 4195328 len 8192 2024-03-08T21:15:30.400 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3072 offset 4195328 len 8192 2024-03-08T21:15:30.400 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 3072 offset 4195328 len 8192 2024-03-08T21:15:30.400 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3072 offset 4195328 len 8192 2024-03-08T21:15:30.400 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 3072 offset 4195328 len 16384 2024-03-08T21:15:30.400 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3072 offset 4195328 len 16384 2024-03-08T21:15:30.400 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 3072 offset 4195328 len 16384 2024-03-08T21:15:30.400 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3072 offset 4195328 len 16384 2024-03-08T21:15:30.400 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 3072 offset 4195840 len 1024 2024-03-08T21:15:30.400 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3072 offset 4195840 len 1024 2024-03-08T21:15:30.400 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 3072 offset 4195840 len 1024 2024-03-08T21:15:30.400 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3072 offset 4195840 len 1024 2024-03-08T21:15:30.400 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 3072 offset 4195840 len 2048 2024-03-08T21:15:30.401 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3072 offset 4195840 len 2048 2024-03-08T21:15:30.401 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 3072 offset 4195840 len 2048 2024-03-08T21:15:30.401 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3072 offset 4195840 len 2048 2024-03-08T21:15:30.401 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 3072 offset 4195840 len 4096 2024-03-08T21:15:30.401 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3072 offset 4195840 len 4096 2024-03-08T21:15:30.401 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 3072 offset 4195840 len 4096 2024-03-08T21:15:30.401 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3072 offset 4195840 len 4096 2024-03-08T21:15:30.401 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 3072 offset 4195840 len 8192 2024-03-08T21:15:30.401 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3072 offset 4195840 len 8192 2024-03-08T21:15:30.401 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 3072 offset 4195840 len 8192 2024-03-08T21:15:30.401 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3072 offset 4195840 len 8192 2024-03-08T21:15:30.401 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 3072 offset 4195840 len 16384 2024-03-08T21:15:30.401 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3072 offset 4195840 len 16384 2024-03-08T21:15:30.401 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 3072 offset 4195840 len 16384 2024-03-08T21:15:30.401 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3072 offset 4195840 len 16384 2024-03-08T21:15:30.402 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 3072 offset 4196352 len 1024 2024-03-08T21:15:30.402 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3072 offset 4196352 len 1024 2024-03-08T21:15:30.402 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 3072 offset 4196352 len 1024 2024-03-08T21:15:30.402 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3072 offset 4196352 len 1024 2024-03-08T21:15:30.402 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 3072 offset 4196352 len 2048 2024-03-08T21:15:30.402 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3072 offset 4196352 len 2048 2024-03-08T21:15:30.402 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 3072 offset 4196352 len 2048 2024-03-08T21:15:30.402 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3072 offset 4196352 len 2048 2024-03-08T21:15:30.402 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 3072 offset 4196352 len 4096 2024-03-08T21:15:30.402 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3072 offset 4196352 len 4096 2024-03-08T21:15:30.402 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 3072 offset 4196352 len 4096 2024-03-08T21:15:30.402 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3072 offset 4196352 len 4096 2024-03-08T21:15:30.402 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 3072 offset 4196352 len 8192 2024-03-08T21:15:30.402 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3072 offset 4196352 len 8192 2024-03-08T21:15:30.402 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 3072 offset 4196352 len 8192 2024-03-08T21:15:30.403 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3072 offset 4196352 len 8192 2024-03-08T21:15:30.403 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 3072 offset 4196352 len 16384 2024-03-08T21:15:30.403 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3072 offset 4196352 len 16384 2024-03-08T21:15:30.403 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 3072 offset 4196352 len 16384 2024-03-08T21:15:30.403 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3072 offset 4196352 len 16384 2024-03-08T21:15:30.403 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 3072 offset 4196864 len 1024 2024-03-08T21:15:30.403 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3072 offset 4196864 len 1024 2024-03-08T21:15:30.403 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 3072 offset 4196864 len 1024 2024-03-08T21:15:30.403 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3072 offset 4196864 len 1024 2024-03-08T21:15:30.403 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 3072 offset 4196864 len 2048 2024-03-08T21:15:30.403 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3072 offset 4196864 len 2048 2024-03-08T21:15:30.403 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 3072 offset 4196864 len 2048 2024-03-08T21:15:30.403 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3072 offset 4196864 len 2048 2024-03-08T21:15:30.403 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 3072 offset 4196864 len 4096 2024-03-08T21:15:30.403 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3072 offset 4196864 len 4096 2024-03-08T21:15:30.403 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 3072 offset 4196864 len 4096 2024-03-08T21:15:30.404 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3072 offset 4196864 len 4096 2024-03-08T21:15:30.404 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 3072 offset 4196864 len 8192 2024-03-08T21:15:30.404 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3072 offset 4196864 len 8192 2024-03-08T21:15:30.404 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 3072 offset 4196864 len 8192 2024-03-08T21:15:30.404 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3072 offset 4196864 len 8192 2024-03-08T21:15:30.404 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 3072 offset 4196864 len 16384 2024-03-08T21:15:30.404 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3072 offset 4196864 len 16384 2024-03-08T21:15:30.404 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 3072 offset 4196864 len 16384 2024-03-08T21:15:30.404 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3072 offset 4196864 len 16384 2024-03-08T21:15:30.404 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 3072 offset 4197376 len 1024 2024-03-08T21:15:30.404 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3072 offset 4197376 len 1024 2024-03-08T21:15:30.404 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 3072 offset 4197376 len 1024 2024-03-08T21:15:30.871 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:15:30 smithi204 ceph-mon[34935]: pgmap v475: 97 pgs: 97 active+clean; 268 MiB data, 481 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 250 op/s 2024-03-08T21:15:30.961 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:15:30 smithi107 ceph-mon[31381]: pgmap v475: 97 pgs: 97 active+clean; 268 MiB data, 481 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 250 op/s 2024-03-08T21:15:31.008 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:15:30 smithi149 ceph-mon[35042]: pgmap v475: 97 pgs: 97 active+clean; 268 MiB data, 481 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 250 op/s 2024-03-08T21:15:31.497 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3072 offset 4197376 len 1024 2024-03-08T21:15:31.497 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 3072 offset 4197376 len 2048 2024-03-08T21:15:31.498 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3072 offset 4197376 len 2048 2024-03-08T21:15:31.498 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 3072 offset 4197376 len 2048 2024-03-08T21:15:31.498 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3072 offset 4197376 len 2048 2024-03-08T21:15:31.498 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 3072 offset 4197376 len 4096 2024-03-08T21:15:31.498 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3072 offset 4197376 len 4096 2024-03-08T21:15:31.498 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 3072 offset 4197376 len 4096 2024-03-08T21:15:31.498 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3072 offset 4197376 len 4096 2024-03-08T21:15:31.498 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 3072 offset 4197376 len 8192 2024-03-08T21:15:31.498 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3072 offset 4197376 len 8192 2024-03-08T21:15:31.498 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 3072 offset 4197376 len 8192 2024-03-08T21:15:31.498 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3072 offset 4197376 len 8192 2024-03-08T21:15:31.498 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 3072 offset 4197376 len 16384 2024-03-08T21:15:31.499 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3072 offset 4197376 len 16384 2024-03-08T21:15:31.499 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 3072 offset 4197376 len 16384 2024-03-08T21:15:31.499 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3072 offset 4197376 len 16384 2024-03-08T21:15:31.499 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 3072 offset 4197888 len 1024 2024-03-08T21:15:31.499 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3072 offset 4197888 len 1024 2024-03-08T21:15:31.499 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 3072 offset 4197888 len 1024 2024-03-08T21:15:31.499 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3072 offset 4197888 len 1024 2024-03-08T21:15:31.499 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 3072 offset 4197888 len 2048 2024-03-08T21:15:31.499 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3072 offset 4197888 len 2048 2024-03-08T21:15:31.499 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 3072 offset 4197888 len 2048 2024-03-08T21:15:31.499 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3072 offset 4197888 len 2048 2024-03-08T21:15:31.499 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 3072 offset 4197888 len 4096 2024-03-08T21:15:31.499 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3072 offset 4197888 len 4096 2024-03-08T21:15:31.499 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 3072 offset 4197888 len 4096 2024-03-08T21:15:31.500 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3072 offset 4197888 len 4096 2024-03-08T21:15:31.500 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 3072 offset 4197888 len 8192 2024-03-08T21:15:31.500 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3072 offset 4197888 len 8192 2024-03-08T21:15:31.500 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 3072 offset 4197888 len 8192 2024-03-08T21:15:31.500 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3072 offset 4197888 len 8192 2024-03-08T21:15:31.500 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 3072 offset 4197888 len 16384 2024-03-08T21:15:31.500 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3072 offset 4197888 len 16384 2024-03-08T21:15:31.500 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 3072 offset 4197888 len 16384 2024-03-08T21:15:31.500 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3072 offset 4197888 len 16384 2024-03-08T21:15:31.500 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 3584 offset 4190720 len 1024 2024-03-08T21:15:31.500 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3584 offset 4190720 len 1024 2024-03-08T21:15:31.500 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 3584 offset 4190720 len 1024 2024-03-08T21:15:31.500 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3584 offset 4190720 len 1024 2024-03-08T21:15:31.501 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 3584 offset 4190720 len 2048 2024-03-08T21:15:31.501 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3584 offset 4190720 len 2048 2024-03-08T21:15:31.501 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 3584 offset 4190720 len 2048 2024-03-08T21:15:31.501 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3584 offset 4190720 len 2048 2024-03-08T21:15:31.501 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 3584 offset 4190720 len 4096 2024-03-08T21:15:31.501 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3584 offset 4190720 len 4096 2024-03-08T21:15:31.501 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 3584 offset 4190720 len 4096 2024-03-08T21:15:31.501 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3584 offset 4190720 len 4096 2024-03-08T21:15:31.501 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 3584 offset 4190720 len 8192 2024-03-08T21:15:31.501 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3584 offset 4190720 len 8192 2024-03-08T21:15:31.501 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 3584 offset 4190720 len 8192 2024-03-08T21:15:31.501 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3584 offset 4190720 len 8192 2024-03-08T21:15:31.501 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 3584 offset 4190720 len 16384 2024-03-08T21:15:31.501 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3584 offset 4190720 len 16384 2024-03-08T21:15:31.502 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 3584 offset 4190720 len 16384 2024-03-08T21:15:31.502 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3584 offset 4190720 len 16384 2024-03-08T21:15:31.502 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 3584 offset 4191232 len 1024 2024-03-08T21:15:31.502 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3584 offset 4191232 len 1024 2024-03-08T21:15:31.502 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 3584 offset 4191232 len 1024 2024-03-08T21:15:31.502 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3584 offset 4191232 len 1024 2024-03-08T21:15:31.502 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 3584 offset 4191232 len 2048 2024-03-08T21:15:31.502 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3584 offset 4191232 len 2048 2024-03-08T21:15:31.502 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 3584 offset 4191232 len 2048 2024-03-08T21:15:31.502 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3584 offset 4191232 len 2048 2024-03-08T21:15:31.502 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 3584 offset 4191232 len 4096 2024-03-08T21:15:31.502 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3584 offset 4191232 len 4096 2024-03-08T21:15:31.502 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 3584 offset 4191232 len 4096 2024-03-08T21:15:31.502 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3584 offset 4191232 len 4096 2024-03-08T21:15:31.503 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 3584 offset 4191232 len 8192 2024-03-08T21:15:31.503 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3584 offset 4191232 len 8192 2024-03-08T21:15:31.503 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 3584 offset 4191232 len 8192 2024-03-08T21:15:31.503 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3584 offset 4191232 len 8192 2024-03-08T21:15:31.503 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 3584 offset 4191232 len 16384 2024-03-08T21:15:31.503 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3584 offset 4191232 len 16384 2024-03-08T21:15:31.503 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 3584 offset 4191232 len 16384 2024-03-08T21:15:31.503 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3584 offset 4191232 len 16384 2024-03-08T21:15:31.503 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 3584 offset 4191744 len 1024 2024-03-08T21:15:31.503 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3584 offset 4191744 len 1024 2024-03-08T21:15:31.503 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 3584 offset 4191744 len 1024 2024-03-08T21:15:31.503 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3584 offset 4191744 len 1024 2024-03-08T21:15:32.585 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 3584 offset 4191744 len 2048 2024-03-08T21:15:32.586 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3584 offset 4191744 len 2048 2024-03-08T21:15:32.586 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 3584 offset 4191744 len 2048 2024-03-08T21:15:32.586 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3584 offset 4191744 len 2048 2024-03-08T21:15:32.586 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 3584 offset 4191744 len 4096 2024-03-08T21:15:32.586 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3584 offset 4191744 len 4096 2024-03-08T21:15:32.586 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 3584 offset 4191744 len 4096 2024-03-08T21:15:32.586 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3584 offset 4191744 len 4096 2024-03-08T21:15:32.587 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 3584 offset 4191744 len 8192 2024-03-08T21:15:32.587 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3584 offset 4191744 len 8192 2024-03-08T21:15:32.587 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 3584 offset 4191744 len 8192 2024-03-08T21:15:32.587 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3584 offset 4191744 len 8192 2024-03-08T21:15:32.587 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 3584 offset 4191744 len 16384 2024-03-08T21:15:32.587 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3584 offset 4191744 len 16384 2024-03-08T21:15:32.587 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 3584 offset 4191744 len 16384 2024-03-08T21:15:32.587 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3584 offset 4191744 len 16384 2024-03-08T21:15:32.587 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 3584 offset 4192256 len 1024 2024-03-08T21:15:32.587 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3584 offset 4192256 len 1024 2024-03-08T21:15:32.587 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 3584 offset 4192256 len 1024 2024-03-08T21:15:32.587 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3584 offset 4192256 len 1024 2024-03-08T21:15:32.588 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 3584 offset 4192256 len 2048 2024-03-08T21:15:32.588 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3584 offset 4192256 len 2048 2024-03-08T21:15:32.588 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 3584 offset 4192256 len 2048 2024-03-08T21:15:32.588 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3584 offset 4192256 len 2048 2024-03-08T21:15:32.588 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 3584 offset 4192256 len 4096 2024-03-08T21:15:32.588 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3584 offset 4192256 len 4096 2024-03-08T21:15:32.588 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 3584 offset 4192256 len 4096 2024-03-08T21:15:32.588 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3584 offset 4192256 len 4096 2024-03-08T21:15:32.588 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 3584 offset 4192256 len 8192 2024-03-08T21:15:32.588 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3584 offset 4192256 len 8192 2024-03-08T21:15:32.588 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 3584 offset 4192256 len 8192 2024-03-08T21:15:32.588 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3584 offset 4192256 len 8192 2024-03-08T21:15:32.588 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 3584 offset 4192256 len 16384 2024-03-08T21:15:32.588 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3584 offset 4192256 len 16384 2024-03-08T21:15:32.589 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 3584 offset 4192256 len 16384 2024-03-08T21:15:32.589 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3584 offset 4192256 len 16384 2024-03-08T21:15:32.589 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 3584 offset 4192768 len 1024 2024-03-08T21:15:32.589 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3584 offset 4192768 len 1024 2024-03-08T21:15:32.589 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 3584 offset 4192768 len 1024 2024-03-08T21:15:32.589 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3584 offset 4192768 len 1024 2024-03-08T21:15:32.589 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 3584 offset 4192768 len 2048 2024-03-08T21:15:32.589 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3584 offset 4192768 len 2048 2024-03-08T21:15:32.589 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 3584 offset 4192768 len 2048 2024-03-08T21:15:32.589 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3584 offset 4192768 len 2048 2024-03-08T21:15:32.589 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 3584 offset 4192768 len 4096 2024-03-08T21:15:32.589 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3584 offset 4192768 len 4096 2024-03-08T21:15:32.589 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 3584 offset 4192768 len 4096 2024-03-08T21:15:32.589 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3584 offset 4192768 len 4096 2024-03-08T21:15:32.590 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 3584 offset 4192768 len 8192 2024-03-08T21:15:32.590 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3584 offset 4192768 len 8192 2024-03-08T21:15:32.590 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 3584 offset 4192768 len 8192 2024-03-08T21:15:32.590 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3584 offset 4192768 len 8192 2024-03-08T21:15:32.590 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 3584 offset 4192768 len 16384 2024-03-08T21:15:32.590 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3584 offset 4192768 len 16384 2024-03-08T21:15:32.590 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 3584 offset 4192768 len 16384 2024-03-08T21:15:32.590 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3584 offset 4192768 len 16384 2024-03-08T21:15:32.590 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 3584 offset 4193280 len 1024 2024-03-08T21:15:32.590 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3584 offset 4193280 len 1024 2024-03-08T21:15:32.590 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 3584 offset 4193280 len 1024 2024-03-08T21:15:32.590 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3584 offset 4193280 len 1024 2024-03-08T21:15:32.590 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 3584 offset 4193280 len 2048 2024-03-08T21:15:32.590 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3584 offset 4193280 len 2048 2024-03-08T21:15:32.591 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 3584 offset 4193280 len 2048 2024-03-08T21:15:32.591 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3584 offset 4193280 len 2048 2024-03-08T21:15:32.591 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 3584 offset 4193280 len 4096 2024-03-08T21:15:32.591 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3584 offset 4193280 len 4096 2024-03-08T21:15:32.591 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 3584 offset 4193280 len 4096 2024-03-08T21:15:32.591 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3584 offset 4193280 len 4096 2024-03-08T21:15:32.591 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 3584 offset 4193280 len 8192 2024-03-08T21:15:32.591 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3584 offset 4193280 len 8192 2024-03-08T21:15:32.591 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 3584 offset 4193280 len 8192 2024-03-08T21:15:32.591 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3584 offset 4193280 len 8192 2024-03-08T21:15:32.591 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 3584 offset 4193280 len 16384 2024-03-08T21:15:32.591 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3584 offset 4193280 len 16384 2024-03-08T21:15:32.591 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 3584 offset 4193280 len 16384 2024-03-08T21:15:32.591 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3584 offset 4193280 len 16384 2024-03-08T21:15:32.591 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 3584 offset 4193792 len 1024 2024-03-08T21:15:32.592 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3584 offset 4193792 len 1024 2024-03-08T21:15:32.592 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 3584 offset 4193792 len 1024 2024-03-08T21:15:32.592 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3584 offset 4193792 len 1024 2024-03-08T21:15:32.871 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:15:32 smithi204 ceph-mon[34935]: pgmap v476: 97 pgs: 97 active+clean; 268 MiB data, 481 MiB used, 1.0 TiB / 1.0 TiB avail; 881 KiB/s rd, 979 KiB/s wr, 155 op/s 2024-03-08T21:15:32.960 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:15:32 smithi107 ceph-mon[31381]: pgmap v476: 97 pgs: 97 active+clean; 268 MiB data, 481 MiB used, 1.0 TiB / 1.0 TiB avail; 881 KiB/s rd, 979 KiB/s wr, 155 op/s 2024-03-08T21:15:33.008 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:15:32 smithi149 ceph-mon[35042]: pgmap v476: 97 pgs: 97 active+clean; 268 MiB data, 481 MiB used, 1.0 TiB / 1.0 TiB avail; 881 KiB/s rd, 979 KiB/s wr, 155 op/s 2024-03-08T21:15:33.861 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 3584 offset 4193792 len 2048 2024-03-08T21:15:33.861 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3584 offset 4193792 len 2048 2024-03-08T21:15:33.861 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 3584 offset 4193792 len 2048 2024-03-08T21:15:33.861 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3584 offset 4193792 len 2048 2024-03-08T21:15:33.861 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 3584 offset 4193792 len 4096 2024-03-08T21:15:33.861 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3584 offset 4193792 len 4096 2024-03-08T21:15:33.861 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 3584 offset 4193792 len 4096 2024-03-08T21:15:33.861 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3584 offset 4193792 len 4096 2024-03-08T21:15:33.861 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 3584 offset 4193792 len 8192 2024-03-08T21:15:33.861 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3584 offset 4193792 len 8192 2024-03-08T21:15:33.862 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 3584 offset 4193792 len 8192 2024-03-08T21:15:33.862 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3584 offset 4193792 len 8192 2024-03-08T21:15:33.862 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 3584 offset 4193792 len 16384 2024-03-08T21:15:33.862 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3584 offset 4193792 len 16384 2024-03-08T21:15:33.862 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 3584 offset 4193792 len 16384 2024-03-08T21:15:33.862 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3584 offset 4193792 len 16384 2024-03-08T21:15:33.862 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 3584 offset 4194304 len 1024 2024-03-08T21:15:33.862 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3584 offset 4194304 len 1024 2024-03-08T21:15:33.862 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 3584 offset 4194304 len 1024 2024-03-08T21:15:33.862 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3584 offset 4194304 len 1024 2024-03-08T21:15:33.862 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 3584 offset 4194304 len 2048 2024-03-08T21:15:33.862 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3584 offset 4194304 len 2048 2024-03-08T21:15:33.862 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 3584 offset 4194304 len 2048 2024-03-08T21:15:33.863 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3584 offset 4194304 len 2048 2024-03-08T21:15:33.863 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 3584 offset 4194304 len 4096 2024-03-08T21:15:33.863 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3584 offset 4194304 len 4096 2024-03-08T21:15:33.863 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 3584 offset 4194304 len 4096 2024-03-08T21:15:33.863 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3584 offset 4194304 len 4096 2024-03-08T21:15:33.863 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 3584 offset 4194304 len 8192 2024-03-08T21:15:33.863 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3584 offset 4194304 len 8192 2024-03-08T21:15:33.863 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 3584 offset 4194304 len 8192 2024-03-08T21:15:33.863 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3584 offset 4194304 len 8192 2024-03-08T21:15:33.863 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 3584 offset 4194304 len 16384 2024-03-08T21:15:33.863 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3584 offset 4194304 len 16384 2024-03-08T21:15:33.863 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 3584 offset 4194304 len 16384 2024-03-08T21:15:33.863 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3584 offset 4194304 len 16384 2024-03-08T21:15:33.863 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 3584 offset 4194816 len 1024 2024-03-08T21:15:33.864 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3584 offset 4194816 len 1024 2024-03-08T21:15:33.864 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 3584 offset 4194816 len 1024 2024-03-08T21:15:33.864 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3584 offset 4194816 len 1024 2024-03-08T21:15:33.864 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 3584 offset 4194816 len 2048 2024-03-08T21:15:33.864 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3584 offset 4194816 len 2048 2024-03-08T21:15:33.864 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 3584 offset 4194816 len 2048 2024-03-08T21:15:33.864 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3584 offset 4194816 len 2048 2024-03-08T21:15:33.864 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 3584 offset 4194816 len 4096 2024-03-08T21:15:33.864 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3584 offset 4194816 len 4096 2024-03-08T21:15:33.864 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 3584 offset 4194816 len 4096 2024-03-08T21:15:33.864 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3584 offset 4194816 len 4096 2024-03-08T21:15:33.864 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 3584 offset 4194816 len 8192 2024-03-08T21:15:33.864 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3584 offset 4194816 len 8192 2024-03-08T21:15:33.864 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 3584 offset 4194816 len 8192 2024-03-08T21:15:33.864 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3584 offset 4194816 len 8192 2024-03-08T21:15:33.865 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 3584 offset 4194816 len 16384 2024-03-08T21:15:33.865 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3584 offset 4194816 len 16384 2024-03-08T21:15:33.865 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 3584 offset 4194816 len 16384 2024-03-08T21:15:33.865 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3584 offset 4194816 len 16384 2024-03-08T21:15:33.865 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 3584 offset 4195328 len 1024 2024-03-08T21:15:33.865 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3584 offset 4195328 len 1024 2024-03-08T21:15:33.865 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 3584 offset 4195328 len 1024 2024-03-08T21:15:33.865 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3584 offset 4195328 len 1024 2024-03-08T21:15:33.865 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 3584 offset 4195328 len 2048 2024-03-08T21:15:33.865 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3584 offset 4195328 len 2048 2024-03-08T21:15:33.865 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 3584 offset 4195328 len 2048 2024-03-08T21:15:33.865 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3584 offset 4195328 len 2048 2024-03-08T21:15:33.865 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 3584 offset 4195328 len 4096 2024-03-08T21:15:33.865 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3584 offset 4195328 len 4096 2024-03-08T21:15:33.866 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 3584 offset 4195328 len 4096 2024-03-08T21:15:33.866 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3584 offset 4195328 len 4096 2024-03-08T21:15:33.866 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 3584 offset 4195328 len 8192 2024-03-08T21:15:33.866 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3584 offset 4195328 len 8192 2024-03-08T21:15:33.866 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 3584 offset 4195328 len 8192 2024-03-08T21:15:33.866 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3584 offset 4195328 len 8192 2024-03-08T21:15:33.866 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 3584 offset 4195328 len 16384 2024-03-08T21:15:33.866 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3584 offset 4195328 len 16384 2024-03-08T21:15:33.866 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 3584 offset 4195328 len 16384 2024-03-08T21:15:33.866 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3584 offset 4195328 len 16384 2024-03-08T21:15:33.866 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 3584 offset 4195840 len 1024 2024-03-08T21:15:33.866 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3584 offset 4195840 len 1024 2024-03-08T21:15:33.866 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 3584 offset 4195840 len 1024 2024-03-08T21:15:33.866 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3584 offset 4195840 len 1024 2024-03-08T21:15:33.866 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 3584 offset 4195840 len 2048 2024-03-08T21:15:34.871 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:15:34 smithi204 ceph-mon[34935]: pgmap v477: 97 pgs: 97 active+clean; 268 MiB data, 481 MiB used, 1.0 TiB / 1.0 TiB avail; 881 KiB/s rd, 979 KiB/s wr, 155 op/s 2024-03-08T21:15:34.960 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:15:34 smithi107 ceph-mon[31381]: pgmap v477: 97 pgs: 97 active+clean; 268 MiB data, 481 MiB used, 1.0 TiB / 1.0 TiB avail; 881 KiB/s rd, 979 KiB/s wr, 155 op/s 2024-03-08T21:15:35.008 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:15:34 smithi149 ceph-mon[35042]: pgmap v477: 97 pgs: 97 active+clean; 268 MiB data, 481 MiB used, 1.0 TiB / 1.0 TiB avail; 881 KiB/s rd, 979 KiB/s wr, 155 op/s 2024-03-08T21:15:35.227 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3584 offset 4195840 len 2048 2024-03-08T21:15:35.228 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 3584 offset 4195840 len 2048 2024-03-08T21:15:35.228 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3584 offset 4195840 len 2048 2024-03-08T21:15:35.228 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 3584 offset 4195840 len 4096 2024-03-08T21:15:35.228 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3584 offset 4195840 len 4096 2024-03-08T21:15:35.228 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 3584 offset 4195840 len 4096 2024-03-08T21:15:35.228 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3584 offset 4195840 len 4096 2024-03-08T21:15:35.228 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 3584 offset 4195840 len 8192 2024-03-08T21:15:35.228 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3584 offset 4195840 len 8192 2024-03-08T21:15:35.228 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 3584 offset 4195840 len 8192 2024-03-08T21:15:35.228 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3584 offset 4195840 len 8192 2024-03-08T21:15:35.228 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 3584 offset 4195840 len 16384 2024-03-08T21:15:35.228 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3584 offset 4195840 len 16384 2024-03-08T21:15:35.229 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 3584 offset 4195840 len 16384 2024-03-08T21:15:35.229 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3584 offset 4195840 len 16384 2024-03-08T21:15:35.229 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 3584 offset 4196352 len 1024 2024-03-08T21:15:35.229 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3584 offset 4196352 len 1024 2024-03-08T21:15:35.229 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 3584 offset 4196352 len 1024 2024-03-08T21:15:35.229 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3584 offset 4196352 len 1024 2024-03-08T21:15:35.229 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 3584 offset 4196352 len 2048 2024-03-08T21:15:35.229 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3584 offset 4196352 len 2048 2024-03-08T21:15:35.229 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 3584 offset 4196352 len 2048 2024-03-08T21:15:35.229 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3584 offset 4196352 len 2048 2024-03-08T21:15:35.229 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 3584 offset 4196352 len 4096 2024-03-08T21:15:35.229 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3584 offset 4196352 len 4096 2024-03-08T21:15:35.229 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 3584 offset 4196352 len 4096 2024-03-08T21:15:35.229 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3584 offset 4196352 len 4096 2024-03-08T21:15:35.230 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 3584 offset 4196352 len 8192 2024-03-08T21:15:35.230 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3584 offset 4196352 len 8192 2024-03-08T21:15:35.230 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 3584 offset 4196352 len 8192 2024-03-08T21:15:35.230 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3584 offset 4196352 len 8192 2024-03-08T21:15:35.230 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 3584 offset 4196352 len 16384 2024-03-08T21:15:35.230 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3584 offset 4196352 len 16384 2024-03-08T21:15:35.230 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 3584 offset 4196352 len 16384 2024-03-08T21:15:35.230 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3584 offset 4196352 len 16384 2024-03-08T21:15:35.230 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 3584 offset 4196864 len 1024 2024-03-08T21:15:35.230 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3584 offset 4196864 len 1024 2024-03-08T21:15:35.230 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 3584 offset 4196864 len 1024 2024-03-08T21:15:35.230 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3584 offset 4196864 len 1024 2024-03-08T21:15:35.230 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 3584 offset 4196864 len 2048 2024-03-08T21:15:35.230 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3584 offset 4196864 len 2048 2024-03-08T21:15:35.230 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 3584 offset 4196864 len 2048 2024-03-08T21:15:35.231 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3584 offset 4196864 len 2048 2024-03-08T21:15:35.231 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 3584 offset 4196864 len 4096 2024-03-08T21:15:35.231 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3584 offset 4196864 len 4096 2024-03-08T21:15:35.231 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 3584 offset 4196864 len 4096 2024-03-08T21:15:35.231 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3584 offset 4196864 len 4096 2024-03-08T21:15:35.231 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 3584 offset 4196864 len 8192 2024-03-08T21:15:35.231 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3584 offset 4196864 len 8192 2024-03-08T21:15:35.231 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 3584 offset 4196864 len 8192 2024-03-08T21:15:35.231 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3584 offset 4196864 len 8192 2024-03-08T21:15:35.231 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 3584 offset 4196864 len 16384 2024-03-08T21:15:35.231 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3584 offset 4196864 len 16384 2024-03-08T21:15:35.231 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 3584 offset 4196864 len 16384 2024-03-08T21:15:35.231 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3584 offset 4196864 len 16384 2024-03-08T21:15:35.231 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 3584 offset 4197376 len 1024 2024-03-08T21:15:35.231 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3584 offset 4197376 len 1024 2024-03-08T21:15:35.232 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 3584 offset 4197376 len 1024 2024-03-08T21:15:35.232 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3584 offset 4197376 len 1024 2024-03-08T21:15:35.232 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 3584 offset 4197376 len 2048 2024-03-08T21:15:35.232 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3584 offset 4197376 len 2048 2024-03-08T21:15:35.232 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 3584 offset 4197376 len 2048 2024-03-08T21:15:35.232 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3584 offset 4197376 len 2048 2024-03-08T21:15:35.232 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 3584 offset 4197376 len 4096 2024-03-08T21:15:35.232 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3584 offset 4197376 len 4096 2024-03-08T21:15:35.232 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 3584 offset 4197376 len 4096 2024-03-08T21:15:35.232 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3584 offset 4197376 len 4096 2024-03-08T21:15:35.232 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 3584 offset 4197376 len 8192 2024-03-08T21:15:35.232 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3584 offset 4197376 len 8192 2024-03-08T21:15:35.232 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 3584 offset 4197376 len 8192 2024-03-08T21:15:35.232 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3584 offset 4197376 len 8192 2024-03-08T21:15:35.232 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 3584 offset 4197376 len 16384 2024-03-08T21:15:35.233 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3584 offset 4197376 len 16384 2024-03-08T21:15:35.233 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 3584 offset 4197376 len 16384 2024-03-08T21:15:35.233 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3584 offset 4197376 len 16384 2024-03-08T21:15:35.233 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 3584 offset 4197888 len 1024 2024-03-08T21:15:35.233 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3584 offset 4197888 len 1024 2024-03-08T21:15:35.233 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 3584 offset 4197888 len 1024 2024-03-08T21:15:35.233 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3584 offset 4197888 len 1024 2024-03-08T21:15:35.233 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 3584 offset 4197888 len 2048 2024-03-08T21:15:35.233 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3584 offset 4197888 len 2048 2024-03-08T21:15:35.450 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 3584 offset 4197888 len 2048 2024-03-08T21:15:35.450 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3584 offset 4197888 len 2048 2024-03-08T21:15:35.450 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 3584 offset 4197888 len 4096 2024-03-08T21:15:35.450 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3584 offset 4197888 len 4096 2024-03-08T21:15:35.450 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 3584 offset 4197888 len 4096 2024-03-08T21:15:35.451 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3584 offset 4197888 len 4096 2024-03-08T21:15:35.451 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 3584 offset 4197888 len 8192 2024-03-08T21:15:35.451 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3584 offset 4197888 len 8192 2024-03-08T21:15:35.451 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 3584 offset 4197888 len 8192 2024-03-08T21:15:35.451 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3584 offset 4197888 len 8192 2024-03-08T21:15:35.451 INFO:tasks.workunit.client.0.smithi107.stdout:write_direct buf_align 3584 offset 4197888 len 16384 2024-03-08T21:15:35.451 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3584 offset 4197888 len 16384 2024-03-08T21:15:35.451 INFO:tasks.workunit.client.0.smithi107.stdout:write_sync buf_align 3584 offset 4197888 len 16384 2024-03-08T21:15:35.451 INFO:tasks.workunit.client.0.smithi107.stdout:write_file buf_align 3584 offset 4197888 len 16384 2024-03-08T21:15:35.452 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-03-08T21:15:35.453 DEBUG:teuthology.orchestra.run.smithi107:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-03-08T21:15:35.634 INFO:tasks.workunit:Stopping ['direct_io'] on client.0... 2024-03-08T21:15:35.635 DEBUG:teuthology.orchestra.run.smithi107:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-03-08T21:15:36.683 DEBUG:teuthology.parallel:result is None 2024-03-08T21:15:36.684 DEBUG:teuthology.orchestra.run.smithi107:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-03-08T21:15:36.821 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-03-08T21:15:36.821 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2024-03-08T21:15:36.832 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi107.front.sepia.ceph.com 2024-03-08T21:15:36.852 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:15:36 smithi107 ceph-mon[31381]: pgmap v478: 97 pgs: 97 active+clean; 144 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 251 op/s 2024-03-08T21:15:36.856 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2024-03-08T21:15:36.865 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2024-03-08T21:15:36.865 INFO:teuthology.task.exec:Executing custom commands... 2024-03-08T21:15:36.865 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi107.front.sepia.ceph.com 2024-03-08T21:15:36.865 DEBUG:teuthology.orchestra.run.smithi107:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2024-03-08T21:15:36.871 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:15:36 smithi204 ceph-mon[34935]: pgmap v478: 97 pgs: 97 active+clean; 144 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 251 op/s 2024-03-08T21:15:36.998 DEBUG:teuthology.orchestra.run.smithi107:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2024-03-08T21:15:37.008 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:15:36 smithi149 ceph-mon[35042]: pgmap v478: 97 pgs: 97 active+clean; 144 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 251 op/s 2024-03-08T21:15:37.094 DEBUG:teuthology.orchestra.run.smithi107:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-03-08T21:15:37.303 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:37.325+0000 7fc8d2000640 1 -- 172.21.15.107:0/1801581802 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc8cc105190 msgr2=0x7fc8cc105570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:15:37.303 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:37.325+0000 7fc8d2000640 1 --2- 172.21.15.107:0/1801581802 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc8cc105190 0x7fc8cc105570 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fc8bc009900 tx=0x7fc8bc02bc30 comp rx=0 tx=0).stop 2024-03-08T21:15:37.303 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:37.326+0000 7fc8d2000640 1 -- 172.21.15.107:0/1801581802 shutdown_connections 2024-03-08T21:15:37.303 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:37.326+0000 7fc8d2000640 1 --2- 172.21.15.107:0/1801581802 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc8cc100e00 0x7fc8cc10d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:15:37.303 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:37.326+0000 7fc8d2000640 1 --2- 172.21.15.107:0/1801581802 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc8cc100440 0x7fc8cc1008c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:15:37.303 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:37.326+0000 7fc8d2000640 1 --2- 172.21.15.107:0/1801581802 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc8cc105190 0x7fc8cc105570 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:15:37.303 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:37.326+0000 7fc8d2000640 1 -- 172.21.15.107:0/1801581802 >> 172.21.15.107:0/1801581802 conn(0x7fc8cc077040 msgr2=0x7fc8cc077450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:15:37.303 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:37.326+0000 7fc8d2000640 1 -- 172.21.15.107:0/1801581802 shutdown_connections 2024-03-08T21:15:37.304 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:37.326+0000 7fc8d2000640 1 -- 172.21.15.107:0/1801581802 wait complete. 2024-03-08T21:15:37.305 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:37.327+0000 7fc8d2000640 1 Processor -- start 2024-03-08T21:15:37.305 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:37.327+0000 7fc8d2000640 1 -- start start 2024-03-08T21:15:37.305 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:37.327+0000 7fc8d2000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc8cc100440 0x7fc8cc199db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:15:37.305 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:37.327+0000 7fc8d2000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc8cc100e00 0x7fc8cc19a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:15:37.305 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:37.327+0000 7fc8d2000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc8cc105190 0x7fc8cc19e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:15:37.305 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:37.327+0000 7fc8d2000640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8cc110630 con 0x7fc8cc100440 2024-03-08T21:15:37.305 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:37.327+0000 7fc8d2000640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8cc1104b0 con 0x7fc8cc100e00 2024-03-08T21:15:37.305 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:37.327+0000 7fc8d2000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8cc1107b0 con 0x7fc8cc105190 2024-03-08T21:15:37.305 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:37.328+0000 7fc8cb400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc8cc100440 0x7fc8cc199db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:15:37.305 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:37.328+0000 7fc8cb400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc8cc100440 0x7fc8cc199db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:58098/0 (socket says 172.21.15.107:58098) 2024-03-08T21:15:37.305 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:37.328+0000 7fc8cb400640 1 -- 172.21.15.107:0/3016925345 learned_addr learned my addr 172.21.15.107:0/3016925345 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:15:37.307 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:37.328+0000 7fc8cb400640 1 -- 172.21.15.107:0/3016925345 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc8cc105190 msgr2=0x7fc8cc19e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:15:37.307 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:37.328+0000 7fc8cbe00640 1 --2- 172.21.15.107:0/3016925345 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc8cc105190 0x7fc8cc19e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:15:37.307 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:37.328+0000 7fc8cb400640 1 --2- 172.21.15.107:0/3016925345 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc8cc105190 0x7fc8cc19e670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:15:37.307 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:37.328+0000 7fc8cb400640 1 -- 172.21.15.107:0/3016925345 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc8cc100e00 msgr2=0x7fc8cc19a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:15:37.307 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:37.328+0000 7fc8cb400640 1 --2- 172.21.15.107:0/3016925345 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc8cc100e00 0x7fc8cc19a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:15:37.307 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:37.328+0000 7fc8cb400640 1 -- 172.21.15.107:0/3016925345 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8cc19eda0 con 0x7fc8cc100440 2024-03-08T21:15:37.307 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:37.329+0000 7fc8cb400640 1 --2- 172.21.15.107:0/3016925345 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc8cc100440 0x7fc8cc199db0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fc8bc000c00 tx=0x7fc8bc02ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:15:37.307 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:37.329+0000 7fc8c3e00640 1 -- 172.21.15.107:0/3016925345 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8bc043020 con 0x7fc8cc100440 2024-03-08T21:15:37.307 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:37.329+0000 7fc8d2000640 1 -- 172.21.15.107:0/3016925345 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8cc19f030 con 0x7fc8cc100440 2024-03-08T21:15:37.307 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:37.329+0000 7fc8c3e00640 1 -- 172.21.15.107:0/3016925345 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fc8bc02cc20 con 0x7fc8cc100440 2024-03-08T21:15:37.308 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:37.329+0000 7fc8c3e00640 1 -- 172.21.15.107:0/3016925345 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8bc003c20 con 0x7fc8cc100440 2024-03-08T21:15:37.308 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:37.329+0000 7fc8d2000640 1 -- 172.21.15.107:0/3016925345 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8cc1a6920 con 0x7fc8cc100440 2024-03-08T21:15:37.308 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:37.330+0000 7fc8d2000640 1 -- 172.21.15.107:0/3016925345 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8cc04f050 con 0x7fc8cc100440 2024-03-08T21:15:37.309 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:37.332+0000 7fc8c3e00640 1 -- 172.21.15.107:0/3016925345 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) v1 ==== 137422+0+0 (secure 0 0 0) 0x7fc8bc046020 con 0x7fc8cc100440 2024-03-08T21:15:37.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:37.335+0000 7fc8c3e00640 1 --2- 172.21.15.107:0/3016925345 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7fc8b00a2a10 0x7fc8b00a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:15:37.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:37.335+0000 7fc8c3e00640 1 -- 172.21.15.107:0/3016925345 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fc8bc0f12b0 con 0x7fc8cc100440 2024-03-08T21:15:37.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:37.335+0000 7fc8caa00640 1 --2- 172.21.15.107:0/3016925345 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7fc8b00a2a10 0x7fc8b00a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:15:37.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:37.336+0000 7fc8caa00640 1 --2- 172.21.15.107:0/3016925345 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7fc8b00a2a10 0x7fc8b00a4ed0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fc8cc10d3d0 tx=0x7fc8b8006c90 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:15:37.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:37.339+0000 7fc8c3e00640 1 -- 172.21.15.107:0/3016925345 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7fc8bc04d020 con 0x7fc8cc100440 2024-03-08T21:15:37.540 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:37.563+0000 7fc8d2000640 1 -- 172.21.15.107:0/3016925345 --> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) v1 -- 0x7fc8cc069080 con 0x7fc8b00a2a10 2024-03-08T21:15:37.556 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:37.580+0000 7fc8c3e00640 1 -- 172.21.15.107:0/3016925345 <== mgr.15375 v2:172.21.15.107:6800/2792338347 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+350 (secure 0 0 0) 0x7fc8cc069080 con 0x7fc8b00a2a10 2024-03-08T21:15:37.557 INFO:teuthology.orchestra.run.smithi107.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-03-08T00:00:00", "created": "2024-03-08T21:00:33", "first": "2024-03-08T21:01:00", "last": "2024-03-08T21:15:00", "last_pruned": "2024-03-08T21:15:00", "created_count": 15, "pruned_count": 9, "active": true} 2024-03-08T21:15:37.560 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:37.582+0000 7fc8d2000640 1 -- 172.21.15.107:0/3016925345 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7fc8b00a2a10 msgr2=0x7fc8b00a4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:15:37.560 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:37.582+0000 7fc8d2000640 1 --2- 172.21.15.107:0/3016925345 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7fc8b00a2a10 0x7fc8b00a4ed0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fc8cc10d3d0 tx=0x7fc8b8006c90 comp rx=0 tx=0).stop 2024-03-08T21:15:37.560 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:37.582+0000 7fc8d2000640 1 -- 172.21.15.107:0/3016925345 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc8cc100440 msgr2=0x7fc8cc199db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:15:37.560 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:37.582+0000 7fc8d2000640 1 --2- 172.21.15.107:0/3016925345 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc8cc100440 0x7fc8cc199db0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fc8bc000c00 tx=0x7fc8bc02ca30 comp rx=0 tx=0).stop 2024-03-08T21:15:37.560 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:37.583+0000 7fc8d2000640 1 -- 172.21.15.107:0/3016925345 shutdown_connections 2024-03-08T21:15:37.560 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:37.583+0000 7fc8d2000640 1 --2- 172.21.15.107:0/3016925345 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7fc8b00a2a10 0x7fc8b00a4ed0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:15:37.560 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:37.583+0000 7fc8d2000640 1 --2- 172.21.15.107:0/3016925345 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc8cc105190 0x7fc8cc19e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:15:37.560 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:37.583+0000 7fc8d2000640 1 --2- 172.21.15.107:0/3016925345 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc8cc100e00 0x7fc8cc19a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:15:37.560 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:37.583+0000 7fc8d2000640 1 --2- 172.21.15.107:0/3016925345 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc8cc100440 0x7fc8cc199db0 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:15:37.560 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:37.583+0000 7fc8d2000640 1 -- 172.21.15.107:0/3016925345 >> 172.21.15.107:0/3016925345 conn(0x7fc8cc077040 msgr2=0x7fc8cc068970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:15:37.560 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:37.583+0000 7fc8d2000640 1 -- 172.21.15.107:0/3016925345 shutdown_connections 2024-03-08T21:15:37.560 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:37.583+0000 7fc8d2000640 1 -- 172.21.15.107:0/3016925345 wait complete. 2024-03-08T21:15:37.579 DEBUG:teuthology.orchestra.run.smithi107:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-03-08T21:15:37.785 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:37.808+0000 7f265d400640 1 -- 172.21.15.107:0/1319471287 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f26581034c0 msgr2=0x7f2658103940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:15:37.785 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:37.808+0000 7f265d400640 1 --2- 172.21.15.107:0/1319471287 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f26581034c0 0x7f2658103940 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f2640009900 tx=0x7f264002bc30 comp rx=0 tx=0).stop 2024-03-08T21:15:37.785 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:37.808+0000 7f265d400640 1 -- 172.21.15.107:0/1319471287 shutdown_connections 2024-03-08T21:15:37.786 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:37.808+0000 7f265d400640 1 --2- 172.21.15.107:0/1319471287 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2658103e80 0x7f2658110340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:15:37.786 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:37.808+0000 7f265d400640 1 --2- 172.21.15.107:0/1319471287 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f26581034c0 0x7f2658103940 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:15:37.786 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:37.808+0000 7f265d400640 1 --2- 172.21.15.107:0/1319471287 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2658109580 0x7f2658109960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:15:37.786 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:37.808+0000 7f265d400640 1 -- 172.21.15.107:0/1319471287 >> 172.21.15.107:0/1319471287 conn(0x7f26580fef70 msgr2=0x7f2658101390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:15:37.786 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:37.808+0000 7f265d400640 1 -- 172.21.15.107:0/1319471287 shutdown_connections 2024-03-08T21:15:37.786 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:37.808+0000 7f265d400640 1 -- 172.21.15.107:0/1319471287 wait complete. 2024-03-08T21:15:37.788 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:37.809+0000 7f265d400640 1 Processor -- start 2024-03-08T21:15:37.788 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:37.809+0000 7f265d400640 1 -- start start 2024-03-08T21:15:37.788 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:37.810+0000 7f265d400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f26581034c0 0x7f265819f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:15:37.788 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:37.810+0000 7f265d400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2658103e80 0x7f265819fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:15:37.788 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:37.810+0000 7f265d400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2658109580 0x7f2658199940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:15:37.788 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:37.810+0000 7f265d400640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2658113760 con 0x7f2658109580 2024-03-08T21:15:37.788 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:37.810+0000 7f265d400640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26581135e0 con 0x7f2658103e80 2024-03-08T21:15:37.789 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:37.810+0000 7f265d400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26581138e0 con 0x7f26581034c0 2024-03-08T21:15:37.789 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:37.810+0000 7f2657200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2658109580 0x7f2658199940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:15:37.789 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:37.810+0000 7f2657200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2658109580 0x7f2658199940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:58106/0 (socket says 172.21.15.107:58106) 2024-03-08T21:15:37.789 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:37.810+0000 7f2657200640 1 -- 172.21.15.107:0/922386137 learned_addr learned my addr 172.21.15.107:0/922386137 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:15:37.789 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:37.810+0000 7f2656800640 1 --2- 172.21.15.107:0/922386137 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f26581034c0 0x7f265819f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:15:37.789 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:37.810+0000 7f264de00640 1 --2- 172.21.15.107:0/922386137 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2658103e80 0x7f265819fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:15:37.789 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:37.810+0000 7f2657200640 1 -- 172.21.15.107:0/922386137 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f26581034c0 msgr2=0x7f265819f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:15:37.789 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:37.810+0000 7f2657200640 1 --2- 172.21.15.107:0/922386137 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f26581034c0 0x7f265819f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:15:37.789 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:37.810+0000 7f2657200640 1 -- 172.21.15.107:0/922386137 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2658103e80 msgr2=0x7f265819fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:15:37.789 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:37.810+0000 7f2657200640 1 --2- 172.21.15.107:0/922386137 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2658103e80 0x7f265819fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:15:37.789 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:37.811+0000 7f2657200640 1 -- 172.21.15.107:0/922386137 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f265819a0c0 con 0x7f2658109580 2024-03-08T21:15:37.789 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:37.811+0000 7f2657200640 1 --2- 172.21.15.107:0/922386137 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2658109580 0x7f2658199940 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f265000cce0 tx=0x7f265000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:15:37.789 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:37.811+0000 7f264fe00640 1 -- 172.21.15.107:0/922386137 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f265000f450 con 0x7f2658109580 2024-03-08T21:15:37.789 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:37.811+0000 7f264fe00640 1 -- 172.21.15.107:0/922386137 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f265000faa0 con 0x7f2658109580 2024-03-08T21:15:37.789 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:37.811+0000 7f265d400640 1 -- 172.21.15.107:0/922386137 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f265819a3b0 con 0x7f2658109580 2024-03-08T21:15:37.790 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:37.811+0000 7f264fe00640 1 -- 172.21.15.107:0/922386137 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2650018b70 con 0x7f2658109580 2024-03-08T21:15:37.790 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:37.811+0000 7f265d400640 1 -- 172.21.15.107:0/922386137 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26581a6650 con 0x7f2658109580 2024-03-08T21:15:37.790 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:37.812+0000 7f265d400640 1 -- 172.21.15.107:0/922386137 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f265804f050 con 0x7f2658109580 2024-03-08T21:15:37.791 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:37.814+0000 7f264fe00640 1 -- 172.21.15.107:0/922386137 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) v1 ==== 137422+0+0 (secure 0 0 0) 0x7f2650018d10 con 0x7f2658109580 2024-03-08T21:15:37.793 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:37.816+0000 7f264fe00640 1 --2- 172.21.15.107:0/922386137 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f263c0a2a90 0x7f263c0a4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:15:37.794 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:37.817+0000 7f264fe00640 1 -- 172.21.15.107:0/922386137 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f26500cf370 con 0x7f2658109580 2024-03-08T21:15:37.794 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:37.817+0000 7f2656800640 1 --2- 172.21.15.107:0/922386137 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f263c0a2a90 0x7f263c0a4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:15:37.794 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:37.818+0000 7f2656800640 1 --2- 172.21.15.107:0/922386137 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f263c0a2a90 0x7f263c0a4f50 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f2648005fd0 tx=0x7f2648005850 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:15:37.798 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:37.821+0000 7f264fe00640 1 -- 172.21.15.107:0/922386137 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f2650014030 con 0x7f2658109580 2024-03-08T21:15:38.018 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:38.041+0000 7f265d400640 1 -- 172.21.15.107:0/922386137 --> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f2658100be0 con 0x7f263c0a2a90 2024-03-08T21:15:38.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:38.056+0000 7f264fe00640 1 -- 172.21.15.107:0/922386137 <== mgr.15375 v2:172.21.15.107:6800/2792338347 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+10 (secure 0 0 0) 0x7f2658100be0 con 0x7f263c0a2a90 2024-03-08T21:15:38.033 INFO:teuthology.orchestra.run.smithi107.stdout:/ 1m 6m3h 2024-03-08T21:15:38.036 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:38.059+0000 7f265d400640 1 -- 172.21.15.107:0/922386137 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f263c0a2a90 msgr2=0x7f263c0a4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:15:38.036 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:38.059+0000 7f265d400640 1 --2- 172.21.15.107:0/922386137 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f263c0a2a90 0x7f263c0a4f50 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f2648005fd0 tx=0x7f2648005850 comp rx=0 tx=0).stop 2024-03-08T21:15:38.036 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:38.059+0000 7f265d400640 1 -- 172.21.15.107:0/922386137 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2658109580 msgr2=0x7f2658199940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:15:38.036 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:38.059+0000 7f265d400640 1 --2- 172.21.15.107:0/922386137 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2658109580 0x7f2658199940 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f265000cce0 tx=0x7f265000a620 comp rx=0 tx=0).stop 2024-03-08T21:15:38.036 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:38.059+0000 7f265d400640 1 -- 172.21.15.107:0/922386137 shutdown_connections 2024-03-08T21:15:38.036 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:38.059+0000 7f265d400640 1 --2- 172.21.15.107:0/922386137 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f263c0a2a90 0x7f263c0a4f50 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:15:38.036 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:38.059+0000 7f265d400640 1 --2- 172.21.15.107:0/922386137 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2658109580 0x7f2658199940 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:15:38.036 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:38.059+0000 7f265d400640 1 --2- 172.21.15.107:0/922386137 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2658103e80 0x7f265819fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:15:38.036 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:38.059+0000 7f265d400640 1 --2- 172.21.15.107:0/922386137 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f26581034c0 0x7f265819f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:15:38.036 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:38.059+0000 7f265d400640 1 -- 172.21.15.107:0/922386137 >> 172.21.15.107:0/922386137 conn(0x7f26580fef70 msgr2=0x7f26581004d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:15:38.037 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:38.059+0000 7f265d400640 1 -- 172.21.15.107:0/922386137 shutdown_connections 2024-03-08T21:15:38.037 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:38.059+0000 7f265d400640 1 -- 172.21.15.107:0/922386137 wait complete. 2024-03-08T21:15:38.055 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-03-08T21:15:38.066 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-08T21:15:38.282 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:38.305+0000 7fa7a9600640 1 -- 172.21.15.107:0/3133968560 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa7a410d8b0 msgr2=0x7fa7a410dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:15:38.282 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:38.305+0000 7fa7a9600640 1 --2- 172.21.15.107:0/3133968560 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa7a410d8b0 0x7fa7a410dc90 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7fa794009930 tx=0x7fa79402bc20 comp rx=0 tx=0).stop 2024-03-08T21:15:38.282 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:38.305+0000 7fa7a9600640 1 -- 172.21.15.107:0/3133968560 shutdown_connections 2024-03-08T21:15:38.282 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:38.305+0000 7fa7a9600640 1 --2- 172.21.15.107:0/3133968560 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa7a4108260 0x7fa7a4114770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:15:38.283 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:38.305+0000 7fa7a9600640 1 --2- 172.21.15.107:0/3133968560 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa7a41078a0 0x7fa7a4107d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:15:38.283 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:38.305+0000 7fa7a9600640 1 --2- 172.21.15.107:0/3133968560 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa7a410d8b0 0x7fa7a410dc90 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:15:38.283 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:38.305+0000 7fa7a9600640 1 -- 172.21.15.107:0/3133968560 >> 172.21.15.107:0/3133968560 conn(0x7fa7a4075840 msgr2=0x7fa7a4077c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:15:38.283 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:38.305+0000 7fa7a9600640 1 -- 172.21.15.107:0/3133968560 shutdown_connections 2024-03-08T21:15:38.283 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:38.305+0000 7fa7a9600640 1 -- 172.21.15.107:0/3133968560 wait complete. 2024-03-08T21:15:38.285 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:38.306+0000 7fa7a9600640 1 Processor -- start 2024-03-08T21:15:38.285 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:38.306+0000 7fa7a9600640 1 -- start start 2024-03-08T21:15:38.285 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:38.306+0000 7fa7a9600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa7a41078a0 0x7fa7a419e120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:15:38.285 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:38.306+0000 7fa7a9600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa7a4108260 0x7fa7a419e660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:15:38.285 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:38.306+0000 7fa7a9600640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa7a410d8b0 0x7fa7a41a29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:15:38.285 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:38.306+0000 7fa7a9600640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7a4117bc0 con 0x7fa7a41078a0 2024-03-08T21:15:38.285 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:38.306+0000 7fa7a9600640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7a4117a40 con 0x7fa7a410d8b0 2024-03-08T21:15:38.285 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:38.306+0000 7fa7a9600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7a4117d40 con 0x7fa7a4108260 2024-03-08T21:15:38.286 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:38.306+0000 7fa7a2a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa7a41078a0 0x7fa7a419e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:15:38.286 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:38.307+0000 7fa7a2a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa7a41078a0 0x7fa7a419e120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:58112/0 (socket says 172.21.15.107:58112) 2024-03-08T21:15:38.286 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:38.307+0000 7fa7a2a00640 1 -- 172.21.15.107:0/2135183702 learned_addr learned my addr 172.21.15.107:0/2135183702 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:15:38.286 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:38.307+0000 7fa7a2a00640 1 -- 172.21.15.107:0/2135183702 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa7a4108260 msgr2=0x7fa7a419e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:15:38.286 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:38.307+0000 7fa7a2a00640 1 --2- 172.21.15.107:0/2135183702 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa7a4108260 0x7fa7a419e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:15:38.286 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:38.307+0000 7fa7a2a00640 1 -- 172.21.15.107:0/2135183702 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa7a410d8b0 msgr2=0x7fa7a41a29e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:15:38.286 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:38.307+0000 7fa7a2a00640 1 --2- 172.21.15.107:0/2135183702 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa7a410d8b0 0x7fa7a41a29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:15:38.286 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:38.307+0000 7fa7a2a00640 1 -- 172.21.15.107:0/2135183702 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7a41a3110 con 0x7fa7a41078a0 2024-03-08T21:15:38.286 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:38.307+0000 7fa7a2a00640 1 --2- 172.21.15.107:0/2135183702 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa7a41078a0 0x7fa7a419e120 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7fa794000c00 tx=0x7fa79402c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:15:38.286 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:38.307+0000 7fa79b400640 1 -- 172.21.15.107:0/2135183702 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa794043020 con 0x7fa7a41078a0 2024-03-08T21:15:38.286 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:38.308+0000 7fa79b400640 1 -- 172.21.15.107:0/2135183702 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa79402cc50 con 0x7fa7a41078a0 2024-03-08T21:15:38.286 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:38.308+0000 7fa7a9600640 1 -- 172.21.15.107:0/2135183702 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7a41a33a0 con 0x7fa7a41078a0 2024-03-08T21:15:38.286 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:38.308+0000 7fa79b400640 1 -- 172.21.15.107:0/2135183702 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa794003c30 con 0x7fa7a41078a0 2024-03-08T21:15:38.287 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:38.308+0000 7fa7a9600640 1 -- 172.21.15.107:0/2135183702 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7a41aac90 con 0x7fa7a41078a0 2024-03-08T21:15:38.287 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:38.309+0000 7fa7a9600640 1 -- 172.21.15.107:0/2135183702 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7a4068530 con 0x7fa7a41078a0 2024-03-08T21:15:38.288 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:38.311+0000 7fa79b400640 1 -- 172.21.15.107:0/2135183702 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) v1 ==== 137422+0+0 (secure 0 0 0) 0x7fa794046020 con 0x7fa7a41078a0 2024-03-08T21:15:38.289 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:38.312+0000 7fa79b400640 1 --2- 172.21.15.107:0/2135183702 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7fa7880a2970 0x7fa7880a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:15:38.289 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:38.312+0000 7fa79b400640 1 -- 172.21.15.107:0/2135183702 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fa7940f0fb0 con 0x7fa7a41078a0 2024-03-08T21:15:38.289 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:38.312+0000 7fa79be00640 1 --2- 172.21.15.107:0/2135183702 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7fa7880a2970 0x7fa7880a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:15:38.290 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:38.313+0000 7fa79be00640 1 --2- 172.21.15.107:0/2135183702 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7fa7880a2970 0x7fa7880a4e30 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fa78c005fd0 tx=0x7fa78c005850 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:15:38.293 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:38.317+0000 7fa79b400640 1 -- 172.21.15.107:0/2135183702 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7fa79404d020 con 0x7fa7a41078a0 2024-03-08T21:15:38.494 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:38.517+0000 7fa7a9600640 1 -- 172.21.15.107:0/2135183702 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa7a410f880 con 0x7fa7a41078a0 2024-03-08T21:15:38.498 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:38.522+0000 7fa79b400640 1 -- 172.21.15.107:0/2135183702 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v113) v1 ==== 74+0+23317 (secure 0 0 0) 0x7fa7940bdd80 con 0x7fa7a41078a0 2024-03-08T21:15:38.499 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-08T21:15:38.499 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":113,"fsid":"dc20f0f8-dd8c-11ee-95c6-87774f69a715","created":"2024-03-08T20:47:04.190034+0000","modified":"2024-03-08T21:15:14.344992+0000","last_up_change":"2024-03-08T20:56:43.135514+0000","last_in_change":"2024-03-08T20:56:17.577074+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-08T20:51:32.832786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-08T20:57:48.721227+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-08T20:57:49.707779+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":25,"snap_epoch":112,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8b1ddc64-4d20-4e85-ac66-f1c5ac599501","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":2453946095},{"type":"v1","addr":"172.21.15.107:6803","nonce":2453946095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":2453946095},{"type":"v1","addr":"172.21.15.107:6805","nonce":2453946095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":2453946095},{"type":"v1","addr":"172.21.15.107:6809","nonce":2453946095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":2453946095},{"type":"v1","addr":"172.21.15.107:6807","nonce":2453946095}]},"public_addr":"172.21.15.107:6803/2453946095","cluster_addr":"172.21.15.107:6805/2453946095","heartbeat_back_addr":"172.21.15.107:6809/2453946095","heartbeat_front_addr":"172.21.15.107:6807/2453946095","state":["exists","up"]},{"osd":1,"uuid":"5a803c09-202a-4777-ba2a-1da8eaf26a54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":3071316419},{"type":"v1","addr":"172.21.15.149:6801","nonce":3071316419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":3071316419},{"type":"v1","addr":"172.21.15.149:6803","nonce":3071316419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":3071316419},{"type":"v1","addr":"172.21.15.149:6807","nonce":3071316419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":3071316419},{"type":"v1","addr":"172.21.15.149:6805","nonce":3071316419}]},"public_addr":"172.21.15.149:6801/3071316419","cluster_addr":"172.21.15.149:6803/3071316419","heartbeat_back_addr":"172.21.15.149:6807/3071316419","heartbeat_front_addr":"172.21.15.149:6805/3071316419","state":["exists","up"]},{"osd":2,"uuid":"555c634f-347e-47dd-a943-e50ae2be073c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":1087852630},{"type":"v1","addr":"172.21.15.204:6801","nonce":1087852630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":1087852630},{"type":"v1","addr":"172.21.15.204:6803","nonce":1087852630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":1087852630},{"type":"v1","addr":"172.21.15.204:6807","nonce":1087852630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":1087852630},{"type":"v1","addr":"172.21.15.204:6805","nonce":1087852630}]},"public_addr":"172.21.15.204:6801/1087852630","cluster_addr":"172.21.15.204:6803/1087852630","heartbeat_back_addr":"172.21.15.204:6807/1087852630","heartbeat_front_addr":"172.21.15.204:6805/1087852630","state":["exists","up"]},{"osd":3,"uuid":"9e433d97-e202-48e0-9ed1-f0988fa14307","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":3123579789},{"type":"v1","addr":"172.21.15.107:6811","nonce":3123579789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":3123579789},{"type":"v1","addr":"172.21.15.107:6813","nonce":3123579789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":3123579789},{"type":"v1","addr":"172.21.15.107:6817","nonce":3123579789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":3123579789},{"type":"v1","addr":"172.21.15.107:6815","nonce":3123579789}]},"public_addr":"172.21.15.107:6811/3123579789","cluster_addr":"172.21.15.107:6813/3123579789","heartbeat_back_addr":"172.21.15.107:6817/3123579789","heartbeat_front_addr":"172.21.15.107:6815/3123579789","state":["exists","up"]},{"osd":4,"uuid":"27c53794-9d2b-487d-bfcf-ac4fd2ab061e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":726286918},{"type":"v1","addr":"172.21.15.149:6809","nonce":726286918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":726286918},{"type":"v1","addr":"172.21.15.149:6811","nonce":726286918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":726286918},{"type":"v1","addr":"172.21.15.149:6815","nonce":726286918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":726286918},{"type":"v1","addr":"172.21.15.149:6813","nonce":726286918}]},"public_addr":"172.21.15.149:6809/726286918","cluster_addr":"172.21.15.149:6811/726286918","heartbeat_back_addr":"172.21.15.149:6815/726286918","heartbeat_front_addr":"172.21.15.149:6813/726286918","state":["exists","up"]},{"osd":5,"uuid":"3a7778fb-46c5-40c0-842e-e988c45ea24d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":3143887820},{"type":"v1","addr":"172.21.15.204:6809","nonce":3143887820}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":3143887820},{"type":"v1","addr":"172.21.15.204:6811","nonce":3143887820}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":3143887820},{"type":"v1","addr":"172.21.15.204:6815","nonce":3143887820}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":3143887820},{"type":"v1","addr":"172.21.15.204:6813","nonce":3143887820}]},"public_addr":"172.21.15.204:6809/3143887820","cluster_addr":"172.21.15.204:6811/3143887820","heartbeat_back_addr":"172.21.15.204:6815/3143887820","heartbeat_front_addr":"172.21.15.204:6813/3143887820","state":["exists","up"]},{"osd":6,"uuid":"d2e05a56-be4f-4435-9b76-ff9aa01e4ac3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":2912452950},{"type":"v1","addr":"172.21.15.107:6819","nonce":2912452950}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":2912452950},{"type":"v1","addr":"172.21.15.107:6821","nonce":2912452950}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":2912452950},{"type":"v1","addr":"172.21.15.107:6825","nonce":2912452950}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":2912452950},{"type":"v1","addr":"172.21.15.107:6823","nonce":2912452950}]},"public_addr":"172.21.15.107:6819/2912452950","cluster_addr":"172.21.15.107:6821/2912452950","heartbeat_back_addr":"172.21.15.107:6825/2912452950","heartbeat_front_addr":"172.21.15.107:6823/2912452950","state":["exists","up"]},{"osd":7,"uuid":"a7dd26ee-c118-4a9a-9e19-142c2680902d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":3835863809},{"type":"v1","addr":"172.21.15.149:6817","nonce":3835863809}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":3835863809},{"type":"v1","addr":"172.21.15.149:6819","nonce":3835863809}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":3835863809},{"type":"v1","addr":"172.21.15.149:6823","nonce":3835863809}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":3835863809},{"type":"v1","addr":"172.21.15.149:6821","nonce":3835863809}]},"public_addr":"172.21.15.149:6817/3835863809","cluster_addr":"172.21.15.149:6819/3835863809","heartbeat_back_addr":"172.21.15.149:6823/3835863809","heartbeat_front_addr":"172.21.15.149:6821/3835863809","state":["exists","up"]},{"osd":8,"uuid":"baaaa8dd-0e1f-4bae-86bf-9e429d985f23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":1010601477},{"type":"v1","addr":"172.21.15.204:6817","nonce":1010601477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":1010601477},{"type":"v1","addr":"172.21.15.204:6819","nonce":1010601477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":1010601477},{"type":"v1","addr":"172.21.15.204:6823","nonce":1010601477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":1010601477},{"type":"v1","addr":"172.21.15.204:6821","nonce":1010601477}]},"public_addr":"172.21.15.204:6817/1010601477","cluster_addr":"172.21.15.204:6819/1010601477","heartbeat_back_addr":"172.21.15.204:6823/1010601477","heartbeat_front_addr":"172.21.15.204:6821/1010601477","state":["exists","up"]},{"osd":9,"uuid":"8df80dfa-e7ff-410e-824c-b1dce111b02f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":3334759819},{"type":"v1","addr":"172.21.15.107:6827","nonce":3334759819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3334759819},{"type":"v1","addr":"172.21.15.107:6829","nonce":3334759819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":3334759819},{"type":"v1","addr":"172.21.15.107:6833","nonce":3334759819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":3334759819},{"type":"v1","addr":"172.21.15.107:6831","nonce":3334759819}]},"public_addr":"172.21.15.107:6827/3334759819","cluster_addr":"172.21.15.107:6829/3334759819","heartbeat_back_addr":"172.21.15.107:6833/3334759819","heartbeat_front_addr":"172.21.15.107:6831/3334759819","state":["exists","up"]},{"osd":10,"uuid":"338945b2-20c0-40b2-aaca-062cd4adee04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":3469284631},{"type":"v1","addr":"172.21.15.149:6825","nonce":3469284631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":3469284631},{"type":"v1","addr":"172.21.15.149:6827","nonce":3469284631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":3469284631},{"type":"v1","addr":"172.21.15.149:6831","nonce":3469284631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":3469284631},{"type":"v1","addr":"172.21.15.149:6829","nonce":3469284631}]},"public_addr":"172.21.15.149:6825/3469284631","cluster_addr":"172.21.15.149:6827/3469284631","heartbeat_back_addr":"172.21.15.149:6831/3469284631","heartbeat_front_addr":"172.21.15.149:6829/3469284631","state":["exists","up"]},{"osd":11,"uuid":"acccfe4f-d307-4a64-8f4d-fba27bfbe09d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":3097772473},{"type":"v1","addr":"172.21.15.204:6825","nonce":3097772473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":3097772473},{"type":"v1","addr":"172.21.15.204:6827","nonce":3097772473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":3097772473},{"type":"v1","addr":"172.21.15.204:6831","nonce":3097772473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":3097772473},{"type":"v1","addr":"172.21.15.204:6829","nonce":3097772473}]},"public_addr":"172.21.15.204:6825/3097772473","cluster_addr":"172.21.15.204:6827/3097772473","heartbeat_back_addr":"172.21.15.204:6831/3097772473","heartbeat_front_addr":"172.21.15.204:6829/3097772473","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T20:50:25.862753+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T20:50:58.167663+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T20:51:30.312816+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T20:52:05.487917+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T20:52:39.348354+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T20:53:12.899002+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T20:53:47.630837+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T20:54:21.444552+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T20:54:57.101160+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T20:55:31.841786+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T20:56:05.827161+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T20:56:40.611097+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.0","mappings":[{"from":4,"to":6}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.3b","mappings":[{"from":3,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.107:0/808124991":"2024-03-09T20:48:20.149406+0000","172.21.15.107:6801/3376636618":"2024-03-09T21:00:32.431358+0000","172.21.15.107:0/2482452215":"2024-03-09T20:47:45.706790+0000","172.21.15.107:6801/3549936519":"2024-03-09T20:47:45.706790+0000","172.21.15.107:0/3746019488":"2024-03-09T21:00:32.431358+0000","172.21.15.107:0/3691962590":"2024-03-09T21:00:32.431358+0000","172.21.15.107:0/1206995985":"2024-03-09T20:47:45.706790+0000","172.21.15.107:6800/3549936519":"2024-03-09T20:47:45.706790+0000","172.21.15.107:0/2392547434":"2024-03-09T20:47:45.706790+0000","172.21.15.107:6800/426372446":"2024-03-09T20:48:20.149406+0000","172.21.15.107:6801/426372446":"2024-03-09T20:48:20.149406+0000","172.21.15.107:0/1886016707":"2024-03-09T20:48:20.149406+0000","172.21.15.107:6800/3376636618":"2024-03-09T21:00:32.431358+0000","172.21.15.107:0/1514666962":"2024-03-09T20:48:20.149406+0000","172.21.15.107:0/552952579":"2024-03-09T21:00:32.431358+0000","172.21.15.107:0/458107598":"2024-03-09T21:00:32.431358+0000","172.21.15.107:0/356063043":"2024-03-09T21:00:32.431358+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":12,"length":1},{"begin":25,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-08T21:15:38.502 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:38.524+0000 7fa7a9600640 1 -- 172.21.15.107:0/2135183702 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7fa7880a2970 msgr2=0x7fa7880a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:15:38.502 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:38.524+0000 7fa7a9600640 1 --2- 172.21.15.107:0/2135183702 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7fa7880a2970 0x7fa7880a4e30 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fa78c005fd0 tx=0x7fa78c005850 comp rx=0 tx=0).stop 2024-03-08T21:15:38.502 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:38.525+0000 7fa7a9600640 1 -- 172.21.15.107:0/2135183702 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa7a41078a0 msgr2=0x7fa7a419e120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:15:38.502 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:38.525+0000 7fa7a9600640 1 --2- 172.21.15.107:0/2135183702 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa7a41078a0 0x7fa7a419e120 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7fa794000c00 tx=0x7fa79402c850 comp rx=0 tx=0).stop 2024-03-08T21:15:38.502 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:38.525+0000 7fa7a9600640 1 -- 172.21.15.107:0/2135183702 shutdown_connections 2024-03-08T21:15:38.502 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:38.525+0000 7fa7a9600640 1 --2- 172.21.15.107:0/2135183702 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7fa7880a2970 0x7fa7880a4e30 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:15:38.502 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:38.525+0000 7fa7a9600640 1 --2- 172.21.15.107:0/2135183702 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa7a410d8b0 0x7fa7a41a29e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:15:38.502 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:38.525+0000 7fa7a9600640 1 --2- 172.21.15.107:0/2135183702 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa7a4108260 0x7fa7a419e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:15:38.502 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:38.525+0000 7fa7a9600640 1 --2- 172.21.15.107:0/2135183702 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa7a41078a0 0x7fa7a419e120 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:15:38.502 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:38.525+0000 7fa7a9600640 1 -- 172.21.15.107:0/2135183702 >> 172.21.15.107:0/2135183702 conn(0x7fa7a4075840 msgr2=0x7fa7a4076230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:15:38.503 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:38.525+0000 7fa7a9600640 1 -- 172.21.15.107:0/2135183702 shutdown_connections 2024-03-08T21:15:38.503 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:38.525+0000 7fa7a9600640 1 -- 172.21.15.107:0/2135183702 wait complete. 2024-03-08T21:15:38.521 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-08T20:51:32.832786+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-08T20:57:48.721227+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-08T20:57:49.707779+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '112', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 25, 'snap_epoch': 112, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-08T21:15:38.522 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-08T21:15:38.739 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:38.762+0000 7fe20ce00640 1 -- 172.21.15.107:0/3608587488 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe20810d8a0 msgr2=0x7fe20810dc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:15:38.739 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:38.762+0000 7fe20ce00640 1 --2- 172.21.15.107:0/3608587488 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe20810d8a0 0x7fe20810dc80 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7fe1f8009930 tx=0x7fe1f802bc20 comp rx=0 tx=0).stop 2024-03-08T21:15:38.739 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:38.762+0000 7fe20ce00640 1 -- 172.21.15.107:0/3608587488 shutdown_connections 2024-03-08T21:15:38.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:38.762+0000 7fe20ce00640 1 --2- 172.21.15.107:0/3608587488 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe208108250 0x7fe208114760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:15:38.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:38.762+0000 7fe20ce00640 1 --2- 172.21.15.107:0/3608587488 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe208107890 0x7fe208107d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:15:38.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:38.762+0000 7fe20ce00640 1 --2- 172.21.15.107:0/3608587488 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe20810d8a0 0x7fe20810dc80 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:15:38.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:38.762+0000 7fe20ce00640 1 -- 172.21.15.107:0/3608587488 >> 172.21.15.107:0/3608587488 conn(0x7fe208075830 msgr2=0x7fe208077c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:15:38.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:38.762+0000 7fe20ce00640 1 -- 172.21.15.107:0/3608587488 shutdown_connections 2024-03-08T21:15:38.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:38.762+0000 7fe20ce00640 1 -- 172.21.15.107:0/3608587488 wait complete. 2024-03-08T21:15:38.742 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:38.763+0000 7fe20ce00640 1 Processor -- start 2024-03-08T21:15:38.742 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:38.763+0000 7fe20ce00640 1 -- start start 2024-03-08T21:15:38.742 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:38.764+0000 7fe20ce00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe208107890 0x7fe20819e150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:15:38.742 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:38.764+0000 7fe20ce00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe208108250 0x7fe20819e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:15:38.742 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:38.764+0000 7fe20ce00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe20810d8a0 0x7fe2081a2a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:15:38.742 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:38.764+0000 7fe20ce00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe208117bd0 con 0x7fe208108250 2024-03-08T21:15:38.742 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:38.764+0000 7fe20ce00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe208117a50 con 0x7fe20810d8a0 2024-03-08T21:15:38.742 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:38.764+0000 7fe20ce00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe208117d50 con 0x7fe208107890 2024-03-08T21:15:38.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:38.764+0000 7fe205400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe208108250 0x7fe20819e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:15:38.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:38.764+0000 7fe205400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe208108250 0x7fe20819e690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:58144/0 (socket says 172.21.15.107:58144) 2024-03-08T21:15:38.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:38.764+0000 7fe205400640 1 -- 172.21.15.107:0/3607217308 learned_addr learned my addr 172.21.15.107:0/3607217308 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:15:38.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:38.764+0000 7fe205e00640 1 --2- 172.21.15.107:0/3607217308 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe208107890 0x7fe20819e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:15:38.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:38.764+0000 7fe206800640 1 --2- 172.21.15.107:0/3607217308 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe20810d8a0 0x7fe2081a2a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:15:38.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:38.764+0000 7fe205400640 1 -- 172.21.15.107:0/3607217308 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe208107890 msgr2=0x7fe20819e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:15:38.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:38.764+0000 7fe205400640 1 --2- 172.21.15.107:0/3607217308 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe208107890 0x7fe20819e150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:15:38.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:38.764+0000 7fe205400640 1 -- 172.21.15.107:0/3607217308 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe20810d8a0 msgr2=0x7fe2081a2a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:15:38.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:38.764+0000 7fe205400640 1 --2- 172.21.15.107:0/3607217308 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe20810d8a0 0x7fe2081a2a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:15:38.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:38.764+0000 7fe205400640 1 -- 172.21.15.107:0/3607217308 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2081a3140 con 0x7fe208108250 2024-03-08T21:15:38.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:38.765+0000 7fe205400640 1 --2- 172.21.15.107:0/3607217308 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe208108250 0x7fe20819e690 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7fe1f400c970 tx=0x7fe1f400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:15:38.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:38.765+0000 7fe1fea00640 1 -- 172.21.15.107:0/3607217308 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1f4007bb0 con 0x7fe208108250 2024-03-08T21:15:38.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:38.765+0000 7fe20ce00640 1 -- 172.21.15.107:0/3607217308 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2081a3430 con 0x7fe208108250 2024-03-08T21:15:38.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:38.765+0000 7fe1fea00640 1 -- 172.21.15.107:0/3607217308 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fe1f4007d50 con 0x7fe208108250 2024-03-08T21:15:38.744 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:38.765+0000 7fe1fea00640 1 -- 172.21.15.107:0/3607217308 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1f400f450 con 0x7fe208108250 2024-03-08T21:15:38.744 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:38.765+0000 7fe20ce00640 1 -- 172.21.15.107:0/3607217308 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2081aacc0 con 0x7fe208108250 2024-03-08T21:15:38.744 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:38.766+0000 7fe20ce00640 1 -- 172.21.15.107:0/3607217308 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe208068530 con 0x7fe208108250 2024-03-08T21:15:38.746 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:38.769+0000 7fe1fea00640 1 -- 172.21.15.107:0/3607217308 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) v1 ==== 137422+0+0 (secure 0 0 0) 0x7fe1f4004f20 con 0x7fe208108250 2024-03-08T21:15:38.747 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:38.770+0000 7fe1fea00640 1 --2- 172.21.15.107:0/3607217308 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7fe1ec0a2970 0x7fe1ec0a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:15:38.747 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:38.771+0000 7fe1fea00640 1 -- 172.21.15.107:0/3607217308 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fe1f40ced20 con 0x7fe208108250 2024-03-08T21:15:38.747 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:38.771+0000 7fe205e00640 1 --2- 172.21.15.107:0/3607217308 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7fe1ec0a2970 0x7fe1ec0a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:15:38.748 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:38.771+0000 7fe205e00640 1 --2- 172.21.15.107:0/3607217308 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7fe1ec0a2970 0x7fe1ec0a4e30 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fe1f8009900 tx=0x7fe1f8005fd0 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:15:38.752 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:38.775+0000 7fe1fea00640 1 -- 172.21.15.107:0/3607217308 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7fe1f4014030 con 0x7fe208108250 2024-03-08T21:15:38.871 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:15:38 smithi204 ceph-mon[34935]: pgmap v479: 97 pgs: 97 active+clean; 144 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 852 KiB/s rd, 945 KiB/s wr, 174 op/s 2024-03-08T21:15:38.871 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:15:38 smithi204 ceph-mon[34935]: from='client.16713 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-03-08T21:15:38.872 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:15:38 smithi204 ceph-mon[34935]: from='client.? 172.21.15.107:0/2135183702' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-08T21:15:38.951 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:38.974+0000 7fe20ce00640 1 -- 172.21.15.107:0/3607217308 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe20810f870 con 0x7fe208108250 2024-03-08T21:15:38.951 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:38.975+0000 7fe1fea00640 1 -- 172.21.15.107:0/3607217308 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v113) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe1f409b9e0 con 0x7fe208108250 2024-03-08T21:15:38.952 INFO:teuthology.orchestra.run.smithi107.stdout:pg_num: 1 2024-03-08T21:15:38.954 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:38.977+0000 7fe20ce00640 1 -- 172.21.15.107:0/3607217308 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7fe1ec0a2970 msgr2=0x7fe1ec0a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:15:38.954 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:38.977+0000 7fe20ce00640 1 --2- 172.21.15.107:0/3607217308 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7fe1ec0a2970 0x7fe1ec0a4e30 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fe1f8009900 tx=0x7fe1f8005fd0 comp rx=0 tx=0).stop 2024-03-08T21:15:38.954 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:38.977+0000 7fe20ce00640 1 -- 172.21.15.107:0/3607217308 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe208108250 msgr2=0x7fe20819e690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:15:38.954 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:38.977+0000 7fe20ce00640 1 --2- 172.21.15.107:0/3607217308 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe208108250 0x7fe20819e690 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7fe1f400c970 tx=0x7fe1f400ce30 comp rx=0 tx=0).stop 2024-03-08T21:15:38.955 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:38.977+0000 7fe20ce00640 1 -- 172.21.15.107:0/3607217308 shutdown_connections 2024-03-08T21:15:38.955 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:38.977+0000 7fe20ce00640 1 --2- 172.21.15.107:0/3607217308 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7fe1ec0a2970 0x7fe1ec0a4e30 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:15:38.955 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:38.977+0000 7fe20ce00640 1 --2- 172.21.15.107:0/3607217308 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe20810d8a0 0x7fe2081a2a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:15:38.955 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:38.977+0000 7fe20ce00640 1 --2- 172.21.15.107:0/3607217308 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe208108250 0x7fe20819e690 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:15:38.955 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:38.977+0000 7fe20ce00640 1 --2- 172.21.15.107:0/3607217308 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe208107890 0x7fe20819e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:15:38.955 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:38.977+0000 7fe20ce00640 1 -- 172.21.15.107:0/3607217308 >> 172.21.15.107:0/3607217308 conn(0x7fe208075830 msgr2=0x7fe208076060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:15:38.955 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:38.977+0000 7fe20ce00640 1 -- 172.21.15.107:0/3607217308 shutdown_connections 2024-03-08T21:15:38.955 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:38.977+0000 7fe20ce00640 1 -- 172.21.15.107:0/3607217308 wait complete. 2024-03-08T21:15:38.960 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:15:38 smithi107 ceph-mon[31381]: pgmap v479: 97 pgs: 97 active+clean; 144 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 852 KiB/s rd, 945 KiB/s wr, 174 op/s 2024-03-08T21:15:38.960 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:15:38 smithi107 ceph-mon[31381]: from='client.16713 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-03-08T21:15:38.960 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:15:38 smithi107 ceph-mon[31381]: from='client.? 172.21.15.107:0/2135183702' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-08T21:15:38.975 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-08T21:15:39.008 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:15:38 smithi149 ceph-mon[35042]: pgmap v479: 97 pgs: 97 active+clean; 144 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 852 KiB/s rd, 945 KiB/s wr, 174 op/s 2024-03-08T21:15:39.008 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:15:38 smithi149 ceph-mon[35042]: from='client.16713 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-03-08T21:15:39.008 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:15:38 smithi149 ceph-mon[35042]: from='client.? 172.21.15.107:0/2135183702' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-08T21:15:39.187 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:39.210+0000 7fe47ec00640 1 -- 172.21.15.107:0/709227952 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe478102e50 msgr2=0x7fe47810b330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:15:39.187 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:39.210+0000 7fe47ec00640 1 --2- 172.21.15.107:0/709227952 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe478102e50 0x7fe47810b330 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7fe460009900 tx=0x7fe46002bcc0 comp rx=0 tx=0).stop 2024-03-08T21:15:39.188 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:39.210+0000 7fe47ec00640 1 -- 172.21.15.107:0/709227952 shutdown_connections 2024-03-08T21:15:39.188 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:39.210+0000 7fe47ec00640 1 --2- 172.21.15.107:0/709227952 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe47810b900 0x7fe47810dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:15:39.188 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:39.210+0000 7fe47ec00640 1 --2- 172.21.15.107:0/709227952 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe478102e50 0x7fe47810b330 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:15:39.188 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:39.210+0000 7fe47ec00640 1 --2- 172.21.15.107:0/709227952 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe478102530 0x7fe478102910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:15:39.188 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:39.210+0000 7fe47ec00640 1 -- 172.21.15.107:0/709227952 >> 172.21.15.107:0/709227952 conn(0x7fe478076570 msgr2=0x7fe478076980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:15:39.188 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:39.210+0000 7fe47ec00640 1 -- 172.21.15.107:0/709227952 shutdown_connections 2024-03-08T21:15:39.188 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:39.210+0000 7fe47ec00640 1 -- 172.21.15.107:0/709227952 wait complete. 2024-03-08T21:15:39.190 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:39.211+0000 7fe47ec00640 1 Processor -- start 2024-03-08T21:15:39.190 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:39.211+0000 7fe47ec00640 1 -- start start 2024-03-08T21:15:39.190 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:39.212+0000 7fe47ec00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe478102530 0x7fe478100180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:15:39.190 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:39.212+0000 7fe47ec00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe478102e50 0x7fe4781006c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:15:39.190 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:39.212+0000 7fe47ec00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe47810b900 0x7fe478100c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:15:39.190 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:39.212+0000 7fe47ec00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe478111590 con 0x7fe478102e50 2024-03-08T21:15:39.191 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:39.212+0000 7fe47ec00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe478111410 con 0x7fe478102530 2024-03-08T21:15:39.191 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:39.212+0000 7fe47ec00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe478111710 con 0x7fe47810b900 2024-03-08T21:15:39.191 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:39.212+0000 7fe46fe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe478102e50 0x7fe4781006c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:15:39.191 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:39.212+0000 7fe46fe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe478102e50 0x7fe4781006c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:58152/0 (socket says 172.21.15.107:58152) 2024-03-08T21:15:39.191 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:39.212+0000 7fe46fe00640 1 -- 172.21.15.107:0/3012182869 learned_addr learned my addr 172.21.15.107:0/3012182869 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:15:39.191 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:39.212+0000 7fe46fe00640 1 -- 172.21.15.107:0/3012182869 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe47810b900 msgr2=0x7fe478100c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:15:39.191 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:39.212+0000 7fe46fe00640 1 --2- 172.21.15.107:0/3012182869 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe47810b900 0x7fe478100c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:15:39.191 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:39.212+0000 7fe46fe00640 1 -- 172.21.15.107:0/3012182869 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe478102530 msgr2=0x7fe478100180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:15:39.191 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:39.212+0000 7fe46fe00640 1 --2- 172.21.15.107:0/3012182869 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe478102530 0x7fe478100180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:15:39.191 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:39.212+0000 7fe46fe00640 1 -- 172.21.15.107:0/3012182869 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4781a5fb0 con 0x7fe478102e50 2024-03-08T21:15:39.191 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:39.213+0000 7fe46fe00640 1 --2- 172.21.15.107:0/3012182869 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe478102e50 0x7fe4781006c0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7fe460005ab0 tx=0x7fe46002db30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:15:39.191 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:39.213+0000 7fe475600640 1 -- 172.21.15.107:0/3012182869 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe460043020 con 0x7fe478102e50 2024-03-08T21:15:39.191 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:39.213+0000 7fe475600640 1 -- 172.21.15.107:0/3012182869 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fe4600052b0 con 0x7fe478102e50 2024-03-08T21:15:39.191 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:39.213+0000 7fe475600640 1 -- 172.21.15.107:0/3012182869 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe460003c50 con 0x7fe478102e50 2024-03-08T21:15:39.191 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:39.213+0000 7fe47ec00640 1 -- 172.21.15.107:0/3012182869 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4781a6240 con 0x7fe478102e50 2024-03-08T21:15:39.192 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:39.213+0000 7fe47ec00640 1 -- 172.21.15.107:0/3012182869 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4781a67e0 con 0x7fe478102e50 2024-03-08T21:15:39.192 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:39.215+0000 7fe47ec00640 1 -- 172.21.15.107:0/3012182869 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe47806a0e0 con 0x7fe478102e50 2024-03-08T21:15:39.193 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:39.216+0000 7fe475600640 1 -- 172.21.15.107:0/3012182869 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) v1 ==== 137422+0+0 (secure 0 0 0) 0x7fe460046020 con 0x7fe478102e50 2024-03-08T21:15:39.194 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:39.217+0000 7fe475600640 1 --2- 172.21.15.107:0/3012182869 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7fe45c0a29f0 0x7fe45c0a4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:15:39.194 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:39.218+0000 7fe475600640 1 -- 172.21.15.107:0/3012182869 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fe4600f10f0 con 0x7fe478102e50 2024-03-08T21:15:39.195 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:39.218+0000 7fe477e00640 1 --2- 172.21.15.107:0/3012182869 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7fe45c0a29f0 0x7fe45c0a4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:15:39.195 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:39.219+0000 7fe477e00640 1 --2- 172.21.15.107:0/3012182869 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7fe45c0a29f0 0x7fe45c0a4eb0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fe46800a990 tx=0x7fe468005c70 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:15:39.198 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:39.222+0000 7fe475600640 1 -- 172.21.15.107:0/3012182869 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7fe46004d020 con 0x7fe478102e50 2024-03-08T21:15:39.397 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:39.420+0000 7fe47ec00640 1 -- 172.21.15.107:0/3012182869 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe478106460 con 0x7fe478102e50 2024-03-08T21:15:39.398 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:39.421+0000 7fe475600640 1 -- 172.21.15.107:0/3012182869 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v113) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe4600bdec0 con 0x7fe478102e50 2024-03-08T21:15:39.399 INFO:teuthology.orchestra.run.smithi107.stdout:pg_num: 32 2024-03-08T21:15:39.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:39.424+0000 7fe47ec00640 1 -- 172.21.15.107:0/3012182869 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7fe45c0a29f0 msgr2=0x7fe45c0a4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:15:39.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:39.424+0000 7fe47ec00640 1 --2- 172.21.15.107:0/3012182869 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7fe45c0a29f0 0x7fe45c0a4eb0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fe46800a990 tx=0x7fe468005c70 comp rx=0 tx=0).stop 2024-03-08T21:15:39.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:39.424+0000 7fe47ec00640 1 -- 172.21.15.107:0/3012182869 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe478102e50 msgr2=0x7fe4781006c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:15:39.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:39.424+0000 7fe47ec00640 1 --2- 172.21.15.107:0/3012182869 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe478102e50 0x7fe4781006c0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7fe460005ab0 tx=0x7fe46002db30 comp rx=0 tx=0).stop 2024-03-08T21:15:39.403 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:39.425+0000 7fe47ec00640 1 -- 172.21.15.107:0/3012182869 shutdown_connections 2024-03-08T21:15:39.403 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:39.425+0000 7fe47ec00640 1 --2- 172.21.15.107:0/3012182869 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7fe45c0a29f0 0x7fe45c0a4eb0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:15:39.403 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:39.425+0000 7fe47ec00640 1 --2- 172.21.15.107:0/3012182869 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fe47810b900 0x7fe478100c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:15:39.403 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:39.425+0000 7fe47ec00640 1 --2- 172.21.15.107:0/3012182869 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe478102e50 0x7fe4781006c0 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:15:39.403 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:39.425+0000 7fe47ec00640 1 --2- 172.21.15.107:0/3012182869 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe478102530 0x7fe478100180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:15:39.403 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:39.425+0000 7fe47ec00640 1 -- 172.21.15.107:0/3012182869 >> 172.21.15.107:0/3012182869 conn(0x7fe478076570 msgr2=0x7fe4780fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:15:39.403 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:39.425+0000 7fe47ec00640 1 -- 172.21.15.107:0/3012182869 shutdown_connections 2024-03-08T21:15:39.403 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:39.425+0000 7fe47ec00640 1 -- 172.21.15.107:0/3012182869 wait complete. 2024-03-08T21:15:39.421 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-08T21:15:39.641 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:39.664+0000 7f9f5e400640 1 -- 172.21.15.107:0/3099412136 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9f581034a0 msgr2=0x7f9f58103920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:15:39.641 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:39.664+0000 7f9f5e400640 1 --2- 172.21.15.107:0/3099412136 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9f581034a0 0x7f9f58103920 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f9f44009900 tx=0x7f9f4402bc40 comp rx=0 tx=0).stop 2024-03-08T21:15:39.641 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:39.664+0000 7f9f5e400640 1 -- 172.21.15.107:0/3099412136 shutdown_connections 2024-03-08T21:15:39.641 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:39.664+0000 7f9f5e400640 1 --2- 172.21.15.107:0/3099412136 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9f58103e60 0x7f9f58110320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:15:39.641 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:39.664+0000 7f9f5e400640 1 --2- 172.21.15.107:0/3099412136 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9f581034a0 0x7f9f58103920 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:15:39.641 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:39.664+0000 7f9f5e400640 1 --2- 172.21.15.107:0/3099412136 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9f58109560 0x7f9f58109940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:15:39.642 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:39.664+0000 7f9f5e400640 1 -- 172.21.15.107:0/3099412136 >> 172.21.15.107:0/3099412136 conn(0x7f9f580fef70 msgr2=0x7f9f58101390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:15:39.642 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:39.664+0000 7f9f5e400640 1 -- 172.21.15.107:0/3099412136 shutdown_connections 2024-03-08T21:15:39.642 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:39.664+0000 7f9f5e400640 1 -- 172.21.15.107:0/3099412136 wait complete. 2024-03-08T21:15:39.644 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:39.665+0000 7f9f5e400640 1 Processor -- start 2024-03-08T21:15:39.644 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:39.665+0000 7f9f5e400640 1 -- start start 2024-03-08T21:15:39.644 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:39.665+0000 7f9f5e400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9f581034a0 0x7f9f5819f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:15:39.644 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:39.665+0000 7f9f5e400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9f58103e60 0x7f9f5819fd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:15:39.644 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:39.665+0000 7f9f5e400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9f58109560 0x7f9f581999e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:15:39.644 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:39.665+0000 7f9f5e400640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f58113720 con 0x7f9f58103e60 2024-03-08T21:15:39.644 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:39.665+0000 7f9f5e400640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f581135a0 con 0x7f9f581034a0 2024-03-08T21:15:39.644 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:39.665+0000 7f9f5e400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f581138a0 con 0x7f9f58109560 2024-03-08T21:15:39.645 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:39.665+0000 7f9f56a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9f58103e60 0x7f9f5819fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:15:39.645 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:39.666+0000 7f9f56a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9f58103e60 0x7f9f5819fd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:58178/0 (socket says 172.21.15.107:58178) 2024-03-08T21:15:39.645 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:39.666+0000 7f9f56a00640 1 -- 172.21.15.107:0/3323855843 learned_addr learned my addr 172.21.15.107:0/3323855843 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:15:39.645 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:39.666+0000 7f9f57400640 1 --2- 172.21.15.107:0/3323855843 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9f581034a0 0x7f9f5819f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:15:39.645 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:39.666+0000 7f9f56a00640 1 -- 172.21.15.107:0/3323855843 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9f58109560 msgr2=0x7f9f581999e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:15:39.645 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:39.666+0000 7f9f56a00640 1 --2- 172.21.15.107:0/3323855843 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9f58109560 0x7f9f581999e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:15:39.645 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:39.666+0000 7f9f56a00640 1 -- 172.21.15.107:0/3323855843 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9f581034a0 msgr2=0x7f9f5819f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:15:39.645 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:39.666+0000 7f9f56a00640 1 --2- 172.21.15.107:0/3323855843 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9f581034a0 0x7f9f5819f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:15:39.645 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:39.666+0000 7f9f56a00640 1 -- 172.21.15.107:0/3323855843 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f5819a160 con 0x7f9f58103e60 2024-03-08T21:15:39.645 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:39.666+0000 7f9f57400640 1 --2- 172.21.15.107:0/3323855843 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9f581034a0 0x7f9f5819f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T21:15:39.645 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:39.666+0000 7f9f56a00640 1 --2- 172.21.15.107:0/3323855843 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9f58103e60 0x7f9f5819fd70 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f9f44000c00 tx=0x7f9f4402ca40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:15:39.645 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:39.667+0000 7f9f4fe00640 1 -- 172.21.15.107:0/3323855843 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f44043020 con 0x7f9f58103e60 2024-03-08T21:15:39.646 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:39.667+0000 7f9f4fe00640 1 -- 172.21.15.107:0/3323855843 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f9f4402cc30 con 0x7f9f58103e60 2024-03-08T21:15:39.646 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:39.667+0000 7f9f4fe00640 1 -- 172.21.15.107:0/3323855843 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f44003c90 con 0x7f9f58103e60 2024-03-08T21:15:39.646 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:39.667+0000 7f9f5e400640 1 -- 172.21.15.107:0/3323855843 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f5819a3f0 con 0x7f9f58103e60 2024-03-08T21:15:39.646 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:39.667+0000 7f9f5e400640 1 -- 172.21.15.107:0/3323855843 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f581a66f0 con 0x7f9f58103e60 2024-03-08T21:15:39.646 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:39.668+0000 7f9f5e400640 1 -- 172.21.15.107:0/3323855843 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f58068530 con 0x7f9f58103e60 2024-03-08T21:15:39.647 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:39.670+0000 7f9f4fe00640 1 -- 172.21.15.107:0/3323855843 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) v1 ==== 137422+0+0 (secure 0 0 0) 0x7f9f44046020 con 0x7f9f58103e60 2024-03-08T21:15:39.648 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:39.672+0000 7f9f4fe00640 1 --2- 172.21.15.107:0/3323855843 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f9f3c0a29f0 0x7f9f3c0a4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:15:39.649 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:39.672+0000 7f9f4fe00640 1 -- 172.21.15.107:0/3323855843 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f9f440f11e0 con 0x7f9f58103e60 2024-03-08T21:15:39.649 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:39.672+0000 7f9f57400640 1 --2- 172.21.15.107:0/3323855843 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f9f3c0a29f0 0x7f9f3c0a4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:15:39.649 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:39.673+0000 7f9f57400640 1 --2- 172.21.15.107:0/3323855843 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f9f3c0a29f0 0x7f9f3c0a4eb0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f9f5819ab20 tx=0x7f9f48005db0 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:15:39.653 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:39.676+0000 7f9f4fe00640 1 -- 172.21.15.107:0/3323855843 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f9f4404d020 con 0x7f9f58103e60 2024-03-08T21:15:39.710 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:15:39 smithi107 ceph-mon[31381]: from='client.16719 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-08T21:15:39.710 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:15:39 smithi107 ceph-mon[31381]: from='client.? 172.21.15.107:0/3607217308' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-08T21:15:39.711 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:15:39 smithi107 ceph-mon[31381]: from='client.? 172.21.15.107:0/3012182869' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-08T21:15:39.850 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:39.873+0000 7f9f5e400640 1 -- 172.21.15.107:0/3323855843 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f9f5802ce70 con 0x7f9f58103e60 2024-03-08T21:15:39.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:39.874+0000 7f9f4fe00640 1 -- 172.21.15.107:0/3323855843 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v113) v1 ==== 100+0+11 (secure 0 0 0) 0x7f9f440be010 con 0x7f9f58103e60 2024-03-08T21:15:39.851 INFO:teuthology.orchestra.run.smithi107.stdout:pg_num: 64 2024-03-08T21:15:39.854 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:39.877+0000 7f9f5e400640 1 -- 172.21.15.107:0/3323855843 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f9f3c0a29f0 msgr2=0x7f9f3c0a4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:15:39.854 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:39.877+0000 7f9f5e400640 1 --2- 172.21.15.107:0/3323855843 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f9f3c0a29f0 0x7f9f3c0a4eb0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f9f5819ab20 tx=0x7f9f48005db0 comp rx=0 tx=0).stop 2024-03-08T21:15:39.854 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:39.877+0000 7f9f5e400640 1 -- 172.21.15.107:0/3323855843 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9f58103e60 msgr2=0x7f9f5819fd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:15:39.854 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:39.877+0000 7f9f5e400640 1 --2- 172.21.15.107:0/3323855843 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9f58103e60 0x7f9f5819fd70 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f9f44000c00 tx=0x7f9f4402ca40 comp rx=0 tx=0).stop 2024-03-08T21:15:39.854 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:39.877+0000 7f9f5e400640 1 -- 172.21.15.107:0/3323855843 shutdown_connections 2024-03-08T21:15:39.854 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:39.877+0000 7f9f5e400640 1 --2- 172.21.15.107:0/3323855843 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f9f3c0a29f0 0x7f9f3c0a4eb0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:15:39.855 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:39.877+0000 7f9f5e400640 1 --2- 172.21.15.107:0/3323855843 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f9f58109560 0x7f9f581999e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:15:39.855 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:39.877+0000 7f9f5e400640 1 --2- 172.21.15.107:0/3323855843 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9f58103e60 0x7f9f5819fd70 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:15:39.855 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:39.877+0000 7f9f5e400640 1 --2- 172.21.15.107:0/3323855843 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9f581034a0 0x7f9f5819f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:15:39.855 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:39.877+0000 7f9f5e400640 1 -- 172.21.15.107:0/3323855843 >> 172.21.15.107:0/3323855843 conn(0x7f9f580fef70 msgr2=0x7f9f58100710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:15:39.855 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:39.877+0000 7f9f5e400640 1 -- 172.21.15.107:0/3323855843 shutdown_connections 2024-03-08T21:15:39.855 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:39.877+0000 7f9f5e400640 1 -- 172.21.15.107:0/3323855843 wait complete. 2024-03-08T21:15:39.871 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:15:39 smithi204 ceph-mon[34935]: from='client.16719 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-08T21:15:39.871 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:15:39 smithi204 ceph-mon[34935]: from='client.? 172.21.15.107:0/3607217308' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-08T21:15:39.871 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:15:39 smithi204 ceph-mon[34935]: from='client.? 172.21.15.107:0/3012182869' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-08T21:15:39.874 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-03-08T21:15:40.008 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:15:39 smithi149 ceph-mon[35042]: from='client.16719 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-08T21:15:40.008 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:15:39 smithi149 ceph-mon[35042]: from='client.? 172.21.15.107:0/3607217308' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-08T21:15:40.009 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:15:39 smithi149 ceph-mon[35042]: from='client.? 172.21.15.107:0/3012182869' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-08T21:15:40.091 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:40.114+0000 7f2746200640 1 -- 172.21.15.107:0/2724625246 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f27401063f0 msgr2=0x7f274010dfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:15:40.091 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:40.114+0000 7f2746200640 1 --2- 172.21.15.107:0/2724625246 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f27401063f0 0x7f274010dfc0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f2738009ca0 tx=0x7f273802c100 comp rx=0 tx=0).stop 2024-03-08T21:15:40.091 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:40.114+0000 7f2746200640 1 -- 172.21.15.107:0/2724625246 shutdown_connections 2024-03-08T21:15:40.091 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:40.114+0000 7f2746200640 1 --2- 172.21.15.107:0/2724625246 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f27401063f0 0x7f274010dfc0 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:15:40.091 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:40.114+0000 7f2746200640 1 --2- 172.21.15.107:0/2724625246 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2740101cd0 0x7f2740105d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:15:40.091 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:40.114+0000 7f2746200640 1 --2- 172.21.15.107:0/2724625246 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2740101320 0x7f2740101700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:15:40.092 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:40.114+0000 7f2746200640 1 -- 172.21.15.107:0/2724625246 >> 172.21.15.107:0/2724625246 conn(0x7f27400fcbc0 msgr2=0x7f27400fefe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:15:40.092 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:40.114+0000 7f2746200640 1 -- 172.21.15.107:0/2724625246 shutdown_connections 2024-03-08T21:15:40.092 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:40.114+0000 7f2746200640 1 -- 172.21.15.107:0/2724625246 wait complete. 2024-03-08T21:15:40.094 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:40.115+0000 7f2746200640 1 Processor -- start 2024-03-08T21:15:40.094 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:40.115+0000 7f2746200640 1 -- start start 2024-03-08T21:15:40.094 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:40.115+0000 7f2746200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2740101320 0x7f27400735f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:15:40.094 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:40.115+0000 7f2746200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f27401063f0 0x7f2740073b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:15:40.094 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:40.115+0000 7f2746200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2740074070 0x7f27401a5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:15:40.094 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:40.115+0000 7f2746200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2740111360 con 0x7f27401063f0 2024-03-08T21:15:40.094 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:40.115+0000 7f2746200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27401111e0 con 0x7f2740074070 2024-03-08T21:15:40.094 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:40.115+0000 7f2746200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27401114e0 con 0x7f2740101320 2024-03-08T21:15:40.094 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:40.116+0000 7f273fe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f27401063f0 0x7f2740073b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:15:40.094 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:40.116+0000 7f273fe00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f27401063f0 0x7f2740073b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:58184/0 (socket says 172.21.15.107:58184) 2024-03-08T21:15:40.094 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:40.116+0000 7f273fe00640 1 -- 172.21.15.107:0/492498522 learned_addr learned my addr 172.21.15.107:0/492498522 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:15:40.095 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:40.116+0000 7f2744e00640 1 --2- 172.21.15.107:0/492498522 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2740101320 0x7f27400735f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:15:40.095 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:40.116+0000 7f273fe00640 1 -- 172.21.15.107:0/492498522 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2740101320 msgr2=0x7f27400735f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:15:40.095 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:40.116+0000 7f273fe00640 1 --2- 172.21.15.107:0/492498522 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2740101320 0x7f27400735f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:15:40.095 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:40.116+0000 7f273fe00640 1 -- 172.21.15.107:0/492498522 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2740074070 msgr2=0x7f27401a5ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:15:40.095 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:40.116+0000 7f273fe00640 1 --2- 172.21.15.107:0/492498522 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2740074070 0x7f27401a5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:15:40.095 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:40.116+0000 7f273fe00640 1 -- 172.21.15.107:0/492498522 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27401a6170 con 0x7f27401063f0 2024-03-08T21:15:40.095 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:40.116+0000 7f2744e00640 1 --2- 172.21.15.107:0/492498522 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2740101320 0x7f27400735f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T21:15:40.095 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:40.116+0000 7f273fe00640 1 --2- 172.21.15.107:0/492498522 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f27401063f0 0x7f2740073b30 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f272c00d740 tx=0x7f272c00dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:15:40.095 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:40.116+0000 7f273d600640 1 -- 172.21.15.107:0/492498522 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f272c017070 con 0x7f27401063f0 2024-03-08T21:15:40.095 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:40.116+0000 7f273d600640 1 -- 172.21.15.107:0/492498522 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f272c00de50 con 0x7f27401063f0 2024-03-08T21:15:40.095 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:40.116+0000 7f2746200640 1 -- 172.21.15.107:0/492498522 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27401a6460 con 0x7f27401063f0 2024-03-08T21:15:40.095 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:40.117+0000 7f273d600640 1 -- 172.21.15.107:0/492498522 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f272c004d60 con 0x7f27401063f0 2024-03-08T21:15:40.095 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:40.117+0000 7f2746200640 1 -- 172.21.15.107:0/492498522 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27401a69b0 con 0x7f27401063f0 2024-03-08T21:15:40.096 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:40.118+0000 7f2746200640 1 -- 172.21.15.107:0/492498522 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2740068450 con 0x7f27401063f0 2024-03-08T21:15:40.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:40.120+0000 7f273d600640 1 -- 172.21.15.107:0/492498522 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) v1 ==== 137422+0+0 (secure 0 0 0) 0x7f272c004f00 con 0x7f27401063f0 2024-03-08T21:15:40.098 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:40.122+0000 7f273d600640 1 --2- 172.21.15.107:0/492498522 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f27280a29f0 0x7f27280a4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:15:40.099 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:40.122+0000 7f273d600640 1 -- 172.21.15.107:0/492498522 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f272c0ce880 con 0x7f27401063f0 2024-03-08T21:15:40.099 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:40.122+0000 7f2744e00640 1 --2- 172.21.15.107:0/492498522 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f27280a29f0 0x7f27280a4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:15:40.099 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:40.123+0000 7f2744e00640 1 --2- 172.21.15.107:0/492498522 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f27280a29f0 0x7f27280a4eb0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f273400baa0 tx=0x7f2734005d90 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:15:40.103 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:40.126+0000 7f273d600640 1 -- 172.21.15.107:0/492498522 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f272c09b540 con 0x7f27401063f0 2024-03-08T21:15:40.352 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:40.375+0000 7f2746200640 1 -- 172.21.15.107:0/492498522 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) v1 -- 0x7f2740109650 con 0x7f27401063f0 2024-03-08T21:15:40.570 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:40.391+0000 7f273d600640 1 -- 172.21.15.107:0/492498522 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v25) v1 ==== 81+0+740372 (secure 0 0 0) 0x7f272c09ae40 con 0x7f27401063f0 2024-03-08T21:15:40.580 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-08T21:15:40.581 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-03-08T21:15:40.581 INFO:teuthology.orchestra.run.smithi107.stdout: "epoch": 25, 2024-03-08T21:15:40.582 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:40.582 INFO:teuthology.orchestra.run.smithi107.stdout: "active_gid": 15375, 2024-03-08T21:15:40.582 INFO:teuthology.orchestra.run.smithi107.stdout: "active_name": "x", 2024-03-08T21:15:40.583 INFO:teuthology.orchestra.run.smithi107.stdout: "active_addrs": { 2024-03-08T21:15:40.583 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-03-08T21:15:40.583 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-08T21:15:40.584 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-03-08T21:15:40.584 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6800", 2024-03-08T21:15:40.584 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 2792338347 2024-03-08T21:15:40.584 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.585 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-08T21:15:40.585 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-03-08T21:15:40.585 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6801", 2024-03-08T21:15:40.586 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 2792338347 2024-03-08T21:15:40.586 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:40.586 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-03-08T21:15:40.586 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.587 INFO:teuthology.orchestra.run.smithi107.stdout: "active_addr": "172.21.15.107:6801/2792338347", 2024-03-08T21:15:40.587 INFO:teuthology.orchestra.run.smithi107.stdout: "active_change": "2024-03-08T21:00:32.432012+0000", 2024-03-08T21:15:40.587 INFO:teuthology.orchestra.run.smithi107.stdout: "active_mgr_features": 4540701547738038271, 2024-03-08T21:15:40.587 INFO:teuthology.orchestra.run.smithi107.stdout: "available": true, 2024-03-08T21:15:40.587 INFO:teuthology.orchestra.run.smithi107.stdout: "standbys": [ 2024-03-08T21:15:40.587 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-08T21:15:40.587 INFO:teuthology.orchestra.run.smithi107.stdout: "gid": 15381, 2024-03-08T21:15:40.587 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "y", 2024-03-08T21:15:40.587 INFO:teuthology.orchestra.run.smithi107.stdout: "mgr_features": 4540701547738038271, 2024-03-08T21:15:40.587 INFO:teuthology.orchestra.run.smithi107.stdout: "available_modules": [ 2024-03-08T21:15:40.587 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-08T21:15:40.587 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "alerts", 2024-03-08T21:15:40.587 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-03-08T21:15:40.587 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-03-08T21:15:40.588 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-03-08T21:15:40.588 INFO:teuthology.orchestra.run.smithi107.stdout: "interval": { 2024-03-08T21:15:40.588 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "interval", 2024-03-08T21:15:40.588 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-03-08T21:15:40.588 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.588 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:40.588 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "60", 2024-03-08T21:15:40.588 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.588 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.588 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.588 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "How frequently to reexamine health status", 2024-03-08T21:15:40.588 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.588 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.588 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.588 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.589 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-03-08T21:15:40.589 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-03-08T21:15:40.589 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:40.589 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.589 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:40.589 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:40.589 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.589 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.589 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:40.589 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-08T21:15:40.589 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-08T21:15:40.589 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-08T21:15:40.589 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-08T21:15:40.589 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-08T21:15:40.589 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-08T21:15:40.590 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:40.590 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.590 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.590 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.590 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.590 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.590 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-03-08T21:15:40.590 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-03-08T21:15:40.590 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:40.590 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.590 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:40.590 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:40.590 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.590 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.590 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.591 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.591 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.591 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.591 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.591 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.591 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-03-08T21:15:40.591 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-03-08T21:15:40.591 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:40.591 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.591 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:40.591 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-03-08T21:15:40.591 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.591 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.591 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:40.591 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-08T21:15:40.592 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-08T21:15:40.592 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-08T21:15:40.592 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-08T21:15:40.592 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-08T21:15:40.592 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-08T21:15:40.592 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:40.592 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.592 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.592 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.592 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.592 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.592 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-03-08T21:15:40.592 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-03-08T21:15:40.592 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:40.592 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.593 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:40.593 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:40.593 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.593 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.593 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.593 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.593 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.593 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.593 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.593 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.593 INFO:teuthology.orchestra.run.smithi107.stdout: "smtp_destination": { 2024-03-08T21:15:40.593 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "smtp_destination", 2024-03-08T21:15:40.593 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:40.593 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.593 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:40.593 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:40.594 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.594 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.594 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.594 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Email address to send alerts to", 2024-03-08T21:15:40.594 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.594 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.594 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.594 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.594 INFO:teuthology.orchestra.run.smithi107.stdout: "smtp_from_name": { 2024-03-08T21:15:40.594 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "smtp_from_name", 2024-03-08T21:15:40.594 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:40.594 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.594 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:40.594 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "Ceph", 2024-03-08T21:15:40.595 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.595 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.595 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.595 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Email From: name", 2024-03-08T21:15:40.595 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.595 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.595 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.595 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.595 INFO:teuthology.orchestra.run.smithi107.stdout: "smtp_host": { 2024-03-08T21:15:40.595 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "smtp_host", 2024-03-08T21:15:40.595 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:40.595 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.595 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:40.595 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:40.595 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.595 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.596 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.596 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "SMTP server", 2024-03-08T21:15:40.596 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.596 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.596 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.596 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.596 INFO:teuthology.orchestra.run.smithi107.stdout: "smtp_password": { 2024-03-08T21:15:40.596 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "smtp_password", 2024-03-08T21:15:40.596 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:40.596 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.596 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:40.596 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:40.596 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.596 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.596 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.597 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Password to authenticate with", 2024-03-08T21:15:40.597 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.597 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.597 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.597 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.597 INFO:teuthology.orchestra.run.smithi107.stdout: "smtp_port": { 2024-03-08T21:15:40.597 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "smtp_port", 2024-03-08T21:15:40.597 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-08T21:15:40.597 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.597 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:40.597 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "465", 2024-03-08T21:15:40.597 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.597 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.597 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.597 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "SMTP port", 2024-03-08T21:15:40.597 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.597 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.598 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.598 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.598 INFO:teuthology.orchestra.run.smithi107.stdout: "smtp_sender": { 2024-03-08T21:15:40.598 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "smtp_sender", 2024-03-08T21:15:40.598 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:40.598 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.598 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:40.598 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:40.598 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.598 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.598 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.598 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "SMTP envelope sender", 2024-03-08T21:15:40.598 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.598 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.599 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.599 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.599 INFO:teuthology.orchestra.run.smithi107.stdout: "smtp_ssl": { 2024-03-08T21:15:40.599 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "smtp_ssl", 2024-03-08T21:15:40.599 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:40.599 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.599 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:40.599 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-08T21:15:40.599 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.599 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.599 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.599 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Use SSL to connect to SMTP server", 2024-03-08T21:15:40.599 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.599 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.600 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.600 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.600 INFO:teuthology.orchestra.run.smithi107.stdout: "smtp_user": { 2024-03-08T21:15:40.600 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "smtp_user", 2024-03-08T21:15:40.600 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:40.600 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.600 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:40.600 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:40.600 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.600 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.600 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.600 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "User to authenticate as", 2024-03-08T21:15:40.600 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.600 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.600 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.600 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:40.601 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:40.601 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.601 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-08T21:15:40.601 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "balancer", 2024-03-08T21:15:40.601 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-03-08T21:15:40.601 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-03-08T21:15:40.601 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-03-08T21:15:40.601 INFO:teuthology.orchestra.run.smithi107.stdout: "active": { 2024-03-08T21:15:40.601 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "active", 2024-03-08T21:15:40.601 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:40.601 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.601 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:40.601 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-08T21:15:40.601 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.602 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.602 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.602 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "automatically balance PGs across cluster", 2024-03-08T21:15:40.602 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.602 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.602 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.602 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.602 INFO:teuthology.orchestra.run.smithi107.stdout: "begin_time": { 2024-03-08T21:15:40.602 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "begin_time", 2024-03-08T21:15:40.602 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:40.602 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.602 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:40.602 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0000", 2024-03-08T21:15:40.602 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.602 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.602 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.603 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "beginning time of day to automatically balance", 2024-03-08T21:15:40.603 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-08T21:15:40.603 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.603 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.603 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.603 INFO:teuthology.orchestra.run.smithi107.stdout: "begin_weekday": { 2024-03-08T21:15:40.603 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "begin_weekday", 2024-03-08T21:15:40.603 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "uint", 2024-03-08T21:15:40.603 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.603 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:40.603 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0", 2024-03-08T21:15:40.603 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "0", 2024-03-08T21:15:40.603 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "6", 2024-03-08T21:15:40.603 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.604 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-03-08T21:15:40.604 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-08T21:15:40.604 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.604 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.604 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.604 INFO:teuthology.orchestra.run.smithi107.stdout: "crush_compat_max_iterations": { 2024-03-08T21:15:40.604 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "crush_compat_max_iterations", 2024-03-08T21:15:40.604 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "uint", 2024-03-08T21:15:40.604 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.604 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:40.604 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "25", 2024-03-08T21:15:40.604 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "1", 2024-03-08T21:15:40.604 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "250", 2024-03-08T21:15:40.604 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.604 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-03-08T21:15:40.605 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.605 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.605 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.605 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.605 INFO:teuthology.orchestra.run.smithi107.stdout: "crush_compat_metrics": { 2024-03-08T21:15:40.605 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "crush_compat_metrics", 2024-03-08T21:15:40.605 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:40.605 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.605 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:40.605 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "pgs,objects,bytes", 2024-03-08T21:15:40.605 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.605 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.605 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.605 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-03-08T21:15:40.605 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-03-08T21:15:40.606 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.606 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.606 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.606 INFO:teuthology.orchestra.run.smithi107.stdout: "crush_compat_step": { 2024-03-08T21:15:40.606 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "crush_compat_step", 2024-03-08T21:15:40.606 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "float", 2024-03-08T21:15:40.606 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.606 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:40.606 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0.5", 2024-03-08T21:15:40.606 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "0.001", 2024-03-08T21:15:40.606 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "0.999", 2024-03-08T21:15:40.606 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.606 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "aggressiveness of optimization", 2024-03-08T21:15:40.607 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-03-08T21:15:40.607 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.607 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.607 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.607 INFO:teuthology.orchestra.run.smithi107.stdout: "end_time": { 2024-03-08T21:15:40.607 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "end_time", 2024-03-08T21:15:40.607 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:40.607 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.607 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:40.607 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "2359", 2024-03-08T21:15:40.607 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.607 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.607 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.607 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "ending time of day to automatically balance", 2024-03-08T21:15:40.607 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-08T21:15:40.608 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.608 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.608 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.608 INFO:teuthology.orchestra.run.smithi107.stdout: "end_weekday": { 2024-03-08T21:15:40.608 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "end_weekday", 2024-03-08T21:15:40.608 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "uint", 2024-03-08T21:15:40.608 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.608 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:40.608 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0", 2024-03-08T21:15:40.608 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "0", 2024-03-08T21:15:40.608 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "6", 2024-03-08T21:15:40.608 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.608 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-03-08T21:15:40.608 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-08T21:15:40.608 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.609 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.609 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.609 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-03-08T21:15:40.609 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-03-08T21:15:40.609 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:40.609 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.609 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:40.609 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:40.609 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.609 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.609 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:40.609 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-08T21:15:40.609 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-08T21:15:40.609 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-08T21:15:40.609 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-08T21:15:40.609 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-08T21:15:40.609 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-08T21:15:40.610 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:40.610 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.610 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.610 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.610 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.610 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.610 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-03-08T21:15:40.610 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-03-08T21:15:40.610 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:40.610 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.610 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:40.610 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:40.610 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.610 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.610 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.611 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.611 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.611 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.611 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.611 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.611 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-03-08T21:15:40.611 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-03-08T21:15:40.611 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:40.611 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.611 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:40.611 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-03-08T21:15:40.611 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.611 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.611 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:40.611 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-08T21:15:40.611 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-08T21:15:40.612 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-08T21:15:40.612 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-08T21:15:40.612 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-08T21:15:40.612 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-08T21:15:40.612 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:40.612 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.612 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.612 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.612 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.612 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.612 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-03-08T21:15:40.612 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-03-08T21:15:40.612 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:40.612 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.612 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:40.613 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:40.613 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.613 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.613 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.613 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.613 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.613 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.613 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.613 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.613 INFO:teuthology.orchestra.run.smithi107.stdout: "min_score": { 2024-03-08T21:15:40.613 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "min_score", 2024-03-08T21:15:40.613 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "float", 2024-03-08T21:15:40.613 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.613 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:40.613 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0", 2024-03-08T21:15:40.613 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.614 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.614 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.614 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-03-08T21:15:40.614 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.614 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.614 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.614 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.614 INFO:teuthology.orchestra.run.smithi107.stdout: "mode": { 2024-03-08T21:15:40.614 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "mode", 2024-03-08T21:15:40.614 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:40.614 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.614 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:40.614 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "upmap", 2024-03-08T21:15:40.614 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.614 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.615 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:40.615 INFO:teuthology.orchestra.run.smithi107.stdout: "crush-compat", 2024-03-08T21:15:40.615 INFO:teuthology.orchestra.run.smithi107.stdout: "none", 2024-03-08T21:15:40.615 INFO:teuthology.orchestra.run.smithi107.stdout: "read", 2024-03-08T21:15:40.615 INFO:teuthology.orchestra.run.smithi107.stdout: "upmap", 2024-03-08T21:15:40.615 INFO:teuthology.orchestra.run.smithi107.stdout: "upmap-read" 2024-03-08T21:15:40.615 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:40.615 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Balancer mode", 2024-03-08T21:15:40.615 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.615 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.615 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.615 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.615 INFO:teuthology.orchestra.run.smithi107.stdout: "pool_ids": { 2024-03-08T21:15:40.615 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "pool_ids", 2024-03-08T21:15:40.615 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:40.616 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.616 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:40.616 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:40.616 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.616 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.616 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.616 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-03-08T21:15:40.616 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.616 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.616 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.616 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.616 INFO:teuthology.orchestra.run.smithi107.stdout: "sleep_interval": { 2024-03-08T21:15:40.616 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "sleep_interval", 2024-03-08T21:15:40.616 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-03-08T21:15:40.617 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.617 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:40.617 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "60", 2024-03-08T21:15:40.617 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.617 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.617 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.617 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-03-08T21:15:40.617 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.617 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.617 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.617 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.617 INFO:teuthology.orchestra.run.smithi107.stdout: "upmap_max_deviation": { 2024-03-08T21:15:40.617 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "upmap_max_deviation", 2024-03-08T21:15:40.617 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-08T21:15:40.617 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.617 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:40.618 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "5", 2024-03-08T21:15:40.618 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "1", 2024-03-08T21:15:40.618 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.618 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.618 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "deviation below which no optimization is attempted", 2024-03-08T21:15:40.618 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-03-08T21:15:40.618 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.618 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.618 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.618 INFO:teuthology.orchestra.run.smithi107.stdout: "upmap_max_optimizations": { 2024-03-08T21:15:40.618 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "upmap_max_optimizations", 2024-03-08T21:15:40.618 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "uint", 2024-03-08T21:15:40.618 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.618 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:40.618 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "10", 2024-03-08T21:15:40.618 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.619 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.619 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.619 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-03-08T21:15:40.619 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.619 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.619 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.619 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:40.619 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:40.619 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.619 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-08T21:15:40.619 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "cephadm", 2024-03-08T21:15:40.619 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-03-08T21:15:40.619 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-03-08T21:15:40.619 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-03-08T21:15:40.619 INFO:teuthology.orchestra.run.smithi107.stdout: "agent_down_multiplier": { 2024-03-08T21:15:40.619 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "agent_down_multiplier", 2024-03-08T21:15:40.620 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "float", 2024-03-08T21:15:40.620 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.620 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:40.620 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "3.0", 2024-03-08T21:15:40.620 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.620 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.620 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.620 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-03-08T21:15:40.620 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.620 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.620 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.620 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.620 INFO:teuthology.orchestra.run.smithi107.stdout: "agent_refresh_rate": { 2024-03-08T21:15:40.620 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "agent_refresh_rate", 2024-03-08T21:15:40.620 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-03-08T21:15:40.621 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.621 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:40.621 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "20", 2024-03-08T21:15:40.621 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.621 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.621 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.621 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-03-08T21:15:40.621 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.621 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.621 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.621 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.621 INFO:teuthology.orchestra.run.smithi107.stdout: "agent_starting_port": { 2024-03-08T21:15:40.621 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "agent_starting_port", 2024-03-08T21:15:40.621 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-08T21:15:40.621 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.621 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:40.622 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "4721", 2024-03-08T21:15:40.622 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.622 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.622 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.622 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-03-08T21:15:40.622 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.622 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.622 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.622 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.622 INFO:teuthology.orchestra.run.smithi107.stdout: "allow_ptrace": { 2024-03-08T21:15:40.622 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "allow_ptrace", 2024-03-08T21:15:40.622 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:40.622 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.622 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:40.622 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:40.623 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.623 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.623 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.623 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-03-08T21:15:40.623 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-03-08T21:15:40.623 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.623 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.623 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.623 INFO:teuthology.orchestra.run.smithi107.stdout: "autotune_interval": { 2024-03-08T21:15:40.623 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "autotune_interval", 2024-03-08T21:15:40.623 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-03-08T21:15:40.623 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.623 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:40.623 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "600", 2024-03-08T21:15:40.624 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.624 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.624 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.624 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "how frequently to autotune daemon memory", 2024-03-08T21:15:40.624 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.624 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.624 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.624 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.624 INFO:teuthology.orchestra.run.smithi107.stdout: "autotune_memory_target_ratio": { 2024-03-08T21:15:40.624 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "autotune_memory_target_ratio", 2024-03-08T21:15:40.624 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "float", 2024-03-08T21:15:40.624 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.624 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:40.624 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0.7", 2024-03-08T21:15:40.624 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.624 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.625 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.625 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-03-08T21:15:40.625 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.625 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.625 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.625 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.625 INFO:teuthology.orchestra.run.smithi107.stdout: "cephadm_log_destination": { 2024-03-08T21:15:40.625 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "cephadm_log_destination", 2024-03-08T21:15:40.625 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:40.625 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.625 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:40.625 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:40.625 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.625 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.625 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:40.625 INFO:teuthology.orchestra.run.smithi107.stdout: "file", 2024-03-08T21:15:40.626 INFO:teuthology.orchestra.run.smithi107.stdout: "file,syslog", 2024-03-08T21:15:40.626 INFO:teuthology.orchestra.run.smithi107.stdout: "syslog" 2024-03-08T21:15:40.626 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:40.626 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-03-08T21:15:40.626 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.626 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.626 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.626 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.626 INFO:teuthology.orchestra.run.smithi107.stdout: "cgroups_split": { 2024-03-08T21:15:40.626 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "cgroups_split", 2024-03-08T21:15:40.626 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:40.626 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.626 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:40.626 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-08T21:15:40.626 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.627 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.627 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.627 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-03-08T21:15:40.627 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.627 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.627 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.627 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.627 INFO:teuthology.orchestra.run.smithi107.stdout: "config_checks_enabled": { 2024-03-08T21:15:40.627 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "config_checks_enabled", 2024-03-08T21:15:40.627 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:40.627 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.627 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:40.627 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:40.627 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.627 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.627 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.628 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-03-08T21:15:40.628 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.628 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.628 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.628 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.628 INFO:teuthology.orchestra.run.smithi107.stdout: "config_dashboard": { 2024-03-08T21:15:40.628 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "config_dashboard", 2024-03-08T21:15:40.628 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:40.628 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.628 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:40.628 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-08T21:15:40.628 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.628 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.628 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.628 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-03-08T21:15:40.628 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.629 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.629 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.629 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.629 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_alertmanager": { 2024-03-08T21:15:40.629 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_alertmanager", 2024-03-08T21:15:40.629 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:40.629 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.629 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:40.629 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-03-08T21:15:40.629 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.629 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.629 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.629 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Prometheus container image", 2024-03-08T21:15:40.629 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.629 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.630 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.630 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.630 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_base": { 2024-03-08T21:15:40.630 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_base", 2024-03-08T21:15:40.630 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:40.630 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.630 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:40.630 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/ceph/ceph", 2024-03-08T21:15:40.630 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.630 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.630 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.630 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Container image name, without the tag", 2024-03-08T21:15:40.630 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.630 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.630 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.631 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.631 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_elasticsearch": { 2024-03-08T21:15:40.631 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_elasticsearch", 2024-03-08T21:15:40.631 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:40.631 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.631 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:40.631 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-03-08T21:15:40.631 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.631 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.631 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.631 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "elasticsearch container image", 2024-03-08T21:15:40.631 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.631 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.631 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.631 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.631 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_grafana": { 2024-03-08T21:15:40.632 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_grafana", 2024-03-08T21:15:40.632 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:40.632 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.632 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:40.632 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.12", 2024-03-08T21:15:40.632 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.632 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.632 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.632 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Prometheus container image", 2024-03-08T21:15:40.632 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.632 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.632 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.632 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.632 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_haproxy": { 2024-03-08T21:15:40.632 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_haproxy", 2024-03-08T21:15:40.632 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:40.633 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.633 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:40.633 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-03-08T21:15:40.633 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.633 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.633 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.633 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "HAproxy container image", 2024-03-08T21:15:40.633 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.633 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.633 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.633 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.633 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_jaeger_agent": { 2024-03-08T21:15:40.633 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_jaeger_agent", 2024-03-08T21:15:40.633 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:40.633 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.634 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:40.634 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-03-08T21:15:40.634 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.634 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.634 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.634 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Jaeger agent container image", 2024-03-08T21:15:40.634 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.634 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.634 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.634 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.634 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_jaeger_collector": { 2024-03-08T21:15:40.634 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_jaeger_collector", 2024-03-08T21:15:40.634 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:40.634 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.634 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:40.634 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-03-08T21:15:40.635 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.635 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.635 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.635 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Jaeger collector container image", 2024-03-08T21:15:40.635 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.635 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.635 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.635 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.635 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_jaeger_query": { 2024-03-08T21:15:40.635 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_jaeger_query", 2024-03-08T21:15:40.635 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:40.635 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.635 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:40.635 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-03-08T21:15:40.635 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.635 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.636 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.636 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Jaeger query container image", 2024-03-08T21:15:40.636 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.636 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.636 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.636 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.636 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_keepalived": { 2024-03-08T21:15:40.636 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_keepalived", 2024-03-08T21:15:40.636 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:40.636 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.636 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:40.636 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-03-08T21:15:40.636 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.636 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.636 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.637 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Keepalived container image", 2024-03-08T21:15:40.637 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.637 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.637 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.637 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.637 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_loki": { 2024-03-08T21:15:40.637 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_loki", 2024-03-08T21:15:40.637 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:40.637 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.637 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:40.637 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-03-08T21:15:40.637 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.637 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.637 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.637 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Loki container image", 2024-03-08T21:15:40.637 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.638 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.638 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.638 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.638 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_node_exporter": { 2024-03-08T21:15:40.638 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_node_exporter", 2024-03-08T21:15:40.638 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:40.638 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.638 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:40.638 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-03-08T21:15:40.638 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.638 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.638 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.638 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Prometheus container image", 2024-03-08T21:15:40.638 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.639 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.639 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.639 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.639 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_nvmeof": { 2024-03-08T21:15:40.639 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_nvmeof", 2024-03-08T21:15:40.639 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:40.639 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.639 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:40.639 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/ceph/nvmeof:1.0.0", 2024-03-08T21:15:40.639 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.639 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.639 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.639 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Nvme-of container image", 2024-03-08T21:15:40.639 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.639 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.639 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.640 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.640 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_prometheus": { 2024-03-08T21:15:40.640 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_prometheus", 2024-03-08T21:15:40.640 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:40.640 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.640 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:40.640 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-03-08T21:15:40.640 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.640 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.640 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.640 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Prometheus container image", 2024-03-08T21:15:40.640 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.640 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.640 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.640 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.641 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_promtail": { 2024-03-08T21:15:40.641 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_promtail", 2024-03-08T21:15:40.641 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:40.641 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.641 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:40.641 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-03-08T21:15:40.641 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.641 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.641 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.641 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Promtail container image", 2024-03-08T21:15:40.641 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.641 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.641 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.641 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.641 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_snmp_gateway": { 2024-03-08T21:15:40.641 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_snmp_gateway", 2024-03-08T21:15:40.642 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:40.642 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.642 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:40.642 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-03-08T21:15:40.642 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.642 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.642 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.642 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "SNMP Gateway container image", 2024-03-08T21:15:40.642 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.642 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.642 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.642 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.642 INFO:teuthology.orchestra.run.smithi107.stdout: "container_init": { 2024-03-08T21:15:40.642 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_init", 2024-03-08T21:15:40.642 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:40.642 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.643 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:40.643 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-08T21:15:40.643 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.643 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.643 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.643 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Run podman/docker with `--init`", 2024-03-08T21:15:40.643 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.643 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.643 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.643 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.643 INFO:teuthology.orchestra.run.smithi107.stdout: "daemon_cache_timeout": { 2024-03-08T21:15:40.643 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "daemon_cache_timeout", 2024-03-08T21:15:40.643 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-03-08T21:15:40.643 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.643 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:40.643 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "600", 2024-03-08T21:15:40.644 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.644 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.644 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.644 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-03-08T21:15:40.644 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.644 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.644 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.644 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.644 INFO:teuthology.orchestra.run.smithi107.stdout: "default_cephadm_command_timeout": { 2024-03-08T21:15:40.644 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "default_cephadm_command_timeout", 2024-03-08T21:15:40.644 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-08T21:15:40.644 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.644 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:40.644 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "900", 2024-03-08T21:15:40.644 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.645 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.645 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.645 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-03-08T21:15:40.645 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.645 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.645 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.645 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.645 INFO:teuthology.orchestra.run.smithi107.stdout: "default_registry": { 2024-03-08T21:15:40.645 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "default_registry", 2024-03-08T21:15:40.645 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:40.645 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.645 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:40.645 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "docker.io", 2024-03-08T21:15:40.645 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.645 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.645 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.646 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-03-08T21:15:40.646 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.646 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.646 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.646 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.646 INFO:teuthology.orchestra.run.smithi107.stdout: "device_cache_timeout": { 2024-03-08T21:15:40.646 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "device_cache_timeout", 2024-03-08T21:15:40.646 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-03-08T21:15:40.646 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.646 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:40.646 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "1800", 2024-03-08T21:15:40.646 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.646 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.646 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.646 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "seconds to cache device inventory", 2024-03-08T21:15:40.646 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.647 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.647 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.647 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.647 INFO:teuthology.orchestra.run.smithi107.stdout: "device_enhanced_scan": { 2024-03-08T21:15:40.647 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "device_enhanced_scan", 2024-03-08T21:15:40.647 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:40.647 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.647 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:40.647 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:40.647 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.647 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.647 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.647 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Use libstoragemgmt during device scans", 2024-03-08T21:15:40.647 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.647 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.648 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.648 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.648 INFO:teuthology.orchestra.run.smithi107.stdout: "facts_cache_timeout": { 2024-03-08T21:15:40.648 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "facts_cache_timeout", 2024-03-08T21:15:40.648 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-03-08T21:15:40.648 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.648 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:40.648 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "60", 2024-03-08T21:15:40.648 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.648 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.648 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.648 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "seconds to cache host facts data", 2024-03-08T21:15:40.648 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.648 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.648 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.648 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.649 INFO:teuthology.orchestra.run.smithi107.stdout: "host_check_interval": { 2024-03-08T21:15:40.649 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "host_check_interval", 2024-03-08T21:15:40.649 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-03-08T21:15:40.649 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.649 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:40.649 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "600", 2024-03-08T21:15:40.649 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.649 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.649 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.649 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "how frequently to perform a host check", 2024-03-08T21:15:40.649 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.649 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.649 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.649 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.649 INFO:teuthology.orchestra.run.smithi107.stdout: "hw_monitoring": { 2024-03-08T21:15:40.649 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "hw_monitoring", 2024-03-08T21:15:40.650 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:40.650 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.650 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:40.650 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:40.650 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.650 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.650 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.650 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-03-08T21:15:40.650 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.650 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.650 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.650 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.650 INFO:teuthology.orchestra.run.smithi107.stdout: "inventory_list_all": { 2024-03-08T21:15:40.650 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "inventory_list_all", 2024-03-08T21:15:40.650 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:40.650 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.651 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:40.651 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:40.651 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.651 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.651 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.651 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-03-08T21:15:40.651 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.651 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.651 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.651 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.651 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-03-08T21:15:40.651 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-03-08T21:15:40.651 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:40.651 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.651 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:40.651 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:40.651 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.652 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.652 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:40.652 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-08T21:15:40.652 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-08T21:15:40.652 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-08T21:15:40.652 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-08T21:15:40.652 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-08T21:15:40.652 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-08T21:15:40.652 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:40.652 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.652 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.652 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.652 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.652 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.652 INFO:teuthology.orchestra.run.smithi107.stdout: "log_refresh_metadata": { 2024-03-08T21:15:40.652 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_refresh_metadata", 2024-03-08T21:15:40.653 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:40.653 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.653 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:40.653 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:40.653 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.653 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.653 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.653 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-03-08T21:15:40.653 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.653 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.653 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.653 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.653 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-03-08T21:15:40.653 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-03-08T21:15:40.653 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:40.653 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.653 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:40.654 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-08T21:15:40.654 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.654 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.654 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.654 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-03-08T21:15:40.654 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.654 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.654 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.654 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.654 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-03-08T21:15:40.654 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-03-08T21:15:40.654 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:40.654 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.654 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:40.654 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-03-08T21:15:40.655 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.655 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.655 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:40.655 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-08T21:15:40.655 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-08T21:15:40.655 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-08T21:15:40.655 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-08T21:15:40.655 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-08T21:15:40.655 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-08T21:15:40.655 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:40.655 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.655 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.655 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.655 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.655 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.655 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-03-08T21:15:40.656 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-03-08T21:15:40.656 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:40.656 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.656 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:40.656 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:40.656 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.656 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.656 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.656 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.656 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.656 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.656 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.656 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.656 INFO:teuthology.orchestra.run.smithi107.stdout: "manage_etc_ceph_ceph_conf": { 2024-03-08T21:15:40.656 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-03-08T21:15:40.656 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:40.657 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.657 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:40.657 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:40.657 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.657 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.657 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.657 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-03-08T21:15:40.657 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.657 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.657 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.657 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.657 INFO:teuthology.orchestra.run.smithi107.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-03-08T21:15:40.657 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-03-08T21:15:40.657 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:40.657 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.658 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:40.658 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "*", 2024-03-08T21:15:40.658 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.658 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.658 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.658 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-03-08T21:15:40.658 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.658 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.658 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.658 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.658 INFO:teuthology.orchestra.run.smithi107.stdout: "max_count_per_host": { 2024-03-08T21:15:40.658 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "max_count_per_host", 2024-03-08T21:15:40.658 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-08T21:15:40.658 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.658 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:40.659 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "10", 2024-03-08T21:15:40.659 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.659 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.659 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.659 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "max number of daemons per service per host", 2024-03-08T21:15:40.659 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.659 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.659 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.659 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.659 INFO:teuthology.orchestra.run.smithi107.stdout: "max_osd_draining_count": { 2024-03-08T21:15:40.659 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "max_osd_draining_count", 2024-03-08T21:15:40.659 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-08T21:15:40.659 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.659 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:40.659 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "10", 2024-03-08T21:15:40.659 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.660 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.660 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.660 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-03-08T21:15:40.660 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.660 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.660 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.660 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.660 INFO:teuthology.orchestra.run.smithi107.stdout: "migration_current": { 2024-03-08T21:15:40.660 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "migration_current", 2024-03-08T21:15:40.660 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-08T21:15:40.660 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.660 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:40.660 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:40.660 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.660 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.660 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.661 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "internal - do not modify", 2024-03-08T21:15:40.661 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.661 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.661 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.661 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.661 INFO:teuthology.orchestra.run.smithi107.stdout: "mode": { 2024-03-08T21:15:40.661 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "mode", 2024-03-08T21:15:40.661 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:40.661 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.661 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:40.661 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "root", 2024-03-08T21:15:40.661 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.661 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.661 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:40.661 INFO:teuthology.orchestra.run.smithi107.stdout: "cephadm-package", 2024-03-08T21:15:40.662 INFO:teuthology.orchestra.run.smithi107.stdout: "root" 2024-03-08T21:15:40.662 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:40.662 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "mode for remote execution of cephadm", 2024-03-08T21:15:40.662 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.662 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.662 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.662 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.662 INFO:teuthology.orchestra.run.smithi107.stdout: "oob_default_addr": { 2024-03-08T21:15:40.662 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "oob_default_addr", 2024-03-08T21:15:40.662 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:40.662 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.662 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:40.662 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "169.254.1.1", 2024-03-08T21:15:40.662 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.662 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.662 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.663 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Default address for RedFish API (oob management).", 2024-03-08T21:15:40.663 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.663 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.663 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.663 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.663 INFO:teuthology.orchestra.run.smithi107.stdout: "prometheus_alerts_path": { 2024-03-08T21:15:40.663 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "prometheus_alerts_path", 2024-03-08T21:15:40.663 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:40.663 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.663 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:40.663 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-03-08T21:15:40.663 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.663 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.663 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.663 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-03-08T21:15:40.664 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.664 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.664 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.664 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.664 INFO:teuthology.orchestra.run.smithi107.stdout: "registry_insecure": { 2024-03-08T21:15:40.664 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "registry_insecure", 2024-03-08T21:15:40.664 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:40.664 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.664 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:40.664 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:40.664 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.664 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.664 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.664 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-03-08T21:15:40.664 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.665 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.665 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.665 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.665 INFO:teuthology.orchestra.run.smithi107.stdout: "registry_password": { 2024-03-08T21:15:40.665 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "registry_password", 2024-03-08T21:15:40.665 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:40.665 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.665 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:40.665 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:40.665 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.665 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.665 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.665 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-03-08T21:15:40.665 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.665 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.666 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.666 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.666 INFO:teuthology.orchestra.run.smithi107.stdout: "registry_url": { 2024-03-08T21:15:40.666 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "registry_url", 2024-03-08T21:15:40.666 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:40.666 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.666 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:40.666 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:40.666 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.666 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.666 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.666 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-03-08T21:15:40.666 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.666 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.666 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.667 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.667 INFO:teuthology.orchestra.run.smithi107.stdout: "registry_username": { 2024-03-08T21:15:40.667 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "registry_username", 2024-03-08T21:15:40.667 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:40.667 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.667 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:40.667 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:40.667 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.667 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.667 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.667 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-03-08T21:15:40.667 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.667 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.667 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.667 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.667 INFO:teuthology.orchestra.run.smithi107.stdout: "secure_monitoring_stack": { 2024-03-08T21:15:40.668 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "secure_monitoring_stack", 2024-03-08T21:15:40.668 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:40.668 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.668 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:40.668 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:40.668 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.668 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.668 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.668 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-03-08T21:15:40.668 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.668 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.668 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.668 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.668 INFO:teuthology.orchestra.run.smithi107.stdout: "service_discovery_port": { 2024-03-08T21:15:40.668 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "service_discovery_port", 2024-03-08T21:15:40.669 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-08T21:15:40.669 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.669 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:40.669 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "8765", 2024-03-08T21:15:40.669 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.669 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.669 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.669 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "cephadm service discovery port", 2024-03-08T21:15:40.669 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.669 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.669 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.669 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.669 INFO:teuthology.orchestra.run.smithi107.stdout: "ssh_config_file": { 2024-03-08T21:15:40.669 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "ssh_config_file", 2024-03-08T21:15:40.669 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:40.670 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.670 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:40.670 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:40.670 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.670 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.670 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.670 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-03-08T21:15:40.670 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.670 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.670 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.670 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.670 INFO:teuthology.orchestra.run.smithi107.stdout: "use_agent": { 2024-03-08T21:15:40.670 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "use_agent", 2024-03-08T21:15:40.670 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:40.670 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.671 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:40.671 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:40.671 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.671 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.671 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.671 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-03-08T21:15:40.671 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.671 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.671 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.671 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.671 INFO:teuthology.orchestra.run.smithi107.stdout: "use_repo_digest": { 2024-03-08T21:15:40.671 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "use_repo_digest", 2024-03-08T21:15:40.671 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:40.671 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.671 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:40.672 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-08T21:15:40.672 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.672 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.672 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.672 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-03-08T21:15:40.672 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.672 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.672 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.672 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.672 INFO:teuthology.orchestra.run.smithi107.stdout: "warn_on_failed_host_check": { 2024-03-08T21:15:40.672 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "warn_on_failed_host_check", 2024-03-08T21:15:40.672 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:40.672 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.672 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:40.672 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-08T21:15:40.672 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.673 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.673 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.673 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "raise a health warning if the host check fails", 2024-03-08T21:15:40.673 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.673 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.673 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.673 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.673 INFO:teuthology.orchestra.run.smithi107.stdout: "warn_on_stray_daemons": { 2024-03-08T21:15:40.673 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "warn_on_stray_daemons", 2024-03-08T21:15:40.673 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:40.673 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.673 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:40.673 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-08T21:15:40.673 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.673 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.673 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.674 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-03-08T21:15:40.674 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.674 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.674 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.674 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.674 INFO:teuthology.orchestra.run.smithi107.stdout: "warn_on_stray_hosts": { 2024-03-08T21:15:40.674 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "warn_on_stray_hosts", 2024-03-08T21:15:40.674 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:40.674 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.674 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:40.674 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-08T21:15:40.674 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.674 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.674 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.674 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-03-08T21:15:40.675 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.675 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.675 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.675 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:40.675 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:40.675 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.675 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-08T21:15:40.675 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "crash", 2024-03-08T21:15:40.675 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-03-08T21:15:40.675 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-03-08T21:15:40.675 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-03-08T21:15:40.675 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-03-08T21:15:40.675 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-03-08T21:15:40.675 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:40.676 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.676 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:40.676 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:40.676 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.676 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.676 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:40.676 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-08T21:15:40.676 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-08T21:15:40.676 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-08T21:15:40.676 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-08T21:15:40.676 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-08T21:15:40.676 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-08T21:15:40.676 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:40.676 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.676 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.676 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.677 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.677 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.677 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-03-08T21:15:40.677 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-03-08T21:15:40.677 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:40.677 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.677 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:40.677 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:40.677 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.677 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.677 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.677 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.677 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.677 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.677 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.677 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.678 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-03-08T21:15:40.678 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-03-08T21:15:40.678 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:40.678 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.678 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:40.678 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-03-08T21:15:40.678 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.678 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.678 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:40.678 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-08T21:15:40.678 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-08T21:15:40.678 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-08T21:15:40.678 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-08T21:15:40.678 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-08T21:15:40.678 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-08T21:15:40.678 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:40.679 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.679 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.679 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.679 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.679 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.679 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-03-08T21:15:40.679 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-03-08T21:15:40.679 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:40.679 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.679 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:40.679 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:40.679 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.679 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.679 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.679 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.680 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.680 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.680 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.680 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.680 INFO:teuthology.orchestra.run.smithi107.stdout: "retain_interval": { 2024-03-08T21:15:40.680 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "retain_interval", 2024-03-08T21:15:40.680 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-03-08T21:15:40.680 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.680 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:40.680 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "31536000", 2024-03-08T21:15:40.680 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.680 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.680 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.680 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "how long to retain crashes before pruning them", 2024-03-08T21:15:40.681 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.681 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.681 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.681 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.681 INFO:teuthology.orchestra.run.smithi107.stdout: "warn_recent_interval": { 2024-03-08T21:15:40.681 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "warn_recent_interval", 2024-03-08T21:15:40.681 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-03-08T21:15:40.681 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.681 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:40.681 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "1209600", 2024-03-08T21:15:40.681 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.681 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.681 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.681 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "time interval in which to warn about recent crashes", 2024-03-08T21:15:40.681 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.682 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.682 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.682 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:40.682 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:40.682 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.682 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-08T21:15:40.682 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "dashboard", 2024-03-08T21:15:40.682 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-03-08T21:15:40.682 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-03-08T21:15:40.682 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-03-08T21:15:40.682 INFO:teuthology.orchestra.run.smithi107.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-03-08T21:15:40.682 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-03-08T21:15:40.682 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-08T21:15:40.682 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.682 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:40.683 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "10", 2024-03-08T21:15:40.683 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.683 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.683 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.683 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.683 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.683 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.683 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.683 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.683 INFO:teuthology.orchestra.run.smithi107.stdout: "ALERTMANAGER_API_HOST": { 2024-03-08T21:15:40.683 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "ALERTMANAGER_API_HOST", 2024-03-08T21:15:40.683 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:40.683 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.683 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:40.683 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:40.683 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.684 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.684 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.684 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.684 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.684 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.684 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.684 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.684 INFO:teuthology.orchestra.run.smithi107.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-03-08T21:15:40.684 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-03-08T21:15:40.684 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:40.684 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.684 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:40.684 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-08T21:15:40.684 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.684 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.685 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.685 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.685 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.685 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.685 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.685 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.685 INFO:teuthology.orchestra.run.smithi107.stdout: "AUDIT_API_ENABLED": { 2024-03-08T21:15:40.685 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "AUDIT_API_ENABLED", 2024-03-08T21:15:40.685 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:40.685 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.685 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:40.685 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:40.685 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.685 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.685 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.685 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.686 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.686 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.686 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.686 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.686 INFO:teuthology.orchestra.run.smithi107.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-03-08T21:15:40.686 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-03-08T21:15:40.686 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:40.686 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.686 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:40.686 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-08T21:15:40.686 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.686 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.686 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.686 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.686 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.687 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.687 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.687 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.687 INFO:teuthology.orchestra.run.smithi107.stdout: "ENABLE_BROWSABLE_API": { 2024-03-08T21:15:40.687 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "ENABLE_BROWSABLE_API", 2024-03-08T21:15:40.687 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:40.687 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.687 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:40.687 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-08T21:15:40.687 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.687 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.687 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.687 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.687 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.687 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.687 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.688 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.688 INFO:teuthology.orchestra.run.smithi107.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-03-08T21:15:40.688 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-03-08T21:15:40.688 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:40.688 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.688 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:40.688 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-08T21:15:40.688 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.688 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.688 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.688 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.688 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.688 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.688 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.688 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.688 INFO:teuthology.orchestra.run.smithi107.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-03-08T21:15:40.689 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-03-08T21:15:40.689 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:40.689 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.689 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:40.689 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-08T21:15:40.689 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.689 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.689 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.689 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.689 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.689 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.689 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.689 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.689 INFO:teuthology.orchestra.run.smithi107.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-03-08T21:15:40.689 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-03-08T21:15:40.690 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:40.690 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.690 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:40.690 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-08T21:15:40.690 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.690 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.690 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.690 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.690 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.690 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.690 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.690 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.690 INFO:teuthology.orchestra.run.smithi107.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-03-08T21:15:40.690 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-03-08T21:15:40.691 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:40.691 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.691 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:40.691 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-08T21:15:40.691 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.691 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.691 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.691 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.691 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.691 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.691 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.691 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.691 INFO:teuthology.orchestra.run.smithi107.stdout: "FEATURE_TOGGLE_NFS": { 2024-03-08T21:15:40.691 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-03-08T21:15:40.692 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:40.692 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.692 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:40.692 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-08T21:15:40.692 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.692 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.692 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.692 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.692 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.692 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.692 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.692 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.692 INFO:teuthology.orchestra.run.smithi107.stdout: "FEATURE_TOGGLE_RBD": { 2024-03-08T21:15:40.692 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-03-08T21:15:40.693 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:40.693 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.693 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:40.693 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-08T21:15:40.693 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.693 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.693 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.693 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.693 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.693 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.693 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.693 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.693 INFO:teuthology.orchestra.run.smithi107.stdout: "FEATURE_TOGGLE_RGW": { 2024-03-08T21:15:40.693 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-03-08T21:15:40.693 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:40.694 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.694 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:40.694 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-08T21:15:40.694 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.694 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.694 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.694 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.694 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.694 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.694 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.694 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.694 INFO:teuthology.orchestra.run.smithi107.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-03-08T21:15:40.694 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-03-08T21:15:40.694 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:40.695 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.695 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:40.695 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:40.695 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.695 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.695 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.695 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.695 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.695 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.695 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.695 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.695 INFO:teuthology.orchestra.run.smithi107.stdout: "GRAFANA_API_PASSWORD": { 2024-03-08T21:15:40.695 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "GRAFANA_API_PASSWORD", 2024-03-08T21:15:40.695 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:40.695 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.696 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:40.696 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "admin", 2024-03-08T21:15:40.696 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.696 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.696 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.696 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.696 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.696 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.696 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.696 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.696 INFO:teuthology.orchestra.run.smithi107.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-03-08T21:15:40.696 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-03-08T21:15:40.696 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:40.696 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.696 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:40.697 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-08T21:15:40.697 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.697 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.697 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.697 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.697 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.697 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.697 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.697 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.697 INFO:teuthology.orchestra.run.smithi107.stdout: "GRAFANA_API_URL": { 2024-03-08T21:15:40.697 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "GRAFANA_API_URL", 2024-03-08T21:15:40.697 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:40.697 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.697 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:40.698 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:40.698 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.698 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.698 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.698 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.698 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.698 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.698 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.698 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.698 INFO:teuthology.orchestra.run.smithi107.stdout: "GRAFANA_API_USERNAME": { 2024-03-08T21:15:40.698 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "GRAFANA_API_USERNAME", 2024-03-08T21:15:40.698 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:40.698 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.699 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:40.699 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "admin", 2024-03-08T21:15:40.699 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.699 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.699 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.699 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.699 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.699 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.699 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.699 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.699 INFO:teuthology.orchestra.run.smithi107.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-03-08T21:15:40.699 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-03-08T21:15:40.699 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:40.699 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.699 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:40.700 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:40.700 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.700 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.700 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.700 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.700 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.700 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.700 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.700 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.700 INFO:teuthology.orchestra.run.smithi107.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-03-08T21:15:40.700 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-03-08T21:15:40.700 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:40.700 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.700 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:40.700 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:40.701 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.701 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.701 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.701 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.701 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.701 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.701 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.701 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.701 INFO:teuthology.orchestra.run.smithi107.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-03-08T21:15:40.701 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-03-08T21:15:40.701 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:40.701 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.701 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:40.701 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-08T21:15:40.701 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.702 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.702 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.702 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.702 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.702 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.702 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.702 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.702 INFO:teuthology.orchestra.run.smithi107.stdout: "ISSUE_TRACKER_API_KEY": { 2024-03-08T21:15:40.702 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-03-08T21:15:40.702 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:40.702 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.702 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:40.702 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:40.702 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.703 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.703 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.703 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.703 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.703 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.703 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.703 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.703 INFO:teuthology.orchestra.run.smithi107.stdout: "MULTICLUSTER_CONFIG": { 2024-03-08T21:15:40.703 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "MULTICLUSTER_CONFIG", 2024-03-08T21:15:40.703 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:40.703 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.703 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:40.703 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "{}", 2024-03-08T21:15:40.704 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.704 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.704 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.704 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.704 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.704 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.704 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.704 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.704 INFO:teuthology.orchestra.run.smithi107.stdout: "PROMETHEUS_API_HOST": { 2024-03-08T21:15:40.704 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PROMETHEUS_API_HOST", 2024-03-08T21:15:40.704 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:40.704 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.704 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:40.704 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:40.704 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.704 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.705 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.705 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.705 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.705 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.705 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.705 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.705 INFO:teuthology.orchestra.run.smithi107.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-03-08T21:15:40.705 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-03-08T21:15:40.705 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:40.705 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.705 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:40.705 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-08T21:15:40.705 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.705 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.705 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.705 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.706 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.706 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.706 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.706 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.706 INFO:teuthology.orchestra.run.smithi107.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-03-08T21:15:40.706 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-03-08T21:15:40.706 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:40.706 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.706 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:40.706 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:40.706 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.706 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.706 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.707 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.707 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.707 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.707 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.707 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.707 INFO:teuthology.orchestra.run.smithi107.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-03-08T21:15:40.707 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-03-08T21:15:40.707 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:40.707 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.707 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:40.707 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:40.707 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.707 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.707 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.707 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.708 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.708 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.708 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.708 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.708 INFO:teuthology.orchestra.run.smithi107.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-03-08T21:15:40.708 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-03-08T21:15:40.708 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:40.708 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.708 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:40.708 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-08T21:15:40.708 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.708 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.708 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.708 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.708 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.709 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.709 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.709 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.709 INFO:teuthology.orchestra.run.smithi107.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-03-08T21:15:40.709 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-03-08T21:15:40.709 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:40.709 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.709 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:40.709 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-08T21:15:40.709 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.709 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.709 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.709 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.709 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.709 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.709 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.710 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.710 INFO:teuthology.orchestra.run.smithi107.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-03-08T21:15:40.710 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-03-08T21:15:40.710 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:40.710 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.710 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:40.710 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:40.710 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.710 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.710 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.710 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.710 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.710 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.710 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.710 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.710 INFO:teuthology.orchestra.run.smithi107.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-03-08T21:15:40.711 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-03-08T21:15:40.711 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:40.711 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.711 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:40.711 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:40.711 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.711 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.711 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.711 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.711 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.711 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.711 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.711 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.711 INFO:teuthology.orchestra.run.smithi107.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-03-08T21:15:40.711 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-03-08T21:15:40.712 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:40.712 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.712 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:40.712 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:40.712 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.712 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.712 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.712 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.712 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.712 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.712 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.712 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.712 INFO:teuthology.orchestra.run.smithi107.stdout: "PWD_POLICY_ENABLED": { 2024-03-08T21:15:40.712 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PWD_POLICY_ENABLED", 2024-03-08T21:15:40.712 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:40.712 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.713 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:40.713 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-08T21:15:40.713 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.713 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.713 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.713 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.713 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.713 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.713 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.713 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.713 INFO:teuthology.orchestra.run.smithi107.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-03-08T21:15:40.713 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-03-08T21:15:40.713 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:40.713 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.713 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:40.713 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-03-08T21:15:40.713 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.714 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.714 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.714 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.714 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.714 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.714 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.714 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.714 INFO:teuthology.orchestra.run.smithi107.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-03-08T21:15:40.714 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-03-08T21:15:40.714 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-08T21:15:40.714 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.714 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:40.714 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "10", 2024-03-08T21:15:40.714 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.714 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.715 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.715 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.715 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.715 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.715 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.715 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.715 INFO:teuthology.orchestra.run.smithi107.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-03-08T21:15:40.715 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-03-08T21:15:40.715 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-08T21:15:40.715 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.715 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:40.715 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "8", 2024-03-08T21:15:40.715 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.715 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.715 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.715 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.716 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.716 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.716 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.716 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.716 INFO:teuthology.orchestra.run.smithi107.stdout: "REST_REQUESTS_TIMEOUT": { 2024-03-08T21:15:40.716 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-03-08T21:15:40.716 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-08T21:15:40.716 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.716 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:40.716 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "45", 2024-03-08T21:15:40.716 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.716 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.716 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.716 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.716 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.716 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.717 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.717 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.717 INFO:teuthology.orchestra.run.smithi107.stdout: "RGW_API_ACCESS_KEY": { 2024-03-08T21:15:40.717 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "RGW_API_ACCESS_KEY", 2024-03-08T21:15:40.717 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:40.717 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.717 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:40.717 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:40.717 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.717 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.717 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.717 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.717 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.717 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.717 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.718 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.718 INFO:teuthology.orchestra.run.smithi107.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-03-08T21:15:40.718 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-03-08T21:15:40.718 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:40.718 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.718 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:40.718 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "admin", 2024-03-08T21:15:40.718 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.718 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.718 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.718 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.718 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.718 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.718 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.718 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.718 INFO:teuthology.orchestra.run.smithi107.stdout: "RGW_API_SECRET_KEY": { 2024-03-08T21:15:40.719 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "RGW_API_SECRET_KEY", 2024-03-08T21:15:40.719 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:40.719 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.719 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:40.719 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:40.719 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.719 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.719 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.719 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.719 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.719 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.719 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.719 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.719 INFO:teuthology.orchestra.run.smithi107.stdout: "RGW_API_SSL_VERIFY": { 2024-03-08T21:15:40.719 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "RGW_API_SSL_VERIFY", 2024-03-08T21:15:40.720 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:40.720 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.720 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:40.720 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-08T21:15:40.720 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.720 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.720 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.720 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.720 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.720 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.720 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.720 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.720 INFO:teuthology.orchestra.run.smithi107.stdout: "UNSAFE_TLS_v1_2": { 2024-03-08T21:15:40.720 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "UNSAFE_TLS_v1_2", 2024-03-08T21:15:40.720 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:40.720 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.720 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:40.721 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:40.721 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.721 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.721 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.721 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.721 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.721 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.721 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.721 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.721 INFO:teuthology.orchestra.run.smithi107.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-03-08T21:15:40.721 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-03-08T21:15:40.721 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-08T21:15:40.721 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.721 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:40.721 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0", 2024-03-08T21:15:40.721 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.722 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.722 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.722 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.722 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.722 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.722 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.722 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.722 INFO:teuthology.orchestra.run.smithi107.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-03-08T21:15:40.722 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-03-08T21:15:40.722 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-08T21:15:40.722 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.722 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:40.722 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "10", 2024-03-08T21:15:40.722 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.722 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.722 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.723 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.723 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.723 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.723 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.723 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.723 INFO:teuthology.orchestra.run.smithi107.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-03-08T21:15:40.723 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-03-08T21:15:40.723 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-08T21:15:40.723 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.723 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:40.723 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "5", 2024-03-08T21:15:40.723 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.723 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.723 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.723 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.723 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.724 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.724 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.724 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.724 INFO:teuthology.orchestra.run.smithi107.stdout: "cross_origin_url": { 2024-03-08T21:15:40.724 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "cross_origin_url", 2024-03-08T21:15:40.724 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:40.724 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.724 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:40.724 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:40.724 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.724 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.724 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.724 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.724 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.724 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.724 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.725 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.725 INFO:teuthology.orchestra.run.smithi107.stdout: "crt_file": { 2024-03-08T21:15:40.725 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "crt_file", 2024-03-08T21:15:40.725 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:40.725 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.725 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:40.725 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:40.725 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.725 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.725 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.725 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.725 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.725 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.725 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.725 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.725 INFO:teuthology.orchestra.run.smithi107.stdout: "debug": { 2024-03-08T21:15:40.725 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "debug", 2024-03-08T21:15:40.725 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:40.726 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.726 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:40.726 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:40.726 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.726 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.726 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.726 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Enable/disable debug options", 2024-03-08T21:15:40.726 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.726 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.726 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.726 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.726 INFO:teuthology.orchestra.run.smithi107.stdout: "jwt_token_ttl": { 2024-03-08T21:15:40.726 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "jwt_token_ttl", 2024-03-08T21:15:40.726 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-08T21:15:40.726 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.727 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:40.727 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "28800", 2024-03-08T21:15:40.727 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.727 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.727 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.727 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.727 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.727 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.727 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.727 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.727 INFO:teuthology.orchestra.run.smithi107.stdout: "key_file": { 2024-03-08T21:15:40.727 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "key_file", 2024-03-08T21:15:40.727 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:40.727 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.727 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:40.728 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:40.728 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.728 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.728 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.728 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.728 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.728 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.728 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.728 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.728 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-03-08T21:15:40.728 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-03-08T21:15:40.728 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:40.728 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.728 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:40.728 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:40.728 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.729 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.729 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:40.729 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-08T21:15:40.729 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-08T21:15:40.729 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-08T21:15:40.729 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-08T21:15:40.729 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-08T21:15:40.729 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-08T21:15:40.729 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:40.729 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.729 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.729 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.729 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.729 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.729 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-03-08T21:15:40.729 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-03-08T21:15:40.730 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:40.730 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.730 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:40.730 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:40.730 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.730 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.730 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.730 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.730 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.730 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.730 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.730 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.730 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-03-08T21:15:40.730 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-03-08T21:15:40.730 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:40.731 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.731 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:40.731 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-03-08T21:15:40.731 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.731 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.731 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:40.731 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-08T21:15:40.731 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-08T21:15:40.731 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-08T21:15:40.731 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-08T21:15:40.731 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-08T21:15:40.731 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-08T21:15:40.731 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:40.731 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.731 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.731 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.732 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.732 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.732 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-03-08T21:15:40.732 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-03-08T21:15:40.732 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:40.732 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.732 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:40.732 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:40.732 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.732 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.732 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.732 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.732 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.732 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.732 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.733 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.733 INFO:teuthology.orchestra.run.smithi107.stdout: "motd": { 2024-03-08T21:15:40.733 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "motd", 2024-03-08T21:15:40.733 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:40.733 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.733 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:40.733 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:40.733 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.733 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.733 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.733 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "The message of the day", 2024-03-08T21:15:40.733 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.733 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.733 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.733 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.733 INFO:teuthology.orchestra.run.smithi107.stdout: "redirect_resolve_ip_addr": { 2024-03-08T21:15:40.734 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "redirect_resolve_ip_addr", 2024-03-08T21:15:40.734 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:40.734 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.734 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:40.734 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:40.734 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.734 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.734 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.734 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.734 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.734 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.734 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.734 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.734 INFO:teuthology.orchestra.run.smithi107.stdout: "server_addr": { 2024-03-08T21:15:40.734 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "server_addr", 2024-03-08T21:15:40.734 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:40.735 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.735 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:40.735 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "::", 2024-03-08T21:15:40.735 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.735 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.735 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.735 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.735 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.735 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.735 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.735 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.735 INFO:teuthology.orchestra.run.smithi107.stdout: "server_port": { 2024-03-08T21:15:40.735 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "server_port", 2024-03-08T21:15:40.735 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-08T21:15:40.735 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.735 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:40.736 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "8080", 2024-03-08T21:15:40.736 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.736 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.736 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.736 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.736 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.736 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.736 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.736 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.736 INFO:teuthology.orchestra.run.smithi107.stdout: "ssl": { 2024-03-08T21:15:40.736 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "ssl", 2024-03-08T21:15:40.736 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:40.736 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.736 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:40.736 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-08T21:15:40.736 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.737 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.737 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.737 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.737 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.737 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.737 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.737 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.737 INFO:teuthology.orchestra.run.smithi107.stdout: "ssl_server_port": { 2024-03-08T21:15:40.737 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "ssl_server_port", 2024-03-08T21:15:40.737 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-08T21:15:40.737 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.737 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:40.737 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "8443", 2024-03-08T21:15:40.737 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.737 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.737 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.738 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.738 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.738 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.738 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.738 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.738 INFO:teuthology.orchestra.run.smithi107.stdout: "standby_behaviour": { 2024-03-08T21:15:40.738 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "standby_behaviour", 2024-03-08T21:15:40.738 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:40.738 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.738 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:40.738 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "redirect", 2024-03-08T21:15:40.738 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.738 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.738 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:40.742 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-08T21:15:40.742 INFO:teuthology.orchestra.run.smithi107.stdout: "redirect" 2024-03-08T21:15:40.742 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:40.742 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.742 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.742 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.742 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.742 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.742 INFO:teuthology.orchestra.run.smithi107.stdout: "standby_error_status_code": { 2024-03-08T21:15:40.742 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "standby_error_status_code", 2024-03-08T21:15:40.743 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-08T21:15:40.743 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.743 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:40.743 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "500", 2024-03-08T21:15:40.743 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "400", 2024-03-08T21:15:40.743 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "599", 2024-03-08T21:15:40.743 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.743 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.743 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.743 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.743 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.743 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.743 INFO:teuthology.orchestra.run.smithi107.stdout: "url_prefix": { 2024-03-08T21:15:40.743 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "url_prefix", 2024-03-08T21:15:40.743 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:40.744 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.744 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:40.744 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:40.744 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.744 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.744 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.744 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.744 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.744 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.744 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.744 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:40.744 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:40.744 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.744 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-08T21:15:40.744 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "devicehealth", 2024-03-08T21:15:40.744 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-03-08T21:15:40.744 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-03-08T21:15:40.745 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-03-08T21:15:40.745 INFO:teuthology.orchestra.run.smithi107.stdout: "enable_monitoring": { 2024-03-08T21:15:40.745 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "enable_monitoring", 2024-03-08T21:15:40.745 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:40.745 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.745 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:40.745 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-08T21:15:40.745 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.745 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.745 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.745 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "monitor device health metrics", 2024-03-08T21:15:40.745 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.745 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.745 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.745 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.745 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-03-08T21:15:40.746 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-03-08T21:15:40.746 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:40.746 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.746 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:40.746 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:40.746 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.746 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.746 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:40.746 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-08T21:15:40.746 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-08T21:15:40.746 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-08T21:15:40.746 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-08T21:15:40.746 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-08T21:15:40.746 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-08T21:15:40.746 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:40.746 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.747 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.747 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.747 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.747 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.747 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-03-08T21:15:40.747 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-03-08T21:15:40.747 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:40.747 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.747 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:40.747 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:40.747 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.747 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.747 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.747 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.747 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.747 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.748 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.748 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.748 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-03-08T21:15:40.748 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-03-08T21:15:40.748 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:40.748 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.748 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:40.748 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-03-08T21:15:40.748 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.748 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.748 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:40.748 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-08T21:15:40.748 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-08T21:15:40.748 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-08T21:15:40.748 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-08T21:15:40.748 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-08T21:15:40.749 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-08T21:15:40.749 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:40.749 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.749 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.749 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.749 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.749 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.749 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-03-08T21:15:40.749 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-03-08T21:15:40.749 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:40.749 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.749 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:40.749 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:40.749 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.749 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.749 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.750 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.750 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.750 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.750 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.750 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.750 INFO:teuthology.orchestra.run.smithi107.stdout: "mark_out_threshold": { 2024-03-08T21:15:40.750 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "mark_out_threshold", 2024-03-08T21:15:40.750 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-03-08T21:15:40.750 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.750 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:40.750 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "2419200", 2024-03-08T21:15:40.750 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.750 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.750 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.751 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-03-08T21:15:40.751 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.751 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.751 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.751 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.751 INFO:teuthology.orchestra.run.smithi107.stdout: "pool_name": { 2024-03-08T21:15:40.751 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "pool_name", 2024-03-08T21:15:40.751 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:40.751 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.751 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:40.751 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "device_health_metrics", 2024-03-08T21:15:40.751 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.751 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.752 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.752 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "name of pool in which to store device health metrics", 2024-03-08T21:15:40.752 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.752 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.752 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.752 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.752 INFO:teuthology.orchestra.run.smithi107.stdout: "retention_period": { 2024-03-08T21:15:40.752 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "retention_period", 2024-03-08T21:15:40.752 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-03-08T21:15:40.752 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.752 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:40.752 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "15552000", 2024-03-08T21:15:40.752 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.752 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.753 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.753 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "how long to retain device health metrics", 2024-03-08T21:15:40.753 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.753 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.753 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.753 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.753 INFO:teuthology.orchestra.run.smithi107.stdout: "scrape_frequency": { 2024-03-08T21:15:40.753 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "scrape_frequency", 2024-03-08T21:15:40.753 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-03-08T21:15:40.753 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.753 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:40.753 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "86400", 2024-03-08T21:15:40.753 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.753 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.753 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.754 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "how frequently to scrape device health metrics", 2024-03-08T21:15:40.754 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.754 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.754 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.754 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.754 INFO:teuthology.orchestra.run.smithi107.stdout: "self_heal": { 2024-03-08T21:15:40.754 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "self_heal", 2024-03-08T21:15:40.754 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:40.754 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.754 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:40.754 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-08T21:15:40.754 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.754 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.755 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.755 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-03-08T21:15:40.755 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.755 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.755 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.755 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.755 INFO:teuthology.orchestra.run.smithi107.stdout: "sleep_interval": { 2024-03-08T21:15:40.755 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "sleep_interval", 2024-03-08T21:15:40.755 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-03-08T21:15:40.755 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.755 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:40.755 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "600", 2024-03-08T21:15:40.755 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.756 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.756 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.756 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "how frequently to wake up and check device health", 2024-03-08T21:15:40.756 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.756 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.756 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.756 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.756 INFO:teuthology.orchestra.run.smithi107.stdout: "warn_threshold": { 2024-03-08T21:15:40.756 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "warn_threshold", 2024-03-08T21:15:40.756 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-03-08T21:15:40.756 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.756 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:40.756 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "7257600", 2024-03-08T21:15:40.756 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.757 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.757 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.757 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-03-08T21:15:40.757 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.757 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.757 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.757 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:40.757 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:40.757 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.757 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-08T21:15:40.757 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "diskprediction_local", 2024-03-08T21:15:40.757 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-03-08T21:15:40.757 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-03-08T21:15:40.757 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-03-08T21:15:40.757 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-03-08T21:15:40.758 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-03-08T21:15:40.758 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:40.758 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.758 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:40.758 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:40.758 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.758 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.758 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:40.758 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-08T21:15:40.758 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-08T21:15:40.758 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-08T21:15:40.758 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-08T21:15:40.758 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-08T21:15:40.758 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-08T21:15:40.759 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:40.759 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.759 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.759 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.759 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.759 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.759 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-03-08T21:15:40.759 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-03-08T21:15:40.759 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:40.759 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.759 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:40.759 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:40.759 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.759 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.759 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.759 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.760 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.760 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.760 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.760 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.760 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-03-08T21:15:40.760 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-03-08T21:15:40.760 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:40.760 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.760 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:40.760 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-03-08T21:15:40.760 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.760 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.760 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:40.760 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-08T21:15:40.761 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-08T21:15:40.761 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-08T21:15:40.761 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-08T21:15:40.761 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-08T21:15:40.761 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-08T21:15:40.761 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:40.761 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.761 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.761 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.761 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.761 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.761 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-03-08T21:15:40.761 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-03-08T21:15:40.761 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:40.761 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.762 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:40.762 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:40.762 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.762 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.762 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.762 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.762 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.762 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.762 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.762 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.762 INFO:teuthology.orchestra.run.smithi107.stdout: "predict_interval": { 2024-03-08T21:15:40.762 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "predict_interval", 2024-03-08T21:15:40.762 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:40.762 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.763 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:40.763 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "86400", 2024-03-08T21:15:40.763 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.763 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.763 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.763 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.763 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.763 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.763 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.763 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.763 INFO:teuthology.orchestra.run.smithi107.stdout: "predictor_model": { 2024-03-08T21:15:40.763 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "predictor_model", 2024-03-08T21:15:40.763 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:40.763 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.763 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:40.764 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "prophetstor", 2024-03-08T21:15:40.764 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.764 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.764 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.764 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.764 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.764 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.764 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.764 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.764 INFO:teuthology.orchestra.run.smithi107.stdout: "sleep_interval": { 2024-03-08T21:15:40.764 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "sleep_interval", 2024-03-08T21:15:40.764 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:40.764 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.764 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:40.764 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "600", 2024-03-08T21:15:40.765 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.765 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.765 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.765 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.765 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.765 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.765 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.765 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:40.765 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:40.765 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.765 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-08T21:15:40.765 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "influx", 2024-03-08T21:15:40.765 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": false, 2024-03-08T21:15:40.765 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "influxdb python module not found", 2024-03-08T21:15:40.765 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-03-08T21:15:40.766 INFO:teuthology.orchestra.run.smithi107.stdout: "batch_size": { 2024-03-08T21:15:40.766 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "batch_size", 2024-03-08T21:15:40.766 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-08T21:15:40.766 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.766 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:40.766 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "5000", 2024-03-08T21:15:40.766 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.766 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.766 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.766 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-03-08T21:15:40.766 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.766 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.766 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.766 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.766 INFO:teuthology.orchestra.run.smithi107.stdout: "database": { 2024-03-08T21:15:40.767 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "database", 2024-03-08T21:15:40.767 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:40.767 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.767 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:40.767 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "ceph", 2024-03-08T21:15:40.767 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.767 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.767 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.767 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-03-08T21:15:40.767 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.767 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.767 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.767 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.767 INFO:teuthology.orchestra.run.smithi107.stdout: "hostname": { 2024-03-08T21:15:40.767 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "hostname", 2024-03-08T21:15:40.768 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:40.768 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.768 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:40.768 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:40.768 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.768 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.768 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.768 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "InfluxDB server hostname", 2024-03-08T21:15:40.768 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.768 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.768 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.768 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.768 INFO:teuthology.orchestra.run.smithi107.stdout: "interval": { 2024-03-08T21:15:40.768 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "interval", 2024-03-08T21:15:40.768 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-03-08T21:15:40.769 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.769 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:40.769 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "30", 2024-03-08T21:15:40.769 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "5", 2024-03-08T21:15:40.769 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.769 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.769 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-03-08T21:15:40.769 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.769 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.769 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.769 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.769 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-03-08T21:15:40.769 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-03-08T21:15:40.769 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:40.769 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.770 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:40.770 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:40.770 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.770 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.770 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:40.770 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-08T21:15:40.770 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-08T21:15:40.770 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-08T21:15:40.770 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-08T21:15:40.770 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-08T21:15:40.770 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-08T21:15:40.770 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:40.770 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.770 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.770 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.771 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.771 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.771 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-03-08T21:15:40.771 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-03-08T21:15:40.771 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:40.771 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.771 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:40.771 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:40.771 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.771 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.771 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.771 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.771 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.771 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.772 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.772 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.772 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-03-08T21:15:40.772 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-03-08T21:15:40.772 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:40.772 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.772 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:40.772 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-03-08T21:15:40.772 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.772 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.772 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:40.772 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-08T21:15:40.772 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-08T21:15:40.772 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-08T21:15:40.772 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-08T21:15:40.773 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-08T21:15:40.773 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-08T21:15:40.773 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:40.773 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.773 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.773 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.773 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.773 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.773 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-03-08T21:15:40.773 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-03-08T21:15:40.773 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:40.773 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.773 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:40.773 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:40.773 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.773 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.774 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.774 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.774 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.774 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.774 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.774 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.774 INFO:teuthology.orchestra.run.smithi107.stdout: "password": { 2024-03-08T21:15:40.774 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "password", 2024-03-08T21:15:40.774 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:40.774 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.774 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:40.774 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:40.774 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.774 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.775 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.775 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "password of InfluxDB server user", 2024-03-08T21:15:40.775 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.775 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.775 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.775 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.775 INFO:teuthology.orchestra.run.smithi107.stdout: "port": { 2024-03-08T21:15:40.775 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "port", 2024-03-08T21:15:40.775 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-08T21:15:40.775 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.775 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:40.775 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "8086", 2024-03-08T21:15:40.775 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.775 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.775 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.776 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "InfluxDB server port", 2024-03-08T21:15:40.776 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.776 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.776 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.776 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.776 INFO:teuthology.orchestra.run.smithi107.stdout: "ssl": { 2024-03-08T21:15:40.776 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "ssl", 2024-03-08T21:15:40.776 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:40.776 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.776 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:40.776 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "false", 2024-03-08T21:15:40.776 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.776 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.776 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.776 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-03-08T21:15:40.776 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.777 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.777 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.777 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.777 INFO:teuthology.orchestra.run.smithi107.stdout: "threads": { 2024-03-08T21:15:40.777 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "threads", 2024-03-08T21:15:40.777 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-08T21:15:40.777 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.777 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:40.777 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "5", 2024-03-08T21:15:40.777 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "1", 2024-03-08T21:15:40.777 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "32", 2024-03-08T21:15:40.777 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.777 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-03-08T21:15:40.777 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.777 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.777 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.777 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.778 INFO:teuthology.orchestra.run.smithi107.stdout: "username": { 2024-03-08T21:15:40.778 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "username", 2024-03-08T21:15:40.778 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:40.778 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.778 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:40.778 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:40.778 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.778 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.778 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.778 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "username of InfluxDB server user", 2024-03-08T21:15:40.778 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.778 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.778 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.778 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.779 INFO:teuthology.orchestra.run.smithi107.stdout: "verify_ssl": { 2024-03-08T21:15:40.779 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "verify_ssl", 2024-03-08T21:15:40.779 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:40.779 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.779 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:40.779 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "true", 2024-03-08T21:15:40.779 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.779 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.779 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.779 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-03-08T21:15:40.779 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.779 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.779 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.779 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:40.779 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:40.780 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.780 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-08T21:15:40.780 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "insights", 2024-03-08T21:15:40.780 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-03-08T21:15:40.780 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-03-08T21:15:40.780 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-03-08T21:15:40.780 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-03-08T21:15:40.780 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-03-08T21:15:40.780 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:40.780 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.780 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:40.780 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:40.780 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.780 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.780 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:40.781 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-08T21:15:40.781 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-08T21:15:40.781 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-08T21:15:40.781 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-08T21:15:40.781 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-08T21:15:40.781 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-08T21:15:40.781 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:40.781 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.781 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.781 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.781 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.781 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.781 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-03-08T21:15:40.781 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-03-08T21:15:40.782 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:40.782 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.782 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:40.782 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:40.782 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.782 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.782 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.782 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.782 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.782 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.782 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.782 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.782 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-03-08T21:15:40.782 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-03-08T21:15:40.783 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:40.783 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.783 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:40.783 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-03-08T21:15:40.783 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.783 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.783 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:40.783 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-08T21:15:40.783 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-08T21:15:40.783 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-08T21:15:40.783 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-08T21:15:40.783 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-08T21:15:40.783 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-08T21:15:40.783 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:40.784 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.784 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.784 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.784 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.784 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.784 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-03-08T21:15:40.784 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-03-08T21:15:40.784 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:40.784 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.784 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:40.784 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:40.784 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.784 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.784 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.784 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.784 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.785 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.785 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.785 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:40.785 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:40.785 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.785 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-08T21:15:40.785 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "iostat", 2024-03-08T21:15:40.785 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-03-08T21:15:40.785 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-03-08T21:15:40.785 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-03-08T21:15:40.785 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-03-08T21:15:40.785 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-03-08T21:15:40.785 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:40.785 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.785 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:40.786 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:40.786 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.786 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.786 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:40.786 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-08T21:15:40.786 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-08T21:15:40.786 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-08T21:15:40.786 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-08T21:15:40.786 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-08T21:15:40.786 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-08T21:15:40.786 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:40.786 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.786 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.786 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.786 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.787 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.787 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-03-08T21:15:40.787 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-03-08T21:15:40.787 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:40.787 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.787 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:40.787 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:40.787 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.787 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.787 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.787 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.787 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.787 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.787 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.787 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.788 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-03-08T21:15:40.788 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-03-08T21:15:40.788 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:40.788 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.788 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:40.788 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-03-08T21:15:40.788 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.788 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.788 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:40.788 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-08T21:15:40.788 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-08T21:15:40.788 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-08T21:15:40.788 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-08T21:15:40.788 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-08T21:15:40.788 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-08T21:15:40.788 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:40.789 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.789 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.789 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.789 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.789 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.789 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-03-08T21:15:40.789 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-03-08T21:15:40.789 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:40.789 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.789 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:40.789 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:40.789 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.789 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.789 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.789 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.789 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.790 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.790 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.790 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:40.790 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:40.790 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.790 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-08T21:15:40.790 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "k8sevents", 2024-03-08T21:15:40.790 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-03-08T21:15:40.790 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-03-08T21:15:40.790 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-03-08T21:15:40.790 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph_event_retention_days": { 2024-03-08T21:15:40.790 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "ceph_event_retention_days", 2024-03-08T21:15:40.790 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-08T21:15:40.790 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.790 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:40.791 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "7", 2024-03-08T21:15:40.791 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.791 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.791 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.791 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Days to hold ceph event information within local cache", 2024-03-08T21:15:40.791 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.791 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.791 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.791 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.791 INFO:teuthology.orchestra.run.smithi107.stdout: "config_check_secs": { 2024-03-08T21:15:40.791 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "config_check_secs", 2024-03-08T21:15:40.791 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-08T21:15:40.791 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.791 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:40.791 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "10", 2024-03-08T21:15:40.792 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "10", 2024-03-08T21:15:40.792 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.792 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.792 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-03-08T21:15:40.792 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.792 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.792 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.792 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.792 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-03-08T21:15:40.792 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-03-08T21:15:40.792 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:40.792 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.792 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:40.792 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:40.793 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.793 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.793 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:40.793 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-08T21:15:40.793 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-08T21:15:40.793 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-08T21:15:40.793 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-08T21:15:40.793 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-08T21:15:40.793 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-08T21:15:40.793 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:40.793 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.793 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.793 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.794 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.794 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.794 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-03-08T21:15:40.794 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-03-08T21:15:40.794 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:40.794 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.794 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:40.794 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:40.794 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.794 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.794 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.794 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.794 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.794 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.795 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.795 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.795 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-03-08T21:15:40.795 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-03-08T21:15:40.795 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:40.795 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.795 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:40.795 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-03-08T21:15:40.795 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.795 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.795 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:40.795 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-08T21:15:40.795 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-08T21:15:40.795 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-08T21:15:40.795 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-08T21:15:40.796 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-08T21:15:40.796 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-08T21:15:40.796 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:40.796 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.796 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.796 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.796 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.796 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.796 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-03-08T21:15:40.796 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-03-08T21:15:40.796 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:40.796 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.796 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:40.796 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:40.797 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.797 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.797 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.797 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.797 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.797 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.797 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.797 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:40.797 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:40.797 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.797 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-08T21:15:40.797 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "localpool", 2024-03-08T21:15:40.797 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-03-08T21:15:40.797 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-03-08T21:15:40.797 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-03-08T21:15:40.798 INFO:teuthology.orchestra.run.smithi107.stdout: "failure_domain": { 2024-03-08T21:15:40.798 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "failure_domain", 2024-03-08T21:15:40.798 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:40.798 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.798 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:40.798 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "host", 2024-03-08T21:15:40.798 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.798 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.798 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.798 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "failure domain for any created local pool", 2024-03-08T21:15:40.798 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-03-08T21:15:40.798 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.798 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.799 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.799 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-03-08T21:15:40.799 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-03-08T21:15:40.799 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:40.799 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.799 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:40.799 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:40.799 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.799 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.799 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:40.799 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-08T21:15:40.799 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-08T21:15:40.799 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-08T21:15:40.799 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-08T21:15:40.800 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-08T21:15:40.800 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-08T21:15:40.800 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:40.800 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.800 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.800 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.800 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.800 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.800 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-03-08T21:15:40.800 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-03-08T21:15:40.800 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:40.800 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.800 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:40.800 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:40.800 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.801 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.801 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.801 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.801 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.801 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.801 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.801 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.801 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-03-08T21:15:40.801 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-03-08T21:15:40.801 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:40.801 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.801 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:40.801 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-03-08T21:15:40.801 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.802 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.802 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:40.802 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-08T21:15:40.802 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-08T21:15:40.802 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-08T21:15:40.802 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-08T21:15:40.802 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-08T21:15:40.802 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-08T21:15:40.802 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:40.802 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.802 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.802 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.802 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.803 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.803 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-03-08T21:15:40.803 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-03-08T21:15:40.803 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:40.803 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.803 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:40.803 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:40.803 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.803 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.803 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.803 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.803 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.803 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.803 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.804 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.804 INFO:teuthology.orchestra.run.smithi107.stdout: "min_size": { 2024-03-08T21:15:40.804 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "min_size", 2024-03-08T21:15:40.804 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-08T21:15:40.804 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.804 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:40.804 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:40.804 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.804 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.804 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.804 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "default min_size for any created local pool", 2024-03-08T21:15:40.804 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-03-08T21:15:40.804 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.804 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.804 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.805 INFO:teuthology.orchestra.run.smithi107.stdout: "num_rep": { 2024-03-08T21:15:40.805 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "num_rep", 2024-03-08T21:15:40.805 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-08T21:15:40.805 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.805 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:40.805 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "3", 2024-03-08T21:15:40.805 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.805 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.805 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.805 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "default replica count for any created local pool", 2024-03-08T21:15:40.805 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.805 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.805 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.805 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.805 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_num": { 2024-03-08T21:15:40.806 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "pg_num", 2024-03-08T21:15:40.806 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-08T21:15:40.806 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.806 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:40.806 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "128", 2024-03-08T21:15:40.806 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.806 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.806 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.806 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "default pg_num for any created local pool", 2024-03-08T21:15:40.806 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.806 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.806 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.806 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.807 INFO:teuthology.orchestra.run.smithi107.stdout: "prefix": { 2024-03-08T21:15:40.807 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "prefix", 2024-03-08T21:15:40.807 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:40.807 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.807 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:40.807 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:40.807 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.807 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.807 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.807 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "name prefix for any created local pool", 2024-03-08T21:15:40.807 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.807 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.807 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.807 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.808 INFO:teuthology.orchestra.run.smithi107.stdout: "subtree": { 2024-03-08T21:15:40.808 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "subtree", 2024-03-08T21:15:40.808 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:40.808 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.808 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:40.808 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "rack", 2024-03-08T21:15:40.808 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.808 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.808 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.808 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "CRUSH level for which to create a local pool", 2024-03-08T21:15:40.808 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-03-08T21:15:40.808 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.808 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.808 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:40.809 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:40.809 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.809 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-08T21:15:40.809 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "mds_autoscaler", 2024-03-08T21:15:40.809 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-03-08T21:15:40.809 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-03-08T21:15:40.809 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-03-08T21:15:40.809 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-03-08T21:15:40.809 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-03-08T21:15:40.809 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:40.809 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.809 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:40.809 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:40.809 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.810 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.810 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:40.810 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-08T21:15:40.810 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-08T21:15:40.810 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-08T21:15:40.810 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-08T21:15:40.810 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-08T21:15:40.810 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-08T21:15:40.810 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:40.810 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.810 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.810 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.810 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.810 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.810 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-03-08T21:15:40.810 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-03-08T21:15:40.811 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:40.811 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.811 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:40.811 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:40.811 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.811 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.811 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.811 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.811 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.811 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.811 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.811 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.811 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-03-08T21:15:40.811 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-03-08T21:15:40.811 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:40.812 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.812 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:40.812 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-03-08T21:15:40.812 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.812 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.812 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:40.812 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-08T21:15:40.812 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-08T21:15:40.812 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-08T21:15:40.812 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-08T21:15:40.812 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-08T21:15:40.812 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-08T21:15:40.812 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:40.812 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.812 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.812 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.813 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.813 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.813 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-03-08T21:15:40.813 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-03-08T21:15:40.813 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:40.813 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.813 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:40.813 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:40.813 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.813 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.813 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.813 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.813 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.813 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.813 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.813 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:40.813 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:40.814 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.814 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-08T21:15:40.814 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "mirroring", 2024-03-08T21:15:40.814 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-03-08T21:15:40.814 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-03-08T21:15:40.814 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-03-08T21:15:40.814 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-03-08T21:15:40.814 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-03-08T21:15:40.814 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:40.814 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.814 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:40.814 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:40.814 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.814 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.814 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:40.815 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-08T21:15:40.815 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-08T21:15:40.815 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-08T21:15:40.815 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-08T21:15:40.815 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-08T21:15:40.815 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-08T21:15:40.815 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:40.815 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.815 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.815 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.815 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.815 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.815 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-03-08T21:15:40.815 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-03-08T21:15:40.815 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:40.815 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.816 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:40.816 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:40.816 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.816 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.816 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.816 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.816 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.816 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.816 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.816 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.816 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-03-08T21:15:40.816 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-03-08T21:15:40.816 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:40.816 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.816 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:40.816 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-03-08T21:15:40.817 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.817 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.817 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:40.817 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-08T21:15:40.817 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-08T21:15:40.817 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-08T21:15:40.817 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-08T21:15:40.817 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-08T21:15:40.817 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-08T21:15:40.817 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:40.817 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.817 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.817 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.817 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.817 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.817 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-03-08T21:15:40.818 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-03-08T21:15:40.818 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:40.818 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.818 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:40.818 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:40.818 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.818 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.818 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.818 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.818 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.818 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.818 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.818 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:40.818 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:40.819 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.819 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-08T21:15:40.819 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "nfs", 2024-03-08T21:15:40.819 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-03-08T21:15:40.819 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-03-08T21:15:40.819 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-03-08T21:15:40.819 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-03-08T21:15:40.819 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-03-08T21:15:40.819 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:40.819 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.819 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:40.819 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:40.819 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.819 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.819 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:40.819 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-08T21:15:40.820 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-08T21:15:40.820 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-08T21:15:40.820 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-08T21:15:40.820 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-08T21:15:40.820 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-08T21:15:40.820 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:40.820 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.820 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.820 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.820 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.820 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.820 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-03-08T21:15:40.820 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-03-08T21:15:40.820 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:40.820 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.820 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:40.821 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:40.821 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.821 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.821 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.821 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.821 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.821 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.821 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.821 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.821 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-03-08T21:15:40.821 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-03-08T21:15:40.821 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:40.821 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.821 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:40.821 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-03-08T21:15:40.822 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.822 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.822 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:40.822 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-08T21:15:40.822 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-08T21:15:40.822 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-08T21:15:40.822 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-08T21:15:40.822 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-08T21:15:40.822 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-08T21:15:40.822 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:40.822 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.822 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.822 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.822 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.822 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.822 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-03-08T21:15:40.823 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-03-08T21:15:40.823 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:40.823 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.823 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:40.823 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:40.823 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.823 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.823 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.823 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.823 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.823 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.823 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.823 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:40.823 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:40.823 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.824 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-08T21:15:40.824 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "orchestrator", 2024-03-08T21:15:40.824 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-03-08T21:15:40.824 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-03-08T21:15:40.824 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-03-08T21:15:40.824 INFO:teuthology.orchestra.run.smithi107.stdout: "fail_fs": { 2024-03-08T21:15:40.824 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "fail_fs", 2024-03-08T21:15:40.824 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:40.824 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.824 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:40.824 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:40.824 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.824 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.824 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.824 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-03-08T21:15:40.824 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.824 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.825 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.825 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.825 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-03-08T21:15:40.825 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-03-08T21:15:40.825 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:40.825 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.825 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:40.825 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:40.825 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.825 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.825 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:40.825 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-08T21:15:40.825 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-08T21:15:40.825 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-08T21:15:40.825 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-08T21:15:40.825 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-08T21:15:40.826 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-08T21:15:40.826 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:40.826 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.826 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.826 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.826 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.826 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.826 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-03-08T21:15:40.826 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-03-08T21:15:40.826 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:40.826 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.826 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:40.826 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:40.826 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.826 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.826 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.827 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.827 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.827 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.827 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.827 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.827 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-03-08T21:15:40.827 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-03-08T21:15:40.827 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:40.827 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.827 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:40.827 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-03-08T21:15:40.827 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.827 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.827 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:40.827 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-08T21:15:40.827 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-08T21:15:40.828 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-08T21:15:40.828 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-08T21:15:40.828 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-08T21:15:40.828 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-08T21:15:40.828 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:40.828 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.828 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.828 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.828 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.828 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.828 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-03-08T21:15:40.828 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-03-08T21:15:40.828 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:40.828 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.828 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:40.828 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:40.829 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.829 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.829 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.829 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.829 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.829 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.829 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.829 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.829 INFO:teuthology.orchestra.run.smithi107.stdout: "orchestrator": { 2024-03-08T21:15:40.829 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "orchestrator", 2024-03-08T21:15:40.829 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:40.829 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.829 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:40.829 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:40.829 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.829 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.829 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:40.830 INFO:teuthology.orchestra.run.smithi107.stdout: "cephadm", 2024-03-08T21:15:40.830 INFO:teuthology.orchestra.run.smithi107.stdout: "rook", 2024-03-08T21:15:40.830 INFO:teuthology.orchestra.run.smithi107.stdout: "test_orchestrator" 2024-03-08T21:15:40.830 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:40.830 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Orchestrator backend", 2024-03-08T21:15:40.830 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.830 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.830 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.830 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:40.830 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:40.830 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.830 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-08T21:15:40.830 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "osd_perf_query", 2024-03-08T21:15:40.830 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-03-08T21:15:40.830 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-03-08T21:15:40.830 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-03-08T21:15:40.831 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-03-08T21:15:40.831 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-03-08T21:15:40.831 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:40.831 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.831 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:40.831 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:40.831 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.831 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.831 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:40.831 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-08T21:15:40.831 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-08T21:15:40.831 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-08T21:15:40.831 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-08T21:15:40.831 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-08T21:15:40.831 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-08T21:15:40.831 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:40.832 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.832 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.832 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.832 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.832 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.832 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-03-08T21:15:40.832 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-03-08T21:15:40.832 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:40.832 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.832 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:40.832 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:40.832 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.832 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.832 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.832 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.833 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.833 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.833 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.833 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.833 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-03-08T21:15:40.833 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-03-08T21:15:40.833 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:40.833 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.833 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:40.833 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-03-08T21:15:40.833 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.833 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.833 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:40.833 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-08T21:15:40.833 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-08T21:15:40.833 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-08T21:15:40.834 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-08T21:15:40.834 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-08T21:15:40.834 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-08T21:15:40.834 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:40.834 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.834 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.834 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.834 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.834 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.834 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-03-08T21:15:40.834 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-03-08T21:15:40.834 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:40.834 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.834 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:40.834 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:40.834 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.835 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.835 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.835 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.835 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.835 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.835 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.835 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:40.835 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:40.835 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.835 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-08T21:15:40.835 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "osd_support", 2024-03-08T21:15:40.835 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-03-08T21:15:40.835 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-03-08T21:15:40.835 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-03-08T21:15:40.835 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-03-08T21:15:40.835 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-03-08T21:15:40.836 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:40.836 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.836 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:40.836 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:40.836 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.836 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.836 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:40.836 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-08T21:15:40.836 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-08T21:15:40.836 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-08T21:15:40.836 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-08T21:15:40.836 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-08T21:15:40.836 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-08T21:15:40.836 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:40.836 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.836 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.837 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.837 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.837 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.837 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-03-08T21:15:40.837 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-03-08T21:15:40.837 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:40.837 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.837 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:40.837 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:40.837 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.837 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.837 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.837 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.837 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.837 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.838 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.838 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.838 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-03-08T21:15:40.838 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-03-08T21:15:40.838 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:40.838 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.838 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:40.838 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-03-08T21:15:40.838 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.838 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.838 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:40.838 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-08T21:15:40.838 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-08T21:15:40.838 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-08T21:15:40.838 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-08T21:15:40.838 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-08T21:15:40.838 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-08T21:15:40.839 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:40.839 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.839 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.839 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.839 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.839 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.839 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-03-08T21:15:40.839 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-03-08T21:15:40.839 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:40.839 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.839 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:40.839 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:40.839 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.839 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.839 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.839 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.840 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.840 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.840 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.840 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:40.840 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:40.840 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.840 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-08T21:15:40.840 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "pg_autoscaler", 2024-03-08T21:15:40.840 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-03-08T21:15:40.840 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-03-08T21:15:40.840 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-03-08T21:15:40.840 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-03-08T21:15:40.840 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-03-08T21:15:40.840 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:40.840 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.840 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:40.841 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:40.841 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.841 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.841 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:40.841 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-08T21:15:40.841 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-08T21:15:40.841 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-08T21:15:40.841 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-08T21:15:40.841 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-08T21:15:40.841 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-08T21:15:40.841 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:40.841 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.841 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.841 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.841 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.841 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.841 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-03-08T21:15:40.842 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-03-08T21:15:40.842 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:40.842 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.842 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:40.842 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:40.842 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.842 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.842 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.842 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.842 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.842 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.842 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.842 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.842 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-03-08T21:15:40.842 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-03-08T21:15:40.842 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:40.843 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.843 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:40.843 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-03-08T21:15:40.843 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.843 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.843 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:40.843 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-08T21:15:40.843 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-08T21:15:40.843 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-08T21:15:40.843 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-08T21:15:40.843 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-08T21:15:40.843 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-08T21:15:40.843 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:40.843 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.843 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.844 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.844 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.844 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.844 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-03-08T21:15:40.844 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-03-08T21:15:40.844 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:40.844 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.844 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:40.844 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:40.844 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.844 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.844 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.844 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.844 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.844 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.844 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.845 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.845 INFO:teuthology.orchestra.run.smithi107.stdout: "sleep_interval": { 2024-03-08T21:15:40.845 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "sleep_interval", 2024-03-08T21:15:40.845 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-03-08T21:15:40.845 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.845 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:40.845 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "60", 2024-03-08T21:15:40.845 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.845 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.845 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.845 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.845 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.845 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.845 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.845 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.845 INFO:teuthology.orchestra.run.smithi107.stdout: "threshold": { 2024-03-08T21:15:40.846 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "threshold", 2024-03-08T21:15:40.846 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "float", 2024-03-08T21:15:40.846 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.846 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:40.846 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "3.0", 2024-03-08T21:15:40.846 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "1.0", 2024-03-08T21:15:40.846 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.846 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.846 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "scaling threshold", 2024-03-08T21:15:40.846 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-03-08T21:15:40.846 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.846 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.846 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:40.846 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:40.846 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.847 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-08T21:15:40.847 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "progress", 2024-03-08T21:15:40.847 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-03-08T21:15:40.847 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-03-08T21:15:40.847 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-03-08T21:15:40.847 INFO:teuthology.orchestra.run.smithi107.stdout: "allow_pg_recovery_event": { 2024-03-08T21:15:40.847 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "allow_pg_recovery_event", 2024-03-08T21:15:40.847 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:40.847 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.847 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:40.847 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:40.847 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.847 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.847 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.847 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "allow the module to show pg recovery progress", 2024-03-08T21:15:40.847 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.848 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.848 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.848 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.848 INFO:teuthology.orchestra.run.smithi107.stdout: "enabled": { 2024-03-08T21:15:40.848 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "enabled", 2024-03-08T21:15:40.848 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:40.848 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.848 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:40.848 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-08T21:15:40.848 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.848 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.848 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.848 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.848 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.848 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.848 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.849 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.849 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-03-08T21:15:40.849 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-03-08T21:15:40.849 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:40.849 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.849 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:40.849 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:40.849 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.849 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.849 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:40.849 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-08T21:15:40.849 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-08T21:15:40.849 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-08T21:15:40.849 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-08T21:15:40.849 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-08T21:15:40.849 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-08T21:15:40.850 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:40.850 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.850 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.850 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.850 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.850 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.850 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-03-08T21:15:40.850 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-03-08T21:15:40.850 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:40.850 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.850 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:40.850 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:40.850 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.850 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.850 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.851 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.851 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.851 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.851 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.851 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.851 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-03-08T21:15:40.851 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-03-08T21:15:40.851 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:40.851 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.851 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:40.851 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-03-08T21:15:40.851 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.851 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.851 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:40.851 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-08T21:15:40.851 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-08T21:15:40.851 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-08T21:15:40.852 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-08T21:15:40.852 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-08T21:15:40.852 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-08T21:15:40.852 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:40.852 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.852 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.852 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.852 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.852 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.852 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-03-08T21:15:40.852 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-03-08T21:15:40.852 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:40.852 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.852 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:40.852 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:40.852 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.853 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.853 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.853 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.853 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.853 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.853 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.853 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.853 INFO:teuthology.orchestra.run.smithi107.stdout: "max_completed_events": { 2024-03-08T21:15:40.853 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "max_completed_events", 2024-03-08T21:15:40.853 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-08T21:15:40.853 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.853 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:40.853 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "50", 2024-03-08T21:15:40.853 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.853 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.853 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.853 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "number of past completed events to remember", 2024-03-08T21:15:40.854 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.854 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.854 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.854 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.854 INFO:teuthology.orchestra.run.smithi107.stdout: "sleep_interval": { 2024-03-08T21:15:40.854 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "sleep_interval", 2024-03-08T21:15:40.854 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-03-08T21:15:40.854 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.854 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:40.854 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "5", 2024-03-08T21:15:40.854 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.854 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.854 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.854 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "how long the module is going to sleep", 2024-03-08T21:15:40.854 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.854 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.855 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.855 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:40.855 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:40.855 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.855 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-08T21:15:40.855 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "prometheus", 2024-03-08T21:15:40.855 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-03-08T21:15:40.855 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-03-08T21:15:40.855 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-03-08T21:15:40.855 INFO:teuthology.orchestra.run.smithi107.stdout: "cache": { 2024-03-08T21:15:40.855 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "cache", 2024-03-08T21:15:40.855 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:40.855 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.855 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:40.855 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-08T21:15:40.856 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.856 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.856 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.856 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.856 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.856 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.856 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.856 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.856 INFO:teuthology.orchestra.run.smithi107.stdout: "exclude_perf_counters": { 2024-03-08T21:15:40.856 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "exclude_perf_counters", 2024-03-08T21:15:40.856 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:40.856 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.856 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:40.856 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-08T21:15:40.856 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.856 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.856 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.857 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-03-08T21:15:40.857 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-03-08T21:15:40.857 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.857 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.857 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.857 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-03-08T21:15:40.857 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-03-08T21:15:40.857 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:40.857 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.857 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:40.857 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:40.857 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.857 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.857 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:40.857 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-08T21:15:40.858 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-08T21:15:40.858 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-08T21:15:40.858 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-08T21:15:40.858 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-08T21:15:40.858 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-08T21:15:40.858 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:40.858 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.858 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.858 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.858 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.858 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.858 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-03-08T21:15:40.858 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-03-08T21:15:40.858 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:40.858 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.858 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:40.859 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:40.859 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.859 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.859 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.859 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.859 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.859 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.859 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.859 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.859 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-03-08T21:15:40.859 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-03-08T21:15:40.859 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:40.859 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.859 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:40.859 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-03-08T21:15:40.859 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.860 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.860 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:40.860 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-08T21:15:40.860 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-08T21:15:40.860 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-08T21:15:40.860 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-08T21:15:40.860 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-08T21:15:40.860 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-08T21:15:40.860 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:40.860 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.860 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.860 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.860 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.860 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.860 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-03-08T21:15:40.860 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-03-08T21:15:40.861 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:40.861 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.861 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:40.861 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:40.861 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.861 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.861 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.861 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.861 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.861 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.861 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.861 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.861 INFO:teuthology.orchestra.run.smithi107.stdout: "rbd_stats_pools": { 2024-03-08T21:15:40.861 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rbd_stats_pools", 2024-03-08T21:15:40.861 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:40.861 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.862 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:40.862 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:40.862 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.862 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.862 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.862 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.862 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.862 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.862 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.862 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.862 INFO:teuthology.orchestra.run.smithi107.stdout: "rbd_stats_pools_refresh_interval": { 2024-03-08T21:15:40.862 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-03-08T21:15:40.862 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-08T21:15:40.862 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.862 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:40.863 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "300", 2024-03-08T21:15:40.863 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.863 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.863 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.863 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.863 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.863 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.863 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.863 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.863 INFO:teuthology.orchestra.run.smithi107.stdout: "scrape_interval": { 2024-03-08T21:15:40.863 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "scrape_interval", 2024-03-08T21:15:40.863 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "float", 2024-03-08T21:15:40.863 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.863 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:40.863 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "15.0", 2024-03-08T21:15:40.863 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.864 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.864 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.864 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.864 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.864 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.864 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.864 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.864 INFO:teuthology.orchestra.run.smithi107.stdout: "server_addr": { 2024-03-08T21:15:40.864 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "server_addr", 2024-03-08T21:15:40.864 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:40.864 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.864 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:40.864 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "::", 2024-03-08T21:15:40.864 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.864 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.864 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.864 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-03-08T21:15:40.865 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.865 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.865 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.865 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.865 INFO:teuthology.orchestra.run.smithi107.stdout: "server_port": { 2024-03-08T21:15:40.865 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "server_port", 2024-03-08T21:15:40.865 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-08T21:15:40.865 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.865 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:40.865 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "9283", 2024-03-08T21:15:40.865 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.865 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.865 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.865 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-03-08T21:15:40.865 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.865 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.866 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.866 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.866 INFO:teuthology.orchestra.run.smithi107.stdout: "stale_cache_strategy": { 2024-03-08T21:15:40.866 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "stale_cache_strategy", 2024-03-08T21:15:40.866 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:40.866 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.866 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:40.866 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "log", 2024-03-08T21:15:40.866 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.866 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.866 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.866 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.866 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.866 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.866 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.867 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.867 INFO:teuthology.orchestra.run.smithi107.stdout: "standby_behaviour": { 2024-03-08T21:15:40.867 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "standby_behaviour", 2024-03-08T21:15:40.867 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:40.867 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.867 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:40.867 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "default", 2024-03-08T21:15:40.867 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.867 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.867 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:40.867 INFO:teuthology.orchestra.run.smithi107.stdout: "default", 2024-03-08T21:15:40.867 INFO:teuthology.orchestra.run.smithi107.stdout: "error" 2024-03-08T21:15:40.867 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:40.867 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.867 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.868 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.868 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.868 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.868 INFO:teuthology.orchestra.run.smithi107.stdout: "standby_error_status_code": { 2024-03-08T21:15:40.868 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "standby_error_status_code", 2024-03-08T21:15:40.868 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-08T21:15:40.868 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.868 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:40.868 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "500", 2024-03-08T21:15:40.868 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "400", 2024-03-08T21:15:40.868 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "599", 2024-03-08T21:15:40.868 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.868 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.868 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.868 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.868 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.869 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:40.869 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:40.869 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.869 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-08T21:15:40.869 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rbd_support", 2024-03-08T21:15:40.869 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-03-08T21:15:40.869 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-03-08T21:15:40.869 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-03-08T21:15:40.869 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-03-08T21:15:40.869 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-03-08T21:15:40.869 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:40.869 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.869 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:40.869 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:40.869 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.869 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.870 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:40.870 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-08T21:15:40.870 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-08T21:15:40.870 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-08T21:15:40.870 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-08T21:15:40.870 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-08T21:15:40.870 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-08T21:15:40.870 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:40.870 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.870 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.870 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.870 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.870 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.870 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-03-08T21:15:40.870 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-03-08T21:15:40.871 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:40.871 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.871 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:40.871 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:40.871 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.871 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.871 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.871 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.871 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.871 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.871 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.871 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.871 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-03-08T21:15:40.871 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-03-08T21:15:40.871 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:40.871 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.872 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:40.872 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-03-08T21:15:40.872 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.872 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.872 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:40.872 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-08T21:15:40.872 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-08T21:15:40.872 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-08T21:15:40.872 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-08T21:15:40.872 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-08T21:15:40.872 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-08T21:15:40.872 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:40.872 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.872 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.872 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.872 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.873 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.873 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-03-08T21:15:40.873 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-03-08T21:15:40.873 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:40.873 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.873 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:40.873 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:40.873 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.873 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.873 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.873 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.873 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.873 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.873 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.873 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.874 INFO:teuthology.orchestra.run.smithi107.stdout: "max_concurrent_snap_create": { 2024-03-08T21:15:40.874 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "max_concurrent_snap_create", 2024-03-08T21:15:40.874 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-08T21:15:40.874 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.874 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:40.874 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "10", 2024-03-08T21:15:40.874 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.874 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.874 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.874 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.874 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.874 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.874 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.874 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.875 INFO:teuthology.orchestra.run.smithi107.stdout: "mirror_snapshot_schedule": { 2024-03-08T21:15:40.875 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "mirror_snapshot_schedule", 2024-03-08T21:15:40.875 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:40.875 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.875 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:40.875 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:40.875 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.875 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.875 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.875 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.875 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.875 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.875 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.875 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.875 INFO:teuthology.orchestra.run.smithi107.stdout: "trash_purge_schedule": { 2024-03-08T21:15:40.875 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "trash_purge_schedule", 2024-03-08T21:15:40.876 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:40.876 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.876 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:40.876 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:40.876 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.876 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.876 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.876 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.876 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.876 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.876 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.876 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:40.876 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:40.876 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.876 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-08T21:15:40.876 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "restful", 2024-03-08T21:15:40.877 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-03-08T21:15:40.877 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-03-08T21:15:40.877 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-03-08T21:15:40.877 INFO:teuthology.orchestra.run.smithi107.stdout: "enable_auth": { 2024-03-08T21:15:40.877 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "enable_auth", 2024-03-08T21:15:40.877 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:40.877 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.877 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:40.877 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-08T21:15:40.877 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.877 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.877 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.877 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.877 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.877 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.877 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.877 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.878 INFO:teuthology.orchestra.run.smithi107.stdout: "key_file": { 2024-03-08T21:15:40.878 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "key_file", 2024-03-08T21:15:40.878 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:40.878 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.878 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:40.878 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:40.878 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.878 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.878 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.878 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.878 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.878 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.878 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.878 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.878 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-03-08T21:15:40.878 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-03-08T21:15:40.879 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:40.879 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.879 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:40.879 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:40.879 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.879 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.879 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:40.879 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-08T21:15:40.879 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-08T21:15:40.879 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-08T21:15:40.879 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-08T21:15:40.879 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-08T21:15:40.879 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-08T21:15:40.880 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:40.880 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.880 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.880 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.880 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.880 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.880 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-03-08T21:15:40.880 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-03-08T21:15:40.880 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:40.880 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.880 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:40.880 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:40.880 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.880 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.880 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.881 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.881 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.881 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.881 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.881 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.881 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-03-08T21:15:40.881 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-03-08T21:15:40.881 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:40.881 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.881 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:40.881 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-03-08T21:15:40.881 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.881 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.881 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:40.881 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-08T21:15:40.881 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-08T21:15:40.882 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-08T21:15:40.882 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-08T21:15:40.882 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-08T21:15:40.882 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-08T21:15:40.882 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:40.882 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.882 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.882 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.882 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.882 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.882 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-03-08T21:15:40.882 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-03-08T21:15:40.882 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:40.882 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.882 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:40.883 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:40.883 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.883 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.883 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.883 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.883 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.883 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.883 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.883 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.883 INFO:teuthology.orchestra.run.smithi107.stdout: "server_addr": { 2024-03-08T21:15:40.883 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "server_addr", 2024-03-08T21:15:40.883 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:40.883 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.883 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:40.883 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:40.883 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.884 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.884 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.884 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.884 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.884 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.884 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.884 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.884 INFO:teuthology.orchestra.run.smithi107.stdout: "server_port": { 2024-03-08T21:15:40.884 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "server_port", 2024-03-08T21:15:40.884 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:40.884 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.884 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:40.884 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:40.884 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.884 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.884 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.885 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.885 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.885 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.885 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.885 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:40.885 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:40.885 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.885 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-08T21:15:40.885 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rgw", 2024-03-08T21:15:40.885 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-03-08T21:15:40.885 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-03-08T21:15:40.885 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-03-08T21:15:40.885 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-03-08T21:15:40.885 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-03-08T21:15:40.885 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:40.885 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.886 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:40.886 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:40.886 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.886 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.886 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:40.886 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-08T21:15:40.886 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-08T21:15:40.886 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-08T21:15:40.886 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-08T21:15:40.886 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-08T21:15:40.886 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-08T21:15:40.886 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:40.886 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.886 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.886 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.886 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.887 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.887 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-03-08T21:15:40.887 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-03-08T21:15:40.887 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:40.887 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.887 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:40.887 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:40.887 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.887 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.887 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.887 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.887 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.887 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.887 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.887 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.887 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-03-08T21:15:40.888 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-03-08T21:15:40.888 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:40.888 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.888 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:40.888 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-03-08T21:15:40.888 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.888 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.888 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:40.888 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-08T21:15:40.888 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-08T21:15:40.888 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-08T21:15:40.888 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-08T21:15:40.888 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-08T21:15:40.888 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-08T21:15:40.888 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:40.888 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.889 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.889 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.889 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.889 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.889 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-03-08T21:15:40.889 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-03-08T21:15:40.889 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:40.889 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.889 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:40.889 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:40.889 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.889 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.889 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.889 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.889 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.890 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.890 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.890 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:40.890 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:40.890 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.890 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-08T21:15:40.890 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rook", 2024-03-08T21:15:40.890 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-03-08T21:15:40.890 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-03-08T21:15:40.890 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-03-08T21:15:40.890 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-03-08T21:15:40.890 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-03-08T21:15:40.890 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:40.890 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.890 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:40.891 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:40.891 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.891 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.891 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:40.891 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-08T21:15:40.891 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-08T21:15:40.891 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-08T21:15:40.891 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-08T21:15:40.891 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-08T21:15:40.891 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-08T21:15:40.891 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:40.891 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.891 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.891 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.891 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.892 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.892 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-03-08T21:15:40.892 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-03-08T21:15:40.892 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:40.892 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.892 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:40.892 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:40.892 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.892 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.892 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.892 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.892 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.892 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.892 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.892 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.892 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-03-08T21:15:40.893 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-03-08T21:15:40.893 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:40.893 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.893 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:40.893 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-03-08T21:15:40.893 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.893 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.893 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:40.893 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-08T21:15:40.893 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-08T21:15:40.893 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-08T21:15:40.893 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-08T21:15:40.893 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-08T21:15:40.893 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-08T21:15:40.893 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:40.893 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.894 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.894 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.894 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.894 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.894 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-03-08T21:15:40.894 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-03-08T21:15:40.894 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:40.894 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.894 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:40.894 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:40.894 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.894 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.894 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.894 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.894 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.895 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.895 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.895 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.895 INFO:teuthology.orchestra.run.smithi107.stdout: "storage_class": { 2024-03-08T21:15:40.895 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "storage_class", 2024-03-08T21:15:40.895 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:40.895 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.895 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:40.895 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "local", 2024-03-08T21:15:40.895 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.895 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.895 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.895 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-03-08T21:15:40.895 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.895 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.895 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.896 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:40.896 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:40.896 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.896 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-08T21:15:40.896 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "selftest", 2024-03-08T21:15:40.896 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-03-08T21:15:40.896 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-03-08T21:15:40.896 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-03-08T21:15:40.896 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-03-08T21:15:40.896 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-03-08T21:15:40.896 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:40.896 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.896 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:40.896 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:40.896 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.896 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.897 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:40.897 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-08T21:15:40.897 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-08T21:15:40.897 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-08T21:15:40.897 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-08T21:15:40.897 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-08T21:15:40.897 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-08T21:15:40.897 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:40.897 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.897 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.897 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.897 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.897 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.897 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-03-08T21:15:40.897 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-03-08T21:15:40.897 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:40.898 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.898 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:40.898 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:40.898 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.898 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.898 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.898 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.898 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.898 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.898 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.898 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.898 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-03-08T21:15:40.898 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-03-08T21:15:40.898 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:40.898 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.899 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:40.899 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-03-08T21:15:40.899 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.899 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.899 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:40.899 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-08T21:15:40.899 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-08T21:15:40.899 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-08T21:15:40.899 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-08T21:15:40.899 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-08T21:15:40.899 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-08T21:15:40.899 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:40.899 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.899 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.899 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.899 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.900 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.900 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-03-08T21:15:40.900 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-03-08T21:15:40.900 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:40.900 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.900 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:40.900 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:40.900 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.900 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.900 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.900 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.900 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.900 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.900 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.900 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.901 INFO:teuthology.orchestra.run.smithi107.stdout: "roption1": { 2024-03-08T21:15:40.901 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "roption1", 2024-03-08T21:15:40.901 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:40.901 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.901 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:40.901 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:40.901 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.901 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.901 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.901 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.901 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.901 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.901 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.901 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.901 INFO:teuthology.orchestra.run.smithi107.stdout: "roption2": { 2024-03-08T21:15:40.902 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "roption2", 2024-03-08T21:15:40.902 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:40.902 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.902 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:40.902 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "xyz", 2024-03-08T21:15:40.902 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.902 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.902 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.902 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.902 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.902 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.902 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.902 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.902 INFO:teuthology.orchestra.run.smithi107.stdout: "rwoption1": { 2024-03-08T21:15:40.902 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rwoption1", 2024-03-08T21:15:40.902 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:40.903 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.903 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:40.903 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:40.903 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.903 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.903 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.903 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.903 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.903 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.903 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.903 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.903 INFO:teuthology.orchestra.run.smithi107.stdout: "rwoption2": { 2024-03-08T21:15:40.903 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rwoption2", 2024-03-08T21:15:40.903 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-08T21:15:40.903 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.903 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:40.904 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:40.904 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.904 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.904 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.904 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.904 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.904 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.904 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.904 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.904 INFO:teuthology.orchestra.run.smithi107.stdout: "rwoption3": { 2024-03-08T21:15:40.904 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rwoption3", 2024-03-08T21:15:40.904 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "float", 2024-03-08T21:15:40.904 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.904 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:40.905 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:40.905 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.905 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.905 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.905 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.905 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.905 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.905 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.905 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.905 INFO:teuthology.orchestra.run.smithi107.stdout: "rwoption4": { 2024-03-08T21:15:40.905 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rwoption4", 2024-03-08T21:15:40.905 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:40.905 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.905 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:40.906 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:40.906 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.906 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.906 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.906 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.906 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.906 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.906 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.906 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.906 INFO:teuthology.orchestra.run.smithi107.stdout: "rwoption5": { 2024-03-08T21:15:40.906 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rwoption5", 2024-03-08T21:15:40.906 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:40.906 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.906 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:40.907 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:40.907 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.907 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.907 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.907 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.907 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.907 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.907 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.907 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.907 INFO:teuthology.orchestra.run.smithi107.stdout: "rwoption6": { 2024-03-08T21:15:40.907 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rwoption6", 2024-03-08T21:15:40.907 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:40.907 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.907 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:40.908 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-08T21:15:40.908 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.908 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.908 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.908 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.908 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.908 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.908 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.908 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.908 INFO:teuthology.orchestra.run.smithi107.stdout: "rwoption7": { 2024-03-08T21:15:40.908 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rwoption7", 2024-03-08T21:15:40.908 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-08T21:15:40.908 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.908 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:40.909 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:40.909 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "1", 2024-03-08T21:15:40.909 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "42", 2024-03-08T21:15:40.909 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.909 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.909 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.909 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.909 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.909 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.909 INFO:teuthology.orchestra.run.smithi107.stdout: "testkey": { 2024-03-08T21:15:40.909 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "testkey", 2024-03-08T21:15:40.909 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:40.909 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.909 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:40.909 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:40.910 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.910 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.910 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.910 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.910 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.910 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.910 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.910 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.910 INFO:teuthology.orchestra.run.smithi107.stdout: "testlkey": { 2024-03-08T21:15:40.910 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "testlkey", 2024-03-08T21:15:40.910 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:40.910 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.910 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:40.910 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:40.910 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.911 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.911 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.911 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.911 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.911 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.911 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.911 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.911 INFO:teuthology.orchestra.run.smithi107.stdout: "testnewline": { 2024-03-08T21:15:40.911 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "testnewline", 2024-03-08T21:15:40.911 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:40.911 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.911 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:40.911 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:40.911 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.911 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.911 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.912 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.912 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.912 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.912 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.912 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:40.912 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:40.912 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.912 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-08T21:15:40.912 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "snap_schedule", 2024-03-08T21:15:40.912 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-03-08T21:15:40.912 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-03-08T21:15:40.912 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-03-08T21:15:40.912 INFO:teuthology.orchestra.run.smithi107.stdout: "allow_m_granularity": { 2024-03-08T21:15:40.912 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "allow_m_granularity", 2024-03-08T21:15:40.912 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:40.912 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.913 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:40.913 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:40.913 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.913 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.913 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.913 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "allow minute scheduled snapshots", 2024-03-08T21:15:40.913 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.913 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.913 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.913 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.913 INFO:teuthology.orchestra.run.smithi107.stdout: "dump_on_update": { 2024-03-08T21:15:40.913 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "dump_on_update", 2024-03-08T21:15:40.913 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:40.913 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.913 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:40.913 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:40.914 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.914 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.914 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.914 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "dump database to debug log on update", 2024-03-08T21:15:40.914 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.914 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.914 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.914 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.914 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-03-08T21:15:40.914 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-03-08T21:15:40.914 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:40.914 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.914 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:40.914 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:40.914 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.915 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.915 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:40.915 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-08T21:15:40.915 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-08T21:15:40.915 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-08T21:15:40.915 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-08T21:15:40.915 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-08T21:15:40.915 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-08T21:15:40.915 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:40.915 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.915 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.915 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.915 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.915 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.915 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-03-08T21:15:40.915 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-03-08T21:15:40.916 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:40.916 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.916 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:40.916 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:40.916 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.916 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.916 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.916 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.916 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.916 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.916 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.916 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.916 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-03-08T21:15:40.916 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-03-08T21:15:40.916 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:40.916 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.916 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:40.917 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-03-08T21:15:40.917 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.917 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.917 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:40.917 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-08T21:15:40.917 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-08T21:15:40.917 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-08T21:15:40.917 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-08T21:15:40.917 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-08T21:15:40.917 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-08T21:15:40.917 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:40.917 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.917 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.917 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.917 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.918 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.918 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-03-08T21:15:40.918 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-03-08T21:15:40.918 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:40.918 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.918 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:40.918 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:40.918 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.918 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.918 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.918 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.918 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.918 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.918 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.918 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:40.918 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:40.919 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.919 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-08T21:15:40.919 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "stats", 2024-03-08T21:15:40.919 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-03-08T21:15:40.919 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-03-08T21:15:40.919 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-03-08T21:15:40.919 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-03-08T21:15:40.919 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-03-08T21:15:40.919 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:40.919 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.919 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:40.919 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:40.919 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.919 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.919 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:40.919 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-08T21:15:40.920 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-08T21:15:40.920 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-08T21:15:40.920 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-08T21:15:40.920 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-08T21:15:40.920 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-08T21:15:40.920 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:40.920 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.920 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.920 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.920 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.920 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.920 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-03-08T21:15:40.920 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-03-08T21:15:40.920 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:40.920 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.920 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:40.921 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:40.921 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.921 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.921 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.921 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.921 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.921 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.921 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.921 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.921 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-03-08T21:15:40.921 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-03-08T21:15:40.921 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:40.921 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.921 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:40.921 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-03-08T21:15:40.921 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.922 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.922 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:40.922 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-08T21:15:40.922 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-08T21:15:40.922 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-08T21:15:40.922 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-08T21:15:40.922 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-08T21:15:40.922 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-08T21:15:40.922 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:40.922 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.922 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.922 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.922 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.922 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.922 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-03-08T21:15:40.923 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-03-08T21:15:40.923 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:40.923 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.923 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:40.923 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:40.923 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.923 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.923 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.923 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.923 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.923 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.923 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.923 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:40.923 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:40.923 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.923 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-08T21:15:40.924 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "status", 2024-03-08T21:15:40.924 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-03-08T21:15:40.924 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-03-08T21:15:40.924 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-03-08T21:15:40.924 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-03-08T21:15:40.924 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-03-08T21:15:40.924 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:40.924 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.924 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:40.924 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:40.924 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.924 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.924 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:40.924 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-08T21:15:40.924 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-08T21:15:40.924 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-08T21:15:40.925 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-08T21:15:40.925 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-08T21:15:40.925 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-08T21:15:40.925 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:40.925 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.925 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.925 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.925 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.925 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.925 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-03-08T21:15:40.925 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-03-08T21:15:40.925 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:40.925 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.925 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:40.925 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:40.925 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.926 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.926 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.926 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.926 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.926 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.926 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.926 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.926 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-03-08T21:15:40.926 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-03-08T21:15:40.926 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:40.926 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.926 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:40.926 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-03-08T21:15:40.926 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.926 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.927 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:40.927 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-08T21:15:40.927 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-08T21:15:40.927 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-08T21:15:40.927 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-08T21:15:40.927 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-08T21:15:40.927 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-08T21:15:40.927 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:40.927 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.927 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.927 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.927 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.927 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.927 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-03-08T21:15:40.927 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-03-08T21:15:40.928 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:40.928 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.928 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:40.928 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:40.928 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.928 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.928 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.928 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.928 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.928 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.928 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.928 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:40.928 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:40.928 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.928 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-08T21:15:40.928 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "telegraf", 2024-03-08T21:15:40.929 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-03-08T21:15:40.929 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-03-08T21:15:40.929 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-03-08T21:15:40.929 INFO:teuthology.orchestra.run.smithi107.stdout: "address": { 2024-03-08T21:15:40.929 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "address", 2024-03-08T21:15:40.929 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:40.929 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.929 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:40.929 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-03-08T21:15:40.929 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.929 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.929 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.929 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.929 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.929 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.929 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.930 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.930 INFO:teuthology.orchestra.run.smithi107.stdout: "interval": { 2024-03-08T21:15:40.930 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "interval", 2024-03-08T21:15:40.930 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-03-08T21:15:40.930 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.930 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:40.930 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "15", 2024-03-08T21:15:40.930 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.930 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.930 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.930 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.930 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.930 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.930 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.930 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.931 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-03-08T21:15:40.931 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-03-08T21:15:40.931 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:40.931 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.931 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:40.931 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:40.931 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.931 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.931 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:40.931 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-08T21:15:40.931 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-08T21:15:40.931 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-08T21:15:40.931 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-08T21:15:40.931 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-08T21:15:40.931 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-08T21:15:40.931 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:40.932 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.932 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.932 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.932 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.932 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.932 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-03-08T21:15:40.932 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-03-08T21:15:40.932 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:40.932 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.932 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:40.932 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:40.932 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.932 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.932 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.932 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.932 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.933 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.933 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.933 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.933 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-03-08T21:15:40.933 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-03-08T21:15:40.933 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:40.933 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.933 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:40.933 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-03-08T21:15:40.933 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.933 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.933 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:40.933 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-08T21:15:40.933 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-08T21:15:40.934 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-08T21:15:40.934 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-08T21:15:40.934 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-08T21:15:40.934 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-08T21:15:40.934 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:40.934 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.934 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.934 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.934 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.934 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.934 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-03-08T21:15:40.934 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-03-08T21:15:40.934 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:40.934 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.934 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:40.935 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:40.935 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.935 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.935 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.935 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.935 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.935 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.935 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.935 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:40.935 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:40.935 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.935 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-08T21:15:40.935 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "telemetry", 2024-03-08T21:15:40.935 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-03-08T21:15:40.935 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-03-08T21:15:40.935 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-03-08T21:15:40.936 INFO:teuthology.orchestra.run.smithi107.stdout: "channel_basic": { 2024-03-08T21:15:40.936 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "channel_basic", 2024-03-08T21:15:40.936 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:40.936 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.936 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:40.936 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-08T21:15:40.936 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.936 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.936 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.936 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Share basic cluster information (size, version)", 2024-03-08T21:15:40.936 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.936 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.936 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.936 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.936 INFO:teuthology.orchestra.run.smithi107.stdout: "channel_crash": { 2024-03-08T21:15:40.936 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "channel_crash", 2024-03-08T21:15:40.937 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:40.937 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.937 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:40.937 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-08T21:15:40.937 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.937 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.937 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.937 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-03-08T21:15:40.937 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.937 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.937 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.937 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.937 INFO:teuthology.orchestra.run.smithi107.stdout: "channel_device": { 2024-03-08T21:15:40.937 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "channel_device", 2024-03-08T21:15:40.937 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:40.937 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.938 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:40.938 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-08T21:15:40.938 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.938 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.938 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.938 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-03-08T21:15:40.938 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.938 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.938 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.938 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.938 INFO:teuthology.orchestra.run.smithi107.stdout: "channel_ident": { 2024-03-08T21:15:40.938 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "channel_ident", 2024-03-08T21:15:40.938 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:40.938 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.938 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:40.938 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:40.939 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.939 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.939 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.939 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-03-08T21:15:40.939 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.939 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.939 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.939 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.939 INFO:teuthology.orchestra.run.smithi107.stdout: "channel_perf": { 2024-03-08T21:15:40.939 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "channel_perf", 2024-03-08T21:15:40.939 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:40.939 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.939 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:40.939 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:40.939 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.939 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.940 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.940 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Share various performance metrics of a cluster", 2024-03-08T21:15:40.940 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.940 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.940 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.940 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.940 INFO:teuthology.orchestra.run.smithi107.stdout: "contact": { 2024-03-08T21:15:40.940 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "contact", 2024-03-08T21:15:40.940 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:40.940 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.940 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:40.940 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:40.940 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.940 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.940 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.940 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.941 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.941 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.941 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.941 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.941 INFO:teuthology.orchestra.run.smithi107.stdout: "description": { 2024-03-08T21:15:40.941 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "description", 2024-03-08T21:15:40.941 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:40.941 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.941 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:40.941 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:40.941 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.941 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.941 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.941 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.941 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.941 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.942 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.942 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.942 INFO:teuthology.orchestra.run.smithi107.stdout: "device_url": { 2024-03-08T21:15:40.942 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "device_url", 2024-03-08T21:15:40.942 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:40.942 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.942 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:40.942 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-03-08T21:15:40.942 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.942 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.942 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.942 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.942 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.942 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.946 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.946 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.946 INFO:teuthology.orchestra.run.smithi107.stdout: "enabled": { 2024-03-08T21:15:40.946 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "enabled", 2024-03-08T21:15:40.946 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:40.946 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.946 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:40.947 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:40.947 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.947 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.947 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.947 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.947 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.947 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.947 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.947 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.947 INFO:teuthology.orchestra.run.smithi107.stdout: "interval": { 2024-03-08T21:15:40.947 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "interval", 2024-03-08T21:15:40.947 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-08T21:15:40.947 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.947 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:40.947 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "24", 2024-03-08T21:15:40.948 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "8", 2024-03-08T21:15:40.948 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.948 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.948 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.948 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.948 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.948 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.948 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.948 INFO:teuthology.orchestra.run.smithi107.stdout: "last_opt_revision": { 2024-03-08T21:15:40.948 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "last_opt_revision", 2024-03-08T21:15:40.948 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-08T21:15:40.948 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.948 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:40.948 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "1", 2024-03-08T21:15:40.948 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.948 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.949 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.949 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.949 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.949 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.949 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.949 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.949 INFO:teuthology.orchestra.run.smithi107.stdout: "leaderboard": { 2024-03-08T21:15:40.949 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "leaderboard", 2024-03-08T21:15:40.949 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:40.949 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.949 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:40.949 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:40.949 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.949 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.949 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.950 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.950 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.950 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.950 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.950 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.950 INFO:teuthology.orchestra.run.smithi107.stdout: "leaderboard_description": { 2024-03-08T21:15:40.950 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "leaderboard_description", 2024-03-08T21:15:40.950 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:40.950 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.950 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:40.950 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:40.950 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.950 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.950 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.950 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.950 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.951 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.951 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.951 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.951 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-03-08T21:15:40.951 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-03-08T21:15:40.951 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:40.951 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.951 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:40.951 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:40.951 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.951 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.951 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:40.951 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-08T21:15:40.951 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-08T21:15:40.952 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-08T21:15:40.952 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-08T21:15:40.952 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-08T21:15:40.952 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-08T21:15:40.952 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:40.952 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.952 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.952 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.952 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.952 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.952 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-03-08T21:15:40.952 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-03-08T21:15:40.952 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:40.952 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.952 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:40.952 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:40.953 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.953 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.953 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.953 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.953 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.953 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.953 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.953 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.953 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-03-08T21:15:40.953 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-03-08T21:15:40.953 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:40.953 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.953 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:40.953 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-03-08T21:15:40.953 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.953 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.954 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:40.954 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-08T21:15:40.954 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-08T21:15:40.954 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-08T21:15:40.954 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-08T21:15:40.954 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-08T21:15:40.954 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-08T21:15:40.954 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:40.954 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.954 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.954 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.954 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.954 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.954 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-03-08T21:15:40.954 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-03-08T21:15:40.954 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:40.955 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.955 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:40.955 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:40.955 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.955 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.955 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.955 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.955 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.955 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.955 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.955 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.955 INFO:teuthology.orchestra.run.smithi107.stdout: "organization": { 2024-03-08T21:15:40.955 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "organization", 2024-03-08T21:15:40.955 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:40.955 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.955 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:40.956 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:40.956 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.956 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.956 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.956 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.956 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.956 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.956 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.956 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.956 INFO:teuthology.orchestra.run.smithi107.stdout: "proxy": { 2024-03-08T21:15:40.956 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "proxy", 2024-03-08T21:15:40.956 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:40.956 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.956 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:40.956 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:40.956 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.956 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.957 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.957 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.957 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.957 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.957 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.957 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.957 INFO:teuthology.orchestra.run.smithi107.stdout: "url": { 2024-03-08T21:15:40.957 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "url", 2024-03-08T21:15:40.957 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:40.957 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.957 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:40.957 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-03-08T21:15:40.957 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.957 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.957 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.957 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.958 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.958 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.958 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.958 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:40.958 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:40.958 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.958 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-08T21:15:40.958 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "test_orchestrator", 2024-03-08T21:15:40.958 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-03-08T21:15:40.958 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-03-08T21:15:40.958 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-03-08T21:15:40.958 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-03-08T21:15:40.958 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-03-08T21:15:40.959 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:40.959 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.959 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:40.959 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:40.959 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.959 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.959 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:40.959 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-08T21:15:40.959 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-08T21:15:40.959 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-08T21:15:40.959 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-08T21:15:40.959 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-08T21:15:40.959 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-08T21:15:40.959 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:40.959 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.959 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.960 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.960 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.960 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.960 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-03-08T21:15:40.960 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-03-08T21:15:40.960 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:40.960 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.960 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:40.960 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:40.960 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.960 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.960 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.960 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.960 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.960 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.960 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.960 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.961 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-03-08T21:15:40.961 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-03-08T21:15:40.961 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:40.961 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.961 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:40.961 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-03-08T21:15:40.961 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.961 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.961 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:40.961 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-08T21:15:40.961 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-08T21:15:40.961 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-08T21:15:40.961 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-08T21:15:40.961 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-08T21:15:40.961 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-08T21:15:40.962 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:40.962 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.962 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.962 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.962 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.962 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.962 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-03-08T21:15:40.962 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-03-08T21:15:40.962 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:40.962 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.962 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:40.962 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:40.962 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.962 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.962 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.963 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.963 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.963 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.963 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.963 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:40.963 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:40.963 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.963 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-08T21:15:40.963 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "volumes", 2024-03-08T21:15:40.963 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-03-08T21:15:40.963 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-03-08T21:15:40.963 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-03-08T21:15:40.963 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-03-08T21:15:40.963 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-03-08T21:15:40.963 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:40.963 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.964 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:40.964 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:40.964 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.964 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.964 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:40.964 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-08T21:15:40.964 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-08T21:15:40.964 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-08T21:15:40.964 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-08T21:15:40.964 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-08T21:15:40.964 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-08T21:15:40.964 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:40.964 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.964 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.964 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.964 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.965 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.965 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-03-08T21:15:40.965 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-03-08T21:15:40.965 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:40.965 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.965 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:40.965 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:40.965 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.965 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.965 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.965 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.965 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.965 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.965 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.966 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.966 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-03-08T21:15:40.966 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-03-08T21:15:40.966 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:40.966 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.966 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:40.966 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-03-08T21:15:40.966 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.966 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.966 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:40.966 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-08T21:15:40.966 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-08T21:15:40.966 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-08T21:15:40.966 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-08T21:15:40.966 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-08T21:15:40.966 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-08T21:15:40.967 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:40.967 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.967 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.967 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.967 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.967 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.967 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-03-08T21:15:40.967 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-03-08T21:15:40.967 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:40.967 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.967 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:40.967 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:40.967 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.967 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.967 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.967 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.968 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.968 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.968 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.968 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.968 INFO:teuthology.orchestra.run.smithi107.stdout: "max_concurrent_clones": { 2024-03-08T21:15:40.968 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "max_concurrent_clones", 2024-03-08T21:15:40.968 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-08T21:15:40.968 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.968 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:40.968 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "4", 2024-03-08T21:15:40.968 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.968 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.968 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.968 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Number of asynchronous cloner threads", 2024-03-08T21:15:40.968 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.968 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.969 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.969 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.969 INFO:teuthology.orchestra.run.smithi107.stdout: "periodic_async_work": { 2024-03-08T21:15:40.969 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "periodic_async_work", 2024-03-08T21:15:40.969 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:40.969 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.969 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:40.969 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:40.969 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.969 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.969 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.969 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Periodically check for async work", 2024-03-08T21:15:40.969 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.969 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.969 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.969 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.970 INFO:teuthology.orchestra.run.smithi107.stdout: "snapshot_clone_delay": { 2024-03-08T21:15:40.970 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "snapshot_clone_delay", 2024-03-08T21:15:40.970 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-08T21:15:40.970 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.970 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:40.970 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0", 2024-03-08T21:15:40.970 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.970 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.970 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.970 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-03-08T21:15:40.970 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.970 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.970 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.970 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.970 INFO:teuthology.orchestra.run.smithi107.stdout: "snapshot_clone_no_wait": { 2024-03-08T21:15:40.970 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "snapshot_clone_no_wait", 2024-03-08T21:15:40.971 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:40.971 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.971 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:40.971 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-08T21:15:40.971 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.971 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.971 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.971 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-03-08T21:15:40.971 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.971 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.971 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.971 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:40.971 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:40.971 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.971 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-08T21:15:40.972 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "zabbix", 2024-03-08T21:15:40.972 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-03-08T21:15:40.972 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-03-08T21:15:40.972 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-03-08T21:15:40.972 INFO:teuthology.orchestra.run.smithi107.stdout: "discovery_interval": { 2024-03-08T21:15:40.972 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "discovery_interval", 2024-03-08T21:15:40.972 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "uint", 2024-03-08T21:15:40.972 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.972 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:40.972 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "100", 2024-03-08T21:15:40.972 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.972 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.972 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.972 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.972 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.973 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.973 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.973 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.973 INFO:teuthology.orchestra.run.smithi107.stdout: "identifier": { 2024-03-08T21:15:40.973 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "identifier", 2024-03-08T21:15:40.973 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:40.973 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.973 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:40.973 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:40.973 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.973 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.973 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.973 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.973 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.973 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.974 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.974 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.974 INFO:teuthology.orchestra.run.smithi107.stdout: "interval": { 2024-03-08T21:15:40.974 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "interval", 2024-03-08T21:15:40.974 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-03-08T21:15:40.974 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.974 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:40.974 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "60", 2024-03-08T21:15:40.974 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.974 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.974 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.974 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.974 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.974 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.974 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.974 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.975 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-03-08T21:15:40.975 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-03-08T21:15:40.975 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:40.975 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.975 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:40.975 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:40.975 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.975 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.975 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:40.975 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-08T21:15:40.975 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-08T21:15:40.975 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-08T21:15:40.975 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-08T21:15:40.975 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-08T21:15:40.975 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-08T21:15:40.976 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:40.976 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.976 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.976 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.976 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.976 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.976 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-03-08T21:15:40.976 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-03-08T21:15:40.976 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:40.976 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.976 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:40.976 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:40.976 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.976 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.976 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.976 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.977 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.977 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.977 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.977 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.977 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-03-08T21:15:40.977 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-03-08T21:15:40.977 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:40.977 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.977 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:40.977 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-03-08T21:15:40.977 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.977 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.977 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:40.977 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-08T21:15:40.977 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-08T21:15:40.977 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-08T21:15:40.978 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-08T21:15:40.978 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-08T21:15:40.978 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-08T21:15:40.978 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:40.978 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.978 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.978 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.978 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.978 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.978 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-03-08T21:15:40.978 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-03-08T21:15:40.978 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:40.978 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.978 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:40.978 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:40.978 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.979 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.979 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.979 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.979 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.979 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.979 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.979 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.979 INFO:teuthology.orchestra.run.smithi107.stdout: "zabbix_host": { 2024-03-08T21:15:40.979 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "zabbix_host", 2024-03-08T21:15:40.979 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:40.979 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.979 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:40.979 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:40.979 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.979 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.979 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.980 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.980 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.980 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.980 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.980 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.980 INFO:teuthology.orchestra.run.smithi107.stdout: "zabbix_port": { 2024-03-08T21:15:40.980 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "zabbix_port", 2024-03-08T21:15:40.980 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-08T21:15:40.980 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.980 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:40.980 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "10051", 2024-03-08T21:15:40.980 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.980 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.980 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.980 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.980 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.980 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.981 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.981 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.981 INFO:teuthology.orchestra.run.smithi107.stdout: "zabbix_sender": { 2024-03-08T21:15:40.981 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "zabbix_sender", 2024-03-08T21:15:40.981 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:40.981 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.981 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:40.981 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-03-08T21:15:40.981 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.981 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.981 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.981 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.981 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.981 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.981 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.981 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:40.982 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:40.982 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:40.982 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-03-08T21:15:40.982 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.982 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-08T21:15:40.982 INFO:teuthology.orchestra.run.smithi107.stdout: "gid": 24796, 2024-03-08T21:15:40.982 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "z", 2024-03-08T21:15:40.982 INFO:teuthology.orchestra.run.smithi107.stdout: "mgr_features": 4540701547738038271, 2024-03-08T21:15:40.982 INFO:teuthology.orchestra.run.smithi107.stdout: "available_modules": [ 2024-03-08T21:15:40.982 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-08T21:15:40.982 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "alerts", 2024-03-08T21:15:40.982 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-03-08T21:15:40.982 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-03-08T21:15:40.982 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-03-08T21:15:40.983 INFO:teuthology.orchestra.run.smithi107.stdout: "interval": { 2024-03-08T21:15:40.983 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "interval", 2024-03-08T21:15:40.983 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-03-08T21:15:40.983 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.983 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:40.983 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "60", 2024-03-08T21:15:40.983 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.983 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.983 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.983 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "How frequently to reexamine health status", 2024-03-08T21:15:40.983 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.983 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.983 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.983 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.983 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-03-08T21:15:40.984 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-03-08T21:15:40.984 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:40.984 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.984 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:40.984 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:40.984 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.984 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.984 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:40.984 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-08T21:15:40.984 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-08T21:15:40.984 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-08T21:15:40.984 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-08T21:15:40.984 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-08T21:15:40.984 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-08T21:15:40.985 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:40.985 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.985 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.985 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.985 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.985 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.985 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-03-08T21:15:40.985 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-03-08T21:15:40.985 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:40.985 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.985 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:40.985 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:40.985 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.985 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.985 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.985 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.986 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.986 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.986 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.986 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.986 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-03-08T21:15:40.986 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-03-08T21:15:40.986 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:40.986 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.986 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:40.986 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-03-08T21:15:40.986 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.986 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.986 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:40.986 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-08T21:15:40.986 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-08T21:15:40.987 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-08T21:15:40.987 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-08T21:15:40.987 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-08T21:15:40.987 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-08T21:15:40.987 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:40.987 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.987 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.987 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.987 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.987 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.987 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-03-08T21:15:40.987 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-03-08T21:15:40.987 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:40.987 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.987 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:40.987 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:40.988 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.988 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.988 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.988 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:40.988 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.988 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.988 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.988 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.988 INFO:teuthology.orchestra.run.smithi107.stdout: "smtp_destination": { 2024-03-08T21:15:40.988 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "smtp_destination", 2024-03-08T21:15:40.988 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:40.988 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.988 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:40.988 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:40.988 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.988 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.989 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.989 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Email address to send alerts to", 2024-03-08T21:15:40.989 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.989 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.989 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.989 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.989 INFO:teuthology.orchestra.run.smithi107.stdout: "smtp_from_name": { 2024-03-08T21:15:40.989 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "smtp_from_name", 2024-03-08T21:15:40.989 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:40.989 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.989 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:40.989 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "Ceph", 2024-03-08T21:15:40.989 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.989 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.989 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.990 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Email From: name", 2024-03-08T21:15:40.990 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.990 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.990 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.990 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.990 INFO:teuthology.orchestra.run.smithi107.stdout: "smtp_host": { 2024-03-08T21:15:40.990 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "smtp_host", 2024-03-08T21:15:40.990 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:40.990 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.990 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:40.990 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:40.990 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.990 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.990 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.990 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "SMTP server", 2024-03-08T21:15:40.990 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.991 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.991 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.991 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.991 INFO:teuthology.orchestra.run.smithi107.stdout: "smtp_password": { 2024-03-08T21:15:40.991 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "smtp_password", 2024-03-08T21:15:40.991 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:40.991 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.991 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:40.991 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:40.991 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.991 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.991 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.991 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Password to authenticate with", 2024-03-08T21:15:40.991 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.991 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.991 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.992 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.992 INFO:teuthology.orchestra.run.smithi107.stdout: "smtp_port": { 2024-03-08T21:15:40.992 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "smtp_port", 2024-03-08T21:15:40.992 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-08T21:15:40.992 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.992 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:40.992 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "465", 2024-03-08T21:15:40.992 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.992 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.992 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.992 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "SMTP port", 2024-03-08T21:15:40.992 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.992 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.992 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.992 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.992 INFO:teuthology.orchestra.run.smithi107.stdout: "smtp_sender": { 2024-03-08T21:15:40.993 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "smtp_sender", 2024-03-08T21:15:40.993 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:40.993 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.993 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:40.993 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:40.993 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.993 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.993 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.993 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "SMTP envelope sender", 2024-03-08T21:15:40.993 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.993 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.993 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.993 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.993 INFO:teuthology.orchestra.run.smithi107.stdout: "smtp_ssl": { 2024-03-08T21:15:40.993 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "smtp_ssl", 2024-03-08T21:15:40.993 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:40.994 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.994 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:40.994 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-08T21:15:40.994 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.994 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.994 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.994 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Use SSL to connect to SMTP server", 2024-03-08T21:15:40.994 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.994 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.994 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.994 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.994 INFO:teuthology.orchestra.run.smithi107.stdout: "smtp_user": { 2024-03-08T21:15:40.994 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "smtp_user", 2024-03-08T21:15:40.994 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:40.994 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.994 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:40.995 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:40.995 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.995 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.995 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.995 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "User to authenticate as", 2024-03-08T21:15:40.995 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.995 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.995 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.995 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:40.995 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:40.995 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.995 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-08T21:15:40.995 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "balancer", 2024-03-08T21:15:40.995 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-03-08T21:15:40.995 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-03-08T21:15:40.996 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-03-08T21:15:40.996 INFO:teuthology.orchestra.run.smithi107.stdout: "active": { 2024-03-08T21:15:40.996 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "active", 2024-03-08T21:15:40.996 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:40.996 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.996 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:40.996 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-08T21:15:40.996 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.996 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.996 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.996 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "automatically balance PGs across cluster", 2024-03-08T21:15:40.996 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.996 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.996 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.996 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.997 INFO:teuthology.orchestra.run.smithi107.stdout: "begin_time": { 2024-03-08T21:15:40.997 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "begin_time", 2024-03-08T21:15:40.997 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:40.997 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.997 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:40.997 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0000", 2024-03-08T21:15:40.997 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:40.997 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:40.997 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.997 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "beginning time of day to automatically balance", 2024-03-08T21:15:40.997 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-08T21:15:40.997 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.997 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.997 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.997 INFO:teuthology.orchestra.run.smithi107.stdout: "begin_weekday": { 2024-03-08T21:15:40.997 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "begin_weekday", 2024-03-08T21:15:40.997 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "uint", 2024-03-08T21:15:40.998 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.998 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:40.998 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0", 2024-03-08T21:15:40.998 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "0", 2024-03-08T21:15:40.998 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "6", 2024-03-08T21:15:40.998 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.998 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-03-08T21:15:40.998 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-08T21:15:40.998 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.998 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.998 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.998 INFO:teuthology.orchestra.run.smithi107.stdout: "crush_compat_max_iterations": { 2024-03-08T21:15:40.998 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "crush_compat_max_iterations", 2024-03-08T21:15:40.998 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "uint", 2024-03-08T21:15:40.998 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.999 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:40.999 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "25", 2024-03-08T21:15:40.999 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "1", 2024-03-08T21:15:40.999 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "250", 2024-03-08T21:15:40.999 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:40.999 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-03-08T21:15:40.999 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:40.999 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:40.999 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:40.999 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:40.999 INFO:teuthology.orchestra.run.smithi107.stdout: "crush_compat_metrics": { 2024-03-08T21:15:40.999 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "crush_compat_metrics", 2024-03-08T21:15:40.999 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:40.999 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:40.999 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:40.999 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "pgs,objects,bytes", 2024-03-08T21:15:41.000 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.000 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.000 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.000 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-03-08T21:15:41.000 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-03-08T21:15:41.000 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.000 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.000 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.000 INFO:teuthology.orchestra.run.smithi107.stdout: "crush_compat_step": { 2024-03-08T21:15:41.000 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "crush_compat_step", 2024-03-08T21:15:41.000 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "float", 2024-03-08T21:15:41.000 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.000 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.000 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0.5", 2024-03-08T21:15:41.001 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "0.001", 2024-03-08T21:15:41.001 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "0.999", 2024-03-08T21:15:41.001 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.001 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "aggressiveness of optimization", 2024-03-08T21:15:41.001 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-03-08T21:15:41.001 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.001 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.001 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.001 INFO:teuthology.orchestra.run.smithi107.stdout: "end_time": { 2024-03-08T21:15:41.001 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "end_time", 2024-03-08T21:15:41.001 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.001 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.001 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.001 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "2359", 2024-03-08T21:15:41.001 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.001 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.001 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.002 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "ending time of day to automatically balance", 2024-03-08T21:15:41.002 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-08T21:15:41.002 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.002 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.002 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.002 INFO:teuthology.orchestra.run.smithi107.stdout: "end_weekday": { 2024-03-08T21:15:41.002 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "end_weekday", 2024-03-08T21:15:41.002 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "uint", 2024-03-08T21:15:41.002 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.002 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.002 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0", 2024-03-08T21:15:41.002 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "0", 2024-03-08T21:15:41.002 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "6", 2024-03-08T21:15:41.002 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.002 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-03-08T21:15:41.002 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-08T21:15:41.003 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.003 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.003 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.003 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-03-08T21:15:41.003 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-03-08T21:15:41.003 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.003 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.003 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.003 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:41.003 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.003 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.003 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:41.003 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-08T21:15:41.003 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-08T21:15:41.003 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-08T21:15:41.004 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-08T21:15:41.004 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-08T21:15:41.004 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-08T21:15:41.004 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:41.004 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.004 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.004 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.004 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.004 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.004 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-03-08T21:15:41.004 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-03-08T21:15:41.004 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.004 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.004 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.004 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:41.004 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.005 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.005 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.005 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.005 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.005 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.005 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.005 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.005 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-03-08T21:15:41.005 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-03-08T21:15:41.005 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.005 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.005 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.005 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-03-08T21:15:41.005 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.005 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.005 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:41.006 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-08T21:15:41.006 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-08T21:15:41.006 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-08T21:15:41.006 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-08T21:15:41.006 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-08T21:15:41.006 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-08T21:15:41.006 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:41.006 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.006 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.006 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.006 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.006 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.006 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-03-08T21:15:41.006 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-03-08T21:15:41.006 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.006 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.007 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.007 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:41.007 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.007 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.007 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.007 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.007 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.007 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.007 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.007 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.007 INFO:teuthology.orchestra.run.smithi107.stdout: "min_score": { 2024-03-08T21:15:41.007 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "min_score", 2024-03-08T21:15:41.007 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "float", 2024-03-08T21:15:41.007 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.008 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.008 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0", 2024-03-08T21:15:41.008 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.008 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.008 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.008 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-03-08T21:15:41.008 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.008 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.008 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.008 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.008 INFO:teuthology.orchestra.run.smithi107.stdout: "mode": { 2024-03-08T21:15:41.008 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "mode", 2024-03-08T21:15:41.008 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.008 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.009 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.009 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "upmap", 2024-03-08T21:15:41.009 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.009 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.009 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:41.009 INFO:teuthology.orchestra.run.smithi107.stdout: "crush-compat", 2024-03-08T21:15:41.009 INFO:teuthology.orchestra.run.smithi107.stdout: "none", 2024-03-08T21:15:41.009 INFO:teuthology.orchestra.run.smithi107.stdout: "read", 2024-03-08T21:15:41.009 INFO:teuthology.orchestra.run.smithi107.stdout: "upmap", 2024-03-08T21:15:41.009 INFO:teuthology.orchestra.run.smithi107.stdout: "upmap-read" 2024-03-08T21:15:41.009 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:41.009 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Balancer mode", 2024-03-08T21:15:41.009 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.009 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.009 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.010 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.010 INFO:teuthology.orchestra.run.smithi107.stdout: "pool_ids": { 2024-03-08T21:15:41.010 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "pool_ids", 2024-03-08T21:15:41.010 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.010 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.010 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.010 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:41.010 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.010 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.010 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.010 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-03-08T21:15:41.010 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.010 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.010 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.010 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.010 INFO:teuthology.orchestra.run.smithi107.stdout: "sleep_interval": { 2024-03-08T21:15:41.011 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "sleep_interval", 2024-03-08T21:15:41.011 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-03-08T21:15:41.011 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.011 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.011 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "60", 2024-03-08T21:15:41.011 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.011 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.011 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.011 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-03-08T21:15:41.011 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.011 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.011 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.011 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.011 INFO:teuthology.orchestra.run.smithi107.stdout: "upmap_max_deviation": { 2024-03-08T21:15:41.011 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "upmap_max_deviation", 2024-03-08T21:15:41.011 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-08T21:15:41.012 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.012 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.012 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "5", 2024-03-08T21:15:41.012 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "1", 2024-03-08T21:15:41.012 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.012 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.012 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "deviation below which no optimization is attempted", 2024-03-08T21:15:41.012 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-03-08T21:15:41.012 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.012 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.012 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.012 INFO:teuthology.orchestra.run.smithi107.stdout: "upmap_max_optimizations": { 2024-03-08T21:15:41.012 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "upmap_max_optimizations", 2024-03-08T21:15:41.012 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "uint", 2024-03-08T21:15:41.012 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.012 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.013 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "10", 2024-03-08T21:15:41.013 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.013 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.013 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.013 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-03-08T21:15:41.013 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.013 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.013 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.013 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:41.013 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:41.013 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.013 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-08T21:15:41.013 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "cephadm", 2024-03-08T21:15:41.013 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-03-08T21:15:41.013 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-03-08T21:15:41.013 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-03-08T21:15:41.014 INFO:teuthology.orchestra.run.smithi107.stdout: "agent_down_multiplier": { 2024-03-08T21:15:41.014 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "agent_down_multiplier", 2024-03-08T21:15:41.014 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "float", 2024-03-08T21:15:41.014 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.014 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.014 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "3.0", 2024-03-08T21:15:41.014 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.014 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.014 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.014 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-03-08T21:15:41.014 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.014 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.014 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.014 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.014 INFO:teuthology.orchestra.run.smithi107.stdout: "agent_refresh_rate": { 2024-03-08T21:15:41.015 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "agent_refresh_rate", 2024-03-08T21:15:41.015 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-03-08T21:15:41.015 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.015 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.015 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "20", 2024-03-08T21:15:41.015 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.015 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.015 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.015 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-03-08T21:15:41.015 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.015 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.015 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.015 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.015 INFO:teuthology.orchestra.run.smithi107.stdout: "agent_starting_port": { 2024-03-08T21:15:41.015 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "agent_starting_port", 2024-03-08T21:15:41.015 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-08T21:15:41.016 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.016 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.016 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "4721", 2024-03-08T21:15:41.016 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.016 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.016 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.016 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-03-08T21:15:41.016 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.016 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.016 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.016 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.016 INFO:teuthology.orchestra.run.smithi107.stdout: "allow_ptrace": { 2024-03-08T21:15:41.016 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "allow_ptrace", 2024-03-08T21:15:41.016 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.016 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.016 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.016 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:41.017 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.017 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.017 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.017 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-03-08T21:15:41.017 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-03-08T21:15:41.017 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.017 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.017 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.017 INFO:teuthology.orchestra.run.smithi107.stdout: "autotune_interval": { 2024-03-08T21:15:41.017 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "autotune_interval", 2024-03-08T21:15:41.017 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-03-08T21:15:41.017 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.017 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.017 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "600", 2024-03-08T21:15:41.017 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.018 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.018 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.018 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "how frequently to autotune daemon memory", 2024-03-08T21:15:41.018 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.018 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.018 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.018 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.018 INFO:teuthology.orchestra.run.smithi107.stdout: "autotune_memory_target_ratio": { 2024-03-08T21:15:41.018 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "autotune_memory_target_ratio", 2024-03-08T21:15:41.018 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "float", 2024-03-08T21:15:41.018 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.018 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.018 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0.7", 2024-03-08T21:15:41.018 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.018 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.019 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.019 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-03-08T21:15:41.019 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.019 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.019 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.019 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.019 INFO:teuthology.orchestra.run.smithi107.stdout: "cephadm_log_destination": { 2024-03-08T21:15:41.019 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "cephadm_log_destination", 2024-03-08T21:15:41.019 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.019 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.019 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.019 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:41.019 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.019 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.019 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:41.020 INFO:teuthology.orchestra.run.smithi107.stdout: "file", 2024-03-08T21:15:41.020 INFO:teuthology.orchestra.run.smithi107.stdout: "file,syslog", 2024-03-08T21:15:41.020 INFO:teuthology.orchestra.run.smithi107.stdout: "syslog" 2024-03-08T21:15:41.020 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:41.020 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-03-08T21:15:41.020 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.020 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.020 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.020 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.020 INFO:teuthology.orchestra.run.smithi107.stdout: "cgroups_split": { 2024-03-08T21:15:41.020 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "cgroups_split", 2024-03-08T21:15:41.020 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.020 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.020 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.020 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-08T21:15:41.020 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.021 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.021 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.021 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-03-08T21:15:41.021 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.021 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.021 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.021 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.021 INFO:teuthology.orchestra.run.smithi107.stdout: "config_checks_enabled": { 2024-03-08T21:15:41.021 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "config_checks_enabled", 2024-03-08T21:15:41.021 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.021 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.021 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.021 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:41.021 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.021 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.021 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.022 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-03-08T21:15:41.022 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.022 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.022 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.022 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.022 INFO:teuthology.orchestra.run.smithi107.stdout: "config_dashboard": { 2024-03-08T21:15:41.022 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "config_dashboard", 2024-03-08T21:15:41.022 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.022 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.022 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.022 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-08T21:15:41.022 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.022 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.022 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.022 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-03-08T21:15:41.022 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.023 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.023 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.023 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.023 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_alertmanager": { 2024-03-08T21:15:41.023 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_alertmanager", 2024-03-08T21:15:41.023 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.023 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.023 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.023 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-03-08T21:15:41.023 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.023 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.023 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.023 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Prometheus container image", 2024-03-08T21:15:41.023 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.023 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.023 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.024 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.024 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_base": { 2024-03-08T21:15:41.024 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_base", 2024-03-08T21:15:41.024 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.024 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.024 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.024 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/ceph/ceph", 2024-03-08T21:15:41.024 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.024 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.024 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.024 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Container image name, without the tag", 2024-03-08T21:15:41.024 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.024 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.024 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.024 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.024 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_elasticsearch": { 2024-03-08T21:15:41.025 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_elasticsearch", 2024-03-08T21:15:41.025 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.025 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.025 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.025 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-03-08T21:15:41.025 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.025 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.025 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.025 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "elasticsearch container image", 2024-03-08T21:15:41.025 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.025 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.025 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.025 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.025 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_grafana": { 2024-03-08T21:15:41.025 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_grafana", 2024-03-08T21:15:41.025 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.025 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.026 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.026 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.12", 2024-03-08T21:15:41.026 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.026 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.026 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.026 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Prometheus container image", 2024-03-08T21:15:41.026 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.026 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.026 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.026 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.026 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_haproxy": { 2024-03-08T21:15:41.026 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_haproxy", 2024-03-08T21:15:41.026 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.026 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.026 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.026 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-03-08T21:15:41.027 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.027 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.027 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.027 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "HAproxy container image", 2024-03-08T21:15:41.027 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.027 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.027 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.027 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.027 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_jaeger_agent": { 2024-03-08T21:15:41.027 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_jaeger_agent", 2024-03-08T21:15:41.027 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.027 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.027 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.027 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-03-08T21:15:41.027 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.027 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.028 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.028 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Jaeger agent container image", 2024-03-08T21:15:41.028 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.028 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.028 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.028 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.028 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_jaeger_collector": { 2024-03-08T21:15:41.028 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_jaeger_collector", 2024-03-08T21:15:41.028 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.028 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.028 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.028 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-03-08T21:15:41.028 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.028 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.028 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.028 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Jaeger collector container image", 2024-03-08T21:15:41.029 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.029 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.029 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.029 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.029 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_jaeger_query": { 2024-03-08T21:15:41.029 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_jaeger_query", 2024-03-08T21:15:41.029 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.029 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.029 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.029 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-03-08T21:15:41.029 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.029 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.029 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.029 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Jaeger query container image", 2024-03-08T21:15:41.029 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.029 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.030 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.030 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.030 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_keepalived": { 2024-03-08T21:15:41.030 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_keepalived", 2024-03-08T21:15:41.030 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.030 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.030 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.030 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-03-08T21:15:41.030 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.030 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.030 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.030 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Keepalived container image", 2024-03-08T21:15:41.030 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.030 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.030 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.031 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.031 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_loki": { 2024-03-08T21:15:41.031 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_loki", 2024-03-08T21:15:41.031 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.031 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.031 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.031 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-03-08T21:15:41.031 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.031 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.031 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.031 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Loki container image", 2024-03-08T21:15:41.031 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.031 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.031 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.031 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.032 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_node_exporter": { 2024-03-08T21:15:41.032 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_node_exporter", 2024-03-08T21:15:41.032 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.032 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.032 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.032 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-03-08T21:15:41.032 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.032 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.032 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.032 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Prometheus container image", 2024-03-08T21:15:41.032 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.032 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.032 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.032 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.032 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_nvmeof": { 2024-03-08T21:15:41.032 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_nvmeof", 2024-03-08T21:15:41.033 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.033 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.033 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.033 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/ceph/nvmeof:1.0.0", 2024-03-08T21:15:41.033 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.033 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.033 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.033 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Nvme-of container image", 2024-03-08T21:15:41.033 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.033 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.033 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.033 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.033 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_prometheus": { 2024-03-08T21:15:41.033 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_prometheus", 2024-03-08T21:15:41.033 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.033 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.033 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.034 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-03-08T21:15:41.034 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.034 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.034 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.034 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Prometheus container image", 2024-03-08T21:15:41.034 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.034 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.034 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.034 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.034 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_promtail": { 2024-03-08T21:15:41.034 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_promtail", 2024-03-08T21:15:41.034 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.034 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.034 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.034 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-03-08T21:15:41.035 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.035 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.035 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.035 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Promtail container image", 2024-03-08T21:15:41.035 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.035 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.035 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.035 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.035 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_snmp_gateway": { 2024-03-08T21:15:41.035 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_snmp_gateway", 2024-03-08T21:15:41.035 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.035 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.035 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.035 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-03-08T21:15:41.035 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.035 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.036 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.036 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "SNMP Gateway container image", 2024-03-08T21:15:41.036 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.036 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.036 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.036 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.036 INFO:teuthology.orchestra.run.smithi107.stdout: "container_init": { 2024-03-08T21:15:41.036 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_init", 2024-03-08T21:15:41.036 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.036 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.036 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.036 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-08T21:15:41.036 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.036 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.036 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.036 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Run podman/docker with `--init`", 2024-03-08T21:15:41.036 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.037 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.037 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.037 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.037 INFO:teuthology.orchestra.run.smithi107.stdout: "daemon_cache_timeout": { 2024-03-08T21:15:41.037 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "daemon_cache_timeout", 2024-03-08T21:15:41.037 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-03-08T21:15:41.037 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.037 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.037 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "600", 2024-03-08T21:15:41.037 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.037 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.037 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.037 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-03-08T21:15:41.037 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.037 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.037 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.038 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.038 INFO:teuthology.orchestra.run.smithi107.stdout: "default_cephadm_command_timeout": { 2024-03-08T21:15:41.038 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "default_cephadm_command_timeout", 2024-03-08T21:15:41.038 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-08T21:15:41.038 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.038 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.038 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "900", 2024-03-08T21:15:41.038 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.038 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.038 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.038 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-03-08T21:15:41.038 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.038 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.038 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.038 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.038 INFO:teuthology.orchestra.run.smithi107.stdout: "default_registry": { 2024-03-08T21:15:41.039 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "default_registry", 2024-03-08T21:15:41.039 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.039 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.039 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.039 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "docker.io", 2024-03-08T21:15:41.039 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.039 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.039 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.039 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-03-08T21:15:41.039 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.039 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.039 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.039 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.039 INFO:teuthology.orchestra.run.smithi107.stdout: "device_cache_timeout": { 2024-03-08T21:15:41.039 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "device_cache_timeout", 2024-03-08T21:15:41.040 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-03-08T21:15:41.040 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.040 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.040 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "1800", 2024-03-08T21:15:41.040 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.040 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.040 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.040 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "seconds to cache device inventory", 2024-03-08T21:15:41.040 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.040 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.040 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.040 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.040 INFO:teuthology.orchestra.run.smithi107.stdout: "device_enhanced_scan": { 2024-03-08T21:15:41.040 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "device_enhanced_scan", 2024-03-08T21:15:41.040 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.040 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.040 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.041 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:41.041 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.041 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.041 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.041 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Use libstoragemgmt during device scans", 2024-03-08T21:15:41.041 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.041 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.041 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.041 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.041 INFO:teuthology.orchestra.run.smithi107.stdout: "facts_cache_timeout": { 2024-03-08T21:15:41.041 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "facts_cache_timeout", 2024-03-08T21:15:41.041 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-03-08T21:15:41.041 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.041 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.041 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "60", 2024-03-08T21:15:41.041 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.042 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.042 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.042 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "seconds to cache host facts data", 2024-03-08T21:15:41.042 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.042 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.042 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.042 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.042 INFO:teuthology.orchestra.run.smithi107.stdout: "host_check_interval": { 2024-03-08T21:15:41.042 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "host_check_interval", 2024-03-08T21:15:41.042 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-03-08T21:15:41.042 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.042 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.042 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "600", 2024-03-08T21:15:41.042 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.042 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.042 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.043 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "how frequently to perform a host check", 2024-03-08T21:15:41.043 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.043 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.043 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.043 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.043 INFO:teuthology.orchestra.run.smithi107.stdout: "hw_monitoring": { 2024-03-08T21:15:41.043 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "hw_monitoring", 2024-03-08T21:15:41.043 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.043 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.043 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.043 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:41.043 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.043 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.043 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.043 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-03-08T21:15:41.044 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.044 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.044 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.044 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.044 INFO:teuthology.orchestra.run.smithi107.stdout: "inventory_list_all": { 2024-03-08T21:15:41.044 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "inventory_list_all", 2024-03-08T21:15:41.044 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.044 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.044 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.044 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:41.044 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.044 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.044 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.044 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-03-08T21:15:41.044 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.044 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.045 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.045 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.045 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-03-08T21:15:41.045 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-03-08T21:15:41.045 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.045 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.045 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.045 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:41.045 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.045 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.045 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:41.045 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-08T21:15:41.045 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-08T21:15:41.045 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-08T21:15:41.045 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-08T21:15:41.045 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-08T21:15:41.046 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-08T21:15:41.046 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:41.046 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.046 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.046 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.046 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.046 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.046 INFO:teuthology.orchestra.run.smithi107.stdout: "log_refresh_metadata": { 2024-03-08T21:15:41.046 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_refresh_metadata", 2024-03-08T21:15:41.046 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.046 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.046 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.046 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:41.046 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.046 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.046 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.047 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-03-08T21:15:41.047 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.047 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.047 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.047 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.047 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-03-08T21:15:41.047 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-03-08T21:15:41.047 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.047 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.047 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.047 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-08T21:15:41.047 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.047 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.047 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.047 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-03-08T21:15:41.048 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.048 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.048 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.048 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.048 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-03-08T21:15:41.048 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-03-08T21:15:41.048 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.048 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.048 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.048 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-03-08T21:15:41.048 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.048 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.048 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:41.048 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-08T21:15:41.048 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-08T21:15:41.048 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-08T21:15:41.048 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-08T21:15:41.049 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-08T21:15:41.049 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-08T21:15:41.049 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:41.049 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.049 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.049 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.049 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.049 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.049 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-03-08T21:15:41.049 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-03-08T21:15:41.049 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.049 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.049 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.049 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:41.049 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.049 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.050 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.050 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.050 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.050 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.050 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.050 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.050 INFO:teuthology.orchestra.run.smithi107.stdout: "manage_etc_ceph_ceph_conf": { 2024-03-08T21:15:41.050 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-03-08T21:15:41.050 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.050 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.050 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.050 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:41.050 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.050 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.050 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.050 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-03-08T21:15:41.051 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.051 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.051 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.051 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.051 INFO:teuthology.orchestra.run.smithi107.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-03-08T21:15:41.051 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-03-08T21:15:41.051 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.051 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.051 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.051 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "*", 2024-03-08T21:15:41.051 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.051 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.051 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.051 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-03-08T21:15:41.051 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.051 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.052 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.052 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.052 INFO:teuthology.orchestra.run.smithi107.stdout: "max_count_per_host": { 2024-03-08T21:15:41.052 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "max_count_per_host", 2024-03-08T21:15:41.052 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-08T21:15:41.052 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.052 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.052 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "10", 2024-03-08T21:15:41.052 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.052 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.052 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.052 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "max number of daemons per service per host", 2024-03-08T21:15:41.052 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.052 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.052 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.052 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.053 INFO:teuthology.orchestra.run.smithi107.stdout: "max_osd_draining_count": { 2024-03-08T21:15:41.053 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "max_osd_draining_count", 2024-03-08T21:15:41.053 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-08T21:15:41.053 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.053 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.053 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "10", 2024-03-08T21:15:41.053 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.053 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.053 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.053 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-03-08T21:15:41.053 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.053 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.053 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.053 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.053 INFO:teuthology.orchestra.run.smithi107.stdout: "migration_current": { 2024-03-08T21:15:41.053 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "migration_current", 2024-03-08T21:15:41.054 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-08T21:15:41.054 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.054 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.054 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:41.054 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.054 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.054 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.054 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "internal - do not modify", 2024-03-08T21:15:41.054 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.054 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.054 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.054 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.054 INFO:teuthology.orchestra.run.smithi107.stdout: "mode": { 2024-03-08T21:15:41.054 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "mode", 2024-03-08T21:15:41.054 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.055 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.055 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.055 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "root", 2024-03-08T21:15:41.055 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.055 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.055 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:41.055 INFO:teuthology.orchestra.run.smithi107.stdout: "cephadm-package", 2024-03-08T21:15:41.055 INFO:teuthology.orchestra.run.smithi107.stdout: "root" 2024-03-08T21:15:41.055 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:41.055 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "mode for remote execution of cephadm", 2024-03-08T21:15:41.055 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.055 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.055 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.055 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.055 INFO:teuthology.orchestra.run.smithi107.stdout: "oob_default_addr": { 2024-03-08T21:15:41.055 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "oob_default_addr", 2024-03-08T21:15:41.056 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.056 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.056 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.056 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "169.254.1.1", 2024-03-08T21:15:41.056 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.056 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.056 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.056 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Default address for RedFish API (oob management).", 2024-03-08T21:15:41.056 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.056 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.056 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.056 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.056 INFO:teuthology.orchestra.run.smithi107.stdout: "prometheus_alerts_path": { 2024-03-08T21:15:41.056 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "prometheus_alerts_path", 2024-03-08T21:15:41.056 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.056 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.056 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.057 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-03-08T21:15:41.057 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.057 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.057 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.057 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-03-08T21:15:41.057 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.057 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.057 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.057 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.057 INFO:teuthology.orchestra.run.smithi107.stdout: "registry_insecure": { 2024-03-08T21:15:41.057 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "registry_insecure", 2024-03-08T21:15:41.057 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.057 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.057 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.057 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:41.057 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.058 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.058 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.058 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-03-08T21:15:41.058 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.058 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.058 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.058 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.058 INFO:teuthology.orchestra.run.smithi107.stdout: "registry_password": { 2024-03-08T21:15:41.058 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "registry_password", 2024-03-08T21:15:41.058 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.058 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.058 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.058 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:41.058 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.058 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.059 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.059 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-03-08T21:15:41.059 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.059 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.059 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.059 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.059 INFO:teuthology.orchestra.run.smithi107.stdout: "registry_url": { 2024-03-08T21:15:41.059 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "registry_url", 2024-03-08T21:15:41.059 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.059 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.059 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.059 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:41.059 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.059 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.059 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.059 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-03-08T21:15:41.059 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.060 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.060 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.060 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.060 INFO:teuthology.orchestra.run.smithi107.stdout: "registry_username": { 2024-03-08T21:15:41.060 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "registry_username", 2024-03-08T21:15:41.060 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.060 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.060 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.060 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:41.060 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.060 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.060 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.060 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-03-08T21:15:41.060 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.060 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.060 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.061 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.061 INFO:teuthology.orchestra.run.smithi107.stdout: "secure_monitoring_stack": { 2024-03-08T21:15:41.061 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "secure_monitoring_stack", 2024-03-08T21:15:41.061 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.061 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.061 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.061 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:41.061 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.061 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.061 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.061 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-03-08T21:15:41.061 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.061 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.061 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.061 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.061 INFO:teuthology.orchestra.run.smithi107.stdout: "service_discovery_port": { 2024-03-08T21:15:41.062 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "service_discovery_port", 2024-03-08T21:15:41.062 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-08T21:15:41.062 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.062 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.062 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "8765", 2024-03-08T21:15:41.062 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.062 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.062 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.062 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "cephadm service discovery port", 2024-03-08T21:15:41.062 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.062 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.062 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.062 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.062 INFO:teuthology.orchestra.run.smithi107.stdout: "ssh_config_file": { 2024-03-08T21:15:41.062 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "ssh_config_file", 2024-03-08T21:15:41.063 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.063 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.063 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.063 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:41.063 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.063 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.063 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.063 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-03-08T21:15:41.063 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.063 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.063 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.063 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.063 INFO:teuthology.orchestra.run.smithi107.stdout: "use_agent": { 2024-03-08T21:15:41.063 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "use_agent", 2024-03-08T21:15:41.063 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.064 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.064 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.064 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:41.064 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.064 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.064 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.064 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-03-08T21:15:41.064 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.064 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.064 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.064 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.064 INFO:teuthology.orchestra.run.smithi107.stdout: "use_repo_digest": { 2024-03-08T21:15:41.064 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "use_repo_digest", 2024-03-08T21:15:41.064 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.064 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.064 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.064 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-08T21:15:41.065 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.065 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.065 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.065 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-03-08T21:15:41.065 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.065 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.065 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.065 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.065 INFO:teuthology.orchestra.run.smithi107.stdout: "warn_on_failed_host_check": { 2024-03-08T21:15:41.065 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "warn_on_failed_host_check", 2024-03-08T21:15:41.065 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.065 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.065 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.065 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-08T21:15:41.065 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.065 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.066 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.066 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "raise a health warning if the host check fails", 2024-03-08T21:15:41.066 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.066 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.066 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.066 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.066 INFO:teuthology.orchestra.run.smithi107.stdout: "warn_on_stray_daemons": { 2024-03-08T21:15:41.066 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "warn_on_stray_daemons", 2024-03-08T21:15:41.066 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.066 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.066 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.066 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-08T21:15:41.066 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.066 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.066 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.066 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-03-08T21:15:41.067 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.067 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.067 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.067 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.067 INFO:teuthology.orchestra.run.smithi107.stdout: "warn_on_stray_hosts": { 2024-03-08T21:15:41.067 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "warn_on_stray_hosts", 2024-03-08T21:15:41.067 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.067 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.067 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.067 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-08T21:15:41.067 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.067 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.067 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.067 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-03-08T21:15:41.067 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.067 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.068 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.068 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:41.068 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:41.068 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.068 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-08T21:15:41.068 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "crash", 2024-03-08T21:15:41.068 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-03-08T21:15:41.068 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-03-08T21:15:41.068 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-03-08T21:15:41.068 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-03-08T21:15:41.068 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-03-08T21:15:41.068 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.068 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.068 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.068 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:41.068 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.069 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.069 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:41.069 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-08T21:15:41.069 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-08T21:15:41.069 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-08T21:15:41.069 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-08T21:15:41.069 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-08T21:15:41.069 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-08T21:15:41.069 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:41.069 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.069 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.069 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.069 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.069 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.069 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-03-08T21:15:41.070 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-03-08T21:15:41.070 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.070 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.070 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.070 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:41.070 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.070 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.070 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.070 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.070 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.070 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.070 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.070 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.070 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-03-08T21:15:41.070 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-03-08T21:15:41.070 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.071 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.071 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.071 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-03-08T21:15:41.071 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.071 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.071 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:41.071 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-08T21:15:41.071 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-08T21:15:41.071 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-08T21:15:41.071 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-08T21:15:41.071 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-08T21:15:41.071 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-08T21:15:41.071 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:41.071 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.071 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.071 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.072 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.072 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.072 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-03-08T21:15:41.072 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-03-08T21:15:41.072 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.072 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.072 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.072 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:41.072 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.072 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.072 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.072 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.072 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.072 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.072 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.072 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.073 INFO:teuthology.orchestra.run.smithi107.stdout: "retain_interval": { 2024-03-08T21:15:41.073 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "retain_interval", 2024-03-08T21:15:41.073 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-03-08T21:15:41.073 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.073 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.073 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "31536000", 2024-03-08T21:15:41.073 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.073 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.073 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.073 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "how long to retain crashes before pruning them", 2024-03-08T21:15:41.073 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.073 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.073 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.073 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.073 INFO:teuthology.orchestra.run.smithi107.stdout: "warn_recent_interval": { 2024-03-08T21:15:41.073 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "warn_recent_interval", 2024-03-08T21:15:41.074 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-03-08T21:15:41.074 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.074 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.074 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "1209600", 2024-03-08T21:15:41.074 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.074 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.074 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.074 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "time interval in which to warn about recent crashes", 2024-03-08T21:15:41.074 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.074 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.074 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.074 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:41.074 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:41.074 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.074 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-08T21:15:41.075 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "dashboard", 2024-03-08T21:15:41.075 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-03-08T21:15:41.075 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-03-08T21:15:41.075 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-03-08T21:15:41.075 INFO:teuthology.orchestra.run.smithi107.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-03-08T21:15:41.075 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-03-08T21:15:41.075 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-08T21:15:41.075 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.075 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.075 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "10", 2024-03-08T21:15:41.075 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.075 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.075 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.075 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.075 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.075 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.076 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.076 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.076 INFO:teuthology.orchestra.run.smithi107.stdout: "ALERTMANAGER_API_HOST": { 2024-03-08T21:15:41.076 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "ALERTMANAGER_API_HOST", 2024-03-08T21:15:41.076 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.076 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.076 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.076 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:41.076 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.076 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.076 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.076 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.076 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.076 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.076 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.077 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.077 INFO:teuthology.orchestra.run.smithi107.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-03-08T21:15:41.077 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-03-08T21:15:41.077 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.077 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.077 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.077 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-08T21:15:41.077 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.077 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.077 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.077 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.077 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.077 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.077 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.077 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.078 INFO:teuthology.orchestra.run.smithi107.stdout: "AUDIT_API_ENABLED": { 2024-03-08T21:15:41.078 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "AUDIT_API_ENABLED", 2024-03-08T21:15:41.078 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.078 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.078 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.078 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:41.078 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.078 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.078 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.078 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.078 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.078 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.078 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.078 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.078 INFO:teuthology.orchestra.run.smithi107.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-03-08T21:15:41.078 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-03-08T21:15:41.079 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.079 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.079 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.079 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-08T21:15:41.079 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.079 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.079 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.079 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.079 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.079 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.079 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.079 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.079 INFO:teuthology.orchestra.run.smithi107.stdout: "ENABLE_BROWSABLE_API": { 2024-03-08T21:15:41.079 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "ENABLE_BROWSABLE_API", 2024-03-08T21:15:41.079 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.079 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.080 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.080 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-08T21:15:41.080 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.080 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.080 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.080 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.080 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.080 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.080 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.080 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.080 INFO:teuthology.orchestra.run.smithi107.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-03-08T21:15:41.080 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-03-08T21:15:41.080 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.080 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.080 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.080 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-08T21:15:41.081 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.081 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.081 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.081 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.081 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.081 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.081 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.081 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.081 INFO:teuthology.orchestra.run.smithi107.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-03-08T21:15:41.081 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-03-08T21:15:41.081 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.081 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.081 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.081 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-08T21:15:41.081 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.081 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.082 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.082 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.082 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.082 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.082 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.082 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.082 INFO:teuthology.orchestra.run.smithi107.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-03-08T21:15:41.082 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-03-08T21:15:41.082 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.082 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.082 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.082 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-08T21:15:41.082 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.082 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.082 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.082 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.082 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.083 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.083 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.083 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.083 INFO:teuthology.orchestra.run.smithi107.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-03-08T21:15:41.083 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-03-08T21:15:41.083 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.083 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.083 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.083 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-08T21:15:41.083 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.083 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.083 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.083 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.083 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.083 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.084 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.084 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.084 INFO:teuthology.orchestra.run.smithi107.stdout: "FEATURE_TOGGLE_NFS": { 2024-03-08T21:15:41.084 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-03-08T21:15:41.084 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.084 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.084 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.084 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-08T21:15:41.084 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.084 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.084 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.084 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.084 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.084 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.084 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.084 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.085 INFO:teuthology.orchestra.run.smithi107.stdout: "FEATURE_TOGGLE_RBD": { 2024-03-08T21:15:41.085 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-03-08T21:15:41.085 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.085 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.085 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.085 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-08T21:15:41.085 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.085 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.085 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.085 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.085 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.085 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.085 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.085 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.085 INFO:teuthology.orchestra.run.smithi107.stdout: "FEATURE_TOGGLE_RGW": { 2024-03-08T21:15:41.085 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-03-08T21:15:41.086 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.086 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.086 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.086 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-08T21:15:41.086 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.086 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.086 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.086 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.086 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.086 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.086 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.086 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.086 INFO:teuthology.orchestra.run.smithi107.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-03-08T21:15:41.086 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-03-08T21:15:41.086 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.086 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.087 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.087 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:41.087 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.087 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.087 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.087 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.087 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.087 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.087 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.087 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.087 INFO:teuthology.orchestra.run.smithi107.stdout: "GRAFANA_API_PASSWORD": { 2024-03-08T21:15:41.087 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "GRAFANA_API_PASSWORD", 2024-03-08T21:15:41.087 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.087 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.087 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.087 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "admin", 2024-03-08T21:15:41.088 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.088 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.088 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.088 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.088 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.088 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.088 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.088 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.088 INFO:teuthology.orchestra.run.smithi107.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-03-08T21:15:41.088 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-03-08T21:15:41.088 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.088 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.088 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.088 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-08T21:15:41.088 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.089 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.089 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.089 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.089 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.089 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.089 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.089 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.089 INFO:teuthology.orchestra.run.smithi107.stdout: "GRAFANA_API_URL": { 2024-03-08T21:15:41.089 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "GRAFANA_API_URL", 2024-03-08T21:15:41.089 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.089 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.089 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.089 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:41.089 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.089 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.089 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.090 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.090 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.090 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.090 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.090 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.090 INFO:teuthology.orchestra.run.smithi107.stdout: "GRAFANA_API_USERNAME": { 2024-03-08T21:15:41.090 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "GRAFANA_API_USERNAME", 2024-03-08T21:15:41.090 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.090 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.090 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.090 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "admin", 2024-03-08T21:15:41.090 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.090 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.090 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.091 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.091 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.091 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.091 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.091 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.091 INFO:teuthology.orchestra.run.smithi107.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-03-08T21:15:41.091 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-03-08T21:15:41.091 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.091 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.091 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.091 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:41.091 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.091 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.091 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.091 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.091 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.092 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.092 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.092 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.092 INFO:teuthology.orchestra.run.smithi107.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-03-08T21:15:41.092 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-03-08T21:15:41.092 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.092 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.092 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.092 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:41.092 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.092 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.092 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.092 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.092 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.092 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.093 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.093 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.093 INFO:teuthology.orchestra.run.smithi107.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-03-08T21:15:41.093 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-03-08T21:15:41.093 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.093 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.093 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.093 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-08T21:15:41.093 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.093 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.093 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.093 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.093 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.093 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.093 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.094 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.094 INFO:teuthology.orchestra.run.smithi107.stdout: "ISSUE_TRACKER_API_KEY": { 2024-03-08T21:15:41.094 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-03-08T21:15:41.094 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.094 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.094 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.094 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:41.094 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.094 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.094 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.094 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.094 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.094 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.094 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.094 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.094 INFO:teuthology.orchestra.run.smithi107.stdout: "MULTICLUSTER_CONFIG": { 2024-03-08T21:15:41.095 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "MULTICLUSTER_CONFIG", 2024-03-08T21:15:41.095 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.095 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.095 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.095 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "{}", 2024-03-08T21:15:41.095 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.095 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.095 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.095 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.095 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.095 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.095 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.095 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.095 INFO:teuthology.orchestra.run.smithi107.stdout: "PROMETHEUS_API_HOST": { 2024-03-08T21:15:41.095 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PROMETHEUS_API_HOST", 2024-03-08T21:15:41.095 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.096 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.096 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.096 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:41.096 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.096 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.096 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.096 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.096 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.096 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.096 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.096 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.096 INFO:teuthology.orchestra.run.smithi107.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-03-08T21:15:41.096 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-03-08T21:15:41.096 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.096 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.096 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.097 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-08T21:15:41.097 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.097 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.097 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.097 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.097 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.097 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.097 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.097 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.097 INFO:teuthology.orchestra.run.smithi107.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-03-08T21:15:41.097 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-03-08T21:15:41.097 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.097 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.097 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.097 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:41.098 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.098 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.098 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.098 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.098 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.098 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.098 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.098 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.098 INFO:teuthology.orchestra.run.smithi107.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-03-08T21:15:41.098 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-03-08T21:15:41.098 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.098 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.098 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.098 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:41.098 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.098 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.099 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.099 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.099 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.099 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.099 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.099 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.099 INFO:teuthology.orchestra.run.smithi107.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-03-08T21:15:41.099 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-03-08T21:15:41.099 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.099 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.099 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.099 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-08T21:15:41.099 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.099 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.099 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.099 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.100 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.100 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.100 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.100 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.100 INFO:teuthology.orchestra.run.smithi107.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-03-08T21:15:41.100 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-03-08T21:15:41.100 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.100 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.100 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.100 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-08T21:15:41.100 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.100 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.100 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.100 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.101 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.101 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.101 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.101 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.101 INFO:teuthology.orchestra.run.smithi107.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-03-08T21:15:41.101 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-03-08T21:15:41.101 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.101 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.101 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.101 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:41.101 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.101 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.101 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.101 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.101 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.102 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.102 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.102 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.102 INFO:teuthology.orchestra.run.smithi107.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-03-08T21:15:41.102 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-03-08T21:15:41.102 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.102 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.102 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.102 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:41.102 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.102 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.102 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.102 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.102 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.102 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.102 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.103 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.103 INFO:teuthology.orchestra.run.smithi107.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-03-08T21:15:41.103 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-03-08T21:15:41.103 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.103 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.103 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.103 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:41.103 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.103 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.103 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.103 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.103 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.103 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.103 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.103 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.104 INFO:teuthology.orchestra.run.smithi107.stdout: "PWD_POLICY_ENABLED": { 2024-03-08T21:15:41.104 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PWD_POLICY_ENABLED", 2024-03-08T21:15:41.104 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.104 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.104 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.104 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-08T21:15:41.104 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.104 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.104 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.104 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.104 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.104 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.104 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.104 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.104 INFO:teuthology.orchestra.run.smithi107.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-03-08T21:15:41.105 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-03-08T21:15:41.105 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.105 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.105 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.105 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-03-08T21:15:41.105 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.105 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.105 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.105 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.105 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.105 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.105 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.105 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.105 INFO:teuthology.orchestra.run.smithi107.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-03-08T21:15:41.105 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-03-08T21:15:41.105 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-08T21:15:41.105 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.106 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.106 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "10", 2024-03-08T21:15:41.106 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.106 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.106 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.106 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.106 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.106 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.106 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.106 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.106 INFO:teuthology.orchestra.run.smithi107.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-03-08T21:15:41.106 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-03-08T21:15:41.106 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-08T21:15:41.106 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.106 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.106 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "8", 2024-03-08T21:15:41.107 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.107 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.107 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.107 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.107 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.107 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.107 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.107 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.107 INFO:teuthology.orchestra.run.smithi107.stdout: "REST_REQUESTS_TIMEOUT": { 2024-03-08T21:15:41.107 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-03-08T21:15:41.107 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-08T21:15:41.107 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.107 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.107 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "45", 2024-03-08T21:15:41.107 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.108 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.108 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.108 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.108 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.108 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.108 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.108 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.108 INFO:teuthology.orchestra.run.smithi107.stdout: "RGW_API_ACCESS_KEY": { 2024-03-08T21:15:41.108 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "RGW_API_ACCESS_KEY", 2024-03-08T21:15:41.108 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.108 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.108 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.108 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:41.108 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.108 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.109 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.109 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.109 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.109 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.109 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.109 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.109 INFO:teuthology.orchestra.run.smithi107.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-03-08T21:15:41.109 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-03-08T21:15:41.109 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.109 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.109 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.109 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "admin", 2024-03-08T21:15:41.109 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.109 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.109 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.109 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.110 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.110 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.110 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.110 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.110 INFO:teuthology.orchestra.run.smithi107.stdout: "RGW_API_SECRET_KEY": { 2024-03-08T21:15:41.110 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "RGW_API_SECRET_KEY", 2024-03-08T21:15:41.110 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.110 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.110 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.110 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:41.110 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.110 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.110 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.110 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.110 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.111 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.111 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.111 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.111 INFO:teuthology.orchestra.run.smithi107.stdout: "RGW_API_SSL_VERIFY": { 2024-03-08T21:15:41.111 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "RGW_API_SSL_VERIFY", 2024-03-08T21:15:41.111 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.111 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.111 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.111 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-08T21:15:41.111 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.111 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.111 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.111 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.111 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.111 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.112 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.112 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.112 INFO:teuthology.orchestra.run.smithi107.stdout: "UNSAFE_TLS_v1_2": { 2024-03-08T21:15:41.112 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "UNSAFE_TLS_v1_2", 2024-03-08T21:15:41.112 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.112 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.112 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.112 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:41.112 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.112 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.112 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.112 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.112 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.112 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.112 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.112 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.113 INFO:teuthology.orchestra.run.smithi107.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-03-08T21:15:41.113 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-03-08T21:15:41.113 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-08T21:15:41.113 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.113 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.113 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0", 2024-03-08T21:15:41.113 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.113 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.113 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.113 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.113 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.113 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.113 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.113 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.113 INFO:teuthology.orchestra.run.smithi107.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-03-08T21:15:41.114 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-03-08T21:15:41.114 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-08T21:15:41.114 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.114 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.114 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "10", 2024-03-08T21:15:41.114 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.114 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.114 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.114 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.114 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.114 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.114 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.114 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.114 INFO:teuthology.orchestra.run.smithi107.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-03-08T21:15:41.114 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-03-08T21:15:41.114 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-08T21:15:41.115 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.115 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.115 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "5", 2024-03-08T21:15:41.115 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.115 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.115 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.115 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.115 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.115 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.115 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.115 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.115 INFO:teuthology.orchestra.run.smithi107.stdout: "cross_origin_url": { 2024-03-08T21:15:41.115 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "cross_origin_url", 2024-03-08T21:15:41.115 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.115 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.115 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.116 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:41.116 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.116 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.116 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.116 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.116 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.116 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.116 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.116 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.116 INFO:teuthology.orchestra.run.smithi107.stdout: "crt_file": { 2024-03-08T21:15:41.116 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "crt_file", 2024-03-08T21:15:41.116 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.116 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.116 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.116 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:41.117 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.117 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.117 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.117 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.117 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.117 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.117 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.117 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.117 INFO:teuthology.orchestra.run.smithi107.stdout: "debug": { 2024-03-08T21:15:41.117 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "debug", 2024-03-08T21:15:41.117 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.117 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.117 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.117 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:41.117 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.117 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.118 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.118 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Enable/disable debug options", 2024-03-08T21:15:41.118 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.118 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.118 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.118 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.118 INFO:teuthology.orchestra.run.smithi107.stdout: "jwt_token_ttl": { 2024-03-08T21:15:41.118 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "jwt_token_ttl", 2024-03-08T21:15:41.118 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-08T21:15:41.118 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.118 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.118 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "28800", 2024-03-08T21:15:41.118 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.118 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.118 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.119 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.119 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.119 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.119 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.119 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.119 INFO:teuthology.orchestra.run.smithi107.stdout: "key_file": { 2024-03-08T21:15:41.119 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "key_file", 2024-03-08T21:15:41.119 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.119 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.119 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.119 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:41.119 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.119 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.119 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.119 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.120 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.120 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.120 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.120 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.120 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-03-08T21:15:41.120 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-03-08T21:15:41.120 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.120 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.120 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.120 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:41.120 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.120 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.120 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:41.121 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-08T21:15:41.121 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-08T21:15:41.121 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-08T21:15:41.121 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-08T21:15:41.121 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-08T21:15:41.121 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-08T21:15:41.121 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:41.121 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.121 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.121 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.121 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.121 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.121 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-03-08T21:15:41.121 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-03-08T21:15:41.121 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.122 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.122 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.122 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:41.122 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.122 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.122 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.122 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.122 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.122 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.122 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.122 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.122 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-03-08T21:15:41.122 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-03-08T21:15:41.123 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.123 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.123 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.123 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-03-08T21:15:41.123 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.123 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.123 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:41.123 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-08T21:15:41.123 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-08T21:15:41.123 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-08T21:15:41.123 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-08T21:15:41.123 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-08T21:15:41.123 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-08T21:15:41.123 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:41.123 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.124 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.124 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.124 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.124 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.124 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-03-08T21:15:41.124 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-03-08T21:15:41.124 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.124 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.124 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.124 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:41.124 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.124 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.124 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.124 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.124 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.125 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.125 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.125 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.125 INFO:teuthology.orchestra.run.smithi107.stdout: "motd": { 2024-03-08T21:15:41.125 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "motd", 2024-03-08T21:15:41.125 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.125 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.125 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.125 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:41.125 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.125 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.125 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.126 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "The message of the day", 2024-03-08T21:15:41.126 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.126 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.126 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.126 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.126 INFO:teuthology.orchestra.run.smithi107.stdout: "redirect_resolve_ip_addr": { 2024-03-08T21:15:41.126 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "redirect_resolve_ip_addr", 2024-03-08T21:15:41.126 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.126 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.126 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.126 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:41.126 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.126 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.126 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.127 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.127 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.127 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.127 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.127 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.127 INFO:teuthology.orchestra.run.smithi107.stdout: "server_addr": { 2024-03-08T21:15:41.127 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "server_addr", 2024-03-08T21:15:41.127 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.127 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.127 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.127 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "::", 2024-03-08T21:15:41.127 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.127 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.127 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.127 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.128 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.128 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.128 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.128 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.128 INFO:teuthology.orchestra.run.smithi107.stdout: "server_port": { 2024-03-08T21:15:41.128 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "server_port", 2024-03-08T21:15:41.128 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-08T21:15:41.128 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.128 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.128 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "8080", 2024-03-08T21:15:41.128 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.128 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.128 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.128 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.128 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.129 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.129 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.129 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.129 INFO:teuthology.orchestra.run.smithi107.stdout: "ssl": { 2024-03-08T21:15:41.129 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "ssl", 2024-03-08T21:15:41.129 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.129 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.129 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.129 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-08T21:15:41.129 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.129 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.129 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.129 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.129 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.130 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.130 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.130 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.130 INFO:teuthology.orchestra.run.smithi107.stdout: "ssl_server_port": { 2024-03-08T21:15:41.130 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "ssl_server_port", 2024-03-08T21:15:41.130 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-08T21:15:41.130 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.130 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.130 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "8443", 2024-03-08T21:15:41.130 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.130 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.130 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.130 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.130 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.131 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.131 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.131 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.131 INFO:teuthology.orchestra.run.smithi107.stdout: "standby_behaviour": { 2024-03-08T21:15:41.131 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "standby_behaviour", 2024-03-08T21:15:41.131 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.131 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.131 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.131 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "redirect", 2024-03-08T21:15:41.131 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.131 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.131 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:41.131 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-08T21:15:41.131 INFO:teuthology.orchestra.run.smithi107.stdout: "redirect" 2024-03-08T21:15:41.131 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:41.132 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.132 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.132 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.132 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.132 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.132 INFO:teuthology.orchestra.run.smithi107.stdout: "standby_error_status_code": { 2024-03-08T21:15:41.132 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "standby_error_status_code", 2024-03-08T21:15:41.132 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-08T21:15:41.132 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.132 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.132 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "500", 2024-03-08T21:15:41.132 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "400", 2024-03-08T21:15:41.132 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "599", 2024-03-08T21:15:41.132 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.132 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.133 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.133 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.133 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.133 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.133 INFO:teuthology.orchestra.run.smithi107.stdout: "url_prefix": { 2024-03-08T21:15:41.133 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "url_prefix", 2024-03-08T21:15:41.133 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.133 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.133 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.133 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:41.133 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.133 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.133 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.133 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.133 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.134 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.134 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.134 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:41.134 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:41.134 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.134 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-08T21:15:41.134 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "devicehealth", 2024-03-08T21:15:41.134 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-03-08T21:15:41.134 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-03-08T21:15:41.134 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-03-08T21:15:41.134 INFO:teuthology.orchestra.run.smithi107.stdout: "enable_monitoring": { 2024-03-08T21:15:41.134 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "enable_monitoring", 2024-03-08T21:15:41.134 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.134 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.135 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.135 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-08T21:15:41.135 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.135 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.135 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.135 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "monitor device health metrics", 2024-03-08T21:15:41.135 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.135 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.135 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.135 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.135 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-03-08T21:15:41.135 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-03-08T21:15:41.135 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.135 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.135 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.136 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:41.136 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.136 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.136 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:41.136 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-08T21:15:41.136 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-08T21:15:41.136 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-08T21:15:41.136 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-08T21:15:41.136 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-08T21:15:41.136 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-08T21:15:41.136 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:41.136 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.136 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.136 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.136 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.137 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.137 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-03-08T21:15:41.137 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-03-08T21:15:41.137 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.137 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.137 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.137 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:41.137 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.137 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.137 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.137 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.137 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.137 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.137 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.137 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.137 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-03-08T21:15:41.138 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-03-08T21:15:41.138 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.138 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.138 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.138 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-03-08T21:15:41.138 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.138 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.138 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:41.138 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-08T21:15:41.138 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-08T21:15:41.138 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-08T21:15:41.138 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-08T21:15:41.138 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-08T21:15:41.138 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-08T21:15:41.138 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:41.139 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.139 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.139 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.139 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.139 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.139 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-03-08T21:15:41.139 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-03-08T21:15:41.139 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.139 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.139 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.139 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:41.139 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.139 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.139 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.139 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.140 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.140 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.140 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.140 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.140 INFO:teuthology.orchestra.run.smithi107.stdout: "mark_out_threshold": { 2024-03-08T21:15:41.140 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "mark_out_threshold", 2024-03-08T21:15:41.140 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-03-08T21:15:41.140 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.140 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.140 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "2419200", 2024-03-08T21:15:41.140 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.140 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.140 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.140 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-03-08T21:15:41.140 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.141 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.141 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.141 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.141 INFO:teuthology.orchestra.run.smithi107.stdout: "pool_name": { 2024-03-08T21:15:41.141 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "pool_name", 2024-03-08T21:15:41.141 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.141 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.141 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.141 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "device_health_metrics", 2024-03-08T21:15:41.141 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.141 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.141 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.141 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "name of pool in which to store device health metrics", 2024-03-08T21:15:41.141 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.141 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.141 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.142 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.142 INFO:teuthology.orchestra.run.smithi107.stdout: "retention_period": { 2024-03-08T21:15:41.142 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "retention_period", 2024-03-08T21:15:41.142 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-03-08T21:15:41.142 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.142 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.142 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "15552000", 2024-03-08T21:15:41.142 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.142 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.142 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.142 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "how long to retain device health metrics", 2024-03-08T21:15:41.142 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.142 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.142 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.142 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.143 INFO:teuthology.orchestra.run.smithi107.stdout: "scrape_frequency": { 2024-03-08T21:15:41.143 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "scrape_frequency", 2024-03-08T21:15:41.143 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-03-08T21:15:41.143 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.143 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.143 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "86400", 2024-03-08T21:15:41.143 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.143 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.143 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.143 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "how frequently to scrape device health metrics", 2024-03-08T21:15:41.143 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.143 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.143 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.143 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.143 INFO:teuthology.orchestra.run.smithi107.stdout: "self_heal": { 2024-03-08T21:15:41.143 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "self_heal", 2024-03-08T21:15:41.144 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.144 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.144 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.144 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-08T21:15:41.144 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.144 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.144 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.144 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-03-08T21:15:41.144 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.144 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.145 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:40.403+0000 7f2746200640 1 -- 172.21.15.107:0/492498522 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f27280a29f0 msgr2=0x7f27280a4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:15:41.145 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:40.403+0000 7f2746200640 1 --2- 172.21.15.107:0/492498522 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f27280a29f0 0x7f27280a4eb0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f273400baa0 tx=0x7f2734005d90 comp rx=0 tx=0).stop 2024-03-08T21:15:41.145 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:40.403+0000 7f2746200640 1 -- 172.21.15.107:0/492498522 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f27401063f0 msgr2=0x7f2740073b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:15:41.145 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:40.403+0000 7f2746200640 1 --2- 172.21.15.107:0/492498522 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f27401063f0 0x7f2740073b30 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f272c00d740 tx=0x7f272c00dc00 comp rx=0 tx=0).stop 2024-03-08T21:15:41.145 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:40.403+0000 7f2746200640 1 -- 172.21.15.107:0/492498522 shutdown_connections 2024-03-08T21:15:41.145 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:40.403+0000 7f2746200640 1 --2- 172.21.15.107:0/492498522 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f27280a29f0 0x7f27280a4eb0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:15:41.145 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:40.403+0000 7f2746200640 1 --2- 172.21.15.107:0/492498522 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2740074070 0x7f27401a5ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:15:41.145 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:40.403+0000 7f2746200640 1 --2- 172.21.15.107:0/492498522 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f27401063f0 0x7f2740073b30 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:15:41.145 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:40.403+0000 7f2746200640 1 --2- 172.21.15.107:0/492498522 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f2740101320 0x7f27400735f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:15:41.146 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:40.403+0000 7f2746200640 1 -- 172.21.15.107:0/492498522 >> 172.21.15.107:0/492498522 conn(0x7f27400fcbc0 msgr2=0x7f27400fef10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:15:41.146 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:40.403+0000 7f2746200640 1 -- 172.21.15.107:0/492498522 shutdown_connections 2024-03-08T21:15:41.146 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:15:40.403+0000 7f2746200640 1 -- 172.21.15.107:0/492498522 wait complete. 2024-03-08T21:15:41.150 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:15:40 smithi149 ceph-mon[35042]: pgmap v480: 97 pgs: 97 active+clean; 17 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 1022 KiB/s rd, 1.1 MiB/s wr, 210 op/s 2024-03-08T21:15:41.150 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:15:40 smithi149 ceph-mon[35042]: from='client.? 172.21.15.107:0/3323855843' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-08T21:15:41.150 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:15:40 smithi149 ceph-mon[35042]: from='client.? 172.21.15.107:0/492498522' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-03-08T21:15:41.150 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:15:40 smithi204 ceph-mon[34935]: pgmap v480: 97 pgs: 97 active+clean; 17 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 1022 KiB/s rd, 1.1 MiB/s wr, 210 op/s 2024-03-08T21:15:41.150 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:15:40 smithi204 ceph-mon[34935]: from='client.? 172.21.15.107:0/3323855843' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-08T21:15:41.151 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:15:40 smithi204 ceph-mon[34935]: from='client.? 172.21.15.107:0/492498522' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-03-08T21:15:41.154 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.154 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.154 INFO:teuthology.orchestra.run.smithi107.stdout: "sleep_interval": { 2024-03-08T21:15:41.154 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "sleep_interval", 2024-03-08T21:15:41.154 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-03-08T21:15:41.154 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.154 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.154 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "600", 2024-03-08T21:15:41.154 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.154 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.154 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.154 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "how frequently to wake up and check device health", 2024-03-08T21:15:41.155 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.155 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.155 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.155 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.155 INFO:teuthology.orchestra.run.smithi107.stdout: "warn_threshold": { 2024-03-08T21:15:41.155 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "warn_threshold", 2024-03-08T21:15:41.155 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-03-08T21:15:41.155 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.155 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.155 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "7257600", 2024-03-08T21:15:41.155 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.155 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.155 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.155 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-03-08T21:15:41.156 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.156 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.156 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.156 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:41.156 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:41.156 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.156 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-08T21:15:41.156 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "diskprediction_local", 2024-03-08T21:15:41.156 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-03-08T21:15:41.156 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-03-08T21:15:41.156 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-03-08T21:15:41.156 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-03-08T21:15:41.156 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-03-08T21:15:41.156 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.157 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.157 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.157 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:41.157 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.157 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.157 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:41.157 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-08T21:15:41.157 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-08T21:15:41.157 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-08T21:15:41.157 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-08T21:15:41.157 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-08T21:15:41.157 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-08T21:15:41.157 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:41.157 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.157 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.158 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.158 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.158 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.158 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-03-08T21:15:41.158 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-03-08T21:15:41.158 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.158 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.158 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.158 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:41.158 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.158 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.158 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.158 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.159 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.159 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.159 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.159 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.159 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-03-08T21:15:41.159 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-03-08T21:15:41.159 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.159 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.159 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.159 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-03-08T21:15:41.159 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.159 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.159 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:41.159 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-08T21:15:41.160 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-08T21:15:41.160 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-08T21:15:41.160 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-08T21:15:41.160 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-08T21:15:41.160 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-08T21:15:41.160 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:41.160 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.160 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.160 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.160 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.160 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.160 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-03-08T21:15:41.160 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-03-08T21:15:41.161 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.161 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.161 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.161 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:41.161 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.161 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.161 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.161 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.161 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.161 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.161 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.161 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.161 INFO:teuthology.orchestra.run.smithi107.stdout: "predict_interval": { 2024-03-08T21:15:41.161 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "predict_interval", 2024-03-08T21:15:41.162 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.162 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.162 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.162 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "86400", 2024-03-08T21:15:41.162 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.162 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.162 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.162 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.162 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.162 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.162 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.162 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.162 INFO:teuthology.orchestra.run.smithi107.stdout: "predictor_model": { 2024-03-08T21:15:41.162 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "predictor_model", 2024-03-08T21:15:41.163 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.163 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.163 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.163 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "prophetstor", 2024-03-08T21:15:41.163 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.163 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.163 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.163 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.163 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.163 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.163 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.163 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.163 INFO:teuthology.orchestra.run.smithi107.stdout: "sleep_interval": { 2024-03-08T21:15:41.163 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "sleep_interval", 2024-03-08T21:15:41.163 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.164 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.164 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.164 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "600", 2024-03-08T21:15:41.164 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.164 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.164 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.164 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.164 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.164 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.164 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.164 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:41.164 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:41.164 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.165 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-08T21:15:41.165 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "influx", 2024-03-08T21:15:41.165 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": false, 2024-03-08T21:15:41.165 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "influxdb python module not found", 2024-03-08T21:15:41.165 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-03-08T21:15:41.165 INFO:teuthology.orchestra.run.smithi107.stdout: "batch_size": { 2024-03-08T21:15:41.165 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "batch_size", 2024-03-08T21:15:41.165 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-08T21:15:41.165 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.165 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.165 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "5000", 2024-03-08T21:15:41.165 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.165 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.165 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.165 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-03-08T21:15:41.166 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.166 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.166 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.166 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.166 INFO:teuthology.orchestra.run.smithi107.stdout: "database": { 2024-03-08T21:15:41.166 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "database", 2024-03-08T21:15:41.166 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.166 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.166 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.166 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "ceph", 2024-03-08T21:15:41.166 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.166 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.166 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.166 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-03-08T21:15:41.167 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.167 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.167 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.167 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.167 INFO:teuthology.orchestra.run.smithi107.stdout: "hostname": { 2024-03-08T21:15:41.167 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "hostname", 2024-03-08T21:15:41.167 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.167 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.167 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.167 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:41.167 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.167 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.167 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.168 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "InfluxDB server hostname", 2024-03-08T21:15:41.168 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.168 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.168 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.168 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.168 INFO:teuthology.orchestra.run.smithi107.stdout: "interval": { 2024-03-08T21:15:41.168 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "interval", 2024-03-08T21:15:41.168 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-03-08T21:15:41.168 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.168 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.168 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "30", 2024-03-08T21:15:41.168 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "5", 2024-03-08T21:15:41.168 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.168 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.169 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-03-08T21:15:41.169 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.169 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.169 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.169 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.169 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-03-08T21:15:41.169 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-03-08T21:15:41.169 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.169 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.169 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.169 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:41.169 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.169 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.169 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:41.170 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-08T21:15:41.170 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-08T21:15:41.170 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-08T21:15:41.170 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-08T21:15:41.170 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-08T21:15:41.170 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-08T21:15:41.170 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:41.170 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.170 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.170 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.170 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.170 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.170 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-03-08T21:15:41.171 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-03-08T21:15:41.171 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.171 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.171 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.171 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:41.171 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.171 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.171 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.171 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.171 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.171 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.171 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.171 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.172 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-03-08T21:15:41.172 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-03-08T21:15:41.172 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.172 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.172 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.172 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-03-08T21:15:41.172 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.172 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.172 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:41.172 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-08T21:15:41.172 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-08T21:15:41.172 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-08T21:15:41.172 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-08T21:15:41.173 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-08T21:15:41.173 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-08T21:15:41.173 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:41.173 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.173 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.173 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.173 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.173 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.173 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-03-08T21:15:41.173 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-03-08T21:15:41.173 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.173 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.173 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.173 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:41.174 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.174 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.174 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.174 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.174 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.174 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.174 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.174 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.174 INFO:teuthology.orchestra.run.smithi107.stdout: "password": { 2024-03-08T21:15:41.174 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "password", 2024-03-08T21:15:41.174 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.174 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.174 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.175 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:41.175 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.175 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.175 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.175 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "password of InfluxDB server user", 2024-03-08T21:15:41.175 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.175 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.175 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.175 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.175 INFO:teuthology.orchestra.run.smithi107.stdout: "port": { 2024-03-08T21:15:41.175 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "port", 2024-03-08T21:15:41.175 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-08T21:15:41.175 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.175 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.176 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "8086", 2024-03-08T21:15:41.176 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.176 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.176 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.176 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "InfluxDB server port", 2024-03-08T21:15:41.176 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.176 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.176 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.176 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.176 INFO:teuthology.orchestra.run.smithi107.stdout: "ssl": { 2024-03-08T21:15:41.176 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "ssl", 2024-03-08T21:15:41.176 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.176 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.176 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.176 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "false", 2024-03-08T21:15:41.177 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.177 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.177 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.177 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-03-08T21:15:41.177 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.177 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.177 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.177 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.177 INFO:teuthology.orchestra.run.smithi107.stdout: "threads": { 2024-03-08T21:15:41.177 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "threads", 2024-03-08T21:15:41.177 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-08T21:15:41.177 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.177 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.177 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "5", 2024-03-08T21:15:41.178 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "1", 2024-03-08T21:15:41.178 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "32", 2024-03-08T21:15:41.178 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.178 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-03-08T21:15:41.178 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.178 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.178 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.178 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.178 INFO:teuthology.orchestra.run.smithi107.stdout: "username": { 2024-03-08T21:15:41.178 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "username", 2024-03-08T21:15:41.178 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.178 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.178 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.178 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:41.179 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.179 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.179 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.179 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "username of InfluxDB server user", 2024-03-08T21:15:41.179 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.179 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.179 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.179 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.179 INFO:teuthology.orchestra.run.smithi107.stdout: "verify_ssl": { 2024-03-08T21:15:41.179 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "verify_ssl", 2024-03-08T21:15:41.179 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.179 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.179 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.179 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "true", 2024-03-08T21:15:41.180 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.180 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.180 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.180 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-03-08T21:15:41.180 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.180 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.180 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.180 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:41.180 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:41.180 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.180 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-08T21:15:41.180 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "insights", 2024-03-08T21:15:41.180 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-03-08T21:15:41.180 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-03-08T21:15:41.181 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-03-08T21:15:41.181 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-03-08T21:15:41.181 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-03-08T21:15:41.181 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.181 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.181 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.181 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:41.181 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.181 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.181 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:41.181 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-08T21:15:41.181 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-08T21:15:41.182 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-08T21:15:41.182 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-08T21:15:41.182 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-08T21:15:41.182 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-08T21:15:41.182 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:41.182 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.182 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.182 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.182 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.182 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.182 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-03-08T21:15:41.182 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-03-08T21:15:41.182 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.183 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.183 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.183 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:41.183 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.183 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.183 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.183 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.183 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.183 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.183 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.183 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.183 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-03-08T21:15:41.183 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-03-08T21:15:41.183 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.184 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.184 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.184 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-03-08T21:15:41.184 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.184 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.184 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:41.184 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-08T21:15:41.184 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-08T21:15:41.184 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-08T21:15:41.184 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-08T21:15:41.184 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-08T21:15:41.184 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-08T21:15:41.184 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:41.185 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.185 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.185 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.185 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.185 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.185 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-03-08T21:15:41.185 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-03-08T21:15:41.185 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.185 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.185 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.185 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:41.185 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.185 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.185 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.186 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.186 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.186 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.186 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.186 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:41.186 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:41.186 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.186 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-08T21:15:41.186 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "iostat", 2024-03-08T21:15:41.186 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-03-08T21:15:41.186 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-03-08T21:15:41.186 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-03-08T21:15:41.186 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-03-08T21:15:41.187 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-03-08T21:15:41.187 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.187 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.187 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.187 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:41.187 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.187 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.187 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:41.187 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-08T21:15:41.187 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-08T21:15:41.187 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-08T21:15:41.187 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-08T21:15:41.187 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-08T21:15:41.187 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-08T21:15:41.187 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:41.188 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.188 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.188 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.188 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.188 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.188 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-03-08T21:15:41.188 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-03-08T21:15:41.188 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.188 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.188 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.188 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:41.188 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.188 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.188 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.188 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.188 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.189 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.189 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.189 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.189 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-03-08T21:15:41.189 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-03-08T21:15:41.189 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.189 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.189 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.189 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-03-08T21:15:41.189 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.189 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.189 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:41.189 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-08T21:15:41.189 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-08T21:15:41.189 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-08T21:15:41.189 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-08T21:15:41.190 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-08T21:15:41.190 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-08T21:15:41.190 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:41.190 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.190 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.190 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.190 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.190 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.190 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-03-08T21:15:41.190 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-03-08T21:15:41.190 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.190 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.190 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.190 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:41.190 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.191 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.191 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.191 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.191 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.191 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.191 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.191 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:41.191 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:41.191 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.191 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-08T21:15:41.191 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "k8sevents", 2024-03-08T21:15:41.191 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-03-08T21:15:41.191 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-03-08T21:15:41.191 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-03-08T21:15:41.191 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph_event_retention_days": { 2024-03-08T21:15:41.191 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "ceph_event_retention_days", 2024-03-08T21:15:41.192 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-08T21:15:41.192 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.192 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.192 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "7", 2024-03-08T21:15:41.192 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.192 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.192 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.192 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Days to hold ceph event information within local cache", 2024-03-08T21:15:41.192 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.192 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.192 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.192 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.192 INFO:teuthology.orchestra.run.smithi107.stdout: "config_check_secs": { 2024-03-08T21:15:41.192 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "config_check_secs", 2024-03-08T21:15:41.192 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-08T21:15:41.192 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.193 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.193 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "10", 2024-03-08T21:15:41.193 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "10", 2024-03-08T21:15:41.193 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.193 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.193 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-03-08T21:15:41.193 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.193 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.193 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.193 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.193 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-03-08T21:15:41.193 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-03-08T21:15:41.193 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.193 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.194 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.194 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:41.194 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.194 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.194 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:41.194 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-08T21:15:41.194 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-08T21:15:41.194 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-08T21:15:41.194 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-08T21:15:41.194 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-08T21:15:41.194 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-08T21:15:41.194 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:41.194 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.195 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.195 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.195 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.195 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.195 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-03-08T21:15:41.195 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-03-08T21:15:41.195 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.195 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.195 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.195 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:41.195 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.195 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.195 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.195 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.196 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.196 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.196 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.196 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.196 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-03-08T21:15:41.196 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-03-08T21:15:41.196 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.196 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.196 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.196 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-03-08T21:15:41.196 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.196 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.196 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:41.197 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-08T21:15:41.197 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-08T21:15:41.197 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-08T21:15:41.197 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-08T21:15:41.197 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-08T21:15:41.197 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-08T21:15:41.197 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:41.197 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.197 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.197 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.197 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.197 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.197 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-03-08T21:15:41.198 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-03-08T21:15:41.198 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.198 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.198 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.198 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:41.198 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.198 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.198 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.198 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.198 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.198 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.198 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.198 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:41.198 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:41.199 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.199 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-08T21:15:41.199 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "localpool", 2024-03-08T21:15:41.199 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-03-08T21:15:41.199 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-03-08T21:15:41.199 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-03-08T21:15:41.199 INFO:teuthology.orchestra.run.smithi107.stdout: "failure_domain": { 2024-03-08T21:15:41.199 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "failure_domain", 2024-03-08T21:15:41.199 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.199 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.199 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.199 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "host", 2024-03-08T21:15:41.199 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.199 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.200 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.200 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "failure domain for any created local pool", 2024-03-08T21:15:41.200 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-03-08T21:15:41.200 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.200 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.200 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.200 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-03-08T21:15:41.200 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-03-08T21:15:41.200 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.200 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.200 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.200 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:41.200 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.200 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.201 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:41.201 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-08T21:15:41.201 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-08T21:15:41.201 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-08T21:15:41.201 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-08T21:15:41.201 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-08T21:15:41.201 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-08T21:15:41.201 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:41.201 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.201 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.201 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.201 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.201 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.201 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-03-08T21:15:41.202 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-03-08T21:15:41.202 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.202 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.202 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.202 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:41.202 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.202 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.202 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.202 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.202 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.202 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.202 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.202 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.202 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-03-08T21:15:41.203 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-03-08T21:15:41.203 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.203 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.203 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.203 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-03-08T21:15:41.203 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.203 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.203 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:41.203 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-08T21:15:41.203 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-08T21:15:41.203 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-08T21:15:41.203 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-08T21:15:41.203 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-08T21:15:41.203 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-08T21:15:41.203 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:41.204 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.204 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.204 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.204 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.204 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.204 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-03-08T21:15:41.204 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-03-08T21:15:41.204 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.204 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.204 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.204 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:41.204 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.204 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.204 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.205 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.205 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.205 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.205 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.205 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.205 INFO:teuthology.orchestra.run.smithi107.stdout: "min_size": { 2024-03-08T21:15:41.205 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "min_size", 2024-03-08T21:15:41.205 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-08T21:15:41.205 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.205 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.205 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:41.205 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.205 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.205 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.205 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "default min_size for any created local pool", 2024-03-08T21:15:41.206 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-03-08T21:15:41.206 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.206 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.206 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.206 INFO:teuthology.orchestra.run.smithi107.stdout: "num_rep": { 2024-03-08T21:15:41.206 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "num_rep", 2024-03-08T21:15:41.206 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-08T21:15:41.206 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.206 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.206 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "3", 2024-03-08T21:15:41.206 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.206 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.206 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.206 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "default replica count for any created local pool", 2024-03-08T21:15:41.207 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.207 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.207 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.207 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.207 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_num": { 2024-03-08T21:15:41.207 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "pg_num", 2024-03-08T21:15:41.207 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-08T21:15:41.207 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.207 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.207 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "128", 2024-03-08T21:15:41.207 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.207 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.207 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.207 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "default pg_num for any created local pool", 2024-03-08T21:15:41.208 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.208 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.208 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.208 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.208 INFO:teuthology.orchestra.run.smithi107.stdout: "prefix": { 2024-03-08T21:15:41.208 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "prefix", 2024-03-08T21:15:41.208 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.208 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.208 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.208 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:41.208 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.208 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.209 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.209 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "name prefix for any created local pool", 2024-03-08T21:15:41.209 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.209 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.209 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.209 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.209 INFO:teuthology.orchestra.run.smithi107.stdout: "subtree": { 2024-03-08T21:15:41.209 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "subtree", 2024-03-08T21:15:41.209 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.209 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.209 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.209 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "rack", 2024-03-08T21:15:41.209 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.210 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.210 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.210 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "CRUSH level for which to create a local pool", 2024-03-08T21:15:41.210 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-03-08T21:15:41.210 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.210 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.210 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:41.210 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:41.210 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.210 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-08T21:15:41.210 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "mds_autoscaler", 2024-03-08T21:15:41.210 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-03-08T21:15:41.210 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-03-08T21:15:41.211 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-03-08T21:15:41.211 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-03-08T21:15:41.211 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-03-08T21:15:41.211 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.211 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.211 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.211 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:41.211 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.211 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.211 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:41.211 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-08T21:15:41.211 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-08T21:15:41.211 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-08T21:15:41.211 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-08T21:15:41.212 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-08T21:15:41.212 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-08T21:15:41.212 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:41.212 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.212 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.212 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.212 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.212 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.212 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-03-08T21:15:41.212 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-03-08T21:15:41.212 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.212 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.212 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.213 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:41.213 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.213 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.213 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.213 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.213 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.213 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.213 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.213 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.213 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-03-08T21:15:41.213 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-03-08T21:15:41.213 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.213 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.214 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.214 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-03-08T21:15:41.214 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.214 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.214 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:41.214 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-08T21:15:41.214 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-08T21:15:41.214 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-08T21:15:41.214 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-08T21:15:41.214 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-08T21:15:41.214 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-08T21:15:41.214 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:41.214 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.214 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.215 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.215 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.215 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.215 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-03-08T21:15:41.215 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-03-08T21:15:41.215 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.215 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.215 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.215 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:41.215 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.215 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.215 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.215 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.216 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.216 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.216 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.216 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:41.216 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:41.216 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.216 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-08T21:15:41.216 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "mirroring", 2024-03-08T21:15:41.216 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-03-08T21:15:41.216 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-03-08T21:15:41.216 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-03-08T21:15:41.216 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-03-08T21:15:41.216 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-03-08T21:15:41.216 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.217 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.217 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.217 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:41.217 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.217 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.217 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:41.217 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-08T21:15:41.217 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-08T21:15:41.217 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-08T21:15:41.217 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-08T21:15:41.217 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-08T21:15:41.217 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-08T21:15:41.217 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:41.218 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.218 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.218 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.218 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.218 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.218 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-03-08T21:15:41.218 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-03-08T21:15:41.218 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.218 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.218 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.218 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:41.218 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.218 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.219 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.219 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.219 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.219 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.219 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.219 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.219 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-03-08T21:15:41.219 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-03-08T21:15:41.219 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.219 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.219 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.219 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-03-08T21:15:41.220 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.220 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.220 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:41.220 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-08T21:15:41.220 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-08T21:15:41.220 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-08T21:15:41.220 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-08T21:15:41.220 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-08T21:15:41.220 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-08T21:15:41.220 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:41.220 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.220 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.220 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.220 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.221 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.221 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-03-08T21:15:41.221 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-03-08T21:15:41.221 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.221 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.221 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.221 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:41.221 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.221 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.221 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.221 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.221 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.221 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.221 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.222 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:41.222 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:41.222 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.222 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-08T21:15:41.222 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "nfs", 2024-03-08T21:15:41.222 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-03-08T21:15:41.222 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-03-08T21:15:41.222 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-03-08T21:15:41.222 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-03-08T21:15:41.222 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-03-08T21:15:41.222 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.222 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.222 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.223 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:41.223 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.223 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.223 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:41.223 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-08T21:15:41.223 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-08T21:15:41.223 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-08T21:15:41.223 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-08T21:15:41.223 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-08T21:15:41.223 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-08T21:15:41.223 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:41.223 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.224 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.224 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.224 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.224 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.224 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-03-08T21:15:41.224 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-03-08T21:15:41.224 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.224 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.224 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.224 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:41.224 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.224 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.224 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.225 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.225 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.225 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.225 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.225 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.225 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-03-08T21:15:41.225 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-03-08T21:15:41.225 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.225 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.225 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.225 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-03-08T21:15:41.225 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.225 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.226 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:41.226 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-08T21:15:41.226 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-08T21:15:41.226 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-08T21:15:41.226 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-08T21:15:41.226 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-08T21:15:41.226 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-08T21:15:41.226 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:41.226 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.226 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.226 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.226 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.227 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.227 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-03-08T21:15:41.227 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-03-08T21:15:41.227 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.227 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.227 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.227 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:41.227 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.227 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.227 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.227 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.227 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.227 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.227 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.228 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:41.228 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:41.228 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.228 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-08T21:15:41.228 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "orchestrator", 2024-03-08T21:15:41.228 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-03-08T21:15:41.228 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-03-08T21:15:41.228 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-03-08T21:15:41.228 INFO:teuthology.orchestra.run.smithi107.stdout: "fail_fs": { 2024-03-08T21:15:41.228 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "fail_fs", 2024-03-08T21:15:41.228 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.228 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.228 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.229 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:41.229 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.229 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.229 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.229 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-03-08T21:15:41.229 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.229 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.229 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.229 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.229 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-03-08T21:15:41.229 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-03-08T21:15:41.229 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.229 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.230 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.230 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:41.230 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.230 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.230 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:41.230 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-08T21:15:41.230 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-08T21:15:41.230 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-08T21:15:41.230 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-08T21:15:41.230 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-08T21:15:41.230 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-08T21:15:41.230 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:41.230 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.231 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.231 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.231 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.231 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.231 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-03-08T21:15:41.231 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-03-08T21:15:41.231 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.231 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.231 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.231 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:41.231 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.231 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.231 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.231 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.232 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.232 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.232 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.232 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.232 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-03-08T21:15:41.232 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-03-08T21:15:41.232 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.232 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.232 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.232 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-03-08T21:15:41.232 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.232 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.233 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:41.233 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-08T21:15:41.233 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-08T21:15:41.233 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-08T21:15:41.233 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-08T21:15:41.233 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-08T21:15:41.233 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-08T21:15:41.233 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:41.233 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.233 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.233 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.233 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.233 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.233 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-03-08T21:15:41.234 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-03-08T21:15:41.234 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.234 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.234 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.234 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:41.234 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.234 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.234 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.234 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.234 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.234 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.234 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.235 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.235 INFO:teuthology.orchestra.run.smithi107.stdout: "orchestrator": { 2024-03-08T21:15:41.235 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "orchestrator", 2024-03-08T21:15:41.235 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.235 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.235 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.235 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:41.235 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.235 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.235 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:41.235 INFO:teuthology.orchestra.run.smithi107.stdout: "cephadm", 2024-03-08T21:15:41.235 INFO:teuthology.orchestra.run.smithi107.stdout: "rook", 2024-03-08T21:15:41.235 INFO:teuthology.orchestra.run.smithi107.stdout: "test_orchestrator" 2024-03-08T21:15:41.235 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:41.236 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Orchestrator backend", 2024-03-08T21:15:41.236 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.236 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.236 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.236 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:41.236 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:41.236 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.236 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-08T21:15:41.236 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "osd_perf_query", 2024-03-08T21:15:41.236 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-03-08T21:15:41.236 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-03-08T21:15:41.236 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-03-08T21:15:41.236 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-03-08T21:15:41.236 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-03-08T21:15:41.237 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.237 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.237 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.237 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:41.237 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.237 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.237 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:41.237 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-08T21:15:41.237 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-08T21:15:41.237 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-08T21:15:41.237 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-08T21:15:41.237 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-08T21:15:41.237 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-08T21:15:41.238 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:41.238 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.238 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.238 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.238 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.238 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.238 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-03-08T21:15:41.238 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-03-08T21:15:41.238 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.238 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.238 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.238 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:41.239 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.239 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.239 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.239 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.239 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.239 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.239 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.239 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.239 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-03-08T21:15:41.239 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-03-08T21:15:41.239 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.239 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.240 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.240 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-03-08T21:15:41.240 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.240 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.240 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:41.240 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-08T21:15:41.240 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-08T21:15:41.240 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-08T21:15:41.240 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-08T21:15:41.240 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-08T21:15:41.240 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-08T21:15:41.240 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:41.241 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.241 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.241 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.241 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.241 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.241 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-03-08T21:15:41.241 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-03-08T21:15:41.241 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.241 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.241 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.241 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:41.241 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.242 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.242 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.242 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.242 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.242 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.242 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.242 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:41.242 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:41.242 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.242 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-08T21:15:41.242 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "osd_support", 2024-03-08T21:15:41.242 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-03-08T21:15:41.242 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-03-08T21:15:41.243 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-03-08T21:15:41.243 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-03-08T21:15:41.243 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-03-08T21:15:41.243 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.243 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.243 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.243 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:41.243 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.243 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.243 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:41.243 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-08T21:15:41.243 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-08T21:15:41.243 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-08T21:15:41.243 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-08T21:15:41.244 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-08T21:15:41.244 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-08T21:15:41.244 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:41.244 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.244 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.244 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.244 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.244 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.244 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-03-08T21:15:41.244 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-03-08T21:15:41.244 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.244 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.244 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.245 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:41.245 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.245 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.245 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.245 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.245 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.245 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.245 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.245 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.245 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-03-08T21:15:41.245 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-03-08T21:15:41.245 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.245 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.246 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.246 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-03-08T21:15:41.246 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.246 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.246 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:41.246 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-08T21:15:41.246 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-08T21:15:41.246 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-08T21:15:41.246 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-08T21:15:41.246 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-08T21:15:41.246 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-08T21:15:41.246 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:41.246 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.247 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.247 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.247 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.247 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.247 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-03-08T21:15:41.247 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-03-08T21:15:41.247 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.247 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.247 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.247 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:41.247 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.247 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.247 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.247 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.247 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.248 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.248 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.248 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:41.248 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:41.248 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.248 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-08T21:15:41.248 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "pg_autoscaler", 2024-03-08T21:15:41.248 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-03-08T21:15:41.248 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-03-08T21:15:41.248 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-03-08T21:15:41.248 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-03-08T21:15:41.248 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-03-08T21:15:41.248 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.248 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.248 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.249 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:41.249 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.249 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.249 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:41.249 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-08T21:15:41.249 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-08T21:15:41.249 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-08T21:15:41.249 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-08T21:15:41.249 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-08T21:15:41.249 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-08T21:15:41.249 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:41.249 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.249 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.249 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.250 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.250 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.250 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-03-08T21:15:41.250 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-03-08T21:15:41.250 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.250 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.250 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.250 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:41.250 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.250 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.250 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.250 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.250 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.250 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.251 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.251 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.251 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-03-08T21:15:41.251 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-03-08T21:15:41.251 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.251 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.251 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.251 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-03-08T21:15:41.251 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.251 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.251 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:41.251 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-08T21:15:41.251 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-08T21:15:41.251 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-08T21:15:41.252 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-08T21:15:41.252 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-08T21:15:41.252 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-08T21:15:41.252 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:41.252 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.252 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.252 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.252 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.252 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.252 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-03-08T21:15:41.252 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-03-08T21:15:41.252 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.252 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.252 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.252 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:41.253 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.253 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.253 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.253 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.253 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.253 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.253 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.253 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.253 INFO:teuthology.orchestra.run.smithi107.stdout: "sleep_interval": { 2024-03-08T21:15:41.253 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "sleep_interval", 2024-03-08T21:15:41.253 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-03-08T21:15:41.253 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.253 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.253 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "60", 2024-03-08T21:15:41.254 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.254 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.254 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.254 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.254 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.254 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.254 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.254 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.254 INFO:teuthology.orchestra.run.smithi107.stdout: "threshold": { 2024-03-08T21:15:41.254 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "threshold", 2024-03-08T21:15:41.254 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "float", 2024-03-08T21:15:41.254 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.254 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.254 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "3.0", 2024-03-08T21:15:41.255 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "1.0", 2024-03-08T21:15:41.255 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.255 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.255 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "scaling threshold", 2024-03-08T21:15:41.255 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-03-08T21:15:41.255 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.255 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.255 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:41.255 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:41.255 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.255 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-08T21:15:41.255 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "progress", 2024-03-08T21:15:41.255 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-03-08T21:15:41.255 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-03-08T21:15:41.255 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-03-08T21:15:41.256 INFO:teuthology.orchestra.run.smithi107.stdout: "allow_pg_recovery_event": { 2024-03-08T21:15:41.256 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "allow_pg_recovery_event", 2024-03-08T21:15:41.256 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.256 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.256 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.256 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:41.256 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.256 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.256 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.256 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "allow the module to show pg recovery progress", 2024-03-08T21:15:41.256 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.256 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.256 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.256 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.257 INFO:teuthology.orchestra.run.smithi107.stdout: "enabled": { 2024-03-08T21:15:41.257 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "enabled", 2024-03-08T21:15:41.257 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.257 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.257 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.257 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-08T21:15:41.257 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.257 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.257 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.257 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.257 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.257 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.257 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.257 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.257 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-03-08T21:15:41.258 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-03-08T21:15:41.258 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.258 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.258 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.258 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:41.258 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.258 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.258 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:41.258 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-08T21:15:41.258 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-08T21:15:41.258 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-08T21:15:41.258 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-08T21:15:41.258 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-08T21:15:41.258 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-08T21:15:41.259 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:41.259 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.259 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.259 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.259 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.259 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.259 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-03-08T21:15:41.259 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-03-08T21:15:41.259 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.259 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.259 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.259 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:41.259 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.259 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.259 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.260 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.260 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.260 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.260 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.260 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.260 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-03-08T21:15:41.260 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-03-08T21:15:41.260 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.260 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.260 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.260 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-03-08T21:15:41.260 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.260 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.260 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:41.260 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-08T21:15:41.261 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-08T21:15:41.261 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-08T21:15:41.261 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-08T21:15:41.261 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-08T21:15:41.261 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-08T21:15:41.261 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:41.261 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.261 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.261 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.261 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.261 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.261 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-03-08T21:15:41.261 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-03-08T21:15:41.261 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.261 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.262 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.262 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:41.262 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.262 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.262 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.262 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.262 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.262 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.262 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.262 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.262 INFO:teuthology.orchestra.run.smithi107.stdout: "max_completed_events": { 2024-03-08T21:15:41.262 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "max_completed_events", 2024-03-08T21:15:41.262 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-08T21:15:41.262 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.262 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.263 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "50", 2024-03-08T21:15:41.263 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.263 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.263 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.263 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "number of past completed events to remember", 2024-03-08T21:15:41.263 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.263 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.263 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.263 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.263 INFO:teuthology.orchestra.run.smithi107.stdout: "sleep_interval": { 2024-03-08T21:15:41.263 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "sleep_interval", 2024-03-08T21:15:41.263 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-03-08T21:15:41.263 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.263 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.263 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "5", 2024-03-08T21:15:41.264 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.264 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.264 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.264 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "how long the module is going to sleep", 2024-03-08T21:15:41.264 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.264 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.264 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.264 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:41.264 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:41.264 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.264 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-08T21:15:41.264 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "prometheus", 2024-03-08T21:15:41.264 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-03-08T21:15:41.264 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-03-08T21:15:41.264 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-03-08T21:15:41.265 INFO:teuthology.orchestra.run.smithi107.stdout: "cache": { 2024-03-08T21:15:41.265 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "cache", 2024-03-08T21:15:41.265 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.265 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.265 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.265 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-08T21:15:41.265 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.265 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.265 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.265 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.265 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.265 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.265 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.265 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.265 INFO:teuthology.orchestra.run.smithi107.stdout: "exclude_perf_counters": { 2024-03-08T21:15:41.266 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "exclude_perf_counters", 2024-03-08T21:15:41.266 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.266 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.266 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.266 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-08T21:15:41.266 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.266 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.266 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.266 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-03-08T21:15:41.266 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-03-08T21:15:41.266 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.266 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.266 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.266 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-03-08T21:15:41.267 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-03-08T21:15:41.267 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.267 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.267 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.267 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:41.267 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.267 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.267 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:41.267 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-08T21:15:41.267 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-08T21:15:41.267 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-08T21:15:41.267 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-08T21:15:41.267 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-08T21:15:41.267 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-08T21:15:41.268 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:41.268 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.268 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.268 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.268 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.268 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.268 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-03-08T21:15:41.268 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-03-08T21:15:41.268 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.268 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.268 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.268 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:41.269 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.269 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.269 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.269 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.269 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.269 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.269 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.269 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.269 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-03-08T21:15:41.269 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-03-08T21:15:41.269 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.269 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.269 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.269 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-03-08T21:15:41.269 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.270 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.270 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:41.270 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-08T21:15:41.270 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-08T21:15:41.270 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-08T21:15:41.270 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-08T21:15:41.270 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-08T21:15:41.270 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-08T21:15:41.270 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:41.270 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.270 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.270 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.270 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.271 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.271 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-03-08T21:15:41.271 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-03-08T21:15:41.271 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.271 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.271 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.271 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:41.271 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.271 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.271 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.271 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.271 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.271 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.271 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.271 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.271 INFO:teuthology.orchestra.run.smithi107.stdout: "rbd_stats_pools": { 2024-03-08T21:15:41.272 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rbd_stats_pools", 2024-03-08T21:15:41.272 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.272 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.272 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.272 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:41.272 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.272 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.272 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.272 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.272 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.272 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.272 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.272 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.272 INFO:teuthology.orchestra.run.smithi107.stdout: "rbd_stats_pools_refresh_interval": { 2024-03-08T21:15:41.272 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-03-08T21:15:41.273 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-08T21:15:41.273 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.273 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.273 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "300", 2024-03-08T21:15:41.273 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.273 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.273 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.273 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.273 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.273 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.273 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.273 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.273 INFO:teuthology.orchestra.run.smithi107.stdout: "scrape_interval": { 2024-03-08T21:15:41.273 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "scrape_interval", 2024-03-08T21:15:41.274 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "float", 2024-03-08T21:15:41.274 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.274 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.274 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "15.0", 2024-03-08T21:15:41.274 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.274 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.274 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.274 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.274 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.274 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.274 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.274 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.274 INFO:teuthology.orchestra.run.smithi107.stdout: "server_addr": { 2024-03-08T21:15:41.274 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "server_addr", 2024-03-08T21:15:41.274 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.274 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.275 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.275 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "::", 2024-03-08T21:15:41.275 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.275 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.275 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.275 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-03-08T21:15:41.275 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.275 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.275 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.275 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.275 INFO:teuthology.orchestra.run.smithi107.stdout: "server_port": { 2024-03-08T21:15:41.275 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "server_port", 2024-03-08T21:15:41.275 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-08T21:15:41.275 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.275 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.276 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "9283", 2024-03-08T21:15:41.276 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.276 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.276 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.276 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-03-08T21:15:41.276 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.276 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.276 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.276 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.276 INFO:teuthology.orchestra.run.smithi107.stdout: "stale_cache_strategy": { 2024-03-08T21:15:41.276 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "stale_cache_strategy", 2024-03-08T21:15:41.276 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.276 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.276 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.276 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "log", 2024-03-08T21:15:41.277 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.277 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.277 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.277 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.277 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.277 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.277 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.277 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.277 INFO:teuthology.orchestra.run.smithi107.stdout: "standby_behaviour": { 2024-03-08T21:15:41.277 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "standby_behaviour", 2024-03-08T21:15:41.277 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.277 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.277 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.277 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "default", 2024-03-08T21:15:41.277 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.277 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.278 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:41.278 INFO:teuthology.orchestra.run.smithi107.stdout: "default", 2024-03-08T21:15:41.278 INFO:teuthology.orchestra.run.smithi107.stdout: "error" 2024-03-08T21:15:41.278 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:41.278 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.278 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.278 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.278 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.278 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.278 INFO:teuthology.orchestra.run.smithi107.stdout: "standby_error_status_code": { 2024-03-08T21:15:41.278 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "standby_error_status_code", 2024-03-08T21:15:41.278 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-08T21:15:41.278 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.278 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.279 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "500", 2024-03-08T21:15:41.279 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "400", 2024-03-08T21:15:41.279 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "599", 2024-03-08T21:15:41.279 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.279 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.279 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.279 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.279 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.279 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:41.279 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:41.279 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.279 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-08T21:15:41.279 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rbd_support", 2024-03-08T21:15:41.279 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-03-08T21:15:41.279 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-03-08T21:15:41.279 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-03-08T21:15:41.280 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-03-08T21:15:41.280 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-03-08T21:15:41.280 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.280 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.280 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.280 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:41.280 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.280 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.280 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:41.280 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-08T21:15:41.280 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-08T21:15:41.280 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-08T21:15:41.280 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-08T21:15:41.280 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-08T21:15:41.281 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-08T21:15:41.281 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:41.281 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.281 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.281 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.281 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.281 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.281 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-03-08T21:15:41.281 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-03-08T21:15:41.281 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.281 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.281 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.281 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:41.281 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.281 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.282 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.282 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.282 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.282 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.282 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.282 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.282 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-03-08T21:15:41.282 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-03-08T21:15:41.282 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.282 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.282 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.282 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-03-08T21:15:41.282 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.282 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.283 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:41.283 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-08T21:15:41.283 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-08T21:15:41.283 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-08T21:15:41.283 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-08T21:15:41.283 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-08T21:15:41.283 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-08T21:15:41.283 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:41.283 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.283 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.283 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.283 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.283 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.283 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-03-08T21:15:41.283 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-03-08T21:15:41.283 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.283 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.284 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.284 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:41.284 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.284 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.284 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.284 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.284 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.284 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.284 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.284 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.284 INFO:teuthology.orchestra.run.smithi107.stdout: "max_concurrent_snap_create": { 2024-03-08T21:15:41.284 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "max_concurrent_snap_create", 2024-03-08T21:15:41.284 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-08T21:15:41.284 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.284 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.284 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "10", 2024-03-08T21:15:41.285 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.285 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.285 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.285 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.285 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.285 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.285 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.285 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.285 INFO:teuthology.orchestra.run.smithi107.stdout: "mirror_snapshot_schedule": { 2024-03-08T21:15:41.285 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "mirror_snapshot_schedule", 2024-03-08T21:15:41.285 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.285 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.285 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.285 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:41.285 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.286 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.286 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.286 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.286 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.286 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.286 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.286 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.286 INFO:teuthology.orchestra.run.smithi107.stdout: "trash_purge_schedule": { 2024-03-08T21:15:41.286 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "trash_purge_schedule", 2024-03-08T21:15:41.286 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.286 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.286 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.286 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:41.286 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.286 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.286 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.287 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.287 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.287 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.287 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.287 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:41.287 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:41.287 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.287 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-08T21:15:41.287 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "restful", 2024-03-08T21:15:41.287 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-03-08T21:15:41.287 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-03-08T21:15:41.287 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-03-08T21:15:41.287 INFO:teuthology.orchestra.run.smithi107.stdout: "enable_auth": { 2024-03-08T21:15:41.287 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "enable_auth", 2024-03-08T21:15:41.287 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.287 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.288 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.288 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-08T21:15:41.288 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.288 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.288 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.288 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.288 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.288 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.288 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.288 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.288 INFO:teuthology.orchestra.run.smithi107.stdout: "key_file": { 2024-03-08T21:15:41.288 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "key_file", 2024-03-08T21:15:41.288 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.288 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.288 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.288 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:41.289 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.289 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.289 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.289 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.289 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.289 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.289 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.289 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.289 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-03-08T21:15:41.289 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-03-08T21:15:41.289 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.289 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.289 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.289 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:41.289 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.289 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.290 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:41.290 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-08T21:15:41.290 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-08T21:15:41.290 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-08T21:15:41.290 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-08T21:15:41.290 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-08T21:15:41.290 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-08T21:15:41.290 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:41.290 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.290 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.290 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.290 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.290 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.290 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-03-08T21:15:41.291 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-03-08T21:15:41.291 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.291 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.291 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.291 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:41.291 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.291 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.291 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.291 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.291 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.291 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.291 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.291 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.291 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-03-08T21:15:41.291 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-03-08T21:15:41.292 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.292 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.292 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.292 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-03-08T21:15:41.292 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.292 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.292 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:41.292 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-08T21:15:41.292 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-08T21:15:41.292 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-08T21:15:41.292 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-08T21:15:41.292 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-08T21:15:41.292 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-08T21:15:41.292 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:41.292 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.293 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.293 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.293 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.293 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.293 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-03-08T21:15:41.293 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-03-08T21:15:41.293 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.293 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.293 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.293 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:41.293 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.293 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.293 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.293 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.293 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.294 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.294 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.294 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.294 INFO:teuthology.orchestra.run.smithi107.stdout: "server_addr": { 2024-03-08T21:15:41.294 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "server_addr", 2024-03-08T21:15:41.294 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.294 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.294 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.294 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:41.294 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.294 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.294 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.294 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.294 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.295 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.295 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.295 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.295 INFO:teuthology.orchestra.run.smithi107.stdout: "server_port": { 2024-03-08T21:15:41.295 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "server_port", 2024-03-08T21:15:41.295 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.295 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.295 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.295 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:41.295 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.295 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.295 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.295 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.295 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.295 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.295 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.296 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:41.296 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:41.296 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.296 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-08T21:15:41.296 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rgw", 2024-03-08T21:15:41.296 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-03-08T21:15:41.296 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-03-08T21:15:41.296 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-03-08T21:15:41.296 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-03-08T21:15:41.296 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-03-08T21:15:41.296 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.296 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.296 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.296 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:41.296 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.296 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.297 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:41.297 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-08T21:15:41.297 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-08T21:15:41.297 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-08T21:15:41.297 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-08T21:15:41.297 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-08T21:15:41.297 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-08T21:15:41.297 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:41.297 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.297 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.297 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.297 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.297 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.297 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-03-08T21:15:41.298 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-03-08T21:15:41.298 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.298 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.298 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.298 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:41.298 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.298 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.298 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.298 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.298 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.298 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.298 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.298 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.299 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-03-08T21:15:41.299 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-03-08T21:15:41.299 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.299 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.299 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.299 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-03-08T21:15:41.299 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.299 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.299 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:41.299 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-08T21:15:41.299 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-08T21:15:41.299 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-08T21:15:41.299 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-08T21:15:41.299 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-08T21:15:41.300 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-08T21:15:41.300 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:41.300 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.300 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.300 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.300 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.300 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.300 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-03-08T21:15:41.300 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-03-08T21:15:41.300 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.300 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.300 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.300 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:41.301 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.301 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.301 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.301 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.301 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.301 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.301 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.301 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:41.301 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:41.301 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.301 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-08T21:15:41.301 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rook", 2024-03-08T21:15:41.301 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-03-08T21:15:41.301 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-03-08T21:15:41.302 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-03-08T21:15:41.302 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-03-08T21:15:41.302 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-03-08T21:15:41.302 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.302 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.302 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.302 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:41.302 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.302 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.302 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:41.302 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-08T21:15:41.302 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-08T21:15:41.302 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-08T21:15:41.302 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-08T21:15:41.303 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-08T21:15:41.303 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-08T21:15:41.303 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:41.303 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.303 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.303 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.303 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.303 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.303 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-03-08T21:15:41.303 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-03-08T21:15:41.303 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.303 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.303 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.303 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:41.303 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.304 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.304 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.304 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.304 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.304 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.304 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.304 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.304 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-03-08T21:15:41.304 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-03-08T21:15:41.304 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.304 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.304 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.304 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-03-08T21:15:41.304 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.305 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.305 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:41.305 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-08T21:15:41.305 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-08T21:15:41.305 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-08T21:15:41.305 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-08T21:15:41.305 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-08T21:15:41.305 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-08T21:15:41.305 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:41.305 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.305 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.305 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.305 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.305 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.306 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-03-08T21:15:41.306 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-03-08T21:15:41.306 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.306 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.306 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.306 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:41.306 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.306 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.306 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.306 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.306 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.306 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.306 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.306 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.307 INFO:teuthology.orchestra.run.smithi107.stdout: "storage_class": { 2024-03-08T21:15:41.307 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "storage_class", 2024-03-08T21:15:41.307 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.307 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.307 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.307 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "local", 2024-03-08T21:15:41.307 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.307 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.307 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.307 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-03-08T21:15:41.307 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.307 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.307 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.308 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:41.308 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:41.308 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.308 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-08T21:15:41.308 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "selftest", 2024-03-08T21:15:41.308 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-03-08T21:15:41.308 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-03-08T21:15:41.308 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-03-08T21:15:41.308 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-03-08T21:15:41.308 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-03-08T21:15:41.308 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.308 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.309 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.309 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:41.309 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.309 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.309 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:41.309 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-08T21:15:41.309 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-08T21:15:41.309 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-08T21:15:41.309 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-08T21:15:41.309 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-08T21:15:41.309 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-08T21:15:41.309 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:41.309 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.309 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.310 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.310 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.310 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.310 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-03-08T21:15:41.310 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-03-08T21:15:41.310 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.310 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.310 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.310 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:41.310 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.310 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.310 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.310 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.311 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.311 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.311 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.311 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.311 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-03-08T21:15:41.311 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-03-08T21:15:41.311 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.311 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.311 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.311 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-03-08T21:15:41.311 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.311 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.311 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:41.311 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-08T21:15:41.312 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-08T21:15:41.312 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-08T21:15:41.312 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-08T21:15:41.312 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-08T21:15:41.312 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-08T21:15:41.312 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:41.312 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.312 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.312 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.312 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.312 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.312 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-03-08T21:15:41.312 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-03-08T21:15:41.312 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.312 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.313 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.313 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:41.313 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.313 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.313 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.313 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.313 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.313 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.313 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.313 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.313 INFO:teuthology.orchestra.run.smithi107.stdout: "roption1": { 2024-03-08T21:15:41.313 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "roption1", 2024-03-08T21:15:41.313 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.313 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.313 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.314 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:41.314 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.314 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.314 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.314 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.314 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.314 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.314 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.314 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.314 INFO:teuthology.orchestra.run.smithi107.stdout: "roption2": { 2024-03-08T21:15:41.314 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "roption2", 2024-03-08T21:15:41.314 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.314 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.314 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.314 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "xyz", 2024-03-08T21:15:41.315 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.315 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.315 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.315 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.315 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.315 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.315 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.315 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.315 INFO:teuthology.orchestra.run.smithi107.stdout: "rwoption1": { 2024-03-08T21:15:41.315 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rwoption1", 2024-03-08T21:15:41.315 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.315 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.315 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.315 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:41.315 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.316 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.316 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.316 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.316 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.316 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.316 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.316 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.316 INFO:teuthology.orchestra.run.smithi107.stdout: "rwoption2": { 2024-03-08T21:15:41.316 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rwoption2", 2024-03-08T21:15:41.316 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-08T21:15:41.316 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.316 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.316 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:41.316 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.316 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.316 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.317 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.317 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.317 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.317 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.317 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.317 INFO:teuthology.orchestra.run.smithi107.stdout: "rwoption3": { 2024-03-08T21:15:41.317 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rwoption3", 2024-03-08T21:15:41.317 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "float", 2024-03-08T21:15:41.317 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.317 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.317 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:41.317 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.317 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.317 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.317 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.318 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.318 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.318 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.318 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.318 INFO:teuthology.orchestra.run.smithi107.stdout: "rwoption4": { 2024-03-08T21:15:41.318 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rwoption4", 2024-03-08T21:15:41.318 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.318 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.318 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.318 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:41.318 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.318 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.318 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.318 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.319 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.319 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.319 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.319 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.319 INFO:teuthology.orchestra.run.smithi107.stdout: "rwoption5": { 2024-03-08T21:15:41.319 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rwoption5", 2024-03-08T21:15:41.319 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.319 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.319 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.319 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:41.319 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.319 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.319 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.319 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.320 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.320 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.320 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.320 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.320 INFO:teuthology.orchestra.run.smithi107.stdout: "rwoption6": { 2024-03-08T21:15:41.320 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rwoption6", 2024-03-08T21:15:41.320 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.320 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.320 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.320 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-08T21:15:41.320 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.320 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.320 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.320 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.320 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.320 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.321 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.321 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.321 INFO:teuthology.orchestra.run.smithi107.stdout: "rwoption7": { 2024-03-08T21:15:41.321 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rwoption7", 2024-03-08T21:15:41.321 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-08T21:15:41.321 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.321 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.321 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:41.321 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "1", 2024-03-08T21:15:41.321 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "42", 2024-03-08T21:15:41.321 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.321 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.321 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.321 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.321 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.322 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.322 INFO:teuthology.orchestra.run.smithi107.stdout: "testkey": { 2024-03-08T21:15:41.322 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "testkey", 2024-03-08T21:15:41.322 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.322 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.322 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.322 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:41.322 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.322 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.322 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.322 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.322 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.322 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.322 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.322 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.323 INFO:teuthology.orchestra.run.smithi107.stdout: "testlkey": { 2024-03-08T21:15:41.323 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "testlkey", 2024-03-08T21:15:41.323 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.323 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.323 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.323 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:41.323 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.323 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.323 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.323 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.323 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.323 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.323 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.323 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.323 INFO:teuthology.orchestra.run.smithi107.stdout: "testnewline": { 2024-03-08T21:15:41.323 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "testnewline", 2024-03-08T21:15:41.324 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.324 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.324 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.324 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:41.324 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.324 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.324 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.324 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.324 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.324 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.324 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.324 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:41.324 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:41.324 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.324 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-08T21:15:41.324 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "snap_schedule", 2024-03-08T21:15:41.325 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-03-08T21:15:41.325 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-03-08T21:15:41.325 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-03-08T21:15:41.325 INFO:teuthology.orchestra.run.smithi107.stdout: "allow_m_granularity": { 2024-03-08T21:15:41.325 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "allow_m_granularity", 2024-03-08T21:15:41.325 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.325 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.325 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.325 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:41.325 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.325 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.325 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.325 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "allow minute scheduled snapshots", 2024-03-08T21:15:41.325 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.325 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.325 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.326 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.326 INFO:teuthology.orchestra.run.smithi107.stdout: "dump_on_update": { 2024-03-08T21:15:41.326 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "dump_on_update", 2024-03-08T21:15:41.326 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.326 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.326 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.326 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:41.326 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.326 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.326 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.326 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "dump database to debug log on update", 2024-03-08T21:15:41.326 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.326 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.326 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.326 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.327 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-03-08T21:15:41.327 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-03-08T21:15:41.327 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.327 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.327 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.327 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:41.327 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.327 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.327 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:41.327 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-08T21:15:41.327 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-08T21:15:41.327 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-08T21:15:41.327 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-08T21:15:41.327 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-08T21:15:41.327 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-08T21:15:41.327 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:41.328 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.328 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.328 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.328 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.328 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.328 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-03-08T21:15:41.328 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-03-08T21:15:41.328 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.328 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.328 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.328 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:41.328 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.328 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.328 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.328 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.328 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.329 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.329 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.329 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.329 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-03-08T21:15:41.329 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-03-08T21:15:41.329 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.329 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.329 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.329 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-03-08T21:15:41.329 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.329 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.329 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:41.329 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-08T21:15:41.329 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-08T21:15:41.330 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-08T21:15:41.330 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-08T21:15:41.330 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-08T21:15:41.330 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-08T21:15:41.330 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:41.330 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.330 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.330 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.330 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.330 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.330 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-03-08T21:15:41.330 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-03-08T21:15:41.330 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.330 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.331 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.331 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:41.331 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.331 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.331 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.331 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.331 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.331 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.331 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.331 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:41.331 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:41.331 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.331 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-08T21:15:41.331 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "stats", 2024-03-08T21:15:41.331 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-03-08T21:15:41.332 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-03-08T21:15:41.332 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-03-08T21:15:41.332 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-03-08T21:15:41.332 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-03-08T21:15:41.332 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.332 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.332 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.332 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:41.332 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.332 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.332 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:41.332 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-08T21:15:41.332 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-08T21:15:41.332 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-08T21:15:41.332 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-08T21:15:41.333 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-08T21:15:41.333 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-08T21:15:41.333 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:41.333 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.333 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.333 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.333 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.333 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.333 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-03-08T21:15:41.333 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-03-08T21:15:41.333 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.333 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.333 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.333 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:41.333 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.333 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.334 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.334 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.334 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.334 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.334 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.334 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.334 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-03-08T21:15:41.334 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-03-08T21:15:41.334 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.334 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.334 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.334 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-03-08T21:15:41.334 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.334 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.335 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:41.335 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-08T21:15:41.335 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-08T21:15:41.335 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-08T21:15:41.335 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-08T21:15:41.335 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-08T21:15:41.335 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-08T21:15:41.335 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:41.335 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.335 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.335 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.335 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.335 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.335 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-03-08T21:15:41.335 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-03-08T21:15:41.335 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.336 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.336 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.336 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:41.336 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.336 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.336 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.336 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.336 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.336 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.336 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.336 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:41.336 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:41.336 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.336 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-08T21:15:41.336 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "status", 2024-03-08T21:15:41.336 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-03-08T21:15:41.336 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-03-08T21:15:41.337 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-03-08T21:15:41.337 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-03-08T21:15:41.337 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-03-08T21:15:41.337 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.337 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.337 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.337 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:41.337 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.337 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.337 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:41.337 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-08T21:15:41.337 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-08T21:15:41.337 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-08T21:15:41.337 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-08T21:15:41.337 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-08T21:15:41.338 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-08T21:15:41.338 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:41.338 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.338 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.338 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.338 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.338 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.338 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-03-08T21:15:41.338 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-03-08T21:15:41.338 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.338 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.338 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.338 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:41.338 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.338 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.339 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.339 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.339 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.339 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.339 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.339 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.339 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-03-08T21:15:41.339 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-03-08T21:15:41.339 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.339 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.339 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.339 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-03-08T21:15:41.339 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.339 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.339 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:41.339 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-08T21:15:41.340 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-08T21:15:41.340 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-08T21:15:41.340 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-08T21:15:41.340 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-08T21:15:41.340 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-08T21:15:41.340 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:41.340 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.340 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.340 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.340 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.340 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.340 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-03-08T21:15:41.340 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-03-08T21:15:41.340 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.340 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.340 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.340 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:41.341 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.341 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.341 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.341 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.341 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.341 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.341 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.341 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:41.341 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:41.341 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.341 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-08T21:15:41.341 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "telegraf", 2024-03-08T21:15:41.341 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-03-08T21:15:41.341 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-03-08T21:15:41.341 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-03-08T21:15:41.342 INFO:teuthology.orchestra.run.smithi107.stdout: "address": { 2024-03-08T21:15:41.342 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "address", 2024-03-08T21:15:41.342 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.342 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.342 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.342 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-03-08T21:15:41.342 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.342 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.342 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.342 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.342 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.342 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.342 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.342 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.343 INFO:teuthology.orchestra.run.smithi107.stdout: "interval": { 2024-03-08T21:15:41.343 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "interval", 2024-03-08T21:15:41.343 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-03-08T21:15:41.343 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.343 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.343 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "15", 2024-03-08T21:15:41.343 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.343 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.343 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.343 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.343 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.343 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.343 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.343 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.343 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-03-08T21:15:41.344 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-03-08T21:15:41.344 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.344 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.344 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.344 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:41.344 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.344 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.344 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:41.344 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-08T21:15:41.344 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-08T21:15:41.344 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-08T21:15:41.344 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-08T21:15:41.344 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-08T21:15:41.344 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-08T21:15:41.344 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:41.345 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.345 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.345 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.345 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.345 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.345 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-03-08T21:15:41.345 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-03-08T21:15:41.345 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.345 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.345 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.345 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:41.345 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.345 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.345 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.345 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.346 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.346 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.346 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.346 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.346 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-03-08T21:15:41.346 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-03-08T21:15:41.346 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.346 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.346 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.346 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-03-08T21:15:41.346 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.346 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.346 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:41.346 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-08T21:15:41.347 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-08T21:15:41.347 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-08T21:15:41.347 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-08T21:15:41.347 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-08T21:15:41.347 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-08T21:15:41.347 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:41.347 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.347 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.347 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.347 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.347 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.347 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-03-08T21:15:41.347 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-03-08T21:15:41.347 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.347 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.348 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.348 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:41.348 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.348 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.348 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.348 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.348 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.348 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.348 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.348 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:41.348 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:41.348 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.348 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-08T21:15:41.348 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "telemetry", 2024-03-08T21:15:41.348 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-03-08T21:15:41.349 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-03-08T21:15:41.349 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-03-08T21:15:41.349 INFO:teuthology.orchestra.run.smithi107.stdout: "channel_basic": { 2024-03-08T21:15:41.349 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "channel_basic", 2024-03-08T21:15:41.349 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.349 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.349 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.349 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-08T21:15:41.349 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.349 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.349 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.349 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Share basic cluster information (size, version)", 2024-03-08T21:15:41.349 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.349 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.349 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.349 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.350 INFO:teuthology.orchestra.run.smithi107.stdout: "channel_crash": { 2024-03-08T21:15:41.350 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "channel_crash", 2024-03-08T21:15:41.350 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.350 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.350 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.350 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-08T21:15:41.350 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.350 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.350 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.350 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-03-08T21:15:41.350 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.350 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.350 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.350 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.350 INFO:teuthology.orchestra.run.smithi107.stdout: "channel_device": { 2024-03-08T21:15:41.351 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "channel_device", 2024-03-08T21:15:41.351 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.351 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.351 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.351 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-08T21:15:41.351 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.351 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.351 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.351 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-03-08T21:15:41.351 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.351 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.351 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.351 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.351 INFO:teuthology.orchestra.run.smithi107.stdout: "channel_ident": { 2024-03-08T21:15:41.351 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "channel_ident", 2024-03-08T21:15:41.352 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.352 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.352 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.352 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:41.352 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.352 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.352 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.352 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-03-08T21:15:41.352 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.352 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.352 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.352 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.352 INFO:teuthology.orchestra.run.smithi107.stdout: "channel_perf": { 2024-03-08T21:15:41.352 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "channel_perf", 2024-03-08T21:15:41.352 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.353 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.353 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.353 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:41.353 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.353 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.353 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.353 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Share various performance metrics of a cluster", 2024-03-08T21:15:41.353 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.353 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.353 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.353 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.353 INFO:teuthology.orchestra.run.smithi107.stdout: "contact": { 2024-03-08T21:15:41.353 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "contact", 2024-03-08T21:15:41.353 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.354 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.354 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.354 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:41.354 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.354 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.354 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.354 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.354 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.354 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.354 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.354 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.354 INFO:teuthology.orchestra.run.smithi107.stdout: "description": { 2024-03-08T21:15:41.354 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "description", 2024-03-08T21:15:41.354 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.354 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.355 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.355 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:41.355 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.355 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.355 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.355 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.355 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.355 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.355 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.355 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.355 INFO:teuthology.orchestra.run.smithi107.stdout: "device_url": { 2024-03-08T21:15:41.355 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "device_url", 2024-03-08T21:15:41.355 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.355 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.355 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.356 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-03-08T21:15:41.356 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.356 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.356 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.356 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.356 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.356 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.356 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.356 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.356 INFO:teuthology.orchestra.run.smithi107.stdout: "enabled": { 2024-03-08T21:15:41.356 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "enabled", 2024-03-08T21:15:41.356 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.356 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.356 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.356 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:41.356 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.357 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.357 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.357 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.357 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.357 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.357 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.357 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.357 INFO:teuthology.orchestra.run.smithi107.stdout: "interval": { 2024-03-08T21:15:41.357 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "interval", 2024-03-08T21:15:41.357 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-08T21:15:41.357 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.357 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.357 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "24", 2024-03-08T21:15:41.357 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "8", 2024-03-08T21:15:41.358 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.358 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.358 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.358 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.358 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.358 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.358 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.358 INFO:teuthology.orchestra.run.smithi107.stdout: "last_opt_revision": { 2024-03-08T21:15:41.358 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "last_opt_revision", 2024-03-08T21:15:41.358 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-08T21:15:41.358 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.358 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.358 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "1", 2024-03-08T21:15:41.359 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.359 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.359 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.359 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.359 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.359 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.359 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.359 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.359 INFO:teuthology.orchestra.run.smithi107.stdout: "leaderboard": { 2024-03-08T21:15:41.359 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "leaderboard", 2024-03-08T21:15:41.359 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.359 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.359 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.359 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:41.359 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.360 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.360 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.360 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.360 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.360 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.360 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.360 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.360 INFO:teuthology.orchestra.run.smithi107.stdout: "leaderboard_description": { 2024-03-08T21:15:41.360 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "leaderboard_description", 2024-03-08T21:15:41.360 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.360 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.360 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.360 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:41.360 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.360 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.361 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.361 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.361 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.361 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.361 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.361 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.361 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-03-08T21:15:41.361 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-03-08T21:15:41.361 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.361 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.361 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.361 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:41.361 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.361 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.362 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:41.362 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-08T21:15:41.362 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-08T21:15:41.362 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-08T21:15:41.362 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-08T21:15:41.362 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-08T21:15:41.362 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-08T21:15:41.362 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:41.362 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.362 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.362 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.362 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.362 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.363 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-03-08T21:15:41.363 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-03-08T21:15:41.363 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.363 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.363 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.363 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:41.363 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.363 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.363 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.363 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.363 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.363 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.363 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.363 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.364 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-03-08T21:15:41.364 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-03-08T21:15:41.364 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.364 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.364 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.364 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-03-08T21:15:41.364 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.364 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.364 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:41.364 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-08T21:15:41.364 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-08T21:15:41.364 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-08T21:15:41.364 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-08T21:15:41.364 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-08T21:15:41.364 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-08T21:15:41.365 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:41.365 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.365 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.365 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.365 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.365 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.365 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-03-08T21:15:41.365 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-03-08T21:15:41.365 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.365 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.365 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.365 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:41.365 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.365 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.365 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.366 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.366 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.366 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.366 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.366 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.366 INFO:teuthology.orchestra.run.smithi107.stdout: "organization": { 2024-03-08T21:15:41.366 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "organization", 2024-03-08T21:15:41.366 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.366 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.366 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.366 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:41.366 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.366 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.366 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.367 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.367 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.367 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.367 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.367 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.367 INFO:teuthology.orchestra.run.smithi107.stdout: "proxy": { 2024-03-08T21:15:41.367 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "proxy", 2024-03-08T21:15:41.367 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.367 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.367 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.367 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:41.367 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.367 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.367 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.367 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.367 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.368 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.368 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.368 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.368 INFO:teuthology.orchestra.run.smithi107.stdout: "url": { 2024-03-08T21:15:41.368 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "url", 2024-03-08T21:15:41.368 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.368 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.368 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.368 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-03-08T21:15:41.368 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.368 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.368 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.368 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.368 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.368 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.368 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.369 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:41.369 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:41.369 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.369 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-08T21:15:41.369 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "test_orchestrator", 2024-03-08T21:15:41.369 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-03-08T21:15:41.369 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-03-08T21:15:41.369 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-03-08T21:15:41.369 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-03-08T21:15:41.369 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-03-08T21:15:41.369 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.369 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.369 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.369 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:41.370 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.370 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.370 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:41.370 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-08T21:15:41.370 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-08T21:15:41.370 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-08T21:15:41.370 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-08T21:15:41.370 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-08T21:15:41.370 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-08T21:15:41.370 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:41.370 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.370 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.370 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.370 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.370 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.371 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-03-08T21:15:41.371 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-03-08T21:15:41.371 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.371 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.371 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.371 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:41.371 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.371 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.371 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.371 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.371 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.371 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.371 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.371 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.372 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-03-08T21:15:41.372 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-03-08T21:15:41.372 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.372 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.372 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.372 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-03-08T21:15:41.372 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.372 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.372 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:41.372 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-08T21:15:41.372 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-08T21:15:41.372 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-08T21:15:41.372 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-08T21:15:41.372 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-08T21:15:41.372 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-08T21:15:41.372 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:41.373 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.373 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.373 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.373 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.373 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.373 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-03-08T21:15:41.373 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-03-08T21:15:41.373 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.373 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.373 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.373 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:41.373 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.373 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.373 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.373 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.373 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.374 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.374 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.374 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:41.374 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:41.374 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.374 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-08T21:15:41.374 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "volumes", 2024-03-08T21:15:41.374 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-03-08T21:15:41.374 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-03-08T21:15:41.374 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-03-08T21:15:41.374 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-03-08T21:15:41.374 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-03-08T21:15:41.374 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.374 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.374 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.375 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:41.375 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.375 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.375 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:41.375 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-08T21:15:41.375 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-08T21:15:41.375 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-08T21:15:41.375 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-08T21:15:41.375 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-08T21:15:41.375 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-08T21:15:41.375 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:41.375 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.375 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.375 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.375 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.375 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.376 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-03-08T21:15:41.376 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-03-08T21:15:41.376 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.376 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.376 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.376 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:41.376 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.376 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.376 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.376 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.376 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.376 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.376 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.376 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.376 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-03-08T21:15:41.376 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-03-08T21:15:41.377 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.377 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.377 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.377 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-03-08T21:15:41.377 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.377 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.377 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:41.377 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-08T21:15:41.377 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-08T21:15:41.377 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-08T21:15:41.377 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-08T21:15:41.377 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-08T21:15:41.377 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-08T21:15:41.377 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:41.377 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.378 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.378 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.378 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.378 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.378 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-03-08T21:15:41.378 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-03-08T21:15:41.378 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.378 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.378 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.378 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:41.378 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.378 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.378 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.378 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.379 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.379 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.379 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.379 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.379 INFO:teuthology.orchestra.run.smithi107.stdout: "max_concurrent_clones": { 2024-03-08T21:15:41.379 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "max_concurrent_clones", 2024-03-08T21:15:41.379 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-08T21:15:41.379 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.379 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.379 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "4", 2024-03-08T21:15:41.379 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.379 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.379 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.379 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Number of asynchronous cloner threads", 2024-03-08T21:15:41.379 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.379 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.380 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.380 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.380 INFO:teuthology.orchestra.run.smithi107.stdout: "periodic_async_work": { 2024-03-08T21:15:41.380 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "periodic_async_work", 2024-03-08T21:15:41.380 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.380 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.380 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.380 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:41.380 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.380 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.380 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.380 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Periodically check for async work", 2024-03-08T21:15:41.380 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.380 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.380 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.380 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.381 INFO:teuthology.orchestra.run.smithi107.stdout: "snapshot_clone_delay": { 2024-03-08T21:15:41.381 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "snapshot_clone_delay", 2024-03-08T21:15:41.381 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-08T21:15:41.381 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.381 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.381 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0", 2024-03-08T21:15:41.381 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.381 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.381 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.381 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-03-08T21:15:41.381 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.381 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.381 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.381 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.381 INFO:teuthology.orchestra.run.smithi107.stdout: "snapshot_clone_no_wait": { 2024-03-08T21:15:41.381 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "snapshot_clone_no_wait", 2024-03-08T21:15:41.382 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.382 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.382 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.382 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-08T21:15:41.382 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.382 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.382 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.382 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-03-08T21:15:41.382 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.382 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.382 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.382 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:41.382 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:41.382 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.382 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-08T21:15:41.383 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "zabbix", 2024-03-08T21:15:41.383 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-03-08T21:15:41.383 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-03-08T21:15:41.383 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-03-08T21:15:41.383 INFO:teuthology.orchestra.run.smithi107.stdout: "discovery_interval": { 2024-03-08T21:15:41.383 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "discovery_interval", 2024-03-08T21:15:41.383 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "uint", 2024-03-08T21:15:41.383 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.383 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.383 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "100", 2024-03-08T21:15:41.383 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.383 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.383 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.383 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.383 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.384 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.384 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.384 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.384 INFO:teuthology.orchestra.run.smithi107.stdout: "identifier": { 2024-03-08T21:15:41.384 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "identifier", 2024-03-08T21:15:41.384 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.384 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.384 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.384 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:41.384 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.384 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.384 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.384 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.384 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.384 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.384 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.385 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.385 INFO:teuthology.orchestra.run.smithi107.stdout: "interval": { 2024-03-08T21:15:41.385 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "interval", 2024-03-08T21:15:41.385 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-03-08T21:15:41.385 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.385 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.385 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "60", 2024-03-08T21:15:41.385 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.385 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.385 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.385 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.385 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.385 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.385 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.385 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.385 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-03-08T21:15:41.386 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-03-08T21:15:41.386 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.386 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.386 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.386 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:41.386 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.386 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.386 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:41.386 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-08T21:15:41.386 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-08T21:15:41.386 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-08T21:15:41.386 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-08T21:15:41.386 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-08T21:15:41.387 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-08T21:15:41.387 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:41.387 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.387 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.387 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.387 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.387 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.387 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-03-08T21:15:41.387 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-03-08T21:15:41.387 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.387 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.387 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.387 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:41.387 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.387 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.388 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.388 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.388 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.388 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.388 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.388 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.388 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-03-08T21:15:41.388 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-03-08T21:15:41.388 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.388 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.388 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.388 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-03-08T21:15:41.388 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.388 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.388 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:41.388 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-08T21:15:41.389 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-08T21:15:41.389 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-08T21:15:41.389 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-08T21:15:41.389 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-08T21:15:41.389 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-08T21:15:41.389 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:41.389 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.389 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.389 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.389 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.389 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.389 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-03-08T21:15:41.389 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-03-08T21:15:41.389 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.389 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.389 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.390 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:41.390 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.390 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.390 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.390 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.390 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.390 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.390 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.390 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.390 INFO:teuthology.orchestra.run.smithi107.stdout: "zabbix_host": { 2024-03-08T21:15:41.390 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "zabbix_host", 2024-03-08T21:15:41.390 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.390 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.390 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.390 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:41.390 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.391 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.391 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.391 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.391 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.391 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.391 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.391 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.391 INFO:teuthology.orchestra.run.smithi107.stdout: "zabbix_port": { 2024-03-08T21:15:41.391 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "zabbix_port", 2024-03-08T21:15:41.391 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-08T21:15:41.391 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.391 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.391 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "10051", 2024-03-08T21:15:41.391 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.391 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.392 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.392 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.392 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.392 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.392 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.392 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.392 INFO:teuthology.orchestra.run.smithi107.stdout: "zabbix_sender": { 2024-03-08T21:15:41.392 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "zabbix_sender", 2024-03-08T21:15:41.392 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.392 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.392 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.392 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-03-08T21:15:41.392 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.392 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.392 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.392 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.392 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.393 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.393 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.393 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:41.393 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:41.393 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:41.393 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-03-08T21:15:41.393 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:41.393 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:41.393 INFO:teuthology.orchestra.run.smithi107.stdout: "modules": [ 2024-03-08T21:15:41.393 INFO:teuthology.orchestra.run.smithi107.stdout: "cephadm", 2024-03-08T21:15:41.393 INFO:teuthology.orchestra.run.smithi107.stdout: "dashboard", 2024-03-08T21:15:41.393 INFO:teuthology.orchestra.run.smithi107.stdout: "iostat", 2024-03-08T21:15:41.393 INFO:teuthology.orchestra.run.smithi107.stdout: "nfs", 2024-03-08T21:15:41.393 INFO:teuthology.orchestra.run.smithi107.stdout: "restful", 2024-03-08T21:15:41.393 INFO:teuthology.orchestra.run.smithi107.stdout: "snap_schedule" 2024-03-08T21:15:41.393 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:41.394 INFO:teuthology.orchestra.run.smithi107.stdout: "available_modules": [ 2024-03-08T21:15:41.394 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-08T21:15:41.394 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "alerts", 2024-03-08T21:15:41.394 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-03-08T21:15:41.394 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-03-08T21:15:41.394 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-03-08T21:15:41.394 INFO:teuthology.orchestra.run.smithi107.stdout: "interval": { 2024-03-08T21:15:41.394 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "interval", 2024-03-08T21:15:41.394 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-03-08T21:15:41.394 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.394 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.394 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "60", 2024-03-08T21:15:41.394 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.394 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.394 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.395 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "How frequently to reexamine health status", 2024-03-08T21:15:41.395 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.395 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.395 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.395 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.395 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-03-08T21:15:41.395 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-03-08T21:15:41.395 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.395 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.395 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.395 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:41.395 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.395 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.395 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:41.395 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-08T21:15:41.396 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-08T21:15:41.396 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-08T21:15:41.396 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-08T21:15:41.396 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-08T21:15:41.396 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-08T21:15:41.396 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:41.396 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.396 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.396 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.396 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.396 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.396 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-03-08T21:15:41.396 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-03-08T21:15:41.396 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.396 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.397 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.397 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:41.397 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.397 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.397 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.397 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.397 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.397 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.397 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.397 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.397 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-03-08T21:15:41.397 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-03-08T21:15:41.397 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.397 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.397 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.398 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-03-08T21:15:41.398 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.398 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.398 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:41.398 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-08T21:15:41.398 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-08T21:15:41.398 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-08T21:15:41.398 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-08T21:15:41.398 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-08T21:15:41.398 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-08T21:15:41.398 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:41.398 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.398 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.398 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.398 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.399 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.399 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-03-08T21:15:41.399 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-03-08T21:15:41.399 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.399 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.399 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.399 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:41.399 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.399 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.399 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.399 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.399 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.399 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.399 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.399 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.399 INFO:teuthology.orchestra.run.smithi107.stdout: "smtp_destination": { 2024-03-08T21:15:41.400 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "smtp_destination", 2024-03-08T21:15:41.400 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.400 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.400 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.400 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:41.400 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.400 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.400 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.400 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Email address to send alerts to", 2024-03-08T21:15:41.400 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.400 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.400 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.400 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.400 INFO:teuthology.orchestra.run.smithi107.stdout: "smtp_from_name": { 2024-03-08T21:15:41.400 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "smtp_from_name", 2024-03-08T21:15:41.400 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.401 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.401 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.401 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "Ceph", 2024-03-08T21:15:41.401 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.401 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.401 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.401 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Email From: name", 2024-03-08T21:15:41.401 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.401 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.401 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.401 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.401 INFO:teuthology.orchestra.run.smithi107.stdout: "smtp_host": { 2024-03-08T21:15:41.401 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "smtp_host", 2024-03-08T21:15:41.401 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.402 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.402 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.402 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:41.402 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.402 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.402 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.402 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "SMTP server", 2024-03-08T21:15:41.402 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.402 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.402 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.402 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.402 INFO:teuthology.orchestra.run.smithi107.stdout: "smtp_password": { 2024-03-08T21:15:41.402 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "smtp_password", 2024-03-08T21:15:41.402 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.402 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.403 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.403 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:41.403 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.403 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.403 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.403 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Password to authenticate with", 2024-03-08T21:15:41.403 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.403 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.403 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.403 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.403 INFO:teuthology.orchestra.run.smithi107.stdout: "smtp_port": { 2024-03-08T21:15:41.403 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "smtp_port", 2024-03-08T21:15:41.403 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-08T21:15:41.403 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.403 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.403 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "465", 2024-03-08T21:15:41.404 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.404 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.404 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.404 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "SMTP port", 2024-03-08T21:15:41.404 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.404 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.404 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.404 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.404 INFO:teuthology.orchestra.run.smithi107.stdout: "smtp_sender": { 2024-03-08T21:15:41.404 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "smtp_sender", 2024-03-08T21:15:41.404 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.404 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.404 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.404 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:41.404 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.404 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.405 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.405 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "SMTP envelope sender", 2024-03-08T21:15:41.405 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.405 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.405 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.405 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.405 INFO:teuthology.orchestra.run.smithi107.stdout: "smtp_ssl": { 2024-03-08T21:15:41.405 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "smtp_ssl", 2024-03-08T21:15:41.405 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.405 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.405 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.405 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-08T21:15:41.405 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.406 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.406 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.406 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Use SSL to connect to SMTP server", 2024-03-08T21:15:41.406 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.406 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.406 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.406 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.406 INFO:teuthology.orchestra.run.smithi107.stdout: "smtp_user": { 2024-03-08T21:15:41.406 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "smtp_user", 2024-03-08T21:15:41.406 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.406 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.406 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.406 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:41.406 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.406 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.407 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.407 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "User to authenticate as", 2024-03-08T21:15:41.407 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.407 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.407 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.407 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:41.407 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:41.407 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.407 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-08T21:15:41.407 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "balancer", 2024-03-08T21:15:41.407 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-03-08T21:15:41.407 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-03-08T21:15:41.407 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-03-08T21:15:41.407 INFO:teuthology.orchestra.run.smithi107.stdout: "active": { 2024-03-08T21:15:41.407 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "active", 2024-03-08T21:15:41.407 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.408 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.408 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.408 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-08T21:15:41.408 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.408 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.408 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.408 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "automatically balance PGs across cluster", 2024-03-08T21:15:41.408 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.408 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.408 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.408 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.408 INFO:teuthology.orchestra.run.smithi107.stdout: "begin_time": { 2024-03-08T21:15:41.408 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "begin_time", 2024-03-08T21:15:41.408 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.409 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.409 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.409 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0000", 2024-03-08T21:15:41.409 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.409 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.409 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.409 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "beginning time of day to automatically balance", 2024-03-08T21:15:41.409 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-08T21:15:41.409 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.409 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.409 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.409 INFO:teuthology.orchestra.run.smithi107.stdout: "begin_weekday": { 2024-03-08T21:15:41.409 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "begin_weekday", 2024-03-08T21:15:41.409 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "uint", 2024-03-08T21:15:41.410 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.410 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.410 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0", 2024-03-08T21:15:41.410 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "0", 2024-03-08T21:15:41.410 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "6", 2024-03-08T21:15:41.410 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.410 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-03-08T21:15:41.410 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-08T21:15:41.410 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.410 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.410 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.410 INFO:teuthology.orchestra.run.smithi107.stdout: "crush_compat_max_iterations": { 2024-03-08T21:15:41.410 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "crush_compat_max_iterations", 2024-03-08T21:15:41.411 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "uint", 2024-03-08T21:15:41.411 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.411 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.411 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "25", 2024-03-08T21:15:41.411 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "1", 2024-03-08T21:15:41.411 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "250", 2024-03-08T21:15:41.411 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.411 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-03-08T21:15:41.411 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.411 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.411 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.411 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.411 INFO:teuthology.orchestra.run.smithi107.stdout: "crush_compat_metrics": { 2024-03-08T21:15:41.411 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "crush_compat_metrics", 2024-03-08T21:15:41.411 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.412 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.412 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.412 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "pgs,objects,bytes", 2024-03-08T21:15:41.412 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.412 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.412 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.412 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-03-08T21:15:41.412 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-03-08T21:15:41.412 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.412 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.412 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.412 INFO:teuthology.orchestra.run.smithi107.stdout: "crush_compat_step": { 2024-03-08T21:15:41.412 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "crush_compat_step", 2024-03-08T21:15:41.412 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "float", 2024-03-08T21:15:41.412 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.412 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.413 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0.5", 2024-03-08T21:15:41.413 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "0.001", 2024-03-08T21:15:41.413 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "0.999", 2024-03-08T21:15:41.413 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.413 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "aggressiveness of optimization", 2024-03-08T21:15:41.413 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-03-08T21:15:41.413 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.413 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.413 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.413 INFO:teuthology.orchestra.run.smithi107.stdout: "end_time": { 2024-03-08T21:15:41.413 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "end_time", 2024-03-08T21:15:41.413 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.413 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.413 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.414 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "2359", 2024-03-08T21:15:41.414 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.414 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.414 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.414 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "ending time of day to automatically balance", 2024-03-08T21:15:41.414 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-08T21:15:41.414 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.414 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.414 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.414 INFO:teuthology.orchestra.run.smithi107.stdout: "end_weekday": { 2024-03-08T21:15:41.414 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "end_weekday", 2024-03-08T21:15:41.414 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "uint", 2024-03-08T21:15:41.414 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.414 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.414 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0", 2024-03-08T21:15:41.414 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "0", 2024-03-08T21:15:41.415 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "6", 2024-03-08T21:15:41.415 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.415 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-03-08T21:15:41.415 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-08T21:15:41.415 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.415 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.415 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.415 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-03-08T21:15:41.415 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-03-08T21:15:41.415 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.415 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.415 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.415 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:41.415 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.415 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.415 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:41.416 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-08T21:15:41.416 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-08T21:15:41.416 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-08T21:15:41.416 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-08T21:15:41.416 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-08T21:15:41.416 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-08T21:15:41.416 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:41.416 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.416 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.416 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.416 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.416 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.416 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-03-08T21:15:41.416 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-03-08T21:15:41.416 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.417 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.417 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.417 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:41.417 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.417 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.417 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.417 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.417 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.417 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.417 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.417 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.417 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-03-08T21:15:41.417 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-03-08T21:15:41.417 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.417 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.417 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.418 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-03-08T21:15:41.418 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.418 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.418 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:41.418 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-08T21:15:41.418 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-08T21:15:41.418 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-08T21:15:41.418 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-08T21:15:41.418 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-08T21:15:41.418 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-08T21:15:41.418 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:41.418 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.418 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.418 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.418 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.418 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.419 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-03-08T21:15:41.419 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-03-08T21:15:41.419 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.419 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.419 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.419 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:41.419 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.419 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.419 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.419 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.419 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.419 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.419 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.419 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.419 INFO:teuthology.orchestra.run.smithi107.stdout: "min_score": { 2024-03-08T21:15:41.420 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "min_score", 2024-03-08T21:15:41.420 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "float", 2024-03-08T21:15:41.420 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.420 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.420 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0", 2024-03-08T21:15:41.420 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.420 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.420 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.420 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-03-08T21:15:41.420 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.420 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.420 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.420 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.420 INFO:teuthology.orchestra.run.smithi107.stdout: "mode": { 2024-03-08T21:15:41.420 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "mode", 2024-03-08T21:15:41.420 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.421 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.421 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.421 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "upmap", 2024-03-08T21:15:41.421 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.421 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.421 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:41.421 INFO:teuthology.orchestra.run.smithi107.stdout: "crush-compat", 2024-03-08T21:15:41.421 INFO:teuthology.orchestra.run.smithi107.stdout: "none", 2024-03-08T21:15:41.421 INFO:teuthology.orchestra.run.smithi107.stdout: "read", 2024-03-08T21:15:41.421 INFO:teuthology.orchestra.run.smithi107.stdout: "upmap", 2024-03-08T21:15:41.421 INFO:teuthology.orchestra.run.smithi107.stdout: "upmap-read" 2024-03-08T21:15:41.421 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:41.421 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Balancer mode", 2024-03-08T21:15:41.421 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.421 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.421 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.422 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.422 INFO:teuthology.orchestra.run.smithi107.stdout: "pool_ids": { 2024-03-08T21:15:41.422 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "pool_ids", 2024-03-08T21:15:41.422 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.422 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.422 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.422 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:41.422 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.422 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.422 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.422 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-03-08T21:15:41.422 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.422 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.422 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.422 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.423 INFO:teuthology.orchestra.run.smithi107.stdout: "sleep_interval": { 2024-03-08T21:15:41.423 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "sleep_interval", 2024-03-08T21:15:41.423 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-03-08T21:15:41.423 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.423 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.423 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "60", 2024-03-08T21:15:41.423 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.423 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.423 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.423 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-03-08T21:15:41.423 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.423 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.423 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.423 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.423 INFO:teuthology.orchestra.run.smithi107.stdout: "upmap_max_deviation": { 2024-03-08T21:15:41.423 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "upmap_max_deviation", 2024-03-08T21:15:41.423 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-08T21:15:41.424 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.424 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.424 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "5", 2024-03-08T21:15:41.424 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "1", 2024-03-08T21:15:41.424 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.424 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.424 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "deviation below which no optimization is attempted", 2024-03-08T21:15:41.424 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-03-08T21:15:41.424 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.424 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.424 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.424 INFO:teuthology.orchestra.run.smithi107.stdout: "upmap_max_optimizations": { 2024-03-08T21:15:41.424 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "upmap_max_optimizations", 2024-03-08T21:15:41.424 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "uint", 2024-03-08T21:15:41.424 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.425 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.425 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "10", 2024-03-08T21:15:41.425 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.425 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.425 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.425 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-03-08T21:15:41.425 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.425 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.425 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.425 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:41.425 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:41.425 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.425 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-08T21:15:41.425 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "cephadm", 2024-03-08T21:15:41.425 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-03-08T21:15:41.425 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-03-08T21:15:41.426 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-03-08T21:15:41.426 INFO:teuthology.orchestra.run.smithi107.stdout: "agent_down_multiplier": { 2024-03-08T21:15:41.426 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "agent_down_multiplier", 2024-03-08T21:15:41.426 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "float", 2024-03-08T21:15:41.426 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.426 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.426 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "3.0", 2024-03-08T21:15:41.426 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.426 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.426 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.426 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-03-08T21:15:41.426 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.426 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.426 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.426 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.427 INFO:teuthology.orchestra.run.smithi107.stdout: "agent_refresh_rate": { 2024-03-08T21:15:41.427 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "agent_refresh_rate", 2024-03-08T21:15:41.427 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-03-08T21:15:41.427 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.427 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.427 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "20", 2024-03-08T21:15:41.427 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.427 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.427 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.427 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-03-08T21:15:41.427 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.427 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.427 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.427 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.427 INFO:teuthology.orchestra.run.smithi107.stdout: "agent_starting_port": { 2024-03-08T21:15:41.427 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "agent_starting_port", 2024-03-08T21:15:41.428 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-08T21:15:41.428 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.428 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.428 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "4721", 2024-03-08T21:15:41.428 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.428 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.428 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.428 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-03-08T21:15:41.428 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.428 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.428 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.428 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.428 INFO:teuthology.orchestra.run.smithi107.stdout: "allow_ptrace": { 2024-03-08T21:15:41.428 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "allow_ptrace", 2024-03-08T21:15:41.428 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.428 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.429 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.429 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:41.429 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.429 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.429 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.429 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-03-08T21:15:41.429 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-03-08T21:15:41.429 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.429 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.429 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.429 INFO:teuthology.orchestra.run.smithi107.stdout: "autotune_interval": { 2024-03-08T21:15:41.429 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "autotune_interval", 2024-03-08T21:15:41.429 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-03-08T21:15:41.429 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.429 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.429 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "600", 2024-03-08T21:15:41.430 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.430 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.430 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.430 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "how frequently to autotune daemon memory", 2024-03-08T21:15:41.430 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.430 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.430 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.430 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.430 INFO:teuthology.orchestra.run.smithi107.stdout: "autotune_memory_target_ratio": { 2024-03-08T21:15:41.430 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "autotune_memory_target_ratio", 2024-03-08T21:15:41.430 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "float", 2024-03-08T21:15:41.430 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.430 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.430 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0.7", 2024-03-08T21:15:41.431 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.431 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.431 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.431 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-03-08T21:15:41.431 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.431 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.431 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.431 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.431 INFO:teuthology.orchestra.run.smithi107.stdout: "cephadm_log_destination": { 2024-03-08T21:15:41.431 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "cephadm_log_destination", 2024-03-08T21:15:41.431 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.431 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.431 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.431 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:41.431 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.431 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.432 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:41.432 INFO:teuthology.orchestra.run.smithi107.stdout: "file", 2024-03-08T21:15:41.432 INFO:teuthology.orchestra.run.smithi107.stdout: "file,syslog", 2024-03-08T21:15:41.432 INFO:teuthology.orchestra.run.smithi107.stdout: "syslog" 2024-03-08T21:15:41.432 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:41.432 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-03-08T21:15:41.432 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.432 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.432 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.432 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.432 INFO:teuthology.orchestra.run.smithi107.stdout: "cgroups_split": { 2024-03-08T21:15:41.432 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "cgroups_split", 2024-03-08T21:15:41.432 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.432 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.432 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.432 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-08T21:15:41.433 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.433 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.433 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.433 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-03-08T21:15:41.433 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.433 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.433 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.433 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.433 INFO:teuthology.orchestra.run.smithi107.stdout: "config_checks_enabled": { 2024-03-08T21:15:41.433 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "config_checks_enabled", 2024-03-08T21:15:41.433 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.433 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.433 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.433 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:41.433 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.434 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.434 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.434 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-03-08T21:15:41.434 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.434 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.434 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.434 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.434 INFO:teuthology.orchestra.run.smithi107.stdout: "config_dashboard": { 2024-03-08T21:15:41.434 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "config_dashboard", 2024-03-08T21:15:41.434 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.434 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.434 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.434 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-08T21:15:41.435 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.435 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.435 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.435 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-03-08T21:15:41.435 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.435 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.435 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.435 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.435 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_alertmanager": { 2024-03-08T21:15:41.435 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_alertmanager", 2024-03-08T21:15:41.435 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.435 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.435 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.435 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-03-08T21:15:41.435 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.435 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.436 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.436 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Prometheus container image", 2024-03-08T21:15:41.436 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.436 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.436 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.436 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.436 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_base": { 2024-03-08T21:15:41.436 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_base", 2024-03-08T21:15:41.436 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.436 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.436 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.436 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/ceph/ceph", 2024-03-08T21:15:41.436 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.436 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.436 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.436 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Container image name, without the tag", 2024-03-08T21:15:41.437 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.437 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.437 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.437 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.437 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_elasticsearch": { 2024-03-08T21:15:41.437 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_elasticsearch", 2024-03-08T21:15:41.437 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.437 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.437 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.437 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-03-08T21:15:41.437 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.437 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.437 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.437 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "elasticsearch container image", 2024-03-08T21:15:41.438 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.438 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.438 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.438 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.438 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_grafana": { 2024-03-08T21:15:41.438 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_grafana", 2024-03-08T21:15:41.438 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.438 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.438 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.438 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.12", 2024-03-08T21:15:41.438 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.438 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.438 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.438 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Prometheus container image", 2024-03-08T21:15:41.439 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.439 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.439 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.439 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.439 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_haproxy": { 2024-03-08T21:15:41.439 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_haproxy", 2024-03-08T21:15:41.439 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.439 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.439 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.439 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-03-08T21:15:41.439 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.439 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.439 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.439 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "HAproxy container image", 2024-03-08T21:15:41.439 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.440 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.440 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.440 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.440 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_jaeger_agent": { 2024-03-08T21:15:41.440 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_jaeger_agent", 2024-03-08T21:15:41.440 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.440 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.440 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.440 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-03-08T21:15:41.440 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.440 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.440 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.440 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Jaeger agent container image", 2024-03-08T21:15:41.440 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.440 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.441 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.441 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.441 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_jaeger_collector": { 2024-03-08T21:15:41.441 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_jaeger_collector", 2024-03-08T21:15:41.441 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.441 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.441 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.441 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-03-08T21:15:41.441 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.441 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.441 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.441 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Jaeger collector container image", 2024-03-08T21:15:41.441 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.441 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.441 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.442 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.442 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_jaeger_query": { 2024-03-08T21:15:41.442 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_jaeger_query", 2024-03-08T21:15:41.442 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.442 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.442 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.442 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-03-08T21:15:41.442 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.442 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.442 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.442 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Jaeger query container image", 2024-03-08T21:15:41.442 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.442 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.442 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.442 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.443 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_keepalived": { 2024-03-08T21:15:41.443 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_keepalived", 2024-03-08T21:15:41.443 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.443 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.443 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.443 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-03-08T21:15:41.443 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.443 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.443 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.443 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Keepalived container image", 2024-03-08T21:15:41.443 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.443 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.443 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.443 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.443 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_loki": { 2024-03-08T21:15:41.444 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_loki", 2024-03-08T21:15:41.444 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.444 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.444 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.444 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-03-08T21:15:41.444 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.444 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.444 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.444 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Loki container image", 2024-03-08T21:15:41.444 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.444 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.444 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.444 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.444 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_node_exporter": { 2024-03-08T21:15:41.444 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_node_exporter", 2024-03-08T21:15:41.444 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.445 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.445 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.445 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-03-08T21:15:41.445 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.445 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.445 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.445 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Prometheus container image", 2024-03-08T21:15:41.445 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.445 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.445 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.445 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.445 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_nvmeof": { 2024-03-08T21:15:41.445 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_nvmeof", 2024-03-08T21:15:41.445 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.445 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.445 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.446 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/ceph/nvmeof:1.0.0", 2024-03-08T21:15:41.446 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.446 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.446 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.446 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Nvme-of container image", 2024-03-08T21:15:41.446 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.446 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.446 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.446 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.446 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_prometheus": { 2024-03-08T21:15:41.446 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_prometheus", 2024-03-08T21:15:41.446 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.446 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.446 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.446 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-03-08T21:15:41.447 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.447 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.447 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.447 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Prometheus container image", 2024-03-08T21:15:41.447 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.447 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.447 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.447 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.447 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_promtail": { 2024-03-08T21:15:41.447 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_promtail", 2024-03-08T21:15:41.447 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.447 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.447 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.447 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-03-08T21:15:41.447 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.447 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.448 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.448 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Promtail container image", 2024-03-08T21:15:41.448 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.448 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.448 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.448 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.448 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_snmp_gateway": { 2024-03-08T21:15:41.448 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_snmp_gateway", 2024-03-08T21:15:41.448 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.448 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.448 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.448 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-03-08T21:15:41.448 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.448 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.448 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.449 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "SNMP Gateway container image", 2024-03-08T21:15:41.449 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.449 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.449 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.449 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.449 INFO:teuthology.orchestra.run.smithi107.stdout: "container_init": { 2024-03-08T21:15:41.449 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_init", 2024-03-08T21:15:41.449 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.449 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.449 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.449 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-08T21:15:41.449 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.449 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.449 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.449 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Run podman/docker with `--init`", 2024-03-08T21:15:41.449 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.450 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.450 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.450 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.450 INFO:teuthology.orchestra.run.smithi107.stdout: "daemon_cache_timeout": { 2024-03-08T21:15:41.450 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "daemon_cache_timeout", 2024-03-08T21:15:41.450 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-03-08T21:15:41.450 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.450 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.450 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "600", 2024-03-08T21:15:41.450 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.450 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.450 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.450 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-03-08T21:15:41.450 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.450 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.451 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.451 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.451 INFO:teuthology.orchestra.run.smithi107.stdout: "default_cephadm_command_timeout": { 2024-03-08T21:15:41.451 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "default_cephadm_command_timeout", 2024-03-08T21:15:41.451 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-08T21:15:41.451 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.451 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.451 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "900", 2024-03-08T21:15:41.451 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.451 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.451 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.451 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-03-08T21:15:41.451 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.451 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.452 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.452 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.452 INFO:teuthology.orchestra.run.smithi107.stdout: "default_registry": { 2024-03-08T21:15:41.452 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "default_registry", 2024-03-08T21:15:41.452 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.452 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.452 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.452 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "docker.io", 2024-03-08T21:15:41.452 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.452 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.452 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.452 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-03-08T21:15:41.452 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.452 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.453 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.453 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.453 INFO:teuthology.orchestra.run.smithi107.stdout: "device_cache_timeout": { 2024-03-08T21:15:41.453 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "device_cache_timeout", 2024-03-08T21:15:41.453 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-03-08T21:15:41.453 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.453 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.453 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "1800", 2024-03-08T21:15:41.453 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.453 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.453 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.453 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "seconds to cache device inventory", 2024-03-08T21:15:41.453 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.453 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.453 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.453 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.454 INFO:teuthology.orchestra.run.smithi107.stdout: "device_enhanced_scan": { 2024-03-08T21:15:41.454 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "device_enhanced_scan", 2024-03-08T21:15:41.454 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.454 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.454 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.454 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:41.454 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.454 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.454 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.454 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Use libstoragemgmt during device scans", 2024-03-08T21:15:41.454 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.454 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.454 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.454 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.454 INFO:teuthology.orchestra.run.smithi107.stdout: "facts_cache_timeout": { 2024-03-08T21:15:41.455 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "facts_cache_timeout", 2024-03-08T21:15:41.455 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-03-08T21:15:41.455 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.455 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.455 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "60", 2024-03-08T21:15:41.455 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.455 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.455 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.455 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "seconds to cache host facts data", 2024-03-08T21:15:41.455 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.455 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.455 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.455 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.455 INFO:teuthology.orchestra.run.smithi107.stdout: "host_check_interval": { 2024-03-08T21:15:41.455 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "host_check_interval", 2024-03-08T21:15:41.455 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-03-08T21:15:41.456 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.456 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.456 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "600", 2024-03-08T21:15:41.456 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.456 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.456 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.456 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "how frequently to perform a host check", 2024-03-08T21:15:41.456 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.456 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.456 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.456 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.456 INFO:teuthology.orchestra.run.smithi107.stdout: "hw_monitoring": { 2024-03-08T21:15:41.456 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "hw_monitoring", 2024-03-08T21:15:41.456 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.456 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.457 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.457 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:41.457 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.457 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.457 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.457 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-03-08T21:15:41.457 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.457 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.457 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.457 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.457 INFO:teuthology.orchestra.run.smithi107.stdout: "inventory_list_all": { 2024-03-08T21:15:41.457 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "inventory_list_all", 2024-03-08T21:15:41.457 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.457 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.458 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.458 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:41.458 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.458 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.458 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.458 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-03-08T21:15:41.458 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.458 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.458 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.458 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.458 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-03-08T21:15:41.458 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-03-08T21:15:41.458 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.458 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.458 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.459 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:41.459 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.459 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.459 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:41.459 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-08T21:15:41.459 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-08T21:15:41.459 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-08T21:15:41.459 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-08T21:15:41.459 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-08T21:15:41.459 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-08T21:15:41.459 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:41.459 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.459 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.459 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.459 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.460 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.460 INFO:teuthology.orchestra.run.smithi107.stdout: "log_refresh_metadata": { 2024-03-08T21:15:41.460 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_refresh_metadata", 2024-03-08T21:15:41.460 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.460 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.460 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.460 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:41.460 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.460 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.460 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.460 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-03-08T21:15:41.460 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.460 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.460 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.460 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.461 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-03-08T21:15:41.461 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-03-08T21:15:41.461 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.461 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.461 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.461 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-08T21:15:41.461 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.461 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.461 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.461 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-03-08T21:15:41.461 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.461 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.461 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.461 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.461 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-03-08T21:15:41.461 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-03-08T21:15:41.462 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.462 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.462 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.462 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-03-08T21:15:41.462 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.462 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.462 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:41.462 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-08T21:15:41.462 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-08T21:15:41.462 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-08T21:15:41.462 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-08T21:15:41.462 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-08T21:15:41.462 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-08T21:15:41.462 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:41.462 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.463 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.463 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.463 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.463 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.463 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-03-08T21:15:41.463 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-03-08T21:15:41.463 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.463 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.463 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.463 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:41.463 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.463 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.463 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.463 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.463 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.463 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.464 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.464 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.464 INFO:teuthology.orchestra.run.smithi107.stdout: "manage_etc_ceph_ceph_conf": { 2024-03-08T21:15:41.464 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-03-08T21:15:41.464 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.464 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.464 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.464 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:41.464 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.464 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.464 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.464 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-03-08T21:15:41.464 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.464 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.465 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.465 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.465 INFO:teuthology.orchestra.run.smithi107.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-03-08T21:15:41.465 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-03-08T21:15:41.465 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.465 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.465 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.465 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "*", 2024-03-08T21:15:41.465 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.465 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.465 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.465 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-03-08T21:15:41.465 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.466 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.466 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.466 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.466 INFO:teuthology.orchestra.run.smithi107.stdout: "max_count_per_host": { 2024-03-08T21:15:41.466 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "max_count_per_host", 2024-03-08T21:15:41.466 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-08T21:15:41.466 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.466 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.466 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "10", 2024-03-08T21:15:41.466 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.466 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.466 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.466 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "max number of daemons per service per host", 2024-03-08T21:15:41.466 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.467 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.467 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.467 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.467 INFO:teuthology.orchestra.run.smithi107.stdout: "max_osd_draining_count": { 2024-03-08T21:15:41.467 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "max_osd_draining_count", 2024-03-08T21:15:41.467 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-08T21:15:41.467 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.467 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.467 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "10", 2024-03-08T21:15:41.467 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.467 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.467 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.467 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-03-08T21:15:41.467 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.468 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.468 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.468 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.468 INFO:teuthology.orchestra.run.smithi107.stdout: "migration_current": { 2024-03-08T21:15:41.468 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "migration_current", 2024-03-08T21:15:41.468 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-08T21:15:41.468 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.468 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.468 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:41.468 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.468 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.468 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.468 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "internal - do not modify", 2024-03-08T21:15:41.469 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.469 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.469 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.469 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.469 INFO:teuthology.orchestra.run.smithi107.stdout: "mode": { 2024-03-08T21:15:41.469 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "mode", 2024-03-08T21:15:41.469 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.469 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.469 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.469 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "root", 2024-03-08T21:15:41.469 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.469 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.469 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:41.469 INFO:teuthology.orchestra.run.smithi107.stdout: "cephadm-package", 2024-03-08T21:15:41.469 INFO:teuthology.orchestra.run.smithi107.stdout: "root" 2024-03-08T21:15:41.469 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:41.470 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "mode for remote execution of cephadm", 2024-03-08T21:15:41.470 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.470 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.470 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.470 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.470 INFO:teuthology.orchestra.run.smithi107.stdout: "oob_default_addr": { 2024-03-08T21:15:41.470 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "oob_default_addr", 2024-03-08T21:15:41.470 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.470 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.470 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.470 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "169.254.1.1", 2024-03-08T21:15:41.470 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.470 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.471 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.471 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Default address for RedFish API (oob management).", 2024-03-08T21:15:41.471 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.471 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.471 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.471 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.471 INFO:teuthology.orchestra.run.smithi107.stdout: "prometheus_alerts_path": { 2024-03-08T21:15:41.471 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "prometheus_alerts_path", 2024-03-08T21:15:41.471 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.471 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.471 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.471 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-03-08T21:15:41.471 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.471 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.471 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.471 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-03-08T21:15:41.472 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.472 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.472 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.472 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.472 INFO:teuthology.orchestra.run.smithi107.stdout: "registry_insecure": { 2024-03-08T21:15:41.472 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "registry_insecure", 2024-03-08T21:15:41.472 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.472 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.472 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.472 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:41.472 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.472 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.472 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.472 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-03-08T21:15:41.473 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.473 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.473 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.473 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.473 INFO:teuthology.orchestra.run.smithi107.stdout: "registry_password": { 2024-03-08T21:15:41.473 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "registry_password", 2024-03-08T21:15:41.473 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.473 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.473 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.473 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:41.473 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.473 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.473 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.473 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-03-08T21:15:41.473 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.473 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.474 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.474 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.474 INFO:teuthology.orchestra.run.smithi107.stdout: "registry_url": { 2024-03-08T21:15:41.474 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "registry_url", 2024-03-08T21:15:41.474 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.474 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.474 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.474 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:41.474 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.474 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.474 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.474 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-03-08T21:15:41.474 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.474 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.474 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.474 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.475 INFO:teuthology.orchestra.run.smithi107.stdout: "registry_username": { 2024-03-08T21:15:41.475 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "registry_username", 2024-03-08T21:15:41.475 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.475 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.475 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.475 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:41.475 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.475 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.475 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.475 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-03-08T21:15:41.475 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.475 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.475 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.475 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.475 INFO:teuthology.orchestra.run.smithi107.stdout: "secure_monitoring_stack": { 2024-03-08T21:15:41.476 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "secure_monitoring_stack", 2024-03-08T21:15:41.476 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.476 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.476 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.476 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:41.476 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.476 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.476 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.476 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-03-08T21:15:41.476 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.476 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.476 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.476 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.476 INFO:teuthology.orchestra.run.smithi107.stdout: "service_discovery_port": { 2024-03-08T21:15:41.476 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "service_discovery_port", 2024-03-08T21:15:41.476 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-08T21:15:41.476 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.477 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.477 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "8765", 2024-03-08T21:15:41.477 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.477 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.477 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.477 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "cephadm service discovery port", 2024-03-08T21:15:41.477 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.477 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.477 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.477 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.477 INFO:teuthology.orchestra.run.smithi107.stdout: "ssh_config_file": { 2024-03-08T21:15:41.477 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "ssh_config_file", 2024-03-08T21:15:41.477 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.477 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.477 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.478 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:41.478 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.478 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.478 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.478 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-03-08T21:15:41.478 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.478 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.478 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.478 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.478 INFO:teuthology.orchestra.run.smithi107.stdout: "use_agent": { 2024-03-08T21:15:41.478 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "use_agent", 2024-03-08T21:15:41.478 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.478 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.478 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.478 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:41.479 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.479 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.479 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.479 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-03-08T21:15:41.479 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.479 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.479 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.479 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.479 INFO:teuthology.orchestra.run.smithi107.stdout: "use_repo_digest": { 2024-03-08T21:15:41.479 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "use_repo_digest", 2024-03-08T21:15:41.479 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.479 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.479 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.479 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-08T21:15:41.479 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.479 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.480 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.480 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-03-08T21:15:41.480 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.480 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.480 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.480 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.480 INFO:teuthology.orchestra.run.smithi107.stdout: "warn_on_failed_host_check": { 2024-03-08T21:15:41.480 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "warn_on_failed_host_check", 2024-03-08T21:15:41.480 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.480 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.480 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.480 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-08T21:15:41.480 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.480 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.480 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.481 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "raise a health warning if the host check fails", 2024-03-08T21:15:41.481 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.481 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.481 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.481 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.481 INFO:teuthology.orchestra.run.smithi107.stdout: "warn_on_stray_daemons": { 2024-03-08T21:15:41.481 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "warn_on_stray_daemons", 2024-03-08T21:15:41.481 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.481 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.481 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.481 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-08T21:15:41.481 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.481 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.481 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.481 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-03-08T21:15:41.482 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.482 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.482 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.482 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.482 INFO:teuthology.orchestra.run.smithi107.stdout: "warn_on_stray_hosts": { 2024-03-08T21:15:41.482 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "warn_on_stray_hosts", 2024-03-08T21:15:41.482 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.482 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.482 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.482 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-08T21:15:41.482 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.482 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.482 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.482 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-03-08T21:15:41.483 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.483 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.483 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.483 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:41.483 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:41.483 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.483 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-08T21:15:41.483 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "crash", 2024-03-08T21:15:41.483 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-03-08T21:15:41.483 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-03-08T21:15:41.483 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-03-08T21:15:41.483 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-03-08T21:15:41.483 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-03-08T21:15:41.483 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.483 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.483 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.484 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:41.484 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.484 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.484 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:41.484 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-08T21:15:41.484 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-08T21:15:41.484 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-08T21:15:41.484 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-08T21:15:41.484 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-08T21:15:41.484 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-08T21:15:41.484 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:41.484 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.484 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.484 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.484 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.485 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.485 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-03-08T21:15:41.485 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-03-08T21:15:41.485 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.485 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.485 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.485 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:41.485 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.485 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.485 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.485 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.485 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.485 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.485 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.485 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.485 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-03-08T21:15:41.486 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-03-08T21:15:41.486 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.486 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.486 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.486 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-03-08T21:15:41.486 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.486 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.486 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:41.486 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-08T21:15:41.486 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-08T21:15:41.486 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-08T21:15:41.486 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-08T21:15:41.486 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-08T21:15:41.486 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-08T21:15:41.486 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:41.487 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.487 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.487 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.487 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.487 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.487 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-03-08T21:15:41.487 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-03-08T21:15:41.487 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.487 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.487 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.487 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:41.487 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.487 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.487 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.487 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.488 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.488 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.488 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.488 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.488 INFO:teuthology.orchestra.run.smithi107.stdout: "retain_interval": { 2024-03-08T21:15:41.488 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "retain_interval", 2024-03-08T21:15:41.488 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-03-08T21:15:41.488 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.488 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.488 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "31536000", 2024-03-08T21:15:41.488 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.488 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.488 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.488 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "how long to retain crashes before pruning them", 2024-03-08T21:15:41.488 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.488 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.489 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.489 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.489 INFO:teuthology.orchestra.run.smithi107.stdout: "warn_recent_interval": { 2024-03-08T21:15:41.489 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "warn_recent_interval", 2024-03-08T21:15:41.489 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-03-08T21:15:41.489 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.489 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.489 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "1209600", 2024-03-08T21:15:41.489 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.489 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.489 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.489 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "time interval in which to warn about recent crashes", 2024-03-08T21:15:41.489 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.489 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.489 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.489 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:41.490 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:41.490 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.490 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-08T21:15:41.490 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "dashboard", 2024-03-08T21:15:41.490 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-03-08T21:15:41.490 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-03-08T21:15:41.490 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-03-08T21:15:41.490 INFO:teuthology.orchestra.run.smithi107.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-03-08T21:15:41.490 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-03-08T21:15:41.490 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-08T21:15:41.490 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.490 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.490 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "10", 2024-03-08T21:15:41.490 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.490 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.490 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.491 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.491 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.491 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.491 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.491 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.491 INFO:teuthology.orchestra.run.smithi107.stdout: "ALERTMANAGER_API_HOST": { 2024-03-08T21:15:41.491 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "ALERTMANAGER_API_HOST", 2024-03-08T21:15:41.491 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.491 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.491 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.491 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:41.491 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.491 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.491 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.491 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.491 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.492 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.492 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.492 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.492 INFO:teuthology.orchestra.run.smithi107.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-03-08T21:15:41.492 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-03-08T21:15:41.492 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.492 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.492 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.492 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-08T21:15:41.492 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.492 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.492 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.492 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.492 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.492 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.492 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.492 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.493 INFO:teuthology.orchestra.run.smithi107.stdout: "AUDIT_API_ENABLED": { 2024-03-08T21:15:41.493 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "AUDIT_API_ENABLED", 2024-03-08T21:15:41.493 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.493 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.493 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.493 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:41.493 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.493 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.493 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.493 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.493 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.493 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.493 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.493 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.493 INFO:teuthology.orchestra.run.smithi107.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-03-08T21:15:41.494 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-03-08T21:15:41.494 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.494 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.494 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.494 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-08T21:15:41.494 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.494 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.494 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.494 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.494 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.494 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.494 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.494 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.494 INFO:teuthology.orchestra.run.smithi107.stdout: "ENABLE_BROWSABLE_API": { 2024-03-08T21:15:41.494 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "ENABLE_BROWSABLE_API", 2024-03-08T21:15:41.495 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.495 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.495 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.495 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-08T21:15:41.495 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.495 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.495 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.495 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.495 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.495 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.495 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.495 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.495 INFO:teuthology.orchestra.run.smithi107.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-03-08T21:15:41.495 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-03-08T21:15:41.496 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.496 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.496 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.496 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-08T21:15:41.496 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.496 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.496 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.496 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.496 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.496 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.496 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.496 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.496 INFO:teuthology.orchestra.run.smithi107.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-03-08T21:15:41.496 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-03-08T21:15:41.496 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.497 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.497 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.497 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-08T21:15:41.497 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.497 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.497 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.497 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.497 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.497 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.497 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.497 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.497 INFO:teuthology.orchestra.run.smithi107.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-03-08T21:15:41.497 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-03-08T21:15:41.497 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.497 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.497 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.498 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-08T21:15:41.498 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.498 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.498 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.498 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.498 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.498 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.498 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.498 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.498 INFO:teuthology.orchestra.run.smithi107.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-03-08T21:15:41.498 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-03-08T21:15:41.498 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.498 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.498 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.498 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-08T21:15:41.499 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.499 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.499 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.499 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.499 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.499 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.499 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.499 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.499 INFO:teuthology.orchestra.run.smithi107.stdout: "FEATURE_TOGGLE_NFS": { 2024-03-08T21:15:41.499 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-03-08T21:15:41.499 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.499 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.499 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.499 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-08T21:15:41.499 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.499 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.500 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.500 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.500 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.500 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.500 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.500 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.500 INFO:teuthology.orchestra.run.smithi107.stdout: "FEATURE_TOGGLE_RBD": { 2024-03-08T21:15:41.500 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-03-08T21:15:41.500 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.500 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.500 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.500 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-08T21:15:41.500 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.500 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.500 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.500 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.501 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.501 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.501 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.501 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.501 INFO:teuthology.orchestra.run.smithi107.stdout: "FEATURE_TOGGLE_RGW": { 2024-03-08T21:15:41.501 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-03-08T21:15:41.501 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.501 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.501 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.501 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-08T21:15:41.501 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.501 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.501 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.501 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.501 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.501 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.502 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.502 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.502 INFO:teuthology.orchestra.run.smithi107.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-03-08T21:15:41.502 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-03-08T21:15:41.502 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.502 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.502 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.502 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:41.502 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.502 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.502 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.502 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.502 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.502 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.502 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.503 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.503 INFO:teuthology.orchestra.run.smithi107.stdout: "GRAFANA_API_PASSWORD": { 2024-03-08T21:15:41.503 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "GRAFANA_API_PASSWORD", 2024-03-08T21:15:41.503 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.503 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.503 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.503 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "admin", 2024-03-08T21:15:41.503 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.503 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.503 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.503 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.503 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.503 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.503 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.503 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.504 INFO:teuthology.orchestra.run.smithi107.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-03-08T21:15:41.504 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-03-08T21:15:41.504 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.504 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.504 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.504 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-08T21:15:41.504 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.504 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.504 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.504 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.504 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.504 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.504 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.504 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.504 INFO:teuthology.orchestra.run.smithi107.stdout: "GRAFANA_API_URL": { 2024-03-08T21:15:41.504 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "GRAFANA_API_URL", 2024-03-08T21:15:41.505 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.505 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.505 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.505 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:41.505 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.505 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.505 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.505 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.505 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.505 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.505 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.505 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.505 INFO:teuthology.orchestra.run.smithi107.stdout: "GRAFANA_API_USERNAME": { 2024-03-08T21:15:41.505 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "GRAFANA_API_USERNAME", 2024-03-08T21:15:41.505 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.505 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.506 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.506 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "admin", 2024-03-08T21:15:41.506 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.506 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.506 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.506 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.506 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.506 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.506 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.506 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.506 INFO:teuthology.orchestra.run.smithi107.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-03-08T21:15:41.506 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-03-08T21:15:41.506 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.506 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.506 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.506 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:41.506 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.507 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.507 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.507 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.507 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.507 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.507 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.507 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.507 INFO:teuthology.orchestra.run.smithi107.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-03-08T21:15:41.507 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-03-08T21:15:41.507 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.507 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.507 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.507 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:41.507 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.507 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.508 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.508 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.508 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.508 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.508 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.508 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.508 INFO:teuthology.orchestra.run.smithi107.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-03-08T21:15:41.508 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-03-08T21:15:41.508 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.508 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.508 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.508 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-08T21:15:41.508 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.508 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.508 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.508 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.509 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.509 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.509 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.509 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.509 INFO:teuthology.orchestra.run.smithi107.stdout: "ISSUE_TRACKER_API_KEY": { 2024-03-08T21:15:41.509 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-03-08T21:15:41.509 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.509 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.509 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.509 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:41.509 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.509 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.509 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.509 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.510 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.510 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.510 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.510 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.510 INFO:teuthology.orchestra.run.smithi107.stdout: "MULTICLUSTER_CONFIG": { 2024-03-08T21:15:41.510 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "MULTICLUSTER_CONFIG", 2024-03-08T21:15:41.510 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.510 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.510 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.510 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "{}", 2024-03-08T21:15:41.510 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.510 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.510 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.510 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.510 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.510 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.511 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.511 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.511 INFO:teuthology.orchestra.run.smithi107.stdout: "PROMETHEUS_API_HOST": { 2024-03-08T21:15:41.511 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PROMETHEUS_API_HOST", 2024-03-08T21:15:41.511 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.511 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.511 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.511 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:41.511 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.511 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.511 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.511 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.511 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.511 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.511 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.511 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.512 INFO:teuthology.orchestra.run.smithi107.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-03-08T21:15:41.512 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-03-08T21:15:41.512 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.512 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.512 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.512 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-08T21:15:41.512 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.512 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.512 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.512 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.512 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.512 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.512 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.512 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.512 INFO:teuthology.orchestra.run.smithi107.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-03-08T21:15:41.512 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-03-08T21:15:41.513 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.513 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.513 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.513 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:41.513 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.513 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.513 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.513 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.513 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.513 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.513 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.513 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.513 INFO:teuthology.orchestra.run.smithi107.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-03-08T21:15:41.513 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-03-08T21:15:41.513 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.513 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.514 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.514 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:41.514 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.514 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.514 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.514 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.514 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.514 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.514 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.514 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.514 INFO:teuthology.orchestra.run.smithi107.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-03-08T21:15:41.514 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-03-08T21:15:41.514 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.514 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.514 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.514 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-08T21:15:41.515 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.515 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.515 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.515 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.515 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.515 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.515 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.515 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.515 INFO:teuthology.orchestra.run.smithi107.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-03-08T21:15:41.515 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-03-08T21:15:41.515 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.515 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.515 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.515 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-08T21:15:41.515 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.515 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.516 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.516 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.516 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.516 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.516 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.516 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.516 INFO:teuthology.orchestra.run.smithi107.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-03-08T21:15:41.516 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-03-08T21:15:41.516 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.516 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.516 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.516 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:41.516 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.516 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.516 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.516 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.517 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.517 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.517 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.517 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.517 INFO:teuthology.orchestra.run.smithi107.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-03-08T21:15:41.517 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-03-08T21:15:41.517 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.517 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.517 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.517 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:41.517 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.517 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.517 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.517 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.517 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.517 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.518 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.518 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.518 INFO:teuthology.orchestra.run.smithi107.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-03-08T21:15:41.518 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-03-08T21:15:41.518 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.518 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.518 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.518 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:41.518 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.518 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.518 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.518 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.518 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.518 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.518 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.519 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.519 INFO:teuthology.orchestra.run.smithi107.stdout: "PWD_POLICY_ENABLED": { 2024-03-08T21:15:41.519 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PWD_POLICY_ENABLED", 2024-03-08T21:15:41.519 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.519 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.519 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.519 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-08T21:15:41.519 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.519 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.519 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.519 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.519 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.519 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.519 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.519 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.519 INFO:teuthology.orchestra.run.smithi107.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-03-08T21:15:41.520 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-03-08T21:15:41.520 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.520 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.520 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.520 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-03-08T21:15:41.520 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.520 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.520 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.520 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.520 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.520 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.520 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.520 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.520 INFO:teuthology.orchestra.run.smithi107.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-03-08T21:15:41.520 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-03-08T21:15:41.520 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-08T21:15:41.520 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.521 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.521 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "10", 2024-03-08T21:15:41.521 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.521 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.521 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.521 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.521 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.521 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.521 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.521 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.521 INFO:teuthology.orchestra.run.smithi107.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-03-08T21:15:41.521 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-03-08T21:15:41.521 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-08T21:15:41.521 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.521 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.522 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "8", 2024-03-08T21:15:41.522 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.522 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.522 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.522 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.522 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.522 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.522 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.522 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.522 INFO:teuthology.orchestra.run.smithi107.stdout: "REST_REQUESTS_TIMEOUT": { 2024-03-08T21:15:41.522 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-03-08T21:15:41.522 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-08T21:15:41.522 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.522 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.522 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "45", 2024-03-08T21:15:41.522 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.523 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.523 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.523 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.523 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.523 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.523 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.523 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.523 INFO:teuthology.orchestra.run.smithi107.stdout: "RGW_API_ACCESS_KEY": { 2024-03-08T21:15:41.523 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "RGW_API_ACCESS_KEY", 2024-03-08T21:15:41.523 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.523 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.523 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.523 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:41.523 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.524 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.524 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.524 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.524 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.524 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.524 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.524 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.524 INFO:teuthology.orchestra.run.smithi107.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-03-08T21:15:41.524 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-03-08T21:15:41.524 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.524 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.524 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.524 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "admin", 2024-03-08T21:15:41.524 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.524 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.524 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.525 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.525 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.525 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.525 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.525 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.525 INFO:teuthology.orchestra.run.smithi107.stdout: "RGW_API_SECRET_KEY": { 2024-03-08T21:15:41.525 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "RGW_API_SECRET_KEY", 2024-03-08T21:15:41.525 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.525 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.525 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.525 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:41.525 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.525 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.525 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.525 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.526 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.526 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.526 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.526 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.526 INFO:teuthology.orchestra.run.smithi107.stdout: "RGW_API_SSL_VERIFY": { 2024-03-08T21:15:41.526 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "RGW_API_SSL_VERIFY", 2024-03-08T21:15:41.526 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.526 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.526 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.526 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-08T21:15:41.526 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.526 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.526 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.526 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.526 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.527 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.527 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.527 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.527 INFO:teuthology.orchestra.run.smithi107.stdout: "UNSAFE_TLS_v1_2": { 2024-03-08T21:15:41.527 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "UNSAFE_TLS_v1_2", 2024-03-08T21:15:41.527 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.527 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.527 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.527 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:41.527 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.527 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.527 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.527 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.527 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.528 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.528 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.528 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.528 INFO:teuthology.orchestra.run.smithi107.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-03-08T21:15:41.528 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-03-08T21:15:41.528 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-08T21:15:41.528 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.528 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.528 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0", 2024-03-08T21:15:41.528 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.528 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.528 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.528 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.528 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.528 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.529 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.529 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.529 INFO:teuthology.orchestra.run.smithi107.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-03-08T21:15:41.529 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-03-08T21:15:41.529 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-08T21:15:41.529 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.529 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.529 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "10", 2024-03-08T21:15:41.529 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.529 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.529 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.529 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.529 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.529 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.529 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.530 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.530 INFO:teuthology.orchestra.run.smithi107.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-03-08T21:15:41.530 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-03-08T21:15:41.530 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-08T21:15:41.530 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.530 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.530 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "5", 2024-03-08T21:15:41.530 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.530 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.530 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.530 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.530 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.530 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.530 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.530 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.530 INFO:teuthology.orchestra.run.smithi107.stdout: "cross_origin_url": { 2024-03-08T21:15:41.531 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "cross_origin_url", 2024-03-08T21:15:41.531 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.531 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.531 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.531 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:41.531 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.531 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.531 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.531 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.531 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.531 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.531 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.531 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.531 INFO:teuthology.orchestra.run.smithi107.stdout: "crt_file": { 2024-03-08T21:15:41.531 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "crt_file", 2024-03-08T21:15:41.531 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.532 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.532 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.532 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:41.532 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.532 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.532 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.532 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.532 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.532 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.532 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.532 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.532 INFO:teuthology.orchestra.run.smithi107.stdout: "debug": { 2024-03-08T21:15:41.532 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "debug", 2024-03-08T21:15:41.532 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.532 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.532 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.533 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:41.533 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.533 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.533 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.533 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Enable/disable debug options", 2024-03-08T21:15:41.533 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.533 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.533 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.533 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.533 INFO:teuthology.orchestra.run.smithi107.stdout: "jwt_token_ttl": { 2024-03-08T21:15:41.533 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "jwt_token_ttl", 2024-03-08T21:15:41.533 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-08T21:15:41.533 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.533 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.533 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "28800", 2024-03-08T21:15:41.533 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.533 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.534 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.534 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.534 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.534 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.534 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.534 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.534 INFO:teuthology.orchestra.run.smithi107.stdout: "key_file": { 2024-03-08T21:15:41.534 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "key_file", 2024-03-08T21:15:41.534 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.534 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.534 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.534 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:41.534 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.534 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.534 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.535 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.535 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.535 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.535 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.535 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.535 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-03-08T21:15:41.535 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-03-08T21:15:41.535 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.535 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.535 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.535 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:41.535 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.535 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.535 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:41.535 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-08T21:15:41.535 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-08T21:15:41.535 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-08T21:15:41.536 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-08T21:15:41.536 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-08T21:15:41.536 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-08T21:15:41.536 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:41.536 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.536 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.536 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.536 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.536 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.536 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-03-08T21:15:41.536 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-03-08T21:15:41.536 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.536 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.536 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.536 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:41.536 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.537 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.537 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.537 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.537 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.537 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.537 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.537 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.537 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-03-08T21:15:41.537 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-03-08T21:15:41.537 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.537 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.537 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.537 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-03-08T21:15:41.537 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.537 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.537 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:41.538 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-08T21:15:41.538 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-08T21:15:41.538 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-08T21:15:41.538 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-08T21:15:41.538 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-08T21:15:41.538 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-08T21:15:41.538 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:41.538 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.538 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.538 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.538 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.538 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.538 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-03-08T21:15:41.538 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-03-08T21:15:41.539 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.539 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.539 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.539 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:41.539 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.539 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.539 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.539 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.539 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.539 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.539 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.539 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.539 INFO:teuthology.orchestra.run.smithi107.stdout: "motd": { 2024-03-08T21:15:41.539 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "motd", 2024-03-08T21:15:41.539 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.539 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.540 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.540 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:41.540 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.540 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.540 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.540 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "The message of the day", 2024-03-08T21:15:41.540 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.540 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.540 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.540 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.540 INFO:teuthology.orchestra.run.smithi107.stdout: "redirect_resolve_ip_addr": { 2024-03-08T21:15:41.540 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "redirect_resolve_ip_addr", 2024-03-08T21:15:41.540 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.540 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.540 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.540 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:41.541 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.541 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.541 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.541 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.541 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.541 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.541 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.541 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.541 INFO:teuthology.orchestra.run.smithi107.stdout: "server_addr": { 2024-03-08T21:15:41.541 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "server_addr", 2024-03-08T21:15:41.541 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.541 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.541 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.541 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "::", 2024-03-08T21:15:41.541 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.541 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.542 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.542 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.542 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.542 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.542 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.542 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.542 INFO:teuthology.orchestra.run.smithi107.stdout: "server_port": { 2024-03-08T21:15:41.542 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "server_port", 2024-03-08T21:15:41.542 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-08T21:15:41.542 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.542 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.542 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "8080", 2024-03-08T21:15:41.542 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.542 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.542 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.542 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.543 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.543 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.543 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.543 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.543 INFO:teuthology.orchestra.run.smithi107.stdout: "ssl": { 2024-03-08T21:15:41.543 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "ssl", 2024-03-08T21:15:41.543 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.543 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.543 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.543 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-08T21:15:41.543 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.543 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.543 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.543 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.543 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.543 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.544 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.544 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.544 INFO:teuthology.orchestra.run.smithi107.stdout: "ssl_server_port": { 2024-03-08T21:15:41.544 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "ssl_server_port", 2024-03-08T21:15:41.544 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-08T21:15:41.544 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.544 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.544 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "8443", 2024-03-08T21:15:41.544 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.544 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.544 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.544 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.544 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.544 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.544 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.544 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.545 INFO:teuthology.orchestra.run.smithi107.stdout: "standby_behaviour": { 2024-03-08T21:15:41.545 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "standby_behaviour", 2024-03-08T21:15:41.545 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.545 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.545 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.545 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "redirect", 2024-03-08T21:15:41.545 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.545 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.545 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:41.545 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-08T21:15:41.545 INFO:teuthology.orchestra.run.smithi107.stdout: "redirect" 2024-03-08T21:15:41.545 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:41.545 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.545 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.545 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.545 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.546 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.546 INFO:teuthology.orchestra.run.smithi107.stdout: "standby_error_status_code": { 2024-03-08T21:15:41.546 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "standby_error_status_code", 2024-03-08T21:15:41.546 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-08T21:15:41.546 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.546 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.546 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "500", 2024-03-08T21:15:41.546 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "400", 2024-03-08T21:15:41.546 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "599", 2024-03-08T21:15:41.546 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.546 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.546 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.546 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.546 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.546 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.547 INFO:teuthology.orchestra.run.smithi107.stdout: "url_prefix": { 2024-03-08T21:15:41.547 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "url_prefix", 2024-03-08T21:15:41.547 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.547 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.547 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.547 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:41.547 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.547 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.547 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.547 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.547 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.547 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.547 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.547 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:41.547 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:41.547 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.548 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-08T21:15:41.548 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "devicehealth", 2024-03-08T21:15:41.548 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-03-08T21:15:41.548 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-03-08T21:15:41.548 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-03-08T21:15:41.548 INFO:teuthology.orchestra.run.smithi107.stdout: "enable_monitoring": { 2024-03-08T21:15:41.548 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "enable_monitoring", 2024-03-08T21:15:41.548 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.548 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.548 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.548 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-08T21:15:41.548 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.548 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.548 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.548 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "monitor device health metrics", 2024-03-08T21:15:41.548 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.549 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.549 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.549 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.549 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-03-08T21:15:41.549 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-03-08T21:15:41.549 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.549 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.549 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.549 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:41.549 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.549 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.549 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:41.549 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-08T21:15:41.549 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-08T21:15:41.549 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-08T21:15:41.549 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-08T21:15:41.550 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-08T21:15:41.550 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-08T21:15:41.550 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:41.550 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.550 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.550 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.550 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.550 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.550 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-03-08T21:15:41.550 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-03-08T21:15:41.550 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.550 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.550 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.550 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:41.550 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.550 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.550 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.551 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.551 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.551 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.551 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.551 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.551 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-03-08T21:15:41.551 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-03-08T21:15:41.551 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.551 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.551 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.551 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-03-08T21:15:41.551 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.551 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.551 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:41.551 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-08T21:15:41.551 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-08T21:15:41.552 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-08T21:15:41.552 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-08T21:15:41.552 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-08T21:15:41.552 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-08T21:15:41.552 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:41.552 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.552 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.552 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.552 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.552 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.552 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-03-08T21:15:41.552 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-03-08T21:15:41.552 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.552 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.552 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.553 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:41.553 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.553 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.553 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.553 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.553 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.553 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.553 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.553 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.553 INFO:teuthology.orchestra.run.smithi107.stdout: "mark_out_threshold": { 2024-03-08T21:15:41.553 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "mark_out_threshold", 2024-03-08T21:15:41.553 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-03-08T21:15:41.553 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.553 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.553 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "2419200", 2024-03-08T21:15:41.554 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.554 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.554 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.554 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-03-08T21:15:41.554 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.554 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.554 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.554 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.554 INFO:teuthology.orchestra.run.smithi107.stdout: "pool_name": { 2024-03-08T21:15:41.554 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "pool_name", 2024-03-08T21:15:41.554 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.554 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.554 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.554 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "device_health_metrics", 2024-03-08T21:15:41.554 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.555 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.555 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.555 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "name of pool in which to store device health metrics", 2024-03-08T21:15:41.555 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.555 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.555 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.555 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.555 INFO:teuthology.orchestra.run.smithi107.stdout: "retention_period": { 2024-03-08T21:15:41.555 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "retention_period", 2024-03-08T21:15:41.555 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-03-08T21:15:41.555 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.555 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.555 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "15552000", 2024-03-08T21:15:41.555 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.555 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.555 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.556 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "how long to retain device health metrics", 2024-03-08T21:15:41.556 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.556 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.556 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.556 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.556 INFO:teuthology.orchestra.run.smithi107.stdout: "scrape_frequency": { 2024-03-08T21:15:41.556 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "scrape_frequency", 2024-03-08T21:15:41.556 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-03-08T21:15:41.556 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.556 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.556 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "86400", 2024-03-08T21:15:41.556 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.556 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.556 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.556 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "how frequently to scrape device health metrics", 2024-03-08T21:15:41.556 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.557 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.557 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.557 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.557 INFO:teuthology.orchestra.run.smithi107.stdout: "self_heal": { 2024-03-08T21:15:41.557 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "self_heal", 2024-03-08T21:15:41.557 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.557 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.557 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.557 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-08T21:15:41.557 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.557 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.557 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.557 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-03-08T21:15:41.557 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.557 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.558 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.558 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.558 INFO:teuthology.orchestra.run.smithi107.stdout: "sleep_interval": { 2024-03-08T21:15:41.558 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "sleep_interval", 2024-03-08T21:15:41.558 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-03-08T21:15:41.558 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.558 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.558 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "600", 2024-03-08T21:15:41.558 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.558 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.558 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.558 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "how frequently to wake up and check device health", 2024-03-08T21:15:41.558 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.558 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.558 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.559 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.559 INFO:teuthology.orchestra.run.smithi107.stdout: "warn_threshold": { 2024-03-08T21:15:41.559 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "warn_threshold", 2024-03-08T21:15:41.559 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-03-08T21:15:41.559 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.559 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.559 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "7257600", 2024-03-08T21:15:41.559 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.559 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.559 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.559 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-03-08T21:15:41.559 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.559 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.559 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.559 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:41.559 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:41.560 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.560 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-08T21:15:41.560 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "diskprediction_local", 2024-03-08T21:15:41.560 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-03-08T21:15:41.560 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-03-08T21:15:41.560 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-03-08T21:15:41.560 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-03-08T21:15:41.560 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-03-08T21:15:41.560 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.560 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.560 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.560 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:41.560 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.560 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.560 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:41.560 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-08T21:15:41.561 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-08T21:15:41.561 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-08T21:15:41.561 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-08T21:15:41.561 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-08T21:15:41.561 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-08T21:15:41.561 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:41.561 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.561 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.561 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.561 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.561 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.561 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-03-08T21:15:41.561 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-03-08T21:15:41.561 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.561 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.562 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.562 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:41.562 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.562 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.562 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.562 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.562 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.562 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.562 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.562 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.562 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-03-08T21:15:41.562 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-03-08T21:15:41.562 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.562 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.563 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.563 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-03-08T21:15:41.563 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.563 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.563 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:41.563 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-08T21:15:41.563 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-08T21:15:41.563 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-08T21:15:41.563 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-08T21:15:41.563 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-08T21:15:41.563 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-08T21:15:41.563 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:41.563 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.564 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.564 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.564 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.564 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.564 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-03-08T21:15:41.564 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-03-08T21:15:41.564 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.564 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.564 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.564 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:41.564 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.564 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.564 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.564 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.564 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.564 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.565 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.565 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.565 INFO:teuthology.orchestra.run.smithi107.stdout: "predict_interval": { 2024-03-08T21:15:41.565 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "predict_interval", 2024-03-08T21:15:41.565 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.565 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.565 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.565 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "86400", 2024-03-08T21:15:41.565 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.565 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.565 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.565 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.565 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.565 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.565 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.565 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.566 INFO:teuthology.orchestra.run.smithi107.stdout: "predictor_model": { 2024-03-08T21:15:41.566 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "predictor_model", 2024-03-08T21:15:41.566 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.566 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.566 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.566 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "prophetstor", 2024-03-08T21:15:41.566 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.566 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.566 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.566 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.566 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.566 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.566 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.566 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.566 INFO:teuthology.orchestra.run.smithi107.stdout: "sleep_interval": { 2024-03-08T21:15:41.567 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "sleep_interval", 2024-03-08T21:15:41.567 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.567 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.567 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.567 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "600", 2024-03-08T21:15:41.567 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.567 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.567 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.567 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.567 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.567 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.567 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.567 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:41.567 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:41.567 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.568 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-08T21:15:41.568 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "influx", 2024-03-08T21:15:41.568 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": false, 2024-03-08T21:15:41.568 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "influxdb python module not found", 2024-03-08T21:15:41.568 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-03-08T21:15:41.568 INFO:teuthology.orchestra.run.smithi107.stdout: "batch_size": { 2024-03-08T21:15:41.568 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "batch_size", 2024-03-08T21:15:41.568 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-08T21:15:41.568 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.568 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.568 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "5000", 2024-03-08T21:15:41.568 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.568 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.568 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.568 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-03-08T21:15:41.568 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.569 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.569 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.569 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.569 INFO:teuthology.orchestra.run.smithi107.stdout: "database": { 2024-03-08T21:15:41.569 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "database", 2024-03-08T21:15:41.569 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.569 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.569 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.569 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "ceph", 2024-03-08T21:15:41.569 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.569 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.569 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.569 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-03-08T21:15:41.569 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.569 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.569 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.570 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.570 INFO:teuthology.orchestra.run.smithi107.stdout: "hostname": { 2024-03-08T21:15:41.570 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "hostname", 2024-03-08T21:15:41.570 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.570 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.570 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.570 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:41.570 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.570 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.570 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.570 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "InfluxDB server hostname", 2024-03-08T21:15:41.570 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.570 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.570 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.570 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.571 INFO:teuthology.orchestra.run.smithi107.stdout: "interval": { 2024-03-08T21:15:41.571 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "interval", 2024-03-08T21:15:41.571 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-03-08T21:15:41.571 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.571 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.571 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "30", 2024-03-08T21:15:41.571 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "5", 2024-03-08T21:15:41.571 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.571 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.571 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-03-08T21:15:41.571 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.571 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.571 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.571 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.571 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-03-08T21:15:41.571 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-03-08T21:15:41.572 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.572 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.572 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.572 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:41.572 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.572 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.572 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:41.572 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-08T21:15:41.572 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-08T21:15:41.572 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-08T21:15:41.572 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-08T21:15:41.572 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-08T21:15:41.572 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-08T21:15:41.572 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:41.572 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.572 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.573 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.573 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.573 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.573 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-03-08T21:15:41.573 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-03-08T21:15:41.573 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.573 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.573 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.573 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:41.573 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.573 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.573 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.573 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.573 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.573 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.573 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.574 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.574 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-03-08T21:15:41.574 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-03-08T21:15:41.574 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.574 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.574 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.574 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-03-08T21:15:41.574 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.574 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.574 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:41.574 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-08T21:15:41.574 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-08T21:15:41.574 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-08T21:15:41.574 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-08T21:15:41.574 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-08T21:15:41.575 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-08T21:15:41.575 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:41.575 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.575 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.575 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.575 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.575 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.575 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-03-08T21:15:41.575 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-03-08T21:15:41.575 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.575 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.575 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.575 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:41.575 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.575 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.575 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.576 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.576 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.576 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.576 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.576 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.576 INFO:teuthology.orchestra.run.smithi107.stdout: "password": { 2024-03-08T21:15:41.576 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "password", 2024-03-08T21:15:41.576 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.576 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.576 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.576 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:41.576 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.576 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.576 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.576 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "password of InfluxDB server user", 2024-03-08T21:15:41.576 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.577 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.577 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.577 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.577 INFO:teuthology.orchestra.run.smithi107.stdout: "port": { 2024-03-08T21:15:41.577 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "port", 2024-03-08T21:15:41.577 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-08T21:15:41.577 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.577 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.577 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "8086", 2024-03-08T21:15:41.577 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.577 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.577 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.577 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "InfluxDB server port", 2024-03-08T21:15:41.577 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.577 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.577 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.578 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.578 INFO:teuthology.orchestra.run.smithi107.stdout: "ssl": { 2024-03-08T21:15:41.578 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "ssl", 2024-03-08T21:15:41.578 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.578 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.578 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.578 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "false", 2024-03-08T21:15:41.578 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.578 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.578 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.578 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-03-08T21:15:41.578 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.578 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.578 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.578 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.579 INFO:teuthology.orchestra.run.smithi107.stdout: "threads": { 2024-03-08T21:15:41.579 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "threads", 2024-03-08T21:15:41.579 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-08T21:15:41.579 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.579 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.579 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "5", 2024-03-08T21:15:41.579 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "1", 2024-03-08T21:15:41.579 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "32", 2024-03-08T21:15:41.579 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.579 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-03-08T21:15:41.579 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.579 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.579 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.579 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.579 INFO:teuthology.orchestra.run.smithi107.stdout: "username": { 2024-03-08T21:15:41.579 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "username", 2024-03-08T21:15:41.580 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.580 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.580 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.580 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:41.580 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.580 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.580 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.580 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "username of InfluxDB server user", 2024-03-08T21:15:41.580 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.580 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.580 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.580 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.580 INFO:teuthology.orchestra.run.smithi107.stdout: "verify_ssl": { 2024-03-08T21:15:41.580 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "verify_ssl", 2024-03-08T21:15:41.580 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.580 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.581 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.581 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "true", 2024-03-08T21:15:41.581 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.581 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.581 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.581 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-03-08T21:15:41.581 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.581 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.581 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.581 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:41.581 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:41.581 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.581 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-08T21:15:41.581 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "insights", 2024-03-08T21:15:41.581 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-03-08T21:15:41.581 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-03-08T21:15:41.582 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-03-08T21:15:41.582 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-03-08T21:15:41.582 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-03-08T21:15:41.582 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.582 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.582 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.582 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:41.582 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.582 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.582 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:41.582 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-08T21:15:41.582 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-08T21:15:41.582 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-08T21:15:41.582 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-08T21:15:41.582 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-08T21:15:41.583 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-08T21:15:41.583 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:41.583 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.583 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.583 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.583 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.583 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.583 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-03-08T21:15:41.583 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-03-08T21:15:41.583 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.583 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.583 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.583 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:41.583 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.583 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.583 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.584 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.584 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.584 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.584 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.584 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.584 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-03-08T21:15:41.584 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-03-08T21:15:41.584 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.584 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.584 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.584 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-03-08T21:15:41.584 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.584 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.584 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:41.584 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-08T21:15:41.584 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-08T21:15:41.585 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-08T21:15:41.585 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-08T21:15:41.585 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-08T21:15:41.585 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-08T21:15:41.585 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:41.585 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.585 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.585 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.585 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.585 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.585 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-03-08T21:15:41.585 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-03-08T21:15:41.585 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.585 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.585 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.585 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:41.586 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.586 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.586 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.586 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.586 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.586 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.586 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.586 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:41.586 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:41.586 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.586 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-08T21:15:41.586 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "iostat", 2024-03-08T21:15:41.586 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-03-08T21:15:41.586 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-03-08T21:15:41.586 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-03-08T21:15:41.587 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-03-08T21:15:41.587 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-03-08T21:15:41.587 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.587 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.587 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.587 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:41.587 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.587 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.587 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:41.587 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-08T21:15:41.587 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-08T21:15:41.587 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-08T21:15:41.587 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-08T21:15:41.587 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-08T21:15:41.587 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-08T21:15:41.587 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:41.588 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.588 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.588 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.588 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.588 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.588 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-03-08T21:15:41.588 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-03-08T21:15:41.588 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.588 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.588 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.588 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:41.588 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.588 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.588 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.588 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.588 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.589 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.589 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.589 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.589 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-03-08T21:15:41.589 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-03-08T21:15:41.589 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.589 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.589 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.589 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-03-08T21:15:41.589 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.589 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.589 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:41.589 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-08T21:15:41.589 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-08T21:15:41.589 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-08T21:15:41.589 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-08T21:15:41.590 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-08T21:15:41.590 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-08T21:15:41.590 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:41.590 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.590 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.590 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.590 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.590 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.590 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-03-08T21:15:41.590 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-03-08T21:15:41.590 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.590 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.590 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.590 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:41.590 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.591 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.591 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.591 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.591 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.591 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.591 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.591 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:41.591 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:41.591 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.591 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-08T21:15:41.591 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "k8sevents", 2024-03-08T21:15:41.591 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-03-08T21:15:41.591 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-03-08T21:15:41.591 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-03-08T21:15:41.591 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph_event_retention_days": { 2024-03-08T21:15:41.591 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "ceph_event_retention_days", 2024-03-08T21:15:41.591 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-08T21:15:41.592 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.592 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.592 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "7", 2024-03-08T21:15:41.592 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.592 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.592 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.592 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Days to hold ceph event information within local cache", 2024-03-08T21:15:41.592 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.592 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.592 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.592 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.592 INFO:teuthology.orchestra.run.smithi107.stdout: "config_check_secs": { 2024-03-08T21:15:41.592 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "config_check_secs", 2024-03-08T21:15:41.592 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-08T21:15:41.592 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.592 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.593 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "10", 2024-03-08T21:15:41.593 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "10", 2024-03-08T21:15:41.593 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.593 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.593 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-03-08T21:15:41.593 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.593 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.593 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.593 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.593 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-03-08T21:15:41.593 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-03-08T21:15:41.593 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.593 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.593 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.593 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:41.593 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.594 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.594 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:41.594 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-08T21:15:41.594 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-08T21:15:41.594 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-08T21:15:41.594 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-08T21:15:41.594 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-08T21:15:41.594 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-08T21:15:41.594 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:41.594 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.594 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.594 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.594 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.594 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.594 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-03-08T21:15:41.594 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-03-08T21:15:41.595 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.595 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.595 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.595 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:41.595 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.595 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.595 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.595 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.595 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.595 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.595 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.595 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.595 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-03-08T21:15:41.595 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-03-08T21:15:41.596 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.596 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.596 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.596 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-03-08T21:15:41.596 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.596 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.596 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:41.596 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-08T21:15:41.596 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-08T21:15:41.596 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-08T21:15:41.596 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-08T21:15:41.596 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-08T21:15:41.596 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-08T21:15:41.596 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:41.596 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.596 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.596 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.597 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.597 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.597 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-03-08T21:15:41.597 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-03-08T21:15:41.597 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.597 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.597 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.597 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:41.597 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.597 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.597 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.597 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.597 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.597 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.597 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.598 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:41.598 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:41.598 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.598 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-08T21:15:41.598 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "localpool", 2024-03-08T21:15:41.598 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-03-08T21:15:41.598 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-03-08T21:15:41.598 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-03-08T21:15:41.598 INFO:teuthology.orchestra.run.smithi107.stdout: "failure_domain": { 2024-03-08T21:15:41.598 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "failure_domain", 2024-03-08T21:15:41.598 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.598 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.598 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.599 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "host", 2024-03-08T21:15:41.599 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.599 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.599 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.599 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "failure domain for any created local pool", 2024-03-08T21:15:41.599 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-03-08T21:15:41.599 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.599 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.599 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.599 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-03-08T21:15:41.599 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-03-08T21:15:41.599 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.599 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.599 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.599 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:41.599 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.600 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.600 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:41.600 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-08T21:15:41.600 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-08T21:15:41.600 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-08T21:15:41.600 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-08T21:15:41.600 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-08T21:15:41.600 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-08T21:15:41.600 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:41.600 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.600 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.600 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.600 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.600 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.600 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-03-08T21:15:41.601 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-03-08T21:15:41.601 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.601 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.601 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.601 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:41.601 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.601 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.601 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.601 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.601 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.601 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.601 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.601 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.601 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-03-08T21:15:41.601 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-03-08T21:15:41.601 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.602 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.602 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.602 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-03-08T21:15:41.602 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.602 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.602 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:41.602 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-08T21:15:41.602 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-08T21:15:41.602 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-08T21:15:41.602 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-08T21:15:41.602 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-08T21:15:41.602 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-08T21:15:41.602 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:41.602 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.602 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.602 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.603 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.603 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.603 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-03-08T21:15:41.603 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-03-08T21:15:41.603 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.603 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.603 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.603 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:41.603 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.603 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.603 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.603 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.603 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.603 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.603 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.604 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.604 INFO:teuthology.orchestra.run.smithi107.stdout: "min_size": { 2024-03-08T21:15:41.604 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "min_size", 2024-03-08T21:15:41.604 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-08T21:15:41.604 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.604 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.604 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:41.604 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.604 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.604 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.604 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "default min_size for any created local pool", 2024-03-08T21:15:41.604 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-03-08T21:15:41.604 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.604 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.604 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.605 INFO:teuthology.orchestra.run.smithi107.stdout: "num_rep": { 2024-03-08T21:15:41.605 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "num_rep", 2024-03-08T21:15:41.605 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-08T21:15:41.605 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.605 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.605 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "3", 2024-03-08T21:15:41.605 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.605 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.605 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.605 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "default replica count for any created local pool", 2024-03-08T21:15:41.605 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.605 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.605 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.605 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.605 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_num": { 2024-03-08T21:15:41.606 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "pg_num", 2024-03-08T21:15:41.606 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-08T21:15:41.606 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.606 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.606 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "128", 2024-03-08T21:15:41.606 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.606 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.606 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.606 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "default pg_num for any created local pool", 2024-03-08T21:15:41.606 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.606 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.606 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.606 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.606 INFO:teuthology.orchestra.run.smithi107.stdout: "prefix": { 2024-03-08T21:15:41.606 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "prefix", 2024-03-08T21:15:41.607 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.607 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.607 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.607 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:41.607 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.607 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.607 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.607 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "name prefix for any created local pool", 2024-03-08T21:15:41.607 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.607 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.607 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.607 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.607 INFO:teuthology.orchestra.run.smithi107.stdout: "subtree": { 2024-03-08T21:15:41.607 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "subtree", 2024-03-08T21:15:41.607 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.608 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.608 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.608 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "rack", 2024-03-08T21:15:41.608 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.608 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.608 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.608 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "CRUSH level for which to create a local pool", 2024-03-08T21:15:41.608 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-03-08T21:15:41.608 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.608 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.608 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:41.608 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:41.608 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.608 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-08T21:15:41.609 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "mds_autoscaler", 2024-03-08T21:15:41.609 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-03-08T21:15:41.609 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-03-08T21:15:41.609 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-03-08T21:15:41.609 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-03-08T21:15:41.609 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-03-08T21:15:41.609 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.609 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.609 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.609 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:41.609 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.609 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.609 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:41.609 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-08T21:15:41.609 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-08T21:15:41.610 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-08T21:15:41.610 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-08T21:15:41.610 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-08T21:15:41.610 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-08T21:15:41.610 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:41.610 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.610 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.610 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.610 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.610 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.610 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-03-08T21:15:41.610 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-03-08T21:15:41.610 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.610 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.610 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.611 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:41.611 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.611 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.611 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.611 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.611 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.611 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.611 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.611 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.611 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-03-08T21:15:41.611 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-03-08T21:15:41.611 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.611 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.611 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.611 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-03-08T21:15:41.612 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.612 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.612 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:41.612 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-08T21:15:41.612 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-08T21:15:41.612 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-08T21:15:41.612 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-08T21:15:41.612 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-08T21:15:41.612 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-08T21:15:41.612 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:41.612 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.612 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.612 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.612 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.612 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.612 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-03-08T21:15:41.613 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-03-08T21:15:41.613 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.613 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.613 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.613 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:41.613 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.613 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.613 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.613 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.613 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.613 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.613 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.613 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:41.613 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:41.613 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.614 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-08T21:15:41.614 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "mirroring", 2024-03-08T21:15:41.614 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-03-08T21:15:41.614 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-03-08T21:15:41.614 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-03-08T21:15:41.614 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-03-08T21:15:41.614 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-03-08T21:15:41.614 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.614 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.614 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.614 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:41.614 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.614 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.614 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:41.615 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-08T21:15:41.615 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-08T21:15:41.615 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-08T21:15:41.615 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-08T21:15:41.615 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-08T21:15:41.615 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-08T21:15:41.615 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:41.615 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.615 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.615 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.615 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.615 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.615 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-03-08T21:15:41.615 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-03-08T21:15:41.615 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.615 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.616 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.616 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:41.616 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.616 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.616 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.616 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.616 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.616 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.616 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.616 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.616 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-03-08T21:15:41.616 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-03-08T21:15:41.616 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.616 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.616 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.617 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-03-08T21:15:41.617 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.617 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.617 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:41.617 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-08T21:15:41.617 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-08T21:15:41.617 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-08T21:15:41.617 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-08T21:15:41.617 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-08T21:15:41.617 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-08T21:15:41.617 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:41.617 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.617 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.617 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.617 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.617 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.618 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-03-08T21:15:41.618 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-03-08T21:15:41.618 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.618 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.618 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.618 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:41.618 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.618 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.618 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.618 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.618 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.618 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.618 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.618 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:41.619 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:41.619 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.619 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-08T21:15:41.619 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "nfs", 2024-03-08T21:15:41.619 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-03-08T21:15:41.619 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-03-08T21:15:41.619 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-03-08T21:15:41.619 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-03-08T21:15:41.619 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-03-08T21:15:41.619 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.619 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.619 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.619 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:41.619 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.619 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.619 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:41.620 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-08T21:15:41.620 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-08T21:15:41.620 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-08T21:15:41.620 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-08T21:15:41.620 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-08T21:15:41.620 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-08T21:15:41.620 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:41.620 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.620 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.620 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.620 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.620 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.620 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-03-08T21:15:41.620 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-03-08T21:15:41.620 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.620 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.621 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.621 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:41.621 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.621 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.621 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.621 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.621 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.621 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.621 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.621 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.621 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-03-08T21:15:41.621 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-03-08T21:15:41.621 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.621 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.621 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.621 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-03-08T21:15:41.622 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.622 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.622 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:41.622 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-08T21:15:41.622 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-08T21:15:41.622 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-08T21:15:41.622 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-08T21:15:41.622 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-08T21:15:41.622 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-08T21:15:41.622 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:41.622 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.622 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.622 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.622 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.622 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.622 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-03-08T21:15:41.623 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-03-08T21:15:41.623 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.623 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.623 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.623 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:41.623 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.623 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.623 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.623 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.623 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.623 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.623 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.623 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:41.623 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:41.623 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.623 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-08T21:15:41.624 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "orchestrator", 2024-03-08T21:15:41.624 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-03-08T21:15:41.624 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-03-08T21:15:41.624 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-03-08T21:15:41.624 INFO:teuthology.orchestra.run.smithi107.stdout: "fail_fs": { 2024-03-08T21:15:41.624 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "fail_fs", 2024-03-08T21:15:41.624 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.624 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.624 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.624 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:41.624 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.624 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.624 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.624 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-03-08T21:15:41.624 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.624 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.625 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.625 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.625 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-03-08T21:15:41.625 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-03-08T21:15:41.625 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.625 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.625 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.625 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:41.625 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.625 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.625 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:41.625 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-08T21:15:41.625 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-08T21:15:41.625 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-08T21:15:41.625 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-08T21:15:41.625 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-08T21:15:41.625 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-08T21:15:41.626 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:41.626 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.626 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.626 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.626 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.626 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.626 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-03-08T21:15:41.626 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-03-08T21:15:41.626 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.626 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.626 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.626 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:41.626 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.626 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.626 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.627 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.627 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.627 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.627 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.627 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.627 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-03-08T21:15:41.627 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-03-08T21:15:41.627 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.627 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.627 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.627 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-03-08T21:15:41.627 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.627 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.627 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:41.627 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-08T21:15:41.627 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-08T21:15:41.628 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-08T21:15:41.628 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-08T21:15:41.628 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-08T21:15:41.628 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-08T21:15:41.628 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:41.628 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.628 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.628 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.628 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.628 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.628 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-03-08T21:15:41.628 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-03-08T21:15:41.628 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.628 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.628 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.628 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:41.629 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.629 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.629 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.629 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.629 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.629 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.629 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.629 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.629 INFO:teuthology.orchestra.run.smithi107.stdout: "orchestrator": { 2024-03-08T21:15:41.629 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "orchestrator", 2024-03-08T21:15:41.629 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.629 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.629 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.629 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:41.629 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.629 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.630 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:41.630 INFO:teuthology.orchestra.run.smithi107.stdout: "cephadm", 2024-03-08T21:15:41.630 INFO:teuthology.orchestra.run.smithi107.stdout: "rook", 2024-03-08T21:15:41.630 INFO:teuthology.orchestra.run.smithi107.stdout: "test_orchestrator" 2024-03-08T21:15:41.630 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:41.630 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Orchestrator backend", 2024-03-08T21:15:41.630 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.630 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.630 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.630 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:41.630 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:41.630 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.630 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-08T21:15:41.630 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "osd_perf_query", 2024-03-08T21:15:41.630 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-03-08T21:15:41.631 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-03-08T21:15:41.631 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-03-08T21:15:41.631 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-03-08T21:15:41.631 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-03-08T21:15:41.631 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.631 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.631 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.631 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:41.631 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.631 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.631 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:41.631 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-08T21:15:41.631 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-08T21:15:41.631 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-08T21:15:41.631 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-08T21:15:41.631 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-08T21:15:41.632 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-08T21:15:41.632 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:41.632 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.632 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.632 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.632 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.632 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.632 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-03-08T21:15:41.632 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-03-08T21:15:41.632 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.632 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.632 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.632 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:41.632 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.632 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.632 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.633 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.633 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.633 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.633 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.633 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.633 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-03-08T21:15:41.633 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-03-08T21:15:41.633 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.633 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.633 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.633 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-03-08T21:15:41.633 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.633 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.633 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:41.633 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-08T21:15:41.633 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-08T21:15:41.634 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-08T21:15:41.634 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-08T21:15:41.634 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-08T21:15:41.634 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-08T21:15:41.634 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:41.634 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.634 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.634 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.634 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.634 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.634 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-03-08T21:15:41.634 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-03-08T21:15:41.634 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.634 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.634 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.635 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:41.635 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.635 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.635 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.635 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.635 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.635 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.635 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.635 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:41.635 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:41.635 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.635 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-08T21:15:41.635 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "osd_support", 2024-03-08T21:15:41.635 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-03-08T21:15:41.635 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-03-08T21:15:41.635 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-03-08T21:15:41.636 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-03-08T21:15:41.636 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-03-08T21:15:41.636 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.636 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.636 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.636 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:41.636 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.636 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.636 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:41.636 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-08T21:15:41.636 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-08T21:15:41.636 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-08T21:15:41.636 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-08T21:15:41.636 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-08T21:15:41.636 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-08T21:15:41.636 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:41.637 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.637 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.637 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.637 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.637 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.637 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-03-08T21:15:41.637 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-03-08T21:15:41.637 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.637 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.637 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.637 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:41.637 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.637 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.637 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.637 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.637 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.637 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.638 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.638 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.638 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-03-08T21:15:41.638 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-03-08T21:15:41.638 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.638 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.638 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.638 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-03-08T21:15:41.638 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.638 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.638 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:41.638 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-08T21:15:41.638 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-08T21:15:41.638 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-08T21:15:41.638 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-08T21:15:41.639 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-08T21:15:41.639 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-08T21:15:41.639 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:41.639 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.639 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.639 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.639 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.639 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.639 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-03-08T21:15:41.639 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-03-08T21:15:41.639 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.639 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.639 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.639 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:41.639 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.640 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.640 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.640 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.640 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.640 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.640 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.640 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:41.640 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:41.640 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.640 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-08T21:15:41.640 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "pg_autoscaler", 2024-03-08T21:15:41.640 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-03-08T21:15:41.640 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-03-08T21:15:41.640 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-03-08T21:15:41.640 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-03-08T21:15:41.640 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-03-08T21:15:41.640 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.641 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.641 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.641 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:41.641 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.641 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.641 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:41.641 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-08T21:15:41.641 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-08T21:15:41.641 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-08T21:15:41.641 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-08T21:15:41.641 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-08T21:15:41.641 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-08T21:15:41.641 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:41.641 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.641 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.641 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.641 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.642 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.642 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-03-08T21:15:41.642 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-03-08T21:15:41.642 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.642 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.642 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.642 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:41.642 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.642 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.642 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.642 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.642 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.642 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.642 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.643 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.643 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-03-08T21:15:41.643 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-03-08T21:15:41.643 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.643 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.643 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.643 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-03-08T21:15:41.643 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.643 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.643 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:41.643 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-08T21:15:41.643 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-08T21:15:41.643 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-08T21:15:41.643 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-08T21:15:41.644 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-08T21:15:41.644 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-08T21:15:41.644 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:41.644 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.644 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.644 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.644 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.644 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.644 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-03-08T21:15:41.644 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-03-08T21:15:41.644 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.644 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.644 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.644 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:41.644 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.645 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.645 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.645 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.645 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.645 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.645 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.645 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.645 INFO:teuthology.orchestra.run.smithi107.stdout: "sleep_interval": { 2024-03-08T21:15:41.645 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "sleep_interval", 2024-03-08T21:15:41.645 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-03-08T21:15:41.645 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.645 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.645 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "60", 2024-03-08T21:15:41.645 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.645 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.646 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.646 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.646 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.646 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.646 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.646 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.646 INFO:teuthology.orchestra.run.smithi107.stdout: "threshold": { 2024-03-08T21:15:41.646 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "threshold", 2024-03-08T21:15:41.646 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "float", 2024-03-08T21:15:41.646 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.646 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.646 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "3.0", 2024-03-08T21:15:41.646 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "1.0", 2024-03-08T21:15:41.646 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.646 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.647 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "scaling threshold", 2024-03-08T21:15:41.647 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-03-08T21:15:41.647 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.647 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.647 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:41.647 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:41.647 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.647 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-08T21:15:41.647 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "progress", 2024-03-08T21:15:41.647 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-03-08T21:15:41.647 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-03-08T21:15:41.647 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-03-08T21:15:41.647 INFO:teuthology.orchestra.run.smithi107.stdout: "allow_pg_recovery_event": { 2024-03-08T21:15:41.647 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "allow_pg_recovery_event", 2024-03-08T21:15:41.647 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.648 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.648 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.648 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:41.648 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.648 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.648 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.648 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "allow the module to show pg recovery progress", 2024-03-08T21:15:41.648 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.648 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.648 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.648 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.648 INFO:teuthology.orchestra.run.smithi107.stdout: "enabled": { 2024-03-08T21:15:41.648 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "enabled", 2024-03-08T21:15:41.648 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.648 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.648 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.649 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-08T21:15:41.649 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.649 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.649 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.649 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.649 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.649 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.649 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.649 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.649 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-03-08T21:15:41.649 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-03-08T21:15:41.649 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.649 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.649 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.649 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:41.649 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.649 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.650 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:41.650 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-08T21:15:41.650 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-08T21:15:41.650 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-08T21:15:41.650 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-08T21:15:41.650 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-08T21:15:41.650 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-08T21:15:41.650 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:41.650 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.650 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.650 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.650 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.650 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.650 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-03-08T21:15:41.650 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-03-08T21:15:41.651 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.651 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.651 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.651 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:41.651 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.651 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.651 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.651 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.651 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.651 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.651 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.651 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.651 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-03-08T21:15:41.651 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-03-08T21:15:41.651 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.651 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.652 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.652 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-03-08T21:15:41.652 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.652 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.652 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:41.652 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-08T21:15:41.652 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-08T21:15:41.652 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-08T21:15:41.652 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-08T21:15:41.652 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-08T21:15:41.652 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-08T21:15:41.652 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:41.652 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.652 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.652 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.653 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.653 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.653 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-03-08T21:15:41.653 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-03-08T21:15:41.653 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.653 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.653 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.653 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:41.653 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.653 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.653 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.653 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.653 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.653 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.653 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.653 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.654 INFO:teuthology.orchestra.run.smithi107.stdout: "max_completed_events": { 2024-03-08T21:15:41.654 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "max_completed_events", 2024-03-08T21:15:41.654 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-08T21:15:41.654 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.654 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.654 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "50", 2024-03-08T21:15:41.654 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.654 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.654 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.654 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "number of past completed events to remember", 2024-03-08T21:15:41.654 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.654 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.654 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.654 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.655 INFO:teuthology.orchestra.run.smithi107.stdout: "sleep_interval": { 2024-03-08T21:15:41.655 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "sleep_interval", 2024-03-08T21:15:41.655 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-03-08T21:15:41.655 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.655 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.655 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "5", 2024-03-08T21:15:41.655 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.655 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.655 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.655 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "how long the module is going to sleep", 2024-03-08T21:15:41.655 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.655 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.655 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.655 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:41.655 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:41.656 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.656 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-08T21:15:41.656 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "prometheus", 2024-03-08T21:15:41.656 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-03-08T21:15:41.656 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-03-08T21:15:41.656 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-03-08T21:15:41.656 INFO:teuthology.orchestra.run.smithi107.stdout: "cache": { 2024-03-08T21:15:41.656 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "cache", 2024-03-08T21:15:41.656 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.656 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.656 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.656 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-08T21:15:41.656 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.656 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.656 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.656 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.657 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.657 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.657 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.657 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.657 INFO:teuthology.orchestra.run.smithi107.stdout: "exclude_perf_counters": { 2024-03-08T21:15:41.657 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "exclude_perf_counters", 2024-03-08T21:15:41.657 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.657 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.657 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.657 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-08T21:15:41.657 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.657 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.657 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.657 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-03-08T21:15:41.657 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-03-08T21:15:41.658 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.658 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.658 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.658 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-03-08T21:15:41.658 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-03-08T21:15:41.658 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.658 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.658 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.658 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:41.658 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.658 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.658 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:41.658 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-08T21:15:41.659 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-08T21:15:41.659 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-08T21:15:41.659 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-08T21:15:41.659 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-08T21:15:41.659 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-08T21:15:41.659 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:41.659 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.659 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.659 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.659 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.659 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.659 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-03-08T21:15:41.659 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-03-08T21:15:41.659 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.660 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.660 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.660 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:41.660 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.660 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.660 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.660 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.660 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.660 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.660 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.660 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.660 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-03-08T21:15:41.660 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-03-08T21:15:41.660 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.660 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.661 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.661 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-03-08T21:15:41.661 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.661 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.661 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:41.661 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-08T21:15:41.661 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-08T21:15:41.661 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-08T21:15:41.661 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-08T21:15:41.661 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-08T21:15:41.661 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-08T21:15:41.661 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:41.661 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.661 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.661 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.662 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.662 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.662 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-03-08T21:15:41.662 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-03-08T21:15:41.662 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.662 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.662 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.662 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:41.662 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.662 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.662 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.662 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.663 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.663 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.663 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.663 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.663 INFO:teuthology.orchestra.run.smithi107.stdout: "rbd_stats_pools": { 2024-03-08T21:15:41.663 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rbd_stats_pools", 2024-03-08T21:15:41.663 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.663 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.663 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.663 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:41.663 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.663 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.663 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.663 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.663 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.664 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.664 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.664 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.664 INFO:teuthology.orchestra.run.smithi107.stdout: "rbd_stats_pools_refresh_interval": { 2024-03-08T21:15:41.664 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-03-08T21:15:41.664 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-08T21:15:41.664 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.664 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.664 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "300", 2024-03-08T21:15:41.664 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.664 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.664 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.664 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.664 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.664 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.665 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.665 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.665 INFO:teuthology.orchestra.run.smithi107.stdout: "scrape_interval": { 2024-03-08T21:15:41.665 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "scrape_interval", 2024-03-08T21:15:41.665 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "float", 2024-03-08T21:15:41.665 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.665 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.665 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "15.0", 2024-03-08T21:15:41.665 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.665 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.665 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.665 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.665 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.665 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.666 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.666 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.666 INFO:teuthology.orchestra.run.smithi107.stdout: "server_addr": { 2024-03-08T21:15:41.666 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "server_addr", 2024-03-08T21:15:41.666 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.666 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.666 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.666 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "::", 2024-03-08T21:15:41.666 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.666 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.666 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.666 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-03-08T21:15:41.666 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.667 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.667 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.667 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.667 INFO:teuthology.orchestra.run.smithi107.stdout: "server_port": { 2024-03-08T21:15:41.667 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "server_port", 2024-03-08T21:15:41.667 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-08T21:15:41.667 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.667 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.667 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "9283", 2024-03-08T21:15:41.667 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.667 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.667 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.667 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-03-08T21:15:41.668 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.668 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.668 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.668 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.668 INFO:teuthology.orchestra.run.smithi107.stdout: "stale_cache_strategy": { 2024-03-08T21:15:41.668 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "stale_cache_strategy", 2024-03-08T21:15:41.668 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.668 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.668 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.668 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "log", 2024-03-08T21:15:41.668 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.668 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.668 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.668 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.668 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.669 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.669 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.669 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.669 INFO:teuthology.orchestra.run.smithi107.stdout: "standby_behaviour": { 2024-03-08T21:15:41.669 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "standby_behaviour", 2024-03-08T21:15:41.669 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.669 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.669 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.669 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "default", 2024-03-08T21:15:41.669 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.669 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.669 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:41.669 INFO:teuthology.orchestra.run.smithi107.stdout: "default", 2024-03-08T21:15:41.669 INFO:teuthology.orchestra.run.smithi107.stdout: "error" 2024-03-08T21:15:41.670 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:41.670 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.670 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.670 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.670 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.670 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.670 INFO:teuthology.orchestra.run.smithi107.stdout: "standby_error_status_code": { 2024-03-08T21:15:41.670 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "standby_error_status_code", 2024-03-08T21:15:41.670 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-08T21:15:41.670 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.670 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.670 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "500", 2024-03-08T21:15:41.670 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "400", 2024-03-08T21:15:41.670 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "599", 2024-03-08T21:15:41.670 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.671 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.671 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.671 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.671 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.671 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:41.671 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:41.671 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.671 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-08T21:15:41.671 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rbd_support", 2024-03-08T21:15:41.671 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-03-08T21:15:41.671 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-03-08T21:15:41.671 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-03-08T21:15:41.671 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-03-08T21:15:41.671 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-03-08T21:15:41.671 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.671 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.672 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.672 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:41.672 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.672 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.672 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:41.672 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-08T21:15:41.672 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-08T21:15:41.672 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-08T21:15:41.672 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-08T21:15:41.672 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-08T21:15:41.672 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-08T21:15:41.672 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:41.672 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.672 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.672 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.672 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.673 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.673 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-03-08T21:15:41.673 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-03-08T21:15:41.673 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.673 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.673 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.673 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:41.673 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.673 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.673 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.673 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.673 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.673 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.673 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.673 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.674 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-03-08T21:15:41.674 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-03-08T21:15:41.674 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.674 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.674 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.674 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-03-08T21:15:41.674 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.674 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.674 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:41.674 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-08T21:15:41.674 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-08T21:15:41.674 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-08T21:15:41.674 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-08T21:15:41.674 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-08T21:15:41.675 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-08T21:15:41.675 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:41.675 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.675 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.675 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.675 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.675 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.675 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-03-08T21:15:41.675 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-03-08T21:15:41.675 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.675 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.675 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.675 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:41.675 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.675 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.676 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.676 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.676 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.676 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.676 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.676 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.676 INFO:teuthology.orchestra.run.smithi107.stdout: "max_concurrent_snap_create": { 2024-03-08T21:15:41.676 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "max_concurrent_snap_create", 2024-03-08T21:15:41.676 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-08T21:15:41.676 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.676 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.676 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "10", 2024-03-08T21:15:41.676 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.676 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.676 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.676 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.677 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.677 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.677 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.677 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.677 INFO:teuthology.orchestra.run.smithi107.stdout: "mirror_snapshot_schedule": { 2024-03-08T21:15:41.677 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "mirror_snapshot_schedule", 2024-03-08T21:15:41.677 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.677 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.677 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.677 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:41.677 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.677 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.677 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.677 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.677 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.677 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.678 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.678 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.678 INFO:teuthology.orchestra.run.smithi107.stdout: "trash_purge_schedule": { 2024-03-08T21:15:41.678 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "trash_purge_schedule", 2024-03-08T21:15:41.678 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.678 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.678 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.678 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:41.678 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.678 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.678 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.678 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.678 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.678 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.679 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.679 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:41.679 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:41.679 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.679 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-08T21:15:41.679 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "restful", 2024-03-08T21:15:41.679 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-03-08T21:15:41.679 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-03-08T21:15:41.679 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-03-08T21:15:41.679 INFO:teuthology.orchestra.run.smithi107.stdout: "enable_auth": { 2024-03-08T21:15:41.679 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "enable_auth", 2024-03-08T21:15:41.679 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.679 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.679 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.679 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-08T21:15:41.680 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.680 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.680 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.680 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.680 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.680 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.680 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.680 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.680 INFO:teuthology.orchestra.run.smithi107.stdout: "key_file": { 2024-03-08T21:15:41.680 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "key_file", 2024-03-08T21:15:41.680 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.680 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.680 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.680 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:41.680 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.680 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.680 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.681 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.681 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.681 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.681 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.681 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.681 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-03-08T21:15:41.681 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-03-08T21:15:41.681 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.681 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.681 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.681 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:41.681 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.681 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.681 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:41.681 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-08T21:15:41.681 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-08T21:15:41.682 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-08T21:15:41.682 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-08T21:15:41.682 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-08T21:15:41.682 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-08T21:15:41.682 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:41.682 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.682 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.682 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.682 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.682 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.682 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-03-08T21:15:41.682 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-03-08T21:15:41.682 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.682 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.683 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.683 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:41.683 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.683 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.683 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.683 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.683 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.683 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.683 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.683 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.683 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-03-08T21:15:41.683 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-03-08T21:15:41.683 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.683 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.683 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.684 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-03-08T21:15:41.684 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.684 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.684 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:41.684 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-08T21:15:41.684 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-08T21:15:41.684 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-08T21:15:41.684 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-08T21:15:41.684 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-08T21:15:41.684 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-08T21:15:41.684 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:41.684 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.684 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.684 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.684 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.684 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.685 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-03-08T21:15:41.685 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-03-08T21:15:41.685 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.685 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.685 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.685 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:41.685 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.685 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.685 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.685 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.685 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.685 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.685 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.685 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.685 INFO:teuthology.orchestra.run.smithi107.stdout: "server_addr": { 2024-03-08T21:15:41.685 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "server_addr", 2024-03-08T21:15:41.686 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.686 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.686 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.686 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:41.686 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.686 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.686 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.686 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.686 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.686 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.686 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.686 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.686 INFO:teuthology.orchestra.run.smithi107.stdout: "server_port": { 2024-03-08T21:15:41.686 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "server_port", 2024-03-08T21:15:41.686 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.687 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.687 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.687 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:41.687 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.687 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.687 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.687 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.687 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.687 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.687 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.687 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:41.687 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:41.687 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.687 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-08T21:15:41.687 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rgw", 2024-03-08T21:15:41.687 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-03-08T21:15:41.688 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-03-08T21:15:41.688 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-03-08T21:15:41.688 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-03-08T21:15:41.688 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-03-08T21:15:41.688 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.688 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.688 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.688 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:41.688 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.688 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.688 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:41.688 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-08T21:15:41.688 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-08T21:15:41.688 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-08T21:15:41.688 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-08T21:15:41.688 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-08T21:15:41.689 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-08T21:15:41.689 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:41.689 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.689 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.689 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.689 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.689 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.689 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-03-08T21:15:41.689 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-03-08T21:15:41.689 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.689 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.689 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.689 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:41.689 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.689 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.690 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.690 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.690 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.690 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.690 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.690 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.690 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-03-08T21:15:41.690 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-03-08T21:15:41.690 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.690 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.690 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.690 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-03-08T21:15:41.690 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.690 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.691 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:41.691 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-08T21:15:41.691 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-08T21:15:41.691 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-08T21:15:41.691 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-08T21:15:41.691 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-08T21:15:41.691 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-08T21:15:41.691 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:41.691 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.691 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.691 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.691 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.691 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.691 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-03-08T21:15:41.691 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-03-08T21:15:41.692 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.692 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.692 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.692 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:41.692 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.692 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.692 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.692 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.692 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.692 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.692 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.692 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:41.692 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:41.692 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.692 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-08T21:15:41.692 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rook", 2024-03-08T21:15:41.693 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-03-08T21:15:41.693 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-03-08T21:15:41.693 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-03-08T21:15:41.693 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-03-08T21:15:41.693 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-03-08T21:15:41.693 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.693 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.693 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.693 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:41.693 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.693 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.693 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:41.693 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-08T21:15:41.693 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-08T21:15:41.693 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-08T21:15:41.694 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-08T21:15:41.694 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-08T21:15:41.694 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-08T21:15:41.694 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:41.694 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.694 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.694 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.694 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.694 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.694 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-03-08T21:15:41.694 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-03-08T21:15:41.694 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.694 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.694 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.695 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:41.695 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.695 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.695 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.695 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.695 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.695 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.695 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.695 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.695 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-03-08T21:15:41.695 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-03-08T21:15:41.695 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.695 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.695 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.695 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-03-08T21:15:41.695 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.696 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.696 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:41.696 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-08T21:15:41.696 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-08T21:15:41.696 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-08T21:15:41.696 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-08T21:15:41.696 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-08T21:15:41.696 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-08T21:15:41.696 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:41.696 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.696 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.696 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.696 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.696 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.696 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-03-08T21:15:41.696 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-03-08T21:15:41.697 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.697 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.697 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.697 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:41.697 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.697 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.697 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.697 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.697 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.697 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.697 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.697 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.697 INFO:teuthology.orchestra.run.smithi107.stdout: "storage_class": { 2024-03-08T21:15:41.697 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "storage_class", 2024-03-08T21:15:41.698 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.698 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.698 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.698 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "local", 2024-03-08T21:15:41.698 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.698 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.698 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.698 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-03-08T21:15:41.698 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.698 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.698 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.698 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:41.698 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:41.698 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.699 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-08T21:15:41.699 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "selftest", 2024-03-08T21:15:41.699 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-03-08T21:15:41.699 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-03-08T21:15:41.699 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-03-08T21:15:41.699 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-03-08T21:15:41.699 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-03-08T21:15:41.699 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.699 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.699 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.699 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:41.699 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.699 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.699 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:41.699 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-08T21:15:41.699 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-08T21:15:41.700 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-08T21:15:41.700 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-08T21:15:41.700 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-08T21:15:41.700 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-08T21:15:41.700 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:41.700 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.700 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.700 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.700 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.700 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.700 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-03-08T21:15:41.700 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-03-08T21:15:41.700 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.700 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.700 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.700 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:41.701 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.701 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.701 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.701 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.701 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.701 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.701 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.701 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.701 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-03-08T21:15:41.701 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-03-08T21:15:41.701 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.701 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.701 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.701 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-03-08T21:15:41.701 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.701 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.702 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:41.702 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-08T21:15:41.702 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-08T21:15:41.702 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-08T21:15:41.702 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-08T21:15:41.702 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-08T21:15:41.702 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-08T21:15:41.702 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:41.702 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.702 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.702 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.702 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.702 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.703 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-03-08T21:15:41.703 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-03-08T21:15:41.703 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.703 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.703 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.703 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:41.703 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.703 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.703 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.703 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.703 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.703 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.703 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.703 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.703 INFO:teuthology.orchestra.run.smithi107.stdout: "roption1": { 2024-03-08T21:15:41.703 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "roption1", 2024-03-08T21:15:41.703 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.704 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.704 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.704 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:41.704 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.704 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.704 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.704 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.704 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.704 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.704 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.704 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.704 INFO:teuthology.orchestra.run.smithi107.stdout: "roption2": { 2024-03-08T21:15:41.704 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "roption2", 2024-03-08T21:15:41.704 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.704 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.705 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.705 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "xyz", 2024-03-08T21:15:41.705 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.705 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.705 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.705 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.705 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.705 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.705 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.705 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.705 INFO:teuthology.orchestra.run.smithi107.stdout: "rwoption1": { 2024-03-08T21:15:41.705 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rwoption1", 2024-03-08T21:15:41.705 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.705 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.705 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.705 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:41.706 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.706 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.706 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.706 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.706 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.706 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.706 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.706 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.706 INFO:teuthology.orchestra.run.smithi107.stdout: "rwoption2": { 2024-03-08T21:15:41.706 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rwoption2", 2024-03-08T21:15:41.706 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-08T21:15:41.706 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.706 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.706 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:41.706 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.706 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.707 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.707 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.707 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.707 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.707 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.707 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.707 INFO:teuthology.orchestra.run.smithi107.stdout: "rwoption3": { 2024-03-08T21:15:41.707 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rwoption3", 2024-03-08T21:15:41.707 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "float", 2024-03-08T21:15:41.707 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.707 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.707 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:41.707 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.707 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.707 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.708 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.708 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.708 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.708 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.708 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.708 INFO:teuthology.orchestra.run.smithi107.stdout: "rwoption4": { 2024-03-08T21:15:41.708 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rwoption4", 2024-03-08T21:15:41.708 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.708 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.708 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.708 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:41.708 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.708 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.708 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.708 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.708 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.709 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.709 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.709 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.709 INFO:teuthology.orchestra.run.smithi107.stdout: "rwoption5": { 2024-03-08T21:15:41.709 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rwoption5", 2024-03-08T21:15:41.709 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.709 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.709 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.709 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:41.709 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.709 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.709 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.709 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.709 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.709 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.709 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.710 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.710 INFO:teuthology.orchestra.run.smithi107.stdout: "rwoption6": { 2024-03-08T21:15:41.710 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rwoption6", 2024-03-08T21:15:41.710 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.710 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.710 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.710 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-08T21:15:41.710 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.710 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.710 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.710 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.710 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.710 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.710 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.710 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.711 INFO:teuthology.orchestra.run.smithi107.stdout: "rwoption7": { 2024-03-08T21:15:41.711 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rwoption7", 2024-03-08T21:15:41.711 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-08T21:15:41.711 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.711 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.711 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:41.711 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "1", 2024-03-08T21:15:41.711 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "42", 2024-03-08T21:15:41.711 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.711 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.711 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.711 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.711 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.711 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.711 INFO:teuthology.orchestra.run.smithi107.stdout: "testkey": { 2024-03-08T21:15:41.711 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "testkey", 2024-03-08T21:15:41.712 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.712 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.712 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.712 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:41.712 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.712 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.712 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.712 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.712 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.712 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.712 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.712 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.712 INFO:teuthology.orchestra.run.smithi107.stdout: "testlkey": { 2024-03-08T21:15:41.712 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "testlkey", 2024-03-08T21:15:41.712 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.712 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.713 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.713 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:41.713 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.713 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.713 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.713 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.713 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.713 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.713 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.713 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.713 INFO:teuthology.orchestra.run.smithi107.stdout: "testnewline": { 2024-03-08T21:15:41.713 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "testnewline", 2024-03-08T21:15:41.713 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.713 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.713 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.713 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:41.713 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.714 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.714 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.714 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.714 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.714 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.714 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.714 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:41.714 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:41.714 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.714 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-08T21:15:41.714 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "snap_schedule", 2024-03-08T21:15:41.714 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-03-08T21:15:41.714 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-03-08T21:15:41.714 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-03-08T21:15:41.714 INFO:teuthology.orchestra.run.smithi107.stdout: "allow_m_granularity": { 2024-03-08T21:15:41.715 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "allow_m_granularity", 2024-03-08T21:15:41.715 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.715 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.715 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.715 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:41.715 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.715 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.715 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.715 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "allow minute scheduled snapshots", 2024-03-08T21:15:41.715 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.715 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.715 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.715 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.715 INFO:teuthology.orchestra.run.smithi107.stdout: "dump_on_update": { 2024-03-08T21:15:41.715 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "dump_on_update", 2024-03-08T21:15:41.715 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.715 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.716 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.716 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:41.716 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.716 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.716 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.716 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "dump database to debug log on update", 2024-03-08T21:15:41.716 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.716 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.716 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.716 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.716 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-03-08T21:15:41.716 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-03-08T21:15:41.716 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.716 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.716 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.716 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:41.717 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.717 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.717 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:41.717 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-08T21:15:41.717 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-08T21:15:41.717 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-08T21:15:41.717 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-08T21:15:41.717 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-08T21:15:41.717 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-08T21:15:41.717 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:41.717 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.717 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.717 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.717 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.717 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.718 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-03-08T21:15:41.718 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-03-08T21:15:41.718 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.718 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.718 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.718 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:41.718 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.718 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.718 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.718 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.718 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.718 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.718 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.718 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.718 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-03-08T21:15:41.718 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-03-08T21:15:41.719 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.719 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.719 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.719 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-03-08T21:15:41.719 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.719 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.719 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:41.719 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-08T21:15:41.719 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-08T21:15:41.719 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-08T21:15:41.719 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-08T21:15:41.719 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-08T21:15:41.719 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-08T21:15:41.719 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:41.719 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.719 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.720 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.720 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.720 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.720 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-03-08T21:15:41.720 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-03-08T21:15:41.720 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.720 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.720 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.720 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:41.720 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.720 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.720 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.720 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.720 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.720 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.720 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.721 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:41.721 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:41.721 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.721 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-08T21:15:41.721 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "stats", 2024-03-08T21:15:41.721 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-03-08T21:15:41.721 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-03-08T21:15:41.721 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-03-08T21:15:41.721 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-03-08T21:15:41.721 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-03-08T21:15:41.721 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.721 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.721 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.721 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:41.721 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.722 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.722 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:41.722 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-08T21:15:41.722 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-08T21:15:41.722 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-08T21:15:41.722 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-08T21:15:41.722 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-08T21:15:41.722 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-08T21:15:41.722 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:41.722 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.722 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.722 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.722 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.722 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.723 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-03-08T21:15:41.723 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-03-08T21:15:41.723 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.723 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.723 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.723 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:41.723 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.723 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.723 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.723 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.723 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.723 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.723 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.723 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.723 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-03-08T21:15:41.723 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-03-08T21:15:41.724 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.724 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.724 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.724 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-03-08T21:15:41.724 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.724 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.724 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:41.724 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-08T21:15:41.724 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-08T21:15:41.724 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-08T21:15:41.724 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-08T21:15:41.724 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-08T21:15:41.724 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-08T21:15:41.724 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:41.724 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.724 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.725 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.725 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.725 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.725 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-03-08T21:15:41.725 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-03-08T21:15:41.725 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.725 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.725 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.725 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:41.725 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.725 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.725 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.725 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.725 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.725 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.726 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.726 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:41.726 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:41.726 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.726 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-08T21:15:41.726 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "status", 2024-03-08T21:15:41.726 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-03-08T21:15:41.726 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-03-08T21:15:41.726 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-03-08T21:15:41.726 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-03-08T21:15:41.726 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-03-08T21:15:41.726 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.726 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.726 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.726 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:41.726 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.727 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.727 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:41.727 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-08T21:15:41.727 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-08T21:15:41.727 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-08T21:15:41.727 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-08T21:15:41.727 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-08T21:15:41.727 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-08T21:15:41.727 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:41.727 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.727 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.727 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.727 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.727 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.727 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-03-08T21:15:41.727 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-03-08T21:15:41.728 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.728 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.728 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.728 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:41.728 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.728 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.728 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.728 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.728 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.728 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.728 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.728 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.728 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-03-08T21:15:41.728 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-03-08T21:15:41.728 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.728 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.729 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.729 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-03-08T21:15:41.729 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.729 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.729 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:41.729 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-08T21:15:41.729 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-08T21:15:41.729 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-08T21:15:41.729 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-08T21:15:41.729 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-08T21:15:41.729 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-08T21:15:41.729 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:41.729 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.729 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.729 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.729 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.730 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.730 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-03-08T21:15:41.730 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-03-08T21:15:41.730 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.730 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.730 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.730 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:41.730 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.730 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.730 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.730 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.730 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.730 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.730 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.730 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:41.731 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:41.731 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.731 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-08T21:15:41.731 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "telegraf", 2024-03-08T21:15:41.731 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-03-08T21:15:41.731 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-03-08T21:15:41.731 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-03-08T21:15:41.731 INFO:teuthology.orchestra.run.smithi107.stdout: "address": { 2024-03-08T21:15:41.731 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "address", 2024-03-08T21:15:41.731 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.731 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.731 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.731 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-03-08T21:15:41.731 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.731 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.731 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.732 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.732 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.732 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.732 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.732 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.732 INFO:teuthology.orchestra.run.smithi107.stdout: "interval": { 2024-03-08T21:15:41.732 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "interval", 2024-03-08T21:15:41.732 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-03-08T21:15:41.732 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.732 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.732 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "15", 2024-03-08T21:15:41.732 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.732 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.732 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.732 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.732 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.733 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.733 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.733 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.733 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-03-08T21:15:41.733 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-03-08T21:15:41.733 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.733 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.733 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.733 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:41.733 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.733 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.733 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:41.733 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-08T21:15:41.733 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-08T21:15:41.733 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-08T21:15:41.733 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-08T21:15:41.733 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-08T21:15:41.734 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-08T21:15:41.734 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:41.734 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.734 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.734 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.734 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.734 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.734 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-03-08T21:15:41.734 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-03-08T21:15:41.734 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.734 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.734 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.734 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:41.734 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.734 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.735 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.735 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.735 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.735 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.735 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.735 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.735 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-03-08T21:15:41.735 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-03-08T21:15:41.735 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.735 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.735 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.735 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-03-08T21:15:41.735 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.735 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.735 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:41.735 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-08T21:15:41.736 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-08T21:15:41.736 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-08T21:15:41.736 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-08T21:15:41.736 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-08T21:15:41.736 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-08T21:15:41.736 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:41.736 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.736 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.736 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.736 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.736 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.736 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-03-08T21:15:41.736 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-03-08T21:15:41.736 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.736 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.736 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.737 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:41.737 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.737 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.737 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.737 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.737 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.737 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.737 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.737 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:41.737 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:41.737 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.737 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-08T21:15:41.737 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "telemetry", 2024-03-08T21:15:41.737 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-03-08T21:15:41.737 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-03-08T21:15:41.737 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-03-08T21:15:41.738 INFO:teuthology.orchestra.run.smithi107.stdout: "channel_basic": { 2024-03-08T21:15:41.738 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "channel_basic", 2024-03-08T21:15:41.738 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.738 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.738 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.738 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-08T21:15:41.738 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.738 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.738 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.738 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Share basic cluster information (size, version)", 2024-03-08T21:15:41.738 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.738 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.738 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.738 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.738 INFO:teuthology.orchestra.run.smithi107.stdout: "channel_crash": { 2024-03-08T21:15:41.739 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "channel_crash", 2024-03-08T21:15:41.739 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.739 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.739 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.739 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-08T21:15:41.739 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.739 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.739 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.739 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-03-08T21:15:41.739 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.739 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.739 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.739 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.739 INFO:teuthology.orchestra.run.smithi107.stdout: "channel_device": { 2024-03-08T21:15:41.739 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "channel_device", 2024-03-08T21:15:41.739 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.740 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.740 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.740 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-08T21:15:41.740 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.740 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.740 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.740 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-03-08T21:15:41.740 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.740 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.740 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.740 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.740 INFO:teuthology.orchestra.run.smithi107.stdout: "channel_ident": { 2024-03-08T21:15:41.740 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "channel_ident", 2024-03-08T21:15:41.740 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.740 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.740 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.741 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:41.741 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.741 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.741 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.741 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-03-08T21:15:41.741 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.741 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.741 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.741 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.741 INFO:teuthology.orchestra.run.smithi107.stdout: "channel_perf": { 2024-03-08T21:15:41.741 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "channel_perf", 2024-03-08T21:15:41.741 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.741 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.741 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.741 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:41.741 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.741 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.742 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.742 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Share various performance metrics of a cluster", 2024-03-08T21:15:41.742 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.742 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.742 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.742 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.742 INFO:teuthology.orchestra.run.smithi107.stdout: "contact": { 2024-03-08T21:15:41.742 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "contact", 2024-03-08T21:15:41.742 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.742 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.742 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.742 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:41.742 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.742 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.742 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.743 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.743 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.743 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.743 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.743 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.743 INFO:teuthology.orchestra.run.smithi107.stdout: "description": { 2024-03-08T21:15:41.743 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "description", 2024-03-08T21:15:41.743 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.743 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.743 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.743 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:41.743 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.743 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.743 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.743 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.743 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.744 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.744 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.744 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.744 INFO:teuthology.orchestra.run.smithi107.stdout: "device_url": { 2024-03-08T21:15:41.744 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "device_url", 2024-03-08T21:15:41.744 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.744 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.744 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.744 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-03-08T21:15:41.744 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.744 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.744 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.744 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.744 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.744 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.744 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.745 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.745 INFO:teuthology.orchestra.run.smithi107.stdout: "enabled": { 2024-03-08T21:15:41.745 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "enabled", 2024-03-08T21:15:41.745 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.745 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.745 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.745 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:41.745 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.745 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.745 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.745 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.745 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.745 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.745 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.745 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.745 INFO:teuthology.orchestra.run.smithi107.stdout: "interval": { 2024-03-08T21:15:41.745 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "interval", 2024-03-08T21:15:41.746 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-08T21:15:41.746 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.746 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.746 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "24", 2024-03-08T21:15:41.746 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "8", 2024-03-08T21:15:41.746 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.746 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.746 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.746 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.746 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.746 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.746 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.746 INFO:teuthology.orchestra.run.smithi107.stdout: "last_opt_revision": { 2024-03-08T21:15:41.746 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "last_opt_revision", 2024-03-08T21:15:41.746 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-08T21:15:41.747 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.747 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.747 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "1", 2024-03-08T21:15:41.747 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.747 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.747 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.747 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.747 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.747 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.747 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.747 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.747 INFO:teuthology.orchestra.run.smithi107.stdout: "leaderboard": { 2024-03-08T21:15:41.747 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "leaderboard", 2024-03-08T21:15:41.747 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.747 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.747 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.748 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:41.748 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.748 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.748 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.748 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.748 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.748 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.748 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.748 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.748 INFO:teuthology.orchestra.run.smithi107.stdout: "leaderboard_description": { 2024-03-08T21:15:41.748 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "leaderboard_description", 2024-03-08T21:15:41.748 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.748 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.748 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.748 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:41.748 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.749 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.749 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.749 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.749 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.749 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.749 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.749 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.749 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-03-08T21:15:41.749 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-03-08T21:15:41.749 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.749 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.749 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.749 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:41.749 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.749 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.750 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:41.750 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-08T21:15:41.750 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-08T21:15:41.750 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-08T21:15:41.750 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-08T21:15:41.750 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-08T21:15:41.750 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-08T21:15:41.750 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:41.750 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.750 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.750 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.750 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.750 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.750 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-03-08T21:15:41.750 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-03-08T21:15:41.750 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.751 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.751 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.751 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:41.751 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.751 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.751 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.751 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.751 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.751 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.751 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.751 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.751 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-03-08T21:15:41.751 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-03-08T21:15:41.751 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.751 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.752 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.752 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-03-08T21:15:41.752 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.752 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.752 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:41.752 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-08T21:15:41.752 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-08T21:15:41.752 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-08T21:15:41.752 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-08T21:15:41.752 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-08T21:15:41.752 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-08T21:15:41.752 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:41.752 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.753 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.753 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.753 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.753 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.753 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-03-08T21:15:41.753 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-03-08T21:15:41.753 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.753 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.753 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.753 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:41.753 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.753 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.753 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.753 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.754 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.754 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.754 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.754 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.754 INFO:teuthology.orchestra.run.smithi107.stdout: "organization": { 2024-03-08T21:15:41.754 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "organization", 2024-03-08T21:15:41.754 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.754 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.754 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.754 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:41.754 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.754 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.754 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.754 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.755 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.755 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.755 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.755 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.755 INFO:teuthology.orchestra.run.smithi107.stdout: "proxy": { 2024-03-08T21:15:41.755 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "proxy", 2024-03-08T21:15:41.755 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.755 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.755 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.755 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:41.755 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.755 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.755 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.755 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.755 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.756 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.756 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.756 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.756 INFO:teuthology.orchestra.run.smithi107.stdout: "url": { 2024-03-08T21:15:41.756 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "url", 2024-03-08T21:15:41.756 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.756 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.756 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.756 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-03-08T21:15:41.756 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.756 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.756 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.756 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.756 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.756 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.757 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.757 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:41.757 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:41.757 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.757 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-08T21:15:41.757 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "test_orchestrator", 2024-03-08T21:15:41.757 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-03-08T21:15:41.757 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-03-08T21:15:41.757 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-03-08T21:15:41.757 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-03-08T21:15:41.757 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-03-08T21:15:41.757 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.757 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.757 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.758 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:41.758 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.758 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.758 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:41.758 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-08T21:15:41.758 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-08T21:15:41.758 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-08T21:15:41.758 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-08T21:15:41.758 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-08T21:15:41.758 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-08T21:15:41.758 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:41.758 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.758 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.758 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.759 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.759 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.759 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-03-08T21:15:41.759 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-03-08T21:15:41.759 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.759 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.759 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.759 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:41.759 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.759 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.759 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.759 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.759 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.759 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.759 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.760 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.760 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-03-08T21:15:41.760 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-03-08T21:15:41.760 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.760 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.760 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.760 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-03-08T21:15:41.760 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.760 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.760 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:41.760 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-08T21:15:41.760 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-08T21:15:41.760 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-08T21:15:41.760 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-08T21:15:41.760 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-08T21:15:41.761 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-08T21:15:41.761 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:41.761 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.761 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.761 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.761 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.761 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.761 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-03-08T21:15:41.761 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-03-08T21:15:41.761 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.761 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.761 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.761 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:41.761 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.761 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.761 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.762 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.762 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.762 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.762 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.762 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:41.762 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:41.762 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.762 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-08T21:15:41.762 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "volumes", 2024-03-08T21:15:41.762 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-03-08T21:15:41.762 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-03-08T21:15:41.762 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-03-08T21:15:41.762 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-03-08T21:15:41.762 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-03-08T21:15:41.763 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.763 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.763 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.763 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:41.763 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.763 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.763 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:41.763 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-08T21:15:41.763 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-08T21:15:41.763 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-08T21:15:41.763 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-08T21:15:41.763 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-08T21:15:41.763 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-08T21:15:41.763 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:41.763 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.763 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.763 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.764 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.764 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.764 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-03-08T21:15:41.764 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-03-08T21:15:41.764 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.764 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.764 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.764 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:41.764 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.764 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.764 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.764 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.764 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.764 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.764 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.764 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.764 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-03-08T21:15:41.765 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-03-08T21:15:41.765 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.765 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.765 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.765 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-03-08T21:15:41.765 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.765 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.765 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:41.765 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-08T21:15:41.765 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-08T21:15:41.765 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-08T21:15:41.765 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-08T21:15:41.765 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-08T21:15:41.765 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-08T21:15:41.765 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:41.765 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.766 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.766 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.766 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.766 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.766 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-03-08T21:15:41.766 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-03-08T21:15:41.766 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.766 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.766 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.766 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:41.766 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.766 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.766 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.767 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.767 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.767 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.767 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.767 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.767 INFO:teuthology.orchestra.run.smithi107.stdout: "max_concurrent_clones": { 2024-03-08T21:15:41.767 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "max_concurrent_clones", 2024-03-08T21:15:41.767 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-08T21:15:41.767 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.767 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.767 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "4", 2024-03-08T21:15:41.767 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.767 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.767 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.767 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Number of asynchronous cloner threads", 2024-03-08T21:15:41.767 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.768 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.768 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.768 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.768 INFO:teuthology.orchestra.run.smithi107.stdout: "periodic_async_work": { 2024-03-08T21:15:41.768 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "periodic_async_work", 2024-03-08T21:15:41.768 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.768 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.768 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.768 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:41.768 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.768 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.768 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.768 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Periodically check for async work", 2024-03-08T21:15:41.768 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.769 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.769 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.769 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.769 INFO:teuthology.orchestra.run.smithi107.stdout: "snapshot_clone_delay": { 2024-03-08T21:15:41.769 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "snapshot_clone_delay", 2024-03-08T21:15:41.769 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-08T21:15:41.769 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.769 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.769 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0", 2024-03-08T21:15:41.769 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.769 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.769 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.769 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-03-08T21:15:41.769 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.769 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.770 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.770 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.770 INFO:teuthology.orchestra.run.smithi107.stdout: "snapshot_clone_no_wait": { 2024-03-08T21:15:41.770 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "snapshot_clone_no_wait", 2024-03-08T21:15:41.770 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.770 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.770 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.770 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-03-08T21:15:41.770 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.770 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.770 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.770 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-03-08T21:15:41.770 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.770 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.771 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.771 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:41.771 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:41.771 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.771 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-08T21:15:41.771 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "zabbix", 2024-03-08T21:15:41.771 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-03-08T21:15:41.771 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-03-08T21:15:41.771 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-03-08T21:15:41.771 INFO:teuthology.orchestra.run.smithi107.stdout: "discovery_interval": { 2024-03-08T21:15:41.771 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "discovery_interval", 2024-03-08T21:15:41.771 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "uint", 2024-03-08T21:15:41.771 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.771 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.771 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "100", 2024-03-08T21:15:41.771 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.772 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.772 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.772 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.772 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.772 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.772 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.772 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.772 INFO:teuthology.orchestra.run.smithi107.stdout: "identifier": { 2024-03-08T21:15:41.772 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "identifier", 2024-03-08T21:15:41.772 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.772 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.772 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.772 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:41.772 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.772 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.773 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.773 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.773 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.773 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.773 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.773 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.773 INFO:teuthology.orchestra.run.smithi107.stdout: "interval": { 2024-03-08T21:15:41.773 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "interval", 2024-03-08T21:15:41.773 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-03-08T21:15:41.773 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.773 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.773 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "60", 2024-03-08T21:15:41.773 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.773 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.773 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.773 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.774 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.774 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.774 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.774 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.774 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-03-08T21:15:41.774 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-03-08T21:15:41.774 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.774 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.774 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.774 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:41.774 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.774 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.774 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:41.774 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-08T21:15:41.774 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-08T21:15:41.775 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-08T21:15:41.775 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-08T21:15:41.775 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-08T21:15:41.775 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-08T21:15:41.775 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:41.775 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.775 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.775 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.775 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.775 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.775 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-03-08T21:15:41.775 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-03-08T21:15:41.775 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.775 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.775 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.775 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:41.776 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.776 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.776 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.776 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.776 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.776 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.776 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.776 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.776 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-03-08T21:15:41.776 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-03-08T21:15:41.776 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.776 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.776 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.776 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-03-08T21:15:41.776 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.776 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.777 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-03-08T21:15:41.777 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-03-08T21:15:41.777 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-03-08T21:15:41.777 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-03-08T21:15:41.777 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-03-08T21:15:41.777 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-03-08T21:15:41.777 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-03-08T21:15:41.777 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:41.777 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.777 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.777 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.777 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.777 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.777 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-03-08T21:15:41.777 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-03-08T21:15:41.777 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-03-08T21:15:41.778 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.778 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-03-08T21:15:41.778 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-03-08T21:15:41.778 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.778 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.778 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.778 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.778 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.778 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.778 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.778 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.778 INFO:teuthology.orchestra.run.smithi107.stdout: "zabbix_host": { 2024-03-08T21:15:41.778 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "zabbix_host", 2024-03-08T21:15:41.778 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.779 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.779 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.779 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-03-08T21:15:41.779 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.779 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.779 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.779 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.779 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.779 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.779 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.779 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.779 INFO:teuthology.orchestra.run.smithi107.stdout: "zabbix_port": { 2024-03-08T21:15:41.779 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "zabbix_port", 2024-03-08T21:15:41.779 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-03-08T21:15:41.779 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.779 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.780 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "10051", 2024-03-08T21:15:41.780 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.780 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.780 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.780 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.780 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.780 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.780 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.780 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.780 INFO:teuthology.orchestra.run.smithi107.stdout: "zabbix_sender": { 2024-03-08T21:15:41.780 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "zabbix_sender", 2024-03-08T21:15:41.780 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-03-08T21:15:41.780 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-03-08T21:15:41.780 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-03-08T21:15:41.780 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-03-08T21:15:41.780 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-03-08T21:15:41.780 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-03-08T21:15:41.781 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-03-08T21:15:41.781 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-03-08T21:15:41.781 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-03-08T21:15:41.781 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-03-08T21:15:41.781 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-03-08T21:15:41.781 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:41.781 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:41.781 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:41.781 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:41.781 INFO:teuthology.orchestra.run.smithi107.stdout: "services": { 2024-03-08T21:15:41.781 INFO:teuthology.orchestra.run.smithi107.stdout: "dashboard": "https://172.21.15.107:8443/" 2024-03-08T21:15:41.781 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.781 INFO:teuthology.orchestra.run.smithi107.stdout: "always_on_modules": { 2024-03-08T21:15:41.781 INFO:teuthology.orchestra.run.smithi107.stdout: "octopus": [ 2024-03-08T21:15:41.781 INFO:teuthology.orchestra.run.smithi107.stdout: "balancer", 2024-03-08T21:15:41.781 INFO:teuthology.orchestra.run.smithi107.stdout: "crash", 2024-03-08T21:15:41.782 INFO:teuthology.orchestra.run.smithi107.stdout: "devicehealth", 2024-03-08T21:15:41.782 INFO:teuthology.orchestra.run.smithi107.stdout: "orchestrator", 2024-03-08T21:15:41.782 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_autoscaler", 2024-03-08T21:15:41.782 INFO:teuthology.orchestra.run.smithi107.stdout: "progress", 2024-03-08T21:15:41.782 INFO:teuthology.orchestra.run.smithi107.stdout: "rbd_support", 2024-03-08T21:15:41.782 INFO:teuthology.orchestra.run.smithi107.stdout: "status", 2024-03-08T21:15:41.782 INFO:teuthology.orchestra.run.smithi107.stdout: "telemetry", 2024-03-08T21:15:41.782 INFO:teuthology.orchestra.run.smithi107.stdout: "volumes" 2024-03-08T21:15:41.782 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:41.782 INFO:teuthology.orchestra.run.smithi107.stdout: "pacific": [ 2024-03-08T21:15:41.782 INFO:teuthology.orchestra.run.smithi107.stdout: "balancer", 2024-03-08T21:15:41.782 INFO:teuthology.orchestra.run.smithi107.stdout: "crash", 2024-03-08T21:15:41.782 INFO:teuthology.orchestra.run.smithi107.stdout: "devicehealth", 2024-03-08T21:15:41.782 INFO:teuthology.orchestra.run.smithi107.stdout: "orchestrator", 2024-03-08T21:15:41.782 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_autoscaler", 2024-03-08T21:15:41.783 INFO:teuthology.orchestra.run.smithi107.stdout: "progress", 2024-03-08T21:15:41.783 INFO:teuthology.orchestra.run.smithi107.stdout: "rbd_support", 2024-03-08T21:15:41.783 INFO:teuthology.orchestra.run.smithi107.stdout: "status", 2024-03-08T21:15:41.783 INFO:teuthology.orchestra.run.smithi107.stdout: "telemetry", 2024-03-08T21:15:41.783 INFO:teuthology.orchestra.run.smithi107.stdout: "volumes" 2024-03-08T21:15:41.783 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:41.783 INFO:teuthology.orchestra.run.smithi107.stdout: "quincy": [ 2024-03-08T21:15:41.783 INFO:teuthology.orchestra.run.smithi107.stdout: "balancer", 2024-03-08T21:15:41.783 INFO:teuthology.orchestra.run.smithi107.stdout: "crash", 2024-03-08T21:15:41.783 INFO:teuthology.orchestra.run.smithi107.stdout: "devicehealth", 2024-03-08T21:15:41.783 INFO:teuthology.orchestra.run.smithi107.stdout: "orchestrator", 2024-03-08T21:15:41.783 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_autoscaler", 2024-03-08T21:15:41.783 INFO:teuthology.orchestra.run.smithi107.stdout: "progress", 2024-03-08T21:15:41.783 INFO:teuthology.orchestra.run.smithi107.stdout: "rbd_support", 2024-03-08T21:15:41.783 INFO:teuthology.orchestra.run.smithi107.stdout: "status", 2024-03-08T21:15:41.783 INFO:teuthology.orchestra.run.smithi107.stdout: "telemetry", 2024-03-08T21:15:41.784 INFO:teuthology.orchestra.run.smithi107.stdout: "volumes" 2024-03-08T21:15:41.784 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:41.784 INFO:teuthology.orchestra.run.smithi107.stdout: "reef": [ 2024-03-08T21:15:41.784 INFO:teuthology.orchestra.run.smithi107.stdout: "balancer", 2024-03-08T21:15:41.784 INFO:teuthology.orchestra.run.smithi107.stdout: "crash", 2024-03-08T21:15:41.784 INFO:teuthology.orchestra.run.smithi107.stdout: "devicehealth", 2024-03-08T21:15:41.784 INFO:teuthology.orchestra.run.smithi107.stdout: "orchestrator", 2024-03-08T21:15:41.784 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_autoscaler", 2024-03-08T21:15:41.784 INFO:teuthology.orchestra.run.smithi107.stdout: "progress", 2024-03-08T21:15:41.784 INFO:teuthology.orchestra.run.smithi107.stdout: "rbd_support", 2024-03-08T21:15:41.784 INFO:teuthology.orchestra.run.smithi107.stdout: "status", 2024-03-08T21:15:41.784 INFO:teuthology.orchestra.run.smithi107.stdout: "telemetry", 2024-03-08T21:15:41.784 INFO:teuthology.orchestra.run.smithi107.stdout: "volumes" 2024-03-08T21:15:41.784 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-03-08T21:15:41.784 INFO:teuthology.orchestra.run.smithi107.stdout: "squid": [ 2024-03-08T21:15:41.784 INFO:teuthology.orchestra.run.smithi107.stdout: "balancer", 2024-03-08T21:15:41.785 INFO:teuthology.orchestra.run.smithi107.stdout: "crash", 2024-03-08T21:15:41.785 INFO:teuthology.orchestra.run.smithi107.stdout: "devicehealth", 2024-03-08T21:15:41.785 INFO:teuthology.orchestra.run.smithi107.stdout: "orchestrator", 2024-03-08T21:15:41.785 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_autoscaler", 2024-03-08T21:15:41.785 INFO:teuthology.orchestra.run.smithi107.stdout: "progress", 2024-03-08T21:15:41.785 INFO:teuthology.orchestra.run.smithi107.stdout: "rbd_support", 2024-03-08T21:15:41.785 INFO:teuthology.orchestra.run.smithi107.stdout: "status", 2024-03-08T21:15:41.785 INFO:teuthology.orchestra.run.smithi107.stdout: "telemetry", 2024-03-08T21:15:41.785 INFO:teuthology.orchestra.run.smithi107.stdout: "volumes" 2024-03-08T21:15:41.785 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-03-08T21:15:41.785 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.785 INFO:teuthology.orchestra.run.smithi107.stdout: "last_failure_osd_epoch": 89, 2024-03-08T21:15:41.785 INFO:teuthology.orchestra.run.smithi107.stdout: "active_clients": [ 2024-03-08T21:15:41.785 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-08T21:15:41.785 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "devicehealth", 2024-03-08T21:15:41.785 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-03-08T21:15:41.785 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-08T21:15:41.786 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-03-08T21:15:41.786 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:0", 2024-03-08T21:15:41.786 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3694811581 2024-03-08T21:15:41.786 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:41.786 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-03-08T21:15:41.786 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.786 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-08T21:15:41.786 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "libcephsqlite", 2024-03-08T21:15:41.786 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-03-08T21:15:41.786 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-08T21:15:41.786 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-03-08T21:15:41.786 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:0", 2024-03-08T21:15:41.786 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 1621279818 2024-03-08T21:15:41.786 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:41.786 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-03-08T21:15:41.787 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.787 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-08T21:15:41.787 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rbd_support", 2024-03-08T21:15:41.787 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-03-08T21:15:41.787 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-08T21:15:41.787 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-03-08T21:15:41.787 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:0", 2024-03-08T21:15:41.787 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 486877722 2024-03-08T21:15:41.787 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:41.787 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-03-08T21:15:41.787 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.787 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-08T21:15:41.787 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "snap_schedule", 2024-03-08T21:15:41.787 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-03-08T21:15:41.787 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-08T21:15:41.787 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-03-08T21:15:41.788 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:0", 2024-03-08T21:15:41.788 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 2871391428 2024-03-08T21:15:41.788 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:41.788 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-03-08T21:15:41.788 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.788 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-08T21:15:41.788 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "snap_schedule", 2024-03-08T21:15:41.788 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-03-08T21:15:41.788 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-08T21:15:41.788 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-03-08T21:15:41.788 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:0", 2024-03-08T21:15:41.788 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3391605585 2024-03-08T21:15:41.788 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:41.788 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-03-08T21:15:41.788 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:41.789 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-08T21:15:41.789 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "volumes", 2024-03-08T21:15:41.789 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-03-08T21:15:41.789 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-03-08T21:15:41.789 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-03-08T21:15:41.789 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:0", 2024-03-08T21:15:41.789 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 2008684429 2024-03-08T21:15:41.789 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:41.789 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-03-08T21:15:41.789 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:41.789 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-03-08T21:15:41.789 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-03-08T21:15:41.794 DEBUG:tasks.check_counter:Getting stats from a 2024-03-08T21:15:41.795 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761 shell --fsid dc20f0f8-dd8c-11ee-95c6-87774f69a715 -- ceph daemon mds.a perf dump 2024-03-08T21:15:41.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:15:40 smithi107 ceph-mon[31381]: pgmap v480: 97 pgs: 97 active+clean; 17 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 1022 KiB/s rd, 1.1 MiB/s wr, 210 op/s 2024-03-08T21:15:41.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:15:40 smithi107 ceph-mon[31381]: from='client.? 172.21.15.107:0/3323855843' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-08T21:15:41.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:15:40 smithi107 ceph-mon[31381]: from='client.? 172.21.15.107:0/492498522' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-03-08T21:15:42.248 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/mon.a/config 2024-03-08T21:15:42.711 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:15:42 smithi107 ceph-mon[31381]: pgmap v481: 97 pgs: 97 active+clean; 17 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 623 KiB/s rd, 587 KiB/s wr, 131 op/s 2024-03-08T21:15:42.871 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:15:42 smithi204 ceph-mon[34935]: pgmap v481: 97 pgs: 97 active+clean; 17 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 623 KiB/s rd, 587 KiB/s wr, 131 op/s 2024-03-08T21:15:43.008 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:15:42 smithi149 ceph-mon[35042]: pgmap v481: 97 pgs: 97 active+clean; 17 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 623 KiB/s rd, 587 KiB/s wr, 131 op/s 2024-03-08T21:15:44.711 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:15:44 smithi107 ceph-mon[31381]: pgmap v482: 97 pgs: 97 active+clean; 17 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 623 KiB/s rd, 587 KiB/s wr, 131 op/s 2024-03-08T21:15:44.714 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-03-08T21:15:44.714 INFO:teuthology.orchestra.run.smithi107.stdout: "AsyncMessenger::Worker-0": { 2024-03-08T21:15:44.714 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_messages": 1089, 2024-03-08T21:15:44.714 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages": 1091, 2024-03-08T21:15:44.714 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_bytes": 364645, 2024-03-08T21:15:44.714 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_bytes": 346303, 2024-03-08T21:15:44.715 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_created_connections": 6, 2024-03-08T21:15:44.715 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_active_connections": 2, 2024-03-08T21:15:44.715 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_total_time": 0.248662100, 2024-03-08T21:15:44.715 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_send_time": 0.091029872, 2024-03-08T21:15:44.715 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_recv_time": 0.061237002, 2024-03-08T21:15:44.715 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_fast_dispatch_time": 0.068504757, 2024-03-08T21:15:44.715 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages_queue_lat": { 2024-03-08T21:15:44.715 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 1089, 2024-03-08T21:15:44.715 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.099659123, 2024-03-08T21:15:44.715 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000091514 2024-03-08T21:15:44.715 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:44.715 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_handle_ack_lat": { 2024-03-08T21:15:44.715 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-03-08T21:15:44.716 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-03-08T21:15:44.716 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-03-08T21:15:44.716 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:44.716 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-08T21:15:44.716 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_encrypted_bytes": 0 2024-03-08T21:15:44.716 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:44.716 INFO:teuthology.orchestra.run.smithi107.stdout: "AsyncMessenger::Worker-1": { 2024-03-08T21:15:44.716 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_messages": 933, 2024-03-08T21:15:44.716 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages": 1150, 2024-03-08T21:15:44.716 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_bytes": 232054, 2024-03-08T21:15:44.716 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_bytes": 846694, 2024-03-08T21:15:44.716 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_created_connections": 8, 2024-03-08T21:15:44.716 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_active_connections": 5, 2024-03-08T21:15:44.716 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_total_time": 0.256638953, 2024-03-08T21:15:44.716 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_send_time": 0.105922842, 2024-03-08T21:15:44.717 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_recv_time": 0.054193052, 2024-03-08T21:15:44.717 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_fast_dispatch_time": 0.069360856, 2024-03-08T21:15:44.717 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages_queue_lat": { 2024-03-08T21:15:44.717 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 1148, 2024-03-08T21:15:44.717 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.106353253, 2024-03-08T21:15:44.717 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000092642 2024-03-08T21:15:44.717 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:44.717 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_handle_ack_lat": { 2024-03-08T21:15:44.717 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-03-08T21:15:44.717 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-03-08T21:15:44.717 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-03-08T21:15:44.717 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:44.717 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_encrypted_bytes": 3360, 2024-03-08T21:15:44.718 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_encrypted_bytes": 551840 2024-03-08T21:15:44.718 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:44.718 INFO:teuthology.orchestra.run.smithi107.stdout: "AsyncMessenger::Worker-2": { 2024-03-08T21:15:44.718 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_messages": 1391, 2024-03-08T21:15:44.718 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages": 1366, 2024-03-08T21:15:44.718 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_bytes": 1851639, 2024-03-08T21:15:44.718 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_bytes": 512010, 2024-03-08T21:15:44.718 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_created_connections": 3, 2024-03-08T21:15:44.718 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_active_connections": 2, 2024-03-08T21:15:44.718 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_total_time": 0.352452368, 2024-03-08T21:15:44.718 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_send_time": 0.131648656, 2024-03-08T21:15:44.718 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_recv_time": 0.155204870, 2024-03-08T21:15:44.718 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_fast_dispatch_time": 0.070914314, 2024-03-08T21:15:44.718 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages_queue_lat": { 2024-03-08T21:15:44.719 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 1365, 2024-03-08T21:15:44.719 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.111244008, 2024-03-08T21:15:44.719 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000081497 2024-03-08T21:15:44.719 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:44.719 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_handle_ack_lat": { 2024-03-08T21:15:44.719 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-03-08T21:15:44.719 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-03-08T21:15:44.719 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-03-08T21:15:44.719 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:44.719 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_encrypted_bytes": 1577456, 2024-03-08T21:15:44.719 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_encrypted_bytes": 180336 2024-03-08T21:15:44.719 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:44.719 INFO:teuthology.orchestra.run.smithi107.stdout: "cct": { 2024-03-08T21:15:44.720 INFO:teuthology.orchestra.run.smithi107.stdout: "total_workers": 1, 2024-03-08T21:15:44.720 INFO:teuthology.orchestra.run.smithi107.stdout: "unhealthy_workers": 0 2024-03-08T21:15:44.720 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:44.720 INFO:teuthology.orchestra.run.smithi107.stdout: "finisher-MDSRank": { 2024-03-08T21:15:44.720 INFO:teuthology.orchestra.run.smithi107.stdout: "queue_len": 0, 2024-03-08T21:15:44.720 INFO:teuthology.orchestra.run.smithi107.stdout: "complete_latency": { 2024-03-08T21:15:44.720 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 4963, 2024-03-08T21:15:44.720 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.288067584, 2024-03-08T21:15:44.720 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000058043 2024-03-08T21:15:44.720 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:44.720 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:44.720 INFO:teuthology.orchestra.run.smithi107.stdout: "finisher-PurgeQueue": { 2024-03-08T21:15:44.720 INFO:teuthology.orchestra.run.smithi107.stdout: "queue_len": 0, 2024-03-08T21:15:44.720 INFO:teuthology.orchestra.run.smithi107.stdout: "complete_latency": { 2024-03-08T21:15:44.721 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-03-08T21:15:44.721 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-03-08T21:15:44.721 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-03-08T21:15:44.721 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:44.721 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:44.721 INFO:teuthology.orchestra.run.smithi107.stdout: "mds": { 2024-03-08T21:15:44.721 INFO:teuthology.orchestra.run.smithi107.stdout: "request": 0, 2024-03-08T21:15:44.721 INFO:teuthology.orchestra.run.smithi107.stdout: "reply": 0, 2024-03-08T21:15:44.721 INFO:teuthology.orchestra.run.smithi107.stdout: "reply_latency": { 2024-03-08T21:15:44.721 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-03-08T21:15:44.721 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-03-08T21:15:44.721 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-03-08T21:15:44.721 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:44.721 INFO:teuthology.orchestra.run.smithi107.stdout: "slow_reply": 0, 2024-03-08T21:15:44.721 INFO:teuthology.orchestra.run.smithi107.stdout: "forward": 0, 2024-03-08T21:15:44.722 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_fetch_complete": 0, 2024-03-08T21:15:44.722 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_fetch_keys": 0, 2024-03-08T21:15:44.722 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_commit": 0, 2024-03-08T21:15:44.722 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_split": 0, 2024-03-08T21:15:44.722 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_merge": 0, 2024-03-08T21:15:44.722 INFO:teuthology.orchestra.run.smithi107.stdout: "inodes": 0, 2024-03-08T21:15:44.722 INFO:teuthology.orchestra.run.smithi107.stdout: "inodes_top": 0, 2024-03-08T21:15:44.722 INFO:teuthology.orchestra.run.smithi107.stdout: "inodes_bottom": 0, 2024-03-08T21:15:44.722 INFO:teuthology.orchestra.run.smithi107.stdout: "inodes_pin_tail": 0, 2024-03-08T21:15:44.722 INFO:teuthology.orchestra.run.smithi107.stdout: "inodes_pinned": 0, 2024-03-08T21:15:44.722 INFO:teuthology.orchestra.run.smithi107.stdout: "inodes_expired": 0, 2024-03-08T21:15:44.722 INFO:teuthology.orchestra.run.smithi107.stdout: "inodes_with_caps": 0, 2024-03-08T21:15:44.722 INFO:teuthology.orchestra.run.smithi107.stdout: "caps": 0, 2024-03-08T21:15:44.722 INFO:teuthology.orchestra.run.smithi107.stdout: "subtrees": 1, 2024-03-08T21:15:44.723 INFO:teuthology.orchestra.run.smithi107.stdout: "traverse": 0, 2024-03-08T21:15:44.723 INFO:teuthology.orchestra.run.smithi107.stdout: "traverse_hit": 0, 2024-03-08T21:15:44.723 INFO:teuthology.orchestra.run.smithi107.stdout: "traverse_forward": 0, 2024-03-08T21:15:44.723 INFO:teuthology.orchestra.run.smithi107.stdout: "traverse_discover": 0, 2024-03-08T21:15:44.723 INFO:teuthology.orchestra.run.smithi107.stdout: "traverse_dir_fetch": 0, 2024-03-08T21:15:44.723 INFO:teuthology.orchestra.run.smithi107.stdout: "traverse_remote_ino": 0, 2024-03-08T21:15:44.723 INFO:teuthology.orchestra.run.smithi107.stdout: "traverse_lock": 0, 2024-03-08T21:15:44.723 INFO:teuthology.orchestra.run.smithi107.stdout: "load_cent": 0, 2024-03-08T21:15:44.723 INFO:teuthology.orchestra.run.smithi107.stdout: "q": 0, 2024-03-08T21:15:44.723 INFO:teuthology.orchestra.run.smithi107.stdout: "exported": 0, 2024-03-08T21:15:44.723 INFO:teuthology.orchestra.run.smithi107.stdout: "exported_inodes": 0, 2024-03-08T21:15:44.723 INFO:teuthology.orchestra.run.smithi107.stdout: "imported": 0, 2024-03-08T21:15:44.723 INFO:teuthology.orchestra.run.smithi107.stdout: "imported_inodes": 0, 2024-03-08T21:15:44.723 INFO:teuthology.orchestra.run.smithi107.stdout: "openino_dir_fetch": 0, 2024-03-08T21:15:44.723 INFO:teuthology.orchestra.run.smithi107.stdout: "openino_backtrace_fetch": 0, 2024-03-08T21:15:44.724 INFO:teuthology.orchestra.run.smithi107.stdout: "openino_peer_discover": 0, 2024-03-08T21:15:44.724 INFO:teuthology.orchestra.run.smithi107.stdout: "root_rfiles": 0, 2024-03-08T21:15:44.724 INFO:teuthology.orchestra.run.smithi107.stdout: "root_rbytes": 0, 2024-03-08T21:15:44.724 INFO:teuthology.orchestra.run.smithi107.stdout: "root_rsnaps": 0, 2024-03-08T21:15:44.724 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_backtrace_fetch": 0, 2024-03-08T21:15:44.724 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_set_tag": 0, 2024-03-08T21:15:44.724 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_backtrace_repaired": 0, 2024-03-08T21:15:44.724 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_inotable_repaired": 0, 2024-03-08T21:15:44.724 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_dir_inodes": 0, 2024-03-08T21:15:44.724 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_dir_base_inodes": 0, 2024-03-08T21:15:44.724 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_dirfrag_rstats": 0, 2024-03-08T21:15:44.724 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_file_inodes": 0, 2024-03-08T21:15:44.724 INFO:teuthology.orchestra.run.smithi107.stdout: "handle_inode_file_caps": 0, 2024-03-08T21:15:44.724 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph_cap_op_revoke": 0, 2024-03-08T21:15:44.725 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph_cap_op_grant": 0, 2024-03-08T21:15:44.725 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph_cap_op_trunc": 0, 2024-03-08T21:15:44.725 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-08T21:15:44.725 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-08T21:15:44.725 INFO:teuthology.orchestra.run.smithi107.stdout: "handle_client_caps": 0, 2024-03-08T21:15:44.725 INFO:teuthology.orchestra.run.smithi107.stdout: "handle_client_caps_dirty": 0, 2024-03-08T21:15:44.725 INFO:teuthology.orchestra.run.smithi107.stdout: "handle_client_cap_release": 0, 2024-03-08T21:15:44.725 INFO:teuthology.orchestra.run.smithi107.stdout: "process_request_cap_release": 0 2024-03-08T21:15:44.725 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:44.725 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_cache": { 2024-03-08T21:15:44.725 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_update": 0, 2024-03-08T21:15:44.725 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_update_receipt": 0, 2024-03-08T21:15:44.725 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_try_discover": 0, 2024-03-08T21:15:44.725 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_send_discover": 0, 2024-03-08T21:15:44.726 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_handle_discover": 0, 2024-03-08T21:15:44.726 INFO:teuthology.orchestra.run.smithi107.stdout: "num_strays": 0, 2024-03-08T21:15:44.726 INFO:teuthology.orchestra.run.smithi107.stdout: "num_strays_delayed": 0, 2024-03-08T21:15:44.726 INFO:teuthology.orchestra.run.smithi107.stdout: "num_strays_enqueuing": 0, 2024-03-08T21:15:44.726 INFO:teuthology.orchestra.run.smithi107.stdout: "strays_created": 0, 2024-03-08T21:15:44.726 INFO:teuthology.orchestra.run.smithi107.stdout: "strays_enqueued": 0, 2024-03-08T21:15:44.726 INFO:teuthology.orchestra.run.smithi107.stdout: "strays_reintegrated": 0, 2024-03-08T21:15:44.726 INFO:teuthology.orchestra.run.smithi107.stdout: "strays_migrated": 0, 2024-03-08T21:15:44.726 INFO:teuthology.orchestra.run.smithi107.stdout: "num_recovering_processing": 0, 2024-03-08T21:15:44.726 INFO:teuthology.orchestra.run.smithi107.stdout: "num_recovering_enqueued": 0, 2024-03-08T21:15:44.726 INFO:teuthology.orchestra.run.smithi107.stdout: "num_recovering_prioritized": 0, 2024-03-08T21:15:44.726 INFO:teuthology.orchestra.run.smithi107.stdout: "recovery_started": 0, 2024-03-08T21:15:44.726 INFO:teuthology.orchestra.run.smithi107.stdout: "recovery_completed": 0, 2024-03-08T21:15:44.727 INFO:teuthology.orchestra.run.smithi107.stdout: "ireq_enqueue_scrub": 0, 2024-03-08T21:15:44.727 INFO:teuthology.orchestra.run.smithi107.stdout: "ireq_exportdir": 0, 2024-03-08T21:15:44.727 INFO:teuthology.orchestra.run.smithi107.stdout: "ireq_flush": 0, 2024-03-08T21:15:44.727 INFO:teuthology.orchestra.run.smithi107.stdout: "ireq_fragmentdir": 0, 2024-03-08T21:15:44.727 INFO:teuthology.orchestra.run.smithi107.stdout: "ireq_fragstats": 0, 2024-03-08T21:15:44.727 INFO:teuthology.orchestra.run.smithi107.stdout: "ireq_inodestats": 0 2024-03-08T21:15:44.727 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:44.727 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_log": { 2024-03-08T21:15:44.727 INFO:teuthology.orchestra.run.smithi107.stdout: "evlrg": 0, 2024-03-08T21:15:44.727 INFO:teuthology.orchestra.run.smithi107.stdout: "evadd": 0, 2024-03-08T21:15:44.727 INFO:teuthology.orchestra.run.smithi107.stdout: "evex": 0, 2024-03-08T21:15:44.727 INFO:teuthology.orchestra.run.smithi107.stdout: "evtrm": 0, 2024-03-08T21:15:44.727 INFO:teuthology.orchestra.run.smithi107.stdout: "ev": 3, 2024-03-08T21:15:44.727 INFO:teuthology.orchestra.run.smithi107.stdout: "evexg": 0, 2024-03-08T21:15:44.728 INFO:teuthology.orchestra.run.smithi107.stdout: "evexd": 0, 2024-03-08T21:15:44.728 INFO:teuthology.orchestra.run.smithi107.stdout: "segadd": 0, 2024-03-08T21:15:44.728 INFO:teuthology.orchestra.run.smithi107.stdout: "segex": 0, 2024-03-08T21:15:44.728 INFO:teuthology.orchestra.run.smithi107.stdout: "segtrm": 0, 2024-03-08T21:15:44.728 INFO:teuthology.orchestra.run.smithi107.stdout: "seg": 2, 2024-03-08T21:15:44.728 INFO:teuthology.orchestra.run.smithi107.stdout: "segmjr": 2, 2024-03-08T21:15:44.728 INFO:teuthology.orchestra.run.smithi107.stdout: "segexg": 0, 2024-03-08T21:15:44.728 INFO:teuthology.orchestra.run.smithi107.stdout: "segexd": 0, 2024-03-08T21:15:44.728 INFO:teuthology.orchestra.run.smithi107.stdout: "expos": 4194304, 2024-03-08T21:15:44.728 INFO:teuthology.orchestra.run.smithi107.stdout: "wrpos": 4195385, 2024-03-08T21:15:44.728 INFO:teuthology.orchestra.run.smithi107.stdout: "rdpos": 4194892, 2024-03-08T21:15:44.728 INFO:teuthology.orchestra.run.smithi107.stdout: "jlat": { 2024-03-08T21:15:44.728 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-03-08T21:15:44.728 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-03-08T21:15:44.729 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-03-08T21:15:44.729 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:44.729 INFO:teuthology.orchestra.run.smithi107.stdout: "replayed": 3 2024-03-08T21:15:44.729 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:44.729 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_mem": { 2024-03-08T21:15:44.729 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 3, 2024-03-08T21:15:44.729 INFO:teuthology.orchestra.run.smithi107.stdout: "ino+": 3, 2024-03-08T21:15:44.729 INFO:teuthology.orchestra.run.smithi107.stdout: "ino-": 0, 2024-03-08T21:15:44.729 INFO:teuthology.orchestra.run.smithi107.stdout: "dir": 1, 2024-03-08T21:15:44.729 INFO:teuthology.orchestra.run.smithi107.stdout: "dir+": 1, 2024-03-08T21:15:44.729 INFO:teuthology.orchestra.run.smithi107.stdout: "dir-": 0, 2024-03-08T21:15:44.729 INFO:teuthology.orchestra.run.smithi107.stdout: "dn": 0, 2024-03-08T21:15:44.730 INFO:teuthology.orchestra.run.smithi107.stdout: "dn+": 0, 2024-03-08T21:15:44.730 INFO:teuthology.orchestra.run.smithi107.stdout: "dn-": 0, 2024-03-08T21:15:44.730 INFO:teuthology.orchestra.run.smithi107.stdout: "cap": 0, 2024-03-08T21:15:44.730 INFO:teuthology.orchestra.run.smithi107.stdout: "cap+": 0, 2024-03-08T21:15:44.730 INFO:teuthology.orchestra.run.smithi107.stdout: "cap-": 0, 2024-03-08T21:15:44.730 INFO:teuthology.orchestra.run.smithi107.stdout: "rss": 48640, 2024-03-08T21:15:44.730 INFO:teuthology.orchestra.run.smithi107.stdout: "heap": 215420 2024-03-08T21:15:44.730 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:44.730 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_server": { 2024-03-08T21:15:44.730 INFO:teuthology.orchestra.run.smithi107.stdout: "dispatch_client_request": 0, 2024-03-08T21:15:44.730 INFO:teuthology.orchestra.run.smithi107.stdout: "dispatch_server_request": 0, 2024-03-08T21:15:44.730 INFO:teuthology.orchestra.run.smithi107.stdout: "handle_client_request": 0, 2024-03-08T21:15:44.730 INFO:teuthology.orchestra.run.smithi107.stdout: "handle_client_session": 0, 2024-03-08T21:15:44.730 INFO:teuthology.orchestra.run.smithi107.stdout: "handle_peer_request": 0, 2024-03-08T21:15:44.731 INFO:teuthology.orchestra.run.smithi107.stdout: "req_create_latency": { 2024-03-08T21:15:44.731 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-03-08T21:15:44.731 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-03-08T21:15:44.731 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-03-08T21:15:44.731 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:44.731 INFO:teuthology.orchestra.run.smithi107.stdout: "req_getattr_latency": { 2024-03-08T21:15:44.731 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-03-08T21:15:44.731 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-03-08T21:15:44.731 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-03-08T21:15:44.731 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:44.731 INFO:teuthology.orchestra.run.smithi107.stdout: "req_getfilelock_latency": { 2024-03-08T21:15:44.731 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-03-08T21:15:44.731 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-03-08T21:15:44.732 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-03-08T21:15:44.732 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:44.732 INFO:teuthology.orchestra.run.smithi107.stdout: "req_link_latency": { 2024-03-08T21:15:44.732 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-03-08T21:15:44.732 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-03-08T21:15:44.732 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-03-08T21:15:44.732 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:44.732 INFO:teuthology.orchestra.run.smithi107.stdout: "req_lookup_latency": { 2024-03-08T21:15:44.732 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-03-08T21:15:44.732 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-03-08T21:15:44.732 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-03-08T21:15:44.732 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:44.732 INFO:teuthology.orchestra.run.smithi107.stdout: "req_lookuphash_latency": { 2024-03-08T21:15:44.732 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-03-08T21:15:44.732 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-03-08T21:15:44.733 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-03-08T21:15:44.733 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:44.733 INFO:teuthology.orchestra.run.smithi107.stdout: "req_lookupino_latency": { 2024-03-08T21:15:44.733 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-03-08T21:15:44.733 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-03-08T21:15:44.733 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-03-08T21:15:44.733 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:44.733 INFO:teuthology.orchestra.run.smithi107.stdout: "req_lookupname_latency": { 2024-03-08T21:15:44.733 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-03-08T21:15:44.733 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-03-08T21:15:44.733 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-03-08T21:15:44.733 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:44.733 INFO:teuthology.orchestra.run.smithi107.stdout: "req_lookupparent_latency": { 2024-03-08T21:15:44.733 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-03-08T21:15:44.733 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-03-08T21:15:44.734 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-03-08T21:15:44.734 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:44.734 INFO:teuthology.orchestra.run.smithi107.stdout: "req_lookupsnap_latency": { 2024-03-08T21:15:44.734 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-03-08T21:15:44.734 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-03-08T21:15:44.734 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-03-08T21:15:44.734 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:44.734 INFO:teuthology.orchestra.run.smithi107.stdout: "req_lssnap_latency": { 2024-03-08T21:15:44.734 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-03-08T21:15:44.734 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-03-08T21:15:44.734 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-03-08T21:15:44.734 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:44.734 INFO:teuthology.orchestra.run.smithi107.stdout: "req_mkdir_latency": { 2024-03-08T21:15:44.734 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-03-08T21:15:44.735 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-03-08T21:15:44.735 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-03-08T21:15:44.735 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:44.735 INFO:teuthology.orchestra.run.smithi107.stdout: "req_mknod_latency": { 2024-03-08T21:15:44.736 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-03-08T21:15:44.736 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-03-08T21:15:44.736 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-03-08T21:15:44.736 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:44.736 INFO:teuthology.orchestra.run.smithi107.stdout: "req_mksnap_latency": { 2024-03-08T21:15:44.736 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-03-08T21:15:44.736 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-03-08T21:15:44.736 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-03-08T21:15:44.736 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:44.736 INFO:teuthology.orchestra.run.smithi107.stdout: "req_open_latency": { 2024-03-08T21:15:44.736 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-03-08T21:15:44.736 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-03-08T21:15:44.737 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-03-08T21:15:44.737 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:44.737 INFO:teuthology.orchestra.run.smithi107.stdout: "req_readdir_latency": { 2024-03-08T21:15:44.737 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-03-08T21:15:44.737 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-03-08T21:15:44.737 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-03-08T21:15:44.737 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:44.737 INFO:teuthology.orchestra.run.smithi107.stdout: "req_rename_latency": { 2024-03-08T21:15:44.737 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-03-08T21:15:44.737 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-03-08T21:15:44.737 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-03-08T21:15:44.737 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:44.737 INFO:teuthology.orchestra.run.smithi107.stdout: "req_renamesnap_latency": { 2024-03-08T21:15:44.737 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-03-08T21:15:44.738 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-03-08T21:15:44.738 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-03-08T21:15:44.738 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:44.738 INFO:teuthology.orchestra.run.smithi107.stdout: "req_snapdiff_latency": { 2024-03-08T21:15:44.738 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-03-08T21:15:44.738 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-03-08T21:15:44.738 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-03-08T21:15:44.738 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:44.738 INFO:teuthology.orchestra.run.smithi107.stdout: "req_rmdir_latency": { 2024-03-08T21:15:44.738 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-03-08T21:15:44.738 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-03-08T21:15:44.738 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-03-08T21:15:44.738 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:44.739 INFO:teuthology.orchestra.run.smithi107.stdout: "req_rmsnap_latency": { 2024-03-08T21:15:44.739 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-03-08T21:15:44.739 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-03-08T21:15:44.739 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-03-08T21:15:44.739 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:44.739 INFO:teuthology.orchestra.run.smithi107.stdout: "req_rmxattr_latency": { 2024-03-08T21:15:44.739 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-03-08T21:15:44.739 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-03-08T21:15:44.739 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-03-08T21:15:44.739 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:44.739 INFO:teuthology.orchestra.run.smithi107.stdout: "req_setattr_latency": { 2024-03-08T21:15:44.739 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-03-08T21:15:44.739 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-03-08T21:15:44.740 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-03-08T21:15:44.740 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:44.740 INFO:teuthology.orchestra.run.smithi107.stdout: "req_setdirlayout_latency": { 2024-03-08T21:15:44.740 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-03-08T21:15:44.740 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-03-08T21:15:44.740 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-03-08T21:15:44.740 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:44.740 INFO:teuthology.orchestra.run.smithi107.stdout: "req_setfilelock_latency": { 2024-03-08T21:15:44.740 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-03-08T21:15:44.740 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-03-08T21:15:44.740 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-03-08T21:15:44.740 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:44.741 INFO:teuthology.orchestra.run.smithi107.stdout: "req_setlayout_latency": { 2024-03-08T21:15:44.741 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-03-08T21:15:44.741 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-03-08T21:15:44.741 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-03-08T21:15:44.741 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:44.741 INFO:teuthology.orchestra.run.smithi107.stdout: "req_setxattr_latency": { 2024-03-08T21:15:44.741 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-03-08T21:15:44.741 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-03-08T21:15:44.741 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-03-08T21:15:44.741 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:44.741 INFO:teuthology.orchestra.run.smithi107.stdout: "req_symlink_latency": { 2024-03-08T21:15:44.741 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-03-08T21:15:44.741 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-03-08T21:15:44.741 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-03-08T21:15:44.742 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:44.742 INFO:teuthology.orchestra.run.smithi107.stdout: "req_unlink_latency": { 2024-03-08T21:15:44.742 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-03-08T21:15:44.742 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-03-08T21:15:44.742 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-03-08T21:15:44.742 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:44.742 INFO:teuthology.orchestra.run.smithi107.stdout: "cap_revoke_eviction": 0, 2024-03-08T21:15:44.742 INFO:teuthology.orchestra.run.smithi107.stdout: "cap_acquisition_throttle": 0, 2024-03-08T21:15:44.742 INFO:teuthology.orchestra.run.smithi107.stdout: "req_getvxattr_latency": { 2024-03-08T21:15:44.742 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-03-08T21:15:44.742 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-03-08T21:15:44.742 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-03-08T21:15:44.742 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:44.742 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:44.743 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_sessions": { 2024-03-08T21:15:44.743 INFO:teuthology.orchestra.run.smithi107.stdout: "session_count": 1, 2024-03-08T21:15:44.743 INFO:teuthology.orchestra.run.smithi107.stdout: "session_add": 1, 2024-03-08T21:15:44.743 INFO:teuthology.orchestra.run.smithi107.stdout: "session_remove": 0, 2024-03-08T21:15:44.743 INFO:teuthology.orchestra.run.smithi107.stdout: "sessions_open": 1, 2024-03-08T21:15:44.743 INFO:teuthology.orchestra.run.smithi107.stdout: "sessions_stale": 0, 2024-03-08T21:15:44.743 INFO:teuthology.orchestra.run.smithi107.stdout: "total_load": 0, 2024-03-08T21:15:44.743 INFO:teuthology.orchestra.run.smithi107.stdout: "average_load": 0, 2024-03-08T21:15:44.743 INFO:teuthology.orchestra.run.smithi107.stdout: "avg_session_uptime": 2387, 2024-03-08T21:15:44.743 INFO:teuthology.orchestra.run.smithi107.stdout: "mdthresh_evicted": 0 2024-03-08T21:15:44.743 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:44.743 INFO:teuthology.orchestra.run.smithi107.stdout: "mempool": { 2024-03-08T21:15:44.743 INFO:teuthology.orchestra.run.smithi107.stdout: "bloom_filter_bytes": 0, 2024-03-08T21:15:44.743 INFO:teuthology.orchestra.run.smithi107.stdout: "bloom_filter_items": 0, 2024-03-08T21:15:44.744 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_alloc_bytes": 0, 2024-03-08T21:15:44.744 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_alloc_items": 0, 2024-03-08T21:15:44.744 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_data_bytes": 0, 2024-03-08T21:15:44.744 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_data_items": 0, 2024-03-08T21:15:44.744 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-08T21:15:44.744 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_onode_items": 0, 2024-03-08T21:15:44.744 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-08T21:15:44.744 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_meta_items": 0, 2024-03-08T21:15:44.744 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_other_bytes": 0, 2024-03-08T21:15:44.744 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_other_items": 0, 2024-03-08T21:15:44.744 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-08T21:15:44.744 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_buffer_items": 0, 2024-03-08T21:15:44.744 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_extent_bytes": 0, 2024-03-08T21:15:44.744 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_extent_items": 0, 2024-03-08T21:15:44.745 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_blob_bytes": 0, 2024-03-08T21:15:44.745 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_blob_items": 0, 2024-03-08T21:15:44.745 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-08T21:15:44.745 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_shared_blob_items": 0, 2024-03-08T21:15:44.745 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-08T21:15:44.745 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_inline_bl_items": 0, 2024-03-08T21:15:44.745 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_fsck_bytes": 0, 2024-03-08T21:15:44.745 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_fsck_items": 0, 2024-03-08T21:15:44.745 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_txc_bytes": 0, 2024-03-08T21:15:44.745 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_txc_items": 0, 2024-03-08T21:15:44.745 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-08T21:15:44.745 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_writing_deferred_items": 0, 2024-03-08T21:15:44.745 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_writing_bytes": 0, 2024-03-08T21:15:44.745 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_writing_items": 0, 2024-03-08T21:15:44.745 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_bytes": 0, 2024-03-08T21:15:44.746 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_items": 0, 2024-03-08T21:15:44.746 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_file_reader_bytes": 0, 2024-03-08T21:15:44.746 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_file_reader_items": 0, 2024-03-08T21:15:44.746 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_file_writer_bytes": 0, 2024-03-08T21:15:44.746 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_file_writer_items": 0, 2024-03-08T21:15:44.746 INFO:teuthology.orchestra.run.smithi107.stdout: "buffer_anon_bytes": 189473, 2024-03-08T21:15:44.746 INFO:teuthology.orchestra.run.smithi107.stdout: "buffer_anon_items": 53, 2024-03-08T21:15:44.746 INFO:teuthology.orchestra.run.smithi107.stdout: "buffer_meta_bytes": 0, 2024-03-08T21:15:44.746 INFO:teuthology.orchestra.run.smithi107.stdout: "buffer_meta_items": 0, 2024-03-08T21:15:44.746 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_bytes": 0, 2024-03-08T21:15:44.746 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_items": 0, 2024-03-08T21:15:44.746 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_mapbl_bytes": 0, 2024-03-08T21:15:44.746 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_mapbl_items": 0, 2024-03-08T21:15:44.747 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_pglog_bytes": 0, 2024-03-08T21:15:44.747 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_pglog_items": 0, 2024-03-08T21:15:44.747 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_bytes": 8352, 2024-03-08T21:15:44.747 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_items": 190, 2024-03-08T21:15:44.747 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_mapping_bytes": 0, 2024-03-08T21:15:44.747 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_mapping_items": 0, 2024-03-08T21:15:44.747 INFO:teuthology.orchestra.run.smithi107.stdout: "pgmap_bytes": 0, 2024-03-08T21:15:44.747 INFO:teuthology.orchestra.run.smithi107.stdout: "pgmap_items": 0, 2024-03-08T21:15:44.747 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_co_bytes": 8320, 2024-03-08T21:15:44.747 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_co_items": 13, 2024-03-08T21:15:44.747 INFO:teuthology.orchestra.run.smithi107.stdout: "unittest_1_bytes": 0, 2024-03-08T21:15:44.747 INFO:teuthology.orchestra.run.smithi107.stdout: "unittest_1_items": 0, 2024-03-08T21:15:44.747 INFO:teuthology.orchestra.run.smithi107.stdout: "unittest_2_bytes": 0, 2024-03-08T21:15:44.747 INFO:teuthology.orchestra.run.smithi107.stdout: "unittest_2_items": 0 2024-03-08T21:15:44.748 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:44.748 INFO:teuthology.orchestra.run.smithi107.stdout: "objecter": { 2024-03-08T21:15:44.748 INFO:teuthology.orchestra.run.smithi107.stdout: "op_active": 0, 2024-03-08T21:15:44.748 INFO:teuthology.orchestra.run.smithi107.stdout: "op_laggy": 0, 2024-03-08T21:15:44.748 INFO:teuthology.orchestra.run.smithi107.stdout: "op_send": 3059, 2024-03-08T21:15:44.748 INFO:teuthology.orchestra.run.smithi107.stdout: "op_send_bytes": 21, 2024-03-08T21:15:44.748 INFO:teuthology.orchestra.run.smithi107.stdout: "op_resend": 3, 2024-03-08T21:15:44.748 INFO:teuthology.orchestra.run.smithi107.stdout: "op_reply": 3059, 2024-03-08T21:15:44.748 INFO:teuthology.orchestra.run.smithi107.stdout: "op_latency": { 2024-03-08T21:15:44.748 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 3059, 2024-03-08T21:15:44.748 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 4.367928917, 2024-03-08T21:15:44.748 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.001427894 2024-03-08T21:15:44.748 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:44.748 INFO:teuthology.orchestra.run.smithi107.stdout: "op_inflight": 0, 2024-03-08T21:15:44.749 INFO:teuthology.orchestra.run.smithi107.stdout: "oplen_avg": { 2024-03-08T21:15:44.749 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 3059, 2024-03-08T21:15:44.749 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 3060 2024-03-08T21:15:44.749 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:44.749 INFO:teuthology.orchestra.run.smithi107.stdout: "op": 3059, 2024-03-08T21:15:44.749 INFO:teuthology.orchestra.run.smithi107.stdout: "op_r": 3059, 2024-03-08T21:15:44.749 INFO:teuthology.orchestra.run.smithi107.stdout: "op_w": 0, 2024-03-08T21:15:44.749 INFO:teuthology.orchestra.run.smithi107.stdout: "op_rmw": 0, 2024-03-08T21:15:44.749 INFO:teuthology.orchestra.run.smithi107.stdout: "op_pg": 0, 2024-03-08T21:15:44.749 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_stat": 1988, 2024-03-08T21:15:44.749 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_create": 0, 2024-03-08T21:15:44.749 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_read": 1069, 2024-03-08T21:15:44.749 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_write": 0, 2024-03-08T21:15:44.750 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_writefull": 0, 2024-03-08T21:15:44.750 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_writesame": 0, 2024-03-08T21:15:44.750 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_append": 0, 2024-03-08T21:15:44.750 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_zero": 0, 2024-03-08T21:15:44.750 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_truncate": 0, 2024-03-08T21:15:44.750 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_delete": 0, 2024-03-08T21:15:44.750 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_mapext": 0, 2024-03-08T21:15:44.750 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_sparse_read": 0, 2024-03-08T21:15:44.750 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_clonerange": 0, 2024-03-08T21:15:44.750 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_getxattr": 1, 2024-03-08T21:15:44.750 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_setxattr": 0, 2024-03-08T21:15:44.750 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_cmpxattr": 0, 2024-03-08T21:15:44.750 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_rmxattr": 0, 2024-03-08T21:15:44.751 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_resetxattrs": 0, 2024-03-08T21:15:44.751 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_call": 0, 2024-03-08T21:15:44.751 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_watch": 0, 2024-03-08T21:15:44.751 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_notify": 0, 2024-03-08T21:15:44.751 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_src_cmpxattr": 0, 2024-03-08T21:15:44.751 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_pgls": 0, 2024-03-08T21:15:44.751 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_pgls_filter": 0, 2024-03-08T21:15:44.751 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_other": 0, 2024-03-08T21:15:44.751 INFO:teuthology.orchestra.run.smithi107.stdout: "linger_active": 0, 2024-03-08T21:15:44.751 INFO:teuthology.orchestra.run.smithi107.stdout: "linger_send": 0, 2024-03-08T21:15:44.751 INFO:teuthology.orchestra.run.smithi107.stdout: "linger_resend": 0, 2024-03-08T21:15:44.751 INFO:teuthology.orchestra.run.smithi107.stdout: "linger_ping": 0, 2024-03-08T21:15:44.751 INFO:teuthology.orchestra.run.smithi107.stdout: "poolop_active": 0, 2024-03-08T21:15:44.752 INFO:teuthology.orchestra.run.smithi107.stdout: "poolop_send": 0, 2024-03-08T21:15:44.752 INFO:teuthology.orchestra.run.smithi107.stdout: "poolop_resend": 0, 2024-03-08T21:15:44.752 INFO:teuthology.orchestra.run.smithi107.stdout: "poolstat_active": 0, 2024-03-08T21:15:44.752 INFO:teuthology.orchestra.run.smithi107.stdout: "poolstat_send": 0, 2024-03-08T21:15:44.752 INFO:teuthology.orchestra.run.smithi107.stdout: "poolstat_resend": 0, 2024-03-08T21:15:44.752 INFO:teuthology.orchestra.run.smithi107.stdout: "statfs_active": 0, 2024-03-08T21:15:44.752 INFO:teuthology.orchestra.run.smithi107.stdout: "statfs_send": 0, 2024-03-08T21:15:44.752 INFO:teuthology.orchestra.run.smithi107.stdout: "statfs_resend": 0, 2024-03-08T21:15:44.752 INFO:teuthology.orchestra.run.smithi107.stdout: "command_active": 0, 2024-03-08T21:15:44.752 INFO:teuthology.orchestra.run.smithi107.stdout: "command_send": 0, 2024-03-08T21:15:44.752 INFO:teuthology.orchestra.run.smithi107.stdout: "command_resend": 0, 2024-03-08T21:15:44.752 INFO:teuthology.orchestra.run.smithi107.stdout: "map_epoch": 113, 2024-03-08T21:15:44.752 INFO:teuthology.orchestra.run.smithi107.stdout: "map_full": 0, 2024-03-08T21:15:44.752 INFO:teuthology.orchestra.run.smithi107.stdout: "map_inc": 27, 2024-03-08T21:15:44.752 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_sessions": 5, 2024-03-08T21:15:44.752 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_session_open": 6, 2024-03-08T21:15:44.753 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_session_close": 1, 2024-03-08T21:15:44.753 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_laggy": 0, 2024-03-08T21:15:44.753 INFO:teuthology.orchestra.run.smithi107.stdout: "omap_wr": 0, 2024-03-08T21:15:44.753 INFO:teuthology.orchestra.run.smithi107.stdout: "omap_rd": 2, 2024-03-08T21:15:44.753 INFO:teuthology.orchestra.run.smithi107.stdout: "omap_del": 0 2024-03-08T21:15:44.753 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:44.753 INFO:teuthology.orchestra.run.smithi107.stdout: "oft": { 2024-03-08T21:15:44.753 INFO:teuthology.orchestra.run.smithi107.stdout: "omap_total_objs": 0, 2024-03-08T21:15:44.753 INFO:teuthology.orchestra.run.smithi107.stdout: "omap_total_kv_pairs": 0, 2024-03-08T21:15:44.753 INFO:teuthology.orchestra.run.smithi107.stdout: "omap_total_updates": 0, 2024-03-08T21:15:44.753 INFO:teuthology.orchestra.run.smithi107.stdout: "omap_total_removes": 0 2024-03-08T21:15:44.753 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:44.753 INFO:teuthology.orchestra.run.smithi107.stdout: "osd-slow-ops": { 2024-03-08T21:15:44.753 INFO:teuthology.orchestra.run.smithi107.stdout: "slow_ops_count": 0 2024-03-08T21:15:44.753 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:44.754 INFO:teuthology.orchestra.run.smithi107.stdout: "purge_queue": { 2024-03-08T21:15:44.754 INFO:teuthology.orchestra.run.smithi107.stdout: "pq_executing_ops": 0, 2024-03-08T21:15:44.754 INFO:teuthology.orchestra.run.smithi107.stdout: "pq_executing_ops_high_water": 0, 2024-03-08T21:15:44.754 INFO:teuthology.orchestra.run.smithi107.stdout: "pq_executing": 0, 2024-03-08T21:15:44.754 INFO:teuthology.orchestra.run.smithi107.stdout: "pq_executing_high_water": 0, 2024-03-08T21:15:44.754 INFO:teuthology.orchestra.run.smithi107.stdout: "pq_executed_ops": 0, 2024-03-08T21:15:44.754 INFO:teuthology.orchestra.run.smithi107.stdout: "pq_executed": 0, 2024-03-08T21:15:44.754 INFO:teuthology.orchestra.run.smithi107.stdout: "pq_item_in_journal": 0 2024-03-08T21:15:44.754 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:44.754 INFO:teuthology.orchestra.run.smithi107.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-08T21:15:44.754 INFO:teuthology.orchestra.run.smithi107.stdout: "val": 0, 2024-03-08T21:15:44.754 INFO:teuthology.orchestra.run.smithi107.stdout: "max": 104857600, 2024-03-08T21:15:44.754 INFO:teuthology.orchestra.run.smithi107.stdout: "get_started": 0, 2024-03-08T21:15:44.754 INFO:teuthology.orchestra.run.smithi107.stdout: "get": 3409, 2024-03-08T21:15:44.754 INFO:teuthology.orchestra.run.smithi107.stdout: "get_sum": 2120935, 2024-03-08T21:15:44.755 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_fail": 0, 2024-03-08T21:15:44.755 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_success": 3409, 2024-03-08T21:15:44.755 INFO:teuthology.orchestra.run.smithi107.stdout: "take": 0, 2024-03-08T21:15:44.755 INFO:teuthology.orchestra.run.smithi107.stdout: "take_sum": 0, 2024-03-08T21:15:44.755 INFO:teuthology.orchestra.run.smithi107.stdout: "put": 3409, 2024-03-08T21:15:44.755 INFO:teuthology.orchestra.run.smithi107.stdout: "put_sum": 2120935, 2024-03-08T21:15:44.755 INFO:teuthology.orchestra.run.smithi107.stdout: "wait": { 2024-03-08T21:15:44.755 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-03-08T21:15:44.755 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-03-08T21:15:44.755 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-03-08T21:15:44.755 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:44.755 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:44.755 INFO:teuthology.orchestra.run.smithi107.stdout: "throttle-objecter_bytes": { 2024-03-08T21:15:44.755 INFO:teuthology.orchestra.run.smithi107.stdout: "val": 0, 2024-03-08T21:15:44.755 INFO:teuthology.orchestra.run.smithi107.stdout: "max": 104857600, 2024-03-08T21:15:44.756 INFO:teuthology.orchestra.run.smithi107.stdout: "get_started": 0, 2024-03-08T21:15:44.756 INFO:teuthology.orchestra.run.smithi107.stdout: "get": 0, 2024-03-08T21:15:44.756 INFO:teuthology.orchestra.run.smithi107.stdout: "get_sum": 0, 2024-03-08T21:15:44.756 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_fail": 0, 2024-03-08T21:15:44.756 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_success": 0, 2024-03-08T21:15:44.756 INFO:teuthology.orchestra.run.smithi107.stdout: "take": 3059, 2024-03-08T21:15:44.756 INFO:teuthology.orchestra.run.smithi107.stdout: "take_sum": 1086, 2024-03-08T21:15:44.756 INFO:teuthology.orchestra.run.smithi107.stdout: "put": 3059, 2024-03-08T21:15:44.756 INFO:teuthology.orchestra.run.smithi107.stdout: "put_sum": 1086, 2024-03-08T21:15:44.756 INFO:teuthology.orchestra.run.smithi107.stdout: "wait": { 2024-03-08T21:15:44.756 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-03-08T21:15:44.756 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-03-08T21:15:44.756 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-03-08T21:15:44.756 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:44.757 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:44.757 INFO:teuthology.orchestra.run.smithi107.stdout: "throttle-objecter_ops": { 2024-03-08T21:15:44.757 INFO:teuthology.orchestra.run.smithi107.stdout: "val": 0, 2024-03-08T21:15:44.757 INFO:teuthology.orchestra.run.smithi107.stdout: "max": 1024, 2024-03-08T21:15:44.757 INFO:teuthology.orchestra.run.smithi107.stdout: "get_started": 0, 2024-03-08T21:15:44.757 INFO:teuthology.orchestra.run.smithi107.stdout: "get": 0, 2024-03-08T21:15:44.757 INFO:teuthology.orchestra.run.smithi107.stdout: "get_sum": 0, 2024-03-08T21:15:44.757 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_fail": 0, 2024-03-08T21:15:44.757 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_success": 0, 2024-03-08T21:15:44.757 INFO:teuthology.orchestra.run.smithi107.stdout: "take": 3059, 2024-03-08T21:15:44.757 INFO:teuthology.orchestra.run.smithi107.stdout: "take_sum": 3059, 2024-03-08T21:15:44.757 INFO:teuthology.orchestra.run.smithi107.stdout: "put": 3059, 2024-03-08T21:15:44.757 INFO:teuthology.orchestra.run.smithi107.stdout: "put_sum": 3059, 2024-03-08T21:15:44.758 INFO:teuthology.orchestra.run.smithi107.stdout: "wait": { 2024-03-08T21:15:44.758 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-03-08T21:15:44.758 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-03-08T21:15:44.758 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-03-08T21:15:44.758 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:44.758 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:44.758 INFO:teuthology.orchestra.run.smithi107.stdout: "throttle-write_buf_throttle": { 2024-03-08T21:15:44.758 INFO:teuthology.orchestra.run.smithi107.stdout: "val": 0, 2024-03-08T21:15:44.758 INFO:teuthology.orchestra.run.smithi107.stdout: "max": 3758096384, 2024-03-08T21:15:44.758 INFO:teuthology.orchestra.run.smithi107.stdout: "get_started": 0, 2024-03-08T21:15:44.758 INFO:teuthology.orchestra.run.smithi107.stdout: "get": 0, 2024-03-08T21:15:44.758 INFO:teuthology.orchestra.run.smithi107.stdout: "get_sum": 0, 2024-03-08T21:15:44.758 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_fail": 0, 2024-03-08T21:15:44.759 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_success": 0, 2024-03-08T21:15:44.759 INFO:teuthology.orchestra.run.smithi107.stdout: "take": 0, 2024-03-08T21:15:44.759 INFO:teuthology.orchestra.run.smithi107.stdout: "take_sum": 0, 2024-03-08T21:15:44.759 INFO:teuthology.orchestra.run.smithi107.stdout: "put": 0, 2024-03-08T21:15:44.759 INFO:teuthology.orchestra.run.smithi107.stdout: "put_sum": 0, 2024-03-08T21:15:44.759 INFO:teuthology.orchestra.run.smithi107.stdout: "wait": { 2024-03-08T21:15:44.759 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-03-08T21:15:44.759 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-03-08T21:15:44.759 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-03-08T21:15:44.759 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:44.759 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:44.759 INFO:teuthology.orchestra.run.smithi107.stdout: "throttle-write_buf_throttle-0x560f2feb2a00": { 2024-03-08T21:15:44.759 INFO:teuthology.orchestra.run.smithi107.stdout: "val": 0, 2024-03-08T21:15:44.759 INFO:teuthology.orchestra.run.smithi107.stdout: "max": 3758096384, 2024-03-08T21:15:44.760 INFO:teuthology.orchestra.run.smithi107.stdout: "get_started": 0, 2024-03-08T21:15:44.760 INFO:teuthology.orchestra.run.smithi107.stdout: "get": 0, 2024-03-08T21:15:44.760 INFO:teuthology.orchestra.run.smithi107.stdout: "get_sum": 0, 2024-03-08T21:15:44.760 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_fail": 0, 2024-03-08T21:15:44.760 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_success": 0, 2024-03-08T21:15:44.760 INFO:teuthology.orchestra.run.smithi107.stdout: "take": 0, 2024-03-08T21:15:44.760 INFO:teuthology.orchestra.run.smithi107.stdout: "take_sum": 0, 2024-03-08T21:15:44.760 INFO:teuthology.orchestra.run.smithi107.stdout: "put": 0, 2024-03-08T21:15:44.760 INFO:teuthology.orchestra.run.smithi107.stdout: "put_sum": 0, 2024-03-08T21:15:44.760 INFO:teuthology.orchestra.run.smithi107.stdout: "wait": { 2024-03-08T21:15:44.760 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-03-08T21:15:44.760 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-03-08T21:15:44.760 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-03-08T21:15:44.761 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:44.761 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:44.761 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-03-08T21:15:44.871 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:15:44 smithi204 ceph-mon[34935]: pgmap v482: 97 pgs: 97 active+clean; 17 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 623 KiB/s rd, 587 KiB/s wr, 131 op/s 2024-03-08T21:15:45.008 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:15:44 smithi149 ceph-mon[35042]: pgmap v482: 97 pgs: 97 active+clean; 17 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 623 KiB/s rd, 587 KiB/s wr, 131 op/s 2024-03-08T21:15:45.679 INFO:tasks.check_counter:Daemon mds.a mds.exported=0 2024-03-08T21:15:45.679 INFO:tasks.check_counter:Daemon mds.a mds.imported=0 2024-03-08T21:15:45.679 INFO:tasks.check_counter:Daemon mds.a mds_cache.dir_update=0 2024-03-08T21:15:45.679 INFO:tasks.check_counter:Daemon mds.a mds_cache.dir_update_receipt=0 2024-03-08T21:15:45.679 INFO:tasks.check_counter:Daemon mds.a mds.root_rsnaps=0 2024-03-08T21:15:45.679 INFO:tasks.check_counter:Daemon mds.a mds_server.req_mksnap_latency.avgcount=0 2024-03-08T21:15:45.679 INFO:tasks.check_counter:Daemon mds.a mds_server.req_rmsnap_latency.avgcount=0 2024-03-08T21:15:45.679 DEBUG:tasks.check_counter:Getting stats from d 2024-03-08T21:15:45.680 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761 shell --fsid dc20f0f8-dd8c-11ee-95c6-87774f69a715 -- ceph daemon mds.d perf dump 2024-03-08T21:15:46.111 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/mon.a/config 2024-03-08T21:15:46.871 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:15:46 smithi204 ceph-mon[34935]: pgmap v483: 97 pgs: 97 active+clean; 17 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 630 KiB/s rd, 593 KiB/s wr, 137 op/s 2024-03-08T21:15:46.960 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:15:46 smithi107 ceph-mon[31381]: pgmap v483: 97 pgs: 97 active+clean; 17 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 630 KiB/s rd, 593 KiB/s wr, 137 op/s 2024-03-08T21:15:47.008 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:15:46 smithi149 ceph-mon[35042]: pgmap v483: 97 pgs: 97 active+clean; 17 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 630 KiB/s rd, 593 KiB/s wr, 137 op/s 2024-03-08T21:15:48.465 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-03-08T21:15:48.466 INFO:teuthology.orchestra.run.smithi107.stdout: "AsyncMessenger::Worker-0": { 2024-03-08T21:15:48.466 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_messages": 411, 2024-03-08T21:15:48.466 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages": 410, 2024-03-08T21:15:48.466 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_bytes": 16106141, 2024-03-08T21:15:48.466 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_bytes": 140062, 2024-03-08T21:15:48.466 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_created_connections": 6, 2024-03-08T21:15:48.466 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_active_connections": 2, 2024-03-08T21:15:48.466 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_total_time": 0.147466689, 2024-03-08T21:15:48.466 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_send_time": 0.034617692, 2024-03-08T21:15:48.466 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_recv_time": 0.080162780, 2024-03-08T21:15:48.467 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_fast_dispatch_time": 0.023357283, 2024-03-08T21:15:48.467 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages_queue_lat": { 2024-03-08T21:15:48.467 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 409, 2024-03-08T21:15:48.467 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.037313680, 2024-03-08T21:15:48.467 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000091231 2024-03-08T21:15:48.467 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:48.467 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_handle_ack_lat": { 2024-03-08T21:15:48.467 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-03-08T21:15:48.467 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-03-08T21:15:48.467 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-03-08T21:15:48.467 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:48.467 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_encrypted_bytes": 170864, 2024-03-08T21:15:48.468 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_encrypted_bytes": 27168 2024-03-08T21:15:48.468 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:48.468 INFO:teuthology.orchestra.run.smithi107.stdout: "AsyncMessenger::Worker-1": { 2024-03-08T21:15:48.468 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_messages": 39, 2024-03-08T21:15:48.468 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages": 71, 2024-03-08T21:15:48.468 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_bytes": 8400132, 2024-03-08T21:15:48.468 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_bytes": 152518, 2024-03-08T21:15:48.468 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_created_connections": 5, 2024-03-08T21:15:48.468 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_active_connections": 4, 2024-03-08T21:15:48.468 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_total_time": 0.031388369, 2024-03-08T21:15:48.468 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_send_time": 0.008926619, 2024-03-08T21:15:48.469 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_recv_time": 0.018845313, 2024-03-08T21:15:48.469 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_fast_dispatch_time": 0.001970427, 2024-03-08T21:15:48.469 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages_queue_lat": { 2024-03-08T21:15:48.469 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 69, 2024-03-08T21:15:48.469 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.012841301, 2024-03-08T21:15:48.469 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000186105 2024-03-08T21:15:48.469 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:48.469 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_handle_ack_lat": { 2024-03-08T21:15:48.469 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-03-08T21:15:48.469 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-03-08T21:15:48.469 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-03-08T21:15:48.470 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:48.470 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_encrypted_bytes": 160, 2024-03-08T21:15:48.470 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_encrypted_bytes": 140368 2024-03-08T21:15:48.470 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:48.470 INFO:teuthology.orchestra.run.smithi107.stdout: "AsyncMessenger::Worker-2": { 2024-03-08T21:15:48.470 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_messages": 145, 2024-03-08T21:15:48.470 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages": 145, 2024-03-08T21:15:48.470 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_bytes": 8427274, 2024-03-08T21:15:48.470 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_bytes": 45293, 2024-03-08T21:15:48.470 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_created_connections": 5, 2024-03-08T21:15:48.470 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_active_connections": 4, 2024-03-08T21:15:48.471 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_total_time": 0.057964998, 2024-03-08T21:15:48.471 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_send_time": 0.011681459, 2024-03-08T21:15:48.471 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_recv_time": 0.032999532, 2024-03-08T21:15:48.471 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_fast_dispatch_time": 0.010026581, 2024-03-08T21:15:48.471 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages_queue_lat": { 2024-03-08T21:15:48.471 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 144, 2024-03-08T21:15:48.471 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.019716299, 2024-03-08T21:15:48.471 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000136918 2024-03-08T21:15:48.471 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:48.471 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_handle_ack_lat": { 2024-03-08T21:15:48.471 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-03-08T21:15:48.471 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-03-08T21:15:48.471 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-03-08T21:15:48.472 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:48.472 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_encrypted_bytes": 3040, 2024-03-08T21:15:48.472 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_encrypted_bytes": 448 2024-03-08T21:15:48.472 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:48.472 INFO:teuthology.orchestra.run.smithi107.stdout: "cct": { 2024-03-08T21:15:48.472 INFO:teuthology.orchestra.run.smithi107.stdout: "total_workers": 1, 2024-03-08T21:15:48.472 INFO:teuthology.orchestra.run.smithi107.stdout: "unhealthy_workers": 0 2024-03-08T21:15:48.472 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:48.472 INFO:teuthology.orchestra.run.smithi107.stdout: "finisher-MDSRank": { 2024-03-08T21:15:48.472 INFO:teuthology.orchestra.run.smithi107.stdout: "queue_len": 0, 2024-03-08T21:15:48.472 INFO:teuthology.orchestra.run.smithi107.stdout: "complete_latency": { 2024-03-08T21:15:48.472 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 855, 2024-03-08T21:15:48.472 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.061091240, 2024-03-08T21:15:48.473 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000071451 2024-03-08T21:15:48.473 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:48.473 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:48.473 INFO:teuthology.orchestra.run.smithi107.stdout: "finisher-PurgeQueue": { 2024-03-08T21:15:48.473 INFO:teuthology.orchestra.run.smithi107.stdout: "queue_len": 0, 2024-03-08T21:15:48.473 INFO:teuthology.orchestra.run.smithi107.stdout: "complete_latency": { 2024-03-08T21:15:48.473 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-03-08T21:15:48.473 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-03-08T21:15:48.473 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-03-08T21:15:48.473 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:48.473 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:48.474 INFO:teuthology.orchestra.run.smithi107.stdout: "mds": { 2024-03-08T21:15:48.474 INFO:teuthology.orchestra.run.smithi107.stdout: "request": 0, 2024-03-08T21:15:48.474 INFO:teuthology.orchestra.run.smithi107.stdout: "reply": 0, 2024-03-08T21:15:48.474 INFO:teuthology.orchestra.run.smithi107.stdout: "reply_latency": { 2024-03-08T21:15:48.474 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-03-08T21:15:48.474 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-03-08T21:15:48.474 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-03-08T21:15:48.474 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:48.474 INFO:teuthology.orchestra.run.smithi107.stdout: "slow_reply": 0, 2024-03-08T21:15:48.474 INFO:teuthology.orchestra.run.smithi107.stdout: "forward": 0, 2024-03-08T21:15:48.474 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_fetch_complete": 0, 2024-03-08T21:15:48.474 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_fetch_keys": 0, 2024-03-08T21:15:48.475 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_commit": 0, 2024-03-08T21:15:48.475 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_split": 0, 2024-03-08T21:15:48.475 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_merge": 0, 2024-03-08T21:15:48.475 INFO:teuthology.orchestra.run.smithi107.stdout: "inodes": 1225, 2024-03-08T21:15:48.475 INFO:teuthology.orchestra.run.smithi107.stdout: "inodes_top": 666, 2024-03-08T21:15:48.475 INFO:teuthology.orchestra.run.smithi107.stdout: "inodes_bottom": 559, 2024-03-08T21:15:48.475 INFO:teuthology.orchestra.run.smithi107.stdout: "inodes_pin_tail": 0, 2024-03-08T21:15:48.475 INFO:teuthology.orchestra.run.smithi107.stdout: "inodes_pinned": 274, 2024-03-08T21:15:48.475 INFO:teuthology.orchestra.run.smithi107.stdout: "inodes_expired": 0, 2024-03-08T21:15:48.475 INFO:teuthology.orchestra.run.smithi107.stdout: "inodes_with_caps": 0, 2024-03-08T21:15:48.475 INFO:teuthology.orchestra.run.smithi107.stdout: "caps": 0, 2024-03-08T21:15:48.475 INFO:teuthology.orchestra.run.smithi107.stdout: "subtrees": 2, 2024-03-08T21:15:48.476 INFO:teuthology.orchestra.run.smithi107.stdout: "traverse": 0, 2024-03-08T21:15:48.476 INFO:teuthology.orchestra.run.smithi107.stdout: "traverse_hit": 0, 2024-03-08T21:15:48.476 INFO:teuthology.orchestra.run.smithi107.stdout: "traverse_forward": 0, 2024-03-08T21:15:48.476 INFO:teuthology.orchestra.run.smithi107.stdout: "traverse_discover": 0, 2024-03-08T21:15:48.476 INFO:teuthology.orchestra.run.smithi107.stdout: "traverse_dir_fetch": 0, 2024-03-08T21:15:48.476 INFO:teuthology.orchestra.run.smithi107.stdout: "traverse_remote_ino": 0, 2024-03-08T21:15:48.476 INFO:teuthology.orchestra.run.smithi107.stdout: "traverse_lock": 0, 2024-03-08T21:15:48.476 INFO:teuthology.orchestra.run.smithi107.stdout: "load_cent": 0, 2024-03-08T21:15:48.476 INFO:teuthology.orchestra.run.smithi107.stdout: "q": 0, 2024-03-08T21:15:48.476 INFO:teuthology.orchestra.run.smithi107.stdout: "exported": 0, 2024-03-08T21:15:48.476 INFO:teuthology.orchestra.run.smithi107.stdout: "exported_inodes": 0, 2024-03-08T21:15:48.476 INFO:teuthology.orchestra.run.smithi107.stdout: "imported": 0, 2024-03-08T21:15:48.477 INFO:teuthology.orchestra.run.smithi107.stdout: "imported_inodes": 0, 2024-03-08T21:15:48.477 INFO:teuthology.orchestra.run.smithi107.stdout: "openino_dir_fetch": 0, 2024-03-08T21:15:48.477 INFO:teuthology.orchestra.run.smithi107.stdout: "openino_backtrace_fetch": 0, 2024-03-08T21:15:48.477 INFO:teuthology.orchestra.run.smithi107.stdout: "openino_peer_discover": 0, 2024-03-08T21:15:48.477 INFO:teuthology.orchestra.run.smithi107.stdout: "root_rfiles": 4, 2024-03-08T21:15:48.477 INFO:teuthology.orchestra.run.smithi107.stdout: "root_rbytes": 234, 2024-03-08T21:15:48.477 INFO:teuthology.orchestra.run.smithi107.stdout: "root_rsnaps": 6, 2024-03-08T21:15:48.477 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_backtrace_fetch": 0, 2024-03-08T21:15:48.477 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_set_tag": 0, 2024-03-08T21:15:48.477 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_backtrace_repaired": 0, 2024-03-08T21:15:48.477 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_inotable_repaired": 0, 2024-03-08T21:15:48.478 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_dir_inodes": 0, 2024-03-08T21:15:48.478 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_dir_base_inodes": 0, 2024-03-08T21:15:48.478 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_dirfrag_rstats": 0, 2024-03-08T21:15:48.478 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_file_inodes": 0, 2024-03-08T21:15:48.478 INFO:teuthology.orchestra.run.smithi107.stdout: "handle_inode_file_caps": 0, 2024-03-08T21:15:48.478 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph_cap_op_revoke": 0, 2024-03-08T21:15:48.478 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph_cap_op_grant": 0, 2024-03-08T21:15:48.478 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph_cap_op_trunc": 0, 2024-03-08T21:15:48.478 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-08T21:15:48.478 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-08T21:15:48.478 INFO:teuthology.orchestra.run.smithi107.stdout: "handle_client_caps": 0, 2024-03-08T21:15:48.479 INFO:teuthology.orchestra.run.smithi107.stdout: "handle_client_caps_dirty": 0, 2024-03-08T21:15:48.479 INFO:teuthology.orchestra.run.smithi107.stdout: "handle_client_cap_release": 0, 2024-03-08T21:15:48.479 INFO:teuthology.orchestra.run.smithi107.stdout: "process_request_cap_release": 0 2024-03-08T21:15:48.479 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:48.479 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_cache": { 2024-03-08T21:15:48.479 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_update": 0, 2024-03-08T21:15:48.479 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_update_receipt": 0, 2024-03-08T21:15:48.479 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_try_discover": 0, 2024-03-08T21:15:48.479 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_send_discover": 0, 2024-03-08T21:15:48.479 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_handle_discover": 0, 2024-03-08T21:15:48.479 INFO:teuthology.orchestra.run.smithi107.stdout: "num_strays": 0, 2024-03-08T21:15:48.480 INFO:teuthology.orchestra.run.smithi107.stdout: "num_strays_delayed": 0, 2024-03-08T21:15:48.480 INFO:teuthology.orchestra.run.smithi107.stdout: "num_strays_enqueuing": 0, 2024-03-08T21:15:48.480 INFO:teuthology.orchestra.run.smithi107.stdout: "strays_created": 0, 2024-03-08T21:15:48.480 INFO:teuthology.orchestra.run.smithi107.stdout: "strays_enqueued": 0, 2024-03-08T21:15:48.480 INFO:teuthology.orchestra.run.smithi107.stdout: "strays_reintegrated": 0, 2024-03-08T21:15:48.480 INFO:teuthology.orchestra.run.smithi107.stdout: "strays_migrated": 0, 2024-03-08T21:15:48.480 INFO:teuthology.orchestra.run.smithi107.stdout: "num_recovering_processing": 0, 2024-03-08T21:15:48.480 INFO:teuthology.orchestra.run.smithi107.stdout: "num_recovering_enqueued": 0, 2024-03-08T21:15:48.480 INFO:teuthology.orchestra.run.smithi107.stdout: "num_recovering_prioritized": 0, 2024-03-08T21:15:48.480 INFO:teuthology.orchestra.run.smithi107.stdout: "recovery_started": 0, 2024-03-08T21:15:48.480 INFO:teuthology.orchestra.run.smithi107.stdout: "recovery_completed": 0, 2024-03-08T21:15:48.481 INFO:teuthology.orchestra.run.smithi107.stdout: "ireq_enqueue_scrub": 0, 2024-03-08T21:15:48.481 INFO:teuthology.orchestra.run.smithi107.stdout: "ireq_exportdir": 0, 2024-03-08T21:15:48.481 INFO:teuthology.orchestra.run.smithi107.stdout: "ireq_flush": 0, 2024-03-08T21:15:48.481 INFO:teuthology.orchestra.run.smithi107.stdout: "ireq_fragmentdir": 0, 2024-03-08T21:15:48.481 INFO:teuthology.orchestra.run.smithi107.stdout: "ireq_fragstats": 0, 2024-03-08T21:15:48.481 INFO:teuthology.orchestra.run.smithi107.stdout: "ireq_inodestats": 0 2024-03-08T21:15:48.481 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:48.481 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_log": { 2024-03-08T21:15:48.481 INFO:teuthology.orchestra.run.smithi107.stdout: "evlrg": 0, 2024-03-08T21:15:48.481 INFO:teuthology.orchestra.run.smithi107.stdout: "evadd": 0, 2024-03-08T21:15:48.481 INFO:teuthology.orchestra.run.smithi107.stdout: "evex": 0, 2024-03-08T21:15:48.481 INFO:teuthology.orchestra.run.smithi107.stdout: "evtrm": 0, 2024-03-08T21:15:48.481 INFO:teuthology.orchestra.run.smithi107.stdout: "ev": 6411, 2024-03-08T21:15:48.482 INFO:teuthology.orchestra.run.smithi107.stdout: "evexg": 0, 2024-03-08T21:15:48.482 INFO:teuthology.orchestra.run.smithi107.stdout: "evexd": 0, 2024-03-08T21:15:48.482 INFO:teuthology.orchestra.run.smithi107.stdout: "segadd": 0, 2024-03-08T21:15:48.482 INFO:teuthology.orchestra.run.smithi107.stdout: "segex": 0, 2024-03-08T21:15:48.482 INFO:teuthology.orchestra.run.smithi107.stdout: "segtrm": 0, 2024-03-08T21:15:48.482 INFO:teuthology.orchestra.run.smithi107.stdout: "seg": 2, 2024-03-08T21:15:48.482 INFO:teuthology.orchestra.run.smithi107.stdout: "segmjr": 11, 2024-03-08T21:15:48.482 INFO:teuthology.orchestra.run.smithi107.stdout: "segexg": 0, 2024-03-08T21:15:48.482 INFO:teuthology.orchestra.run.smithi107.stdout: "segexd": 0, 2024-03-08T21:15:48.482 INFO:teuthology.orchestra.run.smithi107.stdout: "expos": 32972229, 2024-03-08T21:15:48.482 INFO:teuthology.orchestra.run.smithi107.stdout: "wrpos": 37633829, 2024-03-08T21:15:48.482 INFO:teuthology.orchestra.run.smithi107.stdout: "rdpos": 37632178, 2024-03-08T21:15:48.483 INFO:teuthology.orchestra.run.smithi107.stdout: "jlat": { 2024-03-08T21:15:48.483 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-03-08T21:15:48.483 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-03-08T21:15:48.483 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-03-08T21:15:48.483 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:48.483 INFO:teuthology.orchestra.run.smithi107.stdout: "replayed": 6411 2024-03-08T21:15:48.483 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:48.483 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_mem": { 2024-03-08T21:15:48.483 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 22, 2024-03-08T21:15:48.483 INFO:teuthology.orchestra.run.smithi107.stdout: "ino+": 1223, 2024-03-08T21:15:48.483 INFO:teuthology.orchestra.run.smithi107.stdout: "ino-": 1201, 2024-03-08T21:15:48.484 INFO:teuthology.orchestra.run.smithi107.stdout: "dir": 18, 2024-03-08T21:15:48.484 INFO:teuthology.orchestra.run.smithi107.stdout: "dir+": 18, 2024-03-08T21:15:48.484 INFO:teuthology.orchestra.run.smithi107.stdout: "dir-": 0, 2024-03-08T21:15:48.484 INFO:teuthology.orchestra.run.smithi107.stdout: "dn": 1225, 2024-03-08T21:15:48.484 INFO:teuthology.orchestra.run.smithi107.stdout: "dn+": 1225, 2024-03-08T21:15:48.484 INFO:teuthology.orchestra.run.smithi107.stdout: "dn-": 0, 2024-03-08T21:15:48.484 INFO:teuthology.orchestra.run.smithi107.stdout: "cap": 0, 2024-03-08T21:15:48.484 INFO:teuthology.orchestra.run.smithi107.stdout: "cap+": 0, 2024-03-08T21:15:48.484 INFO:teuthology.orchestra.run.smithi107.stdout: "cap-": 0, 2024-03-08T21:15:48.484 INFO:teuthology.orchestra.run.smithi107.stdout: "rss": 50984, 2024-03-08T21:15:48.484 INFO:teuthology.orchestra.run.smithi107.stdout: "heap": 215420 2024-03-08T21:15:48.484 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:48.484 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_server": { 2024-03-08T21:15:48.485 INFO:teuthology.orchestra.run.smithi107.stdout: "dispatch_client_request": 0, 2024-03-08T21:15:48.485 INFO:teuthology.orchestra.run.smithi107.stdout: "dispatch_server_request": 0, 2024-03-08T21:15:48.485 INFO:teuthology.orchestra.run.smithi107.stdout: "handle_client_request": 0, 2024-03-08T21:15:48.485 INFO:teuthology.orchestra.run.smithi107.stdout: "handle_client_session": 0, 2024-03-08T21:15:48.485 INFO:teuthology.orchestra.run.smithi107.stdout: "handle_peer_request": 0, 2024-03-08T21:15:48.485 INFO:teuthology.orchestra.run.smithi107.stdout: "req_create_latency": { 2024-03-08T21:15:48.485 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-03-08T21:15:48.485 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-03-08T21:15:48.485 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-03-08T21:15:48.485 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:48.485 INFO:teuthology.orchestra.run.smithi107.stdout: "req_getattr_latency": { 2024-03-08T21:15:48.485 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-03-08T21:15:48.485 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-03-08T21:15:48.486 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-03-08T21:15:48.486 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:48.486 INFO:teuthology.orchestra.run.smithi107.stdout: "req_getfilelock_latency": { 2024-03-08T21:15:48.486 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-03-08T21:15:48.486 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-03-08T21:15:48.486 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-03-08T21:15:48.486 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:48.486 INFO:teuthology.orchestra.run.smithi107.stdout: "req_link_latency": { 2024-03-08T21:15:48.486 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-03-08T21:15:48.486 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-03-08T21:15:48.486 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-03-08T21:15:48.486 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:48.487 INFO:teuthology.orchestra.run.smithi107.stdout: "req_lookup_latency": { 2024-03-08T21:15:48.487 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-03-08T21:15:48.487 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-03-08T21:15:48.487 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-03-08T21:15:48.487 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:48.487 INFO:teuthology.orchestra.run.smithi107.stdout: "req_lookuphash_latency": { 2024-03-08T21:15:48.487 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-03-08T21:15:48.487 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-03-08T21:15:48.487 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-03-08T21:15:48.487 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:48.487 INFO:teuthology.orchestra.run.smithi107.stdout: "req_lookupino_latency": { 2024-03-08T21:15:48.487 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-03-08T21:15:48.488 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-03-08T21:15:48.488 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-03-08T21:15:48.488 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:48.488 INFO:teuthology.orchestra.run.smithi107.stdout: "req_lookupname_latency": { 2024-03-08T21:15:48.488 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-03-08T21:15:48.488 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-03-08T21:15:48.488 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-03-08T21:15:48.488 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:48.488 INFO:teuthology.orchestra.run.smithi107.stdout: "req_lookupparent_latency": { 2024-03-08T21:15:48.488 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-03-08T21:15:48.488 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-03-08T21:15:48.488 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-03-08T21:15:48.489 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:48.489 INFO:teuthology.orchestra.run.smithi107.stdout: "req_lookupsnap_latency": { 2024-03-08T21:15:48.489 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-03-08T21:15:48.489 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-03-08T21:15:48.489 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-03-08T21:15:48.489 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:48.489 INFO:teuthology.orchestra.run.smithi107.stdout: "req_lssnap_latency": { 2024-03-08T21:15:48.489 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-03-08T21:15:48.489 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-03-08T21:15:48.489 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-03-08T21:15:48.489 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:48.489 INFO:teuthology.orchestra.run.smithi107.stdout: "req_mkdir_latency": { 2024-03-08T21:15:48.490 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-03-08T21:15:48.490 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-03-08T21:15:48.490 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-03-08T21:15:48.491 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:48.491 INFO:teuthology.orchestra.run.smithi107.stdout: "req_mknod_latency": { 2024-03-08T21:15:48.491 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-03-08T21:15:48.491 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-03-08T21:15:48.491 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-03-08T21:15:48.491 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:48.491 INFO:teuthology.orchestra.run.smithi107.stdout: "req_mksnap_latency": { 2024-03-08T21:15:48.491 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-03-08T21:15:48.491 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-03-08T21:15:48.491 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-03-08T21:15:48.491 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:48.491 INFO:teuthology.orchestra.run.smithi107.stdout: "req_open_latency": { 2024-03-08T21:15:48.492 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-03-08T21:15:48.492 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-03-08T21:15:48.492 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-03-08T21:15:48.492 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:48.492 INFO:teuthology.orchestra.run.smithi107.stdout: "req_readdir_latency": { 2024-03-08T21:15:48.492 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-03-08T21:15:48.492 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-03-08T21:15:48.492 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-03-08T21:15:48.492 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:48.492 INFO:teuthology.orchestra.run.smithi107.stdout: "req_rename_latency": { 2024-03-08T21:15:48.493 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-03-08T21:15:48.493 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-03-08T21:15:48.493 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-03-08T21:15:48.493 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:48.493 INFO:teuthology.orchestra.run.smithi107.stdout: "req_renamesnap_latency": { 2024-03-08T21:15:48.493 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-03-08T21:15:48.493 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-03-08T21:15:48.493 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-03-08T21:15:48.493 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:48.493 INFO:teuthology.orchestra.run.smithi107.stdout: "req_snapdiff_latency": { 2024-03-08T21:15:48.493 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-03-08T21:15:48.493 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-03-08T21:15:48.494 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-03-08T21:15:48.494 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:48.494 INFO:teuthology.orchestra.run.smithi107.stdout: "req_rmdir_latency": { 2024-03-08T21:15:48.494 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-03-08T21:15:48.494 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-03-08T21:15:48.494 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-03-08T21:15:48.494 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:48.494 INFO:teuthology.orchestra.run.smithi107.stdout: "req_rmsnap_latency": { 2024-03-08T21:15:48.494 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-03-08T21:15:48.494 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-03-08T21:15:48.494 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-03-08T21:15:48.494 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:48.495 INFO:teuthology.orchestra.run.smithi107.stdout: "req_rmxattr_latency": { 2024-03-08T21:15:48.495 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-03-08T21:15:48.495 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-03-08T21:15:48.495 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-03-08T21:15:48.495 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:48.495 INFO:teuthology.orchestra.run.smithi107.stdout: "req_setattr_latency": { 2024-03-08T21:15:48.495 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-03-08T21:15:48.495 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-03-08T21:15:48.495 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-03-08T21:15:48.495 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:48.495 INFO:teuthology.orchestra.run.smithi107.stdout: "req_setdirlayout_latency": { 2024-03-08T21:15:48.495 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-03-08T21:15:48.496 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-03-08T21:15:48.496 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-03-08T21:15:48.496 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:48.496 INFO:teuthology.orchestra.run.smithi107.stdout: "req_setfilelock_latency": { 2024-03-08T21:15:48.496 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-03-08T21:15:48.496 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-03-08T21:15:48.496 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-03-08T21:15:48.496 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:48.496 INFO:teuthology.orchestra.run.smithi107.stdout: "req_setlayout_latency": { 2024-03-08T21:15:48.496 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-03-08T21:15:48.496 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-03-08T21:15:48.496 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-03-08T21:15:48.496 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:48.497 INFO:teuthology.orchestra.run.smithi107.stdout: "req_setxattr_latency": { 2024-03-08T21:15:48.497 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-03-08T21:15:48.497 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-03-08T21:15:48.497 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-03-08T21:15:48.497 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:48.497 INFO:teuthology.orchestra.run.smithi107.stdout: "req_symlink_latency": { 2024-03-08T21:15:48.497 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-03-08T21:15:48.497 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-03-08T21:15:48.497 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-03-08T21:15:48.497 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:48.497 INFO:teuthology.orchestra.run.smithi107.stdout: "req_unlink_latency": { 2024-03-08T21:15:48.497 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-03-08T21:15:48.498 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-03-08T21:15:48.498 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-03-08T21:15:48.498 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:48.498 INFO:teuthology.orchestra.run.smithi107.stdout: "cap_revoke_eviction": 0, 2024-03-08T21:15:48.498 INFO:teuthology.orchestra.run.smithi107.stdout: "cap_acquisition_throttle": 0, 2024-03-08T21:15:48.498 INFO:teuthology.orchestra.run.smithi107.stdout: "req_getvxattr_latency": { 2024-03-08T21:15:48.498 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-03-08T21:15:48.498 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-03-08T21:15:48.498 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-03-08T21:15:48.498 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:48.498 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:48.499 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_sessions": { 2024-03-08T21:15:48.499 INFO:teuthology.orchestra.run.smithi107.stdout: "session_count": 2, 2024-03-08T21:15:48.499 INFO:teuthology.orchestra.run.smithi107.stdout: "session_add": 2, 2024-03-08T21:15:48.499 INFO:teuthology.orchestra.run.smithi107.stdout: "session_remove": 0, 2024-03-08T21:15:48.499 INFO:teuthology.orchestra.run.smithi107.stdout: "sessions_open": 0, 2024-03-08T21:15:48.499 INFO:teuthology.orchestra.run.smithi107.stdout: "sessions_stale": 0, 2024-03-08T21:15:48.499 INFO:teuthology.orchestra.run.smithi107.stdout: "total_load": 0, 2024-03-08T21:15:48.499 INFO:teuthology.orchestra.run.smithi107.stdout: "average_load": 0, 2024-03-08T21:15:48.499 INFO:teuthology.orchestra.run.smithi107.stdout: "avg_session_uptime": 2394, 2024-03-08T21:15:48.499 INFO:teuthology.orchestra.run.smithi107.stdout: "mdthresh_evicted": 0 2024-03-08T21:15:48.499 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:48.499 INFO:teuthology.orchestra.run.smithi107.stdout: "mempool": { 2024-03-08T21:15:48.500 INFO:teuthology.orchestra.run.smithi107.stdout: "bloom_filter_bytes": 0, 2024-03-08T21:15:48.500 INFO:teuthology.orchestra.run.smithi107.stdout: "bloom_filter_items": 0, 2024-03-08T21:15:48.500 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_alloc_bytes": 0, 2024-03-08T21:15:48.500 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_alloc_items": 0, 2024-03-08T21:15:48.500 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_data_bytes": 0, 2024-03-08T21:15:48.500 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_data_items": 0, 2024-03-08T21:15:48.500 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-08T21:15:48.500 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_onode_items": 0, 2024-03-08T21:15:48.500 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-08T21:15:48.500 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_meta_items": 0, 2024-03-08T21:15:48.500 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_other_bytes": 0, 2024-03-08T21:15:48.500 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_other_items": 0, 2024-03-08T21:15:48.501 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-08T21:15:48.501 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_buffer_items": 0, 2024-03-08T21:15:48.501 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_extent_bytes": 0, 2024-03-08T21:15:48.501 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_extent_items": 0, 2024-03-08T21:15:48.501 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_blob_bytes": 0, 2024-03-08T21:15:48.501 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_blob_items": 0, 2024-03-08T21:15:48.501 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-08T21:15:48.501 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_shared_blob_items": 0, 2024-03-08T21:15:48.501 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-08T21:15:48.501 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_inline_bl_items": 0, 2024-03-08T21:15:48.501 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_fsck_bytes": 0, 2024-03-08T21:15:48.501 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_fsck_items": 0, 2024-03-08T21:15:48.501 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_txc_bytes": 0, 2024-03-08T21:15:48.502 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_txc_items": 0, 2024-03-08T21:15:48.502 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-08T21:15:48.502 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_writing_deferred_items": 0, 2024-03-08T21:15:48.502 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_writing_bytes": 0, 2024-03-08T21:15:48.502 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_writing_items": 0, 2024-03-08T21:15:48.502 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_bytes": 0, 2024-03-08T21:15:48.502 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_items": 0, 2024-03-08T21:15:48.502 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_file_reader_bytes": 0, 2024-03-08T21:15:48.502 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_file_reader_items": 0, 2024-03-08T21:15:48.502 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_file_writer_bytes": 0, 2024-03-08T21:15:48.502 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_file_writer_items": 0, 2024-03-08T21:15:48.502 INFO:teuthology.orchestra.run.smithi107.stdout: "buffer_anon_bytes": 204599, 2024-03-08T21:15:48.503 INFO:teuthology.orchestra.run.smithi107.stdout: "buffer_anon_items": 53, 2024-03-08T21:15:48.503 INFO:teuthology.orchestra.run.smithi107.stdout: "buffer_meta_bytes": 0, 2024-03-08T21:15:48.503 INFO:teuthology.orchestra.run.smithi107.stdout: "buffer_meta_items": 0, 2024-03-08T21:15:48.503 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_bytes": 0, 2024-03-08T21:15:48.503 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_items": 0, 2024-03-08T21:15:48.503 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_mapbl_bytes": 0, 2024-03-08T21:15:48.503 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_mapbl_items": 0, 2024-03-08T21:15:48.503 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_pglog_bytes": 0, 2024-03-08T21:15:48.503 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_pglog_items": 0, 2024-03-08T21:15:48.503 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_bytes": 8352, 2024-03-08T21:15:48.503 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_items": 190, 2024-03-08T21:15:48.503 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_mapping_bytes": 0, 2024-03-08T21:15:48.503 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_mapping_items": 0, 2024-03-08T21:15:48.504 INFO:teuthology.orchestra.run.smithi107.stdout: "pgmap_bytes": 0, 2024-03-08T21:15:48.504 INFO:teuthology.orchestra.run.smithi107.stdout: "pgmap_items": 0, 2024-03-08T21:15:48.504 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_co_bytes": 969631, 2024-03-08T21:15:48.504 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_co_items": 5329, 2024-03-08T21:15:48.504 INFO:teuthology.orchestra.run.smithi107.stdout: "unittest_1_bytes": 0, 2024-03-08T21:15:48.504 INFO:teuthology.orchestra.run.smithi107.stdout: "unittest_1_items": 0, 2024-03-08T21:15:48.504 INFO:teuthology.orchestra.run.smithi107.stdout: "unittest_2_bytes": 0, 2024-03-08T21:15:48.504 INFO:teuthology.orchestra.run.smithi107.stdout: "unittest_2_items": 0 2024-03-08T21:15:48.504 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:48.504 INFO:teuthology.orchestra.run.smithi107.stdout: "objecter": { 2024-03-08T21:15:48.504 INFO:teuthology.orchestra.run.smithi107.stdout: "op_active": 0, 2024-03-08T21:15:48.504 INFO:teuthology.orchestra.run.smithi107.stdout: "op_laggy": 0, 2024-03-08T21:15:48.505 INFO:teuthology.orchestra.run.smithi107.stdout: "op_send": 534, 2024-03-08T21:15:48.505 INFO:teuthology.orchestra.run.smithi107.stdout: "op_send_bytes": 26, 2024-03-08T21:15:48.505 INFO:teuthology.orchestra.run.smithi107.stdout: "op_resend": 4, 2024-03-08T21:15:48.505 INFO:teuthology.orchestra.run.smithi107.stdout: "op_reply": 534, 2024-03-08T21:15:48.505 INFO:teuthology.orchestra.run.smithi107.stdout: "op_latency": { 2024-03-08T21:15:48.505 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 534, 2024-03-08T21:15:48.505 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.964984257, 2024-03-08T21:15:48.505 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.001807086 2024-03-08T21:15:48.505 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:48.505 INFO:teuthology.orchestra.run.smithi107.stdout: "op_inflight": 0, 2024-03-08T21:15:48.505 INFO:teuthology.orchestra.run.smithi107.stdout: "oplen_avg": { 2024-03-08T21:15:48.505 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 534, 2024-03-08T21:15:48.505 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 535 2024-03-08T21:15:48.506 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:48.506 INFO:teuthology.orchestra.run.smithi107.stdout: "op": 534, 2024-03-08T21:15:48.506 INFO:teuthology.orchestra.run.smithi107.stdout: "op_r": 534, 2024-03-08T21:15:48.506 INFO:teuthology.orchestra.run.smithi107.stdout: "op_w": 0, 2024-03-08T21:15:48.506 INFO:teuthology.orchestra.run.smithi107.stdout: "op_rmw": 0, 2024-03-08T21:15:48.506 INFO:teuthology.orchestra.run.smithi107.stdout: "op_pg": 0, 2024-03-08T21:15:48.506 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_stat": 321, 2024-03-08T21:15:48.506 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_create": 0, 2024-03-08T21:15:48.506 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_read": 210, 2024-03-08T21:15:48.506 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_write": 0, 2024-03-08T21:15:48.506 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_writefull": 0, 2024-03-08T21:15:48.506 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_writesame": 0, 2024-03-08T21:15:48.507 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_append": 0, 2024-03-08T21:15:48.507 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_zero": 0, 2024-03-08T21:15:48.507 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_truncate": 0, 2024-03-08T21:15:48.507 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_delete": 0, 2024-03-08T21:15:48.507 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_mapext": 0, 2024-03-08T21:15:48.507 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_sparse_read": 0, 2024-03-08T21:15:48.507 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_clonerange": 0, 2024-03-08T21:15:48.507 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_getxattr": 2, 2024-03-08T21:15:48.507 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_setxattr": 0, 2024-03-08T21:15:48.507 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_cmpxattr": 0, 2024-03-08T21:15:48.507 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_rmxattr": 0, 2024-03-08T21:15:48.508 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_resetxattrs": 0, 2024-03-08T21:15:48.508 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_call": 0, 2024-03-08T21:15:48.508 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_watch": 0, 2024-03-08T21:15:48.508 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_notify": 0, 2024-03-08T21:15:48.508 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_src_cmpxattr": 0, 2024-03-08T21:15:48.508 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_pgls": 0, 2024-03-08T21:15:48.508 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_pgls_filter": 0, 2024-03-08T21:15:48.508 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_other": 0, 2024-03-08T21:15:48.508 INFO:teuthology.orchestra.run.smithi107.stdout: "linger_active": 0, 2024-03-08T21:15:48.508 INFO:teuthology.orchestra.run.smithi107.stdout: "linger_send": 0, 2024-03-08T21:15:48.509 INFO:teuthology.orchestra.run.smithi107.stdout: "linger_resend": 0, 2024-03-08T21:15:48.509 INFO:teuthology.orchestra.run.smithi107.stdout: "linger_ping": 0, 2024-03-08T21:15:48.509 INFO:teuthology.orchestra.run.smithi107.stdout: "poolop_active": 0, 2024-03-08T21:15:48.509 INFO:teuthology.orchestra.run.smithi107.stdout: "poolop_send": 0, 2024-03-08T21:15:48.509 INFO:teuthology.orchestra.run.smithi107.stdout: "poolop_resend": 0, 2024-03-08T21:15:48.509 INFO:teuthology.orchestra.run.smithi107.stdout: "poolstat_active": 0, 2024-03-08T21:15:48.509 INFO:teuthology.orchestra.run.smithi107.stdout: "poolstat_send": 0, 2024-03-08T21:15:48.509 INFO:teuthology.orchestra.run.smithi107.stdout: "poolstat_resend": 0, 2024-03-08T21:15:48.509 INFO:teuthology.orchestra.run.smithi107.stdout: "statfs_active": 0, 2024-03-08T21:15:48.509 INFO:teuthology.orchestra.run.smithi107.stdout: "statfs_send": 0, 2024-03-08T21:15:48.509 INFO:teuthology.orchestra.run.smithi107.stdout: "statfs_resend": 0, 2024-03-08T21:15:48.509 INFO:teuthology.orchestra.run.smithi107.stdout: "command_active": 0, 2024-03-08T21:15:48.509 INFO:teuthology.orchestra.run.smithi107.stdout: "command_send": 0, 2024-03-08T21:15:48.510 INFO:teuthology.orchestra.run.smithi107.stdout: "command_resend": 0, 2024-03-08T21:15:48.510 INFO:teuthology.orchestra.run.smithi107.stdout: "map_epoch": 113, 2024-03-08T21:15:48.510 INFO:teuthology.orchestra.run.smithi107.stdout: "map_full": 0, 2024-03-08T21:15:48.510 INFO:teuthology.orchestra.run.smithi107.stdout: "map_inc": 4, 2024-03-08T21:15:48.510 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_sessions": 7, 2024-03-08T21:15:48.510 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_session_open": 7, 2024-03-08T21:15:48.510 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_session_close": 0, 2024-03-08T21:15:48.510 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_laggy": 0, 2024-03-08T21:15:48.510 INFO:teuthology.orchestra.run.smithi107.stdout: "omap_wr": 0, 2024-03-08T21:15:48.511 INFO:teuthology.orchestra.run.smithi107.stdout: "omap_rd": 2, 2024-03-08T21:15:48.511 INFO:teuthology.orchestra.run.smithi107.stdout: "omap_del": 0 2024-03-08T21:15:48.511 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:48.511 INFO:teuthology.orchestra.run.smithi107.stdout: "oft": { 2024-03-08T21:15:48.511 INFO:teuthology.orchestra.run.smithi107.stdout: "omap_total_objs": 0, 2024-03-08T21:15:48.511 INFO:teuthology.orchestra.run.smithi107.stdout: "omap_total_kv_pairs": 0, 2024-03-08T21:15:48.511 INFO:teuthology.orchestra.run.smithi107.stdout: "omap_total_updates": 0, 2024-03-08T21:15:48.511 INFO:teuthology.orchestra.run.smithi107.stdout: "omap_total_removes": 0 2024-03-08T21:15:48.511 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:48.511 INFO:teuthology.orchestra.run.smithi107.stdout: "osd-slow-ops": { 2024-03-08T21:15:48.511 INFO:teuthology.orchestra.run.smithi107.stdout: "slow_ops_count": 0 2024-03-08T21:15:48.511 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:48.512 INFO:teuthology.orchestra.run.smithi107.stdout: "purge_queue": { 2024-03-08T21:15:48.512 INFO:teuthology.orchestra.run.smithi107.stdout: "pq_executing_ops": 0, 2024-03-08T21:15:48.512 INFO:teuthology.orchestra.run.smithi107.stdout: "pq_executing_ops_high_water": 0, 2024-03-08T21:15:48.512 INFO:teuthology.orchestra.run.smithi107.stdout: "pq_executing": 0, 2024-03-08T21:15:48.512 INFO:teuthology.orchestra.run.smithi107.stdout: "pq_executing_high_water": 0, 2024-03-08T21:15:48.512 INFO:teuthology.orchestra.run.smithi107.stdout: "pq_executed_ops": 0, 2024-03-08T21:15:48.512 INFO:teuthology.orchestra.run.smithi107.stdout: "pq_executed": 0, 2024-03-08T21:15:48.512 INFO:teuthology.orchestra.run.smithi107.stdout: "pq_item_in_journal": 0 2024-03-08T21:15:48.512 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:48.512 INFO:teuthology.orchestra.run.smithi107.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-08T21:15:48.512 INFO:teuthology.orchestra.run.smithi107.stdout: "val": 0, 2024-03-08T21:15:48.512 INFO:teuthology.orchestra.run.smithi107.stdout: "max": 104857600, 2024-03-08T21:15:48.512 INFO:teuthology.orchestra.run.smithi107.stdout: "get_started": 0, 2024-03-08T21:15:48.513 INFO:teuthology.orchestra.run.smithi107.stdout: "get": 591, 2024-03-08T21:15:48.513 INFO:teuthology.orchestra.run.smithi107.stdout: "get_sum": 32874470, 2024-03-08T21:15:48.513 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_fail": 0, 2024-03-08T21:15:48.513 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_success": 591, 2024-03-08T21:15:48.513 INFO:teuthology.orchestra.run.smithi107.stdout: "take": 0, 2024-03-08T21:15:48.513 INFO:teuthology.orchestra.run.smithi107.stdout: "take_sum": 0, 2024-03-08T21:15:48.513 INFO:teuthology.orchestra.run.smithi107.stdout: "put": 591, 2024-03-08T21:15:48.513 INFO:teuthology.orchestra.run.smithi107.stdout: "put_sum": 32874470, 2024-03-08T21:15:48.513 INFO:teuthology.orchestra.run.smithi107.stdout: "wait": { 2024-03-08T21:15:48.513 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-03-08T21:15:48.513 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-03-08T21:15:48.514 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-03-08T21:15:48.514 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:48.514 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:48.514 INFO:teuthology.orchestra.run.smithi107.stdout: "throttle-objecter_bytes": { 2024-03-08T21:15:48.514 INFO:teuthology.orchestra.run.smithi107.stdout: "val": 0, 2024-03-08T21:15:48.514 INFO:teuthology.orchestra.run.smithi107.stdout: "max": 104857600, 2024-03-08T21:15:48.514 INFO:teuthology.orchestra.run.smithi107.stdout: "get_started": 0, 2024-03-08T21:15:48.514 INFO:teuthology.orchestra.run.smithi107.stdout: "get": 0, 2024-03-08T21:15:48.514 INFO:teuthology.orchestra.run.smithi107.stdout: "get_sum": 0, 2024-03-08T21:15:48.514 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_fail": 0, 2024-03-08T21:15:48.514 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_success": 0, 2024-03-08T21:15:48.515 INFO:teuthology.orchestra.run.smithi107.stdout: "take": 534, 2024-03-08T21:15:48.515 INFO:teuthology.orchestra.run.smithi107.stdout: "take_sum": 32605121, 2024-03-08T21:15:48.515 INFO:teuthology.orchestra.run.smithi107.stdout: "put": 534, 2024-03-08T21:15:48.515 INFO:teuthology.orchestra.run.smithi107.stdout: "put_sum": 32605121, 2024-03-08T21:15:48.515 INFO:teuthology.orchestra.run.smithi107.stdout: "wait": { 2024-03-08T21:15:48.515 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-03-08T21:15:48.515 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-03-08T21:15:48.515 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-03-08T21:15:48.515 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:48.515 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:48.515 INFO:teuthology.orchestra.run.smithi107.stdout: "throttle-objecter_ops": { 2024-03-08T21:15:48.515 INFO:teuthology.orchestra.run.smithi107.stdout: "val": 0, 2024-03-08T21:15:48.516 INFO:teuthology.orchestra.run.smithi107.stdout: "max": 1024, 2024-03-08T21:15:48.516 INFO:teuthology.orchestra.run.smithi107.stdout: "get_started": 0, 2024-03-08T21:15:48.516 INFO:teuthology.orchestra.run.smithi107.stdout: "get": 0, 2024-03-08T21:15:48.516 INFO:teuthology.orchestra.run.smithi107.stdout: "get_sum": 0, 2024-03-08T21:15:48.516 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_fail": 0, 2024-03-08T21:15:48.516 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_success": 0, 2024-03-08T21:15:48.516 INFO:teuthology.orchestra.run.smithi107.stdout: "take": 534, 2024-03-08T21:15:48.516 INFO:teuthology.orchestra.run.smithi107.stdout: "take_sum": 534, 2024-03-08T21:15:48.516 INFO:teuthology.orchestra.run.smithi107.stdout: "put": 534, 2024-03-08T21:15:48.516 INFO:teuthology.orchestra.run.smithi107.stdout: "put_sum": 534, 2024-03-08T21:15:48.516 INFO:teuthology.orchestra.run.smithi107.stdout: "wait": { 2024-03-08T21:15:48.516 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-03-08T21:15:48.517 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-03-08T21:15:48.517 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-03-08T21:15:48.517 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:48.517 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:48.517 INFO:teuthology.orchestra.run.smithi107.stdout: "throttle-write_buf_throttle": { 2024-03-08T21:15:48.517 INFO:teuthology.orchestra.run.smithi107.stdout: "val": 0, 2024-03-08T21:15:48.517 INFO:teuthology.orchestra.run.smithi107.stdout: "max": 3758096384, 2024-03-08T21:15:48.517 INFO:teuthology.orchestra.run.smithi107.stdout: "get_started": 0, 2024-03-08T21:15:48.517 INFO:teuthology.orchestra.run.smithi107.stdout: "get": 0, 2024-03-08T21:15:48.517 INFO:teuthology.orchestra.run.smithi107.stdout: "get_sum": 0, 2024-03-08T21:15:48.517 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_fail": 0, 2024-03-08T21:15:48.517 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_success": 0, 2024-03-08T21:15:48.518 INFO:teuthology.orchestra.run.smithi107.stdout: "take": 0, 2024-03-08T21:15:48.518 INFO:teuthology.orchestra.run.smithi107.stdout: "take_sum": 0, 2024-03-08T21:15:48.518 INFO:teuthology.orchestra.run.smithi107.stdout: "put": 0, 2024-03-08T21:15:48.518 INFO:teuthology.orchestra.run.smithi107.stdout: "put_sum": 0, 2024-03-08T21:15:48.518 INFO:teuthology.orchestra.run.smithi107.stdout: "wait": { 2024-03-08T21:15:48.518 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-03-08T21:15:48.518 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-03-08T21:15:48.518 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-03-08T21:15:48.518 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:48.518 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:48.518 INFO:teuthology.orchestra.run.smithi107.stdout: "throttle-write_buf_throttle-0x559e5d400a00": { 2024-03-08T21:15:48.518 INFO:teuthology.orchestra.run.smithi107.stdout: "val": 0, 2024-03-08T21:15:48.519 INFO:teuthology.orchestra.run.smithi107.stdout: "max": 3758096384, 2024-03-08T21:15:48.519 INFO:teuthology.orchestra.run.smithi107.stdout: "get_started": 0, 2024-03-08T21:15:48.519 INFO:teuthology.orchestra.run.smithi107.stdout: "get": 0, 2024-03-08T21:15:48.519 INFO:teuthology.orchestra.run.smithi107.stdout: "get_sum": 0, 2024-03-08T21:15:48.519 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_fail": 0, 2024-03-08T21:15:48.519 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_success": 0, 2024-03-08T21:15:48.519 INFO:teuthology.orchestra.run.smithi107.stdout: "take": 0, 2024-03-08T21:15:48.519 INFO:teuthology.orchestra.run.smithi107.stdout: "take_sum": 0, 2024-03-08T21:15:48.519 INFO:teuthology.orchestra.run.smithi107.stdout: "put": 0, 2024-03-08T21:15:48.519 INFO:teuthology.orchestra.run.smithi107.stdout: "put_sum": 0, 2024-03-08T21:15:48.519 INFO:teuthology.orchestra.run.smithi107.stdout: "wait": { 2024-03-08T21:15:48.519 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-03-08T21:15:48.519 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-03-08T21:15:48.520 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-03-08T21:15:48.520 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:48.520 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:48.520 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-03-08T21:15:48.871 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:15:48 smithi204 ceph-mon[34935]: pgmap v484: 97 pgs: 97 active+clean; 17 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 140 KiB/s wr, 41 op/s 2024-03-08T21:15:48.961 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:15:48 smithi107 ceph-mon[31381]: pgmap v484: 97 pgs: 97 active+clean; 17 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 140 KiB/s wr, 41 op/s 2024-03-08T21:15:49.008 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:15:48 smithi149 ceph-mon[35042]: pgmap v484: 97 pgs: 97 active+clean; 17 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 140 KiB/s wr, 41 op/s 2024-03-08T21:15:49.444 INFO:tasks.check_counter:Daemon mds.d mds.exported=0 2024-03-08T21:15:49.444 INFO:tasks.check_counter:Daemon mds.d mds.imported=0 2024-03-08T21:15:49.444 INFO:tasks.check_counter:Daemon mds.d mds_cache.dir_update=0 2024-03-08T21:15:49.444 INFO:tasks.check_counter:Daemon mds.d mds_cache.dir_update_receipt=0 2024-03-08T21:15:49.444 INFO:tasks.check_counter:Daemon mds.d mds.root_rsnaps=6 2024-03-08T21:15:49.444 INFO:tasks.check_counter:Daemon mds.d mds_server.req_mksnap_latency.avgcount=0 2024-03-08T21:15:49.444 INFO:tasks.check_counter:Daemon mds.d mds_server.req_rmsnap_latency.avgcount=0 2024-03-08T21:15:49.445 DEBUG:tasks.check_counter:Getting stats from g 2024-03-08T21:15:49.445 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761 shell --fsid dc20f0f8-dd8c-11ee-95c6-87774f69a715 -- ceph daemon mds.g perf dump 2024-03-08T21:15:49.892 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/mon.a/config 2024-03-08T21:15:50.711 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:15:50 smithi107 ceph-mon[31381]: pgmap v485: 97 pgs: 97 active+clean; 17 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 140 KiB/s wr, 46 op/s 2024-03-08T21:15:50.871 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:15:50 smithi204 ceph-mon[34935]: pgmap v485: 97 pgs: 97 active+clean; 17 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 140 KiB/s wr, 46 op/s 2024-03-08T21:15:51.008 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:15:50 smithi149 ceph-mon[35042]: pgmap v485: 97 pgs: 97 active+clean; 17 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 140 KiB/s wr, 46 op/s 2024-03-08T21:15:52.214 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-03-08T21:15:52.215 INFO:teuthology.orchestra.run.smithi107.stdout: "AsyncMessenger::Worker-0": { 2024-03-08T21:15:52.215 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_messages": 1077, 2024-03-08T21:15:52.215 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages": 1077, 2024-03-08T21:15:52.215 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_bytes": 363513, 2024-03-08T21:15:52.215 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_bytes": 341663, 2024-03-08T21:15:52.215 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_created_connections": 7, 2024-03-08T21:15:52.215 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_active_connections": 2, 2024-03-08T21:15:52.215 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_total_time": 0.282373495, 2024-03-08T21:15:52.215 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_send_time": 0.127023360, 2024-03-08T21:15:52.215 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_recv_time": 0.062864946, 2024-03-08T21:15:52.215 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_fast_dispatch_time": 0.065258562, 2024-03-08T21:15:52.215 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages_queue_lat": { 2024-03-08T21:15:52.215 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 1076, 2024-03-08T21:15:52.216 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.104572053, 2024-03-08T21:15:52.216 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000097185 2024-03-08T21:15:52.216 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:52.216 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_handle_ack_lat": { 2024-03-08T21:15:52.216 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-03-08T21:15:52.216 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-03-08T21:15:52.216 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-03-08T21:15:52.216 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:52.216 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_encrypted_bytes": 3040, 2024-03-08T21:15:52.216 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_encrypted_bytes": 448 2024-03-08T21:15:52.216 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:52.216 INFO:teuthology.orchestra.run.smithi107.stdout: "AsyncMessenger::Worker-1": { 2024-03-08T21:15:52.216 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_messages": 2008, 2024-03-08T21:15:52.217 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages": 2229, 2024-03-08T21:15:52.217 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_bytes": 512093, 2024-03-08T21:15:52.217 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_bytes": 1191309, 2024-03-08T21:15:52.217 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_created_connections": 7, 2024-03-08T21:15:52.217 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_active_connections": 5, 2024-03-08T21:15:52.217 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_total_time": 0.385310730, 2024-03-08T21:15:52.217 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_send_time": 0.152125175, 2024-03-08T21:15:52.217 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_recv_time": 0.090431114, 2024-03-08T21:15:52.217 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_fast_dispatch_time": 0.109178639, 2024-03-08T21:15:52.217 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages_queue_lat": { 2024-03-08T21:15:52.217 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 2226, 2024-03-08T21:15:52.217 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.204039391, 2024-03-08T21:15:52.217 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000091661 2024-03-08T21:15:52.217 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:52.218 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_handle_ack_lat": { 2024-03-08T21:15:52.218 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-03-08T21:15:52.218 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-03-08T21:15:52.218 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-03-08T21:15:52.218 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:52.218 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_encrypted_bytes": 320, 2024-03-08T21:15:52.218 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_encrypted_bytes": 553344 2024-03-08T21:15:52.218 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:52.218 INFO:teuthology.orchestra.run.smithi107.stdout: "AsyncMessenger::Worker-2": { 2024-03-08T21:15:52.218 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_messages": 352, 2024-03-08T21:15:52.218 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages": 328, 2024-03-08T21:15:52.218 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_bytes": 1579348, 2024-03-08T21:15:52.219 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_bytes": 182549, 2024-03-08T21:15:52.219 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_created_connections": 5, 2024-03-08T21:15:52.219 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_active_connections": 3, 2024-03-08T21:15:52.219 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_total_time": 0.150199750, 2024-03-08T21:15:52.219 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_send_time": 0.051619894, 2024-03-08T21:15:52.219 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_recv_time": 0.118559660, 2024-03-08T21:15:52.219 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_fast_dispatch_time": 0.014599048, 2024-03-08T21:15:52.219 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages_queue_lat": { 2024-03-08T21:15:52.219 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 327, 2024-03-08T21:15:52.219 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.030448545, 2024-03-08T21:15:52.219 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000093114 2024-03-08T21:15:52.219 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:52.219 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_handle_ack_lat": { 2024-03-08T21:15:52.219 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-03-08T21:15:52.220 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-03-08T21:15:52.220 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-03-08T21:15:52.220 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:52.220 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_encrypted_bytes": 1578000, 2024-03-08T21:15:52.220 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_encrypted_bytes": 181584 2024-03-08T21:15:52.220 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:52.220 INFO:teuthology.orchestra.run.smithi107.stdout: "cct": { 2024-03-08T21:15:52.220 INFO:teuthology.orchestra.run.smithi107.stdout: "total_workers": 1, 2024-03-08T21:15:52.220 INFO:teuthology.orchestra.run.smithi107.stdout: "unhealthy_workers": 0 2024-03-08T21:15:52.220 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:52.220 INFO:teuthology.orchestra.run.smithi107.stdout: "finisher-MDSRank": { 2024-03-08T21:15:52.220 INFO:teuthology.orchestra.run.smithi107.stdout: "queue_len": 0, 2024-03-08T21:15:52.220 INFO:teuthology.orchestra.run.smithi107.stdout: "complete_latency": { 2024-03-08T21:15:52.220 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 5220, 2024-03-08T21:15:52.221 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.277193648, 2024-03-08T21:15:52.221 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000053102 2024-03-08T21:15:52.221 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:52.221 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:52.221 INFO:teuthology.orchestra.run.smithi107.stdout: "finisher-PurgeQueue": { 2024-03-08T21:15:52.221 INFO:teuthology.orchestra.run.smithi107.stdout: "queue_len": 0, 2024-03-08T21:15:52.221 INFO:teuthology.orchestra.run.smithi107.stdout: "complete_latency": { 2024-03-08T21:15:52.221 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-03-08T21:15:52.221 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-03-08T21:15:52.221 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-03-08T21:15:52.221 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:52.221 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:52.221 INFO:teuthology.orchestra.run.smithi107.stdout: "mds": { 2024-03-08T21:15:52.221 INFO:teuthology.orchestra.run.smithi107.stdout: "request": 0, 2024-03-08T21:15:52.222 INFO:teuthology.orchestra.run.smithi107.stdout: "reply": 0, 2024-03-08T21:15:52.222 INFO:teuthology.orchestra.run.smithi107.stdout: "reply_latency": { 2024-03-08T21:15:52.222 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-03-08T21:15:52.222 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-03-08T21:15:52.222 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-03-08T21:15:52.222 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:52.222 INFO:teuthology.orchestra.run.smithi107.stdout: "slow_reply": 0, 2024-03-08T21:15:52.222 INFO:teuthology.orchestra.run.smithi107.stdout: "forward": 0, 2024-03-08T21:15:52.222 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_fetch_complete": 0, 2024-03-08T21:15:52.222 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_fetch_keys": 0, 2024-03-08T21:15:52.222 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_commit": 0, 2024-03-08T21:15:52.222 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_split": 0, 2024-03-08T21:15:52.222 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_merge": 0, 2024-03-08T21:15:52.222 INFO:teuthology.orchestra.run.smithi107.stdout: "inodes": 0, 2024-03-08T21:15:52.223 INFO:teuthology.orchestra.run.smithi107.stdout: "inodes_top": 0, 2024-03-08T21:15:52.223 INFO:teuthology.orchestra.run.smithi107.stdout: "inodes_bottom": 0, 2024-03-08T21:15:52.223 INFO:teuthology.orchestra.run.smithi107.stdout: "inodes_pin_tail": 0, 2024-03-08T21:15:52.223 INFO:teuthology.orchestra.run.smithi107.stdout: "inodes_pinned": 0, 2024-03-08T21:15:52.223 INFO:teuthology.orchestra.run.smithi107.stdout: "inodes_expired": 0, 2024-03-08T21:15:52.223 INFO:teuthology.orchestra.run.smithi107.stdout: "inodes_with_caps": 0, 2024-03-08T21:15:52.223 INFO:teuthology.orchestra.run.smithi107.stdout: "caps": 0, 2024-03-08T21:15:52.223 INFO:teuthology.orchestra.run.smithi107.stdout: "subtrees": 1, 2024-03-08T21:15:52.223 INFO:teuthology.orchestra.run.smithi107.stdout: "traverse": 0, 2024-03-08T21:15:52.223 INFO:teuthology.orchestra.run.smithi107.stdout: "traverse_hit": 0, 2024-03-08T21:15:52.223 INFO:teuthology.orchestra.run.smithi107.stdout: "traverse_forward": 0, 2024-03-08T21:15:52.223 INFO:teuthology.orchestra.run.smithi107.stdout: "traverse_discover": 0, 2024-03-08T21:15:52.223 INFO:teuthology.orchestra.run.smithi107.stdout: "traverse_dir_fetch": 0, 2024-03-08T21:15:52.224 INFO:teuthology.orchestra.run.smithi107.stdout: "traverse_remote_ino": 0, 2024-03-08T21:15:52.224 INFO:teuthology.orchestra.run.smithi107.stdout: "traverse_lock": 0, 2024-03-08T21:15:52.224 INFO:teuthology.orchestra.run.smithi107.stdout: "load_cent": 0, 2024-03-08T21:15:52.224 INFO:teuthology.orchestra.run.smithi107.stdout: "q": 0, 2024-03-08T21:15:52.224 INFO:teuthology.orchestra.run.smithi107.stdout: "exported": 0, 2024-03-08T21:15:52.224 INFO:teuthology.orchestra.run.smithi107.stdout: "exported_inodes": 0, 2024-03-08T21:15:52.224 INFO:teuthology.orchestra.run.smithi107.stdout: "imported": 0, 2024-03-08T21:15:52.224 INFO:teuthology.orchestra.run.smithi107.stdout: "imported_inodes": 0, 2024-03-08T21:15:52.224 INFO:teuthology.orchestra.run.smithi107.stdout: "openino_dir_fetch": 0, 2024-03-08T21:15:52.224 INFO:teuthology.orchestra.run.smithi107.stdout: "openino_backtrace_fetch": 0, 2024-03-08T21:15:52.224 INFO:teuthology.orchestra.run.smithi107.stdout: "openino_peer_discover": 0, 2024-03-08T21:15:52.224 INFO:teuthology.orchestra.run.smithi107.stdout: "root_rfiles": 0, 2024-03-08T21:15:52.224 INFO:teuthology.orchestra.run.smithi107.stdout: "root_rbytes": 0, 2024-03-08T21:15:52.224 INFO:teuthology.orchestra.run.smithi107.stdout: "root_rsnaps": 0, 2024-03-08T21:15:52.224 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_backtrace_fetch": 0, 2024-03-08T21:15:52.224 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_set_tag": 0, 2024-03-08T21:15:52.225 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_backtrace_repaired": 0, 2024-03-08T21:15:52.225 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_inotable_repaired": 0, 2024-03-08T21:15:52.225 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_dir_inodes": 0, 2024-03-08T21:15:52.225 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_dir_base_inodes": 0, 2024-03-08T21:15:52.225 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_dirfrag_rstats": 0, 2024-03-08T21:15:52.225 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_file_inodes": 0, 2024-03-08T21:15:52.225 INFO:teuthology.orchestra.run.smithi107.stdout: "handle_inode_file_caps": 0, 2024-03-08T21:15:52.225 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph_cap_op_revoke": 0, 2024-03-08T21:15:52.225 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph_cap_op_grant": 0, 2024-03-08T21:15:52.225 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph_cap_op_trunc": 0, 2024-03-08T21:15:52.225 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-08T21:15:52.225 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-08T21:15:52.225 INFO:teuthology.orchestra.run.smithi107.stdout: "handle_client_caps": 0, 2024-03-08T21:15:52.225 INFO:teuthology.orchestra.run.smithi107.stdout: "handle_client_caps_dirty": 0, 2024-03-08T21:15:52.225 INFO:teuthology.orchestra.run.smithi107.stdout: "handle_client_cap_release": 0, 2024-03-08T21:15:52.225 INFO:teuthology.orchestra.run.smithi107.stdout: "process_request_cap_release": 0 2024-03-08T21:15:52.226 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:52.226 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_cache": { 2024-03-08T21:15:52.226 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_update": 0, 2024-03-08T21:15:52.226 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_update_receipt": 0, 2024-03-08T21:15:52.226 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_try_discover": 0, 2024-03-08T21:15:52.226 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_send_discover": 0, 2024-03-08T21:15:52.226 INFO:teuthology.orchestra.run.smithi107.stdout: "dir_handle_discover": 0, 2024-03-08T21:15:52.226 INFO:teuthology.orchestra.run.smithi107.stdout: "num_strays": 0, 2024-03-08T21:15:52.226 INFO:teuthology.orchestra.run.smithi107.stdout: "num_strays_delayed": 0, 2024-03-08T21:15:52.226 INFO:teuthology.orchestra.run.smithi107.stdout: "num_strays_enqueuing": 0, 2024-03-08T21:15:52.226 INFO:teuthology.orchestra.run.smithi107.stdout: "strays_created": 0, 2024-03-08T21:15:52.226 INFO:teuthology.orchestra.run.smithi107.stdout: "strays_enqueued": 0, 2024-03-08T21:15:52.226 INFO:teuthology.orchestra.run.smithi107.stdout: "strays_reintegrated": 0, 2024-03-08T21:15:52.226 INFO:teuthology.orchestra.run.smithi107.stdout: "strays_migrated": 0, 2024-03-08T21:15:52.227 INFO:teuthology.orchestra.run.smithi107.stdout: "num_recovering_processing": 0, 2024-03-08T21:15:52.227 INFO:teuthology.orchestra.run.smithi107.stdout: "num_recovering_enqueued": 0, 2024-03-08T21:15:52.227 INFO:teuthology.orchestra.run.smithi107.stdout: "num_recovering_prioritized": 0, 2024-03-08T21:15:52.227 INFO:teuthology.orchestra.run.smithi107.stdout: "recovery_started": 0, 2024-03-08T21:15:52.227 INFO:teuthology.orchestra.run.smithi107.stdout: "recovery_completed": 0, 2024-03-08T21:15:52.227 INFO:teuthology.orchestra.run.smithi107.stdout: "ireq_enqueue_scrub": 0, 2024-03-08T21:15:52.227 INFO:teuthology.orchestra.run.smithi107.stdout: "ireq_exportdir": 0, 2024-03-08T21:15:52.227 INFO:teuthology.orchestra.run.smithi107.stdout: "ireq_flush": 0, 2024-03-08T21:15:52.227 INFO:teuthology.orchestra.run.smithi107.stdout: "ireq_fragmentdir": 0, 2024-03-08T21:15:52.227 INFO:teuthology.orchestra.run.smithi107.stdout: "ireq_fragstats": 0, 2024-03-08T21:15:52.227 INFO:teuthology.orchestra.run.smithi107.stdout: "ireq_inodestats": 0 2024-03-08T21:15:52.227 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:52.227 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_log": { 2024-03-08T21:15:52.227 INFO:teuthology.orchestra.run.smithi107.stdout: "evlrg": 0, 2024-03-08T21:15:52.227 INFO:teuthology.orchestra.run.smithi107.stdout: "evadd": 0, 2024-03-08T21:15:52.228 INFO:teuthology.orchestra.run.smithi107.stdout: "evex": 0, 2024-03-08T21:15:52.228 INFO:teuthology.orchestra.run.smithi107.stdout: "evtrm": 0, 2024-03-08T21:15:52.228 INFO:teuthology.orchestra.run.smithi107.stdout: "ev": 3, 2024-03-08T21:15:52.228 INFO:teuthology.orchestra.run.smithi107.stdout: "evexg": 0, 2024-03-08T21:15:52.228 INFO:teuthology.orchestra.run.smithi107.stdout: "evexd": 0, 2024-03-08T21:15:52.228 INFO:teuthology.orchestra.run.smithi107.stdout: "segadd": 0, 2024-03-08T21:15:52.228 INFO:teuthology.orchestra.run.smithi107.stdout: "segex": 0, 2024-03-08T21:15:52.228 INFO:teuthology.orchestra.run.smithi107.stdout: "segtrm": 0, 2024-03-08T21:15:52.228 INFO:teuthology.orchestra.run.smithi107.stdout: "seg": 2, 2024-03-08T21:15:52.228 INFO:teuthology.orchestra.run.smithi107.stdout: "segmjr": 2, 2024-03-08T21:15:52.228 INFO:teuthology.orchestra.run.smithi107.stdout: "segexg": 0, 2024-03-08T21:15:52.228 INFO:teuthology.orchestra.run.smithi107.stdout: "segexd": 0, 2024-03-08T21:15:52.228 INFO:teuthology.orchestra.run.smithi107.stdout: "expos": 4194304, 2024-03-08T21:15:52.228 INFO:teuthology.orchestra.run.smithi107.stdout: "wrpos": 4195385, 2024-03-08T21:15:52.228 INFO:teuthology.orchestra.run.smithi107.stdout: "rdpos": 4194892, 2024-03-08T21:15:52.228 INFO:teuthology.orchestra.run.smithi107.stdout: "jlat": { 2024-03-08T21:15:52.229 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-03-08T21:15:52.229 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-03-08T21:15:52.229 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-03-08T21:15:52.229 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:52.229 INFO:teuthology.orchestra.run.smithi107.stdout: "replayed": 3 2024-03-08T21:15:52.229 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:52.229 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_mem": { 2024-03-08T21:15:52.229 INFO:teuthology.orchestra.run.smithi107.stdout: "ino": 3, 2024-03-08T21:15:52.229 INFO:teuthology.orchestra.run.smithi107.stdout: "ino+": 3, 2024-03-08T21:15:52.229 INFO:teuthology.orchestra.run.smithi107.stdout: "ino-": 0, 2024-03-08T21:15:52.229 INFO:teuthology.orchestra.run.smithi107.stdout: "dir": 1, 2024-03-08T21:15:52.229 INFO:teuthology.orchestra.run.smithi107.stdout: "dir+": 1, 2024-03-08T21:15:52.229 INFO:teuthology.orchestra.run.smithi107.stdout: "dir-": 0, 2024-03-08T21:15:52.229 INFO:teuthology.orchestra.run.smithi107.stdout: "dn": 0, 2024-03-08T21:15:52.230 INFO:teuthology.orchestra.run.smithi107.stdout: "dn+": 0, 2024-03-08T21:15:52.230 INFO:teuthology.orchestra.run.smithi107.stdout: "dn-": 0, 2024-03-08T21:15:52.230 INFO:teuthology.orchestra.run.smithi107.stdout: "cap": 0, 2024-03-08T21:15:52.230 INFO:teuthology.orchestra.run.smithi107.stdout: "cap+": 0, 2024-03-08T21:15:52.230 INFO:teuthology.orchestra.run.smithi107.stdout: "cap-": 0, 2024-03-08T21:15:52.230 INFO:teuthology.orchestra.run.smithi107.stdout: "rss": 49152, 2024-03-08T21:15:52.230 INFO:teuthology.orchestra.run.smithi107.stdout: "heap": 215420 2024-03-08T21:15:52.230 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:52.230 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_server": { 2024-03-08T21:15:52.230 INFO:teuthology.orchestra.run.smithi107.stdout: "dispatch_client_request": 0, 2024-03-08T21:15:52.230 INFO:teuthology.orchestra.run.smithi107.stdout: "dispatch_server_request": 0, 2024-03-08T21:15:52.230 INFO:teuthology.orchestra.run.smithi107.stdout: "handle_client_request": 0, 2024-03-08T21:15:52.230 INFO:teuthology.orchestra.run.smithi107.stdout: "handle_client_session": 0, 2024-03-08T21:15:52.231 INFO:teuthology.orchestra.run.smithi107.stdout: "handle_peer_request": 0, 2024-03-08T21:15:52.231 INFO:teuthology.orchestra.run.smithi107.stdout: "req_create_latency": { 2024-03-08T21:15:52.231 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-03-08T21:15:52.231 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-03-08T21:15:52.231 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-03-08T21:15:52.231 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:52.231 INFO:teuthology.orchestra.run.smithi107.stdout: "req_getattr_latency": { 2024-03-08T21:15:52.231 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-03-08T21:15:52.231 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-03-08T21:15:52.231 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-03-08T21:15:52.231 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:52.231 INFO:teuthology.orchestra.run.smithi107.stdout: "req_getfilelock_latency": { 2024-03-08T21:15:52.231 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-03-08T21:15:52.231 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-03-08T21:15:52.232 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-03-08T21:15:52.232 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:52.232 INFO:teuthology.orchestra.run.smithi107.stdout: "req_link_latency": { 2024-03-08T21:15:52.232 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-03-08T21:15:52.232 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-03-08T21:15:52.232 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-03-08T21:15:52.232 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:52.232 INFO:teuthology.orchestra.run.smithi107.stdout: "req_lookup_latency": { 2024-03-08T21:15:52.232 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-03-08T21:15:52.232 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-03-08T21:15:52.232 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-03-08T21:15:52.232 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:52.232 INFO:teuthology.orchestra.run.smithi107.stdout: "req_lookuphash_latency": { 2024-03-08T21:15:52.232 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-03-08T21:15:52.232 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-03-08T21:15:52.233 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-03-08T21:15:52.233 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:52.233 INFO:teuthology.orchestra.run.smithi107.stdout: "req_lookupino_latency": { 2024-03-08T21:15:52.233 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-03-08T21:15:52.233 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-03-08T21:15:52.233 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-03-08T21:15:52.233 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:52.233 INFO:teuthology.orchestra.run.smithi107.stdout: "req_lookupname_latency": { 2024-03-08T21:15:52.233 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-03-08T21:15:52.233 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-03-08T21:15:52.233 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-03-08T21:15:52.233 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:52.233 INFO:teuthology.orchestra.run.smithi107.stdout: "req_lookupparent_latency": { 2024-03-08T21:15:52.233 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-03-08T21:15:52.233 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-03-08T21:15:52.234 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-03-08T21:15:52.234 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:52.234 INFO:teuthology.orchestra.run.smithi107.stdout: "req_lookupsnap_latency": { 2024-03-08T21:15:52.234 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-03-08T21:15:52.234 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-03-08T21:15:52.234 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-03-08T21:15:52.234 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:52.234 INFO:teuthology.orchestra.run.smithi107.stdout: "req_lssnap_latency": { 2024-03-08T21:15:52.234 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-03-08T21:15:52.234 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-03-08T21:15:52.234 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-03-08T21:15:52.234 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:52.234 INFO:teuthology.orchestra.run.smithi107.stdout: "req_mkdir_latency": { 2024-03-08T21:15:52.235 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-03-08T21:15:52.235 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-03-08T21:15:52.235 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-03-08T21:15:52.235 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:52.235 INFO:teuthology.orchestra.run.smithi107.stdout: "req_mknod_latency": { 2024-03-08T21:15:52.236 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-03-08T21:15:52.236 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-03-08T21:15:52.236 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-03-08T21:15:52.236 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:52.236 INFO:teuthology.orchestra.run.smithi107.stdout: "req_mksnap_latency": { 2024-03-08T21:15:52.236 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-03-08T21:15:52.236 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-03-08T21:15:52.236 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-03-08T21:15:52.236 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:52.236 INFO:teuthology.orchestra.run.smithi107.stdout: "req_open_latency": { 2024-03-08T21:15:52.236 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-03-08T21:15:52.236 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-03-08T21:15:52.236 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-03-08T21:15:52.236 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:52.237 INFO:teuthology.orchestra.run.smithi107.stdout: "req_readdir_latency": { 2024-03-08T21:15:52.237 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-03-08T21:15:52.237 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-03-08T21:15:52.237 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-03-08T21:15:52.237 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:52.237 INFO:teuthology.orchestra.run.smithi107.stdout: "req_rename_latency": { 2024-03-08T21:15:52.237 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-03-08T21:15:52.237 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-03-08T21:15:52.237 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-03-08T21:15:52.237 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:52.237 INFO:teuthology.orchestra.run.smithi107.stdout: "req_renamesnap_latency": { 2024-03-08T21:15:52.237 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-03-08T21:15:52.237 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-03-08T21:15:52.237 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-03-08T21:15:52.237 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:52.238 INFO:teuthology.orchestra.run.smithi107.stdout: "req_snapdiff_latency": { 2024-03-08T21:15:52.238 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-03-08T21:15:52.238 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-03-08T21:15:52.238 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-03-08T21:15:52.238 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:52.238 INFO:teuthology.orchestra.run.smithi107.stdout: "req_rmdir_latency": { 2024-03-08T21:15:52.238 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-03-08T21:15:52.238 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-03-08T21:15:52.238 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-03-08T21:15:52.238 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:52.238 INFO:teuthology.orchestra.run.smithi107.stdout: "req_rmsnap_latency": { 2024-03-08T21:15:52.238 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-03-08T21:15:52.238 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-03-08T21:15:52.238 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-03-08T21:15:52.239 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:52.239 INFO:teuthology.orchestra.run.smithi107.stdout: "req_rmxattr_latency": { 2024-03-08T21:15:52.239 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-03-08T21:15:52.239 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-03-08T21:15:52.239 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-03-08T21:15:52.239 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:52.239 INFO:teuthology.orchestra.run.smithi107.stdout: "req_setattr_latency": { 2024-03-08T21:15:52.239 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-03-08T21:15:52.239 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-03-08T21:15:52.239 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-03-08T21:15:52.239 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:52.239 INFO:teuthology.orchestra.run.smithi107.stdout: "req_setdirlayout_latency": { 2024-03-08T21:15:52.239 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-03-08T21:15:52.239 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-03-08T21:15:52.239 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-03-08T21:15:52.239 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:52.240 INFO:teuthology.orchestra.run.smithi107.stdout: "req_setfilelock_latency": { 2024-03-08T21:15:52.240 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-03-08T21:15:52.240 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-03-08T21:15:52.240 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-03-08T21:15:52.240 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:52.240 INFO:teuthology.orchestra.run.smithi107.stdout: "req_setlayout_latency": { 2024-03-08T21:15:52.240 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-03-08T21:15:52.240 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-03-08T21:15:52.240 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-03-08T21:15:52.240 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:52.240 INFO:teuthology.orchestra.run.smithi107.stdout: "req_setxattr_latency": { 2024-03-08T21:15:52.240 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-03-08T21:15:52.240 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-03-08T21:15:52.240 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-03-08T21:15:52.241 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:52.241 INFO:teuthology.orchestra.run.smithi107.stdout: "req_symlink_latency": { 2024-03-08T21:15:52.241 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-03-08T21:15:52.241 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-03-08T21:15:52.241 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-03-08T21:15:52.241 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:52.241 INFO:teuthology.orchestra.run.smithi107.stdout: "req_unlink_latency": { 2024-03-08T21:15:52.241 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-03-08T21:15:52.241 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-03-08T21:15:52.241 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-03-08T21:15:52.241 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:52.241 INFO:teuthology.orchestra.run.smithi107.stdout: "cap_revoke_eviction": 0, 2024-03-08T21:15:52.241 INFO:teuthology.orchestra.run.smithi107.stdout: "cap_acquisition_throttle": 0, 2024-03-08T21:15:52.241 INFO:teuthology.orchestra.run.smithi107.stdout: "req_getvxattr_latency": { 2024-03-08T21:15:52.241 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-03-08T21:15:52.242 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-03-08T21:15:52.242 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-03-08T21:15:52.242 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:52.242 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:52.242 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_sessions": { 2024-03-08T21:15:52.242 INFO:teuthology.orchestra.run.smithi107.stdout: "session_count": 1, 2024-03-08T21:15:52.242 INFO:teuthology.orchestra.run.smithi107.stdout: "session_add": 1, 2024-03-08T21:15:52.242 INFO:teuthology.orchestra.run.smithi107.stdout: "session_remove": 0, 2024-03-08T21:15:52.242 INFO:teuthology.orchestra.run.smithi107.stdout: "sessions_open": 1, 2024-03-08T21:15:52.242 INFO:teuthology.orchestra.run.smithi107.stdout: "sessions_stale": 0, 2024-03-08T21:15:52.242 INFO:teuthology.orchestra.run.smithi107.stdout: "total_load": 0, 2024-03-08T21:15:52.242 INFO:teuthology.orchestra.run.smithi107.stdout: "average_load": 0, 2024-03-08T21:15:52.242 INFO:teuthology.orchestra.run.smithi107.stdout: "avg_session_uptime": 2398, 2024-03-08T21:15:52.242 INFO:teuthology.orchestra.run.smithi107.stdout: "mdthresh_evicted": 0 2024-03-08T21:15:52.243 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:52.243 INFO:teuthology.orchestra.run.smithi107.stdout: "mempool": { 2024-03-08T21:15:52.243 INFO:teuthology.orchestra.run.smithi107.stdout: "bloom_filter_bytes": 0, 2024-03-08T21:15:52.243 INFO:teuthology.orchestra.run.smithi107.stdout: "bloom_filter_items": 0, 2024-03-08T21:15:52.243 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_alloc_bytes": 0, 2024-03-08T21:15:52.243 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_alloc_items": 0, 2024-03-08T21:15:52.243 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_data_bytes": 0, 2024-03-08T21:15:52.243 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_data_items": 0, 2024-03-08T21:15:52.243 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-08T21:15:52.243 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_onode_items": 0, 2024-03-08T21:15:52.243 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-08T21:15:52.243 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_meta_items": 0, 2024-03-08T21:15:52.243 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_other_bytes": 0, 2024-03-08T21:15:52.243 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_other_items": 0, 2024-03-08T21:15:52.243 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-08T21:15:52.243 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_buffer_items": 0, 2024-03-08T21:15:52.243 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_extent_bytes": 0, 2024-03-08T21:15:52.244 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_extent_items": 0, 2024-03-08T21:15:52.244 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_blob_bytes": 0, 2024-03-08T21:15:52.244 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_blob_items": 0, 2024-03-08T21:15:52.244 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-08T21:15:52.244 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_shared_blob_items": 0, 2024-03-08T21:15:52.244 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-08T21:15:52.244 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_inline_bl_items": 0, 2024-03-08T21:15:52.244 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_fsck_bytes": 0, 2024-03-08T21:15:52.244 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_fsck_items": 0, 2024-03-08T21:15:52.244 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_txc_bytes": 0, 2024-03-08T21:15:52.244 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_txc_items": 0, 2024-03-08T21:15:52.244 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-08T21:15:52.244 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_writing_deferred_items": 0, 2024-03-08T21:15:52.244 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_writing_bytes": 0, 2024-03-08T21:15:52.244 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_writing_items": 0, 2024-03-08T21:15:52.244 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_bytes": 0, 2024-03-08T21:15:52.245 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_items": 0, 2024-03-08T21:15:52.245 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_file_reader_bytes": 0, 2024-03-08T21:15:52.245 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_file_reader_items": 0, 2024-03-08T21:15:52.245 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_file_writer_bytes": 0, 2024-03-08T21:15:52.245 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_file_writer_items": 0, 2024-03-08T21:15:52.245 INFO:teuthology.orchestra.run.smithi107.stdout: "buffer_anon_bytes": 198291, 2024-03-08T21:15:52.245 INFO:teuthology.orchestra.run.smithi107.stdout: "buffer_anon_items": 59, 2024-03-08T21:15:52.245 INFO:teuthology.orchestra.run.smithi107.stdout: "buffer_meta_bytes": 0, 2024-03-08T21:15:52.245 INFO:teuthology.orchestra.run.smithi107.stdout: "buffer_meta_items": 0, 2024-03-08T21:15:52.245 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_bytes": 0, 2024-03-08T21:15:52.245 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_items": 0, 2024-03-08T21:15:52.245 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_mapbl_bytes": 0, 2024-03-08T21:15:52.245 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_mapbl_items": 0, 2024-03-08T21:15:52.245 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_pglog_bytes": 0, 2024-03-08T21:15:52.245 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_pglog_items": 0, 2024-03-08T21:15:52.245 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_bytes": 8352, 2024-03-08T21:15:52.246 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_items": 190, 2024-03-08T21:15:52.246 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_mapping_bytes": 0, 2024-03-08T21:15:52.246 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_mapping_items": 0, 2024-03-08T21:15:52.246 INFO:teuthology.orchestra.run.smithi107.stdout: "pgmap_bytes": 0, 2024-03-08T21:15:52.246 INFO:teuthology.orchestra.run.smithi107.stdout: "pgmap_items": 0, 2024-03-08T21:15:52.246 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_co_bytes": 8320, 2024-03-08T21:15:52.246 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_co_items": 13, 2024-03-08T21:15:52.246 INFO:teuthology.orchestra.run.smithi107.stdout: "unittest_1_bytes": 0, 2024-03-08T21:15:52.246 INFO:teuthology.orchestra.run.smithi107.stdout: "unittest_1_items": 0, 2024-03-08T21:15:52.246 INFO:teuthology.orchestra.run.smithi107.stdout: "unittest_2_bytes": 0, 2024-03-08T21:15:52.246 INFO:teuthology.orchestra.run.smithi107.stdout: "unittest_2_items": 0 2024-03-08T21:15:52.246 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:52.246 INFO:teuthology.orchestra.run.smithi107.stdout: "objecter": { 2024-03-08T21:15:52.246 INFO:teuthology.orchestra.run.smithi107.stdout: "op_active": 0, 2024-03-08T21:15:52.246 INFO:teuthology.orchestra.run.smithi107.stdout: "op_laggy": 0, 2024-03-08T21:15:52.247 INFO:teuthology.orchestra.run.smithi107.stdout: "op_send": 3082, 2024-03-08T21:15:52.247 INFO:teuthology.orchestra.run.smithi107.stdout: "op_send_bytes": 21, 2024-03-08T21:15:52.247 INFO:teuthology.orchestra.run.smithi107.stdout: "op_resend": 3, 2024-03-08T21:15:52.247 INFO:teuthology.orchestra.run.smithi107.stdout: "op_reply": 3082, 2024-03-08T21:15:52.247 INFO:teuthology.orchestra.run.smithi107.stdout: "op_latency": { 2024-03-08T21:15:52.247 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 3082, 2024-03-08T21:15:52.247 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 3.785938463, 2024-03-08T21:15:52.247 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.001228403 2024-03-08T21:15:52.247 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:52.247 INFO:teuthology.orchestra.run.smithi107.stdout: "op_inflight": 0, 2024-03-08T21:15:52.247 INFO:teuthology.orchestra.run.smithi107.stdout: "oplen_avg": { 2024-03-08T21:15:52.247 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 3082, 2024-03-08T21:15:52.247 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 3083 2024-03-08T21:15:52.247 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:52.247 INFO:teuthology.orchestra.run.smithi107.stdout: "op": 3082, 2024-03-08T21:15:52.248 INFO:teuthology.orchestra.run.smithi107.stdout: "op_r": 3082, 2024-03-08T21:15:52.248 INFO:teuthology.orchestra.run.smithi107.stdout: "op_w": 0, 2024-03-08T21:15:52.248 INFO:teuthology.orchestra.run.smithi107.stdout: "op_rmw": 0, 2024-03-08T21:15:52.248 INFO:teuthology.orchestra.run.smithi107.stdout: "op_pg": 0, 2024-03-08T21:15:52.248 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_stat": 2003, 2024-03-08T21:15:52.248 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_create": 0, 2024-03-08T21:15:52.248 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_read": 1077, 2024-03-08T21:15:52.248 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_write": 0, 2024-03-08T21:15:52.248 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_writefull": 0, 2024-03-08T21:15:52.248 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_writesame": 0, 2024-03-08T21:15:52.248 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_append": 0, 2024-03-08T21:15:52.248 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_zero": 0, 2024-03-08T21:15:52.248 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_truncate": 0, 2024-03-08T21:15:52.248 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_delete": 0, 2024-03-08T21:15:52.248 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_mapext": 0, 2024-03-08T21:15:52.249 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_sparse_read": 0, 2024-03-08T21:15:52.249 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_clonerange": 0, 2024-03-08T21:15:52.249 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_getxattr": 1, 2024-03-08T21:15:52.249 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_setxattr": 0, 2024-03-08T21:15:52.249 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_cmpxattr": 0, 2024-03-08T21:15:52.249 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_rmxattr": 0, 2024-03-08T21:15:52.249 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_resetxattrs": 0, 2024-03-08T21:15:52.249 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_call": 0, 2024-03-08T21:15:52.249 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_watch": 0, 2024-03-08T21:15:52.249 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_notify": 0, 2024-03-08T21:15:52.249 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_src_cmpxattr": 0, 2024-03-08T21:15:52.249 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_pgls": 0, 2024-03-08T21:15:52.249 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_pgls_filter": 0, 2024-03-08T21:15:52.249 INFO:teuthology.orchestra.run.smithi107.stdout: "osdop_other": 0, 2024-03-08T21:15:52.249 INFO:teuthology.orchestra.run.smithi107.stdout: "linger_active": 0, 2024-03-08T21:15:52.249 INFO:teuthology.orchestra.run.smithi107.stdout: "linger_send": 0, 2024-03-08T21:15:52.250 INFO:teuthology.orchestra.run.smithi107.stdout: "linger_resend": 0, 2024-03-08T21:15:52.250 INFO:teuthology.orchestra.run.smithi107.stdout: "linger_ping": 0, 2024-03-08T21:15:52.250 INFO:teuthology.orchestra.run.smithi107.stdout: "poolop_active": 0, 2024-03-08T21:15:52.250 INFO:teuthology.orchestra.run.smithi107.stdout: "poolop_send": 0, 2024-03-08T21:15:52.250 INFO:teuthology.orchestra.run.smithi107.stdout: "poolop_resend": 0, 2024-03-08T21:15:52.250 INFO:teuthology.orchestra.run.smithi107.stdout: "poolstat_active": 0, 2024-03-08T21:15:52.250 INFO:teuthology.orchestra.run.smithi107.stdout: "poolstat_send": 0, 2024-03-08T21:15:52.250 INFO:teuthology.orchestra.run.smithi107.stdout: "poolstat_resend": 0, 2024-03-08T21:15:52.250 INFO:teuthology.orchestra.run.smithi107.stdout: "statfs_active": 0, 2024-03-08T21:15:52.250 INFO:teuthology.orchestra.run.smithi107.stdout: "statfs_send": 0, 2024-03-08T21:15:52.250 INFO:teuthology.orchestra.run.smithi107.stdout: "statfs_resend": 0, 2024-03-08T21:15:52.250 INFO:teuthology.orchestra.run.smithi107.stdout: "command_active": 0, 2024-03-08T21:15:52.250 INFO:teuthology.orchestra.run.smithi107.stdout: "command_send": 0, 2024-03-08T21:15:52.250 INFO:teuthology.orchestra.run.smithi107.stdout: "command_resend": 0, 2024-03-08T21:15:52.251 INFO:teuthology.orchestra.run.smithi107.stdout: "map_epoch": 113, 2024-03-08T21:15:52.251 INFO:teuthology.orchestra.run.smithi107.stdout: "map_full": 0, 2024-03-08T21:15:52.251 INFO:teuthology.orchestra.run.smithi107.stdout: "map_inc": 27, 2024-03-08T21:15:52.251 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_sessions": 6, 2024-03-08T21:15:52.251 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_session_open": 8, 2024-03-08T21:15:52.251 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_session_close": 2, 2024-03-08T21:15:52.251 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_laggy": 0, 2024-03-08T21:15:52.251 INFO:teuthology.orchestra.run.smithi107.stdout: "omap_wr": 0, 2024-03-08T21:15:52.251 INFO:teuthology.orchestra.run.smithi107.stdout: "omap_rd": 2, 2024-03-08T21:15:52.251 INFO:teuthology.orchestra.run.smithi107.stdout: "omap_del": 0 2024-03-08T21:15:52.251 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:52.251 INFO:teuthology.orchestra.run.smithi107.stdout: "oft": { 2024-03-08T21:15:52.251 INFO:teuthology.orchestra.run.smithi107.stdout: "omap_total_objs": 0, 2024-03-08T21:15:52.251 INFO:teuthology.orchestra.run.smithi107.stdout: "omap_total_kv_pairs": 0, 2024-03-08T21:15:52.252 INFO:teuthology.orchestra.run.smithi107.stdout: "omap_total_updates": 0, 2024-03-08T21:15:52.252 INFO:teuthology.orchestra.run.smithi107.stdout: "omap_total_removes": 0 2024-03-08T21:15:52.252 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:52.252 INFO:teuthology.orchestra.run.smithi107.stdout: "osd-slow-ops": { 2024-03-08T21:15:52.252 INFO:teuthology.orchestra.run.smithi107.stdout: "slow_ops_count": 0 2024-03-08T21:15:52.252 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:52.252 INFO:teuthology.orchestra.run.smithi107.stdout: "purge_queue": { 2024-03-08T21:15:52.252 INFO:teuthology.orchestra.run.smithi107.stdout: "pq_executing_ops": 0, 2024-03-08T21:15:52.252 INFO:teuthology.orchestra.run.smithi107.stdout: "pq_executing_ops_high_water": 0, 2024-03-08T21:15:52.252 INFO:teuthology.orchestra.run.smithi107.stdout: "pq_executing": 0, 2024-03-08T21:15:52.252 INFO:teuthology.orchestra.run.smithi107.stdout: "pq_executing_high_water": 0, 2024-03-08T21:15:52.252 INFO:teuthology.orchestra.run.smithi107.stdout: "pq_executed_ops": 0, 2024-03-08T21:15:52.252 INFO:teuthology.orchestra.run.smithi107.stdout: "pq_executed": 0, 2024-03-08T21:15:52.252 INFO:teuthology.orchestra.run.smithi107.stdout: "pq_item_in_journal": 0 2024-03-08T21:15:52.252 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:52.253 INFO:teuthology.orchestra.run.smithi107.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-08T21:15:52.253 INFO:teuthology.orchestra.run.smithi107.stdout: "val": 0, 2024-03-08T21:15:52.253 INFO:teuthology.orchestra.run.smithi107.stdout: "max": 104857600, 2024-03-08T21:15:52.253 INFO:teuthology.orchestra.run.smithi107.stdout: "get_started": 0, 2024-03-08T21:15:52.253 INFO:teuthology.orchestra.run.smithi107.stdout: "get": 3433, 2024-03-08T21:15:52.253 INFO:teuthology.orchestra.run.smithi107.stdout: "get_sum": 2125287, 2024-03-08T21:15:52.253 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_fail": 0, 2024-03-08T21:15:52.253 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_success": 3433, 2024-03-08T21:15:52.253 INFO:teuthology.orchestra.run.smithi107.stdout: "take": 0, 2024-03-08T21:15:52.253 INFO:teuthology.orchestra.run.smithi107.stdout: "take_sum": 0, 2024-03-08T21:15:52.253 INFO:teuthology.orchestra.run.smithi107.stdout: "put": 3433, 2024-03-08T21:15:52.253 INFO:teuthology.orchestra.run.smithi107.stdout: "put_sum": 2125287, 2024-03-08T21:15:52.253 INFO:teuthology.orchestra.run.smithi107.stdout: "wait": { 2024-03-08T21:15:52.253 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-03-08T21:15:52.253 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-03-08T21:15:52.253 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-03-08T21:15:52.253 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:52.254 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:52.254 INFO:teuthology.orchestra.run.smithi107.stdout: "throttle-objecter_bytes": { 2024-03-08T21:15:52.254 INFO:teuthology.orchestra.run.smithi107.stdout: "val": 0, 2024-03-08T21:15:52.254 INFO:teuthology.orchestra.run.smithi107.stdout: "max": 104857600, 2024-03-08T21:15:52.254 INFO:teuthology.orchestra.run.smithi107.stdout: "get_started": 0, 2024-03-08T21:15:52.254 INFO:teuthology.orchestra.run.smithi107.stdout: "get": 0, 2024-03-08T21:15:52.254 INFO:teuthology.orchestra.run.smithi107.stdout: "get_sum": 0, 2024-03-08T21:15:52.254 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_fail": 0, 2024-03-08T21:15:52.254 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_success": 0, 2024-03-08T21:15:52.254 INFO:teuthology.orchestra.run.smithi107.stdout: "take": 3082, 2024-03-08T21:15:52.254 INFO:teuthology.orchestra.run.smithi107.stdout: "take_sum": 1086, 2024-03-08T21:15:52.254 INFO:teuthology.orchestra.run.smithi107.stdout: "put": 3082, 2024-03-08T21:15:52.254 INFO:teuthology.orchestra.run.smithi107.stdout: "put_sum": 1086, 2024-03-08T21:15:52.255 INFO:teuthology.orchestra.run.smithi107.stdout: "wait": { 2024-03-08T21:15:52.255 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-03-08T21:15:52.255 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-03-08T21:15:52.255 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-03-08T21:15:52.255 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:52.255 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:52.255 INFO:teuthology.orchestra.run.smithi107.stdout: "throttle-objecter_ops": { 2024-03-08T21:15:52.255 INFO:teuthology.orchestra.run.smithi107.stdout: "val": 0, 2024-03-08T21:15:52.255 INFO:teuthology.orchestra.run.smithi107.stdout: "max": 1024, 2024-03-08T21:15:52.255 INFO:teuthology.orchestra.run.smithi107.stdout: "get_started": 0, 2024-03-08T21:15:52.255 INFO:teuthology.orchestra.run.smithi107.stdout: "get": 0, 2024-03-08T21:15:52.255 INFO:teuthology.orchestra.run.smithi107.stdout: "get_sum": 0, 2024-03-08T21:15:52.255 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_fail": 0, 2024-03-08T21:15:52.256 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_success": 0, 2024-03-08T21:15:52.256 INFO:teuthology.orchestra.run.smithi107.stdout: "take": 3082, 2024-03-08T21:15:52.256 INFO:teuthology.orchestra.run.smithi107.stdout: "take_sum": 3082, 2024-03-08T21:15:52.256 INFO:teuthology.orchestra.run.smithi107.stdout: "put": 3082, 2024-03-08T21:15:52.256 INFO:teuthology.orchestra.run.smithi107.stdout: "put_sum": 3082, 2024-03-08T21:15:52.256 INFO:teuthology.orchestra.run.smithi107.stdout: "wait": { 2024-03-08T21:15:52.256 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-03-08T21:15:52.256 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-03-08T21:15:52.256 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-03-08T21:15:52.256 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:52.256 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:52.256 INFO:teuthology.orchestra.run.smithi107.stdout: "throttle-write_buf_throttle": { 2024-03-08T21:15:52.256 INFO:teuthology.orchestra.run.smithi107.stdout: "val": 0, 2024-03-08T21:15:52.256 INFO:teuthology.orchestra.run.smithi107.stdout: "max": 3758096384, 2024-03-08T21:15:52.257 INFO:teuthology.orchestra.run.smithi107.stdout: "get_started": 0, 2024-03-08T21:15:52.257 INFO:teuthology.orchestra.run.smithi107.stdout: "get": 0, 2024-03-08T21:15:52.257 INFO:teuthology.orchestra.run.smithi107.stdout: "get_sum": 0, 2024-03-08T21:15:52.257 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_fail": 0, 2024-03-08T21:15:52.257 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_success": 0, 2024-03-08T21:15:52.257 INFO:teuthology.orchestra.run.smithi107.stdout: "take": 0, 2024-03-08T21:15:52.257 INFO:teuthology.orchestra.run.smithi107.stdout: "take_sum": 0, 2024-03-08T21:15:52.257 INFO:teuthology.orchestra.run.smithi107.stdout: "put": 0, 2024-03-08T21:15:52.257 INFO:teuthology.orchestra.run.smithi107.stdout: "put_sum": 0, 2024-03-08T21:15:52.257 INFO:teuthology.orchestra.run.smithi107.stdout: "wait": { 2024-03-08T21:15:52.257 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-03-08T21:15:52.257 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-03-08T21:15:52.257 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-03-08T21:15:52.257 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:52.257 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:52.258 INFO:teuthology.orchestra.run.smithi107.stdout: "throttle-write_buf_throttle-0x564123762a00": { 2024-03-08T21:15:52.258 INFO:teuthology.orchestra.run.smithi107.stdout: "val": 0, 2024-03-08T21:15:52.258 INFO:teuthology.orchestra.run.smithi107.stdout: "max": 3758096384, 2024-03-08T21:15:52.258 INFO:teuthology.orchestra.run.smithi107.stdout: "get_started": 0, 2024-03-08T21:15:52.258 INFO:teuthology.orchestra.run.smithi107.stdout: "get": 0, 2024-03-08T21:15:52.258 INFO:teuthology.orchestra.run.smithi107.stdout: "get_sum": 0, 2024-03-08T21:15:52.258 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_fail": 0, 2024-03-08T21:15:52.258 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_success": 0, 2024-03-08T21:15:52.258 INFO:teuthology.orchestra.run.smithi107.stdout: "take": 0, 2024-03-08T21:15:52.258 INFO:teuthology.orchestra.run.smithi107.stdout: "take_sum": 0, 2024-03-08T21:15:52.258 INFO:teuthology.orchestra.run.smithi107.stdout: "put": 0, 2024-03-08T21:15:52.258 INFO:teuthology.orchestra.run.smithi107.stdout: "put_sum": 0, 2024-03-08T21:15:52.258 INFO:teuthology.orchestra.run.smithi107.stdout: "wait": { 2024-03-08T21:15:52.258 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-03-08T21:15:52.259 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-03-08T21:15:52.259 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-03-08T21:15:52.259 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:52.259 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:52.259 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-03-08T21:15:52.889 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:15:52 smithi107 ceph-mon[31381]: pgmap v486: 97 pgs: 97 active+clean; 17 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 5.4 KiB/s wr, 10 op/s 2024-03-08T21:15:53.008 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:15:52 smithi149 ceph-mon[35042]: pgmap v486: 97 pgs: 97 active+clean; 17 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 5.4 KiB/s wr, 10 op/s 2024-03-08T21:15:53.121 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:15:52 smithi204 ceph-mon[34935]: pgmap v486: 97 pgs: 97 active+clean; 17 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 5.4 KiB/s wr, 10 op/s 2024-03-08T21:15:53.325 INFO:tasks.check_counter:Daemon mds.g mds.exported=0 2024-03-08T21:15:53.325 INFO:tasks.check_counter:Daemon mds.g mds.imported=0 2024-03-08T21:15:53.325 INFO:tasks.check_counter:Daemon mds.g mds_cache.dir_update=0 2024-03-08T21:15:53.325 INFO:tasks.check_counter:Daemon mds.g mds_cache.dir_update_receipt=0 2024-03-08T21:15:53.325 INFO:tasks.check_counter:Daemon mds.g mds.root_rsnaps=0 2024-03-08T21:15:53.325 INFO:tasks.check_counter:Daemon mds.g mds_server.req_mksnap_latency.avgcount=0 2024-03-08T21:15:53.325 INFO:tasks.check_counter:Daemon mds.g mds_server.req_rmsnap_latency.avgcount=0 2024-03-08T21:15:53.325 DEBUG:tasks.check_counter:Getting stats from j 2024-03-08T21:15:53.326 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761 shell --fsid dc20f0f8-dd8c-11ee-95c6-87774f69a715 -- ceph daemon mds.j perf dump 2024-03-08T21:15:53.763 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/mon.a/config 2024-03-08T21:15:54.960 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:15:54 smithi107 ceph-mon[31381]: pgmap v487: 97 pgs: 97 active+clean; 17 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 5.4 KiB/s wr, 10 op/s 2024-03-08T21:15:55.008 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:15:54 smithi149 ceph-mon[35042]: pgmap v487: 97 pgs: 97 active+clean; 17 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 5.4 KiB/s wr, 10 op/s 2024-03-08T21:15:55.121 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:15:54 smithi204 ceph-mon[34935]: pgmap v487: 97 pgs: 97 active+clean; 17 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 5.4 KiB/s wr, 10 op/s 2024-03-08T21:15:56.150 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-03-08T21:15:56.150 INFO:teuthology.orchestra.run.smithi107.stdout: "AsyncMessenger::Worker-0": { 2024-03-08T21:15:56.151 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_messages": 4, 2024-03-08T21:15:56.151 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages": 4, 2024-03-08T21:15:56.151 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_bytes": 3040, 2024-03-08T21:15:56.151 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_bytes": 448, 2024-03-08T21:15:56.151 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_created_connections": 4, 2024-03-08T21:15:56.151 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_active_connections": 0, 2024-03-08T21:15:56.151 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_total_time": 0.003955908, 2024-03-08T21:15:56.152 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_send_time": 0.000080108, 2024-03-08T21:15:56.152 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_recv_time": 0.001178976, 2024-03-08T21:15:56.152 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-08T21:15:56.152 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages_queue_lat": { 2024-03-08T21:15:56.152 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 3, 2024-03-08T21:15:56.153 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.001462226, 2024-03-08T21:15:56.153 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000487408 2024-03-08T21:15:56.153 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:56.153 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_handle_ack_lat": { 2024-03-08T21:15:56.153 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-03-08T21:15:56.153 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-03-08T21:15:56.154 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-03-08T21:15:56.154 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:56.154 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_encrypted_bytes": 3040, 2024-03-08T21:15:56.154 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_encrypted_bytes": 448 2024-03-08T21:15:56.154 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:56.154 INFO:teuthology.orchestra.run.smithi107.stdout: "AsyncMessenger::Worker-1": { 2024-03-08T21:15:56.155 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_messages": 312, 2024-03-08T21:15:56.155 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages": 302, 2024-03-08T21:15:56.155 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_bytes": 1463600, 2024-03-08T21:15:56.155 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_bytes": 181616, 2024-03-08T21:15:56.155 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_created_connections": 2, 2024-03-08T21:15:56.155 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_active_connections": 1, 2024-03-08T21:15:56.155 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_total_time": 0.130595006, 2024-03-08T21:15:56.155 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_send_time": 0.049889636, 2024-03-08T21:15:56.155 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_recv_time": 0.084004320, 2024-03-08T21:15:56.156 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_fast_dispatch_time": 0.015364479, 2024-03-08T21:15:56.156 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages_queue_lat": { 2024-03-08T21:15:56.156 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 301, 2024-03-08T21:15:56.156 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.025679780, 2024-03-08T21:15:56.156 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000085314 2024-03-08T21:15:56.156 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:56.156 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_handle_ack_lat": { 2024-03-08T21:15:56.156 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-03-08T21:15:56.156 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-03-08T21:15:56.156 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-03-08T21:15:56.156 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:56.156 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_encrypted_bytes": 1463600, 2024-03-08T21:15:56.156 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_encrypted_bytes": 181616 2024-03-08T21:15:56.156 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:56.156 INFO:teuthology.orchestra.run.smithi107.stdout: "AsyncMessenger::Worker-2": { 2024-03-08T21:15:56.157 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_messages": 3, 2024-03-08T21:15:56.157 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages": 231, 2024-03-08T21:15:56.157 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_bytes": 480, 2024-03-08T21:15:56.157 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_bytes": 273552, 2024-03-08T21:15:56.157 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_created_connections": 6, 2024-03-08T21:15:56.157 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_active_connections": 3, 2024-03-08T21:15:56.157 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_total_time": 0.048153622, 2024-03-08T21:15:56.157 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_send_time": 0.036132364, 2024-03-08T21:15:56.157 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_recv_time": 0.004229101, 2024-03-08T21:15:56.157 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-08T21:15:56.157 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages_queue_lat": { 2024-03-08T21:15:56.157 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 228, 2024-03-08T21:15:56.157 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.024736822, 2024-03-08T21:15:56.157 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000108494 2024-03-08T21:15:56.158 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:56.158 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_handle_ack_lat": { 2024-03-08T21:15:56.158 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-03-08T21:15:56.158 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-03-08T21:15:56.158 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-03-08T21:15:56.158 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:56.158 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_encrypted_bytes": 480, 2024-03-08T21:15:56.158 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_encrypted_bytes": 273552 2024-03-08T21:15:56.158 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:56.158 INFO:teuthology.orchestra.run.smithi107.stdout: "cct": { 2024-03-08T21:15:56.158 INFO:teuthology.orchestra.run.smithi107.stdout: "total_workers": 0, 2024-03-08T21:15:56.159 INFO:teuthology.orchestra.run.smithi107.stdout: "unhealthy_workers": 0 2024-03-08T21:15:56.159 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:56.159 INFO:teuthology.orchestra.run.smithi107.stdout: "mempool": { 2024-03-08T21:15:56.159 INFO:teuthology.orchestra.run.smithi107.stdout: "bloom_filter_bytes": 0, 2024-03-08T21:15:56.159 INFO:teuthology.orchestra.run.smithi107.stdout: "bloom_filter_items": 0, 2024-03-08T21:15:56.159 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_alloc_bytes": 0, 2024-03-08T21:15:56.159 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_alloc_items": 0, 2024-03-08T21:15:56.159 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_data_bytes": 0, 2024-03-08T21:15:56.159 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_data_items": 0, 2024-03-08T21:15:56.159 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-08T21:15:56.159 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_onode_items": 0, 2024-03-08T21:15:56.159 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-08T21:15:56.159 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_meta_items": 0, 2024-03-08T21:15:56.159 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_other_bytes": 0, 2024-03-08T21:15:56.160 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_other_items": 0, 2024-03-08T21:15:56.160 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-08T21:15:56.160 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_buffer_items": 0, 2024-03-08T21:15:56.160 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_extent_bytes": 0, 2024-03-08T21:15:56.160 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_extent_items": 0, 2024-03-08T21:15:56.160 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_blob_bytes": 0, 2024-03-08T21:15:56.160 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_blob_items": 0, 2024-03-08T21:15:56.160 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-08T21:15:56.160 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_shared_blob_items": 0, 2024-03-08T21:15:56.160 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-08T21:15:56.160 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_inline_bl_items": 0, 2024-03-08T21:15:56.160 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_fsck_bytes": 0, 2024-03-08T21:15:56.160 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_fsck_items": 0, 2024-03-08T21:15:56.160 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_txc_bytes": 0, 2024-03-08T21:15:56.160 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_txc_items": 0, 2024-03-08T21:15:56.160 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-08T21:15:56.161 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_writing_deferred_items": 0, 2024-03-08T21:15:56.161 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_writing_bytes": 0, 2024-03-08T21:15:56.161 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_writing_items": 0, 2024-03-08T21:15:56.161 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_bytes": 0, 2024-03-08T21:15:56.161 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_items": 0, 2024-03-08T21:15:56.161 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_file_reader_bytes": 0, 2024-03-08T21:15:56.161 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_file_reader_items": 0, 2024-03-08T21:15:56.161 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_file_writer_bytes": 0, 2024-03-08T21:15:56.161 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_file_writer_items": 0, 2024-03-08T21:15:56.161 INFO:teuthology.orchestra.run.smithi107.stdout: "buffer_anon_bytes": 147930, 2024-03-08T21:15:56.161 INFO:teuthology.orchestra.run.smithi107.stdout: "buffer_anon_items": 28, 2024-03-08T21:15:56.161 INFO:teuthology.orchestra.run.smithi107.stdout: "buffer_meta_bytes": 0, 2024-03-08T21:15:56.161 INFO:teuthology.orchestra.run.smithi107.stdout: "buffer_meta_items": 0, 2024-03-08T21:15:56.161 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_bytes": 0, 2024-03-08T21:15:56.161 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_items": 0, 2024-03-08T21:15:56.162 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_mapbl_bytes": 0, 2024-03-08T21:15:56.162 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_mapbl_items": 0, 2024-03-08T21:15:56.162 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_pglog_bytes": 0, 2024-03-08T21:15:56.162 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_pglog_items": 0, 2024-03-08T21:15:56.162 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_bytes": 0, 2024-03-08T21:15:56.162 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_items": 0, 2024-03-08T21:15:56.162 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_mapping_bytes": 0, 2024-03-08T21:15:56.162 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_mapping_items": 0, 2024-03-08T21:15:56.162 INFO:teuthology.orchestra.run.smithi107.stdout: "pgmap_bytes": 0, 2024-03-08T21:15:56.162 INFO:teuthology.orchestra.run.smithi107.stdout: "pgmap_items": 0, 2024-03-08T21:15:56.162 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_co_bytes": 928, 2024-03-08T21:15:56.162 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_co_items": 2, 2024-03-08T21:15:56.162 INFO:teuthology.orchestra.run.smithi107.stdout: "unittest_1_bytes": 0, 2024-03-08T21:15:56.162 INFO:teuthology.orchestra.run.smithi107.stdout: "unittest_1_items": 0, 2024-03-08T21:15:56.163 INFO:teuthology.orchestra.run.smithi107.stdout: "unittest_2_bytes": 0, 2024-03-08T21:15:56.163 INFO:teuthology.orchestra.run.smithi107.stdout: "unittest_2_items": 0 2024-03-08T21:15:56.163 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-03-08T21:15:56.163 INFO:teuthology.orchestra.run.smithi107.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-08T21:15:56.163 INFO:teuthology.orchestra.run.smithi107.stdout: "val": 0, 2024-03-08T21:15:56.163 INFO:teuthology.orchestra.run.smithi107.stdout: "max": 104857600, 2024-03-08T21:15:56.163 INFO:teuthology.orchestra.run.smithi107.stdout: "get_started": 0, 2024-03-08T21:15:56.163 INFO:teuthology.orchestra.run.smithi107.stdout: "get": 315, 2024-03-08T21:15:56.163 INFO:teuthology.orchestra.run.smithi107.stdout: "get_sum": 1419595, 2024-03-08T21:15:56.163 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_fail": 0, 2024-03-08T21:15:56.163 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_success": 315, 2024-03-08T21:15:56.163 INFO:teuthology.orchestra.run.smithi107.stdout: "take": 0, 2024-03-08T21:15:56.163 INFO:teuthology.orchestra.run.smithi107.stdout: "take_sum": 0, 2024-03-08T21:15:56.163 INFO:teuthology.orchestra.run.smithi107.stdout: "put": 315, 2024-03-08T21:15:56.163 INFO:teuthology.orchestra.run.smithi107.stdout: "put_sum": 1419595, 2024-03-08T21:15:56.163 INFO:teuthology.orchestra.run.smithi107.stdout: "wait": { 2024-03-08T21:15:56.164 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-03-08T21:15:56.164 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-03-08T21:15:56.164 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-03-08T21:15:56.164 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:56.164 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-03-08T21:15:56.164 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-03-08T21:15:56.932 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:15:56 smithi107 ceph-mon[31381]: pgmap v488: 97 pgs: 97 active+clean; 17 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.2 KiB/s wr, 14 op/s 2024-03-08T21:15:57.000 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-03-08T21:15:57.001 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-03-08T21:15:57.001 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.j 2024-03-08T21:15:57.001 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.j 2024-03-08T21:15:57.001 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.j 2024-03-08T21:15:57.001 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.j 2024-03-08T21:15:57.001 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.j 2024-03-08T21:15:57.001 DEBUG:tasks.check_counter:Getting stats from b 2024-03-08T21:15:57.001 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761 shell --fsid dc20f0f8-dd8c-11ee-95c6-87774f69a715 -- ceph daemon mds.b perf dump 2024-03-08T21:15:57.010 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:15:56 smithi149 ceph-mon[35042]: pgmap v488: 97 pgs: 97 active+clean; 17 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.2 KiB/s wr, 14 op/s 2024-03-08T21:15:57.121 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:15:56 smithi204 ceph-mon[34935]: pgmap v488: 97 pgs: 97 active+clean; 17 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.2 KiB/s wr, 14 op/s 2024-03-08T21:15:57.459 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/mon.b/config 2024-03-08T21:15:58.758 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:15:58 smithi149 ceph-mon[35042]: pgmap v489: 97 pgs: 97 active+clean; 17 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-03-08T21:15:58.960 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:15:58 smithi107 ceph-mon[31381]: pgmap v489: 97 pgs: 97 active+clean; 17 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-03-08T21:15:59.121 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:15:58 smithi204 ceph-mon[34935]: pgmap v489: 97 pgs: 97 active+clean; 17 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-03-08T21:15:59.838 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-03-08T21:15:59.838 INFO:teuthology.orchestra.run.smithi149.stdout: "AsyncMessenger::Worker-0": { 2024-03-08T21:15:59.838 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_messages": 4198, 2024-03-08T21:15:59.839 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages": 4729, 2024-03-08T21:15:59.839 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_bytes": 1416034, 2024-03-08T21:15:59.839 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_bytes": 14835678, 2024-03-08T21:15:59.839 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_created_connections": 11, 2024-03-08T21:15:59.839 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_active_connections": 4, 2024-03-08T21:15:59.839 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_total_time": 1.175518718, 2024-03-08T21:15:59.839 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_send_time": 0.542251809, 2024-03-08T21:15:59.839 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_recv_time": 0.370560238, 2024-03-08T21:15:59.839 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_fast_dispatch_time": 0.194355776, 2024-03-08T21:15:59.839 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages_queue_lat": { 2024-03-08T21:15:59.839 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 4732, 2024-03-08T21:15:59.839 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 1.005345829, 2024-03-08T21:15:59.840 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000212456 2024-03-08T21:15:59.840 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:15:59.840 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_handle_ack_lat": { 2024-03-08T21:15:59.840 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 4260, 2024-03-08T21:15:59.840 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.002286085, 2024-03-08T21:15:59.840 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000536 2024-03-08T21:15:59.840 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:15:59.840 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_encrypted_bytes": 160, 2024-03-08T21:15:59.840 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_encrypted_bytes": 388752 2024-03-08T21:15:59.840 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:15:59.840 INFO:teuthology.orchestra.run.smithi149.stdout: "AsyncMessenger::Worker-1": { 2024-03-08T21:15:59.840 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_messages": 5712, 2024-03-08T21:15:59.840 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages": 5174, 2024-03-08T21:15:59.840 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_bytes": 3667466, 2024-03-08T21:15:59.841 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_bytes": 14656284, 2024-03-08T21:15:59.841 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_created_connections": 11, 2024-03-08T21:15:59.841 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_active_connections": 6, 2024-03-08T21:15:59.841 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_total_time": 1.382726068, 2024-03-08T21:15:59.841 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_send_time": 0.555379900, 2024-03-08T21:15:59.841 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_recv_time": 0.561770777, 2024-03-08T21:15:59.841 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_fast_dispatch_time": 0.284989564, 2024-03-08T21:15:59.841 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages_queue_lat": { 2024-03-08T21:15:59.841 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 5178, 2024-03-08T21:15:59.841 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 1.084505697, 2024-03-08T21:15:59.841 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000209444 2024-03-08T21:15:59.841 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:15:59.841 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_handle_ack_lat": { 2024-03-08T21:15:59.841 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 5571, 2024-03-08T21:15:59.841 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.002499610, 2024-03-08T21:15:59.842 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000448 2024-03-08T21:15:59.842 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:15:59.842 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_encrypted_bytes": 2092176, 2024-03-08T21:15:59.842 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_encrypted_bytes": 335552 2024-03-08T21:15:59.842 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:15:59.842 INFO:teuthology.orchestra.run.smithi149.stdout: "AsyncMessenger::Worker-2": { 2024-03-08T21:15:59.842 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_messages": 9641, 2024-03-08T21:15:59.842 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages": 13686, 2024-03-08T21:15:59.842 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_bytes": 3311575, 2024-03-08T21:15:59.842 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_bytes": 15478179, 2024-03-08T21:15:59.842 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_created_connections": 153, 2024-03-08T21:15:59.842 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_active_connections": 13, 2024-03-08T21:15:59.842 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_total_time": 2.555510953, 2024-03-08T21:15:59.843 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_send_time": 1.026057486, 2024-03-08T21:15:59.843 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_recv_time": 1.375911319, 2024-03-08T21:15:59.843 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_fast_dispatch_time": 0.212819809, 2024-03-08T21:15:59.843 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages_queue_lat": { 2024-03-08T21:15:59.843 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 13673, 2024-03-08T21:15:59.843 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 1.231833297, 2024-03-08T21:15:59.843 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000090092 2024-03-08T21:15:59.843 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:15:59.843 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_handle_ack_lat": { 2024-03-08T21:15:59.843 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 18129, 2024-03-08T21:15:59.843 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.010680173, 2024-03-08T21:15:59.843 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000589 2024-03-08T21:15:59.843 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:15:59.843 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_encrypted_bytes": 144192, 2024-03-08T21:15:59.843 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_encrypted_bytes": 150336 2024-03-08T21:15:59.844 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:15:59.844 INFO:teuthology.orchestra.run.smithi149.stdout: "cct": { 2024-03-08T21:15:59.844 INFO:teuthology.orchestra.run.smithi149.stdout: "total_workers": 1, 2024-03-08T21:15:59.844 INFO:teuthology.orchestra.run.smithi149.stdout: "unhealthy_workers": 0 2024-03-08T21:15:59.844 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:15:59.844 INFO:teuthology.orchestra.run.smithi149.stdout: "finisher-MDSRank": { 2024-03-08T21:15:59.844 INFO:teuthology.orchestra.run.smithi149.stdout: "queue_len": 0, 2024-03-08T21:15:59.844 INFO:teuthology.orchestra.run.smithi149.stdout: "complete_latency": { 2024-03-08T21:15:59.844 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 3628, 2024-03-08T21:15:59.844 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 6.451917518, 2024-03-08T21:15:59.844 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.001778367 2024-03-08T21:15:59.844 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-03-08T21:15:59.844 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:15:59.844 INFO:teuthology.orchestra.run.smithi149.stdout: "finisher-PurgeQueue": { 2024-03-08T21:15:59.844 INFO:teuthology.orchestra.run.smithi149.stdout: "queue_len": 0, 2024-03-08T21:15:59.845 INFO:teuthology.orchestra.run.smithi149.stdout: "complete_latency": { 2024-03-08T21:15:59.845 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 5482, 2024-03-08T21:15:59.845 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.554653754, 2024-03-08T21:15:59.845 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000101177 2024-03-08T21:15:59.845 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-03-08T21:15:59.845 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:15:59.845 INFO:teuthology.orchestra.run.smithi149.stdout: "mds": { 2024-03-08T21:15:59.845 INFO:teuthology.orchestra.run.smithi149.stdout: "request": 2665, 2024-03-08T21:15:59.845 INFO:teuthology.orchestra.run.smithi149.stdout: "reply": 2665, 2024-03-08T21:15:59.845 INFO:teuthology.orchestra.run.smithi149.stdout: "reply_latency": { 2024-03-08T21:15:59.845 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 2665, 2024-03-08T21:15:59.845 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 8.085934635, 2024-03-08T21:15:59.845 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.003034121 2024-03-08T21:15:59.845 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:15:59.845 INFO:teuthology.orchestra.run.smithi149.stdout: "slow_reply": 0, 2024-03-08T21:15:59.846 INFO:teuthology.orchestra.run.smithi149.stdout: "forward": 0, 2024-03-08T21:15:59.846 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_fetch_complete": 0, 2024-03-08T21:15:59.846 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_fetch_keys": 0, 2024-03-08T21:15:59.846 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_commit": 189, 2024-03-08T21:15:59.846 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_split": 0, 2024-03-08T21:15:59.846 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_merge": 0, 2024-03-08T21:15:59.846 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes": 104, 2024-03-08T21:15:59.846 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes_top": 17, 2024-03-08T21:15:59.846 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes_bottom": 87, 2024-03-08T21:15:59.846 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes_pin_tail": 0, 2024-03-08T21:15:59.846 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes_pinned": 79, 2024-03-08T21:15:59.846 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes_expired": 0, 2024-03-08T21:15:59.846 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes_with_caps": 3, 2024-03-08T21:15:59.846 INFO:teuthology.orchestra.run.smithi149.stdout: "caps": 3, 2024-03-08T21:15:59.847 INFO:teuthology.orchestra.run.smithi149.stdout: "subtrees": 2, 2024-03-08T21:15:59.847 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse": 2741, 2024-03-08T21:15:59.847 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse_hit": 2720, 2024-03-08T21:15:59.847 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse_forward": 0, 2024-03-08T21:15:59.847 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse_discover": 0, 2024-03-08T21:15:59.847 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse_dir_fetch": 0, 2024-03-08T21:15:59.847 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse_remote_ino": 0, 2024-03-08T21:15:59.847 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse_lock": 0, 2024-03-08T21:15:59.847 INFO:teuthology.orchestra.run.smithi149.stdout: "load_cent": 0, 2024-03-08T21:15:59.847 INFO:teuthology.orchestra.run.smithi149.stdout: "q": 0, 2024-03-08T21:15:59.847 INFO:teuthology.orchestra.run.smithi149.stdout: "exported": 0, 2024-03-08T21:15:59.847 INFO:teuthology.orchestra.run.smithi149.stdout: "exported_inodes": 0, 2024-03-08T21:15:59.847 INFO:teuthology.orchestra.run.smithi149.stdout: "imported": 0, 2024-03-08T21:15:59.847 INFO:teuthology.orchestra.run.smithi149.stdout: "imported_inodes": 0, 2024-03-08T21:15:59.847 INFO:teuthology.orchestra.run.smithi149.stdout: "openino_dir_fetch": 0, 2024-03-08T21:15:59.848 INFO:teuthology.orchestra.run.smithi149.stdout: "openino_backtrace_fetch": 0, 2024-03-08T21:15:59.848 INFO:teuthology.orchestra.run.smithi149.stdout: "openino_peer_discover": 0, 2024-03-08T21:15:59.848 INFO:teuthology.orchestra.run.smithi149.stdout: "root_rfiles": 4, 2024-03-08T21:15:59.848 INFO:teuthology.orchestra.run.smithi149.stdout: "root_rbytes": 234, 2024-03-08T21:15:59.848 INFO:teuthology.orchestra.run.smithi149.stdout: "root_rsnaps": 6, 2024-03-08T21:15:59.848 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_backtrace_fetch": 0, 2024-03-08T21:15:59.848 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_set_tag": 0, 2024-03-08T21:15:59.848 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_backtrace_repaired": 0, 2024-03-08T21:15:59.848 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_inotable_repaired": 0, 2024-03-08T21:15:59.848 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_dir_inodes": 0, 2024-03-08T21:15:59.848 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_dir_base_inodes": 0, 2024-03-08T21:15:59.848 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_dirfrag_rstats": 0, 2024-03-08T21:15:59.848 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_file_inodes": 0, 2024-03-08T21:15:59.848 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_inode_file_caps": 0, 2024-03-08T21:15:59.848 INFO:teuthology.orchestra.run.smithi149.stdout: "ceph_cap_op_revoke": 1302, 2024-03-08T21:15:59.849 INFO:teuthology.orchestra.run.smithi149.stdout: "ceph_cap_op_grant": 3597, 2024-03-08T21:15:59.849 INFO:teuthology.orchestra.run.smithi149.stdout: "ceph_cap_op_trunc": 0, 2024-03-08T21:15:59.849 INFO:teuthology.orchestra.run.smithi149.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-08T21:15:59.849 INFO:teuthology.orchestra.run.smithi149.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-08T21:15:59.849 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_client_caps": 3934, 2024-03-08T21:15:59.849 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_client_caps_dirty": 1789, 2024-03-08T21:15:59.849 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_client_cap_release": 42, 2024-03-08T21:15:59.849 INFO:teuthology.orchestra.run.smithi149.stdout: "process_request_cap_release": 3653 2024-03-08T21:15:59.849 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:15:59.849 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_cache": { 2024-03-08T21:15:59.849 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_update": 27, 2024-03-08T21:15:59.849 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_update_receipt": 0, 2024-03-08T21:15:59.849 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_try_discover": 0, 2024-03-08T21:15:59.849 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_send_discover": 0, 2024-03-08T21:15:59.849 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_handle_discover": 46, 2024-03-08T21:15:59.850 INFO:teuthology.orchestra.run.smithi149.stdout: "num_strays": 3, 2024-03-08T21:15:59.850 INFO:teuthology.orchestra.run.smithi149.stdout: "num_strays_delayed": 0, 2024-03-08T21:15:59.850 INFO:teuthology.orchestra.run.smithi149.stdout: "num_strays_enqueuing": 0, 2024-03-08T21:15:59.850 INFO:teuthology.orchestra.run.smithi149.stdout: "strays_created": 1212, 2024-03-08T21:15:59.850 INFO:teuthology.orchestra.run.smithi149.stdout: "strays_enqueued": 1209, 2024-03-08T21:15:59.850 INFO:teuthology.orchestra.run.smithi149.stdout: "strays_reintegrated": 0, 2024-03-08T21:15:59.850 INFO:teuthology.orchestra.run.smithi149.stdout: "strays_migrated": 0, 2024-03-08T21:15:59.850 INFO:teuthology.orchestra.run.smithi149.stdout: "num_recovering_processing": 0, 2024-03-08T21:15:59.850 INFO:teuthology.orchestra.run.smithi149.stdout: "num_recovering_enqueued": 0, 2024-03-08T21:15:59.850 INFO:teuthology.orchestra.run.smithi149.stdout: "num_recovering_prioritized": 0, 2024-03-08T21:15:59.850 INFO:teuthology.orchestra.run.smithi149.stdout: "recovery_started": 0, 2024-03-08T21:15:59.850 INFO:teuthology.orchestra.run.smithi149.stdout: "recovery_completed": 0, 2024-03-08T21:15:59.850 INFO:teuthology.orchestra.run.smithi149.stdout: "ireq_enqueue_scrub": 0, 2024-03-08T21:15:59.851 INFO:teuthology.orchestra.run.smithi149.stdout: "ireq_exportdir": 0, 2024-03-08T21:15:59.851 INFO:teuthology.orchestra.run.smithi149.stdout: "ireq_flush": 0, 2024-03-08T21:15:59.851 INFO:teuthology.orchestra.run.smithi149.stdout: "ireq_fragmentdir": 0, 2024-03-08T21:15:59.851 INFO:teuthology.orchestra.run.smithi149.stdout: "ireq_fragstats": 0, 2024-03-08T21:15:59.851 INFO:teuthology.orchestra.run.smithi149.stdout: "ireq_inodestats": 0 2024-03-08T21:15:59.851 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:15:59.851 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_log": { 2024-03-08T21:15:59.851 INFO:teuthology.orchestra.run.smithi149.stdout: "evlrg": 0, 2024-03-08T21:15:59.851 INFO:teuthology.orchestra.run.smithi149.stdout: "evadd": 6669, 2024-03-08T21:15:59.851 INFO:teuthology.orchestra.run.smithi149.stdout: "evex": 5565, 2024-03-08T21:15:59.851 INFO:teuthology.orchestra.run.smithi149.stdout: "evtrm": 5565, 2024-03-08T21:15:59.851 INFO:teuthology.orchestra.run.smithi149.stdout: "ev": 1104, 2024-03-08T21:15:59.851 INFO:teuthology.orchestra.run.smithi149.stdout: "evexg": 0, 2024-03-08T21:15:59.851 INFO:teuthology.orchestra.run.smithi149.stdout: "evexd": 2409, 2024-03-08T21:15:59.851 INFO:teuthology.orchestra.run.smithi149.stdout: "segadd": 76, 2024-03-08T21:15:59.852 INFO:teuthology.orchestra.run.smithi149.stdout: "segex": 74, 2024-03-08T21:15:59.852 INFO:teuthology.orchestra.run.smithi149.stdout: "segtrm": 74, 2024-03-08T21:15:59.852 INFO:teuthology.orchestra.run.smithi149.stdout: "seg": 2, 2024-03-08T21:15:59.852 INFO:teuthology.orchestra.run.smithi149.stdout: "segmjr": 1, 2024-03-08T21:15:59.852 INFO:teuthology.orchestra.run.smithi149.stdout: "segexg": 0, 2024-03-08T21:15:59.852 INFO:teuthology.orchestra.run.smithi149.stdout: "segexd": 4, 2024-03-08T21:15:59.852 INFO:teuthology.orchestra.run.smithi149.stdout: "expos": 32972229, 2024-03-08T21:15:59.852 INFO:teuthology.orchestra.run.smithi149.stdout: "wrpos": 37664542, 2024-03-08T21:15:59.852 INFO:teuthology.orchestra.run.smithi149.stdout: "rdpos": 0, 2024-03-08T21:15:59.852 INFO:teuthology.orchestra.run.smithi149.stdout: "jlat": { 2024-03-08T21:15:59.852 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 1445, 2024-03-08T21:15:59.852 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 7.256823532, 2024-03-08T21:15:59.852 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.005022023 2024-03-08T21:15:59.852 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:15:59.852 INFO:teuthology.orchestra.run.smithi149.stdout: "replayed": 0 2024-03-08T21:15:59.853 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:15:59.853 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_mem": { 2024-03-08T21:15:59.853 INFO:teuthology.orchestra.run.smithi149.stdout: "ino": 40, 2024-03-08T21:15:59.853 INFO:teuthology.orchestra.run.smithi149.stdout: "ino+": 1249, 2024-03-08T21:15:59.853 INFO:teuthology.orchestra.run.smithi149.stdout: "ino-": 1209, 2024-03-08T21:15:59.853 INFO:teuthology.orchestra.run.smithi149.stdout: "dir": 21, 2024-03-08T21:15:59.853 INFO:teuthology.orchestra.run.smithi149.stdout: "dir+": 24, 2024-03-08T21:15:59.853 INFO:teuthology.orchestra.run.smithi149.stdout: "dir-": 3, 2024-03-08T21:15:59.853 INFO:teuthology.orchestra.run.smithi149.stdout: "dn": 104, 2024-03-08T21:15:59.853 INFO:teuthology.orchestra.run.smithi149.stdout: "dn+": 1261, 2024-03-08T21:15:59.853 INFO:teuthology.orchestra.run.smithi149.stdout: "dn-": 1157, 2024-03-08T21:15:59.853 INFO:teuthology.orchestra.run.smithi149.stdout: "cap": 3, 2024-03-08T21:15:59.853 INFO:teuthology.orchestra.run.smithi149.stdout: "cap+": 1233, 2024-03-08T21:15:59.853 INFO:teuthology.orchestra.run.smithi149.stdout: "cap-": 1230, 2024-03-08T21:15:59.853 INFO:teuthology.orchestra.run.smithi149.stdout: "rss": 58940, 2024-03-08T21:15:59.853 INFO:teuthology.orchestra.run.smithi149.stdout: "heap": 215420 2024-03-08T21:15:59.854 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:15:59.854 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_server": { 2024-03-08T21:15:59.854 INFO:teuthology.orchestra.run.smithi149.stdout: "dispatch_client_request": 3119, 2024-03-08T21:15:59.854 INFO:teuthology.orchestra.run.smithi149.stdout: "dispatch_server_request": 1, 2024-03-08T21:15:59.854 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_client_request": 2665, 2024-03-08T21:15:59.854 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_client_session": 41, 2024-03-08T21:15:59.854 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_peer_request": 2, 2024-03-08T21:15:59.854 INFO:teuthology.orchestra.run.smithi149.stdout: "req_create_latency": { 2024-03-08T21:15:59.854 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 1210, 2024-03-08T21:15:59.854 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 2.336366931, 2024-03-08T21:15:59.854 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.001930881 2024-03-08T21:15:59.854 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:15:59.854 INFO:teuthology.orchestra.run.smithi149.stdout: "req_getattr_latency": { 2024-03-08T21:15:59.854 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 169, 2024-03-08T21:15:59.855 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.091567438, 2024-03-08T21:15:59.855 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000541819 2024-03-08T21:15:59.855 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:15:59.855 INFO:teuthology.orchestra.run.smithi149.stdout: "req_getfilelock_latency": { 2024-03-08T21:15:59.855 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-08T21:15:59.855 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-08T21:15:59.855 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-08T21:15:59.855 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:15:59.855 INFO:teuthology.orchestra.run.smithi149.stdout: "req_link_latency": { 2024-03-08T21:15:59.855 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-08T21:15:59.855 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-08T21:15:59.855 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-08T21:15:59.855 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:15:59.855 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lookup_latency": { 2024-03-08T21:15:59.855 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 6, 2024-03-08T21:15:59.856 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.007085316, 2024-03-08T21:15:59.856 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.001180886 2024-03-08T21:15:59.856 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:15:59.856 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lookuphash_latency": { 2024-03-08T21:15:59.856 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-08T21:15:59.856 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-08T21:15:59.856 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-08T21:15:59.856 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:15:59.856 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lookupino_latency": { 2024-03-08T21:15:59.856 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-08T21:15:59.856 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-08T21:15:59.856 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-08T21:15:59.856 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:15:59.856 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lookupname_latency": { 2024-03-08T21:15:59.856 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-08T21:15:59.856 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-08T21:15:59.857 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-08T21:15:59.857 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:15:59.857 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lookupparent_latency": { 2024-03-08T21:15:59.857 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-08T21:15:59.857 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-08T21:15:59.857 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-08T21:15:59.857 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:15:59.857 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lookupsnap_latency": { 2024-03-08T21:15:59.857 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-08T21:15:59.857 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-08T21:15:59.857 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-08T21:15:59.857 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:15:59.857 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lssnap_latency": { 2024-03-08T21:15:59.857 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 15, 2024-03-08T21:15:59.857 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.007419563, 2024-03-08T21:15:59.857 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000494637 2024-03-08T21:15:59.858 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:15:59.858 INFO:teuthology.orchestra.run.smithi149.stdout: "req_mkdir_latency": { 2024-03-08T21:15:59.858 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 12, 2024-03-08T21:15:59.858 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.048078337, 2024-03-08T21:15:59.858 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.004006528 2024-03-08T21:15:59.858 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:15:59.859 INFO:teuthology.orchestra.run.smithi149.stdout: "req_mknod_latency": { 2024-03-08T21:15:59.859 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-08T21:15:59.859 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-08T21:15:59.859 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-08T21:15:59.859 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:15:59.859 INFO:teuthology.orchestra.run.smithi149.stdout: "req_mksnap_latency": { 2024-03-08T21:15:59.859 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 15, 2024-03-08T21:15:59.859 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.212198817, 2024-03-08T21:15:59.859 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.014146587 2024-03-08T21:15:59.859 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:15:59.859 INFO:teuthology.orchestra.run.smithi149.stdout: "req_open_latency": { 2024-03-08T21:15:59.859 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-08T21:15:59.859 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-08T21:15:59.859 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-08T21:15:59.859 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:15:59.860 INFO:teuthology.orchestra.run.smithi149.stdout: "req_readdir_latency": { 2024-03-08T21:15:59.860 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 6, 2024-03-08T21:15:59.860 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.043797957, 2024-03-08T21:15:59.860 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.007299659 2024-03-08T21:15:59.860 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:15:59.860 INFO:teuthology.orchestra.run.smithi149.stdout: "req_rename_latency": { 2024-03-08T21:15:59.860 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 2, 2024-03-08T21:15:59.860 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.008051261, 2024-03-08T21:15:59.860 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.004025630 2024-03-08T21:15:59.860 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:15:59.860 INFO:teuthology.orchestra.run.smithi149.stdout: "req_renamesnap_latency": { 2024-03-08T21:15:59.860 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-08T21:15:59.860 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-08T21:15:59.860 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-08T21:15:59.860 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:15:59.861 INFO:teuthology.orchestra.run.smithi149.stdout: "req_snapdiff_latency": { 2024-03-08T21:15:59.861 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-08T21:15:59.861 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-08T21:15:59.861 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-08T21:15:59.861 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:15:59.861 INFO:teuthology.orchestra.run.smithi149.stdout: "req_rmdir_latency": { 2024-03-08T21:15:59.861 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 6, 2024-03-08T21:15:59.861 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.123665079, 2024-03-08T21:15:59.861 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.020610846 2024-03-08T21:15:59.861 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:15:59.861 INFO:teuthology.orchestra.run.smithi149.stdout: "req_rmsnap_latency": { 2024-03-08T21:15:59.861 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 9, 2024-03-08T21:15:59.861 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.139487890, 2024-03-08T21:15:59.861 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.015498654 2024-03-08T21:15:59.862 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:15:59.862 INFO:teuthology.orchestra.run.smithi149.stdout: "req_rmxattr_latency": { 2024-03-08T21:15:59.862 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-08T21:15:59.862 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-08T21:15:59.862 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-08T21:15:59.862 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:15:59.862 INFO:teuthology.orchestra.run.smithi149.stdout: "req_setattr_latency": { 2024-03-08T21:15:59.862 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 1, 2024-03-08T21:15:59.862 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.002284577, 2024-03-08T21:15:59.862 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.002284577 2024-03-08T21:15:59.862 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:15:59.862 INFO:teuthology.orchestra.run.smithi149.stdout: "req_setdirlayout_latency": { 2024-03-08T21:15:59.862 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-08T21:15:59.862 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-08T21:15:59.863 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-08T21:15:59.863 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:15:59.863 INFO:teuthology.orchestra.run.smithi149.stdout: "req_setfilelock_latency": { 2024-03-08T21:15:59.863 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-08T21:15:59.863 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-08T21:15:59.863 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-08T21:15:59.863 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:15:59.863 INFO:teuthology.orchestra.run.smithi149.stdout: "req_setlayout_latency": { 2024-03-08T21:15:59.863 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-08T21:15:59.863 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-08T21:15:59.863 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-08T21:15:59.863 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:15:59.863 INFO:teuthology.orchestra.run.smithi149.stdout: "req_setxattr_latency": { 2024-03-08T21:15:59.863 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 8, 2024-03-08T21:15:59.863 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.055824825, 2024-03-08T21:15:59.863 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.006978103 2024-03-08T21:15:59.864 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:15:59.864 INFO:teuthology.orchestra.run.smithi149.stdout: "req_symlink_latency": { 2024-03-08T21:15:59.864 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-08T21:15:59.864 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-08T21:15:59.864 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-08T21:15:59.864 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:15:59.864 INFO:teuthology.orchestra.run.smithi149.stdout: "req_unlink_latency": { 2024-03-08T21:15:59.864 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 1206, 2024-03-08T21:15:59.864 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 5.010106644, 2024-03-08T21:15:59.864 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.004154317 2024-03-08T21:15:59.864 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:15:59.864 INFO:teuthology.orchestra.run.smithi149.stdout: "cap_revoke_eviction": 0, 2024-03-08T21:15:59.864 INFO:teuthology.orchestra.run.smithi149.stdout: "cap_acquisition_throttle": 0, 2024-03-08T21:15:59.864 INFO:teuthology.orchestra.run.smithi149.stdout: "req_getvxattr_latency": { 2024-03-08T21:15:59.864 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-08T21:15:59.865 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-08T21:15:59.865 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-08T21:15:59.865 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-03-08T21:15:59.865 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:15:59.865 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_sessions": { 2024-03-08T21:15:59.865 INFO:teuthology.orchestra.run.smithi149.stdout: "session_count": 2, 2024-03-08T21:15:59.865 INFO:teuthology.orchestra.run.smithi149.stdout: "session_add": 6, 2024-03-08T21:15:59.865 INFO:teuthology.orchestra.run.smithi149.stdout: "session_remove": 4, 2024-03-08T21:15:59.865 INFO:teuthology.orchestra.run.smithi149.stdout: "sessions_open": 2, 2024-03-08T21:15:59.865 INFO:teuthology.orchestra.run.smithi149.stdout: "sessions_stale": 0, 2024-03-08T21:15:59.865 INFO:teuthology.orchestra.run.smithi149.stdout: "total_load": 1968, 2024-03-08T21:15:59.865 INFO:teuthology.orchestra.run.smithi149.stdout: "average_load": 984, 2024-03-08T21:15:59.865 INFO:teuthology.orchestra.run.smithi149.stdout: "avg_session_uptime": 919, 2024-03-08T21:15:59.865 INFO:teuthology.orchestra.run.smithi149.stdout: "mdthresh_evicted": 0 2024-03-08T21:15:59.866 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:15:59.866 INFO:teuthology.orchestra.run.smithi149.stdout: "mempool": { 2024-03-08T21:15:59.866 INFO:teuthology.orchestra.run.smithi149.stdout: "bloom_filter_bytes": 0, 2024-03-08T21:15:59.866 INFO:teuthology.orchestra.run.smithi149.stdout: "bloom_filter_items": 0, 2024-03-08T21:15:59.866 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_alloc_bytes": 0, 2024-03-08T21:15:59.866 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_alloc_items": 0, 2024-03-08T21:15:59.866 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_data_bytes": 0, 2024-03-08T21:15:59.866 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_data_items": 0, 2024-03-08T21:15:59.866 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-08T21:15:59.866 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_onode_items": 0, 2024-03-08T21:15:59.866 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-08T21:15:59.866 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_meta_items": 0, 2024-03-08T21:15:59.866 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_other_bytes": 0, 2024-03-08T21:15:59.867 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_other_items": 0, 2024-03-08T21:15:59.867 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-08T21:15:59.867 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_buffer_items": 0, 2024-03-08T21:15:59.867 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_extent_bytes": 0, 2024-03-08T21:15:59.867 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_extent_items": 0, 2024-03-08T21:15:59.867 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_blob_bytes": 0, 2024-03-08T21:15:59.867 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_blob_items": 0, 2024-03-08T21:15:59.867 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-08T21:15:59.867 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_shared_blob_items": 0, 2024-03-08T21:15:59.867 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-08T21:15:59.867 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_inline_bl_items": 0, 2024-03-08T21:15:59.867 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_fsck_bytes": 0, 2024-03-08T21:15:59.867 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_fsck_items": 0, 2024-03-08T21:15:59.867 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_txc_bytes": 0, 2024-03-08T21:15:59.868 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_txc_items": 0, 2024-03-08T21:15:59.868 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-08T21:15:59.868 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_writing_deferred_items": 0, 2024-03-08T21:15:59.868 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_writing_bytes": 0, 2024-03-08T21:15:59.868 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_writing_items": 0, 2024-03-08T21:15:59.868 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_bytes": 0, 2024-03-08T21:15:59.868 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_items": 0, 2024-03-08T21:15:59.868 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_file_reader_bytes": 0, 2024-03-08T21:15:59.868 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_file_reader_items": 0, 2024-03-08T21:15:59.868 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_file_writer_bytes": 0, 2024-03-08T21:15:59.868 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_file_writer_items": 0, 2024-03-08T21:15:59.868 INFO:teuthology.orchestra.run.smithi149.stdout: "buffer_anon_bytes": 332816, 2024-03-08T21:15:59.868 INFO:teuthology.orchestra.run.smithi149.stdout: "buffer_anon_items": 134, 2024-03-08T21:15:59.868 INFO:teuthology.orchestra.run.smithi149.stdout: "buffer_meta_bytes": 0, 2024-03-08T21:15:59.868 INFO:teuthology.orchestra.run.smithi149.stdout: "buffer_meta_items": 0, 2024-03-08T21:15:59.869 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_bytes": 0, 2024-03-08T21:15:59.869 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_items": 0, 2024-03-08T21:15:59.869 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_mapbl_bytes": 0, 2024-03-08T21:15:59.869 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_mapbl_items": 0, 2024-03-08T21:15:59.869 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_pglog_bytes": 0, 2024-03-08T21:15:59.869 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_pglog_items": 0, 2024-03-08T21:15:59.869 INFO:teuthology.orchestra.run.smithi149.stdout: "osdmap_bytes": 8352, 2024-03-08T21:15:59.869 INFO:teuthology.orchestra.run.smithi149.stdout: "osdmap_items": 190, 2024-03-08T21:15:59.869 INFO:teuthology.orchestra.run.smithi149.stdout: "osdmap_mapping_bytes": 0, 2024-03-08T21:15:59.869 INFO:teuthology.orchestra.run.smithi149.stdout: "osdmap_mapping_items": 0, 2024-03-08T21:15:59.869 INFO:teuthology.orchestra.run.smithi149.stdout: "pgmap_bytes": 0, 2024-03-08T21:15:59.869 INFO:teuthology.orchestra.run.smithi149.stdout: "pgmap_items": 0, 2024-03-08T21:15:59.869 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_co_bytes": 230023, 2024-03-08T21:15:59.870 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_co_items": 1756, 2024-03-08T21:15:59.870 INFO:teuthology.orchestra.run.smithi149.stdout: "unittest_1_bytes": 0, 2024-03-08T21:15:59.870 INFO:teuthology.orchestra.run.smithi149.stdout: "unittest_1_items": 0, 2024-03-08T21:15:59.870 INFO:teuthology.orchestra.run.smithi149.stdout: "unittest_2_bytes": 0, 2024-03-08T21:15:59.870 INFO:teuthology.orchestra.run.smithi149.stdout: "unittest_2_items": 0 2024-03-08T21:15:59.870 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:15:59.870 INFO:teuthology.orchestra.run.smithi149.stdout: "objecter": { 2024-03-08T21:15:59.870 INFO:teuthology.orchestra.run.smithi149.stdout: "op_active": 0, 2024-03-08T21:15:59.870 INFO:teuthology.orchestra.run.smithi149.stdout: "op_laggy": 0, 2024-03-08T21:15:59.870 INFO:teuthology.orchestra.run.smithi149.stdout: "op_send": 6236, 2024-03-08T21:15:59.870 INFO:teuthology.orchestra.run.smithi149.stdout: "op_send_bytes": 34426313, 2024-03-08T21:15:59.870 INFO:teuthology.orchestra.run.smithi149.stdout: "op_resend": 37, 2024-03-08T21:15:59.870 INFO:teuthology.orchestra.run.smithi149.stdout: "op_reply": 6236, 2024-03-08T21:15:59.870 INFO:teuthology.orchestra.run.smithi149.stdout: "op_latency": { 2024-03-08T21:15:59.871 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 6236, 2024-03-08T21:15:59.871 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 52.872935415, 2024-03-08T21:15:59.871 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.008478661 2024-03-08T21:15:59.871 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:15:59.871 INFO:teuthology.orchestra.run.smithi149.stdout: "op_inflight": 0, 2024-03-08T21:15:59.871 INFO:teuthology.orchestra.run.smithi149.stdout: "oplen_avg": { 2024-03-08T21:15:59.871 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 6236, 2024-03-08T21:15:59.871 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 7056 2024-03-08T21:15:59.871 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:15:59.872 INFO:teuthology.orchestra.run.smithi149.stdout: "op": 6236, 2024-03-08T21:15:59.872 INFO:teuthology.orchestra.run.smithi149.stdout: "op_r": 78, 2024-03-08T21:15:59.872 INFO:teuthology.orchestra.run.smithi149.stdout: "op_w": 6158, 2024-03-08T21:15:59.872 INFO:teuthology.orchestra.run.smithi149.stdout: "op_rmw": 0, 2024-03-08T21:15:59.872 INFO:teuthology.orchestra.run.smithi149.stdout: "op_pg": 0, 2024-03-08T21:15:59.872 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_stat": 167, 2024-03-08T21:15:59.872 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_create": 184, 2024-03-08T21:15:59.872 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_read": 78, 2024-03-08T21:15:59.872 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_write": 1523, 2024-03-08T21:15:59.872 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_writefull": 244, 2024-03-08T21:15:59.872 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_writesame": 0, 2024-03-08T21:15:59.872 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_append": 0, 2024-03-08T21:15:59.872 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_zero": 0, 2024-03-08T21:15:59.873 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_truncate": 0, 2024-03-08T21:15:59.873 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_delete": 3864, 2024-03-08T21:15:59.873 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_mapext": 0, 2024-03-08T21:15:59.873 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_sparse_read": 0, 2024-03-08T21:15:59.873 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_clonerange": 0, 2024-03-08T21:15:59.873 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_getxattr": 0, 2024-03-08T21:15:59.873 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_setxattr": 368, 2024-03-08T21:15:59.873 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_cmpxattr": 0, 2024-03-08T21:15:59.873 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_rmxattr": 0, 2024-03-08T21:15:59.873 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_resetxattrs": 0, 2024-03-08T21:15:59.873 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_call": 0, 2024-03-08T21:15:59.873 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_watch": 0, 2024-03-08T21:15:59.873 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_notify": 0, 2024-03-08T21:15:59.874 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_src_cmpxattr": 0, 2024-03-08T21:15:59.874 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_pgls": 0, 2024-03-08T21:15:59.874 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_pgls_filter": 0, 2024-03-08T21:15:59.874 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_other": 0, 2024-03-08T21:15:59.874 INFO:teuthology.orchestra.run.smithi149.stdout: "linger_active": 0, 2024-03-08T21:15:59.874 INFO:teuthology.orchestra.run.smithi149.stdout: "linger_send": 0, 2024-03-08T21:15:59.874 INFO:teuthology.orchestra.run.smithi149.stdout: "linger_resend": 0, 2024-03-08T21:15:59.874 INFO:teuthology.orchestra.run.smithi149.stdout: "linger_ping": 0, 2024-03-08T21:15:59.874 INFO:teuthology.orchestra.run.smithi149.stdout: "poolop_active": 0, 2024-03-08T21:15:59.874 INFO:teuthology.orchestra.run.smithi149.stdout: "poolop_send": 0, 2024-03-08T21:15:59.874 INFO:teuthology.orchestra.run.smithi149.stdout: "poolop_resend": 0, 2024-03-08T21:15:59.874 INFO:teuthology.orchestra.run.smithi149.stdout: "poolstat_active": 0, 2024-03-08T21:15:59.874 INFO:teuthology.orchestra.run.smithi149.stdout: "poolstat_send": 0, 2024-03-08T21:15:59.875 INFO:teuthology.orchestra.run.smithi149.stdout: "poolstat_resend": 0, 2024-03-08T21:15:59.875 INFO:teuthology.orchestra.run.smithi149.stdout: "statfs_active": 0, 2024-03-08T21:15:59.875 INFO:teuthology.orchestra.run.smithi149.stdout: "statfs_send": 0, 2024-03-08T21:15:59.875 INFO:teuthology.orchestra.run.smithi149.stdout: "statfs_resend": 0, 2024-03-08T21:15:59.875 INFO:teuthology.orchestra.run.smithi149.stdout: "command_active": 0, 2024-03-08T21:15:59.875 INFO:teuthology.orchestra.run.smithi149.stdout: "command_send": 0, 2024-03-08T21:15:59.875 INFO:teuthology.orchestra.run.smithi149.stdout: "command_resend": 0, 2024-03-08T21:15:59.875 INFO:teuthology.orchestra.run.smithi149.stdout: "map_epoch": 113, 2024-03-08T21:15:59.875 INFO:teuthology.orchestra.run.smithi149.stdout: "map_full": 0, 2024-03-08T21:15:59.875 INFO:teuthology.orchestra.run.smithi149.stdout: "map_inc": 28, 2024-03-08T21:15:59.875 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_sessions": 12, 2024-03-08T21:15:59.876 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_session_open": 12, 2024-03-08T21:15:59.876 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_session_close": 0, 2024-03-08T21:15:59.876 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_laggy": 0, 2024-03-08T21:15:59.876 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_wr": 571, 2024-03-08T21:15:59.876 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_rd": 0, 2024-03-08T21:15:59.876 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_del": 57 2024-03-08T21:15:59.876 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:15:59.876 INFO:teuthology.orchestra.run.smithi149.stdout: "oft": { 2024-03-08T21:15:59.876 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_total_objs": 1, 2024-03-08T21:15:59.876 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_total_kv_pairs": 0, 2024-03-08T21:15:59.876 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_total_updates": 0, 2024-03-08T21:15:59.876 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_total_removes": 0 2024-03-08T21:15:59.876 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:15:59.877 INFO:teuthology.orchestra.run.smithi149.stdout: "osd-slow-ops": { 2024-03-08T21:15:59.877 INFO:teuthology.orchestra.run.smithi149.stdout: "slow_ops_count": 0 2024-03-08T21:15:59.877 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:15:59.877 INFO:teuthology.orchestra.run.smithi149.stdout: "purge_queue": { 2024-03-08T21:15:59.877 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_executing_ops": 0, 2024-03-08T21:15:59.877 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_executing_ops_high_water": 8, 2024-03-08T21:15:59.877 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_executing": 0, 2024-03-08T21:15:59.877 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_executing_high_water": 5, 2024-03-08T21:15:59.877 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_executed_ops": 3339, 2024-03-08T21:15:59.877 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_executed": 1209, 2024-03-08T21:15:59.877 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_item_in_journal": 0 2024-03-08T21:15:59.877 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:15:59.877 INFO:teuthology.orchestra.run.smithi149.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-08T21:15:59.877 INFO:teuthology.orchestra.run.smithi149.stdout: "val": 0, 2024-03-08T21:15:59.877 INFO:teuthology.orchestra.run.smithi149.stdout: "max": 104857600, 2024-03-08T21:15:59.878 INFO:teuthology.orchestra.run.smithi149.stdout: "get_started": 0, 2024-03-08T21:15:59.878 INFO:teuthology.orchestra.run.smithi149.stdout: "get": 19547, 2024-03-08T21:15:59.878 INFO:teuthology.orchestra.run.smithi149.stdout: "get_sum": 6547381, 2024-03-08T21:15:59.878 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_fail": 0, 2024-03-08T21:15:59.878 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_success": 19547, 2024-03-08T21:15:59.878 INFO:teuthology.orchestra.run.smithi149.stdout: "take": 0, 2024-03-08T21:15:59.878 INFO:teuthology.orchestra.run.smithi149.stdout: "take_sum": 0, 2024-03-08T21:15:59.878 INFO:teuthology.orchestra.run.smithi149.stdout: "put": 19547, 2024-03-08T21:15:59.878 INFO:teuthology.orchestra.run.smithi149.stdout: "put_sum": 6547381, 2024-03-08T21:15:59.878 INFO:teuthology.orchestra.run.smithi149.stdout: "wait": { 2024-03-08T21:15:59.878 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-08T21:15:59.878 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-08T21:15:59.878 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-08T21:15:59.878 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-03-08T21:15:59.879 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:15:59.879 INFO:teuthology.orchestra.run.smithi149.stdout: "throttle-objecter_bytes": { 2024-03-08T21:15:59.879 INFO:teuthology.orchestra.run.smithi149.stdout: "val": 0, 2024-03-08T21:15:59.879 INFO:teuthology.orchestra.run.smithi149.stdout: "max": 104857600, 2024-03-08T21:15:59.879 INFO:teuthology.orchestra.run.smithi149.stdout: "get_started": 0, 2024-03-08T21:15:59.879 INFO:teuthology.orchestra.run.smithi149.stdout: "get": 0, 2024-03-08T21:15:59.879 INFO:teuthology.orchestra.run.smithi149.stdout: "get_sum": 0, 2024-03-08T21:15:59.879 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_fail": 0, 2024-03-08T21:15:59.879 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_success": 0, 2024-03-08T21:15:59.879 INFO:teuthology.orchestra.run.smithi149.stdout: "take": 6236, 2024-03-08T21:15:59.879 INFO:teuthology.orchestra.run.smithi149.stdout: "take_sum": 34548478, 2024-03-08T21:15:59.879 INFO:teuthology.orchestra.run.smithi149.stdout: "put": 6236, 2024-03-08T21:15:59.879 INFO:teuthology.orchestra.run.smithi149.stdout: "put_sum": 34548478, 2024-03-08T21:15:59.879 INFO:teuthology.orchestra.run.smithi149.stdout: "wait": { 2024-03-08T21:15:59.879 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-08T21:15:59.880 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-08T21:15:59.880 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-08T21:15:59.880 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-03-08T21:15:59.880 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:15:59.880 INFO:teuthology.orchestra.run.smithi149.stdout: "throttle-objecter_ops": { 2024-03-08T21:15:59.880 INFO:teuthology.orchestra.run.smithi149.stdout: "val": 0, 2024-03-08T21:15:59.880 INFO:teuthology.orchestra.run.smithi149.stdout: "max": 1024, 2024-03-08T21:15:59.880 INFO:teuthology.orchestra.run.smithi149.stdout: "get_started": 0, 2024-03-08T21:15:59.880 INFO:teuthology.orchestra.run.smithi149.stdout: "get": 0, 2024-03-08T21:15:59.880 INFO:teuthology.orchestra.run.smithi149.stdout: "get_sum": 0, 2024-03-08T21:15:59.880 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_fail": 0, 2024-03-08T21:15:59.880 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_success": 0, 2024-03-08T21:15:59.880 INFO:teuthology.orchestra.run.smithi149.stdout: "take": 6236, 2024-03-08T21:15:59.880 INFO:teuthology.orchestra.run.smithi149.stdout: "take_sum": 6236, 2024-03-08T21:15:59.880 INFO:teuthology.orchestra.run.smithi149.stdout: "put": 6236, 2024-03-08T21:15:59.881 INFO:teuthology.orchestra.run.smithi149.stdout: "put_sum": 6236, 2024-03-08T21:15:59.881 INFO:teuthology.orchestra.run.smithi149.stdout: "wait": { 2024-03-08T21:15:59.881 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-08T21:15:59.881 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-08T21:15:59.881 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-08T21:15:59.881 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-03-08T21:15:59.881 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:15:59.881 INFO:teuthology.orchestra.run.smithi149.stdout: "throttle-write_buf_throttle": { 2024-03-08T21:15:59.881 INFO:teuthology.orchestra.run.smithi149.stdout: "val": 0, 2024-03-08T21:15:59.881 INFO:teuthology.orchestra.run.smithi149.stdout: "max": 3758096384, 2024-03-08T21:15:59.881 INFO:teuthology.orchestra.run.smithi149.stdout: "get_started": 0, 2024-03-08T21:15:59.881 INFO:teuthology.orchestra.run.smithi149.stdout: "get": 1209, 2024-03-08T21:15:59.881 INFO:teuthology.orchestra.run.smithi149.stdout: "get_sum": 122165, 2024-03-08T21:15:59.881 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_fail": 0, 2024-03-08T21:15:59.881 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_success": 1209, 2024-03-08T21:15:59.881 INFO:teuthology.orchestra.run.smithi149.stdout: "take": 0, 2024-03-08T21:15:59.882 INFO:teuthology.orchestra.run.smithi149.stdout: "take_sum": 0, 2024-03-08T21:15:59.882 INFO:teuthology.orchestra.run.smithi149.stdout: "put": 78, 2024-03-08T21:15:59.882 INFO:teuthology.orchestra.run.smithi149.stdout: "put_sum": 122165, 2024-03-08T21:15:59.882 INFO:teuthology.orchestra.run.smithi149.stdout: "wait": { 2024-03-08T21:15:59.882 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-08T21:15:59.882 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-08T21:15:59.882 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-08T21:15:59.882 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-03-08T21:15:59.882 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:15:59.882 INFO:teuthology.orchestra.run.smithi149.stdout: "throttle-write_buf_throttle-0x556d1958c6e0": { 2024-03-08T21:15:59.882 INFO:teuthology.orchestra.run.smithi149.stdout: "val": 0, 2024-03-08T21:15:59.882 INFO:teuthology.orchestra.run.smithi149.stdout: "max": 3758096384, 2024-03-08T21:15:59.882 INFO:teuthology.orchestra.run.smithi149.stdout: "get_started": 0, 2024-03-08T21:15:59.883 INFO:teuthology.orchestra.run.smithi149.stdout: "get": 6669, 2024-03-08T21:15:59.883 INFO:teuthology.orchestra.run.smithi149.stdout: "get_sum": 33470238, 2024-03-08T21:15:59.883 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_fail": 0, 2024-03-08T21:15:59.883 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_success": 6669, 2024-03-08T21:15:59.883 INFO:teuthology.orchestra.run.smithi149.stdout: "take": 0, 2024-03-08T21:15:59.883 INFO:teuthology.orchestra.run.smithi149.stdout: "take_sum": 0, 2024-03-08T21:15:59.883 INFO:teuthology.orchestra.run.smithi149.stdout: "put": 1445, 2024-03-08T21:15:59.883 INFO:teuthology.orchestra.run.smithi149.stdout: "put_sum": 33470238, 2024-03-08T21:15:59.883 INFO:teuthology.orchestra.run.smithi149.stdout: "wait": { 2024-03-08T21:15:59.883 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-08T21:15:59.883 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-08T21:15:59.883 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-08T21:15:59.883 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-03-08T21:15:59.883 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-03-08T21:15:59.883 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-03-08T21:16:00.864 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:16:00 smithi149 ceph-mon[35042]: pgmap v490: 97 pgs: 97 active+clean; 17 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 1.7 KiB/s wr, 13 op/s 2024-03-08T21:16:00.942 INFO:tasks.check_counter:Daemon mds.b mds.exported=0 2024-03-08T21:16:00.942 INFO:tasks.check_counter:Daemon mds.b mds.imported=0 2024-03-08T21:16:00.942 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update=27 2024-03-08T21:16:00.942 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update_receipt=0 2024-03-08T21:16:00.942 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=6 2024-03-08T21:16:00.942 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=15 2024-03-08T21:16:00.942 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=9 2024-03-08T21:16:00.942 DEBUG:tasks.check_counter:Getting stats from e 2024-03-08T21:16:00.943 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761 shell --fsid dc20f0f8-dd8c-11ee-95c6-87774f69a715 -- ceph daemon mds.e perf dump 2024-03-08T21:16:00.961 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:16:00 smithi107 ceph-mon[31381]: pgmap v490: 97 pgs: 97 active+clean; 17 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 1.7 KiB/s wr, 13 op/s 2024-03-08T21:16:01.121 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:16:00 smithi204 ceph-mon[34935]: pgmap v490: 97 pgs: 97 active+clean; 17 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 1.7 KiB/s wr, 13 op/s 2024-03-08T21:16:01.393 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/mon.b/config 2024-03-08T21:16:02.758 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:16:02 smithi149 ceph-mon[35042]: pgmap v491: 97 pgs: 97 active+clean; 17 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-03-08T21:16:02.960 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:16:02 smithi107 ceph-mon[31381]: pgmap v491: 97 pgs: 97 active+clean; 17 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-03-08T21:16:03.121 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:16:02 smithi204 ceph-mon[34935]: pgmap v491: 97 pgs: 97 active+clean; 17 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-03-08T21:16:03.617 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-03-08T21:16:03.617 INFO:teuthology.orchestra.run.smithi149.stdout: "AsyncMessenger::Worker-0": { 2024-03-08T21:16:03.617 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_messages": 431, 2024-03-08T21:16:03.617 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages": 638, 2024-03-08T21:16:03.617 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_bytes": 1613349, 2024-03-08T21:16:03.617 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_bytes": 642234, 2024-03-08T21:16:03.618 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_created_connections": 13, 2024-03-08T21:16:03.618 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_active_connections": 3, 2024-03-08T21:16:03.618 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_total_time": 0.183989614, 2024-03-08T21:16:03.618 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_send_time": 0.071629534, 2024-03-08T21:16:03.618 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_recv_time": 0.109008446, 2024-03-08T21:16:03.618 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_fast_dispatch_time": 0.015676841, 2024-03-08T21:16:03.618 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages_queue_lat": { 2024-03-08T21:16:03.618 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 638, 2024-03-08T21:16:03.618 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.092937652, 2024-03-08T21:16:03.618 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000145670 2024-03-08T21:16:03.618 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:16:03.619 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_handle_ack_lat": { 2024-03-08T21:16:03.619 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 76, 2024-03-08T21:16:03.619 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000159557, 2024-03-08T21:16:03.619 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000002099 2024-03-08T21:16:03.619 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:16:03.619 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_encrypted_bytes": 1606272, 2024-03-08T21:16:03.619 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_encrypted_bytes": 579680 2024-03-08T21:16:03.619 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:16:03.619 INFO:teuthology.orchestra.run.smithi149.stdout: "AsyncMessenger::Worker-1": { 2024-03-08T21:16:03.619 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_messages": 38, 2024-03-08T21:16:03.619 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages": 133, 2024-03-08T21:16:03.620 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_bytes": 9298, 2024-03-08T21:16:03.620 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_bytes": 142318, 2024-03-08T21:16:03.620 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_created_connections": 11, 2024-03-08T21:16:03.620 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_active_connections": 5, 2024-03-08T21:16:03.620 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_total_time": 0.028432044, 2024-03-08T21:16:03.620 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_send_time": 0.010644575, 2024-03-08T21:16:03.620 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_recv_time": 0.014564384, 2024-03-08T21:16:03.620 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_fast_dispatch_time": 0.000459586, 2024-03-08T21:16:03.620 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages_queue_lat": { 2024-03-08T21:16:03.620 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 130, 2024-03-08T21:16:03.620 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.047152078, 2024-03-08T21:16:03.620 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000362708 2024-03-08T21:16:03.621 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:16:03.621 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_handle_ack_lat": { 2024-03-08T21:16:03.621 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 87, 2024-03-08T21:16:03.621 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000089565, 2024-03-08T21:16:03.621 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000001029 2024-03-08T21:16:03.621 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:16:03.621 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_encrypted_bytes": 5216, 2024-03-08T21:16:03.621 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_encrypted_bytes": 129760 2024-03-08T21:16:03.621 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:16:03.621 INFO:teuthology.orchestra.run.smithi149.stdout: "AsyncMessenger::Worker-2": { 2024-03-08T21:16:03.621 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_messages": 1198, 2024-03-08T21:16:03.621 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages": 1023, 2024-03-08T21:16:03.622 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_bytes": 403137, 2024-03-08T21:16:03.622 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_bytes": 543421, 2024-03-08T21:16:03.622 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_created_connections": 7, 2024-03-08T21:16:03.622 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_active_connections": 4, 2024-03-08T21:16:03.622 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_total_time": 0.391004453, 2024-03-08T21:16:03.622 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_send_time": 0.209193265, 2024-03-08T21:16:03.622 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_recv_time": 0.200547930, 2024-03-08T21:16:03.622 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_fast_dispatch_time": 0.011935129, 2024-03-08T21:16:03.622 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages_queue_lat": { 2024-03-08T21:16:03.622 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 1021, 2024-03-08T21:16:03.623 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.131895719, 2024-03-08T21:16:03.623 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000129182 2024-03-08T21:16:03.623 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:16:03.623 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_handle_ack_lat": { 2024-03-08T21:16:03.623 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 2160, 2024-03-08T21:16:03.623 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.001879195, 2024-03-08T21:16:03.623 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000869 2024-03-08T21:16:03.623 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:16:03.623 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_encrypted_bytes": 928, 2024-03-08T21:16:03.623 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_encrypted_bytes": 1216 2024-03-08T21:16:03.623 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:16:03.623 INFO:teuthology.orchestra.run.smithi149.stdout: "cct": { 2024-03-08T21:16:03.624 INFO:teuthology.orchestra.run.smithi149.stdout: "total_workers": 1, 2024-03-08T21:16:03.624 INFO:teuthology.orchestra.run.smithi149.stdout: "unhealthy_workers": 0 2024-03-08T21:16:03.624 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:16:03.624 INFO:teuthology.orchestra.run.smithi149.stdout: "finisher-MDSRank": { 2024-03-08T21:16:03.624 INFO:teuthology.orchestra.run.smithi149.stdout: "queue_len": 0, 2024-03-08T21:16:03.624 INFO:teuthology.orchestra.run.smithi149.stdout: "complete_latency": { 2024-03-08T21:16:03.624 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 59, 2024-03-08T21:16:03.624 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.002581133, 2024-03-08T21:16:03.624 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000043748 2024-03-08T21:16:03.624 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-03-08T21:16:03.624 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:16:03.625 INFO:teuthology.orchestra.run.smithi149.stdout: "finisher-PurgeQueue": { 2024-03-08T21:16:03.625 INFO:teuthology.orchestra.run.smithi149.stdout: "queue_len": 0, 2024-03-08T21:16:03.625 INFO:teuthology.orchestra.run.smithi149.stdout: "complete_latency": { 2024-03-08T21:16:03.625 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 2, 2024-03-08T21:16:03.625 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000020218, 2024-03-08T21:16:03.625 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000010109 2024-03-08T21:16:03.625 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-03-08T21:16:03.625 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:16:03.625 INFO:teuthology.orchestra.run.smithi149.stdout: "mds": { 2024-03-08T21:16:03.625 INFO:teuthology.orchestra.run.smithi149.stdout: "request": 7, 2024-03-08T21:16:03.625 INFO:teuthology.orchestra.run.smithi149.stdout: "reply": 4, 2024-03-08T21:16:03.625 INFO:teuthology.orchestra.run.smithi149.stdout: "reply_latency": { 2024-03-08T21:16:03.626 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 4, 2024-03-08T21:16:03.626 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.002066513, 2024-03-08T21:16:03.626 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000516628 2024-03-08T21:16:03.626 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:16:03.626 INFO:teuthology.orchestra.run.smithi149.stdout: "slow_reply": 0, 2024-03-08T21:16:03.626 INFO:teuthology.orchestra.run.smithi149.stdout: "forward": 3, 2024-03-08T21:16:03.626 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_fetch_complete": 0, 2024-03-08T21:16:03.626 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_fetch_keys": 0, 2024-03-08T21:16:03.626 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_commit": 11, 2024-03-08T21:16:03.626 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_split": 0, 2024-03-08T21:16:03.626 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_merge": 0, 2024-03-08T21:16:03.626 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes": 18, 2024-03-08T21:16:03.627 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes_top": 2, 2024-03-08T21:16:03.627 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes_bottom": 16, 2024-03-08T21:16:03.627 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes_pin_tail": 0, 2024-03-08T21:16:03.627 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes_pinned": 13, 2024-03-08T21:16:03.627 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes_expired": 8, 2024-03-08T21:16:03.627 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes_with_caps": 2, 2024-03-08T21:16:03.627 INFO:teuthology.orchestra.run.smithi149.stdout: "caps": 2, 2024-03-08T21:16:03.627 INFO:teuthology.orchestra.run.smithi149.stdout: "subtrees": 3, 2024-03-08T21:16:03.627 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse": 19, 2024-03-08T21:16:03.627 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse_hit": 14, 2024-03-08T21:16:03.627 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse_forward": 0, 2024-03-08T21:16:03.627 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse_discover": 4, 2024-03-08T21:16:03.628 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse_dir_fetch": 0, 2024-03-08T21:16:03.628 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse_remote_ino": 0, 2024-03-08T21:16:03.628 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse_lock": 0, 2024-03-08T21:16:03.628 INFO:teuthology.orchestra.run.smithi149.stdout: "load_cent": 0, 2024-03-08T21:16:03.628 INFO:teuthology.orchestra.run.smithi149.stdout: "q": 0, 2024-03-08T21:16:03.628 INFO:teuthology.orchestra.run.smithi149.stdout: "exported": 0, 2024-03-08T21:16:03.628 INFO:teuthology.orchestra.run.smithi149.stdout: "exported_inodes": 0, 2024-03-08T21:16:03.628 INFO:teuthology.orchestra.run.smithi149.stdout: "imported": 0, 2024-03-08T21:16:03.628 INFO:teuthology.orchestra.run.smithi149.stdout: "imported_inodes": 0, 2024-03-08T21:16:03.628 INFO:teuthology.orchestra.run.smithi149.stdout: "openino_dir_fetch": 0, 2024-03-08T21:16:03.628 INFO:teuthology.orchestra.run.smithi149.stdout: "openino_backtrace_fetch": 0, 2024-03-08T21:16:03.628 INFO:teuthology.orchestra.run.smithi149.stdout: "openino_peer_discover": 0, 2024-03-08T21:16:03.628 INFO:teuthology.orchestra.run.smithi149.stdout: "root_rfiles": 5, 2024-03-08T21:16:03.629 INFO:teuthology.orchestra.run.smithi149.stdout: "root_rbytes": 4202218, 2024-03-08T21:16:03.629 INFO:teuthology.orchestra.run.smithi149.stdout: "root_rsnaps": 6, 2024-03-08T21:16:03.629 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_backtrace_fetch": 0, 2024-03-08T21:16:03.629 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_set_tag": 0, 2024-03-08T21:16:03.629 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_backtrace_repaired": 0, 2024-03-08T21:16:03.629 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_inotable_repaired": 0, 2024-03-08T21:16:03.629 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_dir_inodes": 0, 2024-03-08T21:16:03.629 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_dir_base_inodes": 0, 2024-03-08T21:16:03.629 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_dirfrag_rstats": 0, 2024-03-08T21:16:03.629 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_file_inodes": 0, 2024-03-08T21:16:03.629 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_inode_file_caps": 0, 2024-03-08T21:16:03.629 INFO:teuthology.orchestra.run.smithi149.stdout: "ceph_cap_op_revoke": 0, 2024-03-08T21:16:03.630 INFO:teuthology.orchestra.run.smithi149.stdout: "ceph_cap_op_grant": 0, 2024-03-08T21:16:03.630 INFO:teuthology.orchestra.run.smithi149.stdout: "ceph_cap_op_trunc": 0, 2024-03-08T21:16:03.630 INFO:teuthology.orchestra.run.smithi149.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-08T21:16:03.630 INFO:teuthology.orchestra.run.smithi149.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-08T21:16:03.630 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_client_caps": 0, 2024-03-08T21:16:03.630 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_client_caps_dirty": 0, 2024-03-08T21:16:03.630 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_client_cap_release": 0, 2024-03-08T21:16:03.630 INFO:teuthology.orchestra.run.smithi149.stdout: "process_request_cap_release": 0 2024-03-08T21:16:03.631 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:16:03.631 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_cache": { 2024-03-08T21:16:03.631 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_update": 0, 2024-03-08T21:16:03.631 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_update_receipt": 18, 2024-03-08T21:16:03.631 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_try_discover": 11, 2024-03-08T21:16:03.631 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_send_discover": 12, 2024-03-08T21:16:03.631 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_handle_discover": 0, 2024-03-08T21:16:03.631 INFO:teuthology.orchestra.run.smithi149.stdout: "num_strays": 0, 2024-03-08T21:16:03.631 INFO:teuthology.orchestra.run.smithi149.stdout: "num_strays_delayed": 0, 2024-03-08T21:16:03.631 INFO:teuthology.orchestra.run.smithi149.stdout: "num_strays_enqueuing": 0, 2024-03-08T21:16:03.631 INFO:teuthology.orchestra.run.smithi149.stdout: "strays_created": 0, 2024-03-08T21:16:03.631 INFO:teuthology.orchestra.run.smithi149.stdout: "strays_enqueued": 0, 2024-03-08T21:16:03.632 INFO:teuthology.orchestra.run.smithi149.stdout: "strays_reintegrated": 0, 2024-03-08T21:16:03.632 INFO:teuthology.orchestra.run.smithi149.stdout: "strays_migrated": 0, 2024-03-08T21:16:03.632 INFO:teuthology.orchestra.run.smithi149.stdout: "num_recovering_processing": 0, 2024-03-08T21:16:03.632 INFO:teuthology.orchestra.run.smithi149.stdout: "num_recovering_enqueued": 0, 2024-03-08T21:16:03.632 INFO:teuthology.orchestra.run.smithi149.stdout: "num_recovering_prioritized": 0, 2024-03-08T21:16:03.632 INFO:teuthology.orchestra.run.smithi149.stdout: "recovery_started": 0, 2024-03-08T21:16:03.632 INFO:teuthology.orchestra.run.smithi149.stdout: "recovery_completed": 0, 2024-03-08T21:16:03.632 INFO:teuthology.orchestra.run.smithi149.stdout: "ireq_enqueue_scrub": 0, 2024-03-08T21:16:03.632 INFO:teuthology.orchestra.run.smithi149.stdout: "ireq_exportdir": 0, 2024-03-08T21:16:03.632 INFO:teuthology.orchestra.run.smithi149.stdout: "ireq_flush": 0, 2024-03-08T21:16:03.632 INFO:teuthology.orchestra.run.smithi149.stdout: "ireq_fragmentdir": 0, 2024-03-08T21:16:03.632 INFO:teuthology.orchestra.run.smithi149.stdout: "ireq_fragstats": 0, 2024-03-08T21:16:03.632 INFO:teuthology.orchestra.run.smithi149.stdout: "ireq_inodestats": 0 2024-03-08T21:16:03.633 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:16:03.633 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_log": { 2024-03-08T21:16:03.633 INFO:teuthology.orchestra.run.smithi149.stdout: "evlrg": 0, 2024-03-08T21:16:03.633 INFO:teuthology.orchestra.run.smithi149.stdout: "evadd": 8, 2024-03-08T21:16:03.633 INFO:teuthology.orchestra.run.smithi149.stdout: "evex": 0, 2024-03-08T21:16:03.633 INFO:teuthology.orchestra.run.smithi149.stdout: "evtrm": 0, 2024-03-08T21:16:03.633 INFO:teuthology.orchestra.run.smithi149.stdout: "ev": 8, 2024-03-08T21:16:03.633 INFO:teuthology.orchestra.run.smithi149.stdout: "evexg": 0, 2024-03-08T21:16:03.633 INFO:teuthology.orchestra.run.smithi149.stdout: "evexd": 0, 2024-03-08T21:16:03.633 INFO:teuthology.orchestra.run.smithi149.stdout: "segadd": 2, 2024-03-08T21:16:03.633 INFO:teuthology.orchestra.run.smithi149.stdout: "segex": 0, 2024-03-08T21:16:03.633 INFO:teuthology.orchestra.run.smithi149.stdout: "segtrm": 0, 2024-03-08T21:16:03.634 INFO:teuthology.orchestra.run.smithi149.stdout: "seg": 2, 2024-03-08T21:16:03.634 INFO:teuthology.orchestra.run.smithi149.stdout: "segmjr": 2, 2024-03-08T21:16:03.634 INFO:teuthology.orchestra.run.smithi149.stdout: "segexg": 0, 2024-03-08T21:16:03.634 INFO:teuthology.orchestra.run.smithi149.stdout: "segexd": 0, 2024-03-08T21:16:03.634 INFO:teuthology.orchestra.run.smithi149.stdout: "expos": 4194304, 2024-03-08T21:16:03.634 INFO:teuthology.orchestra.run.smithi149.stdout: "wrpos": 4196840, 2024-03-08T21:16:03.634 INFO:teuthology.orchestra.run.smithi149.stdout: "rdpos": 0, 2024-03-08T21:16:03.634 INFO:teuthology.orchestra.run.smithi149.stdout: "jlat": { 2024-03-08T21:16:03.634 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 7, 2024-03-08T21:16:03.634 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.033374659, 2024-03-08T21:16:03.635 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.004767808 2024-03-08T21:16:03.635 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:16:03.635 INFO:teuthology.orchestra.run.smithi149.stdout: "replayed": 0 2024-03-08T21:16:03.635 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:16:03.635 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_mem": { 2024-03-08T21:16:03.635 INFO:teuthology.orchestra.run.smithi149.stdout: "ino": 22, 2024-03-08T21:16:03.635 INFO:teuthology.orchestra.run.smithi149.stdout: "ino+": 28, 2024-03-08T21:16:03.635 INFO:teuthology.orchestra.run.smithi149.stdout: "ino-": 6, 2024-03-08T21:16:03.635 INFO:teuthology.orchestra.run.smithi149.stdout: "dir": 21, 2024-03-08T21:16:03.635 INFO:teuthology.orchestra.run.smithi149.stdout: "dir+": 26, 2024-03-08T21:16:03.635 INFO:teuthology.orchestra.run.smithi149.stdout: "dir-": 5, 2024-03-08T21:16:03.635 INFO:teuthology.orchestra.run.smithi149.stdout: "dn": 18, 2024-03-08T21:16:03.636 INFO:teuthology.orchestra.run.smithi149.stdout: "dn+": 26, 2024-03-08T21:16:03.636 INFO:teuthology.orchestra.run.smithi149.stdout: "dn-": 8, 2024-03-08T21:16:03.636 INFO:teuthology.orchestra.run.smithi149.stdout: "cap": 2, 2024-03-08T21:16:03.636 INFO:teuthology.orchestra.run.smithi149.stdout: "cap+": 2, 2024-03-08T21:16:03.636 INFO:teuthology.orchestra.run.smithi149.stdout: "cap-": 0, 2024-03-08T21:16:03.636 INFO:teuthology.orchestra.run.smithi149.stdout: "rss": 50176, 2024-03-08T21:16:03.636 INFO:teuthology.orchestra.run.smithi149.stdout: "heap": 207228 2024-03-08T21:16:03.636 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:16:03.636 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_server": { 2024-03-08T21:16:03.636 INFO:teuthology.orchestra.run.smithi149.stdout: "dispatch_client_request": 7, 2024-03-08T21:16:03.636 INFO:teuthology.orchestra.run.smithi149.stdout: "dispatch_server_request": 0, 2024-03-08T21:16:03.637 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_client_request": 7, 2024-03-08T21:16:03.637 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_client_session": 32, 2024-03-08T21:16:03.637 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_peer_request": 0, 2024-03-08T21:16:03.637 INFO:teuthology.orchestra.run.smithi149.stdout: "req_create_latency": { 2024-03-08T21:16:03.637 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-08T21:16:03.637 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-08T21:16:03.637 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-08T21:16:03.637 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:16:03.637 INFO:teuthology.orchestra.run.smithi149.stdout: "req_getattr_latency": { 2024-03-08T21:16:03.637 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 4, 2024-03-08T21:16:03.637 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.002066513, 2024-03-08T21:16:03.637 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000516628 2024-03-08T21:16:03.638 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:16:03.638 INFO:teuthology.orchestra.run.smithi149.stdout: "req_getfilelock_latency": { 2024-03-08T21:16:03.638 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-08T21:16:03.638 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-08T21:16:03.638 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-08T21:16:03.638 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:16:03.638 INFO:teuthology.orchestra.run.smithi149.stdout: "req_link_latency": { 2024-03-08T21:16:03.638 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-08T21:16:03.638 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-08T21:16:03.638 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-08T21:16:03.638 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:16:03.638 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lookup_latency": { 2024-03-08T21:16:03.639 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-08T21:16:03.639 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-08T21:16:03.639 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-08T21:16:03.639 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:16:03.639 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lookuphash_latency": { 2024-03-08T21:16:03.639 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-08T21:16:03.639 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-08T21:16:03.639 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-08T21:16:03.639 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:16:03.639 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lookupino_latency": { 2024-03-08T21:16:03.639 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-08T21:16:03.640 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-08T21:16:03.640 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-08T21:16:03.640 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:16:03.640 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lookupname_latency": { 2024-03-08T21:16:03.640 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-08T21:16:03.640 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-08T21:16:03.640 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-08T21:16:03.640 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:16:03.640 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lookupparent_latency": { 2024-03-08T21:16:03.640 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-08T21:16:03.640 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-08T21:16:03.640 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-08T21:16:03.641 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:16:03.641 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lookupsnap_latency": { 2024-03-08T21:16:03.641 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-08T21:16:03.641 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-08T21:16:03.641 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-08T21:16:03.641 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:16:03.641 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lssnap_latency": { 2024-03-08T21:16:03.641 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-08T21:16:03.641 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-08T21:16:03.641 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-08T21:16:03.641 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:16:03.641 INFO:teuthology.orchestra.run.smithi149.stdout: "req_mkdir_latency": { 2024-03-08T21:16:03.642 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-08T21:16:03.642 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-08T21:16:03.642 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-08T21:16:03.643 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:16:03.643 INFO:teuthology.orchestra.run.smithi149.stdout: "req_mknod_latency": { 2024-03-08T21:16:03.643 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-08T21:16:03.643 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-08T21:16:03.643 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-08T21:16:03.643 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:16:03.643 INFO:teuthology.orchestra.run.smithi149.stdout: "req_mksnap_latency": { 2024-03-08T21:16:03.643 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-08T21:16:03.643 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-08T21:16:03.643 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-08T21:16:03.643 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:16:03.644 INFO:teuthology.orchestra.run.smithi149.stdout: "req_open_latency": { 2024-03-08T21:16:03.644 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-08T21:16:03.644 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-08T21:16:03.644 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-08T21:16:03.644 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:16:03.644 INFO:teuthology.orchestra.run.smithi149.stdout: "req_readdir_latency": { 2024-03-08T21:16:03.644 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-08T21:16:03.644 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-08T21:16:03.644 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-08T21:16:03.644 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:16:03.645 INFO:teuthology.orchestra.run.smithi149.stdout: "req_rename_latency": { 2024-03-08T21:16:03.645 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-08T21:16:03.645 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-08T21:16:03.645 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-08T21:16:03.645 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:16:03.645 INFO:teuthology.orchestra.run.smithi149.stdout: "req_renamesnap_latency": { 2024-03-08T21:16:03.645 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-08T21:16:03.645 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-08T21:16:03.645 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-08T21:16:03.645 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:16:03.645 INFO:teuthology.orchestra.run.smithi149.stdout: "req_snapdiff_latency": { 2024-03-08T21:16:03.645 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-08T21:16:03.646 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-08T21:16:03.646 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-08T21:16:03.646 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:16:03.646 INFO:teuthology.orchestra.run.smithi149.stdout: "req_rmdir_latency": { 2024-03-08T21:16:03.646 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-08T21:16:03.646 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-08T21:16:03.646 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-08T21:16:03.646 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:16:03.646 INFO:teuthology.orchestra.run.smithi149.stdout: "req_rmsnap_latency": { 2024-03-08T21:16:03.646 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-08T21:16:03.646 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-08T21:16:03.647 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-08T21:16:03.647 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:16:03.647 INFO:teuthology.orchestra.run.smithi149.stdout: "req_rmxattr_latency": { 2024-03-08T21:16:03.647 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-08T21:16:03.647 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-08T21:16:03.647 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-08T21:16:03.647 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:16:03.647 INFO:teuthology.orchestra.run.smithi149.stdout: "req_setattr_latency": { 2024-03-08T21:16:03.647 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-08T21:16:03.647 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-08T21:16:03.647 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-08T21:16:03.648 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:16:03.648 INFO:teuthology.orchestra.run.smithi149.stdout: "req_setdirlayout_latency": { 2024-03-08T21:16:03.648 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-08T21:16:03.648 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-08T21:16:03.648 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-08T21:16:03.648 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:16:03.648 INFO:teuthology.orchestra.run.smithi149.stdout: "req_setfilelock_latency": { 2024-03-08T21:16:03.648 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-08T21:16:03.648 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-08T21:16:03.648 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-08T21:16:03.648 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:16:03.649 INFO:teuthology.orchestra.run.smithi149.stdout: "req_setlayout_latency": { 2024-03-08T21:16:03.649 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-08T21:16:03.649 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-08T21:16:03.649 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-08T21:16:03.649 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:16:03.649 INFO:teuthology.orchestra.run.smithi149.stdout: "req_setxattr_latency": { 2024-03-08T21:16:03.649 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-08T21:16:03.649 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-08T21:16:03.649 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-08T21:16:03.649 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:16:03.650 INFO:teuthology.orchestra.run.smithi149.stdout: "req_symlink_latency": { 2024-03-08T21:16:03.650 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-08T21:16:03.650 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-08T21:16:03.650 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-08T21:16:03.650 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:16:03.650 INFO:teuthology.orchestra.run.smithi149.stdout: "req_unlink_latency": { 2024-03-08T21:16:03.650 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-08T21:16:03.650 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-08T21:16:03.650 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-08T21:16:03.650 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:16:03.651 INFO:teuthology.orchestra.run.smithi149.stdout: "cap_revoke_eviction": 0, 2024-03-08T21:16:03.651 INFO:teuthology.orchestra.run.smithi149.stdout: "cap_acquisition_throttle": 0, 2024-03-08T21:16:03.651 INFO:teuthology.orchestra.run.smithi149.stdout: "req_getvxattr_latency": { 2024-03-08T21:16:03.651 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-08T21:16:03.651 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-08T21:16:03.651 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-08T21:16:03.651 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-03-08T21:16:03.651 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:16:03.651 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_sessions": { 2024-03-08T21:16:03.651 INFO:teuthology.orchestra.run.smithi149.stdout: "session_count": 2, 2024-03-08T21:16:03.651 INFO:teuthology.orchestra.run.smithi149.stdout: "session_add": 4, 2024-03-08T21:16:03.651 INFO:teuthology.orchestra.run.smithi149.stdout: "session_remove": 2, 2024-03-08T21:16:03.652 INFO:teuthology.orchestra.run.smithi149.stdout: "sessions_open": 2, 2024-03-08T21:16:03.652 INFO:teuthology.orchestra.run.smithi149.stdout: "sessions_stale": 0, 2024-03-08T21:16:03.652 INFO:teuthology.orchestra.run.smithi149.stdout: "total_load": 3, 2024-03-08T21:16:03.652 INFO:teuthology.orchestra.run.smithi149.stdout: "average_load": 0, 2024-03-08T21:16:03.652 INFO:teuthology.orchestra.run.smithi149.stdout: "avg_session_uptime": 921, 2024-03-08T21:16:03.652 INFO:teuthology.orchestra.run.smithi149.stdout: "mdthresh_evicted": 0 2024-03-08T21:16:03.652 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:16:03.652 INFO:teuthology.orchestra.run.smithi149.stdout: "mempool": { 2024-03-08T21:16:03.652 INFO:teuthology.orchestra.run.smithi149.stdout: "bloom_filter_bytes": 0, 2024-03-08T21:16:03.652 INFO:teuthology.orchestra.run.smithi149.stdout: "bloom_filter_items": 0, 2024-03-08T21:16:03.653 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_alloc_bytes": 0, 2024-03-08T21:16:03.653 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_alloc_items": 0, 2024-03-08T21:16:03.653 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_data_bytes": 0, 2024-03-08T21:16:03.653 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_data_items": 0, 2024-03-08T21:16:03.653 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-08T21:16:03.653 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_onode_items": 0, 2024-03-08T21:16:03.653 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-08T21:16:03.653 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_meta_items": 0, 2024-03-08T21:16:03.653 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_other_bytes": 0, 2024-03-08T21:16:03.653 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_other_items": 0, 2024-03-08T21:16:03.653 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-08T21:16:03.654 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_buffer_items": 0, 2024-03-08T21:16:03.654 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_extent_bytes": 0, 2024-03-08T21:16:03.654 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_extent_items": 0, 2024-03-08T21:16:03.654 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_blob_bytes": 0, 2024-03-08T21:16:03.654 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_blob_items": 0, 2024-03-08T21:16:03.654 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-08T21:16:03.654 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_shared_blob_items": 0, 2024-03-08T21:16:03.654 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-08T21:16:03.654 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_inline_bl_items": 0, 2024-03-08T21:16:03.654 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_fsck_bytes": 0, 2024-03-08T21:16:03.654 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_fsck_items": 0, 2024-03-08T21:16:03.655 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_txc_bytes": 0, 2024-03-08T21:16:03.655 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_txc_items": 0, 2024-03-08T21:16:03.655 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-08T21:16:03.655 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_writing_deferred_items": 0, 2024-03-08T21:16:03.655 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_writing_bytes": 0, 2024-03-08T21:16:03.655 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_writing_items": 0, 2024-03-08T21:16:03.655 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_bytes": 0, 2024-03-08T21:16:03.655 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_items": 0, 2024-03-08T21:16:03.655 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_file_reader_bytes": 0, 2024-03-08T21:16:03.655 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_file_reader_items": 0, 2024-03-08T21:16:03.655 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_file_writer_bytes": 0, 2024-03-08T21:16:03.655 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_file_writer_items": 0, 2024-03-08T21:16:03.656 INFO:teuthology.orchestra.run.smithi149.stdout: "buffer_anon_bytes": 256262, 2024-03-08T21:16:03.656 INFO:teuthology.orchestra.run.smithi149.stdout: "buffer_anon_items": 91, 2024-03-08T21:16:03.656 INFO:teuthology.orchestra.run.smithi149.stdout: "buffer_meta_bytes": 176, 2024-03-08T21:16:03.656 INFO:teuthology.orchestra.run.smithi149.stdout: "buffer_meta_items": 2, 2024-03-08T21:16:03.656 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_bytes": 0, 2024-03-08T21:16:03.656 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_items": 0, 2024-03-08T21:16:03.656 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_mapbl_bytes": 0, 2024-03-08T21:16:03.656 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_mapbl_items": 0, 2024-03-08T21:16:03.656 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_pglog_bytes": 0, 2024-03-08T21:16:03.656 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_pglog_items": 0, 2024-03-08T21:16:03.656 INFO:teuthology.orchestra.run.smithi149.stdout: "osdmap_bytes": 8352, 2024-03-08T21:16:03.656 INFO:teuthology.orchestra.run.smithi149.stdout: "osdmap_items": 190, 2024-03-08T21:16:03.657 INFO:teuthology.orchestra.run.smithi149.stdout: "osdmap_mapping_bytes": 0, 2024-03-08T21:16:03.657 INFO:teuthology.orchestra.run.smithi149.stdout: "osdmap_mapping_items": 0, 2024-03-08T21:16:03.657 INFO:teuthology.orchestra.run.smithi149.stdout: "pgmap_bytes": 0, 2024-03-08T21:16:03.657 INFO:teuthology.orchestra.run.smithi149.stdout: "pgmap_items": 0, 2024-03-08T21:16:03.657 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_co_bytes": 91141, 2024-03-08T21:16:03.657 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_co_items": 356, 2024-03-08T21:16:03.657 INFO:teuthology.orchestra.run.smithi149.stdout: "unittest_1_bytes": 0, 2024-03-08T21:16:03.657 INFO:teuthology.orchestra.run.smithi149.stdout: "unittest_1_items": 0, 2024-03-08T21:16:03.657 INFO:teuthology.orchestra.run.smithi149.stdout: "unittest_2_bytes": 0, 2024-03-08T21:16:03.657 INFO:teuthology.orchestra.run.smithi149.stdout: "unittest_2_items": 0 2024-03-08T21:16:03.657 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:16:03.657 INFO:teuthology.orchestra.run.smithi149.stdout: "objecter": { 2024-03-08T21:16:03.657 INFO:teuthology.orchestra.run.smithi149.stdout: "op_active": 0, 2024-03-08T21:16:03.658 INFO:teuthology.orchestra.run.smithi149.stdout: "op_laggy": 0, 2024-03-08T21:16:03.658 INFO:teuthology.orchestra.run.smithi149.stdout: "op_send": 45, 2024-03-08T21:16:03.658 INFO:teuthology.orchestra.run.smithi149.stdout: "op_send_bytes": 13022, 2024-03-08T21:16:03.658 INFO:teuthology.orchestra.run.smithi149.stdout: "op_resend": 31, 2024-03-08T21:16:03.658 INFO:teuthology.orchestra.run.smithi149.stdout: "op_reply": 45, 2024-03-08T21:16:03.658 INFO:teuthology.orchestra.run.smithi149.stdout: "op_latency": { 2024-03-08T21:16:03.658 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 45, 2024-03-08T21:16:03.658 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.505989826, 2024-03-08T21:16:03.658 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.011244218 2024-03-08T21:16:03.658 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:16:03.658 INFO:teuthology.orchestra.run.smithi149.stdout: "op_inflight": 0, 2024-03-08T21:16:03.659 INFO:teuthology.orchestra.run.smithi149.stdout: "oplen_avg": { 2024-03-08T21:16:03.659 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 45, 2024-03-08T21:16:03.659 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 68 2024-03-08T21:16:03.659 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:16:03.659 INFO:teuthology.orchestra.run.smithi149.stdout: "op": 45, 2024-03-08T21:16:03.659 INFO:teuthology.orchestra.run.smithi149.stdout: "op_r": 0, 2024-03-08T21:16:03.659 INFO:teuthology.orchestra.run.smithi149.stdout: "op_w": 45, 2024-03-08T21:16:03.659 INFO:teuthology.orchestra.run.smithi149.stdout: "op_rmw": 0, 2024-03-08T21:16:03.659 INFO:teuthology.orchestra.run.smithi149.stdout: "op_pg": 0, 2024-03-08T21:16:03.659 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_stat": 0, 2024-03-08T21:16:03.659 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_create": 10, 2024-03-08T21:16:03.659 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_read": 0, 2024-03-08T21:16:03.659 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_write": 7, 2024-03-08T21:16:03.660 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_writefull": 8, 2024-03-08T21:16:03.660 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_writesame": 0, 2024-03-08T21:16:03.660 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_append": 0, 2024-03-08T21:16:03.660 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_zero": 0, 2024-03-08T21:16:03.660 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_truncate": 0, 2024-03-08T21:16:03.660 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_delete": 6, 2024-03-08T21:16:03.660 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_mapext": 0, 2024-03-08T21:16:03.660 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_sparse_read": 0, 2024-03-08T21:16:03.660 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_clonerange": 0, 2024-03-08T21:16:03.660 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_getxattr": 0, 2024-03-08T21:16:03.660 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_setxattr": 20, 2024-03-08T21:16:03.660 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_cmpxattr": 0, 2024-03-08T21:16:03.660 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_rmxattr": 0, 2024-03-08T21:16:03.661 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_resetxattrs": 0, 2024-03-08T21:16:03.661 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_call": 0, 2024-03-08T21:16:03.661 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_watch": 0, 2024-03-08T21:16:03.661 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_notify": 0, 2024-03-08T21:16:03.661 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_src_cmpxattr": 0, 2024-03-08T21:16:03.661 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_pgls": 0, 2024-03-08T21:16:03.661 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_pgls_filter": 0, 2024-03-08T21:16:03.661 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_other": 0, 2024-03-08T21:16:03.661 INFO:teuthology.orchestra.run.smithi149.stdout: "linger_active": 0, 2024-03-08T21:16:03.661 INFO:teuthology.orchestra.run.smithi149.stdout: "linger_send": 0, 2024-03-08T21:16:03.661 INFO:teuthology.orchestra.run.smithi149.stdout: "linger_resend": 0, 2024-03-08T21:16:03.661 INFO:teuthology.orchestra.run.smithi149.stdout: "linger_ping": 0, 2024-03-08T21:16:03.662 INFO:teuthology.orchestra.run.smithi149.stdout: "poolop_active": 0, 2024-03-08T21:16:03.662 INFO:teuthology.orchestra.run.smithi149.stdout: "poolop_send": 0, 2024-03-08T21:16:03.662 INFO:teuthology.orchestra.run.smithi149.stdout: "poolop_resend": 0, 2024-03-08T21:16:03.662 INFO:teuthology.orchestra.run.smithi149.stdout: "poolstat_active": 0, 2024-03-08T21:16:03.662 INFO:teuthology.orchestra.run.smithi149.stdout: "poolstat_send": 0, 2024-03-08T21:16:03.662 INFO:teuthology.orchestra.run.smithi149.stdout: "poolstat_resend": 0, 2024-03-08T21:16:03.662 INFO:teuthology.orchestra.run.smithi149.stdout: "statfs_active": 0, 2024-03-08T21:16:03.662 INFO:teuthology.orchestra.run.smithi149.stdout: "statfs_send": 0, 2024-03-08T21:16:03.662 INFO:teuthology.orchestra.run.smithi149.stdout: "statfs_resend": 0, 2024-03-08T21:16:03.662 INFO:teuthology.orchestra.run.smithi149.stdout: "command_active": 0, 2024-03-08T21:16:03.662 INFO:teuthology.orchestra.run.smithi149.stdout: "command_send": 0, 2024-03-08T21:16:03.663 INFO:teuthology.orchestra.run.smithi149.stdout: "command_resend": 0, 2024-03-08T21:16:03.663 INFO:teuthology.orchestra.run.smithi149.stdout: "map_epoch": 113, 2024-03-08T21:16:03.663 INFO:teuthology.orchestra.run.smithi149.stdout: "map_full": 0, 2024-03-08T21:16:03.663 INFO:teuthology.orchestra.run.smithi149.stdout: "map_inc": 27, 2024-03-08T21:16:03.663 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_sessions": 9, 2024-03-08T21:16:03.663 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_session_open": 10, 2024-03-08T21:16:03.663 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_session_close": 1, 2024-03-08T21:16:03.663 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_laggy": 0, 2024-03-08T21:16:03.663 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_wr": 16, 2024-03-08T21:16:03.663 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_rd": 0, 2024-03-08T21:16:03.663 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_del": 1 2024-03-08T21:16:03.663 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:16:03.663 INFO:teuthology.orchestra.run.smithi149.stdout: "oft": { 2024-03-08T21:16:03.664 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_total_objs": 1, 2024-03-08T21:16:03.664 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_total_kv_pairs": 0, 2024-03-08T21:16:03.664 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_total_updates": 0, 2024-03-08T21:16:03.664 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_total_removes": 0 2024-03-08T21:16:03.664 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:16:03.664 INFO:teuthology.orchestra.run.smithi149.stdout: "osd-slow-ops": { 2024-03-08T21:16:03.664 INFO:teuthology.orchestra.run.smithi149.stdout: "slow_ops_count": 0 2024-03-08T21:16:03.664 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:16:03.664 INFO:teuthology.orchestra.run.smithi149.stdout: "purge_queue": { 2024-03-08T21:16:03.664 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_executing_ops": 0, 2024-03-08T21:16:03.664 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_executing_ops_high_water": 0, 2024-03-08T21:16:03.664 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_executing": 0, 2024-03-08T21:16:03.665 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_executing_high_water": 0, 2024-03-08T21:16:03.665 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_executed_ops": 0, 2024-03-08T21:16:03.665 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_executed": 0, 2024-03-08T21:16:03.665 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_item_in_journal": 0 2024-03-08T21:16:03.665 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:16:03.665 INFO:teuthology.orchestra.run.smithi149.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-08T21:16:03.665 INFO:teuthology.orchestra.run.smithi149.stdout: "val": 0, 2024-03-08T21:16:03.665 INFO:teuthology.orchestra.run.smithi149.stdout: "max": 104857600, 2024-03-08T21:16:03.665 INFO:teuthology.orchestra.run.smithi149.stdout: "get_started": 0, 2024-03-08T21:16:03.665 INFO:teuthology.orchestra.run.smithi149.stdout: "get": 1663, 2024-03-08T21:16:03.665 INFO:teuthology.orchestra.run.smithi149.stdout: "get_sum": 1852231, 2024-03-08T21:16:03.665 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_fail": 0, 2024-03-08T21:16:03.666 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_success": 1663, 2024-03-08T21:16:03.666 INFO:teuthology.orchestra.run.smithi149.stdout: "take": 0, 2024-03-08T21:16:03.666 INFO:teuthology.orchestra.run.smithi149.stdout: "take_sum": 0, 2024-03-08T21:16:03.666 INFO:teuthology.orchestra.run.smithi149.stdout: "put": 1663, 2024-03-08T21:16:03.666 INFO:teuthology.orchestra.run.smithi149.stdout: "put_sum": 1852231, 2024-03-08T21:16:03.666 INFO:teuthology.orchestra.run.smithi149.stdout: "wait": { 2024-03-08T21:16:03.666 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-08T21:16:03.666 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-08T21:16:03.666 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-08T21:16:03.666 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-03-08T21:16:03.666 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:16:03.666 INFO:teuthology.orchestra.run.smithi149.stdout: "throttle-objecter_bytes": { 2024-03-08T21:16:03.667 INFO:teuthology.orchestra.run.smithi149.stdout: "val": 0, 2024-03-08T21:16:03.667 INFO:teuthology.orchestra.run.smithi149.stdout: "max": 104857600, 2024-03-08T21:16:03.667 INFO:teuthology.orchestra.run.smithi149.stdout: "get_started": 0, 2024-03-08T21:16:03.667 INFO:teuthology.orchestra.run.smithi149.stdout: "get": 0, 2024-03-08T21:16:03.667 INFO:teuthology.orchestra.run.smithi149.stdout: "get_sum": 0, 2024-03-08T21:16:03.667 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_fail": 0, 2024-03-08T21:16:03.667 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_success": 0, 2024-03-08T21:16:03.667 INFO:teuthology.orchestra.run.smithi149.stdout: "take": 45, 2024-03-08T21:16:03.667 INFO:teuthology.orchestra.run.smithi149.stdout: "take_sum": 13022, 2024-03-08T21:16:03.667 INFO:teuthology.orchestra.run.smithi149.stdout: "put": 45, 2024-03-08T21:16:03.667 INFO:teuthology.orchestra.run.smithi149.stdout: "put_sum": 13022, 2024-03-08T21:16:03.667 INFO:teuthology.orchestra.run.smithi149.stdout: "wait": { 2024-03-08T21:16:03.668 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-08T21:16:03.668 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-08T21:16:03.668 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-08T21:16:03.668 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-03-08T21:16:03.668 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:16:03.668 INFO:teuthology.orchestra.run.smithi149.stdout: "throttle-objecter_ops": { 2024-03-08T21:16:03.668 INFO:teuthology.orchestra.run.smithi149.stdout: "val": 0, 2024-03-08T21:16:03.668 INFO:teuthology.orchestra.run.smithi149.stdout: "max": 1024, 2024-03-08T21:16:03.668 INFO:teuthology.orchestra.run.smithi149.stdout: "get_started": 0, 2024-03-08T21:16:03.668 INFO:teuthology.orchestra.run.smithi149.stdout: "get": 0, 2024-03-08T21:16:03.668 INFO:teuthology.orchestra.run.smithi149.stdout: "get_sum": 0, 2024-03-08T21:16:03.668 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_fail": 0, 2024-03-08T21:16:03.669 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_success": 0, 2024-03-08T21:16:03.669 INFO:teuthology.orchestra.run.smithi149.stdout: "take": 45, 2024-03-08T21:16:03.669 INFO:teuthology.orchestra.run.smithi149.stdout: "take_sum": 45, 2024-03-08T21:16:03.669 INFO:teuthology.orchestra.run.smithi149.stdout: "put": 45, 2024-03-08T21:16:03.669 INFO:teuthology.orchestra.run.smithi149.stdout: "put_sum": 45, 2024-03-08T21:16:03.669 INFO:teuthology.orchestra.run.smithi149.stdout: "wait": { 2024-03-08T21:16:03.669 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-08T21:16:03.669 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-08T21:16:03.669 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-08T21:16:03.669 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-03-08T21:16:03.669 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:16:03.669 INFO:teuthology.orchestra.run.smithi149.stdout: "throttle-write_buf_throttle": { 2024-03-08T21:16:03.670 INFO:teuthology.orchestra.run.smithi149.stdout: "val": 0, 2024-03-08T21:16:03.670 INFO:teuthology.orchestra.run.smithi149.stdout: "max": 3758096384, 2024-03-08T21:16:03.670 INFO:teuthology.orchestra.run.smithi149.stdout: "get_started": 0, 2024-03-08T21:16:03.670 INFO:teuthology.orchestra.run.smithi149.stdout: "get": 0, 2024-03-08T21:16:03.670 INFO:teuthology.orchestra.run.smithi149.stdout: "get_sum": 0, 2024-03-08T21:16:03.670 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_fail": 0, 2024-03-08T21:16:03.670 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_success": 0, 2024-03-08T21:16:03.670 INFO:teuthology.orchestra.run.smithi149.stdout: "take": 0, 2024-03-08T21:16:03.670 INFO:teuthology.orchestra.run.smithi149.stdout: "take_sum": 0, 2024-03-08T21:16:03.670 INFO:teuthology.orchestra.run.smithi149.stdout: "put": 0, 2024-03-08T21:16:03.670 INFO:teuthology.orchestra.run.smithi149.stdout: "put_sum": 0, 2024-03-08T21:16:03.670 INFO:teuthology.orchestra.run.smithi149.stdout: "wait": { 2024-03-08T21:16:03.671 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-08T21:16:03.671 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-08T21:16:03.671 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-08T21:16:03.671 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-03-08T21:16:03.671 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:16:03.671 INFO:teuthology.orchestra.run.smithi149.stdout: "throttle-write_buf_throttle-0x560ed2b446e0": { 2024-03-08T21:16:03.671 INFO:teuthology.orchestra.run.smithi149.stdout: "val": 0, 2024-03-08T21:16:03.671 INFO:teuthology.orchestra.run.smithi149.stdout: "max": 3758096384, 2024-03-08T21:16:03.671 INFO:teuthology.orchestra.run.smithi149.stdout: "get_started": 0, 2024-03-08T21:16:03.671 INFO:teuthology.orchestra.run.smithi149.stdout: "get": 8, 2024-03-08T21:16:03.671 INFO:teuthology.orchestra.run.smithi149.stdout: "get_sum": 2536, 2024-03-08T21:16:03.671 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_fail": 0, 2024-03-08T21:16:03.672 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_success": 8, 2024-03-08T21:16:03.672 INFO:teuthology.orchestra.run.smithi149.stdout: "take": 0, 2024-03-08T21:16:03.672 INFO:teuthology.orchestra.run.smithi149.stdout: "take_sum": 0, 2024-03-08T21:16:03.672 INFO:teuthology.orchestra.run.smithi149.stdout: "put": 7, 2024-03-08T21:16:03.672 INFO:teuthology.orchestra.run.smithi149.stdout: "put_sum": 2536, 2024-03-08T21:16:03.672 INFO:teuthology.orchestra.run.smithi149.stdout: "wait": { 2024-03-08T21:16:03.672 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-08T21:16:03.672 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-08T21:16:03.672 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-08T21:16:03.673 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-03-08T21:16:03.673 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-03-08T21:16:03.673 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-03-08T21:16:04.548 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-03-08T21:16:04.549 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-03-08T21:16:04.549 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update=0 2024-03-08T21:16:04.549 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update_receipt=18 2024-03-08T21:16:04.549 INFO:tasks.check_counter:Daemon mds.e mds.root_rsnaps=6 2024-03-08T21:16:04.549 INFO:tasks.check_counter:Daemon mds.e mds_server.req_mksnap_latency.avgcount=0 2024-03-08T21:16:04.549 INFO:tasks.check_counter:Daemon mds.e mds_server.req_rmsnap_latency.avgcount=0 2024-03-08T21:16:04.549 DEBUG:tasks.check_counter:Getting stats from h 2024-03-08T21:16:04.549 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761 shell --fsid dc20f0f8-dd8c-11ee-95c6-87774f69a715 -- ceph daemon mds.h perf dump 2024-03-08T21:16:04.758 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:16:04 smithi149 ceph-mon[35042]: pgmap v492: 97 pgs: 97 active+clean; 17 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-03-08T21:16:04.960 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:16:04 smithi107 ceph-mon[31381]: pgmap v492: 97 pgs: 97 active+clean; 17 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-03-08T21:16:04.999 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/mon.b/config 2024-03-08T21:16:05.121 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:16:04 smithi204 ceph-mon[34935]: pgmap v492: 97 pgs: 97 active+clean; 17 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-03-08T21:16:06.840 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:16:06 smithi149 ceph-mon[35042]: pgmap v493: 97 pgs: 97 active+clean; 17 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 47 KiB/s wr, 30 op/s 2024-03-08T21:16:06.960 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:16:06 smithi107 ceph-mon[31381]: pgmap v493: 97 pgs: 97 active+clean; 17 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 47 KiB/s wr, 30 op/s 2024-03-08T21:16:07.121 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:16:06 smithi204 ceph-mon[34935]: pgmap v493: 97 pgs: 97 active+clean; 17 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 47 KiB/s wr, 30 op/s 2024-03-08T21:16:07.177 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-03-08T21:16:07.178 INFO:teuthology.orchestra.run.smithi149.stdout: "AsyncMessenger::Worker-0": { 2024-03-08T21:16:07.178 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_messages": 1205, 2024-03-08T21:16:07.178 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages": 1031, 2024-03-08T21:16:07.178 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_bytes": 404074, 2024-03-08T21:16:07.178 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_bytes": 515919, 2024-03-08T21:16:07.178 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_created_connections": 8, 2024-03-08T21:16:07.178 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_active_connections": 3, 2024-03-08T21:16:07.178 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_total_time": 0.397770655, 2024-03-08T21:16:07.178 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_send_time": 0.209751087, 2024-03-08T21:16:07.179 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_recv_time": 0.201940041, 2024-03-08T21:16:07.179 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_fast_dispatch_time": 0.012983798, 2024-03-08T21:16:07.179 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages_queue_lat": { 2024-03-08T21:16:07.179 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 1029, 2024-03-08T21:16:07.179 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.199680246, 2024-03-08T21:16:07.179 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000194052 2024-03-08T21:16:07.179 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:16:07.179 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_handle_ack_lat": { 2024-03-08T21:16:07.179 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 2162, 2024-03-08T21:16:07.179 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.001094623, 2024-03-08T21:16:07.179 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000506 2024-03-08T21:16:07.179 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:16:07.180 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-08T21:16:07.180 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_encrypted_bytes": 0 2024-03-08T21:16:07.180 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:16:07.180 INFO:teuthology.orchestra.run.smithi149.stdout: "AsyncMessenger::Worker-1": { 2024-03-08T21:16:07.180 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_messages": 421, 2024-03-08T21:16:07.180 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages": 439, 2024-03-08T21:16:07.180 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_bytes": 1614718, 2024-03-08T21:16:07.180 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_bytes": 249321, 2024-03-08T21:16:07.180 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_created_connections": 13, 2024-03-08T21:16:07.180 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_active_connections": 4, 2024-03-08T21:16:07.180 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_total_time": 0.168531522, 2024-03-08T21:16:07.180 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_send_time": 0.053868480, 2024-03-08T21:16:07.181 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_recv_time": 0.123808470, 2024-03-08T21:16:07.181 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_fast_dispatch_time": 0.015421192, 2024-03-08T21:16:07.181 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages_queue_lat": { 2024-03-08T21:16:07.181 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 439, 2024-03-08T21:16:07.181 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.103399291, 2024-03-08T21:16:07.181 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000235533 2024-03-08T21:16:07.181 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:16:07.181 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_handle_ack_lat": { 2024-03-08T21:16:07.181 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 79, 2024-03-08T21:16:07.181 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000057048, 2024-03-08T21:16:07.181 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000722 2024-03-08T21:16:07.182 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:16:07.182 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_encrypted_bytes": 1608672, 2024-03-08T21:16:07.182 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_encrypted_bytes": 190240 2024-03-08T21:16:07.182 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:16:07.182 INFO:teuthology.orchestra.run.smithi149.stdout: "AsyncMessenger::Worker-2": { 2024-03-08T21:16:07.182 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_messages": 24, 2024-03-08T21:16:07.182 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages": 276, 2024-03-08T21:16:07.182 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_bytes": 5207, 2024-03-08T21:16:07.182 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_bytes": 603142, 2024-03-08T21:16:07.182 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_created_connections": 12, 2024-03-08T21:16:07.183 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_active_connections": 5, 2024-03-08T21:16:07.183 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_total_time": 0.049415312, 2024-03-08T21:16:07.183 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_send_time": 0.026187140, 2024-03-08T21:16:07.183 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_recv_time": 0.015323657, 2024-03-08T21:16:07.183 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_fast_dispatch_time": 0.000745113, 2024-03-08T21:16:07.183 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages_queue_lat": { 2024-03-08T21:16:07.183 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 273, 2024-03-08T21:16:07.183 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.055019277, 2024-03-08T21:16:07.183 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000201535 2024-03-08T21:16:07.184 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:16:07.184 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_handle_ack_lat": { 2024-03-08T21:16:07.184 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 50, 2024-03-08T21:16:07.184 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000029808, 2024-03-08T21:16:07.184 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000596 2024-03-08T21:16:07.184 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:16:07.184 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_encrypted_bytes": 2688, 2024-03-08T21:16:07.184 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_encrypted_bytes": 593568 2024-03-08T21:16:07.184 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:16:07.184 INFO:teuthology.orchestra.run.smithi149.stdout: "cct": { 2024-03-08T21:16:07.184 INFO:teuthology.orchestra.run.smithi149.stdout: "total_workers": 1, 2024-03-08T21:16:07.185 INFO:teuthology.orchestra.run.smithi149.stdout: "unhealthy_workers": 0 2024-03-08T21:16:07.185 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:16:07.185 INFO:teuthology.orchestra.run.smithi149.stdout: "finisher-MDSRank": { 2024-03-08T21:16:07.185 INFO:teuthology.orchestra.run.smithi149.stdout: "queue_len": 0, 2024-03-08T21:16:07.185 INFO:teuthology.orchestra.run.smithi149.stdout: "complete_latency": { 2024-03-08T21:16:07.185 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 50, 2024-03-08T21:16:07.185 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.002224655, 2024-03-08T21:16:07.185 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000044493 2024-03-08T21:16:07.185 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-03-08T21:16:07.185 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:16:07.185 INFO:teuthology.orchestra.run.smithi149.stdout: "finisher-PurgeQueue": { 2024-03-08T21:16:07.185 INFO:teuthology.orchestra.run.smithi149.stdout: "queue_len": 0, 2024-03-08T21:16:07.185 INFO:teuthology.orchestra.run.smithi149.stdout: "complete_latency": { 2024-03-08T21:16:07.185 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 2, 2024-03-08T21:16:07.186 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000014362, 2024-03-08T21:16:07.186 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000007181 2024-03-08T21:16:07.186 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-03-08T21:16:07.186 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:16:07.186 INFO:teuthology.orchestra.run.smithi149.stdout: "mds": { 2024-03-08T21:16:07.186 INFO:teuthology.orchestra.run.smithi149.stdout: "request": 2, 2024-03-08T21:16:07.186 INFO:teuthology.orchestra.run.smithi149.stdout: "reply": 2, 2024-03-08T21:16:07.186 INFO:teuthology.orchestra.run.smithi149.stdout: "reply_latency": { 2024-03-08T21:16:07.186 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 2, 2024-03-08T21:16:07.186 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.005497469, 2024-03-08T21:16:07.186 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.002748734 2024-03-08T21:16:07.186 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:16:07.186 INFO:teuthology.orchestra.run.smithi149.stdout: "slow_reply": 0, 2024-03-08T21:16:07.186 INFO:teuthology.orchestra.run.smithi149.stdout: "forward": 0, 2024-03-08T21:16:07.187 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_fetch_complete": 0, 2024-03-08T21:16:07.187 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_fetch_keys": 0, 2024-03-08T21:16:07.187 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_commit": 11, 2024-03-08T21:16:07.187 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_split": 0, 2024-03-08T21:16:07.187 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_merge": 0, 2024-03-08T21:16:07.187 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes": 18, 2024-03-08T21:16:07.187 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes_top": 2, 2024-03-08T21:16:07.187 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes_bottom": 16, 2024-03-08T21:16:07.187 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes_pin_tail": 0, 2024-03-08T21:16:07.187 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes_pinned": 13, 2024-03-08T21:16:07.187 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes_expired": 8, 2024-03-08T21:16:07.187 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes_with_caps": 2, 2024-03-08T21:16:07.187 INFO:teuthology.orchestra.run.smithi149.stdout: "caps": 2, 2024-03-08T21:16:07.188 INFO:teuthology.orchestra.run.smithi149.stdout: "subtrees": 3, 2024-03-08T21:16:07.188 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse": 14, 2024-03-08T21:16:07.188 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse_hit": 10, 2024-03-08T21:16:07.188 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse_forward": 0, 2024-03-08T21:16:07.188 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse_discover": 4, 2024-03-08T21:16:07.188 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse_dir_fetch": 0, 2024-03-08T21:16:07.188 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse_remote_ino": 0, 2024-03-08T21:16:07.188 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse_lock": 0, 2024-03-08T21:16:07.188 INFO:teuthology.orchestra.run.smithi149.stdout: "load_cent": 0, 2024-03-08T21:16:07.188 INFO:teuthology.orchestra.run.smithi149.stdout: "q": 0, 2024-03-08T21:16:07.188 INFO:teuthology.orchestra.run.smithi149.stdout: "exported": 0, 2024-03-08T21:16:07.188 INFO:teuthology.orchestra.run.smithi149.stdout: "exported_inodes": 0, 2024-03-08T21:16:07.188 INFO:teuthology.orchestra.run.smithi149.stdout: "imported": 0, 2024-03-08T21:16:07.189 INFO:teuthology.orchestra.run.smithi149.stdout: "imported_inodes": 0, 2024-03-08T21:16:07.189 INFO:teuthology.orchestra.run.smithi149.stdout: "openino_dir_fetch": 0, 2024-03-08T21:16:07.189 INFO:teuthology.orchestra.run.smithi149.stdout: "openino_backtrace_fetch": 0, 2024-03-08T21:16:07.189 INFO:teuthology.orchestra.run.smithi149.stdout: "openino_peer_discover": 0, 2024-03-08T21:16:07.189 INFO:teuthology.orchestra.run.smithi149.stdout: "root_rfiles": 5, 2024-03-08T21:16:07.189 INFO:teuthology.orchestra.run.smithi149.stdout: "root_rbytes": 4202218, 2024-03-08T21:16:07.189 INFO:teuthology.orchestra.run.smithi149.stdout: "root_rsnaps": 6, 2024-03-08T21:16:07.189 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_backtrace_fetch": 0, 2024-03-08T21:16:07.189 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_set_tag": 0, 2024-03-08T21:16:07.189 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_backtrace_repaired": 0, 2024-03-08T21:16:07.189 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_inotable_repaired": 0, 2024-03-08T21:16:07.189 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_dir_inodes": 0, 2024-03-08T21:16:07.189 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_dir_base_inodes": 0, 2024-03-08T21:16:07.189 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_dirfrag_rstats": 0, 2024-03-08T21:16:07.190 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_file_inodes": 0, 2024-03-08T21:16:07.190 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_inode_file_caps": 0, 2024-03-08T21:16:07.190 INFO:teuthology.orchestra.run.smithi149.stdout: "ceph_cap_op_revoke": 1, 2024-03-08T21:16:07.190 INFO:teuthology.orchestra.run.smithi149.stdout: "ceph_cap_op_grant": 0, 2024-03-08T21:16:07.190 INFO:teuthology.orchestra.run.smithi149.stdout: "ceph_cap_op_trunc": 0, 2024-03-08T21:16:07.190 INFO:teuthology.orchestra.run.smithi149.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-08T21:16:07.190 INFO:teuthology.orchestra.run.smithi149.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-08T21:16:07.190 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_client_caps": 1, 2024-03-08T21:16:07.190 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_client_caps_dirty": 0, 2024-03-08T21:16:07.190 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_client_cap_release": 0, 2024-03-08T21:16:07.190 INFO:teuthology.orchestra.run.smithi149.stdout: "process_request_cap_release": 0 2024-03-08T21:16:07.190 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:16:07.190 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_cache": { 2024-03-08T21:16:07.191 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_update": 0, 2024-03-08T21:16:07.191 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_update_receipt": 18, 2024-03-08T21:16:07.191 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_try_discover": 11, 2024-03-08T21:16:07.191 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_send_discover": 12, 2024-03-08T21:16:07.191 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_handle_discover": 0, 2024-03-08T21:16:07.191 INFO:teuthology.orchestra.run.smithi149.stdout: "num_strays": 0, 2024-03-08T21:16:07.191 INFO:teuthology.orchestra.run.smithi149.stdout: "num_strays_delayed": 0, 2024-03-08T21:16:07.191 INFO:teuthology.orchestra.run.smithi149.stdout: "num_strays_enqueuing": 0, 2024-03-08T21:16:07.191 INFO:teuthology.orchestra.run.smithi149.stdout: "strays_created": 0, 2024-03-08T21:16:07.191 INFO:teuthology.orchestra.run.smithi149.stdout: "strays_enqueued": 0, 2024-03-08T21:16:07.191 INFO:teuthology.orchestra.run.smithi149.stdout: "strays_reintegrated": 0, 2024-03-08T21:16:07.191 INFO:teuthology.orchestra.run.smithi149.stdout: "strays_migrated": 0, 2024-03-08T21:16:07.191 INFO:teuthology.orchestra.run.smithi149.stdout: "num_recovering_processing": 0, 2024-03-08T21:16:07.191 INFO:teuthology.orchestra.run.smithi149.stdout: "num_recovering_enqueued": 0, 2024-03-08T21:16:07.191 INFO:teuthology.orchestra.run.smithi149.stdout: "num_recovering_prioritized": 0, 2024-03-08T21:16:07.191 INFO:teuthology.orchestra.run.smithi149.stdout: "recovery_started": 0, 2024-03-08T21:16:07.192 INFO:teuthology.orchestra.run.smithi149.stdout: "recovery_completed": 0, 2024-03-08T21:16:07.192 INFO:teuthology.orchestra.run.smithi149.stdout: "ireq_enqueue_scrub": 0, 2024-03-08T21:16:07.192 INFO:teuthology.orchestra.run.smithi149.stdout: "ireq_exportdir": 0, 2024-03-08T21:16:07.192 INFO:teuthology.orchestra.run.smithi149.stdout: "ireq_flush": 0, 2024-03-08T21:16:07.192 INFO:teuthology.orchestra.run.smithi149.stdout: "ireq_fragmentdir": 0, 2024-03-08T21:16:07.192 INFO:teuthology.orchestra.run.smithi149.stdout: "ireq_fragstats": 0, 2024-03-08T21:16:07.192 INFO:teuthology.orchestra.run.smithi149.stdout: "ireq_inodestats": 0 2024-03-08T21:16:07.192 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:16:07.192 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_log": { 2024-03-08T21:16:07.192 INFO:teuthology.orchestra.run.smithi149.stdout: "evlrg": 0, 2024-03-08T21:16:07.192 INFO:teuthology.orchestra.run.smithi149.stdout: "evadd": 4, 2024-03-08T21:16:07.192 INFO:teuthology.orchestra.run.smithi149.stdout: "evex": 0, 2024-03-08T21:16:07.192 INFO:teuthology.orchestra.run.smithi149.stdout: "evtrm": 0, 2024-03-08T21:16:07.192 INFO:teuthology.orchestra.run.smithi149.stdout: "ev": 4, 2024-03-08T21:16:07.192 INFO:teuthology.orchestra.run.smithi149.stdout: "evexg": 0, 2024-03-08T21:16:07.193 INFO:teuthology.orchestra.run.smithi149.stdout: "evexd": 0, 2024-03-08T21:16:07.193 INFO:teuthology.orchestra.run.smithi149.stdout: "segadd": 2, 2024-03-08T21:16:07.193 INFO:teuthology.orchestra.run.smithi149.stdout: "segex": 0, 2024-03-08T21:16:07.193 INFO:teuthology.orchestra.run.smithi149.stdout: "segtrm": 0, 2024-03-08T21:16:07.193 INFO:teuthology.orchestra.run.smithi149.stdout: "seg": 2, 2024-03-08T21:16:07.193 INFO:teuthology.orchestra.run.smithi149.stdout: "segmjr": 2, 2024-03-08T21:16:07.193 INFO:teuthology.orchestra.run.smithi149.stdout: "segexg": 0, 2024-03-08T21:16:07.193 INFO:teuthology.orchestra.run.smithi149.stdout: "segexd": 0, 2024-03-08T21:16:07.193 INFO:teuthology.orchestra.run.smithi149.stdout: "expos": 4194304, 2024-03-08T21:16:07.193 INFO:teuthology.orchestra.run.smithi149.stdout: "wrpos": 4195776, 2024-03-08T21:16:07.193 INFO:teuthology.orchestra.run.smithi149.stdout: "rdpos": 0, 2024-03-08T21:16:07.193 INFO:teuthology.orchestra.run.smithi149.stdout: "jlat": { 2024-03-08T21:16:07.193 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 3, 2024-03-08T21:16:07.193 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.012091615, 2024-03-08T21:16:07.194 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.004030538 2024-03-08T21:16:07.194 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:16:07.194 INFO:teuthology.orchestra.run.smithi149.stdout: "replayed": 0 2024-03-08T21:16:07.194 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:16:07.194 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_mem": { 2024-03-08T21:16:07.194 INFO:teuthology.orchestra.run.smithi149.stdout: "ino": 22, 2024-03-08T21:16:07.194 INFO:teuthology.orchestra.run.smithi149.stdout: "ino+": 28, 2024-03-08T21:16:07.194 INFO:teuthology.orchestra.run.smithi149.stdout: "ino-": 6, 2024-03-08T21:16:07.194 INFO:teuthology.orchestra.run.smithi149.stdout: "dir": 21, 2024-03-08T21:16:07.194 INFO:teuthology.orchestra.run.smithi149.stdout: "dir+": 26, 2024-03-08T21:16:07.194 INFO:teuthology.orchestra.run.smithi149.stdout: "dir-": 5, 2024-03-08T21:16:07.194 INFO:teuthology.orchestra.run.smithi149.stdout: "dn": 18, 2024-03-08T21:16:07.194 INFO:teuthology.orchestra.run.smithi149.stdout: "dn+": 26, 2024-03-08T21:16:07.194 INFO:teuthology.orchestra.run.smithi149.stdout: "dn-": 8, 2024-03-08T21:16:07.194 INFO:teuthology.orchestra.run.smithi149.stdout: "cap": 2, 2024-03-08T21:16:07.195 INFO:teuthology.orchestra.run.smithi149.stdout: "cap+": 2, 2024-03-08T21:16:07.195 INFO:teuthology.orchestra.run.smithi149.stdout: "cap-": 0, 2024-03-08T21:16:07.195 INFO:teuthology.orchestra.run.smithi149.stdout: "rss": 50688, 2024-03-08T21:16:07.195 INFO:teuthology.orchestra.run.smithi149.stdout: "heap": 207228 2024-03-08T21:16:07.195 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:16:07.195 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_server": { 2024-03-08T21:16:07.195 INFO:teuthology.orchestra.run.smithi149.stdout: "dispatch_client_request": 5, 2024-03-08T21:16:07.195 INFO:teuthology.orchestra.run.smithi149.stdout: "dispatch_server_request": 0, 2024-03-08T21:16:07.195 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_client_request": 2, 2024-03-08T21:16:07.195 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_client_session": 24, 2024-03-08T21:16:07.195 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_peer_request": 1, 2024-03-08T21:16:07.195 INFO:teuthology.orchestra.run.smithi149.stdout: "req_create_latency": { 2024-03-08T21:16:07.195 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-08T21:16:07.196 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-08T21:16:07.196 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-08T21:16:07.196 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:16:07.196 INFO:teuthology.orchestra.run.smithi149.stdout: "req_getattr_latency": { 2024-03-08T21:16:07.196 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 2, 2024-03-08T21:16:07.196 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.005497469, 2024-03-08T21:16:07.196 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.002748734 2024-03-08T21:16:07.196 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:16:07.196 INFO:teuthology.orchestra.run.smithi149.stdout: "req_getfilelock_latency": { 2024-03-08T21:16:07.196 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-08T21:16:07.196 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-08T21:16:07.196 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-08T21:16:07.196 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:16:07.197 INFO:teuthology.orchestra.run.smithi149.stdout: "req_link_latency": { 2024-03-08T21:16:07.197 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-08T21:16:07.197 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-08T21:16:07.197 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-08T21:16:07.197 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:16:07.197 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lookup_latency": { 2024-03-08T21:16:07.197 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-08T21:16:07.197 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-08T21:16:07.197 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-08T21:16:07.197 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:16:07.197 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lookuphash_latency": { 2024-03-08T21:16:07.197 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-08T21:16:07.197 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-08T21:16:07.197 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-08T21:16:07.198 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:16:07.198 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lookupino_latency": { 2024-03-08T21:16:07.198 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-08T21:16:07.198 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-08T21:16:07.198 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-08T21:16:07.198 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:16:07.198 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lookupname_latency": { 2024-03-08T21:16:07.198 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-08T21:16:07.198 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-08T21:16:07.198 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-08T21:16:07.198 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:16:07.198 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lookupparent_latency": { 2024-03-08T21:16:07.198 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-08T21:16:07.198 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-08T21:16:07.199 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-08T21:16:07.199 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:16:07.199 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lookupsnap_latency": { 2024-03-08T21:16:07.199 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-08T21:16:07.199 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-08T21:16:07.199 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-08T21:16:07.199 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:16:07.199 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lssnap_latency": { 2024-03-08T21:16:07.199 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-08T21:16:07.199 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-08T21:16:07.199 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-08T21:16:07.199 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:16:07.199 INFO:teuthology.orchestra.run.smithi149.stdout: "req_mkdir_latency": { 2024-03-08T21:16:07.199 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-08T21:16:07.199 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-08T21:16:07.200 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-08T21:16:07.200 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:16:07.200 INFO:teuthology.orchestra.run.smithi149.stdout: "req_mknod_latency": { 2024-03-08T21:16:07.200 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-08T21:16:07.200 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-08T21:16:07.200 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-08T21:16:07.201 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:16:07.201 INFO:teuthology.orchestra.run.smithi149.stdout: "req_mksnap_latency": { 2024-03-08T21:16:07.201 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-08T21:16:07.201 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-08T21:16:07.201 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-08T21:16:07.201 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:16:07.201 INFO:teuthology.orchestra.run.smithi149.stdout: "req_open_latency": { 2024-03-08T21:16:07.201 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-08T21:16:07.201 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-08T21:16:07.201 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-08T21:16:07.201 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:16:07.201 INFO:teuthology.orchestra.run.smithi149.stdout: "req_readdir_latency": { 2024-03-08T21:16:07.201 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-08T21:16:07.201 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-08T21:16:07.201 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-08T21:16:07.202 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:16:07.202 INFO:teuthology.orchestra.run.smithi149.stdout: "req_rename_latency": { 2024-03-08T21:16:07.202 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-08T21:16:07.202 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-08T21:16:07.202 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-08T21:16:07.202 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:16:07.202 INFO:teuthology.orchestra.run.smithi149.stdout: "req_renamesnap_latency": { 2024-03-08T21:16:07.202 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-08T21:16:07.202 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-08T21:16:07.202 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-08T21:16:07.202 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:16:07.202 INFO:teuthology.orchestra.run.smithi149.stdout: "req_snapdiff_latency": { 2024-03-08T21:16:07.202 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-08T21:16:07.203 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-08T21:16:07.203 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-08T21:16:07.203 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:16:07.203 INFO:teuthology.orchestra.run.smithi149.stdout: "req_rmdir_latency": { 2024-03-08T21:16:07.203 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-08T21:16:07.203 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-08T21:16:07.203 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-08T21:16:07.203 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:16:07.203 INFO:teuthology.orchestra.run.smithi149.stdout: "req_rmsnap_latency": { 2024-03-08T21:16:07.203 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-08T21:16:07.203 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-08T21:16:07.203 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-08T21:16:07.203 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:16:07.203 INFO:teuthology.orchestra.run.smithi149.stdout: "req_rmxattr_latency": { 2024-03-08T21:16:07.204 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-08T21:16:07.204 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-08T21:16:07.204 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-08T21:16:07.204 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:16:07.204 INFO:teuthology.orchestra.run.smithi149.stdout: "req_setattr_latency": { 2024-03-08T21:16:07.204 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-08T21:16:07.204 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-08T21:16:07.204 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-08T21:16:07.204 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:16:07.204 INFO:teuthology.orchestra.run.smithi149.stdout: "req_setdirlayout_latency": { 2024-03-08T21:16:07.204 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-08T21:16:07.204 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-08T21:16:07.204 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-08T21:16:07.204 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:16:07.204 INFO:teuthology.orchestra.run.smithi149.stdout: "req_setfilelock_latency": { 2024-03-08T21:16:07.204 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-08T21:16:07.205 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-08T21:16:07.205 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-08T21:16:07.205 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:16:07.205 INFO:teuthology.orchestra.run.smithi149.stdout: "req_setlayout_latency": { 2024-03-08T21:16:07.205 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-08T21:16:07.205 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-08T21:16:07.205 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-08T21:16:07.205 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:16:07.205 INFO:teuthology.orchestra.run.smithi149.stdout: "req_setxattr_latency": { 2024-03-08T21:16:07.205 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-08T21:16:07.205 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-08T21:16:07.205 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-08T21:16:07.205 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:16:07.205 INFO:teuthology.orchestra.run.smithi149.stdout: "req_symlink_latency": { 2024-03-08T21:16:07.206 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-08T21:16:07.206 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-08T21:16:07.206 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-08T21:16:07.206 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:16:07.206 INFO:teuthology.orchestra.run.smithi149.stdout: "req_unlink_latency": { 2024-03-08T21:16:07.206 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-08T21:16:07.206 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-08T21:16:07.206 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-08T21:16:07.206 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:16:07.206 INFO:teuthology.orchestra.run.smithi149.stdout: "cap_revoke_eviction": 0, 2024-03-08T21:16:07.206 INFO:teuthology.orchestra.run.smithi149.stdout: "cap_acquisition_throttle": 0, 2024-03-08T21:16:07.206 INFO:teuthology.orchestra.run.smithi149.stdout: "req_getvxattr_latency": { 2024-03-08T21:16:07.206 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-08T21:16:07.206 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-08T21:16:07.207 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-08T21:16:07.207 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-03-08T21:16:07.207 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:16:07.207 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_sessions": { 2024-03-08T21:16:07.207 INFO:teuthology.orchestra.run.smithi149.stdout: "session_count": 2, 2024-03-08T21:16:07.207 INFO:teuthology.orchestra.run.smithi149.stdout: "session_add": 2, 2024-03-08T21:16:07.207 INFO:teuthology.orchestra.run.smithi149.stdout: "session_remove": 0, 2024-03-08T21:16:07.207 INFO:teuthology.orchestra.run.smithi149.stdout: "sessions_open": 2, 2024-03-08T21:16:07.207 INFO:teuthology.orchestra.run.smithi149.stdout: "sessions_stale": 0, 2024-03-08T21:16:07.207 INFO:teuthology.orchestra.run.smithi149.stdout: "total_load": 1, 2024-03-08T21:16:07.207 INFO:teuthology.orchestra.run.smithi149.stdout: "average_load": 1, 2024-03-08T21:16:07.207 INFO:teuthology.orchestra.run.smithi149.stdout: "avg_session_uptime": 923, 2024-03-08T21:16:07.207 INFO:teuthology.orchestra.run.smithi149.stdout: "mdthresh_evicted": 0 2024-03-08T21:16:07.208 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:16:07.208 INFO:teuthology.orchestra.run.smithi149.stdout: "mempool": { 2024-03-08T21:16:07.208 INFO:teuthology.orchestra.run.smithi149.stdout: "bloom_filter_bytes": 0, 2024-03-08T21:16:07.208 INFO:teuthology.orchestra.run.smithi149.stdout: "bloom_filter_items": 0, 2024-03-08T21:16:07.208 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_alloc_bytes": 0, 2024-03-08T21:16:07.208 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_alloc_items": 0, 2024-03-08T21:16:07.208 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_data_bytes": 0, 2024-03-08T21:16:07.208 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_data_items": 0, 2024-03-08T21:16:07.208 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-08T21:16:07.208 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_onode_items": 0, 2024-03-08T21:16:07.208 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-08T21:16:07.208 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_meta_items": 0, 2024-03-08T21:16:07.208 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_other_bytes": 0, 2024-03-08T21:16:07.208 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_other_items": 0, 2024-03-08T21:16:07.208 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-08T21:16:07.209 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_buffer_items": 0, 2024-03-08T21:16:07.209 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_extent_bytes": 0, 2024-03-08T21:16:07.209 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_extent_items": 0, 2024-03-08T21:16:07.209 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_blob_bytes": 0, 2024-03-08T21:16:07.209 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_blob_items": 0, 2024-03-08T21:16:07.209 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-08T21:16:07.209 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_shared_blob_items": 0, 2024-03-08T21:16:07.209 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-08T21:16:07.209 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_inline_bl_items": 0, 2024-03-08T21:16:07.209 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_fsck_bytes": 0, 2024-03-08T21:16:07.209 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_fsck_items": 0, 2024-03-08T21:16:07.209 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_txc_bytes": 0, 2024-03-08T21:16:07.209 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_txc_items": 0, 2024-03-08T21:16:07.209 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-08T21:16:07.210 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_writing_deferred_items": 0, 2024-03-08T21:16:07.210 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_writing_bytes": 0, 2024-03-08T21:16:07.210 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_writing_items": 0, 2024-03-08T21:16:07.210 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_bytes": 0, 2024-03-08T21:16:07.210 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_items": 0, 2024-03-08T21:16:07.210 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_file_reader_bytes": 0, 2024-03-08T21:16:07.210 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_file_reader_items": 0, 2024-03-08T21:16:07.210 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_file_writer_bytes": 0, 2024-03-08T21:16:07.210 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_file_writer_items": 0, 2024-03-08T21:16:07.210 INFO:teuthology.orchestra.run.smithi149.stdout: "buffer_anon_bytes": 255281, 2024-03-08T21:16:07.210 INFO:teuthology.orchestra.run.smithi149.stdout: "buffer_anon_items": 84, 2024-03-08T21:16:07.210 INFO:teuthology.orchestra.run.smithi149.stdout: "buffer_meta_bytes": 0, 2024-03-08T21:16:07.210 INFO:teuthology.orchestra.run.smithi149.stdout: "buffer_meta_items": 0, 2024-03-08T21:16:07.211 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_bytes": 0, 2024-03-08T21:16:07.211 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_items": 0, 2024-03-08T21:16:07.211 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_mapbl_bytes": 0, 2024-03-08T21:16:07.211 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_mapbl_items": 0, 2024-03-08T21:16:07.211 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_pglog_bytes": 0, 2024-03-08T21:16:07.211 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_pglog_items": 0, 2024-03-08T21:16:07.211 INFO:teuthology.orchestra.run.smithi149.stdout: "osdmap_bytes": 8352, 2024-03-08T21:16:07.211 INFO:teuthology.orchestra.run.smithi149.stdout: "osdmap_items": 190, 2024-03-08T21:16:07.211 INFO:teuthology.orchestra.run.smithi149.stdout: "osdmap_mapping_bytes": 0, 2024-03-08T21:16:07.211 INFO:teuthology.orchestra.run.smithi149.stdout: "osdmap_mapping_items": 0, 2024-03-08T21:16:07.211 INFO:teuthology.orchestra.run.smithi149.stdout: "pgmap_bytes": 0, 2024-03-08T21:16:07.211 INFO:teuthology.orchestra.run.smithi149.stdout: "pgmap_items": 0, 2024-03-08T21:16:07.211 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_co_bytes": 91181, 2024-03-08T21:16:07.211 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_co_items": 361, 2024-03-08T21:16:07.211 INFO:teuthology.orchestra.run.smithi149.stdout: "unittest_1_bytes": 0, 2024-03-08T21:16:07.212 INFO:teuthology.orchestra.run.smithi149.stdout: "unittest_1_items": 0, 2024-03-08T21:16:07.212 INFO:teuthology.orchestra.run.smithi149.stdout: "unittest_2_bytes": 0, 2024-03-08T21:16:07.212 INFO:teuthology.orchestra.run.smithi149.stdout: "unittest_2_items": 0 2024-03-08T21:16:07.212 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:16:07.212 INFO:teuthology.orchestra.run.smithi149.stdout: "objecter": { 2024-03-08T21:16:07.212 INFO:teuthology.orchestra.run.smithi149.stdout: "op_active": 0, 2024-03-08T21:16:07.212 INFO:teuthology.orchestra.run.smithi149.stdout: "op_laggy": 0, 2024-03-08T21:16:07.212 INFO:teuthology.orchestra.run.smithi149.stdout: "op_send": 40, 2024-03-08T21:16:07.212 INFO:teuthology.orchestra.run.smithi149.stdout: "op_send_bytes": 11929, 2024-03-08T21:16:07.212 INFO:teuthology.orchestra.run.smithi149.stdout: "op_resend": 27, 2024-03-08T21:16:07.212 INFO:teuthology.orchestra.run.smithi149.stdout: "op_reply": 40, 2024-03-08T21:16:07.212 INFO:teuthology.orchestra.run.smithi149.stdout: "op_latency": { 2024-03-08T21:16:07.212 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 40, 2024-03-08T21:16:07.212 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.527989403, 2024-03-08T21:16:07.212 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.013199735 2024-03-08T21:16:07.213 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:16:07.213 INFO:teuthology.orchestra.run.smithi149.stdout: "op_inflight": 0, 2024-03-08T21:16:07.213 INFO:teuthology.orchestra.run.smithi149.stdout: "oplen_avg": { 2024-03-08T21:16:07.213 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 40, 2024-03-08T21:16:07.213 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 63 2024-03-08T21:16:07.213 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:16:07.213 INFO:teuthology.orchestra.run.smithi149.stdout: "op": 40, 2024-03-08T21:16:07.213 INFO:teuthology.orchestra.run.smithi149.stdout: "op_r": 0, 2024-03-08T21:16:07.213 INFO:teuthology.orchestra.run.smithi149.stdout: "op_w": 40, 2024-03-08T21:16:07.213 INFO:teuthology.orchestra.run.smithi149.stdout: "op_rmw": 0, 2024-03-08T21:16:07.213 INFO:teuthology.orchestra.run.smithi149.stdout: "op_pg": 0, 2024-03-08T21:16:07.213 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_stat": 0, 2024-03-08T21:16:07.213 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_create": 10, 2024-03-08T21:16:07.213 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_read": 0, 2024-03-08T21:16:07.213 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_write": 3, 2024-03-08T21:16:07.213 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_writefull": 7, 2024-03-08T21:16:07.213 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_writesame": 0, 2024-03-08T21:16:07.214 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_append": 0, 2024-03-08T21:16:07.214 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_zero": 0, 2024-03-08T21:16:07.214 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_truncate": 0, 2024-03-08T21:16:07.214 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_delete": 6, 2024-03-08T21:16:07.214 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_mapext": 0, 2024-03-08T21:16:07.214 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_sparse_read": 0, 2024-03-08T21:16:07.214 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_clonerange": 0, 2024-03-08T21:16:07.214 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_getxattr": 0, 2024-03-08T21:16:07.214 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_setxattr": 20, 2024-03-08T21:16:07.214 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_cmpxattr": 0, 2024-03-08T21:16:07.214 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_rmxattr": 0, 2024-03-08T21:16:07.214 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_resetxattrs": 0, 2024-03-08T21:16:07.214 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_call": 0, 2024-03-08T21:16:07.214 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_watch": 0, 2024-03-08T21:16:07.215 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_notify": 0, 2024-03-08T21:16:07.215 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_src_cmpxattr": 0, 2024-03-08T21:16:07.215 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_pgls": 0, 2024-03-08T21:16:07.215 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_pgls_filter": 0, 2024-03-08T21:16:07.215 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_other": 0, 2024-03-08T21:16:07.215 INFO:teuthology.orchestra.run.smithi149.stdout: "linger_active": 0, 2024-03-08T21:16:07.215 INFO:teuthology.orchestra.run.smithi149.stdout: "linger_send": 0, 2024-03-08T21:16:07.215 INFO:teuthology.orchestra.run.smithi149.stdout: "linger_resend": 0, 2024-03-08T21:16:07.215 INFO:teuthology.orchestra.run.smithi149.stdout: "linger_ping": 0, 2024-03-08T21:16:07.215 INFO:teuthology.orchestra.run.smithi149.stdout: "poolop_active": 0, 2024-03-08T21:16:07.215 INFO:teuthology.orchestra.run.smithi149.stdout: "poolop_send": 0, 2024-03-08T21:16:07.215 INFO:teuthology.orchestra.run.smithi149.stdout: "poolop_resend": 0, 2024-03-08T21:16:07.215 INFO:teuthology.orchestra.run.smithi149.stdout: "poolstat_active": 0, 2024-03-08T21:16:07.215 INFO:teuthology.orchestra.run.smithi149.stdout: "poolstat_send": 0, 2024-03-08T21:16:07.215 INFO:teuthology.orchestra.run.smithi149.stdout: "poolstat_resend": 0, 2024-03-08T21:16:07.216 INFO:teuthology.orchestra.run.smithi149.stdout: "statfs_active": 0, 2024-03-08T21:16:07.216 INFO:teuthology.orchestra.run.smithi149.stdout: "statfs_send": 0, 2024-03-08T21:16:07.216 INFO:teuthology.orchestra.run.smithi149.stdout: "statfs_resend": 0, 2024-03-08T21:16:07.216 INFO:teuthology.orchestra.run.smithi149.stdout: "command_active": 0, 2024-03-08T21:16:07.216 INFO:teuthology.orchestra.run.smithi149.stdout: "command_send": 0, 2024-03-08T21:16:07.216 INFO:teuthology.orchestra.run.smithi149.stdout: "command_resend": 0, 2024-03-08T21:16:07.216 INFO:teuthology.orchestra.run.smithi149.stdout: "map_epoch": 113, 2024-03-08T21:16:07.216 INFO:teuthology.orchestra.run.smithi149.stdout: "map_full": 0, 2024-03-08T21:16:07.216 INFO:teuthology.orchestra.run.smithi149.stdout: "map_inc": 27, 2024-03-08T21:16:07.216 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_sessions": 9, 2024-03-08T21:16:07.216 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_session_open": 13, 2024-03-08T21:16:07.216 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_session_close": 4, 2024-03-08T21:16:07.216 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_laggy": 0, 2024-03-08T21:16:07.216 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_wr": 16, 2024-03-08T21:16:07.216 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_rd": 0, 2024-03-08T21:16:07.216 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_del": 1 2024-03-08T21:16:07.217 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:16:07.217 INFO:teuthology.orchestra.run.smithi149.stdout: "oft": { 2024-03-08T21:16:07.217 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_total_objs": 1, 2024-03-08T21:16:07.217 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_total_kv_pairs": 0, 2024-03-08T21:16:07.217 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_total_updates": 0, 2024-03-08T21:16:07.217 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_total_removes": 0 2024-03-08T21:16:07.217 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:16:07.217 INFO:teuthology.orchestra.run.smithi149.stdout: "osd-slow-ops": { 2024-03-08T21:16:07.217 INFO:teuthology.orchestra.run.smithi149.stdout: "slow_ops_count": 0 2024-03-08T21:16:07.217 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:16:07.217 INFO:teuthology.orchestra.run.smithi149.stdout: "purge_queue": { 2024-03-08T21:16:07.217 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_executing_ops": 0, 2024-03-08T21:16:07.217 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_executing_ops_high_water": 0, 2024-03-08T21:16:07.217 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_executing": 0, 2024-03-08T21:16:07.217 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_executing_high_water": 0, 2024-03-08T21:16:07.217 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_executed_ops": 0, 2024-03-08T21:16:07.218 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_executed": 0, 2024-03-08T21:16:07.218 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_item_in_journal": 0 2024-03-08T21:16:07.218 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:16:07.218 INFO:teuthology.orchestra.run.smithi149.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-08T21:16:07.218 INFO:teuthology.orchestra.run.smithi149.stdout: "val": 0, 2024-03-08T21:16:07.218 INFO:teuthology.orchestra.run.smithi149.stdout: "max": 104857600, 2024-03-08T21:16:07.218 INFO:teuthology.orchestra.run.smithi149.stdout: "get_started": 0, 2024-03-08T21:16:07.218 INFO:teuthology.orchestra.run.smithi149.stdout: "get": 1646, 2024-03-08T21:16:07.218 INFO:teuthology.orchestra.run.smithi149.stdout: "get_sum": 1852755, 2024-03-08T21:16:07.218 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_fail": 0, 2024-03-08T21:16:07.218 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_success": 1646, 2024-03-08T21:16:07.218 INFO:teuthology.orchestra.run.smithi149.stdout: "take": 0, 2024-03-08T21:16:07.218 INFO:teuthology.orchestra.run.smithi149.stdout: "take_sum": 0, 2024-03-08T21:16:07.218 INFO:teuthology.orchestra.run.smithi149.stdout: "put": 1646, 2024-03-08T21:16:07.218 INFO:teuthology.orchestra.run.smithi149.stdout: "put_sum": 1852755, 2024-03-08T21:16:07.219 INFO:teuthology.orchestra.run.smithi149.stdout: "wait": { 2024-03-08T21:16:07.219 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-08T21:16:07.219 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-08T21:16:07.219 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-08T21:16:07.219 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-03-08T21:16:07.219 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:16:07.219 INFO:teuthology.orchestra.run.smithi149.stdout: "throttle-objecter_bytes": { 2024-03-08T21:16:07.219 INFO:teuthology.orchestra.run.smithi149.stdout: "val": 0, 2024-03-08T21:16:07.219 INFO:teuthology.orchestra.run.smithi149.stdout: "max": 104857600, 2024-03-08T21:16:07.219 INFO:teuthology.orchestra.run.smithi149.stdout: "get_started": 0, 2024-03-08T21:16:07.219 INFO:teuthology.orchestra.run.smithi149.stdout: "get": 0, 2024-03-08T21:16:07.219 INFO:teuthology.orchestra.run.smithi149.stdout: "get_sum": 0, 2024-03-08T21:16:07.219 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_fail": 0, 2024-03-08T21:16:07.219 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_success": 0, 2024-03-08T21:16:07.219 INFO:teuthology.orchestra.run.smithi149.stdout: "take": 40, 2024-03-08T21:16:07.220 INFO:teuthology.orchestra.run.smithi149.stdout: "take_sum": 11929, 2024-03-08T21:16:07.220 INFO:teuthology.orchestra.run.smithi149.stdout: "put": 40, 2024-03-08T21:16:07.220 INFO:teuthology.orchestra.run.smithi149.stdout: "put_sum": 11929, 2024-03-08T21:16:07.220 INFO:teuthology.orchestra.run.smithi149.stdout: "wait": { 2024-03-08T21:16:07.220 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-08T21:16:07.220 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-08T21:16:07.220 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-08T21:16:07.220 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-03-08T21:16:07.220 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:16:07.220 INFO:teuthology.orchestra.run.smithi149.stdout: "throttle-objecter_ops": { 2024-03-08T21:16:07.220 INFO:teuthology.orchestra.run.smithi149.stdout: "val": 0, 2024-03-08T21:16:07.220 INFO:teuthology.orchestra.run.smithi149.stdout: "max": 1024, 2024-03-08T21:16:07.220 INFO:teuthology.orchestra.run.smithi149.stdout: "get_started": 0, 2024-03-08T21:16:07.220 INFO:teuthology.orchestra.run.smithi149.stdout: "get": 0, 2024-03-08T21:16:07.220 INFO:teuthology.orchestra.run.smithi149.stdout: "get_sum": 0, 2024-03-08T21:16:07.221 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_fail": 0, 2024-03-08T21:16:07.221 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_success": 0, 2024-03-08T21:16:07.221 INFO:teuthology.orchestra.run.smithi149.stdout: "take": 40, 2024-03-08T21:16:07.221 INFO:teuthology.orchestra.run.smithi149.stdout: "take_sum": 40, 2024-03-08T21:16:07.221 INFO:teuthology.orchestra.run.smithi149.stdout: "put": 40, 2024-03-08T21:16:07.221 INFO:teuthology.orchestra.run.smithi149.stdout: "put_sum": 40, 2024-03-08T21:16:07.221 INFO:teuthology.orchestra.run.smithi149.stdout: "wait": { 2024-03-08T21:16:07.221 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-08T21:16:07.221 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-08T21:16:07.221 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-08T21:16:07.221 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-03-08T21:16:07.221 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:16:07.221 INFO:teuthology.orchestra.run.smithi149.stdout: "throttle-write_buf_throttle": { 2024-03-08T21:16:07.221 INFO:teuthology.orchestra.run.smithi149.stdout: "val": 0, 2024-03-08T21:16:07.221 INFO:teuthology.orchestra.run.smithi149.stdout: "max": 3758096384, 2024-03-08T21:16:07.222 INFO:teuthology.orchestra.run.smithi149.stdout: "get_started": 0, 2024-03-08T21:16:07.222 INFO:teuthology.orchestra.run.smithi149.stdout: "get": 0, 2024-03-08T21:16:07.222 INFO:teuthology.orchestra.run.smithi149.stdout: "get_sum": 0, 2024-03-08T21:16:07.222 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_fail": 0, 2024-03-08T21:16:07.222 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_success": 0, 2024-03-08T21:16:07.222 INFO:teuthology.orchestra.run.smithi149.stdout: "take": 0, 2024-03-08T21:16:07.222 INFO:teuthology.orchestra.run.smithi149.stdout: "take_sum": 0, 2024-03-08T21:16:07.222 INFO:teuthology.orchestra.run.smithi149.stdout: "put": 0, 2024-03-08T21:16:07.222 INFO:teuthology.orchestra.run.smithi149.stdout: "put_sum": 0, 2024-03-08T21:16:07.222 INFO:teuthology.orchestra.run.smithi149.stdout: "wait": { 2024-03-08T21:16:07.222 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-08T21:16:07.222 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-08T21:16:07.222 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-08T21:16:07.222 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-03-08T21:16:07.222 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:16:07.223 INFO:teuthology.orchestra.run.smithi149.stdout: "throttle-write_buf_throttle-0x55aeb51ca6e0": { 2024-03-08T21:16:07.223 INFO:teuthology.orchestra.run.smithi149.stdout: "val": 0, 2024-03-08T21:16:07.223 INFO:teuthology.orchestra.run.smithi149.stdout: "max": 3758096384, 2024-03-08T21:16:07.223 INFO:teuthology.orchestra.run.smithi149.stdout: "get_started": 0, 2024-03-08T21:16:07.223 INFO:teuthology.orchestra.run.smithi149.stdout: "get": 4, 2024-03-08T21:16:07.223 INFO:teuthology.orchestra.run.smithi149.stdout: "get_sum": 1472, 2024-03-08T21:16:07.223 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_fail": 0, 2024-03-08T21:16:07.223 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_success": 4, 2024-03-08T21:16:07.223 INFO:teuthology.orchestra.run.smithi149.stdout: "take": 0, 2024-03-08T21:16:07.223 INFO:teuthology.orchestra.run.smithi149.stdout: "take_sum": 0, 2024-03-08T21:16:07.223 INFO:teuthology.orchestra.run.smithi149.stdout: "put": 3, 2024-03-08T21:16:07.223 INFO:teuthology.orchestra.run.smithi149.stdout: "put_sum": 1472, 2024-03-08T21:16:07.223 INFO:teuthology.orchestra.run.smithi149.stdout: "wait": { 2024-03-08T21:16:07.223 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-08T21:16:07.223 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-08T21:16:07.223 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-08T21:16:07.224 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-03-08T21:16:07.224 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-03-08T21:16:07.224 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-03-08T21:16:08.056 INFO:tasks.check_counter:Daemon mds.h mds.exported=0 2024-03-08T21:16:08.056 INFO:tasks.check_counter:Daemon mds.h mds.imported=0 2024-03-08T21:16:08.056 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update=0 2024-03-08T21:16:08.056 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update_receipt=18 2024-03-08T21:16:08.056 INFO:tasks.check_counter:Daemon mds.h mds.root_rsnaps=6 2024-03-08T21:16:08.056 INFO:tasks.check_counter:Daemon mds.h mds_server.req_mksnap_latency.avgcount=0 2024-03-08T21:16:08.056 INFO:tasks.check_counter:Daemon mds.h mds_server.req_rmsnap_latency.avgcount=0 2024-03-08T21:16:08.056 DEBUG:tasks.check_counter:Getting stats from k 2024-03-08T21:16:08.057 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761 shell --fsid dc20f0f8-dd8c-11ee-95c6-87774f69a715 -- ceph daemon mds.k perf dump 2024-03-08T21:16:08.506 INFO:teuthology.orchestra.run.smithi149.stderr:Inferring config /var/lib/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/mon.b/config 2024-03-08T21:16:08.924 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:16:08 smithi204 ceph-mon[34935]: pgmap v494: 97 pgs: 97 active+clean; 17 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 46 KiB/s wr, 25 op/s 2024-03-08T21:16:08.949 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:16:08 smithi149 ceph-mon[35042]: pgmap v494: 97 pgs: 97 active+clean; 17 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 46 KiB/s wr, 25 op/s 2024-03-08T21:16:08.960 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:16:08 smithi107 ceph-mon[31381]: pgmap v494: 97 pgs: 97 active+clean; 17 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 46 KiB/s wr, 25 op/s 2024-03-08T21:16:10.644 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-03-08T21:16:10.644 INFO:teuthology.orchestra.run.smithi149.stdout: "AsyncMessenger::Worker-0": { 2024-03-08T21:16:10.644 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_messages": 35, 2024-03-08T21:16:10.644 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages": 63, 2024-03-08T21:16:10.645 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_bytes": 7579, 2024-03-08T21:16:10.645 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_bytes": 55660, 2024-03-08T21:16:10.645 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_created_connections": 14, 2024-03-08T21:16:10.645 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_active_connections": 4, 2024-03-08T21:16:10.645 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_total_time": 0.022343586, 2024-03-08T21:16:10.645 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_send_time": 0.004184385, 2024-03-08T21:16:10.645 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_recv_time": 0.012315167, 2024-03-08T21:16:10.645 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_fast_dispatch_time": 0.000635496, 2024-03-08T21:16:10.645 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages_queue_lat": { 2024-03-08T21:16:10.645 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 61, 2024-03-08T21:16:10.645 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.045268664, 2024-03-08T21:16:10.645 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000742109 2024-03-08T21:16:10.645 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:16:10.645 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_handle_ack_lat": { 2024-03-08T21:16:10.645 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 56, 2024-03-08T21:16:10.646 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000038379, 2024-03-08T21:16:10.646 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000685 2024-03-08T21:16:10.646 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:16:10.646 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-08T21:16:10.646 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_encrypted_bytes": 0 2024-03-08T21:16:10.646 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:16:10.646 INFO:teuthology.orchestra.run.smithi149.stdout: "AsyncMessenger::Worker-1": { 2024-03-08T21:16:10.646 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_messages": 408, 2024-03-08T21:16:10.646 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages": 378, 2024-03-08T21:16:10.646 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_bytes": 1606065, 2024-03-08T21:16:10.646 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_bytes": 200642, 2024-03-08T21:16:10.646 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_created_connections": 5, 2024-03-08T21:16:10.646 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_active_connections": 4, 2024-03-08T21:16:10.646 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_total_time": 0.148231224, 2024-03-08T21:16:10.647 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_send_time": 0.049200941, 2024-03-08T21:16:10.647 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_recv_time": 0.113013866, 2024-03-08T21:16:10.647 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_fast_dispatch_time": 0.015937646, 2024-03-08T21:16:10.647 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages_queue_lat": { 2024-03-08T21:16:10.647 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 378, 2024-03-08T21:16:10.647 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.066312834, 2024-03-08T21:16:10.647 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000175430 2024-03-08T21:16:10.647 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:16:10.647 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_handle_ack_lat": { 2024-03-08T21:16:10.647 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-08T21:16:10.647 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-08T21:16:10.647 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-08T21:16:10.647 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:16:10.647 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_encrypted_bytes": 1602528, 2024-03-08T21:16:10.648 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_encrypted_bytes": 194160 2024-03-08T21:16:10.648 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:16:10.648 INFO:teuthology.orchestra.run.smithi149.stdout: "AsyncMessenger::Worker-2": { 2024-03-08T21:16:10.648 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_messages": 1186, 2024-03-08T21:16:10.648 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages": 1242, 2024-03-08T21:16:10.648 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_bytes": 400182, 2024-03-08T21:16:10.648 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_bytes": 893659, 2024-03-08T21:16:10.648 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_created_connections": 10, 2024-03-08T21:16:10.648 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_active_connections": 5, 2024-03-08T21:16:10.648 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_total_time": 0.404943853, 2024-03-08T21:16:10.648 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_send_time": 0.215244435, 2024-03-08T21:16:10.648 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_recv_time": 0.202958549, 2024-03-08T21:16:10.648 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_running_fast_dispatch_time": 0.011588398, 2024-03-08T21:16:10.649 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_messages_queue_lat": { 2024-03-08T21:16:10.649 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 1239, 2024-03-08T21:16:10.649 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.152818806, 2024-03-08T21:16:10.649 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000123340 2024-03-08T21:16:10.649 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:16:10.649 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_handle_ack_lat": { 2024-03-08T21:16:10.649 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 2150, 2024-03-08T21:16:10.649 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.001008558, 2024-03-08T21:16:10.649 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000469 2024-03-08T21:16:10.649 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:16:10.649 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_recv_encrypted_bytes": 320, 2024-03-08T21:16:10.649 INFO:teuthology.orchestra.run.smithi149.stdout: "msgr_send_encrypted_bytes": 529856 2024-03-08T21:16:10.649 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:16:10.649 INFO:teuthology.orchestra.run.smithi149.stdout: "cct": { 2024-03-08T21:16:10.650 INFO:teuthology.orchestra.run.smithi149.stdout: "total_workers": 1, 2024-03-08T21:16:10.650 INFO:teuthology.orchestra.run.smithi149.stdout: "unhealthy_workers": 0 2024-03-08T21:16:10.650 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:16:10.650 INFO:teuthology.orchestra.run.smithi149.stdout: "finisher-MDSRank": { 2024-03-08T21:16:10.650 INFO:teuthology.orchestra.run.smithi149.stdout: "queue_len": 0, 2024-03-08T21:16:10.650 INFO:teuthology.orchestra.run.smithi149.stdout: "complete_latency": { 2024-03-08T21:16:10.650 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 49, 2024-03-08T21:16:10.650 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.001989630, 2024-03-08T21:16:10.650 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000040604 2024-03-08T21:16:10.650 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-03-08T21:16:10.650 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:16:10.650 INFO:teuthology.orchestra.run.smithi149.stdout: "finisher-PurgeQueue": { 2024-03-08T21:16:10.650 INFO:teuthology.orchestra.run.smithi149.stdout: "queue_len": 0, 2024-03-08T21:16:10.651 INFO:teuthology.orchestra.run.smithi149.stdout: "complete_latency": { 2024-03-08T21:16:10.651 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 2, 2024-03-08T21:16:10.651 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000007785, 2024-03-08T21:16:10.651 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000003892 2024-03-08T21:16:10.651 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-03-08T21:16:10.651 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:16:10.651 INFO:teuthology.orchestra.run.smithi149.stdout: "mds": { 2024-03-08T21:16:10.651 INFO:teuthology.orchestra.run.smithi149.stdout: "request": 1, 2024-03-08T21:16:10.651 INFO:teuthology.orchestra.run.smithi149.stdout: "reply": 0, 2024-03-08T21:16:10.651 INFO:teuthology.orchestra.run.smithi149.stdout: "reply_latency": { 2024-03-08T21:16:10.651 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-08T21:16:10.651 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-08T21:16:10.651 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-08T21:16:10.651 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:16:10.651 INFO:teuthology.orchestra.run.smithi149.stdout: "slow_reply": 0, 2024-03-08T21:16:10.652 INFO:teuthology.orchestra.run.smithi149.stdout: "forward": 1, 2024-03-08T21:16:10.652 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_fetch_complete": 0, 2024-03-08T21:16:10.652 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_fetch_keys": 0, 2024-03-08T21:16:10.652 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_commit": 11, 2024-03-08T21:16:10.652 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_split": 0, 2024-03-08T21:16:10.652 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_merge": 0, 2024-03-08T21:16:10.652 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes": 18, 2024-03-08T21:16:10.652 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes_top": 2, 2024-03-08T21:16:10.652 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes_bottom": 16, 2024-03-08T21:16:10.652 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes_pin_tail": 0, 2024-03-08T21:16:10.652 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes_pinned": 13, 2024-03-08T21:16:10.652 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes_expired": 8, 2024-03-08T21:16:10.652 INFO:teuthology.orchestra.run.smithi149.stdout: "inodes_with_caps": 0, 2024-03-08T21:16:10.652 INFO:teuthology.orchestra.run.smithi149.stdout: "caps": 0, 2024-03-08T21:16:10.652 INFO:teuthology.orchestra.run.smithi149.stdout: "subtrees": 3, 2024-03-08T21:16:10.653 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse": 12, 2024-03-08T21:16:10.653 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse_hit": 6, 2024-03-08T21:16:10.653 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse_forward": 1, 2024-03-08T21:16:10.653 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse_discover": 5, 2024-03-08T21:16:10.653 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse_dir_fetch": 0, 2024-03-08T21:16:10.653 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse_remote_ino": 0, 2024-03-08T21:16:10.653 INFO:teuthology.orchestra.run.smithi149.stdout: "traverse_lock": 0, 2024-03-08T21:16:10.653 INFO:teuthology.orchestra.run.smithi149.stdout: "load_cent": 0, 2024-03-08T21:16:10.653 INFO:teuthology.orchestra.run.smithi149.stdout: "q": 0, 2024-03-08T21:16:10.653 INFO:teuthology.orchestra.run.smithi149.stdout: "exported": 0, 2024-03-08T21:16:10.653 INFO:teuthology.orchestra.run.smithi149.stdout: "exported_inodes": 0, 2024-03-08T21:16:10.653 INFO:teuthology.orchestra.run.smithi149.stdout: "imported": 0, 2024-03-08T21:16:10.653 INFO:teuthology.orchestra.run.smithi149.stdout: "imported_inodes": 0, 2024-03-08T21:16:10.653 INFO:teuthology.orchestra.run.smithi149.stdout: "openino_dir_fetch": 0, 2024-03-08T21:16:10.654 INFO:teuthology.orchestra.run.smithi149.stdout: "openino_backtrace_fetch": 0, 2024-03-08T21:16:10.654 INFO:teuthology.orchestra.run.smithi149.stdout: "openino_peer_discover": 0, 2024-03-08T21:16:10.654 INFO:teuthology.orchestra.run.smithi149.stdout: "root_rfiles": 5, 2024-03-08T21:16:10.654 INFO:teuthology.orchestra.run.smithi149.stdout: "root_rbytes": 4202218, 2024-03-08T21:16:10.654 INFO:teuthology.orchestra.run.smithi149.stdout: "root_rsnaps": 6, 2024-03-08T21:16:10.654 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_backtrace_fetch": 0, 2024-03-08T21:16:10.654 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_set_tag": 0, 2024-03-08T21:16:10.654 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_backtrace_repaired": 0, 2024-03-08T21:16:10.654 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_inotable_repaired": 0, 2024-03-08T21:16:10.654 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_dir_inodes": 0, 2024-03-08T21:16:10.654 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_dir_base_inodes": 0, 2024-03-08T21:16:10.654 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_dirfrag_rstats": 0, 2024-03-08T21:16:10.654 INFO:teuthology.orchestra.run.smithi149.stdout: "scrub_file_inodes": 0, 2024-03-08T21:16:10.654 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_inode_file_caps": 0, 2024-03-08T21:16:10.654 INFO:teuthology.orchestra.run.smithi149.stdout: "ceph_cap_op_revoke": 0, 2024-03-08T21:16:10.655 INFO:teuthology.orchestra.run.smithi149.stdout: "ceph_cap_op_grant": 0, 2024-03-08T21:16:10.655 INFO:teuthology.orchestra.run.smithi149.stdout: "ceph_cap_op_trunc": 0, 2024-03-08T21:16:10.655 INFO:teuthology.orchestra.run.smithi149.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-08T21:16:10.655 INFO:teuthology.orchestra.run.smithi149.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-08T21:16:10.655 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_client_caps": 0, 2024-03-08T21:16:10.655 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_client_caps_dirty": 0, 2024-03-08T21:16:10.655 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_client_cap_release": 0, 2024-03-08T21:16:10.655 INFO:teuthology.orchestra.run.smithi149.stdout: "process_request_cap_release": 0 2024-03-08T21:16:10.655 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:16:10.655 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_cache": { 2024-03-08T21:16:10.655 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_update": 0, 2024-03-08T21:16:10.655 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_update_receipt": 18, 2024-03-08T21:16:10.655 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_try_discover": 11, 2024-03-08T21:16:10.656 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_send_discover": 11, 2024-03-08T21:16:10.656 INFO:teuthology.orchestra.run.smithi149.stdout: "dir_handle_discover": 0, 2024-03-08T21:16:10.656 INFO:teuthology.orchestra.run.smithi149.stdout: "num_strays": 0, 2024-03-08T21:16:10.656 INFO:teuthology.orchestra.run.smithi149.stdout: "num_strays_delayed": 0, 2024-03-08T21:16:10.656 INFO:teuthology.orchestra.run.smithi149.stdout: "num_strays_enqueuing": 0, 2024-03-08T21:16:10.656 INFO:teuthology.orchestra.run.smithi149.stdout: "strays_created": 0, 2024-03-08T21:16:10.656 INFO:teuthology.orchestra.run.smithi149.stdout: "strays_enqueued": 0, 2024-03-08T21:16:10.656 INFO:teuthology.orchestra.run.smithi149.stdout: "strays_reintegrated": 0, 2024-03-08T21:16:10.656 INFO:teuthology.orchestra.run.smithi149.stdout: "strays_migrated": 0, 2024-03-08T21:16:10.656 INFO:teuthology.orchestra.run.smithi149.stdout: "num_recovering_processing": 0, 2024-03-08T21:16:10.656 INFO:teuthology.orchestra.run.smithi149.stdout: "num_recovering_enqueued": 0, 2024-03-08T21:16:10.656 INFO:teuthology.orchestra.run.smithi149.stdout: "num_recovering_prioritized": 0, 2024-03-08T21:16:10.656 INFO:teuthology.orchestra.run.smithi149.stdout: "recovery_started": 0, 2024-03-08T21:16:10.657 INFO:teuthology.orchestra.run.smithi149.stdout: "recovery_completed": 0, 2024-03-08T21:16:10.657 INFO:teuthology.orchestra.run.smithi149.stdout: "ireq_enqueue_scrub": 0, 2024-03-08T21:16:10.657 INFO:teuthology.orchestra.run.smithi149.stdout: "ireq_exportdir": 0, 2024-03-08T21:16:10.657 INFO:teuthology.orchestra.run.smithi149.stdout: "ireq_flush": 0, 2024-03-08T21:16:10.657 INFO:teuthology.orchestra.run.smithi149.stdout: "ireq_fragmentdir": 0, 2024-03-08T21:16:10.657 INFO:teuthology.orchestra.run.smithi149.stdout: "ireq_fragstats": 0, 2024-03-08T21:16:10.657 INFO:teuthology.orchestra.run.smithi149.stdout: "ireq_inodestats": 0 2024-03-08T21:16:10.657 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:16:10.657 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_log": { 2024-03-08T21:16:10.657 INFO:teuthology.orchestra.run.smithi149.stdout: "evlrg": 0, 2024-03-08T21:16:10.657 INFO:teuthology.orchestra.run.smithi149.stdout: "evadd": 3, 2024-03-08T21:16:10.657 INFO:teuthology.orchestra.run.smithi149.stdout: "evex": 0, 2024-03-08T21:16:10.657 INFO:teuthology.orchestra.run.smithi149.stdout: "evtrm": 0, 2024-03-08T21:16:10.657 INFO:teuthology.orchestra.run.smithi149.stdout: "ev": 3, 2024-03-08T21:16:10.658 INFO:teuthology.orchestra.run.smithi149.stdout: "evexg": 0, 2024-03-08T21:16:10.658 INFO:teuthology.orchestra.run.smithi149.stdout: "evexd": 0, 2024-03-08T21:16:10.658 INFO:teuthology.orchestra.run.smithi149.stdout: "segadd": 2, 2024-03-08T21:16:10.658 INFO:teuthology.orchestra.run.smithi149.stdout: "segex": 0, 2024-03-08T21:16:10.658 INFO:teuthology.orchestra.run.smithi149.stdout: "segtrm": 0, 2024-03-08T21:16:10.658 INFO:teuthology.orchestra.run.smithi149.stdout: "seg": 2, 2024-03-08T21:16:10.658 INFO:teuthology.orchestra.run.smithi149.stdout: "segmjr": 2, 2024-03-08T21:16:10.658 INFO:teuthology.orchestra.run.smithi149.stdout: "segexg": 0, 2024-03-08T21:16:10.658 INFO:teuthology.orchestra.run.smithi149.stdout: "segexd": 0, 2024-03-08T21:16:10.658 INFO:teuthology.orchestra.run.smithi149.stdout: "expos": 4194304, 2024-03-08T21:16:10.658 INFO:teuthology.orchestra.run.smithi149.stdout: "wrpos": 4195385, 2024-03-08T21:16:10.658 INFO:teuthology.orchestra.run.smithi149.stdout: "rdpos": 0, 2024-03-08T21:16:10.658 INFO:teuthology.orchestra.run.smithi149.stdout: "jlat": { 2024-03-08T21:16:10.658 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 2, 2024-03-08T21:16:10.659 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.009220336, 2024-03-08T21:16:10.659 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.004610168 2024-03-08T21:16:10.659 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:16:10.659 INFO:teuthology.orchestra.run.smithi149.stdout: "replayed": 0 2024-03-08T21:16:10.659 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:16:10.659 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_mem": { 2024-03-08T21:16:10.659 INFO:teuthology.orchestra.run.smithi149.stdout: "ino": 22, 2024-03-08T21:16:10.659 INFO:teuthology.orchestra.run.smithi149.stdout: "ino+": 28, 2024-03-08T21:16:10.659 INFO:teuthology.orchestra.run.smithi149.stdout: "ino-": 6, 2024-03-08T21:16:10.659 INFO:teuthology.orchestra.run.smithi149.stdout: "dir": 21, 2024-03-08T21:16:10.659 INFO:teuthology.orchestra.run.smithi149.stdout: "dir+": 25, 2024-03-08T21:16:10.659 INFO:teuthology.orchestra.run.smithi149.stdout: "dir-": 4, 2024-03-08T21:16:10.659 INFO:teuthology.orchestra.run.smithi149.stdout: "dn": 18, 2024-03-08T21:16:10.660 INFO:teuthology.orchestra.run.smithi149.stdout: "dn+": 26, 2024-03-08T21:16:10.660 INFO:teuthology.orchestra.run.smithi149.stdout: "dn-": 8, 2024-03-08T21:16:10.660 INFO:teuthology.orchestra.run.smithi149.stdout: "cap": 0, 2024-03-08T21:16:10.660 INFO:teuthology.orchestra.run.smithi149.stdout: "cap+": 0, 2024-03-08T21:16:10.660 INFO:teuthology.orchestra.run.smithi149.stdout: "cap-": 0, 2024-03-08T21:16:10.660 INFO:teuthology.orchestra.run.smithi149.stdout: "rss": 50688, 2024-03-08T21:16:10.660 INFO:teuthology.orchestra.run.smithi149.stdout: "heap": 207228 2024-03-08T21:16:10.660 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:16:10.660 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_server": { 2024-03-08T21:16:10.660 INFO:teuthology.orchestra.run.smithi149.stdout: "dispatch_client_request": 1, 2024-03-08T21:16:10.660 INFO:teuthology.orchestra.run.smithi149.stdout: "dispatch_server_request": 0, 2024-03-08T21:16:10.660 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_client_request": 1, 2024-03-08T21:16:10.660 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_client_session": 12, 2024-03-08T21:16:10.660 INFO:teuthology.orchestra.run.smithi149.stdout: "handle_peer_request": 0, 2024-03-08T21:16:10.660 INFO:teuthology.orchestra.run.smithi149.stdout: "req_create_latency": { 2024-03-08T21:16:10.661 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-08T21:16:10.661 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-08T21:16:10.661 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-08T21:16:10.661 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:16:10.661 INFO:teuthology.orchestra.run.smithi149.stdout: "req_getattr_latency": { 2024-03-08T21:16:10.661 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-08T21:16:10.661 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-08T21:16:10.661 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-08T21:16:10.661 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:16:10.661 INFO:teuthology.orchestra.run.smithi149.stdout: "req_getfilelock_latency": { 2024-03-08T21:16:10.661 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-08T21:16:10.661 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-08T21:16:10.661 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-08T21:16:10.661 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:16:10.661 INFO:teuthology.orchestra.run.smithi149.stdout: "req_link_latency": { 2024-03-08T21:16:10.661 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-08T21:16:10.662 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-08T21:16:10.662 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-08T21:16:10.662 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:16:10.662 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lookup_latency": { 2024-03-08T21:16:10.662 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-08T21:16:10.662 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-08T21:16:10.662 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-08T21:16:10.662 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:16:10.662 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lookuphash_latency": { 2024-03-08T21:16:10.662 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-08T21:16:10.662 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-08T21:16:10.662 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-08T21:16:10.662 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:16:10.662 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lookupino_latency": { 2024-03-08T21:16:10.662 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-08T21:16:10.663 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-08T21:16:10.663 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-08T21:16:10.663 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:16:10.663 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lookupname_latency": { 2024-03-08T21:16:10.663 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-08T21:16:10.663 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-08T21:16:10.663 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-08T21:16:10.663 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:16:10.663 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lookupparent_latency": { 2024-03-08T21:16:10.663 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-08T21:16:10.663 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-08T21:16:10.663 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-08T21:16:10.663 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:16:10.663 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lookupsnap_latency": { 2024-03-08T21:16:10.663 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-08T21:16:10.663 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-08T21:16:10.664 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-08T21:16:10.664 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:16:10.664 INFO:teuthology.orchestra.run.smithi149.stdout: "req_lssnap_latency": { 2024-03-08T21:16:10.664 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-08T21:16:10.664 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-08T21:16:10.664 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-08T21:16:10.664 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:16:10.664 INFO:teuthology.orchestra.run.smithi149.stdout: "req_mkdir_latency": { 2024-03-08T21:16:10.664 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-08T21:16:10.664 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-08T21:16:10.664 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-08T21:16:10.665 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:16:10.665 INFO:teuthology.orchestra.run.smithi149.stdout: "req_mknod_latency": { 2024-03-08T21:16:10.665 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-08T21:16:10.665 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-08T21:16:10.665 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-08T21:16:10.665 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:16:10.665 INFO:teuthology.orchestra.run.smithi149.stdout: "req_mksnap_latency": { 2024-03-08T21:16:10.665 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-08T21:16:10.665 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-08T21:16:10.665 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-08T21:16:10.665 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:16:10.665 INFO:teuthology.orchestra.run.smithi149.stdout: "req_open_latency": { 2024-03-08T21:16:10.666 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-08T21:16:10.666 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-08T21:16:10.666 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-08T21:16:10.666 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:16:10.666 INFO:teuthology.orchestra.run.smithi149.stdout: "req_readdir_latency": { 2024-03-08T21:16:10.666 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-08T21:16:10.666 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-08T21:16:10.666 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-08T21:16:10.666 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:16:10.666 INFO:teuthology.orchestra.run.smithi149.stdout: "req_rename_latency": { 2024-03-08T21:16:10.666 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-08T21:16:10.666 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-08T21:16:10.666 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-08T21:16:10.667 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:16:10.667 INFO:teuthology.orchestra.run.smithi149.stdout: "req_renamesnap_latency": { 2024-03-08T21:16:10.667 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-08T21:16:10.667 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-08T21:16:10.667 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-08T21:16:10.667 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:16:10.667 INFO:teuthology.orchestra.run.smithi149.stdout: "req_snapdiff_latency": { 2024-03-08T21:16:10.667 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-08T21:16:10.667 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-08T21:16:10.667 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-08T21:16:10.667 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:16:10.667 INFO:teuthology.orchestra.run.smithi149.stdout: "req_rmdir_latency": { 2024-03-08T21:16:10.667 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-08T21:16:10.667 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-08T21:16:10.668 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-08T21:16:10.668 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:16:10.668 INFO:teuthology.orchestra.run.smithi149.stdout: "req_rmsnap_latency": { 2024-03-08T21:16:10.668 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-08T21:16:10.668 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-08T21:16:10.668 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-08T21:16:10.668 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:16:10.668 INFO:teuthology.orchestra.run.smithi149.stdout: "req_rmxattr_latency": { 2024-03-08T21:16:10.668 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-08T21:16:10.668 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-08T21:16:10.668 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-08T21:16:10.668 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:16:10.668 INFO:teuthology.orchestra.run.smithi149.stdout: "req_setattr_latency": { 2024-03-08T21:16:10.669 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-08T21:16:10.669 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-08T21:16:10.669 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-08T21:16:10.669 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:16:10.669 INFO:teuthology.orchestra.run.smithi149.stdout: "req_setdirlayout_latency": { 2024-03-08T21:16:10.669 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-08T21:16:10.669 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-08T21:16:10.669 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-08T21:16:10.669 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:16:10.669 INFO:teuthology.orchestra.run.smithi149.stdout: "req_setfilelock_latency": { 2024-03-08T21:16:10.669 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-08T21:16:10.669 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-08T21:16:10.669 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-08T21:16:10.669 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:16:10.670 INFO:teuthology.orchestra.run.smithi149.stdout: "req_setlayout_latency": { 2024-03-08T21:16:10.670 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-08T21:16:10.670 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-08T21:16:10.670 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-08T21:16:10.670 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:16:10.670 INFO:teuthology.orchestra.run.smithi149.stdout: "req_setxattr_latency": { 2024-03-08T21:16:10.670 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-08T21:16:10.670 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-08T21:16:10.670 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-08T21:16:10.670 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:16:10.670 INFO:teuthology.orchestra.run.smithi149.stdout: "req_symlink_latency": { 2024-03-08T21:16:10.670 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-08T21:16:10.670 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-08T21:16:10.670 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-08T21:16:10.671 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:16:10.671 INFO:teuthology.orchestra.run.smithi149.stdout: "req_unlink_latency": { 2024-03-08T21:16:10.671 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-08T21:16:10.671 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-08T21:16:10.671 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-08T21:16:10.671 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:16:10.671 INFO:teuthology.orchestra.run.smithi149.stdout: "cap_revoke_eviction": 0, 2024-03-08T21:16:10.671 INFO:teuthology.orchestra.run.smithi149.stdout: "cap_acquisition_throttle": 0, 2024-03-08T21:16:10.671 INFO:teuthology.orchestra.run.smithi149.stdout: "req_getvxattr_latency": { 2024-03-08T21:16:10.671 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-08T21:16:10.671 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-08T21:16:10.671 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-08T21:16:10.671 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-03-08T21:16:10.671 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:16:10.671 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_sessions": { 2024-03-08T21:16:10.672 INFO:teuthology.orchestra.run.smithi149.stdout: "session_count": 1, 2024-03-08T21:16:10.672 INFO:teuthology.orchestra.run.smithi149.stdout: "session_add": 1, 2024-03-08T21:16:10.672 INFO:teuthology.orchestra.run.smithi149.stdout: "session_remove": 0, 2024-03-08T21:16:10.672 INFO:teuthology.orchestra.run.smithi149.stdout: "sessions_open": 1, 2024-03-08T21:16:10.672 INFO:teuthology.orchestra.run.smithi149.stdout: "sessions_stale": 0, 2024-03-08T21:16:10.672 INFO:teuthology.orchestra.run.smithi149.stdout: "total_load": 0, 2024-03-08T21:16:10.672 INFO:teuthology.orchestra.run.smithi149.stdout: "average_load": 0, 2024-03-08T21:16:10.672 INFO:teuthology.orchestra.run.smithi149.stdout: "avg_session_uptime": 950, 2024-03-08T21:16:10.672 INFO:teuthology.orchestra.run.smithi149.stdout: "mdthresh_evicted": 0 2024-03-08T21:16:10.672 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:16:10.672 INFO:teuthology.orchestra.run.smithi149.stdout: "mempool": { 2024-03-08T21:16:10.672 INFO:teuthology.orchestra.run.smithi149.stdout: "bloom_filter_bytes": 0, 2024-03-08T21:16:10.672 INFO:teuthology.orchestra.run.smithi149.stdout: "bloom_filter_items": 0, 2024-03-08T21:16:10.672 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_alloc_bytes": 0, 2024-03-08T21:16:10.673 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_alloc_items": 0, 2024-03-08T21:16:10.673 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_data_bytes": 0, 2024-03-08T21:16:10.673 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_data_items": 0, 2024-03-08T21:16:10.673 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-08T21:16:10.673 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_onode_items": 0, 2024-03-08T21:16:10.673 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-08T21:16:10.673 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_meta_items": 0, 2024-03-08T21:16:10.673 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_other_bytes": 0, 2024-03-08T21:16:10.673 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_other_items": 0, 2024-03-08T21:16:10.673 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-08T21:16:10.673 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_cache_buffer_items": 0, 2024-03-08T21:16:10.673 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_extent_bytes": 0, 2024-03-08T21:16:10.673 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_extent_items": 0, 2024-03-08T21:16:10.674 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_blob_bytes": 0, 2024-03-08T21:16:10.674 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_blob_items": 0, 2024-03-08T21:16:10.674 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-08T21:16:10.674 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_shared_blob_items": 0, 2024-03-08T21:16:10.674 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-08T21:16:10.674 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_inline_bl_items": 0, 2024-03-08T21:16:10.674 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_fsck_bytes": 0, 2024-03-08T21:16:10.674 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_fsck_items": 0, 2024-03-08T21:16:10.674 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_txc_bytes": 0, 2024-03-08T21:16:10.674 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_txc_items": 0, 2024-03-08T21:16:10.674 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-08T21:16:10.674 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_writing_deferred_items": 0, 2024-03-08T21:16:10.674 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_writing_bytes": 0, 2024-03-08T21:16:10.674 INFO:teuthology.orchestra.run.smithi149.stdout: "bluestore_writing_items": 0, 2024-03-08T21:16:10.675 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_bytes": 0, 2024-03-08T21:16:10.675 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_items": 0, 2024-03-08T21:16:10.675 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_file_reader_bytes": 0, 2024-03-08T21:16:10.675 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_file_reader_items": 0, 2024-03-08T21:16:10.675 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_file_writer_bytes": 0, 2024-03-08T21:16:10.675 INFO:teuthology.orchestra.run.smithi149.stdout: "bluefs_file_writer_items": 0, 2024-03-08T21:16:10.675 INFO:teuthology.orchestra.run.smithi149.stdout: "buffer_anon_bytes": 239917, 2024-03-08T21:16:10.675 INFO:teuthology.orchestra.run.smithi149.stdout: "buffer_anon_items": 81, 2024-03-08T21:16:10.675 INFO:teuthology.orchestra.run.smithi149.stdout: "buffer_meta_bytes": 0, 2024-03-08T21:16:10.675 INFO:teuthology.orchestra.run.smithi149.stdout: "buffer_meta_items": 0, 2024-03-08T21:16:10.675 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_bytes": 0, 2024-03-08T21:16:10.675 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_items": 0, 2024-03-08T21:16:10.675 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_mapbl_bytes": 0, 2024-03-08T21:16:10.675 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_mapbl_items": 0, 2024-03-08T21:16:10.675 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_pglog_bytes": 0, 2024-03-08T21:16:10.676 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_pglog_items": 0, 2024-03-08T21:16:10.676 INFO:teuthology.orchestra.run.smithi149.stdout: "osdmap_bytes": 8352, 2024-03-08T21:16:10.676 INFO:teuthology.orchestra.run.smithi149.stdout: "osdmap_items": 190, 2024-03-08T21:16:10.676 INFO:teuthology.orchestra.run.smithi149.stdout: "osdmap_mapping_bytes": 0, 2024-03-08T21:16:10.676 INFO:teuthology.orchestra.run.smithi149.stdout: "osdmap_mapping_items": 0, 2024-03-08T21:16:10.676 INFO:teuthology.orchestra.run.smithi149.stdout: "pgmap_bytes": 0, 2024-03-08T21:16:10.676 INFO:teuthology.orchestra.run.smithi149.stdout: "pgmap_items": 0, 2024-03-08T21:16:10.676 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_co_bytes": 90397, 2024-03-08T21:16:10.676 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_co_items": 351, 2024-03-08T21:16:10.676 INFO:teuthology.orchestra.run.smithi149.stdout: "unittest_1_bytes": 0, 2024-03-08T21:16:10.676 INFO:teuthology.orchestra.run.smithi149.stdout: "unittest_1_items": 0, 2024-03-08T21:16:10.676 INFO:teuthology.orchestra.run.smithi149.stdout: "unittest_2_bytes": 0, 2024-03-08T21:16:10.676 INFO:teuthology.orchestra.run.smithi149.stdout: "unittest_2_items": 0 2024-03-08T21:16:10.676 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:16:10.677 INFO:teuthology.orchestra.run.smithi149.stdout: "objecter": { 2024-03-08T21:16:10.677 INFO:teuthology.orchestra.run.smithi149.stdout: "op_active": 0, 2024-03-08T21:16:10.677 INFO:teuthology.orchestra.run.smithi149.stdout: "op_laggy": 0, 2024-03-08T21:16:10.677 INFO:teuthology.orchestra.run.smithi149.stdout: "op_send": 37, 2024-03-08T21:16:10.677 INFO:teuthology.orchestra.run.smithi149.stdout: "op_send_bytes": 11293, 2024-03-08T21:16:10.677 INFO:teuthology.orchestra.run.smithi149.stdout: "op_resend": 27, 2024-03-08T21:16:10.677 INFO:teuthology.orchestra.run.smithi149.stdout: "op_reply": 37, 2024-03-08T21:16:10.677 INFO:teuthology.orchestra.run.smithi149.stdout: "op_latency": { 2024-03-08T21:16:10.677 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 37, 2024-03-08T21:16:10.677 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.468990580, 2024-03-08T21:16:10.677 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.012675421 2024-03-08T21:16:10.677 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:16:10.677 INFO:teuthology.orchestra.run.smithi149.stdout: "op_inflight": 0, 2024-03-08T21:16:10.677 INFO:teuthology.orchestra.run.smithi149.stdout: "oplen_avg": { 2024-03-08T21:16:10.678 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 37, 2024-03-08T21:16:10.678 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 59 2024-03-08T21:16:10.678 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:16:10.678 INFO:teuthology.orchestra.run.smithi149.stdout: "op": 37, 2024-03-08T21:16:10.678 INFO:teuthology.orchestra.run.smithi149.stdout: "op_r": 0, 2024-03-08T21:16:10.678 INFO:teuthology.orchestra.run.smithi149.stdout: "op_w": 37, 2024-03-08T21:16:10.678 INFO:teuthology.orchestra.run.smithi149.stdout: "op_rmw": 0, 2024-03-08T21:16:10.678 INFO:teuthology.orchestra.run.smithi149.stdout: "op_pg": 0, 2024-03-08T21:16:10.678 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_stat": 0, 2024-03-08T21:16:10.678 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_create": 10, 2024-03-08T21:16:10.678 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_read": 0, 2024-03-08T21:16:10.678 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_write": 2, 2024-03-08T21:16:10.678 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_writefull": 6, 2024-03-08T21:16:10.679 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_writesame": 0, 2024-03-08T21:16:10.679 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_append": 0, 2024-03-08T21:16:10.679 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_zero": 0, 2024-03-08T21:16:10.679 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_truncate": 0, 2024-03-08T21:16:10.679 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_delete": 6, 2024-03-08T21:16:10.679 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_mapext": 0, 2024-03-08T21:16:10.679 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_sparse_read": 0, 2024-03-08T21:16:10.679 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_clonerange": 0, 2024-03-08T21:16:10.679 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_getxattr": 0, 2024-03-08T21:16:10.679 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_setxattr": 20, 2024-03-08T21:16:10.679 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_cmpxattr": 0, 2024-03-08T21:16:10.679 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_rmxattr": 0, 2024-03-08T21:16:10.679 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_resetxattrs": 0, 2024-03-08T21:16:10.679 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_call": 0, 2024-03-08T21:16:10.680 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_watch": 0, 2024-03-08T21:16:10.680 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_notify": 0, 2024-03-08T21:16:10.680 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_src_cmpxattr": 0, 2024-03-08T21:16:10.680 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_pgls": 0, 2024-03-08T21:16:10.680 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_pgls_filter": 0, 2024-03-08T21:16:10.680 INFO:teuthology.orchestra.run.smithi149.stdout: "osdop_other": 0, 2024-03-08T21:16:10.680 INFO:teuthology.orchestra.run.smithi149.stdout: "linger_active": 0, 2024-03-08T21:16:10.680 INFO:teuthology.orchestra.run.smithi149.stdout: "linger_send": 0, 2024-03-08T21:16:10.680 INFO:teuthology.orchestra.run.smithi149.stdout: "linger_resend": 0, 2024-03-08T21:16:10.680 INFO:teuthology.orchestra.run.smithi149.stdout: "linger_ping": 0, 2024-03-08T21:16:10.680 INFO:teuthology.orchestra.run.smithi149.stdout: "poolop_active": 0, 2024-03-08T21:16:10.680 INFO:teuthology.orchestra.run.smithi149.stdout: "poolop_send": 0, 2024-03-08T21:16:10.680 INFO:teuthology.orchestra.run.smithi149.stdout: "poolop_resend": 0, 2024-03-08T21:16:10.681 INFO:teuthology.orchestra.run.smithi149.stdout: "poolstat_active": 0, 2024-03-08T21:16:10.681 INFO:teuthology.orchestra.run.smithi149.stdout: "poolstat_send": 0, 2024-03-08T21:16:10.681 INFO:teuthology.orchestra.run.smithi149.stdout: "poolstat_resend": 0, 2024-03-08T21:16:10.681 INFO:teuthology.orchestra.run.smithi149.stdout: "statfs_active": 0, 2024-03-08T21:16:10.681 INFO:teuthology.orchestra.run.smithi149.stdout: "statfs_send": 0, 2024-03-08T21:16:10.681 INFO:teuthology.orchestra.run.smithi149.stdout: "statfs_resend": 0, 2024-03-08T21:16:10.681 INFO:teuthology.orchestra.run.smithi149.stdout: "command_active": 0, 2024-03-08T21:16:10.681 INFO:teuthology.orchestra.run.smithi149.stdout: "command_send": 0, 2024-03-08T21:16:10.681 INFO:teuthology.orchestra.run.smithi149.stdout: "command_resend": 0, 2024-03-08T21:16:10.681 INFO:teuthology.orchestra.run.smithi149.stdout: "map_epoch": 114, 2024-03-08T21:16:10.681 INFO:teuthology.orchestra.run.smithi149.stdout: "map_full": 0, 2024-03-08T21:16:10.681 INFO:teuthology.orchestra.run.smithi149.stdout: "map_inc": 28, 2024-03-08T21:16:10.681 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_sessions": 8, 2024-03-08T21:16:10.681 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_session_open": 11, 2024-03-08T21:16:10.681 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_session_close": 3, 2024-03-08T21:16:10.682 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_laggy": 0, 2024-03-08T21:16:10.682 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_wr": 14, 2024-03-08T21:16:10.682 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_rd": 0, 2024-03-08T21:16:10.682 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_del": 1 2024-03-08T21:16:10.682 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:16:10.682 INFO:teuthology.orchestra.run.smithi149.stdout: "oft": { 2024-03-08T21:16:10.682 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_total_objs": 1, 2024-03-08T21:16:10.682 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_total_kv_pairs": 0, 2024-03-08T21:16:10.682 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_total_updates": 0, 2024-03-08T21:16:10.682 INFO:teuthology.orchestra.run.smithi149.stdout: "omap_total_removes": 0 2024-03-08T21:16:10.682 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:16:10.682 INFO:teuthology.orchestra.run.smithi149.stdout: "osd-slow-ops": { 2024-03-08T21:16:10.682 INFO:teuthology.orchestra.run.smithi149.stdout: "slow_ops_count": 0 2024-03-08T21:16:10.682 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:16:10.682 INFO:teuthology.orchestra.run.smithi149.stdout: "purge_queue": { 2024-03-08T21:16:10.683 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_executing_ops": 0, 2024-03-08T21:16:10.683 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_executing_ops_high_water": 0, 2024-03-08T21:16:10.683 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_executing": 0, 2024-03-08T21:16:10.683 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_executing_high_water": 0, 2024-03-08T21:16:10.683 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_executed_ops": 0, 2024-03-08T21:16:10.683 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_executed": 0, 2024-03-08T21:16:10.683 INFO:teuthology.orchestra.run.smithi149.stdout: "pq_item_in_journal": 0 2024-03-08T21:16:10.683 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:16:10.683 INFO:teuthology.orchestra.run.smithi149.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-08T21:16:10.683 INFO:teuthology.orchestra.run.smithi149.stdout: "val": 0, 2024-03-08T21:16:10.683 INFO:teuthology.orchestra.run.smithi149.stdout: "max": 104857600, 2024-03-08T21:16:10.683 INFO:teuthology.orchestra.run.smithi149.stdout: "get_started": 0, 2024-03-08T21:16:10.683 INFO:teuthology.orchestra.run.smithi149.stdout: "get": 1625, 2024-03-08T21:16:10.683 INFO:teuthology.orchestra.run.smithi149.stdout: "get_sum": 1844850, 2024-03-08T21:16:10.683 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_fail": 0, 2024-03-08T21:16:10.683 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_success": 1625, 2024-03-08T21:16:10.684 INFO:teuthology.orchestra.run.smithi149.stdout: "take": 0, 2024-03-08T21:16:10.684 INFO:teuthology.orchestra.run.smithi149.stdout: "take_sum": 0, 2024-03-08T21:16:10.684 INFO:teuthology.orchestra.run.smithi149.stdout: "put": 1625, 2024-03-08T21:16:10.684 INFO:teuthology.orchestra.run.smithi149.stdout: "put_sum": 1844850, 2024-03-08T21:16:10.684 INFO:teuthology.orchestra.run.smithi149.stdout: "wait": { 2024-03-08T21:16:10.684 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-08T21:16:10.684 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-08T21:16:10.684 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-08T21:16:10.684 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-03-08T21:16:10.684 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:16:10.684 INFO:teuthology.orchestra.run.smithi149.stdout: "throttle-objecter_bytes": { 2024-03-08T21:16:10.684 INFO:teuthology.orchestra.run.smithi149.stdout: "val": 0, 2024-03-08T21:16:10.684 INFO:teuthology.orchestra.run.smithi149.stdout: "max": 104857600, 2024-03-08T21:16:10.684 INFO:teuthology.orchestra.run.smithi149.stdout: "get_started": 0, 2024-03-08T21:16:10.684 INFO:teuthology.orchestra.run.smithi149.stdout: "get": 0, 2024-03-08T21:16:10.685 INFO:teuthology.orchestra.run.smithi149.stdout: "get_sum": 0, 2024-03-08T21:16:10.685 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_fail": 0, 2024-03-08T21:16:10.685 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_success": 0, 2024-03-08T21:16:10.685 INFO:teuthology.orchestra.run.smithi149.stdout: "take": 37, 2024-03-08T21:16:10.685 INFO:teuthology.orchestra.run.smithi149.stdout: "take_sum": 11293, 2024-03-08T21:16:10.685 INFO:teuthology.orchestra.run.smithi149.stdout: "put": 37, 2024-03-08T21:16:10.685 INFO:teuthology.orchestra.run.smithi149.stdout: "put_sum": 11293, 2024-03-08T21:16:10.685 INFO:teuthology.orchestra.run.smithi149.stdout: "wait": { 2024-03-08T21:16:10.685 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-08T21:16:10.685 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-08T21:16:10.685 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-08T21:16:10.685 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-03-08T21:16:10.685 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:16:10.685 INFO:teuthology.orchestra.run.smithi149.stdout: "throttle-objecter_ops": { 2024-03-08T21:16:10.686 INFO:teuthology.orchestra.run.smithi149.stdout: "val": 0, 2024-03-08T21:16:10.686 INFO:teuthology.orchestra.run.smithi149.stdout: "max": 1024, 2024-03-08T21:16:10.686 INFO:teuthology.orchestra.run.smithi149.stdout: "get_started": 0, 2024-03-08T21:16:10.686 INFO:teuthology.orchestra.run.smithi149.stdout: "get": 0, 2024-03-08T21:16:10.686 INFO:teuthology.orchestra.run.smithi149.stdout: "get_sum": 0, 2024-03-08T21:16:10.686 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_fail": 0, 2024-03-08T21:16:10.686 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_success": 0, 2024-03-08T21:16:10.686 INFO:teuthology.orchestra.run.smithi149.stdout: "take": 37, 2024-03-08T21:16:10.686 INFO:teuthology.orchestra.run.smithi149.stdout: "take_sum": 37, 2024-03-08T21:16:10.686 INFO:teuthology.orchestra.run.smithi149.stdout: "put": 37, 2024-03-08T21:16:10.686 INFO:teuthology.orchestra.run.smithi149.stdout: "put_sum": 37, 2024-03-08T21:16:10.686 INFO:teuthology.orchestra.run.smithi149.stdout: "wait": { 2024-03-08T21:16:10.686 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-08T21:16:10.686 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-08T21:16:10.686 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-08T21:16:10.687 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-03-08T21:16:10.687 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:16:10.687 INFO:teuthology.orchestra.run.smithi149.stdout: "throttle-write_buf_throttle": { 2024-03-08T21:16:10.687 INFO:teuthology.orchestra.run.smithi149.stdout: "val": 0, 2024-03-08T21:16:10.687 INFO:teuthology.orchestra.run.smithi149.stdout: "max": 3758096384, 2024-03-08T21:16:10.687 INFO:teuthology.orchestra.run.smithi149.stdout: "get_started": 0, 2024-03-08T21:16:10.687 INFO:teuthology.orchestra.run.smithi149.stdout: "get": 0, 2024-03-08T21:16:10.687 INFO:teuthology.orchestra.run.smithi149.stdout: "get_sum": 0, 2024-03-08T21:16:10.687 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_fail": 0, 2024-03-08T21:16:10.687 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_success": 0, 2024-03-08T21:16:10.687 INFO:teuthology.orchestra.run.smithi149.stdout: "take": 0, 2024-03-08T21:16:10.687 INFO:teuthology.orchestra.run.smithi149.stdout: "take_sum": 0, 2024-03-08T21:16:10.687 INFO:teuthology.orchestra.run.smithi149.stdout: "put": 0, 2024-03-08T21:16:10.687 INFO:teuthology.orchestra.run.smithi149.stdout: "put_sum": 0, 2024-03-08T21:16:10.687 INFO:teuthology.orchestra.run.smithi149.stdout: "wait": { 2024-03-08T21:16:10.687 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-08T21:16:10.688 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-08T21:16:10.688 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-08T21:16:10.688 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-03-08T21:16:10.688 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-03-08T21:16:10.688 INFO:teuthology.orchestra.run.smithi149.stdout: "throttle-write_buf_throttle-0x55644ab086e0": { 2024-03-08T21:16:10.688 INFO:teuthology.orchestra.run.smithi149.stdout: "val": 0, 2024-03-08T21:16:10.688 INFO:teuthology.orchestra.run.smithi149.stdout: "max": 3758096384, 2024-03-08T21:16:10.688 INFO:teuthology.orchestra.run.smithi149.stdout: "get_started": 0, 2024-03-08T21:16:10.688 INFO:teuthology.orchestra.run.smithi149.stdout: "get": 3, 2024-03-08T21:16:10.688 INFO:teuthology.orchestra.run.smithi149.stdout: "get_sum": 1081, 2024-03-08T21:16:10.688 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_fail": 0, 2024-03-08T21:16:10.688 INFO:teuthology.orchestra.run.smithi149.stdout: "get_or_fail_success": 3, 2024-03-08T21:16:10.688 INFO:teuthology.orchestra.run.smithi149.stdout: "take": 0, 2024-03-08T21:16:10.688 INFO:teuthology.orchestra.run.smithi149.stdout: "take_sum": 0, 2024-03-08T21:16:10.689 INFO:teuthology.orchestra.run.smithi149.stdout: "put": 2, 2024-03-08T21:16:10.689 INFO:teuthology.orchestra.run.smithi149.stdout: "put_sum": 1081, 2024-03-08T21:16:10.689 INFO:teuthology.orchestra.run.smithi149.stdout: "wait": { 2024-03-08T21:16:10.689 INFO:teuthology.orchestra.run.smithi149.stdout: "avgcount": 0, 2024-03-08T21:16:10.689 INFO:teuthology.orchestra.run.smithi149.stdout: "sum": 0.000000000, 2024-03-08T21:16:10.689 INFO:teuthology.orchestra.run.smithi149.stdout: "avgtime": 0.000000000 2024-03-08T21:16:10.689 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-03-08T21:16:10.689 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-03-08T21:16:10.689 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-03-08T21:16:10.960 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:16:10 smithi107 ceph-mon[31381]: pgmap v495: 97 pgs: 97 active+clean; 17 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 46 KiB/s wr, 29 op/s 2024-03-08T21:16:10.970 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:16:10 smithi107 ceph-mon[31381]: osdmap e114: 12 total, 12 up, 12 in 2024-03-08T21:16:11.008 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:16:10 smithi149 ceph-mon[35042]: pgmap v495: 97 pgs: 97 active+clean; 17 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 46 KiB/s wr, 29 op/s 2024-03-08T21:16:11.008 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:16:10 smithi149 ceph-mon[35042]: osdmap e114: 12 total, 12 up, 12 in 2024-03-08T21:16:11.121 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:16:10 smithi204 ceph-mon[34935]: pgmap v495: 97 pgs: 97 active+clean; 17 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 46 KiB/s wr, 29 op/s 2024-03-08T21:16:11.121 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:16:10 smithi204 ceph-mon[34935]: osdmap e114: 12 total, 12 up, 12 in 2024-03-08T21:16:11.805 INFO:tasks.check_counter:Daemon mds.k mds.exported=0 2024-03-08T21:16:11.805 INFO:tasks.check_counter:Daemon mds.k mds.imported=0 2024-03-08T21:16:11.805 INFO:tasks.check_counter:Daemon mds.k mds_cache.dir_update=0 2024-03-08T21:16:11.805 INFO:tasks.check_counter:Daemon mds.k mds_cache.dir_update_receipt=18 2024-03-08T21:16:11.805 INFO:tasks.check_counter:Daemon mds.k mds.root_rsnaps=6 2024-03-08T21:16:11.805 INFO:tasks.check_counter:Daemon mds.k mds_server.req_mksnap_latency.avgcount=0 2024-03-08T21:16:11.805 INFO:tasks.check_counter:Daemon mds.k mds_server.req_rmsnap_latency.avgcount=0 2024-03-08T21:16:11.805 DEBUG:tasks.check_counter:Getting stats from c 2024-03-08T21:16:11.806 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761 shell --fsid dc20f0f8-dd8c-11ee-95c6-87774f69a715 -- ceph daemon mds.c perf dump 2024-03-08T21:16:12.264 INFO:teuthology.orchestra.run.smithi204.stderr:Inferring config /var/lib/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/mon.c/config 2024-03-08T21:16:12.960 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:16:12 smithi107 ceph-mon[31381]: pgmap v497: 97 pgs: 97 active+clean; 17 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 55 KiB/s wr, 30 op/s 2024-03-08T21:16:12.960 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:16:12 smithi107 ceph-mon[31381]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T21:16:13.008 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:16:12 smithi149 ceph-mon[35042]: pgmap v497: 97 pgs: 97 active+clean; 17 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 55 KiB/s wr, 30 op/s 2024-03-08T21:16:13.008 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:16:12 smithi149 ceph-mon[35042]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T21:16:13.122 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:16:12 smithi204 ceph-mon[34935]: pgmap v497: 97 pgs: 97 active+clean; 17 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 55 KiB/s wr, 30 op/s 2024-03-08T21:16:13.122 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:16:12 smithi204 ceph-mon[34935]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-08T21:16:13.724 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:16:13 smithi204 ceph-mon[34935]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T21:16:13.724 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:16:13 smithi204 ceph-mon[34935]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T21:16:13.724 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:16:13 smithi204 ceph-mon[34935]: from='mgr.15375 ' entity='mgr.x' 2024-03-08T21:16:13.960 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:16:13 smithi107 ceph-mon[31381]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T21:16:13.960 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:16:13 smithi107 ceph-mon[31381]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T21:16:13.961 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:16:13 smithi107 ceph-mon[31381]: from='mgr.15375 ' entity='mgr.x' 2024-03-08T21:16:14.008 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:16:13 smithi149 ceph-mon[35042]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-08T21:16:14.008 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:16:13 smithi149 ceph-mon[35042]: from='mgr.15375 172.21.15.107:0/4242172275' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-08T21:16:14.009 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:16:13 smithi149 ceph-mon[35042]: from='mgr.15375 ' entity='mgr.x' 2024-03-08T21:16:14.596 INFO:teuthology.orchestra.run.smithi204.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2024-03-08T21:16:14.960 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:16:14 smithi107 ceph-mon[31381]: pgmap v498: 97 pgs: 97 active+clean; 17 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 55 KiB/s wr, 30 op/s 2024-03-08T21:16:14.979 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:16:14 smithi204 ceph-mon[34935]: pgmap v498: 97 pgs: 97 active+clean; 17 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 55 KiB/s wr, 30 op/s 2024-03-08T21:16:15.008 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:16:14 smithi149 ceph-mon[35042]: pgmap v498: 97 pgs: 97 active+clean; 17 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 55 KiB/s wr, 30 op/s 2024-03-08T21:16:15.396 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-03-08T21:16:15.397 ERROR:teuthology.run_tasks:Manager failed: check-counter Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/task/__init__.py", line 132, in __exit__ self.end() File "/home/teuthworker/src/git.ceph.com_ceph-c_7d63482aa89628d733e298984a762389b625c761/qa/tasks/check_counter.py", line 84, in end proc = manager.admin_socket(daemon_type, daemon_id, ["perf", "dump"]) File "/home/teuthworker/src/git.ceph.com_ceph-c_7d63482aa89628d733e298984a762389b625c761/qa/tasks/ceph_manager.py", line 1875, in admin_socket return shell( File "/home/teuthworker/src/git.ceph.com_ceph-c_7d63482aa89628d733e298984a762389b625c761/qa/tasks/ceph_manager.py", line 43, in shell return remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi204 with status 22: 'sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761 shell --fsid dc20f0f8-dd8c-11ee-95c6-87774f69a715 -- ceph daemon mds.c perf dump' 2024-03-08T21:16:15.399 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2024-03-08T21:16:15.410 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-03-08T21:16:15.410 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-03-08T21:16:15.410 DEBUG:teuthology.orchestra.run.smithi107:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-08T21:16:15.470 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-03-08T21:16:15.470 DEBUG:teuthology.orchestra.run.smithi107:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-08T21:16:15.521 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-08T21:16:15.783 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:15.807+0000 7f106dc00640 1 -- 172.21.15.107:0/3504246638 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1068109580 msgr2=0x7f1068109960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:16:15.783 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:15.807+0000 7f106dc00640 1 --2- 172.21.15.107:0/3504246638 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1068109580 0x7f1068109960 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f1058009900 tx=0x7f105802bc50 comp rx=0 tx=0).stop 2024-03-08T21:16:15.783 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:15.807+0000 7f106dc00640 1 -- 172.21.15.107:0/3504246638 shutdown_connections 2024-03-08T21:16:15.783 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:15.807+0000 7f106dc00640 1 --2- 172.21.15.107:0/3504246638 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1068103e80 0x7f1068110340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:16:15.783 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:15.807+0000 7f106dc00640 1 --2- 172.21.15.107:0/3504246638 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f10681034c0 0x7f1068103940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:16:15.783 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:15.807+0000 7f106dc00640 1 --2- 172.21.15.107:0/3504246638 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1068109580 0x7f1068109960 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:16:15.784 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:15.807+0000 7f106dc00640 1 -- 172.21.15.107:0/3504246638 >> 172.21.15.107:0/3504246638 conn(0x7f10680fef30 msgr2=0x7f1068101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:16:15.784 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:15.807+0000 7f106dc00640 1 -- 172.21.15.107:0/3504246638 shutdown_connections 2024-03-08T21:16:15.784 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:15.807+0000 7f106dc00640 1 -- 172.21.15.107:0/3504246638 wait complete. 2024-03-08T21:16:15.786 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:15.808+0000 7f106dc00640 1 Processor -- start 2024-03-08T21:16:15.786 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:15.808+0000 7f106dc00640 1 -- start start 2024-03-08T21:16:15.786 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:15.808+0000 7f106dc00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f10681034c0 0x7f1068199dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:16:15.786 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:15.808+0000 7f106dc00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1068103e80 0x7f106819a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:16:15.786 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:15.809+0000 7f106dc00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1068109580 0x7f106819e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:16:15.786 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:15.809+0000 7f106dc00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1068113840 con 0x7f10681034c0 2024-03-08T21:16:15.786 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:15.809+0000 7f106dc00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10681136c0 con 0x7f1068103e80 2024-03-08T21:16:15.787 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:15.809+0000 7f106dc00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10681139c0 con 0x7f1068109580 2024-03-08T21:16:15.787 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:15.809+0000 7f1066a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f10681034c0 0x7f1068199dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:16:15.787 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:15.809+0000 7f1066a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f10681034c0 0x7f1068199dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:55324/0 (socket says 172.21.15.107:55324) 2024-03-08T21:16:15.787 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:15.809+0000 7f1066a00640 1 -- 172.21.15.107:0/3629324684 learned_addr learned my addr 172.21.15.107:0/3629324684 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:16:15.787 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:15.809+0000 7f105e000640 1 --2- 172.21.15.107:0/3629324684 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1068103e80 0x7f106819a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:16:15.787 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:15.809+0000 7f1067400640 1 --2- 172.21.15.107:0/3629324684 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1068109580 0x7f106819e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:16:15.787 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:15.809+0000 7f1066a00640 1 -- 172.21.15.107:0/3629324684 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1068109580 msgr2=0x7f106819e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:16:15.787 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:15.809+0000 7f1066a00640 1 --2- 172.21.15.107:0/3629324684 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1068109580 0x7f106819e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:16:15.787 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:15.809+0000 7f1066a00640 1 -- 172.21.15.107:0/3629324684 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1068103e80 msgr2=0x7f106819a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:16:15.787 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:15.809+0000 7f1066a00640 1 --2- 172.21.15.107:0/3629324684 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1068103e80 0x7f106819a300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:16:15.787 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:15.809+0000 7f1066a00640 1 -- 172.21.15.107:0/3629324684 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f106819edb0 con 0x7f10681034c0 2024-03-08T21:16:15.787 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:15.809+0000 7f105e000640 1 --2- 172.21.15.107:0/3629324684 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1068103e80 0x7f106819a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T21:16:15.787 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:15.809+0000 7f1067400640 1 --2- 172.21.15.107:0/3629324684 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1068109580 0x7f106819e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T21:16:15.788 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:15.810+0000 7f1066a00640 1 --2- 172.21.15.107:0/3629324684 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f10681034c0 0x7f1068199dc0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f1058000c00 tx=0x7f105802ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:16:15.788 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:15.810+0000 7f105fe00640 1 -- 172.21.15.107:0/3629324684 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1058043020 con 0x7f10681034c0 2024-03-08T21:16:15.788 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:15.810+0000 7f105fe00640 1 -- 172.21.15.107:0/3629324684 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f105802cc40 con 0x7f10681034c0 2024-03-08T21:16:15.788 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:15.810+0000 7f106dc00640 1 -- 172.21.15.107:0/3629324684 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f106819f040 con 0x7f10681034c0 2024-03-08T21:16:15.788 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:15.810+0000 7f105fe00640 1 -- 172.21.15.107:0/3629324684 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1058003c90 con 0x7f10681034c0 2024-03-08T21:16:15.788 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:15.810+0000 7f106dc00640 1 -- 172.21.15.107:0/3629324684 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10681a6930 con 0x7f10681034c0 2024-03-08T21:16:15.789 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:15.811+0000 7f106dc00640 1 -- 172.21.15.107:0/3629324684 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10680680e0 con 0x7f10681034c0 2024-03-08T21:16:15.789 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:15.813+0000 7f105fe00640 1 -- 172.21.15.107:0/3629324684 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) v1 ==== 137422+0+0 (secure 0 0 0) 0x7f1058046020 con 0x7f10681034c0 2024-03-08T21:16:15.790 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:15.814+0000 7f105fe00640 1 --2- 172.21.15.107:0/3629324684 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f104c0a2970 0x7f104c0a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:16:15.790 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:15.814+0000 7f105fe00640 1 -- 172.21.15.107:0/3629324684 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 8629+0+0 (secure 0 0 0) 0x7f10580f1610 con 0x7f10681034c0 2024-03-08T21:16:15.790 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:15.815+0000 7f105e000640 1 --2- 172.21.15.107:0/3629324684 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f104c0a2970 0x7f104c0a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:16:15.791 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:15.815+0000 7f105e000640 1 --2- 172.21.15.107:0/3629324684 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f104c0a2970 0x7f104c0a4e30 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f1050006fd0 tx=0x7f1050008040 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:16:15.793 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:15.817+0000 7f105fe00640 1 -- 172.21.15.107:0/3629324684 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f105804d020 con 0x7f10681034c0 2024-03-08T21:16:15.992 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:16.016+0000 7f106dc00640 1 -- 172.21.15.107:0/3629324684 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f106810b970 con 0x7f10681034c0 2024-03-08T21:16:15.995 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:16.020+0000 7f105fe00640 1 -- 172.21.15.107:0/3629324684 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v114) v1 ==== 74+0+23385 (secure 0 0 0) 0x7f10580be3b0 con 0x7f10681034c0 2024-03-08T21:16:15.996 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-08T21:16:15.996 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":114,"fsid":"dc20f0f8-dd8c-11ee-95c6-87774f69a715","created":"2024-03-08T20:47:04.190034+0000","modified":"2024-03-08T21:16:09.695540+0000","last_up_change":"2024-03-08T20:56:43.135514+0000","last_in_change":"2024-03-08T20:56:17.577074+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-08T20:51:32.832786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-08T20:57:48.721227+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-08T20:57:49.707779+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"114","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":27,"snap_epoch":114,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8b1ddc64-4d20-4e85-ac66-f1c5ac599501","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":2453946095},{"type":"v1","addr":"172.21.15.107:6803","nonce":2453946095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":2453946095},{"type":"v1","addr":"172.21.15.107:6805","nonce":2453946095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":2453946095},{"type":"v1","addr":"172.21.15.107:6809","nonce":2453946095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":2453946095},{"type":"v1","addr":"172.21.15.107:6807","nonce":2453946095}]},"public_addr":"172.21.15.107:6803/2453946095","cluster_addr":"172.21.15.107:6805/2453946095","heartbeat_back_addr":"172.21.15.107:6809/2453946095","heartbeat_front_addr":"172.21.15.107:6807/2453946095","state":["exists","up"]},{"osd":1,"uuid":"5a803c09-202a-4777-ba2a-1da8eaf26a54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":3071316419},{"type":"v1","addr":"172.21.15.149:6801","nonce":3071316419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":3071316419},{"type":"v1","addr":"172.21.15.149:6803","nonce":3071316419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":3071316419},{"type":"v1","addr":"172.21.15.149:6807","nonce":3071316419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":3071316419},{"type":"v1","addr":"172.21.15.149:6805","nonce":3071316419}]},"public_addr":"172.21.15.149:6801/3071316419","cluster_addr":"172.21.15.149:6803/3071316419","heartbeat_back_addr":"172.21.15.149:6807/3071316419","heartbeat_front_addr":"172.21.15.149:6805/3071316419","state":["exists","up"]},{"osd":2,"uuid":"555c634f-347e-47dd-a943-e50ae2be073c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":1087852630},{"type":"v1","addr":"172.21.15.204:6801","nonce":1087852630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":1087852630},{"type":"v1","addr":"172.21.15.204:6803","nonce":1087852630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":1087852630},{"type":"v1","addr":"172.21.15.204:6807","nonce":1087852630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":1087852630},{"type":"v1","addr":"172.21.15.204:6805","nonce":1087852630}]},"public_addr":"172.21.15.204:6801/1087852630","cluster_addr":"172.21.15.204:6803/1087852630","heartbeat_back_addr":"172.21.15.204:6807/1087852630","heartbeat_front_addr":"172.21.15.204:6805/1087852630","state":["exists","up"]},{"osd":3,"uuid":"9e433d97-e202-48e0-9ed1-f0988fa14307","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":3123579789},{"type":"v1","addr":"172.21.15.107:6811","nonce":3123579789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":3123579789},{"type":"v1","addr":"172.21.15.107:6813","nonce":3123579789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":3123579789},{"type":"v1","addr":"172.21.15.107:6817","nonce":3123579789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":3123579789},{"type":"v1","addr":"172.21.15.107:6815","nonce":3123579789}]},"public_addr":"172.21.15.107:6811/3123579789","cluster_addr":"172.21.15.107:6813/3123579789","heartbeat_back_addr":"172.21.15.107:6817/3123579789","heartbeat_front_addr":"172.21.15.107:6815/3123579789","state":["exists","up"]},{"osd":4,"uuid":"27c53794-9d2b-487d-bfcf-ac4fd2ab061e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":726286918},{"type":"v1","addr":"172.21.15.149:6809","nonce":726286918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":726286918},{"type":"v1","addr":"172.21.15.149:6811","nonce":726286918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":726286918},{"type":"v1","addr":"172.21.15.149:6815","nonce":726286918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":726286918},{"type":"v1","addr":"172.21.15.149:6813","nonce":726286918}]},"public_addr":"172.21.15.149:6809/726286918","cluster_addr":"172.21.15.149:6811/726286918","heartbeat_back_addr":"172.21.15.149:6815/726286918","heartbeat_front_addr":"172.21.15.149:6813/726286918","state":["exists","up"]},{"osd":5,"uuid":"3a7778fb-46c5-40c0-842e-e988c45ea24d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":3143887820},{"type":"v1","addr":"172.21.15.204:6809","nonce":3143887820}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":3143887820},{"type":"v1","addr":"172.21.15.204:6811","nonce":3143887820}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":3143887820},{"type":"v1","addr":"172.21.15.204:6815","nonce":3143887820}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":3143887820},{"type":"v1","addr":"172.21.15.204:6813","nonce":3143887820}]},"public_addr":"172.21.15.204:6809/3143887820","cluster_addr":"172.21.15.204:6811/3143887820","heartbeat_back_addr":"172.21.15.204:6815/3143887820","heartbeat_front_addr":"172.21.15.204:6813/3143887820","state":["exists","up"]},{"osd":6,"uuid":"d2e05a56-be4f-4435-9b76-ff9aa01e4ac3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":2912452950},{"type":"v1","addr":"172.21.15.107:6819","nonce":2912452950}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":2912452950},{"type":"v1","addr":"172.21.15.107:6821","nonce":2912452950}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":2912452950},{"type":"v1","addr":"172.21.15.107:6825","nonce":2912452950}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":2912452950},{"type":"v1","addr":"172.21.15.107:6823","nonce":2912452950}]},"public_addr":"172.21.15.107:6819/2912452950","cluster_addr":"172.21.15.107:6821/2912452950","heartbeat_back_addr":"172.21.15.107:6825/2912452950","heartbeat_front_addr":"172.21.15.107:6823/2912452950","state":["exists","up"]},{"osd":7,"uuid":"a7dd26ee-c118-4a9a-9e19-142c2680902d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":3835863809},{"type":"v1","addr":"172.21.15.149:6817","nonce":3835863809}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":3835863809},{"type":"v1","addr":"172.21.15.149:6819","nonce":3835863809}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":3835863809},{"type":"v1","addr":"172.21.15.149:6823","nonce":3835863809}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":3835863809},{"type":"v1","addr":"172.21.15.149:6821","nonce":3835863809}]},"public_addr":"172.21.15.149:6817/3835863809","cluster_addr":"172.21.15.149:6819/3835863809","heartbeat_back_addr":"172.21.15.149:6823/3835863809","heartbeat_front_addr":"172.21.15.149:6821/3835863809","state":["exists","up"]},{"osd":8,"uuid":"baaaa8dd-0e1f-4bae-86bf-9e429d985f23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":1010601477},{"type":"v1","addr":"172.21.15.204:6817","nonce":1010601477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":1010601477},{"type":"v1","addr":"172.21.15.204:6819","nonce":1010601477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":1010601477},{"type":"v1","addr":"172.21.15.204:6823","nonce":1010601477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":1010601477},{"type":"v1","addr":"172.21.15.204:6821","nonce":1010601477}]},"public_addr":"172.21.15.204:6817/1010601477","cluster_addr":"172.21.15.204:6819/1010601477","heartbeat_back_addr":"172.21.15.204:6823/1010601477","heartbeat_front_addr":"172.21.15.204:6821/1010601477","state":["exists","up"]},{"osd":9,"uuid":"8df80dfa-e7ff-410e-824c-b1dce111b02f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":3334759819},{"type":"v1","addr":"172.21.15.107:6827","nonce":3334759819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3334759819},{"type":"v1","addr":"172.21.15.107:6829","nonce":3334759819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":3334759819},{"type":"v1","addr":"172.21.15.107:6833","nonce":3334759819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":3334759819},{"type":"v1","addr":"172.21.15.107:6831","nonce":3334759819}]},"public_addr":"172.21.15.107:6827/3334759819","cluster_addr":"172.21.15.107:6829/3334759819","heartbeat_back_addr":"172.21.15.107:6833/3334759819","heartbeat_front_addr":"172.21.15.107:6831/3334759819","state":["exists","up"]},{"osd":10,"uuid":"338945b2-20c0-40b2-aaca-062cd4adee04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":3469284631},{"type":"v1","addr":"172.21.15.149:6825","nonce":3469284631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":3469284631},{"type":"v1","addr":"172.21.15.149:6827","nonce":3469284631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":3469284631},{"type":"v1","addr":"172.21.15.149:6831","nonce":3469284631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":3469284631},{"type":"v1","addr":"172.21.15.149:6829","nonce":3469284631}]},"public_addr":"172.21.15.149:6825/3469284631","cluster_addr":"172.21.15.149:6827/3469284631","heartbeat_back_addr":"172.21.15.149:6831/3469284631","heartbeat_front_addr":"172.21.15.149:6829/3469284631","state":["exists","up"]},{"osd":11,"uuid":"acccfe4f-d307-4a64-8f4d-fba27bfbe09d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":3097772473},{"type":"v1","addr":"172.21.15.204:6825","nonce":3097772473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":3097772473},{"type":"v1","addr":"172.21.15.204:6827","nonce":3097772473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":3097772473},{"type":"v1","addr":"172.21.15.204:6831","nonce":3097772473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":3097772473},{"type":"v1","addr":"172.21.15.204:6829","nonce":3097772473}]},"public_addr":"172.21.15.204:6825/3097772473","cluster_addr":"172.21.15.204:6827/3097772473","heartbeat_back_addr":"172.21.15.204:6831/3097772473","heartbeat_front_addr":"172.21.15.204:6829/3097772473","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T20:50:25.862753+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T20:50:58.167663+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T20:51:30.312816+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T20:52:05.487917+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T20:52:39.348354+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T20:53:12.899002+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T20:53:47.630837+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T20:54:21.444552+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T20:54:57.101160+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T20:55:31.841786+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T20:56:05.827161+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T20:56:40.611097+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.0","mappings":[{"from":4,"to":6}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.3b","mappings":[{"from":3,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.107:0/808124991":"2024-03-09T20:48:20.149406+0000","172.21.15.107:6801/3376636618":"2024-03-09T21:00:32.431358+0000","172.21.15.107:0/2482452215":"2024-03-09T20:47:45.706790+0000","172.21.15.107:6801/3549936519":"2024-03-09T20:47:45.706790+0000","172.21.15.107:0/3746019488":"2024-03-09T21:00:32.431358+0000","172.21.15.107:0/3691962590":"2024-03-09T21:00:32.431358+0000","172.21.15.107:0/1206995985":"2024-03-09T20:47:45.706790+0000","172.21.15.107:6800/3549936519":"2024-03-09T20:47:45.706790+0000","172.21.15.107:0/2392547434":"2024-03-09T20:47:45.706790+0000","172.21.15.107:6800/426372446":"2024-03-09T20:48:20.149406+0000","172.21.15.107:6801/426372446":"2024-03-09T20:48:20.149406+0000","172.21.15.107:0/1886016707":"2024-03-09T20:48:20.149406+0000","172.21.15.107:6800/3376636618":"2024-03-09T21:00:32.431358+0000","172.21.15.107:0/1514666962":"2024-03-09T20:48:20.149406+0000","172.21.15.107:0/552952579":"2024-03-09T21:00:32.431358+0000","172.21.15.107:0/458107598":"2024-03-09T21:00:32.431358+0000","172.21.15.107:0/356063043":"2024-03-09T21:00:32.431358+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":3,"snaps":[{"begin":14,"length":1},{"begin":27,"length":1}]}],"new_removed_snaps":[{"pool":3,"snaps":[{"begin":14,"length":1},{"begin":27,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-08T21:16:15.999 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:16.023+0000 7f106dc00640 1 -- 172.21.15.107:0/3629324684 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f104c0a2970 msgr2=0x7f104c0a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:16:15.999 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:16.023+0000 7f106dc00640 1 --2- 172.21.15.107:0/3629324684 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f104c0a2970 0x7f104c0a4e30 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f1050006fd0 tx=0x7f1050008040 comp rx=0 tx=0).stop 2024-03-08T21:16:15.999 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:16.023+0000 7f106dc00640 1 -- 172.21.15.107:0/3629324684 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f10681034c0 msgr2=0x7f1068199dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:16:15.999 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:16.023+0000 7f106dc00640 1 --2- 172.21.15.107:0/3629324684 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f10681034c0 0x7f1068199dc0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f1058000c00 tx=0x7f105802ca50 comp rx=0 tx=0).stop 2024-03-08T21:16:15.999 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:16.023+0000 7f106dc00640 1 -- 172.21.15.107:0/3629324684 shutdown_connections 2024-03-08T21:16:15.999 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:16.023+0000 7f106dc00640 1 --2- 172.21.15.107:0/3629324684 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f104c0a2970 0x7f104c0a4e30 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:16:15.999 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:16.023+0000 7f106dc00640 1 --2- 172.21.15.107:0/3629324684 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1068109580 0x7f106819e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:16:15.999 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:16.023+0000 7f106dc00640 1 --2- 172.21.15.107:0/3629324684 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1068103e80 0x7f106819a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:16:15.999 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:16.023+0000 7f106dc00640 1 --2- 172.21.15.107:0/3629324684 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f10681034c0 0x7f1068199dc0 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:16:16.000 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:16.023+0000 7f106dc00640 1 -- 172.21.15.107:0/3629324684 >> 172.21.15.107:0/3629324684 conn(0x7f10680fef30 msgr2=0x7f1068100860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:16:16.000 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:16.023+0000 7f106dc00640 1 -- 172.21.15.107:0/3629324684 shutdown_connections 2024-03-08T21:16:16.000 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:16.023+0000 7f106dc00640 1 -- 172.21.15.107:0/3629324684 wait complete. 2024-03-08T21:16:16.019 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-08T20:51:32.832786+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-08T20:57:48.721227+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-08T20:57:49.707779+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '114', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 27, 'snap_epoch': 114, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-08T21:16:16.020 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-08T21:16:16.237 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:16.260+0000 7fa4a4a00640 1 -- 172.21.15.107:0/1284300664 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa4a01081d0 msgr2=0x7fa4a0114620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:16:16.237 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:16.260+0000 7fa4a4a00640 1 --2- 172.21.15.107:0/1284300664 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa4a01081d0 0x7fa4a0114620 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7fa498009ca0 tx=0x7fa49802c180 comp rx=0 tx=0).stop 2024-03-08T21:16:16.237 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:16.261+0000 7fa4a4a00640 1 -- 172.21.15.107:0/1284300664 shutdown_connections 2024-03-08T21:16:16.237 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:16.261+0000 7fa4a4a00640 1 --2- 172.21.15.107:0/1284300664 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa4a01081d0 0x7fa4a0114620 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:16:16.237 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:16.261+0000 7fa4a4a00640 1 --2- 172.21.15.107:0/1284300664 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa4a0107830 0x7fa4a0107c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:16:16.237 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:16.261+0000 7fa4a4a00640 1 --2- 172.21.15.107:0/1284300664 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa4a010d7d0 0x7fa4a010dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:16:16.237 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:16.261+0000 7fa4a4a00640 1 -- 172.21.15.107:0/1284300664 >> 172.21.15.107:0/1284300664 conn(0x7fa4a0077110 msgr2=0x7fa4a0106110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:16:16.237 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:16.261+0000 7fa4a4a00640 1 -- 172.21.15.107:0/1284300664 shutdown_connections 2024-03-08T21:16:16.237 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:16.261+0000 7fa4a4a00640 1 -- 172.21.15.107:0/1284300664 wait complete. 2024-03-08T21:16:16.240 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:16.262+0000 7fa4a4a00640 1 Processor -- start 2024-03-08T21:16:16.240 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:16.262+0000 7fa4a4a00640 1 -- start start 2024-03-08T21:16:16.240 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:16.262+0000 7fa4a4a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa4a0107830 0x7fa4a019dec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:16:16.240 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:16.262+0000 7fa4a4a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa4a01081d0 0x7fa4a019e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:16:16.240 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:16.262+0000 7fa4a4a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa4a010d7d0 0x7fa4a01a2730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:16:16.240 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:16.262+0000 7fa4a4a00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4a0117a40 con 0x7fa4a010d7d0 2024-03-08T21:16:16.240 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:16.262+0000 7fa4a4a00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4a01178c0 con 0x7fa4a0107830 2024-03-08T21:16:16.240 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:16.262+0000 7fa4a4a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4a0117bc0 con 0x7fa4a01081d0 2024-03-08T21:16:16.240 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:16.263+0000 7fa49f200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa4a010d7d0 0x7fa4a01a2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:16:16.240 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:16.263+0000 7fa49f200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa4a010d7d0 0x7fa4a01a2730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46016/0 (socket says 172.21.15.107:46016) 2024-03-08T21:16:16.240 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:16.263+0000 7fa49f200640 1 -- 172.21.15.107:0/3614076001 learned_addr learned my addr 172.21.15.107:0/3614076001 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:16:16.240 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:16.263+0000 7fa49de00640 1 --2- 172.21.15.107:0/3614076001 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa4a01081d0 0x7fa4a019e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:16:16.240 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:16.263+0000 7fa49e800640 1 --2- 172.21.15.107:0/3614076001 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa4a0107830 0x7fa4a019dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:16:16.241 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:16.263+0000 7fa49f200640 1 -- 172.21.15.107:0/3614076001 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa4a01081d0 msgr2=0x7fa4a019e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:16:16.241 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:16.263+0000 7fa49f200640 1 --2- 172.21.15.107:0/3614076001 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa4a01081d0 0x7fa4a019e400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:16:16.241 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:16.263+0000 7fa49f200640 1 -- 172.21.15.107:0/3614076001 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa4a0107830 msgr2=0x7fa4a019dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:16:16.241 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:16.263+0000 7fa49f200640 1 --2- 172.21.15.107:0/3614076001 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa4a0107830 0x7fa4a019dec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:16:16.241 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:16.263+0000 7fa49f200640 1 -- 172.21.15.107:0/3614076001 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4a01a2e60 con 0x7fa4a010d7d0 2024-03-08T21:16:16.241 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:16.264+0000 7fa49f200640 1 --2- 172.21.15.107:0/3614076001 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa4a010d7d0 0x7fa4a01a2730 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7fa49802c790 tx=0x7fa498003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:16:16.241 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:16.264+0000 7fa497400640 1 -- 172.21.15.107:0/3614076001 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa498039040 con 0x7fa4a010d7d0 2024-03-08T21:16:16.241 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:16.264+0000 7fa497400640 1 -- 172.21.15.107:0/3614076001 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa498044070 con 0x7fa4a010d7d0 2024-03-08T21:16:16.241 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:16.264+0000 7fa4a4a00640 1 -- 172.21.15.107:0/3614076001 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4a01a30f0 con 0x7fa4a010d7d0 2024-03-08T21:16:16.241 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:16.264+0000 7fa497400640 1 -- 172.21.15.107:0/3614076001 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4980329f0 con 0x7fa4a010d7d0 2024-03-08T21:16:16.241 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:16.264+0000 7fa4a4a00640 1 -- 172.21.15.107:0/3614076001 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4a01aa9e0 con 0x7fa4a010d7d0 2024-03-08T21:16:16.242 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:16.265+0000 7fa4a4a00640 1 -- 172.21.15.107:0/3614076001 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4a0068450 con 0x7fa4a010d7d0 2024-03-08T21:16:16.243 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:16.267+0000 7fa497400640 1 -- 172.21.15.107:0/3614076001 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) v1 ==== 137422+0+0 (secure 0 0 0) 0x7fa498032b90 con 0x7fa4a010d7d0 2024-03-08T21:16:16.244 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:16.268+0000 7fa497400640 1 --2- 172.21.15.107:0/3614076001 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7fa4840a2970 0x7fa4840a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:16:16.244 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:16.269+0000 7fa497400640 1 -- 172.21.15.107:0/3614076001 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 8629+0+0 (secure 0 0 0) 0x7fa4980f16c0 con 0x7fa4a010d7d0 2024-03-08T21:16:16.244 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:16.269+0000 7fa49e800640 1 --2- 172.21.15.107:0/3614076001 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7fa4840a2970 0x7fa4840a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:16:16.245 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:16.269+0000 7fa49e800640 1 --2- 172.21.15.107:0/3614076001 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7fa4840a2970 0x7fa4840a4e30 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fa490005d90 tx=0x7fa490005d20 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:16:16.248 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:16.273+0000 7fa497400640 1 -- 172.21.15.107:0/3614076001 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7fa4980be3c0 con 0x7fa4a010d7d0 2024-03-08T21:16:16.447 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:16.472+0000 7fa4a4a00640 1 -- 172.21.15.107:0/3614076001 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa4a010fc70 con 0x7fa4a010d7d0 2024-03-08T21:16:16.448 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:16.472+0000 7fa497400640 1 -- 172.21.15.107:0/3614076001 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v114) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa498035e20 con 0x7fa4a010d7d0 2024-03-08T21:16:16.448 INFO:teuthology.orchestra.run.smithi107.stdout:pg_num: 1 2024-03-08T21:16:16.451 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:16.475+0000 7fa4a4a00640 1 -- 172.21.15.107:0/3614076001 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7fa4840a2970 msgr2=0x7fa4840a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:16:16.451 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:16.475+0000 7fa4a4a00640 1 --2- 172.21.15.107:0/3614076001 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7fa4840a2970 0x7fa4840a4e30 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fa490005d90 tx=0x7fa490005d20 comp rx=0 tx=0).stop 2024-03-08T21:16:16.451 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:16.475+0000 7fa4a4a00640 1 -- 172.21.15.107:0/3614076001 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa4a010d7d0 msgr2=0x7fa4a01a2730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:16:16.451 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:16.475+0000 7fa4a4a00640 1 --2- 172.21.15.107:0/3614076001 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa4a010d7d0 0x7fa4a01a2730 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7fa49802c790 tx=0x7fa498003c20 comp rx=0 tx=0).stop 2024-03-08T21:16:16.451 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:16.475+0000 7fa4a4a00640 1 -- 172.21.15.107:0/3614076001 shutdown_connections 2024-03-08T21:16:16.451 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:16.475+0000 7fa4a4a00640 1 --2- 172.21.15.107:0/3614076001 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7fa4840a2970 0x7fa4840a4e30 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:16:16.451 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:16.475+0000 7fa4a4a00640 1 --2- 172.21.15.107:0/3614076001 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa4a010d7d0 0x7fa4a01a2730 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:16:16.452 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:16.475+0000 7fa4a4a00640 1 --2- 172.21.15.107:0/3614076001 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa4a01081d0 0x7fa4a019e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:16:16.452 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:16.475+0000 7fa4a4a00640 1 --2- 172.21.15.107:0/3614076001 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa4a0107830 0x7fa4a019dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:16:16.452 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:16.475+0000 7fa4a4a00640 1 -- 172.21.15.107:0/3614076001 >> 172.21.15.107:0/3614076001 conn(0x7fa4a0077110 msgr2=0x7fa4a0078170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:16:16.452 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:16.475+0000 7fa4a4a00640 1 -- 172.21.15.107:0/3614076001 shutdown_connections 2024-03-08T21:16:16.452 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:16.475+0000 7fa4a4a00640 1 -- 172.21.15.107:0/3614076001 wait complete. 2024-03-08T21:16:16.470 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-08T21:16:16.685 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:16.709+0000 7f7bf5600640 1 -- 172.21.15.107:0/548370356 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7bf0103e70 msgr2=0x7f7bf0110330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:16:16.685 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:16.709+0000 7f7bf5600640 1 --2- 172.21.15.107:0/548370356 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7bf0103e70 0x7f7bf0110330 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f7be8009900 tx=0x7f7be802bc30 comp rx=0 tx=0).stop 2024-03-08T21:16:16.685 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:16.709+0000 7f7bf5600640 1 -- 172.21.15.107:0/548370356 shutdown_connections 2024-03-08T21:16:16.685 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:16.709+0000 7f7bf5600640 1 --2- 172.21.15.107:0/548370356 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7bf0103e70 0x7f7bf0110330 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:16:16.685 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:16.709+0000 7f7bf5600640 1 --2- 172.21.15.107:0/548370356 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7bf01034b0 0x7f7bf0103930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:16:16.685 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:16.709+0000 7f7bf5600640 1 --2- 172.21.15.107:0/548370356 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7bf0109570 0x7f7bf0109950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:16:16.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:16.709+0000 7f7bf5600640 1 -- 172.21.15.107:0/548370356 >> 172.21.15.107:0/548370356 conn(0x7f7bf00fef60 msgr2=0x7f7bf0101380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:16:16.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:16.709+0000 7f7bf5600640 1 -- 172.21.15.107:0/548370356 shutdown_connections 2024-03-08T21:16:16.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:16.709+0000 7f7bf5600640 1 -- 172.21.15.107:0/548370356 wait complete. 2024-03-08T21:16:16.688 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:16.710+0000 7f7bf5600640 1 Processor -- start 2024-03-08T21:16:16.688 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:16.710+0000 7f7bf5600640 1 -- start start 2024-03-08T21:16:16.688 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:16.711+0000 7f7bf5600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7bf01034b0 0x7f7bf019f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:16:16.688 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:16.711+0000 7f7bf5600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7bf0103e70 0x7f7bf019fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:16:16.688 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:16.711+0000 7f7bf5600640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7bf0109570 0x7f7bf01999f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:16:16.688 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:16.711+0000 7f7bf5600640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bf0113750 con 0x7f7bf0103e70 2024-03-08T21:16:16.688 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:16.711+0000 7f7bf5600640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bf01135d0 con 0x7f7bf0109570 2024-03-08T21:16:16.688 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:16.711+0000 7f7bf5600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bf01138d0 con 0x7f7bf01034b0 2024-03-08T21:16:16.688 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:16.711+0000 7f7be6000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7bf0103e70 0x7f7bf019fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:16:16.689 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:16.711+0000 7f7be6000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7bf0103e70 0x7f7bf019fd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46034/0 (socket says 172.21.15.107:46034) 2024-03-08T21:16:16.689 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:16.711+0000 7f7be6000640 1 -- 172.21.15.107:0/240691090 learned_addr learned my addr 172.21.15.107:0/240691090 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:16:16.689 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:16.711+0000 7f7beea00640 1 --2- 172.21.15.107:0/240691090 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7bf01034b0 0x7f7bf019f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:16:16.689 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:16.711+0000 7f7bef400640 1 --2- 172.21.15.107:0/240691090 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7bf0109570 0x7f7bf01999f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:16:16.689 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:16.711+0000 7f7be6000640 1 -- 172.21.15.107:0/240691090 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7bf01034b0 msgr2=0x7f7bf019f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:16:16.689 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:16.711+0000 7f7be6000640 1 --2- 172.21.15.107:0/240691090 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7bf01034b0 0x7f7bf019f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:16:16.689 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:16.711+0000 7f7be6000640 1 -- 172.21.15.107:0/240691090 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7bf0109570 msgr2=0x7f7bf01999f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:16:16.689 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:16.711+0000 7f7be6000640 1 --2- 172.21.15.107:0/240691090 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7bf0109570 0x7f7bf01999f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:16:16.689 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:16.711+0000 7f7be6000640 1 -- 172.21.15.107:0/240691090 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bf019a170 con 0x7f7bf0103e70 2024-03-08T21:16:16.689 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:16.712+0000 7f7be6000640 1 --2- 172.21.15.107:0/240691090 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7bf0103e70 0x7f7bf019fd80 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f7bd800eab0 tx=0x7f7bd800ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:16:16.689 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:16.712+0000 7f7be7e00640 1 -- 172.21.15.107:0/240691090 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bd800ce40 con 0x7f7bf0103e70 2024-03-08T21:16:16.689 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:16.712+0000 7f7be7e00640 1 -- 172.21.15.107:0/240691090 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f7bd8011920 con 0x7f7bf0103e70 2024-03-08T21:16:16.689 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:16.712+0000 7f7be7e00640 1 -- 172.21.15.107:0/240691090 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bd8010a00 con 0x7f7bf0103e70 2024-03-08T21:16:16.690 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:16.712+0000 7f7bf5600640 1 -- 172.21.15.107:0/240691090 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bf019a460 con 0x7f7bf0103e70 2024-03-08T21:16:16.690 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:16.712+0000 7f7bf5600640 1 -- 172.21.15.107:0/240691090 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bf01a6700 con 0x7f7bf0103e70 2024-03-08T21:16:16.690 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:16.714+0000 7f7bf5600640 1 -- 172.21.15.107:0/240691090 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7bf0068530 con 0x7f7bf0103e70 2024-03-08T21:16:16.691 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:16.715+0000 7f7be7e00640 1 -- 172.21.15.107:0/240691090 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) v1 ==== 137422+0+0 (secure 0 0 0) 0x7f7bd8011ad0 con 0x7f7bf0103e70 2024-03-08T21:16:16.692 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:16.717+0000 7f7be7e00640 1 --2- 172.21.15.107:0/240691090 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f7bd40a29f0 0x7f7bd40a4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:16:16.692 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:16.717+0000 7f7be7e00640 1 -- 172.21.15.107:0/240691090 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 8629+0+0 (secure 0 0 0) 0x7f7bd80cf5b0 con 0x7f7bf0103e70 2024-03-08T21:16:16.693 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:16.717+0000 7f7beea00640 1 --2- 172.21.15.107:0/240691090 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f7bd40a29f0 0x7f7bd40a4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:16:16.693 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:16.718+0000 7f7beea00640 1 --2- 172.21.15.107:0/240691090 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f7bd40a29f0 0x7f7bd40a4eb0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f7be0005fd0 tx=0x7f7be0005850 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:16:16.696 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:16.721+0000 7f7be7e00640 1 -- 172.21.15.107:0/240691090 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f7bd8016030 con 0x7f7bf0103e70 2024-03-08T21:16:16.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:16.919+0000 7f7bf5600640 1 -- 172.21.15.107:0/240691090 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f7bf002ce70 con 0x7f7bf0103e70 2024-03-08T21:16:16.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:16.920+0000 7f7be7e00640 1 -- 172.21.15.107:0/240691090 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v114) v1 ==== 104+0+11 (secure 0 0 0) 0x7f7bd809c240 con 0x7f7bf0103e70 2024-03-08T21:16:16.896 INFO:teuthology.orchestra.run.smithi107.stdout:pg_num: 32 2024-03-08T21:16:16.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:16.923+0000 7f7bf5600640 1 -- 172.21.15.107:0/240691090 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f7bd40a29f0 msgr2=0x7f7bd40a4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:16:16.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:16.923+0000 7f7bf5600640 1 --2- 172.21.15.107:0/240691090 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f7bd40a29f0 0x7f7bd40a4eb0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f7be0005fd0 tx=0x7f7be0005850 comp rx=0 tx=0).stop 2024-03-08T21:16:16.899 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:16.923+0000 7f7bf5600640 1 -- 172.21.15.107:0/240691090 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7bf0103e70 msgr2=0x7f7bf019fd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:16:16.899 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:16.923+0000 7f7bf5600640 1 --2- 172.21.15.107:0/240691090 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7bf0103e70 0x7f7bf019fd80 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f7bd800eab0 tx=0x7f7bd800ef70 comp rx=0 tx=0).stop 2024-03-08T21:16:16.899 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:16.923+0000 7f7bf5600640 1 -- 172.21.15.107:0/240691090 shutdown_connections 2024-03-08T21:16:16.899 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:16.923+0000 7f7bf5600640 1 --2- 172.21.15.107:0/240691090 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f7bd40a29f0 0x7f7bd40a4eb0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:16:16.899 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:16.923+0000 7f7bf5600640 1 --2- 172.21.15.107:0/240691090 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7bf0109570 0x7f7bf01999f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:16:16.899 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:16.923+0000 7f7bf5600640 1 --2- 172.21.15.107:0/240691090 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7bf0103e70 0x7f7bf019fd80 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:16:16.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:16.923+0000 7f7bf5600640 1 --2- 172.21.15.107:0/240691090 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f7bf01034b0 0x7f7bf019f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:16:16.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:16.923+0000 7f7bf5600640 1 -- 172.21.15.107:0/240691090 >> 172.21.15.107:0/240691090 conn(0x7f7bf00fef60 msgr2=0x7f7bf01006e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:16:16.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:16.923+0000 7f7bf5600640 1 -- 172.21.15.107:0/240691090 shutdown_connections 2024-03-08T21:16:16.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:16.923+0000 7f7bf5600640 1 -- 172.21.15.107:0/240691090 wait complete. 2024-03-08T21:16:16.915 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:16:16 smithi107 ceph-mon[31381]: pgmap v499: 97 pgs: 97 active+clean; 17 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 307 B/s wr, 10 op/s 2024-03-08T21:16:16.915 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:16:16 smithi107 ceph-mon[31381]: from='client.? 172.21.15.107:0/3629324684' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-08T21:16:16.915 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:16:16 smithi107 ceph-mon[31381]: from='client.? 172.21.15.107:0/3614076001' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-08T21:16:16.917 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-08T21:16:17.008 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:16:16 smithi149 ceph-mon[35042]: pgmap v499: 97 pgs: 97 active+clean; 17 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 307 B/s wr, 10 op/s 2024-03-08T21:16:17.008 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:16:16 smithi149 ceph-mon[35042]: from='client.? 172.21.15.107:0/3629324684' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-08T21:16:17.009 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:16:16 smithi149 ceph-mon[35042]: from='client.? 172.21.15.107:0/3614076001' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-08T21:16:17.121 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:16:16 smithi204 ceph-mon[34935]: pgmap v499: 97 pgs: 97 active+clean; 17 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 307 B/s wr, 10 op/s 2024-03-08T21:16:17.122 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:16:16 smithi204 ceph-mon[34935]: from='client.? 172.21.15.107:0/3629324684' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-08T21:16:17.122 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:16:16 smithi204 ceph-mon[34935]: from='client.? 172.21.15.107:0/3614076001' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-08T21:16:17.128 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:17.152+0000 7efd29600640 1 -- 172.21.15.107:0/2238672735 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efd241034b0 msgr2=0x7efd24103930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:16:17.128 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:17.152+0000 7efd29600640 1 --2- 172.21.15.107:0/2238672735 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efd241034b0 0x7efd24103930 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7efd10009900 tx=0x7efd1002bc30 comp rx=0 tx=0).stop 2024-03-08T21:16:17.128 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:17.152+0000 7efd29600640 1 -- 172.21.15.107:0/2238672735 shutdown_connections 2024-03-08T21:16:17.128 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:17.152+0000 7efd29600640 1 --2- 172.21.15.107:0/2238672735 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efd24103e70 0x7efd24110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:16:17.129 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:17.152+0000 7efd29600640 1 --2- 172.21.15.107:0/2238672735 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efd241034b0 0x7efd24103930 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:16:17.129 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:17.152+0000 7efd29600640 1 --2- 172.21.15.107:0/2238672735 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efd24109570 0x7efd24109950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:16:17.129 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:17.152+0000 7efd29600640 1 -- 172.21.15.107:0/2238672735 >> 172.21.15.107:0/2238672735 conn(0x7efd240fef40 msgr2=0x7efd24101360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:16:17.129 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:17.153+0000 7efd29600640 1 -- 172.21.15.107:0/2238672735 shutdown_connections 2024-03-08T21:16:17.129 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:17.153+0000 7efd29600640 1 -- 172.21.15.107:0/2238672735 wait complete. 2024-03-08T21:16:17.131 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:17.153+0000 7efd29600640 1 Processor -- start 2024-03-08T21:16:17.131 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:17.154+0000 7efd29600640 1 -- start start 2024-03-08T21:16:17.131 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:17.154+0000 7efd29600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efd241034b0 0x7efd2419f920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:16:17.131 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:17.154+0000 7efd29600640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efd24103e70 0x7efd2419fe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:16:17.132 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:17.154+0000 7efd29600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efd24109570 0x7efd24199b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:16:17.132 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:17.154+0000 7efd29600640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd24113700 con 0x7efd241034b0 2024-03-08T21:16:17.132 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:17.154+0000 7efd29600640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd24113580 con 0x7efd24103e70 2024-03-08T21:16:17.132 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:17.154+0000 7efd29600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd24113880 con 0x7efd24109570 2024-03-08T21:16:17.132 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:17.154+0000 7efd22a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efd241034b0 0x7efd2419f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:16:17.132 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:17.154+0000 7efd22a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efd241034b0 0x7efd2419f920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46042/0 (socket says 172.21.15.107:46042) 2024-03-08T21:16:17.132 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:17.154+0000 7efd22a00640 1 -- 172.21.15.107:0/4075050977 learned_addr learned my addr 172.21.15.107:0/4075050977 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:16:17.132 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:17.154+0000 7efd22000640 1 --2- 172.21.15.107:0/4075050977 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efd24103e70 0x7efd2419fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:16:17.132 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:17.155+0000 7efd23400640 1 --2- 172.21.15.107:0/4075050977 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efd24109570 0x7efd24199b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:16:17.132 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:17.155+0000 7efd22a00640 1 -- 172.21.15.107:0/4075050977 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efd24109570 msgr2=0x7efd24199b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:16:17.132 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:17.155+0000 7efd22a00640 1 --2- 172.21.15.107:0/4075050977 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efd24109570 0x7efd24199b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:16:17.132 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:17.155+0000 7efd22a00640 1 -- 172.21.15.107:0/4075050977 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efd24103e70 msgr2=0x7efd2419fe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:16:17.132 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:17.155+0000 7efd22a00640 1 --2- 172.21.15.107:0/4075050977 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efd24103e70 0x7efd2419fe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:16:17.132 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:17.155+0000 7efd22a00640 1 -- 172.21.15.107:0/4075050977 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd2419a2a0 con 0x7efd241034b0 2024-03-08T21:16:17.132 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:17.155+0000 7efd22000640 1 --2- 172.21.15.107:0/4075050977 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efd24103e70 0x7efd2419fe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T21:16:17.133 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:17.155+0000 7efd22a00640 1 --2- 172.21.15.107:0/4075050977 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efd241034b0 0x7efd2419f920 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7efd1400d950 tx=0x7efd1400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:16:17.133 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:17.155+0000 7efd1b400640 1 -- 172.21.15.107:0/4075050977 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd14017070 con 0x7efd241034b0 2024-03-08T21:16:17.133 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:17.155+0000 7efd1b400640 1 -- 172.21.15.107:0/4075050977 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7efd14009c30 con 0x7efd241034b0 2024-03-08T21:16:17.133 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:17.155+0000 7efd29600640 1 -- 172.21.15.107:0/4075050977 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd2419a590 con 0x7efd241034b0 2024-03-08T21:16:17.133 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:17.155+0000 7efd1b400640 1 -- 172.21.15.107:0/4075050977 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd1400f620 con 0x7efd241034b0 2024-03-08T21:16:17.133 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:17.156+0000 7efd29600640 1 -- 172.21.15.107:0/4075050977 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd241a67e0 con 0x7efd241034b0 2024-03-08T21:16:17.133 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:17.157+0000 7efd29600640 1 -- 172.21.15.107:0/4075050977 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd24068530 con 0x7efd241034b0 2024-03-08T21:16:17.134 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:17.158+0000 7efd1b400640 1 -- 172.21.15.107:0/4075050977 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) v1 ==== 137422+0+0 (secure 0 0 0) 0x7efd140109a0 con 0x7efd241034b0 2024-03-08T21:16:17.135 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:17.160+0000 7efd1b400640 1 --2- 172.21.15.107:0/4075050977 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7efd080a29f0 0x7efd080a4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:16:17.135 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:17.160+0000 7efd1b400640 1 -- 172.21.15.107:0/4075050977 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 8629+0+0 (secure 0 0 0) 0x7efd140ceb60 con 0x7efd241034b0 2024-03-08T21:16:17.136 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:17.160+0000 7efd22000640 1 --2- 172.21.15.107:0/4075050977 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7efd080a29f0 0x7efd080a4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:16:17.136 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:17.161+0000 7efd22000640 1 --2- 172.21.15.107:0/4075050977 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7efd080a29f0 0x7efd080a4eb0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7efd241087b0 tx=0x7efd10005db0 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:16:17.139 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:17.164+0000 7efd1b400640 1 -- 172.21.15.107:0/4075050977 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7efd14009de0 con 0x7efd241034b0 2024-03-08T21:16:17.339 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:17.363+0000 7efd29600640 1 -- 172.21.15.107:0/4075050977 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7efd2402ce70 con 0x7efd241034b0 2024-03-08T21:16:17.340 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:17.364+0000 7efd1b400640 1 -- 172.21.15.107:0/4075050977 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v114) v1 ==== 100+0+11 (secure 0 0 0) 0x7efd1409b7f0 con 0x7efd241034b0 2024-03-08T21:16:17.340 INFO:teuthology.orchestra.run.smithi107.stdout:pg_num: 64 2024-03-08T21:16:17.343 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:17.367+0000 7efd29600640 1 -- 172.21.15.107:0/4075050977 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7efd080a29f0 msgr2=0x7efd080a4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:16:17.343 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:17.367+0000 7efd29600640 1 --2- 172.21.15.107:0/4075050977 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7efd080a29f0 0x7efd080a4eb0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7efd241087b0 tx=0x7efd10005db0 comp rx=0 tx=0).stop 2024-03-08T21:16:17.343 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:17.367+0000 7efd29600640 1 -- 172.21.15.107:0/4075050977 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efd241034b0 msgr2=0x7efd2419f920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:16:17.343 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:17.367+0000 7efd29600640 1 --2- 172.21.15.107:0/4075050977 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efd241034b0 0x7efd2419f920 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7efd1400d950 tx=0x7efd1400de10 comp rx=0 tx=0).stop 2024-03-08T21:16:17.343 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:17.367+0000 7efd29600640 1 -- 172.21.15.107:0/4075050977 shutdown_connections 2024-03-08T21:16:17.343 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:17.367+0000 7efd29600640 1 --2- 172.21.15.107:0/4075050977 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7efd080a29f0 0x7efd080a4eb0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:16:17.343 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:17.367+0000 7efd29600640 1 --2- 172.21.15.107:0/4075050977 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7efd24109570 0x7efd24199b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:16:17.344 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:17.367+0000 7efd29600640 1 --2- 172.21.15.107:0/4075050977 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efd24103e70 0x7efd2419fe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:16:17.344 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:17.367+0000 7efd29600640 1 --2- 172.21.15.107:0/4075050977 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efd241034b0 0x7efd2419f920 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:16:17.344 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:17.367+0000 7efd29600640 1 -- 172.21.15.107:0/4075050977 >> 172.21.15.107:0/4075050977 conn(0x7efd240fef40 msgr2=0x7efd24100930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:16:17.344 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:17.368+0000 7efd29600640 1 -- 172.21.15.107:0/4075050977 shutdown_connections 2024-03-08T21:16:17.344 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:17.368+0000 7efd29600640 1 -- 172.21.15.107:0/4075050977 wait complete. 2024-03-08T21:16:17.362 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-08T21:16:17.573 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:17.597+0000 7f165be00640 1 -- 172.21.15.107:0/1251481449 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1654108250 msgr2=0x7f1654114760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:16:17.573 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:17.597+0000 7f165be00640 1 --2- 172.21.15.107:0/1251481449 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1654108250 0x7f1654114760 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f164c009cd0 tx=0x7f164c02c0e0 comp rx=0 tx=0).stop 2024-03-08T21:16:17.573 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:17.597+0000 7f165be00640 1 -- 172.21.15.107:0/1251481449 shutdown_connections 2024-03-08T21:16:17.573 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:17.597+0000 7f165be00640 1 --2- 172.21.15.107:0/1251481449 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1654108250 0x7f1654114760 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:16:17.573 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:17.597+0000 7f165be00640 1 --2- 172.21.15.107:0/1251481449 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1654107890 0x7f1654107d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:16:17.573 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:17.597+0000 7f165be00640 1 --2- 172.21.15.107:0/1251481449 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f165410d8a0 0x7f165410dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:16:17.573 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:17.597+0000 7f165be00640 1 -- 172.21.15.107:0/1251481449 >> 172.21.15.107:0/1251481449 conn(0x7f1654075830 msgr2=0x7f1654077c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:16:17.573 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:17.597+0000 7f165be00640 1 -- 172.21.15.107:0/1251481449 shutdown_connections 2024-03-08T21:16:17.573 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:17.597+0000 7f165be00640 1 -- 172.21.15.107:0/1251481449 wait complete. 2024-03-08T21:16:17.576 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:17.598+0000 7f165be00640 1 Processor -- start 2024-03-08T21:16:17.576 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:17.598+0000 7f165be00640 1 -- start start 2024-03-08T21:16:17.576 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:17.598+0000 7f165be00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1654107890 0x7f165419e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:16:17.576 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:17.598+0000 7f165be00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1654108250 0x7f165419e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:16:17.576 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:17.598+0000 7f165be00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f165410d8a0 0x7f16541a2980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:16:17.576 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:17.598+0000 7f165be00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1654117b60 con 0x7f1654107890 2024-03-08T21:16:17.576 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:17.598+0000 7f165be00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16541179e0 con 0x7f1654108250 2024-03-08T21:16:17.576 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:17.598+0000 7f165be00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1654117ce0 con 0x7f165410d8a0 2024-03-08T21:16:17.576 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:17.599+0000 7f1659400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1654107890 0x7f165419e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:16:17.576 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:17.599+0000 7f1659400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1654107890 0x7f165419e0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46052/0 (socket says 172.21.15.107:46052) 2024-03-08T21:16:17.576 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:17.599+0000 7f1659400640 1 -- 172.21.15.107:0/4207178412 learned_addr learned my addr 172.21.15.107:0/4207178412 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:16:17.576 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:17.599+0000 7f1658a00640 1 --2- 172.21.15.107:0/4207178412 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1654108250 0x7f165419e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:16:17.577 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:17.599+0000 7f1659e00640 1 --2- 172.21.15.107:0/4207178412 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f165410d8a0 0x7f16541a2980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:16:17.577 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:17.599+0000 7f1659400640 1 -- 172.21.15.107:0/4207178412 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f165410d8a0 msgr2=0x7f16541a2980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:16:17.577 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:17.599+0000 7f1659400640 1 --2- 172.21.15.107:0/4207178412 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f165410d8a0 0x7f16541a2980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:16:17.577 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:17.599+0000 7f1659400640 1 -- 172.21.15.107:0/4207178412 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1654108250 msgr2=0x7f165419e600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:16:17.577 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:17.599+0000 7f1659400640 1 --2- 172.21.15.107:0/4207178412 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1654108250 0x7f165419e600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:16:17.577 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:17.599+0000 7f1659400640 1 -- 172.21.15.107:0/4207178412 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16541a30b0 con 0x7f1654107890 2024-03-08T21:16:17.577 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:17.599+0000 7f1658a00640 1 --2- 172.21.15.107:0/4207178412 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1654108250 0x7f165419e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T21:16:17.577 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:17.599+0000 7f1659400640 1 --2- 172.21.15.107:0/4207178412 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1654107890 0x7f165419e0c0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f164800d950 tx=0x7f164800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:16:17.577 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:17.600+0000 7f1652000640 1 -- 172.21.15.107:0/4207178412 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1648017030 con 0x7f1654107890 2024-03-08T21:16:17.577 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:17.600+0000 7f1652000640 1 -- 172.21.15.107:0/4207178412 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f1648004020 con 0x7f1654107890 2024-03-08T21:16:17.577 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:17.600+0000 7f1652000640 1 -- 172.21.15.107:0/4207178412 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16480047a0 con 0x7f1654107890 2024-03-08T21:16:17.577 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:17.600+0000 7f165be00640 1 -- 172.21.15.107:0/4207178412 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16541a33a0 con 0x7f1654107890 2024-03-08T21:16:17.577 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:17.600+0000 7f165be00640 1 -- 172.21.15.107:0/4207178412 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16541aac30 con 0x7f1654107890 2024-03-08T21:16:17.578 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:17.601+0000 7f165be00640 1 -- 172.21.15.107:0/4207178412 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1654068530 con 0x7f1654107890 2024-03-08T21:16:17.578 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:17.603+0000 7f1652000640 1 -- 172.21.15.107:0/4207178412 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) v1 ==== 137422+0+0 (secure 0 0 0) 0x7f16480041d0 con 0x7f1654107890 2024-03-08T21:16:17.580 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:17.604+0000 7f1652000640 1 --2- 172.21.15.107:0/4207178412 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f163c0a29f0 0x7f163c0a4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:16:17.580 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:17.604+0000 7f1652000640 1 -- 172.21.15.107:0/4207178412 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 8629+0+0 (secure 0 0 0) 0x7f16480ce990 con 0x7f1654107890 2024-03-08T21:16:17.580 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:17.604+0000 7f1658a00640 1 --2- 172.21.15.107:0/4207178412 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f163c0a29f0 0x7f163c0a4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:16:17.581 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:17.605+0000 7f1658a00640 1 --2- 172.21.15.107:0/4207178412 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f163c0a29f0 0x7f163c0a4eb0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f16400058c0 tx=0x7f1640005850 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:16:17.584 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:17.608+0000 7f1652000640 1 -- 172.21.15.107:0/4207178412 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f164809b620 con 0x7f1654107890 2024-03-08T21:16:17.758 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:16:17 smithi149 ceph-mon[35042]: from='client.? 172.21.15.107:0/240691090' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-08T21:16:17.758 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:16:17 smithi149 ceph-mon[35042]: from='client.? 172.21.15.107:0/4075050977' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-08T21:16:17.798 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:17.822+0000 7f165be00640 1 -- 172.21.15.107:0/4207178412 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f165410fe80 con 0x7f1654107890 2024-03-08T21:16:17.799 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:17.823+0000 7f1652000640 1 -- 172.21.15.107:0/4207178412 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 17 entries v114) v1 ==== 81+0+1015 (secure 0 0 0) 0x7f164809af20 con 0x7f1654107890 2024-03-08T21:16:17.800 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:0/808124991 2024-03-09T20:48:20.149406+0000 2024-03-08T21:16:17.800 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:6801/3376636618 2024-03-09T21:00:32.431358+0000 2024-03-08T21:16:17.800 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:0/2482452215 2024-03-09T20:47:45.706790+0000 2024-03-08T21:16:17.800 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:6801/3549936519 2024-03-09T20:47:45.706790+0000 2024-03-08T21:16:17.800 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:0/3746019488 2024-03-09T21:00:32.431358+0000 2024-03-08T21:16:17.800 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:0/3691962590 2024-03-09T21:00:32.431358+0000 2024-03-08T21:16:17.800 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:0/1206995985 2024-03-09T20:47:45.706790+0000 2024-03-08T21:16:17.800 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:6800/3549936519 2024-03-09T20:47:45.706790+0000 2024-03-08T21:16:17.800 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:0/2392547434 2024-03-09T20:47:45.706790+0000 2024-03-08T21:16:17.800 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:6800/426372446 2024-03-09T20:48:20.149406+0000 2024-03-08T21:16:17.800 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:6801/426372446 2024-03-09T20:48:20.149406+0000 2024-03-08T21:16:17.800 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:0/1886016707 2024-03-09T20:48:20.149406+0000 2024-03-08T21:16:17.801 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:6800/3376636618 2024-03-09T21:00:32.431358+0000 2024-03-08T21:16:17.801 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:0/1514666962 2024-03-09T20:48:20.149406+0000 2024-03-08T21:16:17.801 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:0/552952579 2024-03-09T21:00:32.431358+0000 2024-03-08T21:16:17.801 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:0/458107598 2024-03-09T21:00:32.431358+0000 2024-03-08T21:16:17.801 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:0/356063043 2024-03-09T21:00:32.431358+0000 2024-03-08T21:16:17.801 INFO:teuthology.orchestra.run.smithi107.stderr:listed 17 entries 2024-03-08T21:16:17.802 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:17.826+0000 7f165be00640 1 -- 172.21.15.107:0/4207178412 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f163c0a29f0 msgr2=0x7f163c0a4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:16:17.802 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:17.826+0000 7f165be00640 1 --2- 172.21.15.107:0/4207178412 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f163c0a29f0 0x7f163c0a4eb0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f16400058c0 tx=0x7f1640005850 comp rx=0 tx=0).stop 2024-03-08T21:16:17.802 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:17.826+0000 7f165be00640 1 -- 172.21.15.107:0/4207178412 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1654107890 msgr2=0x7f165419e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:16:17.802 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:17.826+0000 7f165be00640 1 --2- 172.21.15.107:0/4207178412 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1654107890 0x7f165419e0c0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f164800d950 tx=0x7f164800de10 comp rx=0 tx=0).stop 2024-03-08T21:16:17.803 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:17.826+0000 7f165be00640 1 -- 172.21.15.107:0/4207178412 shutdown_connections 2024-03-08T21:16:17.803 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:17.826+0000 7f165be00640 1 --2- 172.21.15.107:0/4207178412 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f163c0a29f0 0x7f163c0a4eb0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:16:17.803 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:17.826+0000 7f165be00640 1 --2- 172.21.15.107:0/4207178412 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f165410d8a0 0x7f16541a2980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:16:17.803 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:17.826+0000 7f165be00640 1 --2- 172.21.15.107:0/4207178412 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1654108250 0x7f165419e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:16:17.803 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:17.826+0000 7f165be00640 1 --2- 172.21.15.107:0/4207178412 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1654107890 0x7f165419e0c0 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:16:17.803 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:17.826+0000 7f165be00640 1 -- 172.21.15.107:0/4207178412 >> 172.21.15.107:0/4207178412 conn(0x7f1654075830 msgr2=0x7f1654076070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:16:17.803 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:17.826+0000 7f165be00640 1 -- 172.21.15.107:0/4207178412 shutdown_connections 2024-03-08T21:16:17.803 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:17.826+0000 7f165be00640 1 -- 172.21.15.107:0/4207178412 wait complete. 2024-03-08T21:16:17.822 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-03-08T21:16:17.823 DEBUG:teuthology.orchestra.run.smithi107:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-08T21:16:17.875 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-08T21:16:18.086 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:18.110+0000 7f4bd6400640 1 -- 172.21.15.107:0/3035652275 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4bd0111f70 msgr2=0x7f4bd0114360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:16:18.087 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:18.110+0000 7f4bd6400640 1 --2- 172.21.15.107:0/3035652275 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4bd0111f70 0x7f4bd0114360 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f4bc8009c10 tx=0x7f4bc802bee0 comp rx=0 tx=0).stop 2024-03-08T21:16:18.087 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:18.110+0000 7f4bd6400640 1 -- 172.21.15.107:0/3035652275 shutdown_connections 2024-03-08T21:16:18.087 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:18.110+0000 7f4bd6400640 1 --2- 172.21.15.107:0/3035652275 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4bd0111f70 0x7f4bd0114360 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:16:18.087 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:18.110+0000 7f4bd6400640 1 --2- 172.21.15.107:0/3035652275 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4bd0076140 0x7f4bd0111a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:16:18.087 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:18.110+0000 7f4bd6400640 1 --2- 172.21.15.107:0/3035652275 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4bd0075820 0x7f4bd0075c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:16:18.087 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:18.110+0000 7f4bd6400640 1 -- 172.21.15.107:0/3035652275 >> 172.21.15.107:0/3035652275 conn(0x7f4bd00fed80 msgr2=0x7f4bd01011a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:16:18.087 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:18.111+0000 7f4bd6400640 1 -- 172.21.15.107:0/3035652275 shutdown_connections 2024-03-08T21:16:18.087 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:18.111+0000 7f4bd6400640 1 -- 172.21.15.107:0/3035652275 wait complete. 2024-03-08T21:16:18.089 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:18.112+0000 7f4bd6400640 1 Processor -- start 2024-03-08T21:16:18.089 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:18.112+0000 7f4bd6400640 1 -- start start 2024-03-08T21:16:18.089 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:18.112+0000 7f4bd6400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4bd0075820 0x7f4bd019ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:16:18.090 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:18.112+0000 7f4bd6400640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4bd0076140 0x7f4bd019e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:16:18.090 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:18.112+0000 7f4bd6400640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4bd0111f70 0x7f4bd01a2790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:16:18.090 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:18.112+0000 7f4bd6400640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bd01179d0 con 0x7f4bd0075820 2024-03-08T21:16:18.090 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:18.112+0000 7f4bd6400640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bd0117850 con 0x7f4bd0076140 2024-03-08T21:16:18.090 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:18.112+0000 7f4bd6400640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bd0117b50 con 0x7f4bd0111f70 2024-03-08T21:16:18.090 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:18.113+0000 7f4bd5000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4bd0075820 0x7f4bd019ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:16:18.090 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:18.113+0000 7f4bd5000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4bd0075820 0x7f4bd019ded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46056/0 (socket says 172.21.15.107:46056) 2024-03-08T21:16:18.091 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:18.113+0000 7f4bd5000640 1 -- 172.21.15.107:0/2793960681 learned_addr learned my addr 172.21.15.107:0/2793960681 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:16:18.091 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:18.113+0000 7f4bcfe00640 1 --2- 172.21.15.107:0/2793960681 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4bd0076140 0x7f4bd019e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:16:18.091 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:18.113+0000 7f4bd5000640 1 -- 172.21.15.107:0/2793960681 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4bd0111f70 msgr2=0x7f4bd01a2790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:16:18.091 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:18.113+0000 7f4bd5000640 1 --2- 172.21.15.107:0/2793960681 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4bd0111f70 0x7f4bd01a2790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:16:18.091 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:18.113+0000 7f4bd5000640 1 -- 172.21.15.107:0/2793960681 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4bd0076140 msgr2=0x7f4bd019e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:16:18.091 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:18.113+0000 7f4bd5000640 1 --2- 172.21.15.107:0/2793960681 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4bd0076140 0x7f4bd019e410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:16:18.091 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:18.113+0000 7f4bd5000640 1 -- 172.21.15.107:0/2793960681 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bd01a2ec0 con 0x7f4bd0075820 2024-03-08T21:16:18.091 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:18.113+0000 7f4bd5000640 1 --2- 172.21.15.107:0/2793960681 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4bd0075820 0x7f4bd019ded0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f4bc400e9c0 tx=0x7f4bc400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:16:18.091 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:18.114+0000 7f4bcd600640 1 -- 172.21.15.107:0/2793960681 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bc400cde0 con 0x7f4bd0075820 2024-03-08T21:16:18.091 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:18.114+0000 7f4bd6400640 1 -- 172.21.15.107:0/2793960681 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4bd01a31b0 con 0x7f4bd0075820 2024-03-08T21:16:18.091 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:18.114+0000 7f4bcd600640 1 -- 172.21.15.107:0/2793960681 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f4bc40035f0 con 0x7f4bd0075820 2024-03-08T21:16:18.091 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:18.114+0000 7f4bcd600640 1 -- 172.21.15.107:0/2793960681 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bc4010690 con 0x7f4bd0075820 2024-03-08T21:16:18.092 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:18.114+0000 7f4bd6400640 1 -- 172.21.15.107:0/2793960681 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bd01aaa40 con 0x7f4bd0075820 2024-03-08T21:16:18.092 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:18.115+0000 7f4bd6400640 1 -- 172.21.15.107:0/2793960681 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4bd0068450 con 0x7f4bd0075820 2024-03-08T21:16:18.092 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:18.117+0000 7f4bcd600640 1 -- 172.21.15.107:0/2793960681 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) v1 ==== 137422+0+0 (secure 0 0 0) 0x7f4bc40037a0 con 0x7f4bd0075820 2024-03-08T21:16:18.094 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:18.118+0000 7f4bcd600640 1 --2- 172.21.15.107:0/2793960681 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f4bb80a2970 0x7f4bb80a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:16:18.094 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:18.118+0000 7f4bcd600640 1 -- 172.21.15.107:0/2793960681 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 8629+0+0 (secure 0 0 0) 0x7f4bc40cf3b0 con 0x7f4bd0075820 2024-03-08T21:16:18.094 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:18.119+0000 7f4bcfe00640 1 --2- 172.21.15.107:0/2793960681 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f4bb80a2970 0x7f4bb80a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:16:18.095 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:18.119+0000 7f4bcfe00640 1 --2- 172.21.15.107:0/2793960681 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f4bb80a2970 0x7f4bb80a4e30 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f4bd0101270 tx=0x7f4bbc005f90 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:16:18.098 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:18.123+0000 7f4bcd600640 1 -- 172.21.15.107:0/2793960681 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f4bc4009510 con 0x7f4bd0075820 2024-03-08T21:16:18.121 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:16:17 smithi204 ceph-mon[34935]: from='client.? 172.21.15.107:0/240691090' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-08T21:16:18.121 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:16:17 smithi204 ceph-mon[34935]: from='client.? 172.21.15.107:0/4075050977' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-08T21:16:18.210 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:16:17 smithi107 ceph-mon[31381]: from='client.? 172.21.15.107:0/240691090' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-08T21:16:18.210 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:16:17 smithi107 ceph-mon[31381]: from='client.? 172.21.15.107:0/4075050977' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-08T21:16:18.298 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:18.322+0000 7f4bd6400640 1 -- 172.21.15.107:0/2793960681 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4bd010cb70 con 0x7f4bd0075820 2024-03-08T21:16:18.301 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:18.326+0000 7f4bcd600640 1 -- 172.21.15.107:0/2793960681 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v114) v1 ==== 74+0+23385 (secure 0 0 0) 0x7f4bc409c040 con 0x7f4bd0075820 2024-03-08T21:16:18.302 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-03-08T21:16:18.302 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":114,"fsid":"dc20f0f8-dd8c-11ee-95c6-87774f69a715","created":"2024-03-08T20:47:04.190034+0000","modified":"2024-03-08T21:16:09.695540+0000","last_up_change":"2024-03-08T20:56:43.135514+0000","last_in_change":"2024-03-08T20:56:17.577074+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-08T20:51:32.832786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-08T20:57:48.721227+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-08T20:57:49.707779+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"114","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":27,"snap_epoch":114,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8b1ddc64-4d20-4e85-ac66-f1c5ac599501","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":2453946095},{"type":"v1","addr":"172.21.15.107:6803","nonce":2453946095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":2453946095},{"type":"v1","addr":"172.21.15.107:6805","nonce":2453946095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":2453946095},{"type":"v1","addr":"172.21.15.107:6809","nonce":2453946095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":2453946095},{"type":"v1","addr":"172.21.15.107:6807","nonce":2453946095}]},"public_addr":"172.21.15.107:6803/2453946095","cluster_addr":"172.21.15.107:6805/2453946095","heartbeat_back_addr":"172.21.15.107:6809/2453946095","heartbeat_front_addr":"172.21.15.107:6807/2453946095","state":["exists","up"]},{"osd":1,"uuid":"5a803c09-202a-4777-ba2a-1da8eaf26a54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":3071316419},{"type":"v1","addr":"172.21.15.149:6801","nonce":3071316419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":3071316419},{"type":"v1","addr":"172.21.15.149:6803","nonce":3071316419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":3071316419},{"type":"v1","addr":"172.21.15.149:6807","nonce":3071316419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":3071316419},{"type":"v1","addr":"172.21.15.149:6805","nonce":3071316419}]},"public_addr":"172.21.15.149:6801/3071316419","cluster_addr":"172.21.15.149:6803/3071316419","heartbeat_back_addr":"172.21.15.149:6807/3071316419","heartbeat_front_addr":"172.21.15.149:6805/3071316419","state":["exists","up"]},{"osd":2,"uuid":"555c634f-347e-47dd-a943-e50ae2be073c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6800","nonce":1087852630},{"type":"v1","addr":"172.21.15.204:6801","nonce":1087852630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6802","nonce":1087852630},{"type":"v1","addr":"172.21.15.204:6803","nonce":1087852630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6806","nonce":1087852630},{"type":"v1","addr":"172.21.15.204:6807","nonce":1087852630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6804","nonce":1087852630},{"type":"v1","addr":"172.21.15.204:6805","nonce":1087852630}]},"public_addr":"172.21.15.204:6801/1087852630","cluster_addr":"172.21.15.204:6803/1087852630","heartbeat_back_addr":"172.21.15.204:6807/1087852630","heartbeat_front_addr":"172.21.15.204:6805/1087852630","state":["exists","up"]},{"osd":3,"uuid":"9e433d97-e202-48e0-9ed1-f0988fa14307","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":3123579789},{"type":"v1","addr":"172.21.15.107:6811","nonce":3123579789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":3123579789},{"type":"v1","addr":"172.21.15.107:6813","nonce":3123579789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":3123579789},{"type":"v1","addr":"172.21.15.107:6817","nonce":3123579789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":3123579789},{"type":"v1","addr":"172.21.15.107:6815","nonce":3123579789}]},"public_addr":"172.21.15.107:6811/3123579789","cluster_addr":"172.21.15.107:6813/3123579789","heartbeat_back_addr":"172.21.15.107:6817/3123579789","heartbeat_front_addr":"172.21.15.107:6815/3123579789","state":["exists","up"]},{"osd":4,"uuid":"27c53794-9d2b-487d-bfcf-ac4fd2ab061e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":726286918},{"type":"v1","addr":"172.21.15.149:6809","nonce":726286918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":726286918},{"type":"v1","addr":"172.21.15.149:6811","nonce":726286918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":726286918},{"type":"v1","addr":"172.21.15.149:6815","nonce":726286918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":726286918},{"type":"v1","addr":"172.21.15.149:6813","nonce":726286918}]},"public_addr":"172.21.15.149:6809/726286918","cluster_addr":"172.21.15.149:6811/726286918","heartbeat_back_addr":"172.21.15.149:6815/726286918","heartbeat_front_addr":"172.21.15.149:6813/726286918","state":["exists","up"]},{"osd":5,"uuid":"3a7778fb-46c5-40c0-842e-e988c45ea24d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6808","nonce":3143887820},{"type":"v1","addr":"172.21.15.204:6809","nonce":3143887820}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6810","nonce":3143887820},{"type":"v1","addr":"172.21.15.204:6811","nonce":3143887820}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6814","nonce":3143887820},{"type":"v1","addr":"172.21.15.204:6815","nonce":3143887820}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6812","nonce":3143887820},{"type":"v1","addr":"172.21.15.204:6813","nonce":3143887820}]},"public_addr":"172.21.15.204:6809/3143887820","cluster_addr":"172.21.15.204:6811/3143887820","heartbeat_back_addr":"172.21.15.204:6815/3143887820","heartbeat_front_addr":"172.21.15.204:6813/3143887820","state":["exists","up"]},{"osd":6,"uuid":"d2e05a56-be4f-4435-9b76-ff9aa01e4ac3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":2912452950},{"type":"v1","addr":"172.21.15.107:6819","nonce":2912452950}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":2912452950},{"type":"v1","addr":"172.21.15.107:6821","nonce":2912452950}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":2912452950},{"type":"v1","addr":"172.21.15.107:6825","nonce":2912452950}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":2912452950},{"type":"v1","addr":"172.21.15.107:6823","nonce":2912452950}]},"public_addr":"172.21.15.107:6819/2912452950","cluster_addr":"172.21.15.107:6821/2912452950","heartbeat_back_addr":"172.21.15.107:6825/2912452950","heartbeat_front_addr":"172.21.15.107:6823/2912452950","state":["exists","up"]},{"osd":7,"uuid":"a7dd26ee-c118-4a9a-9e19-142c2680902d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":3835863809},{"type":"v1","addr":"172.21.15.149:6817","nonce":3835863809}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":3835863809},{"type":"v1","addr":"172.21.15.149:6819","nonce":3835863809}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":3835863809},{"type":"v1","addr":"172.21.15.149:6823","nonce":3835863809}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":3835863809},{"type":"v1","addr":"172.21.15.149:6821","nonce":3835863809}]},"public_addr":"172.21.15.149:6817/3835863809","cluster_addr":"172.21.15.149:6819/3835863809","heartbeat_back_addr":"172.21.15.149:6823/3835863809","heartbeat_front_addr":"172.21.15.149:6821/3835863809","state":["exists","up"]},{"osd":8,"uuid":"baaaa8dd-0e1f-4bae-86bf-9e429d985f23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6816","nonce":1010601477},{"type":"v1","addr":"172.21.15.204:6817","nonce":1010601477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6818","nonce":1010601477},{"type":"v1","addr":"172.21.15.204:6819","nonce":1010601477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6822","nonce":1010601477},{"type":"v1","addr":"172.21.15.204:6823","nonce":1010601477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6820","nonce":1010601477},{"type":"v1","addr":"172.21.15.204:6821","nonce":1010601477}]},"public_addr":"172.21.15.204:6817/1010601477","cluster_addr":"172.21.15.204:6819/1010601477","heartbeat_back_addr":"172.21.15.204:6823/1010601477","heartbeat_front_addr":"172.21.15.204:6821/1010601477","state":["exists","up"]},{"osd":9,"uuid":"8df80dfa-e7ff-410e-824c-b1dce111b02f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":3334759819},{"type":"v1","addr":"172.21.15.107:6827","nonce":3334759819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3334759819},{"type":"v1","addr":"172.21.15.107:6829","nonce":3334759819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":3334759819},{"type":"v1","addr":"172.21.15.107:6833","nonce":3334759819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":3334759819},{"type":"v1","addr":"172.21.15.107:6831","nonce":3334759819}]},"public_addr":"172.21.15.107:6827/3334759819","cluster_addr":"172.21.15.107:6829/3334759819","heartbeat_back_addr":"172.21.15.107:6833/3334759819","heartbeat_front_addr":"172.21.15.107:6831/3334759819","state":["exists","up"]},{"osd":10,"uuid":"338945b2-20c0-40b2-aaca-062cd4adee04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":3469284631},{"type":"v1","addr":"172.21.15.149:6825","nonce":3469284631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":3469284631},{"type":"v1","addr":"172.21.15.149:6827","nonce":3469284631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":3469284631},{"type":"v1","addr":"172.21.15.149:6831","nonce":3469284631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":3469284631},{"type":"v1","addr":"172.21.15.149:6829","nonce":3469284631}]},"public_addr":"172.21.15.149:6825/3469284631","cluster_addr":"172.21.15.149:6827/3469284631","heartbeat_back_addr":"172.21.15.149:6831/3469284631","heartbeat_front_addr":"172.21.15.149:6829/3469284631","state":["exists","up"]},{"osd":11,"uuid":"acccfe4f-d307-4a64-8f4d-fba27bfbe09d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6824","nonce":3097772473},{"type":"v1","addr":"172.21.15.204:6825","nonce":3097772473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6826","nonce":3097772473},{"type":"v1","addr":"172.21.15.204:6827","nonce":3097772473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6830","nonce":3097772473},{"type":"v1","addr":"172.21.15.204:6831","nonce":3097772473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.204:6828","nonce":3097772473},{"type":"v1","addr":"172.21.15.204:6829","nonce":3097772473}]},"public_addr":"172.21.15.204:6825/3097772473","cluster_addr":"172.21.15.204:6827/3097772473","heartbeat_back_addr":"172.21.15.204:6831/3097772473","heartbeat_front_addr":"172.21.15.204:6829/3097772473","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T20:50:25.862753+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T20:50:58.167663+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T20:51:30.312816+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T20:52:05.487917+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T20:52:39.348354+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T20:53:12.899002+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T20:53:47.630837+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T20:54:21.444552+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T20:54:57.101160+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T20:55:31.841786+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T20:56:05.827161+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-08T20:56:40.611097+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.0","mappings":[{"from":4,"to":6}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.3b","mappings":[{"from":3,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.107:0/808124991":"2024-03-09T20:48:20.149406+0000","172.21.15.107:6801/3376636618":"2024-03-09T21:00:32.431358+0000","172.21.15.107:0/2482452215":"2024-03-09T20:47:45.706790+0000","172.21.15.107:6801/3549936519":"2024-03-09T20:47:45.706790+0000","172.21.15.107:0/3746019488":"2024-03-09T21:00:32.431358+0000","172.21.15.107:0/3691962590":"2024-03-09T21:00:32.431358+0000","172.21.15.107:0/1206995985":"2024-03-09T20:47:45.706790+0000","172.21.15.107:6800/3549936519":"2024-03-09T20:47:45.706790+0000","172.21.15.107:0/2392547434":"2024-03-09T20:47:45.706790+0000","172.21.15.107:6800/426372446":"2024-03-09T20:48:20.149406+0000","172.21.15.107:6801/426372446":"2024-03-09T20:48:20.149406+0000","172.21.15.107:0/1886016707":"2024-03-09T20:48:20.149406+0000","172.21.15.107:6800/3376636618":"2024-03-09T21:00:32.431358+0000","172.21.15.107:0/1514666962":"2024-03-09T20:48:20.149406+0000","172.21.15.107:0/552952579":"2024-03-09T21:00:32.431358+0000","172.21.15.107:0/458107598":"2024-03-09T21:00:32.431358+0000","172.21.15.107:0/356063043":"2024-03-09T21:00:32.431358+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":3,"snaps":[{"begin":14,"length":1},{"begin":27,"length":1}]}],"new_removed_snaps":[{"pool":3,"snaps":[{"begin":14,"length":1},{"begin":27,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-08T21:16:18.305 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:18.329+0000 7f4bd6400640 1 -- 172.21.15.107:0/2793960681 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f4bb80a2970 msgr2=0x7f4bb80a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:16:18.305 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:18.329+0000 7f4bd6400640 1 --2- 172.21.15.107:0/2793960681 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f4bb80a2970 0x7f4bb80a4e30 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f4bd0101270 tx=0x7f4bbc005f90 comp rx=0 tx=0).stop 2024-03-08T21:16:18.305 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:18.329+0000 7f4bd6400640 1 -- 172.21.15.107:0/2793960681 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4bd0075820 msgr2=0x7f4bd019ded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:16:18.305 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:18.329+0000 7f4bd6400640 1 --2- 172.21.15.107:0/2793960681 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4bd0075820 0x7f4bd019ded0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f4bc400e9c0 tx=0x7f4bc400ee80 comp rx=0 tx=0).stop 2024-03-08T21:16:18.305 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:18.329+0000 7f4bd6400640 1 -- 172.21.15.107:0/2793960681 shutdown_connections 2024-03-08T21:16:18.306 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:18.329+0000 7f4bd6400640 1 --2- 172.21.15.107:0/2793960681 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f4bb80a2970 0x7f4bb80a4e30 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:16:18.306 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:18.329+0000 7f4bd6400640 1 --2- 172.21.15.107:0/2793960681 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f4bd0111f70 0x7f4bd01a2790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:16:18.306 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:18.329+0000 7f4bd6400640 1 --2- 172.21.15.107:0/2793960681 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4bd0076140 0x7f4bd019e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:16:18.306 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:18.329+0000 7f4bd6400640 1 --2- 172.21.15.107:0/2793960681 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4bd0075820 0x7f4bd019ded0 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:16:18.306 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:18.329+0000 7f4bd6400640 1 -- 172.21.15.107:0/2793960681 >> 172.21.15.107:0/2793960681 conn(0x7f4bd00fed80 msgr2=0x7f4bd0113690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:16:18.306 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:18.329+0000 7f4bd6400640 1 -- 172.21.15.107:0/2793960681 shutdown_connections 2024-03-08T21:16:18.306 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:18.329+0000 7f4bd6400640 1 -- 172.21.15.107:0/2793960681 wait complete. 2024-03-08T21:16:18.325 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-08T20:51:32.832786+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-08T20:57:48.721227+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-08T20:57:49.707779+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '114', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 27, 'snap_epoch': 114, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-08T21:16:18.326 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-08T21:16:18.541 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:18.565+0000 7fa123200640 1 -- 172.21.15.107:0/3508401447 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa11c1009d0 msgr2=0x7fa11c10d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:16:18.541 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:18.565+0000 7fa123200640 1 --2- 172.21.15.107:0/3508401447 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa11c1009d0 0x7fa11c10d720 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7fa104009900 tx=0x7fa10402bc30 comp rx=0 tx=0).stop 2024-03-08T21:16:18.541 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:18.565+0000 7fa123200640 1 -- 172.21.15.107:0/3508401447 shutdown_connections 2024-03-08T21:16:18.541 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:18.565+0000 7fa123200640 1 --2- 172.21.15.107:0/3508401447 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa11c10dc60 0x7fa11c110050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:16:18.542 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:18.565+0000 7fa123200640 1 --2- 172.21.15.107:0/3508401447 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa11c1009d0 0x7fa11c10d720 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:16:18.542 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:18.565+0000 7fa123200640 1 --2- 172.21.15.107:0/3508401447 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa11c1000b0 0x7fa11c100490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:16:18.542 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:18.565+0000 7fa123200640 1 -- 172.21.15.107:0/3508401447 >> 172.21.15.107:0/3508401447 conn(0x7fa11c076570 msgr2=0x7fa11c076980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:16:18.542 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:18.565+0000 7fa123200640 1 -- 172.21.15.107:0/3508401447 shutdown_connections 2024-03-08T21:16:18.542 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:18.565+0000 7fa123200640 1 -- 172.21.15.107:0/3508401447 wait complete. 2024-03-08T21:16:18.544 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:18.566+0000 7fa123200640 1 Processor -- start 2024-03-08T21:16:18.544 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:18.566+0000 7fa123200640 1 -- start start 2024-03-08T21:16:18.544 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:18.567+0000 7fa123200640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa11c1000b0 0x7fa11c199dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:16:18.544 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:18.567+0000 7fa123200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa11c1009d0 0x7fa11c19a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:16:18.545 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:18.567+0000 7fa123200640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa11c10dc60 0x7fa11c19e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:16:18.545 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:18.567+0000 7fa123200640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa11c113760 con 0x7fa11c1009d0 2024-03-08T21:16:18.545 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:18.567+0000 7fa123200640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa11c1135e0 con 0x7fa11c10dc60 2024-03-08T21:16:18.545 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:18.567+0000 7fa123200640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa11c1138e0 con 0x7fa11c1000b0 2024-03-08T21:16:18.545 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:18.567+0000 7fa11b400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa11c1009d0 0x7fa11c19a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:16:18.545 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:18.567+0000 7fa11b400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa11c1009d0 0x7fa11c19a310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46066/0 (socket says 172.21.15.107:46066) 2024-03-08T21:16:18.545 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:18.567+0000 7fa11b400640 1 -- 172.21.15.107:0/3136869170 learned_addr learned my addr 172.21.15.107:0/3136869170 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:16:18.545 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:18.567+0000 7fa11be00640 1 --2- 172.21.15.107:0/3136869170 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa11c1000b0 0x7fa11c199dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:16:18.545 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:18.567+0000 7fa121200640 1 --2- 172.21.15.107:0/3136869170 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa11c10dc60 0x7fa11c19e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:16:18.545 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:18.568+0000 7fa11b400640 1 -- 172.21.15.107:0/3136869170 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa11c1000b0 msgr2=0x7fa11c199dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:16:18.545 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:18.568+0000 7fa11b400640 1 --2- 172.21.15.107:0/3136869170 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa11c1000b0 0x7fa11c199dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:16:18.545 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:18.568+0000 7fa11b400640 1 -- 172.21.15.107:0/3136869170 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa11c10dc60 msgr2=0x7fa11c19e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:16:18.545 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:18.568+0000 7fa11b400640 1 --2- 172.21.15.107:0/3136869170 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa11c10dc60 0x7fa11c19e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:16:18.546 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:18.568+0000 7fa11b400640 1 -- 172.21.15.107:0/3136869170 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa11c19edc0 con 0x7fa11c1009d0 2024-03-08T21:16:18.546 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:18.568+0000 7fa121200640 1 --2- 172.21.15.107:0/3136869170 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa11c10dc60 0x7fa11c19e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-08T21:16:18.546 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:18.568+0000 7fa11b400640 1 --2- 172.21.15.107:0/3136869170 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa11c1009d0 0x7fa11c19a310 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7fa104005940 tx=0x7fa10402c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:16:18.546 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:18.568+0000 7fa118c00640 1 -- 172.21.15.107:0/3136869170 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa104043020 con 0x7fa11c1009d0 2024-03-08T21:16:18.546 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:18.568+0000 7fa118c00640 1 -- 172.21.15.107:0/3136869170 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fa10402cbe0 con 0x7fa11c1009d0 2024-03-08T21:16:18.546 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:18.568+0000 7fa123200640 1 -- 172.21.15.107:0/3136869170 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa11c19f050 con 0x7fa11c1009d0 2024-03-08T21:16:18.546 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:18.569+0000 7fa118c00640 1 -- 172.21.15.107:0/3136869170 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa104003bf0 con 0x7fa11c1009d0 2024-03-08T21:16:18.546 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:18.569+0000 7fa123200640 1 -- 172.21.15.107:0/3136869170 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa11c1a6940 con 0x7fa11c1009d0 2024-03-08T21:16:18.547 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:18.570+0000 7fa123200640 1 -- 172.21.15.107:0/3136869170 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa11c06a0e0 con 0x7fa11c1009d0 2024-03-08T21:16:18.547 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:18.572+0000 7fa118c00640 1 -- 172.21.15.107:0/3136869170 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) v1 ==== 137422+0+0 (secure 0 0 0) 0x7fa104046020 con 0x7fa11c1009d0 2024-03-08T21:16:18.549 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:18.573+0000 7fa118c00640 1 --2- 172.21.15.107:0/3136869170 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7fa1000a2970 0x7fa1000a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:16:18.549 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:18.573+0000 7fa118c00640 1 -- 172.21.15.107:0/3136869170 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 8629+0+0 (secure 0 0 0) 0x7fa1040f10e0 con 0x7fa11c1009d0 2024-03-08T21:16:18.549 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:18.573+0000 7fa11be00640 1 --2- 172.21.15.107:0/3136869170 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7fa1000a2970 0x7fa1000a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:16:18.549 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:18.574+0000 7fa11be00640 1 --2- 172.21.15.107:0/3136869170 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7fa1000a2970 0x7fa1000a4e30 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fa10c00bbe0 tx=0x7fa10c00b390 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:16:18.553 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:18.577+0000 7fa118c00640 1 -- 172.21.15.107:0/3136869170 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7fa10404d020 con 0x7fa11c1009d0 2024-03-08T21:16:18.751 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:18.776+0000 7fa123200640 1 -- 172.21.15.107:0/3136869170 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa11c108df0 con 0x7fa11c1009d0 2024-03-08T21:16:18.752 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:18.777+0000 7fa118c00640 1 -- 172.21.15.107:0/3136869170 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v114) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa1040bde80 con 0x7fa11c1009d0 2024-03-08T21:16:18.752 INFO:teuthology.orchestra.run.smithi107.stdout:pg_num: 1 2024-03-08T21:16:18.755 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:18.779+0000 7fa123200640 1 -- 172.21.15.107:0/3136869170 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7fa1000a2970 msgr2=0x7fa1000a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:16:18.755 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:18.779+0000 7fa123200640 1 --2- 172.21.15.107:0/3136869170 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7fa1000a2970 0x7fa1000a4e30 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fa10c00bbe0 tx=0x7fa10c00b390 comp rx=0 tx=0).stop 2024-03-08T21:16:18.755 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:18.779+0000 7fa123200640 1 -- 172.21.15.107:0/3136869170 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa11c1009d0 msgr2=0x7fa11c19a310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:16:18.755 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:18.779+0000 7fa123200640 1 --2- 172.21.15.107:0/3136869170 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa11c1009d0 0x7fa11c19a310 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7fa104005940 tx=0x7fa10402c7e0 comp rx=0 tx=0).stop 2024-03-08T21:16:18.755 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:18.779+0000 7fa123200640 1 -- 172.21.15.107:0/3136869170 shutdown_connections 2024-03-08T21:16:18.755 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:18.779+0000 7fa123200640 1 --2- 172.21.15.107:0/3136869170 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7fa1000a2970 0x7fa1000a4e30 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:16:18.756 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:18.779+0000 7fa123200640 1 --2- 172.21.15.107:0/3136869170 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa11c10dc60 0x7fa11c19e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:16:18.756 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:18.779+0000 7fa123200640 1 --2- 172.21.15.107:0/3136869170 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa11c1009d0 0x7fa11c19a310 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:16:18.756 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:18.779+0000 7fa123200640 1 --2- 172.21.15.107:0/3136869170 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fa11c1000b0 0x7fa11c199dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:16:18.756 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:18.779+0000 7fa123200640 1 -- 172.21.15.107:0/3136869170 >> 172.21.15.107:0/3136869170 conn(0x7fa11c076570 msgr2=0x7fa11c10f370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:16:18.756 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:18.779+0000 7fa123200640 1 -- 172.21.15.107:0/3136869170 shutdown_connections 2024-03-08T21:16:18.756 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:18.779+0000 7fa123200640 1 -- 172.21.15.107:0/3136869170 wait complete. 2024-03-08T21:16:18.774 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-08T21:16:18.960 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:16:18 smithi107 ceph-mon[31381]: pgmap v500: 97 pgs: 97 active+clean; 17 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 307 B/s wr, 10 op/s 2024-03-08T21:16:18.960 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:16:18 smithi107 ceph-mon[31381]: from='client.? 172.21.15.107:0/4207178412' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-08T21:16:18.960 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:16:18 smithi107 ceph-mon[31381]: from='client.? 172.21.15.107:0/2793960681' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-08T21:16:18.989 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:19.013+0000 7f1f71a00640 1 -- 172.21.15.107:0/1152538928 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1f6c103e70 msgr2=0x7f1f6c110330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:16:18.989 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:19.013+0000 7f1f71a00640 1 --2- 172.21.15.107:0/1152538928 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1f6c103e70 0x7f1f6c110330 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f1f64009ca0 tx=0x7f1f6402c0f0 comp rx=0 tx=0).stop 2024-03-08T21:16:18.989 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:19.013+0000 7f1f71a00640 1 -- 172.21.15.107:0/1152538928 shutdown_connections 2024-03-08T21:16:18.989 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:19.013+0000 7f1f71a00640 1 --2- 172.21.15.107:0/1152538928 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1f6c103e70 0x7f1f6c110330 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:16:18.989 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:19.013+0000 7f1f71a00640 1 --2- 172.21.15.107:0/1152538928 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1f6c1034b0 0x7f1f6c103930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:16:18.989 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:19.013+0000 7f1f71a00640 1 --2- 172.21.15.107:0/1152538928 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1f6c109570 0x7f1f6c109950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:16:18.989 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:19.013+0000 7f1f71a00640 1 -- 172.21.15.107:0/1152538928 >> 172.21.15.107:0/1152538928 conn(0x7f1f6c0fef60 msgr2=0x7f1f6c101380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:16:18.989 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:19.013+0000 7f1f71a00640 1 -- 172.21.15.107:0/1152538928 shutdown_connections 2024-03-08T21:16:18.989 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:19.013+0000 7f1f71a00640 1 -- 172.21.15.107:0/1152538928 wait complete. 2024-03-08T21:16:18.991 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:19.014+0000 7f1f71a00640 1 Processor -- start 2024-03-08T21:16:18.991 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:19.014+0000 7f1f71a00640 1 -- start start 2024-03-08T21:16:18.991 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:19.014+0000 7f1f71a00640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1f6c1034b0 0x7f1f6c19f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:16:18.991 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:19.014+0000 7f1f71a00640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1f6c103e70 0x7f1f6c19fd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:16:18.991 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:19.014+0000 7f1f71a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1f6c109570 0x7f1f6c1999e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:16:18.991 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:19.014+0000 7f1f71a00640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f6c113700 con 0x7f1f6c109570 2024-03-08T21:16:18.992 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:19.014+0000 7f1f71a00640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f6c113580 con 0x7f1f6c1034b0 2024-03-08T21:16:18.992 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:19.014+0000 7f1f71a00640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f6c113880 con 0x7f1f6c103e70 2024-03-08T21:16:18.992 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:19.015+0000 7f1f6b400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1f6c109570 0x7f1f6c1999e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:16:18.992 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:19.015+0000 7f1f6b400640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1f6c109570 0x7f1f6c1999e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46086/0 (socket says 172.21.15.107:46086) 2024-03-08T21:16:18.992 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:19.015+0000 7f1f6b400640 1 -- 172.21.15.107:0/3269645986 learned_addr learned my addr 172.21.15.107:0/3269645986 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:16:18.992 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:19.015+0000 7f1f6aa00640 1 --2- 172.21.15.107:0/3269645986 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1f6c1034b0 0x7f1f6c19f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:16:18.992 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:19.015+0000 7f1f6b400640 1 -- 172.21.15.107:0/3269645986 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1f6c103e70 msgr2=0x7f1f6c19fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:16:18.992 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:19.015+0000 7f1f6b400640 1 --2- 172.21.15.107:0/3269645986 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1f6c103e70 0x7f1f6c19fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:16:18.992 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:19.015+0000 7f1f6b400640 1 -- 172.21.15.107:0/3269645986 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1f6c1034b0 msgr2=0x7f1f6c19f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:16:18.992 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:19.015+0000 7f1f6b400640 1 --2- 172.21.15.107:0/3269645986 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1f6c1034b0 0x7f1f6c19f830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:16:18.992 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:19.015+0000 7f1f6b400640 1 -- 172.21.15.107:0/3269645986 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f6c19a160 con 0x7f1f6c109570 2024-03-08T21:16:18.992 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:19.015+0000 7f1f6b400640 1 --2- 172.21.15.107:0/3269645986 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1f6c109570 0x7f1f6c1999e0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f1f6402c700 tx=0x7f1f64035d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:16:18.992 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:19.015+0000 7f1f63400640 1 -- 172.21.15.107:0/3269645986 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f64039040 con 0x7f1f6c109570 2024-03-08T21:16:18.992 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:19.015+0000 7f1f63400640 1 -- 172.21.15.107:0/3269645986 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f1f64044070 con 0x7f1f6c109570 2024-03-08T21:16:18.992 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:19.015+0000 7f1f63400640 1 -- 172.21.15.107:0/3269645986 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f64004590 con 0x7f1f6c109570 2024-03-08T21:16:18.993 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:19.016+0000 7f1f71a00640 1 -- 172.21.15.107:0/3269645986 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f6c19a3f0 con 0x7f1f6c109570 2024-03-08T21:16:18.993 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:19.016+0000 7f1f71a00640 1 -- 172.21.15.107:0/3269645986 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f6c1a66f0 con 0x7f1f6c109570 2024-03-08T21:16:18.993 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:19.017+0000 7f1f71a00640 1 -- 172.21.15.107:0/3269645986 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f6c068530 con 0x7f1f6c109570 2024-03-08T21:16:18.994 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:19.018+0000 7f1f63400640 1 -- 172.21.15.107:0/3269645986 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) v1 ==== 137422+0+0 (secure 0 0 0) 0x7f1f64032840 con 0x7f1f6c109570 2024-03-08T21:16:18.995 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:19.020+0000 7f1f63400640 1 --2- 172.21.15.107:0/3269645986 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f1f500a2970 0x7f1f500a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:16:18.995 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:19.020+0000 7f1f63400640 1 -- 172.21.15.107:0/3269645986 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 8629+0+0 (secure 0 0 0) 0x7f1f640f16b0 con 0x7f1f6c109570 2024-03-08T21:16:18.996 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:19.020+0000 7f1f6aa00640 1 --2- 172.21.15.107:0/3269645986 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f1f500a2970 0x7f1f500a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:16:18.996 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:19.021+0000 7f1f6aa00640 1 --2- 172.21.15.107:0/3269645986 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f1f500a2970 0x7f1f500a4e30 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f1f5c0058c0 tx=0x7f1f5c005c80 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:16:19.000 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:19.025+0000 7f1f63400640 1 -- 172.21.15.107:0/3269645986 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f1f640be4b0 con 0x7f1f6c109570 2024-03-08T21:16:19.008 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:16:18 smithi149 ceph-mon[35042]: pgmap v500: 97 pgs: 97 active+clean; 17 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 307 B/s wr, 10 op/s 2024-03-08T21:16:19.008 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:16:18 smithi149 ceph-mon[35042]: from='client.? 172.21.15.107:0/4207178412' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-08T21:16:19.008 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:16:18 smithi149 ceph-mon[35042]: from='client.? 172.21.15.107:0/2793960681' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-08T21:16:19.121 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:16:18 smithi204 ceph-mon[34935]: pgmap v500: 97 pgs: 97 active+clean; 17 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 307 B/s wr, 10 op/s 2024-03-08T21:16:19.121 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:16:18 smithi204 ceph-mon[34935]: from='client.? 172.21.15.107:0/4207178412' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-08T21:16:19.121 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:16:18 smithi204 ceph-mon[34935]: from='client.? 172.21.15.107:0/2793960681' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-08T21:16:19.193 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:19.217+0000 7f1f71a00640 1 -- 172.21.15.107:0/3269645986 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f1f6c02ce70 con 0x7f1f6c109570 2024-03-08T21:16:19.193 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:19.218+0000 7f1f63400640 1 -- 172.21.15.107:0/3269645986 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v114) v1 ==== 104+0+11 (secure 0 0 0) 0x7f1f640bddb0 con 0x7f1f6c109570 2024-03-08T21:16:19.194 INFO:teuthology.orchestra.run.smithi107.stdout:pg_num: 32 2024-03-08T21:16:19.196 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:19.220+0000 7f1f71a00640 1 -- 172.21.15.107:0/3269645986 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f1f500a2970 msgr2=0x7f1f500a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:16:19.196 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:19.220+0000 7f1f71a00640 1 --2- 172.21.15.107:0/3269645986 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f1f500a2970 0x7f1f500a4e30 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f1f5c0058c0 tx=0x7f1f5c005c80 comp rx=0 tx=0).stop 2024-03-08T21:16:19.197 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:19.220+0000 7f1f71a00640 1 -- 172.21.15.107:0/3269645986 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1f6c109570 msgr2=0x7f1f6c1999e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:16:19.197 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:19.220+0000 7f1f71a00640 1 --2- 172.21.15.107:0/3269645986 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1f6c109570 0x7f1f6c1999e0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f1f6402c700 tx=0x7f1f64035d60 comp rx=0 tx=0).stop 2024-03-08T21:16:19.197 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:19.220+0000 7f1f71a00640 1 -- 172.21.15.107:0/3269645986 shutdown_connections 2024-03-08T21:16:19.197 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:19.220+0000 7f1f71a00640 1 --2- 172.21.15.107:0/3269645986 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f1f500a2970 0x7f1f500a4e30 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:16:19.197 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:19.220+0000 7f1f71a00640 1 --2- 172.21.15.107:0/3269645986 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1f6c109570 0x7f1f6c1999e0 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:16:19.197 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:19.220+0000 7f1f71a00640 1 --2- 172.21.15.107:0/3269645986 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f1f6c103e70 0x7f1f6c19fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:16:19.197 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:19.220+0000 7f1f71a00640 1 --2- 172.21.15.107:0/3269645986 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1f6c1034b0 0x7f1f6c19f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:16:19.197 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:19.220+0000 7f1f71a00640 1 -- 172.21.15.107:0/3269645986 >> 172.21.15.107:0/3269645986 conn(0x7f1f6c0fef60 msgr2=0x7f1f6c100870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:16:19.197 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:19.221+0000 7f1f71a00640 1 -- 172.21.15.107:0/3269645986 shutdown_connections 2024-03-08T21:16:19.197 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:19.221+0000 7f1f71a00640 1 -- 172.21.15.107:0/3269645986 wait complete. 2024-03-08T21:16:19.215 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-08T21:16:19.414 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:19.438+0000 7f5833600640 1 -- 172.21.15.107:0/232056557 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f582c1034b0 msgr2=0x7f582c103930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:16:19.415 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:19.438+0000 7f5833600640 1 --2- 172.21.15.107:0/232056557 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f582c1034b0 0x7f582c103930 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f5818009900 tx=0x7f581802bc30 comp rx=0 tx=0).stop 2024-03-08T21:16:19.415 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:19.438+0000 7f5833600640 1 -- 172.21.15.107:0/232056557 shutdown_connections 2024-03-08T21:16:19.415 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:19.438+0000 7f5833600640 1 --2- 172.21.15.107:0/232056557 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f582c103e70 0x7f582c110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:16:19.415 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:19.438+0000 7f5833600640 1 --2- 172.21.15.107:0/232056557 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f582c1034b0 0x7f582c103930 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:16:19.415 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:19.438+0000 7f5833600640 1 --2- 172.21.15.107:0/232056557 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f582c109570 0x7f582c109950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:16:19.415 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:19.438+0000 7f5833600640 1 -- 172.21.15.107:0/232056557 >> 172.21.15.107:0/232056557 conn(0x7f582c0fefa0 msgr2=0x7f582c1013c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:16:19.415 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:19.438+0000 7f5833600640 1 -- 172.21.15.107:0/232056557 shutdown_connections 2024-03-08T21:16:19.415 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:19.438+0000 7f5833600640 1 -- 172.21.15.107:0/232056557 wait complete. 2024-03-08T21:16:19.417 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:19.439+0000 7f5833600640 1 Processor -- start 2024-03-08T21:16:19.417 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:19.439+0000 7f5833600640 1 -- start start 2024-03-08T21:16:19.418 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:19.440+0000 7f5833600640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f582c1034b0 0x7f582c19f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:16:19.418 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:19.440+0000 7f5833600640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f582c103e70 0x7f582c19fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:16:19.418 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:19.440+0000 7f5833600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f582c109570 0x7f582c1999d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:16:19.418 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:19.440+0000 7f5833600640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f582c113730 con 0x7f582c109570 2024-03-08T21:16:19.418 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:19.440+0000 7f5833600640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f582c1135b0 con 0x7f582c1034b0 2024-03-08T21:16:19.418 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:19.440+0000 7f5833600640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f582c1138b0 con 0x7f582c103e70 2024-03-08T21:16:19.418 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:19.440+0000 7f5831600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f582c109570 0x7f582c1999d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:16:19.418 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:19.440+0000 7f5831600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f582c109570 0x7f582c1999d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46102/0 (socket says 172.21.15.107:46102) 2024-03-08T21:16:19.418 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:19.440+0000 7f5831600640 1 -- 172.21.15.107:0/284398546 learned_addr learned my addr 172.21.15.107:0/284398546 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:16:19.418 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:19.440+0000 7f582be00640 1 --2- 172.21.15.107:0/284398546 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f582c103e70 0x7f582c19fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:16:19.419 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:19.440+0000 7f5830c00640 1 --2- 172.21.15.107:0/284398546 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f582c1034b0 0x7f582c19f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:16:19.419 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:19.440+0000 7f5831600640 1 -- 172.21.15.107:0/284398546 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f582c103e70 msgr2=0x7f582c19fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:16:19.419 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:19.440+0000 7f5831600640 1 --2- 172.21.15.107:0/284398546 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f582c103e70 0x7f582c19fd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:16:19.419 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:19.440+0000 7f5831600640 1 -- 172.21.15.107:0/284398546 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f582c1034b0 msgr2=0x7f582c19f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:16:19.419 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:19.440+0000 7f5831600640 1 --2- 172.21.15.107:0/284398546 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f582c1034b0 0x7f582c19f820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:16:19.419 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:19.440+0000 7f5831600640 1 -- 172.21.15.107:0/284398546 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f582c19a150 con 0x7f582c109570 2024-03-08T21:16:19.419 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:19.440+0000 7f582be00640 1 --2- 172.21.15.107:0/284398546 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f582c103e70 0x7f582c19fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T21:16:19.419 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:19.440+0000 7f5830c00640 1 --2- 172.21.15.107:0/284398546 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f582c1034b0 0x7f582c19f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T21:16:19.419 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:19.441+0000 7f5831600640 1 --2- 172.21.15.107:0/284398546 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f582c109570 0x7f582c1999d0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f582400c970 tx=0x7f582400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:16:19.419 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:19.441+0000 7f5829600640 1 -- 172.21.15.107:0/284398546 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5824007bb0 con 0x7f582c109570 2024-03-08T21:16:19.420 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:19.441+0000 7f5829600640 1 -- 172.21.15.107:0/284398546 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7f5824007d50 con 0x7f582c109570 2024-03-08T21:16:19.420 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:19.441+0000 7f5833600640 1 -- 172.21.15.107:0/284398546 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f582c19a440 con 0x7f582c109570 2024-03-08T21:16:19.420 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:19.441+0000 7f5829600640 1 -- 172.21.15.107:0/284398546 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5824004e10 con 0x7f582c109570 2024-03-08T21:16:19.420 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:19.441+0000 7f5833600640 1 -- 172.21.15.107:0/284398546 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f582c1a66e0 con 0x7f582c109570 2024-03-08T21:16:19.420 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:19.442+0000 7f5833600640 1 -- 172.21.15.107:0/284398546 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f582c068530 con 0x7f582c109570 2024-03-08T21:16:19.420 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:19.444+0000 7f5829600640 1 -- 172.21.15.107:0/284398546 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) v1 ==== 137422+0+0 (secure 0 0 0) 0x7f5824004ff0 con 0x7f582c109570 2024-03-08T21:16:19.421 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:19.445+0000 7f5829600640 1 --2- 172.21.15.107:0/284398546 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f58140a2970 0x7f58140a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:16:19.421 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:19.446+0000 7f5829600640 1 -- 172.21.15.107:0/284398546 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f58240cec80 con 0x7f582c109570 2024-03-08T21:16:19.421 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:19.446+0000 7f5830c00640 1 --2- 172.21.15.107:0/284398546 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f58140a2970 0x7f58140a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:16:19.422 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:19.446+0000 7f5830c00640 1 --2- 172.21.15.107:0/284398546 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f58140a2970 0x7f58140a4e30 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f582c19ab10 tx=0x7f5820008370 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:16:19.429 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:19.454+0000 7f5829600640 1 -- 172.21.15.107:0/284398546 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7f5824014030 con 0x7f582c109570 2024-03-08T21:16:19.628 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:19.652+0000 7f5833600640 1 -- 172.21.15.107:0/284398546 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f582c02ce70 con 0x7f582c109570 2024-03-08T21:16:19.629 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:19.653+0000 7f5829600640 1 -- 172.21.15.107:0/284398546 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v115) v1 ==== 100+0+11 (secure 0 0 0) 0x7f582409b940 con 0x7f582c109570 2024-03-08T21:16:19.629 INFO:teuthology.orchestra.run.smithi107.stdout:pg_num: 64 2024-03-08T21:16:19.632 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:19.655+0000 7f5833600640 1 -- 172.21.15.107:0/284398546 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f58140a2970 msgr2=0x7f58140a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:16:19.632 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:19.655+0000 7f5833600640 1 --2- 172.21.15.107:0/284398546 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f58140a2970 0x7f58140a4e30 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f582c19ab10 tx=0x7f5820008370 comp rx=0 tx=0).stop 2024-03-08T21:16:19.632 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:19.655+0000 7f5833600640 1 -- 172.21.15.107:0/284398546 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f582c109570 msgr2=0x7f582c1999d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:16:19.632 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:19.655+0000 7f5833600640 1 --2- 172.21.15.107:0/284398546 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f582c109570 0x7f582c1999d0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f582400c970 tx=0x7f582400ce30 comp rx=0 tx=0).stop 2024-03-08T21:16:19.632 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:19.656+0000 7f5833600640 1 -- 172.21.15.107:0/284398546 shutdown_connections 2024-03-08T21:16:19.632 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:19.656+0000 7f5833600640 1 --2- 172.21.15.107:0/284398546 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7f58140a2970 0x7f58140a4e30 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:16:19.632 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:19.656+0000 7f5833600640 1 --2- 172.21.15.107:0/284398546 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f582c109570 0x7f582c1999d0 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:16:19.632 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:19.656+0000 7f5833600640 1 --2- 172.21.15.107:0/284398546 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7f582c103e70 0x7f582c19fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:16:19.632 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:19.656+0000 7f5833600640 1 --2- 172.21.15.107:0/284398546 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f582c1034b0 0x7f582c19f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:16:19.632 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:19.656+0000 7f5833600640 1 -- 172.21.15.107:0/284398546 >> 172.21.15.107:0/284398546 conn(0x7f582c0fefa0 msgr2=0x7f582c100780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:16:19.633 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:19.656+0000 7f5833600640 1 -- 172.21.15.107:0/284398546 shutdown_connections 2024-03-08T21:16:19.633 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:19.656+0000 7f5833600640 1 -- 172.21.15.107:0/284398546 wait complete. 2024-03-08T21:16:19.651 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-08T21:16:19.839 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:19.862+0000 7fc5f0000640 1 -- 172.21.15.107:0/1786994681 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc5e81034b0 msgr2=0x7fc5e8103930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:16:19.839 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:19.862+0000 7fc5f0000640 1 --2- 172.21.15.107:0/1786994681 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc5e81034b0 0x7fc5e8103930 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7fc5d4009900 tx=0x7fc5d402bc30 comp rx=0 tx=0).stop 2024-03-08T21:16:19.841 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:19.862+0000 7fc5f0000640 1 -- 172.21.15.107:0/1786994681 shutdown_connections 2024-03-08T21:16:19.841 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:19.862+0000 7fc5f0000640 1 --2- 172.21.15.107:0/1786994681 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc5e8103e70 0x7fc5e8110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:16:19.841 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:19.862+0000 7fc5f0000640 1 --2- 172.21.15.107:0/1786994681 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc5e81034b0 0x7fc5e8103930 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:16:19.842 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:19.862+0000 7fc5f0000640 1 --2- 172.21.15.107:0/1786994681 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc5e8109570 0x7fc5e8109950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:16:19.842 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:19.862+0000 7fc5f0000640 1 -- 172.21.15.107:0/1786994681 >> 172.21.15.107:0/1786994681 conn(0x7fc5e80fefa0 msgr2=0x7fc5e81013c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:16:19.842 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:19.862+0000 7fc5f0000640 1 -- 172.21.15.107:0/1786994681 shutdown_connections 2024-03-08T21:16:19.842 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:19.863+0000 7fc5f0000640 1 -- 172.21.15.107:0/1786994681 wait complete. 2024-03-08T21:16:19.844 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:19.864+0000 7fc5f0000640 1 Processor -- start 2024-03-08T21:16:19.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:19.864+0000 7fc5f0000640 1 -- start start 2024-03-08T21:16:19.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:19.864+0000 7fc5f0000640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc5e81034b0 0x7fc5e819f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:16:19.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:19.864+0000 7fc5f0000640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc5e8103e70 0x7fc5e819fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:16:19.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:19.864+0000 7fc5f0000640 1 --2- >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc5e8109570 0x7fc5e81999d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:16:19.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:19.864+0000 7fc5f0000640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5e8113750 con 0x7fc5e81034b0 2024-03-08T21:16:19.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:19.864+0000 7fc5f0000640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5e81135d0 con 0x7fc5e8103e70 2024-03-08T21:16:19.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:19.864+0000 7fc5f0000640 1 -- --> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5e81138d0 con 0x7fc5e8109570 2024-03-08T21:16:19.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:19.864+0000 7fc5ed600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc5e81034b0 0x7fc5e819f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:16:19.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:19.865+0000 7fc5ed600640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc5e81034b0 0x7fc5e819f820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46128/0 (socket says 172.21.15.107:46128) 2024-03-08T21:16:19.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:19.865+0000 7fc5ed600640 1 -- 172.21.15.107:0/2451916088 learned_addr learned my addr 172.21.15.107:0/2451916088 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-03-08T21:16:19.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:19.865+0000 7fc5ecc00640 1 --2- 172.21.15.107:0/2451916088 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc5e8103e70 0x7fc5e819fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:16:19.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:19.865+0000 7fc5ee000640 1 --2- 172.21.15.107:0/2451916088 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc5e8109570 0x7fc5e81999d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:16:19.846 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:19.865+0000 7fc5ed600640 1 -- 172.21.15.107:0/2451916088 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc5e8109570 msgr2=0x7fc5e81999d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:16:19.846 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:19.865+0000 7fc5ed600640 1 --2- 172.21.15.107:0/2451916088 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc5e8109570 0x7fc5e81999d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:16:19.846 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:19.865+0000 7fc5ed600640 1 -- 172.21.15.107:0/2451916088 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc5e8103e70 msgr2=0x7fc5e819fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:16:19.846 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:19.865+0000 7fc5ed600640 1 --2- 172.21.15.107:0/2451916088 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc5e8103e70 0x7fc5e819fd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:16:19.846 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:19.865+0000 7fc5ed600640 1 -- 172.21.15.107:0/2451916088 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5e819a150 con 0x7fc5e81034b0 2024-03-08T21:16:19.846 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:19.865+0000 7fc5ecc00640 1 --2- 172.21.15.107:0/2451916088 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc5e8103e70 0x7fc5e819fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T21:16:19.846 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:19.865+0000 7fc5ee000640 1 --2- 172.21.15.107:0/2451916088 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc5e8109570 0x7fc5e81999d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-08T21:16:19.846 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:19.865+0000 7fc5ed600640 1 --2- 172.21.15.107:0/2451916088 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc5e81034b0 0x7fc5e819f820 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7fc5dc00c920 tx=0x7fc5dc00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:16:19.846 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:19.866+0000 7fc5e6000640 1 -- 172.21.15.107:0/2451916088 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5dc007c70 con 0x7fc5e81034b0 2024-03-08T21:16:19.846 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:19.866+0000 7fc5e6000640 1 -- 172.21.15.107:0/2451916088 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(26 keys) v1 ==== 1102+0+0 (secure 0 0 0) 0x7fc5dc007e10 con 0x7fc5e81034b0 2024-03-08T21:16:19.846 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:19.866+0000 7fc5f0000640 1 -- 172.21.15.107:0/2451916088 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5e819a440 con 0x7fc5e81034b0 2024-03-08T21:16:19.846 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:19.866+0000 7fc5e6000640 1 -- 172.21.15.107:0/2451916088 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5dc018820 con 0x7fc5e81034b0 2024-03-08T21:16:19.847 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:19.866+0000 7fc5f0000640 1 -- 172.21.15.107:0/2451916088 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5e81a66e0 con 0x7fc5e81034b0 2024-03-08T21:16:19.847 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:19.867+0000 7fc5f0000640 1 -- 172.21.15.107:0/2451916088 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5e8068530 con 0x7fc5e81034b0 2024-03-08T21:16:19.847 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:19.869+0000 7fc5e6000640 1 -- 172.21.15.107:0/2451916088 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) v1 ==== 137422+0+0 (secure 0 0 0) 0x7fc5dc0189c0 con 0x7fc5e81034b0 2024-03-08T21:16:19.847 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:19.870+0000 7fc5e6000640 1 --2- 172.21.15.107:0/2451916088 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7fc5d00a2970 0x7fc5d00a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-08T21:16:19.848 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:19.870+0000 7fc5e6000640 1 -- 172.21.15.107:0/2451916088 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fc5dc0cf380 con 0x7fc5e81034b0 2024-03-08T21:16:19.848 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:19.870+0000 7fc5ecc00640 1 --2- 172.21.15.107:0/2451916088 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7fc5d00a2970 0x7fc5d00a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-08T21:16:19.848 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:19.871+0000 7fc5ecc00640 1 --2- 172.21.15.107:0/2451916088 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7fc5d00a2970 0x7fc5d00a4e30 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fc5d4036040 tx=0x7fc5d4005fd0 comp rx=0 tx=0).ready entity=mgr.15375 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-08T21:16:19.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:19.876+0000 7fc5e6000640 1 -- 172.21.15.107:0/2451916088 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184905 (secure 0 0 0) 0x7fc5dc014030 con 0x7fc5e81034b0 2024-03-08T21:16:19.905 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:16:19 smithi107 ceph-mon[31381]: from='client.? 172.21.15.107:0/3136869170' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-08T21:16:19.905 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:16:19 smithi107 ceph-mon[31381]: from='client.? 172.21.15.107:0/3269645986' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-08T21:16:19.905 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:16:19 smithi107 ceph-mon[31381]: osdmap e115: 12 total, 12 up, 12 in 2024-03-08T21:16:19.905 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:16:19 smithi107 ceph-mon[31381]: from='client.? 172.21.15.107:0/284398546' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-08T21:16:20.008 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:16:19 smithi149 ceph-mon[35042]: from='client.? 172.21.15.107:0/3136869170' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-08T21:16:20.008 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:16:19 smithi149 ceph-mon[35042]: from='client.? 172.21.15.107:0/3269645986' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-08T21:16:20.008 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:16:19 smithi149 ceph-mon[35042]: osdmap e115: 12 total, 12 up, 12 in 2024-03-08T21:16:20.008 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:16:19 smithi149 ceph-mon[35042]: from='client.? 172.21.15.107:0/284398546' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-08T21:16:20.053 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:20.077+0000 7fc5f0000640 1 -- 172.21.15.107:0/2451916088 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fc5e819af00 con 0x7fc5e81034b0 2024-03-08T21:16:20.054 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:20.078+0000 7fc5e6000640 1 -- 172.21.15.107:0/2451916088 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 17 entries v115) v1 ==== 81+0+1015 (secure 0 0 0) 0x7fc5dc09c0d0 con 0x7fc5e81034b0 2024-03-08T21:16:20.054 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:0/808124991 2024-03-09T20:48:20.149406+0000 2024-03-08T21:16:20.055 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:6801/3376636618 2024-03-09T21:00:32.431358+0000 2024-03-08T21:16:20.055 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:0/2482452215 2024-03-09T20:47:45.706790+0000 2024-03-08T21:16:20.055 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:6801/3549936519 2024-03-09T20:47:45.706790+0000 2024-03-08T21:16:20.055 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:0/3746019488 2024-03-09T21:00:32.431358+0000 2024-03-08T21:16:20.055 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:0/3691962590 2024-03-09T21:00:32.431358+0000 2024-03-08T21:16:20.055 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:0/1206995985 2024-03-09T20:47:45.706790+0000 2024-03-08T21:16:20.055 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:6800/3549936519 2024-03-09T20:47:45.706790+0000 2024-03-08T21:16:20.055 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:0/2392547434 2024-03-09T20:47:45.706790+0000 2024-03-08T21:16:20.055 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:6800/426372446 2024-03-09T20:48:20.149406+0000 2024-03-08T21:16:20.055 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:6801/426372446 2024-03-09T20:48:20.149406+0000 2024-03-08T21:16:20.055 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:0/1886016707 2024-03-09T20:48:20.149406+0000 2024-03-08T21:16:20.055 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:6800/3376636618 2024-03-09T21:00:32.431358+0000 2024-03-08T21:16:20.055 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:0/1514666962 2024-03-09T20:48:20.149406+0000 2024-03-08T21:16:20.056 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:0/552952579 2024-03-09T21:00:32.431358+0000 2024-03-08T21:16:20.056 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:0/458107598 2024-03-09T21:00:32.431358+0000 2024-03-08T21:16:20.056 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:0/356063043 2024-03-09T21:00:32.431358+0000 2024-03-08T21:16:20.056 INFO:teuthology.orchestra.run.smithi107.stderr:listed 17 entries 2024-03-08T21:16:20.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:20.080+0000 7fc5f0000640 1 -- 172.21.15.107:0/2451916088 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7fc5d00a2970 msgr2=0x7fc5d00a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:16:20.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:20.080+0000 7fc5f0000640 1 --2- 172.21.15.107:0/2451916088 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7fc5d00a2970 0x7fc5d00a4e30 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fc5d4036040 tx=0x7fc5d4005fd0 comp rx=0 tx=0).stop 2024-03-08T21:16:20.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:20.080+0000 7fc5f0000640 1 -- 172.21.15.107:0/2451916088 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc5e81034b0 msgr2=0x7fc5e819f820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-08T21:16:20.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:20.080+0000 7fc5f0000640 1 --2- 172.21.15.107:0/2451916088 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc5e81034b0 0x7fc5e819f820 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7fc5dc00c920 tx=0x7fc5dc00cde0 comp rx=0 tx=0).stop 2024-03-08T21:16:20.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:20.081+0000 7fc5f0000640 1 -- 172.21.15.107:0/2451916088 shutdown_connections 2024-03-08T21:16:20.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:20.081+0000 7fc5f0000640 1 --2- 172.21.15.107:0/2451916088 >> [v2:172.21.15.107:6800/2792338347,v1:172.21.15.107:6801/2792338347] conn(0x7fc5d00a2970 0x7fc5d00a4e30 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:16:20.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:20.081+0000 7fc5f0000640 1 --2- 172.21.15.107:0/2451916088 >> [v2:172.21.15.204:3300/0,v1:172.21.15.204:6789/0] conn(0x7fc5e8109570 0x7fc5e81999d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:16:20.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:20.081+0000 7fc5f0000640 1 --2- 172.21.15.107:0/2451916088 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc5e8103e70 0x7fc5e819fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:16:20.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:20.081+0000 7fc5f0000640 1 --2- 172.21.15.107:0/2451916088 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc5e81034b0 0x7fc5e819f820 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-08T21:16:20.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:20.081+0000 7fc5f0000640 1 -- 172.21.15.107:0/2451916088 >> 172.21.15.107:0/2451916088 conn(0x7fc5e80fefa0 msgr2=0x7fc5e8100890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-08T21:16:20.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:20.081+0000 7fc5f0000640 1 -- 172.21.15.107:0/2451916088 shutdown_connections 2024-03-08T21:16:20.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-03-08T21:16:20.081+0000 7fc5f0000640 1 -- 172.21.15.107:0/2451916088 wait complete. 2024-03-08T21:16:20.075 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi107.front.sepia.ceph.com... 2024-03-08T21:16:20.075 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T21:16:20.075 DEBUG:teuthology.orchestra.run.smithi107:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-03-08T21:16:20.121 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:16:19 smithi204 ceph-mon[34935]: from='client.? 172.21.15.107:0/3136869170' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-08T21:16:20.121 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:16:19 smithi204 ceph-mon[34935]: from='client.? 172.21.15.107:0/3269645986' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-08T21:16:20.122 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:16:19 smithi204 ceph-mon[34935]: osdmap e115: 12 total, 12 up, 12 in 2024-03-08T21:16:20.122 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:16:19 smithi204 ceph-mon[34935]: from='client.? 172.21.15.107:0/284398546' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-08T21:16:20.163 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi107.stderr:ceph-fuse[84486]: fuse finished with error 0 and tester_r 0 2024-03-08T21:16:20.167 INFO:teuthology.orchestra.run:waiting for 300 2024-03-08T21:16:21.008 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:16:20 smithi149 ceph-mon[35042]: pgmap v501: 97 pgs: 97 active+clean; 17 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 307 B/s wr, 10 op/s 2024-03-08T21:16:21.008 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:16:20 smithi149 ceph-mon[35042]: from='client.? 172.21.15.107:0/2451916088' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-08T21:16:21.114 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:16:20 smithi107 ceph-mon[31381]: pgmap v501: 97 pgs: 97 active+clean; 17 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 307 B/s wr, 10 op/s 2024-03-08T21:16:21.114 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:16:20 smithi107 ceph-mon[31381]: from='client.? 172.21.15.107:0/2451916088' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-08T21:16:21.121 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:16:20 smithi204 ceph-mon[34935]: pgmap v501: 97 pgs: 97 active+clean; 17 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 307 B/s wr, 10 op/s 2024-03-08T21:16:21.121 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:16:20 smithi204 ceph-mon[34935]: from='client.? 172.21.15.107:0/2451916088' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-08T21:16:23.008 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:16:22 smithi149 ceph-mon[35042]: pgmap v503: 97 pgs: 97 active+clean; 17 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 307 B/s wr, 10 op/s 2024-03-08T21:16:23.121 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:16:22 smithi204 ceph-mon[34935]: pgmap v503: 97 pgs: 97 active+clean; 17 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 307 B/s wr, 10 op/s 2024-03-08T21:16:23.210 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:16:22 smithi107 ceph-mon[31381]: pgmap v503: 97 pgs: 97 active+clean; 17 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 307 B/s wr, 10 op/s 2024-03-08T21:16:25.008 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:16:24 smithi149 ceph-mon[35042]: pgmap v504: 97 pgs: 97 active+clean; 17 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 307 B/s wr, 10 op/s 2024-03-08T21:16:25.010 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:16:24 smithi204 ceph-mon[34935]: pgmap v504: 97 pgs: 97 active+clean; 17 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 307 B/s wr, 10 op/s 2024-03-08T21:16:25.210 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:16:24 smithi107 ceph-mon[31381]: pgmap v504: 97 pgs: 97 active+clean; 17 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 307 B/s wr, 10 op/s 2024-03-08T21:16:26.269 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi107.front.sepia.ceph.com 2024-03-08T21:16:26.269 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T21:16:26.270 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-03-08T21:16:26.341 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T21:16:26.341 DEBUG:teuthology.orchestra.run.smithi107:> ip netns list 2024-03-08T21:16:26.396 INFO:teuthology.orchestra.run.smithi107.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-08T21:16:26.397 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T21:16:26.398 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-08T21:16:26.492 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-08T21:16:26.492 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip link delete ceph-brx 2024-03-08T21:16:26.612 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-03-08T21:16:26.624 INFO:tasks.cephadm:Teardown begin 2024-03-08T21:16:26.624 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_ceph-c_7d63482aa89628d733e298984a762389b625c761/qa/tasks/cephadm.py", line 1867, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/task/__init__.py", line 132, in __exit__ self.end() File "/home/teuthworker/src/git.ceph.com_ceph-c_7d63482aa89628d733e298984a762389b625c761/qa/tasks/check_counter.py", line 84, in end proc = manager.admin_socket(daemon_type, daemon_id, ["perf", "dump"]) File "/home/teuthworker/src/git.ceph.com_ceph-c_7d63482aa89628d733e298984a762389b625c761/qa/tasks/ceph_manager.py", line 1875, in admin_socket return shell( File "/home/teuthworker/src/git.ceph.com_ceph-c_7d63482aa89628d733e298984a762389b625c761/qa/tasks/ceph_manager.py", line 43, in shell return remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi204 with status 22: 'sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761 shell --fsid dc20f0f8-dd8c-11ee-95c6-87774f69a715 -- ceph daemon mds.c perf dump' 2024-03-08T21:16:26.626 DEBUG:teuthology.orchestra.run.smithi107:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-08T21:16:26.701 DEBUG:teuthology.orchestra.run.smithi149:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-08T21:16:26.805 DEBUG:teuthology.orchestra.run.smithi204:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-08T21:16:26.900 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-03-08T21:16:26.900 DEBUG:teuthology.orchestra.run.smithi107:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-03-08T21:16:26.957 DEBUG:teuthology.orchestra.run.smithi149:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-03-08T21:16:27.008 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:16:26 smithi149 ceph-mon[35042]: pgmap v505: 97 pgs: 97 active+clean; 17 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.1 KiB/s wr, 12 op/s 2024-03-08T21:16:27.014 DEBUG:teuthology.orchestra.run.smithi204:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-03-08T21:16:27.073 INFO:tasks.cephadm:Stopping all daemons... 2024-03-08T21:16:27.073 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-03-08T21:16:27.073 DEBUG:teuthology.orchestra.run.smithi107:> sudo systemctl stop ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715@mon.a 2024-03-08T21:16:27.121 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:16:26 smithi204 ceph-mon[34935]: pgmap v505: 97 pgs: 97 active+clean; 17 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.1 KiB/s wr, 12 op/s 2024-03-08T21:16:27.161 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:16:26 smithi107 ceph-mon[31381]: pgmap v505: 97 pgs: 97 active+clean; 17 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.1 KiB/s wr, 12 op/s 2024-03-08T21:16:27.460 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:16:27 smithi107 systemd[1]: Stopping Ceph mon.a for dc20f0f8-dd8c-11ee-95c6-87774f69a715... 2024-03-08T21:16:27.891 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:16:27 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mon-a[31377]: 2024-03-08T21:16:27.657+0000 7f41bf000700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-08T21:16:27.891 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:16:27 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mon-a[31377]: 2024-03-08T21:16:27.657+0000 7f41bf000700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-03-08T21:16:28.152 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:16:27 smithi107 podman[94840]: 2024-03-08 21:16:27.915614927 +0000 UTC m=+0.394548429 container died 80e0993b3b702919eb2a96b1a997815c7463c65afe695580e83d63562c53205f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=rishabh-2024mar8-7d63482, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-08T21:16:28.461 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:16:28 smithi107 podman[94840]: 2024-03-08 21:16:28.343040152 +0000 UTC m=+0.821973647 container cleanup 80e0993b3b702919eb2a96b1a997815c7463c65afe695580e83d63562c53205f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mon-a, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=rishabh-2024mar8-7d63482, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-03-08T21:16:28.461 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:16:28 smithi107 bash[94840]: ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mon-a 2024-03-08T21:16:28.960 INFO:journalctl@ceph.mon.a.smithi107.stdout:Mar 08 21:16:28 smithi107 podman[94853]: 2024-03-08 21:16:28.661187248 +0000 UTC m=+0.731685032 container remove 80e0993b3b702919eb2a96b1a997815c7463c65afe695580e83d63562c53205f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mon-a, RELEASE=rishabh-2024mar8-7d63482, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-08T21:16:29.113 DEBUG:teuthology.orchestra.run.smithi107:> sudo pkill -f 'journalctl -f -n 0 -u ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715@mon.a.service' 2024-03-08T21:16:29.235 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-08T21:16:29.235 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-03-08T21:16:29.236 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-03-08T21:16:29.236 DEBUG:teuthology.orchestra.run.smithi149:> sudo systemctl stop ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715@mon.b 2024-03-08T21:16:29.758 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:16:29 smithi149 systemd[1]: Stopping Ceph mon.b for dc20f0f8-dd8c-11ee-95c6-87774f69a715... 2024-03-08T21:16:30.262 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:16:29 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mon-b[35038]: 2024-03-08T21:16:29.795+0000 7f0a04e00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-08T21:16:30.262 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:16:29 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mon-b[35038]: 2024-03-08T21:16:29.795+0000 7f0a04e00700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-03-08T21:16:30.262 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:16:29 smithi149 podman[72066]: 2024-03-08 21:16:29.909498852 +0000 UTC m=+0.242666269 container stop 8cecc7919a787e1ca250082d39d07931dc0f0e8e336b6b9c293973f59ad75088 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mon-b, RELEASE=rishabh-2024mar8-7d63482, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-03-08T21:16:30.262 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:16:29 smithi149 podman[72066]: 2024-03-08 21:16:29.936172248 +0000 UTC m=+0.269339625 container died 8cecc7919a787e1ca250082d39d07931dc0f0e8e336b6b9c293973f59ad75088 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mon-b, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=rishabh-2024mar8-7d63482, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-08T21:16:30.684 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:16:30 smithi149 podman[72066]: 2024-03-08 21:16:30.448566401 +0000 UTC m=+0.781733848 container cleanup 8cecc7919a787e1ca250082d39d07931dc0f0e8e336b6b9c293973f59ad75088 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mon-b, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240307, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=rishabh-2024mar8-7d63482, CEPH_POINT_RELEASE=) 2024-03-08T21:16:30.684 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:16:30 smithi149 bash[72066]: ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mon-b 2024-03-08T21:16:31.008 INFO:journalctl@ceph.mon.b.smithi149.stdout:Mar 08 21:16:30 smithi149 podman[72080]: 2024-03-08 21:16:30.708221259 +0000 UTC m=+0.778967023 container remove 8cecc7919a787e1ca250082d39d07931dc0f0e8e336b6b9c293973f59ad75088 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mon-b, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=rishabh-2024mar8-7d63482, io.buildah.version=1.33.5) 2024-03-08T21:16:31.218 DEBUG:teuthology.orchestra.run.smithi149:> sudo pkill -f 'journalctl -f -n 0 -u ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715@mon.b.service' 2024-03-08T21:16:31.360 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-08T21:16:31.360 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-03-08T21:16:31.360 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-03-08T21:16:31.360 DEBUG:teuthology.orchestra.run.smithi204:> sudo systemctl stop ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715@mon.c 2024-03-08T21:16:31.871 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:16:31 smithi204 systemd[1]: Stopping Ceph mon.c for dc20f0f8-dd8c-11ee-95c6-87774f69a715... 2024-03-08T21:16:32.452 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:16:32 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mon-c[34931]: 2024-03-08T21:16:32.186+0000 7fec0e000700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-08T21:16:32.453 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:16:32 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mon-c[34931]: 2024-03-08T21:16:32.186+0000 7fec0e000700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-03-08T21:16:32.722 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:16:32 smithi204 podman[73236]: 2024-03-08 21:16:32.4762523 +0000 UTC m=+0.684759646 container died c7d2a8d930206e45ef4add3e134a7be18d6cf2f728e047b8e4adb69bdd5f9f36 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mon-c, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=rishabh-2024mar8-7d63482, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-03-08T21:16:33.117 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:16:32 smithi204 podman[73236]: 2024-03-08 21:16:32.88076599 +0000 UTC m=+1.089273364 container cleanup c7d2a8d930206e45ef4add3e134a7be18d6cf2f728e047b8e4adb69bdd5f9f36 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mon-c, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=rishabh-2024mar8-7d63482, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-03-08T21:16:33.117 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:16:32 smithi204 bash[73236]: ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mon-c 2024-03-08T21:16:33.371 INFO:journalctl@ceph.mon.c.smithi204.stdout:Mar 08 21:16:33 smithi204 podman[73248]: 2024-03-08 21:16:33.140895319 +0000 UTC m=+0.649797047 container remove c7d2a8d930206e45ef4add3e134a7be18d6cf2f728e047b8e4adb69bdd5f9f36 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mon-c, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=rishabh-2024mar8-7d63482, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True) 2024-03-08T21:16:33.566 DEBUG:teuthology.orchestra.run.smithi204:> sudo pkill -f 'journalctl -f -n 0 -u ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715@mon.c.service' 2024-03-08T21:16:33.735 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-08T21:16:33.736 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-03-08T21:16:33.736 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-03-08T21:16:33.736 DEBUG:teuthology.orchestra.run.smithi107:> sudo systemctl stop ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715@mgr.x 2024-03-08T21:16:34.210 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 08 21:16:33 smithi107 systemd[1]: Stopping Ceph mgr.x for dc20f0f8-dd8c-11ee-95c6-87774f69a715... 2024-03-08T21:16:34.711 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 08 21:16:34 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-x[31606]: 2024-03-08T21:16:34.318+0000 7f65c7a00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-08T21:16:34.711 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 08 21:16:34 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-x[31606]: 2024-03-08T21:16:34.318+0000 7f65c7a00700 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-03-08T21:16:34.711 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 08 21:16:34 smithi107 podman[94960]: 2024-03-08 21:16:34.424156145 +0000 UTC m=+0.265285214 container died cec518020e7db1061653e72b3437af730f6aee622a32e17e9297c739964d334a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-x, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=rishabh-2024mar8-7d63482, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-03-08T21:16:35.134 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 08 21:16:34 smithi107 podman[94960]: 2024-03-08 21:16:34.948083052 +0000 UTC m=+0.789212102 container cleanup cec518020e7db1061653e72b3437af730f6aee622a32e17e9297c739964d334a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-x, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=rishabh-2024mar8-7d63482, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-03-08T21:16:35.134 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 08 21:16:34 smithi107 bash[94960]: ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-x 2024-03-08T21:16:35.460 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Mar 08 21:16:35 smithi107 podman[94974]: 2024-03-08 21:16:35.157827038 +0000 UTC m=+0.717465778 container remove cec518020e7db1061653e72b3437af730f6aee622a32e17e9297c739964d334a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-x, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, RELEASE=rishabh-2024mar8-7d63482, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-08T21:16:35.617 DEBUG:teuthology.orchestra.run.smithi107:> sudo pkill -f 'journalctl -f -n 0 -u ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715@mgr.x.service' 2024-03-08T21:16:35.737 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-08T21:16:35.738 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-03-08T21:16:35.738 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-03-08T21:16:35.738 DEBUG:teuthology.orchestra.run.smithi149:> sudo systemctl stop ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715@mgr.y 2024-03-08T21:16:36.258 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Mar 08 21:16:35 smithi149 systemd[1]: Stopping Ceph mgr.y for dc20f0f8-dd8c-11ee-95c6-87774f69a715... 2024-03-08T21:16:36.651 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Mar 08 21:16:36 smithi149 podman[72197]: 2024-03-08 21:16:36.387869183 +0000 UTC m=+0.240828564 container died 47118d6ce41b9f0ee58b1e2d5b80236fac84a34a388d680e12bad92fdb3c6fc6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-y, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=rishabh-2024mar8-7d63482, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240307) 2024-03-08T21:16:37.008 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Mar 08 21:16:36 smithi149 podman[72197]: 2024-03-08 21:16:36.853365285 +0000 UTC m=+0.706324682 container cleanup 47118d6ce41b9f0ee58b1e2d5b80236fac84a34a388d680e12bad92fdb3c6fc6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-y, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=rishabh-2024mar8-7d63482, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, GIT_CLEAN=True) 2024-03-08T21:16:37.008 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Mar 08 21:16:36 smithi149 bash[72197]: ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-y 2024-03-08T21:16:37.508 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Mar 08 21:16:37 smithi149 podman[72212]: 2024-03-08 21:16:37.105256015 +0000 UTC m=+0.702132733 container remove 47118d6ce41b9f0ee58b1e2d5b80236fac84a34a388d680e12bad92fdb3c6fc6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-y, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=rishabh-2024mar8-7d63482, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-03-08T21:16:37.508 INFO:journalctl@ceph.mgr.y.smithi149.stdout:Mar 08 21:16:37 smithi149 systemd[1]: ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715@mgr.y.service: Main process exited, code=exited, status=143/n/a 2024-03-08T21:16:37.600 DEBUG:teuthology.orchestra.run.smithi149:> sudo pkill -f 'journalctl -f -n 0 -u ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715@mgr.y.service' 2024-03-08T21:16:37.716 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-08T21:16:37.716 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-03-08T21:16:37.716 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-03-08T21:16:37.716 DEBUG:teuthology.orchestra.run.smithi204:> sudo systemctl stop ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715@mgr.z 2024-03-08T21:16:38.121 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Mar 08 21:16:37 smithi204 systemd[1]: Stopping Ceph mgr.z for dc20f0f8-dd8c-11ee-95c6-87774f69a715... 2024-03-08T21:16:38.661 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Mar 08 21:16:38 smithi204 podman[73361]: 2024-03-08 21:16:38.417300301 +0000 UTC m=+0.233166509 container died 6cff0bd156dc85b8535cf852375d41f3f744dbecbae8e925992a62e5e62ae8d9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-z, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=rishabh-2024mar8-7d63482, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-03-08T21:16:39.030 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Mar 08 21:16:38 smithi204 podman[73361]: 2024-03-08 21:16:38.912221823 +0000 UTC m=+0.728088024 container cleanup 6cff0bd156dc85b8535cf852375d41f3f744dbecbae8e925992a62e5e62ae8d9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-z, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=rishabh-2024mar8-7d63482, org.label-schema.schema-version=1.0) 2024-03-08T21:16:39.030 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Mar 08 21:16:38 smithi204 bash[73361]: ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-z 2024-03-08T21:16:39.331 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Mar 08 21:16:39 smithi204 podman[73375]: 2024-03-08 21:16:39.147246227 +0000 UTC m=+0.717697967 container remove 6cff0bd156dc85b8535cf852375d41f3f744dbecbae8e925992a62e5e62ae8d9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mgr-z, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, RELEASE=rishabh-2024mar8-7d63482, CEPH_POINT_RELEASE=) 2024-03-08T21:16:39.331 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Mar 08 21:16:39 smithi204 systemd[1]: ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715@mgr.z.service: Main process exited, code=exited, status=143/n/a 2024-03-08T21:16:39.541 DEBUG:teuthology.orchestra.run.smithi204:> sudo pkill -f 'journalctl -f -n 0 -u ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715@mgr.z.service' 2024-03-08T21:16:39.613 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Mar 08 21:16:39 smithi204 systemd[1]: ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715@mgr.z.service: Failed with result 'exit-code'. 2024-03-08T21:16:39.614 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Mar 08 21:16:39 smithi204 systemd[1]: Stopped Ceph mgr.z for dc20f0f8-dd8c-11ee-95c6-87774f69a715. 2024-03-08T21:16:39.614 INFO:journalctl@ceph.mgr.z.smithi204.stdout:Mar 08 21:16:39 smithi204 systemd[1]: ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715@mgr.z.service: Consumed 33.486s CPU time. 2024-03-08T21:16:39.648 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-08T21:16:39.649 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-03-08T21:16:39.649 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-03-08T21:16:39.649 DEBUG:teuthology.orchestra.run.smithi107:> sudo systemctl stop ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715@osd.0 2024-03-08T21:16:40.209 INFO:journalctl@ceph.osd.0.smithi107.stdout:Mar 08 21:16:39 smithi107 systemd[1]: Stopping Ceph osd.0 for dc20f0f8-dd8c-11ee-95c6-87774f69a715... 2024-03-08T21:16:40.460 INFO:journalctl@ceph.osd.0.smithi107.stdout:Mar 08 21:16:40 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-0[46080]: 2024-03-08T21:16:40.232+0000 7f34df200700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-08T21:16:40.461 INFO:journalctl@ceph.osd.0.smithi107.stdout:Mar 08 21:16:40 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-0[46080]: 2024-03-08T21:16:40.232+0000 7f34df200700 -1 osd.0 115 *** Got signal Terminated *** 2024-03-08T21:16:40.461 INFO:journalctl@ceph.osd.0.smithi107.stdout:Mar 08 21:16:40 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-0[46080]: 2024-03-08T21:16:40.232+0000 7f34df200700 -1 osd.0 115 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-08T21:16:45.961 INFO:journalctl@ceph.osd.0.smithi107.stdout:Mar 08 21:16:45 smithi107 podman[95081]: 2024-03-08 21:16:45.654495874 +0000 UTC m=+5.570247839 container died 6269d4bb525de9dde54d0d806c006eac79d340d3058444bba6ccd4d090807976 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-0, RELEASE=rishabh-2024mar8-7d63482, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-08T21:16:46.372 INFO:journalctl@ceph.osd.0.smithi107.stdout:Mar 08 21:16:46 smithi107 podman[95081]: 2024-03-08 21:16:46.160921988 +0000 UTC m=+6.076673989 container cleanup 6269d4bb525de9dde54d0d806c006eac79d340d3058444bba6ccd4d090807976 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=rishabh-2024mar8-7d63482, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-03-08T21:16:46.372 INFO:journalctl@ceph.osd.0.smithi107.stdout:Mar 08 21:16:46 smithi107 bash[95081]: ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-0 2024-03-08T21:16:46.710 INFO:journalctl@ceph.osd.0.smithi107.stdout:Mar 08 21:16:46 smithi107 podman[95102]: 2024-03-08 21:16:46.395486359 +0000 UTC m=+0.731084856 container remove 6269d4bb525de9dde54d0d806c006eac79d340d3058444bba6ccd4d090807976 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-0, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=rishabh-2024mar8-7d63482, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-03-08T21:16:47.460 INFO:journalctl@ceph.osd.0.smithi107.stdout:Mar 08 21:16:47 smithi107 podman[95192]: 2024-03-08 21:16:47.079506318 +0000 UTC m=+0.033638080 image pull a272146cb810b1fd4a53627f019472132e30966d5c25b01bfc44a001295f1249 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd 2024-03-08T21:16:47.461 INFO:journalctl@ceph.osd.0.smithi107.stdout:Mar 08 21:16:47 smithi107 podman[95192]: 2024-03-08 21:16:47.204775533 +0000 UTC m=+0.158907321 container create 6b63c16d34a902c0fbb270849371f3cbd4198d1411f46fa50eebfe73f6355ba5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-0-deactivate, RELEASE=rishabh-2024mar8-7d63482, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-08T21:16:47.922 INFO:journalctl@ceph.osd.0.smithi107.stdout:Mar 08 21:16:47 smithi107 podman[95192]: 2024-03-08 21:16:47.509062198 +0000 UTC m=+0.463194016 container init 6b63c16d34a902c0fbb270849371f3cbd4198d1411f46fa50eebfe73f6355ba5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-0-deactivate, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , RELEASE=rishabh-2024mar8-7d63482, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-03-08T21:16:47.922 INFO:journalctl@ceph.osd.0.smithi107.stdout:Mar 08 21:16:47 smithi107 podman[95192]: 2024-03-08 21:16:47.520696231 +0000 UTC m=+0.474828046 container start 6b63c16d34a902c0fbb270849371f3cbd4198d1411f46fa50eebfe73f6355ba5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-0-deactivate, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, RELEASE=rishabh-2024mar8-7d63482, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-08T21:16:47.922 INFO:journalctl@ceph.osd.0.smithi107.stdout:Mar 08 21:16:47 smithi107 podman[95192]: 2024-03-08 21:16:47.649739256 +0000 UTC m=+0.603871074 container attach 6b63c16d34a902c0fbb270849371f3cbd4198d1411f46fa50eebfe73f6355ba5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-0-deactivate, org.label-schema.license=GPLv2, RELEASE=rishabh-2024mar8-7d63482, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-08T21:16:48.210 INFO:journalctl@ceph.osd.0.smithi107.stdout:Mar 08 21:16:47 smithi107 podman[95192]: 2024-03-08 21:16:47.946750013 +0000 UTC m=+0.900881819 container died 6b63c16d34a902c0fbb270849371f3cbd4198d1411f46fa50eebfe73f6355ba5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-0-deactivate, RELEASE=rishabh-2024mar8-7d63482, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240307) 2024-03-08T21:16:48.705 DEBUG:teuthology.orchestra.run.smithi107:> sudo pkill -f 'journalctl -f -n 0 -u ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715@osd.0.service' 2024-03-08T21:16:48.819 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-08T21:16:48.819 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-03-08T21:16:48.820 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-03-08T21:16:48.820 DEBUG:teuthology.orchestra.run.smithi149:> sudo systemctl stop ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715@osd.1 2024-03-08T21:16:49.258 INFO:journalctl@ceph.osd.1.smithi149.stdout:Mar 08 21:16:48 smithi149 systemd[1]: Stopping Ceph osd.1 for dc20f0f8-dd8c-11ee-95c6-87774f69a715... 2024-03-08T21:16:49.615 INFO:journalctl@ceph.osd.1.smithi149.stdout:Mar 08 21:16:49 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-1[39636]: 2024-03-08T21:16:49.388+0000 7f9d3c400700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-08T21:16:49.615 INFO:journalctl@ceph.osd.1.smithi149.stdout:Mar 08 21:16:49 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-1[39636]: 2024-03-08T21:16:49.388+0000 7f9d3c400700 -1 osd.1 115 *** Got signal Terminated *** 2024-03-08T21:16:49.615 INFO:journalctl@ceph.osd.1.smithi149.stdout:Mar 08 21:16:49 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-1[39636]: 2024-03-08T21:16:49.388+0000 7f9d3c400700 -1 osd.1 115 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-08T21:16:55.008 INFO:journalctl@ceph.osd.1.smithi149.stdout:Mar 08 21:16:54 smithi149 podman[72321]: 2024-03-08 21:16:54.753556095 +0000 UTC m=+5.511114508 container died e9c05da3e3f30f9c2d75f5414bba505bca401344fdac8b523a22a18f40f4b050 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-1, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=rishabh-2024mar8-7d63482, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=) 2024-03-08T21:16:55.436 INFO:journalctl@ceph.osd.1.smithi149.stdout:Mar 08 21:16:55 smithi149 podman[72321]: 2024-03-08 21:16:55.217380044 +0000 UTC m=+5.974938504 container cleanup e9c05da3e3f30f9c2d75f5414bba505bca401344fdac8b523a22a18f40f4b050 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-1, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=rishabh-2024mar8-7d63482, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-03-08T21:16:55.436 INFO:journalctl@ceph.osd.1.smithi149.stdout:Mar 08 21:16:55 smithi149 bash[72321]: ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-1 2024-03-08T21:16:55.758 INFO:journalctl@ceph.osd.1.smithi149.stdout:Mar 08 21:16:55 smithi149 podman[72351]: 2024-03-08 21:16:55.460489071 +0000 UTC m=+0.692651478 container remove e9c05da3e3f30f9c2d75f5414bba505bca401344fdac8b523a22a18f40f4b050 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-1, GIT_BRANCH=HEAD, RELEASE=rishabh-2024mar8-7d63482, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=) 2024-03-08T21:16:56.508 INFO:journalctl@ceph.osd.1.smithi149.stdout:Mar 08 21:16:56 smithi149 podman[72442]: 2024-03-08 21:16:56.173387616 +0000 UTC m=+0.036216599 image pull a272146cb810b1fd4a53627f019472132e30966d5c25b01bfc44a001295f1249 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd 2024-03-08T21:16:56.508 INFO:journalctl@ceph.osd.1.smithi149.stdout:Mar 08 21:16:56 smithi149 podman[72442]: 2024-03-08 21:16:56.294975083 +0000 UTC m=+0.157804103 container create fa2bbd1286c98439bfc032cf6239ef83268117fecd040bb032fd6d2cdb327e88 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-1-deactivate, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, ceph=True, RELEASE=rishabh-2024mar8-7d63482, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-03-08T21:16:57.009 INFO:journalctl@ceph.osd.1.smithi149.stdout:Mar 08 21:16:56 smithi149 podman[72442]: 2024-03-08 21:16:56.590806311 +0000 UTC m=+0.453635374 container init fa2bbd1286c98439bfc032cf6239ef83268117fecd040bb032fd6d2cdb327e88 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-1-deactivate, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=rishabh-2024mar8-7d63482, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-03-08T21:16:57.009 INFO:journalctl@ceph.osd.1.smithi149.stdout:Mar 08 21:16:56 smithi149 podman[72442]: 2024-03-08 21:16:56.603901188 +0000 UTC m=+0.466730178 container start fa2bbd1286c98439bfc032cf6239ef83268117fecd040bb032fd6d2cdb327e88 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-1-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=rishabh-2024mar8-7d63482, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2) 2024-03-08T21:16:57.009 INFO:journalctl@ceph.osd.1.smithi149.stdout:Mar 08 21:16:56 smithi149 podman[72442]: 2024-03-08 21:16:56.689284178 +0000 UTC m=+0.552113261 container attach fa2bbd1286c98439bfc032cf6239ef83268117fecd040bb032fd6d2cdb327e88 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-1-deactivate, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, io.buildah.version=1.33.5, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=rishabh-2024mar8-7d63482, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-08T21:16:57.365 INFO:journalctl@ceph.osd.1.smithi149.stdout:Mar 08 21:16:57 smithi149 podman[72442]: 2024-03-08 21:16:57.04057314 +0000 UTC m=+0.903402155 container died fa2bbd1286c98439bfc032cf6239ef83268117fecd040bb032fd6d2cdb327e88 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-1-deactivate, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=rishabh-2024mar8-7d63482) 2024-03-08T21:16:57.747 DEBUG:teuthology.orchestra.run.smithi149:> sudo pkill -f 'journalctl -f -n 0 -u ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715@osd.1.service' 2024-03-08T21:16:57.873 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-08T21:16:57.874 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-03-08T21:16:57.874 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-03-08T21:16:57.874 DEBUG:teuthology.orchestra.run.smithi204:> sudo systemctl stop ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715@osd.2 2024-03-08T21:16:58.372 INFO:journalctl@ceph.osd.2.smithi204.stdout:Mar 08 21:16:58 smithi204 systemd[1]: Stopping Ceph osd.2 for dc20f0f8-dd8c-11ee-95c6-87774f69a715... 2024-03-08T21:16:58.871 INFO:journalctl@ceph.osd.2.smithi204.stdout:Mar 08 21:16:58 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-2[40216]: 2024-03-08T21:16:58.480+0000 7f1998600700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-08T21:16:58.872 INFO:journalctl@ceph.osd.2.smithi204.stdout:Mar 08 21:16:58 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-2[40216]: 2024-03-08T21:16:58.480+0000 7f1998600700 -1 osd.2 115 *** Got signal Terminated *** 2024-03-08T21:16:58.872 INFO:journalctl@ceph.osd.2.smithi204.stdout:Mar 08 21:16:58 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-2[40216]: 2024-03-08T21:16:58.480+0000 7f1998600700 -1 osd.2 115 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-08T21:17:04.281 INFO:journalctl@ceph.osd.2.smithi204.stdout:Mar 08 21:17:03 smithi204 podman[73497]: 2024-03-08 21:17:03.988413818 +0000 UTC m=+5.654187230 container died ff79fc65d95c1ec53eb09b74806b8921361e9fa37b5099ef7664d0397c6017ce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-2, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240307, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=rishabh-2024mar8-7d63482, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-03-08T21:17:04.621 INFO:journalctl@ceph.osd.2.smithi204.stdout:Mar 08 21:17:04 smithi204 podman[73497]: 2024-03-08 21:17:04.442425254 +0000 UTC m=+6.108198703 container cleanup ff79fc65d95c1ec53eb09b74806b8921361e9fa37b5099ef7664d0397c6017ce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-2, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=rishabh-2024mar8-7d63482) 2024-03-08T21:17:04.621 INFO:journalctl@ceph.osd.2.smithi204.stdout:Mar 08 21:17:04 smithi204 bash[73497]: ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-2 2024-03-08T21:17:05.121 INFO:journalctl@ceph.osd.2.smithi204.stdout:Mar 08 21:17:04 smithi204 podman[73515]: 2024-03-08 21:17:04.735872911 +0000 UTC m=+0.727059484 container remove ff79fc65d95c1ec53eb09b74806b8921361e9fa37b5099ef7664d0397c6017ce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-2, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=rishabh-2024mar8-7d63482) 2024-03-08T21:17:05.850 INFO:journalctl@ceph.osd.2.smithi204.stdout:Mar 08 21:17:05 smithi204 podman[73605]: 2024-03-08 21:17:05.460549728 +0000 UTC m=+0.032809369 image pull a272146cb810b1fd4a53627f019472132e30966d5c25b01bfc44a001295f1249 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd 2024-03-08T21:17:05.850 INFO:journalctl@ceph.osd.2.smithi204.stdout:Mar 08 21:17:05 smithi204 podman[73605]: 2024-03-08 21:17:05.611986378 +0000 UTC m=+0.184246044 container create d35cac79bec81b4ea9bc3dc7959ea571d063ce5ea37050cb9b675f031576b8ac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-2-deactivate, io.buildah.version=1.33.5, RELEASE=rishabh-2024mar8-7d63482, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-03-08T21:17:06.121 INFO:journalctl@ceph.osd.2.smithi204.stdout:Mar 08 21:17:05 smithi204 podman[73605]: 2024-03-08 21:17:05.87441119 +0000 UTC m=+0.446670899 container init d35cac79bec81b4ea9bc3dc7959ea571d063ce5ea37050cb9b675f031576b8ac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-2-deactivate, RELEASE=rishabh-2024mar8-7d63482, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-08T21:17:06.121 INFO:journalctl@ceph.osd.2.smithi204.stdout:Mar 08 21:17:05 smithi204 podman[73605]: 2024-03-08 21:17:05.885865701 +0000 UTC m=+0.458125347 container start d35cac79bec81b4ea9bc3dc7959ea571d063ce5ea37050cb9b675f031576b8ac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-2-deactivate, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, RELEASE=rishabh-2024mar8-7d63482, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-03-08T21:17:06.121 INFO:journalctl@ceph.osd.2.smithi204.stdout:Mar 08 21:17:05 smithi204 podman[73605]: 2024-03-08 21:17:05.989879613 +0000 UTC m=+0.562139333 container attach d35cac79bec81b4ea9bc3dc7959ea571d063ce5ea37050cb9b675f031576b8ac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-2-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, RELEASE=rishabh-2024mar8-7d63482, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-03-08T21:17:06.508 INFO:journalctl@ceph.osd.4.smithi149.stdout:Mar 08 21:17:06 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-4[46289]: 2024-03-08T21:17:06.107+0000 7f297fc00700 -1 osd.4 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:41.916650+0000 front 2024-03-08T21:16:41.917178+0000 (oldest deadline 2024-03-08T21:17:06.016911+0000) 2024-03-08T21:17:06.621 INFO:journalctl@ceph.osd.2.smithi204.stdout:Mar 08 21:17:06 smithi204 podman[73605]: 2024-03-08 21:17:06.322534745 +0000 UTC m=+0.894794414 container died d35cac79bec81b4ea9bc3dc7959ea571d063ce5ea37050cb9b675f031576b8ac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-2-deactivate, RELEASE=rishabh-2024mar8-7d63482, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-03-08T21:17:07.117 DEBUG:teuthology.orchestra.run.smithi204:> sudo pkill -f 'journalctl -f -n 0 -u ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715@osd.2.service' 2024-03-08T21:17:07.232 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-08T21:17:07.232 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-03-08T21:17:07.232 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-03-08T21:17:07.232 DEBUG:teuthology.orchestra.run.smithi107:> sudo systemctl stop ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715@osd.3 2024-03-08T21:17:07.257 INFO:journalctl@ceph.osd.4.smithi149.stdout:Mar 08 21:17:07 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-4[46289]: 2024-03-08T21:17:07.138+0000 7f297fc00700 -1 osd.4 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:41.916650+0000 front 2024-03-08T21:16:41.917178+0000 (oldest deadline 2024-03-08T21:17:06.016911+0000) 2024-03-08T21:17:07.265 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:17:06 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:17:06.967+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:44.858572+0000 front 2024-03-08T21:16:44.858829+0000 (oldest deadline 2024-03-08T21:17:06.558274+0000) 2024-03-08T21:17:07.710 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:17:07 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:17:07.249+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:40.479965+0000 front 2024-03-08T21:16:40.480241+0000 (oldest deadline 2024-03-08T21:17:06.379830+0000) 2024-03-08T21:17:07.710 INFO:journalctl@ceph.osd.3.smithi107.stdout:Mar 08 21:17:07 smithi107 systemd[1]: Stopping Ceph osd.3 for dc20f0f8-dd8c-11ee-95c6-87774f69a715... 2024-03-08T21:17:08.210 INFO:journalctl@ceph.osd.3.smithi107.stdout:Mar 08 21:17:07 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-3[53018]: 2024-03-08T21:17:07.820+0000 7fd3f2e00700 -1 osd.3 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:42.809095+0000 front 2024-03-08T21:16:42.810267+0000 (oldest deadline 2024-03-08T21:17:07.509217+0000) 2024-03-08T21:17:08.211 INFO:journalctl@ceph.osd.3.smithi107.stdout:Mar 08 21:17:07 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-3[53018]: 2024-03-08T21:17:07.827+0000 7fd3fc600700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-08T21:17:08.211 INFO:journalctl@ceph.osd.3.smithi107.stdout:Mar 08 21:17:07 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-3[53018]: 2024-03-08T21:17:07.827+0000 7fd3fc600700 -1 osd.3 115 *** Got signal Terminated *** 2024-03-08T21:17:08.211 INFO:journalctl@ceph.osd.3.smithi107.stdout:Mar 08 21:17:07 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-3[53018]: 2024-03-08T21:17:07.827+0000 7fd3fc600700 -1 osd.3 115 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-08T21:17:08.258 INFO:journalctl@ceph.osd.4.smithi149.stdout:Mar 08 21:17:08 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-4[46289]: 2024-03-08T21:17:08.172+0000 7f297fc00700 -1 osd.4 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:41.916650+0000 front 2024-03-08T21:16:41.917178+0000 (oldest deadline 2024-03-08T21:17:06.016911+0000) 2024-03-08T21:17:08.258 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:17:07 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:17:07.955+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:44.858572+0000 front 2024-03-08T21:16:44.858829+0000 (oldest deadline 2024-03-08T21:17:06.558274+0000) 2024-03-08T21:17:08.710 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:17:08 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:17:08.257+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:40.479965+0000 front 2024-03-08T21:16:40.480241+0000 (oldest deadline 2024-03-08T21:17:06.379830+0000) 2024-03-08T21:17:08.713 INFO:journalctl@ceph.osd.6.smithi107.stdout:Mar 08 21:17:08 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-6[59834]: 2024-03-08T21:17:08.331+0000 7f6132600700 -1 osd.6 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:44.641569+0000 front 2024-03-08T21:16:44.641810+0000 (oldest deadline 2024-03-08T21:17:07.541743+0000) 2024-03-08T21:17:08.806 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:17:08 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:17:08.472+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:42.614355+0000 front 2024-03-08T21:16:42.614271+0000 (oldest deadline 2024-03-08T21:17:07.914284+0000) 2024-03-08T21:17:09.121 INFO:journalctl@ceph.osd.5.smithi204.stdout:Mar 08 21:17:08 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-5[47140]: 2024-03-08T21:17:08.830+0000 7f038c200700 -1 osd.5 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:43.793658+0000 front 2024-03-08T21:16:43.793735+0000 (oldest deadline 2024-03-08T21:17:07.893906+0000) 2024-03-08T21:17:09.209 INFO:journalctl@ceph.osd.3.smithi107.stdout:Mar 08 21:17:08 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-3[53018]: 2024-03-08T21:17:08.797+0000 7fd3f2e00700 -1 osd.3 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:42.809095+0000 front 2024-03-08T21:16:42.810267+0000 (oldest deadline 2024-03-08T21:17:07.509217+0000) 2024-03-08T21:17:09.258 INFO:journalctl@ceph.osd.4.smithi149.stdout:Mar 08 21:17:09 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-4[46289]: 2024-03-08T21:17:09.169+0000 7f297fc00700 -1 osd.4 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:41.916650+0000 front 2024-03-08T21:16:41.917178+0000 (oldest deadline 2024-03-08T21:17:06.016911+0000) 2024-03-08T21:17:09.258 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:17:08 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:17:08.954+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:44.858572+0000 front 2024-03-08T21:16:44.858829+0000 (oldest deadline 2024-03-08T21:17:06.558274+0000) 2024-03-08T21:17:09.460 INFO:journalctl@ceph.osd.6.smithi107.stdout:Mar 08 21:17:09 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-6[59834]: 2024-03-08T21:17:09.288+0000 7f6132600700 -1 osd.6 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:44.641569+0000 front 2024-03-08T21:16:44.641810+0000 (oldest deadline 2024-03-08T21:17:07.541743+0000) 2024-03-08T21:17:09.460 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:17:09 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:17:09.233+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:40.479965+0000 front 2024-03-08T21:16:40.480241+0000 (oldest deadline 2024-03-08T21:17:06.379830+0000) 2024-03-08T21:17:09.772 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:17:09 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:17:09.422+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:42.614355+0000 front 2024-03-08T21:16:42.614271+0000 (oldest deadline 2024-03-08T21:17:07.914284+0000) 2024-03-08T21:17:10.121 INFO:journalctl@ceph.osd.5.smithi204.stdout:Mar 08 21:17:09 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-5[47140]: 2024-03-08T21:17:09.796+0000 7f038c200700 -1 osd.5 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:43.793658+0000 front 2024-03-08T21:16:43.793735+0000 (oldest deadline 2024-03-08T21:17:07.893906+0000) 2024-03-08T21:17:10.210 INFO:journalctl@ceph.osd.3.smithi107.stdout:Mar 08 21:17:09 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-3[53018]: 2024-03-08T21:17:09.810+0000 7fd3f2e00700 -1 osd.3 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:42.809095+0000 front 2024-03-08T21:16:42.810267+0000 (oldest deadline 2024-03-08T21:17:07.509217+0000) 2024-03-08T21:17:10.258 INFO:journalctl@ceph.osd.4.smithi149.stdout:Mar 08 21:17:10 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-4[46289]: 2024-03-08T21:17:10.178+0000 7f297fc00700 -1 osd.4 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:41.916650+0000 front 2024-03-08T21:16:41.917178+0000 (oldest deadline 2024-03-08T21:17:06.016911+0000) 2024-03-08T21:17:10.258 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:17:09 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:17:09.945+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:44.858572+0000 front 2024-03-08T21:16:44.858829+0000 (oldest deadline 2024-03-08T21:17:06.558274+0000) 2024-03-08T21:17:10.621 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:17:10 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:17:10.455+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:42.614355+0000 front 2024-03-08T21:16:42.614271+0000 (oldest deadline 2024-03-08T21:17:07.914284+0000) 2024-03-08T21:17:10.621 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 08 21:17:10 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-8[53709]: 2024-03-08T21:17:10.275+0000 7f3fae200700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:43.996391+0000 front 2024-03-08T21:16:43.996418+0000 (oldest deadline 2024-03-08T21:17:09.296545+0000) 2024-03-08T21:17:10.710 INFO:journalctl@ceph.osd.6.smithi107.stdout:Mar 08 21:17:10 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-6[59834]: 2024-03-08T21:17:10.303+0000 7f6132600700 -1 osd.6 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:44.641569+0000 front 2024-03-08T21:16:44.641810+0000 (oldest deadline 2024-03-08T21:17:07.541743+0000) 2024-03-08T21:17:10.710 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:17:10 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:17:10.281+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:40.479965+0000 front 2024-03-08T21:16:40.480241+0000 (oldest deadline 2024-03-08T21:17:06.379830+0000) 2024-03-08T21:17:11.121 INFO:journalctl@ceph.osd.5.smithi204.stdout:Mar 08 21:17:10 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-5[47140]: 2024-03-08T21:17:10.755+0000 7f038c200700 -1 osd.5 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:43.793658+0000 front 2024-03-08T21:16:43.793735+0000 (oldest deadline 2024-03-08T21:17:07.893906+0000) 2024-03-08T21:17:11.183 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:17:10 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:17:10.902+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:44.858572+0000 front 2024-03-08T21:16:44.858829+0000 (oldest deadline 2024-03-08T21:17:06.558274+0000) 2024-03-08T21:17:11.210 INFO:journalctl@ceph.osd.3.smithi107.stdout:Mar 08 21:17:10 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-3[53018]: 2024-03-08T21:17:10.859+0000 7fd3f2e00700 -1 osd.3 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:42.809095+0000 front 2024-03-08T21:16:42.810267+0000 (oldest deadline 2024-03-08T21:17:07.509217+0000) 2024-03-08T21:17:11.508 INFO:journalctl@ceph.osd.4.smithi149.stdout:Mar 08 21:17:11 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-4[46289]: 2024-03-08T21:17:11.207+0000 7f297fc00700 -1 osd.4 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:41.916650+0000 front 2024-03-08T21:16:41.917178+0000 (oldest deadline 2024-03-08T21:17:06.016911+0000) 2024-03-08T21:17:11.621 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:17:11 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:17:11.463+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:42.614355+0000 front 2024-03-08T21:16:42.614271+0000 (oldest deadline 2024-03-08T21:17:07.914284+0000) 2024-03-08T21:17:11.621 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 08 21:17:11 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-8[53709]: 2024-03-08T21:17:11.298+0000 7f3fae200700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:43.996391+0000 front 2024-03-08T21:16:43.996418+0000 (oldest deadline 2024-03-08T21:17:09.296545+0000) 2024-03-08T21:17:11.710 INFO:journalctl@ceph.osd.6.smithi107.stdout:Mar 08 21:17:11 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-6[59834]: 2024-03-08T21:17:11.308+0000 7f6132600700 -1 osd.6 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:44.641569+0000 front 2024-03-08T21:16:44.641810+0000 (oldest deadline 2024-03-08T21:17:07.541743+0000) 2024-03-08T21:17:11.710 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:17:11 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:17:11.239+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:40.479965+0000 front 2024-03-08T21:16:40.480241+0000 (oldest deadline 2024-03-08T21:17:06.379830+0000) 2024-03-08T21:17:12.121 INFO:journalctl@ceph.osd.5.smithi204.stdout:Mar 08 21:17:11 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-5[47140]: 2024-03-08T21:17:11.731+0000 7f038c200700 -1 osd.5 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:43.793658+0000 front 2024-03-08T21:16:43.793735+0000 (oldest deadline 2024-03-08T21:17:07.893906+0000) 2024-03-08T21:17:12.210 INFO:journalctl@ceph.osd.3.smithi107.stdout:Mar 08 21:17:11 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-3[53018]: 2024-03-08T21:17:11.857+0000 7fd3f2e00700 -1 osd.3 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:42.809095+0000 front 2024-03-08T21:16:42.810267+0000 (oldest deadline 2024-03-08T21:17:07.509217+0000) 2024-03-08T21:17:12.229 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:17:11 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:17:11.947+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:44.858572+0000 front 2024-03-08T21:16:44.858829+0000 (oldest deadline 2024-03-08T21:17:06.558274+0000) 2024-03-08T21:17:12.508 INFO:journalctl@ceph.osd.4.smithi149.stdout:Mar 08 21:17:12 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-4[46289]: 2024-03-08T21:17:12.253+0000 7f297fc00700 -1 osd.4 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:41.916650+0000 front 2024-03-08T21:16:41.917178+0000 (oldest deadline 2024-03-08T21:17:06.016911+0000) 2024-03-08T21:17:12.621 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:17:12 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:17:12.427+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:42.614355+0000 front 2024-03-08T21:16:42.614271+0000 (oldest deadline 2024-03-08T21:17:07.914284+0000) 2024-03-08T21:17:12.621 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 08 21:17:12 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-8[53709]: 2024-03-08T21:17:12.269+0000 7f3fae200700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:43.996391+0000 front 2024-03-08T21:16:43.996418+0000 (oldest deadline 2024-03-08T21:17:09.296545+0000) 2024-03-08T21:17:12.710 INFO:journalctl@ceph.osd.6.smithi107.stdout:Mar 08 21:17:12 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-6[59834]: 2024-03-08T21:17:12.329+0000 7f6132600700 -1 osd.6 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:44.641569+0000 front 2024-03-08T21:16:44.641810+0000 (oldest deadline 2024-03-08T21:17:07.541743+0000) 2024-03-08T21:17:12.710 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:17:12 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:17:12.247+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:40.479965+0000 front 2024-03-08T21:16:40.480241+0000 (oldest deadline 2024-03-08T21:17:06.379830+0000) 2024-03-08T21:17:13.121 INFO:journalctl@ceph.osd.5.smithi204.stdout:Mar 08 21:17:12 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-5[47140]: 2024-03-08T21:17:12.712+0000 7f038c200700 -1 osd.5 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:43.793658+0000 front 2024-03-08T21:16:43.793735+0000 (oldest deadline 2024-03-08T21:17:07.893906+0000) 2024-03-08T21:17:13.192 INFO:journalctl@ceph.osd.3.smithi107.stdout:Mar 08 21:17:12 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-3[53018]: 2024-03-08T21:17:12.827+0000 7fd3f2e00700 -1 osd.3 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:42.809095+0000 front 2024-03-08T21:16:42.810267+0000 (oldest deadline 2024-03-08T21:17:07.509217+0000) 2024-03-08T21:17:13.257 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:17:12 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:17:12.930+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:44.858572+0000 front 2024-03-08T21:16:44.858829+0000 (oldest deadline 2024-03-08T21:17:06.558274+0000) 2024-03-08T21:17:13.460 INFO:journalctl@ceph.osd.6.smithi107.stdout:Mar 08 21:17:13 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-6[59834]: 2024-03-08T21:17:13.348+0000 7f6132600700 -1 osd.6 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:44.641569+0000 front 2024-03-08T21:16:44.641810+0000 (oldest deadline 2024-03-08T21:17:07.541743+0000) 2024-03-08T21:17:13.460 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:17:13 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:17:13.216+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:40.479965+0000 front 2024-03-08T21:16:40.480241+0000 (oldest deadline 2024-03-08T21:17:06.379830+0000) 2024-03-08T21:17:13.461 INFO:journalctl@ceph.osd.3.smithi107.stdout:Mar 08 21:17:13 smithi107 podman[95309]: 2024-03-08 21:17:13.228852849 +0000 UTC m=+5.537578779 container died bb1b3a2897bc91b546597d66b90a32acb38a57c43e6ffc23a1b350dc0f489855 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-3, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=rishabh-2024mar8-7d63482, maintainer=Guillaume Abrioux ) 2024-03-08T21:17:13.621 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 08 21:17:13 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-8[53709]: 2024-03-08T21:17:13.238+0000 7f3fae200700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:43.996391+0000 front 2024-03-08T21:16:43.996418+0000 (oldest deadline 2024-03-08T21:17:09.296545+0000) 2024-03-08T21:17:13.621 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:17:13 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:17:13.469+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:42.614355+0000 front 2024-03-08T21:16:42.614271+0000 (oldest deadline 2024-03-08T21:17:07.914284+0000) 2024-03-08T21:17:13.758 INFO:journalctl@ceph.osd.4.smithi149.stdout:Mar 08 21:17:13 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-4[46289]: 2024-03-08T21:17:13.294+0000 7f297fc00700 -1 osd.4 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:41.916650+0000 front 2024-03-08T21:16:41.917178+0000 (oldest deadline 2024-03-08T21:17:06.016911+0000) 2024-03-08T21:17:13.897 INFO:journalctl@ceph.osd.3.smithi107.stdout:Mar 08 21:17:13 smithi107 podman[95309]: 2024-03-08 21:17:13.678307188 +0000 UTC m=+5.987033162 container cleanup bb1b3a2897bc91b546597d66b90a32acb38a57c43e6ffc23a1b350dc0f489855 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-3, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, RELEASE=rishabh-2024mar8-7d63482, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-03-08T21:17:13.897 INFO:journalctl@ceph.osd.3.smithi107.stdout:Mar 08 21:17:13 smithi107 bash[95309]: ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-3 2024-03-08T21:17:14.121 INFO:journalctl@ceph.osd.5.smithi204.stdout:Mar 08 21:17:13 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-5[47140]: 2024-03-08T21:17:13.725+0000 7f038c200700 -1 osd.5 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:43.793658+0000 front 2024-03-08T21:16:43.793735+0000 (oldest deadline 2024-03-08T21:17:07.893906+0000) 2024-03-08T21:17:14.199 INFO:journalctl@ceph.osd.3.smithi107.stdout:Mar 08 21:17:13 smithi107 podman[95330]: 2024-03-08 21:17:13.921438865 +0000 UTC m=+0.680877966 container remove bb1b3a2897bc91b546597d66b90a32acb38a57c43e6ffc23a1b350dc0f489855 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-3, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, RELEASE=rishabh-2024mar8-7d63482, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-08T21:17:14.258 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:17:13 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:17:13.939+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:44.858572+0000 front 2024-03-08T21:16:44.858829+0000 (oldest deadline 2024-03-08T21:17:06.558274+0000) 2024-03-08T21:17:14.460 INFO:journalctl@ceph.osd.6.smithi107.stdout:Mar 08 21:17:14 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-6[59834]: 2024-03-08T21:17:14.299+0000 7f6132600700 -1 osd.6 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:44.641569+0000 front 2024-03-08T21:16:44.641810+0000 (oldest deadline 2024-03-08T21:17:07.541743+0000) 2024-03-08T21:17:14.460 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:17:14 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:17:14.223+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:40.479965+0000 front 2024-03-08T21:16:40.480241+0000 (oldest deadline 2024-03-08T21:17:06.379830+0000) 2024-03-08T21:17:14.621 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 08 21:17:14 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-8[53709]: 2024-03-08T21:17:14.279+0000 7f3fae200700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:43.996391+0000 front 2024-03-08T21:16:43.996418+0000 (oldest deadline 2024-03-08T21:17:09.296545+0000) 2024-03-08T21:17:14.621 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:17:14 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:17:14.469+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:42.614355+0000 front 2024-03-08T21:16:42.614271+0000 (oldest deadline 2024-03-08T21:17:07.914284+0000) 2024-03-08T21:17:14.758 INFO:journalctl@ceph.osd.4.smithi149.stdout:Mar 08 21:17:14 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-4[46289]: 2024-03-08T21:17:14.293+0000 7f297fc00700 -1 osd.4 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:41.916650+0000 front 2024-03-08T21:16:41.917178+0000 (oldest deadline 2024-03-08T21:17:06.016911+0000) 2024-03-08T21:17:15.061 INFO:journalctl@ceph.osd.3.smithi107.stdout:Mar 08 21:17:14 smithi107 podman[95417]: 2024-03-08 21:17:14.669837744 +0000 UTC m=+0.034684274 image pull a272146cb810b1fd4a53627f019472132e30966d5c25b01bfc44a001295f1249 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd 2024-03-08T21:17:15.062 INFO:journalctl@ceph.osd.3.smithi107.stdout:Mar 08 21:17:14 smithi107 podman[95417]: 2024-03-08 21:17:14.788809202 +0000 UTC m=+0.153655729 container create 415f0df95fa6235d181d24c597bbeac1355fbc07e652c011e7ab474672a1cbd8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-3-deactivate, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=rishabh-2024mar8-7d63482) 2024-03-08T21:17:15.062 INFO:journalctl@ceph.osd.3.smithi107.stdout:Mar 08 21:17:15 smithi107 podman[95417]: 2024-03-08 21:17:15.084138244 +0000 UTC m=+0.448984815 container init 415f0df95fa6235d181d24c597bbeac1355fbc07e652c011e7ab474672a1cbd8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-3-deactivate, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, RELEASE=rishabh-2024mar8-7d63482, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-03-08T21:17:15.121 INFO:journalctl@ceph.osd.5.smithi204.stdout:Mar 08 21:17:14 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-5[47140]: 2024-03-08T21:17:14.775+0000 7f038c200700 -1 osd.5 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:43.793658+0000 front 2024-03-08T21:16:43.793735+0000 (oldest deadline 2024-03-08T21:17:07.893906+0000) 2024-03-08T21:17:15.258 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:17:14 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:17:14.952+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:44.858572+0000 front 2024-03-08T21:16:44.858829+0000 (oldest deadline 2024-03-08T21:17:06.558274+0000) 2024-03-08T21:17:15.460 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:17:15 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:17:15.192+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:40.479965+0000 front 2024-03-08T21:16:40.480241+0000 (oldest deadline 2024-03-08T21:17:06.379830+0000) 2024-03-08T21:17:15.460 INFO:journalctl@ceph.osd.3.smithi107.stdout:Mar 08 21:17:15 smithi107 podman[95417]: 2024-03-08 21:17:15.09313557 +0000 UTC m=+0.457982162 container start 415f0df95fa6235d181d24c597bbeac1355fbc07e652c011e7ab474672a1cbd8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-3-deactivate, GIT_CLEAN=True, RELEASE=rishabh-2024mar8-7d63482, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=) 2024-03-08T21:17:15.460 INFO:journalctl@ceph.osd.3.smithi107.stdout:Mar 08 21:17:15 smithi107 podman[95417]: 2024-03-08 21:17:15.216503911 +0000 UTC m=+0.581350519 container attach 415f0df95fa6235d181d24c597bbeac1355fbc07e652c011e7ab474672a1cbd8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-3-deactivate, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, ceph=True, RELEASE=rishabh-2024mar8-7d63482, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-08T21:17:15.461 INFO:journalctl@ceph.osd.6.smithi107.stdout:Mar 08 21:17:15 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-6[59834]: 2024-03-08T21:17:15.253+0000 7f6132600700 -1 osd.6 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:44.641569+0000 front 2024-03-08T21:16:44.641810+0000 (oldest deadline 2024-03-08T21:17:07.541743+0000) 2024-03-08T21:17:15.621 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 08 21:17:15 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-8[53709]: 2024-03-08T21:17:15.251+0000 7f3fae200700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:43.996391+0000 front 2024-03-08T21:16:43.996418+0000 (oldest deadline 2024-03-08T21:17:09.296545+0000) 2024-03-08T21:17:15.621 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:17:15 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:17:15.489+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:42.614355+0000 front 2024-03-08T21:16:42.614271+0000 (oldest deadline 2024-03-08T21:17:07.914284+0000) 2024-03-08T21:17:15.757 INFO:journalctl@ceph.osd.4.smithi149.stdout:Mar 08 21:17:15 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-4[46289]: 2024-03-08T21:17:15.319+0000 7f297fc00700 -1 osd.4 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:41.916650+0000 front 2024-03-08T21:16:41.917178+0000 (oldest deadline 2024-03-08T21:17:06.016911+0000) 2024-03-08T21:17:15.850 INFO:journalctl@ceph.osd.3.smithi107.stdout:Mar 08 21:17:15 smithi107 podman[95417]: 2024-03-08 21:17:15.515537056 +0000 UTC m=+0.880383608 container died 415f0df95fa6235d181d24c597bbeac1355fbc07e652c011e7ab474672a1cbd8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-3-deactivate, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=rishabh-2024mar8-7d63482, io.buildah.version=1.33.5, org.label-schema.build-date=20240307) 2024-03-08T21:17:16.121 INFO:journalctl@ceph.osd.5.smithi204.stdout:Mar 08 21:17:15 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-5[47140]: 2024-03-08T21:17:15.771+0000 7f038c200700 -1 osd.5 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:43.793658+0000 front 2024-03-08T21:16:43.793735+0000 (oldest deadline 2024-03-08T21:17:07.893906+0000) 2024-03-08T21:17:16.121 INFO:journalctl@ceph.osd.5.smithi204.stdout:Mar 08 21:17:15 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-5[47140]: 2024-03-08T21:17:15.771+0000 7f038c200700 -1 osd.5 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:53.697831+0000 front 2024-03-08T21:16:53.697757+0000 (oldest deadline 2024-03-08T21:17:14.797782+0000) 2024-03-08T21:17:16.258 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:17:15 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:17:15.977+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:44.858572+0000 front 2024-03-08T21:16:44.858829+0000 (oldest deadline 2024-03-08T21:17:06.558274+0000) 2024-03-08T21:17:16.302 DEBUG:teuthology.orchestra.run.smithi107:> sudo pkill -f 'journalctl -f -n 0 -u ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715@osd.3.service' 2024-03-08T21:17:16.396 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-08T21:17:16.396 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-03-08T21:17:16.396 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-03-08T21:17:16.396 DEBUG:teuthology.orchestra.run.smithi149:> sudo systemctl stop ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715@osd.4 2024-03-08T21:17:16.460 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:17:16 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:17:16.206+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:40.479965+0000 front 2024-03-08T21:16:40.480241+0000 (oldest deadline 2024-03-08T21:17:06.379830+0000) 2024-03-08T21:17:16.460 INFO:journalctl@ceph.osd.6.smithi107.stdout:Mar 08 21:17:16 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-6[59834]: 2024-03-08T21:17:16.228+0000 7f6132600700 -1 osd.6 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:44.641569+0000 front 2024-03-08T21:16:44.641810+0000 (oldest deadline 2024-03-08T21:17:07.541743+0000) 2024-03-08T21:17:16.461 INFO:journalctl@ceph.osd.6.smithi107.stdout:Mar 08 21:17:16 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-6[59834]: 2024-03-08T21:17:16.228+0000 7f6132600700 -1 osd.6 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:53.344721+0000 front 2024-03-08T21:16:53.344770+0000 (oldest deadline 2024-03-08T21:17:15.644922+0000) 2024-03-08T21:17:16.621 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 08 21:17:16 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-8[53709]: 2024-03-08T21:17:16.229+0000 7f3fae200700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:43.996391+0000 front 2024-03-08T21:16:43.996418+0000 (oldest deadline 2024-03-08T21:17:09.296545+0000) 2024-03-08T21:17:16.621 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:17:16 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:17:16.464+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:42.614355+0000 front 2024-03-08T21:16:42.614271+0000 (oldest deadline 2024-03-08T21:17:07.914284+0000) 2024-03-08T21:17:16.621 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:17:16 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:17:16.464+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:52.016503+0000 front 2024-03-08T21:16:52.017017+0000 (oldest deadline 2024-03-08T21:17:16.116550+0000) 2024-03-08T21:17:16.758 INFO:journalctl@ceph.osd.4.smithi149.stdout:Mar 08 21:17:16 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-4[46289]: 2024-03-08T21:17:16.328+0000 7f297fc00700 -1 osd.4 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:41.916650+0000 front 2024-03-08T21:16:41.917178+0000 (oldest deadline 2024-03-08T21:17:06.016911+0000) 2024-03-08T21:17:16.758 INFO:journalctl@ceph.osd.4.smithi149.stdout:Mar 08 21:17:16 smithi149 systemd[1]: Stopping Ceph osd.4 for dc20f0f8-dd8c-11ee-95c6-87774f69a715... 2024-03-08T21:17:17.121 INFO:journalctl@ceph.osd.5.smithi204.stdout:Mar 08 21:17:16 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-5[47140]: 2024-03-08T21:17:16.808+0000 7f038c200700 -1 osd.5 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:43.793658+0000 front 2024-03-08T21:16:43.793735+0000 (oldest deadline 2024-03-08T21:17:07.893906+0000) 2024-03-08T21:17:17.121 INFO:journalctl@ceph.osd.5.smithi204.stdout:Mar 08 21:17:16 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-5[47140]: 2024-03-08T21:17:16.808+0000 7f038c200700 -1 osd.5 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:53.697831+0000 front 2024-03-08T21:16:53.697757+0000 (oldest deadline 2024-03-08T21:17:14.797782+0000) 2024-03-08T21:17:17.258 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:17:16 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:17:16.944+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:44.858572+0000 front 2024-03-08T21:16:44.858829+0000 (oldest deadline 2024-03-08T21:17:06.558274+0000) 2024-03-08T21:17:17.258 INFO:journalctl@ceph.osd.4.smithi149.stdout:Mar 08 21:17:16 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-4[46289]: 2024-03-08T21:17:16.994+0000 7f2989400700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-08T21:17:17.258 INFO:journalctl@ceph.osd.4.smithi149.stdout:Mar 08 21:17:16 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-4[46289]: 2024-03-08T21:17:16.994+0000 7f2989400700 -1 osd.4 115 *** Got signal Terminated *** 2024-03-08T21:17:17.258 INFO:journalctl@ceph.osd.4.smithi149.stdout:Mar 08 21:17:16 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-4[46289]: 2024-03-08T21:17:16.994+0000 7f2989400700 -1 osd.4 115 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-08T21:17:17.460 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:17:17 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:17:17.216+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:40.479965+0000 front 2024-03-08T21:16:40.480241+0000 (oldest deadline 2024-03-08T21:17:06.379830+0000) 2024-03-08T21:17:17.460 INFO:journalctl@ceph.osd.6.smithi107.stdout:Mar 08 21:17:17 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-6[59834]: 2024-03-08T21:17:17.275+0000 7f6132600700 -1 osd.6 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:44.641569+0000 front 2024-03-08T21:16:44.641810+0000 (oldest deadline 2024-03-08T21:17:07.541743+0000) 2024-03-08T21:17:17.461 INFO:journalctl@ceph.osd.6.smithi107.stdout:Mar 08 21:17:17 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-6[59834]: 2024-03-08T21:17:17.275+0000 7f6132600700 -1 osd.6 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:53.344721+0000 front 2024-03-08T21:16:53.344770+0000 (oldest deadline 2024-03-08T21:17:15.644922+0000) 2024-03-08T21:17:17.621 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 08 21:17:17 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-8[53709]: 2024-03-08T21:17:17.220+0000 7f3fae200700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:43.996391+0000 front 2024-03-08T21:16:43.996418+0000 (oldest deadline 2024-03-08T21:17:09.296545+0000) 2024-03-08T21:17:17.621 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 08 21:17:17 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-8[53709]: 2024-03-08T21:17:17.220+0000 7f3fae200700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:52.199334+0000 front 2024-03-08T21:16:52.199113+0000 (oldest deadline 2024-03-08T21:17:16.899180+0000) 2024-03-08T21:17:17.621 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:17:17 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:17:17.441+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:42.614355+0000 front 2024-03-08T21:16:42.614271+0000 (oldest deadline 2024-03-08T21:17:07.914284+0000) 2024-03-08T21:17:17.622 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:17:17 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:17:17.441+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:52.016503+0000 front 2024-03-08T21:16:52.017017+0000 (oldest deadline 2024-03-08T21:17:16.116550+0000) 2024-03-08T21:17:17.758 INFO:journalctl@ceph.osd.4.smithi149.stdout:Mar 08 21:17:17 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-4[46289]: 2024-03-08T21:17:17.342+0000 7f297fc00700 -1 osd.4 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:41.916650+0000 front 2024-03-08T21:16:41.917178+0000 (oldest deadline 2024-03-08T21:17:06.016911+0000) 2024-03-08T21:17:18.121 INFO:journalctl@ceph.osd.5.smithi204.stdout:Mar 08 21:17:17 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-5[47140]: 2024-03-08T21:17:17.769+0000 7f038c200700 -1 osd.5 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:43.793658+0000 front 2024-03-08T21:16:43.793735+0000 (oldest deadline 2024-03-08T21:17:07.893906+0000) 2024-03-08T21:17:18.121 INFO:journalctl@ceph.osd.5.smithi204.stdout:Mar 08 21:17:17 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-5[47140]: 2024-03-08T21:17:17.769+0000 7f038c200700 -1 osd.5 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:53.697831+0000 front 2024-03-08T21:16:53.697757+0000 (oldest deadline 2024-03-08T21:17:14.797782+0000) 2024-03-08T21:17:18.258 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:17:17 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:17:17.970+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:44.858572+0000 front 2024-03-08T21:16:44.858829+0000 (oldest deadline 2024-03-08T21:17:06.558274+0000) 2024-03-08T21:17:18.520 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 08 21:17:18 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-8[53709]: 2024-03-08T21:17:18.257+0000 7f3fae200700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:43.996391+0000 front 2024-03-08T21:16:43.996418+0000 (oldest deadline 2024-03-08T21:17:09.296545+0000) 2024-03-08T21:17:18.520 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 08 21:17:18 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-8[53709]: 2024-03-08T21:17:18.257+0000 7f3fae200700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:52.199334+0000 front 2024-03-08T21:16:52.199113+0000 (oldest deadline 2024-03-08T21:17:16.899180+0000) 2024-03-08T21:17:18.520 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:17:18 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:17:18.426+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:42.614355+0000 front 2024-03-08T21:16:42.614271+0000 (oldest deadline 2024-03-08T21:17:07.914284+0000) 2024-03-08T21:17:18.520 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:17:18 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:17:18.426+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:52.016503+0000 front 2024-03-08T21:16:52.017017+0000 (oldest deadline 2024-03-08T21:17:16.116550+0000) 2024-03-08T21:17:18.629 INFO:journalctl@ceph.osd.4.smithi149.stdout:Mar 08 21:17:18 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-4[46289]: 2024-03-08T21:17:18.372+0000 7f297fc00700 -1 osd.4 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:41.916650+0000 front 2024-03-08T21:16:41.917178+0000 (oldest deadline 2024-03-08T21:17:06.016911+0000) 2024-03-08T21:17:18.710 INFO:journalctl@ceph.osd.6.smithi107.stdout:Mar 08 21:17:18 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-6[59834]: 2024-03-08T21:17:18.308+0000 7f6132600700 -1 osd.6 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:44.641569+0000 front 2024-03-08T21:16:44.641810+0000 (oldest deadline 2024-03-08T21:17:07.541743+0000) 2024-03-08T21:17:18.710 INFO:journalctl@ceph.osd.6.smithi107.stdout:Mar 08 21:17:18 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-6[59834]: 2024-03-08T21:17:18.308+0000 7f6132600700 -1 osd.6 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:53.344721+0000 front 2024-03-08T21:16:53.344770+0000 (oldest deadline 2024-03-08T21:17:15.644922+0000) 2024-03-08T21:17:18.711 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:17:18 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:17:18.251+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:40.479965+0000 front 2024-03-08T21:16:40.480241+0000 (oldest deadline 2024-03-08T21:17:06.379830+0000) 2024-03-08T21:17:18.711 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:17:18 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:17:18.251+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:53.283252+0000 front 2024-03-08T21:16:53.283300+0000 (oldest deadline 2024-03-08T21:17:17.383498+0000) 2024-03-08T21:17:18.937 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 08 21:17:18 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-7[53366]: 2024-03-08T21:17:18.653+0000 7f48e9a00700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:53.119608+0000 front 2024-03-08T21:16:53.119745+0000 (oldest deadline 2024-03-08T21:17:17.819760+0000) 2024-03-08T21:17:19.121 INFO:journalctl@ceph.osd.5.smithi204.stdout:Mar 08 21:17:18 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-5[47140]: 2024-03-08T21:17:18.804+0000 7f038c200700 -1 osd.5 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:43.793658+0000 front 2024-03-08T21:16:43.793735+0000 (oldest deadline 2024-03-08T21:17:07.893906+0000) 2024-03-08T21:17:19.121 INFO:journalctl@ceph.osd.5.smithi204.stdout:Mar 08 21:17:18 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-5[47140]: 2024-03-08T21:17:18.804+0000 7f038c200700 -1 osd.5 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:53.697831+0000 front 2024-03-08T21:16:53.697757+0000 (oldest deadline 2024-03-08T21:17:14.797782+0000) 2024-03-08T21:17:19.258 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:17:18 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:17:18.960+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:44.858572+0000 front 2024-03-08T21:16:44.858829+0000 (oldest deadline 2024-03-08T21:17:06.558274+0000) 2024-03-08T21:17:19.258 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:17:18 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:17:18.960+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:52.961888+0000 front 2024-03-08T21:16:52.961762+0000 (oldest deadline 2024-03-08T21:17:18.862022+0000) 2024-03-08T21:17:19.460 INFO:journalctl@ceph.osd.6.smithi107.stdout:Mar 08 21:17:19 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-6[59834]: 2024-03-08T21:17:19.306+0000 7f6132600700 -1 osd.6 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:44.641569+0000 front 2024-03-08T21:16:44.641810+0000 (oldest deadline 2024-03-08T21:17:07.541743+0000) 2024-03-08T21:17:19.460 INFO:journalctl@ceph.osd.6.smithi107.stdout:Mar 08 21:17:19 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-6[59834]: 2024-03-08T21:17:19.306+0000 7f6132600700 -1 osd.6 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:53.344721+0000 front 2024-03-08T21:16:53.344770+0000 (oldest deadline 2024-03-08T21:17:15.644922+0000) 2024-03-08T21:17:19.461 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:17:19 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:17:19.227+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:40.479965+0000 front 2024-03-08T21:16:40.480241+0000 (oldest deadline 2024-03-08T21:17:06.379830+0000) 2024-03-08T21:17:19.461 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:17:19 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:17:19.227+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:53.283252+0000 front 2024-03-08T21:16:53.283300+0000 (oldest deadline 2024-03-08T21:17:17.383498+0000) 2024-03-08T21:17:19.621 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 08 21:17:19 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-8[53709]: 2024-03-08T21:17:19.297+0000 7f3fae200700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:43.996391+0000 front 2024-03-08T21:16:43.996418+0000 (oldest deadline 2024-03-08T21:17:09.296545+0000) 2024-03-08T21:17:19.621 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 08 21:17:19 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-8[53709]: 2024-03-08T21:17:19.297+0000 7f3fae200700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:52.199334+0000 front 2024-03-08T21:16:52.199113+0000 (oldest deadline 2024-03-08T21:17:16.899180+0000) 2024-03-08T21:17:19.621 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:17:19 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:17:19.397+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:42.614355+0000 front 2024-03-08T21:16:42.614271+0000 (oldest deadline 2024-03-08T21:17:07.914284+0000) 2024-03-08T21:17:19.622 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:17:19 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:17:19.397+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:52.016503+0000 front 2024-03-08T21:16:52.017017+0000 (oldest deadline 2024-03-08T21:17:16.116550+0000) 2024-03-08T21:17:19.666 INFO:journalctl@ceph.osd.4.smithi149.stdout:Mar 08 21:17:19 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-4[46289]: 2024-03-08T21:17:19.423+0000 7f297fc00700 -1 osd.4 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:41.916650+0000 front 2024-03-08T21:16:41.917178+0000 (oldest deadline 2024-03-08T21:17:06.016911+0000) 2024-03-08T21:17:20.008 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 08 21:17:19 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-7[53366]: 2024-03-08T21:17:19.690+0000 7f48e9a00700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:53.119608+0000 front 2024-03-08T21:16:53.119745+0000 (oldest deadline 2024-03-08T21:17:17.819760+0000) 2024-03-08T21:17:20.008 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:17:19 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:17:19.933+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:44.858572+0000 front 2024-03-08T21:16:44.858829+0000 (oldest deadline 2024-03-08T21:17:06.558274+0000) 2024-03-08T21:17:20.008 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:17:19 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:17:19.933+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:52.961888+0000 front 2024-03-08T21:16:52.961762+0000 (oldest deadline 2024-03-08T21:17:18.862022+0000) 2024-03-08T21:17:20.121 INFO:journalctl@ceph.osd.5.smithi204.stdout:Mar 08 21:17:19 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-5[47140]: 2024-03-08T21:17:19.829+0000 7f038c200700 -1 osd.5 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:43.793658+0000 front 2024-03-08T21:16:43.793735+0000 (oldest deadline 2024-03-08T21:17:07.893906+0000) 2024-03-08T21:17:20.121 INFO:journalctl@ceph.osd.5.smithi204.stdout:Mar 08 21:17:19 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-5[47140]: 2024-03-08T21:17:19.829+0000 7f038c200700 -1 osd.5 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:53.697831+0000 front 2024-03-08T21:16:53.697757+0000 (oldest deadline 2024-03-08T21:17:14.797782+0000) 2024-03-08T21:17:20.621 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 08 21:17:20 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-8[53709]: 2024-03-08T21:17:20.260+0000 7f3fae200700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:43.996391+0000 front 2024-03-08T21:16:43.996418+0000 (oldest deadline 2024-03-08T21:17:09.296545+0000) 2024-03-08T21:17:20.621 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 08 21:17:20 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-8[53709]: 2024-03-08T21:17:20.260+0000 7f3fae200700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:52.199334+0000 front 2024-03-08T21:16:52.199113+0000 (oldest deadline 2024-03-08T21:17:16.899180+0000) 2024-03-08T21:17:20.621 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:17:20 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:17:20.439+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:42.614355+0000 front 2024-03-08T21:16:42.614271+0000 (oldest deadline 2024-03-08T21:17:07.914284+0000) 2024-03-08T21:17:20.621 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:17:20 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:17:20.439+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:52.016503+0000 front 2024-03-08T21:16:52.017017+0000 (oldest deadline 2024-03-08T21:17:16.116550+0000) 2024-03-08T21:17:20.670 INFO:journalctl@ceph.osd.4.smithi149.stdout:Mar 08 21:17:20 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-4[46289]: 2024-03-08T21:17:20.391+0000 7f297fc00700 -1 osd.4 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:41.916650+0000 front 2024-03-08T21:16:41.917178+0000 (oldest deadline 2024-03-08T21:17:06.016911+0000) 2024-03-08T21:17:20.710 INFO:journalctl@ceph.osd.6.smithi107.stdout:Mar 08 21:17:20 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-6[59834]: 2024-03-08T21:17:20.355+0000 7f6132600700 -1 osd.6 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:44.641569+0000 front 2024-03-08T21:16:44.641810+0000 (oldest deadline 2024-03-08T21:17:07.541743+0000) 2024-03-08T21:17:20.710 INFO:journalctl@ceph.osd.6.smithi107.stdout:Mar 08 21:17:20 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-6[59834]: 2024-03-08T21:17:20.355+0000 7f6132600700 -1 osd.6 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:53.344721+0000 front 2024-03-08T21:16:53.344770+0000 (oldest deadline 2024-03-08T21:17:15.644922+0000) 2024-03-08T21:17:20.710 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:17:20 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:17:20.268+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:40.479965+0000 front 2024-03-08T21:16:40.480241+0000 (oldest deadline 2024-03-08T21:17:06.379830+0000) 2024-03-08T21:17:20.710 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:17:20 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:17:20.268+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:53.283252+0000 front 2024-03-08T21:16:53.283300+0000 (oldest deadline 2024-03-08T21:17:17.383498+0000) 2024-03-08T21:17:21.008 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 08 21:17:20 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-7[53366]: 2024-03-08T21:17:20.694+0000 7f48e9a00700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:53.119608+0000 front 2024-03-08T21:16:53.119745+0000 (oldest deadline 2024-03-08T21:17:17.819760+0000) 2024-03-08T21:17:21.008 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:17:20 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:17:20.928+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:44.858572+0000 front 2024-03-08T21:16:44.858829+0000 (oldest deadline 2024-03-08T21:17:06.558274+0000) 2024-03-08T21:17:21.008 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:17:20 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:17:20.928+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:52.961888+0000 front 2024-03-08T21:16:52.961762+0000 (oldest deadline 2024-03-08T21:17:18.862022+0000) 2024-03-08T21:17:21.121 INFO:journalctl@ceph.osd.5.smithi204.stdout:Mar 08 21:17:20 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-5[47140]: 2024-03-08T21:17:20.787+0000 7f038c200700 -1 osd.5 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:43.793658+0000 front 2024-03-08T21:16:43.793735+0000 (oldest deadline 2024-03-08T21:17:07.893906+0000) 2024-03-08T21:17:21.121 INFO:journalctl@ceph.osd.5.smithi204.stdout:Mar 08 21:17:20 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-5[47140]: 2024-03-08T21:17:20.787+0000 7f038c200700 -1 osd.5 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:53.697831+0000 front 2024-03-08T21:16:53.697757+0000 (oldest deadline 2024-03-08T21:17:14.797782+0000) 2024-03-08T21:17:21.621 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 08 21:17:21 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-8[53709]: 2024-03-08T21:17:21.284+0000 7f3fae200700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:43.996391+0000 front 2024-03-08T21:16:43.996418+0000 (oldest deadline 2024-03-08T21:17:09.296545+0000) 2024-03-08T21:17:21.621 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 08 21:17:21 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-8[53709]: 2024-03-08T21:17:21.284+0000 7f3fae200700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:52.199334+0000 front 2024-03-08T21:16:52.199113+0000 (oldest deadline 2024-03-08T21:17:16.899180+0000) 2024-03-08T21:17:21.621 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:17:21 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:17:21.459+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:42.614355+0000 front 2024-03-08T21:16:42.614271+0000 (oldest deadline 2024-03-08T21:17:07.914284+0000) 2024-03-08T21:17:21.621 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:17:21 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:17:21.459+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:52.016503+0000 front 2024-03-08T21:16:52.017017+0000 (oldest deadline 2024-03-08T21:17:16.116550+0000) 2024-03-08T21:17:21.663 INFO:journalctl@ceph.osd.4.smithi149.stdout:Mar 08 21:17:21 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-4[46289]: 2024-03-08T21:17:21.382+0000 7f297fc00700 -1 osd.4 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:41.916650+0000 front 2024-03-08T21:16:41.917178+0000 (oldest deadline 2024-03-08T21:17:06.016911+0000) 2024-03-08T21:17:21.710 INFO:journalctl@ceph.osd.6.smithi107.stdout:Mar 08 21:17:21 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-6[59834]: 2024-03-08T21:17:21.332+0000 7f6132600700 -1 osd.6 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:44.641569+0000 front 2024-03-08T21:16:44.641810+0000 (oldest deadline 2024-03-08T21:17:07.541743+0000) 2024-03-08T21:17:21.710 INFO:journalctl@ceph.osd.6.smithi107.stdout:Mar 08 21:17:21 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-6[59834]: 2024-03-08T21:17:21.332+0000 7f6132600700 -1 osd.6 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:53.344721+0000 front 2024-03-08T21:16:53.344770+0000 (oldest deadline 2024-03-08T21:17:15.644922+0000) 2024-03-08T21:17:21.711 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:17:21 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:17:21.277+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:40.479965+0000 front 2024-03-08T21:16:40.480241+0000 (oldest deadline 2024-03-08T21:17:06.379830+0000) 2024-03-08T21:17:21.711 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:17:21 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:17:21.277+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:53.283252+0000 front 2024-03-08T21:16:53.283300+0000 (oldest deadline 2024-03-08T21:17:17.383498+0000) 2024-03-08T21:17:21.953 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 08 21:17:21 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-7[53366]: 2024-03-08T21:17:21.687+0000 7f48e9a00700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:53.119608+0000 front 2024-03-08T21:16:53.119745+0000 (oldest deadline 2024-03-08T21:17:17.819760+0000) 2024-03-08T21:17:22.121 INFO:journalctl@ceph.osd.5.smithi204.stdout:Mar 08 21:17:21 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-5[47140]: 2024-03-08T21:17:21.832+0000 7f038c200700 -1 osd.5 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:43.793658+0000 front 2024-03-08T21:16:43.793735+0000 (oldest deadline 2024-03-08T21:17:07.893906+0000) 2024-03-08T21:17:22.121 INFO:journalctl@ceph.osd.5.smithi204.stdout:Mar 08 21:17:21 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-5[47140]: 2024-03-08T21:17:21.832+0000 7f038c200700 -1 osd.5 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:53.697831+0000 front 2024-03-08T21:16:53.697757+0000 (oldest deadline 2024-03-08T21:17:14.797782+0000) 2024-03-08T21:17:22.225 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:17:21 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:17:21.977+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:44.858572+0000 front 2024-03-08T21:16:44.858829+0000 (oldest deadline 2024-03-08T21:17:06.558274+0000) 2024-03-08T21:17:22.225 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:17:21 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:17:21.977+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:52.961888+0000 front 2024-03-08T21:16:52.961762+0000 (oldest deadline 2024-03-08T21:17:18.862022+0000) 2024-03-08T21:17:22.509 INFO:journalctl@ceph.osd.4.smithi149.stdout:Mar 08 21:17:22 smithi149 podman[72548]: 2024-03-08 21:17:22.415348382 +0000 UTC m=+5.569927050 container died ce49b985891b3e0450c66bc2a9ee6f2d6824dcff061b574bde606453d331dd1b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-4, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=rishabh-2024mar8-7d63482, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD) 2024-03-08T21:17:22.621 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 08 21:17:22 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-8[53709]: 2024-03-08T21:17:22.267+0000 7f3fae200700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:43.996391+0000 front 2024-03-08T21:16:43.996418+0000 (oldest deadline 2024-03-08T21:17:09.296545+0000) 2024-03-08T21:17:22.621 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 08 21:17:22 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-8[53709]: 2024-03-08T21:17:22.267+0000 7f3fae200700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:52.199334+0000 front 2024-03-08T21:16:52.199113+0000 (oldest deadline 2024-03-08T21:17:16.899180+0000) 2024-03-08T21:17:22.621 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:17:22 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:17:22.504+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:42.614355+0000 front 2024-03-08T21:16:42.614271+0000 (oldest deadline 2024-03-08T21:17:07.914284+0000) 2024-03-08T21:17:22.621 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:17:22 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:17:22.504+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:52.016503+0000 front 2024-03-08T21:16:52.017017+0000 (oldest deadline 2024-03-08T21:17:16.116550+0000) 2024-03-08T21:17:22.710 INFO:journalctl@ceph.osd.6.smithi107.stdout:Mar 08 21:17:22 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-6[59834]: 2024-03-08T21:17:22.288+0000 7f6132600700 -1 osd.6 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:44.641569+0000 front 2024-03-08T21:16:44.641810+0000 (oldest deadline 2024-03-08T21:17:07.541743+0000) 2024-03-08T21:17:22.711 INFO:journalctl@ceph.osd.6.smithi107.stdout:Mar 08 21:17:22 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-6[59834]: 2024-03-08T21:17:22.288+0000 7f6132600700 -1 osd.6 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:53.344721+0000 front 2024-03-08T21:16:53.344770+0000 (oldest deadline 2024-03-08T21:17:15.644922+0000) 2024-03-08T21:17:22.711 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:17:22 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:17:22.238+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:40.479965+0000 front 2024-03-08T21:16:40.480241+0000 (oldest deadline 2024-03-08T21:17:06.379830+0000) 2024-03-08T21:17:22.711 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:17:22 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:17:22.238+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:53.283252+0000 front 2024-03-08T21:16:53.283300+0000 (oldest deadline 2024-03-08T21:17:17.383498+0000) 2024-03-08T21:17:22.991 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 08 21:17:22 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-7[53366]: 2024-03-08T21:17:22.734+0000 7f48e9a00700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:53.119608+0000 front 2024-03-08T21:16:53.119745+0000 (oldest deadline 2024-03-08T21:17:17.819760+0000) 2024-03-08T21:17:22.991 INFO:journalctl@ceph.osd.4.smithi149.stdout:Mar 08 21:17:22 smithi149 podman[72548]: 2024-03-08 21:17:22.854443874 +0000 UTC m=+6.009022519 container cleanup ce49b985891b3e0450c66bc2a9ee6f2d6824dcff061b574bde606453d331dd1b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-4, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, RELEASE=rishabh-2024mar8-7d63482, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-03-08T21:17:22.992 INFO:journalctl@ceph.osd.4.smithi149.stdout:Mar 08 21:17:22 smithi149 bash[72548]: ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-4 2024-03-08T21:17:23.121 INFO:journalctl@ceph.osd.5.smithi204.stdout:Mar 08 21:17:22 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-5[47140]: 2024-03-08T21:17:22.806+0000 7f038c200700 -1 osd.5 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:43.793658+0000 front 2024-03-08T21:16:43.793735+0000 (oldest deadline 2024-03-08T21:17:07.893906+0000) 2024-03-08T21:17:23.121 INFO:journalctl@ceph.osd.5.smithi204.stdout:Mar 08 21:17:22 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-5[47140]: 2024-03-08T21:17:22.806+0000 7f038c200700 -1 osd.5 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:53.697831+0000 front 2024-03-08T21:16:53.697757+0000 (oldest deadline 2024-03-08T21:17:14.797782+0000) 2024-03-08T21:17:23.258 INFO:journalctl@ceph.osd.4.smithi149.stdout:Mar 08 21:17:23 smithi149 podman[72580]: 2024-03-08 21:17:23.139697311 +0000 UTC m=+0.706922283 container remove ce49b985891b3e0450c66bc2a9ee6f2d6824dcff061b574bde606453d331dd1b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-4, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=rishabh-2024mar8-7d63482, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-03-08T21:17:23.258 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:17:23 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:17:23.015+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:44.858572+0000 front 2024-03-08T21:16:44.858829+0000 (oldest deadline 2024-03-08T21:17:06.558274+0000) 2024-03-08T21:17:23.258 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:17:23 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:17:23.015+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:52.961888+0000 front 2024-03-08T21:16:52.961762+0000 (oldest deadline 2024-03-08T21:17:18.862022+0000) 2024-03-08T21:17:23.515 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 08 21:17:23 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-8[53709]: 2024-03-08T21:17:23.249+0000 7f3fae200700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:43.996391+0000 front 2024-03-08T21:16:43.996418+0000 (oldest deadline 2024-03-08T21:17:09.296545+0000) 2024-03-08T21:17:23.515 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 08 21:17:23 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-8[53709]: 2024-03-08T21:17:23.249+0000 7f3fae200700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:52.199334+0000 front 2024-03-08T21:16:52.199113+0000 (oldest deadline 2024-03-08T21:17:16.899180+0000) 2024-03-08T21:17:23.710 INFO:journalctl@ceph.osd.6.smithi107.stdout:Mar 08 21:17:23 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-6[59834]: 2024-03-08T21:17:23.261+0000 7f6132600700 -1 osd.6 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:44.641569+0000 front 2024-03-08T21:16:44.641810+0000 (oldest deadline 2024-03-08T21:17:07.541743+0000) 2024-03-08T21:17:23.711 INFO:journalctl@ceph.osd.6.smithi107.stdout:Mar 08 21:17:23 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-6[59834]: 2024-03-08T21:17:23.261+0000 7f6132600700 -1 osd.6 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:53.344721+0000 front 2024-03-08T21:16:53.344770+0000 (oldest deadline 2024-03-08T21:17:15.644922+0000) 2024-03-08T21:17:23.711 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:17:23 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:17:23.278+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:40.479965+0000 front 2024-03-08T21:16:40.480241+0000 (oldest deadline 2024-03-08T21:17:06.379830+0000) 2024-03-08T21:17:23.711 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:17:23 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:17:23.278+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:53.283252+0000 front 2024-03-08T21:16:53.283300+0000 (oldest deadline 2024-03-08T21:17:17.383498+0000) 2024-03-08T21:17:23.820 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:17:23 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:17:23.538+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:42.614355+0000 front 2024-03-08T21:16:42.614271+0000 (oldest deadline 2024-03-08T21:17:07.914284+0000) 2024-03-08T21:17:23.821 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:17:23 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:17:23.538+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:52.016503+0000 front 2024-03-08T21:16:52.017017+0000 (oldest deadline 2024-03-08T21:17:16.116550+0000) 2024-03-08T21:17:24.008 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 08 21:17:23 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-7[53366]: 2024-03-08T21:17:23.760+0000 7f48e9a00700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:53.119608+0000 front 2024-03-08T21:16:53.119745+0000 (oldest deadline 2024-03-08T21:17:17.819760+0000) 2024-03-08T21:17:24.008 INFO:journalctl@ceph.osd.4.smithi149.stdout:Mar 08 21:17:23 smithi149 podman[72666]: 2024-03-08 21:17:23.870598687 +0000 UTC m=+0.033073600 image pull a272146cb810b1fd4a53627f019472132e30966d5c25b01bfc44a001295f1249 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd 2024-03-08T21:17:24.008 INFO:journalctl@ceph.osd.4.smithi149.stdout:Mar 08 21:17:23 smithi149 podman[72666]: 2024-03-08 21:17:23.99894231 +0000 UTC m=+0.161417259 container create 38b283c695bbef99e0fb05544405d7a10bce92fac3ef8068cd6a914546a6b737 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-4-deactivate, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=rishabh-2024mar8-7d63482, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True) 2024-03-08T21:17:24.121 INFO:journalctl@ceph.osd.5.smithi204.stdout:Mar 08 21:17:23 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-5[47140]: 2024-03-08T21:17:23.844+0000 7f038c200700 -1 osd.5 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:43.793658+0000 front 2024-03-08T21:16:43.793735+0000 (oldest deadline 2024-03-08T21:17:07.893906+0000) 2024-03-08T21:17:24.121 INFO:journalctl@ceph.osd.5.smithi204.stdout:Mar 08 21:17:23 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-5[47140]: 2024-03-08T21:17:23.844+0000 7f038c200700 -1 osd.5 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:53.697831+0000 front 2024-03-08T21:16:53.697757+0000 (oldest deadline 2024-03-08T21:17:14.797782+0000) 2024-03-08T21:17:24.281 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:17:24 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:17:24.046+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:44.858572+0000 front 2024-03-08T21:16:44.858829+0000 (oldest deadline 2024-03-08T21:17:06.558274+0000) 2024-03-08T21:17:24.281 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:17:24 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:17:24.046+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:52.961888+0000 front 2024-03-08T21:16:52.961762+0000 (oldest deadline 2024-03-08T21:17:18.862022+0000) 2024-03-08T21:17:24.281 INFO:journalctl@ceph.osd.4.smithi149.stdout:Mar 08 21:17:24 smithi149 podman[72666]: 2024-03-08 21:17:24.294299115 +0000 UTC m=+0.456774071 container init 38b283c695bbef99e0fb05544405d7a10bce92fac3ef8068cd6a914546a6b737 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-4-deactivate, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=rishabh-2024mar8-7d63482) 2024-03-08T21:17:24.281 INFO:journalctl@ceph.osd.4.smithi149.stdout:Mar 08 21:17:24 smithi149 podman[72666]: 2024-03-08 21:17:24.304991581 +0000 UTC m=+0.467466537 container start 38b283c695bbef99e0fb05544405d7a10bce92fac3ef8068cd6a914546a6b737 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-4-deactivate, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=rishabh-2024mar8-7d63482, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-03-08T21:17:24.528 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 08 21:17:24 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-8[53709]: 2024-03-08T21:17:24.291+0000 7f3fae200700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:43.996391+0000 front 2024-03-08T21:16:43.996418+0000 (oldest deadline 2024-03-08T21:17:09.296545+0000) 2024-03-08T21:17:24.528 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 08 21:17:24 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-8[53709]: 2024-03-08T21:17:24.291+0000 7f3fae200700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:52.199334+0000 front 2024-03-08T21:16:52.199113+0000 (oldest deadline 2024-03-08T21:17:16.899180+0000) 2024-03-08T21:17:24.710 INFO:journalctl@ceph.osd.6.smithi107.stdout:Mar 08 21:17:24 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-6[59834]: 2024-03-08T21:17:24.306+0000 7f6132600700 -1 osd.6 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:44.641569+0000 front 2024-03-08T21:16:44.641810+0000 (oldest deadline 2024-03-08T21:17:07.541743+0000) 2024-03-08T21:17:24.711 INFO:journalctl@ceph.osd.6.smithi107.stdout:Mar 08 21:17:24 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-6[59834]: 2024-03-08T21:17:24.306+0000 7f6132600700 -1 osd.6 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:53.344721+0000 front 2024-03-08T21:16:53.344770+0000 (oldest deadline 2024-03-08T21:17:15.644922+0000) 2024-03-08T21:17:24.711 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:17:24 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:17:24.322+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:40.479965+0000 front 2024-03-08T21:16:40.480241+0000 (oldest deadline 2024-03-08T21:17:06.379830+0000) 2024-03-08T21:17:24.711 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:17:24 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:17:24.322+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:53.283252+0000 front 2024-03-08T21:16:53.283300+0000 (oldest deadline 2024-03-08T21:17:17.383498+0000) 2024-03-08T21:17:24.712 INFO:journalctl@ceph.osd.4.smithi149.stdout:Mar 08 21:17:24 smithi149 podman[72666]: 2024-03-08 21:17:24.402098172 +0000 UTC m=+0.564573128 container attach 38b283c695bbef99e0fb05544405d7a10bce92fac3ef8068cd6a914546a6b737 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-4-deactivate, RELEASE=rishabh-2024mar8-7d63482, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5) 2024-03-08T21:17:24.830 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:17:24 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:17:24.552+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:42.614355+0000 front 2024-03-08T21:16:42.614271+0000 (oldest deadline 2024-03-08T21:17:07.914284+0000) 2024-03-08T21:17:24.831 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:17:24 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:17:24.552+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:52.016503+0000 front 2024-03-08T21:16:52.017017+0000 (oldest deadline 2024-03-08T21:17:16.116550+0000) 2024-03-08T21:17:24.975 INFO:journalctl@ceph.osd.4.smithi149.stdout:Mar 08 21:17:24 smithi149 podman[72692]: 2024-03-08 21:17:24.790024853 +0000 UTC m=+0.032228699 container died 38b283c695bbef99e0fb05544405d7a10bce92fac3ef8068cd6a914546a6b737 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-4-deactivate, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=rishabh-2024mar8-7d63482, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-08T21:17:24.975 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 08 21:17:24 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-7[53366]: 2024-03-08T21:17:24.738+0000 7f48e9a00700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:53.119608+0000 front 2024-03-08T21:16:53.119745+0000 (oldest deadline 2024-03-08T21:17:17.819760+0000) 2024-03-08T21:17:25.121 INFO:journalctl@ceph.osd.5.smithi204.stdout:Mar 08 21:17:24 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-5[47140]: 2024-03-08T21:17:24.854+0000 7f038c200700 -1 osd.5 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:43.793658+0000 front 2024-03-08T21:16:43.793735+0000 (oldest deadline 2024-03-08T21:17:07.893906+0000) 2024-03-08T21:17:25.121 INFO:journalctl@ceph.osd.5.smithi204.stdout:Mar 08 21:17:24 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-5[47140]: 2024-03-08T21:17:24.854+0000 7f038c200700 -1 osd.5 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:53.697831+0000 front 2024-03-08T21:16:53.697757+0000 (oldest deadline 2024-03-08T21:17:14.797782+0000) 2024-03-08T21:17:25.258 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:17:25 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:17:24.999+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:44.858572+0000 front 2024-03-08T21:16:44.858829+0000 (oldest deadline 2024-03-08T21:17:06.558274+0000) 2024-03-08T21:17:25.258 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:17:25 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:17:24.999+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:52.961888+0000 front 2024-03-08T21:16:52.961762+0000 (oldest deadline 2024-03-08T21:17:18.862022+0000) 2024-03-08T21:17:25.258 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:17:25 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:17:24.999+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:17:01.764390+0000 front 2024-03-08T21:17:01.764438+0000 (oldest deadline 2024-03-08T21:17:24.064211+0000) 2024-03-08T21:17:25.409 DEBUG:teuthology.orchestra.run.smithi149:> sudo pkill -f 'journalctl -f -n 0 -u ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715@osd.4.service' 2024-03-08T21:17:25.524 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-08T21:17:25.524 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-03-08T21:17:25.524 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-03-08T21:17:25.524 DEBUG:teuthology.orchestra.run.smithi204:> sudo systemctl stop ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715@osd.5 2024-03-08T21:17:25.556 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 08 21:17:25 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-8[53709]: 2024-03-08T21:17:25.281+0000 7f3fae200700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:43.996391+0000 front 2024-03-08T21:16:43.996418+0000 (oldest deadline 2024-03-08T21:17:09.296545+0000) 2024-03-08T21:17:25.556 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 08 21:17:25 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-8[53709]: 2024-03-08T21:17:25.281+0000 7f3fae200700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:52.199334+0000 front 2024-03-08T21:16:52.199113+0000 (oldest deadline 2024-03-08T21:17:16.899180+0000) 2024-03-08T21:17:25.710 INFO:journalctl@ceph.osd.6.smithi107.stdout:Mar 08 21:17:25 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-6[59834]: 2024-03-08T21:17:25.266+0000 7f6132600700 -1 osd.6 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:44.641569+0000 front 2024-03-08T21:16:44.641810+0000 (oldest deadline 2024-03-08T21:17:07.541743+0000) 2024-03-08T21:17:25.711 INFO:journalctl@ceph.osd.6.smithi107.stdout:Mar 08 21:17:25 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-6[59834]: 2024-03-08T21:17:25.266+0000 7f6132600700 -1 osd.6 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:53.344721+0000 front 2024-03-08T21:16:53.344770+0000 (oldest deadline 2024-03-08T21:17:15.644922+0000) 2024-03-08T21:17:25.711 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:17:25 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:17:25.341+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:40.479965+0000 front 2024-03-08T21:16:40.480241+0000 (oldest deadline 2024-03-08T21:17:06.379830+0000) 2024-03-08T21:17:25.711 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:17:25 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:17:25.341+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:53.283252+0000 front 2024-03-08T21:16:53.283300+0000 (oldest deadline 2024-03-08T21:17:17.383498+0000) 2024-03-08T21:17:25.871 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:17:25 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:17:25.580+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:42.614355+0000 front 2024-03-08T21:16:42.614271+0000 (oldest deadline 2024-03-08T21:17:07.914284+0000) 2024-03-08T21:17:25.872 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:17:25 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:17:25.580+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:52.016503+0000 front 2024-03-08T21:16:52.017017+0000 (oldest deadline 2024-03-08T21:17:16.116550+0000) 2024-03-08T21:17:25.872 INFO:journalctl@ceph.osd.5.smithi204.stdout:Mar 08 21:17:25 smithi204 systemd[1]: Stopping Ceph osd.5 for dc20f0f8-dd8c-11ee-95c6-87774f69a715... 2024-03-08T21:17:25.872 INFO:journalctl@ceph.osd.5.smithi204.stdout:Mar 08 21:17:25 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-5[47140]: 2024-03-08T21:17:25.828+0000 7f038c200700 -1 osd.5 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:43.793658+0000 front 2024-03-08T21:16:43.793735+0000 (oldest deadline 2024-03-08T21:17:07.893906+0000) 2024-03-08T21:17:25.872 INFO:journalctl@ceph.osd.5.smithi204.stdout:Mar 08 21:17:25 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-5[47140]: 2024-03-08T21:17:25.828+0000 7f038c200700 -1 osd.5 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:53.697831+0000 front 2024-03-08T21:16:53.697757+0000 (oldest deadline 2024-03-08T21:17:14.797782+0000) 2024-03-08T21:17:25.994 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 08 21:17:25 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-7[53366]: 2024-03-08T21:17:25.709+0000 7f48e9a00700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:53.119608+0000 front 2024-03-08T21:16:53.119745+0000 (oldest deadline 2024-03-08T21:17:17.819760+0000) 2024-03-08T21:17:26.258 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:17:26 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:17:26.018+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:44.858572+0000 front 2024-03-08T21:16:44.858829+0000 (oldest deadline 2024-03-08T21:17:06.558274+0000) 2024-03-08T21:17:26.258 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:17:26 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:17:26.018+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:52.961888+0000 front 2024-03-08T21:16:52.961762+0000 (oldest deadline 2024-03-08T21:17:18.862022+0000) 2024-03-08T21:17:26.258 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:17:26 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:17:26.018+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:17:01.764390+0000 front 2024-03-08T21:17:01.764438+0000 (oldest deadline 2024-03-08T21:17:24.064211+0000) 2024-03-08T21:17:26.371 INFO:journalctl@ceph.osd.5.smithi204.stdout:Mar 08 21:17:26 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-5[47140]: 2024-03-08T21:17:26.128+0000 7f0395a00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-08T21:17:26.371 INFO:journalctl@ceph.osd.5.smithi204.stdout:Mar 08 21:17:26 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-5[47140]: 2024-03-08T21:17:26.128+0000 7f0395a00700 -1 osd.5 115 *** Got signal Terminated *** 2024-03-08T21:17:26.371 INFO:journalctl@ceph.osd.5.smithi204.stdout:Mar 08 21:17:26 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-5[47140]: 2024-03-08T21:17:26.128+0000 7f0395a00700 -1 osd.5 115 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-08T21:17:26.372 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 08 21:17:26 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-8[53709]: 2024-03-08T21:17:26.302+0000 7f3fae200700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:43.996391+0000 front 2024-03-08T21:16:43.996418+0000 (oldest deadline 2024-03-08T21:17:09.296545+0000) 2024-03-08T21:17:26.372 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 08 21:17:26 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-8[53709]: 2024-03-08T21:17:26.303+0000 7f3fae200700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:52.199334+0000 front 2024-03-08T21:16:52.199113+0000 (oldest deadline 2024-03-08T21:17:16.899180+0000) 2024-03-08T21:17:26.372 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 08 21:17:26 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-8[53709]: 2024-03-08T21:17:26.303+0000 7f3fae200700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:17:00.401255+0000 front 2024-03-08T21:17:00.401120+0000 (oldest deadline 2024-03-08T21:17:25.701340+0000) 2024-03-08T21:17:26.710 INFO:journalctl@ceph.osd.6.smithi107.stdout:Mar 08 21:17:26 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-6[59834]: 2024-03-08T21:17:26.248+0000 7f6132600700 -1 osd.6 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:44.641569+0000 front 2024-03-08T21:16:44.641810+0000 (oldest deadline 2024-03-08T21:17:07.541743+0000) 2024-03-08T21:17:26.711 INFO:journalctl@ceph.osd.6.smithi107.stdout:Mar 08 21:17:26 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-6[59834]: 2024-03-08T21:17:26.248+0000 7f6132600700 -1 osd.6 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:53.344721+0000 front 2024-03-08T21:16:53.344770+0000 (oldest deadline 2024-03-08T21:17:15.644922+0000) 2024-03-08T21:17:26.711 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:17:26 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:17:26.337+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:40.479965+0000 front 2024-03-08T21:16:40.480241+0000 (oldest deadline 2024-03-08T21:17:06.379830+0000) 2024-03-08T21:17:26.711 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:17:26 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:17:26.337+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:53.283252+0000 front 2024-03-08T21:16:53.283300+0000 (oldest deadline 2024-03-08T21:17:17.383498+0000) 2024-03-08T21:17:26.711 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:17:26 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:17:26.337+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:17:03.088480+0000 front 2024-03-08T21:17:03.088666+0000 (oldest deadline 2024-03-08T21:17:25.388404+0000) 2024-03-08T21:17:26.836 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:17:26 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:17:26.567+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:42.614355+0000 front 2024-03-08T21:16:42.614271+0000 (oldest deadline 2024-03-08T21:17:07.914284+0000) 2024-03-08T21:17:26.836 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:17:26 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:17:26.567+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:52.016503+0000 front 2024-03-08T21:16:52.017017+0000 (oldest deadline 2024-03-08T21:17:16.116550+0000) 2024-03-08T21:17:27.008 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 08 21:17:26 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-7[53366]: 2024-03-08T21:17:26.709+0000 7f48e9a00700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:53.119608+0000 front 2024-03-08T21:16:53.119745+0000 (oldest deadline 2024-03-08T21:17:17.819760+0000) 2024-03-08T21:17:27.121 INFO:journalctl@ceph.osd.5.smithi204.stdout:Mar 08 21:17:26 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-5[47140]: 2024-03-08T21:17:26.859+0000 7f038c200700 -1 osd.5 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:43.793658+0000 front 2024-03-08T21:16:43.793735+0000 (oldest deadline 2024-03-08T21:17:07.893906+0000) 2024-03-08T21:17:27.121 INFO:journalctl@ceph.osd.5.smithi204.stdout:Mar 08 21:17:26 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-5[47140]: 2024-03-08T21:17:26.859+0000 7f038c200700 -1 osd.5 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:53.697831+0000 front 2024-03-08T21:16:53.697757+0000 (oldest deadline 2024-03-08T21:17:14.797782+0000) 2024-03-08T21:17:27.121 INFO:journalctl@ceph.osd.5.smithi204.stdout:Mar 08 21:17:26 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-5[47140]: 2024-03-08T21:17:26.859+0000 7f038c200700 -1 osd.5 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:17:00.000861+0000 front 2024-03-08T21:17:00.001127+0000 (oldest deadline 2024-03-08T21:17:25.900961+0000) 2024-03-08T21:17:27.508 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:17:27 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:17:27.060+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:44.858572+0000 front 2024-03-08T21:16:44.858829+0000 (oldest deadline 2024-03-08T21:17:06.558274+0000) 2024-03-08T21:17:27.508 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:17:27 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:17:27.060+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:52.961888+0000 front 2024-03-08T21:16:52.961762+0000 (oldest deadline 2024-03-08T21:17:18.862022+0000) 2024-03-08T21:17:27.508 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:17:27 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:17:27.060+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:17:01.764390+0000 front 2024-03-08T21:17:01.764438+0000 (oldest deadline 2024-03-08T21:17:24.064211+0000) 2024-03-08T21:17:27.571 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 08 21:17:27 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-8[53709]: 2024-03-08T21:17:27.275+0000 7f3fae200700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:43.996391+0000 front 2024-03-08T21:16:43.996418+0000 (oldest deadline 2024-03-08T21:17:09.296545+0000) 2024-03-08T21:17:27.571 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 08 21:17:27 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-8[53709]: 2024-03-08T21:17:27.275+0000 7f3fae200700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:52.199334+0000 front 2024-03-08T21:16:52.199113+0000 (oldest deadline 2024-03-08T21:17:16.899180+0000) 2024-03-08T21:17:27.571 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 08 21:17:27 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-8[53709]: 2024-03-08T21:17:27.275+0000 7f3fae200700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:17:00.401255+0000 front 2024-03-08T21:17:00.401120+0000 (oldest deadline 2024-03-08T21:17:25.701340+0000) 2024-03-08T21:17:27.710 INFO:journalctl@ceph.osd.6.smithi107.stdout:Mar 08 21:17:27 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-6[59834]: 2024-03-08T21:17:27.236+0000 7f6132600700 -1 osd.6 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:44.641569+0000 front 2024-03-08T21:16:44.641810+0000 (oldest deadline 2024-03-08T21:17:07.541743+0000) 2024-03-08T21:17:27.711 INFO:journalctl@ceph.osd.6.smithi107.stdout:Mar 08 21:17:27 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-6[59834]: 2024-03-08T21:17:27.236+0000 7f6132600700 -1 osd.6 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:53.344721+0000 front 2024-03-08T21:16:53.344770+0000 (oldest deadline 2024-03-08T21:17:15.644922+0000) 2024-03-08T21:17:27.711 INFO:journalctl@ceph.osd.6.smithi107.stdout:Mar 08 21:17:27 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-6[59834]: 2024-03-08T21:17:27.236+0000 7f6132600700 -1 osd.6 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:17:02.047607+0000 front 2024-03-08T21:17:02.047791+0000 (oldest deadline 2024-03-08T21:17:26.747841+0000) 2024-03-08T21:17:27.711 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:17:27 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:17:27.341+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:40.479965+0000 front 2024-03-08T21:16:40.480241+0000 (oldest deadline 2024-03-08T21:17:06.379830+0000) 2024-03-08T21:17:27.711 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:17:27 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:17:27.341+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:53.283252+0000 front 2024-03-08T21:16:53.283300+0000 (oldest deadline 2024-03-08T21:17:17.383498+0000) 2024-03-08T21:17:27.711 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:17:27 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:17:27.341+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:17:03.088480+0000 front 2024-03-08T21:17:03.088666+0000 (oldest deadline 2024-03-08T21:17:25.388404+0000) 2024-03-08T21:17:27.871 INFO:journalctl@ceph.osd.5.smithi204.stdout:Mar 08 21:17:27 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-5[47140]: 2024-03-08T21:17:27.817+0000 7f038c200700 -1 osd.5 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:43.793658+0000 front 2024-03-08T21:16:43.793735+0000 (oldest deadline 2024-03-08T21:17:07.893906+0000) 2024-03-08T21:17:27.871 INFO:journalctl@ceph.osd.5.smithi204.stdout:Mar 08 21:17:27 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-5[47140]: 2024-03-08T21:17:27.817+0000 7f038c200700 -1 osd.5 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:53.697831+0000 front 2024-03-08T21:16:53.697757+0000 (oldest deadline 2024-03-08T21:17:14.797782+0000) 2024-03-08T21:17:27.871 INFO:journalctl@ceph.osd.5.smithi204.stdout:Mar 08 21:17:27 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-5[47140]: 2024-03-08T21:17:27.817+0000 7f038c200700 -1 osd.5 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:17:00.000861+0000 front 2024-03-08T21:17:00.001127+0000 (oldest deadline 2024-03-08T21:17:25.900961+0000) 2024-03-08T21:17:27.872 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:17:27 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:17:27.594+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:42.614355+0000 front 2024-03-08T21:16:42.614271+0000 (oldest deadline 2024-03-08T21:17:07.914284+0000) 2024-03-08T21:17:27.872 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:17:27 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:17:27.594+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:52.016503+0000 front 2024-03-08T21:16:52.017017+0000 (oldest deadline 2024-03-08T21:17:16.116550+0000) 2024-03-08T21:17:27.872 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:17:27 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:17:27.594+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:16:57.818698+0000 front 2024-03-08T21:16:57.819201+0000 (oldest deadline 2024-03-08T21:17:26.619635+0000) 2024-03-08T21:17:28.008 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 08 21:17:27 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-7[53366]: 2024-03-08T21:17:27.725+0000 7f48e9a00700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:53.119608+0000 front 2024-03-08T21:16:53.119745+0000 (oldest deadline 2024-03-08T21:17:17.819760+0000) 2024-03-08T21:17:28.008 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 08 21:17:27 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-7[53366]: 2024-03-08T21:17:27.725+0000 7f48e9a00700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:17:01.822810+0000 front 2024-03-08T21:17:01.822741+0000 (oldest deadline 2024-03-08T21:17:27.122609+0000) 2024-03-08T21:17:28.460 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:17:28 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:17:28.299+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:40.479965+0000 front 2024-03-08T21:16:40.480241+0000 (oldest deadline 2024-03-08T21:17:06.379830+0000) 2024-03-08T21:17:28.460 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:17:28 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:17:28.299+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:53.283252+0000 front 2024-03-08T21:16:53.283300+0000 (oldest deadline 2024-03-08T21:17:17.383498+0000) 2024-03-08T21:17:28.460 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:17:28 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:17:28.299+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:17:03.088480+0000 front 2024-03-08T21:17:03.088666+0000 (oldest deadline 2024-03-08T21:17:25.388404+0000) 2024-03-08T21:17:28.461 INFO:journalctl@ceph.osd.6.smithi107.stdout:Mar 08 21:17:28 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-6[59834]: 2024-03-08T21:17:28.226+0000 7f6132600700 -1 osd.6 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:44.641569+0000 front 2024-03-08T21:16:44.641810+0000 (oldest deadline 2024-03-08T21:17:07.541743+0000) 2024-03-08T21:17:28.461 INFO:journalctl@ceph.osd.6.smithi107.stdout:Mar 08 21:17:28 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-6[59834]: 2024-03-08T21:17:28.226+0000 7f6132600700 -1 osd.6 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:53.344721+0000 front 2024-03-08T21:16:53.344770+0000 (oldest deadline 2024-03-08T21:17:15.644922+0000) 2024-03-08T21:17:28.461 INFO:journalctl@ceph.osd.6.smithi107.stdout:Mar 08 21:17:28 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-6[59834]: 2024-03-08T21:17:28.226+0000 7f6132600700 -1 osd.6 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:17:02.047607+0000 front 2024-03-08T21:17:02.047791+0000 (oldest deadline 2024-03-08T21:17:26.747841+0000) 2024-03-08T21:17:28.507 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:17:28 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:17:28.104+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:44.858572+0000 front 2024-03-08T21:16:44.858829+0000 (oldest deadline 2024-03-08T21:17:06.558274+0000) 2024-03-08T21:17:28.507 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:17:28 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:17:28.104+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:52.961888+0000 front 2024-03-08T21:16:52.961762+0000 (oldest deadline 2024-03-08T21:17:18.862022+0000) 2024-03-08T21:17:28.508 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:17:28 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:17:28.104+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:17:01.764390+0000 front 2024-03-08T21:17:01.764438+0000 (oldest deadline 2024-03-08T21:17:24.064211+0000) 2024-03-08T21:17:28.544 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 08 21:17:28 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-8[53709]: 2024-03-08T21:17:28.237+0000 7f3fae200700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:43.996391+0000 front 2024-03-08T21:16:43.996418+0000 (oldest deadline 2024-03-08T21:17:09.296545+0000) 2024-03-08T21:17:28.544 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 08 21:17:28 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-8[53709]: 2024-03-08T21:17:28.237+0000 7f3fae200700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:52.199334+0000 front 2024-03-08T21:16:52.199113+0000 (oldest deadline 2024-03-08T21:17:16.899180+0000) 2024-03-08T21:17:28.544 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 08 21:17:28 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-8[53709]: 2024-03-08T21:17:28.237+0000 7f3fae200700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:17:00.401255+0000 front 2024-03-08T21:17:00.401120+0000 (oldest deadline 2024-03-08T21:17:25.701340+0000) 2024-03-08T21:17:28.839 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:17:28 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:17:28.567+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:42.614355+0000 front 2024-03-08T21:16:42.614271+0000 (oldest deadline 2024-03-08T21:17:07.914284+0000) 2024-03-08T21:17:28.840 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:17:28 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:17:28.567+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:52.016503+0000 front 2024-03-08T21:16:52.017017+0000 (oldest deadline 2024-03-08T21:17:16.116550+0000) 2024-03-08T21:17:28.840 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:17:28 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:17:28.567+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:16:57.818698+0000 front 2024-03-08T21:16:57.819201+0000 (oldest deadline 2024-03-08T21:17:26.619635+0000) 2024-03-08T21:17:29.008 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 08 21:17:28 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-7[53366]: 2024-03-08T21:17:28.739+0000 7f48e9a00700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:53.119608+0000 front 2024-03-08T21:16:53.119745+0000 (oldest deadline 2024-03-08T21:17:17.819760+0000) 2024-03-08T21:17:29.008 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 08 21:17:28 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-7[53366]: 2024-03-08T21:17:28.739+0000 7f48e9a00700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:17:01.822810+0000 front 2024-03-08T21:17:01.822741+0000 (oldest deadline 2024-03-08T21:17:27.122609+0000) 2024-03-08T21:17:29.121 INFO:journalctl@ceph.osd.5.smithi204.stdout:Mar 08 21:17:28 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-5[47140]: 2024-03-08T21:17:28.863+0000 7f038c200700 -1 osd.5 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:43.793658+0000 front 2024-03-08T21:16:43.793735+0000 (oldest deadline 2024-03-08T21:17:07.893906+0000) 2024-03-08T21:17:29.121 INFO:journalctl@ceph.osd.5.smithi204.stdout:Mar 08 21:17:28 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-5[47140]: 2024-03-08T21:17:28.863+0000 7f038c200700 -1 osd.5 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:53.697831+0000 front 2024-03-08T21:16:53.697757+0000 (oldest deadline 2024-03-08T21:17:14.797782+0000) 2024-03-08T21:17:29.121 INFO:journalctl@ceph.osd.5.smithi204.stdout:Mar 08 21:17:28 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-5[47140]: 2024-03-08T21:17:28.863+0000 7f038c200700 -1 osd.5 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:17:00.000861+0000 front 2024-03-08T21:17:00.001127+0000 (oldest deadline 2024-03-08T21:17:25.900961+0000) 2024-03-08T21:17:29.508 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:17:29 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:17:29.071+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:44.858572+0000 front 2024-03-08T21:16:44.858829+0000 (oldest deadline 2024-03-08T21:17:06.558274+0000) 2024-03-08T21:17:29.508 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:17:29 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:17:29.071+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:52.961888+0000 front 2024-03-08T21:16:52.961762+0000 (oldest deadline 2024-03-08T21:17:18.862022+0000) 2024-03-08T21:17:29.508 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:17:29 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:17:29.071+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:17:01.764390+0000 front 2024-03-08T21:17:01.764438+0000 (oldest deadline 2024-03-08T21:17:24.064211+0000) 2024-03-08T21:17:29.532 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 08 21:17:29 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-8[53709]: 2024-03-08T21:17:29.205+0000 7f3fae200700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:43.996391+0000 front 2024-03-08T21:16:43.996418+0000 (oldest deadline 2024-03-08T21:17:09.296545+0000) 2024-03-08T21:17:29.532 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 08 21:17:29 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-8[53709]: 2024-03-08T21:17:29.205+0000 7f3fae200700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:52.199334+0000 front 2024-03-08T21:16:52.199113+0000 (oldest deadline 2024-03-08T21:17:16.899180+0000) 2024-03-08T21:17:29.532 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 08 21:17:29 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-8[53709]: 2024-03-08T21:17:29.205+0000 7f3fae200700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:17:00.401255+0000 front 2024-03-08T21:17:00.401120+0000 (oldest deadline 2024-03-08T21:17:25.701340+0000) 2024-03-08T21:17:29.710 INFO:journalctl@ceph.osd.6.smithi107.stdout:Mar 08 21:17:29 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-6[59834]: 2024-03-08T21:17:29.275+0000 7f6132600700 -1 osd.6 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:44.641569+0000 front 2024-03-08T21:16:44.641810+0000 (oldest deadline 2024-03-08T21:17:07.541743+0000) 2024-03-08T21:17:29.710 INFO:journalctl@ceph.osd.6.smithi107.stdout:Mar 08 21:17:29 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-6[59834]: 2024-03-08T21:17:29.275+0000 7f6132600700 -1 osd.6 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:53.344721+0000 front 2024-03-08T21:16:53.344770+0000 (oldest deadline 2024-03-08T21:17:15.644922+0000) 2024-03-08T21:17:29.710 INFO:journalctl@ceph.osd.6.smithi107.stdout:Mar 08 21:17:29 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-6[59834]: 2024-03-08T21:17:29.275+0000 7f6132600700 -1 osd.6 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:17:02.047607+0000 front 2024-03-08T21:17:02.047791+0000 (oldest deadline 2024-03-08T21:17:26.747841+0000) 2024-03-08T21:17:29.711 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:17:29 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:17:29.299+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:40.479965+0000 front 2024-03-08T21:16:40.480241+0000 (oldest deadline 2024-03-08T21:17:06.379830+0000) 2024-03-08T21:17:29.711 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:17:29 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:17:29.299+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:53.283252+0000 front 2024-03-08T21:16:53.283300+0000 (oldest deadline 2024-03-08T21:17:17.383498+0000) 2024-03-08T21:17:29.711 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:17:29 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:17:29.299+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:17:03.088480+0000 front 2024-03-08T21:17:03.088666+0000 (oldest deadline 2024-03-08T21:17:25.388404+0000) 2024-03-08T21:17:29.808 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:17:29 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:17:29.555+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:42.614355+0000 front 2024-03-08T21:16:42.614271+0000 (oldest deadline 2024-03-08T21:17:07.914284+0000) 2024-03-08T21:17:29.808 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:17:29 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:17:29.555+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:52.016503+0000 front 2024-03-08T21:16:52.017017+0000 (oldest deadline 2024-03-08T21:17:16.116550+0000) 2024-03-08T21:17:29.808 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:17:29 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:17:29.555+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:16:57.818698+0000 front 2024-03-08T21:16:57.819201+0000 (oldest deadline 2024-03-08T21:17:26.619635+0000) 2024-03-08T21:17:30.008 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 08 21:17:29 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-7[53366]: 2024-03-08T21:17:29.715+0000 7f48e9a00700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:53.119608+0000 front 2024-03-08T21:16:53.119745+0000 (oldest deadline 2024-03-08T21:17:17.819760+0000) 2024-03-08T21:17:30.008 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 08 21:17:29 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-7[53366]: 2024-03-08T21:17:29.715+0000 7f48e9a00700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:17:01.822810+0000 front 2024-03-08T21:17:01.822741+0000 (oldest deadline 2024-03-08T21:17:27.122609+0000) 2024-03-08T21:17:30.121 INFO:journalctl@ceph.osd.5.smithi204.stdout:Mar 08 21:17:29 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-5[47140]: 2024-03-08T21:17:29.831+0000 7f038c200700 -1 osd.5 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:43.793658+0000 front 2024-03-08T21:16:43.793735+0000 (oldest deadline 2024-03-08T21:17:07.893906+0000) 2024-03-08T21:17:30.121 INFO:journalctl@ceph.osd.5.smithi204.stdout:Mar 08 21:17:29 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-5[47140]: 2024-03-08T21:17:29.831+0000 7f038c200700 -1 osd.5 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:53.697831+0000 front 2024-03-08T21:16:53.697757+0000 (oldest deadline 2024-03-08T21:17:14.797782+0000) 2024-03-08T21:17:30.121 INFO:journalctl@ceph.osd.5.smithi204.stdout:Mar 08 21:17:29 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-5[47140]: 2024-03-08T21:17:29.831+0000 7f038c200700 -1 osd.5 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:17:00.000861+0000 front 2024-03-08T21:17:00.001127+0000 (oldest deadline 2024-03-08T21:17:25.900961+0000) 2024-03-08T21:17:30.502 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 08 21:17:30 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-8[53709]: 2024-03-08T21:17:30.217+0000 7f3fae200700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:43.996391+0000 front 2024-03-08T21:16:43.996418+0000 (oldest deadline 2024-03-08T21:17:09.296545+0000) 2024-03-08T21:17:30.502 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 08 21:17:30 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-8[53709]: 2024-03-08T21:17:30.217+0000 7f3fae200700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:52.199334+0000 front 2024-03-08T21:16:52.199113+0000 (oldest deadline 2024-03-08T21:17:16.899180+0000) 2024-03-08T21:17:30.502 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 08 21:17:30 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-8[53709]: 2024-03-08T21:17:30.217+0000 7f3fae200700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:17:00.401255+0000 front 2024-03-08T21:17:00.401120+0000 (oldest deadline 2024-03-08T21:17:25.701340+0000) 2024-03-08T21:17:30.507 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:17:30 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:17:30.062+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:44.858572+0000 front 2024-03-08T21:16:44.858829+0000 (oldest deadline 2024-03-08T21:17:06.558274+0000) 2024-03-08T21:17:30.508 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:17:30 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:17:30.062+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:52.961888+0000 front 2024-03-08T21:16:52.961762+0000 (oldest deadline 2024-03-08T21:17:18.862022+0000) 2024-03-08T21:17:30.508 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:17:30 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:17:30.062+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:17:01.764390+0000 front 2024-03-08T21:17:01.764438+0000 (oldest deadline 2024-03-08T21:17:24.064211+0000) 2024-03-08T21:17:30.710 INFO:journalctl@ceph.osd.6.smithi107.stdout:Mar 08 21:17:30 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-6[59834]: 2024-03-08T21:17:30.279+0000 7f6132600700 -1 osd.6 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:44.641569+0000 front 2024-03-08T21:16:44.641810+0000 (oldest deadline 2024-03-08T21:17:07.541743+0000) 2024-03-08T21:17:30.710 INFO:journalctl@ceph.osd.6.smithi107.stdout:Mar 08 21:17:30 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-6[59834]: 2024-03-08T21:17:30.279+0000 7f6132600700 -1 osd.6 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:53.344721+0000 front 2024-03-08T21:16:53.344770+0000 (oldest deadline 2024-03-08T21:17:15.644922+0000) 2024-03-08T21:17:30.710 INFO:journalctl@ceph.osd.6.smithi107.stdout:Mar 08 21:17:30 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-6[59834]: 2024-03-08T21:17:30.279+0000 7f6132600700 -1 osd.6 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:17:02.047607+0000 front 2024-03-08T21:17:02.047791+0000 (oldest deadline 2024-03-08T21:17:26.747841+0000) 2024-03-08T21:17:30.710 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:17:30 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:17:30.327+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:40.479965+0000 front 2024-03-08T21:16:40.480241+0000 (oldest deadline 2024-03-08T21:17:06.379830+0000) 2024-03-08T21:17:30.711 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:17:30 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:17:30.327+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:53.283252+0000 front 2024-03-08T21:16:53.283300+0000 (oldest deadline 2024-03-08T21:17:17.383498+0000) 2024-03-08T21:17:30.711 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:17:30 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:17:30.327+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:17:03.088480+0000 front 2024-03-08T21:17:03.088666+0000 (oldest deadline 2024-03-08T21:17:25.388404+0000) 2024-03-08T21:17:30.851 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:17:30 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:17:30.525+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:42.614355+0000 front 2024-03-08T21:16:42.614271+0000 (oldest deadline 2024-03-08T21:17:07.914284+0000) 2024-03-08T21:17:30.851 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:17:30 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:17:30.525+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:52.016503+0000 front 2024-03-08T21:16:52.017017+0000 (oldest deadline 2024-03-08T21:17:16.116550+0000) 2024-03-08T21:17:30.851 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:17:30 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:17:30.525+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:16:57.818698+0000 front 2024-03-08T21:16:57.819201+0000 (oldest deadline 2024-03-08T21:17:26.619635+0000) 2024-03-08T21:17:31.000 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 08 21:17:30 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-7[53366]: 2024-03-08T21:17:30.687+0000 7f48e9a00700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:53.119608+0000 front 2024-03-08T21:16:53.119745+0000 (oldest deadline 2024-03-08T21:17:17.819760+0000) 2024-03-08T21:17:31.000 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 08 21:17:30 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-7[53366]: 2024-03-08T21:17:30.687+0000 7f48e9a00700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:17:01.822810+0000 front 2024-03-08T21:17:01.822741+0000 (oldest deadline 2024-03-08T21:17:27.122609+0000) 2024-03-08T21:17:31.121 INFO:journalctl@ceph.osd.5.smithi204.stdout:Mar 08 21:17:30 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-5[47140]: 2024-03-08T21:17:30.874+0000 7f038c200700 -1 osd.5 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:43.793658+0000 front 2024-03-08T21:16:43.793735+0000 (oldest deadline 2024-03-08T21:17:07.893906+0000) 2024-03-08T21:17:31.121 INFO:journalctl@ceph.osd.5.smithi204.stdout:Mar 08 21:17:30 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-5[47140]: 2024-03-08T21:17:30.874+0000 7f038c200700 -1 osd.5 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:53.697831+0000 front 2024-03-08T21:16:53.697757+0000 (oldest deadline 2024-03-08T21:17:14.797782+0000) 2024-03-08T21:17:31.121 INFO:journalctl@ceph.osd.5.smithi204.stdout:Mar 08 21:17:30 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-5[47140]: 2024-03-08T21:17:30.874+0000 7f038c200700 -1 osd.5 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:17:00.000861+0000 front 2024-03-08T21:17:00.001127+0000 (oldest deadline 2024-03-08T21:17:25.900961+0000) 2024-03-08T21:17:31.258 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:17:31 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:17:31.023+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:44.858572+0000 front 2024-03-08T21:16:44.858829+0000 (oldest deadline 2024-03-08T21:17:06.558274+0000) 2024-03-08T21:17:31.258 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:17:31 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:17:31.023+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:52.961888+0000 front 2024-03-08T21:16:52.961762+0000 (oldest deadline 2024-03-08T21:17:18.862022+0000) 2024-03-08T21:17:31.258 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:17:31 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:17:31.023+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:17:01.764390+0000 front 2024-03-08T21:17:01.764438+0000 (oldest deadline 2024-03-08T21:17:24.064211+0000) 2024-03-08T21:17:31.448 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 08 21:17:31 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-8[53709]: 2024-03-08T21:17:31.216+0000 7f3fae200700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:43.996391+0000 front 2024-03-08T21:16:43.996418+0000 (oldest deadline 2024-03-08T21:17:09.296545+0000) 2024-03-08T21:17:31.448 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 08 21:17:31 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-8[53709]: 2024-03-08T21:17:31.216+0000 7f3fae200700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:52.199334+0000 front 2024-03-08T21:16:52.199113+0000 (oldest deadline 2024-03-08T21:17:16.899180+0000) 2024-03-08T21:17:31.448 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 08 21:17:31 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-8[53709]: 2024-03-08T21:17:31.216+0000 7f3fae200700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:17:00.401255+0000 front 2024-03-08T21:17:00.401120+0000 (oldest deadline 2024-03-08T21:17:25.701340+0000) 2024-03-08T21:17:31.710 INFO:journalctl@ceph.osd.6.smithi107.stdout:Mar 08 21:17:31 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-6[59834]: 2024-03-08T21:17:31.278+0000 7f6132600700 -1 osd.6 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:44.641569+0000 front 2024-03-08T21:16:44.641810+0000 (oldest deadline 2024-03-08T21:17:07.541743+0000) 2024-03-08T21:17:31.710 INFO:journalctl@ceph.osd.6.smithi107.stdout:Mar 08 21:17:31 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-6[59834]: 2024-03-08T21:17:31.278+0000 7f6132600700 -1 osd.6 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:53.344721+0000 front 2024-03-08T21:16:53.344770+0000 (oldest deadline 2024-03-08T21:17:15.644922+0000) 2024-03-08T21:17:31.710 INFO:journalctl@ceph.osd.6.smithi107.stdout:Mar 08 21:17:31 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-6[59834]: 2024-03-08T21:17:31.278+0000 7f6132600700 -1 osd.6 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:17:02.047607+0000 front 2024-03-08T21:17:02.047791+0000 (oldest deadline 2024-03-08T21:17:26.747841+0000) 2024-03-08T21:17:31.711 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:17:31 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:17:31.293+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:40.479965+0000 front 2024-03-08T21:16:40.480241+0000 (oldest deadline 2024-03-08T21:17:06.379830+0000) 2024-03-08T21:17:31.711 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:17:31 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:17:31.293+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:53.283252+0000 front 2024-03-08T21:16:53.283300+0000 (oldest deadline 2024-03-08T21:17:17.383498+0000) 2024-03-08T21:17:31.711 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:17:31 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:17:31.293+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:17:03.088480+0000 front 2024-03-08T21:17:03.088666+0000 (oldest deadline 2024-03-08T21:17:25.388404+0000) 2024-03-08T21:17:31.795 INFO:journalctl@ceph.osd.5.smithi204.stdout:Mar 08 21:17:31 smithi204 podman[73722]: 2024-03-08 21:17:31.472664192 +0000 UTC m=+5.473404201 container died 1a43b081163ee85a7f3e1eec5962a162fcd85b53d201445e48a70a223b92bb94 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=rishabh-2024mar8-7d63482, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , ceph=True) 2024-03-08T21:17:31.796 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:17:31 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:17:31.522+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:42.614355+0000 front 2024-03-08T21:16:42.614271+0000 (oldest deadline 2024-03-08T21:17:07.914284+0000) 2024-03-08T21:17:31.796 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:17:31 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:17:31.522+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:52.016503+0000 front 2024-03-08T21:16:52.017017+0000 (oldest deadline 2024-03-08T21:17:16.116550+0000) 2024-03-08T21:17:31.796 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:17:31 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:17:31.522+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:16:57.818698+0000 front 2024-03-08T21:16:57.819201+0000 (oldest deadline 2024-03-08T21:17:26.619635+0000) 2024-03-08T21:17:32.008 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 08 21:17:31 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-7[53366]: 2024-03-08T21:17:31.645+0000 7f48e9a00700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:53.119608+0000 front 2024-03-08T21:16:53.119745+0000 (oldest deadline 2024-03-08T21:17:17.819760+0000) 2024-03-08T21:17:32.008 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 08 21:17:31 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-7[53366]: 2024-03-08T21:17:31.646+0000 7f48e9a00700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:17:01.822810+0000 front 2024-03-08T21:17:01.822741+0000 (oldest deadline 2024-03-08T21:17:27.122609+0000) 2024-03-08T21:17:32.121 INFO:journalctl@ceph.osd.5.smithi204.stdout:Mar 08 21:17:31 smithi204 podman[73722]: 2024-03-08 21:17:31.94851505 +0000 UTC m=+5.949255080 container cleanup 1a43b081163ee85a7f3e1eec5962a162fcd85b53d201445e48a70a223b92bb94 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-5, RELEASE=rishabh-2024mar8-7d63482, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True) 2024-03-08T21:17:32.121 INFO:journalctl@ceph.osd.5.smithi204.stdout:Mar 08 21:17:31 smithi204 bash[73722]: ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-5 2024-03-08T21:17:32.356 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:17:32 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:17:32.034+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:44.858572+0000 front 2024-03-08T21:16:44.858829+0000 (oldest deadline 2024-03-08T21:17:06.558274+0000) 2024-03-08T21:17:32.356 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:17:32 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:17:32.034+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:52.961888+0000 front 2024-03-08T21:16:52.961762+0000 (oldest deadline 2024-03-08T21:17:18.862022+0000) 2024-03-08T21:17:32.356 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:17:32 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:17:32.034+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:17:01.764390+0000 front 2024-03-08T21:17:01.764438+0000 (oldest deadline 2024-03-08T21:17:24.064211+0000) 2024-03-08T21:17:32.456 INFO:journalctl@ceph.osd.5.smithi204.stdout:Mar 08 21:17:32 smithi204 podman[73740]: 2024-03-08 21:17:32.166727731 +0000 UTC m=+0.678400114 container remove 1a43b081163ee85a7f3e1eec5962a162fcd85b53d201445e48a70a223b92bb94 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-5, RELEASE=rishabh-2024mar8-7d63482, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-03-08T21:17:32.456 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 08 21:17:32 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-8[53709]: 2024-03-08T21:17:32.263+0000 7f3fae200700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:43.996391+0000 front 2024-03-08T21:16:43.996418+0000 (oldest deadline 2024-03-08T21:17:09.296545+0000) 2024-03-08T21:17:32.456 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 08 21:17:32 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-8[53709]: 2024-03-08T21:17:32.263+0000 7f3fae200700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:52.199334+0000 front 2024-03-08T21:16:52.199113+0000 (oldest deadline 2024-03-08T21:17:16.899180+0000) 2024-03-08T21:17:32.456 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 08 21:17:32 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-8[53709]: 2024-03-08T21:17:32.263+0000 7f3fae200700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:17:00.401255+0000 front 2024-03-08T21:17:00.401120+0000 (oldest deadline 2024-03-08T21:17:25.701340+0000) 2024-03-08T21:17:32.710 INFO:journalctl@ceph.osd.6.smithi107.stdout:Mar 08 21:17:32 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-6[59834]: 2024-03-08T21:17:32.289+0000 7f6132600700 -1 osd.6 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:44.641569+0000 front 2024-03-08T21:16:44.641810+0000 (oldest deadline 2024-03-08T21:17:07.541743+0000) 2024-03-08T21:17:32.710 INFO:journalctl@ceph.osd.6.smithi107.stdout:Mar 08 21:17:32 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-6[59834]: 2024-03-08T21:17:32.289+0000 7f6132600700 -1 osd.6 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:53.344721+0000 front 2024-03-08T21:16:53.344770+0000 (oldest deadline 2024-03-08T21:17:15.644922+0000) 2024-03-08T21:17:32.710 INFO:journalctl@ceph.osd.6.smithi107.stdout:Mar 08 21:17:32 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-6[59834]: 2024-03-08T21:17:32.289+0000 7f6132600700 -1 osd.6 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:17:02.047607+0000 front 2024-03-08T21:17:02.047791+0000 (oldest deadline 2024-03-08T21:17:26.747841+0000) 2024-03-08T21:17:32.711 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:17:32 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:17:32.320+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:40.479965+0000 front 2024-03-08T21:16:40.480241+0000 (oldest deadline 2024-03-08T21:17:06.379830+0000) 2024-03-08T21:17:32.711 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:17:32 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:17:32.320+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:53.283252+0000 front 2024-03-08T21:16:53.283300+0000 (oldest deadline 2024-03-08T21:17:17.383498+0000) 2024-03-08T21:17:32.711 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:17:32 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:17:32.320+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:17:03.088480+0000 front 2024-03-08T21:17:03.088666+0000 (oldest deadline 2024-03-08T21:17:25.388404+0000) 2024-03-08T21:17:32.757 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 08 21:17:32 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-7[53366]: 2024-03-08T21:17:32.620+0000 7f48e9a00700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:53.119608+0000 front 2024-03-08T21:16:53.119745+0000 (oldest deadline 2024-03-08T21:17:17.819760+0000) 2024-03-08T21:17:32.758 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 08 21:17:32 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-7[53366]: 2024-03-08T21:17:32.620+0000 7f48e9a00700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:17:01.822810+0000 front 2024-03-08T21:17:01.822741+0000 (oldest deadline 2024-03-08T21:17:27.122609+0000) 2024-03-08T21:17:32.871 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:17:32 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:17:32.479+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:42.614355+0000 front 2024-03-08T21:16:42.614271+0000 (oldest deadline 2024-03-08T21:17:07.914284+0000) 2024-03-08T21:17:32.871 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:17:32 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:17:32.479+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:52.016503+0000 front 2024-03-08T21:16:52.017017+0000 (oldest deadline 2024-03-08T21:17:16.116550+0000) 2024-03-08T21:17:32.871 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:17:32 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:17:32.479+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:16:57.818698+0000 front 2024-03-08T21:16:57.819201+0000 (oldest deadline 2024-03-08T21:17:26.619635+0000) 2024-03-08T21:17:33.230 INFO:journalctl@ceph.osd.5.smithi204.stdout:Mar 08 21:17:32 smithi204 podman[73828]: 2024-03-08 21:17:32.853271716 +0000 UTC m=+0.033230757 image pull a272146cb810b1fd4a53627f019472132e30966d5c25b01bfc44a001295f1249 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd 2024-03-08T21:17:33.231 INFO:journalctl@ceph.osd.5.smithi204.stdout:Mar 08 21:17:32 smithi204 podman[73828]: 2024-03-08 21:17:32.967694815 +0000 UTC m=+0.147653875 container create 073d447387efa0486b39d7e09fd6c92dd0b1cab12daccb9d226cb91d97823a63 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-5-deactivate, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=rishabh-2024mar8-7d63482, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-08T21:17:33.231 INFO:journalctl@ceph.osd.5.smithi204.stdout:Mar 08 21:17:33 smithi204 podman[73828]: 2024-03-08 21:17:33.254790455 +0000 UTC m=+0.434749539 container init 073d447387efa0486b39d7e09fd6c92dd0b1cab12daccb9d226cb91d97823a63 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-5-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=rishabh-2024mar8-7d63482) 2024-03-08T21:17:33.258 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:17:33 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:17:33.012+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:44.858572+0000 front 2024-03-08T21:16:44.858829+0000 (oldest deadline 2024-03-08T21:17:06.558274+0000) 2024-03-08T21:17:33.258 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:17:33 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:17:33.012+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:52.961888+0000 front 2024-03-08T21:16:52.961762+0000 (oldest deadline 2024-03-08T21:17:18.862022+0000) 2024-03-08T21:17:33.258 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:17:33 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:17:33.012+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:17:01.764390+0000 front 2024-03-08T21:17:01.764438+0000 (oldest deadline 2024-03-08T21:17:24.064211+0000) 2024-03-08T21:17:33.621 INFO:journalctl@ceph.osd.5.smithi204.stdout:Mar 08 21:17:33 smithi204 podman[73828]: 2024-03-08 21:17:33.267031678 +0000 UTC m=+0.446990762 container start 073d447387efa0486b39d7e09fd6c92dd0b1cab12daccb9d226cb91d97823a63 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-5-deactivate, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=rishabh-2024mar8-7d63482, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2) 2024-03-08T21:17:33.621 INFO:journalctl@ceph.osd.5.smithi204.stdout:Mar 08 21:17:33 smithi204 podman[73828]: 2024-03-08 21:17:33.370712526 +0000 UTC m=+0.550671585 container attach 073d447387efa0486b39d7e09fd6c92dd0b1cab12daccb9d226cb91d97823a63 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-5-deactivate, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, RELEASE=rishabh-2024mar8-7d63482, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-03-08T21:17:33.621 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:17:33 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:17:33.480+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:42.614355+0000 front 2024-03-08T21:16:42.614271+0000 (oldest deadline 2024-03-08T21:17:07.914284+0000) 2024-03-08T21:17:33.621 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:17:33 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:17:33.480+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:52.016503+0000 front 2024-03-08T21:16:52.017017+0000 (oldest deadline 2024-03-08T21:17:16.116550+0000) 2024-03-08T21:17:33.622 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:17:33 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:17:33.480+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:16:57.818698+0000 front 2024-03-08T21:16:57.819201+0000 (oldest deadline 2024-03-08T21:17:26.619635+0000) 2024-03-08T21:17:33.622 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 08 21:17:33 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-8[53709]: 2024-03-08T21:17:33.273+0000 7f3fae200700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:43.996391+0000 front 2024-03-08T21:16:43.996418+0000 (oldest deadline 2024-03-08T21:17:09.296545+0000) 2024-03-08T21:17:33.622 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 08 21:17:33 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-8[53709]: 2024-03-08T21:17:33.273+0000 7f3fae200700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:52.199334+0000 front 2024-03-08T21:16:52.199113+0000 (oldest deadline 2024-03-08T21:17:16.899180+0000) 2024-03-08T21:17:33.622 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 08 21:17:33 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-8[53709]: 2024-03-08T21:17:33.273+0000 7f3fae200700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:17:00.401255+0000 front 2024-03-08T21:17:00.401120+0000 (oldest deadline 2024-03-08T21:17:25.701340+0000) 2024-03-08T21:17:33.710 INFO:journalctl@ceph.osd.6.smithi107.stdout:Mar 08 21:17:33 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-6[59834]: 2024-03-08T21:17:33.329+0000 7f6132600700 -1 osd.6 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:44.641569+0000 front 2024-03-08T21:16:44.641810+0000 (oldest deadline 2024-03-08T21:17:07.541743+0000) 2024-03-08T21:17:33.710 INFO:journalctl@ceph.osd.6.smithi107.stdout:Mar 08 21:17:33 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-6[59834]: 2024-03-08T21:17:33.329+0000 7f6132600700 -1 osd.6 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:53.344721+0000 front 2024-03-08T21:16:53.344770+0000 (oldest deadline 2024-03-08T21:17:15.644922+0000) 2024-03-08T21:17:33.710 INFO:journalctl@ceph.osd.6.smithi107.stdout:Mar 08 21:17:33 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-6[59834]: 2024-03-08T21:17:33.329+0000 7f6132600700 -1 osd.6 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:17:02.047607+0000 front 2024-03-08T21:17:02.047791+0000 (oldest deadline 2024-03-08T21:17:26.747841+0000) 2024-03-08T21:17:33.711 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:17:33 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:17:33.285+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:40.479965+0000 front 2024-03-08T21:16:40.480241+0000 (oldest deadline 2024-03-08T21:17:06.379830+0000) 2024-03-08T21:17:33.711 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:17:33 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:17:33.285+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:53.283252+0000 front 2024-03-08T21:16:53.283300+0000 (oldest deadline 2024-03-08T21:17:17.383498+0000) 2024-03-08T21:17:33.711 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:17:33 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:17:33.285+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:17:03.088480+0000 front 2024-03-08T21:17:03.088666+0000 (oldest deadline 2024-03-08T21:17:25.388404+0000) 2024-03-08T21:17:34.008 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 08 21:17:33 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-7[53366]: 2024-03-08T21:17:33.629+0000 7f48e9a00700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:53.119608+0000 front 2024-03-08T21:16:53.119745+0000 (oldest deadline 2024-03-08T21:17:17.819760+0000) 2024-03-08T21:17:34.008 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 08 21:17:33 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-7[53366]: 2024-03-08T21:17:33.629+0000 7f48e9a00700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:17:01.822810+0000 front 2024-03-08T21:17:01.822741+0000 (oldest deadline 2024-03-08T21:17:27.122609+0000) 2024-03-08T21:17:34.008 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 08 21:17:33 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-7[53366]: 2024-03-08T21:17:33.629+0000 7f48e9a00700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-03-08T21:17:08.725647+0000 front 2024-03-08T21:17:08.725828+0000 (oldest deadline 2024-03-08T21:17:33.425499+0000) 2024-03-08T21:17:34.096 INFO:journalctl@ceph.osd.5.smithi204.stdout:Mar 08 21:17:33 smithi204 podman[73828]: 2024-03-08 21:17:33.709532541 +0000 UTC m=+0.889491640 container died 073d447387efa0486b39d7e09fd6c92dd0b1cab12daccb9d226cb91d97823a63 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-5-deactivate, RELEASE=rishabh-2024mar8-7d63482, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0) 2024-03-08T21:17:34.371 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 08 21:17:34 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-8[53709]: 2024-03-08T21:17:34.279+0000 7f3fae200700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:43.996391+0000 front 2024-03-08T21:16:43.996418+0000 (oldest deadline 2024-03-08T21:17:09.296545+0000) 2024-03-08T21:17:34.371 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 08 21:17:34 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-8[53709]: 2024-03-08T21:17:34.279+0000 7f3fae200700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:52.199334+0000 front 2024-03-08T21:16:52.199113+0000 (oldest deadline 2024-03-08T21:17:16.899180+0000) 2024-03-08T21:17:34.372 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 08 21:17:34 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-8[53709]: 2024-03-08T21:17:34.279+0000 7f3fae200700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:17:00.401255+0000 front 2024-03-08T21:17:00.401120+0000 (oldest deadline 2024-03-08T21:17:25.701340+0000) 2024-03-08T21:17:34.508 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:17:34 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:17:34.052+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:44.858572+0000 front 2024-03-08T21:16:44.858829+0000 (oldest deadline 2024-03-08T21:17:06.558274+0000) 2024-03-08T21:17:34.508 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:17:34 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:17:34.052+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:52.961888+0000 front 2024-03-08T21:16:52.961762+0000 (oldest deadline 2024-03-08T21:17:18.862022+0000) 2024-03-08T21:17:34.508 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:17:34 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:17:34.052+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:17:01.764390+0000 front 2024-03-08T21:17:01.764438+0000 (oldest deadline 2024-03-08T21:17:24.064211+0000) 2024-03-08T21:17:34.539 DEBUG:teuthology.orchestra.run.smithi204:> sudo pkill -f 'journalctl -f -n 0 -u ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715@osd.5.service' 2024-03-08T21:17:34.664 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:17:34 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:17:34.441+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:42.614355+0000 front 2024-03-08T21:16:42.614271+0000 (oldest deadline 2024-03-08T21:17:07.914284+0000) 2024-03-08T21:17:34.665 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:17:34 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:17:34.441+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:52.016503+0000 front 2024-03-08T21:16:52.017017+0000 (oldest deadline 2024-03-08T21:17:16.116550+0000) 2024-03-08T21:17:34.665 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:17:34 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:17:34.441+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:16:57.818698+0000 front 2024-03-08T21:16:57.819201+0000 (oldest deadline 2024-03-08T21:17:26.619635+0000) 2024-03-08T21:17:34.665 INFO:journalctl@ceph.osd.5.smithi204.stdout:Mar 08 21:17:34 smithi204 podman[73854]: 2024-03-08 21:17:34.517741096 +0000 UTC m=+0.791053280 container remove 073d447387efa0486b39d7e09fd6c92dd0b1cab12daccb9d226cb91d97823a63 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-5-deactivate, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=rishabh-2024mar8-7d63482, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-08T21:17:34.665 INFO:journalctl@ceph.osd.5.smithi204.stdout:Mar 08 21:17:34 smithi204 systemd[1]: ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715@osd.5.service: Deactivated successfully. 2024-03-08T21:17:34.665 INFO:journalctl@ceph.osd.5.smithi204.stdout:Mar 08 21:17:34 smithi204 systemd[1]: Stopped Ceph osd.5 for dc20f0f8-dd8c-11ee-95c6-87774f69a715. 2024-03-08T21:17:34.665 INFO:journalctl@ceph.osd.5.smithi204.stdout:Mar 08 21:17:34 smithi204 systemd[1]: ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715@osd.5.service: Consumed 53.137s CPU time. 2024-03-08T21:17:34.703 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-08T21:17:34.703 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-03-08T21:17:34.703 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-03-08T21:17:34.703 DEBUG:teuthology.orchestra.run.smithi107:> sudo systemctl stop ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715@osd.6 2024-03-08T21:17:34.712 INFO:journalctl@ceph.osd.6.smithi107.stdout:Mar 08 21:17:34 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-6[59834]: 2024-03-08T21:17:34.322+0000 7f6132600700 -1 osd.6 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:44.641569+0000 front 2024-03-08T21:16:44.641810+0000 (oldest deadline 2024-03-08T21:17:07.541743+0000) 2024-03-08T21:17:34.712 INFO:journalctl@ceph.osd.6.smithi107.stdout:Mar 08 21:17:34 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-6[59834]: 2024-03-08T21:17:34.322+0000 7f6132600700 -1 osd.6 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:53.344721+0000 front 2024-03-08T21:16:53.344770+0000 (oldest deadline 2024-03-08T21:17:15.644922+0000) 2024-03-08T21:17:34.712 INFO:journalctl@ceph.osd.6.smithi107.stdout:Mar 08 21:17:34 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-6[59834]: 2024-03-08T21:17:34.322+0000 7f6132600700 -1 osd.6 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:17:02.047607+0000 front 2024-03-08T21:17:02.047791+0000 (oldest deadline 2024-03-08T21:17:26.747841+0000) 2024-03-08T21:17:34.713 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:17:34 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:17:34.303+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:40.479965+0000 front 2024-03-08T21:16:40.480241+0000 (oldest deadline 2024-03-08T21:17:06.379830+0000) 2024-03-08T21:17:34.713 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:17:34 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:17:34.303+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:53.283252+0000 front 2024-03-08T21:16:53.283300+0000 (oldest deadline 2024-03-08T21:17:17.383498+0000) 2024-03-08T21:17:34.713 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:17:34 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:17:34.303+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:17:03.088480+0000 front 2024-03-08T21:17:03.088666+0000 (oldest deadline 2024-03-08T21:17:25.388404+0000) 2024-03-08T21:17:34.713 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:17:34 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:17:34.303+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-03-08T21:17:10.690024+0000 front 2024-03-08T21:17:10.690629+0000 (oldest deadline 2024-03-08T21:17:34.189860+0000) 2024-03-08T21:17:35.008 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 08 21:17:34 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-7[53366]: 2024-03-08T21:17:34.622+0000 7f48e9a00700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:53.119608+0000 front 2024-03-08T21:16:53.119745+0000 (oldest deadline 2024-03-08T21:17:17.819760+0000) 2024-03-08T21:17:35.008 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 08 21:17:34 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-7[53366]: 2024-03-08T21:17:34.623+0000 7f48e9a00700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:17:01.822810+0000 front 2024-03-08T21:17:01.822741+0000 (oldest deadline 2024-03-08T21:17:27.122609+0000) 2024-03-08T21:17:35.008 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 08 21:17:34 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-7[53366]: 2024-03-08T21:17:34.623+0000 7f48e9a00700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-03-08T21:17:08.725647+0000 front 2024-03-08T21:17:08.725828+0000 (oldest deadline 2024-03-08T21:17:33.425499+0000) 2024-03-08T21:17:35.210 INFO:journalctl@ceph.osd.6.smithi107.stdout:Mar 08 21:17:34 smithi107 systemd[1]: Stopping Ceph osd.6 for dc20f0f8-dd8c-11ee-95c6-87774f69a715... 2024-03-08T21:17:35.508 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:17:35 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:17:35.037+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:44.858572+0000 front 2024-03-08T21:16:44.858829+0000 (oldest deadline 2024-03-08T21:17:06.558274+0000) 2024-03-08T21:17:35.508 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:17:35 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:17:35.037+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:52.961888+0000 front 2024-03-08T21:16:52.961762+0000 (oldest deadline 2024-03-08T21:17:18.862022+0000) 2024-03-08T21:17:35.508 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:17:35 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:17:35.037+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:17:01.764390+0000 front 2024-03-08T21:17:01.764438+0000 (oldest deadline 2024-03-08T21:17:24.064211+0000) 2024-03-08T21:17:35.508 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:17:35 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:17:35.037+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-03-08T21:17:10.869088+0000 front 2024-03-08T21:17:10.869147+0000 (oldest deadline 2024-03-08T21:17:34.969351+0000) 2024-03-08T21:17:35.621 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 08 21:17:35 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-8[53709]: 2024-03-08T21:17:35.323+0000 7f3fae200700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:43.996391+0000 front 2024-03-08T21:16:43.996418+0000 (oldest deadline 2024-03-08T21:17:09.296545+0000) 2024-03-08T21:17:35.621 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 08 21:17:35 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-8[53709]: 2024-03-08T21:17:35.323+0000 7f3fae200700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:52.199334+0000 front 2024-03-08T21:16:52.199113+0000 (oldest deadline 2024-03-08T21:17:16.899180+0000) 2024-03-08T21:17:35.621 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 08 21:17:35 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-8[53709]: 2024-03-08T21:17:35.323+0000 7f3fae200700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:17:00.401255+0000 front 2024-03-08T21:17:00.401120+0000 (oldest deadline 2024-03-08T21:17:25.701340+0000) 2024-03-08T21:17:35.621 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:17:35 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:17:35.456+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:42.614355+0000 front 2024-03-08T21:16:42.614271+0000 (oldest deadline 2024-03-08T21:17:07.914284+0000) 2024-03-08T21:17:35.621 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:17:35 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:17:35.456+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:52.016503+0000 front 2024-03-08T21:16:52.017017+0000 (oldest deadline 2024-03-08T21:17:16.116550+0000) 2024-03-08T21:17:35.622 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:17:35 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:17:35.456+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:16:57.818698+0000 front 2024-03-08T21:16:57.819201+0000 (oldest deadline 2024-03-08T21:17:26.619635+0000) 2024-03-08T21:17:35.710 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:17:35 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:17:35.272+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:40.479965+0000 front 2024-03-08T21:16:40.480241+0000 (oldest deadline 2024-03-08T21:17:06.379830+0000) 2024-03-08T21:17:35.710 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:17:35 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:17:35.272+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:53.283252+0000 front 2024-03-08T21:16:53.283300+0000 (oldest deadline 2024-03-08T21:17:17.383498+0000) 2024-03-08T21:17:35.711 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:17:35 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:17:35.272+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:17:03.088480+0000 front 2024-03-08T21:17:03.088666+0000 (oldest deadline 2024-03-08T21:17:25.388404+0000) 2024-03-08T21:17:35.711 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:17:35 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:17:35.272+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-03-08T21:17:10.690024+0000 front 2024-03-08T21:17:10.690629+0000 (oldest deadline 2024-03-08T21:17:34.189860+0000) 2024-03-08T21:17:35.711 INFO:journalctl@ceph.osd.6.smithi107.stdout:Mar 08 21:17:35 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-6[59834]: 2024-03-08T21:17:35.311+0000 7f613be00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-08T21:17:35.711 INFO:journalctl@ceph.osd.6.smithi107.stdout:Mar 08 21:17:35 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-6[59834]: 2024-03-08T21:17:35.311+0000 7f613be00700 -1 osd.6 115 *** Got signal Terminated *** 2024-03-08T21:17:35.711 INFO:journalctl@ceph.osd.6.smithi107.stdout:Mar 08 21:17:35 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-6[59834]: 2024-03-08T21:17:35.311+0000 7f613be00700 -1 osd.6 115 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-08T21:17:35.711 INFO:journalctl@ceph.osd.6.smithi107.stdout:Mar 08 21:17:35 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-6[59834]: 2024-03-08T21:17:35.359+0000 7f6132600700 -1 osd.6 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:44.641569+0000 front 2024-03-08T21:16:44.641810+0000 (oldest deadline 2024-03-08T21:17:07.541743+0000) 2024-03-08T21:17:35.711 INFO:journalctl@ceph.osd.6.smithi107.stdout:Mar 08 21:17:35 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-6[59834]: 2024-03-08T21:17:35.359+0000 7f6132600700 -1 osd.6 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:53.344721+0000 front 2024-03-08T21:16:53.344770+0000 (oldest deadline 2024-03-08T21:17:15.644922+0000) 2024-03-08T21:17:35.711 INFO:journalctl@ceph.osd.6.smithi107.stdout:Mar 08 21:17:35 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-6[59834]: 2024-03-08T21:17:35.359+0000 7f6132600700 -1 osd.6 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:17:02.047607+0000 front 2024-03-08T21:17:02.047791+0000 (oldest deadline 2024-03-08T21:17:26.747841+0000) 2024-03-08T21:17:36.008 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 08 21:17:35 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-7[53366]: 2024-03-08T21:17:35.637+0000 7f48e9a00700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:53.119608+0000 front 2024-03-08T21:16:53.119745+0000 (oldest deadline 2024-03-08T21:17:17.819760+0000) 2024-03-08T21:17:36.008 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 08 21:17:35 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-7[53366]: 2024-03-08T21:17:35.637+0000 7f48e9a00700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:17:01.822810+0000 front 2024-03-08T21:17:01.822741+0000 (oldest deadline 2024-03-08T21:17:27.122609+0000) 2024-03-08T21:17:36.008 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 08 21:17:35 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-7[53366]: 2024-03-08T21:17:35.637+0000 7f48e9a00700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-03-08T21:17:08.725647+0000 front 2024-03-08T21:17:08.725828+0000 (oldest deadline 2024-03-08T21:17:33.425499+0000) 2024-03-08T21:17:36.373 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 08 21:17:36 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-8[53709]: 2024-03-08T21:17:36.304+0000 7f3fae200700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:43.996391+0000 front 2024-03-08T21:16:43.996418+0000 (oldest deadline 2024-03-08T21:17:09.296545+0000) 2024-03-08T21:17:36.373 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 08 21:17:36 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-8[53709]: 2024-03-08T21:17:36.304+0000 7f3fae200700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:52.199334+0000 front 2024-03-08T21:16:52.199113+0000 (oldest deadline 2024-03-08T21:17:16.899180+0000) 2024-03-08T21:17:36.374 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 08 21:17:36 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-8[53709]: 2024-03-08T21:17:36.304+0000 7f3fae200700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:17:00.401255+0000 front 2024-03-08T21:17:00.401120+0000 (oldest deadline 2024-03-08T21:17:25.701340+0000) 2024-03-08T21:17:36.508 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:17:36 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:17:36.071+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:44.858572+0000 front 2024-03-08T21:16:44.858829+0000 (oldest deadline 2024-03-08T21:17:06.558274+0000) 2024-03-08T21:17:36.508 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:17:36 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:17:36.071+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:52.961888+0000 front 2024-03-08T21:16:52.961762+0000 (oldest deadline 2024-03-08T21:17:18.862022+0000) 2024-03-08T21:17:36.508 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:17:36 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:17:36.071+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:17:01.764390+0000 front 2024-03-08T21:17:01.764438+0000 (oldest deadline 2024-03-08T21:17:24.064211+0000) 2024-03-08T21:17:36.508 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:17:36 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:17:36.071+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-03-08T21:17:10.869088+0000 front 2024-03-08T21:17:10.869147+0000 (oldest deadline 2024-03-08T21:17:34.969351+0000) 2024-03-08T21:17:36.710 INFO:journalctl@ceph.osd.6.smithi107.stdout:Mar 08 21:17:36 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-6[59834]: 2024-03-08T21:17:36.354+0000 7f6132600700 -1 osd.6 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:44.641569+0000 front 2024-03-08T21:16:44.641810+0000 (oldest deadline 2024-03-08T21:17:07.541743+0000) 2024-03-08T21:17:36.710 INFO:journalctl@ceph.osd.6.smithi107.stdout:Mar 08 21:17:36 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-6[59834]: 2024-03-08T21:17:36.354+0000 7f6132600700 -1 osd.6 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:53.344721+0000 front 2024-03-08T21:16:53.344770+0000 (oldest deadline 2024-03-08T21:17:15.644922+0000) 2024-03-08T21:17:36.711 INFO:journalctl@ceph.osd.6.smithi107.stdout:Mar 08 21:17:36 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-6[59834]: 2024-03-08T21:17:36.354+0000 7f6132600700 -1 osd.6 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:17:02.047607+0000 front 2024-03-08T21:17:02.047791+0000 (oldest deadline 2024-03-08T21:17:26.747841+0000) 2024-03-08T21:17:36.711 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:17:36 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:17:36.300+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:40.479965+0000 front 2024-03-08T21:16:40.480241+0000 (oldest deadline 2024-03-08T21:17:06.379830+0000) 2024-03-08T21:17:36.711 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:17:36 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:17:36.300+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:53.283252+0000 front 2024-03-08T21:16:53.283300+0000 (oldest deadline 2024-03-08T21:17:17.383498+0000) 2024-03-08T21:17:36.711 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:17:36 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:17:36.300+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:17:03.088480+0000 front 2024-03-08T21:17:03.088666+0000 (oldest deadline 2024-03-08T21:17:25.388404+0000) 2024-03-08T21:17:36.711 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:17:36 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:17:36.300+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-03-08T21:17:10.690024+0000 front 2024-03-08T21:17:10.690629+0000 (oldest deadline 2024-03-08T21:17:34.189860+0000) 2024-03-08T21:17:36.871 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:17:36 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:17:36.465+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:42.614355+0000 front 2024-03-08T21:16:42.614271+0000 (oldest deadline 2024-03-08T21:17:07.914284+0000) 2024-03-08T21:17:36.871 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:17:36 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:17:36.465+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:52.016503+0000 front 2024-03-08T21:16:52.017017+0000 (oldest deadline 2024-03-08T21:17:16.116550+0000) 2024-03-08T21:17:36.871 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:17:36 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:17:36.465+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:16:57.818698+0000 front 2024-03-08T21:16:57.819201+0000 (oldest deadline 2024-03-08T21:17:26.619635+0000) 2024-03-08T21:17:37.008 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 08 21:17:36 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-7[53366]: 2024-03-08T21:17:36.614+0000 7f48e9a00700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:53.119608+0000 front 2024-03-08T21:16:53.119745+0000 (oldest deadline 2024-03-08T21:17:17.819760+0000) 2024-03-08T21:17:37.008 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 08 21:17:36 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-7[53366]: 2024-03-08T21:17:36.614+0000 7f48e9a00700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:17:01.822810+0000 front 2024-03-08T21:17:01.822741+0000 (oldest deadline 2024-03-08T21:17:27.122609+0000) 2024-03-08T21:17:37.008 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 08 21:17:36 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-7[53366]: 2024-03-08T21:17:36.614+0000 7f48e9a00700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-03-08T21:17:08.725647+0000 front 2024-03-08T21:17:08.725828+0000 (oldest deadline 2024-03-08T21:17:33.425499+0000) 2024-03-08T21:17:37.508 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:17:37 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:17:37.093+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:44.858572+0000 front 2024-03-08T21:16:44.858829+0000 (oldest deadline 2024-03-08T21:17:06.558274+0000) 2024-03-08T21:17:37.508 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:17:37 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:17:37.093+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:52.961888+0000 front 2024-03-08T21:16:52.961762+0000 (oldest deadline 2024-03-08T21:17:18.862022+0000) 2024-03-08T21:17:37.508 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:17:37 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:17:37.093+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:17:01.764390+0000 front 2024-03-08T21:17:01.764438+0000 (oldest deadline 2024-03-08T21:17:24.064211+0000) 2024-03-08T21:17:37.508 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:17:37 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:17:37.093+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-03-08T21:17:10.869088+0000 front 2024-03-08T21:17:10.869147+0000 (oldest deadline 2024-03-08T21:17:34.969351+0000) 2024-03-08T21:17:37.621 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:17:37 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:17:37.433+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:42.614355+0000 front 2024-03-08T21:16:42.614271+0000 (oldest deadline 2024-03-08T21:17:07.914284+0000) 2024-03-08T21:17:37.622 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:17:37 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:17:37.433+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:52.016503+0000 front 2024-03-08T21:16:52.017017+0000 (oldest deadline 2024-03-08T21:17:16.116550+0000) 2024-03-08T21:17:37.622 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:17:37 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:17:37.433+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:16:57.818698+0000 front 2024-03-08T21:16:57.819201+0000 (oldest deadline 2024-03-08T21:17:26.619635+0000) 2024-03-08T21:17:37.622 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 08 21:17:37 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-8[53709]: 2024-03-08T21:17:37.344+0000 7f3fae200700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:43.996391+0000 front 2024-03-08T21:16:43.996418+0000 (oldest deadline 2024-03-08T21:17:09.296545+0000) 2024-03-08T21:17:37.622 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 08 21:17:37 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-8[53709]: 2024-03-08T21:17:37.344+0000 7f3fae200700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:52.199334+0000 front 2024-03-08T21:16:52.199113+0000 (oldest deadline 2024-03-08T21:17:16.899180+0000) 2024-03-08T21:17:37.622 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 08 21:17:37 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-8[53709]: 2024-03-08T21:17:37.344+0000 7f3fae200700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:17:00.401255+0000 front 2024-03-08T21:17:00.401120+0000 (oldest deadline 2024-03-08T21:17:25.701340+0000) 2024-03-08T21:17:37.710 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:17:37 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:17:37.293+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:40.479965+0000 front 2024-03-08T21:16:40.480241+0000 (oldest deadline 2024-03-08T21:17:06.379830+0000) 2024-03-08T21:17:37.710 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:17:37 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:17:37.293+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:53.283252+0000 front 2024-03-08T21:16:53.283300+0000 (oldest deadline 2024-03-08T21:17:17.383498+0000) 2024-03-08T21:17:37.710 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:17:37 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:17:37.293+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:17:03.088480+0000 front 2024-03-08T21:17:03.088666+0000 (oldest deadline 2024-03-08T21:17:25.388404+0000) 2024-03-08T21:17:37.711 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:17:37 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:17:37.293+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-03-08T21:17:10.690024+0000 front 2024-03-08T21:17:10.690629+0000 (oldest deadline 2024-03-08T21:17:34.189860+0000) 2024-03-08T21:17:37.711 INFO:journalctl@ceph.osd.6.smithi107.stdout:Mar 08 21:17:37 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-6[59834]: 2024-03-08T21:17:37.360+0000 7f6132600700 -1 osd.6 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:44.641569+0000 front 2024-03-08T21:16:44.641810+0000 (oldest deadline 2024-03-08T21:17:07.541743+0000) 2024-03-08T21:17:37.711 INFO:journalctl@ceph.osd.6.smithi107.stdout:Mar 08 21:17:37 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-6[59834]: 2024-03-08T21:17:37.360+0000 7f6132600700 -1 osd.6 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:53.344721+0000 front 2024-03-08T21:16:53.344770+0000 (oldest deadline 2024-03-08T21:17:15.644922+0000) 2024-03-08T21:17:37.711 INFO:journalctl@ceph.osd.6.smithi107.stdout:Mar 08 21:17:37 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-6[59834]: 2024-03-08T21:17:37.360+0000 7f6132600700 -1 osd.6 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:17:02.047607+0000 front 2024-03-08T21:17:02.047791+0000 (oldest deadline 2024-03-08T21:17:26.747841+0000) 2024-03-08T21:17:37.711 INFO:journalctl@ceph.osd.6.smithi107.stdout:Mar 08 21:17:37 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-6[59834]: 2024-03-08T21:17:37.360+0000 7f6132600700 -1 osd.6 115 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-03-08T21:17:10.849847+0000 front 2024-03-08T21:17:10.849957+0000 (oldest deadline 2024-03-08T21:17:36.750099+0000) 2024-03-08T21:17:38.008 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 08 21:17:37 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-7[53366]: 2024-03-08T21:17:37.567+0000 7f48e9a00700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:53.119608+0000 front 2024-03-08T21:16:53.119745+0000 (oldest deadline 2024-03-08T21:17:17.819760+0000) 2024-03-08T21:17:38.008 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 08 21:17:37 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-7[53366]: 2024-03-08T21:17:37.567+0000 7f48e9a00700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:17:01.822810+0000 front 2024-03-08T21:17:01.822741+0000 (oldest deadline 2024-03-08T21:17:27.122609+0000) 2024-03-08T21:17:38.008 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 08 21:17:37 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-7[53366]: 2024-03-08T21:17:37.567+0000 7f48e9a00700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-03-08T21:17:08.725647+0000 front 2024-03-08T21:17:08.725828+0000 (oldest deadline 2024-03-08T21:17:33.425499+0000) 2024-03-08T21:17:38.508 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:17:38 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:17:38.119+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:44.858572+0000 front 2024-03-08T21:16:44.858829+0000 (oldest deadline 2024-03-08T21:17:06.558274+0000) 2024-03-08T21:17:38.508 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:17:38 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:17:38.120+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:52.961888+0000 front 2024-03-08T21:16:52.961762+0000 (oldest deadline 2024-03-08T21:17:18.862022+0000) 2024-03-08T21:17:38.508 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:17:38 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:17:38.120+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:17:01.764390+0000 front 2024-03-08T21:17:01.764438+0000 (oldest deadline 2024-03-08T21:17:24.064211+0000) 2024-03-08T21:17:38.508 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:17:38 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:17:38.120+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-03-08T21:17:10.869088+0000 front 2024-03-08T21:17:10.869147+0000 (oldest deadline 2024-03-08T21:17:34.969351+0000) 2024-03-08T21:17:38.621 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 08 21:17:38 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-8[53709]: 2024-03-08T21:17:38.372+0000 7f3fae200700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:43.996391+0000 front 2024-03-08T21:16:43.996418+0000 (oldest deadline 2024-03-08T21:17:09.296545+0000) 2024-03-08T21:17:38.622 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 08 21:17:38 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-8[53709]: 2024-03-08T21:17:38.372+0000 7f3fae200700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:52.199334+0000 front 2024-03-08T21:16:52.199113+0000 (oldest deadline 2024-03-08T21:17:16.899180+0000) 2024-03-08T21:17:38.622 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 08 21:17:38 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-8[53709]: 2024-03-08T21:17:38.372+0000 7f3fae200700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:17:00.401255+0000 front 2024-03-08T21:17:00.401120+0000 (oldest deadline 2024-03-08T21:17:25.701340+0000) 2024-03-08T21:17:38.622 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:17:38 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:17:38.479+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:42.614355+0000 front 2024-03-08T21:16:42.614271+0000 (oldest deadline 2024-03-08T21:17:07.914284+0000) 2024-03-08T21:17:38.622 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:17:38 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:17:38.479+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:52.016503+0000 front 2024-03-08T21:16:52.017017+0000 (oldest deadline 2024-03-08T21:17:16.116550+0000) 2024-03-08T21:17:38.622 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:17:38 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:17:38.479+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:16:57.818698+0000 front 2024-03-08T21:16:57.819201+0000 (oldest deadline 2024-03-08T21:17:26.619635+0000) 2024-03-08T21:17:38.711 INFO:journalctl@ceph.osd.6.smithi107.stdout:Mar 08 21:17:38 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-6[59834]: 2024-03-08T21:17:38.406+0000 7f6132600700 -1 osd.6 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:44.641569+0000 front 2024-03-08T21:16:44.641810+0000 (oldest deadline 2024-03-08T21:17:07.541743+0000) 2024-03-08T21:17:38.711 INFO:journalctl@ceph.osd.6.smithi107.stdout:Mar 08 21:17:38 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-6[59834]: 2024-03-08T21:17:38.406+0000 7f6132600700 -1 osd.6 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:53.344721+0000 front 2024-03-08T21:16:53.344770+0000 (oldest deadline 2024-03-08T21:17:15.644922+0000) 2024-03-08T21:17:38.711 INFO:journalctl@ceph.osd.6.smithi107.stdout:Mar 08 21:17:38 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-6[59834]: 2024-03-08T21:17:38.406+0000 7f6132600700 -1 osd.6 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:17:02.047607+0000 front 2024-03-08T21:17:02.047791+0000 (oldest deadline 2024-03-08T21:17:26.747841+0000) 2024-03-08T21:17:38.711 INFO:journalctl@ceph.osd.6.smithi107.stdout:Mar 08 21:17:38 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-6[59834]: 2024-03-08T21:17:38.406+0000 7f6132600700 -1 osd.6 115 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-03-08T21:17:10.849847+0000 front 2024-03-08T21:17:10.849957+0000 (oldest deadline 2024-03-08T21:17:36.750099+0000) 2024-03-08T21:17:38.711 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:17:38 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:17:38.307+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:40.479965+0000 front 2024-03-08T21:16:40.480241+0000 (oldest deadline 2024-03-08T21:17:06.379830+0000) 2024-03-08T21:17:38.711 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:17:38 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:17:38.307+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:53.283252+0000 front 2024-03-08T21:16:53.283300+0000 (oldest deadline 2024-03-08T21:17:17.383498+0000) 2024-03-08T21:17:38.711 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:17:38 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:17:38.307+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:17:03.088480+0000 front 2024-03-08T21:17:03.088666+0000 (oldest deadline 2024-03-08T21:17:25.388404+0000) 2024-03-08T21:17:38.711 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:17:38 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:17:38.307+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-03-08T21:17:10.690024+0000 front 2024-03-08T21:17:10.690629+0000 (oldest deadline 2024-03-08T21:17:34.189860+0000) 2024-03-08T21:17:39.009 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 08 21:17:38 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-7[53366]: 2024-03-08T21:17:38.565+0000 7f48e9a00700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:53.119608+0000 front 2024-03-08T21:16:53.119745+0000 (oldest deadline 2024-03-08T21:17:17.819760+0000) 2024-03-08T21:17:39.009 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 08 21:17:38 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-7[53366]: 2024-03-08T21:17:38.565+0000 7f48e9a00700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:17:01.822810+0000 front 2024-03-08T21:17:01.822741+0000 (oldest deadline 2024-03-08T21:17:27.122609+0000) 2024-03-08T21:17:39.010 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 08 21:17:38 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-7[53366]: 2024-03-08T21:17:38.565+0000 7f48e9a00700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-03-08T21:17:08.725647+0000 front 2024-03-08T21:17:08.725828+0000 (oldest deadline 2024-03-08T21:17:33.425499+0000) 2024-03-08T21:17:39.508 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:17:39 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:17:39.153+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:44.858572+0000 front 2024-03-08T21:16:44.858829+0000 (oldest deadline 2024-03-08T21:17:06.558274+0000) 2024-03-08T21:17:39.508 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:17:39 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:17:39.153+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:52.961888+0000 front 2024-03-08T21:16:52.961762+0000 (oldest deadline 2024-03-08T21:17:18.862022+0000) 2024-03-08T21:17:39.508 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:17:39 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:17:39.153+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:17:01.764390+0000 front 2024-03-08T21:17:01.764438+0000 (oldest deadline 2024-03-08T21:17:24.064211+0000) 2024-03-08T21:17:39.508 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:17:39 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:17:39.153+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-03-08T21:17:10.869088+0000 front 2024-03-08T21:17:10.869147+0000 (oldest deadline 2024-03-08T21:17:34.969351+0000) 2024-03-08T21:17:39.710 INFO:journalctl@ceph.osd.6.smithi107.stdout:Mar 08 21:17:39 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-6[59834]: 2024-03-08T21:17:39.383+0000 7f6132600700 -1 osd.6 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:44.641569+0000 front 2024-03-08T21:16:44.641810+0000 (oldest deadline 2024-03-08T21:17:07.541743+0000) 2024-03-08T21:17:39.711 INFO:journalctl@ceph.osd.6.smithi107.stdout:Mar 08 21:17:39 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-6[59834]: 2024-03-08T21:17:39.383+0000 7f6132600700 -1 osd.6 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:53.344721+0000 front 2024-03-08T21:16:53.344770+0000 (oldest deadline 2024-03-08T21:17:15.644922+0000) 2024-03-08T21:17:39.711 INFO:journalctl@ceph.osd.6.smithi107.stdout:Mar 08 21:17:39 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-6[59834]: 2024-03-08T21:17:39.383+0000 7f6132600700 -1 osd.6 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:17:02.047607+0000 front 2024-03-08T21:17:02.047791+0000 (oldest deadline 2024-03-08T21:17:26.747841+0000) 2024-03-08T21:17:39.711 INFO:journalctl@ceph.osd.6.smithi107.stdout:Mar 08 21:17:39 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-6[59834]: 2024-03-08T21:17:39.383+0000 7f6132600700 -1 osd.6 115 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-03-08T21:17:10.849847+0000 front 2024-03-08T21:17:10.849957+0000 (oldest deadline 2024-03-08T21:17:36.750099+0000) 2024-03-08T21:17:39.711 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:17:39 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:17:39.282+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:40.479965+0000 front 2024-03-08T21:16:40.480241+0000 (oldest deadline 2024-03-08T21:17:06.379830+0000) 2024-03-08T21:17:39.711 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:17:39 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:17:39.282+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:53.283252+0000 front 2024-03-08T21:16:53.283300+0000 (oldest deadline 2024-03-08T21:17:17.383498+0000) 2024-03-08T21:17:39.711 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:17:39 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:17:39.282+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:17:03.088480+0000 front 2024-03-08T21:17:03.088666+0000 (oldest deadline 2024-03-08T21:17:25.388404+0000) 2024-03-08T21:17:39.711 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:17:39 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:17:39.282+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-03-08T21:17:10.690024+0000 front 2024-03-08T21:17:10.690629+0000 (oldest deadline 2024-03-08T21:17:34.189860+0000) 2024-03-08T21:17:39.871 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 08 21:17:39 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-8[53709]: 2024-03-08T21:17:39.395+0000 7f3fae200700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:43.996391+0000 front 2024-03-08T21:16:43.996418+0000 (oldest deadline 2024-03-08T21:17:09.296545+0000) 2024-03-08T21:17:39.871 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 08 21:17:39 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-8[53709]: 2024-03-08T21:17:39.395+0000 7f3fae200700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:52.199334+0000 front 2024-03-08T21:16:52.199113+0000 (oldest deadline 2024-03-08T21:17:16.899180+0000) 2024-03-08T21:17:39.871 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 08 21:17:39 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-8[53709]: 2024-03-08T21:17:39.395+0000 7f3fae200700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:17:00.401255+0000 front 2024-03-08T21:17:00.401120+0000 (oldest deadline 2024-03-08T21:17:25.701340+0000) 2024-03-08T21:17:39.871 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:17:39 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:17:39.461+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:42.614355+0000 front 2024-03-08T21:16:42.614271+0000 (oldest deadline 2024-03-08T21:17:07.914284+0000) 2024-03-08T21:17:39.872 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:17:39 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:17:39.461+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:52.016503+0000 front 2024-03-08T21:16:52.017017+0000 (oldest deadline 2024-03-08T21:17:16.116550+0000) 2024-03-08T21:17:39.872 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:17:39 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:17:39.461+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:16:57.818698+0000 front 2024-03-08T21:16:57.819201+0000 (oldest deadline 2024-03-08T21:17:26.619635+0000) 2024-03-08T21:17:40.008 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 08 21:17:39 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-7[53366]: 2024-03-08T21:17:39.536+0000 7f48e9a00700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:53.119608+0000 front 2024-03-08T21:16:53.119745+0000 (oldest deadline 2024-03-08T21:17:17.819760+0000) 2024-03-08T21:17:40.008 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 08 21:17:39 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-7[53366]: 2024-03-08T21:17:39.536+0000 7f48e9a00700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:17:01.822810+0000 front 2024-03-08T21:17:01.822741+0000 (oldest deadline 2024-03-08T21:17:27.122609+0000) 2024-03-08T21:17:40.008 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 08 21:17:39 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-7[53366]: 2024-03-08T21:17:39.536+0000 7f48e9a00700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-03-08T21:17:08.725647+0000 front 2024-03-08T21:17:08.725828+0000 (oldest deadline 2024-03-08T21:17:33.425499+0000) 2024-03-08T21:17:40.491 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:17:40 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:17:40.179+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:44.858572+0000 front 2024-03-08T21:16:44.858829+0000 (oldest deadline 2024-03-08T21:17:06.558274+0000) 2024-03-08T21:17:40.491 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:17:40 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:17:40.179+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:52.961888+0000 front 2024-03-08T21:16:52.961762+0000 (oldest deadline 2024-03-08T21:17:18.862022+0000) 2024-03-08T21:17:40.491 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:17:40 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:17:40.179+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:17:01.764390+0000 front 2024-03-08T21:17:01.764438+0000 (oldest deadline 2024-03-08T21:17:24.064211+0000) 2024-03-08T21:17:40.491 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:17:40 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:17:40.179+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-03-08T21:17:10.869088+0000 front 2024-03-08T21:17:10.869147+0000 (oldest deadline 2024-03-08T21:17:34.969351+0000) 2024-03-08T21:17:40.621 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 08 21:17:40 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-8[53709]: 2024-03-08T21:17:40.347+0000 7f3fae200700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:43.996391+0000 front 2024-03-08T21:16:43.996418+0000 (oldest deadline 2024-03-08T21:17:09.296545+0000) 2024-03-08T21:17:40.621 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 08 21:17:40 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-8[53709]: 2024-03-08T21:17:40.347+0000 7f3fae200700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:52.199334+0000 front 2024-03-08T21:16:52.199113+0000 (oldest deadline 2024-03-08T21:17:16.899180+0000) 2024-03-08T21:17:40.621 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 08 21:17:40 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-8[53709]: 2024-03-08T21:17:40.347+0000 7f3fae200700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:17:00.401255+0000 front 2024-03-08T21:17:00.401120+0000 (oldest deadline 2024-03-08T21:17:25.701340+0000) 2024-03-08T21:17:40.621 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:17:40 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:17:40.430+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:42.614355+0000 front 2024-03-08T21:16:42.614271+0000 (oldest deadline 2024-03-08T21:17:07.914284+0000) 2024-03-08T21:17:40.621 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:17:40 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:17:40.430+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:52.016503+0000 front 2024-03-08T21:16:52.017017+0000 (oldest deadline 2024-03-08T21:17:16.116550+0000) 2024-03-08T21:17:40.622 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:17:40 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:17:40.430+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:16:57.818698+0000 front 2024-03-08T21:16:57.819201+0000 (oldest deadline 2024-03-08T21:17:26.619635+0000) 2024-03-08T21:17:40.710 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:17:40 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:17:40.261+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:40.479965+0000 front 2024-03-08T21:16:40.480241+0000 (oldest deadline 2024-03-08T21:17:06.379830+0000) 2024-03-08T21:17:40.710 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:17:40 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:17:40.261+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:53.283252+0000 front 2024-03-08T21:16:53.283300+0000 (oldest deadline 2024-03-08T21:17:17.383498+0000) 2024-03-08T21:17:40.710 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:17:40 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:17:40.261+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:17:03.088480+0000 front 2024-03-08T21:17:03.088666+0000 (oldest deadline 2024-03-08T21:17:25.388404+0000) 2024-03-08T21:17:40.710 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:17:40 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:17:40.261+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-03-08T21:17:10.690024+0000 front 2024-03-08T21:17:10.690629+0000 (oldest deadline 2024-03-08T21:17:34.189860+0000) 2024-03-08T21:17:40.757 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 08 21:17:40 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-7[53366]: 2024-03-08T21:17:40.514+0000 7f48e9a00700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:53.119608+0000 front 2024-03-08T21:16:53.119745+0000 (oldest deadline 2024-03-08T21:17:17.819760+0000) 2024-03-08T21:17:40.758 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 08 21:17:40 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-7[53366]: 2024-03-08T21:17:40.514+0000 7f48e9a00700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:17:01.822810+0000 front 2024-03-08T21:17:01.822741+0000 (oldest deadline 2024-03-08T21:17:27.122609+0000) 2024-03-08T21:17:40.758 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 08 21:17:40 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-7[53366]: 2024-03-08T21:17:40.514+0000 7f48e9a00700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-03-08T21:17:08.725647+0000 front 2024-03-08T21:17:08.725828+0000 (oldest deadline 2024-03-08T21:17:33.425499+0000) 2024-03-08T21:17:41.035 INFO:journalctl@ceph.osd.6.smithi107.stdout:Mar 08 21:17:40 smithi107 podman[95537]: 2024-03-08 21:17:40.7568348 +0000 UTC m=+5.590890732 container died b9c3ca32bbc42ff51240e44bb86ec54ec0778407834712e2eb7ecfb9d29d1cc3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-6, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, RELEASE=rishabh-2024mar8-7d63482, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=) 2024-03-08T21:17:41.460 INFO:journalctl@ceph.osd.6.smithi107.stdout:Mar 08 21:17:41 smithi107 podman[95537]: 2024-03-08 21:17:41.195725085 +0000 UTC m=+6.029781062 container cleanup b9c3ca32bbc42ff51240e44bb86ec54ec0778407834712e2eb7ecfb9d29d1cc3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-6, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=rishabh-2024mar8-7d63482, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=) 2024-03-08T21:17:41.460 INFO:journalctl@ceph.osd.6.smithi107.stdout:Mar 08 21:17:41 smithi107 bash[95537]: ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-6 2024-03-08T21:17:41.461 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:17:41 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:17:41.219+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:40.479965+0000 front 2024-03-08T21:16:40.480241+0000 (oldest deadline 2024-03-08T21:17:06.379830+0000) 2024-03-08T21:17:41.461 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:17:41 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:17:41.219+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:53.283252+0000 front 2024-03-08T21:16:53.283300+0000 (oldest deadline 2024-03-08T21:17:17.383498+0000) 2024-03-08T21:17:41.461 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:17:41 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:17:41.219+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:17:03.088480+0000 front 2024-03-08T21:17:03.088666+0000 (oldest deadline 2024-03-08T21:17:25.388404+0000) 2024-03-08T21:17:41.461 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:17:41 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:17:41.219+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-03-08T21:17:10.690024+0000 front 2024-03-08T21:17:10.690629+0000 (oldest deadline 2024-03-08T21:17:34.189860+0000) 2024-03-08T21:17:41.477 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:17:41 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:17:41.175+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:44.858572+0000 front 2024-03-08T21:16:44.858829+0000 (oldest deadline 2024-03-08T21:17:06.558274+0000) 2024-03-08T21:17:41.477 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:17:41 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:17:41.175+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:52.961888+0000 front 2024-03-08T21:16:52.961762+0000 (oldest deadline 2024-03-08T21:17:18.862022+0000) 2024-03-08T21:17:41.477 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:17:41 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:17:41.175+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:17:01.764390+0000 front 2024-03-08T21:17:01.764438+0000 (oldest deadline 2024-03-08T21:17:24.064211+0000) 2024-03-08T21:17:41.477 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:17:41 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:17:41.175+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-03-08T21:17:10.869088+0000 front 2024-03-08T21:17:10.869147+0000 (oldest deadline 2024-03-08T21:17:34.969351+0000) 2024-03-08T21:17:41.621 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 08 21:17:41 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-8[53709]: 2024-03-08T21:17:41.375+0000 7f3fae200700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:43.996391+0000 front 2024-03-08T21:16:43.996418+0000 (oldest deadline 2024-03-08T21:17:09.296545+0000) 2024-03-08T21:17:41.621 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 08 21:17:41 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-8[53709]: 2024-03-08T21:17:41.375+0000 7f3fae200700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:52.199334+0000 front 2024-03-08T21:16:52.199113+0000 (oldest deadline 2024-03-08T21:17:16.899180+0000) 2024-03-08T21:17:41.621 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 08 21:17:41 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-8[53709]: 2024-03-08T21:17:41.375+0000 7f3fae200700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:17:00.401255+0000 front 2024-03-08T21:17:00.401120+0000 (oldest deadline 2024-03-08T21:17:25.701340+0000) 2024-03-08T21:17:41.621 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:17:41 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:17:41.417+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:42.614355+0000 front 2024-03-08T21:16:42.614271+0000 (oldest deadline 2024-03-08T21:17:07.914284+0000) 2024-03-08T21:17:41.621 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:17:41 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:17:41.417+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:52.016503+0000 front 2024-03-08T21:16:52.017017+0000 (oldest deadline 2024-03-08T21:17:16.116550+0000) 2024-03-08T21:17:41.621 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:17:41 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:17:41.417+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:16:57.818698+0000 front 2024-03-08T21:16:57.819201+0000 (oldest deadline 2024-03-08T21:17:26.619635+0000) 2024-03-08T21:17:41.758 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 08 21:17:41 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-7[53366]: 2024-03-08T21:17:41.500+0000 7f48e9a00700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:53.119608+0000 front 2024-03-08T21:16:53.119745+0000 (oldest deadline 2024-03-08T21:17:17.819760+0000) 2024-03-08T21:17:41.758 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 08 21:17:41 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-7[53366]: 2024-03-08T21:17:41.500+0000 7f48e9a00700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:17:01.822810+0000 front 2024-03-08T21:17:01.822741+0000 (oldest deadline 2024-03-08T21:17:27.122609+0000) 2024-03-08T21:17:41.758 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 08 21:17:41 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-7[53366]: 2024-03-08T21:17:41.500+0000 7f48e9a00700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-03-08T21:17:08.725647+0000 front 2024-03-08T21:17:08.725828+0000 (oldest deadline 2024-03-08T21:17:33.425499+0000) 2024-03-08T21:17:41.960 INFO:journalctl@ceph.osd.6.smithi107.stdout:Mar 08 21:17:41 smithi107 podman[95555]: 2024-03-08 21:17:41.505198488 +0000 UTC m=+0.737647262 container remove b9c3ca32bbc42ff51240e44bb86ec54ec0778407834712e2eb7ecfb9d29d1cc3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-6, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=rishabh-2024mar8-7d63482, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-03-08T21:17:42.470 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:17:42 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:17:42.156+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:44.858572+0000 front 2024-03-08T21:16:44.858829+0000 (oldest deadline 2024-03-08T21:17:06.558274+0000) 2024-03-08T21:17:42.471 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:17:42 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:17:42.156+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:52.961888+0000 front 2024-03-08T21:16:52.961762+0000 (oldest deadline 2024-03-08T21:17:18.862022+0000) 2024-03-08T21:17:42.471 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:17:42 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:17:42.156+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:17:01.764390+0000 front 2024-03-08T21:17:01.764438+0000 (oldest deadline 2024-03-08T21:17:24.064211+0000) 2024-03-08T21:17:42.471 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:17:42 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:17:42.156+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-03-08T21:17:10.869088+0000 front 2024-03-08T21:17:10.869147+0000 (oldest deadline 2024-03-08T21:17:34.969351+0000) 2024-03-08T21:17:42.504 INFO:journalctl@ceph.osd.6.smithi107.stdout:Mar 08 21:17:42 smithi107 podman[95642]: 2024-03-08 21:17:42.241086992 +0000 UTC m=+0.031086381 image pull a272146cb810b1fd4a53627f019472132e30966d5c25b01bfc44a001295f1249 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd 2024-03-08T21:17:42.504 INFO:journalctl@ceph.osd.6.smithi107.stdout:Mar 08 21:17:42 smithi107 podman[95642]: 2024-03-08 21:17:42.364504378 +0000 UTC m=+0.154503803 container create e9ec1150592fe7d43e61b6c6f7d2ff4fc66ce7e659996b563b0938a04cbea1f3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-6-deactivate, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, RELEASE=rishabh-2024mar8-7d63482, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-03-08T21:17:42.504 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:17:42 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:17:42.260+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:40.479965+0000 front 2024-03-08T21:16:40.480241+0000 (oldest deadline 2024-03-08T21:17:06.379830+0000) 2024-03-08T21:17:42.504 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:17:42 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:17:42.260+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:53.283252+0000 front 2024-03-08T21:16:53.283300+0000 (oldest deadline 2024-03-08T21:17:17.383498+0000) 2024-03-08T21:17:42.504 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:17:42 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:17:42.260+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:17:03.088480+0000 front 2024-03-08T21:17:03.088666+0000 (oldest deadline 2024-03-08T21:17:25.388404+0000) 2024-03-08T21:17:42.504 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:17:42 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:17:42.260+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-03-08T21:17:10.690024+0000 front 2024-03-08T21:17:10.690629+0000 (oldest deadline 2024-03-08T21:17:34.189860+0000) 2024-03-08T21:17:42.621 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 08 21:17:42 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-8[53709]: 2024-03-08T21:17:42.423+0000 7f3fae200700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:43.996391+0000 front 2024-03-08T21:16:43.996418+0000 (oldest deadline 2024-03-08T21:17:09.296545+0000) 2024-03-08T21:17:42.621 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 08 21:17:42 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-8[53709]: 2024-03-08T21:17:42.423+0000 7f3fae200700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:52.199334+0000 front 2024-03-08T21:16:52.199113+0000 (oldest deadline 2024-03-08T21:17:16.899180+0000) 2024-03-08T21:17:42.621 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 08 21:17:42 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-8[53709]: 2024-03-08T21:17:42.423+0000 7f3fae200700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:17:00.401255+0000 front 2024-03-08T21:17:00.401120+0000 (oldest deadline 2024-03-08T21:17:25.701340+0000) 2024-03-08T21:17:42.621 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:17:42 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:17:42.389+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:42.614355+0000 front 2024-03-08T21:16:42.614271+0000 (oldest deadline 2024-03-08T21:17:07.914284+0000) 2024-03-08T21:17:42.622 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:17:42 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:17:42.389+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:52.016503+0000 front 2024-03-08T21:16:52.017017+0000 (oldest deadline 2024-03-08T21:17:16.116550+0000) 2024-03-08T21:17:42.622 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:17:42 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:17:42.389+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:16:57.818698+0000 front 2024-03-08T21:16:57.819201+0000 (oldest deadline 2024-03-08T21:17:26.619635+0000) 2024-03-08T21:17:42.758 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 08 21:17:42 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-7[53366]: 2024-03-08T21:17:42.494+0000 7f48e9a00700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:53.119608+0000 front 2024-03-08T21:16:53.119745+0000 (oldest deadline 2024-03-08T21:17:17.819760+0000) 2024-03-08T21:17:42.758 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 08 21:17:42 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-7[53366]: 2024-03-08T21:17:42.494+0000 7f48e9a00700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:17:01.822810+0000 front 2024-03-08T21:17:01.822741+0000 (oldest deadline 2024-03-08T21:17:27.122609+0000) 2024-03-08T21:17:42.758 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 08 21:17:42 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-7[53366]: 2024-03-08T21:17:42.494+0000 7f48e9a00700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-03-08T21:17:08.725647+0000 front 2024-03-08T21:17:08.725828+0000 (oldest deadline 2024-03-08T21:17:33.425499+0000) 2024-03-08T21:17:42.960 INFO:journalctl@ceph.osd.6.smithi107.stdout:Mar 08 21:17:42 smithi107 podman[95642]: 2024-03-08 21:17:42.643296067 +0000 UTC m=+0.433295499 container init e9ec1150592fe7d43e61b6c6f7d2ff4fc66ce7e659996b563b0938a04cbea1f3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-6-deactivate, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=rishabh-2024mar8-7d63482, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-08T21:17:42.960 INFO:journalctl@ceph.osd.6.smithi107.stdout:Mar 08 21:17:42 smithi107 podman[95642]: 2024-03-08 21:17:42.654075522 +0000 UTC m=+0.444074967 container start e9ec1150592fe7d43e61b6c6f7d2ff4fc66ce7e659996b563b0938a04cbea1f3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-6-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, RELEASE=rishabh-2024mar8-7d63482) 2024-03-08T21:17:42.960 INFO:journalctl@ceph.osd.6.smithi107.stdout:Mar 08 21:17:42 smithi107 podman[95642]: 2024-03-08 21:17:42.775559336 +0000 UTC m=+0.565558752 container attach e9ec1150592fe7d43e61b6c6f7d2ff4fc66ce7e659996b563b0938a04cbea1f3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-6-deactivate, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240307, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=rishabh-2024mar8-7d63482, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-03-08T21:17:43.438 INFO:journalctl@ceph.osd.6.smithi107.stdout:Mar 08 21:17:43 smithi107 podman[95642]: 2024-03-08 21:17:43.090989902 +0000 UTC m=+0.880989334 container died e9ec1150592fe7d43e61b6c6f7d2ff4fc66ce7e659996b563b0938a04cbea1f3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-6-deactivate, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, io.buildah.version=1.33.5, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=rishabh-2024mar8-7d63482) 2024-03-08T21:17:43.438 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:17:43 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:17:43.259+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:40.479965+0000 front 2024-03-08T21:16:40.480241+0000 (oldest deadline 2024-03-08T21:17:06.379830+0000) 2024-03-08T21:17:43.439 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:17:43 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:17:43.259+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:53.283252+0000 front 2024-03-08T21:16:53.283300+0000 (oldest deadline 2024-03-08T21:17:17.383498+0000) 2024-03-08T21:17:43.439 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:17:43 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:17:43.259+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:17:03.088480+0000 front 2024-03-08T21:17:03.088666+0000 (oldest deadline 2024-03-08T21:17:25.388404+0000) 2024-03-08T21:17:43.439 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:17:43 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:17:43.259+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-03-08T21:17:10.690024+0000 front 2024-03-08T21:17:10.690629+0000 (oldest deadline 2024-03-08T21:17:34.189860+0000) 2024-03-08T21:17:43.469 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:17:43 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:17:43.197+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:44.858572+0000 front 2024-03-08T21:16:44.858829+0000 (oldest deadline 2024-03-08T21:17:06.558274+0000) 2024-03-08T21:17:43.469 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:17:43 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:17:43.198+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:52.961888+0000 front 2024-03-08T21:16:52.961762+0000 (oldest deadline 2024-03-08T21:17:18.862022+0000) 2024-03-08T21:17:43.469 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:17:43 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:17:43.198+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:17:01.764390+0000 front 2024-03-08T21:17:01.764438+0000 (oldest deadline 2024-03-08T21:17:24.064211+0000) 2024-03-08T21:17:43.469 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:17:43 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:17:43.198+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-03-08T21:17:10.869088+0000 front 2024-03-08T21:17:10.869147+0000 (oldest deadline 2024-03-08T21:17:34.969351+0000) 2024-03-08T21:17:43.621 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 08 21:17:43 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-8[53709]: 2024-03-08T21:17:43.446+0000 7f3fae200700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:43.996391+0000 front 2024-03-08T21:16:43.996418+0000 (oldest deadline 2024-03-08T21:17:09.296545+0000) 2024-03-08T21:17:43.621 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 08 21:17:43 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-8[53709]: 2024-03-08T21:17:43.446+0000 7f3fae200700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:52.199334+0000 front 2024-03-08T21:16:52.199113+0000 (oldest deadline 2024-03-08T21:17:16.899180+0000) 2024-03-08T21:17:43.621 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 08 21:17:43 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-8[53709]: 2024-03-08T21:17:43.446+0000 7f3fae200700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:17:00.401255+0000 front 2024-03-08T21:17:00.401120+0000 (oldest deadline 2024-03-08T21:17:25.701340+0000) 2024-03-08T21:17:43.621 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:17:43 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:17:43.348+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:42.614355+0000 front 2024-03-08T21:16:42.614271+0000 (oldest deadline 2024-03-08T21:17:07.914284+0000) 2024-03-08T21:17:43.621 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:17:43 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:17:43.348+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:52.016503+0000 front 2024-03-08T21:16:52.017017+0000 (oldest deadline 2024-03-08T21:17:16.116550+0000) 2024-03-08T21:17:43.621 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:17:43 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:17:43.348+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:16:57.818698+0000 front 2024-03-08T21:16:57.819201+0000 (oldest deadline 2024-03-08T21:17:26.619635+0000) 2024-03-08T21:17:43.758 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 08 21:17:43 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-7[53366]: 2024-03-08T21:17:43.493+0000 7f48e9a00700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:53.119608+0000 front 2024-03-08T21:16:53.119745+0000 (oldest deadline 2024-03-08T21:17:17.819760+0000) 2024-03-08T21:17:43.758 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 08 21:17:43 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-7[53366]: 2024-03-08T21:17:43.493+0000 7f48e9a00700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:17:01.822810+0000 front 2024-03-08T21:17:01.822741+0000 (oldest deadline 2024-03-08T21:17:27.122609+0000) 2024-03-08T21:17:43.758 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 08 21:17:43 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-7[53366]: 2024-03-08T21:17:43.493+0000 7f48e9a00700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-03-08T21:17:08.725647+0000 front 2024-03-08T21:17:08.725828+0000 (oldest deadline 2024-03-08T21:17:33.425499+0000) 2024-03-08T21:17:43.859 DEBUG:teuthology.orchestra.run.smithi107:> sudo pkill -f 'journalctl -f -n 0 -u ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715@osd.6.service' 2024-03-08T21:17:43.977 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-08T21:17:43.977 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-03-08T21:17:43.977 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-03-08T21:17:43.978 DEBUG:teuthology.orchestra.run.smithi149:> sudo systemctl stop ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715@osd.7 2024-03-08T21:17:44.460 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:17:44 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:17:44.236+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:40.479965+0000 front 2024-03-08T21:16:40.480241+0000 (oldest deadline 2024-03-08T21:17:06.379830+0000) 2024-03-08T21:17:44.460 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:17:44 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:17:44.236+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:53.283252+0000 front 2024-03-08T21:16:53.283300+0000 (oldest deadline 2024-03-08T21:17:17.383498+0000) 2024-03-08T21:17:44.461 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:17:44 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:17:44.236+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:17:03.088480+0000 front 2024-03-08T21:17:03.088666+0000 (oldest deadline 2024-03-08T21:17:25.388404+0000) 2024-03-08T21:17:44.461 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:17:44 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:17:44.236+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-03-08T21:17:10.690024+0000 front 2024-03-08T21:17:10.690629+0000 (oldest deadline 2024-03-08T21:17:34.189860+0000) 2024-03-08T21:17:44.496 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 08 21:17:44 smithi149 systemd[1]: Stopping Ceph osd.7 for dc20f0f8-dd8c-11ee-95c6-87774f69a715... 2024-03-08T21:17:44.497 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:17:44 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:17:44.225+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:44.858572+0000 front 2024-03-08T21:16:44.858829+0000 (oldest deadline 2024-03-08T21:17:06.558274+0000) 2024-03-08T21:17:44.497 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:17:44 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:17:44.225+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:52.961888+0000 front 2024-03-08T21:16:52.961762+0000 (oldest deadline 2024-03-08T21:17:18.862022+0000) 2024-03-08T21:17:44.497 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:17:44 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:17:44.225+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:17:01.764390+0000 front 2024-03-08T21:17:01.764438+0000 (oldest deadline 2024-03-08T21:17:24.064211+0000) 2024-03-08T21:17:44.497 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:17:44 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:17:44.225+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-03-08T21:17:10.869088+0000 front 2024-03-08T21:17:10.869147+0000 (oldest deadline 2024-03-08T21:17:34.969351+0000) 2024-03-08T21:17:44.497 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:17:44 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:17:44.225+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-08T21:17:17.871105+0000 front 2024-03-08T21:17:17.871226+0000 (oldest deadline 2024-03-08T21:17:43.770901+0000) 2024-03-08T21:17:44.621 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 08 21:17:44 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-8[53709]: 2024-03-08T21:17:44.464+0000 7f3fae200700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:43.996391+0000 front 2024-03-08T21:16:43.996418+0000 (oldest deadline 2024-03-08T21:17:09.296545+0000) 2024-03-08T21:17:44.621 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 08 21:17:44 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-8[53709]: 2024-03-08T21:17:44.464+0000 7f3fae200700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:52.199334+0000 front 2024-03-08T21:16:52.199113+0000 (oldest deadline 2024-03-08T21:17:16.899180+0000) 2024-03-08T21:17:44.621 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 08 21:17:44 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-8[53709]: 2024-03-08T21:17:44.464+0000 7f3fae200700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:17:00.401255+0000 front 2024-03-08T21:17:00.401120+0000 (oldest deadline 2024-03-08T21:17:25.701340+0000) 2024-03-08T21:17:44.621 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 08 21:17:44 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-8[53709]: 2024-03-08T21:17:44.464+0000 7f3fae200700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-08T21:17:21.406602+0000 front 2024-03-08T21:17:21.406238+0000 (oldest deadline 2024-03-08T21:17:44.305767+0000) 2024-03-08T21:17:44.621 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:17:44 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:17:44.334+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:42.614355+0000 front 2024-03-08T21:16:42.614271+0000 (oldest deadline 2024-03-08T21:17:07.914284+0000) 2024-03-08T21:17:44.622 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:17:44 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:17:44.334+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:52.016503+0000 front 2024-03-08T21:16:52.017017+0000 (oldest deadline 2024-03-08T21:17:16.116550+0000) 2024-03-08T21:17:44.622 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:17:44 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:17:44.334+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:16:57.818698+0000 front 2024-03-08T21:16:57.819201+0000 (oldest deadline 2024-03-08T21:17:26.619635+0000) 2024-03-08T21:17:44.622 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:17:44 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:17:44.334+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-08T21:17:19.424875+0000 front 2024-03-08T21:17:19.424454+0000 (oldest deadline 2024-03-08T21:17:43.425189+0000) 2024-03-08T21:17:44.758 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 08 21:17:44 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-7[53366]: 2024-03-08T21:17:44.520+0000 7f48e9a00700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:53.119608+0000 front 2024-03-08T21:16:53.119745+0000 (oldest deadline 2024-03-08T21:17:17.819760+0000) 2024-03-08T21:17:44.758 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 08 21:17:44 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-7[53366]: 2024-03-08T21:17:44.520+0000 7f48e9a00700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:17:01.822810+0000 front 2024-03-08T21:17:01.822741+0000 (oldest deadline 2024-03-08T21:17:27.122609+0000) 2024-03-08T21:17:44.758 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 08 21:17:44 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-7[53366]: 2024-03-08T21:17:44.520+0000 7f48e9a00700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-03-08T21:17:08.725647+0000 front 2024-03-08T21:17:08.725828+0000 (oldest deadline 2024-03-08T21:17:33.425499+0000) 2024-03-08T21:17:44.758 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 08 21:17:44 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-7[53366]: 2024-03-08T21:17:44.564+0000 7f48ee200700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-08T21:17:44.758 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 08 21:17:44 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-7[53366]: 2024-03-08T21:17:44.564+0000 7f48ee200700 -1 osd.7 115 *** Got signal Terminated *** 2024-03-08T21:17:44.761 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 08 21:17:44 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-7[53366]: 2024-03-08T21:17:44.564+0000 7f48ee200700 -1 osd.7 115 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-08T21:17:45.354 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:17:45 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:17:45.238+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:40.479965+0000 front 2024-03-08T21:16:40.480241+0000 (oldest deadline 2024-03-08T21:17:06.379830+0000) 2024-03-08T21:17:45.354 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:17:45 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:17:45.238+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:53.283252+0000 front 2024-03-08T21:16:53.283300+0000 (oldest deadline 2024-03-08T21:17:17.383498+0000) 2024-03-08T21:17:45.354 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:17:45 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:17:45.238+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:17:03.088480+0000 front 2024-03-08T21:17:03.088666+0000 (oldest deadline 2024-03-08T21:17:25.388404+0000) 2024-03-08T21:17:45.354 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:17:45 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:17:45.238+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-03-08T21:17:10.690024+0000 front 2024-03-08T21:17:10.690629+0000 (oldest deadline 2024-03-08T21:17:34.189860+0000) 2024-03-08T21:17:45.508 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:17:45 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:17:45.219+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:44.858572+0000 front 2024-03-08T21:16:44.858829+0000 (oldest deadline 2024-03-08T21:17:06.558274+0000) 2024-03-08T21:17:45.508 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:17:45 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:17:45.219+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:52.961888+0000 front 2024-03-08T21:16:52.961762+0000 (oldest deadline 2024-03-08T21:17:18.862022+0000) 2024-03-08T21:17:45.508 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:17:45 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:17:45.219+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:17:01.764390+0000 front 2024-03-08T21:17:01.764438+0000 (oldest deadline 2024-03-08T21:17:24.064211+0000) 2024-03-08T21:17:45.508 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:17:45 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:17:45.219+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-03-08T21:17:10.869088+0000 front 2024-03-08T21:17:10.869147+0000 (oldest deadline 2024-03-08T21:17:34.969351+0000) 2024-03-08T21:17:45.508 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:17:45 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:17:45.219+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-08T21:17:17.871105+0000 front 2024-03-08T21:17:17.871226+0000 (oldest deadline 2024-03-08T21:17:43.770901+0000) 2024-03-08T21:17:45.621 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 08 21:17:45 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-8[53709]: 2024-03-08T21:17:45.479+0000 7f3fae200700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:43.996391+0000 front 2024-03-08T21:16:43.996418+0000 (oldest deadline 2024-03-08T21:17:09.296545+0000) 2024-03-08T21:17:45.621 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 08 21:17:45 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-8[53709]: 2024-03-08T21:17:45.479+0000 7f3fae200700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:52.199334+0000 front 2024-03-08T21:16:52.199113+0000 (oldest deadline 2024-03-08T21:17:16.899180+0000) 2024-03-08T21:17:45.621 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 08 21:17:45 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-8[53709]: 2024-03-08T21:17:45.479+0000 7f3fae200700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:17:00.401255+0000 front 2024-03-08T21:17:00.401120+0000 (oldest deadline 2024-03-08T21:17:25.701340+0000) 2024-03-08T21:17:45.621 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 08 21:17:45 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-8[53709]: 2024-03-08T21:17:45.479+0000 7f3fae200700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-08T21:17:21.406602+0000 front 2024-03-08T21:17:21.406238+0000 (oldest deadline 2024-03-08T21:17:44.305767+0000) 2024-03-08T21:17:45.621 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:17:45 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:17:45.318+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:42.614355+0000 front 2024-03-08T21:16:42.614271+0000 (oldest deadline 2024-03-08T21:17:07.914284+0000) 2024-03-08T21:17:45.621 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:17:45 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:17:45.318+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:52.016503+0000 front 2024-03-08T21:16:52.017017+0000 (oldest deadline 2024-03-08T21:17:16.116550+0000) 2024-03-08T21:17:45.622 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:17:45 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:17:45.318+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:16:57.818698+0000 front 2024-03-08T21:16:57.819201+0000 (oldest deadline 2024-03-08T21:17:26.619635+0000) 2024-03-08T21:17:45.622 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:17:45 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:17:45.318+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-08T21:17:19.424875+0000 front 2024-03-08T21:17:19.424454+0000 (oldest deadline 2024-03-08T21:17:43.425189+0000) 2024-03-08T21:17:46.008 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 08 21:17:45 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-7[53366]: 2024-03-08T21:17:45.563+0000 7f48e9a00700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:53.119608+0000 front 2024-03-08T21:16:53.119745+0000 (oldest deadline 2024-03-08T21:17:17.819760+0000) 2024-03-08T21:17:46.008 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 08 21:17:45 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-7[53366]: 2024-03-08T21:17:45.563+0000 7f48e9a00700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:17:01.822810+0000 front 2024-03-08T21:17:01.822741+0000 (oldest deadline 2024-03-08T21:17:27.122609+0000) 2024-03-08T21:17:46.008 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 08 21:17:45 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-7[53366]: 2024-03-08T21:17:45.563+0000 7f48e9a00700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-03-08T21:17:08.725647+0000 front 2024-03-08T21:17:08.725828+0000 (oldest deadline 2024-03-08T21:17:33.425499+0000) 2024-03-08T21:17:46.508 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:17:46 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:17:46.235+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:44.858572+0000 front 2024-03-08T21:16:44.858829+0000 (oldest deadline 2024-03-08T21:17:06.558274+0000) 2024-03-08T21:17:46.508 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:17:46 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:17:46.235+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:52.961888+0000 front 2024-03-08T21:16:52.961762+0000 (oldest deadline 2024-03-08T21:17:18.862022+0000) 2024-03-08T21:17:46.508 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:17:46 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:17:46.235+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:17:01.764390+0000 front 2024-03-08T21:17:01.764438+0000 (oldest deadline 2024-03-08T21:17:24.064211+0000) 2024-03-08T21:17:46.508 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:17:46 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:17:46.235+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-03-08T21:17:10.869088+0000 front 2024-03-08T21:17:10.869147+0000 (oldest deadline 2024-03-08T21:17:34.969351+0000) 2024-03-08T21:17:46.508 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:17:46 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:17:46.235+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-08T21:17:17.871105+0000 front 2024-03-08T21:17:17.871226+0000 (oldest deadline 2024-03-08T21:17:43.770901+0000) 2024-03-08T21:17:46.554 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:17:46 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:17:46.238+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:40.479965+0000 front 2024-03-08T21:16:40.480241+0000 (oldest deadline 2024-03-08T21:17:06.379830+0000) 2024-03-08T21:17:46.554 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:17:46 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:17:46.238+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:53.283252+0000 front 2024-03-08T21:16:53.283300+0000 (oldest deadline 2024-03-08T21:17:17.383498+0000) 2024-03-08T21:17:46.554 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:17:46 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:17:46.238+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:17:03.088480+0000 front 2024-03-08T21:17:03.088666+0000 (oldest deadline 2024-03-08T21:17:25.388404+0000) 2024-03-08T21:17:46.554 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:17:46 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:17:46.238+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-03-08T21:17:10.690024+0000 front 2024-03-08T21:17:10.690629+0000 (oldest deadline 2024-03-08T21:17:34.189860+0000) 2024-03-08T21:17:46.621 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 08 21:17:46 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-8[53709]: 2024-03-08T21:17:46.528+0000 7f3fae200700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:43.996391+0000 front 2024-03-08T21:16:43.996418+0000 (oldest deadline 2024-03-08T21:17:09.296545+0000) 2024-03-08T21:17:46.621 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 08 21:17:46 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-8[53709]: 2024-03-08T21:17:46.528+0000 7f3fae200700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:52.199334+0000 front 2024-03-08T21:16:52.199113+0000 (oldest deadline 2024-03-08T21:17:16.899180+0000) 2024-03-08T21:17:46.621 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 08 21:17:46 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-8[53709]: 2024-03-08T21:17:46.528+0000 7f3fae200700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:17:00.401255+0000 front 2024-03-08T21:17:00.401120+0000 (oldest deadline 2024-03-08T21:17:25.701340+0000) 2024-03-08T21:17:46.621 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 08 21:17:46 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-8[53709]: 2024-03-08T21:17:46.528+0000 7f3fae200700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-08T21:17:21.406602+0000 front 2024-03-08T21:17:21.406238+0000 (oldest deadline 2024-03-08T21:17:44.305767+0000) 2024-03-08T21:17:46.622 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:17:46 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:17:46.351+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:42.614355+0000 front 2024-03-08T21:16:42.614271+0000 (oldest deadline 2024-03-08T21:17:07.914284+0000) 2024-03-08T21:17:46.622 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:17:46 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:17:46.351+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:52.016503+0000 front 2024-03-08T21:16:52.017017+0000 (oldest deadline 2024-03-08T21:17:16.116550+0000) 2024-03-08T21:17:46.622 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:17:46 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:17:46.351+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:16:57.818698+0000 front 2024-03-08T21:16:57.819201+0000 (oldest deadline 2024-03-08T21:17:26.619635+0000) 2024-03-08T21:17:46.622 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:17:46 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:17:46.351+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-08T21:17:19.424875+0000 front 2024-03-08T21:17:19.424454+0000 (oldest deadline 2024-03-08T21:17:43.425189+0000) 2024-03-08T21:17:47.008 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 08 21:17:46 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-7[53366]: 2024-03-08T21:17:46.604+0000 7f48e9a00700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:53.119608+0000 front 2024-03-08T21:16:53.119745+0000 (oldest deadline 2024-03-08T21:17:17.819760+0000) 2024-03-08T21:17:47.008 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 08 21:17:46 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-7[53366]: 2024-03-08T21:17:46.604+0000 7f48e9a00700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:17:01.822810+0000 front 2024-03-08T21:17:01.822741+0000 (oldest deadline 2024-03-08T21:17:27.122609+0000) 2024-03-08T21:17:47.008 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 08 21:17:46 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-7[53366]: 2024-03-08T21:17:46.604+0000 7f48e9a00700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-03-08T21:17:08.725647+0000 front 2024-03-08T21:17:08.725828+0000 (oldest deadline 2024-03-08T21:17:33.425499+0000) 2024-03-08T21:17:47.008 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 08 21:17:46 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-7[53366]: 2024-03-08T21:17:46.604+0000 7f48e9a00700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-08T21:17:21.029125+0000 front 2024-03-08T21:17:21.028896+0000 (oldest deadline 2024-03-08T21:17:45.729211+0000) 2024-03-08T21:17:47.460 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:17:47 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:17:47.211+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:40.479965+0000 front 2024-03-08T21:16:40.480241+0000 (oldest deadline 2024-03-08T21:17:06.379830+0000) 2024-03-08T21:17:47.460 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:17:47 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:17:47.211+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:53.283252+0000 front 2024-03-08T21:16:53.283300+0000 (oldest deadline 2024-03-08T21:17:17.383498+0000) 2024-03-08T21:17:47.460 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:17:47 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:17:47.211+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:17:03.088480+0000 front 2024-03-08T21:17:03.088666+0000 (oldest deadline 2024-03-08T21:17:25.388404+0000) 2024-03-08T21:17:47.460 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:17:47 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:17:47.211+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-03-08T21:17:10.690024+0000 front 2024-03-08T21:17:10.690629+0000 (oldest deadline 2024-03-08T21:17:34.189860+0000) 2024-03-08T21:17:47.460 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:17:47 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:17:47.211+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-08T21:17:18.192173+0000 front 2024-03-08T21:17:18.192324+0000 (oldest deadline 2024-03-08T21:17:46.992906+0000) 2024-03-08T21:17:47.508 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:17:47 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:17:47.190+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:44.858572+0000 front 2024-03-08T21:16:44.858829+0000 (oldest deadline 2024-03-08T21:17:06.558274+0000) 2024-03-08T21:17:47.508 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:17:47 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:17:47.190+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:52.961888+0000 front 2024-03-08T21:16:52.961762+0000 (oldest deadline 2024-03-08T21:17:18.862022+0000) 2024-03-08T21:17:47.508 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:17:47 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:17:47.190+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:17:01.764390+0000 front 2024-03-08T21:17:01.764438+0000 (oldest deadline 2024-03-08T21:17:24.064211+0000) 2024-03-08T21:17:47.508 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:17:47 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:17:47.190+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-03-08T21:17:10.869088+0000 front 2024-03-08T21:17:10.869147+0000 (oldest deadline 2024-03-08T21:17:34.969351+0000) 2024-03-08T21:17:47.508 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:17:47 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:17:47.190+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-08T21:17:17.871105+0000 front 2024-03-08T21:17:17.871226+0000 (oldest deadline 2024-03-08T21:17:43.770901+0000) 2024-03-08T21:17:47.621 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:17:47 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:17:47.325+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:42.614355+0000 front 2024-03-08T21:16:42.614271+0000 (oldest deadline 2024-03-08T21:17:07.914284+0000) 2024-03-08T21:17:47.621 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:17:47 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:17:47.325+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:52.016503+0000 front 2024-03-08T21:16:52.017017+0000 (oldest deadline 2024-03-08T21:17:16.116550+0000) 2024-03-08T21:17:47.622 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:17:47 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:17:47.325+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:16:57.818698+0000 front 2024-03-08T21:16:57.819201+0000 (oldest deadline 2024-03-08T21:17:26.619635+0000) 2024-03-08T21:17:47.622 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:17:47 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:17:47.325+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-08T21:17:19.424875+0000 front 2024-03-08T21:17:19.424454+0000 (oldest deadline 2024-03-08T21:17:43.425189+0000) 2024-03-08T21:17:47.622 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 08 21:17:47 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-8[53709]: 2024-03-08T21:17:47.496+0000 7f3fae200700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:43.996391+0000 front 2024-03-08T21:16:43.996418+0000 (oldest deadline 2024-03-08T21:17:09.296545+0000) 2024-03-08T21:17:47.622 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 08 21:17:47 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-8[53709]: 2024-03-08T21:17:47.496+0000 7f3fae200700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:52.199334+0000 front 2024-03-08T21:16:52.199113+0000 (oldest deadline 2024-03-08T21:17:16.899180+0000) 2024-03-08T21:17:47.623 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 08 21:17:47 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-8[53709]: 2024-03-08T21:17:47.496+0000 7f3fae200700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:17:00.401255+0000 front 2024-03-08T21:17:00.401120+0000 (oldest deadline 2024-03-08T21:17:25.701340+0000) 2024-03-08T21:17:47.623 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 08 21:17:47 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-8[53709]: 2024-03-08T21:17:47.496+0000 7f3fae200700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-08T21:17:21.406602+0000 front 2024-03-08T21:17:21.406238+0000 (oldest deadline 2024-03-08T21:17:44.305767+0000) 2024-03-08T21:17:48.008 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 08 21:17:47 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-7[53366]: 2024-03-08T21:17:47.558+0000 7f48e9a00700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:53.119608+0000 front 2024-03-08T21:16:53.119745+0000 (oldest deadline 2024-03-08T21:17:17.819760+0000) 2024-03-08T21:17:48.008 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 08 21:17:47 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-7[53366]: 2024-03-08T21:17:47.558+0000 7f48e9a00700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:17:01.822810+0000 front 2024-03-08T21:17:01.822741+0000 (oldest deadline 2024-03-08T21:17:27.122609+0000) 2024-03-08T21:17:48.008 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 08 21:17:47 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-7[53366]: 2024-03-08T21:17:47.558+0000 7f48e9a00700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-03-08T21:17:08.725647+0000 front 2024-03-08T21:17:08.725828+0000 (oldest deadline 2024-03-08T21:17:33.425499+0000) 2024-03-08T21:17:48.008 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 08 21:17:47 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-7[53366]: 2024-03-08T21:17:47.558+0000 7f48e9a00700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-08T21:17:21.029125+0000 front 2024-03-08T21:17:21.028896+0000 (oldest deadline 2024-03-08T21:17:45.729211+0000) 2024-03-08T21:17:48.460 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:17:48 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:17:48.192+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:40.479965+0000 front 2024-03-08T21:16:40.480241+0000 (oldest deadline 2024-03-08T21:17:06.379830+0000) 2024-03-08T21:17:48.461 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:17:48 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:17:48.192+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:53.283252+0000 front 2024-03-08T21:16:53.283300+0000 (oldest deadline 2024-03-08T21:17:17.383498+0000) 2024-03-08T21:17:48.461 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:17:48 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:17:48.192+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:17:03.088480+0000 front 2024-03-08T21:17:03.088666+0000 (oldest deadline 2024-03-08T21:17:25.388404+0000) 2024-03-08T21:17:48.461 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:17:48 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:17:48.192+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-03-08T21:17:10.690024+0000 front 2024-03-08T21:17:10.690629+0000 (oldest deadline 2024-03-08T21:17:34.189860+0000) 2024-03-08T21:17:48.461 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:17:48 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:17:48.192+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-08T21:17:18.192173+0000 front 2024-03-08T21:17:18.192324+0000 (oldest deadline 2024-03-08T21:17:46.992906+0000) 2024-03-08T21:17:48.508 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:17:48 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:17:48.217+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:44.858572+0000 front 2024-03-08T21:16:44.858829+0000 (oldest deadline 2024-03-08T21:17:06.558274+0000) 2024-03-08T21:17:48.508 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:17:48 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:17:48.217+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:52.961888+0000 front 2024-03-08T21:16:52.961762+0000 (oldest deadline 2024-03-08T21:17:18.862022+0000) 2024-03-08T21:17:48.508 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:17:48 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:17:48.217+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:17:01.764390+0000 front 2024-03-08T21:17:01.764438+0000 (oldest deadline 2024-03-08T21:17:24.064211+0000) 2024-03-08T21:17:48.508 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:17:48 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:17:48.217+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-03-08T21:17:10.869088+0000 front 2024-03-08T21:17:10.869147+0000 (oldest deadline 2024-03-08T21:17:34.969351+0000) 2024-03-08T21:17:48.508 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:17:48 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:17:48.217+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-08T21:17:17.871105+0000 front 2024-03-08T21:17:17.871226+0000 (oldest deadline 2024-03-08T21:17:43.770901+0000) 2024-03-08T21:17:48.621 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 08 21:17:48 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-8[53709]: 2024-03-08T21:17:48.493+0000 7f3fae200700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:43.996391+0000 front 2024-03-08T21:16:43.996418+0000 (oldest deadline 2024-03-08T21:17:09.296545+0000) 2024-03-08T21:17:48.621 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 08 21:17:48 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-8[53709]: 2024-03-08T21:17:48.493+0000 7f3fae200700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:52.199334+0000 front 2024-03-08T21:16:52.199113+0000 (oldest deadline 2024-03-08T21:17:16.899180+0000) 2024-03-08T21:17:48.621 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 08 21:17:48 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-8[53709]: 2024-03-08T21:17:48.493+0000 7f3fae200700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:17:00.401255+0000 front 2024-03-08T21:17:00.401120+0000 (oldest deadline 2024-03-08T21:17:25.701340+0000) 2024-03-08T21:17:48.622 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 08 21:17:48 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-8[53709]: 2024-03-08T21:17:48.493+0000 7f3fae200700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-08T21:17:21.406602+0000 front 2024-03-08T21:17:21.406238+0000 (oldest deadline 2024-03-08T21:17:44.305767+0000) 2024-03-08T21:17:48.622 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:17:48 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:17:48.364+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:42.614355+0000 front 2024-03-08T21:16:42.614271+0000 (oldest deadline 2024-03-08T21:17:07.914284+0000) 2024-03-08T21:17:48.622 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:17:48 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:17:48.364+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:52.016503+0000 front 2024-03-08T21:16:52.017017+0000 (oldest deadline 2024-03-08T21:17:16.116550+0000) 2024-03-08T21:17:48.622 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:17:48 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:17:48.364+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:16:57.818698+0000 front 2024-03-08T21:16:57.819201+0000 (oldest deadline 2024-03-08T21:17:26.619635+0000) 2024-03-08T21:17:48.622 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:17:48 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:17:48.364+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-08T21:17:19.424875+0000 front 2024-03-08T21:17:19.424454+0000 (oldest deadline 2024-03-08T21:17:43.425189+0000) 2024-03-08T21:17:49.008 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 08 21:17:48 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-7[53366]: 2024-03-08T21:17:48.558+0000 7f48e9a00700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:53.119608+0000 front 2024-03-08T21:16:53.119745+0000 (oldest deadline 2024-03-08T21:17:17.819760+0000) 2024-03-08T21:17:49.008 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 08 21:17:48 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-7[53366]: 2024-03-08T21:17:48.558+0000 7f48e9a00700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:17:01.822810+0000 front 2024-03-08T21:17:01.822741+0000 (oldest deadline 2024-03-08T21:17:27.122609+0000) 2024-03-08T21:17:49.008 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 08 21:17:48 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-7[53366]: 2024-03-08T21:17:48.558+0000 7f48e9a00700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-03-08T21:17:08.725647+0000 front 2024-03-08T21:17:08.725828+0000 (oldest deadline 2024-03-08T21:17:33.425499+0000) 2024-03-08T21:17:49.008 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 08 21:17:48 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-7[53366]: 2024-03-08T21:17:48.558+0000 7f48e9a00700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-08T21:17:21.029125+0000 front 2024-03-08T21:17:21.028896+0000 (oldest deadline 2024-03-08T21:17:45.729211+0000) 2024-03-08T21:17:49.460 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:17:49 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:17:49.172+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:40.479965+0000 front 2024-03-08T21:16:40.480241+0000 (oldest deadline 2024-03-08T21:17:06.379830+0000) 2024-03-08T21:17:49.461 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:17:49 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:17:49.172+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:53.283252+0000 front 2024-03-08T21:16:53.283300+0000 (oldest deadline 2024-03-08T21:17:17.383498+0000) 2024-03-08T21:17:49.461 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:17:49 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:17:49.172+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:17:03.088480+0000 front 2024-03-08T21:17:03.088666+0000 (oldest deadline 2024-03-08T21:17:25.388404+0000) 2024-03-08T21:17:49.461 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:17:49 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:17:49.172+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-03-08T21:17:10.690024+0000 front 2024-03-08T21:17:10.690629+0000 (oldest deadline 2024-03-08T21:17:34.189860+0000) 2024-03-08T21:17:49.461 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:17:49 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:17:49.172+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-08T21:17:18.192173+0000 front 2024-03-08T21:17:18.192324+0000 (oldest deadline 2024-03-08T21:17:46.992906+0000) 2024-03-08T21:17:49.508 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:17:49 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:17:49.217+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:44.858572+0000 front 2024-03-08T21:16:44.858829+0000 (oldest deadline 2024-03-08T21:17:06.558274+0000) 2024-03-08T21:17:49.508 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:17:49 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:17:49.217+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:52.961888+0000 front 2024-03-08T21:16:52.961762+0000 (oldest deadline 2024-03-08T21:17:18.862022+0000) 2024-03-08T21:17:49.508 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:17:49 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:17:49.217+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:17:01.764390+0000 front 2024-03-08T21:17:01.764438+0000 (oldest deadline 2024-03-08T21:17:24.064211+0000) 2024-03-08T21:17:49.508 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:17:49 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:17:49.217+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-03-08T21:17:10.869088+0000 front 2024-03-08T21:17:10.869147+0000 (oldest deadline 2024-03-08T21:17:34.969351+0000) 2024-03-08T21:17:49.508 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:17:49 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:17:49.217+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-08T21:17:17.871105+0000 front 2024-03-08T21:17:17.871226+0000 (oldest deadline 2024-03-08T21:17:43.770901+0000) 2024-03-08T21:17:49.621 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 08 21:17:49 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-8[53709]: 2024-03-08T21:17:49.541+0000 7f3fae200700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:43.996391+0000 front 2024-03-08T21:16:43.996418+0000 (oldest deadline 2024-03-08T21:17:09.296545+0000) 2024-03-08T21:17:49.621 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 08 21:17:49 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-8[53709]: 2024-03-08T21:17:49.541+0000 7f3fae200700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:52.199334+0000 front 2024-03-08T21:16:52.199113+0000 (oldest deadline 2024-03-08T21:17:16.899180+0000) 2024-03-08T21:17:49.621 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 08 21:17:49 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-8[53709]: 2024-03-08T21:17:49.541+0000 7f3fae200700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:17:00.401255+0000 front 2024-03-08T21:17:00.401120+0000 (oldest deadline 2024-03-08T21:17:25.701340+0000) 2024-03-08T21:17:49.621 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 08 21:17:49 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-8[53709]: 2024-03-08T21:17:49.541+0000 7f3fae200700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-08T21:17:21.406602+0000 front 2024-03-08T21:17:21.406238+0000 (oldest deadline 2024-03-08T21:17:44.305767+0000) 2024-03-08T21:17:49.622 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:17:49 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:17:49.355+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:42.614355+0000 front 2024-03-08T21:16:42.614271+0000 (oldest deadline 2024-03-08T21:17:07.914284+0000) 2024-03-08T21:17:49.622 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:17:49 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:17:49.355+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:52.016503+0000 front 2024-03-08T21:16:52.017017+0000 (oldest deadline 2024-03-08T21:17:16.116550+0000) 2024-03-08T21:17:49.622 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:17:49 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:17:49.355+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:16:57.818698+0000 front 2024-03-08T21:16:57.819201+0000 (oldest deadline 2024-03-08T21:17:26.619635+0000) 2024-03-08T21:17:49.622 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:17:49 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:17:49.355+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-08T21:17:19.424875+0000 front 2024-03-08T21:17:19.424454+0000 (oldest deadline 2024-03-08T21:17:43.425189+0000) 2024-03-08T21:17:50.008 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 08 21:17:49 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-7[53366]: 2024-03-08T21:17:49.553+0000 7f48e9a00700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:53.119608+0000 front 2024-03-08T21:16:53.119745+0000 (oldest deadline 2024-03-08T21:17:17.819760+0000) 2024-03-08T21:17:50.008 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 08 21:17:49 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-7[53366]: 2024-03-08T21:17:49.553+0000 7f48e9a00700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:17:01.822810+0000 front 2024-03-08T21:17:01.822741+0000 (oldest deadline 2024-03-08T21:17:27.122609+0000) 2024-03-08T21:17:50.009 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 08 21:17:49 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-7[53366]: 2024-03-08T21:17:49.553+0000 7f48e9a00700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-03-08T21:17:08.725647+0000 front 2024-03-08T21:17:08.725828+0000 (oldest deadline 2024-03-08T21:17:33.425499+0000) 2024-03-08T21:17:50.009 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 08 21:17:49 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-7[53366]: 2024-03-08T21:17:49.553+0000 7f48e9a00700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-08T21:17:21.029125+0000 front 2024-03-08T21:17:21.028896+0000 (oldest deadline 2024-03-08T21:17:45.729211+0000) 2024-03-08T21:17:50.349 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 08 21:17:50 smithi149 podman[72772]: 2024-03-08 21:17:50.074131265 +0000 UTC m=+5.671164731 container died 9d448c10c2d5e7df5eb49107119d95fa4bed7529fff727f3ef1ccf7fd596e792 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-7, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=rishabh-2024mar8-7d63482, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-08T21:17:50.349 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:17:50 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:17:50.178+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:44.858572+0000 front 2024-03-08T21:16:44.858829+0000 (oldest deadline 2024-03-08T21:17:06.558274+0000) 2024-03-08T21:17:50.349 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:17:50 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:17:50.178+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:52.961888+0000 front 2024-03-08T21:16:52.961762+0000 (oldest deadline 2024-03-08T21:17:18.862022+0000) 2024-03-08T21:17:50.349 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:17:50 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:17:50.178+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:17:01.764390+0000 front 2024-03-08T21:17:01.764438+0000 (oldest deadline 2024-03-08T21:17:24.064211+0000) 2024-03-08T21:17:50.350 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:17:50 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:17:50.178+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-03-08T21:17:10.869088+0000 front 2024-03-08T21:17:10.869147+0000 (oldest deadline 2024-03-08T21:17:34.969351+0000) 2024-03-08T21:17:50.350 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:17:50 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:17:50.178+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-08T21:17:17.871105+0000 front 2024-03-08T21:17:17.871226+0000 (oldest deadline 2024-03-08T21:17:43.770901+0000) 2024-03-08T21:17:50.460 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:17:50 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:17:50.199+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:40.479965+0000 front 2024-03-08T21:16:40.480241+0000 (oldest deadline 2024-03-08T21:17:06.379830+0000) 2024-03-08T21:17:50.460 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:17:50 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:17:50.199+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:53.283252+0000 front 2024-03-08T21:16:53.283300+0000 (oldest deadline 2024-03-08T21:17:17.383498+0000) 2024-03-08T21:17:50.460 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:17:50 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:17:50.199+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:17:03.088480+0000 front 2024-03-08T21:17:03.088666+0000 (oldest deadline 2024-03-08T21:17:25.388404+0000) 2024-03-08T21:17:50.460 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:17:50 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:17:50.199+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-03-08T21:17:10.690024+0000 front 2024-03-08T21:17:10.690629+0000 (oldest deadline 2024-03-08T21:17:34.189860+0000) 2024-03-08T21:17:50.461 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:17:50 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:17:50.199+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-08T21:17:18.192173+0000 front 2024-03-08T21:17:18.192324+0000 (oldest deadline 2024-03-08T21:17:46.992906+0000) 2024-03-08T21:17:50.621 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 08 21:17:50 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-8[53709]: 2024-03-08T21:17:50.522+0000 7f3fae200700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:43.996391+0000 front 2024-03-08T21:16:43.996418+0000 (oldest deadline 2024-03-08T21:17:09.296545+0000) 2024-03-08T21:17:50.621 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 08 21:17:50 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-8[53709]: 2024-03-08T21:17:50.522+0000 7f3fae200700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:52.199334+0000 front 2024-03-08T21:16:52.199113+0000 (oldest deadline 2024-03-08T21:17:16.899180+0000) 2024-03-08T21:17:50.621 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 08 21:17:50 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-8[53709]: 2024-03-08T21:17:50.522+0000 7f3fae200700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:17:00.401255+0000 front 2024-03-08T21:17:00.401120+0000 (oldest deadline 2024-03-08T21:17:25.701340+0000) 2024-03-08T21:17:50.621 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 08 21:17:50 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-8[53709]: 2024-03-08T21:17:50.522+0000 7f3fae200700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-08T21:17:21.406602+0000 front 2024-03-08T21:17:21.406238+0000 (oldest deadline 2024-03-08T21:17:44.305767+0000) 2024-03-08T21:17:50.621 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:17:50 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:17:50.384+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:42.614355+0000 front 2024-03-08T21:16:42.614271+0000 (oldest deadline 2024-03-08T21:17:07.914284+0000) 2024-03-08T21:17:50.622 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:17:50 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:17:50.384+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:52.016503+0000 front 2024-03-08T21:16:52.017017+0000 (oldest deadline 2024-03-08T21:17:16.116550+0000) 2024-03-08T21:17:50.622 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:17:50 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:17:50.384+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:16:57.818698+0000 front 2024-03-08T21:16:57.819201+0000 (oldest deadline 2024-03-08T21:17:26.619635+0000) 2024-03-08T21:17:50.622 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:17:50 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:17:50.384+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-08T21:17:19.424875+0000 front 2024-03-08T21:17:19.424454+0000 (oldest deadline 2024-03-08T21:17:43.425189+0000) 2024-03-08T21:17:50.677 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 08 21:17:50 smithi149 podman[72772]: 2024-03-08 21:17:50.502675445 +0000 UTC m=+6.099708914 container cleanup 9d448c10c2d5e7df5eb49107119d95fa4bed7529fff727f3ef1ccf7fd596e792 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-7, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , RELEASE=rishabh-2024mar8-7d63482, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-08T21:17:50.677 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 08 21:17:50 smithi149 bash[72772]: ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-7 2024-03-08T21:17:51.007 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 08 21:17:50 smithi149 podman[72791]: 2024-03-08 21:17:50.787225287 +0000 UTC m=+0.703956175 container remove 9d448c10c2d5e7df5eb49107119d95fa4bed7529fff727f3ef1ccf7fd596e792 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-7, RELEASE=rishabh-2024mar8-7d63482, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-08T21:17:51.460 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:17:51 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:17:51.227+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:40.479965+0000 front 2024-03-08T21:16:40.480241+0000 (oldest deadline 2024-03-08T21:17:06.379830+0000) 2024-03-08T21:17:51.460 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:17:51 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:17:51.227+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:53.283252+0000 front 2024-03-08T21:16:53.283300+0000 (oldest deadline 2024-03-08T21:17:17.383498+0000) 2024-03-08T21:17:51.461 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:17:51 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:17:51.227+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:17:03.088480+0000 front 2024-03-08T21:17:03.088666+0000 (oldest deadline 2024-03-08T21:17:25.388404+0000) 2024-03-08T21:17:51.461 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:17:51 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:17:51.227+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-03-08T21:17:10.690024+0000 front 2024-03-08T21:17:10.690629+0000 (oldest deadline 2024-03-08T21:17:34.189860+0000) 2024-03-08T21:17:51.461 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:17:51 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:17:51.227+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-08T21:17:18.192173+0000 front 2024-03-08T21:17:18.192324+0000 (oldest deadline 2024-03-08T21:17:46.992906+0000) 2024-03-08T21:17:51.508 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:17:51 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:17:51.189+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:44.858572+0000 front 2024-03-08T21:16:44.858829+0000 (oldest deadline 2024-03-08T21:17:06.558274+0000) 2024-03-08T21:17:51.508 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:17:51 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:17:51.189+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:52.961888+0000 front 2024-03-08T21:16:52.961762+0000 (oldest deadline 2024-03-08T21:17:18.862022+0000) 2024-03-08T21:17:51.508 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:17:51 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:17:51.189+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:17:01.764390+0000 front 2024-03-08T21:17:01.764438+0000 (oldest deadline 2024-03-08T21:17:24.064211+0000) 2024-03-08T21:17:51.508 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:17:51 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:17:51.189+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-03-08T21:17:10.869088+0000 front 2024-03-08T21:17:10.869147+0000 (oldest deadline 2024-03-08T21:17:34.969351+0000) 2024-03-08T21:17:51.508 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:17:51 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:17:51.189+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-08T21:17:17.871105+0000 front 2024-03-08T21:17:17.871226+0000 (oldest deadline 2024-03-08T21:17:43.770901+0000) 2024-03-08T21:17:51.780 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 08 21:17:51 smithi149 podman[72887]: 2024-03-08 21:17:51.451644709 +0000 UTC m=+0.037783005 image pull a272146cb810b1fd4a53627f019472132e30966d5c25b01bfc44a001295f1249 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd 2024-03-08T21:17:51.780 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 08 21:17:51 smithi149 podman[72887]: 2024-03-08 21:17:51.638312451 +0000 UTC m=+0.224450752 container create 5521c15660195cb28456945dcf3481c2816029479c3e3359f197037906390e9c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-7-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=rishabh-2024mar8-7d63482, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-03-08T21:17:51.871 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:17:51 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:17:51.413+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:42.614355+0000 front 2024-03-08T21:16:42.614271+0000 (oldest deadline 2024-03-08T21:17:07.914284+0000) 2024-03-08T21:17:51.871 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:17:51 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:17:51.413+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:52.016503+0000 front 2024-03-08T21:16:52.017017+0000 (oldest deadline 2024-03-08T21:17:16.116550+0000) 2024-03-08T21:17:51.871 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:17:51 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:17:51.413+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:16:57.818698+0000 front 2024-03-08T21:16:57.819201+0000 (oldest deadline 2024-03-08T21:17:26.619635+0000) 2024-03-08T21:17:51.871 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:17:51 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:17:51.413+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-08T21:17:19.424875+0000 front 2024-03-08T21:17:19.424454+0000 (oldest deadline 2024-03-08T21:17:43.425189+0000) 2024-03-08T21:17:51.871 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 08 21:17:51 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-8[53709]: 2024-03-08T21:17:51.557+0000 7f3fae200700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:43.996391+0000 front 2024-03-08T21:16:43.996418+0000 (oldest deadline 2024-03-08T21:17:09.296545+0000) 2024-03-08T21:17:51.872 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 08 21:17:51 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-8[53709]: 2024-03-08T21:17:51.557+0000 7f3fae200700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:52.199334+0000 front 2024-03-08T21:16:52.199113+0000 (oldest deadline 2024-03-08T21:17:16.899180+0000) 2024-03-08T21:17:51.872 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 08 21:17:51 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-8[53709]: 2024-03-08T21:17:51.557+0000 7f3fae200700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:17:00.401255+0000 front 2024-03-08T21:17:00.401120+0000 (oldest deadline 2024-03-08T21:17:25.701340+0000) 2024-03-08T21:17:51.872 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 08 21:17:51 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-8[53709]: 2024-03-08T21:17:51.557+0000 7f3fae200700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-08T21:17:21.406602+0000 front 2024-03-08T21:17:21.406238+0000 (oldest deadline 2024-03-08T21:17:44.305767+0000) 2024-03-08T21:17:52.197 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 08 21:17:51 smithi149 podman[72887]: 2024-03-08 21:17:51.925606251 +0000 UTC m=+0.511744600 container init 5521c15660195cb28456945dcf3481c2816029479c3e3359f197037906390e9c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-7-deactivate, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=rishabh-2024mar8-7d63482, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-03-08T21:17:52.197 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 08 21:17:51 smithi149 podman[72887]: 2024-03-08 21:17:51.937820704 +0000 UTC m=+0.523959022 container start 5521c15660195cb28456945dcf3481c2816029479c3e3359f197037906390e9c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-7-deactivate, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=rishabh-2024mar8-7d63482, GIT_CLEAN=True) 2024-03-08T21:17:52.197 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 08 21:17:52 smithi149 podman[72887]: 2024-03-08 21:17:52.032650008 +0000 UTC m=+0.618788370 container attach 5521c15660195cb28456945dcf3481c2816029479c3e3359f197037906390e9c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-7-deactivate, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=rishabh-2024mar8-7d63482, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=) 2024-03-08T21:17:52.460 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:17:52 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:17:52.218+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:40.479965+0000 front 2024-03-08T21:16:40.480241+0000 (oldest deadline 2024-03-08T21:17:06.379830+0000) 2024-03-08T21:17:52.460 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:17:52 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:17:52.218+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:53.283252+0000 front 2024-03-08T21:16:53.283300+0000 (oldest deadline 2024-03-08T21:17:17.383498+0000) 2024-03-08T21:17:52.460 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:17:52 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:17:52.218+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:17:03.088480+0000 front 2024-03-08T21:17:03.088666+0000 (oldest deadline 2024-03-08T21:17:25.388404+0000) 2024-03-08T21:17:52.461 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:17:52 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:17:52.218+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-03-08T21:17:10.690024+0000 front 2024-03-08T21:17:10.690629+0000 (oldest deadline 2024-03-08T21:17:34.189860+0000) 2024-03-08T21:17:52.461 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:17:52 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:17:52.218+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-08T21:17:18.192173+0000 front 2024-03-08T21:17:18.192324+0000 (oldest deadline 2024-03-08T21:17:46.992906+0000) 2024-03-08T21:17:52.507 INFO:journalctl@ceph.osd.7.smithi149.stdout:Mar 08 21:17:52 smithi149 podman[72887]: 2024-03-08 21:17:52.365446201 +0000 UTC m=+0.951584526 container died 5521c15660195cb28456945dcf3481c2816029479c3e3359f197037906390e9c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-7-deactivate, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, RELEASE=rishabh-2024mar8-7d63482) 2024-03-08T21:17:52.508 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:17:52 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:17:52.221+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:44.858572+0000 front 2024-03-08T21:16:44.858829+0000 (oldest deadline 2024-03-08T21:17:06.558274+0000) 2024-03-08T21:17:52.508 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:17:52 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:17:52.221+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:52.961888+0000 front 2024-03-08T21:16:52.961762+0000 (oldest deadline 2024-03-08T21:17:18.862022+0000) 2024-03-08T21:17:52.508 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:17:52 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:17:52.221+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:17:01.764390+0000 front 2024-03-08T21:17:01.764438+0000 (oldest deadline 2024-03-08T21:17:24.064211+0000) 2024-03-08T21:17:52.508 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:17:52 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:17:52.221+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-03-08T21:17:10.869088+0000 front 2024-03-08T21:17:10.869147+0000 (oldest deadline 2024-03-08T21:17:34.969351+0000) 2024-03-08T21:17:52.508 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:17:52 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:17:52.221+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-08T21:17:17.871105+0000 front 2024-03-08T21:17:17.871226+0000 (oldest deadline 2024-03-08T21:17:43.770901+0000) 2024-03-08T21:17:52.508 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:17:52 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:17:52.221+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-03-08T21:17:30.173392+0000 front 2024-03-08T21:17:30.173231+0000 (oldest deadline 2024-03-08T21:17:51.773639+0000) 2024-03-08T21:17:52.872 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 08 21:17:52 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-8[53709]: 2024-03-08T21:17:52.572+0000 7f3fae200700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:43.996391+0000 front 2024-03-08T21:16:43.996418+0000 (oldest deadline 2024-03-08T21:17:09.296545+0000) 2024-03-08T21:17:52.872 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 08 21:17:52 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-8[53709]: 2024-03-08T21:17:52.572+0000 7f3fae200700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:52.199334+0000 front 2024-03-08T21:16:52.199113+0000 (oldest deadline 2024-03-08T21:17:16.899180+0000) 2024-03-08T21:17:52.872 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 08 21:17:52 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-8[53709]: 2024-03-08T21:17:52.572+0000 7f3fae200700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:17:00.401255+0000 front 2024-03-08T21:17:00.401120+0000 (oldest deadline 2024-03-08T21:17:25.701340+0000) 2024-03-08T21:17:52.872 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 08 21:17:52 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-8[53709]: 2024-03-08T21:17:52.572+0000 7f3fae200700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-08T21:17:21.406602+0000 front 2024-03-08T21:17:21.406238+0000 (oldest deadline 2024-03-08T21:17:44.305767+0000) 2024-03-08T21:17:52.872 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 08 21:17:52 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-8[53709]: 2024-03-08T21:17:52.572+0000 7f3fae200700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-03-08T21:17:30.009723+0000 front 2024-03-08T21:17:30.009797+0000 (oldest deadline 2024-03-08T21:17:52.309808+0000) 2024-03-08T21:17:52.872 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:17:52 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:17:52.463+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:42.614355+0000 front 2024-03-08T21:16:42.614271+0000 (oldest deadline 2024-03-08T21:17:07.914284+0000) 2024-03-08T21:17:52.873 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:17:52 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:17:52.463+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:52.016503+0000 front 2024-03-08T21:16:52.017017+0000 (oldest deadline 2024-03-08T21:17:16.116550+0000) 2024-03-08T21:17:52.873 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:17:52 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:17:52.463+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:16:57.818698+0000 front 2024-03-08T21:16:57.819201+0000 (oldest deadline 2024-03-08T21:17:26.619635+0000) 2024-03-08T21:17:52.873 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:17:52 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:17:52.463+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-08T21:17:19.424875+0000 front 2024-03-08T21:17:19.424454+0000 (oldest deadline 2024-03-08T21:17:43.425189+0000) 2024-03-08T21:17:52.873 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:17:52 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:17:52.463+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-03-08T21:17:29.327497+0000 front 2024-03-08T21:17:29.328128+0000 (oldest deadline 2024-03-08T21:17:51.627384+0000) 2024-03-08T21:17:53.139 DEBUG:teuthology.orchestra.run.smithi149:> sudo pkill -f 'journalctl -f -n 0 -u ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715@osd.7.service' 2024-03-08T21:17:53.260 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-08T21:17:53.260 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-03-08T21:17:53.260 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-03-08T21:17:53.260 DEBUG:teuthology.orchestra.run.smithi204:> sudo systemctl stop ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715@osd.8 2024-03-08T21:17:53.460 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:17:53 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:17:53.194+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:40.479965+0000 front 2024-03-08T21:16:40.480241+0000 (oldest deadline 2024-03-08T21:17:06.379830+0000) 2024-03-08T21:17:53.460 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:17:53 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:17:53.194+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:53.283252+0000 front 2024-03-08T21:16:53.283300+0000 (oldest deadline 2024-03-08T21:17:17.383498+0000) 2024-03-08T21:17:53.461 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:17:53 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:17:53.194+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:17:03.088480+0000 front 2024-03-08T21:17:03.088666+0000 (oldest deadline 2024-03-08T21:17:25.388404+0000) 2024-03-08T21:17:53.461 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:17:53 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:17:53.194+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-03-08T21:17:10.690024+0000 front 2024-03-08T21:17:10.690629+0000 (oldest deadline 2024-03-08T21:17:34.189860+0000) 2024-03-08T21:17:53.461 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:17:53 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:17:53.194+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-08T21:17:18.192173+0000 front 2024-03-08T21:17:18.192324+0000 (oldest deadline 2024-03-08T21:17:46.992906+0000) 2024-03-08T21:17:53.461 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:17:53 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:17:53.194+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-03-08T21:17:28.694794+0000 front 2024-03-08T21:17:28.695136+0000 (oldest deadline 2024-03-08T21:17:52.794933+0000) 2024-03-08T21:17:53.508 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:17:53 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:17:53.235+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:44.858572+0000 front 2024-03-08T21:16:44.858829+0000 (oldest deadline 2024-03-08T21:17:06.558274+0000) 2024-03-08T21:17:53.508 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:17:53 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:17:53.235+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:52.961888+0000 front 2024-03-08T21:16:52.961762+0000 (oldest deadline 2024-03-08T21:17:18.862022+0000) 2024-03-08T21:17:53.508 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:17:53 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:17:53.235+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:17:01.764390+0000 front 2024-03-08T21:17:01.764438+0000 (oldest deadline 2024-03-08T21:17:24.064211+0000) 2024-03-08T21:17:53.508 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:17:53 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:17:53.235+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-03-08T21:17:10.869088+0000 front 2024-03-08T21:17:10.869147+0000 (oldest deadline 2024-03-08T21:17:34.969351+0000) 2024-03-08T21:17:53.508 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:17:53 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:17:53.235+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-08T21:17:17.871105+0000 front 2024-03-08T21:17:17.871226+0000 (oldest deadline 2024-03-08T21:17:43.770901+0000) 2024-03-08T21:17:53.508 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:17:53 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:17:53.235+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-03-08T21:17:30.173392+0000 front 2024-03-08T21:17:30.173231+0000 (oldest deadline 2024-03-08T21:17:51.773639+0000) 2024-03-08T21:17:53.621 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:17:53 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:17:53.454+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:42.614355+0000 front 2024-03-08T21:16:42.614271+0000 (oldest deadline 2024-03-08T21:17:07.914284+0000) 2024-03-08T21:17:53.621 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:17:53 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:17:53.454+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:52.016503+0000 front 2024-03-08T21:16:52.017017+0000 (oldest deadline 2024-03-08T21:17:16.116550+0000) 2024-03-08T21:17:53.621 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:17:53 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:17:53.454+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:16:57.818698+0000 front 2024-03-08T21:16:57.819201+0000 (oldest deadline 2024-03-08T21:17:26.619635+0000) 2024-03-08T21:17:53.621 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:17:53 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:17:53.454+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-08T21:17:19.424875+0000 front 2024-03-08T21:17:19.424454+0000 (oldest deadline 2024-03-08T21:17:43.425189+0000) 2024-03-08T21:17:53.621 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:17:53 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:17:53.454+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-03-08T21:17:29.327497+0000 front 2024-03-08T21:17:29.328128+0000 (oldest deadline 2024-03-08T21:17:51.627384+0000) 2024-03-08T21:17:53.622 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 08 21:17:53 smithi204 systemd[1]: Stopping Ceph osd.8 for dc20f0f8-dd8c-11ee-95c6-87774f69a715... 2024-03-08T21:17:53.622 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 08 21:17:53 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-8[53709]: 2024-03-08T21:17:53.618+0000 7f3fae200700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:43.996391+0000 front 2024-03-08T21:16:43.996418+0000 (oldest deadline 2024-03-08T21:17:09.296545+0000) 2024-03-08T21:17:53.622 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 08 21:17:53 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-8[53709]: 2024-03-08T21:17:53.618+0000 7f3fae200700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:52.199334+0000 front 2024-03-08T21:16:52.199113+0000 (oldest deadline 2024-03-08T21:17:16.899180+0000) 2024-03-08T21:17:53.622 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 08 21:17:53 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-8[53709]: 2024-03-08T21:17:53.618+0000 7f3fae200700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:17:00.401255+0000 front 2024-03-08T21:17:00.401120+0000 (oldest deadline 2024-03-08T21:17:25.701340+0000) 2024-03-08T21:17:53.622 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 08 21:17:53 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-8[53709]: 2024-03-08T21:17:53.618+0000 7f3fae200700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-08T21:17:21.406602+0000 front 2024-03-08T21:17:21.406238+0000 (oldest deadline 2024-03-08T21:17:44.305767+0000) 2024-03-08T21:17:53.622 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 08 21:17:53 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-8[53709]: 2024-03-08T21:17:53.618+0000 7f3fae200700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-03-08T21:17:30.009723+0000 front 2024-03-08T21:17:30.009797+0000 (oldest deadline 2024-03-08T21:17:52.309808+0000) 2024-03-08T21:17:54.121 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 08 21:17:53 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-8[53709]: 2024-03-08T21:17:53.834+0000 7f3fb2a00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-08T21:17:54.121 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 08 21:17:53 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-8[53709]: 2024-03-08T21:17:53.834+0000 7f3fb2a00700 -1 osd.8 115 *** Got signal Terminated *** 2024-03-08T21:17:54.121 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 08 21:17:53 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-8[53709]: 2024-03-08T21:17:53.834+0000 7f3fb2a00700 -1 osd.8 115 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-08T21:17:54.460 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:17:54 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:17:54.152+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:40.479965+0000 front 2024-03-08T21:16:40.480241+0000 (oldest deadline 2024-03-08T21:17:06.379830+0000) 2024-03-08T21:17:54.461 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:17:54 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:17:54.152+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:53.283252+0000 front 2024-03-08T21:16:53.283300+0000 (oldest deadline 2024-03-08T21:17:17.383498+0000) 2024-03-08T21:17:54.461 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:17:54 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:17:54.152+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:17:03.088480+0000 front 2024-03-08T21:17:03.088666+0000 (oldest deadline 2024-03-08T21:17:25.388404+0000) 2024-03-08T21:17:54.461 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:17:54 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:17:54.152+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-03-08T21:17:10.690024+0000 front 2024-03-08T21:17:10.690629+0000 (oldest deadline 2024-03-08T21:17:34.189860+0000) 2024-03-08T21:17:54.461 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:17:54 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:17:54.152+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-08T21:17:18.192173+0000 front 2024-03-08T21:17:18.192324+0000 (oldest deadline 2024-03-08T21:17:46.992906+0000) 2024-03-08T21:17:54.461 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:17:54 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:17:54.152+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-03-08T21:17:28.694794+0000 front 2024-03-08T21:17:28.695136+0000 (oldest deadline 2024-03-08T21:17:52.794933+0000) 2024-03-08T21:17:54.508 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:17:54 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:17:54.261+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:44.858572+0000 front 2024-03-08T21:16:44.858829+0000 (oldest deadline 2024-03-08T21:17:06.558274+0000) 2024-03-08T21:17:54.508 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:17:54 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:17:54.261+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:52.961888+0000 front 2024-03-08T21:16:52.961762+0000 (oldest deadline 2024-03-08T21:17:18.862022+0000) 2024-03-08T21:17:54.508 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:17:54 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:17:54.261+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:17:01.764390+0000 front 2024-03-08T21:17:01.764438+0000 (oldest deadline 2024-03-08T21:17:24.064211+0000) 2024-03-08T21:17:54.508 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:17:54 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:17:54.261+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-03-08T21:17:10.869088+0000 front 2024-03-08T21:17:10.869147+0000 (oldest deadline 2024-03-08T21:17:34.969351+0000) 2024-03-08T21:17:54.508 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:17:54 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:17:54.261+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-08T21:17:17.871105+0000 front 2024-03-08T21:17:17.871226+0000 (oldest deadline 2024-03-08T21:17:43.770901+0000) 2024-03-08T21:17:54.508 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:17:54 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:17:54.261+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-03-08T21:17:30.173392+0000 front 2024-03-08T21:17:30.173231+0000 (oldest deadline 2024-03-08T21:17:51.773639+0000) 2024-03-08T21:17:54.871 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:17:54 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:17:54.457+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:42.614355+0000 front 2024-03-08T21:16:42.614271+0000 (oldest deadline 2024-03-08T21:17:07.914284+0000) 2024-03-08T21:17:54.871 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:17:54 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:17:54.457+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:52.016503+0000 front 2024-03-08T21:16:52.017017+0000 (oldest deadline 2024-03-08T21:17:16.116550+0000) 2024-03-08T21:17:54.871 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:17:54 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:17:54.457+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:16:57.818698+0000 front 2024-03-08T21:16:57.819201+0000 (oldest deadline 2024-03-08T21:17:26.619635+0000) 2024-03-08T21:17:54.871 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:17:54 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:17:54.457+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-08T21:17:19.424875+0000 front 2024-03-08T21:17:19.424454+0000 (oldest deadline 2024-03-08T21:17:43.425189+0000) 2024-03-08T21:17:54.871 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:17:54 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:17:54.457+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-03-08T21:17:29.327497+0000 front 2024-03-08T21:17:29.328128+0000 (oldest deadline 2024-03-08T21:17:51.627384+0000) 2024-03-08T21:17:54.872 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 08 21:17:54 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-8[53709]: 2024-03-08T21:17:54.660+0000 7f3fae200700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:43.996391+0000 front 2024-03-08T21:16:43.996418+0000 (oldest deadline 2024-03-08T21:17:09.296545+0000) 2024-03-08T21:17:54.872 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 08 21:17:54 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-8[53709]: 2024-03-08T21:17:54.660+0000 7f3fae200700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:52.199334+0000 front 2024-03-08T21:16:52.199113+0000 (oldest deadline 2024-03-08T21:17:16.899180+0000) 2024-03-08T21:17:54.872 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 08 21:17:54 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-8[53709]: 2024-03-08T21:17:54.660+0000 7f3fae200700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:17:00.401255+0000 front 2024-03-08T21:17:00.401120+0000 (oldest deadline 2024-03-08T21:17:25.701340+0000) 2024-03-08T21:17:54.872 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 08 21:17:54 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-8[53709]: 2024-03-08T21:17:54.660+0000 7f3fae200700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-08T21:17:21.406602+0000 front 2024-03-08T21:17:21.406238+0000 (oldest deadline 2024-03-08T21:17:44.305767+0000) 2024-03-08T21:17:54.872 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 08 21:17:54 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-8[53709]: 2024-03-08T21:17:54.660+0000 7f3fae200700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-03-08T21:17:30.009723+0000 front 2024-03-08T21:17:30.009797+0000 (oldest deadline 2024-03-08T21:17:52.309808+0000) 2024-03-08T21:17:55.460 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:17:55 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:17:55.110+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:40.479965+0000 front 2024-03-08T21:16:40.480241+0000 (oldest deadline 2024-03-08T21:17:06.379830+0000) 2024-03-08T21:17:55.460 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:17:55 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:17:55.110+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:53.283252+0000 front 2024-03-08T21:16:53.283300+0000 (oldest deadline 2024-03-08T21:17:17.383498+0000) 2024-03-08T21:17:55.461 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:17:55 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:17:55.110+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:17:03.088480+0000 front 2024-03-08T21:17:03.088666+0000 (oldest deadline 2024-03-08T21:17:25.388404+0000) 2024-03-08T21:17:55.461 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:17:55 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:17:55.110+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-03-08T21:17:10.690024+0000 front 2024-03-08T21:17:10.690629+0000 (oldest deadline 2024-03-08T21:17:34.189860+0000) 2024-03-08T21:17:55.461 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:17:55 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:17:55.110+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-08T21:17:18.192173+0000 front 2024-03-08T21:17:18.192324+0000 (oldest deadline 2024-03-08T21:17:46.992906+0000) 2024-03-08T21:17:55.461 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:17:55 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:17:55.110+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-03-08T21:17:28.694794+0000 front 2024-03-08T21:17:28.695136+0000 (oldest deadline 2024-03-08T21:17:52.794933+0000) 2024-03-08T21:17:55.758 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:17:55 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:17:55.298+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:44.858572+0000 front 2024-03-08T21:16:44.858829+0000 (oldest deadline 2024-03-08T21:17:06.558274+0000) 2024-03-08T21:17:55.758 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:17:55 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:17:55.298+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:52.961888+0000 front 2024-03-08T21:16:52.961762+0000 (oldest deadline 2024-03-08T21:17:18.862022+0000) 2024-03-08T21:17:55.758 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:17:55 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:17:55.298+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:17:01.764390+0000 front 2024-03-08T21:17:01.764438+0000 (oldest deadline 2024-03-08T21:17:24.064211+0000) 2024-03-08T21:17:55.758 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:17:55 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:17:55.298+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-03-08T21:17:10.869088+0000 front 2024-03-08T21:17:10.869147+0000 (oldest deadline 2024-03-08T21:17:34.969351+0000) 2024-03-08T21:17:55.758 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:17:55 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:17:55.298+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-08T21:17:17.871105+0000 front 2024-03-08T21:17:17.871226+0000 (oldest deadline 2024-03-08T21:17:43.770901+0000) 2024-03-08T21:17:55.758 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:17:55 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:17:55.298+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-03-08T21:17:30.173392+0000 front 2024-03-08T21:17:30.173231+0000 (oldest deadline 2024-03-08T21:17:51.773639+0000) 2024-03-08T21:17:55.871 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 08 21:17:55 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-8[53709]: 2024-03-08T21:17:55.682+0000 7f3fae200700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:43.996391+0000 front 2024-03-08T21:16:43.996418+0000 (oldest deadline 2024-03-08T21:17:09.296545+0000) 2024-03-08T21:17:55.871 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 08 21:17:55 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-8[53709]: 2024-03-08T21:17:55.682+0000 7f3fae200700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:52.199334+0000 front 2024-03-08T21:16:52.199113+0000 (oldest deadline 2024-03-08T21:17:16.899180+0000) 2024-03-08T21:17:55.871 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 08 21:17:55 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-8[53709]: 2024-03-08T21:17:55.682+0000 7f3fae200700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:17:00.401255+0000 front 2024-03-08T21:17:00.401120+0000 (oldest deadline 2024-03-08T21:17:25.701340+0000) 2024-03-08T21:17:55.871 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 08 21:17:55 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-8[53709]: 2024-03-08T21:17:55.682+0000 7f3fae200700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-08T21:17:21.406602+0000 front 2024-03-08T21:17:21.406238+0000 (oldest deadline 2024-03-08T21:17:44.305767+0000) 2024-03-08T21:17:55.872 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 08 21:17:55 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-8[53709]: 2024-03-08T21:17:55.682+0000 7f3fae200700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-03-08T21:17:30.009723+0000 front 2024-03-08T21:17:30.009797+0000 (oldest deadline 2024-03-08T21:17:52.309808+0000) 2024-03-08T21:17:55.872 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:17:55 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:17:55.444+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:42.614355+0000 front 2024-03-08T21:16:42.614271+0000 (oldest deadline 2024-03-08T21:17:07.914284+0000) 2024-03-08T21:17:55.872 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:17:55 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:17:55.444+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:52.016503+0000 front 2024-03-08T21:16:52.017017+0000 (oldest deadline 2024-03-08T21:17:16.116550+0000) 2024-03-08T21:17:55.872 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:17:55 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:17:55.444+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:16:57.818698+0000 front 2024-03-08T21:16:57.819201+0000 (oldest deadline 2024-03-08T21:17:26.619635+0000) 2024-03-08T21:17:55.872 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:17:55 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:17:55.444+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-08T21:17:19.424875+0000 front 2024-03-08T21:17:19.424454+0000 (oldest deadline 2024-03-08T21:17:43.425189+0000) 2024-03-08T21:17:55.872 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:17:55 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:17:55.444+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-03-08T21:17:29.327497+0000 front 2024-03-08T21:17:29.328128+0000 (oldest deadline 2024-03-08T21:17:51.627384+0000) 2024-03-08T21:17:56.460 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:17:56 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:17:56.152+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:40.479965+0000 front 2024-03-08T21:16:40.480241+0000 (oldest deadline 2024-03-08T21:17:06.379830+0000) 2024-03-08T21:17:56.460 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:17:56 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:17:56.152+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:53.283252+0000 front 2024-03-08T21:16:53.283300+0000 (oldest deadline 2024-03-08T21:17:17.383498+0000) 2024-03-08T21:17:56.460 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:17:56 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:17:56.152+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:17:03.088480+0000 front 2024-03-08T21:17:03.088666+0000 (oldest deadline 2024-03-08T21:17:25.388404+0000) 2024-03-08T21:17:56.460 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:17:56 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:17:56.152+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-03-08T21:17:10.690024+0000 front 2024-03-08T21:17:10.690629+0000 (oldest deadline 2024-03-08T21:17:34.189860+0000) 2024-03-08T21:17:56.461 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:17:56 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:17:56.152+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-08T21:17:18.192173+0000 front 2024-03-08T21:17:18.192324+0000 (oldest deadline 2024-03-08T21:17:46.992906+0000) 2024-03-08T21:17:56.461 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:17:56 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:17:56.152+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-03-08T21:17:28.694794+0000 front 2024-03-08T21:17:28.695136+0000 (oldest deadline 2024-03-08T21:17:52.794933+0000) 2024-03-08T21:17:56.758 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:17:56 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:17:56.344+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:44.858572+0000 front 2024-03-08T21:16:44.858829+0000 (oldest deadline 2024-03-08T21:17:06.558274+0000) 2024-03-08T21:17:56.758 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:17:56 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:17:56.344+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:52.961888+0000 front 2024-03-08T21:16:52.961762+0000 (oldest deadline 2024-03-08T21:17:18.862022+0000) 2024-03-08T21:17:56.758 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:17:56 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:17:56.344+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:17:01.764390+0000 front 2024-03-08T21:17:01.764438+0000 (oldest deadline 2024-03-08T21:17:24.064211+0000) 2024-03-08T21:17:56.758 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:17:56 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:17:56.344+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-03-08T21:17:10.869088+0000 front 2024-03-08T21:17:10.869147+0000 (oldest deadline 2024-03-08T21:17:34.969351+0000) 2024-03-08T21:17:56.758 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:17:56 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:17:56.344+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-08T21:17:17.871105+0000 front 2024-03-08T21:17:17.871226+0000 (oldest deadline 2024-03-08T21:17:43.770901+0000) 2024-03-08T21:17:56.758 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:17:56 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:17:56.344+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-03-08T21:17:30.173392+0000 front 2024-03-08T21:17:30.173231+0000 (oldest deadline 2024-03-08T21:17:51.773639+0000) 2024-03-08T21:17:56.871 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 08 21:17:56 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-8[53709]: 2024-03-08T21:17:56.650+0000 7f3fae200700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:43.996391+0000 front 2024-03-08T21:16:43.996418+0000 (oldest deadline 2024-03-08T21:17:09.296545+0000) 2024-03-08T21:17:56.871 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 08 21:17:56 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-8[53709]: 2024-03-08T21:17:56.650+0000 7f3fae200700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:52.199334+0000 front 2024-03-08T21:16:52.199113+0000 (oldest deadline 2024-03-08T21:17:16.899180+0000) 2024-03-08T21:17:56.871 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 08 21:17:56 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-8[53709]: 2024-03-08T21:17:56.650+0000 7f3fae200700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:17:00.401255+0000 front 2024-03-08T21:17:00.401120+0000 (oldest deadline 2024-03-08T21:17:25.701340+0000) 2024-03-08T21:17:56.871 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 08 21:17:56 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-8[53709]: 2024-03-08T21:17:56.650+0000 7f3fae200700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-08T21:17:21.406602+0000 front 2024-03-08T21:17:21.406238+0000 (oldest deadline 2024-03-08T21:17:44.305767+0000) 2024-03-08T21:17:56.871 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 08 21:17:56 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-8[53709]: 2024-03-08T21:17:56.650+0000 7f3fae200700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-03-08T21:17:30.009723+0000 front 2024-03-08T21:17:30.009797+0000 (oldest deadline 2024-03-08T21:17:52.309808+0000) 2024-03-08T21:17:56.872 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:17:56 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:17:56.460+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:42.614355+0000 front 2024-03-08T21:16:42.614271+0000 (oldest deadline 2024-03-08T21:17:07.914284+0000) 2024-03-08T21:17:56.872 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:17:56 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:17:56.460+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:52.016503+0000 front 2024-03-08T21:16:52.017017+0000 (oldest deadline 2024-03-08T21:17:16.116550+0000) 2024-03-08T21:17:56.872 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:17:56 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:17:56.460+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:16:57.818698+0000 front 2024-03-08T21:16:57.819201+0000 (oldest deadline 2024-03-08T21:17:26.619635+0000) 2024-03-08T21:17:56.872 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:17:56 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:17:56.460+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-08T21:17:19.424875+0000 front 2024-03-08T21:17:19.424454+0000 (oldest deadline 2024-03-08T21:17:43.425189+0000) 2024-03-08T21:17:56.872 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:17:56 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:17:56.460+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-03-08T21:17:29.327497+0000 front 2024-03-08T21:17:29.328128+0000 (oldest deadline 2024-03-08T21:17:51.627384+0000) 2024-03-08T21:17:57.460 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:17:57 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:17:57.174+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:40.479965+0000 front 2024-03-08T21:16:40.480241+0000 (oldest deadline 2024-03-08T21:17:06.379830+0000) 2024-03-08T21:17:57.460 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:17:57 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:17:57.174+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:53.283252+0000 front 2024-03-08T21:16:53.283300+0000 (oldest deadline 2024-03-08T21:17:17.383498+0000) 2024-03-08T21:17:57.460 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:17:57 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:17:57.174+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:17:03.088480+0000 front 2024-03-08T21:17:03.088666+0000 (oldest deadline 2024-03-08T21:17:25.388404+0000) 2024-03-08T21:17:57.460 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:17:57 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:17:57.174+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-03-08T21:17:10.690024+0000 front 2024-03-08T21:17:10.690629+0000 (oldest deadline 2024-03-08T21:17:34.189860+0000) 2024-03-08T21:17:57.460 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:17:57 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:17:57.174+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-08T21:17:18.192173+0000 front 2024-03-08T21:17:18.192324+0000 (oldest deadline 2024-03-08T21:17:46.992906+0000) 2024-03-08T21:17:57.461 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:17:57 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:17:57.174+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-03-08T21:17:28.694794+0000 front 2024-03-08T21:17:28.695136+0000 (oldest deadline 2024-03-08T21:17:52.794933+0000) 2024-03-08T21:17:57.758 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:17:57 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:17:57.343+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:44.858572+0000 front 2024-03-08T21:16:44.858829+0000 (oldest deadline 2024-03-08T21:17:06.558274+0000) 2024-03-08T21:17:57.758 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:17:57 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:17:57.343+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:52.961888+0000 front 2024-03-08T21:16:52.961762+0000 (oldest deadline 2024-03-08T21:17:18.862022+0000) 2024-03-08T21:17:57.758 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:17:57 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:17:57.343+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:17:01.764390+0000 front 2024-03-08T21:17:01.764438+0000 (oldest deadline 2024-03-08T21:17:24.064211+0000) 2024-03-08T21:17:57.758 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:17:57 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:17:57.343+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-03-08T21:17:10.869088+0000 front 2024-03-08T21:17:10.869147+0000 (oldest deadline 2024-03-08T21:17:34.969351+0000) 2024-03-08T21:17:57.758 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:17:57 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:17:57.343+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-08T21:17:17.871105+0000 front 2024-03-08T21:17:17.871226+0000 (oldest deadline 2024-03-08T21:17:43.770901+0000) 2024-03-08T21:17:57.758 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:17:57 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:17:57.343+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-03-08T21:17:30.173392+0000 front 2024-03-08T21:17:30.173231+0000 (oldest deadline 2024-03-08T21:17:51.773639+0000) 2024-03-08T21:17:57.871 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 08 21:17:57 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-8[53709]: 2024-03-08T21:17:57.679+0000 7f3fae200700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:43.996391+0000 front 2024-03-08T21:16:43.996418+0000 (oldest deadline 2024-03-08T21:17:09.296545+0000) 2024-03-08T21:17:57.871 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 08 21:17:57 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-8[53709]: 2024-03-08T21:17:57.679+0000 7f3fae200700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:52.199334+0000 front 2024-03-08T21:16:52.199113+0000 (oldest deadline 2024-03-08T21:17:16.899180+0000) 2024-03-08T21:17:57.871 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 08 21:17:57 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-8[53709]: 2024-03-08T21:17:57.679+0000 7f3fae200700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:17:00.401255+0000 front 2024-03-08T21:17:00.401120+0000 (oldest deadline 2024-03-08T21:17:25.701340+0000) 2024-03-08T21:17:57.871 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 08 21:17:57 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-8[53709]: 2024-03-08T21:17:57.679+0000 7f3fae200700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-08T21:17:21.406602+0000 front 2024-03-08T21:17:21.406238+0000 (oldest deadline 2024-03-08T21:17:44.305767+0000) 2024-03-08T21:17:57.871 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 08 21:17:57 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-8[53709]: 2024-03-08T21:17:57.679+0000 7f3fae200700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-03-08T21:17:30.009723+0000 front 2024-03-08T21:17:30.009797+0000 (oldest deadline 2024-03-08T21:17:52.309808+0000) 2024-03-08T21:17:57.871 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:17:57 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:17:57.508+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:42.614355+0000 front 2024-03-08T21:16:42.614271+0000 (oldest deadline 2024-03-08T21:17:07.914284+0000) 2024-03-08T21:17:57.871 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:17:57 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:17:57.508+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:52.016503+0000 front 2024-03-08T21:16:52.017017+0000 (oldest deadline 2024-03-08T21:17:16.116550+0000) 2024-03-08T21:17:57.871 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:17:57 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:17:57.508+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:16:57.818698+0000 front 2024-03-08T21:16:57.819201+0000 (oldest deadline 2024-03-08T21:17:26.619635+0000) 2024-03-08T21:17:57.872 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:17:57 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:17:57.508+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-08T21:17:19.424875+0000 front 2024-03-08T21:17:19.424454+0000 (oldest deadline 2024-03-08T21:17:43.425189+0000) 2024-03-08T21:17:57.872 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:17:57 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:17:57.508+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-03-08T21:17:29.327497+0000 front 2024-03-08T21:17:29.328128+0000 (oldest deadline 2024-03-08T21:17:51.627384+0000) 2024-03-08T21:17:58.460 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:17:58 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:17:58.140+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:40.479965+0000 front 2024-03-08T21:16:40.480241+0000 (oldest deadline 2024-03-08T21:17:06.379830+0000) 2024-03-08T21:17:58.460 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:17:58 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:17:58.140+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:53.283252+0000 front 2024-03-08T21:16:53.283300+0000 (oldest deadline 2024-03-08T21:17:17.383498+0000) 2024-03-08T21:17:58.460 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:17:58 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:17:58.140+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:17:03.088480+0000 front 2024-03-08T21:17:03.088666+0000 (oldest deadline 2024-03-08T21:17:25.388404+0000) 2024-03-08T21:17:58.460 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:17:58 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:17:58.140+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-03-08T21:17:10.690024+0000 front 2024-03-08T21:17:10.690629+0000 (oldest deadline 2024-03-08T21:17:34.189860+0000) 2024-03-08T21:17:58.461 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:17:58 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:17:58.140+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-08T21:17:18.192173+0000 front 2024-03-08T21:17:18.192324+0000 (oldest deadline 2024-03-08T21:17:46.992906+0000) 2024-03-08T21:17:58.461 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:17:58 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:17:58.140+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-03-08T21:17:28.694794+0000 front 2024-03-08T21:17:28.695136+0000 (oldest deadline 2024-03-08T21:17:52.794933+0000) 2024-03-08T21:17:58.758 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:17:58 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:17:58.336+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:44.858572+0000 front 2024-03-08T21:16:44.858829+0000 (oldest deadline 2024-03-08T21:17:06.558274+0000) 2024-03-08T21:17:58.758 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:17:58 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:17:58.336+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:52.961888+0000 front 2024-03-08T21:16:52.961762+0000 (oldest deadline 2024-03-08T21:17:18.862022+0000) 2024-03-08T21:17:58.758 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:17:58 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:17:58.336+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:17:01.764390+0000 front 2024-03-08T21:17:01.764438+0000 (oldest deadline 2024-03-08T21:17:24.064211+0000) 2024-03-08T21:17:58.758 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:17:58 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:17:58.336+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-03-08T21:17:10.869088+0000 front 2024-03-08T21:17:10.869147+0000 (oldest deadline 2024-03-08T21:17:34.969351+0000) 2024-03-08T21:17:58.758 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:17:58 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:17:58.336+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-08T21:17:17.871105+0000 front 2024-03-08T21:17:17.871226+0000 (oldest deadline 2024-03-08T21:17:43.770901+0000) 2024-03-08T21:17:58.758 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:17:58 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:17:58.336+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-03-08T21:17:30.173392+0000 front 2024-03-08T21:17:30.173231+0000 (oldest deadline 2024-03-08T21:17:51.773639+0000) 2024-03-08T21:17:58.871 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 08 21:17:58 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-8[53709]: 2024-03-08T21:17:58.678+0000 7f3fae200700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:43.996391+0000 front 2024-03-08T21:16:43.996418+0000 (oldest deadline 2024-03-08T21:17:09.296545+0000) 2024-03-08T21:17:58.871 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 08 21:17:58 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-8[53709]: 2024-03-08T21:17:58.678+0000 7f3fae200700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:52.199334+0000 front 2024-03-08T21:16:52.199113+0000 (oldest deadline 2024-03-08T21:17:16.899180+0000) 2024-03-08T21:17:58.871 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 08 21:17:58 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-8[53709]: 2024-03-08T21:17:58.678+0000 7f3fae200700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:17:00.401255+0000 front 2024-03-08T21:17:00.401120+0000 (oldest deadline 2024-03-08T21:17:25.701340+0000) 2024-03-08T21:17:58.872 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 08 21:17:58 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-8[53709]: 2024-03-08T21:17:58.678+0000 7f3fae200700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-08T21:17:21.406602+0000 front 2024-03-08T21:17:21.406238+0000 (oldest deadline 2024-03-08T21:17:44.305767+0000) 2024-03-08T21:17:58.872 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 08 21:17:58 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-8[53709]: 2024-03-08T21:17:58.678+0000 7f3fae200700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-03-08T21:17:30.009723+0000 front 2024-03-08T21:17:30.009797+0000 (oldest deadline 2024-03-08T21:17:52.309808+0000) 2024-03-08T21:17:58.872 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:17:58 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:17:58.540+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:42.614355+0000 front 2024-03-08T21:16:42.614271+0000 (oldest deadline 2024-03-08T21:17:07.914284+0000) 2024-03-08T21:17:58.872 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:17:58 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:17:58.540+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:52.016503+0000 front 2024-03-08T21:16:52.017017+0000 (oldest deadline 2024-03-08T21:17:16.116550+0000) 2024-03-08T21:17:58.872 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:17:58 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:17:58.540+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:16:57.818698+0000 front 2024-03-08T21:16:57.819201+0000 (oldest deadline 2024-03-08T21:17:26.619635+0000) 2024-03-08T21:17:58.872 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:17:58 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:17:58.540+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-08T21:17:19.424875+0000 front 2024-03-08T21:17:19.424454+0000 (oldest deadline 2024-03-08T21:17:43.425189+0000) 2024-03-08T21:17:58.872 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:17:58 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:17:58.540+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-03-08T21:17:29.327497+0000 front 2024-03-08T21:17:29.328128+0000 (oldest deadline 2024-03-08T21:17:51.627384+0000) 2024-03-08T21:17:59.460 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:17:59 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:17:59.103+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:40.479965+0000 front 2024-03-08T21:16:40.480241+0000 (oldest deadline 2024-03-08T21:17:06.379830+0000) 2024-03-08T21:17:59.460 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:17:59 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:17:59.103+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:53.283252+0000 front 2024-03-08T21:16:53.283300+0000 (oldest deadline 2024-03-08T21:17:17.383498+0000) 2024-03-08T21:17:59.460 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:17:59 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:17:59.103+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:17:03.088480+0000 front 2024-03-08T21:17:03.088666+0000 (oldest deadline 2024-03-08T21:17:25.388404+0000) 2024-03-08T21:17:59.461 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:17:59 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:17:59.103+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-03-08T21:17:10.690024+0000 front 2024-03-08T21:17:10.690629+0000 (oldest deadline 2024-03-08T21:17:34.189860+0000) 2024-03-08T21:17:59.461 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:17:59 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:17:59.103+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-08T21:17:18.192173+0000 front 2024-03-08T21:17:18.192324+0000 (oldest deadline 2024-03-08T21:17:46.992906+0000) 2024-03-08T21:17:59.461 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:17:59 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:17:59.103+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-03-08T21:17:28.694794+0000 front 2024-03-08T21:17:28.695136+0000 (oldest deadline 2024-03-08T21:17:52.794933+0000) 2024-03-08T21:17:59.621 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 08 21:17:59 smithi204 podman[73948]: 2024-03-08 21:17:59.374356328 +0000 UTC m=+5.653082663 container died 2cf02f530e2fbf5d87d3d92ee54e3381cafcff4750f5375352738d8e1cd000db (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-8, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=rishabh-2024mar8-7d63482, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-08T21:17:59.621 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:17:59 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:17:59.547+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:42.614355+0000 front 2024-03-08T21:16:42.614271+0000 (oldest deadline 2024-03-08T21:17:07.914284+0000) 2024-03-08T21:17:59.621 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:17:59 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:17:59.547+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:52.016503+0000 front 2024-03-08T21:16:52.017017+0000 (oldest deadline 2024-03-08T21:17:16.116550+0000) 2024-03-08T21:17:59.621 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:17:59 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:17:59.547+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:16:57.818698+0000 front 2024-03-08T21:16:57.819201+0000 (oldest deadline 2024-03-08T21:17:26.619635+0000) 2024-03-08T21:17:59.621 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:17:59 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:17:59.547+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-08T21:17:19.424875+0000 front 2024-03-08T21:17:19.424454+0000 (oldest deadline 2024-03-08T21:17:43.425189+0000) 2024-03-08T21:17:59.621 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:17:59 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:17:59.547+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-03-08T21:17:29.327497+0000 front 2024-03-08T21:17:29.328128+0000 (oldest deadline 2024-03-08T21:17:51.627384+0000) 2024-03-08T21:17:59.758 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:17:59 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:17:59.308+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:44.858572+0000 front 2024-03-08T21:16:44.858829+0000 (oldest deadline 2024-03-08T21:17:06.558274+0000) 2024-03-08T21:17:59.758 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:17:59 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:17:59.308+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:52.961888+0000 front 2024-03-08T21:16:52.961762+0000 (oldest deadline 2024-03-08T21:17:18.862022+0000) 2024-03-08T21:17:59.758 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:17:59 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:17:59.308+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:17:01.764390+0000 front 2024-03-08T21:17:01.764438+0000 (oldest deadline 2024-03-08T21:17:24.064211+0000) 2024-03-08T21:17:59.758 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:17:59 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:17:59.308+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-03-08T21:17:10.869088+0000 front 2024-03-08T21:17:10.869147+0000 (oldest deadline 2024-03-08T21:17:34.969351+0000) 2024-03-08T21:17:59.758 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:17:59 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:17:59.308+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-08T21:17:17.871105+0000 front 2024-03-08T21:17:17.871226+0000 (oldest deadline 2024-03-08T21:17:43.770901+0000) 2024-03-08T21:17:59.758 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:17:59 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:17:59.308+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-03-08T21:17:30.173392+0000 front 2024-03-08T21:17:30.173231+0000 (oldest deadline 2024-03-08T21:17:51.773639+0000) 2024-03-08T21:18:00.089 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 08 21:17:59 smithi204 podman[73948]: 2024-03-08 21:17:59.821116764 +0000 UTC m=+6.099843146 container cleanup 2cf02f530e2fbf5d87d3d92ee54e3381cafcff4750f5375352738d8e1cd000db (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-8, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=rishabh-2024mar8-7d63482, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-03-08T21:18:00.090 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 08 21:17:59 smithi204 bash[73948]: ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-8 2024-03-08T21:18:00.370 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 08 21:18:00 smithi204 podman[73966]: 2024-03-08 21:18:00.1141144 +0000 UTC m=+0.722313932 container remove 2cf02f530e2fbf5d87d3d92ee54e3381cafcff4750f5375352738d8e1cd000db (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-8, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=rishabh-2024mar8-7d63482, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-03-08T21:18:00.460 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:18:00 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:18:00.061+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:40.479965+0000 front 2024-03-08T21:16:40.480241+0000 (oldest deadline 2024-03-08T21:17:06.379830+0000) 2024-03-08T21:18:00.460 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:18:00 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:18:00.061+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:53.283252+0000 front 2024-03-08T21:16:53.283300+0000 (oldest deadline 2024-03-08T21:17:17.383498+0000) 2024-03-08T21:18:00.460 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:18:00 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:18:00.061+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:17:03.088480+0000 front 2024-03-08T21:17:03.088666+0000 (oldest deadline 2024-03-08T21:17:25.388404+0000) 2024-03-08T21:18:00.460 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:18:00 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:18:00.061+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-03-08T21:17:10.690024+0000 front 2024-03-08T21:17:10.690629+0000 (oldest deadline 2024-03-08T21:17:34.189860+0000) 2024-03-08T21:18:00.460 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:18:00 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:18:00.061+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-08T21:17:18.192173+0000 front 2024-03-08T21:17:18.192324+0000 (oldest deadline 2024-03-08T21:17:46.992906+0000) 2024-03-08T21:18:00.460 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:18:00 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:18:00.061+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-03-08T21:17:28.694794+0000 front 2024-03-08T21:17:28.695136+0000 (oldest deadline 2024-03-08T21:17:52.794933+0000) 2024-03-08T21:18:00.758 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:18:00 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:18:00.298+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:44.858572+0000 front 2024-03-08T21:16:44.858829+0000 (oldest deadline 2024-03-08T21:17:06.558274+0000) 2024-03-08T21:18:00.758 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:18:00 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:18:00.298+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:52.961888+0000 front 2024-03-08T21:16:52.961762+0000 (oldest deadline 2024-03-08T21:17:18.862022+0000) 2024-03-08T21:18:00.758 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:18:00 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:18:00.298+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:17:01.764390+0000 front 2024-03-08T21:17:01.764438+0000 (oldest deadline 2024-03-08T21:17:24.064211+0000) 2024-03-08T21:18:00.758 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:18:00 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:18:00.298+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-03-08T21:17:10.869088+0000 front 2024-03-08T21:17:10.869147+0000 (oldest deadline 2024-03-08T21:17:34.969351+0000) 2024-03-08T21:18:00.758 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:18:00 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:18:00.298+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-08T21:17:17.871105+0000 front 2024-03-08T21:17:17.871226+0000 (oldest deadline 2024-03-08T21:17:43.770901+0000) 2024-03-08T21:18:00.758 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:18:00 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:18:00.298+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-03-08T21:17:30.173392+0000 front 2024-03-08T21:17:30.173231+0000 (oldest deadline 2024-03-08T21:17:51.773639+0000) 2024-03-08T21:18:00.871 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:18:00 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:18:00.517+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:42.614355+0000 front 2024-03-08T21:16:42.614271+0000 (oldest deadline 2024-03-08T21:17:07.914284+0000) 2024-03-08T21:18:00.871 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:18:00 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:18:00.517+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:52.016503+0000 front 2024-03-08T21:16:52.017017+0000 (oldest deadline 2024-03-08T21:17:16.116550+0000) 2024-03-08T21:18:00.871 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:18:00 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:18:00.517+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:16:57.818698+0000 front 2024-03-08T21:16:57.819201+0000 (oldest deadline 2024-03-08T21:17:26.619635+0000) 2024-03-08T21:18:00.871 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:18:00 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:18:00.517+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-08T21:17:19.424875+0000 front 2024-03-08T21:17:19.424454+0000 (oldest deadline 2024-03-08T21:17:43.425189+0000) 2024-03-08T21:18:00.871 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:18:00 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:18:00.517+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-03-08T21:17:29.327497+0000 front 2024-03-08T21:17:29.328128+0000 (oldest deadline 2024-03-08T21:17:51.627384+0000) 2024-03-08T21:18:01.261 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 08 21:18:00 smithi204 podman[74057]: 2024-03-08 21:18:00.835332627 +0000 UTC m=+0.033788323 image pull a272146cb810b1fd4a53627f019472132e30966d5c25b01bfc44a001295f1249 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd 2024-03-08T21:18:01.261 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 08 21:18:00 smithi204 podman[74057]: 2024-03-08 21:18:00.940025474 +0000 UTC m=+0.138481199 container create 4493868a783856c48fb60b0f671851fedec433e9c9a90b2b20a477949f0b63b2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-8-deactivate, RELEASE=rishabh-2024mar8-7d63482, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0) 2024-03-08T21:18:01.460 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:18:01 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:18:01.096+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:40.479965+0000 front 2024-03-08T21:16:40.480241+0000 (oldest deadline 2024-03-08T21:17:06.379830+0000) 2024-03-08T21:18:01.460 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:18:01 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:18:01.096+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:53.283252+0000 front 2024-03-08T21:16:53.283300+0000 (oldest deadline 2024-03-08T21:17:17.383498+0000) 2024-03-08T21:18:01.460 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:18:01 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:18:01.096+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:17:03.088480+0000 front 2024-03-08T21:17:03.088666+0000 (oldest deadline 2024-03-08T21:17:25.388404+0000) 2024-03-08T21:18:01.460 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:18:01 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:18:01.096+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-03-08T21:17:10.690024+0000 front 2024-03-08T21:17:10.690629+0000 (oldest deadline 2024-03-08T21:17:34.189860+0000) 2024-03-08T21:18:01.460 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:18:01 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:18:01.096+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-08T21:17:18.192173+0000 front 2024-03-08T21:17:18.192324+0000 (oldest deadline 2024-03-08T21:17:46.992906+0000) 2024-03-08T21:18:01.460 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:18:01 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:18:01.096+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-03-08T21:17:28.694794+0000 front 2024-03-08T21:17:28.695136+0000 (oldest deadline 2024-03-08T21:17:52.794933+0000) 2024-03-08T21:18:01.621 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:18:01 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:18:01.514+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:42.614355+0000 front 2024-03-08T21:16:42.614271+0000 (oldest deadline 2024-03-08T21:17:07.914284+0000) 2024-03-08T21:18:01.621 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:18:01 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:18:01.514+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:52.016503+0000 front 2024-03-08T21:16:52.017017+0000 (oldest deadline 2024-03-08T21:17:16.116550+0000) 2024-03-08T21:18:01.621 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:18:01 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:18:01.514+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:16:57.818698+0000 front 2024-03-08T21:16:57.819201+0000 (oldest deadline 2024-03-08T21:17:26.619635+0000) 2024-03-08T21:18:01.621 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:18:01 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:18:01.514+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-08T21:17:19.424875+0000 front 2024-03-08T21:17:19.424454+0000 (oldest deadline 2024-03-08T21:17:43.425189+0000) 2024-03-08T21:18:01.621 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:18:01 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:18:01.514+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-03-08T21:17:29.327497+0000 front 2024-03-08T21:17:29.328128+0000 (oldest deadline 2024-03-08T21:17:51.627384+0000) 2024-03-08T21:18:01.621 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:18:01 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:18:01.514+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.107:6822 osd.6 since back 2024-03-08T21:17:40.231398+0000 front 2024-03-08T21:17:40.231589+0000 (oldest deadline 2024-03-08T21:18:01.331142+0000) 2024-03-08T21:18:01.621 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 08 21:18:01 smithi204 podman[74057]: 2024-03-08 21:18:01.28529542 +0000 UTC m=+0.483751151 container init 4493868a783856c48fb60b0f671851fedec433e9c9a90b2b20a477949f0b63b2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-8-deactivate, ceph=True, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , RELEASE=rishabh-2024mar8-7d63482, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-03-08T21:18:01.622 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 08 21:18:01 smithi204 podman[74057]: 2024-03-08 21:18:01.297565967 +0000 UTC m=+0.496021669 container start 4493868a783856c48fb60b0f671851fedec433e9c9a90b2b20a477949f0b63b2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-8-deactivate, GIT_BRANCH=HEAD, RELEASE=rishabh-2024mar8-7d63482, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-08T21:18:01.622 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 08 21:18:01 smithi204 podman[74057]: 2024-03-08 21:18:01.401197769 +0000 UTC m=+0.599653537 container attach 4493868a783856c48fb60b0f671851fedec433e9c9a90b2b20a477949f0b63b2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-8-deactivate, io.buildah.version=1.33.5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=rishabh-2024mar8-7d63482, org.label-schema.build-date=20240307, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-03-08T21:18:01.757 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:18:01 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:18:01.328+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:44.858572+0000 front 2024-03-08T21:16:44.858829+0000 (oldest deadline 2024-03-08T21:17:06.558274+0000) 2024-03-08T21:18:01.757 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:18:01 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:18:01.328+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:52.961888+0000 front 2024-03-08T21:16:52.961762+0000 (oldest deadline 2024-03-08T21:17:18.862022+0000) 2024-03-08T21:18:01.757 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:18:01 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:18:01.328+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:17:01.764390+0000 front 2024-03-08T21:17:01.764438+0000 (oldest deadline 2024-03-08T21:17:24.064211+0000) 2024-03-08T21:18:01.757 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:18:01 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:18:01.328+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-03-08T21:17:10.869088+0000 front 2024-03-08T21:17:10.869147+0000 (oldest deadline 2024-03-08T21:17:34.969351+0000) 2024-03-08T21:18:01.758 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:18:01 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:18:01.328+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-08T21:17:17.871105+0000 front 2024-03-08T21:17:17.871226+0000 (oldest deadline 2024-03-08T21:17:43.770901+0000) 2024-03-08T21:18:01.758 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:18:01 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:18:01.328+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-03-08T21:17:30.173392+0000 front 2024-03-08T21:17:30.173231+0000 (oldest deadline 2024-03-08T21:17:51.773639+0000) 2024-03-08T21:18:02.051 INFO:journalctl@ceph.osd.8.smithi204.stdout:Mar 08 21:18:01 smithi204 podman[74057]: 2024-03-08 21:18:01.727662682 +0000 UTC m=+0.926118397 container died 4493868a783856c48fb60b0f671851fedec433e9c9a90b2b20a477949f0b63b2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-8-deactivate, RELEASE=rishabh-2024mar8-7d63482, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-03-08T21:18:02.460 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:18:02 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:18:02.141+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:40.479965+0000 front 2024-03-08T21:16:40.480241+0000 (oldest deadline 2024-03-08T21:17:06.379830+0000) 2024-03-08T21:18:02.460 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:18:02 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:18:02.141+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:53.283252+0000 front 2024-03-08T21:16:53.283300+0000 (oldest deadline 2024-03-08T21:17:17.383498+0000) 2024-03-08T21:18:02.460 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:18:02 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:18:02.141+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:17:03.088480+0000 front 2024-03-08T21:17:03.088666+0000 (oldest deadline 2024-03-08T21:17:25.388404+0000) 2024-03-08T21:18:02.460 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:18:02 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:18:02.141+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-03-08T21:17:10.690024+0000 front 2024-03-08T21:17:10.690629+0000 (oldest deadline 2024-03-08T21:17:34.189860+0000) 2024-03-08T21:18:02.460 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:18:02 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:18:02.141+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-08T21:17:18.192173+0000 front 2024-03-08T21:17:18.192324+0000 (oldest deadline 2024-03-08T21:17:46.992906+0000) 2024-03-08T21:18:02.461 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:18:02 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:18:02.141+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-03-08T21:17:28.694794+0000 front 2024-03-08T21:17:28.695136+0000 (oldest deadline 2024-03-08T21:17:52.794933+0000) 2024-03-08T21:18:02.488 DEBUG:teuthology.orchestra.run.smithi204:> sudo pkill -f 'journalctl -f -n 0 -u ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715@osd.8.service' 2024-03-08T21:18:02.607 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-08T21:18:02.608 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-03-08T21:18:02.608 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-03-08T21:18:02.608 DEBUG:teuthology.orchestra.run.smithi107:> sudo systemctl stop ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715@osd.9 2024-03-08T21:18:02.758 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:18:02 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:18:02.334+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:44.858572+0000 front 2024-03-08T21:16:44.858829+0000 (oldest deadline 2024-03-08T21:17:06.558274+0000) 2024-03-08T21:18:02.758 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:18:02 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:18:02.334+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:52.961888+0000 front 2024-03-08T21:16:52.961762+0000 (oldest deadline 2024-03-08T21:17:18.862022+0000) 2024-03-08T21:18:02.758 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:18:02 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:18:02.334+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:17:01.764390+0000 front 2024-03-08T21:17:01.764438+0000 (oldest deadline 2024-03-08T21:17:24.064211+0000) 2024-03-08T21:18:02.758 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:18:02 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:18:02.334+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-03-08T21:17:10.869088+0000 front 2024-03-08T21:17:10.869147+0000 (oldest deadline 2024-03-08T21:17:34.969351+0000) 2024-03-08T21:18:02.758 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:18:02 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:18:02.334+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-08T21:17:17.871105+0000 front 2024-03-08T21:17:17.871226+0000 (oldest deadline 2024-03-08T21:17:43.770901+0000) 2024-03-08T21:18:02.758 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:18:02 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:18:02.334+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-03-08T21:17:30.173392+0000 front 2024-03-08T21:17:30.173231+0000 (oldest deadline 2024-03-08T21:17:51.773639+0000) 2024-03-08T21:18:02.871 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:18:02 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:18:02.562+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:42.614355+0000 front 2024-03-08T21:16:42.614271+0000 (oldest deadline 2024-03-08T21:17:07.914284+0000) 2024-03-08T21:18:02.871 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:18:02 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:18:02.562+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:52.016503+0000 front 2024-03-08T21:16:52.017017+0000 (oldest deadline 2024-03-08T21:17:16.116550+0000) 2024-03-08T21:18:02.871 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:18:02 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:18:02.562+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:16:57.818698+0000 front 2024-03-08T21:16:57.819201+0000 (oldest deadline 2024-03-08T21:17:26.619635+0000) 2024-03-08T21:18:02.872 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:18:02 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:18:02.562+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-08T21:17:19.424875+0000 front 2024-03-08T21:17:19.424454+0000 (oldest deadline 2024-03-08T21:17:43.425189+0000) 2024-03-08T21:18:02.872 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:18:02 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:18:02.562+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-03-08T21:17:29.327497+0000 front 2024-03-08T21:17:29.328128+0000 (oldest deadline 2024-03-08T21:17:51.627384+0000) 2024-03-08T21:18:02.872 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:18:02 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:18:02.562+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.107:6822 osd.6 since back 2024-03-08T21:17:40.231398+0000 front 2024-03-08T21:17:40.231589+0000 (oldest deadline 2024-03-08T21:18:01.331142+0000) 2024-03-08T21:18:02.960 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:18:02 smithi107 systemd[1]: Stopping Ceph osd.9 for dc20f0f8-dd8c-11ee-95c6-87774f69a715... 2024-03-08T21:18:03.460 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:18:03 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:18:03.191+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:40.479965+0000 front 2024-03-08T21:16:40.480241+0000 (oldest deadline 2024-03-08T21:17:06.379830+0000) 2024-03-08T21:18:03.461 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:18:03 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:18:03.191+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:53.283252+0000 front 2024-03-08T21:16:53.283300+0000 (oldest deadline 2024-03-08T21:17:17.383498+0000) 2024-03-08T21:18:03.461 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:18:03 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:18:03.191+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:17:03.088480+0000 front 2024-03-08T21:17:03.088666+0000 (oldest deadline 2024-03-08T21:17:25.388404+0000) 2024-03-08T21:18:03.461 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:18:03 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:18:03.191+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-03-08T21:17:10.690024+0000 front 2024-03-08T21:17:10.690629+0000 (oldest deadline 2024-03-08T21:17:34.189860+0000) 2024-03-08T21:18:03.461 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:18:03 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:18:03.191+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-08T21:17:18.192173+0000 front 2024-03-08T21:17:18.192324+0000 (oldest deadline 2024-03-08T21:17:46.992906+0000) 2024-03-08T21:18:03.461 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:18:03 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:18:03.191+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-03-08T21:17:28.694794+0000 front 2024-03-08T21:17:28.695136+0000 (oldest deadline 2024-03-08T21:17:52.794933+0000) 2024-03-08T21:18:03.461 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:18:03 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:18:03.278+0000 7f4539e00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-08T21:18:03.461 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:18:03 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:18:03.278+0000 7f4539e00700 -1 osd.9 115 *** Got signal Terminated *** 2024-03-08T21:18:03.461 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:18:03 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:18:03.278+0000 7f4539e00700 -1 osd.9 115 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-08T21:18:03.757 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:18:03 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:18:03.378+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:44.858572+0000 front 2024-03-08T21:16:44.858829+0000 (oldest deadline 2024-03-08T21:17:06.558274+0000) 2024-03-08T21:18:03.757 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:18:03 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:18:03.378+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:52.961888+0000 front 2024-03-08T21:16:52.961762+0000 (oldest deadline 2024-03-08T21:17:18.862022+0000) 2024-03-08T21:18:03.758 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:18:03 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:18:03.378+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:17:01.764390+0000 front 2024-03-08T21:17:01.764438+0000 (oldest deadline 2024-03-08T21:17:24.064211+0000) 2024-03-08T21:18:03.758 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:18:03 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:18:03.378+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-03-08T21:17:10.869088+0000 front 2024-03-08T21:17:10.869147+0000 (oldest deadline 2024-03-08T21:17:34.969351+0000) 2024-03-08T21:18:03.758 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:18:03 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:18:03.378+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-08T21:17:17.871105+0000 front 2024-03-08T21:17:17.871226+0000 (oldest deadline 2024-03-08T21:17:43.770901+0000) 2024-03-08T21:18:03.758 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:18:03 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:18:03.378+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-03-08T21:17:30.173392+0000 front 2024-03-08T21:17:30.173231+0000 (oldest deadline 2024-03-08T21:17:51.773639+0000) 2024-03-08T21:18:03.871 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:18:03 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:18:03.571+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:42.614355+0000 front 2024-03-08T21:16:42.614271+0000 (oldest deadline 2024-03-08T21:17:07.914284+0000) 2024-03-08T21:18:03.871 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:18:03 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:18:03.571+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:52.016503+0000 front 2024-03-08T21:16:52.017017+0000 (oldest deadline 2024-03-08T21:17:16.116550+0000) 2024-03-08T21:18:03.871 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:18:03 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:18:03.571+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:16:57.818698+0000 front 2024-03-08T21:16:57.819201+0000 (oldest deadline 2024-03-08T21:17:26.619635+0000) 2024-03-08T21:18:03.871 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:18:03 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:18:03.571+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-08T21:17:19.424875+0000 front 2024-03-08T21:17:19.424454+0000 (oldest deadline 2024-03-08T21:17:43.425189+0000) 2024-03-08T21:18:03.871 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:18:03 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:18:03.571+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-03-08T21:17:29.327497+0000 front 2024-03-08T21:17:29.328128+0000 (oldest deadline 2024-03-08T21:17:51.627384+0000) 2024-03-08T21:18:03.871 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:18:03 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:18:03.571+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.107:6822 osd.6 since back 2024-03-08T21:17:40.231398+0000 front 2024-03-08T21:17:40.231589+0000 (oldest deadline 2024-03-08T21:18:01.331142+0000) 2024-03-08T21:18:04.460 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:18:04 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:18:04.203+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:40.479965+0000 front 2024-03-08T21:16:40.480241+0000 (oldest deadline 2024-03-08T21:17:06.379830+0000) 2024-03-08T21:18:04.460 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:18:04 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:18:04.203+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:53.283252+0000 front 2024-03-08T21:16:53.283300+0000 (oldest deadline 2024-03-08T21:17:17.383498+0000) 2024-03-08T21:18:04.460 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:18:04 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:18:04.203+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:17:03.088480+0000 front 2024-03-08T21:17:03.088666+0000 (oldest deadline 2024-03-08T21:17:25.388404+0000) 2024-03-08T21:18:04.460 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:18:04 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:18:04.203+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-03-08T21:17:10.690024+0000 front 2024-03-08T21:17:10.690629+0000 (oldest deadline 2024-03-08T21:17:34.189860+0000) 2024-03-08T21:18:04.461 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:18:04 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:18:04.203+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-08T21:17:18.192173+0000 front 2024-03-08T21:17:18.192324+0000 (oldest deadline 2024-03-08T21:17:46.992906+0000) 2024-03-08T21:18:04.461 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:18:04 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:18:04.203+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-03-08T21:17:28.694794+0000 front 2024-03-08T21:17:28.695136+0000 (oldest deadline 2024-03-08T21:17:52.794933+0000) 2024-03-08T21:18:04.758 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:18:04 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:18:04.331+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:44.858572+0000 front 2024-03-08T21:16:44.858829+0000 (oldest deadline 2024-03-08T21:17:06.558274+0000) 2024-03-08T21:18:04.758 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:18:04 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:18:04.331+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:52.961888+0000 front 2024-03-08T21:16:52.961762+0000 (oldest deadline 2024-03-08T21:17:18.862022+0000) 2024-03-08T21:18:04.758 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:18:04 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:18:04.331+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:17:01.764390+0000 front 2024-03-08T21:17:01.764438+0000 (oldest deadline 2024-03-08T21:17:24.064211+0000) 2024-03-08T21:18:04.758 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:18:04 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:18:04.331+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-03-08T21:17:10.869088+0000 front 2024-03-08T21:17:10.869147+0000 (oldest deadline 2024-03-08T21:17:34.969351+0000) 2024-03-08T21:18:04.758 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:18:04 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:18:04.331+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-08T21:17:17.871105+0000 front 2024-03-08T21:17:17.871226+0000 (oldest deadline 2024-03-08T21:17:43.770901+0000) 2024-03-08T21:18:04.758 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:18:04 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:18:04.331+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-03-08T21:17:30.173392+0000 front 2024-03-08T21:17:30.173231+0000 (oldest deadline 2024-03-08T21:17:51.773639+0000) 2024-03-08T21:18:04.871 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:18:04 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:18:04.557+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:42.614355+0000 front 2024-03-08T21:16:42.614271+0000 (oldest deadline 2024-03-08T21:17:07.914284+0000) 2024-03-08T21:18:04.871 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:18:04 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:18:04.557+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:52.016503+0000 front 2024-03-08T21:16:52.017017+0000 (oldest deadline 2024-03-08T21:17:16.116550+0000) 2024-03-08T21:18:04.871 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:18:04 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:18:04.557+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:16:57.818698+0000 front 2024-03-08T21:16:57.819201+0000 (oldest deadline 2024-03-08T21:17:26.619635+0000) 2024-03-08T21:18:04.871 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:18:04 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:18:04.557+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-08T21:17:19.424875+0000 front 2024-03-08T21:17:19.424454+0000 (oldest deadline 2024-03-08T21:17:43.425189+0000) 2024-03-08T21:18:04.871 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:18:04 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:18:04.557+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-03-08T21:17:29.327497+0000 front 2024-03-08T21:17:29.328128+0000 (oldest deadline 2024-03-08T21:17:51.627384+0000) 2024-03-08T21:18:04.871 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:18:04 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:18:04.557+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.107:6822 osd.6 since back 2024-03-08T21:17:40.231398+0000 front 2024-03-08T21:17:40.231589+0000 (oldest deadline 2024-03-08T21:18:01.331142+0000) 2024-03-08T21:18:05.460 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:18:05 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:18:05.185+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:40.479965+0000 front 2024-03-08T21:16:40.480241+0000 (oldest deadline 2024-03-08T21:17:06.379830+0000) 2024-03-08T21:18:05.460 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:18:05 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:18:05.185+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:53.283252+0000 front 2024-03-08T21:16:53.283300+0000 (oldest deadline 2024-03-08T21:17:17.383498+0000) 2024-03-08T21:18:05.461 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:18:05 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:18:05.185+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:17:03.088480+0000 front 2024-03-08T21:17:03.088666+0000 (oldest deadline 2024-03-08T21:17:25.388404+0000) 2024-03-08T21:18:05.461 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:18:05 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:18:05.185+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-03-08T21:17:10.690024+0000 front 2024-03-08T21:17:10.690629+0000 (oldest deadline 2024-03-08T21:17:34.189860+0000) 2024-03-08T21:18:05.461 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:18:05 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:18:05.185+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-08T21:17:18.192173+0000 front 2024-03-08T21:17:18.192324+0000 (oldest deadline 2024-03-08T21:17:46.992906+0000) 2024-03-08T21:18:05.461 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:18:05 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:18:05.185+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-03-08T21:17:28.694794+0000 front 2024-03-08T21:17:28.695136+0000 (oldest deadline 2024-03-08T21:17:52.794933+0000) 2024-03-08T21:18:05.461 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:18:05 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:18:05.185+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.107:6822 osd.6 since back 2024-03-08T21:17:38.498151+0000 front 2024-03-08T21:17:38.498257+0000 (oldest deadline 2024-03-08T21:18:04.398068+0000) 2024-03-08T21:18:05.758 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:18:05 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:18:05.375+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:44.858572+0000 front 2024-03-08T21:16:44.858829+0000 (oldest deadline 2024-03-08T21:17:06.558274+0000) 2024-03-08T21:18:05.758 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:18:05 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:18:05.375+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:52.961888+0000 front 2024-03-08T21:16:52.961762+0000 (oldest deadline 2024-03-08T21:17:18.862022+0000) 2024-03-08T21:18:05.758 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:18:05 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:18:05.375+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:17:01.764390+0000 front 2024-03-08T21:17:01.764438+0000 (oldest deadline 2024-03-08T21:17:24.064211+0000) 2024-03-08T21:18:05.758 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:18:05 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:18:05.375+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-03-08T21:17:10.869088+0000 front 2024-03-08T21:17:10.869147+0000 (oldest deadline 2024-03-08T21:17:34.969351+0000) 2024-03-08T21:18:05.758 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:18:05 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:18:05.375+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-08T21:17:17.871105+0000 front 2024-03-08T21:17:17.871226+0000 (oldest deadline 2024-03-08T21:17:43.770901+0000) 2024-03-08T21:18:05.758 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:18:05 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:18:05.375+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-03-08T21:17:30.173392+0000 front 2024-03-08T21:17:30.173231+0000 (oldest deadline 2024-03-08T21:17:51.773639+0000) 2024-03-08T21:18:05.871 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:18:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:18:05.574+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:42.614355+0000 front 2024-03-08T21:16:42.614271+0000 (oldest deadline 2024-03-08T21:17:07.914284+0000) 2024-03-08T21:18:05.871 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:18:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:18:05.574+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:52.016503+0000 front 2024-03-08T21:16:52.017017+0000 (oldest deadline 2024-03-08T21:17:16.116550+0000) 2024-03-08T21:18:05.871 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:18:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:18:05.574+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:16:57.818698+0000 front 2024-03-08T21:16:57.819201+0000 (oldest deadline 2024-03-08T21:17:26.619635+0000) 2024-03-08T21:18:05.871 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:18:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:18:05.574+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-08T21:17:19.424875+0000 front 2024-03-08T21:17:19.424454+0000 (oldest deadline 2024-03-08T21:17:43.425189+0000) 2024-03-08T21:18:05.871 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:18:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:18:05.574+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-03-08T21:17:29.327497+0000 front 2024-03-08T21:17:29.328128+0000 (oldest deadline 2024-03-08T21:17:51.627384+0000) 2024-03-08T21:18:05.871 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:18:05 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:18:05.574+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.107:6822 osd.6 since back 2024-03-08T21:17:40.231398+0000 front 2024-03-08T21:17:40.231589+0000 (oldest deadline 2024-03-08T21:18:01.331142+0000) 2024-03-08T21:18:06.460 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:18:06 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:18:06.225+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:40.479965+0000 front 2024-03-08T21:16:40.480241+0000 (oldest deadline 2024-03-08T21:17:06.379830+0000) 2024-03-08T21:18:06.460 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:18:06 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:18:06.225+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:53.283252+0000 front 2024-03-08T21:16:53.283300+0000 (oldest deadline 2024-03-08T21:17:17.383498+0000) 2024-03-08T21:18:06.461 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:18:06 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:18:06.225+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:17:03.088480+0000 front 2024-03-08T21:17:03.088666+0000 (oldest deadline 2024-03-08T21:17:25.388404+0000) 2024-03-08T21:18:06.461 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:18:06 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:18:06.225+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-03-08T21:17:10.690024+0000 front 2024-03-08T21:17:10.690629+0000 (oldest deadline 2024-03-08T21:17:34.189860+0000) 2024-03-08T21:18:06.461 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:18:06 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:18:06.225+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-08T21:17:18.192173+0000 front 2024-03-08T21:17:18.192324+0000 (oldest deadline 2024-03-08T21:17:46.992906+0000) 2024-03-08T21:18:06.461 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:18:06 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:18:06.225+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-03-08T21:17:28.694794+0000 front 2024-03-08T21:17:28.695136+0000 (oldest deadline 2024-03-08T21:17:52.794933+0000) 2024-03-08T21:18:06.461 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:18:06 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:18:06.225+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.107:6822 osd.6 since back 2024-03-08T21:17:38.498151+0000 front 2024-03-08T21:17:38.498257+0000 (oldest deadline 2024-03-08T21:18:04.398068+0000) 2024-03-08T21:18:06.758 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:18:06 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:18:06.402+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:44.858572+0000 front 2024-03-08T21:16:44.858829+0000 (oldest deadline 2024-03-08T21:17:06.558274+0000) 2024-03-08T21:18:06.758 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:18:06 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:18:06.402+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:52.961888+0000 front 2024-03-08T21:16:52.961762+0000 (oldest deadline 2024-03-08T21:17:18.862022+0000) 2024-03-08T21:18:06.758 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:18:06 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:18:06.402+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:17:01.764390+0000 front 2024-03-08T21:17:01.764438+0000 (oldest deadline 2024-03-08T21:17:24.064211+0000) 2024-03-08T21:18:06.758 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:18:06 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:18:06.402+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-03-08T21:17:10.869088+0000 front 2024-03-08T21:17:10.869147+0000 (oldest deadline 2024-03-08T21:17:34.969351+0000) 2024-03-08T21:18:06.758 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:18:06 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:18:06.402+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-08T21:17:17.871105+0000 front 2024-03-08T21:17:17.871226+0000 (oldest deadline 2024-03-08T21:17:43.770901+0000) 2024-03-08T21:18:06.758 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:18:06 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:18:06.402+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-03-08T21:17:30.173392+0000 front 2024-03-08T21:17:30.173231+0000 (oldest deadline 2024-03-08T21:17:51.773639+0000) 2024-03-08T21:18:06.758 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:18:06 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:18:06.402+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.107:6822 osd.6 since back 2024-03-08T21:17:39.976161+0000 front 2024-03-08T21:17:39.976402+0000 (oldest deadline 2024-03-08T21:18:05.876072+0000) 2024-03-08T21:18:06.871 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:18:06 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:18:06.574+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:42.614355+0000 front 2024-03-08T21:16:42.614271+0000 (oldest deadline 2024-03-08T21:17:07.914284+0000) 2024-03-08T21:18:06.871 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:18:06 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:18:06.574+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:52.016503+0000 front 2024-03-08T21:16:52.017017+0000 (oldest deadline 2024-03-08T21:17:16.116550+0000) 2024-03-08T21:18:06.871 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:18:06 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:18:06.574+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:16:57.818698+0000 front 2024-03-08T21:16:57.819201+0000 (oldest deadline 2024-03-08T21:17:26.619635+0000) 2024-03-08T21:18:06.871 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:18:06 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:18:06.574+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-08T21:17:19.424875+0000 front 2024-03-08T21:17:19.424454+0000 (oldest deadline 2024-03-08T21:17:43.425189+0000) 2024-03-08T21:18:06.871 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:18:06 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:18:06.574+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-03-08T21:17:29.327497+0000 front 2024-03-08T21:17:29.328128+0000 (oldest deadline 2024-03-08T21:17:51.627384+0000) 2024-03-08T21:18:06.871 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:18:06 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:18:06.574+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.107:6822 osd.6 since back 2024-03-08T21:17:40.231398+0000 front 2024-03-08T21:17:40.231589+0000 (oldest deadline 2024-03-08T21:18:01.331142+0000) 2024-03-08T21:18:07.710 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:18:07 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:18:07.252+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:40.479965+0000 front 2024-03-08T21:16:40.480241+0000 (oldest deadline 2024-03-08T21:17:06.379830+0000) 2024-03-08T21:18:07.710 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:18:07 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:18:07.252+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:53.283252+0000 front 2024-03-08T21:16:53.283300+0000 (oldest deadline 2024-03-08T21:17:17.383498+0000) 2024-03-08T21:18:07.711 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:18:07 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:18:07.252+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:17:03.088480+0000 front 2024-03-08T21:17:03.088666+0000 (oldest deadline 2024-03-08T21:17:25.388404+0000) 2024-03-08T21:18:07.711 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:18:07 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:18:07.252+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-03-08T21:17:10.690024+0000 front 2024-03-08T21:17:10.690629+0000 (oldest deadline 2024-03-08T21:17:34.189860+0000) 2024-03-08T21:18:07.711 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:18:07 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:18:07.252+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-08T21:17:18.192173+0000 front 2024-03-08T21:17:18.192324+0000 (oldest deadline 2024-03-08T21:17:46.992906+0000) 2024-03-08T21:18:07.711 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:18:07 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:18:07.252+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-03-08T21:17:28.694794+0000 front 2024-03-08T21:17:28.695136+0000 (oldest deadline 2024-03-08T21:17:52.794933+0000) 2024-03-08T21:18:07.711 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:18:07 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:18:07.252+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.107:6822 osd.6 since back 2024-03-08T21:17:38.498151+0000 front 2024-03-08T21:17:38.498257+0000 (oldest deadline 2024-03-08T21:18:04.398068+0000) 2024-03-08T21:18:07.758 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:18:07 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:18:07.441+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:44.858572+0000 front 2024-03-08T21:16:44.858829+0000 (oldest deadline 2024-03-08T21:17:06.558274+0000) 2024-03-08T21:18:07.758 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:18:07 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:18:07.441+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:52.961888+0000 front 2024-03-08T21:16:52.961762+0000 (oldest deadline 2024-03-08T21:17:18.862022+0000) 2024-03-08T21:18:07.758 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:18:07 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:18:07.441+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:17:01.764390+0000 front 2024-03-08T21:17:01.764438+0000 (oldest deadline 2024-03-08T21:17:24.064211+0000) 2024-03-08T21:18:07.758 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:18:07 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:18:07.441+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-03-08T21:17:10.869088+0000 front 2024-03-08T21:17:10.869147+0000 (oldest deadline 2024-03-08T21:17:34.969351+0000) 2024-03-08T21:18:07.758 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:18:07 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:18:07.441+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-08T21:17:17.871105+0000 front 2024-03-08T21:17:17.871226+0000 (oldest deadline 2024-03-08T21:17:43.770901+0000) 2024-03-08T21:18:07.758 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:18:07 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:18:07.441+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-03-08T21:17:30.173392+0000 front 2024-03-08T21:17:30.173231+0000 (oldest deadline 2024-03-08T21:17:51.773639+0000) 2024-03-08T21:18:07.758 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:18:07 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:18:07.441+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.107:6822 osd.6 since back 2024-03-08T21:17:39.976161+0000 front 2024-03-08T21:17:39.976402+0000 (oldest deadline 2024-03-08T21:18:05.876072+0000) 2024-03-08T21:18:07.871 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:18:07 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:18:07.605+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:42.614355+0000 front 2024-03-08T21:16:42.614271+0000 (oldest deadline 2024-03-08T21:17:07.914284+0000) 2024-03-08T21:18:07.871 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:18:07 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:18:07.605+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:52.016503+0000 front 2024-03-08T21:16:52.017017+0000 (oldest deadline 2024-03-08T21:17:16.116550+0000) 2024-03-08T21:18:07.871 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:18:07 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:18:07.605+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:16:57.818698+0000 front 2024-03-08T21:16:57.819201+0000 (oldest deadline 2024-03-08T21:17:26.619635+0000) 2024-03-08T21:18:07.871 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:18:07 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:18:07.605+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-08T21:17:19.424875+0000 front 2024-03-08T21:17:19.424454+0000 (oldest deadline 2024-03-08T21:17:43.425189+0000) 2024-03-08T21:18:07.871 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:18:07 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:18:07.605+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-03-08T21:17:29.327497+0000 front 2024-03-08T21:17:29.328128+0000 (oldest deadline 2024-03-08T21:17:51.627384+0000) 2024-03-08T21:18:07.872 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:18:07 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:18:07.605+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.107:6822 osd.6 since back 2024-03-08T21:17:40.231398+0000 front 2024-03-08T21:17:40.231589+0000 (oldest deadline 2024-03-08T21:18:01.331142+0000) 2024-03-08T21:18:08.710 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:18:08 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:18:08.242+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:40.479965+0000 front 2024-03-08T21:16:40.480241+0000 (oldest deadline 2024-03-08T21:17:06.379830+0000) 2024-03-08T21:18:08.711 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:18:08 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:18:08.242+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:53.283252+0000 front 2024-03-08T21:16:53.283300+0000 (oldest deadline 2024-03-08T21:17:17.383498+0000) 2024-03-08T21:18:08.711 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:18:08 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:18:08.242+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:17:03.088480+0000 front 2024-03-08T21:17:03.088666+0000 (oldest deadline 2024-03-08T21:17:25.388404+0000) 2024-03-08T21:18:08.711 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:18:08 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:18:08.242+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-03-08T21:17:10.690024+0000 front 2024-03-08T21:17:10.690629+0000 (oldest deadline 2024-03-08T21:17:34.189860+0000) 2024-03-08T21:18:08.711 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:18:08 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:18:08.242+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-08T21:17:18.192173+0000 front 2024-03-08T21:17:18.192324+0000 (oldest deadline 2024-03-08T21:17:46.992906+0000) 2024-03-08T21:18:08.711 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:18:08 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:18:08.242+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-03-08T21:17:28.694794+0000 front 2024-03-08T21:17:28.695136+0000 (oldest deadline 2024-03-08T21:17:52.794933+0000) 2024-03-08T21:18:08.711 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:18:08 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9[66871]: 2024-03-08T21:18:08.242+0000 7f4530600700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.107:6822 osd.6 since back 2024-03-08T21:17:38.498151+0000 front 2024-03-08T21:17:38.498257+0000 (oldest deadline 2024-03-08T21:18:04.398068+0000) 2024-03-08T21:18:08.758 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:18:08 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:18:08.450+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:44.858572+0000 front 2024-03-08T21:16:44.858829+0000 (oldest deadline 2024-03-08T21:17:06.558274+0000) 2024-03-08T21:18:08.758 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:18:08 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:18:08.450+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:52.961888+0000 front 2024-03-08T21:16:52.961762+0000 (oldest deadline 2024-03-08T21:17:18.862022+0000) 2024-03-08T21:18:08.758 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:18:08 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:18:08.450+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:17:01.764390+0000 front 2024-03-08T21:17:01.764438+0000 (oldest deadline 2024-03-08T21:17:24.064211+0000) 2024-03-08T21:18:08.758 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:18:08 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:18:08.450+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-03-08T21:17:10.869088+0000 front 2024-03-08T21:17:10.869147+0000 (oldest deadline 2024-03-08T21:17:34.969351+0000) 2024-03-08T21:18:08.758 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:18:08 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:18:08.450+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-08T21:17:17.871105+0000 front 2024-03-08T21:17:17.871226+0000 (oldest deadline 2024-03-08T21:17:43.770901+0000) 2024-03-08T21:18:08.758 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:18:08 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:18:08.450+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-03-08T21:17:30.173392+0000 front 2024-03-08T21:17:30.173231+0000 (oldest deadline 2024-03-08T21:17:51.773639+0000) 2024-03-08T21:18:08.758 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:18:08 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:18:08.450+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.107:6822 osd.6 since back 2024-03-08T21:17:39.976161+0000 front 2024-03-08T21:17:39.976402+0000 (oldest deadline 2024-03-08T21:18:05.876072+0000) 2024-03-08T21:18:08.807 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:18:08 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:18:08.579+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:42.614355+0000 front 2024-03-08T21:16:42.614271+0000 (oldest deadline 2024-03-08T21:17:07.914284+0000) 2024-03-08T21:18:08.807 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:18:08 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:18:08.579+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:52.016503+0000 front 2024-03-08T21:16:52.017017+0000 (oldest deadline 2024-03-08T21:17:16.116550+0000) 2024-03-08T21:18:08.807 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:18:08 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:18:08.579+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:16:57.818698+0000 front 2024-03-08T21:16:57.819201+0000 (oldest deadline 2024-03-08T21:17:26.619635+0000) 2024-03-08T21:18:08.808 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:18:08 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:18:08.579+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-08T21:17:19.424875+0000 front 2024-03-08T21:17:19.424454+0000 (oldest deadline 2024-03-08T21:17:43.425189+0000) 2024-03-08T21:18:08.808 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:18:08 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:18:08.579+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-03-08T21:17:29.327497+0000 front 2024-03-08T21:17:29.328128+0000 (oldest deadline 2024-03-08T21:17:51.627384+0000) 2024-03-08T21:18:08.808 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:18:08 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:18:08.579+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.107:6822 osd.6 since back 2024-03-08T21:17:40.231398+0000 front 2024-03-08T21:17:40.231589+0000 (oldest deadline 2024-03-08T21:18:01.331142+0000) 2024-03-08T21:18:09.120 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:18:08 smithi107 podman[95758]: 2024-03-08 21:18:08.831225217 +0000 UTC m=+5.734408701 container died 78eeaf6f48927bc3960c9b58a346fc8f34f744286805a712782b3bb41ab9e9d2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, RELEASE=rishabh-2024mar8-7d63482) 2024-03-08T21:18:09.460 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:18:09 smithi107 podman[95758]: 2024-03-08 21:18:09.337330161 +0000 UTC m=+6.240513672 container cleanup 78eeaf6f48927bc3960c9b58a346fc8f34f744286805a712782b3bb41ab9e9d2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=rishabh-2024mar8-7d63482, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-08T21:18:09.460 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:18:09 smithi107 bash[95758]: ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9 2024-03-08T21:18:09.758 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:18:09 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:18:09.411+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:44.858572+0000 front 2024-03-08T21:16:44.858829+0000 (oldest deadline 2024-03-08T21:17:06.558274+0000) 2024-03-08T21:18:09.758 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:18:09 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:18:09.411+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:52.961888+0000 front 2024-03-08T21:16:52.961762+0000 (oldest deadline 2024-03-08T21:17:18.862022+0000) 2024-03-08T21:18:09.758 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:18:09 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:18:09.411+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:17:01.764390+0000 front 2024-03-08T21:17:01.764438+0000 (oldest deadline 2024-03-08T21:17:24.064211+0000) 2024-03-08T21:18:09.758 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:18:09 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:18:09.411+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-03-08T21:17:10.869088+0000 front 2024-03-08T21:17:10.869147+0000 (oldest deadline 2024-03-08T21:17:34.969351+0000) 2024-03-08T21:18:09.758 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:18:09 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:18:09.411+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-08T21:17:17.871105+0000 front 2024-03-08T21:17:17.871226+0000 (oldest deadline 2024-03-08T21:17:43.770901+0000) 2024-03-08T21:18:09.759 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:18:09 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:18:09.411+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-03-08T21:17:30.173392+0000 front 2024-03-08T21:17:30.173231+0000 (oldest deadline 2024-03-08T21:17:51.773639+0000) 2024-03-08T21:18:09.759 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:18:09 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:18:09.411+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.107:6822 osd.6 since back 2024-03-08T21:17:39.976161+0000 front 2024-03-08T21:17:39.976402+0000 (oldest deadline 2024-03-08T21:18:05.876072+0000) 2024-03-08T21:18:09.871 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:18:09 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:18:09.613+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:42.614355+0000 front 2024-03-08T21:16:42.614271+0000 (oldest deadline 2024-03-08T21:17:07.914284+0000) 2024-03-08T21:18:09.871 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:18:09 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:18:09.613+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:52.016503+0000 front 2024-03-08T21:16:52.017017+0000 (oldest deadline 2024-03-08T21:17:16.116550+0000) 2024-03-08T21:18:09.871 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:18:09 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:18:09.613+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:16:57.818698+0000 front 2024-03-08T21:16:57.819201+0000 (oldest deadline 2024-03-08T21:17:26.619635+0000) 2024-03-08T21:18:09.871 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:18:09 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:18:09.613+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-08T21:17:19.424875+0000 front 2024-03-08T21:17:19.424454+0000 (oldest deadline 2024-03-08T21:17:43.425189+0000) 2024-03-08T21:18:09.871 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:18:09 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:18:09.613+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-03-08T21:17:29.327497+0000 front 2024-03-08T21:17:29.328128+0000 (oldest deadline 2024-03-08T21:17:51.627384+0000) 2024-03-08T21:18:09.871 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:18:09 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:18:09.613+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.107:6822 osd.6 since back 2024-03-08T21:17:40.231398+0000 front 2024-03-08T21:17:40.231589+0000 (oldest deadline 2024-03-08T21:18:01.331142+0000) 2024-03-08T21:18:09.960 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:18:09 smithi107 podman[95778]: 2024-03-08 21:18:09.66374282 +0000 UTC m=+0.821537441 container remove 78eeaf6f48927bc3960c9b58a346fc8f34f744286805a712782b3bb41ab9e9d2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=rishabh-2024mar8-7d63482, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-08T21:18:10.621 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:18:10 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:18:10.582+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:42.614355+0000 front 2024-03-08T21:16:42.614271+0000 (oldest deadline 2024-03-08T21:17:07.914284+0000) 2024-03-08T21:18:10.621 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:18:10 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:18:10.582+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:52.016503+0000 front 2024-03-08T21:16:52.017017+0000 (oldest deadline 2024-03-08T21:17:16.116550+0000) 2024-03-08T21:18:10.621 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:18:10 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:18:10.582+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:16:57.818698+0000 front 2024-03-08T21:16:57.819201+0000 (oldest deadline 2024-03-08T21:17:26.619635+0000) 2024-03-08T21:18:10.621 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:18:10 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:18:10.582+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-08T21:17:19.424875+0000 front 2024-03-08T21:17:19.424454+0000 (oldest deadline 2024-03-08T21:17:43.425189+0000) 2024-03-08T21:18:10.621 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:18:10 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:18:10.582+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-03-08T21:17:29.327497+0000 front 2024-03-08T21:17:29.328128+0000 (oldest deadline 2024-03-08T21:17:51.627384+0000) 2024-03-08T21:18:10.622 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:18:10 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:18:10.582+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.107:6822 osd.6 since back 2024-03-08T21:17:40.231398+0000 front 2024-03-08T21:17:40.231589+0000 (oldest deadline 2024-03-08T21:18:01.331142+0000) 2024-03-08T21:18:10.758 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:18:10 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:18:10.367+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:44.858572+0000 front 2024-03-08T21:16:44.858829+0000 (oldest deadline 2024-03-08T21:17:06.558274+0000) 2024-03-08T21:18:10.758 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:18:10 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:18:10.367+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:52.961888+0000 front 2024-03-08T21:16:52.961762+0000 (oldest deadline 2024-03-08T21:17:18.862022+0000) 2024-03-08T21:18:10.758 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:18:10 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:18:10.367+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:17:01.764390+0000 front 2024-03-08T21:17:01.764438+0000 (oldest deadline 2024-03-08T21:17:24.064211+0000) 2024-03-08T21:18:10.758 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:18:10 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:18:10.367+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-03-08T21:17:10.869088+0000 front 2024-03-08T21:17:10.869147+0000 (oldest deadline 2024-03-08T21:17:34.969351+0000) 2024-03-08T21:18:10.758 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:18:10 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:18:10.367+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-08T21:17:17.871105+0000 front 2024-03-08T21:17:17.871226+0000 (oldest deadline 2024-03-08T21:17:43.770901+0000) 2024-03-08T21:18:10.758 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:18:10 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:18:10.367+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-03-08T21:17:30.173392+0000 front 2024-03-08T21:17:30.173231+0000 (oldest deadline 2024-03-08T21:17:51.773639+0000) 2024-03-08T21:18:10.758 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:18:10 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:18:10.367+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.107:6822 osd.6 since back 2024-03-08T21:17:39.976161+0000 front 2024-03-08T21:17:39.976402+0000 (oldest deadline 2024-03-08T21:18:05.876072+0000) 2024-03-08T21:18:10.835 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:18:10 smithi107 podman[95866]: 2024-03-08 21:18:10.478311478 +0000 UTC m=+0.031907389 image pull a272146cb810b1fd4a53627f019472132e30966d5c25b01bfc44a001295f1249 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd 2024-03-08T21:18:10.835 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:18:10 smithi107 podman[95866]: 2024-03-08 21:18:10.597902855 +0000 UTC m=+0.151498807 container create 9150efdd5fb69575d1d2bdbaccc26a86214d2d01d5762caaa42d1ef1d7bdc6fe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9-deactivate, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=rishabh-2024mar8-7d63482, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True) 2024-03-08T21:18:10.836 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:18:10 smithi107 podman[95866]: 2024-03-08 21:18:10.859399717 +0000 UTC m=+0.412995675 container init 9150efdd5fb69575d1d2bdbaccc26a86214d2d01d5762caaa42d1ef1d7bdc6fe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9-deactivate, RELEASE=rishabh-2024mar8-7d63482, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD) 2024-03-08T21:18:11.210 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:18:10 smithi107 podman[95866]: 2024-03-08 21:18:10.870675685 +0000 UTC m=+0.424271616 container start 9150efdd5fb69575d1d2bdbaccc26a86214d2d01d5762caaa42d1ef1d7bdc6fe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9-deactivate, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, RELEASE=rishabh-2024mar8-7d63482, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-08T21:18:11.210 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:18:10 smithi107 podman[95866]: 2024-03-08 21:18:10.966801963 +0000 UTC m=+0.520398004 container attach 9150efdd5fb69575d1d2bdbaccc26a86214d2d01d5762caaa42d1ef1d7bdc6fe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9-deactivate, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=rishabh-2024mar8-7d63482, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-03-08T21:18:11.667 INFO:journalctl@ceph.osd.9.smithi107.stdout:Mar 08 21:18:11 smithi107 podman[95866]: 2024-03-08 21:18:11.297206894 +0000 UTC m=+0.850802848 container died 9150efdd5fb69575d1d2bdbaccc26a86214d2d01d5762caaa42d1ef1d7bdc6fe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-9-deactivate, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=rishabh-2024mar8-7d63482, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-03-08T21:18:11.758 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:18:11 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:18:11.328+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:44.858572+0000 front 2024-03-08T21:16:44.858829+0000 (oldest deadline 2024-03-08T21:17:06.558274+0000) 2024-03-08T21:18:11.758 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:18:11 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:18:11.328+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:52.961888+0000 front 2024-03-08T21:16:52.961762+0000 (oldest deadline 2024-03-08T21:17:18.862022+0000) 2024-03-08T21:18:11.758 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:18:11 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:18:11.328+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:17:01.764390+0000 front 2024-03-08T21:17:01.764438+0000 (oldest deadline 2024-03-08T21:17:24.064211+0000) 2024-03-08T21:18:11.758 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:18:11 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:18:11.328+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-03-08T21:17:10.869088+0000 front 2024-03-08T21:17:10.869147+0000 (oldest deadline 2024-03-08T21:17:34.969351+0000) 2024-03-08T21:18:11.758 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:18:11 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:18:11.328+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-08T21:17:17.871105+0000 front 2024-03-08T21:17:17.871226+0000 (oldest deadline 2024-03-08T21:17:43.770901+0000) 2024-03-08T21:18:11.758 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:18:11 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:18:11.328+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-03-08T21:17:30.173392+0000 front 2024-03-08T21:17:30.173231+0000 (oldest deadline 2024-03-08T21:17:51.773639+0000) 2024-03-08T21:18:11.758 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:18:11 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:18:11.328+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.107:6822 osd.6 since back 2024-03-08T21:17:39.976161+0000 front 2024-03-08T21:17:39.976402+0000 (oldest deadline 2024-03-08T21:18:05.876072+0000) 2024-03-08T21:18:11.871 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:18:11 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:18:11.612+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:42.614355+0000 front 2024-03-08T21:16:42.614271+0000 (oldest deadline 2024-03-08T21:17:07.914284+0000) 2024-03-08T21:18:11.871 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:18:11 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:18:11.612+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:52.016503+0000 front 2024-03-08T21:16:52.017017+0000 (oldest deadline 2024-03-08T21:17:16.116550+0000) 2024-03-08T21:18:11.871 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:18:11 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:18:11.612+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:16:57.818698+0000 front 2024-03-08T21:16:57.819201+0000 (oldest deadline 2024-03-08T21:17:26.619635+0000) 2024-03-08T21:18:11.871 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:18:11 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:18:11.612+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-08T21:17:19.424875+0000 front 2024-03-08T21:17:19.424454+0000 (oldest deadline 2024-03-08T21:17:43.425189+0000) 2024-03-08T21:18:11.871 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:18:11 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:18:11.612+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-03-08T21:17:29.327497+0000 front 2024-03-08T21:17:29.328128+0000 (oldest deadline 2024-03-08T21:17:51.627384+0000) 2024-03-08T21:18:11.871 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:18:11 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:18:11.612+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.107:6822 osd.6 since back 2024-03-08T21:17:40.231398+0000 front 2024-03-08T21:17:40.231589+0000 (oldest deadline 2024-03-08T21:18:01.331142+0000) 2024-03-08T21:18:11.871 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:18:11 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:18:11.612+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.149:6820 osd.7 since back 2024-03-08T21:17:45.933742+0000 front 2024-03-08T21:17:45.933685+0000 (oldest deadline 2024-03-08T21:18:10.633316+0000) 2024-03-08T21:18:12.003 DEBUG:teuthology.orchestra.run.smithi107:> sudo pkill -f 'journalctl -f -n 0 -u ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715@osd.9.service' 2024-03-08T21:18:12.128 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-08T21:18:12.128 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-03-08T21:18:12.128 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-03-08T21:18:12.129 DEBUG:teuthology.orchestra.run.smithi149:> sudo systemctl stop ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715@osd.10 2024-03-08T21:18:12.508 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:18:12 smithi149 systemd[1]: Stopping Ceph osd.10 for dc20f0f8-dd8c-11ee-95c6-87774f69a715... 2024-03-08T21:18:12.508 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:18:12 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:18:12.372+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:44.858572+0000 front 2024-03-08T21:16:44.858829+0000 (oldest deadline 2024-03-08T21:17:06.558274+0000) 2024-03-08T21:18:12.508 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:18:12 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:18:12.372+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:52.961888+0000 front 2024-03-08T21:16:52.961762+0000 (oldest deadline 2024-03-08T21:17:18.862022+0000) 2024-03-08T21:18:12.508 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:18:12 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:18:12.372+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:17:01.764390+0000 front 2024-03-08T21:17:01.764438+0000 (oldest deadline 2024-03-08T21:17:24.064211+0000) 2024-03-08T21:18:12.508 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:18:12 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:18:12.372+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-03-08T21:17:10.869088+0000 front 2024-03-08T21:17:10.869147+0000 (oldest deadline 2024-03-08T21:17:34.969351+0000) 2024-03-08T21:18:12.508 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:18:12 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:18:12.372+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-08T21:17:17.871105+0000 front 2024-03-08T21:17:17.871226+0000 (oldest deadline 2024-03-08T21:17:43.770901+0000) 2024-03-08T21:18:12.508 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:18:12 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:18:12.372+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-03-08T21:17:30.173392+0000 front 2024-03-08T21:17:30.173231+0000 (oldest deadline 2024-03-08T21:17:51.773639+0000) 2024-03-08T21:18:12.509 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:18:12 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:18:12.372+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.107:6822 osd.6 since back 2024-03-08T21:17:39.976161+0000 front 2024-03-08T21:17:39.976402+0000 (oldest deadline 2024-03-08T21:18:05.876072+0000) 2024-03-08T21:18:12.871 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:18:12 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:18:12.615+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:42.614355+0000 front 2024-03-08T21:16:42.614271+0000 (oldest deadline 2024-03-08T21:17:07.914284+0000) 2024-03-08T21:18:12.871 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:18:12 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:18:12.615+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:52.016503+0000 front 2024-03-08T21:16:52.017017+0000 (oldest deadline 2024-03-08T21:17:16.116550+0000) 2024-03-08T21:18:12.871 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:18:12 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:18:12.615+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:16:57.818698+0000 front 2024-03-08T21:16:57.819201+0000 (oldest deadline 2024-03-08T21:17:26.619635+0000) 2024-03-08T21:18:12.871 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:18:12 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:18:12.615+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-08T21:17:19.424875+0000 front 2024-03-08T21:17:19.424454+0000 (oldest deadline 2024-03-08T21:17:43.425189+0000) 2024-03-08T21:18:12.871 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:18:12 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:18:12.615+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-03-08T21:17:29.327497+0000 front 2024-03-08T21:17:29.328128+0000 (oldest deadline 2024-03-08T21:17:51.627384+0000) 2024-03-08T21:18:12.871 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:18:12 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:18:12.615+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.107:6822 osd.6 since back 2024-03-08T21:17:40.231398+0000 front 2024-03-08T21:17:40.231589+0000 (oldest deadline 2024-03-08T21:18:01.331142+0000) 2024-03-08T21:18:12.872 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:18:12 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:18:12.615+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.149:6820 osd.7 since back 2024-03-08T21:17:45.933742+0000 front 2024-03-08T21:17:45.933685+0000 (oldest deadline 2024-03-08T21:18:10.633316+0000) 2024-03-08T21:18:13.007 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:18:12 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:18:12.686+0000 7f77a4600700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-08T21:18:13.008 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:18:12 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:18:12.686+0000 7f77a4600700 -1 osd.10 115 *** Got signal Terminated *** 2024-03-08T21:18:13.008 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:18:12 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:18:12.686+0000 7f77a4600700 -1 osd.10 115 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-08T21:18:13.758 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:18:13 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:18:13.399+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:44.858572+0000 front 2024-03-08T21:16:44.858829+0000 (oldest deadline 2024-03-08T21:17:06.558274+0000) 2024-03-08T21:18:13.758 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:18:13 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:18:13.399+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:52.961888+0000 front 2024-03-08T21:16:52.961762+0000 (oldest deadline 2024-03-08T21:17:18.862022+0000) 2024-03-08T21:18:13.758 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:18:13 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:18:13.399+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:17:01.764390+0000 front 2024-03-08T21:17:01.764438+0000 (oldest deadline 2024-03-08T21:17:24.064211+0000) 2024-03-08T21:18:13.758 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:18:13 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:18:13.399+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-03-08T21:17:10.869088+0000 front 2024-03-08T21:17:10.869147+0000 (oldest deadline 2024-03-08T21:17:34.969351+0000) 2024-03-08T21:18:13.758 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:18:13 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:18:13.399+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-08T21:17:17.871105+0000 front 2024-03-08T21:17:17.871226+0000 (oldest deadline 2024-03-08T21:17:43.770901+0000) 2024-03-08T21:18:13.758 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:18:13 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:18:13.399+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-03-08T21:17:30.173392+0000 front 2024-03-08T21:17:30.173231+0000 (oldest deadline 2024-03-08T21:17:51.773639+0000) 2024-03-08T21:18:13.758 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:18:13 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:18:13.399+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.107:6822 osd.6 since back 2024-03-08T21:17:39.976161+0000 front 2024-03-08T21:17:39.976402+0000 (oldest deadline 2024-03-08T21:18:05.876072+0000) 2024-03-08T21:18:13.871 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:18:13 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:18:13.587+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:42.614355+0000 front 2024-03-08T21:16:42.614271+0000 (oldest deadline 2024-03-08T21:17:07.914284+0000) 2024-03-08T21:18:13.871 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:18:13 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:18:13.587+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:52.016503+0000 front 2024-03-08T21:16:52.017017+0000 (oldest deadline 2024-03-08T21:17:16.116550+0000) 2024-03-08T21:18:13.871 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:18:13 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:18:13.587+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:16:57.818698+0000 front 2024-03-08T21:16:57.819201+0000 (oldest deadline 2024-03-08T21:17:26.619635+0000) 2024-03-08T21:18:13.871 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:18:13 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:18:13.587+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-08T21:17:19.424875+0000 front 2024-03-08T21:17:19.424454+0000 (oldest deadline 2024-03-08T21:17:43.425189+0000) 2024-03-08T21:18:13.871 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:18:13 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:18:13.587+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-03-08T21:17:29.327497+0000 front 2024-03-08T21:17:29.328128+0000 (oldest deadline 2024-03-08T21:17:51.627384+0000) 2024-03-08T21:18:13.872 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:18:13 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:18:13.587+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.107:6822 osd.6 since back 2024-03-08T21:17:40.231398+0000 front 2024-03-08T21:17:40.231589+0000 (oldest deadline 2024-03-08T21:18:01.331142+0000) 2024-03-08T21:18:13.872 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:18:13 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:18:13.587+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.149:6820 osd.7 since back 2024-03-08T21:17:45.933742+0000 front 2024-03-08T21:17:45.933685+0000 (oldest deadline 2024-03-08T21:18:10.633316+0000) 2024-03-08T21:18:14.758 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:18:14 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:18:14.392+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:44.858572+0000 front 2024-03-08T21:16:44.858829+0000 (oldest deadline 2024-03-08T21:17:06.558274+0000) 2024-03-08T21:18:14.758 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:18:14 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:18:14.392+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:52.961888+0000 front 2024-03-08T21:16:52.961762+0000 (oldest deadline 2024-03-08T21:17:18.862022+0000) 2024-03-08T21:18:14.758 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:18:14 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:18:14.392+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:17:01.764390+0000 front 2024-03-08T21:17:01.764438+0000 (oldest deadline 2024-03-08T21:17:24.064211+0000) 2024-03-08T21:18:14.758 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:18:14 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:18:14.392+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-03-08T21:17:10.869088+0000 front 2024-03-08T21:17:10.869147+0000 (oldest deadline 2024-03-08T21:17:34.969351+0000) 2024-03-08T21:18:14.758 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:18:14 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:18:14.392+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-08T21:17:17.871105+0000 front 2024-03-08T21:17:17.871226+0000 (oldest deadline 2024-03-08T21:17:43.770901+0000) 2024-03-08T21:18:14.758 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:18:14 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:18:14.392+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-03-08T21:17:30.173392+0000 front 2024-03-08T21:17:30.173231+0000 (oldest deadline 2024-03-08T21:17:51.773639+0000) 2024-03-08T21:18:14.758 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:18:14 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:18:14.392+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.107:6822 osd.6 since back 2024-03-08T21:17:39.976161+0000 front 2024-03-08T21:17:39.976402+0000 (oldest deadline 2024-03-08T21:18:05.876072+0000) 2024-03-08T21:18:14.871 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:18:14 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:18:14.602+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:42.614355+0000 front 2024-03-08T21:16:42.614271+0000 (oldest deadline 2024-03-08T21:17:07.914284+0000) 2024-03-08T21:18:14.871 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:18:14 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:18:14.602+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:52.016503+0000 front 2024-03-08T21:16:52.017017+0000 (oldest deadline 2024-03-08T21:17:16.116550+0000) 2024-03-08T21:18:14.871 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:18:14 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:18:14.602+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:16:57.818698+0000 front 2024-03-08T21:16:57.819201+0000 (oldest deadline 2024-03-08T21:17:26.619635+0000) 2024-03-08T21:18:14.871 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:18:14 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:18:14.602+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-08T21:17:19.424875+0000 front 2024-03-08T21:17:19.424454+0000 (oldest deadline 2024-03-08T21:17:43.425189+0000) 2024-03-08T21:18:14.871 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:18:14 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:18:14.602+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-03-08T21:17:29.327497+0000 front 2024-03-08T21:17:29.328128+0000 (oldest deadline 2024-03-08T21:17:51.627384+0000) 2024-03-08T21:18:14.871 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:18:14 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:18:14.602+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.107:6822 osd.6 since back 2024-03-08T21:17:40.231398+0000 front 2024-03-08T21:17:40.231589+0000 (oldest deadline 2024-03-08T21:18:01.331142+0000) 2024-03-08T21:18:14.872 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:18:14 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:18:14.603+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.149:6820 osd.7 since back 2024-03-08T21:17:45.933742+0000 front 2024-03-08T21:17:45.933685+0000 (oldest deadline 2024-03-08T21:18:10.633316+0000) 2024-03-08T21:18:15.758 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:18:15 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:18:15.352+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:44.858572+0000 front 2024-03-08T21:16:44.858829+0000 (oldest deadline 2024-03-08T21:17:06.558274+0000) 2024-03-08T21:18:15.758 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:18:15 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:18:15.352+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:52.961888+0000 front 2024-03-08T21:16:52.961762+0000 (oldest deadline 2024-03-08T21:17:18.862022+0000) 2024-03-08T21:18:15.758 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:18:15 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:18:15.352+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:17:01.764390+0000 front 2024-03-08T21:17:01.764438+0000 (oldest deadline 2024-03-08T21:17:24.064211+0000) 2024-03-08T21:18:15.758 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:18:15 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:18:15.352+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-03-08T21:17:10.869088+0000 front 2024-03-08T21:17:10.869147+0000 (oldest deadline 2024-03-08T21:17:34.969351+0000) 2024-03-08T21:18:15.758 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:18:15 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:18:15.352+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-08T21:17:17.871105+0000 front 2024-03-08T21:17:17.871226+0000 (oldest deadline 2024-03-08T21:17:43.770901+0000) 2024-03-08T21:18:15.758 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:18:15 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:18:15.352+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-03-08T21:17:30.173392+0000 front 2024-03-08T21:17:30.173231+0000 (oldest deadline 2024-03-08T21:17:51.773639+0000) 2024-03-08T21:18:15.758 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:18:15 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:18:15.352+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.107:6822 osd.6 since back 2024-03-08T21:17:39.976161+0000 front 2024-03-08T21:17:39.976402+0000 (oldest deadline 2024-03-08T21:18:05.876072+0000) 2024-03-08T21:18:15.871 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:18:15 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:18:15.616+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:42.614355+0000 front 2024-03-08T21:16:42.614271+0000 (oldest deadline 2024-03-08T21:17:07.914284+0000) 2024-03-08T21:18:15.871 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:18:15 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:18:15.616+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:52.016503+0000 front 2024-03-08T21:16:52.017017+0000 (oldest deadline 2024-03-08T21:17:16.116550+0000) 2024-03-08T21:18:15.871 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:18:15 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:18:15.616+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:16:57.818698+0000 front 2024-03-08T21:16:57.819201+0000 (oldest deadline 2024-03-08T21:17:26.619635+0000) 2024-03-08T21:18:15.871 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:18:15 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:18:15.616+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-08T21:17:19.424875+0000 front 2024-03-08T21:17:19.424454+0000 (oldest deadline 2024-03-08T21:17:43.425189+0000) 2024-03-08T21:18:15.871 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:18:15 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:18:15.616+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-03-08T21:17:29.327497+0000 front 2024-03-08T21:17:29.328128+0000 (oldest deadline 2024-03-08T21:17:51.627384+0000) 2024-03-08T21:18:15.872 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:18:15 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:18:15.616+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.107:6822 osd.6 since back 2024-03-08T21:17:40.231398+0000 front 2024-03-08T21:17:40.231589+0000 (oldest deadline 2024-03-08T21:18:01.331142+0000) 2024-03-08T21:18:15.872 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:18:15 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:18:15.616+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.149:6820 osd.7 since back 2024-03-08T21:17:45.933742+0000 front 2024-03-08T21:17:45.933685+0000 (oldest deadline 2024-03-08T21:18:10.633316+0000) 2024-03-08T21:18:16.758 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:18:16 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:18:16.333+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:44.858572+0000 front 2024-03-08T21:16:44.858829+0000 (oldest deadline 2024-03-08T21:17:06.558274+0000) 2024-03-08T21:18:16.758 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:18:16 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:18:16.333+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:52.961888+0000 front 2024-03-08T21:16:52.961762+0000 (oldest deadline 2024-03-08T21:17:18.862022+0000) 2024-03-08T21:18:16.758 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:18:16 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:18:16.333+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:17:01.764390+0000 front 2024-03-08T21:17:01.764438+0000 (oldest deadline 2024-03-08T21:17:24.064211+0000) 2024-03-08T21:18:16.758 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:18:16 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:18:16.333+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-03-08T21:17:10.869088+0000 front 2024-03-08T21:17:10.869147+0000 (oldest deadline 2024-03-08T21:17:34.969351+0000) 2024-03-08T21:18:16.758 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:18:16 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:18:16.333+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-08T21:17:17.871105+0000 front 2024-03-08T21:17:17.871226+0000 (oldest deadline 2024-03-08T21:17:43.770901+0000) 2024-03-08T21:18:16.758 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:18:16 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:18:16.333+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-03-08T21:17:30.173392+0000 front 2024-03-08T21:17:30.173231+0000 (oldest deadline 2024-03-08T21:17:51.773639+0000) 2024-03-08T21:18:16.758 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:18:16 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:18:16.333+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.107:6822 osd.6 since back 2024-03-08T21:17:39.976161+0000 front 2024-03-08T21:17:39.976402+0000 (oldest deadline 2024-03-08T21:18:05.876072+0000) 2024-03-08T21:18:16.871 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:18:16 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:18:16.596+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:42.614355+0000 front 2024-03-08T21:16:42.614271+0000 (oldest deadline 2024-03-08T21:17:07.914284+0000) 2024-03-08T21:18:16.871 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:18:16 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:18:16.596+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:52.016503+0000 front 2024-03-08T21:16:52.017017+0000 (oldest deadline 2024-03-08T21:17:16.116550+0000) 2024-03-08T21:18:16.871 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:18:16 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:18:16.596+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:16:57.818698+0000 front 2024-03-08T21:16:57.819201+0000 (oldest deadline 2024-03-08T21:17:26.619635+0000) 2024-03-08T21:18:16.871 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:18:16 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:18:16.596+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-08T21:17:19.424875+0000 front 2024-03-08T21:17:19.424454+0000 (oldest deadline 2024-03-08T21:17:43.425189+0000) 2024-03-08T21:18:16.871 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:18:16 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:18:16.596+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-03-08T21:17:29.327497+0000 front 2024-03-08T21:17:29.328128+0000 (oldest deadline 2024-03-08T21:17:51.627384+0000) 2024-03-08T21:18:16.871 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:18:16 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:18:16.596+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.107:6822 osd.6 since back 2024-03-08T21:17:40.231398+0000 front 2024-03-08T21:17:40.231589+0000 (oldest deadline 2024-03-08T21:18:01.331142+0000) 2024-03-08T21:18:16.871 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:18:16 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:18:16.596+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.149:6820 osd.7 since back 2024-03-08T21:17:45.933742+0000 front 2024-03-08T21:17:45.933685+0000 (oldest deadline 2024-03-08T21:18:10.633316+0000) 2024-03-08T21:18:17.758 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:18:17 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:18:17.371+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:44.858572+0000 front 2024-03-08T21:16:44.858829+0000 (oldest deadline 2024-03-08T21:17:06.558274+0000) 2024-03-08T21:18:17.758 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:18:17 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:18:17.371+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:52.961888+0000 front 2024-03-08T21:16:52.961762+0000 (oldest deadline 2024-03-08T21:17:18.862022+0000) 2024-03-08T21:18:17.758 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:18:17 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:18:17.371+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:17:01.764390+0000 front 2024-03-08T21:17:01.764438+0000 (oldest deadline 2024-03-08T21:17:24.064211+0000) 2024-03-08T21:18:17.758 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:18:17 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:18:17.371+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-03-08T21:17:10.869088+0000 front 2024-03-08T21:17:10.869147+0000 (oldest deadline 2024-03-08T21:17:34.969351+0000) 2024-03-08T21:18:17.758 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:18:17 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:18:17.371+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-08T21:17:17.871105+0000 front 2024-03-08T21:17:17.871226+0000 (oldest deadline 2024-03-08T21:17:43.770901+0000) 2024-03-08T21:18:17.758 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:18:17 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:18:17.371+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-03-08T21:17:30.173392+0000 front 2024-03-08T21:17:30.173231+0000 (oldest deadline 2024-03-08T21:17:51.773639+0000) 2024-03-08T21:18:17.758 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:18:17 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10[60237]: 2024-03-08T21:18:17.371+0000 7f779ae00700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.107:6822 osd.6 since back 2024-03-08T21:17:39.976161+0000 front 2024-03-08T21:17:39.976402+0000 (oldest deadline 2024-03-08T21:18:05.876072+0000) 2024-03-08T21:18:17.871 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:18:17 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:18:17.564+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:42.614355+0000 front 2024-03-08T21:16:42.614271+0000 (oldest deadline 2024-03-08T21:17:07.914284+0000) 2024-03-08T21:18:17.871 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:18:17 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:18:17.564+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:52.016503+0000 front 2024-03-08T21:16:52.017017+0000 (oldest deadline 2024-03-08T21:17:16.116550+0000) 2024-03-08T21:18:17.871 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:18:17 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:18:17.564+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:16:57.818698+0000 front 2024-03-08T21:16:57.819201+0000 (oldest deadline 2024-03-08T21:17:26.619635+0000) 2024-03-08T21:18:17.871 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:18:17 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:18:17.564+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-08T21:17:19.424875+0000 front 2024-03-08T21:17:19.424454+0000 (oldest deadline 2024-03-08T21:17:43.425189+0000) 2024-03-08T21:18:17.871 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:18:17 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:18:17.564+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-03-08T21:17:29.327497+0000 front 2024-03-08T21:17:29.328128+0000 (oldest deadline 2024-03-08T21:17:51.627384+0000) 2024-03-08T21:18:17.872 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:18:17 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:18:17.564+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.107:6822 osd.6 since back 2024-03-08T21:17:40.231398+0000 front 2024-03-08T21:17:40.231589+0000 (oldest deadline 2024-03-08T21:18:01.331142+0000) 2024-03-08T21:18:17.872 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:18:17 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:18:17.564+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.149:6820 osd.7 since back 2024-03-08T21:17:45.933742+0000 front 2024-03-08T21:17:45.933685+0000 (oldest deadline 2024-03-08T21:18:10.633316+0000) 2024-03-08T21:18:18.444 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:18:18 smithi149 podman[72992]: 2024-03-08 21:18:18.143749257 +0000 UTC m=+5.579194269 container died d91396563d89635c8863427e7fb6b7a59f9080369b01da05e2b516ff87eebd58 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10, io.buildah.version=1.33.5, RELEASE=rishabh-2024mar8-7d63482, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-03-08T21:18:18.757 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:18:18 smithi149 podman[72992]: 2024-03-08 21:18:18.581201621 +0000 UTC m=+6.016646673 container cleanup d91396563d89635c8863427e7fb6b7a59f9080369b01da05e2b516ff87eebd58 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10, maintainer=Guillaume Abrioux , RELEASE=rishabh-2024mar8-7d63482, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-03-08T21:18:18.758 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:18:18 smithi149 bash[72992]: ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10 2024-03-08T21:18:18.871 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:18:18 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:18:18.552+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:42.614355+0000 front 2024-03-08T21:16:42.614271+0000 (oldest deadline 2024-03-08T21:17:07.914284+0000) 2024-03-08T21:18:18.871 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:18:18 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:18:18.552+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:52.016503+0000 front 2024-03-08T21:16:52.017017+0000 (oldest deadline 2024-03-08T21:17:16.116550+0000) 2024-03-08T21:18:18.871 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:18:18 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:18:18.552+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:16:57.818698+0000 front 2024-03-08T21:16:57.819201+0000 (oldest deadline 2024-03-08T21:17:26.619635+0000) 2024-03-08T21:18:18.871 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:18:18 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:18:18.552+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-08T21:17:19.424875+0000 front 2024-03-08T21:17:19.424454+0000 (oldest deadline 2024-03-08T21:17:43.425189+0000) 2024-03-08T21:18:18.871 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:18:18 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:18:18.552+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-03-08T21:17:29.327497+0000 front 2024-03-08T21:17:29.328128+0000 (oldest deadline 2024-03-08T21:17:51.627384+0000) 2024-03-08T21:18:18.872 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:18:18 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:18:18.552+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.107:6822 osd.6 since back 2024-03-08T21:17:40.231398+0000 front 2024-03-08T21:17:40.231589+0000 (oldest deadline 2024-03-08T21:18:01.331142+0000) 2024-03-08T21:18:18.872 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:18:18 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:18:18.552+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.149:6820 osd.7 since back 2024-03-08T21:17:45.933742+0000 front 2024-03-08T21:17:45.933685+0000 (oldest deadline 2024-03-08T21:18:10.633316+0000) 2024-03-08T21:18:19.257 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:18:18 smithi149 podman[73011]: 2024-03-08 21:18:18.866101958 +0000 UTC m=+0.707754639 container remove d91396563d89635c8863427e7fb6b7a59f9080369b01da05e2b516ff87eebd58 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=rishabh-2024mar8-7d63482, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-03-08T21:18:19.842 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:18:19 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:18:19.529+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:42.614355+0000 front 2024-03-08T21:16:42.614271+0000 (oldest deadline 2024-03-08T21:17:07.914284+0000) 2024-03-08T21:18:19.842 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:18:19 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:18:19.529+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:52.016503+0000 front 2024-03-08T21:16:52.017017+0000 (oldest deadline 2024-03-08T21:17:16.116550+0000) 2024-03-08T21:18:19.842 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:18:19 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:18:19.529+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:16:57.818698+0000 front 2024-03-08T21:16:57.819201+0000 (oldest deadline 2024-03-08T21:17:26.619635+0000) 2024-03-08T21:18:19.842 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:18:19 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:18:19.529+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-08T21:17:19.424875+0000 front 2024-03-08T21:17:19.424454+0000 (oldest deadline 2024-03-08T21:17:43.425189+0000) 2024-03-08T21:18:19.842 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:18:19 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:18:19.529+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-03-08T21:17:29.327497+0000 front 2024-03-08T21:17:29.328128+0000 (oldest deadline 2024-03-08T21:17:51.627384+0000) 2024-03-08T21:18:19.842 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:18:19 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:18:19.529+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.107:6822 osd.6 since back 2024-03-08T21:17:40.231398+0000 front 2024-03-08T21:17:40.231589+0000 (oldest deadline 2024-03-08T21:18:01.331142+0000) 2024-03-08T21:18:19.843 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:18:19 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:18:19.529+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.149:6820 osd.7 since back 2024-03-08T21:17:45.933742+0000 front 2024-03-08T21:17:45.933685+0000 (oldest deadline 2024-03-08T21:18:10.633316+0000) 2024-03-08T21:18:20.007 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:18:19 smithi149 podman[73100]: 2024-03-08 21:18:19.596281642 +0000 UTC m=+0.033949490 image pull a272146cb810b1fd4a53627f019472132e30966d5c25b01bfc44a001295f1249 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd 2024-03-08T21:18:20.008 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:18:19 smithi149 podman[73100]: 2024-03-08 21:18:19.82153522 +0000 UTC m=+0.259203119 container create 3967c0ace5e5e642d5ad82d6d532b27cef0fbb6aa896090796971fd6febbc7bb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10-deactivate, org.label-schema.build-date=20240307, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=rishabh-2024mar8-7d63482, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-03-08T21:18:20.387 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:18:20 smithi149 podman[73100]: 2024-03-08 21:18:20.154320586 +0000 UTC m=+0.591988475 container init 3967c0ace5e5e642d5ad82d6d532b27cef0fbb6aa896090796971fd6febbc7bb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10-deactivate, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=rishabh-2024mar8-7d63482, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=) 2024-03-08T21:18:20.387 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:18:20 smithi149 podman[73100]: 2024-03-08 21:18:20.165414477 +0000 UTC m=+0.603082349 container start 3967c0ace5e5e642d5ad82d6d532b27cef0fbb6aa896090796971fd6febbc7bb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, RELEASE=rishabh-2024mar8-7d63482, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-03-08T21:18:20.387 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:18:20 smithi149 podman[73100]: 2024-03-08 21:18:20.294510593 +0000 UTC m=+0.732178482 container attach 3967c0ace5e5e642d5ad82d6d532b27cef0fbb6aa896090796971fd6febbc7bb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10-deactivate, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=rishabh-2024mar8-7d63482) 2024-03-08T21:18:20.674 INFO:journalctl@ceph.osd.10.smithi149.stdout:Mar 08 21:18:20 smithi149 podman[73100]: 2024-03-08 21:18:20.606975634 +0000 UTC m=+1.044643505 container died 3967c0ace5e5e642d5ad82d6d532b27cef0fbb6aa896090796971fd6febbc7bb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-10-deactivate, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, RELEASE=rishabh-2024mar8-7d63482, org.label-schema.license=GPLv2) 2024-03-08T21:18:20.871 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:18:20 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:18:20.504+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:42.614355+0000 front 2024-03-08T21:16:42.614271+0000 (oldest deadline 2024-03-08T21:17:07.914284+0000) 2024-03-08T21:18:20.871 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:18:20 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:18:20.504+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:52.016503+0000 front 2024-03-08T21:16:52.017017+0000 (oldest deadline 2024-03-08T21:17:16.116550+0000) 2024-03-08T21:18:20.872 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:18:20 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:18:20.504+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:16:57.818698+0000 front 2024-03-08T21:16:57.819201+0000 (oldest deadline 2024-03-08T21:17:26.619635+0000) 2024-03-08T21:18:20.872 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:18:20 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:18:20.504+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-08T21:17:19.424875+0000 front 2024-03-08T21:17:19.424454+0000 (oldest deadline 2024-03-08T21:17:43.425189+0000) 2024-03-08T21:18:20.872 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:18:20 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:18:20.504+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-03-08T21:17:29.327497+0000 front 2024-03-08T21:17:29.328128+0000 (oldest deadline 2024-03-08T21:17:51.627384+0000) 2024-03-08T21:18:20.872 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:18:20 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:18:20.504+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.107:6822 osd.6 since back 2024-03-08T21:17:40.231398+0000 front 2024-03-08T21:17:40.231589+0000 (oldest deadline 2024-03-08T21:18:01.331142+0000) 2024-03-08T21:18:20.872 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:18:20 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:18:20.504+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.149:6820 osd.7 since back 2024-03-08T21:17:45.933742+0000 front 2024-03-08T21:17:45.933685+0000 (oldest deadline 2024-03-08T21:18:10.633316+0000) 2024-03-08T21:18:21.284 DEBUG:teuthology.orchestra.run.smithi149:> sudo pkill -f 'journalctl -f -n 0 -u ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715@osd.10.service' 2024-03-08T21:18:21.393 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-08T21:18:21.393 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-03-08T21:18:21.394 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-03-08T21:18:21.394 DEBUG:teuthology.orchestra.run.smithi204:> sudo systemctl stop ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715@osd.11 2024-03-08T21:18:21.871 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:18:21 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:18:21.472+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:42.614355+0000 front 2024-03-08T21:16:42.614271+0000 (oldest deadline 2024-03-08T21:17:07.914284+0000) 2024-03-08T21:18:21.871 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:18:21 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:18:21.472+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:52.016503+0000 front 2024-03-08T21:16:52.017017+0000 (oldest deadline 2024-03-08T21:17:16.116550+0000) 2024-03-08T21:18:21.871 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:18:21 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:18:21.472+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:16:57.818698+0000 front 2024-03-08T21:16:57.819201+0000 (oldest deadline 2024-03-08T21:17:26.619635+0000) 2024-03-08T21:18:21.871 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:18:21 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:18:21.472+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-08T21:17:19.424875+0000 front 2024-03-08T21:17:19.424454+0000 (oldest deadline 2024-03-08T21:17:43.425189+0000) 2024-03-08T21:18:21.871 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:18:21 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:18:21.472+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-03-08T21:17:29.327497+0000 front 2024-03-08T21:17:29.328128+0000 (oldest deadline 2024-03-08T21:17:51.627384+0000) 2024-03-08T21:18:21.871 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:18:21 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:18:21.472+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.107:6822 osd.6 since back 2024-03-08T21:17:40.231398+0000 front 2024-03-08T21:17:40.231589+0000 (oldest deadline 2024-03-08T21:18:01.331142+0000) 2024-03-08T21:18:21.871 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:18:21 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:18:21.472+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.149:6820 osd.7 since back 2024-03-08T21:17:45.933742+0000 front 2024-03-08T21:17:45.933685+0000 (oldest deadline 2024-03-08T21:18:10.633316+0000) 2024-03-08T21:18:21.872 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:18:21 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:18:21.472+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.204:6820 osd.8 since back 2024-03-08T21:17:56.436116+0000 front 2024-03-08T21:17:56.436212+0000 (oldest deadline 2024-03-08T21:18:21.136100+0000) 2024-03-08T21:18:21.872 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:18:21 smithi204 systemd[1]: Stopping Ceph osd.11 for dc20f0f8-dd8c-11ee-95c6-87774f69a715... 2024-03-08T21:18:22.371 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:18:22 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:18:22.058+0000 7fdccc800700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-08T21:18:22.371 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:18:22 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:18:22.058+0000 7fdccc800700 -1 osd.11 115 *** Got signal Terminated *** 2024-03-08T21:18:22.371 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:18:22 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:18:22.058+0000 7fdccc800700 -1 osd.11 115 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-08T21:18:22.874 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:18:22 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:18:22.472+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:42.614355+0000 front 2024-03-08T21:16:42.614271+0000 (oldest deadline 2024-03-08T21:17:07.914284+0000) 2024-03-08T21:18:22.874 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:18:22 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:18:22.472+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:52.016503+0000 front 2024-03-08T21:16:52.017017+0000 (oldest deadline 2024-03-08T21:17:16.116550+0000) 2024-03-08T21:18:22.874 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:18:22 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:18:22.472+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:16:57.818698+0000 front 2024-03-08T21:16:57.819201+0000 (oldest deadline 2024-03-08T21:17:26.619635+0000) 2024-03-08T21:18:22.874 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:18:22 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:18:22.473+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-08T21:17:19.424875+0000 front 2024-03-08T21:17:19.424454+0000 (oldest deadline 2024-03-08T21:17:43.425189+0000) 2024-03-08T21:18:22.874 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:18:22 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:18:22.473+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-03-08T21:17:29.327497+0000 front 2024-03-08T21:17:29.328128+0000 (oldest deadline 2024-03-08T21:17:51.627384+0000) 2024-03-08T21:18:22.874 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:18:22 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:18:22.473+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.107:6822 osd.6 since back 2024-03-08T21:17:40.231398+0000 front 2024-03-08T21:17:40.231589+0000 (oldest deadline 2024-03-08T21:18:01.331142+0000) 2024-03-08T21:18:22.874 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:18:22 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:18:22.473+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.149:6820 osd.7 since back 2024-03-08T21:17:45.933742+0000 front 2024-03-08T21:17:45.933685+0000 (oldest deadline 2024-03-08T21:18:10.633316+0000) 2024-03-08T21:18:22.874 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:18:22 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:18:22.473+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.204:6820 osd.8 since back 2024-03-08T21:17:56.436116+0000 front 2024-03-08T21:17:56.436212+0000 (oldest deadline 2024-03-08T21:18:21.136100+0000) 2024-03-08T21:18:23.871 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:18:23 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:18:23.521+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:42.614355+0000 front 2024-03-08T21:16:42.614271+0000 (oldest deadline 2024-03-08T21:17:07.914284+0000) 2024-03-08T21:18:23.871 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:18:23 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:18:23.521+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:52.016503+0000 front 2024-03-08T21:16:52.017017+0000 (oldest deadline 2024-03-08T21:17:16.116550+0000) 2024-03-08T21:18:23.871 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:18:23 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:18:23.521+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:16:57.818698+0000 front 2024-03-08T21:16:57.819201+0000 (oldest deadline 2024-03-08T21:17:26.619635+0000) 2024-03-08T21:18:23.872 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:18:23 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:18:23.521+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-08T21:17:19.424875+0000 front 2024-03-08T21:17:19.424454+0000 (oldest deadline 2024-03-08T21:17:43.425189+0000) 2024-03-08T21:18:23.872 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:18:23 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:18:23.521+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-03-08T21:17:29.327497+0000 front 2024-03-08T21:17:29.328128+0000 (oldest deadline 2024-03-08T21:17:51.627384+0000) 2024-03-08T21:18:23.872 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:18:23 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:18:23.521+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.107:6822 osd.6 since back 2024-03-08T21:17:40.231398+0000 front 2024-03-08T21:17:40.231589+0000 (oldest deadline 2024-03-08T21:18:01.331142+0000) 2024-03-08T21:18:23.872 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:18:23 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:18:23.521+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.149:6820 osd.7 since back 2024-03-08T21:17:45.933742+0000 front 2024-03-08T21:17:45.933685+0000 (oldest deadline 2024-03-08T21:18:10.633316+0000) 2024-03-08T21:18:23.872 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:18:23 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:18:23.521+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.204:6820 osd.8 since back 2024-03-08T21:17:56.436116+0000 front 2024-03-08T21:17:56.436212+0000 (oldest deadline 2024-03-08T21:18:21.136100+0000) 2024-03-08T21:18:24.871 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:18:24 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:18:24.557+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:42.614355+0000 front 2024-03-08T21:16:42.614271+0000 (oldest deadline 2024-03-08T21:17:07.914284+0000) 2024-03-08T21:18:24.872 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:18:24 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:18:24.557+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:52.016503+0000 front 2024-03-08T21:16:52.017017+0000 (oldest deadline 2024-03-08T21:17:16.116550+0000) 2024-03-08T21:18:24.872 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:18:24 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:18:24.557+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:16:57.818698+0000 front 2024-03-08T21:16:57.819201+0000 (oldest deadline 2024-03-08T21:17:26.619635+0000) 2024-03-08T21:18:24.872 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:18:24 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:18:24.557+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-08T21:17:19.424875+0000 front 2024-03-08T21:17:19.424454+0000 (oldest deadline 2024-03-08T21:17:43.425189+0000) 2024-03-08T21:18:24.872 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:18:24 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:18:24.557+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-03-08T21:17:29.327497+0000 front 2024-03-08T21:17:29.328128+0000 (oldest deadline 2024-03-08T21:17:51.627384+0000) 2024-03-08T21:18:24.872 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:18:24 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:18:24.557+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.107:6822 osd.6 since back 2024-03-08T21:17:40.231398+0000 front 2024-03-08T21:17:40.231589+0000 (oldest deadline 2024-03-08T21:18:01.331142+0000) 2024-03-08T21:18:24.872 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:18:24 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:18:24.557+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.149:6820 osd.7 since back 2024-03-08T21:17:45.933742+0000 front 2024-03-08T21:17:45.933685+0000 (oldest deadline 2024-03-08T21:18:10.633316+0000) 2024-03-08T21:18:24.872 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:18:24 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:18:24.557+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.204:6820 osd.8 since back 2024-03-08T21:17:56.436116+0000 front 2024-03-08T21:17:56.436212+0000 (oldest deadline 2024-03-08T21:18:21.136100+0000) 2024-03-08T21:18:25.871 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:18:25 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:18:25.606+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:42.614355+0000 front 2024-03-08T21:16:42.614271+0000 (oldest deadline 2024-03-08T21:17:07.914284+0000) 2024-03-08T21:18:25.871 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:18:25 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:18:25.606+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:52.016503+0000 front 2024-03-08T21:16:52.017017+0000 (oldest deadline 2024-03-08T21:17:16.116550+0000) 2024-03-08T21:18:25.871 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:18:25 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:18:25.606+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:16:57.818698+0000 front 2024-03-08T21:16:57.819201+0000 (oldest deadline 2024-03-08T21:17:26.619635+0000) 2024-03-08T21:18:25.871 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:18:25 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:18:25.606+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-08T21:17:19.424875+0000 front 2024-03-08T21:17:19.424454+0000 (oldest deadline 2024-03-08T21:17:43.425189+0000) 2024-03-08T21:18:25.872 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:18:25 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:18:25.606+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-03-08T21:17:29.327497+0000 front 2024-03-08T21:17:29.328128+0000 (oldest deadline 2024-03-08T21:17:51.627384+0000) 2024-03-08T21:18:25.872 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:18:25 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:18:25.607+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.107:6822 osd.6 since back 2024-03-08T21:17:40.231398+0000 front 2024-03-08T21:17:40.231589+0000 (oldest deadline 2024-03-08T21:18:01.331142+0000) 2024-03-08T21:18:25.872 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:18:25 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:18:25.607+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.149:6820 osd.7 since back 2024-03-08T21:17:45.933742+0000 front 2024-03-08T21:17:45.933685+0000 (oldest deadline 2024-03-08T21:18:10.633316+0000) 2024-03-08T21:18:25.872 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:18:25 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:18:25.607+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.204:6820 osd.8 since back 2024-03-08T21:17:56.436116+0000 front 2024-03-08T21:17:56.436212+0000 (oldest deadline 2024-03-08T21:18:21.136100+0000) 2024-03-08T21:18:26.871 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:18:26 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:18:26.582+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-03-08T21:16:42.614355+0000 front 2024-03-08T21:16:42.614271+0000 (oldest deadline 2024-03-08T21:17:07.914284+0000) 2024-03-08T21:18:26.871 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:18:26 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:18:26.582+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.149:6804 osd.1 since back 2024-03-08T21:16:52.016503+0000 front 2024-03-08T21:16:52.017017+0000 (oldest deadline 2024-03-08T21:17:16.116550+0000) 2024-03-08T21:18:26.871 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:18:26 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:18:26.582+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.204:6804 osd.2 since back 2024-03-08T21:16:57.818698+0000 front 2024-03-08T21:16:57.819201+0000 (oldest deadline 2024-03-08T21:17:26.619635+0000) 2024-03-08T21:18:26.871 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:18:26 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:18:26.582+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.149:6812 osd.4 since back 2024-03-08T21:17:19.424875+0000 front 2024-03-08T21:17:19.424454+0000 (oldest deadline 2024-03-08T21:17:43.425189+0000) 2024-03-08T21:18:26.871 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:18:26 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:18:26.582+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.204:6812 osd.5 since back 2024-03-08T21:17:29.327497+0000 front 2024-03-08T21:17:29.328128+0000 (oldest deadline 2024-03-08T21:17:51.627384+0000) 2024-03-08T21:18:26.871 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:18:26 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:18:26.582+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.107:6822 osd.6 since back 2024-03-08T21:17:40.231398+0000 front 2024-03-08T21:17:40.231589+0000 (oldest deadline 2024-03-08T21:18:01.331142+0000) 2024-03-08T21:18:26.871 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:18:26 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:18:26.582+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.149:6820 osd.7 since back 2024-03-08T21:17:45.933742+0000 front 2024-03-08T21:17:45.933685+0000 (oldest deadline 2024-03-08T21:18:10.633316+0000) 2024-03-08T21:18:26.872 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:18:26 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11[60676]: 2024-03-08T21:18:26.582+0000 7fdcc3000700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.204:6820 osd.8 since back 2024-03-08T21:17:56.436116+0000 front 2024-03-08T21:17:56.436212+0000 (oldest deadline 2024-03-08T21:18:21.136100+0000) 2024-03-08T21:18:27.708 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:18:27 smithi204 podman[74173]: 2024-03-08 21:18:27.468002509 +0000 UTC m=+5.662697192 container died 3d7919046eb54dc0ec4781eb966834ea42fc3305266e185b04ace2d2f210b061 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=rishabh-2024mar8-7d63482, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5) 2024-03-08T21:18:28.020 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:18:27 smithi204 podman[74173]: 2024-03-08 21:18:27.83472598 +0000 UTC m=+6.029420700 container cleanup 3d7919046eb54dc0ec4781eb966834ea42fc3305266e185b04ace2d2f210b061 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, ceph=True, org.label-schema.license=GPLv2, RELEASE=rishabh-2024mar8-7d63482, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-03-08T21:18:28.020 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:18:27 smithi204 bash[74173]: ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11 2024-03-08T21:18:28.370 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:18:28 smithi204 podman[74193]: 2024-03-08 21:18:28.044156689 +0000 UTC m=+0.561825823 container remove 3d7919046eb54dc0ec4781eb966834ea42fc3305266e185b04ace2d2f210b061 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=rishabh-2024mar8-7d63482, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-03-08T21:18:29.065 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:18:28 smithi204 podman[74280]: 2024-03-08 21:18:28.721586136 +0000 UTC m=+0.035994353 image pull a272146cb810b1fd4a53627f019472132e30966d5c25b01bfc44a001295f1249 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd 2024-03-08T21:18:29.066 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:18:28 smithi204 podman[74280]: 2024-03-08 21:18:28.828441408 +0000 UTC m=+0.142849649 container create 091b44bee0fd95b9a0b6cf232ca3249fe17728cf09ac481d4c53f40d02276f90 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11-deactivate, RELEASE=rishabh-2024mar8-7d63482, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-08T21:18:29.066 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:18:29 smithi204 podman[74280]: 2024-03-08 21:18:29.090377432 +0000 UTC m=+0.404785680 container init 091b44bee0fd95b9a0b6cf232ca3249fe17728cf09ac481d4c53f40d02276f90 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11-deactivate, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=rishabh-2024mar8-7d63482, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-08T21:18:29.370 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:18:29 smithi204 podman[74280]: 2024-03-08 21:18:29.101599947 +0000 UTC m=+0.416008194 container start 091b44bee0fd95b9a0b6cf232ca3249fe17728cf09ac481d4c53f40d02276f90 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11-deactivate, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, RELEASE=rishabh-2024mar8-7d63482, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-03-08T21:18:29.371 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:18:29 smithi204 podman[74280]: 2024-03-08 21:18:29.205863331 +0000 UTC m=+0.520271616 container attach 091b44bee0fd95b9a0b6cf232ca3249fe17728cf09ac481d4c53f40d02276f90 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11-deactivate, RELEASE=rishabh-2024mar8-7d63482, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307) 2024-03-08T21:18:29.847 INFO:journalctl@ceph.osd.11.smithi204.stdout:Mar 08 21:18:29 smithi204 podman[74280]: 2024-03-08 21:18:29.533186745 +0000 UTC m=+0.847595019 container died 091b44bee0fd95b9a0b6cf232ca3249fe17728cf09ac481d4c53f40d02276f90 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-osd-11-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=rishabh-2024mar8-7d63482, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-03-08T21:18:30.245 DEBUG:teuthology.orchestra.run.smithi204:> sudo pkill -f 'journalctl -f -n 0 -u ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715@osd.11.service' 2024-03-08T21:18:30.360 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-08T21:18:30.361 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-03-08T21:18:30.361 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-03-08T21:18:30.361 DEBUG:teuthology.orchestra.run.smithi107:> sudo systemctl stop ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715@mds.a 2024-03-08T21:18:30.709 INFO:journalctl@ceph.mds.a.smithi107.stdout:Mar 08 21:18:30 smithi107 systemd[1]: Stopping Ceph mds.a for dc20f0f8-dd8c-11ee-95c6-87774f69a715... 2024-03-08T21:18:31.210 INFO:journalctl@ceph.mds.a.smithi107.stdout:Mar 08 21:18:30 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-a[73957]: 2024-03-08T21:18:30.941+0000 7f07e4000700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-08T21:18:31.210 INFO:journalctl@ceph.mds.a.smithi107.stdout:Mar 08 21:18:30 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-a[73957]: 2024-03-08T21:18:30.941+0000 7f07e4000700 -1 mds.a *** got signal Terminated *** 2024-03-08T21:18:32.460 INFO:journalctl@ceph.mds.a.smithi107.stdout:Mar 08 21:18:32 smithi107 podman[95970]: 2024-03-08 21:18:32.371103704 +0000 UTC m=+1.573055322 container died b3abd6c77684313e6b255934f6df11e4748740b54d37c859790bfb33748917de (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-a, RELEASE=rishabh-2024mar8-7d63482, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0) 2024-03-08T21:18:32.960 INFO:journalctl@ceph.mds.a.smithi107.stdout:Mar 08 21:18:32 smithi107 podman[95970]: 2024-03-08 21:18:32.836107931 +0000 UTC m=+2.038059570 container cleanup b3abd6c77684313e6b255934f6df11e4748740b54d37c859790bfb33748917de (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-a, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=rishabh-2024mar8-7d63482, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0) 2024-03-08T21:18:32.960 INFO:journalctl@ceph.mds.a.smithi107.stdout:Mar 08 21:18:32 smithi107 bash[95970]: ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-a 2024-03-08T21:18:33.385 INFO:journalctl@ceph.mds.a.smithi107.stdout:Mar 08 21:18:33 smithi107 podman[95993]: 2024-03-08 21:18:33.020483851 +0000 UTC m=+0.629969724 container remove b3abd6c77684313e6b255934f6df11e4748740b54d37c859790bfb33748917de (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-a, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=rishabh-2024mar8-7d63482, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-03-08T21:18:33.421 DEBUG:teuthology.orchestra.run.smithi107:> sudo pkill -f 'journalctl -f -n 0 -u ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715@mds.a.service' 2024-03-08T21:18:33.540 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-08T21:18:33.540 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-03-08T21:18:33.540 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-03-08T21:18:33.540 DEBUG:teuthology.orchestra.run.smithi107:> sudo systemctl stop ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715@mds.d 2024-03-08T21:18:33.960 INFO:journalctl@ceph.mds.d.smithi107.stdout:Mar 08 21:18:33 smithi107 systemd[1]: Stopping Ceph mds.d for dc20f0f8-dd8c-11ee-95c6-87774f69a715... 2024-03-08T21:18:34.460 INFO:journalctl@ceph.mds.d.smithi107.stdout:Mar 08 21:18:34 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-d[92234]: 2024-03-08T21:18:34.144+0000 7f4ffee00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-08T21:18:34.460 INFO:journalctl@ceph.mds.d.smithi107.stdout:Mar 08 21:18:34 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-d[92234]: 2024-03-08T21:18:34.144+0000 7f4ffee00700 -1 mds.d *** got signal Terminated *** 2024-03-08T21:18:38.543 INFO:journalctl@ceph.mds.d.smithi107.stdout:Mar 08 21:18:38 smithi107 podman[96101]: 2024-03-08 21:18:38.239024164 +0000 UTC m=+4.240840353 container died 4591026eeab2fd936f64158ced5c82a9e6430b80aea54d4523d7fc9b1184554f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-d, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=rishabh-2024mar8-7d63482, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240307) 2024-03-08T21:18:38.841 INFO:journalctl@ceph.mds.d.smithi107.stdout:Mar 08 21:18:38 smithi107 podman[96101]: 2024-03-08 21:18:38.681057891 +0000 UTC m=+4.682874116 container cleanup 4591026eeab2fd936f64158ced5c82a9e6430b80aea54d4523d7fc9b1184554f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-d, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, RELEASE=rishabh-2024mar8-7d63482, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-03-08T21:18:38.841 INFO:journalctl@ceph.mds.d.smithi107.stdout:Mar 08 21:18:38 smithi107 bash[96101]: ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-d 2024-03-08T21:18:39.210 INFO:journalctl@ceph.mds.d.smithi107.stdout:Mar 08 21:18:38 smithi107 podman[96115]: 2024-03-08 21:18:38.865452948 +0000 UTC m=+0.610869621 container remove 4591026eeab2fd936f64158ced5c82a9e6430b80aea54d4523d7fc9b1184554f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-d, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=rishabh-2024mar8-7d63482, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True) 2024-03-08T21:18:39.283 DEBUG:teuthology.orchestra.run.smithi107:> sudo pkill -f 'journalctl -f -n 0 -u ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715@mds.d.service' 2024-03-08T21:18:39.401 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-08T21:18:39.401 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-03-08T21:18:39.401 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-03-08T21:18:39.401 DEBUG:teuthology.orchestra.run.smithi107:> sudo systemctl stop ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715@mds.g 2024-03-08T21:18:39.960 INFO:journalctl@ceph.mds.g.smithi107.stdout:Mar 08 21:18:39 smithi107 systemd[1]: Stopping Ceph mds.g for dc20f0f8-dd8c-11ee-95c6-87774f69a715... 2024-03-08T21:18:40.460 INFO:journalctl@ceph.mds.g.smithi107.stdout:Mar 08 21:18:40 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-g[73590]: 2024-03-08T21:18:40.006+0000 7f0000800700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-08T21:18:40.460 INFO:journalctl@ceph.mds.g.smithi107.stdout:Mar 08 21:18:40 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-g[73590]: 2024-03-08T21:18:40.006+0000 7f0000800700 -1 mds.g *** got signal Terminated *** 2024-03-08T21:18:44.395 INFO:journalctl@ceph.mds.g.smithi107.stdout:Mar 08 21:18:44 smithi107 podman[96220]: 2024-03-08 21:18:44.15115762 +0000 UTC m=+4.263280768 container died 35370fcd36cddd9c2db9902163381717421fce830fb0393705ae6398d3dd16b5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-g, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=rishabh-2024mar8-7d63482, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD) 2024-03-08T21:18:44.683 INFO:journalctl@ceph.mds.g.smithi107.stdout:Mar 08 21:18:44 smithi107 podman[96220]: 2024-03-08 21:18:44.531824539 +0000 UTC m=+4.643947680 container cleanup 35370fcd36cddd9c2db9902163381717421fce830fb0393705ae6398d3dd16b5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-g, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=rishabh-2024mar8-7d63482, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-08T21:18:44.683 INFO:journalctl@ceph.mds.g.smithi107.stdout:Mar 08 21:18:44 smithi107 bash[96220]: ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-g 2024-03-08T21:18:44.959 INFO:journalctl@ceph.mds.g.smithi107.stdout:Mar 08 21:18:44 smithi107 podman[96234]: 2024-03-08 21:18:44.707738955 +0000 UTC m=+0.540077347 container remove 35370fcd36cddd9c2db9902163381717421fce830fb0393705ae6398d3dd16b5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-g, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=rishabh-2024mar8-7d63482, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-03-08T21:18:45.176 DEBUG:teuthology.orchestra.run.smithi107:> sudo pkill -f 'journalctl -f -n 0 -u ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715@mds.g.service' 2024-03-08T21:18:45.283 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-08T21:18:45.283 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-03-08T21:18:45.284 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-03-08T21:18:45.284 DEBUG:teuthology.orchestra.run.smithi107:> sudo systemctl stop ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715@mds.j 2024-03-08T21:18:45.709 INFO:journalctl@ceph.mds.j.smithi107.stdout:Mar 08 21:18:45 smithi107 systemd[1]: Stopping Ceph mds.j for dc20f0f8-dd8c-11ee-95c6-87774f69a715... 2024-03-08T21:18:46.210 INFO:journalctl@ceph.mds.j.smithi107.stdout:Mar 08 21:18:45 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-j[72498]: 2024-03-08T21:18:45.898+0000 7f6f7a600700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-08T21:18:46.210 INFO:journalctl@ceph.mds.j.smithi107.stdout:Mar 08 21:18:45 smithi107 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-j[72498]: 2024-03-08T21:18:45.898+0000 7f6f7a600700 -1 mds.j *** got signal Terminated *** 2024-03-08T21:18:48.364 INFO:journalctl@ceph.mds.j.smithi107.stdout:Mar 08 21:18:48 smithi107 podman[96340]: 2024-03-08 21:18:48.096249558 +0000 UTC m=+2.327801111 container died eee17e8f22b6e13bc0a41e9b669f9c0e67b19dd92698cffab1999ec6f44e8dce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-j, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=rishabh-2024mar8-7d63482, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-03-08T21:18:48.710 INFO:journalctl@ceph.mds.j.smithi107.stdout:Mar 08 21:18:48 smithi107 podman[96340]: 2024-03-08 21:18:48.509657947 +0000 UTC m=+2.741209521 container cleanup eee17e8f22b6e13bc0a41e9b669f9c0e67b19dd92698cffab1999ec6f44e8dce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-j, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=rishabh-2024mar8-7d63482, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-03-08T21:18:48.710 INFO:journalctl@ceph.mds.j.smithi107.stdout:Mar 08 21:18:48 smithi107 bash[96340]: ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-j 2024-03-08T21:18:49.201 INFO:journalctl@ceph.mds.j.smithi107.stdout:Mar 08 21:18:48 smithi107 podman[96354]: 2024-03-08 21:18:48.785984534 +0000 UTC m=+0.681315686 container remove eee17e8f22b6e13bc0a41e9b669f9c0e67b19dd92698cffab1999ec6f44e8dce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-j, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=rishabh-2024mar8-7d63482, ceph=True, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-03-08T21:18:49.235 DEBUG:teuthology.orchestra.run.smithi107:> sudo pkill -f 'journalctl -f -n 0 -u ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715@mds.j.service' 2024-03-08T21:18:49.349 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-08T21:18:49.349 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-03-08T21:18:49.349 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-03-08T21:18:49.350 DEBUG:teuthology.orchestra.run.smithi149:> sudo systemctl stop ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715@mds.b 2024-03-08T21:18:49.757 INFO:journalctl@ceph.mds.b.smithi149.stdout:Mar 08 21:18:49 smithi149 systemd[1]: Stopping Ceph mds.b for dc20f0f8-dd8c-11ee-95c6-87774f69a715... 2024-03-08T21:18:50.257 INFO:journalctl@ceph.mds.b.smithi149.stdout:Mar 08 21:18:49 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-b[65616]: 2024-03-08T21:18:49.931+0000 7fb50d800700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-08T21:18:50.257 INFO:journalctl@ceph.mds.b.smithi149.stdout:Mar 08 21:18:49 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-b[65616]: 2024-03-08T21:18:49.931+0000 7fb50d800700 -1 mds.b *** got signal Terminated *** 2024-03-08T21:18:57.355 INFO:journalctl@ceph.mds.b.smithi149.stdout:Mar 08 21:18:57 smithi149 podman[73218]: 2024-03-08 21:18:57.088911371 +0000 UTC m=+7.287667082 container died e8a745fa0d1435df404db22d43b0e4216e2aadefca5629219b664e920814c4de (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-b, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=rishabh-2024mar8-7d63482, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-03-08T21:18:57.758 INFO:journalctl@ceph.mds.b.smithi149.stdout:Mar 08 21:18:57 smithi149 podman[73218]: 2024-03-08 21:18:57.474516725 +0000 UTC m=+7.673272476 container cleanup e8a745fa0d1435df404db22d43b0e4216e2aadefca5629219b664e920814c4de (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-b, RELEASE=rishabh-2024mar8-7d63482, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-03-08T21:18:57.758 INFO:journalctl@ceph.mds.b.smithi149.stdout:Mar 08 21:18:57 smithi149 bash[73218]: ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-b 2024-03-08T21:18:58.011 DEBUG:teuthology.orchestra.run.smithi149:> sudo pkill -f 'journalctl -f -n 0 -u ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715@mds.b.service' 2024-03-08T21:18:58.094 INFO:journalctl@ceph.mds.b.smithi149.stdout:Mar 08 21:18:57 smithi149 podman[73244]: 2024-03-08 21:18:57.784471095 +0000 UTC m=+0.678533370 container remove e8a745fa0d1435df404db22d43b0e4216e2aadefca5629219b664e920814c4de (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-b, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240307, ceph=True, RELEASE=rishabh-2024mar8-7d63482, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-08T21:18:58.094 INFO:journalctl@ceph.mds.b.smithi149.stdout:Mar 08 21:18:58 smithi149 systemd[1]: ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715@mds.b.service: Deactivated successfully. 2024-03-08T21:18:58.094 INFO:journalctl@ceph.mds.b.smithi149.stdout:Mar 08 21:18:58 smithi149 systemd[1]: Stopped Ceph mds.b for dc20f0f8-dd8c-11ee-95c6-87774f69a715. 2024-03-08T21:18:58.094 INFO:journalctl@ceph.mds.b.smithi149.stdout:Mar 08 21:18:58 smithi149 systemd[1]: ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715@mds.b.service: Consumed 47.154s CPU time. 2024-03-08T21:18:58.131 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-08T21:18:58.132 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-03-08T21:18:58.132 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-03-08T21:18:58.132 DEBUG:teuthology.orchestra.run.smithi149:> sudo systemctl stop ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715@mds.e 2024-03-08T21:18:58.507 INFO:journalctl@ceph.mds.e.smithi149.stdout:Mar 08 21:18:58 smithi149 systemd[1]: Stopping Ceph mds.e for dc20f0f8-dd8c-11ee-95c6-87774f69a715... 2024-03-08T21:18:59.007 INFO:journalctl@ceph.mds.e.smithi149.stdout:Mar 08 21:18:58 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-e[65000]: 2024-03-08T21:18:58.700+0000 7f90ec400700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-08T21:18:59.008 INFO:journalctl@ceph.mds.e.smithi149.stdout:Mar 08 21:18:58 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-e[65000]: 2024-03-08T21:18:58.700+0000 7f90ec400700 -1 mds.e *** got signal Terminated *** 2024-03-08T21:19:04.491 INFO:journalctl@ceph.mds.e.smithi149.stdout:Mar 08 21:19:04 smithi149 podman[73350]: 2024-03-08 21:19:04.215476207 +0000 UTC m=+5.641390670 container died 112663c35378aa0d66806c78bb216536c2968fe367b8d25bec33bd87cc111981 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-e, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=rishabh-2024mar8-7d63482, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-03-08T21:19:04.758 INFO:journalctl@ceph.mds.e.smithi149.stdout:Mar 08 21:19:04 smithi149 podman[73350]: 2024-03-08 21:19:04.628131847 +0000 UTC m=+6.054046367 container cleanup 112663c35378aa0d66806c78bb216536c2968fe367b8d25bec33bd87cc111981 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-e, RELEASE=rishabh-2024mar8-7d63482, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True) 2024-03-08T21:19:04.758 INFO:journalctl@ceph.mds.e.smithi149.stdout:Mar 08 21:19:04 smithi149 bash[73350]: ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-e 2024-03-08T21:19:05.258 INFO:journalctl@ceph.mds.e.smithi149.stdout:Mar 08 21:19:04 smithi149 podman[73364]: 2024-03-08 21:19:04.904027328 +0000 UTC m=+0.674920367 container remove 112663c35378aa0d66806c78bb216536c2968fe367b8d25bec33bd87cc111981 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-e, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=rishabh-2024mar8-7d63482, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-03-08T21:19:05.329 DEBUG:teuthology.orchestra.run.smithi149:> sudo pkill -f 'journalctl -f -n 0 -u ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715@mds.e.service' 2024-03-08T21:19:05.443 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-08T21:19:05.444 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-03-08T21:19:05.444 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-03-08T21:19:05.444 DEBUG:teuthology.orchestra.run.smithi149:> sudo systemctl stop ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715@mds.h 2024-03-08T21:19:05.758 INFO:journalctl@ceph.mds.h.smithi149.stdout:Mar 08 21:19:05 smithi149 systemd[1]: Stopping Ceph mds.h for dc20f0f8-dd8c-11ee-95c6-87774f69a715... 2024-03-08T21:19:06.508 INFO:journalctl@ceph.mds.h.smithi149.stdout:Mar 08 21:19:06 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-h[65300]: 2024-03-08T21:19:06.036+0000 7f84dca00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-08T21:19:06.508 INFO:journalctl@ceph.mds.h.smithi149.stdout:Mar 08 21:19:06 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-h[65300]: 2024-03-08T21:19:06.036+0000 7f84dca00700 -1 mds.h *** got signal Terminated *** 2024-03-08T21:19:11.345 INFO:journalctl@ceph.mds.h.smithi149.stdout:Mar 08 21:19:11 smithi149 podman[73470]: 2024-03-08 21:19:11.04800613 +0000 UTC m=+5.151969180 container died 76101b0dc8103f89bc0f1d9b80bec5781a96e22b239a1a08d12a2c45d6c14cf0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-h, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=rishabh-2024mar8-7d63482, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240307, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-03-08T21:19:11.683 INFO:journalctl@ceph.mds.h.smithi149.stdout:Mar 08 21:19:11 smithi149 podman[73470]: 2024-03-08 21:19:11.506223071 +0000 UTC m=+5.610186114 container cleanup 76101b0dc8103f89bc0f1d9b80bec5781a96e22b239a1a08d12a2c45d6c14cf0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-h, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=rishabh-2024mar8-7d63482, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, ceph=True) 2024-03-08T21:19:11.683 INFO:journalctl@ceph.mds.h.smithi149.stdout:Mar 08 21:19:11 smithi149 bash[73470]: ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-h 2024-03-08T21:19:12.008 INFO:journalctl@ceph.mds.h.smithi149.stdout:Mar 08 21:19:11 smithi149 podman[73484]: 2024-03-08 21:19:11.707451771 +0000 UTC m=+0.640572577 container remove 76101b0dc8103f89bc0f1d9b80bec5781a96e22b239a1a08d12a2c45d6c14cf0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-h, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, RELEASE=rishabh-2024mar8-7d63482, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0) 2024-03-08T21:19:12.150 DEBUG:teuthology.orchestra.run.smithi149:> sudo pkill -f 'journalctl -f -n 0 -u ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715@mds.h.service' 2024-03-08T21:19:12.261 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-08T21:19:12.261 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-03-08T21:19:12.261 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-03-08T21:19:12.262 DEBUG:teuthology.orchestra.run.smithi149:> sudo systemctl stop ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715@mds.k 2024-03-08T21:19:12.508 INFO:journalctl@ceph.mds.k.smithi149.stdout:Mar 08 21:19:12 smithi149 systemd[1]: Stopping Ceph mds.k for dc20f0f8-dd8c-11ee-95c6-87774f69a715... 2024-03-08T21:19:13.258 INFO:journalctl@ceph.mds.k.smithi149.stdout:Mar 08 21:19:12 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-k[64685]: 2024-03-08T21:19:12.781+0000 7f80f1a00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-08T21:19:13.258 INFO:journalctl@ceph.mds.k.smithi149.stdout:Mar 08 21:19:12 smithi149 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-k[64685]: 2024-03-08T21:19:12.781+0000 7f80f1a00700 -1 mds.k *** got signal Terminated *** 2024-03-08T21:19:17.478 INFO:journalctl@ceph.mds.k.smithi149.stdout:Mar 08 21:19:17 smithi149 podman[73590]: 2024-03-08 21:19:17.173468907 +0000 UTC m=+4.511650686 container died 4bdbc491acceb2fa6e5863a64c8e0662249afcef287b2870c557152e597f3d65 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-k, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=rishabh-2024mar8-7d63482, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-03-08T21:19:17.758 INFO:journalctl@ceph.mds.k.smithi149.stdout:Mar 08 21:19:17 smithi149 podman[73590]: 2024-03-08 21:19:17.667405842 +0000 UTC m=+5.005587648 container cleanup 4bdbc491acceb2fa6e5863a64c8e0662249afcef287b2870c557152e597f3d65 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-k, RELEASE=rishabh-2024mar8-7d63482, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-08T21:19:17.758 INFO:journalctl@ceph.mds.k.smithi149.stdout:Mar 08 21:19:17 smithi149 bash[73590]: ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-k 2024-03-08T21:19:18.164 INFO:journalctl@ceph.mds.k.smithi149.stdout:Mar 08 21:19:17 smithi149 podman[73604]: 2024-03-08 21:19:17.968435592 +0000 UTC m=+0.774446752 container remove 4bdbc491acceb2fa6e5863a64c8e0662249afcef287b2870c557152e597f3d65 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-k, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240307, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=rishabh-2024mar8-7d63482, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-08T21:19:18.193 DEBUG:teuthology.orchestra.run.smithi149:> sudo pkill -f 'journalctl -f -n 0 -u ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715@mds.k.service' 2024-03-08T21:19:18.312 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-08T21:19:18.313 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-03-08T21:19:18.313 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-03-08T21:19:18.313 DEBUG:teuthology.orchestra.run.smithi204:> sudo systemctl stop ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715@mds.c 2024-03-08T21:19:18.425 DEBUG:teuthology.orchestra.run.smithi204:> sudo pkill -f 'journalctl -f -n 0 -u ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715@mds.c.service' 2024-03-08T21:19:18.536 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-08T21:19:18.536 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-03-08T21:19:18.536 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-03-08T21:19:18.537 DEBUG:teuthology.orchestra.run.smithi204:> sudo systemctl stop ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715@mds.f 2024-03-08T21:19:18.871 INFO:journalctl@ceph.mds.f.smithi204.stdout:Mar 08 21:19:18 smithi204 systemd[1]: Stopping Ceph mds.f for dc20f0f8-dd8c-11ee-95c6-87774f69a715... 2024-03-08T21:19:19.371 INFO:journalctl@ceph.mds.f.smithi204.stdout:Mar 08 21:19:19 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-f[65700]: 2024-03-08T21:19:19.108+0000 7f46a0a00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-08T21:19:19.371 INFO:journalctl@ceph.mds.f.smithi204.stdout:Mar 08 21:19:19 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-f[65700]: 2024-03-08T21:19:19.108+0000 7f46a0a00700 -1 mds.f *** got signal Terminated *** 2024-03-08T21:19:20.539 INFO:journalctl@ceph.mds.f.smithi204.stdout:Mar 08 21:19:20 smithi204 podman[74420]: 2024-03-08 21:19:20.394132736 +0000 UTC m=+1.443773096 container died 05a8d06ffdecbaff4a9e91335b55fc86902670d41b43c5737cd37154a216508f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-f, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=rishabh-2024mar8-7d63482, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-08T21:19:20.789 INFO:journalctl@ceph.mds.f.smithi204.stdout:Mar 08 21:19:20 smithi204 podman[74420]: 2024-03-08 21:19:20.808532899 +0000 UTC m=+1.858173300 container cleanup 05a8d06ffdecbaff4a9e91335b55fc86902670d41b43c5737cd37154a216508f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-f, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=rishabh-2024mar8-7d63482, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-08T21:19:20.790 INFO:journalctl@ceph.mds.f.smithi204.stdout:Mar 08 21:19:20 smithi204 bash[74420]: ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-f 2024-03-08T21:19:21.121 INFO:journalctl@ceph.mds.f.smithi204.stdout:Mar 08 21:19:21 smithi204 podman[74437]: 2024-03-08 21:19:21.051290611 +0000 UTC m=+0.647912383 container remove 05a8d06ffdecbaff4a9e91335b55fc86902670d41b43c5737cd37154a216508f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-f, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240307, RELEASE=rishabh-2024mar8-7d63482, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-08T21:19:21.503 DEBUG:teuthology.orchestra.run.smithi204:> sudo pkill -f 'journalctl -f -n 0 -u ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715@mds.f.service' 2024-03-08T21:19:21.595 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-08T21:19:21.596 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-03-08T21:19:21.596 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-03-08T21:19:21.596 DEBUG:teuthology.orchestra.run.smithi204:> sudo systemctl stop ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715@mds.i 2024-03-08T21:19:22.121 INFO:journalctl@ceph.mds.i.smithi204.stdout:Mar 08 21:19:21 smithi204 systemd[1]: Stopping Ceph mds.i for dc20f0f8-dd8c-11ee-95c6-87774f69a715... 2024-03-08T21:19:22.621 INFO:journalctl@ceph.mds.i.smithi204.stdout:Mar 08 21:19:22 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-i[66312]: 2024-03-08T21:19:22.185+0000 7f48c1800700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-08T21:19:22.621 INFO:journalctl@ceph.mds.i.smithi204.stdout:Mar 08 21:19:22 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-i[66312]: 2024-03-08T21:19:22.185+0000 7f48c1800700 -1 mds.i *** got signal Terminated *** 2024-03-08T21:19:26.371 INFO:journalctl@ceph.mds.i.smithi204.stdout:Mar 08 21:19:26 smithi204 podman[74553]: 2024-03-08 21:19:26.045134621 +0000 UTC m=+3.992797117 container died 728070e41a85ef6dc95d06a9e86ddb324894b510b7348b37576273167f33b7cc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-i, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=rishabh-2024mar8-7d63482, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240307, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-08T21:19:26.807 INFO:journalctl@ceph.mds.i.smithi204.stdout:Mar 08 21:19:26 smithi204 podman[74553]: 2024-03-08 21:19:26.57132258 +0000 UTC m=+4.518985118 container cleanup 728070e41a85ef6dc95d06a9e86ddb324894b510b7348b37576273167f33b7cc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-i, RELEASE=rishabh-2024mar8-7d63482, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240307, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS) 2024-03-08T21:19:26.807 INFO:journalctl@ceph.mds.i.smithi204.stdout:Mar 08 21:19:26 smithi204 bash[74553]: ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-i 2024-03-08T21:19:27.121 INFO:journalctl@ceph.mds.i.smithi204.stdout:Mar 08 21:19:26 smithi204 podman[74567]: 2024-03-08 21:19:26.830602584 +0000 UTC m=+0.772268473 container remove 728070e41a85ef6dc95d06a9e86ddb324894b510b7348b37576273167f33b7cc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-i, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=rishabh-2024mar8-7d63482, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240307, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-03-08T21:19:27.381 DEBUG:teuthology.orchestra.run.smithi204:> sudo pkill -f 'journalctl -f -n 0 -u ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715@mds.i.service' 2024-03-08T21:19:27.495 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-08T21:19:27.495 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-03-08T21:19:27.495 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-03-08T21:19:27.495 DEBUG:teuthology.orchestra.run.smithi204:> sudo systemctl stop ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715@mds.l 2024-03-08T21:19:27.621 INFO:journalctl@ceph.mds.l.smithi204.stdout:Mar 08 21:19:27 smithi204 systemd[1]: Stopping Ceph mds.l for dc20f0f8-dd8c-11ee-95c6-87774f69a715... 2024-03-08T21:19:28.371 INFO:journalctl@ceph.mds.l.smithi204.stdout:Mar 08 21:19:28 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-l[65382]: 2024-03-08T21:19:28.080+0000 7f5cbbe00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-08T21:19:28.371 INFO:journalctl@ceph.mds.l.smithi204.stdout:Mar 08 21:19:28 smithi204 ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-l[65382]: 2024-03-08T21:19:28.080+0000 7f5cbbe00700 -1 mds.l *** got signal Terminated *** 2024-03-08T21:19:32.417 INFO:journalctl@ceph.mds.l.smithi204.stdout:Mar 08 21:19:32 smithi204 podman[74672]: 2024-03-08 21:19:32.185724592 +0000 UTC m=+4.242936305 container died 859782e18848f12e8625c6bbbfc37b077526221cf043677aa87f0d19aaaf2a28 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-l, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=rishabh-2024mar8-7d63482, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240307) 2024-03-08T21:19:32.702 INFO:journalctl@ceph.mds.l.smithi204.stdout:Mar 08 21:19:32 smithi204 podman[74672]: 2024-03-08 21:19:32.541538675 +0000 UTC m=+4.598750422 container cleanup 859782e18848f12e8625c6bbbfc37b077526221cf043677aa87f0d19aaaf2a28 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-l, GIT_CLEAN=True, RELEASE=rishabh-2024mar8-7d63482, org.label-schema.build-date=20240307, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD) 2024-03-08T21:19:32.702 INFO:journalctl@ceph.mds.l.smithi204.stdout:Mar 08 21:19:32 smithi204 bash[74672]: ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-l 2024-03-08T21:19:33.121 INFO:journalctl@ceph.mds.l.smithi204.stdout:Mar 08 21:19:32 smithi204 podman[74686]: 2024-03-08 21:19:32.725734233 +0000 UTC m=+0.530985106 container remove 859782e18848f12e8625c6bbbfc37b077526221cf043677aa87f0d19aaaf2a28 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:70b1c28ffbd1be392f87b7c6c0373971a00211c0624517727a0cab908539babd, name=ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715-mds-l, RELEASE=rishabh-2024mar8-7d63482, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240307, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS) 2024-03-08T21:19:33.185 DEBUG:teuthology.orchestra.run.smithi204:> sudo pkill -f 'journalctl -f -n 0 -u ceph-dc20f0f8-dd8c-11ee-95c6-87774f69a715@mds.l.service' 2024-03-08T21:19:33.285 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-08T21:19:33.285 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-03-08T21:19:33.285 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid dc20f0f8-dd8c-11ee-95c6-87774f69a715 --force --keep-logs 2024-03-08T21:19:33.608 INFO:teuthology.orchestra.run.smithi107.stdout:Deleting cluster with fsid: dc20f0f8-dd8c-11ee-95c6-87774f69a715 2024-03-08T21:19:38.347 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid dc20f0f8-dd8c-11ee-95c6-87774f69a715 --force --keep-logs 2024-03-08T21:19:38.667 INFO:teuthology.orchestra.run.smithi149.stdout:Deleting cluster with fsid: dc20f0f8-dd8c-11ee-95c6-87774f69a715 2024-03-08T21:19:43.553 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid dc20f0f8-dd8c-11ee-95c6-87774f69a715 --force --keep-logs 2024-03-08T21:19:43.872 INFO:teuthology.orchestra.run.smithi204.stdout:Deleting cluster with fsid: dc20f0f8-dd8c-11ee-95c6-87774f69a715 2024-03-08T21:19:48.646 DEBUG:teuthology.orchestra.run.smithi107:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-08T21:19:48.748 DEBUG:teuthology.orchestra.run.smithi149:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-08T21:19:48.849 DEBUG:teuthology.orchestra.run.smithi204:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-08T21:19:48.942 INFO:tasks.cephadm:Archiving crash dumps... 2024-03-08T21:19:48.944 DEBUG:teuthology.misc:Transferring archived files from smithi107:/var/lib/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/crash to /home/teuthworker/archive/rishabh-2024-03-08_19:54:47-fs-rishabh-2024mar8-testing-default-smithi/7588131/remote/smithi107/crash 2024-03-08T21:19:48.945 DEBUG:teuthology.orchestra.run.smithi107:> sudo tar c -f - -C /var/lib/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/crash -- . 2024-03-08T21:19:49.052 INFO:teuthology.orchestra.run.smithi107.stderr:tar: /var/lib/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/crash: Cannot open: No such file or directory 2024-03-08T21:19:49.052 INFO:teuthology.orchestra.run.smithi107.stderr:tar: Error is not recoverable: exiting now 2024-03-08T21:19:49.057 DEBUG:teuthology.misc:Transferring archived files from smithi149:/var/lib/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/crash to /home/teuthworker/archive/rishabh-2024-03-08_19:54:47-fs-rishabh-2024mar8-testing-default-smithi/7588131/remote/smithi149/crash 2024-03-08T21:19:49.058 DEBUG:teuthology.orchestra.run.smithi149:> sudo tar c -f - -C /var/lib/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/crash -- . 2024-03-08T21:19:49.170 INFO:teuthology.orchestra.run.smithi149.stderr:tar: /var/lib/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/crash: Cannot open: No such file or directory 2024-03-08T21:19:49.170 INFO:teuthology.orchestra.run.smithi149.stderr:tar: Error is not recoverable: exiting now 2024-03-08T21:19:49.176 DEBUG:teuthology.misc:Transferring archived files from smithi204:/var/lib/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/crash to /home/teuthworker/archive/rishabh-2024-03-08_19:54:47-fs-rishabh-2024mar8-testing-default-smithi/7588131/remote/smithi204/crash 2024-03-08T21:19:49.177 DEBUG:teuthology.orchestra.run.smithi204:> sudo tar c -f - -C /var/lib/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/crash -- . 2024-03-08T21:19:49.286 INFO:teuthology.orchestra.run.smithi204.stderr:tar: /var/lib/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/crash: Cannot open: No such file or directory 2024-03-08T21:19:49.286 INFO:teuthology.orchestra.run.smithi204.stderr:tar: Error is not recoverable: exiting now 2024-03-08T21:19:49.291 INFO:tasks.cephadm:Checking cluster log for badness... 2024-03-08T21:19:49.291 DEBUG:teuthology.orchestra.run.smithi107:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | head -n 1 2024-03-08T21:19:49.400 INFO:tasks.cephadm:Compressing logs... 2024-03-08T21:19:49.401 DEBUG:teuthology.orchestra.run.smithi107:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-08T21:19:49.405 DEBUG:teuthology.orchestra.run.smithi149:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-08T21:19:49.414 DEBUG:teuthology.orchestra.run.smithi204:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-08T21:19:49.469 INFO:teuthology.orchestra.run.smithi107.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-03-08T21:19:49.469 INFO:teuthology.orchestra.run.smithi107.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-03-08T21:19:49.470 INFO:teuthology.orchestra.run.smithi107.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.log 2024-03-08T21:19:49.472 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/ceph-osd.6.log 2024-03-08T21:19:49.473 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.0.log: gzip -5 --verbose -- /var/log/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/ceph-mon.a.log 2024-03-08T21:19:49.474 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/ceph.audit.log 2024-03-08T21:19:49.475 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/ceph-mds.g.log 2024-03-08T21:19:49.476 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/ceph-mds.j.log 2024-03-08T21:19:49.478 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/ceph-osd.3.log 2024-03-08T21:19:49.479 INFO:teuthology.orchestra.run.smithi107.stderr: 93.2% -- replaced with /var/log/ceph/cephadm.log.gz 2024-03-08T21:19:49.479 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/ceph-mgr.x.log 2024-03-08T21:19:49.482 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/ceph-osd.3.log: 90.0% -- replaced with /var/log/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/ceph.audit.log.gz 2024-03-08T21:19:49.482 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/ceph-osd.0.log 2024-03-08T21:19:49.485 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/ceph.cephadm.log 2024-03-08T21:19:49.488 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/ceph.cephadm.log: 94.6% -- replaced with /var/log/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/ceph-mds.j.log.gz 2024-03-08T21:19:49.489 INFO:teuthology.orchestra.run.smithi107.stderr:gzip -5 --verbose -- /var/log/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/ceph.log 2024-03-08T21:19:49.490 INFO:teuthology.orchestra.run.smithi107.stderr: 83.5% -- replaced with /var/log/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/ceph.cephadm.log.gz 2024-03-08T21:19:49.490 INFO:teuthology.orchestra.run.smithi107.stderr:gzip -5 --verbose -- /var/log/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/ceph-volume.log 2024-03-08T21:19:49.493 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/ceph.log: gzip -5 --verbose -- /var/log/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/ceph-osd.9.log 2024-03-08T21:19:49.496 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/ceph-mds.d.log 2024-03-08T21:19:49.497 INFO:teuthology.orchestra.run.smithi149.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-03-08T21:19:49.498 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/ceph-osd.9.log: 87.9% -- replaced with /var/log/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/ceph.log.gz 2024-03-08T21:19:49.498 INFO:teuthology.orchestra.run.smithi149.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-03-08T21:19:49.500 INFO:teuthology.orchestra.run.smithi149.stderr:gzip -5 --verbose -- /var/log/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/ceph-osd.10.log 2024-03-08T21:19:49.501 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/ceph.audit.log 2024-03-08T21:19:49.502 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/ceph-mds.a.log 2024-03-08T21:19:49.502 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/ceph-mds.b.log 2024-03-08T21:19:49.502 INFO:teuthology.orchestra.run.smithi149.stderr: 92.0% -- replaced with /var/log/ceph/cephadm.log.gz 2024-03-08T21:19:49.503 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/ceph-osd.4.log 2024-03-08T21:19:49.504 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/ceph-osd.1.log 2024-03-08T21:19:49.505 INFO:teuthology.orchestra.run.smithi204.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-03-08T21:19:49.506 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-03-08T21:19:49.507 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/ceph-osd.4.log: 90.2% -- replaced with /var/log/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/ceph.audit.log.gz 2024-03-08T21:19:49.507 INFO:teuthology.orchestra.run.smithi149.stderr:gzip -5 --verbose -- /var/log/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/ceph-mds.k.log 2024-03-08T21:19:49.508 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/ceph-mds.h.log 2024-03-08T21:19:49.508 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/ceph-mds.l.log 2024-03-08T21:19:49.508 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/ceph.audit.log 2024-03-08T21:19:49.509 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/ceph-mds.k.log: gzip -5 --verbose -- /var/log/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/ceph-osd.7.log 2024-03-08T21:19:49.509 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/ceph-mds.l.log: gzip -5 --verbose -- /var/log/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/ceph-mds.i.log 2024-03-08T21:19:49.510 INFO:teuthology.orchestra.run.smithi204.stderr: 92.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-03-08T21:19:49.510 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/ceph-osd.11.log 2024-03-08T21:19:49.510 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/ceph-mds.e.log 2024-03-08T21:19:49.511 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/ceph-mgr.z.log 2024-03-08T21:19:49.513 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/ceph-osd.8.log 2024-03-08T21:19:49.513 INFO:teuthology.orchestra.run.smithi204.stderr: 90.2% -- replaced with /var/log/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/ceph.audit.log.gz 2024-03-08T21:19:49.514 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/ceph-osd.5.log 2024-03-08T21:19:49.515 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/ceph-osd.7.log: /var/log/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/ceph-mds.e.log: gzip -5 --verbose -- /var/log/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/ceph-mgr.y.log 2024-03-08T21:19:49.515 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/ceph-mds.a.log: 90.9% -- replaced with /var/log/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/ceph-volume.log.gz 2024-03-08T21:19:49.516 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/ceph-mgr.z.log: /var/log/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/ceph-mds.f.log 2024-03-08T21:19:49.516 INFO:teuthology.orchestra.run.smithi149.stderr:gzip -5 --verbose -- /var/log/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/ceph.cephadm.log 2024-03-08T21:19:49.517 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/ceph.cephadm.log 2024-03-08T21:19:49.520 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/ceph.cephadm.log: /var/log/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/ceph.log 2024-03-08T21:19:49.522 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/ceph-mds.f.log: /var/log/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/ceph.cephadm.log: 83.0% -- replaced with /var/log/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/ceph.cephadm.log.gz 2024-03-08T21:19:49.522 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/ceph.log 2024-03-08T21:19:49.524 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/ceph.log: gzip -5 --verbose -- /var/log/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/ceph-osd.2.log 2024-03-08T21:19:49.525 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/ceph-mds.c.log 2024-03-08T21:19:49.528 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/ceph.log: 83.1% -- replaced with /var/log/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/ceph.cephadm.log.gz 2024-03-08T21:19:49.528 INFO:teuthology.orchestra.run.smithi149.stderr:gzip -5 --verbose -- /var/log/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/ceph-volume.log 2024-03-08T21:19:49.530 INFO:teuthology.orchestra.run.smithi149.stderr:gzip -5 --verbose -- /var/log/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/ceph-mon.b.log 2024-03-08T21:19:49.531 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/ceph-osd.2.log: 92.3% -- replaced with /var/log/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/ceph-mgr.z.log.gz 2024-03-08T21:19:49.532 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/ceph-mds.c.log: 88.0%gzip -5 --verbose -- /var/log/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/ceph-volume.log 2024-03-08T21:19:49.533 INFO:teuthology.orchestra.run.smithi204.stderr: -- replaced with /var/log/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/ceph.log.gz 2024-03-08T21:19:49.534 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/ceph-volume.log: 88.0% -- replaced with /var/log/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/ceph.log.gz 2024-03-08T21:19:49.536 INFO:teuthology.orchestra.run.smithi204.stderr:gzip -5 --verbose -- /var/log/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/ceph-mon.c.log 2024-03-08T21:19:49.550 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/ceph-mon.b.log: 92.3% -- replaced with /var/log/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/ceph-mgr.y.log.gz 2024-03-08T21:19:49.559 INFO:teuthology.orchestra.run.smithi107.stderr: 94.9% -- replaced with /var/log/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/ceph-mds.g.log.gz 2024-03-08T21:19:49.565 INFO:teuthology.orchestra.run.smithi149.stderr: 90.8% -- replaced with /var/log/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/ceph-volume.log.gz 2024-03-08T21:19:49.569 INFO:teuthology.orchestra.run.smithi204.stderr:/var/log/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/ceph-volume.log: /var/log/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/ceph-mon.c.log: 90.8% -- replaced with /var/log/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/ceph-volume.log.gz 2024-03-08T21:19:49.591 INFO:teuthology.orchestra.run.smithi107.stderr: 94.9% -- replaced with /var/log/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/ceph-mds.a.log.gz 2024-03-08T21:19:49.591 INFO:teuthology.orchestra.run.smithi204.stderr: 94.9% -- replaced with /var/log/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/ceph-mds.l.log.gz 2024-03-08T21:19:49.591 INFO:teuthology.orchestra.run.smithi149.stderr: 94.2% -- replaced with /var/log/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/ceph-mds.k.log.gz 2024-03-08T21:19:49.596 INFO:teuthology.orchestra.run.smithi204.stderr: 94.2% -- replaced with /var/log/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/ceph-mds.i.log.gz 2024-03-08T21:19:49.600 INFO:teuthology.orchestra.run.smithi204.stderr: 95.0% -- replaced with /var/log/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/ceph-mds.f.log.gz 2024-03-08T21:19:49.616 INFO:teuthology.orchestra.run.smithi149.stderr: 94.1% -- replaced with /var/log/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/ceph-mds.h.log.gz 2024-03-08T21:19:49.619 INFO:teuthology.orchestra.run.smithi149.stderr: 94.0% -- replaced with /var/log/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/ceph-mds.e.log.gz 2024-03-08T21:19:49.722 INFO:teuthology.orchestra.run.smithi204.stderr: 92.1% -- replaced with /var/log/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/ceph-mds.c.log.gz 2024-03-08T21:19:49.948 INFO:teuthology.orchestra.run.smithi107.stderr: 89.8% -- replaced with /var/log/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/ceph-mgr.x.log.gz 2024-03-08T21:19:49.997 INFO:teuthology.orchestra.run.smithi204.stderr: 91.7% -- replaced with /var/log/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/ceph-mon.c.log.gz 2024-03-08T21:19:50.029 INFO:teuthology.orchestra.run.smithi107.stderr: 95.5% -- replaced with /var/log/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/ceph-mds.d.log.gz 2024-03-08T21:19:50.067 INFO:teuthology.orchestra.run.smithi149.stderr: 92.1% -- replaced with /var/log/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/ceph-mon.b.log.gz 2024-03-08T21:19:50.705 INFO:teuthology.orchestra.run.smithi107.stderr: 90.4% -- replaced with /var/log/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/ceph-mon.a.log.gz 2024-03-08T21:19:51.147 INFO:teuthology.orchestra.run.smithi204.stderr: 91.6% -- replaced with /var/log/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/ceph-osd.8.log.gz 2024-03-08T21:19:51.207 INFO:teuthology.orchestra.run.smithi204.stderr: 92.0% -- replaced with /var/log/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/ceph-osd.2.log.gz 2024-03-08T21:19:51.256 INFO:teuthology.orchestra.run.smithi149.stderr: 91.9% -- replaced with /var/log/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/ceph-osd.1.log.gz 2024-03-08T21:19:51.391 INFO:teuthology.orchestra.run.smithi204.stderr: 92.6% -- replaced with /var/log/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/ceph-osd.11.log.gz 2024-03-08T21:19:51.701 INFO:teuthology.orchestra.run.smithi149.stderr: 91.8% -- replaced with /var/log/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/ceph-osd.7.log.gz 2024-03-08T21:19:51.701 INFO:teuthology.orchestra.run.smithi107.stderr: 92.4% -- replaced with /var/log/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/ceph-osd.3.log.gz 2024-03-08T21:19:51.766 INFO:teuthology.orchestra.run.smithi107.stderr: 92.6% -- replaced with /var/log/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/ceph-osd.0.log.gz 2024-03-08T21:19:51.967 INFO:teuthology.orchestra.run.smithi107.stderr: 92.2% -- replaced with /var/log/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/ceph-osd.9.log.gz 2024-03-08T21:19:52.023 INFO:teuthology.orchestra.run.smithi107.stderr: 92.3% -- replaced with /var/log/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/ceph-osd.6.log.gz 2024-03-08T21:19:52.024 INFO:teuthology.orchestra.run.smithi149.stderr: 92.4% -- replaced with /var/log/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/ceph-osd.4.log.gz 2024-03-08T21:19:52.034 INFO:teuthology.orchestra.run.smithi149.stderr: 92.2% -- replaced with /var/log/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/ceph-osd.10.log.gz 2024-03-08T21:19:52.343 INFO:teuthology.orchestra.run.smithi204.stderr: 93.1% -- replaced with /var/log/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/ceph-osd.5.log.gz 2024-03-08T21:19:52.349 INFO:teuthology.orchestra.run.smithi204.stderr: 2024-03-08T21:19:52.350 INFO:teuthology.orchestra.run.smithi204.stderr:real 0m2.881s 2024-03-08T21:19:52.350 INFO:teuthology.orchestra.run.smithi204.stderr:user 0m8.225s 2024-03-08T21:19:52.350 INFO:teuthology.orchestra.run.smithi204.stderr:sys 0m0.773s 2024-03-08T21:19:53.410 INFO:teuthology.orchestra.run.smithi149.stderr: 93.3% -- replaced with /var/log/ceph/dc20f0f8-dd8c-11ee-95c6-87774f69a715/ceph-mds.b.log.gz 2024-03-08T21:19:53.414 INFO:teuthology.orchestra.run.smithi149.stderr: 2024-03-08T21:19:53.415 INFO:teuthology.orchestra.run.smithi149.stderr:real 0m3.954s 2024-03-08T21:19:53.415 INFO:teuthology.orchestra.run.smithi149.stderr:user 0m12.709s 2024-03-08T21:19:53.415 INFO:teuthology.orchestra.run.smithi149.stderr:sys 0m1.061s 2024-03-08T21:20:25.428 INFO:teuthology.orchestra.run.smithi107.stderr: 96.6% -- replaced with /var/log/ceph/ceph-client.0.log.gz 2024-03-08T21:20:25.431 INFO:teuthology.orchestra.run.smithi107.stderr: 2024-03-08T21:20:25.431 INFO:teuthology.orchestra.run.smithi107.stderr:real 0m35.993s 2024-03-08T21:20:25.431 INFO:teuthology.orchestra.run.smithi107.stderr:user 0m43.851s 2024-03-08T21:20:25.431 INFO:teuthology.orchestra.run.smithi107.stderr:sys 0m3.985s 2024-03-08T21:20:25.432 INFO:tasks.cephadm:Archiving logs... 2024-03-08T21:20:25.432 DEBUG:teuthology.misc:Transferring archived files from smithi107:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-03-08_19:54:47-fs-rishabh-2024mar8-testing-default-smithi/7588131/remote/smithi107/log 2024-03-08T21:20:25.433 DEBUG:teuthology.orchestra.run.smithi107:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-08T21:20:30.542 DEBUG:teuthology.misc:Transferring archived files from smithi149:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-03-08_19:54:47-fs-rishabh-2024mar8-testing-default-smithi/7588131/remote/smithi149/log 2024-03-08T21:20:30.543 DEBUG:teuthology.orchestra.run.smithi149:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-08T21:20:32.443 DEBUG:teuthology.misc:Transferring archived files from smithi204:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-03-08_19:54:47-fs-rishabh-2024mar8-testing-default-smithi/7588131/remote/smithi204/log 2024-03-08T21:20:32.444 DEBUG:teuthology.orchestra.run.smithi204:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-08T21:20:33.848 INFO:tasks.cephadm:Removing cluster... 2024-03-08T21:20:33.849 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid dc20f0f8-dd8c-11ee-95c6-87774f69a715 --force 2024-03-08T21:20:34.172 INFO:teuthology.orchestra.run.smithi107.stdout:Deleting cluster with fsid: dc20f0f8-dd8c-11ee-95c6-87774f69a715 2024-03-08T21:20:34.767 DEBUG:teuthology.orchestra.run.smithi149:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid dc20f0f8-dd8c-11ee-95c6-87774f69a715 --force 2024-03-08T21:20:35.089 INFO:teuthology.orchestra.run.smithi149.stdout:Deleting cluster with fsid: dc20f0f8-dd8c-11ee-95c6-87774f69a715 2024-03-08T21:20:35.743 DEBUG:teuthology.orchestra.run.smithi204:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid dc20f0f8-dd8c-11ee-95c6-87774f69a715 --force 2024-03-08T21:20:36.065 INFO:teuthology.orchestra.run.smithi204.stdout:Deleting cluster with fsid: dc20f0f8-dd8c-11ee-95c6-87774f69a715 2024-03-08T21:20:36.655 INFO:tasks.cephadm:Removing cephadm ... 2024-03-08T21:20:36.656 DEBUG:teuthology.orchestra.run.smithi107:> rm -rf /home/ubuntu/cephtest/cephadm 2024-03-08T21:20:36.714 DEBUG:teuthology.orchestra.run.smithi149:> rm -rf /home/ubuntu/cephtest/cephadm 2024-03-08T21:20:36.771 DEBUG:teuthology.orchestra.run.smithi204:> rm -rf /home/ubuntu/cephtest/cephadm 2024-03-08T21:20:36.826 INFO:tasks.cephadm:Teardown complete 2024-03-08T21:20:36.826 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-03-08T21:20:36.837 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/task/install/__init__.py", line 626, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/task/__init__.py", line 132, in __exit__ self.end() File "/home/teuthworker/src/git.ceph.com_ceph-c_7d63482aa89628d733e298984a762389b625c761/qa/tasks/check_counter.py", line 84, in end proc = manager.admin_socket(daemon_type, daemon_id, ["perf", "dump"]) File "/home/teuthworker/src/git.ceph.com_ceph-c_7d63482aa89628d733e298984a762389b625c761/qa/tasks/ceph_manager.py", line 1875, in admin_socket return shell( File "/home/teuthworker/src/git.ceph.com_ceph-c_7d63482aa89628d733e298984a762389b625c761/qa/tasks/ceph_manager.py", line 43, in shell return remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi204 with status 22: 'sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761 shell --fsid dc20f0f8-dd8c-11ee-95c6-87774f69a715 -- ceph daemon mds.c perf dump' 2024-03-08T21:20:36.838 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-03-08T21:20:36.838 DEBUG:teuthology.orchestra.run.smithi107:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-08T21:20:36.846 DEBUG:teuthology.orchestra.run.smithi149:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-08T21:20:36.853 DEBUG:teuthology.orchestra.run.smithi204:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-08T21:20:37.020 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-03-08T21:20:37.039 INFO:teuthology.task.clock:Checking final clock skew... 2024-03-08T21:20:37.040 DEBUG:teuthology.orchestra.run.smithi107:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-08T21:20:37.048 DEBUG:teuthology.orchestra.run.smithi149:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-08T21:20:37.056 DEBUG:teuthology.orchestra.run.smithi204:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-08T21:20:37.094 INFO:teuthology.orchestra.run.smithi107.stderr:bash: line 1: ntpq: command not found 2024-03-08T21:20:37.103 INFO:teuthology.orchestra.run.smithi149.stderr:bash: line 1: ntpq: command not found 2024-03-08T21:20:37.104 INFO:teuthology.orchestra.run.smithi107.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-08T21:20:37.104 INFO:teuthology.orchestra.run.smithi107.stdout:=============================================================================== 2024-03-08T21:20:37.104 INFO:teuthology.orchestra.run.smithi107.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 52 -470us[ -429us] +/- 118ms 2024-03-08T21:20:37.105 INFO:teuthology.orchestra.run.smithi107.stdout:^* hv02.front.sepia.ceph.com 3 6 377 49 -1423us[-1382us] +/- 68ms 2024-03-08T21:20:37.105 INFO:teuthology.orchestra.run.smithi107.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 49 +1800us[+1841us] +/- 71ms 2024-03-08T21:20:37.105 INFO:teuthology.orchestra.run.smithi107.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-03-08T21:20:37.109 INFO:teuthology.orchestra.run.smithi204.stderr:bash: line 1: ntpq: command not found 2024-03-08T21:20:37.112 INFO:teuthology.orchestra.run.smithi149.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-08T21:20:37.112 INFO:teuthology.orchestra.run.smithi149.stdout:=============================================================================== 2024-03-08T21:20:37.113 INFO:teuthology.orchestra.run.smithi149.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 48 -331us[ -331us] +/- 118ms 2024-03-08T21:20:37.113 INFO:teuthology.orchestra.run.smithi149.stdout:^* hv02.front.sepia.ceph.com 3 8 377 51 -1241us[-1221us] +/- 68ms 2024-03-08T21:20:37.113 INFO:teuthology.orchestra.run.smithi149.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 50 +1939us[+1939us] +/- 71ms 2024-03-08T21:20:37.113 INFO:teuthology.orchestra.run.smithi149.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-03-08T21:20:37.118 INFO:teuthology.orchestra.run.smithi204.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-08T21:20:37.118 INFO:teuthology.orchestra.run.smithi204.stdout:=============================================================================== 2024-03-08T21:20:37.118 INFO:teuthology.orchestra.run.smithi204.stdout:^+ hv01.front.sepia.ceph.com 2 7 377 46 -284us[ -284us] +/- 118ms 2024-03-08T21:20:37.119 INFO:teuthology.orchestra.run.smithi204.stdout:^* hv02.front.sepia.ceph.com 3 8 377 111 -1190us[-1151us] +/- 67ms 2024-03-08T21:20:37.119 INFO:teuthology.orchestra.run.smithi204.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 47 +1961us[+1961us] +/- 71ms 2024-03-08T21:20:37.119 INFO:teuthology.orchestra.run.smithi204.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-03-08T21:20:37.120 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-03-08T21:20:37.138 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-03-08T21:20:37.138 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-03-08T21:20:37.160 DEBUG:teuthology.orchestra.run.smithi107:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-08T21:20:37.395 DEBUG:teuthology.orchestra.run.smithi149:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-08T21:20:37.592 DEBUG:teuthology.orchestra.run.smithi204:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-08T21:20:37.815 DEBUG:teuthology.orchestra.run.smithi107:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-08T21:20:37.997 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-08T21:20:37.998 DEBUG:teuthology.orchestra.run.smithi149:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-08T21:20:38.100 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-08T21:20:38.101 DEBUG:teuthology.orchestra.run.smithi204:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-08T21:20:38.198 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-08T21:20:38.199 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-03-08T21:20:38.213 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-03-08T21:20:38.239 INFO:teuthology.task.internal:Duration was 2614.122840 seconds 2024-03-08T21:20:38.240 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-03-08T21:20:38.271 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-03-08T21:20:38.271 DEBUG:teuthology.orchestra.run.smithi107:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-08T21:20:38.277 DEBUG:teuthology.orchestra.run.smithi149:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-08T21:20:38.285 DEBUG:teuthology.orchestra.run.smithi204:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-08T21:20:38.421 INFO:teuthology.orchestra.run.smithi107.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-08T21:20:38.434 INFO:teuthology.orchestra.run.smithi149.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-08T21:20:38.449 INFO:teuthology.orchestra.run.smithi204.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-08T21:20:38.840 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-03-08T21:20:38.840 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi107.front.sepia.ceph.com 2024-03-08T21:20:38.840 DEBUG:teuthology.orchestra.run.smithi107:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-08T21:20:38.900 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi149.front.sepia.ceph.com 2024-03-08T21:20:38.901 DEBUG:teuthology.orchestra.run.smithi149:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-08T21:20:38.975 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi204.front.sepia.ceph.com 2024-03-08T21:20:38.975 DEBUG:teuthology.orchestra.run.smithi204:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-08T21:20:39.048 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-03-08T21:20:39.049 DEBUG:teuthology.orchestra.run.smithi107:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-08T21:20:39.053 DEBUG:teuthology.orchestra.run.smithi149:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-08T21:20:39.061 DEBUG:teuthology.orchestra.run.smithi204:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-08T21:20:39.152 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-03-08T21:20:39.152 DEBUG:teuthology.orchestra.run.smithi107:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-08T21:20:39.156 DEBUG:teuthology.orchestra.run.smithi149:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-08T21:20:39.161 DEBUG:teuthology.orchestra.run.smithi204:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-08T21:20:39.644 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-03-08T21:20:39.659 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-03-08T21:20:39.659 DEBUG:teuthology.orchestra.run.smithi107:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-08T21:20:39.749 DEBUG:teuthology.orchestra.run.smithi149:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-08T21:20:39.845 DEBUG:teuthology.orchestra.run.smithi204:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-08T21:20:39.944 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-03-08T21:20:39.956 DEBUG:teuthology.orchestra.run.smithi107:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-08T21:20:39.964 DEBUG:teuthology.orchestra.run.smithi149:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-08T21:20:39.973 DEBUG:teuthology.orchestra.run.smithi204:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-08T21:20:40.066 INFO:teuthology.orchestra.run.smithi107.stdout:kernel.core_pattern = core 2024-03-08T21:20:40.075 INFO:teuthology.orchestra.run.smithi149.stdout:kernel.core_pattern = core 2024-03-08T21:20:40.086 INFO:teuthology.orchestra.run.smithi204.stdout:kernel.core_pattern = core 2024-03-08T21:20:40.156 DEBUG:teuthology.orchestra.run.smithi107:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-08T21:20:40.209 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-08T21:20:40.210 DEBUG:teuthology.orchestra.run.smithi149:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-08T21:20:40.264 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-08T21:20:40.264 DEBUG:teuthology.orchestra.run.smithi204:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-08T21:20:40.320 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-08T21:20:40.321 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-03-08T21:20:40.335 INFO:teuthology.task.internal:Transferring archived files... 2024-03-08T21:20:40.336 DEBUG:teuthology.misc:Transferring archived files from smithi107:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-03-08_19:54:47-fs-rishabh-2024mar8-testing-default-smithi/7588131/remote/smithi107 2024-03-08T21:20:40.337 DEBUG:teuthology.orchestra.run.smithi107:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-08T21:20:40.453 DEBUG:teuthology.misc:Transferring archived files from smithi149:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-03-08_19:54:47-fs-rishabh-2024mar8-testing-default-smithi/7588131/remote/smithi149 2024-03-08T21:20:40.453 DEBUG:teuthology.orchestra.run.smithi149:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-08T21:20:40.565 DEBUG:teuthology.misc:Transferring archived files from smithi204:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-03-08_19:54:47-fs-rishabh-2024mar8-testing-default-smithi/7588131/remote/smithi204 2024-03-08T21:20:40.565 DEBUG:teuthology.orchestra.run.smithi204:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-08T21:20:40.676 INFO:teuthology.task.internal:Removing archive directory... 2024-03-08T21:20:40.676 DEBUG:teuthology.orchestra.run.smithi107:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-08T21:20:40.684 DEBUG:teuthology.orchestra.run.smithi149:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-08T21:20:40.692 DEBUG:teuthology.orchestra.run.smithi204:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-08T21:20:40.748 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-03-08T21:20:40.763 INFO:teuthology.task.internal:Not uploading archives. 2024-03-08T21:20:40.763 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-03-08T21:20:40.789 INFO:teuthology.task.internal:Tidying up after the test... 2024-03-08T21:20:40.789 DEBUG:teuthology.orchestra.run.smithi107:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-08T21:20:40.797 DEBUG:teuthology.orchestra.run.smithi149:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-08T21:20:40.804 DEBUG:teuthology.orchestra.run.smithi204:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-08T21:20:40.846 INFO:teuthology.orchestra.run.smithi107.stdout: 393776 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 8 21:20 /home/ubuntu/cephtest 2024-03-08T21:20:40.851 INFO:teuthology.orchestra.run.smithi149.stdout: 394652 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 8 21:20 /home/ubuntu/cephtest 2024-03-08T21:20:40.860 INFO:teuthology.orchestra.run.smithi204.stdout: 393229 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 8 21:20 /home/ubuntu/cephtest 2024-03-08T21:20:40.864 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-03-08T21:20:40.877 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-03-08T21:20:40.901 DEBUG:teuthology.run_tasks:Exception was not quenched, exiting: CommandFailedError: Command failed on smithi204 with status 22: 'sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761 shell --fsid dc20f0f8-dd8c-11ee-95c6-87774f69a715 -- ceph daemon mds.c perf dump' 2024-03-08T21:20:40.901 INFO:teuthology.nuke:Checking targets against current locks 2024-03-08T21:20:40.924 DEBUG:teuthology.nuke:shortname: smithi107 2024-03-08T21:20:40.924 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-08T21:20:40.949 DEBUG:teuthology.nuke:shortname: smithi149 2024-03-08T21:20:40.949 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-08T21:20:40.954 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi107.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-03-08_19:54:47-fs-rishabh-2024mar8-testing-default-smithi/7588131', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-08 20:27:02.209988', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJHUqpW3DO5uh/UyCHrj4wHu9VgAgBFBM3KOq23ZnIIXBbGc51e1OwI4pL48TyShR7tAWT7OWIZaqaGRteH51no='} 2024-03-08T21:20:40.974 DEBUG:teuthology.nuke:shortname: smithi204 2024-03-08T21:20:40.975 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-08T21:20:40.980 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi149.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-03-08_19:54:47-fs-rishabh-2024mar8-testing-default-smithi/7588131', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-08 20:27:02.208222', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGpna6/R7UJgNVAK3aRrFGF8fsUEJ0NGWiCWa9MYoBhKnZKZ1j7tDvHfMdVuXmpNKHo+IFqZuaP3Lrum5b/bDRA='} 2024-03-08T21:20:41.012 INFO:teuthology.orchestra.console.smithi107:Power off 2024-03-08T21:20:41.012 DEBUG:teuthology.orchestra.console.smithi107:pexpect command: ipmitool -H smithi107.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-08T21:20:41.028 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi204.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-03-08_19:54:47-fs-rishabh-2024mar8-testing-default-smithi/7588131', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-08 20:27:02.206287', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGpHY52wqgFcelYlJ7D0yf12pxTEgCbwvmGmeK8S4HFDZ/4nEWVlPEtLpbNVhfgCTTiPaJMNAyvt+O/rU/tHA3Y='} 2024-03-08T21:20:41.041 DEBUG:teuthology.orchestra.console.smithi107:power off output: Chassis Power Control: Down/Off 2024-03-08T21:20:41.041 DEBUG:teuthology.orchestra.console.smithi107:pexpect command: ipmitool -H smithi107.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-08T21:20:41.057 INFO:teuthology.orchestra.console.smithi149:Power off 2024-03-08T21:20:41.057 DEBUG:teuthology.orchestra.console.smithi149:pexpect command: ipmitool -H smithi149.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-08T21:20:41.071 DEBUG:teuthology.orchestra.console.smithi107:check power output: Chassis Power is on 2024-03-08T21:20:41.081 DEBUG:teuthology.orchestra.console.smithi149:power off output: Chassis Power Control: Down/Off 2024-03-08T21:20:41.081 DEBUG:teuthology.orchestra.console.smithi149:pexpect command: ipmitool -H smithi149.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-08T21:20:41.097 INFO:teuthology.orchestra.console.smithi204:Power off 2024-03-08T21:20:41.097 DEBUG:teuthology.orchestra.console.smithi204:pexpect command: ipmitool -H smithi204.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-08T21:20:41.107 DEBUG:teuthology.orchestra.console.smithi149:check power output: Chassis Power is on 2024-03-08T21:20:41.120 DEBUG:teuthology.orchestra.console.smithi204:power off output: Chassis Power Control: Down/Off 2024-03-08T21:20:41.121 DEBUG:teuthology.orchestra.console.smithi204:pexpect command: ipmitool -H smithi204.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-08T21:20:41.144 DEBUG:teuthology.orchestra.console.smithi204:check power output: Chassis Power is on 2024-03-08T21:20:45.072 DEBUG:teuthology.orchestra.console.smithi107:pexpect command: ipmitool -H smithi107.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-08T21:20:45.108 DEBUG:teuthology.orchestra.console.smithi149:pexpect command: ipmitool -H smithi149.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-08T21:20:45.145 DEBUG:teuthology.orchestra.console.smithi204:pexpect command: ipmitool -H smithi204.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-08T21:20:45.184 DEBUG:teuthology.orchestra.console.smithi107:check power output: Chassis Power is on 2024-03-08T21:20:45.218 DEBUG:teuthology.orchestra.console.smithi149:check power output: Chassis Power is on 2024-03-08T21:20:45.256 DEBUG:teuthology.orchestra.console.smithi204:check power output: Chassis Power is on 2024-03-08T21:20:49.189 DEBUG:teuthology.orchestra.console.smithi107:pexpect command: ipmitool -H smithi107.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-08T21:20:49.219 DEBUG:teuthology.orchestra.console.smithi149:pexpect command: ipmitool -H smithi149.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-08T21:20:49.257 DEBUG:teuthology.orchestra.console.smithi204:pexpect command: ipmitool -H smithi204.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-08T21:20:49.302 DEBUG:teuthology.orchestra.console.smithi107:check power output: Chassis Power is off 2024-03-08T21:20:49.330 DEBUG:teuthology.orchestra.console.smithi149:check power output: Chassis Power is on 2024-03-08T21:20:49.369 DEBUG:teuthology.orchestra.console.smithi204:check power output: Chassis Power is on 2024-03-08T21:20:49.403 INFO:teuthology.orchestra.console.smithi107:Power off completed 2024-03-08T21:20:53.334 DEBUG:teuthology.orchestra.console.smithi149:pexpect command: ipmitool -H smithi149.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-08T21:20:53.370 DEBUG:teuthology.orchestra.console.smithi204:pexpect command: ipmitool -H smithi204.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-08T21:20:53.449 DEBUG:teuthology.orchestra.console.smithi149:check power output: Chassis Power is off 2024-03-08T21:20:53.480 DEBUG:teuthology.orchestra.console.smithi204:check power output: Chassis Power is off 2024-03-08T21:20:53.550 INFO:teuthology.orchestra.console.smithi149:Power off completed 2024-03-08T21:20:53.581 INFO:teuthology.orchestra.console.smithi204:Power off completed 2024-03-08T21:20:53.687 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/fuse objectstore-ec/bluestore-comp omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/random export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-workunit/direct_io}} duration: 2614.1228404045105 failure_reason: 'Command failed on smithi204 with status 22: ''sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:7d63482aa89628d733e298984a762389b625c761 shell --fsid dc20f0f8-dd8c-11ee-95c6-87774f69a715 -- ceph daemon mds.c perf dump''' flavor: default mon.a-kernel-sha1: dd5c1cd171679b4e0ceeab3e8d62f25c2b4e6412 mon.b-kernel-sha1: dd5c1cd171679b4e0ceeab3e8d62f25c2b4e6412 mon.c-kernel-sha1: dd5c1cd171679b4e0ceeab3e8d62f25c2b4e6412 owner: scheduled_rishabh@teuthology status: fail success: false 2024-03-08T21:20:53.687 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-08T21:20:53.780 INFO:teuthology.run:FAIL